2024-04-17T06:45:19.148 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-17T06:45:19.149 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --name rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} --verbose -- /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891/orig.config.yaml 2024-04-17T06:45:19.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T06:45:19.284 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891 branch: wip-rishabh-testing-20240416.193735 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} email: null first_in_suite: false job_id: '7659891' kernel: flavor: default kdb: true sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a8c84f5997950fd7d008204839253291d52e6eda subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3521 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 410/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDWvvi3c9NgpR+VwJWnnDD7K28JTAVPxd+qkTRepn6rnRXmH1+Z9mTT5reI8v39KHZWRjvbw0HYADM8XiGNoVbc= smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNliSfdHzPHo430Yz8TMQz9ReCoSEUJUFaOJjMSzkqlCFzWqtiPzzbfPADZ31P6iRJDHwH7Wh83CBKVAgbkkicA= smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrtn37WGU5iIjblYaX6uLqlB+qcsPvB/jHlbyyMWDiroyLnkQBHtHGJUNEc0B4qUjQTEQuEtwvv0EhoEzBeprI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-17_04:58:44 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-17T06:45:19.285 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-17T06:45:19.285 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-17T06:45:19.285 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-17T06:45:19.287 INFO:teuthology.task.internal:Checking packages... 2024-04-17T06:45:19.308 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-17T06:45:19.309 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T06:45:19.309 INFO:teuthology.packaging:ref: None 2024-04-17T06:45:19.309 INFO:teuthology.packaging:tag: None 2024-04-17T06:45:19.309 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-17T06:45:19.309 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T06:45:19.309 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-17T06:45:19.455 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-17T06:45:19.457 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-17T06:45:19.463 INFO:teuthology.task.internal:no buildpackages task found 2024-04-17T06:45:19.463 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-17T06:45:19.470 INFO:teuthology.task.internal:Saving configuration 2024-04-17T06:45:19.490 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-17T06:45:19.495 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T06:45:19.519 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 06:39:46.425817', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDWvvi3c9NgpR+VwJWnnDD7K28JTAVPxd+qkTRepn6rnRXmH1+Z9mTT5reI8v39KHZWRjvbw0HYADM8XiGNoVbc='} 2024-04-17T06:45:19.537 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 06:39:46.423580', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNliSfdHzPHo430Yz8TMQz9ReCoSEUJUFaOJjMSzkqlCFzWqtiPzzbfPADZ31P6iRJDHwH7Wh83CBKVAgbkkicA='} 2024-04-17T06:45:19.558 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 06:39:46.424858', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrtn37WGU5iIjblYaX6uLqlB+qcsPvB/jHlbyyMWDiroyLnkQBHtHGJUNEc0B4qUjQTEQuEtwvv0EhoEzBeprI='} 2024-04-17T06:45:19.558 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-17T06:45:19.564 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-17T06:45:19.565 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-17T06:45:19.565 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-17T06:45:19.565 INFO:teuthology.run_tasks:Running task console_log... 2024-04-17T06:45:19.669 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fad3a47d160>, signals=[15]) 2024-04-17T06:45:19.669 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-17T06:45:19.675 INFO:teuthology.task.internal:Opening connections... 2024-04-17T06:45:19.675 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-04-17T06:45:19.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:45:19.765 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2024-04-17T06:45:19.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:45:19.850 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-04-17T06:45:19.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:45:19.947 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-17T06:45:19.954 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-04-17T06:45:19.972 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-04-17T06:45:19.973 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-04-17T06:45:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-04-17T06:45:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-04-17T06:45:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-04-17T06:45:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-04-17T06:45:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-04-17T06:45:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-04-17T06:45:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T06:45:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-04-17T06:45:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-04-17T06:45:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T06:45:20.031 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-04-17T06:45:20.031 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T06:45:20.031 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T06:45:20.031 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T06:45:20.031 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-04-17T06:45:20.056 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-04-17T06:45:20.073 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-04-17T06:45:20.073 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-04-17T06:45:20.130 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-04-17T06:45:20.130 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-04-17T06:45:20.130 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-04-17T06:45:20.130 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-04-17T06:45:20.130 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-04-17T06:45:20.130 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-04-17T06:45:20.130 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T06:45:20.130 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-04-17T06:45:20.131 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-04-17T06:45:20.131 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T06:45:20.131 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-04-17T06:45:20.131 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T06:45:20.131 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T06:45:20.131 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T06:45:20.131 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-04-17T06:45:20.156 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-04-17T06:45:20.173 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-04-17T06:45:20.174 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-04-17T06:45:20.230 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-04-17T06:45:20.230 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-04-17T06:45:20.230 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-04-17T06:45:20.230 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-04-17T06:45:20.230 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-04-17T06:45:20.230 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-04-17T06:45:20.230 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T06:45:20.230 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-04-17T06:45:20.230 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-04-17T06:45:20.231 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T06:45:20.231 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-04-17T06:45:20.231 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T06:45:20.231 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T06:45:20.231 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T06:45:20.231 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-04-17T06:45:20.255 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-17T06:45:20.265 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-17T06:45:20.271 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-17T06:45:20.271 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-04-17T06:45:20.273 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2024-04-17T06:45:20.276 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-04-17T06:45:20.291 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-17T06:45:20.297 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-17T06:45:20.297 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-04-17T06:45:20.330 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2024-04-17T06:45:20.332 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-04-17T06:45:20.581 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-17T06:45:20.659 INFO:teuthology.run_tasks:Running task kernel... 2024-04-17T06:45:20.675 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'} 2024-04-17T06:45:20.675 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8'}}, timeout 300 2024-04-17T06:45:20.676 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T06:45:20.676 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T06:45:20.676 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T06:45:20.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:45:20.693 DEBUG:teuthology.task.kernel:sha1 for mon.b is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:45:20.693 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-17T06:45:20.693 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-04-17T06:45:20.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:45:20.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:45:20.696 DEBUG:teuthology.task.kernel:sha1 for mon.c is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:45:20.696 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-17T06:45:20.696 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-04-17T06:45:20.696 DEBUG:teuthology.task.kernel:sha1 for mon.a is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:45:20.696 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-17T06:45:20.697 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-17T06:45:20.749 INFO:teuthology.orchestra.run.smithi098.stdout:5.14.0-361.el9.x86_64 2024-04-17T06:45:20.750 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:45:20.750 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-17T06:45:20.750 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:45:20.755 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-361.el9.x86_64 2024-04-17T06:45:20.755 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-361.el9.x86_64 2024-04-17T06:45:20.755 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:45:20.755 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-17T06:45:20.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:45:20.757 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:45:20.757 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-17T06:45:20.757 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:45:21.064 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum upgrade -y linux-firmware 2024-04-17T06:45:21.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum upgrade -y linux-firmware 2024-04-17T06:45:21.093 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum upgrade -y linux-firmware 2024-04-17T06:45:23.490 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.7 MB 00:00 2024-04-17T06:45:23.524 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS-9 - AppStream 9.3 MB/s | 6.7 MB 00:00 2024-04-17T06:45:23.527 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS-9 - AppStream 9.0 MB/s | 6.7 MB 00:00 2024-04-17T06:45:24.857 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS-9 - BaseOS 7.7 MB/s | 2.0 MB 00:00 2024-04-17T06:45:25.144 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-04-17T06:45:25.299 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS-9 - BaseOS 3.1 MB/s | 2.0 MB 00:00 2024-04-17T06:45:26.690 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-17T06:45:26.909 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.0 MB 00:01 2024-04-17T06:45:28.656 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.0 MB 00:02 2024-04-17T06:45:30.374 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-04-17T06:45:35.091 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.2 MB 00:01 2024-04-17T06:45:36.637 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-04-17T06:45:38.155 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-17T06:45:38.901 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 19 MB 00:10 2024-04-17T06:45:43.177 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-17T06:45:45.622 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 1 Package 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 375 M 2024-04-17T06:45:45.630 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-04-17T06:45:45.968 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.2 MB 00:03 2024-04-17T06:45:45.980 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:18 2024-04-17T06:45:47.769 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-17T06:45:49.348 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-17T06:45:50.840 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.2 MB 00:01 2024-04-17T06:45:54.312 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-17T06:45:56.816 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-17T06:45:56.829 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-17T06:45:56.829 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-04-17T06:45:56.830 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-17T06:45:56.830 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-04-17T06:45:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-17T06:45:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:45:56.830 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-04-17T06:45:56.830 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-17T06:45:56.830 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 1 Package 2024-04-17T06:45:56.830 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:45:56.830 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 375 M 2024-04-17T06:45:56.830 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-04-17T06:46:04.910 INFO:teuthology.orchestra.run.smithi103.stdout:linux-firmware-20240219-143.el9.noarch.rpm 19 MB/s | 375 MB 00:19 2024-04-17T06:46:04.911 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-04-17T06:46:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:Total 19 MB/s | 375 MB 00:19 2024-04-17T06:46:04.912 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-04-17T06:46:05.023 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-04-17T06:46:05.024 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-04-17T06:46:07.571 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-04-17T06:46:07.571 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-04-17T06:46:09.921 INFO:teuthology.orchestra.run.smithi086.stdout:linux-firmware-20240219-143.el9.noarch.rpm 29 MB/s | 375 MB 00:13 2024-04-17T06:46:09.922 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-17T06:46:09.922 INFO:teuthology.orchestra.run.smithi086.stdout:Total 29 MB/s | 375 MB 00:13 2024-04-17T06:46:09.922 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-04-17T06:46:10.007 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-04-17T06:46:10.007 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-04-17T06:46:12.480 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-04-17T06:46:12.481 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-04-17T06:46:14.095 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-04-17T06:46:14.330 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-17T06:46:14.421 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T06:46:15.540 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T06:46:15.540 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-17T06:46:15.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T06:46:15.917 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:46:15.917 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-04-17T06:46:15.917 INFO:teuthology.orchestra.run.smithi103.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-17T06:46:15.917 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:46:15.917 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-04-17T06:46:16.226 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.c... 2024-04-17T06:46:16.226 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:46:16.463 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-17T06:46:16.463 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-17T06:46:16.466 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi103.front.sepia.ceph.com to complete... 2024-04-17T06:46:18.811 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-04-17T06:46:19.012 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-17T06:46:19.087 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T06:46:20.293 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T06:46:20.293 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-17T06:46:20.680 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T06:46:20.680 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:46:20.680 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-04-17T06:46:20.681 INFO:teuthology.orchestra.run.smithi086.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-17T06:46:20.681 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:46:20.681 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-17T06:46:20.972 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.a... 2024-04-17T06:46:20.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:46:21.221 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-17T06:46:21.221 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-17T06:46:21.223 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-04-17T06:46:24.770 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 504 B/s | 16 kB 00:32 2024-04-17T06:46:26.605 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-17T06:46:31.563 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-17T06:46:34.031 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-04-17T06:46:34.045 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-04-17T06:46:34.045 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-04-17T06:46:34.046 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-04-17T06:46:34.046 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2024-04-17T06:46:34.046 INFO:teuthology.orchestra.run.smithi098.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-17T06:46:34.046 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:46:34.046 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-04-17T06:46:34.046 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-04-17T06:46:34.046 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 1 Package 2024-04-17T06:46:34.046 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:46:34.046 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 375 M 2024-04-17T06:46:34.046 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-04-17T06:46:47.972 INFO:teuthology.orchestra.run.smithi098.stdout:linux-firmware-20240219-143.el9.noarch.rpm 27 MB/s | 375 MB 00:13 2024-04-17T06:46:47.973 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-04-17T06:46:47.973 INFO:teuthology.orchestra.run.smithi098.stdout:Total 27 MB/s | 375 MB 00:13 2024-04-17T06:46:47.973 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-04-17T06:46:48.067 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-04-17T06:46:48.067 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-04-17T06:46:50.459 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-04-17T06:46:50.460 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-04-17T06:46:56.879 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-04-17T06:46:57.101 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-17T06:46:57.191 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T06:46:58.448 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T06:46:58.449 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-17T06:46:58.860 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-17T06:46:58.860 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:46:58.860 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2024-04-17T06:46:58.860 INFO:teuthology.orchestra.run.smithi098.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-17T06:46:58.860 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:46:58.860 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-04-17T06:46:59.156 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.b... 2024-04-17T06:46:59.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-17T06:46:59.311 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-17T06:46:59.312 DEBUG:teuthology.orchestra.run.smithi098:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-17T06:46:59.314 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi098.front.sepia.ceph.com to complete... 2024-04-17T06:48:47.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17 06:48:47 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-17T06:48:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:FINISHED --2024-04-17 06:48:47-- 2024-04-17T06:48:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:Total wall clock time: 2m 31s 2024-04-17T06:48:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:Downloaded: 1 files, 341M in 2m 30s (2.27 MB/s) 2024-04-17T06:48:47.075 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.c... 2024-04-17T06:48:47.075 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-17T06:48:49.120 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying... ######################################## 2024-04-17T06:48:49.176 INFO:teuthology.orchestra.run.smithi103.stdout:Preparing... ######################################## 2024-04-17T06:48:49.256 INFO:teuthology.orchestra.run.smithi103.stdout:Updating / installing... 2024-04-17T06:48:57.170 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-17T06:49:00.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17 06:49:00 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-17T06:49:00.125 INFO:teuthology.orchestra.run.smithi098.stderr:FINISHED --2024-04-17 06:49:00-- 2024-04-17T06:49:00.125 INFO:teuthology.orchestra.run.smithi098.stderr:Total wall clock time: 2m 1s 2024-04-17T06:49:00.125 INFO:teuthology.orchestra.run.smithi098.stderr:Downloaded: 1 files, 341M in 2m 0s (2.84 MB/s) 2024-04-17T06:49:00.128 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.b... 2024-04-17T06:49:00.128 DEBUG:teuthology.orchestra.run.smithi098:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-17T06:49:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17 06:49:01 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-17T06:49:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:FINISHED --2024-04-17 06:49:01-- 2024-04-17T06:49:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:Total wall clock time: 2m 40s 2024-04-17T06:49:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:Downloaded: 1 files, 341M in 2m 39s (2.14 MB/s) 2024-04-17T06:49:01.143 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on mon.a... 2024-04-17T06:49:01.143 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-17T06:49:02.176 INFO:teuthology.orchestra.run.smithi098.stdout:Verifying... ######################################## 2024-04-17T06:49:02.299 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing... ######################################## 2024-04-17T06:49:02.378 INFO:teuthology.orchestra.run.smithi098.stdout:Updating / installing... 2024-04-17T06:49:03.193 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying... ######################################## 2024-04-17T06:49:03.254 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing... ######################################## 2024-04-17T06:49:03.334 INFO:teuthology.orchestra.run.smithi086.stdout:Updating / installing... 2024-04-17T06:49:10.324 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-17T06:49:11.311 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-17T06:49:12.721 INFO:teuthology.orchestra.run.smithi103.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-17T06:49:24.761 INFO:teuthology.orchestra.run.smithi098.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-17T06:49:26.752 INFO:teuthology.orchestra.run.smithi086.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-17T06:49:35.791 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi103.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-17T06:49:35.791 DEBUG:teuthology.orchestra.run.smithi103:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-17T06:49:35.885 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:35.886 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:35.887 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:35.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-17T06:49:35.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-17T06:49:35.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-17T06:49:35.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-17T06:49:35.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-17T06:49:35.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-17T06:49:35.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-17T06:49:35.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-17T06:49:35.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-17T06:49:35.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-17T06:49:35.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-17T06:49:35.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-17T06:49:35.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-17T06:49:35.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-17T06:49:35.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-17T06:49:35.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-17T06:49:35.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-17T06:49:35.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-17T06:49:35.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-17T06:49:35.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-17T06:49:35.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-17T06:49:35.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-17T06:49:35.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-17T06:49:35.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-17T06:49:35.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-17T06:49:35.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-17T06:49:35.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-17T06:49:35.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-17T06:49:35.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-17T06:49:35.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-17T06:49:35.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-17T06:49:35.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-17T06:49:35.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-17T06:49:35.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-17T06:49:35.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-17T06:49:35.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-17T06:49:35.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-17T06:49:35.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-17T06:49:35.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-17T06:49:35.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-17T06:49:35.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-17T06:49:35.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-17T06:49:35.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-17T06:49:35.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-17T06:49:35.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-17T06:49:35.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-17T06:49:35.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-17T06:49:35.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-17T06:49:35.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-17T06:49:35.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-17T06:49:35.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-17T06:49:35.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-17T06:49:35.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-17T06:49:35.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-17T06:49:35.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-17T06:49:35.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-17T06:49:35.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-17T06:49:35.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-17T06:49:35.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-17T06:49:35.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-17T06:49:35.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-17T06:49:35.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-17T06:49:35.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-17T06:49:35.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-17T06:49:35.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-17T06:49:35.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-17T06:49:35.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-17T06:49:35.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-17T06:49:35.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-17T06:49:35.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-17T06:49:35.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-17T06:49:35.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-17T06:49:35.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-17T06:49:35.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-17T06:49:35.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-17T06:49:35.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-17T06:49:35.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-17T06:49:35.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-17T06:49:35.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-17T06:49:35.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-17T06:49:35.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-17T06:49:35.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-17T06:49:35.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-17T06:49:35.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-17T06:49:35.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-17T06:49:35.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-17T06:49:35.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-17T06:49:35.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-17T06:49:35.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-17T06:49:35.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-17T06:49:35.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-17T06:49:35.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-17T06:49:35.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-17T06:49:35.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-17T06:49:35.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-17T06:49:35.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-17T06:49:35.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-17T06:49:35.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-17T06:49:35.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-17T06:49:35.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-17T06:49:35.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-17T06:49:35.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-17T06:49:35.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-17T06:49:35.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-17T06:49:35.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-17T06:49:35.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-17T06:49:35.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-17T06:49:35.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-17T06:49:35.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-17T06:49:35.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-17T06:49:35.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-17T06:49:35.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-17T06:49:35.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-17T06:49:35.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-17T06:49:35.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-17T06:49:35.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-17T06:49:35.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-17T06:49:35.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-17T06:49:35.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-17T06:49:35.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-17T06:49:35.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-17T06:49:35.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-17T06:49:35.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-17T06:49:35.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-17T06:49:35.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-17T06:49:35.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-17T06:49:35.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-17T06:49:35.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-17T06:49:35.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-17T06:49:35.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-17T06:49:35.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-17T06:49:35.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-17T06:49:35.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-17T06:49:35.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-17T06:49:35.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-17T06:49:35.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-17T06:49:35.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-17T06:49:35.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-17T06:49:35.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-17T06:49:35.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-17T06:49:35.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-17T06:49:35.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-17T06:49:35.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-17T06:49:35.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-17T06:49:35.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-17T06:49:35.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-17T06:49:35.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-17T06:49:35.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-17T06:49:35.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-17T06:49:35.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-17T06:49:35.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-17T06:49:35.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-17T06:49:35.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-17T06:49:35.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-17T06:49:35.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-17T06:49:35.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-17T06:49:35.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-17T06:49:35.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-17T06:49:35.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-17T06:49:35.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-17T06:49:35.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-17T06:49:35.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-17T06:49:35.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-17T06:49:35.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-17T06:49:35.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-17T06:49:35.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-17T06:49:35.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-17T06:49:35.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-17T06:49:35.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-17T06:49:35.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-17T06:49:35.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-17T06:49:35.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-17T06:49:35.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-17T06:49:35.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-17T06:49:35.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-17T06:49:35.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-17T06:49:35.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-17T06:49:35.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-17T06:49:35.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-17T06:49:35.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-17T06:49:35.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-17T06:49:35.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-17T06:49:35.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-17T06:49:35.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-17T06:49:35.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-17T06:49:35.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-17T06:49:35.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-17T06:49:35.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-17T06:49:35.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-17T06:49:35.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-17T06:49:35.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-17T06:49:35.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-17T06:49:35.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-17T06:49:35.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-17T06:49:35.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-17T06:49:35.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-17T06:49:35.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-17T06:49:35.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-17T06:49:35.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-17T06:49:35.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-17T06:49:35.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-17T06:49:35.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-17T06:49:35.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-17T06:49:35.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-17T06:49:35.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-17T06:49:35.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-17T06:49:35.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-17T06:49:35.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-17T06:49:35.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-17T06:49:35.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-17T06:49:35.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-17T06:49:35.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-17T06:49:35.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-17T06:49:35.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-17T06:49:35.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-17T06:49:35.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-17T06:49:35.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-17T06:49:35.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-17T06:49:35.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-17T06:49:35.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-17T06:49:35.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-17T06:49:35.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-17T06:49:35.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-17T06:49:35.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-17T06:49:35.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-17T06:49:35.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-17T06:49:35.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-17T06:49:35.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-17T06:49:35.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-17T06:49:35.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-17T06:49:35.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-17T06:49:35.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-17T06:49:35.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-17T06:49:35.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-17T06:49:35.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-17T06:49:35.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-17T06:49:35.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-17T06:49:35.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-17T06:49:35.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-17T06:49:35.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-17T06:49:35.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-17T06:49:35.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-17T06:49:35.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-17T06:49:35.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-17T06:49:35.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-17T06:49:35.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-17T06:49:35.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-17T06:49:35.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-17T06:49:35.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-17T06:49:35.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-17T06:49:35.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-17T06:49:35.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-17T06:49:35.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-17T06:49:35.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-17T06:49:35.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-17T06:49:35.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-17T06:49:35.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-17T06:49:35.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-17T06:49:35.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-17T06:49:35.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-17T06:49:35.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-17T06:49:35.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-17T06:49:35.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-17T06:49:35.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-17T06:49:35.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-17T06:49:35.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-17T06:49:35.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-17T06:49:35.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-17T06:49:35.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-17T06:49:35.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-17T06:49:35.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-17T06:49:35.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-17T06:49:35.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-17T06:49:35.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-17T06:49:35.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-17T06:49:35.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-17T06:49:35.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-17T06:49:35.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-17T06:49:35.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-17T06:49:35.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-17T06:49:35.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-17T06:49:35.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-17T06:49:35.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-17T06:49:35.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-17T06:49:35.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-17T06:49:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-17T06:49:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-17T06:49:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-17T06:49:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-17T06:49:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-17T06:49:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-17T06:49:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-17T06:49:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-17T06:49:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-17T06:49:35.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-17T06:49:35.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-17T06:49:35.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-17T06:49:35.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-17T06:49:35.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-17T06:49:35.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-17T06:49:35.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-17T06:49:35.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-17T06:49:35.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-17T06:49:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-17T06:49:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-17T06:49:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-17T06:49:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-17T06:49:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-17T06:49:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-17T06:49:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-17T06:49:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-17T06:49:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-17T06:49:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-17T06:49:35.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-17T06:49:35.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-17T06:49:35.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-17T06:49:35.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-17T06:49:35.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-17T06:49:35.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-17T06:49:35.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-17T06:49:35.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-17T06:49:35.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-17T06:49:35.973 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-17T06:49:35.974 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-17T06:49:35.975 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-17T06:49:35.976 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-17T06:49:35.977 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-17T06:49:35.978 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-17T06:49:35.979 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-17T06:49:35.980 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-17T06:49:35.980 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-17T06:49:35.980 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-17T06:49:35.980 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-17T06:49:35.980 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-17T06:49:35.980 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-17T06:49:35.980 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-17T06:49:35.980 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-17T06:49:35.980 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-17T06:49:35.980 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-17T06:49:35.981 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-17T06:49:35.982 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-17T06:49:35.983 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-17T06:49:35.984 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-17T06:49:35.985 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-17T06:49:35.986 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-17T06:49:35.987 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-17T06:49:35.988 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-17T06:49:35.989 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-17T06:49:35.990 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-17T06:49:35.991 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-17T06:49:35.992 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-17T06:49:35.993 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-17T06:49:35.994 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-17T06:49:35.995 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-17T06:49:35.996 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-17T06:49:35.997 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-17T06:49:35.998 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-17T06:49:35.999 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-17T06:49:36.000 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-17T06:49:36.001 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-17T06:49:36.002 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-17T06:49:36.003 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-17T06:49:36.004 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-17T06:49:36.005 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-17T06:49:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-17T06:49:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-17T06:49:36.008 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-17T06:49:36.009 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-17T06:49:36.010 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-17T06:49:36.011 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-17T06:49:36.012 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-17T06:49:36.013 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-17T06:49:36.014 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-17T06:49:36.015 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-17T06:49:36.016 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-17T06:49:36.017 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-17T06:49:36.018 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-17T06:49:36.019 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-17T06:49:36.020 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-17T06:49:36.021 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-17T06:49:36.022 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-17T06:49:36.023 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-17T06:49:36.024 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-17T06:49:36.025 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-17T06:49:36.026 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-17T06:49:36.027 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-17T06:49:36.028 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-17T06:49:36.029 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-17T06:49:36.030 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-17T06:49:36.031 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-17T06:49:36.032 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-17T06:49:36.033 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-17T06:49:36.034 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-17T06:49:36.034 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-17T06:49:36.034 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-17T06:49:36.034 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-17T06:49:36.034 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-17T06:49:36.035 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-17T06:49:36.036 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-17T06:49:36.037 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-17T06:49:36.038 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-17T06:49:36.039 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-17T06:49:36.040 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-17T06:49:36.041 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-17T06:49:36.042 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-17T06:49:36.043 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-17T06:49:36.044 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-17T06:49:36.045 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-17T06:49:36.046 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-17T06:49:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-17T06:49:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-17T06:49:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-17T06:49:36.054 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:36.054 DEBUG:teuthology.orchestra.run.smithi103:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-17T06:49:36.076 INFO:teuthology.orchestra.run.smithi103.stdout:postinstall scriptlet (using /bin/sh): 2024-04-17T06:49:36.076 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout:for file in vmlinuz System.map config; do 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: fi 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout:done 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-17T06:49:36.077 INFO:teuthology.orchestra.run.smithi103.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:36.078 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-04-17T06:49:36.078 INFO:teuthology.orchestra.run.smithi103.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-17T06:49:36.078 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-17T06:49:36.078 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:36.078 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-04-17T06:49:36.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -qi grub2-tools 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Name : grub2-tools 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Epoch : 1 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Version : 2.06 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Release : 68.el9 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Architecture: x86_64 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Group : Unspecified 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Size : 8264254 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:License : GPLv3+ 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-17T06:49:36.153 INFO:teuthology.orchestra.run.smithi103.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:Packager : builder@centos.org 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:Vendor : CentOS 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:Summary : Support tools for GRUB. 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:Description : 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:hardware devices. 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:49:36.154 INFO:teuthology.orchestra.run.smithi103.stdout:This subpackage provides tools for support of all platforms. 2024-04-17T06:49:36.155 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-17T06:49:36.155 INFO:teuthology.task.kernel:Updating grub on smithi103 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:36.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-17T06:49:36.710 INFO:teuthology.orchestra.run.smithi103.stderr:Generating grub configuration file ... 2024-04-17T06:49:38.197 INFO:teuthology.orchestra.run.smithi103.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-17T06:49:38.220 INFO:teuthology.orchestra.run.smithi103.stderr:done 2024-04-17T06:49:38.226 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-04-17T06:49:38.241 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.CmOE9btHhx 2024-04-17T06:49:38.242 DEBUG:teuthology.orchestra.run.smithi103:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.CmOE9btHhx 2024-04-17T06:49:38.311 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.CmOE9btHhx 2024-04-17T06:49:38.449 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.CmOE9btHhx is 6KB 2024-04-17T06:49:38.497 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.CmOE9btHhx 2024-04-17T06:49:38.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo /bin/ls /boot/loader/entries || true 2024-04-17T06:49:38.576 INFO:teuthology.orchestra.run.smithi103.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-17T06:49:38.576 INFO:teuthology.orchestra.run.smithi103.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-17T06:49:38.576 INFO:teuthology.orchestra.run.smithi103.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-17T06:49:38.576 INFO:teuthology.orchestra.run.smithi103.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-17T06:49:38.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:38.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo shutdown -r now 2024-04-17T06:49:47.540 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi098.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-17T06:49:47.540 DEBUG:teuthology.orchestra.run.smithi098:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-17T06:49:47.614 INFO:teuthology.orchestra.run.smithi098.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:47.614 INFO:teuthology.orchestra.run.smithi098.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-17T06:49:47.615 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-17T06:49:47.616 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-17T06:49:47.617 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-17T06:49:47.618 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-17T06:49:47.619 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-17T06:49:47.620 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-17T06:49:47.621 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-17T06:49:47.622 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-17T06:49:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-17T06:49:47.624 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-17T06:49:47.625 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-17T06:49:47.626 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-17T06:49:47.627 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-17T06:49:47.628 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-17T06:49:47.629 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-17T06:49:47.629 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-17T06:49:47.629 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-17T06:49:47.629 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-17T06:49:47.629 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-17T06:49:47.629 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-17T06:49:47.629 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-17T06:49:47.629 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-17T06:49:47.630 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-17T06:49:47.630 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-17T06:49:47.630 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-17T06:49:47.630 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-17T06:49:47.630 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-17T06:49:47.630 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-17T06:49:47.631 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-17T06:49:47.631 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-17T06:49:47.631 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-17T06:49:47.631 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-17T06:49:47.631 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-17T06:49:47.631 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-17T06:49:47.632 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-17T06:49:47.632 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-17T06:49:47.632 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-17T06:49:47.632 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-17T06:49:47.632 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-17T06:49:47.632 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-17T06:49:47.632 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-17T06:49:47.632 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-17T06:49:47.633 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-17T06:49:47.634 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-17T06:49:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-17T06:49:47.636 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-17T06:49:47.636 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-17T06:49:47.636 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-17T06:49:47.636 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-17T06:49:47.636 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-17T06:49:47.636 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-17T06:49:47.636 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-17T06:49:47.636 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-17T06:49:47.637 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-17T06:49:47.638 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-17T06:49:47.639 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-17T06:49:47.640 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-17T06:49:47.641 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-17T06:49:47.642 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-17T06:49:47.643 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-17T06:49:47.644 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-17T06:49:47.645 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-17T06:49:47.646 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-17T06:49:47.647 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-17T06:49:47.648 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-17T06:49:47.649 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-17T06:49:47.650 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-17T06:49:47.650 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-17T06:49:47.650 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-17T06:49:47.650 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-17T06:49:47.650 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-17T06:49:47.650 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-17T06:49:47.650 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-17T06:49:47.650 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-17T06:49:47.651 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-17T06:49:47.651 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-17T06:49:47.651 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-17T06:49:47.651 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-17T06:49:47.651 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-17T06:49:47.651 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-17T06:49:47.652 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-17T06:49:47.653 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-17T06:49:47.654 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-17T06:49:47.655 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-17T06:49:47.656 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-17T06:49:47.657 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-17T06:49:47.658 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-17T06:49:47.659 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-17T06:49:47.660 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-17T06:49:47.661 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-17T06:49:47.662 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-17T06:49:47.662 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-17T06:49:47.662 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-17T06:49:47.662 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-17T06:49:47.662 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-17T06:49:47.662 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-17T06:49:47.662 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-17T06:49:47.663 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-17T06:49:47.663 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-17T06:49:47.663 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-17T06:49:47.663 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-17T06:49:47.663 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-17T06:49:47.663 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-17T06:49:47.664 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-17T06:49:47.664 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-17T06:49:47.664 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-17T06:49:47.664 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-17T06:49:47.664 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-17T06:49:47.665 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-17T06:49:47.665 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-17T06:49:47.665 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-17T06:49:47.665 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-17T06:49:47.665 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-17T06:49:47.665 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-17T06:49:47.666 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-17T06:49:47.666 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-17T06:49:47.666 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-17T06:49:47.666 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-17T06:49:47.666 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-17T06:49:47.666 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-17T06:49:47.667 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-17T06:49:47.667 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-17T06:49:47.667 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-17T06:49:47.667 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-17T06:49:47.667 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-17T06:49:47.668 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-17T06:49:47.668 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-17T06:49:47.668 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-17T06:49:47.668 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-17T06:49:47.668 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-17T06:49:47.669 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-17T06:49:47.669 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-17T06:49:47.669 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-17T06:49:47.669 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-17T06:49:47.669 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-17T06:49:47.670 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-17T06:49:47.670 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-17T06:49:47.670 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-17T06:49:47.670 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-17T06:49:47.670 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-17T06:49:47.670 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-17T06:49:47.671 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-17T06:49:47.671 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-17T06:49:47.671 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-17T06:49:47.671 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-17T06:49:47.671 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-17T06:49:47.671 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-17T06:49:47.671 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-17T06:49:47.671 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-17T06:49:47.672 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-17T06:49:47.672 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-17T06:49:47.672 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-17T06:49:47.672 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-17T06:49:47.672 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-17T06:49:47.672 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-17T06:49:47.673 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-17T06:49:47.673 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-17T06:49:47.673 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-17T06:49:47.673 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-17T06:49:47.673 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-17T06:49:47.673 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-17T06:49:47.673 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-17T06:49:47.674 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-17T06:49:47.674 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-17T06:49:47.674 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-17T06:49:47.674 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-17T06:49:47.674 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-17T06:49:47.674 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-17T06:49:47.674 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-17T06:49:47.675 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-17T06:49:47.675 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-17T06:49:47.675 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-17T06:49:47.675 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-17T06:49:47.675 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-17T06:49:47.675 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-17T06:49:47.675 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-17T06:49:47.675 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-17T06:49:47.676 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-17T06:49:47.676 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-17T06:49:47.676 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-17T06:49:47.676 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-17T06:49:47.676 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-17T06:49:47.676 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-17T06:49:47.677 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-17T06:49:47.677 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-17T06:49:47.677 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-17T06:49:47.677 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-17T06:49:47.677 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-17T06:49:47.677 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-17T06:49:47.677 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-17T06:49:47.678 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-17T06:49:47.678 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-17T06:49:47.678 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-17T06:49:47.678 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-17T06:49:47.678 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-17T06:49:47.678 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-17T06:49:47.679 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-17T06:49:47.679 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-17T06:49:47.679 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-17T06:49:47.679 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-17T06:49:47.679 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-17T06:49:47.680 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-17T06:49:47.680 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-17T06:49:47.680 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-17T06:49:47.680 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-17T06:49:47.680 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-17T06:49:47.680 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-17T06:49:47.680 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-17T06:49:47.681 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-17T06:49:47.681 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-17T06:49:47.681 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-17T06:49:47.681 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-17T06:49:47.681 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-17T06:49:47.681 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-17T06:49:47.682 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-17T06:49:47.682 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-17T06:49:47.682 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-17T06:49:47.682 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-17T06:49:47.682 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-17T06:49:47.682 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-17T06:49:47.683 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-17T06:49:47.683 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-17T06:49:47.683 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-17T06:49:47.683 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-17T06:49:47.683 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-17T06:49:47.683 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-17T06:49:47.683 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-17T06:49:47.683 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-17T06:49:47.684 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-17T06:49:47.684 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-17T06:49:47.684 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-17T06:49:47.684 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-17T06:49:47.684 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-17T06:49:47.684 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-17T06:49:47.684 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-17T06:49:47.684 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-17T06:49:47.684 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-17T06:49:47.685 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-17T06:49:47.685 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-17T06:49:47.685 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-17T06:49:47.685 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-17T06:49:47.685 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-17T06:49:47.685 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-17T06:49:47.685 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-17T06:49:47.685 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-17T06:49:47.686 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-17T06:49:47.686 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-17T06:49:47.686 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-17T06:49:47.686 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-17T06:49:47.686 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-17T06:49:47.686 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-17T06:49:47.686 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-17T06:49:47.687 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-17T06:49:47.687 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-17T06:49:47.687 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-17T06:49:47.687 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-17T06:49:47.687 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-17T06:49:47.687 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-17T06:49:47.687 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-17T06:49:47.688 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-17T06:49:47.688 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-17T06:49:47.688 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-17T06:49:47.688 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-17T06:49:47.688 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-17T06:49:47.688 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-17T06:49:47.689 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-17T06:49:47.689 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-17T06:49:47.689 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-17T06:49:47.689 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-17T06:49:47.689 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-17T06:49:47.689 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-17T06:49:47.689 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-17T06:49:47.690 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-17T06:49:47.690 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-17T06:49:47.690 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-17T06:49:47.690 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-17T06:49:47.690 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-17T06:49:47.690 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-17T06:49:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-17T06:49:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-17T06:49:47.693 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-17T06:49:47.694 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-17T06:49:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-17T06:49:47.696 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-17T06:49:47.697 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-17T06:49:47.698 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-17T06:49:47.699 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-17T06:49:47.700 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-17T06:49:47.701 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-17T06:49:47.701 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-17T06:49:47.701 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-17T06:49:47.701 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-17T06:49:47.701 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-17T06:49:47.701 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-17T06:49:47.701 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-17T06:49:47.701 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-17T06:49:47.702 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-17T06:49:47.703 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-17T06:49:47.704 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-17T06:49:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-17T06:49:47.706 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-17T06:49:47.707 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-17T06:49:47.708 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-17T06:49:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-17T06:49:47.710 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-17T06:49:47.711 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-17T06:49:47.712 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-17T06:49:47.712 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-17T06:49:47.712 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-17T06:49:47.712 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-17T06:49:47.712 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-17T06:49:47.712 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-17T06:49:47.712 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-17T06:49:47.713 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-17T06:49:47.713 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-17T06:49:47.713 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-17T06:49:47.713 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-17T06:49:47.713 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-17T06:49:47.713 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-17T06:49:47.713 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-17T06:49:47.713 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-17T06:49:47.713 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-17T06:49:47.714 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-17T06:49:47.714 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-17T06:49:47.714 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-17T06:49:47.714 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-17T06:49:47.714 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-17T06:49:47.714 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-17T06:49:47.714 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-17T06:49:47.715 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-17T06:49:47.715 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-17T06:49:47.715 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-17T06:49:47.715 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-17T06:49:47.715 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-17T06:49:47.715 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-17T06:49:47.715 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-17T06:49:47.715 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-17T06:49:47.715 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-17T06:49:47.715 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-17T06:49:47.716 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-17T06:49:47.716 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-17T06:49:47.716 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-17T06:49:47.716 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-17T06:49:47.716 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-17T06:49:47.716 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-17T06:49:47.716 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-17T06:49:47.716 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-17T06:49:47.717 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-17T06:49:47.717 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-17T06:49:47.717 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-17T06:49:47.717 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-17T06:49:47.717 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-17T06:49:47.717 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-17T06:49:47.717 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-17T06:49:47.717 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-17T06:49:47.717 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-17T06:49:47.718 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-17T06:49:47.719 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-17T06:49:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-17T06:49:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-17T06:49:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-17T06:49:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-17T06:49:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-17T06:49:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-17T06:49:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-17T06:49:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-17T06:49:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-17T06:49:47.720 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-17T06:49:47.721 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-17T06:49:47.722 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-17T06:49:47.723 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-17T06:49:47.724 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-17T06:49:47.725 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-17T06:49:47.726 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-17T06:49:47.727 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-17T06:49:47.728 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-17T06:49:47.729 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-17T06:49:47.730 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-17T06:49:47.731 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-17T06:49:47.732 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-17T06:49:47.733 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-17T06:49:47.733 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-17T06:49:47.733 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-17T06:49:47.733 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-17T06:49:47.733 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-17T06:49:47.733 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-17T06:49:47.733 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-17T06:49:47.733 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-17T06:49:47.734 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-17T06:49:47.735 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-17T06:49:47.736 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-17T06:49:47.736 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-17T06:49:47.736 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-17T06:49:47.736 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-17T06:49:47.736 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-17T06:49:47.736 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-17T06:49:47.736 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-17T06:49:47.736 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-17T06:49:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-17T06:49:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-17T06:49:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-17T06:49:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-17T06:49:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-17T06:49:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-17T06:49:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-17T06:49:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-17T06:49:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-17T06:49:47.738 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-17T06:49:47.738 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-17T06:49:47.738 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-17T06:49:47.738 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-17T06:49:47.738 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-17T06:49:47.738 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-17T06:49:47.738 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-17T06:49:47.739 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-17T06:49:47.739 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-17T06:49:47.739 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-17T06:49:47.739 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-17T06:49:47.739 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-17T06:49:47.739 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-17T06:49:47.739 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-17T06:49:47.739 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-17T06:49:47.739 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-17T06:49:47.740 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-17T06:49:47.740 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-17T06:49:47.740 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-17T06:49:47.740 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-17T06:49:47.740 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-17T06:49:47.740 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-17T06:49:47.740 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-17T06:49:47.741 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-17T06:49:47.741 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-17T06:49:47.741 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-17T06:49:47.741 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-17T06:49:47.741 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-17T06:49:47.741 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-17T06:49:47.742 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-17T06:49:47.742 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-17T06:49:47.742 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-17T06:49:47.742 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-17T06:49:47.742 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-17T06:49:47.742 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-17T06:49:47.742 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-17T06:49:47.743 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-17T06:49:47.743 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-17T06:49:47.743 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-17T06:49:47.743 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-17T06:49:47.743 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-17T06:49:47.743 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-17T06:49:47.743 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-17T06:49:47.744 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-17T06:49:47.744 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-17T06:49:47.744 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-17T06:49:47.744 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-17T06:49:47.744 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-17T06:49:47.744 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-17T06:49:47.744 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-17T06:49:47.744 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-17T06:49:47.745 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-17T06:49:47.745 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-17T06:49:47.745 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-17T06:49:47.745 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-17T06:49:47.745 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-17T06:49:47.745 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-17T06:49:47.745 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-17T06:49:47.745 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-17T06:49:47.746 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-17T06:49:47.746 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-17T06:49:47.746 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-17T06:49:47.746 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-17T06:49:47.746 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-17T06:49:47.746 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-17T06:49:47.747 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-17T06:49:47.747 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-17T06:49:47.747 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-17T06:49:47.747 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-17T06:49:47.747 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-17T06:49:47.747 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-17T06:49:47.747 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-17T06:49:47.748 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-17T06:49:47.748 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-17T06:49:47.748 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-17T06:49:47.748 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-17T06:49:47.748 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-17T06:49:47.748 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-17T06:49:47.748 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-17T06:49:47.748 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-17T06:49:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-17T06:49:47.750 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-17T06:49:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-17T06:49:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-17T06:49:47.753 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-17T06:49:47.754 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-17T06:49:47.755 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-17T06:49:47.756 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-17T06:49:47.757 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-17T06:49:47.758 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-17T06:49:47.759 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-17T06:49:47.760 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-17T06:49:47.761 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-17T06:49:47.762 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-17T06:49:47.763 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-17T06:49:47.764 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-17T06:49:47.765 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-17T06:49:47.766 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-17T06:49:47.767 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-17T06:49:47.768 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-17T06:49:47.769 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-17T06:49:47.770 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-17T06:49:47.771 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-17T06:49:47.772 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-17T06:49:47.773 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-17T06:49:47.774 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-17T06:49:47.775 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-17T06:49:47.776 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-17T06:49:47.777 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-17T06:49:47.778 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-17T06:49:47.778 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-17T06:49:47.778 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-17T06:49:47.778 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-17T06:49:47.778 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-17T06:49:47.778 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-17T06:49:47.778 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-17T06:49:47.778 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-17T06:49:47.778 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-17T06:49:47.779 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-17T06:49:47.780 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-17T06:49:47.781 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-17T06:49:47.782 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-17T06:49:47.783 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-17T06:49:47.784 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-17T06:49:47.785 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-17T06:49:47.785 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-17T06:49:47.785 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-17T06:49:47.785 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-17T06:49:47.785 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-17T06:49:47.785 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-17T06:49:47.785 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-17T06:49:47.785 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-17T06:49:47.785 INFO:teuthology.orchestra.run.smithi098.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-17T06:49:47.789 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:47.789 DEBUG:teuthology.orchestra.run.smithi098:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-17T06:49:47.812 INFO:teuthology.orchestra.run.smithi098.stdout:postinstall scriptlet (using /bin/sh): 2024-04-17T06:49:47.812 INFO:teuthology.orchestra.run.smithi098.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-17T06:49:47.812 INFO:teuthology.orchestra.run.smithi098.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-17T06:49:47.812 INFO:teuthology.orchestra.run.smithi098.stdout:fi 2024-04-17T06:49:47.812 INFO:teuthology.orchestra.run.smithi098.stdout:for file in vmlinuz System.map config; do 2024-04-17T06:49:47.812 INFO:teuthology.orchestra.run.smithi098.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-17T06:49:47.812 INFO:teuthology.orchestra.run.smithi098.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T06:49:47.812 INFO:teuthology.orchestra.run.smithi098.stdout: fi 2024-04-17T06:49:47.812 INFO:teuthology.orchestra.run.smithi098.stdout:done 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout:fi 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout:fi 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:47.813 INFO:teuthology.orchestra.run.smithi098.stdout:fi 2024-04-17T06:49:47.814 DEBUG:teuthology.orchestra.run.smithi098:> sudo rpm -qi grub2-tools 2024-04-17T06:49:47.889 INFO:teuthology.orchestra.run.smithi098.stdout:Name : grub2-tools 2024-04-17T06:49:47.889 INFO:teuthology.orchestra.run.smithi098.stdout:Epoch : 1 2024-04-17T06:49:47.889 INFO:teuthology.orchestra.run.smithi098.stdout:Version : 2.06 2024-04-17T06:49:47.890 INFO:teuthology.orchestra.run.smithi098.stdout:Release : 68.el9 2024-04-17T06:49:47.890 INFO:teuthology.orchestra.run.smithi098.stdout:Architecture: x86_64 2024-04-17T06:49:47.890 INFO:teuthology.orchestra.run.smithi098.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-17T06:49:47.890 INFO:teuthology.orchestra.run.smithi098.stdout:Group : Unspecified 2024-04-17T06:49:47.890 INFO:teuthology.orchestra.run.smithi098.stdout:Size : 8264254 2024-04-17T06:49:47.890 INFO:teuthology.orchestra.run.smithi098.stdout:License : GPLv3+ 2024-04-17T06:49:47.890 INFO:teuthology.orchestra.run.smithi098.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-17T06:49:47.890 INFO:teuthology.orchestra.run.smithi098.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-17T06:49:47.890 INFO:teuthology.orchestra.run.smithi098.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-17T06:49:47.891 INFO:teuthology.orchestra.run.smithi098.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-17T06:49:47.891 INFO:teuthology.orchestra.run.smithi098.stdout:Packager : builder@centos.org 2024-04-17T06:49:47.891 INFO:teuthology.orchestra.run.smithi098.stdout:Vendor : CentOS 2024-04-17T06:49:47.891 INFO:teuthology.orchestra.run.smithi098.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-17T06:49:47.891 INFO:teuthology.orchestra.run.smithi098.stdout:Summary : Support tools for GRUB. 2024-04-17T06:49:47.891 INFO:teuthology.orchestra.run.smithi098.stdout:Description : 2024-04-17T06:49:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:49:47.891 INFO:teuthology.orchestra.run.smithi098.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-17T06:49:47.891 INFO:teuthology.orchestra.run.smithi098.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-17T06:49:47.891 INFO:teuthology.orchestra.run.smithi098.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-17T06:49:47.892 INFO:teuthology.orchestra.run.smithi098.stdout:hardware devices. 2024-04-17T06:49:47.892 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:49:47.892 INFO:teuthology.orchestra.run.smithi098.stdout:This subpackage provides tools for support of all platforms. 2024-04-17T06:49:47.893 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-17T06:49:47.893 INFO:teuthology.task.kernel:Updating grub on smithi098 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:47.893 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-17T06:49:48.458 INFO:teuthology.orchestra.run.smithi098.stderr:Generating grub configuration file ... 2024-04-17T06:49:49.686 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-17T06:49:49.687 DEBUG:teuthology.orchestra.run.smithi086:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-17T06:49:49.757 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:49.757 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:49.757 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-17T06:49:49.758 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-17T06:49:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-17T06:49:49.760 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-17T06:49:49.761 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-17T06:49:49.762 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-17T06:49:49.763 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-17T06:49:49.764 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-17T06:49:49.765 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-17T06:49:49.766 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-17T06:49:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-17T06:49:49.768 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-17T06:49:49.769 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-17T06:49:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-17T06:49:49.771 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-17T06:49:49.772 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-17T06:49:49.773 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-17T06:49:49.774 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-17T06:49:49.775 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-17T06:49:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-17T06:49:49.777 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-17T06:49:49.778 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-17T06:49:49.779 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-17T06:49:49.780 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-17T06:49:49.781 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-17T06:49:49.782 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-17T06:49:49.783 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-17T06:49:49.784 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-17T06:49:49.785 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-17T06:49:49.786 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-17T06:49:49.787 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-17T06:49:49.788 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-17T06:49:49.789 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-17T06:49:49.790 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-17T06:49:49.791 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-17T06:49:49.792 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-17T06:49:49.793 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-17T06:49:49.794 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-17T06:49:49.795 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-17T06:49:49.796 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-17T06:49:49.797 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-17T06:49:49.798 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-17T06:49:49.799 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-17T06:49:49.800 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-17T06:49:49.801 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-17T06:49:49.802 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-17T06:49:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-17T06:49:49.804 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-17T06:49:49.805 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-17T06:49:49.806 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-17T06:49:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-17T06:49:49.808 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-17T06:49:49.809 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-17T06:49:49.810 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-17T06:49:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-17T06:49:49.812 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-17T06:49:49.813 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-17T06:49:49.814 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-17T06:49:49.815 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-17T06:49:49.816 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-17T06:49:49.817 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-17T06:49:49.818 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-17T06:49:49.819 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-17T06:49:49.820 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-17T06:49:49.821 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-17T06:49:49.822 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-17T06:49:49.823 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-17T06:49:49.824 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-17T06:49:49.825 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-17T06:49:49.826 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-17T06:49:49.827 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-17T06:49:49.828 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-17T06:49:49.829 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-17T06:49:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-17T06:49:49.831 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-17T06:49:49.832 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-17T06:49:49.833 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-17T06:49:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-17T06:49:49.835 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-17T06:49:49.836 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-17T06:49:49.837 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-17T06:49:49.838 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-17T06:49:49.839 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-17T06:49:49.840 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-17T06:49:49.841 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-17T06:49:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-17T06:49:49.843 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-17T06:49:49.844 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-17T06:49:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-17T06:49:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-17T06:49:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-17T06:49:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-17T06:49:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-17T06:49:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-17T06:49:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-17T06:49:49.852 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-17T06:49:49.853 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-17T06:49:49.854 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-17T06:49:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-17T06:49:49.856 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-17T06:49:49.857 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-17T06:49:49.858 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-17T06:49:49.859 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-17T06:49:49.860 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-17T06:49:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-17T06:49:49.862 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-17T06:49:49.863 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-17T06:49:49.864 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-17T06:49:49.865 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-17T06:49:49.866 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-17T06:49:49.867 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-17T06:49:49.868 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-17T06:49:49.869 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-17T06:49:49.870 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-17T06:49:49.871 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-17T06:49:49.872 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-17T06:49:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-17T06:49:49.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-17T06:49:49.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-17T06:49:49.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-17T06:49:49.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-17T06:49:49.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-17T06:49:49.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-17T06:49:49.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-17T06:49:49.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-17T06:49:49.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-17T06:49:49.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-17T06:49:49.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-17T06:49:49.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-17T06:49:49.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-17T06:49:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-17T06:49:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-17T06:49:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-17T06:49:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-17T06:49:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-17T06:49:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-17T06:49:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-17T06:49:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-17T06:49:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-17T06:49:49.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-17T06:49:49.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-17T06:49:49.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-17T06:49:49.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-17T06:49:49.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-17T06:49:49.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-17T06:49:49.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-17T06:49:49.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-17T06:49:49.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-17T06:49:49.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-17T06:49:49.903 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:49.903 DEBUG:teuthology.orchestra.run.smithi086:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-17T06:49:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-17T06:49:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:done 2024-04-17T06:49:49.905 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-04-17T06:49:49.921 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.Kh4ouKjnQF 2024-04-17T06:49:49.921 DEBUG:teuthology.orchestra.run.smithi098:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Kh4ouKjnQF 2024-04-17T06:49:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:postinstall scriptlet (using /bin/sh): 2024-04-17T06:49:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-17T06:49:49.926 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-17T06:49:49.926 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:for file in vmlinuz System.map config; do 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout: fi 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:done 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-17T06:49:49.927 INFO:teuthology.orchestra.run.smithi086.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-17T06:49:49.928 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-17T06:49:49.928 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:49.928 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-17T06:49:49.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -qi grub2-tools 2024-04-17T06:49:49.996 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.Kh4ouKjnQF 2024-04-17T06:49:50.004 INFO:teuthology.orchestra.run.smithi086.stdout:Name : grub2-tools 2024-04-17T06:49:50.005 INFO:teuthology.orchestra.run.smithi086.stdout:Epoch : 1 2024-04-17T06:49:50.005 INFO:teuthology.orchestra.run.smithi086.stdout:Version : 2.06 2024-04-17T06:49:50.005 INFO:teuthology.orchestra.run.smithi086.stdout:Release : 68.el9 2024-04-17T06:49:50.005 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: x86_64 2024-04-17T06:49:50.005 INFO:teuthology.orchestra.run.smithi086.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-17T06:49:50.005 INFO:teuthology.orchestra.run.smithi086.stdout:Group : Unspecified 2024-04-17T06:49:50.005 INFO:teuthology.orchestra.run.smithi086.stdout:Size : 8264254 2024-04-17T06:49:50.005 INFO:teuthology.orchestra.run.smithi086.stdout:License : GPLv3+ 2024-04-17T06:49:50.005 INFO:teuthology.orchestra.run.smithi086.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-17T06:49:50.005 INFO:teuthology.orchestra.run.smithi086.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:Packager : builder@centos.org 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:Vendor : CentOS 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:Summary : Support tools for GRUB. 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:Description : 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-17T06:49:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-17T06:49:50.007 INFO:teuthology.orchestra.run.smithi086.stdout:hardware devices. 2024-04-17T06:49:50.007 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:49:50.007 INFO:teuthology.orchestra.run.smithi086.stdout:This subpackage provides tools for support of all platforms. 2024-04-17T06:49:50.008 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-17T06:49:50.008 INFO:teuthology.task.kernel:Updating grub on smithi086 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:50.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-17T06:49:50.131 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.Kh4ouKjnQF is 6KB 2024-04-17T06:49:50.179 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.Kh4ouKjnQF 2024-04-17T06:49:50.193 DEBUG:teuthology.orchestra.run.smithi098:> sudo /bin/ls /boot/loader/entries || true 2024-04-17T06:49:50.267 INFO:teuthology.orchestra.run.smithi098.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-17T06:49:50.267 INFO:teuthology.orchestra.run.smithi098.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-17T06:49:50.267 INFO:teuthology.orchestra.run.smithi098.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-17T06:49:50.267 INFO:teuthology.orchestra.run.smithi098.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-17T06:49:50.268 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:50.433 DEBUG:teuthology.orchestra.run.smithi098:> sudo shutdown -r now 2024-04-17T06:49:50.565 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-04-17T06:49:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-17T06:49:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-04-17T06:49:52.021 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-17T06:49:52.035 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.yOtihbkOnf 2024-04-17T06:49:52.036 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yOtihbkOnf 2024-04-17T06:49:52.104 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.yOtihbkOnf 2024-04-17T06:49:52.247 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.yOtihbkOnf is 6KB 2024-04-17T06:49:52.296 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.yOtihbkOnf 2024-04-17T06:49:52.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo /bin/ls /boot/loader/entries || true 2024-04-17T06:49:52.375 INFO:teuthology.orchestra.run.smithi086.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-17T06:49:52.375 INFO:teuthology.orchestra.run.smithi086.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-17T06:49:52.375 INFO:teuthology.orchestra.run.smithi086.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-17T06:49:52.375 INFO:teuthology.orchestra.run.smithi086.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-17T06:49:52.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:49:52.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-04-17T06:50:08.744 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-04-17T06:50:08.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-17T06:50:08.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:50:20.446 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-17T06:50:20.447 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-17T06:50:20.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:50:22.562 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-17T06:50:22.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-17T06:50:22.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:50:27.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-04-17T06:50:36.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-17T06:50:36.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:50:39.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-04-17T06:50:39.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-04-17T06:50:41.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-17T06:50:48.014 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-17T06:50:48.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:50:50.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-17T06:50:50.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:50:51.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-04-17T06:50:51.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-17T06:50:51.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:50:53.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-17T06:50:54.370 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-04-17T06:51:03.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-17T06:51:03.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:51:05.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-17T06:51:05.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:51:09.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-17T06:51:09.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:51:27.903 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-04-17T06:51:34.915 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-04-17T06:51:35.518 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-17T06:51:35.519 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-17T06:51:35.519 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-04-17T06:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:51:35.558 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:51:35.558 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T06:51:35.558 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-17T06:51:36.558 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-17T06:51:36.559 DEBUG:teuthology.orchestra.run.smithi098:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T06:51:36.666 INFO:teuthology.orchestra.run.smithi098.stdout:ttyS1 2024-04-17T06:51:36.729 DEBUG:teuthology.parallel:result is None 2024-04-17T06:51:36.975 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-17T06:51:37.679 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-17T06:51:37.679 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-17T06:51:37.679 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-17T06:51:37.715 INFO:teuthology.orchestra.run.smithi086.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:51:37.715 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:51:37.715 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T06:51:37.715 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-17T06:51:38.716 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-17T06:51:38.716 DEBUG:teuthology.orchestra.run.smithi086:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T06:51:38.836 INFO:teuthology.orchestra.run.smithi086.stdout:ttyS1 2024-04-17T06:51:38.900 DEBUG:teuthology.parallel:result is None 2024-04-17T06:51:45.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-17T06:51:45.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:51:46.345 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-04-17T06:51:46.917 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-17T06:51:46.917 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-17T06:51:46.917 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-04-17T06:51:46.960 INFO:teuthology.orchestra.run.smithi103.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:51:46.960 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-17T06:51:46.961 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T06:51:46.961 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-17T06:51:47.961 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-17T06:51:47.962 DEBUG:teuthology.orchestra.run.smithi103:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-17T06:51:48.090 INFO:teuthology.orchestra.run.smithi103.stdout:ttyS1 2024-04-17T06:51:48.150 DEBUG:teuthology.parallel:result is None 2024-04-17T06:51:48.150 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-17T06:51:48.158 INFO:teuthology.task.internal:Creating test directory... 2024-04-17T06:51:48.158 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T06:51:48.164 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T06:51:48.178 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T06:51:48.248 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-17T06:51:48.257 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-17T06:51:48.267 INFO:teuthology.task.internal:Creating archive directory... 2024-04-17T06:51:48.267 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T06:51:48.273 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T06:51:48.282 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T06:51:48.366 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-17T06:51:48.373 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-17T06:51:48.373 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T06:51:48.377 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T06:51:48.388 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T06:51:48.450 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T06:51:48.483 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T06:51:48.492 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T06:51:48.525 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T06:51:48.530 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T06:51:48.588 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T06:51:48.602 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-17T06:51:48.610 INFO:teuthology.task.internal:Configuring sudo... 2024-04-17T06:51:48.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T06:51:48.614 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T06:51:48.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T06:51:48.750 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-17T06:51:48.759 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-17T06:51:48.760 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T06:51:48.763 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T06:51:48.771 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T06:51:48.841 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T06:51:48.911 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T06:51:49.007 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T06:51:49.074 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T06:51:49.160 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T06:51:49.160 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T06:51:49.242 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T06:51:49.320 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T06:51:49.408 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T06:51:49.476 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T06:51:49.565 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T06:51:49.565 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T06:51:49.643 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T06:51:49.714 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T06:51:49.820 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T06:51:49.892 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T06:51:49.980 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T06:51:49.981 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T06:51:50.064 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-04-17T06:51:50.071 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2024-04-17T06:51:50.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-04-17T06:51:50.194 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T06:51:50.196 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T06:51:50.198 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T06:51:50.707 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-17T06:51:50.715 INFO:teuthology.task.internal:Starting timer... 2024-04-17T06:51:50.715 INFO:teuthology.run_tasks:Running task pcp... 2024-04-17T06:51:50.731 INFO:teuthology.run_tasks:Running task selinux... 2024-04-17T06:51:50.739 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-17T06:51:50.740 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-04-17T06:51:50.882 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-04-17T06:51:50.887 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2024-04-17T06:51:51.045 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2024-04-17T06:51:51.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-04-17T06:51:51.160 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-04-17T06:51:51.166 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-17T06:51:51.166 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-04-17T06:51:51.212 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-04-17T06:51:51.216 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2024-04-17T06:51:51.280 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2024-04-17T06:51:51.280 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-04-17T06:51:51.322 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-04-17T06:51:51.323 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi098.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi103.front.sepia.ceph.com': 'permissive'} 2024-04-17T06:51:51.323 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T06:51:51.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:51:51.465 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T06:51:51.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:51:51.563 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T06:51:51.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:51:51.639 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-17T06:51:51.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-04-17T06:51:51.773 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2024-04-17T06:51:51.877 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-04-17T06:51:51.959 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-17T06:51:51.970 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-17T06:51:51.976 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-17T06:51:52.108 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-17T06:51:52.125 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-17T06:51:52.126 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com,smithi098.front.sepia.ceph.com,smithi103.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-17T06:57:55.074 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi103.front.sepia.ceph.com')] 2024-04-17T06:57:55.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-17T06:57:55.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:57:55.226 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-17T06:57:55.383 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-17T06:57:55.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-17T06:57:55.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:57:55.524 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-04-17T06:57:55.677 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-17T06:57:55.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-17T06:57:55.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:57:55.852 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-04-17T06:57:56.019 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-17T06:57:56.020 INFO:teuthology.run_tasks:Running task clock... 2024-04-17T06:57:56.031 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-17T06:57:56.032 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T06:57:56.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T06:57:56.040 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T06:57:56.040 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T06:57:56.047 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T06:57:56.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T06:57:56.133 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T06:57:56.146 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T06:57:56.148 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T06:57:56.184 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T06:57:56.197 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T06:57:56.201 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T06:57:56.276 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-04-17T06:57:56.296 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2024-04-17T06:57:56.296 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-04-17T06:57:56.319 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-04-17T06:57:56.336 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2024-04-17T06:57:56.343 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-04-17T06:57:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T06:57:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T06:57:56.392 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T06:57:56.425 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T06:57:56.441 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T06:57:56.441 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T06:57:56.569 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-04-17T06:57:56.573 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-17T06:57:56.577 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T06:57:56.577 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-04-17T06:57:56.577 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.577 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.578 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.578 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.579 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-04-17T06:57:56.581 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T06:57:56.581 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-17T06:57:56.581 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.581 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.581 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.581 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.588 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T06:57:56.588 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-04-17T06:57:56.588 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.589 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.589 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.589 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:57:56.590 INFO:teuthology.run_tasks:Running task install... 2024-04-17T06:57:56.598 DEBUG:teuthology.task.install:project ceph 2024-04-17T06:57:56.599 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-17T06:57:56.599 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-17T06:57:56.599 INFO:teuthology.task.install:Using flavor: default 2024-04-17T06:57:56.604 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-17T06:57:56.605 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-17T06:57:56.605 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-17T06:57:56.605 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T06:57:56.607 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-17T06:57:56.607 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T06:57:56.608 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-17T06:57:56.609 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T06:57:56.827 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-17T06:57:56.828 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-17T06:57:56.836 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-17T06:57:56.836 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-17T06:57:56.843 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-17T06:57:56.843 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-17T06:57:56.950 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T06:57:56.950 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T06:57:56.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T06:57:56.953 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T06:57:56.954 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T06:57:56.954 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T06:57:57.008 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T06:57:57.008 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T06:57:57.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T06:57:57.056 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T06:57:57.057 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T06:57:57.058 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T06:57:57.058 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T06:57:57.106 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T06:57:57.106 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T06:57:57.197 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T06:57:57.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T06:57:57.245 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T06:57:57.328 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T06:57:57.329 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T06:57:57.374 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T06:57:57.418 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2024-04-17T06:57:57.420 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-04-17T06:57:57.422 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2024-04-17T06:57:57.425 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-04-17T06:57:57.460 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-04-17T06:57:57.465 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-04-17T06:57:57.785 INFO:teuthology.orchestra.run.smithi086.stdout:74 files removed 2024-04-17T06:57:57.786 INFO:teuthology.orchestra.run.smithi098.stdout:74 files removed 2024-04-17T06:57:57.812 INFO:teuthology.orchestra.run.smithi103.stdout:74 files removed 2024-04-17T06:57:57.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T06:57:57.825 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T06:57:57.851 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T06:57:58.806 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 189 kB/s | 84 kB 00:00 2024-04-17T06:57:58.812 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 176 kB/s | 84 kB 00:00 2024-04-17T06:57:58.840 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 165 kB/s | 84 kB 00:00 2024-04-17T06:57:59.371 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-17T06:57:59.373 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-17T06:57:59.408 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-17T06:57:59.816 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-04-17T06:57:59.913 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-04-17T06:57:59.924 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-04-17T06:58:00.537 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-17T06:58:00.541 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-17T06:58:00.701 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS-9 - AppStream 8.2 MB/s | 6.7 MB 00:00 2024-04-17T06:58:02.485 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS-9 - BaseOS 2.8 MB/s | 2.0 MB 00:00 2024-04-17T06:58:02.503 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-04-17T06:58:02.512 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-04-17T06:58:03.439 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-17T06:58:03.474 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-17T06:58:05.559 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-17T06:58:06.006 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-17T06:58:06.016 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.0 MB 00:03 2024-04-17T06:58:09.550 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 19 MB 00:02 2024-04-17T06:58:10.626 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.2 MB 00:01 2024-04-17T06:58:11.182 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.2 MB 00:01 2024-04-17T06:58:12.335 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-04-17T06:58:13.017 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-17T06:58:14.801 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 21 MB 00:02 2024-04-17T06:58:14.876 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-04-17T06:58:14.884 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-17T06:58:16.662 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-17T06:58:17.887 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-17T06:58:20.745 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-04-17T06:58:20.818 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-17T06:58:23.718 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T06:58:23.719 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T06:58:23.740 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T06:58:23.742 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T06:58:23.743 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T06:58:23.744 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T06:58:23.747 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T06:58:23.751 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T06:58:23.752 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T06:58:23.812 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-17T06:58:23.829 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T06:58:23.831 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T06:58:23.841 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-04-17T06:58:23.854 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T06:58:23.855 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T06:58:23.857 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T06:58:23.858 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T06:58:23.860 INFO:teuthology.orchestra.run.smithi103.stdout:============================================================================================ 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout:============================================================================================ 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-17T06:58:23.861 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-17T06:58:23.862 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T06:58:23.863 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T06:58:23.864 INFO:teuthology.orchestra.run.smithi103.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T06:58:23.865 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-17T06:58:23.866 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T06:58:23.867 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T06:58:23.868 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T06:58:23.869 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T06:58:23.870 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T06:58:23.871 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T06:58:23.872 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T06:58:23.873 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T06:58:23.874 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T06:58:23.874 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-04-17T06:58:23.874 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T06:58:23.874 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:58:23.874 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-04-17T06:58:23.874 INFO:teuthology.orchestra.run.smithi103.stdout:============================================================================================ 2024-04-17T06:58:23.874 INFO:teuthology.orchestra.run.smithi103.stdout:Install 150 Packages 2024-04-17T06:58:23.874 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 24 Packages 2024-04-17T06:58:23.874 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:58:23.875 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 229 M 2024-04-17T06:58:23.875 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-04-17T06:58:23.875 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T06:58:23.875 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T06:58:23.875 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T06:58:23.956 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-04-17T06:58:23.981 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================ 2024-04-17T06:58:23.981 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-04-17T06:58:23.981 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================ 2024-04-17T06:58:23.981 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-04-17T06:58:23.981 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T06:58:23.981 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-17T06:58:23.982 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T06:58:23.983 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T06:58:23.984 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T06:58:23.985 INFO:teuthology.orchestra.run.smithi098.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-17T06:58:23.986 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-17T06:58:23.987 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T06:58:23.988 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-17T06:58:23.989 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T06:58:23.990 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T06:58:23.991 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T06:58:23.992 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T06:58:23.993 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout:Installing weak dependencies: 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================================ 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout:Install 150 Packages 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 24 Packages 2024-04-17T06:58:23.994 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:58:23.995 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 229 M 2024-04-17T06:58:23.995 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-04-17T06:58:24.897 INFO:teuthology.orchestra.run.smithi103.stdout:(1/174): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-17T06:58:25.255 INFO:teuthology.orchestra.run.smithi103.stdout:(2/174): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 2.4 MB/s | 893 kB 00:00 2024-04-17T06:58:25.339 INFO:teuthology.orchestra.run.smithi103.stdout:(3/174): ceph-immutable-object-cache-19.0.0-313 1.7 MB/s | 145 kB 00:00 2024-04-17T06:58:25.420 INFO:teuthology.orchestra.run.smithi098.stdout:(1/174): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-17T06:58:25.449 INFO:teuthology.orchestra.run.smithi103.stdout:(4/174): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 6.2 MB/s | 5.4 MB 00:00 2024-04-17T06:58:25.690 INFO:teuthology.orchestra.run.smithi103.stdout:(5/174): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 6.2 MB/s | 1.5 MB 00:00 2024-04-17T06:58:26.073 INFO:teuthology.orchestra.run.smithi103.stdout:(6/174): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 3.2 MB/s | 2.4 MB 00:00 2024-04-17T06:58:26.398 INFO:teuthology.orchestra.run.smithi103.stdout:(7/174): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-04-17T06:58:26.503 INFO:teuthology.orchestra.run.smithi098.stdout:(2/174): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 3.8 MB/s | 5.4 MB 00:01 2024-04-17T06:58:26.595 INFO:teuthology.orchestra.run.smithi098.stdout:(3/174): ceph-immutable-object-cache-19.0.0-313 1.6 MB/s | 145 kB 00:00 2024-04-17T06:58:26.637 INFO:teuthology.orchestra.run.smithi098.stdout:(4/174): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 734 kB/s | 893 kB 00:01 2024-04-17T06:58:26.688 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T06:58:26.690 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T06:58:26.712 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T06:58:26.713 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T06:58:26.714 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T06:58:26.716 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T06:58:26.718 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T06:58:26.722 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T06:58:26.723 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T06:58:26.810 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================================ 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================================ 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-17T06:58:26.837 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T06:58:26.838 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-17T06:58:26.839 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T06:58:26.840 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T06:58:26.841 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-17T06:58:26.842 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T06:58:26.843 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T06:58:26.844 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T06:58:26.845 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T06:58:26.846 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T06:58:26.847 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T06:58:26.848 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T06:58:26.849 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================================ 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout:Install 150 Packages 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 24 Packages 2024-04-17T06:58:26.850 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:58:26.851 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 229 M 2024-04-17T06:58:26.851 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-04-17T06:58:27.024 INFO:teuthology.orchestra.run.smithi103.stdout:(8/174): ceph-common-19.0.0-3133.ga8c84f59.el9. 8.7 MB/s | 21 MB 00:02 2024-04-17T06:58:27.071 INFO:teuthology.orchestra.run.smithi098.stdout:(5/174): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 5.0 MB/s | 2.4 MB 00:00 2024-04-17T06:58:27.074 INFO:teuthology.orchestra.run.smithi103.stdout:(9/174): ceph-selinux-19.0.0-3133.ga8c84f59.el9 513 kB/s | 25 kB 00:00 2024-04-17T06:58:27.958 INFO:teuthology.orchestra.run.smithi103.stdout:(10/174): ceph-radosgw-19.0.0-3133.ga8c84f59.el 6.8 MB/s | 11 MB 00:01 2024-04-17T06:58:28.049 INFO:teuthology.orchestra.run.smithi103.stdout:(11/174): cephfs-mirror-19.0.0-3133.ga8c84f59.e 2.4 MB/s | 225 kB 00:00 2024-04-17T06:58:28.071 INFO:teuthology.orchestra.run.smithi098.stdout:(6/174): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 4.7 MB/s | 4.7 MB 00:00 2024-04-17T06:58:28.106 INFO:teuthology.orchestra.run.smithi086.stdout:(1/174): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-17T06:58:28.108 INFO:teuthology.orchestra.run.smithi103.stdout:(12/174): cephfs-shell-19.0.0-3133.ga8c84f59.el 468 kB/s | 27 kB 00:00 2024-04-17T06:58:28.155 INFO:teuthology.orchestra.run.smithi098.stdout:(7/174): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 1.0 MB/s | 1.5 MB 00:01 2024-04-17T06:58:28.183 INFO:teuthology.orchestra.run.smithi103.stdout:(13/174): libcephfs-devel-19.0.0-3133.ga8c84f59 428 kB/s | 32 kB 00:00 2024-04-17T06:58:28.484 INFO:teuthology.orchestra.run.smithi103.stdout:(14/174): libcephfs2-19.0.0-3133.ga8c84f59.el9. 2.4 MB/s | 748 kB 00:00 2024-04-17T06:58:28.626 INFO:teuthology.orchestra.run.smithi103.stdout:(15/174): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.1 MB/s | 163 kB 00:00 2024-04-17T06:58:28.689 INFO:teuthology.orchestra.run.smithi086.stdout:(2/174): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.5 MB/s | 893 kB 00:00 2024-04-17T06:58:28.693 INFO:teuthology.orchestra.run.smithi103.stdout:(16/174): librados-devel-19.0.0-3133.ga8c84f59. 1.9 MB/s | 126 kB 00:00 2024-04-17T06:58:28.747 INFO:teuthology.orchestra.run.smithi086.stdout:(3/174): ceph-immutable-object-cache-19.0.0-313 2.5 MB/s | 145 kB 00:00 2024-04-17T06:58:28.847 INFO:teuthology.orchestra.run.smithi098.stdout:(8/174): ceph-common-19.0.0-3133.ga8c84f59.el9. 5.6 MB/s | 21 MB 00:03 2024-04-17T06:58:28.918 INFO:teuthology.orchestra.run.smithi103.stdout:(17/174): libradosstriper1-19.0.0-3133.ga8c84f5 2.2 MB/s | 501 kB 00:00 2024-04-17T06:58:28.947 INFO:teuthology.orchestra.run.smithi098.stdout:(9/174): ceph-selinux-19.0.0-3133.ga8c84f59.el9 254 kB/s | 25 kB 00:00 2024-04-17T06:58:29.748 INFO:teuthology.orchestra.run.smithi086.stdout:(4/174): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-04-17T06:58:30.523 INFO:teuthology.orchestra.run.smithi086.stdout:(5/174): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 1.9 MB/s | 1.5 MB 00:00 2024-04-17T06:58:30.970 INFO:teuthology.orchestra.run.smithi103.stdout:(18/174): librgw2-19.0.0-3133.ga8c84f59.el9.x86 2.6 MB/s | 5.4 MB 00:02 2024-04-17T06:58:31.068 INFO:teuthology.orchestra.run.smithi103.stdout:(19/174): python3-ceph-argparse-19.0.0-3133.ga8 456 kB/s | 45 kB 00:00 2024-04-17T06:58:31.160 INFO:teuthology.orchestra.run.smithi103.stdout:(20/174): python3-ceph-common-19.0.0-3133.ga8c8 1.4 MB/s | 135 kB 00:00 2024-04-17T06:58:31.260 INFO:teuthology.orchestra.run.smithi103.stdout:(21/174): python3-cephfs-19.0.0-3133.ga8c84f59. 1.6 MB/s | 162 kB 00:00 2024-04-17T06:58:31.420 INFO:teuthology.orchestra.run.smithi103.stdout:(22/174): python3-rados-19.0.0-3133.ga8c84f59.e 2.0 MB/s | 321 kB 00:00 2024-04-17T06:58:31.531 INFO:teuthology.orchestra.run.smithi086.stdout:(6/174): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-04-17T06:58:31.545 INFO:teuthology.orchestra.run.smithi103.stdout:(23/174): python3-rbd-19.0.0-3133.ga8c84f59.el9 2.4 MB/s | 299 kB 00:00 2024-04-17T06:58:31.620 INFO:teuthology.orchestra.run.smithi103.stdout:(24/174): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.3 MB/s | 99 kB 00:00 2024-04-17T06:58:31.688 INFO:teuthology.orchestra.run.smithi103.stdout:(25/174): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-17T06:58:31.798 INFO:teuthology.orchestra.run.smithi086.stdout:(7/174): ceph-common-19.0.0-3133.ga8c84f59.el9. 5.2 MB/s | 21 MB 00:04 2024-04-17T06:58:31.864 INFO:teuthology.orchestra.run.smithi098.stdout:(10/174): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 4.4 MB/s | 17 MB 00:03 2024-04-17T06:58:32.030 INFO:teuthology.orchestra.run.smithi098.stdout:(11/174): cephfs-mirror-19.0.0-3133.ga8c84f59.e 1.3 MB/s | 225 kB 00:00 2024-04-17T06:58:32.131 INFO:teuthology.orchestra.run.smithi098.stdout:(12/174): cephfs-shell-19.0.0-3133.ga8c84f59.el 272 kB/s | 27 kB 00:00 2024-04-17T06:58:32.223 INFO:teuthology.orchestra.run.smithi098.stdout:(13/174): libcephfs-devel-19.0.0-3133.ga8c84f59 350 kB/s | 32 kB 00:00 2024-04-17T06:58:32.329 INFO:teuthology.orchestra.run.smithi103.stdout:(26/174): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 4.8 MB/s | 3.1 MB 00:00 2024-04-17T06:58:32.456 INFO:teuthology.orchestra.run.smithi098.stdout:(14/174): libcephfs2-19.0.0-3133.ga8c84f59.el9. 3.1 MB/s | 748 kB 00:00 2024-04-17T06:58:32.532 INFO:teuthology.orchestra.run.smithi098.stdout:(15/174): libcephsqlite-19.0.0-3133.ga8c84f59.e 2.1 MB/s | 163 kB 00:00 2024-04-17T06:58:32.554 INFO:teuthology.orchestra.run.smithi103.stdout:(27/174): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 2.6 MB/s | 17 MB 00:06 2024-04-17T06:58:32.587 INFO:teuthology.orchestra.run.smithi103.stdout:(28/174): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 661 kB/s | 170 kB 00:00 2024-04-17T06:58:32.607 INFO:teuthology.orchestra.run.smithi098.stdout:(16/174): librados-devel-19.0.0-3133.ga8c84f59. 1.7 MB/s | 126 kB 00:00 2024-04-17T06:58:32.632 INFO:teuthology.orchestra.run.smithi086.stdout:(8/174): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 2.2 MB/s | 4.7 MB 00:02 2024-04-17T06:58:32.638 INFO:teuthology.orchestra.run.smithi103.stdout:(29/174): ceph-grafana-dashboards-19.0.0-3133.g 418 kB/s | 34 kB 00:00 2024-04-17T06:58:32.663 INFO:teuthology.orchestra.run.smithi103.stdout:(30/174): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.9 MB/s | 142 kB 00:00 2024-04-17T06:58:32.707 INFO:teuthology.orchestra.run.smithi086.stdout:(9/174): ceph-selinux-19.0.0-3133.ga8c84f59.el9 340 kB/s | 25 kB 00:00 2024-04-17T06:58:32.757 INFO:teuthology.orchestra.run.smithi098.stdout:(17/174): libradosstriper1-19.0.0-3133.ga8c84f5 3.3 MB/s | 501 kB 00:00 2024-04-17T06:58:33.235 INFO:teuthology.orchestra.run.smithi086.stdout:(10/174): ceph-radosgw-19.0.0-3133.ga8c84f59.el 7.4 MB/s | 11 MB 00:01 2024-04-17T06:58:33.324 INFO:teuthology.orchestra.run.smithi086.stdout:(11/174): cephfs-mirror-19.0.0-3133.ga8c84f59.e 2.5 MB/s | 225 kB 00:00 2024-04-17T06:58:33.408 INFO:teuthology.orchestra.run.smithi086.stdout:(12/174): cephfs-shell-19.0.0-3133.ga8c84f59.el 327 kB/s | 27 kB 00:00 2024-04-17T06:58:33.483 INFO:teuthology.orchestra.run.smithi086.stdout:(13/174): libcephfs-devel-19.0.0-3133.ga8c84f59 428 kB/s | 32 kB 00:00 2024-04-17T06:58:33.667 INFO:teuthology.orchestra.run.smithi086.stdout:(14/174): libcephfs2-19.0.0-3133.ga8c84f59.el9. 4.0 MB/s | 748 kB 00:00 2024-04-17T06:58:33.759 INFO:teuthology.orchestra.run.smithi086.stdout:(15/174): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.8 MB/s | 163 kB 00:00 2024-04-17T06:58:33.859 INFO:teuthology.orchestra.run.smithi086.stdout:(16/174): librados-devel-19.0.0-3133.ga8c84f59. 1.2 MB/s | 126 kB 00:00 2024-04-17T06:58:34.026 INFO:teuthology.orchestra.run.smithi086.stdout:(17/174): libradosstriper1-19.0.0-3133.ga8c84f5 3.0 MB/s | 501 kB 00:00 2024-04-17T06:58:34.072 INFO:teuthology.orchestra.run.smithi103.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3133.ga8c84 2.5 MB/s | 3.5 MB 00:01 2024-04-17T06:58:34.280 INFO:teuthology.orchestra.run.smithi103.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3133.ga8 1.2 MB/s | 249 kB 00:00 2024-04-17T06:58:34.391 INFO:teuthology.orchestra.run.smithi098.stdout:(18/174): librgw2-19.0.0-3133.ga8c84f59.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-04-17T06:58:34.414 INFO:teuthology.orchestra.run.smithi103.stdout:(33/174): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-04-17T06:58:34.439 INFO:teuthology.orchestra.run.smithi103.stdout:(34/174): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 313 kB/s | 49 kB 00:00 2024-04-17T06:58:34.458 INFO:teuthology.orchestra.run.smithi098.stdout:(19/174): python3-ceph-argparse-19.0.0-3133.ga8 689 kB/s | 45 kB 00:00 2024-04-17T06:58:34.481 INFO:teuthology.orchestra.run.smithi103.stdout:(35/174): ceph-prometheus-alerts-19.0.0-3133.ga 250 kB/s | 16 kB 00:00 2024-04-17T06:58:34.517 INFO:teuthology.orchestra.run.smithi098.stdout:(20/174): python3-ceph-common-19.0.0-3133.ga8c8 2.3 MB/s | 135 kB 00:00 2024-04-17T06:58:34.556 INFO:teuthology.orchestra.run.smithi103.stdout:(36/174): ceph-volume-19.0.0-3133.ga8c84f59.el9 2.3 MB/s | 273 kB 00:00 2024-04-17T06:58:34.584 INFO:teuthology.orchestra.run.smithi098.stdout:(21/174): python3-cephfs-19.0.0-3133.ga8c84f59. 2.4 MB/s | 162 kB 00:00 2024-04-17T06:58:34.632 INFO:teuthology.orchestra.run.smithi103.stdout:(37/174): cephadm-19.0.0-3133.ga8c84f59.el9.noa 5.0 MB/s | 763 kB 00:00 2024-04-17T06:58:34.657 INFO:teuthology.orchestra.run.smithi103.stdout:(38/174): cephfs-top-19.0.0-3133.ga8c84f59.el9. 183 kB/s | 18 kB 00:00 2024-04-17T06:58:34.667 INFO:teuthology.orchestra.run.smithi098.stdout:(22/174): python3-rados-19.0.0-3133.ga8c84f59.e 3.8 MB/s | 321 kB 00:00 2024-04-17T06:58:34.668 INFO:teuthology.orchestra.run.smithi086.stdout:(18/174): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 5.4 MB/s | 17 MB 00:03 2024-04-17T06:58:34.759 INFO:teuthology.orchestra.run.smithi098.stdout:(23/174): python3-rbd-19.0.0-3133.ga8c84f59.el9 3.2 MB/s | 299 kB 00:00 2024-04-17T06:58:34.818 INFO:teuthology.orchestra.run.smithi098.stdout:(24/174): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.7 MB/s | 99 kB 00:00 2024-04-17T06:58:34.843 INFO:teuthology.orchestra.run.smithi086.stdout:(19/174): librgw2-19.0.0-3133.ga8c84f59.el9.x86 6.6 MB/s | 5.4 MB 00:00 2024-04-17T06:58:34.868 INFO:teuthology.orchestra.run.smithi086.stdout:(20/174): python3-ceph-argparse-19.0.0-3133.ga8 227 kB/s | 45 kB 00:00 2024-04-17T06:58:34.868 INFO:teuthology.orchestra.run.smithi098.stdout:(25/174): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-17T06:58:34.883 INFO:teuthology.orchestra.run.smithi103.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 662 kB 00:00 2024-04-17T06:58:34.918 INFO:teuthology.orchestra.run.smithi086.stdout:(21/174): python3-ceph-common-19.0.0-3133.ga8c8 1.8 MB/s | 135 kB 00:00 2024-04-17T06:58:34.933 INFO:teuthology.orchestra.run.smithi103.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 920 kB 00:00 2024-04-17T06:58:34.958 INFO:teuthology.orchestra.run.smithi103.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.5 MB/s | 106 kB 00:00 2024-04-17T06:58:34.977 INFO:teuthology.orchestra.run.smithi086.stdout:(22/174): python3-cephfs-19.0.0-3133.ga8c84f59. 1.5 MB/s | 162 kB 00:00 2024-04-17T06:58:34.992 INFO:teuthology.orchestra.run.smithi103.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-04-17T06:58:35.017 INFO:teuthology.orchestra.run.smithi103.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 574 kB/s | 32 kB 00:00 2024-04-17T06:58:35.027 INFO:teuthology.orchestra.run.smithi086.stdout:(23/174): python3-rados-19.0.0-3133.ga8c84f59.e 2.9 MB/s | 321 kB 00:00 2024-04-17T06:58:35.086 INFO:teuthology.orchestra.run.smithi086.stdout:(24/174): python3-rbd-19.0.0-3133.ga8c84f59.el9 2.7 MB/s | 299 kB 00:00 2024-04-17T06:58:35.117 INFO:teuthology.orchestra.run.smithi103.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-17T06:58:35.143 INFO:teuthology.orchestra.run.smithi103.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 17 kB 00:00 2024-04-17T06:58:35.161 INFO:teuthology.orchestra.run.smithi086.stdout:(25/174): python3-rgw-19.0.0-3133.ga8c84f59.el9 749 kB/s | 99 kB 00:00 2024-04-17T06:58:35.185 INFO:teuthology.orchestra.run.smithi103.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 249 kB/s | 10 kB 00:00 2024-04-17T06:58:35.210 INFO:teuthology.orchestra.run.smithi103.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 159 kB 00:00 2024-04-17T06:58:35.212 INFO:teuthology.orchestra.run.smithi086.stdout:(26/174): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 684 kB/s | 85 kB 00:00 2024-04-17T06:58:35.252 INFO:teuthology.orchestra.run.smithi103.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-04-17T06:58:35.285 INFO:teuthology.orchestra.run.smithi103.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-04-17T06:58:35.293 INFO:teuthology.orchestra.run.smithi098.stdout:(26/174): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 7.3 MB/s | 3.1 MB 00:00 2024-04-17T06:58:35.312 INFO:teuthology.orchestra.run.smithi086.stdout:(27/174): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 1.7 MB/s | 170 kB 00:00 2024-04-17T06:58:35.335 INFO:teuthology.orchestra.run.smithi103.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-04-17T06:58:35.352 INFO:teuthology.orchestra.run.smithi098.stdout:(27/174): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 2.9 MB/s | 170 kB 00:00 2024-04-17T06:58:35.361 INFO:teuthology.orchestra.run.smithi103.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 46 kB 00:00 2024-04-17T06:58:35.387 INFO:teuthology.orchestra.run.smithi086.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3133.g 462 kB/s | 34 kB 00:00 2024-04-17T06:58:35.402 INFO:teuthology.orchestra.run.smithi103.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 282 kB 00:00 2024-04-17T06:58:35.419 INFO:teuthology.orchestra.run.smithi098.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3133.g 519 kB/s | 34 kB 00:00 2024-04-17T06:58:35.428 INFO:teuthology.orchestra.run.smithi103.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2024-04-17T06:58:35.470 INFO:teuthology.orchestra.run.smithi103.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.2 MB/s | 217 kB 00:00 2024-04-17T06:58:35.486 INFO:teuthology.orchestra.run.smithi098.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 2.1 MB/s | 142 kB 00:00 2024-04-17T06:58:35.495 INFO:teuthology.orchestra.run.smithi103.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 522 kB/s | 34 kB 00:00 2024-04-17T06:58:35.538 INFO:teuthology.orchestra.run.smithi086.stdout:(29/174): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 8.2 MB/s | 3.1 MB 00:00 2024-04-17T06:58:35.588 INFO:teuthology.orchestra.run.smithi086.stdout:(30/174): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 713 kB/s | 142 kB 00:00 2024-04-17T06:58:35.595 INFO:teuthology.orchestra.run.smithi103.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.5 MB 00:00 2024-04-17T06:58:35.695 INFO:teuthology.orchestra.run.smithi103.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-04-17T06:58:35.737 INFO:teuthology.orchestra.run.smithi103.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 6.2 MB/s | 817 kB 00:00 2024-04-17T06:58:35.796 INFO:teuthology.orchestra.run.smithi103.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 132 kB 00:00 2024-04-17T06:58:35.963 INFO:teuthology.orchestra.run.smithi103.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.7 MB 00:00 2024-04-17T06:58:36.006 INFO:teuthology.orchestra.run.smithi103.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.2 MB/s | 1.0 MB 00:00 2024-04-17T06:58:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.9 MB/s | 78 kB 00:00 2024-04-17T06:58:36.063 INFO:teuthology.orchestra.run.smithi086.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3133.ga8c84 6.8 MB/s | 3.5 MB 00:00 2024-04-17T06:58:36.147 INFO:teuthology.orchestra.run.smithi086.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3133.ga8 3.0 MB/s | 249 kB 00:00 2024-04-17T06:58:36.197 INFO:teuthology.orchestra.run.smithi086.stdout:(33/174): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 1.0 MB/s | 49 kB 00:00 2024-04-17T06:58:36.197 INFO:teuthology.orchestra.run.smithi103.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2024-04-17T06:58:36.222 INFO:teuthology.orchestra.run.smithi103.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 154 kB/s | 24 kB 00:00 2024-04-17T06:58:36.228 INFO:teuthology.orchestra.run.smithi098.stdout:(30/174): ceph-mgr-dashboard-19.0.0-3133.ga8c84 4.8 MB/s | 3.5 MB 00:00 2024-04-17T06:58:36.239 INFO:teuthology.orchestra.run.smithi086.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3133.ga 404 kB/s | 16 kB 00:00 2024-04-17T06:58:36.256 INFO:teuthology.orchestra.run.smithi103.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 566 kB/s | 18 kB 00:00 2024-04-17T06:58:36.281 INFO:teuthology.orchestra.run.smithi103.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-04-17T06:58:36.306 INFO:teuthology.orchestra.run.smithi086.stdout:(35/174): ceph-volume-19.0.0-3133.ga8c84f59.el9 4.1 MB/s | 273 kB 00:00 2024-04-17T06:58:36.306 INFO:teuthology.orchestra.run.smithi103.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 656 kB/s | 32 kB 00:00 2024-04-17T06:58:36.456 INFO:teuthology.orchestra.run.smithi086.stdout:(36/174): cephadm-19.0.0-3133.ga8c84f59.el9.noa 5.0 MB/s | 763 kB 00:00 2024-04-17T06:58:36.457 INFO:teuthology.orchestra.run.smithi103.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-04-17T06:58:36.483 INFO:teuthology.orchestra.run.smithi103.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-04-17T06:58:36.532 INFO:teuthology.orchestra.run.smithi103.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.5 MB/s | 174 kB 00:00 2024-04-17T06:58:36.557 INFO:teuthology.orchestra.run.smithi103.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 929 kB/s | 69 kB 00:00 2024-04-17T06:58:36.600 INFO:teuthology.orchestra.run.smithi103.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2024-04-17T06:58:36.606 INFO:teuthology.orchestra.run.smithi086.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2024-04-17T06:58:36.625 INFO:teuthology.orchestra.run.smithi103.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-04-17T06:58:36.632 INFO:teuthology.orchestra.run.smithi086.stdout:(38/174): cephfs-top-19.0.0-3133.ga8c84f59.el9. 105 kB/s | 18 kB 00:00 2024-04-17T06:58:36.666 INFO:teuthology.orchestra.run.smithi103.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 132 kB 00:00 2024-04-17T06:58:36.692 INFO:teuthology.orchestra.run.smithi103.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-04-17T06:58:36.725 INFO:teuthology.orchestra.run.smithi103.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-04-17T06:58:36.751 INFO:teuthology.orchestra.run.smithi103.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 749 kB/s | 43 kB 00:00 2024-04-17T06:58:36.826 INFO:teuthology.orchestra.run.smithi103.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 523 kB/s | 39 kB 00:00 2024-04-17T06:58:36.866 INFO:teuthology.orchestra.run.smithi086.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 662 kB 00:00 2024-04-17T06:58:36.916 INFO:teuthology.orchestra.run.smithi086.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.2 MB/s | 106 kB 00:00 2024-04-17T06:58:36.966 INFO:teuthology.orchestra.run.smithi086.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 920 kB 00:00 2024-04-17T06:58:37.000 INFO:teuthology.orchestra.run.smithi086.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 307 kB 00:00 2024-04-17T06:58:37.054 INFO:teuthology.orchestra.run.smithi086.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 376 kB/s | 32 kB 00:00 2024-04-17T06:58:37.159 INFO:teuthology.orchestra.run.smithi086.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-17T06:58:37.184 INFO:teuthology.orchestra.run.smithi086.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 17 kB 00:00 2024-04-17T06:58:37.217 INFO:teuthology.orchestra.run.smithi086.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 314 kB/s | 10 kB 00:00 2024-04-17T06:58:37.218 INFO:teuthology.orchestra.run.smithi103.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-04-17T06:58:37.237 INFO:teuthology.orchestra.run.smithi098.stdout:(31/174): ceph-radosgw-19.0.0-3133.ga8c84f59.el 1.2 MB/s | 11 MB 00:09 2024-04-17T06:58:37.243 INFO:teuthology.orchestra.run.smithi103.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 102 kB/s | 41 kB 00:00 2024-04-17T06:58:37.251 INFO:teuthology.orchestra.run.smithi086.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 159 kB 00:00 2024-04-17T06:58:37.276 INFO:teuthology.orchestra.run.smithi086.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 785 kB/s | 44 kB 00:00 2024-04-17T06:58:37.301 INFO:teuthology.orchestra.run.smithi103.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.8 MB/s | 220 kB 00:00 2024-04-17T06:58:37.318 INFO:teuthology.orchestra.run.smithi086.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.9 MB/s | 661 kB 00:00 2024-04-17T06:58:37.327 INFO:teuthology.orchestra.run.smithi103.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 424 kB/s | 35 kB 00:00 2024-04-17T06:58:37.360 INFO:teuthology.orchestra.run.smithi086.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.3 MB/s | 578 kB 00:00 2024-04-17T06:58:37.368 INFO:teuthology.orchestra.run.smithi103.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-17T06:58:37.379 INFO:teuthology.orchestra.run.smithi098.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 6.4 MB/s | 7.4 MB 00:01 2024-04-17T06:58:37.385 INFO:teuthology.orchestra.run.smithi086.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 756 kB/s | 46 kB 00:00 2024-04-17T06:58:37.411 INFO:teuthology.orchestra.run.smithi086.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2024-04-17T06:58:37.419 INFO:teuthology.orchestra.run.smithi103.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 300 kB 00:00 2024-04-17T06:58:37.429 INFO:teuthology.orchestra.run.smithi098.stdout:(33/174): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 1.0 MB/s | 49 kB 00:00 2024-04-17T06:58:37.444 INFO:teuthology.orchestra.run.smithi086.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.3 MB/s | 187 kB 00:00 2024-04-17T06:58:37.454 INFO:teuthology.orchestra.run.smithi098.stdout:(34/174): ceph-mgr-modules-core-19.0.0-3133.ga8 1.1 MB/s | 249 kB 00:00 2024-04-17T06:58:37.461 INFO:teuthology.orchestra.run.smithi103.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 56 kB 00:00 2024-04-17T06:58:37.470 INFO:teuthology.orchestra.run.smithi086.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.8 MB/s | 217 kB 00:00 2024-04-17T06:58:37.486 INFO:teuthology.orchestra.run.smithi103.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 956 kB/s | 63 kB 00:00 2024-04-17T06:58:37.495 INFO:teuthology.orchestra.run.smithi086.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 720 kB/s | 34 kB 00:00 2024-04-17T06:58:37.497 INFO:teuthology.orchestra.run.smithi098.stdout:(35/174): ceph-prometheus-alerts-19.0.0-3133.ga 248 kB/s | 16 kB 00:00 2024-04-17T06:58:37.530 INFO:teuthology.orchestra.run.smithi098.stdout:(36/174): ceph-volume-19.0.0-3133.ga8c84f59.el9 3.6 MB/s | 273 kB 00:00 2024-04-17T06:58:37.536 INFO:teuthology.orchestra.run.smithi103.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 900 kB/s | 44 kB 00:00 2024-04-17T06:58:37.578 INFO:teuthology.orchestra.run.smithi103.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 868 kB/s | 79 kB 00:00 2024-04-17T06:58:37.580 INFO:teuthology.orchestra.run.smithi098.stdout:(37/174): cephfs-top-19.0.0-3133.ga8c84f59.el9. 370 kB/s | 18 kB 00:00 2024-04-17T06:58:37.645 INFO:teuthology.orchestra.run.smithi103.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2024-04-17T06:58:37.646 INFO:teuthology.orchestra.run.smithi086.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-17T06:58:37.696 INFO:teuthology.orchestra.run.smithi086.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 8.9 MB/s | 1.5 MB 00:00 2024-04-17T06:58:37.712 INFO:teuthology.orchestra.run.smithi103.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.1 MB 00:00 2024-04-17T06:58:37.737 INFO:teuthology.orchestra.run.smithi086.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 8.8 MB/s | 817 kB 00:00 2024-04-17T06:58:37.763 INFO:teuthology.orchestra.run.smithi086.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 132 kB 00:00 2024-04-17T06:58:37.770 INFO:teuthology.orchestra.run.smithi103.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-04-17T06:58:37.798 INFO:teuthology.orchestra.run.smithi098.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 662 kB 00:00 2024-04-17T06:58:37.837 INFO:teuthology.orchestra.run.smithi103.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 124 kB 00:00 2024-04-17T06:58:37.864 INFO:teuthology.orchestra.run.smithi098.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2024-04-17T06:58:37.896 INFO:teuthology.orchestra.run.smithi103.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.5 MB/s | 187 kB 00:00 2024-04-17T06:58:37.898 INFO:teuthology.orchestra.run.smithi098.stdout:(40/174): cephadm-19.0.0-3133.ga8c84f59.el9.noa 1.9 MB/s | 763 kB 00:00 2024-04-17T06:58:37.924 INFO:teuthology.orchestra.run.smithi098.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-04-17T06:58:37.938 INFO:teuthology.orchestra.run.smithi086.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.7 MB 00:00 2024-04-17T06:58:37.957 INFO:teuthology.orchestra.run.smithi098.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-04-17T06:58:37.971 INFO:teuthology.orchestra.run.smithi086.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.6 MB/s | 1.0 MB 00:00 2024-04-17T06:58:37.983 INFO:teuthology.orchestra.run.smithi098.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 573 kB/s | 32 kB 00:00 2024-04-17T06:58:38.047 INFO:teuthology.orchestra.run.smithi086.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 78 kB 00:00 2024-04-17T06:58:38.181 INFO:teuthology.orchestra.run.smithi086.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-04-17T06:58:38.183 INFO:teuthology.orchestra.run.smithi098.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-17T06:58:38.200 INFO:teuthology.orchestra.run.smithi098.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 85 kB/s | 17 kB 00:00 2024-04-17T06:58:38.205 INFO:teuthology.orchestra.run.smithi086.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 174 kB/s | 24 kB 00:00 2024-04-17T06:58:38.217 INFO:teuthology.orchestra.run.smithi098.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 310 kB/s | 10 kB 00:00 2024-04-17T06:58:38.239 INFO:teuthology.orchestra.run.smithi086.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 4.4 MB/s | 144 kB 00:00 2024-04-17T06:58:38.242 INFO:teuthology.orchestra.run.smithi098.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2024-04-17T06:58:38.259 INFO:teuthology.orchestra.run.smithi098.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-04-17T06:58:38.264 INFO:teuthology.orchestra.run.smithi086.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 314 kB/s | 18 kB 00:00 2024-04-17T06:58:38.290 INFO:teuthology.orchestra.run.smithi086.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 641 kB/s | 32 kB 00:00 2024-04-17T06:58:38.319 INFO:teuthology.orchestra.run.smithi098.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-04-17T06:58:38.360 INFO:teuthology.orchestra.run.smithi098.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.0 MB/s | 578 kB 00:00 2024-04-17T06:58:38.377 INFO:teuthology.orchestra.run.smithi098.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 877 kB/s | 46 kB 00:00 2024-04-17T06:58:38.411 INFO:teuthology.orchestra.run.smithi098.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2024-04-17T06:58:38.440 INFO:teuthology.orchestra.run.smithi086.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-04-17T06:58:38.444 INFO:teuthology.orchestra.run.smithi098.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-04-17T06:58:38.465 INFO:teuthology.orchestra.run.smithi086.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-04-17T06:58:38.477 INFO:teuthology.orchestra.run.smithi098.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-04-17T06:58:38.489 INFO:teuthology.orchestra.run.smithi103.stdout:(94/174): ceph-test-19.0.0-3133.ga8c84f59.el9.x 4.3 MB/s | 49 MB 00:11 2024-04-17T06:58:38.503 INFO:teuthology.orchestra.run.smithi098.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 612 kB/s | 34 kB 00:00 2024-04-17T06:58:38.507 INFO:teuthology.orchestra.run.smithi086.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 174 kB 00:00 2024-04-17T06:58:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-04-17T06:58:38.583 INFO:teuthology.orchestra.run.smithi086.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 41 kB 00:00 2024-04-17T06:58:38.608 INFO:teuthology.orchestra.run.smithi086.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-17T06:58:38.630 INFO:teuthology.orchestra.run.smithi103.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 237 kB/s | 187 kB 00:00 2024-04-17T06:58:38.650 INFO:teuthology.orchestra.run.smithi086.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 132 kB 00:00 2024-04-17T06:58:38.672 INFO:teuthology.orchestra.run.smithi103.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-17T06:58:38.675 INFO:teuthology.orchestra.run.smithi086.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-04-17T06:58:38.700 INFO:teuthology.orchestra.run.smithi086.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 966 kB/s | 47 kB 00:00 2024-04-17T06:58:38.705 INFO:teuthology.orchestra.run.smithi103.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-17T06:58:38.720 INFO:teuthology.orchestra.run.smithi098.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-17T06:58:38.726 INFO:teuthology.orchestra.run.smithi086.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 866 kB/s | 43 kB 00:00 2024-04-17T06:58:38.756 INFO:teuthology.orchestra.run.smithi103.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 935 kB/s | 803 kB 00:00 2024-04-17T06:58:38.771 INFO:teuthology.orchestra.run.smithi098.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 6.1 MB/s | 1.5 MB 00:00 2024-04-17T06:58:38.781 INFO:teuthology.orchestra.run.smithi103.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-04-17T06:58:38.812 INFO:teuthology.orchestra.run.smithi098.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 8.8 MB/s | 817 kB 00:00 2024-04-17T06:58:38.815 INFO:teuthology.orchestra.run.smithi103.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 236 kB/s | 25 kB 00:00 2024-04-17T06:58:38.838 INFO:teuthology.orchestra.run.smithi098.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 132 kB 00:00 2024-04-17T06:58:38.882 INFO:teuthology.orchestra.run.smithi103.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.4 MB/s | 245 kB 00:00 2024-04-17T06:58:38.909 INFO:teuthology.orchestra.run.smithi086.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 246 kB/s | 39 kB 00:00 2024-04-17T06:58:38.915 INFO:teuthology.orchestra.run.smithi103.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.4 MB/s | 248 kB 00:00 2024-04-17T06:58:38.941 INFO:teuthology.orchestra.run.smithi103.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-04-17T06:58:39.030 INFO:teuthology.orchestra.run.smithi098.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2024-04-17T06:58:39.072 INFO:teuthology.orchestra.run.smithi098.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-04-17T06:58:39.083 INFO:teuthology.orchestra.run.smithi103.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 13 MB/s | 4.1 MB 00:00 2024-04-17T06:58:39.122 INFO:teuthology.orchestra.run.smithi098.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.6 MB/s | 78 kB 00:00 2024-04-17T06:58:39.126 INFO:teuthology.orchestra.run.smithi086.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-17T06:58:39.143 INFO:teuthology.orchestra.run.smithi086.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 179 kB/s | 41 kB 00:00 2024-04-17T06:58:39.177 INFO:teuthology.orchestra.run.smithi086.stdout:(81/174): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 35 kB 00:00 2024-04-17T06:58:39.219 INFO:teuthology.orchestra.run.smithi086.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-04-17T06:58:39.236 INFO:teuthology.orchestra.run.smithi086.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 824 kB/s | 46 kB 00:00 2024-04-17T06:58:39.256 INFO:teuthology.orchestra.run.smithi098.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 5.8 MB 00:00 2024-04-17T06:58:39.269 INFO:teuthology.orchestra.run.smithi086.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 300 kB 00:00 2024-04-17T06:58:39.281 INFO:teuthology.orchestra.run.smithi098.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 171 kB/s | 24 kB 00:00 2024-04-17T06:58:39.295 INFO:teuthology.orchestra.run.smithi086.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-04-17T06:58:39.320 INFO:teuthology.orchestra.run.smithi086.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-17T06:58:39.323 INFO:teuthology.orchestra.run.smithi098.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 447 kB/s | 18 kB 00:00 2024-04-17T06:58:39.348 INFO:teuthology.orchestra.run.smithi098.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 144 kB 00:00 2024-04-17T06:58:39.353 INFO:teuthology.orchestra.run.smithi086.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 771 kB/s | 44 kB 00:00 2024-04-17T06:58:39.379 INFO:teuthology.orchestra.run.smithi086.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-04-17T06:58:39.390 INFO:teuthology.orchestra.run.smithi098.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 772 kB/s | 32 kB 00:00 2024-04-17T06:58:39.413 INFO:teuthology.orchestra.run.smithi086.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-04-17T06:58:39.471 INFO:teuthology.orchestra.run.smithi086.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-04-17T06:58:39.488 INFO:teuthology.orchestra.run.smithi086.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2024-04-17T06:58:39.491 INFO:teuthology.orchestra.run.smithi103.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-04-17T06:58:39.513 INFO:teuthology.orchestra.run.smithi086.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 3.0 MB/s | 124 kB 00:00 2024-04-17T06:58:39.523 INFO:teuthology.orchestra.run.smithi098.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-04-17T06:58:39.547 INFO:teuthology.orchestra.run.smithi086.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-04-17T06:58:39.548 INFO:teuthology.orchestra.run.smithi098.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-04-17T06:58:39.550 INFO:teuthology.orchestra.run.smithi103.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 5.3 MB/s | 309 kB 00:00 2024-04-17T06:58:39.591 INFO:teuthology.orchestra.run.smithi098.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 1.7 MB/s | 69 kB 00:00 2024-04-17T06:58:39.609 INFO:teuthology.orchestra.run.smithi103.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 337 kB/s | 19 kB 00:00 2024-04-17T06:58:39.624 INFO:teuthology.orchestra.run.smithi098.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.3 MB/s | 174 kB 00:00 2024-04-17T06:58:39.634 INFO:teuthology.orchestra.run.smithi103.stdout:(108/174): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-04-17T06:58:39.658 INFO:teuthology.orchestra.run.smithi098.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 636 kB/s | 41 kB 00:00 2024-04-17T06:58:39.689 INFO:teuthology.orchestra.run.smithi086.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-04-17T06:58:39.691 INFO:teuthology.orchestra.run.smithi098.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-04-17T06:58:39.701 INFO:teuthology.orchestra.run.smithi103.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 393 kB/s | 26 kB 00:00 2024-04-17T06:58:39.717 INFO:teuthology.orchestra.run.smithi098.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-04-17T06:58:39.731 INFO:teuthology.orchestra.run.smithi086.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-17T06:58:39.750 INFO:teuthology.orchestra.run.smithi098.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-04-17T06:58:39.765 INFO:teuthology.orchestra.run.smithi086.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-17T06:58:39.768 INFO:teuthology.orchestra.run.smithi103.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 810 kB/s | 49 kB 00:00 2024-04-17T06:58:39.776 INFO:teuthology.orchestra.run.smithi098.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 845 kB/s | 47 kB 00:00 2024-04-17T06:58:39.790 INFO:teuthology.orchestra.run.smithi086.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-17T06:58:39.801 INFO:teuthology.orchestra.run.smithi098.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 875 kB/s | 43 kB 00:00 2024-04-17T06:58:39.876 INFO:teuthology.orchestra.run.smithi103.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 669 kB/s | 67 kB 00:00 2024-04-17T06:58:40.018 INFO:teuthology.orchestra.run.smithi103.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-17T06:58:40.043 INFO:teuthology.orchestra.run.smithi103.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 938 kB/s | 151 kB 00:00 2024-04-17T06:58:40.161 INFO:teuthology.orchestra.run.smithi103.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.1 MB/s | 838 kB 00:00 2024-04-17T06:58:40.193 INFO:teuthology.orchestra.run.smithi098.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-17T06:58:40.194 INFO:teuthology.orchestra.run.smithi103.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-17T06:58:40.210 INFO:teuthology.orchestra.run.smithi098.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 98 kB/s | 39 kB 00:00 2024-04-17T06:58:40.211 INFO:teuthology.orchestra.run.smithi103.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 912 kB/s | 43 kB 00:00 2024-04-17T06:58:40.227 INFO:teuthology.orchestra.run.smithi098.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 41 kB 00:00 2024-04-17T06:58:40.249 INFO:teuthology.orchestra.run.smithi103.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 602 kB/s | 32 kB 00:00 2024-04-17T06:58:40.274 INFO:teuthology.orchestra.run.smithi103.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 240 kB/s | 15 kB 00:00 2024-04-17T06:58:40.277 INFO:teuthology.orchestra.run.smithi098.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 220 kB 00:00 2024-04-17T06:58:40.294 INFO:teuthology.orchestra.run.smithi098.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 543 kB/s | 35 kB 00:00 2024-04-17T06:58:40.308 INFO:teuthology.orchestra.run.smithi103.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-17T06:58:40.311 INFO:teuthology.orchestra.run.smithi098.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-17T06:58:40.342 INFO:teuthology.orchestra.run.smithi103.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 5.4 MB/s | 358 kB 00:00 2024-04-17T06:58:40.362 INFO:teuthology.orchestra.run.smithi098.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 300 kB 00:00 2024-04-17T06:58:40.376 INFO:teuthology.orchestra.run.smithi103.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.7 MB/s | 303 kB 00:00 2024-04-17T06:58:40.378 INFO:teuthology.orchestra.run.smithi098.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 888 kB/s | 56 kB 00:00 2024-04-17T06:58:40.398 INFO:teuthology.orchestra.run.smithi086.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 34 kB/s | 21 kB 00:00 2024-04-17T06:58:40.401 INFO:teuthology.orchestra.run.smithi103.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 641 kB/s | 35 kB 00:00 2024-04-17T06:58:40.412 INFO:teuthology.orchestra.run.smithi098.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-17T06:58:40.429 INFO:teuthology.orchestra.run.smithi098.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 895 kB/s | 44 kB 00:00 2024-04-17T06:58:40.434 INFO:teuthology.orchestra.run.smithi103.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.5 MB/s | 203 kB 00:00 2024-04-17T06:58:40.457 INFO:teuthology.orchestra.run.smithi086.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 38 kB/s | 25 kB 00:00 2024-04-17T06:58:40.471 INFO:teuthology.orchestra.run.smithi098.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-04-17T06:58:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-04-17T06:58:40.526 INFO:teuthology.orchestra.run.smithi103.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-04-17T06:58:40.597 INFO:teuthology.orchestra.run.smithi098.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 9.3 MB/s | 1.1 MB 00:00 2024-04-17T06:58:40.610 INFO:teuthology.orchestra.run.smithi103.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-04-17T06:58:40.636 INFO:teuthology.orchestra.run.smithi103.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 102 kB/s | 11 kB 00:00 2024-04-17T06:58:40.639 INFO:teuthology.orchestra.run.smithi098.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 966 kB/s | 103 kB 00:00 2024-04-17T06:58:40.661 INFO:teuthology.orchestra.run.smithi103.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 359 kB/s | 18 kB 00:00 2024-04-17T06:58:40.678 INFO:teuthology.orchestra.run.smithi103.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 566 kB/s | 23 kB 00:00 2024-04-17T06:58:40.681 INFO:teuthology.orchestra.run.smithi098.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-17T06:58:40.703 INFO:teuthology.orchestra.run.smithi103.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 476 kB/s | 19 kB 00:00 2024-04-17T06:58:40.720 INFO:teuthology.orchestra.run.smithi103.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 477 kB/s | 20 kB 00:00 2024-04-17T06:58:40.782 INFO:teuthology.orchestra.run.smithi098.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.3 MB/s | 187 kB 00:00 2024-04-17T06:58:40.787 INFO:teuthology.orchestra.run.smithi103.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-17T06:58:40.812 INFO:teuthology.orchestra.run.smithi103.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 509 kB/s | 46 kB 00:00 2024-04-17T06:58:40.854 INFO:teuthology.orchestra.run.smithi103.stdout:(133/174): protobuf-compiler-3.14.0-13.el9.x86_ 451 kB/s | 863 kB 00:01 2024-04-17T06:58:40.905 INFO:teuthology.orchestra.run.smithi103.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 866 kB/s | 79 kB 00:00 2024-04-17T06:58:40.930 INFO:teuthology.orchestra.run.smithi103.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 498 kB/s | 58 kB 00:00 2024-04-17T06:58:40.963 INFO:teuthology.orchestra.run.smithi103.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-17T06:58:40.989 INFO:teuthology.orchestra.run.smithi103.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 298 kB/s | 16 kB 00:00 2024-04-17T06:58:41.022 INFO:teuthology.orchestra.run.smithi103.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-17T06:58:41.048 INFO:teuthology.orchestra.run.smithi103.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 342 kB/s | 28 kB 00:00 2024-04-17T06:58:41.073 INFO:teuthology.orchestra.run.smithi103.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 373 kB/s | 31 kB 00:00 2024-04-17T06:58:41.106 INFO:teuthology.orchestra.run.smithi103.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-17T06:58:41.132 INFO:teuthology.orchestra.run.smithi103.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-17T06:58:41.157 INFO:teuthology.orchestra.run.smithi103.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 443 kB/s | 36 kB 00:00 2024-04-17T06:58:41.190 INFO:teuthology.orchestra.run.smithi103.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 622 kB/s | 51 kB 00:00 2024-04-17T06:58:41.224 INFO:teuthology.orchestra.run.smithi103.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-17T06:58:41.249 INFO:teuthology.orchestra.run.smithi103.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 993 kB/s | 90 kB 00:00 2024-04-17T06:58:41.255 INFO:teuthology.orchestra.run.smithi098.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 327 kB/s | 187 kB 00:00 2024-04-17T06:58:41.283 INFO:teuthology.orchestra.run.smithi103.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.7 MB/s | 427 kB 00:00 2024-04-17T06:58:41.308 INFO:teuthology.orchestra.run.smithi103.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 373 kB/s | 20 kB 00:00 2024-04-17T06:58:41.342 INFO:teuthology.orchestra.run.smithi103.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-17T06:58:41.390 INFO:teuthology.orchestra.run.smithi086.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 263 kB/s | 245 kB 00:00 2024-04-17T06:58:41.400 INFO:teuthology.orchestra.run.smithi103.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-17T06:58:41.465 INFO:teuthology.orchestra.run.smithi098.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 931 kB/s | 195 kB 00:00 2024-04-17T06:58:41.509 INFO:teuthology.orchestra.run.smithi103.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 21 MB/s | 2.3 MB 00:00 2024-04-17T06:58:41.607 INFO:teuthology.orchestra.run.smithi098.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 974 kB/s | 803 kB 00:00 2024-04-17T06:58:41.649 INFO:teuthology.orchestra.run.smithi098.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 683 kB/s | 125 kB 00:00 2024-04-17T06:58:41.659 INFO:teuthology.orchestra.run.smithi103.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-04-17T06:58:41.701 INFO:teuthology.orchestra.run.smithi103.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.0 MB/s | 41 kB 00:00 2024-04-17T06:58:41.717 INFO:teuthology.orchestra.run.smithi098.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 200 kB/s | 21 kB 00:00 2024-04-17T06:58:41.741 INFO:teuthology.orchestra.run.smithi098.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 280 kB/s | 25 kB 00:00 2024-04-17T06:58:41.743 INFO:teuthology.orchestra.run.smithi103.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.9 MB/s | 248 kB 00:00 2024-04-17T06:58:41.785 INFO:teuthology.orchestra.run.smithi103.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-04-17T06:58:41.852 INFO:teuthology.orchestra.run.smithi103.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 71 kB 00:00 2024-04-17T06:58:41.924 INFO:teuthology.orchestra.run.smithi103.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 513 kB/s | 36 kB 00:00 2024-04-17T06:58:42.003 INFO:teuthology.orchestra.run.smithi103.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.4 MB/s | 192 kB 00:00 2024-04-17T06:58:42.061 INFO:teuthology.orchestra.run.smithi103.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-04-17T06:58:42.112 INFO:teuthology.orchestra.run.smithi103.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 465 kB/s | 23 kB 00:00 2024-04-17T06:58:42.153 INFO:teuthology.orchestra.run.smithi103.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 108 kB 00:00 2024-04-17T06:58:42.204 INFO:teuthology.orchestra.run.smithi103.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.1 MB/s | 155 kB 00:00 2024-04-17T06:58:42.254 INFO:teuthology.orchestra.run.smithi103.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.7 MB/s | 136 kB 00:00 2024-04-17T06:58:42.321 INFO:teuthology.orchestra.run.smithi103.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 968 kB/s | 64 kB 00:00 2024-04-17T06:58:42.342 INFO:teuthology.orchestra.run.smithi098.stdout:(100/174): ceph-test-19.0.0-3133.ga8c84f59.el9. 3.7 MB/s | 49 MB 00:13 2024-04-17T06:58:42.371 INFO:teuthology.orchestra.run.smithi103.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 585 kB/s | 29 kB 00:00 2024-04-17T06:58:42.391 INFO:teuthology.orchestra.run.smithi086.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 248 kB/s | 248 kB 00:00 2024-04-17T06:58:42.402 INFO:teuthology.orchestra.run.smithi098.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 373 kB/s | 245 kB 00:00 2024-04-17T06:58:42.472 INFO:teuthology.orchestra.run.smithi103.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-04-17T06:58:42.501 INFO:teuthology.orchestra.run.smithi098.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.4 MB/s | 248 kB 00:00 2024-04-17T06:58:42.526 INFO:teuthology.orchestra.run.smithi098.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 752 kB/s | 94 kB 00:00 2024-04-17T06:58:42.589 INFO:teuthology.orchestra.run.smithi103.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 19 MB/s | 2.2 MB 00:00 2024-04-17T06:58:42.668 INFO:teuthology.orchestra.run.smithi098.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.3 MB/s | 4.1 MB 00:00 2024-04-17T06:58:42.705 INFO:teuthology.orchestra.run.smithi103.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 20 MB/s | 2.3 MB 00:00 2024-04-17T06:58:42.781 INFO:teuthology.orchestra.run.smithi103.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.1 MB/s | 465 kB 00:00 2024-04-17T06:58:42.831 INFO:teuthology.orchestra.run.smithi103.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 634 kB/s | 31 kB 00:00 2024-04-17T06:58:42.977 INFO:teuthology.orchestra.run.smithi103.stdout:(171/174): librados2-19.0.0-3133.ga8c84f59.el9. 2.0 MB/s | 3.4 MB 00:01 2024-04-17T06:58:43.019 INFO:teuthology.orchestra.run.smithi103.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.7 MB/s | 1.1 MB 00:00 2024-04-17T06:58:43.061 INFO:teuthology.orchestra.run.smithi103.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 898 kB/s | 74 kB 00:00 2024-04-17T06:58:43.068 INFO:teuthology.orchestra.run.smithi098.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-04-17T06:58:43.127 INFO:teuthology.orchestra.run.smithi098.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 5.5 MB/s | 309 kB 00:00 2024-04-17T06:58:43.177 INFO:teuthology.orchestra.run.smithi098.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-04-17T06:58:43.244 INFO:teuthology.orchestra.run.smithi098.stdout:(108/174): lua-devel-5.4.4-4.el9.x86_64.rpm 30 kB/s | 22 kB 00:00 2024-04-17T06:58:43.249 INFO:teuthology.orchestra.run.smithi086.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 109 kB/s | 94 kB 00:00 2024-04-17T06:58:43.333 INFO:teuthology.orchestra.run.smithi086.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 273 kB/s | 22 kB 00:00 2024-04-17T06:58:43.403 INFO:teuthology.orchestra.run.smithi103.stdout:(174/174): librbd1-19.0.0-3133.ga8c84f59.el9.x8 1.5 MB/s | 3.1 MB 00:02 2024-04-17T06:58:43.414 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-04-17T06:58:43.414 INFO:teuthology.orchestra.run.smithi103.stdout:Total 12 MB/s | 229 MB 00:19 2024-04-17T06:58:43.483 INFO:teuthology.orchestra.run.smithi086.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.7 MB/s | 863 kB 00:00 2024-04-17T06:58:43.536 INFO:teuthology.orchestra.run.smithi098.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-17T06:58:43.561 INFO:teuthology.orchestra.run.smithi098.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 83 kB/s | 26 kB 00:00 2024-04-17T06:58:43.612 INFO:teuthology.orchestra.run.smithi098.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 997 kB/s | 49 kB 00:00 2024-04-17T06:58:43.645 INFO:teuthology.orchestra.run.smithi098.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 819 kB/s | 67 kB 00:00 2024-04-17T06:58:43.704 INFO:teuthology.orchestra.run.smithi098.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-04-17T06:58:43.896 INFO:teuthology.orchestra.run.smithi098.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-04-17T06:58:43.938 INFO:teuthology.orchestra.run.smithi098.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-04-17T06:58:43.942 INFO:teuthology.orchestra.run.smithi086.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-17T06:58:43.955 INFO:teuthology.orchestra.run.smithi098.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 831 kB/s | 43 kB 00:00 2024-04-17T06:58:43.985 INFO:teuthology.orchestra.run.smithi098.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 711 kB/s | 32 kB 00:00 2024-04-17T06:58:44.010 INFO:teuthology.orchestra.run.smithi098.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 277 kB/s | 15 kB 00:00 2024-04-17T06:58:44.035 INFO:teuthology.orchestra.run.smithi098.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-17T06:58:44.041 INFO:teuthology.orchestra.run.smithi086.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.1 MB/s | 309 kB 00:00 2024-04-17T06:58:44.069 INFO:teuthology.orchestra.run.smithi098.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 6.2 MB/s | 358 kB 00:00 2024-04-17T06:58:44.100 INFO:teuthology.orchestra.run.smithi086.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 336 kB/s | 19 kB 00:00 2024-04-17T06:58:44.103 INFO:teuthology.orchestra.run.smithi098.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.7 MB/s | 303 kB 00:00 2024-04-17T06:58:44.128 INFO:teuthology.orchestra.run.smithi098.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 637 kB/s | 35 kB 00:00 2024-04-17T06:58:44.162 INFO:teuthology.orchestra.run.smithi098.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-17T06:58:44.363 INFO:teuthology.orchestra.run.smithi098.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.8 MB/s | 2.0 MB 00:00 2024-04-17T06:58:44.388 INFO:teuthology.orchestra.run.smithi098.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 660 kB/s | 144 kB 00:00 2024-04-17T06:58:44.413 INFO:teuthology.orchestra.run.smithi098.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 222 kB/s | 11 kB 00:00 2024-04-17T06:58:44.438 INFO:teuthology.orchestra.run.smithi098.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 357 kB/s | 18 kB 00:00 2024-04-17T06:58:44.455 INFO:teuthology.orchestra.run.smithi098.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 566 kB/s | 23 kB 00:00 2024-04-17T06:58:44.498 INFO:teuthology.orchestra.run.smithi098.stdout:(129/174): protobuf-compiler-3.14.0-13.el9.x86_ 438 kB/s | 863 kB 00:01 2024-04-17T06:58:44.514 INFO:teuthology.orchestra.run.smithi098.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-17T06:58:44.531 INFO:teuthology.orchestra.run.smithi098.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2024-04-17T06:58:44.583 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-04-17T06:58:44.606 INFO:teuthology.orchestra.run.smithi098.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 9.5 MB/s | 1.0 MB 00:00 2024-04-17T06:58:44.632 INFO:teuthology.orchestra.run.smithi098.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-04-17T06:58:44.649 INFO:teuthology.orchestra.run.smithi098.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 724 kB/s | 79 kB 00:00 2024-04-17T06:58:44.697 INFO:teuthology.orchestra.run.smithi098.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 651 kB/s | 58 kB 00:00 2024-04-17T06:58:44.733 INFO:teuthology.orchestra.run.smithi098.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-17T06:58:44.758 INFO:teuthology.orchestra.run.smithi098.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 153 kB/s | 16 kB 00:00 2024-04-17T06:58:44.767 INFO:teuthology.orchestra.run.smithi086.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-04-17T06:58:44.775 INFO:teuthology.orchestra.run.smithi098.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-17T06:58:44.809 INFO:teuthology.orchestra.run.smithi098.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 584 kB/s | 28 kB 00:00 2024-04-17T06:58:44.826 INFO:teuthology.orchestra.run.smithi086.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 452 kB/s | 26 kB 00:00 2024-04-17T06:58:44.834 INFO:teuthology.orchestra.run.smithi098.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 419 kB/s | 31 kB 00:00 2024-04-17T06:58:44.867 INFO:teuthology.orchestra.run.smithi098.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-17T06:58:44.885 INFO:teuthology.orchestra.run.smithi086.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 840 kB/s | 49 kB 00:00 2024-04-17T06:58:44.892 INFO:teuthology.orchestra.run.smithi098.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-17T06:58:44.918 INFO:teuthology.orchestra.run.smithi098.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 443 kB/s | 36 kB 00:00 2024-04-17T06:58:44.943 INFO:teuthology.orchestra.run.smithi098.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 692 kB/s | 51 kB 00:00 2024-04-17T06:58:44.943 INFO:teuthology.orchestra.run.smithi086.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-17T06:58:44.970 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-04-17T06:58:44.970 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-04-17T06:58:44.976 INFO:teuthology.orchestra.run.smithi098.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-17T06:58:45.002 INFO:teuthology.orchestra.run.smithi098.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-17T06:58:45.002 INFO:teuthology.orchestra.run.smithi086.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-04-17T06:58:45.036 INFO:teuthology.orchestra.run.smithi098.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.7 MB/s | 427 kB 00:00 2024-04-17T06:58:45.061 INFO:teuthology.orchestra.run.smithi098.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 372 kB/s | 20 kB 00:00 2024-04-17T06:58:45.077 INFO:teuthology.orchestra.run.smithi098.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.7 MB/s | 192 kB 00:00 2024-04-17T06:58:45.136 INFO:teuthology.orchestra.run.smithi086.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.2 MB/s | 838 kB 00:00 2024-04-17T06:58:45.153 INFO:teuthology.orchestra.run.smithi098.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-17T06:58:45.236 INFO:teuthology.orchestra.run.smithi086.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-17T06:58:45.295 INFO:teuthology.orchestra.run.smithi086.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 756 kB/s | 43 kB 00:00 2024-04-17T06:58:45.370 INFO:teuthology.orchestra.run.smithi086.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 433 kB/s | 32 kB 00:00 2024-04-17T06:58:45.429 INFO:teuthology.orchestra.run.smithi086.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 260 kB/s | 15 kB 00:00 2024-04-17T06:58:45.436 INFO:teuthology.orchestra.run.smithi098.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 8.1 MB/s | 2.3 MB 00:00 2024-04-17T06:58:45.495 INFO:teuthology.orchestra.run.smithi086.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-17T06:58:45.562 INFO:teuthology.orchestra.run.smithi098.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-04-17T06:58:45.563 INFO:teuthology.orchestra.run.smithi086.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-17T06:58:45.595 INFO:teuthology.orchestra.run.smithi098.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-17T06:58:45.629 INFO:teuthology.orchestra.run.smithi098.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 7.4 MB/s | 248 kB 00:00 2024-04-17T06:58:45.630 INFO:teuthology.orchestra.run.smithi086.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-04-17T06:58:45.671 INFO:teuthology.orchestra.run.smithi098.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-04-17T06:58:45.680 INFO:teuthology.orchestra.run.smithi086.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 718 kB/s | 35 kB 00:00 2024-04-17T06:58:45.747 INFO:teuthology.orchestra.run.smithi086.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-04-17T06:58:45.938 INFO:teuthology.orchestra.run.smithi098.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 268 kB/s | 71 kB 00:00 2024-04-17T06:58:46.009 INFO:teuthology.orchestra.run.smithi086.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-04-17T06:58:46.047 INFO:teuthology.orchestra.run.smithi098.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 339 kB/s | 36 kB 00:00 2024-04-17T06:58:46.076 INFO:teuthology.orchestra.run.smithi086.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-17T06:58:46.134 INFO:teuthology.orchestra.run.smithi086.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 186 kB/s | 11 kB 00:00 2024-04-17T06:58:46.193 INFO:teuthology.orchestra.run.smithi086.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 306 kB/s | 18 kB 00:00 2024-04-17T06:58:46.252 INFO:teuthology.orchestra.run.smithi086.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 401 kB/s | 23 kB 00:00 2024-04-17T06:58:46.311 INFO:teuthology.orchestra.run.smithi086.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 338 kB/s | 19 kB 00:00 2024-04-17T06:58:46.314 INFO:teuthology.orchestra.run.smithi098.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 723 kB/s | 192 kB 00:00 2024-04-17T06:58:46.428 INFO:teuthology.orchestra.run.smithi086.stdout:(129/174): openssl-devel-3.0.7-27.el9.x86_64.rp 702 kB/s | 4.1 MB 00:06 2024-04-17T06:58:46.453 INFO:teuthology.orchestra.run.smithi086.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 139 kB/s | 20 kB 00:00 2024-04-17T06:58:46.505 INFO:teuthology.orchestra.run.smithi098.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.3 MB/s | 256 kB 00:00 2024-04-17T06:58:46.553 INFO:teuthology.orchestra.run.smithi086.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 466 kB/s | 46 kB 00:00 2024-04-17T06:58:46.614 INFO:teuthology.orchestra.run.smithi098.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 214 kB/s | 23 kB 00:00 2024-04-17T06:58:46.637 INFO:teuthology.orchestra.run.smithi086.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 953 kB/s | 79 kB 00:00 2024-04-17T06:58:46.731 INFO:teuthology.orchestra.run.smithi098.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 934 kB/s | 108 kB 00:00 2024-04-17T06:58:46.804 INFO:teuthology.orchestra.run.smithi086.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 349 kB/s | 58 kB 00:00 2024-04-17T06:58:46.806 INFO:teuthology.orchestra.run.smithi098.stdout:(162/174): librbd1-19.0.0-3133.ga8c84f59.el9.x8 1.8 MB/s | 3.1 MB 00:01 2024-04-17T06:58:46.840 INFO:teuthology.orchestra.run.smithi098.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-04-17T06:58:46.871 INFO:teuthology.orchestra.run.smithi086.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-17T06:58:46.921 INFO:teuthology.orchestra.run.smithi086.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 334 kB/s | 16 kB 00:00 2024-04-17T06:58:46.957 INFO:teuthology.orchestra.run.smithi098.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 550 kB/s | 64 kB 00:00 2024-04-17T06:58:46.980 INFO:teuthology.orchestra.run.smithi086.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-17T06:58:47.030 INFO:teuthology.orchestra.run.smithi086.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 582 kB/s | 28 kB 00:00 2024-04-17T06:58:47.080 INFO:teuthology.orchestra.run.smithi086.stdout:(138/174): python3-kubernetes-26.1.0-2.el9.noar 1.6 MB/s | 1.0 MB 00:00 2024-04-17T06:58:47.106 INFO:teuthology.orchestra.run.smithi086.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 413 kB/s | 31 kB 00:00 2024-04-17T06:58:47.107 INFO:teuthology.orchestra.run.smithi098.stdout:(165/174): librados2-19.0.0-3133.ga8c84f59.el9. 1.7 MB/s | 3.4 MB 00:02 2024-04-17T06:58:47.133 INFO:teuthology.orchestra.run.smithi098.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 164 kB/s | 29 kB 00:00 2024-04-17T06:58:47.164 INFO:teuthology.orchestra.run.smithi086.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-17T06:58:47.190 INFO:teuthology.orchestra.run.smithi086.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-17T06:58:47.223 INFO:teuthology.orchestra.run.smithi086.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 619 kB/s | 36 kB 00:00 2024-04-17T06:58:47.248 INFO:teuthology.orchestra.run.smithi086.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 881 kB/s | 51 kB 00:00 2024-04-17T06:58:47.311 INFO:teuthology.orchestra.run.smithi086.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-17T06:58:47.325 INFO:teuthology.orchestra.run.smithi098.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 262 kB/s | 136 kB 00:00 2024-04-17T06:58:47.332 INFO:teuthology.orchestra.run.smithi086.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-17T06:58:47.374 INFO:teuthology.orchestra.run.smithi086.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-04-17T06:58:47.385 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-04-17T06:58:47.385 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-04-17T06:58:47.400 INFO:teuthology.orchestra.run.smithi086.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 302 kB/s | 20 kB 00:00 2024-04-17T06:58:47.442 INFO:teuthology.orchestra.run.smithi086.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-17T06:58:47.584 INFO:teuthology.orchestra.run.smithi098.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.8 MB/s | 2.2 MB 00:00 2024-04-17T06:58:47.709 INFO:teuthology.orchestra.run.smithi098.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.7 MB/s | 465 kB 00:00 2024-04-17T06:58:47.718 INFO:teuthology.orchestra.run.smithi086.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-04-17T06:58:47.817 INFO:teuthology.orchestra.run.smithi098.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 289 kB/s | 31 kB 00:00 2024-04-17T06:58:47.926 INFO:teuthology.orchestra.run.smithi098.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.8 MB/s | 2.3 MB 00:00 2024-04-17T06:58:47.990 INFO:teuthology.orchestra.run.smithi098.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.3 MB/s | 1.1 MB 00:00 2024-04-17T06:58:48.015 INFO:teuthology.orchestra.run.smithi098.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 844 kB/s | 74 kB 00:00 2024-04-17T06:58:48.057 INFO:teuthology.orchestra.run.smithi098.stdout:(174/174): openssl-3.0.7-27.el9.x86_64.rpm 1.3 MB/s | 1.2 MB 00:00 2024-04-17T06:58:48.068 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-04-17T06:58:48.068 INFO:teuthology.orchestra.run.smithi098.stdout:Total 9.5 MB/s | 229 MB 00:24 2024-04-17T06:58:48.459 INFO:teuthology.orchestra.run.smithi086.stdout:(150/174): librados2-19.0.0-3133.ga8c84f59.el9. 3.3 MB/s | 3.4 MB 00:01 2024-04-17T06:58:48.951 INFO:teuthology.orchestra.run.smithi086.stdout:(151/174): ceph-test-19.0.0-3133.ga8c84f59.el9. 3.0 MB/s | 49 MB 00:16 2024-04-17T06:58:49.026 INFO:teuthology.orchestra.run.smithi086.stdout:(152/174): NetworkManager-1.47.4-1.el9.x86_64.r 4.1 MB/s | 2.3 MB 00:00 2024-04-17T06:58:49.101 INFO:teuthology.orchestra.run.smithi086.stdout:(153/174): NetworkManager-libnm-1.47.4-1.el9.x8 13 MB/s | 1.8 MB 00:00 2024-04-17T06:58:49.127 INFO:teuthology.orchestra.run.smithi086.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 475 kB/s | 41 kB 00:00 2024-04-17T06:58:49.160 INFO:teuthology.orchestra.run.smithi086.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-04-17T06:58:49.206 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-04-17T06:58:49.219 INFO:teuthology.orchestra.run.smithi086.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.5 MB/s | 321 kB 00:00 2024-04-17T06:58:49.244 INFO:teuthology.orchestra.run.smithi086.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 892 kB/s | 71 kB 00:00 2024-04-17T06:58:49.269 INFO:teuthology.orchestra.run.smithi086.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 737 kB/s | 36 kB 00:00 2024-04-17T06:58:49.303 INFO:teuthology.orchestra.run.smithi086.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.3 MB/s | 192 kB 00:00 2024-04-17T06:58:49.336 INFO:teuthology.orchestra.run.smithi086.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.8 MB/s | 256 kB 00:00 2024-04-17T06:58:49.362 INFO:teuthology.orchestra.run.smithi086.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 399 kB/s | 23 kB 00:00 2024-04-17T06:58:49.395 INFO:teuthology.orchestra.run.smithi086.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-04-17T06:58:49.429 INFO:teuthology.orchestra.run.smithi086.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-04-17T06:58:49.463 INFO:teuthology.orchestra.run.smithi086.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-04-17T06:58:49.488 INFO:teuthology.orchestra.run.smithi086.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-04-17T06:58:49.513 INFO:teuthology.orchestra.run.smithi086.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 578 kB/s | 29 kB 00:00 2024-04-17T06:58:49.588 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-04-17T06:58:49.588 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-04-17T06:58:49.589 INFO:teuthology.orchestra.run.smithi086.stdout:(167/174): librbd1-19.0.0-3133.ga8c84f59.el9.x8 1.7 MB/s | 3.1 MB 00:01 2024-04-17T06:58:49.906 INFO:teuthology.orchestra.run.smithi086.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.2 MB/s | 2.3 MB 00:00 2024-04-17T06:58:49.973 INFO:teuthology.orchestra.run.smithi086.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.7 MB/s | 2.2 MB 00:00 2024-04-17T06:58:50.014 INFO:teuthology.orchestra.run.smithi086.stdout:(170/174): openssl-3.0.7-27.el9.x86_64.rpm 2.3 MB/s | 1.2 MB 00:00 2024-04-17T06:58:50.031 INFO:teuthology.orchestra.run.smithi086.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 540 kB/s | 31 kB 00:00 2024-04-17T06:58:50.073 INFO:teuthology.orchestra.run.smithi086.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 2.7 MB/s | 465 kB 00:00 2024-04-17T06:58:50.123 INFO:teuthology.orchestra.run.smithi086.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 10 MB/s | 1.1 MB 00:00 2024-04-17T06:58:50.782 INFO:teuthology.orchestra.run.smithi086.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 99 kB/s | 74 kB 00:00 2024-04-17T06:58:50.797 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-17T06:58:50.797 INFO:teuthology.orchestra.run.smithi086.stdout:Total 9.6 MB/s | 229 MB 00:23 2024-04-17T06:58:51.937 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-04-17T06:58:52.006 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-04-17T06:58:52.007 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-04-17T06:58:52.318 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-04-17T06:58:52.318 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-04-17T06:58:54.699 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-04-17T06:58:54.700 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-04-17T06:58:54.784 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-04-17T06:58:54.894 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T06:58:54.966 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T06:58:55.015 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T06:58:55.055 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T06:58:55.139 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T06:58:55.237 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T06:58:55.297 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T06:58:55.330 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T06:58:55.493 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T06:58:55.925 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T06:58:55.979 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T06:58:56.136 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T06:58:56.172 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T06:58:56.310 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T06:58:56.345 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T06:58:56.403 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T06:58:56.710 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T06:58:56.768 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T06:58:56.794 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T06:58:56.951 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T06:58:56.986 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T06:58:57.085 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T06:58:57.139 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T06:58:57.164 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T06:58:57.224 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T06:58:57.515 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T06:58:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T06:58:57.614 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T06:58:57.917 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T06:58:58.089 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T06:58:58.176 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T06:58:58.358 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:58:58.425 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:58:58.737 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:58:58.921 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T06:58:59.035 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T06:58:59.073 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T06:58:59.109 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T06:58:59.167 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T06:58:59.278 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T06:58:59.361 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-04-17T06:58:59.365 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T06:58:59.415 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T06:58:59.457 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T06:58:59.505 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T06:58:59.507 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T06:58:59.543 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T06:58:59.545 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T06:58:59.589 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T06:58:59.605 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T06:58:59.649 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T06:58:59.679 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T06:58:59.681 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T06:58:59.792 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T06:58:59.795 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T06:58:59.837 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 39/198 2024-04-17T06:58:59.852 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T06:58:59.883 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T06:58:59.892 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 40/198 2024-04-17T06:58:59.926 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 41/198 2024-04-17T06:58:59.971 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T06:59:00.044 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T06:59:00.131 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T06:59:00.252 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T06:59:00.386 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T06:59:00.423 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T06:59:00.428 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T06:59:00.491 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T06:59:00.582 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T06:59:00.608 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T06:59:00.728 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T06:59:00.764 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T06:59:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T06:59:01.175 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T06:59:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T06:59:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T06:59:01.255 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T06:59:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T06:59:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T06:59:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T06:59:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T06:59:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T06:59:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T06:59:01.529 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T06:59:01.553 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T06:59:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T06:59:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T06:59:01.615 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T06:59:01.665 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 53/198 2024-04-17T06:59:01.671 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T06:59:01.706 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T06:59:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T06:59:02.023 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T06:59:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T06:59:02.154 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T06:59:02.179 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-04-17T06:59:02.282 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T06:59:02.331 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T06:59:02.377 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T06:59:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T06:59:02.424 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T06:59:02.508 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T06:59:02.582 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T06:59:02.615 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T06:59:02.668 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T06:59:02.670 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T06:59:02.726 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T06:59:02.800 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T06:59:02.823 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T06:59:02.885 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:59:02.890 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T06:59:02.935 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:59:02.962 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T06:59:03.165 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T06:59:03.226 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T06:59:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T06:59:03.277 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:59:03.302 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T06:59:03.307 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T06:59:03.397 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T06:59:03.467 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T06:59:03.468 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T06:59:03.470 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T06:59:03.491 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T06:59:03.510 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T06:59:03.580 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T06:59:03.601 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T06:59:03.612 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T06:59:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T06:59:03.641 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T06:59:03.648 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T06:59:03.678 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T06:59:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T06:59:03.715 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T06:59:03.736 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T06:59:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T06:59:03.844 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T06:59:03.906 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T06:59:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T06:59:03.948 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T06:59:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T06:59:04.008 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T06:59:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T06:59:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T06:59:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T06:59:04.079 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T06:59:04.088 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T06:59:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T06:59:04.139 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T06:59:04.183 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T06:59:04.215 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T06:59:04.216 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T06:59:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T06:59:04.247 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T06:59:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T06:59:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T06:59:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T06:59:04.337 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 17/198 2024-04-17T06:59:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T06:59:04.361 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T06:59:04.391 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T06:59:04.404 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 39/198 2024-04-17T06:59:04.408 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T06:59:04.440 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 40/198 2024-04-17T06:59:04.460 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 41/198 2024-04-17T06:59:04.462 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T06:59:04.511 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T06:59:04.569 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T06:59:04.620 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T06:59:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T06:59:04.715 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T06:59:04.781 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T06:59:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T06:59:04.843 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T06:59:04.891 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T06:59:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T06:59:04.895 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T06:59:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T06:59:04.957 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T06:59:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T06:59:05.216 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T06:59:05.277 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T06:59:05.337 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T06:59:05.415 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T06:59:05.496 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T06:59:05.503 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T06:59:05.551 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T06:59:05.592 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T06:59:05.631 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T06:59:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T06:59:05.671 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T06:59:05.692 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:59:05.717 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T06:59:05.737 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T06:59:05.742 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:59:05.778 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T06:59:05.836 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T06:59:05.874 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T06:59:05.875 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T06:59:05.923 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T06:59:05.950 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T06:59:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T06:59:06.002 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T06:59:06.047 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T06:59:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T06:59:06.063 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:59:06.082 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T06:59:06.106 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 53/198 2024-04-17T06:59:06.147 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T06:59:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T06:59:06.177 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T06:59:06.255 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T06:59:06.320 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T06:59:06.387 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T06:59:06.423 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T06:59:06.430 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T06:59:06.460 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T06:59:06.513 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T06:59:06.519 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T06:59:06.553 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T06:59:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T06:59:06.609 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T06:59:06.629 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T06:59:06.730 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T06:59:06.748 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T06:59:06.780 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T06:59:06.790 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T06:59:06.827 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T06:59:06.861 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T06:59:06.875 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T06:59:06.916 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T06:59:06.921 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T06:59:06.983 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T06:59:07.022 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T06:59:07.055 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T06:59:07.098 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T06:59:07.149 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T06:59:07.168 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T06:59:07.202 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 39/198 2024-04-17T06:59:07.212 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T06:59:07.247 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 40/198 2024-04-17T06:59:07.276 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 41/198 2024-04-17T06:59:07.306 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T06:59:07.349 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T06:59:07.388 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T06:59:07.417 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T06:59:07.457 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T06:59:07.473 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T06:59:07.496 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T06:59:07.551 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T06:59:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T06:59:07.605 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T06:59:07.761 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T06:59:07.782 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T06:59:07.794 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T06:59:07.826 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T06:59:07.834 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T06:59:07.854 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T06:59:07.867 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T06:59:07.891 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T06:59:07.923 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T06:59:07.949 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T06:59:08.036 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T06:59:08.066 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T06:59:08.085 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T06:59:08.107 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T06:59:08.151 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T06:59:08.180 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T06:59:08.213 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T06:59:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T06:59:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T06:59:08.378 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T06:59:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T06:59:08.511 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T06:59:08.546 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T06:59:08.606 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T06:59:08.688 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T06:59:08.719 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T06:59:08.741 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T06:59:08.781 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T06:59:08.803 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T06:59:08.809 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T06:59:08.854 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T06:59:08.878 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T06:59:08.914 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T06:59:08.945 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 53/198 2024-04-17T06:59:08.978 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T06:59:09.132 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T06:59:09.184 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T06:59:09.251 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T06:59:09.353 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T06:59:09.365 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T06:59:09.379 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T06:59:09.404 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T06:59:09.446 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T06:59:09.477 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T06:59:09.517 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T06:59:09.555 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T06:59:09.574 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T06:59:09.630 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T06:59:09.643 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T06:59:09.681 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T06:59:09.699 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T06:59:09.730 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T06:59:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T06:59:09.902 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T06:59:09.908 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T06:59:09.945 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T06:59:09.990 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T06:59:09.998 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T06:59:10.064 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T06:59:10.065 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T06:59:10.161 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T06:59:10.206 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T06:59:10.210 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T06:59:10.246 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T06:59:10.312 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T06:59:10.325 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T06:59:10.354 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T06:59:10.378 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T06:59:10.414 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T06:59:10.416 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T06:59:10.468 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T06:59:10.473 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T06:59:10.607 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T06:59:10.671 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T06:59:10.694 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T06:59:10.743 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T06:59:10.780 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T06:59:10.829 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T06:59:10.938 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T06:59:10.939 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T06:59:10.979 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T06:59:11.022 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T06:59:11.039 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T06:59:11.052 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T06:59:11.076 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T06:59:11.085 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T06:59:11.107 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T06:59:11.146 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T06:59:11.233 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T06:59:11.405 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T06:59:11.523 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T06:59:11.537 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T06:59:11.554 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T06:59:11.596 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T06:59:11.632 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T06:59:11.658 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T06:59:11.678 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T06:59:11.700 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T06:59:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T06:59:11.732 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T06:59:11.735 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 125/198 2024-04-17T06:59:11.768 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T06:59:11.782 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T06:59:11.869 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T06:59:11.922 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T06:59:11.983 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T06:59:12.046 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T06:59:12.154 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T06:59:12.194 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T06:59:12.208 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T06:59:12.252 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T06:59:12.287 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T06:59:12.348 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T06:59:12.382 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T06:59:12.413 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T06:59:12.483 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T06:59:12.510 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 129/198 2024-04-17T06:59:12.714 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T06:59:12.766 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T06:59:12.817 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T06:59:12.892 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T06:59:12.946 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T06:59:12.994 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T06:59:13.105 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T06:59:13.111 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T06:59:13.146 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T06:59:13.170 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T06:59:13.231 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T06:59:13.256 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T06:59:13.275 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T06:59:13.417 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T06:59:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T06:59:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T06:59:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T06:59:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T06:59:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T06:59:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T06:59:14.786 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T06:59:14.861 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T06:59:14.874 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T06:59:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T06:59:14.914 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T06:59:14.939 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T06:59:14.942 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T06:59:14.981 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T06:59:14.997 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T06:59:15.108 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T06:59:15.129 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T06:59:15.148 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T06:59:15.165 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T06:59:15.342 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T06:59:15.372 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T06:59:15.381 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T06:59:15.411 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T06:59:15.502 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T06:59:15.509 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T06:59:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 125/198 2024-04-17T06:59:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T06:59:15.788 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T06:59:15.827 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T06:59:15.988 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T06:59:15.988 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T06:59:15.988 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T06:59:15.988 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:16.036 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T06:59:16.105 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T06:59:16.134 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T06:59:16.226 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T06:59:16.253 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 129/198 2024-04-17T06:59:16.319 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T06:59:16.489 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T06:59:18.024 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T06:59:18.215 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T06:59:18.255 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T06:59:18.293 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T06:59:18.352 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T06:59:18.740 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T06:59:18.772 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T06:59:18.858 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T06:59:18.891 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 124/198 2024-04-17T06:59:18.908 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T06:59:18.919 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 125/198 2024-04-17T06:59:18.994 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T06:59:19.535 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T06:59:19.537 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T06:59:19.566 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T06:59:19.578 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T06:59:19.657 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/198 2024-04-17T06:59:19.685 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 129/198 2024-04-17T06:59:19.744 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T06:59:19.744 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T06:59:19.744 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T06:59:19.744 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:19.804 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T06:59:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T06:59:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T06:59:23.046 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/198 2024-04-17T06:59:23.083 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T06:59:23.235 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/198 2024-04-17T06:59:23.235 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T06:59:23.235 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T06:59:23.235 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:23.276 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T06:59:28.951 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T06:59:28.951 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-04-17T06:59:28.951 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-04-17T06:59:28.951 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-04-17T06:59:28.951 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-04-17T06:59:28.951 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-04-17T06:59:28.951 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-04-17T06:59:28.951 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-04-17T06:59:28.951 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:29.197 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T06:59:29.244 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T06:59:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:29.244 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T06:59:29.244 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:30.893 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T06:59:30.970 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T06:59:30.970 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:30.971 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T06:59:30.971 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T06:59:30.971 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T06:59:30.971 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:31.210 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T06:59:31.280 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T06:59:31.280 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:31.280 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T06:59:31.281 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T06:59:31.281 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T06:59:31.281 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:31.769 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T06:59:31.839 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T06:59:31.839 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:31.839 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T06:59:31.839 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T06:59:31.839 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T06:59:31.840 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:31.962 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T06:59:32.433 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T06:59:32.434 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2024-04-17T06:59:32.434 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2024-04-17T06:59:32.434 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2024-04-17T06:59:32.434 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2024-04-17T06:59:32.434 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2024-04-17T06:59:32.434 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /root 2024-04-17T06:59:32.434 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2024-04-17T06:59:32.434 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:32.595 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T06:59:32.622 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T06:59:32.664 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 139/198 2024-04-17T06:59:32.664 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T06:59:32.711 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T06:59:32.711 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:32.711 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T06:59:32.711 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:32.762 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 140/198 2024-04-17T06:59:32.802 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T06:59:34.120 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T06:59:34.162 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T06:59:34.292 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T06:59:34.370 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T06:59:34.370 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T06:59:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T06:59:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T06:59:34.371 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:34.617 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T06:59:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T06:59:34.683 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:34.683 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T06:59:34.683 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T06:59:34.684 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T06:59:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T06:59:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T06:59:35.228 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:35.228 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T06:59:35.228 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T06:59:35.228 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T06:59:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:35.260 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T06:59:35.287 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T06:59:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T06:59:35.433 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T06:59:35.615 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 144/198 2024-04-17T06:59:35.639 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T06:59:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T06:59:35.699 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:35.699 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T06:59:35.699 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T06:59:35.699 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T06:59:35.699 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:35.757 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T06:59:35.793 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T06:59:35.853 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 147/198 2024-04-17T06:59:35.857 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 132/198 2024-04-17T06:59:35.857 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-04-17T06:59:35.857 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-04-17T06:59:35.857 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-04-17T06:59:35.857 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-04-17T06:59:35.858 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-04-17T06:59:35.858 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-04-17T06:59:35.858 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-04-17T06:59:35.858 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:35.958 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T06:59:35.996 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T06:59:36.046 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 139/198 2024-04-17T06:59:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T06:59:36.125 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 133/198 2024-04-17T06:59:36.125 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:36.125 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T06:59:36.125 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:36.175 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 140/198 2024-04-17T06:59:36.209 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T06:59:36.891 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T06:59:36.926 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T06:59:36.997 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T06:59:36.997 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:36.997 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T06:59:36.997 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T06:59:36.997 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T06:59:36.997 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:37.052 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T06:59:37.120 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T06:59:37.120 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:37.121 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T06:59:37.121 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:37.211 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T06:59:37.295 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T06:59:37.295 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:37.295 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T06:59:37.295 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T06:59:37.295 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T06:59:37.295 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T06:59:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T06:59:37.661 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T06:59:37.724 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T06:59:37.732 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T06:59:37.732 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:37.732 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T06:59:37.732 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T06:59:37.733 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T06:59:37.733 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:37.831 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 134/198 2024-04-17T06:59:37.831 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:37.831 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T06:59:37.831 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T06:59:37.831 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T06:59:37.831 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:38.099 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T06:59:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/198 2024-04-17T06:59:38.195 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:38.195 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T06:59:38.195 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T06:59:38.195 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T06:59:38.195 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:38.645 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T06:59:38.675 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T06:59:38.677 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T06:59:38.743 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/198 2024-04-17T06:59:38.743 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:38.743 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T06:59:38.743 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T06:59:38.743 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T06:59:38.743 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:38.821 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T06:59:38.860 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T06:59:39.011 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 144/198 2024-04-17T06:59:39.038 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T06:59:39.108 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T06:59:39.108 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:39.108 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T06:59:39.108 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T06:59:39.108 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T06:59:39.108 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:39.172 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T06:59:39.212 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T06:59:39.290 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 147/198 2024-04-17T06:59:39.532 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T06:59:39.559 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 138/198 2024-04-17T06:59:39.612 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 139/198 2024-04-17T06:59:39.713 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 140/198 2024-04-17T06:59:39.740 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T06:59:40.363 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T06:59:40.399 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T06:59:40.477 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T06:59:40.477 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:40.478 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T06:59:40.478 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T06:59:40.478 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T06:59:40.478 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:40.534 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T06:59:40.627 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T06:59:40.627 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:40.627 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T06:59:40.627 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:40.718 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T06:59:40.811 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T06:59:40.811 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:40.811 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T06:59:40.811 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T06:59:40.811 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T06:59:40.811 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:41.048 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 141/198 2024-04-17T06:59:41.083 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T06:59:41.202 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T06:59:41.280 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T06:59:41.281 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:41.281 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T06:59:41.281 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T06:59:41.281 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T06:59:41.281 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:42.178 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 142/198 2024-04-17T06:59:42.208 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T06:59:42.353 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 143/198 2024-04-17T06:59:42.530 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 144/198 2024-04-17T06:59:42.588 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T06:59:42.655 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 145/198 2024-04-17T06:59:42.655 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:42.655 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T06:59:42.655 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T06:59:42.655 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T06:59:42.655 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:42.719 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T06:59:42.760 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 146/198 2024-04-17T06:59:42.771 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T06:59:42.826 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 154/198 2024-04-17T06:59:42.850 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 147/198 2024-04-17T06:59:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 155/198 2024-04-17T06:59:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T06:59:43.211 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 157/198 2024-04-17T06:59:43.338 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T06:59:43.385 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 159/198 2024-04-17T06:59:43.432 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T06:59:43.494 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T06:59:43.538 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T06:59:43.574 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 163/198 2024-04-17T06:59:43.618 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T06:59:43.671 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T06:59:43.754 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T06:59:43.850 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T06:59:43.882 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T06:59:43.894 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T06:59:43.913 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T06:59:43.979 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/198 2024-04-17T06:59:43.979 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:43.979 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T06:59:43.979 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T06:59:43.979 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T06:59:43.979 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:44.030 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T06:59:44.118 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 150/198 2024-04-17T06:59:44.118 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:44.118 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T06:59:44.118 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:44.197 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T06:59:44.295 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T06:59:44.295 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:44.295 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T06:59:44.296 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T06:59:44.296 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T06:59:44.296 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:44.664 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T06:59:44.735 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 152/198 2024-04-17T06:59:44.735 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:59:44.735 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T06:59:44.735 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T06:59:44.736 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T06:59:44.736 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:44.826 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:59:44.876 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:59:44.932 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:59:46.318 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T06:59:46.332 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 170/198 2024-04-17T06:59:46.373 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 154/198 2024-04-17T06:59:46.407 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 155/198 2024-04-17T06:59:46.416 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T06:59:46.461 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T06:59:46.470 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T06:59:46.565 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T06:59:46.601 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T06:59:46.643 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T06:59:46.677 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T06:59:46.718 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T06:59:46.719 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T06:59:46.745 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 157/198 2024-04-17T06:59:46.858 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T06:59:46.869 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T06:59:46.899 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T06:59:46.900 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T06:59:46.917 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 159/198 2024-04-17T06:59:46.969 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T06:59:47.032 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T06:59:47.070 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T06:59:47.078 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T06:59:47.112 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 163/198 2024-04-17T06:59:47.116 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T06:59:47.157 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T06:59:47.169 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T06:59:47.203 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T06:59:47.209 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T06:59:47.235 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T06:59:47.299 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T06:59:47.305 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T06:59:47.375 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T06:59:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T06:59:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T06:59:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T06:59:47.465 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T06:59:47.499 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T06:59:47.533 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T06:59:47.566 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T06:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:59:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:59:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:59:47.941 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T06:59:48.114 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T06:59:48.165 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T06:59:48.194 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T06:59:48.231 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T06:59:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T06:59:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:59:48.389 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T06:59:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:59:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:59:48.641 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T06:59:48.641 INFO:teuthology.orchestra.run.smithi103.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-17T06:59:48.642 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:49.798 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T06:59:49.822 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 170/198 2024-04-17T06:59:49.854 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 154/198 2024-04-17T06:59:49.895 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 155/198 2024-04-17T06:59:49.933 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T06:59:49.962 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T06:59:50.016 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T06:59:50.162 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T06:59:50.198 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T06:59:50.220 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 157/198 2024-04-17T06:59:50.231 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T06:59:50.266 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T06:59:50.307 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T06:59:50.308 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T06:59:50.368 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T06:59:50.420 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 159/198 2024-04-17T06:59:50.436 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T06:59:50.469 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T06:59:50.478 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T06:59:50.479 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T06:59:50.528 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T06:59:50.572 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T06:59:50.607 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 163/198 2024-04-17T06:59:50.608 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T06:59:50.643 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T06:59:50.653 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T06:59:50.696 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T06:59:50.706 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T06:59:50.740 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T06:59:50.772 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T06:59:50.778 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T06:59:50.842 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T06:59:50.889 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T06:59:50.909 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T06:59:50.944 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T06:59:50.961 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T06:59:50.994 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T06:59:51.027 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T06:59:51.086 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T06:59:51.119 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T06:59:51.157 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:59:51.157 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:59:51.461 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:59:51.462 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T06:59:51.592 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T06:59:51.651 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T06:59:51.681 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T06:59:51.717 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T06:59:51.757 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T06:59:51.838 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T06:59:51.853 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:59:51.899 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:59:51.954 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:59:52.089 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T06:59:52.089 INFO:teuthology.orchestra.run.smithi098.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-17T06:59:52.089 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T06:59:53.308 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 170/198 2024-04-17T06:59:53.395 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T06:59:53.438 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T06:59:53.577 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T06:59:53.615 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T06:59:53.648 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T06:59:53.682 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T06:59:53.724 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T06:59:53.724 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T06:59:53.861 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T06:59:53.903 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T06:59:53.903 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T06:59:54.023 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T06:59:54.069 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T06:59:54.122 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T06:59:54.157 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T06:59:54.191 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/198 2024-04-17T06:59:54.191 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/198 2024-04-17T06:59:54.191 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/198 2024-04-17T06:59:54.191 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/198 2024-04-17T06:59:54.192 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/198 2024-04-17T06:59:54.193 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/198 2024-04-17T06:59:54.194 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T06:59:54.209 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/198 2024-04-17T06:59:54.209 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/198 2024-04-17T06:59:54.209 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/198 2024-04-17T06:59:54.209 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/198 2024-04-17T06:59:54.209 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/198 2024-04-17T06:59:54.209 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/198 2024-04-17T06:59:54.209 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T06:59:54.210 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T06:59:54.211 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T06:59:54.212 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T06:59:54.213 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T06:59:54.214 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T06:59:54.215 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T06:59:54.216 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T06:59:54.217 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T06:59:54.218 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T06:59:54.219 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T06:59:54.220 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T06:59:54.221 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T06:59:54.221 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T06:59:54.221 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T06:59:54.221 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T06:59:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T06:59:54.346 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T06:59:54.395 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T06:59:54.427 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T06:59:54.461 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T06:59:54.495 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T06:59:54.528 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T06:59:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:59:54.565 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:59:54.878 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:59:54.878 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T06:59:55.017 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T06:59:55.069 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T06:59:55.106 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T06:59:55.142 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T06:59:55.182 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T06:59:55.302 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T06:59:55.579 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T06:59:55.580 INFO:teuthology.orchestra.run.smithi086.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-17T06:59:55.580 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T06:59:56.713 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T06:59:56.713 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:56.713 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-04-17T06:59:56.713 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T06:59:56.713 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T06:59:56.713 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.714 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T06:59:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T06:59:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T06:59:56.717 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: jq-1.6-16.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T06:59:56.718 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T06:59:56.719 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T06:59:56.720 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T06:59:56.721 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T06:59:56.722 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T06:59:56.723 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T06:59:56.724 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T06:59:56.725 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T06:59:56.726 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T06:59:56.726 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T06:59:56.726 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T06:59:56.726 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T06:59:56.726 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T06:59:56.726 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T06:59:56.726 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T06:59:56.726 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-04-17T06:59:57.402 DEBUG:teuthology.parallel:result is None 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/198 2024-04-17T06:59:57.463 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/198 2024-04-17T06:59:57.464 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T06:59:57.465 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/198 2024-04-17T06:59:57.465 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/198 2024-04-17T06:59:57.465 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/198 2024-04-17T06:59:57.465 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/198 2024-04-17T06:59:57.465 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/198 2024-04-17T06:59:57.465 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/198 2024-04-17T06:59:57.465 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/198 2024-04-17T06:59:57.465 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/198 2024-04-17T06:59:57.465 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/198 2024-04-17T06:59:57.465 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/198 2024-04-17T06:59:57.480 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T06:59:57.481 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T06:59:57.482 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T06:59:57.483 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T06:59:57.484 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T06:59:57.485 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T06:59:57.486 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T06:59:57.487 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T06:59:57.488 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T06:59:57.489 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T06:59:57.490 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T06:59:57.491 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T06:59:57.492 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T07:00:00.000 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T07:00:00.001 INFO:teuthology.orchestra.run.smithi098.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T07:00:00.002 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.003 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:00.004 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: jq-1.6-16.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T07:00:00.005 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T07:00:00.006 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T07:00:00.007 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.008 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T07:00:00.009 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T07:00:00.010 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.011 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T07:00:00.012 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T07:00:00.013 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T07:00:00.014 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T07:00:00.014 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-17T07:00:00.014 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-04-17T07:00:00.685 DEBUG:teuthology.parallel:result is None 2024-04-17T07:00:00.760 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/198 2024-04-17T07:00:00.760 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/198 2024-04-17T07:00:00.760 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/198 2024-04-17T07:00:00.760 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/198 2024-04-17T07:00:00.760 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/198 2024-04-17T07:00:00.760 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/198 2024-04-17T07:00:00.760 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/198 2024-04-17T07:00:00.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/198 2024-04-17T07:00:00.762 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/198 2024-04-17T07:00:00.777 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/198 2024-04-17T07:00:00.777 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/198 2024-04-17T07:00:00.777 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/198 2024-04-17T07:00:00.777 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/198 2024-04-17T07:00:00.777 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/198 2024-04-17T07:00:00.777 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T07:00:00.778 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T07:00:00.779 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T07:00:00.780 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T07:00:00.781 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T07:00:00.782 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T07:00:00.783 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T07:00:00.784 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T07:00:00.785 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T07:00:00.786 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T07:00:00.786 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T07:00:00.786 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T07:00:00.786 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/198 2024-04-17T07:00:00.786 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T07:00:00.786 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 153/198 2024-04-17T07:00:00.787 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T07:00:00.787 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T07:00:00.787 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T07:00:00.787 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T07:00:00.787 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T07:00:00.787 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T07:00:00.787 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T07:00:00.787 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T07:00:00.788 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T07:00:00.789 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T07:00:00.790 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T07:00:03.235 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T07:00:03.235 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:00:03.235 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T07:00:03.236 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T07:00:03.237 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T07:00:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T07:00:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T07:00:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:03.239 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T07:00:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: jq-1.6-16.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T07:00:03.241 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T07:00:03.242 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T07:00:03.243 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T07:00:03.244 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T07:00:03.245 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T07:00:03.246 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T07:00:03.247 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T07:00:03.248 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T07:00:03.248 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T07:00:03.248 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T07:00:03.248 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T07:00:03.248 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T07:00:03.248 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T07:00:03.248 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T07:00:03.248 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T07:00:03.248 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T07:00:03.248 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T07:00:03.249 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T07:00:03.250 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T07:00:03.250 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:00:03.250 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-17T07:00:03.918 DEBUG:teuthology.parallel:result is None 2024-04-17T07:00:03.919 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:00:04.067 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T07:00:04.123 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-17T07:00:04.123 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-17T07:00:04.123 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-17T07:00:04.125 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:00:04.271 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T07:00:04.333 INFO:teuthology.orchestra.run.smithi098.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-17T07:00:04.333 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-17T07:00:04.334 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-17T07:00:04.335 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:00:04.558 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T07:00:04.633 INFO:teuthology.orchestra.run.smithi103.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-17T07:00:04.633 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-17T07:00:04.633 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-17T07:00:04.636 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-17T07:00:04.637 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:00:04.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T07:00:04.722 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T07:00:04.722 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T07:00:04.799 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T07:00:04.799 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T07:00:04.907 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-17T07:00:04.907 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:00:04.907 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T07:00:04.984 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T07:00:05.096 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T07:00:05.096 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T07:00:05.190 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T07:00:05.327 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T07:00:05.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T07:00:05.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T07:00:05.572 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-17T07:00:05.573 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:00:05.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T07:00:05.648 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T07:00:05.757 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T07:00:05.757 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T07:00:05.874 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T07:00:06.008 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T07:00:06.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T07:00:06.109 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T07:00:06.253 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-17T07:00:06.253 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:00:06.253 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T07:00:06.321 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T07:00:06.432 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T07:00:06.432 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T07:00:06.531 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T07:00:06.624 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T07:00:06.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T07:00:06.724 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T07:00:06.817 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-17T07:00:06.921 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-17T07:00:06.921 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:00:06.922 INFO:tasks.cephadm:Cluster fsid is 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:00:06.922 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-17T07:00:06.922 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.86', 'mon.b': '172.21.15.98', 'mon.c': '172.21.15.103'} 2024-04-17T07:00:06.922 INFO:tasks.cephadm:First mon is mon.a on smithi086 2024-04-17T07:00:06.922 INFO:tasks.cephadm:First mgr is x 2024-04-17T07:00:06.923 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-17T07:00:06.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-04-17T07:00:07.005 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname $(hostname -s) 2024-04-17T07:00:07.105 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-04-17T07:00:07.208 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-17T07:00:07.209 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:00:07.359 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78637/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3133-ga8c84f59', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3133.ga8c84f59'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 21:41:56.138585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240416.193735', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-17T07:00:07.507 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-rishabh-testing-20240416.193735, sha1 a8c84f5997950fd7d008204839253291d52e6eda from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:00:07.508 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm 2024-04-17T07:00:07.508 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-17T07:00:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 17 07:00 /home/ubuntu/cephtest/cephadm 2024-04-17T07:00:08.518 DEBUG:teuthology.orchestra.run.smithi098:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-17T07:00:09.988 INFO:teuthology.orchestra.run.smithi098.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 17 07:00 /home/ubuntu/cephtest/cephadm 2024-04-17T07:00:09.988 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-17T07:00:11.255 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 17 07:00 /home/ubuntu/cephtest/cephadm 2024-04-17T07:00:11.255 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-17T07:00:11.316 DEBUG:teuthology.orchestra.run.smithi098:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-17T07:00:11.376 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-17T07:00:11.453 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda on all hosts... 2024-04-17T07:00:11.453 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-17T07:00:11.460 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-17T07:00:11.468 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-17T07:00:12.724 INFO:teuthology.orchestra.run.smithi098.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T07:00:12.778 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T07:00:12.919 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T07:00:49.185 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-17T07:00:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T07:00:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-17T07:00:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-04-17T07:00:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-17T07:00:49.185 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-04-17T07:00:49.185 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-17T07:00:52.042 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-17T07:00:52.042 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T07:00:52.043 INFO:teuthology.orchestra.run.smithi098.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-17T07:00:52.043 INFO:teuthology.orchestra.run.smithi098.stdout: "repo_digests": [ 2024-04-17T07:00:52.043 INFO:teuthology.orchestra.run.smithi098.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-17T07:00:52.043 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-04-17T07:00:52.043 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-17T07:01:02.701 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:01:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T07:01:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-17T07:01:02.701 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-04-17T07:01:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-17T07:01:02.702 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-17T07:01:02.702 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:01:02.770 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-04-17T07:01:02.860 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph 2024-04-17T07:01:02.946 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-04-17T07:01:03.050 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-04-17T07:01:03.141 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /etc/ceph 2024-04-17T07:01:03.232 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-04-17T07:01:03.325 INFO:tasks.cephadm:Writing seed config... 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-17T07:01:03.327 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-17T07:01:03.328 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-17T07:01:03.329 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-17T07:01:03.330 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:01:03.330 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-17T07:01:03.384 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1f7ed99a-fc88-11ee-bc91-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-17T07:01:03.384 DEBUG:teuthology.orchestra.run.smithi086:mon.a> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.a.service 2024-04-17T07:01:03.431 DEBUG:teuthology.orchestra.run.smithi086:mgr.x> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.x.service 2024-04-17T07:01:03.478 INFO:tasks.cephadm:Bootstrapping... 2024-04-17T07:01:03.478 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda -v bootstrap --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.86 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:01:03.758 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-17T07:01:03.759 INFO:teuthology.orchestra.run.smithi086.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda', '-v', 'bootstrap', '--fsid', '1f7ed99a-fc88-11ee-bc91-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.86', '--skip-admin-label'] 2024-04-17T07:01:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-17T07:01:03.759 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying podman|docker is present... 2024-04-17T07:01:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-17T07:01:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying lvm2 is present... 2024-04-17T07:01:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying time synchronization is in place... 2024-04-17T07:01:04.003 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-17T07:01:04.003 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-17T07:01:04.019 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-17T07:01:04.020 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-17T07:01:04.040 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-04-17T07:01:04.055 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-04-17T07:01:04.056 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-04-17T07:01:04.056 INFO:teuthology.orchestra.run.smithi086.stdout:Repeating the final host check... 2024-04-17T07:01:04.111 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-17T07:01:04.111 INFO:teuthology.orchestra.run.smithi086.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-17T07:01:04.111 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl is present 2024-04-17T07:01:04.111 INFO:teuthology.orchestra.run.smithi086.stdout:lvcreate is present 2024-04-17T07:01:04.126 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-17T07:01:04.126 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-17T07:01:04.140 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-17T07:01:04.140 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-17T07:01:04.155 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-04-17T07:01:04.170 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-04-17T07:01:04.171 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-04-17T07:01:04.171 INFO:teuthology.orchestra.run.smithi086.stdout:Host looks OK 2024-04-17T07:01:04.171 INFO:teuthology.orchestra.run.smithi086.stdout:Cluster fsid: 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:04.171 INFO:teuthology.orchestra.run.smithi086.stdout:Acquiring lock 140083279860640 on /run/cephadm/1f7ed99a-fc88-11ee-bc91-c7b262605968.lock 2024-04-17T07:01:04.171 INFO:teuthology.orchestra.run.smithi086.stdout:Lock 140083279860640 acquired on /run/cephadm/1f7ed99a-fc88-11ee-bc91-c7b262605968.lock 2024-04-17T07:01:04.172 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 3300 ... 2024-04-17T07:01:04.172 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 6789 ... 2024-04-17T07:01:04.173 INFO:teuthology.orchestra.run.smithi086.stdout:Base mon IP(s) is [172.21.15.86:3300, 172.21.15.86:6789], mon addrv is [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-04-17T07:01:04.181 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.86 metric 100 2024-04-17T07:01:04.181 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.86 metric 100 2024-04-17T07:01:04.189 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-17T07:01:04.189 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-17T07:01:04.189 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1775sec hoplimit 64 pref medium 2024-04-17T07:01:04.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-17T07:01:04.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-17T07:01:04.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-17T07:01:04.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-17T07:01:04.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-17T07:01:04.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-17T07:01:04.198 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-04-17T07:01:04.198 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-04-17T07:01:04.198 INFO:teuthology.orchestra.run.smithi086.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-17T07:01:04.199 INFO:teuthology.orchestra.run.smithi086.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-17T07:01:04.200 INFO:teuthology.orchestra.run.smithi086.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T07:01:09.907 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-17T07:01:09.907 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-17T07:01:09.908 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Getting image source signatures 2024-04-17T07:01:09.908 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-17T07:01:09.908 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:9473d3fd79377e8e2904ab0f292b04da81cb2f2bfccde15ec927b481a725f6f4 2024-04-17T07:01:09.908 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying config sha256:76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-17T07:01:09.908 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-17T07:01:11.158 INFO:teuthology.orchestra.run.smithi086.stdout:ceph: stdout ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T07:01:11.158 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph version: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-17T07:01:11.158 INFO:teuthology.orchestra.run.smithi086.stdout:Extracting ceph user uid/gid from container image... 2024-04-17T07:01:12.238 INFO:teuthology.orchestra.run.smithi086.stdout:stat: stdout 167 167 2024-04-17T07:01:12.238 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial keys... 2024-04-17T07:01:13.218 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQC4cx9mwsGhJhAAWdiXLj9IUztopdKtQ4V+xw== 2024-04-17T07:01:14.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQC5cx9mcGBuLBAAWXmkAF79uAblrwvhqD9pFQ== 2024-04-17T07:01:15.536 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQC6cx9m4tc0NRAAOsOeeGEXVYZ3wAhBgdIi/Q== 2024-04-17T07:01:15.540 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial monmap... 2024-04-17T07:01:16.647 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-17T07:01:16.648 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-17T07:01:16.648 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:16.648 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-17T07:01:16.648 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool for a [v2:172.21.15.86:3300,v1:172.21.15.86:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-17T07:01:16.648 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = quincy 2024-04-17T07:01:16.648 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: set fsid to 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:16.648 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-17T07:01:16.648 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:01:16.649 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mon... 2024-04-17T07:01:17.655 INFO:teuthology.orchestra.run.smithi086.stdout:create mon.a on 2024-04-17T07:01:18.037 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-17T07:01:18.284 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-17T07:01:18.563 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968.target → /etc/systemd/system/ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968.target. 2024-04-17T07:01:18.564 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968.target → /etc/systemd/system/ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968.target. 2024-04-17T07:01:18.950 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.a 2024-04-17T07:01:18.950 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.a.service: Unit ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.a.service not loaded. 2024-04-17T07:01:19.224 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968.target.wants/ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@.service. 2024-04-17T07:01:19.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:19 smithi086 podman[31346]: 2024-04-17 07:01:19.856049429 +0000 UTC m=+0.371037669 container init ec8bfdf036663d9f7584124376a52bd2b125ac9ccc94f9d2cdc5168156b7424d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:01:19.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:19 smithi086 podman[31346]: 2024-04-17 07:01:19.866750743 +0000 UTC m=+0.381738954 container start ec8bfdf036663d9f7584124376a52bd2b125ac9ccc94f9d2cdc5168156b7424d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-17T07:01:19.983 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T07:01:19.983 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-17T07:01:19.999 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T07:01:19.999 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-17T07:01:19.999 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-17T07:01:19.999 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-04-17T07:01:19.999 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon to start... 2024-04-17T07:01:20.000 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon... 2024-04-17T07:01:20.147 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:19 smithi086 ceph-mon[31365]: mkfs 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:20.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:19 smithi086 ceph-mon[31365]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-17T07:01:20.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:19 smithi086 bash[31346]: ec8bfdf036663d9f7584124376a52bd2b125ac9ccc94f9d2cdc5168156b7424d 2024-04-17T07:01:20.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:19 smithi086 systemd[1]: Started Ceph mon.a for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:01:21.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-17T07:01:21.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: monmap epoch 1 2024-04-17T07:01:21.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:21.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: last_changed 2024-04-17T07:01:16.008725+0000 2024-04-17T07:01:21.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:01:21.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: min_mon_release 19 (squid) 2024-04-17T07:01:21.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: election_strategy: 1 2024-04-17T07:01:21.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:01:21.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: fsmap 2024-04-17T07:01:21.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: osdmap e1: 0 total, 0 up, 0 in 2024-04-17T07:01:21.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: mgrmap e1: no daemons active 2024-04-17T07:01:21.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:20 smithi086 ceph-mon[31365]: from='client.? 172.21.15.86:0/353670554' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-17T07:01:21.362 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout cluster: 2024-04-17T07:01:21.363 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout id: 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:21.363 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-17T07:01:21.363 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:21.363 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout services: 2024-04-17T07:01:21.368 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.766005s) 2024-04-17T07:01:21.368 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout data: 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pgs: 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.613+0000 7f7a91800700 1 Processor -- start 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.613+0000 7f7a91800700 1 -- start start 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.613+0000 7f7a91800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c10a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:21.369 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.613+0000 7f7a91800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c10af60 con 0x7f7a8c10a570 2024-04-17T07:01:21.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.614+0000 7f7a8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c10a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:21.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.614+0000 7f7a8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c10a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54010/0 (socket says 172.21.15.86:54010) 2024-04-17T07:01:21.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.614+0000 7f7a8aa00700 1 -- 172.21.15.86:0/3067316892 learned_addr learned my addr 172.21.15.86:0/3067316892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:21.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.615+0000 7f7a8aa00700 1 -- 172.21.15.86:0/3067316892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c10b720 con 0x7f7a8c10a570 2024-04-17T07:01:21.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.615+0000 7f7a8aa00700 1 --2- 172.21.15.86:0/3067316892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c10a990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098a0 tx=0x7f7a7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2ae9508dccf98c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:21.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.616+0000 7f7a8a000700 1 -- 172.21.15.86:0/3067316892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a7c004160 con 0x7f7a8c10a570 2024-04-17T07:01:21.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.616+0000 7f7a8a000700 1 -- 172.21.15.86:0/3067316892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a7c004300 con 0x7f7a8c10a570 2024-04-17T07:01:21.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.616+0000 7f7a8a000700 1 -- 172.21.15.86:0/3067316892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a7c0036a0 con 0x7f7a8c10a570 2024-04-17T07:01:21.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.616+0000 7f7a91800700 1 -- 172.21.15.86:0/3067316892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 msgr2=0x7f7a8c10a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:21.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.616+0000 7f7a91800700 1 --2- 172.21.15.86:0/3067316892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c10a990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098a0 tx=0x7f7a7c009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.616+0000 7f7a91800700 1 -- 172.21.15.86:0/3067316892 shutdown_connections 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.616+0000 7f7a91800700 1 --2- 172.21.15.86:0/3067316892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c10a990 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.616+0000 7f7a91800700 1 -- 172.21.15.86:0/3067316892 >> 172.21.15.86:0/3067316892 conn(0x7f7a8c07dc10 msgr2=0x7f7a8c1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.617+0000 7f7a91800700 1 -- 172.21.15.86:0/3067316892 shutdown_connections 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.617+0000 7f7a91800700 1 -- 172.21.15.86:0/3067316892 wait complete. 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.618+0000 7f7a91800700 1 Processor -- start 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.618+0000 7f7a91800700 1 -- start start 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.619+0000 7f7a91800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.619+0000 7f7a91800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c10bc50 con 0x7f7a8c10a570 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.619+0000 7f7a8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:21.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.619+0000 7f7a8aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54020/0 (socket says 172.21.15.86:54020) 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.619+0000 7f7a8aa00700 1 -- 172.21.15.86:0/353670554 learned_addr learned my addr 172.21.15.86:0/353670554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.620+0000 7f7a8aa00700 1 -- 172.21.15.86:0/353670554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c19e0e0 con 0x7f7a8c10a570 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.620+0000 7f7a8aa00700 1 --2- 172.21.15.86:0/353670554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c19dba0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000c00 tx=0x7f7a7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.620+0000 7f7a83400700 1 -- 172.21.15.86:0/353670554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a7c003db0 con 0x7f7a8c10a570 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.620+0000 7f7a83400700 1 -- 172.21.15.86:0/353670554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a7c011630 con 0x7f7a8c10a570 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.620+0000 7f7a83400700 1 -- 172.21.15.86:0/353670554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a7c011950 con 0x7f7a8c10a570 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.620+0000 7f7a91800700 1 -- 172.21.15.86:0/353670554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c19e370 con 0x7f7a8c10a570 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.620+0000 7f7a91800700 1 -- 172.21.15.86:0/353670554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1a1060 con 0x7f7a8c10a570 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.621+0000 7f7a83400700 1 -- 172.21.15.86:0/353670554 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a7c01a460 con 0x7f7a8c10a570 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.621+0000 7f7a83400700 1 -- 172.21.15.86:0/353670554 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7a7c01b780 con 0x7f7a8c10a570 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.622+0000 7f7a91800700 1 -- 172.21.15.86:0/353670554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c0640b0 con 0x7f7a8c10a570 2024-04-17T07:01:21.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.625+0000 7f7a83400700 1 -- 172.21.15.86:0/353670554 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f7a7c01b980 con 0x7f7a8c10a570 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.704+0000 7f7a91800700 1 -- 172.21.15.86:0/353670554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7a8c197100 con 0x7f7a8c10a570 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.705+0000 7f7a83400700 1 -- 172.21.15.86:0/353670554 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7a8c197100 con 0x7f7a8c10a570 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.705+0000 7f7a91800700 1 -- 172.21.15.86:0/353670554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 msgr2=0x7f7a8c19dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.705+0000 7f7a91800700 1 --2- 172.21.15.86:0/353670554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c19dba0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000c00 tx=0x7f7a7c004750 comp rx=0 tx=0).stop 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.706+0000 7f7a91800700 1 -- 172.21.15.86:0/353670554 shutdown_connections 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.706+0000 7f7a91800700 1 --2- 172.21.15.86:0/353670554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c10a570 0x7f7a8c19dba0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.706+0000 7f7a91800700 1 -- 172.21.15.86:0/353670554 >> 172.21.15.86:0/353670554 conn(0x7f7a8c07dc10 msgr2=0x7f7a8c191f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.706+0000 7f7a91800700 1 -- 172.21.15.86:0/353670554 shutdown_connections 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:20.706+0000 7f7a91800700 1 -- 172.21.15.86:0/353670554 wait complete. 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:mon is available 2024-04-17T07:01:21.373 INFO:teuthology.orchestra.run.smithi086.stdout:Assimilating anything we can from ceph.conf... 2024-04-17T07:01:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:22 smithi086 ceph-mon[31365]: from='client.? 172.21.15.86:0/3266656711' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-17T07:01:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:22 smithi086 ceph-mon[31365]: from='client.? 172.21.15.86:0/3266656711' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-17T07:01:22.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:22.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-04-17T07:01:22.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-17T07:01:22.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.934+0000 7fe626000700 1 Processor -- start 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.934+0000 7fe626000700 1 -- start start 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.935+0000 7fe626000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe620111340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.935+0000 7fe626000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe620111910 con 0x7fe62010ef50 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.935+0000 7fe624c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe620111340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.935+0000 7fe624c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe620111340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54032/0 (socket says 172.21.15.86:54032) 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.935+0000 7fe624c00700 1 -- 172.21.15.86:0/2893674034 learned_addr learned my addr 172.21.15.86:0/2893674034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.936+0000 7fe624c00700 1 -- 172.21.15.86:0/2893674034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe620111a90 con 0x7fe62010ef50 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.936+0000 7fe624c00700 1 --2- 172.21.15.86:0/2893674034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe620111340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe6140098a0 tx=0x7fe614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283790268ed40b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.937+0000 7fe61f400700 1 -- 172.21.15.86:0/2893674034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe614004160 con 0x7fe62010ef50 2024-04-17T07:01:22.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.937+0000 7fe61f400700 1 -- 172.21.15.86:0/2893674034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe614004300 con 0x7fe62010ef50 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.937+0000 7fe61f400700 1 -- 172.21.15.86:0/2893674034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6140036a0 con 0x7fe62010ef50 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.937+0000 7fe626000700 1 -- 172.21.15.86:0/2893674034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 msgr2=0x7fe620111340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.937+0000 7fe626000700 1 --2- 172.21.15.86:0/2893674034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe620111340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe6140098a0 tx=0x7fe614009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.937+0000 7fe626000700 1 -- 172.21.15.86:0/2893674034 shutdown_connections 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.937+0000 7fe626000700 1 --2- 172.21.15.86:0/2893674034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe620111340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.937+0000 7fe626000700 1 -- 172.21.15.86:0/2893674034 >> 172.21.15.86:0/2893674034 conn(0x7fe62007da10 msgr2=0x7fe62007fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.938+0000 7fe626000700 1 -- 172.21.15.86:0/2893674034 shutdown_connections 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.938+0000 7fe626000700 1 -- 172.21.15.86:0/2893674034 wait complete. 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.939+0000 7fe626000700 1 Processor -- start 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.939+0000 7fe626000700 1 -- start start 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.939+0000 7fe626000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe6201a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:22.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.939+0000 7fe626000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe620111f60 con 0x7fe62010ef50 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.939+0000 7fe624c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe6201a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.939+0000 7fe624c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe6201a1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54036/0 (socket says 172.21.15.86:54036) 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.939+0000 7fe624c00700 1 -- 172.21.15.86:0/3266656711 learned_addr learned my addr 172.21.15.86:0/3266656711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.940+0000 7fe624c00700 1 -- 172.21.15.86:0/3266656711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6201a2240 con 0x7fe62010ef50 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.940+0000 7fe624c00700 1 --2- 172.21.15.86:0/3266656711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe6201a1d00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe614000c00 tx=0x7fe614004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.940+0000 7fe61d600700 1 -- 172.21.15.86:0/3266656711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe614003db0 con 0x7fe62010ef50 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.940+0000 7fe61d600700 1 -- 172.21.15.86:0/3266656711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe614011630 con 0x7fe62010ef50 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.940+0000 7fe61d600700 1 -- 172.21.15.86:0/3266656711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe614011950 con 0x7fe62010ef50 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.940+0000 7fe626000700 1 -- 172.21.15.86:0/3266656711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6201a2530 con 0x7fe62010ef50 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.941+0000 7fe626000700 1 -- 172.21.15.86:0/3266656711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6201a51b0 con 0x7fe62010ef50 2024-04-17T07:01:22.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.941+0000 7fe61d600700 1 -- 172.21.15.86:0/3266656711 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe614011af0 con 0x7fe62010ef50 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.941+0000 7fe61d600700 1 -- 172.21.15.86:0/3266656711 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe61401c450 con 0x7fe62010ef50 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.942+0000 7fe626000700 1 -- 172.21.15.86:0/3266656711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6200640b0 con 0x7fe62010ef50 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:21.945+0000 7fe61d600700 1 -- 172.21.15.86:0/3266656711 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fe614032020 con 0x7fe62010ef50 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:22.020+0000 7fe626000700 1 -- 172.21.15.86:0/3266656711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe6201a6920 con 0x7fe62010ef50 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:22.027+0000 7fe61d600700 1 -- 172.21.15.86:0/3266656711 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fe614004260 con 0x7fe62010ef50 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:22.028+0000 7fe61d600700 1 -- 172.21.15.86:0/3266656711 <== mon.0 v2:172.21.15.86:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe61401b450 con 0x7fe62010ef50 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:22.029+0000 7fe626000700 1 -- 172.21.15.86:0/3266656711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 msgr2=0x7fe6201a1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:22.029+0000 7fe626000700 1 --2- 172.21.15.86:0/3266656711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe6201a1d00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe614000c00 tx=0x7fe614004750 comp rx=0 tx=0).stop 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:22.029+0000 7fe626000700 1 -- 172.21.15.86:0/3266656711 shutdown_connections 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:22.029+0000 7fe626000700 1 --2- 172.21.15.86:0/3266656711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe62010ef50 0x7fe6201a1d00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:22.029+0000 7fe626000700 1 -- 172.21.15.86:0/3266656711 >> 172.21.15.86:0/3266656711 conn(0x7fe62007da10 msgr2=0x7fe62007f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:22.029+0000 7fe626000700 1 -- 172.21.15.86:0/3266656711 shutdown_connections 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:22.029+0000 7fe626000700 1 -- 172.21.15.86:0/3266656711 wait complete. 2024-04-17T07:01:22.667 INFO:teuthology.orchestra.run.smithi086.stdout:Generating new minimal ceph.conf... 2024-04-17T07:01:23.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:23 smithi086 ceph-mon[31365]: from='client.? 172.21.15.86:0/567291054' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:01:23.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.257+0000 7f2e41600700 1 Processor -- start 2024-04-17T07:01:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.257+0000 7f2e41600700 1 -- start start 2024-04-17T07:01:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.258+0000 7f2e41600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c10a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.258+0000 7f2e41600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c10af60 con 0x7f2e3c10a570 2024-04-17T07:01:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.259+0000 7f2e3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c10a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.259+0000 7f2e3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c10a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54040/0 (socket says 172.21.15.86:54040) 2024-04-17T07:01:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.259+0000 7f2e3aa00700 1 -- 172.21.15.86:0/1046240321 learned_addr learned my addr 172.21.15.86:0/1046240321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.260+0000 7f2e3aa00700 1 -- 172.21.15.86:0/1046240321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c10b720 con 0x7f2e3c10a570 2024-04-17T07:01:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.260+0000 7f2e3aa00700 1 --2- 172.21.15.86:0/1046240321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c10a990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0098a0 tx=0x7f2e2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d3f308aea24515 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.260+0000 7f2e39600700 1 -- 172.21.15.86:0/1046240321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e2c004160 con 0x7f2e3c10a570 2024-04-17T07:01:23.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.260+0000 7f2e39600700 1 -- 172.21.15.86:0/1046240321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2e2c004300 con 0x7f2e3c10a570 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.260+0000 7f2e39600700 1 -- 172.21.15.86:0/1046240321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e2c011a80 con 0x7f2e3c10a570 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.261+0000 7f2e41600700 1 -- 172.21.15.86:0/1046240321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 msgr2=0x7f2e3c10a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.261+0000 7f2e41600700 1 --2- 172.21.15.86:0/1046240321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c10a990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0098a0 tx=0x7f2e2c009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.261+0000 7f2e41600700 1 -- 172.21.15.86:0/1046240321 shutdown_connections 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.261+0000 7f2e41600700 1 --2- 172.21.15.86:0/1046240321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c10a990 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.261+0000 7f2e41600700 1 -- 172.21.15.86:0/1046240321 >> 172.21.15.86:0/1046240321 conn(0x7f2e3c07dc10 msgr2=0x7f2e3c1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.261+0000 7f2e41600700 1 -- 172.21.15.86:0/1046240321 shutdown_connections 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.261+0000 7f2e41600700 1 -- 172.21.15.86:0/1046240321 wait complete. 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.262+0000 7f2e41600700 1 Processor -- start 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.263+0000 7f2e41600700 1 -- start start 2024-04-17T07:01:23.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.263+0000 7f2e41600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.263+0000 7f2e41600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c10ead0 con 0x7f2e3c10a570 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.263+0000 7f2e3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.263+0000 7f2e3aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c19e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54042/0 (socket says 172.21.15.86:54042) 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.263+0000 7f2e3aa00700 1 -- 172.21.15.86:0/567291054 learned_addr learned my addr 172.21.15.86:0/567291054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.264+0000 7f2e3aa00700 1 -- 172.21.15.86:0/567291054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c19e930 con 0x7f2e3c10a570 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.264+0000 7f2e3aa00700 1 --2- 172.21.15.86:0/567291054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c19e3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c000c00 tx=0x7f2e2c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.264+0000 7f2e33400700 1 -- 172.21.15.86:0/567291054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e2c0039c0 con 0x7f2e3c10a570 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.264+0000 7f2e33400700 1 -- 172.21.15.86:0/567291054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2e2c003b60 con 0x7f2e3c10a570 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.264+0000 7f2e33400700 1 -- 172.21.15.86:0/567291054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e2c01a6f0 con 0x7f2e3c10a570 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.264+0000 7f2e41600700 1 -- 172.21.15.86:0/567291054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3c19ebc0 con 0x7f2e3c10a570 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.264+0000 7f2e41600700 1 -- 172.21.15.86:0/567291054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3c1a18b0 con 0x7f2e3c10a570 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.265+0000 7f2e33400700 1 -- 172.21.15.86:0/567291054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e2c01ac20 con 0x7f2e3c10a570 2024-04-17T07:01:23.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.265+0000 7f2e33400700 1 -- 172.21.15.86:0/567291054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2e2c019700 con 0x7f2e3c10a570 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.266+0000 7f2e41600700 1 -- 172.21.15.86:0/567291054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3c0640b0 con 0x7f2e3c10a570 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.268+0000 7f2e33400700 1 -- 172.21.15.86:0/567291054 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f2e2c004300 con 0x7f2e3c10a570 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.346+0000 7f2e41600700 1 -- 172.21.15.86:0/567291054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f2e3c1a1a80 con 0x7f2e3c10a570 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.346+0000 7f2e33400700 1 -- 172.21.15.86:0/567291054 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f2e2c024a40 con 0x7f2e3c10a570 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.348+0000 7f2e41600700 1 -- 172.21.15.86:0/567291054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 msgr2=0x7f2e3c19e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.348+0000 7f2e41600700 1 --2- 172.21.15.86:0/567291054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c19e3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c000c00 tx=0x7f2e2c004290 comp rx=0 tx=0).stop 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.348+0000 7f2e41600700 1 -- 172.21.15.86:0/567291054 shutdown_connections 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.348+0000 7f2e41600700 1 --2- 172.21.15.86:0/567291054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c10a570 0x7f2e3c19e3f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.348+0000 7f2e41600700 1 -- 172.21.15.86:0/567291054 >> 172.21.15.86:0/567291054 conn(0x7f2e3c07dc10 msgr2=0x7f2e3c107c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.348+0000 7f2e41600700 1 -- 172.21.15.86:0/567291054 shutdown_connections 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:23.348+0000 7f2e41600700 1 -- 172.21.15.86:0/567291054 wait complete. 2024-04-17T07:01:23.972 INFO:teuthology.orchestra.run.smithi086.stdout:Restarting the monitor... 2024-04-17T07:01:24.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:24 smithi086 systemd[1]: Stopping Ceph mon.a for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:01:24.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a[31361]: 2024-04-17T07:01:24.465+0000 7f1c2a200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:01:24.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a[31361]: 2024-04-17T07:01:24.465+0000 7f1c2a200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-17T07:01:24.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:24 smithi086 podman[31609]: 2024-04-17 07:01:24.580029321 +0000 UTC m=+0.259724204 container died ec8bfdf036663d9f7584124376a52bd2b125ac9ccc94f9d2cdc5168156b7424d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-17T07:01:25.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:24 smithi086 podman[31609]: 2024-04-17 07:01:24.984667503 +0000 UTC m=+0.664362411 container cleanup ec8bfdf036663d9f7584124376a52bd2b125ac9ccc94f9d2cdc5168156b7424d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-17T07:01:25.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:24 smithi086 bash[31609]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a 2024-04-17T07:01:25.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:25 smithi086 podman[31623]: 2024-04-17 07:01:25.160091776 +0000 UTC m=+0.562091733 container remove ec8bfdf036663d9f7584124376a52bd2b125ac9ccc94f9d2cdc5168156b7424d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-17T07:01:26.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:25 smithi086 systemd[1]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-17T07:01:26.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:25 smithi086 systemd[1]: Stopped Ceph mon.a for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:01:26.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:25 smithi086 systemd[1]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.a.service: Consumed 1.945s CPU time. 2024-04-17T07:01:26.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:25 smithi086 systemd[1]: Starting Ceph mon.a for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:01:26.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 podman[31714]: 2024-04-17 07:01:26.017036001 +0000 UTC m=+0.035114744 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:01:26.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 podman[31714]: 2024-04-17 07:01:26.127799487 +0000 UTC m=+0.145878264 container create c352ceee4038beb26a994da8880f0ab0d213c1f1e1dac4497541958f6e1fb122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:01:26.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 podman[31714]: 2024-04-17 07:01:26.389574904 +0000 UTC m=+0.407653688 container init c352ceee4038beb26a994da8880f0ab0d213c1f1e1dac4497541958f6e1fb122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-17T07:01:26.488 INFO:teuthology.orchestra.run.smithi086.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-17T07:01:26.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 podman[31714]: 2024-04-17 07:01:26.400493858 +0000 UTC m=+0.418572606 container start c352ceee4038beb26a994da8880f0ab0d213c1f1e1dac4497541958f6e1fb122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-17T07:01:26.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: set uid:gid to 167:167 (ceph:ceph) 2024-04-17T07:01:26.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-17T07:01:26.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: pidfile_write: ignore empty --pid-file 2024-04-17T07:01:26.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: load: jerasure load: lrc 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: RocksDB version: 7.9.2 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Git sha 0 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: DB SUMMARY 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: DB Session ID: FZP3WRUKRSHFWTYZAPO0 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: CURRENT file: CURRENT 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: IDENTITY file: IDENTITY 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104684 ; 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.error_if_exists: 0 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.create_if_missing: 0 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.paranoid_checks: 1 2024-04-17T07:01:26.648 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.env: 0x559ec53efcc0 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.fs: PosixFileSystem 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.info_log: 0x559ed9bebc60 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_file_opening_threads: 16 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.statistics: (nil) 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.use_fsync: 0 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_log_file_size: 0 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.keep_log_file_num: 1000 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.recycle_log_file_num: 0 2024-04-17T07:01:26.649 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.allow_fallocate: 1 2024-04-17T07:01:26.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.allow_mmap_reads: 0 2024-04-17T07:01:26.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.allow_mmap_writes: 0 2024-04-17T07:01:26.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.use_direct_reads: 0 2024-04-17T07:01:26.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-17T07:01:26.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.create_missing_column_families: 0 2024-04-17T07:01:26.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.db_log_dir: 2024-04-17T07:01:26.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.wal_dir: 2024-04-17T07:01:26.651 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.advise_random_on_open: 1 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.db_write_buffer_size: 0 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.write_buffer_manager: 0x559ed9bba1e0 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.rate_limiter: (nil) 2024-04-17T07:01:26.652 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.wal_recovery_mode: 2 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.enable_thread_tracking: 0 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.enable_pipelined_write: 0 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.unordered_write: 0 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.row_cache: None 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.wal_filter: None 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.allow_ingest_behind: 0 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.two_write_queues: 0 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.manual_wal_flush: 0 2024-04-17T07:01:26.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.wal_compression: 0 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.atomic_flush: 0 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.log_readahead_size: 0 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.best_efforts_recovery: 0 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.allow_data_in_errors: 0 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.db_host_id: __hostname__ 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_background_jobs: 2 2024-04-17T07:01:26.654 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_background_compactions: -1 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_subcompactions: 1 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_total_wal_size: 0 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_open_files: -1 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bytes_per_sync: 0 2024-04-17T07:01:26.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_readahead_size: 0 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_background_flushes: -1 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Compression algorithms supported: 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: kZSTD supported: 0 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: kXpressCompression supported: 0 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: kLZ4HCCompression supported: 1 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: kZlibCompression supported: 1 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: kSnappyCompression supported: 1 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: kLZ4Compression supported: 1 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: kBZip2Compression supported: 0 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-17T07:01:26.656 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.merge_operator: 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_filter: None 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_filter_factory: None 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.sst_partitioner_factory: None 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559ed9bebe20) 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-17T07:01:26.657 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_type: 0 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_index_type: 0 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_shortening: 1 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: checksum: 4 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: no_block_cache: 0 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache: 0x559ed9bed090 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_options: 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: capacity : 536870912 2024-04-17T07:01:26.658 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_shard_bits : 4 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: strict_capacity_limit : 0 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_compressed: (nil) 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: persistent_cache: (nil) 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size: 4096 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size_deviation: 10 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_restart_interval: 16 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_block_restart_interval: 1 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: metadata_block_size: 4096 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: partition_filters: 0 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: use_delta_encoding: 1 2024-04-17T07:01:26.659 INFO:journalctl@ceph.mon.a.smithi086.stdout: filter_policy: bloomfilter 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout: whole_key_filtering: 1 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout: verify_compression: 0 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout: format_version: 5 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_index_compression: 1 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_align: 0 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout: max_auto_readahead_size: 262144 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout: prepopulate_block_cache: 0 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout: initial_auto_readahead_size: 8192 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.write_buffer_size: 33554432 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_write_buffer_number: 2 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compression: NoCompression 2024-04-17T07:01:26.660 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bottommost_compression: Disabled 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.prefix_extractor: nullptr 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.num_levels: 7 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-17T07:01:26.661 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compression_opts.level: 32767 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compression_opts.strategy: 0 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compression_opts.enabled: false 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-17T07:01:26.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.target_file_size_base: 67108864 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.arena_block_size: 1048576 2024-04-17T07:01:26.663 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.disable_auto_compactions: 0 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.inplace_update_support: 0 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-17T07:01:26.664 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.bloom_locality: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.max_successive_merges: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.paranoid_file_checks: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.force_consistency_checks: 1 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.report_bg_io_stats: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.ttl: 2592000 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.enable_blob_files: false 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.min_blob_size: 0 2024-04-17T07:01:26.665 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.blob_file_size: 268435456 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.blob_file_starting_level: 0 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 041e55b4-7a48-4892-ad3c-084768fb49bc 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713337286454080, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713337286456041, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99887, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97769, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14060, "raw_average_key_size": 49, "raw_value_size": 90119, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713337286, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "041e55b4-7a48-4892-ad3c-084768fb49bc", "db_session_id": "FZP3WRUKRSHFWTYZAPO0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-17T07:01:26.666 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713337286456199, "job": 1, "event": "recovery_finished"} 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559ed9cda000 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: DB pointer 0x559ed9cc4000 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** DB Stats ** 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-17T07:01:26.667 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: L0 2/0 99.41 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: Sum 2/0 99.41 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 80.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 80.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-17T07:01:26.668 INFO:journalctl@ceph.mon.a.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative compaction: 0.00 GB write, 14.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval compaction: 0.00 GB write, 14.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache BinnedLRUCache@0x559ed9bed090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-17T07:01:26.669 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: starting mon.a rank 0 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: mon.a@-1(???) e1 preinit fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: mon.a@-1(???).mds e1 new map 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: mon.a@-1(???).mds e1 print_map 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout: e1 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout: legacy client fscid: -1 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout: No filesystems configured 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:01:26.670 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 bash[31714]: c352ceee4038beb26a994da8880f0ab0d213c1f1e1dac4497541958f6e1fb122 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 systemd[1]: Started Ceph mon.a for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: monmap epoch 1 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: last_changed 2024-04-17T07:01:16.008725+0000 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: min_mon_release 19 (squid) 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: election_strategy: 1 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: fsmap 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: osdmap e1: 0 total, 0 up, 0 in 2024-04-17T07:01:26.671 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:26 smithi086 ceph-mon[31733]: mgrmap e1: no daemons active 2024-04-17T07:01:27.963 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.159+0000 7f8cb5c00700 1 Processor -- start 2024-04-17T07:01:27.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.159+0000 7f8cb5c00700 1 -- start start 2024-04-17T07:01:27.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.160+0000 7f8cb5c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:27.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.160+0000 7f8cb5c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb007c840 con 0x7f8cb007dd90 2024-04-17T07:01:27.964 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.161+0000 7f8caea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:27.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.161+0000 7f8caea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47400/0 (socket says 172.21.15.86:47400) 2024-04-17T07:01:27.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.161+0000 7f8caea00700 1 -- 172.21.15.86:0/2952752750 learned_addr learned my addr 172.21.15.86:0/2952752750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:27.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.161+0000 7f8caea00700 1 -- 172.21.15.86:0/2952752750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb007c9c0 con 0x7f8cb007dd90 2024-04-17T07:01:27.965 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.162+0000 7f8caea00700 1 --2- 172.21.15.86:0/2952752750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb007c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00098a0 tx=0x7f8ca0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c205416ac86a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:27.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.162+0000 7f8cad600700 1 -- 172.21.15.86:0/2952752750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ca0004160 con 0x7f8cb007dd90 2024-04-17T07:01:27.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.162+0000 7f8cad600700 1 -- 172.21.15.86:0/2952752750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8ca0004300 con 0x7f8cb007dd90 2024-04-17T07:01:27.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.162+0000 7f8cad600700 1 -- 172.21.15.86:0/2952752750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ca0011a80 con 0x7f8cb007dd90 2024-04-17T07:01:27.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.163+0000 7f8cb5c00700 1 -- 172.21.15.86:0/2952752750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 msgr2=0x7f8cb007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:27.966 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.163+0000 7f8cb5c00700 1 --2- 172.21.15.86:0/2952752750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb007c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00098a0 tx=0x7f8ca0009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:27.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.163+0000 7f8cb5c00700 1 -- 172.21.15.86:0/2952752750 shutdown_connections 2024-04-17T07:01:27.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.163+0000 7f8cb5c00700 1 --2- 172.21.15.86:0/2952752750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb007c300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:27.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.163+0000 7f8cb5c00700 1 -- 172.21.15.86:0/2952752750 >> 172.21.15.86:0/2952752750 conn(0x7f8cb01039b0 msgr2=0x7f8cb0105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:27.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.163+0000 7f8cb5c00700 1 -- 172.21.15.86:0/2952752750 shutdown_connections 2024-04-17T07:01:27.967 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.163+0000 7f8cb5c00700 1 -- 172.21.15.86:0/2952752750 wait complete. 2024-04-17T07:01:27.968 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.164+0000 7f8cb5c00700 1 Processor -- start 2024-04-17T07:01:27.968 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.165+0000 7f8cb5c00700 1 -- start start 2024-04-17T07:01:27.968 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.165+0000 7f8cb5c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:27.969 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.165+0000 7f8cb5c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb010e7c0 con 0x7f8cb007dd90 2024-04-17T07:01:27.969 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.165+0000 7f8caea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:27.969 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.165+0000 7f8caea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb019e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47416/0 (socket says 172.21.15.86:47416) 2024-04-17T07:01:27.969 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.165+0000 7f8caea00700 1 -- 172.21.15.86:0/417361849 learned_addr learned my addr 172.21.15.86:0/417361849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:27.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.166+0000 7f8caea00700 1 -- 172.21.15.86:0/417361849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb019e8d0 con 0x7f8cb007dd90 2024-04-17T07:01:27.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.166+0000 7f8caea00700 1 --2- 172.21.15.86:0/417361849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb019e390 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0006af0 tx=0x7f8ca0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:27.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.166+0000 7f8ca7400700 1 -- 172.21.15.86:0/417361849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ca0003930 con 0x7f8cb007dd90 2024-04-17T07:01:27.970 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.166+0000 7f8ca7400700 1 -- 172.21.15.86:0/417361849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8ca0003ad0 con 0x7f8cb007dd90 2024-04-17T07:01:27.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.166+0000 7f8cb5c00700 1 -- 172.21.15.86:0/417361849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb019eb60 con 0x7f8cb007dd90 2024-04-17T07:01:27.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.166+0000 7f8ca7400700 1 -- 172.21.15.86:0/417361849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ca00195e0 con 0x7f8cb007dd90 2024-04-17T07:01:27.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.166+0000 7f8cb5c00700 1 -- 172.21.15.86:0/417361849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb01a1850 con 0x7f8cb007dd90 2024-04-17T07:01:27.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.167+0000 7f8ca7400700 1 -- 172.21.15.86:0/417361849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8ca0019c70 con 0x7f8cb007dd90 2024-04-17T07:01:27.971 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.167+0000 7f8ca7400700 1 -- 172.21.15.86:0/417361849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8ca0024730 con 0x7f8cb007dd90 2024-04-17T07:01:27.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.168+0000 7f8cb5c00700 1 -- 172.21.15.86:0/417361849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb00640b0 con 0x7f8cb007dd90 2024-04-17T07:01:27.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.171+0000 7f8ca7400700 1 -- 172.21.15.86:0/417361849 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f8ca002d440 con 0x7f8cb007dd90 2024-04-17T07:01:27.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.246+0000 7f8cb5c00700 1 -- 172.21.15.86:0/417361849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f8cb01a1e60 con 0x7f8cb007dd90 2024-04-17T07:01:27.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.249+0000 7f8ca7400700 1 -- 172.21.15.86:0/417361849 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f8ca002d620 con 0x7f8cb007dd90 2024-04-17T07:01:27.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.251+0000 7f8cb5c00700 1 -- 172.21.15.86:0/417361849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 msgr2=0x7f8cb019e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:27.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.251+0000 7f8cb5c00700 1 --2- 172.21.15.86:0/417361849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb019e390 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0006af0 tx=0x7f8ca0011b70 comp rx=0 tx=0).stop 2024-04-17T07:01:27.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.252+0000 7f8cb5c00700 1 -- 172.21.15.86:0/417361849 shutdown_connections 2024-04-17T07:01:27.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.252+0000 7f8cb5c00700 1 --2- 172.21.15.86:0/417361849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cb007dd90 0x7f8cb019e390 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:27.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.252+0000 7f8cb5c00700 1 -- 172.21.15.86:0/417361849 >> 172.21.15.86:0/417361849 conn(0x7f8cb01039b0 msgr2=0x7f8cb0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:27.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.252+0000 7f8cb5c00700 1 -- 172.21.15.86:0/417361849 shutdown_connections 2024-04-17T07:01:27.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:27.252+0000 7f8cb5c00700 1 -- 172.21.15.86:0/417361849 wait complete. 2024-04-17T07:01:27.974 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-17T07:01:28.001 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:01:28.002 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mgr... 2024-04-17T07:01:28.004 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-17T07:01:28.005 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-17T07:01:28.410 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.x 2024-04-17T07:01:28.410 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.x.service: Unit ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.x.service not loaded. 2024-04-17T07:01:28.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:28 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/417361849' entity='client.admin' 2024-04-17T07:01:28.866 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968.target.wants/ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@.service. 2024-04-17T07:01:29.247 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:28 smithi086 systemd[1]: Starting Ceph mgr.x for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:01:29.580 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:29 smithi086 podman[31955]: 2024-04-17 07:01:29.150770049 +0000 UTC m=+0.032522502 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-17T07:01:29.580 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:29 smithi086 podman[31955]: 2024-04-17 07:01:29.247405734 +0000 UTC m=+0.129158246 container create e4f189e0cff0f8b221bf89b8875d6315a7d6c33a207f0363e747a41f80f1034c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:01:29.580 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:29 smithi086 podman[31955]: 2024-04-17 07:01:29.485164391 +0000 UTC m=+0.366916923 container init e4f189e0cff0f8b221bf89b8875d6315a7d6c33a207f0363e747a41f80f1034c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-17T07:01:29.580 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:29 smithi086 podman[31955]: 2024-04-17 07:01:29.496267211 +0000 UTC m=+0.378019672 container start e4f189e0cff0f8b221bf89b8875d6315a7d6c33a207f0363e747a41f80f1034c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-17T07:01:29.634 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T07:01:29.635 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-17T07:01:29.653 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T07:01:29.653 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-17T07:01:29.653 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-17T07:01:29.653 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-04-17T07:01:29.682 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T07:01:29.682 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-17T07:01:29.703 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T07:01:29.704 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-17T07:01:29.704 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-17T07:01:29.704 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-17T07:01:29.704 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr to start... 2024-04-17T07:01:29.704 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr... 2024-04-17T07:01:29.862 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:29 smithi086 bash[31955]: e4f189e0cff0f8b221bf89b8875d6315a7d6c33a207f0363e747a41f80f1034c 2024-04-17T07:01:29.862 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:29 smithi086 systemd[1]: Started Ceph mgr.x for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:01:30.456 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:30.269+0000 7ff22ede6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T07:01:30.456 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:30.395+0000 7ff22ede6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T07:01:30.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:30 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/245016584' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T07:01:31.132 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:31.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:01:31.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:01:31.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T07:01:31.133 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T07:01:31.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T07:01:31.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T07:01:31.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:31.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T07:01:31.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T07:01:31.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-17T07:01:31.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:31.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T07:01:31.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T07:01:31.136 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:31.137 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T07:01:31.139 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T07:01:31.139 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T07:01:31.139 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T07:01:31.139 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T07:01:31.139 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T07:01:31.139 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T07:01:31.139 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T07:01:31.139 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T07:01:31.139 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:31.139 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T07:01:31.146 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:31.146 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T07:01:31.146 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T07:01:31.146 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:31.146 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T07:01:31.146 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-17T07:01:31.146 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T07:01:31.146 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T07:01:31.146 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T07:01:31.146 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T07:01:19.941840+0000", 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.348+0000 7f343fe00700 1 Processor -- start 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.348+0000 7f343fe00700 1 -- start start 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.349+0000 7f343fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.349+0000 7f343fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f344007c8f0 con 0x7f344007de70 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.349+0000 7f343ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.350+0000 7f343ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47454/0 (socket says 172.21.15.86:47454) 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.350+0000 7f343ea00700 1 -- 172.21.15.86:0/2646518153 learned_addr learned my addr 172.21.15.86:0/2646518153 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.350+0000 7f343ea00700 1 -- 172.21.15.86:0/2646518153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344007ca70 con 0x7f344007de70 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.350+0000 7f343ea00700 1 --2- 172.21.15.86:0/2646518153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344007c320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f34300098a0 tx=0x7f3430009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56319f44ced9b7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.351+0000 7f343d600700 1 -- 172.21.15.86:0/2646518153 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3430004160 con 0x7f344007de70 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.351+0000 7f343d600700 1 -- 172.21.15.86:0/2646518153 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3430004300 con 0x7f344007de70 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.351+0000 7f343d600700 1 -- 172.21.15.86:0/2646518153 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3430011a80 con 0x7f344007de70 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.351+0000 7f343fe00700 1 -- 172.21.15.86:0/2646518153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 msgr2=0x7f344007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.351+0000 7f343fe00700 1 --2- 172.21.15.86:0/2646518153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344007c320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f34300098a0 tx=0x7f3430009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.352+0000 7f343fe00700 1 -- 172.21.15.86:0/2646518153 shutdown_connections 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.352+0000 7f343fe00700 1 --2- 172.21.15.86:0/2646518153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344007c320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.352+0000 7f343fe00700 1 -- 172.21.15.86:0/2646518153 >> 172.21.15.86:0/2646518153 conn(0x7f34401037d0 msgr2=0x7f3440105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:31.148 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.352+0000 7f343fe00700 1 -- 172.21.15.86:0/2646518153 shutdown_connections 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.352+0000 7f343fe00700 1 -- 172.21.15.86:0/2646518153 wait complete. 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.353+0000 7f343fe00700 1 Processor -- start 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.353+0000 7f343fe00700 1 -- start start 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.353+0000 7f343fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344019e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.353+0000 7f343fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f344010e6e0 con 0x7f344007de70 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.353+0000 7f343ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344019e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.353+0000 7f343ea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344019e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47460/0 (socket says 172.21.15.86:47460) 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.353+0000 7f343ea00700 1 -- 172.21.15.86:0/245016584 learned_addr learned my addr 172.21.15.86:0/245016584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.354+0000 7f343ea00700 1 -- 172.21.15.86:0/245016584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344019e720 con 0x7f344007de70 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.354+0000 7f343ea00700 1 --2- 172.21.15.86:0/245016584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344019e1e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3430011a50 tx=0x7f3430004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.354+0000 7f3444a00700 1 -- 172.21.15.86:0/245016584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34300039f0 con 0x7f344007de70 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.354+0000 7f3444a00700 1 -- 172.21.15.86:0/245016584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3430003b90 con 0x7f344007de70 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.354+0000 7f3444a00700 1 -- 172.21.15.86:0/245016584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f343001a600 con 0x7f344007de70 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.355+0000 7f343fe00700 1 -- 172.21.15.86:0/245016584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f344019e9b0 con 0x7f344007de70 2024-04-17T07:01:31.149 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.355+0000 7f343fe00700 1 -- 172.21.15.86:0/245016584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34401a16a0 con 0x7f344007de70 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.355+0000 7f3444a00700 1 -- 172.21.15.86:0/245016584 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f343001ac40 con 0x7f344007de70 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.355+0000 7f3444a00700 1 -- 172.21.15.86:0/245016584 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3430019700 con 0x7f344007de70 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.356+0000 7f343fe00700 1 -- 172.21.15.86:0/245016584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34400640b0 con 0x7f344007de70 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.358+0000 7f3444a00700 1 -- 172.21.15.86:0/245016584 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f3430024a40 con 0x7f344007de70 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.437+0000 7f343fe00700 1 -- 172.21.15.86:0/245016584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f34401a1bf0 con 0x7f344007de70 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.438+0000 7f3444a00700 1 -- 172.21.15.86:0/245016584 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3430024440 con 0x7f344007de70 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.439+0000 7f343fe00700 1 -- 172.21.15.86:0/245016584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 msgr2=0x7f344019e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.439+0000 7f343fe00700 1 --2- 172.21.15.86:0/245016584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344019e1e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3430011a50 tx=0x7f3430004290 comp rx=0 tx=0).stop 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.439+0000 7f343fe00700 1 -- 172.21.15.86:0/245016584 shutdown_connections 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.439+0000 7f343fe00700 1 --2- 172.21.15.86:0/245016584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344007de70 0x7f344019e1e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.439+0000 7f343fe00700 1 -- 172.21.15.86:0/245016584 >> 172.21.15.86:0/245016584 conn(0x7f34401037d0 msgr2=0x7f3440104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.439+0000 7f343fe00700 1 -- 172.21.15.86:0/245016584 shutdown_connections 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:30.439+0000 7f343fe00700 1 -- 172.21.15.86:0/245016584 wait complete. 2024-04-17T07:01:31.150 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (1/15)... 2024-04-17T07:01:31.557 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:31.248+0000 7ff22ede6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T07:01:31.557 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:31.369+0000 7ff22ede6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T07:01:31.557 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:31.556+0000 7ff22ede6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T07:01:31.830 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:31.680+0000 7ff22ede6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T07:01:31.830 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:31.789+0000 7ff22ede6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T07:01:32.386 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:32.133+0000 7ff22ede6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T07:01:32.386 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:32.273+0000 7ff22ede6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T07:01:32.739 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:32.385+0000 7ff22ede6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T07:01:32.739 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:32.508+0000 7ff22ede6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T07:01:33.580 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:32.738+0000 7ff22ede6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T07:01:34.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:33 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/529119250' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T07:01:34.260 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:34 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:34.084+0000 7ff22ede6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T07:01:34.260 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:34 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:34.260+0000 7ff22ede6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T07:01:34.508 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:34.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:01:34.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:01:34.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T07:01:34.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T07:01:34.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T07:01:34.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T07:01:34.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:34.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T07:01:34.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T07:01:34.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-17T07:01:34.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:34.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T07:01:34.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T07:01:34.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:34.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-17T07:01:34.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T07:01:34.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:34.512 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T07:01:34.513 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T07:01:34.513 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:34.514 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T07:01:34.514 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:34.514 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T07:01:34.514 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T07:01:34.514 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T07:01:34.514 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T07:01:34.514 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T07:01:34.514 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T07:01:34.514 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:34.515 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T07:01:34.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T07:01:34.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T07:01:34.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T07:01:34.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T07:01:34.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T07:01:34.516 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T07:01:34.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T07:01:34.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T07:01:34.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:34.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T07:01:34.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:34.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T07:01:34.517 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T07:01:34.518 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T07:01:34.519 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:34.521 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T07:01:19.941840+0000", 2024-04-17T07:01:34.521 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:01:34.521 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:34.521 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T07:01:34.521 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:01:34.521 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.799+0000 7f7536e00700 1 Processor -- start 2024-04-17T07:01:34.521 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.800+0000 7f7536e00700 1 -- start start 2024-04-17T07:01:34.522 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.800+0000 7f7536e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f7530108750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:34.522 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.800+0000 7f7536e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7530108d20 con 0x7f7530108330 2024-04-17T07:01:34.522 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.800+0000 7f752fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f7530108750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:34.522 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.801+0000 7f752fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f7530108750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47474/0 (socket says 172.21.15.86:47474) 2024-04-17T07:01:34.522 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.801+0000 7f752fe00700 1 -- 172.21.15.86:0/2250677766 learned_addr learned my addr 172.21.15.86:0/2250677766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:34.522 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.801+0000 7f752fe00700 1 -- 172.21.15.86:0/2250677766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75301094e0 con 0x7f7530108330 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.801+0000 7f752fe00700 1 --2- 172.21.15.86:0/2250677766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f7530108750 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f75200098a0 tx=0x7f7520009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e271edd3c1cb24c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.801+0000 7f752f400700 1 -- 172.21.15.86:0/2250677766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7520004160 con 0x7f7530108330 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.802+0000 7f752f400700 1 -- 172.21.15.86:0/2250677766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7520004300 con 0x7f7530108330 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.802+0000 7f752f400700 1 -- 172.21.15.86:0/2250677766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7520011a80 con 0x7f7530108330 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.802+0000 7f7536e00700 1 -- 172.21.15.86:0/2250677766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 msgr2=0x7f7530108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.802+0000 7f7536e00700 1 --2- 172.21.15.86:0/2250677766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f7530108750 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f75200098a0 tx=0x7f7520009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.802+0000 7f7536e00700 1 -- 172.21.15.86:0/2250677766 shutdown_connections 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.802+0000 7f7536e00700 1 --2- 172.21.15.86:0/2250677766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f7530108750 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.803+0000 7f7536e00700 1 -- 172.21.15.86:0/2250677766 >> 172.21.15.86:0/2250677766 conn(0x7f75301039b0 msgr2=0x7f7530105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.803+0000 7f7536e00700 1 -- 172.21.15.86:0/2250677766 shutdown_connections 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.803+0000 7f7536e00700 1 -- 172.21.15.86:0/2250677766 wait complete. 2024-04-17T07:01:34.524 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.804+0000 7f7536e00700 1 Processor -- start 2024-04-17T07:01:34.525 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.804+0000 7f7536e00700 1 -- start start 2024-04-17T07:01:34.525 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.804+0000 7f7536e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f753019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:34.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.804+0000 7f7536e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f753010c890 con 0x7f7530108330 2024-04-17T07:01:34.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.804+0000 7f752fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f753019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:34.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.804+0000 7f752fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f753019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47480/0 (socket says 172.21.15.86:47480) 2024-04-17T07:01:34.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.804+0000 7f752fe00700 1 -- 172.21.15.86:0/529119250 learned_addr learned my addr 172.21.15.86:0/529119250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:34.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.805+0000 7f752fe00700 1 -- 172.21.15.86:0/529119250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753019e920 con 0x7f7530108330 2024-04-17T07:01:34.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.805+0000 7f752fe00700 1 --2- 172.21.15.86:0/529119250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f753019e3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7520000c00 tx=0x7f7520004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:34.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.805+0000 7f752d600700 1 -- 172.21.15.86:0/529119250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75200039c0 con 0x7f7530108330 2024-04-17T07:01:34.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.805+0000 7f752d600700 1 -- 172.21.15.86:0/529119250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7520003b60 con 0x7f7530108330 2024-04-17T07:01:34.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.805+0000 7f7536e00700 1 -- 172.21.15.86:0/529119250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753019ebb0 con 0x7f7530108330 2024-04-17T07:01:34.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.805+0000 7f7536e00700 1 -- 172.21.15.86:0/529119250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75301a18a0 con 0x7f7530108330 2024-04-17T07:01:34.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.806+0000 7f752d600700 1 -- 172.21.15.86:0/529119250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f752001a6f0 con 0x7f7530108330 2024-04-17T07:01:34.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.806+0000 7f752d600700 1 -- 172.21.15.86:0/529119250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f752001ace0 con 0x7f7530108330 2024-04-17T07:01:34.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.806+0000 7f752d600700 1 -- 172.21.15.86:0/529119250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7520019700 con 0x7f7530108330 2024-04-17T07:01:34.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.807+0000 7f7536e00700 1 -- 172.21.15.86:0/529119250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750c005360 con 0x7f7530108330 2024-04-17T07:01:34.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.809+0000 7f752d600700 1 -- 172.21.15.86:0/529119250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f7520024a40 con 0x7f7530108330 2024-04-17T07:01:34.528 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.886+0000 7f7536e00700 1 -- 172.21.15.86:0/529119250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f750c005920 con 0x7f7530108330 2024-04-17T07:01:34.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.886+0000 7f752d600700 1 -- 172.21.15.86:0/529119250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7520024440 con 0x7f7530108330 2024-04-17T07:01:34.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.887+0000 7f7536e00700 1 -- 172.21.15.86:0/529119250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 msgr2=0x7f753019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:34.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.887+0000 7f7536e00700 1 --2- 172.21.15.86:0/529119250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f753019e3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7520000c00 tx=0x7f7520004290 comp rx=0 tx=0).stop 2024-04-17T07:01:34.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.887+0000 7f7536e00700 1 -- 172.21.15.86:0/529119250 shutdown_connections 2024-04-17T07:01:34.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.887+0000 7f7536e00700 1 --2- 172.21.15.86:0/529119250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7530108330 0x7f753019e3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:34.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.887+0000 7f7536e00700 1 -- 172.21.15.86:0/529119250 >> 172.21.15.86:0/529119250 conn(0x7f75301039b0 msgr2=0x7f7530104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:34.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.887+0000 7f7536e00700 1 -- 172.21.15.86:0/529119250 shutdown_connections 2024-04-17T07:01:34.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:33.887+0000 7f7536e00700 1 -- 172.21.15.86:0/529119250 wait complete. 2024-04-17T07:01:34.530 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (2/15)... 2024-04-17T07:01:35.060 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:34 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:34.685+0000 7ff22ede6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T07:01:35.060 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:34 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:34.802+0000 7ff22ede6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T07:01:35.330 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:35.060+0000 7ff22ede6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:01:35.330 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:35.301+0000 7ff22ede6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T07:01:35.994 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:35.679+0000 7ff22ede6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:01:35.994 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:35.994+0000 7ff22ede6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T07:01:36.579 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:36 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:36.299+0000 7ff22ede6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T07:01:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:37 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2839277079' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T07:01:37.688 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:37 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:37.372+0000 7ff22ede6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T07:01:37.865 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T07:01:37.866 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T07:01:37.867 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T07:01:37.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T07:01:37.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T07:01:37.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T07:01:37.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T07:01:37.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:37.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T07:01:37.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T07:01:37.875 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:37.876 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T07:01:19.941840+0000", 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:37.877 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.064+0000 7fbcf3400700 1 Processor -- start 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.064+0000 7fbcf3400700 1 -- start start 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.064+0000 7fbcf3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec108850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.064+0000 7fbcf3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec108e20 con 0x7fbcec108430 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.065+0000 7fbcf0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.065+0000 7fbcf0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec108850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55160/0 (socket says 172.21.15.86:55160) 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.065+0000 7fbcf0a00700 1 -- 172.21.15.86:0/495050294 learned_addr learned my addr 172.21.15.86:0/495050294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.065+0000 7fbcf0a00700 1 -- 172.21.15.86:0/495050294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec1095e0 con 0x7fbcec108430 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.066+0000 7fbcf0a00700 1 --2- 172.21.15.86:0/495050294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec108850 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbce00098a0 tx=0x7fbce0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc230e63929df5f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.066+0000 7fbceb400700 1 -- 172.21.15.86:0/495050294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce0004160 con 0x7fbcec108430 2024-04-17T07:01:37.878 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.066+0000 7fbceb400700 1 -- 172.21.15.86:0/495050294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbce0004300 con 0x7fbcec108430 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.066+0000 7fbceb400700 1 -- 172.21.15.86:0/495050294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce0011a80 con 0x7fbcec108430 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.067+0000 7fbcf3400700 1 -- 172.21.15.86:0/495050294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 msgr2=0x7fbcec108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.067+0000 7fbcf3400700 1 --2- 172.21.15.86:0/495050294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec108850 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbce00098a0 tx=0x7fbce0009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.067+0000 7fbcf3400700 1 -- 172.21.15.86:0/495050294 shutdown_connections 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.067+0000 7fbcf3400700 1 --2- 172.21.15.86:0/495050294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec108850 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.067+0000 7fbcf3400700 1 -- 172.21.15.86:0/495050294 >> 172.21.15.86:0/495050294 conn(0x7fbcec103a90 msgr2=0x7fbcec105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.067+0000 7fbcf3400700 1 -- 172.21.15.86:0/495050294 shutdown_connections 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.067+0000 7fbcf3400700 1 -- 172.21.15.86:0/495050294 wait complete. 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.068+0000 7fbcf3400700 1 Processor -- start 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.068+0000 7fbcf3400700 1 -- start start 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.068+0000 7fbcf3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:37.879 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.068+0000 7fbcf3400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec10c990 con 0x7fbcec108430 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.069+0000 7fbcf0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.069+0000 7fbcf0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec19e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55174/0 (socket says 172.21.15.86:55174) 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.069+0000 7fbcf0a00700 1 -- 172.21.15.86:0/2839277079 learned_addr learned my addr 172.21.15.86:0/2839277079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.069+0000 7fbcf0a00700 1 -- 172.21.15.86:0/2839277079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec19ea10 con 0x7fbcec108430 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.069+0000 7fbcf0a00700 1 --2- 172.21.15.86:0/2839277079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec19e4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000c00 tx=0x7fbce0004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.070+0000 7fbce9600700 1 -- 172.21.15.86:0/2839277079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce00039c0 con 0x7fbcec108430 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.070+0000 7fbce9600700 1 -- 172.21.15.86:0/2839277079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fbce0003b60 con 0x7fbcec108430 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.070+0000 7fbcf3400700 1 -- 172.21.15.86:0/2839277079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec19eca0 con 0x7fbcec108430 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.070+0000 7fbce9600700 1 -- 172.21.15.86:0/2839277079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce001a6f0 con 0x7fbcec108430 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.070+0000 7fbcf3400700 1 -- 172.21.15.86:0/2839277079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec1a1990 con 0x7fbcec108430 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.070+0000 7fbce9600700 1 -- 172.21.15.86:0/2839277079 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbce001ac20 con 0x7fbcec108430 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.070+0000 7fbce9600700 1 -- 172.21.15.86:0/2839277079 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbce0019700 con 0x7fbcec108430 2024-04-17T07:01:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.071+0000 7fbcf3400700 1 -- 172.21.15.86:0/2839277079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcec0640b0 con 0x7fbcec108430 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.073+0000 7fbce9600700 1 -- 172.21.15.86:0/2839277079 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fbce0004300 con 0x7fbcec108430 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.153+0000 7fbcf3400700 1 -- 172.21.15.86:0/2839277079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbcec1a1b60 con 0x7fbcec108430 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.154+0000 7fbce9600700 1 -- 172.21.15.86:0/2839277079 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fbce0024a40 con 0x7fbcec108430 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.156+0000 7fbcf3400700 1 -- 172.21.15.86:0/2839277079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 msgr2=0x7fbcec19e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.156+0000 7fbcf3400700 1 --2- 172.21.15.86:0/2839277079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec19e4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000c00 tx=0x7fbce0004290 comp rx=0 tx=0).stop 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.156+0000 7fbcf3400700 1 -- 172.21.15.86:0/2839277079 shutdown_connections 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.156+0000 7fbcf3400700 1 --2- 172.21.15.86:0/2839277079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcec108430 0x7fbcec19e4d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.156+0000 7fbcf3400700 1 -- 172.21.15.86:0/2839277079 >> 172.21.15.86:0/2839277079 conn(0x7fbcec103a90 msgr2=0x7fbcec104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.156+0000 7fbcf3400700 1 -- 172.21.15.86:0/2839277079 shutdown_connections 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:37.156+0000 7fbcf3400700 1 -- 172.21.15.86:0/2839277079 wait complete. 2024-04-17T07:01:37.881 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (3/15)... 2024-04-17T07:01:38.492 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:38.080+0000 7ff22ede6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T07:01:38.492 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:38.209+0000 7ff22ede6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T07:01:38.830 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:38.492+0000 7ff22ede6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T07:01:38.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: Activating manager daemon x 2024-04-17T07:01:38.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: mgrmap e2: x(active, starting, since 0.00545743s) 2024-04-17T07:01:38.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: Manager daemon x is now available 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' 2024-04-17T07:01:38.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:38 smithi086 ceph-mon[31733]: from='mgr.14100 172.21.15.86:0/2346929162' entity='mgr.x' 2024-04-17T07:01:40.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:40 smithi086 ceph-mon[31733]: mgrmap e3: x(active, since 1.02069s) 2024-04-17T07:01:41.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:41.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:01:41.476 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:01:41.477 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-17T07:01:41.477 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-17T07:01:41.477 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-17T07:01:41.477 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-17T07:01:41.477 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:41.479 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-17T07:01:41.479 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-17T07:01:41.479 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-17T07:01:41.479 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:41.479 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-17T07:01:41.479 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-17T07:01:41.479 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:41.479 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-17T07:01:41.479 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-17T07:01:41.479 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:41.480 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-17T07:01:41.480 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-17T07:01:41.480 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:41.480 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-17T07:01:41.480 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:41.480 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-17T07:01:41.480 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-17T07:01:41.480 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-17T07:01:41.486 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-17T07:01:41.487 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-17T07:01:19.941840+0000", 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.481+0000 7f9b21200700 1 Processor -- start 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.481+0000 7f9b21200700 1 -- start start 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.482+0000 7f9b21200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c10b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.482+0000 7f9b21200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c07be30 con 0x7f9b1c108e00 2024-04-17T07:01:41.488 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.482+0000 7f9b19e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c10b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.482+0000 7f9b19e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c10b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55244/0 (socket says 172.21.15.86:55244) 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.482+0000 7f9b19e00700 1 -- 172.21.15.86:0/63459278 learned_addr learned my addr 172.21.15.86:0/63459278 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.482+0000 7f9b19e00700 1 -- 172.21.15.86:0/63459278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c10b730 con 0x7f9b1c108e00 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.483+0000 7f9b19e00700 1 --2- 172.21.15.86:0/63459278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c10b1f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0098a0 tx=0x7f9b0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98be52020afd22cc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.483+0000 7f9b18a00700 1 -- 172.21.15.86:0/63459278 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b0c004160 con 0x7f9b1c108e00 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.483+0000 7f9b18a00700 1 -- 172.21.15.86:0/63459278 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9b0c004300 con 0x7f9b1c108e00 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.483+0000 7f9b18a00700 1 -- 172.21.15.86:0/63459278 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b0c011a80 con 0x7f9b1c108e00 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.484+0000 7f9b21200700 1 -- 172.21.15.86:0/63459278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 msgr2=0x7f9b1c10b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.484+0000 7f9b21200700 1 --2- 172.21.15.86:0/63459278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c10b1f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0098a0 tx=0x7f9b0c009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.484+0000 7f9b21200700 1 -- 172.21.15.86:0/63459278 shutdown_connections 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.484+0000 7f9b21200700 1 --2- 172.21.15.86:0/63459278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c10b1f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.484+0000 7f9b21200700 1 -- 172.21.15.86:0/63459278 >> 172.21.15.86:0/63459278 conn(0x7f9b1c102830 msgr2=0x7f9b1c104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.484+0000 7f9b21200700 1 -- 172.21.15.86:0/63459278 shutdown_connections 2024-04-17T07:01:41.489 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.484+0000 7f9b21200700 1 -- 172.21.15.86:0/63459278 wait complete. 2024-04-17T07:01:41.490 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.485+0000 7f9b21200700 1 Processor -- start 2024-04-17T07:01:41.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.485+0000 7f9b21200700 1 -- start start 2024-04-17T07:01:41.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.486+0000 7f9b21200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:41.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.486+0000 7f9b21200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c10c690 con 0x7f9b1c108e00 2024-04-17T07:01:41.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.486+0000 7f9b19e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:41.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.486+0000 7f9b19e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c19c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55256/0 (socket says 172.21.15.86:55256) 2024-04-17T07:01:41.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.486+0000 7f9b19e00700 1 -- 172.21.15.86:0/2896351450 learned_addr learned my addr 172.21.15.86:0/2896351450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:41.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.486+0000 7f9b19e00700 1 -- 172.21.15.86:0/2896351450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c19c6e0 con 0x7f9b1c108e00 2024-04-17T07:01:41.493 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.486+0000 7f9b19e00700 1 --2- 172.21.15.86:0/2896351450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c19c1a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c011a50 tx=0x7f9b0c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.487+0000 7f9b12a00700 1 -- 172.21.15.86:0/2896351450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b0c0039f0 con 0x7f9b1c108e00 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.487+0000 7f9b12a00700 1 -- 172.21.15.86:0/2896351450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9b0c003b90 con 0x7f9b1c108e00 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.487+0000 7f9b21200700 1 -- 172.21.15.86:0/2896351450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1c19c970 con 0x7f9b1c108e00 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.487+0000 7f9b12a00700 1 -- 172.21.15.86:0/2896351450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b0c01a600 con 0x7f9b1c108e00 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.487+0000 7f9b21200700 1 -- 172.21.15.86:0/2896351450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1c19f660 con 0x7f9b1c108e00 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.488+0000 7f9b12a00700 1 -- 172.21.15.86:0/2896351450 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7f9b0c024a40 con 0x7f9b1c108e00 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.488+0000 7f9b21200700 1 -- 172.21.15.86:0/2896351450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1c0640b0 con 0x7f9b1c108e00 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.488+0000 7f9b12a00700 1 --2- 172.21.15.86:0/2896351450 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9b00039550 0x7f9b0003ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.489+0000 7f9b12a00700 1 -- 172.21.15.86:0/2896351450 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b0c04d880 con 0x7f9b1c108e00 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.489+0000 7f9b19400700 1 --2- 172.21.15.86:0/2896351450 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9b00039550 0x7f9b0003ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.489+0000 7f9b19400700 1 --2- 172.21.15.86:0/2896351450 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9b00039550 0x7f9b0003ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9b08006fd0 tx=0x7f9b08006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.495+0000 7f9b12a00700 1 -- 172.21.15.86:0/2896351450 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9b0c02d770 con 0x7f9b1c108e00 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.510+0000 7f9b12a00700 1 -- 172.21.15.86:0/2896351450 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f9b0c02cc20 con 0x7f9b1c108e00 2024-04-17T07:01:41.494 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.802+0000 7f9b21200700 1 -- 172.21.15.86:0/2896351450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9b1c19f830 con 0x7f9b1c108e00 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.803+0000 7f9b12a00700 1 -- 172.21.15.86:0/2896351450 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9b1c19f830 con 0x7f9b1c108e00 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.806+0000 7f9b21200700 1 -- 172.21.15.86:0/2896351450 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9b00039550 msgr2=0x7f9b0003ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.806+0000 7f9b21200700 1 --2- 172.21.15.86:0/2896351450 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9b00039550 0x7f9b0003ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9b08006fd0 tx=0x7f9b08006e40 comp rx=0 tx=0).stop 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.806+0000 7f9b21200700 1 -- 172.21.15.86:0/2896351450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 msgr2=0x7f9b1c19c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.806+0000 7f9b21200700 1 --2- 172.21.15.86:0/2896351450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c19c1a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c011a50 tx=0x7f9b0c004290 comp rx=0 tx=0).stop 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.806+0000 7f9b21200700 1 -- 172.21.15.86:0/2896351450 shutdown_connections 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.806+0000 7f9b21200700 1 --2- 172.21.15.86:0/2896351450 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9b00039550 0x7f9b0003ba10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.806+0000 7f9b21200700 1 --2- 172.21.15.86:0/2896351450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b1c108e00 0x7f9b1c19c1a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.806+0000 7f9b21200700 1 -- 172.21.15.86:0/2896351450 >> 172.21.15.86:0/2896351450 conn(0x7f9b1c102830 msgr2=0x7f9b1c1032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.807+0000 7f9b21200700 1 -- 172.21.15.86:0/2896351450 shutdown_connections 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:40.807+0000 7f9b21200700 1 -- 172.21.15.86:0/2896351450 wait complete. 2024-04-17T07:01:41.495 INFO:teuthology.orchestra.run.smithi086.stdout:mgr is available 2024-04-17T07:01:41.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:41 smithi086 ceph-mon[31733]: mgrmap e4: x(active, since 2s) 2024-04-17T07:01:41.794 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:41 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2896351450' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-17T07:01:42.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:42 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2986887585' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-17T07:01:42.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:42.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-04-17T07:01:42.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:42.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-17T07:01:42.979 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-17T07:01:42.979 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-17T07:01:42.979 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-17T07:01:42.979 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:42.979 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-17T07:01:42.979 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-17T07:01:42.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-17T07:01:42.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-04-17T07:01:42.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-17T07:01:42.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-17T07:01:42.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-17T07:01:42.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-17T07:01:42.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-17T07:01:42.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.085+0000 7f0c61800700 1 Processor -- start 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.085+0000 7f0c61800700 1 -- start start 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.086+0000 7f0c61800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c1087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.086+0000 7f0c61800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c108dc0 con 0x7f0c5c1083d0 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.086+0000 7f0c5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c1087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.086+0000 7f0c5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c1087f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55258/0 (socket says 172.21.15.86:55258) 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.086+0000 7f0c5aa00700 1 -- 172.21.15.86:0/633042821 learned_addr learned my addr 172.21.15.86:0/633042821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.086+0000 7f0c5aa00700 1 -- 172.21.15.86:0/633042821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5c109530 con 0x7f0c5c1083d0 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.087+0000 7f0c5aa00700 1 --2- 172.21.15.86:0/633042821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c1087f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0098a0 tx=0x7f0c4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d71969c42e80f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.087+0000 7f0c59600700 1 -- 172.21.15.86:0/633042821 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c4c004160 con 0x7f0c5c1083d0 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.087+0000 7f0c59600700 1 -- 172.21.15.86:0/633042821 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0c4c004300 con 0x7f0c5c1083d0 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.087+0000 7f0c59600700 1 -- 172.21.15.86:0/633042821 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c4c011a80 con 0x7f0c5c1083d0 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.088+0000 7f0c61800700 1 -- 172.21.15.86:0/633042821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 msgr2=0x7f0c5c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.088+0000 7f0c61800700 1 --2- 172.21.15.86:0/633042821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c1087f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0098a0 tx=0x7f0c4c009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:42.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.088+0000 7f0c61800700 1 -- 172.21.15.86:0/633042821 shutdown_connections 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.088+0000 7f0c61800700 1 --2- 172.21.15.86:0/633042821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c1087f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.088+0000 7f0c61800700 1 -- 172.21.15.86:0/633042821 >> 172.21.15.86:0/633042821 conn(0x7f0c5c1039f0 msgr2=0x7f0c5c105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.088+0000 7f0c61800700 1 -- 172.21.15.86:0/633042821 shutdown_connections 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.088+0000 7f0c61800700 1 -- 172.21.15.86:0/633042821 wait complete. 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.089+0000 7f0c61800700 1 Processor -- start 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.089+0000 7f0c61800700 1 -- start start 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.089+0000 7f0c61800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c07db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.089+0000 7f0c61800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c10c8e0 con 0x7f0c5c1083d0 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.089+0000 7f0c5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c07db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.089+0000 7f0c5aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c07db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55268/0 (socket says 172.21.15.86:55268) 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.089+0000 7f0c5aa00700 1 -- 172.21.15.86:0/2986887585 learned_addr learned my addr 172.21.15.86:0/2986887585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.090+0000 7f0c5aa00700 1 -- 172.21.15.86:0/2986887585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5c07e0d0 con 0x7f0c5c1083d0 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.090+0000 7f0c5aa00700 1 --2- 172.21.15.86:0/2986887585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c07db90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c006af0 tx=0x7f0c4c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:42.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.090+0000 7f0c53400700 1 -- 172.21.15.86:0/2986887585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c4c003930 con 0x7f0c5c1083d0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.090+0000 7f0c53400700 1 -- 172.21.15.86:0/2986887585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0c4c003ad0 con 0x7f0c5c1083d0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.090+0000 7f0c61800700 1 -- 172.21.15.86:0/2986887585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5c07c2c0 con 0x7f0c5c1083d0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.090+0000 7f0c53400700 1 -- 172.21.15.86:0/2986887585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c4c0195e0 con 0x7f0c5c1083d0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.090+0000 7f0c61800700 1 -- 172.21.15.86:0/2986887585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c5c07c800 con 0x7f0c5c1083d0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.091+0000 7f0c53400700 1 -- 172.21.15.86:0/2986887585 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f0c4c019890 con 0x7f0c5c1083d0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.091+0000 7f0c61800700 1 -- 172.21.15.86:0/2986887585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c5c0640b0 con 0x7f0c5c1083d0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.092+0000 7f0c53400700 1 --2- 172.21.15.86:0/2986887585 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f0c40039620 0x7f0c4003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.092+0000 7f0c53400700 1 -- 172.21.15.86:0/2986887585 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0c4c04db40 con 0x7f0c5c1083d0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.092+0000 7f0c5a000700 1 --2- 172.21.15.86:0/2986887585 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f0c40039620 0x7f0c4003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.093+0000 7f0c5a000700 1 --2- 172.21.15.86:0/2986887585 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f0c40039620 0x7f0c4003bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c48006fd0 tx=0x7f0c48006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.097+0000 7f0c53400700 1 -- 172.21.15.86:0/2986887585 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0c4c04d560 con 0x7f0c5c1083d0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.302+0000 7f0c61800700 1 -- 172.21.15.86:0/2986887585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0c5c07cff0 con 0x7f0c5c1083d0 2024-04-17T07:01:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.304+0000 7f0c53400700 1 -- 172.21.15.86:0/2986887585 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f0c4c00fda0 con 0x7f0c5c1083d0 2024-04-17T07:01:42.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.306+0000 7f0c61800700 1 -- 172.21.15.86:0/2986887585 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f0c40039620 msgr2=0x7f0c4003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:42.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.306+0000 7f0c61800700 1 --2- 172.21.15.86:0/2986887585 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f0c40039620 0x7f0c4003bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c48006fd0 tx=0x7f0c48006e40 comp rx=0 tx=0).stop 2024-04-17T07:01:42.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.306+0000 7f0c61800700 1 -- 172.21.15.86:0/2986887585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 msgr2=0x7f0c5c07db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:42.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.306+0000 7f0c61800700 1 --2- 172.21.15.86:0/2986887585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c07db90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c006af0 tx=0x7f0c4c011b70 comp rx=0 tx=0).stop 2024-04-17T07:01:42.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.307+0000 7f0c61800700 1 -- 172.21.15.86:0/2986887585 shutdown_connections 2024-04-17T07:01:42.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.307+0000 7f0c61800700 1 --2- 172.21.15.86:0/2986887585 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f0c40039620 0x7f0c4003bae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:42.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.307+0000 7f0c61800700 1 --2- 172.21.15.86:0/2986887585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c5c1083d0 0x7f0c5c07db90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:42.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.307+0000 7f0c61800700 1 -- 172.21.15.86:0/2986887585 >> 172.21.15.86:0/2986887585 conn(0x7f0c5c1039f0 msgr2=0x7f0c5c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:42.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.307+0000 7f0c61800700 1 -- 172.21.15.86:0/2986887585 shutdown_connections 2024-04-17T07:01:42.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:42.307+0000 7f0c61800700 1 -- 172.21.15.86:0/2986887585 wait complete. 2024-04-17T07:01:42.991 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling cephadm module... 2024-04-17T07:01:44.300 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:43 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2002944161' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-17T07:01:44.300 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:44 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: ignoring --setuser ceph since I am not root 2024-04-17T07:01:44.300 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:44 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: ignoring --setgroup ceph since I am not root 2024-04-17T07:01:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.573+0000 7fe060600700 1 Processor -- start 2024-04-17T07:01:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.573+0000 7fe060600700 1 -- start start 2024-04-17T07:01:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.573+0000 7fe060600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe05810f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.573+0000 7fe060600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810f920 con 0x7fe05810ef50 2024-04-17T07:01:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.574+0000 7fe05dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe05810f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.574+0000 7fe05dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe05810f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55270/0 (socket says 172.21.15.86:55270) 2024-04-17T07:01:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.574+0000 7fe05dc00700 1 -- 172.21.15.86:0/1194479324 learned_addr learned my addr 172.21.15.86:0/1194479324 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.574+0000 7fe05dc00700 1 -- 172.21.15.86:0/1194479324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05810faa0 con 0x7fe05810ef50 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.574+0000 7fe05dc00700 1 --2- 172.21.15.86:0/1194479324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe05810f350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0098a0 tx=0x7fe04c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c549d7ded0d29cac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.575+0000 7fe057e00700 1 -- 172.21.15.86:0/1194479324 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe04c004160 con 0x7fe05810ef50 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.575+0000 7fe057e00700 1 -- 172.21.15.86:0/1194479324 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe04c004300 con 0x7fe05810ef50 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.575+0000 7fe057e00700 1 -- 172.21.15.86:0/1194479324 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe04c011a80 con 0x7fe05810ef50 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.575+0000 7fe060600700 1 -- 172.21.15.86:0/1194479324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 msgr2=0x7fe05810f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.575+0000 7fe060600700 1 --2- 172.21.15.86:0/1194479324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe05810f350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0098a0 tx=0x7fe04c009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.575+0000 7fe060600700 1 -- 172.21.15.86:0/1194479324 shutdown_connections 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.575+0000 7fe060600700 1 --2- 172.21.15.86:0/1194479324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe05810f350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.575+0000 7fe060600700 1 -- 172.21.15.86:0/1194479324 >> 172.21.15.86:0/1194479324 conn(0x7fe05807de30 msgr2=0x7fe058080250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.575+0000 7fe060600700 1 -- 172.21.15.86:0/1194479324 shutdown_connections 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.576+0000 7fe060600700 1 -- 172.21.15.86:0/1194479324 wait complete. 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.576+0000 7fe060600700 1 Processor -- start 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.577+0000 7fe060600700 1 -- start start 2024-04-17T07:01:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.577+0000 7fe060600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe0581a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.577+0000 7fe060600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe058110a60 con 0x7fe05810ef50 2024-04-17T07:01:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.577+0000 7fe05dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe0581a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.577+0000 7fe05dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe0581a27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55282/0 (socket says 172.21.15.86:55282) 2024-04-17T07:01:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.577+0000 7fe05dc00700 1 -- 172.21.15.86:0/2002944161 learned_addr learned my addr 172.21.15.86:0/2002944161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.578+0000 7fe05dc00700 1 -- 172.21.15.86:0/2002944161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0581a2d00 con 0x7fe05810ef50 2024-04-17T07:01:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.578+0000 7fe05dc00700 1 --2- 172.21.15.86:0/2002944161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe0581a27c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe04c000c00 tx=0x7fe04c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.578+0000 7fe056000700 1 -- 172.21.15.86:0/2002944161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe04c0039c0 con 0x7fe05810ef50 2024-04-17T07:01:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.578+0000 7fe056000700 1 -- 172.21.15.86:0/2002944161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe04c003b60 con 0x7fe05810ef50 2024-04-17T07:01:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.578+0000 7fe060600700 1 -- 172.21.15.86:0/2002944161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0581a2f90 con 0x7fe05810ef50 2024-04-17T07:01:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.578+0000 7fe056000700 1 -- 172.21.15.86:0/2002944161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe04c01a6f0 con 0x7fe05810ef50 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.578+0000 7fe060600700 1 -- 172.21.15.86:0/2002944161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0581a5c80 con 0x7fe05810ef50 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.579+0000 7fe056000700 1 -- 172.21.15.86:0/2002944161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fe04c01a890 con 0x7fe05810ef50 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.579+0000 7fe060600700 1 -- 172.21.15.86:0/2002944161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe058112f30 con 0x7fe05810ef50 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.580+0000 7fe056000700 1 --2- 172.21.15.86:0/2002944161 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7fe040039670 0x7fe04003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.580+0000 7fe056000700 1 -- 172.21.15.86:0/2002944161 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe04c04dab0 con 0x7fe05810ef50 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.580+0000 7fe05d200700 1 --2- 172.21.15.86:0/2002944161 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7fe040039670 0x7fe04003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.581+0000 7fe05d200700 1 --2- 172.21.15.86:0/2002944161 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7fe040039670 0x7fe04003bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe044006fd0 tx=0x7fe044006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.585+0000 7fe056000700 1 -- 172.21.15.86:0/2002944161 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe04c04d4d0 con 0x7fe05810ef50 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.872+0000 7fe060600700 1 -- 172.21.15.86:0/2002944161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fe0581a62d0 con 0x7fe05810ef50 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.926+0000 7fe056000700 1 -- 172.21.15.86:0/2002944161 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fe04c024780 con 0x7fe05810ef50 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.930+0000 7fe056000700 1 -- 172.21.15.86:0/2002944161 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fe04c02d910 con 0x7fe05810ef50 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.931+0000 7fe060600700 1 -- 172.21.15.86:0/2002944161 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7fe040039670 msgr2=0x7fe04003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.931+0000 7fe060600700 1 --2- 172.21.15.86:0/2002944161 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7fe040039670 0x7fe04003bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe044006fd0 tx=0x7fe044006e40 comp rx=0 tx=0).stop 2024-04-17T07:01:44.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.931+0000 7fe060600700 1 -- 172.21.15.86:0/2002944161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 msgr2=0x7fe0581a27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:44.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.931+0000 7fe060600700 1 --2- 172.21.15.86:0/2002944161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe0581a27c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe04c000c00 tx=0x7fe04c004290 comp rx=0 tx=0).stop 2024-04-17T07:01:44.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.932+0000 7fe060600700 1 -- 172.21.15.86:0/2002944161 shutdown_connections 2024-04-17T07:01:44.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.932+0000 7fe060600700 1 --2- 172.21.15.86:0/2002944161 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7fe040039670 0x7fe04003bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:44.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.932+0000 7fe060600700 1 --2- 172.21.15.86:0/2002944161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe05810ef50 0x7fe0581a27c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:44.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.932+0000 7fe060600700 1 -- 172.21.15.86:0/2002944161 >> 172.21.15.86:0/2002944161 conn(0x7fe05807de30 msgr2=0x7fe05807f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:44.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.932+0000 7fe060600700 1 -- 172.21.15.86:0/2002944161 shutdown_connections 2024-04-17T07:01:44.580 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:43.932+0000 7fe060600700 1 -- 172.21.15.86:0/2002944161 wait complete. 2024-04-17T07:01:44.829 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:44 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:44.711+0000 7f0d1fab9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T07:01:45.122 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:44 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:44.835+0000 7f0d1fab9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T07:01:45.122 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:44 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2002944161' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-17T07:01:45.122 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:44 smithi086 ceph-mon[31733]: mgrmap e5: x(active, since 5s) 2024-04-17T07:01:45.936 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:45 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:45.664+0000 7f0d1fab9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T07:01:45.937 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:45 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:45.789+0000 7f0d1fab9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T07:01:46.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:01:46.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-17T07:01:46.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-17T07:01:46.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-17T07:01:46.173 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-17T07:01:46.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:01:46.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.171+0000 7f6359200700 1 Processor -- start 2024-04-17T07:01:46.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.172+0000 7f6359200700 1 -- start start 2024-04-17T07:01:46.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.172+0000 7f6359200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f63541087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:46.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.172+0000 7f6359200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6354108d70 con 0x7f6354108380 2024-04-17T07:01:46.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.172+0000 7f6351e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f63541087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:46.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.172+0000 7f6351e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f63541087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55318/0 (socket says 172.21.15.86:55318) 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.172+0000 7f6351e00700 1 -- 172.21.15.86:0/1653835409 learned_addr learned my addr 172.21.15.86:0/1653835409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.173+0000 7f6351e00700 1 -- 172.21.15.86:0/1653835409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63541094e0 con 0x7f6354108380 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.173+0000 7f6351e00700 1 --2- 172.21.15.86:0/1653835409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f63541087a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f63440098a0 tx=0x7f6344009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6596fcc5a126cca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.173+0000 7f6350a00700 1 -- 172.21.15.86:0/1653835409 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6344004160 con 0x7f6354108380 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.173+0000 7f6350a00700 1 -- 172.21.15.86:0/1653835409 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6344004300 con 0x7f6354108380 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.173+0000 7f6350a00700 1 -- 172.21.15.86:0/1653835409 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6344011a80 con 0x7f6354108380 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.174+0000 7f6359200700 1 -- 172.21.15.86:0/1653835409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 msgr2=0x7f63541087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.174+0000 7f6359200700 1 --2- 172.21.15.86:0/1653835409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f63541087a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f63440098a0 tx=0x7f6344009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.174+0000 7f6359200700 1 -- 172.21.15.86:0/1653835409 shutdown_connections 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.174+0000 7f6359200700 1 --2- 172.21.15.86:0/1653835409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f63541087a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.174+0000 7f6359200700 1 -- 172.21.15.86:0/1653835409 >> 172.21.15.86:0/1653835409 conn(0x7f6354103980 msgr2=0x7f6354105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.174+0000 7f6359200700 1 -- 172.21.15.86:0/1653835409 shutdown_connections 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.174+0000 7f6359200700 1 -- 172.21.15.86:0/1653835409 wait complete. 2024-04-17T07:01:46.179 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.175+0000 7f6359200700 1 Processor -- start 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.175+0000 7f6359200700 1 -- start start 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.175+0000 7f6359200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f635407db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.175+0000 7f6359200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f635410c890 con 0x7f6354108380 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.176+0000 7f6351e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f635407db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.176+0000 7f6351e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f635407db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55332/0 (socket says 172.21.15.86:55332) 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.176+0000 7f6351e00700 1 -- 172.21.15.86:0/2688929498 learned_addr learned my addr 172.21.15.86:0/2688929498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.176+0000 7f6351e00700 1 -- 172.21.15.86:0/2688929498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635407e0d0 con 0x7f6354108380 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.176+0000 7f6351e00700 1 --2- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f635407db90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6344000c00 tx=0x7f6344004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.177+0000 7f634aa00700 1 -- 172.21.15.86:0/2688929498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63440039c0 con 0x7f6354108380 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.177+0000 7f634aa00700 1 -- 172.21.15.86:0/2688929498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6344003b60 con 0x7f6354108380 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.177+0000 7f6359200700 1 -- 172.21.15.86:0/2688929498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635407c2c0 con 0x7f6354108380 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.177+0000 7f634aa00700 1 -- 172.21.15.86:0/2688929498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f634401a6f0 con 0x7f6354108380 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.177+0000 7f6359200700 1 -- 172.21.15.86:0/2688929498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635407c800 con 0x7f6354108380 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.178+0000 7f6359200700 1 -- 172.21.15.86:0/2688929498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63540640b0 con 0x7f6354108380 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.178+0000 7f634aa00700 1 -- 172.21.15.86:0/2688929498 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f6344024a40 con 0x7f6354108380 2024-04-17T07:01:46.180 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.178+0000 7f634aa00700 1 --2- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f63380396c0 0x7f633803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:46.181 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.179+0000 7f634aa00700 1 -- 172.21.15.86:0/2688929498 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f634404d9d0 con 0x7f6354108380 2024-04-17T07:01:46.181 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.179+0000 7f6351400700 1 -- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f63380396c0 msgr2=0x7f633803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4176177513 2024-04-17T07:01:46.181 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.179+0000 7f6351400700 1 --2- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f63380396c0 0x7f633803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:01:46.181 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.184+0000 7f634aa00700 1 -- 172.21.15.86:0/2688929498 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f634400f2e0 con 0x7f6354108380 2024-04-17T07:01:46.181 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.380+0000 7f6351400700 1 -- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f63380396c0 msgr2=0x7f633803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4176177513 2024-04-17T07:01:46.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.380+0000 7f6351400700 1 --2- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f63380396c0 0x7f633803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T07:01:46.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.467+0000 7f6359200700 1 -- 172.21.15.86:0/2688929498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f635407d130 con 0x7f6354108380 2024-04-17T07:01:46.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.468+0000 7f634aa00700 1 -- 172.21.15.86:0/2688929498 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f6344004300 con 0x7f6354108380 2024-04-17T07:01:46.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.471+0000 7f6359200700 1 -- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f63380396c0 msgr2=0x7f633803bb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:01:46.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.471+0000 7f6359200700 1 --2- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f63380396c0 0x7f633803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:46.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.471+0000 7f6359200700 1 -- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 msgr2=0x7f635407db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:46.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.471+0000 7f6359200700 1 --2- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f635407db90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6344000c00 tx=0x7f6344004290 comp rx=0 tx=0).stop 2024-04-17T07:01:46.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.471+0000 7f6359200700 1 -- 172.21.15.86:0/2688929498 shutdown_connections 2024-04-17T07:01:46.182 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.471+0000 7f6359200700 1 --2- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f63380396c0 0x7f633803bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:46.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.471+0000 7f6359200700 1 --2- 172.21.15.86:0/2688929498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6354108380 0x7f635407db90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:46.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.471+0000 7f6359200700 1 -- 172.21.15.86:0/2688929498 >> 172.21.15.86:0/2688929498 conn(0x7f6354103980 msgr2=0x7f63541044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:46.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.471+0000 7f6359200700 1 -- 172.21.15.86:0/2688929498 shutdown_connections 2024-04-17T07:01:46.183 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:45.471+0000 7f6359200700 1 -- 172.21.15.86:0/2688929498 wait complete. 2024-04-17T07:01:46.183 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-04-17T07:01:46.183 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 5... 2024-04-17T07:01:46.213 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:45 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2688929498' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-17T07:01:46.213 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:45 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:45.976+0000 7f0d1fab9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T07:01:46.213 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:46.101+0000 7f0d1fab9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T07:01:46.213 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:46.213+0000 7f0d1fab9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T07:01:46.824 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:46.560+0000 7f0d1fab9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T07:01:46.824 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:46.708+0000 7f0d1fab9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T07:01:46.825 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:46.823+0000 7f0d1fab9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T07:01:47.080 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:46.947+0000 7f0d1fab9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T07:01:47.579 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:47 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:47.165+0000 7f0d1fab9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T07:01:48.830 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:48 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:48.436+0000 7f0d1fab9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T07:01:48.830 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:48 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:48.611+0000 7f0d1fab9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T07:01:49.330 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:49 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:49.026+0000 7f0d1fab9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T07:01:49.330 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:49 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:49.145+0000 7f0d1fab9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T07:01:49.747 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:49 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:49.401+0000 7f0d1fab9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:01:49.747 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:49 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:49.647+0000 7f0d1fab9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T07:01:50.330 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:50 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:50.027+0000 7f0d1fab9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:01:50.645 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:50 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:50.342+0000 7f0d1fab9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T07:01:50.645 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:50 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:50.644+0000 7f0d1fab9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T07:01:52.080 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:51 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:51.665+0000 7f0d1fab9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T07:01:52.768 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:52.358+0000 7f0d1fab9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T07:01:52.768 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:52.487+0000 7f0d1fab9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T07:01:53.080 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:01:52.767+0000 7f0d1fab9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T07:01:53.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-mon[31733]: Active manager daemon x restarted 2024-04-17T07:01:53.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-mon[31733]: Activating manager daemon x 2024-04-17T07:01:53.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-mon[31733]: osdmap e2: 0 total, 0 up, 0 in 2024-04-17T07:01:53.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-mon[31733]: mgrmap e6: x(active, starting, since 0.00813536s) 2024-04-17T07:01:53.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:01:53.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T07:01:53.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T07:01:53.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T07:01:53.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T07:01:53.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:52 smithi086 ceph-mon[31733]: Manager daemon x is now available 2024-04-17T07:01:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:53 smithi086 ceph-mon[31733]: Found migration_current of "None". Setting to last migration. 2024-04-17T07:01:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:53 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:01:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:53 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:01:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:53 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:01:54.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:53 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:01:54.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:53 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T07:01:54.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:53 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T07:01:54.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:53 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:01:54.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:53 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:01:54.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:53 smithi086 ceph-mon[31733]: mgrmap e7: x(active, since 1.01385s) 2024-04-17T07:01:54.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:01:54.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-17T07:01:54.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-17T07:01:54.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:01:54.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.734+0000 7f9e8ec00700 1 Processor -- start 2024-04-17T07:01:54.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.734+0000 7f9e8ec00700 1 -- start start 2024-04-17T07:01:54.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.735+0000 7f9e8ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e881087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:54.499 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.735+0000 7f9e8ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88108dc0 con 0x7f9e881083d0 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.735+0000 7f9e87e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e881087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.735+0000 7f9e87e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e881087f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55344/0 (socket says 172.21.15.86:55344) 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.735+0000 7f9e87e00700 1 -- 172.21.15.86:0/2537474325 learned_addr learned my addr 172.21.15.86:0/2537474325 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.736+0000 7f9e87e00700 1 -- 172.21.15.86:0/2537474325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e88109530 con 0x7f9e881083d0 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.736+0000 7f9e87e00700 1 --2- 172.21.15.86:0/2537474325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e881087f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9e780098a0 tx=0x7f9e78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbe8eb5bc1cd32f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.736+0000 7f9e86a00700 1 -- 172.21.15.86:0/2537474325 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e78004160 con 0x7f9e881083d0 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.736+0000 7f9e86a00700 1 -- 172.21.15.86:0/2537474325 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9e78004300 con 0x7f9e881083d0 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.736+0000 7f9e86a00700 1 -- 172.21.15.86:0/2537474325 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e78011a80 con 0x7f9e881083d0 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.737+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2537474325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 msgr2=0x7f9e881087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.737+0000 7f9e8ec00700 1 --2- 172.21.15.86:0/2537474325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e881087f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9e780098a0 tx=0x7f9e78009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:54.501 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.737+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2537474325 shutdown_connections 2024-04-17T07:01:54.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.737+0000 7f9e8ec00700 1 --2- 172.21.15.86:0/2537474325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e881087f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:54.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.737+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2537474325 >> 172.21.15.86:0/2537474325 conn(0x7f9e881039f0 msgr2=0x7f9e88105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:54.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.737+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2537474325 shutdown_connections 2024-04-17T07:01:54.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.737+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2537474325 wait complete. 2024-04-17T07:01:54.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.739+0000 7f9e8ec00700 1 Processor -- start 2024-04-17T07:01:54.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.739+0000 7f9e8ec00700 1 -- start start 2024-04-17T07:01:54.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.739+0000 7f9e8ec00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e8819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:54.504 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.739+0000 7f9e8ec00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8810c760 con 0x7f9e881083d0 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.739+0000 7f9e87e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e8819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.739+0000 7f9e87e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e8819e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55354/0 (socket says 172.21.15.86:55354) 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.739+0000 7f9e87e00700 1 -- 172.21.15.86:0/2454681788 learned_addr learned my addr 172.21.15.86:0/2454681788 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.740+0000 7f9e87e00700 1 -- 172.21.15.86:0/2454681788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8819e910 con 0x7f9e881083d0 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.740+0000 7f9e87e00700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e8819e3d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9e78000c00 tx=0x7f9e78004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.740+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e780039c0 con 0x7f9e881083d0 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.741+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2454681788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8819eba0 con 0x7f9e881083d0 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.741+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9e78003b60 con 0x7f9e881083d0 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.741+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e7801a6f0 con 0x7f9e881083d0 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.741+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2454681788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e881a1890 con 0x7f9e881083d0 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.742+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f9e7801a9a0 con 0x7f9e881083d0 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.742+0000 7f9e84c00700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 0x7f9e6c03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:54.505 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.742+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2454681788 --> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9e880640b0 con 0x7f9e6c039670 2024-04-17T07:01:54.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.742+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9e7804db70 con 0x7f9e881083d0 2024-04-17T07:01:54.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.742+0000 7f9e87400700 1 -- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 msgr2=0x7f9e6c03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4176177513 2024-04-17T07:01:54.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.742+0000 7f9e87400700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 0x7f9e6c03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:01:54.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.943+0000 7f9e87400700 1 -- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 msgr2=0x7f9e6c03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4176177513 2024-04-17T07:01:54.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:46.943+0000 7f9e87400700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 0x7f9e6c03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T07:01:54.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:47.344+0000 7f9e87400700 1 -- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 msgr2=0x7f9e6c03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4176177513 2024-04-17T07:01:54.509 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:47.344+0000 7f9e87400700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 0x7f9e6c03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:48.145+0000 7f9e87400700 1 -- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 msgr2=0x7f9e6c03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4176177513 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:48.145+0000 7f9e87400700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 0x7f9e6c03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:49.747+0000 7f9e87400700 1 -- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 msgr2=0x7f9e6c03bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/4176177513 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:49.747+0000 7f9e87400700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 0x7f9e6c03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:52.779+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f9e7802d910 con 0x7f9e881083d0 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:52.780+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 msgr2=0x7f9e6c03bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:52.780+0000 7f9e84c00700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 0x7f9e6c03bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.782+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f9e7804f6e0 con 0x7f9e881083d0 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.783+0000 7f9e84c00700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9e6c03cd70 0x7f9e6c03f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.783+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 --> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9e880640b0 con 0x7f9e6c03cd70 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.785+0000 7f9e87400700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9e6c03cd70 0x7f9e6c03f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.788+0000 7f9e87400700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9e6c03cd70 0x7f9e6c03f160 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9e74003a10 tx=0x7f9e74009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.789+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 <== mgr.14120 v2:172.21.15.86:6800/2497304884 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f9e880640b0 con 0x7f9e6c03cd70 2024-04-17T07:01:54.510 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.797+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2454681788 --> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9e8810c8e0 con 0x7f9e6c03cd70 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.798+0000 7f9e84c00700 1 -- 172.21.15.86:0/2454681788 <== mgr.14120 v2:172.21.15.86:6800/2497304884 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f9e8810c8e0 con 0x7f9e6c03cd70 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.798+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9e6c03cd70 msgr2=0x7f9e6c03f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.798+0000 7f9e8ec00700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9e6c03cd70 0x7f9e6c03f160 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9e74003a10 tx=0x7f9e74009290 comp rx=0 tx=0).stop 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.798+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 msgr2=0x7f9e8819e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.798+0000 7f9e8ec00700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e8819e3d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9e78000c00 tx=0x7f9e78004290 comp rx=0 tx=0).stop 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.799+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2454681788 shutdown_connections 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.799+0000 7f9e8ec00700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9e6c03cd70 0x7f9e6c03f160 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.799+0000 7f9e8ec00700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e881083d0 0x7f9e8819e3d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.799+0000 7f9e8ec00700 1 --2- 172.21.15.86:0/2454681788 >> [v2:172.21.15.86:6800/4176177513,v1:172.21.15.86:6801/4176177513] conn(0x7f9e6c039670 0x7f9e6c03bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.799+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2454681788 >> 172.21.15.86:0/2454681788 conn(0x7f9e881039f0 msgr2=0x7f9e88104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.799+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2454681788 shutdown_connections 2024-04-17T07:01:54.511 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:53.799+0000 7f9e8ec00700 1 -- 172.21.15.86:0/2454681788 wait complete. 2024-04-17T07:01:54.512 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 5 is available 2024-04-17T07:01:54.512 INFO:teuthology.orchestra.run.smithi086.stdout:Setting orchestrator backend to cephadm... 2024-04-17T07:01:55.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:54 smithi086 ceph-mon[31733]: [17/Apr/2024:07:01:53] ENGINE Bus STARTING 2024-04-17T07:01:55.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:54 smithi086 ceph-mon[31733]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-17T07:01:55.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:54 smithi086 ceph-mon[31733]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-17T07:01:55.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:54 smithi086 ceph-mon[31733]: [17/Apr/2024:07:01:53] ENGINE Serving on http://172.21.15.86:8765 2024-04-17T07:01:55.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:54 smithi086 ceph-mon[31733]: [17/Apr/2024:07:01:53] ENGINE Serving on https://172.21.15.86:7150 2024-04-17T07:01:55.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:54 smithi086 ceph-mon[31733]: [17/Apr/2024:07:01:53] ENGINE Bus STARTED 2024-04-17T07:01:55.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:54 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:01:55.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:55 smithi086 ceph-mon[31733]: mgrmap e8: x(active, since 2s) 2024-04-17T07:01:55.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:55 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:01:55.988 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:55 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:01:56.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.092+0000 7f2ddc000700 1 Processor -- start 2024-04-17T07:01:56.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.092+0000 7f2ddc000700 1 -- start start 2024-04-17T07:01:56.002 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.093+0000 7f2ddc000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd410a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.093+0000 7f2ddc000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410af80 con 0x7f2dd410a590 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.093+0000 7f2dd9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd410a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.093+0000 7f2dd9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd410a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60540/0 (socket says 172.21.15.86:60540) 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.093+0000 7f2dd9600700 1 -- 172.21.15.86:0/2751233859 learned_addr learned my addr 172.21.15.86:0/2751233859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.094+0000 7f2dd9600700 1 -- 172.21.15.86:0/2751233859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd410b740 con 0x7f2dd410a590 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.094+0000 7f2dd9600700 1 --2- 172.21.15.86:0/2751233859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd410a9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80098a0 tx=0x7f2dc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c68077771eff66 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.094+0000 7f2dd3e00700 1 -- 172.21.15.86:0/2751233859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc8004160 con 0x7f2dd410a590 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.094+0000 7f2dd3e00700 1 -- 172.21.15.86:0/2751233859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2dc8004300 con 0x7f2dd410a590 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.094+0000 7f2dd3e00700 1 -- 172.21.15.86:0/2751233859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc8011a80 con 0x7f2dd410a590 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.095+0000 7f2ddc000700 1 -- 172.21.15.86:0/2751233859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 msgr2=0x7f2dd410a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.095+0000 7f2ddc000700 1 --2- 172.21.15.86:0/2751233859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd410a9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80098a0 tx=0x7f2dc8009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.095+0000 7f2ddc000700 1 -- 172.21.15.86:0/2751233859 shutdown_connections 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.095+0000 7f2ddc000700 1 --2- 172.21.15.86:0/2751233859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd410a9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:56.003 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.095+0000 7f2ddc000700 1 -- 172.21.15.86:0/2751233859 >> 172.21.15.86:0/2751233859 conn(0x7f2dd407dc10 msgr2=0x7f2dd41084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.095+0000 7f2ddc000700 1 -- 172.21.15.86:0/2751233859 shutdown_connections 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.095+0000 7f2ddc000700 1 -- 172.21.15.86:0/2751233859 wait complete. 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.096+0000 7f2ddc000700 1 Processor -- start 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.096+0000 7f2ddc000700 1 -- start start 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.096+0000 7f2ddc000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.096+0000 7f2ddc000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410eaf0 con 0x7f2dd410a590 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.097+0000 7f2dd9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.097+0000 7f2dd9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60546/0 (socket says 172.21.15.86:60546) 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.097+0000 7f2dd9600700 1 -- 172.21.15.86:0/4192680443 learned_addr learned my addr 172.21.15.86:0/4192680443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.097+0000 7f2dd9600700 1 -- 172.21.15.86:0/4192680443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd419e940 con 0x7f2dd410a590 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.097+0000 7f2dd9600700 1 --2- 172.21.15.86:0/4192680443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd419e400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8006af0 tx=0x7f2dc8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.098+0000 7f2dd2000700 1 -- 172.21.15.86:0/4192680443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc8003930 con 0x7f2dd410a590 2024-04-17T07:01:56.004 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.098+0000 7f2dd2000700 1 -- 172.21.15.86:0/4192680443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2dc8003ad0 con 0x7f2dd410a590 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.098+0000 7f2ddc000700 1 -- 172.21.15.86:0/4192680443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd419ebd0 con 0x7f2dd410a590 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.098+0000 7f2dd2000700 1 -- 172.21.15.86:0/4192680443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc80195e0 con 0x7f2dd410a590 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.098+0000 7f2ddc000700 1 -- 172.21.15.86:0/4192680443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd41a18c0 con 0x7f2dd410a590 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.099+0000 7f2dd2000700 1 -- 172.21.15.86:0/4192680443 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2dc801a430 con 0x7f2dd410a590 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.099+0000 7f2ddc000700 1 -- 172.21.15.86:0/4192680443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd40640b0 con 0x7f2dd410a590 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.099+0000 7f2dd2000700 1 --2- 172.21.15.86:0/4192680443 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f2dbc039670 0x7f2dbc03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.099+0000 7f2dd2000700 1 -- 172.21.15.86:0/4192680443 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2dc804dc20 con 0x7f2dd410a590 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.100+0000 7f2dd8c00700 1 --2- 172.21.15.86:0/4192680443 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f2dbc039670 0x7f2dbc03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.100+0000 7f2dd8c00700 1 --2- 172.21.15.86:0/4192680443 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f2dbc039670 0x7f2dbc03bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006fd0 tx=0x7f2dc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.106+0000 7f2dd2000700 1 -- 172.21.15.86:0/4192680443 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2dc801a720 con 0x7f2dd410a590 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.346+0000 7f2ddc000700 1 -- 172.21.15.86:0/4192680443 --> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f2dd41082f0 con 0x7f2dbc039670 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.359+0000 7f2dd2000700 1 -- 172.21.15.86:0/4192680443 <== mgr.14120 v2:172.21.15.86:6800/2497304884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2dd41082f0 con 0x7f2dbc039670 2024-04-17T07:01:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.362+0000 7f2ddc000700 1 -- 172.21.15.86:0/4192680443 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f2dbc039670 msgr2=0x7f2dbc03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:56.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.362+0000 7f2ddc000700 1 --2- 172.21.15.86:0/4192680443 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f2dbc039670 0x7f2dbc03bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006fd0 tx=0x7f2dc0006e40 comp rx=0 tx=0).stop 2024-04-17T07:01:56.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.362+0000 7f2ddc000700 1 -- 172.21.15.86:0/4192680443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 msgr2=0x7f2dd419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:56.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.362+0000 7f2ddc000700 1 --2- 172.21.15.86:0/4192680443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd419e400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8006af0 tx=0x7f2dc8011b70 comp rx=0 tx=0).stop 2024-04-17T07:01:56.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.362+0000 7f2ddc000700 1 -- 172.21.15.86:0/4192680443 shutdown_connections 2024-04-17T07:01:56.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.362+0000 7f2ddc000700 1 --2- 172.21.15.86:0/4192680443 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f2dbc039670 0x7f2dbc03bb30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:56.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.362+0000 7f2ddc000700 1 --2- 172.21.15.86:0/4192680443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd410a590 0x7f2dd419e400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:56.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.362+0000 7f2ddc000700 1 -- 172.21.15.86:0/4192680443 >> 172.21.15.86:0/4192680443 conn(0x7f2dd407dc10 msgr2=0x7f2dd4107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:56.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.362+0000 7f2ddc000700 1 -- 172.21.15.86:0/4192680443 shutdown_connections 2024-04-17T07:01:56.006 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:55.363+0000 7f2ddc000700 1 -- 172.21.15.86:0/4192680443 wait complete. 2024-04-17T07:01:56.909 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:56 smithi086 ceph-mon[31733]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:01:57.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-17T07:01:57.538 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.647+0000 7f7256400700 1 Processor -- start 2024-04-17T07:01:57.538 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.647+0000 7f7256400700 1 -- start start 2024-04-17T07:01:57.538 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.647+0000 7f7256400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f725010f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:57.538 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.647+0000 7f7256400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f725010f940 con 0x7f725010ef50 2024-04-17T07:01:57.538 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.648+0000 7f724f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f725010f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:57.538 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.648+0000 7f724f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f725010f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60554/0 (socket says 172.21.15.86:60554) 2024-04-17T07:01:57.538 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.648+0000 7f724f400700 1 -- 172.21.15.86:0/3129539214 learned_addr learned my addr 172.21.15.86:0/3129539214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:57.538 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.648+0000 7f724f400700 1 -- 172.21.15.86:0/3129539214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725010fac0 con 0x7f725010ef50 2024-04-17T07:01:57.538 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.649+0000 7f724f400700 1 --2- 172.21.15.86:0/3129539214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f725010f370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f72400098a0 tx=0x7f7240009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e1902447f768c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.649+0000 7f724e000700 1 -- 172.21.15.86:0/3129539214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7240004160 con 0x7f725010ef50 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.649+0000 7f724e000700 1 -- 172.21.15.86:0/3129539214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7240004300 con 0x7f725010ef50 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.649+0000 7f724e000700 1 -- 172.21.15.86:0/3129539214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7240011a80 con 0x7f725010ef50 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.650+0000 7f7256400700 1 -- 172.21.15.86:0/3129539214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 msgr2=0x7f725010f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.650+0000 7f7256400700 1 --2- 172.21.15.86:0/3129539214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f725010f370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f72400098a0 tx=0x7f7240009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.650+0000 7f7256400700 1 -- 172.21.15.86:0/3129539214 shutdown_connections 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.650+0000 7f7256400700 1 --2- 172.21.15.86:0/3129539214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f725010f370 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.650+0000 7f7256400700 1 -- 172.21.15.86:0/3129539214 >> 172.21.15.86:0/3129539214 conn(0x7f725007de30 msgr2=0x7f7250080250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.650+0000 7f7256400700 1 -- 172.21.15.86:0/3129539214 shutdown_connections 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.650+0000 7f7256400700 1 -- 172.21.15.86:0/3129539214 wait complete. 2024-04-17T07:01:57.539 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.651+0000 7f7256400700 1 Processor -- start 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.651+0000 7f7256400700 1 -- start start 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.652+0000 7f7256400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f72501a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.652+0000 7f7256400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7250110a80 con 0x7f725010ef50 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.652+0000 7f724f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f72501a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.652+0000 7f724f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f72501a27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60556/0 (socket says 172.21.15.86:60556) 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.652+0000 7f724f400700 1 -- 172.21.15.86:0/57500789 learned_addr learned my addr 172.21.15.86:0/57500789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.652+0000 7f724f400700 1 -- 172.21.15.86:0/57500789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72501a2d10 con 0x7f725010ef50 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.653+0000 7f724f400700 1 --2- 172.21.15.86:0/57500789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f72501a27d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7240006af0 tx=0x7f7240011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.653+0000 7f7247e00700 1 -- 172.21.15.86:0/57500789 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7240003930 con 0x7f725010ef50 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.653+0000 7f7247e00700 1 -- 172.21.15.86:0/57500789 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7240003ad0 con 0x7f725010ef50 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.653+0000 7f7247e00700 1 -- 172.21.15.86:0/57500789 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72400195e0 con 0x7f725010ef50 2024-04-17T07:01:57.540 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.653+0000 7f7256400700 1 -- 172.21.15.86:0/57500789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72501a2fa0 con 0x7f725010ef50 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.653+0000 7f7256400700 1 -- 172.21.15.86:0/57500789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72501a5c90 con 0x7f725010ef50 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.654+0000 7f7247e00700 1 -- 172.21.15.86:0/57500789 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7240019890 con 0x7f725010ef50 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.654+0000 7f7256400700 1 -- 172.21.15.86:0/57500789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7250112f30 con 0x7f725010ef50 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.655+0000 7f7247e00700 1 --2- 172.21.15.86:0/57500789 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f7234039670 0x7f723403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.655+0000 7f7247e00700 1 -- 172.21.15.86:0/57500789 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f724004dc50 con 0x7f725010ef50 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.655+0000 7f724ea00700 1 --2- 172.21.15.86:0/57500789 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f7234039670 0x7f723403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.656+0000 7f724ea00700 1 --2- 172.21.15.86:0/57500789 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f7234039670 0x7f723403bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f723c006fd0 tx=0x7f723c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.661+0000 7f7247e00700 1 -- 172.21.15.86:0/57500789 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f724004d5a0 con 0x7f725010ef50 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.887+0000 7f7256400700 1 -- 172.21.15.86:0/57500789 --> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f7250080030 con 0x7f7234039670 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.889+0000 7f7247e00700 1 -- 172.21.15.86:0/57500789 <== mgr.14120 v2:172.21.15.86:6800/2497304884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f7250080030 con 0x7f7234039670 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.892+0000 7f7256400700 1 -- 172.21.15.86:0/57500789 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f7234039670 msgr2=0x7f723403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.892+0000 7f7256400700 1 --2- 172.21.15.86:0/57500789 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f7234039670 0x7f723403bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f723c006fd0 tx=0x7f723c006e40 comp rx=0 tx=0).stop 2024-04-17T07:01:57.541 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.892+0000 7f7256400700 1 -- 172.21.15.86:0/57500789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 msgr2=0x7f72501a27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:57.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.892+0000 7f7256400700 1 --2- 172.21.15.86:0/57500789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f72501a27d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7240006af0 tx=0x7f7240011b70 comp rx=0 tx=0).stop 2024-04-17T07:01:57.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.892+0000 7f7256400700 1 -- 172.21.15.86:0/57500789 shutdown_connections 2024-04-17T07:01:57.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.892+0000 7f7256400700 1 --2- 172.21.15.86:0/57500789 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f7234039670 0x7f723403bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:57.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.892+0000 7f7256400700 1 --2- 172.21.15.86:0/57500789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725010ef50 0x7f72501a27d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:57.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.892+0000 7f7256400700 1 -- 172.21.15.86:0/57500789 >> 172.21.15.86:0/57500789 conn(0x7f725007de30 msgr2=0x7f725007f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:57.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.892+0000 7f7256400700 1 -- 172.21.15.86:0/57500789 shutdown_connections 2024-04-17T07:01:57.542 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:56.892+0000 7f7256400700 1 -- 172.21.15.86:0/57500789 wait complete. 2024-04-17T07:01:57.542 INFO:teuthology.orchestra.run.smithi086.stdout:Generating ssh key... 2024-04-17T07:01:58.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:57 smithi086 ceph-mon[31733]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: Generating public/private rsa key pair. 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: Your identification has been saved in /tmp/tmpm0n7as22/key. 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: Your public key has been saved in /tmp/tmpm0n7as22/key.pub. 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: The key fingerprint is: 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: SHA256:iiG4NPp+piOOt4l8NUJNxICCiR8HhWTx16I8trh6SHU ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: The key's randomart image is: 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: +---[RSA 3072]----+ 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: |oo**=. | 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: |=oo..o . | 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: |.. o+ o . | 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: | ..+ E . | 2024-04-17T07:01:59.284 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: |.o+ B S | 2024-04-17T07:01:59.285 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: |oo.= B . | 2024-04-17T07:01:59.285 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: |+.. = o | 2024-04-17T07:01:59.285 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: |=o++o | 2024-04-17T07:01:59.285 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: |=OO* | 2024-04-17T07:01:59.285 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:01:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: +----[SHA256]-----+ 2024-04-17T07:01:59.633 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.147+0000 7fa969800700 1 Processor -- start 2024-04-17T07:01:59.633 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.147+0000 7fa969800700 1 -- start start 2024-04-17T07:01:59.634 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.148+0000 7fa969800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:59.634 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.148+0000 7fa969800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa96407c860 con 0x7fa96407de20 2024-04-17T07:01:59.634 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.148+0000 7fa962a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:59.634 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.148+0000 7fa962a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40840/0 (socket says 172.21.15.86:40840) 2024-04-17T07:01:59.635 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.148+0000 7fa962a00700 1 -- 172.21.15.86:0/76158285 learned_addr learned my addr 172.21.15.86:0/76158285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:59.635 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.149+0000 7fa962a00700 1 -- 172.21.15.86:0/76158285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96407c9e0 con 0x7fa96407de20 2024-04-17T07:01:59.635 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.149+0000 7fa962a00700 1 --2- 172.21.15.86:0/76158285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96407c320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098a0 tx=0x7fa954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a177cee0fced45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:59.636 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.149+0000 7fa961600700 1 -- 172.21.15.86:0/76158285 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa954004160 con 0x7fa96407de20 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.149+0000 7fa961600700 1 -- 172.21.15.86:0/76158285 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa954004300 con 0x7fa96407de20 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.149+0000 7fa961600700 1 -- 172.21.15.86:0/76158285 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa954011a80 con 0x7fa96407de20 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.150+0000 7fa969800700 1 -- 172.21.15.86:0/76158285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 msgr2=0x7fa96407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.150+0000 7fa969800700 1 --2- 172.21.15.86:0/76158285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96407c320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098a0 tx=0x7fa954009bb0 comp rx=0 tx=0).stop 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.150+0000 7fa969800700 1 -- 172.21.15.86:0/76158285 shutdown_connections 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.150+0000 7fa969800700 1 --2- 172.21.15.86:0/76158285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96407c320 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.150+0000 7fa969800700 1 -- 172.21.15.86:0/76158285 >> 172.21.15.86:0/76158285 conn(0x7fa9641039f0 msgr2=0x7fa964105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.151+0000 7fa969800700 1 -- 172.21.15.86:0/76158285 shutdown_connections 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.151+0000 7fa969800700 1 -- 172.21.15.86:0/76158285 wait complete. 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.152+0000 7fa969800700 1 Processor -- start 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.152+0000 7fa969800700 1 -- start start 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.152+0000 7fa969800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.152+0000 7fa969800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa96410e860 con 0x7fa96407de20 2024-04-17T07:01:59.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.152+0000 7fa962a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:59.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.152+0000 7fa962a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40850/0 (socket says 172.21.15.86:40850) 2024-04-17T07:01:59.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.152+0000 7fa962a00700 1 -- 172.21.15.86:0/1973951818 learned_addr learned my addr 172.21.15.86:0/1973951818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:01:59.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.153+0000 7fa962a00700 1 -- 172.21.15.86:0/1973951818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96419e8b0 con 0x7fa96407de20 2024-04-17T07:01:59.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.153+0000 7fa962a00700 1 --2- 172.21.15.86:0/1973951818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96419e370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa954011a50 tx=0x7fa954004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:59.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.153+0000 7fa95b400700 1 -- 172.21.15.86:0/1973951818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9540039f0 con 0x7fa96407de20 2024-04-17T07:01:59.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.153+0000 7fa95b400700 1 -- 172.21.15.86:0/1973951818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa954003b90 con 0x7fa96407de20 2024-04-17T07:01:59.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.153+0000 7fa969800700 1 -- 172.21.15.86:0/1973951818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96419eb40 con 0x7fa96407de20 2024-04-17T07:01:59.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.153+0000 7fa95b400700 1 -- 172.21.15.86:0/1973951818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa95401a600 con 0x7fa96407de20 2024-04-17T07:01:59.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.154+0000 7fa969800700 1 -- 172.21.15.86:0/1973951818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9641a1830 con 0x7fa96407de20 2024-04-17T07:01:59.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.155+0000 7fa95b400700 1 -- 172.21.15.86:0/1973951818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa954024440 con 0x7fa96407de20 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.155+0000 7fa969800700 1 -- 172.21.15.86:0/1973951818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9640640b0 con 0x7fa96407de20 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.155+0000 7fa95b400700 1 --2- 172.21.15.86:0/1973951818 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa9480396c0 0x7fa94803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.155+0000 7fa95b400700 1 -- 172.21.15.86:0/1973951818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa95404dbb0 con 0x7fa96407de20 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.156+0000 7fa962000700 1 --2- 172.21.15.86:0/1973951818 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa9480396c0 0x7fa94803bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.156+0000 7fa962000700 1 --2- 172.21.15.86:0/1973951818 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa9480396c0 0x7fa94803bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa950006fd0 tx=0x7fa950006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.161+0000 7fa95b400700 1 -- 172.21.15.86:0/1973951818 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa95400f2e0 con 0x7fa96407de20 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.389+0000 7fa969800700 1 -- 172.21.15.86:0/1973951818 --> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa964104bc0 con 0x7fa9480396c0 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.863+0000 7fa95b400700 1 -- 172.21.15.86:0/1973951818 <== mgr.14120 v2:172.21.15.86:6800/2497304884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa964104bc0 con 0x7fa9480396c0 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.866+0000 7fa969800700 1 -- 172.21.15.86:0/1973951818 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa9480396c0 msgr2=0x7fa94803bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.866+0000 7fa969800700 1 --2- 172.21.15.86:0/1973951818 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa9480396c0 0x7fa94803bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa950006fd0 tx=0x7fa950006e40 comp rx=0 tx=0).stop 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.866+0000 7fa969800700 1 -- 172.21.15.86:0/1973951818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 msgr2=0x7fa96419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:01:59.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.866+0000 7fa969800700 1 --2- 172.21.15.86:0/1973951818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96419e370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa954011a50 tx=0x7fa954004290 comp rx=0 tx=0).stop 2024-04-17T07:01:59.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.867+0000 7fa969800700 1 -- 172.21.15.86:0/1973951818 shutdown_connections 2024-04-17T07:01:59.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.867+0000 7fa969800700 1 --2- 172.21.15.86:0/1973951818 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa9480396c0 0x7fa94803bb80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:59.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.867+0000 7fa969800700 1 --2- 172.21.15.86:0/1973951818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa96407de20 0x7fa96419e370 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:01:59.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.867+0000 7fa969800700 1 -- 172.21.15.86:0/1973951818 >> 172.21.15.86:0/1973951818 conn(0x7fa9641039f0 msgr2=0x7fa9641044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:01:59.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.867+0000 7fa969800700 1 -- 172.21.15.86:0/1973951818 shutdown_connections 2024-04-17T07:01:59.643 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:01:58.867+0000 7fa969800700 1 -- 172.21.15.86:0/1973951818 wait complete. 2024-04-17T07:01:59.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:59 smithi086 ceph-mon[31733]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:01:59.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:59 smithi086 ceph-mon[31733]: Generating ssh key... 2024-04-17T07:01:59.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:59 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:01:59.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:01:59 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:02:01.029 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw53rmGFF8KqqZ4oy0aVQ0lEbBlfj1GCJvP4xh+/wzWoZ+egUCWdgV8At2h5bBo7PARB8ZLJayfP6XuDcEskGS1yUrLBQdRACeVBnejfOKwyAZuvokr5Yi0nwy/+yApHEwjxjW6nj7e+CVOYsB0/Yo1NeSRrtVw1yZHkM51Uly+rc0QnfSLWxYOQigWmQXVDqRXhh3IT03A+uolDtMW39tjQK/usz4ZrS74tG/h13Nyl7nizJiElUtnyUh7P7dJkQYaUMWIapO4+3vNXrIGat1gWksEeAXaKDEPEYxytbhUQbuw2KkvtITolxc/WV1S7MhvHAGf2OETHHjwb8dcaqQp6Bn5yyyf1/Y2Gp0PflCZHMkP98X+YBDLrqOxr4resOO2dXuwwDLpNhcemT8A+jO/Gfvdk5q4gqA2EUNqvikZpzSSdXZTbeCv9mwPI0W+bavk0iGG5xggTviUeGPgxLEX4AJ95+0+yrcTn6mc+tXhifJ7w7XAIAjxltGt5mnc1M= ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:02:01.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.183+0000 7f46f2800700 1 Processor -- start 2024-04-17T07:02:01.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.183+0000 7f46f2800700 1 -- start start 2024-04-17T07:02:01.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.183+0000 7f46f2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec10a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:01.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.183+0000 7f46f2800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10ad90 con 0x7f46ec10a3a0 2024-04-17T07:02:01.034 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.184+0000 7f46f1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec10a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.184+0000 7f46f1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec10a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40866/0 (socket says 172.21.15.86:40866) 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.184+0000 7f46f1400700 1 -- 172.21.15.86:0/703231192 learned_addr learned my addr 172.21.15.86:0/703231192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.185+0000 7f46f1400700 1 -- 172.21.15.86:0/703231192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec10b550 con 0x7f46ec10a3a0 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.185+0000 7f46f1400700 1 --2- 172.21.15.86:0/703231192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec10a7c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46e00098a0 tx=0x7f46e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f7a85bdec12b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.185+0000 7f46ebe00700 1 -- 172.21.15.86:0/703231192 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46e0004160 con 0x7f46ec10a3a0 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.186+0000 7f46ebe00700 1 -- 172.21.15.86:0/703231192 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f46e0004300 con 0x7f46ec10a3a0 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.186+0000 7f46ebe00700 1 -- 172.21.15.86:0/703231192 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46e0011a80 con 0x7f46ec10a3a0 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.186+0000 7f46f2800700 1 -- 172.21.15.86:0/703231192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 msgr2=0x7f46ec10a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.186+0000 7f46f2800700 1 --2- 172.21.15.86:0/703231192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec10a7c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46e00098a0 tx=0x7f46e0009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.186+0000 7f46f2800700 1 -- 172.21.15.86:0/703231192 shutdown_connections 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.186+0000 7f46f2800700 1 --2- 172.21.15.86:0/703231192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec10a7c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.186+0000 7f46f2800700 1 -- 172.21.15.86:0/703231192 >> 172.21.15.86:0/703231192 conn(0x7f46ec07da10 msgr2=0x7f46ec1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.186+0000 7f46f2800700 1 -- 172.21.15.86:0/703231192 shutdown_connections 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.187+0000 7f46f2800700 1 -- 172.21.15.86:0/703231192 wait complete. 2024-04-17T07:02:01.035 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.188+0000 7f46f2800700 1 Processor -- start 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.188+0000 7f46f2800700 1 -- start start 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.188+0000 7f46f2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec19e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.188+0000 7f46f2800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec10e900 con 0x7f46ec10a3a0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.188+0000 7f46f1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec19e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.188+0000 7f46f1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec19e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40872/0 (socket says 172.21.15.86:40872) 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.188+0000 7f46f1400700 1 -- 172.21.15.86:0/4074950357 learned_addr learned my addr 172.21.15.86:0/4074950357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.189+0000 7f46f1400700 1 -- 172.21.15.86:0/4074950357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec19e700 con 0x7f46ec10a3a0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.189+0000 7f46f1400700 1 --2- 172.21.15.86:0/4074950357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec19e1c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f46e0000c00 tx=0x7f46e0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.189+0000 7f46ea000700 1 -- 172.21.15.86:0/4074950357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46e00039f0 con 0x7f46ec10a3a0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.189+0000 7f46ea000700 1 -- 172.21.15.86:0/4074950357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f46e0003b90 con 0x7f46ec10a3a0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.189+0000 7f46ea000700 1 -- 172.21.15.86:0/4074950357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46e001a740 con 0x7f46ec10a3a0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.189+0000 7f46f2800700 1 -- 172.21.15.86:0/4074950357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec19e990 con 0x7f46ec10a3a0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.190+0000 7f46f2800700 1 -- 172.21.15.86:0/4074950357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec1a1680 con 0x7f46ec10a3a0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.190+0000 7f46ea000700 1 -- 172.21.15.86:0/4074950357 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f46e0024af0 con 0x7f46ec10a3a0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.190+0000 7f46f2800700 1 -- 172.21.15.86:0/4074950357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec0640b0 con 0x7f46ec10a3a0 2024-04-17T07:02:01.036 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.191+0000 7f46ea000700 1 --2- 172.21.15.86:0/4074950357 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f46d40396c0 0x7f46d403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.191+0000 7f46ea000700 1 -- 172.21.15.86:0/4074950357 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f46e004dab0 con 0x7f46ec10a3a0 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.191+0000 7f46f0a00700 1 --2- 172.21.15.86:0/4074950357 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f46d40396c0 0x7f46d403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.192+0000 7f46f0a00700 1 --2- 172.21.15.86:0/4074950357 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f46d40396c0 0x7f46d403bb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46d8006fd0 tx=0x7f46d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.196+0000 7f46ea000700 1 -- 172.21.15.86:0/4074950357 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f46e0020db0 con 0x7f46ec10a3a0 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.417+0000 7f46f2800700 1 -- 172.21.15.86:0/4074950357 --> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f46ec1082b0 con 0x7f46d40396c0 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.419+0000 7f46ea000700 1 -- 172.21.15.86:0/4074950357 <== mgr.14120 v2:172.21.15.86:6800/2497304884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f46ec1082b0 con 0x7f46d40396c0 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.421+0000 7f46f2800700 1 -- 172.21.15.86:0/4074950357 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f46d40396c0 msgr2=0x7f46d403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.421+0000 7f46f2800700 1 --2- 172.21.15.86:0/4074950357 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f46d40396c0 0x7f46d403bb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46d8006fd0 tx=0x7f46d8006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.421+0000 7f46f2800700 1 -- 172.21.15.86:0/4074950357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 msgr2=0x7f46ec19e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.421+0000 7f46f2800700 1 --2- 172.21.15.86:0/4074950357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec19e1c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f46e0000c00 tx=0x7f46e0011b00 comp rx=0 tx=0).stop 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.422+0000 7f46f2800700 1 -- 172.21.15.86:0/4074950357 shutdown_connections 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.422+0000 7f46f2800700 1 --2- 172.21.15.86:0/4074950357 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f46d40396c0 0x7f46d403bb80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.422+0000 7f46f2800700 1 --2- 172.21.15.86:0/4074950357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec10a3a0 0x7f46ec19e1c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:01.037 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.422+0000 7f46f2800700 1 -- 172.21.15.86:0/4074950357 >> 172.21.15.86:0/4074950357 conn(0x7f46ec07da10 msgr2=0x7f46ec107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:01.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.422+0000 7f46f2800700 1 -- 172.21.15.86:0/4074950357 shutdown_connections 2024-04-17T07:02:01.038 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:00.422+0000 7f46f2800700 1 -- 172.21.15.86:0/4074950357 wait complete. 2024-04-17T07:02:01.038 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-17T07:02:01.038 INFO:teuthology.orchestra.run.smithi086.stdout:Adding key to root@localhost authorized_keys... 2024-04-17T07:02:01.038 INFO:teuthology.orchestra.run.smithi086.stdout:Adding host smithi086... 2024-04-17T07:02:01.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:01 smithi086 ceph-mon[31733]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:02:02.307 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:02 smithi086 ceph-mon[31733]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:02:04.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:04 smithi086 ceph-mon[31733]: Deploying cephadm binary to smithi086 2024-04-17T07:02:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Added host 'smithi086' with addr '172.21.15.86' 2024-04-17T07:02:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.644+0000 7f9b93e00700 1 Processor -- start 2024-04-17T07:02:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.644+0000 7f9b93e00700 1 -- start start 2024-04-17T07:02:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.645+0000 7f9b93e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9410a3a0 0x7f9b9410a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.645+0000 7f9b93e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9410ad90 con 0x7f9b9410a3a0 2024-04-17T07:02:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.645+0000 7f9b92a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9410a3a0 0x7f9b9410a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.646+0000 7f9b92a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9410a3a0 0x7f9b9410a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40874/0 (socket says 172.21.15.86:40874) 2024-04-17T07:02:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.646+0000 7f9b92a00700 1 -- 172.21.15.86:0/975708165 learned_addr learned my addr 172.21.15.86:0/975708165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.646+0000 7f9b92a00700 1 -- 172.21.15.86:0/975708165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9410b550 con 0x7f9b9410a3a0 2024-04-17T07:02:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.647+0000 7f9b92a00700 1 --2- 172.21.15.86:0/975708165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9410a3a0 0x7f9b9410a7c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b840098a0 tx=0x7f9b84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4490088d885554 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.647+0000 7f9b91600700 1 -- 172.21.15.86:0/975708165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b84004160 con 0x7f9b9410a3a0 2024-04-17T07:02:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.647+0000 7f9b91600700 1 -- 172.21.15.86:0/975708165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9b84004300 con 0x7f9b9410a3a0 2024-04-17T07:02:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.647+0000 7f9b91600700 1 -- 172.21.15.86:0/975708165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b84011a80 con 0x7f9b9410a3a0 2024-04-17T07:02:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.648+0000 7f9b93e00700 1 -- 172.21.15.86:0/975708165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9410a3a0 msgr2=0x7f9b9410a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.648+0000 7f9b93e00700 1 --2- 172.21.15.86:0/975708165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9410a3a0 0x7f9b9410a7c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b840098a0 tx=0x7f9b84009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.648+0000 7f9b93e00700 1 -- 172.21.15.86:0/975708165 shutdown_connections 2024-04-17T07:02:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.648+0000 7f9b93e00700 1 --2- 172.21.15.86:0/975708165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9410a3a0 0x7f9b9410a7c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.648+0000 7f9b93e00700 1 -- 172.21.15.86:0/975708165 >> 172.21.15.86:0/975708165 conn(0x7f9b9407da10 msgr2=0x7f9b941084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.648+0000 7f9b93e00700 1 -- 172.21.15.86:0/975708165 shutdown_connections 2024-04-17T07:02:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.648+0000 7f9b93e00700 1 -- 172.21.15.86:0/975708165 wait complete. 2024-04-17T07:02:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.649+0000 7f9b93e00700 1 Processor -- start 2024-04-17T07:02:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.650+0000 7f9b93e00700 1 -- start start 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.650+0000 7f9b93e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9419e410 0x7f9b9419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.650+0000 7f9b93e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9410e900 con 0x7f9b9419e410 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.650+0000 7f9b92a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9419e410 0x7f9b9419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.650+0000 7f9b92a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9419e410 0x7f9b9419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40882/0 (socket says 172.21.15.86:40882) 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.650+0000 7f9b92a00700 1 -- 172.21.15.86:0/1299491824 learned_addr learned my addr 172.21.15.86:0/1299491824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.651+0000 7f9b92a00700 1 -- 172.21.15.86:0/1299491824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9419ed70 con 0x7f9b9419e410 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.651+0000 7f9b92a00700 1 --2- 172.21.15.86:0/1299491824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9419e410 0x7f9b9419e830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9b84011740 tx=0x7f9b84011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.651+0000 7f9b8b400700 1 -- 172.21.15.86:0/1299491824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b840038b0 con 0x7f9b9419e410 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.651+0000 7f9b8b400700 1 -- 172.21.15.86:0/1299491824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f9b84003a50 con 0x7f9b9419e410 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.651+0000 7f9b8b400700 1 -- 172.21.15.86:0/1299491824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b8401a5d0 con 0x7f9b9419e410 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.651+0000 7f9b93e00700 1 -- 172.21.15.86:0/1299491824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9419f000 con 0x7f9b9419e410 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.651+0000 7f9b93e00700 1 -- 172.21.15.86:0/1299491824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b941a1ae0 con 0x7f9b9419e410 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.652+0000 7f9b93e00700 1 -- 172.21.15.86:0/1299491824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b940640b0 con 0x7f9b9419e410 2024-04-17T07:02:09.087 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.653+0000 7f9b8b400700 1 -- 172.21.15.86:0/1299491824 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9b84026480 con 0x7f9b9419e410 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.653+0000 7f9b8b400700 1 --2- 172.21.15.86:0/1299491824 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9b78039670 0x7f9b7803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.653+0000 7f9b8b400700 1 -- 172.21.15.86:0/1299491824 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9b8404dc60 con 0x7f9b9419e410 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.653+0000 7f9b92000700 1 --2- 172.21.15.86:0/1299491824 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9b78039670 0x7f9b7803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.654+0000 7f9b92000700 1 --2- 172.21.15.86:0/1299491824 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9b78039670 0x7f9b7803bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b80006fd0 tx=0x7f9b80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.659+0000 7f9b8b400700 1 -- 172.21.15.86:0/1299491824 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9b8400be50 con 0x7f9b9419e410 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:01.887+0000 7f9b93e00700 1 -- 172.21.15.86:0/1299491824 --> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}) -- 0x7f9b94108340 con 0x7f9b78039670 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.366+0000 7f9b8b400700 1 -- 172.21.15.86:0/1299491824 <== mgr.14120 v2:172.21.15.86:6800/2497304884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9b94108340 con 0x7f9b78039670 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.370+0000 7f9b93e00700 1 -- 172.21.15.86:0/1299491824 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9b78039670 msgr2=0x7f9b7803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.370+0000 7f9b93e00700 1 --2- 172.21.15.86:0/1299491824 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9b78039670 0x7f9b7803bb30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b80006fd0 tx=0x7f9b80006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.370+0000 7f9b93e00700 1 -- 172.21.15.86:0/1299491824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9419e410 msgr2=0x7f9b9419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.370+0000 7f9b93e00700 1 --2- 172.21.15.86:0/1299491824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9419e410 0x7f9b9419e830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9b84011740 tx=0x7f9b84011b00 comp rx=0 tx=0).stop 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.371+0000 7f9b93e00700 1 -- 172.21.15.86:0/1299491824 shutdown_connections 2024-04-17T07:02:09.088 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.371+0000 7f9b93e00700 1 --2- 172.21.15.86:0/1299491824 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f9b78039670 0x7f9b7803bb30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:09.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.371+0000 7f9b93e00700 1 --2- 172.21.15.86:0/1299491824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b9419e410 0x7f9b9419e830 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:09.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.371+0000 7f9b93e00700 1 -- 172.21.15.86:0/1299491824 >> 172.21.15.86:0/1299491824 conn(0x7f9b9407da10 msgr2=0x7f9b94107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:09.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.371+0000 7f9b93e00700 1 -- 172.21.15.86:0/1299491824 shutdown_connections 2024-04-17T07:02:09.089 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:08.371+0000 7f9b93e00700 1 -- 172.21.15.86:0/1299491824 wait complete. 2024-04-17T07:02:09.089 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mon service... 2024-04-17T07:02:09.659 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:09 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:02:09.659 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:09 smithi086 ceph-mon[31733]: Added host smithi086 2024-04-17T07:02:09.659 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:09 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:02:11.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:11 smithi086 ceph-mon[31733]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:02:11.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:11 smithi086 ceph-mon[31733]: Saving service mon spec with placement count:5 2024-04-17T07:02:11.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:11 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:02:11.364 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-17T07:02:11.365 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.825+0000 7f4734a00700 1 Processor -- start 2024-04-17T07:02:11.365 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.825+0000 7f4734a00700 1 -- start start 2024-04-17T07:02:11.365 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.826+0000 7f4734a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f4730106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:11.365 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.826+0000 7f4734a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f473007be70 con 0x7f4730106bb0 2024-04-17T07:02:11.365 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.826+0000 7f472de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f4730106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.826+0000 7f472de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f4730106fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48170/0 (socket says 172.21.15.86:48170) 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.826+0000 7f472de00700 1 -- 172.21.15.86:0/2904178762 learned_addr learned my addr 172.21.15.86:0/2904178762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.827+0000 7f472de00700 1 -- 172.21.15.86:0/2904178762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4730107510 con 0x7f4730106bb0 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.827+0000 7f472de00700 1 --2- 172.21.15.86:0/2904178762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f4730106fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4720009e80 tx=0x7f472000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8affc76990fcfb1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.827+0000 7f472d400700 1 -- 172.21.15.86:0/2904178762 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4720017070 con 0x7f4730106bb0 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.827+0000 7f472d400700 1 -- 172.21.15.86:0/2904178762 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f472000b8c0 con 0x7f4730106bb0 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.827+0000 7f472d400700 1 -- 172.21.15.86:0/2904178762 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4720004520 con 0x7f4730106bb0 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.828+0000 7f4734a00700 1 -- 172.21.15.86:0/2904178762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 msgr2=0x7f4730106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.828+0000 7f4734a00700 1 --2- 172.21.15.86:0/2904178762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f4730106fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4720009e80 tx=0x7f472000b330 comp rx=0 tx=0).stop 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.828+0000 7f4734a00700 1 -- 172.21.15.86:0/2904178762 shutdown_connections 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.828+0000 7f4734a00700 1 --2- 172.21.15.86:0/2904178762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f4730106fd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:11.367 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.828+0000 7f4734a00700 1 -- 172.21.15.86:0/2904178762 >> 172.21.15.86:0/2904178762 conn(0x7f4730102830 msgr2=0x7f4730104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:11.368 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.828+0000 7f4734a00700 1 -- 172.21.15.86:0/2904178762 shutdown_connections 2024-04-17T07:02:11.368 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.828+0000 7f4734a00700 1 -- 172.21.15.86:0/2904178762 wait complete. 2024-04-17T07:02:11.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.829+0000 7f4734a00700 1 Processor -- start 2024-04-17T07:02:11.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.829+0000 7f4734a00700 1 -- start start 2024-04-17T07:02:11.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.830+0000 7f4734a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f473019e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:11.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.830+0000 7f4734a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4730108540 con 0x7f4730106bb0 2024-04-17T07:02:11.370 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.830+0000 7f472de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f473019e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.830+0000 7f472de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f473019e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48182/0 (socket says 172.21.15.86:48182) 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.830+0000 7f472de00700 1 -- 172.21.15.86:0/3997163241 learned_addr learned my addr 172.21.15.86:0/3997163241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.830+0000 7f472de00700 1 -- 172.21.15.86:0/3997163241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473019e8b0 con 0x7f4730106bb0 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.831+0000 7f472de00700 1 --2- 172.21.15.86:0/3997163241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f473019e370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f47200042b0 tx=0x7f4720004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.831+0000 7f4726a00700 1 -- 172.21.15.86:0/3997163241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4720020020 con 0x7f4730106bb0 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.831+0000 7f4726a00700 1 -- 172.21.15.86:0/3997163241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f472000bba0 con 0x7f4730106bb0 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.831+0000 7f4734a00700 1 -- 172.21.15.86:0/3997163241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473019eb40 con 0x7f4730106bb0 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.831+0000 7f4726a00700 1 -- 172.21.15.86:0/3997163241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4720017040 con 0x7f4730106bb0 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.831+0000 7f4734a00700 1 -- 172.21.15.86:0/3997163241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47301a1830 con 0x7f4730106bb0 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.832+0000 7f4734a00700 1 -- 172.21.15.86:0/3997163241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47300640b0 con 0x7f4730106bb0 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.832+0000 7f4726a00700 1 -- 172.21.15.86:0/3997163241 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4720023020 con 0x7f4730106bb0 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.833+0000 7f4726a00700 1 --2- 172.21.15.86:0/3997163241 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f47140396c0 0x7f471403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:11.371 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.833+0000 7f4726a00700 1 -- 172.21.15.86:0/3997163241 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f472000f050 con 0x7f4730106bb0 2024-04-17T07:02:11.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.833+0000 7f4727e00700 1 --2- 172.21.15.86:0/3997163241 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f47140396c0 0x7f471403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:11.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.834+0000 7f4727e00700 1 --2- 172.21.15.86:0/3997163241 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f47140396c0 0x7f471403bb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4718006fd0 tx=0x7f4718006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:11.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:09.838+0000 7f4726a00700 1 -- 172.21.15.86:0/3997163241 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f472002a020 con 0x7f4730106bb0 2024-04-17T07:02:11.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.085+0000 7f4734a00700 1 -- 172.21.15.86:0/3997163241 --> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f473002e340 con 0x7f47140396c0 2024-04-17T07:02:11.372 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.091+0000 7f4726a00700 1 -- 172.21.15.86:0/3997163241 <== mgr.14120 v2:172.21.15.86:6800/2497304884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f473002e340 con 0x7f47140396c0 2024-04-17T07:02:11.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.094+0000 7f4734a00700 1 -- 172.21.15.86:0/3997163241 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f47140396c0 msgr2=0x7f471403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:11.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.094+0000 7f4734a00700 1 --2- 172.21.15.86:0/3997163241 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f47140396c0 0x7f471403bb80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4718006fd0 tx=0x7f4718006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:11.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.094+0000 7f4734a00700 1 -- 172.21.15.86:0/3997163241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 msgr2=0x7f473019e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:11.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.094+0000 7f4734a00700 1 --2- 172.21.15.86:0/3997163241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f473019e370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f47200042b0 tx=0x7f4720004670 comp rx=0 tx=0).stop 2024-04-17T07:02:11.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.094+0000 7f4734a00700 1 -- 172.21.15.86:0/3997163241 shutdown_connections 2024-04-17T07:02:11.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.094+0000 7f4734a00700 1 --2- 172.21.15.86:0/3997163241 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f47140396c0 0x7f471403bb80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:11.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.094+0000 7f4734a00700 1 --2- 172.21.15.86:0/3997163241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4730106bb0 0x7f473019e370 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:11.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.094+0000 7f4734a00700 1 -- 172.21.15.86:0/3997163241 >> 172.21.15.86:0/3997163241 conn(0x7f4730102830 msgr2=0x7f4730194fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:11.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.095+0000 7f4734a00700 1 -- 172.21.15.86:0/3997163241 shutdown_connections 2024-04-17T07:02:11.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:10.095+0000 7f4734a00700 1 -- 172.21.15.86:0/3997163241 wait complete. 2024-04-17T07:02:11.374 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mgr service... 2024-04-17T07:02:13.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-17T07:02:13.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.039+0000 7f831b200700 1 Processor -- start 2024-04-17T07:02:13.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.039+0000 7f831b200700 1 -- start start 2024-04-17T07:02:13.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.039+0000 7f831b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:13.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.040+0000 7f831b200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f831407c840 con 0x7f831407de50 2024-04-17T07:02:13.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.040+0000 7f8319e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:13.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.040+0000 7f8319e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48192/0 (socket says 172.21.15.86:48192) 2024-04-17T07:02:13.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.040+0000 7f8319e00700 1 -- 172.21.15.86:0/297274140 learned_addr learned my addr 172.21.15.86:0/297274140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:13.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.040+0000 7f8319e00700 1 -- 172.21.15.86:0/297274140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831407c9c0 con 0x7f831407de50 2024-04-17T07:02:13.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.041+0000 7f8319e00700 1 --2- 172.21.15.86:0/297274140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831407c300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f83080098a0 tx=0x7f8308009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe9a8788ac162d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.041+0000 7f8318a00700 1 -- 172.21.15.86:0/297274140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8308004160 con 0x7f831407de50 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.041+0000 7f8318a00700 1 -- 172.21.15.86:0/297274140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8308004300 con 0x7f831407de50 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.041+0000 7f8318a00700 1 -- 172.21.15.86:0/297274140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8308011a80 con 0x7f831407de50 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.042+0000 7f831b200700 1 -- 172.21.15.86:0/297274140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 msgr2=0x7f831407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.042+0000 7f831b200700 1 --2- 172.21.15.86:0/297274140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831407c300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f83080098a0 tx=0x7f8308009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.042+0000 7f831b200700 1 -- 172.21.15.86:0/297274140 shutdown_connections 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.042+0000 7f831b200700 1 --2- 172.21.15.86:0/297274140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831407c300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.042+0000 7f831b200700 1 -- 172.21.15.86:0/297274140 >> 172.21.15.86:0/297274140 conn(0x7f83141037d0 msgr2=0x7f8314105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.042+0000 7f831b200700 1 -- 172.21.15.86:0/297274140 shutdown_connections 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.042+0000 7f831b200700 1 -- 172.21.15.86:0/297274140 wait complete. 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.044+0000 7f831b200700 1 Processor -- start 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.044+0000 7f831b200700 1 -- start start 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.044+0000 7f831b200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831419e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:13.021 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.044+0000 7f831b200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f831410e700 con 0x7f831407de50 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.045+0000 7f8319e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831419e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.045+0000 7f8319e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831419e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48208/0 (socket says 172.21.15.86:48208) 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.045+0000 7f8319e00700 1 -- 172.21.15.86:0/3820154795 learned_addr learned my addr 172.21.15.86:0/3820154795 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.045+0000 7f8319e00700 1 -- 172.21.15.86:0/3820154795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831419e700 con 0x7f831407de50 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.045+0000 7f8319e00700 1 --2- 172.21.15.86:0/3820154795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831419e1c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8308006af0 tx=0x7f8308011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.046+0000 7f830ea00700 1 -- 172.21.15.86:0/3820154795 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8308003930 con 0x7f831407de50 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.046+0000 7f830ea00700 1 -- 172.21.15.86:0/3820154795 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8308003ad0 con 0x7f831407de50 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.046+0000 7f830ea00700 1 -- 172.21.15.86:0/3820154795 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83080195e0 con 0x7f831407de50 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.046+0000 7f831b200700 1 -- 172.21.15.86:0/3820154795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831419e990 con 0x7f831407de50 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.046+0000 7f831b200700 1 -- 172.21.15.86:0/3820154795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83141a1680 con 0x7f831407de50 2024-04-17T07:02:13.022 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.047+0000 7f830ea00700 1 -- 172.21.15.86:0/3820154795 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8308019890 con 0x7f831407de50 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.047+0000 7f831b200700 1 -- 172.21.15.86:0/3820154795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83140640b0 con 0x7f831407de50 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.047+0000 7f830ea00700 1 --2- 172.21.15.86:0/3820154795 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f82fc039670 0x7f82fc03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.047+0000 7f830ea00700 1 -- 172.21.15.86:0/3820154795 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f830804dbd0 con 0x7f831407de50 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.048+0000 7f8319400700 1 --2- 172.21.15.86:0/3820154795 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f82fc039670 0x7f82fc03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.048+0000 7f8319400700 1 --2- 172.21.15.86:0/3820154795 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f82fc039670 0x7f82fc03bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8304006fd0 tx=0x7f8304006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.053+0000 7f830ea00700 1 -- 172.21.15.86:0/3820154795 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8308023e00 con 0x7f831407de50 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.229+0000 7f830ea00700 1 -- 172.21.15.86:0/3820154795 <== mon.0 v2:172.21.15.86:3300/0 7 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8308053080 con 0x7f831407de50 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.320+0000 7f831b200700 1 -- 172.21.15.86:0/3820154795 --> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f83141049f0 con 0x7f82fc039670 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.327+0000 7f830ea00700 1 -- 172.21.15.86:0/3820154795 <== mgr.14120 v2:172.21.15.86:6800/2497304884 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f83141049f0 con 0x7f82fc039670 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.331+0000 7f831b200700 1 -- 172.21.15.86:0/3820154795 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f82fc039670 msgr2=0x7f82fc03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.331+0000 7f831b200700 1 --2- 172.21.15.86:0/3820154795 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f82fc039670 0x7f82fc03bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8304006fd0 tx=0x7f8304006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.331+0000 7f831b200700 1 -- 172.21.15.86:0/3820154795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 msgr2=0x7f831419e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:13.023 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.331+0000 7f831b200700 1 --2- 172.21.15.86:0/3820154795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831419e1c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8308006af0 tx=0x7f8308011b70 comp rx=0 tx=0).stop 2024-04-17T07:02:13.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.331+0000 7f831b200700 1 -- 172.21.15.86:0/3820154795 shutdown_connections 2024-04-17T07:02:13.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.331+0000 7f831b200700 1 --2- 172.21.15.86:0/3820154795 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f82fc039670 0x7f82fc03bb30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:13.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.331+0000 7f831b200700 1 --2- 172.21.15.86:0/3820154795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f831407de50 0x7f831419e1c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:13.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.331+0000 7f831b200700 1 -- 172.21.15.86:0/3820154795 >> 172.21.15.86:0/3820154795 conn(0x7f83141037d0 msgr2=0x7f83141042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:13.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.331+0000 7f831b200700 1 -- 172.21.15.86:0/3820154795 shutdown_connections 2024-04-17T07:02:13.024 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:12.331+0000 7f831b200700 1 -- 172.21.15.86:0/3820154795 wait complete. 2024-04-17T07:02:13.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:13 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:02:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:13 smithi086 ceph-mon[31733]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:02:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:13 smithi086 ceph-mon[31733]: Saving service mgr spec with placement count:2 2024-04-17T07:02:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:13 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:02:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:13 smithi086 ceph-mon[31733]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:02:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:13 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:02:14.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.611+0000 7f989da00700 1 Processor -- start 2024-04-17T07:02:14.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.612+0000 7f989da00700 1 -- start start 2024-04-17T07:02:14.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.612+0000 7f989da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f9898108750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:14.547 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.612+0000 7f989da00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9898108c90 con 0x7f9898108330 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.613+0000 7f9896a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f9898108750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.613+0000 7f9896a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f9898108750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48222/0 (socket says 172.21.15.86:48222) 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.613+0000 7f9896a00700 1 -- 172.21.15.86:0/962837778 learned_addr learned my addr 172.21.15.86:0/962837778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.613+0000 7f9896a00700 1 -- 172.21.15.86:0/962837778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9898108e10 con 0x7f9898108330 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.613+0000 7f9896a00700 1 --2- 172.21.15.86:0/962837778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f9898108750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f98880098a0 tx=0x7f9888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fff9d168d469020 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.614+0000 7f9895600700 1 -- 172.21.15.86:0/962837778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9888004160 con 0x7f9898108330 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.614+0000 7f9895600700 1 -- 172.21.15.86:0/962837778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9888004300 con 0x7f9898108330 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.614+0000 7f9895600700 1 -- 172.21.15.86:0/962837778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9888011a50 con 0x7f9898108330 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.615+0000 7f989da00700 1 -- 172.21.15.86:0/962837778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 msgr2=0x7f9898108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.615+0000 7f989da00700 1 --2- 172.21.15.86:0/962837778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f9898108750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f98880098a0 tx=0x7f9888009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.615+0000 7f989da00700 1 -- 172.21.15.86:0/962837778 shutdown_connections 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.615+0000 7f989da00700 1 --2- 172.21.15.86:0/962837778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f9898108750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:14.548 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.615+0000 7f989da00700 1 -- 172.21.15.86:0/962837778 >> 172.21.15.86:0/962837778 conn(0x7f98981039b0 msgr2=0x7f9898105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.615+0000 7f989da00700 1 -- 172.21.15.86:0/962837778 shutdown_connections 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.615+0000 7f989da00700 1 -- 172.21.15.86:0/962837778 wait complete. 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.616+0000 7f989da00700 1 Processor -- start 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.616+0000 7f989da00700 1 -- start start 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.616+0000 7f989da00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f989819e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.616+0000 7f989da00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f989810a370 con 0x7f9898108330 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.617+0000 7f9896a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f989819e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.617+0000 7f9896a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f989819e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48236/0 (socket says 172.21.15.86:48236) 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.617+0000 7f9896a00700 1 -- 172.21.15.86:0/4104242095 learned_addr learned my addr 172.21.15.86:0/4104242095 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.617+0000 7f9896a00700 1 -- 172.21.15.86:0/4104242095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989819e890 con 0x7f9898108330 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.617+0000 7f9896a00700 1 --2- 172.21.15.86:0/4104242095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f989819e350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9888011a00 tx=0x7f9888011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.618+0000 7f988f400700 1 -- 172.21.15.86:0/4104242095 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9888003af0 con 0x7f9898108330 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.618+0000 7f988f400700 1 -- 172.21.15.86:0/4104242095 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f988801a8e0 con 0x7f9898108330 2024-04-17T07:02:14.549 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.618+0000 7f989da00700 1 -- 172.21.15.86:0/4104242095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989819eb20 con 0x7f9898108330 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.618+0000 7f988f400700 1 -- 172.21.15.86:0/4104242095 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9888024840 con 0x7f9898108330 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.618+0000 7f989da00700 1 -- 172.21.15.86:0/4104242095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98981a1810 con 0x7f9898108330 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.619+0000 7f988f400700 1 -- 172.21.15.86:0/4104242095 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9888022020 con 0x7f9898108330 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.619+0000 7f989da00700 1 -- 172.21.15.86:0/4104242095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98980640b0 con 0x7f9898108330 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.620+0000 7f988f400700 1 --2- 172.21.15.86:0/4104242095 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f987c039720 0x7f987c03bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.620+0000 7f988f400700 1 -- 172.21.15.86:0/4104242095 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f988804daf0 con 0x7f9898108330 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.620+0000 7f9896000700 1 --2- 172.21.15.86:0/4104242095 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f987c039720 0x7f987c03bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.621+0000 7f9896000700 1 --2- 172.21.15.86:0/4104242095 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f987c039720 0x7f987c03bbe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9884006fd0 tx=0x7f9884006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.625+0000 7f988f400700 1 -- 172.21.15.86:0/4104242095 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f988804fe20 con 0x7f9898108330 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.850+0000 7f989da00700 1 -- 172.21.15.86:0/4104242095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f98981a1dc0 con 0x7f9898108330 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.853+0000 7f988f400700 1 -- 172.21.15.86:0/4104242095 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f988800b3e0 con 0x7f9898108330 2024-04-17T07:02:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.856+0000 7f989da00700 1 -- 172.21.15.86:0/4104242095 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f987c039720 msgr2=0x7f987c03bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:14.551 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.856+0000 7f989da00700 1 --2- 172.21.15.86:0/4104242095 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f987c039720 0x7f987c03bbe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9884006fd0 tx=0x7f9884006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:14.551 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.856+0000 7f989da00700 1 -- 172.21.15.86:0/4104242095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 msgr2=0x7f989819e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:14.551 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.856+0000 7f989da00700 1 --2- 172.21.15.86:0/4104242095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f989819e350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9888011a00 tx=0x7f9888011ae0 comp rx=0 tx=0).stop 2024-04-17T07:02:14.551 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.856+0000 7f989da00700 1 -- 172.21.15.86:0/4104242095 shutdown_connections 2024-04-17T07:02:14.551 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.856+0000 7f989da00700 1 --2- 172.21.15.86:0/4104242095 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f987c039720 0x7f987c03bbe0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:14.551 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.856+0000 7f989da00700 1 --2- 172.21.15.86:0/4104242095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9898108330 0x7f989819e350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:14.551 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.856+0000 7f989da00700 1 -- 172.21.15.86:0/4104242095 >> 172.21.15.86:0/4104242095 conn(0x7f98981039b0 msgr2=0x7f9898104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:14.551 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.857+0000 7f989da00700 1 -- 172.21.15.86:0/4104242095 shutdown_connections 2024-04-17T07:02:14.551 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:13.857+0000 7f989da00700 1 -- 172.21.15.86:0/4104242095 wait complete. 2024-04-17T07:02:15.164 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:14 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4104242095' entity='client.admin' 2024-04-17T07:02:16.310 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:15 smithi086 ceph-mon[31733]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:02:16.310 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:15 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3490095600' entity='client.admin' 2024-04-17T07:02:16.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.339+0000 7f752cc00700 1 Processor -- start 2024-04-17T07:02:16.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.339+0000 7f752cc00700 1 -- start start 2024-04-17T07:02:16.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.339+0000 7f752cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752810b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:16.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.339+0000 7f752cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f752807bed0 con 0x7f7528108d60 2024-04-17T07:02:16.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.340+0000 7f7525e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752810b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:16.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.340+0000 7f7525e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752810b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48246/0 (socket says 172.21.15.86:48246) 2024-04-17T07:02:16.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.340+0000 7f7525e00700 1 -- 172.21.15.86:0/2610247533 learned_addr learned my addr 172.21.15.86:0/2610247533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:16.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.340+0000 7f7525e00700 1 -- 172.21.15.86:0/2610247533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752810b690 con 0x7f7528108d60 2024-04-17T07:02:16.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.341+0000 7f7525e00700 1 --2- 172.21.15.86:0/2610247533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752810b150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f75180098a0 tx=0x7f7518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dc2057ca3024a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:16.325 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.341+0000 7f7524a00700 1 -- 172.21.15.86:0/2610247533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7518004160 con 0x7f7528108d60 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.341+0000 7f7524a00700 1 -- 172.21.15.86:0/2610247533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7518004300 con 0x7f7528108d60 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.341+0000 7f7524a00700 1 -- 172.21.15.86:0/2610247533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7518011a50 con 0x7f7528108d60 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.342+0000 7f752cc00700 1 -- 172.21.15.86:0/2610247533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 msgr2=0x7f752810b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.342+0000 7f752cc00700 1 --2- 172.21.15.86:0/2610247533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752810b150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f75180098a0 tx=0x7f7518009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.342+0000 7f752cc00700 1 -- 172.21.15.86:0/2610247533 shutdown_connections 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.342+0000 7f752cc00700 1 --2- 172.21.15.86:0/2610247533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752810b150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.342+0000 7f752cc00700 1 -- 172.21.15.86:0/2610247533 >> 172.21.15.86:0/2610247533 conn(0x7f7528102830 msgr2=0x7f7528104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.342+0000 7f752cc00700 1 -- 172.21.15.86:0/2610247533 shutdown_connections 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.342+0000 7f752cc00700 1 -- 172.21.15.86:0/2610247533 wait complete. 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.343+0000 7f752cc00700 1 Processor -- start 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.343+0000 7f752cc00700 1 -- start start 2024-04-17T07:02:16.326 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.343+0000 7f752cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.343+0000 7f752cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f752810c6a0 con 0x7f7528108d60 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.344+0000 7f7525e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.344+0000 7f7525e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752819e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48252/0 (socket says 172.21.15.86:48252) 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.344+0000 7f7525e00700 1 -- 172.21.15.86:0/3490095600 learned_addr learned my addr 172.21.15.86:0/3490095600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.344+0000 7f7525e00700 1 -- 172.21.15.86:0/3490095600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752819e8b0 con 0x7f7528108d60 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.344+0000 7f7525e00700 1 --2- 172.21.15.86:0/3490095600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752819e370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7518011a20 tx=0x7f7518011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.345+0000 7f751ea00700 1 -- 172.21.15.86:0/3490095600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75180039c0 con 0x7f7528108d60 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.345+0000 7f751ea00700 1 -- 172.21.15.86:0/3490095600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7518003b60 con 0x7f7528108d60 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.345+0000 7f752cc00700 1 -- 172.21.15.86:0/3490095600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752819eb40 con 0x7f7528108d60 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.345+0000 7f751ea00700 1 -- 172.21.15.86:0/3490095600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f751801a6b0 con 0x7f7528108d60 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.345+0000 7f752cc00700 1 -- 172.21.15.86:0/3490095600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75281a1830 con 0x7f7528108d60 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.346+0000 7f752cc00700 1 -- 172.21.15.86:0/3490095600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75280640b0 con 0x7f7528108d60 2024-04-17T07:02:16.327 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.346+0000 7f751ea00700 1 -- 172.21.15.86:0/3490095600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7518024430 con 0x7f7528108d60 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.346+0000 7f751ea00700 1 --2- 172.21.15.86:0/3490095600 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f750c039670 0x7f750c03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.346+0000 7f751ea00700 1 -- 172.21.15.86:0/3490095600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f751804dd70 con 0x7f7528108d60 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.347+0000 7f7525400700 1 --2- 172.21.15.86:0/3490095600 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f750c039670 0x7f750c03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.347+0000 7f7525400700 1 --2- 172.21.15.86:0/3490095600 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f750c039670 0x7f750c03bb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7514006fd0 tx=0x7f7514006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.352+0000 7f751ea00700 1 -- 172.21.15.86:0/3490095600 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7518017ca0 con 0x7f7528108d60 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.576+0000 7f752cc00700 1 -- 172.21.15.86:0/3490095600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f75281a1e50 con 0x7f7528108d60 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.580+0000 7f751ea00700 1 -- 172.21.15.86:0/3490095600 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f7518024a70 con 0x7f7528108d60 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.583+0000 7f752cc00700 1 -- 172.21.15.86:0/3490095600 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f750c039670 msgr2=0x7f750c03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.583+0000 7f752cc00700 1 --2- 172.21.15.86:0/3490095600 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f750c039670 0x7f750c03bb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7514006fd0 tx=0x7f7514006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.583+0000 7f752cc00700 1 -- 172.21.15.86:0/3490095600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 msgr2=0x7f752819e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.583+0000 7f752cc00700 1 --2- 172.21.15.86:0/3490095600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752819e370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7518011a20 tx=0x7f7518011b00 comp rx=0 tx=0).stop 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.584+0000 7f752cc00700 1 -- 172.21.15.86:0/3490095600 shutdown_connections 2024-04-17T07:02:16.328 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.584+0000 7f752cc00700 1 --2- 172.21.15.86:0/3490095600 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f750c039670 0x7f750c03bb30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:16.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.584+0000 7f752cc00700 1 --2- 172.21.15.86:0/3490095600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528108d60 0x7f752819e370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:16.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.584+0000 7f752cc00700 1 -- 172.21.15.86:0/3490095600 >> 172.21.15.86:0/3490095600 conn(0x7f7528102830 msgr2=0x7f7528103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:16.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.584+0000 7f752cc00700 1 -- 172.21.15.86:0/3490095600 shutdown_connections 2024-04-17T07:02:16.329 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:15.584+0000 7f752cc00700 1 -- 172.21.15.86:0/3490095600 wait complete. 2024-04-17T07:02:16.329 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling the dashboard module... 2024-04-17T07:02:17.175 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:16 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:02:18.286 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:17 smithi086 ceph-mon[31733]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:02:18.286 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:17 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/561941831' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-17T07:02:18.286 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:17 smithi086 ceph-mon[31733]: from='mgr.14120 172.21.15.86:0/4208603152' entity='mgr.x' 2024-04-17T07:02:18.286 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:18 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: ignoring --setuser ceph since I am not root 2024-04-17T07:02:18.287 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:18 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: ignoring --setgroup ceph since I am not root 2024-04-17T07:02:18.612 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.867+0000 7fa582c00700 1 Processor -- start 2024-04-17T07:02:18.614 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.867+0000 7fa582c00700 1 -- start start 2024-04-17T07:02:18.614 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.868+0000 7fa582c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:18.614 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.868+0000 7fa582c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c07be90 con 0x7fa57c106bd0 2024-04-17T07:02:18.614 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.868+0000 7fa57be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:18.614 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.868+0000 7fa57be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49278/0 (socket says 172.21.15.86:49278) 2024-04-17T07:02:18.615 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.868+0000 7fa57be00700 1 -- 172.21.15.86:0/2350642855 learned_addr learned my addr 172.21.15.86:0/2350642855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:18.615 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.869+0000 7fa57be00700 1 -- 172.21.15.86:0/2350642855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c107530 con 0x7fa57c106bd0 2024-04-17T07:02:18.615 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.869+0000 7fa57be00700 1 --2- 172.21.15.86:0/2350642855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c106ff0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0098a0 tx=0x7fa56c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553ba741a9d63b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:18.615 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.869+0000 7fa57aa00700 1 -- 172.21.15.86:0/2350642855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa56c004160 con 0x7fa57c106bd0 2024-04-17T07:02:18.616 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.869+0000 7fa57aa00700 1 -- 172.21.15.86:0/2350642855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa56c004300 con 0x7fa57c106bd0 2024-04-17T07:02:18.616 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.869+0000 7fa57aa00700 1 -- 172.21.15.86:0/2350642855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa56c011a50 con 0x7fa57c106bd0 2024-04-17T07:02:18.616 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.870+0000 7fa582c00700 1 -- 172.21.15.86:0/2350642855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 msgr2=0x7fa57c106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:18.616 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.870+0000 7fa582c00700 1 --2- 172.21.15.86:0/2350642855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c106ff0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0098a0 tx=0x7fa56c009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:18.616 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.870+0000 7fa582c00700 1 -- 172.21.15.86:0/2350642855 shutdown_connections 2024-04-17T07:02:18.617 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.870+0000 7fa582c00700 1 --2- 172.21.15.86:0/2350642855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c106ff0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:18.617 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.870+0000 7fa582c00700 1 -- 172.21.15.86:0/2350642855 >> 172.21.15.86:0/2350642855 conn(0x7fa57c102890 msgr2=0x7fa57c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:18.617 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.870+0000 7fa582c00700 1 -- 172.21.15.86:0/2350642855 shutdown_connections 2024-04-17T07:02:18.617 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.870+0000 7fa582c00700 1 -- 172.21.15.86:0/2350642855 wait complete. 2024-04-17T07:02:18.617 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.871+0000 7fa582c00700 1 Processor -- start 2024-04-17T07:02:18.618 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.871+0000 7fa582c00700 1 -- start start 2024-04-17T07:02:18.618 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.872+0000 7fa582c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c19e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:18.618 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.872+0000 7fa582c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c1085a0 con 0x7fa57c106bd0 2024-04-17T07:02:18.618 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.872+0000 7fa57be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c19e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:18.618 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.872+0000 7fa57be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c19e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49288/0 (socket says 172.21.15.86:49288) 2024-04-17T07:02:18.618 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.872+0000 7fa57be00700 1 -- 172.21.15.86:0/561941831 learned_addr learned my addr 172.21.15.86:0/561941831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:18.618 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.872+0000 7fa57be00700 1 -- 172.21.15.86:0/561941831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c19e8c0 con 0x7fa57c106bd0 2024-04-17T07:02:18.619 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.873+0000 7fa57be00700 1 --2- 172.21.15.86:0/561941831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c19e380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa56c011a20 tx=0x7fa56c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:18.619 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.873+0000 7fa578c00700 1 -- 172.21.15.86:0/561941831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa56c0039c0 con 0x7fa57c106bd0 2024-04-17T07:02:18.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.873+0000 7fa578c00700 1 -- 172.21.15.86:0/561941831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa56c003b60 con 0x7fa57c106bd0 2024-04-17T07:02:18.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.873+0000 7fa582c00700 1 -- 172.21.15.86:0/561941831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57c19eb50 con 0x7fa57c106bd0 2024-04-17T07:02:18.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.873+0000 7fa578c00700 1 -- 172.21.15.86:0/561941831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa56c01a6b0 con 0x7fa57c106bd0 2024-04-17T07:02:18.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.873+0000 7fa582c00700 1 -- 172.21.15.86:0/561941831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57c1a1840 con 0x7fa57c106bd0 2024-04-17T07:02:18.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.874+0000 7fa578c00700 1 -- 172.21.15.86:0/561941831 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa56c01a960 con 0x7fa57c106bd0 2024-04-17T07:02:18.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.874+0000 7fa582c00700 1 -- 172.21.15.86:0/561941831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57c0640b0 con 0x7fa57c106bd0 2024-04-17T07:02:18.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.875+0000 7fa578c00700 1 --2- 172.21.15.86:0/561941831 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa560039670 0x7fa56003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:18.620 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.875+0000 7fa578c00700 1 -- 172.21.15.86:0/561941831 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa56c04ded0 con 0x7fa57c106bd0 2024-04-17T07:02:18.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.875+0000 7fa57b400700 1 --2- 172.21.15.86:0/561941831 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa560039670 0x7fa56003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:18.621 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.876+0000 7fa57b400700 1 --2- 172.21.15.86:0/561941831 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa560039670 0x7fa56003bb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa568006fd0 tx=0x7fa568006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:18.622 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:16.881+0000 7fa578c00700 1 -- 172.21.15.86:0/561941831 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa56c00ba30 con 0x7fa57c106bd0 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.175+0000 7fa582c00700 1 -- 172.21.15.86:0/561941831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa57c02e1e0 con 0x7fa57c106bd0 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.915+0000 7fa578c00700 1 -- 172.21.15.86:0/561941831 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa56c04d820 con 0x7fa57c106bd0 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.919+0000 7fa578c00700 1 -- 172.21.15.86:0/561941831 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa56c02d910 con 0x7fa57c106bd0 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.920+0000 7fa582c00700 1 -- 172.21.15.86:0/561941831 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa560039670 msgr2=0x7fa56003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.920+0000 7fa582c00700 1 --2- 172.21.15.86:0/561941831 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa560039670 0x7fa56003bb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa568006fd0 tx=0x7fa568006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.920+0000 7fa582c00700 1 -- 172.21.15.86:0/561941831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 msgr2=0x7fa57c19e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.920+0000 7fa582c00700 1 --2- 172.21.15.86:0/561941831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c19e380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa56c011a20 tx=0x7fa56c011b00 comp rx=0 tx=0).stop 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.921+0000 7fa582c00700 1 -- 172.21.15.86:0/561941831 shutdown_connections 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.921+0000 7fa582c00700 1 --2- 172.21.15.86:0/561941831 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7fa560039670 0x7fa56003bb30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.921+0000 7fa582c00700 1 --2- 172.21.15.86:0/561941831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c106bd0 0x7fa57c19e380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.921+0000 7fa582c00700 1 -- 172.21.15.86:0/561941831 >> 172.21.15.86:0/561941831 conn(0x7fa57c102890 msgr2=0x7fa57c194ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.921+0000 7fa582c00700 1 -- 172.21.15.86:0/561941831 shutdown_connections 2024-04-17T07:02:18.623 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:17.922+0000 7fa582c00700 1 -- 172.21.15.86:0/561941831 wait complete. 2024-04-17T07:02:18.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:18 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:18.740+0000 7ffaf9396200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T07:02:19.135 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:18 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/561941831' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-17T07:02:19.136 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:18 smithi086 ceph-mon[31733]: mgrmap e9: x(active, since 25s) 2024-04-17T07:02:19.136 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:18 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:18.865+0000 7ffaf9396200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T07:02:20.001 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:19 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/193895606' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-17T07:02:20.002 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:19 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:19.695+0000 7ffaf9396200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T07:02:20.002 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:19 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:19.820+0000 7ffaf9396200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T07:02:20.002 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:20.001+0000 7ffaf9396200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T07:02:20.177 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:02:20.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-17T07:02:20.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-17T07:02:20.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-17T07:02:20.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-17T07:02:20.178 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.208+0000 7f107fa00700 1 Processor -- start 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.208+0000 7f107fa00700 1 -- start start 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.208+0000 7f107fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.208+0000 7f107fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f107807c840 con 0x7f107807de00 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.209+0000 7f107d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.209+0000 7f107d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49316/0 (socket says 172.21.15.86:49316) 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.209+0000 7f107d000700 1 -- 172.21.15.86:0/384651733 learned_addr learned my addr 172.21.15.86:0/384651733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.209+0000 7f107d000700 1 -- 172.21.15.86:0/384651733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107807c9c0 con 0x7f107807de00 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.210+0000 7f107d000700 1 --2- 172.21.15.86:0/384651733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107807c300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f106c0098a0 tx=0x7f106c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6502714f768ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.210+0000 7f1077400700 1 -- 172.21.15.86:0/384651733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f106c004160 con 0x7f107807de00 2024-04-17T07:02:20.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.210+0000 7f1077400700 1 -- 172.21.15.86:0/384651733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f106c004300 con 0x7f107807de00 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.210+0000 7f1077400700 1 -- 172.21.15.86:0/384651733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f106c011a50 con 0x7f107807de00 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.211+0000 7f107fa00700 1 -- 172.21.15.86:0/384651733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 msgr2=0x7f107807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.211+0000 7f107fa00700 1 --2- 172.21.15.86:0/384651733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107807c300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f106c0098a0 tx=0x7f106c009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.211+0000 7f107fa00700 1 -- 172.21.15.86:0/384651733 shutdown_connections 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.211+0000 7f107fa00700 1 --2- 172.21.15.86:0/384651733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107807c300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.211+0000 7f107fa00700 1 -- 172.21.15.86:0/384651733 >> 172.21.15.86:0/384651733 conn(0x7f10781039f0 msgr2=0x7f1078105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.211+0000 7f107fa00700 1 -- 172.21.15.86:0/384651733 shutdown_connections 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.211+0000 7f107fa00700 1 -- 172.21.15.86:0/384651733 wait complete. 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.212+0000 7f107fa00700 1 Processor -- start 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.212+0000 7f107fa00700 1 -- start start 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.212+0000 7f107fa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107819e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.212+0000 7f107fa00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f107810e8d0 con 0x7f107807de00 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.213+0000 7f107d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107819e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:20.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.213+0000 7f107d000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107819e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49322/0 (socket says 172.21.15.86:49322) 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.213+0000 7f107d000700 1 -- 172.21.15.86:0/193895606 learned_addr learned my addr 172.21.15.86:0/193895606 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.213+0000 7f107d000700 1 -- 172.21.15.86:0/193895606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107819e820 con 0x7f107807de00 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.213+0000 7f107d000700 1 --2- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107819e2e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f106c011a00 tx=0x7f106c011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.214+0000 7f1075600700 1 -- 172.21.15.86:0/193895606 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f106c003af0 con 0x7f107807de00 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.214+0000 7f1075600700 1 -- 172.21.15.86:0/193895606 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f106c01a8e0 con 0x7f107807de00 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.214+0000 7f1075600700 1 -- 172.21.15.86:0/193895606 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f106c024840 con 0x7f107807de00 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.214+0000 7f107fa00700 1 -- 172.21.15.86:0/193895606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107819eab0 con 0x7f107807de00 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.214+0000 7f107fa00700 1 -- 172.21.15.86:0/193895606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10781a17a0 con 0x7f107807de00 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.215+0000 7f107fa00700 1 -- 172.21.15.86:0/193895606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10780640b0 con 0x7f107807de00 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.215+0000 7f1075600700 1 -- 172.21.15.86:0/193895606 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f106c022020 con 0x7f107807de00 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.216+0000 7f1075600700 1 --2- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f1060039720 0x7f106003bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.216+0000 7f1075600700 1 -- 172.21.15.86:0/193895606 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f106c04db50 con 0x7f107807de00 2024-04-17T07:02:20.186 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.216+0000 7f1077e00700 1 -- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f1060039720 msgr2=0x7f106003bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2497304884 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.216+0000 7f1077e00700 1 --2- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f1060039720 0x7f106003bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.221+0000 7f1075600700 1 -- 172.21.15.86:0/193895606 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f106c00b3e0 con 0x7f107807de00 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.417+0000 7f1077e00700 1 -- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f1060039720 msgr2=0x7f106003bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2497304884 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.417+0000 7f1077e00700 1 --2- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f1060039720 0x7f106003bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.504+0000 7f107fa00700 1 -- 172.21.15.86:0/193895606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f10781a1a10 con 0x7f107807de00 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.504+0000 7f1075600700 1 -- 172.21.15.86:0/193895606 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f106c02ddb0 con 0x7f107807de00 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.507+0000 7f107fa00700 1 -- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f1060039720 msgr2=0x7f106003bbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.507+0000 7f107fa00700 1 --2- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f1060039720 0x7f106003bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.507+0000 7f107fa00700 1 -- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 msgr2=0x7f107819e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.507+0000 7f107fa00700 1 --2- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107819e2e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f106c011a00 tx=0x7f106c011ae0 comp rx=0 tx=0).stop 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.507+0000 7f107fa00700 1 -- 172.21.15.86:0/193895606 shutdown_connections 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.507+0000 7f107fa00700 1 --2- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f1060039720 0x7f106003bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.507+0000 7f107fa00700 1 --2- 172.21.15.86:0/193895606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107807de00 0x7f107819e2e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:20.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.507+0000 7f107fa00700 1 -- 172.21.15.86:0/193895606 >> 172.21.15.86:0/193895606 conn(0x7f10781039f0 msgr2=0x7f10781043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:20.188 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.508+0000 7f107fa00700 1 -- 172.21.15.86:0/193895606 shutdown_connections 2024-04-17T07:02:20.188 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:19.508+0000 7f107fa00700 1 -- 172.21.15.86:0/193895606 wait complete. 2024-04-17T07:02:20.188 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-04-17T07:02:20.188 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 9... 2024-04-17T07:02:20.329 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:20.127+0000 7ffaf9396200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T07:02:20.330 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:20.240+0000 7ffaf9396200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T07:02:20.846 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:20.584+0000 7ffaf9396200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T07:02:20.846 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:20.731+0000 7ffaf9396200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T07:02:20.847 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:20.846+0000 7ffaf9396200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T07:02:21.189 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:20.970+0000 7ffaf9396200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T07:02:21.579 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:21 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:21.188+0000 7ffaf9396200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T07:02:22.830 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:22 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:22.469+0000 7ffaf9396200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T07:02:22.830 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:22 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:22.646+0000 7ffaf9396200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T07:02:23.330 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:23.068+0000 7ffaf9396200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T07:02:23.330 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:23.190+0000 7ffaf9396200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T07:02:23.778 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:23.452+0000 7ffaf9396200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:02:23.779 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:23.702+0000 7ffaf9396200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T07:02:24.405 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:24.088+0000 7ffaf9396200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:02:24.405 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:24.404+0000 7ffaf9396200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T07:02:25.080 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:24.716+0000 7ffaf9396200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T07:02:26.080 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:25 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:25.751+0000 7ffaf9396200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T07:02:26.829 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:26.458+0000 7ffaf9396200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T07:02:26.830 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:26.588+0000 7ffaf9396200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T07:02:27.091 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:02:26.877+0000 7ffaf9396200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T07:02:27.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-mon[31733]: Active manager daemon x restarted 2024-04-17T07:02:27.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-mon[31733]: Activating manager daemon x 2024-04-17T07:02:27.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-mon[31733]: osdmap e3: 0 total, 0 up, 0 in 2024-04-17T07:02:27.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-mon[31733]: mgrmap e10: x(active, starting, since 0.00915553s) 2024-04-17T07:02:27.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:02:27.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-17T07:02:27.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-17T07:02:27.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-17T07:02:27.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-17T07:02:27.092 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:26 smithi086 ceph-mon[31733]: Manager daemon x is now available 2024-04-17T07:02:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:02:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-17T07:02:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-17T07:02:28.237 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:27 smithi086 ceph-mon[31733]: mgrmap e11: x(active, since 1.01227s) 2024-04-17T07:02:28.764 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-17T07:02:28.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-17T07:02:28.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-17T07:02:28.772 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.765+0000 7f25be400700 1 Processor -- start 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.765+0000 7f25be400700 1 -- start start 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.766+0000 7f25be400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b807c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.766+0000 7f25be400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25b807c880 con 0x7f25b807de40 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.766+0000 7f25b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b807c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.766+0000 7f25b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b807c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49332/0 (socket says 172.21.15.86:49332) 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.766+0000 7f25b7400700 1 -- 172.21.15.86:0/390795421 learned_addr learned my addr 172.21.15.86:0/390795421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.767+0000 7f25b7400700 1 -- 172.21.15.86:0/390795421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b807ca00 con 0x7f25b807de40 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.767+0000 7f25b7400700 1 --2- 172.21.15.86:0/390795421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b807c340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25a80098a0 tx=0x7f25a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e072a9f78f82da server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.767+0000 7f25b6a00700 1 -- 172.21.15.86:0/390795421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a8004160 con 0x7f25b807de40 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.768+0000 7f25b6a00700 1 -- 172.21.15.86:0/390795421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25a8004300 con 0x7f25b807de40 2024-04-17T07:02:28.773 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.768+0000 7f25b6a00700 1 -- 172.21.15.86:0/390795421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a8011a50 con 0x7f25b807de40 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.768+0000 7f25be400700 1 -- 172.21.15.86:0/390795421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 msgr2=0x7f25b807c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.768+0000 7f25be400700 1 --2- 172.21.15.86:0/390795421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b807c340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25a80098a0 tx=0x7f25a8009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.768+0000 7f25be400700 1 -- 172.21.15.86:0/390795421 shutdown_connections 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.768+0000 7f25be400700 1 --2- 172.21.15.86:0/390795421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b807c340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.768+0000 7f25be400700 1 -- 172.21.15.86:0/390795421 >> 172.21.15.86:0/390795421 conn(0x7f25b81039f0 msgr2=0x7f25b8105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.768+0000 7f25be400700 1 -- 172.21.15.86:0/390795421 shutdown_connections 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.769+0000 7f25be400700 1 -- 172.21.15.86:0/390795421 wait complete. 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.770+0000 7f25be400700 1 Processor -- start 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.770+0000 7f25be400700 1 -- start start 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.770+0000 7f25be400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.770+0000 7f25be400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810e8f0 con 0x7f25b807de40 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.770+0000 7f25b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:28.774 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.770+0000 7f25b7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b819e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49336/0 (socket says 172.21.15.86:49336) 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.770+0000 7f25b7400700 1 -- 172.21.15.86:0/2569680554 learned_addr learned my addr 172.21.15.86:0/2569680554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.771+0000 7f25b7400700 1 -- 172.21.15.86:0/2569680554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b819e840 con 0x7f25b807de40 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.771+0000 7f25b7400700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b819e300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25a8011a00 tx=0x7f25a8011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.771+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a8003af0 con 0x7f25b807de40 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.771+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25a801a8e0 con 0x7f25b807de40 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.771+0000 7f25be400700 1 -- 172.21.15.86:0/2569680554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b819ead0 con 0x7f25b807de40 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.771+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25a8024840 con 0x7f25b807de40 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.772+0000 7f25be400700 1 -- 172.21.15.86:0/2569680554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b81a17c0 con 0x7f25b807de40 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.773+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f25a8022020 con 0x7f25b807de40 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.773+0000 7f25b4c00700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 0x7f259c03bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.773+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 --> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f259c03c2f0 con 0x7f259c039720 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.773+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f25a804dc90 con 0x7f25b807de40 2024-04-17T07:02:28.775 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.773+0000 7f25afe00700 1 -- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 msgr2=0x7f259c03bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2497304884 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.773+0000 7f25afe00700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 0x7f259c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.974+0000 7f25afe00700 1 -- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 msgr2=0x7f259c03bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2497304884 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:20.974+0000 7f25afe00700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 0x7f259c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:21.375+0000 7f25afe00700 1 -- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 msgr2=0x7f259c03bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2497304884 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:21.375+0000 7f25afe00700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 0x7f259c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:22.176+0000 7f25afe00700 1 -- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 msgr2=0x7f259c03bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2497304884 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:22.176+0000 7f25afe00700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 0x7f259c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:23.778+0000 7f25afe00700 1 -- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 msgr2=0x7f259c03bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2497304884 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:23.778+0000 7f25afe00700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 0x7f259c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:26.887+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f25a8024af0 con 0x7f25b807de40 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:26.887+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 msgr2=0x7f259c03bbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:26.887+0000 7f25b4c00700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 0x7f259c03bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:28.776 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.889+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f25a804fd70 con 0x7f25b807de40 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.889+0000 7f25b4c00700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f259c03cfc0 0x7f259c03f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.890+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f259c03c2f0 con 0x7f259c03cfc0 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.891+0000 7f25afe00700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f259c03cfc0 0x7f259c03f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.892+0000 7f25afe00700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f259c03cfc0 0x7f259c03f3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25a0003a10 tx=0x7f25a0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.893+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f259c03c2f0 con 0x7f259c03cfc0 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.902+0000 7f25be400700 1 -- 172.21.15.86:0/2569680554 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f25b80640b0 con 0x7f259c03cfc0 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.902+0000 7f25b4c00700 1 -- 172.21.15.86:0/2569680554 <== mgr.14152 v2:172.21.15.86:6800/561620030 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f25b80640b0 con 0x7f259c03cfc0 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.902+0000 7f25be400700 1 -- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f259c03cfc0 msgr2=0x7f259c03f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.902+0000 7f25be400700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f259c03cfc0 0x7f259c03f3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25a0003a10 tx=0x7f25a0009270 comp rx=0 tx=0).stop 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.902+0000 7f25be400700 1 -- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 msgr2=0x7f25b819e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.902+0000 7f25be400700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b819e300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25a8011a00 tx=0x7f25a8011ae0 comp rx=0 tx=0).stop 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.903+0000 7f25be400700 1 -- 172.21.15.86:0/2569680554 shutdown_connections 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.903+0000 7f25be400700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f259c03cfc0 0x7f259c03f3b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:28.777 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.903+0000 7f25be400700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b807de40 0x7f25b819e300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:28.778 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.903+0000 7f25be400700 1 --2- 172.21.15.86:0/2569680554 >> [v2:172.21.15.86:6800/2497304884,v1:172.21.15.86:6801/2497304884] conn(0x7f259c039720 0x7f259c03bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:28.778 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.903+0000 7f25be400700 1 -- 172.21.15.86:0/2569680554 >> 172.21.15.86:0/2569680554 conn(0x7f25b81039f0 msgr2=0x7f25b81043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:28.778 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.903+0000 7f25be400700 1 -- 172.21.15.86:0/2569680554 shutdown_connections 2024-04-17T07:02:28.778 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:27.903+0000 7f25be400700 1 -- 172.21.15.86:0/2569680554 wait complete. 2024-04-17T07:02:28.778 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 9 is available 2024-04-17T07:02:28.778 INFO:teuthology.orchestra.run.smithi086.stdout:Generating a dashboard self-signed certificate... 2024-04-17T07:02:29.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:28 smithi086 ceph-mon[31733]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-17T07:02:29.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:28 smithi086 ceph-mon[31733]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-17T07:02:30.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:29 smithi086 ceph-mon[31733]: [17/Apr/2024:07:02:28] ENGINE Bus STARTING 2024-04-17T07:02:30.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:29 smithi086 ceph-mon[31733]: [17/Apr/2024:07:02:28] ENGINE Serving on https://172.21.15.86:7150 2024-04-17T07:02:30.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:29 smithi086 ceph-mon[31733]: [17/Apr/2024:07:02:28] ENGINE Serving on http://172.21.15.86:8765 2024-04-17T07:02:30.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:29 smithi086 ceph-mon[31733]: [17/Apr/2024:07:02:28] ENGINE Bus STARTED 2024-04-17T07:02:30.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:29 smithi086 ceph-mon[31733]: mgrmap e12: x(active, since 2s) 2024-04-17T07:02:30.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:30.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:30.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:30.637 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-17T07:02:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.513+0000 7f4186800700 1 Processor -- start 2024-04-17T07:02:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.513+0000 7f4186800700 1 -- start start 2024-04-17T07:02:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.513+0000 7f4186800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f418010f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.513+0000 7f4186800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f418010f920 con 0x7f418010ef50 2024-04-17T07:02:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.514+0000 7f417f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f418010f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.514+0000 7f417f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f418010f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47496/0 (socket says 172.21.15.86:47496) 2024-04-17T07:02:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.514+0000 7f417f400700 1 -- 172.21.15.86:0/970354984 learned_addr learned my addr 172.21.15.86:0/970354984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.514+0000 7f417f400700 1 -- 172.21.15.86:0/970354984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418010faa0 con 0x7f418010ef50 2024-04-17T07:02:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.515+0000 7f417f400700 1 --2- 172.21.15.86:0/970354984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f418010f350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41700098a0 tx=0x7f4170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d84bc1dd4a41761 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.515+0000 7f417e000700 1 -- 172.21.15.86:0/970354984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4170004160 con 0x7f418010ef50 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.515+0000 7f417e000700 1 -- 172.21.15.86:0/970354984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4170004300 con 0x7f418010ef50 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.515+0000 7f417e000700 1 -- 172.21.15.86:0/970354984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4170011a50 con 0x7f418010ef50 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.515+0000 7f4186800700 1 -- 172.21.15.86:0/970354984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 msgr2=0x7f418010f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.515+0000 7f4186800700 1 --2- 172.21.15.86:0/970354984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f418010f350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41700098a0 tx=0x7f4170009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.516+0000 7f4186800700 1 -- 172.21.15.86:0/970354984 shutdown_connections 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.516+0000 7f4186800700 1 --2- 172.21.15.86:0/970354984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f418010f350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.516+0000 7f4186800700 1 -- 172.21.15.86:0/970354984 >> 172.21.15.86:0/970354984 conn(0x7f418007de30 msgr2=0x7f4180080250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.516+0000 7f4186800700 1 -- 172.21.15.86:0/970354984 shutdown_connections 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.516+0000 7f4186800700 1 -- 172.21.15.86:0/970354984 wait complete. 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.517+0000 7f4186800700 1 Processor -- start 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.517+0000 7f4186800700 1 -- start start 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.517+0000 7f4186800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f41801a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:30.639 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.517+0000 7f4186800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4180110b10 con 0x7f418010ef50 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.517+0000 7f417f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f41801a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.517+0000 7f417f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f41801a27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47506/0 (socket says 172.21.15.86:47506) 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.517+0000 7f417f400700 1 -- 172.21.15.86:0/3807108598 learned_addr learned my addr 172.21.15.86:0/3807108598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.518+0000 7f417f400700 1 -- 172.21.15.86:0/3807108598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41801a2d00 con 0x7f418010ef50 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.518+0000 7f417f400700 1 --2- 172.21.15.86:0/3807108598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f41801a27c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4170011710 tx=0x7f4170011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.518+0000 7f4177e00700 1 -- 172.21.15.86:0/3807108598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41700039c0 con 0x7f418010ef50 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.518+0000 7f4177e00700 1 -- 172.21.15.86:0/3807108598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4170003b60 con 0x7f418010ef50 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.518+0000 7f4186800700 1 -- 172.21.15.86:0/3807108598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41801a2f90 con 0x7f418010ef50 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.518+0000 7f4177e00700 1 -- 172.21.15.86:0/3807108598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f417001a690 con 0x7f418010ef50 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.519+0000 7f4186800700 1 -- 172.21.15.86:0/3807108598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801a33f0 con 0x7f418010ef50 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.520+0000 7f4177e00700 1 -- 172.21.15.86:0/3807108598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f4170024430 con 0x7f418010ef50 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.520+0000 7f4186800700 1 -- 172.21.15.86:0/3807108598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41800640b0 con 0x7f418010ef50 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.520+0000 7f4177e00700 1 --2- 172.21.15.86:0/3807108598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4164039670 0x7f416403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:30.640 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.520+0000 7f4177e00700 1 -- 172.21.15.86:0/3807108598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f417004dee0 con 0x7f418010ef50 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.520+0000 7f417ea00700 1 --2- 172.21.15.86:0/3807108598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4164039670 0x7f416403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.521+0000 7f417ea00700 1 --2- 172.21.15.86:0/3807108598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4164039670 0x7f416403bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f416c006fd0 tx=0x7f416c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.526+0000 7f4177e00700 1 -- 172.21.15.86:0/3807108598 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4170024a70 con 0x7f418010ef50 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.758+0000 7f4186800700 1 -- 172.21.15.86:0/3807108598 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f418007fee0 con 0x7f4164039670 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.899+0000 7f4177e00700 1 -- 172.21.15.86:0/3807108598 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f418007fee0 con 0x7f4164039670 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.902+0000 7f4186800700 1 -- 172.21.15.86:0/3807108598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4164039670 msgr2=0x7f416403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.902+0000 7f4186800700 1 --2- 172.21.15.86:0/3807108598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4164039670 0x7f416403bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f416c006fd0 tx=0x7f416c006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.902+0000 7f4186800700 1 -- 172.21.15.86:0/3807108598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 msgr2=0x7f41801a27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.902+0000 7f4186800700 1 --2- 172.21.15.86:0/3807108598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f41801a27c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4170011710 tx=0x7f4170011ad0 comp rx=0 tx=0).stop 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.902+0000 7f4186800700 1 -- 172.21.15.86:0/3807108598 shutdown_connections 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.902+0000 7f4186800700 1 --2- 172.21.15.86:0/3807108598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4164039670 0x7f416403bb30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.902+0000 7f4186800700 1 --2- 172.21.15.86:0/3807108598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f418010ef50 0x7f41801a27c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:30.641 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.902+0000 7f4186800700 1 -- 172.21.15.86:0/3807108598 >> 172.21.15.86:0/3807108598 conn(0x7f418007de30 msgr2=0x7f418007f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:30.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.903+0000 7f4186800700 1 -- 172.21.15.86:0/3807108598 shutdown_connections 2024-04-17T07:02:30.642 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:29.903+0000 7f4186800700 1 -- 172.21.15.86:0/3807108598 wait complete. 2024-04-17T07:02:30.642 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial admin user... 2024-04-17T07:02:31.241 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:30 smithi086 ceph-mon[31733]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:02:32.458 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$RWBR8Jxb819nfXaUDxmBXeXtEVdOsLZL5mFFo0qXPPMfx0LYRvHeS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713337351, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.240+0000 7f71f6c00700 1 Processor -- start 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.240+0000 7f71f6c00700 1 -- start start 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.241+0000 7f71f6c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f010aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.241+0000 7f71f6c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71f010b080 con 0x7f71f010a690 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.241+0000 7f71efe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f010aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.241+0000 7f71efe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f010aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47508/0 (socket says 172.21.15.86:47508) 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.241+0000 7f71efe00700 1 -- 172.21.15.86:0/2811971554 learned_addr learned my addr 172.21.15.86:0/2811971554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.242+0000 7f71efe00700 1 -- 172.21.15.86:0/2811971554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f010b7f0 con 0x7f71f010a690 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.242+0000 7f71efe00700 1 --2- 172.21.15.86:0/2811971554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f010aab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71e00098a0 tx=0x7f71e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849037db1e4789d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.242+0000 7f71ef400700 1 -- 172.21.15.86:0/2811971554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71e0004160 con 0x7f71f010a690 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.242+0000 7f71ef400700 1 -- 172.21.15.86:0/2811971554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e0004300 con 0x7f71f010a690 2024-04-17T07:02:32.459 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.242+0000 7f71ef400700 1 -- 172.21.15.86:0/2811971554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71e0011a50 con 0x7f71f010a690 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.243+0000 7f71f6c00700 1 -- 172.21.15.86:0/2811971554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 msgr2=0x7f71f010aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.243+0000 7f71f6c00700 1 --2- 172.21.15.86:0/2811971554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f010aab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71e00098a0 tx=0x7f71e0009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.243+0000 7f71f6c00700 1 -- 172.21.15.86:0/2811971554 shutdown_connections 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.243+0000 7f71f6c00700 1 --2- 172.21.15.86:0/2811971554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f010aab0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.243+0000 7f71f6c00700 1 -- 172.21.15.86:0/2811971554 >> 172.21.15.86:0/2811971554 conn(0x7f71f007dc10 msgr2=0x7f71f01084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.243+0000 7f71f6c00700 1 -- 172.21.15.86:0/2811971554 shutdown_connections 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.244+0000 7f71f6c00700 1 -- 172.21.15.86:0/2811971554 wait complete. 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.245+0000 7f71f6c00700 1 Processor -- start 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.245+0000 7f71f6c00700 1 -- start start 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.245+0000 7f71f6c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.245+0000 7f71f6c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71f010eba0 con 0x7f71f010a690 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.245+0000 7f71efe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.245+0000 7f71efe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47510/0 (socket says 172.21.15.86:47510) 2024-04-17T07:02:32.460 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.245+0000 7f71efe00700 1 -- 172.21.15.86:0/3810067683 learned_addr learned my addr 172.21.15.86:0/3810067683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.246+0000 7f71efe00700 1 -- 172.21.15.86:0/3810067683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f019e930 con 0x7f71f010a690 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.247+0000 7f71efe00700 1 --2- 172.21.15.86:0/3810067683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f019e3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f71e0011a20 tx=0x7f71e0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.247+0000 7f71ed600700 1 -- 172.21.15.86:0/3810067683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71e0003af0 con 0x7f71f010a690 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.247+0000 7f71ed600700 1 -- 172.21.15.86:0/3810067683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e001a8e0 con 0x7f71f010a690 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.247+0000 7f71f6c00700 1 -- 172.21.15.86:0/3810067683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f019ebc0 con 0x7f71f010a690 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.247+0000 7f71ed600700 1 -- 172.21.15.86:0/3810067683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71e0024860 con 0x7f71f010a690 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.247+0000 7f71f6c00700 1 -- 172.21.15.86:0/3810067683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f01a18b0 con 0x7f71f010a690 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.248+0000 7f71ed600700 1 -- 172.21.15.86:0/3810067683 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f71e0022020 con 0x7f71f010a690 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.248+0000 7f71f6c00700 1 -- 172.21.15.86:0/3810067683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f00640b0 con 0x7f71f010a690 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.249+0000 7f71ed600700 1 --2- 172.21.15.86:0/3810067683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71d4039720 0x7f71d403bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.249+0000 7f71ed600700 1 -- 172.21.15.86:0/3810067683 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f71e004dbd0 con 0x7f71f010a690 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.249+0000 7f71e7400700 1 --2- 172.21.15.86:0/3810067683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71d4039720 0x7f71d403bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:32.461 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.250+0000 7f71e7400700 1 --2- 172.21.15.86:0/3810067683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71d4039720 0x7f71d403bbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f71d800ad30 tx=0x7f71d80093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.255+0000 7f71ed600700 1 -- 172.21.15.86:0/3810067683 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f71e000b3e0 con 0x7f71f010a690 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.501+0000 7f71f6c00700 1 -- 172.21.15.86:0/3810067683 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f71f01a1a80 con 0x7f71d4039720 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.733+0000 7f71ed600700 1 -- 172.21.15.86:0/3810067683 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f71f01a1a80 con 0x7f71d4039720 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.735+0000 7f71f6c00700 1 -- 172.21.15.86:0/3810067683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71d4039720 msgr2=0x7f71d403bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.735+0000 7f71f6c00700 1 --2- 172.21.15.86:0/3810067683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71d4039720 0x7f71d403bbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f71d800ad30 tx=0x7f71d80093f0 comp rx=0 tx=0).stop 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.735+0000 7f71f6c00700 1 -- 172.21.15.86:0/3810067683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 msgr2=0x7f71f019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.735+0000 7f71f6c00700 1 --2- 172.21.15.86:0/3810067683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f019e3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f71e0011a20 tx=0x7f71e0011b00 comp rx=0 tx=0).stop 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.736+0000 7f71f6c00700 1 -- 172.21.15.86:0/3810067683 shutdown_connections 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.736+0000 7f71f6c00700 1 --2- 172.21.15.86:0/3810067683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71d4039720 0x7f71d403bbe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.736+0000 7f71f6c00700 1 --2- 172.21.15.86:0/3810067683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71f010a690 0x7f71f019e3f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.736+0000 7f71f6c00700 1 -- 172.21.15.86:0/3810067683 >> 172.21.15.86:0/3810067683 conn(0x7f71f007dc10 msgr2=0x7f71f0107ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.736+0000 7f71f6c00700 1 -- 172.21.15.86:0/3810067683 shutdown_connections 2024-04-17T07:02:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:31.736+0000 7f71f6c00700 1 -- 172.21.15.86:0/3810067683 wait complete. 2024-04-17T07:02:32.463 INFO:teuthology.orchestra.run.smithi086.stdout:Fetching dashboard port number... 2024-04-17T07:02:32.980 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:32 smithi086 ceph-mon[31733]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:02:32.980 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:32 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:32.980 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:32 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:33.969 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:33 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4021238191' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-17T07:02:33.981 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 8443 2024-04-17T07:02:33.982 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.997+0000 7f193d400700 1 Processor -- start 2024-04-17T07:02:33.982 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.998+0000 7f193d400700 1 -- start start 2024-04-17T07:02:33.982 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.998+0000 7f193d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193810b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:33.982 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.998+0000 7f193d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f193807bed0 con 0x7f1938108d60 2024-04-17T07:02:33.982 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.998+0000 7f1936800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193810b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:33.982 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.998+0000 7f1936800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193810b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47520/0 (socket says 172.21.15.86:47520) 2024-04-17T07:02:33.982 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.998+0000 7f1936800700 1 -- 172.21.15.86:0/3180825607 learned_addr learned my addr 172.21.15.86:0/3180825607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:33.982 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.999+0000 7f1936800700 1 -- 172.21.15.86:0/3180825607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193810b690 con 0x7f1938108d60 2024-04-17T07:02:33.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.999+0000 7f1936800700 1 --2- 172.21.15.86:0/3180825607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193810b150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19280098a0 tx=0x7f1928009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bdf88cd71a539c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:33.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.999+0000 7f1935400700 1 -- 172.21.15.86:0/3180825607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1928004160 con 0x7f1938108d60 2024-04-17T07:02:33.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:32.999+0000 7f1935400700 1 -- 172.21.15.86:0/3180825607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1928004300 con 0x7f1938108d60 2024-04-17T07:02:33.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.000+0000 7f1935400700 1 -- 172.21.15.86:0/3180825607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1928011a50 con 0x7f1938108d60 2024-04-17T07:02:33.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.000+0000 7f193d400700 1 -- 172.21.15.86:0/3180825607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 msgr2=0x7f193810b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:33.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.000+0000 7f193d400700 1 --2- 172.21.15.86:0/3180825607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193810b150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19280098a0 tx=0x7f1928009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:33.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.000+0000 7f193d400700 1 -- 172.21.15.86:0/3180825607 shutdown_connections 2024-04-17T07:02:33.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.000+0000 7f193d400700 1 --2- 172.21.15.86:0/3180825607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193810b150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.000+0000 7f193d400700 1 -- 172.21.15.86:0/3180825607 >> 172.21.15.86:0/3180825607 conn(0x7f1938102830 msgr2=0x7f1938104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.000+0000 7f193d400700 1 -- 172.21.15.86:0/3180825607 shutdown_connections 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.001+0000 7f193d400700 1 -- 172.21.15.86:0/3180825607 wait complete. 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.002+0000 7f193d400700 1 Processor -- start 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.002+0000 7f193d400700 1 -- start start 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.002+0000 7f193d400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193819e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.002+0000 7f193d400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f193810c6a0 con 0x7f1938108d60 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.002+0000 7f1936800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193819e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.003+0000 7f1936800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193819e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47528/0 (socket says 172.21.15.86:47528) 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.003+0000 7f1936800700 1 -- 172.21.15.86:0/4021238191 learned_addr learned my addr 172.21.15.86:0/4021238191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.003+0000 7f1936800700 1 -- 172.21.15.86:0/4021238191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193819e880 con 0x7f1938108d60 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.003+0000 7f1936800700 1 --2- 172.21.15.86:0/4021238191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193819e340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1928011a00 tx=0x7f1928011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.003+0000 7f192f400700 1 -- 172.21.15.86:0/4021238191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1928003af0 con 0x7f1938108d60 2024-04-17T07:02:33.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.004+0000 7f192f400700 1 -- 172.21.15.86:0/4021238191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f192801a8e0 con 0x7f1938108d60 2024-04-17T07:02:33.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.004+0000 7f192f400700 1 -- 172.21.15.86:0/4021238191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1928024840 con 0x7f1938108d60 2024-04-17T07:02:33.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.004+0000 7f193d400700 1 -- 172.21.15.86:0/4021238191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193819eb10 con 0x7f1938108d60 2024-04-17T07:02:33.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.004+0000 7f193d400700 1 -- 172.21.15.86:0/4021238191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19381a1800 con 0x7f1938108d60 2024-04-17T07:02:33.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.005+0000 7f193d400700 1 -- 172.21.15.86:0/4021238191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19380640b0 con 0x7f1938108d60 2024-04-17T07:02:33.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.005+0000 7f192f400700 1 -- 172.21.15.86:0/4021238191 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f1928022020 con 0x7f1938108d60 2024-04-17T07:02:33.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.005+0000 7f192f400700 1 --2- 172.21.15.86:0/4021238191 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f191c039720 0x7f191c03bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:33.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.005+0000 7f192f400700 1 -- 172.21.15.86:0/4021238191 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f192804dbe0 con 0x7f1938108d60 2024-04-17T07:02:33.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.006+0000 7f1935e00700 1 --2- 172.21.15.86:0/4021238191 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f191c039720 0x7f191c03bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:33.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.006+0000 7f1935e00700 1 --2- 172.21.15.86:0/4021238191 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f191c039720 0x7f191c03bbe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1924006fd0 tx=0x7f1924006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.011+0000 7f192f400700 1 -- 172.21.15.86:0/4021238191 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f192802ddb0 con 0x7f1938108d60 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.246+0000 7f193d400700 1 -- 172.21.15.86:0/4021238191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f19381a1db0 con 0x7f1938108d60 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.246+0000 7f192f400700 1 -- 172.21.15.86:0/4021238191 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f192802ad90 con 0x7f1938108d60 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.250+0000 7f193d400700 1 -- 172.21.15.86:0/4021238191 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f191c039720 msgr2=0x7f191c03bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.250+0000 7f193d400700 1 --2- 172.21.15.86:0/4021238191 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f191c039720 0x7f191c03bbe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1924006fd0 tx=0x7f1924006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.250+0000 7f193d400700 1 -- 172.21.15.86:0/4021238191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 msgr2=0x7f193819e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.250+0000 7f193d400700 1 --2- 172.21.15.86:0/4021238191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193819e340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1928011a00 tx=0x7f1928011ae0 comp rx=0 tx=0).stop 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.250+0000 7f193d400700 1 -- 172.21.15.86:0/4021238191 shutdown_connections 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.250+0000 7f193d400700 1 --2- 172.21.15.86:0/4021238191 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f191c039720 0x7f191c03bbe0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.250+0000 7f193d400700 1 --2- 172.21.15.86:0/4021238191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1938108d60 0x7f193819e340 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:33.990 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.250+0000 7f193d400700 1 -- 172.21.15.86:0/4021238191 >> 172.21.15.86:0/4021238191 conn(0x7f1938102830 msgr2=0x7f19381031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:33.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.250+0000 7f193d400700 1 -- 172.21.15.86:0/4021238191 shutdown_connections 2024-04-17T07:02:33.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:33.251+0000 7f193d400700 1 -- 172.21.15.86:0/4021238191 wait complete. 2024-04-17T07:02:34.024 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-17T07:02:34.024 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-17T07:02:34.050 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-17T07:02:34.050 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-17T07:02:34.050 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-17T07:02:34.051 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-17T07:02:34.057 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph Dashboard is now available at: 2024-04-17T07:02:34.057 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:34.057 INFO:teuthology.orchestra.run.smithi086.stdout: URL: https://smithi086.front.sepia.ceph.com:8443/ 2024-04-17T07:02:34.057 INFO:teuthology.orchestra.run.smithi086.stdout: User: admin 2024-04-17T07:02:34.057 INFO:teuthology.orchestra.run.smithi086.stdout: Password: 968l8a50qf 2024-04-17T07:02:34.058 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:34.058 INFO:teuthology.orchestra.run.smithi086.stdout:Saving cluster configuration to /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config directory 2024-04-17T07:02:34.061 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling autotune for osd_memory_target 2024-04-17T07:02:34.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:34 smithi086 ceph-mon[31733]: mgrmap e13: x(active, since 6s) 2024-04-17T07:02:36.184 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.419+0000 7f774ae00700 1 Processor -- start 2024-04-17T07:02:36.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.420+0000 7f774ae00700 1 -- start start 2024-04-17T07:02:36.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.420+0000 7f774ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f774407eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:36.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.420+0000 7f774ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f774407f270 con 0x7f774407e880 2024-04-17T07:02:36.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.421+0000 7f7749a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f774407eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:36.185 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.421+0000 7f7749a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f774407eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47540/0 (socket says 172.21.15.86:47540) 2024-04-17T07:02:36.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.421+0000 7f7749a00700 1 -- 172.21.15.86:0/3356509091 learned_addr learned my addr 172.21.15.86:0/3356509091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:36.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.421+0000 7f7749a00700 1 -- 172.21.15.86:0/3356509091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774407fa80 con 0x7f774407e880 2024-04-17T07:02:36.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.422+0000 7f7749a00700 1 --2- 172.21.15.86:0/3356509091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f774407eca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77380098a0 tx=0x7f7738009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97accca2c4acf73e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:36.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.422+0000 7f7743e00700 1 -- 172.21.15.86:0/3356509091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7738004160 con 0x7f774407e880 2024-04-17T07:02:36.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.422+0000 7f7743e00700 1 -- 172.21.15.86:0/3356509091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7738004300 con 0x7f774407e880 2024-04-17T07:02:36.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.422+0000 7f7743e00700 1 -- 172.21.15.86:0/3356509091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7738011a50 con 0x7f774407e880 2024-04-17T07:02:36.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.422+0000 7f774ae00700 1 -- 172.21.15.86:0/3356509091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 msgr2=0x7f774407eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:36.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.422+0000 7f774ae00700 1 --2- 172.21.15.86:0/3356509091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f774407eca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77380098a0 tx=0x7f7738009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:36.187 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.423+0000 7f774ae00700 1 -- 172.21.15.86:0/3356509091 shutdown_connections 2024-04-17T07:02:36.188 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.423+0000 7f774ae00700 1 --2- 172.21.15.86:0/3356509091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f774407eca0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:36.188 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.423+0000 7f774ae00700 1 -- 172.21.15.86:0/3356509091 >> 172.21.15.86:0/3356509091 conn(0x7f774407da10 msgr2=0x7f77441084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:36.191 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.423+0000 7f774ae00700 1 -- 172.21.15.86:0/3356509091 shutdown_connections 2024-04-17T07:02:36.191 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.423+0000 7f774ae00700 1 -- 172.21.15.86:0/3356509091 wait complete. 2024-04-17T07:02:36.191 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.424+0000 7f774ae00700 1 Processor -- start 2024-04-17T07:02:36.191 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.424+0000 7f774ae00700 1 -- start start 2024-04-17T07:02:36.191 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.424+0000 7f774ae00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f77441a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:36.191 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.424+0000 7f774ae00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7744080e40 con 0x7f774407e880 2024-04-17T07:02:36.191 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.425+0000 7f7749a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f77441a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:36.191 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.425+0000 7f7749a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f77441a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47554/0 (socket says 172.21.15.86:47554) 2024-04-17T07:02:36.191 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.425+0000 7f7749a00700 1 -- 172.21.15.86:0/3959862274 learned_addr learned my addr 172.21.15.86:0/3959862274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.425+0000 7f7749a00700 1 -- 172.21.15.86:0/3959862274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77441a28c0 con 0x7f774407e880 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.425+0000 7f7749a00700 1 --2- 172.21.15.86:0/3959862274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f77441a2380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7738003900 tx=0x7f77380039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.425+0000 7f7742000700 1 -- 172.21.15.86:0/3959862274 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77380044b0 con 0x7f774407e880 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.426+0000 7f7742000700 1 -- 172.21.15.86:0/3959862274 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f773801aaf0 con 0x7f774407e880 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.426+0000 7f774ae00700 1 -- 172.21.15.86:0/3959862274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77441a2b50 con 0x7f774407e880 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.426+0000 7f7742000700 1 -- 172.21.15.86:0/3959862274 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7738024820 con 0x7f774407e880 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.426+0000 7f774ae00700 1 -- 172.21.15.86:0/3959862274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77441a5840 con 0x7f774407e880 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.427+0000 7f7742000700 1 -- 172.21.15.86:0/3959862274 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f773801aca0 con 0x7f774407e880 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.427+0000 7f774ae00700 1 -- 172.21.15.86:0/3959862274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77440640b0 con 0x7f774407e880 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.427+0000 7f7742000700 1 --2- 172.21.15.86:0/3959862274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f772c039760 0x7f772c03bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.427+0000 7f7742000700 1 -- 172.21.15.86:0/3959862274 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f773804df70 con 0x7f774407e880 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.428+0000 7f7749000700 1 --2- 172.21.15.86:0/3959862274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f772c039760 0x7f772c03bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:36.192 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.428+0000 7f7749000700 1 --2- 172.21.15.86:0/3959862274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f772c039760 0x7f772c03bc20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7730006fd0 tx=0x7f7730006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:36.193 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.432+0000 7f7742000700 1 -- 172.21.15.86:0/3959862274 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f773802d3f0 con 0x7f774407e880 2024-04-17T07:02:36.193 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.658+0000 7f774ae00700 1 -- 172.21.15.86:0/3959862274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f774402e270 con 0x7f774407e880 2024-04-17T07:02:36.193 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.659+0000 7f7742000700 1 -- 172.21.15.86:0/3959862274 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f773802c350 con 0x7f774407e880 2024-04-17T07:02:36.193 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.662+0000 7f774ae00700 1 -- 172.21.15.86:0/3959862274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f772c039760 msgr2=0x7f772c03bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:36.193 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.662+0000 7f774ae00700 1 --2- 172.21.15.86:0/3959862274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f772c039760 0x7f772c03bc20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7730006fd0 tx=0x7f7730006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:36.194 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.662+0000 7f774ae00700 1 -- 172.21.15.86:0/3959862274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 msgr2=0x7f77441a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:36.194 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.662+0000 7f774ae00700 1 --2- 172.21.15.86:0/3959862274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f77441a2380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7738003900 tx=0x7f77380039e0 comp rx=0 tx=0).stop 2024-04-17T07:02:36.194 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.662+0000 7f774ae00700 1 -- 172.21.15.86:0/3959862274 shutdown_connections 2024-04-17T07:02:36.194 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.662+0000 7f774ae00700 1 --2- 172.21.15.86:0/3959862274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f772c039760 0x7f772c03bc20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:36.194 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.662+0000 7f774ae00700 1 --2- 172.21.15.86:0/3959862274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774407e880 0x7f77441a2380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:36.194 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.662+0000 7f774ae00700 1 -- 172.21.15.86:0/3959862274 >> 172.21.15.86:0/3959862274 conn(0x7f774407da10 msgr2=0x7f7744108880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:36.194 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.662+0000 7f774ae00700 1 -- 172.21.15.86:0/3959862274 shutdown_connections 2024-04-17T07:02:36.195 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:35.662+0000 7f774ae00700 1 -- 172.21.15.86:0/3959862274 wait complete. 2024-04-17T07:02:37.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.778+0000 7fe040a00700 1 Processor -- start 2024-04-17T07:02:37.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.778+0000 7fe040a00700 1 -- start start 2024-04-17T07:02:37.796 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.779+0000 7fe040a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c10a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:37.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.779+0000 7fe040a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c10af60 con 0x7fe03c10a570 2024-04-17T07:02:37.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.780+0000 7fe039e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c10a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:37.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.780+0000 7fe039e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c10a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47562/0 (socket says 172.21.15.86:47562) 2024-04-17T07:02:37.797 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.780+0000 7fe039e00700 1 -- 172.21.15.86:0/3863157105 learned_addr learned my addr 172.21.15.86:0/3863157105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:37.798 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.780+0000 7fe039e00700 1 -- 172.21.15.86:0/3863157105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c10b720 con 0x7fe03c10a570 2024-04-17T07:02:37.798 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.780+0000 7fe039e00700 1 --2- 172.21.15.86:0/3863157105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c10a990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0098a0 tx=0x7fe02c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1ad459c66dd9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:37.798 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.781+0000 7fe038a00700 1 -- 172.21.15.86:0/3863157105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe02c004160 con 0x7fe03c10a570 2024-04-17T07:02:37.798 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.781+0000 7fe038a00700 1 -- 172.21.15.86:0/3863157105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe02c004300 con 0x7fe03c10a570 2024-04-17T07:02:37.798 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.781+0000 7fe038a00700 1 -- 172.21.15.86:0/3863157105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe02c011a50 con 0x7fe03c10a570 2024-04-17T07:02:37.799 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.781+0000 7fe040a00700 1 -- 172.21.15.86:0/3863157105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 msgr2=0x7fe03c10a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:37.799 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.781+0000 7fe040a00700 1 --2- 172.21.15.86:0/3863157105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c10a990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0098a0 tx=0x7fe02c009bb0 comp rx=0 tx=0).stop 2024-04-17T07:02:37.800 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.781+0000 7fe040a00700 1 -- 172.21.15.86:0/3863157105 shutdown_connections 2024-04-17T07:02:37.800 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.781+0000 7fe040a00700 1 --2- 172.21.15.86:0/3863157105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c10a990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:37.800 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.781+0000 7fe040a00700 1 -- 172.21.15.86:0/3863157105 >> 172.21.15.86:0/3863157105 conn(0x7fe03c07dc10 msgr2=0x7fe03c1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:37.800 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.782+0000 7fe040a00700 1 -- 172.21.15.86:0/3863157105 shutdown_connections 2024-04-17T07:02:37.801 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.782+0000 7fe040a00700 1 -- 172.21.15.86:0/3863157105 wait complete. 2024-04-17T07:02:37.801 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.783+0000 7fe040a00700 1 Processor -- start 2024-04-17T07:02:37.801 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.783+0000 7fe040a00700 1 -- start start 2024-04-17T07:02:37.801 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.783+0000 7fe040a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:37.801 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.783+0000 7fe040a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c10ead0 con 0x7fe03c10a570 2024-04-17T07:02:37.801 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.783+0000 7fe039e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:37.801 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.783+0000 7fe039e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c19e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47566/0 (socket says 172.21.15.86:47566) 2024-04-17T07:02:37.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.783+0000 7fe039e00700 1 -- 172.21.15.86:0/2032148631 learned_addr learned my addr 172.21.15.86:0/2032148631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:37.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.784+0000 7fe039e00700 1 -- 172.21.15.86:0/2032148631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c19e8a0 con 0x7fe03c10a570 2024-04-17T07:02:37.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.784+0000 7fe039e00700 1 --2- 172.21.15.86:0/2032148631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c19e360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe02c011a20 tx=0x7fe02c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:37.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.784+0000 7fe032a00700 1 -- 172.21.15.86:0/2032148631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe02c003af0 con 0x7fe03c10a570 2024-04-17T07:02:37.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.784+0000 7fe032a00700 1 -- 172.21.15.86:0/2032148631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe02c01a8e0 con 0x7fe03c10a570 2024-04-17T07:02:37.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.784+0000 7fe032a00700 1 -- 172.21.15.86:0/2032148631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe02c024860 con 0x7fe03c10a570 2024-04-17T07:02:37.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.784+0000 7fe040a00700 1 -- 172.21.15.86:0/2032148631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03c19eb30 con 0x7fe03c10a570 2024-04-17T07:02:37.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.785+0000 7fe040a00700 1 -- 172.21.15.86:0/2032148631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03c1a1820 con 0x7fe03c10a570 2024-04-17T07:02:37.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.786+0000 7fe032a00700 1 -- 172.21.15.86:0/2032148631 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe02c022020 con 0x7fe03c10a570 2024-04-17T07:02:37.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.786+0000 7fe040a00700 1 -- 172.21.15.86:0/2032148631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe03c0640b0 con 0x7fe03c10a570 2024-04-17T07:02:37.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.786+0000 7fe032a00700 1 --2- 172.21.15.86:0/2032148631 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe020039760 0x7fe02003bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:37.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.786+0000 7fe032a00700 1 -- 172.21.15.86:0/2032148631 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe02c04dc80 con 0x7fe03c10a570 2024-04-17T07:02:37.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.786+0000 7fe039400700 1 --2- 172.21.15.86:0/2032148631 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe020039760 0x7fe02003bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:37.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.787+0000 7fe039400700 1 --2- 172.21.15.86:0/2032148631 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe020039760 0x7fe02003bc20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe028006fd0 tx=0x7fe028006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:37.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:36.792+0000 7fe032a00700 1 -- 172.21.15.86:0/2032148631 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe02c0094f0 con 0x7fe03c10a570 2024-04-17T07:02:37.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.131+0000 7fe040a00700 1 -- 172.21.15.86:0/2032148631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe03c1a1dd0 con 0x7fe03c10a570 2024-04-17T07:02:37.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.134+0000 7fe032a00700 1 -- 172.21.15.86:0/2032148631 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fe02c02ad60 con 0x7fe03c10a570 2024-04-17T07:02:37.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.137+0000 7fe040a00700 1 -- 172.21.15.86:0/2032148631 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe020039760 msgr2=0x7fe02003bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:37.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.137+0000 7fe040a00700 1 --2- 172.21.15.86:0/2032148631 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe020039760 0x7fe02003bc20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe028006fd0 tx=0x7fe028006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:37.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.137+0000 7fe040a00700 1 -- 172.21.15.86:0/2032148631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 msgr2=0x7fe03c19e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:37.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.137+0000 7fe040a00700 1 --2- 172.21.15.86:0/2032148631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c19e360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe02c011a20 tx=0x7fe02c011b00 comp rx=0 tx=0).stop 2024-04-17T07:02:37.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.138+0000 7fe040a00700 1 -- 172.21.15.86:0/2032148631 shutdown_connections 2024-04-17T07:02:37.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.138+0000 7fe040a00700 1 --2- 172.21.15.86:0/2032148631 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe020039760 0x7fe02003bc20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.138+0000 7fe040a00700 1 --2- 172.21.15.86:0/2032148631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe03c10a570 0x7fe03c19e360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.138+0000 7fe040a00700 1 -- 172.21.15.86:0/2032148631 >> 172.21.15.86:0/2032148631 conn(0x7fe03c07dc10 msgr2=0x7fe03c107ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.138+0000 7fe040a00700 1 -- 172.21.15.86:0/2032148631 shutdown_connections 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-17T07:02:37.138+0000 7fe040a00700 1 -- 172.21.15.86:0/2032148631 wait complete. 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout:Or, if you are only running a single cluster on this host: 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:37.807 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-17T07:02:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:37.808 INFO:teuthology.orchestra.run.smithi086.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-17T07:02:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: ceph telemetry on 2024-04-17T07:02:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:37.808 INFO:teuthology.orchestra.run.smithi086.stdout:For more information see: 2024-04-17T07:02:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-17T07:02:37.808 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:37.808 INFO:teuthology.orchestra.run.smithi086.stdout:Bootstrap complete. 2024-04-17T07:02:37.924 INFO:tasks.cephadm:Fetching config... 2024-04-17T07:02:37.925 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:02:37.925 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-17T07:02:37.969 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-17T07:02:37.969 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:02:37.969 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-17T07:02:38.055 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-17T07:02:38.055 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:02:38.055 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-17T07:02:38.175 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-17T07:02:38.176 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:02:38.176 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-17T07:02:38.264 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-17T07:02:38.264 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw53rmGFF8KqqZ4oy0aVQ0lEbBlfj1GCJvP4xh+/wzWoZ+egUCWdgV8At2h5bBo7PARB8ZLJayfP6XuDcEskGS1yUrLBQdRACeVBnejfOKwyAZuvokr5Yi0nwy/+yApHEwjxjW6nj7e+CVOYsB0/Yo1NeSRrtVw1yZHkM51Uly+rc0QnfSLWxYOQigWmQXVDqRXhh3IT03A+uolDtMW39tjQK/usz4ZrS74tG/h13Nyl7nizJiElUtnyUh7P7dJkQYaUMWIapO4+3vNXrIGat1gWksEeAXaKDEPEYxytbhUQbuw2KkvtITolxc/WV1S7MhvHAGf2OETHHjwb8dcaqQp6Bn5yyyf1/Y2Gp0PflCZHMkP98X+YBDLrqOxr4resOO2dXuwwDLpNhcemT8A+jO/Gfvdk5q4gqA2EUNqvikZpzSSdXZTbeCv9mwPI0W+bavk0iGG5xggTviUeGPgxLEX4AJ95+0+yrcTn6mc+tXhifJ7w7XAIAjxltGt5mnc1M= ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-17T07:02:38.408 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:38 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2032148631' entity='client.admin' 2024-04-17T07:02:38.415 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw53rmGFF8KqqZ4oy0aVQ0lEbBlfj1GCJvP4xh+/wzWoZ+egUCWdgV8At2h5bBo7PARB8ZLJayfP6XuDcEskGS1yUrLBQdRACeVBnejfOKwyAZuvokr5Yi0nwy/+yApHEwjxjW6nj7e+CVOYsB0/Yo1NeSRrtVw1yZHkM51Uly+rc0QnfSLWxYOQigWmQXVDqRXhh3IT03A+uolDtMW39tjQK/usz4ZrS74tG/h13Nyl7nizJiElUtnyUh7P7dJkQYaUMWIapO4+3vNXrIGat1gWksEeAXaKDEPEYxytbhUQbuw2KkvtITolxc/WV1S7MhvHAGf2OETHHjwb8dcaqQp6Bn5yyyf1/Y2Gp0PflCZHMkP98X+YBDLrqOxr4resOO2dXuwwDLpNhcemT8A+jO/Gfvdk5q4gqA2EUNqvikZpzSSdXZTbeCv9mwPI0W+bavk0iGG5xggTviUeGPgxLEX4AJ95+0+yrcTn6mc+tXhifJ7w7XAIAjxltGt5mnc1M= ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:02:38.459 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw53rmGFF8KqqZ4oy0aVQ0lEbBlfj1GCJvP4xh+/wzWoZ+egUCWdgV8At2h5bBo7PARB8ZLJayfP6XuDcEskGS1yUrLBQdRACeVBnejfOKwyAZuvokr5Yi0nwy/+yApHEwjxjW6nj7e+CVOYsB0/Yo1NeSRrtVw1yZHkM51Uly+rc0QnfSLWxYOQigWmQXVDqRXhh3IT03A+uolDtMW39tjQK/usz4ZrS74tG/h13Nyl7nizJiElUtnyUh7P7dJkQYaUMWIapO4+3vNXrIGat1gWksEeAXaKDEPEYxytbhUQbuw2KkvtITolxc/WV1S7MhvHAGf2OETHHjwb8dcaqQp6Bn5yyyf1/Y2Gp0PflCZHMkP98X+YBDLrqOxr4resOO2dXuwwDLpNhcemT8A+jO/Gfvdk5q4gqA2EUNqvikZpzSSdXZTbeCv9mwPI0W+bavk0iGG5xggTviUeGPgxLEX4AJ95+0+yrcTn6mc+tXhifJ7w7XAIAjxltGt5mnc1M= ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-17T07:02:38.601 INFO:teuthology.orchestra.run.smithi098.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw53rmGFF8KqqZ4oy0aVQ0lEbBlfj1GCJvP4xh+/wzWoZ+egUCWdgV8At2h5bBo7PARB8ZLJayfP6XuDcEskGS1yUrLBQdRACeVBnejfOKwyAZuvokr5Yi0nwy/+yApHEwjxjW6nj7e+CVOYsB0/Yo1NeSRrtVw1yZHkM51Uly+rc0QnfSLWxYOQigWmQXVDqRXhh3IT03A+uolDtMW39tjQK/usz4ZrS74tG/h13Nyl7nizJiElUtnyUh7P7dJkQYaUMWIapO4+3vNXrIGat1gWksEeAXaKDEPEYxytbhUQbuw2KkvtITolxc/WV1S7MhvHAGf2OETHHjwb8dcaqQp6Bn5yyyf1/Y2Gp0PflCZHMkP98X+YBDLrqOxr4resOO2dXuwwDLpNhcemT8A+jO/Gfvdk5q4gqA2EUNqvikZpzSSdXZTbeCv9mwPI0W+bavk0iGG5xggTviUeGPgxLEX4AJ95+0+yrcTn6mc+tXhifJ7w7XAIAjxltGt5mnc1M= ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:02:38.666 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw53rmGFF8KqqZ4oy0aVQ0lEbBlfj1GCJvP4xh+/wzWoZ+egUCWdgV8At2h5bBo7PARB8ZLJayfP6XuDcEskGS1yUrLBQdRACeVBnejfOKwyAZuvokr5Yi0nwy/+yApHEwjxjW6nj7e+CVOYsB0/Yo1NeSRrtVw1yZHkM51Uly+rc0QnfSLWxYOQigWmQXVDqRXhh3IT03A+uolDtMW39tjQK/usz4ZrS74tG/h13Nyl7nizJiElUtnyUh7P7dJkQYaUMWIapO4+3vNXrIGat1gWksEeAXaKDEPEYxytbhUQbuw2KkvtITolxc/WV1S7MhvHAGf2OETHHjwb8dcaqQp6Bn5yyyf1/Y2Gp0PflCZHMkP98X+YBDLrqOxr4resOO2dXuwwDLpNhcemT8A+jO/Gfvdk5q4gqA2EUNqvikZpzSSdXZTbeCv9mwPI0W+bavk0iGG5xggTviUeGPgxLEX4AJ95+0+yrcTn6mc+tXhifJ7w7XAIAjxltGt5mnc1M= ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-17T07:02:38.799 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCw53rmGFF8KqqZ4oy0aVQ0lEbBlfj1GCJvP4xh+/wzWoZ+egUCWdgV8At2h5bBo7PARB8ZLJayfP6XuDcEskGS1yUrLBQdRACeVBnejfOKwyAZuvokr5Yi0nwy/+yApHEwjxjW6nj7e+CVOYsB0/Yo1NeSRrtVw1yZHkM51Uly+rc0QnfSLWxYOQigWmQXVDqRXhh3IT03A+uolDtMW39tjQK/usz4ZrS74tG/h13Nyl7nizJiElUtnyUh7P7dJkQYaUMWIapO4+3vNXrIGat1gWksEeAXaKDEPEYxytbhUQbuw2KkvtITolxc/WV1S7MhvHAGf2OETHHjwb8dcaqQp6Bn5yyyf1/Y2Gp0PflCZHMkP98X+YBDLrqOxr4resOO2dXuwwDLpNhcemT8A+jO/Gfvdk5q4gqA2EUNqvikZpzSSdXZTbeCv9mwPI0W+bavk0iGG5xggTviUeGPgxLEX4AJ95+0+yrcTn6mc+tXhifJ7w7XAIAjxltGt5mnc1M= ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:02:38.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-17T07:02:39.294 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:02:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.867+0000 7ff790c00700 1 -- 172.21.15.86:0/4245735762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c104700 msgr2=0x7ff78c104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.867+0000 7ff790c00700 1 --2- 172.21.15.86:0/4245735762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c104700 0x7ff78c104b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009910 tx=0x7ff77c009c20 comp rx=0 tx=0).stop 2024-04-17T07:02:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.867+0000 7ff790c00700 1 -- 172.21.15.86:0/4245735762 shutdown_connections 2024-04-17T07:02:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.867+0000 7ff790c00700 1 --2- 172.21.15.86:0/4245735762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c104700 0x7ff78c104b20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.867+0000 7ff790c00700 1 -- 172.21.15.86:0/4245735762 >> 172.21.15.86:0/4245735762 conn(0x7ff78c0ffc80 msgr2=0x7ff78c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.868+0000 7ff790c00700 1 -- 172.21.15.86:0/4245735762 shutdown_connections 2024-04-17T07:02:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.868+0000 7ff790c00700 1 -- 172.21.15.86:0/4245735762 wait complete. 2024-04-17T07:02:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.869+0000 7ff790c00700 1 Processor -- start 2024-04-17T07:02:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.869+0000 7ff790c00700 1 -- start start 2024-04-17T07:02:41.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.869+0000 7ff790c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c104700 0x7ff78c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.869+0000 7ff790c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c10a640 con 0x7ff78c104700 2024-04-17T07:02:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.869+0000 7ff789e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c104700 0x7ff78c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.869+0000 7ff789e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c104700 0x7ff78c199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33736/0 (socket says 172.21.15.86:33736) 2024-04-17T07:02:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.869+0000 7ff789e00700 1 -- 172.21.15.86:0/239976278 learned_addr learned my addr 172.21.15.86:0/239976278 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.870+0000 7ff789e00700 1 -- 172.21.15.86:0/239976278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c19a250 con 0x7ff78c104700 2024-04-17T07:02:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.870+0000 7ff789e00700 1 --2- 172.21.15.86:0/239976278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c104700 0x7ff78c199d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff77c005800 tx=0x7ff77c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.870+0000 7ff782a00700 1 -- 172.21.15.86:0/239976278 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff77c027070 con 0x7ff78c104700 2024-04-17T07:02:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.871+0000 7ff782a00700 1 -- 172.21.15.86:0/239976278 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff77c00be10 con 0x7ff78c104700 2024-04-17T07:02:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.871+0000 7ff790c00700 1 -- 172.21.15.86:0/239976278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78c19a4e0 con 0x7ff78c104700 2024-04-17T07:02:41.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.871+0000 7ff782a00700 1 -- 172.21.15.86:0/239976278 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff77c00f460 con 0x7ff78c104700 2024-04-17T07:02:41.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.871+0000 7ff790c00700 1 -- 172.21.15.86:0/239976278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78c19d1c0 con 0x7ff78c104700 2024-04-17T07:02:41.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.872+0000 7ff790c00700 1 -- 172.21.15.86:0/239976278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff78c10adc0 con 0x7ff78c104700 2024-04-17T07:02:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.872+0000 7ff782a00700 1 -- 172.21.15.86:0/239976278 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff77c02a070 con 0x7ff78c104700 2024-04-17T07:02:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.872+0000 7ff782a00700 1 --2- 172.21.15.86:0/239976278 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff770039650 0x7ff77003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.872+0000 7ff782a00700 1 -- 172.21.15.86:0/239976278 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff77c01e030 con 0x7ff78c104700 2024-04-17T07:02:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.873+0000 7ff789400700 1 --2- 172.21.15.86:0/239976278 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff770039650 0x7ff77003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.873+0000 7ff789400700 1 --2- 172.21.15.86:0/239976278 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff770039650 0x7ff77003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff778006fd0 tx=0x7ff778006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:41.878+0000 7ff782a00700 1 -- 172.21.15.86:0/239976278 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff77c013070 con 0x7ff78c104700 2024-04-17T07:02:42.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:42.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:42.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:02:42.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:42.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:02:42.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.104+0000 7ff790c00700 1 -- 172.21.15.86:0/239976278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7ff78c02e0a0 con 0x7ff78c104700 2024-04-17T07:02:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.110+0000 7ff782a00700 1 -- 172.21.15.86:0/239976278 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7ff77c031020 con 0x7ff78c104700 2024-04-17T07:02:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.112+0000 7ff790c00700 1 -- 172.21.15.86:0/239976278 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff770039650 msgr2=0x7ff77003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.113+0000 7ff790c00700 1 --2- 172.21.15.86:0/239976278 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff770039650 0x7ff77003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff778006fd0 tx=0x7ff778006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.113+0000 7ff790c00700 1 -- 172.21.15.86:0/239976278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c104700 msgr2=0x7ff78c199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.113+0000 7ff790c00700 1 --2- 172.21.15.86:0/239976278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c104700 0x7ff78c199d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff77c005800 tx=0x7ff77c009f60 comp rx=0 tx=0).stop 2024-04-17T07:02:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.113+0000 7ff790c00700 1 -- 172.21.15.86:0/239976278 shutdown_connections 2024-04-17T07:02:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.113+0000 7ff790c00700 1 --2- 172.21.15.86:0/239976278 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff770039650 0x7ff77003bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.113+0000 7ff790c00700 1 --2- 172.21.15.86:0/239976278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c104700 0x7ff78c199d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.113+0000 7ff790c00700 1 -- 172.21.15.86:0/239976278 >> 172.21.15.86:0/239976278 conn(0x7ff78c0ffc80 msgr2=0x7ff78c100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.113+0000 7ff790c00700 1 -- 172.21.15.86:0/239976278 shutdown_connections 2024-04-17T07:02:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:42.113+0000 7ff790c00700 1 -- 172.21.15.86:0/239976278 wait complete. 2024-04-17T07:02:42.823 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-17T07:02:42.823 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-17T07:02:43.260 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:02:43.407 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:43 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/239976278' entity='client.admin' 2024-04-17T07:02:43.407 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:02:43.407 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.173+0000 7fb246c00700 1 -- 172.21.15.86:0/3279282816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb240102f40 msgr2=0x7fb240105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.173+0000 7fb246c00700 1 --2- 172.21.15.86:0/3279282816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb240102f40 0x7fb240105330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb230009910 tx=0x7fb230009c20 comp rx=0 tx=0).stop 2024-04-17T07:02:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.173+0000 7fb246c00700 1 -- 172.21.15.86:0/3279282816 shutdown_connections 2024-04-17T07:02:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.173+0000 7fb246c00700 1 --2- 172.21.15.86:0/3279282816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb240102f40 0x7fb240105330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.173+0000 7fb246c00700 1 -- 172.21.15.86:0/3279282816 >> 172.21.15.86:0/3279282816 conn(0x7fb2400fc950 msgr2=0x7fb2400fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.173+0000 7fb246c00700 1 -- 172.21.15.86:0/3279282816 shutdown_connections 2024-04-17T07:02:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.174+0000 7fb246c00700 1 -- 172.21.15.86:0/3279282816 wait complete. 2024-04-17T07:02:45.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.175+0000 7fb246c00700 1 Processor -- start 2024-04-17T07:02:45.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.175+0000 7fb246c00700 1 -- start start 2024-04-17T07:02:45.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.175+0000 7fb246c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb240102f40 0x7fb240197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:45.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.175+0000 7fb246c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb240108390 con 0x7fb240102f40 2024-04-17T07:02:45.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.175+0000 7fb23fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb240102f40 0x7fb240197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.176+0000 7fb23fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb240102f40 0x7fb240197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33748/0 (socket says 172.21.15.86:33748) 2024-04-17T07:02:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.176+0000 7fb23fe00700 1 -- 172.21.15.86:0/3045638684 learned_addr learned my addr 172.21.15.86:0/3045638684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.176+0000 7fb23fe00700 1 -- 172.21.15.86:0/3045638684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb240197fe0 con 0x7fb240102f40 2024-04-17T07:02:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.176+0000 7fb23fe00700 1 --2- 172.21.15.86:0/3045638684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb240102f40 0x7fb240197aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb230005190 tx=0x7fb230004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.176+0000 7fb23d600700 1 -- 172.21.15.86:0/3045638684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb230027070 con 0x7fb240102f40 2024-04-17T07:02:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.177+0000 7fb246c00700 1 -- 172.21.15.86:0/3045638684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2401982d0 con 0x7fb240102f40 2024-04-17T07:02:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.177+0000 7fb23d600700 1 -- 172.21.15.86:0/3045638684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb230022470 con 0x7fb240102f40 2024-04-17T07:02:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.177+0000 7fb23d600700 1 -- 172.21.15.86:0/3045638684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb23000f5f0 con 0x7fb240102f40 2024-04-17T07:02:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.177+0000 7fb246c00700 1 -- 172.21.15.86:0/3045638684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24019af10 con 0x7fb240102f40 2024-04-17T07:02:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.178+0000 7fb23d600700 1 -- 172.21.15.86:0/3045638684 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb23002a070 con 0x7fb240102f40 2024-04-17T07:02:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.178+0000 7fb246c00700 1 -- 172.21.15.86:0/3045638684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb240108b10 con 0x7fb240102f40 2024-04-17T07:02:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.178+0000 7fb23d600700 1 --2- 172.21.15.86:0/3045638684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb224039650 0x7fb22403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.178+0000 7fb23d600700 1 -- 172.21.15.86:0/3045638684 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb23001e030 con 0x7fb240102f40 2024-04-17T07:02:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.179+0000 7fb237400700 1 --2- 172.21.15.86:0/3045638684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb224039650 0x7fb22403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:45.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.179+0000 7fb237400700 1 --2- 172.21.15.86:0/3045638684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb224039650 0x7fb22403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb228006fd0 tx=0x7fb228006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.184+0000 7fb23d600700 1 -- 172.21.15.86:0/3045638684 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb230013070 con 0x7fb240102f40 2024-04-17T07:02:45.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.412+0000 7fb246c00700 1 -- 172.21.15.86:0/3045638684 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb24002e180 con 0x7fb224039650 2024-04-17T07:02:45.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.420+0000 7fb23d600700 1 -- 172.21.15.86:0/3045638684 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb24002e180 con 0x7fb224039650 2024-04-17T07:02:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.423+0000 7fb246c00700 1 -- 172.21.15.86:0/3045638684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb224039650 msgr2=0x7fb22403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.423+0000 7fb246c00700 1 --2- 172.21.15.86:0/3045638684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb224039650 0x7fb22403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb228006fd0 tx=0x7fb228006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.423+0000 7fb246c00700 1 -- 172.21.15.86:0/3045638684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb240102f40 msgr2=0x7fb240197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.423+0000 7fb246c00700 1 --2- 172.21.15.86:0/3045638684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb240102f40 0x7fb240197aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb230005190 tx=0x7fb230004f80 comp rx=0 tx=0).stop 2024-04-17T07:02:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.424+0000 7fb246c00700 1 -- 172.21.15.86:0/3045638684 shutdown_connections 2024-04-17T07:02:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.424+0000 7fb246c00700 1 --2- 172.21.15.86:0/3045638684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb224039650 0x7fb22403bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.424+0000 7fb246c00700 1 --2- 172.21.15.86:0/3045638684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb240102f40 0x7fb240197aa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.424+0000 7fb246c00700 1 -- 172.21.15.86:0/3045638684 >> 172.21.15.86:0/3045638684 conn(0x7fb2400fc950 msgr2=0x7fb2400fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.424+0000 7fb246c00700 1 -- 172.21.15.86:0/3045638684 shutdown_connections 2024-04-17T07:02:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:45.424+0000 7fb246c00700 1 -- 172.21.15.86:0/3045638684 wait complete. 2024-04-17T07:02:46.146 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi098 2024-04-17T07:02:46.146 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T07:02:46.146 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.conf 2024-04-17T07:02:46.207 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T07:02:46.207 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:02:46.303 INFO:tasks.cephadm:Adding host smithi098 to orchestrator... 2024-04-17T07:02:46.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch host add smithi098 2024-04-17T07:02:46.713 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:02:46.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:46 smithi086 ceph-mon[31733]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:02:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:02:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:02:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:02:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:46 smithi086 ceph-mon[31733]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-17T07:02:47.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:47 smithi086 ceph-mon[31733]: Updating smithi086:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:02:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:47 smithi086 ceph-mon[31733]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:02:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:47 smithi086 ceph-mon[31733]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:02:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.775+0000 7f9b3a400700 1 -- 172.21.15.86:0/2440789108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34069e70 msgr2=0x7f9b3406a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.775+0000 7f9b3a400700 1 --2- 172.21.15.86:0/2440789108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34069e70 0x7f9b3406a250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009910 tx=0x7f9b24009c20 comp rx=0 tx=0).stop 2024-04-17T07:02:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.775+0000 7f9b3a400700 1 -- 172.21.15.86:0/2440789108 shutdown_connections 2024-04-17T07:02:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.775+0000 7f9b3a400700 1 --2- 172.21.15.86:0/2440789108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34069e70 0x7f9b3406a250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.775+0000 7f9b3a400700 1 -- 172.21.15.86:0/2440789108 >> 172.21.15.86:0/2440789108 conn(0x7f9b34076eb0 msgr2=0x7f9b340772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.775+0000 7f9b3a400700 1 -- 172.21.15.86:0/2440789108 shutdown_connections 2024-04-17T07:02:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.776+0000 7f9b3a400700 1 -- 172.21.15.86:0/2440789108 wait complete. 2024-04-17T07:02:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.777+0000 7f9b3a400700 1 Processor -- start 2024-04-17T07:02:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.777+0000 7f9b3a400700 1 -- start start 2024-04-17T07:02:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.777+0000 7f9b3a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34069e70 0x7f9b3419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.777+0000 7f9b3a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3410eba0 con 0x7f9b34069e70 2024-04-17T07:02:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.777+0000 7f9b33400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34069e70 0x7f9b3419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.777+0000 7f9b33400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34069e70 0x7f9b3419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40980/0 (socket says 172.21.15.86:40980) 2024-04-17T07:02:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.777+0000 7f9b33400700 1 -- 172.21.15.86:0/1679317710 learned_addr learned my addr 172.21.15.86:0/1679317710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.778+0000 7f9b33400700 1 -- 172.21.15.86:0/1679317710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3419a650 con 0x7f9b34069e70 2024-04-17T07:02:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.778+0000 7f9b33400700 1 --2- 172.21.15.86:0/1679317710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34069e70 0x7f9b3419a110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b240056d0 tx=0x7f9b240057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:48.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.778+0000 7f9b30c00700 1 -- 172.21.15.86:0/1679317710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b24026070 con 0x7f9b34069e70 2024-04-17T07:02:48.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.778+0000 7f9b3a400700 1 -- 172.21.15.86:0/1679317710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3419a8e0 con 0x7f9b34069e70 2024-04-17T07:02:48.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.778+0000 7f9b30c00700 1 -- 172.21.15.86:0/1679317710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b2400bc00 con 0x7f9b34069e70 2024-04-17T07:02:48.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.778+0000 7f9b30c00700 1 -- 172.21.15.86:0/1679317710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b240039b0 con 0x7f9b34069e70 2024-04-17T07:02:48.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.779+0000 7f9b3a400700 1 -- 172.21.15.86:0/1679317710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3419ad40 con 0x7f9b34069e70 2024-04-17T07:02:48.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.780+0000 7f9b3a400700 1 -- 172.21.15.86:0/1679317710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b34101980 con 0x7f9b34069e70 2024-04-17T07:02:48.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.780+0000 7f9b30c00700 1 -- 172.21.15.86:0/1679317710 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9b24029070 con 0x7f9b34069e70 2024-04-17T07:02:48.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.780+0000 7f9b30c00700 1 --2- 172.21.15.86:0/1679317710 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b18039650 0x7f9b1803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:48.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.780+0000 7f9b30c00700 1 -- 172.21.15.86:0/1679317710 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9b2401e070 con 0x7f9b34069e70 2024-04-17T07:02:48.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.781+0000 7f9b2be00700 1 --2- 172.21.15.86:0/1679317710 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b18039650 0x7f9b1803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:48.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.781+0000 7f9b2be00700 1 --2- 172.21.15.86:0/1679317710 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b18039650 0x7f9b1803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c006fd0 tx=0x7f9b1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:48.786+0000 7f9b30c00700 1 -- 172.21.15.86:0/1679317710 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9b24013070 con 0x7f9b34069e70 2024-04-17T07:02:49.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:49.016+0000 7f9b3a400700 1 -- 172.21.15.86:0/1679317710 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}) -- 0x7f9b3419b470 con 0x7f9b18039650 2024-04-17T07:02:49.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:48 smithi086 ceph-mon[31733]: Updating smithi086:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T07:02:49.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:49.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:49.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:49 smithi086 ceph-mon[31733]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:02:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:49 smithi086 ceph-mon[31733]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:02:52.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:51 smithi086 ceph-mon[31733]: Deploying cephadm binary to smithi098 2024-04-17T07:02:52.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:51 smithi086 ceph-mon[31733]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:02:54.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:53 smithi086 ceph-mon[31733]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:02:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.528+0000 7f9b30c00700 1 -- 172.21.15.86:0/1679317710 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9b3419b470 con 0x7f9b18039650 2024-04-17T07:02:55.528 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi098' with addr '172.21.15.98' 2024-04-17T07:02:55.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.531+0000 7f9b3a400700 1 -- 172.21.15.86:0/1679317710 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b18039650 msgr2=0x7f9b1803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:55.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.531+0000 7f9b3a400700 1 --2- 172.21.15.86:0/1679317710 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b18039650 0x7f9b1803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c006fd0 tx=0x7f9b1c006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:55.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.531+0000 7f9b3a400700 1 -- 172.21.15.86:0/1679317710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34069e70 msgr2=0x7f9b3419a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:55.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.531+0000 7f9b3a400700 1 --2- 172.21.15.86:0/1679317710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34069e70 0x7f9b3419a110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b240056d0 tx=0x7f9b240057b0 comp rx=0 tx=0).stop 2024-04-17T07:02:55.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.531+0000 7f9b3a400700 1 -- 172.21.15.86:0/1679317710 shutdown_connections 2024-04-17T07:02:55.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.531+0000 7f9b3a400700 1 --2- 172.21.15.86:0/1679317710 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b18039650 0x7f9b1803bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:55.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.531+0000 7f9b3a400700 1 --2- 172.21.15.86:0/1679317710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b34069e70 0x7f9b3419a110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:55.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.531+0000 7f9b3a400700 1 -- 172.21.15.86:0/1679317710 >> 172.21.15.86:0/1679317710 conn(0x7f9b34076eb0 msgr2=0x7f9b341099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:55.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.532+0000 7f9b3a400700 1 -- 172.21.15.86:0/1679317710 shutdown_connections 2024-04-17T07:02:55.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:55.532+0000 7f9b3a400700 1 -- 172.21.15.86:0/1679317710 wait complete. 2024-04-17T07:02:56.249 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:55 smithi086 ceph-mon[31733]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:02:56.249 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:56.249 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:02:56.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch host ls --format=json 2024-04-17T07:02:56.729 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:02:57.137 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:56 smithi086 ceph-mon[31733]: Added host smithi098 2024-04-17T07:02:57.137 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:57.995 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:57 smithi086 ceph-mon[31733]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:02:58.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:02:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.682+0000 7f7961600700 1 -- 172.21.15.86:0/547111691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795c1084f0 msgr2=0x7f795c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.682+0000 7f7961600700 1 --2- 172.21.15.86:0/547111691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795c1084f0 0x7f795c1088d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f794c009910 tx=0x7f794c009c20 comp rx=0 tx=0).stop 2024-04-17T07:02:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.682+0000 7f7961600700 1 -- 172.21.15.86:0/547111691 shutdown_connections 2024-04-17T07:02:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.682+0000 7f7961600700 1 --2- 172.21.15.86:0/547111691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795c1084f0 0x7f795c1088d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.682+0000 7f7961600700 1 -- 172.21.15.86:0/547111691 >> 172.21.15.86:0/547111691 conn(0x7f795c0fdf90 msgr2=0x7f795c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.682+0000 7f7961600700 1 -- 172.21.15.86:0/547111691 shutdown_connections 2024-04-17T07:02:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.683+0000 7f7961600700 1 -- 172.21.15.86:0/547111691 wait complete. 2024-04-17T07:02:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.683+0000 7f7961600700 1 Processor -- start 2024-04-17T07:02:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.684+0000 7f7961600700 1 -- start start 2024-04-17T07:02:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.684+0000 7f7961600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795c1084f0 0x7f795c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.684+0000 7f7961600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10c950 con 0x7f795c1084f0 2024-04-17T07:02:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.684+0000 7f795aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795c1084f0 0x7f795c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:58.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.684+0000 7f795aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795c1084f0 0x7f795c19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45708/0 (socket says 172.21.15.86:45708) 2024-04-17T07:02:58.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.684+0000 7f795aa00700 1 -- 172.21.15.86:0/224274375 learned_addr learned my addr 172.21.15.86:0/224274375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:02:58.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.685+0000 7f795aa00700 1 -- 172.21.15.86:0/224274375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c19a610 con 0x7f795c1084f0 2024-04-17T07:02:58.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.685+0000 7f795aa00700 1 --2- 172.21.15.86:0/224274375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795c1084f0 0x7f795c19a0d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f794c0056d0 tx=0x7f794c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:58.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.685+0000 7f7953400700 1 -- 172.21.15.86:0/224274375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c026070 con 0x7f795c1084f0 2024-04-17T07:02:58.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.685+0000 7f7953400700 1 -- 172.21.15.86:0/224274375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f794c00bba0 con 0x7f795c1084f0 2024-04-17T07:02:58.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.685+0000 7f7961600700 1 -- 172.21.15.86:0/224274375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c19a900 con 0x7f795c1084f0 2024-04-17T07:02:58.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.685+0000 7f7953400700 1 -- 172.21.15.86:0/224274375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c0037f0 con 0x7f795c1084f0 2024-04-17T07:02:58.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.686+0000 7f7961600700 1 -- 172.21.15.86:0/224274375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c19e9a0 con 0x7f795c1084f0 2024-04-17T07:02:58.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.687+0000 7f7961600700 1 -- 172.21.15.86:0/224274375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c103ba0 con 0x7f795c1084f0 2024-04-17T07:02:58.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.687+0000 7f7953400700 1 -- 172.21.15.86:0/224274375 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f794c029070 con 0x7f795c1084f0 2024-04-17T07:02:58.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.687+0000 7f7953400700 1 --2- 172.21.15.86:0/224274375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7940039650 0x7f794003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:02:58.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.687+0000 7f7953400700 1 -- 172.21.15.86:0/224274375 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f794c01e070 con 0x7f795c1084f0 2024-04-17T07:02:58.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.688+0000 7f795a000700 1 --2- 172.21.15.86:0/224274375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7940039650 0x7f794003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:02:58.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.688+0000 7f795a000700 1 --2- 172.21.15.86:0/224274375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7940039650 0x7f794003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7948006fd0 tx=0x7f7948006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:02:58.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.693+0000 7f7953400700 1 -- 172.21.15.86:0/224274375 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f794c013070 con 0x7f795c1084f0 2024-04-17T07:02:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.923+0000 7f7961600700 1 -- 172.21.15.86:0/224274375 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f795c19b430 con 0x7f7940039650 2024-04-17T07:02:58.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.926+0000 7f7953400700 1 -- 172.21.15.86:0/224274375 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f795c19b430 con 0x7f7940039650 2024-04-17T07:02:58.926 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:02:58.926 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}] 2024-04-17T07:02:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.929+0000 7f7961600700 1 -- 172.21.15.86:0/224274375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7940039650 msgr2=0x7f794003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.929+0000 7f7961600700 1 --2- 172.21.15.86:0/224274375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7940039650 0x7f794003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7948006fd0 tx=0x7f7948006e40 comp rx=0 tx=0).stop 2024-04-17T07:02:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.929+0000 7f7961600700 1 -- 172.21.15.86:0/224274375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795c1084f0 msgr2=0x7f795c19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:02:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.929+0000 7f7961600700 1 --2- 172.21.15.86:0/224274375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795c1084f0 0x7f795c19a0d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f794c0056d0 tx=0x7f794c0057b0 comp rx=0 tx=0).stop 2024-04-17T07:02:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.929+0000 7f7961600700 1 -- 172.21.15.86:0/224274375 shutdown_connections 2024-04-17T07:02:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.929+0000 7f7961600700 1 --2- 172.21.15.86:0/224274375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7940039650 0x7f794003bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.929+0000 7f7961600700 1 --2- 172.21.15.86:0/224274375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f795c1084f0 0x7f795c19a0d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:02:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.929+0000 7f7961600700 1 -- 172.21.15.86:0/224274375 >> 172.21.15.86:0/224274375 conn(0x7f795c0fdf90 msgr2=0x7f795c10a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:02:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.929+0000 7f7961600700 1 -- 172.21.15.86:0/224274375 shutdown_connections 2024-04-17T07:02:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:02:58.929+0000 7f7961600700 1 -- 172.21.15.86:0/224274375 wait complete. 2024-04-17T07:02:59.588 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi103 2024-04-17T07:02:59.588 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T07:02:59.588 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.conf 2024-04-17T07:02:59.645 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T07:02:59.645 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:02:59.741 INFO:tasks.cephadm:Adding host smithi103 to orchestrator... 2024-04-17T07:02:59.742 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch host add smithi103 2024-04-17T07:03:00.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:59 smithi086 ceph-mon[31733]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:00.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:59 smithi086 ceph-mon[31733]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:03:00.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:02:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:00.167 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:03:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.013+0000 7fe471600700 1 -- 172.21.15.86:0/3606565074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c074d70 msgr2=0x7fe46c075150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.013+0000 7fe471600700 1 --2- 172.21.15.86:0/3606565074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c074d70 0x7fe46c075150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009910 tx=0x7fe45c009c20 comp rx=0 tx=0).stop 2024-04-17T07:03:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.013+0000 7fe471600700 1 -- 172.21.15.86:0/3606565074 shutdown_connections 2024-04-17T07:03:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.013+0000 7fe471600700 1 --2- 172.21.15.86:0/3606565074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c074d70 0x7fe46c075150 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.013+0000 7fe471600700 1 -- 172.21.15.86:0/3606565074 >> 172.21.15.86:0/3606565074 conn(0x7fe46c0fdf90 msgr2=0x7fe46c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.013+0000 7fe471600700 1 -- 172.21.15.86:0/3606565074 shutdown_connections 2024-04-17T07:03:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.013+0000 7fe471600700 1 -- 172.21.15.86:0/3606565074 wait complete. 2024-04-17T07:03:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.014+0000 7fe471600700 1 Processor -- start 2024-04-17T07:03:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.014+0000 7fe471600700 1 -- start start 2024-04-17T07:03:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.014+0000 7fe471600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c074d70 0x7fe46c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.014+0000 7fe471600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c10eba0 con 0x7fe46c074d70 2024-04-17T07:03:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.015+0000 7fe46aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c074d70 0x7fe46c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.015+0000 7fe46aa00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c074d70 0x7fe46c19a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45724/0 (socket says 172.21.15.86:45724) 2024-04-17T07:03:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.015+0000 7fe46aa00700 1 -- 172.21.15.86:0/2531130203 learned_addr learned my addr 172.21.15.86:0/2531130203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:03:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.015+0000 7fe46aa00700 1 -- 172.21.15.86:0/2531130203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c19a640 con 0x7fe46c074d70 2024-04-17T07:03:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.015+0000 7fe46aa00700 1 --2- 172.21.15.86:0/2531130203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c074d70 0x7fe46c19a100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe45c006010 tx=0x7fe45c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.016+0000 7fe463400700 1 -- 172.21.15.86:0/2531130203 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c026070 con 0x7fe46c074d70 2024-04-17T07:03:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.016+0000 7fe463400700 1 -- 172.21.15.86:0/2531130203 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe45c00bc00 con 0x7fe46c074d70 2024-04-17T07:03:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.016+0000 7fe471600700 1 -- 172.21.15.86:0/2531130203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c19a8d0 con 0x7fe46c074d70 2024-04-17T07:03:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.016+0000 7fe463400700 1 -- 172.21.15.86:0/2531130203 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c003910 con 0x7fe46c074d70 2024-04-17T07:03:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.016+0000 7fe471600700 1 -- 172.21.15.86:0/2531130203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c19ad30 con 0x7fe46c074d70 2024-04-17T07:03:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.017+0000 7fe463400700 1 -- 172.21.15.86:0/2531130203 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe45c029070 con 0x7fe46c074d70 2024-04-17T07:03:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.017+0000 7fe471600700 1 -- 172.21.15.86:0/2531130203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46c0764b0 con 0x7fe46c074d70 2024-04-17T07:03:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.018+0000 7fe463400700 1 --2- 172.21.15.86:0/2531130203 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4500396b0 0x7fe45003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.018+0000 7fe463400700 1 -- 172.21.15.86:0/2531130203 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe45c01e070 con 0x7fe46c074d70 2024-04-17T07:03:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.018+0000 7fe463e00700 1 --2- 172.21.15.86:0/2531130203 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4500396b0 0x7fe45003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.019+0000 7fe463e00700 1 --2- 172.21.15.86:0/2531130203 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4500396b0 0x7fe45003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe454006fd0 tx=0x7fe454006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.023+0000 7fe463400700 1 -- 172.21.15.86:0/2531130203 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe45c013070 con 0x7fe46c074d70 2024-04-17T07:03:02.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:02.251+0000 7fe471600700 1 -- 172.21.15.86:0/2531130203 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}) -- 0x7fe46c19b4d0 con 0x7fe4500396b0 2024-04-17T07:03:02.252 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:01 smithi086 ceph-mon[31733]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:04.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:04 smithi086 ceph-mon[31733]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:03:04.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:04 smithi086 ceph-mon[31733]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:05.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:05 smithi086 ceph-mon[31733]: Deploying cephadm binary to smithi103 2024-04-17T07:03:05.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:05.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:05.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:05.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:05.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:03:05.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:05.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:06.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:06 smithi086 ceph-mon[31733]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T07:03:06.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:06 smithi086 ceph-mon[31733]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:08 smithi086 ceph-mon[31733]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.786+0000 7fe463400700 1 -- 172.21.15.86:0/2531130203 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fe46c19b4d0 con 0x7fe4500396b0 2024-04-17T07:03:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi103' with addr '172.21.15.103' 2024-04-17T07:03:08.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.790+0000 7fe471600700 1 -- 172.21.15.86:0/2531130203 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4500396b0 msgr2=0x7fe45003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:08.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.790+0000 7fe471600700 1 --2- 172.21.15.86:0/2531130203 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4500396b0 0x7fe45003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe454006fd0 tx=0x7fe454006e40 comp rx=0 tx=0).stop 2024-04-17T07:03:08.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.790+0000 7fe471600700 1 -- 172.21.15.86:0/2531130203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c074d70 msgr2=0x7fe46c19a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:08.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.790+0000 7fe471600700 1 --2- 172.21.15.86:0/2531130203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c074d70 0x7fe46c19a100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe45c006010 tx=0x7fe45c005780 comp rx=0 tx=0).stop 2024-04-17T07:03:08.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.790+0000 7fe471600700 1 -- 172.21.15.86:0/2531130203 shutdown_connections 2024-04-17T07:03:08.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.790+0000 7fe471600700 1 --2- 172.21.15.86:0/2531130203 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4500396b0 0x7fe45003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:08.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.790+0000 7fe471600700 1 --2- 172.21.15.86:0/2531130203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c074d70 0x7fe46c19a100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:08.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.790+0000 7fe471600700 1 -- 172.21.15.86:0/2531130203 >> 172.21.15.86:0/2531130203 conn(0x7fe46c0fdf90 msgr2=0x7fe46c106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:08.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.790+0000 7fe471600700 1 -- 172.21.15.86:0/2531130203 shutdown_connections 2024-04-17T07:03:08.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:08.790+0000 7fe471600700 1 -- 172.21.15.86:0/2531130203 wait complete. 2024-04-17T07:03:09.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:09 smithi086 ceph-mon[31733]: Updating smithi098:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:09.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:09.626 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch host ls --format=json 2024-04-17T07:03:10.069 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:03:10.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:10 smithi086 ceph-mon[31733]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:03:10.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:10 smithi086 ceph-mon[31733]: Added host smithi103 2024-04-17T07:03:10.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:10 smithi086 ceph-mon[31733]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:10.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:10.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:10.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:11.223 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:11 smithi086 ceph-mon[31733]: Updating smithi098:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T07:03:11.223 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:11 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:12.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.061+0000 7fc8c3200700 1 -- 172.21.15.86:0/239352116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8bc10d970 msgr2=0x7fc8bc10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.061+0000 7fc8c3200700 1 --2- 172.21.15.86:0/239352116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8bc10d970 0x7fc8bc10dd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009910 tx=0x7fc8ac009c20 comp rx=0 tx=0).stop 2024-04-17T07:03:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.061+0000 7fc8c3200700 1 -- 172.21.15.86:0/239352116 shutdown_connections 2024-04-17T07:03:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.061+0000 7fc8c3200700 1 --2- 172.21.15.86:0/239352116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8bc10d970 0x7fc8bc10dd50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.061+0000 7fc8c3200700 1 -- 172.21.15.86:0/239352116 >> 172.21.15.86:0/239352116 conn(0x7fc8bc0781f0 msgr2=0x7fc8bc07a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.061+0000 7fc8c3200700 1 -- 172.21.15.86:0/239352116 shutdown_connections 2024-04-17T07:03:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.061+0000 7fc8c3200700 1 -- 172.21.15.86:0/239352116 wait complete. 2024-04-17T07:03:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.062+0000 7fc8c3200700 1 Processor -- start 2024-04-17T07:03:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.062+0000 7fc8c3200700 1 -- start start 2024-04-17T07:03:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.063+0000 7fc8c3200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8bc10d970 0x7fc8bc19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.063+0000 7fc8c3200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc112f70 con 0x7fc8bc10d970 2024-04-17T07:03:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.063+0000 7fc8bbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8bc10d970 0x7fc8bc19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.063+0000 7fc8bbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8bc10d970 0x7fc8bc19e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50760/0 (socket says 172.21.15.86:50760) 2024-04-17T07:03:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.063+0000 7fc8bbe00700 1 -- 172.21.15.86:0/2040574102 learned_addr learned my addr 172.21.15.86:0/2040574102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:03:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.063+0000 7fc8bbe00700 1 -- 172.21.15.86:0/2040574102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc19e9c0 con 0x7fc8bc10d970 2024-04-17T07:03:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.064+0000 7fc8bbe00700 1 --2- 172.21.15.86:0/2040574102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8bc10d970 0x7fc8bc19e480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac006010 tx=0x7fc8ac005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.064+0000 7fc8b8c00700 1 -- 172.21.15.86:0/2040574102 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8ac026070 con 0x7fc8bc10d970 2024-04-17T07:03:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.064+0000 7fc8b8c00700 1 -- 172.21.15.86:0/2040574102 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8ac00bba0 con 0x7fc8bc10d970 2024-04-17T07:03:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.064+0000 7fc8c3200700 1 -- 172.21.15.86:0/2040574102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8bc19ecb0 con 0x7fc8bc10d970 2024-04-17T07:03:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.064+0000 7fc8b8c00700 1 -- 172.21.15.86:0/2040574102 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8ac003910 con 0x7fc8bc10d970 2024-04-17T07:03:12.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.064+0000 7fc8c3200700 1 -- 172.21.15.86:0/2040574102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8bc1a2d50 con 0x7fc8bc10d970 2024-04-17T07:03:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.065+0000 7fc8b8c00700 1 -- 172.21.15.86:0/2040574102 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc8ac029070 con 0x7fc8bc10d970 2024-04-17T07:03:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.065+0000 7fc8c3200700 1 -- 172.21.15.86:0/2040574102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8bc10a190 con 0x7fc8bc10d970 2024-04-17T07:03:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.066+0000 7fc8b8c00700 1 --2- 172.21.15.86:0/2040574102 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8a0039650 0x7fc8a003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.066+0000 7fc8b8c00700 1 -- 172.21.15.86:0/2040574102 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc8ac01e070 con 0x7fc8bc10d970 2024-04-17T07:03:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.066+0000 7fc8bb400700 1 --2- 172.21.15.86:0/2040574102 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8a0039650 0x7fc8a003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.067+0000 7fc8bb400700 1 --2- 172.21.15.86:0/2040574102 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8a0039650 0x7fc8a003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8006fd0 tx=0x7fc8a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.071+0000 7fc8b8c00700 1 -- 172.21.15.86:0/2040574102 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc8ac013070 con 0x7fc8bc10d970 2024-04-17T07:03:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.303+0000 7fc8c3200700 1 -- 172.21.15.86:0/2040574102 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8bc19f7e0 con 0x7fc8a0039650 2024-04-17T07:03:12.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:12 smithi086 ceph-mon[31733]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:12.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:12 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:12.303 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:12 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:12.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.305+0000 7fc8b8c00700 1 -- 172.21.15.86:0/2040574102 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fc8bc19f7e0 con 0x7fc8a0039650 2024-04-17T07:03:12.305 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:03:12.305 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}] 2024-04-17T07:03:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.307+0000 7fc8c3200700 1 -- 172.21.15.86:0/2040574102 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8a0039650 msgr2=0x7fc8a003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.307+0000 7fc8c3200700 1 --2- 172.21.15.86:0/2040574102 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8a0039650 0x7fc8a003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8006fd0 tx=0x7fc8a8006e40 comp rx=0 tx=0).stop 2024-04-17T07:03:12.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.307+0000 7fc8c3200700 1 -- 172.21.15.86:0/2040574102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8bc10d970 msgr2=0x7fc8bc19e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.307+0000 7fc8c3200700 1 --2- 172.21.15.86:0/2040574102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8bc10d970 0x7fc8bc19e480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac006010 tx=0x7fc8ac005780 comp rx=0 tx=0).stop 2024-04-17T07:03:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.308+0000 7fc8c3200700 1 -- 172.21.15.86:0/2040574102 shutdown_connections 2024-04-17T07:03:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.308+0000 7fc8c3200700 1 --2- 172.21.15.86:0/2040574102 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8a0039650 0x7fc8a003bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.308+0000 7fc8c3200700 1 --2- 172.21.15.86:0/2040574102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8bc10d970 0x7fc8bc19e480 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.308+0000 7fc8c3200700 1 -- 172.21.15.86:0/2040574102 >> 172.21.15.86:0/2040574102 conn(0x7fc8bc0781f0 msgr2=0x7fc8bc078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.308+0000 7fc8c3200700 1 -- 172.21.15.86:0/2040574102 shutdown_connections 2024-04-17T07:03:12.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:12.308+0000 7fc8c3200700 1 -- 172.21.15.86:0/2040574102 wait complete. 2024-04-17T07:03:13.004 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-17T07:03:13.005 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd crush tunables default 2024-04-17T07:03:13.441 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:03:14.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:14 smithi086 ceph-mon[31733]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:03:14.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:14 smithi086 ceph-mon[31733]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:15.213 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:15.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.381+0000 7f8be8e00700 1 -- 172.21.15.86:0/3552178271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be4074df0 msgr2=0x7f8be40751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.381+0000 7f8be8e00700 1 --2- 172.21.15.86:0/3552178271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be4074df0 0x7f8be40751d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400b0a0 tx=0x7f8bd400b3b0 comp rx=0 tx=0).stop 2024-04-17T07:03:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.381+0000 7f8be8e00700 1 -- 172.21.15.86:0/3552178271 shutdown_connections 2024-04-17T07:03:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.381+0000 7f8be8e00700 1 --2- 172.21.15.86:0/3552178271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be4074df0 0x7f8be40751d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.381+0000 7f8be8e00700 1 -- 172.21.15.86:0/3552178271 >> 172.21.15.86:0/3552178271 conn(0x7f8be40fdf90 msgr2=0x7f8be41003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.381+0000 7f8be8e00700 1 -- 172.21.15.86:0/3552178271 shutdown_connections 2024-04-17T07:03:15.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.381+0000 7f8be8e00700 1 -- 172.21.15.86:0/3552178271 wait complete. 2024-04-17T07:03:15.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.382+0000 7f8be8e00700 1 Processor -- start 2024-04-17T07:03:15.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.382+0000 7f8be8e00700 1 -- start start 2024-04-17T07:03:15.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.383+0000 7f8be8e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be4074df0 0x7f8be419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:15.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.383+0000 7f8be8e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8be4112fb0 con 0x7f8be4074df0 2024-04-17T07:03:15.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.383+0000 7f8be1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be4074df0 0x7f8be419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:15.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.383+0000 7f8be1e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be4074df0 0x7f8be419e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50784/0 (socket says 172.21.15.86:50784) 2024-04-17T07:03:15.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.383+0000 7f8be1e00700 1 -- 172.21.15.86:0/900910452 learned_addr learned my addr 172.21.15.86:0/900910452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:03:15.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.383+0000 7f8be1e00700 1 -- 172.21.15.86:0/900910452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be419e9b0 con 0x7f8be4074df0 2024-04-17T07:03:15.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.384+0000 7f8be1e00700 1 --2- 172.21.15.86:0/900910452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be4074df0 0x7f8be419e470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4000f80 tx=0x7f8bd4009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:15.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.384+0000 7f8bdb400700 1 -- 172.21.15.86:0/900910452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bd4017040 con 0x7f8be4074df0 2024-04-17T07:03:15.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.384+0000 7f8bdb400700 1 -- 172.21.15.86:0/900910452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bd4022070 con 0x7f8be4074df0 2024-04-17T07:03:15.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.384+0000 7f8be8e00700 1 -- 172.21.15.86:0/900910452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be419ec40 con 0x7f8be4074df0 2024-04-17T07:03:15.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.384+0000 7f8bdb400700 1 -- 172.21.15.86:0/900910452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bd4004910 con 0x7f8be4074df0 2024-04-17T07:03:15.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.384+0000 7f8be8e00700 1 -- 172.21.15.86:0/900910452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be419f0a0 con 0x7f8be4074df0 2024-04-17T07:03:15.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.386+0000 7f8be8e00700 1 -- 172.21.15.86:0/900910452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be4076530 con 0x7f8be4074df0 2024-04-17T07:03:15.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.386+0000 7f8bdb400700 1 -- 172.21.15.86:0/900910452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8bd4004040 con 0x7f8be4074df0 2024-04-17T07:03:15.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.386+0000 7f8bdb400700 1 --2- 172.21.15.86:0/900910452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8bc80396f0 0x7f8bc803bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:15.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.386+0000 7f8bdb400700 1 -- 172.21.15.86:0/900910452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8bd400f050 con 0x7f8be4074df0 2024-04-17T07:03:15.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.387+0000 7f8bd9400700 1 --2- 172.21.15.86:0/900910452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8bc80396f0 0x7f8bc803bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:15.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.387+0000 7f8bd9400700 1 --2- 172.21.15.86:0/900910452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8bc80396f0 0x7f8bc803bbb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006fd0 tx=0x7f8bcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.392+0000 7f8bdb400700 1 -- 172.21.15.86:0/900910452 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8bd401c5f0 con 0x7f8be4074df0 2024-04-17T07:03:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:15.625+0000 7f8be8e00700 1 -- 172.21.15.86:0/900910452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8be419f7d0 con 0x7f8be4074df0 2024-04-17T07:03:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.059+0000 7f8bdb400700 1 -- 172.21.15.86:0/900910452 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f8bd40192e0 con 0x7f8be4074df0 2024-04-17T07:03:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.063+0000 7f8be8e00700 1 -- 172.21.15.86:0/900910452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8bc80396f0 msgr2=0x7f8bc803bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.063+0000 7f8be8e00700 1 --2- 172.21.15.86:0/900910452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8bc80396f0 0x7f8bc803bbb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc006fd0 tx=0x7f8bcc006e40 comp rx=0 tx=0).stop 2024-04-17T07:03:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.063+0000 7f8be8e00700 1 -- 172.21.15.86:0/900910452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be4074df0 msgr2=0x7f8be419e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.063+0000 7f8be8e00700 1 --2- 172.21.15.86:0/900910452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be4074df0 0x7f8be419e470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4000f80 tx=0x7f8bd4009790 comp rx=0 tx=0).stop 2024-04-17T07:03:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.063+0000 7f8be8e00700 1 -- 172.21.15.86:0/900910452 shutdown_connections 2024-04-17T07:03:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.063+0000 7f8be8e00700 1 --2- 172.21.15.86:0/900910452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8bc80396f0 0x7f8bc803bbb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.063+0000 7f8be8e00700 1 --2- 172.21.15.86:0/900910452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8be4074df0 0x7f8be419e470 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.063+0000 7f8be8e00700 1 -- 172.21.15.86:0/900910452 >> 172.21.15.86:0/900910452 conn(0x7f8be40fdf90 msgr2=0x7f8be40fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.064+0000 7f8be8e00700 1 -- 172.21.15.86:0/900910452 shutdown_connections 2024-04-17T07:03:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:16.064+0000 7f8be8e00700 1 -- 172.21.15.86:0/900910452 wait complete. 2024-04-17T07:03:16.065 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-04-17T07:03:16.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:16 smithi086 ceph-mon[31733]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:16.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:16 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/900910452' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-17T07:03:16.839 INFO:tasks.cephadm:Adding mon.a on smithi086 2024-04-17T07:03:16.839 INFO:tasks.cephadm:Adding mon.b on smithi098 2024-04-17T07:03:16.839 INFO:tasks.cephadm:Adding mon.c on smithi103 2024-04-17T07:03:16.839 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch apply mon '3;smithi086:172.21.15.86=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c' 2024-04-17T07:03:17.261 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-17T07:03:17.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:17 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/900910452' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-17T07:03:17.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:17 smithi086 ceph-mon[31733]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:17.364 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-17T07:03:18.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:18 smithi086 ceph-mon[31733]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:19.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:19 smithi086 ceph-mon[31733]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:19.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.827+0000 7f3fc2800700 1 -- 172.21.15.103:0/4230276068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102da0 msgr2=0x7f3fbc105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.827+0000 7f3fc2800700 1 --2- 172.21.15.103:0/4230276068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102da0 0x7f3fbc105190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009910 tx=0x7f3fb0009c20 comp rx=0 tx=0).stop 2024-04-17T07:03:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.827+0000 7f3fc2800700 1 -- 172.21.15.103:0/4230276068 shutdown_connections 2024-04-17T07:03:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.827+0000 7f3fc2800700 1 --2- 172.21.15.103:0/4230276068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102da0 0x7f3fbc105190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.827+0000 7f3fc2800700 1 -- 172.21.15.103:0/4230276068 >> 172.21.15.103:0/4230276068 conn(0x7f3fbc0fc7b0 msgr2=0x7f3fbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.827+0000 7f3fc2800700 1 -- 172.21.15.103:0/4230276068 shutdown_connections 2024-04-17T07:03:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.827+0000 7f3fc2800700 1 -- 172.21.15.103:0/4230276068 wait complete. 2024-04-17T07:03:19.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.828+0000 7f3fc2800700 1 Processor -- start 2024-04-17T07:03:19.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.828+0000 7f3fc2800700 1 -- start start 2024-04-17T07:03:19.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.829+0000 7f3fc2800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102da0 0x7f3fbc197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:19.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.829+0000 7f3fc2800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1081f0 con 0x7f3fbc102da0 2024-04-17T07:03:19.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.829+0000 7f3fc1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102da0 0x7f3fbc197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:19.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.829+0000 7f3fc1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102da0 0x7f3fbc197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.103:33192/0 (socket says 172.21.15.103:33192) 2024-04-17T07:03:19.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.830+0000 7f3fc1400700 1 -- 172.21.15.103:0/98442885 learned_addr learned my addr 172.21.15.103:0/98442885 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:03:19.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.830+0000 7f3fc1400700 1 -- 172.21.15.103:0/98442885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc197dd0 con 0x7f3fbc102da0 2024-04-17T07:03:19.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.831+0000 7f3fc1400700 1 --2- 172.21.15.103:0/98442885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102da0 0x7f3fbc197890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0005b40 tx=0x7f3fb0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:19.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.831+0000 7f3fba000700 1 -- 172.21.15.103:0/98442885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0027070 con 0x7f3fbc102da0 2024-04-17T07:03:19.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.831+0000 7f3fc2800700 1 -- 172.21.15.103:0/98442885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fbc1980c0 con 0x7f3fbc102da0 2024-04-17T07:03:19.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.831+0000 7f3fc2800700 1 -- 172.21.15.103:0/98442885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fbc19ad50 con 0x7f3fbc102da0 2024-04-17T07:03:19.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.831+0000 7f3fba000700 1 -- 172.21.15.103:0/98442885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb0004800 con 0x7f3fbc102da0 2024-04-17T07:03:19.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.832+0000 7f3fba000700 1 -- 172.21.15.103:0/98442885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0031060 con 0x7f3fbc102da0 2024-04-17T07:03:19.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.832+0000 7f3fc2800700 1 -- 172.21.15.103:0/98442885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f9c005360 con 0x7f3fbc102da0 2024-04-17T07:03:19.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.833+0000 7f3fba000700 1 -- 172.21.15.103:0/98442885 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3fb002a070 con 0x7f3fbc102da0 2024-04-17T07:03:19.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.833+0000 7f3fba000700 1 --2- 172.21.15.103:0/98442885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa40396b0 0x7f3fa403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:19.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.833+0000 7f3fba000700 1 -- 172.21.15.103:0/98442885 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3fb001e030 con 0x7f3fbc102da0 2024-04-17T07:03:19.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.834+0000 7f3fc0a00700 1 --2- 172.21.15.103:0/98442885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa40396b0 0x7f3fa403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:19.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.835+0000 7f3fc0a00700 1 --2- 172.21.15.103:0/98442885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa40396b0 0x7f3fa403bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8006fd0 tx=0x7f3fa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:19.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:19.849+0000 7f3fba000700 1 -- 172.21.15.103:0/98442885 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3fb0013070 con 0x7f3fbc102da0 2024-04-17T07:03:20.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.091+0000 7f3fc2800700 1 -- 172.21.15.103:0/98442885 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c", "target": ["mon-mgr", ""]}) -- 0x7f3f9c000cb0 con 0x7f3fa40396b0 2024-04-17T07:03:20.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.106+0000 7f3fba000700 1 -- 172.21.15.103:0/98442885 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3f9c000cb0 con 0x7f3fa40396b0 2024-04-17T07:03:20.106 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled mon update... 2024-04-17T07:03:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.109+0000 7f3fc2800700 1 -- 172.21.15.103:0/98442885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa40396b0 msgr2=0x7f3fa403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.109+0000 7f3fc2800700 1 --2- 172.21.15.103:0/98442885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa40396b0 0x7f3fa403bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8006fd0 tx=0x7f3fa8006e40 comp rx=0 tx=0).stop 2024-04-17T07:03:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.109+0000 7f3fc2800700 1 -- 172.21.15.103:0/98442885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102da0 msgr2=0x7f3fbc197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.109+0000 7f3fc2800700 1 --2- 172.21.15.103:0/98442885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102da0 0x7f3fbc197890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0005b40 tx=0x7f3fb0004f80 comp rx=0 tx=0).stop 2024-04-17T07:03:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.109+0000 7f3fc2800700 1 -- 172.21.15.103:0/98442885 shutdown_connections 2024-04-17T07:03:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.109+0000 7f3fc2800700 1 --2- 172.21.15.103:0/98442885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa40396b0 0x7f3fa403bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.109+0000 7f3fc2800700 1 --2- 172.21.15.103:0/98442885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fbc102da0 0x7f3fbc197890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.109+0000 7f3fc2800700 1 -- 172.21.15.103:0/98442885 >> 172.21.15.103:0/98442885 conn(0x7f3fbc0fc7b0 msgr2=0x7f3fbc0fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.109+0000 7f3fc2800700 1 -- 172.21.15.103:0/98442885 shutdown_connections 2024-04-17T07:03:20.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:20.109+0000 7f3fc2800700 1 -- 172.21.15.103:0/98442885 wait complete. 2024-04-17T07:03:20.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:20.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:20.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:20.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:20.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:03:20.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:20.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:20.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:20 smithi086 ceph-mon[31733]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T07:03:20.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:20.858 DEBUG:teuthology.orchestra.run.smithi098:mon.b> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.b.service 2024-04-17T07:03:20.866 DEBUG:teuthology.orchestra.run.smithi103:mon.c> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.c.service 2024-04-17T07:03:20.871 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-17T07:03:20.871 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph mon dump -f json 2024-04-17T07:03:21.310 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:21.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:21 smithi086 ceph-mon[31733]: Updating smithi103:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:21.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:21 smithi086 ceph-mon[31733]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:03:21.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:21 smithi086 ceph-mon[31733]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c;count:3 2024-04-17T07:03:21.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:21 smithi086 ceph-mon[31733]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:03:21.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:21 smithi086 ceph-mon[31733]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:22.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:22 smithi086 ceph-mon[31733]: Updating smithi103:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T07:03:22.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:22.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:22.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:22.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:22.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.365+0000 7f434f600700 1 -- 172.21.15.103:0/3397451840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43481084f0 msgr2=0x7f43481088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.365+0000 7f434f600700 1 --2- 172.21.15.103:0/3397451840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43481084f0 0x7f43481088d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f433c009910 tx=0x7f433c009c20 comp rx=0 tx=0).stop 2024-04-17T07:03:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.365+0000 7f434f600700 1 -- 172.21.15.103:0/3397451840 shutdown_connections 2024-04-17T07:03:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.365+0000 7f434f600700 1 --2- 172.21.15.103:0/3397451840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43481084f0 0x7f43481088d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.365+0000 7f434f600700 1 -- 172.21.15.103:0/3397451840 >> 172.21.15.103:0/3397451840 conn(0x7f43480fdf90 msgr2=0x7f43481003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.365+0000 7f434f600700 1 -- 172.21.15.103:0/3397451840 shutdown_connections 2024-04-17T07:03:23.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.365+0000 7f434f600700 1 -- 172.21.15.103:0/3397451840 wait complete. 2024-04-17T07:03:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.366+0000 7f434f600700 1 Processor -- start 2024-04-17T07:03:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.367+0000 7f434f600700 1 -- start start 2024-04-17T07:03:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.367+0000 7f434f600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43481084f0 0x7f434819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:23.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.367+0000 7f434f600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f434810c950 con 0x7f43481084f0 2024-04-17T07:03:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.368+0000 7f434cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43481084f0 0x7f434819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.368+0000 7f434cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43481084f0 0x7f434819a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.103:53376/0 (socket says 172.21.15.103:53376) 2024-04-17T07:03:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.368+0000 7f434cc00700 1 -- 172.21.15.103:0/99176379 learned_addr learned my addr 172.21.15.103:0/99176379 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:03:23.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.368+0000 7f434cc00700 1 -- 172.21.15.103:0/99176379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434819a610 con 0x7f43481084f0 2024-04-17T07:03:23.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.369+0000 7f434cc00700 1 --2- 172.21.15.103:0/99176379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43481084f0 0x7f434819a0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f433c0056d0 tx=0x7f433c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:23.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.369+0000 7f4345600700 1 -- 172.21.15.103:0/99176379 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f433c026070 con 0x7f43481084f0 2024-04-17T07:03:23.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.369+0000 7f434f600700 1 -- 172.21.15.103:0/99176379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434819a900 con 0x7f43481084f0 2024-04-17T07:03:23.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.369+0000 7f434f600700 1 -- 172.21.15.103:0/99176379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434819e9a0 con 0x7f43481084f0 2024-04-17T07:03:23.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.369+0000 7f4345600700 1 -- 172.21.15.103:0/99176379 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f433c00bba0 con 0x7f43481084f0 2024-04-17T07:03:23.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.370+0000 7f4345600700 1 -- 172.21.15.103:0/99176379 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f433c0037f0 con 0x7f43481084f0 2024-04-17T07:03:23.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.370+0000 7f434f600700 1 -- 172.21.15.103:0/99176379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4328005330 con 0x7f43481084f0 2024-04-17T07:03:23.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.371+0000 7f4345600700 1 -- 172.21.15.103:0/99176379 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f433c029070 con 0x7f43481084f0 2024-04-17T07:03:23.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.372+0000 7f4345600700 1 --2- 172.21.15.103:0/99176379 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43300396b0 0x7f433003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:23.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.372+0000 7f4345600700 1 -- 172.21.15.103:0/99176379 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f433c01e070 con 0x7f43481084f0 2024-04-17T07:03:23.371 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.372+0000 7f4347e00700 1 --2- 172.21.15.103:0/99176379 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43300396b0 0x7f433003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:23.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.373+0000 7f4347e00700 1 --2- 172.21.15.103:0/99176379 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43300396b0 0x7f433003bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4334006fd0 tx=0x7f4334006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:23.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.388+0000 7f4345600700 1 -- 172.21.15.103:0/99176379 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f433c013070 con 0x7f43481084f0 2024-04-17T07:03:23.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.713+0000 7f434f600700 1 -- 172.21.15.103:0/99176379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4328005620 con 0x7f43481084f0 2024-04-17T07:03:23.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.714+0000 7f4345600700 1 -- 172.21.15.103:0/99176379 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f433c029360 con 0x7f43481084f0 2024-04-17T07:03:23.713 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T07:03:23.713 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":1,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","modified":"2024-04-17T07:01:16.008725Z","created":"2024-04-17T07:01:16.008725Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-17T07:03:23.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.717+0000 7f434f600700 1 -- 172.21.15.103:0/99176379 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43300396b0 msgr2=0x7f433003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:23.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.717+0000 7f434f600700 1 --2- 172.21.15.103:0/99176379 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43300396b0 0x7f433003bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4334006fd0 tx=0x7f4334006e40 comp rx=0 tx=0).stop 2024-04-17T07:03:23.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.717+0000 7f434f600700 1 -- 172.21.15.103:0/99176379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43481084f0 msgr2=0x7f434819a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:23.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.717+0000 7f434f600700 1 --2- 172.21.15.103:0/99176379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43481084f0 0x7f434819a0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f433c0056d0 tx=0x7f433c0057b0 comp rx=0 tx=0).stop 2024-04-17T07:03:23.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.717+0000 7f434f600700 1 -- 172.21.15.103:0/99176379 shutdown_connections 2024-04-17T07:03:23.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.717+0000 7f434f600700 1 --2- 172.21.15.103:0/99176379 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43300396b0 0x7f433003bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:23.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.717+0000 7f434f600700 1 --2- 172.21.15.103:0/99176379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43481084f0 0x7f434819a0d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:23.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.717+0000 7f434f600700 1 -- 172.21.15.103:0/99176379 >> 172.21.15.103:0/99176379 conn(0x7f43480fdf90 msgr2=0x7f4348109fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:23.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.717+0000 7f434f600700 1 -- 172.21.15.103:0/99176379 shutdown_connections 2024-04-17T07:03:23.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:23.717+0000 7f434f600700 1 -- 172.21.15.103:0/99176379 wait complete. 2024-04-17T07:03:23.717 INFO:teuthology.orchestra.run.smithi103.stderr:dumped monmap epoch 1 2024-04-17T07:03:23.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:23 smithi086 ceph-mon[31733]: Deploying daemon mon.c on smithi103 2024-04-17T07:03:23.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:23 smithi086 ceph-mon[31733]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:24.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:24 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/99176379' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:03:25.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:25 smithi086 ceph-mon[31733]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:27.456 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-17T07:03:27.456 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph mon dump -f json 2024-04-17T07:03:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:27 smithi086 ceph-mon[31733]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.455 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 podman[35452]: 2024-04-17 07:03:28.097100396 +0000 UTC m=+0.043462821 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:03:28.622 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:03:28.804 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 podman[35452]: 2024-04-17 07:03:28.538186039 +0000 UTC m=+0.484548469 container init 819a7f1e426ca449ea42647d3e7dbaad8bc4a0ebe724a949ceb31f6398b88bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-17T07:03:28.804 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 podman[35452]: 2024-04-17 07:03:28.546026103 +0000 UTC m=+0.492388527 container start 819a7f1e426ca449ea42647d3e7dbaad8bc4a0ebe724a949ceb31f6398b88bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-17T07:03:28.804 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: set uid:gid to 167:167 (ceph:ceph) 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pidfile_write: ignore empty --pid-file 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: load: jerasure load: lrc 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: RocksDB version: 7.9.2 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Git sha 0 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: DB SUMMARY 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: DB Session ID: HLVGFDU5E5O8LHOUIMZG 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: CURRENT file: CURRENT 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: IDENTITY file: IDENTITY 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-17T07:03:28.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.error_if_exists: 0 2024-04-17T07:03:28.806 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.create_if_missing: 0 2024-04-17T07:03:28.806 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.paranoid_checks: 1 2024-04-17T07:03:28.806 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-17T07:03:28.806 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-17T07:03:28.806 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-17T07:03:28.806 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.env: 0x55ea26b54cc0 2024-04-17T07:03:28.806 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.fs: PosixFileSystem 2024-04-17T07:03:28.806 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.info_log: 0x55ea312e7c60 2024-04-17T07:03:28.806 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_file_opening_threads: 16 2024-04-17T07:03:28.807 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.statistics: (nil) 2024-04-17T07:03:28.807 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.use_fsync: 0 2024-04-17T07:03:28.807 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_log_file_size: 0 2024-04-17T07:03:28.807 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-17T07:03:28.807 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-17T07:03:28.807 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.keep_log_file_num: 1000 2024-04-17T07:03:28.807 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.recycle_log_file_num: 0 2024-04-17T07:03:28.807 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.allow_fallocate: 1 2024-04-17T07:03:28.807 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.allow_mmap_reads: 0 2024-04-17T07:03:28.807 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.allow_mmap_writes: 0 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.use_direct_reads: 0 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.create_missing_column_families: 0 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.db_log_dir: 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.wal_dir: 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.advise_random_on_open: 1 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.db_write_buffer_size: 0 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.write_buffer_manager: 0x55ea312b61e0 2024-04-17T07:03:28.808 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.rate_limiter: (nil) 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.wal_recovery_mode: 2 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.enable_thread_tracking: 0 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.enable_pipelined_write: 0 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.unordered_write: 0 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.row_cache: None 2024-04-17T07:03:28.809 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.wal_filter: None 2024-04-17T07:03:28.811 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-17T07:03:28.811 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.allow_ingest_behind: 0 2024-04-17T07:03:28.811 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.two_write_queues: 0 2024-04-17T07:03:28.811 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.manual_wal_flush: 0 2024-04-17T07:03:28.811 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.wal_compression: 0 2024-04-17T07:03:28.811 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.atomic_flush: 0 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.log_readahead_size: 0 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.best_efforts_recovery: 0 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.allow_data_in_errors: 0 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.db_host_id: __hostname__ 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_background_jobs: 2 2024-04-17T07:03:28.812 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_background_compactions: -1 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_subcompactions: 1 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_total_wal_size: 0 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_open_files: -1 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bytes_per_sync: 0 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_readahead_size: 0 2024-04-17T07:03:28.813 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_background_flushes: -1 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Compression algorithms supported: 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: kZSTD supported: 0 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: kXpressCompression supported: 0 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: kLZ4HCCompression supported: 1 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: kZlibCompression supported: 1 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: kSnappyCompression supported: 1 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: kLZ4Compression supported: 1 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: kBZip2Compression supported: 0 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-17T07:03:28.814 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.merge_operator: 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_filter: None 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_filter_factory: None 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.sst_partitioner_factory: None 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ea312e7e20) 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout: cache_index_and_filter_blocks: 1 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout: pin_top_level_index_and_filter: 1 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout: index_type: 0 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout: data_block_index_type: 0 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout: index_shortening: 1 2024-04-17T07:03:28.815 INFO:journalctl@ceph.mon.c.smithi103.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: checksum: 4 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: no_block_cache: 0 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: block_cache: 0x55ea312e9090 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: block_cache_name: BinnedLRUCache 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: block_cache_options: 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: capacity : 536870912 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: num_shard_bits : 4 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: strict_capacity_limit : 0 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: high_pri_pool_ratio: 0.000 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: block_cache_compressed: (nil) 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: persistent_cache: (nil) 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: block_size: 4096 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: block_size_deviation: 10 2024-04-17T07:03:28.816 INFO:journalctl@ceph.mon.c.smithi103.stdout: block_restart_interval: 16 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: index_block_restart_interval: 1 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: metadata_block_size: 4096 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: partition_filters: 0 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: use_delta_encoding: 1 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: filter_policy: bloomfilter 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: whole_key_filtering: 1 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: verify_compression: 0 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: read_amp_bytes_per_bit: 0 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: format_version: 5 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: enable_index_compression: 1 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: block_align: 0 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: max_auto_readahead_size: 262144 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: prepopulate_block_cache: 0 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: initial_auto_readahead_size: 8192 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout: num_file_reads_for_auto_readahead: 2 2024-04-17T07:03:28.817 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.write_buffer_size: 33554432 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_write_buffer_number: 2 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compression: NoCompression 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bottommost_compression: Disabled 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.prefix_extractor: nullptr 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.num_levels: 7 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-17T07:03:28.818 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compression_opts.level: 32767 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compression_opts.strategy: 0 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compression_opts.enabled: false 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-17T07:03:28.819 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.target_file_size_base: 67108864 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-17T07:03:28.820 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.arena_block_size: 1048576 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.disable_auto_compactions: 0 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-17T07:03:28.821 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.inplace_update_support: 0 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.bloom_locality: 0 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.max_successive_merges: 0 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.paranoid_file_checks: 0 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.force_consistency_checks: 1 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.report_bg_io_stats: 0 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.ttl: 2592000 2024-04-17T07:03:28.822 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.enable_blob_files: false 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.min_blob_size: 0 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.blob_file_size: 268435456 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.blob_file_starting_level: 0 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b6c48e7e-7822-4185-be33-cbd537a9f2fb 2024-04-17T07:03:28.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713337408595672, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713337408596501, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713337408, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b6c48e7e-7822-4185-be33-cbd537a9f2fb", "db_session_id": "HLVGFDU5E5O8LHOUIMZG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713337408596755, "job": 1, "event": "recovery_finished"} 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ea313d6000 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: DB pointer 0x55ea313c0000 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout: ** DB Stats ** 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-17T07:03:28.824 INFO:journalctl@ceph.mon.c.smithi103.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: ** Compaction Stats [default] ** 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: ** Compaction Stats [default] ** 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-17T07:03:28.825 INFO:journalctl@ceph.mon.c.smithi103.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: Block cache BinnedLRUCache@0x55ea312e9090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-17T07:03:28.826 INFO:journalctl@ceph.mon.c.smithi103.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout: 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: using public_addr v2:172.21.15.103:0/0 -> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: starting mon.c rank -1 at public addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] at bind addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(???) e0 preinit fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 bash[35452]: 819a7f1e426ca449ea42647d3e7dbaad8bc4a0ebe724a949ceb31f6398b88bb7 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 systemd[1]: Started Ceph mon.c for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).mds e1 new map 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout: e1 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout: legacy client fscid: -1 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout: 2024-04-17T07:03:28.827 INFO:journalctl@ceph.mon.c.smithi103.stdout: No filesystems configured 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Deploying cephadm binary to smithi098 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Added host smithi098 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.828 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Deploying cephadm binary to smithi103 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.829 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Updating smithi098:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Added host smithi103 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.830 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Updating smithi098:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/900910452' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/900910452' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.831 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Updating smithi103:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c;count:3 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Updating smithi103:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.832 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.833 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:28.833 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:28.833 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:28.833 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: Deploying daemon mon.c on smithi103 2024-04-17T07:03:28.833 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.833 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/99176379' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:03:28.833 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.833 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:28.833 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:28 smithi103 ceph-mon[35495]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-17T07:03:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.869+0000 7fcfc0a00700 1 -- 172.21.15.103:0/558024724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfbc103550 msgr2=0x7fcfa0005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.869+0000 7fcfc0a00700 1 --2- 172.21.15.103:0/558024724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfbc103550 0x7fcfa0005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcfac005fa0 tx=0x7fcfac00f4a0 comp rx=0 tx=0).stop 2024-04-17T07:03:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.870+0000 7fcfc0a00700 1 -- 172.21.15.103:0/558024724 shutdown_connections 2024-04-17T07:03:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.870+0000 7fcfc0a00700 1 --2- 172.21.15.103:0/558024724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfbc103550 0x7fcfa0005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.870+0000 7fcfc0a00700 1 -- 172.21.15.103:0/558024724 >> 172.21.15.103:0/558024724 conn(0x7fcfbc0fcdc0 msgr2=0x7fcfbc0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.870+0000 7fcfc0a00700 1 -- 172.21.15.103:0/558024724 shutdown_connections 2024-04-17T07:03:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.870+0000 7fcfc0a00700 1 -- 172.21.15.103:0/558024724 wait complete. 2024-04-17T07:03:33.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.874+0000 7fcfc0a00700 1 Processor -- start 2024-04-17T07:03:33.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.874+0000 7fcfc0a00700 1 -- start start 2024-04-17T07:03:33.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.875+0000 7fcfc0a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfbc103550 0x7fcfbc197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:33.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.875+0000 7fcfc0a00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfbc198320 0x7fcfbc19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:33.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.875+0000 7fcfc0a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc10a2e0 con 0x7fcfbc103550 2024-04-17T07:03:33.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.875+0000 7fcfc0a00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc10a160 con 0x7fcfbc198320 2024-04-17T07:03:33.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.876+0000 7fcfb9e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfbc198320 0x7fcfbc19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:33.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.876+0000 7fcfb9e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfbc198320 0x7fcfbc19c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33006/0 (socket says 172.21.15.103:33006) 2024-04-17T07:03:33.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.876+0000 7fcfb9e00700 1 -- 172.21.15.103:0/3324187226 learned_addr learned my addr 172.21.15.103:0/3324187226 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:03:33.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.876+0000 7fcfba800700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfbc103550 0x7fcfbc197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:33.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.877+0000 7fcfb9e00700 1 -- 172.21.15.103:0/3324187226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfbc198320 msgr2=0x7fcfbc19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-17T07:03:33.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.877+0000 7fcfb9e00700 1 -- 172.21.15.103:0/3324187226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfbc198320 msgr2=0x7fcfbc19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-17T07:03:33.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.877+0000 7fcfb9e00700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfbc198320 0x7fcfbc19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T07:03:33.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.877+0000 7fcfb9e00700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfbc198320 0x7fcfbc19c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:03:33.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.877+0000 7fcfba800700 1 -- 172.21.15.103:0/3324187226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfbc198320 msgr2=0x7fcfbc19c6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:03:33.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.877+0000 7fcfba800700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfbc198320 0x7fcfbc19c6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:33.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.877+0000 7fcfba800700 1 -- 172.21.15.103:0/3324187226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc198960 con 0x7fcfbc103550 2024-04-17T07:03:33.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.878+0000 7fcfba800700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfbc103550 0x7fcfbc197de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcfac005fa0 tx=0x7fcfac005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:33.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.878+0000 7fcfb3400700 1 -- 172.21.15.103:0/3324187226 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcfac004bf0 con 0x7fcfbc103550 2024-04-17T07:03:33.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.878+0000 7fcfb3400700 1 -- 172.21.15.103:0/3324187226 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfac02c070 con 0x7fcfbc103550 2024-04-17T07:03:33.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.878+0000 7fcfc0a00700 1 -- 172.21.15.103:0/3324187226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc19cbf0 con 0x7fcfbc103550 2024-04-17T07:03:33.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.878+0000 7fcfb3400700 1 -- 172.21.15.103:0/3324187226 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcfac01c410 con 0x7fcfbc103550 2024-04-17T07:03:33.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.879+0000 7fcfc0a00700 1 -- 172.21.15.103:0/3324187226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc19d150 con 0x7fcfbc103550 2024-04-17T07:03:33.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.880+0000 7fcfb3400700 1 -- 172.21.15.103:0/3324187226 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcfac004d90 con 0x7fcfbc103550 2024-04-17T07:03:33.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.880+0000 7fcfc0a00700 1 -- 172.21.15.103:0/3324187226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfbc06acb0 con 0x7fcfbc103550 2024-04-17T07:03:33.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.881+0000 7fcfb3400700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf9c039460 0x7fcf9c03b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:33.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.881+0000 7fcfb3400700 1 -- 172.21.15.103:0/3324187226 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcfac019070 con 0x7fcfbc103550 2024-04-17T07:03:33.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.881+0000 7fcfb9e00700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf9c039460 0x7fcf9c03b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:33.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.883+0000 7fcfb9e00700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf9c039460 0x7fcf9c03b920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009180 tx=0x7fcfa8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:33.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:33.896+0000 7fcfb3400700 1 -- 172.21.15.103:0/3324187226 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcfac010a40 con 0x7fcfbc103550 2024-04-17T07:03:34.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:03:34.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: mon.a calling monitor election 2024-04-17T07:03:34.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:34.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: mon.c calling monitor election 2024-04-17T07:03:34.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:34.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: monmap epoch 2 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: last_changed 2024-04-17T07:03:28.656511+0000 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: min_mon_release 19 (squid) 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: election_strategy: 1 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: fsmap 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:34.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: mgrmap e13: x(active, since 66s) 2024-04-17T07:03:34.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: overall HEALTH_OK 2024-04-17T07:03:34.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:34.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:34.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:34.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:34.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: mon.a calling monitor election 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: mon.c calling monitor election 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:34.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T07:03:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: monmap epoch 2 2024-04-17T07:03:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: last_changed 2024-04-17T07:03:28.656511+0000 2024-04-17T07:03:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:03:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: min_mon_release 19 (squid) 2024-04-17T07:03:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: election_strategy: 1 2024-04-17T07:03:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:03:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T07:03:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: fsmap 2024-04-17T07:03:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:34.083 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: mgrmap e13: x(active, since 66s) 2024-04-17T07:03:34.083 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: overall HEALTH_OK 2024-04-17T07:03:34.083 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:34.083 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:34.083 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:34.083 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:34.083 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:34.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.197+0000 7fcfc0a00700 1 -- 172.21.15.103:0/3324187226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcfbc19d400 con 0x7fcfbc103550 2024-04-17T07:03:34.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.199+0000 7fcfb3400700 1 -- 172.21.15.103:0/3324187226 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fcfac013940 con 0x7fcfbc103550 2024-04-17T07:03:34.198 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T07:03:34.199 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":2,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","modified":"2024-04-17T07:03:28.656511Z","created":"2024-04-17T07:01:16.008725Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-17T07:03:34.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 -- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf9c039460 msgr2=0x7fcf9c03b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:34.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf9c039460 0x7fcf9c03b920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009180 tx=0x7fcfa8008040 comp rx=0 tx=0).stop 2024-04-17T07:03:34.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 -- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfbc103550 msgr2=0x7fcfbc197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:34.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfbc103550 0x7fcfbc197de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcfac005fa0 tx=0x7fcfac005740 comp rx=0 tx=0).stop 2024-04-17T07:03:34.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 -- 172.21.15.103:0/3324187226 shutdown_connections 2024-04-17T07:03:34.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf9c039460 0x7fcf9c03b920 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfbc103550 0x7fcfbc197de0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 --2- 172.21.15.103:0/3324187226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfbc198320 0x7fcfbc19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 -- 172.21.15.103:0/3324187226 >> 172.21.15.103:0/3324187226 conn(0x7fcfbc0fcdc0 msgr2=0x7fcfbc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 -- 172.21.15.103:0/3324187226 shutdown_connections 2024-04-17T07:03:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:34.202+0000 7fcfc0a00700 1 -- 172.21.15.103:0/3324187226 wait complete. 2024-04-17T07:03:34.202 INFO:teuthology.orchestra.run.smithi103.stderr:dumped monmap epoch 2 2024-04-17T07:03:35.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:34 smithi103 ceph-mon[35495]: Deploying daemon mon.b on smithi098 2024-04-17T07:03:35.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:34 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/3324187226' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:03:35.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:34 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:35.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:34 smithi086 ceph-mon[31733]: Deploying daemon mon.b on smithi098 2024-04-17T07:03:35.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:34 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/3324187226' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:03:35.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:34 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:35.827 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-17T07:03:35.828 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph mon dump -f json 2024-04-17T07:03:36.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:35 smithi103 ceph-mon[35495]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:36.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:35 smithi086 ceph-mon[31733]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:36.268 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:03:38.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.164+0000 7f0bd2e00700 1 -- 172.21.15.103:0/1740191410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bcc1037c0 msgr2=0x7f0bcc103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:38.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.164+0000 7f0bd2e00700 1 --2- 172.21.15.103:0/1740191410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bcc1037c0 0x7f0bcc103ba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009910 tx=0x7f0bbc009c20 comp rx=0 tx=0).stop 2024-04-17T07:03:38.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.164+0000 7f0bd2e00700 1 -- 172.21.15.103:0/1740191410 shutdown_connections 2024-04-17T07:03:38.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.164+0000 7f0bd2e00700 1 --2- 172.21.15.103:0/1740191410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bcc1037c0 0x7f0bcc103ba0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:38.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.164+0000 7f0bd2e00700 1 -- 172.21.15.103:0/1740191410 >> 172.21.15.103:0/1740191410 conn(0x7f0bcc0fcf30 msgr2=0x7f0bcc0ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:38.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.164+0000 7f0bd2e00700 1 -- 172.21.15.103:0/1740191410 shutdown_connections 2024-04-17T07:03:38.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.164+0000 7f0bd2e00700 1 -- 172.21.15.103:0/1740191410 wait complete. 2024-04-17T07:03:38.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.166+0000 7f0bd2e00700 1 Processor -- start 2024-04-17T07:03:38.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.166+0000 7f0bd2e00700 1 -- start start 2024-04-17T07:03:38.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.166+0000 7f0bd2e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bcc1037c0 0x7f0bcc198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:38.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.166+0000 7f0bd2e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bcc1986c0 0x7f0bcc19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:38.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.166+0000 7f0bd2e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc10ca90 con 0x7f0bcc1037c0 2024-04-17T07:03:38.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.166+0000 7f0bd2e00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc10c910 con 0x7f0bcc1986c0 2024-04-17T07:03:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.167+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bcc1037c0 0x7f0bcc198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.167+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bcc1037c0 0x7f0bcc198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.103:52018/0 (socket says 172.21.15.103:52018) 2024-04-17T07:03:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.167+0000 7f0bcbe00700 1 -- 172.21.15.103:0/3698870131 learned_addr learned my addr 172.21.15.103:0/3698870131 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:03:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.167+0000 7f0bc3400700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bcc1986c0 0x7f0bcc19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.167+0000 7f0bc3400700 1 -- 172.21.15.103:0/3698870131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bcc1986c0 msgr2=0x7f0bcc19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-17T07:03:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.167+0000 7f0bc3400700 1 -- 172.21.15.103:0/3698870131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bcc1986c0 msgr2=0x7f0bcc19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-17T07:03:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.167+0000 7f0bc3400700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bcc1986c0 0x7f0bcc19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T07:03:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.167+0000 7f0bc3400700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bcc1986c0 0x7f0bcc19ca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:03:38.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.168+0000 7f0bcbe00700 1 -- 172.21.15.103:0/3698870131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bcc1986c0 msgr2=0x7f0bcc19ca50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:03:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.168+0000 7f0bcbe00700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bcc1986c0 0x7f0bcc19ca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.168+0000 7f0bcbe00700 1 -- 172.21.15.103:0/3698870131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc198d00 con 0x7f0bcc1037c0 2024-04-17T07:03:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.168+0000 7f0bcbe00700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bcc1037c0 0x7f0bcc198180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006010 tx=0x7f0bbc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.168+0000 7f0bc9600700 1 -- 172.21.15.103:0/3698870131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0bbc005dc0 con 0x7f0bcc1037c0 2024-04-17T07:03:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.169+0000 7f0bd2e00700 1 -- 172.21.15.103:0/3698870131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc19cf90 con 0x7f0bcc1037c0 2024-04-17T07:03:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.169+0000 7f0bc9600700 1 -- 172.21.15.103:0/3698870131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bbc00be10 con 0x7f0bcc1037c0 2024-04-17T07:03:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.169+0000 7f0bc9600700 1 -- 172.21.15.103:0/3698870131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0bbc00f460 con 0x7f0bcc1037c0 2024-04-17T07:03:38.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.169+0000 7f0bd2e00700 1 -- 172.21.15.103:0/3698870131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc19d490 con 0x7f0bcc1037c0 2024-04-17T07:03:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.170+0000 7f0bd2e00700 1 -- 172.21.15.103:0/3698870131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bcc06abf0 con 0x7f0bcc1037c0 2024-04-17T07:03:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.170+0000 7f0bc9600700 1 -- 172.21.15.103:0/3698870131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0bbc02a070 con 0x7f0bcc1037c0 2024-04-17T07:03:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.171+0000 7f0bc9600700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0bb00393f0 0x7f0bb003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.171+0000 7f0bc9600700 1 -- 172.21.15.103:0/3698870131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0bbc01e030 con 0x7f0bcc1037c0 2024-04-17T07:03:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.171+0000 7f0bc3400700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0bb00393f0 0x7f0bb003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.173+0000 7f0bc3400700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0bb00393f0 0x7f0bb003b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009850 tx=0x7f0bb4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:38.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.187+0000 7f0bc9600700 1 -- 172.21.15.103:0/3698870131 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0bbc013070 con 0x7f0bcc1037c0 2024-04-17T07:03:38.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:37 smithi103 ceph-mon[35495]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:38.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:37 smithi086 ceph-mon[31733]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:38.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.500+0000 7f0bd2e00700 1 -- 172.21.15.103:0/3698870131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0bcc19d740 con 0x7f0bcc1037c0 2024-04-17T07:03:38.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.502+0000 7f0bc9600700 1 -- 172.21.15.103:0/3698870131 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f0bbc031030 con 0x7f0bcc1037c0 2024-04-17T07:03:38.502 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T07:03:38.502 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":2,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","modified":"2024-04-17T07:03:28.656511Z","created":"2024-04-17T07:01:16.008725Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-17T07:03:38.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 -- 172.21.15.103:0/3698870131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0bb00393f0 msgr2=0x7f0bb003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:38.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0bb00393f0 0x7f0bb003b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009850 tx=0x7f0bb4008040 comp rx=0 tx=0).stop 2024-04-17T07:03:38.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 -- 172.21.15.103:0/3698870131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bcc1037c0 msgr2=0x7f0bcc198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:38.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bcc1037c0 0x7f0bcc198180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006010 tx=0x7f0bbc005030 comp rx=0 tx=0).stop 2024-04-17T07:03:38.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 -- 172.21.15.103:0/3698870131 shutdown_connections 2024-04-17T07:03:38.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0bb00393f0 0x7f0bb003b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:38.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bcc1037c0 0x7f0bcc198180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:38.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 --2- 172.21.15.103:0/3698870131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bcc1986c0 0x7f0bcc19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:38.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 -- 172.21.15.103:0/3698870131 >> 172.21.15.103:0/3698870131 conn(0x7f0bcc0fcf30 msgr2=0x7f0bcc106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:38.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 -- 172.21.15.103:0/3698870131 shutdown_connections 2024-04-17T07:03:38.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:38.505+0000 7f0bd2e00700 1 -- 172.21.15.103:0/3698870131 wait complete. 2024-04-17T07:03:38.506 INFO:teuthology.orchestra.run.smithi103.stderr:dumped monmap epoch 2 2024-04-17T07:03:39.102 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:38 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/3698870131' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:03:39.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:38 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/3698870131' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:03:39.351 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 podman[35400]: 2024-04-17 07:03:39.018909834 +0000 UTC m=+0.038042608 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:03:39.352 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 podman[35400]: 2024-04-17 07:03:39.350531704 +0000 UTC m=+0.369664505 container init 35a692606e822c829b6d2c25081ab972f8c15656794bc9cd38a42d34a238d37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 podman[35400]: 2024-04-17 07:03:39.360772593 +0000 UTC m=+0.379905394 container start 35a692606e822c829b6d2c25081ab972f8c15656794bc9cd38a42d34a238d37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: set uid:gid to 167:167 (ceph:ceph) 2024-04-17T07:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-17T07:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pidfile_write: ignore empty --pid-file 2024-04-17T07:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: load: jerasure load: lrc 2024-04-17T07:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: RocksDB version: 7.9.2 2024-04-17T07:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Git sha 0 2024-04-17T07:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-17T07:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: DB SUMMARY 2024-04-17T07:03:39.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: DB Session ID: 9RO7NSPXHJMXKU74HET7 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: CURRENT file: CURRENT 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: IDENTITY file: IDENTITY 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.error_if_exists: 0 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.create_if_missing: 0 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.paranoid_checks: 1 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-17T07:03:39.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.env: 0x558b63b89cc0 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.fs: PosixFileSystem 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.info_log: 0x558ba0011c60 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_file_opening_threads: 16 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.statistics: (nil) 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.use_fsync: 0 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_log_file_size: 0 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.keep_log_file_num: 1000 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.recycle_log_file_num: 0 2024-04-17T07:03:39.756 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.allow_fallocate: 1 2024-04-17T07:03:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.allow_mmap_reads: 0 2024-04-17T07:03:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.allow_mmap_writes: 0 2024-04-17T07:03:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.use_direct_reads: 0 2024-04-17T07:03:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-17T07:03:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.create_missing_column_families: 0 2024-04-17T07:03:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.db_log_dir: 2024-04-17T07:03:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.wal_dir: 2024-04-17T07:03:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.advise_random_on_open: 1 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.db_write_buffer_size: 0 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.write_buffer_manager: 0x558b9ffe01e0 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.rate_limiter: (nil) 2024-04-17T07:03:39.758 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.wal_recovery_mode: 2 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.enable_thread_tracking: 0 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.enable_pipelined_write: 0 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.unordered_write: 0 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.row_cache: None 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.wal_filter: None 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.allow_ingest_behind: 0 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.two_write_queues: 0 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.manual_wal_flush: 0 2024-04-17T07:03:39.759 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.wal_compression: 0 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.atomic_flush: 0 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.log_readahead_size: 0 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.best_efforts_recovery: 0 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.allow_data_in_errors: 0 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.db_host_id: __hostname__ 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_background_jobs: 2 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_background_compactions: -1 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_subcompactions: 1 2024-04-17T07:03:39.760 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_total_wal_size: 0 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_open_files: -1 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bytes_per_sync: 0 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_readahead_size: 0 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_background_flushes: -1 2024-04-17T07:03:39.761 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Compression algorithms supported: 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: kZSTD supported: 0 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: kXpressCompression supported: 0 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: kLZ4HCCompression supported: 1 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: kZlibCompression supported: 1 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: kSnappyCompression supported: 1 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: kLZ4Compression supported: 1 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: kBZip2Compression supported: 0 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-17T07:03:39.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.merge_operator: 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_filter: None 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_filter_factory: None 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.sst_partitioner_factory: None 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558ba0011e20) 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout: cache_index_and_filter_blocks: 1 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout: pin_top_level_index_and_filter: 1 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_type: 0 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout: data_block_index_type: 0 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_shortening: 1 2024-04-17T07:03:39.763 INFO:journalctl@ceph.mon.b.smithi098.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: checksum: 4 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: no_block_cache: 0 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache: 0x558ba0013090 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_name: BinnedLRUCache 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_options: 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: capacity : 536870912 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: num_shard_bits : 4 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: strict_capacity_limit : 0 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: high_pri_pool_ratio: 0.000 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_compressed: (nil) 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: persistent_cache: (nil) 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_size: 4096 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_size_deviation: 10 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_restart_interval: 16 2024-04-17T07:03:39.764 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_block_restart_interval: 1 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: metadata_block_size: 4096 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: partition_filters: 0 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: use_delta_encoding: 1 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: filter_policy: bloomfilter 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: whole_key_filtering: 1 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: verify_compression: 0 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: read_amp_bytes_per_bit: 0 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: format_version: 5 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: enable_index_compression: 1 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_align: 0 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: max_auto_readahead_size: 262144 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: prepopulate_block_cache: 0 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: initial_auto_readahead_size: 8192 2024-04-17T07:03:39.765 INFO:journalctl@ceph.mon.b.smithi098.stdout: num_file_reads_for_auto_readahead: 2 2024-04-17T07:03:39.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.write_buffer_size: 33554432 2024-04-17T07:03:39.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_write_buffer_number: 2 2024-04-17T07:03:39.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compression: NoCompression 2024-04-17T07:03:39.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bottommost_compression: Disabled 2024-04-17T07:03:39.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.prefix_extractor: nullptr 2024-04-17T07:03:39.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-17T07:03:39.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.num_levels: 7 2024-04-17T07:03:39.766 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-17T07:03:39.767 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-17T07:03:39.767 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-17T07:03:39.767 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compression_opts.level: 32767 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compression_opts.strategy: 0 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-17T07:03:39.768 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compression_opts.enabled: false 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.target_file_size_base: 67108864 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-17T07:03:39.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.arena_block_size: 1048576 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.disable_auto_compactions: 0 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-17T07:03:39.770 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.inplace_update_support: 0 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-17T07:03:39.771 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.bloom_locality: 0 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.max_successive_merges: 0 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.paranoid_file_checks: 0 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.force_consistency_checks: 1 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.report_bg_io_stats: 0 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.ttl: 2592000 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.enable_blob_files: false 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.min_blob_size: 0 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.blob_file_size: 268435456 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-17T07:03:39.772 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.blob_file_starting_level: 0 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: dffdfed8-3af9-492f-bfc7-54bd0a740b7e 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713337419420280, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713337419421615, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713337419, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "dffdfed8-3af9-492f-bfc7-54bd0a740b7e", "db_session_id": "9RO7NSPXHJMXKU74HET7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713337419421983, "job": 1, "event": "recovery_finished"} 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-17T07:03:39.773 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558ba0100000 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: DB pointer 0x558ba00ea000 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** DB Stats ** 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-17T07:03:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** Compaction Stats [default] ** 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** Compaction Stats [default] ** 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-17T07:03:39.775 INFO:journalctl@ceph.mon.b.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative compaction: 0.00 GB write, 0.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval compaction: 0.00 GB write, 0.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: Block cache BinnedLRUCache@0x558ba0013090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.5e-05 secs_since: 0 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: using public_addr v2:172.21.15.98:0/0 -> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2024-04-17T07:03:39.776 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: starting mon.b rank -1 at public addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] at bind addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(???) e0 preinit fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 bash[35400]: 35a692606e822c829b6d2c25081ab972f8c15656794bc9cd38a42d34a238d37a 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 systemd[1]: Started Ceph mon.b for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).mds e1 new map 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: e1 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: legacy client fscid: -1 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout: No filesystems configured 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-17T07:03:39.777 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Deploying cephadm binary to smithi098 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.778 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Added host smithi098 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Deploying cephadm binary to smithi103 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.779 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Updating smithi098:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:03:39.780 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Added host smithi103 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Updating smithi098:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.781 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/900910452' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/900910452' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T07:03:39.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Updating smithi103:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c;count:3 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Updating smithi103:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.client.admin.keyring 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:39.783 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Deploying daemon mon.c on smithi103 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/99176379' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.a calling monitor election 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.c calling monitor election 2024-04-17T07:03:39.784 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: monmap epoch 2 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: last_changed 2024-04-17T07:03:28.656511+0000 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: min_mon_release 19 (squid) 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: election_strategy: 1 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T07:03:39.785 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: fsmap 2024-04-17T07:03:39.786 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:39.786 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mgrmap e13: x(active, since 66s) 2024-04-17T07:03:39.786 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: overall HEALTH_OK 2024-04-17T07:03:39.786 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.786 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.786 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:39.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:39.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:39.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: Deploying daemon mon.b on smithi098 2024-04-17T07:03:39.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/3324187226' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:03:39.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:39.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:39.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/3698870131' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-17T07:03:39.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:39 smithi098 ceph-mon[35418]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-04-17T07:03:40.157 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-17T07:03:40.157 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph mon dump -f json 2024-04-17T07:03:40.582 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:03:44.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:03:44.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: mon.a calling monitor election 2024-04-17T07:03:44.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: mon.c calling monitor election 2024-04-17T07:03:44.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:44.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:44.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:44.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:44.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:44.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:44.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:44.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:44.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:44.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T07:03:44.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: monmap epoch 3 2024-04-17T07:03:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: last_changed 2024-04-17T07:03:39.485487+0000 2024-04-17T07:03:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:03:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: min_mon_release 19 (squid) 2024-04-17T07:03:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: election_strategy: 1 2024-04-17T07:03:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:03:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T07:03:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-17T07:03:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: fsmap 2024-04-17T07:03:44.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:44.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: mgrmap e13: x(active, since 77s) 2024-04-17T07:03:44.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: overall HEALTH_OK 2024-04-17T07:03:44.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:44.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:45.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:03:45.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: mon.a calling monitor election 2024-04-17T07:03:45.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: mon.c calling monitor election 2024-04-17T07:03:45.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:45.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:45.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:45.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:45.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:45.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:45.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:45.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:45.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:45.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T07:03:45.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: monmap epoch 3 2024-04-17T07:03:45.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:45.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: last_changed 2024-04-17T07:03:39.485487+0000 2024-04-17T07:03:45.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:03:45.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: min_mon_release 19 (squid) 2024-04-17T07:03:45.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: election_strategy: 1 2024-04-17T07:03:45.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:03:45.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T07:03:45.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-17T07:03:45.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: fsmap 2024-04-17T07:03:45.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:45.029 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: mgrmap e13: x(active, since 77s) 2024-04-17T07:03:45.030 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: overall HEALTH_OK 2024-04-17T07:03:45.030 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:45.030 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:45.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.423+0000 7fb5fac00700 1 -- 172.21.15.103:0/1947733235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5d80035c0 msgr2=0x7fb5d8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:45.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.423+0000 7fb5fac00700 1 --2- 172.21.15.103:0/1947733235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5d80035c0 0x7fb5d8005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb5f41024c0 tx=0x7fb5e400feb0 comp rx=0 tx=0).stop 2024-04-17T07:03:45.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.423+0000 7fb5fac00700 1 -- 172.21.15.103:0/1947733235 shutdown_connections 2024-04-17T07:03:45.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.423+0000 7fb5fac00700 1 --2- 172.21.15.103:0/1947733235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5d80035c0 0x7fb5d8005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:45.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.423+0000 7fb5fac00700 1 -- 172.21.15.103:0/1947733235 >> 172.21.15.103:0/1947733235 conn(0x7fb5f40fe040 msgr2=0x7fb5f4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:45.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.423+0000 7fb5fac00700 1 -- 172.21.15.103:0/1947733235 shutdown_connections 2024-04-17T07:03:45.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.423+0000 7fb5fac00700 1 -- 172.21.15.103:0/1947733235 wait complete. 2024-04-17T07:03:45.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.425+0000 7fb5fac00700 1 Processor -- start 2024-04-17T07:03:45.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.425+0000 7fb5fac00700 1 -- start start 2024-04-17T07:03:45.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.426+0000 7fb5fac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5d80035c0 0x7fb5f419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:45.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.426+0000 7fb5fac00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f419a830 0x7fb5f419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:45.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.426+0000 7fb5fac00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f419f100 0x7fb5f419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:45.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.426+0000 7fb5fac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f410a4e0 con 0x7fb5d80035c0 2024-04-17T07:03:45.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.426+0000 7fb5fac00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f410a360 con 0x7fb5f419a830 2024-04-17T07:03:45.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.426+0000 7fb5fac00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f410a660 con 0x7fb5f419f100 2024-04-17T07:03:45.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.427+0000 7fb5f3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5d80035c0 0x7fb5f419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:45.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.427+0000 7fb5f3e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5d80035c0 0x7fb5f419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.103:60678/0 (socket says 172.21.15.103:60678) 2024-04-17T07:03:45.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.427+0000 7fb5f3e00700 1 -- 172.21.15.103:0/2203524705 learned_addr learned my addr 172.21.15.103:0/2203524705 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:03:45.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.427+0000 7fb5f8c00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f419f100 0x7fb5f419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:45.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.427+0000 7fb5f3400700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f419a830 0x7fb5f419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:45.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.427+0000 7fb5f3400700 1 -- 172.21.15.103:0/2203524705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f419a830 msgr2=0x7fb5f419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-17T07:03:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.427+0000 7fb5f3400700 1 -- 172.21.15.103:0/2203524705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f419a830 msgr2=0x7fb5f419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-17T07:03:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.427+0000 7fb5f3400700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f419a830 0x7fb5f419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T07:03:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.427+0000 7fb5f3400700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f419a830 0x7fb5f419ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T07:03:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.427+0000 7fb5f3e00700 1 -- 172.21.15.103:0/2203524705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f419f100 msgr2=0x7fb5f419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.428+0000 7fb5f3e00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f419f100 0x7fb5f419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.428+0000 7fb5f3e00700 1 -- 172.21.15.103:0/2203524705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f419a830 msgr2=0x7fb5f419ebc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:03:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.428+0000 7fb5f3e00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f419a830 0x7fb5f419ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:45.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.428+0000 7fb5f3e00700 1 -- 172.21.15.103:0/2203524705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f41a6e70 con 0x7fb5d80035c0 2024-04-17T07:03:45.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.428+0000 7fb5f8c00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f419f100 0x7fb5f419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:03:45.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.428+0000 7fb5f3e00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5d80035c0 0x7fb5f419a2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4010010 tx=0x7fb5e4013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:45.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.429+0000 7fb5f0c00700 1 -- 172.21.15.103:0/2203524705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4005020 con 0x7fb5d80035c0 2024-04-17T07:03:45.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.429+0000 7fb5f0c00700 1 -- 172.21.15.103:0/2203524705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5e40051c0 con 0x7fb5d80035c0 2024-04-17T07:03:45.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.429+0000 7fb5f0c00700 1 -- 172.21.15.103:0/2203524705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4003b60 con 0x7fb5d80035c0 2024-04-17T07:03:45.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.429+0000 7fb5fac00700 1 -- 172.21.15.103:0/2203524705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f41a7040 con 0x7fb5d80035c0 2024-04-17T07:03:45.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.429+0000 7fb5fac00700 1 -- 172.21.15.103:0/2203524705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f41a75b0 con 0x7fb5d80035c0 2024-04-17T07:03:45.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.430+0000 7fb5fac00700 1 -- 172.21.15.103:0/2203524705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f40640b0 con 0x7fb5d80035c0 2024-04-17T07:03:45.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.431+0000 7fb5f0c00700 1 -- 172.21.15.103:0/2203524705 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb5e4023070 con 0x7fb5d80035c0 2024-04-17T07:03:45.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.432+0000 7fb5f0c00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5d4039ae0 0x7fb5d403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:45.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.432+0000 7fb5f0c00700 1 -- 172.21.15.103:0/2203524705 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb5e404ef20 con 0x7fb5d80035c0 2024-04-17T07:03:45.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.432+0000 7fb5f3400700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5d4039ae0 0x7fb5d403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:45.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.434+0000 7fb5f3400700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5d4039ae0 0x7fb5d403bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009850 tx=0x7fb5e0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:45.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.442+0000 7fb5f0c00700 1 -- 172.21.15.103:0/2203524705 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb5e4028020 con 0x7fb5d80035c0 2024-04-17T07:03:45.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.771+0000 7fb5fac00700 1 -- 172.21.15.103:0/2203524705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb5f402df90 con 0x7fb5d80035c0 2024-04-17T07:03:45.771 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:45.771 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:45.771 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:45 smithi103 ceph-mon[35495]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:45.771 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:45.772 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:45.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.772+0000 7fb5f0c00700 1 -- 172.21.15.103:0/2203524705 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7fb5e400ee10 con 0x7fb5d80035c0 2024-04-17T07:03:45.772 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-17T07:03:45.772 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":3,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","modified":"2024-04-17T07:03:39.485487Z","created":"2024-04-17T07:01:16.008725Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-17T07:03:45.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 -- 172.21.15.103:0/2203524705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5d4039ae0 msgr2=0x7fb5d403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:45.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5d4039ae0 0x7fb5d403bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009850 tx=0x7fb5e0008040 comp rx=0 tx=0).stop 2024-04-17T07:03:45.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 -- 172.21.15.103:0/2203524705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5d80035c0 msgr2=0x7fb5f419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5d80035c0 0x7fb5f419a2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4010010 tx=0x7fb5e4013f90 comp rx=0 tx=0).stop 2024-04-17T07:03:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 -- 172.21.15.103:0/2203524705 shutdown_connections 2024-04-17T07:03:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5d4039ae0 0x7fb5d403bfa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f419f100 0x7fb5f419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5d80035c0 0x7fb5f419a2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 --2- 172.21.15.103:0/2203524705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f419a830 0x7fb5f419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 -- 172.21.15.103:0/2203524705 >> 172.21.15.103:0/2203524705 conn(0x7fb5f40fe040 msgr2=0x7fb5f40fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.775+0000 7fb5fac00700 1 -- 172.21.15.103:0/2203524705 shutdown_connections 2024-04-17T07:03:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:45.776+0000 7fb5fac00700 1 -- 172.21.15.103:0/2203524705 wait complete. 2024-04-17T07:03:45.777 INFO:teuthology.orchestra.run.smithi103.stderr:dumped monmap epoch 3 2024-04-17T07:03:46.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:46.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:46.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:45 smithi086 ceph-mon[31733]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:46.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:46.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:46.399 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-17T07:03:46.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph config generate-minimal-conf 2024-04-17T07:03:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: mon.b calling monitor election 2024-04-17T07:03:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: mon.b calling monitor election 2024-04-17T07:03:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: mon.c calling monitor election 2024-04-17T07:03:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: mon.a calling monitor election 2024-04-17T07:03:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-17T07:03:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: monmap epoch 3 2024-04-17T07:03:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: last_changed 2024-04-17T07:03:39.485487+0000 2024-04-17T07:03:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:03:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: min_mon_release 19 (squid) 2024-04-17T07:03:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: election_strategy: 1 2024-04-17T07:03:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:03:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T07:03:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-17T07:03:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: fsmap 2024-04-17T07:03:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: mgrmap e13: x(active, since 79s) 2024-04-17T07:03:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:46 smithi086 ceph-mon[31733]: overall HEALTH_OK 2024-04-17T07:03:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:03:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:03:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: mon.a calling monitor election 2024-04-17T07:03:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: mon.c calling monitor election 2024-04-17T07:03:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:03:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:46.890 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-17T07:03:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: monmap epoch 3 2024-04-17T07:03:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: last_changed 2024-04-17T07:03:39.485487+0000 2024-04-17T07:03:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:03:46.891 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: min_mon_release 19 (squid) 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: election_strategy: 1 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: fsmap 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: mgrmap e13: x(active, since 77s) 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: overall HEALTH_OK 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:46.892 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: mon.b calling monitor election 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: mon.b calling monitor election 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: mon.c calling monitor election 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: mon.a calling monitor election 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: monmap epoch 3 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: last_changed 2024-04-17T07:03:39.485487+0000 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: min_mon_release 19 (squid) 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: election_strategy: 1 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-17T07:03:46.893 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: fsmap 2024-04-17T07:03:46.894 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:46.894 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: mgrmap e13: x(active, since 79s) 2024-04-17T07:03:46.894 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:46 smithi098 ceph-mon[35418]: overall HEALTH_OK 2024-04-17T07:03:47.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: mon.b calling monitor election 2024-04-17T07:03:47.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: mon.b calling monitor election 2024-04-17T07:03:47.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: mon.c calling monitor election 2024-04-17T07:03:47.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: mon.a calling monitor election 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: monmap epoch 3 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: last_changed 2024-04-17T07:03:39.485487+0000 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: created 2024-04-17T07:01:16.008725+0000 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: min_mon_release 19 (squid) 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: election_strategy: 1 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: 1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.c 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: fsmap 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: osdmap e4: 0 total, 0 up, 0 in 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: mgrmap e13: x(active, since 79s) 2024-04-17T07:03:47.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:46 smithi103 ceph-mon[35495]: overall HEALTH_OK 2024-04-17T07:03:48.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:47 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:47 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:47 smithi086 ceph-mon[31733]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:48.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:47 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:47 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:47 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:47 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:48.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:47 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:48.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:47 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:48.114 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:47 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.114 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:47 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.114 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:47 smithi098 ceph-mon[35418]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:48.114 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:47 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.114 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:47 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.115 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:47 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.115 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:47 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:48.115 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:47 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:48.115 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:47 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:47 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:47 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:47 smithi103 ceph-mon[35495]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:47 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:47 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:47 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:47 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:47 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:47 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.971+0000 7f3157e00700 1 -- 172.21.15.86:0/3179641223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315010a670 msgr2=0x7f315010aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.971+0000 7f3157e00700 1 --2- 172.21.15.86:0/3179641223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315010a670 0x7f315010aa50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f314400b0a0 tx=0x7f314400b3b0 comp rx=0 tx=0).stop 2024-04-17T07:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.971+0000 7f3157e00700 1 -- 172.21.15.86:0/3179641223 shutdown_connections 2024-04-17T07:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.971+0000 7f3157e00700 1 --2- 172.21.15.86:0/3179641223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315010a670 0x7f315010aa50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.971+0000 7f3157e00700 1 -- 172.21.15.86:0/3179641223 >> 172.21.15.86:0/3179641223 conn(0x7f3150100160 msgr2=0x7f3150102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.971+0000 7f3157e00700 1 -- 172.21.15.86:0/3179641223 shutdown_connections 2024-04-17T07:03:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.971+0000 7f3157e00700 1 -- 172.21.15.86:0/3179641223 wait complete. 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: Updating smithi098:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: Updating smithi086:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: Updating smithi103:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.972 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.973 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.973 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.972+0000 7f3157e00700 1 Processor -- start 2024-04-17T07:03:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.972+0000 7f3157e00700 1 -- start start 2024-04-17T07:03:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.973+0000 7f3157e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315010a670 0x7f315019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.973+0000 7f3157e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315019a7a0 0x7f315019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.973+0000 7f3157e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f315019f070 0x7f315019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.973+0000 7f3157e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f315010eb10 con 0x7f315010a670 2024-04-17T07:03:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.973+0000 7f3157e00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f315010e990 con 0x7f315019f070 2024-04-17T07:03:48.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.973+0000 7f3157e00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f315010ec90 con 0x7f315019a7a0 2024-04-17T07:03:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.973+0000 7f3155400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315010a670 0x7f315019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.973+0000 7f3155400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315010a670 0x7f315019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52746/0 (socket says 172.21.15.86:52746) 2024-04-17T07:03:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.973+0000 7f3155400700 1 -- 172.21.15.86:0/447733489 learned_addr learned my addr 172.21.15.86:0/447733489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:03:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.974+0000 7f3155400700 1 -- 172.21.15.86:0/447733489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315019a7a0 msgr2=0x7f315019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.974+0000 7f3155400700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315019a7a0 0x7f315019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.974+0000 7f3155400700 1 -- 172.21.15.86:0/447733489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f315019f070 msgr2=0x7f315019f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.974+0000 7f3155400700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f315019f070 0x7f315019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.974+0000 7f3155400700 1 -- 172.21.15.86:0/447733489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31501a6de0 con 0x7f315010a670 2024-04-17T07:03:48.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.974+0000 7f3155400700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315010a670 0x7f315019a260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f314400bf50 tx=0x7f31440075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:48.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.974+0000 7f314e000700 1 -- 172.21.15.86:0/447733489 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314401f020 con 0x7f315010a670 2024-04-17T07:03:48.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.974+0000 7f314e000700 1 -- 172.21.15.86:0/447733489 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3144005520 con 0x7f315010a670 2024-04-17T07:03:48.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.975+0000 7f314e000700 1 -- 172.21.15.86:0/447733489 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144017040 con 0x7f315010a670 2024-04-17T07:03:48.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.975+0000 7f3157e00700 1 -- 172.21.15.86:0/447733489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31501a6fb0 con 0x7f315010a670 2024-04-17T07:03:48.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.975+0000 7f3157e00700 1 -- 172.21.15.86:0/447733489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31501a7440 con 0x7f315010a670 2024-04-17T07:03:48.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.976+0000 7f3157e00700 1 -- 172.21.15.86:0/447733489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31500640b0 con 0x7f315010a670 2024-04-17T07:03:48.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.976+0000 7f314e000700 1 -- 172.21.15.86:0/447733489 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3144022020 con 0x7f315010a670 2024-04-17T07:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.977+0000 7f314e000700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3138039ae0 0x7f313803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.977+0000 7f314e000700 1 -- 172.21.15.86:0/447733489 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f314400f020 con 0x7f315010a670 2024-04-17T07:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.977+0000 7f3154a00700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3138039ae0 0x7f313803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:48.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.978+0000 7f3154a00700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3138039ae0 0x7f313803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f313c0098e0 tx=0x7f313c006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:48.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:48.983+0000 7f314e000700 1 -- 172.21.15.86:0/447733489 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f31440102f0 con 0x7f315010a670 2024-04-17T07:03:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.210+0000 7f3157e00700 1 -- 172.21.15.86:0/447733489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f31501a78c0 con 0x7f315010a670 2024-04-17T07:03:49.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.210+0000 7f314e000700 1 -- 172.21.15.86:0/447733489 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f3144012960 con 0x7f315010a670 2024-04-17T07:03:49.211 INFO:teuthology.orchestra.run.smithi086.stdout:# minimal ceph.conf for 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:49.211 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-17T07:03:49.211 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:03:49.211 INFO:teuthology.orchestra.run.smithi086.stdout: mon_host = [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] 2024-04-17T07:03:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 -- 172.21.15.86:0/447733489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3138039ae0 msgr2=0x7f313803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3138039ae0 0x7f313803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f313c0098e0 tx=0x7f313c006dd0 comp rx=0 tx=0).stop 2024-04-17T07:03:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 -- 172.21.15.86:0/447733489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315010a670 msgr2=0x7f315019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315010a670 0x7f315019a260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f314400bf50 tx=0x7f31440075c0 comp rx=0 tx=0).stop 2024-04-17T07:03:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 -- 172.21.15.86:0/447733489 shutdown_connections 2024-04-17T07:03:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3138039ae0 0x7f313803bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:49.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f315019f070 0x7f315019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:49.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315010a670 0x7f315019a260 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:49.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 --2- 172.21.15.86:0/447733489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f315019a7a0 0x7f315019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:49.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 -- 172.21.15.86:0/447733489 >> 172.21.15.86:0/447733489 conn(0x7f3150100160 msgr2=0x7f3150106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:49.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 -- 172.21.15.86:0/447733489 shutdown_connections 2024-04-17T07:03:49.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:03:49.213+0000 7f3157e00700 1 -- 172.21.15.86:0/447733489 wait complete. 2024-04-17T07:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-17T07:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T07:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T07:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: Updating smithi098:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: Updating smithi086:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: Updating smithi103:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-17T07:03:49.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-17T07:03:49.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-17T07:03:49.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: Updating smithi098:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: Updating smithi086:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: Updating smithi103:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/config/ceph.conf 2024-04-17T07:03:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:49 smithi086 ceph-mon[31733]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:49.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:49.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:49.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:03:49.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:49.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:49 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/447733489' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:49.950 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-17T07:03:49.951 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:03:49.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-04-17T07:03:50.026 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:03:50.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:03:50.149 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T07:03:50.149 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.conf 2024-04-17T07:03:50.234 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:49 smithi098 ceph-mon[35418]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:50.234 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:50.234 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:50.234 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:50.234 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:50.234 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:50.234 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:03:50.234 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:50.234 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:49 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/447733489' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:50.249 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T07:03:50.249 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:03:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:49 smithi103 ceph-mon[35495]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:03:50.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:50.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:49 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/447733489' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:50.385 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T07:03:50.386 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-04-17T07:03:50.491 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T07:03:50.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-17T07:03:50.633 INFO:tasks.cephadm:Adding mgr.x on smithi086 2024-04-17T07:03:50.633 INFO:tasks.cephadm:Adding mgr.y on smithi098 2024-04-17T07:03:50.633 INFO:tasks.cephadm:Adding mgr.z on smithi103 2024-04-17T07:03:50.633 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch apply mgr '3;smithi086=x;smithi098=y;smithi103=z' 2024-04-17T07:03:51.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:50 smithi103 ceph-mon[35495]: Reconfiguring mon.a (unknown last config time)... 2024-04-17T07:03:51.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:50 smithi103 ceph-mon[35495]: Reconfiguring daemon mon.a on smithi086 2024-04-17T07:03:51.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:50 smithi086 ceph-mon[31733]: Reconfiguring mon.a (unknown last config time)... 2024-04-17T07:03:51.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:50 smithi086 ceph-mon[31733]: Reconfiguring daemon mon.a on smithi086 2024-04-17T07:03:51.129 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:03:51.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:50 smithi098 ceph-mon[35418]: Reconfiguring mon.a (unknown last config time)... 2024-04-17T07:03:51.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:50 smithi098 ceph-mon[35418]: Reconfiguring daemon mon.a on smithi086 2024-04-17T07:03:51.989 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:51 smithi103 ceph-mon[35495]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:51.989 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:51.989 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:51.990 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:51.990 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:03:51.990 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:52.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:51 smithi086 ceph-mon[31733]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:52.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:52.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:52.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:52.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:03:52.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:52.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:51 smithi098 ceph-mon[35418]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:03:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:53.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:52 smithi086 ceph-mon[31733]: Reconfiguring mon.b (monmap changed)... 2024-04-17T07:03:53.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:52 smithi086 ceph-mon[31733]: Reconfiguring daemon mon.b on smithi098 2024-04-17T07:03:53.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.090+0000 7f37abe00700 1 -- 172.21.15.103:0/91839501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37a4100e30 msgr2=0x7f37a4103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:53.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.090+0000 7f37abe00700 1 --2- 172.21.15.103:0/91839501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37a4100e30 0x7f37a4103260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f379800b0a0 tx=0x7f379800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:03:53.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.091+0000 7f37abe00700 1 -- 172.21.15.103:0/91839501 shutdown_connections 2024-04-17T07:03:53.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.091+0000 7f37abe00700 1 --2- 172.21.15.103:0/91839501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37a4100e30 0x7f37a4103260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:53.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.091+0000 7f37abe00700 1 -- 172.21.15.103:0/91839501 >> 172.21.15.103:0/91839501 conn(0x7f37a40fca50 msgr2=0x7f37a40fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.091+0000 7f37abe00700 1 -- 172.21.15.103:0/91839501 shutdown_connections 2024-04-17T07:03:53.092 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:52 smithi103 ceph-mon[35495]: Reconfiguring mon.b (monmap changed)... 2024-04-17T07:03:53.092 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:52 smithi103 ceph-mon[35495]: Reconfiguring daemon mon.b on smithi098 2024-04-17T07:03:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.091+0000 7f37abe00700 1 -- 172.21.15.103:0/91839501 wait complete. 2024-04-17T07:03:53.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.092+0000 7f37abe00700 1 Processor -- start 2024-04-17T07:03:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.093+0000 7f37abe00700 1 -- start start 2024-04-17T07:03:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.093+0000 7f37abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37a4100e30 0x7f37a419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.093+0000 7f37abe00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37a419e7b0 0x7f37a41a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.093+0000 7f37abe00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37a419ec30 0x7f37a41a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:53.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.093+0000 7f37abe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37a4106350 con 0x7f37a4100e30 2024-04-17T07:03:53.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.093+0000 7f37abe00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41061d0 con 0x7f37a419ec30 2024-04-17T07:03:53.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.093+0000 7f37abe00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41064d0 con 0x7f37a419e7b0 2024-04-17T07:03:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.093+0000 7f37a8a00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37a419e7b0 0x7f37a41a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.093+0000 7f37a8a00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37a419e7b0 0x7f37a41a3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:52862/0 (socket says 172.21.15.103:52862) 2024-04-17T07:03:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.093+0000 7f37a8a00700 1 -- 172.21.15.103:0/2121135653 learned_addr learned my addr 172.21.15.103:0/2121135653 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:03:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.094+0000 7f37a9400700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37a4100e30 0x7f37a419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.094+0000 7f37a9e00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37a419ec30 0x7f37a41a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.094+0000 7f37a8a00700 1 -- 172.21.15.103:0/2121135653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37a419ec30 msgr2=0x7f37a41a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:53.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.094+0000 7f37a8a00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37a419ec30 0x7f37a41a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.094+0000 7f37a8a00700 1 -- 172.21.15.103:0/2121135653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37a4100e30 msgr2=0x7f37a419e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.094+0000 7f37a8a00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37a4100e30 0x7f37a419e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.094+0000 7f37a8a00700 1 -- 172.21.15.103:0/2121135653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a41a6680 con 0x7f37a419e7b0 2024-04-17T07:03:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.094+0000 7f37a9e00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37a419ec30 0x7f37a41a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:03:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.094+0000 7f37a8a00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37a419e7b0 0x7f37a41a3820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f379000b780 tx=0x7f379000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.095+0000 7f37a2000700 1 -- 172.21.15.103:0/2121135653 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790004230 con 0x7f37a419e7b0 2024-04-17T07:03:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.095+0000 7f37a2000700 1 -- 172.21.15.103:0/2121135653 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37900043d0 con 0x7f37a419e7b0 2024-04-17T07:03:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.095+0000 7f37abe00700 1 -- 172.21.15.103:0/2121135653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a41a6970 con 0x7f37a419e7b0 2024-04-17T07:03:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.095+0000 7f37a2000700 1 -- 172.21.15.103:0/2121135653 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790011b60 con 0x7f37a419e7b0 2024-04-17T07:03:53.096 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.095+0000 7f37abe00700 1 -- 172.21.15.103:0/2121135653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a41a6f80 con 0x7f37a419e7b0 2024-04-17T07:03:53.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.096+0000 7f37abe00700 1 -- 172.21.15.103:0/2121135653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a40640b0 con 0x7f37a419e7b0 2024-04-17T07:03:53.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.096+0000 7f37a2000700 1 -- 172.21.15.103:0/2121135653 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f37900036a0 con 0x7f37a419e7b0 2024-04-17T07:03:53.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.097+0000 7f37a2000700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f378c039ae0 0x7f378c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:03:53.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.097+0000 7f37a2000700 1 -- 172.21.15.103:0/2121135653 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f379004c420 con 0x7f37a419e7b0 2024-04-17T07:03:53.097 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.097+0000 7f37a9400700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f378c039ae0 0x7f378c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:03:53.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.098+0000 7f37a9400700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f378c039ae0 0x7f378c03bfa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3798003f00 tx=0x7f3798005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:03:53.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.103+0000 7f37a2000700 1 -- 172.21.15.103:0/2121135653 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3790021e00 con 0x7f37a419e7b0 2024-04-17T07:03:53.231 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:52 smithi098 ceph-mon[35418]: Reconfiguring mon.b (monmap changed)... 2024-04-17T07:03:53.231 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:52 smithi098 ceph-mon[35418]: Reconfiguring daemon mon.b on smithi098 2024-04-17T07:03:53.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.343+0000 7f37abe00700 1 -- 172.21.15.103:0/2121135653 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi098=y;smithi103=z", "target": ["mon-mgr", ""]}) -- 0x7f37a4062f20 con 0x7f378c039ae0 2024-04-17T07:03:53.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.358+0000 7f37a2000700 1 -- 172.21.15.103:0/2121135653 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f37a4062f20 con 0x7f378c039ae0 2024-04-17T07:03:53.358 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled mgr update... 2024-04-17T07:03:53.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.361+0000 7f37abe00700 1 -- 172.21.15.103:0/2121135653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f378c039ae0 msgr2=0x7f378c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:53.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.361+0000 7f37abe00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f378c039ae0 0x7f378c03bfa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3798003f00 tx=0x7f3798005040 comp rx=0 tx=0).stop 2024-04-17T07:03:53.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.361+0000 7f37abe00700 1 -- 172.21.15.103:0/2121135653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37a419e7b0 msgr2=0x7f37a41a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:03:53.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.361+0000 7f37abe00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37a419e7b0 0x7f37a41a3820 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f379000b780 tx=0x7f379000ba90 comp rx=0 tx=0).stop 2024-04-17T07:03:53.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.361+0000 7f37abe00700 1 -- 172.21.15.103:0/2121135653 shutdown_connections 2024-04-17T07:03:53.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.361+0000 7f37abe00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f378c039ae0 0x7f378c03bfa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:53.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.361+0000 7f37abe00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37a419ec30 0x7f37a41a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.362+0000 7f37abe00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37a4100e30 0x7f37a419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.362+0000 7f37abe00700 1 --2- 172.21.15.103:0/2121135653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37a419e7b0 0x7f37a41a3820 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:03:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.362+0000 7f37abe00700 1 -- 172.21.15.103:0/2121135653 >> 172.21.15.103:0/2121135653 conn(0x7f37a40fca50 msgr2=0x7f37a40fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:03:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.362+0000 7f37abe00700 1 -- 172.21.15.103:0/2121135653 shutdown_connections 2024-04-17T07:03:53.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:03:53.362+0000 7f37abe00700 1 -- 172.21.15.103:0/2121135653 wait complete. 2024-04-17T07:03:54.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:53 smithi103 ceph-mon[35495]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:54.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:54.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:54.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:54.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:54.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:03:54.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:53 smithi086 ceph-mon[31733]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:03:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:54.104 DEBUG:teuthology.orchestra.run.smithi098:mgr.y> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.y.service 2024-04-17T07:03:54.112 DEBUG:teuthology.orchestra.run.smithi103:mgr.z> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.z.service 2024-04-17T07:03:54.119 INFO:tasks.cephadm:Deploying OSDs... 2024-04-17T07:03:54.119 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:03:54.119 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T07:03:54.178 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T07:03:54.240 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-04-17T07:03:54.241 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:53 smithi098 ceph-mon[35418]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:54.241 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:54.242 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:54.242 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:54.242 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-17T07:03:54.242 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-17T07:03:54.242 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T07:03:54.307 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:54.307 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-17T07:03:54.307 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:54.307 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:54.307 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-17 07:02:39.384197344 +0000 2024-04-17T07:03:54.307 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-17 06:57:21.386240572 +0000 2024-04-17T07:03:54.307 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-17 06:57:21.386240572 +0000 2024-04-17T07:03:54.308 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-17 06:57:21.386240572 +0000 2024-04-17T07:03:54.308 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T07:03:54.399 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-17T07:03:54.400 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-17T07:03:54.400 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000532569 s, 961 kB/s 2024-04-17T07:03:54.403 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T07:03:54.463 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-04-17T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:54.514 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-17T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-17 07:02:39.448195962 +0000 2024-04-17T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-17 06:57:21.985227557 +0000 2024-04-17T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-17 06:57:21.985227557 +0000 2024-04-17T07:03:54.515 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-17 06:57:21.985227557 +0000 2024-04-17T07:03:54.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T07:03:54.602 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-17T07:03:54.602 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-17T07:03:54.602 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000355268 s, 1.4 MB/s 2024-04-17T07:03:54.605 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T07:03:54.656 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-04-17T07:03:54.706 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T07:03:54.706 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:54.706 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-17T07:03:54.707 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:54.707 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:54.707 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-17 07:02:39.520194407 +0000 2024-04-17T07:03:54.707 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-17 06:57:22.596214281 +0000 2024-04-17T07:03:54.707 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-17 06:57:22.596214281 +0000 2024-04-17T07:03:54.707 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-17 06:57:22.596214281 +0000 2024-04-17T07:03:54.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T07:03:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-17T07:03:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-17T07:03:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000487258 s, 1.1 MB/s 2024-04-17T07:03:54.802 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T07:03:54.856 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-04-17T07:03:54.875 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:54 smithi103 ceph-mon[35495]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi098=y;smithi103=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:03:54.875 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:54 smithi103 ceph-mon[35495]: Saving service mgr spec with placement smithi086=x;smithi098=y;smithi103=z;count:3 2024-04-17T07:03:54.875 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:54 smithi103 ceph-mon[35495]: Reconfiguring mon.c (monmap changed)... 2024-04-17T07:03:54.875 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:54 smithi103 ceph-mon[35495]: Reconfiguring daemon mon.c on smithi103 2024-04-17T07:03:54.902 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T07:03:54.903 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:54.903 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-17T07:03:54.903 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:54.903 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:54.903 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-17 07:02:39.592192852 +0000 2024-04-17T07:03:54.903 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-17 06:57:23.359197702 +0000 2024-04-17T07:03:54.903 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-17 06:57:23.359197702 +0000 2024-04-17T07:03:54.903 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-17 06:57:23.359197702 +0000 2024-04-17T07:03:54.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T07:03:54.984 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-17T07:03:54.984 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-17T07:03:54.984 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000315223 s, 1.6 MB/s 2024-04-17T07:03:54.987 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T07:03:55.039 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-17T07:03:55.039 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T07:03:55.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:54 smithi086 ceph-mon[31733]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi098=y;smithi103=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:03:55.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:54 smithi086 ceph-mon[31733]: Saving service mgr spec with placement smithi086=x;smithi098=y;smithi103=z;count:3 2024-04-17T07:03:55.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:54 smithi086 ceph-mon[31733]: Reconfiguring mon.c (monmap changed)... 2024-04-17T07:03:55.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:54 smithi086 ceph-mon[31733]: Reconfiguring daemon mon.c on smithi103 2024-04-17T07:03:55.099 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T07:03:55.099 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2024-04-17T07:03:55.151 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T07:03:55.151 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:55.151 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-17T07:03:55.151 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:55.151 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:55.151 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-17 07:03:38.596551105 +0000 2024-04-17T07:03:55.152 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-17 06:57:22.095582612 +0000 2024-04-17T07:03:55.152 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-17 06:57:22.095582612 +0000 2024-04-17T07:03:55.152 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-17 06:57:22.095582612 +0000 2024-04-17T07:03:55.152 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T07:03:55.231 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:54 smithi098 ceph-mon[35418]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi098=y;smithi103=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:03:55.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:54 smithi098 ceph-mon[35418]: Saving service mgr spec with placement smithi086=x;smithi098=y;smithi103=z;count:3 2024-04-17T07:03:55.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:54 smithi098 ceph-mon[35418]: Reconfiguring mon.c (monmap changed)... 2024-04-17T07:03:55.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:54 smithi098 ceph-mon[35418]: Reconfiguring daemon mon.c on smithi103 2024-04-17T07:03:55.242 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-17T07:03:55.242 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-17T07:03:55.242 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000362005 s, 1.4 MB/s 2024-04-17T07:03:55.247 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T07:03:55.301 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2024-04-17T07:03:55.350 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T07:03:55.351 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:55.351 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-17T07:03:55.351 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:55.351 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:55.351 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-17 07:03:38.597551092 +0000 2024-04-17T07:03:55.351 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-17 06:57:22.715574646 +0000 2024-04-17T07:03:55.351 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-17 06:57:22.715574646 +0000 2024-04-17T07:03:55.351 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-17 06:57:22.715574646 +0000 2024-04-17T07:03:55.351 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T07:03:55.441 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-17T07:03:55.441 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-17T07:03:55.441 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.00037655 s, 1.4 MB/s 2024-04-17T07:03:55.445 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T07:03:55.502 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2024-04-17T07:03:55.552 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T07:03:55.552 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:55.552 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-17T07:03:55.552 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:55.552 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:55.552 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-17 07:03:38.598551079 +0000 2024-04-17T07:03:55.552 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-17 06:57:23.328566770 +0000 2024-04-17T07:03:55.552 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-17 06:57:23.328566770 +0000 2024-04-17T07:03:55.552 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-17 06:57:23.328566770 +0000 2024-04-17T07:03:55.552 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T07:03:55.641 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-17T07:03:55.642 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-17T07:03:55.642 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000376468 s, 1.4 MB/s 2024-04-17T07:03:55.646 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T07:03:55.699 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2024-04-17T07:03:55.748 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T07:03:55.748 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:55.748 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-17T07:03:55.748 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:55.748 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:55.748 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-17 07:03:38.599551066 +0000 2024-04-17T07:03:55.749 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-17 06:57:23.951558767 +0000 2024-04-17T07:03:55.749 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-17 06:57:23.951558767 +0000 2024-04-17T07:03:55.749 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-17 06:57:23.951558767 +0000 2024-04-17T07:03:55.749 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T07:03:55.829 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-17T07:03:55.830 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-17T07:03:55.830 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000373164 s, 1.4 MB/s 2024-04-17T07:03:55.832 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T07:03:55.885 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-17T07:03:55.885 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T07:03:55.943 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T07:03:55.944 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-04-17T07:03:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T07:03:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:55.992 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-17T07:03:55.992 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:55.992 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:55.992 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-17 07:03:27.630193640 +0000 2024-04-17T07:03:55.992 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-17 06:57:22.876472425 +0000 2024-04-17T07:03:55.992 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-17 06:57:22.876472425 +0000 2024-04-17T07:03:55.992 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-17 06:57:22.876472425 +0000 2024-04-17T07:03:55.993 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T07:03:56.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:55 smithi098 ceph-mon[35418]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:56.025 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:55 smithi103 ceph-mon[35495]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-17T07:03:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-17T07:03:56.063 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000309982 s, 1.7 MB/s 2024-04-17T07:03:56.066 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T07:03:56.149 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-04-17T07:03:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T07:03:56.237 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:56.237 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-17T07:03:56.237 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:56.237 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:56.237 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-17 07:03:27.631193622 +0000 2024-04-17T07:03:56.237 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-17 06:57:23.473461096 +0000 2024-04-17T07:03:56.237 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-17 06:57:23.473461096 +0000 2024-04-17T07:03:56.238 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-17 06:57:23.473461096 +0000 2024-04-17T07:03:56.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T07:03:56.312 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-17T07:03:56.312 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-17T07:03:56.312 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000327158 s, 1.6 MB/s 2024-04-17T07:03:56.315 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T07:03:56.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:55 smithi086 ceph-mon[31733]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:56.365 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-04-17T07:03:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T07:03:56.410 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:56.410 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-17T07:03:56.410 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:56.410 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:56.411 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-17 07:03:27.632193603 +0000 2024-04-17T07:03:56.411 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-17 06:57:24.059449975 +0000 2024-04-17T07:03:56.411 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-17 06:57:24.059449975 +0000 2024-04-17T07:03:56.411 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-17 06:57:24.059449975 +0000 2024-04-17T07:03:56.411 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T07:03:56.538 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-17T07:03:56.538 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-17T07:03:56.538 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000345998 s, 1.5 MB/s 2024-04-17T07:03:56.543 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T07:03:56.586 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-04-17T07:03:56.670 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T07:03:56.670 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T07:03:56.671 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-17T07:03:56.671 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T07:03:56.671 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T07:03:56.671 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-17 07:03:27.633193584 +0000 2024-04-17T07:03:56.671 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-17 06:57:24.676438267 +0000 2024-04-17T07:03:56.671 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-17 06:57:24.676438267 +0000 2024-04-17T07:03:56.671 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-17 06:57:24.676438267 +0000 2024-04-17T07:03:56.671 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T07:03:56.777 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-17T07:03:56.777 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-17T07:03:56.777 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000271406 s, 1.9 MB/s 2024-04-17T07:03:56.780 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T07:03:56.819 INFO:tasks.cephadm:Deploying osd.0 on smithi086 with /dev/vg_nvme/lv_4... 2024-04-17T07:03:56.819 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-17T07:03:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:03:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:03:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:03:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:57.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:56 smithi103 ceph-mon[35495]: Deploying daemon mgr.z on smithi103 2024-04-17T07:03:57.248 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:56 smithi098 ceph-mon[35418]: Deploying daemon mgr.z on smithi103 2024-04-17T07:03:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:03:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:03:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:03:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:03:57.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:03:57.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:03:57.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:03:57.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:56 smithi086 ceph-mon[31733]: Deploying daemon mgr.z on smithi103 2024-04-17T07:03:58.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:57 smithi086 ceph-mon[31733]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:58.240 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:57 smithi098 ceph-mon[35418]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:58.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:57 smithi103 ceph-mon[35495]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:03:58.988 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:03:58 smithi103 systemd[1]: Starting Ceph mgr.z for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:03:59.242 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:03:58 smithi103 podman[37427]: 2024-04-17 07:03:58.987227067 +0000 UTC m=+0.126253463 container create afa69e5a544e93d35f242876214a13db4306cc95f994a247af597ef92e2ec8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-17T07:03:59.242 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:03:58 smithi103 podman[37427]: 2024-04-17 07:03:58.896063122 +0000 UTC m=+0.035089497 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:03:59.242 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:03:59 smithi103 podman[37427]: 2024-04-17 07:03:59.232249843 +0000 UTC m=+0.371276249 container init afa69e5a544e93d35f242876214a13db4306cc95f994a247af597ef92e2ec8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-17T07:03:59.243 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:03:59 smithi103 podman[37427]: 2024-04-17 07:03:59.242458099 +0000 UTC m=+0.381484479 container start afa69e5a544e93d35f242876214a13db4306cc95f994a247af597ef92e2ec8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-17T07:03:59.526 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:03:59 smithi103 ceph-mgr[37446]: -- 172.21.15.103:0/3473506848 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a714965a0 con 0x559a7146ac00 2024-04-17T07:03:59.526 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:03:59 smithi103 bash[37427]: afa69e5a544e93d35f242876214a13db4306cc95f994a247af597ef92e2ec8fc 2024-04-17T07:03:59.526 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:03:59 smithi103 systemd[1]: Started Ceph mgr.z for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:03:59.977 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-17T07:03:59.977 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-17T07:03:59.977 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-17T07:03:59.977 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-17T07:03:59.977 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.037224 s, 282 MB/s 2024-04-17T07:03:59.977 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-17T07:04:00.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:59 smithi098 ceph-mon[35418]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:00.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:04:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:04:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:04:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:03:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:00.276 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:00.033+0000 7ffb0385f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T07:04:00.276 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:00.154+0000 7ffb0385f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T07:04:00.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:59 smithi103 ceph-mon[35495]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:00.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:00.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:00.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:00.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:04:00.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:04:00.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:04:00.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:03:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:59 smithi086 ceph-mon[31733]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:00.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:00.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:04:00.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-17T07:04:00.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:04:00.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:03:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:00.773 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-04-17T07:04:01.213 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:04:01.217 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:00 smithi098 ceph-mon[35418]: Deploying daemon mgr.y on smithi098 2024-04-17T07:04:01.260 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:00 smithi103 ceph-mon[35495]: Deploying daemon mgr.y on smithi098 2024-04-17T07:04:01.260 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:00.956+0000 7ffb0385f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T07:04:01.261 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:01.082+0000 7ffb0385f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T07:04:01.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:00 smithi086 ceph-mon[31733]: Deploying daemon mgr.y on smithi098 2024-04-17T07:04:01.526 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:01.260+0000 7ffb0385f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T07:04:01.526 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:01.385+0000 7ffb0385f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T07:04:01.526 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:01.495+0000 7ffb0385f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T07:04:02.087 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:01 smithi103 ceph-mon[35495]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:02.088 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:01.830+0000 7ffb0385f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T07:04:02.088 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:01.970+0000 7ffb0385f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T07:04:02.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:01 smithi098 ceph-mon[35418]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:02.307 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:01 smithi086 ceph-mon[31733]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:02.427 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:02.087+0000 7ffb0385f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T07:04:02.428 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:02.212+0000 7ffb0385f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T07:04:02.428 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:02.427+0000 7ffb0385f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T07:04:02.634 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:02 smithi098 systemd[1]: Starting Ceph mgr.y for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:04:02.924 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:02 smithi098 podman[36782]: 2024-04-17 07:04:02.540884541 +0000 UTC m=+0.037074640 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:04:02.924 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:02 smithi098 podman[36782]: 2024-04-17 07:04:02.676102245 +0000 UTC m=+0.172292371 container create 7f5ef95ce723a59537bf9beaf7b81e4e744596ab08a7b4ab3db9bbbec0d30c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:04:02.924 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:02 smithi098 podman[36782]: 2024-04-17 07:04:02.921978067 +0000 UTC m=+0.418168200 container init 7f5ef95ce723a59537bf9beaf7b81e4e744596ab08a7b4ab3db9bbbec0d30c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-17T07:04:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.138+0000 7fddc2400700 1 -- 172.21.15.86:0/4194411981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddbc10a2f0 msgr2=0x7fddbc10a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:04:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.138+0000 7fddc2400700 1 --2- 172.21.15.86:0/4194411981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddbc10a2f0 0x7fddbc10a750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdda80099c0 tx=0x7fdda8009cd0 comp rx=0 tx=0).stop 2024-04-17T07:04:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.138+0000 7fddc2400700 1 -- 172.21.15.86:0/4194411981 shutdown_connections 2024-04-17T07:04:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.138+0000 7fddc2400700 1 --2- 172.21.15.86:0/4194411981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddbc10ac90 0x7fddbc111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.138+0000 7fddc2400700 1 --2- 172.21.15.86:0/4194411981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddbc1090f0 0x7fddbc109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.138+0000 7fddc2400700 1 --2- 172.21.15.86:0/4194411981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddbc10a2f0 0x7fddbc10a750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.138+0000 7fddc2400700 1 -- 172.21.15.86:0/4194411981 >> 172.21.15.86:0/4194411981 conn(0x7fddbc0780c0 msgr2=0x7fddbc07a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:04:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.138+0000 7fddc2400700 1 -- 172.21.15.86:0/4194411981 shutdown_connections 2024-04-17T07:04:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.138+0000 7fddc2400700 1 -- 172.21.15.86:0/4194411981 wait complete. 2024-04-17T07:04:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.139+0000 7fddc2400700 1 Processor -- start 2024-04-17T07:04:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.139+0000 7fddc2400700 1 -- start start 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddc2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddbc1090f0 0x7fddbc19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddc2400700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddbc10a2f0 0x7fddbc19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddc2400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddbc10ac90 0x7fddbc1a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddc2400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc114c30 con 0x7fddbc1090f0 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddc2400700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc114ab0 con 0x7fddbc10a2f0 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddc2400700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc114db0 con 0x7fddbc10ac90 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddbb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddbc1090f0 0x7fddbc19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddbb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddbc1090f0 0x7fddbc19f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46716/0 (socket says 172.21.15.86:46716) 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddbb400700 1 -- 172.21.15.86:0/4269482584 learned_addr learned my addr 172.21.15.86:0/4269482584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddbaa00700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddbc10a2f0 0x7fddbc19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddbbe00700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddbc10ac90 0x7fddbc1a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:04:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddbb400700 1 -- 172.21.15.86:0/4269482584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddbc10ac90 msgr2=0x7fddbc1a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:04:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddbb400700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddbc10ac90 0x7fddbc1a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddbb400700 1 -- 172.21.15.86:0/4269482584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddbc10a2f0 msgr2=0x7fddbc19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:04:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddbb400700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddbc10a2f0 0x7fddbc19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.140+0000 7fddbb400700 1 -- 172.21.15.86:0/4269482584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc1a6eb0 con 0x7fddbc1090f0 2024-04-17T07:04:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.141+0000 7fddbb400700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddbc1090f0 0x7fddbc19f250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fddac00b530 tx=0x7fddac00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:04:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.141+0000 7fddb3e00700 1 -- 172.21.15.86:0/4269482584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac015020 con 0x7fddbc1090f0 2024-04-17T07:04:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.141+0000 7fddb3e00700 1 -- 172.21.15.86:0/4269482584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddac011ce0 con 0x7fddbc1090f0 2024-04-17T07:04:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.141+0000 7fddc2400700 1 -- 172.21.15.86:0/4269482584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc1a71a0 con 0x7fddbc1090f0 2024-04-17T07:04:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.141+0000 7fddb3e00700 1 -- 172.21.15.86:0/4269482584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac010d60 con 0x7fddbc1090f0 2024-04-17T07:04:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.141+0000 7fddc2400700 1 -- 172.21.15.86:0/4269482584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc1a7760 con 0x7fddbc1090f0 2024-04-17T07:04:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.142+0000 7fddb3e00700 1 -- 172.21.15.86:0/4269482584 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fddac018020 con 0x7fddbc1090f0 2024-04-17T07:04:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.142+0000 7fddc2400700 1 -- 172.21.15.86:0/4269482584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddbc069740 con 0x7fddbc1090f0 2024-04-17T07:04:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.143+0000 7fddb3e00700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdda0039890 0x7fdda003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:04:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.143+0000 7fddb3e00700 1 -- 172.21.15.86:0/4269482584 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fddac04c1c0 con 0x7fddbc1090f0 2024-04-17T07:04:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.143+0000 7fddbaa00700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdda0039890 0x7fdda003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:04:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.144+0000 7fddbaa00700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdda0039890 0x7fdda003bd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdda80038f0 tx=0x7fdda8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:04:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.149+0000 7fddb3e00700 1 -- 172.21.15.86:0/4269482584 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fddac01f020 con 0x7fddbc1090f0 2024-04-17T07:04:03.253 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:02 smithi098 podman[36782]: 2024-04-17 07:04:02.933647332 +0000 UTC m=+0.429837491 container start 7f5ef95ce723a59537bf9beaf7b81e4e744596ab08a7b4ab3db9bbbec0d30c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-17T07:04:03.253 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:03 smithi098 bash[36782]: 7f5ef95ce723a59537bf9beaf7b81e4e744596ab08a7b4ab3db9bbbec0d30c16 2024-04-17T07:04:03.253 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:03 smithi098 systemd[1]: Started Ceph mgr.y for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:04:03.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:03.391+0000 7fddc2400700 1 -- 172.21.15.86:0/4269482584 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fddbc062e90 con 0x7fdda0039890 2024-04-17T07:04:03.986 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:03.703+0000 7fe9ac34c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T07:04:03.987 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:03.828+0000 7fe9ac34c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T07:04:04.026 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:03.760+0000 7ffb0385f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T07:04:04.026 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:03.943+0000 7ffb0385f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T07:04:04.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-mon[35495]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:04.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:04:04.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:04:04.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:04.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:04.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-mon[35418]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:04.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:04:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:04:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:04.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:03 smithi086 ceph-mon[31733]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:04.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:04:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:04:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:04.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:04.776 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:04.381+0000 7ffb0385f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T07:04:04.777 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:04.523+0000 7ffb0385f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T07:04:04.991 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:04 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:04.707+0000 7fe9ac34c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T07:04:04.991 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:04 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:04.832+0000 7fe9ac34c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T07:04:05.046 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:04 smithi103 ceph-mon[35495]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:04:05.046 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:04.788+0000 7ffb0385f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:04:05.047 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:05.047+0000 7ffb0385f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T07:04:05.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:04 smithi098 ceph-mon[35418]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:04:05.253 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:05.018+0000 7fe9ac34c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T07:04:05.253 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:05.145+0000 7fe9ac34c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T07:04:05.325 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:04 smithi086 ceph-mon[31733]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:04:05.615 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:05.257+0000 7fe9ac34c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T07:04:05.615 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:05.615+0000 7fe9ac34c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T07:04:05.764 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:05.445+0000 7ffb0385f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:04:05.764 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:05.764+0000 7ffb0385f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T07:04:05.882 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:05.763+0000 7fe9ac34c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T07:04:05.882 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:05.882+0000 7fe9ac34c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T07:04:06.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:05 smithi103 ceph-mon[35495]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:06.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:06 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.237 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:06.011+0000 7fe9ac34c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T07:04:06.238 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:06 smithi098 ceph-mon[35418]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:06.239 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:06 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.239 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:06 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.239 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:06 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.239 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:06 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:05 smithi086 ceph-mon[31733]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:06.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:06 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:06.502 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:06.237+0000 7fe9ac34c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T07:04:06.526 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:06.074+0000 7ffb0385f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T07:04:07.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:07.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:07.276 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:07.117+0000 7ffb0385f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T07:04:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:07.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:07.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:07.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:07.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:08.002 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:07.567+0000 7fe9ac34c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T07:04:08.003 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:07.744+0000 7fe9ac34c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T07:04:08.234 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:07.814+0000 7ffb0385f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T07:04:08.234 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:07.943+0000 7ffb0385f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: Reconfiguring mgr.x (unknown last config time)... 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: Reconfiguring daemon mgr.x on smithi086 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1163806026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "770a17ec-a9cd-4339-b3d5-1a841ac60e52"}]: dispatch 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1163806026' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "770a17ec-a9cd-4339-b3d5-1a841ac60e52"}]': finished 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: osdmap e5: 1 total, 0 up, 1 in 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:08.235 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1977402922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:04:08.290 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:08.170+0000 7fe9ac34c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T07:04:08.290 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:08.290+0000 7fe9ac34c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T07:04:08.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:08.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: Reconfiguring mgr.x (unknown last config time)... 2024-04-17T07:04:08.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:04:08.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:04:08.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:08.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: Reconfiguring daemon mgr.x on smithi086 2024-04-17T07:04:08.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1163806026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "770a17ec-a9cd-4339-b3d5-1a841ac60e52"}]: dispatch 2024-04-17T07:04:08.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1163806026' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "770a17ec-a9cd-4339-b3d5-1a841ac60e52"}]': finished 2024-04-17T07:04:08.291 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: osdmap e5: 1 total, 0 up, 1 in 2024-04-17T07:04:08.292 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:08.292 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1977402922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:04:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: Reconfiguring mgr.x (unknown last config time)... 2024-04-17T07:04:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-17T07:04:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-17T07:04:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: Reconfiguring daemon mgr.x on smithi086 2024-04-17T07:04:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1163806026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "770a17ec-a9cd-4339-b3d5-1a841ac60e52"}]: dispatch 2024-04-17T07:04:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1163806026' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "770a17ec-a9cd-4339-b3d5-1a841ac60e52"}]': finished 2024-04-17T07:04:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: osdmap e5: 1 total, 0 up, 1 in 2024-04-17T07:04:08.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:08.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:08 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1977402922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:04:08.526 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:04:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z[37442]: 2024-04-17T07:04:08.234+0000 7ffb0385f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T07:04:09.002 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:08.550+0000 7fe9ac34c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:04:09.003 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:08.797+0000 7fe9ac34c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T07:04:09.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:09.026+0000 7fddb3e00700 1 -- 172.21.15.86:0/4269482584 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fddac01a920 con 0x7fddbc1090f0 2024-04-17T07:04:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:09 smithi103 ceph-mon[35495]: Standby manager daemon z started 2024-04-17T07:04:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:09 smithi103 ceph-mon[35495]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T07:04:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:09 smithi103 ceph-mon[35495]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:04:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:09 smithi103 ceph-mon[35495]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T07:04:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:09 smithi103 ceph-mon[35495]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:04:09.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:09 smithi086 ceph-mon[31733]: Standby manager daemon z started 2024-04-17T07:04:09.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:09 smithi086 ceph-mon[31733]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T07:04:09.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:09 smithi086 ceph-mon[31733]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:04:09.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:09 smithi086 ceph-mon[31733]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T07:04:09.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:09 smithi086 ceph-mon[31733]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:04:09.502 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:09.179+0000 7fe9ac34c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T07:04:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:09 smithi098 ceph-mon[35418]: Standby manager daemon z started 2024-04-17T07:04:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:09 smithi098 ceph-mon[35418]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-17T07:04:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:09 smithi098 ceph-mon[35418]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:04:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:09 smithi098 ceph-mon[35418]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-17T07:04:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:09 smithi098 ceph-mon[35418]: from='mgr.? 172.21.15.103:0/3890898759' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:04:09.829 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:09.518+0000 7fe9ac34c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T07:04:10.252 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:09.826+0000 7fe9ac34c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T07:04:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:10 smithi098 ceph-mon[35418]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:10 smithi098 ceph-mon[35418]: mgrmap e14: x(active, since 102s), standbys: z 2024-04-17T07:04:10.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T07:04:10.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:10 smithi103 ceph-mon[35495]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:10.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:10 smithi103 ceph-mon[35495]: mgrmap e14: x(active, since 102s), standbys: z 2024-04-17T07:04:10.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T07:04:10.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:10 smithi086 ceph-mon[31733]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:10.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:10 smithi086 ceph-mon[31733]: mgrmap e14: x(active, since 102s), standbys: z 2024-04-17T07:04:10.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-17T07:04:11.252 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:10.865+0000 7fe9ac34c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T07:04:11.987 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:11.573+0000 7fe9ac34c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T07:04:11.987 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:11.702+0000 7fe9ac34c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T07:04:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:12.063+0000 7fddb3e00700 1 -- 172.21.15.86:0/4269482584 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fddac01a400 con 0x7fddbc1090f0 2024-04-17T07:04:12.252 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:04:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y[36798]: 2024-04-17T07:04:11.987+0000 7fe9ac34c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T07:04:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:12 smithi098 ceph-mon[35418]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:12 smithi098 ceph-mon[35418]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T07:04:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:12 smithi098 ceph-mon[35418]: Standby manager daemon y started 2024-04-17T07:04:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:12 smithi098 ceph-mon[35418]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:04:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:12 smithi098 ceph-mon[35418]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T07:04:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:12 smithi098 ceph-mon[35418]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:04:12.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:12 smithi103 ceph-mon[35495]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:12.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:12 smithi103 ceph-mon[35495]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T07:04:12.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:12 smithi103 ceph-mon[35495]: Standby manager daemon y started 2024-04-17T07:04:12.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:12 smithi103 ceph-mon[35495]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:04:12.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:12 smithi103 ceph-mon[35495]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T07:04:12.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:12 smithi103 ceph-mon[35495]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:04:12.279 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:12 smithi086 ceph-mon[31733]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:12.281 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:12 smithi086 ceph-mon[31733]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-17T07:04:12.281 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:12 smithi086 ceph-mon[31733]: Standby manager daemon y started 2024-04-17T07:04:12.281 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:12 smithi086 ceph-mon[31733]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-17T07:04:12.281 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:12 smithi086 ceph-mon[31733]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-17T07:04:12.281 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:12 smithi086 ceph-mon[31733]: from='mgr.? 172.21.15.98:0/336574969' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-17T07:04:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:13 smithi098 ceph-mon[35418]: mgrmap e15: x(active, since 105s), standbys: z, y 2024-04-17T07:04:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T07:04:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:13 smithi086 ceph-mon[31733]: mgrmap e15: x(active, since 105s), standbys: z, y 2024-04-17T07:04:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T07:04:13.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:13 smithi103 ceph-mon[35495]: mgrmap e15: x(active, since 105s), standbys: z, y 2024-04-17T07:04:13.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-17T07:04:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:14 smithi086 ceph-mon[31733]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:14.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:14 smithi098 ceph-mon[35418]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:14.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:14 smithi103 ceph-mon[35495]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:14.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:14.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:14.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:15.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:15.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:15.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:15.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:16.220 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:16 smithi086 ceph-mon[31733]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:16.404 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:16 smithi098 ceph-mon[35418]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:16 smithi103 ceph-mon[35495]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:17.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-17T07:04:17.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:17.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-17T07:04:17.441 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-17T07:04:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:18.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:18 smithi086 ceph-mon[31733]: Deploying daemon osd.0 on smithi086 2024-04-17T07:04:18.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:18 smithi086 ceph-mon[31733]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:18 smithi098 ceph-mon[35418]: Deploying daemon osd.0 on smithi086 2024-04-17T07:04:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:18 smithi098 ceph-mon[35418]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:18.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:18 smithi103 ceph-mon[35495]: Deploying daemon osd.0 on smithi086 2024-04-17T07:04:18.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:18 smithi103 ceph-mon[35495]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:19.390 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:19 smithi086 ceph-mon[31733]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:19.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:19 smithi098 ceph-mon[35418]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:19.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:19 smithi103 ceph-mon[35495]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:22.226 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:21 smithi086 ceph-mon[31733]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:22.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:21 smithi098 ceph-mon[35418]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:22.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:21 smithi103 ceph-mon[35495]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:24.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:23 smithi086 ceph-mon[31733]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:24.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:23 smithi098 ceph-mon[35418]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:24.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:23 smithi103 ceph-mon[35495]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:25.933 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:25.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:25.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:25.934 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:25 smithi086 ceph-mon[31733]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:25 smithi098 ceph-mon[35418]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:26.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:26.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:26.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:26.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:25 smithi103 ceph-mon[35495]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:27 smithi098 ceph-mon[35418]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:28.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:27 smithi103 ceph-mon[35495]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:28.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:28.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:28.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:28.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:28.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:27 smithi086 ceph-mon[31733]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:29.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:28 smithi098 ceph-mon[35418]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-17T07:04:29.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:28 smithi103 ceph-mon[35495]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-17T07:04:29.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:28 smithi086 ceph-mon[31733]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-17T07:04:29.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.496+0000 7fddb3e00700 1 -- 172.21.15.86:0/4269482584 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fddbc062e90 con 0x7fdda0039890 2024-04-17T07:04:29.496 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 0 on host 'smithi086' 2024-04-17T07:04:29.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.500+0000 7fddc2400700 1 -- 172.21.15.86:0/4269482584 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdda0039890 msgr2=0x7fdda003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:04:29.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.500+0000 7fddc2400700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdda0039890 0x7fdda003bd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdda80038f0 tx=0x7fdda8005fd0 comp rx=0 tx=0).stop 2024-04-17T07:04:29.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.500+0000 7fddc2400700 1 -- 172.21.15.86:0/4269482584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddbc1090f0 msgr2=0x7fddbc19f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:04:29.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.500+0000 7fddc2400700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddbc1090f0 0x7fddbc19f250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fddac00b530 tx=0x7fddac00b8f0 comp rx=0 tx=0).stop 2024-04-17T07:04:29.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.501+0000 7fddc2400700 1 -- 172.21.15.86:0/4269482584 shutdown_connections 2024-04-17T07:04:29.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.501+0000 7fddc2400700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdda0039890 0x7fdda003bd50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:29.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.501+0000 7fddc2400700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddbc10ac90 0x7fddbc1a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:29.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.501+0000 7fddc2400700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddbc1090f0 0x7fddbc19f250 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:29.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.501+0000 7fddc2400700 1 --2- 172.21.15.86:0/4269482584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddbc10a2f0 0x7fddbc19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:29.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.501+0000 7fddc2400700 1 -- 172.21.15.86:0/4269482584 >> 172.21.15.86:0/4269482584 conn(0x7fddbc0780c0 msgr2=0x7fddbc10d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:04:29.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.501+0000 7fddc2400700 1 -- 172.21.15.86:0/4269482584 shutdown_connections 2024-04-17T07:04:29.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:04:29.501+0000 7fddc2400700 1 -- 172.21.15.86:0/4269482584 wait complete. 2024-04-17T07:04:30.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:30 smithi086 ceph-mon[31733]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:30.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:30 smithi086 ceph-mon[31733]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-17T07:04:30.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:30 smithi086 ceph-mon[31733]: osdmap e6: 1 total, 0 up, 1 in 2024-04-17T07:04:30.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:30 smithi086 ceph-mon[31733]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:04:30.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:30.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:30.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:30.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:30 smithi098 ceph-mon[35418]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:30 smithi098 ceph-mon[35418]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-17T07:04:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:30 smithi098 ceph-mon[35418]: osdmap e6: 1 total, 0 up, 1 in 2024-04-17T07:04:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:30 smithi098 ceph-mon[35418]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:04:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:30.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:30.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:30 smithi103 ceph-mon[35495]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:30.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:30 smithi103 ceph-mon[35495]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-17T07:04:30.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:30 smithi103 ceph-mon[35495]: osdmap e6: 1 total, 0 up, 1 in 2024-04-17T07:04:30.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:30 smithi103 ceph-mon[35495]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:04:30.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:30.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:30.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:30.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:30.284 DEBUG:teuthology.orchestra.run.smithi086:osd.0> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.0.service 2024-04-17T07:04:30.289 INFO:tasks.cephadm:Deploying osd.1 on smithi098 with /dev/vg_nvme/lv_4... 2024-04-17T07:04:30.289 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-17T07:04:30.708 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:04:31.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:31 smithi086 ceph-mon[31733]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:04:31.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:31 smithi086 ceph-mon[31733]: osdmap e7: 1 total, 0 up, 1 in 2024-04-17T07:04:31.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:31 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:31.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:31 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:31.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:31 smithi098 ceph-mon[35418]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:04:31.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:31 smithi098 ceph-mon[35418]: osdmap e7: 1 total, 0 up, 1 in 2024-04-17T07:04:31.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:31 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:31.135 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:31 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:31.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:31 smithi103 ceph-mon[35495]: from='osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:04:31.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:31 smithi103 ceph-mon[35495]: osdmap e7: 1 total, 0 up, 1 in 2024-04-17T07:04:31.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:31 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:31.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:31 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:32.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:32 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:04:32.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:32 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:04:32.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:32 smithi086 ceph-mon[31733]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:32.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:32 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:32.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:32 smithi086 ceph-mon[31733]: osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] boot 2024-04-17T07:04:32.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:32 smithi086 ceph-mon[31733]: osdmap e8: 1 total, 1 up, 1 in 2024-04-17T07:04:32.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:32 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:32.156 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:32 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:04:32.156 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:32 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:04:32.156 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:32 smithi098 ceph-mon[35418]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:32.156 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:32 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:32.156 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:32 smithi098 ceph-mon[35418]: osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] boot 2024-04-17T07:04:32.156 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:32 smithi098 ceph-mon[35418]: osdmap e8: 1 total, 1 up, 1 in 2024-04-17T07:04:32.156 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:32 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:32.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:32 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:04:32.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:32 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:04:32.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:32 smithi103 ceph-mon[35495]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-17T07:04:32.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:32 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:32.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:32 smithi103 ceph-mon[35495]: osd.0 [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] boot 2024-04-17T07:04:32.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:32 smithi103 ceph-mon[35495]: osdmap e8: 1 total, 1 up, 1 in 2024-04-17T07:04:32.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:32 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-17T07:04:33.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:33 smithi086 ceph-mon[31733]: osdmap e9: 1 total, 1 up, 1 in 2024-04-17T07:04:33.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:33 smithi098 ceph-mon[35418]: osdmap e9: 1 total, 1 up, 1 in 2024-04-17T07:04:33.455 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-17T07:04:33.455 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-17T07:04:33.455 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-17T07:04:33.455 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-17T07:04:33.455 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0301035 s, 348 MB/s 2024-04-17T07:04:33.455 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-17T07:04:33.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:33 smithi103 ceph-mon[35495]: osdmap e9: 1 total, 1 up, 1 in 2024-04-17T07:04:34.125 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:34 smithi098 ceph-mon[35418]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:34.218 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_4 2024-04-17T07:04:34.225 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:34 smithi086 ceph-mon[31733]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:34 smithi103 ceph-mon[35495]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:34.635 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:04:35.805 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:35 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi086 2024-04-17T07:04:35.805 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:35.805 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:35.806 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:04:35.806 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:35 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi086 to 17089M 2024-04-17T07:04:35.806 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:35.806 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:35.806 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:35.806 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:35.806 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:35 smithi098 ceph-mon[35418]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:36.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:35 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi086 2024-04-17T07:04:36.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:36.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:36.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:04:36.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:35 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi086 to 17089M 2024-04-17T07:04:36.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:36.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:36.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:36.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:36.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:35 smithi103 ceph-mon[35495]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:36.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:35 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi086 2024-04-17T07:04:36.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:36.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:36.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:04:36.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:35 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi086 to 17089M 2024-04-17T07:04:36.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:36.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:36.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:04:36.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:36.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:35 smithi086 ceph-mon[31733]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.624+0000 7febcd400700 1 -- 172.21.15.98:0/1486547846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81087a0 msgr2=0x7febc810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:04:36.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.624+0000 7febcd400700 1 --2- 172.21.15.98:0/1486547846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81087a0 0x7febc810ac10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7febc000b0a0 tx=0x7febc000b3b0 comp rx=0 tx=0).stop 2024-04-17T07:04:36.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.624+0000 7febcd400700 1 -- 172.21.15.98:0/1486547846 shutdown_connections 2024-04-17T07:04:36.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.624+0000 7febcd400700 1 --2- 172.21.15.98:0/1486547846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81087a0 0x7febc810ac10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:36.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.624+0000 7febcd400700 1 --2- 172.21.15.98:0/1486547846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8103540 0x7febc8105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:36.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.624+0000 7febcd400700 1 --2- 172.21.15.98:0/1486547846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8105e70 0x7febc8108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:36.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.624+0000 7febcd400700 1 -- 172.21.15.98:0/1486547846 >> 172.21.15.98:0/1486547846 conn(0x7febc80fcf50 msgr2=0x7febc80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:04:36.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.624+0000 7febcd400700 1 -- 172.21.15.98:0/1486547846 shutdown_connections 2024-04-17T07:04:36.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.624+0000 7febcd400700 1 -- 172.21.15.98:0/1486547846 wait complete. 2024-04-17T07:04:36.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.625+0000 7febcd400700 1 Processor -- start 2024-04-17T07:04:36.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.625+0000 7febcd400700 1 -- start start 2024-04-17T07:04:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febcd400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8103540 0x7febc8102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:04:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febcd400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8105e70 0x7febc81014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:04:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febcd400700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81087a0 0x7febc8101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:04:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febcd400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febc810e590 con 0x7febc8103540 2024-04-17T07:04:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febcd400700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7febc810e410 con 0x7febc81087a0 2024-04-17T07:04:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febcd400700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febc810e710 con 0x7febc8105e70 2024-04-17T07:04:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febc7200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81087a0 0x7febc8101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:04:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febc7200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81087a0 0x7febc8101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48904/0 (socket says 172.21.15.98:48904) 2024-04-17T07:04:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febc7200700 1 -- 172.21.15.98:0/1195478658 learned_addr learned my addr 172.21.15.98:0/1195478658 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-17T07:04:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febc5e00700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8105e70 0x7febc81014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:04:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.626+0000 7febc6800700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8103540 0x7febc8102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:04:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febc7200700 1 -- 172.21.15.98:0/1195478658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8105e70 msgr2=0x7febc81014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:04:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febc7200700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8105e70 0x7febc81014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febc7200700 1 -- 172.21.15.98:0/1195478658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8103540 msgr2=0x7febc8102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:04:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febc7200700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8103540 0x7febc8102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:04:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febc7200700 1 -- 172.21.15.98:0/1195478658 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc81022e0 con 0x7febc81087a0 2024-04-17T07:04:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febc7200700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81087a0 0x7febc8101a30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7febc0009fd0 tx=0x7febc0007770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:04:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febbf400700 1 -- 172.21.15.98:0/1195478658 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0021030 con 0x7febc81087a0 2024-04-17T07:04:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febcd400700 1 -- 172.21.15.98:0/1195478658 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc81a2d00 con 0x7febc81087a0 2024-04-17T07:04:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febbf400700 1 -- 172.21.15.98:0/1195478658 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc0007d20 con 0x7febc81087a0 2024-04-17T07:04:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febbf400700 1 -- 172.21.15.98:0/1195478658 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc001d470 con 0x7febc81087a0 2024-04-17T07:04:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.627+0000 7febcd400700 1 -- 172.21.15.98:0/1195478658 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc81a31b0 con 0x7febc81087a0 2024-04-17T07:04:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.628+0000 7febcd400700 1 -- 172.21.15.98:0/1195478658 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc8069740 con 0x7febc81087a0 2024-04-17T07:04:36.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.632+0000 7febbf400700 1 -- 172.21.15.98:0/1195478658 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7febc0024070 con 0x7febc81087a0 2024-04-17T07:04:36.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.633+0000 7febbf400700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febac0a3cb0 0x7febac0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:04:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.634+0000 7febbf400700 1 -- 172.21.15.98:0/1195478658 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7febc000f020 con 0x7febc81087a0 2024-04-17T07:04:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.634+0000 7febc6800700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febac0a3cb0 0x7febac0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:04:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.635+0000 7febc6800700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febac0a3cb0 0x7febac0a6170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7febb8005ea0 tx=0x7febb8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:04:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.637+0000 7febbf400700 1 -- 172.21.15.98:0/1195478658 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7febc009e1a0 con 0x7febc81087a0 2024-04-17T07:04:36.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:04:36.872+0000 7febcd400700 1 -- 172.21.15.98:0/1195478658 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7febc8062e90 con 0x7febac0a3cb0 2024-04-17T07:04:37.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:04:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:04:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:37.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:04:37.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:04:37.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:04:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:04:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:38.099 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:37 smithi098 ceph-mon[35418]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:04:38.099 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:37 smithi098 ceph-mon[35418]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:38.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:37 smithi103 ceph-mon[35495]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:04:38.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:37 smithi103 ceph-mon[35495]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:38.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:37 smithi086 ceph-mon[31733]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:04:38.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:37 smithi086 ceph-mon[31733]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:40.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:39 smithi098 ceph-mon[35418]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:40.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:39 smithi103 ceph-mon[35495]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:40.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:39 smithi086 ceph-mon[31733]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:41.235 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:40 smithi098 ceph-mon[35418]: from='client.? 172.21.15.98:0/1583367254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ded7c4aa-ef6b-4065-9f7f-b26aca9c1299"}]: dispatch 2024-04-17T07:04:41.236 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:40 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ded7c4aa-ef6b-4065-9f7f-b26aca9c1299"}]: dispatch 2024-04-17T07:04:41.236 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:40 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ded7c4aa-ef6b-4065-9f7f-b26aca9c1299"}]': finished 2024-04-17T07:04:41.236 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:40 smithi098 ceph-mon[35418]: osdmap e10: 2 total, 1 up, 2 in 2024-04-17T07:04:41.236 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:04:41.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:40 smithi103 ceph-mon[35495]: from='client.? 172.21.15.98:0/1583367254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ded7c4aa-ef6b-4065-9f7f-b26aca9c1299"}]: dispatch 2024-04-17T07:04:41.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:40 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ded7c4aa-ef6b-4065-9f7f-b26aca9c1299"}]: dispatch 2024-04-17T07:04:41.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:40 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ded7c4aa-ef6b-4065-9f7f-b26aca9c1299"}]': finished 2024-04-17T07:04:41.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:40 smithi103 ceph-mon[35495]: osdmap e10: 2 total, 1 up, 2 in 2024-04-17T07:04:41.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:04:41.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:40 smithi086 ceph-mon[31733]: from='client.? 172.21.15.98:0/1583367254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ded7c4aa-ef6b-4065-9f7f-b26aca9c1299"}]: dispatch 2024-04-17T07:04:41.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:40 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ded7c4aa-ef6b-4065-9f7f-b26aca9c1299"}]: dispatch 2024-04-17T07:04:41.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:40 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ded7c4aa-ef6b-4065-9f7f-b26aca9c1299"}]': finished 2024-04-17T07:04:41.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:40 smithi086 ceph-mon[31733]: osdmap e10: 2 total, 1 up, 2 in 2024-04-17T07:04:41.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:04:42.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:41 smithi098 ceph-mon[35418]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:42.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:41 smithi098 ceph-mon[35418]: from='client.? 172.21.15.98:0/1635260651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:04:42.278 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:41 smithi103 ceph-mon[35495]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:41 smithi103 ceph-mon[35495]: from='client.? 172.21.15.98:0/1635260651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:04:42.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:41 smithi086 ceph-mon[31733]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:42.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:41 smithi086 ceph-mon[31733]: from='client.? 172.21.15.98:0/1635260651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:04:44.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:43 smithi098 ceph-mon[35418]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:44.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:43 smithi103 ceph-mon[35495]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:44.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:43 smithi086 ceph-mon[31733]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:46.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:46 smithi098 ceph-mon[35418]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:46.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:46 smithi103 ceph-mon[35495]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:46 smithi086 ceph-mon[31733]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:48.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:48 smithi098 ceph-mon[35418]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:48 smithi103 ceph-mon[35495]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:48.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:48 smithi086 ceph-mon[31733]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:50 smithi103 ceph-mon[35495]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-17T07:04:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:50.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:50 smithi086 ceph-mon[31733]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-17T07:04:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:50.422 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:50 smithi098 ceph-mon[35418]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:50.422 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-17T07:04:50.422 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:04:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:51 smithi103 ceph-mon[35495]: Deploying daemon osd.1 on smithi098 2024-04-17T07:04:51.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:51 smithi086 ceph-mon[31733]: Deploying daemon osd.1 on smithi098 2024-04-17T07:04:51.438 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:51 smithi098 ceph-mon[35418]: Deploying daemon osd.1 on smithi098 2024-04-17T07:04:52.235 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:52 smithi098 ceph-mon[35418]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:52.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:52 smithi086 ceph-mon[31733]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:52 smithi103 ceph-mon[35495]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:54 smithi098 ceph-mon[35418]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:54.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:54 smithi086 ceph-mon[31733]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:54 smithi103 ceph-mon[35495]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:56.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:56 smithi098 ceph-mon[35418]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:56.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:56 smithi086 ceph-mon[31733]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:56 smithi103 ceph-mon[35495]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:58.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:58 smithi086 ceph-mon[31733]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:58.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:58 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:58.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:58 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:58.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:04:58 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:58.395 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:58 smithi098 ceph-mon[35418]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:58.396 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:58 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:58.396 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:58 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:58.396 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:04:58 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:04:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:58 smithi103 ceph-mon[35495]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:04:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:58 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:58 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:04:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:04:58 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:05:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:00 smithi098 ceph-mon[35418]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:00 smithi086 ceph-mon[31733]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:00 smithi103 ceph-mon[35495]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:01.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.480+0000 7febbf400700 1 -- 172.21.15.98:0/1195478658 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7febc8062e90 con 0x7febac0a3cb0 2024-04-17T07:05:01.480 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 1 on host 'smithi098' 2024-04-17T07:05:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.483+0000 7febcd400700 1 -- 172.21.15.98:0/1195478658 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febac0a3cb0 msgr2=0x7febac0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:05:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.483+0000 7febcd400700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febac0a3cb0 0x7febac0a6170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7febb8005ea0 tx=0x7febb8005e30 comp rx=0 tx=0).stop 2024-04-17T07:05:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.483+0000 7febcd400700 1 -- 172.21.15.98:0/1195478658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81087a0 msgr2=0x7febc8101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:05:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.483+0000 7febcd400700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81087a0 0x7febc8101a30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7febc0009fd0 tx=0x7febc0007770 comp rx=0 tx=0).stop 2024-04-17T07:05:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.483+0000 7febcd400700 1 -- 172.21.15.98:0/1195478658 shutdown_connections 2024-04-17T07:05:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.483+0000 7febcd400700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febac0a3cb0 0x7febac0a6170 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.483+0000 7febcd400700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81087a0 0x7febc8101a30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.483+0000 7febcd400700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc8103540 0x7febc8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.483+0000 7febcd400700 1 --2- 172.21.15.98:0/1195478658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8105e70 0x7febc81014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.483+0000 7febcd400700 1 -- 172.21.15.98:0/1195478658 >> 172.21.15.98:0/1195478658 conn(0x7febc80fcf50 msgr2=0x7febc8103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:05:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.484+0000 7febcd400700 1 -- 172.21.15.98:0/1195478658 shutdown_connections 2024-04-17T07:05:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:05:01.484+0000 7febcd400700 1 -- 172.21.15.98:0/1195478658 wait complete. 2024-04-17T07:05:01.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:01.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: from='osd.1 [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:05:02.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:02.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:02.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: from='osd.1 [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:05:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:05:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:05:02.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:02.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:02.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:02.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:02.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:02.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:02.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:02.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:02.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: from='osd.1 [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:05:02.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-17T07:05:02.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:05:02.285 DEBUG:teuthology.orchestra.run.smithi098:osd.1> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.1.service 2024-04-17T07:05:02.290 INFO:tasks.cephadm:Deploying osd.2 on smithi103 with /dev/vg_nvme/lv_4... 2024-04-17T07:05:02.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-17T07:05:02.738 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:05:03.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:03 smithi098 ceph-mon[35418]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-17T07:05:03.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:03 smithi098 ceph-mon[35418]: from='osd.1 [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:05:03.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:03 smithi098 ceph-mon[35418]: osdmap e11: 2 total, 1 up, 2 in 2024-04-17T07:05:03.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:03.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:03 smithi098 ceph-mon[35418]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:05:03.653 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:03 smithi098 ceph-mon[35418]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:03.653 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:05:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1[40012]: 2024-04-17T07:05:03.468+0000 7f89bac00700 -1 osd.1 0 waiting for initial osdmap 2024-04-17T07:05:03.653 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:05:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1[40012]: 2024-04-17T07:05:03.477+0000 7f89b5a00700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:05:03.707 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:03 smithi103 ceph-mon[35495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-17T07:05:03.707 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:03 smithi103 ceph-mon[35495]: from='osd.1 [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:05:03.707 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:03 smithi103 ceph-mon[35495]: osdmap e11: 2 total, 1 up, 2 in 2024-04-17T07:05:03.707 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:03.707 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:03 smithi103 ceph-mon[35495]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:05:03.707 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:03 smithi103 ceph-mon[35495]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:03 smithi086 ceph-mon[31733]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-17T07:05:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:03 smithi086 ceph-mon[31733]: from='osd.1 [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:05:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:03 smithi086 ceph-mon[31733]: osdmap e11: 2 total, 1 up, 2 in 2024-04-17T07:05:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:03 smithi086 ceph-mon[31733]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:05:03.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:03 smithi086 ceph-mon[31733]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:04 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:05:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:04 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:05:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:04 smithi098 ceph-mon[35418]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:05:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:04 smithi098 ceph-mon[35418]: osdmap e12: 2 total, 1 up, 2 in 2024-04-17T07:05:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:04 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:04 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:04.591 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:04 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:05:04.591 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:04 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:05:04.591 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:04 smithi103 ceph-mon[35495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:05:04.591 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:04 smithi103 ceph-mon[35495]: osdmap e12: 2 total, 1 up, 2 in 2024-04-17T07:05:04.591 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:04 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:04.591 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:04 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:04.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:04 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:05:04.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:04 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:05:04.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:04 smithi086 ceph-mon[31733]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:05:04.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:04 smithi086 ceph-mon[31733]: osdmap e12: 2 total, 1 up, 2 in 2024-04-17T07:05:04.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:04 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:04.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:04 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:05.502 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-17T07:05:05.503 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-17T07:05:05.503 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-17T07:05:05.503 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-17T07:05:05.503 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0340793 s, 308 MB/s 2024-04-17T07:05:05.503 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-17T07:05:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:05 smithi098 ceph-mon[35418]: osd.1 [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] boot 2024-04-17T07:05:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:05 smithi098 ceph-mon[35418]: osdmap e13: 2 total, 2 up, 2 in 2024-04-17T07:05:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:05 smithi098 ceph-mon[35418]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:05 smithi103 ceph-mon[35495]: osd.1 [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] boot 2024-04-17T07:05:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:05 smithi103 ceph-mon[35495]: osdmap e13: 2 total, 2 up, 2 in 2024-04-17T07:05:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:05 smithi103 ceph-mon[35495]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:05 smithi086 ceph-mon[31733]: osd.1 [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] boot 2024-04-17T07:05:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:05 smithi086 ceph-mon[31733]: osdmap e13: 2 total, 2 up, 2 in 2024-04-17T07:05:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-17T07:05:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:05 smithi086 ceph-mon[31733]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-17T07:05:06.234 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-04-17T07:05:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:06 smithi098 ceph-mon[35418]: osdmap e14: 2 total, 2 up, 2 in 2024-04-17T07:05:06.676 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:05:06.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:06 smithi103 ceph-mon[35495]: osdmap e14: 2 total, 2 up, 2 in 2024-04-17T07:05:06.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:06 smithi086 ceph-mon[31733]: osdmap e14: 2 total, 2 up, 2 in 2024-04-17T07:05:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:07 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi098 2024-04-17T07:05:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:05:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:07 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi098 to 17089M 2024-04-17T07:05:07.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:07.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:07.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:07.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:07.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:07 smithi103 ceph-mon[35495]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:07 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi098 2024-04-17T07:05:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:05:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:07 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi098 to 17089M 2024-04-17T07:05:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:08.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:07 smithi098 ceph-mon[35418]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:08.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:07 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi098 2024-04-17T07:05:08.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:08.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:08.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:05:08.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:07 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi098 to 17089M 2024-04-17T07:05:08.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:08.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:08.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:08.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:08.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:07 smithi086 ceph-mon[31733]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.414+0000 7f41d9600700 1 -- 172.21.15.103:0/3949024112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41d4102ab0 msgr2=0x7f41d4102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:05:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.414+0000 7f41d9600700 1 --2- 172.21.15.103:0/3949024112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41d4102ab0 0x7f41d4102ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009990 tx=0x7f41c4009ca0 comp rx=0 tx=0).stop 2024-04-17T07:05:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.414+0000 7f41d9600700 1 -- 172.21.15.103:0/3949024112 shutdown_connections 2024-04-17T07:05:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.414+0000 7f41d9600700 1 --2- 172.21.15.103:0/3949024112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4104650 0x7f41d410af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.414+0000 7f41d9600700 1 --2- 172.21.15.103:0/3949024112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41d4102ab0 0x7f41d4102ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.414+0000 7f41d9600700 1 --2- 172.21.15.103:0/3949024112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41d4103cb0 0x7f41d4104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.414+0000 7f41d9600700 1 -- 172.21.15.103:0/3949024112 >> 172.21.15.103:0/3949024112 conn(0x7f41d40fe050 msgr2=0x7f41d4100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:05:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.414+0000 7f41d9600700 1 -- 172.21.15.103:0/3949024112 shutdown_connections 2024-04-17T07:05:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.414+0000 7f41d9600700 1 -- 172.21.15.103:0/3949024112 wait complete. 2024-04-17T07:05:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.416+0000 7f41d9600700 1 Processor -- start 2024-04-17T07:05:08.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.416+0000 7f41d9600700 1 -- start start 2024-04-17T07:05:08.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.416+0000 7f41d9600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4102ab0 0x7f41d419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.416+0000 7f41d9600700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41d4103cb0 0x7f41d419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.416+0000 7f41d9600700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41d4104650 0x7f41d41a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.416+0000 7f41d9600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41d410e520 con 0x7f41d4102ab0 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.416+0000 7f41d9600700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f41d410e3a0 con 0x7f41d4103cb0 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.416+0000 7f41d9600700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41d410e6a0 con 0x7f41d4104650 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d3400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41d4104650 0x7f41d41a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d3400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41d4104650 0x7f41d41a2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55854/0 (socket says 172.21.15.103:55854) 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d3400700 1 -- 172.21.15.103:0/2608071065 learned_addr learned my addr 172.21.15.103:0/2608071065 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d2a00700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4102ab0 0x7f41d419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d2000700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41d4103cb0 0x7f41d419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:05:08.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d3400700 1 -- 172.21.15.103:0/2608071065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41d4103cb0 msgr2=0x7f41d419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:05:08.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d3400700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41d4103cb0 0x7f41d419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:08.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d3400700 1 -- 172.21.15.103:0/2608071065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4102ab0 msgr2=0x7f41d419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:05:08.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d3400700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4102ab0 0x7f41d419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:08.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d3400700 1 -- 172.21.15.103:0/2608071065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d41a2a00 con 0x7f41d4104650 2024-04-17T07:05:08.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.417+0000 7f41d3400700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41d4104650 0x7f41d41a2360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00add0 tx=0x7f41cc00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:05:08.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.418+0000 7f41cb400700 1 -- 172.21.15.103:0/2608071065 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc017020 con 0x7f41d4104650 2024-04-17T07:05:08.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.418+0000 7f41cb400700 1 -- 172.21.15.103:0/2608071065 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41cc00cdf0 con 0x7f41d4104650 2024-04-17T07:05:08.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.418+0000 7f41d9600700 1 -- 172.21.15.103:0/2608071065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d41a2cf0 con 0x7f41d4104650 2024-04-17T07:05:08.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.418+0000 7f41cb400700 1 -- 172.21.15.103:0/2608071065 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc005460 con 0x7f41d4104650 2024-04-17T07:05:08.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.418+0000 7f41d9600700 1 -- 172.21.15.103:0/2608071065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d41a3180 con 0x7f41d4104650 2024-04-17T07:05:08.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.419+0000 7f41d9600700 1 -- 172.21.15.103:0/2608071065 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d4069740 con 0x7f41d4104650 2024-04-17T07:05:08.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.421+0000 7f41cb400700 1 -- 172.21.15.103:0/2608071065 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f41cc004ad0 con 0x7f41d4104650 2024-04-17T07:05:08.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.423+0000 7f41cb400700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41b80a3bb0 0x7f41b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:05:08.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.423+0000 7f41cb400700 1 -- 172.21.15.103:0/2608071065 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f41cc0ce9e0 con 0x7f41d4104650 2024-04-17T07:05:08.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.424+0000 7f41d2a00700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41b80a3bb0 0x7f41b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:05:08.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.425+0000 7f41d2a00700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41b80a3bb0 0x7f41b80a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41c4003920 tx=0x7f41c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:05:08.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.427+0000 7f41cb400700 1 -- 172.21.15.103:0/2608071065 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f41cc021020 con 0x7f41d4104650 2024-04-17T07:05:08.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:08.670+0000 7f41d9600700 1 -- 172.21.15.103:0/2608071065 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f41d40763a0 con 0x7f41b80a3bb0 2024-04-17T07:05:09.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:05:09.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:05:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:09.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:05:09.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:05:09.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:09.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:05:09.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:05:09.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:09.889 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:09 smithi103 ceph-mon[35495]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:05:09.889 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:09 smithi103 ceph-mon[35495]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:10.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:09 smithi098 ceph-mon[35418]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:05:10.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:09 smithi098 ceph-mon[35418]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:10.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:09 smithi086 ceph-mon[31733]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:05:10.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:09 smithi086 ceph-mon[31733]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:12.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:11 smithi098 ceph-mon[35418]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:12.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:11 smithi103 ceph-mon[35495]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:12.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:11 smithi086 ceph-mon[31733]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/596112604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29d6fda2-deeb-4a3f-aa06-16bec72d0cc7"}]: dispatch 2024-04-17T07:05:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:12 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29d6fda2-deeb-4a3f-aa06-16bec72d0cc7"}]: dispatch 2024-04-17T07:05:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:12 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29d6fda2-deeb-4a3f-aa06-16bec72d0cc7"}]': finished 2024-04-17T07:05:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:12 smithi098 ceph-mon[35418]: osdmap e15: 3 total, 2 up, 3 in 2024-04-17T07:05:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:12 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:13.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/596112604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29d6fda2-deeb-4a3f-aa06-16bec72d0cc7"}]: dispatch 2024-04-17T07:05:13.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:12 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29d6fda2-deeb-4a3f-aa06-16bec72d0cc7"}]: dispatch 2024-04-17T07:05:13.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:12 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29d6fda2-deeb-4a3f-aa06-16bec72d0cc7"}]': finished 2024-04-17T07:05:13.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:12 smithi103 ceph-mon[35495]: osdmap e15: 3 total, 2 up, 3 in 2024-04-17T07:05:13.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:12 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/596112604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29d6fda2-deeb-4a3f-aa06-16bec72d0cc7"}]: dispatch 2024-04-17T07:05:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:12 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29d6fda2-deeb-4a3f-aa06-16bec72d0cc7"}]: dispatch 2024-04-17T07:05:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:12 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29d6fda2-deeb-4a3f-aa06-16bec72d0cc7"}]': finished 2024-04-17T07:05:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:12 smithi086 ceph-mon[31733]: osdmap e15: 3 total, 2 up, 3 in 2024-04-17T07:05:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:12 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:14.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:13 smithi098 ceph-mon[35418]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:14.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:13 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/4089256887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:05:14.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:13 smithi103 ceph-mon[35495]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:14.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:13 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/4089256887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:05:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:13 smithi086 ceph-mon[31733]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:13 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/4089256887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:05:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:15 smithi098 ceph-mon[35418]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:16.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:15 smithi103 ceph-mon[35495]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:16.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:15 smithi086 ceph-mon[31733]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:18.257 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:18 smithi103 ceph-mon[35495]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:18.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:18 smithi086 ceph-mon[31733]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:18.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:18 smithi098 ceph-mon[35418]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:20.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:20 smithi103 ceph-mon[35495]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:20 smithi086 ceph-mon[31733]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:20.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:20 smithi098 ceph-mon[35418]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:22.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:22 smithi103 ceph-mon[35495]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:22.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:22 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-17T07:05:22.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:22 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:22 smithi086 ceph-mon[31733]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-17T07:05:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:22.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:22 smithi098 ceph-mon[35418]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:22.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:22 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-17T07:05:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:22 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:23.213 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:23 smithi103 ceph-mon[35495]: Deploying daemon osd.2 on smithi103 2024-04-17T07:05:23.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:23 smithi086 ceph-mon[31733]: Deploying daemon osd.2 on smithi103 2024-04-17T07:05:23.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:23 smithi098 ceph-mon[35418]: Deploying daemon osd.2 on smithi103 2024-04-17T07:05:24.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:24 smithi103 ceph-mon[35495]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:24.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:24 smithi086 ceph-mon[31733]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:24 smithi098 ceph-mon[35418]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:26.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:26 smithi086 ceph-mon[31733]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:26.387 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:26 smithi103 ceph-mon[35495]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:26.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:26 smithi098 ceph-mon[35418]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:28.267 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:28 smithi103 ceph-mon[35495]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:28.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:28 smithi086 ceph-mon[31733]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:28.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:28 smithi098 ceph-mon[35418]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:30.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:30 smithi103 ceph-mon[35495]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:30.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:30.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:30.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:05:30.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:30 smithi098 ceph-mon[35418]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:05:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:30 smithi086 ceph-mon[31733]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:05:31.331 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:31 smithi103 ceph-mon[35495]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:31.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:31 smithi098 ceph-mon[35418]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:31 smithi086 ceph-mon[31733]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:33.093 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:33.093 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:33.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:33.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:33.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.678+0000 7f41cb400700 1 -- 172.21.15.103:0/2608071065 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f41d40763a0 con 0x7f41b80a3bb0 2024-04-17T07:05:33.679 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 2 on host 'smithi103' 2024-04-17T07:05:33.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 -- 172.21.15.103:0/2608071065 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41b80a3bb0 msgr2=0x7f41b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41b80a3bb0 0x7f41b80a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41c4003920 tx=0x7f41c4005fb0 comp rx=0 tx=0).stop 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 -- 172.21.15.103:0/2608071065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41d4104650 msgr2=0x7f41d41a2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41d4104650 0x7f41d41a2360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00add0 tx=0x7f41cc00c330 comp rx=0 tx=0).stop 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 -- 172.21.15.103:0/2608071065 shutdown_connections 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41b80a3bb0 0x7f41b80a6070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41d4104650 0x7f41d41a2360 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41d4102ab0 0x7f41d419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 --2- 172.21.15.103:0/2608071065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41d4103cb0 0x7f41d419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 -- 172.21.15.103:0/2608071065 >> 172.21.15.103:0/2608071065 conn(0x7f41d40fe050 msgr2=0x7f41d4074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 -- 172.21.15.103:0/2608071065 shutdown_connections 2024-04-17T07:05:33.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:05:33.682+0000 7f41d9600700 1 -- 172.21.15.103:0/2608071065 wait complete. 2024-04-17T07:05:34.474 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:34 smithi103 ceph-mon[35495]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:34.475 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:34 smithi103 ceph-mon[35495]: from='osd.2 [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:05:34.475 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:34 smithi103 ceph-mon[35495]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:05:34.475 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:34 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:34.475 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:34 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:34.475 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:34 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:34.475 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:34 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:34.475 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:34 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:34.475 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:34 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:05:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:34 smithi098 ceph-mon[35418]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:34 smithi098 ceph-mon[35418]: from='osd.2 [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:05:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:34 smithi098 ceph-mon[35418]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:05:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:34 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:34 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:34 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:34 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:34 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:34 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:05:34.540 DEBUG:teuthology.orchestra.run.smithi103:osd.2> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.2.service 2024-04-17T07:05:34.544 INFO:tasks.cephadm:Deploying osd.3 on smithi086 with /dev/vg_nvme/lv_3... 2024-04-17T07:05:34.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-17T07:05:34.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:34 smithi086 ceph-mon[31733]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:34 smithi086 ceph-mon[31733]: from='osd.2 [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:05:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:34 smithi086 ceph-mon[31733]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-17T07:05:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:34 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:34 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:34 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:34 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:34 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:34 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:05:34.987 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:05:35.423 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:35 smithi086 ceph-mon[31733]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-17T07:05:35.423 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:35 smithi086 ceph-mon[31733]: osdmap e16: 3 total, 2 up, 3 in 2024-04-17T07:05:35.423 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:35.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:35 smithi086 ceph-mon[31733]: from='osd.2 [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:05:35.424 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:35 smithi086 ceph-mon[31733]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:05:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:35 smithi098 ceph-mon[35418]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-17T07:05:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:35 smithi098 ceph-mon[35418]: osdmap e16: 3 total, 2 up, 3 in 2024-04-17T07:05:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:35 smithi098 ceph-mon[35418]: from='osd.2 [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:05:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:35 smithi098 ceph-mon[35418]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:05:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:35 smithi103 ceph-mon[35495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-17T07:05:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:35 smithi103 ceph-mon[35495]: osdmap e16: 3 total, 2 up, 3 in 2024-04-17T07:05:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:35 smithi103 ceph-mon[35495]: from='osd.2 [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:05:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:35 smithi103 ceph-mon[35495]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:05:35.527 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:05:35 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2[40737]: 2024-04-17T07:05:35.132+0000 7f4770a00700 -1 osd.2 0 waiting for initial osdmap 2024-04-17T07:05:35.527 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:05:35 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2[40737]: 2024-04-17T07:05:35.141+0000 7f476b800700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:05:36.250 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:36 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:05:36.250 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:36 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:05:36.250 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:36 smithi103 ceph-mon[35495]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:36.250 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:36 smithi103 ceph-mon[35495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:05:36.250 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:36 smithi103 ceph-mon[35495]: osdmap e17: 3 total, 2 up, 3 in 2024-04-17T07:05:36.250 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:36.250 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:36 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:05:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:36 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:05:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:36 smithi098 ceph-mon[35418]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:36 smithi098 ceph-mon[35418]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:05:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:36 smithi098 ceph-mon[35418]: osdmap e17: 3 total, 2 up, 3 in 2024-04-17T07:05:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:36.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:36 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:05:36.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:36 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:05:36.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:36 smithi086 ceph-mon[31733]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-17T07:05:36.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:36 smithi086 ceph-mon[31733]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:05:36.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:36 smithi086 ceph-mon[31733]: osdmap e17: 3 total, 2 up, 3 in 2024-04-17T07:05:36.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:36.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:37 smithi098 ceph-mon[35418]: osd.2 [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] boot 2024-04-17T07:05:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:37 smithi098 ceph-mon[35418]: osdmap e18: 3 total, 3 up, 3 in 2024-04-17T07:05:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:37 smithi098 ceph-mon[35418]: pgmap v103: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:05:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:37 smithi103 ceph-mon[35495]: osd.2 [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] boot 2024-04-17T07:05:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:37 smithi103 ceph-mon[35495]: osdmap e18: 3 total, 3 up, 3 in 2024-04-17T07:05:37.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:37.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:37 smithi103 ceph-mon[35495]: pgmap v103: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:37.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:05:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:37 smithi086 ceph-mon[31733]: osd.2 [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] boot 2024-04-17T07:05:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:37 smithi086 ceph-mon[31733]: osdmap e18: 3 total, 3 up, 3 in 2024-04-17T07:05:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-17T07:05:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:37 smithi086 ceph-mon[31733]: pgmap v103: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:05:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-17T07:05:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-17T07:05:37.811 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-17T07:05:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-17T07:05:37.812 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0338466 s, 310 MB/s 2024-04-17T07:05:37.812 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-17T07:05:38.460 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:38 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-17T07:05:38.460 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:38 smithi086 ceph-mon[31733]: osdmap e19: 3 total, 3 up, 3 in 2024-04-17T07:05:38.460 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:38 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:05:38.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:38 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-17T07:05:38.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:38 smithi098 ceph-mon[35418]: osdmap e19: 3 total, 3 up, 3 in 2024-04-17T07:05:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:38 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:05:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:38 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-17T07:05:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:38 smithi103 ceph-mon[35495]: osdmap e19: 3 total, 3 up, 3 in 2024-04-17T07:05:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:38 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-17T07:05:38.697 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-04-17T07:05:38.822 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48374]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:05:38.823 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48374]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:38.823 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48374]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:38.823 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48374]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:38.823 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48377]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:05:38.823 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48377]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:38.823 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48377]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:38.823 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48377]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:05:38 smithi098 sudo[44088]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:05:39.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:05:38 smithi098 sudo[44088]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:39.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:05:38 smithi098 sudo[44088]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:39.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:05:38 smithi098 sudo[44088]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:05:38 smithi098 sudo[44091]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:05:39.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:05:38 smithi098 sudo[44091]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:39.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:05:38 smithi098 sudo[44091]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:39.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:05:38 smithi098 sudo[44091]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.026 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:05:38 smithi103 sudo[44419]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:05:39.026 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:05:38 smithi103 sudo[44419]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:39.026 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:05:38 smithi103 sudo[44419]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:39.026 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:05:38 smithi103 sudo[44419]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.026 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:05:38 smithi103 sudo[44422]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:05:39.027 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:05:38 smithi103 sudo[44422]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:39.027 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:05:38 smithi103 sudo[44422]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:39.027 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:05:38 smithi103 sudo[44422]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48398]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:05:39.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48398]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:39.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:38 smithi086 sudo[48398]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:39.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 sudo[48398]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 sudo[48403]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:05:39.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 sudo[48403]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:39.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 sudo[48403]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:39.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 sudo[48403]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.174 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:05:39.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-17T07:05:39.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: osdmap e20: 3 total, 3 up, 3 in 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: pgmap v106: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:05:39.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:05:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 sudo[44094]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:05:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 sudo[44094]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 sudo[44094]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 sudo[44094]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 sudo[44097]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:05:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 sudo[44097]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 sudo[44097]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 sudo[44097]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: osdmap e20: 3 total, 3 up, 3 in 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: pgmap v106: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:05:39.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 sudo[44425]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 sudo[44425]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 sudo[44425]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 sudo[44425]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 sudo[44428]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 sudo[44428]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 sudo[44428]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 sudo[44428]: pam_unix(sudo:session): session closed for user root 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: osdmap e20: 3 total, 3 up, 3 in 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:39.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: pgmap v106: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:39.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:39.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:05:39.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:05:39.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:05:39.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:05:39.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:05:39.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:05:39.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:05:39.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi103 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: osdmap e21: 3 total, 3 up, 3 in 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:05:40.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi103 to 17089M 2024-04-17T07:05:40.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:40.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:40.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:40 smithi103 ceph-mon[35495]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-17T07:05:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:05:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi103 2024-04-17T07:05:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:05:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:05:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: osdmap e21: 3 total, 3 up, 3 in 2024-04-17T07:05:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:05:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:05:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:05:40.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:05:40.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:05:40.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi103 to 17089M 2024-04-17T07:05:40.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:40.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:40.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:40 smithi086 ceph-mon[31733]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-17T07:05:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:05:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi103 2024-04-17T07:05:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:05:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-17T07:05:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: osdmap e21: 3 total, 3 up, 3 in 2024-04-17T07:05:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-17T07:05:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi103 to 17089M 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:05:40.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:40 smithi098 ceph-mon[35418]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-17T07:05:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.350+0000 7f0294a00700 1 -- 172.21.15.86:0/3242152936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0290076a80 msgr2=0x7f0290074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:05:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.350+0000 7f0294a00700 1 --2- 172.21.15.86:0/3242152936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0290076a80 0x7f0290074ee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0280009990 tx=0x7f0280009ca0 comp rx=0 tx=0).stop 2024-04-17T07:05:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.350+0000 7f0294a00700 1 -- 172.21.15.86:0/3242152936 shutdown_connections 2024-04-17T07:05:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.350+0000 7f0294a00700 1 --2- 172.21.15.86:0/3242152936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f029010f090 0x7f0290111480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.350+0000 7f0294a00700 1 --2- 172.21.15.86:0/3242152936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0290076a80 0x7f0290074ee0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.350+0000 7f0294a00700 1 --2- 172.21.15.86:0/3242152936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02900754b0 0x7f0290075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.350+0000 7f0294a00700 1 -- 172.21.15.86:0/3242152936 >> 172.21.15.86:0/3242152936 conn(0x7f02900fe050 msgr2=0x7f0290100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:05:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.350+0000 7f0294a00700 1 -- 172.21.15.86:0/3242152936 shutdown_connections 2024-04-17T07:05:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.350+0000 7f0294a00700 1 -- 172.21.15.86:0/3242152936 wait complete. 2024-04-17T07:05:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.351+0000 7f0294a00700 1 Processor -- start 2024-04-17T07:05:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.351+0000 7f0294a00700 1 -- start start 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.352+0000 7f0294a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900754b0 0x7f029019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.352+0000 7f0294a00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0290076a80 0x7f029019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.352+0000 7f0294a00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f029010f090 0x7f02901a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.352+0000 7f0294a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0290114c90 con 0x7f02900754b0 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.352+0000 7f0294a00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0290114b10 con 0x7f029010f090 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.352+0000 7f0294a00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0290114e10 con 0x7f0290076a80 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.352+0000 7f028de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900754b0 0x7f029019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.352+0000 7f028de00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900754b0 0x7f029019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50426/0 (socket says 172.21.15.86:50426) 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.352+0000 7f028de00700 1 -- 172.21.15.86:0/1550843519 learned_addr learned my addr 172.21.15.86:0/1550843519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.352+0000 7f028d400700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0290076a80 0x7f029019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.353+0000 7f028e800700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f029010f090 0x7f02901a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:05:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.353+0000 7f028de00700 1 -- 172.21.15.86:0/1550843519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0290076a80 msgr2=0x7f029019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:05:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.353+0000 7f028de00700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0290076a80 0x7f029019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.353+0000 7f028de00700 1 -- 172.21.15.86:0/1550843519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f029010f090 msgr2=0x7f02901a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:05:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.353+0000 7f028de00700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f029010f090 0x7f02901a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:05:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.353+0000 7f028de00700 1 -- 172.21.15.86:0/1550843519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02901a6fe0 con 0x7f02900754b0 2024-04-17T07:05:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.353+0000 7f028de00700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900754b0 0x7f029019f380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f02800057c0 tx=0x7f0280005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:05:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.353+0000 7f0286a00700 1 -- 172.21.15.86:0/1550843519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280027070 con 0x7f02900754b0 2024-04-17T07:05:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.354+0000 7f0286a00700 1 -- 172.21.15.86:0/1550843519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0280022470 con 0x7f02900754b0 2024-04-17T07:05:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.354+0000 7f0294a00700 1 -- 172.21.15.86:0/1550843519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02901a7270 con 0x7f02900754b0 2024-04-17T07:05:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.354+0000 7f0286a00700 1 -- 172.21.15.86:0/1550843519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028000f6c0 con 0x7f02900754b0 2024-04-17T07:05:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.354+0000 7f0294a00700 1 -- 172.21.15.86:0/1550843519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02901a7740 con 0x7f02900754b0 2024-04-17T07:05:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.355+0000 7f0294a00700 1 -- 172.21.15.86:0/1550843519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0290069740 con 0x7f02900754b0 2024-04-17T07:05:41.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.357+0000 7f0286a00700 1 -- 172.21.15.86:0/1550843519 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f028002a070 con 0x7f02900754b0 2024-04-17T07:05:41.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.358+0000 7f0286a00700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02740a3c40 0x7f02740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:05:41.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.358+0000 7f0286a00700 1 -- 172.21.15.86:0/1550843519 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f02800d0150 con 0x7f02900754b0 2024-04-17T07:05:41.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.358+0000 7f028d400700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02740a3c40 0x7f02740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:05:41.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.359+0000 7f028d400700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02740a3c40 0x7f02740a6100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f027c006fd0 tx=0x7f027c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:05:41.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.363+0000 7f0286a00700 1 -- 172.21.15.86:0/1550843519 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f028005eff0 con 0x7f02900754b0 2024-04-17T07:05:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:41 smithi086 ceph-mon[31733]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:41.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:05:41.597+0000 7f0294a00700 1 -- 172.21.15.86:0/1550843519 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0290062e90 con 0x7f02740a3c40 2024-04-17T07:05:41.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:41 smithi098 ceph-mon[35418]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:41.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:41 smithi103 ceph-mon[35495]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:42.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:42 smithi086 ceph-mon[31733]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:05:42.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:05:42.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:05:42.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:42.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:42 smithi098 ceph-mon[35418]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:05:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:05:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:05:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:42.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:42 smithi103 ceph-mon[35495]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:05:42.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:05:42.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:05:42.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:43 smithi086 ceph-mon[31733]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:43 smithi098 ceph-mon[35418]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:43.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:43 smithi103 ceph-mon[35495]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:45 smithi098 ceph-mon[35418]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:45 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2645280921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f"}]: dispatch 2024-04-17T07:05:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:45 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2645280921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f"}]': finished 2024-04-17T07:05:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:45 smithi098 ceph-mon[35418]: osdmap e22: 4 total, 3 up, 4 in 2024-04-17T07:05:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:05:46.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:45 smithi103 ceph-mon[35495]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:46.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:45 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2645280921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f"}]: dispatch 2024-04-17T07:05:46.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:45 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2645280921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f"}]': finished 2024-04-17T07:05:46.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:45 smithi103 ceph-mon[35495]: osdmap e22: 4 total, 3 up, 4 in 2024-04-17T07:05:46.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:05:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:45 smithi086 ceph-mon[31733]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:45 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2645280921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f"}]: dispatch 2024-04-17T07:05:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:45 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2645280921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f"}]': finished 2024-04-17T07:05:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:45 smithi086 ceph-mon[31733]: osdmap e22: 4 total, 3 up, 4 in 2024-04-17T07:05:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:05:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:46 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/942145006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:05:47.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:46 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/942145006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:05:47.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:46 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/942145006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:05:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:48 smithi103 ceph-mon[35495]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:48.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:48 smithi086 ceph-mon[31733]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:48 smithi098 ceph-mon[35418]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:50.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:50 smithi086 ceph-mon[31733]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:50 smithi103 ceph-mon[35495]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:50 smithi098 ceph-mon[35418]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:52.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:52 smithi103 ceph-mon[35495]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:52.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:52 smithi086 ceph-mon[31733]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:52 smithi098 ceph-mon[35418]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:54.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:54 smithi103 ceph-mon[35495]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:54.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:54 smithi086 ceph-mon[31733]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:54.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:54 smithi098 ceph-mon[35418]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:55.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-17T07:05:55.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-17T07:05:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-17T07:05:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:05:56.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:56 smithi086 ceph-mon[31733]: Deploying daemon osd.3 on smithi086 2024-04-17T07:05:56.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:56 smithi086 ceph-mon[31733]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:56 smithi098 ceph-mon[35418]: Deploying daemon osd.3 on smithi086 2024-04-17T07:05:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:56 smithi098 ceph-mon[35418]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:56 smithi103 ceph-mon[35495]: Deploying daemon osd.3 on smithi086 2024-04-17T07:05:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:56 smithi103 ceph-mon[35495]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:58.231 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:05:58 smithi086 ceph-mon[31733]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:58.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:05:58 smithi098 ceph-mon[35418]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:05:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:05:58 smithi103 ceph-mon[35495]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:00 smithi086 ceph-mon[31733]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:00.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:00 smithi098 ceph-mon[35418]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:00 smithi103 ceph-mon[35495]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:02.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:02 smithi086 ceph-mon[31733]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:02.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:02 smithi098 ceph-mon[35418]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:02 smithi103 ceph-mon[35495]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:04.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:04.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:04.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:04.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:03 smithi086 ceph-mon[31733]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:04.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:03 smithi098 ceph-mon[35418]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:04.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:04.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:04.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:04.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:03 smithi103 ceph-mon[35495]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:06.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:05 smithi086 ceph-mon[31733]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:06.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:05 smithi098 ceph-mon[35418]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:06.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:05 smithi103 ceph-mon[35495]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:07.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:07.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:07.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:07 smithi086 ceph-mon[31733]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T07:06:07.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:07 smithi086 ceph-mon[31733]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:07 smithi103 ceph-mon[35495]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T07:06:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:07 smithi103 ceph-mon[35495]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:07.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.535+0000 7f0286a00700 1 -- 172.21.15.86:0/1550843519 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0290062e90 con 0x7f02740a3c40 2024-04-17T07:06:07.536 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 3 on host 'smithi086' 2024-04-17T07:06:07.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 -- 172.21.15.86:0/1550843519 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02740a3c40 msgr2=0x7f02740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:06:07.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02740a3c40 0x7f02740a6100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f027c006fd0 tx=0x7f027c008040 comp rx=0 tx=0).stop 2024-04-17T07:06:07.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 -- 172.21.15.86:0/1550843519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900754b0 msgr2=0x7f029019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:06:07.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900754b0 0x7f029019f380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f02800057c0 tx=0x7f0280005f10 comp rx=0 tx=0).stop 2024-04-17T07:06:07.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 -- 172.21.15.86:0/1550843519 shutdown_connections 2024-04-17T07:06:07.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02740a3c40 0x7f02740a6100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:07.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f029010f090 0x7f02901a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:07.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900754b0 0x7f029019f380 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:07.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 --2- 172.21.15.86:0/1550843519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0290076a80 0x7f029019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:07.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 -- 172.21.15.86:0/1550843519 >> 172.21.15.86:0/1550843519 conn(0x7f02900fe050 msgr2=0x7f0290102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:06:07.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 -- 172.21.15.86:0/1550843519 shutdown_connections 2024-04-17T07:06:07.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:06:07.539+0000 7f0294a00700 1 -- 172.21.15.86:0/1550843519 wait complete. 2024-04-17T07:06:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:07 smithi098 ceph-mon[35418]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-17T07:06:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:07 smithi098 ceph-mon[35418]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:08.348 DEBUG:teuthology.orchestra.run.smithi086:osd.3> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.3.service 2024-04-17T07:06:08.355 INFO:tasks.cephadm:Deploying osd.4 on smithi098 with /dev/vg_nvme/lv_3... 2024-04-17T07:06:08.356 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-17T07:06:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-mon[31733]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-17T07:06:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-mon[31733]: osdmap e23: 4 total, 3 up, 4 in 2024-04-17T07:06:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-mon[31733]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:06:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:08 smithi098 ceph-mon[35418]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-17T07:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:08 smithi098 ceph-mon[35418]: osdmap e23: 4 total, 3 up, 4 in 2024-04-17T07:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:08 smithi098 ceph-mon[35418]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:08.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:08 smithi103 ceph-mon[35495]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-17T07:06:08.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:08 smithi103 ceph-mon[35495]: osdmap e23: 4 total, 3 up, 4 in 2024-04-17T07:06:08.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:08 smithi103 ceph-mon[35495]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:06:08.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:08.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:08.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:08.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:08.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:08.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:08.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:08.798 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:06:09.249 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:06:08 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:06:08.976+0000 7f3c9ec00700 -1 osd.3 0 waiting for initial osdmap 2024-04-17T07:06:09.249 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:06:09 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:06:09.009+0000 7f3c94a00700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:06:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:09 smithi098 ceph-mon[35418]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:06:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:09 smithi098 ceph-mon[35418]: osdmap e24: 4 total, 3 up, 4 in 2024-04-17T07:06:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:09 smithi098 ceph-mon[35418]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:09 smithi098 ceph-mon[35418]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' 2024-04-17T07:06:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:09 smithi086 ceph-mon[31733]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:06:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:09 smithi086 ceph-mon[31733]: osdmap e24: 4 total, 3 up, 4 in 2024-04-17T07:06:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:09 smithi086 ceph-mon[31733]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:09 smithi086 ceph-mon[31733]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' 2024-04-17T07:06:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:09 smithi103 ceph-mon[35495]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:06:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:09 smithi103 ceph-mon[35495]: osdmap e24: 4 total, 3 up, 4 in 2024-04-17T07:06:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:09 smithi103 ceph-mon[35495]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-17T07:06:09.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:09 smithi103 ceph-mon[35495]: from='osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177]' entity='osd.3' 2024-04-17T07:06:10.458 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:10 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:06:10.458 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:10 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:06:10.458 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:10.458 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:10 smithi086 ceph-mon[31733]: osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] boot 2024-04-17T07:06:10.458 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:10 smithi086 ceph-mon[31733]: osdmap e25: 4 total, 4 up, 4 in 2024-04-17T07:06:10.458 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:10.578 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:10 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:06:10.578 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:10 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:06:10.578 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:10.578 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:10 smithi098 ceph-mon[35418]: osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] boot 2024-04-17T07:06:10.578 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:10 smithi098 ceph-mon[35418]: osdmap e25: 4 total, 4 up, 4 in 2024-04-17T07:06:10.578 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:10.678 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:10 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:06:10.678 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:10 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:06:10.678 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:10.678 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:10 smithi103 ceph-mon[35495]: osd.3 [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] boot 2024-04-17T07:06:10.678 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:10 smithi103 ceph-mon[35495]: osdmap e25: 4 total, 4 up, 4 in 2024-04-17T07:06:10.678 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-17T07:06:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:11 smithi086 ceph-mon[31733]: osdmap e26: 4 total, 4 up, 4 in 2024-04-17T07:06:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:11 smithi086 ceph-mon[31733]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:11 smithi086 ceph-mon[31733]: osdmap e27: 4 total, 4 up, 4 in 2024-04-17T07:06:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:11 smithi098 ceph-mon[35418]: osdmap e26: 4 total, 4 up, 4 in 2024-04-17T07:06:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:11 smithi098 ceph-mon[35418]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:11 smithi098 ceph-mon[35418]: osdmap e27: 4 total, 4 up, 4 in 2024-04-17T07:06:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:11 smithi103 ceph-mon[35495]: osdmap e26: 4 total, 4 up, 4 in 2024-04-17T07:06:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:11 smithi103 ceph-mon[35495]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:11 smithi103 ceph-mon[35495]: osdmap e27: 4 total, 4 up, 4 in 2024-04-17T07:06:11.839 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-17T07:06:11.839 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-17T07:06:11.839 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-17T07:06:11.839 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-17T07:06:11.839 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0372684 s, 281 MB/s 2024-04-17T07:06:11.839 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-17T07:06:12.669 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_3 2024-04-17T07:06:13.098 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:06:13.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:13 smithi098 ceph-mon[35418]: osdmap e28: 4 total, 4 up, 4 in 2024-04-17T07:06:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:13 smithi098 ceph-mon[35418]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:13 smithi086 ceph-mon[31733]: osdmap e28: 4 total, 4 up, 4 in 2024-04-17T07:06:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:13 smithi086 ceph-mon[31733]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:13.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:13 smithi103 ceph-mon[35495]: osdmap e28: 4 total, 4 up, 4 in 2024-04-17T07:06:13.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:13 smithi103 ceph-mon[35495]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:14.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: osdmap e29: 4 total, 4 up, 4 in 2024-04-17T07:06:14.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi086 2024-04-17T07:06:14.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:14.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:14.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-17T07:06:14.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:14.481 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:14.482 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: osdmap e29: 4 total, 4 up, 4 in 2024-04-17T07:06:14.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi086 2024-04-17T07:06:14.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:14.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:14.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-17T07:06:14.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:14.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:14.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: osdmap e29: 4 total, 4 up, 4 in 2024-04-17T07:06:14.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi086 2024-04-17T07:06:14.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:14.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:14.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-17T07:06:14.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:14.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:14.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:14.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:15.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.314+0000 7fd556000700 1 -- 172.21.15.98:0/4181025368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd550106770 msgr2=0x7fd55010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:06:15.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.314+0000 7fd556000700 1 --2- 172.21.15.98:0/4181025368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd550106770 0x7fd55010d020 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b0d0 tx=0x7fd54800b3e0 comp rx=0 tx=0).stop 2024-04-17T07:06:15.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.315+0000 7fd556000700 1 -- 172.21.15.98:0/4181025368 shutdown_connections 2024-04-17T07:06:15.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.315+0000 7fd556000700 1 --2- 172.21.15.98:0/4181025368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd550106770 0x7fd55010d020 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:15.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.315+0000 7fd556000700 1 --2- 172.21.15.98:0/4181025368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd550104bd0 0x7fd550104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:15.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.315+0000 7fd556000700 1 --2- 172.21.15.98:0/4181025368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd550105dd0 0x7fd550106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:15.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.315+0000 7fd556000700 1 -- 172.21.15.98:0/4181025368 >> 172.21.15.98:0/4181025368 conn(0x7fd550100170 msgr2=0x7fd5501025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:06:15.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.315+0000 7fd556000700 1 -- 172.21.15.98:0/4181025368 shutdown_connections 2024-04-17T07:06:15.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.315+0000 7fd556000700 1 -- 172.21.15.98:0/4181025368 wait complete. 2024-04-17T07:06:15.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.316+0000 7fd556000700 1 Processor -- start 2024-04-17T07:06:15.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.316+0000 7fd556000700 1 -- start start 2024-04-17T07:06:15.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.317+0000 7fd556000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd550104bd0 0x7fd55019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:06:15.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.317+0000 7fd556000700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd550105dd0 0x7fd55019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:06:15.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.317+0000 7fd556000700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd550106770 0x7fd5501a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:06:15.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.317+0000 7fd556000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5501106a0 con 0x7fd550104bd0 2024-04-17T07:06:15.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.317+0000 7fd556000700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd550110520 con 0x7fd550106770 2024-04-17T07:06:15.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.317+0000 7fd556000700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd550110820 con 0x7fd550105dd0 2024-04-17T07:06:15.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.317+0000 7fd555600700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd550106770 0x7fd5501a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:06:15.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.317+0000 7fd555600700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd550106770 0x7fd5501a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59614/0 (socket says 172.21.15.98:59614) 2024-04-17T07:06:15.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.317+0000 7fd555600700 1 -- 172.21.15.98:0/485295881 learned_addr learned my addr 172.21.15.98:0/485295881 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-17T07:06:15.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.317+0000 7fd554c00700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd550104bd0 0x7fd55019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:06:15.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd54fe00700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd550105dd0 0x7fd55019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:06:15.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd555600700 1 -- 172.21.15.98:0/485295881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd550105dd0 msgr2=0x7fd55019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:06:15.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd555600700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd550105dd0 0x7fd55019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:15.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd555600700 1 -- 172.21.15.98:0/485295881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd550104bd0 msgr2=0x7fd55019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:06:15.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd555600700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd550104bd0 0x7fd55019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:15.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd555600700 1 -- 172.21.15.98:0/485295881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5501a28a0 con 0x7fd550106770 2024-04-17T07:06:15.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd54fe00700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd550105dd0 0x7fd55019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:06:15.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd555600700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd550106770 0x7fd5501a21d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd548004c80 tx=0x7fd548004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:06:15.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd54d600700 1 -- 172.21.15.98:0/485295881 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54801f030 con 0x7fd550106770 2024-04-17T07:06:15.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd54d600700 1 -- 172.21.15.98:0/485295881 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd548005040 con 0x7fd550106770 2024-04-17T07:06:15.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd54d600700 1 -- 172.21.15.98:0/485295881 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54801b690 con 0x7fd550106770 2024-04-17T07:06:15.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.318+0000 7fd556000700 1 -- 172.21.15.98:0/485295881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5501a2b30 con 0x7fd550106770 2024-04-17T07:06:15.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.319+0000 7fd556000700 1 -- 172.21.15.98:0/485295881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5501a2f30 con 0x7fd550106770 2024-04-17T07:06:15.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.320+0000 7fd556000700 1 -- 172.21.15.98:0/485295881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd550069740 con 0x7fd550106770 2024-04-17T07:06:15.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.322+0000 7fd54d600700 1 -- 172.21.15.98:0/485295881 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd548022030 con 0x7fd550106770 2024-04-17T07:06:15.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.323+0000 7fd54d600700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5380a3cc0 0x7fd5380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:06:15.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.323+0000 7fd54d600700 1 -- 172.21.15.98:0/485295881 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fd54800f020 con 0x7fd550106770 2024-04-17T07:06:15.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.324+0000 7fd554c00700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5380a3cc0 0x7fd5380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:06:15.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.325+0000 7fd554c00700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5380a3cc0 0x7fd5380a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd55019c130 tx=0x7fd54400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:06:15.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.327+0000 7fd54d600700 1 -- 172.21.15.98:0/485295881 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd548029020 con 0x7fd550106770 2024-04-17T07:06:15.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:15 smithi098 ceph-mon[35418]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:15.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:15.564+0000 7fd556000700 1 -- 172.21.15.98:0/485295881 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd550062e90 con 0x7fd5380a3cc0 2024-04-17T07:06:15.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:15 smithi103 ceph-mon[35495]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:15.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:15 smithi086 ceph-mon[31733]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:16.706 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:16 smithi098 ceph-mon[35418]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:06:16.706 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:16 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:06:16.706 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:16 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:06:16.707 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:16 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:16 smithi103 ceph-mon[35495]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:06:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:16 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:06:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:16 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:06:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:16 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:16 smithi086 ceph-mon[31733]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:06:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:16 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:06:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:16 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:06:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:16 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:17 smithi098 ceph-mon[35418]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T07:06:17.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:17 smithi103 ceph-mon[35495]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T07:06:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:17 smithi086 ceph-mon[31733]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-17T07:06:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:20 smithi098 ceph-mon[35418]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-17T07:06:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:20 smithi098 ceph-mon[35418]: from='client.? 172.21.15.98:0/1722328820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0591e547-4420-43e7-9454-663746016d3a"}]: dispatch 2024-04-17T07:06:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:20 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0591e547-4420-43e7-9454-663746016d3a"}]: dispatch 2024-04-17T07:06:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:20 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0591e547-4420-43e7-9454-663746016d3a"}]': finished 2024-04-17T07:06:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:20 smithi098 ceph-mon[35418]: osdmap e30: 5 total, 4 up, 5 in 2024-04-17T07:06:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:20.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:20 smithi103 ceph-mon[35495]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-17T07:06:20.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:20 smithi103 ceph-mon[35495]: from='client.? 172.21.15.98:0/1722328820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0591e547-4420-43e7-9454-663746016d3a"}]: dispatch 2024-04-17T07:06:20.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:20 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0591e547-4420-43e7-9454-663746016d3a"}]: dispatch 2024-04-17T07:06:20.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:20 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0591e547-4420-43e7-9454-663746016d3a"}]': finished 2024-04-17T07:06:20.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:20 smithi103 ceph-mon[35495]: osdmap e30: 5 total, 4 up, 5 in 2024-04-17T07:06:20.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:20 smithi086 ceph-mon[31733]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-17T07:06:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:20 smithi086 ceph-mon[31733]: from='client.? 172.21.15.98:0/1722328820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0591e547-4420-43e7-9454-663746016d3a"}]: dispatch 2024-04-17T07:06:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:20 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0591e547-4420-43e7-9454-663746016d3a"}]: dispatch 2024-04-17T07:06:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:20 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0591e547-4420-43e7-9454-663746016d3a"}]': finished 2024-04-17T07:06:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:20 smithi086 ceph-mon[31733]: osdmap e30: 5 total, 4 up, 5 in 2024-04-17T07:06:20.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:21.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:21 smithi103 ceph-mon[35495]: from='client.? 172.21.15.98:0/1402636507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:06:21.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:21 smithi086 ceph-mon[31733]: from='client.? 172.21.15.98:0/1402636507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:06:21.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:21 smithi098 ceph-mon[35418]: from='client.? 172.21.15.98:0/1402636507' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:06:22.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:22 smithi103 ceph-mon[35495]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:06:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:22 smithi086 ceph-mon[31733]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:06:22.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:22 smithi098 ceph-mon[35418]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:06:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:24 smithi098 ceph-mon[35418]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-17T07:06:24.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:24 smithi103 ceph-mon[35495]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-17T07:06:24.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:24 smithi086 ceph-mon[31733]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-17T07:06:26.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:26 smithi086 ceph-mon[31733]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:06:26.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:26 smithi098 ceph-mon[35418]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:06:26.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:26 smithi103 ceph-mon[35495]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:06:28.247 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:28 smithi098 ceph-mon[35418]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:28 smithi086 ceph-mon[31733]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:28 smithi103 ceph-mon[35495]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:29.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-17T07:06:29.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:29.437 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:29 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-17T07:06:29.437 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:29 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:29 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-17T07:06:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:29 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:30.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:30 smithi086 ceph-mon[31733]: Deploying daemon osd.4 on smithi098 2024-04-17T07:06:30.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:30 smithi086 ceph-mon[31733]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:30.446 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:30 smithi098 ceph-mon[35418]: Deploying daemon osd.4 on smithi098 2024-04-17T07:06:30.446 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:30 smithi098 ceph-mon[35418]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:30 smithi103 ceph-mon[35495]: Deploying daemon osd.4 on smithi098 2024-04-17T07:06:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:30 smithi103 ceph-mon[35495]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:32 smithi098 ceph-mon[35418]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:32.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:32 smithi086 ceph-mon[31733]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:32 smithi103 ceph-mon[35495]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:34.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:34 smithi086 ceph-mon[31733]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:34 smithi098 ceph-mon[35418]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:34 smithi103 ceph-mon[35495]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:36.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:36 smithi086 ceph-mon[31733]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:36.366 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:36 smithi098 ceph-mon[35418]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:36.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:36 smithi103 ceph-mon[35495]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:37.819 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:37.819 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:37.819 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:37.819 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:37 smithi098 ceph-mon[35418]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:38.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:38.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:38.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:38.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:37 smithi103 ceph-mon[35495]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:38.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:38.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:38.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:38.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:37 smithi086 ceph-mon[31733]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:40.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:40 smithi086 ceph-mon[31733]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:40.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:40 smithi098 ceph-mon[35418]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:40.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:40 smithi103 ceph-mon[35495]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:41.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:41 smithi098 ceph-mon[35418]: from='osd.4 [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:06:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:41 smithi098 ceph-mon[35418]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:06:41.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:41.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:41.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:41 smithi103 ceph-mon[35495]: from='osd.4 [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:06:41.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:41 smithi103 ceph-mon[35495]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:06:41.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:41.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:41.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:41 smithi086 ceph-mon[31733]: from='osd.4 [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:06:41.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:41 smithi086 ceph-mon[31733]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-17T07:06:41.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.435+0000 7fd54d600700 1 -- 172.21.15.98:0/485295881 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd550062e90 con 0x7fd5380a3cc0 2024-04-17T07:06:41.436 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 4 on host 'smithi098' 2024-04-17T07:06:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.439+0000 7fd556000700 1 -- 172.21.15.98:0/485295881 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5380a3cc0 msgr2=0x7fd5380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:06:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.439+0000 7fd556000700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5380a3cc0 0x7fd5380a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd55019c130 tx=0x7fd54400a3b0 comp rx=0 tx=0).stop 2024-04-17T07:06:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.439+0000 7fd556000700 1 -- 172.21.15.98:0/485295881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd550106770 msgr2=0x7fd5501a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:06:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.439+0000 7fd556000700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd550106770 0x7fd5501a21d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd548004c80 tx=0x7fd548004cb0 comp rx=0 tx=0).stop 2024-04-17T07:06:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.440+0000 7fd556000700 1 -- 172.21.15.98:0/485295881 shutdown_connections 2024-04-17T07:06:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.440+0000 7fd556000700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5380a3cc0 0x7fd5380a6180 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:41.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.440+0000 7fd556000700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd550106770 0x7fd5501a21d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:41.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.440+0000 7fd556000700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd550104bd0 0x7fd55019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:41.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.440+0000 7fd556000700 1 --2- 172.21.15.98:0/485295881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd550105dd0 0x7fd55019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:41.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.440+0000 7fd556000700 1 -- 172.21.15.98:0/485295881 >> 172.21.15.98:0/485295881 conn(0x7fd550100170 msgr2=0x7fd5501090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:06:41.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.440+0000 7fd556000700 1 -- 172.21.15.98:0/485295881 shutdown_connections 2024-04-17T07:06:41.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:06:41.440+0000 7fd556000700 1 -- 172.21.15.98:0/485295881 wait complete. 2024-04-17T07:06:42.093 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:42.093 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-17T07:06:42.093 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: from='osd.4 [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:06:42.093 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: osdmap e31: 5 total, 4 up, 5 in 2024-04-17T07:06:42.094 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:42.094 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:06:42.094 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:42.094 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:42.094 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:42.094 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:42.094 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:42.094 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:42.265 DEBUG:teuthology.orchestra.run.smithi098:osd.4> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.4.service 2024-04-17T07:06:42.271 INFO:tasks.cephadm:Deploying osd.5 on smithi103 with /dev/vg_nvme/lv_3... 2024-04-17T07:06:42.271 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: from='osd.4 [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: osdmap e31: 5 total, 4 up, 5 in 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:42.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:42.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:42.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-17T07:06:42.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: from='osd.4 [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:06:42.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: osdmap e31: 5 total, 4 up, 5 in 2024-04-17T07:06:42.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:42.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:06:42.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:42.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:42.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:42.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:42.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:42.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:06:42.674 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:06:43.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:43 smithi098 ceph-mon[35418]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:06:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:43 smithi098 ceph-mon[35418]: osdmap e32: 5 total, 4 up, 5 in 2024-04-17T07:06:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:43.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:43 smithi086 ceph-mon[31733]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:06:43.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:43 smithi086 ceph-mon[31733]: osdmap e32: 5 total, 4 up, 5 in 2024-04-17T07:06:43.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:43.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:43.525 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:43 smithi103 ceph-mon[35495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:06:43.525 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:43 smithi103 ceph-mon[35495]: osdmap e32: 5 total, 4 up, 5 in 2024-04-17T07:06:43.525 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:43.525 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:44.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:44 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:06:44.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:44 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:06:44.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:44 smithi086 ceph-mon[31733]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:44.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:44 smithi086 ceph-mon[31733]: osd.4 [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] boot 2024-04-17T07:06:44.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:44 smithi086 ceph-mon[31733]: osdmap e33: 5 total, 5 up, 5 in 2024-04-17T07:06:44.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:44.378 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:44 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:06:44.378 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:44 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:06:44.378 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:44 smithi098 ceph-mon[35418]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:44.379 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:44 smithi098 ceph-mon[35418]: osd.4 [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] boot 2024-04-17T07:06:44.379 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:44 smithi098 ceph-mon[35418]: osdmap e33: 5 total, 5 up, 5 in 2024-04-17T07:06:44.379 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:44 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:44.417 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:44 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:06:44.417 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:44 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:06:44.417 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:44 smithi103 ceph-mon[35495]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-17T07:06:44.417 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:44 smithi103 ceph-mon[35495]: osd.4 [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] boot 2024-04-17T07:06:44.418 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:44 smithi103 ceph-mon[35495]: osdmap e33: 5 total, 5 up, 5 in 2024-04-17T07:06:44.418 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-17T07:06:45.464 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:45 smithi103 ceph-mon[35495]: osdmap e34: 5 total, 5 up, 5 in 2024-04-17T07:06:45.464 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-17T07:06:45.464 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-17T07:06:45.464 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-17T07:06:45.464 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-17T07:06:45.464 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0343731 s, 305 MB/s 2024-04-17T07:06:45.465 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-17T07:06:45.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:45 smithi098 ceph-mon[35418]: osdmap e34: 5 total, 5 up, 5 in 2024-04-17T07:06:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:45 smithi086 ceph-mon[31733]: osdmap e34: 5 total, 5 up, 5 in 2024-04-17T07:06:46.210 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:46 smithi103 ceph-mon[35495]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:46.286 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-04-17T07:06:46.374 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:46 smithi098 ceph-mon[35418]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:46 smithi086 ceph-mon[31733]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:46.732 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:06:48.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:48 smithi103 ceph-mon[35495]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:48.196 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.196 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.196 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:48.196 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:48.196 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.196 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:48.196 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:48.196 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:48 smithi098 ceph-mon[35418]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:48.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:48 smithi086 ceph-mon[31733]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:06:48.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:48.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:06:48.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:06:48.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.702+0000 7ffb15a00700 1 -- 172.21.15.103:0/908548504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb100754b0 msgr2=0x7ffb10075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:06:48.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.702+0000 7ffb15a00700 1 --2- 172.21.15.103:0/908548504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb100754b0 0x7ffb10075930 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009990 tx=0x7ffafc009ca0 comp rx=0 tx=0).stop 2024-04-17T07:06:48.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.702+0000 7ffb15a00700 1 -- 172.21.15.103:0/908548504 shutdown_connections 2024-04-17T07:06:48.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.702+0000 7ffb15a00700 1 --2- 172.21.15.103:0/908548504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb1010a980 0x7ffb1010ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:48.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.702+0000 7ffb15a00700 1 --2- 172.21.15.103:0/908548504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb10076a80 0x7ffb10074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:48.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.702+0000 7ffb15a00700 1 --2- 172.21.15.103:0/908548504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb100754b0 0x7ffb10075930 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:48.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.702+0000 7ffb15a00700 1 -- 172.21.15.103:0/908548504 >> 172.21.15.103:0/908548504 conn(0x7ffb100fe050 msgr2=0x7ffb10100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:06:48.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.703+0000 7ffb15a00700 1 -- 172.21.15.103:0/908548504 shutdown_connections 2024-04-17T07:06:48.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.703+0000 7ffb15a00700 1 -- 172.21.15.103:0/908548504 wait complete. 2024-04-17T07:06:48.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.704+0000 7ffb15a00700 1 Processor -- start 2024-04-17T07:06:48.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.704+0000 7ffb15a00700 1 -- start start 2024-04-17T07:06:48.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb15a00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb100754b0 0x7ffb1019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:06:48.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb15a00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb10076a80 0x7ffb1019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:06:48.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb15a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1010a980 0x7ffb101a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:06:48.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb15a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10110760 con 0x7ffb1010a980 2024-04-17T07:06:48.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb15a00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101105e0 con 0x7ffb100754b0 2024-04-17T07:06:48.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb15a00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101108e0 con 0x7ffb10076a80 2024-04-17T07:06:48.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0e000700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb10076a80 0x7ffb1019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:06:48.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0e000700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb10076a80 0x7ffb1019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35914/0 (socket says 172.21.15.103:35914) 2024-04-17T07:06:48.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0e000700 1 -- 172.21.15.103:0/3304294906 learned_addr learned my addr 172.21.15.103:0/3304294906 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:06:48.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0ea00700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb100754b0 0x7ffb1019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:06:48.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0f400700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1010a980 0x7ffb101a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:06:48.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0e000700 1 -- 172.21.15.103:0/3304294906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb100754b0 msgr2=0x7ffb1019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:06:48.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0e000700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb100754b0 0x7ffb1019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:48.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0e000700 1 -- 172.21.15.103:0/3304294906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1010a980 msgr2=0x7ffb101a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:06:48.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0e000700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1010a980 0x7ffb101a23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:06:48.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0e000700 1 -- 172.21.15.103:0/3304294906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb101a2a70 con 0x7ffb10076a80 2024-04-17T07:06:48.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.705+0000 7ffb0ea00700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb100754b0 0x7ffb1019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:06:48.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.706+0000 7ffb0e000700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb10076a80 0x7ffb1019b3a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffafc000c00 tx=0x7ffafc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:06:48.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.706+0000 7ffb07400700 1 -- 172.21.15.103:0/3304294906 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc027070 con 0x7ffb10076a80 2024-04-17T07:06:48.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.706+0000 7ffb07400700 1 -- 172.21.15.103:0/3304294906 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffafc004190 con 0x7ffb10076a80 2024-04-17T07:06:48.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.706+0000 7ffb15a00700 1 -- 172.21.15.103:0/3304294906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb101a2d00 con 0x7ffb10076a80 2024-04-17T07:06:48.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.706+0000 7ffb07400700 1 -- 172.21.15.103:0/3304294906 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc022e60 con 0x7ffb10076a80 2024-04-17T07:06:48.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.706+0000 7ffb15a00700 1 -- 172.21.15.103:0/3304294906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb101a3180 con 0x7ffb10076a80 2024-04-17T07:06:48.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.708+0000 7ffb15a00700 1 -- 172.21.15.103:0/3304294906 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1002df90 con 0x7ffb10076a80 2024-04-17T07:06:48.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.709+0000 7ffb07400700 1 -- 172.21.15.103:0/3304294906 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffafc02a070 con 0x7ffb10076a80 2024-04-17T07:06:48.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.711+0000 7ffb07400700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffaf40a3cc0 0x7ffaf40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:06:48.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.711+0000 7ffb07400700 1 -- 172.21.15.103:0/3304294906 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7ffafc0d0040 con 0x7ffb10076a80 2024-04-17T07:06:48.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.712+0000 7ffb0ea00700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffaf40a3cc0 0x7ffaf40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:06:48.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.713+0000 7ffb0ea00700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffaf40a3cc0 0x7ffaf40a6180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffb00005950 tx=0x7ffb000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:06:48.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.716+0000 7ffb07400700 1 -- 172.21.15.103:0/3304294906 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffafc013070 con 0x7ffb10076a80 2024-04-17T07:06:48.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:06:48.950+0000 7ffb15a00700 1 -- 172.21.15.103:0/3304294906 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ffb10062e90 con 0x7ffaf40a3cc0 2024-04-17T07:06:49.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:49 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi098 2024-04-17T07:06:49.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:49 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi098 to 8544M 2024-04-17T07:06:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:06:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:06:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:49.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:49 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi098 2024-04-17T07:06:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:49 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi098 to 8544M 2024-04-17T07:06:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:06:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:06:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:49 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi098 2024-04-17T07:06:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:49 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi098 to 8544M 2024-04-17T07:06:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:06:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:06:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:06:50.241 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:50 smithi103 ceph-mon[35495]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:06:50.241 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:50 smithi103 ceph-mon[35495]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:50 smithi098 ceph-mon[35418]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:06:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:50 smithi098 ceph-mon[35418]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:50 smithi086 ceph-mon[31733]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:06:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:50 smithi086 ceph-mon[31733]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:51 smithi103 ceph-mon[35495]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:51.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:51 smithi098 ceph-mon[35418]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:51 smithi086 ceph-mon[31733]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:52 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/573792996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3965d5b0-7cd1-40fe-9b78-287e5a584a5b"}]: dispatch 2024-04-17T07:06:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:52 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3965d5b0-7cd1-40fe-9b78-287e5a584a5b"}]: dispatch 2024-04-17T07:06:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:52 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3965d5b0-7cd1-40fe-9b78-287e5a584a5b"}]': finished 2024-04-17T07:06:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:52 smithi098 ceph-mon[35418]: osdmap e35: 6 total, 5 up, 6 in 2024-04-17T07:06:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:52 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:06:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:52 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/573792996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3965d5b0-7cd1-40fe-9b78-287e5a584a5b"}]: dispatch 2024-04-17T07:06:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:52 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3965d5b0-7cd1-40fe-9b78-287e5a584a5b"}]: dispatch 2024-04-17T07:06:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:52 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3965d5b0-7cd1-40fe-9b78-287e5a584a5b"}]': finished 2024-04-17T07:06:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:52 smithi103 ceph-mon[35495]: osdmap e35: 6 total, 5 up, 6 in 2024-04-17T07:06:53.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:52 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:06:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:52 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/573792996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3965d5b0-7cd1-40fe-9b78-287e5a584a5b"}]: dispatch 2024-04-17T07:06:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:52 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3965d5b0-7cd1-40fe-9b78-287e5a584a5b"}]: dispatch 2024-04-17T07:06:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:52 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3965d5b0-7cd1-40fe-9b78-287e5a584a5b"}]': finished 2024-04-17T07:06:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:52 smithi086 ceph-mon[31733]: osdmap e35: 6 total, 5 up, 6 in 2024-04-17T07:06:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:52 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:06:54.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:53 smithi098 ceph-mon[35418]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:54.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:53 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/1767574283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:06:54.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:53 smithi103 ceph-mon[35495]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:54.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:53 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/1767574283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:06:54.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:53 smithi086 ceph-mon[31733]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:54.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:53 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/1767574283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:06:56.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:56 smithi103 ceph-mon[35495]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:56.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:56 smithi086 ceph-mon[31733]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:56 smithi098 ceph-mon[35418]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:58.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:06:58 smithi103 ceph-mon[35495]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:58.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:06:58 smithi086 ceph-mon[31733]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:06:58.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:06:58 smithi098 ceph-mon[35418]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:00.272 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:00 smithi103 ceph-mon[35495]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:00 smithi086 ceph-mon[31733]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:00.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:00 smithi098 ceph-mon[35418]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:02.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:02 smithi103 ceph-mon[35495]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:02.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:02 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-17T07:07:02.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:02 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:02.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:02 smithi086 ceph-mon[31733]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:02.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:02 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-17T07:07:02.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:02 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:02.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:02 smithi098 ceph-mon[35418]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:02 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-17T07:07:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:02 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:03.222 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:03 smithi103 ceph-mon[35495]: Deploying daemon osd.5 on smithi103 2024-04-17T07:07:03.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:03 smithi086 ceph-mon[31733]: Deploying daemon osd.5 on smithi103 2024-04-17T07:07:03.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:03 smithi098 ceph-mon[35418]: Deploying daemon osd.5 on smithi103 2024-04-17T07:07:04.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:04 smithi103 ceph-mon[35495]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:04.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:04 smithi086 ceph-mon[31733]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:04 smithi098 ceph-mon[35418]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:06.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:06 smithi086 ceph-mon[31733]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:06.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:06 smithi098 ceph-mon[35418]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:06.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:06 smithi103 ceph-mon[35495]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:08 smithi086 ceph-mon[31733]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:08.446 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:08 smithi103 ceph-mon[35495]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:08.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:08 smithi098 ceph-mon[35418]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:10.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:10 smithi086 ceph-mon[31733]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:10.275 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:10 smithi103 ceph-mon[35495]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:10.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:10 smithi098 ceph-mon[35418]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:11 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:11 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:11 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:11 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:11 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:11 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:11.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:11 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:11 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:11 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:12.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:12 smithi098 ceph-mon[35418]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:12 smithi103 ceph-mon[35495]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:12 smithi086 ceph-mon[31733]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:14.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:14 smithi103 ceph-mon[35495]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:14.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:14.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:14.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:14.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:14.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:14 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:14.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:14 smithi103 ceph-mon[35495]: from='osd.5 [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:07:14.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:14 smithi103 ceph-mon[35495]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:14 smithi098 ceph-mon[35418]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:14 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:14 smithi098 ceph-mon[35418]: from='osd.5 [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:07:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:14 smithi098 ceph-mon[35418]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:07:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:14 smithi086 ceph-mon[31733]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:14 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:14 smithi086 ceph-mon[31733]: from='osd.5 [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:07:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:14 smithi086 ceph-mon[31733]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-17T07:07:14.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.903+0000 7ffb07400700 1 -- 172.21.15.103:0/3304294906 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffb10062e90 con 0x7ffaf40a3cc0 2024-04-17T07:07:14.905 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 5 on host 'smithi103' 2024-04-17T07:07:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.907+0000 7ffb15a00700 1 -- 172.21.15.103:0/3304294906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffaf40a3cc0 msgr2=0x7ffaf40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:07:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.907+0000 7ffb15a00700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffaf40a3cc0 0x7ffaf40a6180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffb00005950 tx=0x7ffb000058e0 comp rx=0 tx=0).stop 2024-04-17T07:07:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.907+0000 7ffb15a00700 1 -- 172.21.15.103:0/3304294906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb10076a80 msgr2=0x7ffb1019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:07:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.907+0000 7ffb15a00700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb10076a80 0x7ffb1019b3a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffafc000c00 tx=0x7ffafc009580 comp rx=0 tx=0).stop 2024-04-17T07:07:14.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.908+0000 7ffb15a00700 1 -- 172.21.15.103:0/3304294906 shutdown_connections 2024-04-17T07:07:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.908+0000 7ffb15a00700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffaf40a3cc0 0x7ffaf40a6180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.908+0000 7ffb15a00700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb1010a980 0x7ffb101a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.908+0000 7ffb15a00700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb100754b0 0x7ffb1019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.908+0000 7ffb15a00700 1 --2- 172.21.15.103:0/3304294906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb10076a80 0x7ffb1019b3a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.908+0000 7ffb15a00700 1 -- 172.21.15.103:0/3304294906 >> 172.21.15.103:0/3304294906 conn(0x7ffb100fe050 msgr2=0x7ffb10106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:07:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.908+0000 7ffb15a00700 1 -- 172.21.15.103:0/3304294906 shutdown_connections 2024-04-17T07:07:14.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:07:14.908+0000 7ffb15a00700 1 -- 172.21.15.103:0/3304294906 wait complete. 2024-04-17T07:07:15.699 DEBUG:teuthology.orchestra.run.smithi103:osd.5> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.5.service 2024-04-17T07:07:15.703 INFO:tasks.cephadm:Deploying osd.6 on smithi086 with /dev/vg_nvme/lv_2... 2024-04-17T07:07:15.704 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-17T07:07:15.928 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:15 smithi103 ceph-mon[35495]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-17T07:07:15.928 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:15 smithi103 ceph-mon[35495]: from='osd.5 [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:07:15.928 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:15 smithi103 ceph-mon[35495]: osdmap e36: 6 total, 5 up, 6 in 2024-04-17T07:07:15.929 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:15.929 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:15 smithi103 ceph-mon[35495]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:07:15.929 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:15.929 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:15.929 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:15.929 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:15 smithi103 ceph-mon[35495]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:15 smithi098 ceph-mon[35418]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-17T07:07:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:15 smithi098 ceph-mon[35418]: from='osd.5 [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:07:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:15 smithi098 ceph-mon[35418]: osdmap e36: 6 total, 5 up, 6 in 2024-04-17T07:07:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:15 smithi098 ceph-mon[35418]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:07:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:15 smithi098 ceph-mon[35418]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:16.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:15 smithi086 ceph-mon[31733]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-17T07:07:16.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:15 smithi086 ceph-mon[31733]: from='osd.5 [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:07:16.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:15 smithi086 ceph-mon[31733]: osdmap e36: 6 total, 5 up, 6 in 2024-04-17T07:07:16.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:16.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:15 smithi086 ceph-mon[31733]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:07:16.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:16.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:16.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:16.083 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:15 smithi086 ceph-mon[31733]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-04-17T07:07:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:07:16.527 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:07:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:07:16.324+0000 7f5b3e200700 -1 osd.5 0 waiting for initial osdmap 2024-04-17T07:07:16.527 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:07:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:07:16.353+0000 7f5b33600700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:07:16.950 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:16 smithi103 ceph-mon[35495]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:07:16.950 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:16 smithi103 ceph-mon[35495]: osdmap e37: 6 total, 5 up, 6 in 2024-04-17T07:07:16.950 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:16 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:16.950 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:16 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:16.950 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:16 smithi103 ceph-mon[35495]: from='osd.5 ' entity='osd.5' 2024-04-17T07:07:16.951 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:16 smithi103 ceph-mon[35495]: osd.5 [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] boot 2024-04-17T07:07:16.951 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:16 smithi103 ceph-mon[35495]: osdmap e38: 6 total, 6 up, 6 in 2024-04-17T07:07:16.951 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:16 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:16 smithi098 ceph-mon[35418]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:07:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:16 smithi098 ceph-mon[35418]: osdmap e37: 6 total, 5 up, 6 in 2024-04-17T07:07:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:16 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:16 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:16 smithi098 ceph-mon[35418]: from='osd.5 ' entity='osd.5' 2024-04-17T07:07:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:16 smithi098 ceph-mon[35418]: osd.5 [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] boot 2024-04-17T07:07:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:16 smithi098 ceph-mon[35418]: osdmap e38: 6 total, 6 up, 6 in 2024-04-17T07:07:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:16 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:17.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:16 smithi086 ceph-mon[31733]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:07:17.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:16 smithi086 ceph-mon[31733]: osdmap e37: 6 total, 5 up, 6 in 2024-04-17T07:07:17.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:16 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:17.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:16 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:17.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:16 smithi086 ceph-mon[31733]: from='osd.5 ' entity='osd.5' 2024-04-17T07:07:17.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:16 smithi086 ceph-mon[31733]: osd.5 [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] boot 2024-04-17T07:07:17.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:16 smithi086 ceph-mon[31733]: osdmap e38: 6 total, 6 up, 6 in 2024-04-17T07:07:17.054 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:16 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-17T07:07:18.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:17 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:07:18.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:17 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:07:18.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:17 smithi086 ceph-mon[31733]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:18.012 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:17 smithi086 ceph-mon[31733]: osdmap e39: 6 total, 6 up, 6 in 2024-04-17T07:07:18.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:17 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:07:18.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:17 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:07:18.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:17 smithi103 ceph-mon[35495]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:18.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:17 smithi103 ceph-mon[35495]: osdmap e39: 6 total, 6 up, 6 in 2024-04-17T07:07:18.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:17 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:07:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:17 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:07:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:17 smithi098 ceph-mon[35418]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:17 smithi098 ceph-mon[35418]: osdmap e39: 6 total, 6 up, 6 in 2024-04-17T07:07:19.104 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-17T07:07:19.104 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-17T07:07:19.104 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-17T07:07:19.104 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-17T07:07:19.104 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0354532 s, 296 MB/s 2024-04-17T07:07:19.104 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-17T07:07:19.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:19 smithi103 ceph-mon[35495]: osdmap e40: 6 total, 6 up, 6 in 2024-04-17T07:07:19.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:19 smithi103 ceph-mon[35495]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:19.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:19 smithi086 ceph-mon[31733]: osdmap e40: 6 total, 6 up, 6 in 2024-04-17T07:07:19.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:19 smithi086 ceph-mon[31733]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:19 smithi098 ceph-mon[35418]: osdmap e40: 6 total, 6 up, 6 in 2024-04-17T07:07:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:19 smithi098 ceph-mon[35418]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:20.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-04-17T07:07:20.464 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:07:20.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:20 smithi086 ceph-mon[31733]: osdmap e41: 6 total, 6 up, 6 in 2024-04-17T07:07:20.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:20.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:20.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:20.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:20.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:20.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:20.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:20.937 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:20 smithi098 ceph-mon[35418]: osdmap e41: 6 total, 6 up, 6 in 2024-04-17T07:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:21.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:20 smithi103 ceph-mon[35495]: osdmap e41: 6 total, 6 up, 6 in 2024-04-17T07:07:21.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:21.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:21.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:21.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:21.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:21.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:21.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:21.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:21.814 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:21 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi103 2024-04-17T07:07:21.814 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:21 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi103 to 8544M 2024-04-17T07:07:21.814 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:21 smithi086 ceph-mon[31733]: osdmap e42: 6 total, 6 up, 6 in 2024-04-17T07:07:21.814 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:21 smithi086 ceph-mon[31733]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-17T07:07:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:21 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi103 2024-04-17T07:07:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:21 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi103 to 8544M 2024-04-17T07:07:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:21 smithi098 ceph-mon[35418]: osdmap e42: 6 total, 6 up, 6 in 2024-04-17T07:07:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:21 smithi098 ceph-mon[35418]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-17T07:07:22.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:21 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi103 2024-04-17T07:07:22.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:21 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi103 to 8544M 2024-04-17T07:07:22.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:21 smithi103 ceph-mon[35495]: osdmap e42: 6 total, 6 up, 6 in 2024-04-17T07:07:22.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:21 smithi103 ceph-mon[35495]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 197 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-17T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.666+0000 7f5ed9400700 1 -- 172.21.15.86:0/123702284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed41090f0 msgr2=0x7f5ed4109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.666+0000 7f5ed9400700 1 --2- 172.21.15.86:0/123702284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed41090f0 0x7f5ed4109510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40099c0 tx=0x7f5ec4009cd0 comp rx=0 tx=0).stop 2024-04-17T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.666+0000 7f5ed9400700 1 -- 172.21.15.86:0/123702284 shutdown_connections 2024-04-17T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.666+0000 7f5ed9400700 1 --2- 172.21.15.86:0/123702284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed410ac90 0x7f5ed4111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.666+0000 7f5ed9400700 1 --2- 172.21.15.86:0/123702284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed41090f0 0x7f5ed4109510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.666+0000 7f5ed9400700 1 --2- 172.21.15.86:0/123702284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ed410a2f0 0x7f5ed410a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:22.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.666+0000 7f5ed9400700 1 -- 172.21.15.86:0/123702284 >> 172.21.15.86:0/123702284 conn(0x7f5ed40780c0 msgr2=0x7f5ed407a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.667+0000 7f5ed9400700 1 -- 172.21.15.86:0/123702284 shutdown_connections 2024-04-17T07:07:22.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.667+0000 7f5ed9400700 1 -- 172.21.15.86:0/123702284 wait complete. 2024-04-17T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.668+0000 7f5ed9400700 1 Processor -- start 2024-04-17T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.668+0000 7f5ed9400700 1 -- start start 2024-04-17T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.668+0000 7f5ed9400700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed41090f0 0x7f5ed419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:07:22.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.668+0000 7f5ed9400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed410a2f0 0x7f5ed419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.668+0000 7f5ed9400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ed410ac90 0x7f5ed41a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.668+0000 7f5ed9400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4114bb0 con 0x7f5ed410a2f0 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.668+0000 7f5ed9400700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4114a30 con 0x7f5ed41090f0 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.668+0000 7f5ed9400700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4114d30 con 0x7f5ed410ac90 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ecbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed410a2f0 0x7f5ed419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ecbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed410a2f0 0x7f5ed419f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38100/0 (socket says 172.21.15.86:38100) 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ecbe00700 1 -- 172.21.15.86:0/3488478896 learned_addr learned my addr 172.21.15.86:0/3488478896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ed2800700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed41090f0 0x7f5ed419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ed3200700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ed410ac90 0x7f5ed41a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ecbe00700 1 -- 172.21.15.86:0/3488478896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ed410ac90 msgr2=0x7f5ed41a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ecbe00700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ed410ac90 0x7f5ed41a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ecbe00700 1 -- 172.21.15.86:0/3488478896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed41090f0 msgr2=0x7f5ed419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:07:22.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ecbe00700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed41090f0 0x7f5ed419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:22.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ecbe00700 1 -- 172.21.15.86:0/3488478896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41a6e80 con 0x7f5ed410a2f0 2024-04-17T07:07:22.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.669+0000 7f5ecbe00700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed410a2f0 0x7f5ed419f7b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d980 tx=0x7f5ebc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:07:22.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.670+0000 7f5ecb400700 1 -- 172.21.15.86:0/3488478896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc016070 con 0x7f5ed410a2f0 2024-04-17T07:07:22.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.670+0000 7f5ecb400700 1 -- 172.21.15.86:0/3488478896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ebc010460 con 0x7f5ed410a2f0 2024-04-17T07:07:22.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.670+0000 7f5ecb400700 1 -- 172.21.15.86:0/3488478896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00f620 con 0x7f5ed410a2f0 2024-04-17T07:07:22.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.670+0000 7f5ed9400700 1 -- 172.21.15.86:0/3488478896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41a7170 con 0x7f5ed410a2f0 2024-04-17T07:07:22.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.670+0000 7f5ed9400700 1 -- 172.21.15.86:0/3488478896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41a7700 con 0x7f5ed410a2f0 2024-04-17T07:07:22.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.671+0000 7f5ed9400700 1 -- 172.21.15.86:0/3488478896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed4069740 con 0x7f5ed410a2f0 2024-04-17T07:07:22.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.675+0000 7f5ecb400700 1 -- 172.21.15.86:0/3488478896 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5ebc019070 con 0x7f5ed410a2f0 2024-04-17T07:07:22.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.676+0000 7f5ecb400700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5eb80a3cc0 0x7f5eb80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:07:22.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.676+0000 7f5ecb400700 1 -- 172.21.15.86:0/3488478896 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f5ebc0cf100 con 0x7f5ed410a2f0 2024-04-17T07:07:22.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.676+0000 7f5ed2800700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5eb80a3cc0 0x7f5eb80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:07:22.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.677+0000 7f5ed2800700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5eb80a3cc0 0x7f5eb80a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4000c00 tx=0x7f5ec4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:07:22.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.679+0000 7f5ecb400700 1 -- 172.21.15.86:0/3488478896 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5ebc09c900 con 0x7f5ed410a2f0 2024-04-17T07:07:22.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:22.916+0000 7f5ed9400700 1 -- 172.21.15.86:0/3488478896 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5ed4062e90 con 0x7f5eb80a3cc0 2024-04-17T07:07:23.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:22 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:07:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:22 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:07:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:22 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:23.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:22 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:07:23.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:22 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:07:23.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:22 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:23.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:07:23.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:07:23.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:24.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:24 smithi086 ceph-mon[31733]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:07:24.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:24 smithi086 ceph-mon[31733]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:07:24.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:24 smithi103 ceph-mon[35495]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:07:24.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:24 smithi103 ceph-mon[35495]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:07:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:24 smithi098 ceph-mon[35418]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:07:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:24 smithi098 ceph-mon[35418]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:07:26.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:26 smithi103 ceph-mon[35495]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:07:26.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:26 smithi086 ceph-mon[31733]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:07:26.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:26 smithi098 ceph-mon[35418]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:07:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:27 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2024196862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c695ed77-6e45-4218-88cd-40094d317983"}]: dispatch 2024-04-17T07:07:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:27 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2024196862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c695ed77-6e45-4218-88cd-40094d317983"}]': finished 2024-04-17T07:07:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:27 smithi086 ceph-mon[31733]: osdmap e43: 7 total, 6 up, 7 in 2024-04-17T07:07:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:27.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:27 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2024196862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c695ed77-6e45-4218-88cd-40094d317983"}]: dispatch 2024-04-17T07:07:27.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:27 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2024196862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c695ed77-6e45-4218-88cd-40094d317983"}]': finished 2024-04-17T07:07:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:27 smithi098 ceph-mon[35418]: osdmap e43: 7 total, 6 up, 7 in 2024-04-17T07:07:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:27.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:27 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2024196862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c695ed77-6e45-4218-88cd-40094d317983"}]: dispatch 2024-04-17T07:07:27.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:27 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2024196862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c695ed77-6e45-4218-88cd-40094d317983"}]': finished 2024-04-17T07:07:27.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:27 smithi103 ceph-mon[35495]: osdmap e43: 7 total, 6 up, 7 in 2024-04-17T07:07:27.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:28 smithi086 ceph-mon[31733]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-17T07:07:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:28 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/106077400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:07:28.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:28 smithi098 ceph-mon[35418]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-17T07:07:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:28 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/106077400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:07:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:28 smithi103 ceph-mon[35495]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-17T07:07:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:28 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/106077400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:07:30.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:30 smithi086 ceph-mon[31733]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:30.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:30 smithi098 ceph-mon[35418]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:30 smithi103 ceph-mon[35495]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:32.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:32 smithi086 ceph-mon[31733]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:32 smithi098 ceph-mon[35418]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:32 smithi103 ceph-mon[35495]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:34.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:34 smithi086 ceph-mon[31733]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:34 smithi098 ceph-mon[35418]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:34 smithi103 ceph-mon[35495]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:36.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:36 smithi086 ceph-mon[31733]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:36.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:36 smithi098 ceph-mon[35418]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:36.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:36 smithi103 ceph-mon[35495]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-17T07:07:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-17T07:07:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-17T07:07:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:38.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:38 smithi086 ceph-mon[31733]: Deploying daemon osd.6 on smithi086 2024-04-17T07:07:38.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:38 smithi086 ceph-mon[31733]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:38.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:38 smithi098 ceph-mon[35418]: Deploying daemon osd.6 on smithi086 2024-04-17T07:07:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:38 smithi098 ceph-mon[35418]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:38 smithi103 ceph-mon[35495]: Deploying daemon osd.6 on smithi086 2024-04-17T07:07:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:38 smithi103 ceph-mon[35495]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:40.329 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:40 smithi103 ceph-mon[35495]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:40.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:40 smithi086 ceph-mon[31733]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:40.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:40 smithi098 ceph-mon[35418]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:42.193 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:42 smithi086 ceph-mon[31733]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:42.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:42 smithi103 ceph-mon[35495]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:42.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:42 smithi098 ceph-mon[35418]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:44.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:44 smithi086 ceph-mon[31733]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:44.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:44 smithi098 ceph-mon[35418]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:44.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:44 smithi103 ceph-mon[35495]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:46.239 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:46.239 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:45 smithi086 ceph-mon[31733]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:46.239 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:46.239 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:46.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:46.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:45 smithi098 ceph-mon[35418]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:45 smithi103 ceph-mon[35495]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:48.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:48 smithi086 ceph-mon[31733]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:48 smithi098 ceph-mon[35418]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:48 smithi103 ceph-mon[35495]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:50.006 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 6 on host 'smithi086' 2024-04-17T07:07:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.005+0000 7f5ecb400700 1 -- 172.21.15.86:0/3488478896 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5ed4062e90 con 0x7f5eb80a3cc0 2024-04-17T07:07:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.009+0000 7f5ed9400700 1 -- 172.21.15.86:0/3488478896 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5eb80a3cc0 msgr2=0x7f5eb80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:07:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.009+0000 7f5ed9400700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5eb80a3cc0 0x7f5eb80a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4000c00 tx=0x7f5ec4005fb0 comp rx=0 tx=0).stop 2024-04-17T07:07:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.009+0000 7f5ed9400700 1 -- 172.21.15.86:0/3488478896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed410a2f0 msgr2=0x7f5ed419f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:07:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.009+0000 7f5ed9400700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed410a2f0 0x7f5ed419f7b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d980 tx=0x7f5ebc00dd40 comp rx=0 tx=0).stop 2024-04-17T07:07:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.009+0000 7f5ed9400700 1 -- 172.21.15.86:0/3488478896 shutdown_connections 2024-04-17T07:07:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.009+0000 7f5ed9400700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5eb80a3cc0 0x7f5eb80a6180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.009+0000 7f5ed9400700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ed410ac90 0x7f5ed41a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.009+0000 7f5ed9400700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed41090f0 0x7f5ed419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.009+0000 7f5ed9400700 1 --2- 172.21.15.86:0/3488478896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed410a2f0 0x7f5ed419f7b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.009+0000 7f5ed9400700 1 -- 172.21.15.86:0/3488478896 >> 172.21.15.86:0/3488478896 conn(0x7f5ed40780c0 msgr2=0x7f5ed410d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:07:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.010+0000 7f5ed9400700 1 -- 172.21.15.86:0/3488478896 shutdown_connections 2024-04-17T07:07:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:07:50.010+0000 7f5ed9400700 1 -- 172.21.15.86:0/3488478896 wait complete. 2024-04-17T07:07:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:49 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:49 smithi098 ceph-mon[35418]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-17T07:07:50.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:49 smithi098 ceph-mon[35418]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:49 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:50.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:49 smithi103 ceph-mon[35495]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-17T07:07:50.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:49 smithi103 ceph-mon[35495]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:49 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:49 smithi086 ceph-mon[31733]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-17T07:07:50.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:49 smithi086 ceph-mon[31733]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:50.822 DEBUG:teuthology.orchestra.run.smithi086:osd.6> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.6.service 2024-04-17T07:07:50.827 INFO:tasks.cephadm:Deploying osd.7 on smithi098 with /dev/vg_nvme/lv_2... 2024-04-17T07:07:50.827 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-17T07:07:51.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:50 smithi086 ceph-mon[31733]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-17T07:07:51.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:50 smithi086 ceph-mon[31733]: osdmap e44: 7 total, 6 up, 7 in 2024-04-17T07:07:51.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:50 smithi086 ceph-mon[31733]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:07:51.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:51.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:51.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:51.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:51.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:50 smithi098 ceph-mon[35418]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-17T07:07:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:50 smithi098 ceph-mon[35418]: osdmap e44: 7 total, 6 up, 7 in 2024-04-17T07:07:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:50 smithi098 ceph-mon[35418]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:07:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:51.271 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:07:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:50 smithi103 ceph-mon[35495]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-17T07:07:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:50 smithi103 ceph-mon[35495]: osdmap e44: 7 total, 6 up, 7 in 2024-04-17T07:07:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:50 smithi103 ceph-mon[35495]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:07:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:07:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:51.832 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:07:51 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:07:51.524+0000 7f5909400700 -1 osd.6 0 waiting for initial osdmap 2024-04-17T07:07:51.832 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:07:51 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:07:51.565+0000 7f58ff200700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:07:51.924 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:51 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:07:51.924 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:51 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:07:51.924 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:51 smithi098 ceph-mon[35418]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:07:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:51 smithi098 ceph-mon[35418]: osdmap e45: 7 total, 6 up, 7 in 2024-04-17T07:07:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:51 smithi098 ceph-mon[35418]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:51 smithi098 ceph-mon[35418]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' 2024-04-17T07:07:52.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:51 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:07:52.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:51 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:07:52.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:51 smithi103 ceph-mon[35495]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:07:52.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:51 smithi103 ceph-mon[35495]: osdmap e45: 7 total, 6 up, 7 in 2024-04-17T07:07:52.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:52.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:52.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:51 smithi103 ceph-mon[35495]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:52.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:51 smithi103 ceph-mon[35495]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' 2024-04-17T07:07:52.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:51 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:07:52.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:51 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:07:52.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:51 smithi086 ceph-mon[31733]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:07:52.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:51 smithi086 ceph-mon[31733]: osdmap e45: 7 total, 6 up, 7 in 2024-04-17T07:07:52.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:52.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:52.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:51 smithi086 ceph-mon[31733]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-17T07:07:52.302 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:51 smithi086 ceph-mon[31733]: from='osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330]' entity='osd.6' 2024-04-17T07:07:53.182 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:52 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:53.182 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:52 smithi098 ceph-mon[35418]: osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] boot 2024-04-17T07:07:53.182 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:52 smithi098 ceph-mon[35418]: osdmap e46: 7 total, 7 up, 7 in 2024-04-17T07:07:53.182 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:52 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:52 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:52 smithi103 ceph-mon[35495]: osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] boot 2024-04-17T07:07:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:52 smithi103 ceph-mon[35495]: osdmap e46: 7 total, 7 up, 7 in 2024-04-17T07:07:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:52 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:52 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:52 smithi086 ceph-mon[31733]: osd.6 [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] boot 2024-04-17T07:07:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:52 smithi086 ceph-mon[31733]: osdmap e46: 7 total, 7 up, 7 in 2024-04-17T07:07:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:52 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-17T07:07:54.121 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:53 smithi098 ceph-mon[35418]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:07:54.121 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-17T07:07:54.121 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-17T07:07:54.121 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-17T07:07:54.121 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-17T07:07:54.121 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0317994 s, 330 MB/s 2024-04-17T07:07:54.122 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-17T07:07:54.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:53 smithi103 ceph-mon[35495]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:07:54.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:53 smithi086 ceph-mon[31733]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:07:54.912 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_2 2024-04-17T07:07:55.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:54 smithi098 ceph-mon[35418]: osdmap e47: 7 total, 7 up, 7 in 2024-04-17T07:07:55.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:54 smithi103 ceph-mon[35495]: osdmap e47: 7 total, 7 up, 7 in 2024-04-17T07:07:55.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:54 smithi086 ceph-mon[31733]: osdmap e47: 7 total, 7 up, 7 in 2024-04-17T07:07:55.358 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:07:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:55 smithi098 ceph-mon[35418]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:07:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:56.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:55 smithi103 ceph-mon[35495]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:07:56.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:56.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:56.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:56.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:56.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:56.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:55 smithi086 ceph-mon[31733]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:07:56.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:56.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:56.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:07:56.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:56.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:56.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:07:56.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:07:57.109 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:56 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi086 2024-04-17T07:07:57.109 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:56 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-17T07:07:57.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:56 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi086 2024-04-17T07:07:57.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:56 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-17T07:07:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:56 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi086 2024-04-17T07:07:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:56 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-17T07:07:57.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.618+0000 7f9bd4e00700 1 -- 172.21.15.98:0/2154609855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd010ac90 msgr2=0x7f9bd0111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:07:57.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.618+0000 7f9bd4e00700 1 --2- 172.21.15.98:0/2154609855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd010ac90 0x7f9bd0111540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800b0d0 tx=0x7f9bc800b3e0 comp rx=0 tx=0).stop 2024-04-17T07:07:57.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.618+0000 7f9bd4e00700 1 -- 172.21.15.98:0/2154609855 shutdown_connections 2024-04-17T07:07:57.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.618+0000 7f9bd4e00700 1 --2- 172.21.15.98:0/2154609855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd010ac90 0x7f9bd0111540 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:57.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.618+0000 7f9bd4e00700 1 --2- 172.21.15.98:0/2154609855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd01090f0 0x7f9bd0109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:57.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.618+0000 7f9bd4e00700 1 --2- 172.21.15.98:0/2154609855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bd010a2f0 0x7f9bd010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:57.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.618+0000 7f9bd4e00700 1 -- 172.21.15.98:0/2154609855 >> 172.21.15.98:0/2154609855 conn(0x7f9bd00780c0 msgr2=0x7f9bd007a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:07:57.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.618+0000 7f9bd4e00700 1 -- 172.21.15.98:0/2154609855 shutdown_connections 2024-04-17T07:07:57.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.618+0000 7f9bd4e00700 1 -- 172.21.15.98:0/2154609855 wait complete. 2024-04-17T07:07:57.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.619+0000 7f9bd4e00700 1 Processor -- start 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.619+0000 7f9bd4e00700 1 -- start start 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.619+0000 7f9bd4e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd01090f0 0x7f9bd019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.619+0000 7f9bd4e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bd010a2f0 0x7f9bd019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.619+0000 7f9bd4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd010ac90 0x7f9bd01a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.619+0000 7f9bd4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0114b70 con 0x7f9bd010ac90 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.619+0000 7f9bd4e00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01149f0 con 0x7f9bd01090f0 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bd4e00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0114cf0 con 0x7f9bd010a2f0 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bcde00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd01090f0 0x7f9bd019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bcde00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd01090f0 0x7f9bd019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44816/0 (socket says 172.21.15.98:44816) 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bcde00700 1 -- 172.21.15.98:0/816495371 learned_addr learned my addr 172.21.15.98:0/816495371 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bcd400700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bd010a2f0 0x7f9bd019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bce800700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd010ac90 0x7f9bd01a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:07:57.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bcde00700 1 -- 172.21.15.98:0/816495371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bd010a2f0 msgr2=0x7f9bd019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bcde00700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bd010a2f0 0x7f9bd019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bcde00700 1 -- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd010ac90 msgr2=0x7f9bd01a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bcde00700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd010ac90 0x7f9bd01a65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bcde00700 1 -- 172.21.15.98:0/816495371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd01a6c70 con 0x7f9bd01090f0 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.620+0000 7f9bce800700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd010ac90 0x7f9bd01a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.621+0000 7f9bcde00700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd01090f0 0x7f9bd019f010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000c990 tx=0x7f9bc000cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.621+0000 7f9bc6a00700 1 -- 172.21.15.98:0/816495371 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0007b00 con 0x7f9bd01090f0 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.621+0000 7f9bc6a00700 1 -- 172.21.15.98:0/816495371 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc0007ca0 con 0x7f9bd01090f0 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.621+0000 7f9bd4e00700 1 -- 172.21.15.98:0/816495371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd01a6f60 con 0x7f9bd01090f0 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.621+0000 7f9bc6a00700 1 -- 172.21.15.98:0/816495371 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0018640 con 0x7f9bd01090f0 2024-04-17T07:07:57.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.621+0000 7f9bd4e00700 1 -- 172.21.15.98:0/816495371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd01a73b0 con 0x7f9bd01090f0 2024-04-17T07:07:57.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.622+0000 7f9bd4e00700 1 -- 172.21.15.98:0/816495371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd002df90 con 0x7f9bd01090f0 2024-04-17T07:07:57.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.626+0000 7f9bc6a00700 1 -- 172.21.15.98:0/816495371 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9bc000f450 con 0x7f9bd01090f0 2024-04-17T07:07:57.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.627+0000 7f9bc6a00700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9bb40a3c40 0x7f9bb40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:07:57.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.628+0000 7f9bc6a00700 1 -- 172.21.15.98:0/816495371 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f9bc00cf800 con 0x7f9bd01090f0 2024-04-17T07:07:57.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.628+0000 7f9bcd400700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9bb40a3c40 0x7f9bb40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:07:57.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.629+0000 7f9bcd400700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9bb40a3c40 0x7f9bb40a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8005ea0 tx=0x7f9bb8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:07:57.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.631+0000 7f9bc6a00700 1 -- 172.21.15.98:0/816495371 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9bc0014030 con 0x7f9bd01090f0 2024-04-17T07:07:57.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:07:57.865+0000 7f9bd4e00700 1 -- 172.21.15.98:0/816495371 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9bd00638c0 con 0x7f9bb40a3c40 2024-04-17T07:07:58.232 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:57 smithi086 ceph-mon[31733]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:07:58.232 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:07:58.232 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:07:58.232 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:58.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:57 smithi098 ceph-mon[35418]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:07:58.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:07:58.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:07:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:58.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:57 smithi103 ceph-mon[35495]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:07:58.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:07:58.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:07:58.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:07:59.087 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:07:58 smithi098 ceph-mon[35418]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:07:59.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:07:58 smithi103 ceph-mon[35495]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:07:59.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:07:58 smithi086 ceph-mon[31733]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:08:00.144 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:00 smithi098 ceph-mon[35418]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:00 smithi086 ceph-mon[31733]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:00 smithi103 ceph-mon[35495]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:02.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:02 smithi086 ceph-mon[31733]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:02.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:02 smithi086 ceph-mon[31733]: from='client.? 172.21.15.98:0/2204333174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eacf793e-e041-433e-ba5b-374fb4f1c6d6"}]: dispatch 2024-04-17T07:08:02.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:02 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eacf793e-e041-433e-ba5b-374fb4f1c6d6"}]: dispatch 2024-04-17T07:08:02.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:02 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eacf793e-e041-433e-ba5b-374fb4f1c6d6"}]': finished 2024-04-17T07:08:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:02 smithi098 ceph-mon[35418]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:02 smithi098 ceph-mon[35418]: from='client.? 172.21.15.98:0/2204333174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eacf793e-e041-433e-ba5b-374fb4f1c6d6"}]: dispatch 2024-04-17T07:08:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:02 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eacf793e-e041-433e-ba5b-374fb4f1c6d6"}]: dispatch 2024-04-17T07:08:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:02 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eacf793e-e041-433e-ba5b-374fb4f1c6d6"}]': finished 2024-04-17T07:08:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:02 smithi103 ceph-mon[35495]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:02 smithi103 ceph-mon[35495]: from='client.? 172.21.15.98:0/2204333174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eacf793e-e041-433e-ba5b-374fb4f1c6d6"}]: dispatch 2024-04-17T07:08:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:02 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eacf793e-e041-433e-ba5b-374fb4f1c6d6"}]: dispatch 2024-04-17T07:08:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:02 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eacf793e-e041-433e-ba5b-374fb4f1c6d6"}]': finished 2024-04-17T07:08:03.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:03 smithi086 ceph-mon[31733]: osdmap e48: 8 total, 7 up, 8 in 2024-04-17T07:08:03.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:03.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:03 smithi086 ceph-mon[31733]: from='client.? 172.21.15.98:0/2838508975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:08:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:03 smithi098 ceph-mon[35418]: osdmap e48: 8 total, 7 up, 8 in 2024-04-17T07:08:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:03 smithi098 ceph-mon[35418]: from='client.? 172.21.15.98:0/2838508975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:08:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:03 smithi103 ceph-mon[35495]: osdmap e48: 8 total, 7 up, 8 in 2024-04-17T07:08:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:03 smithi103 ceph-mon[35495]: from='client.? 172.21.15.98:0/2838508975' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:08:04.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:04 smithi086 ceph-mon[31733]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:04 smithi098 ceph-mon[35418]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:04.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:04 smithi103 ceph-mon[35495]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:06.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:06 smithi086 ceph-mon[31733]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:06.335 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:06 smithi098 ceph-mon[35418]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:06.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:06 smithi103 ceph-mon[35495]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:08 smithi086 ceph-mon[31733]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:08.332 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:08 smithi098 ceph-mon[35418]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:08.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:08 smithi103 ceph-mon[35495]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:10.329 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:10 smithi103 ceph-mon[35495]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:10.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:10 smithi086 ceph-mon[31733]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:10 smithi098 ceph-mon[35418]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:12.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:12 smithi086 ceph-mon[31733]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:12.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:12 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-17T07:08:12.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:12 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:12.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:12 smithi098 ceph-mon[35418]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:12.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:12 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-17T07:08:12.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:12 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:12 smithi103 ceph-mon[35495]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:12 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-17T07:08:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:12 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:13 smithi098 ceph-mon[35418]: Deploying daemon osd.7 on smithi098 2024-04-17T07:08:13.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:13 smithi103 ceph-mon[35495]: Deploying daemon osd.7 on smithi098 2024-04-17T07:08:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:13 smithi086 ceph-mon[31733]: Deploying daemon osd.7 on smithi098 2024-04-17T07:08:14.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:14 smithi098 ceph-mon[35418]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:14.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:14 smithi103 ceph-mon[35495]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:14 smithi086 ceph-mon[31733]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:16.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:16 smithi098 ceph-mon[35418]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:16 smithi103 ceph-mon[35495]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:16.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:16 smithi086 ceph-mon[31733]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:18.471 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:18 smithi098 ceph-mon[35418]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:18.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:18 smithi103 ceph-mon[35495]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:18.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:18 smithi086 ceph-mon[31733]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:20.363 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:20 smithi098 ceph-mon[35418]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:20.363 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:20.363 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:20.364 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:20 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:08:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:20 smithi103 ceph-mon[35495]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:20 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:08:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:20 smithi086 ceph-mon[31733]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:20 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:08:22.231 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:22 smithi098 ceph-mon[35418]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:22.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:22 smithi103 ceph-mon[35495]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:22.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:22 smithi086 ceph-mon[31733]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:24.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:24 smithi098 ceph-mon[35418]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:24 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:24 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:24 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:24 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:24 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:24 smithi098 ceph-mon[35418]: from='osd.7 [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:08:24.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:24 smithi098 ceph-mon[35418]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:08:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:24 smithi103 ceph-mon[35495]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:24 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:24 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:24.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:24 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:24.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:24 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:08:24.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:24 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:24.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:24 smithi103 ceph-mon[35495]: from='osd.7 [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:08:24.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:24 smithi103 ceph-mon[35495]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:08:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:24 smithi086 ceph-mon[31733]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:24 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:24 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:24 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:24 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:08:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:24 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:24 smithi086 ceph-mon[31733]: from='osd.7 [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:08:24.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:24 smithi086 ceph-mon[31733]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-17T07:08:24.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.883+0000 7f9bc6a00700 1 -- 172.21.15.98:0/816495371 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9bd00638c0 con 0x7f9bb40a3c40 2024-04-17T07:08:24.884 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 7 on host 'smithi098' 2024-04-17T07:08:24.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 -- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9bb40a3c40 msgr2=0x7f9bb40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:08:24.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9bb40a3c40 0x7f9bb40a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8005ea0 tx=0x7f9bb8005e30 comp rx=0 tx=0).stop 2024-04-17T07:08:24.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 -- 172.21.15.98:0/816495371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd01090f0 msgr2=0x7f9bd019f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:08:24.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd01090f0 0x7f9bd019f010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000c990 tx=0x7f9bc000cd50 comp rx=0 tx=0).stop 2024-04-17T07:08:24.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 -- 172.21.15.98:0/816495371 shutdown_connections 2024-04-17T07:08:24.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9bb40a3c40 0x7f9bb40a6100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:24.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bd010ac90 0x7f9bd01a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:24.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd01090f0 0x7f9bd019f010 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:24.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 --2- 172.21.15.98:0/816495371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bd010a2f0 0x7f9bd019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:24.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 -- 172.21.15.98:0/816495371 >> 172.21.15.98:0/816495371 conn(0x7f9bd00780c0 msgr2=0x7f9bd010d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:08:24.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 -- 172.21.15.98:0/816495371 shutdown_connections 2024-04-17T07:08:24.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:08:24.887+0000 7f9bd4e00700 1 -- 172.21.15.98:0/816495371 wait complete. 2024-04-17T07:08:25.691 DEBUG:teuthology.orchestra.run.smithi098:osd.7> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.7.service 2024-04-17T07:08:25.696 INFO:tasks.cephadm:Deploying osd.8 on smithi103 with /dev/vg_nvme/lv_2... 2024-04-17T07:08:25.696 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-17T07:08:25.887 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:25.887 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:08:25.887 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:25.888 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-mon[35418]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-17T07:08:25.888 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-mon[35418]: osdmap e49: 8 total, 7 up, 8 in 2024-04-17T07:08:25.888 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-mon[35418]: from='osd.7 [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:08:25.888 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:25.888 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-mon[35418]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:08:25.888 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-mon[35418]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:26.134 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:08:26.230 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:08:25.905+0000 7f2668c00700 -1 osd.7 0 waiting for initial osdmap 2024-04-17T07:08:26.231 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:08:25 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:08:25.939+0000 7f2663a00700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:08:26.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:26.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:08:26.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:26.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:25 smithi103 ceph-mon[35495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-17T07:08:26.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:25 smithi103 ceph-mon[35495]: osdmap e49: 8 total, 7 up, 8 in 2024-04-17T07:08:26.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:25 smithi103 ceph-mon[35495]: from='osd.7 [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:08:26.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:26.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:25 smithi103 ceph-mon[35495]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:08:26.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:25 smithi103 ceph-mon[35495]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:26.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:26.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:08:26.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:26.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:25 smithi086 ceph-mon[31733]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-17T07:08:26.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:25 smithi086 ceph-mon[31733]: osdmap e49: 8 total, 7 up, 8 in 2024-04-17T07:08:26.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:25 smithi086 ceph-mon[31733]: from='osd.7 [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:08:26.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:26.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:25 smithi086 ceph-mon[31733]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:08:26.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:25 smithi086 ceph-mon[31733]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-17T07:08:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:26 smithi098 ceph-mon[35418]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:08:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:26 smithi098 ceph-mon[35418]: osdmap e50: 8 total, 7 up, 8 in 2024-04-17T07:08:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:26 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:26 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:26 smithi098 ceph-mon[35418]: osd.7 [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] boot 2024-04-17T07:08:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:26 smithi098 ceph-mon[35418]: osdmap e51: 8 total, 8 up, 8 in 2024-04-17T07:08:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:26 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:27.150 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:26 smithi103 ceph-mon[35495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:08:27.150 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:26 smithi103 ceph-mon[35495]: osdmap e50: 8 total, 7 up, 8 in 2024-04-17T07:08:27.150 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:26 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:27.150 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:26 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:27.150 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:26 smithi103 ceph-mon[35495]: osd.7 [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] boot 2024-04-17T07:08:27.150 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:26 smithi103 ceph-mon[35495]: osdmap e51: 8 total, 8 up, 8 in 2024-04-17T07:08:27.151 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:26 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:26 smithi086 ceph-mon[31733]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:08:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:26 smithi086 ceph-mon[31733]: osdmap e50: 8 total, 7 up, 8 in 2024-04-17T07:08:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:26 smithi086 ceph-mon[31733]: osd.7 [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] boot 2024-04-17T07:08:27.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:26 smithi086 ceph-mon[31733]: osdmap e51: 8 total, 8 up, 8 in 2024-04-17T07:08:27.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-17T07:08:28.170 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:27 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:08:28.170 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:27 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:08:28.170 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:27 smithi103 ceph-mon[35495]: pgmap v221: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:28.170 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:27 smithi103 ceph-mon[35495]: osdmap e52: 8 total, 8 up, 8 in 2024-04-17T07:08:28.232 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:27 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:08:28.232 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:27 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:08:28.232 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:27 smithi086 ceph-mon[31733]: pgmap v221: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:28.232 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:27 smithi086 ceph-mon[31733]: osdmap e52: 8 total, 8 up, 8 in 2024-04-17T07:08:28.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:27 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:08:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:27 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:08:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:27 smithi098 ceph-mon[35418]: pgmap v221: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:28.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:27 smithi098 ceph-mon[35418]: osdmap e52: 8 total, 8 up, 8 in 2024-04-17T07:08:29.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:28 smithi103 ceph-mon[35495]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-17T07:08:29.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:28 smithi103 ceph-mon[35495]: osdmap e53: 8 total, 8 up, 8 in 2024-04-17T07:08:29.058 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-17T07:08:29.058 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-17T07:08:29.058 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-17T07:08:29.058 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-17T07:08:29.058 INFO:teuthology.orchestra.run.smithi103.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0341067 s, 307 MB/s 2024-04-17T07:08:29.058 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-17T07:08:29.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:28 smithi098 ceph-mon[35418]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-17T07:08:29.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:28 smithi098 ceph-mon[35418]: osdmap e53: 8 total, 8 up, 8 in 2024-04-17T07:08:29.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:28 smithi086 ceph-mon[31733]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-17T07:08:29.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:28 smithi086 ceph-mon[31733]: osdmap e53: 8 total, 8 up, 8 in 2024-04-17T07:08:29.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-04-17T07:08:30.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:29 smithi098 ceph-mon[35418]: pgmap v224: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:30.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:29 smithi103 ceph-mon[35495]: pgmap v224: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:30.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:29 smithi086 ceph-mon[31733]: pgmap v224: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:30.431 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:08:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:30 smithi098 ceph-mon[35418]: osdmap e54: 8 total, 8 up, 8 in 2024-04-17T07:08:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:08:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:08:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:08:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:08:31.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:30 smithi103 ceph-mon[35495]: osdmap e54: 8 total, 8 up, 8 in 2024-04-17T07:08:31.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:08:31.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:08:31.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:08:31.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:31.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:08:31.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:30 smithi086 ceph-mon[31733]: osdmap e54: 8 total, 8 up, 8 in 2024-04-17T07:08:31.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:08:31.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:08:31.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:08:31.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:31.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:31.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:08:31.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:32.005 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:31 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi098 2024-04-17T07:08:32.005 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:31 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi098 to 5696M 2024-04-17T07:08:32.005 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:31 smithi103 ceph-mon[35495]: osdmap e55: 8 total, 8 up, 8 in 2024-04-17T07:08:32.005 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:31 smithi103 ceph-mon[35495]: pgmap v227: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:31 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi098 2024-04-17T07:08:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:31 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi098 to 5696M 2024-04-17T07:08:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:31 smithi098 ceph-mon[35418]: osdmap e55: 8 total, 8 up, 8 in 2024-04-17T07:08:32.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:31 smithi098 ceph-mon[35418]: pgmap v227: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:32.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:31 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi098 2024-04-17T07:08:32.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:31 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi098 to 5696M 2024-04-17T07:08:32.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:31 smithi086 ceph-mon[31733]: osdmap e55: 8 total, 8 up, 8 in 2024-04-17T07:08:32.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:31 smithi086 ceph-mon[31733]: pgmap v227: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.649+0000 7f92f1400700 1 -- 172.21.15.103:0/356717519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92ec10ac90 msgr2=0x7f92ec111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:08:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.649+0000 7f92f1400700 1 --2- 172.21.15.103:0/356717519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92ec10ac90 0x7f92ec111540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f92e400b0a0 tx=0x7f92e400b3b0 comp rx=0 tx=0).stop 2024-04-17T07:08:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.649+0000 7f92f1400700 1 -- 172.21.15.103:0/356717519 shutdown_connections 2024-04-17T07:08:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.649+0000 7f92f1400700 1 --2- 172.21.15.103:0/356717519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92ec10ac90 0x7f92ec111540 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.649+0000 7f92f1400700 1 --2- 172.21.15.103:0/356717519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92ec1090f0 0x7f92ec109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.649+0000 7f92f1400700 1 --2- 172.21.15.103:0/356717519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec10a2f0 0x7f92ec10a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.649+0000 7f92f1400700 1 -- 172.21.15.103:0/356717519 >> 172.21.15.103:0/356717519 conn(0x7f92ec0780c0 msgr2=0x7f92ec07a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:08:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.649+0000 7f92f1400700 1 -- 172.21.15.103:0/356717519 shutdown_connections 2024-04-17T07:08:32.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.649+0000 7f92f1400700 1 -- 172.21.15.103:0/356717519 wait complete. 2024-04-17T07:08:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.650+0000 7f92f1400700 1 Processor -- start 2024-04-17T07:08:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.650+0000 7f92f1400700 1 -- start start 2024-04-17T07:08:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92f1400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec1090f0 0x7f92ec19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:08:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92f1400700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92ec10a2f0 0x7f92ec19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:08:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92f1400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92ec10ac90 0x7f92ec1a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:08:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92f1400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec114bf0 con 0x7f92ec1090f0 2024-04-17T07:08:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92f1400700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec114a70 con 0x7f92ec10a2f0 2024-04-17T07:08:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92f1400700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec114d70 con 0x7f92ec10ac90 2024-04-17T07:08:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92eb200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92ec10ac90 0x7f92ec1a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:08:32.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92eb200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92ec10ac90 0x7f92ec1a6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38346/0 (socket says 172.21.15.103:38346) 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92eb200700 1 -- 172.21.15.103:0/1261368828 learned_addr learned my addr 172.21.15.103:0/1261368828 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92ea800700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec1090f0 0x7f92ec19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.651+0000 7f92e9e00700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92ec10a2f0 0x7f92ec19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92eb200700 1 -- 172.21.15.103:0/1261368828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92ec10a2f0 msgr2=0x7f92ec19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92eb200700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92ec10a2f0 0x7f92ec19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92eb200700 1 -- 172.21.15.103:0/1261368828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec1090f0 msgr2=0x7f92ec19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92eb200700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec1090f0 0x7f92ec19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92eb200700 1 -- 172.21.15.103:0/1261368828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec1a6da0 con 0x7f92ec10ac90 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92eb200700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92ec10ac90 0x7f92ec1a6700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f92e4007fa0 tx=0x7f92e400cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92e3400700 1 -- 172.21.15.103:0/1261368828 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4021020 con 0x7f92ec10ac90 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92e3400700 1 -- 172.21.15.103:0/1261368828 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92e401d920 con 0x7f92ec10ac90 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92f1400700 1 -- 172.21.15.103:0/1261368828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec1a7030 con 0x7f92ec10ac90 2024-04-17T07:08:32.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92e3400700 1 -- 172.21.15.103:0/1261368828 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4005640 con 0x7f92ec10ac90 2024-04-17T07:08:32.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.652+0000 7f92f1400700 1 -- 172.21.15.103:0/1261368828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec1a74e0 con 0x7f92ec10ac90 2024-04-17T07:08:32.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.654+0000 7f92f1400700 1 -- 172.21.15.103:0/1261368828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec069740 con 0x7f92ec10ac90 2024-04-17T07:08:32.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.656+0000 7f92e3400700 1 -- 172.21.15.103:0/1261368828 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f92e4024020 con 0x7f92ec10ac90 2024-04-17T07:08:32.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.657+0000 7f92e3400700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92d00a3cc0 0x7f92d00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:08:32.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.657+0000 7f92e3400700 1 -- 172.21.15.103:0/1261368828 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f92e40d15a0 con 0x7f92ec10ac90 2024-04-17T07:08:32.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.658+0000 7f92ea800700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92d00a3cc0 0x7f92d00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:08:32.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.659+0000 7f92ea800700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92d00a3cc0 0x7f92d00a6180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92dc005fd0 tx=0x7f92dc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:08:32.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.660+0000 7f92e3400700 1 -- 172.21.15.103:0/1261368828 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f92e402b020 con 0x7f92ec10ac90 2024-04-17T07:08:32.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:32.897+0000 7f92f1400700 1 -- 172.21.15.103:0/1261368828 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f92ec062f50 con 0x7f92d00a3cc0 2024-04-17T07:08:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:32 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:08:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:32 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:08:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:32 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:33.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:32 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:08:33.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:32 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:08:33.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:32 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:33.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:32 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:08:33.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:32 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:08:33.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:32 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:34.129 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:33 smithi103 ceph-mon[35495]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:08:34.130 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:33 smithi103 ceph-mon[35495]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:34.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:33 smithi098 ceph-mon[35418]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:08:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:33 smithi098 ceph-mon[35418]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:34.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:33 smithi086 ceph-mon[31733]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:08:34.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:33 smithi086 ceph-mon[31733]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:35.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:34 smithi098 ceph-mon[35418]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-17T07:08:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:34 smithi098 ceph-mon[35418]: Cluster is now healthy 2024-04-17T07:08:35.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:34 smithi103 ceph-mon[35495]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-17T07:08:35.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:34 smithi103 ceph-mon[35495]: Cluster is now healthy 2024-04-17T07:08:35.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:34 smithi086 ceph-mon[31733]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-17T07:08:35.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:34 smithi086 ceph-mon[31733]: Cluster is now healthy 2024-04-17T07:08:36.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:35 smithi103 ceph-mon[35495]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:36.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:35 smithi098 ceph-mon[35418]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:36.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:35 smithi086 ceph-mon[31733]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:36 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/1846979139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c7c6330-3ffb-45e1-b9b5-77db0f63a735"}]: dispatch 2024-04-17T07:08:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:36 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c7c6330-3ffb-45e1-b9b5-77db0f63a735"}]: dispatch 2024-04-17T07:08:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:36 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c7c6330-3ffb-45e1-b9b5-77db0f63a735"}]': finished 2024-04-17T07:08:37.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:36 smithi098 ceph-mon[35418]: osdmap e56: 9 total, 8 up, 9 in 2024-04-17T07:08:37.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:36 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/1846979139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c7c6330-3ffb-45e1-b9b5-77db0f63a735"}]: dispatch 2024-04-17T07:08:37.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:36 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c7c6330-3ffb-45e1-b9b5-77db0f63a735"}]: dispatch 2024-04-17T07:08:37.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:36 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c7c6330-3ffb-45e1-b9b5-77db0f63a735"}]': finished 2024-04-17T07:08:37.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:36 smithi103 ceph-mon[35495]: osdmap e56: 9 total, 8 up, 9 in 2024-04-17T07:08:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:36 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/1846979139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c7c6330-3ffb-45e1-b9b5-77db0f63a735"}]: dispatch 2024-04-17T07:08:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:36 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c7c6330-3ffb-45e1-b9b5-77db0f63a735"}]: dispatch 2024-04-17T07:08:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:36 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c7c6330-3ffb-45e1-b9b5-77db0f63a735"}]': finished 2024-04-17T07:08:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:36 smithi086 ceph-mon[31733]: osdmap e56: 9 total, 8 up, 9 in 2024-04-17T07:08:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:08:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:37 smithi098 ceph-mon[35418]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:37 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/854370294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:08:38.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:08:38.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:37 smithi103 ceph-mon[35495]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:38.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:37 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/854370294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:08:38.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:08:38.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:37 smithi086 ceph-mon[31733]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:38.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:37 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/854370294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:08:40.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:40 smithi086 ceph-mon[31733]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:40.384 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:40 smithi103 ceph-mon[35495]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:40.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:40 smithi098 ceph-mon[35418]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:42.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:42 smithi103 ceph-mon[35495]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:42.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:42 smithi086 ceph-mon[31733]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:42.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:42 smithi098 ceph-mon[35418]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:44.248 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:44 smithi103 ceph-mon[35495]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:44.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:44 smithi086 ceph-mon[31733]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:44.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:44 smithi098 ceph-mon[35418]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:46.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:46 smithi103 ceph-mon[35495]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:46.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-17T07:08:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:46 smithi086 ceph-mon[31733]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-17T07:08:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:46 smithi098 ceph-mon[35418]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-17T07:08:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:08:47.270 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:47 smithi103 ceph-mon[35495]: Deploying daemon osd.8 on smithi103 2024-04-17T07:08:47.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:47 smithi086 ceph-mon[31733]: Deploying daemon osd.8 on smithi103 2024-04-17T07:08:47.340 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:47 smithi098 ceph-mon[35418]: Deploying daemon osd.8 on smithi103 2024-04-17T07:08:48.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:48 smithi098 ceph-mon[35418]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:48.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:48 smithi086 ceph-mon[31733]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:48.458 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:48 smithi103 ceph-mon[35495]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:50.233 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:50 smithi103 ceph-mon[35495]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:50 smithi098 ceph-mon[35418]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:50 smithi086 ceph-mon[31733]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:52.468 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:52 smithi103 ceph-mon[35495]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:52 smithi098 ceph-mon[35418]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:52.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:52 smithi086 ceph-mon[31733]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:54.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:54 smithi103 ceph-mon[35495]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:54.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:54 smithi098 ceph-mon[35418]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:54.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:54 smithi086 ceph-mon[31733]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:08:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:55 smithi098 ceph-mon[35418]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:55.516 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:55.516 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:55.517 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:08:55.517 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:55 smithi103 ceph-mon[35495]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:08:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:55 smithi086 ceph-mon[31733]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:58.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:58 smithi086 ceph-mon[31733]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:58.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:58 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:58.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:58 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:58.350 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:58 smithi098 ceph-mon[35418]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:58.350 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:58 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:58.350 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:58 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:58.504 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:58 smithi103 ceph-mon[35495]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:08:58.504 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:58 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:58.504 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:58 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:08:59.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.173+0000 7f92e3400700 1 -- 172.21.15.103:0/1261368828 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f92ec062f50 con 0x7f92d00a3cc0 2024-04-17T07:08:59.174 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 8 on host 'smithi103' 2024-04-17T07:08:59.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.177+0000 7f92f1400700 1 -- 172.21.15.103:0/1261368828 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92d00a3cc0 msgr2=0x7f92d00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:08:59.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.177+0000 7f92f1400700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92d00a3cc0 0x7f92d00a6180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92dc005fd0 tx=0x7f92dc00a560 comp rx=0 tx=0).stop 2024-04-17T07:08:59.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.177+0000 7f92f1400700 1 -- 172.21.15.103:0/1261368828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92ec10ac90 msgr2=0x7f92ec1a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:08:59.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.177+0000 7f92f1400700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92ec10ac90 0x7f92ec1a6700 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f92e4007fa0 tx=0x7f92e400cf80 comp rx=0 tx=0).stop 2024-04-17T07:08:59.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.177+0000 7f92f1400700 1 -- 172.21.15.103:0/1261368828 shutdown_connections 2024-04-17T07:08:59.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.177+0000 7f92f1400700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92d00a3cc0 0x7f92d00a6180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:59.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.177+0000 7f92f1400700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92ec10ac90 0x7f92ec1a6700 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:59.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.177+0000 7f92f1400700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec1090f0 0x7f92ec19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:59.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.177+0000 7f92f1400700 1 --2- 172.21.15.103:0/1261368828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92ec10a2f0 0x7f92ec19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:08:59.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.177+0000 7f92f1400700 1 -- 172.21.15.103:0/1261368828 >> 172.21.15.103:0/1261368828 conn(0x7f92ec0780c0 msgr2=0x7f92ec10d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:08:59.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.178+0000 7f92f1400700 1 -- 172.21.15.103:0/1261368828 shutdown_connections 2024-04-17T07:08:59.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:08:59.178+0000 7f92f1400700 1 -- 172.21.15.103:0/1261368828 wait complete. 2024-04-17T07:08:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:59 smithi098 ceph-mon[35418]: from='osd.8 [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:08:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:08:59 smithi098 ceph-mon[35418]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:08:59.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:59 smithi086 ceph-mon[31733]: from='osd.8 [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:08:59.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:08:59 smithi086 ceph-mon[31733]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:08:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:59 smithi103 ceph-mon[35495]: from='osd.8 [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:08:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:08:59 smithi103 ceph-mon[35495]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-17T07:08:59.951 DEBUG:teuthology.orchestra.run.smithi103:osd.8> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.8.service 2024-04-17T07:08:59.956 INFO:tasks.cephadm:Deploying osd.9 on smithi086 with /dev/vg_nvme/lv_1... 2024-04-17T07:08:59.956 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-17T07:09:00.190 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:00 smithi103 ceph-mon[35495]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:09:00.190 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:00 smithi103 ceph-mon[35495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-17T07:09:00.190 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:00 smithi103 ceph-mon[35495]: osdmap e57: 9 total, 8 up, 9 in 2024-04-17T07:09:00.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:00 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:00.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:00 smithi103 ceph-mon[35495]: from='osd.8 [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:09:00.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:00 smithi103 ceph-mon[35495]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:09:00.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:00 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:00.191 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:00 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:00 smithi086 ceph-mon[31733]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:09:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:00 smithi086 ceph-mon[31733]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-17T07:09:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:00 smithi086 ceph-mon[31733]: osdmap e57: 9 total, 8 up, 9 in 2024-04-17T07:09:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:00 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:00 smithi086 ceph-mon[31733]: from='osd.8 [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:09:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:00 smithi086 ceph-mon[31733]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:09:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:00 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:00.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:00 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:09:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:00 smithi098 ceph-mon[35418]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:09:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:00 smithi098 ceph-mon[35418]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-17T07:09:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:00 smithi098 ceph-mon[35418]: osdmap e57: 9 total, 8 up, 9 in 2024-04-17T07:09:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:00 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:00 smithi098 ceph-mon[35418]: from='osd.8 [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:09:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:00 smithi098 ceph-mon[35418]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:09:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:00 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:00.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:00 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:00.778 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:09:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:09:00.668+0000 7f4967c00700 -1 osd.8 0 waiting for initial osdmap 2024-04-17T07:09:00.778 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:09:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:09:00.702+0000 7f495d000700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:09:01.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:01 smithi086 ceph-mon[31733]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:09:01.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:01 smithi086 ceph-mon[31733]: osdmap e58: 9 total, 8 up, 9 in 2024-04-17T07:09:01.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:01.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:01.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:01 smithi086 ceph-mon[31733]: from='osd.8 ' entity='osd.8' 2024-04-17T07:09:01.442 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:01 smithi103 ceph-mon[35495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:09:01.443 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:01 smithi103 ceph-mon[35495]: osdmap e58: 9 total, 8 up, 9 in 2024-04-17T07:09:01.443 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:01.443 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:01.443 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:01 smithi103 ceph-mon[35495]: from='osd.8 ' entity='osd.8' 2024-04-17T07:09:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:01 smithi098 ceph-mon[35418]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:09:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:01 smithi098 ceph-mon[35418]: osdmap e58: 9 total, 8 up, 9 in 2024-04-17T07:09:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:01 smithi098 ceph-mon[35418]: from='osd.8 ' entity='osd.8' 2024-04-17T07:09:02.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:02 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:09:02.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:02 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:09:02.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:02 smithi086 ceph-mon[31733]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:09:02.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:02 smithi086 ceph-mon[31733]: osd.8 [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] boot 2024-04-17T07:09:02.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:02 smithi086 ceph-mon[31733]: osdmap e59: 9 total, 9 up, 9 in 2024-04-17T07:09:02.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:02 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:02.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:02 smithi086 ceph-mon[31733]: osdmap e60: 9 total, 9 up, 9 in 2024-04-17T07:09:02.433 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:02 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:09:02.433 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:02 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:09:02.434 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:02 smithi103 ceph-mon[35495]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:09:02.434 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:02 smithi103 ceph-mon[35495]: osd.8 [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] boot 2024-04-17T07:09:02.434 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:02 smithi103 ceph-mon[35495]: osdmap e59: 9 total, 9 up, 9 in 2024-04-17T07:09:02.434 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:02 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:02.434 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:02 smithi103 ceph-mon[35495]: osdmap e60: 9 total, 9 up, 9 in 2024-04-17T07:09:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:02 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:09:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:02 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:09:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:02 smithi098 ceph-mon[35418]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-17T07:09:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:02 smithi098 ceph-mon[35418]: osd.8 [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] boot 2024-04-17T07:09:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:02 smithi098 ceph-mon[35418]: osdmap e59: 9 total, 9 up, 9 in 2024-04-17T07:09:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:02 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-17T07:09:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:02 smithi098 ceph-mon[35418]: osdmap e60: 9 total, 9 up, 9 in 2024-04-17T07:09:03.418 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-17T07:09:03.418 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-17T07:09:03.418 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-17T07:09:03.418 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-17T07:09:03.418 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0352622 s, 297 MB/s 2024-04-17T07:09:03.418 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-17T07:09:03.816 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:03 smithi086 ceph-mon[31733]: osdmap e61: 9 total, 9 up, 9 in 2024-04-17T07:09:03.816 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:03 smithi086 ceph-mon[31733]: pgmap v249: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:03 smithi098 ceph-mon[35418]: osdmap e61: 9 total, 9 up, 9 in 2024-04-17T07:09:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:03 smithi098 ceph-mon[35418]: pgmap v249: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:04.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:03 smithi103 ceph-mon[35495]: osdmap e61: 9 total, 9 up, 9 in 2024-04-17T07:09:04.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:03 smithi103 ceph-mon[35495]: pgmap v249: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:04.266 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-04-17T07:09:04.701 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:09:04.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:04 smithi103 ceph-mon[35495]: osdmap e62: 9 total, 9 up, 9 in 2024-04-17T07:09:05.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:04 smithi098 ceph-mon[35418]: osdmap e62: 9 total, 9 up, 9 in 2024-04-17T07:09:05.016 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:04 smithi086 ceph-mon[31733]: osdmap e62: 9 total, 9 up, 9 in 2024-04-17T07:09:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: osdmap e63: 9 total, 9 up, 9 in 2024-04-17T07:09:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi103 2024-04-17T07:09:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi103 to 5696M 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: pgmap v252: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:06.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:06.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: osdmap e63: 9 total, 9 up, 9 in 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi103 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi103 to 5696M 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: pgmap v252: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:06.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:09:06.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:06.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:06.028 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: osdmap e63: 9 total, 9 up, 9 in 2024-04-17T07:09:06.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi103 2024-04-17T07:09:06.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi103 to 5696M 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: pgmap v252: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.003+0000 7f39f6400700 1 -- 172.21.15.86:0/126358733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f010a2f0 msgr2=0x7f39f010a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:09:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.003+0000 7f39f6400700 1 --2- 172.21.15.86:0/126358733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f010a2f0 0x7f39f010a750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f39d80099c0 tx=0x7f39d8009cd0 comp rx=0 tx=0).stop 2024-04-17T07:09:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.004+0000 7f39f6400700 1 -- 172.21.15.86:0/126358733 shutdown_connections 2024-04-17T07:09:07.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.004+0000 7f39f6400700 1 --2- 172.21.15.86:0/126358733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f010ac90 0x7f39f0111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.004+0000 7f39f6400700 1 --2- 172.21.15.86:0/126358733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39f01090f0 0x7f39f0109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.004+0000 7f39f6400700 1 --2- 172.21.15.86:0/126358733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f010a2f0 0x7f39f010a750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.004+0000 7f39f6400700 1 -- 172.21.15.86:0/126358733 >> 172.21.15.86:0/126358733 conn(0x7f39f00780c0 msgr2=0x7f39f007a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:09:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.004+0000 7f39f6400700 1 -- 172.21.15.86:0/126358733 shutdown_connections 2024-04-17T07:09:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.004+0000 7f39f6400700 1 -- 172.21.15.86:0/126358733 wait complete. 2024-04-17T07:09:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.005+0000 7f39f6400700 1 Processor -- start 2024-04-17T07:09:07.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.006+0000 7f39f6400700 1 -- start start 2024-04-17T07:09:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.006+0000 7f39f6400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39f01090f0 0x7f39f019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:09:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.006+0000 7f39f6400700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f010a2f0 0x7f39f019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:09:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.006+0000 7f39f6400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f010ac90 0x7f39f01a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:09:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.006+0000 7f39f6400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0114c50 con 0x7f39f010ac90 2024-04-17T07:09:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.006+0000 7f39f6400700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0114ad0 con 0x7f39f010a2f0 2024-04-17T07:09:07.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.006+0000 7f39f6400700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0114dd0 con 0x7f39f01090f0 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39efe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f010ac90 0x7f39f01a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39efe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f010ac90 0x7f39f01a6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36182/0 (socket says 172.21.15.86:36182) 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39efe00700 1 -- 172.21.15.86:0/773466235 learned_addr learned my addr 172.21.15.86:0/773466235 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39e7e00700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f010a2f0 0x7f39f019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39ef400700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39f01090f0 0x7f39f019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39efe00700 1 -- 172.21.15.86:0/773466235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39f01090f0 msgr2=0x7f39f019f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39efe00700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39f01090f0 0x7f39f019f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39efe00700 1 -- 172.21.15.86:0/773466235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f010a2f0 msgr2=0x7f39f019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39efe00700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f010a2f0 0x7f39f019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39efe00700 1 -- 172.21.15.86:0/773466235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f01a6ef0 con 0x7f39f010ac90 2024-04-17T07:09:07.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.007+0000 7f39efe00700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f010ac90 0x7f39f01a6850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f39e800add0 tx=0x7f39e800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:09:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.008+0000 7f39ecc00700 1 -- 172.21.15.86:0/773466235 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8017020 con 0x7f39f010ac90 2024-04-17T07:09:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.008+0000 7f39ecc00700 1 -- 172.21.15.86:0/773466235 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39e800cdf0 con 0x7f39f010ac90 2024-04-17T07:09:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.008+0000 7f39f6400700 1 -- 172.21.15.86:0/773466235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f01a71e0 con 0x7f39f010ac90 2024-04-17T07:09:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.008+0000 7f39ecc00700 1 -- 172.21.15.86:0/773466235 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8005460 con 0x7f39f010ac90 2024-04-17T07:09:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.008+0000 7f39f6400700 1 -- 172.21.15.86:0/773466235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f01a7750 con 0x7f39f010ac90 2024-04-17T07:09:07.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.009+0000 7f39f6400700 1 -- 172.21.15.86:0/773466235 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f0069740 con 0x7f39f010ac90 2024-04-17T07:09:07.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.011+0000 7f39ecc00700 1 -- 172.21.15.86:0/773466235 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f39e8004ad0 con 0x7f39f010ac90 2024-04-17T07:09:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.013+0000 7f39ecc00700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39d00a3c40 0x7f39d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:09:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.014+0000 7f39ecc00700 1 -- 172.21.15.86:0/773466235 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f39e80cf8f0 con 0x7f39f010ac90 2024-04-17T07:09:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.014+0000 7f39ef400700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39d00a3c40 0x7f39d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:09:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.014+0000 7f39ef400700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39d00a3c40 0x7f39d00a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39e0006fd0 tx=0x7f39e0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:09:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.017+0000 7f39ecc00700 1 -- 172.21.15.86:0/773466235 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f39e8021020 con 0x7f39f010ac90 2024-04-17T07:09:07.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:07.261+0000 7f39f6400700 1 -- 172.21.15.86:0/773466235 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f39f0062e90 con 0x7f39d00a3c40 2024-04-17T07:09:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:08 smithi086 ceph-mon[31733]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:09:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:09:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:09:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:08 smithi098 ceph-mon[35418]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:09:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:09:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:09:08.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:08.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:08 smithi103 ceph-mon[35495]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:09:08.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:09:08.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:09:08.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:09.211 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:09 smithi086 ceph-mon[31733]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:09:09.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:09 smithi098 ceph-mon[35418]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:09:09.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:09 smithi103 ceph-mon[35495]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:09:10.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:10 smithi086 ceph-mon[31733]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:09:10.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:10 smithi103 ceph-mon[35495]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:09:10.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:10 smithi098 ceph-mon[35418]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:09:12.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:12 smithi086 ceph-mon[31733]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:09:12.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/601938304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4f5184c-d482-4569-9b76-78377f60226d"}]: dispatch 2024-04-17T07:09:12.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/601938304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4f5184c-d482-4569-9b76-78377f60226d"}]': finished 2024-04-17T07:09:12.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:12 smithi086 ceph-mon[31733]: osdmap e64: 10 total, 9 up, 10 in 2024-04-17T07:09:12.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:12 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:12.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1131672723' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:09:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:12 smithi098 ceph-mon[35418]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:09:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/601938304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4f5184c-d482-4569-9b76-78377f60226d"}]: dispatch 2024-04-17T07:09:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/601938304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4f5184c-d482-4569-9b76-78377f60226d"}]': finished 2024-04-17T07:09:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:12 smithi098 ceph-mon[35418]: osdmap e64: 10 total, 9 up, 10 in 2024-04-17T07:09:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:12 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1131672723' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:09:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:12 smithi103 ceph-mon[35495]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:09:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/601938304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c4f5184c-d482-4569-9b76-78377f60226d"}]: dispatch 2024-04-17T07:09:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/601938304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c4f5184c-d482-4569-9b76-78377f60226d"}]': finished 2024-04-17T07:09:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:12 smithi103 ceph-mon[35495]: osdmap e64: 10 total, 9 up, 10 in 2024-04-17T07:09:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:12 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1131672723' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:09:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:14 smithi086 ceph-mon[31733]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-17T07:09:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:14 smithi098 ceph-mon[35418]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-17T07:09:14.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:14 smithi103 ceph-mon[35495]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-17T07:09:16.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:16 smithi086 ceph-mon[31733]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:09:16.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:16 smithi098 ceph-mon[35418]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:09:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:16 smithi103 ceph-mon[35495]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:09:18.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:18 smithi086 ceph-mon[31733]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:18.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:18 smithi098 ceph-mon[35418]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:18.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:18 smithi103 ceph-mon[35495]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:20.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:20 smithi098 ceph-mon[35418]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:20.504 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:20 smithi103 ceph-mon[35495]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:20.505 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:20 smithi086 ceph-mon[31733]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-17T07:09:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:21.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-17T07:09:21.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:21.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-17T07:09:21.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:22 smithi086 ceph-mon[31733]: Deploying daemon osd.9 on smithi086 2024-04-17T07:09:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:22 smithi086 ceph-mon[31733]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:22.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:22 smithi098 ceph-mon[35418]: Deploying daemon osd.9 on smithi086 2024-04-17T07:09:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:22 smithi098 ceph-mon[35418]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:22.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:22 smithi103 ceph-mon[35495]: Deploying daemon osd.9 on smithi086 2024-04-17T07:09:22.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:22 smithi103 ceph-mon[35495]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:24.267 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:24 smithi086 ceph-mon[31733]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:24 smithi098 ceph-mon[35418]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:24 smithi103 ceph-mon[35495]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:26.403 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:26 smithi086 ceph-mon[31733]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:26.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:26 smithi098 ceph-mon[35418]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:26.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:26 smithi103 ceph-mon[35495]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:28.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:28 smithi098 ceph-mon[35418]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:28 smithi103 ceph-mon[35495]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:28.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:28 smithi086 ceph-mon[31733]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:30.437 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:30 smithi086 ceph-mon[31733]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:30.438 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:30.438 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:30.438 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:30 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:09:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:30 smithi098 ceph-mon[35418]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:30 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:09:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:30 smithi103 ceph-mon[35495]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:30 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:09:31.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:31 smithi086 ceph-mon[31733]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:31.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:31 smithi098 ceph-mon[35418]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:31 smithi103 ceph-mon[35495]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.258+0000 7f39ecc00700 1 -- 172.21.15.86:0/773466235 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f39f0062e90 con 0x7f39d00a3c40 2024-04-17T07:09:34.259 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 9 on host 'smithi086' 2024-04-17T07:09:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.262+0000 7f39f6400700 1 -- 172.21.15.86:0/773466235 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39d00a3c40 msgr2=0x7f39d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:09:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39d00a3c40 0x7f39d00a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39e0006fd0 tx=0x7f39e0008040 comp rx=0 tx=0).stop 2024-04-17T07:09:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 -- 172.21.15.86:0/773466235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f010ac90 msgr2=0x7f39f01a6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:09:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f010ac90 0x7f39f01a6850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f39e800add0 tx=0x7f39e800c330 comp rx=0 tx=0).stop 2024-04-17T07:09:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 -- 172.21.15.86:0/773466235 shutdown_connections 2024-04-17T07:09:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39d00a3c40 0x7f39d00a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f010ac90 0x7f39f01a6850 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39f01090f0 0x7f39f019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 --2- 172.21.15.86:0/773466235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f010a2f0 0x7f39f019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 -- 172.21.15.86:0/773466235 >> 172.21.15.86:0/773466235 conn(0x7f39f00780c0 msgr2=0x7f39f010d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:09:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 -- 172.21.15.86:0/773466235 shutdown_connections 2024-04-17T07:09:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:09:34.263+0000 7f39f6400700 1 -- 172.21.15.86:0/773466235 wait complete. 2024-04-17T07:09:34.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:34 smithi086 ceph-mon[31733]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:34.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:34 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:34.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:34 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:34.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:34 smithi086 ceph-mon[31733]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T07:09:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:34 smithi098 ceph-mon[35418]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:34 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:34 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:34 smithi098 ceph-mon[35418]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T07:09:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:34 smithi103 ceph-mon[35495]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:34 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:34 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:34 smithi103 ceph-mon[35495]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-17T07:09:35.057 DEBUG:teuthology.orchestra.run.smithi086:osd.9> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.9.service 2024-04-17T07:09:35.062 INFO:tasks.cephadm:Deploying osd.10 on smithi098 with /dev/vg_nvme/lv_1... 2024-04-17T07:09:35.062 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-17T07:09:35.259 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:35 smithi086 ceph-mon[31733]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-17T07:09:35.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:35 smithi086 ceph-mon[31733]: osdmap e65: 10 total, 9 up, 10 in 2024-04-17T07:09:35.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:35 smithi086 ceph-mon[31733]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:09:35.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:35.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:35.260 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:35 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:35 smithi098 ceph-mon[35418]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-17T07:09:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:35 smithi098 ceph-mon[35418]: osdmap e65: 10 total, 9 up, 10 in 2024-04-17T07:09:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:35 smithi098 ceph-mon[35418]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:09:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:35 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:35.504 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:09:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:35 smithi103 ceph-mon[35495]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-17T07:09:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:35 smithi103 ceph-mon[35495]: osdmap e65: 10 total, 9 up, 10 in 2024-04-17T07:09:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:35 smithi103 ceph-mon[35495]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-17T07:09:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:35 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:36.009 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:09:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:09:35.661+0000 7fbb8ce00700 -1 osd.9 0 waiting for initial osdmap 2024-04-17T07:09:36.009 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:09:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:09:35.691+0000 7fbb82c00700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:09:36.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:36 smithi086 ceph-mon[31733]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:36.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:36 smithi086 ceph-mon[31733]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:09:36.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:36 smithi086 ceph-mon[31733]: osdmap e66: 10 total, 9 up, 10 in 2024-04-17T07:09:36.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:36.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:36.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:36 smithi086 ceph-mon[31733]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' 2024-04-17T07:09:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:36 smithi098 ceph-mon[35418]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:36 smithi098 ceph-mon[35418]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:09:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:36 smithi098 ceph-mon[35418]: osdmap e66: 10 total, 9 up, 10 in 2024-04-17T07:09:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:36.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:36 smithi098 ceph-mon[35418]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' 2024-04-17T07:09:36.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:36 smithi103 ceph-mon[35495]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:36.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:36 smithi103 ceph-mon[35495]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-17T07:09:36.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:36 smithi103 ceph-mon[35495]: osdmap e66: 10 total, 9 up, 10 in 2024-04-17T07:09:36.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:36.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:36.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:36 smithi103 ceph-mon[35495]: from='osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207]' entity='osd.9' 2024-04-17T07:09:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:37 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:09:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:37 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:09:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:37 smithi086 ceph-mon[31733]: osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] boot 2024-04-17T07:09:37.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:37 smithi086 ceph-mon[31733]: osdmap e67: 10 total, 10 up, 10 in 2024-04-17T07:09:37.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:37.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:37 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:09:37.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:37 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:09:37.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:37.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:37 smithi098 ceph-mon[35418]: osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] boot 2024-04-17T07:09:37.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:37 smithi098 ceph-mon[35418]: osdmap e67: 10 total, 10 up, 10 in 2024-04-17T07:09:37.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:37 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:09:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:37 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:09:37.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:37.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:37 smithi103 ceph-mon[35495]: osd.9 [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] boot 2024-04-17T07:09:37.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:37 smithi103 ceph-mon[35495]: osdmap e67: 10 total, 10 up, 10 in 2024-04-17T07:09:37.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-17T07:09:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:38 smithi098 ceph-mon[35418]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:38 smithi103 ceph-mon[35495]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:38.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:38 smithi086 ceph-mon[31733]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-17T07:09:38.625 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-17T07:09:38.625 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-17T07:09:38.625 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-17T07:09:38.625 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-17T07:09:38.625 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0429676 s, 244 MB/s 2024-04-17T07:09:38.626 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-17T07:09:39.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:39 smithi098 ceph-mon[35418]: osdmap e68: 10 total, 10 up, 10 in 2024-04-17T07:09:39.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:39 smithi103 ceph-mon[35495]: osdmap e68: 10 total, 10 up, 10 in 2024-04-17T07:09:39.539 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_1 2024-04-17T07:09:39.564 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:39 smithi086 ceph-mon[31733]: osdmap e68: 10 total, 10 up, 10 in 2024-04-17T07:09:39.945 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:40.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:40 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:40.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:40.429 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:40.429 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:40 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:40.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:09:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:40 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:41.214 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:41 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi086 2024-04-17T07:09:41.214 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:41 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-17T07:09:41.214 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:09:41.214 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:41.214 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:41.214 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:41 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi086 2024-04-17T07:09:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:41 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-17T07:09:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:09:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:41 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi086 2024-04-17T07:09:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:41 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-17T07:09:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:09:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:41.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:09:41.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:09:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.016+0000 7f0f1e200700 1 -- 172.21.15.98:0/3290300764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18101c40 msgr2=0x7f0f181020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:09:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.016+0000 7f0f1e200700 1 --2- 172.21.15.98:0/3290300764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18101c40 0x7f0f181020c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009990 tx=0x7f0f04009ca0 comp rx=0 tx=0).stop 2024-04-17T07:09:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.017+0000 7f0f1e200700 1 -- 172.21.15.98:0/3290300764 shutdown_connections 2024-04-17T07:09:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.017+0000 7f0f1e200700 1 --2- 172.21.15.98:0/3290300764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18102600 0x7f0f18102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.017+0000 7f0f1e200700 1 --2- 172.21.15.98:0/3290300764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f181012e0 0x7f0f18101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.017+0000 7f0f1e200700 1 --2- 172.21.15.98:0/3290300764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18101c40 0x7f0f181020c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.017+0000 7f0f1e200700 1 -- 172.21.15.98:0/3290300764 >> 172.21.15.98:0/3290300764 conn(0x7f0f180fce60 msgr2=0x7f0f180ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:09:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.017+0000 7f0f1e200700 1 -- 172.21.15.98:0/3290300764 shutdown_connections 2024-04-17T07:09:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.017+0000 7f0f1e200700 1 -- 172.21.15.98:0/3290300764 wait complete. 2024-04-17T07:09:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.018+0000 7f0f1e200700 1 Processor -- start 2024-04-17T07:09:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.018+0000 7f0f1e200700 1 -- start start 2024-04-17T07:09:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f1e200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f181012e0 0x7f0f1819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f1e200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18101c40 0x7f0f1819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f1e200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18102600 0x7f0f181a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f1e200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18110660 con 0x7f0f18102600 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f1e200700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181104e0 con 0x7f0f18101c40 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f1e200700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0f181107e0 con 0x7f0f181012e0 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f17e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18101c40 0x7f0f1819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f17e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18101c40 0x7f0f1819b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53910/0 (socket says 172.21.15.98:53910) 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f17e00700 1 -- 172.21.15.98:0/1741821486 learned_addr learned my addr 172.21.15.98:0/1741821486 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f1ce00700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f181012e0 0x7f0f1819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.019+0000 7f0f1d800700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18102600 0x7f0f181a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f17e00700 1 -- 172.21.15.98:0/1741821486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f181012e0 msgr2=0x7f0f1819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f17e00700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f181012e0 0x7f0f1819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:42.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f17e00700 1 -- 172.21.15.98:0/1741821486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18102600 msgr2=0x7f0f181a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:09:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f17e00700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18102600 0x7f0f181a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:09:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f17e00700 1 -- 172.21.15.98:0/1741821486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f181a29a0 con 0x7f0f18101c40 2024-04-17T07:09:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f1ce00700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f181012e0 0x7f0f1819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:09:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f17e00700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18101c40 0x7f0f1819b280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f040036e0 tx=0x7f0f04005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:09:42.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f15600700 1 -- 172.21.15.98:0/1741821486 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04027070 con 0x7f0f18101c40 2024-04-17T07:09:42.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f15600700 1 -- 172.21.15.98:0/1741821486 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f04022470 con 0x7f0f18101c40 2024-04-17T07:09:42.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f15600700 1 -- 172.21.15.98:0/1741821486 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0400f6c0 con 0x7f0f18101c40 2024-04-17T07:09:42.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.020+0000 7f0f1e200700 1 -- 172.21.15.98:0/1741821486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f181a2c30 con 0x7f0f18101c40 2024-04-17T07:09:42.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.021+0000 7f0f1e200700 1 -- 172.21.15.98:0/1741821486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a30e0 con 0x7f0f18101c40 2024-04-17T07:09:42.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.022+0000 7f0f1e200700 1 -- 172.21.15.98:0/1741821486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f18069740 con 0x7f0f18101c40 2024-04-17T07:09:42.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.024+0000 7f0f15600700 1 -- 172.21.15.98:0/1741821486 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0f0402a070 con 0x7f0f18101c40 2024-04-17T07:09:42.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.025+0000 7f0f15600700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0f000a3cc0 0x7f0f000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:09:42.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.025+0000 7f0f15600700 1 -- 172.21.15.98:0/1741821486 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f0f040d0a80 con 0x7f0f18101c40 2024-04-17T07:09:42.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.026+0000 7f0f1ce00700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0f000a3cc0 0x7f0f000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:09:42.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.027+0000 7f0f1ce00700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0f000a3cc0 0x7f0f000a6180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00bcf0 tx=0x7f0f0c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:09:42.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.030+0000 7f0f15600700 1 -- 172.21.15.98:0/1741821486 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0f0405eff0 con 0x7f0f18101c40 2024-04-17T07:09:42.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:09:42.271+0000 7f0f1e200700 1 -- 172.21.15.98:0/1741821486 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0f18062e90 con 0x7f0f000a3cc0 2024-04-17T07:09:42.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:42 smithi086 ceph-mon[31733]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:42.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:42 smithi098 ceph-mon[35418]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:42.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:42 smithi103 ceph-mon[35495]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:43.407 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:09:43.407 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:09:43.407 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:09:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:09:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:09:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:09:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:44.469 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:44 smithi098 ceph-mon[35418]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:09:44.469 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:44 smithi098 ceph-mon[35418]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:44.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:44 smithi103 ceph-mon[35495]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:09:44.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:44 smithi103 ceph-mon[35495]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:44.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:44 smithi086 ceph-mon[31733]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:09:44.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:44 smithi086 ceph-mon[31733]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:45 smithi098 ceph-mon[35418]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:45.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:45 smithi103 ceph-mon[35495]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:45 smithi086 ceph-mon[31733]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:46 smithi098 ceph-mon[35418]: from='client.? 172.21.15.98:0/576303746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f42db156-1323-4e33-9d20-d3401ef400fd"}]: dispatch 2024-04-17T07:09:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:46 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f42db156-1323-4e33-9d20-d3401ef400fd"}]: dispatch 2024-04-17T07:09:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:46 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f42db156-1323-4e33-9d20-d3401ef400fd"}]': finished 2024-04-17T07:09:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:46 smithi098 ceph-mon[35418]: osdmap e69: 11 total, 10 up, 11 in 2024-04-17T07:09:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:09:46.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:46 smithi103 ceph-mon[35495]: from='client.? 172.21.15.98:0/576303746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f42db156-1323-4e33-9d20-d3401ef400fd"}]: dispatch 2024-04-17T07:09:46.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:46 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f42db156-1323-4e33-9d20-d3401ef400fd"}]: dispatch 2024-04-17T07:09:46.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:46 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f42db156-1323-4e33-9d20-d3401ef400fd"}]': finished 2024-04-17T07:09:46.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:46 smithi103 ceph-mon[35495]: osdmap e69: 11 total, 10 up, 11 in 2024-04-17T07:09:46.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:09:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:46 smithi086 ceph-mon[31733]: from='client.? 172.21.15.98:0/576303746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f42db156-1323-4e33-9d20-d3401ef400fd"}]: dispatch 2024-04-17T07:09:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:46 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f42db156-1323-4e33-9d20-d3401ef400fd"}]: dispatch 2024-04-17T07:09:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:46 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f42db156-1323-4e33-9d20-d3401ef400fd"}]': finished 2024-04-17T07:09:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:46 smithi086 ceph-mon[31733]: osdmap e69: 11 total, 10 up, 11 in 2024-04-17T07:09:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:09:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:47 smithi098 ceph-mon[35418]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:47 smithi098 ceph-mon[35418]: from='client.? 172.21.15.98:0/2327989871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:09:47.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:47 smithi103 ceph-mon[35495]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:47.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:47 smithi103 ceph-mon[35495]: from='client.? 172.21.15.98:0/2327989871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:09:47.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:47 smithi086 ceph-mon[31733]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:47.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:47 smithi086 ceph-mon[31733]: from='client.? 172.21.15.98:0/2327989871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:09:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:50 smithi086 ceph-mon[31733]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:50 smithi098 ceph-mon[35418]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:50.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:50 smithi103 ceph-mon[35495]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:52.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:52 smithi086 ceph-mon[31733]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:52 smithi098 ceph-mon[35418]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:52 smithi103 ceph-mon[35495]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:54.177 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:54 smithi098 ceph-mon[35418]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:54.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:54 smithi086 ceph-mon[31733]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:54 smithi103 ceph-mon[35495]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:56.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:56 smithi098 ceph-mon[35418]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-17T07:09:56.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:56 smithi103 ceph-mon[35495]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-17T07:09:56.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:56.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:56 smithi086 ceph-mon[31733]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:56.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-17T07:09:56.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:09:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:57 smithi098 ceph-mon[35418]: Deploying daemon osd.10 on smithi098 2024-04-17T07:09:57.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:57 smithi103 ceph-mon[35495]: Deploying daemon osd.10 on smithi098 2024-04-17T07:09:57.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:57 smithi086 ceph-mon[31733]: Deploying daemon osd.10 on smithi098 2024-04-17T07:09:58.482 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:09:58 smithi086 ceph-mon[31733]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:09:58 smithi098 ceph-mon[35418]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:09:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:09:58 smithi103 ceph-mon[35495]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:00.181 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:00 smithi086 ceph-mon[31733]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:00.182 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:00 smithi086 ceph-mon[31733]: overall HEALTH_OK 2024-04-17T07:10:00.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:00 smithi098 ceph-mon[35418]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:00.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:00 smithi098 ceph-mon[35418]: overall HEALTH_OK 2024-04-17T07:10:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:00 smithi103 ceph-mon[35495]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:00 smithi103 ceph-mon[35495]: overall HEALTH_OK 2024-04-17T07:10:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:02 smithi098 ceph-mon[35418]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:02 smithi103 ceph-mon[35495]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:02 smithi086 ceph-mon[31733]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:04.231 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:04 smithi098 ceph-mon[35418]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:04.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:04 smithi103 ceph-mon[35495]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:04.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:04 smithi086 ceph-mon[31733]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:05.750 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:05.750 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:05.750 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:05.750 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:05 smithi098 ceph-mon[35418]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:05 smithi103 ceph-mon[35495]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:05 smithi086 ceph-mon[31733]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:08.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:08 smithi098 ceph-mon[35418]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:08 smithi086 ceph-mon[31733]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:08.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:08 smithi103 ceph-mon[35495]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:09.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:09 smithi098 ceph-mon[35418]: from='osd.10 [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:10:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:09 smithi098 ceph-mon[35418]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:10:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:09 smithi098 ceph-mon[35418]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:09 smithi086 ceph-mon[31733]: from='osd.10 [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:10:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:09 smithi086 ceph-mon[31733]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:10:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:09 smithi086 ceph-mon[31733]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:09.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.657+0000 7f0f15600700 1 -- 172.21.15.98:0/1741821486 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0f18062e90 con 0x7f0f000a3cc0 2024-04-17T07:10:09.657 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 10 on host 'smithi098' 2024-04-17T07:10:09.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 -- 172.21.15.98:0/1741821486 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0f000a3cc0 msgr2=0x7f0f000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:09.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0f000a3cc0 0x7f0f000a6180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00bcf0 tx=0x7f0f0c00b2c0 comp rx=0 tx=0).stop 2024-04-17T07:10:09.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 -- 172.21.15.98:0/1741821486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18101c40 msgr2=0x7f0f1819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:09.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18101c40 0x7f0f1819b280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f040036e0 tx=0x7f0f04005ef0 comp rx=0 tx=0).stop 2024-04-17T07:10:09.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 -- 172.21.15.98:0/1741821486 shutdown_connections 2024-04-17T07:10:09.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0f000a3cc0 0x7f0f000a6180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:09.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f18102600 0x7f0f181a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:09.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f181012e0 0x7f0f1819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:09.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 --2- 172.21.15.98:0/1741821486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f18101c40 0x7f0f1819b280 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:09.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 -- 172.21.15.98:0/1741821486 >> 172.21.15.98:0/1741821486 conn(0x7f0f180fce60 msgr2=0x7f0f180fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:10:09.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 -- 172.21.15.98:0/1741821486 shutdown_connections 2024-04-17T07:10:09.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-17T07:10:09.661+0000 7f0f1e200700 1 -- 172.21.15.98:0/1741821486 wait complete. 2024-04-17T07:10:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:09 smithi103 ceph-mon[35495]: from='osd.10 [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:10:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:09 smithi103 ceph-mon[35495]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-17T07:10:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:09 smithi103 ceph-mon[35495]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:10.512 DEBUG:teuthology.orchestra.run.smithi098:osd.10> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.10.service 2024-04-17T07:10:10.518 INFO:tasks.cephadm:Deploying osd.11 on smithi103 with /dev/vg_nvme/lv_1... 2024-04-17T07:10:10.518 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-17T07:10:10.613 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-17T07:10:10.613 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: osdmap e70: 11 total, 10 up, 11 in 2024-04-17T07:10:10.613 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:10.613 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: from='osd.10 [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:10:10.614 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:10:10.614 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:10.614 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:10.614 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:10.614 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:10.614 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:10.614 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:10.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-17T07:10:10.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: osdmap e70: 11 total, 10 up, 11 in 2024-04-17T07:10:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: from='osd.10 [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:10:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:10:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:10.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-17T07:10:10.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: osdmap e70: 11 total, 10 up, 11 in 2024-04-17T07:10:10.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:10.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: from='osd.10 [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:10:10.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-17T07:10:10.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:10.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:10.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:10.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:10.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:10.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:10.974 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:10:11.253 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:10:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:10:10.999+0000 7fab7fa00700 -1 osd.10 0 waiting for initial osdmap 2024-04-17T07:10:11.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:10:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:10:11.030+0000 7fab75800700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:10:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:11 smithi086 ceph-mon[31733]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:10:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:11 smithi086 ceph-mon[31733]: osdmap e71: 11 total, 10 up, 11 in 2024-04-17T07:10:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:11 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:11 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:11 smithi086 ceph-mon[31733]: from='osd.10 ' entity='osd.10' 2024-04-17T07:10:11.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:11 smithi086 ceph-mon[31733]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:11.615 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:11 smithi103 ceph-mon[35495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:10:11.615 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:11 smithi103 ceph-mon[35495]: osdmap e71: 11 total, 10 up, 11 in 2024-04-17T07:10:11.615 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:11 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:11.615 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:11 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:11.615 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:11 smithi103 ceph-mon[35495]: from='osd.10 ' entity='osd.10' 2024-04-17T07:10:11.615 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:11 smithi103 ceph-mon[35495]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:11 smithi098 ceph-mon[35418]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-17T07:10:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:11 smithi098 ceph-mon[35418]: osdmap e71: 11 total, 10 up, 11 in 2024-04-17T07:10:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:11 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:11 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:11 smithi098 ceph-mon[35418]: from='osd.10 ' entity='osd.10' 2024-04-17T07:10:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:11 smithi098 ceph-mon[35418]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-17T07:10:12.688 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:12 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:10:12.689 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:12 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:10:12.689 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:12 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:12.689 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:12 smithi103 ceph-mon[35495]: osd.10 [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] boot 2024-04-17T07:10:12.689 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:12 smithi103 ceph-mon[35495]: osdmap e72: 11 total, 11 up, 11 in 2024-04-17T07:10:12.689 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:12 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:12.689 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:12 smithi103 ceph-mon[35495]: osdmap e73: 11 total, 11 up, 11 in 2024-04-17T07:10:12.711 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:12 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:10:12.712 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:12 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:10:12.712 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:12 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:12.712 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:12 smithi098 ceph-mon[35418]: osd.10 [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] boot 2024-04-17T07:10:12.712 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:12 smithi098 ceph-mon[35418]: osdmap e72: 11 total, 11 up, 11 in 2024-04-17T07:10:12.712 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:12 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:12.712 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:12 smithi098 ceph-mon[35418]: osdmap e73: 11 total, 11 up, 11 in 2024-04-17T07:10:12.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:12 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:10:12.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:12 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:10:12.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:12 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:12.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:12 smithi086 ceph-mon[31733]: osd.10 [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] boot 2024-04-17T07:10:12.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:12 smithi086 ceph-mon[31733]: osdmap e72: 11 total, 11 up, 11 in 2024-04-17T07:10:12.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:12 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-17T07:10:12.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:12 smithi086 ceph-mon[31733]: osdmap e73: 11 total, 11 up, 11 in 2024-04-17T07:10:13.909 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:13 smithi103 ceph-mon[35495]: osdmap e74: 11 total, 11 up, 11 in 2024-04-17T07:10:13.909 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:13 smithi103 ceph-mon[35495]: pgmap v297: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:13.909 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-17T07:10:13.909 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-17T07:10:13.909 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-17T07:10:13.910 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-17T07:10:13.910 INFO:teuthology.orchestra.run.smithi103.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.034474 s, 304 MB/s 2024-04-17T07:10:13.910 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-17T07:10:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:13 smithi098 ceph-mon[35418]: osdmap e74: 11 total, 11 up, 11 in 2024-04-17T07:10:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:13 smithi098 ceph-mon[35418]: pgmap v297: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:14.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:13 smithi086 ceph-mon[31733]: osdmap e74: 11 total, 11 up, 11 in 2024-04-17T07:10:14.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:13 smithi086 ceph-mon[31733]: pgmap v297: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:14.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-04-17T07:10:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:14 smithi098 ceph-mon[35418]: osdmap e75: 11 total, 11 up, 11 in 2024-04-17T07:10:15.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:14 smithi103 ceph-mon[35495]: osdmap e75: 11 total, 11 up, 11 in 2024-04-17T07:10:15.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:14 smithi086 ceph-mon[31733]: osdmap e75: 11 total, 11 up, 11 in 2024-04-17T07:10:15.213 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:10:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: osdmap e76: 11 total, 11 up, 11 in 2024-04-17T07:10:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:16.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: osdmap e76: 11 total, 11 up, 11 in 2024-04-17T07:10:16.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:16.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:16.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:16.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: osdmap e76: 11 total, 11 up, 11 in 2024-04-17T07:10:16.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:16.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:16.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:16.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:16.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:16.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:17.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:16 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi098 2024-04-17T07:10:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:16 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi098 to 4272M 2024-04-17T07:10:17.018 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:16 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi098 2024-04-17T07:10:17.018 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:16 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi098 to 4272M 2024-04-17T07:10:17.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:16 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi098 2024-04-17T07:10:17.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:16 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi098 to 4272M 2024-04-17T07:10:17.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.475+0000 7f9ee9200700 1 -- 172.21.15.103:0/3969073321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee4105ef0 msgr2=0x7f9ee4106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:17.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.475+0000 7f9ee9200700 1 --2- 172.21.15.103:0/3969073321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee4105ef0 0x7f9ee4106350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009990 tx=0x7f9ed0009ca0 comp rx=0 tx=0).stop 2024-04-17T07:10:17.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.475+0000 7f9ee9200700 1 -- 172.21.15.103:0/3969073321 shutdown_connections 2024-04-17T07:10:17.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.475+0000 7f9ee9200700 1 --2- 172.21.15.103:0/3969073321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4106890 0x7f9ee410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:17.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.475+0000 7f9ee9200700 1 --2- 172.21.15.103:0/3969073321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee4104cf0 0x7f9ee4105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:17.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.475+0000 7f9ee9200700 1 --2- 172.21.15.103:0/3969073321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee4105ef0 0x7f9ee4106350 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:17.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.475+0000 7f9ee9200700 1 -- 172.21.15.103:0/3969073321 >> 172.21.15.103:0/3969073321 conn(0x7f9ee41002b0 msgr2=0x7f9ee41026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:10:17.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.475+0000 7f9ee9200700 1 -- 172.21.15.103:0/3969073321 shutdown_connections 2024-04-17T07:10:17.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.475+0000 7f9ee9200700 1 -- 172.21.15.103:0/3969073321 wait complete. 2024-04-17T07:10:17.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.476+0000 7f9ee9200700 1 Processor -- start 2024-04-17T07:10:17.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.477+0000 7f9ee9200700 1 -- start start 2024-04-17T07:10:17.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.477+0000 7f9ee9200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4104cf0 0x7f9ee419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:17.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.477+0000 7f9ee9200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee4105ef0 0x7f9ee419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:17.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.477+0000 7f9ee9200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee4106890 0x7f9ee41a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:17.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.477+0000 7f9ee9200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4110760 con 0x7f9ee4106890 2024-04-17T07:10:17.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.477+0000 7f9ee9200700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41105e0 con 0x7f9ee4104cf0 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.477+0000 7f9ee9200700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41108e0 con 0x7f9ee4105ef0 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.477+0000 7f9ee1400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee4105ef0 0x7f9ee419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.477+0000 7f9ee1400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee4105ef0 0x7f9ee419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45916/0 (socket says 172.21.15.103:45916) 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.477+0000 7f9ee1400700 1 -- 172.21.15.103:0/2568037633 learned_addr learned my addr 172.21.15.103:0/2568037633 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.478+0000 7f9ee1e00700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4104cf0 0x7f9ee419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.478+0000 7f9ee2800700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee4106890 0x7f9ee41a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.478+0000 7f9ee1400700 1 -- 172.21.15.103:0/2568037633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4104cf0 msgr2=0x7f9ee419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.478+0000 7f9ee1400700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4104cf0 0x7f9ee419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.478+0000 7f9ee1400700 1 -- 172.21.15.103:0/2568037633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee4106890 msgr2=0x7f9ee41a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.478+0000 7f9ee1400700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee4106890 0x7f9ee41a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:17.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.478+0000 7f9ee1400700 1 -- 172.21.15.103:0/2568037633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee41a2a30 con 0x7f9ee4105ef0 2024-04-17T07:10:17.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.478+0000 7f9ee1400700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee4105ef0 0x7f9ee419b310 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0004790 tx=0x7f9ed0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:10:17.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.479+0000 7f9edaa00700 1 -- 172.21.15.103:0/2568037633 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0027070 con 0x7f9ee4105ef0 2024-04-17T07:10:17.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.479+0000 7f9edaa00700 1 -- 172.21.15.103:0/2568037633 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ed0004080 con 0x7f9ee4105ef0 2024-04-17T07:10:17.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.479+0000 7f9ee9200700 1 -- 172.21.15.103:0/2568037633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee41a2cc0 con 0x7f9ee4105ef0 2024-04-17T07:10:17.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.479+0000 7f9edaa00700 1 -- 172.21.15.103:0/2568037633 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0022d70 con 0x7f9ee4105ef0 2024-04-17T07:10:17.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.479+0000 7f9ee9200700 1 -- 172.21.15.103:0/2568037633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a3110 con 0x7f9ee4105ef0 2024-04-17T07:10:17.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.480+0000 7f9ee9200700 1 -- 172.21.15.103:0/2568037633 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee402df90 con 0x7f9ee4105ef0 2024-04-17T07:10:17.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.483+0000 7f9edaa00700 1 -- 172.21.15.103:0/2568037633 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9ed002a070 con 0x7f9ee4105ef0 2024-04-17T07:10:17.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.484+0000 7f9edaa00700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec80a3c40 0x7f9ec80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:17.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.484+0000 7f9edaa00700 1 -- 172.21.15.103:0/2568037633 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f9ed00d11a0 con 0x7f9ee4105ef0 2024-04-17T07:10:17.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.485+0000 7f9ee1e00700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec80a3c40 0x7f9ec80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:17.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.485+0000 7f9ee1e00700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec80a3c40 0x7f9ec80a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4105d50 tx=0x7f9ed400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:10:17.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.487+0000 7f9edaa00700 1 -- 172.21.15.103:0/2568037633 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9ed0031020 con 0x7f9ee4105ef0 2024-04-17T07:10:17.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:17.725+0000 7f9ee9200700 1 -- 172.21.15.103:0/2568037633 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9ee4062e90 con 0x7f9ec80a3c40 2024-04-17T07:10:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:17 smithi098 ceph-mon[35418]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:10:18.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:17 smithi103 ceph-mon[35495]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:10:18.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:17 smithi086 ceph-mon[31733]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-04-17T07:10:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:18 smithi098 ceph-mon[35418]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:10:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:18 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:10:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:18 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:10:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:18 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:19.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:18 smithi103 ceph-mon[35495]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:10:19.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:18 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:10:19.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:18 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:10:19.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:18 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:19.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:18 smithi086 ceph-mon[31733]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:10:19.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:18 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-17T07:10:19.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:18 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-17T07:10:19.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:18 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:20.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:19 smithi098 ceph-mon[35418]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:10:20.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:19 smithi103 ceph-mon[35495]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:10:20.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:19 smithi086 ceph-mon[31733]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-17T07:10:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:22 smithi086 ceph-mon[31733]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:10:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:22 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/3660799791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c2d8c2-4547-437f-a8f4-d12f4a8047b3"}]: dispatch 2024-04-17T07:10:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:22 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c2d8c2-4547-437f-a8f4-d12f4a8047b3"}]: dispatch 2024-04-17T07:10:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:22 smithi086 ceph-mon[31733]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86c2d8c2-4547-437f-a8f4-d12f4a8047b3"}]': finished 2024-04-17T07:10:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:22 smithi086 ceph-mon[31733]: osdmap e77: 12 total, 11 up, 12 in 2024-04-17T07:10:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:22.419 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:22 smithi103 ceph-mon[35495]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:10:22.419 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:22 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/3660799791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c2d8c2-4547-437f-a8f4-d12f4a8047b3"}]: dispatch 2024-04-17T07:10:22.419 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:22 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c2d8c2-4547-437f-a8f4-d12f4a8047b3"}]: dispatch 2024-04-17T07:10:22.419 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:22 smithi103 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86c2d8c2-4547-437f-a8f4-d12f4a8047b3"}]': finished 2024-04-17T07:10:22.419 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:22 smithi103 ceph-mon[35495]: osdmap e77: 12 total, 11 up, 12 in 2024-04-17T07:10:22.419 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:22 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:22 smithi098 ceph-mon[35418]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-17T07:10:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:22 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/3660799791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c2d8c2-4547-437f-a8f4-d12f4a8047b3"}]: dispatch 2024-04-17T07:10:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:22 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86c2d8c2-4547-437f-a8f4-d12f4a8047b3"}]: dispatch 2024-04-17T07:10:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:22 smithi098 ceph-mon[35418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86c2d8c2-4547-437f-a8f4-d12f4a8047b3"}]': finished 2024-04-17T07:10:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:22 smithi098 ceph-mon[35418]: osdmap e77: 12 total, 11 up, 12 in 2024-04-17T07:10:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:22 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:23.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:23 smithi098 ceph-mon[35418]: from='client.? 172.21.15.103:0/2766980956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:10:23.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:23 smithi103 ceph-mon[35495]: from='client.? 172.21.15.103:0/2766980956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:10:23.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:23 smithi086 ceph-mon[31733]: from='client.? 172.21.15.103:0/2766980956' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-17T07:10:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:24 smithi098 ceph-mon[35418]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-17T07:10:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:24 smithi103 ceph-mon[35495]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-17T07:10:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:24 smithi086 ceph-mon[31733]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-17T07:10:26.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:26 smithi098 ceph-mon[35418]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:10:26.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:26 smithi103 ceph-mon[35495]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:10:26.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:26 smithi086 ceph-mon[31733]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-17T07:10:28.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:28 smithi086 ceph-mon[31733]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:28.439 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:28 smithi103 ceph-mon[35495]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:28.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:28 smithi098 ceph-mon[35418]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:30.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:30 smithi098 ceph-mon[35418]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:30.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:30 smithi103 ceph-mon[35495]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:30 smithi086 ceph-mon[31733]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:32 smithi098 ceph-mon[35418]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:32 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-17T07:10:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:32 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:32 smithi103 ceph-mon[35495]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:32 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-17T07:10:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:32 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:32.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:32 smithi086 ceph-mon[31733]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:32.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:32 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-17T07:10:32.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:32 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:33.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:33 smithi103 ceph-mon[35495]: Deploying daemon osd.11 on smithi103 2024-04-17T07:10:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:33 smithi098 ceph-mon[35418]: Deploying daemon osd.11 on smithi103 2024-04-17T07:10:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:33 smithi086 ceph-mon[31733]: Deploying daemon osd.11 on smithi103 2024-04-17T07:10:34.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:34 smithi103 ceph-mon[35495]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:34 smithi098 ceph-mon[35418]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:34 smithi086 ceph-mon[31733]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:36 smithi098 ceph-mon[35418]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:36.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:36 smithi103 ceph-mon[35495]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:36 smithi086 ceph-mon[31733]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:38 smithi098 ceph-mon[35418]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:38 smithi103 ceph-mon[35495]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:38.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:38 smithi086 ceph-mon[31733]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:40.436 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:40 smithi103 ceph-mon[35495]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:40.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:40 smithi098 ceph-mon[35418]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:40.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:40 smithi086 ceph-mon[31733]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:41 smithi098 ceph-mon[35418]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:41 smithi103 ceph-mon[35495]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:41 smithi086 ceph-mon[31733]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:44.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:44 smithi086 ceph-mon[31733]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:44.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:44.467 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:44 smithi103 ceph-mon[35495]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:44.468 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:44.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:44 smithi098 ceph-mon[35418]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:44 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:45.232 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:45.233 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:45.233 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:45.233 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:45.233 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:45 smithi103 ceph-mon[35495]: from='osd.11 [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:10:45.233 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:45 smithi103 ceph-mon[35495]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:10:45.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.338+0000 7f9edaa00700 1 -- 172.21.15.103:0/2568037633 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9ee4062e90 con 0x7f9ec80a3c40 2024-04-17T07:10:45.340 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 11 on host 'smithi103' 2024-04-17T07:10:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.341+0000 7f9ee9200700 1 -- 172.21.15.103:0/2568037633 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec80a3c40 msgr2=0x7f9ec80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.341+0000 7f9ee9200700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec80a3c40 0x7f9ec80a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4105d50 tx=0x7f9ed400b410 comp rx=0 tx=0).stop 2024-04-17T07:10:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.341+0000 7f9ee9200700 1 -- 172.21.15.103:0/2568037633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee4105ef0 msgr2=0x7f9ee419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.341+0000 7f9ee9200700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee4105ef0 0x7f9ee419b310 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0004790 tx=0x7f9ed0009580 comp rx=0 tx=0).stop 2024-04-17T07:10:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.342+0000 7f9ee9200700 1 -- 172.21.15.103:0/2568037633 shutdown_connections 2024-04-17T07:10:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.342+0000 7f9ee9200700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec80a3c40 0x7f9ec80a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.342+0000 7f9ee9200700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ee4106890 0x7f9ee41a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.342+0000 7f9ee9200700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ee4104cf0 0x7f9ee419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.342+0000 7f9ee9200700 1 --2- 172.21.15.103:0/2568037633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ee4105ef0 0x7f9ee419b310 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.342+0000 7f9ee9200700 1 -- 172.21.15.103:0/2568037633 >> 172.21.15.103:0/2568037633 conn(0x7f9ee41002b0 msgr2=0x7f9ee4109270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:10:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.342+0000 7f9ee9200700 1 -- 172.21.15.103:0/2568037633 shutdown_connections 2024-04-17T07:10:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:10:45.342+0000 7f9ee9200700 1 -- 172.21.15.103:0/2568037633 wait complete. 2024-04-17T07:10:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:45 smithi098 ceph-mon[35418]: from='osd.11 [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:10:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:45 smithi098 ceph-mon[35418]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:10:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:45.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:45 smithi086 ceph-mon[31733]: from='osd.11 [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:10:45.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:45 smithi086 ceph-mon[31733]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-17T07:10:46.021 DEBUG:teuthology.orchestra.run.smithi103:osd.11> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.11.service 2024-04-17T07:10:46.026 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-17T07:10:46.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd stat -f json 2024-04-17T07:10:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-mon[35495]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-mon[35495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-17T07:10:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-mon[35495]: osdmap e78: 12 total, 11 up, 12 in 2024-04-17T07:10:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-mon[35495]: from='osd.11 [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:10:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-mon[35495]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:10:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:46.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:46 smithi098 ceph-mon[35418]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:46.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:46 smithi098 ceph-mon[35418]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-17T07:10:46.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:46 smithi098 ceph-mon[35418]: osdmap e78: 12 total, 11 up, 12 in 2024-04-17T07:10:46.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:46.406 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:46 smithi098 ceph-mon[35418]: from='osd.11 [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:10:46.406 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:46 smithi098 ceph-mon[35418]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:10:46.406 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:46.406 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:46.406 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:46.477 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:10:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:46 smithi086 ceph-mon[31733]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:46 smithi086 ceph-mon[31733]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-17T07:10:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:46 smithi086 ceph-mon[31733]: osdmap e78: 12 total, 11 up, 12 in 2024-04-17T07:10:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:46 smithi086 ceph-mon[31733]: from='osd.11 [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:10:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:46 smithi086 ceph-mon[31733]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-17T07:10:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:46.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:46.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:10:47.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:10:46.716+0000 7f1625400700 -1 osd.11 0 waiting for initial osdmap 2024-04-17T07:10:47.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:10:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:10:46.749+0000 7f161b200700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T07:10:47.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:47 smithi086 ceph-mon[31733]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:10:47.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:47 smithi086 ceph-mon[31733]: osdmap e79: 12 total, 11 up, 12 in 2024-04-17T07:10:47.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:47 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:47.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:47 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:47.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:47 smithi086 ceph-mon[31733]: from='osd.11 ' entity='osd.11' 2024-04-17T07:10:47.367 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:47 smithi103 ceph-mon[35495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:10:47.367 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:47 smithi103 ceph-mon[35495]: osdmap e79: 12 total, 11 up, 12 in 2024-04-17T07:10:47.367 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:47 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:47.367 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:47 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:47.368 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:47 smithi103 ceph-mon[35495]: from='osd.11 ' entity='osd.11' 2024-04-17T07:10:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:47 smithi098 ceph-mon[35418]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-17T07:10:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:47 smithi098 ceph-mon[35418]: osdmap e79: 12 total, 11 up, 12 in 2024-04-17T07:10:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:47 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:47 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:47 smithi098 ceph-mon[35418]: from='osd.11 ' entity='osd.11' 2024-04-17T07:10:48.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:48 smithi086 ceph-mon[31733]: purged_snaps scrub starts 2024-04-17T07:10:48.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:48 smithi086 ceph-mon[31733]: purged_snaps scrub ok 2024-04-17T07:10:48.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:48 smithi086 ceph-mon[31733]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:48.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:48.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:48 smithi086 ceph-mon[31733]: osd.11 [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] boot 2024-04-17T07:10:48.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:48 smithi086 ceph-mon[31733]: osdmap e80: 12 total, 12 up, 12 in 2024-04-17T07:10:48.422 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:48 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:48 smithi098 ceph-mon[35418]: purged_snaps scrub starts 2024-04-17T07:10:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:48 smithi098 ceph-mon[35418]: purged_snaps scrub ok 2024-04-17T07:10:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:48 smithi098 ceph-mon[35418]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:48 smithi098 ceph-mon[35418]: osd.11 [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] boot 2024-04-17T07:10:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:48 smithi098 ceph-mon[35418]: osdmap e80: 12 total, 12 up, 12 in 2024-04-17T07:10:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:48 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:48 smithi103 ceph-mon[35495]: purged_snaps scrub starts 2024-04-17T07:10:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:48 smithi103 ceph-mon[35495]: purged_snaps scrub ok 2024-04-17T07:10:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:48 smithi103 ceph-mon[35495]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-17T07:10:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:48 smithi103 ceph-mon[35495]: osd.11 [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] boot 2024-04-17T07:10:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:48 smithi103 ceph-mon[35495]: osdmap e80: 12 total, 12 up, 12 in 2024-04-17T07:10:48.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:48 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-17T07:10:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.741+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1326722228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc102ab0 msgr2=0x7f8abc102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.741+0000 7f8ac1c00700 1 --2- 172.21.15.86:0/1326722228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc102ab0 0x7f8abc102ed0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0099c0 tx=0x7f8aac009cd0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.741+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1326722228 shutdown_connections 2024-04-17T07:10:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.741+0000 7f8ac1c00700 1 --2- 172.21.15.86:0/1326722228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8abc104650 0x7f8abc10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.741+0000 7f8ac1c00700 1 --2- 172.21.15.86:0/1326722228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc102ab0 0x7f8abc102ed0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.741+0000 7f8ac1c00700 1 --2- 172.21.15.86:0/1326722228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8abc103cb0 0x7f8abc104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.741+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1326722228 >> 172.21.15.86:0/1326722228 conn(0x7f8abc0fe050 msgr2=0x7f8abc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:10:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.741+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1326722228 shutdown_connections 2024-04-17T07:10:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.741+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1326722228 wait complete. 2024-04-17T07:10:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.742+0000 7f8ac1c00700 1 Processor -- start 2024-04-17T07:10:48.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.742+0000 7f8ac1c00700 1 -- start start 2024-04-17T07:10:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8ac1c00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8abc102ab0 0x7f8abc19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8ac1c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc103cb0 0x7f8abc19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8ac1c00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8abc104650 0x7f8abc1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8ac1c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc10e540 con 0x7f8abc103cb0 2024-04-17T07:10:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8ac1c00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc10e3c0 con 0x7f8abc102ab0 2024-04-17T07:10:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8ac1c00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc10e6c0 con 0x7f8abc104650 2024-04-17T07:10:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8aba000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc103cb0 0x7f8abc19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8aba000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc103cb0 0x7f8abc19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36314/0 (socket says 172.21.15.86:36314) 2024-04-17T07:10:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8aba000700 1 -- 172.21.15.86:0/1564190907 learned_addr learned my addr 172.21.15.86:0/1564190907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:10:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8abb400700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8abc104650 0x7f8abc1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8abaa00700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8abc102ab0 0x7f8abc19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8aba000700 1 -- 172.21.15.86:0/1564190907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8abc104650 msgr2=0x7f8abc1a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.743+0000 7f8aba000700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8abc104650 0x7f8abc1a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.744+0000 7f8aba000700 1 -- 172.21.15.86:0/1564190907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8abc102ab0 msgr2=0x7f8abc19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.744+0000 7f8aba000700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8abc102ab0 0x7f8abc19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.744+0000 7f8aba000700 1 -- 172.21.15.86:0/1564190907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc1a2a80 con 0x7f8abc103cb0 2024-04-17T07:10:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.744+0000 7f8aba000700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc103cb0 0x7f8abc19b360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800d9b0 tx=0x7f8aa800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:10:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.744+0000 7f8ab3400700 1 -- 172.21.15.86:0/1564190907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8017070 con 0x7f8abc103cb0 2024-04-17T07:10:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.744+0000 7f8ab3400700 1 -- 172.21.15.86:0/1564190907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8aa8005020 con 0x7f8abc103cb0 2024-04-17T07:10:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.744+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1564190907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc1a2d70 con 0x7f8abc103cb0 2024-04-17T07:10:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.744+0000 7f8ab3400700 1 -- 172.21.15.86:0/1564190907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8009b50 con 0x7f8abc103cb0 2024-04-17T07:10:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.744+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1564190907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc1a3200 con 0x7f8abc103cb0 2024-04-17T07:10:48.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.745+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1564190907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc069770 con 0x7f8abc103cb0 2024-04-17T07:10:48.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.748+0000 7f8ab3400700 1 -- 172.21.15.86:0/1564190907 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8aa8009cf0 con 0x7f8abc103cb0 2024-04-17T07:10:48.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.749+0000 7f8ab3400700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8aa00a3c40 0x7f8aa00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:48.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.749+0000 7f8ab3400700 1 -- 172.21.15.86:0/1564190907 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f8aa80d02e0 con 0x7f8abc103cb0 2024-04-17T07:10:48.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.750+0000 7f8abaa00700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8aa00a3c40 0x7f8aa00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:48.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.750+0000 7f8abaa00700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8aa00a3c40 0x7f8aa00a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000c00 tx=0x7f8aac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:10:48.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.752+0000 7f8ab3400700 1 -- 172.21.15.86:0/1564190907 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8abc069770 con 0x7f8abc103cb0 2024-04-17T07:10:48.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.980+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1564190907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8abc1a3770 con 0x7f8abc103cb0 2024-04-17T07:10:48.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.981+0000 7f8ab3400700 1 -- 172.21.15.86:0/1564190907 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f8abc1a3770 con 0x7f8abc103cb0 2024-04-17T07:10:48.981 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:10:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.984+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1564190907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8aa00a3c40 msgr2=0x7f8aa00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.984+0000 7f8ac1c00700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8aa00a3c40 0x7f8aa00a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000c00 tx=0x7f8aac005fd0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.984+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1564190907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc103cb0 msgr2=0x7f8abc19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.984+0000 7f8ac1c00700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc103cb0 0x7f8abc19b360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800d9b0 tx=0x7f8aa800dcc0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.985+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1564190907 shutdown_connections 2024-04-17T07:10:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.985+0000 7f8ac1c00700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8aa00a3c40 0x7f8aa00a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.985+0000 7f8ac1c00700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8abc104650 0x7f8abc1a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.985+0000 7f8ac1c00700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8abc102ab0 0x7f8abc19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.985+0000 7f8ac1c00700 1 --2- 172.21.15.86:0/1564190907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc103cb0 0x7f8abc19b360 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:48.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.985+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1564190907 >> 172.21.15.86:0/1564190907 conn(0x7f8abc0fe050 msgr2=0x7f8abc074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:10:48.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.985+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1564190907 shutdown_connections 2024-04-17T07:10:48.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:48.985+0000 7f8ac1c00700 1 -- 172.21.15.86:0/1564190907 wait complete. 2024-04-17T07:10:49.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:49 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1564190907' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-17T07:10:49.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:49 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1564190907' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-17T07:10:49.525 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:49 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1564190907' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-17T07:10:49.903 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1713337847,"num_in_osds":12,"osd_in_since":1713337821,"num_remapped_pgs":0} 2024-04-17T07:10:49.904 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd dump --format=json 2024-04-17T07:10:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:10:50.490 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:50 smithi086 ceph-mon[31733]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:50.490 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:50 smithi086 ceph-mon[31733]: osdmap e81: 12 total, 12 up, 12 in 2024-04-17T07:10:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:50 smithi098 ceph-mon[35418]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:50 smithi098 ceph-mon[35418]: osdmap e81: 12 total, 12 up, 12 in 2024-04-17T07:10:50.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:50 smithi103 ceph-mon[35495]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:50.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:50 smithi103 ceph-mon[35495]: osdmap e81: 12 total, 12 up, 12 in 2024-04-17T07:10:52.024 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: Detected new or changed devices on smithi103 2024-04-17T07:10:52.024 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.024 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.024 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.024 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.024 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.025 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.025 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi103 to 4272M 2024-04-17T07:10:52.025 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.025 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:52.025 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:52.025 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.025 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:51 smithi086 ceph-mon[31733]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:52.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: Detected new or changed devices on smithi103 2024-04-17T07:10:52.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi103 to 4272M 2024-04-17T07:10:52.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:52.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:52.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:51 smithi103 ceph-mon[35495]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: Detected new or changed devices on smithi103 2024-04-17T07:10:52.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi103 to 4272M 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:10:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:51 smithi098 ceph-mon[35418]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.545+0000 7f4272200700 1 -- 172.21.15.86:0/313487148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c10f210 msgr2=0x7f426c111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.545+0000 7f4272200700 1 --2- 172.21.15.86:0/313487148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c10f210 0x7f426c111620 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f426400b0a0 tx=0x7f426400b3b0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.545+0000 7f4272200700 1 -- 172.21.15.86:0/313487148 shutdown_connections 2024-04-17T07:10:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.545+0000 7f4272200700 1 --2- 172.21.15.86:0/313487148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c10f210 0x7f426c111620 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.545+0000 7f4272200700 1 --2- 172.21.15.86:0/313487148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f426c074e20 0x7f426c075200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.545+0000 7f4272200700 1 --2- 172.21.15.86:0/313487148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f426c075740 0x7f426c10ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.545+0000 7f4272200700 1 -- 172.21.15.86:0/313487148 >> 172.21.15.86:0/313487148 conn(0x7f426c0fe460 msgr2=0x7f426c100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:10:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.545+0000 7f4272200700 1 -- 172.21.15.86:0/313487148 shutdown_connections 2024-04-17T07:10:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.545+0000 7f4272200700 1 -- 172.21.15.86:0/313487148 wait complete. 2024-04-17T07:10:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.546+0000 7f4272200700 1 Processor -- start 2024-04-17T07:10:52.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.547+0000 7f4272200700 1 -- start start 2024-04-17T07:10:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.547+0000 7f4272200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c074e20 0x7f426c102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.547+0000 7f4272200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f426c075740 0x7f426c102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.547+0000 7f4272200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f426c10f210 0x7f426c1033b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.547+0000 7f4272200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f426c114d50 con 0x7f426c074e20 2024-04-17T07:10:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.547+0000 7f4272200700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f426c114bd0 con 0x7f426c075740 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.547+0000 7f4272200700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f426c114ed0 con 0x7f426c10f210 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.547+0000 7f4270e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c074e20 0x7f426c102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.548+0000 7f4270e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c074e20 0x7f426c102930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36332/0 (socket says 172.21.15.86:36332) 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.548+0000 7f4270e00700 1 -- 172.21.15.86:0/4155187885 learned_addr learned my addr 172.21.15.86:0/4155187885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.548+0000 7f426be00700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f426c075740 0x7f426c102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.548+0000 7f4271800700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f426c10f210 0x7f426c1033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.548+0000 7f4270e00700 1 -- 172.21.15.86:0/4155187885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f426c10f210 msgr2=0x7f426c1033b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.548+0000 7f4270e00700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f426c10f210 0x7f426c1033b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.548+0000 7f4270e00700 1 -- 172.21.15.86:0/4155187885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f426c075740 msgr2=0x7f426c102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.548+0000 7f4270e00700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f426c075740 0x7f426c102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.548+0000 7f4270e00700 1 -- 172.21.15.86:0/4155187885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c1ab8e0 con 0x7f426c074e20 2024-04-17T07:10:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.548+0000 7f4270e00700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c074e20 0x7f426c102930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f426000e9a0 tx=0x7f426000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:10:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.549+0000 7f4269600700 1 -- 172.21.15.86:0/4155187885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426000cbd0 con 0x7f426c074e20 2024-04-17T07:10:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.549+0000 7f4269600700 1 -- 172.21.15.86:0/4155187885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f426000cd70 con 0x7f426c074e20 2024-04-17T07:10:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.549+0000 7f4272200700 1 -- 172.21.15.86:0/4155187885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426c1abbd0 con 0x7f426c074e20 2024-04-17T07:10:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.549+0000 7f4269600700 1 -- 172.21.15.86:0/4155187885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260018680 con 0x7f426c074e20 2024-04-17T07:10:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.549+0000 7f4272200700 1 -- 172.21.15.86:0/4155187885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426c1ac110 con 0x7f426c074e20 2024-04-17T07:10:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.550+0000 7f4272200700 1 -- 172.21.15.86:0/4155187885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426c0764d0 con 0x7f426c074e20 2024-04-17T07:10:52.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.552+0000 7f4269600700 1 -- 172.21.15.86:0/4155187885 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4260010430 con 0x7f426c074e20 2024-04-17T07:10:52.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.553+0000 7f4269600700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42540a3c40 0x7f42540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:52.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.554+0000 7f4269600700 1 -- 172.21.15.86:0/4155187885 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f42600d0230 con 0x7f426c074e20 2024-04-17T07:10:52.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.554+0000 7f426be00700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42540a3c40 0x7f42540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.555+0000 7f426be00700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42540a3c40 0x7f42540a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f426c104050 tx=0x7f4258009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:10:52.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.558+0000 7f4269600700 1 -- 172.21.15.86:0/4155187885 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f426009d010 con 0x7f426c074e20 2024-04-17T07:10:52.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.785+0000 7f4272200700 1 -- 172.21.15.86:0/4155187885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f426c02df90 con 0x7f426c074e20 2024-04-17T07:10:52.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.788+0000 7f4269600700 1 -- 172.21.15.86:0/4155187885 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18385 (secure 0 0 0) 0x7f426009c910 con 0x7f426c074e20 2024-04-17T07:10:52.788 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:10:52.789 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:10:49.137829+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:10:52.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.791+0000 7f4272200700 1 -- 172.21.15.86:0/4155187885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42540a3c40 msgr2=0x7f42540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:52.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.791+0000 7f4272200700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42540a3c40 0x7f42540a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f426c104050 tx=0x7f4258009260 comp rx=0 tx=0).stop 2024-04-17T07:10:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.791+0000 7f4272200700 1 -- 172.21.15.86:0/4155187885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c074e20 msgr2=0x7f426c102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.791+0000 7f4272200700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c074e20 0x7f426c102930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f426000e9a0 tx=0x7f426000ecb0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.791+0000 7f4272200700 1 -- 172.21.15.86:0/4155187885 shutdown_connections 2024-04-17T07:10:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.791+0000 7f4272200700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42540a3c40 0x7f42540a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.791+0000 7f4272200700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f426c10f210 0x7f426c1033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.791+0000 7f4272200700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f426c074e20 0x7f426c102930 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.791+0000 7f4272200700 1 --2- 172.21.15.86:0/4155187885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f426c075740 0x7f426c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.791+0000 7f4272200700 1 -- 172.21.15.86:0/4155187885 >> 172.21.15.86:0/4155187885 conn(0x7f426c0fe460 msgr2=0x7f426c0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:10:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.792+0000 7f4272200700 1 -- 172.21.15.86:0/4155187885 shutdown_connections 2024-04-17T07:10:52.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:52.792+0000 7f4272200700 1 -- 172.21.15.86:0/4155187885 wait complete. 2024-04-17T07:10:53.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:52 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4155187885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:10:53.100 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:52 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4155187885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:10:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:52 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4155187885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:10:53.715 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T07:10:53.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-17T07:10:54.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:53 smithi086 ceph-mon[31733]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:54.159 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:10:54.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:53 smithi098 ceph-mon[35418]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:54.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:53 smithi103 ceph-mon[35495]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:56.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:56 smithi086 ceph-mon[31733]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:56.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.374+0000 7f801fe00700 1 -- 172.21.15.86:0/2371500875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801806ba80 msgr2=0x7f801810f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:56.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.374+0000 7f801fe00700 1 --2- 172.21.15.86:0/2371500875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801806ba80 0x7f801810f050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8004009990 tx=0x7f8004009ca0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.374+0000 7f801fe00700 1 -- 172.21.15.86:0/2371500875 shutdown_connections 2024-04-17T07:10:56.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.374+0000 7f801fe00700 1 --2- 172.21.15.86:0/2371500875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801810f590 0x7f8018111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.374+0000 7f801fe00700 1 --2- 172.21.15.86:0/2371500875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f801806b160 0x7f801806b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.374+0000 7f801fe00700 1 --2- 172.21.15.86:0/2371500875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801806ba80 0x7f801810f050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.374+0000 7f801fe00700 1 -- 172.21.15.86:0/2371500875 >> 172.21.15.86:0/2371500875 conn(0x7f80180779a0 msgr2=0x7f8018077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:10:56.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.374+0000 7f801fe00700 1 -- 172.21.15.86:0/2371500875 shutdown_connections 2024-04-17T07:10:56.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.375+0000 7f801fe00700 1 -- 172.21.15.86:0/2371500875 wait complete. 2024-04-17T07:10:56.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.375+0000 7f801fe00700 1 Processor -- start 2024-04-17T07:10:56.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.376+0000 7f801fe00700 1 -- start start 2024-04-17T07:10:56.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.376+0000 7f801fe00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801806b160 0x7f801819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:56.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.376+0000 7f801fe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801806ba80 0x7f801819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:56.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.376+0000 7f801fe00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f801810f590 0x7f801819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:56.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.376+0000 7f801fe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8018114e70 con 0x7f801806ba80 2024-04-17T07:10:56.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.376+0000 7f801fe00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8018114cf0 con 0x7f801810f590 2024-04-17T07:10:56.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.376+0000 7f801fe00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8018114ff0 con 0x7f801806b160 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.376+0000 7f801ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801806ba80 0x7f801819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.376+0000 7f801ca00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801806ba80 0x7f801819b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36348/0 (socket says 172.21.15.86:36348) 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.376+0000 7f801ca00700 1 -- 172.21.15.86:0/3938168114 learned_addr learned my addr 172.21.15.86:0/3938168114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.377+0000 7f801d400700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801806b160 0x7f801819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.377+0000 7f801de00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f801810f590 0x7f801819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.377+0000 7f801ca00700 1 -- 172.21.15.86:0/3938168114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801806b160 msgr2=0x7f801819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.377+0000 7f801ca00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801806b160 0x7f801819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.377+0000 7f801ca00700 1 -- 172.21.15.86:0/3938168114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f801810f590 msgr2=0x7f801819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.377+0000 7f801ca00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f801810f590 0x7f801819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.377+0000 7f801ca00700 1 -- 172.21.15.86:0/3938168114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80181a0310 con 0x7f801806ba80 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.377+0000 7f801de00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f801810f590 0x7f801819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:10:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.377+0000 7f801ca00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801806ba80 0x7f801819b840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80040047b0 tx=0x7f800400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:10:56.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.377+0000 7f8016000700 1 -- 172.21.15.86:0/3938168114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004026070 con 0x7f801806ba80 2024-04-17T07:10:56.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.378+0000 7f8016000700 1 -- 172.21.15.86:0/3938168114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8004004100 con 0x7f801806ba80 2024-04-17T07:10:56.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.378+0000 7f801fe00700 1 -- 172.21.15.86:0/3938168114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80181a05a0 con 0x7f801806ba80 2024-04-17T07:10:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.378+0000 7f8016000700 1 -- 172.21.15.86:0/3938168114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004005290 con 0x7f801806ba80 2024-04-17T07:10:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.378+0000 7f801fe00700 1 -- 172.21.15.86:0/3938168114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80181a7e80 con 0x7f801806ba80 2024-04-17T07:10:56.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.379+0000 7f801fe00700 1 -- 172.21.15.86:0/3938168114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8018101fb0 con 0x7f801806ba80 2024-04-17T07:10:56.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.381+0000 7f8016000700 1 -- 172.21.15.86:0/3938168114 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8004029070 con 0x7f801806ba80 2024-04-17T07:10:56.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.382+0000 7f8016000700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f80000a3c40 0x7f80000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:10:56.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.382+0000 7f8016000700 1 -- 172.21.15.86:0/3938168114 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f80040d0ee0 con 0x7f801806ba80 2024-04-17T07:10:56.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.383+0000 7f801d400700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f80000a3c40 0x7f80000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:10:56.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.383+0000 7f801d400700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f80000a3c40 0x7f80000a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f800c005ea0 tx=0x7f800c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:10:56.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.385+0000 7f8016000700 1 -- 172.21.15.86:0/3938168114 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8004030020 con 0x7f801806ba80 2024-04-17T07:10:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:56 smithi098 ceph-mon[35418]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:56 smithi103 ceph-mon[35495]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:56.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.610+0000 7f801fe00700 1 -- 172.21.15.86:0/3938168114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f801819c660 con 0x7f801806ba80 2024-04-17T07:10:56.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.611+0000 7f8016000700 1 -- 172.21.15.86:0/3938168114 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f8004013070 con 0x7f801806ba80 2024-04-17T07:10:56.611 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:10:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.614+0000 7f801fe00700 1 -- 172.21.15.86:0/3938168114 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f80000a3c40 msgr2=0x7f80000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.614+0000 7f801fe00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f80000a3c40 0x7f80000a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f800c005ea0 tx=0x7f800c005e30 comp rx=0 tx=0).stop 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.614+0000 7f801fe00700 1 -- 172.21.15.86:0/3938168114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801806ba80 msgr2=0x7f801819b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.614+0000 7f801fe00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801806ba80 0x7f801819b840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80040047b0 tx=0x7f800400beb0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.614+0000 7f801fe00700 1 -- 172.21.15.86:0/3938168114 shutdown_connections 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.614+0000 7f801fe00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f80000a3c40 0x7f80000a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.614+0000 7f801fe00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f801810f590 0x7f801819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.614+0000 7f801fe00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f801806b160 0x7f801819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.614+0000 7f801fe00700 1 --2- 172.21.15.86:0/3938168114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801806ba80 0x7f801819b840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.614+0000 7f801fe00700 1 -- 172.21.15.86:0/3938168114 >> 172.21.15.86:0/3938168114 conn(0x7f80180779a0 msgr2=0x7f80181109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.615+0000 7f801fe00700 1 -- 172.21.15.86:0/3938168114 shutdown_connections 2024-04-17T07:10:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:10:56.615+0000 7f801fe00700 1 -- 172.21.15.86:0/3938168114 wait complete. 2024-04-17T07:10:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:57 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3938168114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.a on smithi086 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.d on smithi086 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.g on smithi086 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.j on smithi086 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.b on smithi098 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.e on smithi098 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.h on smithi098 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.k on smithi098 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.c on smithi103 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.f on smithi103 2024-04-17T07:10:57.498 INFO:tasks.cephadm:Adding mds.i on smithi103 2024-04-17T07:10:57.499 INFO:tasks.cephadm:Adding mds.l on smithi103 2024-04-17T07:10:57.499 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph orch apply mds all '12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l' 2024-04-17T07:10:57.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:57 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3938168114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:10:57.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:57 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3938168114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:10:57.960 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:10:58.352 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:10:58 smithi103 ceph-mon[35495]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:58.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:10:58 smithi098 ceph-mon[35418]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:10:58.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:10:58 smithi086 ceph-mon[31733]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:00.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.194+0000 7f3aa8c00700 1 -- 172.21.15.103:0/3771506757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3aa4103bf0 msgr2=0x7f3aa4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:00.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.194+0000 7f3aa8c00700 1 --2- 172.21.15.103:0/3771506757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3aa4103bf0 0x7f3aa4104050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0098f0 tx=0x7f3a8c009c00 comp rx=0 tx=0).stop 2024-04-17T07:11:00.195 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:00 smithi103 ceph-mon[35495]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.194+0000 7f3aa8c00700 1 -- 172.21.15.103:0/3771506757 shutdown_connections 2024-04-17T07:11:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.194+0000 7f3aa8c00700 1 --2- 172.21.15.103:0/3771506757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa4104590 0x7f3aa410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.194+0000 7f3aa8c00700 1 --2- 172.21.15.103:0/3771506757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3aa41029f0 0x7f3aa4102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.194+0000 7f3aa8c00700 1 --2- 172.21.15.103:0/3771506757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3aa4103bf0 0x7f3aa4104050 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.194+0000 7f3aa8c00700 1 -- 172.21.15.103:0/3771506757 >> 172.21.15.103:0/3771506757 conn(0x7f3aa40fdf90 msgr2=0x7f3aa41003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.194+0000 7f3aa8c00700 1 -- 172.21.15.103:0/3771506757 shutdown_connections 2024-04-17T07:11:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.194+0000 7f3aa8c00700 1 -- 172.21.15.103:0/3771506757 wait complete. 2024-04-17T07:11:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.195+0000 7f3aa8c00700 1 Processor -- start 2024-04-17T07:11:00.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa8c00700 1 -- start start 2024-04-17T07:11:00.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa8c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3aa41029f0 0x7f3aa4198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:00.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa8c00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa4103bf0 0x7f3aa41990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa8c00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3aa4104590 0x7f3aa41a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa8c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa410e4b0 con 0x7f3aa41029f0 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa8c00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa410e330 con 0x7f3aa4103bf0 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa8c00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa410e630 con 0x7f3aa4104590 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa3e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3aa4104590 0x7f3aa41a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa3400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3aa41029f0 0x7f3aa4198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa3e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3aa4104590 0x7f3aa41a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47906/0 (socket says 172.21.15.103:47906) 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.196+0000 7f3aa3e00700 1 -- 172.21.15.103:0/2636545009 learned_addr learned my addr 172.21.15.103:0/2636545009 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.197+0000 7f3a9aa00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa4103bf0 0x7f3aa41990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.197+0000 7f3aa3e00700 1 -- 172.21.15.103:0/2636545009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa4103bf0 msgr2=0x7f3aa41990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:00.199 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.197+0000 7f3aa3e00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa4103bf0 0x7f3aa41990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.197+0000 7f3aa3e00700 1 -- 172.21.15.103:0/2636545009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3aa41029f0 msgr2=0x7f3aa4198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.197+0000 7f3aa3e00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3aa41029f0 0x7f3aa4198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.197+0000 7f3aa3e00700 1 -- 172.21.15.103:0/2636545009 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa41a07f0 con 0x7f3aa4104590 2024-04-17T07:11:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.197+0000 7f3aa3e00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3aa4104590 0x7f3aa41a0150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c007f10 tx=0x7f3a9c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.197+0000 7f3aa0c00700 1 -- 172.21.15.103:0/2636545009 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c004bb0 con 0x7f3aa4104590 2024-04-17T07:11:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.198+0000 7f3aa0c00700 1 -- 172.21.15.103:0/2636545009 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a9c004d50 con 0x7f3aa4104590 2024-04-17T07:11:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.198+0000 7f3aa8c00700 1 -- 172.21.15.103:0/2636545009 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa41a0ae0 con 0x7f3aa4104590 2024-04-17T07:11:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.198+0000 7f3aa0c00700 1 -- 172.21.15.103:0/2636545009 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c005990 con 0x7f3aa4104590 2024-04-17T07:11:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.198+0000 7f3aa8c00700 1 -- 172.21.15.103:0/2636545009 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa41a0fd0 con 0x7f3aa4104590 2024-04-17T07:11:00.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.199+0000 7f3aa8c00700 1 -- 172.21.15.103:0/2636545009 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa4069740 con 0x7f3aa4104590 2024-04-17T07:11:00.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.201+0000 7f3aa0c00700 1 -- 172.21.15.103:0/2636545009 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3a9c01a020 con 0x7f3aa4104590 2024-04-17T07:11:00.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.202+0000 7f3aa0c00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a880a3c40 0x7f3a880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.203+0000 7f3aa0c00700 1 -- 172.21.15.103:0/2636545009 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3a9c0cf7a0 con 0x7f3aa4104590 2024-04-17T07:11:00.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.203+0000 7f3aa3400700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a880a3c40 0x7f3a880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:00.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.204+0000 7f3aa3400700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a880a3c40 0x7f3a880a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4103a50 tx=0x7f3a94009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:00.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.207+0000 7f3aa0c00700 1 -- 172.21.15.103:0/2636545009 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3a9c010070 con 0x7f3aa4104590 2024-04-17T07:11:00.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.441+0000 7f3aa8c00700 1 -- 172.21.15.103:0/2636545009 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l", "target": ["mon-mgr", ""]}) -- 0x7f3aa4063150 con 0x7f3a880a3c40 2024-04-17T07:11:00.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.455+0000 7f3aa0c00700 1 -- 172.21.15.103:0/2636545009 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f3aa4063150 con 0x7f3a880a3c40 2024-04-17T07:11:00.455 INFO:teuthology.orchestra.run.smithi103.stdout:Scheduled mds.all update... 2024-04-17T07:11:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 -- 172.21.15.103:0/2636545009 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a880a3c40 msgr2=0x7f3a880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a880a3c40 0x7f3a880a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4103a50 tx=0x7f3a94009310 comp rx=0 tx=0).stop 2024-04-17T07:11:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 -- 172.21.15.103:0/2636545009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3aa4104590 msgr2=0x7f3aa41a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3aa4104590 0x7f3aa41a0150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c007f10 tx=0x7f3a9c00d3b0 comp rx=0 tx=0).stop 2024-04-17T07:11:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 -- 172.21.15.103:0/2636545009 shutdown_connections 2024-04-17T07:11:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a880a3c40 0x7f3a880a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3aa4104590 0x7f3aa41a0150 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3aa41029f0 0x7f3aa4198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:00.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 --2- 172.21.15.103:0/2636545009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa4103bf0 0x7f3aa41990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 -- 172.21.15.103:0/2636545009 >> 172.21.15.103:0/2636545009 conn(0x7f3aa40fdf90 msgr2=0x7f3aa4106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.458+0000 7f3aa8c00700 1 -- 172.21.15.103:0/2636545009 shutdown_connections 2024-04-17T07:11:00.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-17T07:11:00.459+0000 7f3aa8c00700 1 -- 172.21.15.103:0/2636545009 wait complete. 2024-04-17T07:11:00.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:00 smithi086 ceph-mon[31733]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:00.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:00 smithi098 ceph-mon[35418]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:01.220 DEBUG:teuthology.orchestra.run.smithi086:mds.a> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.a.service 2024-04-17T07:11:01.229 DEBUG:teuthology.orchestra.run.smithi086:mds.d> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.d.service 2024-04-17T07:11:01.275 DEBUG:teuthology.orchestra.run.smithi086:mds.g> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.g.service 2024-04-17T07:11:01.321 DEBUG:teuthology.orchestra.run.smithi086:mds.j> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.j.service 2024-04-17T07:11:01.367 DEBUG:teuthology.orchestra.run.smithi098:mds.b> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.b.service 2024-04-17T07:11:01.375 DEBUG:teuthology.orchestra.run.smithi098:mds.e> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.e.service 2024-04-17T07:11:01.422 DEBUG:teuthology.orchestra.run.smithi098:mds.h> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.h.service 2024-04-17T07:11:01.467 DEBUG:teuthology.orchestra.run.smithi098:mds.k> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.k.service 2024-04-17T07:11:01.512 DEBUG:teuthology.orchestra.run.smithi103:mds.c> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.c.service 2024-04-17T07:11:01.519 DEBUG:teuthology.orchestra.run.smithi103:mds.f> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.f.service 2024-04-17T07:11:01.565 DEBUG:teuthology.orchestra.run.smithi103:mds.i> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.i.service 2024-04-17T07:11:01.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:11:01.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l;count:12 2024-04-17T07:11:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:11:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:11:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: Deploying daemon mds.l on smithi103 2024-04-17T07:11:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:01 smithi086 ceph-mon[31733]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:01.611 DEBUG:teuthology.orchestra.run.smithi103:mds.l> sudo journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.l.service 2024-04-17T07:11:01.657 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-17T07:11:01.657 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:11:01.713 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:11:01.713 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l;count:12 2024-04-17T07:11:01.713 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:01.713 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:11:01.713 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:01.713 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:11:01.713 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:01.713 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:01.714 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:01.714 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:01.714 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: Deploying daemon mds.l on smithi103 2024-04-17T07:11:01.714 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:01 smithi103 ceph-mon[35495]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:11:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l;count:12 2024-04-17T07:11:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:11:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:11:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: Deploying daemon mds.l on smithi103 2024-04-17T07:11:01.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:01 smithi098 ceph-mon[35418]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:01.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.885+0000 7f063b200640 1 -- 172.21.15.86:0/3805293049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06341053b0 msgr2=0x7f0634105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.885+0000 7f063b200640 1 --2- 172.21.15.86:0/3805293049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06341053b0 0x7f0634105830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f061c009900 tx=0x7f061c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:11:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.885+0000 7f063b200640 1 -- 172.21.15.86:0/3805293049 shutdown_connections 2024-04-17T07:11:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.885+0000 7f063b200640 1 --2- 172.21.15.86:0/3805293049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0634105d70 0x7f063410c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.885+0000 7f063b200640 1 --2- 172.21.15.86:0/3805293049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06341053b0 0x7f0634105830 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.885+0000 7f063b200640 1 --2- 172.21.15.86:0/3805293049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06341041c0 0x7f06341045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.885+0000 7f063b200640 1 -- 172.21.15.86:0/3805293049 >> 172.21.15.86:0/3805293049 conn(0x7f06340ff750 msgr2=0x7f0634101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.885+0000 7f063b200640 1 -- 172.21.15.86:0/3805293049 shutdown_connections 2024-04-17T07:11:01.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.885+0000 7f063b200640 1 -- 172.21.15.86:0/3805293049 wait complete. 2024-04-17T07:11:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.886+0000 7f063b200640 1 Processor -- start 2024-04-17T07:11:01.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.886+0000 7f063b200640 1 -- start start 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f063b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06341041c0 0x7f063419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f063b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06341053b0 0x7f063419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f063b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0634105d70 0x7f06341a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f063b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f063410fd50 con 0x7f06341041c0 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f063b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f063410fbd0 con 0x7f0634105d70 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f063b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f063410fed0 con 0x7f06341053b0 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f0633e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06341041c0 0x7f063419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f0633e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06341041c0 0x7f063419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56202/0 (socket says 172.21.15.86:56202) 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f0633e00640 1 -- 172.21.15.86:0/2248260990 learned_addr learned my addr 172.21.15.86:0/2248260990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f062b400640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06341053b0 0x7f063419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f0633e00640 1 -- 172.21.15.86:0/2248260990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06341053b0 msgr2=0x7f063419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f0633e00640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06341053b0 0x7f063419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:01.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f0633e00640 1 -- 172.21.15.86:0/2248260990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0634105d70 msgr2=0x7f06341a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f0633e00640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0634105d70 0x7f06341a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.887+0000 7f0633e00640 1 -- 172.21.15.86:0/2248260990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06341a1f70 con 0x7f06341041c0 2024-04-17T07:11:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.888+0000 7f0633e00640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06341041c0 0x7f063419a350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f062400c970 tx=0x7f062400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.888+0000 7f0631600640 1 -- 172.21.15.86:0/2248260990 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624004220 con 0x7f06341041c0 2024-04-17T07:11:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.888+0000 7f0631600640 1 -- 172.21.15.86:0/2248260990 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06240043c0 con 0x7f06341041c0 2024-04-17T07:11:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.888+0000 7f063b200640 1 -- 172.21.15.86:0/2248260990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06341a2260 con 0x7f06341041c0 2024-04-17T07:11:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.888+0000 7f0631600640 1 -- 172.21.15.86:0/2248260990 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062400f450 con 0x7f06341041c0 2024-04-17T07:11:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.888+0000 7f063b200640 1 -- 172.21.15.86:0/2248260990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06341a27d0 con 0x7f06341041c0 2024-04-17T07:11:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.889+0000 7f063b200640 1 -- 172.21.15.86:0/2248260990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063410b480 con 0x7f06341041c0 2024-04-17T07:11:01.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.891+0000 7f0631600640 1 -- 172.21.15.86:0/2248260990 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f06240079b0 con 0x7f06341041c0 2024-04-17T07:11:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.892+0000 7f0631600640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06180a3c30 0x7f06180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.893+0000 7f0631600640 1 -- 172.21.15.86:0/2248260990 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f06240cfc70 con 0x7f06341041c0 2024-04-17T07:11:01.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.893+0000 7f062b400640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06180a3c30 0x7f06180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.893+0000 7f062b400640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06180a3c30 0x7f06180a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f061c006010 tx=0x7f061c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:01.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:01.897+0000 7f0631600640 1 -- 172.21.15.86:0/2248260990 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0624014030 con 0x7f06341041c0 2024-04-17T07:11:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.104+0000 7f063b200640 1 -- 172.21.15.86:0/2248260990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0634062c00 con 0x7f06341041c0 2024-04-17T07:11:02.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.107+0000 7f0631600640 1 -- 172.21.15.86:0/2248260990 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18385 (secure 0 0 0) 0x7f062409caf0 con 0x7f06341041c0 2024-04-17T07:11:02.109 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:02.109 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:10:49.137829+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:11:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.109+0000 7f063b200640 1 -- 172.21.15.86:0/2248260990 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06180a3c30 msgr2=0x7f06180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.109+0000 7f063b200640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06180a3c30 0x7f06180a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f061c006010 tx=0x7f061c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.110+0000 7f063b200640 1 -- 172.21.15.86:0/2248260990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06341041c0 msgr2=0x7f063419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.110+0000 7f063b200640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06341041c0 0x7f063419a350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f062400c970 tx=0x7f062400ce30 comp rx=0 tx=0).stop 2024-04-17T07:11:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.110+0000 7f063b200640 1 -- 172.21.15.86:0/2248260990 shutdown_connections 2024-04-17T07:11:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.110+0000 7f063b200640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06180a3c30 0x7f06180a60f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.110+0000 7f063b200640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0634105d70 0x7f06341a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.110+0000 7f063b200640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06341053b0 0x7f063419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.110+0000 7f063b200640 1 --2- 172.21.15.86:0/2248260990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06341041c0 0x7f063419a350 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.110+0000 7f063b200640 1 -- 172.21.15.86:0/2248260990 >> 172.21.15.86:0/2248260990 conn(0x7f06340ff750 msgr2=0x7f0634108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.110+0000 7f063b200640 1 -- 172.21.15.86:0/2248260990 shutdown_connections 2024-04-17T07:11:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.110+0000 7f063b200640 1 -- 172.21.15.86:0/2248260990 wait complete. 2024-04-17T07:11:02.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T07:11:02.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:11:02.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.346+0000 7fbf6e400640 1 -- 172.21.15.86:0/1561908197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf68102f70 msgr2=0x7fbf681033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:02.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.346+0000 7fbf6e400640 1 --2- 172.21.15.86:0/1561908197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf68102f70 0x7fbf681033f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009900 tx=0x7fbf5402bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:02.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.347+0000 7fbf6e400640 1 -- 172.21.15.86:0/1561908197 shutdown_connections 2024-04-17T07:11:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.347+0000 7fbf6e400640 1 --2- 172.21.15.86:0/1561908197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf68103930 0x7fbf6810a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.347+0000 7fbf6e400640 1 --2- 172.21.15.86:0/1561908197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf68102f70 0x7fbf681033f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.347+0000 7fbf6e400640 1 --2- 172.21.15.86:0/1561908197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf68101d80 0x7fbf681021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.347+0000 7fbf6e400640 1 -- 172.21.15.86:0/1561908197 >> 172.21.15.86:0/1561908197 conn(0x7fbf680fd2f0 msgr2=0x7fbf680ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.347+0000 7fbf6e400640 1 -- 172.21.15.86:0/1561908197 shutdown_connections 2024-04-17T07:11:02.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.347+0000 7fbf6e400640 1 -- 172.21.15.86:0/1561908197 wait complete. 2024-04-17T07:11:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf6e400640 1 Processor -- start 2024-04-17T07:11:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf6e400640 1 -- start start 2024-04-17T07:11:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf6e400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf68101d80 0x7fbf6819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf6e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf68102f70 0x7fbf6819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf6e400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf68103930 0x7fbf681a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf6e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6810d900 con 0x7fbf68102f70 2024-04-17T07:11:02.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf6e400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6810d780 con 0x7fbf68101d80 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf6e400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6810da80 con 0x7fbf68103930 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf68102f70 0x7fbf6819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf68102f70 0x7fbf6819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56224/0 (socket says 172.21.15.86:56224) 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.348+0000 7fbf67e00640 1 -- 172.21.15.86:0/2170051016 learned_addr learned my addr 172.21.15.86:0/2170051016 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.349+0000 7fbf6d000640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf68101d80 0x7fbf6819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.349+0000 7fbf6da00640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf68103930 0x7fbf681a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.349+0000 7fbf67e00640 1 -- 172.21.15.86:0/2170051016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf68103930 msgr2=0x7fbf681a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.349+0000 7fbf67e00640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf68103930 0x7fbf681a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.349+0000 7fbf67e00640 1 -- 172.21.15.86:0/2170051016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf68101d80 msgr2=0x7fbf6819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.349+0000 7fbf67e00640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf68101d80 0x7fbf6819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.349+0000 7fbf67e00640 1 -- 172.21.15.86:0/2170051016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf681a1e00 con 0x7fbf68102f70 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.349+0000 7fbf67e00640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf68102f70 0x7fbf6819a720 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf54000c00 tx=0x7fbf540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.350+0000 7fbf65600640 1 -- 172.21.15.86:0/2170051016 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54043020 con 0x7fbf68102f70 2024-04-17T07:11:02.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.350+0000 7fbf65600640 1 -- 172.21.15.86:0/2170051016 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf540054c0 con 0x7fbf68102f70 2024-04-17T07:11:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.350+0000 7fbf6e400640 1 -- 172.21.15.86:0/2170051016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf681a2090 con 0x7fbf68102f70 2024-04-17T07:11:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.350+0000 7fbf65600640 1 -- 172.21.15.86:0/2170051016 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54003d80 con 0x7fbf68102f70 2024-04-17T07:11:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.350+0000 7fbf6e400640 1 -- 172.21.15.86:0/2170051016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf681a2630 con 0x7fbf68102f70 2024-04-17T07:11:02.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.351+0000 7fbf6e400640 1 -- 172.21.15.86:0/2170051016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf681a2950 con 0x7fbf68102f70 2024-04-17T07:11:02.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.353+0000 7fbf65600640 1 -- 172.21.15.86:0/2170051016 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbf54046020 con 0x7fbf68102f70 2024-04-17T07:11:02.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.354+0000 7fbf65600640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf500a3c30 0x7fbf500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:02.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.355+0000 7fbf65600640 1 -- 172.21.15.86:0/2170051016 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbf540f2410 con 0x7fbf68102f70 2024-04-17T07:11:02.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.355+0000 7fbf6d000640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf500a3c30 0x7fbf500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:02.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.355+0000 7fbf6d000640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf500a3c30 0x7fbf500a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0058c0 tx=0x7fbf5c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:02.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.359+0000 7fbf65600640 1 -- 172.21.15.86:0/2170051016 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbf5404d020 con 0x7fbf68102f70 2024-04-17T07:11:02.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.562+0000 7fbf6e400640 1 -- 172.21.15.86:0/2170051016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf681a2b20 con 0x7fbf68102f70 2024-04-17T07:11:02.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.562+0000 7fbf65600640 1 -- 172.21.15.86:0/2170051016 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fbf540bf1f0 con 0x7fbf68102f70 2024-04-17T07:11:02.563 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:11:02.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 -- 172.21.15.86:0/2170051016 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf500a3c30 msgr2=0x7fbf500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:02.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf500a3c30 0x7fbf500a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0058c0 tx=0x7fbf5c00a4c0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 -- 172.21.15.86:0/2170051016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf68102f70 msgr2=0x7fbf6819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:02.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf68102f70 0x7fbf6819a720 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf54000c00 tx=0x7fbf540042f0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 -- 172.21.15.86:0/2170051016 shutdown_connections 2024-04-17T07:11:02.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf500a3c30 0x7fbf500a60f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf68103930 0x7fbf681a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf68102f70 0x7fbf6819a720 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 --2- 172.21.15.86:0/2170051016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf68101d80 0x7fbf6819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 -- 172.21.15.86:0/2170051016 >> 172.21.15.86:0/2170051016 conn(0x7fbf680fd2f0 msgr2=0x7fbf68074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:02.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 -- 172.21.15.86:0/2170051016 shutdown_connections 2024-04-17T07:11:02.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.565+0000 7fbf6e400640 1 -- 172.21.15.86:0/2170051016 wait complete. 2024-04-17T07:11:02.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:02.624 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:02 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2248260990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:11:02.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:02 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2248260990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:11:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.793+0000 7f50b3200640 1 -- 172.21.15.86:0/504225716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50b4105b90 msgr2=0x7f50b410c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.793+0000 7f50b3200640 1 --2- 172.21.15.86:0/504225716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50b4105b90 0x7f50b410c420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009ca0 tx=0x7f50ac02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.793+0000 7f50b3200640 1 -- 172.21.15.86:0/504225716 shutdown_connections 2024-04-17T07:11:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.793+0000 7f50b3200640 1 --2- 172.21.15.86:0/504225716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50b4105b90 0x7f50b410c420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.793+0000 7f50b3200640 1 --2- 172.21.15.86:0/504225716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50b41051d0 0x7f50b4105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.793+0000 7f50b3200640 1 --2- 172.21.15.86:0/504225716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50b4103fe0 0x7f50b4104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.793+0000 7f50b3200640 1 -- 172.21.15.86:0/504225716 >> 172.21.15.86:0/504225716 conn(0x7f50b40ff570 msgr2=0x7f50b4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.794+0000 7f50b3200640 1 -- 172.21.15.86:0/504225716 shutdown_connections 2024-04-17T07:11:02.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.794+0000 7f50b3200640 1 -- 172.21.15.86:0/504225716 wait complete. 2024-04-17T07:11:02.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.794+0000 7f50b3200640 1 Processor -- start 2024-04-17T07:11:02.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b3200640 1 -- start start 2024-04-17T07:11:02.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50b4103fe0 0x7f50b419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:02.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50b41051d0 0x7f50b419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50b4105b90 0x7f50b41a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410fb20 con 0x7f50b4105b90 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b3200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410f9a0 con 0x7f50b41051d0 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410fca0 con 0x7f50b4103fe0 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50b4105b90 0x7f50b41a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50b4105b90 0x7f50b41a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56240/0 (socket says 172.21.15.86:56240) 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b2800640 1 -- 172.21.15.86:0/1759164299 learned_addr learned my addr 172.21.15.86:0/1759164299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b1e00640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50b4103fe0 0x7f50b419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b2800640 1 -- 172.21.15.86:0/1759164299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50b4103fe0 msgr2=0x7f50b419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b2800640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50b4103fe0 0x7f50b419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b2800640 1 -- 172.21.15.86:0/1759164299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50b41051d0 msgr2=0x7f50b419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.795+0000 7f50b2800640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50b41051d0 0x7f50b419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.796+0000 7f50b2800640 1 -- 172.21.15.86:0/1759164299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b41a1dd0 con 0x7f50b4105b90 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.796+0000 7f50b2800640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50b4105b90 0x7f50b41a1760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50ac032380 tx=0x7f50ac032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.796+0000 7f50aaa00640 1 -- 172.21.15.86:0/1759164299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac041020 con 0x7f50b4105b90 2024-04-17T07:11:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.796+0000 7f50aaa00640 1 -- 172.21.15.86:0/1759164299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50ac003e20 con 0x7f50b4105b90 2024-04-17T07:11:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.796+0000 7f50b3200640 1 -- 172.21.15.86:0/1759164299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b41a2060 con 0x7f50b4105b90 2024-04-17T07:11:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.796+0000 7f50aaa00640 1 -- 172.21.15.86:0/1759164299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac039040 con 0x7f50b4105b90 2024-04-17T07:11:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.796+0000 7f50b3200640 1 -- 172.21.15.86:0/1759164299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b41a2550 con 0x7f50b4105b90 2024-04-17T07:11:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.797+0000 7f50b3200640 1 -- 172.21.15.86:0/1759164299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b410b380 con 0x7f50b4105b90 2024-04-17T07:11:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.800+0000 7f50aaa00640 1 -- 172.21.15.86:0/1759164299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f50ac044020 con 0x7f50b4105b90 2024-04-17T07:11:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.801+0000 7f50aaa00640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50980a3bb0 0x7f50980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.801+0000 7f50aaa00640 1 -- 172.21.15.86:0/1759164299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f50ac0f27a0 con 0x7f50b4105b90 2024-04-17T07:11:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.801+0000 7f50b1e00640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50980a3bb0 0x7f50980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.802+0000 7f50b1e00640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50980a3bb0 0x7f50980a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f50a4005fd0 tx=0x7f50a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:02.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:02.804+0000 7f50aaa00640 1 -- 172.21.15.86:0/1759164299 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f50ac04b020 con 0x7f50b4105b90 2024-04-17T07:11:02.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:02 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2248260990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:11:03.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.046+0000 7f50b3200640 1 -- 172.21.15.86:0/1759164299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50b402d5d0 con 0x7f50b4105b90 2024-04-17T07:11:03.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.047+0000 7f50aaa00640 1 -- 172.21.15.86:0/1759164299 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f50ac0bf580 con 0x7f50b4105b90 2024-04-17T07:11:03.047 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:03.047 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-17T07:11:03.047 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 1 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.049+0000 7f50b3200640 1 -- 172.21.15.86:0/1759164299 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50980a3bb0 msgr2=0x7f50980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.049+0000 7f50b3200640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50980a3bb0 0x7f50980a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f50a4005fd0 tx=0x7f50a4005850 comp rx=0 tx=0).stop 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.049+0000 7f50b3200640 1 -- 172.21.15.86:0/1759164299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50b4105b90 msgr2=0x7f50b41a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.049+0000 7f50b3200640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50b4105b90 0x7f50b41a1760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f50ac032380 tx=0x7f50ac032820 comp rx=0 tx=0).stop 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.050+0000 7f50b3200640 1 -- 172.21.15.86:0/1759164299 shutdown_connections 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.050+0000 7f50b3200640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50980a3bb0 0x7f50980a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.050+0000 7f50b3200640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50b4105b90 0x7f50b41a1760 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.050+0000 7f50b3200640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50b41051d0 0x7f50b419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.050+0000 7f50b3200640 1 --2- 172.21.15.86:0/1759164299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50b4103fe0 0x7f50b419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.050+0000 7f50b3200640 1 -- 172.21.15.86:0/1759164299 >> 172.21.15.86:0/1759164299 conn(0x7f50b40ff570 msgr2=0x7f50b4108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.050+0000 7f50b3200640 1 -- 172.21.15.86:0/1759164299 shutdown_connections 2024-04-17T07:11:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:03.050+0000 7f50b3200640 1 -- 172.21.15.86:0/1759164299 wait complete. 2024-04-17T07:11:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:03 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2170051016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:11:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:03 smithi098 ceph-mon[35418]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:03 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1759164299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:03.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:03 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2170051016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:11:03.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:03 smithi103 ceph-mon[35495]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:03.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:03 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1759164299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:03 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2170051016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:11:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:03 smithi086 ceph-mon[31733]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:03 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1759164299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:04.090 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:11:03 smithi103 podman[66144]: 2024-04-17 07:11:03.836207792 +0000 UTC m=+0.171193049 container create 8cccce9551256533fc1da323353b8dc824e98278d843ada695a3b4f5ce3ada4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-17T07:11:04.091 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:11:04 smithi103 podman[66144]: 2024-04-17 07:11:04.090023349 +0000 UTC m=+0.425008670 container init 8cccce9551256533fc1da323353b8dc824e98278d843ada695a3b4f5ce3ada4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-17T07:11:04.526 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:11:04 smithi103 podman[66144]: 2024-04-17 07:11:04.100750109 +0000 UTC m=+0.435735372 container start 8cccce9551256533fc1da323353b8dc824e98278d843ada695a3b4f5ce3ada4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:11:04.527 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:11:04 smithi103 ceph-mds[66163]: -- 172.21.15.103:0/2603936439 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56371de15a40 con 0x56371dde8c00 2024-04-17T07:11:04.527 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:11:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-l[66159]: starting mds.l at 2024-04-17T07:11:04.527 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:11:04 smithi103 bash[66144]: 8cccce9551256533fc1da323353b8dc824e98278d843ada695a3b4f5ce3ada4f 2024-04-17T07:11:04.527 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:11:04 smithi103 systemd[1]: Started Ceph mds.l for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:05.068 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.288+0000 7f7714a00640 1 -- 172.21.15.86:0/2486260723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77101095d0 msgr2=0x7f7710109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.288+0000 7f7714a00640 1 --2- 172.21.15.86:0/2486260723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77101095d0 0x7f7710109a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009900 tx=0x7f76f802bc50 comp rx=0 tx=0).stop 2024-04-17T07:11:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.289+0000 7f7714a00640 1 -- 172.21.15.86:0/2486260723 shutdown_connections 2024-04-17T07:11:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.289+0000 7f7714a00640 1 --2- 172.21.15.86:0/2486260723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7710109f90 0x7f7710110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.289+0000 7f7714a00640 1 --2- 172.21.15.86:0/2486260723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77101095d0 0x7f7710109a50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:05.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.289+0000 7f7714a00640 1 --2- 172.21.15.86:0/2486260723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77101083e0 0x7f7710108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:05.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.289+0000 7f7714a00640 1 -- 172.21.15.86:0/2486260723 >> 172.21.15.86:0/2486260723 conn(0x7f77100773a0 msgr2=0x7f77100797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:05.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.289+0000 7f7714a00640 1 -- 172.21.15.86:0/2486260723 shutdown_connections 2024-04-17T07:11:05.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.289+0000 7f7714a00640 1 -- 172.21.15.86:0/2486260723 wait complete. 2024-04-17T07:11:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.290+0000 7f7714a00640 1 Processor -- start 2024-04-17T07:11:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.290+0000 7f7714a00640 1 -- start start 2024-04-17T07:11:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.290+0000 7f7714a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77101083e0 0x7f771019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.290+0000 7f7714a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77101095d0 0x7f771019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.290+0000 7f7714a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7710109f90 0x7f77101a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.290+0000 7f7714a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7710113f70 con 0x7f77101095d0 2024-04-17T07:11:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.290+0000 7f7714a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7710113df0 con 0x7f7710109f90 2024-04-17T07:11:05.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.290+0000 7f7714a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77101140f0 con 0x7f77101083e0 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.290+0000 7f7705e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77101095d0 0x7f771019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.291+0000 7f7705e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77101095d0 0x7f771019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56250/0 (socket says 172.21.15.86:56250) 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.291+0000 7f7705e00640 1 -- 172.21.15.86:0/2474031336 learned_addr learned my addr 172.21.15.86:0/2474031336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.291+0000 7f770f200640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7710109f90 0x7f77101a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.291+0000 7f770e800640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77101083e0 0x7f771019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.291+0000 7f7705e00640 1 -- 172.21.15.86:0/2474031336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77101083e0 msgr2=0x7f771019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.291+0000 7f7705e00640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77101083e0 0x7f771019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.291+0000 7f7705e00640 1 -- 172.21.15.86:0/2474031336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7710109f90 msgr2=0x7f77101a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.291+0000 7f7705e00640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7710109f90 0x7f77101a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.291+0000 7f7705e00640 1 -- 172.21.15.86:0/2474031336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77101a6190 con 0x7f77101095d0 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.291+0000 7f7705e00640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77101095d0 0x7f771019eab0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76f8000c00 tx=0x7f76f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.292+0000 7f7707e00640 1 -- 172.21.15.86:0/2474031336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8043020 con 0x7f77101095d0 2024-04-17T07:11:05.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.292+0000 7f7707e00640 1 -- 172.21.15.86:0/2474031336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76f8004aa0 con 0x7f77101095d0 2024-04-17T07:11:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.292+0000 7f7714a00640 1 -- 172.21.15.86:0/2474031336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77101a6420 con 0x7f77101095d0 2024-04-17T07:11:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.292+0000 7f7707e00640 1 -- 172.21.15.86:0/2474031336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8003d00 con 0x7f77101095d0 2024-04-17T07:11:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.292+0000 7f7714a00640 1 -- 172.21.15.86:0/2474031336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77101a69c0 con 0x7f77101095d0 2024-04-17T07:11:05.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.293+0000 7f7714a00640 1 -- 172.21.15.86:0/2474031336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771010f6a0 con 0x7f77101095d0 2024-04-17T07:11:05.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.295+0000 7f7707e00640 1 -- 172.21.15.86:0/2474031336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f76f8046020 con 0x7f77101095d0 2024-04-17T07:11:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.297+0000 7f7707e00640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76f40a3c30 0x7f76f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.297+0000 7f7707e00640 1 -- 172.21.15.86:0/2474031336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f76f80f2310 con 0x7f77101095d0 2024-04-17T07:11:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.297+0000 7f770e800640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76f40a3c30 0x7f76f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:05.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.298+0000 7f770e800640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76f40a3c30 0x7f76f40a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f77000058c0 tx=0x7f7700005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:05.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.301+0000 7f7707e00640 1 -- 172.21.15.86:0/2474031336 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f76f804d020 con 0x7f77101095d0 2024-04-17T07:11:05.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:05 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.103:6832/1707978686,v1:172.21.15.103:6833/1707978686] up:boot 2024-04-17T07:11:05.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:05 smithi098 ceph-mon[35418]: fsmap 1 up:standby 2024-04-17T07:11:05.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T07:11:05.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:05.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:05.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:05.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:05.443 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:05.444 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:05.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:05 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.103:6832/1707978686,v1:172.21.15.103:6833/1707978686] up:boot 2024-04-17T07:11:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:05 smithi103 ceph-mon[35495]: fsmap 1 up:standby 2024-04-17T07:11:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T07:11:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:05.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.543+0000 7f7714a00640 1 -- 172.21.15.86:0/2474031336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7710062b60 con 0x7f77101095d0 2024-04-17T07:11:05.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.544+0000 7f7707e00640 1 -- 172.21.15.86:0/2474031336 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f76f80bf0f0 con 0x7f77101095d0 2024-04-17T07:11:05.544 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:05.544 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-17T07:11:05.544 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 2 2024-04-17T07:11:05.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 -- 172.21.15.86:0/2474031336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76f40a3c30 msgr2=0x7f76f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:05.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76f40a3c30 0x7f76f40a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f77000058c0 tx=0x7f7700005850 comp rx=0 tx=0).stop 2024-04-17T07:11:05.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 -- 172.21.15.86:0/2474031336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77101095d0 msgr2=0x7f771019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:05.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77101095d0 0x7f771019eab0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76f8000c00 tx=0x7f76f80042f0 comp rx=0 tx=0).stop 2024-04-17T07:11:05.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 -- 172.21.15.86:0/2474031336 shutdown_connections 2024-04-17T07:11:05.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76f40a3c30 0x7f76f40a60f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:05.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7710109f90 0x7f77101a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:05.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77101095d0 0x7f771019eab0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:05.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 --2- 172.21.15.86:0/2474031336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77101083e0 0x7f771019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:05.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 -- 172.21.15.86:0/2474031336 >> 172.21.15.86:0/2474031336 conn(0x7f77100773a0 msgr2=0x7f771010c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:05.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 -- 172.21.15.86:0/2474031336 shutdown_connections 2024-04-17T07:11:05.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:05.546+0000 7f7714a00640 1 -- 172.21.15.86:0/2474031336 wait complete. 2024-04-17T07:11:05.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:05 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.103:6832/1707978686,v1:172.21.15.103:6833/1707978686] up:boot 2024-04-17T07:11:05.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:05 smithi086 ceph-mon[31733]: fsmap 1 up:standby 2024-04-17T07:11:05.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-17T07:11:05.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:05.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:05.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:05.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:05.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:05.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:06.474 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:06 smithi098 ceph-mon[35418]: Deploying daemon mds.k on smithi098 2024-04-17T07:11:06.475 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:06 smithi098 ceph-mon[35418]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:06.475 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:06 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2474031336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:06.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:06 smithi103 ceph-mon[35495]: Deploying daemon mds.k on smithi098 2024-04-17T07:11:06.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:06 smithi103 ceph-mon[35495]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:06.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:06 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2474031336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:06.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:06 smithi086 ceph-mon[31733]: Deploying daemon mds.k on smithi098 2024-04-17T07:11:06.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:06 smithi086 ceph-mon[31733]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:06.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:06 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2474031336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:07.372 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:07 smithi098 ceph-mon[35418]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:07 smithi103 ceph-mon[35495]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:07.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:07.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:07 smithi086 ceph-mon[31733]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:07.729 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:11:07 smithi098 systemd[1]: Starting Ceph mds.k for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:11:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.790+0000 7f3433e00640 1 -- 172.21.15.86:0/4243463459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34341051d0 msgr2=0x7f3434105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.790+0000 7f3433e00640 1 --2- 172.21.15.86:0/4243463459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34341051d0 0x7f3434105650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f341c009900 tx=0x7f341c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.790+0000 7f3433e00640 1 -- 172.21.15.86:0/4243463459 shutdown_connections 2024-04-17T07:11:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.790+0000 7f3433e00640 1 --2- 172.21.15.86:0/4243463459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3434105b90 0x7f343410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.790+0000 7f3433e00640 1 --2- 172.21.15.86:0/4243463459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34341051d0 0x7f3434105650 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.790+0000 7f3433e00640 1 --2- 172.21.15.86:0/4243463459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3434103fe0 0x7f3434104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.790+0000 7f3433e00640 1 -- 172.21.15.86:0/4243463459 >> 172.21.15.86:0/4243463459 conn(0x7f34340ff570 msgr2=0x7f3434101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.790+0000 7f3433e00640 1 -- 172.21.15.86:0/4243463459 shutdown_connections 2024-04-17T07:11:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.790+0000 7f3433e00640 1 -- 172.21.15.86:0/4243463459 wait complete. 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.791+0000 7f3433e00640 1 Processor -- start 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f3433e00640 1 -- start start 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f3433e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3434103fe0 0x7f343419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f3433e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34341051d0 0x7f343419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f3433e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3434105b90 0x7f34341a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f3433e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f343410fb70 con 0x7f34341051d0 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f3433e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f343410f9f0 con 0x7f3434105b90 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f3433e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f343410fcf0 con 0x7f3434103fe0 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f342be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34341051d0 0x7f343419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f342be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34341051d0 0x7f343419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36220/0 (socket says 172.21.15.86:36220) 2024-04-17T07:11:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f342be00640 1 -- 172.21.15.86:0/4066752501 learned_addr learned my addr 172.21.15.86:0/4066752501 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f3432a00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3434103fe0 0x7f343419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.792+0000 7f3433400640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3434105b90 0x7f34341a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f342be00640 1 -- 172.21.15.86:0/4066752501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3434103fe0 msgr2=0x7f343419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f342be00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3434103fe0 0x7f343419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f342be00640 1 -- 172.21.15.86:0/4066752501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3434105b90 msgr2=0x7f34341a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f342be00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3434105b90 0x7f34341a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f342be00640 1 -- 172.21.15.86:0/4066752501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34341a1e50 con 0x7f34341051d0 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f3433400640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3434105b90 0x7f34341a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f342be00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34341051d0 0x7f343419a770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f341c000c00 tx=0x7f341c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f342b400640 1 -- 172.21.15.86:0/4066752501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c043020 con 0x7f34341051d0 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f342b400640 1 -- 172.21.15.86:0/4066752501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341c0054c0 con 0x7f34341051d0 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f3433e00640 1 -- 172.21.15.86:0/4066752501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34341a20e0 con 0x7f34341051d0 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f342b400640 1 -- 172.21.15.86:0/4066752501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c003d80 con 0x7f34341051d0 2024-04-17T07:11:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.793+0000 7f3433e00640 1 -- 172.21.15.86:0/4066752501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34341a2630 con 0x7f34341051d0 2024-04-17T07:11:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.795+0000 7f3433e00640 1 -- 172.21.15.86:0/4066752501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343410b380 con 0x7f34341051d0 2024-04-17T07:11:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.797+0000 7f342b400640 1 -- 172.21.15.86:0/4066752501 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f341c046020 con 0x7f34341051d0 2024-04-17T07:11:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.798+0000 7f342b400640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34180a3bb0 0x7f34180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.798+0000 7f342b400640 1 -- 172.21.15.86:0/4066752501 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f341c0f24e0 con 0x7f34341051d0 2024-04-17T07:11:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.798+0000 7f3432a00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34180a3bb0 0x7f34180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.799+0000 7f3432a00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34180a3bb0 0x7f34180a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34240058c0 tx=0x7f342400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:07.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:07.802+0000 7f342b400640 1 -- 172.21.15.86:0/4066752501 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f341c04d020 con 0x7f34341051d0 2024-04-17T07:11:08.004 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:11:07 smithi098 podman[65220]: 2024-04-17 07:11:07.635912295 +0000 UTC m=+0.036392626 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:11:08.004 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:11:07 smithi098 podman[65220]: 2024-04-17 07:11:07.767868211 +0000 UTC m=+0.168348559 container create 6b3e665830c96c209a39bb971f362fda288a0c4f127f9265335097409985749a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-k, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-17T07:11:08.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.038+0000 7f3433e00640 1 -- 172.21.15.86:0/4066752501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3434062b60 con 0x7f34341051d0 2024-04-17T07:11:08.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.039+0000 7f342b400640 1 -- 172.21.15.86:0/4066752501 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f341c0bf2c0 con 0x7f34341051d0 2024-04-17T07:11:08.040 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:08.040 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-17T07:11:08.040 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 2 2024-04-17T07:11:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 -- 172.21.15.86:0/4066752501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34180a3bb0 msgr2=0x7f34180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34180a3bb0 0x7f34180a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34240058c0 tx=0x7f342400a4c0 comp rx=0 tx=0).stop 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 -- 172.21.15.86:0/4066752501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34341051d0 msgr2=0x7f343419a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34341051d0 0x7f343419a770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f341c000c00 tx=0x7f341c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 -- 172.21.15.86:0/4066752501 shutdown_connections 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34180a3bb0 0x7f34180a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3434105b90 0x7f34341a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34341051d0 0x7f343419a770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 --2- 172.21.15.86:0/4066752501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3434103fe0 0x7f343419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 -- 172.21.15.86:0/4066752501 >> 172.21.15.86:0/4066752501 conn(0x7f34340ff570 msgr2=0x7f3434108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 -- 172.21.15.86:0/4066752501 shutdown_connections 2024-04-17T07:11:08.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:08.042+0000 7f3433e00640 1 -- 172.21.15.86:0/4066752501 wait complete. 2024-04-17T07:11:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:08 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4066752501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:08.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:08 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4066752501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:08.503 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:11:08 smithi098 podman[65220]: 2024-04-17 07:11:08.080120736 +0000 UTC m=+0.480601090 container init 6b3e665830c96c209a39bb971f362fda288a0c4f127f9265335097409985749a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-17T07:11:08.503 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:11:08 smithi098 podman[65220]: 2024-04-17 07:11:08.089848231 +0000 UTC m=+0.490328586 container start 6b3e665830c96c209a39bb971f362fda288a0c4f127f9265335097409985749a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:11:08.503 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:11:08 smithi098 ceph-mds[65238]: -- 172.21.15.98:0/4067644559 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb97673a40 con 0x55eb97646800 2024-04-17T07:11:08.504 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:11:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-k[65234]: starting mds.k at 2024-04-17T07:11:08.504 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:11:08 smithi098 bash[65220]: 6b3e665830c96c209a39bb971f362fda288a0c4f127f9265335097409985749a 2024-04-17T07:11:08.504 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:11:08 smithi098 systemd[1]: Started Ceph mds.k for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:08.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:08 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4066752501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] up:boot 2024-04-17T07:11:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: fsmap 2 up:standby 2024-04-17T07:11:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T07:11:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: Deploying daemon mds.j on smithi086 2024-04-17T07:11:09.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:09 smithi098 ceph-mon[35418]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:09.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] up:boot 2024-04-17T07:11:09.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: fsmap 2 up:standby 2024-04-17T07:11:09.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T07:11:09.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:09.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:09.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:09.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:09.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:09.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:09.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: Deploying daemon mds.j on smithi086 2024-04-17T07:11:09.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:09 smithi103 ceph-mon[35495]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:09.537 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] up:boot 2024-04-17T07:11:09.537 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: fsmap 2 up:standby 2024-04-17T07:11:09.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-17T07:11:09.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:09.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:09.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:09.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:09.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:09.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:09.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: Deploying daemon mds.j on smithi086 2024-04-17T07:11:09.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:09 smithi086 ceph-mon[31733]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:10.061 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:10.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.273+0000 7f67c1200640 1 -- 172.21.15.86:0/3526373517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67bc101e00 msgr2=0x7f67bc102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:10.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.273+0000 7f67c1200640 1 --2- 172.21.15.86:0/3526373517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67bc101e00 0x7f67bc102220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009ca0 tx=0x7f67ac02c100 comp rx=0 tx=0).stop 2024-04-17T07:11:10.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.274+0000 7f67c1200640 1 -- 172.21.15.86:0/3526373517 shutdown_connections 2024-04-17T07:11:10.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.274+0000 7f67c1200640 1 --2- 172.21.15.86:0/3526373517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc1039b0 0x7f67bc10a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:10.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.274+0000 7f67c1200640 1 --2- 172.21.15.86:0/3526373517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67bc102ff0 0x7f67bc103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:10.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.274+0000 7f67c1200640 1 --2- 172.21.15.86:0/3526373517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67bc101e00 0x7f67bc102220 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:10.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.274+0000 7f67c1200640 1 -- 172.21.15.86:0/3526373517 >> 172.21.15.86:0/3526373517 conn(0x7f67bc0fd370 msgr2=0x7f67bc0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:10.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.274+0000 7f67c1200640 1 -- 172.21.15.86:0/3526373517 shutdown_connections 2024-04-17T07:11:10.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.274+0000 7f67c1200640 1 -- 172.21.15.86:0/3526373517 wait complete. 2024-04-17T07:11:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.275+0000 7f67c1200640 1 Processor -- start 2024-04-17T07:11:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.275+0000 7f67c1200640 1 -- start start 2024-04-17T07:11:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.275+0000 7f67c1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67bc101e00 0x7f67bc19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.275+0000 7f67c1200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102ff0 0x7f67bc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.275+0000 7f67c1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67bc1039b0 0x7f67bc1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.275+0000 7f67c1200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10d930 con 0x7f67bc101e00 2024-04-17T07:11:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.275+0000 7f67c1200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10d7b0 con 0x7f67bc1039b0 2024-04-17T07:11:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.275+0000 7f67c1200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10dab0 con 0x7f67bc102ff0 2024-04-17T07:11:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.275+0000 7f67bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67bc101e00 0x7f67bc19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:10.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.275+0000 7f67bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67bc101e00 0x7f67bc19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36234/0 (socket says 172.21.15.86:36234) 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67bb400640 1 -- 172.21.15.86:0/4252959435 learned_addr learned my addr 172.21.15.86:0/4252959435 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67b2a00640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102ff0 0x7f67bc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67bb400640 1 -- 172.21.15.86:0/4252959435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102ff0 msgr2=0x7f67bc19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67bb400640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102ff0 0x7f67bc19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67bb400640 1 -- 172.21.15.86:0/4252959435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67bc1039b0 msgr2=0x7f67bc1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67bb400640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67bc1039b0 0x7f67bc1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67bb400640 1 -- 172.21.15.86:0/4252959435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc1a1dd0 con 0x7f67bc101e00 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67b2a00640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102ff0 0x7f67bc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67bb400640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67bc101e00 0x7f67bc19a1b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f67ac033000 tx=0x7f67ac035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67b8c00640 1 -- 172.21.15.86:0/4252959435 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac039040 con 0x7f67bc101e00 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.276+0000 7f67b8c00640 1 -- 172.21.15.86:0/4252959435 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67ac044070 con 0x7f67bc101e00 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.277+0000 7f67c1200640 1 -- 172.21.15.86:0/4252959435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc1a2060 con 0x7f67bc101e00 2024-04-17T07:11:10.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.277+0000 7f67b8c00640 1 -- 172.21.15.86:0/4252959435 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0044d0 con 0x7f67bc101e00 2024-04-17T07:11:10.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.277+0000 7f67c1200640 1 -- 172.21.15.86:0/4252959435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a2550 con 0x7f67bc101e00 2024-04-17T07:11:10.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.278+0000 7f67c1200640 1 -- 172.21.15.86:0/4252959435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67bc1a27b0 con 0x7f67bc101e00 2024-04-17T07:11:10.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.280+0000 7f67b8c00640 1 -- 172.21.15.86:0/4252959435 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f67ac0327c0 con 0x7f67bc101e00 2024-04-17T07:11:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.281+0000 7f67b8c00640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f67a00a3bb0 0x7f67a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:10.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.282+0000 7f67b8c00640 1 -- 172.21.15.86:0/4252959435 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f67ac0f2750 con 0x7f67bc101e00 2024-04-17T07:11:10.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.282+0000 7f67b2a00640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f67a00a3bb0 0x7f67a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:10.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.283+0000 7f67b2a00640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f67a00a3bb0 0x7f67a00a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67bc19b6c0 tx=0x7f67a4005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:10.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.286+0000 7f67b8c00640 1 -- 172.21.15.86:0/4252959435 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f67ac0bf530 con 0x7f67bc101e00 2024-04-17T07:11:10.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.535+0000 7f67c1200640 1 -- 172.21.15.86:0/4252959435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67bc1a2980 con 0x7f67bc101e00 2024-04-17T07:11:10.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.536+0000 7f67b8c00640 1 -- 172.21.15.86:0/4252959435 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f67ac0bee30 con 0x7f67bc101e00 2024-04-17T07:11:10.535 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:10.535 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T07:11:10.535 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 3 2024-04-17T07:11:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 -- 172.21.15.86:0/4252959435 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f67a00a3bb0 msgr2=0x7f67a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f67a00a3bb0 0x7f67a00a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67bc19b6c0 tx=0x7f67a4005d90 comp rx=0 tx=0).stop 2024-04-17T07:11:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 -- 172.21.15.86:0/4252959435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67bc101e00 msgr2=0x7f67bc19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67bc101e00 0x7f67bc19a1b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f67ac033000 tx=0x7f67ac035d60 comp rx=0 tx=0).stop 2024-04-17T07:11:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 -- 172.21.15.86:0/4252959435 shutdown_connections 2024-04-17T07:11:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f67a00a3bb0 0x7f67a00a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67bc1039b0 0x7f67bc1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67bc102ff0 0x7f67bc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 --2- 172.21.15.86:0/4252959435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67bc101e00 0x7f67bc19a1b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 -- 172.21.15.86:0/4252959435 >> 172.21.15.86:0/4252959435 conn(0x7f67bc0fd370 msgr2=0x7f67bc074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:10.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 -- 172.21.15.86:0/4252959435 shutdown_connections 2024-04-17T07:11:10.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:10.539+0000 7f67c1200640 1 -- 172.21.15.86:0/4252959435 wait complete. 2024-04-17T07:11:10.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:10 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4252959435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:11.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:10 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4252959435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:11.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:10 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4252959435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:11.742 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:11 smithi103 ceph-mon[35495]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:11.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:11 smithi086 ceph-mon[31733]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:12.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:11 smithi098 ceph-mon[35418]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:12.196 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:11:11 smithi086 podman[69695]: 2024-04-17 07:11:11.909516192 +0000 UTC m=+0.149123850 container create 97cd7ae16a23e8cebac3a9faa576bff966348322c40061fbc4c5c9e8aa6db61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-j, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-17T07:11:12.196 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:11:12 smithi086 podman[69695]: 2024-04-17 07:11:12.196522101 +0000 UTC m=+0.436129784 container init 97cd7ae16a23e8cebac3a9faa576bff966348322c40061fbc4c5c9e8aa6db61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-17T07:11:12.558 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:12.580 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:11:12 smithi086 podman[69695]: 2024-04-17 07:11:12.207172516 +0000 UTC m=+0.446780182 container start 97cd7ae16a23e8cebac3a9faa576bff966348322c40061fbc4c5c9e8aa6db61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-j, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:11:12.580 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:11:12 smithi086 ceph-mds[69714]: -- 172.21.15.86:0/988835767 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cfc9923a40 con 0x55cfc98f7000 2024-04-17T07:11:12.580 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:11:12 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-j[69710]: starting mds.j at 2024-04-17T07:11:12.580 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:11:12 smithi086 bash[69695]: 97cd7ae16a23e8cebac3a9faa576bff966348322c40061fbc4c5c9e8aa6db61e 2024-04-17T07:11:12.580 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:11:12 smithi086 systemd[1]: Started Ceph mds.j for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.763+0000 7f3134a00640 1 -- 172.21.15.86:0/3574455911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130076920 msgr2=0x7f3130076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.763+0000 7f3134a00640 1 --2- 172.21.15.86:0/3574455911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130076920 0x7f3130076d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3120009900 tx=0x7f312002bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.763+0000 7f3134a00640 1 -- 172.21.15.86:0/3574455911 shutdown_connections 2024-04-17T07:11:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.763+0000 7f3134a00640 1 --2- 172.21.15.86:0/3574455911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3130078500 0x7f3130078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.763+0000 7f3134a00640 1 --2- 172.21.15.86:0/3574455911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3130077b60 0x7f3130077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.763+0000 7f3134a00640 1 --2- 172.21.15.86:0/3574455911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130076920 0x7f3130076d40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.763+0000 7f3134a00640 1 -- 172.21.15.86:0/3574455911 >> 172.21.15.86:0/3574455911 conn(0x7f31300ff750 msgr2=0x7f3130101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.764+0000 7f3134a00640 1 -- 172.21.15.86:0/3574455911 shutdown_connections 2024-04-17T07:11:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.764+0000 7f3134a00640 1 -- 172.21.15.86:0/3574455911 wait complete. 2024-04-17T07:11:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.764+0000 7f3134a00640 1 Processor -- start 2024-04-17T07:11:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f3134a00640 1 -- start start 2024-04-17T07:11:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f3134a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130076920 0x7f313019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f3134a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3130077b60 0x7f313019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f3134a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3130078500 0x7f31301a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f3134a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3130114140 con 0x7f3130076920 2024-04-17T07:11:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f3134a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3130113fc0 con 0x7f3130077b60 2024-04-17T07:11:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f3134a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31301142c0 con 0x7f3130078500 2024-04-17T07:11:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f312de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130076920 0x7f313019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f312de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130076920 0x7f313019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36264/0 (socket says 172.21.15.86:36264) 2024-04-17T07:11:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f312de00640 1 -- 172.21.15.86:0/2702078680 learned_addr learned my addr 172.21.15.86:0/2702078680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.765+0000 7f312e800640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3130078500 0x7f31301a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f312d400640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3130077b60 0x7f313019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f312de00640 1 -- 172.21.15.86:0/2702078680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3130078500 msgr2=0x7f31301a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f312de00640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3130078500 0x7f31301a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f312de00640 1 -- 172.21.15.86:0/2702078680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3130077b60 msgr2=0x7f313019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f312de00640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3130077b60 0x7f313019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f312de00640 1 -- 172.21.15.86:0/2702078680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31301a6400 con 0x7f3130076920 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f312d400640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3130077b60 0x7f313019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f312de00640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130076920 0x7f313019e7e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3120000c00 tx=0x7f31200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f3126a00640 1 -- 172.21.15.86:0/2702078680 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120043020 con 0x7f3130076920 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f3126a00640 1 -- 172.21.15.86:0/2702078680 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31200054c0 con 0x7f3130076920 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f3126a00640 1 -- 172.21.15.86:0/2702078680 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120003d40 con 0x7f3130076920 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.766+0000 7f3134a00640 1 -- 172.21.15.86:0/2702078680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31301a6690 con 0x7f3130076920 2024-04-17T07:11:12.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.767+0000 7f3134a00640 1 -- 172.21.15.86:0/2702078680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31301a6b90 con 0x7f3130076920 2024-04-17T07:11:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.768+0000 7f3134a00640 1 -- 172.21.15.86:0/2702078680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3130110d60 con 0x7f3130076920 2024-04-17T07:11:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.770+0000 7f3126a00640 1 -- 172.21.15.86:0/2702078680 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3120046020 con 0x7f3130076920 2024-04-17T07:11:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.771+0000 7f3126a00640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31140a3c30 0x7f31140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.771+0000 7f3126a00640 1 -- 172.21.15.86:0/2702078680 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f31200f2b10 con 0x7f3130076920 2024-04-17T07:11:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.771+0000 7f312d400640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31140a3c30 0x7f31140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.772+0000 7f312d400640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31140a3c30 0x7f31140a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f313019fcf0 tx=0x7f311c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:12.774+0000 7f3126a00640 1 -- 172.21.15.86:0/2702078680 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f312004d020 con 0x7f3130076920 2024-04-17T07:11:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.027+0000 7f3134a00640 1 -- 172.21.15.86:0/2702078680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f313002d5d0 con 0x7f3130076920 2024-04-17T07:11:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.028+0000 7f3126a00640 1 -- 172.21.15.86:0/2702078680 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7f31200bfa10 con 0x7f3130076920 2024-04-17T07:11:13.027 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:13.027 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T07:11:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 4 2024-04-17T07:11:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 -- 172.21.15.86:0/2702078680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31140a3c30 msgr2=0x7f31140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31140a3c30 0x7f31140a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f313019fcf0 tx=0x7f311c005e50 comp rx=0 tx=0).stop 2024-04-17T07:11:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 -- 172.21.15.86:0/2702078680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130076920 msgr2=0x7f313019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130076920 0x7f313019e7e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3120000c00 tx=0x7f31200042f0 comp rx=0 tx=0).stop 2024-04-17T07:11:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 -- 172.21.15.86:0/2702078680 shutdown_connections 2024-04-17T07:11:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31140a3c30 0x7f31140a60f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3130078500 0x7f31301a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3130077b60 0x7f313019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 --2- 172.21.15.86:0/2702078680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3130076920 0x7f313019e7e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 -- 172.21.15.86:0/2702078680 >> 172.21.15.86:0/2702078680 conn(0x7f31300ff750 msgr2=0x7f3130100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.031+0000 7f3134a00640 1 -- 172.21.15.86:0/2702078680 shutdown_connections 2024-04-17T07:11:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:13.032+0000 7f3134a00640 1 -- 172.21.15.86:0/2702078680 wait complete. 2024-04-17T07:11:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: Deploying daemon mds.d on smithi086 2024-04-17T07:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.86:6834/22224874,v1:172.21.15.86:6835/22224874] up:boot 2024-04-17T07:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: fsmap 3 up:standby 2024-04-17T07:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T07:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2702078680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:13 smithi098 ceph-mon[35418]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:13.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: Deploying daemon mds.d on smithi086 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.86:6834/22224874,v1:172.21.15.86:6835/22224874] up:boot 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: fsmap 3 up:standby 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2702078680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:13 smithi103 ceph-mon[35495]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:13.817 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:13.817 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:13.817 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:13.817 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:13.817 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:13.817 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:13.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: Deploying daemon mds.d on smithi086 2024-04-17T07:11:13.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.86:6834/22224874,v1:172.21.15.86:6835/22224874] up:boot 2024-04-17T07:11:13.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: fsmap 3 up:standby 2024-04-17T07:11:13.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-17T07:11:13.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2702078680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:13.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:13 smithi086 ceph-mon[31733]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:15.052 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.442+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2965705715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e5410e2a0 msgr2=0x7f6e5410e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.442+0000 7f6e5ac00640 1 --2- 172.21.15.86:0/2965705715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e5410e2a0 0x7f6e5410e750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009ca0 tx=0x7f6e4c02c100 comp rx=0 tx=0).stop 2024-04-17T07:11:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.442+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2965705715 shutdown_connections 2024-04-17T07:11:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.442+0000 7f6e5ac00640 1 --2- 172.21.15.86:0/2965705715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e5410e2a0 0x7f6e5410e750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.442+0000 7f6e5ac00640 1 --2- 172.21.15.86:0/2965705715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e5410b8d0 0x7f6e5410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.442+0000 7f6e5ac00640 1 --2- 172.21.15.86:0/2965705715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e54108f60 0x7f6e5410b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.442+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2965705715 >> 172.21.15.86:0/2965705715 conn(0x7f6e5406ee00 msgr2=0x7f6e54071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:15.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.442+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2965705715 shutdown_connections 2024-04-17T07:11:15.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.442+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2965705715 wait complete. 2024-04-17T07:11:15.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.443+0000 7f6e5ac00640 1 Processor -- start 2024-04-17T07:11:15.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.443+0000 7f6e5ac00640 1 -- start start 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.443+0000 7f6e5ac00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e54108f60 0x7f6e541a7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.443+0000 7f6e5ac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e5410b8d0 0x7f6e541a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e5ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e5410e2a0 0x7f6e541ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e5ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5411c960 con 0x7f6e5410e2a0 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e5ac00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5411c7e0 con 0x7f6e54108f60 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e5ac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5411cae0 con 0x7f6e5410b8d0 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e58c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e5410e2a0 0x7f6e541ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e58c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e5410e2a0 0x7f6e541ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36286/0 (socket says 172.21.15.86:36286) 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e58c00640 1 -- 172.21.15.86:0/2782720265 learned_addr learned my addr 172.21.15.86:0/2782720265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e53e00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e54108f60 0x7f6e541a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e53400640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e5410b8d0 0x7f6e541a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e58c00640 1 -- 172.21.15.86:0/2782720265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e5410b8d0 msgr2=0x7f6e541a75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e58c00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e5410b8d0 0x7f6e541a75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e58c00640 1 -- 172.21.15.86:0/2782720265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e54108f60 msgr2=0x7f6e541a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e58c00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e54108f60 0x7f6e541a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.444+0000 7f6e58c00640 1 -- 172.21.15.86:0/2782720265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e541aec80 con 0x7f6e5410e2a0 2024-04-17T07:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.445+0000 7f6e58c00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e5410e2a0 0x7f6e541ae610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c02c710 tx=0x7f6e4c035dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.445+0000 7f6e50c00640 1 -- 172.21.15.86:0/2782720265 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c039040 con 0x7f6e5410e2a0 2024-04-17T07:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.445+0000 7f6e50c00640 1 -- 172.21.15.86:0/2782720265 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e4c044070 con 0x7f6e5410e2a0 2024-04-17T07:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.445+0000 7f6e50c00640 1 -- 172.21.15.86:0/2782720265 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c004670 con 0x7f6e5410e2a0 2024-04-17T07:11:15.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.446+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2782720265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e541aef10 con 0x7f6e5410e2a0 2024-04-17T07:11:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.446+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2782720265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e541af3a0 con 0x7f6e5410e2a0 2024-04-17T07:11:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.447+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2782720265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e54071900 con 0x7f6e5410e2a0 2024-04-17T07:11:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.450+0000 7f6e50c00640 1 -- 172.21.15.86:0/2782720265 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e4c003970 con 0x7f6e5410e2a0 2024-04-17T07:11:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.451+0000 7f6e50c00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c30 0x7f6e380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.451+0000 7f6e50c00640 1 -- 172.21.15.86:0/2782720265 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6e4c0f29a0 con 0x7f6e5410e2a0 2024-04-17T07:11:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.451+0000 7f6e53e00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c30 0x7f6e380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.452+0000 7f6e53e00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c30 0x7f6e380a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e400058c0 tx=0x7f6e4000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:15.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.454+0000 7f6e50c00640 1 -- 172.21.15.86:0/2782720265 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6e4c04d020 con 0x7f6e5410e2a0 2024-04-17T07:11:15.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.703+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2782720265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e5402d5d0 con 0x7f6e5410e2a0 2024-04-17T07:11:15.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.704+0000 7f6e50c00640 1 -- 172.21.15.86:0/2782720265 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7f6e4c0bf780 con 0x7f6e5410e2a0 2024-04-17T07:11:15.703 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:15.703 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T07:11:15.704 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 4 2024-04-17T07:11:15.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2782720265 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c30 msgr2=0x7f6e380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:15.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c30 0x7f6e380a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e400058c0 tx=0x7f6e4000a4c0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2782720265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e5410e2a0 msgr2=0x7f6e541ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:15.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e5410e2a0 0x7f6e541ae610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c02c710 tx=0x7f6e4c035dd0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2782720265 shutdown_connections 2024-04-17T07:11:15.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c30 0x7f6e380a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e5410e2a0 0x7f6e541ae610 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e5410b8d0 0x7f6e541a75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 --2- 172.21.15.86:0/2782720265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e54108f60 0x7f6e541a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:15.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2782720265 >> 172.21.15.86:0/2782720265 conn(0x7f6e5406ee00 msgr2=0x7f6e5410cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:15.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2782720265 shutdown_connections 2024-04-17T07:11:15.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:15.707+0000 7f6e5ac00640 1 -- 172.21.15.86:0/2782720265 wait complete. 2024-04-17T07:11:16.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:16 smithi086 ceph-mon[31733]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:16.269 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:16 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2782720265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:16.459 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:16 smithi098 ceph-mon[35418]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:16.459 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:16 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2782720265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:16 smithi103 ceph-mon[35495]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:16 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2782720265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:16.830 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 17 07:11:16 smithi086 systemd[1]: Started Ceph mds.d for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:17.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: Deploying daemon mds.e on smithi098 2024-04-17T07:11:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:17.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.86:6836/267440661,v1:172.21.15.86:6837/267440661] up:boot 2024-04-17T07:11:17.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: fsmap 4 up:standby 2024-04-17T07:11:17.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T07:11:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: Deploying daemon mds.e on smithi098 2024-04-17T07:11:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.86:6836/267440661,v1:172.21.15.86:6837/267440661] up:boot 2024-04-17T07:11:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: fsmap 4 up:standby 2024-04-17T07:11:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T07:11:17.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.953+0000 7f682a400640 1 -- 172.21.15.86:0/3091649178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68241051d0 msgr2=0x7f6824105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:17.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.953+0000 7f682a400640 1 --2- 172.21.15.86:0/3091649178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68241051d0 0x7f6824105650 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f680c009900 tx=0x7f680c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:17.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.953+0000 7f682a400640 1 -- 172.21.15.86:0/3091649178 shutdown_connections 2024-04-17T07:11:17.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.953+0000 7f682a400640 1 --2- 172.21.15.86:0/3091649178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6824105b90 0x7f682410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:17.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.953+0000 7f682a400640 1 --2- 172.21.15.86:0/3091649178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68241051d0 0x7f6824105650 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:17.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.953+0000 7f682a400640 1 --2- 172.21.15.86:0/3091649178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6824103fe0 0x7f6824104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:17.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.953+0000 7f682a400640 1 -- 172.21.15.86:0/3091649178 >> 172.21.15.86:0/3091649178 conn(0x7f68240ff570 msgr2=0x7f6824101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:17.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.953+0000 7f682a400640 1 -- 172.21.15.86:0/3091649178 shutdown_connections 2024-04-17T07:11:17.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.953+0000 7f682a400640 1 -- 172.21.15.86:0/3091649178 wait complete. 2024-04-17T07:11:17.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.954+0000 7f682a400640 1 Processor -- start 2024-04-17T07:11:17.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.954+0000 7f682a400640 1 -- start start 2024-04-17T07:11:17.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f682a400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6824103fe0 0x7f682419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:17.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f682a400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68241051d0 0x7f682419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:17.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f682a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6824105b90 0x7f68241a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:17.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f682a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f682410fb70 con 0x7f6824105b90 2024-04-17T07:11:17.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f682a400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f682410f9f0 con 0x7f68241051d0 2024-04-17T07:11:17.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f682a400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f682410fcf0 con 0x7f6824103fe0 2024-04-17T07:11:17.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f6829a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6824105b90 0x7f68241a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f6829a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6824105b90 0x7f68241a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33066/0 (socket says 172.21.15.86:33066) 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f6829a00640 1 -- 172.21.15.86:0/3652110359 learned_addr learned my addr 172.21.15.86:0/3652110359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f681be00640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68241051d0 0x7f682419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f6829000640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6824103fe0 0x7f682419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f6829a00640 1 -- 172.21.15.86:0/3652110359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6824103fe0 msgr2=0x7f682419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f6829a00640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6824103fe0 0x7f682419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f6829a00640 1 -- 172.21.15.86:0/3652110359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68241051d0 msgr2=0x7f682419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.955+0000 7f6829a00640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68241051d0 0x7f682419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.956+0000 7f6829a00640 1 -- 172.21.15.86:0/3652110359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68241a1df0 con 0x7f6824105b90 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.956+0000 7f6829a00640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6824105b90 0x7f68241a1780 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f681c00eaa0 tx=0x7f681c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.956+0000 7f6822000640 1 -- 172.21.15.86:0/3652110359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c00cdb0 con 0x7f6824105b90 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.956+0000 7f6822000640 1 -- 172.21.15.86:0/3652110359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f681c005020 con 0x7f6824105b90 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.956+0000 7f682a400640 1 -- 172.21.15.86:0/3652110359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68241a20e0 con 0x7f6824105b90 2024-04-17T07:11:17.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.956+0000 7f6822000640 1 -- 172.21.15.86:0/3652110359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c0116d0 con 0x7f6824105b90 2024-04-17T07:11:17.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.956+0000 7f682a400640 1 -- 172.21.15.86:0/3652110359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68241a25b0 con 0x7f6824105b90 2024-04-17T07:11:17.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.958+0000 7f682a400640 1 -- 172.21.15.86:0/3652110359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f682410b380 con 0x7f6824105b90 2024-04-17T07:11:17.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.959+0000 7f6822000640 1 -- 172.21.15.86:0/3652110359 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f681c0051d0 con 0x7f6824105b90 2024-04-17T07:11:17.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.961+0000 7f6822000640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68080a3bb0 0x7f68080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:17.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.961+0000 7f6822000640 1 -- 172.21.15.86:0/3652110359 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f681c0d01f0 con 0x7f6824105b90 2024-04-17T07:11:17.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.961+0000 7f6829000640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68080a3bb0 0x7f68080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:17.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.962+0000 7f6829000640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68080a3bb0 0x7f68080a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6814005fd0 tx=0x7f6814005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:17.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:17.966+0000 7f6822000640 1 -- 172.21.15.86:0/3652110359 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f681c016030 con 0x7f6824105b90 2024-04-17T07:11:18.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:18.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:18.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:18.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:18.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:18.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:18.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: Deploying daemon mds.e on smithi098 2024-04-17T07:11:18.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:18.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.86:6836/267440661,v1:172.21.15.86:6837/267440661] up:boot 2024-04-17T07:11:18.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: fsmap 4 up:standby 2024-04-17T07:11:18.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-17T07:11:18.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.211+0000 7f682a400640 1 -- 172.21.15.86:0/3652110359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f682402d5d0 con 0x7f6824105b90 2024-04-17T07:11:18.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.212+0000 7f6822000640 1 -- 172.21.15.86:0/3652110359 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3862 (secure 0 0 0) 0x7f681c09cfd0 con 0x7f6824105b90 2024-04-17T07:11:18.211 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T07:11:18.211 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 5 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.214+0000 7f682a400640 1 -- 172.21.15.86:0/3652110359 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68080a3bb0 msgr2=0x7f68080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.214+0000 7f682a400640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68080a3bb0 0x7f68080a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6814005fd0 tx=0x7f6814005850 comp rx=0 tx=0).stop 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.214+0000 7f682a400640 1 -- 172.21.15.86:0/3652110359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6824105b90 msgr2=0x7f68241a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.214+0000 7f682a400640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6824105b90 0x7f68241a1780 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f681c00eaa0 tx=0x7f681c00ef60 comp rx=0 tx=0).stop 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.214+0000 7f682a400640 1 -- 172.21.15.86:0/3652110359 shutdown_connections 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.214+0000 7f682a400640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68080a3bb0 0x7f68080a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.214+0000 7f682a400640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6824105b90 0x7f68241a1780 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.214+0000 7f682a400640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68241051d0 0x7f682419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.214+0000 7f682a400640 1 --2- 172.21.15.86:0/3652110359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6824103fe0 0x7f682419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.214+0000 7f682a400640 1 -- 172.21.15.86:0/3652110359 >> 172.21.15.86:0/3652110359 conn(0x7f68240ff570 msgr2=0x7f6824108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.215+0000 7f682a400640 1 -- 172.21.15.86:0/3652110359 shutdown_connections 2024-04-17T07:11:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:18.215+0000 7f682a400640 1 -- 172.21.15.86:0/3652110359 wait complete. 2024-04-17T07:11:18.929 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:18 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3652110359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:19.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:18 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3652110359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:19.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:18 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3652110359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:19.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:19 smithi098 ceph-mon[35418]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:20.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:19 smithi103 ceph-mon[35495]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:20.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:19 smithi086 ceph-mon[31733]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:20.124 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:11:19 smithi098 systemd[1]: Starting Ceph mds.e for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:11:20.234 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:20.378 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:11:20 smithi098 podman[65535]: 2024-04-17 07:11:20.12353885 +0000 UTC m=+0.127977841 container create 19cc2df441fd76c09b0450f17eac474a11fc52d8f1a87fd5a4c3452dd6ce529d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-04-17T07:11:20.378 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:11:20 smithi098 podman[65535]: 2024-04-17 07:11:20.030143476 +0000 UTC m=+0.034582417 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:11:20.378 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:11:20 smithi098 podman[65535]: 2024-04-17 07:11:20.377669057 +0000 UTC m=+0.382108066 container init 19cc2df441fd76c09b0450f17eac474a11fc52d8f1a87fd5a4c3452dd6ce529d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:11:20.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.456+0000 7fb1f4e00640 1 -- 172.21.15.86:0/1358149603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f0077b60 msgr2=0x7fb1f0077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:20.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.456+0000 7fb1f4e00640 1 --2- 172.21.15.86:0/1358149603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f0077b60 0x7fb1f0077fc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009900 tx=0x7fb1d802bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:20.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.456+0000 7fb1f4e00640 1 -- 172.21.15.86:0/1358149603 shutdown_connections 2024-04-17T07:11:20.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.456+0000 7fb1f4e00640 1 --2- 172.21.15.86:0/1358149603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1f0078500 0x7fb1f0078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.456+0000 7fb1f4e00640 1 --2- 172.21.15.86:0/1358149603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f0077b60 0x7fb1f0077fc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.456+0000 7fb1f4e00640 1 --2- 172.21.15.86:0/1358149603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f0076920 0x7fb1f0076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.456+0000 7fb1f4e00640 1 -- 172.21.15.86:0/1358149603 >> 172.21.15.86:0/1358149603 conn(0x7fb1f00ff570 msgr2=0x7fb1f0101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.456+0000 7fb1f4e00640 1 -- 172.21.15.86:0/1358149603 shutdown_connections 2024-04-17T07:11:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.456+0000 7fb1f4e00640 1 -- 172.21.15.86:0/1358149603 wait complete. 2024-04-17T07:11:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.457+0000 7fb1f4e00640 1 Processor -- start 2024-04-17T07:11:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.457+0000 7fb1f4e00640 1 -- start start 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.457+0000 7fb1f4e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f0076920 0x7fb1f019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.457+0000 7fb1f4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1f0077b60 0x7fb1f019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1f4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f0078500 0x7fb1f01a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1f4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0113f20 con 0x7fb1f0078500 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1f4e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0113da0 con 0x7fb1f0076920 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1f4e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01140a0 con 0x7fb1f0077b60 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f0078500 0x7fb1f01a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f0078500 0x7fb1f01a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33092/0 (socket says 172.21.15.86:33092) 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1efe00640 1 -- 172.21.15.86:0/3834110773 learned_addr learned my addr 172.21.15.86:0/3834110773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1ef400640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f0076920 0x7fb1f019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1e6a00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1f0077b60 0x7fb1f019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1efe00640 1 -- 172.21.15.86:0/3834110773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1f0077b60 msgr2=0x7fb1f019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1efe00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1f0077b60 0x7fb1f019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1efe00640 1 -- 172.21.15.86:0/3834110773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f0076920 msgr2=0x7fb1f019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1efe00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f0076920 0x7fb1f019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1efe00640 1 -- 172.21.15.86:0/3834110773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f01a6250 con 0x7fb1f0078500 2024-04-17T07:11:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1e6a00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1f0077b60 0x7fb1f019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.458+0000 7fb1ef400640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f0076920 0x7fb1f019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.459+0000 7fb1efe00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f0078500 0x7fb1f01a5be0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800eaa0 tx=0x7fb1e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.459+0000 7fb1ecc00640 1 -- 172.21.15.86:0/3834110773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e800cdb0 con 0x7fb1f0078500 2024-04-17T07:11:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.459+0000 7fb1ecc00640 1 -- 172.21.15.86:0/3834110773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1e8005020 con 0x7fb1f0078500 2024-04-17T07:11:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.459+0000 7fb1f4e00640 1 -- 172.21.15.86:0/3834110773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f01a6540 con 0x7fb1f0078500 2024-04-17T07:11:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.459+0000 7fb1ecc00640 1 -- 172.21.15.86:0/3834110773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e80116d0 con 0x7fb1f0078500 2024-04-17T07:11:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.459+0000 7fb1f4e00640 1 -- 172.21.15.86:0/3834110773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01a6a40 con 0x7fb1f0078500 2024-04-17T07:11:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.460+0000 7fb1f4e00640 1 -- 172.21.15.86:0/3834110773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f0110bd0 con 0x7fb1f0078500 2024-04-17T07:11:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.462+0000 7fb1ecc00640 1 -- 172.21.15.86:0/3834110773 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb1e80051d0 con 0x7fb1f0078500 2024-04-17T07:11:20.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.464+0000 7fb1ecc00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3bb0 0x7fb1d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:20.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.464+0000 7fb1ecc00640 1 -- 172.21.15.86:0/3834110773 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb1e80d02b0 con 0x7fb1f0078500 2024-04-17T07:11:20.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.464+0000 7fb1ef400640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3bb0 0x7fb1d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:20.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.465+0000 7fb1ef400640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3bb0 0x7fb1d00a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009b50 tx=0x7fb1e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:20.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.469+0000 7fb1ecc00640 1 -- 172.21.15.86:0/3834110773 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb1e8016030 con 0x7fb1f0078500 2024-04-17T07:11:20.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.717+0000 7fb1f4e00640 1 -- 172.21.15.86:0/3834110773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1f0062b60 con 0x7fb1f0078500 2024-04-17T07:11:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.719+0000 7fb1ecc00640 1 -- 172.21.15.86:0/3834110773 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4684 (secure 0 0 0) 0x7fb1e809d090 con 0x7fb1f0078500 2024-04-17T07:11:20.719 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:20.719 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-17T07:11:20.719 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 6 2024-04-17T07:11:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.721+0000 7fb1f4e00640 1 -- 172.21.15.86:0/3834110773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3bb0 msgr2=0x7fb1d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.721+0000 7fb1f4e00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3bb0 0x7fb1d00a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009b50 tx=0x7fb1e0005f90 comp rx=0 tx=0).stop 2024-04-17T07:11:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.721+0000 7fb1f4e00640 1 -- 172.21.15.86:0/3834110773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f0078500 msgr2=0x7fb1f01a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:20.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.721+0000 7fb1f4e00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f0078500 0x7fb1f01a5be0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800eaa0 tx=0x7fb1e800ef60 comp rx=0 tx=0).stop 2024-04-17T07:11:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.722+0000 7fb1f4e00640 1 -- 172.21.15.86:0/3834110773 shutdown_connections 2024-04-17T07:11:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.722+0000 7fb1f4e00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3bb0 0x7fb1d00a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.722+0000 7fb1f4e00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1f0078500 0x7fb1f01a5be0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.722+0000 7fb1f4e00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1f0077b60 0x7fb1f019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.722+0000 7fb1f4e00640 1 --2- 172.21.15.86:0/3834110773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1f0076920 0x7fb1f019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:20.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.722+0000 7fb1f4e00640 1 -- 172.21.15.86:0/3834110773 >> 172.21.15.86:0/3834110773 conn(0x7fb1f00ff570 msgr2=0x7fb1f0100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.722+0000 7fb1f4e00640 1 -- 172.21.15.86:0/3834110773 shutdown_connections 2024-04-17T07:11:20.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:20.722+0000 7fb1f4e00640 1 -- 172.21.15.86:0/3834110773 wait complete. 2024-04-17T07:11:20.753 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:11:20 smithi098 podman[65535]: 2024-04-17 07:11:20.387680369 +0000 UTC m=+0.392119341 container start 19cc2df441fd76c09b0450f17eac474a11fc52d8f1a87fd5a4c3452dd6ce529d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-17T07:11:20.753 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:11:20 smithi098 ceph-mds[65554]: -- 172.21.15.98:0/296743361 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db576f9a40 con 0x55db576cd000 2024-04-17T07:11:20.753 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:11:20 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-e[65550]: starting mds.e at 2024-04-17T07:11:20.754 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:11:20 smithi098 bash[65535]: 19cc2df441fd76c09b0450f17eac474a11fc52d8f1a87fd5a4c3452dd6ce529d 2024-04-17T07:11:20.754 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:11:20 smithi098 systemd[1]: Started Ceph mds.e for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:21.896 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:21.896 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] up:boot 2024-04-17T07:11:21.896 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: fsmap 5 up:standby 2024-04-17T07:11:21.896 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T07:11:21.896 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:21.896 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:21.896 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:21.896 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:21.896 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3834110773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:21.897 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:21.897 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: Deploying daemon mds.f on smithi103 2024-04-17T07:11:21.897 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:21 smithi103 ceph-mon[35495]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] up:boot 2024-04-17T07:11:22.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: fsmap 5 up:standby 2024-04-17T07:11:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T07:11:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3834110773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:22.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: Deploying daemon mds.f on smithi103 2024-04-17T07:11:22.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:21 smithi098 ceph-mon[35418]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:22.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:22.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] up:boot 2024-04-17T07:11:22.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: fsmap 5 up:standby 2024-04-17T07:11:22.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-17T07:11:22.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:22.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:22.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:22.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:22.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3834110773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:22.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:22.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: Deploying daemon mds.f on smithi103 2024-04-17T07:11:22.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:21 smithi086 ceph-mon[31733]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:22.743 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.963+0000 7ff69d200640 1 -- 172.21.15.86:0/2453547605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6981051d0 msgr2=0x7ff698105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.963+0000 7ff69d200640 1 --2- 172.21.15.86:0/2453547605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6981051d0 0x7ff698105650 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff680009900 tx=0x7ff68002bc50 comp rx=0 tx=0).stop 2024-04-17T07:11:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.964+0000 7ff69d200640 1 -- 172.21.15.86:0/2453547605 shutdown_connections 2024-04-17T07:11:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.964+0000 7ff69d200640 1 --2- 172.21.15.86:0/2453547605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff698105b90 0x7ff69810c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.964+0000 7ff69d200640 1 --2- 172.21.15.86:0/2453547605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6981051d0 0x7ff698105650 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.964+0000 7ff69d200640 1 --2- 172.21.15.86:0/2453547605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff698103fe0 0x7ff698104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.964+0000 7ff69d200640 1 -- 172.21.15.86:0/2453547605 >> 172.21.15.86:0/2453547605 conn(0x7ff6980ff570 msgr2=0x7ff698101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.964+0000 7ff69d200640 1 -- 172.21.15.86:0/2453547605 shutdown_connections 2024-04-17T07:11:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.964+0000 7ff69d200640 1 -- 172.21.15.86:0/2453547605 wait complete. 2024-04-17T07:11:22.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff69d200640 1 Processor -- start 2024-04-17T07:11:22.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff69d200640 1 -- start start 2024-04-17T07:11:22.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff69d200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff698103fe0 0x7ff69819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:22.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff69d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6981051d0 0x7ff69819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:22.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff69d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff698105b90 0x7ff6981a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:22.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff69d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff69810fb70 con 0x7ff698105b90 2024-04-17T07:11:22.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff69d200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff69810f9f0 con 0x7ff698103fe0 2024-04-17T07:11:22.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff69d200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff69810fcf0 con 0x7ff6981051d0 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff697e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff698105b90 0x7ff6981a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff697e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff698105b90 0x7ff6981a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33116/0 (socket says 172.21.15.86:33116) 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.965+0000 7ff697e00640 1 -- 172.21.15.86:0/87668653 learned_addr learned my addr 172.21.15.86:0/87668653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff697400640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff698103fe0 0x7ff69819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff696a00640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6981051d0 0x7ff69819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff697e00640 1 -- 172.21.15.86:0/87668653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6981051d0 msgr2=0x7ff69819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff697e00640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6981051d0 0x7ff69819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff697e00640 1 -- 172.21.15.86:0/87668653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff698103fe0 msgr2=0x7ff69819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff697e00640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff698103fe0 0x7ff69819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff697e00640 1 -- 172.21.15.86:0/87668653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6981a1de0 con 0x7ff698105b90 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff696a00640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6981051d0 0x7ff69819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff697e00640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff698105b90 0x7ff6981a1770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff69000cce0 tx=0x7ff69000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff68fe00640 1 -- 172.21.15.86:0/87668653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69000f450 con 0x7ff698105b90 2024-04-17T07:11:22.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff69d200640 1 -- 172.21.15.86:0/87668653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6981a20d0 con 0x7ff698105b90 2024-04-17T07:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff68fe00640 1 -- 172.21.15.86:0/87668653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff69000faa0 con 0x7ff698105b90 2024-04-17T07:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff68fe00640 1 -- 172.21.15.86:0/87668653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690018b70 con 0x7ff698105b90 2024-04-17T07:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.966+0000 7ff69d200640 1 -- 172.21.15.86:0/87668653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6981a2620 con 0x7ff698105b90 2024-04-17T07:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.967+0000 7ff69d200640 1 -- 172.21.15.86:0/87668653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69810b300 con 0x7ff698105b90 2024-04-17T07:11:22.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.970+0000 7ff68fe00640 1 -- 172.21.15.86:0/87668653 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff690018d10 con 0x7ff698105b90 2024-04-17T07:11:22.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.971+0000 7ff68fe00640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff67c0a3c30 0x7ff67c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:22.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.971+0000 7ff68fe00640 1 -- 172.21.15.86:0/87668653 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff6900d03c0 con 0x7ff698105b90 2024-04-17T07:11:22.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.971+0000 7ff697400640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff67c0a3c30 0x7ff67c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:22.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.972+0000 7ff697400640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff67c0a3c30 0x7ff67c0a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff688005fd0 tx=0x7ff688005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:22.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:22.976+0000 7ff68fe00640 1 -- 172.21.15.86:0/87668653 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff690014030 con 0x7ff698105b90 2024-04-17T07:11:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.220+0000 7ff69d200640 1 -- 172.21.15.86:0/87668653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff698062b60 con 0x7ff698105b90 2024-04-17T07:11:23.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.221+0000 7ff68fe00640 1 -- 172.21.15.86:0/87668653 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4684 (secure 0 0 0) 0x7ff69009d240 con 0x7ff698105b90 2024-04-17T07:11:23.221 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:23.221 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-17T07:11:23.222 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 6 2024-04-17T07:11:23.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 -- 172.21.15.86:0/87668653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff67c0a3c30 msgr2=0x7ff67c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:23.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff67c0a3c30 0x7ff67c0a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff688005fd0 tx=0x7ff688005850 comp rx=0 tx=0).stop 2024-04-17T07:11:23.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 -- 172.21.15.86:0/87668653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff698105b90 msgr2=0x7ff6981a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:23.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff698105b90 0x7ff6981a1770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff69000cce0 tx=0x7ff69000a620 comp rx=0 tx=0).stop 2024-04-17T07:11:23.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 -- 172.21.15.86:0/87668653 shutdown_connections 2024-04-17T07:11:23.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff67c0a3c30 0x7ff67c0a60f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:23.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff698105b90 0x7ff6981a1770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:23.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6981051d0 0x7ff69819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:23.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 --2- 172.21.15.86:0/87668653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff698103fe0 0x7ff69819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:23.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 -- 172.21.15.86:0/87668653 >> 172.21.15.86:0/87668653 conn(0x7ff6980ff570 msgr2=0x7ff6981084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:23.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 -- 172.21.15.86:0/87668653 shutdown_connections 2024-04-17T07:11:23.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:23.224+0000 7ff69d200640 1 -- 172.21.15.86:0/87668653 wait complete. 2024-04-17T07:11:24.026 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:11:23 smithi103 systemd[1]: Starting Ceph mds.f for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:11:24.355 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:11:24 smithi103 podman[66458]: 2024-04-17 07:11:24.026307464 +0000 UTC m=+0.038520472 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:11:24.356 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:11:24 smithi103 podman[66458]: 2024-04-17 07:11:24.142020429 +0000 UTC m=+0.154233456 container create 7b5269347c63d0b52bc31fdb37d33f096b594db53eb6ec4ddb64b0388b84a4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-17T07:11:24.356 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:24 smithi103 ceph-mon[35495]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:24.356 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:24 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/87668653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:24 smithi098 ceph-mon[35418]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:24 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/87668653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:24 smithi086 ceph-mon[31733]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:24 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/87668653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:24.717 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:11:24 smithi103 podman[66458]: 2024-04-17 07:11:24.42931911 +0000 UTC m=+0.441532153 container init 7b5269347c63d0b52bc31fdb37d33f096b594db53eb6ec4ddb64b0388b84a4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-17T07:11:24.718 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:11:24 smithi103 podman[66458]: 2024-04-17 07:11:24.438953036 +0000 UTC m=+0.451166052 container start 7b5269347c63d0b52bc31fdb37d33f096b594db53eb6ec4ddb64b0388b84a4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-17T07:11:24.718 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:11:24 smithi103 ceph-mds[66477]: -- 172.21.15.103:0/3507318675 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5630c02f3a40 con 0x5630c02c6800 2024-04-17T07:11:24.718 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:11:24 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-f[66472]: starting mds.f at 2024-04-17T07:11:24.718 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:11:24 smithi103 bash[66458]: 7b5269347c63d0b52bc31fdb37d33f096b594db53eb6ec4ddb64b0388b84a4b0 2024-04-17T07:11:24.718 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:11:24 smithi103 systemd[1]: Started Ceph mds.f for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:25.243 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.467+0000 7ffb6a000640 1 -- 172.21.15.86:0/341783167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb641083e0 msgr2=0x7ffb64108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.467+0000 7ffb6a000640 1 --2- 172.21.15.86:0/341783167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb641083e0 0x7ffb64108800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009900 tx=0x7ffb5402bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.467+0000 7ffb6a000640 1 -- 172.21.15.86:0/341783167 shutdown_connections 2024-04-17T07:11:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.467+0000 7ffb6a000640 1 --2- 172.21.15.86:0/341783167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb64109f90 0x7ffb64110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.467+0000 7ffb6a000640 1 --2- 172.21.15.86:0/341783167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb641095d0 0x7ffb64109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.467+0000 7ffb6a000640 1 --2- 172.21.15.86:0/341783167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb641083e0 0x7ffb64108800 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.467+0000 7ffb6a000640 1 -- 172.21.15.86:0/341783167 >> 172.21.15.86:0/341783167 conn(0x7ffb640773a0 msgr2=0x7ffb640797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:25.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.467+0000 7ffb6a000640 1 -- 172.21.15.86:0/341783167 shutdown_connections 2024-04-17T07:11:25.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.467+0000 7ffb6a000640 1 -- 172.21.15.86:0/341783167 wait complete. 2024-04-17T07:11:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.468+0000 7ffb6a000640 1 Processor -- start 2024-04-17T07:11:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.468+0000 7ffb6a000640 1 -- start start 2024-04-17T07:11:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.468+0000 7ffb6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb641083e0 0x7ffb6419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.468+0000 7ffb6a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb641095d0 0x7ffb6419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.468+0000 7ffb6a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb64109f90 0x7ffb641a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.468+0000 7ffb6a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64113f70 con 0x7ffb641083e0 2024-04-17T07:11:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.468+0000 7ffb6a000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64113df0 con 0x7ffb641095d0 2024-04-17T07:11:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.468+0000 7ffb6a000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641140f0 con 0x7ffb64109f90 2024-04-17T07:11:25.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.469+0000 7ffb68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb641083e0 0x7ffb6419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:25.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.469+0000 7ffb69600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb64109f90 0x7ffb641a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:25.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.469+0000 7ffb68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb641083e0 0x7ffb6419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33130/0 (socket says 172.21.15.86:33130) 2024-04-17T07:11:25.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.469+0000 7ffb68c00640 1 -- 172.21.15.86:0/2622992257 learned_addr learned my addr 172.21.15.86:0/2622992257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:25.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.469+0000 7ffb5be00640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb641095d0 0x7ffb6419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:25.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.469+0000 7ffb68c00640 1 -- 172.21.15.86:0/2622992257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb64109f90 msgr2=0x7ffb641a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:25.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.469+0000 7ffb68c00640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb64109f90 0x7ffb641a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:25.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.469+0000 7ffb68c00640 1 -- 172.21.15.86:0/2622992257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb641095d0 msgr2=0x7ffb6419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:25.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.469+0000 7ffb68c00640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb641095d0 0x7ffb6419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.469+0000 7ffb68c00640 1 -- 172.21.15.86:0/2622992257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb641a6250 con 0x7ffb641083e0 2024-04-17T07:11:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.470+0000 7ffb68c00640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb641083e0 0x7ffb6419e6d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffb54000c00 tx=0x7ffb540040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.470+0000 7ffb62000640 1 -- 172.21.15.86:0/2622992257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54043020 con 0x7ffb641083e0 2024-04-17T07:11:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.470+0000 7ffb62000640 1 -- 172.21.15.86:0/2622992257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb540044c0 con 0x7ffb641083e0 2024-04-17T07:11:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.470+0000 7ffb62000640 1 -- 172.21.15.86:0/2622992257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54003c90 con 0x7ffb641083e0 2024-04-17T07:11:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.470+0000 7ffb6a000640 1 -- 172.21.15.86:0/2622992257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb641a64e0 con 0x7ffb641083e0 2024-04-17T07:11:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.470+0000 7ffb6a000640 1 -- 172.21.15.86:0/2622992257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb641a69e0 con 0x7ffb641083e0 2024-04-17T07:11:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.471+0000 7ffb6a000640 1 -- 172.21.15.86:0/2622992257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6410f750 con 0x7ffb641083e0 2024-04-17T07:11:25.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.473+0000 7ffb62000640 1 -- 172.21.15.86:0/2622992257 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffb54046020 con 0x7ffb641083e0 2024-04-17T07:11:25.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.474+0000 7ffb62000640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb480a3c30 0x7ffb480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:25.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.475+0000 7ffb62000640 1 -- 172.21.15.86:0/2622992257 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ffb540f21e0 con 0x7ffb641083e0 2024-04-17T07:11:25.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.475+0000 7ffb5be00640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb480a3c30 0x7ffb480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:25.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.475+0000 7ffb5be00640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb480a3c30 0x7ffb480a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0095f0 tx=0x7ffb4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:25.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.479+0000 7ffb62000640 1 -- 172.21.15.86:0/2622992257 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffb5404d020 con 0x7ffb641083e0 2024-04-17T07:11:25.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.725+0000 7ffb6a000640 1 -- 172.21.15.86:0/2622992257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb6402d5d0 con 0x7ffb641083e0 2024-04-17T07:11:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.726+0000 7ffb62000640 1 -- 172.21.15.86:0/2622992257 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5509 (secure 0 0 0) 0x7ffb540befc0 con 0x7ffb641083e0 2024-04-17T07:11:25.727 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:25.727 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-17T07:11:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-17T07:11:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 -- 172.21.15.86:0/2622992257 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb480a3c30 msgr2=0x7ffb480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb480a3c30 0x7ffb480a60f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0095f0 tx=0x7ffb4c005f90 comp rx=0 tx=0).stop 2024-04-17T07:11:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 -- 172.21.15.86:0/2622992257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb641083e0 msgr2=0x7ffb6419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb641083e0 0x7ffb6419e6d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffb54000c00 tx=0x7ffb540040c0 comp rx=0 tx=0).stop 2024-04-17T07:11:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 -- 172.21.15.86:0/2622992257 shutdown_connections 2024-04-17T07:11:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb480a3c30 0x7ffb480a60f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb64109f90 0x7ffb641a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb641095d0 0x7ffb6419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 --2- 172.21.15.86:0/2622992257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb641083e0 0x7ffb6419e6d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 -- 172.21.15.86:0/2622992257 >> 172.21.15.86:0/2622992257 conn(0x7ffb640773a0 msgr2=0x7ffb6410c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 -- 172.21.15.86:0/2622992257 shutdown_connections 2024-04-17T07:11:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:25.729+0000 7ffb6a000640 1 -- 172.21.15.86:0/2622992257 wait complete. 2024-04-17T07:11:25.955 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:25.955 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:25.955 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:25.955 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:25.955 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:25.955 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:25.955 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: Deploying daemon mds.c on smithi103 2024-04-17T07:11:25.956 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:25.956 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.103:6834/423518277,v1:172.21.15.103:6835/423518277] up:boot 2024-04-17T07:11:25.956 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: fsmap 6 up:standby 2024-04-17T07:11:25.956 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:25 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T07:11:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:26.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: Deploying daemon mds.c on smithi103 2024-04-17T07:11:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.103:6834/423518277,v1:172.21.15.103:6835/423518277] up:boot 2024-04-17T07:11:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: fsmap 6 up:standby 2024-04-17T07:11:26.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:25 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T07:11:26.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:26.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:26.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:26.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:26.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:26.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:26.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: Deploying daemon mds.c on smithi103 2024-04-17T07:11:26.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:26.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.103:6834/423518277,v1:172.21.15.103:6835/423518277] up:boot 2024-04-17T07:11:26.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: fsmap 6 up:standby 2024-04-17T07:11:26.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:25 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-17T07:11:26.897 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:26 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2622992257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:27.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:26 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2622992257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:27.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:26 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2622992257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:27.749 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.972+0000 7f1596400640 1 -- 172.21.15.86:0/82285409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1590109c90 msgr2=0x7f159010c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.972+0000 7f1596400640 1 --2- 172.21.15.86:0/82285409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1590109c90 0x7f159010c120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1588009900 tx=0x7f158802bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.973+0000 7f1596400640 1 -- 172.21.15.86:0/82285409 shutdown_connections 2024-04-17T07:11:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.973+0000 7f1596400640 1 --2- 172.21.15.86:0/82285409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1590109c90 0x7f159010c120 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.973+0000 7f1596400640 1 --2- 172.21.15.86:0/82285409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15900749c0 0x7f1590074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.973+0000 7f1596400640 1 --2- 172.21.15.86:0/82285409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1590075fa0 0x7f15900743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.973+0000 7f1596400640 1 -- 172.21.15.86:0/82285409 >> 172.21.15.86:0/82285409 conn(0x7f15900fd340 msgr2=0x7f15900ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.973+0000 7f1596400640 1 -- 172.21.15.86:0/82285409 shutdown_connections 2024-04-17T07:11:27.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.973+0000 7f1596400640 1 -- 172.21.15.86:0/82285409 wait complete. 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1596400640 1 Processor -- start 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1596400640 1 -- start start 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1596400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15900749c0 0x7f159019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1596400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1590075fa0 0x7f159019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1596400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1590109c90 0x7f15901a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1596400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f159010fac0 con 0x7f15900749c0 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1596400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f159010f940 con 0x7f1590109c90 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1596400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f159010fc40 con 0x7f1590075fa0 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1595000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15900749c0 0x7f159019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1595000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15900749c0 0x7f159019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37584/0 (socket says 172.21.15.86:37584) 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.974+0000 7f1595000640 1 -- 172.21.15.86:0/842603416 learned_addr learned my addr 172.21.15.86:0/842603416 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:27.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f1587e00640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1590075fa0 0x7f159019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f1595a00640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1590109c90 0x7f15901a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f1595000640 1 -- 172.21.15.86:0/842603416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1590075fa0 msgr2=0x7f159019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f1595000640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1590075fa0 0x7f159019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f1595000640 1 -- 172.21.15.86:0/842603416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1590109c90 msgr2=0x7f15901a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f1595000640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1590109c90 0x7f15901a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f1595000640 1 -- 172.21.15.86:0/842603416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15901a1da0 con 0x7f15900749c0 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f1587e00640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1590075fa0 0x7f159019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f1595000640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15900749c0 0x7f159019a180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f158000eab0 tx=0x7f158000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f158e000640 1 -- 172.21.15.86:0/842603416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158000ce40 con 0x7f15900749c0 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f158e000640 1 -- 172.21.15.86:0/842603416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1580011920 con 0x7f15900749c0 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f1596400640 1 -- 172.21.15.86:0/842603416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15901a2090 con 0x7f15900749c0 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.975+0000 7f158e000640 1 -- 172.21.15.86:0/842603416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580010a00 con 0x7f15900749c0 2024-04-17T07:11:27.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.976+0000 7f1596400640 1 -- 172.21.15.86:0/842603416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15901a25e0 con 0x7f15900749c0 2024-04-17T07:11:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.977+0000 7f1596400640 1 -- 172.21.15.86:0/842603416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15901089c0 con 0x7f15900749c0 2024-04-17T07:11:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.978+0000 7f158e000640 1 -- 172.21.15.86:0/842603416 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1580011ad0 con 0x7f15900749c0 2024-04-17T07:11:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.980+0000 7f158e000640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f15740a3bb0 0x7f15740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.980+0000 7f158e000640 1 -- 172.21.15.86:0/842603416 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f15800d0650 con 0x7f15900749c0 2024-04-17T07:11:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.980+0000 7f1587e00640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f15740a3bb0 0x7f15740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.981+0000 7f1587e00640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f15740a3bb0 0x7f15740a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f159019b690 tx=0x7f1578005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:27.986+0000 7f158e000640 1 -- 172.21.15.86:0/842603416 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1580016030 con 0x7f15900749c0 2024-04-17T07:11:28.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:27 smithi098 ceph-mon[35418]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:28.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:27 smithi103 ceph-mon[35495]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:28.027 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:11:27 smithi103 systemd[1]: Starting Ceph mds.c for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:11:28.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:27 smithi086 ceph-mon[31733]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:28.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.224+0000 7f1596400640 1 -- 172.21.15.86:0/842603416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f159002d5d0 con 0x7f15900749c0 2024-04-17T07:11:28.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.225+0000 7f158e000640 1 -- 172.21.15.86:0/842603416 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5509 (secure 0 0 0) 0x7f158009d4d0 con 0x7f15900749c0 2024-04-17T07:11:28.226 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:28.226 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-17T07:11:28.226 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-17T07:11:28.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 -- 172.21.15.86:0/842603416 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f15740a3bb0 msgr2=0x7f15740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:28.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f15740a3bb0 0x7f15740a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f159019b690 tx=0x7f1578005ec0 comp rx=0 tx=0).stop 2024-04-17T07:11:28.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 -- 172.21.15.86:0/842603416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15900749c0 msgr2=0x7f159019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15900749c0 0x7f159019a180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f158000eab0 tx=0x7f158000ef70 comp rx=0 tx=0).stop 2024-04-17T07:11:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 -- 172.21.15.86:0/842603416 shutdown_connections 2024-04-17T07:11:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f15740a3bb0 0x7f15740a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1590109c90 0x7f15901a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1590075fa0 0x7f159019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 --2- 172.21.15.86:0/842603416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15900749c0 0x7f159019a180 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 -- 172.21.15.86:0/842603416 >> 172.21.15.86:0/842603416 conn(0x7f15900fd340 msgr2=0x7f1590105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 -- 172.21.15.86:0/842603416 shutdown_connections 2024-04-17T07:11:28.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:28.228+0000 7f1596400640 1 -- 172.21.15.86:0/842603416 wait complete. 2024-04-17T07:11:28.487 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:11:28 smithi103 podman[66771]: 2024-04-17 07:11:28.050048361 +0000 UTC m=+0.035357689 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:11:28.487 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:11:28 smithi103 podman[66771]: 2024-04-17 07:11:28.166203503 +0000 UTC m=+0.151512868 container create e7940f396d362baa7ad05506634effde6f59ffcbad0db602ab0b2ff7ab7fe12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:11:28.487 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:11:28 smithi103 podman[66771]: 2024-04-17 07:11:28.487478518 +0000 UTC m=+0.472787893 container init e7940f396d362baa7ad05506634effde6f59ffcbad0db602ab0b2ff7ab7fe12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-17T07:11:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:28 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/842603416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:28.776 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:11:28 smithi103 podman[66771]: 2024-04-17 07:11:28.499048613 +0000 UTC m=+0.484357951 container start e7940f396d362baa7ad05506634effde6f59ffcbad0db602ab0b2ff7ab7fe12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-17T07:11:28.777 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:11:28 smithi103 ceph-mds[66790]: -- 172.21.15.103:0/2832204841 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556318e93a40 con 0x556318e66c00 2024-04-17T07:11:28.777 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:11:28 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-c[66786]: starting mds.c at 2024-04-17T07:11:28.777 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:11:28 smithi103 bash[66771]: e7940f396d362baa7ad05506634effde6f59ffcbad0db602ab0b2ff7ab7fe12b 2024-04-17T07:11:28.777 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:11:28 smithi103 systemd[1]: Started Ceph mds.c for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:29.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:28 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/842603416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:29.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:28 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/842603416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.103:6836/299191474,v1:172.21.15.103:6837/299191474] up:boot 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: fsmap 7 up:standby 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: Deploying daemon mds.h on smithi098 2024-04-17T07:11:29.955 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:29 smithi086 ceph-mon[31733]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.103:6836/299191474,v1:172.21.15.103:6837/299191474] up:boot 2024-04-17T07:11:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: fsmap 7 up:standby 2024-04-17T07:11:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T07:11:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:30.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: Deploying daemon mds.h on smithi098 2024-04-17T07:11:30.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:29 smithi098 ceph-mon[35418]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:30.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.103:6836/299191474,v1:172.21.15.103:6837/299191474] up:boot 2024-04-17T07:11:30.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: fsmap 7 up:standby 2024-04-17T07:11:30.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-17T07:11:30.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:30.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:30.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:30.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:30.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:30.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:30.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: Deploying daemon mds.h on smithi098 2024-04-17T07:11:30.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:29 smithi103 ceph-mon[35495]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:30.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.445+0000 7f4775a00640 1 -- 172.21.15.86:0/2310308566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47700749c0 msgr2=0x7f4770074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.445+0000 7f4775a00640 1 --2- 172.21.15.86:0/2310308566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47700749c0 0x7f4770074e40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4758009900 tx=0x7f475802bc40 comp rx=0 tx=0).stop 2024-04-17T07:11:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.445+0000 7f4775a00640 1 -- 172.21.15.86:0/2310308566 shutdown_connections 2024-04-17T07:11:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.445+0000 7f4775a00640 1 --2- 172.21.15.86:0/2310308566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770109cc0 0x7f477010c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.445+0000 7f4775a00640 1 --2- 172.21.15.86:0/2310308566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47700749c0 0x7f4770074e40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.445+0000 7f4775a00640 1 --2- 172.21.15.86:0/2310308566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770075fa0 0x7f47700743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.445+0000 7f4775a00640 1 -- 172.21.15.86:0/2310308566 >> 172.21.15.86:0/2310308566 conn(0x7f47700fd370 msgr2=0x7f47700ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.445+0000 7f4775a00640 1 -- 172.21.15.86:0/2310308566 shutdown_connections 2024-04-17T07:11:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.445+0000 7f4775a00640 1 -- 172.21.15.86:0/2310308566 wait complete. 2024-04-17T07:11:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.446+0000 7f4775a00640 1 Processor -- start 2024-04-17T07:11:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.446+0000 7f4775a00640 1 -- start start 2024-04-17T07:11:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f4775a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47700749c0 0x7f477019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f4775a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770075fa0 0x7f477019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f4775a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770109cc0 0x7f47701a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f4775a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f477010fb40 con 0x7f47700749c0 2024-04-17T07:11:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f4775a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f477010f9c0 con 0x7f4770109cc0 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f4775a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f477010fcc0 con 0x7f4770075fa0 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f476fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47700749c0 0x7f477019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f476fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47700749c0 0x7f477019a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37606/0 (socket says 172.21.15.86:37606) 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f476fe00640 1 -- 172.21.15.86:0/3970145921 learned_addr learned my addr 172.21.15.86:0/3970145921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f476fe00640 1 -- 172.21.15.86:0/3970145921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770075fa0 msgr2=0x7f477019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f4767e00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770075fa0 0x7f477019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f476fe00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770075fa0 0x7f477019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f476fe00640 1 -- 172.21.15.86:0/3970145921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770109cc0 msgr2=0x7f47701a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f476fe00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770109cc0 0x7f47701a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.447+0000 7f476fe00640 1 -- 172.21.15.86:0/3970145921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47701a1d40 con 0x7f47700749c0 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.448+0000 7f476fe00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47700749c0 0x7f477019a120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f476000c920 tx=0x7f476000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.448+0000 7f476d600640 1 -- 172.21.15.86:0/3970145921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760007c70 con 0x7f47700749c0 2024-04-17T07:11:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.448+0000 7f476d600640 1 -- 172.21.15.86:0/3970145921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4760007e10 con 0x7f47700749c0 2024-04-17T07:11:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.448+0000 7f4775a00640 1 -- 172.21.15.86:0/3970145921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47701a1fd0 con 0x7f47700749c0 2024-04-17T07:11:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.448+0000 7f476d600640 1 -- 172.21.15.86:0/3970145921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760018820 con 0x7f47700749c0 2024-04-17T07:11:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.448+0000 7f4775a00640 1 -- 172.21.15.86:0/3970145921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47701a24c0 con 0x7f47700749c0 2024-04-17T07:11:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.449+0000 7f4775a00640 1 -- 172.21.15.86:0/3970145921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47701089f0 con 0x7f47700749c0 2024-04-17T07:11:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.451+0000 7f476d600640 1 -- 172.21.15.86:0/3970145921 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f47600189c0 con 0x7f47700749c0 2024-04-17T07:11:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.453+0000 7f476d600640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47540a3bb0 0x7f47540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.454+0000 7f476d600640 1 -- 172.21.15.86:0/3970145921 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f47600d04a0 con 0x7f47700749c0 2024-04-17T07:11:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.454+0000 7f4767e00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47540a3bb0 0x7f47540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.454+0000 7f4767e00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47540a3bb0 0x7f47540a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f47580096c0 tx=0x7f4758005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.457+0000 7f476d600640 1 -- 172.21.15.86:0/3970145921 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4760014030 con 0x7f47700749c0 2024-04-17T07:11:30.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.699+0000 7f4775a00640 1 -- 172.21.15.86:0/3970145921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f477002d5d0 con 0x7f47700749c0 2024-04-17T07:11:30.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.700+0000 7f476d600640 1 -- 172.21.15.86:0/3970145921 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6334 (secure 0 0 0) 0x7f476009d310 con 0x7f47700749c0 2024-04-17T07:11:30.700 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:30.700 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-17T07:11:30.700 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 8 2024-04-17T07:11:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.702+0000 7f4775a00640 1 -- 172.21.15.86:0/3970145921 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47540a3bb0 msgr2=0x7f47540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:30.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.702+0000 7f4775a00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47540a3bb0 0x7f47540a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f47580096c0 tx=0x7f4758005fd0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.702+0000 7f4775a00640 1 -- 172.21.15.86:0/3970145921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47700749c0 msgr2=0x7f477019a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.702+0000 7f4775a00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47700749c0 0x7f477019a120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f476000c920 tx=0x7f476000cde0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.702+0000 7f4775a00640 1 -- 172.21.15.86:0/3970145921 shutdown_connections 2024-04-17T07:11:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.702+0000 7f4775a00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47540a3bb0 0x7f47540a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.702+0000 7f4775a00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4770109cc0 0x7f47701a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.702+0000 7f4775a00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4770075fa0 0x7f477019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.702+0000 7f4775a00640 1 --2- 172.21.15.86:0/3970145921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47700749c0 0x7f477019a120 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.702+0000 7f4775a00640 1 -- 172.21.15.86:0/3970145921 >> 172.21.15.86:0/3970145921 conn(0x7f47700fd370 msgr2=0x7f4770105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.703+0000 7f4775a00640 1 -- 172.21.15.86:0/3970145921 shutdown_connections 2024-04-17T07:11:30.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:30.703+0000 7f4775a00640 1 -- 172.21.15.86:0/3970145921 wait complete. 2024-04-17T07:11:31.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:30 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3970145921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:31.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:30 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3970145921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:31.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:30 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3970145921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:32.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:31 smithi098 ceph-mon[35418]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:32.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:31 smithi103 ceph-mon[35495]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:32.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:31 smithi086 ceph-mon[31733]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:32.376 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:11:32 smithi098 systemd[1]: Starting Ceph mds.h for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:11:32.651 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:11:32 smithi098 podman[65837]: 2024-04-17 07:11:32.281878132 +0000 UTC m=+0.033333868 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:11:32.651 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:11:32 smithi098 podman[65837]: 2024-04-17 07:11:32.396237922 +0000 UTC m=+0.147693696 container create 2590f5f40a5fc92449cb813d6746588c87b0940bc1a50e5237cd467844365266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:11:32.651 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:11:32 smithi098 podman[65837]: 2024-04-17 07:11:32.65058757 +0000 UTC m=+0.402043357 container init 2590f5f40a5fc92449cb813d6746588c87b0940bc1a50e5237cd467844365266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-h, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-17T07:11:32.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:32.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.948+0000 7fb31cc00640 1 -- 172.21.15.86:0/713571496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb318105b90 msgr2=0x7fb31810c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:32.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.948+0000 7fb31cc00640 1 --2- 172.21.15.86:0/713571496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb318105b90 0x7fb31810c420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb310009ca0 tx=0x7fb31002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:11:32.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.948+0000 7fb31cc00640 1 -- 172.21.15.86:0/713571496 shutdown_connections 2024-04-17T07:11:32.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.948+0000 7fb31cc00640 1 --2- 172.21.15.86:0/713571496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb318105b90 0x7fb31810c420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:32.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.948+0000 7fb31cc00640 1 --2- 172.21.15.86:0/713571496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3181051d0 0x7fb318105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:32.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.948+0000 7fb31cc00640 1 --2- 172.21.15.86:0/713571496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb318103fe0 0x7fb318104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:32.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.948+0000 7fb31cc00640 1 -- 172.21.15.86:0/713571496 >> 172.21.15.86:0/713571496 conn(0x7fb3180ff570 msgr2=0x7fb318101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:32.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.948+0000 7fb31cc00640 1 -- 172.21.15.86:0/713571496 shutdown_connections 2024-04-17T07:11:32.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.948+0000 7fb31cc00640 1 -- 172.21.15.86:0/713571496 wait complete. 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.949+0000 7fb31cc00640 1 Processor -- start 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.949+0000 7fb31cc00640 1 -- start start 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.949+0000 7fb31cc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb318103fe0 0x7fb31819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.949+0000 7fb31cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3181051d0 0x7fb31819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.949+0000 7fb31cc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb318105b90 0x7fb3181a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.949+0000 7fb31cc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb31810fb20 con 0x7fb3181051d0 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.949+0000 7fb31cc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb31810f9a0 con 0x7fb318103fe0 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.949+0000 7fb31cc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb31810fca0 con 0x7fb318105b90 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb316a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3181051d0 0x7fb31819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb316a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3181051d0 0x7fb31819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37610/0 (socket says 172.21.15.86:37610) 2024-04-17T07:11:32.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb316a00640 1 -- 172.21.15.86:0/834655053 learned_addr learned my addr 172.21.15.86:0/834655053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb317e00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb318105b90 0x7fb3181a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb317400640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb318103fe0 0x7fb31819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb316a00640 1 -- 172.21.15.86:0/834655053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb318105b90 msgr2=0x7fb3181a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb316a00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb318105b90 0x7fb3181a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb316a00640 1 -- 172.21.15.86:0/834655053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb318103fe0 msgr2=0x7fb31819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb316a00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb318103fe0 0x7fb31819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb316a00640 1 -- 172.21.15.86:0/834655053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3181a1d20 con 0x7fb3181051d0 2024-04-17T07:11:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.950+0000 7fb317400640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb318103fe0 0x7fb31819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:32.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.951+0000 7fb316a00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3181051d0 0x7fb31819a640 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb30400c920 tx=0x7fb30400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:32.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.952+0000 7fb30fe00640 1 -- 172.21.15.86:0/834655053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304007c70 con 0x7fb3181051d0 2024-04-17T07:11:32.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.952+0000 7fb30fe00640 1 -- 172.21.15.86:0/834655053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb304007e10 con 0x7fb3181051d0 2024-04-17T07:11:32.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.953+0000 7fb31cc00640 1 -- 172.21.15.86:0/834655053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3181a1fb0 con 0x7fb3181051d0 2024-04-17T07:11:32.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.953+0000 7fb30fe00640 1 -- 172.21.15.86:0/834655053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304018820 con 0x7fb3181051d0 2024-04-17T07:11:32.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.953+0000 7fb31cc00640 1 -- 172.21.15.86:0/834655053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3181a2520 con 0x7fb3181051d0 2024-04-17T07:11:32.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.954+0000 7fb31cc00640 1 -- 172.21.15.86:0/834655053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31810b300 con 0x7fb3181051d0 2024-04-17T07:11:32.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.957+0000 7fb30fe00640 1 -- 172.21.15.86:0/834655053 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb3040189c0 con 0x7fb3181051d0 2024-04-17T07:11:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.959+0000 7fb30fe00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2fc0a3c30 0x7fb2fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.959+0000 7fb30fe00640 1 -- 172.21.15.86:0/834655053 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb3040d00d0 con 0x7fb3181051d0 2024-04-17T07:11:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.959+0000 7fb317400640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2fc0a3c30 0x7fb2fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:32.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.960+0000 7fb317400640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2fc0a3c30 0x7fb2fc0a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb318105030 tx=0x7fb308005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:32.965+0000 7fb30fe00640 1 -- 172.21.15.86:0/834655053 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb304014030 con 0x7fb3181051d0 2024-04-17T07:11:33.003 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:11:32 smithi098 podman[65837]: 2024-04-17 07:11:32.660982598 +0000 UTC m=+0.412438412 container start 2590f5f40a5fc92449cb813d6746588c87b0940bc1a50e5237cd467844365266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-17T07:11:33.003 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:11:32 smithi098 ceph-mds[65855]: -- 172.21.15.98:0/3418560767 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56243016ba40 con 0x56243013e800 2024-04-17T07:11:33.003 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:11:32 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-h[65851]: starting mds.h at 2024-04-17T07:11:33.003 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:11:32 smithi098 bash[65837]: 2590f5f40a5fc92449cb813d6746588c87b0940bc1a50e5237cd467844365266 2024-04-17T07:11:33.003 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:11:32 smithi098 systemd[1]: Started Ceph mds.h for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.210+0000 7fb31cc00640 1 -- 172.21.15.86:0/834655053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb31802d840 con 0x7fb3181051d0 2024-04-17T07:11:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.211+0000 7fb30fe00640 1 -- 172.21.15.86:0/834655053 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7159 (secure 0 0 0) 0x7fb30400de20 con 0x7fb3181051d0 2024-04-17T07:11:33.212 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:33.212 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T07:11:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-17T07:11:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.214+0000 7fb31cc00640 1 -- 172.21.15.86:0/834655053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2fc0a3c30 msgr2=0x7fb2fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.214+0000 7fb31cc00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2fc0a3c30 0x7fb2fc0a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb318105030 tx=0x7fb308005d30 comp rx=0 tx=0).stop 2024-04-17T07:11:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.214+0000 7fb31cc00640 1 -- 172.21.15.86:0/834655053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3181051d0 msgr2=0x7fb31819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.214+0000 7fb31cc00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3181051d0 0x7fb31819a640 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb30400c920 tx=0x7fb30400cde0 comp rx=0 tx=0).stop 2024-04-17T07:11:33.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.214+0000 7fb31cc00640 1 -- 172.21.15.86:0/834655053 shutdown_connections 2024-04-17T07:11:33.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.214+0000 7fb31cc00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2fc0a3c30 0x7fb2fc0a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:33.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.214+0000 7fb31cc00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb318105b90 0x7fb3181a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:33.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.214+0000 7fb31cc00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3181051d0 0x7fb31819a640 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:33.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.214+0000 7fb31cc00640 1 --2- 172.21.15.86:0/834655053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb318103fe0 0x7fb31819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:33.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.214+0000 7fb31cc00640 1 -- 172.21.15.86:0/834655053 >> 172.21.15.86:0/834655053 conn(0x7fb3180ff570 msgr2=0x7fb3181084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:33.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.215+0000 7fb31cc00640 1 -- 172.21.15.86:0/834655053 shutdown_connections 2024-04-17T07:11:33.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:33.215+0000 7fb31cc00640 1 -- 172.21.15.86:0/834655053 wait complete. 2024-04-17T07:11:34.159 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] up:boot 2024-04-17T07:11:34.159 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: fsmap 8 up:standby 2024-04-17T07:11:34.159 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T07:11:34.159 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:34.159 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:34.159 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:34.159 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:34.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:34.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:34.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: Deploying daemon mds.i on smithi103 2024-04-17T07:11:34.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:34.160 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:33 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/834655053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] up:boot 2024-04-17T07:11:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: fsmap 8 up:standby 2024-04-17T07:11:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T07:11:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:34.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: Deploying daemon mds.i on smithi103 2024-04-17T07:11:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:34.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:33 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/834655053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:34.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] up:boot 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: fsmap 8 up:standby 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: Deploying daemon mds.i on smithi103 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:34.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:33 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/834655053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:35.234 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:35.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.457+0000 7fdddf200640 1 -- 172.21.15.86:0/130803635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdde0109f90 msgr2=0x7fdde0110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:35.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.457+0000 7fdddf200640 1 --2- 172.21.15.86:0/130803635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdde0109f90 0x7fdde0110820 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009ca0 tx=0x7fddd802c0f0 comp rx=0 tx=0).stop 2024-04-17T07:11:35.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.457+0000 7fdddf200640 1 -- 172.21.15.86:0/130803635 shutdown_connections 2024-04-17T07:11:35.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.457+0000 7fdddf200640 1 --2- 172.21.15.86:0/130803635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdde0109f90 0x7fdde0110820 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:35.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.457+0000 7fdddf200640 1 --2- 172.21.15.86:0/130803635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde01095d0 0x7fdde0109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:35.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.457+0000 7fdddf200640 1 --2- 172.21.15.86:0/130803635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdde01083e0 0x7fdde0108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:35.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.457+0000 7fdddf200640 1 -- 172.21.15.86:0/130803635 >> 172.21.15.86:0/130803635 conn(0x7fdde00773a0 msgr2=0x7fdde00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:35.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.457+0000 7fdddf200640 1 -- 172.21.15.86:0/130803635 shutdown_connections 2024-04-17T07:11:35.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.457+0000 7fdddf200640 1 -- 172.21.15.86:0/130803635 wait complete. 2024-04-17T07:11:35.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.458+0000 7fdddf200640 1 Processor -- start 2024-04-17T07:11:35.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.458+0000 7fdddf200640 1 -- start start 2024-04-17T07:11:35.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fdddf200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde01083e0 0x7fdde019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:35.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fdddf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdde01095d0 0x7fdde019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fdddf200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdde0109f90 0x7fdde01a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fdddf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdde0113ef0 con 0x7fdde01095d0 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fdddf200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdde0113d70 con 0x7fdde0109f90 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fdddf200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdde0114070 con 0x7fdde01083e0 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fdddd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdde01095d0 0x7fdde019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fdddd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdde01095d0 0x7fdde019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37632/0 (socket says 172.21.15.86:37632) 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fdddde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde01083e0 0x7fdde019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fdddd400640 1 -- 172.21.15.86:0/2288823083 learned_addr learned my addr 172.21.15.86:0/2288823083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.459+0000 7fddde800640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdde0109f90 0x7fdde01a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fdddd400640 1 -- 172.21.15.86:0/2288823083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde01083e0 msgr2=0x7fdde019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fdddd400640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde01083e0 0x7fdde019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fdddd400640 1 -- 172.21.15.86:0/2288823083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdde0109f90 msgr2=0x7fdde01a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:35.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fdddd400640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdde0109f90 0x7fdde01a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fdddd400640 1 -- 172.21.15.86:0/2288823083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde01a61e0 con 0x7fdde01095d0 2024-04-17T07:11:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fddde800640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdde0109f90 0x7fdde01a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fdddd400640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdde01095d0 0x7fdde019eb00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00d950 tx=0x7fddcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fddd6a00640 1 -- 172.21.15.86:0/2288823083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc017070 con 0x7fdde01095d0 2024-04-17T07:11:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fddd6a00640 1 -- 172.21.15.86:0/2288823083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddcc009c30 con 0x7fdde01095d0 2024-04-17T07:11:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fdddf200640 1 -- 172.21.15.86:0/2288823083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde01a64d0 con 0x7fdde01095d0 2024-04-17T07:11:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fddd6a00640 1 -- 172.21.15.86:0/2288823083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddcc00f620 con 0x7fdde01095d0 2024-04-17T07:11:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.460+0000 7fdddf200640 1 -- 172.21.15.86:0/2288823083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde01a6a20 con 0x7fdde01095d0 2024-04-17T07:11:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.461+0000 7fdddf200640 1 -- 172.21.15.86:0/2288823083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde010f7a0 con 0x7fdde01095d0 2024-04-17T07:11:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.464+0000 7fddd6a00640 1 -- 172.21.15.86:0/2288823083 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fddcc0109a0 con 0x7fdde01095d0 2024-04-17T07:11:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.465+0000 7fddd6a00640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddc40a3bb0 0x7fddc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.465+0000 7fddd6a00640 1 -- 172.21.15.86:0/2288823083 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fddcc0cfc30 con 0x7fdde01095d0 2024-04-17T07:11:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.466+0000 7fdddde00640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddc40a3bb0 0x7fddc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.466+0000 7fdddde00640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddc40a3bb0 0x7fddc40a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdde0109430 tx=0x7fddd000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.470+0000 7fddd6a00640 1 -- 172.21.15.86:0/2288823083 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fddcc009de0 con 0x7fdde01095d0 2024-04-17T07:11:35.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.713+0000 7fdddf200640 1 -- 172.21.15.86:0/2288823083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdde0062b60 con 0x7fdde01095d0 2024-04-17T07:11:35.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.714+0000 7fddd6a00640 1 -- 172.21.15.86:0/2288823083 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7159 (secure 0 0 0) 0x7fddcc09ca10 con 0x7fdde01095d0 2024-04-17T07:11:35.715 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:35.715 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T07:11:35.715 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-17T07:11:35.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.717+0000 7fdddf200640 1 -- 172.21.15.86:0/2288823083 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddc40a3bb0 msgr2=0x7fddc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.717+0000 7fdddf200640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddc40a3bb0 0x7fddc40a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdde0109430 tx=0x7fddd000b390 comp rx=0 tx=0).stop 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.717+0000 7fdddf200640 1 -- 172.21.15.86:0/2288823083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdde01095d0 msgr2=0x7fdde019eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.717+0000 7fdddf200640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdde01095d0 0x7fdde019eb00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00d950 tx=0x7fddcc00de10 comp rx=0 tx=0).stop 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.717+0000 7fdddf200640 1 -- 172.21.15.86:0/2288823083 shutdown_connections 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.717+0000 7fdddf200640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddc40a3bb0 0x7fddc40a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.717+0000 7fdddf200640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdde0109f90 0x7fdde01a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.717+0000 7fdddf200640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdde01095d0 0x7fdde019eb00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.717+0000 7fdddf200640 1 --2- 172.21.15.86:0/2288823083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde01083e0 0x7fdde019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.717+0000 7fdddf200640 1 -- 172.21.15.86:0/2288823083 >> 172.21.15.86:0/2288823083 conn(0x7fdde00773a0 msgr2=0x7fdde010c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.718+0000 7fdddf200640 1 -- 172.21.15.86:0/2288823083 shutdown_connections 2024-04-17T07:11:35.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:35.718+0000 7fdddf200640 1 -- 172.21.15.86:0/2288823083 wait complete. 2024-04-17T07:11:36.396 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:36 smithi103 ceph-mon[35495]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:36.397 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:36 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2288823083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:36.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:36 smithi098 ceph-mon[35418]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:36 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2288823083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:36 smithi086 ceph-mon[31733]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:36 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2288823083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:36.752 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:11:36 smithi103 podman[67076]: 2024-04-17 07:11:36.438803707 +0000 UTC m=+0.172635518 container create 8e3e7493cc70d8a7c9e861ea01966912ee235ae7cd6bc1d032c07e9980f22294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:11:36.752 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:11:36 smithi103 podman[67076]: 2024-04-17 07:11:36.75102563 +0000 UTC m=+0.484857477 container init 8e3e7493cc70d8a7c9e861ea01966912ee235ae7cd6bc1d032c07e9980f22294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:11:37.027 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:11:36 smithi103 podman[67076]: 2024-04-17 07:11:36.761890017 +0000 UTC m=+0.495721816 container start 8e3e7493cc70d8a7c9e861ea01966912ee235ae7cd6bc1d032c07e9980f22294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:11:37.027 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:11:36 smithi103 ceph-mds[67096]: -- 172.21.15.103:0/1970709414 <== mon.1 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5629393bba40 con 0x56293938f000 2024-04-17T07:11:37.027 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:11:36 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-i[67092]: starting mds.i at 2024-04-17T07:11:37.027 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:11:36 smithi103 bash[67076]: 8e3e7493cc70d8a7c9e861ea01966912ee235ae7cd6bc1d032c07e9980f22294 2024-04-17T07:11:37.027 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:11:36 smithi103 systemd[1]: Started Ceph mds.i for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:37.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.930+0000 7fd053800640 1 -- 172.21.15.86:0/3154029027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c1041c0 msgr2=0x7fd04c1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.930+0000 7fd053800640 1 --2- 172.21.15.86:0/3154029027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c1041c0 0x7fd04c1045e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009900 tx=0x7fd03c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.930+0000 7fd053800640 1 -- 172.21.15.86:0/3154029027 shutdown_connections 2024-04-17T07:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.930+0000 7fd053800640 1 --2- 172.21.15.86:0/3154029027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd04c105d70 0x7fd04c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.930+0000 7fd053800640 1 --2- 172.21.15.86:0/3154029027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd04c1053b0 0x7fd04c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.930+0000 7fd053800640 1 --2- 172.21.15.86:0/3154029027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c1041c0 0x7fd04c1045e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:37.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.930+0000 7fd053800640 1 -- 172.21.15.86:0/3154029027 >> 172.21.15.86:0/3154029027 conn(0x7fd04c0ff750 msgr2=0x7fd04c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:37.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.930+0000 7fd053800640 1 -- 172.21.15.86:0/3154029027 shutdown_connections 2024-04-17T07:11:37.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.930+0000 7fd053800640 1 -- 172.21.15.86:0/3154029027 wait complete. 2024-04-17T07:11:37.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.931+0000 7fd053800640 1 Processor -- start 2024-04-17T07:11:37.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.931+0000 7fd053800640 1 -- start start 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd053800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd04c1041c0 0x7fd04c079d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd053800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c1053b0 0x7fd04c07a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd053800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd04c105d70 0x7fd04c076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd053800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c10fd40 con 0x7fd04c1053b0 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd053800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c10fbc0 con 0x7fd04c1041c0 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd053800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c10fec0 con 0x7fd04c105d70 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd043e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c1053b0 0x7fd04c07a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd043e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c1053b0 0x7fd04c07a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42340/0 (socket says 172.21.15.86:42340) 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd043e00640 1 -- 172.21.15.86:0/4014903003 learned_addr learned my addr 172.21.15.86:0/4014903003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd050e00640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd04c1041c0 0x7fd04c079d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd051800640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd04c105d70 0x7fd04c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:37.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd043e00640 1 -- 172.21.15.86:0/4014903003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd04c105d70 msgr2=0x7fd04c076880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd043e00640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd04c105d70 0x7fd04c076880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd043e00640 1 -- 172.21.15.86:0/4014903003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd04c1041c0 msgr2=0x7fd04c079d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd043e00640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd04c1041c0 0x7fd04c079d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd043e00640 1 -- 172.21.15.86:0/4014903003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c077140 con 0x7fd04c1053b0 2024-04-17T07:11:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.932+0000 7fd050e00640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd04c1041c0 0x7fd04c079d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.933+0000 7fd051800640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd04c105d70 0x7fd04c076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.933+0000 7fd043e00640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c1053b0 0x7fd04c07a250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c890 tx=0x7fd03400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.933+0000 7fd04a000640 1 -- 172.21.15.86:0/4014903003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034004220 con 0x7fd04c1053b0 2024-04-17T07:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.933+0000 7fd053800640 1 -- 172.21.15.86:0/4014903003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c1a68e0 con 0x7fd04c1053b0 2024-04-17T07:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.933+0000 7fd04a000640 1 -- 172.21.15.86:0/4014903003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0340043c0 con 0x7fd04c1053b0 2024-04-17T07:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.933+0000 7fd04a000640 1 -- 172.21.15.86:0/4014903003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03401d060 con 0x7fd04c1053b0 2024-04-17T07:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.933+0000 7fd053800640 1 -- 172.21.15.86:0/4014903003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c1a6dc0 con 0x7fd04c1053b0 2024-04-17T07:11:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.935+0000 7fd053800640 1 -- 172.21.15.86:0/4014903003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c10b580 con 0x7fd04c1053b0 2024-04-17T07:11:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.937+0000 7fd04a000640 1 -- 172.21.15.86:0/4014903003 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd0340049a0 con 0x7fd04c1053b0 2024-04-17T07:11:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.938+0000 7fd04a000640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0300a3bb0 0x7fd0300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:37.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.938+0000 7fd04a000640 1 -- 172.21.15.86:0/4014903003 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd0340cfd40 con 0x7fd04c1053b0 2024-04-17T07:11:37.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.938+0000 7fd050e00640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0300a3bb0 0x7fd0300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:37.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.939+0000 7fd050e00640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0300a3bb0 0x7fd0300a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0096c0 tx=0x7fd03c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:37.943+0000 7fd04a000640 1 -- 172.21.15.86:0/4014903003 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd034014030 con 0x7fd04c1053b0 2024-04-17T07:11:38.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:38.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:38.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:38.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:38.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:38.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:38.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: Deploying daemon mds.g on smithi086 2024-04-17T07:11:38.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:38.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] up:boot 2024-04-17T07:11:38.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: fsmap 9 up:standby 2024-04-17T07:11:38.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:37 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T07:11:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.194+0000 7fd053800640 1 -- 172.21.15.86:0/4014903003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd04c02d5d0 con 0x7fd04c1053b0 2024-04-17T07:11:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.196+0000 7fd04a000640 1 -- 172.21.15.86:0/4014903003 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7989 (secure 0 0 0) 0x7fd03409cbc0 con 0x7fd04c1053b0 2024-04-17T07:11:38.196 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:38.196 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T07:11:38.196 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 10 2024-04-17T07:11:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 -- 172.21.15.86:0/4014903003 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0300a3bb0 msgr2=0x7fd0300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0300a3bb0 0x7fd0300a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0096c0 tx=0x7fd03c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 -- 172.21.15.86:0/4014903003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c1053b0 msgr2=0x7fd04c07a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c1053b0 0x7fd04c07a250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c890 tx=0x7fd03400cd50 comp rx=0 tx=0).stop 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 -- 172.21.15.86:0/4014903003 shutdown_connections 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0300a3bb0 0x7fd0300a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd04c105d70 0x7fd04c076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04c1053b0 0x7fd04c07a250 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 --2- 172.21.15.86:0/4014903003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd04c1041c0 0x7fd04c079d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 -- 172.21.15.86:0/4014903003 >> 172.21.15.86:0/4014903003 conn(0x7fd04c0ff750 msgr2=0x7fd04c108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.198+0000 7fd053800640 1 -- 172.21.15.86:0/4014903003 shutdown_connections 2024-04-17T07:11:38.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:38.199+0000 7fd053800640 1 -- 172.21.15.86:0/4014903003 wait complete. 2024-04-17T07:11:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:38.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: Deploying daemon mds.g on smithi086 2024-04-17T07:11:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] up:boot 2024-04-17T07:11:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: fsmap 9 up:standby 2024-04-17T07:11:38.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:37 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T07:11:38.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:38.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:38.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:38.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:38.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:38.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:38.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: Deploying daemon mds.g on smithi086 2024-04-17T07:11:38.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:38.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] up:boot 2024-04-17T07:11:38.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: fsmap 9 up:standby 2024-04-17T07:11:38.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:37 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-17T07:11:39.201 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:39 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4014903003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:39.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:39 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4014903003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:39.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:39 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4014903003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:40.220 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:40.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:40 smithi103 ceph-mon[35495]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:40.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:40 smithi086 ceph-mon[31733]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.412+0000 7f8fdea00640 1 -- 172.21.15.86:0/2765784045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd8105190 msgr2=0x7f8fd8107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.412+0000 7f8fdea00640 1 --2- 172.21.15.86:0/2765784045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd8105190 0x7f8fd8107580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009900 tx=0x7f8fc802bc50 comp rx=0 tx=0).stop 2024-04-17T07:11:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.412+0000 7f8fdea00640 1 -- 172.21.15.86:0/2765784045 shutdown_connections 2024-04-17T07:11:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.412+0000 7f8fdea00640 1 --2- 172.21.15.86:0/2765784045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd8107ac0 0x7f8fd8109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.412+0000 7f8fdea00640 1 --2- 172.21.15.86:0/2765784045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd8105190 0x7f8fd8107580 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.412+0000 7f8fdea00640 1 --2- 172.21.15.86:0/2765784045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd8102860 0x7f8fd8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.412+0000 7f8fdea00640 1 -- 172.21.15.86:0/2765784045 >> 172.21.15.86:0/2765784045 conn(0x7f8fd80fc270 msgr2=0x7f8fd80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.412+0000 7f8fdea00640 1 -- 172.21.15.86:0/2765784045 shutdown_connections 2024-04-17T07:11:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.412+0000 7f8fdea00640 1 -- 172.21.15.86:0/2765784045 wait complete. 2024-04-17T07:11:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.413+0000 7f8fdea00640 1 Processor -- start 2024-04-17T07:11:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.413+0000 7f8fdea00640 1 -- start start 2024-04-17T07:11:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fdea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd8102860 0x7f8fd819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fdea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd8105190 0x7f8fd819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fdea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd8107ac0 0x7f8fd81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fdea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd810d950 con 0x7f8fd8107ac0 2024-04-17T07:11:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fdea00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd810d7d0 con 0x7f8fd8105190 2024-04-17T07:11:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fdea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd810dad0 con 0x7f8fd8102860 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fde000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd8107ac0 0x7f8fd81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fde000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd8107ac0 0x7f8fd81a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42368/0 (socket says 172.21.15.86:42368) 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fde000640 1 -- 172.21.15.86:0/2585428702 learned_addr learned my addr 172.21.15.86:0/2585428702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fdd600640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd8102860 0x7f8fd819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.414+0000 7f8fdcc00640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd8105190 0x7f8fd819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fde000640 1 -- 172.21.15.86:0/2585428702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd8102860 msgr2=0x7f8fd819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fde000640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd8102860 0x7f8fd819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fde000640 1 -- 172.21.15.86:0/2585428702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd8105190 msgr2=0x7f8fd819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fde000640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd8105190 0x7f8fd819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fde000640 1 -- 172.21.15.86:0/2585428702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd81a1d80 con 0x7f8fd8107ac0 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fdd600640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd8102860 0x7f8fd819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fde000640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd8107ac0 0x7f8fd81a1710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000cce0 tx=0x7f8fd000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fd6000640 1 -- 172.21.15.86:0/2585428702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd000f450 con 0x7f8fd8107ac0 2024-04-17T07:11:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fdea00640 1 -- 172.21.15.86:0/2585428702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd81a2070 con 0x7f8fd8107ac0 2024-04-17T07:11:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fd6000640 1 -- 172.21.15.86:0/2585428702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fd000faa0 con 0x7f8fd8107ac0 2024-04-17T07:11:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fd6000640 1 -- 172.21.15.86:0/2585428702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0018b70 con 0x7f8fd8107ac0 2024-04-17T07:11:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.415+0000 7f8fdea00640 1 -- 172.21.15.86:0/2585428702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd81a2540 con 0x7f8fd8107ac0 2024-04-17T07:11:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.416+0000 7f8fdea00640 1 -- 172.21.15.86:0/2585428702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd8109610 con 0x7f8fd8107ac0 2024-04-17T07:11:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.418+0000 7f8fd6000640 1 -- 172.21.15.86:0/2585428702 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fd0018d10 con 0x7f8fd8107ac0 2024-04-17T07:11:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.420+0000 7f8fd6000640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fc00a3bb0 0x7f8fc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.420+0000 7f8fd6000640 1 -- 172.21.15.86:0/2585428702 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8fd00d04d0 con 0x7f8fd8107ac0 2024-04-17T07:11:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.420+0000 7f8fdd600640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fc00a3bb0 0x7f8fc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.421+0000 7f8fdd600640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fc00a3bb0 0x7f8fc00a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00abe0 tx=0x7f8fcc00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.426+0000 7f8fd6000640 1 -- 172.21.15.86:0/2585428702 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8fd0014030 con 0x7f8fd8107ac0 2024-04-17T07:11:40.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:40 smithi098 ceph-mon[35418]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:40.580 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:11:40 smithi086 podman[70766]: 2024-04-17 07:11:40.420823386 +0000 UTC m=+0.160703241 container create ed9543eec6ff30c144e770a438537b88689dbb73ad9b4c0daf7f5f9be8f1869b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-17T07:11:40.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.677+0000 7f8fdea00640 1 -- 172.21.15.86:0/2585428702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fd8062b60 con 0x7f8fd8107ac0 2024-04-17T07:11:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.678+0000 7f8fd6000640 1 -- 172.21.15.86:0/2585428702 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7989 (secure 0 0 0) 0x7f8fd009d2b0 con 0x7f8fd8107ac0 2024-04-17T07:11:40.679 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:40.679 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T07:11:40.679 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 10 2024-04-17T07:11:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 -- 172.21.15.86:0/2585428702 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fc00a3bb0 msgr2=0x7f8fc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fc00a3bb0 0x7f8fc00a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00abe0 tx=0x7f8fcc00a5c0 comp rx=0 tx=0).stop 2024-04-17T07:11:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 -- 172.21.15.86:0/2585428702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd8107ac0 msgr2=0x7f8fd81a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd8107ac0 0x7f8fd81a1710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000cce0 tx=0x7f8fd000a620 comp rx=0 tx=0).stop 2024-04-17T07:11:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 -- 172.21.15.86:0/2585428702 shutdown_connections 2024-04-17T07:11:40.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fc00a3bb0 0x7f8fc00a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:40.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fd8107ac0 0x7f8fd81a1710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:40.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fd8105190 0x7f8fd819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:40.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 --2- 172.21.15.86:0/2585428702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd8102860 0x7f8fd819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:40.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 -- 172.21.15.86:0/2585428702 >> 172.21.15.86:0/2585428702 conn(0x7f8fd80fc270 msgr2=0x7f8fd8101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:40.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 -- 172.21.15.86:0/2585428702 shutdown_connections 2024-04-17T07:11:40.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:40.681+0000 7f8fdea00640 1 -- 172.21.15.86:0/2585428702 wait complete. 2024-04-17T07:11:41.036 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:11:40 smithi086 podman[70766]: 2024-04-17 07:11:40.699798033 +0000 UTC m=+0.439677938 container init ed9543eec6ff30c144e770a438537b88689dbb73ad9b4c0daf7f5f9be8f1869b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-17T07:11:41.036 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:11:40 smithi086 podman[70766]: 2024-04-17 07:11:40.707097748 +0000 UTC m=+0.446977677 container start ed9543eec6ff30c144e770a438537b88689dbb73ad9b4c0daf7f5f9be8f1869b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True) 2024-04-17T07:11:41.036 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:11:40 smithi086 ceph-mds[70817]: -- 172.21.15.86:0/3397470607 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc5554ba40 con 0x55dc5551ec00 2024-04-17T07:11:41.036 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:11:40 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-g[70811]: starting mds.g at 2024-04-17T07:11:41.036 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:11:40 smithi086 bash[70766]: ed9543eec6ff30c144e770a438537b88689dbb73ad9b4c0daf7f5f9be8f1869b 2024-04-17T07:11:41.036 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:11:40 smithi086 systemd[1]: Started Ceph mds.g for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:41.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:41 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2585428702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:41.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:41.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:41.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:41.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:41.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:41.320 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:41 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2585428702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:41 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2585428702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:41.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:42.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:42 smithi103 ceph-mon[35495]: Deploying daemon mds.a on smithi086 2024-04-17T07:11:42.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:42 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.86:6838/2923053510,v1:172.21.15.86:6839/2923053510] up:boot 2024-04-17T07:11:42.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:42 smithi103 ceph-mon[35495]: fsmap 10 up:standby 2024-04-17T07:11:42.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:42 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T07:11:42.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:42 smithi103 ceph-mon[35495]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:42.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:42 smithi086 ceph-mon[31733]: Deploying daemon mds.a on smithi086 2024-04-17T07:11:42.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:42 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.86:6838/2923053510,v1:172.21.15.86:6839/2923053510] up:boot 2024-04-17T07:11:42.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:42 smithi086 ceph-mon[31733]: fsmap 10 up:standby 2024-04-17T07:11:42.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:42 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T07:11:42.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:42 smithi086 ceph-mon[31733]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:42 smithi098 ceph-mon[35418]: Deploying daemon mds.a on smithi086 2024-04-17T07:11:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:42 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.86:6838/2923053510,v1:172.21.15.86:6839/2923053510] up:boot 2024-04-17T07:11:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:42 smithi098 ceph-mon[35418]: fsmap 10 up:standby 2024-04-17T07:11:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:42 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-17T07:11:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:42 smithi098 ceph-mon[35418]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:42.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.936+0000 7f3742200640 1 -- 172.21.15.86:0/4153910583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f373c103fe0 msgr2=0x7f373c104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.936+0000 7f3742200640 1 --2- 172.21.15.86:0/4153910583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f373c103fe0 0x7f373c104400 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3734009ca0 tx=0x7f373402c120 comp rx=0 tx=0).stop 2024-04-17T07:11:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.936+0000 7f3742200640 1 -- 172.21.15.86:0/4153910583 shutdown_connections 2024-04-17T07:11:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.936+0000 7f3742200640 1 --2- 172.21.15.86:0/4153910583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f373c105b90 0x7f373c10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.936+0000 7f3742200640 1 --2- 172.21.15.86:0/4153910583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373c1051d0 0x7f373c105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.936+0000 7f3742200640 1 --2- 172.21.15.86:0/4153910583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f373c103fe0 0x7f373c104400 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.936+0000 7f3742200640 1 -- 172.21.15.86:0/4153910583 >> 172.21.15.86:0/4153910583 conn(0x7f373c0ff570 msgr2=0x7f373c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.936+0000 7f3742200640 1 -- 172.21.15.86:0/4153910583 shutdown_connections 2024-04-17T07:11:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.936+0000 7f3742200640 1 -- 172.21.15.86:0/4153910583 wait complete. 2024-04-17T07:11:42.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.937+0000 7f3742200640 1 Processor -- start 2024-04-17T07:11:42.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.937+0000 7f3742200640 1 -- start start 2024-04-17T07:11:42.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.937+0000 7f3742200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373c103fe0 0x7f373c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.937+0000 7f3742200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f373c1051d0 0x7f373c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.937+0000 7f3742200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f373c105b90 0x7f373c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.937+0000 7f3742200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f373c10fba0 con 0x7f373c105b90 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.937+0000 7f3742200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f373c10fa20 con 0x7f373c1051d0 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.937+0000 7f3742200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f373c10fd20 con 0x7f373c103fe0 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.938+0000 7f3741800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f373c105b90 0x7f373c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.938+0000 7f3740e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373c103fe0 0x7f373c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.938+0000 7f3741800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f373c105b90 0x7f373c1a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42408/0 (socket says 172.21.15.86:42408) 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.938+0000 7f3741800640 1 -- 172.21.15.86:0/842309095 learned_addr learned my addr 172.21.15.86:0/842309095 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.938+0000 7f373be00640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f373c1051d0 0x7f373c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.938+0000 7f3741800640 1 -- 172.21.15.86:0/842309095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373c103fe0 msgr2=0x7f373c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.938+0000 7f3741800640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373c103fe0 0x7f373c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:42.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.938+0000 7f3741800640 1 -- 172.21.15.86:0/842309095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f373c1051d0 msgr2=0x7f373c19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.938+0000 7f3741800640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f373c1051d0 0x7f373c19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.938+0000 7f3741800640 1 -- 172.21.15.86:0/842309095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373c1a1f10 con 0x7f373c105b90 2024-04-17T07:11:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.939+0000 7f3741800640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f373c105b90 0x7f373c1a18a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f372c00b3f0 tx=0x7f372c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.939+0000 7f3739600640 1 -- 172.21.15.86:0/842309095 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c015020 con 0x7f373c105b90 2024-04-17T07:11:42.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.939+0000 7f3739600640 1 -- 172.21.15.86:0/842309095 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f372c011ce0 con 0x7f373c105b90 2024-04-17T07:11:42.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.939+0000 7f3739600640 1 -- 172.21.15.86:0/842309095 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c010da0 con 0x7f373c105b90 2024-04-17T07:11:42.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.939+0000 7f3742200640 1 -- 172.21.15.86:0/842309095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373c1a2200 con 0x7f373c105b90 2024-04-17T07:11:42.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.939+0000 7f3742200640 1 -- 172.21.15.86:0/842309095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373c1a2700 con 0x7f373c105b90 2024-04-17T07:11:42.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.941+0000 7f3742200640 1 -- 172.21.15.86:0/842309095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f373c10b3d0 con 0x7f373c105b90 2024-04-17T07:11:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.943+0000 7f3739600640 1 -- 172.21.15.86:0/842309095 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f372c018020 con 0x7f373c105b90 2024-04-17T07:11:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.944+0000 7f3739600640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37200a3bb0 0x7f37200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.944+0000 7f3739600640 1 -- 172.21.15.86:0/842309095 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f372c0cf840 con 0x7f373c105b90 2024-04-17T07:11:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.944+0000 7f3740e00640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37200a3bb0 0x7f37200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.945+0000 7f3740e00640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37200a3bb0 0x7f37200a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3734032420 tx=0x7f3734032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:42.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:42.949+0000 7f3739600640 1 -- 172.21.15.86:0/842309095 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f372c01f020 con 0x7f373c105b90 2024-04-17T07:11:43.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.198+0000 7f3742200640 1 -- 172.21.15.86:0/842309095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f373c02d5d0 con 0x7f373c105b90 2024-04-17T07:11:43.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.199+0000 7f3739600640 1 -- 172.21.15.86:0/842309095 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8815 (secure 0 0 0) 0x7f372c09c6a0 con 0x7f373c105b90 2024-04-17T07:11:43.199 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:43.200 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T07:11:43.200 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 11 2024-04-17T07:11:43.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 -- 172.21.15.86:0/842309095 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37200a3bb0 msgr2=0x7f37200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:43.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37200a3bb0 0x7f37200a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3734032420 tx=0x7f3734032340 comp rx=0 tx=0).stop 2024-04-17T07:11:43.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 -- 172.21.15.86:0/842309095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f373c105b90 msgr2=0x7f373c1a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:43.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f373c105b90 0x7f373c1a18a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f372c00b3f0 tx=0x7f372c00b8b0 comp rx=0 tx=0).stop 2024-04-17T07:11:43.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 -- 172.21.15.86:0/842309095 shutdown_connections 2024-04-17T07:11:43.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37200a3bb0 0x7f37200a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f373c105b90 0x7f373c1a18a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f373c1051d0 0x7f373c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 --2- 172.21.15.86:0/842309095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f373c103fe0 0x7f373c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 -- 172.21.15.86:0/842309095 >> 172.21.15.86:0/842309095 conn(0x7f373c0ff570 msgr2=0x7f373c1085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 -- 172.21.15.86:0/842309095 shutdown_connections 2024-04-17T07:11:43.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:43.202+0000 7f3742200640 1 -- 172.21.15.86:0/842309095 wait complete. 2024-04-17T07:11:44.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:44 smithi086 ceph-mon[31733]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:44.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:44 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/842309095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:44 smithi098 ceph-mon[35418]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:44 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/842309095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:44.515 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:11:44 smithi086 systemd[1]: Starting Ceph mds.a for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:11:44.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:44 smithi103 ceph-mon[35495]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:44.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:44 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/842309095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:44.831 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:11:44 smithi086 podman[71139]: 2024-04-17 07:11:44.420559624 +0000 UTC m=+0.037183363 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:11:44.831 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:11:44 smithi086 podman[71139]: 2024-04-17 07:11:44.544737408 +0000 UTC m=+0.161361177 container create f0157b6dc715473c6b19f5b21465f444e49c573c6e7d7634bd0d4c3e67fb4c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-04-17T07:11:45.223 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:45.311 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:11:44 smithi086 podman[71139]: 2024-04-17 07:11:44.840730883 +0000 UTC m=+0.457354676 container init f0157b6dc715473c6b19f5b21465f444e49c573c6e7d7634bd0d4c3e67fb4c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-17T07:11:45.312 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:11:44 smithi086 podman[71139]: 2024-04-17 07:11:44.850806258 +0000 UTC m=+0.467430057 container start f0157b6dc715473c6b19f5b21465f444e49c573c6e7d7634bd0d4c3e67fb4c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:11:45.312 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:11:44 smithi086 ceph-mds[71158]: -- 172.21.15.86:0/3842095971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c130393a40 con 0x55c130366c00 2024-04-17T07:11:45.312 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:11:44 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-a[71154]: starting mds.a at 2024-04-17T07:11:45.312 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:11:44 smithi086 bash[71139]: f0157b6dc715473c6b19f5b21465f444e49c573c6e7d7634bd0d4c3e67fb4c66 2024-04-17T07:11:45.312 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:11:44 smithi086 systemd[1]: Started Ceph mds.a for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:45.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.442+0000 7fa027200640 1 -- 172.21.15.86:0/972095305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa028103980 msgr2=0x7fa02810a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:45.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.442+0000 7fa027200640 1 --2- 172.21.15.86:0/972095305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa028103980 0x7fa02810a210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa020009ca0 tx=0x7fa02002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:11:45.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.442+0000 7fa027200640 1 -- 172.21.15.86:0/972095305 shutdown_connections 2024-04-17T07:11:45.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.442+0000 7fa027200640 1 --2- 172.21.15.86:0/972095305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa028103980 0x7fa02810a210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:45.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.442+0000 7fa027200640 1 --2- 172.21.15.86:0/972095305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa028102fc0 0x7fa028103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:45.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.442+0000 7fa027200640 1 --2- 172.21.15.86:0/972095305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa028101dd0 0x7fa0281021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:45.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.442+0000 7fa027200640 1 -- 172.21.15.86:0/972095305 >> 172.21.15.86:0/972095305 conn(0x7fa0280fd340 msgr2=0x7fa0280ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:45.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.442+0000 7fa027200640 1 -- 172.21.15.86:0/972095305 shutdown_connections 2024-04-17T07:11:45.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.442+0000 7fa027200640 1 -- 172.21.15.86:0/972095305 wait complete. 2024-04-17T07:11:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.443+0000 7fa027200640 1 Processor -- start 2024-04-17T07:11:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.443+0000 7fa027200640 1 -- start start 2024-04-17T07:11:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.443+0000 7fa027200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa028101dd0 0x7fa028075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.443+0000 7fa027200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa028102fc0 0x7fa028074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa027200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa028103980 0x7fa028074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa027200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa02810d900 con 0x7fa028101dd0 2024-04-17T07:11:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa027200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa02810d780 con 0x7fa028103980 2024-04-17T07:11:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa027200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa02810da80 con 0x7fa028102fc0 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa028101dd0 0x7fa028075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa028101dd0 0x7fa028075cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42454/0 (socket says 172.21.15.86:42454) 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa025e00640 1 -- 172.21.15.86:0/1347746682 learned_addr learned my addr 172.21.15.86:0/1347746682 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa026800640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa028103980 0x7fa028074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa025400640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa028102fc0 0x7fa028074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa025e00640 1 -- 172.21.15.86:0/1347746682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa028102fc0 msgr2=0x7fa028074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa025e00640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa028102fc0 0x7fa028074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa025e00640 1 -- 172.21.15.86:0/1347746682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa028103980 msgr2=0x7fa028074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa025e00640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa028103980 0x7fa028074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.444+0000 7fa025e00640 1 -- 172.21.15.86:0/1347746682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa028075150 con 0x7fa028101dd0 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.445+0000 7fa025e00640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa028101dd0 0x7fa028075cc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa01800d950 tx=0x7fa01800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.445+0000 7fa01ea00640 1 -- 172.21.15.86:0/1347746682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018017070 con 0x7fa028101dd0 2024-04-17T07:11:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.445+0000 7fa01ea00640 1 -- 172.21.15.86:0/1347746682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa018009c30 con 0x7fa028101dd0 2024-04-17T07:11:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.445+0000 7fa01ea00640 1 -- 172.21.15.86:0/1347746682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01800f620 con 0x7fa028101dd0 2024-04-17T07:11:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.445+0000 7fa027200640 1 -- 172.21.15.86:0/1347746682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0281a2170 con 0x7fa028101dd0 2024-04-17T07:11:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.445+0000 7fa027200640 1 -- 172.21.15.86:0/1347746682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0281a2640 con 0x7fa028101dd0 2024-04-17T07:11:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.446+0000 7fa027200640 1 -- 172.21.15.86:0/1347746682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa028109210 con 0x7fa028101dd0 2024-04-17T07:11:45.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.448+0000 7fa01ea00640 1 -- 172.21.15.86:0/1347746682 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa01801a070 con 0x7fa028101dd0 2024-04-17T07:11:45.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.449+0000 7fa01ea00640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa00c0a3bb0 0x7fa00c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:45.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.450+0000 7fa01ea00640 1 -- 172.21.15.86:0/1347746682 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa0180cfb20 con 0x7fa028101dd0 2024-04-17T07:11:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.450+0000 7fa025400640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa00c0a3bb0 0x7fa00c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.450+0000 7fa025400640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa00c0a3bb0 0x7fa00c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa014005fd0 tx=0x7fa014005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.454+0000 7fa01ea00640 1 -- 172.21.15.86:0/1347746682 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa018009de0 con 0x7fa028101dd0 2024-04-17T07:11:45.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.697+0000 7fa027200640 1 -- 172.21.15.86:0/1347746682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa02802d5d0 con 0x7fa028101dd0 2024-04-17T07:11:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.699+0000 7fa01ea00640 1 -- 172.21.15.86:0/1347746682 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9641 (secure 0 0 0) 0x7fa01809c900 con 0x7fa028101dd0 2024-04-17T07:11:45.699 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:45.699 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T07:11:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 12 2024-04-17T07:11:45.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 -- 172.21.15.86:0/1347746682 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa00c0a3bb0 msgr2=0x7fa00c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:45.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa00c0a3bb0 0x7fa00c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa014005fd0 tx=0x7fa014005850 comp rx=0 tx=0).stop 2024-04-17T07:11:45.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 -- 172.21.15.86:0/1347746682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa028101dd0 msgr2=0x7fa028075cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:45.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa028101dd0 0x7fa028075cc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa01800d950 tx=0x7fa01800de10 comp rx=0 tx=0).stop 2024-04-17T07:11:45.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 -- 172.21.15.86:0/1347746682 shutdown_connections 2024-04-17T07:11:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa00c0a3bb0 0x7fa00c0a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa028103980 0x7fa028074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa028102fc0 0x7fa028074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 --2- 172.21.15.86:0/1347746682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa028101dd0 0x7fa028075cc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 -- 172.21.15.86:0/1347746682 >> 172.21.15.86:0/1347746682 conn(0x7fa0280fd340 msgr2=0x7fa0281063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 -- 172.21.15.86:0/1347746682 shutdown_connections 2024-04-17T07:11:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:45.702+0000 7fa027200640 1 -- 172.21.15.86:0/1347746682 wait complete. 2024-04-17T07:11:46.307 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:46.307 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:46.307 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:46.307 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.86:6840/4256331668,v1:172.21.15.86:6841/4256331668] up:boot 2024-04-17T07:11:46.307 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: fsmap 11 up:standby 2024-04-17T07:11:46.307 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T07:11:46.307 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:46.307 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:46.308 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:46.308 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:46.308 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: Deploying daemon mds.b on smithi098 2024-04-17T07:11:46.308 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:46 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1347746682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.86:6840/4256331668,v1:172.21.15.86:6841/4256331668] up:boot 2024-04-17T07:11:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: fsmap 11 up:standby 2024-04-17T07:11:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T07:11:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: Deploying daemon mds.b on smithi098 2024-04-17T07:11:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:46 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1347746682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.86:6840/4256331668,v1:172.21.15.86:6841/4256331668] up:boot 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: fsmap 11 up:standby 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: Deploying daemon mds.b on smithi098 2024-04-17T07:11:46.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:46 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1347746682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:47.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:47.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.942+0000 7f5f6e400640 1 -- 172.21.15.86:0/1084665574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68101e00 msgr2=0x7f5f68102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:47.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.942+0000 7f5f6e400640 1 --2- 172.21.15.86:0/1084665574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68101e00 0x7f5f68102220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009900 tx=0x7f5f5c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:47.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.942+0000 7f5f6e400640 1 -- 172.21.15.86:0/1084665574 shutdown_connections 2024-04-17T07:11:47.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.942+0000 7f5f6e400640 1 --2- 172.21.15.86:0/1084665574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f681039b0 0x7f5f6810a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:47.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.942+0000 7f5f6e400640 1 --2- 172.21.15.86:0/1084665574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f68102ff0 0x7f5f68103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:47.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.942+0000 7f5f6e400640 1 --2- 172.21.15.86:0/1084665574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68101e00 0x7f5f68102220 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.942+0000 7f5f6e400640 1 -- 172.21.15.86:0/1084665574 >> 172.21.15.86:0/1084665574 conn(0x7f5f680fd370 msgr2=0x7f5f680ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.942+0000 7f5f6e400640 1 -- 172.21.15.86:0/1084665574 shutdown_connections 2024-04-17T07:11:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.943+0000 7f5f6e400640 1 -- 172.21.15.86:0/1084665574 wait complete. 2024-04-17T07:11:47.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.944+0000 7f5f6e400640 1 Processor -- start 2024-04-17T07:11:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.944+0000 7f5f6e400640 1 -- start start 2024-04-17T07:11:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.944+0000 7f5f6e400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f68101e00 0x7f5f6819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.944+0000 7f5f6e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68102ff0 0x7f5f6819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.944+0000 7f5f6e400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f681039b0 0x7f5f681a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.944+0000 7f5f6e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6810d9d0 con 0x7f5f68102ff0 2024-04-17T07:11:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.944+0000 7f5f6e400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6810d850 con 0x7f5f68101e00 2024-04-17T07:11:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.944+0000 7f5f6e400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6810db50 con 0x7f5f681039b0 2024-04-17T07:11:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.944+0000 7f5f67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68102ff0 0x7f5f6819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f67e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68102ff0 0x7f5f6819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33762/0 (socket says 172.21.15.86:33762) 2024-04-17T07:11:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f67e00640 1 -- 172.21.15.86:0/4257383598 learned_addr learned my addr 172.21.15.86:0/4257383598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f6d000640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f68101e00 0x7f5f6819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f6da00640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f681039b0 0x7f5f681a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f67e00640 1 -- 172.21.15.86:0/4257383598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f681039b0 msgr2=0x7f5f681a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f67e00640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f681039b0 0x7f5f681a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f67e00640 1 -- 172.21.15.86:0/4257383598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f68101e00 msgr2=0x7f5f6819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f67e00640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f68101e00 0x7f5f6819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f67e00640 1 -- 172.21.15.86:0/4257383598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f681a1ec0 con 0x7f5f68102ff0 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f67e00640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68102ff0 0x7f5f6819a830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400c920 tx=0x7f5f5400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f65600640 1 -- 172.21.15.86:0/4257383598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54007c70 con 0x7f5f68102ff0 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.945+0000 7f5f65600640 1 -- 172.21.15.86:0/4257383598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f54007e10 con 0x7f5f68102ff0 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.946+0000 7f5f6e400640 1 -- 172.21.15.86:0/4257383598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f681a21b0 con 0x7f5f68102ff0 2024-04-17T07:11:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.946+0000 7f5f65600640 1 -- 172.21.15.86:0/4257383598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54018820 con 0x7f5f68102ff0 2024-04-17T07:11:47.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.946+0000 7f5f6e400640 1 -- 172.21.15.86:0/4257383598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681a27d0 con 0x7f5f68102ff0 2024-04-17T07:11:47.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.947+0000 7f5f6e400640 1 -- 172.21.15.86:0/4257383598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f681a2a90 con 0x7f5f68102ff0 2024-04-17T07:11:47.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.949+0000 7f5f65600640 1 -- 172.21.15.86:0/4257383598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5f540189c0 con 0x7f5f68102ff0 2024-04-17T07:11:47.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.951+0000 7f5f65600640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f500a3c30 0x7f5f500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:47.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.951+0000 7f5f65600640 1 -- 172.21.15.86:0/4257383598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5f540d0370 con 0x7f5f68102ff0 2024-04-17T07:11:47.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.951+0000 7f5f6d000640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f500a3c30 0x7f5f500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:47.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.952+0000 7f5f6d000640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f500a3c30 0x7f5f500a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c02c240 tx=0x7f5f5c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:47.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:47.953+0000 7f5f65600640 1 -- 172.21.15.86:0/4257383598 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5f54014030 con 0x7f5f68102ff0 2024-04-17T07:11:48.141 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:48 smithi098 ceph-mon[35418]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:48.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.197+0000 7f5f6e400640 1 -- 172.21.15.86:0/4257383598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f681a2c60 con 0x7f5f68102ff0 2024-04-17T07:11:48.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.199+0000 7f5f65600640 1 -- 172.21.15.86:0/4257383598 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9641 (secure 0 0 0) 0x7f5f5409d260 con 0x7f5f68102ff0 2024-04-17T07:11:48.202 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:48.202 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-17T07:11:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 12 2024-04-17T07:11:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 -- 172.21.15.86:0/4257383598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f500a3c30 msgr2=0x7f5f500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f500a3c30 0x7f5f500a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c02c240 tx=0x7f5f5c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:11:48.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 -- 172.21.15.86:0/4257383598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68102ff0 msgr2=0x7f5f6819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:48.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68102ff0 0x7f5f6819a830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400c920 tx=0x7f5f5400cde0 comp rx=0 tx=0).stop 2024-04-17T07:11:48.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 -- 172.21.15.86:0/4257383598 shutdown_connections 2024-04-17T07:11:48.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f500a3c30 0x7f5f500a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:48.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f681039b0 0x7f5f681a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:48.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f68102ff0 0x7f5f6819a830 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:48.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 --2- 172.21.15.86:0/4257383598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f68101e00 0x7f5f6819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:48.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 -- 172.21.15.86:0/4257383598 >> 172.21.15.86:0/4257383598 conn(0x7f5f680fd370 msgr2=0x7f5f68074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:48.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 -- 172.21.15.86:0/4257383598 shutdown_connections 2024-04-17T07:11:48.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:48.202+0000 7f5f6e400640 1 -- 172.21.15.86:0/4257383598 wait complete. 2024-04-17T07:11:48.503 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:11:48 smithi098 systemd[1]: Starting Ceph mds.b for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:11:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:48 smithi103 ceph-mon[35495]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:48 smithi086 ceph-mon[31733]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:48.858 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:11:48 smithi098 podman[66151]: 2024-04-17 07:11:48.452251688 +0000 UTC m=+0.035331195 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:11:48.859 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:11:48 smithi098 podman[66151]: 2024-04-17 07:11:48.570094964 +0000 UTC m=+0.153174509 container create 28e0e021f6c9f36ec9fc8e2a5b2f9da8016b2fe61e34b064eaeb8d3862f225b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-17T07:11:48.859 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:11:48 smithi098 podman[66151]: 2024-04-17 07:11:48.857935574 +0000 UTC m=+0.441015122 container init 28e0e021f6c9f36ec9fc8e2a5b2f9da8016b2fe61e34b064eaeb8d3862f225b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:11:49.111 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:11:48 smithi098 podman[66151]: 2024-04-17 07:11:48.869221978 +0000 UTC m=+0.452301491 container start 28e0e021f6c9f36ec9fc8e2a5b2f9da8016b2fe61e34b064eaeb8d3862f225b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-17T07:11:49.112 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:11:48 smithi098 ceph-mds[66170]: -- 172.21.15.98:0/2340096386 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb07087a40 con 0x55cb0705a800 2024-04-17T07:11:49.112 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:11:48 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b[66166]: starting mds.b at 2024-04-17T07:11:49.112 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:11:48 smithi098 bash[66151]: 28e0e021f6c9f36ec9fc8e2a5b2f9da8016b2fe61e34b064eaeb8d3862f225b5 2024-04-17T07:11:49.112 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:11:48 smithi098 systemd[1]: Started Ceph mds.b for 1f7ed99a-fc88-11ee-bc91-c7b262605968. 2024-04-17T07:11:49.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:49 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4257383598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:49.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:49 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4257383598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:49 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4257383598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:50.223 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:11:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.452+0000 7feaa9600640 1 -- 172.21.15.86:0/2345329359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa41053b0 msgr2=0x7feaa4105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.452+0000 7feaa9600640 1 --2- 172.21.15.86:0/2345329359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa41053b0 0x7feaa4105830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009860 tx=0x7fea8c02bb70 comp rx=0 tx=0).stop 2024-04-17T07:11:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.452+0000 7feaa9600640 1 -- 172.21.15.86:0/2345329359 shutdown_connections 2024-04-17T07:11:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.452+0000 7feaa9600640 1 --2- 172.21.15.86:0/2345329359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaa4105d70 0x7feaa410c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.452+0000 7feaa9600640 1 --2- 172.21.15.86:0/2345329359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa41053b0 0x7feaa4105830 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.452+0000 7feaa9600640 1 --2- 172.21.15.86:0/2345329359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa41041c0 0x7feaa41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.453+0000 7feaa9600640 1 -- 172.21.15.86:0/2345329359 >> 172.21.15.86:0/2345329359 conn(0x7feaa40ff750 msgr2=0x7feaa4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:50.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.453+0000 7feaa9600640 1 -- 172.21.15.86:0/2345329359 shutdown_connections 2024-04-17T07:11:50.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.453+0000 7feaa9600640 1 -- 172.21.15.86:0/2345329359 wait complete. 2024-04-17T07:11:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa9600640 1 Processor -- start 2024-04-17T07:11:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa9600640 1 -- start start 2024-04-17T07:11:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa9600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaa41041c0 0x7feaa419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa9600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa41053b0 0x7feaa419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4105d70 0x7feaa41a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feaa410fd00 con 0x7feaa4105d70 2024-04-17T07:11:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa9600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feaa410fb80 con 0x7feaa41041c0 2024-04-17T07:11:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa9600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feaa410fe80 con 0x7feaa41053b0 2024-04-17T07:11:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4105d70 0x7feaa41a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaa41041c0 0x7feaa419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7fea9a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa41053b0 0x7feaa419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaa41041c0 0x7feaa419a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:34038/0 (socket says 172.21.15.86:34038) 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4105d70 0x7feaa41a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33786/0 (socket says 172.21.15.86:33786) 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.454+0000 7feaa2a00640 1 -- 172.21.15.86:0/868898793 learned_addr learned my addr 172.21.15.86:0/868898793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7feaa3400640 1 -- 172.21.15.86:0/868898793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa41053b0 msgr2=0x7feaa419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7feaa3400640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa41053b0 0x7feaa419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7feaa3400640 1 -- 172.21.15.86:0/868898793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaa41041c0 msgr2=0x7feaa419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7feaa3400640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaa41041c0 0x7feaa419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7feaa3400640 1 -- 172.21.15.86:0/868898793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa41a1fb0 con 0x7feaa4105d70 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7fea9a000640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa41053b0 0x7feaa419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7feaa3400640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4105d70 0x7feaa41a1940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00d680 tx=0x7fea9c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7fea9be00640 1 -- 172.21.15.86:0/868898793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c017070 con 0x7feaa4105d70 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7fea9be00640 1 -- 172.21.15.86:0/868898793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea9c009820 con 0x7feaa4105d70 2024-04-17T07:11:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7feaa9600640 1 -- 172.21.15.86:0/868898793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa41a22a0 con 0x7feaa4105d70 2024-04-17T07:11:50.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.455+0000 7fea9be00640 1 -- 172.21.15.86:0/868898793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c004cf0 con 0x7feaa4105d70 2024-04-17T07:11:50.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.456+0000 7feaa9600640 1 -- 172.21.15.86:0/868898793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa41a27c0 con 0x7feaa4105d70 2024-04-17T07:11:50.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.457+0000 7feaa9600640 1 -- 172.21.15.86:0/868898793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa410b4e0 con 0x7feaa4105d70 2024-04-17T07:11:50.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.459+0000 7fea9be00640 1 -- 172.21.15.86:0/868898793 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fea9c0099d0 con 0x7feaa4105d70 2024-04-17T07:11:50.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.460+0000 7fea9be00640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea880a3bb0 0x7fea880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:50.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.460+0000 7fea9be00640 1 -- 172.21.15.86:0/868898793 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fea9c0cf880 con 0x7feaa4105d70 2024-04-17T07:11:50.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.460+0000 7feaa2a00640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea880a3bb0 0x7fea880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.461+0000 7feaa2a00640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea880a3bb0 0x7fea880a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea9400e510 tx=0x7fea94010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.465+0000 7fea9be00640 1 -- 172.21.15.86:0/868898793 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fea9c09c010 con 0x7feaa4105d70 2024-04-17T07:11:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:50 smithi098 ceph-mon[35418]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:50 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] up:boot 2024-04-17T07:11:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:50 smithi098 ceph-mon[35418]: fsmap 12 up:standby 2024-04-17T07:11:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T07:11:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:11:50.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:50 smithi103 ceph-mon[35495]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:50.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:50 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] up:boot 2024-04-17T07:11:50.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:50 smithi103 ceph-mon[35495]: fsmap 12 up:standby 2024-04-17T07:11:50.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T07:11:50.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:11:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:50 smithi086 ceph-mon[31733]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:50 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] up:boot 2024-04-17T07:11:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:50 smithi086 ceph-mon[31733]: fsmap 12 up:standby 2024-04-17T07:11:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-17T07:11:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:50.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:11:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.725+0000 7feaa9600640 1 -- 172.21.15.86:0/868898793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaa4062b60 con 0x7feaa4105d70 2024-04-17T07:11:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.725+0000 7fea9be00640 1 -- 172.21.15.86:0/868898793 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10464 (secure 0 0 0) 0x7fea9c0a0310 con 0x7feaa4105d70 2024-04-17T07:11:50.726 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:50.726 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-17T07:11:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 13 2024-04-17T07:11:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 -- 172.21.15.86:0/868898793 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea880a3bb0 msgr2=0x7fea880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea880a3bb0 0x7fea880a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea9400e510 tx=0x7fea94010040 comp rx=0 tx=0).stop 2024-04-17T07:11:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 -- 172.21.15.86:0/868898793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4105d70 msgr2=0x7feaa41a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4105d70 0x7feaa41a1940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00d680 tx=0x7fea9c00db40 comp rx=0 tx=0).stop 2024-04-17T07:11:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 -- 172.21.15.86:0/868898793 shutdown_connections 2024-04-17T07:11:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea880a3bb0 0x7fea880a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa4105d70 0x7feaa41a1940 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaa41053b0 0x7feaa419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 --2- 172.21.15.86:0/868898793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaa41041c0 0x7feaa419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 -- 172.21.15.86:0/868898793 >> 172.21.15.86:0/868898793 conn(0x7feaa40ff750 msgr2=0x7feaa41086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 -- 172.21.15.86:0/868898793 shutdown_connections 2024-04-17T07:11:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.728+0000 7feaa9600640 1 -- 172.21.15.86:0/868898793 wait complete. 2024-04-17T07:11:50.746 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:11:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.942+0000 7f075e600640 1 -- 172.21.15.86:0/2530068331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0758102ff0 msgr2=0x7f0758103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.942+0000 7f075e600640 1 --2- 172.21.15.86:0/2530068331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0758102ff0 0x7f0758103470 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0744009900 tx=0x7f074402bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.942+0000 7f075e600640 1 -- 172.21.15.86:0/2530068331 shutdown_connections 2024-04-17T07:11:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.942+0000 7f075e600640 1 --2- 172.21.15.86:0/2530068331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07581039b0 0x7f075810a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.942+0000 7f075e600640 1 --2- 172.21.15.86:0/2530068331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0758102ff0 0x7f0758103470 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.942+0000 7f075e600640 1 --2- 172.21.15.86:0/2530068331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0758101e00 0x7f0758102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.942+0000 7f075e600640 1 -- 172.21.15.86:0/2530068331 >> 172.21.15.86:0/2530068331 conn(0x7f07580fd370 msgr2=0x7f07580ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.942+0000 7f075e600640 1 -- 172.21.15.86:0/2530068331 shutdown_connections 2024-04-17T07:11:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.942+0000 7f075e600640 1 -- 172.21.15.86:0/2530068331 wait complete. 2024-04-17T07:11:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.944+0000 7f075e600640 1 Processor -- start 2024-04-17T07:11:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.944+0000 7f075e600640 1 -- start start 2024-04-17T07:11:50.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.944+0000 7f075e600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0758101e00 0x7f0758198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:50.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.944+0000 7f075e600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0758102ff0 0x7f0758198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07581039b0 0x7f075819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f075810d980 con 0x7f07581039b0 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075e600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f075810d800 con 0x7f0758101e00 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075e600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f075810db00 con 0x7f0758102ff0 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07581039b0 0x7f075819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07581039b0 0x7f075819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33806/0 (socket says 172.21.15.86:33806) 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075dc00640 1 -- 172.21.15.86:0/2532697512 learned_addr learned my addr 172.21.15.86:0/2532697512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f0757e00640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0758102ff0 0x7f0758198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075d200640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0758101e00 0x7f0758198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075dc00640 1 -- 172.21.15.86:0/2532697512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0758102ff0 msgr2=0x7f0758198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075dc00640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0758102ff0 0x7f0758198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075dc00640 1 -- 172.21.15.86:0/2532697512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0758101e00 msgr2=0x7f0758198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075dc00640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0758101e00 0x7f0758198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.945+0000 7f075dc00640 1 -- 172.21.15.86:0/2532697512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075819fc40 con 0x7f07581039b0 2024-04-17T07:11:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.946+0000 7f075dc00640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07581039b0 0x7f075819f5d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f075000eaa0 tx=0x7f075000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.946+0000 7f0755600640 1 -- 172.21.15.86:0/2532697512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075000cdb0 con 0x7f07581039b0 2024-04-17T07:11:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.946+0000 7f0755600640 1 -- 172.21.15.86:0/2532697512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0750005020 con 0x7f07581039b0 2024-04-17T07:11:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.946+0000 7f0755600640 1 -- 172.21.15.86:0/2532697512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07500116d0 con 0x7f07581039b0 2024-04-17T07:11:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.946+0000 7f075e600640 1 -- 172.21.15.86:0/2532697512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075819ff30 con 0x7f07581039b0 2024-04-17T07:11:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.946+0000 7f075e600640 1 -- 172.21.15.86:0/2532697512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07581a0340 con 0x7f07581039b0 2024-04-17T07:11:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.948+0000 7f075e600640 1 -- 172.21.15.86:0/2532697512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0758109120 con 0x7f07581039b0 2024-04-17T07:11:50.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.949+0000 7f0755600640 1 -- 172.21.15.86:0/2532697512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f07500051d0 con 0x7f07581039b0 2024-04-17T07:11:50.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.951+0000 7f0755600640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f07400a3bb0 0x7f07400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:50.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.951+0000 7f0755600640 1 -- 172.21.15.86:0/2532697512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f07500d0020 con 0x7f07581039b0 2024-04-17T07:11:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.951+0000 7f075d200640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f07400a3bb0 0x7f07400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.952+0000 7f075d200640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f07400a3bb0 0x7f07400a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0758102e50 tx=0x7f074c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:50.956+0000 7f0755600640 1 -- 172.21.15.86:0/2532697512 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0750016030 con 0x7f07581039b0 2024-04-17T07:11:51.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.164+0000 7f075e600640 1 -- 172.21.15.86:0/2532697512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f075802d7c0 con 0x7f07581039b0 2024-04-17T07:11:51.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.166+0000 7f0755600640 1 -- 172.21.15.86:0/2532697512 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18385 (secure 0 0 0) 0x7f075009c850 con 0x7f07581039b0 2024-04-17T07:11:51.167 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:51.167 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:10:49.137829+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:11:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 -- 172.21.15.86:0/2532697512 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f07400a3bb0 msgr2=0x7f07400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f07400a3bb0 0x7f07400a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0758102e50 tx=0x7f074c00b390 comp rx=0 tx=0).stop 2024-04-17T07:11:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 -- 172.21.15.86:0/2532697512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07581039b0 msgr2=0x7f075819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07581039b0 0x7f075819f5d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f075000eaa0 tx=0x7f075000ef60 comp rx=0 tx=0).stop 2024-04-17T07:11:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 -- 172.21.15.86:0/2532697512 shutdown_connections 2024-04-17T07:11:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f07400a3bb0 0x7f07400a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07581039b0 0x7f075819f5d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0758102ff0 0x7f0758198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 --2- 172.21.15.86:0/2532697512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0758101e00 0x7f0758198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 -- 172.21.15.86:0/2532697512 >> 172.21.15.86:0/2532697512 conn(0x7f07580fd370 msgr2=0x7f07581062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 -- 172.21.15.86:0/2532697512 shutdown_connections 2024-04-17T07:11:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.169+0000 7f075e600640 1 -- 172.21.15.86:0/2532697512 wait complete. 2024-04-17T07:11:51.187 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T07:11:51.187 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:11:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:51 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/868898793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:51 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/868898793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.384+0000 7fe58e600640 1 -- 172.21.15.86:0/1570814372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe588105b70 msgr2=0x7fe58810c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.385+0000 7fe58e600640 1 --2- 172.21.15.86:0/1570814372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe588105b70 0x7fe58810c400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe580009ca0 tx=0x7fe58002c100 comp rx=0 tx=0).stop 2024-04-17T07:11:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.385+0000 7fe58e600640 1 -- 172.21.15.86:0/1570814372 shutdown_connections 2024-04-17T07:11:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.385+0000 7fe58e600640 1 --2- 172.21.15.86:0/1570814372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe588105b70 0x7fe58810c400 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.385+0000 7fe58e600640 1 --2- 172.21.15.86:0/1570814372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5881051b0 0x7fe588105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.385+0000 7fe58e600640 1 --2- 172.21.15.86:0/1570814372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe588103fc0 0x7fe5881043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.385+0000 7fe58e600640 1 -- 172.21.15.86:0/1570814372 >> 172.21.15.86:0/1570814372 conn(0x7fe5880ff530 msgr2=0x7fe588101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.385+0000 7fe58e600640 1 -- 172.21.15.86:0/1570814372 shutdown_connections 2024-04-17T07:11:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.385+0000 7fe58e600640 1 -- 172.21.15.86:0/1570814372 wait complete. 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe58e600640 1 Processor -- start 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe58e600640 1 -- start start 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe58e600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe588103fc0 0x7fe58819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe58e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5881051b0 0x7fe58819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe58e600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe588105b70 0x7fe5881a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe58e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe58810fb00 con 0x7fe5881051b0 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe58e600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe58810f980 con 0x7fe588105b70 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe58e600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe58810fc80 con 0x7fe588103fc0 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe587e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5881051b0 0x7fe58819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe587e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5881051b0 0x7fe58819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33820/0 (socket says 172.21.15.86:33820) 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.386+0000 7fe587e00640 1 -- 172.21.15.86:0/279209067 learned_addr learned my addr 172.21.15.86:0/279209067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe58d200640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe588103fc0 0x7fe58819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe58dc00640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe588105b70 0x7fe5881a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe587e00640 1 -- 172.21.15.86:0/279209067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe588103fc0 msgr2=0x7fe58819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe587e00640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe588103fc0 0x7fe58819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe587e00640 1 -- 172.21.15.86:0/279209067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe588105b70 msgr2=0x7fe5881a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe587e00640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe588105b70 0x7fe5881a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe587e00640 1 -- 172.21.15.86:0/279209067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5881a1cd0 con 0x7fe5881051b0 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe587e00640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5881051b0 0x7fe58819a5f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c920 tx=0x7fe57400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe585600640 1 -- 172.21.15.86:0/279209067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574007c70 con 0x7fe5881051b0 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe585600640 1 -- 172.21.15.86:0/279209067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe574007e10 con 0x7fe5881051b0 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe58e600640 1 -- 172.21.15.86:0/279209067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5881a1fc0 con 0x7fe5881051b0 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe585600640 1 -- 172.21.15.86:0/279209067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574018820 con 0x7fe5881051b0 2024-04-17T07:11:51.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.387+0000 7fe58e600640 1 -- 172.21.15.86:0/279209067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5881a2480 con 0x7fe5881051b0 2024-04-17T07:11:51.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.388+0000 7fe58e600640 1 -- 172.21.15.86:0/279209067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe58810b210 con 0x7fe5881051b0 2024-04-17T07:11:51.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.391+0000 7fe585600640 1 -- 172.21.15.86:0/279209067 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe5740189c0 con 0x7fe5881051b0 2024-04-17T07:11:51.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.393+0000 7fe585600640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe5700a3bb0 0x7fe5700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:51.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.393+0000 7fe585600640 1 -- 172.21.15.86:0/279209067 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe5740d0160 con 0x7fe5881051b0 2024-04-17T07:11:51.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.393+0000 7fe58d200640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe5700a3bb0 0x7fe5700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:51.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.394+0000 7fe58d200640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe5700a3bb0 0x7fe5700a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0058c0 tx=0x7fe57c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.397+0000 7fe585600640 1 -- 172.21.15.86:0/279209067 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe574014030 con 0x7fe5881051b0 2024-04-17T07:11:51.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:51 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/868898793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:11:51.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.602+0000 7fe58e600640 1 -- 172.21.15.86:0/279209067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe588062c60 con 0x7fe5881051b0 2024-04-17T07:11:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.603+0000 7fe585600640 1 -- 172.21.15.86:0/279209067 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fe57400de20 con 0x7fe5881051b0 2024-04-17T07:11:51.603 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:11:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 -- 172.21.15.86:0/279209067 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe5700a3bb0 msgr2=0x7fe5700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe5700a3bb0 0x7fe5700a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0058c0 tx=0x7fe57c005850 comp rx=0 tx=0).stop 2024-04-17T07:11:51.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 -- 172.21.15.86:0/279209067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5881051b0 msgr2=0x7fe58819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5881051b0 0x7fe58819a5f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe57400c920 tx=0x7fe57400cde0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 -- 172.21.15.86:0/279209067 shutdown_connections 2024-04-17T07:11:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe5700a3bb0 0x7fe5700a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe588105b70 0x7fe5881a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5881051b0 0x7fe58819a5f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 --2- 172.21.15.86:0/279209067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe588103fc0 0x7fe58819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 -- 172.21.15.86:0/279209067 >> 172.21.15.86:0/279209067 conn(0x7fe5880ff530 msgr2=0x7fe5881083e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.605+0000 7fe58e600640 1 -- 172.21.15.86:0/279209067 shutdown_connections 2024-04-17T07:11:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.606+0000 7fe58e600640 1 -- 172.21.15.86:0/279209067 wait complete. 2024-04-17T07:11:51.627 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T07:11:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.815+0000 7facddc00640 1 -- 172.21.15.86:0/3382341304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facd8109e70 msgr2=0x7facd810c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.815+0000 7facddc00640 1 --2- 172.21.15.86:0/3382341304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facd8109e70 0x7facd810c300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7facd0009ca0 tx=0x7facd002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.815+0000 7facddc00640 1 -- 172.21.15.86:0/3382341304 shutdown_connections 2024-04-17T07:11:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.815+0000 7facddc00640 1 --2- 172.21.15.86:0/3382341304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facd8109e70 0x7facd810c300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.815+0000 7facddc00640 1 --2- 172.21.15.86:0/3382341304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd80749c0 0x7facd8074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.815+0000 7facddc00640 1 --2- 172.21.15.86:0/3382341304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8075fa0 0x7facd80743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.815+0000 7facddc00640 1 -- 172.21.15.86:0/3382341304 >> 172.21.15.86:0/3382341304 conn(0x7facd80fd540 msgr2=0x7facd80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.815+0000 7facddc00640 1 -- 172.21.15.86:0/3382341304 shutdown_connections 2024-04-17T07:11:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.815+0000 7facddc00640 1 -- 172.21.15.86:0/3382341304 wait complete. 2024-04-17T07:11:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.816+0000 7facddc00640 1 Processor -- start 2024-04-17T07:11:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.816+0000 7facddc00640 1 -- start start 2024-04-17T07:11:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.816+0000 7facddc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd80749c0 0x7facd819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.816+0000 7facddc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8075fa0 0x7facd819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.816+0000 7facddc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facd8109e70 0x7facd81a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.816+0000 7facddc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facd810fcb0 con 0x7facd8109e70 2024-04-17T07:11:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.816+0000 7facddc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7facd810fb30 con 0x7facd80749c0 2024-04-17T07:11:51.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.816+0000 7facddc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7facd810fe30 con 0x7facd8075fa0 2024-04-17T07:11:51.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facd8109e70 0x7facd81a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:51.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8075fa0 0x7facd819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:51.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facd8109e70 0x7facd81a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33836/0 (socket says 172.21.15.86:33836) 2024-04-17T07:11:51.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd7400640 1 -- 172.21.15.86:0/2730049113 learned_addr learned my addr 172.21.15.86:0/2730049113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:51.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd6a00640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd80749c0 0x7facd819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd7400640 1 -- 172.21.15.86:0/2730049113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8075fa0 msgr2=0x7facd819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd7400640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8075fa0 0x7facd819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd7400640 1 -- 172.21.15.86:0/2730049113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd80749c0 msgr2=0x7facd819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd7400640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd80749c0 0x7facd819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd7400640 1 -- 172.21.15.86:0/2730049113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd81a1fa0 con 0x7facd8109e70 2024-04-17T07:11:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.817+0000 7facd6a00640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd80749c0 0x7facd819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:11:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.818+0000 7facd7400640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facd8109e70 0x7facd81a1930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7facd0032380 tx=0x7facd0032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.818+0000 7faccf400640 1 -- 172.21.15.86:0/2730049113 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0041020 con 0x7facd8109e70 2024-04-17T07:11:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.818+0000 7faccf400640 1 -- 172.21.15.86:0/2730049113 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facd0003e20 con 0x7facd8109e70 2024-04-17T07:11:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.818+0000 7facddc00640 1 -- 172.21.15.86:0/2730049113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd81a2230 con 0x7facd8109e70 2024-04-17T07:11:51.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.818+0000 7faccf400640 1 -- 172.21.15.86:0/2730049113 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0039040 con 0x7facd8109e70 2024-04-17T07:11:51.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.818+0000 7facddc00640 1 -- 172.21.15.86:0/2730049113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd81a2720 con 0x7facd8109e70 2024-04-17T07:11:51.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.819+0000 7facddc00640 1 -- 172.21.15.86:0/2730049113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd8108ba0 con 0x7facd8109e70 2024-04-17T07:11:51.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.823+0000 7faccf400640 1 -- 172.21.15.86:0/2730049113 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7facd0044020 con 0x7facd8109e70 2024-04-17T07:11:51.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.824+0000 7faccf400640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7facbc0a3bb0 0x7facbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:51.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.825+0000 7faccf400640 1 -- 172.21.15.86:0/2730049113 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7facd00f2570 con 0x7facd8109e70 2024-04-17T07:11:51.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.825+0000 7facd6a00640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7facbc0a3bb0 0x7facbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:51.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.825+0000 7faccf400640 1 -- 172.21.15.86:0/2730049113 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7facd004b020 con 0x7facd8109e70 2024-04-17T07:11:51.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:51.826+0000 7facd6a00640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7facbc0a3bb0 0x7facbc0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facc8009c20 tx=0x7facc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:52.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.045+0000 7facddc00640 1 -- 172.21.15.86:0/2730049113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7facd802d7a0 con 0x7facd8109e70 2024-04-17T07:11:52.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.046+0000 7faccf400640 1 -- 172.21.15.86:0/2730049113 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7facd00bf350 con 0x7facd8109e70 2024-04-17T07:11:52.046 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:11:52.046 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-17T07:11:52.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:11:52.046 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-17T07:11:52.046 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-17T07:11:52.046 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:11:52.046 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-17T07:11:52.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.048+0000 7facddc00640 1 -- 172.21.15.86:0/2730049113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7facbc0a3bb0 msgr2=0x7facbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:52.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.048+0000 7facddc00640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7facbc0a3bb0 0x7facbc0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7facc8009c20 tx=0x7facc8005f90 comp rx=0 tx=0).stop 2024-04-17T07:11:52.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.048+0000 7facddc00640 1 -- 172.21.15.86:0/2730049113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facd8109e70 msgr2=0x7facd81a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:52.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.048+0000 7facddc00640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facd8109e70 0x7facd81a1930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7facd0032380 tx=0x7facd0032820 comp rx=0 tx=0).stop 2024-04-17T07:11:52.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.048+0000 7facddc00640 1 -- 172.21.15.86:0/2730049113 shutdown_connections 2024-04-17T07:11:52.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.048+0000 7facddc00640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7facbc0a3bb0 0x7facbc0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:52.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.048+0000 7facddc00640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facd8109e70 0x7facd81a1930 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:52.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.048+0000 7facddc00640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8075fa0 0x7facd819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:52.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.048+0000 7facddc00640 1 --2- 172.21.15.86:0/2730049113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd80749c0 0x7facd819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:52.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.048+0000 7facddc00640 1 -- 172.21.15.86:0/2730049113 >> 172.21.15.86:0/2730049113 conn(0x7facd80fd540 msgr2=0x7facd8105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:52.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.049+0000 7facddc00640 1 -- 172.21.15.86:0/2730049113 shutdown_connections 2024-04-17T07:11:52.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.049+0000 7facddc00640 1 -- 172.21.15.86:0/2730049113 wait complete. 2024-04-17T07:11:52.067 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T07:11:52.068 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T07:11:52.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.268+0000 7fdaed400640 1 -- 172.21.15.86:0/1157429263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae8105d90 msgr2=0x7fdae810c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:52.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.268+0000 7fdaed400640 1 --2- 172.21.15.86:0/1157429263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae8105d90 0x7fdae810c620 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009ca0 tx=0x7fdae002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:11:52.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.268+0000 7fdaed400640 1 -- 172.21.15.86:0/1157429263 shutdown_connections 2024-04-17T07:11:52.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.268+0000 7fdaed400640 1 --2- 172.21.15.86:0/1157429263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae8105d90 0x7fdae810c620 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:52.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.268+0000 7fdaed400640 1 --2- 172.21.15.86:0/1157429263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdae81053d0 0x7fdae8105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:52.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.268+0000 7fdaed400640 1 --2- 172.21.15.86:0/1157429263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae81041e0 0x7fdae8104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:52.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.268+0000 7fdaed400640 1 -- 172.21.15.86:0/1157429263 >> 172.21.15.86:0/1157429263 conn(0x7fdae80ff750 msgr2=0x7fdae8101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:52.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.268+0000 7fdaed400640 1 -- 172.21.15.86:0/1157429263 shutdown_connections 2024-04-17T07:11:52.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.268+0000 7fdaed400640 1 -- 172.21.15.86:0/1157429263 wait complete. 2024-04-17T07:11:52.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.269+0000 7fdaed400640 1 Processor -- start 2024-04-17T07:11:52.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdaed400640 1 -- start start 2024-04-17T07:11:52.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdaed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae81041e0 0x7fdae819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:52.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdaed400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdae81053d0 0x7fdae819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:52.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdaed400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae8105d90 0x7fdae81a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdaed400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810fd20 con 0x7fdae81041e0 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdaed400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810fba0 con 0x7fdae81053d0 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdaed400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810fea0 con 0x7fdae8105d90 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdae6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae81041e0 0x7fdae819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdae6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae81041e0 0x7fdae819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33860/0 (socket says 172.21.15.86:33860) 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdae6800640 1 -- 172.21.15.86:0/4043665697 learned_addr learned my addr 172.21.15.86:0/4043665697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdae5e00640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdae81053d0 0x7fdae819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.270+0000 7fdae7200640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae8105d90 0x7fdae81a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.271+0000 7fdae6800640 1 -- 172.21.15.86:0/4043665697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae8105d90 msgr2=0x7fdae81a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.271+0000 7fdae6800640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae8105d90 0x7fdae81a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.271+0000 7fdae6800640 1 -- 172.21.15.86:0/4043665697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdae81053d0 msgr2=0x7fdae819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.271+0000 7fdae6800640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdae81053d0 0x7fdae819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:52.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.271+0000 7fdae6800640 1 -- 172.21.15.86:0/4043665697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae81a1fd0 con 0x7fdae81041e0 2024-04-17T07:11:52.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.271+0000 7fdae6800640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae81041e0 0x7fdae819a3b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdad800e9c0 tx=0x7fdad800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:52.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.271+0000 7fdadf400640 1 -- 172.21.15.86:0/4043665697 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad800cde0 con 0x7fdae81041e0 2024-04-17T07:11:52.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.271+0000 7fdadf400640 1 -- 172.21.15.86:0/4043665697 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdad80108e0 con 0x7fdae81041e0 2024-04-17T07:11:52.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.271+0000 7fdaed400640 1 -- 172.21.15.86:0/4043665697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae81a22c0 con 0x7fdae81041e0 2024-04-17T07:11:52.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.271+0000 7fdadf400640 1 -- 172.21.15.86:0/4043665697 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8018990 con 0x7fdae81041e0 2024-04-17T07:11:52.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.272+0000 7fdaed400640 1 -- 172.21.15.86:0/4043665697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae81a2860 con 0x7fdae81041e0 2024-04-17T07:11:52.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.273+0000 7fdaed400640 1 -- 172.21.15.86:0/4043665697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae810b520 con 0x7fdae81041e0 2024-04-17T07:11:52.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.275+0000 7fdadf400640 1 -- 172.21.15.86:0/4043665697 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdad801f020 con 0x7fdae81041e0 2024-04-17T07:11:52.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.277+0000 7fdadf400640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdacc0a3c30 0x7fdacc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:52.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.277+0000 7fdadf400640 1 -- 172.21.15.86:0/4043665697 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdad80d0150 con 0x7fdae81041e0 2024-04-17T07:11:52.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.278+0000 7fdae5e00640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdacc0a3c30 0x7fdacc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:52.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.278+0000 7fdae5e00640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdacc0a3c30 0x7fdacc0a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdae819b8c0 tx=0x7fdad4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:52.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.282+0000 7fdadf400640 1 -- 172.21.15.86:0/4043665697 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdad809cf30 con 0x7fdae81041e0 2024-04-17T07:11:52.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:52 smithi086 ceph-mon[31733]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:52.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:52 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2532697512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:11:52.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:52 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/279209067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:11:52.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:52 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:52.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:52 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2730049113' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-17T07:11:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:52.494+0000 7fdaed400640 1 -- 172.21.15.86:0/4043665697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdae802d5d0 con 0x7fdae81041e0 2024-04-17T07:11:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:52 smithi098 ceph-mon[35418]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:52 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2532697512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:11:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:52 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/279209067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:11:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:52 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:52 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2730049113' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-17T07:11:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:52 smithi103 ceph-mon[35495]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:52 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2532697512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:11:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:52 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/279209067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:11:52.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:52 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:52.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:52 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2730049113' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-17T07:11:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.418+0000 7fdadf400640 1 -- 172.21.15.86:0/4043665697 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fdad809c830 con 0x7fdae81041e0 2024-04-17T07:11:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-17T07:11:53.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.421+0000 7fdaed400640 1 -- 172.21.15.86:0/4043665697 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdacc0a3c30 msgr2=0x7fdacc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:53.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.421+0000 7fdaed400640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdacc0a3c30 0x7fdacc0a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdae819b8c0 tx=0x7fdad4005f90 comp rx=0 tx=0).stop 2024-04-17T07:11:53.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.421+0000 7fdaed400640 1 -- 172.21.15.86:0/4043665697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae81041e0 msgr2=0x7fdae819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:53.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.421+0000 7fdaed400640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae81041e0 0x7fdae819a3b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdad800e9c0 tx=0x7fdad800ee80 comp rx=0 tx=0).stop 2024-04-17T07:11:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.422+0000 7fdaed400640 1 -- 172.21.15.86:0/4043665697 shutdown_connections 2024-04-17T07:11:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.422+0000 7fdaed400640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdacc0a3c30 0x7fdacc0a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.422+0000 7fdaed400640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae8105d90 0x7fdae81a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.422+0000 7fdaed400640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdae81053d0 0x7fdae819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.422+0000 7fdaed400640 1 --2- 172.21.15.86:0/4043665697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae81041e0 0x7fdae819a3b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.422+0000 7fdaed400640 1 -- 172.21.15.86:0/4043665697 >> 172.21.15.86:0/4043665697 conn(0x7fdae80ff750 msgr2=0x7fdae81086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:53.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.422+0000 7fdaed400640 1 -- 172.21.15.86:0/4043665697 shutdown_connections 2024-04-17T07:11:53.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.422+0000 7fdaed400640 1 -- 172.21.15.86:0/4043665697 wait complete. 2024-04-17T07:11:53.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-17T07:11:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.647+0000 7fb753200640 1 -- 172.21.15.86:0/1019798310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb754103a40 msgr2=0x7fb75410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.647+0000 7fb753200640 1 --2- 172.21.15.86:0/1019798310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb754103a40 0x7fb75410a360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009ca0 tx=0x7fb74c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:11:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.647+0000 7fb753200640 1 -- 172.21.15.86:0/1019798310 shutdown_connections 2024-04-17T07:11:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.647+0000 7fb753200640 1 --2- 172.21.15.86:0/1019798310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb754103a40 0x7fb75410a360 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.647+0000 7fb753200640 1 --2- 172.21.15.86:0/1019798310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb754103080 0x7fb754103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.647+0000 7fb753200640 1 --2- 172.21.15.86:0/1019798310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754101ea0 0x7fb7541022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.647+0000 7fb753200640 1 -- 172.21.15.86:0/1019798310 >> 172.21.15.86:0/1019798310 conn(0x7fb7540fd460 msgr2=0x7fb7540ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.647+0000 7fb753200640 1 -- 172.21.15.86:0/1019798310 shutdown_connections 2024-04-17T07:11:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.647+0000 7fb753200640 1 -- 172.21.15.86:0/1019798310 wait complete. 2024-04-17T07:11:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.648+0000 7fb753200640 1 Processor -- start 2024-04-17T07:11:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.648+0000 7fb753200640 1 -- start start 2024-04-17T07:11:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.648+0000 7fb753200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb754101ea0 0x7fb754198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.648+0000 7fb753200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754103080 0x7fb754198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.648+0000 7fb753200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb754103a40 0x7fb75419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.648+0000 7fb753200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb75410d9e0 con 0x7fb754103a40 2024-04-17T07:11:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.648+0000 7fb753200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb75410d860 con 0x7fb754101ea0 2024-04-17T07:11:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.648+0000 7fb753200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb75410db60 con 0x7fb754103080 2024-04-17T07:11:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb752800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb754103a40 0x7fb75419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb752800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb754103a40 0x7fb75419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33876/0 (socket says 172.21.15.86:33876) 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb752800640 1 -- 172.21.15.86:0/145746384 learned_addr learned my addr 172.21.15.86:0/145746384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb74be00640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754103080 0x7fb754198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb751e00640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb754101ea0 0x7fb754198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb752800640 1 -- 172.21.15.86:0/145746384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754103080 msgr2=0x7fb754198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb752800640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754103080 0x7fb754198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb752800640 1 -- 172.21.15.86:0/145746384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb754101ea0 msgr2=0x7fb754198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb752800640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb754101ea0 0x7fb754198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb752800640 1 -- 172.21.15.86:0/145746384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75419fc70 con 0x7fb754103a40 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.649+0000 7fb752800640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb754103a40 0x7fb75419f600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb74c02c700 tx=0x7fb74c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.650+0000 7fb74aa00640 1 -- 172.21.15.86:0/145746384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c041020 con 0x7fb754103a40 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.650+0000 7fb753200640 1 -- 172.21.15.86:0/145746384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75419ff00 con 0x7fb754103a40 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.650+0000 7fb74aa00640 1 -- 172.21.15.86:0/145746384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb74c003e20 con 0x7fb754103a40 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.650+0000 7fb753200640 1 -- 172.21.15.86:0/145746384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7541a03c0 con 0x7fb754103a40 2024-04-17T07:11:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.650+0000 7fb74aa00640 1 -- 172.21.15.86:0/145746384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c039040 con 0x7fb754103a40 2024-04-17T07:11:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.651+0000 7fb753200640 1 -- 172.21.15.86:0/145746384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb754109240 con 0x7fb754103a40 2024-04-17T07:11:53.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.654+0000 7fb74aa00640 1 -- 172.21.15.86:0/145746384 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb74c044020 con 0x7fb754103a40 2024-04-17T07:11:53.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.655+0000 7fb74aa00640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7380a3bb0 0x7fb7380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:53.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.655+0000 7fb74aa00640 1 -- 172.21.15.86:0/145746384 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fb74c0f2a20 con 0x7fb754103a40 2024-04-17T07:11:53.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.655+0000 7fb751e00640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7380a3bb0 0x7fb7380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:53.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.656+0000 7fb751e00640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7380a3bb0 0x7fb7380a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb744005fd0 tx=0x7fb744005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:53.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.671+0000 7fb74aa00640 1 -- 172.21.15.86:0/145746384 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb74c04b020 con 0x7fb754103a40 2024-04-17T07:11:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:53 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4043665697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T07:11:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:53 smithi098 ceph-mon[35418]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:53.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:53 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4043665697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T07:11:53.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:53 smithi103 ceph-mon[35495]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:53.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:53 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4043665697' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-17T07:11:53.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:53.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:53 smithi086 ceph-mon[31733]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:53.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:53.883+0000 7fb753200640 1 -- 172.21.15.86:0/145746384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fb75402d5d0 con 0x7fb754103a40 2024-04-17T07:11:54.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.429+0000 7fb74aa00640 1 -- 172.21.15.86:0/145746384 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7fb74c0bf690 con 0x7fb754103a40 2024-04-17T07:11:54.430 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-17T07:11:54.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.433+0000 7fb753200640 1 -- 172.21.15.86:0/145746384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7380a3bb0 msgr2=0x7fb7380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:54.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.433+0000 7fb753200640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7380a3bb0 0x7fb7380a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb744005fd0 tx=0x7fb744005850 comp rx=0 tx=0).stop 2024-04-17T07:11:54.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.433+0000 7fb753200640 1 -- 172.21.15.86:0/145746384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb754103a40 msgr2=0x7fb75419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:54.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.433+0000 7fb753200640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb754103a40 0x7fb75419f600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb74c02c700 tx=0x7fb74c0328a0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.434+0000 7fb753200640 1 -- 172.21.15.86:0/145746384 shutdown_connections 2024-04-17T07:11:54.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.434+0000 7fb753200640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7380a3bb0 0x7fb7380a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.434+0000 7fb753200640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb754103a40 0x7fb75419f600 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.434+0000 7fb753200640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754103080 0x7fb754198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.434+0000 7fb753200640 1 --2- 172.21.15.86:0/145746384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb754101ea0 0x7fb754198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.434+0000 7fb753200640 1 -- 172.21.15.86:0/145746384 >> 172.21.15.86:0/145746384 conn(0x7fb7540fd460 msgr2=0x7fb754106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:54.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.434+0000 7fb753200640 1 -- 172.21.15.86:0/145746384 shutdown_connections 2024-04-17T07:11:54.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.434+0000 7fb753200640 1 -- 172.21.15.86:0/145746384 wait complete. 2024-04-17T07:11:54.477 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T07:11:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.661+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1125282209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ac109db0 msgr2=0x7ff8ac10c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.661+0000 7ff8b2a00640 1 --2- 172.21.15.86:0/1125282209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ac109db0 0x7ff8ac10c240 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009ca0 tx=0x7ff8a402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.662+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1125282209 shutdown_connections 2024-04-17T07:11:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.662+0000 7ff8b2a00640 1 --2- 172.21.15.86:0/1125282209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ac109db0 0x7ff8ac10c240 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.662+0000 7ff8b2a00640 1 --2- 172.21.15.86:0/1125282209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8ac0749c0 0x7ff8ac074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.662+0000 7ff8b2a00640 1 --2- 172.21.15.86:0/1125282209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac075fa0 0x7ff8ac0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.662+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1125282209 >> 172.21.15.86:0/1125282209 conn(0x7ff8ac0fd460 msgr2=0x7ff8ac0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.662+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1125282209 shutdown_connections 2024-04-17T07:11:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.662+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1125282209 wait complete. 2024-04-17T07:11:54.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.663+0000 7ff8b2a00640 1 Processor -- start 2024-04-17T07:11:54.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.663+0000 7ff8b2a00640 1 -- start start 2024-04-17T07:11:54.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.663+0000 7ff8b2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac0749c0 0x7ff8ac19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:54.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.663+0000 7ff8b2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ac075fa0 0x7ff8ac19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:54.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.663+0000 7ff8b2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8ac109db0 0x7ff8ac1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:54.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.663+0000 7ff8b2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac10fbb0 con 0x7ff8ac075fa0 2024-04-17T07:11:54.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.663+0000 7ff8b2a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac10fa30 con 0x7ff8ac0749c0 2024-04-17T07:11:54.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.663+0000 7ff8b2a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac10fd30 con 0x7ff8ac109db0 2024-04-17T07:11:54.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ac075fa0 0x7ff8ac19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ac075fa0 0x7ff8ac19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33892/0 (socket says 172.21.15.86:33892) 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b0c00640 1 -- 172.21.15.86:0/1723455412 learned_addr learned my addr 172.21.15.86:0/1723455412 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b1600640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac0749c0 0x7ff8ac19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b2000640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8ac109db0 0x7ff8ac1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b0c00640 1 -- 172.21.15.86:0/1723455412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8ac109db0 msgr2=0x7ff8ac1a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b0c00640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8ac109db0 0x7ff8ac1a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b0c00640 1 -- 172.21.15.86:0/1723455412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac0749c0 msgr2=0x7ff8ac19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b0c00640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac0749c0 0x7ff8ac19a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b0c00640 1 -- 172.21.15.86:0/1723455412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac1a1e40 con 0x7ff8ac075fa0 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.664+0000 7ff8b0c00640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ac075fa0 0x7ff8ac19a760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff89800d770 tx=0x7ff89800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.665+0000 7ff8aa000640 1 -- 172.21.15.86:0/1723455412 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898017070 con 0x7ff8ac075fa0 2024-04-17T07:11:54.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.665+0000 7ff8aa000640 1 -- 172.21.15.86:0/1723455412 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8980054d0 con 0x7ff8ac075fa0 2024-04-17T07:11:54.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.665+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1723455412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac1a2130 con 0x7ff8ac075fa0 2024-04-17T07:11:54.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.665+0000 7ff8aa000640 1 -- 172.21.15.86:0/1723455412 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898004cf0 con 0x7ff8ac075fa0 2024-04-17T07:11:54.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.665+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1723455412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac1a2630 con 0x7ff8ac075fa0 2024-04-17T07:11:54.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.666+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1723455412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ac108ae0 con 0x7ff8ac075fa0 2024-04-17T07:11:54.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.669+0000 7ff8aa000640 1 -- 172.21.15.86:0/1723455412 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff898004e90 con 0x7ff8ac075fa0 2024-04-17T07:11:54.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.670+0000 7ff8aa000640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff8940a3bb0 0x7ff8940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:54.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.670+0000 7ff8aa000640 1 -- 172.21.15.86:0/1723455412 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7ff8980cfd60 con 0x7ff8ac075fa0 2024-04-17T07:11:54.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.670+0000 7ff8b1600640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff8940a3bb0 0x7ff8940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:54.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.671+0000 7ff8b1600640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff8940a3bb0 0x7ff8940a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000bbe0 tx=0x7ff8a000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:54.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.676+0000 7ff8aa000640 1 -- 172.21.15.86:0/1723455412 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff89809c850 con 0x7ff8ac075fa0 2024-04-17T07:11:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:54 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4043665697' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-17T07:11:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:54 smithi098 ceph-mon[35418]: osdmap e82: 12 total, 12 up, 12 in 2024-04-17T07:11:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:54 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:54 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:11:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:54 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:54 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/145746384' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-17T07:11:54.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:54 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:11:54.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:54 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4043665697' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-17T07:11:54.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:54 smithi103 ceph-mon[35495]: osdmap e82: 12 total, 12 up, 12 in 2024-04-17T07:11:54.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:54 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:54.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:54 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:11:54.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:54 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:54.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:54 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/145746384' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-17T07:11:54.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:54 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:11:54.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:54 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4043665697' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-17T07:11:54.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:54 smithi086 ceph-mon[31733]: osdmap e82: 12 total, 12 up, 12 in 2024-04-17T07:11:54.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:54 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:54.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:54 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:11:54.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:54 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:54.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:54 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/145746384' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-17T07:11:54.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:54 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:11:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:54.932+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1723455412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff8ac062b60 con 0x7ff8ac075fa0 2024-04-17T07:11:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.458+0000 7ff8aa000640 1 -- 172.21.15.86:0/1723455412 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ff89809c150 con 0x7ff8ac075fa0 2024-04-17T07:11:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 2 and data pool 3 2024-04-17T07:11:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.466+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1723455412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff8940a3bb0 msgr2=0x7ff8940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.466+0000 7ff8b2a00640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff8940a3bb0 0x7ff8940a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000bbe0 tx=0x7ff8a000b390 comp rx=0 tx=0).stop 2024-04-17T07:11:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.467+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1723455412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ac075fa0 msgr2=0x7ff8ac19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.467+0000 7ff8b2a00640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ac075fa0 0x7ff8ac19a760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff89800d770 tx=0x7ff89800dc30 comp rx=0 tx=0).stop 2024-04-17T07:11:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.467+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1723455412 shutdown_connections 2024-04-17T07:11:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.467+0000 7ff8b2a00640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff8940a3bb0 0x7ff8940a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.467+0000 7ff8b2a00640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8ac109db0 0x7ff8ac1a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.467+0000 7ff8b2a00640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ac075fa0 0x7ff8ac19a760 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.467+0000 7ff8b2a00640 1 --2- 172.21.15.86:0/1723455412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac0749c0 0x7ff8ac19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.467+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1723455412 >> 172.21.15.86:0/1723455412 conn(0x7ff8ac0fd460 msgr2=0x7ff8ac105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.468+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1723455412 shutdown_connections 2024-04-17T07:11:55.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.468+0000 7ff8b2a00640 1 -- 172.21.15.86:0/1723455412 wait complete. 2024-04-17T07:11:55.500 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-17T07:11:55.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-17T07:11:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.692+0000 7f72bca00640 1 -- 172.21.15.86:0/710608625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8105220 msgr2=0x7f72b8107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.692+0000 7f72bca00640 1 --2- 172.21.15.86:0/710608625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8105220 0x7f72b8107610 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009900 tx=0x7f72a402bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.692+0000 7f72bca00640 1 -- 172.21.15.86:0/710608625 shutdown_connections 2024-04-17T07:11:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.692+0000 7f72bca00640 1 --2- 172.21.15.86:0/710608625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72b8107b50 0x7f72b8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.692+0000 7f72bca00640 1 --2- 172.21.15.86:0/710608625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8105220 0x7f72b8107610 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.692+0000 7f72bca00640 1 --2- 172.21.15.86:0/710608625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72b81028f0 0x7f72b8104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.692+0000 7f72bca00640 1 -- 172.21.15.86:0/710608625 >> 172.21.15.86:0/710608625 conn(0x7f72b80fc300 msgr2=0x7f72b80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.692+0000 7f72bca00640 1 -- 172.21.15.86:0/710608625 shutdown_connections 2024-04-17T07:11:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.692+0000 7f72bca00640 1 -- 172.21.15.86:0/710608625 wait complete. 2024-04-17T07:11:55.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.693+0000 7f72bca00640 1 Processor -- start 2024-04-17T07:11:55.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.693+0000 7f72bca00640 1 -- start start 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.693+0000 7f72bca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72b81028f0 0x7f72b819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72bca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8105220 0x7f72b819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72bca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72b8107b50 0x7f72b81a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72bca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72b810d9e0 con 0x7f72b8105220 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72bca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f72b810d860 con 0x7f72b81028f0 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72bca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72b810db60 con 0x7f72b8107b50 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8105220 0x7f72b819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8105220 0x7f72b819a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33906/0 (socket says 172.21.15.86:33906) 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b5e00640 1 -- 172.21.15.86:0/629512212 learned_addr learned my addr 172.21.15.86:0/629512212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72b8107b50 0x7f72b81a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b6800640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72b81028f0 0x7f72b819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b5e00640 1 -- 172.21.15.86:0/629512212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72b8107b50 msgr2=0x7f72b81a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b5e00640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72b8107b50 0x7f72b81a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:55.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b5e00640 1 -- 172.21.15.86:0/629512212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72b81028f0 msgr2=0x7f72b819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:55.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b5e00640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72b81028f0 0x7f72b819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:55.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b5e00640 1 -- 172.21.15.86:0/629512212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b81a1f00 con 0x7f72b8105220 2024-04-17T07:11:55.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.694+0000 7f72b7200640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72b8107b50 0x7f72b81a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:11:55.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.695+0000 7f72b5e00640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8105220 0x7f72b819a820 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f72a4000c00 tx=0x7f72a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:55.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.695+0000 7f72af400640 1 -- 172.21.15.86:0/629512212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4043020 con 0x7f72b8105220 2024-04-17T07:11:55.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.695+0000 7f72af400640 1 -- 172.21.15.86:0/629512212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72a40044c0 con 0x7f72b8105220 2024-04-17T07:11:55.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.695+0000 7f72bca00640 1 -- 172.21.15.86:0/629512212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b81a2190 con 0x7f72b8105220 2024-04-17T07:11:55.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.695+0000 7f72af400640 1 -- 172.21.15.86:0/629512212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4003b40 con 0x7f72b8105220 2024-04-17T07:11:55.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.695+0000 7f72bca00640 1 -- 172.21.15.86:0/629512212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b81a2730 con 0x7f72b8105220 2024-04-17T07:11:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.696+0000 7f72bca00640 1 -- 172.21.15.86:0/629512212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b81096a0 con 0x7f72b8105220 2024-04-17T07:11:55.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.699+0000 7f72af400640 1 -- 172.21.15.86:0/629512212 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f72a4046020 con 0x7f72b8105220 2024-04-17T07:11:55.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.702+0000 7f72af400640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f729c0a3c30 0x7f729c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:55.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.702+0000 7f72af400640 1 -- 172.21.15.86:0/629512212 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f72a40f2850 con 0x7f72b8105220 2024-04-17T07:11:55.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.702+0000 7f72b6800640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f729c0a3c30 0x7f729c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:55.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.703+0000 7f72b6800640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f729c0a3c30 0x7f729c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f72a800bbe0 tx=0x7f72a800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:55.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.704+0000 7f72af400640 1 -- 172.21.15.86:0/629512212 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f72a404d020 con 0x7f72b8105220 2024-04-17T07:11:55.746 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:55 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/145746384' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-17T07:11:55.746 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:55 smithi098 ceph-mon[35418]: osdmap e83: 12 total, 12 up, 12 in 2024-04-17T07:11:55.746 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:55 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1723455412' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T07:11:55.746 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:55 smithi098 ceph-mon[35418]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-17T07:11:55.746 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:55 smithi098 ceph-mon[35418]: pgmap v357: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:55.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:55 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/145746384' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-17T07:11:55.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:55 smithi103 ceph-mon[35495]: osdmap e83: 12 total, 12 up, 12 in 2024-04-17T07:11:55.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:55 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1723455412' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T07:11:55.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:55 smithi103 ceph-mon[35495]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-17T07:11:55.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:55 smithi103 ceph-mon[35495]: pgmap v357: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:55.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:55 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/145746384' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-17T07:11:55.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:55 smithi086 ceph-mon[31733]: osdmap e83: 12 total, 12 up, 12 in 2024-04-17T07:11:55.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:55 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1723455412' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-17T07:11:55.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:55 smithi086 ceph-mon[31733]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-17T07:11:55.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:55 smithi086 ceph-mon[31733]: pgmap v357: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:11:55.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:55.917+0000 7f72bca00640 1 -- 172.21.15.86:0/629512212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f72b802d820 con 0x7f72b8105220 2024-04-17T07:11:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.437+0000 7f72af400640 1 -- 172.21.15.86:0/629512212 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f72a40bf400 con 0x7f72b8105220 2024-04-17T07:11:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.440+0000 7f72bca00640 1 -- 172.21.15.86:0/629512212 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f729c0a3c30 msgr2=0x7f729c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.440+0000 7f72bca00640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f729c0a3c30 0x7f729c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f72a800bbe0 tx=0x7f72a800b390 comp rx=0 tx=0).stop 2024-04-17T07:11:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.440+0000 7f72bca00640 1 -- 172.21.15.86:0/629512212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8105220 msgr2=0x7f72b819a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.440+0000 7f72bca00640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8105220 0x7f72b819a820 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f72a4000c00 tx=0x7f72a40040c0 comp rx=0 tx=0).stop 2024-04-17T07:11:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.440+0000 7f72bca00640 1 -- 172.21.15.86:0/629512212 shutdown_connections 2024-04-17T07:11:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.440+0000 7f72bca00640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f729c0a3c30 0x7f729c0a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.440+0000 7f72bca00640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72b8107b50 0x7f72b81a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.440+0000 7f72bca00640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b8105220 0x7f72b819a820 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.440+0000 7f72bca00640 1 --2- 172.21.15.86:0/629512212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72b81028f0 0x7f72b819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.440+0000 7f72bca00640 1 -- 172.21.15.86:0/629512212 >> 172.21.15.86:0/629512212 conn(0x7f72b80fc300 msgr2=0x7f72b8101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.441+0000 7f72bca00640 1 -- 172.21.15.86:0/629512212 shutdown_connections 2024-04-17T07:11:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.441+0000 7f72bca00640 1 -- 172.21.15.86:0/629512212 wait complete. 2024-04-17T07:11:56.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T07:11:56.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.672+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2735019265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b24103190 msgr2=0x7f5b24105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.672+0000 7f5b2bc00640 1 --2- 172.21.15.86:0/2735019265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b24103190 0x7f5b24105600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009860 tx=0x7f5b1402bb70 comp rx=0 tx=0).stop 2024-04-17T07:11:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.672+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2735019265 shutdown_connections 2024-04-17T07:11:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.672+0000 7f5b2bc00640 1 --2- 172.21.15.86:0/2735019265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b24105b40 0x7f5b24105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.672+0000 7f5b2bc00640 1 --2- 172.21.15.86:0/2735019265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b24103190 0x7f5b24105600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.672+0000 7f5b2bc00640 1 --2- 172.21.15.86:0/2735019265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b24100820 0x7f5b24102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.672+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2735019265 >> 172.21.15.86:0/2735019265 conn(0x7f5b240fc440 msgr2=0x7f5b240fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.672+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2735019265 shutdown_connections 2024-04-17T07:11:56.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.672+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2735019265 wait complete. 2024-04-17T07:11:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.673+0000 7f5b2bc00640 1 Processor -- start 2024-04-17T07:11:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.673+0000 7f5b2bc00640 1 -- start start 2024-04-17T07:11:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b2bc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b24100820 0x7f5b2419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b2bc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b24103190 0x7f5b2419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b2bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b24105b40 0x7f5b241a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b2bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b241140e0 con 0x7f5b24105b40 2024-04-17T07:11:56.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b2bc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24113f60 con 0x7f5b24100820 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b2bc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24114260 con 0x7f5b24103190 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b29c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b24105b40 0x7f5b241a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b29c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b24105b40 0x7f5b241a5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33928/0 (socket says 172.21.15.86:33928) 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b29c00640 1 -- 172.21.15.86:0/2971891335 learned_addr learned my addr 172.21.15.86:0/2971891335 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b23e00640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b24103190 0x7f5b2419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b29200640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b24100820 0x7f5b2419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b29c00640 1 -- 172.21.15.86:0/2971891335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b24103190 msgr2=0x7f5b2419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b29c00640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b24103190 0x7f5b2419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b29c00640 1 -- 172.21.15.86:0/2971891335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b24100820 msgr2=0x7f5b2419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b29c00640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b24100820 0x7f5b2419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.674+0000 7f5b29c00640 1 -- 172.21.15.86:0/2971891335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b241a6410 con 0x7f5b24105b40 2024-04-17T07:11:56.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.675+0000 7f5b29c00640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b24105b40 0x7f5b241a5da0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00efc0 tx=0x7f5b1c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:56.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.675+0000 7f5b21600640 1 -- 172.21.15.86:0/2971891335 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c003970 con 0x7f5b24105b40 2024-04-17T07:11:56.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.675+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2971891335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b241a6700 con 0x7f5b24105b40 2024-04-17T07:11:56.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.675+0000 7f5b21600640 1 -- 172.21.15.86:0/2971891335 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b1c004020 con 0x7f5b24105b40 2024-04-17T07:11:56.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.675+0000 7f5b21600640 1 -- 172.21.15.86:0/2971891335 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c004d40 con 0x7f5b24105b40 2024-04-17T07:11:56.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.675+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2971891335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b241a6bd0 con 0x7f5b24105b40 2024-04-17T07:11:56.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.676+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2971891335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b240fdf30 con 0x7f5b24105b40 2024-04-17T07:11:56.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.679+0000 7f5b21600640 1 -- 172.21.15.86:0/2971891335 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b1c01d020 con 0x7f5b24105b40 2024-04-17T07:11:56.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.681+0000 7f5b21600640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b0c0a3bb0 0x7f5b0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:56.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.681+0000 7f5b21600640 1 -- 172.21.15.86:0/2971891335 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f5b1c0d0ef0 con 0x7f5b24105b40 2024-04-17T07:11:56.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.681+0000 7f5b29200640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b0c0a3bb0 0x7f5b0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:56.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.682+0000 7f5b29200640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b0c0a3bb0 0x7f5b0c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005fd0 tx=0x7f5b18005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:56.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.685+0000 7f5b21600640 1 -- 172.21.15.86:0/2971891335 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5b1c024030 con 0x7f5b24105b40 2024-04-17T07:11:56.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:56 smithi098 ceph-mon[35418]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:11:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:56 smithi098 ceph-mon[35418]: osdmap e84: 12 total, 12 up, 12 in 2024-04-17T07:11:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:56 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1723455412' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-17T07:11:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:56 smithi098 ceph-mon[35418]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-17T07:11:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:56 smithi098 ceph-mon[35418]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-17T07:11:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:56 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/629512212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-17T07:11:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:56 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/629512212' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-17T07:11:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:56 smithi098 ceph-mon[35418]: osdmap e85: 12 total, 12 up, 12 in 2024-04-17T07:11:56.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:56 smithi103 ceph-mon[35495]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:11:56.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:56 smithi103 ceph-mon[35495]: osdmap e84: 12 total, 12 up, 12 in 2024-04-17T07:11:56.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:56 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1723455412' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-17T07:11:56.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:56 smithi103 ceph-mon[35495]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-17T07:11:56.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:56 smithi103 ceph-mon[35495]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-17T07:11:56.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:56 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/629512212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-17T07:11:56.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:56 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/629512212' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-17T07:11:56.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:56 smithi103 ceph-mon[35495]: osdmap e85: 12 total, 12 up, 12 in 2024-04-17T07:11:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:56 smithi086 ceph-mon[31733]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:11:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:56 smithi086 ceph-mon[31733]: osdmap e84: 12 total, 12 up, 12 in 2024-04-17T07:11:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:56 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1723455412' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-17T07:11:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:56 smithi086 ceph-mon[31733]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-17T07:11:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:56 smithi086 ceph-mon[31733]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-17T07:11:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:56 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/629512212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-17T07:11:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:56 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/629512212' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-17T07:11:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:56 smithi086 ceph-mon[31733]: osdmap e85: 12 total, 12 up, 12 in 2024-04-17T07:11:56.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:56.906+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2971891335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5b24062b60 con 0x7f5b24105b40 2024-04-17T07:11:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] up:active 2024-04-17T07:11:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2971891335' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: pgmap v360: 97 pgs: 11 creating+peering, 48 active+clean, 38 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:57.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:11:57.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] up:active 2024-04-17T07:11:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:11:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:11:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2971891335' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T07:11:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: pgmap v360: 97 pgs: 11 creating+peering, 48 active+clean, 38 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-04-17T07:11:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:57.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:11:57.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] up:active 2024-04-17T07:11:57.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2971891335' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: pgmap v360: 97 pgs: 11 creating+peering, 48 active+clean, 38 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:11:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:11:58.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.478+0000 7f5b21600640 1 -- 172.21.15.86:0/2971891335 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f5b1c09d840 con 0x7f5b24105b40 2024-04-17T07:11:58.477 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' created 2024-04-17T07:11:58.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.481+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2971891335 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b0c0a3bb0 msgr2=0x7f5b0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:58.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.482+0000 7f5b2bc00640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b0c0a3bb0 0x7f5b0c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005fd0 tx=0x7f5b18005850 comp rx=0 tx=0).stop 2024-04-17T07:11:58.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.482+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2971891335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b24105b40 msgr2=0x7f5b241a5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:58.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.482+0000 7f5b2bc00640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b24105b40 0x7f5b241a5da0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00efc0 tx=0x7f5b1c00c800 comp rx=0 tx=0).stop 2024-04-17T07:11:58.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.482+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2971891335 shutdown_connections 2024-04-17T07:11:58.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.482+0000 7f5b2bc00640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b0c0a3bb0 0x7f5b0c0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:58.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.482+0000 7f5b2bc00640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b24105b40 0x7f5b241a5da0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:58.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.482+0000 7f5b2bc00640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b24103190 0x7f5b2419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:58.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.482+0000 7f5b2bc00640 1 --2- 172.21.15.86:0/2971891335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b24100820 0x7f5b2419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:58.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.482+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2971891335 >> 172.21.15.86:0/2971891335 conn(0x7f5b240fc440 msgr2=0x7f5b241012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:58.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.483+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2971891335 shutdown_connections 2024-04-17T07:11:58.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.483+0000 7f5b2bc00640 1 -- 172.21.15.86:0/2971891335 wait complete. 2024-04-17T07:11:58.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-17T07:11:58.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.712+0000 7fcb1d400640 1 -- 172.21.15.86:0/2129397084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181097b0 msgr2=0x7fcb18109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:58.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.712+0000 7fcb1d400640 1 --2- 172.21.15.86:0/2129397084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181097b0 0x7fcb18109c30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009900 tx=0x7fcb0002bc40 comp rx=0 tx=0).stop 2024-04-17T07:11:58.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.713+0000 7fcb1d400640 1 -- 172.21.15.86:0/2129397084 shutdown_connections 2024-04-17T07:11:58.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.713+0000 7fcb1d400640 1 --2- 172.21.15.86:0/2129397084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb1810a170 0x7fcb18110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:58.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.713+0000 7fcb1d400640 1 --2- 172.21.15.86:0/2129397084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181097b0 0x7fcb18109c30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:58.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.713+0000 7fcb1d400640 1 --2- 172.21.15.86:0/2129397084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb181085c0 0x7fcb181089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:58.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.713+0000 7fcb1d400640 1 -- 172.21.15.86:0/2129397084 >> 172.21.15.86:0/2129397084 conn(0x7fcb18077560 msgr2=0x7fcb180799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:58.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.713+0000 7fcb1d400640 1 -- 172.21.15.86:0/2129397084 shutdown_connections 2024-04-17T07:11:58.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.713+0000 7fcb1d400640 1 -- 172.21.15.86:0/2129397084 wait complete. 2024-04-17T07:11:58.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.714+0000 7fcb1d400640 1 Processor -- start 2024-04-17T07:11:58.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.714+0000 7fcb1d400640 1 -- start start 2024-04-17T07:11:58.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.714+0000 7fcb1d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb181085c0 0x7fcb1819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:58.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.714+0000 7fcb1d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181097b0 0x7fcb1819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb1d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb1810a170 0x7fcb181a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb1d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18114130 con 0x7fcb181097b0 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb1d400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb18113fb0 con 0x7fcb181085c0 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb1d400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb181142b0 con 0x7fcb1810a170 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb15e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181097b0 0x7fcb1819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb15e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181097b0 0x7fcb1819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59200/0 (socket says 172.21.15.86:59200) 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb15e00640 1 -- 172.21.15.86:0/425002336 learned_addr learned my addr 172.21.15.86:0/425002336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb16800640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb181085c0 0x7fcb1819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb17200640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb1810a170 0x7fcb181a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb15e00640 1 -- 172.21.15.86:0/425002336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb1810a170 msgr2=0x7fcb181a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb15e00640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb1810a170 0x7fcb181a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb15e00640 1 -- 172.21.15.86:0/425002336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb181085c0 msgr2=0x7fcb1819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb15e00640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb181085c0 0x7fcb1819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:58.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb15e00640 1 -- 172.21.15.86:0/425002336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb181a6330 con 0x7fcb181097b0 2024-04-17T07:11:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.715+0000 7fcb16800640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb181085c0 0x7fcb1819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.716+0000 7fcb15e00640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181097b0 0x7fcb1819ec50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcb00000c00 tx=0x7fcb000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.716+0000 7fcb0f400640 1 -- 172.21.15.86:0/425002336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00043020 con 0x7fcb181097b0 2024-04-17T07:11:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.716+0000 7fcb0f400640 1 -- 172.21.15.86:0/425002336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb000054c0 con 0x7fcb181097b0 2024-04-17T07:11:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.716+0000 7fcb1d400640 1 -- 172.21.15.86:0/425002336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb181a65c0 con 0x7fcb181097b0 2024-04-17T07:11:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.716+0000 7fcb0f400640 1 -- 172.21.15.86:0/425002336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00003d80 con 0x7fcb181097b0 2024-04-17T07:11:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.716+0000 7fcb1d400640 1 -- 172.21.15.86:0/425002336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb181a6b10 con 0x7fcb181097b0 2024-04-17T07:11:58.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.717+0000 7fcb1d400640 1 -- 172.21.15.86:0/425002336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1810f8e0 con 0x7fcb181097b0 2024-04-17T07:11:58.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.720+0000 7fcb0f400640 1 -- 172.21.15.86:0/425002336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcb00046020 con 0x7fcb181097b0 2024-04-17T07:11:58.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.721+0000 7fcb0f400640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcafc0a3bb0 0x7fcafc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:58.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.721+0000 7fcb0f400640 1 -- 172.21.15.86:0/425002336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7fcb000f2f80 con 0x7fcb181097b0 2024-04-17T07:11:58.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.721+0000 7fcb16800640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcafc0a3bb0 0x7fcafc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:58.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.722+0000 7fcb16800640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcafc0a3bb0 0x7fcafc0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb18109610 tx=0x7fcb08005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:58.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.724+0000 7fcb0f400640 1 -- 172.21.15.86:0/425002336 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcb0004d020 con 0x7fcb181097b0 2024-04-17T07:11:58.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:58 smithi103 ceph-mon[35495]: osdmap e86: 12 total, 12 up, 12 in 2024-04-17T07:11:58.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:58 smithi103 ceph-mon[35495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-17T07:11:58.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:58 smithi103 ceph-mon[35495]: Cluster is now healthy 2024-04-17T07:11:58.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:58 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2971891335' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-17T07:11:58.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:58 smithi103 ceph-mon[35495]: osdmap e87: 12 total, 12 up, 12 in 2024-04-17T07:11:58.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:58 smithi086 ceph-mon[31733]: osdmap e86: 12 total, 12 up, 12 in 2024-04-17T07:11:58.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:58 smithi086 ceph-mon[31733]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-17T07:11:58.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:58 smithi086 ceph-mon[31733]: Cluster is now healthy 2024-04-17T07:11:58.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:58 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2971891335' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-17T07:11:58.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:58 smithi086 ceph-mon[31733]: osdmap e87: 12 total, 12 up, 12 in 2024-04-17T07:11:58.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:58.927+0000 7fcb1d400640 1 -- 172.21.15.86:0/425002336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fcb18062b60 con 0x7fcb181097b0 2024-04-17T07:11:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:58 smithi098 ceph-mon[35418]: osdmap e86: 12 total, 12 up, 12 in 2024-04-17T07:11:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:58 smithi098 ceph-mon[35418]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-17T07:11:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:58 smithi098 ceph-mon[35418]: Cluster is now healthy 2024-04-17T07:11:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:58 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2971891335' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-17T07:11:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:58 smithi098 ceph-mon[35418]: osdmap e87: 12 total, 12 up, 12 in 2024-04-17T07:11:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.482+0000 7fcb0f400640 1 -- 172.21.15.86:0/425002336 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7fcb000bf800 con 0x7fcb181097b0 2024-04-17T07:11:59.482 INFO:teuthology.orchestra.run.smithi086.stderr:set pool 4 allow_ec_overwrites to true 2024-04-17T07:11:59.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.486+0000 7fcb1d400640 1 -- 172.21.15.86:0/425002336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcafc0a3bb0 msgr2=0x7fcafc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:59.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.486+0000 7fcb1d400640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcafc0a3bb0 0x7fcafc0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb18109610 tx=0x7fcb08005db0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.486+0000 7fcb1d400640 1 -- 172.21.15.86:0/425002336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181097b0 msgr2=0x7fcb1819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:59.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.486+0000 7fcb1d400640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181097b0 0x7fcb1819ec50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcb00000c00 tx=0x7fcb000042f0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.486+0000 7fcb1d400640 1 -- 172.21.15.86:0/425002336 shutdown_connections 2024-04-17T07:11:59.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.486+0000 7fcb1d400640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcafc0a3bb0 0x7fcafc0a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.486+0000 7fcb1d400640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb1810a170 0x7fcb181a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.486+0000 7fcb1d400640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb181097b0 0x7fcb1819ec50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.486+0000 7fcb1d400640 1 --2- 172.21.15.86:0/425002336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb181085c0 0x7fcb1819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.487+0000 7fcb1d400640 1 -- 172.21.15.86:0/425002336 >> 172.21.15.86:0/425002336 conn(0x7fcb18077560 msgr2=0x7fcb1810cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:59.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.487+0000 7fcb1d400640 1 -- 172.21.15.86:0/425002336 shutdown_connections 2024-04-17T07:11:59.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.487+0000 7fcb1d400640 1 -- 172.21.15.86:0/425002336 wait complete. 2024-04-17T07:11:59.505 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:11:59 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b[66166]: 2024-04-17T07:11:59.129+0000 7f64d0a00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T07:11:59.520 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-17T07:11:59.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.709+0000 7f5285c00640 1 -- 172.21.15.86:0/4284093209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280105370 msgr2=0x7f5280107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:59.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.709+0000 7f5285c00640 1 --2- 172.21.15.86:0/4284093209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280105370 0x7f5280107760 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f526c009900 tx=0x7f526c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:11:59.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.709+0000 7f5285c00640 1 -- 172.21.15.86:0/4284093209 shutdown_connections 2024-04-17T07:11:59.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.709+0000 7f5285c00640 1 --2- 172.21.15.86:0/4284093209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280107ca0 0x7f528010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.709+0000 7f5285c00640 1 --2- 172.21.15.86:0/4284093209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280105370 0x7f5280107760 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.709+0000 7f5285c00640 1 --2- 172.21.15.86:0/4284093209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5280102a40 0x7f5280104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.709+0000 7f5285c00640 1 -- 172.21.15.86:0/4284093209 >> 172.21.15.86:0/4284093209 conn(0x7f52800fc430 msgr2=0x7f52800fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:11:59.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.709+0000 7f5285c00640 1 -- 172.21.15.86:0/4284093209 shutdown_connections 2024-04-17T07:11:59.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.709+0000 7f5285c00640 1 -- 172.21.15.86:0/4284093209 wait complete. 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.710+0000 7f5285c00640 1 Processor -- start 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.710+0000 7f5285c00640 1 -- start start 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.711+0000 7f5285c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280102a40 0x7f52801023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.711+0000 7f5285c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280105370 0x7f52801009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.711+0000 7f5285c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5280107ca0 0x7f5280100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.711+0000 7f5285c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f528010db50 con 0x7f5280105370 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.711+0000 7f5285c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f528010d9d0 con 0x7f5280102a40 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.711+0000 7f5285c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f528010dcd0 con 0x7f5280107ca0 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.711+0000 7f527e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280105370 0x7f52801009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.711+0000 7f527e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280105370 0x7f52801009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59218/0 (socket says 172.21.15.86:59218) 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.711+0000 7f527e000640 1 -- 172.21.15.86:0/3028625539 learned_addr learned my addr 172.21.15.86:0/3028625539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:11:59.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.711+0000 7f527ea00640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280102a40 0x7f52801023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f527f400640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5280107ca0 0x7f5280100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f527e000640 1 -- 172.21.15.86:0/3028625539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5280107ca0 msgr2=0x7f5280100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f527e000640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5280107ca0 0x7f5280100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f527e000640 1 -- 172.21.15.86:0/3028625539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280102a40 msgr2=0x7f52801023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f527e000640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280102a40 0x7f52801023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f527e000640 1 -- 172.21.15.86:0/3028625539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52801017c0 con 0x7f5280105370 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f527f400640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5280107ca0 0x7f5280100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f527e000640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280105370 0x7f52801009f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f526c005ab0 tx=0x7f526c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f5277400640 1 -- 172.21.15.86:0/3028625539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c043020 con 0x7f5280105370 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f5277400640 1 -- 172.21.15.86:0/3028625539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f526c0054c0 con 0x7f5280105370 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f5277400640 1 -- 172.21.15.86:0/3028625539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c003de0 con 0x7f5280105370 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f5285c00640 1 -- 172.21.15.86:0/3028625539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52801a23a0 con 0x7f5280105370 2024-04-17T07:11:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.712+0000 7f5285c00640 1 -- 172.21.15.86:0/3028625539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a2910 con 0x7f5280105370 2024-04-17T07:11:59.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.714+0000 7f5285c00640 1 -- 172.21.15.86:0/3028625539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52800fdf20 con 0x7f5280105370 2024-04-17T07:11:59.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.716+0000 7f5277400640 1 -- 172.21.15.86:0/3028625539 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f526c046020 con 0x7f5280105370 2024-04-17T07:11:59.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.717+0000 7f5277400640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52640a3c30 0x7f52640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:11:59.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.717+0000 7f5277400640 1 -- 172.21.15.86:0/3028625539 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f526c0f30e0 con 0x7f5280105370 2024-04-17T07:11:59.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.717+0000 7f527ea00640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52640a3c30 0x7f52640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:11:59.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.718+0000 7f527ea00640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52640a3c30 0x7f52640a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52700058c0 tx=0x7f527000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:11:59.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.720+0000 7f5277400640 1 -- 172.21.15.86:0/3028625539 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f526c04d020 con 0x7f5280105370 2024-04-17T07:11:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:59 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/425002336' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-17T07:11:59.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:59 smithi103 ceph-mon[35495]: pgmap v363: 129 pgs: 11 creating+peering, 84 active+clean, 34 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-04-17T07:11:59.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:59 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/425002336' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-17T07:11:59.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:11:59 smithi103 ceph-mon[35495]: osdmap e88: 12 total, 12 up, 12 in 2024-04-17T07:11:59.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:59 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/425002336' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-17T07:11:59.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:59 smithi086 ceph-mon[31733]: pgmap v363: 129 pgs: 11 creating+peering, 84 active+clean, 34 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-04-17T07:11:59.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:59 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/425002336' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-17T07:11:59.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:11:59 smithi086 ceph-mon[31733]: osdmap e88: 12 total, 12 up, 12 in 2024-04-17T07:11:59.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:11:59.970+0000 7f5285c00640 1 -- 172.21.15.86:0/3028625539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f528002d820 con 0x7f5280105370 2024-04-17T07:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:59 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/425002336' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-17T07:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:59 smithi098 ceph-mon[35418]: pgmap v363: 129 pgs: 11 creating+peering, 84 active+clean, 34 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-04-17T07:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:59 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/425002336' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-17T07:12:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:11:59 smithi098 ceph-mon[35418]: osdmap e88: 12 total, 12 up, 12 in 2024-04-17T07:12:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.543+0000 7f5277400640 1 -- 172.21.15.86:0/3028625539 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f526c0bf9c0 con 0x7f5280105370 2024-04-17T07:12:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:added data pool 4 to fsmap 2024-04-17T07:12:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.547+0000 7f5285c00640 1 -- 172.21.15.86:0/3028625539 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52640a3c30 msgr2=0x7f52640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.547+0000 7f5285c00640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52640a3c30 0x7f52640a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52700058c0 tx=0x7f527000a4c0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.547+0000 7f5285c00640 1 -- 172.21.15.86:0/3028625539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280105370 msgr2=0x7f52801009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.547+0000 7f5285c00640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280105370 0x7f52801009f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f526c005ab0 tx=0x7f526c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.548+0000 7f5285c00640 1 -- 172.21.15.86:0/3028625539 shutdown_connections 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.548+0000 7f5285c00640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52640a3c30 0x7f52640a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.548+0000 7f5285c00640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5280107ca0 0x7f5280100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.548+0000 7f5285c00640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280105370 0x7f52801009f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.548+0000 7f5285c00640 1 --2- 172.21.15.86:0/3028625539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5280102a40 0x7f52801023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.548+0000 7f5285c00640 1 -- 172.21.15.86:0/3028625539 >> 172.21.15.86:0/3028625539 conn(0x7f52800fc430 msgr2=0x7f5280103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.548+0000 7f5285c00640 1 -- 172.21.15.86:0/3028625539 shutdown_connections 2024-04-17T07:12:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.548+0000 7f5285c00640 1 -- 172.21.15.86:0/3028625539 wait complete. 2024-04-17T07:12:00.568 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:12:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.761+0000 7fe475000640 1 -- 172.21.15.86:0/407924920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe470100650 msgr2=0x7fe470100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.761+0000 7fe475000640 1 --2- 172.21.15.86:0/407924920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe470100650 0x7fe470100a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe460009900 tx=0x7fe46002bc40 comp rx=0 tx=0).stop 2024-04-17T07:12:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.762+0000 7fe475000640 1 -- 172.21.15.86:0/407924920 shutdown_connections 2024-04-17T07:12:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.762+0000 7fe475000640 1 --2- 172.21.15.86:0/407924920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe470101950 0x7fe470101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.762+0000 7fe475000640 1 --2- 172.21.15.86:0/407924920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe470100fb0 0x7fe470101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.762+0000 7fe475000640 1 --2- 172.21.15.86:0/407924920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe470100650 0x7fe470100a70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.762+0000 7fe475000640 1 -- 172.21.15.86:0/407924920 >> 172.21.15.86:0/407924920 conn(0x7fe4700fc230 msgr2=0x7fe4700fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.762+0000 7fe475000640 1 -- 172.21.15.86:0/407924920 shutdown_connections 2024-04-17T07:12:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.762+0000 7fe475000640 1 -- 172.21.15.86:0/407924920 wait complete. 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe475000640 1 Processor -- start 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe475000640 1 -- start start 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe475000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe470100650 0x7fe47019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe475000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe470100fb0 0x7fe47019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe475000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe470101950 0x7fe4701a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe475000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe47010fb80 con 0x7fe470101950 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe475000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe47010fa00 con 0x7fe470100650 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe475000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe47010fd00 con 0x7fe470100fb0 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe46fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe470101950 0x7fe4701a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe46fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe470101950 0x7fe4701a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59238/0 (socket says 172.21.15.86:59238) 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.763+0000 7fe46fe00640 1 -- 172.21.15.86:0/518056411 learned_addr learned my addr 172.21.15.86:0/518056411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe46f400640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe470100650 0x7fe47019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe467e00640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe470100fb0 0x7fe47019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe46fe00640 1 -- 172.21.15.86:0/518056411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe470100fb0 msgr2=0x7fe47019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe46fe00640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe470100fb0 0x7fe47019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe46fe00640 1 -- 172.21.15.86:0/518056411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe470100650 msgr2=0x7fe47019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe46fe00640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe470100650 0x7fe47019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe46fe00640 1 -- 172.21.15.86:0/518056411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4701a1d60 con 0x7fe470101950 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe46fe00640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe470101950 0x7fe4701a16f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe46800cce0 tx=0x7fe46800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe46cc00640 1 -- 172.21.15.86:0/518056411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46800f450 con 0x7fe470101950 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe46cc00640 1 -- 172.21.15.86:0/518056411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe46800faa0 con 0x7fe470101950 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe475000640 1 -- 172.21.15.86:0/518056411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4701a1ff0 con 0x7fe470101950 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe46cc00640 1 -- 172.21.15.86:0/518056411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468018b70 con 0x7fe470101950 2024-04-17T07:12:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.764+0000 7fe475000640 1 -- 172.21.15.86:0/518056411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4701a2510 con 0x7fe470101950 2024-04-17T07:12:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.766+0000 7fe475000640 1 -- 172.21.15.86:0/518056411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe470068ca0 con 0x7fe470101950 2024-04-17T07:12:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.768+0000 7fe46cc00640 1 -- 172.21.15.86:0/518056411 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe468018d10 con 0x7fe470101950 2024-04-17T07:12:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.769+0000 7fe46cc00640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3c30 0x7fe4540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.769+0000 7fe46cc00640 1 -- 172.21.15.86:0/518056411 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe4680d0b70 con 0x7fe470101950 2024-04-17T07:12:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.769+0000 7fe46f400640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3c30 0x7fe4540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.770+0000 7fe46f400640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3c30 0x7fe4540a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe46002c850 tx=0x7fe460000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:00.771+0000 7fe46cc00640 1 -- 172.21.15.86:0/518056411 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe468014030 con 0x7fe470101950 2024-04-17T07:12:00.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:00 smithi086 ceph-mon[31733]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:12:00.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:00 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3028625539' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-17T07:12:01.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:00 smithi098 ceph-mon[35418]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:12:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:00 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3028625539' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-17T07:12:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.011+0000 7fe475000640 1 -- 172.21.15.86:0/518056411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe47010b3f0 con 0x7fe470101950 2024-04-17T07:12:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.013+0000 7fe46cc00640 1 -- 172.21.15.86:0/518056411 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7fe47010b3f0 con 0x7fe470101950 2024-04-17T07:12:01.012 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:01.013 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:11:59.971908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34490,"qdb_cluster":[34490]},"id":1}]} 2024-04-17T07:12:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.015+0000 7fe475000640 1 -- 172.21.15.86:0/518056411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3c30 msgr2=0x7fe4540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.015+0000 7fe475000640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3c30 0x7fe4540a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe46002c850 tx=0x7fe460000bc0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.016+0000 7fe475000640 1 -- 172.21.15.86:0/518056411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe470101950 msgr2=0x7fe4701a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.016+0000 7fe475000640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe470101950 0x7fe4701a16f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe46800cce0 tx=0x7fe46800a620 comp rx=0 tx=0).stop 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.016+0000 7fe475000640 1 -- 172.21.15.86:0/518056411 shutdown_connections 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.016+0000 7fe475000640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3c30 0x7fe4540a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.016+0000 7fe475000640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe470101950 0x7fe4701a16f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.016+0000 7fe475000640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe470100fb0 0x7fe47019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.016+0000 7fe475000640 1 --2- 172.21.15.86:0/518056411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe470100650 0x7fe47019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.016+0000 7fe475000640 1 -- 172.21.15.86:0/518056411 >> 172.21.15.86:0/518056411 conn(0x7fe4700fc230 msgr2=0x7fe4700fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.016+0000 7fe475000640 1 -- 172.21.15.86:0/518056411 shutdown_connections 2024-04-17T07:12:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.016+0000 7fe475000640 1 -- 172.21.15.86:0/518056411 wait complete. 2024-04-17T07:12:01.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:00 smithi103 ceph-mon[35495]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-17T07:12:01.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:00 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3028625539' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-17T07:12:01.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:12:01.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.216+0000 7f3a85400640 1 -- 172.21.15.86:0/474242056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a801095d0 msgr2=0x7f3a80109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.216+0000 7f3a85400640 1 --2- 172.21.15.86:0/474242056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a801095d0 0x7f3a80109a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009900 tx=0x7f3a6c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:01.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.216+0000 7f3a85400640 1 -- 172.21.15.86:0/474242056 shutdown_connections 2024-04-17T07:12:01.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.216+0000 7f3a85400640 1 --2- 172.21.15.86:0/474242056 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a80109f90 0x7f3a80110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.216+0000 7f3a85400640 1 --2- 172.21.15.86:0/474242056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a801095d0 0x7f3a80109a50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.216+0000 7f3a85400640 1 --2- 172.21.15.86:0/474242056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a801083e0 0x7f3a80108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.216+0000 7f3a85400640 1 -- 172.21.15.86:0/474242056 >> 172.21.15.86:0/474242056 conn(0x7f3a800773a0 msgr2=0x7f3a800797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:01.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.216+0000 7f3a85400640 1 -- 172.21.15.86:0/474242056 shutdown_connections 2024-04-17T07:12:01.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.216+0000 7f3a85400640 1 -- 172.21.15.86:0/474242056 wait complete. 2024-04-17T07:12:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.217+0000 7f3a85400640 1 Processor -- start 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.217+0000 7f3a85400640 1 -- start start 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a85400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a801083e0 0x7f3a8019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a85400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a801095d0 0x7f3a8019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a85400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a80109f90 0x7f3a801a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a85400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a80113f20 con 0x7f3a801095d0 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a85400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a80113da0 con 0x7f3a801083e0 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a85400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a801140a0 con 0x7f3a80109f90 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a7f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a801095d0 0x7f3a8019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a7f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a801095d0 0x7f3a8019eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59260/0 (socket says 172.21.15.86:59260) 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a7f400640 1 -- 172.21.15.86:0/4270664349 learned_addr learned my addr 172.21.15.86:0/4270664349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a84a00640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a80109f90 0x7f3a801a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.218+0000 7f3a7fe00640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a801083e0 0x7f3a8019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:01.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a7f400640 1 -- 172.21.15.86:0/4270664349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a80109f90 msgr2=0x7f3a801a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a7f400640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a80109f90 0x7f3a801a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a7f400640 1 -- 172.21.15.86:0/4270664349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a801083e0 msgr2=0x7f3a8019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a7f400640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a801083e0 0x7f3a8019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a7f400640 1 -- 172.21.15.86:0/4270664349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a801a6200 con 0x7f3a801095d0 2024-04-17T07:12:01.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a7f400640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a801095d0 0x7f3a8019eb20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c006010 tx=0x7f3a6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:01.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a7cc00640 1 -- 172.21.15.86:0/4270664349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c043020 con 0x7f3a801095d0 2024-04-17T07:12:01.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a7cc00640 1 -- 172.21.15.86:0/4270664349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a6c004aa0 con 0x7f3a801095d0 2024-04-17T07:12:01.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a7cc00640 1 -- 172.21.15.86:0/4270664349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c003c60 con 0x7f3a801095d0 2024-04-17T07:12:01.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a85400640 1 -- 172.21.15.86:0/4270664349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a801a6490 con 0x7f3a801095d0 2024-04-17T07:12:01.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.219+0000 7f3a85400640 1 -- 172.21.15.86:0/4270664349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a801a6990 con 0x7f3a801095d0 2024-04-17T07:12:01.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.221+0000 7f3a85400640 1 -- 172.21.15.86:0/4270664349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8010f7d0 con 0x7f3a801095d0 2024-04-17T07:12:01.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.223+0000 7f3a7cc00640 1 -- 172.21.15.86:0/4270664349 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3a6c046020 con 0x7f3a801095d0 2024-04-17T07:12:01.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.224+0000 7f3a7cc00640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a640a3c30 0x7f3a640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:01.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.224+0000 7f3a7cc00640 1 -- 172.21.15.86:0/4270664349 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f3a6c0f30f0 con 0x7f3a801095d0 2024-04-17T07:12:01.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.224+0000 7f3a7fe00640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a640a3c30 0x7f3a640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:01.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.225+0000 7f3a7fe00640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a640a3c30 0x7f3a640a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a70005fd0 tx=0x7f3a70005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:01.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.226+0000 7f3a7cc00640 1 -- 172.21.15.86:0/4270664349 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3a6c04d020 con 0x7f3a801095d0 2024-04-17T07:12:01.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.415+0000 7f3a85400640 1 -- 172.21.15.86:0/4270664349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a8002d7a0 con 0x7f3a801095d0 2024-04-17T07:12:01.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.419+0000 7f3a7cc00640 1 -- 172.21.15.86:0/4270664349 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24057 (secure 0 0 0) 0x7f3a8002d7a0 con 0x7f3a801095d0 2024-04-17T07:12:01.419 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:01.419 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:11:59.971687+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:12:01.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 -- 172.21.15.86:0/4270664349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a640a3c30 msgr2=0x7f3a640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a640a3c30 0x7f3a640a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a70005fd0 tx=0x7f3a70005850 comp rx=0 tx=0).stop 2024-04-17T07:12:01.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 -- 172.21.15.86:0/4270664349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a801095d0 msgr2=0x7f3a8019eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a801095d0 0x7f3a8019eb20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c006010 tx=0x7f3a6c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 -- 172.21.15.86:0/4270664349 shutdown_connections 2024-04-17T07:12:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a640a3c30 0x7f3a640a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a80109f90 0x7f3a801a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a801095d0 0x7f3a8019eb20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 --2- 172.21.15.86:0/4270664349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a801083e0 0x7f3a8019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 -- 172.21.15.86:0/4270664349 >> 172.21.15.86:0/4270664349 conn(0x7f3a800773a0 msgr2=0x7f3a8010c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 -- 172.21.15.86:0/4270664349 shutdown_connections 2024-04-17T07:12:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.422+0000 7f3a85400640 1 -- 172.21.15.86:0/4270664349 wait complete. 2024-04-17T07:12:01.441 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:12:01.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.655+0000 7f4444a00640 1 -- 172.21.15.86:0/631749543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44401083e0 msgr2=0x7f4440108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.655+0000 7f4444a00640 1 --2- 172.21.15.86:0/631749543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44401083e0 0x7f4440108800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4430009900 tx=0x7f443002bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:01.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.655+0000 7f4444a00640 1 -- 172.21.15.86:0/631749543 shutdown_connections 2024-04-17T07:12:01.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.655+0000 7f4444a00640 1 --2- 172.21.15.86:0/631749543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4440109f90 0x7f4440110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.655+0000 7f4444a00640 1 --2- 172.21.15.86:0/631749543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44401095d0 0x7f4440109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.655+0000 7f4444a00640 1 --2- 172.21.15.86:0/631749543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44401083e0 0x7f4440108800 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.655+0000 7f4444a00640 1 -- 172.21.15.86:0/631749543 >> 172.21.15.86:0/631749543 conn(0x7f44400773a0 msgr2=0x7f44400797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:01.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.655+0000 7f4444a00640 1 -- 172.21.15.86:0/631749543 shutdown_connections 2024-04-17T07:12:01.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.655+0000 7f4444a00640 1 -- 172.21.15.86:0/631749543 wait complete. 2024-04-17T07:12:01.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.656+0000 7f4444a00640 1 Processor -- start 2024-04-17T07:12:01.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.656+0000 7f4444a00640 1 -- start start 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.656+0000 7f4444a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44401083e0 0x7f444019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.656+0000 7f4444a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44401095d0 0x7f444019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.656+0000 7f4444a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4440109f90 0x7f44401a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.656+0000 7f4444a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4440113f40 con 0x7f44401095d0 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.656+0000 7f4444a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4440113dc0 con 0x7f44401083e0 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.656+0000 7f4444a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44401140c0 con 0x7f4440109f90 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.657+0000 7f4437e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44401095d0 0x7f444019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.657+0000 7f4437e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44401095d0 0x7f444019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59272/0 (socket says 172.21.15.86:59272) 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.657+0000 7f4437e00640 1 -- 172.21.15.86:0/2640276768 learned_addr learned my addr 172.21.15.86:0/2640276768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.657+0000 7f443f400640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44401083e0 0x7f444019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.657+0000 7f4437e00640 1 -- 172.21.15.86:0/2640276768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4440109f90 msgr2=0x7f44401a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.657+0000 7f4437e00640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4440109f90 0x7f44401a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.657+0000 7f4437e00640 1 -- 172.21.15.86:0/2640276768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44401083e0 msgr2=0x7f444019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.657+0000 7f4437e00640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44401083e0 0x7f444019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.657+0000 7f4437e00640 1 -- 172.21.15.86:0/2640276768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44401a6240 con 0x7f44401095d0 2024-04-17T07:12:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.657+0000 7f4437e00640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44401095d0 0x7f444019eb60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f442800e9f0 tx=0x7f442800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.658+0000 7f443cc00640 1 -- 172.21.15.86:0/2640276768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442800cd80 con 0x7f44401095d0 2024-04-17T07:12:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.658+0000 7f443cc00640 1 -- 172.21.15.86:0/2640276768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44280108e0 con 0x7f44401095d0 2024-04-17T07:12:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.658+0000 7f4444a00640 1 -- 172.21.15.86:0/2640276768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44401a6530 con 0x7f44401095d0 2024-04-17T07:12:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.658+0000 7f443cc00640 1 -- 172.21.15.86:0/2640276768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428018920 con 0x7f44401095d0 2024-04-17T07:12:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.658+0000 7f4444a00640 1 -- 172.21.15.86:0/2640276768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44401a6ad0 con 0x7f44401095d0 2024-04-17T07:12:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.659+0000 7f4444a00640 1 -- 172.21.15.86:0/2640276768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444010f720 con 0x7f44401095d0 2024-04-17T07:12:01.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.661+0000 7f443cc00640 1 -- 172.21.15.86:0/2640276768 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4428010a90 con 0x7f44401095d0 2024-04-17T07:12:01.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.662+0000 7f443cc00640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44240a3c30 0x7f44240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:01.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.662+0000 7f443cc00640 1 -- 172.21.15.86:0/2640276768 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f44280d0a00 con 0x7f44401095d0 2024-04-17T07:12:01.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.662+0000 7f443f400640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44240a3c30 0x7f44240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:01.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.663+0000 7f443f400640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44240a3c30 0x7f44240a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f443002c240 tx=0x7f4430005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:01.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.666+0000 7f443cc00640 1 -- 172.21.15.86:0/2640276768 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f442809d1a0 con 0x7f44401095d0 2024-04-17T07:12:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:01 smithi086 ceph-mon[31733]: osdmap e89: 12 total, 12 up, 12 in 2024-04-17T07:12:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:01 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3028625539' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-17T07:12:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:01 smithi086 ceph-mon[31733]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:12:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:01 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/518056411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:12:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:01 smithi086 ceph-mon[31733]: pgmap v366: 129 pgs: 7 creating+peering, 100 active+clean, 22 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-04-17T07:12:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:01 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4270664349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:01.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.875+0000 7f4444a00640 1 -- 172.21.15.86:0/2640276768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4440062c00 con 0x7f44401095d0 2024-04-17T07:12:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.878+0000 7f443cc00640 1 -- 172.21.15.86:0/2640276768 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24057 (secure 0 0 0) 0x7f442809caa0 con 0x7f44401095d0 2024-04-17T07:12:01.877 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:01.877 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:11:59.971687+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:12:01.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.880+0000 7f4444a00640 1 -- 172.21.15.86:0/2640276768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44240a3c30 msgr2=0x7f44240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.880+0000 7f4444a00640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44240a3c30 0x7f44240a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f443002c240 tx=0x7f4430005fd0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.880+0000 7f4444a00640 1 -- 172.21.15.86:0/2640276768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44401095d0 msgr2=0x7f444019eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:01.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.880+0000 7f4444a00640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44401095d0 0x7f444019eb60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f442800e9f0 tx=0x7f442800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.881+0000 7f4444a00640 1 -- 172.21.15.86:0/2640276768 shutdown_connections 2024-04-17T07:12:01.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.881+0000 7f4444a00640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44240a3c30 0x7f44240a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.881+0000 7f4444a00640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4440109f90 0x7f44401a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.881+0000 7f4444a00640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44401095d0 0x7f444019eb60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.881+0000 7f4444a00640 1 --2- 172.21.15.86:0/2640276768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44401083e0 0x7f444019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:01.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.881+0000 7f4444a00640 1 -- 172.21.15.86:0/2640276768 >> 172.21.15.86:0/2640276768 conn(0x7f44400773a0 msgr2=0x7f444010c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:01.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.881+0000 7f4444a00640 1 -- 172.21.15.86:0/2640276768 shutdown_connections 2024-04-17T07:12:01.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:01.881+0000 7f4444a00640 1 -- 172.21.15.86:0/2640276768 wait complete. 2024-04-17T07:12:01.917 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:12:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:01 smithi098 ceph-mon[35418]: osdmap e89: 12 total, 12 up, 12 in 2024-04-17T07:12:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:01 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3028625539' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-17T07:12:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:01 smithi098 ceph-mon[35418]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:12:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:01 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/518056411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:12:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:01 smithi098 ceph-mon[35418]: pgmap v366: 129 pgs: 7 creating+peering, 100 active+clean, 22 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-04-17T07:12:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:01 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4270664349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:02.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:01 smithi103 ceph-mon[35495]: osdmap e89: 12 total, 12 up, 12 in 2024-04-17T07:12:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:01 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3028625539' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-17T07:12:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:01 smithi103 ceph-mon[35495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:12:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:01 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/518056411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:12:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:01 smithi103 ceph-mon[35495]: pgmap v366: 129 pgs: 7 creating+peering, 100 active+clean, 22 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-04-17T07:12:02.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:01 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4270664349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.132+0000 7f3efa000640 1 -- 172.21.15.86:0/1156672962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef41053d0 msgr2=0x7f3ef4105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.132+0000 7f3efa000640 1 --2- 172.21.15.86:0/1156672962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef41053d0 0x7f3ef4105850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009900 tx=0x7f3ee002bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.132+0000 7f3efa000640 1 -- 172.21.15.86:0/1156672962 shutdown_connections 2024-04-17T07:12:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.132+0000 7f3efa000640 1 --2- 172.21.15.86:0/1156672962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4105d90 0x7f3ef410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.132+0000 7f3efa000640 1 --2- 172.21.15.86:0/1156672962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef41053d0 0x7f3ef4105850 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.132+0000 7f3efa000640 1 --2- 172.21.15.86:0/1156672962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef41041e0 0x7f3ef4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.132+0000 7f3efa000640 1 -- 172.21.15.86:0/1156672962 >> 172.21.15.86:0/1156672962 conn(0x7f3ef40ff750 msgr2=0x7f3ef4101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.133+0000 7f3efa000640 1 -- 172.21.15.86:0/1156672962 shutdown_connections 2024-04-17T07:12:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.133+0000 7f3efa000640 1 -- 172.21.15.86:0/1156672962 wait complete. 2024-04-17T07:12:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.133+0000 7f3efa000640 1 Processor -- start 2024-04-17T07:12:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.134+0000 7f3efa000640 1 -- start start 2024-04-17T07:12:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.134+0000 7f3efa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef41041e0 0x7f3ef419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.134+0000 7f3efa000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef41053d0 0x7f3ef419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.134+0000 7f3efa000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4105d90 0x7f3ef41a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.134+0000 7f3efa000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef410fdb0 con 0x7f3ef41041e0 2024-04-17T07:12:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.134+0000 7f3efa000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef410fc30 con 0x7f3ef41053d0 2024-04-17T07:12:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.134+0000 7f3efa000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef410ff30 con 0x7f3ef4105d90 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.134+0000 7f3ef3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef41041e0 0x7f3ef419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.134+0000 7f3ef3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef41041e0 0x7f3ef419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59286/0 (socket says 172.21.15.86:59286) 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.134+0000 7f3ef3400640 1 -- 172.21.15.86:0/3372065169 learned_addr learned my addr 172.21.15.86:0/3372065169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3ef2a00640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef41053d0 0x7f3ef419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3ef3400640 1 -- 172.21.15.86:0/3372065169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4105d90 msgr2=0x7f3ef41a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3ef3e00640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4105d90 0x7f3ef41a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3ef3400640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4105d90 0x7f3ef41a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3ef3400640 1 -- 172.21.15.86:0/3372065169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef41053d0 msgr2=0x7f3ef419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3ef3400640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef41053d0 0x7f3ef419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3ef3400640 1 -- 172.21.15.86:0/3372065169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef41a2020 con 0x7f3ef41041e0 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3ef3e00640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4105d90 0x7f3ef41a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3ef2a00640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef41053d0 0x7f3ef419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:12:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3ef3400640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef41041e0 0x7f3ef419a400 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c970 tx=0x7f3ee400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3eebe00640 1 -- 172.21.15.86:0/3372065169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4007c10 con 0x7f3ef41041e0 2024-04-17T07:12:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3eebe00640 1 -- 172.21.15.86:0/3372065169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ee4007db0 con 0x7f3ef41041e0 2024-04-17T07:12:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3eebe00640 1 -- 172.21.15.86:0/3372065169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400f870 con 0x7f3ef41041e0 2024-04-17T07:12:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.135+0000 7f3efa000640 1 -- 172.21.15.86:0/3372065169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef41a2310 con 0x7f3ef41041e0 2024-04-17T07:12:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.136+0000 7f3efa000640 1 -- 172.21.15.86:0/3372065169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef41a2890 con 0x7f3ef41041e0 2024-04-17T07:12:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.137+0000 7f3efa000640 1 -- 172.21.15.86:0/3372065169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef410b500 con 0x7f3ef41041e0 2024-04-17T07:12:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.139+0000 7f3eebe00640 1 -- 172.21.15.86:0/3372065169 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ee400fa10 con 0x7f3ef41041e0 2024-04-17T07:12:02.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.140+0000 7f3eebe00640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ed80a3c80 0x7f3ed80a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:02.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.140+0000 7f3eebe00640 1 -- 172.21.15.86:0/3372065169 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f3ee40d0a10 con 0x7f3ef41041e0 2024-04-17T07:12:02.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.140+0000 7f3ef2a00640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ed80a3c80 0x7f3ed80a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:02.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.141+0000 7f3ef2a00640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ed80a3c80 0x7f3ed80a6140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3ef419b910 tx=0x7f3ee0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.144+0000 7f3eebe00640 1 -- 172.21.15.86:0/3372065169 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3ee409d240 con 0x7f3ef41041e0 2024-04-17T07:12:02.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.384+0000 7f3efa000640 1 -- 172.21.15.86:0/3372065169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ef4062b90 con 0x7f3ef41041e0 2024-04-17T07:12:02.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.386+0000 7f3eebe00640 1 -- 172.21.15.86:0/3372065169 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7f3ee409cb40 con 0x7f3ef41041e0 2024-04-17T07:12:02.385 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:02.385 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14688,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34400,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34430,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34460,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:11:59.971908+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34490,"qdb_cluster":[34490]},"id":1}]} 2024-04-17T07:12:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.388+0000 7f3efa000640 1 -- 172.21.15.86:0/3372065169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ed80a3c80 msgr2=0x7f3ed80a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.388+0000 7f3efa000640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ed80a3c80 0x7f3ed80a6140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3ef419b910 tx=0x7f3ee0005fd0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.388+0000 7f3efa000640 1 -- 172.21.15.86:0/3372065169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef41041e0 msgr2=0x7f3ef419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.388+0000 7f3efa000640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef41041e0 0x7f3ef419a400 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c970 tx=0x7f3ee400ce30 comp rx=0 tx=0).stop 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.389+0000 7f3efa000640 1 -- 172.21.15.86:0/3372065169 shutdown_connections 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.389+0000 7f3efa000640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ed80a3c80 0x7f3ed80a6140 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.389+0000 7f3efa000640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4105d90 0x7f3ef41a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.389+0000 7f3efa000640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef41053d0 0x7f3ef419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.389+0000 7f3efa000640 1 --2- 172.21.15.86:0/3372065169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ef41041e0 0x7f3ef419a400 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.389+0000 7f3efa000640 1 -- 172.21.15.86:0/3372065169 >> 172.21.15.86:0/3372065169 conn(0x7f3ef40ff750 msgr2=0x7f3ef41086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.389+0000 7f3efa000640 1 -- 172.21.15.86:0/3372065169 shutdown_connections 2024-04-17T07:12:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.389+0000 7f3efa000640 1 -- 172.21.15.86:0/3372065169 wait complete. 2024-04-17T07:12:02.407 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:12:02.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.616+0000 7fdf3ae00640 1 -- 172.21.15.86:0/1847191734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf34103fe0 msgr2=0x7fdf34104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.616+0000 7fdf3ae00640 1 --2- 172.21.15.86:0/1847191734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf34103fe0 0x7fdf34104400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009900 tx=0x7fdf2402bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.616+0000 7fdf3ae00640 1 -- 172.21.15.86:0/1847191734 shutdown_connections 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.616+0000 7fdf3ae00640 1 --2- 172.21.15.86:0/1847191734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf34105b90 0x7fdf3410c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.616+0000 7fdf3ae00640 1 --2- 172.21.15.86:0/1847191734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf341051d0 0x7fdf34105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.616+0000 7fdf3ae00640 1 --2- 172.21.15.86:0/1847191734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf34103fe0 0x7fdf34104400 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.616+0000 7fdf3ae00640 1 -- 172.21.15.86:0/1847191734 >> 172.21.15.86:0/1847191734 conn(0x7fdf340ff570 msgr2=0x7fdf34101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.616+0000 7fdf3ae00640 1 -- 172.21.15.86:0/1847191734 shutdown_connections 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.617+0000 7fdf3ae00640 1 -- 172.21.15.86:0/1847191734 wait complete. 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.617+0000 7fdf3ae00640 1 Processor -- start 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.617+0000 7fdf3ae00640 1 -- start start 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf34103fe0 0x7fdf3419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:02.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3ae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf341051d0 0x7fdf3419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:02.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf34105b90 0x7fdf341a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:02.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3410fb70 con 0x7fdf34105b90 2024-04-17T07:12:02.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3ae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3410f9f0 con 0x7fdf341051d0 2024-04-17T07:12:02.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3410fcf0 con 0x7fdf34103fe0 2024-04-17T07:12:02.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf34105b90 0x7fdf341a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:02.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf34105b90 0x7fdf341a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59306/0 (socket says 172.21.15.86:59306) 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3a400640 1 -- 172.21.15.86:0/2224438428 learned_addr learned my addr 172.21.15.86:0/2224438428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf39a00640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf34103fe0 0x7fdf3419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf39000640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf341051d0 0x7fdf3419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3a400640 1 -- 172.21.15.86:0/2224438428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf34103fe0 msgr2=0x7fdf3419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3a400640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf34103fe0 0x7fdf3419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.618+0000 7fdf3a400640 1 -- 172.21.15.86:0/2224438428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf341051d0 msgr2=0x7fdf3419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.619+0000 7fdf3a400640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf341051d0 0x7fdf3419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.619+0000 7fdf3a400640 1 -- 172.21.15.86:0/2224438428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf341a1ea0 con 0x7fdf34105b90 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.619+0000 7fdf39a00640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf34103fe0 0x7fdf3419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.619+0000 7fdf39000640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf341051d0 0x7fdf3419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.619+0000 7fdf3a400640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf34105b90 0x7fdf341a1830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c970 tx=0x7fdf2c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.619+0000 7fdf32000640 1 -- 172.21.15.86:0/2224438428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c007bb0 con 0x7fdf34105b90 2024-04-17T07:12:02.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.619+0000 7fdf32000640 1 -- 172.21.15.86:0/2224438428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf2c007d50 con 0x7fdf34105b90 2024-04-17T07:12:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.619+0000 7fdf3ae00640 1 -- 172.21.15.86:0/2224438428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf341a2190 con 0x7fdf34105b90 2024-04-17T07:12:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.619+0000 7fdf32000640 1 -- 172.21.15.86:0/2224438428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c004e10 con 0x7fdf34105b90 2024-04-17T07:12:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.619+0000 7fdf3ae00640 1 -- 172.21.15.86:0/2224438428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf341a2690 con 0x7fdf34105b90 2024-04-17T07:12:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.620+0000 7fdf3ae00640 1 -- 172.21.15.86:0/2224438428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3410b320 con 0x7fdf34105b90 2024-04-17T07:12:02.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.622+0000 7fdf32000640 1 -- 172.21.15.86:0/2224438428 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdf2c005020 con 0x7fdf34105b90 2024-04-17T07:12:02.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.623+0000 7fdf32000640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdf180a3bb0 0x7fdf180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:02.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.624+0000 7fdf32000640 1 -- 172.21.15.86:0/2224438428 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fdf2c0d0410 con 0x7fdf34105b90 2024-04-17T07:12:02.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.624+0000 7fdf39a00640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdf180a3bb0 0x7fdf180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:02.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.624+0000 7fdf39a00640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdf180a3bb0 0x7fdf180a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf24036040 tx=0x7fdf24005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:02.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.628+0000 7fdf32000640 1 -- 172.21.15.86:0/2224438428 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdf2c014030 con 0x7fdf34105b90 2024-04-17T07:12:02.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:02 smithi086 ceph-mon[31733]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-17T07:12:02.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:02 smithi086 ceph-mon[31733]: Cluster is now healthy 2024-04-17T07:12:02.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:02 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2640276768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:02.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:02 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3372065169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:12:02.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.833+0000 7fdf3ae00640 1 -- 172.21.15.86:0/2224438428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf34062b60 con 0x7fdf34105b90 2024-04-17T07:12:02.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.837+0000 7fdf32000640 1 -- 172.21.15.86:0/2224438428 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24057 (secure 0 0 0) 0x7fdf2c09cb50 con 0x7fdf34105b90 2024-04-17T07:12:02.836 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:02.836 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:11:59.971687+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:12:02.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 -- 172.21.15.86:0/2224438428 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdf180a3bb0 msgr2=0x7fdf180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:02.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdf180a3bb0 0x7fdf180a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf24036040 tx=0x7fdf24005fd0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 -- 172.21.15.86:0/2224438428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf34105b90 msgr2=0x7fdf341a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:02.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf34105b90 0x7fdf341a1830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c970 tx=0x7fdf2c00ce30 comp rx=0 tx=0).stop 2024-04-17T07:12:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 -- 172.21.15.86:0/2224438428 shutdown_connections 2024-04-17T07:12:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdf180a3bb0 0x7fdf180a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf34105b90 0x7fdf341a1830 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf341051d0 0x7fdf3419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 --2- 172.21.15.86:0/2224438428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf34103fe0 0x7fdf3419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 -- 172.21.15.86:0/2224438428 >> 172.21.15.86:0/2224438428 conn(0x7fdf340ff570 msgr2=0x7fdf341084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 -- 172.21.15.86:0/2224438428 shutdown_connections 2024-04-17T07:12:02.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:02.840+0000 7fdf3ae00640 1 -- 172.21.15.86:0/2224438428 wait complete. 2024-04-17T07:12:02.858 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T07:11:54.934275+0000', 'modified': '2024-04-17T07:11:59.971908+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34490': {'gid': 34490, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/809350102', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 809350102}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 809350102}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34490, 'qdb_cluster': [34490]} 2024-04-17T07:12:02.858 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T07:12:02.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-17T07:12:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:02 smithi098 ceph-mon[35418]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-17T07:12:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:02 smithi098 ceph-mon[35418]: Cluster is now healthy 2024-04-17T07:12:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:02 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2640276768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:02 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3372065169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:12:03.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:02 smithi103 ceph-mon[35495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-17T07:12:03.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:02 smithi103 ceph-mon[35495]: Cluster is now healthy 2024-04-17T07:12:03.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:02 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2640276768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:03.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:02 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3372065169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:12:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.059+0000 7f5783800640 1 -- 172.21.15.86:0/3101250386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577c1031a0 msgr2=0x7f577c103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.059+0000 7f5783800640 1 --2- 172.21.15.86:0/3101250386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577c1031a0 0x7f577c103620 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5764009900 tx=0x7f576402bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.059+0000 7f5783800640 1 -- 172.21.15.86:0/3101250386 shutdown_connections 2024-04-17T07:12:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.059+0000 7f5783800640 1 --2- 172.21.15.86:0/3101250386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f577c103b60 0x7f577c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.059+0000 7f5783800640 1 --2- 172.21.15.86:0/3101250386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577c1031a0 0x7f577c103620 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.059+0000 7f5783800640 1 --2- 172.21.15.86:0/3101250386 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f577c101fb0 0x7f577c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.059+0000 7f5783800640 1 -- 172.21.15.86:0/3101250386 >> 172.21.15.86:0/3101250386 conn(0x7f577c0fd540 msgr2=0x7f577c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.059+0000 7f5783800640 1 -- 172.21.15.86:0/3101250386 shutdown_connections 2024-04-17T07:12:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.059+0000 7f5783800640 1 -- 172.21.15.86:0/3101250386 wait complete. 2024-04-17T07:12:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.060+0000 7f5783800640 1 Processor -- start 2024-04-17T07:12:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.060+0000 7f5783800640 1 -- start start 2024-04-17T07:12:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5783800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f577c101fb0 0x7f577c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5783800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577c1031a0 0x7f577c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5783800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f577c103b60 0x7f577c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5783800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f577c10db80 con 0x7f577c1031a0 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5783800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f577c10da00 con 0x7f577c101fb0 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5783800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f577c10dd00 con 0x7f577c103b60 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5773e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577c1031a0 0x7f577c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5773e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577c1031a0 0x7f577c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59332/0 (socket says 172.21.15.86:59332) 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5773e00640 1 -- 172.21.15.86:0/3095990882 learned_addr learned my addr 172.21.15.86:0/3095990882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5780e00640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f577c101fb0 0x7f577c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5773e00640 1 -- 172.21.15.86:0/3095990882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f577c103b60 msgr2=0x7f577c19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5773e00640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f577c103b60 0x7f577c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5773e00640 1 -- 172.21.15.86:0/3095990882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f577c101fb0 msgr2=0x7f577c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5773e00640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f577c101fb0 0x7f577c198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.061+0000 7f5773e00640 1 -- 172.21.15.86:0/3095990882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c19fea0 con 0x7f577c1031a0 2024-04-17T07:12:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.062+0000 7f5773e00640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577c1031a0 0x7f577c1987c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5764000c00 tx=0x7f57640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.062+0000 7f577a000640 1 -- 172.21.15.86:0/3095990882 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764043020 con 0x7f577c1031a0 2024-04-17T07:12:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.062+0000 7f577a000640 1 -- 172.21.15.86:0/3095990882 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57640054c0 con 0x7f577c1031a0 2024-04-17T07:12:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.062+0000 7f577a000640 1 -- 172.21.15.86:0/3095990882 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764003d80 con 0x7f577c1031a0 2024-04-17T07:12:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.062+0000 7f5783800640 1 -- 172.21.15.86:0/3095990882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c1a0130 con 0x7f577c1031a0 2024-04-17T07:12:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.062+0000 7f5783800640 1 -- 172.21.15.86:0/3095990882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c1a06b0 con 0x7f577c1031a0 2024-04-17T07:12:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.065+0000 7f57863e0740 1 Processor -- start 2024-04-17T07:12:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.065+0000 7f577a000640 1 -- 172.21.15.86:0/3095990882 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5764046020 con 0x7f577c1031a0 2024-04-17T07:12:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.065+0000 7f57863e0740 1 -- start start 2024-04-17T07:12:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f57863e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab57a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f57863e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57a6e0 0x5604ab57f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f57863e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab581ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f57863e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5604ab4dc980 con 0x5604ab57f850 2024-04-17T07:12:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f57863e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5604ab483220 con 0x5604ab57a6e0 2024-04-17T07:12:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f57863e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604ab4d8940 con 0x5604ab579cf0 2024-04-17T07:12:03.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f5773e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab581ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f5773e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab581ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59342/0 (socket says 172.21.15.86:59342) 2024-04-17T07:12:03.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f5773e00640 1 -- 172.21.15.86:0/980111093 learned_addr learned my addr 172.21.15.86:0/980111093 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:03.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f5781800640 1 --2- 172.21.15.86:0/980111093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab57a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f5780e00640 1 --2- 172.21.15.86:0/980111093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57a6e0 0x5604ab57f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f5773e00640 1 -- 172.21.15.86:0/980111093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 msgr2=0x5604ab57a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:12:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f5773e00640 1 --2- 172.21.15.86:0/980111093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab57a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f5773e00640 1 -- 172.21.15.86:0/980111093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57a6e0 msgr2=0x5604ab57f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:12:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f5773e00640 1 --2- 172.21.15.86:0/980111093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57a6e0 0x5604ab57f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.066+0000 7f5773e00640 1 -- 172.21.15.86:0/980111093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604ab1ef2d0 con 0x5604ab57f850 2024-04-17T07:12:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.067+0000 7f577a000640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57600a3c30 0x7f57600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.067+0000 7f5773e00640 1 --2- 172.21.15.86:0/980111093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab581ce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f57640f6ae0 tx=0x7f57640f6bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4eab24ba52ce01 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.067+0000 7f577a000640 1 -- 172.21.15.86:0/3095990882 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f57640f2ab0 con 0x7f577c1031a0 2024-04-17T07:12:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.067+0000 7f5781800640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57600a3c30 0x7f57600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.067+0000 7f5772000640 1 -- 172.21.15.86:0/980111093 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57640bad90 con 0x5604ab57f850 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.067+0000 7f5772000640 1 -- 172.21.15.86:0/980111093 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5764004c50 con 0x5604ab57f850 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.067+0000 7f5772000640 1 -- 172.21.15.86:0/980111093 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57640be450 con 0x5604ab57f850 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f57863e0740 1 -- 172.21.15.86:0/980111093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 msgr2=0x5604ab581ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f57863e0740 1 --2- 172.21.15.86:0/980111093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab581ce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f57640f6ae0 tx=0x7f57640f6bd0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f5781800640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57600a3c30 0x7f57600a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5604ab57af40 tx=0x7f5774009430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f57863e0740 1 -- 172.21.15.86:0/980111093 shutdown_connections 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f57863e0740 1 --2- 172.21.15.86:0/980111093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab581ce0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f57863e0740 1 --2- 172.21.15.86:0/980111093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57a6e0 0x5604ab57f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f57863e0740 1 --2- 172.21.15.86:0/980111093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab57a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f57863e0740 1 -- 172.21.15.86:0/980111093 >> 172.21.15.86:0/980111093 conn(0x5604ab570e00 msgr2=0x5604ab571200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f57863e0740 1 -- 172.21.15.86:0/980111093 shutdown_connections 2024-04-17T07:12:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f57863e0740 1 -- 172.21.15.86:0/980111093 wait complete. 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.068+0000 7f57863e0740 1 -- start start 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f57863e0740 1 Processor -- start 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f57863e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab583f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f57863e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57a6e0 0x5604ab584460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f57863e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57f850 0x5604ab58b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f57863e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5604ab5825c0 con 0x5604ab57a6e0 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f57863e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5604ab496590 con 0x5604ab57f850 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f57863e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604ab582740 con 0x5604ab579cf0 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f57863e0740 10 client.0 _ll_register_callbacks cb 0x5604ab1eeac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f5773e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57a6e0 0x5604ab584460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f5773e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57a6e0 0x5604ab584460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59346/0 (socket says 172.21.15.86:59346) 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.069+0000 7f5773e00640 1 -- 172.21.15.86:0/3558676663 learned_addr learned my addr 172.21.15.86:0/3558676663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f5781800640 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57f850 0x5604ab58b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f5780e00640 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab583f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f5773e00640 1 -- 172.21.15.86:0/3558676663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 msgr2=0x5604ab583f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:12:03.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f5773e00640 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab583f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f5773e00640 1 -- 172.21.15.86:0/3558676663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57f850 msgr2=0x5604ab58b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:12:03.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f5773e00640 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57f850 0x5604ab58b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f5773e00640 1 -- 172.21.15.86:0/3558676663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604ab2b96b0 con 0x5604ab57a6e0 2024-04-17T07:12:03.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f5773e00640 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57a6e0 0x5604ab584460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f57640bb470 tx=0x7f5764005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4d00b32ba0272c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:03.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f576aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:12:03.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f576aa00640 1 -- 172.21.15.86:0/3558676663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57640be630 con 0x5604ab57a6e0 2024-04-17T07:12:03.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f576aa00640 1 -- 172.21.15.86:0/3558676663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f57640bcb90 con 0x5604ab57a6e0 2024-04-17T07:12:03.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f57863e0740 1 -- 172.21.15.86:0/3558676663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5764004c50 con 0x5604ab57a6e0 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f576aa00640 1 -- 172.21.15.86:0/3558676663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f57640be8c0 con 0x5604ab57a6e0 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f57863e0740 10 client.14823 fetch_fsmap learned FSMap version 17 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.070+0000 7f57863e0740 1 -- 172.21.15.86:0/3558676663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5604ab5825c0 con 0x5604ab57a6e0 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.071+0000 7f576aa00640 1 -- 172.21.15.86:0/3558676663 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5764004c50 con 0x5604ab57a6e0 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.071+0000 7f576aa00640 1 -- 172.21.15.86:0/3558676663 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 17) ==== 8343+0+0 (secure 0 0 0) 0x7f57640bed50 con 0x5604ab57a6e0 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.071+0000 7f57863e0740 10 client.14823 fetch_fsmap finished waiting for FSMap version 17 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.071+0000 7f57863e0740 10 client.14823 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.071+0000 7f57863e0740 20 client.14823 populate_metadata read hostname 'smithi086' 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.071+0000 7f57863e0740 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5604ab590af0 0x5604ab591f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.071+0000 7f57863e0740 4 client.14823 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:12:03.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.071+0000 7f57863e0740 1 -- 172.21.15.86:0/3558676663 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604ab4ea3a0 con 0x5604ab590af0 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.072+0000 7f5780e00640 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5604ab590af0 0x5604ab591f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.073+0000 7f5780e00640 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5604ab590af0 0x5604ab591f60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bedcebb6b87d98de server_cookie=84d583fca844b049 in_seq=0 out_seq=0 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.073+0000 7f576aa00640 10 client.14823 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f576aa00640 1 -- 172.21.15.86:0/3558676663 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604ab4ea3a0 con 0x5604ab590af0 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f576aa00640 10 client.14823 handle_command_reply: tid=0 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f57863e0740 1 client.14823 shutdown 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f57863e0740 1 -- 172.21.15.86:0/3558676663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57a6e0 msgr2=0x5604ab584460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f57863e0740 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57a6e0 0x5604ab584460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f57640bb470 tx=0x7f5764005b50 comp rx=0 tx=0).stop 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f57863e0740 1 -- 172.21.15.86:0/3558676663 shutdown_connections 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f57863e0740 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5604ab590af0 0x5604ab591f60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f57863e0740 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57f850 0x5604ab58b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f57863e0740 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57a6e0 0x5604ab584460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f576aa00640 0 client.14823 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:12:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f57863e0740 1 --2- 172.21.15.86:0/3558676663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab583f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.075+0000 7f57863e0740 1 -- 172.21.15.86:0/3558676663 >> 172.21.15.86:0/3558676663 conn(0x5604ab570e00 msgr2=0x5604ab570090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.076+0000 7f57863e0740 1 -- 172.21.15.86:0/3558676663 shutdown_connections 2024-04-17T07:12:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.076+0000 7f57863e0740 1 -- 172.21.15.86:0/3558676663 wait complete. 2024-04-17T07:12:03.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.076+0000 7f57863e0740 20 client.14823 trim_cache size 0 max 16384 2024-04-17T07:12:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.095+0000 7f57863e0740 1 Processor -- start 2024-04-17T07:12:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.095+0000 7f57863e0740 1 -- start start 2024-04-17T07:12:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.095+0000 7f57863e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab579cf0 0x5604ab583b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.095+0000 7f57863e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab57a6e0 0x5604ab584070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.095+0000 7f57863e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57f850 0x5604ab56f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.095+0000 7f57863e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5604ab5902a0 con 0x5604ab579cf0 2024-04-17T07:12:03.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.095+0000 7f57863e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5604ab590970 con 0x5604ab57f850 2024-04-17T07:12:03.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.095+0000 7f57863e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604ab591040 con 0x5604ab57a6e0 2024-04-17T07:12:03.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5780e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab579cf0 0x5604ab583b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5780e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab579cf0 0x5604ab583b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59356/0 (socket says 172.21.15.86:59356) 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5780e00640 1 -- 172.21.15.86:0/525754310 learned_addr learned my addr 172.21.15.86:0/525754310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5781800640 1 --2- 172.21.15.86:0/525754310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57f850 0x5604ab56f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5773e00640 1 --2- 172.21.15.86:0/525754310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab57a6e0 0x5604ab584070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5780e00640 1 -- 172.21.15.86:0/525754310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab57a6e0 msgr2=0x5604ab584070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5780e00640 1 --2- 172.21.15.86:0/525754310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab57a6e0 0x5604ab584070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5780e00640 1 -- 172.21.15.86:0/525754310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57f850 msgr2=0x5604ab56f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5780e00640 1 --2- 172.21.15.86:0/525754310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57f850 0x5604ab56f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5780e00640 1 -- 172.21.15.86:0/525754310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604ab571850 con 0x5604ab579cf0 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.096+0000 7f5780e00640 1 --2- 172.21.15.86:0/525754310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab579cf0 0x5604ab583b30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5604ab57aae0 tx=0x7f576c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2181f43daf8cb7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.097+0000 7f5771600640 1 -- 172.21.15.86:0/525754310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c012b80 con 0x5604ab579cf0 2024-04-17T07:12:03.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.097+0000 7f5771600640 1 -- 172.21.15.86:0/525754310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f576c01a860 con 0x5604ab579cf0 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.097+0000 7f5771600640 1 -- 172.21.15.86:0/525754310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c010be0 con 0x5604ab579cf0 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.097+0000 7f57863e0740 1 -- 172.21.15.86:0/525754310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab579cf0 msgr2=0x5604ab583b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.097+0000 7f57863e0740 1 --2- 172.21.15.86:0/525754310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab579cf0 0x5604ab583b30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5604ab57aae0 tx=0x7f576c005d60 comp rx=0 tx=0).stop 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.097+0000 7f57863e0740 1 -- 172.21.15.86:0/525754310 shutdown_connections 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.097+0000 7f57863e0740 1 --2- 172.21.15.86:0/525754310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57f850 0x5604ab56f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.097+0000 7f57863e0740 1 --2- 172.21.15.86:0/525754310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab57a6e0 0x5604ab584070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.097+0000 7f57863e0740 1 --2- 172.21.15.86:0/525754310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab579cf0 0x5604ab583b30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.098+0000 7f57863e0740 1 -- 172.21.15.86:0/525754310 >> 172.21.15.86:0/525754310 conn(0x5604ab570e00 msgr2=0x5604ab57bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.098+0000 7f57863e0740 1 -- 172.21.15.86:0/525754310 shutdown_connections 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.098+0000 7f57863e0740 1 -- 172.21.15.86:0/525754310 wait complete. 2024-04-17T07:12:03.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.098+0000 7f57863e0740 1 -- start start 2024-04-17T07:12:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.098+0000 7f57863e0740 1 Processor -- start 2024-04-17T07:12:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f57863e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab578ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f57863e0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57a6e0 0x5604ab5793e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f57863e0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab58b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f57863e0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5604ab591de0 con 0x5604ab57f850 2024-04-17T07:12:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f57863e0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5604ab591710 con 0x5604ab57a6e0 2024-04-17T07:12:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f57863e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604ab5924b0 con 0x5604ab579cf0 2024-04-17T07:12:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f57863e0740 10 client.0 _ll_register_callbacks cb 0x7f57640f6980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f5781800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab58b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f5781800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab58b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59370/0 (socket says 172.21.15.86:59370) 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f5781800640 1 -- 172.21.15.86:0/2152915675 learned_addr learned my addr 172.21.15.86:0/2152915675 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f5781800640 1 -- 172.21.15.86:0/2152915675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 msgr2=0x5604ab578ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f5773e00640 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57a6e0 0x5604ab5793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f5781800640 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab578ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f5781800640 1 -- 172.21.15.86:0/2152915675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57a6e0 msgr2=0x5604ab5793e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f5781800640 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57a6e0 0x5604ab5793e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.099+0000 7f5781800640 1 -- 172.21.15.86:0/2152915675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604ab5776b0 con 0x5604ab57f850 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f5781800640 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab58b460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5774017bf0 tx=0x7f5774004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bd0445c1d5d5d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f576aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f576aa00640 1 -- 172.21.15.86:0/2152915675 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774016920 con 0x5604ab57f850 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f57863e0740 1 -- 172.21.15.86:0/2152915675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c01a860 con 0x5604ab57f850 2024-04-17T07:12:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f576aa00640 1 -- 172.21.15.86:0/2152915675 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f5774016ac0 con 0x5604ab57f850 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f576aa00640 1 -- 172.21.15.86:0/2152915675 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5774016cc0 con 0x5604ab57f850 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f57863e0740 10 client.14829 fetch_fsmap learned FSMap version 17 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f57863e0740 1 -- 172.21.15.86:0/2152915675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5604ab5fb840 con 0x5604ab57f850 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f576aa00640 1 -- 172.21.15.86:0/2152915675 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f576c01a860 con 0x5604ab57f850 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f576aa00640 1 -- 172.21.15.86:0/2152915675 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 17) ==== 8343+0+0 (secure 0 0 0) 0x7f5774021420 con 0x5604ab57f850 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.100+0000 7f57863e0740 10 client.14829 fetch_fsmap finished waiting for FSMap version 17 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.101+0000 7f57863e0740 10 client.14829 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.101+0000 7f57863e0740 20 client.14829 populate_metadata read hostname 'smithi086' 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.101+0000 7f57863e0740 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5604ab600530 0x5604ab5ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.101+0000 7f57863e0740 4 client.14829 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.101+0000 7f57863e0740 1 -- 172.21.15.86:0/2152915675 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "status"}) -- 0x5604ab4e4210 con 0x5604ab600530 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.101+0000 7f5780e00640 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5604ab600530 0x5604ab5ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.102+0000 7f5780e00640 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5604ab600530 0x5604ab5ff240 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5941133199a342f5 server_cookie=4e0ba3a1041a0631 in_seq=0 out_seq=0 2024-04-17T07:12:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.102+0000 7f576aa00640 10 client.14829 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:12:03.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f576aa00640 1 -- 172.21.15.86:0/2152915675 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5604ab4e4210 con 0x5604ab600530 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f576aa00640 10 client.14829 handle_command_reply: tid=0 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 client.14829 shutdown 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 -- 172.21.15.86:0/2152915675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 msgr2=0x5604ab58b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab58b460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5774017bf0 tx=0x7f5774004b50 comp rx=0 tx=0).stop 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 -- 172.21.15.86:0/2152915675 shutdown_connections 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5604ab600530 0x5604ab5ff240 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5604ab57f850 0x5604ab58b460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5604ab57a6e0 0x5604ab5793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 --2- 172.21.15.86:0/2152915675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ab579cf0 0x5604ab578ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 -- 172.21.15.86:0/2152915675 >> 172.21.15.86:0/2152915675 conn(0x5604ab570e00 msgr2=0x5604ab574940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f576aa00640 0 client.14829 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:12:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 -- 172.21.15.86:0/2152915675 shutdown_connections 2024-04-17T07:12:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 1 -- 172.21.15.86:0/2152915675 wait complete. 2024-04-17T07:12:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.103+0000 7f57863e0740 20 client.14829 trim_cache size 0 max 16384 2024-04-17T07:12:03.108 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:12:03.108 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:12:03.108 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-17T07:12:03.108 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 34490, 2024-04-17T07:12:03.108 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:12:03.108 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:12:03.109 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:12:03.109 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 7.6363800580000003, 2024-04-17T07:12:03.109 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 17, 2024-04-17T07:12:03.109 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-04-17T07:12:03.109 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-17T07:12:03.109 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 14.181118360999999 2024-04-17T07:12:03.109 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:12:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 -- 172.21.15.86:0/3095990882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57600a3c30 msgr2=0x7f57600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57600a3c30 0x7f57600a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5604ab57af40 tx=0x7f5774009430 comp rx=0 tx=0).stop 2024-04-17T07:12:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 -- 172.21.15.86:0/3095990882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577c1031a0 msgr2=0x7f577c1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577c1031a0 0x7f577c1987c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5764000c00 tx=0x7f57640042f0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 -- 172.21.15.86:0/3095990882 shutdown_connections 2024-04-17T07:12:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57600a3c30 0x7f57600a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f577c103b60 0x7f577c19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f577c1031a0 0x7f577c1987c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 --2- 172.21.15.86:0/3095990882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f577c101fb0 0x7f577c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:03.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 -- 172.21.15.86:0/3095990882 >> 172.21.15.86:0/3095990882 conn(0x7f577c0fd540 msgr2=0x7f577c1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:03.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 -- 172.21.15.86:0/3095990882 shutdown_connections 2024-04-17T07:12:03.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:03.104+0000 7f5783800640 1 -- 172.21.15.86:0/3095990882 wait complete. 2024-04-17T07:12:03.118 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T07:12:03.119 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:12:03.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph fs ls 2024-04-17T07:12:03.553 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:03 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2224438428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:03 smithi086 ceph-mon[31733]: pgmap v367: 129 pgs: 7 creating+peering, 119 active+clean, 3 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-04-17T07:12:04.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:03 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2224438428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:03 smithi098 ceph-mon[35418]: pgmap v367: 129 pgs: 7 creating+peering, 119 active+clean, 3 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-04-17T07:12:04.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:03 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2224438428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:04.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:03 smithi103 ceph-mon[35495]: pgmap v367: 129 pgs: 7 creating+peering, 119 active+clean, 3 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-04-17T07:12:05.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.805+0000 7ff099a00700 1 -- 172.21.15.86:0/253404971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff09410f450 msgr2=0x7ff094111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:05.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.805+0000 7ff099a00700 1 --2- 172.21.15.86:0/253404971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff09410f450 0x7ff094111840 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00b0a0 tx=0x7ff08c00b3b0 comp rx=0 tx=0).stop 2024-04-17T07:12:05.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.805+0000 7ff099a00700 1 -- 172.21.15.86:0/253404971 shutdown_connections 2024-04-17T07:12:05.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.805+0000 7ff099a00700 1 --2- 172.21.15.86:0/253404971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff09410f450 0x7ff094111840 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:05.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.805+0000 7ff099a00700 1 --2- 172.21.15.86:0/253404971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff094074c80 0x7ff094075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:05.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.805+0000 7ff099a00700 1 --2- 172.21.15.86:0/253404971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0940755a0 0x7ff09410ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:05.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.805+0000 7ff099a00700 1 -- 172.21.15.86:0/253404971 >> 172.21.15.86:0/253404971 conn(0x7ff094079bb0 msgr2=0x7ff094079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:05.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.805+0000 7ff099a00700 1 -- 172.21.15.86:0/253404971 shutdown_connections 2024-04-17T07:12:05.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.806+0000 7ff099a00700 1 -- 172.21.15.86:0/253404971 wait complete. 2024-04-17T07:12:05.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.807+0000 7ff099a00700 1 Processor -- start 2024-04-17T07:12:05.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.807+0000 7ff099a00700 1 -- start start 2024-04-17T07:12:05.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff099a00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff094074c80 0x7ff09419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:05.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff099a00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0940755a0 0x7ff09419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:05.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff099a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff09410f450 0x7ff09419fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:05.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff099a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff094114e20 con 0x7ff09410f450 2024-04-17T07:12:05.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff099a00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff094114ca0 con 0x7ff094074c80 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff099a00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff094114fa0 con 0x7ff0940755a0 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff093400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff09410f450 0x7ff09419fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff093400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff09410f450 0x7ff09419fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59384/0 (socket says 172.21.15.86:59384) 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff093400700 1 -- 172.21.15.86:0/2678056871 learned_addr learned my addr 172.21.15.86:0/2678056871 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff08be00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0940755a0 0x7ff09419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff092a00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff094074c80 0x7ff09419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff093400700 1 -- 172.21.15.86:0/2678056871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0940755a0 msgr2=0x7ff09419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff093400700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0940755a0 0x7ff09419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff093400700 1 -- 172.21.15.86:0/2678056871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff094074c80 msgr2=0x7ff09419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.808+0000 7ff093400700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff094074c80 0x7ff09419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.809+0000 7ff093400700 1 -- 172.21.15.86:0/2678056871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0941a0210 con 0x7ff09410f450 2024-04-17T07:12:05.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.809+0000 7ff092a00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff094074c80 0x7ff09419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:05.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.809+0000 7ff093400700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff09410f450 0x7ff09419fad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0062a0 tx=0x7ff08c004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:05.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.809+0000 7ff08b400700 1 -- 172.21.15.86:0/2678056871 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c021070 con 0x7ff09410f450 2024-04-17T07:12:05.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.809+0000 7ff08b400700 1 -- 172.21.15.86:0/2678056871 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff08c003c00 con 0x7ff09410f450 2024-04-17T07:12:05.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.809+0000 7ff08b400700 1 -- 172.21.15.86:0/2678056871 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c004740 con 0x7ff09410f450 2024-04-17T07:12:05.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.809+0000 7ff099a00700 1 -- 172.21.15.86:0/2678056871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0941a04a0 con 0x7ff09410f450 2024-04-17T07:12:05.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.809+0000 7ff099a00700 1 -- 172.21.15.86:0/2678056871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0941a7d80 con 0x7ff09410f450 2024-04-17T07:12:05.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.811+0000 7ff099a00700 1 -- 172.21.15.86:0/2678056871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff094069740 con 0x7ff09410f450 2024-04-17T07:12:05.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.813+0000 7ff08b400700 1 -- 172.21.15.86:0/2678056871 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff08c003db0 con 0x7ff09410f450 2024-04-17T07:12:05.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.814+0000 7ff08b400700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0780a3cc0 0x7ff0780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:05.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.814+0000 7ff08b400700 1 -- 172.21.15.86:0/2678056871 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff08c00f020 con 0x7ff09410f450 2024-04-17T07:12:05.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.814+0000 7ff092a00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0780a3cc0 0x7ff0780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:05.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.815+0000 7ff092a00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0780a3cc0 0x7ff0780a6180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff084005fd0 tx=0x7ff084005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:05.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:05.816+0000 7ff08b400700 1 -- 172.21.15.86:0/2678056871 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff08c02a020 con 0x7ff09410f450 2024-04-17T07:12:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.095+0000 7ff099a00700 1 -- 172.21.15.86:0/2678056871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff0941a7f50 con 0x7ff09410f450 2024-04-17T07:12:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.096+0000 7ff08b400700 1 -- 172.21.15.86:0/2678056871 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7ff08c09e7f0 con 0x7ff09410f450 2024-04-17T07:12:06.096 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-17T07:12:06.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.098+0000 7ff099a00700 1 -- 172.21.15.86:0/2678056871 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0780a3cc0 msgr2=0x7ff0780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:06.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.098+0000 7ff099a00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0780a3cc0 0x7ff0780a6180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff084005fd0 tx=0x7ff084005e20 comp rx=0 tx=0).stop 2024-04-17T07:12:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.098+0000 7ff099a00700 1 -- 172.21.15.86:0/2678056871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff09410f450 msgr2=0x7ff09419fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.098+0000 7ff099a00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff09410f450 0x7ff09419fad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0062a0 tx=0x7ff08c004d90 comp rx=0 tx=0).stop 2024-04-17T07:12:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.099+0000 7ff099a00700 1 -- 172.21.15.86:0/2678056871 shutdown_connections 2024-04-17T07:12:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.099+0000 7ff099a00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0780a3cc0 0x7ff0780a6180 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.099+0000 7ff099a00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff09410f450 0x7ff09419fad0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:06.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.099+0000 7ff099a00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff094074c80 0x7ff09419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:06.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.099+0000 7ff099a00700 1 --2- 172.21.15.86:0/2678056871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0940755a0 0x7ff09419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:06.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.099+0000 7ff099a00700 1 -- 172.21.15.86:0/2678056871 >> 172.21.15.86:0/2678056871 conn(0x7ff094079bb0 msgr2=0x7ff0940ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:06.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.099+0000 7ff099a00700 1 -- 172.21.15.86:0/2678056871 shutdown_connections 2024-04-17T07:12:06.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:06.099+0000 7ff099a00700 1 -- 172.21.15.86:0/2678056871 wait complete. 2024-04-17T07:12:06.447 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:06 smithi086 ceph-mon[31733]: pgmap v368: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-17T07:12:06.448 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:06 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2678056871' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:12:06.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:06 smithi098 ceph-mon[35418]: pgmap v368: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-17T07:12:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:06 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2678056871' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:12:06.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:06 smithi103 ceph-mon[35495]: pgmap v368: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-17T07:12:06.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:06 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2678056871' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:12:06.909 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-17T07:12:06.909 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T07:12:06.909 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-17T07:12:06.909 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-17T07:12:06.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:12:06.909 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-17T07:12:06.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T07:12:06.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T07:12:06.910 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-17T07:12:06.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:06.910 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-17T07:12:06.966 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T07:12:06.966 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T07:12:06.966 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T07:12:06.966 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T07:12:06.967 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41970sec preferred_lft 41970sec 2024-04-17T07:12:06.968 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-17T07:12:06.968 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:12:06.969 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T07:12:06.969 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:12:06.969 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-17T07:12:06.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-17T07:12:06.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-17T07:12:06.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-17T07:12:06.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T07:12:06.969 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-17T07:12:07.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:12:07.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:12:07.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:07.534 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T07:12:07.610 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-17T07:12:07.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:07.615 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-17T07:12:07.673 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-17T07:12:07.673 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T07:12:07.673 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T07:12:07.673 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T07:12:07.673 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T07:12:07.674 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:12:07.674 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-17T07:12:07.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T07:12:07.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T07:12:07.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T07:12:07.674 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-17T07:12:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:12:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:08 smithi086 ceph-mon[31733]: pgmap v369: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-17T07:12:08.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:12:08.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:08 smithi098 ceph-mon[35418]: pgmap v369: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-17T07:12:08.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:08.511 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-17T07:12:08.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:08 smithi103 ceph-mon[35495]: pgmap v369: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-17T07:12:08.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:08.565 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-17T07:12:08.620 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:12:08.620 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-17T07:12:08.620 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:12:08.620 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-17T07:12:08.620 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-17T07:12:08.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:12:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:12:08.831 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-17T07:12:08.831 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:12:08.831 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-17T07:12:08.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-17T07:12:08.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T07:12:08.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-17T07:12:08.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-17T07:12:08.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-17T07:12:08.831 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-17T07:12:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:12:09.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:12:09.175 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:12:09.176 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-17T07:12:09.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-17T07:12:09.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T07:12:09.176 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-17T07:12:09.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:12:09.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:12:09.402 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-17T07:12:09.402 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T07:12:09.403 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:12:09.448 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-17T07:12:09.448 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T07:12:09.449 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:12:09.538 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-17T07:12:09.627 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T07:12:09.676 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-17T07:12:09.676 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-17T07:12:09.676 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-17T07:12:09.677 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/2cd09f6d91b55586871c77946312699cfe780fb744dc7dd04fef954998dd3d6b/merged 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/daa4e0009951020f1d8e0f1a30c631b514d43a7dd9bfbcf2d1eae139c2a7eb2f/merged 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:12:09.678 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/b94ef22031a7435e285187988bebca5e4cef86fe8b13dcf0d1444c00be1fcf3c/merged 2024-04-17T07:12:09.679 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c5a576a8d0f15fde9a8605e3a9629d20764354d21e798f72c6fed568f4a86ccf/merged 2024-04-17T07:12:09.679 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5ca71d6a5746de4bdb3cc50a830763d933d1001eac8f4b450402954a84b1efed/merged 2024-04-17T07:12:09.679 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c49121dbdaf6788fba7930f24e1dc568a87dcc5c8c4158e40ff110b9a4723737/merged 2024-04-17T07:12:09.679 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/93635c477aa616f15f8542b827e2586cbbbb03a902f83386b128502d0c1009e3/merged 2024-04-17T07:12:09.679 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5c0cc5bb9c77431ff4db506e0500c3287d3fe329e5876e767259f50d1396aa30/merged 2024-04-17T07:12:09.679 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/3e44fcef40b275061b17516d4d6207e6e80be758f3e73e8d2710d71d42ccdd1a/merged 2024-04-17T07:12:09.679 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/13eace86438efe164220139509f4b1079c443786d7afc6365f2862a4ea4a1f85/merged 2024-04-17T07:12:09.679 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-17T07:12:09.679 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:12:09.679 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:12:09.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:09.680 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-17T07:12:09.727 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T07:12:09.728 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-17T07:12:09.732 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-17T07:12:09.808 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T07:12:09.852 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-17T07:12:09.852 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-17T07:12:09.852 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-17T07:12:09.852 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-17T07:12:09.852 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-17T07:12:09.852 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-17T07:12:09.852 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-17T07:12:09.852 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T07:12:09.853 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/2cd09f6d91b55586871c77946312699cfe780fb744dc7dd04fef954998dd3d6b/merged 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/daa4e0009951020f1d8e0f1a30c631b514d43a7dd9bfbcf2d1eae139c2a7eb2f/merged 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/b94ef22031a7435e285187988bebca5e4cef86fe8b13dcf0d1444c00be1fcf3c/merged 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c5a576a8d0f15fde9a8605e3a9629d20764354d21e798f72c6fed568f4a86ccf/merged 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5ca71d6a5746de4bdb3cc50a830763d933d1001eac8f4b450402954a84b1efed/merged 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c49121dbdaf6788fba7930f24e1dc568a87dcc5c8c4158e40ff110b9a4723737/merged 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/93635c477aa616f15f8542b827e2586cbbbb03a902f83386b128502d0c1009e3/merged 2024-04-17T07:12:09.854 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5c0cc5bb9c77431ff4db506e0500c3287d3fe329e5876e767259f50d1396aa30/merged 2024-04-17T07:12:09.855 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/3e44fcef40b275061b17516d4d6207e6e80be758f3e73e8d2710d71d42ccdd1a/merged 2024-04-17T07:12:09.855 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/13eace86438efe164220139509f4b1079c443786d7afc6365f2862a4ea4a1f85/merged 2024-04-17T07:12:09.855 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-17T07:12:09.855 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:12:09.855 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:12:09.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:09.856 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-17T07:12:09.935 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:2024-04-17T07:12:09.936+0000 7fd1d8a1d480 -1 init, newargv = 0x5588604112d0 newargc=15 2024-04-17T07:12:09.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[73419]: starting ceph client 2024-04-17T07:12:09.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[73419]: starting fuse 2024-04-17T07:12:10.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:10 smithi098 ceph-mon[35418]: pgmap v370: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:12:10.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:10 smithi103 ceph-mon[35495]: pgmap v370: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:12:10.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:10 smithi086 ceph-mon[31733]: pgmap v370: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:12:11.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-17T07:12:11.151 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T07:12:11.210 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-17T07:12:11.210 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-17T07:12:11.210 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-17T07:12:11.210 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-17T07:12:11.211 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/2cd09f6d91b55586871c77946312699cfe780fb744dc7dd04fef954998dd3d6b/merged 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/daa4e0009951020f1d8e0f1a30c631b514d43a7dd9bfbcf2d1eae139c2a7eb2f/merged 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/b94ef22031a7435e285187988bebca5e4cef86fe8b13dcf0d1444c00be1fcf3c/merged 2024-04-17T07:12:11.212 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c5a576a8d0f15fde9a8605e3a9629d20764354d21e798f72c6fed568f4a86ccf/merged 2024-04-17T07:12:11.213 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5ca71d6a5746de4bdb3cc50a830763d933d1001eac8f4b450402954a84b1efed/merged 2024-04-17T07:12:11.213 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c49121dbdaf6788fba7930f24e1dc568a87dcc5c8c4158e40ff110b9a4723737/merged 2024-04-17T07:12:11.213 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/93635c477aa616f15f8542b827e2586cbbbb03a902f83386b128502d0c1009e3/merged 2024-04-17T07:12:11.213 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5c0cc5bb9c77431ff4db506e0500c3287d3fe329e5876e767259f50d1396aa30/merged 2024-04-17T07:12:11.213 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/3e44fcef40b275061b17516d4d6207e6e80be758f3e73e8d2710d71d42ccdd1a/merged 2024-04-17T07:12:11.213 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/13eace86438efe164220139509f4b1079c443786d7afc6365f2862a4ea4a1f85/merged 2024-04-17T07:12:11.213 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-17T07:12:11.213 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:12:11.213 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:12:11.213 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-17T07:12:11.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:11.214 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-17T07:12:11.269 INFO:teuthology.orchestra.run.smithi086.stdout:119 2024-04-17T07:12:11.269 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-17T07:12:11.270 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.admin")) 2024-04-17T07:12:11.271 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-17T07:12:11.439 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.admin.73419.asok 2024-04-17T07:12:11.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:12:11.458 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73419.asok 2024-04-17T07:12:11.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:11.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73419.asok status 2024-04-17T07:12:11.691 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:12:11.691 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "admin", 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "73419", 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 14841, 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-17T07:12:11.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 14841 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3996643040 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3996643040 2024-04-17T07:12:11.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:12:11.694 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.14841 192.168.0.1:0/3996643040", 2024-04-17T07:12:11.694 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/3996643040", 2024-04-17T07:12:11.694 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-17T07:12:11.694 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 17, 2024-04-17T07:12:11.694 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 89, 2024-04-17T07:12:11.694 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-17T07:12:11.694 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-17T07:12:11.694 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-17T07:12:11.694 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:12:11.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:11.707 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-17T07:12:11.707 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:12:11.761 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-17T07:12:11.761 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:12:11.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:11.761 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:12:11.856 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-17T07:12:11.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T07:12:12.012 INFO:teuthology.orchestra.run.smithi086.stdout:# file: . 2024-04-17T07:12:12.012 INFO:teuthology.orchestra.run.smithi086.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-17T07:12:12.013 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:12.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:12:12.023 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:12:12.023 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T07:12:12.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:12:12.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.289+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3899614086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0078500 msgr2=0x7fa5c0078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.289+0000 7fa5c5a00640 1 --2- 172.21.15.86:0/3899614086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0078500 0x7fa5c0078da0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009ca0 tx=0x7fa5b802c100 comp rx=0 tx=0).stop 2024-04-17T07:12:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.289+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3899614086 shutdown_connections 2024-04-17T07:12:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.289+0000 7fa5c5a00640 1 --2- 172.21.15.86:0/3899614086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0078500 0x7fa5c0078da0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.289+0000 7fa5c5a00640 1 --2- 172.21.15.86:0/3899614086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c0077b60 0x7fa5c0077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.289+0000 7fa5c5a00640 1 --2- 172.21.15.86:0/3899614086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c0076920 0x7fa5c0076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.289+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3899614086 >> 172.21.15.86:0/3899614086 conn(0x7fa5c00ff570 msgr2=0x7fa5c0101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.290+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3899614086 shutdown_connections 2024-04-17T07:12:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.290+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3899614086 wait complete. 2024-04-17T07:12:12.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5c5a00640 1 Processor -- start 2024-04-17T07:12:12.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5c5a00640 1 -- start start 2024-04-17T07:12:12.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5c5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0076920 0x7fa5c019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5c5a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c0077b60 0x7fa5c019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5c5a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c0078500 0x7fa5c01a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5c5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c0113f10 con 0x7fa5c0076920 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5c5a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c0113d90 con 0x7fa5c0078500 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5c5a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c0114090 con 0x7fa5c0077b60 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0076920 0x7fa5c019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0076920 0x7fa5c019e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49642/0 (socket says 172.21.15.86:49642) 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.291+0000 7fa5bfe00640 1 -- 172.21.15.86:0/3910688305 learned_addr learned my addr 172.21.15.86:0/3910688305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5c5000640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c0078500 0x7fa5c01a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5b7e00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c0077b60 0x7fa5c019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5bfe00640 1 -- 172.21.15.86:0/3910688305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c0077b60 msgr2=0x7fa5c019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5bfe00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c0077b60 0x7fa5c019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5bfe00640 1 -- 172.21.15.86:0/3910688305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c0078500 msgr2=0x7fa5c01a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5bfe00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c0078500 0x7fa5c01a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5bfe00640 1 -- 172.21.15.86:0/3910688305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c01a60b0 con 0x7fa5c0076920 2024-04-17T07:12:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5b7e00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c0077b60 0x7fa5c019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5bfe00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0076920 0x7fa5c019e490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000c920 tx=0x7fa5b000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5bd600640 1 -- 172.21.15.86:0/3910688305 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0007c70 con 0x7fa5c0076920 2024-04-17T07:12:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5bd600640 1 -- 172.21.15.86:0/3910688305 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5b0007e10 con 0x7fa5c0076920 2024-04-17T07:12:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5bd600640 1 -- 172.21.15.86:0/3910688305 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0018820 con 0x7fa5c0076920 2024-04-17T07:12:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.292+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3910688305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c01a63a0 con 0x7fa5c0076920 2024-04-17T07:12:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.293+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3910688305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c01a68f0 con 0x7fa5c0076920 2024-04-17T07:12:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.294+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3910688305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c0110950 con 0x7fa5c0076920 2024-04-17T07:12:12.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.296+0000 7fa5bd600640 1 -- 172.21.15.86:0/3910688305 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa5b00189c0 con 0x7fa5c0076920 2024-04-17T07:12:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.297+0000 7fa5bd600640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5a00a3bb0 0x7fa5a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.297+0000 7fa5bd600640 1 -- 172.21.15.86:0/3910688305 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa5b00d0b40 con 0x7fa5c0076920 2024-04-17T07:12:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.297+0000 7fa5b7e00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5a00a3bb0 0x7fa5a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.298+0000 7fa5b7e00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5a00a3bb0 0x7fa5a00a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5c019f9a0 tx=0x7fa5a8005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.300+0000 7fa5bd600640 1 -- 172.21.15.86:0/3910688305 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa5b0014030 con 0x7fa5c0076920 2024-04-17T07:12:12.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:12 smithi098 ceph-mon[35418]: pgmap v371: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:12:12.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.504+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3910688305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5c0062b90 con 0x7fa5c0076920 2024-04-17T07:12:12.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.505+0000 7fa5bd600640 1 -- 172.21.15.86:0/3910688305 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24057 (secure 0 0 0) 0x7fa5b009d310 con 0x7fa5c0076920 2024-04-17T07:12:12.506 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:12.506 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:11:59.971687+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:12:12.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.508+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3910688305 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5a00a3bb0 msgr2=0x7fa5a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.508+0000 7fa5c5a00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5a00a3bb0 0x7fa5a00a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa5c019f9a0 tx=0x7fa5a8005d90 comp rx=0 tx=0).stop 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.508+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3910688305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0076920 msgr2=0x7fa5c019e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.508+0000 7fa5c5a00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0076920 0x7fa5c019e490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000c920 tx=0x7fa5b000cde0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.509+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3910688305 shutdown_connections 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.509+0000 7fa5c5a00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5a00a3bb0 0x7fa5a00a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.509+0000 7fa5c5a00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa5c0078500 0x7fa5c01a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.509+0000 7fa5c5a00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5c0077b60 0x7fa5c019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.509+0000 7fa5c5a00640 1 --2- 172.21.15.86:0/3910688305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c0076920 0x7fa5c019e490 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.509+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3910688305 >> 172.21.15.86:0/3910688305 conn(0x7fa5c00ff570 msgr2=0x7fa5c0101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.509+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3910688305 shutdown_connections 2024-04-17T07:12:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.509+0000 7fa5c5a00640 1 -- 172.21.15.86:0/3910688305 wait complete. 2024-04-17T07:12:12.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:12 smithi086 ceph-mon[31733]: pgmap v371: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:12:12.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:12:12.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:12:12.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:12 smithi103 ceph-mon[35495]: pgmap v371: 129 pgs: 129 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-17T07:12:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.750+0000 7efd6f000640 1 -- 172.21.15.86:0/2667975726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68103930 msgr2=0x7efd6810a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.750+0000 7efd6f000640 1 --2- 172.21.15.86:0/2667975726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68103930 0x7efd6810a1c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efd60009ca0 tx=0x7efd6002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.750+0000 7efd6f000640 1 -- 172.21.15.86:0/2667975726 shutdown_connections 2024-04-17T07:12:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.750+0000 7efd6f000640 1 --2- 172.21.15.86:0/2667975726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68103930 0x7efd6810a1c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.750+0000 7efd6f000640 1 --2- 172.21.15.86:0/2667975726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd68102f70 0x7efd681033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.750+0000 7efd6f000640 1 --2- 172.21.15.86:0/2667975726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd68101d80 0x7efd681021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.750+0000 7efd6f000640 1 -- 172.21.15.86:0/2667975726 >> 172.21.15.86:0/2667975726 conn(0x7efd680fd2f0 msgr2=0x7efd680ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.750+0000 7efd6f000640 1 -- 172.21.15.86:0/2667975726 shutdown_connections 2024-04-17T07:12:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.750+0000 7efd6f000640 1 -- 172.21.15.86:0/2667975726 wait complete. 2024-04-17T07:12:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.751+0000 7efd6f000640 1 Processor -- start 2024-04-17T07:12:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.751+0000 7efd6f000640 1 -- start start 2024-04-17T07:12:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.751+0000 7efd6f000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd68101d80 0x7efd68197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.751+0000 7efd6f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68102f70 0x7efd68198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.751+0000 7efd6f000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd68103930 0x7efd6819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.751+0000 7efd6f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810d8b0 con 0x7efd68102f70 2024-04-17T07:12:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.751+0000 7efd6f000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810d730 con 0x7efd68101d80 2024-04-17T07:12:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.751+0000 7efd6f000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efd6810da30 con 0x7efd68103930 2024-04-17T07:12:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68102f70 0x7efd68198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68102f70 0x7efd68198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49660/0 (socket says 172.21.15.86:49660) 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6d200640 1 -- 172.21.15.86:0/3690832595 learned_addr learned my addr 172.21.15.86:0/3690832595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6e600640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd68103930 0x7efd6819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6dc00640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd68101d80 0x7efd68197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6d200640 1 -- 172.21.15.86:0/3690832595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd68103930 msgr2=0x7efd6819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6d200640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd68103930 0x7efd6819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6d200640 1 -- 172.21.15.86:0/3690832595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd68101d80 msgr2=0x7efd68197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6d200640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd68101d80 0x7efd68197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6d200640 1 -- 172.21.15.86:0/3690832595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6819fb40 con 0x7efd68102f70 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6dc00640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd68101d80 0x7efd68197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.752+0000 7efd6d200640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68102f70 0x7efd68198460 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efd5400d770 tx=0x7efd5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.753+0000 7efd66000640 1 -- 172.21.15.86:0/3690832595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54017070 con 0x7efd68102f70 2024-04-17T07:12:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.753+0000 7efd66000640 1 -- 172.21.15.86:0/3690832595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd540054d0 con 0x7efd68102f70 2024-04-17T07:12:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.753+0000 7efd66000640 1 -- 172.21.15.86:0/3690832595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54004cf0 con 0x7efd68102f70 2024-04-17T07:12:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.753+0000 7efd6f000640 1 -- 172.21.15.86:0/3690832595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6819fe30 con 0x7efd68102f70 2024-04-17T07:12:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.753+0000 7efd6f000640 1 -- 172.21.15.86:0/3690832595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd681a02f0 con 0x7efd68102f70 2024-04-17T07:12:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.754+0000 7efd6f000640 1 -- 172.21.15.86:0/3690832595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd68109120 con 0x7efd68102f70 2024-04-17T07:12:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.759+0000 7efd66000640 1 -- 172.21.15.86:0/3690832595 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efd54004e90 con 0x7efd68102f70 2024-04-17T07:12:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.761+0000 7efd66000640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd500a3c30 0x7efd500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.761+0000 7efd66000640 1 -- 172.21.15.86:0/3690832595 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7efd540d04b0 con 0x7efd68102f70 2024-04-17T07:12:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.762+0000 7efd6dc00640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd500a3c30 0x7efd500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:12.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.762+0000 7efd66000640 1 -- 172.21.15.86:0/3690832595 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efd5409cc70 con 0x7efd68102f70 2024-04-17T07:12:12.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.763+0000 7efd6dc00640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd500a3c30 0x7efd500a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd68102dd0 tx=0x7efd5c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:12.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.965+0000 7efd6f000640 1 -- 172.21.15.86:0/3690832595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd6802d7f0 con 0x7efd68102f70 2024-04-17T07:12:12.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.966+0000 7efd66000640 1 -- 172.21.15.86:0/3690832595 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7efd5409c570 con 0x7efd68102f70 2024-04-17T07:12:12.966 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:12:12.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.968+0000 7efd6f000640 1 -- 172.21.15.86:0/3690832595 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd500a3c30 msgr2=0x7efd500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:12.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.968+0000 7efd6f000640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd500a3c30 0x7efd500a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd68102dd0 tx=0x7efd5c005e70 comp rx=0 tx=0).stop 2024-04-17T07:12:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.968+0000 7efd6f000640 1 -- 172.21.15.86:0/3690832595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68102f70 msgr2=0x7efd68198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.968+0000 7efd6f000640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68102f70 0x7efd68198460 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efd5400d770 tx=0x7efd5400dc30 comp rx=0 tx=0).stop 2024-04-17T07:12:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.968+0000 7efd6f000640 1 -- 172.21.15.86:0/3690832595 shutdown_connections 2024-04-17T07:12:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.968+0000 7efd6f000640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd500a3c30 0x7efd500a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.968+0000 7efd6f000640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd68103930 0x7efd6819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.968+0000 7efd6f000640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd68102f70 0x7efd68198460 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.968+0000 7efd6f000640 1 --2- 172.21.15.86:0/3690832595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd68101d80 0x7efd68197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.968+0000 7efd6f000640 1 -- 172.21.15.86:0/3690832595 >> 172.21.15.86:0/3690832595 conn(0x7efd680fd2f0 msgr2=0x7efd681062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.969+0000 7efd6f000640 1 -- 172.21.15.86:0/3690832595 shutdown_connections 2024-04-17T07:12:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:12.969+0000 7efd6f000640 1 -- 172.21.15.86:0/3690832595 wait complete. 2024-04-17T07:12:12.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:12:13.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.205+0000 7fe236c00640 1 -- 172.21.15.86:0/1859289092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe230075fa0 msgr2=0x7fe2300743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.205+0000 7fe236c00640 1 --2- 172.21.15.86:0/1859289092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe230075fa0 0x7fe2300743f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe224009900 tx=0x7fe22402bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.205+0000 7fe236c00640 1 -- 172.21.15.86:0/1859289092 shutdown_connections 2024-04-17T07:12:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.205+0000 7fe236c00640 1 --2- 172.21.15.86:0/1859289092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe230109db0 0x7fe23010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.205+0000 7fe236c00640 1 --2- 172.21.15.86:0/1859289092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2300749c0 0x7fe230074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.205+0000 7fe236c00640 1 --2- 172.21.15.86:0/1859289092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe230075fa0 0x7fe2300743f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.205+0000 7fe236c00640 1 -- 172.21.15.86:0/1859289092 >> 172.21.15.86:0/1859289092 conn(0x7fe2300fd460 msgr2=0x7fe2300ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.205+0000 7fe236c00640 1 -- 172.21.15.86:0/1859289092 shutdown_connections 2024-04-17T07:12:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.205+0000 7fe236c00640 1 -- 172.21.15.86:0/1859289092 wait complete. 2024-04-17T07:12:13.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.206+0000 7fe236c00640 1 Processor -- start 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.206+0000 7fe236c00640 1 -- start start 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe236c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2300749c0 0x7fe23019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe236c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe230075fa0 0x7fe23019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe236c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe230109db0 0x7fe2301a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe236c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe23010fc00 con 0x7fe230075fa0 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe236c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe23010fa80 con 0x7fe230109db0 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe236c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe23010fd80 con 0x7fe2300749c0 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe234e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe230075fa0 0x7fe23019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe234e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe230075fa0 0x7fe23019a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49680/0 (socket says 172.21.15.86:49680) 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe234e00640 1 -- 172.21.15.86:0/220572847 learned_addr learned my addr 172.21.15.86:0/220572847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe235800640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2300749c0 0x7fe23019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.207+0000 7fe236200640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe230109db0 0x7fe2301a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:13.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe234e00640 1 -- 172.21.15.86:0/220572847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2300749c0 msgr2=0x7fe23019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe234e00640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2300749c0 0x7fe23019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe234e00640 1 -- 172.21.15.86:0/220572847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe230109db0 msgr2=0x7fe2301a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe234e00640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe230109db0 0x7fe2301a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe234e00640 1 -- 172.21.15.86:0/220572847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2301a1ed0 con 0x7fe230075fa0 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe236200640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe230109db0 0x7fe2301a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe235800640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2300749c0 0x7fe23019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe234e00640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe230075fa0 0x7fe23019a840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00d950 tx=0x7fe21c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe22e000640 1 -- 172.21.15.86:0/220572847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c017070 con 0x7fe230075fa0 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe22e000640 1 -- 172.21.15.86:0/220572847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21c009c30 con 0x7fe230075fa0 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe22e000640 1 -- 172.21.15.86:0/220572847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00f620 con 0x7fe230075fa0 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe236c00640 1 -- 172.21.15.86:0/220572847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2301a21c0 con 0x7fe230075fa0 2024-04-17T07:12:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.208+0000 7fe236c00640 1 -- 172.21.15.86:0/220572847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2301a26c0 con 0x7fe230075fa0 2024-04-17T07:12:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.209+0000 7fe236c00640 1 -- 172.21.15.86:0/220572847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe230108ae0 con 0x7fe230075fa0 2024-04-17T07:12:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.211+0000 7fe22e000640 1 -- 172.21.15.86:0/220572847 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe21c0109a0 con 0x7fe230075fa0 2024-04-17T07:12:13.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.213+0000 7fe22e000640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2180a3c30 0x7fe2180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.213+0000 7fe22e000640 1 -- 172.21.15.86:0/220572847 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe21c0d0200 con 0x7fe230075fa0 2024-04-17T07:12:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.213+0000 7fe235800640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2180a3c30 0x7fe2180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.214+0000 7fe235800640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2180a3c30 0x7fe2180a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe224006010 tx=0x7fe22402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:13.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.217+0000 7fe22e000640 1 -- 172.21.15.86:0/220572847 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe21c009de0 con 0x7fe230075fa0 2024-04-17T07:12:13.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:13 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3910688305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:13.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:13 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3690832595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:12:13.332 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:13 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3910688305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:13.332 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:13 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3690832595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:12:13.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.422+0000 7fe236c00640 1 -- 172.21.15.86:0/220572847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe23002d860 con 0x7fe230075fa0 2024-04-17T07:12:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.423+0000 7fe22e000640 1 -- 172.21.15.86:0/220572847 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fe21c09ca40 con 0x7fe230075fa0 2024-04-17T07:12:13.423 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:12:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.425+0000 7fe236c00640 1 -- 172.21.15.86:0/220572847 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2180a3c30 msgr2=0x7fe2180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.425+0000 7fe236c00640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2180a3c30 0x7fe2180a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe224006010 tx=0x7fe22402c7e0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.425+0000 7fe236c00640 1 -- 172.21.15.86:0/220572847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe230075fa0 msgr2=0x7fe23019a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.425+0000 7fe236c00640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe230075fa0 0x7fe23019a840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00d950 tx=0x7fe21c00de10 comp rx=0 tx=0).stop 2024-04-17T07:12:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.426+0000 7fe236c00640 1 -- 172.21.15.86:0/220572847 shutdown_connections 2024-04-17T07:12:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.426+0000 7fe236c00640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2180a3c30 0x7fe2180a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.426+0000 7fe236c00640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe230109db0 0x7fe2301a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.426+0000 7fe236c00640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe230075fa0 0x7fe23019a840 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.426+0000 7fe236c00640 1 --2- 172.21.15.86:0/220572847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2300749c0 0x7fe23019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.426+0000 7fe236c00640 1 -- 172.21.15.86:0/220572847 >> 172.21.15.86:0/220572847 conn(0x7fe2300fd460 msgr2=0x7fe230105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.426+0000 7fe236c00640 1 -- 172.21.15.86:0/220572847 shutdown_connections 2024-04-17T07:12:13.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.426+0000 7fe236c00640 1 -- 172.21.15.86:0/220572847 wait complete. 2024-04-17T07:12:13.445 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:12:13.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:13 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3910688305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:13 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3690832595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:12:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.662+0000 7f4183a00640 1 -- 172.21.15.86:0/1924183923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f417c103bb0 msgr2=0x7f417c10a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.662+0000 7f4183a00640 1 --2- 172.21.15.86:0/1924183923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f417c103bb0 0x7f417c10a440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4174009ca0 tx=0x7f417402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.662+0000 7f4183a00640 1 -- 172.21.15.86:0/1924183923 shutdown_connections 2024-04-17T07:12:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.662+0000 7f4183a00640 1 --2- 172.21.15.86:0/1924183923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f417c103bb0 0x7f417c10a440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.662+0000 7f4183a00640 1 --2- 172.21.15.86:0/1924183923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f417c1031f0 0x7f417c103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.662+0000 7f4183a00640 1 --2- 172.21.15.86:0/1924183923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417c102000 0x7f417c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.662+0000 7f4183a00640 1 -- 172.21.15.86:0/1924183923 >> 172.21.15.86:0/1924183923 conn(0x7f417c0fd550 msgr2=0x7f417c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:13.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.662+0000 7f4183a00640 1 -- 172.21.15.86:0/1924183923 shutdown_connections 2024-04-17T07:12:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.662+0000 7f4183a00640 1 -- 172.21.15.86:0/1924183923 wait complete. 2024-04-17T07:12:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.663+0000 7f4183a00640 1 Processor -- start 2024-04-17T07:12:13.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.663+0000 7f4183a00640 1 -- start start 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4183a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f417c102000 0x7f417c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4183a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f417c1031f0 0x7f417c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4183a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417c103bb0 0x7f417c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4183a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10dad0 con 0x7f417c102000 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4183a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10d950 con 0x7f417c103bb0 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4183a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f417c10dc50 con 0x7f417c1031f0 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4181000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f417c102000 0x7f417c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4181000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f417c102000 0x7f417c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49692/0 (socket says 172.21.15.86:49692) 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4181000640 1 -- 172.21.15.86:0/1743071141 learned_addr learned my addr 172.21.15.86:0/1743071141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f417be00640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f417c1031f0 0x7f417c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4181a00640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417c103bb0 0x7f417c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:13.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4181000640 1 -- 172.21.15.86:0/1743071141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f417c1031f0 msgr2=0x7f417c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4181000640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f417c1031f0 0x7f417c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4181000640 1 -- 172.21.15.86:0/1743071141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417c103bb0 msgr2=0x7f417c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4181000640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417c103bb0 0x7f417c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.664+0000 7f4181000640 1 -- 172.21.15.86:0/1743071141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c19fd40 con 0x7f417c102000 2024-04-17T07:12:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.665+0000 7f4181000640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f417c102000 0x7f417c198120 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f417000e9c0 tx=0x7f417000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.665+0000 7f4179600640 1 -- 172.21.15.86:0/1743071141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417000cd80 con 0x7f417c102000 2024-04-17T07:12:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.665+0000 7f4179600640 1 -- 172.21.15.86:0/1743071141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41700035f0 con 0x7f417c102000 2024-04-17T07:12:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.665+0000 7f4183a00640 1 -- 172.21.15.86:0/1743071141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417c19ffd0 con 0x7f417c102000 2024-04-17T07:12:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.665+0000 7f4179600640 1 -- 172.21.15.86:0/1743071141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170010690 con 0x7f417c102000 2024-04-17T07:12:13.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.665+0000 7f4183a00640 1 -- 172.21.15.86:0/1743071141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417c1a0490 con 0x7f417c102000 2024-04-17T07:12:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.666+0000 7f4183a00640 1 -- 172.21.15.86:0/1743071141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f417c109340 con 0x7f417c102000 2024-04-17T07:12:13.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.668+0000 7f4179600640 1 -- 172.21.15.86:0/1743071141 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f41700037a0 con 0x7f417c102000 2024-04-17T07:12:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.669+0000 7f4179600640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41640a3bb0 0x7f41640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:13.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.670+0000 7f4179600640 1 -- 172.21.15.86:0/1743071141 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f41700d0b60 con 0x7f417c102000 2024-04-17T07:12:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.670+0000 7f417be00640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41640a3bb0 0x7f41640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:13.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.670+0000 7f417be00640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41640a3bb0 0x7f41640a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f417c199630 tx=0x7f4168005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:13.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.674+0000 7f4179600640 1 -- 172.21.15.86:0/1743071141 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f417009d2a0 con 0x7f417c102000 2024-04-17T07:12:13.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.877+0000 7f4183a00640 1 -- 172.21.15.86:0/1743071141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f417c062c00 con 0x7f417c102000 2024-04-17T07:12:13.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.877+0000 7f4179600640 1 -- 172.21.15.86:0/1743071141 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f417009cba0 con 0x7f417c102000 2024-04-17T07:12:13.877 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:12:13.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.880+0000 7f4183a00640 1 -- 172.21.15.86:0/1743071141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41640a3bb0 msgr2=0x7f41640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.880+0000 7f4183a00640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41640a3bb0 0x7f41640a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f417c199630 tx=0x7f4168005f90 comp rx=0 tx=0).stop 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.880+0000 7f4183a00640 1 -- 172.21.15.86:0/1743071141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f417c102000 msgr2=0x7f417c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.880+0000 7f4183a00640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f417c102000 0x7f417c198120 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f417000e9c0 tx=0x7f417000ee80 comp rx=0 tx=0).stop 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.880+0000 7f4183a00640 1 -- 172.21.15.86:0/1743071141 shutdown_connections 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.880+0000 7f4183a00640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f41640a3bb0 0x7f41640a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.880+0000 7f4183a00640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f417c103bb0 0x7f417c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.880+0000 7f4183a00640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f417c1031f0 0x7f417c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.880+0000 7f4183a00640 1 --2- 172.21.15.86:0/1743071141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f417c102000 0x7f417c198120 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.881+0000 7f4183a00640 1 -- 172.21.15.86:0/1743071141 >> 172.21.15.86:0/1743071141 conn(0x7f417c0fd550 msgr2=0x7f417c106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.881+0000 7f4183a00640 1 -- 172.21.15.86:0/1743071141 shutdown_connections 2024-04-17T07:12:13.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:13.881+0000 7f4183a00640 1 -- 172.21.15.86:0/1743071141 wait complete. 2024-04-17T07:12:13.899 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:12:14.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.108+0000 7f3b2b200640 1 -- 172.21.15.86:0/3859692822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b2c075f40 msgr2=0x7f3b2c074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.108+0000 7f3b2b200640 1 --2- 172.21.15.86:0/3859692822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b2c075f40 0x7f3b2c074390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009900 tx=0x7f3b1c02bcd0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.108+0000 7f3b2b200640 1 -- 172.21.15.86:0/3859692822 shutdown_connections 2024-04-17T07:12:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.108+0000 7f3b2b200640 1 --2- 172.21.15.86:0/3859692822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b2c109d00 0x7f3b2c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.108+0000 7f3b2b200640 1 --2- 172.21.15.86:0/3859692822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b2c074960 0x7f3b2c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.108+0000 7f3b2b200640 1 --2- 172.21.15.86:0/3859692822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b2c075f40 0x7f3b2c074390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.108+0000 7f3b2b200640 1 -- 172.21.15.86:0/3859692822 >> 172.21.15.86:0/3859692822 conn(0x7f3b2c0fd400 msgr2=0x7f3b2c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.108+0000 7f3b2b200640 1 -- 172.21.15.86:0/3859692822 shutdown_connections 2024-04-17T07:12:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.109+0000 7f3b2b200640 1 -- 172.21.15.86:0/3859692822 wait complete. 2024-04-17T07:12:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2b200640 1 Processor -- start 2024-04-17T07:12:14.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2b200640 1 -- start start 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b2c074960 0x7f3b2c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b2c075f40 0x7f3b2c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b2c109d00 0x7f3b2c1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c10fc00 con 0x7f3b2c109d00 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c10fa80 con 0x7f3b2c075f40 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c10fd80 con 0x7f3b2c074960 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b2c109d00 0x7f3b2c1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b2c109d00 0x7f3b2c1a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49714/0 (socket says 172.21.15.86:49714) 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.110+0000 7f3b2a800640 1 -- 172.21.15.86:0/15219655 learned_addr learned my addr 172.21.15.86:0/15219655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b29400640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b2c075f40 0x7f3b2c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b29e00640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b2c074960 0x7f3b2c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b2a800640 1 -- 172.21.15.86:0/15219655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b2c074960 msgr2=0x7f3b2c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b2a800640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b2c074960 0x7f3b2c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b2a800640 1 -- 172.21.15.86:0/15219655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b2c075f40 msgr2=0x7f3b2c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b2a800640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b2c075f40 0x7f3b2c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b2a800640 1 -- 172.21.15.86:0/15219655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c1a1ef0 con 0x7f3b2c109d00 2024-04-17T07:12:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b29e00640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b2c074960 0x7f3b2c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b2a800640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b2c109d00 0x7f3b2c1a1880 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400eaa0 tx=0x7f3b2400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b22a00640 1 -- 172.21.15.86:0/15219655 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400cdb0 con 0x7f3b2c109d00 2024-04-17T07:12:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b22a00640 1 -- 172.21.15.86:0/15219655 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b24005020 con 0x7f3b2c109d00 2024-04-17T07:12:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b2b200640 1 -- 172.21.15.86:0/15219655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c1a21e0 con 0x7f3b2c109d00 2024-04-17T07:12:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.111+0000 7f3b22a00640 1 -- 172.21.15.86:0/15219655 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b240116d0 con 0x7f3b2c109d00 2024-04-17T07:12:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.112+0000 7f3b2b200640 1 -- 172.21.15.86:0/15219655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c1a26e0 con 0x7f3b2c109d00 2024-04-17T07:12:14.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.113+0000 7f3b2b200640 1 -- 172.21.15.86:0/15219655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2c108ac0 con 0x7f3b2c109d00 2024-04-17T07:12:14.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.115+0000 7f3b22a00640 1 -- 172.21.15.86:0/15219655 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3b240051d0 con 0x7f3b2c109d00 2024-04-17T07:12:14.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.116+0000 7f3b22a00640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3b100a3bb0 0x7f3b100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:14.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.116+0000 7f3b22a00640 1 -- 172.21.15.86:0/15219655 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f3b240d08e0 con 0x7f3b2c109d00 2024-04-17T07:12:14.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.116+0000 7f3b29e00640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3b100a3bb0 0x7f3b100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:14.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.117+0000 7f3b29e00640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3b100a3bb0 0x7f3b100a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c036040 tx=0x7f3b1c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.120+0000 7f3b22a00640 1 -- 172.21.15.86:0/15219655 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3b24016030 con 0x7f3b2c109d00 2024-04-17T07:12:14.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:14 smithi103 ceph-mon[35495]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:14.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:14 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/220572847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:12:14.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:14 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1743071141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:12:14.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.321+0000 7f3b2b200640 1 -- 172.21.15.86:0/15219655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3b2c062b60 con 0x7f3b2c109d00 2024-04-17T07:12:14.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.322+0000 7f3b22a00640 1 -- 172.21.15.86:0/15219655 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f3b2409d080 con 0x7f3b2c109d00 2024-04-17T07:12:14.322 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:12:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 -- 172.21.15.86:0/15219655 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3b100a3bb0 msgr2=0x7f3b100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:14.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3b100a3bb0 0x7f3b100a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c036040 tx=0x7f3b1c005fd0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 -- 172.21.15.86:0/15219655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b2c109d00 msgr2=0x7f3b2c1a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b2c109d00 0x7f3b2c1a1880 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400eaa0 tx=0x7f3b2400ef60 comp rx=0 tx=0).stop 2024-04-17T07:12:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 -- 172.21.15.86:0/15219655 shutdown_connections 2024-04-17T07:12:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3b100a3bb0 0x7f3b100a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b2c109d00 0x7f3b2c1a1880 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b2c075f40 0x7f3b2c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 --2- 172.21.15.86:0/15219655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b2c074960 0x7f3b2c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 -- 172.21.15.86:0/15219655 >> 172.21.15.86:0/15219655 conn(0x7f3b2c0fd400 msgr2=0x7f3b2c105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 -- 172.21.15.86:0/15219655 shutdown_connections 2024-04-17T07:12:14.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.325+0000 7f3b2b200640 1 -- 172.21.15.86:0/15219655 wait complete. 2024-04-17T07:12:14.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:14 smithi086 ceph-mon[31733]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:14 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/220572847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:12:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:14 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1743071141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:12:14.343 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T07:12:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:14 smithi098 ceph-mon[35418]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:14 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/220572847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:12:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:14 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1743071141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:12:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.556+0000 7f66d7200640 1 -- 172.21.15.86:0/3558109939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d80749c0 msgr2=0x7f66d8074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.556+0000 7f66d7200640 1 --2- 172.21.15.86:0/3558109939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d80749c0 0x7f66d8074e40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009900 tx=0x7f66c002bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.556+0000 7f66d7200640 1 -- 172.21.15.86:0/3558109939 shutdown_connections 2024-04-17T07:12:14.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.556+0000 7f66d7200640 1 --2- 172.21.15.86:0/3558109939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d8109db0 0x7f66d810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.556+0000 7f66d7200640 1 --2- 172.21.15.86:0/3558109939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d80749c0 0x7f66d8074e40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.556+0000 7f66d7200640 1 --2- 172.21.15.86:0/3558109939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66d8075fa0 0x7f66d80743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.556+0000 7f66d7200640 1 -- 172.21.15.86:0/3558109939 >> 172.21.15.86:0/3558109939 conn(0x7f66d80fd460 msgr2=0x7f66d80ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.556+0000 7f66d7200640 1 -- 172.21.15.86:0/3558109939 shutdown_connections 2024-04-17T07:12:14.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.556+0000 7f66d7200640 1 -- 172.21.15.86:0/3558109939 wait complete. 2024-04-17T07:12:14.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.557+0000 7f66d7200640 1 Processor -- start 2024-04-17T07:12:14.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.557+0000 7f66d7200640 1 -- start start 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d7200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66d80749c0 0x7f66d819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d8075fa0 0x7f66d819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d8109db0 0x7f66d81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810fc00 con 0x7f66d8075fa0 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d7200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810fa80 con 0x7f66d80749c0 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810fd80 con 0x7f66d8109db0 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d8075fa0 0x7f66d819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d8075fa0 0x7f66d819a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49738/0 (socket says 172.21.15.86:49738) 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d5400640 1 -- 172.21.15.86:0/1233101791 learned_addr learned my addr 172.21.15.86:0/1233101791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d5e00640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66d80749c0 0x7f66d819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d6800640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d8109db0 0x7f66d81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d5400640 1 -- 172.21.15.86:0/1233101791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d8109db0 msgr2=0x7f66d81a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:14.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d5400640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d8109db0 0x7f66d81a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d5400640 1 -- 172.21.15.86:0/1233101791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66d80749c0 msgr2=0x7f66d819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d5400640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66d80749c0 0x7f66d819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.558+0000 7f66d5400640 1 -- 172.21.15.86:0/1233101791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d81a1ef0 con 0x7f66d8075fa0 2024-04-17T07:12:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.559+0000 7f66d5400640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d8075fa0 0x7f66d819a810 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f66c0000c00 tx=0x7f66c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.559+0000 7f66cea00640 1 -- 172.21.15.86:0/1233101791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0043020 con 0x7f66d8075fa0 2024-04-17T07:12:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.559+0000 7f66cea00640 1 -- 172.21.15.86:0/1233101791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66c00054c0 con 0x7f66d8075fa0 2024-04-17T07:12:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.559+0000 7f66d7200640 1 -- 172.21.15.86:0/1233101791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d81a2180 con 0x7f66d8075fa0 2024-04-17T07:12:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.559+0000 7f66cea00640 1 -- 172.21.15.86:0/1233101791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0003d40 con 0x7f66d8075fa0 2024-04-17T07:12:14.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.559+0000 7f66d7200640 1 -- 172.21.15.86:0/1233101791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d81a26d0 con 0x7f66d8075fa0 2024-04-17T07:12:14.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.560+0000 7f66d7200640 1 -- 172.21.15.86:0/1233101791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d8108ae0 con 0x7f66d8075fa0 2024-04-17T07:12:14.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.562+0000 7f66cea00640 1 -- 172.21.15.86:0/1233101791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f66c0046020 con 0x7f66d8075fa0 2024-04-17T07:12:14.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.564+0000 7f66cea00640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66bc0a3bb0 0x7f66bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:14.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.564+0000 7f66cea00640 1 -- 172.21.15.86:0/1233101791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f66c00f2af0 con 0x7f66d8075fa0 2024-04-17T07:12:14.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.564+0000 7f66d5e00640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66bc0a3bb0 0x7f66bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:14.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.565+0000 7f66d5e00640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66bc0a3bb0 0x7f66bc0a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f66c8005fd0 tx=0x7f66c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:14.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.568+0000 7f66cea00640 1 -- 172.21.15.86:0/1233101791 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f66c004d020 con 0x7f66d8075fa0 2024-04-17T07:12:14.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.785+0000 7f66d7200640 1 -- 172.21.15.86:0/1233101791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f66d802d5d0 con 0x7f66d8075fa0 2024-04-17T07:12:14.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.786+0000 7f66cea00640 1 -- 172.21.15.86:0/1233101791 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+591 (secure 0 0 0) 0x7f66c00bf230 con 0x7f66d8075fa0 2024-04-17T07:12:14.786 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/174556107 2024-04-18T07:02:26.881886+0000 2024-04-17T07:12:14.786 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2497304884 2024-04-18T07:02:26.881886+0000 2024-04-17T07:12:14.786 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2497304884 2024-04-18T07:02:26.881886+0000 2024-04-17T07:12:14.786 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2537127051 2024-04-18T07:01:52.772350+0000 2024-04-17T07:12:14.786 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2158768684 2024-04-18T07:02:26.881886+0000 2024-04-17T07:12:14.787 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3172534015 2024-04-18T07:02:26.881886+0000 2024-04-17T07:12:14.787 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/4176177513 2024-04-18T07:01:52.772350+0000 2024-04-17T07:12:14.787 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3239168822 2024-04-18T07:01:52.772350+0000 2024-04-17T07:12:14.787 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/4176177513 2024-04-18T07:01:52.772350+0000 2024-04-17T07:12:14.787 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3864604926 2024-04-18T07:01:52.772350+0000 2024-04-17T07:12:14.787 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-04-17T07:12:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 -- 172.21.15.86:0/1233101791 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66bc0a3bb0 msgr2=0x7f66bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66bc0a3bb0 0x7f66bc0a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f66c8005fd0 tx=0x7f66c8005850 comp rx=0 tx=0).stop 2024-04-17T07:12:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 -- 172.21.15.86:0/1233101791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d8075fa0 msgr2=0x7f66d819a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d8075fa0 0x7f66d819a810 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f66c0000c00 tx=0x7f66c00042f0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 -- 172.21.15.86:0/1233101791 shutdown_connections 2024-04-17T07:12:14.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66bc0a3bb0 0x7f66bc0a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66d8109db0 0x7f66d81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66d8075fa0 0x7f66d819a810 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 --2- 172.21.15.86:0/1233101791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66d80749c0 0x7f66d819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 -- 172.21.15.86:0/1233101791 >> 172.21.15.86:0/1233101791 conn(0x7f66d80fd460 msgr2=0x7f66d8105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 -- 172.21.15.86:0/1233101791 shutdown_connections 2024-04-17T07:12:14.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:14.789+0000 7f66d7200640 1 -- 172.21.15.86:0/1233101791 wait complete. 2024-04-17T07:12:14.810 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:12:14.810 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T07:12:14.861 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:12:15.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.073+0000 7fd537200640 1 -- 172.21.15.86:0/1261822283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381051d0 msgr2=0x7fd538105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:15.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.073+0000 7fd537200640 1 --2- 172.21.15.86:0/1261822283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381051d0 0x7fd538105650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd524009900 tx=0x7fd52402bc40 comp rx=0 tx=0).stop 2024-04-17T07:12:15.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.073+0000 7fd537200640 1 -- 172.21.15.86:0/1261822283 shutdown_connections 2024-04-17T07:12:15.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.073+0000 7fd537200640 1 --2- 172.21.15.86:0/1261822283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd538105b90 0x7fd53810c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.073+0000 7fd537200640 1 --2- 172.21.15.86:0/1261822283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381051d0 0x7fd538105650 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.073+0000 7fd537200640 1 --2- 172.21.15.86:0/1261822283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd538103fe0 0x7fd538104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.073+0000 7fd537200640 1 -- 172.21.15.86:0/1261822283 >> 172.21.15.86:0/1261822283 conn(0x7fd5380ff570 msgr2=0x7fd538101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:15.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.073+0000 7fd537200640 1 -- 172.21.15.86:0/1261822283 shutdown_connections 2024-04-17T07:12:15.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.073+0000 7fd537200640 1 -- 172.21.15.86:0/1261822283 wait complete. 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.074+0000 7fd537200640 1 Processor -- start 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.074+0000 7fd537200640 1 -- start start 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.074+0000 7fd537200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd538103fe0 0x7fd53819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.074+0000 7fd537200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381051d0 0x7fd53819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.074+0000 7fd537200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd538105b90 0x7fd5381a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.074+0000 7fd537200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd53810fb20 con 0x7fd5381051d0 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.074+0000 7fd537200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd53810f9a0 con 0x7fd538105b90 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.074+0000 7fd537200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd53810fca0 con 0x7fd538103fe0 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.075+0000 7fd535400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381051d0 0x7fd53819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.075+0000 7fd535400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381051d0 0x7fd53819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49770/0 (socket says 172.21.15.86:49770) 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.075+0000 7fd535400640 1 -- 172.21.15.86:0/3075779489 learned_addr learned my addr 172.21.15.86:0/3075779489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:15.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.075+0000 7fd535e00640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd538103fe0 0x7fd53819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.075+0000 7fd535400640 1 -- 172.21.15.86:0/3075779489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd538103fe0 msgr2=0x7fd53819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.075+0000 7fd535400640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd538103fe0 0x7fd53819a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.075+0000 7fd535400640 1 -- 172.21.15.86:0/3075779489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd538105b90 msgr2=0x7fd5381a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.075+0000 7fd536800640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd538105b90 0x7fd5381a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.075+0000 7fd535400640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd538105b90 0x7fd5381a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.075+0000 7fd535400640 1 -- 172.21.15.86:0/3075779489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5381a1d90 con 0x7fd5381051d0 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.076+0000 7fd535400640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381051d0 0x7fd53819a6b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd524006010 tx=0x7fd5240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.076+0000 7fd52ea00640 1 -- 172.21.15.86:0/3075779489 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524043020 con 0x7fd5381051d0 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.076+0000 7fd52ea00640 1 -- 172.21.15.86:0/3075779489 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd524004aa0 con 0x7fd5381051d0 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.076+0000 7fd52ea00640 1 -- 172.21.15.86:0/3075779489 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524003c60 con 0x7fd5381051d0 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.076+0000 7fd537200640 1 -- 172.21.15.86:0/3075779489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5381a2020 con 0x7fd5381051d0 2024-04-17T07:12:15.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.076+0000 7fd537200640 1 -- 172.21.15.86:0/3075779489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5381a2520 con 0x7fd5381051d0 2024-04-17T07:12:15.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.077+0000 7fd537200640 1 -- 172.21.15.86:0/3075779489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd53810b300 con 0x7fd5381051d0 2024-04-17T07:12:15.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.079+0000 7fd52ea00640 1 -- 172.21.15.86:0/3075779489 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd524046020 con 0x7fd5381051d0 2024-04-17T07:12:15.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.080+0000 7fd52ea00640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd51c0a3c30 0x7fd51c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:15.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.081+0000 7fd52ea00640 1 -- 172.21.15.86:0/3075779489 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd5240f2a90 con 0x7fd5381051d0 2024-04-17T07:12:15.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.081+0000 7fd535e00640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd51c0a3c30 0x7fd51c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:15.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.081+0000 7fd535e00640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd51c0a3c30 0x7fd51c0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd528005fd0 tx=0x7fd528005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:15.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.085+0000 7fd52ea00640 1 -- 172.21.15.86:0/3075779489 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd52404d020 con 0x7fd5381051d0 2024-04-17T07:12:15.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.290+0000 7fd537200640 1 -- 172.21.15.86:0/3075779489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd538062b60 con 0x7fd5381051d0 2024-04-17T07:12:15.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.293+0000 7fd52ea00640 1 -- 172.21.15.86:0/3075779489 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24057 (secure 0 0 0) 0x7fd5240bf1d0 con 0x7fd5381051d0 2024-04-17T07:12:15.293 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:15.293 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:11:59.971687+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.295+0000 7fd537200640 1 -- 172.21.15.86:0/3075779489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd51c0a3c30 msgr2=0x7fd51c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.295+0000 7fd537200640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd51c0a3c30 0x7fd51c0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd528005fd0 tx=0x7fd528005850 comp rx=0 tx=0).stop 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.296+0000 7fd537200640 1 -- 172.21.15.86:0/3075779489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381051d0 msgr2=0x7fd53819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.296+0000 7fd537200640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381051d0 0x7fd53819a6b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd524006010 tx=0x7fd5240042f0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.296+0000 7fd537200640 1 -- 172.21.15.86:0/3075779489 shutdown_connections 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.296+0000 7fd537200640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd51c0a3c30 0x7fd51c0a60f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.296+0000 7fd537200640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd538105b90 0x7fd5381a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.296+0000 7fd537200640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5381051d0 0x7fd53819a6b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.296+0000 7fd537200640 1 --2- 172.21.15.86:0/3075779489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd538103fe0 0x7fd53819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.296+0000 7fd537200640 1 -- 172.21.15.86:0/3075779489 >> 172.21.15.86:0/3075779489 conn(0x7fd5380ff570 msgr2=0x7fd5381084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:15.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.296+0000 7fd537200640 1 -- 172.21.15.86:0/3075779489 shutdown_connections 2024-04-17T07:12:15.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.296+0000 7fd537200640 1 -- 172.21.15.86:0/3075779489 wait complete. 2024-04-17T07:12:15.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:12:15.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:12:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:15 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/15219655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:12:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:15 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1233101791' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:12:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:15 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/15219655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:12:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:15 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1233101791' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:12:15.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.533+0000 7f3712600640 1 -- 172.21.15.86:0/332121838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f370c100650 msgr2=0x7f370c100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:15.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.533+0000 7f3712600640 1 --2- 172.21.15.86:0/332121838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f370c100650 0x7f370c100a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009ca0 tx=0x7f36fc02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.533+0000 7f3712600640 1 -- 172.21.15.86:0/332121838 shutdown_connections 2024-04-17T07:12:15.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.533+0000 7f3712600640 1 --2- 172.21.15.86:0/332121838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f370c101950 0x7f370c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.533+0000 7f3712600640 1 --2- 172.21.15.86:0/332121838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f370c100fb0 0x7f370c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.533+0000 7f3712600640 1 --2- 172.21.15.86:0/332121838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f370c100650 0x7f370c100a70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.533+0000 7f3712600640 1 -- 172.21.15.86:0/332121838 >> 172.21.15.86:0/332121838 conn(0x7f370c0fc230 msgr2=0x7f370c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:15.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.533+0000 7f3712600640 1 -- 172.21.15.86:0/332121838 shutdown_connections 2024-04-17T07:12:15.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.534+0000 7f3712600640 1 -- 172.21.15.86:0/332121838 wait complete. 2024-04-17T07:12:15.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.534+0000 7f3712600640 1 Processor -- start 2024-04-17T07:12:15.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f3712600640 1 -- start start 2024-04-17T07:12:15.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f3712600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f370c100650 0x7f370c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:15.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f3712600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f370c100fb0 0x7f370c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:15.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f3712600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f370c101950 0x7f370c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:15.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f3712600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f370c10faa0 con 0x7f370c100fb0 2024-04-17T07:12:15.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f3712600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f370c10f920 con 0x7f370c101950 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f3712600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f370c10fc20 con 0x7f370c100650 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f370be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f370c100fb0 0x7f370c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f370be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f370c100fb0 0x7f370c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49786/0 (socket says 172.21.15.86:49786) 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f370be00640 1 -- 172.21.15.86:0/3456479592 learned_addr learned my addr 172.21.15.86:0/3456479592 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.535+0000 7f3711200640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f370c100650 0x7f370c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f3711c00640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f370c101950 0x7f370c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f370be00640 1 -- 172.21.15.86:0/3456479592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f370c100650 msgr2=0x7f370c19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f370be00640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f370c100650 0x7f370c19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f370be00640 1 -- 172.21.15.86:0/3456479592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f370c101950 msgr2=0x7f370c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f370be00640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f370c101950 0x7f370c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f370be00640 1 -- 172.21.15.86:0/3456479592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c1a1dc0 con 0x7f370c100fb0 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f370be00640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f370c100fb0 0x7f370c19a6e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f36f800d770 tx=0x7f36f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f3709600640 1 -- 172.21.15.86:0/3456479592 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8017070 con 0x7f370c100fb0 2024-04-17T07:12:15.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f3709600640 1 -- 172.21.15.86:0/3456479592 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36f80054d0 con 0x7f370c100fb0 2024-04-17T07:12:15.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f3712600640 1 -- 172.21.15.86:0/3456479592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370c1a20b0 con 0x7f370c100fb0 2024-04-17T07:12:15.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f3709600640 1 -- 172.21.15.86:0/3456479592 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8004cf0 con 0x7f370c100fb0 2024-04-17T07:12:15.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.536+0000 7f3712600640 1 -- 172.21.15.86:0/3456479592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370c1a25b0 con 0x7f370c100fb0 2024-04-17T07:12:15.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.538+0000 7f3712600640 1 -- 172.21.15.86:0/3456479592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f370c10b3a0 con 0x7f370c100fb0 2024-04-17T07:12:15.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.541+0000 7f3709600640 1 -- 172.21.15.86:0/3456479592 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f36f8004e90 con 0x7f370c100fb0 2024-04-17T07:12:15.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.542+0000 7f3709600640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f36f00a3bb0 0x7f36f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:15.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.542+0000 7f3709600640 1 -- 172.21.15.86:0/3456479592 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f36f80d0590 con 0x7f370c100fb0 2024-04-17T07:12:15.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.542+0000 7f3711200640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f36f00a3bb0 0x7f36f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:15.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.543+0000 7f3711200640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f36f00a3bb0 0x7f36f00a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f36fc02c700 tx=0x7f36fc000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:15.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.544+0000 7f3709600640 1 -- 172.21.15.86:0/3456479592 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f36f809cd30 con 0x7f370c100fb0 2024-04-17T07:12:15.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:15 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/15219655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:12:15.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:15 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1233101791' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:12:15.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.747+0000 7f3712600640 1 -- 172.21.15.86:0/3456479592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f370c02d760 con 0x7f370c100fb0 2024-04-17T07:12:15.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.748+0000 7f3709600640 1 -- 172.21.15.86:0/3456479592 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f36f809c630 con 0x7f370c100fb0 2024-04-17T07:12:15.748 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:12:15.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 -- 172.21.15.86:0/3456479592 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f36f00a3bb0 msgr2=0x7f36f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:15.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f36f00a3bb0 0x7f36f00a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f36fc02c700 tx=0x7f36fc000f40 comp rx=0 tx=0).stop 2024-04-17T07:12:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 -- 172.21.15.86:0/3456479592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f370c100fb0 msgr2=0x7f370c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f370c100fb0 0x7f370c19a6e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f36f800d770 tx=0x7f36f800dc30 comp rx=0 tx=0).stop 2024-04-17T07:12:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 -- 172.21.15.86:0/3456479592 shutdown_connections 2024-04-17T07:12:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f36f00a3bb0 0x7f36f00a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f370c101950 0x7f370c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f370c100fb0 0x7f370c19a6e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 --2- 172.21.15.86:0/3456479592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f370c100650 0x7f370c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 -- 172.21.15.86:0/3456479592 >> 172.21.15.86:0/3456479592 conn(0x7f370c0fc230 msgr2=0x7f370c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 -- 172.21.15.86:0/3456479592 shutdown_connections 2024-04-17T07:12:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:15.751+0000 7f3712600640 1 -- 172.21.15.86:0/3456479592 wait complete. 2024-04-17T07:12:15.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:12:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.002+0000 7feeb4a00640 1 -- 172.21.15.86:0/1953031517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0101040 msgr2=0x7feeb01014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.002+0000 7feeb4a00640 1 --2- 172.21.15.86:0/1953031517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0101040 0x7feeb01014a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fee98009900 tx=0x7fee9802bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:16.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.002+0000 7feeb4a00640 1 -- 172.21.15.86:0/1953031517 shutdown_connections 2024-04-17T07:12:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.002+0000 7feeb4a00640 1 --2- 172.21.15.86:0/1953031517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb01019e0 0x7feeb0101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.002+0000 7feeb4a00640 1 --2- 172.21.15.86:0/1953031517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0101040 0x7feeb01014a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.002+0000 7feeb4a00640 1 --2- 172.21.15.86:0/1953031517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feeb01006e0 0x7feeb0100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.002+0000 7feeb4a00640 1 -- 172.21.15.86:0/1953031517 >> 172.21.15.86:0/1953031517 conn(0x7feeb00fc300 msgr2=0x7feeb00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.002+0000 7feeb4a00640 1 -- 172.21.15.86:0/1953031517 shutdown_connections 2024-04-17T07:12:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.002+0000 7feeb4a00640 1 -- 172.21.15.86:0/1953031517 wait complete. 2024-04-17T07:12:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.003+0000 7feeb4a00640 1 Processor -- start 2024-04-17T07:12:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.003+0000 7feeb4a00640 1 -- start start 2024-04-17T07:12:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb01006e0 0x7feeb019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0101040 0x7feeb019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeb4a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feeb01019e0 0x7feeb01a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeb4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feeb010fc10 con 0x7feeb0101040 2024-04-17T07:12:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeb4a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feeb010fa90 con 0x7feeb01006e0 2024-04-17T07:12:16.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeb4a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feeb010fd90 con 0x7feeb01019e0 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0101040 0x7feeb019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0101040 0x7feeb019a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49800/0 (socket says 172.21.15.86:49800) 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeade00640 1 -- 172.21.15.86:0/2466661727 learned_addr learned my addr 172.21.15.86:0/2466661727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeaf200640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feeb01019e0 0x7feeb01a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeae800640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb01006e0 0x7feeb019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeade00640 1 -- 172.21.15.86:0/2466661727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feeb01019e0 msgr2=0x7feeb01a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeade00640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feeb01019e0 0x7feeb01a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeade00640 1 -- 172.21.15.86:0/2466661727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb01006e0 msgr2=0x7feeb019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeade00640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb01006e0 0x7feeb019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.004+0000 7feeade00640 1 -- 172.21.15.86:0/2466661727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb01a1ea0 con 0x7feeb0101040 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.005+0000 7feeade00640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0101040 0x7feeb019a7c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fee98000c00 tx=0x7fee980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.005+0000 7feea7400640 1 -- 172.21.15.86:0/2466661727 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98043020 con 0x7feeb0101040 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.005+0000 7feea7400640 1 -- 172.21.15.86:0/2466661727 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee980054c0 con 0x7feeb0101040 2024-04-17T07:12:16.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.005+0000 7feeb4a00640 1 -- 172.21.15.86:0/2466661727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb01a2130 con 0x7feeb0101040 2024-04-17T07:12:16.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.005+0000 7feea7400640 1 -- 172.21.15.86:0/2466661727 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98003d40 con 0x7feeb0101040 2024-04-17T07:12:16.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.005+0000 7feeb4a00640 1 -- 172.21.15.86:0/2466661727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb01a2680 con 0x7feeb0101040 2024-04-17T07:12:16.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.006+0000 7feeb4a00640 1 -- 172.21.15.86:0/2466661727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb0068ca0 con 0x7feeb0101040 2024-04-17T07:12:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.008+0000 7feea7400640 1 -- 172.21.15.86:0/2466661727 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fee98046020 con 0x7feeb0101040 2024-04-17T07:12:16.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.009+0000 7feea7400640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee940a3bb0 0x7fee940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.010+0000 7feea7400640 1 -- 172.21.15.86:0/2466661727 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fee980f2a90 con 0x7feeb0101040 2024-04-17T07:12:16.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.010+0000 7feeae800640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee940a3bb0 0x7fee940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.010+0000 7feeae800640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee940a3bb0 0x7fee940a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feea0005c40 tx=0x7feea0005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:16.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.014+0000 7feea7400640 1 -- 172.21.15.86:0/2466661727 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fee9804d020 con 0x7feeb0101040 2024-04-17T07:12:16.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.218+0000 7feeb4a00640 1 -- 172.21.15.86:0/2466661727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feeb0062b60 con 0x7feeb0101040 2024-04-17T07:12:16.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.219+0000 7feea7400640 1 -- 172.21.15.86:0/2466661727 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fee980bf230 con 0x7feeb0101040 2024-04-17T07:12:16.219 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:12:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 -- 172.21.15.86:0/2466661727 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee940a3bb0 msgr2=0x7fee940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee940a3bb0 0x7fee940a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feea0005c40 tx=0x7feea0005bd0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 -- 172.21.15.86:0/2466661727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0101040 msgr2=0x7feeb019a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0101040 0x7feeb019a7c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fee98000c00 tx=0x7fee980042f0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 -- 172.21.15.86:0/2466661727 shutdown_connections 2024-04-17T07:12:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee940a3bb0 0x7fee940a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feeb01019e0 0x7feeb01a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feeb0101040 0x7feeb019a7c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 --2- 172.21.15.86:0/2466661727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb01006e0 0x7feeb019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 -- 172.21.15.86:0/2466661727 >> 172.21.15.86:0/2466661727 conn(0x7feeb00fc300 msgr2=0x7feeb00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.222+0000 7feeb4a00640 1 -- 172.21.15.86:0/2466661727 shutdown_connections 2024-04-17T07:12:16.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.223+0000 7feeb4a00640 1 -- 172.21.15.86:0/2466661727 wait complete. 2024-04-17T07:12:16.241 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:12:16.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:16 smithi098 ceph-mon[35418]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:16.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:16 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3075779489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:16.405 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:16 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3456479592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:12:16.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.455+0000 7f879fe00640 1 -- 172.21.15.86:0/3648485902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a01051b0 msgr2=0x7f87a0105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.455+0000 7f879fe00640 1 --2- 172.21.15.86:0/3648485902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a01051b0 0x7f87a0105630 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8788009900 tx=0x7f878802bc50 comp rx=0 tx=0).stop 2024-04-17T07:12:16.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.455+0000 7f879fe00640 1 -- 172.21.15.86:0/3648485902 shutdown_connections 2024-04-17T07:12:16.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.455+0000 7f879fe00640 1 --2- 172.21.15.86:0/3648485902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87a0105b70 0x7f87a010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.455+0000 7f879fe00640 1 --2- 172.21.15.86:0/3648485902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a01051b0 0x7f87a0105630 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.455+0000 7f879fe00640 1 --2- 172.21.15.86:0/3648485902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87a0103fc0 0x7f87a01043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.455+0000 7f879fe00640 1 -- 172.21.15.86:0/3648485902 >> 172.21.15.86:0/3648485902 conn(0x7f87a00ff530 msgr2=0x7f87a0101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:16.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.455+0000 7f879fe00640 1 -- 172.21.15.86:0/3648485902 shutdown_connections 2024-04-17T07:12:16.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.455+0000 7f879fe00640 1 -- 172.21.15.86:0/3648485902 wait complete. 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.456+0000 7f879fe00640 1 Processor -- start 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.456+0000 7f879fe00640 1 -- start start 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.456+0000 7f879fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87a0103fc0 0x7f87a019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.456+0000 7f879fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87a01051b0 0x7f87a019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.456+0000 7f879fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a0105b70 0x7f87a01a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.456+0000 7f879fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87a010fb90 con 0x7f87a0105b70 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.456+0000 7f879fe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87a010fa10 con 0x7f87a0103fc0 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.456+0000 7f879fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f87a010fd10 con 0x7f87a01051b0 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a0105b70 0x7f87a01a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a0105b70 0x7f87a01a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49822/0 (socket says 172.21.15.86:49822) 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879f400640 1 -- 172.21.15.86:0/4281085489 learned_addr learned my addr 172.21.15.86:0/4281085489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f8796000640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87a01051b0 0x7f87a019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879ea00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87a0103fc0 0x7f87a019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879f400640 1 -- 172.21.15.86:0/4281085489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87a01051b0 msgr2=0x7f87a019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879f400640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87a01051b0 0x7f87a019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879f400640 1 -- 172.21.15.86:0/4281085489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87a0103fc0 msgr2=0x7f87a019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879f400640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87a0103fc0 0x7f87a019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879f400640 1 -- 172.21.15.86:0/4281085489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a01a1e50 con 0x7f87a0105b70 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f8796000640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87a01051b0 0x7f87a019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879ea00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87a0103fc0 0x7f87a019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.457+0000 7f879f400640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a0105b70 0x7f87a01a17e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8798009fd0 tx=0x7f879800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.458+0000 7f87a4a00640 1 -- 172.21.15.86:0/4281085489 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798011830 con 0x7f87a0105b70 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.458+0000 7f87a4a00640 1 -- 172.21.15.86:0/4281085489 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8798016070 con 0x7f87a0105b70 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.458+0000 7f879fe00640 1 -- 172.21.15.86:0/4281085489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a01a2140 con 0x7f87a0105b70 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.458+0000 7f87a4a00640 1 -- 172.21.15.86:0/4281085489 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798010de0 con 0x7f87a0105b70 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.458+0000 7f879fe00640 1 -- 172.21.15.86:0/4281085489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a01a2710 con 0x7f87a0105b70 2024-04-17T07:12:16.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.459+0000 7f879fe00640 1 -- 172.21.15.86:0/4281085489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a010b300 con 0x7f87a0105b70 2024-04-17T07:12:16.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.461+0000 7f87a4a00640 1 -- 172.21.15.86:0/4281085489 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f87980119d0 con 0x7f87a0105b70 2024-04-17T07:12:16.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.462+0000 7f87a4a00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87840a3c30 0x7f87840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.463+0000 7f87a4a00640 1 -- 172.21.15.86:0/4281085489 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f87980cfdc0 con 0x7f87a0105b70 2024-04-17T07:12:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.463+0000 7f879ea00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87840a3c30 0x7f87840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.463+0000 7f879ea00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87840a3c30 0x7f87840a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f87a0105010 tx=0x7f87900082f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:16.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.467+0000 7f87a4a00640 1 -- 172.21.15.86:0/4281085489 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f879809c500 con 0x7f87a0105b70 2024-04-17T07:12:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:16 smithi103 ceph-mon[35495]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:16 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3075779489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:16 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3456479592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:12:16.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:16 smithi086 ceph-mon[31733]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:16.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:16 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3075779489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:16.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:16 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3456479592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:12:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.672+0000 7f879fe00640 1 -- 172.21.15.86:0/4281085489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87a0062b60 con 0x7f87a0105b70 2024-04-17T07:12:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.673+0000 7f87a4a00640 1 -- 172.21.15.86:0/4281085489 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f879809be00 con 0x7f87a0105b70 2024-04-17T07:12:16.672 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:12:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 -- 172.21.15.86:0/4281085489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87840a3c30 msgr2=0x7f87840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87840a3c30 0x7f87840a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f87a0105010 tx=0x7f87900082f0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 -- 172.21.15.86:0/4281085489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a0105b70 msgr2=0x7f87a01a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a0105b70 0x7f87a01a17e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8798009fd0 tx=0x7f879800b8e0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 -- 172.21.15.86:0/4281085489 shutdown_connections 2024-04-17T07:12:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87840a3c30 0x7f87840a60f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87a0105b70 0x7f87a01a17e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f87a01051b0 0x7f87a019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 --2- 172.21.15.86:0/4281085489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87a0103fc0 0x7f87a019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 -- 172.21.15.86:0/4281085489 >> 172.21.15.86:0/4281085489 conn(0x7f87a00ff530 msgr2=0x7f87a01084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 -- 172.21.15.86:0/4281085489 shutdown_connections 2024-04-17T07:12:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.676+0000 7f879fe00640 1 -- 172.21.15.86:0/4281085489 wait complete. 2024-04-17T07:12:16.696 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:12:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.909+0000 7f2cb5a00640 1 -- 172.21.15.86:0/4105426167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb0109690 msgr2=0x7f2cb0109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.909+0000 7f2cb5a00640 1 --2- 172.21.15.86:0/4105426167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb0109690 0x7f2cb0109b10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009900 tx=0x7f2c9802bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.909+0000 7f2cb5a00640 1 -- 172.21.15.86:0/4105426167 shutdown_connections 2024-04-17T07:12:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.909+0000 7f2cb5a00640 1 --2- 172.21.15.86:0/4105426167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb010a050 0x7f2cb01108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.909+0000 7f2cb5a00640 1 --2- 172.21.15.86:0/4105426167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb0109690 0x7f2cb0109b10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.909+0000 7f2cb5a00640 1 --2- 172.21.15.86:0/4105426167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb01084a0 0x7f2cb01088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.909+0000 7f2cb5a00640 1 -- 172.21.15.86:0/4105426167 >> 172.21.15.86:0/4105426167 conn(0x7f2cb0077430 msgr2=0x7f2cb0079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.910+0000 7f2cb5a00640 1 -- 172.21.15.86:0/4105426167 shutdown_connections 2024-04-17T07:12:16.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.910+0000 7f2cb5a00640 1 -- 172.21.15.86:0/4105426167 wait complete. 2024-04-17T07:12:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5a00640 1 Processor -- start 2024-04-17T07:12:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5a00640 1 -- start start 2024-04-17T07:12:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb01084a0 0x7f2cb019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb0109690 0x7f2cb019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb010a050 0x7f2cb01a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0113fe0 con 0x7f2cb010a050 2024-04-17T07:12:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0113e60 con 0x7f2cb01084a0 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb0114160 con 0x7f2cb0109690 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb010a050 0x7f2cb01a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb010a050 0x7f2cb01a5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32840/0 (socket says 172.21.15.86:32840) 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.911+0000 7f2cb5000640 1 -- 172.21.15.86:0/2952184506 learned_addr learned my addr 172.21.15.86:0/2952184506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cafe00640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb01084a0 0x7f2cb019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2ca7400640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb0109690 0x7f2cb019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cb5000640 1 -- 172.21.15.86:0/2952184506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb0109690 msgr2=0x7f2cb019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cb5000640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb0109690 0x7f2cb019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cb5000640 1 -- 172.21.15.86:0/2952184506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb01084a0 msgr2=0x7f2cb019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cb5000640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb01084a0 0x7f2cb019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cb5000640 1 -- 172.21.15.86:0/2952184506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb01a62c0 con 0x7f2cb010a050 2024-04-17T07:12:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cafe00640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb01084a0 0x7f2cb019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:12:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cb5000640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb010a050 0x7f2cb01a5c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800cad0 tx=0x7f2ca800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cad600640 1 -- 172.21.15.86:0/2952184506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800ae20 con 0x7f2cb010a050 2024-04-17T07:12:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cb5a00640 1 -- 172.21.15.86:0/2952184506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb01a65b0 con 0x7f2cb010a050 2024-04-17T07:12:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.912+0000 7f2cad600640 1 -- 172.21.15.86:0/2952184506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ca8005020 con 0x7f2cb010a050 2024-04-17T07:12:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.913+0000 7f2cad600640 1 -- 172.21.15.86:0/2952184506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800f6b0 con 0x7f2cb010a050 2024-04-17T07:12:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.913+0000 7f2cb5a00640 1 -- 172.21.15.86:0/2952184506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb01a6b00 con 0x7f2cb010a050 2024-04-17T07:12:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.914+0000 7f2cb5a00640 1 -- 172.21.15.86:0/2952184506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb010f890 con 0x7f2cb010a050 2024-04-17T07:12:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.916+0000 7f2cad600640 1 -- 172.21.15.86:0/2952184506 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2ca80051d0 con 0x7f2cb010a050 2024-04-17T07:12:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.918+0000 7f2cad600640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2c940a3bb0 0x7f2c940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.919+0000 7f2cad600640 1 -- 172.21.15.86:0/2952184506 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2ca80d0b10 con 0x7f2cb010a050 2024-04-17T07:12:16.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.919+0000 7f2cafe00640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2c940a3bb0 0x7f2c940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.919+0000 7f2cafe00640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2c940a3bb0 0x7f2c940a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2cb01094f0 tx=0x7f2ca000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:16.922+0000 7f2cad600640 1 -- 172.21.15.86:0/2952184506 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2ca8014030 con 0x7f2cb010a050 2024-04-17T07:12:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.125+0000 7f2cb5a00640 1 -- 172.21.15.86:0/2952184506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2cb002d5d0 con 0x7f2cb010a050 2024-04-17T07:12:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.126+0000 7f2cad600640 1 -- 172.21.15.86:0/2952184506 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f2ca809d2b0 con 0x7f2cb010a050 2024-04-17T07:12:17.126 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.128+0000 7f2cb5a00640 1 -- 172.21.15.86:0/2952184506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2c940a3bb0 msgr2=0x7f2c940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.128+0000 7f2cb5a00640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2c940a3bb0 0x7f2c940a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2cb01094f0 tx=0x7f2ca000a560 comp rx=0 tx=0).stop 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.128+0000 7f2cb5a00640 1 -- 172.21.15.86:0/2952184506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb010a050 msgr2=0x7f2cb01a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.128+0000 7f2cb5a00640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb010a050 0x7f2cb01a5c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800cad0 tx=0x7f2ca800cf90 comp rx=0 tx=0).stop 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.128+0000 7f2cb5a00640 1 -- 172.21.15.86:0/2952184506 shutdown_connections 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.128+0000 7f2cb5a00640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2c940a3bb0 0x7f2c940a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.128+0000 7f2cb5a00640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cb010a050 0x7f2cb01a5c50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.128+0000 7f2cb5a00640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb0109690 0x7f2cb019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.128+0000 7f2cb5a00640 1 --2- 172.21.15.86:0/2952184506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb01084a0 0x7f2cb019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.129+0000 7f2cb5a00640 1 -- 172.21.15.86:0/2952184506 >> 172.21.15.86:0/2952184506 conn(0x7f2cb0077430 msgr2=0x7f2cb010ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.129+0000 7f2cb5a00640 1 -- 172.21.15.86:0/2952184506 shutdown_connections 2024-04-17T07:12:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.129+0000 7f2cb5a00640 1 -- 172.21.15.86:0/2952184506 wait complete. 2024-04-17T07:12:17.148 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T07:12:17.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.364+0000 7f2468200640 1 -- 172.21.15.86:0/1724912918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2460105b50 msgr2=0x7f2460105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:17.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.364+0000 7f2468200640 1 --2- 172.21.15.86:0/1724912918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2460105b50 0x7f2460105fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2458009ca0 tx=0x7f245802c0f0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.364+0000 7f2468200640 1 -- 172.21.15.86:0/1724912918 shutdown_connections 2024-04-17T07:12:17.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.364+0000 7f2468200640 1 --2- 172.21.15.86:0/1724912918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2460105b50 0x7f2460105fd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.364+0000 7f2468200640 1 --2- 172.21.15.86:0/1724912918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24601031a0 0x7f2460105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.364+0000 7f2468200640 1 --2- 172.21.15.86:0/1724912918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2460100830 0x7f2460102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.364+0000 7f2468200640 1 -- 172.21.15.86:0/1724912918 >> 172.21.15.86:0/1724912918 conn(0x7f24600fc430 msgr2=0x7f24600fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.364+0000 7f2468200640 1 -- 172.21.15.86:0/1724912918 shutdown_connections 2024-04-17T07:12:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.364+0000 7f2468200640 1 -- 172.21.15.86:0/1724912918 wait complete. 2024-04-17T07:12:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.365+0000 7f2468200640 1 Processor -- start 2024-04-17T07:12:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.365+0000 7f2468200640 1 -- start start 2024-04-17T07:12:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2468200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2460100830 0x7f246019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2468200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24601031a0 0x7f246019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2468200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2460105b50 0x7f24601a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2468200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24601140e0 con 0x7f2460105b50 2024-04-17T07:12:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2468200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2460113f60 con 0x7f24601031a0 2024-04-17T07:12:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2468200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2460114260 con 0x7f2460100830 2024-04-17T07:12:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2466200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2460105b50 0x7f24601a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2466200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2460105b50 0x7f24601a5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32860/0 (socket says 172.21.15.86:32860) 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2466200640 1 -- 172.21.15.86:0/3452534607 learned_addr learned my addr 172.21.15.86:0/3452534607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2465800640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2460100830 0x7f246019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2466200640 1 -- 172.21.15.86:0/3452534607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2460100830 msgr2=0x7f246019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2466200640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2460100830 0x7f246019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2466200640 1 -- 172.21.15.86:0/3452534607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24601031a0 msgr2=0x7f246019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2464e00640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24601031a0 0x7f246019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2466200640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24601031a0 0x7f246019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.366+0000 7f2466200640 1 -- 172.21.15.86:0/3452534607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24601a63b0 con 0x7f2460105b50 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.367+0000 7f2466200640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2460105b50 0x7f24601a5d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2458032380 tx=0x7f2458032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.367+0000 7f245e000640 1 -- 172.21.15.86:0/3452534607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458041020 con 0x7f2460105b50 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.367+0000 7f2468200640 1 -- 172.21.15.86:0/3452534607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24601a6640 con 0x7f2460105b50 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.367+0000 7f245e000640 1 -- 172.21.15.86:0/3452534607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2458003e20 con 0x7f2460105b50 2024-04-17T07:12:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.367+0000 7f245e000640 1 -- 172.21.15.86:0/3452534607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458039040 con 0x7f2460105b50 2024-04-17T07:12:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.367+0000 7f2468200640 1 -- 172.21.15.86:0/3452534607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24601a6b80 con 0x7f2460105b50 2024-04-17T07:12:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.368+0000 7f2468200640 1 -- 172.21.15.86:0/3452534607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24600fdf20 con 0x7f2460105b50 2024-04-17T07:12:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.370+0000 7f245e000640 1 -- 172.21.15.86:0/3452534607 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2458044020 con 0x7f2460105b50 2024-04-17T07:12:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.372+0000 7f245e000640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24480a3bb0 0x7f24480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.372+0000 7f245e000640 1 -- 172.21.15.86:0/3452534607 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f24580f29d0 con 0x7f2460105b50 2024-04-17T07:12:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.372+0000 7f2465800640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24480a3bb0 0x7f24480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.373+0000 7f2465800640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24480a3bb0 0x7f24480a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2454005fd0 tx=0x7f2454005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:17.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.376+0000 7f245e000640 1 -- 172.21.15.86:0/3452534607 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f245804b020 con 0x7f2460105b50 2024-04-17T07:12:17.442 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:17 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2466661727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:12:17.442 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:17 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4281085489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:12:17.442 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:17 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2952184506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:12:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:17 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2466661727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:12:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:17 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4281085489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:12:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:17 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2952184506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:12:17.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:17 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2466661727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:12:17.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:17 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4281085489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:12:17.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:17 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2952184506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:12:17.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.594+0000 7f2468200640 1 -- 172.21.15.86:0/3452534607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2460062b60 con 0x7f2460105b50 2024-04-17T07:12:17.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.595+0000 7f245e000640 1 -- 172.21.15.86:0/3452534607 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+591 (secure 0 0 0) 0x7f24580bf170 con 0x7f2460105b50 2024-04-17T07:12:17.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/174556107 2024-04-18T07:02:26.881886+0000 2024-04-17T07:12:17.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2497304884 2024-04-18T07:02:26.881886+0000 2024-04-17T07:12:17.595 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2497304884 2024-04-18T07:02:26.881886+0000 2024-04-17T07:12:17.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2537127051 2024-04-18T07:01:52.772350+0000 2024-04-17T07:12:17.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2158768684 2024-04-18T07:02:26.881886+0000 2024-04-17T07:12:17.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3172534015 2024-04-18T07:02:26.881886+0000 2024-04-17T07:12:17.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/4176177513 2024-04-18T07:01:52.772350+0000 2024-04-17T07:12:17.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3239168822 2024-04-18T07:01:52.772350+0000 2024-04-17T07:12:17.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/4176177513 2024-04-18T07:01:52.772350+0000 2024-04-17T07:12:17.596 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3864604926 2024-04-18T07:01:52.772350+0000 2024-04-17T07:12:17.596 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-04-17T07:12:17.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.598+0000 7f2468200640 1 -- 172.21.15.86:0/3452534607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24480a3bb0 msgr2=0x7f24480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:17.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.598+0000 7f2468200640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24480a3bb0 0x7f24480a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2454005fd0 tx=0x7f2454005850 comp rx=0 tx=0).stop 2024-04-17T07:12:17.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.598+0000 7f2468200640 1 -- 172.21.15.86:0/3452534607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2460105b50 msgr2=0x7f24601a5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:17.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.598+0000 7f2468200640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2460105b50 0x7f24601a5d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2458032380 tx=0x7f2458032820 comp rx=0 tx=0).stop 2024-04-17T07:12:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.598+0000 7f2468200640 1 -- 172.21.15.86:0/3452534607 shutdown_connections 2024-04-17T07:12:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.598+0000 7f2468200640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24480a3bb0 0x7f24480a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.598+0000 7f2468200640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2460105b50 0x7f24601a5d40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.598+0000 7f2468200640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24601031a0 0x7f246019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.598+0000 7f2468200640 1 --2- 172.21.15.86:0/3452534607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2460100830 0x7f246019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.598+0000 7f2468200640 1 -- 172.21.15.86:0/3452534607 >> 172.21.15.86:0/3452534607 conn(0x7f24600fc430 msgr2=0x7f24601011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.599+0000 7f2468200640 1 -- 172.21.15.86:0/3452534607 shutdown_connections 2024-04-17T07:12:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:17.599+0000 7f2468200640 1 -- 172.21.15.86:0/3452534607 wait complete. 2024-04-17T07:12:17.617 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-17T07:12:17.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:17.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-17T07:12:17.703 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[73419]: fuse finished with error 0 and tester_r 0 2024-04-17T07:12:17.708 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T07:12:18.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:18 smithi086 ceph-mon[31733]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:18.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:18 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3452534607' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:12:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:18 smithi098 ceph-mon[35418]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:18 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3452534607' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:12:18.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:18 smithi103 ceph-mon[35495]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:18.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:18 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3452534607' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:12:20.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:20 smithi098 ceph-mon[35418]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:20 smithi103 ceph-mon[35495]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:20 smithi086 ceph-mon[31733]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:21.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:21 smithi098 ceph-mon[35418]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:21.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:21 smithi103 ceph-mon[35495]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:21 smithi086 ceph-mon[31733]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:23.810 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-17T07:12:23.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:12:23.810 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-17T07:12:23.887 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:12:24.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.102+0000 7ff10e000640 1 -- 172.21.15.86:0/3790654352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff108078500 msgr2=0x7ff108078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.102+0000 7ff10e000640 1 --2- 172.21.15.86:0/3790654352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff108078500 0x7ff108078da0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff100009ca0 tx=0x7ff10002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.102+0000 7ff10e000640 1 -- 172.21.15.86:0/3790654352 shutdown_connections 2024-04-17T07:12:24.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.102+0000 7ff10e000640 1 --2- 172.21.15.86:0/3790654352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff108078500 0x7ff108078da0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.102+0000 7ff10e000640 1 --2- 172.21.15.86:0/3790654352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff108077b60 0x7ff108077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.102+0000 7ff10e000640 1 --2- 172.21.15.86:0/3790654352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff108076920 0x7ff108076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.102+0000 7ff10e000640 1 -- 172.21.15.86:0/3790654352 >> 172.21.15.86:0/3790654352 conn(0x7ff1080ff570 msgr2=0x7ff108101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.102+0000 7ff10e000640 1 -- 172.21.15.86:0/3790654352 shutdown_connections 2024-04-17T07:12:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.102+0000 7ff10e000640 1 -- 172.21.15.86:0/3790654352 wait complete. 2024-04-17T07:12:24.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.103+0000 7ff10e000640 1 Processor -- start 2024-04-17T07:12:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.103+0000 7ff10e000640 1 -- start start 2024-04-17T07:12:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.103+0000 7ff10e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff108076920 0x7ff10819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.103+0000 7ff10e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff108077b60 0x7ff10819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.103+0000 7ff10e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff108078500 0x7ff1081a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.103+0000 7ff10e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff108113f20 con 0x7ff108077b60 2024-04-17T07:12:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.103+0000 7ff10e000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff108113da0 con 0x7ff108076920 2024-04-17T07:12:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.103+0000 7ff10e000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081140a0 con 0x7ff108078500 2024-04-17T07:12:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.104+0000 7ff107e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff108077b60 0x7ff10819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.104+0000 7ff107e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff108077b60 0x7ff10819ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32888/0 (socket says 172.21.15.86:32888) 2024-04-17T07:12:24.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.104+0000 7ff107e00640 1 -- 172.21.15.86:0/3090944014 learned_addr learned my addr 172.21.15.86:0/3090944014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.104+0000 7ff10cc00640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff108076920 0x7ff10819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.104+0000 7ff107e00640 1 -- 172.21.15.86:0/3090944014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff108078500 msgr2=0x7ff1081a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.104+0000 7ff107e00640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff108078500 0x7ff1081a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.104+0000 7ff107e00640 1 -- 172.21.15.86:0/3090944014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff108076920 msgr2=0x7ff10819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.104+0000 7ff107e00640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff108076920 0x7ff10819e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.104+0000 7ff107e00640 1 -- 172.21.15.86:0/3090944014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1081a6140 con 0x7ff108077b60 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.104+0000 7ff107e00640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff108077b60 0x7ff10819ea60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c920 tx=0x7ff0f400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.105+0000 7ff105600640 1 -- 172.21.15.86:0/3090944014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4007c70 con 0x7ff108077b60 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.105+0000 7ff105600640 1 -- 172.21.15.86:0/3090944014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0f4007e10 con 0x7ff108077b60 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.105+0000 7ff10e000640 1 -- 172.21.15.86:0/3090944014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1081a6430 con 0x7ff108077b60 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.105+0000 7ff105600640 1 -- 172.21.15.86:0/3090944014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4018820 con 0x7ff108077b60 2024-04-17T07:12:24.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.105+0000 7ff10e000640 1 -- 172.21.15.86:0/3090944014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1081a6980 con 0x7ff108077b60 2024-04-17T07:12:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.106+0000 7ff10e000640 1 -- 172.21.15.86:0/3090944014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff108110aa0 con 0x7ff108077b60 2024-04-17T07:12:24.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.108+0000 7ff105600640 1 -- 172.21.15.86:0/3090944014 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff0f40189c0 con 0x7ff108077b60 2024-04-17T07:12:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.110+0000 7ff105600640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0f00a3bb0 0x7ff0f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.110+0000 7ff105600640 1 -- 172.21.15.86:0/3090944014 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff0f40d0880 con 0x7ff108077b60 2024-04-17T07:12:24.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.110+0000 7ff10cc00640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0f00a3bb0 0x7ff0f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:24.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.111+0000 7ff10cc00640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0f00a3bb0 0x7ff0f00a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0058c0 tx=0x7ff0fc005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:24.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.115+0000 7ff105600640 1 -- 172.21.15.86:0/3090944014 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff0f4014030 con 0x7ff108077b60 2024-04-17T07:12:24.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:24 smithi103 ceph-mon[35495]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-17T07:12:24.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.319+0000 7ff10e000640 1 -- 172.21.15.86:0/3090944014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff108062c00 con 0x7ff108077b60 2024-04-17T07:12:24.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.322+0000 7ff105600640 1 -- 172.21.15.86:0/3090944014 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24057 (secure 0 0 0) 0x7ff0f409cfc0 con 0x7ff108077b60 2024-04-17T07:12:24.323 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:24.323 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:11:59.971687+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:12:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.324+0000 7ff10e000640 1 -- 172.21.15.86:0/3090944014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0f00a3bb0 msgr2=0x7ff0f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.324+0000 7ff10e000640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0f00a3bb0 0x7ff0f00a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0058c0 tx=0x7ff0fc005c60 comp rx=0 tx=0).stop 2024-04-17T07:12:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.324+0000 7ff10e000640 1 -- 172.21.15.86:0/3090944014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff108077b60 msgr2=0x7ff10819ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.324+0000 7ff10e000640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff108077b60 0x7ff10819ea60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c920 tx=0x7ff0f400cde0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.325+0000 7ff10e000640 1 -- 172.21.15.86:0/3090944014 shutdown_connections 2024-04-17T07:12:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.325+0000 7ff10e000640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff0f00a3bb0 0x7ff0f00a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.325+0000 7ff10e000640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff108078500 0x7ff1081a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.325+0000 7ff10e000640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff108077b60 0x7ff10819ea60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.325+0000 7ff10e000640 1 --2- 172.21.15.86:0/3090944014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff108076920 0x7ff10819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.325+0000 7ff10e000640 1 -- 172.21.15.86:0/3090944014 >> 172.21.15.86:0/3090944014 conn(0x7ff1080ff570 msgr2=0x7ff108100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.325+0000 7ff10e000640 1 -- 172.21.15.86:0/3090944014 shutdown_connections 2024-04-17T07:12:24.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.325+0000 7ff10e000640 1 -- 172.21.15.86:0/3090944014 wait complete. 2024-04-17T07:12:24.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:24 smithi086 ceph-mon[31733]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-17T07:12:24.344 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:12:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:24 smithi098 ceph-mon[35418]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-17T07:12:24.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.538+0000 7f9a54800640 1 -- 172.21.15.86:0/1310224929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a4c1097b0 msgr2=0x7f9a4c109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.538+0000 7f9a54800640 1 --2- 172.21.15.86:0/1310224929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a4c1097b0 0x7f9a4c109c30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009900 tx=0x7f9a3802bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:24.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.538+0000 7f9a54800640 1 -- 172.21.15.86:0/1310224929 shutdown_connections 2024-04-17T07:12:24.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.538+0000 7f9a54800640 1 --2- 172.21.15.86:0/1310224929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a4c10a170 0x7f9a4c110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.538+0000 7f9a54800640 1 --2- 172.21.15.86:0/1310224929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a4c1097b0 0x7f9a4c109c30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.538+0000 7f9a54800640 1 --2- 172.21.15.86:0/1310224929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a4c1085c0 0x7f9a4c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.538+0000 7f9a54800640 1 -- 172.21.15.86:0/1310224929 >> 172.21.15.86:0/1310224929 conn(0x7f9a4c077580 msgr2=0x7f9a4c0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:24.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.538+0000 7f9a54800640 1 -- 172.21.15.86:0/1310224929 shutdown_connections 2024-04-17T07:12:24.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.538+0000 7f9a54800640 1 -- 172.21.15.86:0/1310224929 wait complete. 2024-04-17T07:12:24.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.539+0000 7f9a54800640 1 Processor -- start 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a54800640 1 -- start start 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a54800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a4c1085c0 0x7f9a4c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a54800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a4c1097b0 0x7f9a4c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a54800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a4c10a170 0x7f9a4c1a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a54800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c114230 con 0x7f9a4c1085c0 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a54800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1140b0 con 0x7f9a4c10a170 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a54800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1143b0 con 0x7f9a4c1097b0 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a4c1085c0 0x7f9a4c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a4c1085c0 0x7f9a4c19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32896/0 (socket says 172.21.15.86:32896) 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a51e00640 1 -- 172.21.15.86:0/2391864904 learned_addr learned my addr 172.21.15.86:0/2391864904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.540+0000 7f9a51400640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a4c1097b0 0x7f9a4c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a52800640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a4c10a170 0x7f9a4c1a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:24.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a51e00640 1 -- 172.21.15.86:0/2391864904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a4c1097b0 msgr2=0x7f9a4c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a51e00640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a4c1097b0 0x7f9a4c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a51e00640 1 -- 172.21.15.86:0/2391864904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a4c10a170 msgr2=0x7f9a4c1a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a51e00640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a4c10a170 0x7f9a4c1a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a51e00640 1 -- 172.21.15.86:0/2391864904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c1a65a0 con 0x7f9a4c1085c0 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a52800640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a4c10a170 0x7f9a4c1a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a51e00640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a4c1085c0 0x7f9a4c19e980 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000b450 tx=0x7f9a4000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a46a00640 1 -- 172.21.15.86:0/2391864904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40015020 con 0x7f9a4c1085c0 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a54800640 1 -- 172.21.15.86:0/2391864904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c1a6890 con 0x7f9a4c1085c0 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a46a00640 1 -- 172.21.15.86:0/2391864904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a40011d40 con 0x7f9a4c1085c0 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a46a00640 1 -- 172.21.15.86:0/2391864904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40010e60 con 0x7f9a4c1085c0 2024-04-17T07:12:24.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.541+0000 7f9a54800640 1 -- 172.21.15.86:0/2391864904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c1a6eb0 con 0x7f9a4c1085c0 2024-04-17T07:12:24.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.542+0000 7f9a54800640 1 -- 172.21.15.86:0/2391864904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4c069200 con 0x7f9a4c1085c0 2024-04-17T07:12:24.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.544+0000 7f9a46a00640 1 -- 172.21.15.86:0/2391864904 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9a40018020 con 0x7f9a4c1085c0 2024-04-17T07:12:24.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.546+0000 7f9a46a00640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a340a3bb0 0x7f9a340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.547+0000 7f9a46a00640 1 -- 172.21.15.86:0/2391864904 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9a400d0160 con 0x7f9a4c1085c0 2024-04-17T07:12:24.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.547+0000 7f9a51400640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a340a3bb0 0x7f9a340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:24.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.547+0000 7f9a51400640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a340a3bb0 0x7f9a340a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c19fe90 tx=0x7f9a38000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:24.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.551+0000 7f9a46a00640 1 -- 172.21.15.86:0/2391864904 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9a4c069200 con 0x7f9a4c1085c0 2024-04-17T07:12:24.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.750+0000 7f9a54800640 1 -- 172.21.15.86:0/2391864904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a4c062b90 con 0x7f9a4c1085c0 2024-04-17T07:12:24.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.753+0000 7f9a46a00640 1 -- 172.21.15.86:0/2391864904 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24057 (secure 0 0 0) 0x7f9a4001f020 con 0x7f9a4c1085c0 2024-04-17T07:12:24.753 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:24.753 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:11:59.971687+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.755+0000 7f9a54800640 1 -- 172.21.15.86:0/2391864904 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a340a3bb0 msgr2=0x7f9a340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.755+0000 7f9a54800640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a340a3bb0 0x7f9a340a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c19fe90 tx=0x7f9a38000bc0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.755+0000 7f9a54800640 1 -- 172.21.15.86:0/2391864904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a4c1085c0 msgr2=0x7f9a4c19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.755+0000 7f9a54800640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a4c1085c0 0x7f9a4c19e980 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000b450 tx=0x7f9a4000b910 comp rx=0 tx=0).stop 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.756+0000 7f9a54800640 1 -- 172.21.15.86:0/2391864904 shutdown_connections 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.756+0000 7f9a54800640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a340a3bb0 0x7f9a340a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.756+0000 7f9a54800640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a4c10a170 0x7f9a4c1a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.756+0000 7f9a54800640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a4c1097b0 0x7f9a4c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.756+0000 7f9a54800640 1 --2- 172.21.15.86:0/2391864904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a4c1085c0 0x7f9a4c19e980 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.756+0000 7f9a54800640 1 -- 172.21.15.86:0/2391864904 >> 172.21.15.86:0/2391864904 conn(0x7f9a4c077580 msgr2=0x7f9a4c10cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.756+0000 7f9a54800640 1 -- 172.21.15.86:0/2391864904 shutdown_connections 2024-04-17T07:12:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.756+0000 7f9a54800640 1 -- 172.21.15.86:0/2391864904 wait complete. 2024-04-17T07:12:24.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T07:12:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.990+0000 7fac83e00640 1 -- 172.21.15.86:0/1272122894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac841084a0 msgr2=0x7fac841088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.990+0000 7fac83e00640 1 --2- 172.21.15.86:0/1272122894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac841084a0 0x7fac841088c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fac74009900 tx=0x7fac7402bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.990+0000 7fac83e00640 1 -- 172.21.15.86:0/1272122894 shutdown_connections 2024-04-17T07:12:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.990+0000 7fac83e00640 1 --2- 172.21.15.86:0/1272122894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac8410a050 0x7fac841108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.990+0000 7fac83e00640 1 --2- 172.21.15.86:0/1272122894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac84109690 0x7fac84109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.990+0000 7fac83e00640 1 --2- 172.21.15.86:0/1272122894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac841084a0 0x7fac841088c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.990+0000 7fac83e00640 1 -- 172.21.15.86:0/1272122894 >> 172.21.15.86:0/1272122894 conn(0x7fac84077430 msgr2=0x7fac84079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.990+0000 7fac83e00640 1 -- 172.21.15.86:0/1272122894 shutdown_connections 2024-04-17T07:12:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.990+0000 7fac83e00640 1 -- 172.21.15.86:0/1272122894 wait complete. 2024-04-17T07:12:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.991+0000 7fac83e00640 1 Processor -- start 2024-04-17T07:12:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.991+0000 7fac83e00640 1 -- start start 2024-04-17T07:12:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.991+0000 7fac83e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac841084a0 0x7fac8419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac84109690 0x7fac8419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8410a050 0x7fac841a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac84114030 con 0x7fac8410a050 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fac84113eb0 con 0x7fac841084a0 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac841141b0 con 0x7fac84109690 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8410a050 0x7fac841a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8410a050 0x7fac841a5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32904/0 (socket says 172.21.15.86:32904) 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83400640 1 -- 172.21.15.86:0/3085811131 learned_addr learned my addr 172.21.15.86:0/3085811131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac82a00640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac841084a0 0x7fac8419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83400640 1 -- 172.21.15.86:0/3085811131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac84109690 msgr2=0x7fac8419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83400640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac84109690 0x7fac8419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83400640 1 -- 172.21.15.86:0/3085811131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac841084a0 msgr2=0x7fac8419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83400640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac841084a0 0x7fac8419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.992+0000 7fac83400640 1 -- 172.21.15.86:0/3085811131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac841a62d0 con 0x7fac8410a050 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.993+0000 7fac83400640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8410a050 0x7fac841a5c60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00eaa0 tx=0x7fac7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.993+0000 7fac88a00640 1 -- 172.21.15.86:0/3085811131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00cdb0 con 0x7fac8410a050 2024-04-17T07:12:24.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.993+0000 7fac88a00640 1 -- 172.21.15.86:0/3085811131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac7c005020 con 0x7fac8410a050 2024-04-17T07:12:24.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.993+0000 7fac83e00640 1 -- 172.21.15.86:0/3085811131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac841a65c0 con 0x7fac8410a050 2024-04-17T07:12:24.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.993+0000 7fac88a00640 1 -- 172.21.15.86:0/3085811131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0116d0 con 0x7fac8410a050 2024-04-17T07:12:24.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.993+0000 7fac83e00640 1 -- 172.21.15.86:0/3085811131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac841a6b60 con 0x7fac8410a050 2024-04-17T07:12:24.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.994+0000 7fac83e00640 1 -- 172.21.15.86:0/3085811131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8410f840 con 0x7fac8410a050 2024-04-17T07:12:24.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.996+0000 7fac88a00640 1 -- 172.21.15.86:0/3085811131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fac7c0051d0 con 0x7fac8410a050 2024-04-17T07:12:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.997+0000 7fac88a00640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fac680a3c30 0x7fac680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.998+0000 7fac88a00640 1 -- 172.21.15.86:0/3085811131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fac7c0d0bb0 con 0x7fac8410a050 2024-04-17T07:12:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.998+0000 7fac82a00640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fac680a3c30 0x7fac680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:24.999+0000 7fac82a00640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fac680a3c30 0x7fac680a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac740096c0 tx=0x7fac74005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:25.000+0000 7fac88a00640 1 -- 172.21.15.86:0/3085811131 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fac7c016030 con 0x7fac8410a050 2024-04-17T07:12:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:25.245+0000 7fac83e00640 1 -- 172.21.15.86:0/3085811131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fac8402d5d0 con 0x7fac8410a050 2024-04-17T07:12:25.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:25 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3090944014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:25 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2391864904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:25 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3090944014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:25 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2391864904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:25.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:25 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3090944014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:25.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:25 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2391864904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:26.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.132+0000 7fac88a00640 1 -- 172.21.15.86:0/3085811131 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fac7c09d370 con 0x7fac8410a050 2024-04-17T07:12:26.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 -- 172.21.15.86:0/3085811131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fac680a3c30 msgr2=0x7fac680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:26.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fac680a3c30 0x7fac680a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac740096c0 tx=0x7fac74005fd0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 -- 172.21.15.86:0/3085811131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8410a050 msgr2=0x7fac841a5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:26.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8410a050 0x7fac841a5c60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00eaa0 tx=0x7fac7c00ef60 comp rx=0 tx=0).stop 2024-04-17T07:12:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 -- 172.21.15.86:0/3085811131 shutdown_connections 2024-04-17T07:12:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fac680a3c30 0x7fac680a60f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8410a050 0x7fac841a5c60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac84109690 0x7fac8419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 --2- 172.21.15.86:0/3085811131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac841084a0 0x7fac8419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 -- 172.21.15.86:0/3085811131 >> 172.21.15.86:0/3085811131 conn(0x7fac84077430 msgr2=0x7fac8410ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.136+0000 7fac83e00640 1 -- 172.21.15.86:0/3085811131 shutdown_connections 2024-04-17T07:12:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.137+0000 7fac83e00640 1 -- 172.21.15.86:0/3085811131 wait complete. 2024-04-17T07:12:26.161 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-17T07:12:26.161 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-17T07:12:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.380+0000 7faa27200640 1 -- 172.21.15.86:0/903604979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa281040a0 msgr2=0x7faa281044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.380+0000 7faa27200640 1 --2- 172.21.15.86:0/903604979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa281040a0 0x7faa281044c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faa18009900 tx=0x7faa1802bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.380+0000 7faa27200640 1 -- 172.21.15.86:0/903604979 shutdown_connections 2024-04-17T07:12:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.380+0000 7faa27200640 1 --2- 172.21.15.86:0/903604979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa28105c50 0x7faa2810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.380+0000 7faa27200640 1 --2- 172.21.15.86:0/903604979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa28105290 0x7faa28105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.380+0000 7faa27200640 1 --2- 172.21.15.86:0/903604979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa281040a0 0x7faa281044c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.380+0000 7faa27200640 1 -- 172.21.15.86:0/903604979 >> 172.21.15.86:0/903604979 conn(0x7faa280ff610 msgr2=0x7faa28101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.380+0000 7faa27200640 1 -- 172.21.15.86:0/903604979 shutdown_connections 2024-04-17T07:12:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.380+0000 7faa27200640 1 -- 172.21.15.86:0/903604979 wait complete. 2024-04-17T07:12:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.381+0000 7faa27200640 1 Processor -- start 2024-04-17T07:12:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.381+0000 7faa27200640 1 -- start start 2024-04-17T07:12:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa27200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa281040a0 0x7faa2819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa27200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa28105290 0x7faa2819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa27200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa28105c50 0x7faa281a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa27200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa2810fbe0 con 0x7faa28105c50 2024-04-17T07:12:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa27200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa2810fa60 con 0x7faa28105290 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa27200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa2810fd60 con 0x7faa281040a0 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa26800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa28105c50 0x7faa281a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa26800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa28105c50 0x7faa281a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32922/0 (socket says 172.21.15.86:32922) 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa26800640 1 -- 172.21.15.86:0/1012457033 learned_addr learned my addr 172.21.15.86:0/1012457033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa25e00640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa281040a0 0x7faa2819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa25400640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa28105290 0x7faa2819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa26800640 1 -- 172.21.15.86:0/1012457033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa281040a0 msgr2=0x7faa2819a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa26800640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa281040a0 0x7faa2819a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa26800640 1 -- 172.21.15.86:0/1012457033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa28105290 msgr2=0x7faa2819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa26800640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa28105290 0x7faa2819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa26800640 1 -- 172.21.15.86:0/1012457033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa281a1ee0 con 0x7faa28105c50 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.382+0000 7faa25e00640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa281040a0 0x7faa2819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.383+0000 7faa26800640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa28105c50 0x7faa281a1870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faa2000eaa0 tx=0x7faa2000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:26.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.383+0000 7faa1ea00640 1 -- 172.21.15.86:0/1012457033 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2000cdb0 con 0x7faa28105c50 2024-04-17T07:12:26.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.383+0000 7faa1ea00640 1 -- 172.21.15.86:0/1012457033 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa20005020 con 0x7faa28105c50 2024-04-17T07:12:26.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.383+0000 7faa1ea00640 1 -- 172.21.15.86:0/1012457033 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa200116d0 con 0x7faa28105c50 2024-04-17T07:12:26.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.383+0000 7faa27200640 1 -- 172.21.15.86:0/1012457033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa281a21d0 con 0x7faa28105c50 2024-04-17T07:12:26.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.383+0000 7faa27200640 1 -- 172.21.15.86:0/1012457033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa281a2720 con 0x7faa28105c50 2024-04-17T07:12:26.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.384+0000 7faa27200640 1 -- 172.21.15.86:0/1012457033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa2810b530 con 0x7faa28105c50 2024-04-17T07:12:26.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.387+0000 7faa1ea00640 1 -- 172.21.15.86:0/1012457033 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faa200051d0 con 0x7faa28105c50 2024-04-17T07:12:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.388+0000 7faa1ea00640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa0c0a3c30 0x7faa0c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.388+0000 7faa1ea00640 1 -- 172.21.15.86:0/1012457033 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7faa200d0ca0 con 0x7faa28105c50 2024-04-17T07:12:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.388+0000 7faa25e00640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa0c0a3c30 0x7faa0c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:26.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.389+0000 7faa25e00640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa0c0a3c30 0x7faa0c0a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faa180096c0 tx=0x7faa18000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:26.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.391+0000 7faa1ea00640 1 -- 172.21.15.86:0/1012457033 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faa20016030 con 0x7faa28105c50 2024-04-17T07:12:26.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:26 smithi098 ceph-mon[35418]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-17T07:12:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:26 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3085811131' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T07:12:26.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:26 smithi103 ceph-mon[35495]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-17T07:12:26.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:26 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3085811131' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T07:12:26.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:26 smithi086 ceph-mon[31733]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-17T07:12:26.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:26 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3085811131' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-17T07:12:26.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.637+0000 7faa27200640 1 -- 172.21.15.86:0/1012457033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7faa2802d5d0 con 0x7faa28105c50 2024-04-17T07:12:26.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.689+0000 7faa1ea00640 1 -- 172.21.15.86:0/1012457033 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7faa2009d460 con 0x7faa28105c50 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.692+0000 7faa27200640 1 -- 172.21.15.86:0/1012457033 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa0c0a3c30 msgr2=0x7faa0c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.692+0000 7faa27200640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa0c0a3c30 0x7faa0c0a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faa180096c0 tx=0x7faa18000bc0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.692+0000 7faa27200640 1 -- 172.21.15.86:0/1012457033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa28105c50 msgr2=0x7faa281a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.692+0000 7faa27200640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa28105c50 0x7faa281a1870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faa2000eaa0 tx=0x7faa2000ef60 comp rx=0 tx=0).stop 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.692+0000 7faa27200640 1 -- 172.21.15.86:0/1012457033 shutdown_connections 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.692+0000 7faa27200640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa0c0a3c30 0x7faa0c0a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.692+0000 7faa27200640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa28105c50 0x7faa281a1870 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.693+0000 7faa27200640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa28105290 0x7faa2819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.693+0000 7faa27200640 1 --2- 172.21.15.86:0/1012457033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa281040a0 0x7faa2819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.693+0000 7faa27200640 1 -- 172.21.15.86:0/1012457033 >> 172.21.15.86:0/1012457033 conn(0x7faa280ff610 msgr2=0x7faa28108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.693+0000 7faa27200640 1 -- 172.21.15.86:0/1012457033 shutdown_connections 2024-04-17T07:12:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.693+0000 7faa27200640 1 -- 172.21.15.86:0/1012457033 wait complete. 2024-04-17T07:12:26.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-17T07:12:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.922+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3585912058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b301006f0 msgr2=0x7f7b30100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.922+0000 7f7b2fe00640 1 --2- 172.21.15.86:0/3585912058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b301006f0 0x7f7b30100b10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009900 tx=0x7f7b2002bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.922+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3585912058 shutdown_connections 2024-04-17T07:12:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.922+0000 7f7b2fe00640 1 --2- 172.21.15.86:0/3585912058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b301019f0 0x7f7b30101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.922+0000 7f7b2fe00640 1 --2- 172.21.15.86:0/3585912058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b30101050 0x7f7b301014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.922+0000 7f7b2fe00640 1 --2- 172.21.15.86:0/3585912058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b301006f0 0x7f7b30100b10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.922+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3585912058 >> 172.21.15.86:0/3585912058 conn(0x7f7b300fc2d0 msgr2=0x7f7b300fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.922+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3585912058 shutdown_connections 2024-04-17T07:12:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.922+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3585912058 wait complete. 2024-04-17T07:12:26.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.923+0000 7f7b2fe00640 1 Processor -- start 2024-04-17T07:12:26.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.923+0000 7f7b2fe00640 1 -- start start 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b301006f0 0x7f7b3019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b30101050 0x7f7b3019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b301019f0 0x7f7b301a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3010fc20 con 0x7f7b30101050 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2fe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3010faa0 con 0x7f7b301006f0 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3010fda0 con 0x7f7b301019f0 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b30101050 0x7f7b3019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b30101050 0x7f7b3019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54048/0 (socket says 172.21.15.86:54048) 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2e000640 1 -- 172.21.15.86:0/3562476564 learned_addr learned my addr 172.21.15.86:0/3562476564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2f400640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b301019f0 0x7f7b301a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2e000640 1 -- 172.21.15.86:0/3562476564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b301019f0 msgr2=0x7f7b301a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2e000640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b301019f0 0x7f7b301a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2e000640 1 -- 172.21.15.86:0/3562476564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b301006f0 msgr2=0x7f7b3019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2e000640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b301006f0 0x7f7b3019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2e000640 1 -- 172.21.15.86:0/3562476564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b301a1e60 con 0x7f7b30101050 2024-04-17T07:12:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.924+0000 7f7b2f400640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b301019f0 0x7f7b301a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.925+0000 7f7b2e000640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b30101050 0x7f7b3019a780 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00d950 tx=0x7f7b1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.925+0000 7f7b27400640 1 -- 172.21.15.86:0/3562476564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c017070 con 0x7f7b30101050 2024-04-17T07:12:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.925+0000 7f7b27400640 1 -- 172.21.15.86:0/3562476564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b1c009c30 con 0x7f7b30101050 2024-04-17T07:12:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.925+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3562476564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b301a2150 con 0x7f7b30101050 2024-04-17T07:12:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.925+0000 7f7b27400640 1 -- 172.21.15.86:0/3562476564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c00f620 con 0x7f7b30101050 2024-04-17T07:12:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.925+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3562476564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b301a26f0 con 0x7f7b30101050 2024-04-17T07:12:26.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.926+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3562476564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b3010b440 con 0x7f7b30101050 2024-04-17T07:12:26.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.929+0000 7f7b27400640 1 -- 172.21.15.86:0/3562476564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7b1c0109a0 con 0x7f7b30101050 2024-04-17T07:12:26.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.931+0000 7f7b27400640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b140a3c30 0x7f7b140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:26.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.931+0000 7f7b27400640 1 -- 172.21.15.86:0/3562476564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7b1c0d0250 con 0x7f7b30101050 2024-04-17T07:12:26.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.931+0000 7f7b2ea00640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b140a3c30 0x7f7b140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:26.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.932+0000 7f7b2ea00640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b140a3c30 0x7f7b140a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005c20 tx=0x7f7b20000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:26.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:26.933+0000 7f7b27400640 1 -- 172.21.15.86:0/3562476564 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7b1c009de0 con 0x7f7b30101050 2024-04-17T07:12:27.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.179+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3562476564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f7b3002d5d0 con 0x7f7b30101050 2024-04-17T07:12:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3085811131' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-17T07:12:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1012457033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1012457033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]: dispatch 2024-04-17T07:12:27.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-17T07:12:27.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:27 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3085811131' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1012457033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1012457033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-17T07:12:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-17T07:12:27.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-17T07:12:27.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-17T07:12:27.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]: dispatch 2024-04-17T07:12:27.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-17T07:12:27.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:27 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-17T07:12:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3085811131' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-17T07:12:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1012457033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1012457033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-17T07:12:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]: dispatch 2024-04-17T07:12:27.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-04-17T07:12:27.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:27 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-17T07:12:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.693+0000 7f7b27400640 1 -- 172.21.15.86:0/3562476564 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f7b1c09ca70 con 0x7f7b30101050 2024-04-17T07:12:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.696+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3562476564 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b140a3c30 msgr2=0x7f7b140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.696+0000 7f7b2fe00640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b140a3c30 0x7f7b140a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005c20 tx=0x7f7b20000bc0 comp rx=0 tx=0).stop 2024-04-17T07:12:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.696+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3562476564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b30101050 msgr2=0x7f7b3019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.696+0000 7f7b2fe00640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b30101050 0x7f7b3019a780 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00d950 tx=0x7f7b1c00de10 comp rx=0 tx=0).stop 2024-04-17T07:12:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.697+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3562476564 shutdown_connections 2024-04-17T07:12:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.697+0000 7f7b2fe00640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b140a3c30 0x7f7b140a60f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.697+0000 7f7b2fe00640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b301019f0 0x7f7b301a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.697+0000 7f7b2fe00640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b30101050 0x7f7b3019a780 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.697+0000 7f7b2fe00640 1 --2- 172.21.15.86:0/3562476564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b301006f0 0x7f7b3019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.697+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3562476564 >> 172.21.15.86:0/3562476564 conn(0x7f7b300fc2d0 msgr2=0x7f7b300fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.697+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3562476564 shutdown_connections 2024-04-17T07:12:27.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.697+0000 7f7b2fe00640 1 -- 172.21.15.86:0/3562476564 wait complete. 2024-04-17T07:12:27.732 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-17T07:12:27.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.921+0000 7f292ca00640 1 -- 172.21.15.86:0/1749208905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29280749c0 msgr2=0x7f2928074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:27.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.921+0000 7f292ca00640 1 --2- 172.21.15.86:0/1749208905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29280749c0 0x7f2928074e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2914009900 tx=0x7f291402bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:27.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.921+0000 7f292ca00640 1 -- 172.21.15.86:0/1749208905 shutdown_connections 2024-04-17T07:12:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.921+0000 7f292ca00640 1 --2- 172.21.15.86:0/1749208905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2928109d30 0x7f292810c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.921+0000 7f292ca00640 1 --2- 172.21.15.86:0/1749208905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29280749c0 0x7f2928074e40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.921+0000 7f292ca00640 1 --2- 172.21.15.86:0/1749208905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2928075fa0 0x7f29280743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.921+0000 7f292ca00640 1 -- 172.21.15.86:0/1749208905 >> 172.21.15.86:0/1749208905 conn(0x7f29280fd400 msgr2=0x7f29280ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.921+0000 7f292ca00640 1 -- 172.21.15.86:0/1749208905 shutdown_connections 2024-04-17T07:12:27.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.921+0000 7f292ca00640 1 -- 172.21.15.86:0/1749208905 wait complete. 2024-04-17T07:12:27.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.922+0000 7f292ca00640 1 Processor -- start 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.922+0000 7f292ca00640 1 -- start start 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f292ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29280749c0 0x7f292819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f292ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2928075fa0 0x7f292819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f292ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2928109d30 0x7f29281a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f292ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f292810fbf0 con 0x7f2928109d30 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f292ca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f292810fa70 con 0x7f2928075fa0 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f292ca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f292810fd70 con 0x7f29280749c0 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f2927e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2928109d30 0x7f29281a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f2927e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2928109d30 0x7f29281a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54078/0 (socket says 172.21.15.86:54078) 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f2927e00640 1 -- 172.21.15.86:0/4112144362 learned_addr learned my addr 172.21.15.86:0/4112144362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:27.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f2927400640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29280749c0 0x7f292819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.923+0000 7f2926a00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2928075fa0 0x7f292819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f2927e00640 1 -- 172.21.15.86:0/4112144362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29280749c0 msgr2=0x7f292819a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f2927e00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29280749c0 0x7f292819a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f2927e00640 1 -- 172.21.15.86:0/4112144362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2928075fa0 msgr2=0x7f292819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f2927e00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2928075fa0 0x7f292819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f2927e00640 1 -- 172.21.15.86:0/4112144362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29281a1ee0 con 0x7f2928109d30 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f2927400640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29280749c0 0x7f292819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f2927e00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2928109d30 0x7f29281a1870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f292000eaa0 tx=0x7f292000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f291fe00640 1 -- 172.21.15.86:0/4112144362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292000cdb0 con 0x7f2928109d30 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f291fe00640 1 -- 172.21.15.86:0/4112144362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2920005020 con 0x7f2928109d30 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f292ca00640 1 -- 172.21.15.86:0/4112144362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29281a21d0 con 0x7f2928109d30 2024-04-17T07:12:27.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.924+0000 7f291fe00640 1 -- 172.21.15.86:0/4112144362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200116d0 con 0x7f2928109d30 2024-04-17T07:12:27.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.925+0000 7f292ca00640 1 -- 172.21.15.86:0/4112144362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29281a26a0 con 0x7f2928109d30 2024-04-17T07:12:27.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.926+0000 7f292ca00640 1 -- 172.21.15.86:0/4112144362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2928108a60 con 0x7f2928109d30 2024-04-17T07:12:27.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.928+0000 7f291fe00640 1 -- 172.21.15.86:0/4112144362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f29200051d0 con 0x7f2928109d30 2024-04-17T07:12:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.930+0000 7f291fe00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f290c0a3bb0 0x7f290c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.930+0000 7f291fe00640 1 -- 172.21.15.86:0/4112144362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f29200d09b0 con 0x7f2928109d30 2024-04-17T07:12:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.930+0000 7f2927400640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f290c0a3bb0 0x7f290c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.931+0000 7f2927400640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f290c0a3bb0 0x7f290c0a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29180058c0 tx=0x7f2918005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:27.935+0000 7f291fe00640 1 -- 172.21.15.86:0/4112144362 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2920016030 con 0x7f2928109d30 2024-04-17T07:12:28.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:28.180+0000 7f292ca00640 1 -- 172.21.15.86:0/4112144362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f2928062b60 con 0x7f2928109d30 2024-04-17T07:12:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: pgmap v379: 129 pgs: 129 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3562476564' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T07:12:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3562476564' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-17T07:12:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] up:active 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]': finished 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: osdmap e90: 12 total, 12 up, 12 in 2024-04-17T07:12:28.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:28 smithi098 ceph-mon[35418]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-17T07:12:28.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: pgmap v379: 129 pgs: 129 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:28.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3562476564' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T07:12:28.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3562476564' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-17T07:12:28.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] up:active 2024-04-17T07:12:28.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-17T07:12:28.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-17T07:12:28.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-17T07:12:28.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-17T07:12:28.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-04-17T07:12:28.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-17T07:12:28.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-17T07:12:28.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-17T07:12:28.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-17T07:12:28.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]': finished 2024-04-17T07:12:28.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-17T07:12:28.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-17T07:12:28.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: osdmap e90: 12 total, 12 up, 12 in 2024-04-17T07:12:28.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:28 smithi103 ceph-mon[35495]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-17T07:12:28.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: pgmap v379: 129 pgs: 129 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-17T07:12:28.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3562476564' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-17T07:12:28.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3562476564' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] up:active 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 10]}]': finished 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-04-17T07:12:28.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-17T07:12:28.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: osdmap e90: 12 total, 12 up, 12 in 2024-04-17T07:12:28.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:28 smithi086 ceph-mon[31733]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-17T07:12:29.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.163+0000 7f291fe00640 1 -- 172.21.15.86:0/4112144362 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f292009cfd0 con 0x7f2928109d30 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.166+0000 7f292ca00640 1 -- 172.21.15.86:0/4112144362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f290c0a3bb0 msgr2=0x7f290c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.166+0000 7f292ca00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f290c0a3bb0 0x7f290c0a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29180058c0 tx=0x7f2918005850 comp rx=0 tx=0).stop 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.166+0000 7f292ca00640 1 -- 172.21.15.86:0/4112144362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2928109d30 msgr2=0x7f29281a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.166+0000 7f292ca00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2928109d30 0x7f29281a1870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f292000eaa0 tx=0x7f292000ef60 comp rx=0 tx=0).stop 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.167+0000 7f292ca00640 1 -- 172.21.15.86:0/4112144362 shutdown_connections 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.167+0000 7f292ca00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f290c0a3bb0 0x7f290c0a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.167+0000 7f292ca00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2928109d30 0x7f29281a1870 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.167+0000 7f292ca00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2928075fa0 0x7f292819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.167+0000 7f292ca00640 1 --2- 172.21.15.86:0/4112144362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29280749c0 0x7f292819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.167+0000 7f292ca00640 1 -- 172.21.15.86:0/4112144362 >> 172.21.15.86:0/4112144362 conn(0x7f29280fd400 msgr2=0x7f2928105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:29.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.167+0000 7f292ca00640 1 -- 172.21.15.86:0/4112144362 shutdown_connections 2024-04-17T07:12:29.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.167+0000 7f292ca00640 1 -- 172.21.15.86:0/4112144362 wait complete. 2024-04-17T07:12:29.196 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-17T07:12:29.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-17T07:12:29.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.388+0000 7fe0efe00640 1 -- 172.21.15.86:0/1557073505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f0105290 msgr2=0x7fe0f0105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:29.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.388+0000 7fe0efe00640 1 --2- 172.21.15.86:0/1557073505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f0105290 0x7fe0f0105710 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009900 tx=0x7fe0d802bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:29.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.388+0000 7fe0efe00640 1 -- 172.21.15.86:0/1557073505 shutdown_connections 2024-04-17T07:12:29.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.388+0000 7fe0efe00640 1 --2- 172.21.15.86:0/1557073505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f0105c50 0x7fe0f010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:29.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.388+0000 7fe0efe00640 1 --2- 172.21.15.86:0/1557073505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f0105290 0x7fe0f0105710 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:29.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.388+0000 7fe0efe00640 1 --2- 172.21.15.86:0/1557073505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f01040a0 0x7fe0f01044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:29.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.388+0000 7fe0efe00640 1 -- 172.21.15.86:0/1557073505 >> 172.21.15.86:0/1557073505 conn(0x7fe0f00ff610 msgr2=0x7fe0f0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:29.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.388+0000 7fe0efe00640 1 -- 172.21.15.86:0/1557073505 shutdown_connections 2024-04-17T07:12:29.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.388+0000 7fe0efe00640 1 -- 172.21.15.86:0/1557073505 wait complete. 2024-04-17T07:12:29.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.389+0000 7fe0efe00640 1 Processor -- start 2024-04-17T07:12:29.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.389+0000 7fe0efe00640 1 -- start start 2024-04-17T07:12:29.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.389+0000 7fe0efe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f01040a0 0x7fe0f019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:29.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.389+0000 7fe0efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f0105290 0x7fe0f019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:29.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.389+0000 7fe0efe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f0105c50 0x7fe0f01a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:29.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.389+0000 7fe0efe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010fbe0 con 0x7fe0f0105290 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.389+0000 7fe0efe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010fa60 con 0x7fe0f01040a0 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.389+0000 7fe0efe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010fd60 con 0x7fe0f0105c50 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.390+0000 7fe0e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f0105290 0x7fe0f019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.390+0000 7fe0e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f0105290 0x7fe0f019a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54088/0 (socket says 172.21.15.86:54088) 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.390+0000 7fe0e7e00640 1 -- 172.21.15.86:0/1918224204 learned_addr learned my addr 172.21.15.86:0/1918224204 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.390+0000 7fe0eea00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f01040a0 0x7fe0f019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.390+0000 7fe0ef400640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f0105c50 0x7fe0f01a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.390+0000 7fe0e7e00640 1 -- 172.21.15.86:0/1918224204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f0105c50 msgr2=0x7fe0f01a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.390+0000 7fe0e7e00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f0105c50 0x7fe0f01a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.390+0000 7fe0e7e00640 1 -- 172.21.15.86:0/1918224204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f01040a0 msgr2=0x7fe0f019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.390+0000 7fe0e7e00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f01040a0 0x7fe0f019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.390+0000 7fe0e7e00640 1 -- 172.21.15.86:0/1918224204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f01a1eb0 con 0x7fe0f0105290 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.391+0000 7fe0e7e00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f0105290 0x7fe0f019a7d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8006010 tx=0x7fe0d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.391+0000 7fe0f4a00640 1 -- 172.21.15.86:0/1918224204 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8043020 con 0x7fe0f0105290 2024-04-17T07:12:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.391+0000 7fe0f4a00640 1 -- 172.21.15.86:0/1918224204 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0d8004aa0 con 0x7fe0f0105290 2024-04-17T07:12:29.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.391+0000 7fe0f4a00640 1 -- 172.21.15.86:0/1918224204 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8003c60 con 0x7fe0f0105290 2024-04-17T07:12:29.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.391+0000 7fe0efe00640 1 -- 172.21.15.86:0/1918224204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f01a2140 con 0x7fe0f0105290 2024-04-17T07:12:29.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.391+0000 7fe0efe00640 1 -- 172.21.15.86:0/1918224204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01a2690 con 0x7fe0f0105290 2024-04-17T07:12:29.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.392+0000 7fe0efe00640 1 -- 172.21.15.86:0/1918224204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f010b410 con 0x7fe0f0105290 2024-04-17T07:12:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.394+0000 7fe0f4a00640 1 -- 172.21.15.86:0/1918224204 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe0d8046020 con 0x7fe0f0105290 2024-04-17T07:12:29.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.395+0000 7fe0f4a00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0d40a3bb0 0x7fe0d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:29.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.396+0000 7fe0f4a00640 1 -- 172.21.15.86:0/1918224204 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe0d80f3290 con 0x7fe0f0105290 2024-04-17T07:12:29.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.396+0000 7fe0eea00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0d40a3bb0 0x7fe0d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:29.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.396+0000 7fe0eea00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0d40a3bb0 0x7fe0d40a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0005fd0 tx=0x7fe0e0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.398+0000 7fe0f4a00640 1 -- 172.21.15.86:0/1918224204 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe0d804d020 con 0x7fe0f0105290 2024-04-17T07:12:29.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:29 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4112144362' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T07:12:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:29 smithi098 ceph-mon[35418]: osdmap e91: 12 total, 12 up, 12 in 2024-04-17T07:12:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:29 smithi098 ceph-mon[35418]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-17T07:12:29.503 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:12:29 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b[66166]: 2024-04-17T07:12:29.131+0000 7f64d0a00700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-17T07:12:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:29 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4112144362' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T07:12:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:29 smithi103 ceph-mon[35495]: osdmap e91: 12 total, 12 up, 12 in 2024-04-17T07:12:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:29 smithi103 ceph-mon[35495]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-17T07:12:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:29 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4112144362' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-17T07:12:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:29 smithi086 ceph-mon[31733]: osdmap e91: 12 total, 12 up, 12 in 2024-04-17T07:12:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:29 smithi086 ceph-mon[31733]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-17T07:12:29.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:29.629+0000 7fe0efe00640 1 -- 172.21.15.86:0/1918224204 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe0f00623a0 con 0x7fe0d40a3bb0 2024-04-17T07:12:30.081 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:12:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:12:29.641+0000 7ffa4ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:12:30.081 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:12:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:12:29.641+0000 7ffa4ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:12:30.081 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:12:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:12:29.641+0000 7ffa4ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:12:30.081 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:12:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:12:29.641+0000 7ffa4ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:12:30.081 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:12:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:12:29.641+0000 7ffa4ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:12:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.227+0000 7fe0f4a00640 1 -- 172.21.15.86:0/1918224204 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe0f00623a0 con 0x7fe0d40a3bb0 2024-04-17T07:12:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.231+0000 7fe0efe00640 1 -- 172.21.15.86:0/1918224204 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0d40a3bb0 msgr2=0x7fe0d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:30.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.231+0000 7fe0efe00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0d40a3bb0 0x7fe0d40a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0005fd0 tx=0x7fe0e0005850 comp rx=0 tx=0).stop 2024-04-17T07:12:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.231+0000 7fe0efe00640 1 -- 172.21.15.86:0/1918224204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f0105290 msgr2=0x7fe0f019a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.231+0000 7fe0efe00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f0105290 0x7fe0f019a7d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8006010 tx=0x7fe0d80042f0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.231+0000 7fe0efe00640 1 -- 172.21.15.86:0/1918224204 shutdown_connections 2024-04-17T07:12:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.231+0000 7fe0efe00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0d40a3bb0 0x7fe0d40a6070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.231+0000 7fe0efe00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0f0105c50 0x7fe0f01a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.231+0000 7fe0efe00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0f0105290 0x7fe0f019a7d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.231+0000 7fe0efe00640 1 --2- 172.21.15.86:0/1918224204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0f01040a0 0x7fe0f019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.231+0000 7fe0efe00640 1 -- 172.21.15.86:0/1918224204 >> 172.21.15.86:0/1918224204 conn(0x7fe0f00ff610 msgr2=0x7fe0f01085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.232+0000 7fe0efe00640 1 -- 172.21.15.86:0/1918224204 shutdown_connections 2024-04-17T07:12:30.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.232+0000 7fe0efe00640 1 -- 172.21.15.86:0/1918224204 wait complete. 2024-04-17T07:12:30.253 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-17T07:12:30.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.466+0000 7fc53e800640 1 -- 172.21.15.86:0/3187370222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc538102fc0 msgr2=0x7fc538103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:30.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.466+0000 7fc53e800640 1 --2- 172.21.15.86:0/3187370222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc538102fc0 0x7fc538103440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc524009900 tx=0x7fc52402bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:30.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.466+0000 7fc53e800640 1 -- 172.21.15.86:0/3187370222 shutdown_connections 2024-04-17T07:12:30.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.466+0000 7fc53e800640 1 --2- 172.21.15.86:0/3187370222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc538103980 0x7fc53810a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.466+0000 7fc53e800640 1 --2- 172.21.15.86:0/3187370222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc538102fc0 0x7fc538103440 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.466+0000 7fc53e800640 1 --2- 172.21.15.86:0/3187370222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc538101e70 0x7fc538102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.466+0000 7fc53e800640 1 -- 172.21.15.86:0/3187370222 >> 172.21.15.86:0/3187370222 conn(0x7fc5380fd400 msgr2=0x7fc5380ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:30.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.466+0000 7fc53e800640 1 -- 172.21.15.86:0/3187370222 shutdown_connections 2024-04-17T07:12:30.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.466+0000 7fc53e800640 1 -- 172.21.15.86:0/3187370222 wait complete. 2024-04-17T07:12:30.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.467+0000 7fc53e800640 1 Processor -- start 2024-04-17T07:12:30.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.467+0000 7fc53e800640 1 -- start start 2024-04-17T07:12:30.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc538101e70 0x7fc53819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:30.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc538102fc0 0x7fc53819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:30.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc538103980 0x7fc5381a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:30.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810d9f0 con 0x7fc538102fc0 2024-04-17T07:12:30.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53e800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810d870 con 0x7fc538101e70 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53e800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc53810db70 con 0x7fc538103980 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc538102fc0 0x7fc53819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc538102fc0 0x7fc53819a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54114/0 (socket says 172.21.15.86:54114) 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53ca00640 1 -- 172.21.15.86:0/606339050 learned_addr learned my addr 172.21.15.86:0/606339050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53d400640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc538101e70 0x7fc53819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53de00640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc538103980 0x7fc5381a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53ca00640 1 -- 172.21.15.86:0/606339050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc538103980 msgr2=0x7fc5381a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53ca00640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc538103980 0x7fc5381a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53ca00640 1 -- 172.21.15.86:0/606339050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc538101e70 msgr2=0x7fc53819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53ca00640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc538101e70 0x7fc53819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.468+0000 7fc53ca00640 1 -- 172.21.15.86:0/606339050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5381a1f00 con 0x7fc538102fc0 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.469+0000 7fc53ca00640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc538102fc0 0x7fc53819a820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc524005ab0 tx=0x7fc5240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:30.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.469+0000 7fc536000640 1 -- 172.21.15.86:0/606339050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524043020 con 0x7fc538102fc0 2024-04-17T07:12:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.469+0000 7fc53e800640 1 -- 172.21.15.86:0/606339050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5381a2190 con 0x7fc538102fc0 2024-04-17T07:12:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.469+0000 7fc536000640 1 -- 172.21.15.86:0/606339050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5240054c0 con 0x7fc538102fc0 2024-04-17T07:12:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.469+0000 7fc536000640 1 -- 172.21.15.86:0/606339050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524003d10 con 0x7fc538102fc0 2024-04-17T07:12:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.469+0000 7fc53e800640 1 -- 172.21.15.86:0/606339050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5381a2730 con 0x7fc538102fc0 2024-04-17T07:12:30.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.470+0000 7fc53e800640 1 -- 172.21.15.86:0/606339050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5381a2a50 con 0x7fc538102fc0 2024-04-17T07:12:30.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.472+0000 7fc536000640 1 -- 172.21.15.86:0/606339050 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc524046020 con 0x7fc538102fc0 2024-04-17T07:12:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.474+0000 7fc536000640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc5200a3c30 0x7fc5200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.474+0000 7fc536000640 1 -- 172.21.15.86:0/606339050 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc5240f2b20 con 0x7fc538102fc0 2024-04-17T07:12:30.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.474+0000 7fc53d400640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc5200a3c30 0x7fc5200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:30.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.475+0000 7fc53d400640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc5200a3c30 0x7fc5200a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0058c0 tx=0x7fc52c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:30.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.478+0000 7fc536000640 1 -- 172.21.15.86:0/606339050 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc52404d020 con 0x7fc538102fc0 2024-04-17T07:12:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:30 smithi098 ceph-mon[35418]: pgmap v382: 129 pgs: 3 peering, 126 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2024-04-17T07:12:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:30 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4112144362' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-17T07:12:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:30 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] up:active 2024-04-17T07:12:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:30 smithi098 ceph-mon[35418]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-17T07:12:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:30 smithi098 ceph-mon[35418]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-17T07:12:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:30 smithi098 ceph-mon[35418]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-17T07:12:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:30 smithi098 ceph-mon[35418]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-17T07:12:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:30 smithi103 ceph-mon[35495]: pgmap v382: 129 pgs: 3 peering, 126 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2024-04-17T07:12:30.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:30 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4112144362' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-17T07:12:30.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:30 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] up:active 2024-04-17T07:12:30.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:30 smithi103 ceph-mon[35495]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-17T07:12:30.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:30 smithi103 ceph-mon[35495]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-17T07:12:30.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:30 smithi103 ceph-mon[35495]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-17T07:12:30.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:30 smithi103 ceph-mon[35495]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-17T07:12:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:30 smithi086 ceph-mon[31733]: pgmap v382: 129 pgs: 3 peering, 126 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2024-04-17T07:12:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:30 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4112144362' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-17T07:12:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:30 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] up:active 2024-04-17T07:12:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:30 smithi086 ceph-mon[31733]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-17T07:12:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:30 smithi086 ceph-mon[31733]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-17T07:12:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:30 smithi086 ceph-mon[31733]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-17T07:12:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:30 smithi086 ceph-mon[31733]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-17T07:12:30.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.711+0000 7fc53e800640 1 -- 172.21.15.86:0/606339050 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc5380758b0 con 0x7fc5200a3c30 2024-04-17T07:12:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.771+0000 7fc536000640 1 -- 172.21.15.86:0/606339050 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc5380758b0 con 0x7fc5200a3c30 2024-04-17T07:12:30.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 -- 172.21.15.86:0/606339050 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc5200a3c30 msgr2=0x7fc5200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:30.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc5200a3c30 0x7fc5200a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0058c0 tx=0x7fc52c00a4c0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 -- 172.21.15.86:0/606339050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc538102fc0 msgr2=0x7fc53819a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:30.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc538102fc0 0x7fc53819a820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc524005ab0 tx=0x7fc5240042f0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 -- 172.21.15.86:0/606339050 shutdown_connections 2024-04-17T07:12:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc5200a3c30 0x7fc5200a60f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc538103980 0x7fc5381a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc538102fc0 0x7fc53819a820 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 --2- 172.21.15.86:0/606339050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc538101e70 0x7fc53819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 -- 172.21.15.86:0/606339050 >> 172.21.15.86:0/606339050 conn(0x7fc5380fd400 msgr2=0x7fc538074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.774+0000 7fc53e800640 1 -- 172.21.15.86:0/606339050 shutdown_connections 2024-04-17T07:12:30.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:30.775+0000 7fc53e800640 1 -- 172.21.15.86:0/606339050 wait complete. 2024-04-17T07:12:30.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-17T07:12:31.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.020+0000 7fd447200640 1 -- 172.21.15.86:0/518972054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44810a050 msgr2=0x7fd4481108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:31.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.020+0000 7fd447200640 1 --2- 172.21.15.86:0/518972054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44810a050 0x7fd4481108e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd440009ca0 tx=0x7fd44002c100 comp rx=0 tx=0).stop 2024-04-17T07:12:31.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.020+0000 7fd447200640 1 -- 172.21.15.86:0/518972054 shutdown_connections 2024-04-17T07:12:31.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.020+0000 7fd447200640 1 --2- 172.21.15.86:0/518972054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44810a050 0x7fd4481108e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.020+0000 7fd447200640 1 --2- 172.21.15.86:0/518972054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd448109690 0x7fd448109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.020+0000 7fd447200640 1 --2- 172.21.15.86:0/518972054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4481084a0 0x7fd4481088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.020+0000 7fd447200640 1 -- 172.21.15.86:0/518972054 >> 172.21.15.86:0/518972054 conn(0x7fd448077430 msgr2=0x7fd448079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:31.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.021+0000 7fd447200640 1 -- 172.21.15.86:0/518972054 shutdown_connections 2024-04-17T07:12:31.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.021+0000 7fd447200640 1 -- 172.21.15.86:0/518972054 wait complete. 2024-04-17T07:12:31.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.021+0000 7fd447200640 1 Processor -- start 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd447200640 1 -- start start 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd447200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4481084a0 0x7fd44819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd447200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd448109690 0x7fd44819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd447200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44810a050 0x7fd4481a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd447200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd448113fe0 con 0x7fd44810a050 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd447200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd448113e60 con 0x7fd4481084a0 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd447200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd448114160 con 0x7fd448109690 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd446800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44810a050 0x7fd4481a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd446800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44810a050 0x7fd4481a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54124/0 (socket says 172.21.15.86:54124) 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd446800640 1 -- 172.21.15.86:0/1061745284 learned_addr learned my addr 172.21.15.86:0/1061745284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.022+0000 7fd445e00640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4481084a0 0x7fd44819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.023+0000 7fd445400640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd448109690 0x7fd44819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.023+0000 7fd446800640 1 -- 172.21.15.86:0/1061745284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd448109690 msgr2=0x7fd44819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:31.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.023+0000 7fd446800640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd448109690 0x7fd44819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.023+0000 7fd446800640 1 -- 172.21.15.86:0/1061745284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4481084a0 msgr2=0x7fd44819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:31.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.023+0000 7fd446800640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4481084a0 0x7fd44819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.023+0000 7fd446800640 1 -- 172.21.15.86:0/1061745284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4481a6260 con 0x7fd44810a050 2024-04-17T07:12:31.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.023+0000 7fd446800640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44810a050 0x7fd4481a5bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd440032520 tx=0x7fd440034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:31.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.023+0000 7fd43ea00640 1 -- 172.21.15.86:0/1061745284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440039040 con 0x7fd44810a050 2024-04-17T07:12:31.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.024+0000 7fd447200640 1 -- 172.21.15.86:0/1061745284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4481a64f0 con 0x7fd44810a050 2024-04-17T07:12:31.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.024+0000 7fd43ea00640 1 -- 172.21.15.86:0/1061745284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd440044070 con 0x7fd44810a050 2024-04-17T07:12:31.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.024+0000 7fd43ea00640 1 -- 172.21.15.86:0/1061745284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4400043c0 con 0x7fd44810a050 2024-04-17T07:12:31.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.024+0000 7fd447200640 1 -- 172.21.15.86:0/1061745284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4481a6a30 con 0x7fd44810a050 2024-04-17T07:12:31.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.025+0000 7fd447200640 1 -- 172.21.15.86:0/1061745284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd44810f7c0 con 0x7fd44810a050 2024-04-17T07:12:31.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.027+0000 7fd43ea00640 1 -- 172.21.15.86:0/1061745284 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd440004560 con 0x7fd44810a050 2024-04-17T07:12:31.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.028+0000 7fd43ea00640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd42c0a3c30 0x7fd42c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:31.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.028+0000 7fd43ea00640 1 -- 172.21.15.86:0/1061745284 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd4400f2e10 con 0x7fd44810a050 2024-04-17T07:12:31.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.028+0000 7fd445e00640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd42c0a3c30 0x7fd42c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:31.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.029+0000 7fd445e00640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd42c0a3c30 0x7fd42c0a60f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd4481094f0 tx=0x7fd43800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:31.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.033+0000 7fd43ea00640 1 -- 172.21.15.86:0/1061745284 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd4400f6030 con 0x7fd44810a050 2024-04-17T07:12:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.274+0000 7fd447200640 1 -- 172.21.15.86:0/1061745284 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fd448062430 con 0x7fd42c0a3c30 2024-04-17T07:12:31.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.282+0000 7fd43ea00640 1 -- 172.21.15.86:0/1061745284 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd448062430 con 0x7fd42c0a3c30 2024-04-17T07:12:31.282 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_0/20a7ddba-5b0c-401a-839f-309c9dcc27f5 2024-04-17T07:12:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 -- 172.21.15.86:0/1061745284 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd42c0a3c30 msgr2=0x7fd42c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd42c0a3c30 0x7fd42c0a60f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd4481094f0 tx=0x7fd43800b390 comp rx=0 tx=0).stop 2024-04-17T07:12:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 -- 172.21.15.86:0/1061745284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44810a050 msgr2=0x7fd4481a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44810a050 0x7fd4481a5bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd440032520 tx=0x7fd440034880 comp rx=0 tx=0).stop 2024-04-17T07:12:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 -- 172.21.15.86:0/1061745284 shutdown_connections 2024-04-17T07:12:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd42c0a3c30 0x7fd42c0a60f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd44810a050 0x7fd4481a5bf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd448109690 0x7fd44819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 --2- 172.21.15.86:0/1061745284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4481084a0 0x7fd44819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 -- 172.21.15.86:0/1061745284 >> 172.21.15.86:0/1061745284 conn(0x7fd448077430 msgr2=0x7fd44810c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 -- 172.21.15.86:0/1061745284 shutdown_connections 2024-04-17T07:12:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.285+0000 7fd447200640 1 -- 172.21.15.86:0/1061745284 wait complete. 2024-04-17T07:12:31.306 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-17T07:12:31.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:31 smithi098 ceph-mon[35418]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:31.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:31 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] up:active 2024-04-17T07:12:31.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:31 smithi098 ceph-mon[35418]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-17T07:12:31.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:31 smithi098 ceph-mon[35418]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-17T07:12:31.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:31 smithi098 ceph-mon[35418]: Cluster is now healthy 2024-04-17T07:12:31.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:31 smithi098 ceph-mon[35418]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-17T07:12:31.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:31 smithi098 ceph-mon[35418]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-17T07:12:31.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:31 smithi098 ceph-mon[35418]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-17T07:12:31.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.518+0000 7f3145800640 1 -- 172.21.15.86:0/4007862583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3140105c50 msgr2=0x7f314010c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:31.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.518+0000 7f3145800640 1 --2- 172.21.15.86:0/4007862583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3140105c50 0x7f314010c4e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3138009900 tx=0x7f313802bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:31.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.518+0000 7f3145800640 1 -- 172.21.15.86:0/4007862583 shutdown_connections 2024-04-17T07:12:31.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.518+0000 7f3145800640 1 --2- 172.21.15.86:0/4007862583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3140105c50 0x7f314010c4e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.518+0000 7f3145800640 1 --2- 172.21.15.86:0/4007862583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3140105290 0x7f3140105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.518+0000 7f3145800640 1 --2- 172.21.15.86:0/4007862583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31401040a0 0x7f31401044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.518+0000 7f3145800640 1 -- 172.21.15.86:0/4007862583 >> 172.21.15.86:0/4007862583 conn(0x7f31400ff610 msgr2=0x7f3140101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:31.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.518+0000 7f3145800640 1 -- 172.21.15.86:0/4007862583 shutdown_connections 2024-04-17T07:12:31.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.518+0000 7f3145800640 1 -- 172.21.15.86:0/4007862583 wait complete. 2024-04-17T07:12:31.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.519+0000 7f3145800640 1 Processor -- start 2024-04-17T07:12:31.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.519+0000 7f3145800640 1 -- start start 2024-04-17T07:12:31.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.520+0000 7f3145800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31401040a0 0x7f3140079d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:31.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.520+0000 7f3145800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3140105290 0x7f314007a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:31.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.520+0000 7f3145800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140105c50 0x7f3140076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.520+0000 7f3145800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f314010fc30 con 0x7f3140105290 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.520+0000 7f3145800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f314010fab0 con 0x7f31401040a0 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.520+0000 7f3145800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f314010fdb0 con 0x7f3140105c50 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.520+0000 7f313f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3140105290 0x7f314007a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.520+0000 7f313f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3140105290 0x7f314007a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54144/0 (socket says 172.21.15.86:54144) 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.520+0000 7f313f400640 1 -- 172.21.15.86:0/1564059023 learned_addr learned my addr 172.21.15.86:0/1564059023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313fe00640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31401040a0 0x7f3140079d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313f400640 1 -- 172.21.15.86:0/1564059023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140105c50 msgr2=0x7f3140076880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313f400640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140105c50 0x7f3140076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313f400640 1 -- 172.21.15.86:0/1564059023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31401040a0 msgr2=0x7f3140079d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313f400640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31401040a0 0x7f3140079d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313f400640 1 -- 172.21.15.86:0/1564059023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3140077140 con 0x7f3140105290 2024-04-17T07:12:31.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313fe00640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31401040a0 0x7f3140079d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:31.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313f400640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3140105290 0x7f314007a250 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f312800e9f0 tx=0x7f312800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:31.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313cc00640 1 -- 172.21.15.86:0/1564059023 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312800cd80 con 0x7f3140105290 2024-04-17T07:12:31.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313cc00640 1 -- 172.21.15.86:0/1564059023 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31280108e0 con 0x7f3140105290 2024-04-17T07:12:31.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f313cc00640 1 -- 172.21.15.86:0/1564059023 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128018920 con 0x7f3140105290 2024-04-17T07:12:31.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.521+0000 7f3145800640 1 -- 172.21.15.86:0/1564059023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31401a6690 con 0x7f3140105290 2024-04-17T07:12:31.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.522+0000 7f3145800640 1 -- 172.21.15.86:0/1564059023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31401a6af0 con 0x7f3140105290 2024-04-17T07:12:31.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.523+0000 7f3145800640 1 -- 172.21.15.86:0/1564059023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314010b3c0 con 0x7f3140105290 2024-04-17T07:12:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.526+0000 7f313cc00640 1 -- 172.21.15.86:0/1564059023 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3128010a90 con 0x7f3140105290 2024-04-17T07:12:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:31 smithi103 ceph-mon[35495]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:31 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] up:active 2024-04-17T07:12:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:31 smithi103 ceph-mon[35495]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-17T07:12:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:31 smithi103 ceph-mon[35495]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-17T07:12:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:31 smithi103 ceph-mon[35495]: Cluster is now healthy 2024-04-17T07:12:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:31 smithi103 ceph-mon[35495]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-17T07:12:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:31 smithi103 ceph-mon[35495]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-17T07:12:31.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:31 smithi103 ceph-mon[35495]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-17T07:12:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.527+0000 7f313cc00640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31240a3c30 0x7f31240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.528+0000 7f313cc00640 1 -- 172.21.15.86:0/1564059023 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f31280d0b20 con 0x7f3140105290 2024-04-17T07:12:31.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.528+0000 7f313fe00640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31240a3c30 0x7f31240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:31.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.528+0000 7f313fe00640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31240a3c30 0x7f31240a60f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31401050f0 tx=0x7f3130009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:31.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.532+0000 7f313cc00640 1 -- 172.21.15.86:0/1564059023 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f312809d140 con 0x7f3140105290 2024-04-17T07:12:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:31 smithi086 ceph-mon[31733]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:31 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] up:active 2024-04-17T07:12:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:31 smithi086 ceph-mon[31733]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-17T07:12:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:31 smithi086 ceph-mon[31733]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-17T07:12:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:31 smithi086 ceph-mon[31733]: Cluster is now healthy 2024-04-17T07:12:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:31 smithi086 ceph-mon[31733]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-17T07:12:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:31 smithi086 ceph-mon[31733]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-17T07:12:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:31 smithi086 ceph-mon[31733]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-17T07:12:31.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.766+0000 7f3145800640 1 -- 172.21.15.86:0/1564059023 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3140062430 con 0x7f31240a3c30 2024-04-17T07:12:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.830+0000 7f313cc00640 1 -- 172.21.15.86:0/1564059023 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3140062430 con 0x7f31240a3c30 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.833+0000 7f3145800640 1 -- 172.21.15.86:0/1564059023 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31240a3c30 msgr2=0x7f31240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.833+0000 7f3145800640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31240a3c30 0x7f31240a60f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31401050f0 tx=0x7f3130009530 comp rx=0 tx=0).stop 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.834+0000 7f3145800640 1 -- 172.21.15.86:0/1564059023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3140105290 msgr2=0x7f314007a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.834+0000 7f3145800640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3140105290 0x7f314007a250 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f312800e9f0 tx=0x7f312800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.834+0000 7f3145800640 1 -- 172.21.15.86:0/1564059023 shutdown_connections 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.834+0000 7f3145800640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31240a3c30 0x7f31240a60f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.834+0000 7f3145800640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140105c50 0x7f3140076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.834+0000 7f3145800640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3140105290 0x7f314007a250 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.834+0000 7f3145800640 1 --2- 172.21.15.86:0/1564059023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31401040a0 0x7f3140079d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.834+0000 7f3145800640 1 -- 172.21.15.86:0/1564059023 >> 172.21.15.86:0/1564059023 conn(0x7f31400ff610 msgr2=0x7f3140108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.834+0000 7f3145800640 1 -- 172.21.15.86:0/1564059023 shutdown_connections 2024-04-17T07:12:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:31.834+0000 7f3145800640 1 -- 172.21.15.86:0/1564059023 wait complete. 2024-04-17T07:12:31.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-17T07:12:32.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.078+0000 7f8df9600640 1 -- 172.21.15.86:0/3880057359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df4105350 msgr2=0x7f8df4107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:32.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.078+0000 7f8df9600640 1 --2- 172.21.15.86:0/3880057359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df4105350 0x7f8df4107740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009900 tx=0x7f8de002bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:32.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.078+0000 7f8df9600640 1 -- 172.21.15.86:0/3880057359 shutdown_connections 2024-04-17T07:12:32.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.078+0000 7f8df9600640 1 --2- 172.21.15.86:0/3880057359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4107c80 0x7f8df410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.078+0000 7f8df9600640 1 --2- 172.21.15.86:0/3880057359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df4105350 0x7f8df4107740 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.078+0000 7f8df9600640 1 --2- 172.21.15.86:0/3880057359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8df4102a20 0x7f8df4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.078+0000 7f8df9600640 1 -- 172.21.15.86:0/3880057359 >> 172.21.15.86:0/3880057359 conn(0x7f8df40fc410 msgr2=0x7f8df40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:32.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.078+0000 7f8df9600640 1 -- 172.21.15.86:0/3880057359 shutdown_connections 2024-04-17T07:12:32.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.078+0000 7f8df9600640 1 -- 172.21.15.86:0/3880057359 wait complete. 2024-04-17T07:12:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.079+0000 7f8df9600640 1 Processor -- start 2024-04-17T07:12:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.079+0000 7f8df9600640 1 -- start start 2024-04-17T07:12:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.079+0000 7f8df9600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4102a20 0x7f8df419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.079+0000 7f8df9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df4105350 0x7f8df419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.079+0000 7f8df9600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8df4107c80 0x7f8df41a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8df410daf0 con 0x7f8df4105350 2024-04-17T07:12:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df9600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8df410d970 con 0x7f8df4107c80 2024-04-17T07:12:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df9600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8df410dc70 con 0x7f8df4102a20 2024-04-17T07:12:32.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df4105350 0x7f8df419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:32.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df4105350 0x7f8df419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54152/0 (socket says 172.21.15.86:54152) 2024-04-17T07:12:32.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df2000640 1 -- 172.21.15.86:0/3325002356 learned_addr learned my addr 172.21.15.86:0/3325002356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:32.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df3400640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8df4107c80 0x7f8df41a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:32.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df2a00640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4102a20 0x7f8df419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:32.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df2000640 1 -- 172.21.15.86:0/3325002356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4102a20 msgr2=0x7f8df419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:32.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df2000640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4102a20 0x7f8df419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df2000640 1 -- 172.21.15.86:0/3325002356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8df4107c80 msgr2=0x7f8df41a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:32.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df2000640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8df4107c80 0x7f8df41a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df2000640 1 -- 172.21.15.86:0/3325002356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df41a2080 con 0x7f8df4105350 2024-04-17T07:12:32.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.080+0000 7f8df2a00640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4102a20 0x7f8df419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:32.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.081+0000 7f8df2000640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df4105350 0x7f8df419a9a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8de0000c00 tx=0x7f8de00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:32.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.081+0000 7f8deb400640 1 -- 172.21.15.86:0/3325002356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0043020 con 0x7f8df4105350 2024-04-17T07:12:32.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.081+0000 7f8deb400640 1 -- 172.21.15.86:0/3325002356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8de00044c0 con 0x7f8df4105350 2024-04-17T07:12:32.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.081+0000 7f8df9600640 1 -- 172.21.15.86:0/3325002356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df41a2310 con 0x7f8df4105350 2024-04-17T07:12:32.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.081+0000 7f8deb400640 1 -- 172.21.15.86:0/3325002356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0003c60 con 0x7f8df4105350 2024-04-17T07:12:32.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.081+0000 7f8df9600640 1 -- 172.21.15.86:0/3325002356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df41a2860 con 0x7f8df4105350 2024-04-17T07:12:32.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.082+0000 7f8df9600640 1 -- 172.21.15.86:0/3325002356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df41097d0 con 0x7f8df4105350 2024-04-17T07:12:32.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.084+0000 7f8deb400640 1 -- 172.21.15.86:0/3325002356 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8de0046020 con 0x7f8df4105350 2024-04-17T07:12:32.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.085+0000 7f8deb400640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dd80a3bb0 0x7f8dd80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:32.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.085+0000 7f8deb400640 1 -- 172.21.15.86:0/3325002356 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8de00f2a50 con 0x7f8df4105350 2024-04-17T07:12:32.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.086+0000 7f8df2a00640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dd80a3bb0 0x7f8dd80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:32.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.086+0000 7f8df2a00640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dd80a3bb0 0x7f8dd80a6070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009b50 tx=0x7f8de4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:32.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.091+0000 7f8deb400640 1 -- 172.21.15.86:0/3325002356 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8de004d020 con 0x7f8df4105350 2024-04-17T07:12:32.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.239+0000 7f8deb400640 1 -- 172.21.15.86:0/3325002356 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8de00bb160 con 0x7f8df4105350 2024-04-17T07:12:32.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.342+0000 7f8df9600640 1 -- 172.21.15.86:0/3325002356 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8df4062430 con 0x7f8dd80a3bb0 2024-04-17T07:12:32.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.355+0000 7f8deb400640 1 -- 172.21.15.86:0/3325002356 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8df4062430 con 0x7f8dd80a3bb0 2024-04-17T07:12:32.355 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_1/04f00876-29f9-4fff-82a8-cd6ad13e5f78 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.357+0000 7f8df9600640 1 -- 172.21.15.86:0/3325002356 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dd80a3bb0 msgr2=0x7f8dd80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.357+0000 7f8df9600640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dd80a3bb0 0x7f8dd80a6070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009b50 tx=0x7f8de4005f90 comp rx=0 tx=0).stop 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.357+0000 7f8df9600640 1 -- 172.21.15.86:0/3325002356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df4105350 msgr2=0x7f8df419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.357+0000 7f8df9600640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df4105350 0x7f8df419a9a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8de0000c00 tx=0x7f8de00040c0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.358+0000 7f8df9600640 1 -- 172.21.15.86:0/3325002356 shutdown_connections 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.358+0000 7f8df9600640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dd80a3bb0 0x7f8dd80a6070 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.358+0000 7f8df9600640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8df4107c80 0x7f8df41a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.358+0000 7f8df9600640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df4105350 0x7f8df419a9a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.358+0000 7f8df9600640 1 --2- 172.21.15.86:0/3325002356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4102a20 0x7f8df419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.358+0000 7f8df9600640 1 -- 172.21.15.86:0/3325002356 >> 172.21.15.86:0/3325002356 conn(0x7f8df40fc410 msgr2=0x7f8df41022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.358+0000 7f8df9600640 1 -- 172.21.15.86:0/3325002356 shutdown_connections 2024-04-17T07:12:32.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.358+0000 7f8df9600640 1 -- 172.21.15.86:0/3325002356 wait complete. 2024-04-17T07:12:32.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:12:32.455 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:32 smithi086 ceph-mon[31733]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:32.455 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:32 smithi086 ceph-mon[31733]: pgmap v383: 129 pgs: 1 activating, 3 peering, 125 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-17T07:12:32.455 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:32 smithi086 ceph-mon[31733]: mds.? [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] up:active 2024-04-17T07:12:32.455 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:32 smithi086 ceph-mon[31733]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-17T07:12:32.455 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:32 smithi086 ceph-mon[31733]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-17T07:12:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:32 smithi098 ceph-mon[35418]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:32 smithi098 ceph-mon[35418]: pgmap v383: 129 pgs: 1 activating, 3 peering, 125 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-17T07:12:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:32 smithi098 ceph-mon[35418]: mds.? [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] up:active 2024-04-17T07:12:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:32 smithi098 ceph-mon[35418]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-17T07:12:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:32 smithi098 ceph-mon[35418]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-17T07:12:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:32 smithi103 ceph-mon[35495]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:32 smithi103 ceph-mon[35495]: pgmap v383: 129 pgs: 1 activating, 3 peering, 125 active+clean; 583 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-17T07:12:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:32 smithi103 ceph-mon[35495]: mds.? [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] up:active 2024-04-17T07:12:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:32 smithi103 ceph-mon[35495]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-17T07:12:32.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:32 smithi103 ceph-mon[35495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-17T07:12:32.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.585+0000 7f8edc400640 1 -- 172.21.15.86:0/257146521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed4105d70 msgr2=0x7f8ed410c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.585+0000 7f8edc400640 1 --2- 172.21.15.86:0/257146521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed4105d70 0x7f8ed410c600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009ca0 tx=0x7f8ecc02c100 comp rx=0 tx=0).stop 2024-04-17T07:12:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.585+0000 7f8edc400640 1 -- 172.21.15.86:0/257146521 shutdown_connections 2024-04-17T07:12:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.585+0000 7f8edc400640 1 --2- 172.21.15.86:0/257146521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed4105d70 0x7f8ed410c600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.585+0000 7f8edc400640 1 --2- 172.21.15.86:0/257146521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ed41053b0 0x7f8ed4105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.585+0000 7f8edc400640 1 --2- 172.21.15.86:0/257146521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed41041c0 0x7f8ed41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.585+0000 7f8edc400640 1 -- 172.21.15.86:0/257146521 >> 172.21.15.86:0/257146521 conn(0x7f8ed40ff750 msgr2=0x7f8ed4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.585+0000 7f8edc400640 1 -- 172.21.15.86:0/257146521 shutdown_connections 2024-04-17T07:12:32.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.585+0000 7f8edc400640 1 -- 172.21.15.86:0/257146521 wait complete. 2024-04-17T07:12:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.586+0000 7f8edc400640 1 Processor -- start 2024-04-17T07:12:32.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.587+0000 7f8edc400640 1 -- start start 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.587+0000 7f8edc400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ed41041c0 0x7f8ed419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.587+0000 7f8edc400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed41053b0 0x7f8ed419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.587+0000 7f8edc400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4105d70 0x7f8ed41a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.587+0000 7f8edc400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410fcd0 con 0x7f8ed41053b0 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.587+0000 7f8edc400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410fb50 con 0x7f8ed41041c0 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.587+0000 7f8edc400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410fe50 con 0x7f8ed4105d70 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.587+0000 7f8ed9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed41053b0 0x7f8ed419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.587+0000 7f8ed9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed41053b0 0x7f8ed419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54166/0 (socket says 172.21.15.86:54166) 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.587+0000 7f8ed9000640 1 -- 172.21.15.86:0/2209047048 learned_addr learned my addr 172.21.15.86:0/2209047048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8eda400640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4105d70 0x7f8ed41a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8ed9a00640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ed41041c0 0x7f8ed419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8ed9000640 1 -- 172.21.15.86:0/2209047048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4105d70 msgr2=0x7f8ed41a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:32.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8ed9000640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4105d70 0x7f8ed41a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8ed9000640 1 -- 172.21.15.86:0/2209047048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ed41041c0 msgr2=0x7f8ed419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8ed9000640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ed41041c0 0x7f8ed419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8ed9000640 1 -- 172.21.15.86:0/2209047048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed41a1e70 con 0x7f8ed41053b0 2024-04-17T07:12:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8eda400640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4105d70 0x7f8ed41a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8ed9000640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed41053b0 0x7f8ed419a790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000c920 tx=0x7f8ec000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8ed2000640 1 -- 172.21.15.86:0/2209047048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0007c70 con 0x7f8ed41053b0 2024-04-17T07:12:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8ed2000640 1 -- 172.21.15.86:0/2209047048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ec0007e10 con 0x7f8ed41053b0 2024-04-17T07:12:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8edc400640 1 -- 172.21.15.86:0/2209047048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed41a2160 con 0x7f8ed41053b0 2024-04-17T07:12:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8ed2000640 1 -- 172.21.15.86:0/2209047048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0018820 con 0x7f8ed41053b0 2024-04-17T07:12:32.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.588+0000 7f8edc400640 1 -- 172.21.15.86:0/2209047048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41a26b0 con 0x7f8ed41053b0 2024-04-17T07:12:32.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.590+0000 7f8edc400640 1 -- 172.21.15.86:0/2209047048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed410b410 con 0x7f8ed41053b0 2024-04-17T07:12:32.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.591+0000 7f8ed2000640 1 -- 172.21.15.86:0/2209047048 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8ec00189c0 con 0x7f8ed41053b0 2024-04-17T07:12:32.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.592+0000 7f8ed2000640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ebc0a3c40 0x7f8ebc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:32.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.593+0000 7f8ed2000640 1 -- 172.21.15.86:0/2209047048 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8ec00d0ad0 con 0x7f8ed41053b0 2024-04-17T07:12:32.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.593+0000 7f8ed9a00640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ebc0a3c40 0x7f8ebc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:32.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.593+0000 7f8ed9a00640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ebc0a3c40 0x7f8ebc0a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8005fd0 tx=0x7f8ec8005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:32.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.597+0000 7f8ed2000640 1 -- 172.21.15.86:0/2209047048 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8ec0014030 con 0x7f8ed41053b0 2024-04-17T07:12:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.840+0000 7f8edc400640 1 -- 172.21.15.86:0/2209047048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ed402d830 con 0x7f8ed41053b0 2024-04-17T07:12:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.842+0000 7f8ed2000640 1 -- 172.21.15.86:0/2209047048 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f8ec009cff0 con 0x7f8ed41053b0 2024-04-17T07:12:32.842 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:32.842 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:12:31.203694+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[34460,34490,34430,34400,24505]},"id":1}]} 2024-04-17T07:12:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-04-17T07:12:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.844+0000 7f8edc400640 1 -- 172.21.15.86:0/2209047048 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ebc0a3c40 msgr2=0x7f8ebc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.844+0000 7f8edc400640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ebc0a3c40 0x7f8ebc0a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8005fd0 tx=0x7f8ec8005ce0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.844+0000 7f8edc400640 1 -- 172.21.15.86:0/2209047048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed41053b0 msgr2=0x7f8ed419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.844+0000 7f8edc400640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed41053b0 0x7f8ed419a790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000c920 tx=0x7f8ec000cde0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.845+0000 7f8edc400640 1 -- 172.21.15.86:0/2209047048 shutdown_connections 2024-04-17T07:12:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.845+0000 7f8edc400640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ebc0a3c40 0x7f8ebc0a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.845+0000 7f8edc400640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4105d70 0x7f8ed41a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.845+0000 7f8edc400640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed41053b0 0x7f8ed419a790 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.845+0000 7f8edc400640 1 --2- 172.21.15.86:0/2209047048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ed41041c0 0x7f8ed419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.845+0000 7f8edc400640 1 -- 172.21.15.86:0/2209047048 >> 172.21.15.86:0/2209047048 conn(0x7f8ed40ff750 msgr2=0x7f8ed41085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.845+0000 7f8edc400640 1 -- 172.21.15.86:0/2209047048 shutdown_connections 2024-04-17T07:12:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:32.845+0000 7f8edc400640 1 -- 172.21.15.86:0/2209047048 wait complete. 2024-04-17T07:12:32.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:12:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.073+0000 7f70a4a00640 1 -- 172.21.15.86:0/1135030400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a0102ff0 msgr2=0x7f70a0103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.073+0000 7f70a4a00640 1 --2- 172.21.15.86:0/1135030400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a0102ff0 0x7f70a0103470 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7088009900 tx=0x7f708802bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.073+0000 7f70a4a00640 1 -- 172.21.15.86:0/1135030400 shutdown_connections 2024-04-17T07:12:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.073+0000 7f70a4a00640 1 --2- 172.21.15.86:0/1135030400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70a01039b0 0x7f70a010a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.073+0000 7f70a4a00640 1 --2- 172.21.15.86:0/1135030400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a0102ff0 0x7f70a0103470 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.073+0000 7f70a4a00640 1 --2- 172.21.15.86:0/1135030400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a0101e00 0x7f70a0102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.074+0000 7f70a4a00640 1 -- 172.21.15.86:0/1135030400 >> 172.21.15.86:0/1135030400 conn(0x7f70a00fd370 msgr2=0x7f70a00ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.074+0000 7f70a4a00640 1 -- 172.21.15.86:0/1135030400 shutdown_connections 2024-04-17T07:12:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.074+0000 7f70a4a00640 1 -- 172.21.15.86:0/1135030400 wait complete. 2024-04-17T07:12:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f70a4a00640 1 Processor -- start 2024-04-17T07:12:33.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f70a4a00640 1 -- start start 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f70a4a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70a0101e00 0x7f70a019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f70a4a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a0102ff0 0x7f70a019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f70a4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a01039b0 0x7f70a01a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f70a4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70a010d980 con 0x7f70a01039b0 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f70a4a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70a010d800 con 0x7f70a0101e00 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f70a4a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70a010db00 con 0x7f70a0102ff0 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f709f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a01039b0 0x7f70a01a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f709f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a01039b0 0x7f70a01a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54176/0 (socket says 172.21.15.86:54176) 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.075+0000 7f709f200640 1 -- 172.21.15.86:0/1201571890 learned_addr learned my addr 172.21.15.86:0/1201571890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f709e800640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70a0101e00 0x7f70a019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f709de00640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a0102ff0 0x7f70a019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f709f200640 1 -- 172.21.15.86:0/1201571890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a0102ff0 msgr2=0x7f70a019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:33.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f709f200640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a0102ff0 0x7f70a019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f709f200640 1 -- 172.21.15.86:0/1201571890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70a0101e00 msgr2=0x7f70a019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f709f200640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70a0101e00 0x7f70a019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f709f200640 1 -- 172.21.15.86:0/1201571890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a01a1e90 con 0x7f70a01039b0 2024-04-17T07:12:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f709f200640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a01039b0 0x7f70a01a1820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f709800d950 tx=0x7f709800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f7097400640 1 -- 172.21.15.86:0/1201571890 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098017030 con 0x7f70a01039b0 2024-04-17T07:12:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f70a4a00640 1 -- 172.21.15.86:0/1201571890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a01a2180 con 0x7f70a01039b0 2024-04-17T07:12:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f7097400640 1 -- 172.21.15.86:0/1201571890 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7098004020 con 0x7f70a01039b0 2024-04-17T07:12:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f7097400640 1 -- 172.21.15.86:0/1201571890 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098003b20 con 0x7f70a01039b0 2024-04-17T07:12:33.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.076+0000 7f70a4a00640 1 -- 172.21.15.86:0/1201571890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a01a2650 con 0x7f70a01039b0 2024-04-17T07:12:33.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.078+0000 7f70a4a00640 1 -- 172.21.15.86:0/1201571890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a01a2910 con 0x7f70a01039b0 2024-04-17T07:12:33.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.080+0000 7f7097400640 1 -- 172.21.15.86:0/1201571890 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f709801a070 con 0x7f70a01039b0 2024-04-17T07:12:33.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.081+0000 7f7097400640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70840a3c40 0x7f70840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.081+0000 7f7097400640 1 -- 172.21.15.86:0/1201571890 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f70980d01c0 con 0x7f70a01039b0 2024-04-17T07:12:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.081+0000 7f709e800640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70840a3c40 0x7f70840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:33.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.082+0000 7f709e800640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70840a3c40 0x7f70840a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f70a0102e50 tx=0x7f7090009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:33.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.085+0000 7f7097400640 1 -- 172.21.15.86:0/1201571890 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7098021070 con 0x7f70a01039b0 2024-04-17T07:12:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.291+0000 7f70a4a00640 1 -- 172.21.15.86:0/1201571890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70a01a2b80 con 0x7f70a01039b0 2024-04-17T07:12:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.293+0000 7f7097400640 1 -- 172.21.15.86:0/1201571890 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24585 (secure 0 0 0) 0x7f709809c6e0 con 0x7f70a01039b0 2024-04-17T07:12:33.294 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:33.294 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":91,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:12:28.705120+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:12:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 -- 172.21.15.86:0/1201571890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70840a3c40 msgr2=0x7f70840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70840a3c40 0x7f70840a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f70a0102e50 tx=0x7f7090009870 comp rx=0 tx=0).stop 2024-04-17T07:12:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 -- 172.21.15.86:0/1201571890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a01039b0 msgr2=0x7f70a01a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a01039b0 0x7f70a01a1820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f709800d950 tx=0x7f709800de10 comp rx=0 tx=0).stop 2024-04-17T07:12:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 -- 172.21.15.86:0/1201571890 shutdown_connections 2024-04-17T07:12:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70840a3c40 0x7f70840a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70a01039b0 0x7f70a01a1820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a0102ff0 0x7f70a019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 --2- 172.21.15.86:0/1201571890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70a0101e00 0x7f70a019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 -- 172.21.15.86:0/1201571890 >> 172.21.15.86:0/1201571890 conn(0x7f70a00fd370 msgr2=0x7f70a0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 -- 172.21.15.86:0/1201571890 shutdown_connections 2024-04-17T07:12:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.296+0000 7f70a4a00640 1 -- 172.21.15.86:0/1201571890 wait complete. 2024-04-17T07:12:33.316 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T07:12:33.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:12:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:33 smithi098 ceph-mon[35418]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:33 smithi098 ceph-mon[35418]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:33 smithi098 ceph-mon[35418]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-17T07:12:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:33 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2209047048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:12:33.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.525+0000 7f5e66200640 1 -- 172.21.15.86:0/2797640228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e600749c0 msgr2=0x7f5e60074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:33.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.525+0000 7f5e66200640 1 --2- 172.21.15.86:0/2797640228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e600749c0 0x7f5e60074e40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009900 tx=0x7f5e4802bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:33.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.525+0000 7f5e66200640 1 -- 172.21.15.86:0/2797640228 shutdown_connections 2024-04-17T07:12:33.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.525+0000 7f5e66200640 1 --2- 172.21.15.86:0/2797640228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e60109cc0 0x7f5e6010c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.525+0000 7f5e66200640 1 --2- 172.21.15.86:0/2797640228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e600749c0 0x7f5e60074e40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.525+0000 7f5e66200640 1 --2- 172.21.15.86:0/2797640228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e60075fa0 0x7f5e600743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.525+0000 7f5e66200640 1 -- 172.21.15.86:0/2797640228 >> 172.21.15.86:0/2797640228 conn(0x7f5e600fd370 msgr2=0x7f5e600ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:33.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.525+0000 7f5e66200640 1 -- 172.21.15.86:0/2797640228 shutdown_connections 2024-04-17T07:12:33.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.525+0000 7f5e66200640 1 -- 172.21.15.86:0/2797640228 wait complete. 2024-04-17T07:12:33.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.526+0000 7f5e66200640 1 Processor -- start 2024-04-17T07:12:33.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.526+0000 7f5e66200640 1 -- start start 2024-04-17T07:12:33.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.526+0000 7f5e66200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e600749c0 0x7f5e6019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:33.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.526+0000 7f5e66200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e60075fa0 0x7f5e6019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:33.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.526+0000 7f5e66200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e60109cc0 0x7f5e601a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:33.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.526+0000 7f5e66200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6010fbc0 con 0x7f5e60109cc0 2024-04-17T07:12:33.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.526+0000 7f5e66200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6010fa40 con 0x7f5e600749c0 2024-04-17T07:12:33.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.526+0000 7f5e66200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6010fd40 con 0x7f5e60075fa0 2024-04-17T07:12:33.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e65800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e60109cc0 0x7f5e601a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:33.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e65800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e60109cc0 0x7f5e601a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54186/0 (socket says 172.21.15.86:54186) 2024-04-17T07:12:33.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e65800640 1 -- 172.21.15.86:0/1185556313 learned_addr learned my addr 172.21.15.86:0/1185556313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:33.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e64e00640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e600749c0 0x7f5e6019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:33.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e57e00640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e60075fa0 0x7f5e6019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:33.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e65800640 1 -- 172.21.15.86:0/1185556313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e60075fa0 msgr2=0x7f5e6019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:33.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e65800640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e60075fa0 0x7f5e6019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e65800640 1 -- 172.21.15.86:0/1185556313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e600749c0 msgr2=0x7f5e6019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:33.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e65800640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e600749c0 0x7f5e6019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e65800640 1 -- 172.21.15.86:0/1185556313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e601a1e10 con 0x7f5e60109cc0 2024-04-17T07:12:33.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:33 smithi103 ceph-mon[35495]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:33.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:33 smithi103 ceph-mon[35495]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:33.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:33 smithi103 ceph-mon[35495]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-17T07:12:33.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:33 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2209047048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:12:33.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.527+0000 7f5e65800640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e60109cc0 0x7f5e601a17a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009fd0 tx=0x7f5e5800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:33.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.528+0000 7f5e5e000640 1 -- 172.21.15.86:0/1185556313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58011830 con 0x7f5e60109cc0 2024-04-17T07:12:33.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.528+0000 7f5e5e000640 1 -- 172.21.15.86:0/1185556313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e58016070 con 0x7f5e60109cc0 2024-04-17T07:12:33.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.528+0000 7f5e66200640 1 -- 172.21.15.86:0/1185556313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e601a2100 con 0x7f5e60109cc0 2024-04-17T07:12:33.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.528+0000 7f5e5e000640 1 -- 172.21.15.86:0/1185556313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58010de0 con 0x7f5e60109cc0 2024-04-17T07:12:33.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.528+0000 7f5e66200640 1 -- 172.21.15.86:0/1185556313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e601a26d0 con 0x7f5e60109cc0 2024-04-17T07:12:33.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.529+0000 7f5e66200640 1 -- 172.21.15.86:0/1185556313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e601089f0 con 0x7f5e60109cc0 2024-04-17T07:12:33.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.531+0000 7f5e5e000640 1 -- 172.21.15.86:0/1185556313 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5e580119d0 con 0x7f5e60109cc0 2024-04-17T07:12:33.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.532+0000 7f5e5e000640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5e440a3cc0 0x7f5e440a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:33.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.532+0000 7f5e5e000640 1 -- 172.21.15.86:0/1185556313 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5e580cff00 con 0x7f5e60109cc0 2024-04-17T07:12:33.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.533+0000 7f5e64e00640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5e440a3cc0 0x7f5e440a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:33.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.533+0000 7f5e64e00640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5e440a3cc0 0x7f5e440a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5e50007350 tx=0x7f5e500072e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:33.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.537+0000 7f5e5e000640 1 -- 172.21.15.86:0/1185556313 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5e5809c530 con 0x7f5e60109cc0 2024-04-17T07:12:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:33 smithi086 ceph-mon[31733]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:33 smithi086 ceph-mon[31733]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:33 smithi086 ceph-mon[31733]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-17T07:12:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:33 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2209047048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:12:33.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.737+0000 7f5e66200640 1 -- 172.21.15.86:0/1185556313 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e600623a0 con 0x7f5e440a3cc0 2024-04-17T07:12:33.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.753+0000 7f5e5e000640 1 -- 172.21.15.86:0/1185556313 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340706 (secure 0 0 0) 0x7f5e600623a0 con 0x7f5e440a3cc0 2024-04-17T07:12:33.759 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:33.761 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-17T07:12:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.760+0000 7f5e66200640 1 -- 172.21.15.86:0/1185556313 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5e440a3cc0 msgr2=0x7f5e440a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.760+0000 7f5e66200640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5e440a3cc0 0x7f5e440a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5e50007350 tx=0x7f5e500072e0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.760+0000 7f5e66200640 1 -- 172.21.15.86:0/1185556313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e60109cc0 msgr2=0x7f5e601a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.760+0000 7f5e66200640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e60109cc0 0x7f5e601a17a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009fd0 tx=0x7f5e5800b8e0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.761+0000 7f5e66200640 1 -- 172.21.15.86:0/1185556313 shutdown_connections 2024-04-17T07:12:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.761+0000 7f5e66200640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5e440a3cc0 0x7f5e440a6180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.761+0000 7f5e66200640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e60109cc0 0x7f5e601a17a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.761+0000 7f5e66200640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e60075fa0 0x7f5e6019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.761+0000 7f5e66200640 1 --2- 172.21.15.86:0/1185556313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e600749c0 0x7f5e6019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.761+0000 7f5e66200640 1 -- 172.21.15.86:0/1185556313 >> 172.21.15.86:0/1185556313 conn(0x7f5e600fd370 msgr2=0x7f5e60105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.762+0000 7f5e66200640 1 -- 172.21.15.86:0/1185556313 shutdown_connections 2024-04-17T07:12:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:33.762+0000 7f5e66200640 1 -- 172.21.15.86:0/1185556313 wait complete. 2024-04-17T07:12:33.783 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-17T07:12:33.058088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639036,"num_objects":86,"num_object_clones":0,"num_object_copies":259,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":152,"num_read_kb":170,"num_write":378,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181358,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365404,"kb_used_data":12076,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576420,"statfs":{"total":1151940427776,"available":1151566254080,"internally_reserved":0,"allocated":12365824,"data_stored":7783091,"data_compressed":347595,"data_compressed_allocated":471040,"data_compressed_original":2441216,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44028,"num_objects":62,"num_object_clones":0,"num_object_copies":187,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":26,"num_read_kb":61,"num_write":148,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002844"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.738061+0000","last_change":"2024-04-17T07:11:55.486746+0000","last_active":"2024-04-17T07:12:27.738061+0000","last_peered":"2024-04-17T07:12:27.738061+0000","last_clean":"2024-04-17T07:12:27.738061+0000","last_became_active":"2024-04-17T07:11:55.486335+0000","last_became_peered":"2024-04-17T07:11:55.486335+0000","last_unstale":"2024-04-17T07:12:27.738061+0000","last_undegraded":"2024-04-17T07:12:27.738061+0000","last_fullsized":"2024-04-17T07:12:27.738061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:48.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735866+0000","last_change":"2024-04-17T07:11:55.481832+0000","last_active":"2024-04-17T07:12:28.735866+0000","last_peered":"2024-04-17T07:12:28.735866+0000","last_clean":"2024-04-17T07:12:28.735866+0000","last_became_active":"2024-04-17T07:11:55.476428+0000","last_became_peered":"2024-04-17T07:11:55.476428+0000","last_unstale":"2024-04-17T07:12:28.735866+0000","last_undegraded":"2024-04-17T07:12:28.735866+0000","last_fullsized":"2024-04-17T07:12:28.735866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:04.452542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736227+0000","last_change":"2024-04-17T07:11:55.483721+0000","last_active":"2024-04-17T07:12:28.736227+0000","last_peered":"2024-04-17T07:12:28.736227+0000","last_clean":"2024-04-17T07:12:28.736227+0000","last_became_active":"2024-04-17T07:11:55.483268+0000","last_became_peered":"2024-04-17T07:11:55.483268+0000","last_unstale":"2024-04-17T07:12:28.736227+0000","last_undegraded":"2024-04-17T07:12:28.736227+0000","last_fullsized":"2024-04-17T07:12:28.736227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:41.584783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714249+0000","last_change":"2024-04-17T07:11:55.485114+0000","last_active":"2024-04-17T07:12:28.714249+0000","last_peered":"2024-04-17T07:12:28.714249+0000","last_clean":"2024-04-17T07:12:28.714249+0000","last_became_active":"2024-04-17T07:11:55.483913+0000","last_became_peered":"2024-04-17T07:11:55.483913+0000","last_unstale":"2024-04-17T07:12:28.714249+0000","last_undegraded":"2024-04-17T07:12:28.714249+0000","last_fullsized":"2024-04-17T07:12:28.714249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:46.211121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726512+0000","last_change":"2024-04-17T07:11:55.491952+0000","last_active":"2024-04-17T07:12:28.726512+0000","last_peered":"2024-04-17T07:12:28.726512+0000","last_clean":"2024-04-17T07:12:28.726512+0000","last_became_active":"2024-04-17T07:11:55.491238+0000","last_became_peered":"2024-04-17T07:11:55.491238+0000","last_unstale":"2024-04-17T07:12:28.726512+0000","last_undegraded":"2024-04-17T07:12:28.726512+0000","last_fullsized":"2024-04-17T07:12:28.726512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:06.759797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725020+0000","last_change":"2024-04-17T07:11:55.471124+0000","last_active":"2024-04-17T07:12:28.725020+0000","last_peered":"2024-04-17T07:12:28.725020+0000","last_clean":"2024-04-17T07:12:28.725020+0000","last_became_active":"2024-04-17T07:11:55.470781+0000","last_became_peered":"2024-04-17T07:11:55.470781+0000","last_unstale":"2024-04-17T07:12:28.725020+0000","last_undegraded":"2024-04-17T07:12:28.725020+0000","last_fullsized":"2024-04-17T07:12:28.725020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:45.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720008+0000","last_change":"2024-04-17T07:11:55.469099+0000","last_active":"2024-04-17T07:12:28.720008+0000","last_peered":"2024-04-17T07:12:28.720008+0000","last_clean":"2024-04-17T07:12:28.720008+0000","last_became_active":"2024-04-17T07:11:55.468584+0000","last_became_peered":"2024-04-17T07:11:55.468584+0000","last_unstale":"2024-04-17T07:12:28.720008+0000","last_undegraded":"2024-04-17T07:12:28.720008+0000","last_fullsized":"2024-04-17T07:12:28.720008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:52.834846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122568+0000","last_change":"2024-04-17T07:11:55.473794+0000","last_active":"2024-04-17T07:12:29.122568+0000","last_peered":"2024-04-17T07:12:29.122568+0000","last_clean":"2024-04-17T07:12:29.122568+0000","last_became_active":"2024-04-17T07:11:55.471919+0000","last_became_peered":"2024-04-17T07:11:55.471919+0000","last_unstale":"2024-04-17T07:12:29.122568+0000","last_undegraded":"2024-04-17T07:12:29.122568+0000","last_fullsized":"2024-04-17T07:12:29.122568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:16.085117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726090+0000","last_change":"2024-04-17T07:11:55.482445+0000","last_active":"2024-04-17T07:12:28.726090+0000","last_peered":"2024-04-17T07:12:28.726090+0000","last_clean":"2024-04-17T07:12:28.726090+0000","last_became_active":"2024-04-17T07:11:55.481552+0000","last_became_peered":"2024-04-17T07:11:55.481552+0000","last_unstale":"2024-04-17T07:12:28.726090+0000","last_undegraded":"2024-04-17T07:12:28.726090+0000","last_fullsized":"2024-04-17T07:12:28.726090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:48.733585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725405+0000","last_change":"2024-04-17T07:11:55.476726+0000","last_active":"2024-04-17T07:12:28.725405+0000","last_peered":"2024-04-17T07:12:28.725405+0000","last_clean":"2024-04-17T07:12:28.725405+0000","last_became_active":"2024-04-17T07:11:55.476320+0000","last_became_peered":"2024-04-17T07:11:55.476320+0000","last_unstale":"2024-04-17T07:12:28.725405+0000","last_undegraded":"2024-04-17T07:12:28.725405+0000","last_fullsized":"2024-04-17T07:12:28.725405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.230181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721027+0000","last_change":"2024-04-17T07:11:55.485798+0000","last_active":"2024-04-17T07:12:28.721027+0000","last_peered":"2024-04-17T07:12:28.721027+0000","last_clean":"2024-04-17T07:12:28.721027+0000","last_became_active":"2024-04-17T07:11:55.484015+0000","last_became_peered":"2024-04-17T07:11:55.484015+0000","last_unstale":"2024-04-17T07:12:28.721027+0000","last_undegraded":"2024-04-17T07:12:28.721027+0000","last_fullsized":"2024-04-17T07:12:28.721027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:37.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.731078+0000","last_change":"2024-04-17T07:12:28.729834+0000","last_active":"2024-04-17T07:12:28.731078+0000","last_peered":"2024-04-17T07:12:28.731078+0000","last_clean":"2024-04-17T07:12:28.731078+0000","last_became_active":"2024-04-17T07:12:28.728694+0000","last_became_peered":"2024-04-17T07:12:28.728694+0000","last_unstale":"2024-04-17T07:12:28.731078+0000","last_undegraded":"2024-04-17T07:12:28.731078+0000","last_fullsized":"2024-04-17T07:12:28.731078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:46.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720070+0000","last_change":"2024-04-17T07:11:55.450379+0000","last_active":"2024-04-17T07:12:28.720070+0000","last_peered":"2024-04-17T07:12:28.720070+0000","last_clean":"2024-04-17T07:12:28.720070+0000","last_became_active":"2024-04-17T07:11:55.449118+0000","last_became_peered":"2024-04-17T07:11:55.449118+0000","last_unstale":"2024-04-17T07:12:28.720070+0000","last_undegraded":"2024-04-17T07:12:28.720070+0000","last_fullsized":"2024-04-17T07:12:28.720070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:07:34.248044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735022+0000","last_change":"2024-04-17T07:11:55.473839+0000","last_active":"2024-04-17T07:12:28.735022+0000","last_peered":"2024-04-17T07:12:28.735022+0000","last_clean":"2024-04-17T07:12:28.735022+0000","last_became_active":"2024-04-17T07:11:55.473472+0000","last_became_peered":"2024-04-17T07:11:55.473472+0000","last_unstale":"2024-04-17T07:12:28.735022+0000","last_undegraded":"2024-04-17T07:12:28.735022+0000","last_fullsized":"2024-04-17T07:12:28.735022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:02.263272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736022+0000","last_change":"2024-04-17T07:11:55.484820+0000","last_active":"2024-04-17T07:12:28.736022+0000","last_peered":"2024-04-17T07:12:28.736022+0000","last_clean":"2024-04-17T07:12:28.736022+0000","last_became_active":"2024-04-17T07:11:55.484314+0000","last_became_peered":"2024-04-17T07:11:55.484314+0000","last_unstale":"2024-04-17T07:12:28.736022+0000","last_undegraded":"2024-04-17T07:12:28.736022+0000","last_fullsized":"2024-04-17T07:12:28.736022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.186912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725577+0000","last_change":"2024-04-17T07:11:55.492018+0000","last_active":"2024-04-17T07:12:28.725577+0000","last_peered":"2024-04-17T07:12:28.725577+0000","last_clean":"2024-04-17T07:12:28.725577+0000","last_became_active":"2024-04-17T07:11:55.490512+0000","last_became_peered":"2024-04-17T07:11:55.490512+0000","last_unstale":"2024-04-17T07:12:28.725577+0000","last_undegraded":"2024-04-17T07:12:28.725577+0000","last_fullsized":"2024-04-17T07:12:28.725577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:45.436270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.737302+0000","last_change":"2024-04-17T07:11:55.487210+0000","last_active":"2024-04-17T07:12:27.737302+0000","last_peered":"2024-04-17T07:12:27.737302+0000","last_clean":"2024-04-17T07:12:27.737302+0000","last_became_active":"2024-04-17T07:11:55.485713+0000","last_became_peered":"2024-04-17T07:11:55.485713+0000","last_unstale":"2024-04-17T07:12:27.737302+0000","last_undegraded":"2024-04-17T07:12:27.737302+0000","last_fullsized":"2024-04-17T07:12:27.737302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:54.495950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716204+0000","last_change":"2024-04-17T07:11:55.482705+0000","last_active":"2024-04-17T07:12:28.716204+0000","last_peered":"2024-04-17T07:12:28.716204+0000","last_clean":"2024-04-17T07:12:28.716204+0000","last_became_active":"2024-04-17T07:11:55.482349+0000","last_became_peered":"2024-04-17T07:11:55.482349+0000","last_unstale":"2024-04-17T07:12:28.716204+0000","last_undegraded":"2024-04-17T07:12:28.716204+0000","last_fullsized":"2024-04-17T07:12:28.716204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:57.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716213+0000","last_change":"2024-04-17T07:11:55.467065+0000","last_active":"2024-04-17T07:12:28.716213+0000","last_peered":"2024-04-17T07:12:28.716213+0000","last_clean":"2024-04-17T07:12:28.716213+0000","last_became_active":"2024-04-17T07:11:55.466535+0000","last_became_peered":"2024-04-17T07:11:55.466535+0000","last_unstale":"2024-04-17T07:12:28.716213+0000","last_undegraded":"2024-04-17T07:12:28.716213+0000","last_fullsized":"2024-04-17T07:12:28.716213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:49.955453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715012+0000","last_change":"2024-04-17T07:11:59.515778+0000","last_active":"2024-04-17T07:12:28.715012+0000","last_peered":"2024-04-17T07:12:28.715012+0000","last_clean":"2024-04-17T07:12:28.715012+0000","last_became_active":"2024-04-17T07:11:59.515266+0000","last_became_peered":"2024-04-17T07:11:59.515266+0000","last_unstale":"2024-04-17T07:12:28.715012+0000","last_undegraded":"2024-04-17T07:12:28.715012+0000","last_fullsized":"2024-04-17T07:12:28.715012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:43.084724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.219364+0000","last_change":"2024-04-17T07:11:54.467060+0000","last_active":"2024-04-17T07:12:31.219364+0000","last_peered":"2024-04-17T07:12:31.219364+0000","last_clean":"2024-04-17T07:12:31.219364+0000","last_became_active":"2024-04-17T07:11:54.466687+0000","last_became_peered":"2024-04-17T07:11:54.466687+0000","last_unstale":"2024-04-17T07:12:31.219364+0000","last_undegraded":"2024-04-17T07:12:31.219364+0000","last_fullsized":"2024-04-17T07:12:31.219364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:57.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123283+0000","last_change":"2024-04-17T07:11:55.469857+0000","last_active":"2024-04-17T07:12:29.123283+0000","last_peered":"2024-04-17T07:12:29.123283+0000","last_clean":"2024-04-17T07:12:29.123283+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:12:29.123283+0000","last_undegraded":"2024-04-17T07:12:29.123283+0000","last_fullsized":"2024-04-17T07:12:29.123283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:43.565627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"activating","last_fresh":"2024-04-17T07:12:28.723099+0000","last_change":"2024-04-17T07:12:28.714326+0000","last_active":"2024-04-17T07:12:27.714584+0000","last_peered":"2024-04-17T07:12:00.800296+0000","last_clean":"2024-04-17T07:12:00.800296+0000","last_became_active":"2024-04-17T07:11:59.512404+0000","last_became_peered":"2024-04-17T07:11:59.512404+0000","last_unstale":"2024-04-17T07:12:28.723099+0000","last_undegraded":"2024-04-17T07:12:28.723099+0000","last_fullsized":"2024-04-17T07:12:28.723099+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":["4(0)","3(3)","6(1)","8(2)","9(2)"],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123277+0000","last_change":"2024-04-17T07:11:54.464715+0000","last_active":"2024-04-17T07:12:29.123277+0000","last_peered":"2024-04-17T07:12:29.123277+0000","last_clean":"2024-04-17T07:12:29.123277+0000","last_became_active":"2024-04-17T07:11:54.463917+0000","last_became_peered":"2024-04-17T07:11:54.463917+0000","last_unstale":"2024-04-17T07:12:29.123277+0000","last_undegraded":"2024-04-17T07:12:29.123277+0000","last_fullsized":"2024-04-17T07:12:29.123277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:43.738471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736571+0000","last_change":"2024-04-17T07:11:55.482876+0000","last_active":"2024-04-17T07:12:28.736571+0000","last_peered":"2024-04-17T07:12:28.736571+0000","last_clean":"2024-04-17T07:12:28.736571+0000","last_became_active":"2024-04-17T07:11:55.482451+0000","last_became_peered":"2024-04-17T07:11:55.482451+0000","last_unstale":"2024-04-17T07:12:28.736571+0000","last_undegraded":"2024-04-17T07:12:28.736571+0000","last_fullsized":"2024-04-17T07:12:28.736571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:39.394090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713748+0000","last_change":"2024-04-17T07:11:59.522634+0000","last_active":"2024-04-17T07:12:28.713748+0000","last_peered":"2024-04-17T07:12:28.713748+0000","last_clean":"2024-04-17T07:12:28.713748+0000","last_became_active":"2024-04-17T07:11:59.520797+0000","last_became_peered":"2024-04-17T07:11:59.520797+0000","last_unstale":"2024-04-17T07:12:28.713748+0000","last_undegraded":"2024-04-17T07:12:28.713748+0000","last_fullsized":"2024-04-17T07:12:28.713748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:02.316515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-17T07:11:55.467756+0000","last_change":"2024-04-17T07:12:28.153200+0000","last_active":"2024-04-17T07:12:28.153200+0000","last_peered":"2024-04-17T07:11:55.467756+0000","last_clean":"2024-04-17T07:11:55.467756+0000","last_became_active":"2024-04-17T07:11:54.469320+0000","last_became_peered":"2024-04-17T07:11:54.469320+0000","last_unstale":"2024-04-17T07:11:55.467756+0000","last_undegraded":"2024-04-17T07:11:55.467756+0000","last_fullsized":"2024-04-17T07:11:55.467756+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726296+0000","last_change":"2024-04-17T07:11:55.485999+0000","last_active":"2024-04-17T07:12:28.726296+0000","last_peered":"2024-04-17T07:12:28.726296+0000","last_clean":"2024-04-17T07:12:28.726296+0000","last_became_active":"2024-04-17T07:11:55.484048+0000","last_became_peered":"2024-04-17T07:11:55.484048+0000","last_unstale":"2024-04-17T07:12:28.726296+0000","last_undegraded":"2024-04-17T07:12:28.726296+0000","last_fullsized":"2024-04-17T07:12:28.726296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:38.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713330+0000","last_change":"2024-04-17T07:11:59.529929+0000","last_active":"2024-04-17T07:12:28.713330+0000","last_peered":"2024-04-17T07:12:28.713330+0000","last_clean":"2024-04-17T07:12:28.713330+0000","last_became_active":"2024-04-17T07:11:59.529400+0000","last_became_peered":"2024-04-17T07:11:59.529400+0000","last_unstale":"2024-04-17T07:12:28.713330+0000","last_undegraded":"2024-04-17T07:12:28.713330+0000","last_fullsized":"2024-04-17T07:12:28.713330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:54.845529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.209278+0000","last_change":"2024-04-17T07:11:54.462852+0000","last_active":"2024-04-17T07:12:30.209278+0000","last_peered":"2024-04-17T07:12:30.209278+0000","last_clean":"2024-04-17T07:12:30.209278+0000","last_became_active":"2024-04-17T07:11:54.459178+0000","last_became_peered":"2024-04-17T07:11:54.459178+0000","last_unstale":"2024-04-17T07:12:30.209278+0000","last_undegraded":"2024-04-17T07:12:30.209278+0000","last_fullsized":"2024-04-17T07:12:30.209278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:07.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.737683+0000","last_change":"2024-04-17T07:11:55.480951+0000","last_active":"2024-04-17T07:12:28.737683+0000","last_peered":"2024-04-17T07:12:28.737683+0000","last_clean":"2024-04-17T07:12:28.737683+0000","last_became_active":"2024-04-17T07:11:55.480507+0000","last_became_peered":"2024-04-17T07:11:55.480507+0000","last_unstale":"2024-04-17T07:12:28.737683+0000","last_undegraded":"2024-04-17T07:12:28.737683+0000","last_fullsized":"2024-04-17T07:12:28.737683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:42.584180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715898+0000","last_change":"2024-04-17T07:11:59.520897+0000","last_active":"2024-04-17T07:12:28.715898+0000","last_peered":"2024-04-17T07:12:28.715898+0000","last_clean":"2024-04-17T07:12:28.715898+0000","last_became_active":"2024-04-17T07:11:59.520350+0000","last_became_peered":"2024-04-17T07:11:59.520350+0000","last_unstale":"2024-04-17T07:12:28.715898+0000","last_undegraded":"2024-04-17T07:12:28.715898+0000","last_fullsized":"2024-04-17T07:12:28.715898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:39.813097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.222928+0000","last_change":"2024-04-17T07:12:28.753717+0000","last_active":"2024-04-17T07:12:31.222928+0000","last_peered":"2024-04-17T07:12:31.222928+0000","last_clean":"2024-04-17T07:12:31.222928+0000","last_became_active":"2024-04-17T07:12:28.739556+0000","last_became_peered":"2024-04-17T07:12:28.739556+0000","last_unstale":"2024-04-17T07:12:31.222928+0000","last_undegraded":"2024-04-17T07:12:31.222928+0000","last_fullsized":"2024-04-17T07:12:31.222928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:11.967731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.732946+0000","last_change":"2024-04-17T07:11:55.476675+0000","last_active":"2024-04-17T07:12:28.732946+0000","last_peered":"2024-04-17T07:12:28.732946+0000","last_clean":"2024-04-17T07:12:28.732946+0000","last_became_active":"2024-04-17T07:11:55.476273+0000","last_became_peered":"2024-04-17T07:11:55.476273+0000","last_unstale":"2024-04-17T07:12:28.732946+0000","last_undegraded":"2024-04-17T07:12:28.732946+0000","last_fullsized":"2024-04-17T07:12:28.732946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:09.612138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.741537+0000","last_change":"2024-04-17T07:12:28.741537+0000","last_active":"2024-04-17T07:12:28.741537+0000","last_peered":"2024-04-17T07:12:28.741537+0000","last_clean":"2024-04-17T07:12:28.741537+0000","last_became_active":"2024-04-17T07:12:28.741035+0000","last_became_peered":"2024-04-17T07:12:28.741035+0000","last_unstale":"2024-04-17T07:12:28.741537+0000","last_undegraded":"2024-04-17T07:12:28.741537+0000","last_fullsized":"2024-04-17T07:12:28.741537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:15.550112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.195614+0000","last_change":"2024-04-17T07:11:54.471118+0000","last_active":"2024-04-17T07:12:29.195614+0000","last_peered":"2024-04-17T07:12:29.195614+0000","last_clean":"2024-04-17T07:12:29.195614+0000","last_became_active":"2024-04-17T07:11:54.470526+0000","last_became_peered":"2024-04-17T07:11:54.470526+0000","last_unstale":"2024-04-17T07:12:29.195614+0000","last_undegraded":"2024-04-17T07:12:29.195614+0000","last_fullsized":"2024-04-17T07:12:29.195614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:52:47.918086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735651+0000","last_change":"2024-04-17T07:11:55.472850+0000","last_active":"2024-04-17T07:12:28.735651+0000","last_peered":"2024-04-17T07:12:28.735651+0000","last_clean":"2024-04-17T07:12:28.735651+0000","last_became_active":"2024-04-17T07:11:55.472097+0000","last_became_peered":"2024-04-17T07:11:55.472097+0000","last_unstale":"2024-04-17T07:12:28.735651+0000","last_undegraded":"2024-04-17T07:12:28.735651+0000","last_fullsized":"2024-04-17T07:12:28.735651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:41.730809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.737174+0000","last_change":"2024-04-17T07:11:59.522960+0000","last_active":"2024-04-17T07:12:27.737174+0000","last_peered":"2024-04-17T07:12:27.737174+0000","last_clean":"2024-04-17T07:12:27.737174+0000","last_became_active":"2024-04-17T07:11:59.522470+0000","last_became_peered":"2024-04-17T07:11:59.522470+0000","last_unstale":"2024-04-17T07:12:27.737174+0000","last_undegraded":"2024-04-17T07:12:27.737174+0000","last_fullsized":"2024-04-17T07:12:27.737174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:36.620442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720922+0000","last_change":"2024-04-17T07:11:54.470693+0000","last_active":"2024-04-17T07:12:28.720922+0000","last_peered":"2024-04-17T07:12:28.720922+0000","last_clean":"2024-04-17T07:12:28.720922+0000","last_became_active":"2024-04-17T07:11:54.470334+0000","last_became_peered":"2024-04-17T07:11:54.470334+0000","last_unstale":"2024-04-17T07:12:28.720922+0000","last_undegraded":"2024-04-17T07:12:28.720922+0000","last_fullsized":"2024-04-17T07:12:28.720922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:19:04.980786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122063+0000","last_change":"2024-04-17T07:11:55.474755+0000","last_active":"2024-04-17T07:12:29.122063+0000","last_peered":"2024-04-17T07:12:29.122063+0000","last_clean":"2024-04-17T07:12:29.122063+0000","last_became_active":"2024-04-17T07:11:55.474261+0000","last_became_peered":"2024-04-17T07:11:55.474261+0000","last_unstale":"2024-04-17T07:12:29.122063+0000","last_undegraded":"2024-04-17T07:12:29.122063+0000","last_fullsized":"2024-04-17T07:12:29.122063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:00.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735779+0000","last_change":"2024-04-17T07:11:59.517152+0000","last_active":"2024-04-17T07:12:28.735779+0000","last_peered":"2024-04-17T07:12:28.735779+0000","last_clean":"2024-04-17T07:12:28.735779+0000","last_became_active":"2024-04-17T07:11:59.514977+0000","last_became_peered":"2024-04-17T07:11:59.514977+0000","last_unstale":"2024-04-17T07:12:28.735779+0000","last_undegraded":"2024-04-17T07:12:28.735779+0000","last_fullsized":"2024-04-17T07:12:28.735779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:14.974136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.212620+0000","last_change":"2024-04-17T07:11:54.459786+0000","last_active":"2024-04-17T07:12:30.212620+0000","last_peered":"2024-04-17T07:12:30.212620+0000","last_clean":"2024-04-17T07:12:30.212620+0000","last_became_active":"2024-04-17T07:11:54.457589+0000","last_became_peered":"2024-04-17T07:11:54.457589+0000","last_unstale":"2024-04-17T07:12:30.212620+0000","last_undegraded":"2024-04-17T07:12:30.212620+0000","last_fullsized":"2024-04-17T07:12:30.212620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:25.986539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714421+0000","last_change":"2024-04-17T07:11:55.465020+0000","last_active":"2024-04-17T07:12:28.714421+0000","last_peered":"2024-04-17T07:12:28.714421+0000","last_clean":"2024-04-17T07:12:28.714421+0000","last_became_active":"2024-04-17T07:11:55.464608+0000","last_became_peered":"2024-04-17T07:11:55.464608+0000","last_unstale":"2024-04-17T07:12:28.714421+0000","last_undegraded":"2024-04-17T07:12:28.714421+0000","last_fullsized":"2024-04-17T07:12:28.714421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122725+0000","last_change":"2024-04-17T07:11:59.519333+0000","last_active":"2024-04-17T07:12:29.122725+0000","last_peered":"2024-04-17T07:12:29.122725+0000","last_clean":"2024-04-17T07:12:29.122725+0000","last_became_active":"2024-04-17T07:11:59.518865+0000","last_became_peered":"2024-04-17T07:11:59.518865+0000","last_unstale":"2024-04-17T07:12:29.122725+0000","last_undegraded":"2024-04-17T07:12:29.122725+0000","last_fullsized":"2024-04-17T07:12:29.122725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:42.009010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.227304+0000","last_change":"2024-04-17T07:11:54.462283+0000","last_active":"2024-04-17T07:12:32.227304+0000","last_peered":"2024-04-17T07:12:32.227304+0000","last_clean":"2024-04-17T07:12:32.227304+0000","last_became_active":"2024-04-17T07:11:54.461818+0000","last_became_peered":"2024-04-17T07:11:54.461818+0000","last_unstale":"2024-04-17T07:12:32.227304+0000","last_undegraded":"2024-04-17T07:12:32.227304+0000","last_fullsized":"2024-04-17T07:12:32.227304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.765058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726968+0000","last_change":"2024-04-17T07:11:55.486491+0000","last_active":"2024-04-17T07:12:28.726968+0000","last_peered":"2024-04-17T07:12:28.726968+0000","last_clean":"2024-04-17T07:12:28.726968+0000","last_became_active":"2024-04-17T07:11:55.486121+0000","last_became_peered":"2024-04-17T07:11:55.486121+0000","last_unstale":"2024-04-17T07:12:28.726968+0000","last_undegraded":"2024-04-17T07:12:28.726968+0000","last_fullsized":"2024-04-17T07:12:28.726968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:12.998321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713871+0000","last_change":"2024-04-17T07:11:59.511555+0000","last_active":"2024-04-17T07:12:28.713871+0000","last_peered":"2024-04-17T07:12:28.713871+0000","last_clean":"2024-04-17T07:12:28.713871+0000","last_became_active":"2024-04-17T07:11:59.511147+0000","last_became_peered":"2024-04-17T07:11:59.511147+0000","last_unstale":"2024-04-17T07:12:28.713871+0000","last_undegraded":"2024-04-17T07:12:28.713871+0000","last_fullsized":"2024-04-17T07:12:28.713871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:41.854815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.210000+0000","last_change":"2024-04-17T07:12:28.734476+0000","last_active":"2024-04-17T07:12:30.210000+0000","last_peered":"2024-04-17T07:12:30.210000+0000","last_clean":"2024-04-17T07:12:30.210000+0000","last_became_active":"2024-04-17T07:12:28.728187+0000","last_became_peered":"2024-04-17T07:12:28.728187+0000","last_unstale":"2024-04-17T07:12:30.210000+0000","last_undegraded":"2024-04-17T07:12:30.210000+0000","last_fullsized":"2024-04-17T07:12:30.210000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:45.749607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736562+0000","last_change":"2024-04-17T07:11:55.480589+0000","last_active":"2024-04-17T07:12:28.736562+0000","last_peered":"2024-04-17T07:12:28.736562+0000","last_clean":"2024-04-17T07:12:28.736562+0000","last_became_active":"2024-04-17T07:11:55.480224+0000","last_became_peered":"2024-04-17T07:11:55.480224+0000","last_unstale":"2024-04-17T07:12:28.736562+0000","last_undegraded":"2024-04-17T07:12:28.736562+0000","last_fullsized":"2024-04-17T07:12:28.736562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:05.892442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713815+0000","last_change":"2024-04-17T07:11:59.522021+0000","last_active":"2024-04-17T07:12:28.713815+0000","last_peered":"2024-04-17T07:12:28.713815+0000","last_clean":"2024-04-17T07:12:28.713815+0000","last_became_active":"2024-04-17T07:11:59.521493+0000","last_became_peered":"2024-04-17T07:11:59.521493+0000","last_unstale":"2024-04-17T07:12:28.713815+0000","last_undegraded":"2024-04-17T07:12:28.713815+0000","last_fullsized":"2024-04-17T07:12:28.713815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:20.386216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.195512+0000","last_change":"2024-04-17T07:11:54.468225+0000","last_active":"2024-04-17T07:12:29.195512+0000","last_peered":"2024-04-17T07:12:29.195512+0000","last_clean":"2024-04-17T07:12:29.195512+0000","last_became_active":"2024-04-17T07:11:54.467046+0000","last_became_peered":"2024-04-17T07:11:54.467046+0000","last_unstale":"2024-04-17T07:12:29.195512+0000","last_undegraded":"2024-04-17T07:12:29.195512+0000","last_fullsized":"2024-04-17T07:12:29.195512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:17.725255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726428+0000","last_change":"2024-04-17T07:11:55.488980+0000","last_active":"2024-04-17T07:12:28.726428+0000","last_peered":"2024-04-17T07:12:28.726428+0000","last_clean":"2024-04-17T07:12:28.726428+0000","last_became_active":"2024-04-17T07:11:55.488226+0000","last_became_peered":"2024-04-17T07:11:55.488226+0000","last_unstale":"2024-04-17T07:12:28.726428+0000","last_undegraded":"2024-04-17T07:12:28.726428+0000","last_fullsized":"2024-04-17T07:12:28.726428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:00.135973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714388+0000","last_change":"2024-04-17T07:11:59.515134+0000","last_active":"2024-04-17T07:12:28.714388+0000","last_peered":"2024-04-17T07:12:28.714388+0000","last_clean":"2024-04-17T07:12:28.714388+0000","last_became_active":"2024-04-17T07:11:59.514560+0000","last_became_peered":"2024-04-17T07:11:59.514560+0000","last_unstale":"2024-04-17T07:12:28.714388+0000","last_undegraded":"2024-04-17T07:12:28.714388+0000","last_fullsized":"2024-04-17T07:12:28.714388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:34.669789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'26","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.230037+0000","last_change":"2024-04-17T07:11:54.461226+0000","last_active":"2024-04-17T07:12:32.230037+0000","last_peered":"2024-04-17T07:12:32.230037+0000","last_clean":"2024-04-17T07:12:32.230037+0000","last_became_active":"2024-04-17T07:11:54.460573+0000","last_became_peered":"2024-04-17T07:11:54.460573+0000","last_unstale":"2024-04-17T07:12:32.230037+0000","last_undegraded":"2024-04-17T07:12:32.230037+0000","last_fullsized":"2024-04-17T07:12:32.230037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":44,"num_write":28,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.734828+0000","last_change":"2024-04-17T07:11:55.483925+0000","last_active":"2024-04-17T07:12:28.734828+0000","last_peered":"2024-04-17T07:12:28.734828+0000","last_clean":"2024-04-17T07:12:28.734828+0000","last_became_active":"2024-04-17T07:11:55.483584+0000","last_became_peered":"2024-04-17T07:11:55.483584+0000","last_unstale":"2024-04-17T07:12:28.734828+0000","last_undegraded":"2024-04-17T07:12:28.734828+0000","last_fullsized":"2024-04-17T07:12:28.734828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:12.328876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.737367+0000","last_change":"2024-04-17T07:11:59.515597+0000","last_active":"2024-04-17T07:12:28.737367+0000","last_peered":"2024-04-17T07:12:28.737367+0000","last_clean":"2024-04-17T07:12:28.737367+0000","last_became_active":"2024-04-17T07:11:59.514413+0000","last_became_peered":"2024-04-17T07:11:59.514413+0000","last_unstale":"2024-04-17T07:12:28.737367+0000","last_undegraded":"2024-04-17T07:12:28.737367+0000","last_fullsized":"2024-04-17T07:12:28.737367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:45.824127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.209717+0000","last_change":"2024-04-17T07:11:54.449984+0000","last_active":"2024-04-17T07:12:30.209717+0000","last_peered":"2024-04-17T07:12:30.209717+0000","last_clean":"2024-04-17T07:12:30.209717+0000","last_became_active":"2024-04-17T07:11:54.449628+0000","last_became_peered":"2024-04-17T07:11:54.449628+0000","last_unstale":"2024-04-17T07:12:30.209717+0000","last_undegraded":"2024-04-17T07:12:30.209717+0000","last_fullsized":"2024-04-17T07:12:30.209717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:26:03.281684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719923+0000","last_change":"2024-04-17T07:11:55.476912+0000","last_active":"2024-04-17T07:12:28.719923+0000","last_peered":"2024-04-17T07:12:28.719923+0000","last_clean":"2024-04-17T07:12:28.719923+0000","last_became_active":"2024-04-17T07:11:55.471576+0000","last_became_peered":"2024-04-17T07:11:55.471576+0000","last_unstale":"2024-04-17T07:12:28.719923+0000","last_undegraded":"2024-04-17T07:12:28.719923+0000","last_fullsized":"2024-04-17T07:12:28.719923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:35.439271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.733057+0000","last_change":"2024-04-17T07:11:59.515897+0000","last_active":"2024-04-17T07:12:27.733057+0000","last_peered":"2024-04-17T07:12:27.733057+0000","last_clean":"2024-04-17T07:12:27.733057+0000","last_became_active":"2024-04-17T07:11:59.515349+0000","last_became_peered":"2024-04-17T07:11:59.515349+0000","last_unstale":"2024-04-17T07:12:27.733057+0000","last_undegraded":"2024-04-17T07:12:27.733057+0000","last_fullsized":"2024-04-17T07:12:27.733057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:00.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214859+0000","last_change":"2024-04-17T07:11:54.462369+0000","last_active":"2024-04-17T07:12:30.214859+0000","last_peered":"2024-04-17T07:12:30.214859+0000","last_clean":"2024-04-17T07:12:30.214859+0000","last_became_active":"2024-04-17T07:11:54.460357+0000","last_became_peered":"2024-04-17T07:11:54.460357+0000","last_unstale":"2024-04-17T07:12:30.214859+0000","last_undegraded":"2024-04-17T07:12:30.214859+0000","last_fullsized":"2024-04-17T07:12:30.214859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:38.029980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720650+0000","last_change":"2024-04-17T07:11:55.475874+0000","last_active":"2024-04-17T07:12:28.720650+0000","last_peered":"2024-04-17T07:12:28.720650+0000","last_clean":"2024-04-17T07:12:28.720650+0000","last_became_active":"2024-04-17T07:11:55.475550+0000","last_became_peered":"2024-04-17T07:11:55.475550+0000","last_unstale":"2024-04-17T07:12:28.720650+0000","last_undegraded":"2024-04-17T07:12:28.720650+0000","last_fullsized":"2024-04-17T07:12:28.720650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:40.096573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714893+0000","last_change":"2024-04-17T07:11:59.514661+0000","last_active":"2024-04-17T07:12:28.714893+0000","last_peered":"2024-04-17T07:12:28.714893+0000","last_clean":"2024-04-17T07:12:28.714893+0000","last_became_active":"2024-04-17T07:11:59.513030+0000","last_became_peered":"2024-04-17T07:11:59.513030+0000","last_unstale":"2024-04-17T07:12:28.714893+0000","last_undegraded":"2024-04-17T07:12:28.714893+0000","last_fullsized":"2024-04-17T07:12:28.714893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:35.696961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713262+0000","last_change":"2024-04-17T07:11:54.455279+0000","last_active":"2024-04-17T07:12:28.713262+0000","last_peered":"2024-04-17T07:12:28.713262+0000","last_clean":"2024-04-17T07:12:28.713262+0000","last_became_active":"2024-04-17T07:11:54.453620+0000","last_became_peered":"2024-04-17T07:11:54.453620+0000","last_unstale":"2024-04-17T07:12:28.713262+0000","last_undegraded":"2024-04-17T07:12:28.713262+0000","last_fullsized":"2024-04-17T07:12:28.713262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:15.227410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719686+0000","last_change":"2024-04-17T07:11:55.469579+0000","last_active":"2024-04-17T07:12:28.719686+0000","last_peered":"2024-04-17T07:12:28.719686+0000","last_clean":"2024-04-17T07:12:28.719686+0000","last_became_active":"2024-04-17T07:11:55.468389+0000","last_became_peered":"2024-04-17T07:11:55.468389+0000","last_unstale":"2024-04-17T07:12:28.719686+0000","last_undegraded":"2024-04-17T07:12:28.719686+0000","last_fullsized":"2024-04-17T07:12:28.719686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:01.274677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720385+0000","last_change":"2024-04-17T07:11:59.512110+0000","last_active":"2024-04-17T07:12:28.720385+0000","last_peered":"2024-04-17T07:12:28.720385+0000","last_clean":"2024-04-17T07:12:28.720385+0000","last_became_active":"2024-04-17T07:11:59.510573+0000","last_became_peered":"2024-04-17T07:11:59.510573+0000","last_unstale":"2024-04-17T07:12:28.720385+0000","last_undegraded":"2024-04-17T07:12:28.720385+0000","last_fullsized":"2024-04-17T07:12:28.720385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:32.515687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'10","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-17T07:12:00.800050+0000","last_change":"2024-04-17T07:12:28.153142+0000","last_active":"2024-04-17T07:12:28.153142+0000","last_peered":"2024-04-17T07:12:00.800050+0000","last_clean":"2024-04-17T07:12:00.800050+0000","last_became_active":"2024-04-17T07:11:54.471537+0000","last_became_peered":"2024-04-17T07:11:54.471537+0000","last_unstale":"2024-04-17T07:12:00.800050+0000","last_undegraded":"2024-04-17T07:12:00.800050+0000","last_fullsized":"2024-04-17T07:12:00.800050+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.731088+0000","last_change":"2024-04-17T07:11:55.473540+0000","last_active":"2024-04-17T07:12:28.731088+0000","last_peered":"2024-04-17T07:12:28.731088+0000","last_clean":"2024-04-17T07:12:28.731088+0000","last_became_active":"2024-04-17T07:11:55.472436+0000","last_became_peered":"2024-04-17T07:11:55.472436+0000","last_unstale":"2024-04-17T07:12:28.731088+0000","last_undegraded":"2024-04-17T07:12:28.731088+0000","last_fullsized":"2024-04-17T07:12:28.731088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:54.317052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730908+0000","last_change":"2024-04-17T07:11:59.521305+0000","last_active":"2024-04-17T07:12:28.730908+0000","last_peered":"2024-04-17T07:12:28.730908+0000","last_clean":"2024-04-17T07:12:28.730908+0000","last_became_active":"2024-04-17T07:11:59.516860+0000","last_became_peered":"2024-04-17T07:11:59.516860+0000","last_unstale":"2024-04-17T07:12:28.730908+0000","last_undegraded":"2024-04-17T07:12:28.730908+0000","last_fullsized":"2024-04-17T07:12:28.730908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:23.403218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735984+0000","last_change":"2024-04-17T07:11:54.463378+0000","last_active":"2024-04-17T07:12:28.735984+0000","last_peered":"2024-04-17T07:12:28.735984+0000","last_clean":"2024-04-17T07:12:28.735984+0000","last_became_active":"2024-04-17T07:11:54.462120+0000","last_became_peered":"2024-04-17T07:11:54.462120+0000","last_unstale":"2024-04-17T07:12:28.735984+0000","last_undegraded":"2024-04-17T07:12:28.735984+0000","last_fullsized":"2024-04-17T07:12:28.735984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:42.109928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.737028+0000","last_change":"2024-04-17T07:11:55.478604+0000","last_active":"2024-04-17T07:12:27.737028+0000","last_peered":"2024-04-17T07:12:27.737028+0000","last_clean":"2024-04-17T07:12:27.737028+0000","last_became_active":"2024-04-17T07:11:55.478252+0000","last_became_peered":"2024-04-17T07:11:55.478252+0000","last_unstale":"2024-04-17T07:12:27.737028+0000","last_undegraded":"2024-04-17T07:12:27.737028+0000","last_fullsized":"2024-04-17T07:12:27.737028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:08.365896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727344+0000","last_change":"2024-04-17T07:11:59.518619+0000","last_active":"2024-04-17T07:12:28.727344+0000","last_peered":"2024-04-17T07:12:28.727344+0000","last_clean":"2024-04-17T07:12:28.727344+0000","last_became_active":"2024-04-17T07:11:59.518214+0000","last_became_peered":"2024-04-17T07:11:59.518214+0000","last_unstale":"2024-04-17T07:12:28.727344+0000","last_undegraded":"2024-04-17T07:12:28.727344+0000","last_fullsized":"2024-04-17T07:12:28.727344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:46.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715522+0000","last_change":"2024-04-17T07:11:54.453038+0000","last_active":"2024-04-17T07:12:28.715522+0000","last_peered":"2024-04-17T07:12:28.715522+0000","last_clean":"2024-04-17T07:12:28.715522+0000","last_became_active":"2024-04-17T07:11:54.452619+0000","last_became_peered":"2024-04-17T07:11:54.452619+0000","last_unstale":"2024-04-17T07:12:28.715522+0000","last_undegraded":"2024-04-17T07:12:28.715522+0000","last_fullsized":"2024-04-17T07:12:28.715522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:25.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.724948+0000","last_change":"2024-04-17T07:11:55.465990+0000","last_active":"2024-04-17T07:12:28.724948+0000","last_peered":"2024-04-17T07:12:28.724948+0000","last_clean":"2024-04-17T07:12:28.724948+0000","last_became_active":"2024-04-17T07:11:55.465523+0000","last_became_peered":"2024-04-17T07:11:55.465523+0000","last_unstale":"2024-04-17T07:12:28.724948+0000","last_undegraded":"2024-04-17T07:12:28.724948+0000","last_fullsized":"2024-04-17T07:12:28.724948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:40.366027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721070+0000","last_change":"2024-04-17T07:11:59.512119+0000","last_active":"2024-04-17T07:12:28.721070+0000","last_peered":"2024-04-17T07:12:28.721070+0000","last_clean":"2024-04-17T07:12:28.721070+0000","last_became_active":"2024-04-17T07:11:59.511460+0000","last_became_peered":"2024-04-17T07:11:59.511460+0000","last_unstale":"2024-04-17T07:12:28.721070+0000","last_undegraded":"2024-04-17T07:12:28.721070+0000","last_fullsized":"2024-04-17T07:12:28.721070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:50.888203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.221574+0000","last_change":"2024-04-17T07:11:54.469346+0000","last_active":"2024-04-17T07:12:31.221574+0000","last_peered":"2024-04-17T07:12:31.221574+0000","last_clean":"2024-04-17T07:12:31.221574+0000","last_became_active":"2024-04-17T07:11:54.468947+0000","last_became_peered":"2024-04-17T07:11:54.468947+0000","last_unstale":"2024-04-17T07:12:31.221574+0000","last_undegraded":"2024-04-17T07:12:31.221574+0000","last_fullsized":"2024-04-17T07:12:31.221574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:04.854625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.724510+0000","last_change":"2024-04-17T07:11:55.471235+0000","last_active":"2024-04-17T07:12:28.724510+0000","last_peered":"2024-04-17T07:12:28.724510+0000","last_clean":"2024-04-17T07:12:28.724510+0000","last_became_active":"2024-04-17T07:11:55.470857+0000","last_became_peered":"2024-04-17T07:11:55.470857+0000","last_unstale":"2024-04-17T07:12:28.724510+0000","last_undegraded":"2024-04-17T07:12:28.724510+0000","last_fullsized":"2024-04-17T07:12:28.724510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:21.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720514+0000","last_change":"2024-04-17T07:11:59.524519+0000","last_active":"2024-04-17T07:12:28.720514+0000","last_peered":"2024-04-17T07:12:28.720514+0000","last_clean":"2024-04-17T07:12:28.720514+0000","last_became_active":"2024-04-17T07:11:59.519270+0000","last_became_peered":"2024-04-17T07:11:59.519270+0000","last_unstale":"2024-04-17T07:12:28.720514+0000","last_undegraded":"2024-04-17T07:12:28.720514+0000","last_fullsized":"2024-04-17T07:12:28.720514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:41.080249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.734024+0000","last_change":"2024-04-17T07:11:55.473922+0000","last_active":"2024-04-17T07:12:28.734024+0000","last_peered":"2024-04-17T07:12:28.734024+0000","last_clean":"2024-04-17T07:12:28.734024+0000","last_became_active":"2024-04-17T07:11:55.468594+0000","last_became_peered":"2024-04-17T07:11:55.468594+0000","last_unstale":"2024-04-17T07:12:28.734024+0000","last_undegraded":"2024-04-17T07:12:28.734024+0000","last_fullsized":"2024-04-17T07:12:28.734024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:21.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.219511+0000","last_change":"2024-04-17T07:11:54.473066+0000","last_active":"2024-04-17T07:12:31.219511+0000","last_peered":"2024-04-17T07:12:31.219511+0000","last_clean":"2024-04-17T07:12:31.219511+0000","last_became_active":"2024-04-17T07:11:54.472708+0000","last_became_peered":"2024-04-17T07:11:54.472708+0000","last_unstale":"2024-04-17T07:12:31.219511+0000","last_undegraded":"2024-04-17T07:12:31.219511+0000","last_fullsized":"2024-04-17T07:12:31.219511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:27.614819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720813+0000","last_change":"2024-04-17T07:11:59.516939+0000","last_active":"2024-04-17T07:12:28.720813+0000","last_peered":"2024-04-17T07:12:28.720813+0000","last_clean":"2024-04-17T07:12:28.720813+0000","last_became_active":"2024-04-17T07:11:59.516333+0000","last_became_peered":"2024-04-17T07:11:59.516333+0000","last_unstale":"2024-04-17T07:12:28.720813+0000","last_undegraded":"2024-04-17T07:12:28.720813+0000","last_fullsized":"2024-04-17T07:12:28.720813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:15.648831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122635+0000","last_change":"2024-04-17T07:11:55.481006+0000","last_active":"2024-04-17T07:12:29.122635+0000","last_peered":"2024-04-17T07:12:29.122635+0000","last_clean":"2024-04-17T07:12:29.122635+0000","last_became_active":"2024-04-17T07:11:55.479751+0000","last_became_peered":"2024-04-17T07:11:55.479751+0000","last_unstale":"2024-04-17T07:12:29.122635+0000","last_undegraded":"2024-04-17T07:12:29.122635+0000","last_fullsized":"2024-04-17T07:12:29.122635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:13.004650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713874+0000","last_change":"2024-04-17T07:11:54.463106+0000","last_active":"2024-04-17T07:12:28.713874+0000","last_peered":"2024-04-17T07:12:28.713874+0000","last_clean":"2024-04-17T07:12:28.713874+0000","last_became_active":"2024-04-17T07:11:54.462691+0000","last_became_peered":"2024-04-17T07:11:54.462691+0000","last_unstale":"2024-04-17T07:12:28.713874+0000","last_undegraded":"2024-04-17T07:12:28.713874+0000","last_fullsized":"2024-04-17T07:12:28.713874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:34.302019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719632+0000","last_change":"2024-04-17T07:10:14.685883+0000","last_active":"2024-04-17T07:12:28.719632+0000","last_peered":"2024-04-17T07:12:28.719632+0000","last_clean":"2024-04-17T07:12:28.719632+0000","last_became_active":"2024-04-17T07:10:14.684551+0000","last_became_peered":"2024-04-17T07:10:14.684551+0000","last_unstale":"2024-04-17T07:12:28.719632+0000","last_undegraded":"2024-04-17T07:12:28.719632+0000","last_fullsized":"2024-04-17T07:12:28.719632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_clean_scrub_stamp":"2024-04-17T07:05:37.112317+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:04.265147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736804+0000","last_change":"2024-04-17T07:12:28.731807+0000","last_active":"2024-04-17T07:12:28.736804+0000","last_peered":"2024-04-17T07:12:28.736804+0000","last_clean":"2024-04-17T07:12:28.736804+0000","last_became_active":"2024-04-17T07:12:28.730656+0000","last_became_peered":"2024-04-17T07:12:28.730656+0000","last_unstale":"2024-04-17T07:12:28.736804+0000","last_undegraded":"2024-04-17T07:12:28.736804+0000","last_fullsized":"2024-04-17T07:12:28.736804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:58.023333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727980+0000","last_change":"2024-04-17T07:12:28.727980+0000","last_active":"2024-04-17T07:12:28.727980+0000","last_peered":"2024-04-17T07:12:28.727980+0000","last_clean":"2024-04-17T07:12:28.727980+0000","last_became_active":"2024-04-17T07:12:28.727060+0000","last_became_peered":"2024-04-17T07:12:28.727060+0000","last_unstale":"2024-04-17T07:12:28.727980+0000","last_undegraded":"2024-04-17T07:12:28.727980+0000","last_fullsized":"2024-04-17T07:12:28.727980+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:33.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719921+0000","last_change":"2024-04-17T07:11:54.463180+0000","last_active":"2024-04-17T07:12:28.719921+0000","last_peered":"2024-04-17T07:12:28.719921+0000","last_clean":"2024-04-17T07:12:28.719921+0000","last_became_active":"2024-04-17T07:11:54.462793+0000","last_became_peered":"2024-04-17T07:11:54.462793+0000","last_unstale":"2024-04-17T07:12:28.719921+0000","last_undegraded":"2024-04-17T07:12:28.719921+0000","last_fullsized":"2024-04-17T07:12:28.719921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:19.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.733730+0000","last_change":"2024-04-17T07:11:59.518355+0000","last_active":"2024-04-17T07:12:27.733730+0000","last_peered":"2024-04-17T07:12:27.733730+0000","last_clean":"2024-04-17T07:12:27.733730+0000","last_became_active":"2024-04-17T07:11:59.517619+0000","last_became_peered":"2024-04-17T07:11:59.517619+0000","last_unstale":"2024-04-17T07:12:27.733730+0000","last_undegraded":"2024-04-17T07:12:27.733730+0000","last_fullsized":"2024-04-17T07:12:27.733730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:36.826021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.196898+0000","last_change":"2024-04-17T07:11:54.465285+0000","last_active":"2024-04-17T07:12:29.196898+0000","last_peered":"2024-04-17T07:12:29.196898+0000","last_clean":"2024-04-17T07:12:29.196898+0000","last_became_active":"2024-04-17T07:11:54.464818+0000","last_became_peered":"2024-04-17T07:11:54.464818+0000","last_unstale":"2024-04-17T07:12:29.196898+0000","last_undegraded":"2024-04-17T07:12:29.196898+0000","last_fullsized":"2024-04-17T07:12:29.196898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721576+0000","last_change":"2024-04-17T07:11:55.470405+0000","last_active":"2024-04-17T07:12:28.721576+0000","last_peered":"2024-04-17T07:12:28.721576+0000","last_clean":"2024-04-17T07:12:28.721576+0000","last_became_active":"2024-04-17T07:11:55.470058+0000","last_became_peered":"2024-04-17T07:11:55.470058+0000","last_unstale":"2024-04-17T07:12:28.721576+0000","last_undegraded":"2024-04-17T07:12:28.721576+0000","last_fullsized":"2024-04-17T07:12:28.721576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:17.416775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122022+0000","last_change":"2024-04-17T07:11:59.522040+0000","last_active":"2024-04-17T07:12:29.122022+0000","last_peered":"2024-04-17T07:12:29.122022+0000","last_clean":"2024-04-17T07:12:29.122022+0000","last_became_active":"2024-04-17T07:11:59.519771+0000","last_became_peered":"2024-04-17T07:11:59.519771+0000","last_unstale":"2024-04-17T07:12:29.122022+0000","last_undegraded":"2024-04-17T07:12:29.122022+0000","last_fullsized":"2024-04-17T07:12:29.122022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:58.203737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.488533+0000","last_change":"2024-04-17T07:11:54.457954+0000","last_active":"2024-04-17T07:12:30.488533+0000","last_peered":"2024-04-17T07:12:30.488533+0000","last_clean":"2024-04-17T07:12:30.488533+0000","last_became_active":"2024-04-17T07:11:54.457560+0000","last_became_peered":"2024-04-17T07:11:54.457560+0000","last_unstale":"2024-04-17T07:12:30.488533+0000","last_undegraded":"2024-04-17T07:12:30.488533+0000","last_fullsized":"2024-04-17T07:12:30.488533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:31.919730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123098+0000","last_change":"2024-04-17T07:11:55.480475+0000","last_active":"2024-04-17T07:12:29.123098+0000","last_peered":"2024-04-17T07:12:29.123098+0000","last_clean":"2024-04-17T07:12:29.123098+0000","last_became_active":"2024-04-17T07:11:55.480060+0000","last_became_peered":"2024-04-17T07:11:55.480060+0000","last_unstale":"2024-04-17T07:12:29.123098+0000","last_undegraded":"2024-04-17T07:12:29.123098+0000","last_fullsized":"2024-04-17T07:12:29.123098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:30:23.600244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714322+0000","last_change":"2024-04-17T07:11:59.525422+0000","last_active":"2024-04-17T07:12:28.714322+0000","last_peered":"2024-04-17T07:12:28.714322+0000","last_clean":"2024-04-17T07:12:28.714322+0000","last_became_active":"2024-04-17T07:11:59.523649+0000","last_became_peered":"2024-04-17T07:11:59.523649+0000","last_unstale":"2024-04-17T07:12:28.714322+0000","last_undegraded":"2024-04-17T07:12:28.714322+0000","last_fullsized":"2024-04-17T07:12:28.714322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:16.653752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714912+0000","last_change":"2024-04-17T07:11:54.465824+0000","last_active":"2024-04-17T07:12:28.714912+0000","last_peered":"2024-04-17T07:12:28.714912+0000","last_clean":"2024-04-17T07:12:28.714912+0000","last_became_active":"2024-04-17T07:11:54.465348+0000","last_became_peered":"2024-04-17T07:11:54.465348+0000","last_unstale":"2024-04-17T07:12:28.714912+0000","last_undegraded":"2024-04-17T07:12:28.714912+0000","last_fullsized":"2024-04-17T07:12:28.714912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:17.729903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736284+0000","last_change":"2024-04-17T07:11:55.481666+0000","last_active":"2024-04-17T07:12:28.736284+0000","last_peered":"2024-04-17T07:12:28.736284+0000","last_clean":"2024-04-17T07:12:28.736284+0000","last_became_active":"2024-04-17T07:11:55.480332+0000","last_became_peered":"2024-04-17T07:11:55.480332+0000","last_unstale":"2024-04-17T07:12:28.736284+0000","last_undegraded":"2024-04-17T07:12:28.736284+0000","last_fullsized":"2024-04-17T07:12:28.736284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:19.825407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714433+0000","last_change":"2024-04-17T07:11:59.527850+0000","last_active":"2024-04-17T07:12:28.714433+0000","last_peered":"2024-04-17T07:12:28.714433+0000","last_clean":"2024-04-17T07:12:28.714433+0000","last_became_active":"2024-04-17T07:11:59.527306+0000","last_became_peered":"2024-04-17T07:11:59.527306+0000","last_unstale":"2024-04-17T07:12:28.714433+0000","last_undegraded":"2024-04-17T07:12:28.714433+0000","last_fullsized":"2024-04-17T07:12:28.714433+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:35.906222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725682+0000","last_change":"2024-04-17T07:11:54.455862+0000","last_active":"2024-04-17T07:12:28.725682+0000","last_peered":"2024-04-17T07:12:28.725682+0000","last_clean":"2024-04-17T07:12:28.725682+0000","last_became_active":"2024-04-17T07:11:54.453904+0000","last_became_peered":"2024-04-17T07:11:54.453904+0000","last_unstale":"2024-04-17T07:12:28.725682+0000","last_undegraded":"2024-04-17T07:12:28.725682+0000","last_fullsized":"2024-04-17T07:12:28.725682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:01.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725079+0000","last_change":"2024-04-17T07:11:55.470315+0000","last_active":"2024-04-17T07:12:28.725079+0000","last_peered":"2024-04-17T07:12:28.725079+0000","last_clean":"2024-04-17T07:12:28.725079+0000","last_became_active":"2024-04-17T07:11:55.469829+0000","last_became_peered":"2024-04-17T07:11:55.469829+0000","last_unstale":"2024-04-17T07:12:28.725079+0000","last_undegraded":"2024-04-17T07:12:28.725079+0000","last_fullsized":"2024-04-17T07:12:28.725079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:48.197546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.126045+0000","last_change":"2024-04-17T07:12:29.125447+0000","last_active":"2024-04-17T07:12:29.126045+0000","last_peered":"2024-04-17T07:12:29.126045+0000","last_clean":"2024-04-17T07:12:29.126045+0000","last_became_active":"2024-04-17T07:12:29.124551+0000","last_became_peered":"2024-04-17T07:12:29.124551+0000","last_unstale":"2024-04-17T07:12:29.126045+0000","last_undegraded":"2024-04-17T07:12:29.126045+0000","last_fullsized":"2024-04-17T07:12:29.126045+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:09.489755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.216091+0000","last_change":"2024-04-17T07:11:54.453117+0000","last_active":"2024-04-17T07:12:30.216091+0000","last_peered":"2024-04-17T07:12:30.216091+0000","last_clean":"2024-04-17T07:12:30.216091+0000","last_became_active":"2024-04-17T07:11:54.452632+0000","last_became_peered":"2024-04-17T07:11:54.452632+0000","last_unstale":"2024-04-17T07:12:30.216091+0000","last_undegraded":"2024-04-17T07:12:30.216091+0000","last_fullsized":"2024-04-17T07:12:30.216091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:09.366149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.728073+0000","last_change":"2024-04-17T07:11:55.469991+0000","last_active":"2024-04-17T07:12:27.728073+0000","last_peered":"2024-04-17T07:12:27.728073+0000","last_clean":"2024-04-17T07:12:27.728073+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:12:27.728073+0000","last_undegraded":"2024-04-17T07:12:27.728073+0000","last_fullsized":"2024-04-17T07:12:27.728073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:02.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713914+0000","last_change":"2024-04-17T07:11:59.519817+0000","last_active":"2024-04-17T07:12:28.713914+0000","last_peered":"2024-04-17T07:12:28.713914+0000","last_clean":"2024-04-17T07:12:28.713914+0000","last_became_active":"2024-04-17T07:11:59.519108+0000","last_became_peered":"2024-04-17T07:11:59.519108+0000","last_unstale":"2024-04-17T07:12:28.713914+0000","last_undegraded":"2024-04-17T07:12:28.713914+0000","last_fullsized":"2024-04-17T07:12:28.713914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:58.378837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716195+0000","last_change":"2024-04-17T07:11:54.463618+0000","last_active":"2024-04-17T07:12:28.716195+0000","last_peered":"2024-04-17T07:12:28.716195+0000","last_clean":"2024-04-17T07:12:28.716195+0000","last_became_active":"2024-04-17T07:11:54.463257+0000","last_became_peered":"2024-04-17T07:11:54.463257+0000","last_unstale":"2024-04-17T07:12:28.716195+0000","last_undegraded":"2024-04-17T07:12:28.716195+0000","last_fullsized":"2024-04-17T07:12:28.716195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:20.135338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122992+0000","last_change":"2024-04-17T07:11:55.487471+0000","last_active":"2024-04-17T07:12:29.122992+0000","last_peered":"2024-04-17T07:12:29.122992+0000","last_clean":"2024-04-17T07:12:29.122992+0000","last_became_active":"2024-04-17T07:11:55.487024+0000","last_became_peered":"2024-04-17T07:11:55.487024+0000","last_unstale":"2024-04-17T07:12:29.122992+0000","last_undegraded":"2024-04-17T07:12:29.122992+0000","last_fullsized":"2024-04-17T07:12:29.122992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:36.851210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714472+0000","last_change":"2024-04-17T07:11:59.527017+0000","last_active":"2024-04-17T07:12:28.714472+0000","last_peered":"2024-04-17T07:12:28.714472+0000","last_clean":"2024-04-17T07:12:28.714472+0000","last_became_active":"2024-04-17T07:11:59.526541+0000","last_became_peered":"2024-04-17T07:11:59.526541+0000","last_unstale":"2024-04-17T07:12:28.714472+0000","last_undegraded":"2024-04-17T07:12:28.714472+0000","last_fullsized":"2024-04-17T07:12:28.714472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:54.411647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.728776+0000","last_change":"2024-04-17T07:11:54.465671+0000","last_active":"2024-04-17T07:12:28.728776+0000","last_peered":"2024-04-17T07:12:28.728776+0000","last_clean":"2024-04-17T07:12:28.728776+0000","last_became_active":"2024-04-17T07:11:54.464902+0000","last_became_peered":"2024-04-17T07:11:54.464902+0000","last_unstale":"2024-04-17T07:12:28.728776+0000","last_undegraded":"2024-04-17T07:12:28.728776+0000","last_fullsized":"2024-04-17T07:12:28.728776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:13:38.232965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720011+0000","last_change":"2024-04-17T07:11:55.484259+0000","last_active":"2024-04-17T07:12:28.720011+0000","last_peered":"2024-04-17T07:12:28.720011+0000","last_clean":"2024-04-17T07:12:28.720011+0000","last_became_active":"2024-04-17T07:11:55.483911+0000","last_became_peered":"2024-04-17T07:11:55.483911+0000","last_unstale":"2024-04-17T07:12:28.720011+0000","last_undegraded":"2024-04-17T07:12:28.720011+0000","last_fullsized":"2024-04-17T07:12:28.720011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:54.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725852+0000","last_change":"2024-04-17T07:11:59.525369+0000","last_active":"2024-04-17T07:12:28.725852+0000","last_peered":"2024-04-17T07:12:28.725852+0000","last_clean":"2024-04-17T07:12:28.725852+0000","last_became_active":"2024-04-17T07:11:59.524809+0000","last_became_peered":"2024-04-17T07:11:59.524809+0000","last_unstale":"2024-04-17T07:12:28.725852+0000","last_undegraded":"2024-04-17T07:12:28.725852+0000","last_fullsized":"2024-04-17T07:12:28.725852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:20.511001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.197527+0000","last_change":"2024-04-17T07:11:54.459079+0000","last_active":"2024-04-17T07:12:29.197527+0000","last_peered":"2024-04-17T07:12:29.197527+0000","last_clean":"2024-04-17T07:12:29.197527+0000","last_became_active":"2024-04-17T07:11:54.457008+0000","last_became_peered":"2024-04-17T07:11:54.457008+0000","last_unstale":"2024-04-17T07:12:29.197527+0000","last_undegraded":"2024-04-17T07:12:29.197527+0000","last_fullsized":"2024-04-17T07:12:29.197527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:36.084512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721123+0000","last_change":"2024-04-17T07:11:55.484453+0000","last_active":"2024-04-17T07:12:28.721123+0000","last_peered":"2024-04-17T07:12:28.721123+0000","last_clean":"2024-04-17T07:12:28.721123+0000","last_became_active":"2024-04-17T07:11:55.483152+0000","last_became_peered":"2024-04-17T07:11:55.483152+0000","last_unstale":"2024-04-17T07:12:28.721123+0000","last_undegraded":"2024-04-17T07:12:28.721123+0000","last_fullsized":"2024-04-17T07:12:28.721123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:32.224868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735401+0000","last_change":"2024-04-17T07:11:59.521487+0000","last_active":"2024-04-17T07:12:28.735401+0000","last_peered":"2024-04-17T07:12:28.735401+0000","last_clean":"2024-04-17T07:12:28.735401+0000","last_became_active":"2024-04-17T07:11:59.521060+0000","last_became_peered":"2024-04-17T07:11:59.521060+0000","last_unstale":"2024-04-17T07:12:28.735401+0000","last_undegraded":"2024-04-17T07:12:28.735401+0000","last_fullsized":"2024-04-17T07:12:28.735401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:06.692434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214903+0000","last_change":"2024-04-17T07:11:54.459368+0000","last_active":"2024-04-17T07:12:30.214903+0000","last_peered":"2024-04-17T07:12:30.214903+0000","last_clean":"2024-04-17T07:12:30.214903+0000","last_became_active":"2024-04-17T07:11:54.458973+0000","last_became_peered":"2024-04-17T07:11:54.458973+0000","last_unstale":"2024-04-17T07:12:30.214903+0000","last_undegraded":"2024-04-17T07:12:30.214903+0000","last_fullsized":"2024-04-17T07:12:30.214903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:22.034110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721530+0000","last_change":"2024-04-17T07:11:55.470648+0000","last_active":"2024-04-17T07:12:28.721530+0000","last_peered":"2024-04-17T07:12:28.721530+0000","last_clean":"2024-04-17T07:12:28.721530+0000","last_became_active":"2024-04-17T07:11:55.469666+0000","last_became_peered":"2024-04-17T07:11:55.469666+0000","last_unstale":"2024-04-17T07:12:28.721530+0000","last_undegraded":"2024-04-17T07:12:28.721530+0000","last_fullsized":"2024-04-17T07:12:28.721530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:59.208872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730680+0000","last_change":"2024-04-17T07:11:59.524027+0000","last_active":"2024-04-17T07:12:28.730680+0000","last_peered":"2024-04-17T07:12:28.730680+0000","last_clean":"2024-04-17T07:12:28.730680+0000","last_became_active":"2024-04-17T07:11:59.522764+0000","last_became_peered":"2024-04-17T07:11:59.522764+0000","last_unstale":"2024-04-17T07:12:28.730680+0000","last_undegraded":"2024-04-17T07:12:28.730680+0000","last_fullsized":"2024-04-17T07:12:28.730680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:40.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.216540+0000","last_change":"2024-04-17T07:11:54.472884+0000","last_active":"2024-04-17T07:12:30.216540+0000","last_peered":"2024-04-17T07:12:30.216540+0000","last_clean":"2024-04-17T07:12:30.216540+0000","last_became_active":"2024-04-17T07:11:54.472305+0000","last_became_peered":"2024-04-17T07:11:54.472305+0000","last_unstale":"2024-04-17T07:12:30.216540+0000","last_undegraded":"2024-04-17T07:12:30.216540+0000","last_fullsized":"2024-04-17T07:12:30.216540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:21.031442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721232+0000","last_change":"2024-04-17T07:11:55.479982+0000","last_active":"2024-04-17T07:12:28.721232+0000","last_peered":"2024-04-17T07:12:28.721232+0000","last_clean":"2024-04-17T07:12:28.721232+0000","last_became_active":"2024-04-17T07:11:55.477594+0000","last_became_peered":"2024-04-17T07:11:55.477594+0000","last_unstale":"2024-04-17T07:12:28.721232+0000","last_undegraded":"2024-04-17T07:12:28.721232+0000","last_fullsized":"2024-04-17T07:12:28.721232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:23.507312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.732471+0000","last_change":"2024-04-17T07:11:55.480941+0000","last_active":"2024-04-17T07:12:27.732471+0000","last_peered":"2024-04-17T07:12:27.732471+0000","last_clean":"2024-04-17T07:12:27.732471+0000","last_became_active":"2024-04-17T07:11:55.480180+0000","last_became_peered":"2024-04-17T07:11:55.480180+0000","last_unstale":"2024-04-17T07:12:27.732471+0000","last_undegraded":"2024-04-17T07:12:27.732471+0000","last_fullsized":"2024-04-17T07:12:27.732471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:16.321806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715350+0000","last_change":"2024-04-17T07:11:55.477663+0000","last_active":"2024-04-17T07:12:28.715350+0000","last_peered":"2024-04-17T07:12:28.715350+0000","last_clean":"2024-04-17T07:12:28.715350+0000","last_became_active":"2024-04-17T07:11:55.477304+0000","last_became_peered":"2024-04-17T07:11:55.477304+0000","last_unstale":"2024-04-17T07:12:28.715350+0000","last_undegraded":"2024-04-17T07:12:28.715350+0000","last_fullsized":"2024-04-17T07:12:28.715350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:00.796633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714460+0000","last_change":"2024-04-17T07:11:55.475536+0000","last_active":"2024-04-17T07:12:28.714460+0000","last_peered":"2024-04-17T07:12:28.714460+0000","last_clean":"2024-04-17T07:12:28.714460+0000","last_became_active":"2024-04-17T07:11:55.475138+0000","last_became_peered":"2024-04-17T07:11:55.475138+0000","last_unstale":"2024-04-17T07:12:28.714460+0000","last_undegraded":"2024-04-17T07:12:28.714460+0000","last_fullsized":"2024-04-17T07:12:28.714460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:47.849076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735949+0000","last_change":"2024-04-17T07:11:55.483742+0000","last_active":"2024-04-17T07:12:28.735949+0000","last_peered":"2024-04-17T07:12:28.735949+0000","last_clean":"2024-04-17T07:12:28.735949+0000","last_became_active":"2024-04-17T07:11:55.483044+0000","last_became_peered":"2024-04-17T07:11:55.483044+0000","last_unstale":"2024-04-17T07:12:28.735949+0000","last_undegraded":"2024-04-17T07:12:28.735949+0000","last_fullsized":"2024-04-17T07:12:28.735949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:18.972137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721440+0000","last_change":"2024-04-17T07:11:55.471266+0000","last_active":"2024-04-17T07:12:28.721440+0000","last_peered":"2024-04-17T07:12:28.721440+0000","last_clean":"2024-04-17T07:12:28.721440+0000","last_became_active":"2024-04-17T07:11:55.470897+0000","last_became_peered":"2024-04-17T07:11:55.470897+0000","last_unstale":"2024-04-17T07:12:28.721440+0000","last_undegraded":"2024-04-17T07:12:28.721440+0000","last_fullsized":"2024-04-17T07:12:28.721440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:25.811064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.736847+0000","last_change":"2024-04-17T07:11:55.475860+0000","last_active":"2024-04-17T07:12:27.736847+0000","last_peered":"2024-04-17T07:12:27.736847+0000","last_clean":"2024-04-17T07:12:27.736847+0000","last_became_active":"2024-04-17T07:11:55.475522+0000","last_became_peered":"2024-04-17T07:11:55.475522+0000","last_unstale":"2024-04-17T07:12:27.736847+0000","last_undegraded":"2024-04-17T07:12:27.736847+0000","last_fullsized":"2024-04-17T07:12:27.736847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:41.548964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715736+0000","last_change":"2024-04-17T07:11:55.478815+0000","last_active":"2024-04-17T07:12:28.715736+0000","last_peered":"2024-04-17T07:12:28.715736+0000","last_clean":"2024-04-17T07:12:28.715736+0000","last_became_active":"2024-04-17T07:11:55.478347+0000","last_became_peered":"2024-04-17T07:11:55.478347+0000","last_unstale":"2024-04-17T07:12:28.715736+0000","last_undegraded":"2024-04-17T07:12:28.715736+0000","last_fullsized":"2024-04-17T07:12:28.715736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:16.465511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123444+0000","last_change":"2024-04-17T07:11:55.474208+0000","last_active":"2024-04-17T07:12:29.123444+0000","last_peered":"2024-04-17T07:12:29.123444+0000","last_clean":"2024-04-17T07:12:29.123444+0000","last_became_active":"2024-04-17T07:11:55.472042+0000","last_became_peered":"2024-04-17T07:11:55.472042+0000","last_unstale":"2024-04-17T07:12:29.123444+0000","last_undegraded":"2024-04-17T07:12:29.123444+0000","last_fullsized":"2024-04-17T07:12:29.123444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:37:24.248506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.733719+0000","last_change":"2024-04-17T07:11:55.477257+0000","last_active":"2024-04-17T07:12:28.733719+0000","last_peered":"2024-04-17T07:12:28.733719+0000","last_clean":"2024-04-17T07:12:28.733719+0000","last_became_active":"2024-04-17T07:11:55.473491+0000","last_became_peered":"2024-04-17T07:11:55.473491+0000","last_unstale":"2024-04-17T07:12:28.733719+0000","last_undegraded":"2024-04-17T07:12:28.733719+0000","last_fullsized":"2024-04-17T07:12:28.733719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:33.737485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122022+0000","last_change":"2024-04-17T07:11:55.474930+0000","last_active":"2024-04-17T07:12:29.122022+0000","last_peered":"2024-04-17T07:12:29.122022+0000","last_clean":"2024-04-17T07:12:29.122022+0000","last_became_active":"2024-04-17T07:11:55.472007+0000","last_became_peered":"2024-04-17T07:11:55.472007+0000","last_unstale":"2024-04-17T07:12:29.122022+0000","last_undegraded":"2024-04-17T07:12:29.122022+0000","last_fullsized":"2024-04-17T07:12:29.122022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:07.111267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726119+0000","last_change":"2024-04-17T07:11:55.492422+0000","last_active":"2024-04-17T07:12:28.726119+0000","last_peered":"2024-04-17T07:12:28.726119+0000","last_clean":"2024-04-17T07:12:28.726119+0000","last_became_active":"2024-04-17T07:11:55.491906+0000","last_became_peered":"2024-04-17T07:11:55.491906+0000","last_unstale":"2024-04-17T07:12:28.726119+0000","last_undegraded":"2024-04-17T07:12:28.726119+0000","last_fullsized":"2024-04-17T07:12:28.726119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:13:05.322586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720385+0000","last_change":"2024-04-17T07:11:55.479436+0000","last_active":"2024-04-17T07:12:28.720385+0000","last_peered":"2024-04-17T07:12:28.720385+0000","last_clean":"2024-04-17T07:12:28.720385+0000","last_became_active":"2024-04-17T07:11:55.478982+0000","last_became_peered":"2024-04-17T07:11:55.478982+0000","last_unstale":"2024-04-17T07:12:28.720385+0000","last_undegraded":"2024-04-17T07:12:28.720385+0000","last_fullsized":"2024-04-17T07:12:28.720385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:20.663457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.733290+0000","last_change":"2024-04-17T07:11:55.476708+0000","last_active":"2024-04-17T07:12:27.733290+0000","last_peered":"2024-04-17T07:12:27.733290+0000","last_clean":"2024-04-17T07:12:27.733290+0000","last_became_active":"2024-04-17T07:11:55.476344+0000","last_became_peered":"2024-04-17T07:11:55.476344+0000","last_unstale":"2024-04-17T07:12:27.733290+0000","last_undegraded":"2024-04-17T07:12:27.733290+0000","last_fullsized":"2024-04-17T07:12:27.733290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:58.874987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279575,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48551,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":26,"num_read_kb":61,"num_write":196,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":95150,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383702,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14703,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645343,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":1130171,"data_compressed":58079,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":501379,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":497549,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":1094158,"data_compressed":56230,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":496883,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":501598,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":498339,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":507953,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":496569,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":487441,"data_compressed":17684,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1093836,"data_compressed":56230,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":13207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43642,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:12:33.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:12:34.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.034+0000 7f5bc1a00640 1 -- 172.21.15.86:0/2139625534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bbc103fe0 msgr2=0x7f5bbc104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:34.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.034+0000 7f5bc1a00640 1 --2- 172.21.15.86:0/2139625534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bbc103fe0 0x7f5bbc104400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009ca0 tx=0x7f5bac02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.034+0000 7f5bc1a00640 1 -- 172.21.15.86:0/2139625534 shutdown_connections 2024-04-17T07:12:34.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.034+0000 7f5bc1a00640 1 --2- 172.21.15.86:0/2139625534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc105b90 0x7f5bbc10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.034+0000 7f5bc1a00640 1 --2- 172.21.15.86:0/2139625534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bbc1051d0 0x7f5bbc105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.034+0000 7f5bc1a00640 1 --2- 172.21.15.86:0/2139625534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bbc103fe0 0x7f5bbc104400 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.034+0000 7f5bc1a00640 1 -- 172.21.15.86:0/2139625534 >> 172.21.15.86:0/2139625534 conn(0x7f5bbc0ff570 msgr2=0x7f5bbc101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:34.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.034+0000 7f5bc1a00640 1 -- 172.21.15.86:0/2139625534 shutdown_connections 2024-04-17T07:12:34.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.034+0000 7f5bc1a00640 1 -- 172.21.15.86:0/2139625534 wait complete. 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.035+0000 7f5bc1a00640 1 Processor -- start 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.035+0000 7f5bc1a00640 1 -- start start 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.035+0000 7f5bc1a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bbc103fe0 0x7f5bbc19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.035+0000 7f5bc1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bbc1051d0 0x7f5bbc19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.035+0000 7f5bc1a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc105b90 0x7f5bbc1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.035+0000 7f5bc1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc10fae0 con 0x7f5bbc1051d0 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.035+0000 7f5bc1a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc10f960 con 0x7f5bbc103fe0 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.035+0000 7f5bc1a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc10fc60 con 0x7f5bbc105b90 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bbc1051d0 0x7f5bbc19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bbc1051d0 0x7f5bbc19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54190/0 (socket says 172.21.15.86:54190) 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bb3400640 1 -- 172.21.15.86:0/1563312391 learned_addr learned my addr 172.21.15.86:0/1563312391 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bbbe00640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bbc103fe0 0x7f5bbc19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bc1000640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc105b90 0x7f5bbc1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bb3400640 1 -- 172.21.15.86:0/1563312391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc105b90 msgr2=0x7f5bbc1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bb3400640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc105b90 0x7f5bbc1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bb3400640 1 -- 172.21.15.86:0/1563312391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bbc103fe0 msgr2=0x7f5bbc19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bb3400640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bbc103fe0 0x7f5bbc19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bb3400640 1 -- 172.21.15.86:0/1563312391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc1a1e50 con 0x7f5bbc1051d0 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bc1000640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc105b90 0x7f5bbc1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.036+0000 7f5bb3400640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bbc1051d0 0x7f5bbc19a770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400d950 tx=0x7f5ba400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.037+0000 7f5bb9600640 1 -- 172.21.15.86:0/1563312391 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4017070 con 0x7f5bbc1051d0 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.037+0000 7f5bb9600640 1 -- 172.21.15.86:0/1563312391 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ba4009c30 con 0x7f5bbc1051d0 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.037+0000 7f5bc1a00640 1 -- 172.21.15.86:0/1563312391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc1a2140 con 0x7f5bbc1051d0 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.037+0000 7f5bb9600640 1 -- 172.21.15.86:0/1563312391 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba400f620 con 0x7f5bbc1051d0 2024-04-17T07:12:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.037+0000 7f5bc1a00640 1 -- 172.21.15.86:0/1563312391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc1a2690 con 0x7f5bbc1051d0 2024-04-17T07:12:34.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.038+0000 7f5bc1a00640 1 -- 172.21.15.86:0/1563312391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bbc10b3d0 con 0x7f5bbc1051d0 2024-04-17T07:12:34.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.040+0000 7f5bb9600640 1 -- 172.21.15.86:0/1563312391 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ba40109a0 con 0x7f5bbc1051d0 2024-04-17T07:12:34.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.041+0000 7f5bb9600640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5ba00a3c40 0x7f5ba00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:34.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.041+0000 7f5bb9600640 1 -- 172.21.15.86:0/1563312391 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5ba40d0630 con 0x7f5bbc1051d0 2024-04-17T07:12:34.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.042+0000 7f5bbbe00640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5ba00a3c40 0x7f5ba00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:34.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.042+0000 7f5bbbe00640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5ba00a3c40 0x7f5ba00a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5bac02c700 tx=0x7f5bac000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:34.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.046+0000 7f5bb9600640 1 -- 172.21.15.86:0/1563312391 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5ba4009de0 con 0x7f5bbc1051d0 2024-04-17T07:12:34.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.246+0000 7f5bc1a00640 1 -- 172.21.15.86:0/1563312391 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5bbc0623a0 con 0x7f5ba00a3c40 2024-04-17T07:12:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.275+0000 7f5bb9600640 1 -- 172.21.15.86:0/1563312391 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340706 (secure 0 0 0) 0x7f5bbc0623a0 con 0x7f5ba00a3c40 2024-04-17T07:12:34.278 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:34.282 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-17T07:12:34.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.283+0000 7f5bc1a00640 1 -- 172.21.15.86:0/1563312391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5ba00a3c40 msgr2=0x7f5ba00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.283+0000 7f5bc1a00640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5ba00a3c40 0x7f5ba00a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5bac02c700 tx=0x7f5bac000f40 comp rx=0 tx=0).stop 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.283+0000 7f5bc1a00640 1 -- 172.21.15.86:0/1563312391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bbc1051d0 msgr2=0x7f5bbc19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.283+0000 7f5bc1a00640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bbc1051d0 0x7f5bbc19a770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400d950 tx=0x7f5ba400de10 comp rx=0 tx=0).stop 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.283+0000 7f5bc1a00640 1 -- 172.21.15.86:0/1563312391 shutdown_connections 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.283+0000 7f5bc1a00640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5ba00a3c40 0x7f5ba00a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.283+0000 7f5bc1a00640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5bbc105b90 0x7f5bbc1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.283+0000 7f5bc1a00640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bbc1051d0 0x7f5bbc19a770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.283+0000 7f5bc1a00640 1 --2- 172.21.15.86:0/1563312391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bbc103fe0 0x7f5bbc19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.283+0000 7f5bc1a00640 1 -- 172.21.15.86:0/1563312391 >> 172.21.15.86:0/1563312391 conn(0x7f5bbc0ff570 msgr2=0x7f5bbc1085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.284+0000 7f5bc1a00640 1 -- 172.21.15.86:0/1563312391 shutdown_connections 2024-04-17T07:12:34.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.284+0000 7f5bc1a00640 1 -- 172.21.15.86:0/1563312391 wait complete. 2024-04-17T07:12:34.304 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-17T07:12:33.058088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639036,"num_objects":86,"num_object_clones":0,"num_object_copies":259,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":152,"num_read_kb":170,"num_write":378,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181358,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365404,"kb_used_data":12076,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576420,"statfs":{"total":1151940427776,"available":1151566254080,"internally_reserved":0,"allocated":12365824,"data_stored":7783091,"data_compressed":347595,"data_compressed_allocated":471040,"data_compressed_original":2441216,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44028,"num_objects":62,"num_object_clones":0,"num_object_copies":187,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":26,"num_read_kb":61,"num_write":148,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002844"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.738061+0000","last_change":"2024-04-17T07:11:55.486746+0000","last_active":"2024-04-17T07:12:27.738061+0000","last_peered":"2024-04-17T07:12:27.738061+0000","last_clean":"2024-04-17T07:12:27.738061+0000","last_became_active":"2024-04-17T07:11:55.486335+0000","last_became_peered":"2024-04-17T07:11:55.486335+0000","last_unstale":"2024-04-17T07:12:27.738061+0000","last_undegraded":"2024-04-17T07:12:27.738061+0000","last_fullsized":"2024-04-17T07:12:27.738061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:48.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735866+0000","last_change":"2024-04-17T07:11:55.481832+0000","last_active":"2024-04-17T07:12:28.735866+0000","last_peered":"2024-04-17T07:12:28.735866+0000","last_clean":"2024-04-17T07:12:28.735866+0000","last_became_active":"2024-04-17T07:11:55.476428+0000","last_became_peered":"2024-04-17T07:11:55.476428+0000","last_unstale":"2024-04-17T07:12:28.735866+0000","last_undegraded":"2024-04-17T07:12:28.735866+0000","last_fullsized":"2024-04-17T07:12:28.735866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:04.452542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736227+0000","last_change":"2024-04-17T07:11:55.483721+0000","last_active":"2024-04-17T07:12:28.736227+0000","last_peered":"2024-04-17T07:12:28.736227+0000","last_clean":"2024-04-17T07:12:28.736227+0000","last_became_active":"2024-04-17T07:11:55.483268+0000","last_became_peered":"2024-04-17T07:11:55.483268+0000","last_unstale":"2024-04-17T07:12:28.736227+0000","last_undegraded":"2024-04-17T07:12:28.736227+0000","last_fullsized":"2024-04-17T07:12:28.736227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:41.584783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714249+0000","last_change":"2024-04-17T07:11:55.485114+0000","last_active":"2024-04-17T07:12:28.714249+0000","last_peered":"2024-04-17T07:12:28.714249+0000","last_clean":"2024-04-17T07:12:28.714249+0000","last_became_active":"2024-04-17T07:11:55.483913+0000","last_became_peered":"2024-04-17T07:11:55.483913+0000","last_unstale":"2024-04-17T07:12:28.714249+0000","last_undegraded":"2024-04-17T07:12:28.714249+0000","last_fullsized":"2024-04-17T07:12:28.714249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:46.211121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726512+0000","last_change":"2024-04-17T07:11:55.491952+0000","last_active":"2024-04-17T07:12:28.726512+0000","last_peered":"2024-04-17T07:12:28.726512+0000","last_clean":"2024-04-17T07:12:28.726512+0000","last_became_active":"2024-04-17T07:11:55.491238+0000","last_became_peered":"2024-04-17T07:11:55.491238+0000","last_unstale":"2024-04-17T07:12:28.726512+0000","last_undegraded":"2024-04-17T07:12:28.726512+0000","last_fullsized":"2024-04-17T07:12:28.726512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:06.759797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725020+0000","last_change":"2024-04-17T07:11:55.471124+0000","last_active":"2024-04-17T07:12:28.725020+0000","last_peered":"2024-04-17T07:12:28.725020+0000","last_clean":"2024-04-17T07:12:28.725020+0000","last_became_active":"2024-04-17T07:11:55.470781+0000","last_became_peered":"2024-04-17T07:11:55.470781+0000","last_unstale":"2024-04-17T07:12:28.725020+0000","last_undegraded":"2024-04-17T07:12:28.725020+0000","last_fullsized":"2024-04-17T07:12:28.725020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:45.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720008+0000","last_change":"2024-04-17T07:11:55.469099+0000","last_active":"2024-04-17T07:12:28.720008+0000","last_peered":"2024-04-17T07:12:28.720008+0000","last_clean":"2024-04-17T07:12:28.720008+0000","last_became_active":"2024-04-17T07:11:55.468584+0000","last_became_peered":"2024-04-17T07:11:55.468584+0000","last_unstale":"2024-04-17T07:12:28.720008+0000","last_undegraded":"2024-04-17T07:12:28.720008+0000","last_fullsized":"2024-04-17T07:12:28.720008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:52.834846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122568+0000","last_change":"2024-04-17T07:11:55.473794+0000","last_active":"2024-04-17T07:12:29.122568+0000","last_peered":"2024-04-17T07:12:29.122568+0000","last_clean":"2024-04-17T07:12:29.122568+0000","last_became_active":"2024-04-17T07:11:55.471919+0000","last_became_peered":"2024-04-17T07:11:55.471919+0000","last_unstale":"2024-04-17T07:12:29.122568+0000","last_undegraded":"2024-04-17T07:12:29.122568+0000","last_fullsized":"2024-04-17T07:12:29.122568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:16.085117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726090+0000","last_change":"2024-04-17T07:11:55.482445+0000","last_active":"2024-04-17T07:12:28.726090+0000","last_peered":"2024-04-17T07:12:28.726090+0000","last_clean":"2024-04-17T07:12:28.726090+0000","last_became_active":"2024-04-17T07:11:55.481552+0000","last_became_peered":"2024-04-17T07:11:55.481552+0000","last_unstale":"2024-04-17T07:12:28.726090+0000","last_undegraded":"2024-04-17T07:12:28.726090+0000","last_fullsized":"2024-04-17T07:12:28.726090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:48.733585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725405+0000","last_change":"2024-04-17T07:11:55.476726+0000","last_active":"2024-04-17T07:12:28.725405+0000","last_peered":"2024-04-17T07:12:28.725405+0000","last_clean":"2024-04-17T07:12:28.725405+0000","last_became_active":"2024-04-17T07:11:55.476320+0000","last_became_peered":"2024-04-17T07:11:55.476320+0000","last_unstale":"2024-04-17T07:12:28.725405+0000","last_undegraded":"2024-04-17T07:12:28.725405+0000","last_fullsized":"2024-04-17T07:12:28.725405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.230181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721027+0000","last_change":"2024-04-17T07:11:55.485798+0000","last_active":"2024-04-17T07:12:28.721027+0000","last_peered":"2024-04-17T07:12:28.721027+0000","last_clean":"2024-04-17T07:12:28.721027+0000","last_became_active":"2024-04-17T07:11:55.484015+0000","last_became_peered":"2024-04-17T07:11:55.484015+0000","last_unstale":"2024-04-17T07:12:28.721027+0000","last_undegraded":"2024-04-17T07:12:28.721027+0000","last_fullsized":"2024-04-17T07:12:28.721027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:37.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.731078+0000","last_change":"2024-04-17T07:12:28.729834+0000","last_active":"2024-04-17T07:12:28.731078+0000","last_peered":"2024-04-17T07:12:28.731078+0000","last_clean":"2024-04-17T07:12:28.731078+0000","last_became_active":"2024-04-17T07:12:28.728694+0000","last_became_peered":"2024-04-17T07:12:28.728694+0000","last_unstale":"2024-04-17T07:12:28.731078+0000","last_undegraded":"2024-04-17T07:12:28.731078+0000","last_fullsized":"2024-04-17T07:12:28.731078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:46.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720070+0000","last_change":"2024-04-17T07:11:55.450379+0000","last_active":"2024-04-17T07:12:28.720070+0000","last_peered":"2024-04-17T07:12:28.720070+0000","last_clean":"2024-04-17T07:12:28.720070+0000","last_became_active":"2024-04-17T07:11:55.449118+0000","last_became_peered":"2024-04-17T07:11:55.449118+0000","last_unstale":"2024-04-17T07:12:28.720070+0000","last_undegraded":"2024-04-17T07:12:28.720070+0000","last_fullsized":"2024-04-17T07:12:28.720070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:07:34.248044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735022+0000","last_change":"2024-04-17T07:11:55.473839+0000","last_active":"2024-04-17T07:12:28.735022+0000","last_peered":"2024-04-17T07:12:28.735022+0000","last_clean":"2024-04-17T07:12:28.735022+0000","last_became_active":"2024-04-17T07:11:55.473472+0000","last_became_peered":"2024-04-17T07:11:55.473472+0000","last_unstale":"2024-04-17T07:12:28.735022+0000","last_undegraded":"2024-04-17T07:12:28.735022+0000","last_fullsized":"2024-04-17T07:12:28.735022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:02.263272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736022+0000","last_change":"2024-04-17T07:11:55.484820+0000","last_active":"2024-04-17T07:12:28.736022+0000","last_peered":"2024-04-17T07:12:28.736022+0000","last_clean":"2024-04-17T07:12:28.736022+0000","last_became_active":"2024-04-17T07:11:55.484314+0000","last_became_peered":"2024-04-17T07:11:55.484314+0000","last_unstale":"2024-04-17T07:12:28.736022+0000","last_undegraded":"2024-04-17T07:12:28.736022+0000","last_fullsized":"2024-04-17T07:12:28.736022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.186912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725577+0000","last_change":"2024-04-17T07:11:55.492018+0000","last_active":"2024-04-17T07:12:28.725577+0000","last_peered":"2024-04-17T07:12:28.725577+0000","last_clean":"2024-04-17T07:12:28.725577+0000","last_became_active":"2024-04-17T07:11:55.490512+0000","last_became_peered":"2024-04-17T07:11:55.490512+0000","last_unstale":"2024-04-17T07:12:28.725577+0000","last_undegraded":"2024-04-17T07:12:28.725577+0000","last_fullsized":"2024-04-17T07:12:28.725577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:45.436270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.737302+0000","last_change":"2024-04-17T07:11:55.487210+0000","last_active":"2024-04-17T07:12:27.737302+0000","last_peered":"2024-04-17T07:12:27.737302+0000","last_clean":"2024-04-17T07:12:27.737302+0000","last_became_active":"2024-04-17T07:11:55.485713+0000","last_became_peered":"2024-04-17T07:11:55.485713+0000","last_unstale":"2024-04-17T07:12:27.737302+0000","last_undegraded":"2024-04-17T07:12:27.737302+0000","last_fullsized":"2024-04-17T07:12:27.737302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:54.495950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716204+0000","last_change":"2024-04-17T07:11:55.482705+0000","last_active":"2024-04-17T07:12:28.716204+0000","last_peered":"2024-04-17T07:12:28.716204+0000","last_clean":"2024-04-17T07:12:28.716204+0000","last_became_active":"2024-04-17T07:11:55.482349+0000","last_became_peered":"2024-04-17T07:11:55.482349+0000","last_unstale":"2024-04-17T07:12:28.716204+0000","last_undegraded":"2024-04-17T07:12:28.716204+0000","last_fullsized":"2024-04-17T07:12:28.716204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:57.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716213+0000","last_change":"2024-04-17T07:11:55.467065+0000","last_active":"2024-04-17T07:12:28.716213+0000","last_peered":"2024-04-17T07:12:28.716213+0000","last_clean":"2024-04-17T07:12:28.716213+0000","last_became_active":"2024-04-17T07:11:55.466535+0000","last_became_peered":"2024-04-17T07:11:55.466535+0000","last_unstale":"2024-04-17T07:12:28.716213+0000","last_undegraded":"2024-04-17T07:12:28.716213+0000","last_fullsized":"2024-04-17T07:12:28.716213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:49.955453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715012+0000","last_change":"2024-04-17T07:11:59.515778+0000","last_active":"2024-04-17T07:12:28.715012+0000","last_peered":"2024-04-17T07:12:28.715012+0000","last_clean":"2024-04-17T07:12:28.715012+0000","last_became_active":"2024-04-17T07:11:59.515266+0000","last_became_peered":"2024-04-17T07:11:59.515266+0000","last_unstale":"2024-04-17T07:12:28.715012+0000","last_undegraded":"2024-04-17T07:12:28.715012+0000","last_fullsized":"2024-04-17T07:12:28.715012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:43.084724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.219364+0000","last_change":"2024-04-17T07:11:54.467060+0000","last_active":"2024-04-17T07:12:31.219364+0000","last_peered":"2024-04-17T07:12:31.219364+0000","last_clean":"2024-04-17T07:12:31.219364+0000","last_became_active":"2024-04-17T07:11:54.466687+0000","last_became_peered":"2024-04-17T07:11:54.466687+0000","last_unstale":"2024-04-17T07:12:31.219364+0000","last_undegraded":"2024-04-17T07:12:31.219364+0000","last_fullsized":"2024-04-17T07:12:31.219364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:57.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123283+0000","last_change":"2024-04-17T07:11:55.469857+0000","last_active":"2024-04-17T07:12:29.123283+0000","last_peered":"2024-04-17T07:12:29.123283+0000","last_clean":"2024-04-17T07:12:29.123283+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:12:29.123283+0000","last_undegraded":"2024-04-17T07:12:29.123283+0000","last_fullsized":"2024-04-17T07:12:29.123283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:43.565627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"activating","last_fresh":"2024-04-17T07:12:28.723099+0000","last_change":"2024-04-17T07:12:28.714326+0000","last_active":"2024-04-17T07:12:27.714584+0000","last_peered":"2024-04-17T07:12:00.800296+0000","last_clean":"2024-04-17T07:12:00.800296+0000","last_became_active":"2024-04-17T07:11:59.512404+0000","last_became_peered":"2024-04-17T07:11:59.512404+0000","last_unstale":"2024-04-17T07:12:28.723099+0000","last_undegraded":"2024-04-17T07:12:28.723099+0000","last_fullsized":"2024-04-17T07:12:28.723099+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":["4(0)","3(3)","6(1)","8(2)","9(2)"],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123277+0000","last_change":"2024-04-17T07:11:54.464715+0000","last_active":"2024-04-17T07:12:29.123277+0000","last_peered":"2024-04-17T07:12:29.123277+0000","last_clean":"2024-04-17T07:12:29.123277+0000","last_became_active":"2024-04-17T07:11:54.463917+0000","last_became_peered":"2024-04-17T07:11:54.463917+0000","last_unstale":"2024-04-17T07:12:29.123277+0000","last_undegraded":"2024-04-17T07:12:29.123277+0000","last_fullsized":"2024-04-17T07:12:29.123277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:43.738471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736571+0000","last_change":"2024-04-17T07:11:55.482876+0000","last_active":"2024-04-17T07:12:28.736571+0000","last_peered":"2024-04-17T07:12:28.736571+0000","last_clean":"2024-04-17T07:12:28.736571+0000","last_became_active":"2024-04-17T07:11:55.482451+0000","last_became_peered":"2024-04-17T07:11:55.482451+0000","last_unstale":"2024-04-17T07:12:28.736571+0000","last_undegraded":"2024-04-17T07:12:28.736571+0000","last_fullsized":"2024-04-17T07:12:28.736571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:39.394090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713748+0000","last_change":"2024-04-17T07:11:59.522634+0000","last_active":"2024-04-17T07:12:28.713748+0000","last_peered":"2024-04-17T07:12:28.713748+0000","last_clean":"2024-04-17T07:12:28.713748+0000","last_became_active":"2024-04-17T07:11:59.520797+0000","last_became_peered":"2024-04-17T07:11:59.520797+0000","last_unstale":"2024-04-17T07:12:28.713748+0000","last_undegraded":"2024-04-17T07:12:28.713748+0000","last_fullsized":"2024-04-17T07:12:28.713748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:02.316515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-17T07:11:55.467756+0000","last_change":"2024-04-17T07:12:28.153200+0000","last_active":"2024-04-17T07:12:28.153200+0000","last_peered":"2024-04-17T07:11:55.467756+0000","last_clean":"2024-04-17T07:11:55.467756+0000","last_became_active":"2024-04-17T07:11:54.469320+0000","last_became_peered":"2024-04-17T07:11:54.469320+0000","last_unstale":"2024-04-17T07:11:55.467756+0000","last_undegraded":"2024-04-17T07:11:55.467756+0000","last_fullsized":"2024-04-17T07:11:55.467756+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726296+0000","last_change":"2024-04-17T07:11:55.485999+0000","last_active":"2024-04-17T07:12:28.726296+0000","last_peered":"2024-04-17T07:12:28.726296+0000","last_clean":"2024-04-17T07:12:28.726296+0000","last_became_active":"2024-04-17T07:11:55.484048+0000","last_became_peered":"2024-04-17T07:11:55.484048+0000","last_unstale":"2024-04-17T07:12:28.726296+0000","last_undegraded":"2024-04-17T07:12:28.726296+0000","last_fullsized":"2024-04-17T07:12:28.726296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:38.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713330+0000","last_change":"2024-04-17T07:11:59.529929+0000","last_active":"2024-04-17T07:12:28.713330+0000","last_peered":"2024-04-17T07:12:28.713330+0000","last_clean":"2024-04-17T07:12:28.713330+0000","last_became_active":"2024-04-17T07:11:59.529400+0000","last_became_peered":"2024-04-17T07:11:59.529400+0000","last_unstale":"2024-04-17T07:12:28.713330+0000","last_undegraded":"2024-04-17T07:12:28.713330+0000","last_fullsized":"2024-04-17T07:12:28.713330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:54.845529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.209278+0000","last_change":"2024-04-17T07:11:54.462852+0000","last_active":"2024-04-17T07:12:30.209278+0000","last_peered":"2024-04-17T07:12:30.209278+0000","last_clean":"2024-04-17T07:12:30.209278+0000","last_became_active":"2024-04-17T07:11:54.459178+0000","last_became_peered":"2024-04-17T07:11:54.459178+0000","last_unstale":"2024-04-17T07:12:30.209278+0000","last_undegraded":"2024-04-17T07:12:30.209278+0000","last_fullsized":"2024-04-17T07:12:30.209278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:07.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.737683+0000","last_change":"2024-04-17T07:11:55.480951+0000","last_active":"2024-04-17T07:12:28.737683+0000","last_peered":"2024-04-17T07:12:28.737683+0000","last_clean":"2024-04-17T07:12:28.737683+0000","last_became_active":"2024-04-17T07:11:55.480507+0000","last_became_peered":"2024-04-17T07:11:55.480507+0000","last_unstale":"2024-04-17T07:12:28.737683+0000","last_undegraded":"2024-04-17T07:12:28.737683+0000","last_fullsized":"2024-04-17T07:12:28.737683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:42.584180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715898+0000","last_change":"2024-04-17T07:11:59.520897+0000","last_active":"2024-04-17T07:12:28.715898+0000","last_peered":"2024-04-17T07:12:28.715898+0000","last_clean":"2024-04-17T07:12:28.715898+0000","last_became_active":"2024-04-17T07:11:59.520350+0000","last_became_peered":"2024-04-17T07:11:59.520350+0000","last_unstale":"2024-04-17T07:12:28.715898+0000","last_undegraded":"2024-04-17T07:12:28.715898+0000","last_fullsized":"2024-04-17T07:12:28.715898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:39.813097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.222928+0000","last_change":"2024-04-17T07:12:28.753717+0000","last_active":"2024-04-17T07:12:31.222928+0000","last_peered":"2024-04-17T07:12:31.222928+0000","last_clean":"2024-04-17T07:12:31.222928+0000","last_became_active":"2024-04-17T07:12:28.739556+0000","last_became_peered":"2024-04-17T07:12:28.739556+0000","last_unstale":"2024-04-17T07:12:31.222928+0000","last_undegraded":"2024-04-17T07:12:31.222928+0000","last_fullsized":"2024-04-17T07:12:31.222928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:11.967731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.732946+0000","last_change":"2024-04-17T07:11:55.476675+0000","last_active":"2024-04-17T07:12:28.732946+0000","last_peered":"2024-04-17T07:12:28.732946+0000","last_clean":"2024-04-17T07:12:28.732946+0000","last_became_active":"2024-04-17T07:11:55.476273+0000","last_became_peered":"2024-04-17T07:11:55.476273+0000","last_unstale":"2024-04-17T07:12:28.732946+0000","last_undegraded":"2024-04-17T07:12:28.732946+0000","last_fullsized":"2024-04-17T07:12:28.732946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:09.612138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.741537+0000","last_change":"2024-04-17T07:12:28.741537+0000","last_active":"2024-04-17T07:12:28.741537+0000","last_peered":"2024-04-17T07:12:28.741537+0000","last_clean":"2024-04-17T07:12:28.741537+0000","last_became_active":"2024-04-17T07:12:28.741035+0000","last_became_peered":"2024-04-17T07:12:28.741035+0000","last_unstale":"2024-04-17T07:12:28.741537+0000","last_undegraded":"2024-04-17T07:12:28.741537+0000","last_fullsized":"2024-04-17T07:12:28.741537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:15.550112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.195614+0000","last_change":"2024-04-17T07:11:54.471118+0000","last_active":"2024-04-17T07:12:29.195614+0000","last_peered":"2024-04-17T07:12:29.195614+0000","last_clean":"2024-04-17T07:12:29.195614+0000","last_became_active":"2024-04-17T07:11:54.470526+0000","last_became_peered":"2024-04-17T07:11:54.470526+0000","last_unstale":"2024-04-17T07:12:29.195614+0000","last_undegraded":"2024-04-17T07:12:29.195614+0000","last_fullsized":"2024-04-17T07:12:29.195614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:52:47.918086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735651+0000","last_change":"2024-04-17T07:11:55.472850+0000","last_active":"2024-04-17T07:12:28.735651+0000","last_peered":"2024-04-17T07:12:28.735651+0000","last_clean":"2024-04-17T07:12:28.735651+0000","last_became_active":"2024-04-17T07:11:55.472097+0000","last_became_peered":"2024-04-17T07:11:55.472097+0000","last_unstale":"2024-04-17T07:12:28.735651+0000","last_undegraded":"2024-04-17T07:12:28.735651+0000","last_fullsized":"2024-04-17T07:12:28.735651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:41.730809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.737174+0000","last_change":"2024-04-17T07:11:59.522960+0000","last_active":"2024-04-17T07:12:27.737174+0000","last_peered":"2024-04-17T07:12:27.737174+0000","last_clean":"2024-04-17T07:12:27.737174+0000","last_became_active":"2024-04-17T07:11:59.522470+0000","last_became_peered":"2024-04-17T07:11:59.522470+0000","last_unstale":"2024-04-17T07:12:27.737174+0000","last_undegraded":"2024-04-17T07:12:27.737174+0000","last_fullsized":"2024-04-17T07:12:27.737174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:36.620442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720922+0000","last_change":"2024-04-17T07:11:54.470693+0000","last_active":"2024-04-17T07:12:28.720922+0000","last_peered":"2024-04-17T07:12:28.720922+0000","last_clean":"2024-04-17T07:12:28.720922+0000","last_became_active":"2024-04-17T07:11:54.470334+0000","last_became_peered":"2024-04-17T07:11:54.470334+0000","last_unstale":"2024-04-17T07:12:28.720922+0000","last_undegraded":"2024-04-17T07:12:28.720922+0000","last_fullsized":"2024-04-17T07:12:28.720922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:19:04.980786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122063+0000","last_change":"2024-04-17T07:11:55.474755+0000","last_active":"2024-04-17T07:12:29.122063+0000","last_peered":"2024-04-17T07:12:29.122063+0000","last_clean":"2024-04-17T07:12:29.122063+0000","last_became_active":"2024-04-17T07:11:55.474261+0000","last_became_peered":"2024-04-17T07:11:55.474261+0000","last_unstale":"2024-04-17T07:12:29.122063+0000","last_undegraded":"2024-04-17T07:12:29.122063+0000","last_fullsized":"2024-04-17T07:12:29.122063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:00.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735779+0000","last_change":"2024-04-17T07:11:59.517152+0000","last_active":"2024-04-17T07:12:28.735779+0000","last_peered":"2024-04-17T07:12:28.735779+0000","last_clean":"2024-04-17T07:12:28.735779+0000","last_became_active":"2024-04-17T07:11:59.514977+0000","last_became_peered":"2024-04-17T07:11:59.514977+0000","last_unstale":"2024-04-17T07:12:28.735779+0000","last_undegraded":"2024-04-17T07:12:28.735779+0000","last_fullsized":"2024-04-17T07:12:28.735779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:14.974136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.212620+0000","last_change":"2024-04-17T07:11:54.459786+0000","last_active":"2024-04-17T07:12:30.212620+0000","last_peered":"2024-04-17T07:12:30.212620+0000","last_clean":"2024-04-17T07:12:30.212620+0000","last_became_active":"2024-04-17T07:11:54.457589+0000","last_became_peered":"2024-04-17T07:11:54.457589+0000","last_unstale":"2024-04-17T07:12:30.212620+0000","last_undegraded":"2024-04-17T07:12:30.212620+0000","last_fullsized":"2024-04-17T07:12:30.212620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:25.986539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714421+0000","last_change":"2024-04-17T07:11:55.465020+0000","last_active":"2024-04-17T07:12:28.714421+0000","last_peered":"2024-04-17T07:12:28.714421+0000","last_clean":"2024-04-17T07:12:28.714421+0000","last_became_active":"2024-04-17T07:11:55.464608+0000","last_became_peered":"2024-04-17T07:11:55.464608+0000","last_unstale":"2024-04-17T07:12:28.714421+0000","last_undegraded":"2024-04-17T07:12:28.714421+0000","last_fullsized":"2024-04-17T07:12:28.714421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122725+0000","last_change":"2024-04-17T07:11:59.519333+0000","last_active":"2024-04-17T07:12:29.122725+0000","last_peered":"2024-04-17T07:12:29.122725+0000","last_clean":"2024-04-17T07:12:29.122725+0000","last_became_active":"2024-04-17T07:11:59.518865+0000","last_became_peered":"2024-04-17T07:11:59.518865+0000","last_unstale":"2024-04-17T07:12:29.122725+0000","last_undegraded":"2024-04-17T07:12:29.122725+0000","last_fullsized":"2024-04-17T07:12:29.122725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:42.009010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.227304+0000","last_change":"2024-04-17T07:11:54.462283+0000","last_active":"2024-04-17T07:12:32.227304+0000","last_peered":"2024-04-17T07:12:32.227304+0000","last_clean":"2024-04-17T07:12:32.227304+0000","last_became_active":"2024-04-17T07:11:54.461818+0000","last_became_peered":"2024-04-17T07:11:54.461818+0000","last_unstale":"2024-04-17T07:12:32.227304+0000","last_undegraded":"2024-04-17T07:12:32.227304+0000","last_fullsized":"2024-04-17T07:12:32.227304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.765058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726968+0000","last_change":"2024-04-17T07:11:55.486491+0000","last_active":"2024-04-17T07:12:28.726968+0000","last_peered":"2024-04-17T07:12:28.726968+0000","last_clean":"2024-04-17T07:12:28.726968+0000","last_became_active":"2024-04-17T07:11:55.486121+0000","last_became_peered":"2024-04-17T07:11:55.486121+0000","last_unstale":"2024-04-17T07:12:28.726968+0000","last_undegraded":"2024-04-17T07:12:28.726968+0000","last_fullsized":"2024-04-17T07:12:28.726968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:12.998321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713871+0000","last_change":"2024-04-17T07:11:59.511555+0000","last_active":"2024-04-17T07:12:28.713871+0000","last_peered":"2024-04-17T07:12:28.713871+0000","last_clean":"2024-04-17T07:12:28.713871+0000","last_became_active":"2024-04-17T07:11:59.511147+0000","last_became_peered":"2024-04-17T07:11:59.511147+0000","last_unstale":"2024-04-17T07:12:28.713871+0000","last_undegraded":"2024-04-17T07:12:28.713871+0000","last_fullsized":"2024-04-17T07:12:28.713871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:41.854815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.210000+0000","last_change":"2024-04-17T07:12:28.734476+0000","last_active":"2024-04-17T07:12:30.210000+0000","last_peered":"2024-04-17T07:12:30.210000+0000","last_clean":"2024-04-17T07:12:30.210000+0000","last_became_active":"2024-04-17T07:12:28.728187+0000","last_became_peered":"2024-04-17T07:12:28.728187+0000","last_unstale":"2024-04-17T07:12:30.210000+0000","last_undegraded":"2024-04-17T07:12:30.210000+0000","last_fullsized":"2024-04-17T07:12:30.210000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:45.749607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736562+0000","last_change":"2024-04-17T07:11:55.480589+0000","last_active":"2024-04-17T07:12:28.736562+0000","last_peered":"2024-04-17T07:12:28.736562+0000","last_clean":"2024-04-17T07:12:28.736562+0000","last_became_active":"2024-04-17T07:11:55.480224+0000","last_became_peered":"2024-04-17T07:11:55.480224+0000","last_unstale":"2024-04-17T07:12:28.736562+0000","last_undegraded":"2024-04-17T07:12:28.736562+0000","last_fullsized":"2024-04-17T07:12:28.736562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:05.892442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713815+0000","last_change":"2024-04-17T07:11:59.522021+0000","last_active":"2024-04-17T07:12:28.713815+0000","last_peered":"2024-04-17T07:12:28.713815+0000","last_clean":"2024-04-17T07:12:28.713815+0000","last_became_active":"2024-04-17T07:11:59.521493+0000","last_became_peered":"2024-04-17T07:11:59.521493+0000","last_unstale":"2024-04-17T07:12:28.713815+0000","last_undegraded":"2024-04-17T07:12:28.713815+0000","last_fullsized":"2024-04-17T07:12:28.713815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:20.386216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.195512+0000","last_change":"2024-04-17T07:11:54.468225+0000","last_active":"2024-04-17T07:12:29.195512+0000","last_peered":"2024-04-17T07:12:29.195512+0000","last_clean":"2024-04-17T07:12:29.195512+0000","last_became_active":"2024-04-17T07:11:54.467046+0000","last_became_peered":"2024-04-17T07:11:54.467046+0000","last_unstale":"2024-04-17T07:12:29.195512+0000","last_undegraded":"2024-04-17T07:12:29.195512+0000","last_fullsized":"2024-04-17T07:12:29.195512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:17.725255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726428+0000","last_change":"2024-04-17T07:11:55.488980+0000","last_active":"2024-04-17T07:12:28.726428+0000","last_peered":"2024-04-17T07:12:28.726428+0000","last_clean":"2024-04-17T07:12:28.726428+0000","last_became_active":"2024-04-17T07:11:55.488226+0000","last_became_peered":"2024-04-17T07:11:55.488226+0000","last_unstale":"2024-04-17T07:12:28.726428+0000","last_undegraded":"2024-04-17T07:12:28.726428+0000","last_fullsized":"2024-04-17T07:12:28.726428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:00.135973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714388+0000","last_change":"2024-04-17T07:11:59.515134+0000","last_active":"2024-04-17T07:12:28.714388+0000","last_peered":"2024-04-17T07:12:28.714388+0000","last_clean":"2024-04-17T07:12:28.714388+0000","last_became_active":"2024-04-17T07:11:59.514560+0000","last_became_peered":"2024-04-17T07:11:59.514560+0000","last_unstale":"2024-04-17T07:12:28.714388+0000","last_undegraded":"2024-04-17T07:12:28.714388+0000","last_fullsized":"2024-04-17T07:12:28.714388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:34.669789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'26","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.230037+0000","last_change":"2024-04-17T07:11:54.461226+0000","last_active":"2024-04-17T07:12:32.230037+0000","last_peered":"2024-04-17T07:12:32.230037+0000","last_clean":"2024-04-17T07:12:32.230037+0000","last_became_active":"2024-04-17T07:11:54.460573+0000","last_became_peered":"2024-04-17T07:11:54.460573+0000","last_unstale":"2024-04-17T07:12:32.230037+0000","last_undegraded":"2024-04-17T07:12:32.230037+0000","last_fullsized":"2024-04-17T07:12:32.230037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":44,"num_write":28,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.734828+0000","last_change":"2024-04-17T07:11:55.483925+0000","last_active":"2024-04-17T07:12:28.734828+0000","last_peered":"2024-04-17T07:12:28.734828+0000","last_clean":"2024-04-17T07:12:28.734828+0000","last_became_active":"2024-04-17T07:11:55.483584+0000","last_became_peered":"2024-04-17T07:11:55.483584+0000","last_unstale":"2024-04-17T07:12:28.734828+0000","last_undegraded":"2024-04-17T07:12:28.734828+0000","last_fullsized":"2024-04-17T07:12:28.734828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:12.328876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.737367+0000","last_change":"2024-04-17T07:11:59.515597+0000","last_active":"2024-04-17T07:12:28.737367+0000","last_peered":"2024-04-17T07:12:28.737367+0000","last_clean":"2024-04-17T07:12:28.737367+0000","last_became_active":"2024-04-17T07:11:59.514413+0000","last_became_peered":"2024-04-17T07:11:59.514413+0000","last_unstale":"2024-04-17T07:12:28.737367+0000","last_undegraded":"2024-04-17T07:12:28.737367+0000","last_fullsized":"2024-04-17T07:12:28.737367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:45.824127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.209717+0000","last_change":"2024-04-17T07:11:54.449984+0000","last_active":"2024-04-17T07:12:30.209717+0000","last_peered":"2024-04-17T07:12:30.209717+0000","last_clean":"2024-04-17T07:12:30.209717+0000","last_became_active":"2024-04-17T07:11:54.449628+0000","last_became_peered":"2024-04-17T07:11:54.449628+0000","last_unstale":"2024-04-17T07:12:30.209717+0000","last_undegraded":"2024-04-17T07:12:30.209717+0000","last_fullsized":"2024-04-17T07:12:30.209717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:26:03.281684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719923+0000","last_change":"2024-04-17T07:11:55.476912+0000","last_active":"2024-04-17T07:12:28.719923+0000","last_peered":"2024-04-17T07:12:28.719923+0000","last_clean":"2024-04-17T07:12:28.719923+0000","last_became_active":"2024-04-17T07:11:55.471576+0000","last_became_peered":"2024-04-17T07:11:55.471576+0000","last_unstale":"2024-04-17T07:12:28.719923+0000","last_undegraded":"2024-04-17T07:12:28.719923+0000","last_fullsized":"2024-04-17T07:12:28.719923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:35.439271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.733057+0000","last_change":"2024-04-17T07:11:59.515897+0000","last_active":"2024-04-17T07:12:27.733057+0000","last_peered":"2024-04-17T07:12:27.733057+0000","last_clean":"2024-04-17T07:12:27.733057+0000","last_became_active":"2024-04-17T07:11:59.515349+0000","last_became_peered":"2024-04-17T07:11:59.515349+0000","last_unstale":"2024-04-17T07:12:27.733057+0000","last_undegraded":"2024-04-17T07:12:27.733057+0000","last_fullsized":"2024-04-17T07:12:27.733057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:00.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214859+0000","last_change":"2024-04-17T07:11:54.462369+0000","last_active":"2024-04-17T07:12:30.214859+0000","last_peered":"2024-04-17T07:12:30.214859+0000","last_clean":"2024-04-17T07:12:30.214859+0000","last_became_active":"2024-04-17T07:11:54.460357+0000","last_became_peered":"2024-04-17T07:11:54.460357+0000","last_unstale":"2024-04-17T07:12:30.214859+0000","last_undegraded":"2024-04-17T07:12:30.214859+0000","last_fullsized":"2024-04-17T07:12:30.214859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:38.029980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720650+0000","last_change":"2024-04-17T07:11:55.475874+0000","last_active":"2024-04-17T07:12:28.720650+0000","last_peered":"2024-04-17T07:12:28.720650+0000","last_clean":"2024-04-17T07:12:28.720650+0000","last_became_active":"2024-04-17T07:11:55.475550+0000","last_became_peered":"2024-04-17T07:11:55.475550+0000","last_unstale":"2024-04-17T07:12:28.720650+0000","last_undegraded":"2024-04-17T07:12:28.720650+0000","last_fullsized":"2024-04-17T07:12:28.720650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:40.096573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714893+0000","last_change":"2024-04-17T07:11:59.514661+0000","last_active":"2024-04-17T07:12:28.714893+0000","last_peered":"2024-04-17T07:12:28.714893+0000","last_clean":"2024-04-17T07:12:28.714893+0000","last_became_active":"2024-04-17T07:11:59.513030+0000","last_became_peered":"2024-04-17T07:11:59.513030+0000","last_unstale":"2024-04-17T07:12:28.714893+0000","last_undegraded":"2024-04-17T07:12:28.714893+0000","last_fullsized":"2024-04-17T07:12:28.714893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:35.696961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713262+0000","last_change":"2024-04-17T07:11:54.455279+0000","last_active":"2024-04-17T07:12:28.713262+0000","last_peered":"2024-04-17T07:12:28.713262+0000","last_clean":"2024-04-17T07:12:28.713262+0000","last_became_active":"2024-04-17T07:11:54.453620+0000","last_became_peered":"2024-04-17T07:11:54.453620+0000","last_unstale":"2024-04-17T07:12:28.713262+0000","last_undegraded":"2024-04-17T07:12:28.713262+0000","last_fullsized":"2024-04-17T07:12:28.713262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:15.227410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719686+0000","last_change":"2024-04-17T07:11:55.469579+0000","last_active":"2024-04-17T07:12:28.719686+0000","last_peered":"2024-04-17T07:12:28.719686+0000","last_clean":"2024-04-17T07:12:28.719686+0000","last_became_active":"2024-04-17T07:11:55.468389+0000","last_became_peered":"2024-04-17T07:11:55.468389+0000","last_unstale":"2024-04-17T07:12:28.719686+0000","last_undegraded":"2024-04-17T07:12:28.719686+0000","last_fullsized":"2024-04-17T07:12:28.719686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:01.274677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720385+0000","last_change":"2024-04-17T07:11:59.512110+0000","last_active":"2024-04-17T07:12:28.720385+0000","last_peered":"2024-04-17T07:12:28.720385+0000","last_clean":"2024-04-17T07:12:28.720385+0000","last_became_active":"2024-04-17T07:11:59.510573+0000","last_became_peered":"2024-04-17T07:11:59.510573+0000","last_unstale":"2024-04-17T07:12:28.720385+0000","last_undegraded":"2024-04-17T07:12:28.720385+0000","last_fullsized":"2024-04-17T07:12:28.720385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:32.515687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'10","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-17T07:12:00.800050+0000","last_change":"2024-04-17T07:12:28.153142+0000","last_active":"2024-04-17T07:12:28.153142+0000","last_peered":"2024-04-17T07:12:00.800050+0000","last_clean":"2024-04-17T07:12:00.800050+0000","last_became_active":"2024-04-17T07:11:54.471537+0000","last_became_peered":"2024-04-17T07:11:54.471537+0000","last_unstale":"2024-04-17T07:12:00.800050+0000","last_undegraded":"2024-04-17T07:12:00.800050+0000","last_fullsized":"2024-04-17T07:12:00.800050+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.731088+0000","last_change":"2024-04-17T07:11:55.473540+0000","last_active":"2024-04-17T07:12:28.731088+0000","last_peered":"2024-04-17T07:12:28.731088+0000","last_clean":"2024-04-17T07:12:28.731088+0000","last_became_active":"2024-04-17T07:11:55.472436+0000","last_became_peered":"2024-04-17T07:11:55.472436+0000","last_unstale":"2024-04-17T07:12:28.731088+0000","last_undegraded":"2024-04-17T07:12:28.731088+0000","last_fullsized":"2024-04-17T07:12:28.731088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:54.317052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730908+0000","last_change":"2024-04-17T07:11:59.521305+0000","last_active":"2024-04-17T07:12:28.730908+0000","last_peered":"2024-04-17T07:12:28.730908+0000","last_clean":"2024-04-17T07:12:28.730908+0000","last_became_active":"2024-04-17T07:11:59.516860+0000","last_became_peered":"2024-04-17T07:11:59.516860+0000","last_unstale":"2024-04-17T07:12:28.730908+0000","last_undegraded":"2024-04-17T07:12:28.730908+0000","last_fullsized":"2024-04-17T07:12:28.730908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:23.403218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735984+0000","last_change":"2024-04-17T07:11:54.463378+0000","last_active":"2024-04-17T07:12:28.735984+0000","last_peered":"2024-04-17T07:12:28.735984+0000","last_clean":"2024-04-17T07:12:28.735984+0000","last_became_active":"2024-04-17T07:11:54.462120+0000","last_became_peered":"2024-04-17T07:11:54.462120+0000","last_unstale":"2024-04-17T07:12:28.735984+0000","last_undegraded":"2024-04-17T07:12:28.735984+0000","last_fullsized":"2024-04-17T07:12:28.735984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:42.109928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.737028+0000","last_change":"2024-04-17T07:11:55.478604+0000","last_active":"2024-04-17T07:12:27.737028+0000","last_peered":"2024-04-17T07:12:27.737028+0000","last_clean":"2024-04-17T07:12:27.737028+0000","last_became_active":"2024-04-17T07:11:55.478252+0000","last_became_peered":"2024-04-17T07:11:55.478252+0000","last_unstale":"2024-04-17T07:12:27.737028+0000","last_undegraded":"2024-04-17T07:12:27.737028+0000","last_fullsized":"2024-04-17T07:12:27.737028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:08.365896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727344+0000","last_change":"2024-04-17T07:11:59.518619+0000","last_active":"2024-04-17T07:12:28.727344+0000","last_peered":"2024-04-17T07:12:28.727344+0000","last_clean":"2024-04-17T07:12:28.727344+0000","last_became_active":"2024-04-17T07:11:59.518214+0000","last_became_peered":"2024-04-17T07:11:59.518214+0000","last_unstale":"2024-04-17T07:12:28.727344+0000","last_undegraded":"2024-04-17T07:12:28.727344+0000","last_fullsized":"2024-04-17T07:12:28.727344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:46.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715522+0000","last_change":"2024-04-17T07:11:54.453038+0000","last_active":"2024-04-17T07:12:28.715522+0000","last_peered":"2024-04-17T07:12:28.715522+0000","last_clean":"2024-04-17T07:12:28.715522+0000","last_became_active":"2024-04-17T07:11:54.452619+0000","last_became_peered":"2024-04-17T07:11:54.452619+0000","last_unstale":"2024-04-17T07:12:28.715522+0000","last_undegraded":"2024-04-17T07:12:28.715522+0000","last_fullsized":"2024-04-17T07:12:28.715522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:25.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.724948+0000","last_change":"2024-04-17T07:11:55.465990+0000","last_active":"2024-04-17T07:12:28.724948+0000","last_peered":"2024-04-17T07:12:28.724948+0000","last_clean":"2024-04-17T07:12:28.724948+0000","last_became_active":"2024-04-17T07:11:55.465523+0000","last_became_peered":"2024-04-17T07:11:55.465523+0000","last_unstale":"2024-04-17T07:12:28.724948+0000","last_undegraded":"2024-04-17T07:12:28.724948+0000","last_fullsized":"2024-04-17T07:12:28.724948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:40.366027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721070+0000","last_change":"2024-04-17T07:11:59.512119+0000","last_active":"2024-04-17T07:12:28.721070+0000","last_peered":"2024-04-17T07:12:28.721070+0000","last_clean":"2024-04-17T07:12:28.721070+0000","last_became_active":"2024-04-17T07:11:59.511460+0000","last_became_peered":"2024-04-17T07:11:59.511460+0000","last_unstale":"2024-04-17T07:12:28.721070+0000","last_undegraded":"2024-04-17T07:12:28.721070+0000","last_fullsized":"2024-04-17T07:12:28.721070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:50.888203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.221574+0000","last_change":"2024-04-17T07:11:54.469346+0000","last_active":"2024-04-17T07:12:31.221574+0000","last_peered":"2024-04-17T07:12:31.221574+0000","last_clean":"2024-04-17T07:12:31.221574+0000","last_became_active":"2024-04-17T07:11:54.468947+0000","last_became_peered":"2024-04-17T07:11:54.468947+0000","last_unstale":"2024-04-17T07:12:31.221574+0000","last_undegraded":"2024-04-17T07:12:31.221574+0000","last_fullsized":"2024-04-17T07:12:31.221574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:04.854625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.724510+0000","last_change":"2024-04-17T07:11:55.471235+0000","last_active":"2024-04-17T07:12:28.724510+0000","last_peered":"2024-04-17T07:12:28.724510+0000","last_clean":"2024-04-17T07:12:28.724510+0000","last_became_active":"2024-04-17T07:11:55.470857+0000","last_became_peered":"2024-04-17T07:11:55.470857+0000","last_unstale":"2024-04-17T07:12:28.724510+0000","last_undegraded":"2024-04-17T07:12:28.724510+0000","last_fullsized":"2024-04-17T07:12:28.724510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:21.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720514+0000","last_change":"2024-04-17T07:11:59.524519+0000","last_active":"2024-04-17T07:12:28.720514+0000","last_peered":"2024-04-17T07:12:28.720514+0000","last_clean":"2024-04-17T07:12:28.720514+0000","last_became_active":"2024-04-17T07:11:59.519270+0000","last_became_peered":"2024-04-17T07:11:59.519270+0000","last_unstale":"2024-04-17T07:12:28.720514+0000","last_undegraded":"2024-04-17T07:12:28.720514+0000","last_fullsized":"2024-04-17T07:12:28.720514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:41.080249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.734024+0000","last_change":"2024-04-17T07:11:55.473922+0000","last_active":"2024-04-17T07:12:28.734024+0000","last_peered":"2024-04-17T07:12:28.734024+0000","last_clean":"2024-04-17T07:12:28.734024+0000","last_became_active":"2024-04-17T07:11:55.468594+0000","last_became_peered":"2024-04-17T07:11:55.468594+0000","last_unstale":"2024-04-17T07:12:28.734024+0000","last_undegraded":"2024-04-17T07:12:28.734024+0000","last_fullsized":"2024-04-17T07:12:28.734024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:21.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.219511+0000","last_change":"2024-04-17T07:11:54.473066+0000","last_active":"2024-04-17T07:12:31.219511+0000","last_peered":"2024-04-17T07:12:31.219511+0000","last_clean":"2024-04-17T07:12:31.219511+0000","last_became_active":"2024-04-17T07:11:54.472708+0000","last_became_peered":"2024-04-17T07:11:54.472708+0000","last_unstale":"2024-04-17T07:12:31.219511+0000","last_undegraded":"2024-04-17T07:12:31.219511+0000","last_fullsized":"2024-04-17T07:12:31.219511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:27.614819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720813+0000","last_change":"2024-04-17T07:11:59.516939+0000","last_active":"2024-04-17T07:12:28.720813+0000","last_peered":"2024-04-17T07:12:28.720813+0000","last_clean":"2024-04-17T07:12:28.720813+0000","last_became_active":"2024-04-17T07:11:59.516333+0000","last_became_peered":"2024-04-17T07:11:59.516333+0000","last_unstale":"2024-04-17T07:12:28.720813+0000","last_undegraded":"2024-04-17T07:12:28.720813+0000","last_fullsized":"2024-04-17T07:12:28.720813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:15.648831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122635+0000","last_change":"2024-04-17T07:11:55.481006+0000","last_active":"2024-04-17T07:12:29.122635+0000","last_peered":"2024-04-17T07:12:29.122635+0000","last_clean":"2024-04-17T07:12:29.122635+0000","last_became_active":"2024-04-17T07:11:55.479751+0000","last_became_peered":"2024-04-17T07:11:55.479751+0000","last_unstale":"2024-04-17T07:12:29.122635+0000","last_undegraded":"2024-04-17T07:12:29.122635+0000","last_fullsized":"2024-04-17T07:12:29.122635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:13.004650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713874+0000","last_change":"2024-04-17T07:11:54.463106+0000","last_active":"2024-04-17T07:12:28.713874+0000","last_peered":"2024-04-17T07:12:28.713874+0000","last_clean":"2024-04-17T07:12:28.713874+0000","last_became_active":"2024-04-17T07:11:54.462691+0000","last_became_peered":"2024-04-17T07:11:54.462691+0000","last_unstale":"2024-04-17T07:12:28.713874+0000","last_undegraded":"2024-04-17T07:12:28.713874+0000","last_fullsized":"2024-04-17T07:12:28.713874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:34.302019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719632+0000","last_change":"2024-04-17T07:10:14.685883+0000","last_active":"2024-04-17T07:12:28.719632+0000","last_peered":"2024-04-17T07:12:28.719632+0000","last_clean":"2024-04-17T07:12:28.719632+0000","last_became_active":"2024-04-17T07:10:14.684551+0000","last_became_peered":"2024-04-17T07:10:14.684551+0000","last_unstale":"2024-04-17T07:12:28.719632+0000","last_undegraded":"2024-04-17T07:12:28.719632+0000","last_fullsized":"2024-04-17T07:12:28.719632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_clean_scrub_stamp":"2024-04-17T07:05:37.112317+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:04.265147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736804+0000","last_change":"2024-04-17T07:12:28.731807+0000","last_active":"2024-04-17T07:12:28.736804+0000","last_peered":"2024-04-17T07:12:28.736804+0000","last_clean":"2024-04-17T07:12:28.736804+0000","last_became_active":"2024-04-17T07:12:28.730656+0000","last_became_peered":"2024-04-17T07:12:28.730656+0000","last_unstale":"2024-04-17T07:12:28.736804+0000","last_undegraded":"2024-04-17T07:12:28.736804+0000","last_fullsized":"2024-04-17T07:12:28.736804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:58.023333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727980+0000","last_change":"2024-04-17T07:12:28.727980+0000","last_active":"2024-04-17T07:12:28.727980+0000","last_peered":"2024-04-17T07:12:28.727980+0000","last_clean":"2024-04-17T07:12:28.727980+0000","last_became_active":"2024-04-17T07:12:28.727060+0000","last_became_peered":"2024-04-17T07:12:28.727060+0000","last_unstale":"2024-04-17T07:12:28.727980+0000","last_undegraded":"2024-04-17T07:12:28.727980+0000","last_fullsized":"2024-04-17T07:12:28.727980+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:33.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719921+0000","last_change":"2024-04-17T07:11:54.463180+0000","last_active":"2024-04-17T07:12:28.719921+0000","last_peered":"2024-04-17T07:12:28.719921+0000","last_clean":"2024-04-17T07:12:28.719921+0000","last_became_active":"2024-04-17T07:11:54.462793+0000","last_became_peered":"2024-04-17T07:11:54.462793+0000","last_unstale":"2024-04-17T07:12:28.719921+0000","last_undegraded":"2024-04-17T07:12:28.719921+0000","last_fullsized":"2024-04-17T07:12:28.719921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:19.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.733730+0000","last_change":"2024-04-17T07:11:59.518355+0000","last_active":"2024-04-17T07:12:27.733730+0000","last_peered":"2024-04-17T07:12:27.733730+0000","last_clean":"2024-04-17T07:12:27.733730+0000","last_became_active":"2024-04-17T07:11:59.517619+0000","last_became_peered":"2024-04-17T07:11:59.517619+0000","last_unstale":"2024-04-17T07:12:27.733730+0000","last_undegraded":"2024-04-17T07:12:27.733730+0000","last_fullsized":"2024-04-17T07:12:27.733730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:36.826021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.196898+0000","last_change":"2024-04-17T07:11:54.465285+0000","last_active":"2024-04-17T07:12:29.196898+0000","last_peered":"2024-04-17T07:12:29.196898+0000","last_clean":"2024-04-17T07:12:29.196898+0000","last_became_active":"2024-04-17T07:11:54.464818+0000","last_became_peered":"2024-04-17T07:11:54.464818+0000","last_unstale":"2024-04-17T07:12:29.196898+0000","last_undegraded":"2024-04-17T07:12:29.196898+0000","last_fullsized":"2024-04-17T07:12:29.196898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721576+0000","last_change":"2024-04-17T07:11:55.470405+0000","last_active":"2024-04-17T07:12:28.721576+0000","last_peered":"2024-04-17T07:12:28.721576+0000","last_clean":"2024-04-17T07:12:28.721576+0000","last_became_active":"2024-04-17T07:11:55.470058+0000","last_became_peered":"2024-04-17T07:11:55.470058+0000","last_unstale":"2024-04-17T07:12:28.721576+0000","last_undegraded":"2024-04-17T07:12:28.721576+0000","last_fullsized":"2024-04-17T07:12:28.721576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:17.416775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122022+0000","last_change":"2024-04-17T07:11:59.522040+0000","last_active":"2024-04-17T07:12:29.122022+0000","last_peered":"2024-04-17T07:12:29.122022+0000","last_clean":"2024-04-17T07:12:29.122022+0000","last_became_active":"2024-04-17T07:11:59.519771+0000","last_became_peered":"2024-04-17T07:11:59.519771+0000","last_unstale":"2024-04-17T07:12:29.122022+0000","last_undegraded":"2024-04-17T07:12:29.122022+0000","last_fullsized":"2024-04-17T07:12:29.122022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:58.203737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.488533+0000","last_change":"2024-04-17T07:11:54.457954+0000","last_active":"2024-04-17T07:12:30.488533+0000","last_peered":"2024-04-17T07:12:30.488533+0000","last_clean":"2024-04-17T07:12:30.488533+0000","last_became_active":"2024-04-17T07:11:54.457560+0000","last_became_peered":"2024-04-17T07:11:54.457560+0000","last_unstale":"2024-04-17T07:12:30.488533+0000","last_undegraded":"2024-04-17T07:12:30.488533+0000","last_fullsized":"2024-04-17T07:12:30.488533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:31.919730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123098+0000","last_change":"2024-04-17T07:11:55.480475+0000","last_active":"2024-04-17T07:12:29.123098+0000","last_peered":"2024-04-17T07:12:29.123098+0000","last_clean":"2024-04-17T07:12:29.123098+0000","last_became_active":"2024-04-17T07:11:55.480060+0000","last_became_peered":"2024-04-17T07:11:55.480060+0000","last_unstale":"2024-04-17T07:12:29.123098+0000","last_undegraded":"2024-04-17T07:12:29.123098+0000","last_fullsized":"2024-04-17T07:12:29.123098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:30:23.600244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714322+0000","last_change":"2024-04-17T07:11:59.525422+0000","last_active":"2024-04-17T07:12:28.714322+0000","last_peered":"2024-04-17T07:12:28.714322+0000","last_clean":"2024-04-17T07:12:28.714322+0000","last_became_active":"2024-04-17T07:11:59.523649+0000","last_became_peered":"2024-04-17T07:11:59.523649+0000","last_unstale":"2024-04-17T07:12:28.714322+0000","last_undegraded":"2024-04-17T07:12:28.714322+0000","last_fullsized":"2024-04-17T07:12:28.714322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:16.653752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714912+0000","last_change":"2024-04-17T07:11:54.465824+0000","last_active":"2024-04-17T07:12:28.714912+0000","last_peered":"2024-04-17T07:12:28.714912+0000","last_clean":"2024-04-17T07:12:28.714912+0000","last_became_active":"2024-04-17T07:11:54.465348+0000","last_became_peered":"2024-04-17T07:11:54.465348+0000","last_unstale":"2024-04-17T07:12:28.714912+0000","last_undegraded":"2024-04-17T07:12:28.714912+0000","last_fullsized":"2024-04-17T07:12:28.714912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:17.729903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736284+0000","last_change":"2024-04-17T07:11:55.481666+0000","last_active":"2024-04-17T07:12:28.736284+0000","last_peered":"2024-04-17T07:12:28.736284+0000","last_clean":"2024-04-17T07:12:28.736284+0000","last_became_active":"2024-04-17T07:11:55.480332+0000","last_became_peered":"2024-04-17T07:11:55.480332+0000","last_unstale":"2024-04-17T07:12:28.736284+0000","last_undegraded":"2024-04-17T07:12:28.736284+0000","last_fullsized":"2024-04-17T07:12:28.736284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:19.825407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714433+0000","last_change":"2024-04-17T07:11:59.527850+0000","last_active":"2024-04-17T07:12:28.714433+0000","last_peered":"2024-04-17T07:12:28.714433+0000","last_clean":"2024-04-17T07:12:28.714433+0000","last_became_active":"2024-04-17T07:11:59.527306+0000","last_became_peered":"2024-04-17T07:11:59.527306+0000","last_unstale":"2024-04-17T07:12:28.714433+0000","last_undegraded":"2024-04-17T07:12:28.714433+0000","last_fullsized":"2024-04-17T07:12:28.714433+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:35.906222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725682+0000","last_change":"2024-04-17T07:11:54.455862+0000","last_active":"2024-04-17T07:12:28.725682+0000","last_peered":"2024-04-17T07:12:28.725682+0000","last_clean":"2024-04-17T07:12:28.725682+0000","last_became_active":"2024-04-17T07:11:54.453904+0000","last_became_peered":"2024-04-17T07:11:54.453904+0000","last_unstale":"2024-04-17T07:12:28.725682+0000","last_undegraded":"2024-04-17T07:12:28.725682+0000","last_fullsized":"2024-04-17T07:12:28.725682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:01.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725079+0000","last_change":"2024-04-17T07:11:55.470315+0000","last_active":"2024-04-17T07:12:28.725079+0000","last_peered":"2024-04-17T07:12:28.725079+0000","last_clean":"2024-04-17T07:12:28.725079+0000","last_became_active":"2024-04-17T07:11:55.469829+0000","last_became_peered":"2024-04-17T07:11:55.469829+0000","last_unstale":"2024-04-17T07:12:28.725079+0000","last_undegraded":"2024-04-17T07:12:28.725079+0000","last_fullsized":"2024-04-17T07:12:28.725079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:48.197546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.126045+0000","last_change":"2024-04-17T07:12:29.125447+0000","last_active":"2024-04-17T07:12:29.126045+0000","last_peered":"2024-04-17T07:12:29.126045+0000","last_clean":"2024-04-17T07:12:29.126045+0000","last_became_active":"2024-04-17T07:12:29.124551+0000","last_became_peered":"2024-04-17T07:12:29.124551+0000","last_unstale":"2024-04-17T07:12:29.126045+0000","last_undegraded":"2024-04-17T07:12:29.126045+0000","last_fullsized":"2024-04-17T07:12:29.126045+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:09.489755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.216091+0000","last_change":"2024-04-17T07:11:54.453117+0000","last_active":"2024-04-17T07:12:30.216091+0000","last_peered":"2024-04-17T07:12:30.216091+0000","last_clean":"2024-04-17T07:12:30.216091+0000","last_became_active":"2024-04-17T07:11:54.452632+0000","last_became_peered":"2024-04-17T07:11:54.452632+0000","last_unstale":"2024-04-17T07:12:30.216091+0000","last_undegraded":"2024-04-17T07:12:30.216091+0000","last_fullsized":"2024-04-17T07:12:30.216091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:09.366149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.728073+0000","last_change":"2024-04-17T07:11:55.469991+0000","last_active":"2024-04-17T07:12:27.728073+0000","last_peered":"2024-04-17T07:12:27.728073+0000","last_clean":"2024-04-17T07:12:27.728073+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:12:27.728073+0000","last_undegraded":"2024-04-17T07:12:27.728073+0000","last_fullsized":"2024-04-17T07:12:27.728073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:02.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713914+0000","last_change":"2024-04-17T07:11:59.519817+0000","last_active":"2024-04-17T07:12:28.713914+0000","last_peered":"2024-04-17T07:12:28.713914+0000","last_clean":"2024-04-17T07:12:28.713914+0000","last_became_active":"2024-04-17T07:11:59.519108+0000","last_became_peered":"2024-04-17T07:11:59.519108+0000","last_unstale":"2024-04-17T07:12:28.713914+0000","last_undegraded":"2024-04-17T07:12:28.713914+0000","last_fullsized":"2024-04-17T07:12:28.713914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:58.378837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716195+0000","last_change":"2024-04-17T07:11:54.463618+0000","last_active":"2024-04-17T07:12:28.716195+0000","last_peered":"2024-04-17T07:12:28.716195+0000","last_clean":"2024-04-17T07:12:28.716195+0000","last_became_active":"2024-04-17T07:11:54.463257+0000","last_became_peered":"2024-04-17T07:11:54.463257+0000","last_unstale":"2024-04-17T07:12:28.716195+0000","last_undegraded":"2024-04-17T07:12:28.716195+0000","last_fullsized":"2024-04-17T07:12:28.716195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:20.135338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122992+0000","last_change":"2024-04-17T07:11:55.487471+0000","last_active":"2024-04-17T07:12:29.122992+0000","last_peered":"2024-04-17T07:12:29.122992+0000","last_clean":"2024-04-17T07:12:29.122992+0000","last_became_active":"2024-04-17T07:11:55.487024+0000","last_became_peered":"2024-04-17T07:11:55.487024+0000","last_unstale":"2024-04-17T07:12:29.122992+0000","last_undegraded":"2024-04-17T07:12:29.122992+0000","last_fullsized":"2024-04-17T07:12:29.122992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:36.851210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714472+0000","last_change":"2024-04-17T07:11:59.527017+0000","last_active":"2024-04-17T07:12:28.714472+0000","last_peered":"2024-04-17T07:12:28.714472+0000","last_clean":"2024-04-17T07:12:28.714472+0000","last_became_active":"2024-04-17T07:11:59.526541+0000","last_became_peered":"2024-04-17T07:11:59.526541+0000","last_unstale":"2024-04-17T07:12:28.714472+0000","last_undegraded":"2024-04-17T07:12:28.714472+0000","last_fullsized":"2024-04-17T07:12:28.714472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:54.411647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.728776+0000","last_change":"2024-04-17T07:11:54.465671+0000","last_active":"2024-04-17T07:12:28.728776+0000","last_peered":"2024-04-17T07:12:28.728776+0000","last_clean":"2024-04-17T07:12:28.728776+0000","last_became_active":"2024-04-17T07:11:54.464902+0000","last_became_peered":"2024-04-17T07:11:54.464902+0000","last_unstale":"2024-04-17T07:12:28.728776+0000","last_undegraded":"2024-04-17T07:12:28.728776+0000","last_fullsized":"2024-04-17T07:12:28.728776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:13:38.232965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720011+0000","last_change":"2024-04-17T07:11:55.484259+0000","last_active":"2024-04-17T07:12:28.720011+0000","last_peered":"2024-04-17T07:12:28.720011+0000","last_clean":"2024-04-17T07:12:28.720011+0000","last_became_active":"2024-04-17T07:11:55.483911+0000","last_became_peered":"2024-04-17T07:11:55.483911+0000","last_unstale":"2024-04-17T07:12:28.720011+0000","last_undegraded":"2024-04-17T07:12:28.720011+0000","last_fullsized":"2024-04-17T07:12:28.720011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:54.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725852+0000","last_change":"2024-04-17T07:11:59.525369+0000","last_active":"2024-04-17T07:12:28.725852+0000","last_peered":"2024-04-17T07:12:28.725852+0000","last_clean":"2024-04-17T07:12:28.725852+0000","last_became_active":"2024-04-17T07:11:59.524809+0000","last_became_peered":"2024-04-17T07:11:59.524809+0000","last_unstale":"2024-04-17T07:12:28.725852+0000","last_undegraded":"2024-04-17T07:12:28.725852+0000","last_fullsized":"2024-04-17T07:12:28.725852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:20.511001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.197527+0000","last_change":"2024-04-17T07:11:54.459079+0000","last_active":"2024-04-17T07:12:29.197527+0000","last_peered":"2024-04-17T07:12:29.197527+0000","last_clean":"2024-04-17T07:12:29.197527+0000","last_became_active":"2024-04-17T07:11:54.457008+0000","last_became_peered":"2024-04-17T07:11:54.457008+0000","last_unstale":"2024-04-17T07:12:29.197527+0000","last_undegraded":"2024-04-17T07:12:29.197527+0000","last_fullsized":"2024-04-17T07:12:29.197527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:36.084512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721123+0000","last_change":"2024-04-17T07:11:55.484453+0000","last_active":"2024-04-17T07:12:28.721123+0000","last_peered":"2024-04-17T07:12:28.721123+0000","last_clean":"2024-04-17T07:12:28.721123+0000","last_became_active":"2024-04-17T07:11:55.483152+0000","last_became_peered":"2024-04-17T07:11:55.483152+0000","last_unstale":"2024-04-17T07:12:28.721123+0000","last_undegraded":"2024-04-17T07:12:28.721123+0000","last_fullsized":"2024-04-17T07:12:28.721123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:32.224868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735401+0000","last_change":"2024-04-17T07:11:59.521487+0000","last_active":"2024-04-17T07:12:28.735401+0000","last_peered":"2024-04-17T07:12:28.735401+0000","last_clean":"2024-04-17T07:12:28.735401+0000","last_became_active":"2024-04-17T07:11:59.521060+0000","last_became_peered":"2024-04-17T07:11:59.521060+0000","last_unstale":"2024-04-17T07:12:28.735401+0000","last_undegraded":"2024-04-17T07:12:28.735401+0000","last_fullsized":"2024-04-17T07:12:28.735401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:06.692434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214903+0000","last_change":"2024-04-17T07:11:54.459368+0000","last_active":"2024-04-17T07:12:30.214903+0000","last_peered":"2024-04-17T07:12:30.214903+0000","last_clean":"2024-04-17T07:12:30.214903+0000","last_became_active":"2024-04-17T07:11:54.458973+0000","last_became_peered":"2024-04-17T07:11:54.458973+0000","last_unstale":"2024-04-17T07:12:30.214903+0000","last_undegraded":"2024-04-17T07:12:30.214903+0000","last_fullsized":"2024-04-17T07:12:30.214903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:22.034110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721530+0000","last_change":"2024-04-17T07:11:55.470648+0000","last_active":"2024-04-17T07:12:28.721530+0000","last_peered":"2024-04-17T07:12:28.721530+0000","last_clean":"2024-04-17T07:12:28.721530+0000","last_became_active":"2024-04-17T07:11:55.469666+0000","last_became_peered":"2024-04-17T07:11:55.469666+0000","last_unstale":"2024-04-17T07:12:28.721530+0000","last_undegraded":"2024-04-17T07:12:28.721530+0000","last_fullsized":"2024-04-17T07:12:28.721530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:59.208872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730680+0000","last_change":"2024-04-17T07:11:59.524027+0000","last_active":"2024-04-17T07:12:28.730680+0000","last_peered":"2024-04-17T07:12:28.730680+0000","last_clean":"2024-04-17T07:12:28.730680+0000","last_became_active":"2024-04-17T07:11:59.522764+0000","last_became_peered":"2024-04-17T07:11:59.522764+0000","last_unstale":"2024-04-17T07:12:28.730680+0000","last_undegraded":"2024-04-17T07:12:28.730680+0000","last_fullsized":"2024-04-17T07:12:28.730680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:40.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.216540+0000","last_change":"2024-04-17T07:11:54.472884+0000","last_active":"2024-04-17T07:12:30.216540+0000","last_peered":"2024-04-17T07:12:30.216540+0000","last_clean":"2024-04-17T07:12:30.216540+0000","last_became_active":"2024-04-17T07:11:54.472305+0000","last_became_peered":"2024-04-17T07:11:54.472305+0000","last_unstale":"2024-04-17T07:12:30.216540+0000","last_undegraded":"2024-04-17T07:12:30.216540+0000","last_fullsized":"2024-04-17T07:12:30.216540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:21.031442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721232+0000","last_change":"2024-04-17T07:11:55.479982+0000","last_active":"2024-04-17T07:12:28.721232+0000","last_peered":"2024-04-17T07:12:28.721232+0000","last_clean":"2024-04-17T07:12:28.721232+0000","last_became_active":"2024-04-17T07:11:55.477594+0000","last_became_peered":"2024-04-17T07:11:55.477594+0000","last_unstale":"2024-04-17T07:12:28.721232+0000","last_undegraded":"2024-04-17T07:12:28.721232+0000","last_fullsized":"2024-04-17T07:12:28.721232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:23.507312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.732471+0000","last_change":"2024-04-17T07:11:55.480941+0000","last_active":"2024-04-17T07:12:27.732471+0000","last_peered":"2024-04-17T07:12:27.732471+0000","last_clean":"2024-04-17T07:12:27.732471+0000","last_became_active":"2024-04-17T07:11:55.480180+0000","last_became_peered":"2024-04-17T07:11:55.480180+0000","last_unstale":"2024-04-17T07:12:27.732471+0000","last_undegraded":"2024-04-17T07:12:27.732471+0000","last_fullsized":"2024-04-17T07:12:27.732471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:16.321806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715350+0000","last_change":"2024-04-17T07:11:55.477663+0000","last_active":"2024-04-17T07:12:28.715350+0000","last_peered":"2024-04-17T07:12:28.715350+0000","last_clean":"2024-04-17T07:12:28.715350+0000","last_became_active":"2024-04-17T07:11:55.477304+0000","last_became_peered":"2024-04-17T07:11:55.477304+0000","last_unstale":"2024-04-17T07:12:28.715350+0000","last_undegraded":"2024-04-17T07:12:28.715350+0000","last_fullsized":"2024-04-17T07:12:28.715350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:00.796633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714460+0000","last_change":"2024-04-17T07:11:55.475536+0000","last_active":"2024-04-17T07:12:28.714460+0000","last_peered":"2024-04-17T07:12:28.714460+0000","last_clean":"2024-04-17T07:12:28.714460+0000","last_became_active":"2024-04-17T07:11:55.475138+0000","last_became_peered":"2024-04-17T07:11:55.475138+0000","last_unstale":"2024-04-17T07:12:28.714460+0000","last_undegraded":"2024-04-17T07:12:28.714460+0000","last_fullsized":"2024-04-17T07:12:28.714460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:47.849076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735949+0000","last_change":"2024-04-17T07:11:55.483742+0000","last_active":"2024-04-17T07:12:28.735949+0000","last_peered":"2024-04-17T07:12:28.735949+0000","last_clean":"2024-04-17T07:12:28.735949+0000","last_became_active":"2024-04-17T07:11:55.483044+0000","last_became_peered":"2024-04-17T07:11:55.483044+0000","last_unstale":"2024-04-17T07:12:28.735949+0000","last_undegraded":"2024-04-17T07:12:28.735949+0000","last_fullsized":"2024-04-17T07:12:28.735949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:18.972137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721440+0000","last_change":"2024-04-17T07:11:55.471266+0000","last_active":"2024-04-17T07:12:28.721440+0000","last_peered":"2024-04-17T07:12:28.721440+0000","last_clean":"2024-04-17T07:12:28.721440+0000","last_became_active":"2024-04-17T07:11:55.470897+0000","last_became_peered":"2024-04-17T07:11:55.470897+0000","last_unstale":"2024-04-17T07:12:28.721440+0000","last_undegraded":"2024-04-17T07:12:28.721440+0000","last_fullsized":"2024-04-17T07:12:28.721440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:25.811064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.736847+0000","last_change":"2024-04-17T07:11:55.475860+0000","last_active":"2024-04-17T07:12:27.736847+0000","last_peered":"2024-04-17T07:12:27.736847+0000","last_clean":"2024-04-17T07:12:27.736847+0000","last_became_active":"2024-04-17T07:11:55.475522+0000","last_became_peered":"2024-04-17T07:11:55.475522+0000","last_unstale":"2024-04-17T07:12:27.736847+0000","last_undegraded":"2024-04-17T07:12:27.736847+0000","last_fullsized":"2024-04-17T07:12:27.736847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:41.548964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715736+0000","last_change":"2024-04-17T07:11:55.478815+0000","last_active":"2024-04-17T07:12:28.715736+0000","last_peered":"2024-04-17T07:12:28.715736+0000","last_clean":"2024-04-17T07:12:28.715736+0000","last_became_active":"2024-04-17T07:11:55.478347+0000","last_became_peered":"2024-04-17T07:11:55.478347+0000","last_unstale":"2024-04-17T07:12:28.715736+0000","last_undegraded":"2024-04-17T07:12:28.715736+0000","last_fullsized":"2024-04-17T07:12:28.715736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:16.465511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123444+0000","last_change":"2024-04-17T07:11:55.474208+0000","last_active":"2024-04-17T07:12:29.123444+0000","last_peered":"2024-04-17T07:12:29.123444+0000","last_clean":"2024-04-17T07:12:29.123444+0000","last_became_active":"2024-04-17T07:11:55.472042+0000","last_became_peered":"2024-04-17T07:11:55.472042+0000","last_unstale":"2024-04-17T07:12:29.123444+0000","last_undegraded":"2024-04-17T07:12:29.123444+0000","last_fullsized":"2024-04-17T07:12:29.123444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:37:24.248506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.733719+0000","last_change":"2024-04-17T07:11:55.477257+0000","last_active":"2024-04-17T07:12:28.733719+0000","last_peered":"2024-04-17T07:12:28.733719+0000","last_clean":"2024-04-17T07:12:28.733719+0000","last_became_active":"2024-04-17T07:11:55.473491+0000","last_became_peered":"2024-04-17T07:11:55.473491+0000","last_unstale":"2024-04-17T07:12:28.733719+0000","last_undegraded":"2024-04-17T07:12:28.733719+0000","last_fullsized":"2024-04-17T07:12:28.733719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:33.737485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122022+0000","last_change":"2024-04-17T07:11:55.474930+0000","last_active":"2024-04-17T07:12:29.122022+0000","last_peered":"2024-04-17T07:12:29.122022+0000","last_clean":"2024-04-17T07:12:29.122022+0000","last_became_active":"2024-04-17T07:11:55.472007+0000","last_became_peered":"2024-04-17T07:11:55.472007+0000","last_unstale":"2024-04-17T07:12:29.122022+0000","last_undegraded":"2024-04-17T07:12:29.122022+0000","last_fullsized":"2024-04-17T07:12:29.122022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:07.111267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726119+0000","last_change":"2024-04-17T07:11:55.492422+0000","last_active":"2024-04-17T07:12:28.726119+0000","last_peered":"2024-04-17T07:12:28.726119+0000","last_clean":"2024-04-17T07:12:28.726119+0000","last_became_active":"2024-04-17T07:11:55.491906+0000","last_became_peered":"2024-04-17T07:11:55.491906+0000","last_unstale":"2024-04-17T07:12:28.726119+0000","last_undegraded":"2024-04-17T07:12:28.726119+0000","last_fullsized":"2024-04-17T07:12:28.726119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:13:05.322586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720385+0000","last_change":"2024-04-17T07:11:55.479436+0000","last_active":"2024-04-17T07:12:28.720385+0000","last_peered":"2024-04-17T07:12:28.720385+0000","last_clean":"2024-04-17T07:12:28.720385+0000","last_became_active":"2024-04-17T07:11:55.478982+0000","last_became_peered":"2024-04-17T07:11:55.478982+0000","last_unstale":"2024-04-17T07:12:28.720385+0000","last_undegraded":"2024-04-17T07:12:28.720385+0000","last_fullsized":"2024-04-17T07:12:28.720385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:20.663457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.733290+0000","last_change":"2024-04-17T07:11:55.476708+0000","last_active":"2024-04-17T07:12:27.733290+0000","last_peered":"2024-04-17T07:12:27.733290+0000","last_clean":"2024-04-17T07:12:27.733290+0000","last_became_active":"2024-04-17T07:11:55.476344+0000","last_became_peered":"2024-04-17T07:11:55.476344+0000","last_unstale":"2024-04-17T07:12:27.733290+0000","last_undegraded":"2024-04-17T07:12:27.733290+0000","last_fullsized":"2024-04-17T07:12:27.733290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:58.874987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279575,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48551,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":26,"num_read_kb":61,"num_write":196,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":95150,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383702,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14703,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645343,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":1130171,"data_compressed":58079,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":501379,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":497549,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":1094158,"data_compressed":56230,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":496883,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":501598,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":498339,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":507953,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":496569,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":487441,"data_compressed":17684,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1093836,"data_compressed":56230,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":13207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43642,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:12:34.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:12:34.430 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:34 smithi086 ceph-mon[31733]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:34.430 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:34 smithi086 ceph-mon[31733]: pgmap v384: 129 pgs: 1 activating, 2 peering, 126 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 21 op/s; 77 B/s, 0 objects/s recovering 2024-04-17T07:12:34.430 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:34 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1201571890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:34 smithi098 ceph-mon[35418]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:34 smithi098 ceph-mon[35418]: pgmap v384: 129 pgs: 1 activating, 2 peering, 126 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 21 op/s; 77 B/s, 0 objects/s recovering 2024-04-17T07:12:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:34 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1201571890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:34 smithi103 ceph-mon[35495]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:12:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:34 smithi103 ceph-mon[35495]: pgmap v384: 129 pgs: 1 activating, 2 peering, 126 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 21 op/s; 77 B/s, 0 objects/s recovering 2024-04-17T07:12:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:34 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1201571890' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.568+0000 7fae93200640 1 -- 172.21.15.86:0/66266972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c103fe0 msgr2=0x7fae8c104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.568+0000 7fae93200640 1 --2- 172.21.15.86:0/66266972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c103fe0 0x7fae8c104400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fae80009900 tx=0x7fae8002bc30 comp rx=0 tx=0).stop 2024-04-17T07:12:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.569+0000 7fae93200640 1 -- 172.21.15.86:0/66266972 shutdown_connections 2024-04-17T07:12:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.569+0000 7fae93200640 1 --2- 172.21.15.86:0/66266972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae8c105b90 0x7fae8c10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.569+0000 7fae93200640 1 --2- 172.21.15.86:0/66266972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c1051d0 0x7fae8c105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.569+0000 7fae93200640 1 --2- 172.21.15.86:0/66266972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c103fe0 0x7fae8c104400 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.569+0000 7fae93200640 1 -- 172.21.15.86:0/66266972 >> 172.21.15.86:0/66266972 conn(0x7fae8c0ff570 msgr2=0x7fae8c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.569+0000 7fae93200640 1 -- 172.21.15.86:0/66266972 shutdown_connections 2024-04-17T07:12:34.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.569+0000 7fae93200640 1 -- 172.21.15.86:0/66266972 wait complete. 2024-04-17T07:12:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.570+0000 7fae93200640 1 Processor -- start 2024-04-17T07:12:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.570+0000 7fae93200640 1 -- start start 2024-04-17T07:12:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.570+0000 7fae93200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae8c103fe0 0x7fae8c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.570+0000 7fae93200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c1051d0 0x7fae8c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.570+0000 7fae93200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c105b90 0x7fae8c1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.570+0000 7fae93200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c10fb20 con 0x7fae8c105b90 2024-04-17T07:12:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.570+0000 7fae93200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c10f9a0 con 0x7fae8c1051d0 2024-04-17T07:12:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.570+0000 7fae93200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c10fca0 con 0x7fae8c103fe0 2024-04-17T07:12:34.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae92800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c105b90 0x7fae8c1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae92800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c105b90 0x7fae8c1a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54204/0 (socket says 172.21.15.86:54204) 2024-04-17T07:12:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae92800640 1 -- 172.21.15.86:0/485286233 learned_addr learned my addr 172.21.15.86:0/485286233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae91400640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c1051d0 0x7fae8c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae91e00640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae8c103fe0 0x7fae8c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae92800640 1 -- 172.21.15.86:0/485286233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae8c103fe0 msgr2=0x7fae8c19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae92800640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae8c103fe0 0x7fae8c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae92800640 1 -- 172.21.15.86:0/485286233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c1051d0 msgr2=0x7fae8c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae92800640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c1051d0 0x7fae8c19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae92800640 1 -- 172.21.15.86:0/485286233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c1a1e20 con 0x7fae8c105b90 2024-04-17T07:12:34.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.571+0000 7fae92800640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c105b90 0x7fae8c1a17b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fae8800eaa0 tx=0x7fae8800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.572+0000 7fae86a00640 1 -- 172.21.15.86:0/485286233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8800cdb0 con 0x7fae8c105b90 2024-04-17T07:12:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.572+0000 7fae86a00640 1 -- 172.21.15.86:0/485286233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae88005020 con 0x7fae8c105b90 2024-04-17T07:12:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.572+0000 7fae86a00640 1 -- 172.21.15.86:0/485286233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae880116d0 con 0x7fae8c105b90 2024-04-17T07:12:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.572+0000 7fae93200640 1 -- 172.21.15.86:0/485286233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c1a2110 con 0x7fae8c105b90 2024-04-17T07:12:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.572+0000 7fae93200640 1 -- 172.21.15.86:0/485286233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c1a2660 con 0x7fae8c105b90 2024-04-17T07:12:34.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.573+0000 7fae93200640 1 -- 172.21.15.86:0/485286233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8c10b380 con 0x7fae8c105b90 2024-04-17T07:12:34.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.575+0000 7fae86a00640 1 -- 172.21.15.86:0/485286233 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fae880051d0 con 0x7fae8c105b90 2024-04-17T07:12:34.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.576+0000 7fae86a00640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae740a3c40 0x7fae740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:34.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.576+0000 7fae86a00640 1 -- 172.21.15.86:0/485286233 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fae880d0b90 con 0x7fae8c105b90 2024-04-17T07:12:34.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.576+0000 7fae91e00640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae740a3c40 0x7fae740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:34.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.577+0000 7fae91e00640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae740a3c40 0x7fae740a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae80036040 tx=0x7fae80000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:34.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.581+0000 7fae86a00640 1 -- 172.21.15.86:0/485286233 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fae88016030 con 0x7fae8c105b90 2024-04-17T07:12:34.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.782+0000 7fae93200640 1 -- 172.21.15.86:0/485286233 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae8c0623a0 con 0x7fae740a3c40 2024-04-17T07:12:34.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.793+0000 7fae86a00640 1 -- 172.21.15.86:0/485286233 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340706 (secure 0 0 0) 0x7fae8c0623a0 con 0x7fae740a3c40 2024-04-17T07:12:34.797 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:34.799 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-17T07:12:34.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.798+0000 7fae93200640 1 -- 172.21.15.86:0/485286233 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae740a3c40 msgr2=0x7fae740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.798+0000 7fae93200640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae740a3c40 0x7fae740a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae80036040 tx=0x7fae80000bc0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.798+0000 7fae93200640 1 -- 172.21.15.86:0/485286233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c105b90 msgr2=0x7fae8c1a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.798+0000 7fae93200640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c105b90 0x7fae8c1a17b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fae8800eaa0 tx=0x7fae8800ef60 comp rx=0 tx=0).stop 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.799+0000 7fae93200640 1 -- 172.21.15.86:0/485286233 shutdown_connections 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.799+0000 7fae93200640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae740a3c40 0x7fae740a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.799+0000 7fae93200640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae8c105b90 0x7fae8c1a17b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.799+0000 7fae93200640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c1051d0 0x7fae8c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.799+0000 7fae93200640 1 --2- 172.21.15.86:0/485286233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae8c103fe0 0x7fae8c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.799+0000 7fae93200640 1 -- 172.21.15.86:0/485286233 >> 172.21.15.86:0/485286233 conn(0x7fae8c0ff570 msgr2=0x7fae8c108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.799+0000 7fae93200640 1 -- 172.21.15.86:0/485286233 shutdown_connections 2024-04-17T07:12:34.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:34.799+0000 7fae93200640 1 -- 172.21.15.86:0/485286233 wait complete. 2024-04-17T07:12:34.817 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-17T07:12:33.058088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639036,"num_objects":86,"num_object_clones":0,"num_object_copies":259,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":152,"num_read_kb":170,"num_write":378,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1181358,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365404,"kb_used_data":12076,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576420,"statfs":{"total":1151940427776,"available":1151566254080,"internally_reserved":0,"allocated":12365824,"data_stored":7783091,"data_compressed":347595,"data_compressed_allocated":471040,"data_compressed_original":2441216,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44028,"num_objects":62,"num_object_clones":0,"num_object_copies":187,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":26,"num_read_kb":61,"num_write":148,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002844"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.738061+0000","last_change":"2024-04-17T07:11:55.486746+0000","last_active":"2024-04-17T07:12:27.738061+0000","last_peered":"2024-04-17T07:12:27.738061+0000","last_clean":"2024-04-17T07:12:27.738061+0000","last_became_active":"2024-04-17T07:11:55.486335+0000","last_became_peered":"2024-04-17T07:11:55.486335+0000","last_unstale":"2024-04-17T07:12:27.738061+0000","last_undegraded":"2024-04-17T07:12:27.738061+0000","last_fullsized":"2024-04-17T07:12:27.738061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:48.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735866+0000","last_change":"2024-04-17T07:11:55.481832+0000","last_active":"2024-04-17T07:12:28.735866+0000","last_peered":"2024-04-17T07:12:28.735866+0000","last_clean":"2024-04-17T07:12:28.735866+0000","last_became_active":"2024-04-17T07:11:55.476428+0000","last_became_peered":"2024-04-17T07:11:55.476428+0000","last_unstale":"2024-04-17T07:12:28.735866+0000","last_undegraded":"2024-04-17T07:12:28.735866+0000","last_fullsized":"2024-04-17T07:12:28.735866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:04.452542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736227+0000","last_change":"2024-04-17T07:11:55.483721+0000","last_active":"2024-04-17T07:12:28.736227+0000","last_peered":"2024-04-17T07:12:28.736227+0000","last_clean":"2024-04-17T07:12:28.736227+0000","last_became_active":"2024-04-17T07:11:55.483268+0000","last_became_peered":"2024-04-17T07:11:55.483268+0000","last_unstale":"2024-04-17T07:12:28.736227+0000","last_undegraded":"2024-04-17T07:12:28.736227+0000","last_fullsized":"2024-04-17T07:12:28.736227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:41.584783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714249+0000","last_change":"2024-04-17T07:11:55.485114+0000","last_active":"2024-04-17T07:12:28.714249+0000","last_peered":"2024-04-17T07:12:28.714249+0000","last_clean":"2024-04-17T07:12:28.714249+0000","last_became_active":"2024-04-17T07:11:55.483913+0000","last_became_peered":"2024-04-17T07:11:55.483913+0000","last_unstale":"2024-04-17T07:12:28.714249+0000","last_undegraded":"2024-04-17T07:12:28.714249+0000","last_fullsized":"2024-04-17T07:12:28.714249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:46.211121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726512+0000","last_change":"2024-04-17T07:11:55.491952+0000","last_active":"2024-04-17T07:12:28.726512+0000","last_peered":"2024-04-17T07:12:28.726512+0000","last_clean":"2024-04-17T07:12:28.726512+0000","last_became_active":"2024-04-17T07:11:55.491238+0000","last_became_peered":"2024-04-17T07:11:55.491238+0000","last_unstale":"2024-04-17T07:12:28.726512+0000","last_undegraded":"2024-04-17T07:12:28.726512+0000","last_fullsized":"2024-04-17T07:12:28.726512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:06.759797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725020+0000","last_change":"2024-04-17T07:11:55.471124+0000","last_active":"2024-04-17T07:12:28.725020+0000","last_peered":"2024-04-17T07:12:28.725020+0000","last_clean":"2024-04-17T07:12:28.725020+0000","last_became_active":"2024-04-17T07:11:55.470781+0000","last_became_peered":"2024-04-17T07:11:55.470781+0000","last_unstale":"2024-04-17T07:12:28.725020+0000","last_undegraded":"2024-04-17T07:12:28.725020+0000","last_fullsized":"2024-04-17T07:12:28.725020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:45.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720008+0000","last_change":"2024-04-17T07:11:55.469099+0000","last_active":"2024-04-17T07:12:28.720008+0000","last_peered":"2024-04-17T07:12:28.720008+0000","last_clean":"2024-04-17T07:12:28.720008+0000","last_became_active":"2024-04-17T07:11:55.468584+0000","last_became_peered":"2024-04-17T07:11:55.468584+0000","last_unstale":"2024-04-17T07:12:28.720008+0000","last_undegraded":"2024-04-17T07:12:28.720008+0000","last_fullsized":"2024-04-17T07:12:28.720008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:52.834846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122568+0000","last_change":"2024-04-17T07:11:55.473794+0000","last_active":"2024-04-17T07:12:29.122568+0000","last_peered":"2024-04-17T07:12:29.122568+0000","last_clean":"2024-04-17T07:12:29.122568+0000","last_became_active":"2024-04-17T07:11:55.471919+0000","last_became_peered":"2024-04-17T07:11:55.471919+0000","last_unstale":"2024-04-17T07:12:29.122568+0000","last_undegraded":"2024-04-17T07:12:29.122568+0000","last_fullsized":"2024-04-17T07:12:29.122568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:16.085117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726090+0000","last_change":"2024-04-17T07:11:55.482445+0000","last_active":"2024-04-17T07:12:28.726090+0000","last_peered":"2024-04-17T07:12:28.726090+0000","last_clean":"2024-04-17T07:12:28.726090+0000","last_became_active":"2024-04-17T07:11:55.481552+0000","last_became_peered":"2024-04-17T07:11:55.481552+0000","last_unstale":"2024-04-17T07:12:28.726090+0000","last_undegraded":"2024-04-17T07:12:28.726090+0000","last_fullsized":"2024-04-17T07:12:28.726090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:48.733585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725405+0000","last_change":"2024-04-17T07:11:55.476726+0000","last_active":"2024-04-17T07:12:28.725405+0000","last_peered":"2024-04-17T07:12:28.725405+0000","last_clean":"2024-04-17T07:12:28.725405+0000","last_became_active":"2024-04-17T07:11:55.476320+0000","last_became_peered":"2024-04-17T07:11:55.476320+0000","last_unstale":"2024-04-17T07:12:28.725405+0000","last_undegraded":"2024-04-17T07:12:28.725405+0000","last_fullsized":"2024-04-17T07:12:28.725405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.230181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721027+0000","last_change":"2024-04-17T07:11:55.485798+0000","last_active":"2024-04-17T07:12:28.721027+0000","last_peered":"2024-04-17T07:12:28.721027+0000","last_clean":"2024-04-17T07:12:28.721027+0000","last_became_active":"2024-04-17T07:11:55.484015+0000","last_became_peered":"2024-04-17T07:11:55.484015+0000","last_unstale":"2024-04-17T07:12:28.721027+0000","last_undegraded":"2024-04-17T07:12:28.721027+0000","last_fullsized":"2024-04-17T07:12:28.721027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:37.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.731078+0000","last_change":"2024-04-17T07:12:28.729834+0000","last_active":"2024-04-17T07:12:28.731078+0000","last_peered":"2024-04-17T07:12:28.731078+0000","last_clean":"2024-04-17T07:12:28.731078+0000","last_became_active":"2024-04-17T07:12:28.728694+0000","last_became_peered":"2024-04-17T07:12:28.728694+0000","last_unstale":"2024-04-17T07:12:28.731078+0000","last_undegraded":"2024-04-17T07:12:28.731078+0000","last_fullsized":"2024-04-17T07:12:28.731078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:46.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720070+0000","last_change":"2024-04-17T07:11:55.450379+0000","last_active":"2024-04-17T07:12:28.720070+0000","last_peered":"2024-04-17T07:12:28.720070+0000","last_clean":"2024-04-17T07:12:28.720070+0000","last_became_active":"2024-04-17T07:11:55.449118+0000","last_became_peered":"2024-04-17T07:11:55.449118+0000","last_unstale":"2024-04-17T07:12:28.720070+0000","last_undegraded":"2024-04-17T07:12:28.720070+0000","last_fullsized":"2024-04-17T07:12:28.720070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:07:34.248044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735022+0000","last_change":"2024-04-17T07:11:55.473839+0000","last_active":"2024-04-17T07:12:28.735022+0000","last_peered":"2024-04-17T07:12:28.735022+0000","last_clean":"2024-04-17T07:12:28.735022+0000","last_became_active":"2024-04-17T07:11:55.473472+0000","last_became_peered":"2024-04-17T07:11:55.473472+0000","last_unstale":"2024-04-17T07:12:28.735022+0000","last_undegraded":"2024-04-17T07:12:28.735022+0000","last_fullsized":"2024-04-17T07:12:28.735022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:02.263272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736022+0000","last_change":"2024-04-17T07:11:55.484820+0000","last_active":"2024-04-17T07:12:28.736022+0000","last_peered":"2024-04-17T07:12:28.736022+0000","last_clean":"2024-04-17T07:12:28.736022+0000","last_became_active":"2024-04-17T07:11:55.484314+0000","last_became_peered":"2024-04-17T07:11:55.484314+0000","last_unstale":"2024-04-17T07:12:28.736022+0000","last_undegraded":"2024-04-17T07:12:28.736022+0000","last_fullsized":"2024-04-17T07:12:28.736022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.186912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725577+0000","last_change":"2024-04-17T07:11:55.492018+0000","last_active":"2024-04-17T07:12:28.725577+0000","last_peered":"2024-04-17T07:12:28.725577+0000","last_clean":"2024-04-17T07:12:28.725577+0000","last_became_active":"2024-04-17T07:11:55.490512+0000","last_became_peered":"2024-04-17T07:11:55.490512+0000","last_unstale":"2024-04-17T07:12:28.725577+0000","last_undegraded":"2024-04-17T07:12:28.725577+0000","last_fullsized":"2024-04-17T07:12:28.725577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:45.436270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.737302+0000","last_change":"2024-04-17T07:11:55.487210+0000","last_active":"2024-04-17T07:12:27.737302+0000","last_peered":"2024-04-17T07:12:27.737302+0000","last_clean":"2024-04-17T07:12:27.737302+0000","last_became_active":"2024-04-17T07:11:55.485713+0000","last_became_peered":"2024-04-17T07:11:55.485713+0000","last_unstale":"2024-04-17T07:12:27.737302+0000","last_undegraded":"2024-04-17T07:12:27.737302+0000","last_fullsized":"2024-04-17T07:12:27.737302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:54.495950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716204+0000","last_change":"2024-04-17T07:11:55.482705+0000","last_active":"2024-04-17T07:12:28.716204+0000","last_peered":"2024-04-17T07:12:28.716204+0000","last_clean":"2024-04-17T07:12:28.716204+0000","last_became_active":"2024-04-17T07:11:55.482349+0000","last_became_peered":"2024-04-17T07:11:55.482349+0000","last_unstale":"2024-04-17T07:12:28.716204+0000","last_undegraded":"2024-04-17T07:12:28.716204+0000","last_fullsized":"2024-04-17T07:12:28.716204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:57.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716213+0000","last_change":"2024-04-17T07:11:55.467065+0000","last_active":"2024-04-17T07:12:28.716213+0000","last_peered":"2024-04-17T07:12:28.716213+0000","last_clean":"2024-04-17T07:12:28.716213+0000","last_became_active":"2024-04-17T07:11:55.466535+0000","last_became_peered":"2024-04-17T07:11:55.466535+0000","last_unstale":"2024-04-17T07:12:28.716213+0000","last_undegraded":"2024-04-17T07:12:28.716213+0000","last_fullsized":"2024-04-17T07:12:28.716213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:49.955453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715012+0000","last_change":"2024-04-17T07:11:59.515778+0000","last_active":"2024-04-17T07:12:28.715012+0000","last_peered":"2024-04-17T07:12:28.715012+0000","last_clean":"2024-04-17T07:12:28.715012+0000","last_became_active":"2024-04-17T07:11:59.515266+0000","last_became_peered":"2024-04-17T07:11:59.515266+0000","last_unstale":"2024-04-17T07:12:28.715012+0000","last_undegraded":"2024-04-17T07:12:28.715012+0000","last_fullsized":"2024-04-17T07:12:28.715012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:43.084724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.219364+0000","last_change":"2024-04-17T07:11:54.467060+0000","last_active":"2024-04-17T07:12:31.219364+0000","last_peered":"2024-04-17T07:12:31.219364+0000","last_clean":"2024-04-17T07:12:31.219364+0000","last_became_active":"2024-04-17T07:11:54.466687+0000","last_became_peered":"2024-04-17T07:11:54.466687+0000","last_unstale":"2024-04-17T07:12:31.219364+0000","last_undegraded":"2024-04-17T07:12:31.219364+0000","last_fullsized":"2024-04-17T07:12:31.219364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:57.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123283+0000","last_change":"2024-04-17T07:11:55.469857+0000","last_active":"2024-04-17T07:12:29.123283+0000","last_peered":"2024-04-17T07:12:29.123283+0000","last_clean":"2024-04-17T07:12:29.123283+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:12:29.123283+0000","last_undegraded":"2024-04-17T07:12:29.123283+0000","last_fullsized":"2024-04-17T07:12:29.123283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:43.565627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"activating","last_fresh":"2024-04-17T07:12:28.723099+0000","last_change":"2024-04-17T07:12:28.714326+0000","last_active":"2024-04-17T07:12:27.714584+0000","last_peered":"2024-04-17T07:12:00.800296+0000","last_clean":"2024-04-17T07:12:00.800296+0000","last_became_active":"2024-04-17T07:11:59.512404+0000","last_became_peered":"2024-04-17T07:11:59.512404+0000","last_unstale":"2024-04-17T07:12:28.723099+0000","last_undegraded":"2024-04-17T07:12:28.723099+0000","last_fullsized":"2024-04-17T07:12:28.723099+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":["4(0)","3(3)","6(1)","8(2)","9(2)"],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123277+0000","last_change":"2024-04-17T07:11:54.464715+0000","last_active":"2024-04-17T07:12:29.123277+0000","last_peered":"2024-04-17T07:12:29.123277+0000","last_clean":"2024-04-17T07:12:29.123277+0000","last_became_active":"2024-04-17T07:11:54.463917+0000","last_became_peered":"2024-04-17T07:11:54.463917+0000","last_unstale":"2024-04-17T07:12:29.123277+0000","last_undegraded":"2024-04-17T07:12:29.123277+0000","last_fullsized":"2024-04-17T07:12:29.123277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:43.738471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736571+0000","last_change":"2024-04-17T07:11:55.482876+0000","last_active":"2024-04-17T07:12:28.736571+0000","last_peered":"2024-04-17T07:12:28.736571+0000","last_clean":"2024-04-17T07:12:28.736571+0000","last_became_active":"2024-04-17T07:11:55.482451+0000","last_became_peered":"2024-04-17T07:11:55.482451+0000","last_unstale":"2024-04-17T07:12:28.736571+0000","last_undegraded":"2024-04-17T07:12:28.736571+0000","last_fullsized":"2024-04-17T07:12:28.736571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:39.394090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713748+0000","last_change":"2024-04-17T07:11:59.522634+0000","last_active":"2024-04-17T07:12:28.713748+0000","last_peered":"2024-04-17T07:12:28.713748+0000","last_clean":"2024-04-17T07:12:28.713748+0000","last_became_active":"2024-04-17T07:11:59.520797+0000","last_became_peered":"2024-04-17T07:11:59.520797+0000","last_unstale":"2024-04-17T07:12:28.713748+0000","last_undegraded":"2024-04-17T07:12:28.713748+0000","last_fullsized":"2024-04-17T07:12:28.713748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:02.316515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-17T07:11:55.467756+0000","last_change":"2024-04-17T07:12:28.153200+0000","last_active":"2024-04-17T07:12:28.153200+0000","last_peered":"2024-04-17T07:11:55.467756+0000","last_clean":"2024-04-17T07:11:55.467756+0000","last_became_active":"2024-04-17T07:11:54.469320+0000","last_became_peered":"2024-04-17T07:11:54.469320+0000","last_unstale":"2024-04-17T07:11:55.467756+0000","last_undegraded":"2024-04-17T07:11:55.467756+0000","last_fullsized":"2024-04-17T07:11:55.467756+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726296+0000","last_change":"2024-04-17T07:11:55.485999+0000","last_active":"2024-04-17T07:12:28.726296+0000","last_peered":"2024-04-17T07:12:28.726296+0000","last_clean":"2024-04-17T07:12:28.726296+0000","last_became_active":"2024-04-17T07:11:55.484048+0000","last_became_peered":"2024-04-17T07:11:55.484048+0000","last_unstale":"2024-04-17T07:12:28.726296+0000","last_undegraded":"2024-04-17T07:12:28.726296+0000","last_fullsized":"2024-04-17T07:12:28.726296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:38.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713330+0000","last_change":"2024-04-17T07:11:59.529929+0000","last_active":"2024-04-17T07:12:28.713330+0000","last_peered":"2024-04-17T07:12:28.713330+0000","last_clean":"2024-04-17T07:12:28.713330+0000","last_became_active":"2024-04-17T07:11:59.529400+0000","last_became_peered":"2024-04-17T07:11:59.529400+0000","last_unstale":"2024-04-17T07:12:28.713330+0000","last_undegraded":"2024-04-17T07:12:28.713330+0000","last_fullsized":"2024-04-17T07:12:28.713330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:54.845529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.209278+0000","last_change":"2024-04-17T07:11:54.462852+0000","last_active":"2024-04-17T07:12:30.209278+0000","last_peered":"2024-04-17T07:12:30.209278+0000","last_clean":"2024-04-17T07:12:30.209278+0000","last_became_active":"2024-04-17T07:11:54.459178+0000","last_became_peered":"2024-04-17T07:11:54.459178+0000","last_unstale":"2024-04-17T07:12:30.209278+0000","last_undegraded":"2024-04-17T07:12:30.209278+0000","last_fullsized":"2024-04-17T07:12:30.209278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:07.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.737683+0000","last_change":"2024-04-17T07:11:55.480951+0000","last_active":"2024-04-17T07:12:28.737683+0000","last_peered":"2024-04-17T07:12:28.737683+0000","last_clean":"2024-04-17T07:12:28.737683+0000","last_became_active":"2024-04-17T07:11:55.480507+0000","last_became_peered":"2024-04-17T07:11:55.480507+0000","last_unstale":"2024-04-17T07:12:28.737683+0000","last_undegraded":"2024-04-17T07:12:28.737683+0000","last_fullsized":"2024-04-17T07:12:28.737683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:42.584180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715898+0000","last_change":"2024-04-17T07:11:59.520897+0000","last_active":"2024-04-17T07:12:28.715898+0000","last_peered":"2024-04-17T07:12:28.715898+0000","last_clean":"2024-04-17T07:12:28.715898+0000","last_became_active":"2024-04-17T07:11:59.520350+0000","last_became_peered":"2024-04-17T07:11:59.520350+0000","last_unstale":"2024-04-17T07:12:28.715898+0000","last_undegraded":"2024-04-17T07:12:28.715898+0000","last_fullsized":"2024-04-17T07:12:28.715898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:39.813097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.222928+0000","last_change":"2024-04-17T07:12:28.753717+0000","last_active":"2024-04-17T07:12:31.222928+0000","last_peered":"2024-04-17T07:12:31.222928+0000","last_clean":"2024-04-17T07:12:31.222928+0000","last_became_active":"2024-04-17T07:12:28.739556+0000","last_became_peered":"2024-04-17T07:12:28.739556+0000","last_unstale":"2024-04-17T07:12:31.222928+0000","last_undegraded":"2024-04-17T07:12:31.222928+0000","last_fullsized":"2024-04-17T07:12:31.222928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:11.967731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.732946+0000","last_change":"2024-04-17T07:11:55.476675+0000","last_active":"2024-04-17T07:12:28.732946+0000","last_peered":"2024-04-17T07:12:28.732946+0000","last_clean":"2024-04-17T07:12:28.732946+0000","last_became_active":"2024-04-17T07:11:55.476273+0000","last_became_peered":"2024-04-17T07:11:55.476273+0000","last_unstale":"2024-04-17T07:12:28.732946+0000","last_undegraded":"2024-04-17T07:12:28.732946+0000","last_fullsized":"2024-04-17T07:12:28.732946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:09.612138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.741537+0000","last_change":"2024-04-17T07:12:28.741537+0000","last_active":"2024-04-17T07:12:28.741537+0000","last_peered":"2024-04-17T07:12:28.741537+0000","last_clean":"2024-04-17T07:12:28.741537+0000","last_became_active":"2024-04-17T07:12:28.741035+0000","last_became_peered":"2024-04-17T07:12:28.741035+0000","last_unstale":"2024-04-17T07:12:28.741537+0000","last_undegraded":"2024-04-17T07:12:28.741537+0000","last_fullsized":"2024-04-17T07:12:28.741537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:15.550112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.195614+0000","last_change":"2024-04-17T07:11:54.471118+0000","last_active":"2024-04-17T07:12:29.195614+0000","last_peered":"2024-04-17T07:12:29.195614+0000","last_clean":"2024-04-17T07:12:29.195614+0000","last_became_active":"2024-04-17T07:11:54.470526+0000","last_became_peered":"2024-04-17T07:11:54.470526+0000","last_unstale":"2024-04-17T07:12:29.195614+0000","last_undegraded":"2024-04-17T07:12:29.195614+0000","last_fullsized":"2024-04-17T07:12:29.195614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:52:47.918086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735651+0000","last_change":"2024-04-17T07:11:55.472850+0000","last_active":"2024-04-17T07:12:28.735651+0000","last_peered":"2024-04-17T07:12:28.735651+0000","last_clean":"2024-04-17T07:12:28.735651+0000","last_became_active":"2024-04-17T07:11:55.472097+0000","last_became_peered":"2024-04-17T07:11:55.472097+0000","last_unstale":"2024-04-17T07:12:28.735651+0000","last_undegraded":"2024-04-17T07:12:28.735651+0000","last_fullsized":"2024-04-17T07:12:28.735651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:41.730809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.737174+0000","last_change":"2024-04-17T07:11:59.522960+0000","last_active":"2024-04-17T07:12:27.737174+0000","last_peered":"2024-04-17T07:12:27.737174+0000","last_clean":"2024-04-17T07:12:27.737174+0000","last_became_active":"2024-04-17T07:11:59.522470+0000","last_became_peered":"2024-04-17T07:11:59.522470+0000","last_unstale":"2024-04-17T07:12:27.737174+0000","last_undegraded":"2024-04-17T07:12:27.737174+0000","last_fullsized":"2024-04-17T07:12:27.737174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:36.620442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720922+0000","last_change":"2024-04-17T07:11:54.470693+0000","last_active":"2024-04-17T07:12:28.720922+0000","last_peered":"2024-04-17T07:12:28.720922+0000","last_clean":"2024-04-17T07:12:28.720922+0000","last_became_active":"2024-04-17T07:11:54.470334+0000","last_became_peered":"2024-04-17T07:11:54.470334+0000","last_unstale":"2024-04-17T07:12:28.720922+0000","last_undegraded":"2024-04-17T07:12:28.720922+0000","last_fullsized":"2024-04-17T07:12:28.720922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:19:04.980786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122063+0000","last_change":"2024-04-17T07:11:55.474755+0000","last_active":"2024-04-17T07:12:29.122063+0000","last_peered":"2024-04-17T07:12:29.122063+0000","last_clean":"2024-04-17T07:12:29.122063+0000","last_became_active":"2024-04-17T07:11:55.474261+0000","last_became_peered":"2024-04-17T07:11:55.474261+0000","last_unstale":"2024-04-17T07:12:29.122063+0000","last_undegraded":"2024-04-17T07:12:29.122063+0000","last_fullsized":"2024-04-17T07:12:29.122063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:00.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735779+0000","last_change":"2024-04-17T07:11:59.517152+0000","last_active":"2024-04-17T07:12:28.735779+0000","last_peered":"2024-04-17T07:12:28.735779+0000","last_clean":"2024-04-17T07:12:28.735779+0000","last_became_active":"2024-04-17T07:11:59.514977+0000","last_became_peered":"2024-04-17T07:11:59.514977+0000","last_unstale":"2024-04-17T07:12:28.735779+0000","last_undegraded":"2024-04-17T07:12:28.735779+0000","last_fullsized":"2024-04-17T07:12:28.735779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:14.974136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.212620+0000","last_change":"2024-04-17T07:11:54.459786+0000","last_active":"2024-04-17T07:12:30.212620+0000","last_peered":"2024-04-17T07:12:30.212620+0000","last_clean":"2024-04-17T07:12:30.212620+0000","last_became_active":"2024-04-17T07:11:54.457589+0000","last_became_peered":"2024-04-17T07:11:54.457589+0000","last_unstale":"2024-04-17T07:12:30.212620+0000","last_undegraded":"2024-04-17T07:12:30.212620+0000","last_fullsized":"2024-04-17T07:12:30.212620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:25.986539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714421+0000","last_change":"2024-04-17T07:11:55.465020+0000","last_active":"2024-04-17T07:12:28.714421+0000","last_peered":"2024-04-17T07:12:28.714421+0000","last_clean":"2024-04-17T07:12:28.714421+0000","last_became_active":"2024-04-17T07:11:55.464608+0000","last_became_peered":"2024-04-17T07:11:55.464608+0000","last_unstale":"2024-04-17T07:12:28.714421+0000","last_undegraded":"2024-04-17T07:12:28.714421+0000","last_fullsized":"2024-04-17T07:12:28.714421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122725+0000","last_change":"2024-04-17T07:11:59.519333+0000","last_active":"2024-04-17T07:12:29.122725+0000","last_peered":"2024-04-17T07:12:29.122725+0000","last_clean":"2024-04-17T07:12:29.122725+0000","last_became_active":"2024-04-17T07:11:59.518865+0000","last_became_peered":"2024-04-17T07:11:59.518865+0000","last_unstale":"2024-04-17T07:12:29.122725+0000","last_undegraded":"2024-04-17T07:12:29.122725+0000","last_fullsized":"2024-04-17T07:12:29.122725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:42.009010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.227304+0000","last_change":"2024-04-17T07:11:54.462283+0000","last_active":"2024-04-17T07:12:32.227304+0000","last_peered":"2024-04-17T07:12:32.227304+0000","last_clean":"2024-04-17T07:12:32.227304+0000","last_became_active":"2024-04-17T07:11:54.461818+0000","last_became_peered":"2024-04-17T07:11:54.461818+0000","last_unstale":"2024-04-17T07:12:32.227304+0000","last_undegraded":"2024-04-17T07:12:32.227304+0000","last_fullsized":"2024-04-17T07:12:32.227304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.765058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726968+0000","last_change":"2024-04-17T07:11:55.486491+0000","last_active":"2024-04-17T07:12:28.726968+0000","last_peered":"2024-04-17T07:12:28.726968+0000","last_clean":"2024-04-17T07:12:28.726968+0000","last_became_active":"2024-04-17T07:11:55.486121+0000","last_became_peered":"2024-04-17T07:11:55.486121+0000","last_unstale":"2024-04-17T07:12:28.726968+0000","last_undegraded":"2024-04-17T07:12:28.726968+0000","last_fullsized":"2024-04-17T07:12:28.726968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:12.998321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713871+0000","last_change":"2024-04-17T07:11:59.511555+0000","last_active":"2024-04-17T07:12:28.713871+0000","last_peered":"2024-04-17T07:12:28.713871+0000","last_clean":"2024-04-17T07:12:28.713871+0000","last_became_active":"2024-04-17T07:11:59.511147+0000","last_became_peered":"2024-04-17T07:11:59.511147+0000","last_unstale":"2024-04-17T07:12:28.713871+0000","last_undegraded":"2024-04-17T07:12:28.713871+0000","last_fullsized":"2024-04-17T07:12:28.713871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:41.854815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.210000+0000","last_change":"2024-04-17T07:12:28.734476+0000","last_active":"2024-04-17T07:12:30.210000+0000","last_peered":"2024-04-17T07:12:30.210000+0000","last_clean":"2024-04-17T07:12:30.210000+0000","last_became_active":"2024-04-17T07:12:28.728187+0000","last_became_peered":"2024-04-17T07:12:28.728187+0000","last_unstale":"2024-04-17T07:12:30.210000+0000","last_undegraded":"2024-04-17T07:12:30.210000+0000","last_fullsized":"2024-04-17T07:12:30.210000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:45.749607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736562+0000","last_change":"2024-04-17T07:11:55.480589+0000","last_active":"2024-04-17T07:12:28.736562+0000","last_peered":"2024-04-17T07:12:28.736562+0000","last_clean":"2024-04-17T07:12:28.736562+0000","last_became_active":"2024-04-17T07:11:55.480224+0000","last_became_peered":"2024-04-17T07:11:55.480224+0000","last_unstale":"2024-04-17T07:12:28.736562+0000","last_undegraded":"2024-04-17T07:12:28.736562+0000","last_fullsized":"2024-04-17T07:12:28.736562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:05.892442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713815+0000","last_change":"2024-04-17T07:11:59.522021+0000","last_active":"2024-04-17T07:12:28.713815+0000","last_peered":"2024-04-17T07:12:28.713815+0000","last_clean":"2024-04-17T07:12:28.713815+0000","last_became_active":"2024-04-17T07:11:59.521493+0000","last_became_peered":"2024-04-17T07:11:59.521493+0000","last_unstale":"2024-04-17T07:12:28.713815+0000","last_undegraded":"2024-04-17T07:12:28.713815+0000","last_fullsized":"2024-04-17T07:12:28.713815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:20.386216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.195512+0000","last_change":"2024-04-17T07:11:54.468225+0000","last_active":"2024-04-17T07:12:29.195512+0000","last_peered":"2024-04-17T07:12:29.195512+0000","last_clean":"2024-04-17T07:12:29.195512+0000","last_became_active":"2024-04-17T07:11:54.467046+0000","last_became_peered":"2024-04-17T07:11:54.467046+0000","last_unstale":"2024-04-17T07:12:29.195512+0000","last_undegraded":"2024-04-17T07:12:29.195512+0000","last_fullsized":"2024-04-17T07:12:29.195512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:17.725255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726428+0000","last_change":"2024-04-17T07:11:55.488980+0000","last_active":"2024-04-17T07:12:28.726428+0000","last_peered":"2024-04-17T07:12:28.726428+0000","last_clean":"2024-04-17T07:12:28.726428+0000","last_became_active":"2024-04-17T07:11:55.488226+0000","last_became_peered":"2024-04-17T07:11:55.488226+0000","last_unstale":"2024-04-17T07:12:28.726428+0000","last_undegraded":"2024-04-17T07:12:28.726428+0000","last_fullsized":"2024-04-17T07:12:28.726428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:00.135973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714388+0000","last_change":"2024-04-17T07:11:59.515134+0000","last_active":"2024-04-17T07:12:28.714388+0000","last_peered":"2024-04-17T07:12:28.714388+0000","last_clean":"2024-04-17T07:12:28.714388+0000","last_became_active":"2024-04-17T07:11:59.514560+0000","last_became_peered":"2024-04-17T07:11:59.514560+0000","last_unstale":"2024-04-17T07:12:28.714388+0000","last_undegraded":"2024-04-17T07:12:28.714388+0000","last_fullsized":"2024-04-17T07:12:28.714388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:34.669789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'26","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.230037+0000","last_change":"2024-04-17T07:11:54.461226+0000","last_active":"2024-04-17T07:12:32.230037+0000","last_peered":"2024-04-17T07:12:32.230037+0000","last_clean":"2024-04-17T07:12:32.230037+0000","last_became_active":"2024-04-17T07:11:54.460573+0000","last_became_peered":"2024-04-17T07:11:54.460573+0000","last_unstale":"2024-04-17T07:12:32.230037+0000","last_undegraded":"2024-04-17T07:12:32.230037+0000","last_fullsized":"2024-04-17T07:12:32.230037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":44,"num_write":28,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.734828+0000","last_change":"2024-04-17T07:11:55.483925+0000","last_active":"2024-04-17T07:12:28.734828+0000","last_peered":"2024-04-17T07:12:28.734828+0000","last_clean":"2024-04-17T07:12:28.734828+0000","last_became_active":"2024-04-17T07:11:55.483584+0000","last_became_peered":"2024-04-17T07:11:55.483584+0000","last_unstale":"2024-04-17T07:12:28.734828+0000","last_undegraded":"2024-04-17T07:12:28.734828+0000","last_fullsized":"2024-04-17T07:12:28.734828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:12.328876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.737367+0000","last_change":"2024-04-17T07:11:59.515597+0000","last_active":"2024-04-17T07:12:28.737367+0000","last_peered":"2024-04-17T07:12:28.737367+0000","last_clean":"2024-04-17T07:12:28.737367+0000","last_became_active":"2024-04-17T07:11:59.514413+0000","last_became_peered":"2024-04-17T07:11:59.514413+0000","last_unstale":"2024-04-17T07:12:28.737367+0000","last_undegraded":"2024-04-17T07:12:28.737367+0000","last_fullsized":"2024-04-17T07:12:28.737367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:45.824127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.209717+0000","last_change":"2024-04-17T07:11:54.449984+0000","last_active":"2024-04-17T07:12:30.209717+0000","last_peered":"2024-04-17T07:12:30.209717+0000","last_clean":"2024-04-17T07:12:30.209717+0000","last_became_active":"2024-04-17T07:11:54.449628+0000","last_became_peered":"2024-04-17T07:11:54.449628+0000","last_unstale":"2024-04-17T07:12:30.209717+0000","last_undegraded":"2024-04-17T07:12:30.209717+0000","last_fullsized":"2024-04-17T07:12:30.209717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:26:03.281684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719923+0000","last_change":"2024-04-17T07:11:55.476912+0000","last_active":"2024-04-17T07:12:28.719923+0000","last_peered":"2024-04-17T07:12:28.719923+0000","last_clean":"2024-04-17T07:12:28.719923+0000","last_became_active":"2024-04-17T07:11:55.471576+0000","last_became_peered":"2024-04-17T07:11:55.471576+0000","last_unstale":"2024-04-17T07:12:28.719923+0000","last_undegraded":"2024-04-17T07:12:28.719923+0000","last_fullsized":"2024-04-17T07:12:28.719923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:35.439271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.733057+0000","last_change":"2024-04-17T07:11:59.515897+0000","last_active":"2024-04-17T07:12:27.733057+0000","last_peered":"2024-04-17T07:12:27.733057+0000","last_clean":"2024-04-17T07:12:27.733057+0000","last_became_active":"2024-04-17T07:11:59.515349+0000","last_became_peered":"2024-04-17T07:11:59.515349+0000","last_unstale":"2024-04-17T07:12:27.733057+0000","last_undegraded":"2024-04-17T07:12:27.733057+0000","last_fullsized":"2024-04-17T07:12:27.733057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:00.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214859+0000","last_change":"2024-04-17T07:11:54.462369+0000","last_active":"2024-04-17T07:12:30.214859+0000","last_peered":"2024-04-17T07:12:30.214859+0000","last_clean":"2024-04-17T07:12:30.214859+0000","last_became_active":"2024-04-17T07:11:54.460357+0000","last_became_peered":"2024-04-17T07:11:54.460357+0000","last_unstale":"2024-04-17T07:12:30.214859+0000","last_undegraded":"2024-04-17T07:12:30.214859+0000","last_fullsized":"2024-04-17T07:12:30.214859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:38.029980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720650+0000","last_change":"2024-04-17T07:11:55.475874+0000","last_active":"2024-04-17T07:12:28.720650+0000","last_peered":"2024-04-17T07:12:28.720650+0000","last_clean":"2024-04-17T07:12:28.720650+0000","last_became_active":"2024-04-17T07:11:55.475550+0000","last_became_peered":"2024-04-17T07:11:55.475550+0000","last_unstale":"2024-04-17T07:12:28.720650+0000","last_undegraded":"2024-04-17T07:12:28.720650+0000","last_fullsized":"2024-04-17T07:12:28.720650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:40.096573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714893+0000","last_change":"2024-04-17T07:11:59.514661+0000","last_active":"2024-04-17T07:12:28.714893+0000","last_peered":"2024-04-17T07:12:28.714893+0000","last_clean":"2024-04-17T07:12:28.714893+0000","last_became_active":"2024-04-17T07:11:59.513030+0000","last_became_peered":"2024-04-17T07:11:59.513030+0000","last_unstale":"2024-04-17T07:12:28.714893+0000","last_undegraded":"2024-04-17T07:12:28.714893+0000","last_fullsized":"2024-04-17T07:12:28.714893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:35.696961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713262+0000","last_change":"2024-04-17T07:11:54.455279+0000","last_active":"2024-04-17T07:12:28.713262+0000","last_peered":"2024-04-17T07:12:28.713262+0000","last_clean":"2024-04-17T07:12:28.713262+0000","last_became_active":"2024-04-17T07:11:54.453620+0000","last_became_peered":"2024-04-17T07:11:54.453620+0000","last_unstale":"2024-04-17T07:12:28.713262+0000","last_undegraded":"2024-04-17T07:12:28.713262+0000","last_fullsized":"2024-04-17T07:12:28.713262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:15.227410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719686+0000","last_change":"2024-04-17T07:11:55.469579+0000","last_active":"2024-04-17T07:12:28.719686+0000","last_peered":"2024-04-17T07:12:28.719686+0000","last_clean":"2024-04-17T07:12:28.719686+0000","last_became_active":"2024-04-17T07:11:55.468389+0000","last_became_peered":"2024-04-17T07:11:55.468389+0000","last_unstale":"2024-04-17T07:12:28.719686+0000","last_undegraded":"2024-04-17T07:12:28.719686+0000","last_fullsized":"2024-04-17T07:12:28.719686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:01.274677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720385+0000","last_change":"2024-04-17T07:11:59.512110+0000","last_active":"2024-04-17T07:12:28.720385+0000","last_peered":"2024-04-17T07:12:28.720385+0000","last_clean":"2024-04-17T07:12:28.720385+0000","last_became_active":"2024-04-17T07:11:59.510573+0000","last_became_peered":"2024-04-17T07:11:59.510573+0000","last_unstale":"2024-04-17T07:12:28.720385+0000","last_undegraded":"2024-04-17T07:12:28.720385+0000","last_fullsized":"2024-04-17T07:12:28.720385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:32.515687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'10","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-17T07:12:00.800050+0000","last_change":"2024-04-17T07:12:28.153142+0000","last_active":"2024-04-17T07:12:28.153142+0000","last_peered":"2024-04-17T07:12:00.800050+0000","last_clean":"2024-04-17T07:12:00.800050+0000","last_became_active":"2024-04-17T07:11:54.471537+0000","last_became_peered":"2024-04-17T07:11:54.471537+0000","last_unstale":"2024-04-17T07:12:00.800050+0000","last_undegraded":"2024-04-17T07:12:00.800050+0000","last_fullsized":"2024-04-17T07:12:00.800050+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":5,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.731088+0000","last_change":"2024-04-17T07:11:55.473540+0000","last_active":"2024-04-17T07:12:28.731088+0000","last_peered":"2024-04-17T07:12:28.731088+0000","last_clean":"2024-04-17T07:12:28.731088+0000","last_became_active":"2024-04-17T07:11:55.472436+0000","last_became_peered":"2024-04-17T07:11:55.472436+0000","last_unstale":"2024-04-17T07:12:28.731088+0000","last_undegraded":"2024-04-17T07:12:28.731088+0000","last_fullsized":"2024-04-17T07:12:28.731088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:54.317052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730908+0000","last_change":"2024-04-17T07:11:59.521305+0000","last_active":"2024-04-17T07:12:28.730908+0000","last_peered":"2024-04-17T07:12:28.730908+0000","last_clean":"2024-04-17T07:12:28.730908+0000","last_became_active":"2024-04-17T07:11:59.516860+0000","last_became_peered":"2024-04-17T07:11:59.516860+0000","last_unstale":"2024-04-17T07:12:28.730908+0000","last_undegraded":"2024-04-17T07:12:28.730908+0000","last_fullsized":"2024-04-17T07:12:28.730908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:23.403218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735984+0000","last_change":"2024-04-17T07:11:54.463378+0000","last_active":"2024-04-17T07:12:28.735984+0000","last_peered":"2024-04-17T07:12:28.735984+0000","last_clean":"2024-04-17T07:12:28.735984+0000","last_became_active":"2024-04-17T07:11:54.462120+0000","last_became_peered":"2024-04-17T07:11:54.462120+0000","last_unstale":"2024-04-17T07:12:28.735984+0000","last_undegraded":"2024-04-17T07:12:28.735984+0000","last_fullsized":"2024-04-17T07:12:28.735984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:42.109928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.737028+0000","last_change":"2024-04-17T07:11:55.478604+0000","last_active":"2024-04-17T07:12:27.737028+0000","last_peered":"2024-04-17T07:12:27.737028+0000","last_clean":"2024-04-17T07:12:27.737028+0000","last_became_active":"2024-04-17T07:11:55.478252+0000","last_became_peered":"2024-04-17T07:11:55.478252+0000","last_unstale":"2024-04-17T07:12:27.737028+0000","last_undegraded":"2024-04-17T07:12:27.737028+0000","last_fullsized":"2024-04-17T07:12:27.737028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:08.365896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727344+0000","last_change":"2024-04-17T07:11:59.518619+0000","last_active":"2024-04-17T07:12:28.727344+0000","last_peered":"2024-04-17T07:12:28.727344+0000","last_clean":"2024-04-17T07:12:28.727344+0000","last_became_active":"2024-04-17T07:11:59.518214+0000","last_became_peered":"2024-04-17T07:11:59.518214+0000","last_unstale":"2024-04-17T07:12:28.727344+0000","last_undegraded":"2024-04-17T07:12:28.727344+0000","last_fullsized":"2024-04-17T07:12:28.727344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:46.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715522+0000","last_change":"2024-04-17T07:11:54.453038+0000","last_active":"2024-04-17T07:12:28.715522+0000","last_peered":"2024-04-17T07:12:28.715522+0000","last_clean":"2024-04-17T07:12:28.715522+0000","last_became_active":"2024-04-17T07:11:54.452619+0000","last_became_peered":"2024-04-17T07:11:54.452619+0000","last_unstale":"2024-04-17T07:12:28.715522+0000","last_undegraded":"2024-04-17T07:12:28.715522+0000","last_fullsized":"2024-04-17T07:12:28.715522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:25.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.724948+0000","last_change":"2024-04-17T07:11:55.465990+0000","last_active":"2024-04-17T07:12:28.724948+0000","last_peered":"2024-04-17T07:12:28.724948+0000","last_clean":"2024-04-17T07:12:28.724948+0000","last_became_active":"2024-04-17T07:11:55.465523+0000","last_became_peered":"2024-04-17T07:11:55.465523+0000","last_unstale":"2024-04-17T07:12:28.724948+0000","last_undegraded":"2024-04-17T07:12:28.724948+0000","last_fullsized":"2024-04-17T07:12:28.724948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:40.366027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721070+0000","last_change":"2024-04-17T07:11:59.512119+0000","last_active":"2024-04-17T07:12:28.721070+0000","last_peered":"2024-04-17T07:12:28.721070+0000","last_clean":"2024-04-17T07:12:28.721070+0000","last_became_active":"2024-04-17T07:11:59.511460+0000","last_became_peered":"2024-04-17T07:11:59.511460+0000","last_unstale":"2024-04-17T07:12:28.721070+0000","last_undegraded":"2024-04-17T07:12:28.721070+0000","last_fullsized":"2024-04-17T07:12:28.721070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:50.888203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.221574+0000","last_change":"2024-04-17T07:11:54.469346+0000","last_active":"2024-04-17T07:12:31.221574+0000","last_peered":"2024-04-17T07:12:31.221574+0000","last_clean":"2024-04-17T07:12:31.221574+0000","last_became_active":"2024-04-17T07:11:54.468947+0000","last_became_peered":"2024-04-17T07:11:54.468947+0000","last_unstale":"2024-04-17T07:12:31.221574+0000","last_undegraded":"2024-04-17T07:12:31.221574+0000","last_fullsized":"2024-04-17T07:12:31.221574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:04.854625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.724510+0000","last_change":"2024-04-17T07:11:55.471235+0000","last_active":"2024-04-17T07:12:28.724510+0000","last_peered":"2024-04-17T07:12:28.724510+0000","last_clean":"2024-04-17T07:12:28.724510+0000","last_became_active":"2024-04-17T07:11:55.470857+0000","last_became_peered":"2024-04-17T07:11:55.470857+0000","last_unstale":"2024-04-17T07:12:28.724510+0000","last_undegraded":"2024-04-17T07:12:28.724510+0000","last_fullsized":"2024-04-17T07:12:28.724510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:21.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720514+0000","last_change":"2024-04-17T07:11:59.524519+0000","last_active":"2024-04-17T07:12:28.720514+0000","last_peered":"2024-04-17T07:12:28.720514+0000","last_clean":"2024-04-17T07:12:28.720514+0000","last_became_active":"2024-04-17T07:11:59.519270+0000","last_became_peered":"2024-04-17T07:11:59.519270+0000","last_unstale":"2024-04-17T07:12:28.720514+0000","last_undegraded":"2024-04-17T07:12:28.720514+0000","last_fullsized":"2024-04-17T07:12:28.720514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:41.080249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.734024+0000","last_change":"2024-04-17T07:11:55.473922+0000","last_active":"2024-04-17T07:12:28.734024+0000","last_peered":"2024-04-17T07:12:28.734024+0000","last_clean":"2024-04-17T07:12:28.734024+0000","last_became_active":"2024-04-17T07:11:55.468594+0000","last_became_peered":"2024-04-17T07:11:55.468594+0000","last_unstale":"2024-04-17T07:12:28.734024+0000","last_undegraded":"2024-04-17T07:12:28.734024+0000","last_fullsized":"2024-04-17T07:12:28.734024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:21.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.219511+0000","last_change":"2024-04-17T07:11:54.473066+0000","last_active":"2024-04-17T07:12:31.219511+0000","last_peered":"2024-04-17T07:12:31.219511+0000","last_clean":"2024-04-17T07:12:31.219511+0000","last_became_active":"2024-04-17T07:11:54.472708+0000","last_became_peered":"2024-04-17T07:11:54.472708+0000","last_unstale":"2024-04-17T07:12:31.219511+0000","last_undegraded":"2024-04-17T07:12:31.219511+0000","last_fullsized":"2024-04-17T07:12:31.219511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:27.614819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720813+0000","last_change":"2024-04-17T07:11:59.516939+0000","last_active":"2024-04-17T07:12:28.720813+0000","last_peered":"2024-04-17T07:12:28.720813+0000","last_clean":"2024-04-17T07:12:28.720813+0000","last_became_active":"2024-04-17T07:11:59.516333+0000","last_became_peered":"2024-04-17T07:11:59.516333+0000","last_unstale":"2024-04-17T07:12:28.720813+0000","last_undegraded":"2024-04-17T07:12:28.720813+0000","last_fullsized":"2024-04-17T07:12:28.720813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:15.648831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122635+0000","last_change":"2024-04-17T07:11:55.481006+0000","last_active":"2024-04-17T07:12:29.122635+0000","last_peered":"2024-04-17T07:12:29.122635+0000","last_clean":"2024-04-17T07:12:29.122635+0000","last_became_active":"2024-04-17T07:11:55.479751+0000","last_became_peered":"2024-04-17T07:11:55.479751+0000","last_unstale":"2024-04-17T07:12:29.122635+0000","last_undegraded":"2024-04-17T07:12:29.122635+0000","last_fullsized":"2024-04-17T07:12:29.122635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:13.004650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713874+0000","last_change":"2024-04-17T07:11:54.463106+0000","last_active":"2024-04-17T07:12:28.713874+0000","last_peered":"2024-04-17T07:12:28.713874+0000","last_clean":"2024-04-17T07:12:28.713874+0000","last_became_active":"2024-04-17T07:11:54.462691+0000","last_became_peered":"2024-04-17T07:11:54.462691+0000","last_unstale":"2024-04-17T07:12:28.713874+0000","last_undegraded":"2024-04-17T07:12:28.713874+0000","last_fullsized":"2024-04-17T07:12:28.713874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:34.302019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719632+0000","last_change":"2024-04-17T07:10:14.685883+0000","last_active":"2024-04-17T07:12:28.719632+0000","last_peered":"2024-04-17T07:12:28.719632+0000","last_clean":"2024-04-17T07:12:28.719632+0000","last_became_active":"2024-04-17T07:10:14.684551+0000","last_became_peered":"2024-04-17T07:10:14.684551+0000","last_unstale":"2024-04-17T07:12:28.719632+0000","last_undegraded":"2024-04-17T07:12:28.719632+0000","last_fullsized":"2024-04-17T07:12:28.719632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_clean_scrub_stamp":"2024-04-17T07:05:37.112317+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:04.265147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736804+0000","last_change":"2024-04-17T07:12:28.731807+0000","last_active":"2024-04-17T07:12:28.736804+0000","last_peered":"2024-04-17T07:12:28.736804+0000","last_clean":"2024-04-17T07:12:28.736804+0000","last_became_active":"2024-04-17T07:12:28.730656+0000","last_became_peered":"2024-04-17T07:12:28.730656+0000","last_unstale":"2024-04-17T07:12:28.736804+0000","last_undegraded":"2024-04-17T07:12:28.736804+0000","last_fullsized":"2024-04-17T07:12:28.736804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:58.023333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727980+0000","last_change":"2024-04-17T07:12:28.727980+0000","last_active":"2024-04-17T07:12:28.727980+0000","last_peered":"2024-04-17T07:12:28.727980+0000","last_clean":"2024-04-17T07:12:28.727980+0000","last_became_active":"2024-04-17T07:12:28.727060+0000","last_became_peered":"2024-04-17T07:12:28.727060+0000","last_unstale":"2024-04-17T07:12:28.727980+0000","last_undegraded":"2024-04-17T07:12:28.727980+0000","last_fullsized":"2024-04-17T07:12:28.727980+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:33.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719921+0000","last_change":"2024-04-17T07:11:54.463180+0000","last_active":"2024-04-17T07:12:28.719921+0000","last_peered":"2024-04-17T07:12:28.719921+0000","last_clean":"2024-04-17T07:12:28.719921+0000","last_became_active":"2024-04-17T07:11:54.462793+0000","last_became_peered":"2024-04-17T07:11:54.462793+0000","last_unstale":"2024-04-17T07:12:28.719921+0000","last_undegraded":"2024-04-17T07:12:28.719921+0000","last_fullsized":"2024-04-17T07:12:28.719921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:19.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.733730+0000","last_change":"2024-04-17T07:11:59.518355+0000","last_active":"2024-04-17T07:12:27.733730+0000","last_peered":"2024-04-17T07:12:27.733730+0000","last_clean":"2024-04-17T07:12:27.733730+0000","last_became_active":"2024-04-17T07:11:59.517619+0000","last_became_peered":"2024-04-17T07:11:59.517619+0000","last_unstale":"2024-04-17T07:12:27.733730+0000","last_undegraded":"2024-04-17T07:12:27.733730+0000","last_fullsized":"2024-04-17T07:12:27.733730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:36.826021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.196898+0000","last_change":"2024-04-17T07:11:54.465285+0000","last_active":"2024-04-17T07:12:29.196898+0000","last_peered":"2024-04-17T07:12:29.196898+0000","last_clean":"2024-04-17T07:12:29.196898+0000","last_became_active":"2024-04-17T07:11:54.464818+0000","last_became_peered":"2024-04-17T07:11:54.464818+0000","last_unstale":"2024-04-17T07:12:29.196898+0000","last_undegraded":"2024-04-17T07:12:29.196898+0000","last_fullsized":"2024-04-17T07:12:29.196898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721576+0000","last_change":"2024-04-17T07:11:55.470405+0000","last_active":"2024-04-17T07:12:28.721576+0000","last_peered":"2024-04-17T07:12:28.721576+0000","last_clean":"2024-04-17T07:12:28.721576+0000","last_became_active":"2024-04-17T07:11:55.470058+0000","last_became_peered":"2024-04-17T07:11:55.470058+0000","last_unstale":"2024-04-17T07:12:28.721576+0000","last_undegraded":"2024-04-17T07:12:28.721576+0000","last_fullsized":"2024-04-17T07:12:28.721576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:17.416775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122022+0000","last_change":"2024-04-17T07:11:59.522040+0000","last_active":"2024-04-17T07:12:29.122022+0000","last_peered":"2024-04-17T07:12:29.122022+0000","last_clean":"2024-04-17T07:12:29.122022+0000","last_became_active":"2024-04-17T07:11:59.519771+0000","last_became_peered":"2024-04-17T07:11:59.519771+0000","last_unstale":"2024-04-17T07:12:29.122022+0000","last_undegraded":"2024-04-17T07:12:29.122022+0000","last_fullsized":"2024-04-17T07:12:29.122022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:58.203737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.488533+0000","last_change":"2024-04-17T07:11:54.457954+0000","last_active":"2024-04-17T07:12:30.488533+0000","last_peered":"2024-04-17T07:12:30.488533+0000","last_clean":"2024-04-17T07:12:30.488533+0000","last_became_active":"2024-04-17T07:11:54.457560+0000","last_became_peered":"2024-04-17T07:11:54.457560+0000","last_unstale":"2024-04-17T07:12:30.488533+0000","last_undegraded":"2024-04-17T07:12:30.488533+0000","last_fullsized":"2024-04-17T07:12:30.488533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:31.919730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123098+0000","last_change":"2024-04-17T07:11:55.480475+0000","last_active":"2024-04-17T07:12:29.123098+0000","last_peered":"2024-04-17T07:12:29.123098+0000","last_clean":"2024-04-17T07:12:29.123098+0000","last_became_active":"2024-04-17T07:11:55.480060+0000","last_became_peered":"2024-04-17T07:11:55.480060+0000","last_unstale":"2024-04-17T07:12:29.123098+0000","last_undegraded":"2024-04-17T07:12:29.123098+0000","last_fullsized":"2024-04-17T07:12:29.123098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:30:23.600244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714322+0000","last_change":"2024-04-17T07:11:59.525422+0000","last_active":"2024-04-17T07:12:28.714322+0000","last_peered":"2024-04-17T07:12:28.714322+0000","last_clean":"2024-04-17T07:12:28.714322+0000","last_became_active":"2024-04-17T07:11:59.523649+0000","last_became_peered":"2024-04-17T07:11:59.523649+0000","last_unstale":"2024-04-17T07:12:28.714322+0000","last_undegraded":"2024-04-17T07:12:28.714322+0000","last_fullsized":"2024-04-17T07:12:28.714322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:16.653752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714912+0000","last_change":"2024-04-17T07:11:54.465824+0000","last_active":"2024-04-17T07:12:28.714912+0000","last_peered":"2024-04-17T07:12:28.714912+0000","last_clean":"2024-04-17T07:12:28.714912+0000","last_became_active":"2024-04-17T07:11:54.465348+0000","last_became_peered":"2024-04-17T07:11:54.465348+0000","last_unstale":"2024-04-17T07:12:28.714912+0000","last_undegraded":"2024-04-17T07:12:28.714912+0000","last_fullsized":"2024-04-17T07:12:28.714912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:17.729903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736284+0000","last_change":"2024-04-17T07:11:55.481666+0000","last_active":"2024-04-17T07:12:28.736284+0000","last_peered":"2024-04-17T07:12:28.736284+0000","last_clean":"2024-04-17T07:12:28.736284+0000","last_became_active":"2024-04-17T07:11:55.480332+0000","last_became_peered":"2024-04-17T07:11:55.480332+0000","last_unstale":"2024-04-17T07:12:28.736284+0000","last_undegraded":"2024-04-17T07:12:28.736284+0000","last_fullsized":"2024-04-17T07:12:28.736284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:19.825407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714433+0000","last_change":"2024-04-17T07:11:59.527850+0000","last_active":"2024-04-17T07:12:28.714433+0000","last_peered":"2024-04-17T07:12:28.714433+0000","last_clean":"2024-04-17T07:12:28.714433+0000","last_became_active":"2024-04-17T07:11:59.527306+0000","last_became_peered":"2024-04-17T07:11:59.527306+0000","last_unstale":"2024-04-17T07:12:28.714433+0000","last_undegraded":"2024-04-17T07:12:28.714433+0000","last_fullsized":"2024-04-17T07:12:28.714433+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:35.906222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725682+0000","last_change":"2024-04-17T07:11:54.455862+0000","last_active":"2024-04-17T07:12:28.725682+0000","last_peered":"2024-04-17T07:12:28.725682+0000","last_clean":"2024-04-17T07:12:28.725682+0000","last_became_active":"2024-04-17T07:11:54.453904+0000","last_became_peered":"2024-04-17T07:11:54.453904+0000","last_unstale":"2024-04-17T07:12:28.725682+0000","last_undegraded":"2024-04-17T07:12:28.725682+0000","last_fullsized":"2024-04-17T07:12:28.725682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:01.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725079+0000","last_change":"2024-04-17T07:11:55.470315+0000","last_active":"2024-04-17T07:12:28.725079+0000","last_peered":"2024-04-17T07:12:28.725079+0000","last_clean":"2024-04-17T07:12:28.725079+0000","last_became_active":"2024-04-17T07:11:55.469829+0000","last_became_peered":"2024-04-17T07:11:55.469829+0000","last_unstale":"2024-04-17T07:12:28.725079+0000","last_undegraded":"2024-04-17T07:12:28.725079+0000","last_fullsized":"2024-04-17T07:12:28.725079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:48.197546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.126045+0000","last_change":"2024-04-17T07:12:29.125447+0000","last_active":"2024-04-17T07:12:29.126045+0000","last_peered":"2024-04-17T07:12:29.126045+0000","last_clean":"2024-04-17T07:12:29.126045+0000","last_became_active":"2024-04-17T07:12:29.124551+0000","last_became_peered":"2024-04-17T07:12:29.124551+0000","last_unstale":"2024-04-17T07:12:29.126045+0000","last_undegraded":"2024-04-17T07:12:29.126045+0000","last_fullsized":"2024-04-17T07:12:29.126045+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:09.489755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.216091+0000","last_change":"2024-04-17T07:11:54.453117+0000","last_active":"2024-04-17T07:12:30.216091+0000","last_peered":"2024-04-17T07:12:30.216091+0000","last_clean":"2024-04-17T07:12:30.216091+0000","last_became_active":"2024-04-17T07:11:54.452632+0000","last_became_peered":"2024-04-17T07:11:54.452632+0000","last_unstale":"2024-04-17T07:12:30.216091+0000","last_undegraded":"2024-04-17T07:12:30.216091+0000","last_fullsized":"2024-04-17T07:12:30.216091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:09.366149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.728073+0000","last_change":"2024-04-17T07:11:55.469991+0000","last_active":"2024-04-17T07:12:27.728073+0000","last_peered":"2024-04-17T07:12:27.728073+0000","last_clean":"2024-04-17T07:12:27.728073+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:12:27.728073+0000","last_undegraded":"2024-04-17T07:12:27.728073+0000","last_fullsized":"2024-04-17T07:12:27.728073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:02.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713914+0000","last_change":"2024-04-17T07:11:59.519817+0000","last_active":"2024-04-17T07:12:28.713914+0000","last_peered":"2024-04-17T07:12:28.713914+0000","last_clean":"2024-04-17T07:12:28.713914+0000","last_became_active":"2024-04-17T07:11:59.519108+0000","last_became_peered":"2024-04-17T07:11:59.519108+0000","last_unstale":"2024-04-17T07:12:28.713914+0000","last_undegraded":"2024-04-17T07:12:28.713914+0000","last_fullsized":"2024-04-17T07:12:28.713914+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:58.378837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716195+0000","last_change":"2024-04-17T07:11:54.463618+0000","last_active":"2024-04-17T07:12:28.716195+0000","last_peered":"2024-04-17T07:12:28.716195+0000","last_clean":"2024-04-17T07:12:28.716195+0000","last_became_active":"2024-04-17T07:11:54.463257+0000","last_became_peered":"2024-04-17T07:11:54.463257+0000","last_unstale":"2024-04-17T07:12:28.716195+0000","last_undegraded":"2024-04-17T07:12:28.716195+0000","last_fullsized":"2024-04-17T07:12:28.716195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:20.135338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122992+0000","last_change":"2024-04-17T07:11:55.487471+0000","last_active":"2024-04-17T07:12:29.122992+0000","last_peered":"2024-04-17T07:12:29.122992+0000","last_clean":"2024-04-17T07:12:29.122992+0000","last_became_active":"2024-04-17T07:11:55.487024+0000","last_became_peered":"2024-04-17T07:11:55.487024+0000","last_unstale":"2024-04-17T07:12:29.122992+0000","last_undegraded":"2024-04-17T07:12:29.122992+0000","last_fullsized":"2024-04-17T07:12:29.122992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:36.851210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714472+0000","last_change":"2024-04-17T07:11:59.527017+0000","last_active":"2024-04-17T07:12:28.714472+0000","last_peered":"2024-04-17T07:12:28.714472+0000","last_clean":"2024-04-17T07:12:28.714472+0000","last_became_active":"2024-04-17T07:11:59.526541+0000","last_became_peered":"2024-04-17T07:11:59.526541+0000","last_unstale":"2024-04-17T07:12:28.714472+0000","last_undegraded":"2024-04-17T07:12:28.714472+0000","last_fullsized":"2024-04-17T07:12:28.714472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:54.411647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.728776+0000","last_change":"2024-04-17T07:11:54.465671+0000","last_active":"2024-04-17T07:12:28.728776+0000","last_peered":"2024-04-17T07:12:28.728776+0000","last_clean":"2024-04-17T07:12:28.728776+0000","last_became_active":"2024-04-17T07:11:54.464902+0000","last_became_peered":"2024-04-17T07:11:54.464902+0000","last_unstale":"2024-04-17T07:12:28.728776+0000","last_undegraded":"2024-04-17T07:12:28.728776+0000","last_fullsized":"2024-04-17T07:12:28.728776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:13:38.232965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720011+0000","last_change":"2024-04-17T07:11:55.484259+0000","last_active":"2024-04-17T07:12:28.720011+0000","last_peered":"2024-04-17T07:12:28.720011+0000","last_clean":"2024-04-17T07:12:28.720011+0000","last_became_active":"2024-04-17T07:11:55.483911+0000","last_became_peered":"2024-04-17T07:11:55.483911+0000","last_unstale":"2024-04-17T07:12:28.720011+0000","last_undegraded":"2024-04-17T07:12:28.720011+0000","last_fullsized":"2024-04-17T07:12:28.720011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:54.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725852+0000","last_change":"2024-04-17T07:11:59.525369+0000","last_active":"2024-04-17T07:12:28.725852+0000","last_peered":"2024-04-17T07:12:28.725852+0000","last_clean":"2024-04-17T07:12:28.725852+0000","last_became_active":"2024-04-17T07:11:59.524809+0000","last_became_peered":"2024-04-17T07:11:59.524809+0000","last_unstale":"2024-04-17T07:12:28.725852+0000","last_undegraded":"2024-04-17T07:12:28.725852+0000","last_fullsized":"2024-04-17T07:12:28.725852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:20.511001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.197527+0000","last_change":"2024-04-17T07:11:54.459079+0000","last_active":"2024-04-17T07:12:29.197527+0000","last_peered":"2024-04-17T07:12:29.197527+0000","last_clean":"2024-04-17T07:12:29.197527+0000","last_became_active":"2024-04-17T07:11:54.457008+0000","last_became_peered":"2024-04-17T07:11:54.457008+0000","last_unstale":"2024-04-17T07:12:29.197527+0000","last_undegraded":"2024-04-17T07:12:29.197527+0000","last_fullsized":"2024-04-17T07:12:29.197527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:36.084512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721123+0000","last_change":"2024-04-17T07:11:55.484453+0000","last_active":"2024-04-17T07:12:28.721123+0000","last_peered":"2024-04-17T07:12:28.721123+0000","last_clean":"2024-04-17T07:12:28.721123+0000","last_became_active":"2024-04-17T07:11:55.483152+0000","last_became_peered":"2024-04-17T07:11:55.483152+0000","last_unstale":"2024-04-17T07:12:28.721123+0000","last_undegraded":"2024-04-17T07:12:28.721123+0000","last_fullsized":"2024-04-17T07:12:28.721123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:32.224868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735401+0000","last_change":"2024-04-17T07:11:59.521487+0000","last_active":"2024-04-17T07:12:28.735401+0000","last_peered":"2024-04-17T07:12:28.735401+0000","last_clean":"2024-04-17T07:12:28.735401+0000","last_became_active":"2024-04-17T07:11:59.521060+0000","last_became_peered":"2024-04-17T07:11:59.521060+0000","last_unstale":"2024-04-17T07:12:28.735401+0000","last_undegraded":"2024-04-17T07:12:28.735401+0000","last_fullsized":"2024-04-17T07:12:28.735401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:06.692434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214903+0000","last_change":"2024-04-17T07:11:54.459368+0000","last_active":"2024-04-17T07:12:30.214903+0000","last_peered":"2024-04-17T07:12:30.214903+0000","last_clean":"2024-04-17T07:12:30.214903+0000","last_became_active":"2024-04-17T07:11:54.458973+0000","last_became_peered":"2024-04-17T07:11:54.458973+0000","last_unstale":"2024-04-17T07:12:30.214903+0000","last_undegraded":"2024-04-17T07:12:30.214903+0000","last_fullsized":"2024-04-17T07:12:30.214903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:22.034110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721530+0000","last_change":"2024-04-17T07:11:55.470648+0000","last_active":"2024-04-17T07:12:28.721530+0000","last_peered":"2024-04-17T07:12:28.721530+0000","last_clean":"2024-04-17T07:12:28.721530+0000","last_became_active":"2024-04-17T07:11:55.469666+0000","last_became_peered":"2024-04-17T07:11:55.469666+0000","last_unstale":"2024-04-17T07:12:28.721530+0000","last_undegraded":"2024-04-17T07:12:28.721530+0000","last_fullsized":"2024-04-17T07:12:28.721530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:59.208872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730680+0000","last_change":"2024-04-17T07:11:59.524027+0000","last_active":"2024-04-17T07:12:28.730680+0000","last_peered":"2024-04-17T07:12:28.730680+0000","last_clean":"2024-04-17T07:12:28.730680+0000","last_became_active":"2024-04-17T07:11:59.522764+0000","last_became_peered":"2024-04-17T07:11:59.522764+0000","last_unstale":"2024-04-17T07:12:28.730680+0000","last_undegraded":"2024-04-17T07:12:28.730680+0000","last_fullsized":"2024-04-17T07:12:28.730680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:40.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.216540+0000","last_change":"2024-04-17T07:11:54.472884+0000","last_active":"2024-04-17T07:12:30.216540+0000","last_peered":"2024-04-17T07:12:30.216540+0000","last_clean":"2024-04-17T07:12:30.216540+0000","last_became_active":"2024-04-17T07:11:54.472305+0000","last_became_peered":"2024-04-17T07:11:54.472305+0000","last_unstale":"2024-04-17T07:12:30.216540+0000","last_undegraded":"2024-04-17T07:12:30.216540+0000","last_fullsized":"2024-04-17T07:12:30.216540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:21.031442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721232+0000","last_change":"2024-04-17T07:11:55.479982+0000","last_active":"2024-04-17T07:12:28.721232+0000","last_peered":"2024-04-17T07:12:28.721232+0000","last_clean":"2024-04-17T07:12:28.721232+0000","last_became_active":"2024-04-17T07:11:55.477594+0000","last_became_peered":"2024-04-17T07:11:55.477594+0000","last_unstale":"2024-04-17T07:12:28.721232+0000","last_undegraded":"2024-04-17T07:12:28.721232+0000","last_fullsized":"2024-04-17T07:12:28.721232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:23.507312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.732471+0000","last_change":"2024-04-17T07:11:55.480941+0000","last_active":"2024-04-17T07:12:27.732471+0000","last_peered":"2024-04-17T07:12:27.732471+0000","last_clean":"2024-04-17T07:12:27.732471+0000","last_became_active":"2024-04-17T07:11:55.480180+0000","last_became_peered":"2024-04-17T07:11:55.480180+0000","last_unstale":"2024-04-17T07:12:27.732471+0000","last_undegraded":"2024-04-17T07:12:27.732471+0000","last_fullsized":"2024-04-17T07:12:27.732471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:16.321806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715350+0000","last_change":"2024-04-17T07:11:55.477663+0000","last_active":"2024-04-17T07:12:28.715350+0000","last_peered":"2024-04-17T07:12:28.715350+0000","last_clean":"2024-04-17T07:12:28.715350+0000","last_became_active":"2024-04-17T07:11:55.477304+0000","last_became_peered":"2024-04-17T07:11:55.477304+0000","last_unstale":"2024-04-17T07:12:28.715350+0000","last_undegraded":"2024-04-17T07:12:28.715350+0000","last_fullsized":"2024-04-17T07:12:28.715350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:00.796633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714460+0000","last_change":"2024-04-17T07:11:55.475536+0000","last_active":"2024-04-17T07:12:28.714460+0000","last_peered":"2024-04-17T07:12:28.714460+0000","last_clean":"2024-04-17T07:12:28.714460+0000","last_became_active":"2024-04-17T07:11:55.475138+0000","last_became_peered":"2024-04-17T07:11:55.475138+0000","last_unstale":"2024-04-17T07:12:28.714460+0000","last_undegraded":"2024-04-17T07:12:28.714460+0000","last_fullsized":"2024-04-17T07:12:28.714460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:47.849076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735949+0000","last_change":"2024-04-17T07:11:55.483742+0000","last_active":"2024-04-17T07:12:28.735949+0000","last_peered":"2024-04-17T07:12:28.735949+0000","last_clean":"2024-04-17T07:12:28.735949+0000","last_became_active":"2024-04-17T07:11:55.483044+0000","last_became_peered":"2024-04-17T07:11:55.483044+0000","last_unstale":"2024-04-17T07:12:28.735949+0000","last_undegraded":"2024-04-17T07:12:28.735949+0000","last_fullsized":"2024-04-17T07:12:28.735949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:18.972137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721440+0000","last_change":"2024-04-17T07:11:55.471266+0000","last_active":"2024-04-17T07:12:28.721440+0000","last_peered":"2024-04-17T07:12:28.721440+0000","last_clean":"2024-04-17T07:12:28.721440+0000","last_became_active":"2024-04-17T07:11:55.470897+0000","last_became_peered":"2024-04-17T07:11:55.470897+0000","last_unstale":"2024-04-17T07:12:28.721440+0000","last_undegraded":"2024-04-17T07:12:28.721440+0000","last_fullsized":"2024-04-17T07:12:28.721440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:25.811064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.736847+0000","last_change":"2024-04-17T07:11:55.475860+0000","last_active":"2024-04-17T07:12:27.736847+0000","last_peered":"2024-04-17T07:12:27.736847+0000","last_clean":"2024-04-17T07:12:27.736847+0000","last_became_active":"2024-04-17T07:11:55.475522+0000","last_became_peered":"2024-04-17T07:11:55.475522+0000","last_unstale":"2024-04-17T07:12:27.736847+0000","last_undegraded":"2024-04-17T07:12:27.736847+0000","last_fullsized":"2024-04-17T07:12:27.736847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:41.548964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715736+0000","last_change":"2024-04-17T07:11:55.478815+0000","last_active":"2024-04-17T07:12:28.715736+0000","last_peered":"2024-04-17T07:12:28.715736+0000","last_clean":"2024-04-17T07:12:28.715736+0000","last_became_active":"2024-04-17T07:11:55.478347+0000","last_became_peered":"2024-04-17T07:11:55.478347+0000","last_unstale":"2024-04-17T07:12:28.715736+0000","last_undegraded":"2024-04-17T07:12:28.715736+0000","last_fullsized":"2024-04-17T07:12:28.715736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:16.465511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123444+0000","last_change":"2024-04-17T07:11:55.474208+0000","last_active":"2024-04-17T07:12:29.123444+0000","last_peered":"2024-04-17T07:12:29.123444+0000","last_clean":"2024-04-17T07:12:29.123444+0000","last_became_active":"2024-04-17T07:11:55.472042+0000","last_became_peered":"2024-04-17T07:11:55.472042+0000","last_unstale":"2024-04-17T07:12:29.123444+0000","last_undegraded":"2024-04-17T07:12:29.123444+0000","last_fullsized":"2024-04-17T07:12:29.123444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:37:24.248506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.733719+0000","last_change":"2024-04-17T07:11:55.477257+0000","last_active":"2024-04-17T07:12:28.733719+0000","last_peered":"2024-04-17T07:12:28.733719+0000","last_clean":"2024-04-17T07:12:28.733719+0000","last_became_active":"2024-04-17T07:11:55.473491+0000","last_became_peered":"2024-04-17T07:11:55.473491+0000","last_unstale":"2024-04-17T07:12:28.733719+0000","last_undegraded":"2024-04-17T07:12:28.733719+0000","last_fullsized":"2024-04-17T07:12:28.733719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:33.737485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122022+0000","last_change":"2024-04-17T07:11:55.474930+0000","last_active":"2024-04-17T07:12:29.122022+0000","last_peered":"2024-04-17T07:12:29.122022+0000","last_clean":"2024-04-17T07:12:29.122022+0000","last_became_active":"2024-04-17T07:11:55.472007+0000","last_became_peered":"2024-04-17T07:11:55.472007+0000","last_unstale":"2024-04-17T07:12:29.122022+0000","last_undegraded":"2024-04-17T07:12:29.122022+0000","last_fullsized":"2024-04-17T07:12:29.122022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:07.111267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726119+0000","last_change":"2024-04-17T07:11:55.492422+0000","last_active":"2024-04-17T07:12:28.726119+0000","last_peered":"2024-04-17T07:12:28.726119+0000","last_clean":"2024-04-17T07:12:28.726119+0000","last_became_active":"2024-04-17T07:11:55.491906+0000","last_became_peered":"2024-04-17T07:11:55.491906+0000","last_unstale":"2024-04-17T07:12:28.726119+0000","last_undegraded":"2024-04-17T07:12:28.726119+0000","last_fullsized":"2024-04-17T07:12:28.726119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:13:05.322586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720385+0000","last_change":"2024-04-17T07:11:55.479436+0000","last_active":"2024-04-17T07:12:28.720385+0000","last_peered":"2024-04-17T07:12:28.720385+0000","last_clean":"2024-04-17T07:12:28.720385+0000","last_became_active":"2024-04-17T07:11:55.478982+0000","last_became_peered":"2024-04-17T07:11:55.478982+0000","last_unstale":"2024-04-17T07:12:28.720385+0000","last_undegraded":"2024-04-17T07:12:28.720385+0000","last_fullsized":"2024-04-17T07:12:28.720385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:20.663457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-17T07:12:27.733290+0000","last_change":"2024-04-17T07:11:55.476708+0000","last_active":"2024-04-17T07:12:27.733290+0000","last_peered":"2024-04-17T07:12:27.733290+0000","last_clean":"2024-04-17T07:12:27.733290+0000","last_became_active":"2024-04-17T07:11:55.476344+0000","last_became_peered":"2024-04-17T07:11:55.476344+0000","last_unstale":"2024-04-17T07:12:27.733290+0000","last_undegraded":"2024-04-17T07:12:27.733290+0000","last_fullsized":"2024-04-17T07:12:27.733290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:58.874987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279575,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48551,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":26,"num_read_kb":61,"num_write":196,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":95150,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":172,"ondisk_log_size":172,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383702,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":477215,"data_compressed":14703,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645343,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":1130171,"data_compressed":58079,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":501379,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":497549,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":1094158,"data_compressed":56230,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":496883,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":501598,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":498339,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":507953,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":496569,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":487441,"data_compressed":17684,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1093836,"data_compressed":56230,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":13207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43642,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:12:34.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2024-04-17T07:12:34.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 19, 'reported_epoch': 91, 'state': 'activating', 'last_fresh': '2024-04-17T07:12:28.723099+0000', 'last_change': '2024-04-17T07:12:28.714326+0000', 'last_active': '2024-04-17T07:12:27.714584+0000', 'last_peered': '2024-04-17T07:12:00.800296+0000', 'last_clean': '2024-04-17T07:12:00.800296+0000', 'last_became_active': '2024-04-17T07:11:59.512404+0000', 'last_became_peered': '2024-04-17T07:11:59.512404+0000', 'last_unstale': '2024-04-17T07:12:28.723099+0000', 'last_undegraded': '2024-04-17T07:12:28.723099+0000', 'last_fullsized': '2024-04-17T07:12:28.723099+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 88, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T07:11:58.472667+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T07:11:58.472667+0000', 'last_clean_scrub_stamp': '2024-04-17T07:11:58.472667+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6, 8, 3], 'acting': [4, 6, 8, 3], 'avail_no_missing': ['4(0)', '3(3)', '6(1)', '8(2)', '9(2)'], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-17T07:12:34.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-04-17T07:12:34.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "84'2", 'reported_seq': 2, 'reported_epoch': 90, 'state': 'peering', 'last_fresh': '2024-04-17T07:11:55.467756+0000', 'last_change': '2024-04-17T07:12:28.153200+0000', 'last_active': '2024-04-17T07:12:28.153200+0000', 'last_peered': '2024-04-17T07:11:55.467756+0000', 'last_clean': '2024-04-17T07:11:55.467756+0000', 'last_became_active': '2024-04-17T07:11:54.469320+0000', 'last_became_peered': '2024-04-17T07:11:54.469320+0000', 'last_unstale': '2024-04-17T07:11:55.467756+0000', 'last_undegraded': '2024-04-17T07:11:55.467756+0000', 'last_fullsized': '2024-04-17T07:11:55.467756+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 83, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T07:11:53.415144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T07:11:53.415144+0000', 'last_clean_scrub_stamp': '2024-04-17T07:11:53.415144+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 3, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 1, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 1, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 2, 7], 'acting': [11, 2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2024-04-17T07:12:34.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-04-17T07:12:34.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "89'10", 'reported_seq': 2, 'reported_epoch': 90, 'state': 'peering', 'last_fresh': '2024-04-17T07:12:00.800050+0000', 'last_change': '2024-04-17T07:12:28.153142+0000', 'last_active': '2024-04-17T07:12:28.153142+0000', 'last_peered': '2024-04-17T07:12:00.800050+0000', 'last_clean': '2024-04-17T07:12:00.800050+0000', 'last_became_active': '2024-04-17T07:11:54.471537+0000', 'last_became_peered': '2024-04-17T07:11:54.471537+0000', 'last_unstale': '2024-04-17T07:12:00.800050+0000', 'last_undegraded': '2024-04-17T07:12:00.800050+0000', 'last_fullsized': '2024-04-17T07:12:00.800050+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 83, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-17T07:11:53.415144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-17T07:11:53.415144+0000', 'last_clean_scrub_stamp': '2024-04-17T07:11:53.415144+0000', 'objects_scrubbed': 0, 'log_size': 10, 'log_dups_size': 0, 'ondisk_log_size': 10, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 5, 'num_object_clones': 0, 'num_object_copies': 15, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 5, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 5, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 0, 2], 'acting': [11, 0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2024-04-17T07:12:34.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-17T07:12:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.071+0000 7f37f0e00640 1 -- 172.21.15.86:0/4256405335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec1039b0 msgr2=0x7f37ec10a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.071+0000 7f37f0e00640 1 --2- 172.21.15.86:0/4256405335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec1039b0 0x7f37ec10a240 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009ca0 tx=0x7f37e402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.071+0000 7f37f0e00640 1 -- 172.21.15.86:0/4256405335 shutdown_connections 2024-04-17T07:12:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.071+0000 7f37f0e00640 1 --2- 172.21.15.86:0/4256405335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec1039b0 0x7f37ec10a240 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.071+0000 7f37f0e00640 1 --2- 172.21.15.86:0/4256405335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37ec102ff0 0x7f37ec103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.071+0000 7f37f0e00640 1 --2- 172.21.15.86:0/4256405335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37ec101e00 0x7f37ec102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.071+0000 7f37f0e00640 1 -- 172.21.15.86:0/4256405335 >> 172.21.15.86:0/4256405335 conn(0x7f37ec0fd370 msgr2=0x7f37ec0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.071+0000 7f37f0e00640 1 -- 172.21.15.86:0/4256405335 shutdown_connections 2024-04-17T07:12:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.071+0000 7f37f0e00640 1 -- 172.21.15.86:0/4256405335 wait complete. 2024-04-17T07:12:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.072+0000 7f37f0e00640 1 Processor -- start 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.072+0000 7f37f0e00640 1 -- start start 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.072+0000 7f37f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec101e00 0x7f37ec075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.072+0000 7f37f0e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37ec102ff0 0x7f37ec074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.072+0000 7f37f0e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37ec1039b0 0x7f37ec074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.072+0000 7f37f0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec10d930 con 0x7f37ec101e00 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.072+0000 7f37f0e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec10d7b0 con 0x7f37ec102ff0 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.072+0000 7f37f0e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec10dab0 con 0x7f37ec1039b0 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.072+0000 7f37eb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec101e00 0x7f37ec075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37eb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec101e00 0x7f37ec075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54224/0 (socket says 172.21.15.86:54224) 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37eb400640 1 -- 172.21.15.86:0/2649296459 learned_addr learned my addr 172.21.15.86:0/2649296459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37eaa00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37ec102ff0 0x7f37ec074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37eb400640 1 -- 172.21.15.86:0/2649296459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37ec1039b0 msgr2=0x7f37ec074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37ebe00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37ec1039b0 0x7f37ec074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37eb400640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37ec1039b0 0x7f37ec074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37eb400640 1 -- 172.21.15.86:0/2649296459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37ec102ff0 msgr2=0x7f37ec074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37eb400640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37ec102ff0 0x7f37ec074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37eb400640 1 -- 172.21.15.86:0/2649296459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec075120 con 0x7f37ec101e00 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37ebe00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37ec1039b0 0x7f37ec074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37eaa00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37ec102ff0 0x7f37ec074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.073+0000 7f37eb400640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec101e00 0x7f37ec075d10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00d770 tx=0x7f37dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.074+0000 7f37e3e00640 1 -- 172.21.15.86:0/2649296459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc017070 con 0x7f37ec101e00 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.074+0000 7f37e3e00640 1 -- 172.21.15.86:0/2649296459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37dc009c30 con 0x7f37ec101e00 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.074+0000 7f37f0e00640 1 -- 172.21.15.86:0/2649296459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec1a20d0 con 0x7f37ec101e00 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.074+0000 7f37e3e00640 1 -- 172.21.15.86:0/2649296459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc003d00 con 0x7f37ec101e00 2024-04-17T07:12:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.074+0000 7f37f0e00640 1 -- 172.21.15.86:0/2649296459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec1a2670 con 0x7f37ec101e00 2024-04-17T07:12:35.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.075+0000 7f37f0e00640 1 -- 172.21.15.86:0/2649296459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec109120 con 0x7f37ec101e00 2024-04-17T07:12:35.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.077+0000 7f37e3e00640 1 -- 172.21.15.86:0/2649296459 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f37dc003ea0 con 0x7f37ec101e00 2024-04-17T07:12:35.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.078+0000 7f37e3e00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37d00a3cc0 0x7f37d00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:35.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.078+0000 7f37e3e00640 1 -- 172.21.15.86:0/2649296459 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f37dc0d06b0 con 0x7f37ec101e00 2024-04-17T07:12:35.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.078+0000 7f37eaa00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37d00a3cc0 0x7f37d00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:35.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.079+0000 7f37eaa00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37d00a3cc0 0x7f37d00a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f37ec075a80 tx=0x7f37d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:35.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.083+0000 7f37e3e00640 1 -- 172.21.15.86:0/2649296459 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f37dc09cb70 con 0x7f37ec101e00 2024-04-17T07:12:35.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.365+0000 7f37f0e00640 1 -- 172.21.15.86:0/2649296459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f37ec02d880 con 0x7f37ec101e00 2024-04-17T07:12:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.366+0000 7f37e3e00640 1 -- 172.21.15.86:0/2649296459 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2648 (secure 0 0 0) 0x7f37dc009de0 con 0x7f37ec101e00 2024-04-17T07:12:35.366 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:35.366 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":528,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":12,"num_up_osds":12,"osd_up_since":1713337847,"num_in_osds":12,"osd_in_since":1713337821,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":101,"data_bytes":639321,"bytes_used":374247424,"bytes_avail":1151566180352,"bytes_total":1151940427776,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":77,"recovering_keys_per_sec":0,"num_objects_recovered":9,"num_bytes_recovered":622,"num_keys_recovered":0,"read_bytes_sec":9212,"write_bytes_sec":11898,"read_op_per_sec":4,"write_op_per_sec":21},"fsmap":{"epoch":27,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14688},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14700},{"filesystem_id":1,"rank":2,"name":"l","status":"up:standby-replay","gid":24412},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24469},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24487},{"filesystem_id":1,"rank":4,"name":"i","status":"up:active","gid":24505},{"filesystem_id":1,"rank":3,"name":"k","status":"up:active","gid":34400},{"filesystem_id":1,"rank":2,"name":"e","status":"up:active","gid":34430},{"filesystem_id":1,"rank":1,"name":"h","status":"up:active","gid":34460},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":34490}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.86:8443/"}},"servicemap":{"epoch":25,"modified":"2024-04-17T07:12:29.056841+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.368+0000 7f37f0e00640 1 -- 172.21.15.86:0/2649296459 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37d00a3cc0 msgr2=0x7f37d00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.368+0000 7f37f0e00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37d00a3cc0 0x7f37d00a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f37ec075a80 tx=0x7f37d8005f90 comp rx=0 tx=0).stop 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.368+0000 7f37f0e00640 1 -- 172.21.15.86:0/2649296459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec101e00 msgr2=0x7f37ec075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.368+0000 7f37f0e00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec101e00 0x7f37ec075d10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00d770 tx=0x7f37dc00dc30 comp rx=0 tx=0).stop 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.369+0000 7f37f0e00640 1 -- 172.21.15.86:0/2649296459 shutdown_connections 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.369+0000 7f37f0e00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37d00a3cc0 0x7f37d00a6180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.369+0000 7f37f0e00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37ec1039b0 0x7f37ec074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.369+0000 7f37f0e00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37ec102ff0 0x7f37ec074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.369+0000 7f37f0e00640 1 --2- 172.21.15.86:0/2649296459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec101e00 0x7f37ec075d10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.369+0000 7f37f0e00640 1 -- 172.21.15.86:0/2649296459 >> 172.21.15.86:0/2649296459 conn(0x7f37ec0fd370 msgr2=0x7f37ec1062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.369+0000 7f37f0e00640 1 -- 172.21.15.86:0/2649296459 shutdown_connections 2024-04-17T07:12:35.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.369+0000 7f37f0e00640 1 -- 172.21.15.86:0/2649296459 wait complete. 2024-04-17T07:12:35.388 INFO:tasks.cephfs.filesystem.ceph_manager:making progress, resetting timeout 2024-04-17T07:12:35.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:12:35.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:35 smithi086 ceph-mon[31733]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:35.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:35 smithi086 ceph-mon[31733]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:35.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:35 smithi086 ceph-mon[31733]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:35.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:35 smithi086 ceph-mon[31733]: pgmap v385: 129 pgs: 129 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 KiB/s wr, 26 op/s; 77 B/s, 1 objects/s recovering 2024-04-17T07:12:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:35 smithi098 ceph-mon[35418]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:35 smithi098 ceph-mon[35418]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:35 smithi098 ceph-mon[35418]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:35 smithi098 ceph-mon[35418]: pgmap v385: 129 pgs: 129 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 KiB/s wr, 26 op/s; 77 B/s, 1 objects/s recovering 2024-04-17T07:12:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:35 smithi103 ceph-mon[35495]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:35 smithi103 ceph-mon[35495]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:35 smithi103 ceph-mon[35495]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:35 smithi103 ceph-mon[35495]: pgmap v385: 129 pgs: 129 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 KiB/s wr, 26 op/s; 77 B/s, 1 objects/s recovering 2024-04-17T07:12:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.603+0000 7fd41fe00640 1 -- 172.21.15.86:0/2912978310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd420109f90 msgr2=0x7fd420110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.603+0000 7fd41fe00640 1 --2- 172.21.15.86:0/2912978310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd420109f90 0x7fd420110820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd418009ca0 tx=0x7fd41802c100 comp rx=0 tx=0).stop 2024-04-17T07:12:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.603+0000 7fd41fe00640 1 -- 172.21.15.86:0/2912978310 shutdown_connections 2024-04-17T07:12:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.603+0000 7fd41fe00640 1 --2- 172.21.15.86:0/2912978310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd420109f90 0x7fd420110820 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.603+0000 7fd41fe00640 1 --2- 172.21.15.86:0/2912978310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4201095d0 0x7fd420109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.603+0000 7fd41fe00640 1 --2- 172.21.15.86:0/2912978310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4201083e0 0x7fd420108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.603+0000 7fd41fe00640 1 -- 172.21.15.86:0/2912978310 >> 172.21.15.86:0/2912978310 conn(0x7fd4200773a0 msgr2=0x7fd4200797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.604+0000 7fd41fe00640 1 -- 172.21.15.86:0/2912978310 shutdown_connections 2024-04-17T07:12:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.604+0000 7fd41fe00640 1 -- 172.21.15.86:0/2912978310 wait complete. 2024-04-17T07:12:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41fe00640 1 Processor -- start 2024-04-17T07:12:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41fe00640 1 -- start start 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4201083e0 0x7fd42019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4201095d0 0x7fd42019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd420109f90 0x7fd4201a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd420113f20 con 0x7fd4201083e0 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41fe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd420113da0 con 0x7fd4201095d0 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201140a0 con 0x7fd420109f90 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4201083e0 0x7fd42019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4201083e0 0x7fd42019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54236/0 (socket says 172.21.15.86:54236) 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.605+0000 7fd41ea00640 1 -- 172.21.15.86:0/2710055136 learned_addr learned my addr 172.21.15.86:0/2710055136 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd416000640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4201095d0 0x7fd42019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd41f400640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd420109f90 0x7fd4201a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd41ea00640 1 -- 172.21.15.86:0/2710055136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd420109f90 msgr2=0x7fd4201a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd41ea00640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd420109f90 0x7fd4201a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd41ea00640 1 -- 172.21.15.86:0/2710055136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4201095d0 msgr2=0x7fd42019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd41ea00640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4201095d0 0x7fd42019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd41ea00640 1 -- 172.21.15.86:0/2710055136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4201a61e0 con 0x7fd4201083e0 2024-04-17T07:12:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd41ea00640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4201083e0 0x7fd42019e5c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd41000d770 tx=0x7fd41000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd417e00640 1 -- 172.21.15.86:0/2710055136 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410017070 con 0x7fd4201083e0 2024-04-17T07:12:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd417e00640 1 -- 172.21.15.86:0/2710055136 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd410009c30 con 0x7fd4201083e0 2024-04-17T07:12:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd417e00640 1 -- 172.21.15.86:0/2710055136 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410003d00 con 0x7fd4201083e0 2024-04-17T07:12:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.606+0000 7fd41fe00640 1 -- 172.21.15.86:0/2710055136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4201a64d0 con 0x7fd4201083e0 2024-04-17T07:12:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.607+0000 7fd41fe00640 1 -- 172.21.15.86:0/2710055136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4201a69d0 con 0x7fd4201083e0 2024-04-17T07:12:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.608+0000 7fd41fe00640 1 -- 172.21.15.86:0/2710055136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd42010f7d0 con 0x7fd4201083e0 2024-04-17T07:12:35.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.610+0000 7fd417e00640 1 -- 172.21.15.86:0/2710055136 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd410003ea0 con 0x7fd4201083e0 2024-04-17T07:12:35.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.611+0000 7fd417e00640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4040a3cc0 0x7fd4040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:35.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.611+0000 7fd417e00640 1 -- 172.21.15.86:0/2710055136 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd4100d06b0 con 0x7fd4201083e0 2024-04-17T07:12:35.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.612+0000 7fd416000640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4040a3cc0 0x7fd4040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:35.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.612+0000 7fd416000640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4040a3cc0 0x7fd4040a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd408005fd0 tx=0x7fd408005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.616+0000 7fd417e00640 1 -- 172.21.15.86:0/2710055136 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd41009cb70 con 0x7fd4201083e0 2024-04-17T07:12:35.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.815+0000 7fd41fe00640 1 -- 172.21.15.86:0/2710055136 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4200623a0 con 0x7fd4040a3cc0 2024-04-17T07:12:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.828+0000 7fd417e00640 1 -- 172.21.15.86:0/2710055136 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340841 (secure 0 0 0) 0x7fd4200623a0 con 0x7fd4040a3cc0 2024-04-17T07:12:35.833 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-17T07:12:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.835+0000 7fd41fe00640 1 -- 172.21.15.86:0/2710055136 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4040a3cc0 msgr2=0x7fd4040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.835+0000 7fd41fe00640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4040a3cc0 0x7fd4040a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd408005fd0 tx=0x7fd408005850 comp rx=0 tx=0).stop 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.835+0000 7fd41fe00640 1 -- 172.21.15.86:0/2710055136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4201083e0 msgr2=0x7fd42019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.835+0000 7fd41fe00640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4201083e0 0x7fd42019e5c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd41000d770 tx=0x7fd41000dc30 comp rx=0 tx=0).stop 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.836+0000 7fd41fe00640 1 -- 172.21.15.86:0/2710055136 shutdown_connections 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.836+0000 7fd41fe00640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4040a3cc0 0x7fd4040a6180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.836+0000 7fd41fe00640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd420109f90 0x7fd4201a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.836+0000 7fd41fe00640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4201095d0 0x7fd42019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.836+0000 7fd41fe00640 1 --2- 172.21.15.86:0/2710055136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4201083e0 0x7fd42019e5c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.836+0000 7fd41fe00640 1 -- 172.21.15.86:0/2710055136 >> 172.21.15.86:0/2710055136 conn(0x7fd4200773a0 msgr2=0x7fd42010c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.836+0000 7fd41fe00640 1 -- 172.21.15.86:0/2710055136 shutdown_connections 2024-04-17T07:12:35.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:35.836+0000 7fd41fe00640 1 -- 172.21.15.86:0/2710055136 wait complete. 2024-04-17T07:12:35.854 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-04-17T07:12:35.059346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639321,"num_objects":101,"num_object_clones":0,"num_object_copies":305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":166,"num_read_kb":181,"num_write":408,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181358,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":336,"ondisk_log_size":336,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":422,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365476,"kb_used_data":12148,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576348,"statfs":{"total":1151940427776,"available":1151566180352,"internally_reserved":0,"allocated":12439552,"data_stored":7820953,"data_compressed":356542,"data_compressed_allocated":483328,"data_compressed_original":2465792,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43109,"num_objects":72,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":40,"num_read_kb":72,"num_write":169,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003382"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730198+0000","last_change":"2024-04-17T07:11:55.486746+0000","last_active":"2024-04-17T07:12:28.730198+0000","last_peered":"2024-04-17T07:12:28.730198+0000","last_clean":"2024-04-17T07:12:28.730198+0000","last_became_active":"2024-04-17T07:11:55.486335+0000","last_became_peered":"2024-04-17T07:11:55.486335+0000","last_unstale":"2024-04-17T07:12:28.730198+0000","last_undegraded":"2024-04-17T07:12:28.730198+0000","last_fullsized":"2024-04-17T07:12:28.730198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:48.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735866+0000","last_change":"2024-04-17T07:11:55.481832+0000","last_active":"2024-04-17T07:12:28.735866+0000","last_peered":"2024-04-17T07:12:28.735866+0000","last_clean":"2024-04-17T07:12:28.735866+0000","last_became_active":"2024-04-17T07:11:55.476428+0000","last_became_peered":"2024-04-17T07:11:55.476428+0000","last_unstale":"2024-04-17T07:12:28.735866+0000","last_undegraded":"2024-04-17T07:12:28.735866+0000","last_fullsized":"2024-04-17T07:12:28.735866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:04.452542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736227+0000","last_change":"2024-04-17T07:11:55.483721+0000","last_active":"2024-04-17T07:12:28.736227+0000","last_peered":"2024-04-17T07:12:28.736227+0000","last_clean":"2024-04-17T07:12:28.736227+0000","last_became_active":"2024-04-17T07:11:55.483268+0000","last_became_peered":"2024-04-17T07:11:55.483268+0000","last_unstale":"2024-04-17T07:12:28.736227+0000","last_undegraded":"2024-04-17T07:12:28.736227+0000","last_fullsized":"2024-04-17T07:12:28.736227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:41.584783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714249+0000","last_change":"2024-04-17T07:11:55.485114+0000","last_active":"2024-04-17T07:12:28.714249+0000","last_peered":"2024-04-17T07:12:28.714249+0000","last_clean":"2024-04-17T07:12:28.714249+0000","last_became_active":"2024-04-17T07:11:55.483913+0000","last_became_peered":"2024-04-17T07:11:55.483913+0000","last_unstale":"2024-04-17T07:12:28.714249+0000","last_undegraded":"2024-04-17T07:12:28.714249+0000","last_fullsized":"2024-04-17T07:12:28.714249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:46.211121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726512+0000","last_change":"2024-04-17T07:11:55.491952+0000","last_active":"2024-04-17T07:12:28.726512+0000","last_peered":"2024-04-17T07:12:28.726512+0000","last_clean":"2024-04-17T07:12:28.726512+0000","last_became_active":"2024-04-17T07:11:55.491238+0000","last_became_peered":"2024-04-17T07:11:55.491238+0000","last_unstale":"2024-04-17T07:12:28.726512+0000","last_undegraded":"2024-04-17T07:12:28.726512+0000","last_fullsized":"2024-04-17T07:12:28.726512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:06.759797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725020+0000","last_change":"2024-04-17T07:11:55.471124+0000","last_active":"2024-04-17T07:12:28.725020+0000","last_peered":"2024-04-17T07:12:28.725020+0000","last_clean":"2024-04-17T07:12:28.725020+0000","last_became_active":"2024-04-17T07:11:55.470781+0000","last_became_peered":"2024-04-17T07:11:55.470781+0000","last_unstale":"2024-04-17T07:12:28.725020+0000","last_undegraded":"2024-04-17T07:12:28.725020+0000","last_fullsized":"2024-04-17T07:12:28.725020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:45.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720008+0000","last_change":"2024-04-17T07:11:55.469099+0000","last_active":"2024-04-17T07:12:28.720008+0000","last_peered":"2024-04-17T07:12:28.720008+0000","last_clean":"2024-04-17T07:12:28.720008+0000","last_became_active":"2024-04-17T07:11:55.468584+0000","last_became_peered":"2024-04-17T07:11:55.468584+0000","last_unstale":"2024-04-17T07:12:28.720008+0000","last_undegraded":"2024-04-17T07:12:28.720008+0000","last_fullsized":"2024-04-17T07:12:28.720008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:52.834846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122568+0000","last_change":"2024-04-17T07:11:55.473794+0000","last_active":"2024-04-17T07:12:29.122568+0000","last_peered":"2024-04-17T07:12:29.122568+0000","last_clean":"2024-04-17T07:12:29.122568+0000","last_became_active":"2024-04-17T07:11:55.471919+0000","last_became_peered":"2024-04-17T07:11:55.471919+0000","last_unstale":"2024-04-17T07:12:29.122568+0000","last_undegraded":"2024-04-17T07:12:29.122568+0000","last_fullsized":"2024-04-17T07:12:29.122568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:16.085117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726090+0000","last_change":"2024-04-17T07:11:55.482445+0000","last_active":"2024-04-17T07:12:28.726090+0000","last_peered":"2024-04-17T07:12:28.726090+0000","last_clean":"2024-04-17T07:12:28.726090+0000","last_became_active":"2024-04-17T07:11:55.481552+0000","last_became_peered":"2024-04-17T07:11:55.481552+0000","last_unstale":"2024-04-17T07:12:28.726090+0000","last_undegraded":"2024-04-17T07:12:28.726090+0000","last_fullsized":"2024-04-17T07:12:28.726090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:48.733585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725405+0000","last_change":"2024-04-17T07:11:55.476726+0000","last_active":"2024-04-17T07:12:28.725405+0000","last_peered":"2024-04-17T07:12:28.725405+0000","last_clean":"2024-04-17T07:12:28.725405+0000","last_became_active":"2024-04-17T07:11:55.476320+0000","last_became_peered":"2024-04-17T07:11:55.476320+0000","last_unstale":"2024-04-17T07:12:28.725405+0000","last_undegraded":"2024-04-17T07:12:28.725405+0000","last_fullsized":"2024-04-17T07:12:28.725405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.230181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721027+0000","last_change":"2024-04-17T07:11:55.485798+0000","last_active":"2024-04-17T07:12:28.721027+0000","last_peered":"2024-04-17T07:12:28.721027+0000","last_clean":"2024-04-17T07:12:28.721027+0000","last_became_active":"2024-04-17T07:11:55.484015+0000","last_became_peered":"2024-04-17T07:11:55.484015+0000","last_unstale":"2024-04-17T07:12:28.721027+0000","last_undegraded":"2024-04-17T07:12:28.721027+0000","last_fullsized":"2024-04-17T07:12:28.721027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:37.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.731078+0000","last_change":"2024-04-17T07:12:28.729834+0000","last_active":"2024-04-17T07:12:28.731078+0000","last_peered":"2024-04-17T07:12:28.731078+0000","last_clean":"2024-04-17T07:12:28.731078+0000","last_became_active":"2024-04-17T07:12:28.728694+0000","last_became_peered":"2024-04-17T07:12:28.728694+0000","last_unstale":"2024-04-17T07:12:28.731078+0000","last_undegraded":"2024-04-17T07:12:28.731078+0000","last_fullsized":"2024-04-17T07:12:28.731078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:46.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720070+0000","last_change":"2024-04-17T07:11:55.450379+0000","last_active":"2024-04-17T07:12:28.720070+0000","last_peered":"2024-04-17T07:12:28.720070+0000","last_clean":"2024-04-17T07:12:28.720070+0000","last_became_active":"2024-04-17T07:11:55.449118+0000","last_became_peered":"2024-04-17T07:11:55.449118+0000","last_unstale":"2024-04-17T07:12:28.720070+0000","last_undegraded":"2024-04-17T07:12:28.720070+0000","last_fullsized":"2024-04-17T07:12:28.720070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:07:34.248044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735022+0000","last_change":"2024-04-17T07:11:55.473839+0000","last_active":"2024-04-17T07:12:28.735022+0000","last_peered":"2024-04-17T07:12:28.735022+0000","last_clean":"2024-04-17T07:12:28.735022+0000","last_became_active":"2024-04-17T07:11:55.473472+0000","last_became_peered":"2024-04-17T07:11:55.473472+0000","last_unstale":"2024-04-17T07:12:28.735022+0000","last_undegraded":"2024-04-17T07:12:28.735022+0000","last_fullsized":"2024-04-17T07:12:28.735022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:02.263272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736022+0000","last_change":"2024-04-17T07:11:55.484820+0000","last_active":"2024-04-17T07:12:28.736022+0000","last_peered":"2024-04-17T07:12:28.736022+0000","last_clean":"2024-04-17T07:12:28.736022+0000","last_became_active":"2024-04-17T07:11:55.484314+0000","last_became_peered":"2024-04-17T07:11:55.484314+0000","last_unstale":"2024-04-17T07:12:28.736022+0000","last_undegraded":"2024-04-17T07:12:28.736022+0000","last_fullsized":"2024-04-17T07:12:28.736022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.186912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725577+0000","last_change":"2024-04-17T07:11:55.492018+0000","last_active":"2024-04-17T07:12:28.725577+0000","last_peered":"2024-04-17T07:12:28.725577+0000","last_clean":"2024-04-17T07:12:28.725577+0000","last_became_active":"2024-04-17T07:11:55.490512+0000","last_became_peered":"2024-04-17T07:11:55.490512+0000","last_unstale":"2024-04-17T07:12:28.725577+0000","last_undegraded":"2024-04-17T07:12:28.725577+0000","last_fullsized":"2024-04-17T07:12:28.725577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:45.436270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.729938+0000","last_change":"2024-04-17T07:11:55.487210+0000","last_active":"2024-04-17T07:12:28.729938+0000","last_peered":"2024-04-17T07:12:28.729938+0000","last_clean":"2024-04-17T07:12:28.729938+0000","last_became_active":"2024-04-17T07:11:55.485713+0000","last_became_peered":"2024-04-17T07:11:55.485713+0000","last_unstale":"2024-04-17T07:12:28.729938+0000","last_undegraded":"2024-04-17T07:12:28.729938+0000","last_fullsized":"2024-04-17T07:12:28.729938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:54.495950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716204+0000","last_change":"2024-04-17T07:11:55.482705+0000","last_active":"2024-04-17T07:12:28.716204+0000","last_peered":"2024-04-17T07:12:28.716204+0000","last_clean":"2024-04-17T07:12:28.716204+0000","last_became_active":"2024-04-17T07:11:55.482349+0000","last_became_peered":"2024-04-17T07:11:55.482349+0000","last_unstale":"2024-04-17T07:12:28.716204+0000","last_undegraded":"2024-04-17T07:12:28.716204+0000","last_fullsized":"2024-04-17T07:12:28.716204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:57.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716213+0000","last_change":"2024-04-17T07:11:55.467065+0000","last_active":"2024-04-17T07:12:28.716213+0000","last_peered":"2024-04-17T07:12:28.716213+0000","last_clean":"2024-04-17T07:12:28.716213+0000","last_became_active":"2024-04-17T07:11:55.466535+0000","last_became_peered":"2024-04-17T07:11:55.466535+0000","last_unstale":"2024-04-17T07:12:28.716213+0000","last_undegraded":"2024-04-17T07:12:28.716213+0000","last_fullsized":"2024-04-17T07:12:28.716213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:49.955453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715012+0000","last_change":"2024-04-17T07:11:59.515778+0000","last_active":"2024-04-17T07:12:28.715012+0000","last_peered":"2024-04-17T07:12:28.715012+0000","last_clean":"2024-04-17T07:12:28.715012+0000","last_became_active":"2024-04-17T07:11:59.515266+0000","last_became_peered":"2024-04-17T07:11:59.515266+0000","last_unstale":"2024-04-17T07:12:28.715012+0000","last_undegraded":"2024-04-17T07:12:28.715012+0000","last_fullsized":"2024-04-17T07:12:28.715012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:43.084724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.219364+0000","last_change":"2024-04-17T07:11:54.467060+0000","last_active":"2024-04-17T07:12:31.219364+0000","last_peered":"2024-04-17T07:12:31.219364+0000","last_clean":"2024-04-17T07:12:31.219364+0000","last_became_active":"2024-04-17T07:11:54.466687+0000","last_became_peered":"2024-04-17T07:11:54.466687+0000","last_unstale":"2024-04-17T07:12:31.219364+0000","last_undegraded":"2024-04-17T07:12:31.219364+0000","last_fullsized":"2024-04-17T07:12:31.219364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:57.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123283+0000","last_change":"2024-04-17T07:11:55.469857+0000","last_active":"2024-04-17T07:12:29.123283+0000","last_peered":"2024-04-17T07:12:29.123283+0000","last_clean":"2024-04-17T07:12:29.123283+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:12:29.123283+0000","last_undegraded":"2024-04-17T07:12:29.123283+0000","last_fullsized":"2024-04-17T07:12:29.123283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:43.565627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.125397+0000","last_change":"2024-04-17T07:12:29.125397+0000","last_active":"2024-04-17T07:12:29.125397+0000","last_peered":"2024-04-17T07:12:29.125397+0000","last_clean":"2024-04-17T07:12:29.125397+0000","last_became_active":"2024-04-17T07:12:29.124158+0000","last_became_peered":"2024-04-17T07:12:29.124158+0000","last_unstale":"2024-04-17T07:12:29.125397+0000","last_undegraded":"2024-04-17T07:12:29.125397+0000","last_fullsized":"2024-04-17T07:12:29.125397+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:24.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123277+0000","last_change":"2024-04-17T07:11:54.464715+0000","last_active":"2024-04-17T07:12:29.123277+0000","last_peered":"2024-04-17T07:12:29.123277+0000","last_clean":"2024-04-17T07:12:29.123277+0000","last_became_active":"2024-04-17T07:11:54.463917+0000","last_became_peered":"2024-04-17T07:11:54.463917+0000","last_unstale":"2024-04-17T07:12:29.123277+0000","last_undegraded":"2024-04-17T07:12:29.123277+0000","last_fullsized":"2024-04-17T07:12:29.123277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:43.738471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736571+0000","last_change":"2024-04-17T07:11:55.482876+0000","last_active":"2024-04-17T07:12:28.736571+0000","last_peered":"2024-04-17T07:12:28.736571+0000","last_clean":"2024-04-17T07:12:28.736571+0000","last_became_active":"2024-04-17T07:11:55.482451+0000","last_became_peered":"2024-04-17T07:11:55.482451+0000","last_unstale":"2024-04-17T07:12:28.736571+0000","last_undegraded":"2024-04-17T07:12:28.736571+0000","last_fullsized":"2024-04-17T07:12:28.736571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:39.394090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713748+0000","last_change":"2024-04-17T07:11:59.522634+0000","last_active":"2024-04-17T07:12:28.713748+0000","last_peered":"2024-04-17T07:12:28.713748+0000","last_clean":"2024-04-17T07:12:28.713748+0000","last_became_active":"2024-04-17T07:11:59.520797+0000","last_became_peered":"2024-04-17T07:11:59.520797+0000","last_unstale":"2024-04-17T07:12:28.713748+0000","last_undegraded":"2024-04-17T07:12:28.713748+0000","last_fullsized":"2024-04-17T07:12:28.713748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:02.316515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.218005+0000","last_change":"2024-04-17T07:12:28.756528+0000","last_active":"2024-04-17T07:12:31.218005+0000","last_peered":"2024-04-17T07:12:31.218005+0000","last_clean":"2024-04-17T07:12:31.218005+0000","last_became_active":"2024-04-17T07:12:28.737008+0000","last_became_peered":"2024-04-17T07:12:28.737008+0000","last_unstale":"2024-04-17T07:12:31.218005+0000","last_undegraded":"2024-04-17T07:12:31.218005+0000","last_fullsized":"2024-04-17T07:12:31.218005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:56:26.719377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726296+0000","last_change":"2024-04-17T07:11:55.485999+0000","last_active":"2024-04-17T07:12:28.726296+0000","last_peered":"2024-04-17T07:12:28.726296+0000","last_clean":"2024-04-17T07:12:28.726296+0000","last_became_active":"2024-04-17T07:11:55.484048+0000","last_became_peered":"2024-04-17T07:11:55.484048+0000","last_unstale":"2024-04-17T07:12:28.726296+0000","last_undegraded":"2024-04-17T07:12:28.726296+0000","last_fullsized":"2024-04-17T07:12:28.726296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:38.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713330+0000","last_change":"2024-04-17T07:11:59.529929+0000","last_active":"2024-04-17T07:12:28.713330+0000","last_peered":"2024-04-17T07:12:28.713330+0000","last_clean":"2024-04-17T07:12:28.713330+0000","last_became_active":"2024-04-17T07:11:59.529400+0000","last_became_peered":"2024-04-17T07:11:59.529400+0000","last_unstale":"2024-04-17T07:12:28.713330+0000","last_undegraded":"2024-04-17T07:12:28.713330+0000","last_fullsized":"2024-04-17T07:12:28.713330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:54.845529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.209278+0000","last_change":"2024-04-17T07:11:54.462852+0000","last_active":"2024-04-17T07:12:30.209278+0000","last_peered":"2024-04-17T07:12:30.209278+0000","last_clean":"2024-04-17T07:12:30.209278+0000","last_became_active":"2024-04-17T07:11:54.459178+0000","last_became_peered":"2024-04-17T07:11:54.459178+0000","last_unstale":"2024-04-17T07:12:30.209278+0000","last_undegraded":"2024-04-17T07:12:30.209278+0000","last_fullsized":"2024-04-17T07:12:30.209278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:07.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.737683+0000","last_change":"2024-04-17T07:11:55.480951+0000","last_active":"2024-04-17T07:12:28.737683+0000","last_peered":"2024-04-17T07:12:28.737683+0000","last_clean":"2024-04-17T07:12:28.737683+0000","last_became_active":"2024-04-17T07:11:55.480507+0000","last_became_peered":"2024-04-17T07:11:55.480507+0000","last_unstale":"2024-04-17T07:12:28.737683+0000","last_undegraded":"2024-04-17T07:12:28.737683+0000","last_fullsized":"2024-04-17T07:12:28.737683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:42.584180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715898+0000","last_change":"2024-04-17T07:11:59.520897+0000","last_active":"2024-04-17T07:12:28.715898+0000","last_peered":"2024-04-17T07:12:28.715898+0000","last_clean":"2024-04-17T07:12:28.715898+0000","last_became_active":"2024-04-17T07:11:59.520350+0000","last_became_peered":"2024-04-17T07:11:59.520350+0000","last_unstale":"2024-04-17T07:12:28.715898+0000","last_undegraded":"2024-04-17T07:12:28.715898+0000","last_fullsized":"2024-04-17T07:12:28.715898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:39.813097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.222928+0000","last_change":"2024-04-17T07:12:28.753717+0000","last_active":"2024-04-17T07:12:31.222928+0000","last_peered":"2024-04-17T07:12:31.222928+0000","last_clean":"2024-04-17T07:12:31.222928+0000","last_became_active":"2024-04-17T07:12:28.739556+0000","last_became_peered":"2024-04-17T07:12:28.739556+0000","last_unstale":"2024-04-17T07:12:31.222928+0000","last_undegraded":"2024-04-17T07:12:31.222928+0000","last_fullsized":"2024-04-17T07:12:31.222928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:11.967731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.732946+0000","last_change":"2024-04-17T07:11:55.476675+0000","last_active":"2024-04-17T07:12:28.732946+0000","last_peered":"2024-04-17T07:12:28.732946+0000","last_clean":"2024-04-17T07:12:28.732946+0000","last_became_active":"2024-04-17T07:11:55.476273+0000","last_became_peered":"2024-04-17T07:11:55.476273+0000","last_unstale":"2024-04-17T07:12:28.732946+0000","last_undegraded":"2024-04-17T07:12:28.732946+0000","last_fullsized":"2024-04-17T07:12:28.732946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:09.612138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.741537+0000","last_change":"2024-04-17T07:12:28.741537+0000","last_active":"2024-04-17T07:12:28.741537+0000","last_peered":"2024-04-17T07:12:28.741537+0000","last_clean":"2024-04-17T07:12:28.741537+0000","last_became_active":"2024-04-17T07:12:28.741035+0000","last_became_peered":"2024-04-17T07:12:28.741035+0000","last_unstale":"2024-04-17T07:12:28.741537+0000","last_undegraded":"2024-04-17T07:12:28.741537+0000","last_fullsized":"2024-04-17T07:12:28.741537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:15.550112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.195614+0000","last_change":"2024-04-17T07:11:54.471118+0000","last_active":"2024-04-17T07:12:29.195614+0000","last_peered":"2024-04-17T07:12:29.195614+0000","last_clean":"2024-04-17T07:12:29.195614+0000","last_became_active":"2024-04-17T07:11:54.470526+0000","last_became_peered":"2024-04-17T07:11:54.470526+0000","last_unstale":"2024-04-17T07:12:29.195614+0000","last_undegraded":"2024-04-17T07:12:29.195614+0000","last_fullsized":"2024-04-17T07:12:29.195614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:52:47.918086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735651+0000","last_change":"2024-04-17T07:11:55.472850+0000","last_active":"2024-04-17T07:12:28.735651+0000","last_peered":"2024-04-17T07:12:28.735651+0000","last_clean":"2024-04-17T07:12:28.735651+0000","last_became_active":"2024-04-17T07:11:55.472097+0000","last_became_peered":"2024-04-17T07:11:55.472097+0000","last_unstale":"2024-04-17T07:12:28.735651+0000","last_undegraded":"2024-04-17T07:12:28.735651+0000","last_fullsized":"2024-04-17T07:12:28.735651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:41.730809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.729619+0000","last_change":"2024-04-17T07:11:59.522960+0000","last_active":"2024-04-17T07:12:28.729619+0000","last_peered":"2024-04-17T07:12:28.729619+0000","last_clean":"2024-04-17T07:12:28.729619+0000","last_became_active":"2024-04-17T07:11:59.522470+0000","last_became_peered":"2024-04-17T07:11:59.522470+0000","last_unstale":"2024-04-17T07:12:28.729619+0000","last_undegraded":"2024-04-17T07:12:28.729619+0000","last_fullsized":"2024-04-17T07:12:28.729619+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:36.620442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720922+0000","last_change":"2024-04-17T07:11:54.470693+0000","last_active":"2024-04-17T07:12:28.720922+0000","last_peered":"2024-04-17T07:12:28.720922+0000","last_clean":"2024-04-17T07:12:28.720922+0000","last_became_active":"2024-04-17T07:11:54.470334+0000","last_became_peered":"2024-04-17T07:11:54.470334+0000","last_unstale":"2024-04-17T07:12:28.720922+0000","last_undegraded":"2024-04-17T07:12:28.720922+0000","last_fullsized":"2024-04-17T07:12:28.720922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:19:04.980786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122063+0000","last_change":"2024-04-17T07:11:55.474755+0000","last_active":"2024-04-17T07:12:29.122063+0000","last_peered":"2024-04-17T07:12:29.122063+0000","last_clean":"2024-04-17T07:12:29.122063+0000","last_became_active":"2024-04-17T07:11:55.474261+0000","last_became_peered":"2024-04-17T07:11:55.474261+0000","last_unstale":"2024-04-17T07:12:29.122063+0000","last_undegraded":"2024-04-17T07:12:29.122063+0000","last_fullsized":"2024-04-17T07:12:29.122063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:00.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735779+0000","last_change":"2024-04-17T07:11:59.517152+0000","last_active":"2024-04-17T07:12:28.735779+0000","last_peered":"2024-04-17T07:12:28.735779+0000","last_clean":"2024-04-17T07:12:28.735779+0000","last_became_active":"2024-04-17T07:11:59.514977+0000","last_became_peered":"2024-04-17T07:11:59.514977+0000","last_unstale":"2024-04-17T07:12:28.735779+0000","last_undegraded":"2024-04-17T07:12:28.735779+0000","last_fullsized":"2024-04-17T07:12:28.735779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:14.974136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.212620+0000","last_change":"2024-04-17T07:11:54.459786+0000","last_active":"2024-04-17T07:12:30.212620+0000","last_peered":"2024-04-17T07:12:30.212620+0000","last_clean":"2024-04-17T07:12:30.212620+0000","last_became_active":"2024-04-17T07:11:54.457589+0000","last_became_peered":"2024-04-17T07:11:54.457589+0000","last_unstale":"2024-04-17T07:12:30.212620+0000","last_undegraded":"2024-04-17T07:12:30.212620+0000","last_fullsized":"2024-04-17T07:12:30.212620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:25.986539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714421+0000","last_change":"2024-04-17T07:11:55.465020+0000","last_active":"2024-04-17T07:12:28.714421+0000","last_peered":"2024-04-17T07:12:28.714421+0000","last_clean":"2024-04-17T07:12:28.714421+0000","last_became_active":"2024-04-17T07:11:55.464608+0000","last_became_peered":"2024-04-17T07:11:55.464608+0000","last_unstale":"2024-04-17T07:12:28.714421+0000","last_undegraded":"2024-04-17T07:12:28.714421+0000","last_fullsized":"2024-04-17T07:12:28.714421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122725+0000","last_change":"2024-04-17T07:11:59.519333+0000","last_active":"2024-04-17T07:12:29.122725+0000","last_peered":"2024-04-17T07:12:29.122725+0000","last_clean":"2024-04-17T07:12:29.122725+0000","last_became_active":"2024-04-17T07:11:59.518865+0000","last_became_peered":"2024-04-17T07:11:59.518865+0000","last_unstale":"2024-04-17T07:12:29.122725+0000","last_undegraded":"2024-04-17T07:12:29.122725+0000","last_fullsized":"2024-04-17T07:12:29.122725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:42.009010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.227304+0000","last_change":"2024-04-17T07:11:54.462283+0000","last_active":"2024-04-17T07:12:32.227304+0000","last_peered":"2024-04-17T07:12:32.227304+0000","last_clean":"2024-04-17T07:12:32.227304+0000","last_became_active":"2024-04-17T07:11:54.461818+0000","last_became_peered":"2024-04-17T07:11:54.461818+0000","last_unstale":"2024-04-17T07:12:32.227304+0000","last_undegraded":"2024-04-17T07:12:32.227304+0000","last_fullsized":"2024-04-17T07:12:32.227304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.765058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726968+0000","last_change":"2024-04-17T07:11:55.486491+0000","last_active":"2024-04-17T07:12:28.726968+0000","last_peered":"2024-04-17T07:12:28.726968+0000","last_clean":"2024-04-17T07:12:28.726968+0000","last_became_active":"2024-04-17T07:11:55.486121+0000","last_became_peered":"2024-04-17T07:11:55.486121+0000","last_unstale":"2024-04-17T07:12:28.726968+0000","last_undegraded":"2024-04-17T07:12:28.726968+0000","last_fullsized":"2024-04-17T07:12:28.726968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:12.998321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713871+0000","last_change":"2024-04-17T07:11:59.511555+0000","last_active":"2024-04-17T07:12:28.713871+0000","last_peered":"2024-04-17T07:12:28.713871+0000","last_clean":"2024-04-17T07:12:28.713871+0000","last_became_active":"2024-04-17T07:11:59.511147+0000","last_became_peered":"2024-04-17T07:11:59.511147+0000","last_unstale":"2024-04-17T07:12:28.713871+0000","last_undegraded":"2024-04-17T07:12:28.713871+0000","last_fullsized":"2024-04-17T07:12:28.713871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:41.854815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.210000+0000","last_change":"2024-04-17T07:12:28.734476+0000","last_active":"2024-04-17T07:12:30.210000+0000","last_peered":"2024-04-17T07:12:30.210000+0000","last_clean":"2024-04-17T07:12:30.210000+0000","last_became_active":"2024-04-17T07:12:28.728187+0000","last_became_peered":"2024-04-17T07:12:28.728187+0000","last_unstale":"2024-04-17T07:12:30.210000+0000","last_undegraded":"2024-04-17T07:12:30.210000+0000","last_fullsized":"2024-04-17T07:12:30.210000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:45.749607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736562+0000","last_change":"2024-04-17T07:11:55.480589+0000","last_active":"2024-04-17T07:12:28.736562+0000","last_peered":"2024-04-17T07:12:28.736562+0000","last_clean":"2024-04-17T07:12:28.736562+0000","last_became_active":"2024-04-17T07:11:55.480224+0000","last_became_peered":"2024-04-17T07:11:55.480224+0000","last_unstale":"2024-04-17T07:12:28.736562+0000","last_undegraded":"2024-04-17T07:12:28.736562+0000","last_fullsized":"2024-04-17T07:12:28.736562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:05.892442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713815+0000","last_change":"2024-04-17T07:11:59.522021+0000","last_active":"2024-04-17T07:12:28.713815+0000","last_peered":"2024-04-17T07:12:28.713815+0000","last_clean":"2024-04-17T07:12:28.713815+0000","last_became_active":"2024-04-17T07:11:59.521493+0000","last_became_peered":"2024-04-17T07:11:59.521493+0000","last_unstale":"2024-04-17T07:12:28.713815+0000","last_undegraded":"2024-04-17T07:12:28.713815+0000","last_fullsized":"2024-04-17T07:12:28.713815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:20.386216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.195512+0000","last_change":"2024-04-17T07:11:54.468225+0000","last_active":"2024-04-17T07:12:29.195512+0000","last_peered":"2024-04-17T07:12:29.195512+0000","last_clean":"2024-04-17T07:12:29.195512+0000","last_became_active":"2024-04-17T07:11:54.467046+0000","last_became_peered":"2024-04-17T07:11:54.467046+0000","last_unstale":"2024-04-17T07:12:29.195512+0000","last_undegraded":"2024-04-17T07:12:29.195512+0000","last_fullsized":"2024-04-17T07:12:29.195512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:17.725255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726428+0000","last_change":"2024-04-17T07:11:55.488980+0000","last_active":"2024-04-17T07:12:28.726428+0000","last_peered":"2024-04-17T07:12:28.726428+0000","last_clean":"2024-04-17T07:12:28.726428+0000","last_became_active":"2024-04-17T07:11:55.488226+0000","last_became_peered":"2024-04-17T07:11:55.488226+0000","last_unstale":"2024-04-17T07:12:28.726428+0000","last_undegraded":"2024-04-17T07:12:28.726428+0000","last_fullsized":"2024-04-17T07:12:28.726428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:00.135973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714388+0000","last_change":"2024-04-17T07:11:59.515134+0000","last_active":"2024-04-17T07:12:28.714388+0000","last_peered":"2024-04-17T07:12:28.714388+0000","last_clean":"2024-04-17T07:12:28.714388+0000","last_became_active":"2024-04-17T07:11:59.514560+0000","last_became_peered":"2024-04-17T07:11:59.514560+0000","last_unstale":"2024-04-17T07:12:28.714388+0000","last_undegraded":"2024-04-17T07:12:28.714388+0000","last_fullsized":"2024-04-17T07:12:28.714388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:34.669789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'26","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.230037+0000","last_change":"2024-04-17T07:11:54.461226+0000","last_active":"2024-04-17T07:12:32.230037+0000","last_peered":"2024-04-17T07:12:32.230037+0000","last_clean":"2024-04-17T07:12:32.230037+0000","last_became_active":"2024-04-17T07:11:54.460573+0000","last_became_peered":"2024-04-17T07:11:54.460573+0000","last_unstale":"2024-04-17T07:12:32.230037+0000","last_undegraded":"2024-04-17T07:12:32.230037+0000","last_fullsized":"2024-04-17T07:12:32.230037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":44,"num_write":28,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.734828+0000","last_change":"2024-04-17T07:11:55.483925+0000","last_active":"2024-04-17T07:12:28.734828+0000","last_peered":"2024-04-17T07:12:28.734828+0000","last_clean":"2024-04-17T07:12:28.734828+0000","last_became_active":"2024-04-17T07:11:55.483584+0000","last_became_peered":"2024-04-17T07:11:55.483584+0000","last_unstale":"2024-04-17T07:12:28.734828+0000","last_undegraded":"2024-04-17T07:12:28.734828+0000","last_fullsized":"2024-04-17T07:12:28.734828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:12.328876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.737367+0000","last_change":"2024-04-17T07:11:59.515597+0000","last_active":"2024-04-17T07:12:28.737367+0000","last_peered":"2024-04-17T07:12:28.737367+0000","last_clean":"2024-04-17T07:12:28.737367+0000","last_became_active":"2024-04-17T07:11:59.514413+0000","last_became_peered":"2024-04-17T07:11:59.514413+0000","last_unstale":"2024-04-17T07:12:28.737367+0000","last_undegraded":"2024-04-17T07:12:28.737367+0000","last_fullsized":"2024-04-17T07:12:28.737367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:45.824127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.209717+0000","last_change":"2024-04-17T07:11:54.449984+0000","last_active":"2024-04-17T07:12:30.209717+0000","last_peered":"2024-04-17T07:12:30.209717+0000","last_clean":"2024-04-17T07:12:30.209717+0000","last_became_active":"2024-04-17T07:11:54.449628+0000","last_became_peered":"2024-04-17T07:11:54.449628+0000","last_unstale":"2024-04-17T07:12:30.209717+0000","last_undegraded":"2024-04-17T07:12:30.209717+0000","last_fullsized":"2024-04-17T07:12:30.209717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:26:03.281684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719923+0000","last_change":"2024-04-17T07:11:55.476912+0000","last_active":"2024-04-17T07:12:28.719923+0000","last_peered":"2024-04-17T07:12:28.719923+0000","last_clean":"2024-04-17T07:12:28.719923+0000","last_became_active":"2024-04-17T07:11:55.471576+0000","last_became_peered":"2024-04-17T07:11:55.471576+0000","last_unstale":"2024-04-17T07:12:28.719923+0000","last_undegraded":"2024-04-17T07:12:28.719923+0000","last_fullsized":"2024-04-17T07:12:28.719923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:35.439271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726532+0000","last_change":"2024-04-17T07:11:59.515897+0000","last_active":"2024-04-17T07:12:28.726532+0000","last_peered":"2024-04-17T07:12:28.726532+0000","last_clean":"2024-04-17T07:12:28.726532+0000","last_became_active":"2024-04-17T07:11:59.515349+0000","last_became_peered":"2024-04-17T07:11:59.515349+0000","last_unstale":"2024-04-17T07:12:28.726532+0000","last_undegraded":"2024-04-17T07:12:28.726532+0000","last_fullsized":"2024-04-17T07:12:28.726532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:00.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214859+0000","last_change":"2024-04-17T07:11:54.462369+0000","last_active":"2024-04-17T07:12:30.214859+0000","last_peered":"2024-04-17T07:12:30.214859+0000","last_clean":"2024-04-17T07:12:30.214859+0000","last_became_active":"2024-04-17T07:11:54.460357+0000","last_became_peered":"2024-04-17T07:11:54.460357+0000","last_unstale":"2024-04-17T07:12:30.214859+0000","last_undegraded":"2024-04-17T07:12:30.214859+0000","last_fullsized":"2024-04-17T07:12:30.214859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:38.029980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720650+0000","last_change":"2024-04-17T07:11:55.475874+0000","last_active":"2024-04-17T07:12:28.720650+0000","last_peered":"2024-04-17T07:12:28.720650+0000","last_clean":"2024-04-17T07:12:28.720650+0000","last_became_active":"2024-04-17T07:11:55.475550+0000","last_became_peered":"2024-04-17T07:11:55.475550+0000","last_unstale":"2024-04-17T07:12:28.720650+0000","last_undegraded":"2024-04-17T07:12:28.720650+0000","last_fullsized":"2024-04-17T07:12:28.720650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:40.096573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714893+0000","last_change":"2024-04-17T07:11:59.514661+0000","last_active":"2024-04-17T07:12:28.714893+0000","last_peered":"2024-04-17T07:12:28.714893+0000","last_clean":"2024-04-17T07:12:28.714893+0000","last_became_active":"2024-04-17T07:11:59.513030+0000","last_became_peered":"2024-04-17T07:11:59.513030+0000","last_unstale":"2024-04-17T07:12:28.714893+0000","last_undegraded":"2024-04-17T07:12:28.714893+0000","last_fullsized":"2024-04-17T07:12:28.714893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:35.696961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.241199+0000","last_change":"2024-04-17T07:11:54.455279+0000","last_active":"2024-04-17T07:12:32.241199+0000","last_peered":"2024-04-17T07:12:32.241199+0000","last_clean":"2024-04-17T07:12:32.241199+0000","last_became_active":"2024-04-17T07:11:54.453620+0000","last_became_peered":"2024-04-17T07:11:54.453620+0000","last_unstale":"2024-04-17T07:12:32.241199+0000","last_undegraded":"2024-04-17T07:12:32.241199+0000","last_fullsized":"2024-04-17T07:12:32.241199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:15.227410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719686+0000","last_change":"2024-04-17T07:11:55.469579+0000","last_active":"2024-04-17T07:12:28.719686+0000","last_peered":"2024-04-17T07:12:28.719686+0000","last_clean":"2024-04-17T07:12:28.719686+0000","last_became_active":"2024-04-17T07:11:55.468389+0000","last_became_peered":"2024-04-17T07:11:55.468389+0000","last_unstale":"2024-04-17T07:12:28.719686+0000","last_undegraded":"2024-04-17T07:12:28.719686+0000","last_fullsized":"2024-04-17T07:12:28.719686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:01.274677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720385+0000","last_change":"2024-04-17T07:11:59.512110+0000","last_active":"2024-04-17T07:12:28.720385+0000","last_peered":"2024-04-17T07:12:28.720385+0000","last_clean":"2024-04-17T07:12:28.720385+0000","last_became_active":"2024-04-17T07:11:59.510573+0000","last_became_peered":"2024-04-17T07:11:59.510573+0000","last_unstale":"2024-04-17T07:12:28.720385+0000","last_undegraded":"2024-04-17T07:12:28.720385+0000","last_fullsized":"2024-04-17T07:12:28.720385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:32.515687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.209947+0000","last_change":"2024-04-17T07:12:28.767517+0000","last_active":"2024-04-17T07:12:30.209947+0000","last_peered":"2024-04-17T07:12:30.209947+0000","last_clean":"2024-04-17T07:12:30.209947+0000","last_became_active":"2024-04-17T07:12:28.738908+0000","last_became_peered":"2024-04-17T07:12:28.738908+0000","last_unstale":"2024-04-17T07:12:30.209947+0000","last_undegraded":"2024-04-17T07:12:30.209947+0000","last_fullsized":"2024-04-17T07:12:30.209947+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:47:30.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.731088+0000","last_change":"2024-04-17T07:11:55.473540+0000","last_active":"2024-04-17T07:12:28.731088+0000","last_peered":"2024-04-17T07:12:28.731088+0000","last_clean":"2024-04-17T07:12:28.731088+0000","last_became_active":"2024-04-17T07:11:55.472436+0000","last_became_peered":"2024-04-17T07:11:55.472436+0000","last_unstale":"2024-04-17T07:12:28.731088+0000","last_undegraded":"2024-04-17T07:12:28.731088+0000","last_fullsized":"2024-04-17T07:12:28.731088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:54.317052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730908+0000","last_change":"2024-04-17T07:11:59.521305+0000","last_active":"2024-04-17T07:12:28.730908+0000","last_peered":"2024-04-17T07:12:28.730908+0000","last_clean":"2024-04-17T07:12:28.730908+0000","last_became_active":"2024-04-17T07:11:59.516860+0000","last_became_peered":"2024-04-17T07:11:59.516860+0000","last_unstale":"2024-04-17T07:12:28.730908+0000","last_undegraded":"2024-04-17T07:12:28.730908+0000","last_fullsized":"2024-04-17T07:12:28.730908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:23.403218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735984+0000","last_change":"2024-04-17T07:11:54.463378+0000","last_active":"2024-04-17T07:12:28.735984+0000","last_peered":"2024-04-17T07:12:28.735984+0000","last_clean":"2024-04-17T07:12:28.735984+0000","last_became_active":"2024-04-17T07:11:54.462120+0000","last_became_peered":"2024-04-17T07:11:54.462120+0000","last_unstale":"2024-04-17T07:12:28.735984+0000","last_undegraded":"2024-04-17T07:12:28.735984+0000","last_fullsized":"2024-04-17T07:12:28.735984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:42.109928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727537+0000","last_change":"2024-04-17T07:11:55.478604+0000","last_active":"2024-04-17T07:12:28.727537+0000","last_peered":"2024-04-17T07:12:28.727537+0000","last_clean":"2024-04-17T07:12:28.727537+0000","last_became_active":"2024-04-17T07:11:55.478252+0000","last_became_peered":"2024-04-17T07:11:55.478252+0000","last_unstale":"2024-04-17T07:12:28.727537+0000","last_undegraded":"2024-04-17T07:12:28.727537+0000","last_fullsized":"2024-04-17T07:12:28.727537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:08.365896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727344+0000","last_change":"2024-04-17T07:11:59.518619+0000","last_active":"2024-04-17T07:12:28.727344+0000","last_peered":"2024-04-17T07:12:28.727344+0000","last_clean":"2024-04-17T07:12:28.727344+0000","last_became_active":"2024-04-17T07:11:59.518214+0000","last_became_peered":"2024-04-17T07:11:59.518214+0000","last_unstale":"2024-04-17T07:12:28.727344+0000","last_undegraded":"2024-04-17T07:12:28.727344+0000","last_fullsized":"2024-04-17T07:12:28.727344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:46.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715522+0000","last_change":"2024-04-17T07:11:54.453038+0000","last_active":"2024-04-17T07:12:28.715522+0000","last_peered":"2024-04-17T07:12:28.715522+0000","last_clean":"2024-04-17T07:12:28.715522+0000","last_became_active":"2024-04-17T07:11:54.452619+0000","last_became_peered":"2024-04-17T07:11:54.452619+0000","last_unstale":"2024-04-17T07:12:28.715522+0000","last_undegraded":"2024-04-17T07:12:28.715522+0000","last_fullsized":"2024-04-17T07:12:28.715522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:25.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.724948+0000","last_change":"2024-04-17T07:11:55.465990+0000","last_active":"2024-04-17T07:12:28.724948+0000","last_peered":"2024-04-17T07:12:28.724948+0000","last_clean":"2024-04-17T07:12:28.724948+0000","last_became_active":"2024-04-17T07:11:55.465523+0000","last_became_peered":"2024-04-17T07:11:55.465523+0000","last_unstale":"2024-04-17T07:12:28.724948+0000","last_undegraded":"2024-04-17T07:12:28.724948+0000","last_fullsized":"2024-04-17T07:12:28.724948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:40.366027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721070+0000","last_change":"2024-04-17T07:11:59.512119+0000","last_active":"2024-04-17T07:12:28.721070+0000","last_peered":"2024-04-17T07:12:28.721070+0000","last_clean":"2024-04-17T07:12:28.721070+0000","last_became_active":"2024-04-17T07:11:59.511460+0000","last_became_peered":"2024-04-17T07:11:59.511460+0000","last_unstale":"2024-04-17T07:12:28.721070+0000","last_undegraded":"2024-04-17T07:12:28.721070+0000","last_fullsized":"2024-04-17T07:12:28.721070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:50.888203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.221574+0000","last_change":"2024-04-17T07:11:54.469346+0000","last_active":"2024-04-17T07:12:31.221574+0000","last_peered":"2024-04-17T07:12:31.221574+0000","last_clean":"2024-04-17T07:12:31.221574+0000","last_became_active":"2024-04-17T07:11:54.468947+0000","last_became_peered":"2024-04-17T07:11:54.468947+0000","last_unstale":"2024-04-17T07:12:31.221574+0000","last_undegraded":"2024-04-17T07:12:31.221574+0000","last_fullsized":"2024-04-17T07:12:31.221574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:04.854625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.724510+0000","last_change":"2024-04-17T07:11:55.471235+0000","last_active":"2024-04-17T07:12:28.724510+0000","last_peered":"2024-04-17T07:12:28.724510+0000","last_clean":"2024-04-17T07:12:28.724510+0000","last_became_active":"2024-04-17T07:11:55.470857+0000","last_became_peered":"2024-04-17T07:11:55.470857+0000","last_unstale":"2024-04-17T07:12:28.724510+0000","last_undegraded":"2024-04-17T07:12:28.724510+0000","last_fullsized":"2024-04-17T07:12:28.724510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:21.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720514+0000","last_change":"2024-04-17T07:11:59.524519+0000","last_active":"2024-04-17T07:12:28.720514+0000","last_peered":"2024-04-17T07:12:28.720514+0000","last_clean":"2024-04-17T07:12:28.720514+0000","last_became_active":"2024-04-17T07:11:59.519270+0000","last_became_peered":"2024-04-17T07:11:59.519270+0000","last_unstale":"2024-04-17T07:12:28.720514+0000","last_undegraded":"2024-04-17T07:12:28.720514+0000","last_fullsized":"2024-04-17T07:12:28.720514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:41.080249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.734024+0000","last_change":"2024-04-17T07:11:55.473922+0000","last_active":"2024-04-17T07:12:28.734024+0000","last_peered":"2024-04-17T07:12:28.734024+0000","last_clean":"2024-04-17T07:12:28.734024+0000","last_became_active":"2024-04-17T07:11:55.468594+0000","last_became_peered":"2024-04-17T07:11:55.468594+0000","last_unstale":"2024-04-17T07:12:28.734024+0000","last_undegraded":"2024-04-17T07:12:28.734024+0000","last_fullsized":"2024-04-17T07:12:28.734024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:21.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.219511+0000","last_change":"2024-04-17T07:11:54.473066+0000","last_active":"2024-04-17T07:12:31.219511+0000","last_peered":"2024-04-17T07:12:31.219511+0000","last_clean":"2024-04-17T07:12:31.219511+0000","last_became_active":"2024-04-17T07:11:54.472708+0000","last_became_peered":"2024-04-17T07:11:54.472708+0000","last_unstale":"2024-04-17T07:12:31.219511+0000","last_undegraded":"2024-04-17T07:12:31.219511+0000","last_fullsized":"2024-04-17T07:12:31.219511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:27.614819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720813+0000","last_change":"2024-04-17T07:11:59.516939+0000","last_active":"2024-04-17T07:12:28.720813+0000","last_peered":"2024-04-17T07:12:28.720813+0000","last_clean":"2024-04-17T07:12:28.720813+0000","last_became_active":"2024-04-17T07:11:59.516333+0000","last_became_peered":"2024-04-17T07:11:59.516333+0000","last_unstale":"2024-04-17T07:12:28.720813+0000","last_undegraded":"2024-04-17T07:12:28.720813+0000","last_fullsized":"2024-04-17T07:12:28.720813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:15.648831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122635+0000","last_change":"2024-04-17T07:11:55.481006+0000","last_active":"2024-04-17T07:12:29.122635+0000","last_peered":"2024-04-17T07:12:29.122635+0000","last_clean":"2024-04-17T07:12:29.122635+0000","last_became_active":"2024-04-17T07:11:55.479751+0000","last_became_peered":"2024-04-17T07:11:55.479751+0000","last_unstale":"2024-04-17T07:12:29.122635+0000","last_undegraded":"2024-04-17T07:12:29.122635+0000","last_fullsized":"2024-04-17T07:12:29.122635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:13.004650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.218667+0000","last_change":"2024-04-17T07:11:54.463106+0000","last_active":"2024-04-17T07:12:31.218667+0000","last_peered":"2024-04-17T07:12:31.218667+0000","last_clean":"2024-04-17T07:12:31.218667+0000","last_became_active":"2024-04-17T07:11:54.462691+0000","last_became_peered":"2024-04-17T07:11:54.462691+0000","last_unstale":"2024-04-17T07:12:31.218667+0000","last_undegraded":"2024-04-17T07:12:31.218667+0000","last_fullsized":"2024-04-17T07:12:31.218667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:34.302019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719632+0000","last_change":"2024-04-17T07:10:14.685883+0000","last_active":"2024-04-17T07:12:28.719632+0000","last_peered":"2024-04-17T07:12:28.719632+0000","last_clean":"2024-04-17T07:12:28.719632+0000","last_became_active":"2024-04-17T07:10:14.684551+0000","last_became_peered":"2024-04-17T07:10:14.684551+0000","last_unstale":"2024-04-17T07:12:28.719632+0000","last_undegraded":"2024-04-17T07:12:28.719632+0000","last_fullsized":"2024-04-17T07:12:28.719632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_clean_scrub_stamp":"2024-04-17T07:05:37.112317+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:04.265147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736804+0000","last_change":"2024-04-17T07:12:28.731807+0000","last_active":"2024-04-17T07:12:28.736804+0000","last_peered":"2024-04-17T07:12:28.736804+0000","last_clean":"2024-04-17T07:12:28.736804+0000","last_became_active":"2024-04-17T07:12:28.730656+0000","last_became_peered":"2024-04-17T07:12:28.730656+0000","last_unstale":"2024-04-17T07:12:28.736804+0000","last_undegraded":"2024-04-17T07:12:28.736804+0000","last_fullsized":"2024-04-17T07:12:28.736804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:58.023333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727980+0000","last_change":"2024-04-17T07:12:28.727980+0000","last_active":"2024-04-17T07:12:28.727980+0000","last_peered":"2024-04-17T07:12:28.727980+0000","last_clean":"2024-04-17T07:12:28.727980+0000","last_became_active":"2024-04-17T07:12:28.727060+0000","last_became_peered":"2024-04-17T07:12:28.727060+0000","last_unstale":"2024-04-17T07:12:28.727980+0000","last_undegraded":"2024-04-17T07:12:28.727980+0000","last_fullsized":"2024-04-17T07:12:28.727980+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:33.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719921+0000","last_change":"2024-04-17T07:11:54.463180+0000","last_active":"2024-04-17T07:12:28.719921+0000","last_peered":"2024-04-17T07:12:28.719921+0000","last_clean":"2024-04-17T07:12:28.719921+0000","last_became_active":"2024-04-17T07:11:54.462793+0000","last_became_peered":"2024-04-17T07:11:54.462793+0000","last_unstale":"2024-04-17T07:12:28.719921+0000","last_undegraded":"2024-04-17T07:12:28.719921+0000","last_fullsized":"2024-04-17T07:12:28.719921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:19.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725190+0000","last_change":"2024-04-17T07:11:59.518355+0000","last_active":"2024-04-17T07:12:28.725190+0000","last_peered":"2024-04-17T07:12:28.725190+0000","last_clean":"2024-04-17T07:12:28.725190+0000","last_became_active":"2024-04-17T07:11:59.517619+0000","last_became_peered":"2024-04-17T07:11:59.517619+0000","last_unstale":"2024-04-17T07:12:28.725190+0000","last_undegraded":"2024-04-17T07:12:28.725190+0000","last_fullsized":"2024-04-17T07:12:28.725190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:36.826021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.196898+0000","last_change":"2024-04-17T07:11:54.465285+0000","last_active":"2024-04-17T07:12:29.196898+0000","last_peered":"2024-04-17T07:12:29.196898+0000","last_clean":"2024-04-17T07:12:29.196898+0000","last_became_active":"2024-04-17T07:11:54.464818+0000","last_became_peered":"2024-04-17T07:11:54.464818+0000","last_unstale":"2024-04-17T07:12:29.196898+0000","last_undegraded":"2024-04-17T07:12:29.196898+0000","last_fullsized":"2024-04-17T07:12:29.196898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721576+0000","last_change":"2024-04-17T07:11:55.470405+0000","last_active":"2024-04-17T07:12:28.721576+0000","last_peered":"2024-04-17T07:12:28.721576+0000","last_clean":"2024-04-17T07:12:28.721576+0000","last_became_active":"2024-04-17T07:11:55.470058+0000","last_became_peered":"2024-04-17T07:11:55.470058+0000","last_unstale":"2024-04-17T07:12:28.721576+0000","last_undegraded":"2024-04-17T07:12:28.721576+0000","last_fullsized":"2024-04-17T07:12:28.721576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:17.416775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122022+0000","last_change":"2024-04-17T07:11:59.522040+0000","last_active":"2024-04-17T07:12:29.122022+0000","last_peered":"2024-04-17T07:12:29.122022+0000","last_clean":"2024-04-17T07:12:29.122022+0000","last_became_active":"2024-04-17T07:11:59.519771+0000","last_became_peered":"2024-04-17T07:11:59.519771+0000","last_unstale":"2024-04-17T07:12:29.122022+0000","last_undegraded":"2024-04-17T07:12:29.122022+0000","last_fullsized":"2024-04-17T07:12:29.122022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:58.203737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.488533+0000","last_change":"2024-04-17T07:11:54.457954+0000","last_active":"2024-04-17T07:12:30.488533+0000","last_peered":"2024-04-17T07:12:30.488533+0000","last_clean":"2024-04-17T07:12:30.488533+0000","last_became_active":"2024-04-17T07:11:54.457560+0000","last_became_peered":"2024-04-17T07:11:54.457560+0000","last_unstale":"2024-04-17T07:12:30.488533+0000","last_undegraded":"2024-04-17T07:12:30.488533+0000","last_fullsized":"2024-04-17T07:12:30.488533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:31.919730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123098+0000","last_change":"2024-04-17T07:11:55.480475+0000","last_active":"2024-04-17T07:12:29.123098+0000","last_peered":"2024-04-17T07:12:29.123098+0000","last_clean":"2024-04-17T07:12:29.123098+0000","last_became_active":"2024-04-17T07:11:55.480060+0000","last_became_peered":"2024-04-17T07:11:55.480060+0000","last_unstale":"2024-04-17T07:12:29.123098+0000","last_undegraded":"2024-04-17T07:12:29.123098+0000","last_fullsized":"2024-04-17T07:12:29.123098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:30:23.600244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714322+0000","last_change":"2024-04-17T07:11:59.525422+0000","last_active":"2024-04-17T07:12:28.714322+0000","last_peered":"2024-04-17T07:12:28.714322+0000","last_clean":"2024-04-17T07:12:28.714322+0000","last_became_active":"2024-04-17T07:11:59.523649+0000","last_became_peered":"2024-04-17T07:11:59.523649+0000","last_unstale":"2024-04-17T07:12:28.714322+0000","last_undegraded":"2024-04-17T07:12:28.714322+0000","last_fullsized":"2024-04-17T07:12:28.714322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:16.653752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214020+0000","last_change":"2024-04-17T07:11:54.465824+0000","last_active":"2024-04-17T07:12:30.214020+0000","last_peered":"2024-04-17T07:12:30.214020+0000","last_clean":"2024-04-17T07:12:30.214020+0000","last_became_active":"2024-04-17T07:11:54.465348+0000","last_became_peered":"2024-04-17T07:11:54.465348+0000","last_unstale":"2024-04-17T07:12:30.214020+0000","last_undegraded":"2024-04-17T07:12:30.214020+0000","last_fullsized":"2024-04-17T07:12:30.214020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:17.729903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736284+0000","last_change":"2024-04-17T07:11:55.481666+0000","last_active":"2024-04-17T07:12:28.736284+0000","last_peered":"2024-04-17T07:12:28.736284+0000","last_clean":"2024-04-17T07:12:28.736284+0000","last_became_active":"2024-04-17T07:11:55.480332+0000","last_became_peered":"2024-04-17T07:11:55.480332+0000","last_unstale":"2024-04-17T07:12:28.736284+0000","last_undegraded":"2024-04-17T07:12:28.736284+0000","last_fullsized":"2024-04-17T07:12:28.736284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:19.825407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714433+0000","last_change":"2024-04-17T07:11:59.527850+0000","last_active":"2024-04-17T07:12:28.714433+0000","last_peered":"2024-04-17T07:12:28.714433+0000","last_clean":"2024-04-17T07:12:28.714433+0000","last_became_active":"2024-04-17T07:11:59.527306+0000","last_became_peered":"2024-04-17T07:11:59.527306+0000","last_unstale":"2024-04-17T07:12:28.714433+0000","last_undegraded":"2024-04-17T07:12:28.714433+0000","last_fullsized":"2024-04-17T07:12:28.714433+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:35.906222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725682+0000","last_change":"2024-04-17T07:11:54.455862+0000","last_active":"2024-04-17T07:12:28.725682+0000","last_peered":"2024-04-17T07:12:28.725682+0000","last_clean":"2024-04-17T07:12:28.725682+0000","last_became_active":"2024-04-17T07:11:54.453904+0000","last_became_peered":"2024-04-17T07:11:54.453904+0000","last_unstale":"2024-04-17T07:12:28.725682+0000","last_undegraded":"2024-04-17T07:12:28.725682+0000","last_fullsized":"2024-04-17T07:12:28.725682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:01.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725079+0000","last_change":"2024-04-17T07:11:55.470315+0000","last_active":"2024-04-17T07:12:28.725079+0000","last_peered":"2024-04-17T07:12:28.725079+0000","last_clean":"2024-04-17T07:12:28.725079+0000","last_became_active":"2024-04-17T07:11:55.469829+0000","last_became_peered":"2024-04-17T07:11:55.469829+0000","last_unstale":"2024-04-17T07:12:28.725079+0000","last_undegraded":"2024-04-17T07:12:28.725079+0000","last_fullsized":"2024-04-17T07:12:28.725079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:48.197546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.126045+0000","last_change":"2024-04-17T07:12:29.125447+0000","last_active":"2024-04-17T07:12:29.126045+0000","last_peered":"2024-04-17T07:12:29.126045+0000","last_clean":"2024-04-17T07:12:29.126045+0000","last_became_active":"2024-04-17T07:12:29.124551+0000","last_became_peered":"2024-04-17T07:12:29.124551+0000","last_unstale":"2024-04-17T07:12:29.126045+0000","last_undegraded":"2024-04-17T07:12:29.126045+0000","last_fullsized":"2024-04-17T07:12:29.126045+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:09.489755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.216091+0000","last_change":"2024-04-17T07:11:54.453117+0000","last_active":"2024-04-17T07:12:30.216091+0000","last_peered":"2024-04-17T07:12:30.216091+0000","last_clean":"2024-04-17T07:12:30.216091+0000","last_became_active":"2024-04-17T07:11:54.452632+0000","last_became_peered":"2024-04-17T07:11:54.452632+0000","last_unstale":"2024-04-17T07:12:30.216091+0000","last_undegraded":"2024-04-17T07:12:30.216091+0000","last_fullsized":"2024-04-17T07:12:30.216091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:09.366149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726470+0000","last_change":"2024-04-17T07:11:55.469991+0000","last_active":"2024-04-17T07:12:28.726470+0000","last_peered":"2024-04-17T07:12:28.726470+0000","last_clean":"2024-04-17T07:12:28.726470+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:12:28.726470+0000","last_undegraded":"2024-04-17T07:12:28.726470+0000","last_fullsized":"2024-04-17T07:12:28.726470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:02.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713959+0000","last_change":"2024-04-17T07:11:59.519817+0000","last_active":"2024-04-17T07:12:28.713959+0000","last_peered":"2024-04-17T07:12:28.713959+0000","last_clean":"2024-04-17T07:12:28.713959+0000","last_became_active":"2024-04-17T07:11:59.519108+0000","last_became_peered":"2024-04-17T07:11:59.519108+0000","last_unstale":"2024-04-17T07:12:28.713959+0000","last_undegraded":"2024-04-17T07:12:28.713959+0000","last_fullsized":"2024-04-17T07:12:28.713959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:58.378837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716195+0000","last_change":"2024-04-17T07:11:54.463618+0000","last_active":"2024-04-17T07:12:28.716195+0000","last_peered":"2024-04-17T07:12:28.716195+0000","last_clean":"2024-04-17T07:12:28.716195+0000","last_became_active":"2024-04-17T07:11:54.463257+0000","last_became_peered":"2024-04-17T07:11:54.463257+0000","last_unstale":"2024-04-17T07:12:28.716195+0000","last_undegraded":"2024-04-17T07:12:28.716195+0000","last_fullsized":"2024-04-17T07:12:28.716195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:20.135338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122992+0000","last_change":"2024-04-17T07:11:55.487471+0000","last_active":"2024-04-17T07:12:29.122992+0000","last_peered":"2024-04-17T07:12:29.122992+0000","last_clean":"2024-04-17T07:12:29.122992+0000","last_became_active":"2024-04-17T07:11:55.487024+0000","last_became_peered":"2024-04-17T07:11:55.487024+0000","last_unstale":"2024-04-17T07:12:29.122992+0000","last_undegraded":"2024-04-17T07:12:29.122992+0000","last_fullsized":"2024-04-17T07:12:29.122992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:36.851210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714472+0000","last_change":"2024-04-17T07:11:59.527017+0000","last_active":"2024-04-17T07:12:28.714472+0000","last_peered":"2024-04-17T07:12:28.714472+0000","last_clean":"2024-04-17T07:12:28.714472+0000","last_became_active":"2024-04-17T07:11:59.526541+0000","last_became_peered":"2024-04-17T07:11:59.526541+0000","last_unstale":"2024-04-17T07:12:28.714472+0000","last_undegraded":"2024-04-17T07:12:28.714472+0000","last_fullsized":"2024-04-17T07:12:28.714472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:54.411647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.728776+0000","last_change":"2024-04-17T07:11:54.465671+0000","last_active":"2024-04-17T07:12:28.728776+0000","last_peered":"2024-04-17T07:12:28.728776+0000","last_clean":"2024-04-17T07:12:28.728776+0000","last_became_active":"2024-04-17T07:11:54.464902+0000","last_became_peered":"2024-04-17T07:11:54.464902+0000","last_unstale":"2024-04-17T07:12:28.728776+0000","last_undegraded":"2024-04-17T07:12:28.728776+0000","last_fullsized":"2024-04-17T07:12:28.728776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:13:38.232965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720011+0000","last_change":"2024-04-17T07:11:55.484259+0000","last_active":"2024-04-17T07:12:28.720011+0000","last_peered":"2024-04-17T07:12:28.720011+0000","last_clean":"2024-04-17T07:12:28.720011+0000","last_became_active":"2024-04-17T07:11:55.483911+0000","last_became_peered":"2024-04-17T07:11:55.483911+0000","last_unstale":"2024-04-17T07:12:28.720011+0000","last_undegraded":"2024-04-17T07:12:28.720011+0000","last_fullsized":"2024-04-17T07:12:28.720011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:54.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725852+0000","last_change":"2024-04-17T07:11:59.525369+0000","last_active":"2024-04-17T07:12:28.725852+0000","last_peered":"2024-04-17T07:12:28.725852+0000","last_clean":"2024-04-17T07:12:28.725852+0000","last_became_active":"2024-04-17T07:11:59.524809+0000","last_became_peered":"2024-04-17T07:11:59.524809+0000","last_unstale":"2024-04-17T07:12:28.725852+0000","last_undegraded":"2024-04-17T07:12:28.725852+0000","last_fullsized":"2024-04-17T07:12:28.725852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:20.511001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.197527+0000","last_change":"2024-04-17T07:11:54.459079+0000","last_active":"2024-04-17T07:12:29.197527+0000","last_peered":"2024-04-17T07:12:29.197527+0000","last_clean":"2024-04-17T07:12:29.197527+0000","last_became_active":"2024-04-17T07:11:54.457008+0000","last_became_peered":"2024-04-17T07:11:54.457008+0000","last_unstale":"2024-04-17T07:12:29.197527+0000","last_undegraded":"2024-04-17T07:12:29.197527+0000","last_fullsized":"2024-04-17T07:12:29.197527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:36.084512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721123+0000","last_change":"2024-04-17T07:11:55.484453+0000","last_active":"2024-04-17T07:12:28.721123+0000","last_peered":"2024-04-17T07:12:28.721123+0000","last_clean":"2024-04-17T07:12:28.721123+0000","last_became_active":"2024-04-17T07:11:55.483152+0000","last_became_peered":"2024-04-17T07:11:55.483152+0000","last_unstale":"2024-04-17T07:12:28.721123+0000","last_undegraded":"2024-04-17T07:12:28.721123+0000","last_fullsized":"2024-04-17T07:12:28.721123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:32.224868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735401+0000","last_change":"2024-04-17T07:11:59.521487+0000","last_active":"2024-04-17T07:12:28.735401+0000","last_peered":"2024-04-17T07:12:28.735401+0000","last_clean":"2024-04-17T07:12:28.735401+0000","last_became_active":"2024-04-17T07:11:59.521060+0000","last_became_peered":"2024-04-17T07:11:59.521060+0000","last_unstale":"2024-04-17T07:12:28.735401+0000","last_undegraded":"2024-04-17T07:12:28.735401+0000","last_fullsized":"2024-04-17T07:12:28.735401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:06.692434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214903+0000","last_change":"2024-04-17T07:11:54.459368+0000","last_active":"2024-04-17T07:12:30.214903+0000","last_peered":"2024-04-17T07:12:30.214903+0000","last_clean":"2024-04-17T07:12:30.214903+0000","last_became_active":"2024-04-17T07:11:54.458973+0000","last_became_peered":"2024-04-17T07:11:54.458973+0000","last_unstale":"2024-04-17T07:12:30.214903+0000","last_undegraded":"2024-04-17T07:12:30.214903+0000","last_fullsized":"2024-04-17T07:12:30.214903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:22.034110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721530+0000","last_change":"2024-04-17T07:11:55.470648+0000","last_active":"2024-04-17T07:12:28.721530+0000","last_peered":"2024-04-17T07:12:28.721530+0000","last_clean":"2024-04-17T07:12:28.721530+0000","last_became_active":"2024-04-17T07:11:55.469666+0000","last_became_peered":"2024-04-17T07:11:55.469666+0000","last_unstale":"2024-04-17T07:12:28.721530+0000","last_undegraded":"2024-04-17T07:12:28.721530+0000","last_fullsized":"2024-04-17T07:12:28.721530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:59.208872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730680+0000","last_change":"2024-04-17T07:11:59.524027+0000","last_active":"2024-04-17T07:12:28.730680+0000","last_peered":"2024-04-17T07:12:28.730680+0000","last_clean":"2024-04-17T07:12:28.730680+0000","last_became_active":"2024-04-17T07:11:59.522764+0000","last_became_peered":"2024-04-17T07:11:59.522764+0000","last_unstale":"2024-04-17T07:12:28.730680+0000","last_undegraded":"2024-04-17T07:12:28.730680+0000","last_fullsized":"2024-04-17T07:12:28.730680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:40.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.216540+0000","last_change":"2024-04-17T07:11:54.472884+0000","last_active":"2024-04-17T07:12:30.216540+0000","last_peered":"2024-04-17T07:12:30.216540+0000","last_clean":"2024-04-17T07:12:30.216540+0000","last_became_active":"2024-04-17T07:11:54.472305+0000","last_became_peered":"2024-04-17T07:11:54.472305+0000","last_unstale":"2024-04-17T07:12:30.216540+0000","last_undegraded":"2024-04-17T07:12:30.216540+0000","last_fullsized":"2024-04-17T07:12:30.216540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:21.031442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721232+0000","last_change":"2024-04-17T07:11:55.479982+0000","last_active":"2024-04-17T07:12:28.721232+0000","last_peered":"2024-04-17T07:12:28.721232+0000","last_clean":"2024-04-17T07:12:28.721232+0000","last_became_active":"2024-04-17T07:11:55.477594+0000","last_became_peered":"2024-04-17T07:11:55.477594+0000","last_unstale":"2024-04-17T07:12:28.721232+0000","last_undegraded":"2024-04-17T07:12:28.721232+0000","last_fullsized":"2024-04-17T07:12:28.721232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:23.507312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726804+0000","last_change":"2024-04-17T07:11:55.480941+0000","last_active":"2024-04-17T07:12:28.726804+0000","last_peered":"2024-04-17T07:12:28.726804+0000","last_clean":"2024-04-17T07:12:28.726804+0000","last_became_active":"2024-04-17T07:11:55.480180+0000","last_became_peered":"2024-04-17T07:11:55.480180+0000","last_unstale":"2024-04-17T07:12:28.726804+0000","last_undegraded":"2024-04-17T07:12:28.726804+0000","last_fullsized":"2024-04-17T07:12:28.726804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:16.321806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715350+0000","last_change":"2024-04-17T07:11:55.477663+0000","last_active":"2024-04-17T07:12:28.715350+0000","last_peered":"2024-04-17T07:12:28.715350+0000","last_clean":"2024-04-17T07:12:28.715350+0000","last_became_active":"2024-04-17T07:11:55.477304+0000","last_became_peered":"2024-04-17T07:11:55.477304+0000","last_unstale":"2024-04-17T07:12:28.715350+0000","last_undegraded":"2024-04-17T07:12:28.715350+0000","last_fullsized":"2024-04-17T07:12:28.715350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:00.796633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714460+0000","last_change":"2024-04-17T07:11:55.475536+0000","last_active":"2024-04-17T07:12:28.714460+0000","last_peered":"2024-04-17T07:12:28.714460+0000","last_clean":"2024-04-17T07:12:28.714460+0000","last_became_active":"2024-04-17T07:11:55.475138+0000","last_became_peered":"2024-04-17T07:11:55.475138+0000","last_unstale":"2024-04-17T07:12:28.714460+0000","last_undegraded":"2024-04-17T07:12:28.714460+0000","last_fullsized":"2024-04-17T07:12:28.714460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:47.849076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735949+0000","last_change":"2024-04-17T07:11:55.483742+0000","last_active":"2024-04-17T07:12:28.735949+0000","last_peered":"2024-04-17T07:12:28.735949+0000","last_clean":"2024-04-17T07:12:28.735949+0000","last_became_active":"2024-04-17T07:11:55.483044+0000","last_became_peered":"2024-04-17T07:11:55.483044+0000","last_unstale":"2024-04-17T07:12:28.735949+0000","last_undegraded":"2024-04-17T07:12:28.735949+0000","last_fullsized":"2024-04-17T07:12:28.735949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:18.972137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721440+0000","last_change":"2024-04-17T07:11:55.471266+0000","last_active":"2024-04-17T07:12:28.721440+0000","last_peered":"2024-04-17T07:12:28.721440+0000","last_clean":"2024-04-17T07:12:28.721440+0000","last_became_active":"2024-04-17T07:11:55.470897+0000","last_became_peered":"2024-04-17T07:11:55.470897+0000","last_unstale":"2024-04-17T07:12:28.721440+0000","last_undegraded":"2024-04-17T07:12:28.721440+0000","last_fullsized":"2024-04-17T07:12:28.721440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:25.811064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.728989+0000","last_change":"2024-04-17T07:11:55.475860+0000","last_active":"2024-04-17T07:12:28.728989+0000","last_peered":"2024-04-17T07:12:28.728989+0000","last_clean":"2024-04-17T07:12:28.728989+0000","last_became_active":"2024-04-17T07:11:55.475522+0000","last_became_peered":"2024-04-17T07:11:55.475522+0000","last_unstale":"2024-04-17T07:12:28.728989+0000","last_undegraded":"2024-04-17T07:12:28.728989+0000","last_fullsized":"2024-04-17T07:12:28.728989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:41.548964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715736+0000","last_change":"2024-04-17T07:11:55.478815+0000","last_active":"2024-04-17T07:12:28.715736+0000","last_peered":"2024-04-17T07:12:28.715736+0000","last_clean":"2024-04-17T07:12:28.715736+0000","last_became_active":"2024-04-17T07:11:55.478347+0000","last_became_peered":"2024-04-17T07:11:55.478347+0000","last_unstale":"2024-04-17T07:12:28.715736+0000","last_undegraded":"2024-04-17T07:12:28.715736+0000","last_fullsized":"2024-04-17T07:12:28.715736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:16.465511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123444+0000","last_change":"2024-04-17T07:11:55.474208+0000","last_active":"2024-04-17T07:12:29.123444+0000","last_peered":"2024-04-17T07:12:29.123444+0000","last_clean":"2024-04-17T07:12:29.123444+0000","last_became_active":"2024-04-17T07:11:55.472042+0000","last_became_peered":"2024-04-17T07:11:55.472042+0000","last_unstale":"2024-04-17T07:12:29.123444+0000","last_undegraded":"2024-04-17T07:12:29.123444+0000","last_fullsized":"2024-04-17T07:12:29.123444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:37:24.248506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.733719+0000","last_change":"2024-04-17T07:11:55.477257+0000","last_active":"2024-04-17T07:12:28.733719+0000","last_peered":"2024-04-17T07:12:28.733719+0000","last_clean":"2024-04-17T07:12:28.733719+0000","last_became_active":"2024-04-17T07:11:55.473491+0000","last_became_peered":"2024-04-17T07:11:55.473491+0000","last_unstale":"2024-04-17T07:12:28.733719+0000","last_undegraded":"2024-04-17T07:12:28.733719+0000","last_fullsized":"2024-04-17T07:12:28.733719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:33.737485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122022+0000","last_change":"2024-04-17T07:11:55.474930+0000","last_active":"2024-04-17T07:12:29.122022+0000","last_peered":"2024-04-17T07:12:29.122022+0000","last_clean":"2024-04-17T07:12:29.122022+0000","last_became_active":"2024-04-17T07:11:55.472007+0000","last_became_peered":"2024-04-17T07:11:55.472007+0000","last_unstale":"2024-04-17T07:12:29.122022+0000","last_undegraded":"2024-04-17T07:12:29.122022+0000","last_fullsized":"2024-04-17T07:12:29.122022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:07.111267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726119+0000","last_change":"2024-04-17T07:11:55.492422+0000","last_active":"2024-04-17T07:12:28.726119+0000","last_peered":"2024-04-17T07:12:28.726119+0000","last_clean":"2024-04-17T07:12:28.726119+0000","last_became_active":"2024-04-17T07:11:55.491906+0000","last_became_peered":"2024-04-17T07:11:55.491906+0000","last_unstale":"2024-04-17T07:12:28.726119+0000","last_undegraded":"2024-04-17T07:12:28.726119+0000","last_fullsized":"2024-04-17T07:12:28.726119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:13:05.322586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720385+0000","last_change":"2024-04-17T07:11:55.479436+0000","last_active":"2024-04-17T07:12:28.720385+0000","last_peered":"2024-04-17T07:12:28.720385+0000","last_clean":"2024-04-17T07:12:28.720385+0000","last_became_active":"2024-04-17T07:11:55.478982+0000","last_became_peered":"2024-04-17T07:11:55.478982+0000","last_unstale":"2024-04-17T07:12:28.720385+0000","last_undegraded":"2024-04-17T07:12:28.720385+0000","last_fullsized":"2024-04-17T07:12:28.720385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:20.663457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727023+0000","last_change":"2024-04-17T07:11:55.476708+0000","last_active":"2024-04-17T07:12:28.727023+0000","last_peered":"2024-04-17T07:12:28.727023+0000","last_clean":"2024-04-17T07:12:28.727023+0000","last_became_active":"2024-04-17T07:11:55.476344+0000","last_became_peered":"2024-04-17T07:11:55.476344+0000","last_unstale":"2024-04-17T07:12:28.727023+0000","last_undegraded":"2024-04-17T07:12:28.727023+0000","last_fullsized":"2024-04-17T07:12:28.727023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:58.874987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":28672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279575,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48719,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":40,"num_read_kb":72,"num_write":225,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":95928,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383703,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":501561,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645343,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32280,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712872,"statfs":{"total":95995035648,"available":95961980928,"internally_reserved":0,"allocated":1130496,"data_stored":1130171,"data_compressed":58079,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":501379,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":497549,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":1094158,"data_compressed":56230,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495674,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":496883,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":501598,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":498339,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":507953,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":496569,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":500957,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1093836,"data_compressed":56230,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":13207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":43642,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:12:36.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:36 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2649296459' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-17T07:12:36.574 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:36 smithi098 ceph-mon[35418]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:36.575 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:36 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2649296459' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-17T07:12:36.575 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:36 smithi103 ceph-mon[35495]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:36 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2649296459' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-17T07:12:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:36 smithi086 ceph-mon[31733]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:37 smithi098 ceph-mon[35418]: pgmap v386: 129 pgs: 129 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.8 KiB/s wr, 24 op/s; 66 B/s, 0 objects/s recovering 2024-04-17T07:12:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:37 smithi103 ceph-mon[35495]: pgmap v386: 129 pgs: 129 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.8 KiB/s wr, 24 op/s; 66 B/s, 0 objects/s recovering 2024-04-17T07:12:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:37 smithi086 ceph-mon[31733]: pgmap v386: 129 pgs: 129 active+clean; 624 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.8 KiB/s wr, 24 op/s; 66 B/s, 0 objects/s recovering 2024-04-17T07:12:38.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:12:39.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.080+0000 7fee73e00640 1 -- 172.21.15.86:0/391654370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74105b90 msgr2=0x7fee7410c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:39.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.080+0000 7fee73e00640 1 --2- 172.21.15.86:0/391654370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74105b90 0x7fee7410c420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009ca0 tx=0x7fee6c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:12:39.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.080+0000 7fee73e00640 1 -- 172.21.15.86:0/391654370 shutdown_connections 2024-04-17T07:12:39.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.080+0000 7fee73e00640 1 --2- 172.21.15.86:0/391654370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74105b90 0x7fee7410c420 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.080+0000 7fee73e00640 1 --2- 172.21.15.86:0/391654370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee741051d0 0x7fee74105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.080+0000 7fee73e00640 1 --2- 172.21.15.86:0/391654370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee74103fe0 0x7fee74104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.080+0000 7fee73e00640 1 -- 172.21.15.86:0/391654370 >> 172.21.15.86:0/391654370 conn(0x7fee740ff570 msgr2=0x7fee74101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.080+0000 7fee73e00640 1 -- 172.21.15.86:0/391654370 shutdown_connections 2024-04-17T07:12:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.080+0000 7fee73e00640 1 -- 172.21.15.86:0/391654370 wait complete. 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.081+0000 7fee73e00640 1 Processor -- start 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.081+0000 7fee73e00640 1 -- start start 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.081+0000 7fee73e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee74103fe0 0x7fee7419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.081+0000 7fee73e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee741051d0 0x7fee7419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74105b90 0x7fee741a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee7410fb20 con 0x7fee74105b90 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee7410f9a0 con 0x7fee741051d0 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee7410fca0 con 0x7fee74103fe0 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74105b90 0x7fee741a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74105b90 0x7fee741a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58536/0 (socket says 172.21.15.86:58536) 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73400640 1 -- 172.21.15.86:0/1979404643 learned_addr learned my addr 172.21.15.86:0/1979404643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:39.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73400640 1 -- 172.21.15.86:0/1979404643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee74103fe0 msgr2=0x7fee7419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee72a00640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee74103fe0 0x7fee7419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee6a000640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee741051d0 0x7fee7419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73400640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee74103fe0 0x7fee7419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73400640 1 -- 172.21.15.86:0/1979404643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee741051d0 msgr2=0x7fee7419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73400640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee741051d0 0x7fee7419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.082+0000 7fee73400640 1 -- 172.21.15.86:0/1979404643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee741a1da0 con 0x7fee74105b90 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.083+0000 7fee73400640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74105b90 0x7fee741a1730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee6c032380 tx=0x7fee6c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.083+0000 7fee78a00640 1 -- 172.21.15.86:0/1979404643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c041020 con 0x7fee74105b90 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.083+0000 7fee78a00640 1 -- 172.21.15.86:0/1979404643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee6c003e20 con 0x7fee74105b90 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.083+0000 7fee73e00640 1 -- 172.21.15.86:0/1979404643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee741a2030 con 0x7fee74105b90 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.083+0000 7fee78a00640 1 -- 172.21.15.86:0/1979404643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c039040 con 0x7fee74105b90 2024-04-17T07:12:39.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.083+0000 7fee73e00640 1 -- 172.21.15.86:0/1979404643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee741a2520 con 0x7fee74105b90 2024-04-17T07:12:39.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.084+0000 7fee73e00640 1 -- 172.21.15.86:0/1979404643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee7410b3d0 con 0x7fee74105b90 2024-04-17T07:12:39.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.086+0000 7fee78a00640 1 -- 172.21.15.86:0/1979404643 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fee6c044020 con 0x7fee74105b90 2024-04-17T07:12:39.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.087+0000 7fee78a00640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee580a3c40 0x7fee580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:39.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.088+0000 7fee78a00640 1 -- 172.21.15.86:0/1979404643 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fee6c0f2b30 con 0x7fee74105b90 2024-04-17T07:12:39.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.088+0000 7fee72a00640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee580a3c40 0x7fee580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:39.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.088+0000 7fee72a00640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee580a3c40 0x7fee580a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee64005fd0 tx=0x7fee64005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:39.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.093+0000 7fee78a00640 1 -- 172.21.15.86:0/1979404643 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fee6c04b020 con 0x7fee74105b90 2024-04-17T07:12:39.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.295+0000 7fee73e00640 1 -- 172.21.15.86:0/1979404643 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee740623a0 con 0x7fee580a3c40 2024-04-17T07:12:39.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.321+0000 7fee78a00640 1 -- 172.21.15.86:0/1979404643 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340779 (secure 0 0 0) 0x7fee740623a0 con 0x7fee580a3c40 2024-04-17T07:12:39.326 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:39.328 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-17T07:12:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 -- 172.21.15.86:0/1979404643 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee580a3c40 msgr2=0x7fee580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee580a3c40 0x7fee580a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee64005fd0 tx=0x7fee64005850 comp rx=0 tx=0).stop 2024-04-17T07:12:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 -- 172.21.15.86:0/1979404643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74105b90 msgr2=0x7fee741a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74105b90 0x7fee741a1730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee6c032380 tx=0x7fee6c032820 comp rx=0 tx=0).stop 2024-04-17T07:12:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 -- 172.21.15.86:0/1979404643 shutdown_connections 2024-04-17T07:12:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee580a3c40 0x7fee580a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee74105b90 0x7fee741a1730 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee741051d0 0x7fee7419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 --2- 172.21.15.86:0/1979404643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee74103fe0 0x7fee7419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 -- 172.21.15.86:0/1979404643 >> 172.21.15.86:0/1979404643 conn(0x7fee740ff570 msgr2=0x7fee741085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 -- 172.21.15.86:0/1979404643 shutdown_connections 2024-04-17T07:12:39.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:39.328+0000 7fee73e00640 1 -- 172.21.15.86:0/1979404643 wait complete. 2024-04-17T07:12:39.348 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-04-17T07:12:39.061319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643936,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":221,"num_read_kb":220,"num_write":414,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181358,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365564,"kb_used_data":12248,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576260,"statfs":{"total":1151940427776,"available":1151566090240,"internally_reserved":0,"allocated":12541952,"data_stored":7917806,"data_compressed":360240,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47544,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":-6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":95,"num_read_kb":111,"num_write":158,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.350869"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730198+0000","last_change":"2024-04-17T07:11:55.486746+0000","last_active":"2024-04-17T07:12:28.730198+0000","last_peered":"2024-04-17T07:12:28.730198+0000","last_clean":"2024-04-17T07:12:28.730198+0000","last_became_active":"2024-04-17T07:11:55.486335+0000","last_became_peered":"2024-04-17T07:11:55.486335+0000","last_unstale":"2024-04-17T07:12:28.730198+0000","last_undegraded":"2024-04-17T07:12:28.730198+0000","last_fullsized":"2024-04-17T07:12:28.730198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:48.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735866+0000","last_change":"2024-04-17T07:11:55.481832+0000","last_active":"2024-04-17T07:12:28.735866+0000","last_peered":"2024-04-17T07:12:28.735866+0000","last_clean":"2024-04-17T07:12:28.735866+0000","last_became_active":"2024-04-17T07:11:55.476428+0000","last_became_peered":"2024-04-17T07:11:55.476428+0000","last_unstale":"2024-04-17T07:12:28.735866+0000","last_undegraded":"2024-04-17T07:12:28.735866+0000","last_fullsized":"2024-04-17T07:12:28.735866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:04.452542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736227+0000","last_change":"2024-04-17T07:11:55.483721+0000","last_active":"2024-04-17T07:12:28.736227+0000","last_peered":"2024-04-17T07:12:28.736227+0000","last_clean":"2024-04-17T07:12:28.736227+0000","last_became_active":"2024-04-17T07:11:55.483268+0000","last_became_peered":"2024-04-17T07:11:55.483268+0000","last_unstale":"2024-04-17T07:12:28.736227+0000","last_undegraded":"2024-04-17T07:12:28.736227+0000","last_fullsized":"2024-04-17T07:12:28.736227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:41.584783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714249+0000","last_change":"2024-04-17T07:11:55.485114+0000","last_active":"2024-04-17T07:12:28.714249+0000","last_peered":"2024-04-17T07:12:28.714249+0000","last_clean":"2024-04-17T07:12:28.714249+0000","last_became_active":"2024-04-17T07:11:55.483913+0000","last_became_peered":"2024-04-17T07:11:55.483913+0000","last_unstale":"2024-04-17T07:12:28.714249+0000","last_undegraded":"2024-04-17T07:12:28.714249+0000","last_fullsized":"2024-04-17T07:12:28.714249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:46.211121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726512+0000","last_change":"2024-04-17T07:11:55.491952+0000","last_active":"2024-04-17T07:12:28.726512+0000","last_peered":"2024-04-17T07:12:28.726512+0000","last_clean":"2024-04-17T07:12:28.726512+0000","last_became_active":"2024-04-17T07:11:55.491238+0000","last_became_peered":"2024-04-17T07:11:55.491238+0000","last_unstale":"2024-04-17T07:12:28.726512+0000","last_undegraded":"2024-04-17T07:12:28.726512+0000","last_fullsized":"2024-04-17T07:12:28.726512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:06.759797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725020+0000","last_change":"2024-04-17T07:11:55.471124+0000","last_active":"2024-04-17T07:12:28.725020+0000","last_peered":"2024-04-17T07:12:28.725020+0000","last_clean":"2024-04-17T07:12:28.725020+0000","last_became_active":"2024-04-17T07:11:55.470781+0000","last_became_peered":"2024-04-17T07:11:55.470781+0000","last_unstale":"2024-04-17T07:12:28.725020+0000","last_undegraded":"2024-04-17T07:12:28.725020+0000","last_fullsized":"2024-04-17T07:12:28.725020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:45.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720008+0000","last_change":"2024-04-17T07:11:55.469099+0000","last_active":"2024-04-17T07:12:28.720008+0000","last_peered":"2024-04-17T07:12:28.720008+0000","last_clean":"2024-04-17T07:12:28.720008+0000","last_became_active":"2024-04-17T07:11:55.468584+0000","last_became_peered":"2024-04-17T07:11:55.468584+0000","last_unstale":"2024-04-17T07:12:28.720008+0000","last_undegraded":"2024-04-17T07:12:28.720008+0000","last_fullsized":"2024-04-17T07:12:28.720008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:52.834846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122568+0000","last_change":"2024-04-17T07:11:55.473794+0000","last_active":"2024-04-17T07:12:29.122568+0000","last_peered":"2024-04-17T07:12:29.122568+0000","last_clean":"2024-04-17T07:12:29.122568+0000","last_became_active":"2024-04-17T07:11:55.471919+0000","last_became_peered":"2024-04-17T07:11:55.471919+0000","last_unstale":"2024-04-17T07:12:29.122568+0000","last_undegraded":"2024-04-17T07:12:29.122568+0000","last_fullsized":"2024-04-17T07:12:29.122568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:16.085117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726090+0000","last_change":"2024-04-17T07:11:55.482445+0000","last_active":"2024-04-17T07:12:28.726090+0000","last_peered":"2024-04-17T07:12:28.726090+0000","last_clean":"2024-04-17T07:12:28.726090+0000","last_became_active":"2024-04-17T07:11:55.481552+0000","last_became_peered":"2024-04-17T07:11:55.481552+0000","last_unstale":"2024-04-17T07:12:28.726090+0000","last_undegraded":"2024-04-17T07:12:28.726090+0000","last_fullsized":"2024-04-17T07:12:28.726090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:48.733585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725405+0000","last_change":"2024-04-17T07:11:55.476726+0000","last_active":"2024-04-17T07:12:28.725405+0000","last_peered":"2024-04-17T07:12:28.725405+0000","last_clean":"2024-04-17T07:12:28.725405+0000","last_became_active":"2024-04-17T07:11:55.476320+0000","last_became_peered":"2024-04-17T07:11:55.476320+0000","last_unstale":"2024-04-17T07:12:28.725405+0000","last_undegraded":"2024-04-17T07:12:28.725405+0000","last_fullsized":"2024-04-17T07:12:28.725405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.230181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721027+0000","last_change":"2024-04-17T07:11:55.485798+0000","last_active":"2024-04-17T07:12:28.721027+0000","last_peered":"2024-04-17T07:12:28.721027+0000","last_clean":"2024-04-17T07:12:28.721027+0000","last_became_active":"2024-04-17T07:11:55.484015+0000","last_became_peered":"2024-04-17T07:11:55.484015+0000","last_unstale":"2024-04-17T07:12:28.721027+0000","last_undegraded":"2024-04-17T07:12:28.721027+0000","last_fullsized":"2024-04-17T07:12:28.721027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:37.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.731078+0000","last_change":"2024-04-17T07:12:28.729834+0000","last_active":"2024-04-17T07:12:28.731078+0000","last_peered":"2024-04-17T07:12:28.731078+0000","last_clean":"2024-04-17T07:12:28.731078+0000","last_became_active":"2024-04-17T07:12:28.728694+0000","last_became_peered":"2024-04-17T07:12:28.728694+0000","last_unstale":"2024-04-17T07:12:28.731078+0000","last_undegraded":"2024-04-17T07:12:28.731078+0000","last_fullsized":"2024-04-17T07:12:28.731078+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:46.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720070+0000","last_change":"2024-04-17T07:11:55.450379+0000","last_active":"2024-04-17T07:12:28.720070+0000","last_peered":"2024-04-17T07:12:28.720070+0000","last_clean":"2024-04-17T07:12:28.720070+0000","last_became_active":"2024-04-17T07:11:55.449118+0000","last_became_peered":"2024-04-17T07:11:55.449118+0000","last_unstale":"2024-04-17T07:12:28.720070+0000","last_undegraded":"2024-04-17T07:12:28.720070+0000","last_fullsized":"2024-04-17T07:12:28.720070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:07:34.248044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735022+0000","last_change":"2024-04-17T07:11:55.473839+0000","last_active":"2024-04-17T07:12:28.735022+0000","last_peered":"2024-04-17T07:12:28.735022+0000","last_clean":"2024-04-17T07:12:28.735022+0000","last_became_active":"2024-04-17T07:11:55.473472+0000","last_became_peered":"2024-04-17T07:11:55.473472+0000","last_unstale":"2024-04-17T07:12:28.735022+0000","last_undegraded":"2024-04-17T07:12:28.735022+0000","last_fullsized":"2024-04-17T07:12:28.735022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:02.263272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736022+0000","last_change":"2024-04-17T07:11:55.484820+0000","last_active":"2024-04-17T07:12:28.736022+0000","last_peered":"2024-04-17T07:12:28.736022+0000","last_clean":"2024-04-17T07:12:28.736022+0000","last_became_active":"2024-04-17T07:11:55.484314+0000","last_became_peered":"2024-04-17T07:11:55.484314+0000","last_unstale":"2024-04-17T07:12:28.736022+0000","last_undegraded":"2024-04-17T07:12:28.736022+0000","last_fullsized":"2024-04-17T07:12:28.736022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.186912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725577+0000","last_change":"2024-04-17T07:11:55.492018+0000","last_active":"2024-04-17T07:12:28.725577+0000","last_peered":"2024-04-17T07:12:28.725577+0000","last_clean":"2024-04-17T07:12:28.725577+0000","last_became_active":"2024-04-17T07:11:55.490512+0000","last_became_peered":"2024-04-17T07:11:55.490512+0000","last_unstale":"2024-04-17T07:12:28.725577+0000","last_undegraded":"2024-04-17T07:12:28.725577+0000","last_fullsized":"2024-04-17T07:12:28.725577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:45.436270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.729938+0000","last_change":"2024-04-17T07:11:55.487210+0000","last_active":"2024-04-17T07:12:28.729938+0000","last_peered":"2024-04-17T07:12:28.729938+0000","last_clean":"2024-04-17T07:12:28.729938+0000","last_became_active":"2024-04-17T07:11:55.485713+0000","last_became_peered":"2024-04-17T07:11:55.485713+0000","last_unstale":"2024-04-17T07:12:28.729938+0000","last_undegraded":"2024-04-17T07:12:28.729938+0000","last_fullsized":"2024-04-17T07:12:28.729938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:54.495950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716204+0000","last_change":"2024-04-17T07:11:55.482705+0000","last_active":"2024-04-17T07:12:28.716204+0000","last_peered":"2024-04-17T07:12:28.716204+0000","last_clean":"2024-04-17T07:12:28.716204+0000","last_became_active":"2024-04-17T07:11:55.482349+0000","last_became_peered":"2024-04-17T07:11:55.482349+0000","last_unstale":"2024-04-17T07:12:28.716204+0000","last_undegraded":"2024-04-17T07:12:28.716204+0000","last_fullsized":"2024-04-17T07:12:28.716204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:57.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.716213+0000","last_change":"2024-04-17T07:11:55.467065+0000","last_active":"2024-04-17T07:12:28.716213+0000","last_peered":"2024-04-17T07:12:28.716213+0000","last_clean":"2024-04-17T07:12:28.716213+0000","last_became_active":"2024-04-17T07:11:55.466535+0000","last_became_peered":"2024-04-17T07:11:55.466535+0000","last_unstale":"2024-04-17T07:12:28.716213+0000","last_undegraded":"2024-04-17T07:12:28.716213+0000","last_fullsized":"2024-04-17T07:12:28.716213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:49.955453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715012+0000","last_change":"2024-04-17T07:11:59.515778+0000","last_active":"2024-04-17T07:12:28.715012+0000","last_peered":"2024-04-17T07:12:28.715012+0000","last_clean":"2024-04-17T07:12:28.715012+0000","last_became_active":"2024-04-17T07:11:59.515266+0000","last_became_peered":"2024-04-17T07:11:59.515266+0000","last_unstale":"2024-04-17T07:12:28.715012+0000","last_undegraded":"2024-04-17T07:12:28.715012+0000","last_fullsized":"2024-04-17T07:12:28.715012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:43.084724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:35.246694+0000","last_change":"2024-04-17T07:11:54.467060+0000","last_active":"2024-04-17T07:12:35.246694+0000","last_peered":"2024-04-17T07:12:35.246694+0000","last_clean":"2024-04-17T07:12:35.246694+0000","last_became_active":"2024-04-17T07:11:54.466687+0000","last_became_peered":"2024-04-17T07:11:54.466687+0000","last_unstale":"2024-04-17T07:12:35.246694+0000","last_undegraded":"2024-04-17T07:12:35.246694+0000","last_fullsized":"2024-04-17T07:12:35.246694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:57.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123283+0000","last_change":"2024-04-17T07:11:55.469857+0000","last_active":"2024-04-17T07:12:29.123283+0000","last_peered":"2024-04-17T07:12:29.123283+0000","last_clean":"2024-04-17T07:12:29.123283+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:12:29.123283+0000","last_undegraded":"2024-04-17T07:12:29.123283+0000","last_fullsized":"2024-04-17T07:12:29.123283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:43.565627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.125397+0000","last_change":"2024-04-17T07:12:29.125397+0000","last_active":"2024-04-17T07:12:29.125397+0000","last_peered":"2024-04-17T07:12:29.125397+0000","last_clean":"2024-04-17T07:12:29.125397+0000","last_became_active":"2024-04-17T07:12:29.124158+0000","last_became_peered":"2024-04-17T07:12:29.124158+0000","last_unstale":"2024-04-17T07:12:29.125397+0000","last_undegraded":"2024-04-17T07:12:29.125397+0000","last_fullsized":"2024-04-17T07:12:29.125397+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:24.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123277+0000","last_change":"2024-04-17T07:11:54.464715+0000","last_active":"2024-04-17T07:12:29.123277+0000","last_peered":"2024-04-17T07:12:29.123277+0000","last_clean":"2024-04-17T07:12:29.123277+0000","last_became_active":"2024-04-17T07:11:54.463917+0000","last_became_peered":"2024-04-17T07:11:54.463917+0000","last_unstale":"2024-04-17T07:12:29.123277+0000","last_undegraded":"2024-04-17T07:12:29.123277+0000","last_fullsized":"2024-04-17T07:12:29.123277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:43.738471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736571+0000","last_change":"2024-04-17T07:11:55.482876+0000","last_active":"2024-04-17T07:12:28.736571+0000","last_peered":"2024-04-17T07:12:28.736571+0000","last_clean":"2024-04-17T07:12:28.736571+0000","last_became_active":"2024-04-17T07:11:55.482451+0000","last_became_peered":"2024-04-17T07:11:55.482451+0000","last_unstale":"2024-04-17T07:12:28.736571+0000","last_undegraded":"2024-04-17T07:12:28.736571+0000","last_fullsized":"2024-04-17T07:12:28.736571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:39.394090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713748+0000","last_change":"2024-04-17T07:11:59.522634+0000","last_active":"2024-04-17T07:12:28.713748+0000","last_peered":"2024-04-17T07:12:28.713748+0000","last_clean":"2024-04-17T07:12:28.713748+0000","last_became_active":"2024-04-17T07:11:59.520797+0000","last_became_peered":"2024-04-17T07:11:59.520797+0000","last_unstale":"2024-04-17T07:12:28.713748+0000","last_undegraded":"2024-04-17T07:12:28.713748+0000","last_fullsized":"2024-04-17T07:12:28.713748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:02.316515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.218005+0000","last_change":"2024-04-17T07:12:28.756528+0000","last_active":"2024-04-17T07:12:31.218005+0000","last_peered":"2024-04-17T07:12:31.218005+0000","last_clean":"2024-04-17T07:12:31.218005+0000","last_became_active":"2024-04-17T07:12:28.737008+0000","last_became_peered":"2024-04-17T07:12:28.737008+0000","last_unstale":"2024-04-17T07:12:31.218005+0000","last_undegraded":"2024-04-17T07:12:31.218005+0000","last_fullsized":"2024-04-17T07:12:31.218005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:56:26.719377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726296+0000","last_change":"2024-04-17T07:11:55.485999+0000","last_active":"2024-04-17T07:12:28.726296+0000","last_peered":"2024-04-17T07:12:28.726296+0000","last_clean":"2024-04-17T07:12:28.726296+0000","last_became_active":"2024-04-17T07:11:55.484048+0000","last_became_peered":"2024-04-17T07:11:55.484048+0000","last_unstale":"2024-04-17T07:12:28.726296+0000","last_undegraded":"2024-04-17T07:12:28.726296+0000","last_fullsized":"2024-04-17T07:12:28.726296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:38.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713330+0000","last_change":"2024-04-17T07:11:59.529929+0000","last_active":"2024-04-17T07:12:28.713330+0000","last_peered":"2024-04-17T07:12:28.713330+0000","last_clean":"2024-04-17T07:12:28.713330+0000","last_became_active":"2024-04-17T07:11:59.529400+0000","last_became_peered":"2024-04-17T07:11:59.529400+0000","last_unstale":"2024-04-17T07:12:28.713330+0000","last_undegraded":"2024-04-17T07:12:28.713330+0000","last_fullsized":"2024-04-17T07:12:28.713330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:54.845529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.210405+0000","last_change":"2024-04-17T07:11:54.462852+0000","last_active":"2024-04-17T07:12:30.210405+0000","last_peered":"2024-04-17T07:12:30.210405+0000","last_clean":"2024-04-17T07:12:30.210405+0000","last_became_active":"2024-04-17T07:11:54.459178+0000","last_became_peered":"2024-04-17T07:11:54.459178+0000","last_unstale":"2024-04-17T07:12:30.210405+0000","last_undegraded":"2024-04-17T07:12:30.210405+0000","last_fullsized":"2024-04-17T07:12:30.210405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:07.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.737683+0000","last_change":"2024-04-17T07:11:55.480951+0000","last_active":"2024-04-17T07:12:28.737683+0000","last_peered":"2024-04-17T07:12:28.737683+0000","last_clean":"2024-04-17T07:12:28.737683+0000","last_became_active":"2024-04-17T07:11:55.480507+0000","last_became_peered":"2024-04-17T07:11:55.480507+0000","last_unstale":"2024-04-17T07:12:28.737683+0000","last_undegraded":"2024-04-17T07:12:28.737683+0000","last_fullsized":"2024-04-17T07:12:28.737683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:42.584180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715898+0000","last_change":"2024-04-17T07:11:59.520897+0000","last_active":"2024-04-17T07:12:28.715898+0000","last_peered":"2024-04-17T07:12:28.715898+0000","last_clean":"2024-04-17T07:12:28.715898+0000","last_became_active":"2024-04-17T07:11:59.520350+0000","last_became_peered":"2024-04-17T07:11:59.520350+0000","last_unstale":"2024-04-17T07:12:28.715898+0000","last_undegraded":"2024-04-17T07:12:28.715898+0000","last_fullsized":"2024-04-17T07:12:28.715898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:39.813097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:35.238288+0000","last_change":"2024-04-17T07:12:28.753717+0000","last_active":"2024-04-17T07:12:35.238288+0000","last_peered":"2024-04-17T07:12:35.238288+0000","last_clean":"2024-04-17T07:12:35.238288+0000","last_became_active":"2024-04-17T07:12:28.739556+0000","last_became_peered":"2024-04-17T07:12:28.739556+0000","last_unstale":"2024-04-17T07:12:35.238288+0000","last_undegraded":"2024-04-17T07:12:35.238288+0000","last_fullsized":"2024-04-17T07:12:35.238288+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:11.967731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.732946+0000","last_change":"2024-04-17T07:11:55.476675+0000","last_active":"2024-04-17T07:12:28.732946+0000","last_peered":"2024-04-17T07:12:28.732946+0000","last_clean":"2024-04-17T07:12:28.732946+0000","last_became_active":"2024-04-17T07:11:55.476273+0000","last_became_peered":"2024-04-17T07:11:55.476273+0000","last_unstale":"2024-04-17T07:12:28.732946+0000","last_undegraded":"2024-04-17T07:12:28.732946+0000","last_fullsized":"2024-04-17T07:12:28.732946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:09.612138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.741537+0000","last_change":"2024-04-17T07:12:28.741537+0000","last_active":"2024-04-17T07:12:28.741537+0000","last_peered":"2024-04-17T07:12:28.741537+0000","last_clean":"2024-04-17T07:12:28.741537+0000","last_became_active":"2024-04-17T07:12:28.741035+0000","last_became_peered":"2024-04-17T07:12:28.741035+0000","last_unstale":"2024-04-17T07:12:28.741537+0000","last_undegraded":"2024-04-17T07:12:28.741537+0000","last_fullsized":"2024-04-17T07:12:28.741537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:15.550112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.195614+0000","last_change":"2024-04-17T07:11:54.471118+0000","last_active":"2024-04-17T07:12:29.195614+0000","last_peered":"2024-04-17T07:12:29.195614+0000","last_clean":"2024-04-17T07:12:29.195614+0000","last_became_active":"2024-04-17T07:11:54.470526+0000","last_became_peered":"2024-04-17T07:11:54.470526+0000","last_unstale":"2024-04-17T07:12:29.195614+0000","last_undegraded":"2024-04-17T07:12:29.195614+0000","last_fullsized":"2024-04-17T07:12:29.195614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:52:47.918086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735651+0000","last_change":"2024-04-17T07:11:55.472850+0000","last_active":"2024-04-17T07:12:28.735651+0000","last_peered":"2024-04-17T07:12:28.735651+0000","last_clean":"2024-04-17T07:12:28.735651+0000","last_became_active":"2024-04-17T07:11:55.472097+0000","last_became_peered":"2024-04-17T07:11:55.472097+0000","last_unstale":"2024-04-17T07:12:28.735651+0000","last_undegraded":"2024-04-17T07:12:28.735651+0000","last_fullsized":"2024-04-17T07:12:28.735651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:41.730809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.729619+0000","last_change":"2024-04-17T07:11:59.522960+0000","last_active":"2024-04-17T07:12:28.729619+0000","last_peered":"2024-04-17T07:12:28.729619+0000","last_clean":"2024-04-17T07:12:28.729619+0000","last_became_active":"2024-04-17T07:11:59.522470+0000","last_became_peered":"2024-04-17T07:11:59.522470+0000","last_unstale":"2024-04-17T07:12:28.729619+0000","last_undegraded":"2024-04-17T07:12:28.729619+0000","last_fullsized":"2024-04-17T07:12:28.729619+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:36.620442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720922+0000","last_change":"2024-04-17T07:11:54.470693+0000","last_active":"2024-04-17T07:12:28.720922+0000","last_peered":"2024-04-17T07:12:28.720922+0000","last_clean":"2024-04-17T07:12:28.720922+0000","last_became_active":"2024-04-17T07:11:54.470334+0000","last_became_peered":"2024-04-17T07:11:54.470334+0000","last_unstale":"2024-04-17T07:12:28.720922+0000","last_undegraded":"2024-04-17T07:12:28.720922+0000","last_fullsized":"2024-04-17T07:12:28.720922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:19:04.980786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122063+0000","last_change":"2024-04-17T07:11:55.474755+0000","last_active":"2024-04-17T07:12:29.122063+0000","last_peered":"2024-04-17T07:12:29.122063+0000","last_clean":"2024-04-17T07:12:29.122063+0000","last_became_active":"2024-04-17T07:11:55.474261+0000","last_became_peered":"2024-04-17T07:11:55.474261+0000","last_unstale":"2024-04-17T07:12:29.122063+0000","last_undegraded":"2024-04-17T07:12:29.122063+0000","last_fullsized":"2024-04-17T07:12:29.122063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:00.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735779+0000","last_change":"2024-04-17T07:11:59.517152+0000","last_active":"2024-04-17T07:12:28.735779+0000","last_peered":"2024-04-17T07:12:28.735779+0000","last_clean":"2024-04-17T07:12:28.735779+0000","last_became_active":"2024-04-17T07:11:59.514977+0000","last_became_peered":"2024-04-17T07:11:59.514977+0000","last_unstale":"2024-04-17T07:12:28.735779+0000","last_undegraded":"2024-04-17T07:12:28.735779+0000","last_fullsized":"2024-04-17T07:12:28.735779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:14.974136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.212620+0000","last_change":"2024-04-17T07:11:54.459786+0000","last_active":"2024-04-17T07:12:30.212620+0000","last_peered":"2024-04-17T07:12:30.212620+0000","last_clean":"2024-04-17T07:12:30.212620+0000","last_became_active":"2024-04-17T07:11:54.457589+0000","last_became_peered":"2024-04-17T07:11:54.457589+0000","last_unstale":"2024-04-17T07:12:30.212620+0000","last_undegraded":"2024-04-17T07:12:30.212620+0000","last_fullsized":"2024-04-17T07:12:30.212620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:25.986539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714421+0000","last_change":"2024-04-17T07:11:55.465020+0000","last_active":"2024-04-17T07:12:28.714421+0000","last_peered":"2024-04-17T07:12:28.714421+0000","last_clean":"2024-04-17T07:12:28.714421+0000","last_became_active":"2024-04-17T07:11:55.464608+0000","last_became_peered":"2024-04-17T07:11:55.464608+0000","last_unstale":"2024-04-17T07:12:28.714421+0000","last_undegraded":"2024-04-17T07:12:28.714421+0000","last_fullsized":"2024-04-17T07:12:28.714421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122725+0000","last_change":"2024-04-17T07:11:59.519333+0000","last_active":"2024-04-17T07:12:29.122725+0000","last_peered":"2024-04-17T07:12:29.122725+0000","last_clean":"2024-04-17T07:12:29.122725+0000","last_became_active":"2024-04-17T07:11:59.518865+0000","last_became_peered":"2024-04-17T07:11:59.518865+0000","last_unstale":"2024-04-17T07:12:29.122725+0000","last_undegraded":"2024-04-17T07:12:29.122725+0000","last_fullsized":"2024-04-17T07:12:29.122725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:42.009010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:37.243160+0000","last_change":"2024-04-17T07:11:54.462283+0000","last_active":"2024-04-17T07:12:37.243160+0000","last_peered":"2024-04-17T07:12:37.243160+0000","last_clean":"2024-04-17T07:12:37.243160+0000","last_became_active":"2024-04-17T07:11:54.461818+0000","last_became_peered":"2024-04-17T07:11:54.461818+0000","last_unstale":"2024-04-17T07:12:37.243160+0000","last_undegraded":"2024-04-17T07:12:37.243160+0000","last_fullsized":"2024-04-17T07:12:37.243160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.765058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726968+0000","last_change":"2024-04-17T07:11:55.486491+0000","last_active":"2024-04-17T07:12:28.726968+0000","last_peered":"2024-04-17T07:12:28.726968+0000","last_clean":"2024-04-17T07:12:28.726968+0000","last_became_active":"2024-04-17T07:11:55.486121+0000","last_became_peered":"2024-04-17T07:11:55.486121+0000","last_unstale":"2024-04-17T07:12:28.726968+0000","last_undegraded":"2024-04-17T07:12:28.726968+0000","last_fullsized":"2024-04-17T07:12:28.726968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:12.998321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713871+0000","last_change":"2024-04-17T07:11:59.511555+0000","last_active":"2024-04-17T07:12:28.713871+0000","last_peered":"2024-04-17T07:12:28.713871+0000","last_clean":"2024-04-17T07:12:28.713871+0000","last_became_active":"2024-04-17T07:11:59.511147+0000","last_became_peered":"2024-04-17T07:11:59.511147+0000","last_unstale":"2024-04-17T07:12:28.713871+0000","last_undegraded":"2024-04-17T07:12:28.713871+0000","last_fullsized":"2024-04-17T07:12:28.713871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:41.854815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.210000+0000","last_change":"2024-04-17T07:12:28.734476+0000","last_active":"2024-04-17T07:12:30.210000+0000","last_peered":"2024-04-17T07:12:30.210000+0000","last_clean":"2024-04-17T07:12:30.210000+0000","last_became_active":"2024-04-17T07:12:28.728187+0000","last_became_peered":"2024-04-17T07:12:28.728187+0000","last_unstale":"2024-04-17T07:12:30.210000+0000","last_undegraded":"2024-04-17T07:12:30.210000+0000","last_fullsized":"2024-04-17T07:12:30.210000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:45.749607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736562+0000","last_change":"2024-04-17T07:11:55.480589+0000","last_active":"2024-04-17T07:12:28.736562+0000","last_peered":"2024-04-17T07:12:28.736562+0000","last_clean":"2024-04-17T07:12:28.736562+0000","last_became_active":"2024-04-17T07:11:55.480224+0000","last_became_peered":"2024-04-17T07:11:55.480224+0000","last_unstale":"2024-04-17T07:12:28.736562+0000","last_undegraded":"2024-04-17T07:12:28.736562+0000","last_fullsized":"2024-04-17T07:12:28.736562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:05.892442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713815+0000","last_change":"2024-04-17T07:11:59.522021+0000","last_active":"2024-04-17T07:12:28.713815+0000","last_peered":"2024-04-17T07:12:28.713815+0000","last_clean":"2024-04-17T07:12:28.713815+0000","last_became_active":"2024-04-17T07:11:59.521493+0000","last_became_peered":"2024-04-17T07:11:59.521493+0000","last_unstale":"2024-04-17T07:12:28.713815+0000","last_undegraded":"2024-04-17T07:12:28.713815+0000","last_fullsized":"2024-04-17T07:12:28.713815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:20.386216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.195512+0000","last_change":"2024-04-17T07:11:54.468225+0000","last_active":"2024-04-17T07:12:29.195512+0000","last_peered":"2024-04-17T07:12:29.195512+0000","last_clean":"2024-04-17T07:12:29.195512+0000","last_became_active":"2024-04-17T07:11:54.467046+0000","last_became_peered":"2024-04-17T07:11:54.467046+0000","last_unstale":"2024-04-17T07:12:29.195512+0000","last_undegraded":"2024-04-17T07:12:29.195512+0000","last_fullsized":"2024-04-17T07:12:29.195512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:17.725255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726428+0000","last_change":"2024-04-17T07:11:55.488980+0000","last_active":"2024-04-17T07:12:28.726428+0000","last_peered":"2024-04-17T07:12:28.726428+0000","last_clean":"2024-04-17T07:12:28.726428+0000","last_became_active":"2024-04-17T07:11:55.488226+0000","last_became_peered":"2024-04-17T07:11:55.488226+0000","last_unstale":"2024-04-17T07:12:28.726428+0000","last_undegraded":"2024-04-17T07:12:28.726428+0000","last_fullsized":"2024-04-17T07:12:28.726428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:00.135973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714388+0000","last_change":"2024-04-17T07:11:59.515134+0000","last_active":"2024-04-17T07:12:28.714388+0000","last_peered":"2024-04-17T07:12:28.714388+0000","last_clean":"2024-04-17T07:12:28.714388+0000","last_became_active":"2024-04-17T07:11:59.514560+0000","last_became_peered":"2024-04-17T07:11:59.514560+0000","last_unstale":"2024-04-17T07:12:28.714388+0000","last_undegraded":"2024-04-17T07:12:28.714388+0000","last_fullsized":"2024-04-17T07:12:28.714388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:34.669789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'27","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:37.248883+0000","last_change":"2024-04-17T07:11:54.461226+0000","last_active":"2024-04-17T07:12:37.248883+0000","last_peered":"2024-04-17T07:12:37.248883+0000","last_clean":"2024-04-17T07:12:37.248883+0000","last_became_active":"2024-04-17T07:11:54.460573+0000","last_became_peered":"2024-04-17T07:11:54.460573+0000","last_unstale":"2024-04-17T07:12:37.248883+0000","last_undegraded":"2024-04-17T07:12:37.248883+0000","last_fullsized":"2024-04-17T07:12:37.248883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46521,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":54,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.734828+0000","last_change":"2024-04-17T07:11:55.483925+0000","last_active":"2024-04-17T07:12:28.734828+0000","last_peered":"2024-04-17T07:12:28.734828+0000","last_clean":"2024-04-17T07:12:28.734828+0000","last_became_active":"2024-04-17T07:11:55.483584+0000","last_became_peered":"2024-04-17T07:11:55.483584+0000","last_unstale":"2024-04-17T07:12:28.734828+0000","last_undegraded":"2024-04-17T07:12:28.734828+0000","last_fullsized":"2024-04-17T07:12:28.734828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:12.328876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.737367+0000","last_change":"2024-04-17T07:11:59.515597+0000","last_active":"2024-04-17T07:12:28.737367+0000","last_peered":"2024-04-17T07:12:28.737367+0000","last_clean":"2024-04-17T07:12:28.737367+0000","last_became_active":"2024-04-17T07:11:59.514413+0000","last_became_peered":"2024-04-17T07:11:59.514413+0000","last_unstale":"2024-04-17T07:12:28.737367+0000","last_undegraded":"2024-04-17T07:12:28.737367+0000","last_fullsized":"2024-04-17T07:12:28.737367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:45.824127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.216591+0000","last_change":"2024-04-17T07:11:54.449984+0000","last_active":"2024-04-17T07:12:31.216591+0000","last_peered":"2024-04-17T07:12:31.216591+0000","last_clean":"2024-04-17T07:12:31.216591+0000","last_became_active":"2024-04-17T07:11:54.449628+0000","last_became_peered":"2024-04-17T07:11:54.449628+0000","last_unstale":"2024-04-17T07:12:31.216591+0000","last_undegraded":"2024-04-17T07:12:31.216591+0000","last_fullsized":"2024-04-17T07:12:31.216591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:26:03.281684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719923+0000","last_change":"2024-04-17T07:11:55.476912+0000","last_active":"2024-04-17T07:12:28.719923+0000","last_peered":"2024-04-17T07:12:28.719923+0000","last_clean":"2024-04-17T07:12:28.719923+0000","last_became_active":"2024-04-17T07:11:55.471576+0000","last_became_peered":"2024-04-17T07:11:55.471576+0000","last_unstale":"2024-04-17T07:12:28.719923+0000","last_undegraded":"2024-04-17T07:12:28.719923+0000","last_fullsized":"2024-04-17T07:12:28.719923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:35.439271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726532+0000","last_change":"2024-04-17T07:11:59.515897+0000","last_active":"2024-04-17T07:12:28.726532+0000","last_peered":"2024-04-17T07:12:28.726532+0000","last_clean":"2024-04-17T07:12:28.726532+0000","last_became_active":"2024-04-17T07:11:59.515349+0000","last_became_peered":"2024-04-17T07:11:59.515349+0000","last_unstale":"2024-04-17T07:12:28.726532+0000","last_undegraded":"2024-04-17T07:12:28.726532+0000","last_fullsized":"2024-04-17T07:12:28.726532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:00.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214859+0000","last_change":"2024-04-17T07:11:54.462369+0000","last_active":"2024-04-17T07:12:30.214859+0000","last_peered":"2024-04-17T07:12:30.214859+0000","last_clean":"2024-04-17T07:12:30.214859+0000","last_became_active":"2024-04-17T07:11:54.460357+0000","last_became_peered":"2024-04-17T07:11:54.460357+0000","last_unstale":"2024-04-17T07:12:30.214859+0000","last_undegraded":"2024-04-17T07:12:30.214859+0000","last_fullsized":"2024-04-17T07:12:30.214859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:38.029980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720650+0000","last_change":"2024-04-17T07:11:55.475874+0000","last_active":"2024-04-17T07:12:28.720650+0000","last_peered":"2024-04-17T07:12:28.720650+0000","last_clean":"2024-04-17T07:12:28.720650+0000","last_became_active":"2024-04-17T07:11:55.475550+0000","last_became_peered":"2024-04-17T07:11:55.475550+0000","last_unstale":"2024-04-17T07:12:28.720650+0000","last_undegraded":"2024-04-17T07:12:28.720650+0000","last_fullsized":"2024-04-17T07:12:28.720650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:40.096573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714893+0000","last_change":"2024-04-17T07:11:59.514661+0000","last_active":"2024-04-17T07:12:28.714893+0000","last_peered":"2024-04-17T07:12:28.714893+0000","last_clean":"2024-04-17T07:12:28.714893+0000","last_became_active":"2024-04-17T07:11:59.513030+0000","last_became_peered":"2024-04-17T07:11:59.513030+0000","last_unstale":"2024-04-17T07:12:28.714893+0000","last_undegraded":"2024-04-17T07:12:28.714893+0000","last_fullsized":"2024-04-17T07:12:28.714893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:35.696961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.241199+0000","last_change":"2024-04-17T07:11:54.455279+0000","last_active":"2024-04-17T07:12:32.241199+0000","last_peered":"2024-04-17T07:12:32.241199+0000","last_clean":"2024-04-17T07:12:32.241199+0000","last_became_active":"2024-04-17T07:11:54.453620+0000","last_became_peered":"2024-04-17T07:11:54.453620+0000","last_unstale":"2024-04-17T07:12:32.241199+0000","last_undegraded":"2024-04-17T07:12:32.241199+0000","last_fullsized":"2024-04-17T07:12:32.241199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:15.227410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719686+0000","last_change":"2024-04-17T07:11:55.469579+0000","last_active":"2024-04-17T07:12:28.719686+0000","last_peered":"2024-04-17T07:12:28.719686+0000","last_clean":"2024-04-17T07:12:28.719686+0000","last_became_active":"2024-04-17T07:11:55.468389+0000","last_became_peered":"2024-04-17T07:11:55.468389+0000","last_unstale":"2024-04-17T07:12:28.719686+0000","last_undegraded":"2024-04-17T07:12:28.719686+0000","last_fullsized":"2024-04-17T07:12:28.719686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:01.274677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720385+0000","last_change":"2024-04-17T07:11:59.512110+0000","last_active":"2024-04-17T07:12:28.720385+0000","last_peered":"2024-04-17T07:12:28.720385+0000","last_clean":"2024-04-17T07:12:28.720385+0000","last_became_active":"2024-04-17T07:11:59.510573+0000","last_became_peered":"2024-04-17T07:11:59.510573+0000","last_unstale":"2024-04-17T07:12:28.720385+0000","last_undegraded":"2024-04-17T07:12:28.720385+0000","last_fullsized":"2024-04-17T07:12:28.720385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:32.515687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.209947+0000","last_change":"2024-04-17T07:12:28.767517+0000","last_active":"2024-04-17T07:12:30.209947+0000","last_peered":"2024-04-17T07:12:30.209947+0000","last_clean":"2024-04-17T07:12:30.209947+0000","last_became_active":"2024-04-17T07:12:28.738908+0000","last_became_peered":"2024-04-17T07:12:28.738908+0000","last_unstale":"2024-04-17T07:12:30.209947+0000","last_undegraded":"2024-04-17T07:12:30.209947+0000","last_fullsized":"2024-04-17T07:12:30.209947+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:47:30.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.731088+0000","last_change":"2024-04-17T07:11:55.473540+0000","last_active":"2024-04-17T07:12:28.731088+0000","last_peered":"2024-04-17T07:12:28.731088+0000","last_clean":"2024-04-17T07:12:28.731088+0000","last_became_active":"2024-04-17T07:11:55.472436+0000","last_became_peered":"2024-04-17T07:11:55.472436+0000","last_unstale":"2024-04-17T07:12:28.731088+0000","last_undegraded":"2024-04-17T07:12:28.731088+0000","last_fullsized":"2024-04-17T07:12:28.731088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:54.317052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730908+0000","last_change":"2024-04-17T07:11:59.521305+0000","last_active":"2024-04-17T07:12:28.730908+0000","last_peered":"2024-04-17T07:12:28.730908+0000","last_clean":"2024-04-17T07:12:28.730908+0000","last_became_active":"2024-04-17T07:11:59.516860+0000","last_became_peered":"2024-04-17T07:11:59.516860+0000","last_unstale":"2024-04-17T07:12:28.730908+0000","last_undegraded":"2024-04-17T07:12:28.730908+0000","last_fullsized":"2024-04-17T07:12:28.730908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:23.403218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.723396+0000","last_change":"2024-04-17T07:11:54.463378+0000","last_active":"2024-04-17T07:12:29.723396+0000","last_peered":"2024-04-17T07:12:29.723396+0000","last_clean":"2024-04-17T07:12:29.723396+0000","last_became_active":"2024-04-17T07:11:54.462120+0000","last_became_peered":"2024-04-17T07:11:54.462120+0000","last_unstale":"2024-04-17T07:12:29.723396+0000","last_undegraded":"2024-04-17T07:12:29.723396+0000","last_fullsized":"2024-04-17T07:12:29.723396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:42.109928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727537+0000","last_change":"2024-04-17T07:11:55.478604+0000","last_active":"2024-04-17T07:12:28.727537+0000","last_peered":"2024-04-17T07:12:28.727537+0000","last_clean":"2024-04-17T07:12:28.727537+0000","last_became_active":"2024-04-17T07:11:55.478252+0000","last_became_peered":"2024-04-17T07:11:55.478252+0000","last_unstale":"2024-04-17T07:12:28.727537+0000","last_undegraded":"2024-04-17T07:12:28.727537+0000","last_fullsized":"2024-04-17T07:12:28.727537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:08.365896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727344+0000","last_change":"2024-04-17T07:11:59.518619+0000","last_active":"2024-04-17T07:12:28.727344+0000","last_peered":"2024-04-17T07:12:28.727344+0000","last_clean":"2024-04-17T07:12:28.727344+0000","last_became_active":"2024-04-17T07:11:59.518214+0000","last_became_peered":"2024-04-17T07:11:59.518214+0000","last_unstale":"2024-04-17T07:12:28.727344+0000","last_undegraded":"2024-04-17T07:12:28.727344+0000","last_fullsized":"2024-04-17T07:12:28.727344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:46.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715522+0000","last_change":"2024-04-17T07:11:54.453038+0000","last_active":"2024-04-17T07:12:28.715522+0000","last_peered":"2024-04-17T07:12:28.715522+0000","last_clean":"2024-04-17T07:12:28.715522+0000","last_became_active":"2024-04-17T07:11:54.452619+0000","last_became_peered":"2024-04-17T07:11:54.452619+0000","last_unstale":"2024-04-17T07:12:28.715522+0000","last_undegraded":"2024-04-17T07:12:28.715522+0000","last_fullsized":"2024-04-17T07:12:28.715522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:25.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.724948+0000","last_change":"2024-04-17T07:11:55.465990+0000","last_active":"2024-04-17T07:12:28.724948+0000","last_peered":"2024-04-17T07:12:28.724948+0000","last_clean":"2024-04-17T07:12:28.724948+0000","last_became_active":"2024-04-17T07:11:55.465523+0000","last_became_peered":"2024-04-17T07:11:55.465523+0000","last_unstale":"2024-04-17T07:12:28.724948+0000","last_undegraded":"2024-04-17T07:12:28.724948+0000","last_fullsized":"2024-04-17T07:12:28.724948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:40.366027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721070+0000","last_change":"2024-04-17T07:11:59.512119+0000","last_active":"2024-04-17T07:12:28.721070+0000","last_peered":"2024-04-17T07:12:28.721070+0000","last_clean":"2024-04-17T07:12:28.721070+0000","last_became_active":"2024-04-17T07:11:59.511460+0000","last_became_peered":"2024-04-17T07:11:59.511460+0000","last_unstale":"2024-04-17T07:12:28.721070+0000","last_undegraded":"2024-04-17T07:12:28.721070+0000","last_fullsized":"2024-04-17T07:12:28.721070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:50.888203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:35.248132+0000","last_change":"2024-04-17T07:11:54.469346+0000","last_active":"2024-04-17T07:12:35.248132+0000","last_peered":"2024-04-17T07:12:35.248132+0000","last_clean":"2024-04-17T07:12:35.248132+0000","last_became_active":"2024-04-17T07:11:54.468947+0000","last_became_peered":"2024-04-17T07:11:54.468947+0000","last_unstale":"2024-04-17T07:12:35.248132+0000","last_undegraded":"2024-04-17T07:12:35.248132+0000","last_fullsized":"2024-04-17T07:12:35.248132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:04.854625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.724510+0000","last_change":"2024-04-17T07:11:55.471235+0000","last_active":"2024-04-17T07:12:28.724510+0000","last_peered":"2024-04-17T07:12:28.724510+0000","last_clean":"2024-04-17T07:12:28.724510+0000","last_became_active":"2024-04-17T07:11:55.470857+0000","last_became_peered":"2024-04-17T07:11:55.470857+0000","last_unstale":"2024-04-17T07:12:28.724510+0000","last_undegraded":"2024-04-17T07:12:28.724510+0000","last_fullsized":"2024-04-17T07:12:28.724510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:21.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720514+0000","last_change":"2024-04-17T07:11:59.524519+0000","last_active":"2024-04-17T07:12:28.720514+0000","last_peered":"2024-04-17T07:12:28.720514+0000","last_clean":"2024-04-17T07:12:28.720514+0000","last_became_active":"2024-04-17T07:11:59.519270+0000","last_became_peered":"2024-04-17T07:11:59.519270+0000","last_unstale":"2024-04-17T07:12:28.720514+0000","last_undegraded":"2024-04-17T07:12:28.720514+0000","last_fullsized":"2024-04-17T07:12:28.720514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:41.080249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.734024+0000","last_change":"2024-04-17T07:11:55.473922+0000","last_active":"2024-04-17T07:12:28.734024+0000","last_peered":"2024-04-17T07:12:28.734024+0000","last_clean":"2024-04-17T07:12:28.734024+0000","last_became_active":"2024-04-17T07:11:55.468594+0000","last_became_peered":"2024-04-17T07:11:55.468594+0000","last_unstale":"2024-04-17T07:12:28.734024+0000","last_undegraded":"2024-04-17T07:12:28.734024+0000","last_fullsized":"2024-04-17T07:12:28.734024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:21.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:36.251257+0000","last_change":"2024-04-17T07:11:54.473066+0000","last_active":"2024-04-17T07:12:36.251257+0000","last_peered":"2024-04-17T07:12:36.251257+0000","last_clean":"2024-04-17T07:12:36.251257+0000","last_became_active":"2024-04-17T07:11:54.472708+0000","last_became_peered":"2024-04-17T07:11:54.472708+0000","last_unstale":"2024-04-17T07:12:36.251257+0000","last_undegraded":"2024-04-17T07:12:36.251257+0000","last_fullsized":"2024-04-17T07:12:36.251257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:27.614819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720813+0000","last_change":"2024-04-17T07:11:59.516939+0000","last_active":"2024-04-17T07:12:28.720813+0000","last_peered":"2024-04-17T07:12:28.720813+0000","last_clean":"2024-04-17T07:12:28.720813+0000","last_became_active":"2024-04-17T07:11:59.516333+0000","last_became_peered":"2024-04-17T07:11:59.516333+0000","last_unstale":"2024-04-17T07:12:28.720813+0000","last_undegraded":"2024-04-17T07:12:28.720813+0000","last_fullsized":"2024-04-17T07:12:28.720813+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:15.648831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122635+0000","last_change":"2024-04-17T07:11:55.481006+0000","last_active":"2024-04-17T07:12:29.122635+0000","last_peered":"2024-04-17T07:12:29.122635+0000","last_clean":"2024-04-17T07:12:29.122635+0000","last_became_active":"2024-04-17T07:11:55.479751+0000","last_became_peered":"2024-04-17T07:11:55.479751+0000","last_unstale":"2024-04-17T07:12:29.122635+0000","last_undegraded":"2024-04-17T07:12:29.122635+0000","last_fullsized":"2024-04-17T07:12:29.122635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:13.004650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:31.218667+0000","last_change":"2024-04-17T07:11:54.463106+0000","last_active":"2024-04-17T07:12:31.218667+0000","last_peered":"2024-04-17T07:12:31.218667+0000","last_clean":"2024-04-17T07:12:31.218667+0000","last_became_active":"2024-04-17T07:11:54.462691+0000","last_became_peered":"2024-04-17T07:11:54.462691+0000","last_unstale":"2024-04-17T07:12:31.218667+0000","last_undegraded":"2024-04-17T07:12:31.218667+0000","last_fullsized":"2024-04-17T07:12:31.218667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:34.302019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719632+0000","last_change":"2024-04-17T07:10:14.685883+0000","last_active":"2024-04-17T07:12:28.719632+0000","last_peered":"2024-04-17T07:12:28.719632+0000","last_clean":"2024-04-17T07:12:28.719632+0000","last_became_active":"2024-04-17T07:10:14.684551+0000","last_became_peered":"2024-04-17T07:10:14.684551+0000","last_unstale":"2024-04-17T07:12:28.719632+0000","last_undegraded":"2024-04-17T07:12:28.719632+0000","last_fullsized":"2024-04-17T07:12:28.719632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_clean_scrub_stamp":"2024-04-17T07:05:37.112317+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:04.265147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736804+0000","last_change":"2024-04-17T07:12:28.731807+0000","last_active":"2024-04-17T07:12:28.736804+0000","last_peered":"2024-04-17T07:12:28.736804+0000","last_clean":"2024-04-17T07:12:28.736804+0000","last_became_active":"2024-04-17T07:12:28.730656+0000","last_became_peered":"2024-04-17T07:12:28.730656+0000","last_unstale":"2024-04-17T07:12:28.736804+0000","last_undegraded":"2024-04-17T07:12:28.736804+0000","last_fullsized":"2024-04-17T07:12:28.736804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:58.023333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727980+0000","last_change":"2024-04-17T07:12:28.727980+0000","last_active":"2024-04-17T07:12:28.727980+0000","last_peered":"2024-04-17T07:12:28.727980+0000","last_clean":"2024-04-17T07:12:28.727980+0000","last_became_active":"2024-04-17T07:12:28.727060+0000","last_became_peered":"2024-04-17T07:12:28.727060+0000","last_unstale":"2024-04-17T07:12:28.727980+0000","last_undegraded":"2024-04-17T07:12:28.727980+0000","last_fullsized":"2024-04-17T07:12:28.727980+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:33.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.719921+0000","last_change":"2024-04-17T07:11:54.463180+0000","last_active":"2024-04-17T07:12:28.719921+0000","last_peered":"2024-04-17T07:12:28.719921+0000","last_clean":"2024-04-17T07:12:28.719921+0000","last_became_active":"2024-04-17T07:11:54.462793+0000","last_became_peered":"2024-04-17T07:11:54.462793+0000","last_unstale":"2024-04-17T07:12:28.719921+0000","last_undegraded":"2024-04-17T07:12:28.719921+0000","last_fullsized":"2024-04-17T07:12:28.719921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:19.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725190+0000","last_change":"2024-04-17T07:11:59.518355+0000","last_active":"2024-04-17T07:12:28.725190+0000","last_peered":"2024-04-17T07:12:28.725190+0000","last_clean":"2024-04-17T07:12:28.725190+0000","last_became_active":"2024-04-17T07:11:59.517619+0000","last_became_peered":"2024-04-17T07:11:59.517619+0000","last_unstale":"2024-04-17T07:12:28.725190+0000","last_undegraded":"2024-04-17T07:12:28.725190+0000","last_fullsized":"2024-04-17T07:12:28.725190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:36.826021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.210952+0000","last_change":"2024-04-17T07:11:54.465285+0000","last_active":"2024-04-17T07:12:30.210952+0000","last_peered":"2024-04-17T07:12:30.210952+0000","last_clean":"2024-04-17T07:12:30.210952+0000","last_became_active":"2024-04-17T07:11:54.464818+0000","last_became_peered":"2024-04-17T07:11:54.464818+0000","last_unstale":"2024-04-17T07:12:30.210952+0000","last_undegraded":"2024-04-17T07:12:30.210952+0000","last_fullsized":"2024-04-17T07:12:30.210952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721576+0000","last_change":"2024-04-17T07:11:55.470405+0000","last_active":"2024-04-17T07:12:28.721576+0000","last_peered":"2024-04-17T07:12:28.721576+0000","last_clean":"2024-04-17T07:12:28.721576+0000","last_became_active":"2024-04-17T07:11:55.470058+0000","last_became_peered":"2024-04-17T07:11:55.470058+0000","last_unstale":"2024-04-17T07:12:28.721576+0000","last_undegraded":"2024-04-17T07:12:28.721576+0000","last_fullsized":"2024-04-17T07:12:28.721576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:17.416775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122022+0000","last_change":"2024-04-17T07:11:59.522040+0000","last_active":"2024-04-17T07:12:29.122022+0000","last_peered":"2024-04-17T07:12:29.122022+0000","last_clean":"2024-04-17T07:12:29.122022+0000","last_became_active":"2024-04-17T07:11:59.519771+0000","last_became_peered":"2024-04-17T07:11:59.519771+0000","last_unstale":"2024-04-17T07:12:29.122022+0000","last_undegraded":"2024-04-17T07:12:29.122022+0000","last_fullsized":"2024-04-17T07:12:29.122022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:58.203737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'11","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:32.490214+0000","last_change":"2024-04-17T07:11:54.457954+0000","last_active":"2024-04-17T07:12:32.490214+0000","last_peered":"2024-04-17T07:12:32.490214+0000","last_clean":"2024-04-17T07:12:32.490214+0000","last_became_active":"2024-04-17T07:11:54.457560+0000","last_became_peered":"2024-04-17T07:11:54.457560+0000","last_unstale":"2024-04-17T07:12:32.490214+0000","last_undegraded":"2024-04-17T07:12:32.490214+0000","last_fullsized":"2024-04-17T07:12:32.490214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:31.919730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123098+0000","last_change":"2024-04-17T07:11:55.480475+0000","last_active":"2024-04-17T07:12:29.123098+0000","last_peered":"2024-04-17T07:12:29.123098+0000","last_clean":"2024-04-17T07:12:29.123098+0000","last_became_active":"2024-04-17T07:11:55.480060+0000","last_became_peered":"2024-04-17T07:11:55.480060+0000","last_unstale":"2024-04-17T07:12:29.123098+0000","last_undegraded":"2024-04-17T07:12:29.123098+0000","last_fullsized":"2024-04-17T07:12:29.123098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:30:23.600244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714322+0000","last_change":"2024-04-17T07:11:59.525422+0000","last_active":"2024-04-17T07:12:28.714322+0000","last_peered":"2024-04-17T07:12:28.714322+0000","last_clean":"2024-04-17T07:12:28.714322+0000","last_became_active":"2024-04-17T07:11:59.523649+0000","last_became_peered":"2024-04-17T07:11:59.523649+0000","last_unstale":"2024-04-17T07:12:28.714322+0000","last_undegraded":"2024-04-17T07:12:28.714322+0000","last_fullsized":"2024-04-17T07:12:28.714322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:16.653752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214020+0000","last_change":"2024-04-17T07:11:54.465824+0000","last_active":"2024-04-17T07:12:30.214020+0000","last_peered":"2024-04-17T07:12:30.214020+0000","last_clean":"2024-04-17T07:12:30.214020+0000","last_became_active":"2024-04-17T07:11:54.465348+0000","last_became_peered":"2024-04-17T07:11:54.465348+0000","last_unstale":"2024-04-17T07:12:30.214020+0000","last_undegraded":"2024-04-17T07:12:30.214020+0000","last_fullsized":"2024-04-17T07:12:30.214020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:17.729903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.736284+0000","last_change":"2024-04-17T07:11:55.481666+0000","last_active":"2024-04-17T07:12:28.736284+0000","last_peered":"2024-04-17T07:12:28.736284+0000","last_clean":"2024-04-17T07:12:28.736284+0000","last_became_active":"2024-04-17T07:11:55.480332+0000","last_became_peered":"2024-04-17T07:11:55.480332+0000","last_unstale":"2024-04-17T07:12:28.736284+0000","last_undegraded":"2024-04-17T07:12:28.736284+0000","last_fullsized":"2024-04-17T07:12:28.736284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:19.825407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714433+0000","last_change":"2024-04-17T07:11:59.527850+0000","last_active":"2024-04-17T07:12:28.714433+0000","last_peered":"2024-04-17T07:12:28.714433+0000","last_clean":"2024-04-17T07:12:28.714433+0000","last_became_active":"2024-04-17T07:11:59.527306+0000","last_became_peered":"2024-04-17T07:11:59.527306+0000","last_unstale":"2024-04-17T07:12:28.714433+0000","last_undegraded":"2024-04-17T07:12:28.714433+0000","last_fullsized":"2024-04-17T07:12:28.714433+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:35.906222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725682+0000","last_change":"2024-04-17T07:11:54.455862+0000","last_active":"2024-04-17T07:12:28.725682+0000","last_peered":"2024-04-17T07:12:28.725682+0000","last_clean":"2024-04-17T07:12:28.725682+0000","last_became_active":"2024-04-17T07:11:54.453904+0000","last_became_peered":"2024-04-17T07:11:54.453904+0000","last_unstale":"2024-04-17T07:12:28.725682+0000","last_undegraded":"2024-04-17T07:12:28.725682+0000","last_fullsized":"2024-04-17T07:12:28.725682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:01.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725079+0000","last_change":"2024-04-17T07:11:55.470315+0000","last_active":"2024-04-17T07:12:28.725079+0000","last_peered":"2024-04-17T07:12:28.725079+0000","last_clean":"2024-04-17T07:12:28.725079+0000","last_became_active":"2024-04-17T07:11:55.469829+0000","last_became_peered":"2024-04-17T07:11:55.469829+0000","last_unstale":"2024-04-17T07:12:28.725079+0000","last_undegraded":"2024-04-17T07:12:28.725079+0000","last_fullsized":"2024-04-17T07:12:28.725079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:48.197546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.126045+0000","last_change":"2024-04-17T07:12:29.125447+0000","last_active":"2024-04-17T07:12:29.126045+0000","last_peered":"2024-04-17T07:12:29.126045+0000","last_clean":"2024-04-17T07:12:29.126045+0000","last_became_active":"2024-04-17T07:12:29.124551+0000","last_became_peered":"2024-04-17T07:12:29.124551+0000","last_unstale":"2024-04-17T07:12:29.126045+0000","last_undegraded":"2024-04-17T07:12:29.126045+0000","last_fullsized":"2024-04-17T07:12:29.126045+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:09.489755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.216091+0000","last_change":"2024-04-17T07:11:54.453117+0000","last_active":"2024-04-17T07:12:30.216091+0000","last_peered":"2024-04-17T07:12:30.216091+0000","last_clean":"2024-04-17T07:12:30.216091+0000","last_became_active":"2024-04-17T07:11:54.452632+0000","last_became_peered":"2024-04-17T07:11:54.452632+0000","last_unstale":"2024-04-17T07:12:30.216091+0000","last_undegraded":"2024-04-17T07:12:30.216091+0000","last_fullsized":"2024-04-17T07:12:30.216091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:09.366149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726470+0000","last_change":"2024-04-17T07:11:55.469991+0000","last_active":"2024-04-17T07:12:28.726470+0000","last_peered":"2024-04-17T07:12:28.726470+0000","last_clean":"2024-04-17T07:12:28.726470+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:12:28.726470+0000","last_undegraded":"2024-04-17T07:12:28.726470+0000","last_fullsized":"2024-04-17T07:12:28.726470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:02.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.713959+0000","last_change":"2024-04-17T07:11:59.519817+0000","last_active":"2024-04-17T07:12:28.713959+0000","last_peered":"2024-04-17T07:12:28.713959+0000","last_clean":"2024-04-17T07:12:28.713959+0000","last_became_active":"2024-04-17T07:11:59.519108+0000","last_became_peered":"2024-04-17T07:11:59.519108+0000","last_unstale":"2024-04-17T07:12:28.713959+0000","last_undegraded":"2024-04-17T07:12:28.713959+0000","last_fullsized":"2024-04-17T07:12:28.713959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:58.378837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.198814+0000","last_change":"2024-04-17T07:11:54.463618+0000","last_active":"2024-04-17T07:12:29.198814+0000","last_peered":"2024-04-17T07:12:29.198814+0000","last_clean":"2024-04-17T07:12:29.198814+0000","last_became_active":"2024-04-17T07:11:54.463257+0000","last_became_peered":"2024-04-17T07:11:54.463257+0000","last_unstale":"2024-04-17T07:12:29.198814+0000","last_undegraded":"2024-04-17T07:12:29.198814+0000","last_fullsized":"2024-04-17T07:12:29.198814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:20.135338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122992+0000","last_change":"2024-04-17T07:11:55.487471+0000","last_active":"2024-04-17T07:12:29.122992+0000","last_peered":"2024-04-17T07:12:29.122992+0000","last_clean":"2024-04-17T07:12:29.122992+0000","last_became_active":"2024-04-17T07:11:55.487024+0000","last_became_peered":"2024-04-17T07:11:55.487024+0000","last_unstale":"2024-04-17T07:12:29.122992+0000","last_undegraded":"2024-04-17T07:12:29.122992+0000","last_fullsized":"2024-04-17T07:12:29.122992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:36.851210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714472+0000","last_change":"2024-04-17T07:11:59.527017+0000","last_active":"2024-04-17T07:12:28.714472+0000","last_peered":"2024-04-17T07:12:28.714472+0000","last_clean":"2024-04-17T07:12:28.714472+0000","last_became_active":"2024-04-17T07:11:59.526541+0000","last_became_peered":"2024-04-17T07:11:59.526541+0000","last_unstale":"2024-04-17T07:12:28.714472+0000","last_undegraded":"2024-04-17T07:12:28.714472+0000","last_fullsized":"2024-04-17T07:12:28.714472+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:54.411647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.728931+0000","last_change":"2024-04-17T07:11:54.465671+0000","last_active":"2024-04-17T07:12:28.728931+0000","last_peered":"2024-04-17T07:12:28.728931+0000","last_clean":"2024-04-17T07:12:28.728931+0000","last_became_active":"2024-04-17T07:11:54.464902+0000","last_became_peered":"2024-04-17T07:11:54.464902+0000","last_unstale":"2024-04-17T07:12:28.728931+0000","last_undegraded":"2024-04-17T07:12:28.728931+0000","last_fullsized":"2024-04-17T07:12:28.728931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:13:38.232965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720011+0000","last_change":"2024-04-17T07:11:55.484259+0000","last_active":"2024-04-17T07:12:28.720011+0000","last_peered":"2024-04-17T07:12:28.720011+0000","last_clean":"2024-04-17T07:12:28.720011+0000","last_became_active":"2024-04-17T07:11:55.483911+0000","last_became_peered":"2024-04-17T07:11:55.483911+0000","last_unstale":"2024-04-17T07:12:28.720011+0000","last_undegraded":"2024-04-17T07:12:28.720011+0000","last_fullsized":"2024-04-17T07:12:28.720011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:54.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.725852+0000","last_change":"2024-04-17T07:11:59.525369+0000","last_active":"2024-04-17T07:12:28.725852+0000","last_peered":"2024-04-17T07:12:28.725852+0000","last_clean":"2024-04-17T07:12:28.725852+0000","last_became_active":"2024-04-17T07:11:59.524809+0000","last_became_peered":"2024-04-17T07:11:59.524809+0000","last_unstale":"2024-04-17T07:12:28.725852+0000","last_undegraded":"2024-04-17T07:12:28.725852+0000","last_fullsized":"2024-04-17T07:12:28.725852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:20.511001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:34.242770+0000","last_change":"2024-04-17T07:11:54.459079+0000","last_active":"2024-04-17T07:12:34.242770+0000","last_peered":"2024-04-17T07:12:34.242770+0000","last_clean":"2024-04-17T07:12:34.242770+0000","last_became_active":"2024-04-17T07:11:54.457008+0000","last_became_peered":"2024-04-17T07:11:54.457008+0000","last_unstale":"2024-04-17T07:12:34.242770+0000","last_undegraded":"2024-04-17T07:12:34.242770+0000","last_fullsized":"2024-04-17T07:12:34.242770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:36.084512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721123+0000","last_change":"2024-04-17T07:11:55.484453+0000","last_active":"2024-04-17T07:12:28.721123+0000","last_peered":"2024-04-17T07:12:28.721123+0000","last_clean":"2024-04-17T07:12:28.721123+0000","last_became_active":"2024-04-17T07:11:55.483152+0000","last_became_peered":"2024-04-17T07:11:55.483152+0000","last_unstale":"2024-04-17T07:12:28.721123+0000","last_undegraded":"2024-04-17T07:12:28.721123+0000","last_fullsized":"2024-04-17T07:12:28.721123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:32.224868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735401+0000","last_change":"2024-04-17T07:11:59.521487+0000","last_active":"2024-04-17T07:12:28.735401+0000","last_peered":"2024-04-17T07:12:28.735401+0000","last_clean":"2024-04-17T07:12:28.735401+0000","last_became_active":"2024-04-17T07:11:59.521060+0000","last_became_peered":"2024-04-17T07:11:59.521060+0000","last_unstale":"2024-04-17T07:12:28.735401+0000","last_undegraded":"2024-04-17T07:12:28.735401+0000","last_fullsized":"2024-04-17T07:12:28.735401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:06.692434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.214903+0000","last_change":"2024-04-17T07:11:54.459368+0000","last_active":"2024-04-17T07:12:30.214903+0000","last_peered":"2024-04-17T07:12:30.214903+0000","last_clean":"2024-04-17T07:12:30.214903+0000","last_became_active":"2024-04-17T07:11:54.458973+0000","last_became_peered":"2024-04-17T07:11:54.458973+0000","last_unstale":"2024-04-17T07:12:30.214903+0000","last_undegraded":"2024-04-17T07:12:30.214903+0000","last_fullsized":"2024-04-17T07:12:30.214903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:22.034110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721530+0000","last_change":"2024-04-17T07:11:55.470648+0000","last_active":"2024-04-17T07:12:28.721530+0000","last_peered":"2024-04-17T07:12:28.721530+0000","last_clean":"2024-04-17T07:12:28.721530+0000","last_became_active":"2024-04-17T07:11:55.469666+0000","last_became_peered":"2024-04-17T07:11:55.469666+0000","last_unstale":"2024-04-17T07:12:28.721530+0000","last_undegraded":"2024-04-17T07:12:28.721530+0000","last_fullsized":"2024-04-17T07:12:28.721530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:59.208872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.730680+0000","last_change":"2024-04-17T07:11:59.524027+0000","last_active":"2024-04-17T07:12:28.730680+0000","last_peered":"2024-04-17T07:12:28.730680+0000","last_clean":"2024-04-17T07:12:28.730680+0000","last_became_active":"2024-04-17T07:11:59.522764+0000","last_became_peered":"2024-04-17T07:11:59.522764+0000","last_unstale":"2024-04-17T07:12:28.730680+0000","last_undegraded":"2024-04-17T07:12:28.730680+0000","last_fullsized":"2024-04-17T07:12:28.730680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:40.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:30.216540+0000","last_change":"2024-04-17T07:11:54.472884+0000","last_active":"2024-04-17T07:12:30.216540+0000","last_peered":"2024-04-17T07:12:30.216540+0000","last_clean":"2024-04-17T07:12:30.216540+0000","last_became_active":"2024-04-17T07:11:54.472305+0000","last_became_peered":"2024-04-17T07:11:54.472305+0000","last_unstale":"2024-04-17T07:12:30.216540+0000","last_undegraded":"2024-04-17T07:12:30.216540+0000","last_fullsized":"2024-04-17T07:12:30.216540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:21.031442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721232+0000","last_change":"2024-04-17T07:11:55.479982+0000","last_active":"2024-04-17T07:12:28.721232+0000","last_peered":"2024-04-17T07:12:28.721232+0000","last_clean":"2024-04-17T07:12:28.721232+0000","last_became_active":"2024-04-17T07:11:55.477594+0000","last_became_peered":"2024-04-17T07:11:55.477594+0000","last_unstale":"2024-04-17T07:12:28.721232+0000","last_undegraded":"2024-04-17T07:12:28.721232+0000","last_fullsized":"2024-04-17T07:12:28.721232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:23.507312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726804+0000","last_change":"2024-04-17T07:11:55.480941+0000","last_active":"2024-04-17T07:12:28.726804+0000","last_peered":"2024-04-17T07:12:28.726804+0000","last_clean":"2024-04-17T07:12:28.726804+0000","last_became_active":"2024-04-17T07:11:55.480180+0000","last_became_peered":"2024-04-17T07:11:55.480180+0000","last_unstale":"2024-04-17T07:12:28.726804+0000","last_undegraded":"2024-04-17T07:12:28.726804+0000","last_fullsized":"2024-04-17T07:12:28.726804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:16.321806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715350+0000","last_change":"2024-04-17T07:11:55.477663+0000","last_active":"2024-04-17T07:12:28.715350+0000","last_peered":"2024-04-17T07:12:28.715350+0000","last_clean":"2024-04-17T07:12:28.715350+0000","last_became_active":"2024-04-17T07:11:55.477304+0000","last_became_peered":"2024-04-17T07:11:55.477304+0000","last_unstale":"2024-04-17T07:12:28.715350+0000","last_undegraded":"2024-04-17T07:12:28.715350+0000","last_fullsized":"2024-04-17T07:12:28.715350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:00.796633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.714460+0000","last_change":"2024-04-17T07:11:55.475536+0000","last_active":"2024-04-17T07:12:28.714460+0000","last_peered":"2024-04-17T07:12:28.714460+0000","last_clean":"2024-04-17T07:12:28.714460+0000","last_became_active":"2024-04-17T07:11:55.475138+0000","last_became_peered":"2024-04-17T07:11:55.475138+0000","last_unstale":"2024-04-17T07:12:28.714460+0000","last_undegraded":"2024-04-17T07:12:28.714460+0000","last_fullsized":"2024-04-17T07:12:28.714460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:47.849076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.735949+0000","last_change":"2024-04-17T07:11:55.483742+0000","last_active":"2024-04-17T07:12:28.735949+0000","last_peered":"2024-04-17T07:12:28.735949+0000","last_clean":"2024-04-17T07:12:28.735949+0000","last_became_active":"2024-04-17T07:11:55.483044+0000","last_became_peered":"2024-04-17T07:11:55.483044+0000","last_unstale":"2024-04-17T07:12:28.735949+0000","last_undegraded":"2024-04-17T07:12:28.735949+0000","last_fullsized":"2024-04-17T07:12:28.735949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:18.972137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.721440+0000","last_change":"2024-04-17T07:11:55.471266+0000","last_active":"2024-04-17T07:12:28.721440+0000","last_peered":"2024-04-17T07:12:28.721440+0000","last_clean":"2024-04-17T07:12:28.721440+0000","last_became_active":"2024-04-17T07:11:55.470897+0000","last_became_peered":"2024-04-17T07:11:55.470897+0000","last_unstale":"2024-04-17T07:12:28.721440+0000","last_undegraded":"2024-04-17T07:12:28.721440+0000","last_fullsized":"2024-04-17T07:12:28.721440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:25.811064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.728989+0000","last_change":"2024-04-17T07:11:55.475860+0000","last_active":"2024-04-17T07:12:28.728989+0000","last_peered":"2024-04-17T07:12:28.728989+0000","last_clean":"2024-04-17T07:12:28.728989+0000","last_became_active":"2024-04-17T07:11:55.475522+0000","last_became_peered":"2024-04-17T07:11:55.475522+0000","last_unstale":"2024-04-17T07:12:28.728989+0000","last_undegraded":"2024-04-17T07:12:28.728989+0000","last_fullsized":"2024-04-17T07:12:28.728989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:41.548964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.715736+0000","last_change":"2024-04-17T07:11:55.478815+0000","last_active":"2024-04-17T07:12:28.715736+0000","last_peered":"2024-04-17T07:12:28.715736+0000","last_clean":"2024-04-17T07:12:28.715736+0000","last_became_active":"2024-04-17T07:11:55.478347+0000","last_became_peered":"2024-04-17T07:11:55.478347+0000","last_unstale":"2024-04-17T07:12:28.715736+0000","last_undegraded":"2024-04-17T07:12:28.715736+0000","last_fullsized":"2024-04-17T07:12:28.715736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:16.465511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.123444+0000","last_change":"2024-04-17T07:11:55.474208+0000","last_active":"2024-04-17T07:12:29.123444+0000","last_peered":"2024-04-17T07:12:29.123444+0000","last_clean":"2024-04-17T07:12:29.123444+0000","last_became_active":"2024-04-17T07:11:55.472042+0000","last_became_peered":"2024-04-17T07:11:55.472042+0000","last_unstale":"2024-04-17T07:12:29.123444+0000","last_undegraded":"2024-04-17T07:12:29.123444+0000","last_fullsized":"2024-04-17T07:12:29.123444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:37:24.248506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.733719+0000","last_change":"2024-04-17T07:11:55.477257+0000","last_active":"2024-04-17T07:12:28.733719+0000","last_peered":"2024-04-17T07:12:28.733719+0000","last_clean":"2024-04-17T07:12:28.733719+0000","last_became_active":"2024-04-17T07:11:55.473491+0000","last_became_peered":"2024-04-17T07:11:55.473491+0000","last_unstale":"2024-04-17T07:12:28.733719+0000","last_undegraded":"2024-04-17T07:12:28.733719+0000","last_fullsized":"2024-04-17T07:12:28.733719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:33.737485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:29.122022+0000","last_change":"2024-04-17T07:11:55.474930+0000","last_active":"2024-04-17T07:12:29.122022+0000","last_peered":"2024-04-17T07:12:29.122022+0000","last_clean":"2024-04-17T07:12:29.122022+0000","last_became_active":"2024-04-17T07:11:55.472007+0000","last_became_peered":"2024-04-17T07:11:55.472007+0000","last_unstale":"2024-04-17T07:12:29.122022+0000","last_undegraded":"2024-04-17T07:12:29.122022+0000","last_fullsized":"2024-04-17T07:12:29.122022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:07.111267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.726119+0000","last_change":"2024-04-17T07:11:55.492422+0000","last_active":"2024-04-17T07:12:28.726119+0000","last_peered":"2024-04-17T07:12:28.726119+0000","last_clean":"2024-04-17T07:12:28.726119+0000","last_became_active":"2024-04-17T07:11:55.491906+0000","last_became_peered":"2024-04-17T07:11:55.491906+0000","last_unstale":"2024-04-17T07:12:28.726119+0000","last_undegraded":"2024-04-17T07:12:28.726119+0000","last_fullsized":"2024-04-17T07:12:28.726119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:13:05.322586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.720385+0000","last_change":"2024-04-17T07:11:55.479436+0000","last_active":"2024-04-17T07:12:28.720385+0000","last_peered":"2024-04-17T07:12:28.720385+0000","last_clean":"2024-04-17T07:12:28.720385+0000","last_became_active":"2024-04-17T07:11:55.478982+0000","last_became_peered":"2024-04-17T07:11:55.478982+0000","last_unstale":"2024-04-17T07:12:28.720385+0000","last_undegraded":"2024-04-17T07:12:28.720385+0000","last_fullsized":"2024-04-17T07:12:28.720385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:20.663457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-17T07:12:28.727023+0000","last_change":"2024-04-17T07:11:55.476708+0000","last_active":"2024-04-17T07:12:28.727023+0000","last_peered":"2024-04-17T07:12:28.727023+0000","last_clean":"2024-04-17T07:12:28.727023+0000","last_became_active":"2024-04-17T07:11:55.476344+0000","last_became_peered":"2024-04-17T07:11:55.476344+0000","last_unstale":"2024-04-17T07:12:28.727023+0000","last_undegraded":"2024-04-17T07:12:28.727023+0000","last_fullsized":"2024-04-17T07:12:28.727023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:58.874987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279575,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53334,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":95,"num_read_kb":111,"num_write":231,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":160002,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383704,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":501561,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":1134786,"data_compressed":58079,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":505475,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":501645,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1098254,"data_compressed":56230,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":496883,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":545447,"data_compressed":22516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":498339,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":544032,"data_compressed":22516,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":496591,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":500957,"data_compressed":20667,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1093836,"data_compressed":56230,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":49286,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":47871,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":48257,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:12:39.354 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T07:12:39.354 INFO:tasks.cephadm:Setting up client nodes... 2024-04-17T07:12:39.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-17T07:12:39.842 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:40.308 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:40 smithi086 ceph-mon[31733]: pgmap v387: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 24 op/s; 60 B/s, 0 objects/s recovering 2024-04-17T07:12:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:40 smithi098 ceph-mon[35418]: pgmap v387: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 24 op/s; 60 B/s, 0 objects/s recovering 2024-04-17T07:12:40.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:40 smithi103 ceph-mon[35495]: pgmap v387: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 24 op/s; 60 B/s, 0 objects/s recovering 2024-04-17T07:12:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:41 smithi098 ceph-mon[35418]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:41 smithi103 ceph-mon[35495]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:41.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:41 smithi086 ceph-mon[31733]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:12:42.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.250+0000 7fee74200700 1 -- 172.21.15.86:0/1909894358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c075450 msgr2=0x7fee6c0758d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:42.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.250+0000 7fee74200700 1 --2- 172.21.15.86:0/1909894358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c075450 0x7fee6c0758d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fee58009990 tx=0x7fee58009ca0 comp rx=0 tx=0).stop 2024-04-17T07:12:42.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.250+0000 7fee74200700 1 -- 172.21.15.86:0/1909894358 shutdown_connections 2024-04-17T07:12:42.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.250+0000 7fee74200700 1 --2- 172.21.15.86:0/1909894358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c10a9c0 0x7fee6c10ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:42.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.250+0000 7fee74200700 1 --2- 172.21.15.86:0/1909894358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee6c076a20 0x7fee6c074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:42.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.250+0000 7fee74200700 1 --2- 172.21.15.86:0/1909894358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c075450 0x7fee6c0758d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:42.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.250+0000 7fee74200700 1 -- 172.21.15.86:0/1909894358 >> 172.21.15.86:0/1909894358 conn(0x7fee6c0fe080 msgr2=0x7fee6c1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:42.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.250+0000 7fee74200700 1 -- 172.21.15.86:0/1909894358 shutdown_connections 2024-04-17T07:12:42.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.250+0000 7fee74200700 1 -- 172.21.15.86:0/1909894358 wait complete. 2024-04-17T07:12:42.252 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:42 smithi086 ceph-mon[31733]: pgmap v388: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 KiB/s wr, 21 op/s; 51 B/s, 0 objects/s recovering 2024-04-17T07:12:42.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.251+0000 7fee74200700 1 Processor -- start 2024-04-17T07:12:42.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.252+0000 7fee74200700 1 -- start start 2024-04-17T07:12:42.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.252+0000 7fee74200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c075450 0x7fee6c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:42.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.252+0000 7fee74200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee6c076a20 0x7fee6c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:42.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.252+0000 7fee74200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c10a9c0 0x7fee6c1a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:42.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.252+0000 7fee74200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c1107b0 con 0x7fee6c10a9c0 2024-04-17T07:12:42.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.252+0000 7fee74200700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c110630 con 0x7fee6c076a20 2024-04-17T07:12:42.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.252+0000 7fee74200700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee6c110930 con 0x7fee6c075450 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.252+0000 7fee72200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c10a9c0 0x7fee6c1a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.252+0000 7fee72200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c10a9c0 0x7fee6c1a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58560/0 (socket says 172.21.15.86:58560) 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.252+0000 7fee72200700 1 -- 172.21.15.86:0/861111531 learned_addr learned my addr 172.21.15.86:0/861111531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.253+0000 7fee70e00700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee6c076a20 0x7fee6c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.253+0000 7fee71800700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c075450 0x7fee6c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.253+0000 7fee72200700 1 -- 172.21.15.86:0/861111531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c075450 msgr2=0x7fee6c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.253+0000 7fee72200700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c075450 0x7fee6c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.253+0000 7fee72200700 1 -- 172.21.15.86:0/861111531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee6c076a20 msgr2=0x7fee6c19b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.253+0000 7fee72200700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee6c076a20 0x7fee6c19b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.253+0000 7fee72200700 1 -- 172.21.15.86:0/861111531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6c1a29a0 con 0x7fee6c10a9c0 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.253+0000 7fee72200700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c10a9c0 0x7fee6c1a2300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fee6400add0 tx=0x7fee6400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.254+0000 7fee6a000700 1 -- 172.21.15.86:0/861111531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64017020 con 0x7fee6c10a9c0 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.254+0000 7fee6a000700 1 -- 172.21.15.86:0/861111531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee6400ae00 con 0x7fee6c10a9c0 2024-04-17T07:12:42.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.254+0000 7fee6a000700 1 -- 172.21.15.86:0/861111531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64005350 con 0x7fee6c10a9c0 2024-04-17T07:12:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.254+0000 7fee74200700 1 -- 172.21.15.86:0/861111531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6c1a2c90 con 0x7fee6c10a9c0 2024-04-17T07:12:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.254+0000 7fee74200700 1 -- 172.21.15.86:0/861111531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6c1a3180 con 0x7fee6c10a9c0 2024-04-17T07:12:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.255+0000 7fee74200700 1 -- 172.21.15.86:0/861111531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee6c050f90 con 0x7fee6c10a9c0 2024-04-17T07:12:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.257+0000 7fee6a000700 1 -- 172.21.15.86:0/861111531 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fee6400c940 con 0x7fee6c10a9c0 2024-04-17T07:12:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.258+0000 7fee6a000700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee540a3e00 0x7fee540a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.259+0000 7fee6a000700 1 -- 172.21.15.86:0/861111531 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fee640d0090 con 0x7fee6c10a9c0 2024-04-17T07:12:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.259+0000 7fee71800700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee540a3e00 0x7fee540a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.260+0000 7fee71800700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee540a3e00 0x7fee540a62c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee60005fd0 tx=0x7fee6000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:42.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.264+0000 7fee6a000700 1 -- 172.21.15.86:0/861111531 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fee64021020 con 0x7fee6c10a9c0 2024-04-17T07:12:42.428 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:42 smithi103 ceph-mon[35495]: pgmap v388: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 KiB/s wr, 21 op/s; 51 B/s, 0 objects/s recovering 2024-04-17T07:12:42.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:42 smithi098 ceph-mon[35418]: pgmap v388: 129 pgs: 129 active+clean; 629 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 KiB/s wr, 21 op/s; 51 B/s, 0 objects/s recovering 2024-04-17T07:12:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.582+0000 7fee74200700 1 -- 172.21.15.86:0/861111531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fee6c02e060 con 0x7fee6c10a9c0 2024-04-17T07:12:42.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.587+0000 7fee6a000700 1 -- 172.21.15.86:0/861111531 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7fee6409c5e0 con 0x7fee6c10a9c0 2024-04-17T07:12:42.587 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-17T07:12:42.587 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBqdh9mshrZIhAA0uF871vay5CjWLJUsdTvJA== 2024-04-17T07:12:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.589+0000 7fee74200700 1 -- 172.21.15.86:0/861111531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee540a3e00 msgr2=0x7fee540a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.589+0000 7fee74200700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee540a3e00 0x7fee540a62c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee60005fd0 tx=0x7fee6000a560 comp rx=0 tx=0).stop 2024-04-17T07:12:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.590+0000 7fee74200700 1 -- 172.21.15.86:0/861111531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c10a9c0 msgr2=0x7fee6c1a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.590+0000 7fee74200700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c10a9c0 0x7fee6c1a2300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fee6400add0 tx=0x7fee6400c330 comp rx=0 tx=0).stop 2024-04-17T07:12:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.590+0000 7fee74200700 1 -- 172.21.15.86:0/861111531 shutdown_connections 2024-04-17T07:12:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.590+0000 7fee74200700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee540a3e00 0x7fee540a62c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.590+0000 7fee74200700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c10a9c0 0x7fee6c1a2300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.590+0000 7fee74200700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee6c075450 0x7fee6c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.590+0000 7fee74200700 1 --2- 172.21.15.86:0/861111531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee6c076a20 0x7fee6c19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.590+0000 7fee74200700 1 -- 172.21.15.86:0/861111531 >> 172.21.15.86:0/861111531 conn(0x7fee6c0fe080 msgr2=0x7fee6c106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.590+0000 7fee74200700 1 -- 172.21.15.86:0/861111531 shutdown_connections 2024-04-17T07:12:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:42.590+0000 7fee74200700 1 -- 172.21.15.86:0/861111531 wait complete. 2024-04-17T07:12:43.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:43 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/861111531' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T07:12:43.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:43 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/861111531' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-17T07:12:43.498 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:12:43.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-17T07:12:43.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-17T07:12:43.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:43 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/861111531' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T07:12:43.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:43 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/861111531' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-17T07:12:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:43 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/861111531' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-17T07:12:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:43 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/861111531' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-17T07:12:43.638 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-17T07:12:43.638 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-17T07:12:43.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph mgr dump --format=json 2024-04-17T07:12:44.087 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:44.487 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:44 smithi086 ceph-mon[31733]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 24 op/s; 51 B/s, 0 objects/s recovering 2024-04-17T07:12:44.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:44 smithi098 ceph-mon[35418]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 24 op/s; 51 B/s, 0 objects/s recovering 2024-04-17T07:12:44.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:44 smithi103 ceph-mon[35495]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 24 op/s; 51 B/s, 0 objects/s recovering 2024-04-17T07:12:46.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.326+0000 7f8b32000700 1 -- 172.21.15.86:0/655043994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2c104ed0 msgr2=0x7f8b2c1052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:46.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.326+0000 7f8b32000700 1 --2- 172.21.15.86:0/655043994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2c104ed0 0x7f8b2c1052b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0099c0 tx=0x7f8b1c009cd0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.326+0000 7f8b32000700 1 -- 172.21.15.86:0/655043994 shutdown_connections 2024-04-17T07:12:46.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.326+0000 7f8b32000700 1 --2- 172.21.15.86:0/655043994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b2c109fb0 0x7f8b2c111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.326+0000 7f8b32000700 1 --2- 172.21.15.86:0/655043994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2c104ed0 0x7f8b2c1052b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.326+0000 7f8b32000700 1 --2- 172.21.15.86:0/655043994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b2c105880 0x7f8b2c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.326+0000 7f8b32000700 1 -- 172.21.15.86:0/655043994 >> 172.21.15.86:0/655043994 conn(0x7f8b2c100760 msgr2=0x7f8b2c102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.326+0000 7f8b32000700 1 -- 172.21.15.86:0/655043994 shutdown_connections 2024-04-17T07:12:46.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.327+0000 7f8b32000700 1 -- 172.21.15.86:0/655043994 wait complete. 2024-04-17T07:12:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.328+0000 7f8b32000700 1 Processor -- start 2024-04-17T07:12:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.328+0000 7f8b32000700 1 -- start start 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.328+0000 7f8b32000700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b2c104ed0 0x7f8b2c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.329+0000 7f8b32000700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b2c105880 0x7f8b2c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.329+0000 7f8b32000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2c109fb0 0x7f8b2c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.329+0000 7f8b32000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c114e40 con 0x7f8b2c109fb0 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.329+0000 7f8b32000700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c114cc0 con 0x7f8b2c105880 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.329+0000 7f8b32000700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c114fc0 con 0x7f8b2c104ed0 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.329+0000 7f8b2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2c109fb0 0x7f8b2c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.329+0000 7f8b2be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2c109fb0 0x7f8b2c19fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58578/0 (socket says 172.21.15.86:58578) 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.329+0000 7f8b2be00700 1 -- 172.21.15.86:0/1727031229 learned_addr learned my addr 172.21.15.86:0/1727031229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.329+0000 7f8b2aa00700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b2c105880 0x7f8b2c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.329+0000 7f8b2b400700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b2c104ed0 0x7f8b2c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.330+0000 7f8b2be00700 1 -- 172.21.15.86:0/1727031229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b2c104ed0 msgr2=0x7f8b2c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.330+0000 7f8b2be00700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b2c104ed0 0x7f8b2c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.330+0000 7f8b2be00700 1 -- 172.21.15.86:0/1727031229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b2c105880 msgr2=0x7f8b2c19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.330+0000 7f8b2be00700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b2c105880 0x7f8b2c19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.330+0000 7f8b2be00700 1 -- 172.21.15.86:0/1727031229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c1a0350 con 0x7f8b2c109fb0 2024-04-17T07:12:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.330+0000 7f8b2be00700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2c109fb0 0x7f8b2c19fc10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8b24007ee0 tx=0x7f8b2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.330+0000 7f8b23e00700 1 -- 172.21.15.86:0/1727031229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24004bb0 con 0x7f8b2c109fb0 2024-04-17T07:12:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.330+0000 7f8b23e00700 1 -- 172.21.15.86:0/1727031229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b24004d50 con 0x7f8b2c109fb0 2024-04-17T07:12:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.330+0000 7f8b23e00700 1 -- 172.21.15.86:0/1727031229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240058e0 con 0x7f8b2c109fb0 2024-04-17T07:12:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.330+0000 7f8b32000700 1 -- 172.21.15.86:0/1727031229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c1a0640 con 0x7f8b2c109fb0 2024-04-17T07:12:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.331+0000 7f8b32000700 1 -- 172.21.15.86:0/1727031229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c1a7ec0 con 0x7f8b2c109fb0 2024-04-17T07:12:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.332+0000 7f8b32000700 1 -- 172.21.15.86:0/1727031229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c069740 con 0x7f8b2c109fb0 2024-04-17T07:12:46.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.334+0000 7f8b23e00700 1 -- 172.21.15.86:0/1727031229 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8b2401a020 con 0x7f8b2c109fb0 2024-04-17T07:12:46.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.336+0000 7f8b23e00700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8b100a3cd0 0x7f8b100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:46.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.336+0000 7f8b23e00700 1 -- 172.21.15.86:0/1727031229 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8b240cfff0 con 0x7f8b2c109fb0 2024-04-17T07:12:46.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.336+0000 7f8b2b400700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8b100a3cd0 0x7f8b100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:46.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.337+0000 7f8b2b400700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8b100a3cd0 0x7f8b100a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c000c00 tx=0x7f8b1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.341+0000 7f8b23e00700 1 -- 172.21.15.86:0/1727031229 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8b24010070 con 0x7f8b2c109fb0 2024-04-17T07:12:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:46 smithi098 ceph-mon[35418]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-04-17T07:12:46.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:46 smithi103 ceph-mon[35495]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-04-17T07:12:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:46 smithi086 ceph-mon[31733]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-04-17T07:12:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.627+0000 7f8b32000700 1 -- 172.21.15.86:0/1727031229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8b2c02df90 con 0x7f8b2c109fb0 2024-04-17T07:12:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.634+0000 7f8b23e00700 1 -- 172.21.15.86:0/1727031229 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264995 (secure 0 0 0) 0x7f8b2409c4d0 con 0x7f8b2c109fb0 2024-04-17T07:12:46.637 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 -- 172.21.15.86:0/1727031229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8b100a3cd0 msgr2=0x7f8b100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8b100a3cd0 0x7f8b100a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c000c00 tx=0x7f8b1c005fb0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 -- 172.21.15.86:0/1727031229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2c109fb0 msgr2=0x7f8b2c19fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2c109fb0 0x7f8b2c19fc10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8b24007ee0 tx=0x7f8b2400d3b0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 -- 172.21.15.86:0/1727031229 shutdown_connections 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8b100a3cd0 0x7f8b100a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b2c109fb0 0x7f8b2c19fc10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b2c104ed0 0x7f8b2c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 --2- 172.21.15.86:0/1727031229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b2c105880 0x7f8b2c19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 -- 172.21.15.86:0/1727031229 >> 172.21.15.86:0/1727031229 conn(0x7f8b2c100760 msgr2=0x7f8b2c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 -- 172.21.15.86:0/1727031229 shutdown_connections 2024-04-17T07:12:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:46.640+0000 7f8b32000700 1 -- 172.21.15.86:0/1727031229 wait complete. 2024-04-17T07:12:47.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:47 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1727031229' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-17T07:12:47.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:47 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1727031229' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-17T07:12:47.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:47 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1727031229' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-17T07:12:47.623 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":561620030},{"type":"v1","addr":"172.21.15.86:6801","nonce":561620030}]},"active_addr":"172.21.15.86:6801/561620030","active_change":"2024-04-17T07:02:26.882232+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.86:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3234336563}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3393751077}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1168582328}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2573645088}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2896627130}]}]} 2024-04-17T07:12:47.628 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-17T07:12:47.629 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-17T07:12:47.629 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd dump --format=json 2024-04-17T07:12:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:48.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:48 smithi086 ceph-mon[31733]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-04-17T07:12:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:48 smithi098 ceph-mon[35418]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-04-17T07:12:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:48 smithi103 ceph-mon[35495]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-04-17T07:12:50.208 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:50 smithi086 ceph-mon[31733]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-17T07:12:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:50 smithi098 ceph-mon[35418]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-17T07:12:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.515+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3080299130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a78105880 msgr2=0x7f3a781098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.515+0000 7f3a7cc00700 1 --2- 172.21.15.86:0/3080299130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a78105880 0x7f3a781098d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3a640099c0 tx=0x7f3a64009cd0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.515+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3080299130 shutdown_connections 2024-04-17T07:12:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.515+0000 7f3a7cc00700 1 --2- 172.21.15.86:0/3080299130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a78109fb0 0x7f3a78111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.515+0000 7f3a7cc00700 1 --2- 172.21.15.86:0/3080299130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a78104ed0 0x7f3a781052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.515+0000 7f3a7cc00700 1 --2- 172.21.15.86:0/3080299130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a78105880 0x7f3a781098d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.515+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3080299130 >> 172.21.15.86:0/3080299130 conn(0x7f3a78100760 msgr2=0x7f3a78102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.515+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3080299130 shutdown_connections 2024-04-17T07:12:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.515+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3080299130 wait complete. 2024-04-17T07:12:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.516+0000 7f3a7cc00700 1 Processor -- start 2024-04-17T07:12:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.516+0000 7f3a7cc00700 1 -- start start 2024-04-17T07:12:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.517+0000 7f3a7cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a78104ed0 0x7f3a7819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.517+0000 7f3a7cc00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a78105880 0x7f3a7819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.517+0000 7f3a7cc00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a78109fb0 0x7f3a7819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.517+0000 7f3a7cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a78114e20 con 0x7f3a78104ed0 2024-04-17T07:12:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.517+0000 7f3a7cc00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a78114ca0 con 0x7f3a78109fb0 2024-04-17T07:12:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.517+0000 7f3a7cc00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a78114fa0 con 0x7f3a78105880 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.517+0000 7f3a75e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a78104ed0 0x7f3a7819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.517+0000 7f3a75e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a78104ed0 0x7f3a7819b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55042/0 (socket says 172.21.15.86:55042) 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.517+0000 7f3a75e00700 1 -- 172.21.15.86:0/3643431232 learned_addr learned my addr 172.21.15.86:0/3643431232 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.518+0000 7f3a75400700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a78105880 0x7f3a7819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.518+0000 7f3a76800700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a78109fb0 0x7f3a7819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.518+0000 7f3a75e00700 1 -- 172.21.15.86:0/3643431232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a78105880 msgr2=0x7f3a7819b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.518+0000 7f3a75e00700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a78105880 0x7f3a7819b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.518+0000 7f3a75e00700 1 -- 172.21.15.86:0/3643431232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a78109fb0 msgr2=0x7f3a7819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.518+0000 7f3a75e00700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a78109fb0 0x7f3a7819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.518+0000 7f3a75e00700 1 -- 172.21.15.86:0/3643431232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a781a0220 con 0x7f3a78104ed0 2024-04-17T07:12:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.518+0000 7f3a76800700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a78109fb0 0x7f3a7819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:12:50.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.518+0000 7f3a75e00700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a78104ed0 0x7f3a7819b210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c9b0 tx=0x7f3a6800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:50.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.519+0000 7f3a6ea00700 1 -- 172.21.15.86:0/3643431232 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68007b00 con 0x7f3a78104ed0 2024-04-17T07:12:50.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.519+0000 7f3a6ea00700 1 -- 172.21.15.86:0/3643431232 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a68007ca0 con 0x7f3a78104ed0 2024-04-17T07:12:50.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.519+0000 7f3a6ea00700 1 -- 172.21.15.86:0/3643431232 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68018660 con 0x7f3a78104ed0 2024-04-17T07:12:50.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.519+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3643431232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a781a0510 con 0x7f3a78104ed0 2024-04-17T07:12:50.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.519+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3643431232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a781a7d90 con 0x7f3a78104ed0 2024-04-17T07:12:50.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.520+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3643431232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a78069740 con 0x7f3a78104ed0 2024-04-17T07:12:50.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.522+0000 7f3a6ea00700 1 -- 172.21.15.86:0/3643431232 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3a6800f450 con 0x7f3a78104ed0 2024-04-17T07:12:50.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.523+0000 7f3a6ea00700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a5c0a3d50 0x7f3a5c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:50.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.524+0000 7f3a6ea00700 1 -- 172.21.15.86:0/3643431232 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f3a680d08d0 con 0x7f3a78104ed0 2024-04-17T07:12:50.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.524+0000 7f3a75400700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a5c0a3d50 0x7f3a5c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:50.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.525+0000 7f3a75400700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a5c0a3d50 0x7f3a5c0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a64000c00 tx=0x7f3a64005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:50.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:50 smithi103 ceph-mon[35495]: pgmap v392: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-17T07:12:50.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.528+0000 7f3a6ea00700 1 -- 172.21.15.86:0/3643431232 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3a68014030 con 0x7f3a78104ed0 2024-04-17T07:12:50.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.752+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3643431232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a7819c570 con 0x7f3a78104ed0 2024-04-17T07:12:50.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.755+0000 7f3a6ea00700 1 -- 172.21.15.86:0/3643431232 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24585 (secure 0 0 0) 0x7f3a6809cd90 con 0x7f3a78104ed0 2024-04-17T07:12:50.755 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:50.755 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":91,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:12:28.705120+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:12:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.758+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3643431232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a5c0a3d50 msgr2=0x7f3a5c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.758+0000 7f3a7cc00700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a5c0a3d50 0x7f3a5c0a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a64000c00 tx=0x7f3a64005fb0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.758+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3643431232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a78104ed0 msgr2=0x7f3a7819b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.758+0000 7f3a7cc00700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a78104ed0 0x7f3a7819b210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c9b0 tx=0x7f3a6800ccc0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.758+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3643431232 shutdown_connections 2024-04-17T07:12:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.758+0000 7f3a7cc00700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a5c0a3d50 0x7f3a5c0a6210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.758+0000 7f3a7cc00700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a78109fb0 0x7f3a7819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.758+0000 7f3a7cc00700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a78104ed0 0x7f3a7819b210 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.758+0000 7f3a7cc00700 1 --2- 172.21.15.86:0/3643431232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a78105880 0x7f3a7819b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.758+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3643431232 >> 172.21.15.86:0/3643431232 conn(0x7f3a78100760 msgr2=0x7f3a781012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.759+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3643431232 shutdown_connections 2024-04-17T07:12:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:50.759+0000 7f3a7cc00700 1 -- 172.21.15.86:0/3643431232 wait complete. 2024-04-17T07:12:51.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:51 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3643431232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:51.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:51 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3643431232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:51 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3643431232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:51.649 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-17T07:12:51.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd dump --format=json 2024-04-17T07:12:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:52.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:52 smithi086 ceph-mon[31733]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-17T07:12:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:52 smithi098 ceph-mon[35418]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-17T07:12:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:52 smithi103 ceph-mon[35495]: pgmap v393: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-17T07:12:53.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:53 smithi098 ceph-mon[35418]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-17T07:12:53.511 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:53 smithi103 ceph-mon[35495]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-17T07:12:53.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:53 smithi086 ceph-mon[31733]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-17T07:12:54.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.387+0000 7fc100c00700 1 -- 172.21.15.86:0/2606567353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc105880 msgr2=0x7fc0fc1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.387+0000 7fc100c00700 1 --2- 172.21.15.86:0/2606567353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc105880 0x7fc0fc1098d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80099c0 tx=0x7fc0e8009cd0 comp rx=0 tx=0).stop 2024-04-17T07:12:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.387+0000 7fc100c00700 1 -- 172.21.15.86:0/2606567353 shutdown_connections 2024-04-17T07:12:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.387+0000 7fc100c00700 1 --2- 172.21.15.86:0/2606567353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0fc109fb0 0x7fc0fc111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.387+0000 7fc100c00700 1 --2- 172.21.15.86:0/2606567353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0fc104ed0 0x7fc0fc1052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.387+0000 7fc100c00700 1 --2- 172.21.15.86:0/2606567353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc105880 0x7fc0fc1098d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.387+0000 7fc100c00700 1 -- 172.21.15.86:0/2606567353 >> 172.21.15.86:0/2606567353 conn(0x7fc0fc100760 msgr2=0x7fc0fc102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:54.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.387+0000 7fc100c00700 1 -- 172.21.15.86:0/2606567353 shutdown_connections 2024-04-17T07:12:54.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.387+0000 7fc100c00700 1 -- 172.21.15.86:0/2606567353 wait complete. 2024-04-17T07:12:54.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.388+0000 7fc100c00700 1 Processor -- start 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.388+0000 7fc100c00700 1 -- start start 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc100c00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0fc104ed0 0x7fc0fc111a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc100c00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0fc105880 0x7fc0fc10eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc100c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc109fb0 0x7fc0fc10f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc100c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc114e70 con 0x7fc0fc109fb0 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc100c00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc114cf0 con 0x7fc0fc104ed0 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc100c00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc114ff0 con 0x7fc0fc105880 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc0fa800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc109fb0 0x7fc0fc10f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc0fa800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc109fb0 0x7fc0fc10f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55060/0 (socket says 172.21.15.86:55060) 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc0fa800700 1 -- 172.21.15.86:0/638624399 learned_addr learned my addr 172.21.15.86:0/638624399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc0f9e00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0fc104ed0 0x7fc0fc111a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.389+0000 7fc0f9400700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0fc105880 0x7fc0fc10eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc0fa800700 1 -- 172.21.15.86:0/638624399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0fc105880 msgr2=0x7fc0fc10eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:54.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc0fa800700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0fc105880 0x7fc0fc10eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc0fa800700 1 -- 172.21.15.86:0/638624399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0fc104ed0 msgr2=0x7fc0fc111a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc0fa800700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0fc104ed0 0x7fc0fc111a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc0fa800700 1 -- 172.21.15.86:0/638624399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc10f940 con 0x7fc0fc109fb0 2024-04-17T07:12:54.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc0f9e00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0fc104ed0 0x7fc0fc111a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:12:54.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc0fa800700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc109fb0 0x7fc0fc10f040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400ada0 tx=0x7fc0f400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:54.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc0f2a00700 1 -- 172.21.15.86:0/638624399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4017020 con 0x7fc0fc109fb0 2024-04-17T07:12:54.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc100c00700 1 -- 172.21.15.86:0/638624399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0fc1a14b0 con 0x7fc0fc109fb0 2024-04-17T07:12:54.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc0f2a00700 1 -- 172.21.15.86:0/638624399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0f400ce30 con 0x7fc0fc109fb0 2024-04-17T07:12:54.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc0f2a00700 1 -- 172.21.15.86:0/638624399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4005460 con 0x7fc0fc109fb0 2024-04-17T07:12:54.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.390+0000 7fc100c00700 1 -- 172.21.15.86:0/638624399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0fc1a1970 con 0x7fc0fc109fb0 2024-04-17T07:12:54.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.392+0000 7fc100c00700 1 -- 172.21.15.86:0/638624399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0fc1063b0 con 0x7fc0fc109fb0 2024-04-17T07:12:54.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.394+0000 7fc0f2a00700 1 -- 172.21.15.86:0/638624399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc0f400c980 con 0x7fc0fc109fb0 2024-04-17T07:12:54.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.395+0000 7fc0f2a00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc0e00a3cd0 0x7fc0e00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:12:54.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.395+0000 7fc0f2a00700 1 -- 172.21.15.86:0/638624399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc0f40d0170 con 0x7fc0fc109fb0 2024-04-17T07:12:54.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.395+0000 7fc0f9e00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc0e00a3cd0 0x7fc0e00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:12:54.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.396+0000 7fc0f9e00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc0e00a3cd0 0x7fc0e00a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec005fd0 tx=0x7fc0ec005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:12:54.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.400+0000 7fc0f2a00700 1 -- 172.21.15.86:0/638624399 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc0f4021020 con 0x7fc0fc109fb0 2024-04-17T07:12:54.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.628+0000 7fc100c00700 1 -- 172.21.15.86:0/638624399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0fc0780b0 con 0x7fc0fc109fb0 2024-04-17T07:12:54.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.630+0000 7fc0f2a00700 1 -- 172.21.15.86:0/638624399 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24585 (secure 0 0 0) 0x7fc0f409c6c0 con 0x7fc0fc109fb0 2024-04-17T07:12:54.630 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:12:54.631 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":91,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:12:28.705120+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:12:54.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 -- 172.21.15.86:0/638624399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc0e00a3cd0 msgr2=0x7fc0e00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:54.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc0e00a3cd0 0x7fc0e00a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec005fd0 tx=0x7fc0ec005e30 comp rx=0 tx=0).stop 2024-04-17T07:12:54.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 -- 172.21.15.86:0/638624399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc109fb0 msgr2=0x7fc0fc10f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:12:54.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc109fb0 0x7fc0fc10f040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400ada0 tx=0x7fc0f400c330 comp rx=0 tx=0).stop 2024-04-17T07:12:54.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 -- 172.21.15.86:0/638624399 shutdown_connections 2024-04-17T07:12:54.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc0e00a3cd0 0x7fc0e00a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:54.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc109fb0 0x7fc0fc10f040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:54.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0fc104ed0 0x7fc0fc111a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:54.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 --2- 172.21.15.86:0/638624399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0fc105880 0x7fc0fc10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:12:54.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 -- 172.21.15.86:0/638624399 >> 172.21.15.86:0/638624399 conn(0x7fc0fc100760 msgr2=0x7fc0fc1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:12:54.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 -- 172.21.15.86:0/638624399 shutdown_connections 2024-04-17T07:12:54.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:12:54.633+0000 7fc100c00700 1 -- 172.21.15.86:0/638624399 wait complete. 2024-04-17T07:12:55.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:54 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/638624399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:55.022 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:54 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/638624399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:55.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:54 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/638624399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:12:55.540 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-17T07:12:55.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-17T07:12:55.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-17T07:12:55.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-17T07:12:55.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-17T07:12:55.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-17T07:12:55.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-17T07:12:55.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-17T07:12:55.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-17T07:12:55.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-17T07:12:55.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-17T07:12:55.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-17T07:12:55.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:55 smithi086 ceph-mon[31733]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-17T07:12:56.005 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:55 smithi098 ceph-mon[35418]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-17T07:12:56.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:55 smithi103 ceph-mon[35495]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-17T07:12:56.545 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:56.645 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:56.944 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:56.968 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:58.253 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:12:58.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:58 smithi086 ceph-mon[31733]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-17T07:12:58.416 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:58 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:12:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:58 smithi098 ceph-mon[35418]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-17T07:12:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:58 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:12:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:58 smithi103 ceph-mon[35495]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-17T07:12:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:58 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:12:59.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:59 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:12:59 smithi086 ceph-mon[31733]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-17T07:12:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:59 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:12:59 smithi098 ceph-mon[35418]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-17T07:12:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:59 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:12:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:12:59 smithi103 ceph-mon[35495]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-17T07:13:01.554 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.554 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.554 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:01 smithi086 ceph-mon[31733]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-17T07:13:01.554 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.554 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:01 smithi098 ceph-mon[35418]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-17T07:13:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:01 smithi103 ceph-mon[35495]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-17T07:13:01.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:01.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:03.358 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:03.359 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:04 smithi098 ceph-mon[35418]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-17T07:13:04.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:04 smithi103 ceph-mon[35495]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-17T07:13:04.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:04 smithi086 ceph-mon[31733]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-17T07:13:05.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.142+0000 7fbbd4e00700 1 -- 172.21.15.86:0/1406265891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbd010cf70 msgr2=0x7fbbd010f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:05.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.142+0000 7fbbd4e00700 1 --2- 172.21.15.86:0/1406265891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbd010cf70 0x7fbbd010f430 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800b0a0 tx=0x7fbbc800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:05.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.142+0000 7fbbd4e00700 1 -- 172.21.15.86:0/1406265891 shutdown_connections 2024-04-17T07:13:05.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.142+0000 7fbbd4e00700 1 --2- 172.21.15.86:0/1406265891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbd010cf70 0x7fbbd010f430 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:05.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.142+0000 7fbbd4e00700 1 --2- 172.21.15.86:0/1406265891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd0103b40 0x7fbbd0103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:05.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.142+0000 7fbbd4e00700 1 --2- 172.21.15.86:0/1406265891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbd0104460 0x7fbbd010c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:05.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.142+0000 7fbbd4e00700 1 -- 172.21.15.86:0/1406265891 >> 172.21.15.86:0/1406265891 conn(0x7fbbd00fd370 msgr2=0x7fbbd00ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:05.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.143+0000 7fbbd4e00700 1 -- 172.21.15.86:0/1406265891 shutdown_connections 2024-04-17T07:13:05.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.143+0000 7fbbd4e00700 1 -- 172.21.15.86:0/1406265891 wait complete. 2024-04-17T07:13:05.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.144+0000 7fbbd4e00700 1 Processor -- start 2024-04-17T07:13:05.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.144+0000 7fbbd4e00700 1 -- start start 2024-04-17T07:13:05.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.145+0000 7fbbd4e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbd0103b40 0x7fbbd019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.145+0000 7fbbd4e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd0104460 0x7fbbd019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.145+0000 7fbbd4e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbd010cf70 0x7fbbd019fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.145+0000 7fbbd4e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd0112bd0 con 0x7fbbd0103b40 2024-04-17T07:13:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.145+0000 7fbbd4e00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd0112a50 con 0x7fbbd010cf70 2024-04-17T07:13:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.145+0000 7fbbd4e00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd0112d50 con 0x7fbbd0104460 2024-04-17T07:13:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.145+0000 7fbbcf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbd0103b40 0x7fbbd019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.145+0000 7fbbcf400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbd0103b40 0x7fbbd019b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37636/0 (socket says 172.21.15.86:37636) 2024-04-17T07:13:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.145+0000 7fbbcf400700 1 -- 172.21.15.86:0/2420876670 learned_addr learned my addr 172.21.15.86:0/2420876670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.145+0000 7fbbcfe00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbd010cf70 0x7fbbd019fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.146+0000 7fbbcf400700 1 -- 172.21.15.86:0/2420876670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd0104460 msgr2=0x7fbbd019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.146+0000 7fbbcea00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd0104460 0x7fbbd019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.146+0000 7fbbcf400700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbd0104460 0x7fbbd019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.146+0000 7fbbcf400700 1 -- 172.21.15.86:0/2420876670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbd010cf70 msgr2=0x7fbbd019fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.146+0000 7fbbcf400700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbd010cf70 0x7fbbd019fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.146+0000 7fbbcf400700 1 -- 172.21.15.86:0/2420876670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd01a0220 con 0x7fbbd0103b40 2024-04-17T07:13:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.146+0000 7fbbcfe00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbd010cf70 0x7fbbd019fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:13:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.146+0000 7fbbcf400700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbd0103b40 0x7fbbd019b210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000d980 tx=0x7fbbc000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.147+0000 7fbbc7e00700 1 -- 172.21.15.86:0/2420876670 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0017070 con 0x7fbbd0103b40 2024-04-17T07:13:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.148+0000 7fbbc7e00700 1 -- 172.21.15.86:0/2420876670 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc0004830 con 0x7fbbd0103b40 2024-04-17T07:13:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.148+0000 7fbbc7e00700 1 -- 172.21.15.86:0/2420876670 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0005440 con 0x7fbbd0103b40 2024-04-17T07:13:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.148+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd01a0510 con 0x7fbbd0103b40 2024-04-17T07:13:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.148+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd01a7d90 con 0x7fbbd0103b40 2024-04-17T07:13:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.150+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbbd01032c0 con 0x7fbbd0103b40 2024-04-17T07:13:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.151+0000 7fbbc7e00700 1 -- 172.21.15.86:0/2420876670 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbbc00055e0 con 0x7fbbd0103b40 2024-04-17T07:13:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.153+0000 7fbbc7e00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbbb40a3d50 0x7fbbb40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.153+0000 7fbbc7e00700 1 -- 172.21.15.86:0/2420876670 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbbc00d0830 con 0x7fbbd0103b40 2024-04-17T07:13:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.153+0000 7fbbc7e00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] conn(0x7fbbb40aeeb0 0x7fbbb40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.153+0000 7fbbc7e00700 1 -- 172.21.15.86:0/2420876670 --> [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbb40b1a20 con 0x7fbbb40aeeb0 2024-04-17T07:13:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.153+0000 7fbbc7e00700 1 -- 172.21.15.86:0/2420876670 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fbbc00d7050 con 0x7fbbd0103b40 2024-04-17T07:13:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.153+0000 7fbbcea00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbbb40a3d50 0x7fbbb40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.155+0000 7fbbcea00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbbb40a3d50 0x7fbbb40a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbbd010e790 tx=0x7fbbbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.154+0000 7fbbcfe00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] conn(0x7fbbb40aeeb0 0x7fbbb40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.156+0000 7fbbcfe00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] conn(0x7fbbb40aeeb0 0x7fbbb40b1350 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:05.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.158+0000 7fbbc7e00700 1 -- 172.21.15.86:0/2420876670 <== osd.10 v2:172.21.15.98:6824/2308079272 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbb40b1a20 con 0x7fbbb40aeeb0 2024-04-17T07:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.177+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 --> [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbbd006b010 con 0x7fbbb40aeeb0 2024-04-17T07:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.178+0000 7fbbc7e00700 1 -- 172.21.15.86:0/2420876670 <== osd.10 v2:172.21.15.98:6824/2308079272 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbbd006b010 con 0x7fbbb40aeeb0 2024-04-17T07:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 >> [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] conn(0x7fbbb40aeeb0 msgr2=0x7fbbb40b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbd4e00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] conn(0x7fbbb40aeeb0 0x7fbbb40b1350 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbbb40a3d50 msgr2=0x7fbbb40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbd4e00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbbb40a3d50 0x7fbbb40a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbbd010e790 tx=0x7fbbbc009260 comp rx=0 tx=0).stop 2024-04-17T07:13:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbd0103b40 msgr2=0x7fbbd019b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbd4e00700 1 --2- 172.21.15.86:0/2420876670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbd0103b40 0x7fbbd019b210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000d980 tx=0x7fbbc000dd40 comp rx=0 tx=0).stop 2024-04-17T07:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbcfe00700 1 -- 172.21.15.86:0/2420876670 reap_dead start 2024-04-17T07:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 shutdown_connections 2024-04-17T07:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 >> 172.21.15.86:0/2420876670 conn(0x7fbbd00fd370 msgr2=0x7fbbd010f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 shutdown_connections 2024-04-17T07:13:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:04.179+0000 7fbbd4e00700 1 -- 172.21.15.86:0/2420876670 wait complete. 2024-04-17T07:13:06.282 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:06 smithi086 ceph-mon[31733]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-17T07:13:06.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:06 smithi098 ceph-mon[35418]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-17T07:13:06.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:06 smithi103 ceph-mon[35495]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-17T07:13:07.084 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:08.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:08 smithi086 ceph-mon[31733]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-17T07:13:08.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:08 smithi098 ceph-mon[35418]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-17T07:13:08.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:08 smithi103 ceph-mon[35495]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-17T07:13:08.931 INFO:teuthology.orchestra.run.smithi086.stdout:309237645350 2024-04-17T07:13:08.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-17T07:13:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.035+0000 7fd1fee00700 1 -- 172.21.15.86:0/2562767619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f8104ef0 msgr2=0x7fd1f81052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.035+0000 7fd1fee00700 1 --2- 172.21.15.86:0/2562767619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f8104ef0 0x7fd1f81052d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80099c0 tx=0x7fd1e8009cd0 comp rx=0 tx=0).stop 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.035+0000 7fd1fee00700 1 -- 172.21.15.86:0/2562767619 shutdown_connections 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.035+0000 7fd1fee00700 1 --2- 172.21.15.86:0/2562767619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f8109fd0 0x7fd1f8111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.035+0000 7fd1fee00700 1 --2- 172.21.15.86:0/2562767619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f8104ef0 0x7fd1f81052d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.035+0000 7fd1fee00700 1 --2- 172.21.15.86:0/2562767619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f81058a0 0x7fd1f81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.035+0000 7fd1fee00700 1 -- 172.21.15.86:0/2562767619 >> 172.21.15.86:0/2562767619 conn(0x7fd1f8100780 msgr2=0x7fd1f8102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.035+0000 7fd1fee00700 1 -- 172.21.15.86:0/2562767619 shutdown_connections 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.035+0000 7fd1fee00700 1 -- 172.21.15.86:0/2562767619 wait complete. 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.036+0000 7fd1fee00700 1 Processor -- start 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.036+0000 7fd1fee00700 1 -- start start 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.036+0000 7fd1fee00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f8104ef0 0x7fd1f819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.036+0000 7fd1fee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81058a0 0x7fd1f819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1fee00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f8109fd0 0x7fd1f819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1fee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8114ec0 con 0x7fd1f81058a0 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1fee00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8114d40 con 0x7fd1f8109fd0 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1fee00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8115040 con 0x7fd1f8104ef0 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81058a0 0x7fd1f819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1ef400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81058a0 0x7fd1f819b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39516/0 (socket says 172.21.15.86:39516) 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1ef400700 1 -- 172.21.15.86:0/330357317 learned_addr learned my addr 172.21.15.86:0/330357317 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1f7e00700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f8104ef0 0x7fd1f819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1fce00700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f8109fd0 0x7fd1f819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1ef400700 1 -- 172.21.15.86:0/330357317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f8104ef0 msgr2=0x7fd1f819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1ef400700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f8104ef0 0x7fd1f819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1ef400700 1 -- 172.21.15.86:0/330357317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f8109fd0 msgr2=0x7fd1f819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1ef400700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f8109fd0 0x7fd1f819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1ef400700 1 -- 172.21.15.86:0/330357317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f81a03f0 con 0x7fd1f81058a0 2024-04-17T07:13:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.037+0000 7fd1fce00700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f8109fd0 0x7fd1f819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:13:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.038+0000 7fd1ef400700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81058a0 0x7fd1f819b970 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c8f0 tx=0x7fd1e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.038+0000 7fd1f5600700 1 -- 172.21.15.86:0/330357317 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0007b00 con 0x7fd1f81058a0 2024-04-17T07:13:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.038+0000 7fd1f5600700 1 -- 172.21.15.86:0/330357317 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1e0007ca0 con 0x7fd1f81058a0 2024-04-17T07:13:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.038+0000 7fd1f5600700 1 -- 172.21.15.86:0/330357317 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0018660 con 0x7fd1f81058a0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.038+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f81a06e0 con 0x7fd1f81058a0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.038+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f81a7f60 con 0x7fd1f81058a0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.039+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1f8063120 con 0x7fd1f81058a0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.041+0000 7fd1f5600700 1 -- 172.21.15.86:0/330357317 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd1e000f450 con 0x7fd1f81058a0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.043+0000 7fd1f5600700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd1dc0a3d50 0x7fd1dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.043+0000 7fd1f5600700 1 -- 172.21.15.86:0/330357317 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd1e00d0b60 con 0x7fd1f81058a0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.043+0000 7fd1f7e00700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd1dc0a3d50 0x7fd1dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.043+0000 7fd1f5600700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] conn(0x7fd1dc0aeeb0 0x7fd1dc0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.043+0000 7fd1f5600700 1 -- 172.21.15.86:0/330357317 --> [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1dc0b1a20 con 0x7fd1dc0aeeb0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.043+0000 7fd1f5600700 1 -- 172.21.15.86:0/330357317 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fd1e00d0f10 con 0x7fd1f81058a0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.044+0000 7fd1fce00700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] conn(0x7fd1dc0aeeb0 0x7fd1dc0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.044+0000 7fd1f7e00700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd1dc0a3d50 0x7fd1dc0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009750 tx=0x7fd1e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.044+0000 7fd1fce00700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] conn(0x7fd1dc0aeeb0 0x7fd1dc0b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.045+0000 7fd1f5600700 1 -- 172.21.15.86:0/330357317 <== osd.3 v2:172.21.15.86:6810/3089932177 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1dc0b1a20 con 0x7fd1dc0aeeb0 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.063+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 --> [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd1f80635f0 con 0x7fd1dc0aeeb0 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.064+0000 7fd1f5600700 1 -- 172.21.15.86:0/330357317 <== osd.3 v2:172.21.15.86:6810/3089932177 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd1f80635f0 con 0x7fd1dc0aeeb0 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] conn(0x7fd1dc0aeeb0 msgr2=0x7fd1dc0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fee00700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] conn(0x7fd1dc0aeeb0 0x7fd1dc0b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd1dc0a3d50 msgr2=0x7fd1dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fee00700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd1dc0a3d50 0x7fd1dc0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009750 tx=0x7fd1e8005fb0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81058a0 msgr2=0x7fd1f819b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fee00700 1 --2- 172.21.15.86:0/330357317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81058a0 0x7fd1f819b970 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c8f0 tx=0x7fd1e000ccb0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fce00700 1 -- 172.21.15.86:0/330357317 reap_dead start 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 shutdown_connections 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 >> 172.21.15.86:0/330357317 conn(0x7fd1f8100780 msgr2=0x7fd1f8102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 shutdown_connections 2024-04-17T07:13:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.065+0000 7fd1fee00700 1 -- 172.21.15.86:0/330357317 wait complete. 2024-04-17T07:13:09.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.149+0000 7f720c000700 1 -- 172.21.15.86:0/4009985080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7204074c80 msgr2=0x7f7204075060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.149+0000 7f720c000700 1 --2- 172.21.15.86:0/4009985080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7204074c80 0x7f7204075060 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009990 tx=0x7f71f8009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.149+0000 7f720c000700 1 -- 172.21.15.86:0/4009985080 shutdown_connections 2024-04-17T07:13:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.149+0000 7f720c000700 1 --2- 172.21.15.86:0/4009985080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7204113c20 0x7f7204116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.149+0000 7f720c000700 1 --2- 172.21.15.86:0/4009985080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7204074c80 0x7f7204075060 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.149+0000 7f720c000700 1 --2- 172.21.15.86:0/4009985080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72040755a0 0x7f72041136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.149+0000 7f720c000700 1 -- 172.21.15.86:0/4009985080 >> 172.21.15.86:0/4009985080 conn(0x7f7204079bb0 msgr2=0x7f7204079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.150+0000 7f720c000700 1 -- 172.21.15.86:0/4009985080 shutdown_connections 2024-04-17T07:13:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.150+0000 7f720c000700 1 -- 172.21.15.86:0/4009985080 wait complete. 2024-04-17T07:13:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.151+0000 7f720c000700 1 Processor -- start 2024-04-17T07:13:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.151+0000 7f720c000700 1 -- start start 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.152+0000 7f720c000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7204074c80 0x7f72041a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.152+0000 7f720c000700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72040755a0 0x7f72041a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.152+0000 7f720c000700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7204113c20 0x7f720419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.152+0000 7f720c000700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7204119280 con 0x7f7204074c80 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.152+0000 7f720c000700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7204119100 con 0x7f7204113c20 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.152+0000 7f720c000700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7204119400 con 0x7f72040755a0 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.152+0000 7f7209600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7204074c80 0x7f72041a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.152+0000 7f7209600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7204074c80 0x7f72041a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39532/0 (socket says 172.21.15.86:39532) 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.152+0000 7f7209600700 1 -- 172.21.15.86:0/126739224 learned_addr learned my addr 172.21.15.86:0/126739224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.152+0000 7f7208c00700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72040755a0 0x7f72041a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.153+0000 7f720a000700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7204113c20 0x7f720419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.153+0000 7f7209600700 1 -- 172.21.15.86:0/126739224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72040755a0 msgr2=0x7f72041a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.153+0000 7f7209600700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72040755a0 0x7f72041a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.153+0000 7f7209600700 1 -- 172.21.15.86:0/126739224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7204113c20 msgr2=0x7f720419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.153+0000 7f7209600700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7204113c20 0x7f720419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.153+0000 7f7209600700 1 -- 172.21.15.86:0/126739224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720419ff90 con 0x7f7204074c80 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.153+0000 7f7209600700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7204074c80 0x7f72041a5540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005910 tx=0x7f71f80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.153+0000 7f7202000700 1 -- 172.21.15.86:0/126739224 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8027070 con 0x7f7204074c80 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.153+0000 7f7202000700 1 -- 172.21.15.86:0/126739224 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71f8003f10 con 0x7f7204074c80 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.153+0000 7f720c000700 1 -- 172.21.15.86:0/126739224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72041a0220 con 0x7f7204074c80 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.154+0000 7f7202000700 1 -- 172.21.15.86:0/126739224 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8022d00 con 0x7f7204074c80 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.154+0000 7f720c000700 1 -- 172.21.15.86:0/126739224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72041ac4c0 con 0x7f7204074c80 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.154+0000 7f71f7400700 1 -- 172.21.15.86:0/126739224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71e0001030 con 0x7f7204074c80 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.157+0000 7f7202000700 1 -- 172.21.15.86:0/126739224 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f71f802a070 con 0x7f7204074c80 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.158+0000 7f7202000700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71ec0a3d50 0x7f71ec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.159+0000 7f7202000700 1 -- 172.21.15.86:0/126739224 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f71f80d2180 con 0x7f7204074c80 2024-04-17T07:13:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.159+0000 7f7208c00700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71ec0a3d50 0x7f71ec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.159+0000 7f7202000700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] conn(0x7f71ec0aeeb0 0x7f71ec0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.159+0000 7f7202000700 1 -- 172.21.15.86:0/126739224 --> [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71ec0b1a20 con 0x7f71ec0aeeb0 2024-04-17T07:13:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.159+0000 7f7202000700 1 -- 172.21.15.86:0/126739224 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f71f80d2530 con 0x7f7204074c80 2024-04-17T07:13:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.159+0000 7f720a000700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] conn(0x7f71ec0aeeb0 0x7f71ec0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.160+0000 7f7208c00700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71ec0a3d50 0x7f71ec0a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72041a0e70 tx=0x7f71f000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.160+0000 7f720a000700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] conn(0x7f71ec0aeeb0 0x7f71ec0b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.161+0000 7f7202000700 1 -- 172.21.15.86:0/126739224 <== osd.0 v2:172.21.15.86:6802/871033288 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71ec0b1a20 con 0x7f71ec0aeeb0 2024-04-17T07:13:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.179+0000 7f71f7400700 1 -- 172.21.15.86:0/126739224 --> [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71e0002e60 con 0x7f71ec0aeeb0 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.180+0000 7f7202000700 1 -- 172.21.15.86:0/126739224 <== osd.0 v2:172.21.15.86:6802/871033288 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f71e0002e60 con 0x7f71ec0aeeb0 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.180+0000 7f71f7400700 1 -- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] conn(0x7f71ec0aeeb0 msgr2=0x7f71ec0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.180+0000 7f71f7400700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] conn(0x7f71ec0aeeb0 0x7f71ec0b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.180+0000 7f71f7400700 1 -- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71ec0a3d50 msgr2=0x7f71ec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.180+0000 7f71f7400700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71ec0a3d50 0x7f71ec0a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72041a0e70 tx=0x7f71f000b3c0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.180+0000 7f71f7400700 1 -- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7204074c80 msgr2=0x7f72041a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.180+0000 7f71f7400700 1 --2- 172.21.15.86:0/126739224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7204074c80 0x7f72041a5540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005910 tx=0x7f71f80047b0 comp rx=0 tx=0).stop 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.180+0000 7f720a000700 1 -- 172.21.15.86:0/126739224 reap_dead start 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.180+0000 7f71f7400700 1 -- 172.21.15.86:0/126739224 shutdown_connections 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.180+0000 7f71f7400700 1 -- 172.21.15.86:0/126739224 >> 172.21.15.86:0/126739224 conn(0x7f7204079bb0 msgr2=0x7f7204110ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.181+0000 7f71f7400700 1 -- 172.21.15.86:0/126739224 shutdown_connections 2024-04-17T07:13:09.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:07.181+0000 7f71f7400700 1 -- 172.21.15.86:0/126739224 wait complete. 2024-04-17T07:13:10.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:10 smithi086 ceph-mon[31733]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:13:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:10 smithi098 ceph-mon[35418]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:13:10.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:10 smithi103 ceph-mon[35495]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:13:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.101+0000 7f184ee00700 1 -- 172.21.15.86:0/248020709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1848108a60 msgr2=0x7f1848108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.101+0000 7f184ee00700 1 --2- 172.21.15.86:0/248020709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1848108a60 0x7f1848108e40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1838009990 tx=0x7f1838009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.101+0000 7f184ee00700 1 -- 172.21.15.86:0/248020709 shutdown_connections 2024-04-17T07:13:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.101+0000 7f184ee00700 1 --2- 172.21.15.86:0/248020709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1848103460 0x7f184810f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.101+0000 7f184ee00700 1 --2- 172.21.15.86:0/248020709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1848108a60 0x7f1848108e40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:12.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.101+0000 7f184ee00700 1 --2- 172.21.15.86:0/248020709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1848102aa0 0x7f1848102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.101+0000 7f184ee00700 1 -- 172.21.15.86:0/248020709 >> 172.21.15.86:0/248020709 conn(0x7f1848079bb0 msgr2=0x7f1848079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.101+0000 7f184ee00700 1 -- 172.21.15.86:0/248020709 shutdown_connections 2024-04-17T07:13:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.101+0000 7f184ee00700 1 -- 172.21.15.86:0/248020709 wait complete. 2024-04-17T07:13:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.102+0000 7f184ee00700 1 Processor -- start 2024-04-17T07:13:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.103+0000 7f184ee00700 1 -- start start 2024-04-17T07:13:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.103+0000 7f184ee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1848102aa0 0x7f18481991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.103+0000 7f184ee00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1848103460 0x7f1848199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.103+0000 7f184ee00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1848108a60 0x7f184819da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.103+0000 7f184ee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1848112c70 con 0x7f1848102aa0 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.103+0000 7f184ee00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1848112af0 con 0x7f1848103460 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.103+0000 7f184ee00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1848112df0 con 0x7f1848108a60 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.103+0000 7f1847e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1848102aa0 0x7f18481991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.103+0000 7f1847e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1848102aa0 0x7f18481991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39556/0 (socket says 172.21.15.86:39556) 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.103+0000 7f1847e00700 1 -- 172.21.15.86:0/2483588617 learned_addr learned my addr 172.21.15.86:0/2483588617 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.104+0000 7f1847400700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1848103460 0x7f1848199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.104+0000 7f184ce00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1848108a60 0x7f184819da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.104+0000 7f1847e00700 1 -- 172.21.15.86:0/2483588617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1848108a60 msgr2=0x7f184819da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.104+0000 7f1847e00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1848108a60 0x7f184819da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.104+0000 7f1847e00700 1 -- 172.21.15.86:0/2483588617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1848103460 msgr2=0x7f1848199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.104+0000 7f1847e00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1848103460 0x7f1848199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.104+0000 7f1847e00700 1 -- 172.21.15.86:0/2483588617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184819e1d0 con 0x7f1848102aa0 2024-04-17T07:13:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.104+0000 7f1847e00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1848102aa0 0x7f18481991c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1838005910 tx=0x7f18380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.105+0000 7f1844c00700 1 -- 172.21.15.86:0/2483588617 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838027070 con 0x7f1848102aa0 2024-04-17T07:13:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.105+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184819e460 con 0x7f1848102aa0 2024-04-17T07:13:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.105+0000 7f1844c00700 1 -- 172.21.15.86:0/2483588617 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1838003f10 con 0x7f1848102aa0 2024-04-17T07:13:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.105+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18481a5d40 con 0x7f1848102aa0 2024-04-17T07:13:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.105+0000 7f1844c00700 1 -- 172.21.15.86:0/2483588617 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838022ca0 con 0x7f1848102aa0 2024-04-17T07:13:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.106+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1848063120 con 0x7f1848102aa0 2024-04-17T07:13:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.108+0000 7f1844c00700 1 -- 172.21.15.86:0/2483588617 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f183802a070 con 0x7f1848102aa0 2024-04-17T07:13:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.110+0000 7f1844c00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f182c0a3cd0 0x7f182c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.110+0000 7f1844c00700 1 -- 172.21.15.86:0/2483588617 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f18380d16d0 con 0x7f1848102aa0 2024-04-17T07:13:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.110+0000 7f1847400700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f182c0a3cd0 0x7f182c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.110+0000 7f1844c00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] conn(0x7f182c0aee30 0x7f182c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.110+0000 7f1844c00700 1 -- 172.21.15.86:0/2483588617 --> [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1838019e50 con 0x7f182c0aee30 2024-04-17T07:13:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.110+0000 7f1844c00700 1 -- 172.21.15.86:0/2483588617 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f18380d1a80 con 0x7f1848102aa0 2024-04-17T07:13:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.111+0000 7f1847400700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f182c0a3cd0 0x7f182c0a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f184807a2f0 tx=0x7f183400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:12.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.125+0000 7f184ce00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] conn(0x7f182c0aee30 0x7f182c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.126+0000 7f184ce00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] conn(0x7f182c0aee30 0x7f182c0b12d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.129+0000 7f1844c00700 1 -- 172.21.15.86:0/2483588617 <== osd.4 v2:172.21.15.98:6808/2532868809 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1838019e50 con 0x7f182c0aee30 2024-04-17T07:13:12.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.146+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 --> [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1848063550 con 0x7f182c0aee30 2024-04-17T07:13:12.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.147+0000 7f1844c00700 1 -- 172.21.15.86:0/2483588617 <== osd.4 v2:172.21.15.98:6808/2532868809 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1848063550 con 0x7f182c0aee30 2024-04-17T07:13:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 >> [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] conn(0x7f182c0aee30 msgr2=0x7f182c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ee00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] conn(0x7f182c0aee30 0x7f182c0b12d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f182c0a3cd0 msgr2=0x7f182c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ee00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f182c0a3cd0 0x7f182c0a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f184807a2f0 tx=0x7f183400b3c0 comp rx=0 tx=0).stop 2024-04-17T07:13:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1848102aa0 msgr2=0x7f18481991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ee00700 1 --2- 172.21.15.86:0/2483588617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1848102aa0 0x7f18481991c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1838005910 tx=0x7f18380047b0 comp rx=0 tx=0).stop 2024-04-17T07:13:12.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ce00700 1 -- 172.21.15.86:0/2483588617 reap_dead start 2024-04-17T07:13:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 shutdown_connections 2024-04-17T07:13:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 >> 172.21.15.86:0/2483588617 conn(0x7f1848079bb0 msgr2=0x7f1848101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 shutdown_connections 2024-04-17T07:13:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.148+0000 7f184ee00700 1 -- 172.21.15.86:0/2483588617 wait complete. 2024-04-17T07:13:12.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:12 smithi086 ceph-mon[31733]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:12.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:12 smithi098 ceph-mon[35418]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:12 smithi103 ceph-mon[35495]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.810+0000 7f6102400700 1 -- 172.21.15.86:0/467243966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60fc1016d0 msgr2=0x7f60fc101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.810+0000 7f6102400700 1 --2- 172.21.15.86:0/467243966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60fc1016d0 0x7f60fc101ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009990 tx=0x7f60f0009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.811+0000 7f6102400700 1 -- 172.21.15.86:0/467243966 shutdown_connections 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.811+0000 7f6102400700 1 --2- 172.21.15.86:0/467243966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc10f280 0x7f60fc111670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.811+0000 7f6102400700 1 --2- 172.21.15.86:0/467243966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60fc1016d0 0x7f60fc101ab0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.811+0000 7f6102400700 1 --2- 172.21.15.86:0/467243966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60fc102080 0x7f60fc10ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.811+0000 7f6102400700 1 -- 172.21.15.86:0/467243966 >> 172.21.15.86:0/467243966 conn(0x7f60fc0fd370 msgr2=0x7f60fc0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.811+0000 7f6102400700 1 -- 172.21.15.86:0/467243966 shutdown_connections 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.811+0000 7f6102400700 1 -- 172.21.15.86:0/467243966 wait complete. 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.812+0000 7f6102400700 1 Processor -- start 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.812+0000 7f6102400700 1 -- start start 2024-04-17T07:13:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f6102400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc1016d0 0x7f60fc19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f6102400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60fc102080 0x7f60fc19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f6102400700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60fc10f280 0x7f60fc19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f6102400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc114da0 con 0x7f60fc102080 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f6102400700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc114c20 con 0x7f60fc10f280 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f6102400700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc114f20 con 0x7f60fc1016d0 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f60fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60fc102080 0x7f60fc19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f60fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60fc102080 0x7f60fc19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39572/0 (socket says 172.21.15.86:39572) 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f60fbe00700 1 -- 172.21.15.86:0/3688324277 learned_addr learned my addr 172.21.15.86:0/3688324277 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f6101000700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc1016d0 0x7f60fc19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.813+0000 7f6101a00700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60fc10f280 0x7f60fc19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f60fbe00700 1 -- 172.21.15.86:0/3688324277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc1016d0 msgr2=0x7f60fc19b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f60fbe00700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60fc1016d0 0x7f60fc19b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:13.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f60fbe00700 1 -- 172.21.15.86:0/3688324277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60fc10f280 msgr2=0x7f60fc19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f60fbe00700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60fc10f280 0x7f60fc19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f60fbe00700 1 -- 172.21.15.86:0/3688324277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc1a0240 con 0x7f60fc102080 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f6101a00700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60fc10f280 0x7f60fc19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f60fbe00700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60fc102080 0x7f60fc19b770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d950 tx=0x7f60e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f60f9600700 1 -- 172.21.15.86:0/3688324277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8016070 con 0x7f60fc102080 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f60f9600700 1 -- 172.21.15.86:0/3688324277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60e8009df0 con 0x7f60fc102080 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc1a0530 con 0x7f60fc102080 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f60f9600700 1 -- 172.21.15.86:0/3688324277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e800f3c0 con 0x7f60fc102080 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.814+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1a7db0 con 0x7f60fc102080 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.816+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60fc0631e0 con 0x7f60fc102080 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.818+0000 7f60f9600700 1 -- 172.21.15.86:0/3688324277 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f60e8019070 con 0x7f60fc102080 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.819+0000 7f60f9600700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60e40a3d50 0x7f60e40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.819+0000 7f60f9600700 1 -- 172.21.15.86:0/3688324277 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f60e80d09c0 con 0x7f60fc102080 2024-04-17T07:13:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.819+0000 7f6101000700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60e40a3d50 0x7f60e40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.819+0000 7f60f9600700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] conn(0x7f60e40aeeb0 0x7f60e40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.819+0000 7f60f9600700 1 -- 172.21.15.86:0/3688324277 --> [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60e8024ea0 con 0x7f60e40aeeb0 2024-04-17T07:13:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.819+0000 7f60f9600700 1 -- 172.21.15.86:0/3688324277 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f60e80d0d70 con 0x7f60fc102080 2024-04-17T07:13:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.820+0000 7f6101000700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60e40a3d50 0x7f60e40a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b5c0 tx=0x7f60f0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.820+0000 7f6101a00700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] conn(0x7f60e40aeeb0 0x7f60e40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.821+0000 7f6101a00700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] conn(0x7f60e40aeeb0 0x7f60e40b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.824+0000 7f60f9600700 1 -- 172.21.15.86:0/3688324277 <== osd.1 v2:172.21.15.98:6800/140628921 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60e8024ea0 con 0x7f60e40aeeb0 2024-04-17T07:13:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.842+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 --> [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60fc0635e0 con 0x7f60e40aeeb0 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.843+0000 7f60f9600700 1 -- 172.21.15.86:0/3688324277 <== osd.1 v2:172.21.15.98:6800/140628921 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f60fc0635e0 con 0x7f60e40aeeb0 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.843+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 >> [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] conn(0x7f60e40aeeb0 msgr2=0x7f60e40b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.843+0000 7f6102400700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] conn(0x7f60e40aeeb0 0x7f60e40b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.843+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60e40a3d50 msgr2=0x7f60e40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.844+0000 7f6102400700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60e40a3d50 0x7f60e40a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f60f000b5c0 tx=0x7f60f0005960 comp rx=0 tx=0).stop 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.844+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60fc102080 msgr2=0x7f60fc19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.844+0000 7f6102400700 1 --2- 172.21.15.86:0/3688324277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60fc102080 0x7f60fc19b770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d950 tx=0x7f60e800dc60 comp rx=0 tx=0).stop 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.844+0000 7f6101a00700 1 -- 172.21.15.86:0/3688324277 reap_dead start 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.844+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 shutdown_connections 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.844+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 >> 172.21.15.86:0/3688324277 conn(0x7f60fc0fd370 msgr2=0x7f60fc0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.844+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 shutdown_connections 2024-04-17T07:13:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:12.844+0000 7f6102400700 1 -- 172.21.15.86:0/3688324277 wait complete. 2024-04-17T07:13:14.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:14 smithi098 ceph-mon[35418]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:13:14.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:14 smithi103 ceph-mon[35495]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:13:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:14 smithi086 ceph-mon[31733]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:13:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.201+0000 7f9b23e00700 1 -- 172.21.15.86:0/2038307215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b240757a0 msgr2=0x7f9b2410ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.201+0000 7f9b23e00700 1 --2- 172.21.15.86:0/2038307215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b240757a0 0x7f9b2410ec80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009990 tx=0x7f9b10009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.201+0000 7f9b23e00700 1 -- 172.21.15.86:0/2038307215 shutdown_connections 2024-04-17T07:13:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.201+0000 7f9b23e00700 1 --2- 172.21.15.86:0/2038307215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b2410f1c0 0x7f9b24111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.201+0000 7f9b23e00700 1 --2- 172.21.15.86:0/2038307215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b24074e80 0x7f9b24075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.201+0000 7f9b23e00700 1 --2- 172.21.15.86:0/2038307215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b240757a0 0x7f9b2410ec80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.201+0000 7f9b23e00700 1 -- 172.21.15.86:0/2038307215 >> 172.21.15.86:0/2038307215 conn(0x7f9b240fe4d0 msgr2=0x7f9b241008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.201+0000 7f9b23e00700 1 -- 172.21.15.86:0/2038307215 shutdown_connections 2024-04-17T07:13:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.201+0000 7f9b23e00700 1 -- 172.21.15.86:0/2038307215 wait complete. 2024-04-17T07:13:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.202+0000 7f9b23e00700 1 Processor -- start 2024-04-17T07:13:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.202+0000 7f9b23e00700 1 -- start start 2024-04-17T07:13:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.203+0000 7f9b23e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b24074e80 0x7f9b2419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.203+0000 7f9b23e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b240757a0 0x7f9b2419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.203+0000 7f9b23e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2410f1c0 0x7f9b2419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.203+0000 7f9b23e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24114db0 con 0x7f9b2410f1c0 2024-04-17T07:13:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.203+0000 7f9b23e00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24114c30 con 0x7f9b24074e80 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.203+0000 7f9b23e00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24114f30 con 0x7f9b240757a0 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.203+0000 7f9b23400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2410f1c0 0x7f9b2419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.203+0000 7f9b23400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2410f1c0 0x7f9b2419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39592/0 (socket says 172.21.15.86:39592) 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.203+0000 7f9b23400700 1 -- 172.21.15.86:0/1311849062 learned_addr learned my addr 172.21.15.86:0/1311849062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.204+0000 7f9b22a00700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b24074e80 0x7f9b2419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.204+0000 7f9b22000700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b240757a0 0x7f9b2419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.204+0000 7f9b23400700 1 -- 172.21.15.86:0/1311849062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b240757a0 msgr2=0x7f9b2419b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.204+0000 7f9b23400700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b240757a0 0x7f9b2419b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.204+0000 7f9b23400700 1 -- 172.21.15.86:0/1311849062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b24074e80 msgr2=0x7f9b2419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.204+0000 7f9b23400700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b24074e80 0x7f9b2419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.204+0000 7f9b23400700 1 -- 172.21.15.86:0/1311849062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b241a0290 con 0x7f9b2410f1c0 2024-04-17T07:13:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.204+0000 7f9b23400700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2410f1c0 0x7f9b2419fb50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c007ee0 tx=0x7f9b1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.204+0000 7f9b1b400700 1 -- 172.21.15.86:0/1311849062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00dbd0 con 0x7f9b2410f1c0 2024-04-17T07:13:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.205+0000 7f9b1b400700 1 -- 172.21.15.86:0/1311849062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b1c00dd70 con 0x7f9b2410f1c0 2024-04-17T07:13:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.205+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b241a0580 con 0x7f9b2410f1c0 2024-04-17T07:13:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.205+0000 7f9b1b400700 1 -- 172.21.15.86:0/1311849062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c005a20 con 0x7f9b2410f1c0 2024-04-17T07:13:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.205+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b241a7e00 con 0x7f9b2410f1c0 2024-04-17T07:13:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.206+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b24063160 con 0x7f9b2410f1c0 2024-04-17T07:13:15.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.208+0000 7f9b1b400700 1 -- 172.21.15.86:0/1311849062 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b1c01a020 con 0x7f9b2410f1c0 2024-04-17T07:13:15.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.209+0000 7f9b1b400700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b080a3d50 0x7f9b080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:15.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.209+0000 7f9b1b400700 1 -- 172.21.15.86:0/1311849062 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9b1c0d0690 con 0x7f9b2410f1c0 2024-04-17T07:13:15.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.210+0000 7f9b22a00700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b080a3d50 0x7f9b080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:15.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.210+0000 7f9b1b400700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] conn(0x7f9b080aeeb0 0x7f9b080b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:15.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.210+0000 7f9b1b400700 1 -- 172.21.15.86:0/1311849062 --> [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b1c021e70 con 0x7f9b080aeeb0 2024-04-17T07:13:15.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.210+0000 7f9b1b400700 1 -- 172.21.15.86:0/1311849062 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f9b1c0d0a40 con 0x7f9b2410f1c0 2024-04-17T07:13:15.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.210+0000 7f9b22a00700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b080a3d50 0x7f9b080a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9b14005950 tx=0x7f9b140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:15.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.210+0000 7f9b22000700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] conn(0x7f9b080aeeb0 0x7f9b080b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:15.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.211+0000 7f9b22000700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] conn(0x7f9b080aeeb0 0x7f9b080b1350 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:15.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.214+0000 7f9b1b400700 1 -- 172.21.15.86:0/1311849062 <== osd.8 v2:172.21.15.103:6816/55757000 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b1c021e70 con 0x7f9b080aeeb0 2024-04-17T07:13:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.232+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 --> [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b24063560 con 0x7f9b080aeeb0 2024-04-17T07:13:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.233+0000 7f9b1b400700 1 -- 172.21.15.86:0/1311849062 <== osd.8 v2:172.21.15.103:6816/55757000 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b24063560 con 0x7f9b080aeeb0 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.233+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 >> [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] conn(0x7f9b080aeeb0 msgr2=0x7f9b080b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.233+0000 7f9b23e00700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] conn(0x7f9b080aeeb0 0x7f9b080b1350 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.233+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b080a3d50 msgr2=0x7f9b080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.233+0000 7f9b23e00700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9b080a3d50 0x7f9b080a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9b14005950 tx=0x7f9b140058e0 comp rx=0 tx=0).stop 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.233+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2410f1c0 msgr2=0x7f9b2419fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.233+0000 7f9b23e00700 1 --2- 172.21.15.86:0/1311849062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b2410f1c0 0x7f9b2419fb50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c007ee0 tx=0x7f9b1c00d3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.233+0000 7f9b23400700 1 -- 172.21.15.86:0/1311849062 reap_dead start 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.234+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 shutdown_connections 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.234+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 >> 172.21.15.86:0/1311849062 conn(0x7f9b240fe4d0 msgr2=0x7f9b240ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.234+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 shutdown_connections 2024-04-17T07:13:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:15.234+0000 7f9b23e00700 1 -- 172.21.15.86:0/1311849062 wait complete. 2024-04-17T07:13:16.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.380+0000 7fd6e5600700 1 -- 172.21.15.86:0/4255225644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e00757a0 msgr2=0x7fd6e010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:16.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.380+0000 7fd6e5600700 1 --2- 172.21.15.86:0/4255225644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e00757a0 0x7fd6e010ec70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009990 tx=0x7fd6cc009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:16.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.380+0000 7fd6e5600700 1 -- 172.21.15.86:0/4255225644 shutdown_connections 2024-04-17T07:13:16.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.380+0000 7fd6e5600700 1 --2- 172.21.15.86:0/4255225644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e010f1b0 0x7fd6e0111620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:16.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.380+0000 7fd6e5600700 1 --2- 172.21.15.86:0/4255225644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6e0074e80 0x7fd6e0075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:16.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.380+0000 7fd6e5600700 1 --2- 172.21.15.86:0/4255225644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e00757a0 0x7fd6e010ec70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:16.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.380+0000 7fd6e5600700 1 -- 172.21.15.86:0/4255225644 >> 172.21.15.86:0/4255225644 conn(0x7fd6e00fe460 msgr2=0x7fd6e0100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:16.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.380+0000 7fd6e5600700 1 -- 172.21.15.86:0/4255225644 shutdown_connections 2024-04-17T07:13:16.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.380+0000 7fd6e5600700 1 -- 172.21.15.86:0/4255225644 wait complete. 2024-04-17T07:13:16.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:16 smithi086 ceph-mon[31733]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.381+0000 7fd6e5600700 1 Processor -- start 2024-04-17T07:13:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.382+0000 7fd6e5600700 1 -- start start 2024-04-17T07:13:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.382+0000 7fd6e5600700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6e0074e80 0x7fd6e019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.382+0000 7fd6e5600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e00757a0 0x7fd6e019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.382+0000 7fd6e5600700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e010f1b0 0x7fd6e019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.382+0000 7fd6e5600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0114d30 con 0x7fd6e00757a0 2024-04-17T07:13:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.382+0000 7fd6e5600700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0114bb0 con 0x7fd6e010f1b0 2024-04-17T07:13:16.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.382+0000 7fd6e5600700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0114eb0 con 0x7fd6e0074e80 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.382+0000 7fd6df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e00757a0 0x7fd6e019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.382+0000 7fd6df400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e00757a0 0x7fd6e019b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39612/0 (socket says 172.21.15.86:39612) 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.382+0000 7fd6df400700 1 -- 172.21.15.86:0/983441229 learned_addr learned my addr 172.21.15.86:0/983441229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.383+0000 7fd6dfe00700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6e0074e80 0x7fd6e019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.383+0000 7fd6e4c00700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e010f1b0 0x7fd6e019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.383+0000 7fd6df400700 1 -- 172.21.15.86:0/983441229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6e0074e80 msgr2=0x7fd6e019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.383+0000 7fd6df400700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6e0074e80 0x7fd6e019b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.383+0000 7fd6df400700 1 -- 172.21.15.86:0/983441229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e010f1b0 msgr2=0x7fd6e019fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.383+0000 7fd6df400700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e010f1b0 0x7fd6e019fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.383+0000 7fd6df400700 1 -- 172.21.15.86:0/983441229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e01a0190 con 0x7fd6e00757a0 2024-04-17T07:13:16.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.383+0000 7fd6df400700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e00757a0 0x7fd6e019b6c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc000c00 tx=0x7fd6cc003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.384+0000 7fd6dcc00700 1 -- 172.21.15.86:0/983441229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc027070 con 0x7fd6e00757a0 2024-04-17T07:13:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.384+0000 7fd6dcc00700 1 -- 172.21.15.86:0/983441229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6cc00bdc0 con 0x7fd6e00757a0 2024-04-17T07:13:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.384+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e01a0420 con 0x7fd6e00757a0 2024-04-17T07:13:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.384+0000 7fd6dcc00700 1 -- 172.21.15.86:0/983441229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00f6c0 con 0x7fd6e00757a0 2024-04-17T07:13:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.384+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e01a7d00 con 0x7fd6e00757a0 2024-04-17T07:13:16.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.385+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6e0063160 con 0x7fd6e00757a0 2024-04-17T07:13:16.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.387+0000 7fd6dcc00700 1 -- 172.21.15.86:0/983441229 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd6cc02a070 con 0x7fd6e00757a0 2024-04-17T07:13:16.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.388+0000 7fd6dcc00700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6c40a3d50 0x7fd6c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.389+0000 7fd6dcc00700 1 -- 172.21.15.86:0/983441229 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd6cc0d2120 con 0x7fd6e00757a0 2024-04-17T07:13:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.389+0000 7fd6dfe00700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6c40a3d50 0x7fd6c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.389+0000 7fd6dcc00700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] conn(0x7fd6c40aeeb0 0x7fd6c40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.389+0000 7fd6dcc00700 1 -- 172.21.15.86:0/983441229 --> [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6c40b1a20 con 0x7fd6c40aeeb0 2024-04-17T07:13:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.389+0000 7fd6dcc00700 1 -- 172.21.15.86:0/983441229 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fd6cc0d24d0 con 0x7fd6e00757a0 2024-04-17T07:13:16.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.390+0000 7fd6dfe00700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6c40a3d50 0x7fd6c40a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0005ea0 tx=0x7fd6d0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:16.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.390+0000 7fd6e4c00700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] conn(0x7fd6c40aeeb0 0x7fd6c40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:16.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.391+0000 7fd6e4c00700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] conn(0x7fd6c40aeeb0 0x7fd6c40b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:16.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.393+0000 7fd6dcc00700 1 -- 172.21.15.86:0/983441229 <== osd.7 v2:172.21.15.98:6816/3839591799 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6c40b1a20 con 0x7fd6c40aeeb0 2024-04-17T07:13:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.411+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 --> [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd6e0063560 con 0x7fd6c40aeeb0 2024-04-17T07:13:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.412+0000 7fd6dcc00700 1 -- 172.21.15.86:0/983441229 <== osd.7 v2:172.21.15.98:6816/3839591799 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd6e0063560 con 0x7fd6c40aeeb0 2024-04-17T07:13:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.413+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 >> [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] conn(0x7fd6c40aeeb0 msgr2=0x7fd6c40b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.413+0000 7fd6e5600700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] conn(0x7fd6c40aeeb0 0x7fd6c40b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.413+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6c40a3d50 msgr2=0x7fd6c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.413+0000 7fd6e5600700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6c40a3d50 0x7fd6c40a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0005ea0 tx=0x7fd6d0005e30 comp rx=0 tx=0).stop 2024-04-17T07:13:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.413+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e00757a0 msgr2=0x7fd6e019b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.413+0000 7fd6e5600700 1 --2- 172.21.15.86:0/983441229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e00757a0 0x7fd6e019b6c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc000c00 tx=0x7fd6cc003b30 comp rx=0 tx=0).stop 2024-04-17T07:13:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.413+0000 7fd6e4c00700 1 -- 172.21.15.86:0/983441229 reap_dead start 2024-04-17T07:13:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.413+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 shutdown_connections 2024-04-17T07:13:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.413+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 >> 172.21.15.86:0/983441229 conn(0x7fd6e00fe460 msgr2=0x7fd6e00ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.413+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 shutdown_connections 2024-04-17T07:13:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:16.414+0000 7fd6e5600700 1 -- 172.21.15.86:0/983441229 wait complete. 2024-04-17T07:13:16.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:16 smithi098 ceph-mon[35418]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:16 smithi103 ceph-mon[35495]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:17.450 INFO:teuthology.orchestra.run.smithi086.stdout:107374182486 2024-04-17T07:13:17.451 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-17T07:13:17.789 INFO:teuthology.orchestra.run.smithi086.stdout:34359738475 2024-04-17T07:13:17.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-17T07:13:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.019+0000 7fe7f2400700 1 -- 172.21.15.86:0/1570708759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec113c20 msgr2=0x7fe7ec116010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.019+0000 7fe7f2400700 1 --2- 172.21.15.86:0/1570708759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec113c20 0x7fe7ec116010 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400b0a0 tx=0x7fe7e400b3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.019+0000 7fe7f2400700 1 -- 172.21.15.86:0/1570708759 shutdown_connections 2024-04-17T07:13:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.019+0000 7fe7f2400700 1 --2- 172.21.15.86:0/1570708759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec113c20 0x7fe7ec116010 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.019+0000 7fe7f2400700 1 --2- 172.21.15.86:0/1570708759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec074c80 0x7fe7ec075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.019+0000 7fe7f2400700 1 --2- 172.21.15.86:0/1570708759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ec0755a0 0x7fe7ec1136e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.019+0000 7fe7f2400700 1 -- 172.21.15.86:0/1570708759 >> 172.21.15.86:0/1570708759 conn(0x7fe7ec079c00 msgr2=0x7fe7ec07a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.020+0000 7fe7f2400700 1 -- 172.21.15.86:0/1570708759 shutdown_connections 2024-04-17T07:13:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.020+0000 7fe7f2400700 1 -- 172.21.15.86:0/1570708759 wait complete. 2024-04-17T07:13:18.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.021+0000 7fe7f2400700 1 Processor -- start 2024-04-17T07:13:18.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.022+0000 7fe7f2400700 1 -- start start 2024-04-17T07:13:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.022+0000 7fe7f2400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec074c80 0x7fe7ec19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.022+0000 7fe7f2400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec0755a0 0x7fe7ec19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.022+0000 7fe7f2400700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ec113c20 0x7fe7ec1a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.022+0000 7fe7f2400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec119250 con 0x7fe7ec074c80 2024-04-17T07:13:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.022+0000 7fe7f2400700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1190d0 con 0x7fe7ec113c20 2024-04-17T07:13:18.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.022+0000 7fe7f2400700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1193d0 con 0x7fe7ec0755a0 2024-04-17T07:13:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7eb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec074c80 0x7fe7ec19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7eb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec074c80 0x7fe7ec19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37792/0 (socket says 172.21.15.86:37792) 2024-04-17T07:13:18.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7eb400700 1 -- 172.21.15.86:0/3403065729 learned_addr learned my addr 172.21.15.86:0/3403065729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7ebe00700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ec113c20 0x7fe7ec1a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7eaa00700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec0755a0 0x7fe7ec19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7eb400700 1 -- 172.21.15.86:0/3403065729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec0755a0 msgr2=0x7fe7ec19fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7eb400700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec0755a0 0x7fe7ec19fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7eb400700 1 -- 172.21.15.86:0/3403065729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ec113c20 msgr2=0x7fe7ec1a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7eb400700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ec113c20 0x7fe7ec1a3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7eb400700 1 -- 172.21.15.86:0/3403065729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec1a4650 con 0x7fe7ec074c80 2024-04-17T07:13:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7ebe00700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ec113c20 0x7fe7ec1a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:13:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.023+0000 7fe7eaa00700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7ec0755a0 0x7fe7ec19fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:13:18.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.024+0000 7fe7eb400700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec074c80 0x7fe7ec19f640 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c9b0 tx=0x7fe7dc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:18.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.024+0000 7fe7e3e00700 1 -- 172.21.15.86:0/3403065729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc007b00 con 0x7fe7ec074c80 2024-04-17T07:13:18.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.024+0000 7fe7e3e00700 1 -- 172.21.15.86:0/3403065729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7dc007ca0 con 0x7fe7ec074c80 2024-04-17T07:13:18.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.024+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec1a4940 con 0x7fe7ec074c80 2024-04-17T07:13:18.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.024+0000 7fe7e3e00700 1 -- 172.21.15.86:0/3403065729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc018730 con 0x7fe7ec074c80 2024-04-17T07:13:18.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.024+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec1ac1c0 con 0x7fe7ec074c80 2024-04-17T07:13:18.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.025+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7ec113440 con 0x7fe7ec074c80 2024-04-17T07:13:18.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.027+0000 7fe7e3e00700 1 -- 172.21.15.86:0/3403065729 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe7dc00f450 con 0x7fe7ec074c80 2024-04-17T07:13:18.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.030+0000 7fe7e3e00700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7d00a3cd0 0x7fe7d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.030+0000 7fe7e3e00700 1 -- 172.21.15.86:0/3403065729 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe7dc0d0e50 con 0x7fe7ec074c80 2024-04-17T07:13:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.030+0000 7fe7eaa00700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7d00a3cd0 0x7fe7d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.030+0000 7fe7e3e00700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] conn(0x7fe7d00aee30 0x7fe7d00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.030+0000 7fe7e3e00700 1 -- 172.21.15.86:0/3403065729 --> [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7d00b19a0 con 0x7fe7d00aee30 2024-04-17T07:13:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.030+0000 7fe7e3e00700 1 -- 172.21.15.86:0/3403065729 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe7dc0d1200 con 0x7fe7ec074c80 2024-04-17T07:13:18.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.031+0000 7fe7eaa00700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7d00a3cd0 0x7fe7d00a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec110540 tx=0x7fe7d8009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:18.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.031+0000 7fe7ebe00700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] conn(0x7fe7d00aee30 0x7fe7d00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:18.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.032+0000 7fe7ebe00700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] conn(0x7fe7d00aee30 0x7fe7d00b12d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.035+0000 7fe7e3e00700 1 -- 172.21.15.86:0/3403065729 <== osd.2 v2:172.21.15.103:6800/3728485789 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7d00b19a0 con 0x7fe7d00aee30 2024-04-17T07:13:18.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.055+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 --> [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7ec0763a0 con 0x7fe7d00aee30 2024-04-17T07:13:18.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.056+0000 7fe7e3e00700 1 -- 172.21.15.86:0/3403065729 <== osd.2 v2:172.21.15.103:6800/3728485789 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe7ec0763a0 con 0x7fe7d00aee30 2024-04-17T07:13:18.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.057+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 >> [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] conn(0x7fe7d00aee30 msgr2=0x7fe7d00b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.057+0000 7fe7f2400700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] conn(0x7fe7d00aee30 0x7fe7d00b12d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.057+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7d00a3cd0 msgr2=0x7fe7d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.057+0000 7fe7f2400700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7d00a3cd0 0x7fe7d00a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec110540 tx=0x7fe7d8009550 comp rx=0 tx=0).stop 2024-04-17T07:13:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.057+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec074c80 msgr2=0x7fe7ec19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:18.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.057+0000 7fe7f2400700 1 --2- 172.21.15.86:0/3403065729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec074c80 0x7fe7ec19f640 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c9b0 tx=0x7fe7dc00cd70 comp rx=0 tx=0).stop 2024-04-17T07:13:18.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.057+0000 7fe7ebe00700 1 -- 172.21.15.86:0/3403065729 reap_dead start 2024-04-17T07:13:18.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.057+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 shutdown_connections 2024-04-17T07:13:18.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.057+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 >> 172.21.15.86:0/3403065729 conn(0x7fe7ec079c00 msgr2=0x7fe7ec1129e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:18.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.057+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 shutdown_connections 2024-04-17T07:13:18.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:18.058+0000 7fe7f2400700 1 -- 172.21.15.86:0/3403065729 wait complete. 2024-04-17T07:13:18.328 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:18 smithi086 ceph-mon[31733]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-04-17T07:13:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:18 smithi098 ceph-mon[35418]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-04-17T07:13:18.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:18 smithi103 ceph-mon[35495]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-04-17T07:13:20.493 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:20 smithi086 ceph-mon[31733]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-17T07:13:20.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:20 smithi098 ceph-mon[35418]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-17T07:13:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:20 smithi103 ceph-mon[35495]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-17T07:13:20.542 INFO:teuthology.orchestra.run.smithi086.stdout:141733920849 2024-04-17T07:13:20.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-17T07:13:21.782 INFO:teuthology.orchestra.run.smithi086.stdout:219043332156 2024-04-17T07:13:21.783 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-17T07:13:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.516+0000 7efd92600700 1 -- 172.21.15.86:0/570996090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c109f80 msgr2=0x7efd8c111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.516+0000 7efd92600700 1 --2- 172.21.15.86:0/570996090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c109f80 0x7efd8c111b50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efd8400b0d0 tx=0x7efd8400b3e0 comp rx=0 tx=0).stop 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.516+0000 7efd92600700 1 -- 172.21.15.86:0/570996090 shutdown_connections 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.516+0000 7efd92600700 1 --2- 172.21.15.86:0/570996090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c109f80 0x7efd8c111b50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.516+0000 7efd92600700 1 --2- 172.21.15.86:0/570996090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd8c104ea0 0x7efd8c105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.516+0000 7efd92600700 1 --2- 172.21.15.86:0/570996090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd8c105850 0x7efd8c1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.516+0000 7efd92600700 1 -- 172.21.15.86:0/570996090 >> 172.21.15.86:0/570996090 conn(0x7efd8c100730 msgr2=0x7efd8c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.516+0000 7efd92600700 1 -- 172.21.15.86:0/570996090 shutdown_connections 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.517+0000 7efd92600700 1 -- 172.21.15.86:0/570996090 wait complete. 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.517+0000 7efd92600700 1 Processor -- start 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.518+0000 7efd92600700 1 -- start start 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.518+0000 7efd92600700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd8c104ea0 0x7efd8c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.518+0000 7efd92600700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd8c105850 0x7efd8c19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.518+0000 7efd92600700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c109f80 0x7efd8c19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.518+0000 7efd92600700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c114e90 con 0x7efd8c109f80 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.518+0000 7efd92600700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c114d10 con 0x7efd8c105850 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.518+0000 7efd92600700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c115010 con 0x7efd8c104ea0 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.518+0000 7efd8be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c109f80 0x7efd8c19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.518+0000 7efd8be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c109f80 0x7efd8c19fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37816/0 (socket says 172.21.15.86:37816) 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.518+0000 7efd8be00700 1 -- 172.21.15.86:0/1672061181 learned_addr learned my addr 172.21.15.86:0/1672061181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.519+0000 7efd8aa00700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd8c105850 0x7efd8c19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.519+0000 7efd8b400700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd8c104ea0 0x7efd8c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.519+0000 7efd8be00700 1 -- 172.21.15.86:0/1672061181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd8c104ea0 msgr2=0x7efd8c19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.519+0000 7efd8be00700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd8c104ea0 0x7efd8c19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.519+0000 7efd8be00700 1 -- 172.21.15.86:0/1672061181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd8c105850 msgr2=0x7efd8c19b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.519+0000 7efd8be00700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd8c105850 0x7efd8c19b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.519+0000 7efd8be00700 1 -- 172.21.15.86:0/1672061181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c1a02e0 con 0x7efd8c109f80 2024-04-17T07:13:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.519+0000 7efd8aa00700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd8c105850 0x7efd8c19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.519+0000 7efd8b400700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd8c104ea0 0x7efd8c19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.519+0000 7efd8be00700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c109f80 0x7efd8c19fba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efd8400d010 tx=0x7efd84007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.520+0000 7efd83e00700 1 -- 172.21.15.86:0/1672061181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8401f070 con 0x7efd8c109f80 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.520+0000 7efd83e00700 1 -- 172.21.15.86:0/1672061181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd84003c00 con 0x7efd8c109f80 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.520+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c1a0570 con 0x7efd8c109f80 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.520+0000 7efd83e00700 1 -- 172.21.15.86:0/1672061181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8401b6a0 con 0x7efd8c109f80 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.520+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c1a7e50 con 0x7efd8c109f80 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.521+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd8c0631a0 con 0x7efd8c109f80 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.523+0000 7efd83e00700 1 -- 172.21.15.86:0/1672061181 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efd84022070 con 0x7efd8c109f80 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.524+0000 7efd83e00700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd700a3cd0 0x7efd700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.524+0000 7efd83e00700 1 -- 172.21.15.86:0/1672061181 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7efd8400f020 con 0x7efd8c109f80 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.524+0000 7efd8b400700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd700a3cd0 0x7efd700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.525+0000 7efd83e00700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] conn(0x7efd700aee30 0x7efd700b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.525+0000 7efd83e00700 1 -- 172.21.15.86:0/1672061181 --> [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd8402de50 con 0x7efd700aee30 2024-04-17T07:13:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.525+0000 7efd83e00700 1 -- 172.21.15.86:0/1672061181 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7efd840d2100 con 0x7efd8c109f80 2024-04-17T07:13:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.525+0000 7efd8aa00700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] conn(0x7efd700aee30 0x7efd700b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.525+0000 7efd8b400700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd700a3cd0 0x7efd700a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd7c005950 tx=0x7efd7c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.526+0000 7efd8aa00700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] conn(0x7efd700aee30 0x7efd700b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.527+0000 7efd83e00700 1 -- 172.21.15.86:0/1672061181 <== osd.9 v2:172.21.15.86:6826/3368302207 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd8402de50 con 0x7efd700aee30 2024-04-17T07:13:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.545+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 --> [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efd8c0635a0 con 0x7efd700aee30 2024-04-17T07:13:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.546+0000 7efd83e00700 1 -- 172.21.15.86:0/1672061181 <== osd.9 v2:172.21.15.86:6826/3368302207 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efd8c0635a0 con 0x7efd700aee30 2024-04-17T07:13:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.546+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] conn(0x7efd700aee30 msgr2=0x7efd700b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.546+0000 7efd92600700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] conn(0x7efd700aee30 0x7efd700b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.546+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd700a3cd0 msgr2=0x7efd700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.546+0000 7efd92600700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd700a3cd0 0x7efd700a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd7c005950 tx=0x7efd7c0058e0 comp rx=0 tx=0).stop 2024-04-17T07:13:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.547+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c109f80 msgr2=0x7efd8c19fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.547+0000 7efd92600700 1 --2- 172.21.15.86:0/1672061181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c109f80 0x7efd8c19fba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efd8400d010 tx=0x7efd84007a50 comp rx=0 tx=0).stop 2024-04-17T07:13:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.547+0000 7efd8be00700 1 -- 172.21.15.86:0/1672061181 reap_dead start 2024-04-17T07:13:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.547+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 shutdown_connections 2024-04-17T07:13:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.547+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 >> 172.21.15.86:0/1672061181 conn(0x7efd8c100730 msgr2=0x7efd8c1012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.547+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 shutdown_connections 2024-04-17T07:13:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.547+0000 7efd92600700 1 -- 172.21.15.86:0/1672061181 wait complete. 2024-04-17T07:13:22.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:22 smithi098 ceph-mon[35418]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:22.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:22 smithi103 ceph-mon[35495]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:22.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:22 smithi086 ceph-mon[31733]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:22.687 INFO:teuthology.orchestra.run.smithi086.stdout:77309411422 2024-04-17T07:13:22.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-17T07:13:23.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.459+0000 7fc9bee00700 1 -- 172.21.15.86:0/3807854828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b8102980 msgr2=0x7fc9b8102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.459+0000 7fc9bee00700 1 --2- 172.21.15.86:0/3807854828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b8102980 0x7fc9b8102e00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009990 tx=0x7fc9a4009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.459+0000 7fc9bee00700 1 -- 172.21.15.86:0/3807854828 shutdown_connections 2024-04-17T07:13:23.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.459+0000 7fc9bee00700 1 --2- 172.21.15.86:0/3807854828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9b8103340 0x7fc9b810f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.459+0000 7fc9bee00700 1 --2- 172.21.15.86:0/3807854828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b8108980 0x7fc9b8108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.459+0000 7fc9bee00700 1 --2- 172.21.15.86:0/3807854828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b8102980 0x7fc9b8102e00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.459+0000 7fc9bee00700 1 -- 172.21.15.86:0/3807854828 >> 172.21.15.86:0/3807854828 conn(0x7fc9b80fe4d0 msgr2=0x7fc9b81008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:23.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.459+0000 7fc9bee00700 1 -- 172.21.15.86:0/3807854828 shutdown_connections 2024-04-17T07:13:23.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.459+0000 7fc9bee00700 1 -- 172.21.15.86:0/3807854828 wait complete. 2024-04-17T07:13:23.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.461+0000 7fc9bee00700 1 Processor -- start 2024-04-17T07:13:23.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.461+0000 7fc9bee00700 1 -- start start 2024-04-17T07:13:23.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9bee00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b8102980 0x7fc9b80765f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:23.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9bee00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9b8103340 0x7fc9b8071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:23.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9bee00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b8108980 0x7fc9b8071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:23.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9bee00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8112c60 con 0x7fc9b8108980 2024-04-17T07:13:23.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9bee00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8112ae0 con 0x7fc9b8103340 2024-04-17T07:13:23.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9bee00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8112de0 con 0x7fc9b8102980 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9be400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b8108980 0x7fc9b8071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9bd000700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9b8103340 0x7fc9b8071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9bda00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b8102980 0x7fc9b80765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9bd000700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9b8103340 0x7fc9b8071680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:40836/0 (socket says 172.21.15.86:40836) 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9be400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b8108980 0x7fc9b8071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37834/0 (socket says 172.21.15.86:37834) 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.462+0000 7fc9bd000700 1 -- 172.21.15.86:0/1919024843 learned_addr learned my addr 172.21.15.86:0/1919024843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.463+0000 7fc9be400700 1 -- 172.21.15.86:0/1919024843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b8102980 msgr2=0x7fc9b80765f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.463+0000 7fc9be400700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9b8102980 0x7fc9b80765f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.463+0000 7fc9be400700 1 -- 172.21.15.86:0/1919024843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9b8103340 msgr2=0x7fc9b8071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.463+0000 7fc9be400700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9b8103340 0x7fc9b8071680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.463+0000 7fc9be400700 1 -- 172.21.15.86:0/1919024843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b80722f0 con 0x7fc9b8108980 2024-04-17T07:13:23.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.463+0000 7fc9be400700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b8108980 0x7fc9b8071bc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000d750 tx=0x7fc9b000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:23.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.464+0000 7fc9b6000700 1 -- 172.21.15.86:0/1919024843 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0014030 con 0x7fc9b8108980 2024-04-17T07:13:23.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.464+0000 7fc9b6000700 1 -- 172.21.15.86:0/1919024843 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9b00044d0 con 0x7fc9b8108980 2024-04-17T07:13:23.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.464+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b81b0120 con 0x7fc9b8108980 2024-04-17T07:13:23.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.464+0000 7fc9b6000700 1 -- 172.21.15.86:0/1919024843 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0005420 con 0x7fc9b8108980 2024-04-17T07:13:23.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.464+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b81b05d0 con 0x7fc9b8108980 2024-04-17T07:13:23.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.465+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9b80631e0 con 0x7fc9b8108980 2024-04-17T07:13:23.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.467+0000 7fc9b6000700 1 -- 172.21.15.86:0/1919024843 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc9b00055c0 con 0x7fc9b8108980 2024-04-17T07:13:23.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.469+0000 7fc9b6000700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9a00a3d90 0x7fc9a00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:23.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.469+0000 7fc9b6000700 1 -- 172.21.15.86:0/1919024843 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc9b00d0520 con 0x7fc9b8108980 2024-04-17T07:13:23.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.469+0000 7fc9bda00700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9a00a3d90 0x7fc9a00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:23.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.469+0000 7fc9b6000700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] conn(0x7fc9a00aeef0 0x7fc9a00b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:23.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.469+0000 7fc9b6000700 1 -- 172.21.15.86:0/1919024843 --> [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9b0021e50 con 0x7fc9a00aeef0 2024-04-17T07:13:23.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.469+0000 7fc9b6000700 1 -- 172.21.15.86:0/1919024843 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fc9b00d08d0 con 0x7fc9b8108980 2024-04-17T07:13:23.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.470+0000 7fc9bd000700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] conn(0x7fc9a00aeef0 0x7fc9a00b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:23.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.470+0000 7fc9bda00700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9a00a3d90 0x7fc9a00a6250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac003f10 tx=0x7fc9ac009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:23.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.471+0000 7fc9bd000700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] conn(0x7fc9a00aeef0 0x7fc9a00b1390 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:23.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.472+0000 7fc9b6000700 1 -- 172.21.15.86:0/1919024843 <== osd.6 v2:172.21.15.86:6818/994878330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9b0021e50 con 0x7fc9a00aeef0 2024-04-17T07:13:23.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.495+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 --> [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc9b8063660 con 0x7fc9a00aeef0 2024-04-17T07:13:23.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.495+0000 7fc9b6000700 1 -- 172.21.15.86:0/1919024843 <== osd.6 v2:172.21.15.86:6818/994878330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc9b8063660 con 0x7fc9a00aeef0 2024-04-17T07:13:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.496+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] conn(0x7fc9a00aeef0 msgr2=0x7fc9a00b1390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.496+0000 7fc9bee00700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] conn(0x7fc9a00aeef0 0x7fc9a00b1390 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.496+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9a00a3d90 msgr2=0x7fc9a00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.496+0000 7fc9bee00700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9a00a3d90 0x7fc9a00a6250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac003f10 tx=0x7fc9ac009180 comp rx=0 tx=0).stop 2024-04-17T07:13:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.496+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b8108980 msgr2=0x7fc9b8071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.496+0000 7fc9bee00700 1 --2- 172.21.15.86:0/1919024843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9b8108980 0x7fc9b8071bc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000d750 tx=0x7fc9b000da60 comp rx=0 tx=0).stop 2024-04-17T07:13:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.496+0000 7fc9be400700 1 -- 172.21.15.86:0/1919024843 reap_dead start 2024-04-17T07:13:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.496+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 shutdown_connections 2024-04-17T07:13:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.497+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 >> 172.21.15.86:0/1919024843 conn(0x7fc9b80fe4d0 msgr2=0x7fc9b80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.497+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 shutdown_connections 2024-04-17T07:13:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:23.497+0000 7fc9bee00700 1 -- 172.21.15.86:0/1919024843 wait complete. 2024-04-17T07:13:23.506 INFO:teuthology.orchestra.run.smithi086.stdout:55834574948 2024-04-17T07:13:23.507 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-17T07:13:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.383+0000 7f2d4a400700 1 -- 172.21.15.86:0/2066280970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d4410f220 msgr2=0x7f2d44111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.383+0000 7f2d4a400700 1 --2- 172.21.15.86:0/2066280970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d4410f220 0x7f2d44111630 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00b0a0 tx=0x7f2d3c00b3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.383+0000 7f2d4a400700 1 -- 172.21.15.86:0/2066280970 shutdown_connections 2024-04-17T07:13:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.383+0000 7f2d4a400700 1 --2- 172.21.15.86:0/2066280970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d4410f220 0x7f2d44111630 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.383+0000 7f2d4a400700 1 --2- 172.21.15.86:0/2066280970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d44074e20 0x7f2d44075200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.383+0000 7f2d4a400700 1 --2- 172.21.15.86:0/2066280970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d44075740 0x7f2d4410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.383+0000 7f2d4a400700 1 -- 172.21.15.86:0/2066280970 >> 172.21.15.86:0/2066280970 conn(0x7f2d440fe4d0 msgr2=0x7f2d441008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.383+0000 7f2d4a400700 1 -- 172.21.15.86:0/2066280970 shutdown_connections 2024-04-17T07:13:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.383+0000 7f2d4a400700 1 -- 172.21.15.86:0/2066280970 wait complete. 2024-04-17T07:13:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.385+0000 7f2d4a400700 1 Processor -- start 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.385+0000 7f2d4a400700 1 -- start start 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d4a400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d44074e20 0x7f2d4419b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d4a400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d44075740 0x7f2d4419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d4a400700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4410f220 0x7f2d4419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d4a400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44114d90 con 0x7f2d44074e20 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d4a400700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44114c10 con 0x7f2d4410f220 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d4a400700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44114f10 con 0x7f2d44075740 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d49000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d44074e20 0x7f2d4419b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d49000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d44074e20 0x7f2d4419b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37802/0 (socket says 172.21.15.86:37802) 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d49000700 1 -- 172.21.15.86:0/3373922607 learned_addr learned my addr 172.21.15.86:0/3373922607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d43e00700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d44075740 0x7f2d4419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.386+0000 7f2d49a00700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4410f220 0x7f2d4419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.387+0000 7f2d49000700 1 -- 172.21.15.86:0/3373922607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d44075740 msgr2=0x7f2d4419b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.387+0000 7f2d49000700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d44075740 0x7f2d4419b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.387+0000 7f2d49000700 1 -- 172.21.15.86:0/3373922607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4410f220 msgr2=0x7f2d4419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.387+0000 7f2d49000700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4410f220 0x7f2d4419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.387+0000 7f2d49000700 1 -- 172.21.15.86:0/3373922607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d441a0230 con 0x7f2d44074e20 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.387+0000 7f2d49000700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d44074e20 0x7f2d4419b220 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800c920 tx=0x7f2d3800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.387+0000 7f2d41600700 1 -- 172.21.15.86:0/3373922607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38007b00 con 0x7f2d44074e20 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.387+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d441a0520 con 0x7f2d44074e20 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.388+0000 7f2d41600700 1 -- 172.21.15.86:0/3373922607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d38004ce0 con 0x7f2d44074e20 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.388+0000 7f2d41600700 1 -- 172.21.15.86:0/3373922607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d380054b0 con 0x7f2d44074e20 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.388+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d441a7da0 con 0x7f2d44074e20 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.389+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d440631e0 con 0x7f2d44074e20 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.391+0000 7f2d41600700 1 -- 172.21.15.86:0/3373922607 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2d38004750 con 0x7f2d44074e20 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.393+0000 7f2d41600700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d2c0a3d10 0x7f2d2c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.393+0000 7f2d41600700 1 -- 172.21.15.86:0/3373922607 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2d380d08c0 con 0x7f2d44074e20 2024-04-17T07:13:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.394+0000 7f2d43e00700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d2c0a3d10 0x7f2d2c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.394+0000 7f2d41600700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] conn(0x7f2d2c0aee70 0x7f2d2c0b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.394+0000 7f2d41600700 1 -- 172.21.15.86:0/3373922607 --> [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d2c0b19e0 con 0x7f2d2c0aee70 2024-04-17T07:13:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.394+0000 7f2d41600700 1 -- 172.21.15.86:0/3373922607 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f2d380d0c70 con 0x7f2d44074e20 2024-04-17T07:13:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.394+0000 7f2d43e00700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d2c0a3d10 0x7f2d2c0a61d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d44110560 tx=0x7f2d3000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.395+0000 7f2d49a00700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] conn(0x7f2d2c0aee70 0x7f2d2c0b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.396+0000 7f2d49a00700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] conn(0x7f2d2c0aee70 0x7f2d2c0b1310 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.398+0000 7f2d41600700 1 -- 172.21.15.86:0/3373922607 <== osd.5 v2:172.21.15.103:6808/3545437589 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d2c0b19e0 con 0x7f2d2c0aee70 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.416+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 --> [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d44063660 con 0x7f2d2c0aee70 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.417+0000 7f2d41600700 1 -- 172.21.15.86:0/3373922607 <== osd.5 v2:172.21.15.103:6808/3545437589 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2d44063660 con 0x7f2d2c0aee70 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 >> [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] conn(0x7f2d2c0aee70 msgr2=0x7f2d2c0b1310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d4a400700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] conn(0x7f2d2c0aee70 0x7f2d2c0b1310 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d2c0a3d10 msgr2=0x7f2d2c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d4a400700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d2c0a3d10 0x7f2d2c0a61d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d44110560 tx=0x7f2d3000b410 comp rx=0 tx=0).stop 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d44074e20 msgr2=0x7f2d4419b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d4a400700 1 --2- 172.21.15.86:0/3373922607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d44074e20 0x7f2d4419b220 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800c920 tx=0x7f2d3800cc30 comp rx=0 tx=0).stop 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d49a00700 1 -- 172.21.15.86:0/3373922607 reap_dead start 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 shutdown_connections 2024-04-17T07:13:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 >> 172.21.15.86:0/3373922607 conn(0x7f2d440fe4d0 msgr2=0x7f2d440ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 shutdown_connections 2024-04-17T07:13:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:21.418+0000 7f2d4a400700 1 -- 172.21.15.86:0/3373922607 wait complete. 2024-04-17T07:13:24.084 INFO:teuthology.orchestra.run.smithi086.stdout:287762808879 2024-04-17T07:13:24.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-17T07:13:24.401 INFO:teuthology.orchestra.run.smithi086.stdout:163208757323 2024-04-17T07:13:24.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-17T07:13:24.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:24 smithi086 ceph-mon[31733]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:13:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:24 smithi098 ceph-mon[35418]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:13:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:24 smithi103 ceph-mon[35495]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:13:25.253 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:25 smithi086 ceph-mon[31733]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:25.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:25 smithi098 ceph-mon[35418]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:25.514 INFO:teuthology.orchestra.run.smithi086.stdout:253403070517 2024-04-17T07:13:25.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-17T07:13:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:25 smithi103 ceph-mon[35495]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.253+0000 7f347f200700 1 -- 172.21.15.86:0/3711409349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3480103320 msgr2=0x7f348010f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.253+0000 7f347f200700 1 --2- 172.21.15.86:0/3711409349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3480103320 0x7f348010f830 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f347800b0a0 tx=0x7f347800b3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.253+0000 7f347f200700 1 -- 172.21.15.86:0/3711409349 shutdown_connections 2024-04-17T07:13:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.253+0000 7f347f200700 1 --2- 172.21.15.86:0/3711409349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3480103320 0x7f348010f830 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.253+0000 7f347f200700 1 --2- 172.21.15.86:0/3711409349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480108960 0x7f3480108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.253+0000 7f347f200700 1 --2- 172.21.15.86:0/3711409349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3480102960 0x7f3480102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.253+0000 7f347f200700 1 -- 172.21.15.86:0/3711409349 >> 172.21.15.86:0/3711409349 conn(0x7f34800fe470 msgr2=0x7f3480100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.253+0000 7f347f200700 1 -- 172.21.15.86:0/3711409349 shutdown_connections 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.253+0000 7f347f200700 1 -- 172.21.15.86:0/3711409349 wait complete. 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.254+0000 7f347f200700 1 Processor -- start 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.254+0000 7f347f200700 1 -- start start 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347f200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480102960 0x7f348019b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347f200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3480103320 0x7f348019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347f200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3480108960 0x7f348019f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347f200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3480112ba0 con 0x7f3480108960 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347f200700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3480112a20 con 0x7f3480102960 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347f200700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3480112d20 con 0x7f3480103320 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3480108960 0x7f348019f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3480108960 0x7f348019f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37866/0 (socket says 172.21.15.86:37866) 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347e800700 1 -- 172.21.15.86:0/1320109072 learned_addr learned my addr 172.21.15.86:0/1320109072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347de00700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480102960 0x7f348019b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.255+0000 7f347d400700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3480103320 0x7f348019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f347e800700 1 -- 172.21.15.86:0/1320109072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3480103320 msgr2=0x7f348019b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f347e800700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3480103320 0x7f348019b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f347e800700 1 -- 172.21.15.86:0/1320109072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480102960 msgr2=0x7f348019b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f347e800700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480102960 0x7f348019b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f347e800700 1 -- 172.21.15.86:0/1320109072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34801a00e0 con 0x7f3480108960 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f347de00700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480102960 0x7f348019b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f347e800700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3480108960 0x7f348019f9a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f347800b070 tx=0x7f347800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f3476a00700 1 -- 172.21.15.86:0/1320109072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478021070 con 0x7f3480108960 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f3476a00700 1 -- 172.21.15.86:0/1320109072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3478007cf0 con 0x7f3480108960 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34801a0370 con 0x7f3480108960 2024-04-17T07:13:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.256+0000 7f3476a00700 1 -- 172.21.15.86:0/1320109072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478005440 con 0x7f3480108960 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.257+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34801a7c50 con 0x7f3480108960 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.257+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34800ffbb0 con 0x7f3480108960 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.259+0000 7f3476a00700 1 -- 172.21.15.86:0/1320109072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3478024070 con 0x7f3480108960 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.261+0000 7f3476a00700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34640a3cd0 0x7f34640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.261+0000 7f3476a00700 1 -- 172.21.15.86:0/1320109072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f347800f020 con 0x7f3480108960 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.261+0000 7f347de00700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34640a3cd0 0x7f34640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.262+0000 7f3476a00700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] conn(0x7f34640aee30 0x7f34640b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.262+0000 7f3476a00700 1 -- 172.21.15.86:0/1320109072 --> [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f347800cbd0 con 0x7f34640aee30 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.262+0000 7f3476a00700 1 -- 172.21.15.86:0/1320109072 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f34780d2730 con 0x7f3480108960 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.262+0000 7f347de00700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34640a3cd0 0x7f34640a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3470005fd0 tx=0x7f347000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.262+0000 7f347d400700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] conn(0x7f34640aee30 0x7f34640b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.263+0000 7f347d400700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] conn(0x7f34640aee30 0x7f34640b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.265+0000 7f3476a00700 1 -- 172.21.15.86:0/1320109072 <== osd.11 v2:172.21.15.103:6824/1742700108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f347800cbd0 con 0x7f34640aee30 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.283+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 --> [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f34801040a0 con 0x7f34640aee30 2024-04-17T07:13:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.284+0000 7f3476a00700 1 -- 172.21.15.86:0/1320109072 <== osd.11 v2:172.21.15.103:6824/1742700108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f34801040a0 con 0x7f34640aee30 2024-04-17T07:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.285+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 >> [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] conn(0x7f34640aee30 msgr2=0x7f34640b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.285+0000 7f347f200700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] conn(0x7f34640aee30 0x7f34640b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.285+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34640a3cd0 msgr2=0x7f34640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.285+0000 7f347f200700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34640a3cd0 0x7f34640a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3470005fd0 tx=0x7f347000a5e0 comp rx=0 tx=0).stop 2024-04-17T07:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.285+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3480108960 msgr2=0x7f348019f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.285+0000 7f347f200700 1 --2- 172.21.15.86:0/1320109072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3480108960 0x7f348019f9a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f347800b070 tx=0x7f347800cf00 comp rx=0 tx=0).stop 2024-04-17T07:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.285+0000 7f347e800700 1 -- 172.21.15.86:0/1320109072 reap_dead start 2024-04-17T07:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.285+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 shutdown_connections 2024-04-17T07:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.285+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 >> 172.21.15.86:0/1320109072 conn(0x7f34800fe470 msgr2=0x7f348010e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.285+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 shutdown_connections 2024-04-17T07:13:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:25.286+0000 7f347f200700 1 -- 172.21.15.86:0/1320109072 wait complete. 2024-04-17T07:13:26.350 INFO:teuthology.orchestra.run.smithi086.stdout:343597383714 2024-04-17T07:13:26.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-17T07:13:26.943 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:26.973 INFO:teuthology.orchestra.run.smithi086.stdout:197568495685 2024-04-17T07:13:26.973 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-17T07:13:27.018 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:27.086 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:27.202 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:27.209 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:28.384 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:28 smithi086 ceph-mon[31733]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:13:28.384 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-17T07:13:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:28 smithi098 ceph-mon[35418]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:13:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-17T07:13:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:28 smithi103 ceph-mon[35495]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:13:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-17T07:13:28.633 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:29.428 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:29 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-17T07:13:29.429 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:29 smithi086 ceph-mon[31733]: osdmap e92: 12 total, 12 up, 12 in 2024-04-17T07:13:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:29 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-17T07:13:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:29 smithi098 ceph-mon[35418]: osdmap e92: 12 total, 12 up, 12 in 2024-04-17T07:13:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:29 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-17T07:13:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:29 smithi103 ceph-mon[35495]: osdmap e92: 12 total, 12 up, 12 in 2024-04-17T07:13:30.293 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:30 smithi086 ceph-mon[31733]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:13:30.293 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:30 smithi086 ceph-mon[31733]: osdmap e93: 12 total, 12 up, 12 in 2024-04-17T07:13:30.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:30 smithi098 ceph-mon[35418]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:13:30.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:30 smithi098 ceph-mon[35418]: osdmap e93: 12 total, 12 up, 12 in 2024-04-17T07:13:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:30 smithi103 ceph-mon[35495]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:13:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:30 smithi103 ceph-mon[35495]: osdmap e93: 12 total, 12 up, 12 in 2024-04-17T07:13:31.496 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:32.492 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:32 smithi086 ceph-mon[31733]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-17T07:13:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:32 smithi098 ceph-mon[35418]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-17T07:13:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:32 smithi103 ceph-mon[35495]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-17T07:13:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:33.663 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:34.302 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:34 smithi086 ceph-mon[31733]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-17T07:13:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:34 smithi098 ceph-mon[35418]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-17T07:13:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:34 smithi103 ceph-mon[35495]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-17T07:13:36.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:36 smithi086 ceph-mon[31733]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-04-17T07:13:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:36 smithi098 ceph-mon[35418]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-04-17T07:13:36.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:36 smithi103 ceph-mon[35495]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-04-17T07:13:38.194 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:38 smithi086 ceph-mon[31733]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-04-17T07:13:38.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:38 smithi098 ceph-mon[35418]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-04-17T07:13:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:38 smithi103 ceph-mon[35495]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2024-04-17T07:13:38.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.778+0000 7fcfdba00700 1 -- 172.21.15.86:0/1559673871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd4105490 msgr2=0x7fcfd4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:38.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.778+0000 7fcfdba00700 1 --2- 172.21.15.86:0/1559673871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd4105490 0x7fcfd4105920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00b0a0 tx=0x7fcfcc00b3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:38.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.778+0000 7fcfdba00700 1 -- 172.21.15.86:0/1559673871 shutdown_connections 2024-04-17T07:13:38.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.778+0000 7fcfdba00700 1 --2- 172.21.15.86:0/1559673871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd4105490 0x7fcfd4105920 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:38.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.778+0000 7fcfdba00700 1 --2- 172.21.15.86:0/1559673871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd4107980 0x7fcfd406a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.778+0000 7fcfdba00700 1 --2- 172.21.15.86:0/1559673871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd406aca0 0x7fcfd406b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.778+0000 7fcfdba00700 1 -- 172.21.15.86:0/1559673871 >> 172.21.15.86:0/1559673871 conn(0x7fcfd4078900 msgr2=0x7fcfd4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.778+0000 7fcfdba00700 1 -- 172.21.15.86:0/1559673871 shutdown_connections 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.779+0000 7fcfdba00700 1 -- 172.21.15.86:0/1559673871 wait complete. 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.780+0000 7fcfdba00700 1 Processor -- start 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.780+0000 7fcfdba00700 1 -- start start 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.780+0000 7fcfdba00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd406aca0 0x7fcfd419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.780+0000 7fcfdba00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd4105490 0x7fcfd419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.780+0000 7fcfdba00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd4107980 0x7fcfd41a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.780+0000 7fcfdba00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4119240 con 0x7fcfd406aca0 2024-04-17T07:13:38.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.780+0000 7fcfdba00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41190c0 con 0x7fcfd4105490 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.780+0000 7fcfdba00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41193c0 con 0x7fcfd4107980 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.781+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd4105490 0x7fcfd419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.781+0000 7fcfd9000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd406aca0 0x7fcfd419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.781+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd4105490 0x7fcfd419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:35626/0 (socket says 172.21.15.86:35626) 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.781+0000 7fcfd3e00700 1 -- 172.21.15.86:0/606659328 learned_addr learned my addr 172.21.15.86:0/606659328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.781+0000 7fcfd9a00700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd4107980 0x7fcfd41a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.781+0000 7fcfd9000700 1 -- 172.21.15.86:0/606659328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd4107980 msgr2=0x7fcfd41a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.781+0000 7fcfd9000700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd4107980 0x7fcfd41a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.781+0000 7fcfd9000700 1 -- 172.21.15.86:0/606659328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd4105490 msgr2=0x7fcfd419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.781+0000 7fcfd9000700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd4105490 0x7fcfd419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.781+0000 7fcfd9000700 1 -- 172.21.15.86:0/606659328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd41a4700 con 0x7fcfd406aca0 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.782+0000 7fcfd9000700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd406aca0 0x7fcfd419f6f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800c970 tx=0x7fcfc800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:38.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.782+0000 7fcfd1600700 1 -- 172.21.15.86:0/606659328 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800ce60 con 0x7fcfd406aca0 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.782+0000 7fcfd1600700 1 -- 172.21.15.86:0/606659328 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfc8004830 con 0x7fcfd406aca0 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.782+0000 7fcfd1600700 1 -- 172.21.15.86:0/606659328 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8005500 con 0x7fcfd406aca0 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.782+0000 7fcfdba00700 1 -- 172.21.15.86:0/606659328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd41a49f0 con 0x7fcfd406aca0 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.782+0000 7fcfdba00700 1 -- 172.21.15.86:0/606659328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd41ac270 con 0x7fcfd406aca0 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.783+0000 7fcfdba00700 1 -- 172.21.15.86:0/606659328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd4069740 con 0x7fcfd406aca0 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.785+0000 7fcfd1600700 1 -- 172.21.15.86:0/606659328 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcfc80049e0 con 0x7fcfd406aca0 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.787+0000 7fcfd1600700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcfbc0a3d50 0x7fcfbc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.787+0000 7fcfd1600700 1 -- 172.21.15.86:0/606659328 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcfc80d0720 con 0x7fcfd406aca0 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.787+0000 7fcfd3e00700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcfbc0a3d50 0x7fcfbc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.788+0000 7fcfd3e00700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcfbc0a3d50 0x7fcfbc0a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4078fb0 tx=0x7fcfc0009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:38.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:38.792+0000 7fcfd1600700 1 -- 172.21.15.86:0/606659328 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcfc8014030 con 0x7fcfd406aca0 2024-04-17T07:13:39.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.021+0000 7fcfdba00700 1 -- 172.21.15.86:0/606659328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fcfd41a0a50 con 0x7fcfd406aca0 2024-04-17T07:13:39.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.022+0000 7fcfd1600700 1 -- 172.21.15.86:0/606659328 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcfc809cbc0 con 0x7fcfd406aca0 2024-04-17T07:13:39.022 INFO:teuthology.orchestra.run.smithi086.stdout:34359738481 2024-04-17T07:13:39.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 -- 172.21.15.86:0/606659328 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcfbc0a3d50 msgr2=0x7fcfbc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:39.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcfbc0a3d50 0x7fcfbc0a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4078fb0 tx=0x7fcfc0009290 comp rx=0 tx=0).stop 2024-04-17T07:13:39.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 -- 172.21.15.86:0/606659328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd406aca0 msgr2=0x7fcfd419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:39.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd406aca0 0x7fcfd419f6f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800c970 tx=0x7fcfc800cc80 comp rx=0 tx=0).stop 2024-04-17T07:13:39.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 -- 172.21.15.86:0/606659328 shutdown_connections 2024-04-17T07:13:39.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcfbc0a3d50 0x7fcfbc0a6210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:39.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd4107980 0x7fcfd41a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:39.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcfd406aca0 0x7fcfd419f6f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:39.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 --2- 172.21.15.86:0/606659328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd4105490 0x7fcfd419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:39.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 -- 172.21.15.86:0/606659328 >> 172.21.15.86:0/606659328 conn(0x7fcfd4078900 msgr2=0x7fcfd4104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:39.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 -- 172.21.15.86:0/606659328 shutdown_connections 2024-04-17T07:13:39.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:39.025+0000 7fcfdba00700 1 -- 172.21.15.86:0/606659328 wait complete. 2024-04-17T07:13:39.478 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:39 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/606659328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-17T07:13:39.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:39 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/606659328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-17T07:13:39.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:39 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/606659328' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-17T07:13:40.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:40 smithi086 ceph-mon[31733]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-17T07:13:40.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:40 smithi098 ceph-mon[35418]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-17T07:13:40.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:40 smithi103 ceph-mon[35495]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-17T07:13:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.026+0000 7feafea00700 1 -- 172.21.15.86:0/933393121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaf8108a90 msgr2=0x7feaf8108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.026+0000 7feafea00700 1 --2- 172.21.15.86:0/933393121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaf8108a90 0x7feaf8108e70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feae8009990 tx=0x7feae8009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.026+0000 7feafea00700 1 -- 172.21.15.86:0/933393121 shutdown_connections 2024-04-17T07:13:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.026+0000 7feafea00700 1 --2- 172.21.15.86:0/933393121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaf8103460 0x7feaf810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.026+0000 7feafea00700 1 --2- 172.21.15.86:0/933393121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaf8108a90 0x7feaf8108e70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.026+0000 7feafea00700 1 --2- 172.21.15.86:0/933393121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf8102aa0 0x7feaf8102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.026+0000 7feafea00700 1 -- 172.21.15.86:0/933393121 >> 172.21.15.86:0/933393121 conn(0x7feaf8079c00 msgr2=0x7feaf807a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.026+0000 7feafea00700 1 -- 172.21.15.86:0/933393121 shutdown_connections 2024-04-17T07:13:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.026+0000 7feafea00700 1 -- 172.21.15.86:0/933393121 wait complete. 2024-04-17T07:13:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.028+0000 7feafea00700 1 Processor -- start 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.028+0000 7feafea00700 1 -- start start 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.029+0000 7feafea00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaf8102aa0 0x7feaf81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.029+0000 7feafea00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaf8103460 0x7feaf8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.029+0000 7feafea00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf8108a90 0x7feaf819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.029+0000 7feafea00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8112c70 con 0x7feaf8102aa0 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.029+0000 7feafea00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8112af0 con 0x7feaf8103460 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.029+0000 7feafea00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8112df0 con 0x7feaf8108a90 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.029+0000 7feaf7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaf8102aa0 0x7feaf81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.029+0000 7feaf7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaf8102aa0 0x7feaf81990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34060/0 (socket says 172.21.15.86:34060) 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.029+0000 7feaf7e00700 1 -- 172.21.15.86:0/369855339 learned_addr learned my addr 172.21.15.86:0/369855339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.030+0000 7feaf7400700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaf8103460 0x7feaf8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.030+0000 7feafca00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf8108a90 0x7feaf819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.030+0000 7feaf7e00700 1 -- 172.21.15.86:0/369855339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf8108a90 msgr2=0x7feaf819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.030+0000 7feaf7e00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf8108a90 0x7feaf819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.030+0000 7feaf7e00700 1 -- 172.21.15.86:0/369855339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaf8103460 msgr2=0x7feaf8199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.030+0000 7feaf7e00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaf8103460 0x7feaf8199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.030+0000 7feaf7e00700 1 -- 172.21.15.86:0/369855339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf819e0d0 con 0x7feaf8102aa0 2024-04-17T07:13:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.030+0000 7feafca00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf8108a90 0x7feaf819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:13:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.030+0000 7feaf7e00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaf8102aa0 0x7feaf81990c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feae8005910 tx=0x7feae80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.031+0000 7feaf4c00700 1 -- 172.21.15.86:0/369855339 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8027070 con 0x7feaf8102aa0 2024-04-17T07:13:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.031+0000 7feaf4c00700 1 -- 172.21.15.86:0/369855339 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feae8003f10 con 0x7feaf8102aa0 2024-04-17T07:13:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.031+0000 7feafea00700 1 -- 172.21.15.86:0/369855339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf819e360 con 0x7feaf8102aa0 2024-04-17T07:13:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.031+0000 7feaf4c00700 1 -- 172.21.15.86:0/369855339 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8022d00 con 0x7feaf8102aa0 2024-04-17T07:13:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.031+0000 7feafea00700 1 -- 172.21.15.86:0/369855339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf81a5c40 con 0x7feaf8102aa0 2024-04-17T07:13:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.033+0000 7feafea00700 1 -- 172.21.15.86:0/369855339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf8069740 con 0x7feaf8102aa0 2024-04-17T07:13:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.034+0000 7feaf4c00700 1 -- 172.21.15.86:0/369855339 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feae802a070 con 0x7feaf8102aa0 2024-04-17T07:13:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.037+0000 7feaf4c00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feadc0a3d50 0x7feadc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.037+0000 7feaf4c00700 1 -- 172.21.15.86:0/369855339 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feae80d1700 con 0x7feaf8102aa0 2024-04-17T07:13:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.037+0000 7feaf7400700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feadc0a3d50 0x7feadc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.038+0000 7feaf7400700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feadc0a3d50 0x7feadc0a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feaf807a190 tx=0x7feae400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.040+0000 7feaf4c00700 1 -- 172.21.15.86:0/369855339 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7feae805eff0 con 0x7feaf8102aa0 2024-04-17T07:13:41.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.292+0000 7feafea00700 1 -- 172.21.15.86:0/369855339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7feaf802df90 con 0x7feaf8102aa0 2024-04-17T07:13:41.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.293+0000 7feaf4c00700 1 -- 172.21.15.86:0/369855339 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feae8013070 con 0x7feaf8102aa0 2024-04-17T07:13:41.293 INFO:teuthology.orchestra.run.smithi086.stdout:77309411426 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.295+0000 7feafea00700 1 -- 172.21.15.86:0/369855339 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feadc0a3d50 msgr2=0x7feadc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.295+0000 7feafea00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feadc0a3d50 0x7feadc0a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feaf807a190 tx=0x7feae400a380 comp rx=0 tx=0).stop 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.295+0000 7feafea00700 1 -- 172.21.15.86:0/369855339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaf8102aa0 msgr2=0x7feaf81990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.295+0000 7feafea00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaf8102aa0 0x7feaf81990c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feae8005910 tx=0x7feae80047b0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.296+0000 7feafea00700 1 -- 172.21.15.86:0/369855339 shutdown_connections 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.296+0000 7feafea00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feadc0a3d50 0x7feadc0a6210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.296+0000 7feafea00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feaf8108a90 0x7feaf819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.296+0000 7feafea00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaf8102aa0 0x7feaf81990c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.296+0000 7feafea00700 1 --2- 172.21.15.86:0/369855339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feaf8103460 0x7feaf8199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.296+0000 7feafea00700 1 -- 172.21.15.86:0/369855339 >> 172.21.15.86:0/369855339 conn(0x7feaf8079c00 msgr2=0x7feaf8101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.296+0000 7feafea00700 1 -- 172.21.15.86:0/369855339 shutdown_connections 2024-04-17T07:13:41.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.296+0000 7feafea00700 1 -- 172.21.15.86:0/369855339 wait complete. 2024-04-17T07:13:41.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:41 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3985424053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-17T07:13:41.406 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:41 smithi086 ceph-mon[31733]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-17T07:13:41.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:41 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3985424053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-17T07:13:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:41 smithi098 ceph-mon[35418]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-17T07:13:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:41 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3985424053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-17T07:13:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:41 smithi103 ceph-mon[35495]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-17T07:13:41.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.157+0000 7f0199a00700 1 -- 172.21.15.86:0/1902441147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0194108a40 msgr2=0x7f0194108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:41.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.157+0000 7f0199a00700 1 --2- 172.21.15.86:0/1902441147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0194108a40 0x7f0194108e20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0184009990 tx=0x7f0184009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.158+0000 7f0199a00700 1 -- 172.21.15.86:0/1902441147 shutdown_connections 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.158+0000 7f0199a00700 1 --2- 172.21.15.86:0/1902441147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019406b100 0x7f019410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.158+0000 7f0199a00700 1 --2- 172.21.15.86:0/1902441147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0194108a40 0x7f0194108e20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.158+0000 7f0199a00700 1 --2- 172.21.15.86:0/1902441147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f019406a760 0x7f019406abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.158+0000 7f0199a00700 1 -- 172.21.15.86:0/1902441147 >> 172.21.15.86:0/1902441147 conn(0x7f0194078900 msgr2=0x7f0194078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.158+0000 7f0199a00700 1 -- 172.21.15.86:0/1902441147 shutdown_connections 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.158+0000 7f0199a00700 1 -- 172.21.15.86:0/1902441147 wait complete. 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.159+0000 7f0199a00700 1 Processor -- start 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.159+0000 7f0199a00700 1 -- start start 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.159+0000 7f0199a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f019406a760 0x7f0194101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.159+0000 7f0199a00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019406b100 0x7f0194101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.159+0000 7f0199a00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0194108a40 0x7f01941024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.159+0000 7f0199a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0194112c20 con 0x7f019406a760 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.159+0000 7f0199a00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0194112aa0 con 0x7f0194108a40 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.159+0000 7f0199a00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0194112da0 con 0x7f019406b100 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0192a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f019406a760 0x7f0194101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0192a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f019406a760 0x7f0194101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34080/0 (socket says 172.21.15.86:34080) 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0192a00700 1 -- 172.21.15.86:0/3369847679 learned_addr learned my addr 172.21.15.86:0/3369847679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0193400700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0194108a40 0x7f01941024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0192a00700 1 -- 172.21.15.86:0/3369847679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019406b100 msgr2=0x7f0194101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0192000700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019406b100 0x7f0194101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0192a00700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019406b100 0x7f0194101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0192a00700 1 -- 172.21.15.86:0/3369847679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0194108a40 msgr2=0x7f01941024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0192a00700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0194108a40 0x7f01941024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0192a00700 1 -- 172.21.15.86:0/3369847679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01941a76e0 con 0x7f019406a760 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0193400700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0194108a40 0x7f01941024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.160+0000 7f0192000700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019406b100 0x7f0194101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.161+0000 7f0192a00700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f019406a760 0x7f0194101a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0184000c00 tx=0x7f01840057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.161+0000 7f018b400700 1 -- 172.21.15.86:0/3369847679 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184027070 con 0x7f019406a760 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.161+0000 7f0199a00700 1 -- 172.21.15.86:0/3369847679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01941a7970 con 0x7f019406a760 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.161+0000 7f018b400700 1 -- 172.21.15.86:0/3369847679 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0184004010 con 0x7f019406a760 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.161+0000 7f018b400700 1 -- 172.21.15.86:0/3369847679 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0184022d90 con 0x7f019406a760 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.161+0000 7f0199a00700 1 -- 172.21.15.86:0/3369847679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01941a7dc0 con 0x7f019406a760 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.162+0000 7f0199a00700 1 -- 172.21.15.86:0/3369847679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0194069740 con 0x7f019406a760 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.164+0000 7f018b400700 1 -- 172.21.15.86:0/3369847679 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f018402a070 con 0x7f019406a760 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.166+0000 7f018b400700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f01780a3cd0 0x7f01780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.167+0000 7f018b400700 1 -- 172.21.15.86:0/3369847679 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f01840d1810 con 0x7f019406a760 2024-04-17T07:13:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.167+0000 7f0192000700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f01780a3cd0 0x7f01780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:41.745 INFO:teuthology.orchestra.run.smithi086.stdout:55834574954 2024-04-17T07:13:41.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.168+0000 7f0192000700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f01780a3cd0 0x7f01780a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0180005fd0 tx=0x7f0180005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:41.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.171+0000 7f018b400700 1 -- 172.21.15.86:0/3369847679 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f018405eff0 con 0x7f019406a760 2024-04-17T07:13:41.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.406+0000 7f0199a00700 1 -- 172.21.15.86:0/3369847679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f019402e1f0 con 0x7f019406a760 2024-04-17T07:13:41.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.406+0000 7f018b400700 1 -- 172.21.15.86:0/3369847679 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0184099020 con 0x7f019406a760 2024-04-17T07:13:41.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.409+0000 7f0199a00700 1 -- 172.21.15.86:0/3369847679 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f01780a3cd0 msgr2=0x7f01780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.409+0000 7f0199a00700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f01780a3cd0 0x7f01780a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0180005fd0 tx=0x7f0180005ee0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.409+0000 7f0199a00700 1 -- 172.21.15.86:0/3369847679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f019406a760 msgr2=0x7f0194101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.409+0000 7f0199a00700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f019406a760 0x7f0194101a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0184000c00 tx=0x7f01840057c0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.409+0000 7f0199a00700 1 -- 172.21.15.86:0/3369847679 shutdown_connections 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.409+0000 7f0199a00700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f01780a3cd0 0x7f01780a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.409+0000 7f0199a00700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0194108a40 0x7f01941024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.409+0000 7f0199a00700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f019406a760 0x7f0194101a20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.409+0000 7f0199a00700 1 --2- 172.21.15.86:0/3369847679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f019406b100 0x7f0194101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.409+0000 7f0199a00700 1 -- 172.21.15.86:0/3369847679 >> 172.21.15.86:0/3369847679 conn(0x7f0194078900 msgr2=0x7f01940ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.410+0000 7f0199a00700 1 -- 172.21.15.86:0/3369847679 shutdown_connections 2024-04-17T07:13:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:41.410+0000 7f0199a00700 1 -- 172.21.15.86:0/3369847679 wait complete. 2024-04-17T07:13:42.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:42 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/369855339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-17T07:13:42.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:42 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3369847679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-17T07:13:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:42 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/369855339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-17T07:13:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:42 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3369847679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-17T07:13:42.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:42 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/369855339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-17T07:13:42.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:42 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3369847679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-17T07:13:43.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.782+0000 7f49fbe00700 1 -- 172.21.15.86:0/2944504736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc108f90 msgr2=0x7f49fc109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:43.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.782+0000 7f49fbe00700 1 --2- 172.21.15.86:0/2944504736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc108f90 0x7f49fc109410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009990 tx=0x7f49e8009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.782+0000 7f49fbe00700 1 -- 172.21.15.86:0/2944504736 shutdown_connections 2024-04-17T07:13:43.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.782+0000 7f49fbe00700 1 --2- 172.21.15.86:0/2944504736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49fc109950 0x7f49fc115df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.782+0000 7f49fbe00700 1 --2- 172.21.15.86:0/2944504736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49fc10f040 0x7f49fc10f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.782+0000 7f49fbe00700 1 --2- 172.21.15.86:0/2944504736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc108f90 0x7f49fc109410 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.782+0000 7f49fbe00700 1 -- 172.21.15.86:0/2944504736 >> 172.21.15.86:0/2944504736 conn(0x7f49fc0787e0 msgr2=0x7f49fc107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.783+0000 7f49fbe00700 1 -- 172.21.15.86:0/2944504736 shutdown_connections 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.783+0000 7f49fbe00700 1 -- 172.21.15.86:0/2944504736 wait complete. 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.784+0000 7f49fbe00700 1 Processor -- start 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.784+0000 7f49fbe00700 1 -- start start 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.784+0000 7f49fbe00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49fc108f90 0x7f49fc19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.784+0000 7f49fbe00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49fc109950 0x7f49fc19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.784+0000 7f49fbe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc10f040 0x7f49fc1a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.784+0000 7f49fbe00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc119100 con 0x7f49fc10f040 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.784+0000 7f49fbe00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc118f80 con 0x7f49fc109950 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.784+0000 7f49fbe00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc119280 con 0x7f49fc108f90 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.784+0000 7f49fb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc10f040 0x7f49fc1a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:43.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.785+0000 7f49fb400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc10f040 0x7f49fc1a3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34098/0 (socket says 172.21.15.86:34098) 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.785+0000 7f49fb400700 1 -- 172.21.15.86:0/3185287961 learned_addr learned my addr 172.21.15.86:0/3185287961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.785+0000 7f49fa000700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49fc109950 0x7f49fc19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.785+0000 7f49faa00700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49fc108f90 0x7f49fc19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.785+0000 7f49fb400700 1 -- 172.21.15.86:0/3185287961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49fc108f90 msgr2=0x7f49fc19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.785+0000 7f49fb400700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49fc108f90 0x7f49fc19f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.785+0000 7f49fb400700 1 -- 172.21.15.86:0/3185287961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49fc109950 msgr2=0x7f49fc19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.785+0000 7f49fb400700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49fc109950 0x7f49fc19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.785+0000 7f49fb400700 1 -- 172.21.15.86:0/3185287961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc1a4580 con 0x7f49fc10f040 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.785+0000 7f49fb400700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc10f040 0x7f49fc1a3e40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f49f400ada0 tx=0x7f49f400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.786+0000 7f49f3400700 1 -- 172.21.15.86:0/3185287961 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4017020 con 0x7f49fc10f040 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.786+0000 7f49f3400700 1 -- 172.21.15.86:0/3185287961 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49f400ce30 con 0x7f49fc10f040 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.786+0000 7f49f3400700 1 -- 172.21.15.86:0/3185287961 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4005460 con 0x7f49fc10f040 2024-04-17T07:13:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.786+0000 7f49fbe00700 1 -- 172.21.15.86:0/3185287961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc1a4870 con 0x7f49fc10f040 2024-04-17T07:13:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.786+0000 7f49fbe00700 1 -- 172.21.15.86:0/3185287961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc1ac0f0 con 0x7f49fc10f040 2024-04-17T07:13:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.787+0000 7f49fbe00700 1 -- 172.21.15.86:0/3185287961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49fc111390 con 0x7f49fc10f040 2024-04-17T07:13:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.789+0000 7f49f3400700 1 -- 172.21.15.86:0/3185287961 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f49f400c980 con 0x7f49fc10f040 2024-04-17T07:13:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.790+0000 7f49f3400700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f49e00a3d50 0x7f49e00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:43.155 INFO:teuthology.orchestra.run.smithi086.stdout:163208757327 2024-04-17T07:13:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.790+0000 7f49f3400700 1 -- 172.21.15.86:0/3185287961 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f49f40d0180 con 0x7f49fc10f040 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.790+0000 7f49faa00700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f49e00a3d50 0x7f49e00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.791+0000 7f49faa00700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f49e00a3d50 0x7f49e00a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f49ec005950 tx=0x7f49ec0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:42.796+0000 7f49f3400700 1 -- 172.21.15.86:0/3185287961 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f49f4021020 con 0x7f49fc10f040 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.028+0000 7f49fbe00700 1 -- 172.21.15.86:0/3185287961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f49fc1a08d0 con 0x7f49fc10f040 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.029+0000 7f49f3400700 1 -- 172.21.15.86:0/3185287961 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49f409c6b0 con 0x7f49fc10f040 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 -- 172.21.15.86:0/3185287961 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f49e00a3d50 msgr2=0x7f49e00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f49e00a3d50 0x7f49e00a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f49ec005950 tx=0x7f49ec0058e0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 -- 172.21.15.86:0/3185287961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc10f040 msgr2=0x7f49fc1a3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc10f040 0x7f49fc1a3e40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f49f400ada0 tx=0x7f49f400c330 comp rx=0 tx=0).stop 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 -- 172.21.15.86:0/3185287961 shutdown_connections 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f49e00a3d50 0x7f49e00a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49fc10f040 0x7f49fc1a3e40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49fc108f90 0x7f49fc19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 --2- 172.21.15.86:0/3185287961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49fc109950 0x7f49fc19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 -- 172.21.15.86:0/3185287961 >> 172.21.15.86:0/3185287961 conn(0x7f49fc0787e0 msgr2=0x7f49fc079a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:43.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.032+0000 7f49fbe00700 1 -- 172.21.15.86:0/3185287961 shutdown_connections 2024-04-17T07:13:43.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.033+0000 7f49fbe00700 1 -- 172.21.15.86:0/3185287961 wait complete. 2024-04-17T07:13:43.491 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:43 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3185287961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-17T07:13:43.491 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:43 smithi086 ceph-mon[31733]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-17T07:13:43.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:43 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3185287961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-17T07:13:43.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:43 smithi098 ceph-mon[35418]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-17T07:13:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:43 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3185287961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-17T07:13:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:43 smithi103 ceph-mon[35495]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-17T07:13:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.626+0000 7f4efd800700 1 -- 172.21.15.86:0/2213492518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef81092e0 msgr2=0x7f4ef81096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.626+0000 7f4efd800700 1 --2- 172.21.15.86:0/2213492518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef81092e0 0x7f4ef81096c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80099c0 tx=0x7f4ee8009cd0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.626+0000 7f4efd800700 1 -- 172.21.15.86:0/2213492518 shutdown_connections 2024-04-17T07:13:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.626+0000 7f4efd800700 1 --2- 172.21.15.86:0/2213492518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef810e3c0 0x7f4ef8115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.626+0000 7f4efd800700 1 --2- 172.21.15.86:0/2213492518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef81092e0 0x7f4ef81096c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.626+0000 7f4efd800700 1 --2- 172.21.15.86:0/2213492518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef8109c90 0x7f4ef810dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.626+0000 7f4efd800700 1 -- 172.21.15.86:0/2213492518 >> 172.21.15.86:0/2213492518 conn(0x7f4ef8076220 msgr2=0x7f4ef8107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.626+0000 7f4efd800700 1 -- 172.21.15.86:0/2213492518 shutdown_connections 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.626+0000 7f4efd800700 1 -- 172.21.15.86:0/2213492518 wait complete. 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.627+0000 7f4efd800700 1 Processor -- start 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.628+0000 7f4efd800700 1 -- start start 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.628+0000 7f4efd800700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef81092e0 0x7f4ef81a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.628+0000 7f4efd800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef8109c90 0x7f4ef81a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.628+0000 7f4efd800700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef810e3c0 0x7f4ef819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.628+0000 7f4efd800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef81192a0 con 0x7f4ef8109c90 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.628+0000 7f4efd800700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8119120 con 0x7f4ef810e3c0 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.628+0000 7f4efd800700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8119420 con 0x7f4ef81092e0 2024-04-17T07:13:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.628+0000 7f4ef6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef8109c90 0x7f4ef81a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.628+0000 7f4ef6000700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef8109c90 0x7f4ef81a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34038/0 (socket says 172.21.15.86:34038) 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.629+0000 7f4ef6000700 1 -- 172.21.15.86:0/3985424053 learned_addr learned my addr 172.21.15.86:0/3985424053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.629+0000 7f4ef6a00700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef81092e0 0x7f4ef81a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.629+0000 7f4ef7400700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef810e3c0 0x7f4ef819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.629+0000 7f4ef6000700 1 -- 172.21.15.86:0/3985424053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef81092e0 msgr2=0x7f4ef81a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.629+0000 7f4ef6000700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef81092e0 0x7f4ef81a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.629+0000 7f4ef6000700 1 -- 172.21.15.86:0/3985424053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef810e3c0 msgr2=0x7f4ef819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.629+0000 7f4ef6000700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef810e3c0 0x7f4ef819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.629+0000 7f4ef6000700 1 -- 172.21.15.86:0/3985424053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef819fd40 con 0x7f4ef8109c90 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.629+0000 7f4ef6000700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef8109c90 0x7f4ef81a5910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400c8f0 tx=0x7f4ee400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.630+0000 7f4eef400700 1 -- 172.21.15.86:0/3985424053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4007b00 con 0x7f4ef8109c90 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.630+0000 7f4eef400700 1 -- 172.21.15.86:0/3985424053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ee4007ca0 con 0x7f4ef8109c90 2024-04-17T07:13:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.630+0000 7f4efd800700 1 -- 172.21.15.86:0/3985424053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef81a0030 con 0x7f4ef8109c90 2024-04-17T07:13:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.630+0000 7f4eef400700 1 -- 172.21.15.86:0/3985424053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4018660 con 0x7f4ef8109c90 2024-04-17T07:13:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.630+0000 7f4efd800700 1 -- 172.21.15.86:0/3985424053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef81ac2a0 con 0x7f4ef8109c90 2024-04-17T07:13:43.599 INFO:teuthology.orchestra.run.smithi086.stdout:107374182492 2024-04-17T07:13:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.633+0000 7f4efd800700 1 -- 172.21.15.86:0/3985424053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef8069740 con 0x7f4ef8109c90 2024-04-17T07:13:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.633+0000 7f4eef400700 1 -- 172.21.15.86:0/3985424053 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4ee400f450 con 0x7f4ef8109c90 2024-04-17T07:13:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.634+0000 7f4eef400700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4edc0a3d50 0x7f4edc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.635+0000 7f4eef400700 1 -- 172.21.15.86:0/3985424053 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4ee40d0900 con 0x7f4ef8109c90 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.635+0000 7f4ef6a00700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4edc0a3d50 0x7f4edc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.635+0000 7f4ef6a00700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4edc0a3d50 0x7f4edc0a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009fd0 tx=0x7f4ee8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.639+0000 7f4eef400700 1 -- 172.21.15.86:0/3985424053 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4ee4014030 con 0x7f4ef8109c90 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.873+0000 7f4efd800700 1 -- 172.21.15.86:0/3985424053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4ef81a0ac0 con 0x7f4ef8109c90 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.874+0000 7f4eef400700 1 -- 172.21.15.86:0/3985424053 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4ee409cda0 con 0x7f4ef8109c90 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.877+0000 7f4efd800700 1 -- 172.21.15.86:0/3985424053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4edc0a3d50 msgr2=0x7f4edc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.877+0000 7f4efd800700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4edc0a3d50 0x7f4edc0a6210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009fd0 tx=0x7f4ee8005fb0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.877+0000 7f4efd800700 1 -- 172.21.15.86:0/3985424053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef8109c90 msgr2=0x7f4ef81a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.877+0000 7f4efd800700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef8109c90 0x7f4ef81a5910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400c8f0 tx=0x7f4ee400ccb0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.877+0000 7f4efd800700 1 -- 172.21.15.86:0/3985424053 shutdown_connections 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.877+0000 7f4efd800700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4edc0a3d50 0x7f4edc0a6210 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.877+0000 7f4efd800700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ef810e3c0 0x7f4ef819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.877+0000 7f4efd800700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef81092e0 0x7f4ef81a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.877+0000 7f4efd800700 1 --2- 172.21.15.86:0/3985424053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ef8109c90 0x7f4ef81a5910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.877+0000 7f4efd800700 1 -- 172.21.15.86:0/3985424053 >> 172.21.15.86:0/3985424053 conn(0x7f4ef8076220 msgr2=0x7f4ef8077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.878+0000 7f4efd800700 1 -- 172.21.15.86:0/3985424053 shutdown_connections 2024-04-17T07:13:43.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:40.878+0000 7f4efd800700 1 -- 172.21.15.86:0/3985424053 wait complete. 2024-04-17T07:13:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:44 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1099003115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-17T07:13:44.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:44 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1099003115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-17T07:13:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.657+0000 7f82b2e00700 1 -- 172.21.15.86:0/1196674492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac102980 msgr2=0x7f82ac102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.657+0000 7f82b2e00700 1 --2- 172.21.15.86:0/1196674492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac102980 0x7f82ac102e00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8298009990 tx=0x7f8298009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.658+0000 7f82b2e00700 1 -- 172.21.15.86:0/1196674492 shutdown_connections 2024-04-17T07:13:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.658+0000 7f82b2e00700 1 --2- 172.21.15.86:0/1196674492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ac103340 0x7f82ac10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.658+0000 7f82b2e00700 1 --2- 172.21.15.86:0/1196674492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82ac108980 0x7f82ac108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.658+0000 7f82b2e00700 1 --2- 172.21.15.86:0/1196674492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac102980 0x7f82ac102e00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.658+0000 7f82b2e00700 1 -- 172.21.15.86:0/1196674492 >> 172.21.15.86:0/1196674492 conn(0x7f82ac0fe4d0 msgr2=0x7f82ac1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.659+0000 7f82b2e00700 1 -- 172.21.15.86:0/1196674492 shutdown_connections 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.659+0000 7f82b2e00700 1 -- 172.21.15.86:0/1196674492 wait complete. 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.661+0000 7f82b2e00700 1 Processor -- start 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.661+0000 7f82b2e00700 1 -- start start 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b2e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac102980 0x7f82ac199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b2e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ac103340 0x7f82ac199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b2e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82ac108980 0x7f82ac19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b2e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac112b90 con 0x7f82ac102980 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b2e00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac112a10 con 0x7f82ac108980 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b2e00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac112d10 con 0x7f82ac103340 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac102980 0x7f82ac199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b1a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac102980 0x7f82ac199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34112/0 (socket says 172.21.15.86:34112) 2024-04-17T07:13:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b1a00700 1 -- 172.21.15.86:0/1099003115 learned_addr learned my addr 172.21.15.86:0/1099003115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b1000700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ac103340 0x7f82ac199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.662+0000 7f82b2400700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82ac108980 0x7f82ac19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82b1a00700 1 -- 172.21.15.86:0/1099003115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ac103340 msgr2=0x7f82ac199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82b1a00700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ac103340 0x7f82ac199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82b1a00700 1 -- 172.21.15.86:0/1099003115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82ac108980 msgr2=0x7f82ac19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82b1a00700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82ac108980 0x7f82ac19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82b1a00700 1 -- 172.21.15.86:0/1099003115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac19e050 con 0x7f82ac102980 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82b1000700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ac103340 0x7f82ac199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82b1a00700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac102980 0x7f82ac199040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f82a000d950 tx=0x7f82a000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82aa000700 1 -- 172.21.15.86:0/1099003115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0016070 con 0x7f82ac102980 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82b2e00700 1 -- 172.21.15.86:0/1099003115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac19e340 con 0x7f82ac102980 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82aa000700 1 -- 172.21.15.86:0/1099003115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82a0010460 con 0x7f82ac102980 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.663+0000 7f82aa000700 1 -- 172.21.15.86:0/1099003115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a000f620 con 0x7f82ac102980 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.664+0000 7f82b2e00700 1 -- 172.21.15.86:0/1099003115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1a5bc0 con 0x7f82ac102980 2024-04-17T07:13:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.665+0000 7f82b2e00700 1 -- 172.21.15.86:0/1099003115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac069740 con 0x7f82ac102980 2024-04-17T07:13:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.667+0000 7f82aa000700 1 -- 172.21.15.86:0/1099003115 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f82a0019070 con 0x7f82ac102980 2024-04-17T07:13:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.668+0000 7f82aa000700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82940a3d50 0x7f82940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.668+0000 7f82aa000700 1 -- 172.21.15.86:0/1099003115 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f82a00d04e0 con 0x7f82ac102980 2024-04-17T07:13:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.669+0000 7f82b1000700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82940a3d50 0x7f82940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.669+0000 7f82b1000700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82940a3d50 0x7f82940a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8298006010 tx=0x7f829800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.673+0000 7f82aa000700 1 -- 172.21.15.86:0/1099003115 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f82a009c980 con 0x7f82ac102980 2024-04-17T07:13:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.909+0000 7f82b2e00700 1 -- 172.21.15.86:0/1099003115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f82ac19a3a0 con 0x7f82ac102980 2024-04-17T07:13:44.581 INFO:teuthology.orchestra.run.smithi086.stdout:309237645358 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.910+0000 7f82aa000700 1 -- 172.21.15.86:0/1099003115 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f82a009c280 con 0x7f82ac102980 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 -- 172.21.15.86:0/1099003115 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82940a3d50 msgr2=0x7f82940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82940a3d50 0x7f82940a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8298006010 tx=0x7f829800b540 comp rx=0 tx=0).stop 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 -- 172.21.15.86:0/1099003115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac102980 msgr2=0x7f82ac199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac102980 0x7f82ac199040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f82a000d950 tx=0x7f82a000dd10 comp rx=0 tx=0).stop 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 -- 172.21.15.86:0/1099003115 shutdown_connections 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82940a3d50 0x7f82940a6210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82ac108980 0x7f82ac19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac102980 0x7f82ac199040 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 --2- 172.21.15.86:0/1099003115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ac103340 0x7f82ac199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 -- 172.21.15.86:0/1099003115 >> 172.21.15.86:0/1099003115 conn(0x7f82ac0fe4d0 msgr2=0x7f82ac0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:44.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:44 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1099003115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-17T07:13:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.912+0000 7f82b2e00700 1 -- 172.21.15.86:0/1099003115 shutdown_connections 2024-04-17T07:13:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:43.913+0000 7f82b2e00700 1 -- 172.21.15.86:0/1099003115 wait complete. 2024-04-17T07:13:44.718 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182486 got 107374182492 for osd.3 2024-04-17T07:13:44.719 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:45.385 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:45 smithi086 ceph-mon[31733]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:45.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:45 smithi098 ceph-mon[35418]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:45.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:45 smithi103 ceph-mon[35495]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.664+0000 7fe190800700 1 -- 172.21.15.86:0/1859097250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe188103460 msgr2=0x7fe18810f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.664+0000 7fe190800700 1 --2- 172.21.15.86:0/1859097250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe188103460 0x7fe18810f8c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe18400b0a0 tx=0x7fe18400b3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.664+0000 7fe190800700 1 -- 172.21.15.86:0/1859097250 shutdown_connections 2024-04-17T07:13:46.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.664+0000 7fe190800700 1 --2- 172.21.15.86:0/1859097250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe188103460 0x7fe18810f8c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.664+0000 7fe190800700 1 --2- 172.21.15.86:0/1859097250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188108a60 0x7fe188108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.664+0000 7fe190800700 1 --2- 172.21.15.86:0/1859097250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe188102aa0 0x7fe188102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.664+0000 7fe190800700 1 -- 172.21.15.86:0/1859097250 >> 172.21.15.86:0/1859097250 conn(0x7fe188079bb0 msgr2=0x7fe188079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.664+0000 7fe190800700 1 -- 172.21.15.86:0/1859097250 shutdown_connections 2024-04-17T07:13:46.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.664+0000 7fe190800700 1 -- 172.21.15.86:0/1859097250 wait complete. 2024-04-17T07:13:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.665+0000 7fe190800700 1 Processor -- start 2024-04-17T07:13:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.665+0000 7fe190800700 1 -- start start 2024-04-17T07:13:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.666+0000 7fe190800700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188102aa0 0x7fe188074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.666+0000 7fe190800700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe188103460 0x7fe188075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.666+0000 7fe190800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe188108a60 0x7fe1880757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.666+0000 7fe190800700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe188112c80 con 0x7fe188108a60 2024-04-17T07:13:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.666+0000 7fe190800700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe188112b00 con 0x7fe188102aa0 2024-04-17T07:13:46.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.666+0000 7fe190800700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe188112e00 con 0x7fe188103460 2024-04-17T07:13:46.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.666+0000 7fe18e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe188108a60 0x7fe1880757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.666+0000 7fe18e800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe188108a60 0x7fe1880757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34136/0 (socket says 172.21.15.86:34136) 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.666+0000 7fe18e800700 1 -- 172.21.15.86:0/668585254 learned_addr learned my addr 172.21.15.86:0/668585254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.667+0000 7fe18d400700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe188103460 0x7fe188075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.667+0000 7fe18de00700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188102aa0 0x7fe188074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.667+0000 7fe18e800700 1 -- 172.21.15.86:0/668585254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe188103460 msgr2=0x7fe188075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.667+0000 7fe18e800700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe188103460 0x7fe188075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.667+0000 7fe18e800700 1 -- 172.21.15.86:0/668585254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188102aa0 msgr2=0x7fe188074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.667+0000 7fe18e800700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188102aa0 0x7fe188074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.667+0000 7fe18e800700 1 -- 172.21.15.86:0/668585254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1881a7700 con 0x7fe188108a60 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.667+0000 7fe18e800700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe188108a60 0x7fe1880757d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe184009fd0 tx=0x7fe184004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.668+0000 7fe182a00700 1 -- 172.21.15.86:0/668585254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184021070 con 0x7fe188108a60 2024-04-17T07:13:46.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.668+0000 7fe182a00700 1 -- 172.21.15.86:0/668585254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe184007b80 con 0x7fe188108a60 2024-04-17T07:13:46.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.668+0000 7fe190800700 1 -- 172.21.15.86:0/668585254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1881a7990 con 0x7fe188108a60 2024-04-17T07:13:46.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.668+0000 7fe182a00700 1 -- 172.21.15.86:0/668585254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1840052a0 con 0x7fe188108a60 2024-04-17T07:13:46.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.668+0000 7fe190800700 1 -- 172.21.15.86:0/668585254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881a7e40 con 0x7fe188108a60 2024-04-17T07:13:46.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.670+0000 7fe190800700 1 -- 172.21.15.86:0/668585254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe188069740 con 0x7fe188108a60 2024-04-17T07:13:46.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.672+0000 7fe182a00700 1 -- 172.21.15.86:0/668585254 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe184024070 con 0x7fe188108a60 2024-04-17T07:13:46.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.673+0000 7fe182a00700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe1700a3cd0 0x7fe1700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:46.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.673+0000 7fe182a00700 1 -- 172.21.15.86:0/668585254 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe18400f020 con 0x7fe188108a60 2024-04-17T07:13:46.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.673+0000 7fe18de00700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe1700a3cd0 0x7fe1700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:46.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.674+0000 7fe18de00700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe1700a3cd0 0x7fe1700a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe17c005950 tx=0x7fe17c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:46.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.677+0000 7fe182a00700 1 -- 172.21.15.86:0/668585254 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe18402b030 con 0x7fe188108a60 2024-04-17T07:13:46.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.913+0000 7fe190800700 1 -- 172.21.15.86:0/668585254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe18802df90 con 0x7fe188108a60 2024-04-17T07:13:46.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.914+0000 7fe182a00700 1 -- 172.21.15.86:0/668585254 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe184011e50 con 0x7fe188108a60 2024-04-17T07:13:46.914 INFO:teuthology.orchestra.run.smithi086.stdout:287762808884 2024-04-17T07:13:46.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.916+0000 7fe190800700 1 -- 172.21.15.86:0/668585254 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe1700a3cd0 msgr2=0x7fe1700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:46.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.916+0000 7fe190800700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe1700a3cd0 0x7fe1700a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe17c005950 tx=0x7fe17c00b410 comp rx=0 tx=0).stop 2024-04-17T07:13:46.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.916+0000 7fe190800700 1 -- 172.21.15.86:0/668585254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe188108a60 msgr2=0x7fe1880757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:46.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.916+0000 7fe190800700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe188108a60 0x7fe1880757d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe184009fd0 tx=0x7fe184004dc0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.917+0000 7fe190800700 1 -- 172.21.15.86:0/668585254 shutdown_connections 2024-04-17T07:13:46.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.917+0000 7fe190800700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe1700a3cd0 0x7fe1700a6190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.917+0000 7fe190800700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe188108a60 0x7fe1880757d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.917+0000 7fe190800700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe188102aa0 0x7fe188074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.917+0000 7fe190800700 1 --2- 172.21.15.86:0/668585254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe188103460 0x7fe188075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:46.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.917+0000 7fe190800700 1 -- 172.21.15.86:0/668585254 >> 172.21.15.86:0/668585254 conn(0x7fe188079bb0 msgr2=0x7fe1881018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:46.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.917+0000 7fe190800700 1 -- 172.21.15.86:0/668585254 shutdown_connections 2024-04-17T07:13:46.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:46.917+0000 7fe190800700 1 -- 172.21.15.86:0/668585254 wait complete. 2024-04-17T07:13:47.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:46 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/668585254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-17T07:13:47.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:46 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/668585254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-17T07:13:47.315 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:46 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/668585254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-17T07:13:47.380 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738475 got 34359738481 for osd.0 2024-04-17T07:13:47.380 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:47.928 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645350 got 309237645358 for osd.10 2024-04-17T07:13:47.929 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:48.114 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:47 smithi086 ceph-mon[31733]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:13:48.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:47 smithi098 ceph-mon[35418]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:13:48.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:47 smithi103 ceph-mon[35495]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.262+0000 7f7727200700 1 -- 172.21.15.86:0/1554526683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7728109950 msgr2=0x7f7728115df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.262+0000 7f7727200700 1 --2- 172.21.15.86:0/1554526683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7728109950 0x7f7728115df0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f772000b0a0 tx=0x7f772000b3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.262+0000 7f7727200700 1 -- 172.21.15.86:0/1554526683 shutdown_connections 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.262+0000 7f7727200700 1 --2- 172.21.15.86:0/1554526683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7728109950 0x7f7728115df0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.262+0000 7f7727200700 1 --2- 172.21.15.86:0/1554526683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f772810f040 0x7f772810f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.262+0000 7f7727200700 1 --2- 172.21.15.86:0/1554526683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7728108f90 0x7f7728109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.262+0000 7f7727200700 1 -- 172.21.15.86:0/1554526683 >> 172.21.15.86:0/1554526683 conn(0x7f77280787e0 msgr2=0x7f7728107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.262+0000 7f7727200700 1 -- 172.21.15.86:0/1554526683 shutdown_connections 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.262+0000 7f7727200700 1 -- 172.21.15.86:0/1554526683 wait complete. 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.263+0000 7f7727200700 1 Processor -- start 2024-04-17T07:13:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.264+0000 7f7727200700 1 -- start start 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.264+0000 7f7727200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7728108f90 0x7f772819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.264+0000 7f7727200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7728109950 0x7f772819fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.264+0000 7f7727200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f772810f040 0x7f77281a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.264+0000 7f7727200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7728119150 con 0x7f772810f040 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.264+0000 7f7727200700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7728118fd0 con 0x7f7728109950 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.264+0000 7f7727200700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77281192d0 con 0x7f7728108f90 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.264+0000 7f7726800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f772810f040 0x7f77281a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.264+0000 7f7726800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f772810f040 0x7f77281a3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58336/0 (socket says 172.21.15.86:58336) 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.264+0000 7f7726800700 1 -- 172.21.15.86:0/1792958693 learned_addr learned my addr 172.21.15.86:0/1792958693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.265+0000 7f7725400700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7728109950 0x7f772819fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.265+0000 7f7725e00700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7728108f90 0x7f772819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.265+0000 7f7726800700 1 -- 172.21.15.86:0/1792958693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7728108f90 msgr2=0x7f772819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.265+0000 7f7726800700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7728108f90 0x7f772819f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.265+0000 7f7726800700 1 -- 172.21.15.86:0/1792958693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7728109950 msgr2=0x7f772819fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.265+0000 7f7726800700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7728109950 0x7f772819fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.265+0000 7f7726800700 1 -- 172.21.15.86:0/1792958693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77281a4510 con 0x7f772810f040 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.265+0000 7f7726800700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f772810f040 0x7f77281a3dd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f772000b070 tx=0x7f772000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.265+0000 7f771ea00700 1 -- 172.21.15.86:0/1792958693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7720021070 con 0x7f772810f040 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.266+0000 7f771ea00700 1 -- 172.21.15.86:0/1792958693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7720007cf0 con 0x7f772810f040 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.266+0000 7f7727200700 1 -- 172.21.15.86:0/1792958693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77281a47a0 con 0x7f772810f040 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.266+0000 7f771ea00700 1 -- 172.21.15.86:0/1792958693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7720005440 con 0x7f772810f040 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.266+0000 7f7727200700 1 -- 172.21.15.86:0/1792958693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77281ac080 con 0x7f772810f040 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.267+0000 7f7727200700 1 -- 172.21.15.86:0/1792958693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7728111390 con 0x7f772810f040 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.269+0000 7f771ea00700 1 -- 172.21.15.86:0/1792958693 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7720024070 con 0x7f772810f040 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.270+0000 7f771ea00700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f770c0a3cd0 0x7f770c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.270+0000 7f771ea00700 1 -- 172.21.15.86:0/1792958693 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f772000f020 con 0x7f772810f040 2024-04-17T07:13:48.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.270+0000 7f7725e00700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f770c0a3cd0 0x7f770c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:48.570 INFO:teuthology.orchestra.run.smithi086.stdout:343597383718 2024-04-17T07:13:48.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.271+0000 7f7725e00700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f770c0a3cd0 0x7f770c0a6190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7718005fd0 tx=0x7f771800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:48.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.274+0000 7f771ea00700 1 -- 172.21.15.86:0/1792958693 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f772002b020 con 0x7f772810f040 2024-04-17T07:13:48.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.503+0000 7f7727200700 1 -- 172.21.15.86:0/1792958693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f77281a0860 con 0x7f772810f040 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.504+0000 7f771ea00700 1 -- 172.21.15.86:0/1792958693 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f772009e640 con 0x7f772810f040 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 -- 172.21.15.86:0/1792958693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f770c0a3cd0 msgr2=0x7f770c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f770c0a3cd0 0x7f770c0a6190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7718005fd0 tx=0x7f771800a560 comp rx=0 tx=0).stop 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 -- 172.21.15.86:0/1792958693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f772810f040 msgr2=0x7f77281a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f772810f040 0x7f77281a3dd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f772000b070 tx=0x7f772000cf00 comp rx=0 tx=0).stop 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 -- 172.21.15.86:0/1792958693 shutdown_connections 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f770c0a3cd0 0x7f770c0a6190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f772810f040 0x7f77281a3dd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7728108f90 0x7f772819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 --2- 172.21.15.86:0/1792958693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7728109950 0x7f772819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 -- 172.21.15.86:0/1792958693 >> 172.21.15.86:0/1792958693 conn(0x7f77280787e0 msgr2=0x7f77280798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 -- 172.21.15.86:0/1792958693 shutdown_connections 2024-04-17T07:13:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.507+0000 7f7727200700 1 -- 172.21.15.86:0/1792958693 wait complete. 2024-04-17T07:13:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.778+0000 7f0b85a00700 1 -- 172.21.15.86:0/3172945368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b80108a40 msgr2=0x7f0b80108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.778+0000 7f0b85a00700 1 --2- 172.21.15.86:0/3172945368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b80108a40 0x7f0b80108e20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009990 tx=0x7f0b70009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.778+0000 7f0b85a00700 1 -- 172.21.15.86:0/3172945368 shutdown_connections 2024-04-17T07:13:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.778+0000 7f0b85a00700 1 --2- 172.21.15.86:0/3172945368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b8006b100 0x7f0b8010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.778+0000 7f0b85a00700 1 --2- 172.21.15.86:0/3172945368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b80108a40 0x7f0b80108e20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.778+0000 7f0b85a00700 1 --2- 172.21.15.86:0/3172945368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8006a760 0x7f0b8006abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.778+0000 7f0b85a00700 1 -- 172.21.15.86:0/3172945368 >> 172.21.15.86:0/3172945368 conn(0x7f0b80078900 msgr2=0x7f0b80078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.779+0000 7f0b85a00700 1 -- 172.21.15.86:0/3172945368 shutdown_connections 2024-04-17T07:13:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.779+0000 7f0b85a00700 1 -- 172.21.15.86:0/3172945368 wait complete. 2024-04-17T07:13:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.780+0000 7f0b85a00700 1 Processor -- start 2024-04-17T07:13:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.780+0000 7f0b85a00700 1 -- start start 2024-04-17T07:13:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.780+0000 7f0b85a00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b8006a760 0x7f0b8019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:48.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.780+0000 7f0b85a00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8006b100 0x7f0b8019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:48.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.780+0000 7f0b85a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b80108a40 0x7f0b8019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:48.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.780+0000 7f0b85a00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80112bc0 con 0x7f0b80108a40 2024-04-17T07:13:48.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.780+0000 7f0b85a00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80112a40 con 0x7f0b8006b100 2024-04-17T07:13:48.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.780+0000 7f0b85a00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80112d40 con 0x7f0b8006a760 2024-04-17T07:13:48.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.780+0000 7f0b7f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b80108a40 0x7f0b8019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:48.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.781+0000 7f0b7f400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b80108a40 0x7f0b8019fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58324/0 (socket says 172.21.15.86:58324) 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.781+0000 7f0b7f400700 1 -- 172.21.15.86:0/1730100916 learned_addr learned my addr 172.21.15.86:0/1730100916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.781+0000 7f0b7e000700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8006b100 0x7f0b8019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.781+0000 7f0b7ea00700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b8006a760 0x7f0b8019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.781+0000 7f0b7f400700 1 -- 172.21.15.86:0/1730100916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b8006a760 msgr2=0x7f0b8019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.781+0000 7f0b7f400700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b8006a760 0x7f0b8019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.781+0000 7f0b7f400700 1 -- 172.21.15.86:0/1730100916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8006b100 msgr2=0x7f0b8019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.781+0000 7f0b7f400700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8006b100 0x7f0b8019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.781+0000 7f0b7f400700 1 -- 172.21.15.86:0/1730100916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b801a0290 con 0x7f0b80108a40 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.781+0000 7f0b7f400700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b80108a40 0x7f0b8019fb50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0b78007d70 tx=0x7f0b7800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.782+0000 7f0b77400700 1 -- 172.21.15.86:0/1730100916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78016020 con 0x7f0b80108a40 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.782+0000 7f0b85a00700 1 -- 172.21.15.86:0/1730100916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b801a0580 con 0x7f0b80108a40 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.782+0000 7f0b77400700 1 -- 172.21.15.86:0/1730100916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b7800ad40 con 0x7f0b80108a40 2024-04-17T07:13:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.782+0000 7f0b77400700 1 -- 172.21.15.86:0/1730100916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780115e0 con 0x7f0b80108a40 2024-04-17T07:13:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.782+0000 7f0b85a00700 1 -- 172.21.15.86:0/1730100916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b801a7e00 con 0x7f0b80108a40 2024-04-17T07:13:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.783+0000 7f0b85a00700 1 -- 172.21.15.86:0/1730100916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b80069740 con 0x7f0b80108a40 2024-04-17T07:13:48.848 INFO:teuthology.orchestra.run.smithi086.stdout:141733920856 2024-04-17T07:13:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.785+0000 7f0b77400700 1 -- 172.21.15.86:0/1730100916 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0b78019020 con 0x7f0b80108a40 2024-04-17T07:13:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.786+0000 7f0b77400700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0b640a3cd0 0x7f0b640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.787+0000 7f0b77400700 1 -- 172.21.15.86:0/1730100916 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0b780d0800 con 0x7f0b80108a40 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.787+0000 7f0b7ea00700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0b640a3cd0 0x7f0b640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.788+0000 7f0b7ea00700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0b640a3cd0 0x7f0b640a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0b70006010 tx=0x7f0b7001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:47.794+0000 7f0b77400700 1 -- 172.21.15.86:0/1730100916 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0b78020020 con 0x7f0b80108a40 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.049+0000 7f0b85a00700 1 -- 172.21.15.86:0/1730100916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0b8002df90 con 0x7f0b80108a40 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.050+0000 7f0b77400700 1 -- 172.21.15.86:0/1730100916 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0b7809cdb0 con 0x7f0b80108a40 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 -- 172.21.15.86:0/1730100916 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0b640a3cd0 msgr2=0x7f0b640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0b640a3cd0 0x7f0b640a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0b70006010 tx=0x7f0b7001a040 comp rx=0 tx=0).stop 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 -- 172.21.15.86:0/1730100916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b80108a40 msgr2=0x7f0b8019fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b80108a40 0x7f0b8019fb50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0b78007d70 tx=0x7f0b7800a340 comp rx=0 tx=0).stop 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 -- 172.21.15.86:0/1730100916 shutdown_connections 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0b640a3cd0 0x7f0b640a6190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b80108a40 0x7f0b8019fb50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b8006a760 0x7f0b8019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 --2- 172.21.15.86:0/1730100916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b8006b100 0x7f0b8019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:48.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 -- 172.21.15.86:0/1730100916 >> 172.21.15.86:0/1730100916 conn(0x7f0b80078900 msgr2=0x7f0b8010dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:48.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 -- 172.21.15.86:0/1730100916 shutdown_connections 2024-04-17T07:13:48.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:48.053+0000 7f0b85a00700 1 -- 172.21.15.86:0/1730100916 wait complete. 2024-04-17T07:13:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:48 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1730100916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-17T07:13:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:48 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1792958693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-17T07:13:49.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:48 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1730100916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-17T07:13:49.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:48 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1792958693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-17T07:13:49.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:48 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1730100916' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-17T07:13:49.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:48 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1792958693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-17T07:13:49.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411422 got 77309411426 for osd.2 2024-04-17T07:13:49.440 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:49.952 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808879 got 287762808884 for osd.9 2024-04-17T07:13:49.952 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:50.181 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:49 smithi086 ceph-mon[31733]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:13:50.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:49 smithi098 ceph-mon[35418]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:13:50.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:49 smithi103 ceph-mon[35495]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:13:50.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574948 got 55834574954 for osd.1 2024-04-17T07:13:50.565 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:50.897 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920849 got 141733920856 for osd.4 2024-04-17T07:13:50.897 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:51.494 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383714 got 343597383718 for osd.11 2024-04-17T07:13:51.495 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:51.818 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:52.384 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757323 got 163208757327 for osd.5 2024-04-17T07:13:52.385 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:52.502 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:52 smithi086 ceph-mon[31733]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:52 smithi098 ceph-mon[35418]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:52 smithi103 ceph-mon[35495]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:53.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.044+0000 7f56b2200700 1 -- 172.21.15.86:0/2257224009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56ac06b990 msgr2=0x7f56ac10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:53.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.044+0000 7f56b2200700 1 --2- 172.21.15.86:0/2257224009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56ac06b990 0x7f56ac10f900 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f56a400b0a0 tx=0x7f56a400b3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:53.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.044+0000 7f56b2200700 1 -- 172.21.15.86:0/2257224009 shutdown_connections 2024-04-17T07:13:53.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.044+0000 7f56b2200700 1 --2- 172.21.15.86:0/2257224009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56ac06b990 0x7f56ac10f900 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:53.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.044+0000 7f56b2200700 1 --2- 172.21.15.86:0/2257224009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac108a30 0x7f56ac108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:53.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.044+0000 7f56b2200700 1 --2- 172.21.15.86:0/2257224009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56ac06aff0 0x7f56ac06b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.044+0000 7f56b2200700 1 -- 172.21.15.86:0/2257224009 >> 172.21.15.86:0/2257224009 conn(0x7f56ac0788c0 msgr2=0x7f56ac078cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.044+0000 7f56b2200700 1 -- 172.21.15.86:0/2257224009 shutdown_connections 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.044+0000 7f56b2200700 1 -- 172.21.15.86:0/2257224009 wait complete. 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.045+0000 7f56b2200700 1 Processor -- start 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.046+0000 7f56b2200700 1 -- start start 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.046+0000 7f56b2200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56ac06aff0 0x7f56ac101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.046+0000 7f56b2200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac06b990 0x7f56ac101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.046+0000 7f56b2200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56ac108a30 0x7f56ac102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.046+0000 7f56b2200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac112bd0 con 0x7f56ac108a30 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.046+0000 7f56b2200700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac112a50 con 0x7f56ac06aff0 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.046+0000 7f56b2200700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac112d50 con 0x7f56ac06b990 2024-04-17T07:13:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.046+0000 7f56abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56ac108a30 0x7f56ac102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.046+0000 7f56abe00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56ac108a30 0x7f56ac102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58346/0 (socket says 172.21.15.86:58346) 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.046+0000 7f56abe00700 1 -- 172.21.15.86:0/2165383088 learned_addr learned my addr 172.21.15.86:0/2165383088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.047+0000 7f56aaa00700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac06b990 0x7f56ac101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.047+0000 7f56ab400700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56ac06aff0 0x7f56ac101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.047+0000 7f56abe00700 1 -- 172.21.15.86:0/2165383088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac06b990 msgr2=0x7f56ac101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.047+0000 7f56abe00700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac06b990 0x7f56ac101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.047+0000 7f56abe00700 1 -- 172.21.15.86:0/2165383088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56ac06aff0 msgr2=0x7f56ac101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.047+0000 7f56abe00700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56ac06aff0 0x7f56ac101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.047+0000 7f56abe00700 1 -- 172.21.15.86:0/2165383088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac1aba50 con 0x7f56ac108a30 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.047+0000 7f56abe00700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56ac108a30 0x7f56ac102490 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f56a400b830 tx=0x7f56a4004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.048+0000 7f56a3e00700 1 -- 172.21.15.86:0/2165383088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4021070 con 0x7f56ac108a30 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.048+0000 7f56a3e00700 1 -- 172.21.15.86:0/2165383088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56a4003c00 con 0x7f56ac108a30 2024-04-17T07:13:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.048+0000 7f56b2200700 1 -- 172.21.15.86:0/2165383088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac1abc80 con 0x7f56ac108a30 2024-04-17T07:13:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.048+0000 7f56a3e00700 1 -- 172.21.15.86:0/2165383088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a40047c0 con 0x7f56ac108a30 2024-04-17T07:13:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.048+0000 7f56b2200700 1 -- 172.21.15.86:0/2165383088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac1ac190 con 0x7f56ac108a30 2024-04-17T07:13:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.049+0000 7f56b2200700 1 -- 172.21.15.86:0/2165383088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ac104170 con 0x7f56ac108a30 2024-04-17T07:13:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.051+0000 7f56a3e00700 1 -- 172.21.15.86:0/2165383088 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f56a4007a00 con 0x7f56ac108a30 2024-04-17T07:13:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.053+0000 7f56a3e00700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f56900a3d50 0x7f56900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.054+0000 7f56a3e00700 1 -- 172.21.15.86:0/2165383088 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f56a400f020 con 0x7f56ac108a30 2024-04-17T07:13:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.054+0000 7f56ab400700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f56900a3d50 0x7f56900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.054+0000 7f56ab400700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f56900a3d50 0x7f56900a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f569c00bbb0 tx=0x7f569c00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.056+0000 7f56a3e00700 1 -- 172.21.15.86:0/2165383088 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f56a402a020 con 0x7f56ac108a30 2024-04-17T07:13:53.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.292+0000 7f56b2200700 1 -- 172.21.15.86:0/2165383088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f56ac02df90 con 0x7f56ac108a30 2024-04-17T07:13:53.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.292+0000 7f56a3e00700 1 -- 172.21.15.86:0/2165383088 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f56a4007cf0 con 0x7f56ac108a30 2024-04-17T07:13:53.293 INFO:teuthology.orchestra.run.smithi086.stdout:219043332164 2024-04-17T07:13:53.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 -- 172.21.15.86:0/2165383088 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f56900a3d50 msgr2=0x7f56900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:53.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f56900a3d50 0x7f56900a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f569c00bbb0 tx=0x7f569c00b590 comp rx=0 tx=0).stop 2024-04-17T07:13:53.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 -- 172.21.15.86:0/2165383088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56ac108a30 msgr2=0x7f56ac102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:53.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56ac108a30 0x7f56ac102490 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f56a400b830 tx=0x7f56a4004d90 comp rx=0 tx=0).stop 2024-04-17T07:13:53.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 -- 172.21.15.86:0/2165383088 shutdown_connections 2024-04-17T07:13:53.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f56900a3d50 0x7f56900a6210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:53.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56ac108a30 0x7f56ac102490 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:53.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56ac06aff0 0x7f56ac101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:53.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 --2- 172.21.15.86:0/2165383088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f56ac06b990 0x7f56ac101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:53.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 -- 172.21.15.86:0/2165383088 >> 172.21.15.86:0/2165383088 conn(0x7f56ac0788c0 msgr2=0x7f56ac0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:53.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 -- 172.21.15.86:0/2165383088 shutdown_connections 2024-04-17T07:13:53.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:53.296+0000 7f56b2200700 1 -- 172.21.15.86:0/2165383088 wait complete. 2024-04-17T07:13:54.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:54 smithi098 ceph-mon[35418]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:13:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:54 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2165383088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-17T07:13:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:54 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:54 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:54.503 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:54 smithi086 ceph-mon[31733]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:13:54.503 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:54 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2165383088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-17T07:13:54.503 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:54 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:54.503 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:54 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:54 smithi103 ceph-mon[35495]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:13:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:54 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2165383088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-17T07:13:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:54 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:54 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:54.676 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332156 got 219043332164 for osd.7 2024-04-17T07:13:54.676 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:54.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.502+0000 7f5789400700 1 -- 172.21.15.86:0/834684901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578406a760 msgr2=0x7f578406abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:54.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.502+0000 7f5789400700 1 --2- 172.21.15.86:0/834684901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578406a760 0x7f578406abc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5770009990 tx=0x7f5770009ca0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.503+0000 7f5789400700 1 -- 172.21.15.86:0/834684901 shutdown_connections 2024-04-17T07:13:54.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.503+0000 7f5789400700 1 --2- 172.21.15.86:0/834684901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f578406b100 0x7f578410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.503+0000 7f5789400700 1 --2- 172.21.15.86:0/834684901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5784108a40 0x7f5784108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.503+0000 7f5789400700 1 --2- 172.21.15.86:0/834684901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578406a760 0x7f578406abc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.503+0000 7f5789400700 1 -- 172.21.15.86:0/834684901 >> 172.21.15.86:0/834684901 conn(0x7f5784078900 msgr2=0x7f5784078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.503+0000 7f5789400700 1 -- 172.21.15.86:0/834684901 shutdown_connections 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.503+0000 7f5789400700 1 -- 172.21.15.86:0/834684901 wait complete. 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.504+0000 7f5789400700 1 Processor -- start 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.504+0000 7f5789400700 1 -- start start 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.505+0000 7f5789400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578406a760 0x7f578419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.505+0000 7f5789400700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f578406b100 0x7f578419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.505+0000 7f5789400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5784108a40 0x7f578419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.505+0000 7f5789400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5784112c50 con 0x7f578406a760 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.505+0000 7f5789400700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5784112ad0 con 0x7f578406b100 2024-04-17T07:13:54.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.505+0000 7f5789400700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5784112dd0 con 0x7f5784108a40 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.506+0000 7f5783200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5784108a40 0x7f578419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.506+0000 7f5783200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5784108a40 0x7f578419fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:50322/0 (socket says 172.21.15.86:50322) 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.506+0000 7f5781e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f578406b100 0x7f578419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.506+0000 7f5783200700 1 -- 172.21.15.86:0/2185714181 learned_addr learned my addr 172.21.15.86:0/2185714181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.506+0000 7f5782800700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578406a760 0x7f578419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.506+0000 7f5783200700 1 -- 172.21.15.86:0/2185714181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f578406b100 msgr2=0x7f578419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.506+0000 7f5783200700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f578406b100 0x7f578419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.507+0000 7f5783200700 1 -- 172.21.15.86:0/2185714181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578406a760 msgr2=0x7f578419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.507+0000 7f5783200700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578406a760 0x7f578419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.507+0000 7f5783200700 1 -- 172.21.15.86:0/2185714181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57841a0380 con 0x7f5784108a40 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.507+0000 7f5783200700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5784108a40 0x7f578419fc40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f577c007ee0 tx=0x7f577c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.507+0000 7f577b400700 1 -- 172.21.15.86:0/2185714181 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c00dbd0 con 0x7f5784108a40 2024-04-17T07:13:54.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.508+0000 7f5789400700 1 -- 172.21.15.86:0/2185714181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57841a0670 con 0x7f5784108a40 2024-04-17T07:13:54.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.508+0000 7f577b400700 1 -- 172.21.15.86:0/2185714181 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f577c00dd70 con 0x7f5784108a40 2024-04-17T07:13:54.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.508+0000 7f577b400700 1 -- 172.21.15.86:0/2185714181 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c005990 con 0x7f5784108a40 2024-04-17T07:13:54.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.508+0000 7f5789400700 1 -- 172.21.15.86:0/2185714181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57841a7ef0 con 0x7f5784108a40 2024-04-17T07:13:54.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.509+0000 7f5789400700 1 -- 172.21.15.86:0/2185714181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5784069740 con 0x7f5784108a40 2024-04-17T07:13:54.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.512+0000 7f577b400700 1 -- 172.21.15.86:0/2185714181 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f577c01a020 con 0x7f5784108a40 2024-04-17T07:13:54.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.513+0000 7f577b400700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57680a3cd0 0x7f57680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:54.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.513+0000 7f577b400700 1 -- 172.21.15.86:0/2185714181 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f577c0d0690 con 0x7f5784108a40 2024-04-17T07:13:54.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.514+0000 7f5782800700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57680a3cd0 0x7f57680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:54.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.514+0000 7f5782800700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57680a3cd0 0x7f57680a6190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5774009bd0 tx=0x7f5774009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:54.823 INFO:teuthology.orchestra.run.smithi086.stdout:253403070525 2024-04-17T07:13:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.528+0000 7f577b400700 1 -- 172.21.15.86:0/2185714181 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f577c010070 con 0x7f5784108a40 2024-04-17T07:13:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.763+0000 7f5789400700 1 -- 172.21.15.86:0/2185714181 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f57841a8120 con 0x7f5784108a40 2024-04-17T07:13:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.764+0000 7f577b400700 1 -- 172.21.15.86:0/2185714181 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f577c09cb90 con 0x7f5784108a40 2024-04-17T07:13:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.767+0000 7f5789400700 1 -- 172.21.15.86:0/2185714181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57680a3cd0 msgr2=0x7f57680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.767+0000 7f5789400700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57680a3cd0 0x7f57680a6190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5774009bd0 tx=0x7f5774009260 comp rx=0 tx=0).stop 2024-04-17T07:13:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.767+0000 7f5789400700 1 -- 172.21.15.86:0/2185714181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5784108a40 msgr2=0x7f578419fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.767+0000 7f5789400700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5784108a40 0x7f578419fc40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f577c007ee0 tx=0x7f577c00d3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.767+0000 7f5789400700 1 -- 172.21.15.86:0/2185714181 shutdown_connections 2024-04-17T07:13:54.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.767+0000 7f5789400700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57680a3cd0 0x7f57680a6190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.767+0000 7f5789400700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5784108a40 0x7f578419fc40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.767+0000 7f5789400700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f578406a760 0x7f578419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.767+0000 7f5789400700 1 --2- 172.21.15.86:0/2185714181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f578406b100 0x7f578419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.767+0000 7f5789400700 1 -- 172.21.15.86:0/2185714181 >> 172.21.15.86:0/2185714181 conn(0x7f5784078900 msgr2=0x7f57840ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.768+0000 7f5789400700 1 -- 172.21.15.86:0/2185714181 shutdown_connections 2024-04-17T07:13:54.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:54.768+0000 7f5789400700 1 -- 172.21.15.86:0/2185714181 wait complete. 2024-04-17T07:13:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:55 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2185714181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-17T07:13:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:55 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2185714181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-17T07:13:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:55 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2185714181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-17T07:13:56.028 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070517 got 253403070525 for osd.8 2024-04-17T07:13:56.028 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:56.215 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:56 smithi086 ceph-mon[31733]: pgmap v427: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:56.215 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:13:56.215 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:13:56.215 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:56 smithi098 ceph-mon[35418]: pgmap v427: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:13:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:13:56.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:56.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.518+0000 7f5024200700 1 -- 172.21.15.86:0/3410412714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501c10f480 msgr2=0x7f501c111870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.518+0000 7f5024200700 1 --2- 172.21.15.86:0/3410412714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501c10f480 0x7f501c111870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f501400b0a0 tx=0x7f501400b3b0 comp rx=0 tx=0).stop 2024-04-17T07:13:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.518+0000 7f5024200700 1 -- 172.21.15.86:0/3410412714 shutdown_connections 2024-04-17T07:13:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.518+0000 7f5024200700 1 --2- 172.21.15.86:0/3410412714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501c10f480 0x7f501c111870 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.518+0000 7f5024200700 1 --2- 172.21.15.86:0/3410412714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f501c074c80 0x7f501c075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.518+0000 7f5024200700 1 --2- 172.21.15.86:0/3410412714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c0755a0 0x7f501c10ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.518+0000 7f5024200700 1 -- 172.21.15.86:0/3410412714 >> 172.21.15.86:0/3410412714 conn(0x7f501c079b20 msgr2=0x7f501c079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.518+0000 7f5024200700 1 -- 172.21.15.86:0/3410412714 shutdown_connections 2024-04-17T07:13:56.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.518+0000 7f5024200700 1 -- 172.21.15.86:0/3410412714 wait complete. 2024-04-17T07:13:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.520+0000 7f5024200700 1 Processor -- start 2024-04-17T07:13:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.520+0000 7f5024200700 1 -- start start 2024-04-17T07:13:56.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5024200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f501c074c80 0x7f501c19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5024200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c0755a0 0x7f501c19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5024200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501c10f480 0x7f501c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5024200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f501c114e70 con 0x7f501c10f480 2024-04-17T07:13:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5024200700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f501c114cf0 con 0x7f501c074c80 2024-04-17T07:13:56.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5024200700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f501c114ff0 con 0x7f501c0755a0 2024-04-17T07:13:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5022200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501c10f480 0x7f501c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:56.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5022200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501c10f480 0x7f501c19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58390/0 (socket says 172.21.15.86:58390) 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5022200700 1 -- 172.21.15.86:0/2723759087 learned_addr learned my addr 172.21.15.86:0/2723759087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5021800700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f501c074c80 0x7f501c19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.521+0000 7f5020e00700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c0755a0 0x7f501c19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f5022200700 1 -- 172.21.15.86:0/2723759087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c0755a0 msgr2=0x7f501c19b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f5022200700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c0755a0 0x7f501c19b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f5022200700 1 -- 172.21.15.86:0/2723759087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f501c074c80 msgr2=0x7f501c19b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f5022200700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f501c074c80 0x7f501c19b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f5022200700 1 -- 172.21.15.86:0/2723759087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501c1a0200 con 0x7f501c10f480 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f5022200700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501c10f480 0x7f501c19fac0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5014009fd0 tx=0x7f5014004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f501a000700 1 -- 172.21.15.86:0/2723759087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014021070 con 0x7f501c10f480 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f501a000700 1 -- 172.21.15.86:0/2723759087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5014007d00 con 0x7f501c10f480 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f5024200700 1 -- 172.21.15.86:0/2723759087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501c1a0490 con 0x7f501c10f480 2024-04-17T07:13:56.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f501a000700 1 -- 172.21.15.86:0/2723759087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501401d450 con 0x7f501c10f480 2024-04-17T07:13:56.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.522+0000 7f5024200700 1 -- 172.21.15.86:0/2723759087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501c1a7d70 con 0x7f501c10f480 2024-04-17T07:13:56.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.524+0000 7f5024200700 1 -- 172.21.15.86:0/2723759087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501c069740 con 0x7f501c10f480 2024-04-17T07:13:56.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.526+0000 7f501a000700 1 -- 172.21.15.86:0/2723759087 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f50140041b0 con 0x7f501c10f480 2024-04-17T07:13:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:56 smithi103 ceph-mon[35495]: pgmap v427: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:13:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:13:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:13:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:13:56.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.527+0000 7f501a000700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50040a3d50 0x7f50040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:13:56.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.527+0000 7f501a000700 1 -- 172.21.15.86:0/2723759087 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f501400f020 con 0x7f501c10f480 2024-04-17T07:13:56.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.527+0000 7f5021800700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50040a3d50 0x7f50040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:13:56.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.528+0000 7f5021800700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50040a3d50 0x7f50040a6210 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5010005ea0 tx=0x7f5010005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:13:56.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.532+0000 7f501a000700 1 -- 172.21.15.86:0/2723759087 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f501402a020 con 0x7f501c10f480 2024-04-17T07:13:56.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.759+0000 7f5024200700 1 -- 172.21.15.86:0/2723759087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f501c1a7f40 con 0x7f501c10f480 2024-04-17T07:13:56.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.760+0000 7f501a000700 1 -- 172.21.15.86:0/2723759087 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f501409e480 con 0x7f501c10f480 2024-04-17T07:13:56.760 INFO:teuthology.orchestra.run.smithi086.stdout:197568495691 2024-04-17T07:13:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.763+0000 7f5024200700 1 -- 172.21.15.86:0/2723759087 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50040a3d50 msgr2=0x7f50040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.763+0000 7f5024200700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50040a3d50 0x7f50040a6210 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5010005ea0 tx=0x7f5010005e30 comp rx=0 tx=0).stop 2024-04-17T07:13:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.763+0000 7f5024200700 1 -- 172.21.15.86:0/2723759087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501c10f480 msgr2=0x7f501c19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:13:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.763+0000 7f5024200700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501c10f480 0x7f501c19fac0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5014009fd0 tx=0x7f5014004e70 comp rx=0 tx=0).stop 2024-04-17T07:13:56.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.763+0000 7f5024200700 1 -- 172.21.15.86:0/2723759087 shutdown_connections 2024-04-17T07:13:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.763+0000 7f5024200700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50040a3d50 0x7f50040a6210 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.763+0000 7f5024200700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f501c10f480 0x7f501c19fac0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.763+0000 7f5024200700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f501c074c80 0x7f501c19b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.763+0000 7f5024200700 1 --2- 172.21.15.86:0/2723759087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f501c0755a0 0x7f501c19b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:13:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.763+0000 7f5024200700 1 -- 172.21.15.86:0/2723759087 >> 172.21.15.86:0/2723759087 conn(0x7f501c079b20 msgr2=0x7f501c0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:13:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.764+0000 7f5024200700 1 -- 172.21.15.86:0/2723759087 shutdown_connections 2024-04-17T07:13:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:13:56.764+0000 7f5024200700 1 -- 172.21.15.86:0/2723759087 wait complete. 2024-04-17T07:13:57.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:57 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2723759087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-17T07:13:57.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:57 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2723759087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-17T07:13:57.571 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:57 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2723759087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-17T07:13:57.632 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495685 got 197568495691 for osd.6 2024-04-17T07:13:57.632 DEBUG:teuthology.parallel:result is None 2024-04-17T07:13:57.632 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-17T07:13:57.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph pg dump --format=json 2024-04-17T07:13:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:13:58.448 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:58 smithi086 ceph-mon[31733]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:13:58.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:58 smithi098 ceph-mon[35418]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:13:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:58 smithi103 ceph-mon[35495]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:13:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:59 smithi098 ceph-mon[35418]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-17T07:13:59.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:13:59 smithi098 ceph-mon[35418]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-17T07:13:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:59 smithi103 ceph-mon[35495]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-17T07:13:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:13:59 smithi103 ceph-mon[35495]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-17T07:13:59.565 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:59 smithi086 ceph-mon[31733]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-17T07:13:59.566 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:13:59 smithi086 ceph-mon[31733]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-17T07:14:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.377+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2861873302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb010f450 msgr2=0x7fbbb0111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.377+0000 7fbbb6e00700 1 --2- 172.21.15.86:0/2861873302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb010f450 0x7fbbb0111840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009990 tx=0x7fbba8009ca0 comp rx=0 tx=0).stop 2024-04-17T07:14:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.377+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2861873302 shutdown_connections 2024-04-17T07:14:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.377+0000 7fbbb6e00700 1 --2- 172.21.15.86:0/2861873302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb010f450 0x7fbbb0111840 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.377+0000 7fbbb6e00700 1 --2- 172.21.15.86:0/2861873302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb0074c80 0x7fbbb0075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.377+0000 7fbbb6e00700 1 --2- 172.21.15.86:0/2861873302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb00755a0 0x7fbbb010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.377+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2861873302 >> 172.21.15.86:0/2861873302 conn(0x7fbbb0079bb0 msgr2=0x7fbbb0079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.377+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2861873302 shutdown_connections 2024-04-17T07:14:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.377+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2861873302 wait complete. 2024-04-17T07:14:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.378+0000 7fbbb6e00700 1 Processor -- start 2024-04-17T07:14:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.378+0000 7fbbb6e00700 1 -- start start 2024-04-17T07:14:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.379+0000 7fbbb6e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb0074c80 0x7fbbb019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.379+0000 7fbbb6e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb00755a0 0x7fbbb019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.379+0000 7fbbb6e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb010f450 0x7fbbb019fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.379+0000 7fbbb6e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0114e30 con 0x7fbbb00755a0 2024-04-17T07:14:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.379+0000 7fbbb6e00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0114cb0 con 0x7fbbb010f450 2024-04-17T07:14:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.379+0000 7fbbb6e00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0114fb0 con 0x7fbbb0074c80 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.379+0000 7fbba7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb00755a0 0x7fbbb019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.379+0000 7fbba7400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb00755a0 0x7fbbb019b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42664/0 (socket says 172.21.15.86:42664) 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.379+0000 7fbba7400700 1 -- 172.21.15.86:0/2421105466 learned_addr learned my addr 172.21.15.86:0/2421105466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.379+0000 7fbbafe00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb0074c80 0x7fbbb019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbbb4e00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb010f450 0x7fbbb019fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbba7400700 1 -- 172.21.15.86:0/2421105466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb0074c80 msgr2=0x7fbbb019b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbba7400700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb0074c80 0x7fbbb019b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbba7400700 1 -- 172.21.15.86:0/2421105466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb010f450 msgr2=0x7fbbb019fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbba7400700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb010f450 0x7fbbb019fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbba7400700 1 -- 172.21.15.86:0/2421105466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb01a0250 con 0x7fbbb00755a0 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbbafe00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb0074c80 0x7fbbb019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbba7400700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb00755a0 0x7fbbb019b780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800c970 tx=0x7fbb9800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbbad600700 1 -- 172.21.15.86:0/2421105466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98004220 con 0x7fbbb00755a0 2024-04-17T07:14:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbbad600700 1 -- 172.21.15.86:0/2421105466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb98007d50 con 0x7fbbb00755a0 2024-04-17T07:14:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2421105466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb01a0540 con 0x7fbbb00755a0 2024-04-17T07:14:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbbad600700 1 -- 172.21.15.86:0/2421105466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98003d70 con 0x7fbbb00755a0 2024-04-17T07:14:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.380+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2421105466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb01a7dc0 con 0x7fbbb00755a0 2024-04-17T07:14:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.382+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2421105466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb0069740 con 0x7fbbb00755a0 2024-04-17T07:14:00.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.383+0000 7fbbad600700 1 -- 172.21.15.86:0/2421105466 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbb980078a0 con 0x7fbbb00755a0 2024-04-17T07:14:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.385+0000 7fbbad600700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb940a3c40 0x7fbb940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.385+0000 7fbbad600700 1 -- 172.21.15.86:0/2421105466 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbb980d0710 con 0x7fbbb00755a0 2024-04-17T07:14:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.385+0000 7fbbafe00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb940a3c40 0x7fbb940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.385+0000 7fbbafe00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb940a3c40 0x7fbb940a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbba000e300 tx=0x7fbba0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:00.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.389+0000 7fbbad600700 1 -- 172.21.15.86:0/2421105466 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbb98014030 con 0x7fbbb00755a0 2024-04-17T07:14:00.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.594+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2421105466 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbbb0100290 con 0x7fbb940a3c40 2024-04-17T07:14:00.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.611+0000 7fbbad600700 1 -- 172.21.15.86:0/2421105466 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340857 (secure 0 0 0) 0x7fbbb0100290 con 0x7fbb940a3c40 2024-04-17T07:14:00.613 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.618+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2421105466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb940a3c40 msgr2=0x7fbb940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.618+0000 7fbbb6e00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb940a3c40 0x7fbb940a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbba000e300 tx=0x7fbba0009040 comp rx=0 tx=0).stop 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.618+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2421105466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb00755a0 msgr2=0x7fbbb019b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.618+0000 7fbbb6e00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb00755a0 0x7fbbb019b780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800c970 tx=0x7fbb9800cd30 comp rx=0 tx=0).stop 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.618+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2421105466 shutdown_connections 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.618+0000 7fbbb6e00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb940a3c40 0x7fbb940a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.618+0000 7fbbb6e00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb010f450 0x7fbbb019fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.618+0000 7fbbb6e00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb0074c80 0x7fbbb019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.618+0000 7fbbb6e00700 1 --2- 172.21.15.86:0/2421105466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb00755a0 0x7fbbb019b780 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.618+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2421105466 >> 172.21.15.86:0/2421105466 conn(0x7fbbb0079bb0 msgr2=0x7fbbb00ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.619+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2421105466 shutdown_connections 2024-04-17T07:14:00.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:00.619+0000 7fbbb6e00700 1 -- 172.21.15.86:0/2421105466 wait complete. 2024-04-17T07:14:00.622 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-17T07:14:01.481 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-04-17T07:13:59.100534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659665,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1019,"num_read_kb":633,"num_write":428,"num_write_kb":3083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181358,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":568,"ondisk_log_size":568,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365956,"kb_used_data":12576,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575868,"statfs":{"total":1151940427776,"available":1151565688832,"internally_reserved":0,"allocated":12877824,"data_stored":8188643,"data_compressed":432012,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":75,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005915"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078599+0000","last_change":"2024-04-17T07:11:55.486746+0000","last_active":"2024-04-17T07:13:57.078599+0000","last_peered":"2024-04-17T07:13:57.078599+0000","last_clean":"2024-04-17T07:13:57.078599+0000","last_became_active":"2024-04-17T07:11:55.486335+0000","last_became_peered":"2024-04-17T07:11:55.486335+0000","last_unstale":"2024-04-17T07:13:57.078599+0000","last_undegraded":"2024-04-17T07:13:57.078599+0000","last_fullsized":"2024-04-17T07:13:57.078599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:48.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081897+0000","last_change":"2024-04-17T07:11:55.481832+0000","last_active":"2024-04-17T07:13:57.081897+0000","last_peered":"2024-04-17T07:13:57.081897+0000","last_clean":"2024-04-17T07:13:57.081897+0000","last_became_active":"2024-04-17T07:11:55.476428+0000","last_became_peered":"2024-04-17T07:11:55.476428+0000","last_unstale":"2024-04-17T07:13:57.081897+0000","last_undegraded":"2024-04-17T07:13:57.081897+0000","last_fullsized":"2024-04-17T07:13:57.081897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:04.452542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083222+0000","last_change":"2024-04-17T07:11:55.483721+0000","last_active":"2024-04-17T07:13:57.083222+0000","last_peered":"2024-04-17T07:13:57.083222+0000","last_clean":"2024-04-17T07:13:57.083222+0000","last_became_active":"2024-04-17T07:11:55.483268+0000","last_became_peered":"2024-04-17T07:11:55.483268+0000","last_unstale":"2024-04-17T07:13:57.083222+0000","last_undegraded":"2024-04-17T07:13:57.083222+0000","last_fullsized":"2024-04-17T07:13:57.083222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:41.584783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175367+0000","last_change":"2024-04-17T07:11:55.485114+0000","last_active":"2024-04-17T07:13:29.175367+0000","last_peered":"2024-04-17T07:13:29.175367+0000","last_clean":"2024-04-17T07:13:29.175367+0000","last_became_active":"2024-04-17T07:11:55.483913+0000","last_became_peered":"2024-04-17T07:11:55.483913+0000","last_unstale":"2024-04-17T07:13:29.175367+0000","last_undegraded":"2024-04-17T07:13:29.175367+0000","last_fullsized":"2024-04-17T07:13:29.175367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:46.211121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204144+0000","last_change":"2024-04-17T07:11:55.491952+0000","last_active":"2024-04-17T07:13:29.204144+0000","last_peered":"2024-04-17T07:13:29.204144+0000","last_clean":"2024-04-17T07:13:29.204144+0000","last_became_active":"2024-04-17T07:11:55.491238+0000","last_became_peered":"2024-04-17T07:11:55.491238+0000","last_unstale":"2024-04-17T07:13:29.204144+0000","last_undegraded":"2024-04-17T07:13:29.204144+0000","last_fullsized":"2024-04-17T07:13:29.204144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:06.759797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068603+0000","last_change":"2024-04-17T07:11:55.471124+0000","last_active":"2024-04-17T07:13:57.068603+0000","last_peered":"2024-04-17T07:13:57.068603+0000","last_clean":"2024-04-17T07:13:57.068603+0000","last_became_active":"2024-04-17T07:11:55.470781+0000","last_became_peered":"2024-04-17T07:11:55.470781+0000","last_unstale":"2024-04-17T07:13:57.068603+0000","last_undegraded":"2024-04-17T07:13:57.068603+0000","last_fullsized":"2024-04-17T07:13:57.068603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:45.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205948+0000","last_change":"2024-04-17T07:11:55.469099+0000","last_active":"2024-04-17T07:13:29.205948+0000","last_peered":"2024-04-17T07:13:29.205948+0000","last_clean":"2024-04-17T07:13:29.205948+0000","last_became_active":"2024-04-17T07:11:55.468584+0000","last_became_peered":"2024-04-17T07:11:55.468584+0000","last_unstale":"2024-04-17T07:13:29.205948+0000","last_undegraded":"2024-04-17T07:13:29.205948+0000","last_fullsized":"2024-04-17T07:13:29.205948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:52.834846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203165+0000","last_change":"2024-04-17T07:11:55.473794+0000","last_active":"2024-04-17T07:13:29.203165+0000","last_peered":"2024-04-17T07:13:29.203165+0000","last_clean":"2024-04-17T07:13:29.203165+0000","last_became_active":"2024-04-17T07:11:55.471919+0000","last_became_peered":"2024-04-17T07:11:55.471919+0000","last_unstale":"2024-04-17T07:13:29.203165+0000","last_undegraded":"2024-04-17T07:13:29.203165+0000","last_fullsized":"2024-04-17T07:13:29.203165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:16.085117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206105+0000","last_change":"2024-04-17T07:11:55.482445+0000","last_active":"2024-04-17T07:13:29.206105+0000","last_peered":"2024-04-17T07:13:29.206105+0000","last_clean":"2024-04-17T07:13:29.206105+0000","last_became_active":"2024-04-17T07:11:55.481552+0000","last_became_peered":"2024-04-17T07:11:55.481552+0000","last_unstale":"2024-04-17T07:13:29.206105+0000","last_undegraded":"2024-04-17T07:13:29.206105+0000","last_fullsized":"2024-04-17T07:13:29.206105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:48.733585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071209+0000","last_change":"2024-04-17T07:11:55.476726+0000","last_active":"2024-04-17T07:13:57.071209+0000","last_peered":"2024-04-17T07:13:57.071209+0000","last_clean":"2024-04-17T07:13:57.071209+0000","last_became_active":"2024-04-17T07:11:55.476320+0000","last_became_peered":"2024-04-17T07:11:55.476320+0000","last_unstale":"2024-04-17T07:13:57.071209+0000","last_undegraded":"2024-04-17T07:13:57.071209+0000","last_fullsized":"2024-04-17T07:13:57.071209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.230181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209484+0000","last_change":"2024-04-17T07:11:55.485798+0000","last_active":"2024-04-17T07:13:29.209484+0000","last_peered":"2024-04-17T07:13:29.209484+0000","last_clean":"2024-04-17T07:13:29.209484+0000","last_became_active":"2024-04-17T07:11:55.484015+0000","last_became_peered":"2024-04-17T07:11:55.484015+0000","last_unstale":"2024-04-17T07:13:29.209484+0000","last_undegraded":"2024-04-17T07:13:29.209484+0000","last_fullsized":"2024-04-17T07:13:29.209484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:37.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.061275+0000","last_change":"2024-04-17T07:12:28.729834+0000","last_active":"2024-04-17T07:13:57.061275+0000","last_peered":"2024-04-17T07:13:57.061275+0000","last_clean":"2024-04-17T07:13:57.061275+0000","last_became_active":"2024-04-17T07:12:28.728694+0000","last_became_peered":"2024-04-17T07:12:28.728694+0000","last_unstale":"2024-04-17T07:13:57.061275+0000","last_undegraded":"2024-04-17T07:13:57.061275+0000","last_fullsized":"2024-04-17T07:13:57.061275+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:46.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067835+0000","last_change":"2024-04-17T07:13:29.210325+0000","last_active":"2024-04-17T07:13:57.067835+0000","last_peered":"2024-04-17T07:13:57.067835+0000","last_clean":"2024-04-17T07:13:57.067835+0000","last_became_active":"2024-04-17T07:13:29.209759+0000","last_became_peered":"2024-04-17T07:13:29.209759+0000","last_unstale":"2024-04-17T07:13:57.067835+0000","last_undegraded":"2024-04-17T07:13:57.067835+0000","last_fullsized":"2024-04-17T07:13:57.067835+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:17.168931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083523+0000","last_change":"2024-04-17T07:11:55.473839+0000","last_active":"2024-04-17T07:13:57.083523+0000","last_peered":"2024-04-17T07:13:57.083523+0000","last_clean":"2024-04-17T07:13:57.083523+0000","last_became_active":"2024-04-17T07:11:55.473472+0000","last_became_peered":"2024-04-17T07:11:55.473472+0000","last_unstale":"2024-04-17T07:13:57.083523+0000","last_undegraded":"2024-04-17T07:13:57.083523+0000","last_fullsized":"2024-04-17T07:13:57.083523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:02.263272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081336+0000","last_change":"2024-04-17T07:11:55.484820+0000","last_active":"2024-04-17T07:13:57.081336+0000","last_peered":"2024-04-17T07:13:57.081336+0000","last_clean":"2024-04-17T07:13:57.081336+0000","last_became_active":"2024-04-17T07:11:55.484314+0000","last_became_peered":"2024-04-17T07:11:55.484314+0000","last_unstale":"2024-04-17T07:13:57.081336+0000","last_undegraded":"2024-04-17T07:13:57.081336+0000","last_fullsized":"2024-04-17T07:13:57.081336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.186912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204040+0000","last_change":"2024-04-17T07:11:55.492018+0000","last_active":"2024-04-17T07:13:29.204040+0000","last_peered":"2024-04-17T07:13:29.204040+0000","last_clean":"2024-04-17T07:13:29.204040+0000","last_became_active":"2024-04-17T07:11:55.490512+0000","last_became_peered":"2024-04-17T07:11:55.490512+0000","last_unstale":"2024-04-17T07:13:29.204040+0000","last_undegraded":"2024-04-17T07:13:29.204040+0000","last_fullsized":"2024-04-17T07:13:29.204040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:45.436270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081355+0000","last_change":"2024-04-17T07:11:55.487210+0000","last_active":"2024-04-17T07:13:57.081355+0000","last_peered":"2024-04-17T07:13:57.081355+0000","last_clean":"2024-04-17T07:13:57.081355+0000","last_became_active":"2024-04-17T07:11:55.485713+0000","last_became_peered":"2024-04-17T07:11:55.485713+0000","last_unstale":"2024-04-17T07:13:57.081355+0000","last_undegraded":"2024-04-17T07:13:57.081355+0000","last_fullsized":"2024-04-17T07:13:57.081355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:54.495950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082677+0000","last_change":"2024-04-17T07:11:55.482705+0000","last_active":"2024-04-17T07:13:57.082677+0000","last_peered":"2024-04-17T07:13:57.082677+0000","last_clean":"2024-04-17T07:13:57.082677+0000","last_became_active":"2024-04-17T07:11:55.482349+0000","last_became_peered":"2024-04-17T07:11:55.482349+0000","last_unstale":"2024-04-17T07:13:57.082677+0000","last_undegraded":"2024-04-17T07:13:57.082677+0000","last_fullsized":"2024-04-17T07:13:57.082677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:57.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205690+0000","last_change":"2024-04-17T07:11:55.467065+0000","last_active":"2024-04-17T07:13:29.205690+0000","last_peered":"2024-04-17T07:13:29.205690+0000","last_clean":"2024-04-17T07:13:29.205690+0000","last_became_active":"2024-04-17T07:11:55.466535+0000","last_became_peered":"2024-04-17T07:11:55.466535+0000","last_unstale":"2024-04-17T07:13:29.205690+0000","last_undegraded":"2024-04-17T07:13:29.205690+0000","last_fullsized":"2024-04-17T07:13:29.205690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:49.955453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204248+0000","last_change":"2024-04-17T07:11:59.515778+0000","last_active":"2024-04-17T07:13:29.204248+0000","last_peered":"2024-04-17T07:13:29.204248+0000","last_clean":"2024-04-17T07:13:29.204248+0000","last_became_active":"2024-04-17T07:11:59.515266+0000","last_became_peered":"2024-04-17T07:11:59.515266+0000","last_unstale":"2024-04-17T07:13:29.204248+0000","last_undegraded":"2024-04-17T07:13:29.204248+0000","last_fullsized":"2024-04-17T07:13:29.204248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:43.084724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:55.461099+0000","last_change":"2024-04-17T07:11:54.467060+0000","last_active":"2024-04-17T07:13:55.461099+0000","last_peered":"2024-04-17T07:13:55.461099+0000","last_clean":"2024-04-17T07:13:55.461099+0000","last_became_active":"2024-04-17T07:11:54.466687+0000","last_became_peered":"2024-04-17T07:11:54.466687+0000","last_unstale":"2024-04-17T07:13:55.461099+0000","last_undegraded":"2024-04-17T07:13:55.461099+0000","last_fullsized":"2024-04-17T07:13:55.461099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:57.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205055+0000","last_change":"2024-04-17T07:11:55.469857+0000","last_active":"2024-04-17T07:13:29.205055+0000","last_peered":"2024-04-17T07:13:29.205055+0000","last_clean":"2024-04-17T07:13:29.205055+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:13:29.205055+0000","last_undegraded":"2024-04-17T07:13:29.205055+0000","last_fullsized":"2024-04-17T07:13:29.205055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:43.565627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174262+0000","last_change":"2024-04-17T07:12:29.125397+0000","last_active":"2024-04-17T07:13:29.174262+0000","last_peered":"2024-04-17T07:13:29.174262+0000","last_clean":"2024-04-17T07:13:29.174262+0000","last_became_active":"2024-04-17T07:12:29.124158+0000","last_became_peered":"2024-04-17T07:12:29.124158+0000","last_unstale":"2024-04-17T07:13:29.174262+0000","last_undegraded":"2024-04-17T07:13:29.174262+0000","last_fullsized":"2024-04-17T07:13:29.174262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:24.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205266+0000","last_change":"2024-04-17T07:11:54.464715+0000","last_active":"2024-04-17T07:13:29.205266+0000","last_peered":"2024-04-17T07:13:29.205266+0000","last_clean":"2024-04-17T07:13:29.205266+0000","last_became_active":"2024-04-17T07:11:54.463917+0000","last_became_peered":"2024-04-17T07:11:54.463917+0000","last_unstale":"2024-04-17T07:13:29.205266+0000","last_undegraded":"2024-04-17T07:13:29.205266+0000","last_fullsized":"2024-04-17T07:13:29.205266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:43.738471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082948+0000","last_change":"2024-04-17T07:11:55.482876+0000","last_active":"2024-04-17T07:13:57.082948+0000","last_peered":"2024-04-17T07:13:57.082948+0000","last_clean":"2024-04-17T07:13:57.082948+0000","last_became_active":"2024-04-17T07:11:55.482451+0000","last_became_peered":"2024-04-17T07:11:55.482451+0000","last_unstale":"2024-04-17T07:13:57.082948+0000","last_undegraded":"2024-04-17T07:13:57.082948+0000","last_fullsized":"2024-04-17T07:13:57.082948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:39.394090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174779+0000","last_change":"2024-04-17T07:11:59.522634+0000","last_active":"2024-04-17T07:13:29.174779+0000","last_peered":"2024-04-17T07:13:29.174779+0000","last_clean":"2024-04-17T07:13:29.174779+0000","last_became_active":"2024-04-17T07:11:59.520797+0000","last_became_peered":"2024-04-17T07:11:59.520797+0000","last_unstale":"2024-04-17T07:13:29.174779+0000","last_undegraded":"2024-04-17T07:13:29.174779+0000","last_fullsized":"2024-04-17T07:13:29.174779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:02.316515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.334093+0000","last_change":"2024-04-17T07:12:28.756528+0000","last_active":"2024-04-17T07:13:29.334093+0000","last_peered":"2024-04-17T07:13:29.334093+0000","last_clean":"2024-04-17T07:13:29.334093+0000","last_became_active":"2024-04-17T07:12:28.737008+0000","last_became_peered":"2024-04-17T07:12:28.737008+0000","last_unstale":"2024-04-17T07:13:29.334093+0000","last_undegraded":"2024-04-17T07:13:29.334093+0000","last_fullsized":"2024-04-17T07:13:29.334093+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:56:26.719377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204987+0000","last_change":"2024-04-17T07:11:55.485999+0000","last_active":"2024-04-17T07:13:29.204987+0000","last_peered":"2024-04-17T07:13:29.204987+0000","last_clean":"2024-04-17T07:13:29.204987+0000","last_became_active":"2024-04-17T07:11:55.484048+0000","last_became_peered":"2024-04-17T07:11:55.484048+0000","last_unstale":"2024-04-17T07:13:29.204987+0000","last_undegraded":"2024-04-17T07:13:29.204987+0000","last_fullsized":"2024-04-17T07:13:29.204987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:38.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174261+0000","last_change":"2024-04-17T07:11:59.529929+0000","last_active":"2024-04-17T07:13:29.174261+0000","last_peered":"2024-04-17T07:13:29.174261+0000","last_clean":"2024-04-17T07:13:29.174261+0000","last_became_active":"2024-04-17T07:11:59.529400+0000","last_became_peered":"2024-04-17T07:11:59.529400+0000","last_unstale":"2024-04-17T07:13:29.174261+0000","last_undegraded":"2024-04-17T07:13:29.174261+0000","last_fullsized":"2024-04-17T07:13:29.174261+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:54.845529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204741+0000","last_change":"2024-04-17T07:11:54.462852+0000","last_active":"2024-04-17T07:13:29.204741+0000","last_peered":"2024-04-17T07:13:29.204741+0000","last_clean":"2024-04-17T07:13:29.204741+0000","last_became_active":"2024-04-17T07:11:54.459178+0000","last_became_peered":"2024-04-17T07:11:54.459178+0000","last_unstale":"2024-04-17T07:13:29.204741+0000","last_undegraded":"2024-04-17T07:13:29.204741+0000","last_fullsized":"2024-04-17T07:13:29.204741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:07.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174122+0000","last_change":"2024-04-17T07:11:55.480951+0000","last_active":"2024-04-17T07:13:29.174122+0000","last_peered":"2024-04-17T07:13:29.174122+0000","last_clean":"2024-04-17T07:13:29.174122+0000","last_became_active":"2024-04-17T07:11:55.480507+0000","last_became_peered":"2024-04-17T07:11:55.480507+0000","last_unstale":"2024-04-17T07:13:29.174122+0000","last_undegraded":"2024-04-17T07:13:29.174122+0000","last_fullsized":"2024-04-17T07:13:29.174122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:42.584180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205344+0000","last_change":"2024-04-17T07:11:59.520897+0000","last_active":"2024-04-17T07:13:29.205344+0000","last_peered":"2024-04-17T07:13:29.205344+0000","last_clean":"2024-04-17T07:13:29.205344+0000","last_became_active":"2024-04-17T07:11:59.520350+0000","last_became_peered":"2024-04-17T07:11:59.520350+0000","last_unstale":"2024-04-17T07:13:29.205344+0000","last_undegraded":"2024-04-17T07:13:29.205344+0000","last_fullsized":"2024-04-17T07:13:29.205344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:39.813097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":137,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:55.480224+0000","last_change":"2024-04-17T07:12:28.753717+0000","last_active":"2024-04-17T07:13:55.480224+0000","last_peered":"2024-04-17T07:13:55.480224+0000","last_clean":"2024-04-17T07:13:55.480224+0000","last_became_active":"2024-04-17T07:12:28.739556+0000","last_became_peered":"2024-04-17T07:12:28.739556+0000","last_unstale":"2024-04-17T07:13:55.480224+0000","last_undegraded":"2024-04-17T07:13:55.480224+0000","last_fullsized":"2024-04-17T07:13:55.480224+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:11.967731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206407+0000","last_change":"2024-04-17T07:11:55.476675+0000","last_active":"2024-04-17T07:13:29.206407+0000","last_peered":"2024-04-17T07:13:29.206407+0000","last_clean":"2024-04-17T07:13:29.206407+0000","last_became_active":"2024-04-17T07:11:55.476273+0000","last_became_peered":"2024-04-17T07:11:55.476273+0000","last_unstale":"2024-04-17T07:13:29.206407+0000","last_undegraded":"2024-04-17T07:13:29.206407+0000","last_fullsized":"2024-04-17T07:13:29.206407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:09.612138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205437+0000","last_change":"2024-04-17T07:12:28.741537+0000","last_active":"2024-04-17T07:13:29.205437+0000","last_peered":"2024-04-17T07:13:29.205437+0000","last_clean":"2024-04-17T07:13:29.205437+0000","last_became_active":"2024-04-17T07:12:28.741035+0000","last_became_peered":"2024-04-17T07:12:28.741035+0000","last_unstale":"2024-04-17T07:13:29.205437+0000","last_undegraded":"2024-04-17T07:13:29.205437+0000","last_fullsized":"2024-04-17T07:13:29.205437+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:15.550112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.207837+0000","last_change":"2024-04-17T07:11:54.471118+0000","last_active":"2024-04-17T07:13:29.207837+0000","last_peered":"2024-04-17T07:13:29.207837+0000","last_clean":"2024-04-17T07:13:29.207837+0000","last_became_active":"2024-04-17T07:11:54.470526+0000","last_became_peered":"2024-04-17T07:11:54.470526+0000","last_unstale":"2024-04-17T07:13:29.207837+0000","last_undegraded":"2024-04-17T07:13:29.207837+0000","last_fullsized":"2024-04-17T07:13:29.207837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:52:47.918086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174100+0000","last_change":"2024-04-17T07:11:55.472850+0000","last_active":"2024-04-17T07:13:29.174100+0000","last_peered":"2024-04-17T07:13:29.174100+0000","last_clean":"2024-04-17T07:13:29.174100+0000","last_became_active":"2024-04-17T07:11:55.472097+0000","last_became_peered":"2024-04-17T07:11:55.472097+0000","last_unstale":"2024-04-17T07:13:29.174100+0000","last_undegraded":"2024-04-17T07:13:29.174100+0000","last_fullsized":"2024-04-17T07:13:29.174100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:41.730809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.334016+0000","last_change":"2024-04-17T07:11:59.522960+0000","last_active":"2024-04-17T07:13:29.334016+0000","last_peered":"2024-04-17T07:13:29.334016+0000","last_clean":"2024-04-17T07:13:29.334016+0000","last_became_active":"2024-04-17T07:11:59.522470+0000","last_became_peered":"2024-04-17T07:11:59.522470+0000","last_unstale":"2024-04-17T07:13:29.334016+0000","last_undegraded":"2024-04-17T07:13:29.334016+0000","last_fullsized":"2024-04-17T07:13:29.334016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:36.620442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.171594+0000","last_change":"2024-04-17T07:11:54.470693+0000","last_active":"2024-04-17T07:13:29.171594+0000","last_peered":"2024-04-17T07:13:29.171594+0000","last_clean":"2024-04-17T07:13:29.171594+0000","last_became_active":"2024-04-17T07:11:54.470334+0000","last_became_peered":"2024-04-17T07:11:54.470334+0000","last_unstale":"2024-04-17T07:13:29.171594+0000","last_undegraded":"2024-04-17T07:13:29.171594+0000","last_fullsized":"2024-04-17T07:13:29.171594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:19:04.980786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203031+0000","last_change":"2024-04-17T07:11:55.474755+0000","last_active":"2024-04-17T07:13:29.203031+0000","last_peered":"2024-04-17T07:13:29.203031+0000","last_clean":"2024-04-17T07:13:29.203031+0000","last_became_active":"2024-04-17T07:11:55.474261+0000","last_became_peered":"2024-04-17T07:11:55.474261+0000","last_unstale":"2024-04-17T07:13:29.203031+0000","last_undegraded":"2024-04-17T07:13:29.203031+0000","last_fullsized":"2024-04-17T07:13:29.203031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:00.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203132+0000","last_change":"2024-04-17T07:11:59.517152+0000","last_active":"2024-04-17T07:13:29.203132+0000","last_peered":"2024-04-17T07:13:29.203132+0000","last_clean":"2024-04-17T07:13:29.203132+0000","last_became_active":"2024-04-17T07:11:59.514977+0000","last_became_peered":"2024-04-17T07:11:59.514977+0000","last_unstale":"2024-04-17T07:13:29.203132+0000","last_undegraded":"2024-04-17T07:13:29.203132+0000","last_fullsized":"2024-04-17T07:13:29.203132+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:14.974136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172810+0000","last_change":"2024-04-17T07:11:54.459786+0000","last_active":"2024-04-17T07:13:29.172810+0000","last_peered":"2024-04-17T07:13:29.172810+0000","last_clean":"2024-04-17T07:13:29.172810+0000","last_became_active":"2024-04-17T07:11:54.457589+0000","last_became_peered":"2024-04-17T07:11:54.457589+0000","last_unstale":"2024-04-17T07:13:29.172810+0000","last_undegraded":"2024-04-17T07:13:29.172810+0000","last_fullsized":"2024-04-17T07:13:29.172810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:25.986539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067337+0000","last_change":"2024-04-17T07:11:55.465020+0000","last_active":"2024-04-17T07:13:57.067337+0000","last_peered":"2024-04-17T07:13:57.067337+0000","last_clean":"2024-04-17T07:13:57.067337+0000","last_became_active":"2024-04-17T07:11:55.464608+0000","last_became_peered":"2024-04-17T07:11:55.464608+0000","last_unstale":"2024-04-17T07:13:57.067337+0000","last_undegraded":"2024-04-17T07:13:57.067337+0000","last_fullsized":"2024-04-17T07:13:57.067337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204510+0000","last_change":"2024-04-17T07:11:59.519333+0000","last_active":"2024-04-17T07:13:29.204510+0000","last_peered":"2024-04-17T07:13:29.204510+0000","last_clean":"2024-04-17T07:13:29.204510+0000","last_became_active":"2024-04-17T07:11:59.518865+0000","last_became_peered":"2024-04-17T07:11:59.518865+0000","last_unstale":"2024-04-17T07:13:29.204510+0000","last_undegraded":"2024-04-17T07:13:29.204510+0000","last_fullsized":"2024-04-17T07:13:29.204510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:42.009010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":208,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:56.481831+0000","last_change":"2024-04-17T07:11:54.462283+0000","last_active":"2024-04-17T07:13:56.481831+0000","last_peered":"2024-04-17T07:13:56.481831+0000","last_clean":"2024-04-17T07:13:56.481831+0000","last_became_active":"2024-04-17T07:11:54.461818+0000","last_became_peered":"2024-04-17T07:11:54.461818+0000","last_unstale":"2024-04-17T07:13:56.481831+0000","last_undegraded":"2024-04-17T07:13:56.481831+0000","last_fullsized":"2024-04-17T07:13:56.481831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.765058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205503+0000","last_change":"2024-04-17T07:11:55.486491+0000","last_active":"2024-04-17T07:13:29.205503+0000","last_peered":"2024-04-17T07:13:29.205503+0000","last_clean":"2024-04-17T07:13:29.205503+0000","last_became_active":"2024-04-17T07:11:55.486121+0000","last_became_peered":"2024-04-17T07:11:55.486121+0000","last_unstale":"2024-04-17T07:13:29.205503+0000","last_undegraded":"2024-04-17T07:13:29.205503+0000","last_fullsized":"2024-04-17T07:13:29.205503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:12.998321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205900+0000","last_change":"2024-04-17T07:11:59.511555+0000","last_active":"2024-04-17T07:13:29.205900+0000","last_peered":"2024-04-17T07:13:29.205900+0000","last_clean":"2024-04-17T07:13:29.205900+0000","last_became_active":"2024-04-17T07:11:59.511147+0000","last_became_peered":"2024-04-17T07:11:59.511147+0000","last_unstale":"2024-04-17T07:13:29.205900+0000","last_undegraded":"2024-04-17T07:13:29.205900+0000","last_fullsized":"2024-04-17T07:13:29.205900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:41.854815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209414+0000","last_change":"2024-04-17T07:12:28.734476+0000","last_active":"2024-04-17T07:13:29.209414+0000","last_peered":"2024-04-17T07:13:29.209414+0000","last_clean":"2024-04-17T07:13:29.209414+0000","last_became_active":"2024-04-17T07:12:28.728187+0000","last_became_peered":"2024-04-17T07:12:28.728187+0000","last_unstale":"2024-04-17T07:13:29.209414+0000","last_undegraded":"2024-04-17T07:13:29.209414+0000","last_fullsized":"2024-04-17T07:13:29.209414+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:45.749607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081348+0000","last_change":"2024-04-17T07:11:55.480589+0000","last_active":"2024-04-17T07:13:57.081348+0000","last_peered":"2024-04-17T07:13:57.081348+0000","last_clean":"2024-04-17T07:13:57.081348+0000","last_became_active":"2024-04-17T07:11:55.480224+0000","last_became_peered":"2024-04-17T07:11:55.480224+0000","last_unstale":"2024-04-17T07:13:57.081348+0000","last_undegraded":"2024-04-17T07:13:57.081348+0000","last_fullsized":"2024-04-17T07:13:57.081348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:05.892442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175056+0000","last_change":"2024-04-17T07:11:59.522021+0000","last_active":"2024-04-17T07:13:29.175056+0000","last_peered":"2024-04-17T07:13:29.175056+0000","last_clean":"2024-04-17T07:13:29.175056+0000","last_became_active":"2024-04-17T07:11:59.521493+0000","last_became_peered":"2024-04-17T07:11:59.521493+0000","last_unstale":"2024-04-17T07:13:29.175056+0000","last_undegraded":"2024-04-17T07:13:29.175056+0000","last_fullsized":"2024-04-17T07:13:29.175056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:20.386216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174326+0000","last_change":"2024-04-17T07:11:54.468225+0000","last_active":"2024-04-17T07:13:29.174326+0000","last_peered":"2024-04-17T07:13:29.174326+0000","last_clean":"2024-04-17T07:13:29.174326+0000","last_became_active":"2024-04-17T07:11:54.467046+0000","last_became_peered":"2024-04-17T07:11:54.467046+0000","last_unstale":"2024-04-17T07:13:29.174326+0000","last_undegraded":"2024-04-17T07:13:29.174326+0000","last_fullsized":"2024-04-17T07:13:29.174326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:17.725255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205258+0000","last_change":"2024-04-17T07:11:55.488980+0000","last_active":"2024-04-17T07:13:29.205258+0000","last_peered":"2024-04-17T07:13:29.205258+0000","last_clean":"2024-04-17T07:13:29.205258+0000","last_became_active":"2024-04-17T07:11:55.488226+0000","last_became_peered":"2024-04-17T07:11:55.488226+0000","last_unstale":"2024-04-17T07:13:29.205258+0000","last_undegraded":"2024-04-17T07:13:29.205258+0000","last_fullsized":"2024-04-17T07:13:29.205258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:00.135973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205554+0000","last_change":"2024-04-17T07:11:59.515134+0000","last_active":"2024-04-17T07:13:29.205554+0000","last_peered":"2024-04-17T07:13:29.205554+0000","last_clean":"2024-04-17T07:13:29.205554+0000","last_became_active":"2024-04-17T07:11:59.514560+0000","last_became_peered":"2024-04-17T07:11:59.514560+0000","last_unstale":"2024-04-17T07:13:29.205554+0000","last_undegraded":"2024-04-17T07:13:29.205554+0000","last_fullsized":"2024-04-17T07:13:29.205554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:34.669789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'36","reported_seq":245,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.006437+0000","last_change":"2024-04-17T07:11:54.461226+0000","last_active":"2024-04-17T07:13:57.006437+0000","last_peered":"2024-04-17T07:13:57.006437+0000","last_clean":"2024-04-17T07:13:57.006437+0000","last_became_active":"2024-04-17T07:11:54.460573+0000","last_became_peered":"2024-04-17T07:11:54.460573+0000","last_unstale":"2024-04-17T07:13:57.006437+0000","last_undegraded":"2024-04-17T07:13:57.006437+0000","last_fullsized":"2024-04-17T07:13:57.006437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62250,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":178,"num_read_kb":149,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204544+0000","last_change":"2024-04-17T07:11:55.483925+0000","last_active":"2024-04-17T07:13:29.204544+0000","last_peered":"2024-04-17T07:13:29.204544+0000","last_clean":"2024-04-17T07:13:29.204544+0000","last_became_active":"2024-04-17T07:11:55.483584+0000","last_became_peered":"2024-04-17T07:11:55.483584+0000","last_unstale":"2024-04-17T07:13:29.204544+0000","last_undegraded":"2024-04-17T07:13:29.204544+0000","last_fullsized":"2024-04-17T07:13:29.204544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:12.328876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174863+0000","last_change":"2024-04-17T07:11:59.515597+0000","last_active":"2024-04-17T07:13:29.174863+0000","last_peered":"2024-04-17T07:13:29.174863+0000","last_clean":"2024-04-17T07:13:29.174863+0000","last_became_active":"2024-04-17T07:11:59.514413+0000","last_became_peered":"2024-04-17T07:11:59.514413+0000","last_unstale":"2024-04-17T07:13:29.174863+0000","last_undegraded":"2024-04-17T07:13:29.174863+0000","last_fullsized":"2024-04-17T07:13:29.174863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:45.824127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205125+0000","last_change":"2024-04-17T07:11:54.449984+0000","last_active":"2024-04-17T07:13:29.205125+0000","last_peered":"2024-04-17T07:13:29.205125+0000","last_clean":"2024-04-17T07:13:29.205125+0000","last_became_active":"2024-04-17T07:11:54.449628+0000","last_became_peered":"2024-04-17T07:11:54.449628+0000","last_unstale":"2024-04-17T07:13:29.205125+0000","last_undegraded":"2024-04-17T07:13:29.205125+0000","last_fullsized":"2024-04-17T07:13:29.205125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:26:03.281684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069323+0000","last_change":"2024-04-17T07:11:55.476912+0000","last_active":"2024-04-17T07:13:57.069323+0000","last_peered":"2024-04-17T07:13:57.069323+0000","last_clean":"2024-04-17T07:13:57.069323+0000","last_became_active":"2024-04-17T07:11:55.471576+0000","last_became_peered":"2024-04-17T07:11:55.471576+0000","last_unstale":"2024-04-17T07:13:57.069323+0000","last_undegraded":"2024-04-17T07:13:57.069323+0000","last_fullsized":"2024-04-17T07:13:57.069323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:35.439271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333787+0000","last_change":"2024-04-17T07:11:59.515897+0000","last_active":"2024-04-17T07:13:29.333787+0000","last_peered":"2024-04-17T07:13:29.333787+0000","last_clean":"2024-04-17T07:13:29.333787+0000","last_became_active":"2024-04-17T07:11:59.515349+0000","last_became_peered":"2024-04-17T07:11:59.515349+0000","last_unstale":"2024-04-17T07:13:29.333787+0000","last_undegraded":"2024-04-17T07:13:29.333787+0000","last_fullsized":"2024-04-17T07:13:29.333787+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:00.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173500+0000","last_change":"2024-04-17T07:11:54.462369+0000","last_active":"2024-04-17T07:13:29.173500+0000","last_peered":"2024-04-17T07:13:29.173500+0000","last_clean":"2024-04-17T07:13:29.173500+0000","last_became_active":"2024-04-17T07:11:54.460357+0000","last_became_peered":"2024-04-17T07:11:54.460357+0000","last_unstale":"2024-04-17T07:13:29.173500+0000","last_undegraded":"2024-04-17T07:13:29.173500+0000","last_fullsized":"2024-04-17T07:13:29.173500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:38.029980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.208013+0000","last_change":"2024-04-17T07:11:55.475874+0000","last_active":"2024-04-17T07:13:29.208013+0000","last_peered":"2024-04-17T07:13:29.208013+0000","last_clean":"2024-04-17T07:13:29.208013+0000","last_became_active":"2024-04-17T07:11:55.475550+0000","last_became_peered":"2024-04-17T07:11:55.475550+0000","last_unstale":"2024-04-17T07:13:29.208013+0000","last_undegraded":"2024-04-17T07:13:29.208013+0000","last_fullsized":"2024-04-17T07:13:29.208013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:40.096573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206806+0000","last_change":"2024-04-17T07:11:59.514661+0000","last_active":"2024-04-17T07:13:29.206806+0000","last_peered":"2024-04-17T07:13:29.206806+0000","last_clean":"2024-04-17T07:13:29.206806+0000","last_became_active":"2024-04-17T07:11:59.513030+0000","last_became_peered":"2024-04-17T07:11:59.513030+0000","last_unstale":"2024-04-17T07:13:29.206806+0000","last_undegraded":"2024-04-17T07:13:29.206806+0000","last_fullsized":"2024-04-17T07:13:29.206806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:35.696961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":121,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:52.466853+0000","last_change":"2024-04-17T07:11:54.455279+0000","last_active":"2024-04-17T07:13:52.466853+0000","last_peered":"2024-04-17T07:13:52.466853+0000","last_clean":"2024-04-17T07:13:52.466853+0000","last_became_active":"2024-04-17T07:11:54.453620+0000","last_became_peered":"2024-04-17T07:11:54.453620+0000","last_unstale":"2024-04-17T07:13:52.466853+0000","last_undegraded":"2024-04-17T07:13:52.466853+0000","last_fullsized":"2024-04-17T07:13:52.466853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:15.227410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205462+0000","last_change":"2024-04-17T07:11:55.469579+0000","last_active":"2024-04-17T07:13:29.205462+0000","last_peered":"2024-04-17T07:13:29.205462+0000","last_clean":"2024-04-17T07:13:29.205462+0000","last_became_active":"2024-04-17T07:11:55.468389+0000","last_became_peered":"2024-04-17T07:11:55.468389+0000","last_unstale":"2024-04-17T07:13:29.205462+0000","last_undegraded":"2024-04-17T07:13:29.205462+0000","last_fullsized":"2024-04-17T07:13:29.205462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:01.274677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173429+0000","last_change":"2024-04-17T07:11:59.512110+0000","last_active":"2024-04-17T07:13:29.173429+0000","last_peered":"2024-04-17T07:13:29.173429+0000","last_clean":"2024-04-17T07:13:29.173429+0000","last_became_active":"2024-04-17T07:11:59.510573+0000","last_became_peered":"2024-04-17T07:11:59.510573+0000","last_unstale":"2024-04-17T07:13:29.173429+0000","last_undegraded":"2024-04-17T07:13:29.173429+0000","last_fullsized":"2024-04-17T07:13:29.173429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:32.515687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333195+0000","last_change":"2024-04-17T07:12:28.767517+0000","last_active":"2024-04-17T07:13:29.333195+0000","last_peered":"2024-04-17T07:13:29.333195+0000","last_clean":"2024-04-17T07:13:29.333195+0000","last_became_active":"2024-04-17T07:12:28.738908+0000","last_became_peered":"2024-04-17T07:12:28.738908+0000","last_unstale":"2024-04-17T07:13:29.333195+0000","last_undegraded":"2024-04-17T07:13:29.333195+0000","last_fullsized":"2024-04-17T07:13:29.333195+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:47:30.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172811+0000","last_change":"2024-04-17T07:11:55.473540+0000","last_active":"2024-04-17T07:13:29.172811+0000","last_peered":"2024-04-17T07:13:29.172811+0000","last_clean":"2024-04-17T07:13:29.172811+0000","last_became_active":"2024-04-17T07:11:55.472436+0000","last_became_peered":"2024-04-17T07:11:55.472436+0000","last_unstale":"2024-04-17T07:13:29.172811+0000","last_undegraded":"2024-04-17T07:13:29.172811+0000","last_fullsized":"2024-04-17T07:13:29.172811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:54.317052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172782+0000","last_change":"2024-04-17T07:11:59.521305+0000","last_active":"2024-04-17T07:13:29.172782+0000","last_peered":"2024-04-17T07:13:29.172782+0000","last_clean":"2024-04-17T07:13:29.172782+0000","last_became_active":"2024-04-17T07:11:59.516860+0000","last_became_peered":"2024-04-17T07:11:59.516860+0000","last_unstale":"2024-04-17T07:13:29.172782+0000","last_undegraded":"2024-04-17T07:13:29.172782+0000","last_fullsized":"2024-04-17T07:13:29.172782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:23.403218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206663+0000","last_change":"2024-04-17T07:11:54.463378+0000","last_active":"2024-04-17T07:13:29.206663+0000","last_peered":"2024-04-17T07:13:29.206663+0000","last_clean":"2024-04-17T07:13:29.206663+0000","last_became_active":"2024-04-17T07:11:54.462120+0000","last_became_peered":"2024-04-17T07:11:54.462120+0000","last_unstale":"2024-04-17T07:13:29.206663+0000","last_undegraded":"2024-04-17T07:13:29.206663+0000","last_fullsized":"2024-04-17T07:13:29.206663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:42.109928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.065968+0000","last_change":"2024-04-17T07:11:55.478604+0000","last_active":"2024-04-17T07:13:57.065968+0000","last_peered":"2024-04-17T07:13:57.065968+0000","last_clean":"2024-04-17T07:13:57.065968+0000","last_became_active":"2024-04-17T07:11:55.478252+0000","last_became_peered":"2024-04-17T07:11:55.478252+0000","last_unstale":"2024-04-17T07:13:57.065968+0000","last_undegraded":"2024-04-17T07:13:57.065968+0000","last_fullsized":"2024-04-17T07:13:57.065968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:08.365896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206271+0000","last_change":"2024-04-17T07:11:59.518619+0000","last_active":"2024-04-17T07:13:29.206271+0000","last_peered":"2024-04-17T07:13:29.206271+0000","last_clean":"2024-04-17T07:13:29.206271+0000","last_became_active":"2024-04-17T07:11:59.518214+0000","last_became_peered":"2024-04-17T07:11:59.518214+0000","last_unstale":"2024-04-17T07:13:29.206271+0000","last_undegraded":"2024-04-17T07:13:29.206271+0000","last_fullsized":"2024-04-17T07:13:29.206271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:46.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206247+0000","last_change":"2024-04-17T07:11:54.453038+0000","last_active":"2024-04-17T07:13:29.206247+0000","last_peered":"2024-04-17T07:13:29.206247+0000","last_clean":"2024-04-17T07:13:29.206247+0000","last_became_active":"2024-04-17T07:11:54.452619+0000","last_became_peered":"2024-04-17T07:11:54.452619+0000","last_unstale":"2024-04-17T07:13:29.206247+0000","last_undegraded":"2024-04-17T07:13:29.206247+0000","last_fullsized":"2024-04-17T07:13:29.206247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:25.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.063375+0000","last_change":"2024-04-17T07:11:55.465990+0000","last_active":"2024-04-17T07:13:57.063375+0000","last_peered":"2024-04-17T07:13:57.063375+0000","last_clean":"2024-04-17T07:13:57.063375+0000","last_became_active":"2024-04-17T07:11:55.465523+0000","last_became_peered":"2024-04-17T07:11:55.465523+0000","last_unstale":"2024-04-17T07:13:57.063375+0000","last_undegraded":"2024-04-17T07:13:57.063375+0000","last_fullsized":"2024-04-17T07:13:57.063375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:40.366027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172265+0000","last_change":"2024-04-17T07:11:59.512119+0000","last_active":"2024-04-17T07:13:29.172265+0000","last_peered":"2024-04-17T07:13:29.172265+0000","last_clean":"2024-04-17T07:13:29.172265+0000","last_became_active":"2024-04-17T07:11:59.511460+0000","last_became_peered":"2024-04-17T07:11:59.511460+0000","last_unstale":"2024-04-17T07:13:29.172265+0000","last_undegraded":"2024-04-17T07:13:29.172265+0000","last_fullsized":"2024-04-17T07:13:29.172265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:50.888203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":123,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:55.462424+0000","last_change":"2024-04-17T07:11:54.469346+0000","last_active":"2024-04-17T07:13:55.462424+0000","last_peered":"2024-04-17T07:13:55.462424+0000","last_clean":"2024-04-17T07:13:55.462424+0000","last_became_active":"2024-04-17T07:11:54.468947+0000","last_became_peered":"2024-04-17T07:11:54.468947+0000","last_unstale":"2024-04-17T07:13:55.462424+0000","last_undegraded":"2024-04-17T07:13:55.462424+0000","last_fullsized":"2024-04-17T07:13:55.462424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:04.854625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077481+0000","last_change":"2024-04-17T07:11:55.471235+0000","last_active":"2024-04-17T07:13:57.077481+0000","last_peered":"2024-04-17T07:13:57.077481+0000","last_clean":"2024-04-17T07:13:57.077481+0000","last_became_active":"2024-04-17T07:11:55.470857+0000","last_became_peered":"2024-04-17T07:11:55.470857+0000","last_unstale":"2024-04-17T07:13:57.077481+0000","last_undegraded":"2024-04-17T07:13:57.077481+0000","last_fullsized":"2024-04-17T07:13:57.077481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:21.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.171912+0000","last_change":"2024-04-17T07:11:59.524519+0000","last_active":"2024-04-17T07:13:29.171912+0000","last_peered":"2024-04-17T07:13:29.171912+0000","last_clean":"2024-04-17T07:13:29.171912+0000","last_became_active":"2024-04-17T07:11:59.519270+0000","last_became_peered":"2024-04-17T07:11:59.519270+0000","last_unstale":"2024-04-17T07:13:29.171912+0000","last_undegraded":"2024-04-17T07:13:29.171912+0000","last_fullsized":"2024-04-17T07:13:29.171912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:41.080249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173538+0000","last_change":"2024-04-17T07:11:55.473922+0000","last_active":"2024-04-17T07:13:29.173538+0000","last_peered":"2024-04-17T07:13:29.173538+0000","last_clean":"2024-04-17T07:13:29.173538+0000","last_became_active":"2024-04-17T07:11:55.468594+0000","last_became_peered":"2024-04-17T07:11:55.468594+0000","last_unstale":"2024-04-17T07:13:29.173538+0000","last_undegraded":"2024-04-17T07:13:29.173538+0000","last_fullsized":"2024-04-17T07:13:29.173538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:21.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":124,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:56.477269+0000","last_change":"2024-04-17T07:11:54.473066+0000","last_active":"2024-04-17T07:13:56.477269+0000","last_peered":"2024-04-17T07:13:56.477269+0000","last_clean":"2024-04-17T07:13:56.477269+0000","last_became_active":"2024-04-17T07:11:54.472708+0000","last_became_peered":"2024-04-17T07:11:54.472708+0000","last_unstale":"2024-04-17T07:13:56.477269+0000","last_undegraded":"2024-04-17T07:13:56.477269+0000","last_fullsized":"2024-04-17T07:13:56.477269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:27.614819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.208953+0000","last_change":"2024-04-17T07:11:59.516939+0000","last_active":"2024-04-17T07:13:29.208953+0000","last_peered":"2024-04-17T07:13:29.208953+0000","last_clean":"2024-04-17T07:13:29.208953+0000","last_became_active":"2024-04-17T07:11:59.516333+0000","last_became_peered":"2024-04-17T07:11:59.516333+0000","last_unstale":"2024-04-17T07:13:29.208953+0000","last_undegraded":"2024-04-17T07:13:29.208953+0000","last_fullsized":"2024-04-17T07:13:29.208953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:15.648831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204860+0000","last_change":"2024-04-17T07:11:55.481006+0000","last_active":"2024-04-17T07:13:29.204860+0000","last_peered":"2024-04-17T07:13:29.204860+0000","last_clean":"2024-04-17T07:13:29.204860+0000","last_became_active":"2024-04-17T07:11:55.479751+0000","last_became_peered":"2024-04-17T07:11:55.479751+0000","last_unstale":"2024-04-17T07:13:29.204860+0000","last_undegraded":"2024-04-17T07:13:29.204860+0000","last_fullsized":"2024-04-17T07:13:29.204860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:13.004650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174955+0000","last_change":"2024-04-17T07:11:54.463106+0000","last_active":"2024-04-17T07:13:29.174955+0000","last_peered":"2024-04-17T07:13:29.174955+0000","last_clean":"2024-04-17T07:13:29.174955+0000","last_became_active":"2024-04-17T07:11:54.462691+0000","last_became_peered":"2024-04-17T07:11:54.462691+0000","last_unstale":"2024-04-17T07:13:29.174955+0000","last_undegraded":"2024-04-17T07:13:29.174955+0000","last_fullsized":"2024-04-17T07:13:29.174955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:34.302019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204929+0000","last_change":"2024-04-17T07:10:14.685883+0000","last_active":"2024-04-17T07:13:29.204929+0000","last_peered":"2024-04-17T07:13:29.204929+0000","last_clean":"2024-04-17T07:13:29.204929+0000","last_became_active":"2024-04-17T07:10:14.684551+0000","last_became_peered":"2024-04-17T07:10:14.684551+0000","last_unstale":"2024-04-17T07:13:29.204929+0000","last_undegraded":"2024-04-17T07:13:29.204929+0000","last_fullsized":"2024-04-17T07:13:29.204929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_clean_scrub_stamp":"2024-04-17T07:05:37.112317+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:04.265147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206420+0000","last_change":"2024-04-17T07:12:28.731807+0000","last_active":"2024-04-17T07:13:29.206420+0000","last_peered":"2024-04-17T07:13:29.206420+0000","last_clean":"2024-04-17T07:13:29.206420+0000","last_became_active":"2024-04-17T07:12:28.730656+0000","last_became_peered":"2024-04-17T07:12:28.730656+0000","last_unstale":"2024-04-17T07:13:29.206420+0000","last_undegraded":"2024-04-17T07:13:29.206420+0000","last_fullsized":"2024-04-17T07:13:29.206420+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:58.023333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068681+0000","last_change":"2024-04-17T07:12:28.727980+0000","last_active":"2024-04-17T07:13:57.068681+0000","last_peered":"2024-04-17T07:13:57.068681+0000","last_clean":"2024-04-17T07:13:57.068681+0000","last_became_active":"2024-04-17T07:12:28.727060+0000","last_became_peered":"2024-04-17T07:12:28.727060+0000","last_unstale":"2024-04-17T07:13:57.068681+0000","last_undegraded":"2024-04-17T07:13:57.068681+0000","last_fullsized":"2024-04-17T07:13:57.068681+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:33.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205324+0000","last_change":"2024-04-17T07:11:54.463180+0000","last_active":"2024-04-17T07:13:29.205324+0000","last_peered":"2024-04-17T07:13:29.205324+0000","last_clean":"2024-04-17T07:13:29.205324+0000","last_became_active":"2024-04-17T07:11:54.462793+0000","last_became_peered":"2024-04-17T07:11:54.462793+0000","last_unstale":"2024-04-17T07:13:29.205324+0000","last_undegraded":"2024-04-17T07:13:29.205324+0000","last_fullsized":"2024-04-17T07:13:29.205324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:19.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333167+0000","last_change":"2024-04-17T07:11:59.518355+0000","last_active":"2024-04-17T07:13:29.333167+0000","last_peered":"2024-04-17T07:13:29.333167+0000","last_clean":"2024-04-17T07:13:29.333167+0000","last_became_active":"2024-04-17T07:11:59.517619+0000","last_became_peered":"2024-04-17T07:11:59.517619+0000","last_unstale":"2024-04-17T07:13:29.333167+0000","last_undegraded":"2024-04-17T07:13:29.333167+0000","last_fullsized":"2024-04-17T07:13:29.333167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:36.826021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206249+0000","last_change":"2024-04-17T07:11:54.465285+0000","last_active":"2024-04-17T07:13:29.206249+0000","last_peered":"2024-04-17T07:13:29.206249+0000","last_clean":"2024-04-17T07:13:29.206249+0000","last_became_active":"2024-04-17T07:11:54.464818+0000","last_became_peered":"2024-04-17T07:11:54.464818+0000","last_unstale":"2024-04-17T07:13:29.206249+0000","last_undegraded":"2024-04-17T07:13:29.206249+0000","last_fullsized":"2024-04-17T07:13:29.206249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.062697+0000","last_change":"2024-04-17T07:11:55.470405+0000","last_active":"2024-04-17T07:13:57.062697+0000","last_peered":"2024-04-17T07:13:57.062697+0000","last_clean":"2024-04-17T07:13:57.062697+0000","last_became_active":"2024-04-17T07:11:55.470058+0000","last_became_peered":"2024-04-17T07:11:55.470058+0000","last_unstale":"2024-04-17T07:13:57.062697+0000","last_undegraded":"2024-04-17T07:13:57.062697+0000","last_fullsized":"2024-04-17T07:13:57.062697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:17.416775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.202874+0000","last_change":"2024-04-17T07:11:59.522040+0000","last_active":"2024-04-17T07:13:29.202874+0000","last_peered":"2024-04-17T07:13:29.202874+0000","last_clean":"2024-04-17T07:13:29.202874+0000","last_became_active":"2024-04-17T07:11:59.519771+0000","last_became_peered":"2024-04-17T07:11:59.519771+0000","last_unstale":"2024-04-17T07:13:29.202874+0000","last_undegraded":"2024-04-17T07:13:29.202874+0000","last_fullsized":"2024-04-17T07:13:29.202874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:58.203737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206703+0000","last_change":"2024-04-17T07:11:54.457954+0000","last_active":"2024-04-17T07:13:29.206703+0000","last_peered":"2024-04-17T07:13:29.206703+0000","last_clean":"2024-04-17T07:13:29.206703+0000","last_became_active":"2024-04-17T07:11:54.457560+0000","last_became_peered":"2024-04-17T07:11:54.457560+0000","last_unstale":"2024-04-17T07:13:29.206703+0000","last_undegraded":"2024-04-17T07:13:29.206703+0000","last_fullsized":"2024-04-17T07:13:29.206703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:31.919730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205366+0000","last_change":"2024-04-17T07:11:55.480475+0000","last_active":"2024-04-17T07:13:29.205366+0000","last_peered":"2024-04-17T07:13:29.205366+0000","last_clean":"2024-04-17T07:13:29.205366+0000","last_became_active":"2024-04-17T07:11:55.480060+0000","last_became_peered":"2024-04-17T07:11:55.480060+0000","last_unstale":"2024-04-17T07:13:29.205366+0000","last_undegraded":"2024-04-17T07:13:29.205366+0000","last_fullsized":"2024-04-17T07:13:29.205366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:30:23.600244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203636+0000","last_change":"2024-04-17T07:11:59.525422+0000","last_active":"2024-04-17T07:13:29.203636+0000","last_peered":"2024-04-17T07:13:29.203636+0000","last_clean":"2024-04-17T07:13:29.203636+0000","last_became_active":"2024-04-17T07:11:59.523649+0000","last_became_peered":"2024-04-17T07:11:59.523649+0000","last_unstale":"2024-04-17T07:13:29.203636+0000","last_undegraded":"2024-04-17T07:13:29.203636+0000","last_fullsized":"2024-04-17T07:13:29.203636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:16.653752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175977+0000","last_change":"2024-04-17T07:11:54.465824+0000","last_active":"2024-04-17T07:13:29.175977+0000","last_peered":"2024-04-17T07:13:29.175977+0000","last_clean":"2024-04-17T07:13:29.175977+0000","last_became_active":"2024-04-17T07:11:54.465348+0000","last_became_peered":"2024-04-17T07:11:54.465348+0000","last_unstale":"2024-04-17T07:13:29.175977+0000","last_undegraded":"2024-04-17T07:13:29.175977+0000","last_fullsized":"2024-04-17T07:13:29.175977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:17.729903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078824+0000","last_change":"2024-04-17T07:11:55.481666+0000","last_active":"2024-04-17T07:13:57.078824+0000","last_peered":"2024-04-17T07:13:57.078824+0000","last_clean":"2024-04-17T07:13:57.078824+0000","last_became_active":"2024-04-17T07:11:55.480332+0000","last_became_peered":"2024-04-17T07:11:55.480332+0000","last_unstale":"2024-04-17T07:13:57.078824+0000","last_undegraded":"2024-04-17T07:13:57.078824+0000","last_fullsized":"2024-04-17T07:13:57.078824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:19.825407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175471+0000","last_change":"2024-04-17T07:11:59.527850+0000","last_active":"2024-04-17T07:13:29.175471+0000","last_peered":"2024-04-17T07:13:29.175471+0000","last_clean":"2024-04-17T07:13:29.175471+0000","last_became_active":"2024-04-17T07:11:59.527306+0000","last_became_peered":"2024-04-17T07:11:59.527306+0000","last_unstale":"2024-04-17T07:13:29.175471+0000","last_undegraded":"2024-04-17T07:13:29.175471+0000","last_fullsized":"2024-04-17T07:13:29.175471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:35.906222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204070+0000","last_change":"2024-04-17T07:11:54.455862+0000","last_active":"2024-04-17T07:13:29.204070+0000","last_peered":"2024-04-17T07:13:29.204070+0000","last_clean":"2024-04-17T07:13:29.204070+0000","last_became_active":"2024-04-17T07:11:54.453904+0000","last_became_peered":"2024-04-17T07:11:54.453904+0000","last_unstale":"2024-04-17T07:13:29.204070+0000","last_undegraded":"2024-04-17T07:13:29.204070+0000","last_fullsized":"2024-04-17T07:13:29.204070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:01.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203748+0000","last_change":"2024-04-17T07:11:55.470315+0000","last_active":"2024-04-17T07:13:29.203748+0000","last_peered":"2024-04-17T07:13:29.203748+0000","last_clean":"2024-04-17T07:13:29.203748+0000","last_became_active":"2024-04-17T07:11:55.469829+0000","last_became_peered":"2024-04-17T07:11:55.469829+0000","last_unstale":"2024-04-17T07:13:29.203748+0000","last_undegraded":"2024-04-17T07:13:29.203748+0000","last_fullsized":"2024-04-17T07:13:29.203748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:48.197546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209882+0000","last_change":"2024-04-17T07:12:29.125447+0000","last_active":"2024-04-17T07:13:29.209882+0000","last_peered":"2024-04-17T07:13:29.209882+0000","last_clean":"2024-04-17T07:13:29.209882+0000","last_became_active":"2024-04-17T07:12:29.124551+0000","last_became_peered":"2024-04-17T07:12:29.124551+0000","last_unstale":"2024-04-17T07:13:29.209882+0000","last_undegraded":"2024-04-17T07:13:29.209882+0000","last_fullsized":"2024-04-17T07:13:29.209882+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:09.489755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204239+0000","last_change":"2024-04-17T07:11:54.453117+0000","last_active":"2024-04-17T07:13:29.204239+0000","last_peered":"2024-04-17T07:13:29.204239+0000","last_clean":"2024-04-17T07:13:29.204239+0000","last_became_active":"2024-04-17T07:11:54.452632+0000","last_became_peered":"2024-04-17T07:11:54.452632+0000","last_unstale":"2024-04-17T07:13:29.204239+0000","last_undegraded":"2024-04-17T07:13:29.204239+0000","last_fullsized":"2024-04-17T07:13:29.204239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:09.366149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.076821+0000","last_change":"2024-04-17T07:11:55.469991+0000","last_active":"2024-04-17T07:13:57.076821+0000","last_peered":"2024-04-17T07:13:57.076821+0000","last_clean":"2024-04-17T07:13:57.076821+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:13:57.076821+0000","last_undegraded":"2024-04-17T07:13:57.076821+0000","last_fullsized":"2024-04-17T07:13:57.076821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:02.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174957+0000","last_change":"2024-04-17T07:11:59.519817+0000","last_active":"2024-04-17T07:13:29.174957+0000","last_peered":"2024-04-17T07:13:29.174957+0000","last_clean":"2024-04-17T07:13:29.174957+0000","last_became_active":"2024-04-17T07:11:59.519108+0000","last_became_peered":"2024-04-17T07:11:59.519108+0000","last_unstale":"2024-04-17T07:13:29.174957+0000","last_undegraded":"2024-04-17T07:13:29.174957+0000","last_fullsized":"2024-04-17T07:13:29.174957+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:58.378837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204520+0000","last_change":"2024-04-17T07:11:54.463618+0000","last_active":"2024-04-17T07:13:29.204520+0000","last_peered":"2024-04-17T07:13:29.204520+0000","last_clean":"2024-04-17T07:13:29.204520+0000","last_became_active":"2024-04-17T07:11:54.463257+0000","last_became_peered":"2024-04-17T07:11:54.463257+0000","last_unstale":"2024-04-17T07:13:29.204520+0000","last_undegraded":"2024-04-17T07:13:29.204520+0000","last_fullsized":"2024-04-17T07:13:29.204520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:20.135338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204206+0000","last_change":"2024-04-17T07:11:55.487471+0000","last_active":"2024-04-17T07:13:29.204206+0000","last_peered":"2024-04-17T07:13:29.204206+0000","last_clean":"2024-04-17T07:13:29.204206+0000","last_became_active":"2024-04-17T07:11:55.487024+0000","last_became_peered":"2024-04-17T07:11:55.487024+0000","last_unstale":"2024-04-17T07:13:29.204206+0000","last_undegraded":"2024-04-17T07:13:29.204206+0000","last_fullsized":"2024-04-17T07:13:29.204206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:36.851210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175106+0000","last_change":"2024-04-17T07:11:59.527017+0000","last_active":"2024-04-17T07:13:29.175106+0000","last_peered":"2024-04-17T07:13:29.175106+0000","last_clean":"2024-04-17T07:13:29.175106+0000","last_became_active":"2024-04-17T07:11:59.526541+0000","last_became_peered":"2024-04-17T07:11:59.526541+0000","last_unstale":"2024-04-17T07:13:29.175106+0000","last_undegraded":"2024-04-17T07:13:29.175106+0000","last_fullsized":"2024-04-17T07:13:29.175106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:54.411647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204444+0000","last_change":"2024-04-17T07:11:54.465671+0000","last_active":"2024-04-17T07:13:29.204444+0000","last_peered":"2024-04-17T07:13:29.204444+0000","last_clean":"2024-04-17T07:13:29.204444+0000","last_became_active":"2024-04-17T07:11:54.464902+0000","last_became_peered":"2024-04-17T07:11:54.464902+0000","last_unstale":"2024-04-17T07:13:29.204444+0000","last_undegraded":"2024-04-17T07:13:29.204444+0000","last_fullsized":"2024-04-17T07:13:29.204444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:13:38.232965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206970+0000","last_change":"2024-04-17T07:11:55.484259+0000","last_active":"2024-04-17T07:13:29.206970+0000","last_peered":"2024-04-17T07:13:29.206970+0000","last_clean":"2024-04-17T07:13:29.206970+0000","last_became_active":"2024-04-17T07:11:55.483911+0000","last_became_peered":"2024-04-17T07:11:55.483911+0000","last_unstale":"2024-04-17T07:13:29.206970+0000","last_undegraded":"2024-04-17T07:13:29.206970+0000","last_fullsized":"2024-04-17T07:13:29.206970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:54.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204080+0000","last_change":"2024-04-17T07:11:59.525369+0000","last_active":"2024-04-17T07:13:29.204080+0000","last_peered":"2024-04-17T07:13:29.204080+0000","last_clean":"2024-04-17T07:13:29.204080+0000","last_became_active":"2024-04-17T07:11:59.524809+0000","last_became_peered":"2024-04-17T07:11:59.524809+0000","last_unstale":"2024-04-17T07:13:29.204080+0000","last_undegraded":"2024-04-17T07:13:29.204080+0000","last_fullsized":"2024-04-17T07:13:29.204080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:20.511001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":123,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:54.475327+0000","last_change":"2024-04-17T07:11:54.459079+0000","last_active":"2024-04-17T07:13:54.475327+0000","last_peered":"2024-04-17T07:13:54.475327+0000","last_clean":"2024-04-17T07:13:54.475327+0000","last_became_active":"2024-04-17T07:11:54.457008+0000","last_became_peered":"2024-04-17T07:11:54.457008+0000","last_unstale":"2024-04-17T07:13:54.475327+0000","last_undegraded":"2024-04-17T07:13:54.475327+0000","last_fullsized":"2024-04-17T07:13:54.475327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:36.084512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209690+0000","last_change":"2024-04-17T07:11:55.484453+0000","last_active":"2024-04-17T07:13:29.209690+0000","last_peered":"2024-04-17T07:13:29.209690+0000","last_clean":"2024-04-17T07:13:29.209690+0000","last_became_active":"2024-04-17T07:11:55.483152+0000","last_became_peered":"2024-04-17T07:11:55.483152+0000","last_unstale":"2024-04-17T07:13:29.209690+0000","last_undegraded":"2024-04-17T07:13:29.209690+0000","last_fullsized":"2024-04-17T07:13:29.209690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:32.224868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.202572+0000","last_change":"2024-04-17T07:11:59.521487+0000","last_active":"2024-04-17T07:13:29.202572+0000","last_peered":"2024-04-17T07:13:29.202572+0000","last_clean":"2024-04-17T07:13:29.202572+0000","last_became_active":"2024-04-17T07:11:59.521060+0000","last_became_peered":"2024-04-17T07:11:59.521060+0000","last_unstale":"2024-04-17T07:13:29.202572+0000","last_undegraded":"2024-04-17T07:13:29.202572+0000","last_fullsized":"2024-04-17T07:13:29.202572+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:06.692434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174431+0000","last_change":"2024-04-17T07:11:54.459368+0000","last_active":"2024-04-17T07:13:29.174431+0000","last_peered":"2024-04-17T07:13:29.174431+0000","last_clean":"2024-04-17T07:13:29.174431+0000","last_became_active":"2024-04-17T07:11:54.458973+0000","last_became_peered":"2024-04-17T07:11:54.458973+0000","last_unstale":"2024-04-17T07:13:29.174431+0000","last_undegraded":"2024-04-17T07:13:29.174431+0000","last_fullsized":"2024-04-17T07:13:29.174431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:22.034110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204896+0000","last_change":"2024-04-17T07:11:55.470648+0000","last_active":"2024-04-17T07:13:29.204896+0000","last_peered":"2024-04-17T07:13:29.204896+0000","last_clean":"2024-04-17T07:13:29.204896+0000","last_became_active":"2024-04-17T07:11:55.469666+0000","last_became_peered":"2024-04-17T07:11:55.469666+0000","last_unstale":"2024-04-17T07:13:29.204896+0000","last_undegraded":"2024-04-17T07:13:29.204896+0000","last_fullsized":"2024-04-17T07:13:29.204896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:59.208872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172289+0000","last_change":"2024-04-17T07:11:59.524027+0000","last_active":"2024-04-17T07:13:29.172289+0000","last_peered":"2024-04-17T07:13:29.172289+0000","last_clean":"2024-04-17T07:13:29.172289+0000","last_became_active":"2024-04-17T07:11:59.522764+0000","last_became_peered":"2024-04-17T07:11:59.522764+0000","last_unstale":"2024-04-17T07:13:29.172289+0000","last_undegraded":"2024-04-17T07:13:29.172289+0000","last_fullsized":"2024-04-17T07:13:29.172289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:40.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173104+0000","last_change":"2024-04-17T07:11:54.472884+0000","last_active":"2024-04-17T07:13:29.173104+0000","last_peered":"2024-04-17T07:13:29.173104+0000","last_clean":"2024-04-17T07:13:29.173104+0000","last_became_active":"2024-04-17T07:11:54.472305+0000","last_became_peered":"2024-04-17T07:11:54.472305+0000","last_unstale":"2024-04-17T07:13:29.173104+0000","last_undegraded":"2024-04-17T07:13:29.173104+0000","last_fullsized":"2024-04-17T07:13:29.173104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:21.031442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.210143+0000","last_change":"2024-04-17T07:11:55.479982+0000","last_active":"2024-04-17T07:13:29.210143+0000","last_peered":"2024-04-17T07:13:29.210143+0000","last_clean":"2024-04-17T07:13:29.210143+0000","last_became_active":"2024-04-17T07:11:55.477594+0000","last_became_peered":"2024-04-17T07:11:55.477594+0000","last_unstale":"2024-04-17T07:13:29.210143+0000","last_undegraded":"2024-04-17T07:13:29.210143+0000","last_fullsized":"2024-04-17T07:13:29.210143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:23.507312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070282+0000","last_change":"2024-04-17T07:11:55.480941+0000","last_active":"2024-04-17T07:13:57.070282+0000","last_peered":"2024-04-17T07:13:57.070282+0000","last_clean":"2024-04-17T07:13:57.070282+0000","last_became_active":"2024-04-17T07:11:55.480180+0000","last_became_peered":"2024-04-17T07:11:55.480180+0000","last_unstale":"2024-04-17T07:13:57.070282+0000","last_undegraded":"2024-04-17T07:13:57.070282+0000","last_fullsized":"2024-04-17T07:13:57.070282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:16.321806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204782+0000","last_change":"2024-04-17T07:11:55.477663+0000","last_active":"2024-04-17T07:13:29.204782+0000","last_peered":"2024-04-17T07:13:29.204782+0000","last_clean":"2024-04-17T07:13:29.204782+0000","last_became_active":"2024-04-17T07:11:55.477304+0000","last_became_peered":"2024-04-17T07:11:55.477304+0000","last_unstale":"2024-04-17T07:13:29.204782+0000","last_undegraded":"2024-04-17T07:13:29.204782+0000","last_fullsized":"2024-04-17T07:13:29.204782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:00.796633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204220+0000","last_change":"2024-04-17T07:11:55.475536+0000","last_active":"2024-04-17T07:13:29.204220+0000","last_peered":"2024-04-17T07:13:29.204220+0000","last_clean":"2024-04-17T07:13:29.204220+0000","last_became_active":"2024-04-17T07:11:55.475138+0000","last_became_peered":"2024-04-17T07:11:55.475138+0000","last_unstale":"2024-04-17T07:13:29.204220+0000","last_undegraded":"2024-04-17T07:13:29.204220+0000","last_fullsized":"2024-04-17T07:13:29.204220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:47.849076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.080994+0000","last_change":"2024-04-17T07:11:55.483742+0000","last_active":"2024-04-17T07:13:57.080994+0000","last_peered":"2024-04-17T07:13:57.080994+0000","last_clean":"2024-04-17T07:13:57.080994+0000","last_became_active":"2024-04-17T07:11:55.483044+0000","last_became_peered":"2024-04-17T07:11:55.483044+0000","last_unstale":"2024-04-17T07:13:57.080994+0000","last_undegraded":"2024-04-17T07:13:57.080994+0000","last_fullsized":"2024-04-17T07:13:57.080994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:18.972137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078144+0000","last_change":"2024-04-17T07:11:55.471266+0000","last_active":"2024-04-17T07:13:57.078144+0000","last_peered":"2024-04-17T07:13:57.078144+0000","last_clean":"2024-04-17T07:13:57.078144+0000","last_became_active":"2024-04-17T07:11:55.470897+0000","last_became_peered":"2024-04-17T07:11:55.470897+0000","last_unstale":"2024-04-17T07:13:57.078144+0000","last_undegraded":"2024-04-17T07:13:57.078144+0000","last_fullsized":"2024-04-17T07:13:57.078144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:25.811064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.066053+0000","last_change":"2024-04-17T07:11:55.475860+0000","last_active":"2024-04-17T07:13:57.066053+0000","last_peered":"2024-04-17T07:13:57.066053+0000","last_clean":"2024-04-17T07:13:57.066053+0000","last_became_active":"2024-04-17T07:11:55.475522+0000","last_became_peered":"2024-04-17T07:11:55.475522+0000","last_unstale":"2024-04-17T07:13:57.066053+0000","last_undegraded":"2024-04-17T07:13:57.066053+0000","last_fullsized":"2024-04-17T07:13:57.066053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:41.548964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205802+0000","last_change":"2024-04-17T07:11:55.478815+0000","last_active":"2024-04-17T07:13:29.205802+0000","last_peered":"2024-04-17T07:13:29.205802+0000","last_clean":"2024-04-17T07:13:29.205802+0000","last_became_active":"2024-04-17T07:11:55.478347+0000","last_became_peered":"2024-04-17T07:11:55.478347+0000","last_unstale":"2024-04-17T07:13:29.205802+0000","last_undegraded":"2024-04-17T07:13:29.205802+0000","last_fullsized":"2024-04-17T07:13:29.205802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:16.465511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205575+0000","last_change":"2024-04-17T07:11:55.474208+0000","last_active":"2024-04-17T07:13:29.205575+0000","last_peered":"2024-04-17T07:13:29.205575+0000","last_clean":"2024-04-17T07:13:29.205575+0000","last_became_active":"2024-04-17T07:11:55.472042+0000","last_became_peered":"2024-04-17T07:11:55.472042+0000","last_unstale":"2024-04-17T07:13:29.205575+0000","last_undegraded":"2024-04-17T07:13:29.205575+0000","last_fullsized":"2024-04-17T07:13:29.205575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:37:24.248506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172797+0000","last_change":"2024-04-17T07:11:55.477257+0000","last_active":"2024-04-17T07:13:29.172797+0000","last_peered":"2024-04-17T07:13:29.172797+0000","last_clean":"2024-04-17T07:13:29.172797+0000","last_became_active":"2024-04-17T07:11:55.473491+0000","last_became_peered":"2024-04-17T07:11:55.473491+0000","last_unstale":"2024-04-17T07:13:29.172797+0000","last_undegraded":"2024-04-17T07:13:29.172797+0000","last_fullsized":"2024-04-17T07:13:29.172797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:33.737485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204068+0000","last_change":"2024-04-17T07:11:55.474930+0000","last_active":"2024-04-17T07:13:29.204068+0000","last_peered":"2024-04-17T07:13:29.204068+0000","last_clean":"2024-04-17T07:13:29.204068+0000","last_became_active":"2024-04-17T07:11:55.472007+0000","last_became_peered":"2024-04-17T07:11:55.472007+0000","last_unstale":"2024-04-17T07:13:29.204068+0000","last_undegraded":"2024-04-17T07:13:29.204068+0000","last_fullsized":"2024-04-17T07:13:29.204068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:07.111267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204914+0000","last_change":"2024-04-17T07:11:55.492422+0000","last_active":"2024-04-17T07:13:29.204914+0000","last_peered":"2024-04-17T07:13:29.204914+0000","last_clean":"2024-04-17T07:13:29.204914+0000","last_became_active":"2024-04-17T07:11:55.491906+0000","last_became_peered":"2024-04-17T07:11:55.491906+0000","last_unstale":"2024-04-17T07:13:29.204914+0000","last_undegraded":"2024-04-17T07:13:29.204914+0000","last_fullsized":"2024-04-17T07:13:29.204914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:13:05.322586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077807+0000","last_change":"2024-04-17T07:11:55.479436+0000","last_active":"2024-04-17T07:13:57.077807+0000","last_peered":"2024-04-17T07:13:57.077807+0000","last_clean":"2024-04-17T07:13:57.077807+0000","last_became_active":"2024-04-17T07:11:55.478982+0000","last_became_peered":"2024-04-17T07:11:55.478982+0000","last_unstale":"2024-04-17T07:13:57.077807+0000","last_undegraded":"2024-04-17T07:13:57.077807+0000","last_fullsized":"2024-04-17T07:13:57.077807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:20.663457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070158+0000","last_change":"2024-04-17T07:11:55.476708+0000","last_active":"2024-04-17T07:13:57.070158+0000","last_peered":"2024-04-17T07:13:57.070158+0000","last_clean":"2024-04-17T07:13:57.070158+0000","last_became_active":"2024-04-17T07:11:55.476344+0000","last_became_peered":"2024-04-17T07:11:55.476344+0000","last_unstale":"2024-04-17T07:13:57.070158+0000","last_undegraded":"2024-04-17T07:13:57.070158+0000","last_fullsized":"2024-04-17T07:13:57.070158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:58.874987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279575,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69063,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":893,"num_read_kb":524,"num_write":245,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":206691,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":520240,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645361,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1169194,"data_compressed":64060,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":524154,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070526,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":520324,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332165,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1116933,"data_compressed":62211,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495692,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":515562,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":579606,"data_compressed":28497,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738485,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":517018,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":578191,"data_compressed":28497,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182496,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":515270,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920858,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":519636,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757331,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":1112515,"data_compressed":62211,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64766,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63351,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63986,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:14:01.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph pg dump --format=json 2024-04-17T07:14:01.981 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:14:02.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:02 smithi098 ceph-mon[35418]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:14:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:02 smithi098 ceph-mon[35418]: pgmap v430: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-17T07:14:02.503 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:02 smithi086 ceph-mon[31733]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:14:02.503 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:02 smithi086 ceph-mon[31733]: pgmap v430: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-17T07:14:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:02 smithi103 ceph-mon[35495]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:14:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:02 smithi103 ceph-mon[35495]: pgmap v430: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-17T07:14:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.338+0000 7f7b71400700 1 -- 172.21.15.86:0/201446346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b6c1020e0 msgr2=0x7f7b6c10eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.338+0000 7f7b71400700 1 --2- 172.21.15.86:0/201446346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b6c1020e0 0x7f7b6c10eda0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009990 tx=0x7f7b54009ca0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.338+0000 7f7b71400700 1 -- 172.21.15.86:0/201446346 shutdown_connections 2024-04-17T07:14:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.338+0000 7f7b71400700 1 --2- 172.21.15.86:0/201446346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b6c10f2e0 0x7f7b6c1116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.338+0000 7f7b71400700 1 --2- 172.21.15.86:0/201446346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b6c101730 0x7f7b6c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.338+0000 7f7b71400700 1 --2- 172.21.15.86:0/201446346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b6c1020e0 0x7f7b6c10eda0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.338+0000 7f7b71400700 1 -- 172.21.15.86:0/201446346 >> 172.21.15.86:0/201446346 conn(0x7f7b6c0fd370 msgr2=0x7f7b6c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:04.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.338+0000 7f7b71400700 1 -- 172.21.15.86:0/201446346 shutdown_connections 2024-04-17T07:14:04.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.338+0000 7f7b71400700 1 -- 172.21.15.86:0/201446346 wait complete. 2024-04-17T07:14:04.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.339+0000 7f7b71400700 1 Processor -- start 2024-04-17T07:14:04.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.339+0000 7f7b71400700 1 -- start start 2024-04-17T07:14:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.339+0000 7f7b71400700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b6c101730 0x7f7b6c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.339+0000 7f7b71400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b6c1020e0 0x7f7b6c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.339+0000 7f7b71400700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b6c10f2e0 0x7f7b6c19fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.339+0000 7f7b71400700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c114d40 con 0x7f7b6c1020e0 2024-04-17T07:14:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.339+0000 7f7b71400700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c114bc0 con 0x7f7b6c101730 2024-04-17T07:14:04.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.339+0000 7f7b71400700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c114ec0 con 0x7f7b6c10f2e0 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b63400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b6c1020e0 0x7f7b6c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b63400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b6c1020e0 0x7f7b6c19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42690/0 (socket says 172.21.15.86:42690) 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b63400700 1 -- 172.21.15.86:0/899230488 learned_addr learned my addr 172.21.15.86:0/899230488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b6be00700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b6c101730 0x7f7b6c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b70a00700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b6c10f2e0 0x7f7b6c19fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b63400700 1 -- 172.21.15.86:0/899230488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b6c10f2e0 msgr2=0x7f7b6c19fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b63400700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b6c10f2e0 0x7f7b6c19fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b63400700 1 -- 172.21.15.86:0/899230488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b6c101730 msgr2=0x7f7b6c19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b63400700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b6c101730 0x7f7b6c19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b63400700 1 -- 172.21.15.86:0/899230488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c1a0170 con 0x7f7b6c1020e0 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.340+0000 7f7b70a00700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b6c10f2e0 0x7f7b6c19fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:04.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.341+0000 7f7b63400700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b6c1020e0 0x7f7b6c19b6f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7b54003ab0 tx=0x7f7b54003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.341+0000 7f7b69600700 1 -- 172.21.15.86:0/899230488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54027070 con 0x7f7b6c1020e0 2024-04-17T07:14:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.341+0000 7f7b69600700 1 -- 172.21.15.86:0/899230488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b5400bdc0 con 0x7f7b6c1020e0 2024-04-17T07:14:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.341+0000 7f7b71400700 1 -- 172.21.15.86:0/899230488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c1a0400 con 0x7f7b6c1020e0 2024-04-17T07:14:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.341+0000 7f7b69600700 1 -- 172.21.15.86:0/899230488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54022de0 con 0x7f7b6c1020e0 2024-04-17T07:14:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.341+0000 7f7b71400700 1 -- 172.21.15.86:0/899230488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1a7ce0 con 0x7f7b6c1020e0 2024-04-17T07:14:04.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.342+0000 7f7b71400700 1 -- 172.21.15.86:0/899230488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c069740 con 0x7f7b6c1020e0 2024-04-17T07:14:04.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.346+0000 7f7b69600700 1 -- 172.21.15.86:0/899230488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7b5402a070 con 0x7f7b6c1020e0 2024-04-17T07:14:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.347+0000 7f7b69600700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b500a3c40 0x7f7b500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.347+0000 7f7b69600700 1 -- 172.21.15.86:0/899230488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7b540d1a00 con 0x7f7b6c1020e0 2024-04-17T07:14:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.348+0000 7f7b6be00700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b500a3c40 0x7f7b500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.348+0000 7f7b6be00700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b500a3c40 0x7f7b500a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c005ea0 tx=0x7f7b5c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.350+0000 7f7b69600700 1 -- 172.21.15.86:0/899230488 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7b54031020 con 0x7f7b6c1020e0 2024-04-17T07:14:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:04 smithi098 ceph-mon[35418]: pgmap v431: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-17T07:14:04.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:04 smithi103 ceph-mon[35495]: pgmap v431: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-17T07:14:04.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.572+0000 7f7b71400700 1 -- 172.21.15.86:0/899230488 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b6c19c4c0 con 0x7f7b500a3c40 2024-04-17T07:14:04.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:04 smithi086 ceph-mon[31733]: pgmap v431: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-17T07:14:04.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.588+0000 7f7b69600700 1 -- 172.21.15.86:0/899230488 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340965 (secure 0 0 0) 0x7f7b6c19c4c0 con 0x7f7b500a3c40 2024-04-17T07:14:04.594 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.595+0000 7f7b71400700 1 -- 172.21.15.86:0/899230488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b500a3c40 msgr2=0x7f7b500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.595+0000 7f7b71400700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b500a3c40 0x7f7b500a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c005ea0 tx=0x7f7b5c005e30 comp rx=0 tx=0).stop 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.595+0000 7f7b71400700 1 -- 172.21.15.86:0/899230488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b6c1020e0 msgr2=0x7f7b6c19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.595+0000 7f7b71400700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b6c1020e0 0x7f7b6c19b6f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7b54003ab0 tx=0x7f7b54003ae0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.595+0000 7f7b71400700 1 -- 172.21.15.86:0/899230488 shutdown_connections 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.595+0000 7f7b71400700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b500a3c40 0x7f7b500a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.595+0000 7f7b71400700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b6c10f2e0 0x7f7b6c19fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.595+0000 7f7b71400700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b6c101730 0x7f7b6c19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.595+0000 7f7b71400700 1 --2- 172.21.15.86:0/899230488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b6c1020e0 0x7f7b6c19b6f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.595+0000 7f7b71400700 1 -- 172.21.15.86:0/899230488 >> 172.21.15.86:0/899230488 conn(0x7f7b6c0fd370 msgr2=0x7f7b6c1104a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.596+0000 7f7b71400700 1 -- 172.21.15.86:0/899230488 shutdown_connections 2024-04-17T07:14:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:04.596+0000 7f7b71400700 1 -- 172.21.15.86:0/899230488 wait complete. 2024-04-17T07:14:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-17T07:14:05.537 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-04-17T07:14:03.102327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659806,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1071,"num_read_kb":660,"num_write":431,"num_write_kb":3084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181358,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":854,"ondisk_log_size":854,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365896,"kb_used_data":12584,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575928,"statfs":{"total":1151940427776,"available":1151565750272,"internally_reserved":0,"allocated":12886016,"data_stored":8189564,"data_compressed":432012,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":77,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005400"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078599+0000","last_change":"2024-04-17T07:11:55.486746+0000","last_active":"2024-04-17T07:13:57.078599+0000","last_peered":"2024-04-17T07:13:57.078599+0000","last_clean":"2024-04-17T07:13:57.078599+0000","last_became_active":"2024-04-17T07:11:55.486335+0000","last_became_peered":"2024-04-17T07:11:55.486335+0000","last_unstale":"2024-04-17T07:13:57.078599+0000","last_undegraded":"2024-04-17T07:13:57.078599+0000","last_fullsized":"2024-04-17T07:13:57.078599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:48.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082087+0000","last_change":"2024-04-17T07:11:55.481832+0000","last_active":"2024-04-17T07:13:57.082087+0000","last_peered":"2024-04-17T07:13:57.082087+0000","last_clean":"2024-04-17T07:13:57.082087+0000","last_became_active":"2024-04-17T07:11:55.476428+0000","last_became_peered":"2024-04-17T07:11:55.476428+0000","last_unstale":"2024-04-17T07:13:57.082087+0000","last_undegraded":"2024-04-17T07:13:57.082087+0000","last_fullsized":"2024-04-17T07:13:57.082087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:04.452542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083395+0000","last_change":"2024-04-17T07:11:55.483721+0000","last_active":"2024-04-17T07:13:57.083395+0000","last_peered":"2024-04-17T07:13:57.083395+0000","last_clean":"2024-04-17T07:13:57.083395+0000","last_became_active":"2024-04-17T07:11:55.483268+0000","last_became_peered":"2024-04-17T07:11:55.483268+0000","last_unstale":"2024-04-17T07:13:57.083395+0000","last_undegraded":"2024-04-17T07:13:57.083395+0000","last_fullsized":"2024-04-17T07:13:57.083395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:41.584783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067540+0000","last_change":"2024-04-17T07:11:55.485114+0000","last_active":"2024-04-17T07:13:57.067540+0000","last_peered":"2024-04-17T07:13:57.067540+0000","last_clean":"2024-04-17T07:13:57.067540+0000","last_became_active":"2024-04-17T07:11:55.483913+0000","last_became_peered":"2024-04-17T07:11:55.483913+0000","last_unstale":"2024-04-17T07:13:57.067540+0000","last_undegraded":"2024-04-17T07:13:57.067540+0000","last_fullsized":"2024-04-17T07:13:57.067540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:46.211121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.052268+0000","last_change":"2024-04-17T07:11:55.491952+0000","last_active":"2024-04-17T07:13:57.052268+0000","last_peered":"2024-04-17T07:13:57.052268+0000","last_clean":"2024-04-17T07:13:57.052268+0000","last_became_active":"2024-04-17T07:11:55.491238+0000","last_became_peered":"2024-04-17T07:11:55.491238+0000","last_unstale":"2024-04-17T07:13:57.052268+0000","last_undegraded":"2024-04-17T07:13:57.052268+0000","last_fullsized":"2024-04-17T07:13:57.052268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:06.759797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068603+0000","last_change":"2024-04-17T07:11:55.471124+0000","last_active":"2024-04-17T07:13:57.068603+0000","last_peered":"2024-04-17T07:13:57.068603+0000","last_clean":"2024-04-17T07:13:57.068603+0000","last_became_active":"2024-04-17T07:11:55.470781+0000","last_became_peered":"2024-04-17T07:11:55.470781+0000","last_unstale":"2024-04-17T07:13:57.068603+0000","last_undegraded":"2024-04-17T07:13:57.068603+0000","last_fullsized":"2024-04-17T07:13:57.068603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:45.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.063934+0000","last_change":"2024-04-17T07:11:55.469099+0000","last_active":"2024-04-17T07:13:57.063934+0000","last_peered":"2024-04-17T07:13:57.063934+0000","last_clean":"2024-04-17T07:13:57.063934+0000","last_became_active":"2024-04-17T07:11:55.468584+0000","last_became_peered":"2024-04-17T07:11:55.468584+0000","last_unstale":"2024-04-17T07:13:57.063934+0000","last_undegraded":"2024-04-17T07:13:57.063934+0000","last_fullsized":"2024-04-17T07:13:57.063934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:52.834846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070402+0000","last_change":"2024-04-17T07:11:55.473794+0000","last_active":"2024-04-17T07:13:57.070402+0000","last_peered":"2024-04-17T07:13:57.070402+0000","last_clean":"2024-04-17T07:13:57.070402+0000","last_became_active":"2024-04-17T07:11:55.471919+0000","last_became_peered":"2024-04-17T07:11:55.471919+0000","last_unstale":"2024-04-17T07:13:57.070402+0000","last_undegraded":"2024-04-17T07:13:57.070402+0000","last_fullsized":"2024-04-17T07:13:57.070402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:16.085117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078902+0000","last_change":"2024-04-17T07:11:55.482445+0000","last_active":"2024-04-17T07:13:57.078902+0000","last_peered":"2024-04-17T07:13:57.078902+0000","last_clean":"2024-04-17T07:13:57.078902+0000","last_became_active":"2024-04-17T07:11:55.481552+0000","last_became_peered":"2024-04-17T07:11:55.481552+0000","last_unstale":"2024-04-17T07:13:57.078902+0000","last_undegraded":"2024-04-17T07:13:57.078902+0000","last_fullsized":"2024-04-17T07:13:57.078902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:48.733585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071209+0000","last_change":"2024-04-17T07:11:55.476726+0000","last_active":"2024-04-17T07:13:57.071209+0000","last_peered":"2024-04-17T07:13:57.071209+0000","last_clean":"2024-04-17T07:13:57.071209+0000","last_became_active":"2024-04-17T07:11:55.476320+0000","last_became_peered":"2024-04-17T07:11:55.476320+0000","last_unstale":"2024-04-17T07:13:57.071209+0000","last_undegraded":"2024-04-17T07:13:57.071209+0000","last_fullsized":"2024-04-17T07:13:57.071209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.230181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084001+0000","last_change":"2024-04-17T07:11:55.485798+0000","last_active":"2024-04-17T07:13:57.084001+0000","last_peered":"2024-04-17T07:13:57.084001+0000","last_clean":"2024-04-17T07:13:57.084001+0000","last_became_active":"2024-04-17T07:11:55.484015+0000","last_became_peered":"2024-04-17T07:11:55.484015+0000","last_unstale":"2024-04-17T07:13:57.084001+0000","last_undegraded":"2024-04-17T07:13:57.084001+0000","last_fullsized":"2024-04-17T07:13:57.084001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:37.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.061275+0000","last_change":"2024-04-17T07:12:28.729834+0000","last_active":"2024-04-17T07:13:57.061275+0000","last_peered":"2024-04-17T07:13:57.061275+0000","last_clean":"2024-04-17T07:13:57.061275+0000","last_became_active":"2024-04-17T07:12:28.728694+0000","last_became_peered":"2024-04-17T07:12:28.728694+0000","last_unstale":"2024-04-17T07:13:57.061275+0000","last_undegraded":"2024-04-17T07:13:57.061275+0000","last_fullsized":"2024-04-17T07:13:57.061275+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:46.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067835+0000","last_change":"2024-04-17T07:13:29.210325+0000","last_active":"2024-04-17T07:13:57.067835+0000","last_peered":"2024-04-17T07:13:57.067835+0000","last_clean":"2024-04-17T07:13:57.067835+0000","last_became_active":"2024-04-17T07:13:29.209759+0000","last_became_peered":"2024-04-17T07:13:29.209759+0000","last_unstale":"2024-04-17T07:13:57.067835+0000","last_undegraded":"2024-04-17T07:13:57.067835+0000","last_fullsized":"2024-04-17T07:13:57.067835+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:17.168931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083523+0000","last_change":"2024-04-17T07:11:55.473839+0000","last_active":"2024-04-17T07:13:57.083523+0000","last_peered":"2024-04-17T07:13:57.083523+0000","last_clean":"2024-04-17T07:13:57.083523+0000","last_became_active":"2024-04-17T07:11:55.473472+0000","last_became_peered":"2024-04-17T07:11:55.473472+0000","last_unstale":"2024-04-17T07:13:57.083523+0000","last_undegraded":"2024-04-17T07:13:57.083523+0000","last_fullsized":"2024-04-17T07:13:57.083523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:02.263272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081336+0000","last_change":"2024-04-17T07:11:55.484820+0000","last_active":"2024-04-17T07:13:57.081336+0000","last_peered":"2024-04-17T07:13:57.081336+0000","last_clean":"2024-04-17T07:13:57.081336+0000","last_became_active":"2024-04-17T07:11:55.484314+0000","last_became_peered":"2024-04-17T07:11:55.484314+0000","last_unstale":"2024-04-17T07:13:57.081336+0000","last_undegraded":"2024-04-17T07:13:57.081336+0000","last_fullsized":"2024-04-17T07:13:57.081336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.186912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083308+0000","last_change":"2024-04-17T07:11:55.492018+0000","last_active":"2024-04-17T07:13:57.083308+0000","last_peered":"2024-04-17T07:13:57.083308+0000","last_clean":"2024-04-17T07:13:57.083308+0000","last_became_active":"2024-04-17T07:11:55.490512+0000","last_became_peered":"2024-04-17T07:11:55.490512+0000","last_unstale":"2024-04-17T07:13:57.083308+0000","last_undegraded":"2024-04-17T07:13:57.083308+0000","last_fullsized":"2024-04-17T07:13:57.083308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:45.436270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081355+0000","last_change":"2024-04-17T07:11:55.487210+0000","last_active":"2024-04-17T07:13:57.081355+0000","last_peered":"2024-04-17T07:13:57.081355+0000","last_clean":"2024-04-17T07:13:57.081355+0000","last_became_active":"2024-04-17T07:11:55.485713+0000","last_became_peered":"2024-04-17T07:11:55.485713+0000","last_unstale":"2024-04-17T07:13:57.081355+0000","last_undegraded":"2024-04-17T07:13:57.081355+0000","last_fullsized":"2024-04-17T07:13:57.081355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:54.495950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082677+0000","last_change":"2024-04-17T07:11:55.482705+0000","last_active":"2024-04-17T07:13:57.082677+0000","last_peered":"2024-04-17T07:13:57.082677+0000","last_clean":"2024-04-17T07:13:57.082677+0000","last_became_active":"2024-04-17T07:11:55.482349+0000","last_became_peered":"2024-04-17T07:11:55.482349+0000","last_unstale":"2024-04-17T07:13:57.082677+0000","last_undegraded":"2024-04-17T07:13:57.082677+0000","last_fullsized":"2024-04-17T07:13:57.082677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:57.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.058661+0000","last_change":"2024-04-17T07:11:55.467065+0000","last_active":"2024-04-17T07:13:57.058661+0000","last_peered":"2024-04-17T07:13:57.058661+0000","last_clean":"2024-04-17T07:13:57.058661+0000","last_became_active":"2024-04-17T07:11:55.466535+0000","last_became_peered":"2024-04-17T07:11:55.466535+0000","last_unstale":"2024-04-17T07:13:57.058661+0000","last_undegraded":"2024-04-17T07:13:57.058661+0000","last_fullsized":"2024-04-17T07:13:57.058661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:49.955453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204248+0000","last_change":"2024-04-17T07:11:59.515778+0000","last_active":"2024-04-17T07:13:29.204248+0000","last_peered":"2024-04-17T07:13:29.204248+0000","last_clean":"2024-04-17T07:13:29.204248+0000","last_became_active":"2024-04-17T07:11:59.515266+0000","last_became_peered":"2024-04-17T07:11:59.515266+0000","last_unstale":"2024-04-17T07:13:29.204248+0000","last_undegraded":"2024-04-17T07:13:29.204248+0000","last_fullsized":"2024-04-17T07:13:29.204248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:43.084724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'13","reported_seq":137,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:01.477470+0000","last_change":"2024-04-17T07:11:54.467060+0000","last_active":"2024-04-17T07:14:01.477470+0000","last_peered":"2024-04-17T07:14:01.477470+0000","last_clean":"2024-04-17T07:14:01.477470+0000","last_became_active":"2024-04-17T07:11:54.466687+0000","last_became_peered":"2024-04-17T07:11:54.466687+0000","last_unstale":"2024-04-17T07:14:01.477470+0000","last_undegraded":"2024-04-17T07:14:01.477470+0000","last_fullsized":"2024-04-17T07:14:01.477470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:57.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.064498+0000","last_change":"2024-04-17T07:11:55.469857+0000","last_active":"2024-04-17T07:13:57.064498+0000","last_peered":"2024-04-17T07:13:57.064498+0000","last_clean":"2024-04-17T07:13:57.064498+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:13:57.064498+0000","last_undegraded":"2024-04-17T07:13:57.064498+0000","last_fullsized":"2024-04-17T07:13:57.064498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:43.565627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174262+0000","last_change":"2024-04-17T07:12:29.125397+0000","last_active":"2024-04-17T07:13:29.174262+0000","last_peered":"2024-04-17T07:13:29.174262+0000","last_clean":"2024-04-17T07:13:29.174262+0000","last_became_active":"2024-04-17T07:12:29.124158+0000","last_became_peered":"2024-04-17T07:12:29.124158+0000","last_unstale":"2024-04-17T07:13:29.174262+0000","last_undegraded":"2024-04-17T07:13:29.174262+0000","last_fullsized":"2024-04-17T07:13:29.174262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:24.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205266+0000","last_change":"2024-04-17T07:11:54.464715+0000","last_active":"2024-04-17T07:13:29.205266+0000","last_peered":"2024-04-17T07:13:29.205266+0000","last_clean":"2024-04-17T07:13:29.205266+0000","last_became_active":"2024-04-17T07:11:54.463917+0000","last_became_peered":"2024-04-17T07:11:54.463917+0000","last_unstale":"2024-04-17T07:13:29.205266+0000","last_undegraded":"2024-04-17T07:13:29.205266+0000","last_fullsized":"2024-04-17T07:13:29.205266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:43.738471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.085059+0000","last_change":"2024-04-17T07:11:55.482876+0000","last_active":"2024-04-17T07:13:57.085059+0000","last_peered":"2024-04-17T07:13:57.085059+0000","last_clean":"2024-04-17T07:13:57.085059+0000","last_became_active":"2024-04-17T07:11:55.482451+0000","last_became_peered":"2024-04-17T07:11:55.482451+0000","last_unstale":"2024-04-17T07:13:57.085059+0000","last_undegraded":"2024-04-17T07:13:57.085059+0000","last_fullsized":"2024-04-17T07:13:57.085059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:39.394090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174779+0000","last_change":"2024-04-17T07:11:59.522634+0000","last_active":"2024-04-17T07:13:29.174779+0000","last_peered":"2024-04-17T07:13:29.174779+0000","last_clean":"2024-04-17T07:13:29.174779+0000","last_became_active":"2024-04-17T07:11:59.520797+0000","last_became_peered":"2024-04-17T07:11:59.520797+0000","last_unstale":"2024-04-17T07:13:29.174779+0000","last_undegraded":"2024-04-17T07:13:29.174779+0000","last_fullsized":"2024-04-17T07:13:29.174779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:02.316515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.334093+0000","last_change":"2024-04-17T07:12:28.756528+0000","last_active":"2024-04-17T07:13:29.334093+0000","last_peered":"2024-04-17T07:13:29.334093+0000","last_clean":"2024-04-17T07:13:29.334093+0000","last_became_active":"2024-04-17T07:12:28.737008+0000","last_became_peered":"2024-04-17T07:12:28.737008+0000","last_unstale":"2024-04-17T07:13:29.334093+0000","last_undegraded":"2024-04-17T07:13:29.334093+0000","last_fullsized":"2024-04-17T07:13:29.334093+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:56:26.719377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082808+0000","last_change":"2024-04-17T07:11:55.485999+0000","last_active":"2024-04-17T07:13:57.082808+0000","last_peered":"2024-04-17T07:13:57.082808+0000","last_clean":"2024-04-17T07:13:57.082808+0000","last_became_active":"2024-04-17T07:11:55.484048+0000","last_became_peered":"2024-04-17T07:11:55.484048+0000","last_unstale":"2024-04-17T07:13:57.082808+0000","last_undegraded":"2024-04-17T07:13:57.082808+0000","last_fullsized":"2024-04-17T07:13:57.082808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:38.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174261+0000","last_change":"2024-04-17T07:11:59.529929+0000","last_active":"2024-04-17T07:13:29.174261+0000","last_peered":"2024-04-17T07:13:29.174261+0000","last_clean":"2024-04-17T07:13:29.174261+0000","last_became_active":"2024-04-17T07:11:59.529400+0000","last_became_peered":"2024-04-17T07:11:59.529400+0000","last_unstale":"2024-04-17T07:13:29.174261+0000","last_undegraded":"2024-04-17T07:13:29.174261+0000","last_fullsized":"2024-04-17T07:13:29.174261+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:54.845529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204741+0000","last_change":"2024-04-17T07:11:54.462852+0000","last_active":"2024-04-17T07:13:29.204741+0000","last_peered":"2024-04-17T07:13:29.204741+0000","last_clean":"2024-04-17T07:13:29.204741+0000","last_became_active":"2024-04-17T07:11:54.459178+0000","last_became_peered":"2024-04-17T07:11:54.459178+0000","last_unstale":"2024-04-17T07:13:29.204741+0000","last_undegraded":"2024-04-17T07:13:29.204741+0000","last_fullsized":"2024-04-17T07:13:29.204741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:07.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.075088+0000","last_change":"2024-04-17T07:11:55.480951+0000","last_active":"2024-04-17T07:13:57.075088+0000","last_peered":"2024-04-17T07:13:57.075088+0000","last_clean":"2024-04-17T07:13:57.075088+0000","last_became_active":"2024-04-17T07:11:55.480507+0000","last_became_peered":"2024-04-17T07:11:55.480507+0000","last_unstale":"2024-04-17T07:13:57.075088+0000","last_undegraded":"2024-04-17T07:13:57.075088+0000","last_fullsized":"2024-04-17T07:13:57.075088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:42.584180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205344+0000","last_change":"2024-04-17T07:11:59.520897+0000","last_active":"2024-04-17T07:13:29.205344+0000","last_peered":"2024-04-17T07:13:29.205344+0000","last_clean":"2024-04-17T07:13:29.205344+0000","last_became_active":"2024-04-17T07:11:59.520350+0000","last_became_peered":"2024-04-17T07:11:59.520350+0000","last_unstale":"2024-04-17T07:13:29.205344+0000","last_undegraded":"2024-04-17T07:13:29.205344+0000","last_fullsized":"2024-04-17T07:13:29.205344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:39.813097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":144,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:00.495518+0000","last_change":"2024-04-17T07:12:28.753717+0000","last_active":"2024-04-17T07:14:00.495518+0000","last_peered":"2024-04-17T07:14:00.495518+0000","last_clean":"2024-04-17T07:14:00.495518+0000","last_became_active":"2024-04-17T07:12:28.739556+0000","last_became_peered":"2024-04-17T07:12:28.739556+0000","last_unstale":"2024-04-17T07:14:00.495518+0000","last_undegraded":"2024-04-17T07:14:00.495518+0000","last_fullsized":"2024-04-17T07:14:00.495518+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:11.967731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.055403+0000","last_change":"2024-04-17T07:11:55.476675+0000","last_active":"2024-04-17T07:13:57.055403+0000","last_peered":"2024-04-17T07:13:57.055403+0000","last_clean":"2024-04-17T07:13:57.055403+0000","last_became_active":"2024-04-17T07:11:55.476273+0000","last_became_peered":"2024-04-17T07:11:55.476273+0000","last_unstale":"2024-04-17T07:13:57.055403+0000","last_undegraded":"2024-04-17T07:13:57.055403+0000","last_fullsized":"2024-04-17T07:13:57.055403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:09.612138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205437+0000","last_change":"2024-04-17T07:12:28.741537+0000","last_active":"2024-04-17T07:13:29.205437+0000","last_peered":"2024-04-17T07:13:29.205437+0000","last_clean":"2024-04-17T07:13:29.205437+0000","last_became_active":"2024-04-17T07:12:28.741035+0000","last_became_peered":"2024-04-17T07:12:28.741035+0000","last_unstale":"2024-04-17T07:13:29.205437+0000","last_undegraded":"2024-04-17T07:13:29.205437+0000","last_fullsized":"2024-04-17T07:13:29.205437+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:15.550112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.207837+0000","last_change":"2024-04-17T07:11:54.471118+0000","last_active":"2024-04-17T07:13:29.207837+0000","last_peered":"2024-04-17T07:13:29.207837+0000","last_clean":"2024-04-17T07:13:29.207837+0000","last_became_active":"2024-04-17T07:11:54.470526+0000","last_became_peered":"2024-04-17T07:11:54.470526+0000","last_unstale":"2024-04-17T07:13:29.207837+0000","last_undegraded":"2024-04-17T07:13:29.207837+0000","last_fullsized":"2024-04-17T07:13:29.207837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:52:47.918086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070303+0000","last_change":"2024-04-17T07:11:55.472850+0000","last_active":"2024-04-17T07:13:57.070303+0000","last_peered":"2024-04-17T07:13:57.070303+0000","last_clean":"2024-04-17T07:13:57.070303+0000","last_became_active":"2024-04-17T07:11:55.472097+0000","last_became_peered":"2024-04-17T07:11:55.472097+0000","last_unstale":"2024-04-17T07:13:57.070303+0000","last_undegraded":"2024-04-17T07:13:57.070303+0000","last_fullsized":"2024-04-17T07:13:57.070303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:41.730809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.334016+0000","last_change":"2024-04-17T07:11:59.522960+0000","last_active":"2024-04-17T07:13:29.334016+0000","last_peered":"2024-04-17T07:13:29.334016+0000","last_clean":"2024-04-17T07:13:29.334016+0000","last_became_active":"2024-04-17T07:11:59.522470+0000","last_became_peered":"2024-04-17T07:11:59.522470+0000","last_unstale":"2024-04-17T07:13:29.334016+0000","last_undegraded":"2024-04-17T07:13:29.334016+0000","last_fullsized":"2024-04-17T07:13:29.334016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:36.620442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.171594+0000","last_change":"2024-04-17T07:11:54.470693+0000","last_active":"2024-04-17T07:13:29.171594+0000","last_peered":"2024-04-17T07:13:29.171594+0000","last_clean":"2024-04-17T07:13:29.171594+0000","last_became_active":"2024-04-17T07:11:54.470334+0000","last_became_peered":"2024-04-17T07:11:54.470334+0000","last_unstale":"2024-04-17T07:13:29.171594+0000","last_undegraded":"2024-04-17T07:13:29.171594+0000","last_fullsized":"2024-04-17T07:13:29.171594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:19:04.980786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068407+0000","last_change":"2024-04-17T07:11:55.474755+0000","last_active":"2024-04-17T07:13:57.068407+0000","last_peered":"2024-04-17T07:13:57.068407+0000","last_clean":"2024-04-17T07:13:57.068407+0000","last_became_active":"2024-04-17T07:11:55.474261+0000","last_became_peered":"2024-04-17T07:11:55.474261+0000","last_unstale":"2024-04-17T07:13:57.068407+0000","last_undegraded":"2024-04-17T07:13:57.068407+0000","last_fullsized":"2024-04-17T07:13:57.068407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:00.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203208+0000","last_change":"2024-04-17T07:11:59.517152+0000","last_active":"2024-04-17T07:13:29.203208+0000","last_peered":"2024-04-17T07:13:29.203208+0000","last_clean":"2024-04-17T07:13:29.203208+0000","last_became_active":"2024-04-17T07:11:59.514977+0000","last_became_peered":"2024-04-17T07:11:59.514977+0000","last_unstale":"2024-04-17T07:13:29.203208+0000","last_undegraded":"2024-04-17T07:13:29.203208+0000","last_fullsized":"2024-04-17T07:13:29.203208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:14.974136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172810+0000","last_change":"2024-04-17T07:11:54.459786+0000","last_active":"2024-04-17T07:13:29.172810+0000","last_peered":"2024-04-17T07:13:29.172810+0000","last_clean":"2024-04-17T07:13:29.172810+0000","last_became_active":"2024-04-17T07:11:54.457589+0000","last_became_peered":"2024-04-17T07:11:54.457589+0000","last_unstale":"2024-04-17T07:13:29.172810+0000","last_undegraded":"2024-04-17T07:13:29.172810+0000","last_fullsized":"2024-04-17T07:13:29.172810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:25.986539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067337+0000","last_change":"2024-04-17T07:11:55.465020+0000","last_active":"2024-04-17T07:13:57.067337+0000","last_peered":"2024-04-17T07:13:57.067337+0000","last_clean":"2024-04-17T07:13:57.067337+0000","last_became_active":"2024-04-17T07:11:55.464608+0000","last_became_peered":"2024-04-17T07:11:55.464608+0000","last_unstale":"2024-04-17T07:13:57.067337+0000","last_undegraded":"2024-04-17T07:13:57.067337+0000","last_fullsized":"2024-04-17T07:13:57.067337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204510+0000","last_change":"2024-04-17T07:11:59.519333+0000","last_active":"2024-04-17T07:13:29.204510+0000","last_peered":"2024-04-17T07:13:29.204510+0000","last_clean":"2024-04-17T07:13:29.204510+0000","last_became_active":"2024-04-17T07:11:59.518865+0000","last_became_peered":"2024-04-17T07:11:59.518865+0000","last_unstale":"2024-04-17T07:13:29.204510+0000","last_undegraded":"2024-04-17T07:13:29.204510+0000","last_fullsized":"2024-04-17T07:13:29.204510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:42.009010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":218,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:01.494444+0000","last_change":"2024-04-17T07:11:54.462283+0000","last_active":"2024-04-17T07:14:01.494444+0000","last_peered":"2024-04-17T07:14:01.494444+0000","last_clean":"2024-04-17T07:14:01.494444+0000","last_became_active":"2024-04-17T07:11:54.461818+0000","last_became_peered":"2024-04-17T07:11:54.461818+0000","last_unstale":"2024-04-17T07:14:01.494444+0000","last_undegraded":"2024-04-17T07:14:01.494444+0000","last_fullsized":"2024-04-17T07:14:01.494444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.765058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.080879+0000","last_change":"2024-04-17T07:11:55.486491+0000","last_active":"2024-04-17T07:13:57.080879+0000","last_peered":"2024-04-17T07:13:57.080879+0000","last_clean":"2024-04-17T07:13:57.080879+0000","last_became_active":"2024-04-17T07:11:55.486121+0000","last_became_peered":"2024-04-17T07:11:55.486121+0000","last_unstale":"2024-04-17T07:13:57.080879+0000","last_undegraded":"2024-04-17T07:13:57.080879+0000","last_fullsized":"2024-04-17T07:13:57.080879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:12.998321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205900+0000","last_change":"2024-04-17T07:11:59.511555+0000","last_active":"2024-04-17T07:13:29.205900+0000","last_peered":"2024-04-17T07:13:29.205900+0000","last_clean":"2024-04-17T07:13:29.205900+0000","last_became_active":"2024-04-17T07:11:59.511147+0000","last_became_peered":"2024-04-17T07:11:59.511147+0000","last_unstale":"2024-04-17T07:13:29.205900+0000","last_undegraded":"2024-04-17T07:13:29.205900+0000","last_fullsized":"2024-04-17T07:13:29.205900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:41.854815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209414+0000","last_change":"2024-04-17T07:12:28.734476+0000","last_active":"2024-04-17T07:13:29.209414+0000","last_peered":"2024-04-17T07:13:29.209414+0000","last_clean":"2024-04-17T07:13:29.209414+0000","last_became_active":"2024-04-17T07:12:28.728187+0000","last_became_peered":"2024-04-17T07:12:28.728187+0000","last_unstale":"2024-04-17T07:13:29.209414+0000","last_undegraded":"2024-04-17T07:13:29.209414+0000","last_fullsized":"2024-04-17T07:13:29.209414+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:45.749607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081348+0000","last_change":"2024-04-17T07:11:55.480589+0000","last_active":"2024-04-17T07:13:57.081348+0000","last_peered":"2024-04-17T07:13:57.081348+0000","last_clean":"2024-04-17T07:13:57.081348+0000","last_became_active":"2024-04-17T07:11:55.480224+0000","last_became_peered":"2024-04-17T07:11:55.480224+0000","last_unstale":"2024-04-17T07:13:57.081348+0000","last_undegraded":"2024-04-17T07:13:57.081348+0000","last_fullsized":"2024-04-17T07:13:57.081348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:05.892442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175056+0000","last_change":"2024-04-17T07:11:59.522021+0000","last_active":"2024-04-17T07:13:29.175056+0000","last_peered":"2024-04-17T07:13:29.175056+0000","last_clean":"2024-04-17T07:13:29.175056+0000","last_became_active":"2024-04-17T07:11:59.521493+0000","last_became_peered":"2024-04-17T07:11:59.521493+0000","last_unstale":"2024-04-17T07:13:29.175056+0000","last_undegraded":"2024-04-17T07:13:29.175056+0000","last_fullsized":"2024-04-17T07:13:29.175056+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:20.386216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174326+0000","last_change":"2024-04-17T07:11:54.468225+0000","last_active":"2024-04-17T07:13:29.174326+0000","last_peered":"2024-04-17T07:13:29.174326+0000","last_clean":"2024-04-17T07:13:29.174326+0000","last_became_active":"2024-04-17T07:11:54.467046+0000","last_became_peered":"2024-04-17T07:11:54.467046+0000","last_unstale":"2024-04-17T07:13:29.174326+0000","last_undegraded":"2024-04-17T07:13:29.174326+0000","last_fullsized":"2024-04-17T07:13:29.174326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:17.725255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082674+0000","last_change":"2024-04-17T07:11:55.488980+0000","last_active":"2024-04-17T07:13:57.082674+0000","last_peered":"2024-04-17T07:13:57.082674+0000","last_clean":"2024-04-17T07:13:57.082674+0000","last_became_active":"2024-04-17T07:11:55.488226+0000","last_became_peered":"2024-04-17T07:11:55.488226+0000","last_unstale":"2024-04-17T07:13:57.082674+0000","last_undegraded":"2024-04-17T07:13:57.082674+0000","last_fullsized":"2024-04-17T07:13:57.082674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:00.135973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205554+0000","last_change":"2024-04-17T07:11:59.515134+0000","last_active":"2024-04-17T07:13:29.205554+0000","last_peered":"2024-04-17T07:13:29.205554+0000","last_clean":"2024-04-17T07:13:29.205554+0000","last_became_active":"2024-04-17T07:11:59.514560+0000","last_became_peered":"2024-04-17T07:11:59.514560+0000","last_unstale":"2024-04-17T07:13:29.205554+0000","last_undegraded":"2024-04-17T07:13:29.205554+0000","last_fullsized":"2024-04-17T07:13:29.205554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:34.669789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'36","reported_seq":256,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:01.495917+0000","last_change":"2024-04-17T07:11:54.461226+0000","last_active":"2024-04-17T07:14:01.495917+0000","last_peered":"2024-04-17T07:14:01.495917+0000","last_clean":"2024-04-17T07:14:01.495917+0000","last_became_active":"2024-04-17T07:11:54.460573+0000","last_became_peered":"2024-04-17T07:11:54.460573+0000","last_unstale":"2024-04-17T07:14:01.495917+0000","last_undegraded":"2024-04-17T07:14:01.495917+0000","last_fullsized":"2024-04-17T07:14:01.495917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62250,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":189,"num_read_kb":155,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068078+0000","last_change":"2024-04-17T07:11:55.483925+0000","last_active":"2024-04-17T07:13:57.068078+0000","last_peered":"2024-04-17T07:13:57.068078+0000","last_clean":"2024-04-17T07:13:57.068078+0000","last_became_active":"2024-04-17T07:11:55.483584+0000","last_became_peered":"2024-04-17T07:11:55.483584+0000","last_unstale":"2024-04-17T07:13:57.068078+0000","last_undegraded":"2024-04-17T07:13:57.068078+0000","last_fullsized":"2024-04-17T07:13:57.068078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:12.328876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174863+0000","last_change":"2024-04-17T07:11:59.515597+0000","last_active":"2024-04-17T07:13:29.174863+0000","last_peered":"2024-04-17T07:13:29.174863+0000","last_clean":"2024-04-17T07:13:29.174863+0000","last_became_active":"2024-04-17T07:11:59.514413+0000","last_became_peered":"2024-04-17T07:11:59.514413+0000","last_unstale":"2024-04-17T07:13:29.174863+0000","last_undegraded":"2024-04-17T07:13:29.174863+0000","last_fullsized":"2024-04-17T07:13:29.174863+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:45.824127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205125+0000","last_change":"2024-04-17T07:11:54.449984+0000","last_active":"2024-04-17T07:13:29.205125+0000","last_peered":"2024-04-17T07:13:29.205125+0000","last_clean":"2024-04-17T07:13:29.205125+0000","last_became_active":"2024-04-17T07:11:54.449628+0000","last_became_peered":"2024-04-17T07:11:54.449628+0000","last_unstale":"2024-04-17T07:13:29.205125+0000","last_undegraded":"2024-04-17T07:13:29.205125+0000","last_fullsized":"2024-04-17T07:13:29.205125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:26:03.281684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069323+0000","last_change":"2024-04-17T07:11:55.476912+0000","last_active":"2024-04-17T07:13:57.069323+0000","last_peered":"2024-04-17T07:13:57.069323+0000","last_clean":"2024-04-17T07:13:57.069323+0000","last_became_active":"2024-04-17T07:11:55.471576+0000","last_became_peered":"2024-04-17T07:11:55.471576+0000","last_unstale":"2024-04-17T07:13:57.069323+0000","last_undegraded":"2024-04-17T07:13:57.069323+0000","last_fullsized":"2024-04-17T07:13:57.069323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:35.439271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333787+0000","last_change":"2024-04-17T07:11:59.515897+0000","last_active":"2024-04-17T07:13:29.333787+0000","last_peered":"2024-04-17T07:13:29.333787+0000","last_clean":"2024-04-17T07:13:29.333787+0000","last_became_active":"2024-04-17T07:11:59.515349+0000","last_became_peered":"2024-04-17T07:11:59.515349+0000","last_unstale":"2024-04-17T07:13:29.333787+0000","last_undegraded":"2024-04-17T07:13:29.333787+0000","last_fullsized":"2024-04-17T07:13:29.333787+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:00.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173500+0000","last_change":"2024-04-17T07:11:54.462369+0000","last_active":"2024-04-17T07:13:29.173500+0000","last_peered":"2024-04-17T07:13:29.173500+0000","last_clean":"2024-04-17T07:13:29.173500+0000","last_became_active":"2024-04-17T07:11:54.460357+0000","last_became_peered":"2024-04-17T07:11:54.460357+0000","last_unstale":"2024-04-17T07:13:29.173500+0000","last_undegraded":"2024-04-17T07:13:29.173500+0000","last_fullsized":"2024-04-17T07:13:29.173500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:38.029980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071275+0000","last_change":"2024-04-17T07:11:55.475874+0000","last_active":"2024-04-17T07:13:57.071275+0000","last_peered":"2024-04-17T07:13:57.071275+0000","last_clean":"2024-04-17T07:13:57.071275+0000","last_became_active":"2024-04-17T07:11:55.475550+0000","last_became_peered":"2024-04-17T07:11:55.475550+0000","last_unstale":"2024-04-17T07:13:57.071275+0000","last_undegraded":"2024-04-17T07:13:57.071275+0000","last_fullsized":"2024-04-17T07:13:57.071275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:40.096573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206806+0000","last_change":"2024-04-17T07:11:59.514661+0000","last_active":"2024-04-17T07:13:29.206806+0000","last_peered":"2024-04-17T07:13:29.206806+0000","last_clean":"2024-04-17T07:13:29.206806+0000","last_became_active":"2024-04-17T07:11:59.513030+0000","last_became_peered":"2024-04-17T07:11:59.513030+0000","last_unstale":"2024-04-17T07:13:29.206806+0000","last_undegraded":"2024-04-17T07:13:29.206806+0000","last_fullsized":"2024-04-17T07:13:29.206806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:35.696961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.481495+0000","last_change":"2024-04-17T07:11:54.455279+0000","last_active":"2024-04-17T07:13:57.481495+0000","last_peered":"2024-04-17T07:13:57.481495+0000","last_clean":"2024-04-17T07:13:57.481495+0000","last_became_active":"2024-04-17T07:11:54.453620+0000","last_became_peered":"2024-04-17T07:11:54.453620+0000","last_unstale":"2024-04-17T07:13:57.481495+0000","last_undegraded":"2024-04-17T07:13:57.481495+0000","last_fullsized":"2024-04-17T07:13:57.481495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:15.227410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.065132+0000","last_change":"2024-04-17T07:11:55.469579+0000","last_active":"2024-04-17T07:13:57.065132+0000","last_peered":"2024-04-17T07:13:57.065132+0000","last_clean":"2024-04-17T07:13:57.065132+0000","last_became_active":"2024-04-17T07:11:55.468389+0000","last_became_peered":"2024-04-17T07:11:55.468389+0000","last_unstale":"2024-04-17T07:13:57.065132+0000","last_undegraded":"2024-04-17T07:13:57.065132+0000","last_fullsized":"2024-04-17T07:13:57.065132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:01.274677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173429+0000","last_change":"2024-04-17T07:11:59.512110+0000","last_active":"2024-04-17T07:13:29.173429+0000","last_peered":"2024-04-17T07:13:29.173429+0000","last_clean":"2024-04-17T07:13:29.173429+0000","last_became_active":"2024-04-17T07:11:59.510573+0000","last_became_peered":"2024-04-17T07:11:59.510573+0000","last_unstale":"2024-04-17T07:13:29.173429+0000","last_undegraded":"2024-04-17T07:13:29.173429+0000","last_fullsized":"2024-04-17T07:13:29.173429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:32.515687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333195+0000","last_change":"2024-04-17T07:12:28.767517+0000","last_active":"2024-04-17T07:13:29.333195+0000","last_peered":"2024-04-17T07:13:29.333195+0000","last_clean":"2024-04-17T07:13:29.333195+0000","last_became_active":"2024-04-17T07:12:28.738908+0000","last_became_peered":"2024-04-17T07:12:28.738908+0000","last_unstale":"2024-04-17T07:13:29.333195+0000","last_undegraded":"2024-04-17T07:13:29.333195+0000","last_fullsized":"2024-04-17T07:13:29.333195+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:47:30.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067484+0000","last_change":"2024-04-17T07:11:55.473540+0000","last_active":"2024-04-17T07:13:57.067484+0000","last_peered":"2024-04-17T07:13:57.067484+0000","last_clean":"2024-04-17T07:13:57.067484+0000","last_became_active":"2024-04-17T07:11:55.472436+0000","last_became_peered":"2024-04-17T07:11:55.472436+0000","last_unstale":"2024-04-17T07:13:57.067484+0000","last_undegraded":"2024-04-17T07:13:57.067484+0000","last_fullsized":"2024-04-17T07:13:57.067484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:54.317052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172782+0000","last_change":"2024-04-17T07:11:59.521305+0000","last_active":"2024-04-17T07:13:29.172782+0000","last_peered":"2024-04-17T07:13:29.172782+0000","last_clean":"2024-04-17T07:13:29.172782+0000","last_became_active":"2024-04-17T07:11:59.516860+0000","last_became_peered":"2024-04-17T07:11:59.516860+0000","last_unstale":"2024-04-17T07:13:29.172782+0000","last_undegraded":"2024-04-17T07:13:29.172782+0000","last_fullsized":"2024-04-17T07:13:29.172782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:23.403218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206663+0000","last_change":"2024-04-17T07:11:54.463378+0000","last_active":"2024-04-17T07:13:29.206663+0000","last_peered":"2024-04-17T07:13:29.206663+0000","last_clean":"2024-04-17T07:13:29.206663+0000","last_became_active":"2024-04-17T07:11:54.462120+0000","last_became_peered":"2024-04-17T07:11:54.462120+0000","last_unstale":"2024-04-17T07:13:29.206663+0000","last_undegraded":"2024-04-17T07:13:29.206663+0000","last_fullsized":"2024-04-17T07:13:29.206663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:42.109928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.065968+0000","last_change":"2024-04-17T07:11:55.478604+0000","last_active":"2024-04-17T07:13:57.065968+0000","last_peered":"2024-04-17T07:13:57.065968+0000","last_clean":"2024-04-17T07:13:57.065968+0000","last_became_active":"2024-04-17T07:11:55.478252+0000","last_became_peered":"2024-04-17T07:11:55.478252+0000","last_unstale":"2024-04-17T07:13:57.065968+0000","last_undegraded":"2024-04-17T07:13:57.065968+0000","last_fullsized":"2024-04-17T07:13:57.065968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:08.365896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206271+0000","last_change":"2024-04-17T07:11:59.518619+0000","last_active":"2024-04-17T07:13:29.206271+0000","last_peered":"2024-04-17T07:13:29.206271+0000","last_clean":"2024-04-17T07:13:29.206271+0000","last_became_active":"2024-04-17T07:11:59.518214+0000","last_became_peered":"2024-04-17T07:11:59.518214+0000","last_unstale":"2024-04-17T07:13:29.206271+0000","last_undegraded":"2024-04-17T07:13:29.206271+0000","last_fullsized":"2024-04-17T07:13:29.206271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:46.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206316+0000","last_change":"2024-04-17T07:11:54.453038+0000","last_active":"2024-04-17T07:13:29.206316+0000","last_peered":"2024-04-17T07:13:29.206316+0000","last_clean":"2024-04-17T07:13:29.206316+0000","last_became_active":"2024-04-17T07:11:54.452619+0000","last_became_peered":"2024-04-17T07:11:54.452619+0000","last_unstale":"2024-04-17T07:13:29.206316+0000","last_undegraded":"2024-04-17T07:13:29.206316+0000","last_fullsized":"2024-04-17T07:13:29.206316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:25.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.063521+0000","last_change":"2024-04-17T07:11:55.465990+0000","last_active":"2024-04-17T07:13:57.063521+0000","last_peered":"2024-04-17T07:13:57.063521+0000","last_clean":"2024-04-17T07:13:57.063521+0000","last_became_active":"2024-04-17T07:11:55.465523+0000","last_became_peered":"2024-04-17T07:11:55.465523+0000","last_unstale":"2024-04-17T07:13:57.063521+0000","last_undegraded":"2024-04-17T07:13:57.063521+0000","last_fullsized":"2024-04-17T07:13:57.063521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:40.366027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172265+0000","last_change":"2024-04-17T07:11:59.512119+0000","last_active":"2024-04-17T07:13:29.172265+0000","last_peered":"2024-04-17T07:13:29.172265+0000","last_clean":"2024-04-17T07:13:29.172265+0000","last_became_active":"2024-04-17T07:11:59.511460+0000","last_became_peered":"2024-04-17T07:11:59.511460+0000","last_unstale":"2024-04-17T07:13:29.172265+0000","last_undegraded":"2024-04-17T07:13:29.172265+0000","last_fullsized":"2024-04-17T07:13:29.172265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:50.888203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'5","reported_seq":129,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:01.478744+0000","last_change":"2024-04-17T07:11:54.469346+0000","last_active":"2024-04-17T07:14:01.478744+0000","last_peered":"2024-04-17T07:14:01.478744+0000","last_clean":"2024-04-17T07:14:01.478744+0000","last_became_active":"2024-04-17T07:11:54.468947+0000","last_became_peered":"2024-04-17T07:11:54.468947+0000","last_unstale":"2024-04-17T07:14:01.478744+0000","last_undegraded":"2024-04-17T07:14:01.478744+0000","last_fullsized":"2024-04-17T07:14:01.478744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:04.854625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078974+0000","last_change":"2024-04-17T07:11:55.471235+0000","last_active":"2024-04-17T07:13:57.078974+0000","last_peered":"2024-04-17T07:13:57.078974+0000","last_clean":"2024-04-17T07:13:57.078974+0000","last_became_active":"2024-04-17T07:11:55.470857+0000","last_became_peered":"2024-04-17T07:11:55.470857+0000","last_unstale":"2024-04-17T07:13:57.078974+0000","last_undegraded":"2024-04-17T07:13:57.078974+0000","last_fullsized":"2024-04-17T07:13:57.078974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:21.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.171912+0000","last_change":"2024-04-17T07:11:59.524519+0000","last_active":"2024-04-17T07:13:29.171912+0000","last_peered":"2024-04-17T07:13:29.171912+0000","last_clean":"2024-04-17T07:13:29.171912+0000","last_became_active":"2024-04-17T07:11:59.519270+0000","last_became_peered":"2024-04-17T07:11:59.519270+0000","last_unstale":"2024-04-17T07:13:29.171912+0000","last_undegraded":"2024-04-17T07:13:29.171912+0000","last_fullsized":"2024-04-17T07:13:29.171912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:41.080249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070265+0000","last_change":"2024-04-17T07:11:55.473922+0000","last_active":"2024-04-17T07:13:57.070265+0000","last_peered":"2024-04-17T07:13:57.070265+0000","last_clean":"2024-04-17T07:13:57.070265+0000","last_became_active":"2024-04-17T07:11:55.468594+0000","last_became_peered":"2024-04-17T07:11:55.468594+0000","last_unstale":"2024-04-17T07:13:57.070265+0000","last_undegraded":"2024-04-17T07:13:57.070265+0000","last_fullsized":"2024-04-17T07:13:57.070265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:21.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":130,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:02.494711+0000","last_change":"2024-04-17T07:11:54.473066+0000","last_active":"2024-04-17T07:14:02.494711+0000","last_peered":"2024-04-17T07:14:02.494711+0000","last_clean":"2024-04-17T07:14:02.494711+0000","last_became_active":"2024-04-17T07:11:54.472708+0000","last_became_peered":"2024-04-17T07:11:54.472708+0000","last_unstale":"2024-04-17T07:14:02.494711+0000","last_undegraded":"2024-04-17T07:14:02.494711+0000","last_fullsized":"2024-04-17T07:14:02.494711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:27.614819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.208953+0000","last_change":"2024-04-17T07:11:59.516939+0000","last_active":"2024-04-17T07:13:29.208953+0000","last_peered":"2024-04-17T07:13:29.208953+0000","last_clean":"2024-04-17T07:13:29.208953+0000","last_became_active":"2024-04-17T07:11:59.516333+0000","last_became_peered":"2024-04-17T07:11:59.516333+0000","last_unstale":"2024-04-17T07:13:29.208953+0000","last_undegraded":"2024-04-17T07:13:29.208953+0000","last_fullsized":"2024-04-17T07:13:29.208953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:15.648831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082745+0000","last_change":"2024-04-17T07:11:55.481006+0000","last_active":"2024-04-17T07:13:57.082745+0000","last_peered":"2024-04-17T07:13:57.082745+0000","last_clean":"2024-04-17T07:13:57.082745+0000","last_became_active":"2024-04-17T07:11:55.479751+0000","last_became_peered":"2024-04-17T07:11:55.479751+0000","last_unstale":"2024-04-17T07:13:57.082745+0000","last_undegraded":"2024-04-17T07:13:57.082745+0000","last_fullsized":"2024-04-17T07:13:57.082745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:13.004650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175099+0000","last_change":"2024-04-17T07:11:54.463106+0000","last_active":"2024-04-17T07:13:29.175099+0000","last_peered":"2024-04-17T07:13:29.175099+0000","last_clean":"2024-04-17T07:13:29.175099+0000","last_became_active":"2024-04-17T07:11:54.462691+0000","last_became_peered":"2024-04-17T07:11:54.462691+0000","last_unstale":"2024-04-17T07:13:29.175099+0000","last_undegraded":"2024-04-17T07:13:29.175099+0000","last_fullsized":"2024-04-17T07:13:29.175099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:34.302019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204929+0000","last_change":"2024-04-17T07:10:14.685883+0000","last_active":"2024-04-17T07:13:29.204929+0000","last_peered":"2024-04-17T07:13:29.204929+0000","last_clean":"2024-04-17T07:13:29.204929+0000","last_became_active":"2024-04-17T07:10:14.684551+0000","last_became_peered":"2024-04-17T07:10:14.684551+0000","last_unstale":"2024-04-17T07:13:29.204929+0000","last_undegraded":"2024-04-17T07:13:29.204929+0000","last_fullsized":"2024-04-17T07:13:29.204929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_clean_scrub_stamp":"2024-04-17T07:05:37.112317+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:04.265147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206420+0000","last_change":"2024-04-17T07:12:28.731807+0000","last_active":"2024-04-17T07:13:29.206420+0000","last_peered":"2024-04-17T07:13:29.206420+0000","last_clean":"2024-04-17T07:13:29.206420+0000","last_became_active":"2024-04-17T07:12:28.730656+0000","last_became_peered":"2024-04-17T07:12:28.730656+0000","last_unstale":"2024-04-17T07:13:29.206420+0000","last_undegraded":"2024-04-17T07:13:29.206420+0000","last_fullsized":"2024-04-17T07:13:29.206420+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:58.023333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068799+0000","last_change":"2024-04-17T07:12:28.727980+0000","last_active":"2024-04-17T07:13:57.068799+0000","last_peered":"2024-04-17T07:13:57.068799+0000","last_clean":"2024-04-17T07:13:57.068799+0000","last_became_active":"2024-04-17T07:12:28.727060+0000","last_became_peered":"2024-04-17T07:12:28.727060+0000","last_unstale":"2024-04-17T07:13:57.068799+0000","last_undegraded":"2024-04-17T07:13:57.068799+0000","last_fullsized":"2024-04-17T07:13:57.068799+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:33.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205324+0000","last_change":"2024-04-17T07:11:54.463180+0000","last_active":"2024-04-17T07:13:29.205324+0000","last_peered":"2024-04-17T07:13:29.205324+0000","last_clean":"2024-04-17T07:13:29.205324+0000","last_became_active":"2024-04-17T07:11:54.462793+0000","last_became_peered":"2024-04-17T07:11:54.462793+0000","last_unstale":"2024-04-17T07:13:29.205324+0000","last_undegraded":"2024-04-17T07:13:29.205324+0000","last_fullsized":"2024-04-17T07:13:29.205324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:19.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333167+0000","last_change":"2024-04-17T07:11:59.518355+0000","last_active":"2024-04-17T07:13:29.333167+0000","last_peered":"2024-04-17T07:13:29.333167+0000","last_clean":"2024-04-17T07:13:29.333167+0000","last_became_active":"2024-04-17T07:11:59.517619+0000","last_became_peered":"2024-04-17T07:11:59.517619+0000","last_unstale":"2024-04-17T07:13:29.333167+0000","last_undegraded":"2024-04-17T07:13:29.333167+0000","last_fullsized":"2024-04-17T07:13:29.333167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:36.826021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206403+0000","last_change":"2024-04-17T07:11:54.465285+0000","last_active":"2024-04-17T07:13:29.206403+0000","last_peered":"2024-04-17T07:13:29.206403+0000","last_clean":"2024-04-17T07:13:29.206403+0000","last_became_active":"2024-04-17T07:11:54.464818+0000","last_became_peered":"2024-04-17T07:11:54.464818+0000","last_unstale":"2024-04-17T07:13:29.206403+0000","last_undegraded":"2024-04-17T07:13:29.206403+0000","last_fullsized":"2024-04-17T07:13:29.206403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069247+0000","last_change":"2024-04-17T07:11:55.470405+0000","last_active":"2024-04-17T07:13:57.069247+0000","last_peered":"2024-04-17T07:13:57.069247+0000","last_clean":"2024-04-17T07:13:57.069247+0000","last_became_active":"2024-04-17T07:11:55.470058+0000","last_became_peered":"2024-04-17T07:11:55.470058+0000","last_unstale":"2024-04-17T07:13:57.069247+0000","last_undegraded":"2024-04-17T07:13:57.069247+0000","last_fullsized":"2024-04-17T07:13:57.069247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:17.416775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.202954+0000","last_change":"2024-04-17T07:11:59.522040+0000","last_active":"2024-04-17T07:13:29.202954+0000","last_peered":"2024-04-17T07:13:29.202954+0000","last_clean":"2024-04-17T07:13:29.202954+0000","last_became_active":"2024-04-17T07:11:59.519771+0000","last_became_peered":"2024-04-17T07:11:59.519771+0000","last_unstale":"2024-04-17T07:13:29.202954+0000","last_undegraded":"2024-04-17T07:13:29.202954+0000","last_fullsized":"2024-04-17T07:13:29.202954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:58.203737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'14","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206746+0000","last_change":"2024-04-17T07:11:54.457954+0000","last_active":"2024-04-17T07:13:29.206746+0000","last_peered":"2024-04-17T07:13:29.206746+0000","last_clean":"2024-04-17T07:13:29.206746+0000","last_became_active":"2024-04-17T07:11:54.457560+0000","last_became_peered":"2024-04-17T07:11:54.457560+0000","last_unstale":"2024-04-17T07:13:29.206746+0000","last_undegraded":"2024-04-17T07:13:29.206746+0000","last_fullsized":"2024-04-17T07:13:29.206746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:31.919730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068769+0000","last_change":"2024-04-17T07:11:55.480475+0000","last_active":"2024-04-17T07:13:57.068769+0000","last_peered":"2024-04-17T07:13:57.068769+0000","last_clean":"2024-04-17T07:13:57.068769+0000","last_became_active":"2024-04-17T07:11:55.480060+0000","last_became_peered":"2024-04-17T07:11:55.480060+0000","last_unstale":"2024-04-17T07:13:57.068769+0000","last_undegraded":"2024-04-17T07:13:57.068769+0000","last_fullsized":"2024-04-17T07:13:57.068769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:30:23.600244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203636+0000","last_change":"2024-04-17T07:11:59.525422+0000","last_active":"2024-04-17T07:13:29.203636+0000","last_peered":"2024-04-17T07:13:29.203636+0000","last_clean":"2024-04-17T07:13:29.203636+0000","last_became_active":"2024-04-17T07:11:59.523649+0000","last_became_peered":"2024-04-17T07:11:59.523649+0000","last_unstale":"2024-04-17T07:13:29.203636+0000","last_undegraded":"2024-04-17T07:13:29.203636+0000","last_fullsized":"2024-04-17T07:13:29.203636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:16.653752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175977+0000","last_change":"2024-04-17T07:11:54.465824+0000","last_active":"2024-04-17T07:13:29.175977+0000","last_peered":"2024-04-17T07:13:29.175977+0000","last_clean":"2024-04-17T07:13:29.175977+0000","last_became_active":"2024-04-17T07:11:54.465348+0000","last_became_peered":"2024-04-17T07:11:54.465348+0000","last_unstale":"2024-04-17T07:13:29.175977+0000","last_undegraded":"2024-04-17T07:13:29.175977+0000","last_fullsized":"2024-04-17T07:13:29.175977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:17.729903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078824+0000","last_change":"2024-04-17T07:11:55.481666+0000","last_active":"2024-04-17T07:13:57.078824+0000","last_peered":"2024-04-17T07:13:57.078824+0000","last_clean":"2024-04-17T07:13:57.078824+0000","last_became_active":"2024-04-17T07:11:55.480332+0000","last_became_peered":"2024-04-17T07:11:55.480332+0000","last_unstale":"2024-04-17T07:13:57.078824+0000","last_undegraded":"2024-04-17T07:13:57.078824+0000","last_fullsized":"2024-04-17T07:13:57.078824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:19.825407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175471+0000","last_change":"2024-04-17T07:11:59.527850+0000","last_active":"2024-04-17T07:13:29.175471+0000","last_peered":"2024-04-17T07:13:29.175471+0000","last_clean":"2024-04-17T07:13:29.175471+0000","last_became_active":"2024-04-17T07:11:59.527306+0000","last_became_peered":"2024-04-17T07:11:59.527306+0000","last_unstale":"2024-04-17T07:13:29.175471+0000","last_undegraded":"2024-04-17T07:13:29.175471+0000","last_fullsized":"2024-04-17T07:13:29.175471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:35.906222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204070+0000","last_change":"2024-04-17T07:11:54.455862+0000","last_active":"2024-04-17T07:13:29.204070+0000","last_peered":"2024-04-17T07:13:29.204070+0000","last_clean":"2024-04-17T07:13:29.204070+0000","last_became_active":"2024-04-17T07:11:54.453904+0000","last_became_peered":"2024-04-17T07:11:54.453904+0000","last_unstale":"2024-04-17T07:13:29.204070+0000","last_undegraded":"2024-04-17T07:13:29.204070+0000","last_fullsized":"2024-04-17T07:13:29.204070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:01.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.019625+0000","last_change":"2024-04-17T07:11:55.470315+0000","last_active":"2024-04-17T07:13:57.019625+0000","last_peered":"2024-04-17T07:13:57.019625+0000","last_clean":"2024-04-17T07:13:57.019625+0000","last_became_active":"2024-04-17T07:11:55.469829+0000","last_became_peered":"2024-04-17T07:11:55.469829+0000","last_unstale":"2024-04-17T07:13:57.019625+0000","last_undegraded":"2024-04-17T07:13:57.019625+0000","last_fullsized":"2024-04-17T07:13:57.019625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:48.197546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209882+0000","last_change":"2024-04-17T07:12:29.125447+0000","last_active":"2024-04-17T07:13:29.209882+0000","last_peered":"2024-04-17T07:13:29.209882+0000","last_clean":"2024-04-17T07:13:29.209882+0000","last_became_active":"2024-04-17T07:12:29.124551+0000","last_became_peered":"2024-04-17T07:12:29.124551+0000","last_unstale":"2024-04-17T07:13:29.209882+0000","last_undegraded":"2024-04-17T07:13:29.209882+0000","last_fullsized":"2024-04-17T07:13:29.209882+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:09.489755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204239+0000","last_change":"2024-04-17T07:11:54.453117+0000","last_active":"2024-04-17T07:13:29.204239+0000","last_peered":"2024-04-17T07:13:29.204239+0000","last_clean":"2024-04-17T07:13:29.204239+0000","last_became_active":"2024-04-17T07:11:54.452632+0000","last_became_peered":"2024-04-17T07:11:54.452632+0000","last_unstale":"2024-04-17T07:13:29.204239+0000","last_undegraded":"2024-04-17T07:13:29.204239+0000","last_fullsized":"2024-04-17T07:13:29.204239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:09.366149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.076821+0000","last_change":"2024-04-17T07:11:55.469991+0000","last_active":"2024-04-17T07:13:57.076821+0000","last_peered":"2024-04-17T07:13:57.076821+0000","last_clean":"2024-04-17T07:13:57.076821+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:13:57.076821+0000","last_undegraded":"2024-04-17T07:13:57.076821+0000","last_fullsized":"2024-04-17T07:13:57.076821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:02.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174957+0000","last_change":"2024-04-17T07:11:59.519817+0000","last_active":"2024-04-17T07:13:29.174957+0000","last_peered":"2024-04-17T07:13:29.174957+0000","last_clean":"2024-04-17T07:13:29.174957+0000","last_became_active":"2024-04-17T07:11:59.519108+0000","last_became_peered":"2024-04-17T07:11:59.519108+0000","last_unstale":"2024-04-17T07:13:29.174957+0000","last_undegraded":"2024-04-17T07:13:29.174957+0000","last_fullsized":"2024-04-17T07:13:29.174957+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:58.378837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204520+0000","last_change":"2024-04-17T07:11:54.463618+0000","last_active":"2024-04-17T07:13:29.204520+0000","last_peered":"2024-04-17T07:13:29.204520+0000","last_clean":"2024-04-17T07:13:29.204520+0000","last_became_active":"2024-04-17T07:11:54.463257+0000","last_became_peered":"2024-04-17T07:11:54.463257+0000","last_unstale":"2024-04-17T07:13:29.204520+0000","last_undegraded":"2024-04-17T07:13:29.204520+0000","last_fullsized":"2024-04-17T07:13:29.204520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:20.135338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077010+0000","last_change":"2024-04-17T07:11:55.487471+0000","last_active":"2024-04-17T07:13:57.077010+0000","last_peered":"2024-04-17T07:13:57.077010+0000","last_clean":"2024-04-17T07:13:57.077010+0000","last_became_active":"2024-04-17T07:11:55.487024+0000","last_became_peered":"2024-04-17T07:11:55.487024+0000","last_unstale":"2024-04-17T07:13:57.077010+0000","last_undegraded":"2024-04-17T07:13:57.077010+0000","last_fullsized":"2024-04-17T07:13:57.077010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:36.851210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175106+0000","last_change":"2024-04-17T07:11:59.527017+0000","last_active":"2024-04-17T07:13:29.175106+0000","last_peered":"2024-04-17T07:13:29.175106+0000","last_clean":"2024-04-17T07:13:29.175106+0000","last_became_active":"2024-04-17T07:11:59.526541+0000","last_became_peered":"2024-04-17T07:11:59.526541+0000","last_unstale":"2024-04-17T07:13:29.175106+0000","last_undegraded":"2024-04-17T07:13:29.175106+0000","last_fullsized":"2024-04-17T07:13:29.175106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:54.411647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204444+0000","last_change":"2024-04-17T07:11:54.465671+0000","last_active":"2024-04-17T07:13:29.204444+0000","last_peered":"2024-04-17T07:13:29.204444+0000","last_clean":"2024-04-17T07:13:29.204444+0000","last_became_active":"2024-04-17T07:11:54.464902+0000","last_became_peered":"2024-04-17T07:11:54.464902+0000","last_unstale":"2024-04-17T07:13:29.204444+0000","last_undegraded":"2024-04-17T07:13:29.204444+0000","last_fullsized":"2024-04-17T07:13:29.204444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:13:38.232965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084566+0000","last_change":"2024-04-17T07:11:55.484259+0000","last_active":"2024-04-17T07:13:57.084566+0000","last_peered":"2024-04-17T07:13:57.084566+0000","last_clean":"2024-04-17T07:13:57.084566+0000","last_became_active":"2024-04-17T07:11:55.483911+0000","last_became_peered":"2024-04-17T07:11:55.483911+0000","last_unstale":"2024-04-17T07:13:57.084566+0000","last_undegraded":"2024-04-17T07:13:57.084566+0000","last_fullsized":"2024-04-17T07:13:57.084566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:54.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204080+0000","last_change":"2024-04-17T07:11:59.525369+0000","last_active":"2024-04-17T07:13:29.204080+0000","last_peered":"2024-04-17T07:13:29.204080+0000","last_clean":"2024-04-17T07:13:29.204080+0000","last_became_active":"2024-04-17T07:11:59.524809+0000","last_became_peered":"2024-04-17T07:11:59.524809+0000","last_unstale":"2024-04-17T07:13:29.204080+0000","last_undegraded":"2024-04-17T07:13:29.204080+0000","last_fullsized":"2024-04-17T07:13:29.204080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:20.511001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'5","reported_seq":128,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:59.490212+0000","last_change":"2024-04-17T07:11:54.459079+0000","last_active":"2024-04-17T07:13:59.490212+0000","last_peered":"2024-04-17T07:13:59.490212+0000","last_clean":"2024-04-17T07:13:59.490212+0000","last_became_active":"2024-04-17T07:11:54.457008+0000","last_became_peered":"2024-04-17T07:11:54.457008+0000","last_unstale":"2024-04-17T07:13:59.490212+0000","last_undegraded":"2024-04-17T07:13:59.490212+0000","last_fullsized":"2024-04-17T07:13:59.490212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:36.084512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083899+0000","last_change":"2024-04-17T07:11:55.484453+0000","last_active":"2024-04-17T07:13:57.083899+0000","last_peered":"2024-04-17T07:13:57.083899+0000","last_clean":"2024-04-17T07:13:57.083899+0000","last_became_active":"2024-04-17T07:11:55.483152+0000","last_became_peered":"2024-04-17T07:11:55.483152+0000","last_unstale":"2024-04-17T07:13:57.083899+0000","last_undegraded":"2024-04-17T07:13:57.083899+0000","last_fullsized":"2024-04-17T07:13:57.083899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:32.224868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.202572+0000","last_change":"2024-04-17T07:11:59.521487+0000","last_active":"2024-04-17T07:13:29.202572+0000","last_peered":"2024-04-17T07:13:29.202572+0000","last_clean":"2024-04-17T07:13:29.202572+0000","last_became_active":"2024-04-17T07:11:59.521060+0000","last_became_peered":"2024-04-17T07:11:59.521060+0000","last_unstale":"2024-04-17T07:13:29.202572+0000","last_undegraded":"2024-04-17T07:13:29.202572+0000","last_fullsized":"2024-04-17T07:13:29.202572+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:06.692434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174431+0000","last_change":"2024-04-17T07:11:54.459368+0000","last_active":"2024-04-17T07:13:29.174431+0000","last_peered":"2024-04-17T07:13:29.174431+0000","last_clean":"2024-04-17T07:13:29.174431+0000","last_became_active":"2024-04-17T07:11:54.458973+0000","last_became_peered":"2024-04-17T07:11:54.458973+0000","last_unstale":"2024-04-17T07:13:29.174431+0000","last_undegraded":"2024-04-17T07:13:29.174431+0000","last_fullsized":"2024-04-17T07:13:29.174431+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:22.034110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.064313+0000","last_change":"2024-04-17T07:11:55.470648+0000","last_active":"2024-04-17T07:13:57.064313+0000","last_peered":"2024-04-17T07:13:57.064313+0000","last_clean":"2024-04-17T07:13:57.064313+0000","last_became_active":"2024-04-17T07:11:55.469666+0000","last_became_peered":"2024-04-17T07:11:55.469666+0000","last_unstale":"2024-04-17T07:13:57.064313+0000","last_undegraded":"2024-04-17T07:13:57.064313+0000","last_fullsized":"2024-04-17T07:13:57.064313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:59.208872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172289+0000","last_change":"2024-04-17T07:11:59.524027+0000","last_active":"2024-04-17T07:13:29.172289+0000","last_peered":"2024-04-17T07:13:29.172289+0000","last_clean":"2024-04-17T07:13:29.172289+0000","last_became_active":"2024-04-17T07:11:59.522764+0000","last_became_peered":"2024-04-17T07:11:59.522764+0000","last_unstale":"2024-04-17T07:13:29.172289+0000","last_undegraded":"2024-04-17T07:13:29.172289+0000","last_fullsized":"2024-04-17T07:13:29.172289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:40.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173104+0000","last_change":"2024-04-17T07:11:54.472884+0000","last_active":"2024-04-17T07:13:29.173104+0000","last_peered":"2024-04-17T07:13:29.173104+0000","last_clean":"2024-04-17T07:13:29.173104+0000","last_became_active":"2024-04-17T07:11:54.472305+0000","last_became_peered":"2024-04-17T07:11:54.472305+0000","last_unstale":"2024-04-17T07:13:29.173104+0000","last_undegraded":"2024-04-17T07:13:29.173104+0000","last_fullsized":"2024-04-17T07:13:29.173104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:21.031442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070576+0000","last_change":"2024-04-17T07:11:55.479982+0000","last_active":"2024-04-17T07:13:57.070576+0000","last_peered":"2024-04-17T07:13:57.070576+0000","last_clean":"2024-04-17T07:13:57.070576+0000","last_became_active":"2024-04-17T07:11:55.477594+0000","last_became_peered":"2024-04-17T07:11:55.477594+0000","last_unstale":"2024-04-17T07:13:57.070576+0000","last_undegraded":"2024-04-17T07:13:57.070576+0000","last_fullsized":"2024-04-17T07:13:57.070576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:23.507312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070282+0000","last_change":"2024-04-17T07:11:55.480941+0000","last_active":"2024-04-17T07:13:57.070282+0000","last_peered":"2024-04-17T07:13:57.070282+0000","last_clean":"2024-04-17T07:13:57.070282+0000","last_became_active":"2024-04-17T07:11:55.480180+0000","last_became_peered":"2024-04-17T07:11:55.480180+0000","last_unstale":"2024-04-17T07:13:57.070282+0000","last_undegraded":"2024-04-17T07:13:57.070282+0000","last_fullsized":"2024-04-17T07:13:57.070282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:16.321806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071075+0000","last_change":"2024-04-17T07:11:55.477663+0000","last_active":"2024-04-17T07:13:57.071075+0000","last_peered":"2024-04-17T07:13:57.071075+0000","last_clean":"2024-04-17T07:13:57.071075+0000","last_became_active":"2024-04-17T07:11:55.477304+0000","last_became_peered":"2024-04-17T07:11:55.477304+0000","last_unstale":"2024-04-17T07:13:57.071075+0000","last_undegraded":"2024-04-17T07:13:57.071075+0000","last_fullsized":"2024-04-17T07:13:57.071075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:00.796633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077531+0000","last_change":"2024-04-17T07:11:55.475536+0000","last_active":"2024-04-17T07:13:57.077531+0000","last_peered":"2024-04-17T07:13:57.077531+0000","last_clean":"2024-04-17T07:13:57.077531+0000","last_became_active":"2024-04-17T07:11:55.475138+0000","last_became_peered":"2024-04-17T07:11:55.475138+0000","last_unstale":"2024-04-17T07:13:57.077531+0000","last_undegraded":"2024-04-17T07:13:57.077531+0000","last_fullsized":"2024-04-17T07:13:57.077531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:47.849076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081347+0000","last_change":"2024-04-17T07:11:55.483742+0000","last_active":"2024-04-17T07:13:57.081347+0000","last_peered":"2024-04-17T07:13:57.081347+0000","last_clean":"2024-04-17T07:13:57.081347+0000","last_became_active":"2024-04-17T07:11:55.483044+0000","last_became_peered":"2024-04-17T07:11:55.483044+0000","last_unstale":"2024-04-17T07:13:57.081347+0000","last_undegraded":"2024-04-17T07:13:57.081347+0000","last_fullsized":"2024-04-17T07:13:57.081347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:18.972137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078144+0000","last_change":"2024-04-17T07:11:55.471266+0000","last_active":"2024-04-17T07:13:57.078144+0000","last_peered":"2024-04-17T07:13:57.078144+0000","last_clean":"2024-04-17T07:13:57.078144+0000","last_became_active":"2024-04-17T07:11:55.470897+0000","last_became_peered":"2024-04-17T07:11:55.470897+0000","last_unstale":"2024-04-17T07:13:57.078144+0000","last_undegraded":"2024-04-17T07:13:57.078144+0000","last_fullsized":"2024-04-17T07:13:57.078144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:25.811064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.066053+0000","last_change":"2024-04-17T07:11:55.475860+0000","last_active":"2024-04-17T07:13:57.066053+0000","last_peered":"2024-04-17T07:13:57.066053+0000","last_clean":"2024-04-17T07:13:57.066053+0000","last_became_active":"2024-04-17T07:11:55.475522+0000","last_became_peered":"2024-04-17T07:11:55.475522+0000","last_unstale":"2024-04-17T07:13:57.066053+0000","last_undegraded":"2024-04-17T07:13:57.066053+0000","last_fullsized":"2024-04-17T07:13:57.066053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:41.548964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077945+0000","last_change":"2024-04-17T07:11:55.478815+0000","last_active":"2024-04-17T07:13:57.077945+0000","last_peered":"2024-04-17T07:13:57.077945+0000","last_clean":"2024-04-17T07:13:57.077945+0000","last_became_active":"2024-04-17T07:11:55.478347+0000","last_became_peered":"2024-04-17T07:11:55.478347+0000","last_unstale":"2024-04-17T07:13:57.077945+0000","last_undegraded":"2024-04-17T07:13:57.077945+0000","last_fullsized":"2024-04-17T07:13:57.077945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:16.465511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.062327+0000","last_change":"2024-04-17T07:11:55.474208+0000","last_active":"2024-04-17T07:13:57.062327+0000","last_peered":"2024-04-17T07:13:57.062327+0000","last_clean":"2024-04-17T07:13:57.062327+0000","last_became_active":"2024-04-17T07:11:55.472042+0000","last_became_peered":"2024-04-17T07:11:55.472042+0000","last_unstale":"2024-04-17T07:13:57.062327+0000","last_undegraded":"2024-04-17T07:13:57.062327+0000","last_fullsized":"2024-04-17T07:13:57.062327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:37:24.248506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069884+0000","last_change":"2024-04-17T07:11:55.477257+0000","last_active":"2024-04-17T07:13:57.069884+0000","last_peered":"2024-04-17T07:13:57.069884+0000","last_clean":"2024-04-17T07:13:57.069884+0000","last_became_active":"2024-04-17T07:11:55.473491+0000","last_became_peered":"2024-04-17T07:11:55.473491+0000","last_unstale":"2024-04-17T07:13:57.069884+0000","last_undegraded":"2024-04-17T07:13:57.069884+0000","last_fullsized":"2024-04-17T07:13:57.069884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:33.737485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068142+0000","last_change":"2024-04-17T07:11:55.474930+0000","last_active":"2024-04-17T07:13:57.068142+0000","last_peered":"2024-04-17T07:13:57.068142+0000","last_clean":"2024-04-17T07:13:57.068142+0000","last_became_active":"2024-04-17T07:11:55.472007+0000","last_became_peered":"2024-04-17T07:11:55.472007+0000","last_unstale":"2024-04-17T07:13:57.068142+0000","last_undegraded":"2024-04-17T07:13:57.068142+0000","last_fullsized":"2024-04-17T07:13:57.068142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:07.111267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084363+0000","last_change":"2024-04-17T07:11:55.492422+0000","last_active":"2024-04-17T07:13:57.084363+0000","last_peered":"2024-04-17T07:13:57.084363+0000","last_clean":"2024-04-17T07:13:57.084363+0000","last_became_active":"2024-04-17T07:11:55.491906+0000","last_became_peered":"2024-04-17T07:11:55.491906+0000","last_unstale":"2024-04-17T07:13:57.084363+0000","last_undegraded":"2024-04-17T07:13:57.084363+0000","last_fullsized":"2024-04-17T07:13:57.084363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:13:05.322586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077807+0000","last_change":"2024-04-17T07:11:55.479436+0000","last_active":"2024-04-17T07:13:57.077807+0000","last_peered":"2024-04-17T07:13:57.077807+0000","last_clean":"2024-04-17T07:13:57.077807+0000","last_became_active":"2024-04-17T07:11:55.478982+0000","last_became_peered":"2024-04-17T07:11:55.478982+0000","last_unstale":"2024-04-17T07:13:57.077807+0000","last_undegraded":"2024-04-17T07:13:57.077807+0000","last_fullsized":"2024-04-17T07:13:57.077807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:20.663457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070158+0000","last_change":"2024-04-17T07:11:55.476708+0000","last_active":"2024-04-17T07:13:57.070158+0000","last_peered":"2024-04-17T07:13:57.070158+0000","last_clean":"2024-04-17T07:13:57.070158+0000","last_became_active":"2024-04-17T07:11:55.476344+0000","last_became_peered":"2024-04-17T07:11:55.476344+0000","last_unstale":"2024-04-17T07:13:57.070158+0000","last_undegraded":"2024-04-17T07:13:57.070158+0000","last_fullsized":"2024-04-17T07:13:57.070158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:58.874987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279575,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69204,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":945,"num_read_kb":551,"num_write":248,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207612,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":520240,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645362,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32324,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712828,"statfs":{"total":95995035648,"available":95961935872,"internally_reserved":0,"allocated":1175552,"data_stored":1169194,"data_compressed":64060,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808888,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":524154,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070527,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":520324,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332166,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1117074,"data_compressed":62211,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495693,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":515562,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":579996,"data_compressed":28497,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":517018,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":578440,"data_compressed":28497,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182497,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":515270,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920859,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":519636,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757332,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":1112656,"data_compressed":62211,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65156,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63600,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63986,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:14:05.543 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-17T07:14:05.543 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-17T07:14:05.543 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-17T07:14:05.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph health --format=json 2024-04-17T07:14:06.031 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:14:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:06 smithi098 ceph-mon[35418]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:14:06.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:06 smithi098 ceph-mon[35418]: pgmap v432: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-17T07:14:06.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:06 smithi103 ceph-mon[35495]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:14:06.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:06 smithi103 ceph-mon[35495]: pgmap v432: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-17T07:14:06.538 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:06 smithi086 ceph-mon[31733]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:14:06.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:06 smithi086 ceph-mon[31733]: pgmap v432: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-17T07:14:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.282+0000 7f09a7e00700 1 -- 172.21.15.86:0/2510929012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a810f040 msgr2=0x7f09a810f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.282+0000 7f09a7e00700 1 --2- 172.21.15.86:0/2510929012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a810f040 0x7f09a810f420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0998009990 tx=0x7f0998009ca0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.282+0000 7f09a7e00700 1 -- 172.21.15.86:0/2510929012 shutdown_connections 2024-04-17T07:14:08.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.282+0000 7f09a7e00700 1 --2- 172.21.15.86:0/2510929012 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a8109950 0x7f09a8115df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.282+0000 7f09a7e00700 1 --2- 172.21.15.86:0/2510929012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a810f040 0x7f09a810f420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.282+0000 7f09a7e00700 1 --2- 172.21.15.86:0/2510929012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a8108f90 0x7f09a8109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.282+0000 7f09a7e00700 1 -- 172.21.15.86:0/2510929012 >> 172.21.15.86:0/2510929012 conn(0x7f09a80787e0 msgr2=0x7f09a8107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.282+0000 7f09a7e00700 1 -- 172.21.15.86:0/2510929012 shutdown_connections 2024-04-17T07:14:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.282+0000 7f09a7e00700 1 -- 172.21.15.86:0/2510929012 wait complete. 2024-04-17T07:14:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.283+0000 7f09a7e00700 1 Processor -- start 2024-04-17T07:14:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.283+0000 7f09a7e00700 1 -- start start 2024-04-17T07:14:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a7e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a8108f90 0x7f09a819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a7e00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a8109950 0x7f09a819fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a7e00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a810f040 0x7f09a81a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a7e00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8119150 con 0x7f09a8108f90 2024-04-17T07:14:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a7e00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8118fd0 con 0x7f09a810f040 2024-04-17T07:14:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a7e00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81192d0 con 0x7f09a8109950 2024-04-17T07:14:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a8108f90 0x7f09a819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a6a00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a8108f90 0x7f09a819f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37586/0 (socket says 172.21.15.86:37586) 2024-04-17T07:14:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a6a00700 1 -- 172.21.15.86:0/1309762758 learned_addr learned my addr 172.21.15.86:0/1309762758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a6000700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a8109950 0x7f09a819fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a7400700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a810f040 0x7f09a81a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a6a00700 1 -- 172.21.15.86:0/1309762758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a8109950 msgr2=0x7f09a819fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a6a00700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a8109950 0x7f09a819fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.284+0000 7f09a6a00700 1 -- 172.21.15.86:0/1309762758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a810f040 msgr2=0x7f09a81a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.285+0000 7f09a6a00700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a810f040 0x7f09a81a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.285+0000 7f09a6a00700 1 -- 172.21.15.86:0/1309762758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a81a4510 con 0x7f09a8108f90 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.285+0000 7f09a7400700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a810f040 0x7f09a81a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.285+0000 7f09a6a00700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a8108f90 0x7f09a819f500 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0998000c00 tx=0x7f09980036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.285+0000 7f09aca00700 1 -- 172.21.15.86:0/1309762758 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998027070 con 0x7f09a8108f90 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.285+0000 7f09aca00700 1 -- 172.21.15.86:0/1309762758 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0998004090 con 0x7f09a8108f90 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.285+0000 7f09a7e00700 1 -- 172.21.15.86:0/1309762758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a81a47a0 con 0x7f09a8108f90 2024-04-17T07:14:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.285+0000 7f09aca00700 1 -- 172.21.15.86:0/1309762758 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998022e20 con 0x7f09a8108f90 2024-04-17T07:14:08.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.285+0000 7f09a7e00700 1 -- 172.21.15.86:0/1309762758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a81ac080 con 0x7f09a8108f90 2024-04-17T07:14:08.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.287+0000 7f09a7e00700 1 -- 172.21.15.86:0/1309762758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a8069740 con 0x7f09a8108f90 2024-04-17T07:14:08.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.289+0000 7f09aca00700 1 -- 172.21.15.86:0/1309762758 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f099802a070 con 0x7f09a8108f90 2024-04-17T07:14:08.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.290+0000 7f09aca00700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f098c0a3cc0 0x7f098c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:08.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.290+0000 7f09aca00700 1 -- 172.21.15.86:0/1309762758 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f09980d19a0 con 0x7f09a8108f90 2024-04-17T07:14:08.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.290+0000 7f09a6000700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f098c0a3cc0 0x7f098c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.291+0000 7f09a6000700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f098c0a3cc0 0x7f098c0a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0994005ea0 tx=0x7f0994005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.295+0000 7f09aca00700 1 -- 172.21.15.86:0/1309762758 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f099805eff0 con 0x7f09a8108f90 2024-04-17T07:14:08.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:08 smithi098 ceph-mon[35418]: pgmap v433: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-17T07:14:08.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:08 smithi103 ceph-mon[35495]: pgmap v433: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-17T07:14:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.572+0000 7f09a7e00700 1 -- 172.21.15.86:0/1309762758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f09a802df90 con 0x7f09a8108f90 2024-04-17T07:14:08.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:08 smithi086 ceph-mon[31733]: pgmap v433: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-17T07:14:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.573+0000 7f09aca00700 1 -- 172.21.15.86:0/1309762758 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0998013070 con 0x7f09a8108f90 2024-04-17T07:14:08.573 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:08.573 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-17T07:14:08.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.575+0000 7f09a7e00700 1 -- 172.21.15.86:0/1309762758 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f098c0a3cc0 msgr2=0x7f098c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:08.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.575+0000 7f09a7e00700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f098c0a3cc0 0x7f098c0a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0994005ea0 tx=0x7f0994005e30 comp rx=0 tx=0).stop 2024-04-17T07:14:08.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.575+0000 7f09a7e00700 1 -- 172.21.15.86:0/1309762758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a8108f90 msgr2=0x7f09a819f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:08.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.575+0000 7f09a7e00700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a8108f90 0x7f09a819f500 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0998000c00 tx=0x7f09980036f0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.575+0000 7f09a7e00700 1 -- 172.21.15.86:0/1309762758 shutdown_connections 2024-04-17T07:14:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.575+0000 7f09a7e00700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f098c0a3cc0 0x7f098c0a6180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.575+0000 7f09a7e00700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a810f040 0x7f09a81a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.575+0000 7f09a7e00700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09a8108f90 0x7f09a819f500 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.575+0000 7f09a7e00700 1 --2- 172.21.15.86:0/1309762758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09a8109950 0x7f09a819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.575+0000 7f09a7e00700 1 -- 172.21.15.86:0/1309762758 >> 172.21.15.86:0/1309762758 conn(0x7f09a80787e0 msgr2=0x7f09a80798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.576+0000 7f09a7e00700 1 -- 172.21.15.86:0/1309762758 shutdown_connections 2024-04-17T07:14:08.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:08.576+0000 7f09a7e00700 1 -- 172.21.15.86:0/1309762758 wait complete. 2024-04-17T07:14:09.396 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-17T07:14:09.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:14:09.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:09 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1309762758' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-17T07:14:09.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:09 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1309762758' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-17T07:14:09.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:09 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1309762758' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-17T07:14:09.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.619+0000 7f3a8d600640 1 -- 172.21.15.86:0/905339039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a881083e0 msgr2=0x7f3a88108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:09.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.619+0000 7f3a8d600640 1 --2- 172.21.15.86:0/905339039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a881083e0 0x7f3a88108800 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009ca0 tx=0x7f3a7802c0f0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.619+0000 7f3a8d600640 1 -- 172.21.15.86:0/905339039 shutdown_connections 2024-04-17T07:14:09.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.619+0000 7f3a8d600640 1 --2- 172.21.15.86:0/905339039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a88109f90 0x7f3a88110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.619+0000 7f3a8d600640 1 --2- 172.21.15.86:0/905339039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a881095d0 0x7f3a88109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.619+0000 7f3a8d600640 1 --2- 172.21.15.86:0/905339039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a881083e0 0x7f3a88108800 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.619+0000 7f3a8d600640 1 -- 172.21.15.86:0/905339039 >> 172.21.15.86:0/905339039 conn(0x7f3a880773a0 msgr2=0x7f3a880797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:09.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.619+0000 7f3a8d600640 1 -- 172.21.15.86:0/905339039 shutdown_connections 2024-04-17T07:14:09.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.619+0000 7f3a8d600640 1 -- 172.21.15.86:0/905339039 wait complete. 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.620+0000 7f3a8d600640 1 Processor -- start 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.620+0000 7f3a8d600640 1 -- start start 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.620+0000 7f3a8d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a881083e0 0x7f3a8819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.620+0000 7f3a8d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a881095d0 0x7f3a8819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.620+0000 7f3a8d600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a88109f90 0x7f3a881a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.620+0000 7f3a8d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88113fb0 con 0x7f3a881083e0 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.620+0000 7f3a8d600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88113e30 con 0x7f3a88109f90 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.620+0000 7f3a8d600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88114130 con 0x7f3a881095d0 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a881083e0 0x7f3a8819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a881083e0 0x7f3a8819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37596/0 (socket says 172.21.15.86:37596) 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a87e00640 1 -- 172.21.15.86:0/2131422159 learned_addr learned my addr 172.21.15.86:0/2131422159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:09.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a87e00640 1 -- 172.21.15.86:0/2131422159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a881095d0 msgr2=0x7f3a8819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a87e00640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a881095d0 0x7f3a8819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a87e00640 1 -- 172.21.15.86:0/2131422159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a88109f90 msgr2=0x7f3a881a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a8cc00640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a88109f90 0x7f3a881a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a87e00640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a88109f90 0x7f3a881a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a87e00640 1 -- 172.21.15.86:0/2131422159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a881a6300 con 0x7f3a881083e0 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a8cc00640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a88109f90 0x7f3a881a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.621+0000 7f3a87e00640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a881083e0 0x7f3a8819e6e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3a78032380 tx=0x7f3a78032890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.622+0000 7f3a84c00640 1 -- 172.21.15.86:0/2131422159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78041020 con 0x7f3a881083e0 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.622+0000 7f3a84c00640 1 -- 172.21.15.86:0/2131422159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a78003e20 con 0x7f3a881083e0 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.622+0000 7f3a84c00640 1 -- 172.21.15.86:0/2131422159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78039040 con 0x7f3a881083e0 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.622+0000 7f3a8d600640 1 -- 172.21.15.86:0/2131422159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a881a6590 con 0x7f3a881083e0 2024-04-17T07:14:09.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.622+0000 7f3a8d600640 1 -- 172.21.15.86:0/2131422159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a881a6b00 con 0x7f3a881083e0 2024-04-17T07:14:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.623+0000 7f3a8d600640 1 -- 172.21.15.86:0/2131422159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8810f780 con 0x7f3a881083e0 2024-04-17T07:14:09.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.625+0000 7f3a84c00640 1 -- 172.21.15.86:0/2131422159 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3a78044020 con 0x7f3a881083e0 2024-04-17T07:14:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.626+0000 7f3a84c00640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a6c0a3c00 0x7f3a6c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.626+0000 7f3a84c00640 1 -- 172.21.15.86:0/2131422159 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3a780f2c20 con 0x7f3a881083e0 2024-04-17T07:14:09.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.627+0000 7f3a87400640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a6c0a3c00 0x7f3a6c0a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:09.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.627+0000 7f3a87400640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a6c0a3c00 0x7f3a6c0a60c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a8819fbf0 tx=0x7f3a70005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:09.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.631+0000 7f3a84c00640 1 -- 172.21.15.86:0/2131422159 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3a7804b020 con 0x7f3a881083e0 2024-04-17T07:14:09.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.834+0000 7f3a8d600640 1 -- 172.21.15.86:0/2131422159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a88062b60 con 0x7f3a881083e0 2024-04-17T07:14:09.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.839+0000 7f3a84c00640 1 -- 172.21.15.86:0/2131422159 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7f3a780bf220 con 0x7f3a881083e0 2024-04-17T07:14:09.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:09.839 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:14:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.841+0000 7f3a8d600640 1 -- 172.21.15.86:0/2131422159 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a6c0a3c00 msgr2=0x7f3a6c0a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.841+0000 7f3a8d600640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a6c0a3c00 0x7f3a6c0a60c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a8819fbf0 tx=0x7f3a70005bf0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.841+0000 7f3a8d600640 1 -- 172.21.15.86:0/2131422159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a881083e0 msgr2=0x7f3a8819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.841+0000 7f3a8d600640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a881083e0 0x7f3a8819e6e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3a78032380 tx=0x7f3a78032890 comp rx=0 tx=0).stop 2024-04-17T07:14:09.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.842+0000 7f3a8d600640 1 -- 172.21.15.86:0/2131422159 shutdown_connections 2024-04-17T07:14:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.842+0000 7f3a8d600640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a6c0a3c00 0x7f3a6c0a60c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.842+0000 7f3a8d600640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a88109f90 0x7f3a881a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.842+0000 7f3a8d600640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a881095d0 0x7f3a8819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.842+0000 7f3a8d600640 1 --2- 172.21.15.86:0/2131422159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a881083e0 0x7f3a8819e6e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.842+0000 7f3a8d600640 1 -- 172.21.15.86:0/2131422159 >> 172.21.15.86:0/2131422159 conn(0x7f3a880773a0 msgr2=0x7f3a8810c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.842+0000 7f3a8d600640 1 -- 172.21.15.86:0/2131422159 shutdown_connections 2024-04-17T07:14:09.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:09.842+0000 7f3a8d600640 1 -- 172.21.15.86:0/2131422159 wait complete. 2024-04-17T07:14:09.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:14:09.862 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:14:10.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.077+0000 7fa657e00640 1 -- 172.21.15.86:0/4088195860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6581083c0 msgr2=0x7fa6581087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.077+0000 7fa657e00640 1 --2- 172.21.15.86:0/4088195860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6581083c0 0x7fa6581087e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa648009900 tx=0x7fa64802bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:10.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.077+0000 7fa657e00640 1 -- 172.21.15.86:0/4088195860 shutdown_connections 2024-04-17T07:14:10.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.077+0000 7fa657e00640 1 --2- 172.21.15.86:0/4088195860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa658109f70 0x7fa658110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.077+0000 7fa657e00640 1 --2- 172.21.15.86:0/4088195860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6581095b0 0x7fa658109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.077+0000 7fa657e00640 1 --2- 172.21.15.86:0/4088195860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6581083c0 0x7fa6581087e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.077+0000 7fa657e00640 1 -- 172.21.15.86:0/4088195860 >> 172.21.15.86:0/4088195860 conn(0x7fa658077360 msgr2=0x7fa6580797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:10.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.077+0000 7fa657e00640 1 -- 172.21.15.86:0/4088195860 shutdown_connections 2024-04-17T07:14:10.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.077+0000 7fa657e00640 1 -- 172.21.15.86:0/4088195860 wait complete. 2024-04-17T07:14:10.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.079+0000 7fa657e00640 1 Processor -- start 2024-04-17T07:14:10.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.079+0000 7fa657e00640 1 -- start start 2024-04-17T07:14:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa657e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6581083c0 0x7fa65819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa657e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6581095b0 0x7fa65819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa657e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa658109f70 0x7fa6581a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa657e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa658113f50 con 0x7fa6581095b0 2024-04-17T07:14:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa657e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa658113dd0 con 0x7fa658109f70 2024-04-17T07:14:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa657e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581140d0 con 0x7fa6581083c0 2024-04-17T07:14:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa656000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6581095b0 0x7fa65819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:10.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa656000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6581095b0 0x7fa65819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37614/0 (socket says 172.21.15.86:37614) 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa656000640 1 -- 172.21.15.86:0/1460379952 learned_addr learned my addr 172.21.15.86:0/1460379952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa656a00640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6581083c0 0x7fa65819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa656000640 1 -- 172.21.15.86:0/1460379952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6581083c0 msgr2=0x7fa65819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa656000640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6581083c0 0x7fa65819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa657400640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa658109f70 0x7fa6581a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa656000640 1 -- 172.21.15.86:0/1460379952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa658109f70 msgr2=0x7fa6581a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa656000640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa658109f70 0x7fa6581a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.080+0000 7fa656000640 1 -- 172.21.15.86:0/1460379952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6581a6200 con 0x7fa6581095b0 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.081+0000 7fa656000640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6581095b0 0x7fa65819eb70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa64400d770 tx=0x7fa64400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.081+0000 7fa64f400640 1 -- 172.21.15.86:0/1460379952 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644017070 con 0x7fa6581095b0 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.081+0000 7fa64f400640 1 -- 172.21.15.86:0/1460379952 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6440054d0 con 0x7fa6581095b0 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.081+0000 7fa657e00640 1 -- 172.21.15.86:0/1460379952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6581a64f0 con 0x7fa6581095b0 2024-04-17T07:14:10.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.081+0000 7fa64f400640 1 -- 172.21.15.86:0/1460379952 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644004cf0 con 0x7fa6581095b0 2024-04-17T07:14:10.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.081+0000 7fa657e00640 1 -- 172.21.15.86:0/1460379952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6581a6a40 con 0x7fa6581095b0 2024-04-17T07:14:10.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.082+0000 7fa657e00640 1 -- 172.21.15.86:0/1460379952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa65810f7b0 con 0x7fa6581095b0 2024-04-17T07:14:10.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.085+0000 7fa64f400640 1 -- 172.21.15.86:0/1460379952 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa644004e90 con 0x7fa6581095b0 2024-04-17T07:14:10.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.086+0000 7fa64f400640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa63c0a3c30 0x7fa63c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:10.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.087+0000 7fa64f400640 1 -- 172.21.15.86:0/1460379952 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa6440d0280 con 0x7fa6581095b0 2024-04-17T07:14:10.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.087+0000 7fa656a00640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa63c0a3c30 0x7fa63c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:10.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.087+0000 7fa656a00640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa63c0a3c30 0x7fa63c0a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa64802c240 tx=0x7fa648005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:10.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.092+0000 7fa64f400640 1 -- 172.21.15.86:0/1460379952 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa64409c880 con 0x7fa6581095b0 2024-04-17T07:14:10.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.301+0000 7fa657e00640 1 -- 172.21.15.86:0/1460379952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa658062b60 con 0x7fa6581095b0 2024-04-17T07:14:10.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.302+0000 7fa64f400640 1 -- 172.21.15.86:0/1460379952 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fa64409c180 con 0x7fa6581095b0 2024-04-17T07:14:10.302 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:14:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.304+0000 7fa657e00640 1 -- 172.21.15.86:0/1460379952 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa63c0a3c30 msgr2=0x7fa63c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.304+0000 7fa657e00640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa63c0a3c30 0x7fa63c0a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa64802c240 tx=0x7fa648005fd0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.304+0000 7fa657e00640 1 -- 172.21.15.86:0/1460379952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6581095b0 msgr2=0x7fa65819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.304+0000 7fa657e00640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6581095b0 0x7fa65819eb70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa64400d770 tx=0x7fa64400dc30 comp rx=0 tx=0).stop 2024-04-17T07:14:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.304+0000 7fa657e00640 1 -- 172.21.15.86:0/1460379952 shutdown_connections 2024-04-17T07:14:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.304+0000 7fa657e00640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa63c0a3c30 0x7fa63c0a60f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.304+0000 7fa657e00640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa658109f70 0x7fa6581a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.304+0000 7fa657e00640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6581095b0 0x7fa65819eb70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.304+0000 7fa657e00640 1 --2- 172.21.15.86:0/1460379952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6581083c0 0x7fa65819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.304+0000 7fa657e00640 1 -- 172.21.15.86:0/1460379952 >> 172.21.15.86:0/1460379952 conn(0x7fa658077360 msgr2=0x7fa65810c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.305+0000 7fa657e00640 1 -- 172.21.15.86:0/1460379952 shutdown_connections 2024-04-17T07:14:10.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.305+0000 7fa657e00640 1 -- 172.21.15.86:0/1460379952 wait complete. 2024-04-17T07:14:10.325 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:14:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:10 smithi098 ceph-mon[35418]: pgmap v434: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-17T07:14:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:10 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2131422159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:10.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:10 smithi103 ceph-mon[35495]: pgmap v434: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-17T07:14:10.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:10 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2131422159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:10.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.544+0000 7faed3200640 1 -- 172.21.15.86:0/1629727372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed4105290 msgr2=0x7faed4105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.544+0000 7faed3200640 1 --2- 172.21.15.86:0/1629727372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed4105290 0x7faed4105710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faebc009900 tx=0x7faebc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:10.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.544+0000 7faed3200640 1 -- 172.21.15.86:0/1629727372 shutdown_connections 2024-04-17T07:14:10.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.544+0000 7faed3200640 1 --2- 172.21.15.86:0/1629727372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed4105c50 0x7faed410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.544+0000 7faed3200640 1 --2- 172.21.15.86:0/1629727372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed4105290 0x7faed4105710 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.544+0000 7faed3200640 1 --2- 172.21.15.86:0/1629727372 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed41040a0 0x7faed41044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.544+0000 7faed3200640 1 -- 172.21.15.86:0/1629727372 >> 172.21.15.86:0/1629727372 conn(0x7faed40ff610 msgr2=0x7faed4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.544+0000 7faed3200640 1 -- 172.21.15.86:0/1629727372 shutdown_connections 2024-04-17T07:14:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.544+0000 7faed3200640 1 -- 172.21.15.86:0/1629727372 wait complete. 2024-04-17T07:14:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.545+0000 7faed3200640 1 Processor -- start 2024-04-17T07:14:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.545+0000 7faed3200640 1 -- start start 2024-04-17T07:14:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.545+0000 7faed3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed41040a0 0x7faed419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.545+0000 7faed3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4105290 0x7faed419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.545+0000 7faed3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed4105c50 0x7faed41a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.545+0000 7faed3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faed410fbe0 con 0x7faed4105c50 2024-04-17T07:14:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.545+0000 7faed3200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faed410fa60 con 0x7faed4105290 2024-04-17T07:14:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.545+0000 7faed3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faed410fd60 con 0x7faed41040a0 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faed2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed4105c50 0x7faed41a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faed2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed4105c50 0x7faed41a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37638/0 (socket says 172.21.15.86:37638) 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faed2800640 1 -- 172.21.15.86:0/3170744231 learned_addr learned my addr 172.21.15.86:0/3170744231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faec9400640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4105290 0x7faed419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faed1e00640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed41040a0 0x7faed419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faed2800640 1 -- 172.21.15.86:0/3170744231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed41040a0 msgr2=0x7faed419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faed2800640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed41040a0 0x7faed419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faed2800640 1 -- 172.21.15.86:0/3170744231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4105290 msgr2=0x7faed419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faed2800640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4105290 0x7faed419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faed2800640 1 -- 172.21.15.86:0/3170744231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed41a1ee0 con 0x7faed4105c50 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.546+0000 7faec9400640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4105290 0x7faed419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.547+0000 7faed2800640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed4105c50 0x7faed41a1870 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faecc00eaa0 tx=0x7faecc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.547+0000 7faecb400640 1 -- 172.21.15.86:0/3170744231 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc00cdb0 con 0x7faed4105c50 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.547+0000 7faecb400640 1 -- 172.21.15.86:0/3170744231 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faecc005020 con 0x7faed4105c50 2024-04-17T07:14:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.547+0000 7faed3200640 1 -- 172.21.15.86:0/3170744231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed41a21d0 con 0x7faed4105c50 2024-04-17T07:14:10.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.547+0000 7faecb400640 1 -- 172.21.15.86:0/3170744231 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc0116d0 con 0x7faed4105c50 2024-04-17T07:14:10.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.547+0000 7faed3200640 1 -- 172.21.15.86:0/3170744231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed41a2770 con 0x7faed4105c50 2024-04-17T07:14:10.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.548+0000 7faed3200640 1 -- 172.21.15.86:0/3170744231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed410b4e0 con 0x7faed4105c50 2024-04-17T07:14:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.550+0000 7faecb400640 1 -- 172.21.15.86:0/3170744231 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faecc0051d0 con 0x7faed4105c50 2024-04-17T07:14:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.551+0000 7faecb400640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faeb80a3c30 0x7faeb80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.552+0000 7faecb400640 1 -- 172.21.15.86:0/3170744231 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faecc0d0930 con 0x7faed4105c50 2024-04-17T07:14:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.552+0000 7faed1e00640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faeb80a3c30 0x7faeb80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.552+0000 7faed1e00640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faeb80a3c30 0x7faeb80a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faec4005fd0 tx=0x7faec4005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.556+0000 7faecb400640 1 -- 172.21.15.86:0/3170744231 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faecc016030 con 0x7faed4105c50 2024-04-17T07:14:10.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:10 smithi086 ceph-mon[31733]: pgmap v434: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-17T07:14:10.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:10 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2131422159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:10.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.758+0000 7faed3200640 1 -- 172.21.15.86:0/3170744231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faed4062b60 con 0x7faed4105c50 2024-04-17T07:14:10.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.758+0000 7faecb400640 1 -- 172.21.15.86:0/3170744231 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7faecc09cf30 con 0x7faed4105c50 2024-04-17T07:14:10.759 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:14:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 -- 172.21.15.86:0/3170744231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faeb80a3c30 msgr2=0x7faeb80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faeb80a3c30 0x7faeb80a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faec4005fd0 tx=0x7faec4005c70 comp rx=0 tx=0).stop 2024-04-17T07:14:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 -- 172.21.15.86:0/3170744231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed4105c50 msgr2=0x7faed41a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed4105c50 0x7faed41a1870 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faecc00eaa0 tx=0x7faecc00ef60 comp rx=0 tx=0).stop 2024-04-17T07:14:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 -- 172.21.15.86:0/3170744231 shutdown_connections 2024-04-17T07:14:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faeb80a3c30 0x7faeb80a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed4105c50 0x7faed41a1870 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4105290 0x7faed419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 --2- 172.21.15.86:0/3170744231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed41040a0 0x7faed419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 -- 172.21.15.86:0/3170744231 >> 172.21.15.86:0/3170744231 conn(0x7faed40ff610 msgr2=0x7faed41086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 -- 172.21.15.86:0/3170744231 shutdown_connections 2024-04-17T07:14:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.761+0000 7faed3200640 1 -- 172.21.15.86:0/3170744231 wait complete. 2024-04-17T07:14:10.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:14:10.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.998+0000 7fc915800640 1 -- 172.21.15.86:0/603700878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9101041c0 msgr2=0x7fc9101045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.998+0000 7fc915800640 1 --2- 172.21.15.86:0/603700878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9101041c0 0x7fc9101045e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc90002bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.998+0000 7fc915800640 1 -- 172.21.15.86:0/603700878 shutdown_connections 2024-04-17T07:14:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.998+0000 7fc915800640 1 --2- 172.21.15.86:0/603700878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc910105d70 0x7fc91010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.998+0000 7fc915800640 1 --2- 172.21.15.86:0/603700878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9101053b0 0x7fc910105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.998+0000 7fc915800640 1 --2- 172.21.15.86:0/603700878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9101041c0 0x7fc9101045e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.998+0000 7fc915800640 1 -- 172.21.15.86:0/603700878 >> 172.21.15.86:0/603700878 conn(0x7fc9100ff750 msgr2=0x7fc910101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.998+0000 7fc915800640 1 -- 172.21.15.86:0/603700878 shutdown_connections 2024-04-17T07:14:10.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.998+0000 7fc915800640 1 -- 172.21.15.86:0/603700878 wait complete. 2024-04-17T07:14:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.999+0000 7fc915800640 1 Processor -- start 2024-04-17T07:14:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:10.999+0000 7fc915800640 1 -- start start 2024-04-17T07:14:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc915800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9101041c0 0x7fc91019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc915800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101053b0 0x7fc91019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc915800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc910105d70 0x7fc9101a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc915800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc91010fd50 con 0x7fc910105d70 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc915800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc91010fbd0 con 0x7fc9101041c0 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc915800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc91010fed0 con 0x7fc9101053b0 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc90f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc910105d70 0x7fc9101a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc90f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc910105d70 0x7fc9101a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37666/0 (socket says 172.21.15.86:37666) 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc90f400640 1 -- 172.21.15.86:0/3977440231 learned_addr learned my addr 172.21.15.86:0/3977440231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc907e00640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101053b0 0x7fc91019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc90ea00640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9101041c0 0x7fc91019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc90f400640 1 -- 172.21.15.86:0/3977440231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101053b0 msgr2=0x7fc91019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc90f400640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101053b0 0x7fc91019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc90f400640 1 -- 172.21.15.86:0/3977440231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9101041c0 msgr2=0x7fc91019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc90f400640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9101041c0 0x7fc91019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc90f400640 1 -- 172.21.15.86:0/3977440231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9101a1fc0 con 0x7fc910105d70 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.000+0000 7fc907e00640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101053b0 0x7fc91019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.001+0000 7fc90f400640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc910105d70 0x7fc9101a1950 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc90800eaa0 tx=0x7fc90800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.001+0000 7fc907400640 1 -- 172.21.15.86:0/3977440231 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90800cdb0 con 0x7fc910105d70 2024-04-17T07:14:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.001+0000 7fc907400640 1 -- 172.21.15.86:0/3977440231 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc908005020 con 0x7fc910105d70 2024-04-17T07:14:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.001+0000 7fc915800640 1 -- 172.21.15.86:0/3977440231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9101a22b0 con 0x7fc910105d70 2024-04-17T07:14:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.001+0000 7fc907400640 1 -- 172.21.15.86:0/3977440231 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080116d0 con 0x7fc910105d70 2024-04-17T07:14:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.001+0000 7fc915800640 1 -- 172.21.15.86:0/3977440231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9101a2800 con 0x7fc910105d70 2024-04-17T07:14:11.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.002+0000 7fc915800640 1 -- 172.21.15.86:0/3977440231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc91010b530 con 0x7fc910105d70 2024-04-17T07:14:11.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.004+0000 7fc907400640 1 -- 172.21.15.86:0/3977440231 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc9080051d0 con 0x7fc910105d70 2024-04-17T07:14:11.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.006+0000 7fc907400640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8f40a3bb0 0x7fc8f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.006+0000 7fc907400640 1 -- 172.21.15.86:0/3977440231 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc9080d0a20 con 0x7fc910105d70 2024-04-17T07:14:11.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.006+0000 7fc90ea00640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8f40a3bb0 0x7fc8f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.007+0000 7fc90ea00640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8f40a3bb0 0x7fc8f40a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc9000096c0 tx=0x7fc900005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:11.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.010+0000 7fc907400640 1 -- 172.21.15.86:0/3977440231 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc908016030 con 0x7fc910105d70 2024-04-17T07:14:11.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.217+0000 7fc915800640 1 -- 172.21.15.86:0/3977440231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc910062b60 con 0x7fc910105d70 2024-04-17T07:14:11.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.218+0000 7fc907400640 1 -- 172.21.15.86:0/3977440231 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fc90809d020 con 0x7fc910105d70 2024-04-17T07:14:11.219 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.221+0000 7fc915800640 1 -- 172.21.15.86:0/3977440231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8f40a3bb0 msgr2=0x7fc8f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.221+0000 7fc915800640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8f40a3bb0 0x7fc8f40a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc9000096c0 tx=0x7fc900005fd0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.221+0000 7fc915800640 1 -- 172.21.15.86:0/3977440231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc910105d70 msgr2=0x7fc9101a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.221+0000 7fc915800640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc910105d70 0x7fc9101a1950 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc90800eaa0 tx=0x7fc90800ef60 comp rx=0 tx=0).stop 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.222+0000 7fc915800640 1 -- 172.21.15.86:0/3977440231 shutdown_connections 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.222+0000 7fc915800640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8f40a3bb0 0x7fc8f40a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.222+0000 7fc915800640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc910105d70 0x7fc9101a1950 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.222+0000 7fc915800640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9101053b0 0x7fc91019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.222+0000 7fc915800640 1 --2- 172.21.15.86:0/3977440231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9101041c0 0x7fc91019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.222+0000 7fc915800640 1 -- 172.21.15.86:0/3977440231 >> 172.21.15.86:0/3977440231 conn(0x7fc9100ff750 msgr2=0x7fc910108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.222+0000 7fc915800640 1 -- 172.21.15.86:0/3977440231 shutdown_connections 2024-04-17T07:14:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.222+0000 7fc915800640 1 -- 172.21.15.86:0/3977440231 wait complete. 2024-04-17T07:14:11.242 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:14:11.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.456+0000 7ff80ac00640 1 -- 172.21.15.86:0/2719629536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8041084a0 msgr2=0x7ff8041088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.456+0000 7ff80ac00640 1 --2- 172.21.15.86:0/2719629536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8041084a0 0x7ff8041088c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009900 tx=0x7ff7f802bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:11.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.456+0000 7ff80ac00640 1 -- 172.21.15.86:0/2719629536 shutdown_connections 2024-04-17T07:14:11.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.456+0000 7ff80ac00640 1 --2- 172.21.15.86:0/2719629536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff80410a050 0x7ff8041108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.456+0000 7ff80ac00640 1 --2- 172.21.15.86:0/2719629536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff804109690 0x7ff804109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.456+0000 7ff80ac00640 1 --2- 172.21.15.86:0/2719629536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8041084a0 0x7ff8041088c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.456+0000 7ff80ac00640 1 -- 172.21.15.86:0/2719629536 >> 172.21.15.86:0/2719629536 conn(0x7ff804077430 msgr2=0x7ff804079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:11.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.456+0000 7ff80ac00640 1 -- 172.21.15.86:0/2719629536 shutdown_connections 2024-04-17T07:14:11.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.456+0000 7ff80ac00640 1 -- 172.21.15.86:0/2719629536 wait complete. 2024-04-17T07:14:11.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.457+0000 7ff80ac00640 1 Processor -- start 2024-04-17T07:14:11.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff80ac00640 1 -- start start 2024-04-17T07:14:11.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff80ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8041084a0 0x7ff80419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff80ac00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff804109690 0x7ff80419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff80ac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff80410a050 0x7ff8041a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff80ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff804113fe0 con 0x7ff8041084a0 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff80ac00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff804113e60 con 0x7ff804109690 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff80ac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff804114160 con 0x7ff80410a050 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff809800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8041084a0 0x7ff80419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff809800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8041084a0 0x7ff80419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37688/0 (socket says 172.21.15.86:37688) 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff809800640 1 -- 172.21.15.86:0/3905886232 learned_addr learned my addr 172.21.15.86:0/3905886232 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff808e00640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff804109690 0x7ff80419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.458+0000 7ff809800640 1 -- 172.21.15.86:0/3905886232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff80410a050 msgr2=0x7ff8041a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff80a200640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff80410a050 0x7ff8041a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff809800640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff80410a050 0x7ff8041a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff809800640 1 -- 172.21.15.86:0/3905886232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff804109690 msgr2=0x7ff80419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff809800640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff804109690 0x7ff80419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff809800640 1 -- 172.21.15.86:0/3905886232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8041a62b0 con 0x7ff8041084a0 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff808e00640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff804109690 0x7ff80419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:11.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff80a200640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff80410a050 0x7ff8041a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:11.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff809800640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8041084a0 0x7ff80419e690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8000c00 tx=0x7ff7f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:11.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff802000640 1 -- 172.21.15.86:0/3905886232 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8043020 con 0x7ff8041084a0 2024-04-17T07:14:11.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff802000640 1 -- 172.21.15.86:0/3905886232 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7f8004aa0 con 0x7ff8041084a0 2024-04-17T07:14:11.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff802000640 1 -- 172.21.15.86:0/3905886232 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8003d00 con 0x7ff8041084a0 2024-04-17T07:14:11.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff80ac00640 1 -- 172.21.15.86:0/3905886232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8041a6540 con 0x7ff8041084a0 2024-04-17T07:14:11.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.459+0000 7ff80ac00640 1 -- 172.21.15.86:0/3905886232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8041a6ae0 con 0x7ff8041084a0 2024-04-17T07:14:11.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.461+0000 7ff80ac00640 1 -- 172.21.15.86:0/3905886232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff80410f830 con 0x7ff8041084a0 2024-04-17T07:14:11.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.463+0000 7ff802000640 1 -- 172.21.15.86:0/3905886232 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff7f8046020 con 0x7ff8041084a0 2024-04-17T07:14:11.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.464+0000 7ff802000640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7ec0a3c30 0x7ff7ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.464+0000 7ff802000640 1 -- 172.21.15.86:0/3905886232 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff7f80f2b80 con 0x7ff8041084a0 2024-04-17T07:14:11.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.464+0000 7ff808e00640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7ec0a3c30 0x7ff7ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.465+0000 7ff808e00640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7ec0a3c30 0x7ff7ec0a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff80419fba0 tx=0x7ff7f0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:11.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.468+0000 7ff802000640 1 -- 172.21.15.86:0/3905886232 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff7f804d020 con 0x7ff8041084a0 2024-04-17T07:14:11.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:11 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1460379952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:11 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3170744231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:11.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:11 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1460379952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:11.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:11 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3170744231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:11 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1460379952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:11 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3170744231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.676+0000 7ff80ac00640 1 -- 172.21.15.86:0/3905886232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff804062b60 con 0x7ff8041084a0 2024-04-17T07:14:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.677+0000 7ff802000640 1 -- 172.21.15.86:0/3905886232 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7ff7f80bf180 con 0x7ff8041084a0 2024-04-17T07:14:11.677 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:14:11.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.680+0000 7ff80ac00640 1 -- 172.21.15.86:0/3905886232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7ec0a3c30 msgr2=0x7ff7ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.680+0000 7ff80ac00640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7ec0a3c30 0x7ff7ec0a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff80419fba0 tx=0x7ff7f0005e50 comp rx=0 tx=0).stop 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.680+0000 7ff80ac00640 1 -- 172.21.15.86:0/3905886232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8041084a0 msgr2=0x7ff80419e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.680+0000 7ff80ac00640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8041084a0 0x7ff80419e690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8000c00 tx=0x7ff7f8004270 comp rx=0 tx=0).stop 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.681+0000 7ff80ac00640 1 -- 172.21.15.86:0/3905886232 shutdown_connections 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.681+0000 7ff80ac00640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7ec0a3c30 0x7ff7ec0a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.681+0000 7ff80ac00640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff80410a050 0x7ff8041a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.681+0000 7ff80ac00640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff804109690 0x7ff80419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.681+0000 7ff80ac00640 1 --2- 172.21.15.86:0/3905886232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8041084a0 0x7ff80419e690 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.681+0000 7ff80ac00640 1 -- 172.21.15.86:0/3905886232 >> 172.21.15.86:0/3905886232 conn(0x7ff804077430 msgr2=0x7ff80410ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.681+0000 7ff80ac00640 1 -- 172.21.15.86:0/3905886232 shutdown_connections 2024-04-17T07:14:11.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.681+0000 7ff80ac00640 1 -- 172.21.15.86:0/3905886232 wait complete. 2024-04-17T07:14:11.699 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:14:11.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.914+0000 7f7cdd400640 1 -- 172.21.15.86:0/3505635367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cd8101dd0 msgr2=0x7f7cd81021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.914+0000 7f7cdd400640 1 --2- 172.21.15.86:0/3505635367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cd8101dd0 0x7f7cd81021f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009900 tx=0x7f7cc802bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:11.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.914+0000 7f7cdd400640 1 -- 172.21.15.86:0/3505635367 shutdown_connections 2024-04-17T07:14:11.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.914+0000 7f7cdd400640 1 --2- 172.21.15.86:0/3505635367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8103980 0x7f7cd810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.914+0000 7f7cdd400640 1 --2- 172.21.15.86:0/3505635367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cd8102fc0 0x7f7cd8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.914+0000 7f7cdd400640 1 --2- 172.21.15.86:0/3505635367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cd8101dd0 0x7f7cd81021f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.914+0000 7f7cdd400640 1 -- 172.21.15.86:0/3505635367 >> 172.21.15.86:0/3505635367 conn(0x7f7cd80fd340 msgr2=0x7f7cd80ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:11.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.914+0000 7f7cdd400640 1 -- 172.21.15.86:0/3505635367 shutdown_connections 2024-04-17T07:14:11.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.914+0000 7f7cdd400640 1 -- 172.21.15.86:0/3505635367 wait complete. 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.915+0000 7f7cdd400640 1 Processor -- start 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.915+0000 7f7cdd400640 1 -- start start 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.915+0000 7f7cdd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cd8101dd0 0x7f7cd8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.915+0000 7f7cdd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8102fc0 0x7f7cd8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.915+0000 7f7cdd400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cd8103980 0x7f7cd819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.915+0000 7f7cdd400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810d920 con 0x7f7cd8101dd0 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.915+0000 7f7cdd400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810d7a0 con 0x7f7cd8102fc0 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.915+0000 7f7cdd400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810daa0 con 0x7f7cd8103980 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cd8101dd0 0x7f7cd8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cd8101dd0 0x7f7cd8197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37706/0 (socket says 172.21.15.86:37706) 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7e00640 1 -- 172.21.15.86:0/3535313688 learned_addr learned my addr 172.21.15.86:0/3535313688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7400640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8102fc0 0x7f7cd8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cdca00640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cd8103980 0x7f7cd819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7e00640 1 -- 172.21.15.86:0/3535313688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cd8103980 msgr2=0x7f7cd819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7e00640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cd8103980 0x7f7cd819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7e00640 1 -- 172.21.15.86:0/3535313688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8102fc0 msgr2=0x7f7cd8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7e00640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8102fc0 0x7f7cd8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7e00640 1 -- 172.21.15.86:0/3535313688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd819fbe0 con 0x7f7cd8101dd0 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cdca00640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cd8103980 0x7f7cd819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7400640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8102fc0 0x7f7cd8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.916+0000 7f7cd7e00640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cd8101dd0 0x7f7cd8197fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8005ab0 tx=0x7f7cc80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.917+0000 7f7cd4c00640 1 -- 172.21.15.86:0/3535313688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8043020 con 0x7f7cd8101dd0 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.917+0000 7f7cd4c00640 1 -- 172.21.15.86:0/3535313688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cc80054c0 con 0x7f7cd8101dd0 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.917+0000 7f7cdd400640 1 -- 172.21.15.86:0/3535313688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd819fe70 con 0x7f7cd8101dd0 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.917+0000 7f7cd4c00640 1 -- 172.21.15.86:0/3535313688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8003de0 con 0x7f7cd8101dd0 2024-04-17T07:14:11.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.917+0000 7f7cdd400640 1 -- 172.21.15.86:0/3535313688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd81a02d0 con 0x7f7cd8101dd0 2024-04-17T07:14:11.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.918+0000 7f7cdd400640 1 -- 172.21.15.86:0/3535313688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd81090f0 con 0x7f7cd8101dd0 2024-04-17T07:14:11.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.920+0000 7f7cd4c00640 1 -- 172.21.15.86:0/3535313688 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7cc8046020 con 0x7f7cd8101dd0 2024-04-17T07:14:11.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.921+0000 7f7cd4c00640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7cbc0a3bb0 0x7f7cbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:11.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.922+0000 7f7cd4c00640 1 -- 172.21.15.86:0/3535313688 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7cc80f2d60 con 0x7f7cd8101dd0 2024-04-17T07:14:11.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.922+0000 7f7cd7400640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7cbc0a3bb0 0x7f7cbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.922+0000 7f7cd7400640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7cbc0a3bb0 0x7f7cbc0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7cd81994d0 tx=0x7f7cc0005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:11.926+0000 7f7cd4c00640 1 -- 172.21.15.86:0/3535313688 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7cc804d020 con 0x7f7cd8101dd0 2024-04-17T07:14:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.168+0000 7f7cdd400640 1 -- 172.21.15.86:0/3535313688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cd802d5d0 con 0x7f7cd8101dd0 2024-04-17T07:14:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.171+0000 7f7cd4c00640 1 -- 172.21.15.86:0/3535313688 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f7cc80bf480 con 0x7f7cd8101dd0 2024-04-17T07:14:12.171 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:12.171 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:12:31.203694+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[34460,34490,34430,34400,24505]},"id":1}]} 2024-04-17T07:14:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-04-17T07:14:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 -- 172.21.15.86:0/3535313688 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7cbc0a3bb0 msgr2=0x7f7cbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7cbc0a3bb0 0x7f7cbc0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7cd81994d0 tx=0x7f7cc0005d90 comp rx=0 tx=0).stop 2024-04-17T07:14:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 -- 172.21.15.86:0/3535313688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cd8101dd0 msgr2=0x7f7cd8197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cd8101dd0 0x7f7cd8197fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8005ab0 tx=0x7f7cc80042f0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 -- 172.21.15.86:0/3535313688 shutdown_connections 2024-04-17T07:14:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7cbc0a3bb0 0x7f7cbc0a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cd8103980 0x7f7cd819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8102fc0 0x7f7cd8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 --2- 172.21.15.86:0/3535313688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cd8101dd0 0x7f7cd8197fc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 -- 172.21.15.86:0/3535313688 >> 172.21.15.86:0/3535313688 conn(0x7f7cd80fd340 msgr2=0x7f7cd81062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 -- 172.21.15.86:0/3535313688 shutdown_connections 2024-04-17T07:14:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.174+0000 7f7cdd400640 1 -- 172.21.15.86:0/3535313688 wait complete. 2024-04-17T07:14:12.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:14:12.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.404+0000 7f4770e00640 1 -- 172.21.15.86:0/368295289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476c109f90 msgr2=0x7f476c110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:12.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.404+0000 7f4770e00640 1 --2- 172.21.15.86:0/368295289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476c109f90 0x7f476c110820 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4764009ca0 tx=0x7f476402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.404+0000 7f4770e00640 1 -- 172.21.15.86:0/368295289 shutdown_connections 2024-04-17T07:14:12.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.404+0000 7f4770e00640 1 --2- 172.21.15.86:0/368295289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476c109f90 0x7f476c110820 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.404+0000 7f4770e00640 1 --2- 172.21.15.86:0/368295289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f476c1095d0 0x7f476c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.404+0000 7f4770e00640 1 --2- 172.21.15.86:0/368295289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f476c1083e0 0x7f476c108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.404+0000 7f4770e00640 1 -- 172.21.15.86:0/368295289 >> 172.21.15.86:0/368295289 conn(0x7f476c0773a0 msgr2=0x7f476c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:12.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.404+0000 7f4770e00640 1 -- 172.21.15.86:0/368295289 shutdown_connections 2024-04-17T07:14:12.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.404+0000 7f4770e00640 1 -- 172.21.15.86:0/368295289 wait complete. 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.405+0000 7f4770e00640 1 Processor -- start 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.405+0000 7f4770e00640 1 -- start start 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f4770e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f476c1083e0 0x7f476c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f4770e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f476c1095d0 0x7f476c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f4770e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476c109f90 0x7f476c1a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f4770e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f476c113f20 con 0x7f476c109f90 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f4770e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f476c113da0 con 0x7f476c1083e0 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f4770e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1140a0 con 0x7f476c1095d0 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f476be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476c109f90 0x7f476c1a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f476be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476c109f90 0x7f476c1a5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37728/0 (socket says 172.21.15.86:37728) 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f476be00640 1 -- 172.21.15.86:0/2043770193 learned_addr learned my addr 172.21.15.86:0/2043770193 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:12.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f476aa00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f476c1095d0 0x7f476c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.406+0000 7f476b400640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f476c1083e0 0x7f476c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f476be00640 1 -- 172.21.15.86:0/2043770193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f476c1095d0 msgr2=0x7f476c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f476be00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f476c1095d0 0x7f476c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f476be00640 1 -- 172.21.15.86:0/2043770193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f476c1083e0 msgr2=0x7f476c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f476be00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f476c1083e0 0x7f476c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f476be00640 1 -- 172.21.15.86:0/2043770193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c1a61a0 con 0x7f476c109f90 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f476be00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476c109f90 0x7f476c1a5b30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f476402c700 tx=0x7f47640328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f4763e00640 1 -- 172.21.15.86:0/2043770193 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764041020 con 0x7f476c109f90 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f4763e00640 1 -- 172.21.15.86:0/2043770193 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4764003e20 con 0x7f476c109f90 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f4770e00640 1 -- 172.21.15.86:0/2043770193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c1a6430 con 0x7f476c109f90 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f4763e00640 1 -- 172.21.15.86:0/2043770193 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764039040 con 0x7f476c109f90 2024-04-17T07:14:12.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.407+0000 7f4770e00640 1 -- 172.21.15.86:0/2043770193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1a6920 con 0x7f476c109f90 2024-04-17T07:14:12.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.409+0000 7f4770e00640 1 -- 172.21.15.86:0/2043770193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476c10f780 con 0x7f476c109f90 2024-04-17T07:14:12.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.411+0000 7f4763e00640 1 -- 172.21.15.86:0/2043770193 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4764044020 con 0x7f476c109f90 2024-04-17T07:14:12.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.413+0000 7f4763e00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47500a3bb0 0x7f47500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:12.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.413+0000 7f4763e00640 1 -- 172.21.15.86:0/2043770193 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f47640f2d10 con 0x7f476c109f90 2024-04-17T07:14:12.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.413+0000 7f476b400640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47500a3bb0 0x7f47500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:12.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.414+0000 7f476b400640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47500a3bb0 0x7f47500a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f475c005fd0 tx=0x7f475c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:12.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.417+0000 7f4763e00640 1 -- 172.21.15.86:0/2043770193 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f47640f8020 con 0x7f476c109f90 2024-04-17T07:14:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:12 smithi098 ceph-mon[35418]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-17T07:14:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3977440231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3905886232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3535313688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:12 smithi103 ceph-mon[35495]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-17T07:14:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3977440231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:12.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3905886232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:12.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3535313688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:12 smithi086 ceph-mon[31733]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-17T07:14:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3977440231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3905886232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3535313688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.621+0000 7f4770e00640 1 -- 172.21.15.86:0/2043770193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f476c062b60 con 0x7f476c109f90 2024-04-17T07:14:12.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.625+0000 7f4763e00640 1 -- 172.21.15.86:0/2043770193 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7f476404b020 con 0x7f476c109f90 2024-04-17T07:14:12.626 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:12.626 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:14:12.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 -- 172.21.15.86:0/2043770193 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47500a3bb0 msgr2=0x7f47500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:12.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47500a3bb0 0x7f47500a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f475c005fd0 tx=0x7f475c005850 comp rx=0 tx=0).stop 2024-04-17T07:14:12.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 -- 172.21.15.86:0/2043770193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476c109f90 msgr2=0x7f476c1a5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:12.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476c109f90 0x7f476c1a5b30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f476402c700 tx=0x7f47640328a0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 -- 172.21.15.86:0/2043770193 shutdown_connections 2024-04-17T07:14:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f47500a3bb0 0x7f47500a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f476c109f90 0x7f476c1a5b30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f476c1095d0 0x7f476c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 --2- 172.21.15.86:0/2043770193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f476c1083e0 0x7f476c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 -- 172.21.15.86:0/2043770193 >> 172.21.15.86:0/2043770193 conn(0x7f476c0773a0 msgr2=0x7f476c10c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.628+0000 7f4770e00640 1 -- 172.21.15.86:0/2043770193 shutdown_connections 2024-04-17T07:14:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.629+0000 7f4770e00640 1 -- 172.21.15.86:0/2043770193 wait complete. 2024-04-17T07:14:12.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:14:12.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:14:12.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.867+0000 7fa186400640 1 -- 172.21.15.86:0/852140367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1801085c0 msgr2=0x7fa1801089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:12.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.867+0000 7fa186400640 1 --2- 172.21.15.86:0/852140367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1801085c0 0x7fa1801089e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa170009900 tx=0x7fa17002bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:12.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.867+0000 7fa186400640 1 -- 172.21.15.86:0/852140367 shutdown_connections 2024-04-17T07:14:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.867+0000 7fa186400640 1 --2- 172.21.15.86:0/852140367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa18010a170 0x7fa180110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.867+0000 7fa186400640 1 --2- 172.21.15.86:0/852140367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1801097b0 0x7fa180109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.867+0000 7fa186400640 1 --2- 172.21.15.86:0/852140367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1801085c0 0x7fa1801089e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.867+0000 7fa186400640 1 -- 172.21.15.86:0/852140367 >> 172.21.15.86:0/852140367 conn(0x7fa180077560 msgr2=0x7fa1800799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.867+0000 7fa186400640 1 -- 172.21.15.86:0/852140367 shutdown_connections 2024-04-17T07:14:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.867+0000 7fa186400640 1 -- 172.21.15.86:0/852140367 wait complete. 2024-04-17T07:14:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.868+0000 7fa186400640 1 Processor -- start 2024-04-17T07:14:12.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.869+0000 7fa186400640 1 -- start start 2024-04-17T07:14:12.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.869+0000 7fa186400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1801085c0 0x7fa18019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.869+0000 7fa186400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1801097b0 0x7fa18019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.869+0000 7fa186400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa18010a170 0x7fa1801a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.869+0000 7fa186400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa180114140 con 0x7fa1801097b0 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.869+0000 7fa186400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa180113fc0 con 0x7fa1801085c0 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.869+0000 7fa186400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1801142c0 con 0x7fa18010a170 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.869+0000 7fa17ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1801097b0 0x7fa18019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.869+0000 7fa17ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1801097b0 0x7fa18019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37748/0 (socket says 172.21.15.86:37748) 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.869+0000 7fa17ea00640 1 -- 172.21.15.86:0/109990293 learned_addr learned my addr 172.21.15.86:0/109990293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.870+0000 7fa17f400640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1801085c0 0x7fa18019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.870+0000 7fa17fe00640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa18010a170 0x7fa1801a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.870+0000 7fa17ea00640 1 -- 172.21.15.86:0/109990293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa18010a170 msgr2=0x7fa1801a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.870+0000 7fa17ea00640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa18010a170 0x7fa1801a5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.870+0000 7fa17ea00640 1 -- 172.21.15.86:0/109990293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1801085c0 msgr2=0x7fa18019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:12.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.870+0000 7fa17ea00640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1801085c0 0x7fa18019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.870+0000 7fa17ea00640 1 -- 172.21.15.86:0/109990293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1801a6460 con 0x7fa1801097b0 2024-04-17T07:14:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.870+0000 7fa17ea00640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1801097b0 0x7fa18019ed80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00d770 tx=0x7fa16c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.871+0000 7fa177e00640 1 -- 172.21.15.86:0/109990293 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c017070 con 0x7fa1801097b0 2024-04-17T07:14:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.871+0000 7fa177e00640 1 -- 172.21.15.86:0/109990293 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa16c0054d0 con 0x7fa1801097b0 2024-04-17T07:14:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.871+0000 7fa186400640 1 -- 172.21.15.86:0/109990293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1801a6750 con 0x7fa1801097b0 2024-04-17T07:14:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.871+0000 7fa177e00640 1 -- 172.21.15.86:0/109990293 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c004cf0 con 0x7fa1801097b0 2024-04-17T07:14:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.871+0000 7fa186400640 1 -- 172.21.15.86:0/109990293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1801a6c50 con 0x7fa1801097b0 2024-04-17T07:14:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.872+0000 7fa186400640 1 -- 172.21.15.86:0/109990293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18010f950 con 0x7fa1801097b0 2024-04-17T07:14:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.875+0000 7fa177e00640 1 -- 172.21.15.86:0/109990293 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa16c004e90 con 0x7fa1801097b0 2024-04-17T07:14:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.876+0000 7fa177e00640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa1640a3bb0 0x7fa1640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.876+0000 7fa177e00640 1 -- 172.21.15.86:0/109990293 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa16c0d02a0 con 0x7fa1801097b0 2024-04-17T07:14:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.876+0000 7fa17f400640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa1640a3bb0 0x7fa1640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:12.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.877+0000 7fa17f400640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa1640a3bb0 0x7fa1640a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa17002c240 tx=0x7fa170005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:12.881+0000 7fa177e00640 1 -- 172.21.15.86:0/109990293 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa16c09c900 con 0x7fa1801097b0 2024-04-17T07:14:13.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.087+0000 7fa186400640 1 -- 172.21.15.86:0/109990293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa180062b60 con 0x7fa1801097b0 2024-04-17T07:14:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.088+0000 7fa177e00640 1 -- 172.21.15.86:0/109990293 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fa16c09c200 con 0x7fa1801097b0 2024-04-17T07:14:13.089 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:14:13.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.091+0000 7fa186400640 1 -- 172.21.15.86:0/109990293 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa1640a3bb0 msgr2=0x7fa1640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.091+0000 7fa186400640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa1640a3bb0 0x7fa1640a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa17002c240 tx=0x7fa170005fd0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.091+0000 7fa186400640 1 -- 172.21.15.86:0/109990293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1801097b0 msgr2=0x7fa18019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.091+0000 7fa186400640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1801097b0 0x7fa18019ed80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00d770 tx=0x7fa16c00dc30 comp rx=0 tx=0).stop 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.091+0000 7fa186400640 1 -- 172.21.15.86:0/109990293 shutdown_connections 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.091+0000 7fa186400640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa1640a3bb0 0x7fa1640a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.091+0000 7fa186400640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa18010a170 0x7fa1801a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.091+0000 7fa186400640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1801097b0 0x7fa18019ed80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.091+0000 7fa186400640 1 --2- 172.21.15.86:0/109990293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1801085c0 0x7fa18019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.091+0000 7fa186400640 1 -- 172.21.15.86:0/109990293 >> 172.21.15.86:0/109990293 conn(0x7fa180077560 msgr2=0x7fa18010cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.092+0000 7fa186400640 1 -- 172.21.15.86:0/109990293 shutdown_connections 2024-04-17T07:14:13.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.092+0000 7fa186400640 1 -- 172.21.15.86:0/109990293 wait complete. 2024-04-17T07:14:13.111 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:14:13.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.335+0000 7fb623200640 1 -- 172.21.15.86:0/864355248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624102930 msgr2=0x7fb6241091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:13.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.335+0000 7fb623200640 1 --2- 172.21.15.86:0/864355248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624102930 0x7fb6241091c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009c90 tx=0x7fb61c02c0b0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.335+0000 7fb623200640 1 -- 172.21.15.86:0/864355248 shutdown_connections 2024-04-17T07:14:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.335+0000 7fb623200640 1 --2- 172.21.15.86:0/864355248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624102930 0x7fb6241091c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.335+0000 7fb623200640 1 --2- 172.21.15.86:0/864355248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb624101f70 0x7fb6241023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.335+0000 7fb623200640 1 --2- 172.21.15.86:0/864355248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb624100d80 0x7fb6241011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.335+0000 7fb623200640 1 -- 172.21.15.86:0/864355248 >> 172.21.15.86:0/864355248 conn(0x7fb6240fc310 msgr2=0x7fb6240fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.335+0000 7fb623200640 1 -- 172.21.15.86:0/864355248 shutdown_connections 2024-04-17T07:14:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.335+0000 7fb623200640 1 -- 172.21.15.86:0/864355248 wait complete. 2024-04-17T07:14:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.336+0000 7fb623200640 1 Processor -- start 2024-04-17T07:14:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.336+0000 7fb623200640 1 -- start start 2024-04-17T07:14:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.337+0000 7fb623200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb624100d80 0x7fb62419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.337+0000 7fb623200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624101f70 0x7fb62419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.337+0000 7fb623200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb624102930 0x7fb6241a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.337+0000 7fb623200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb62410d990 con 0x7fb624101f70 2024-04-17T07:14:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.337+0000 7fb623200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb62410d810 con 0x7fb624100d80 2024-04-17T07:14:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.337+0000 7fb623200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb62410db10 con 0x7fb624102930 2024-04-17T07:14:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.337+0000 7fb621400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624101f70 0x7fb62419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:13.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.337+0000 7fb621400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624101f70 0x7fb62419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37764/0 (socket says 172.21.15.86:37764) 2024-04-17T07:14:13.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.337+0000 7fb621400640 1 -- 172.21.15.86:0/691335153 learned_addr learned my addr 172.21.15.86:0/691335153 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:13.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.337+0000 7fb621e00640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb624100d80 0x7fb62419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb622800640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb624102930 0x7fb6241a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb621400640 1 -- 172.21.15.86:0/691335153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb624102930 msgr2=0x7fb6241a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb621400640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb624102930 0x7fb6241a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb621400640 1 -- 172.21.15.86:0/691335153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb624100d80 msgr2=0x7fb62419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb621400640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb624100d80 0x7fb62419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb621400640 1 -- 172.21.15.86:0/691335153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6241a1f20 con 0x7fb624101f70 2024-04-17T07:14:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb622800640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb624102930 0x7fb6241a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb621400640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624101f70 0x7fb62419a840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00e9f0 tx=0x7fb60c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:13.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb61aa00640 1 -- 172.21.15.86:0/691335153 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c00cd80 con 0x7fb624101f70 2024-04-17T07:14:13.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb61aa00640 1 -- 172.21.15.86:0/691335153 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb60c0108e0 con 0x7fb624101f70 2024-04-17T07:14:13.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb61aa00640 1 -- 172.21.15.86:0/691335153 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c018920 con 0x7fb624101f70 2024-04-17T07:14:13.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb623200640 1 -- 172.21.15.86:0/691335153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6241a2210 con 0x7fb624101f70 2024-04-17T07:14:13.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.338+0000 7fb623200640 1 -- 172.21.15.86:0/691335153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6241a27b0 con 0x7fb624101f70 2024-04-17T07:14:13.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.340+0000 7fb623200640 1 -- 172.21.15.86:0/691335153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6241a2a10 con 0x7fb624101f70 2024-04-17T07:14:13.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.342+0000 7fb61aa00640 1 -- 172.21.15.86:0/691335153 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb60c010a90 con 0x7fb624101f70 2024-04-17T07:14:13.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.343+0000 7fb61aa00640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6080a3c30 0x7fb6080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:13.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.343+0000 7fb61aa00640 1 -- 172.21.15.86:0/691335153 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb60c0d0ba0 con 0x7fb624101f70 2024-04-17T07:14:13.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.343+0000 7fb621e00640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6080a3c30 0x7fb6080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:13.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.345+0000 7fb621e00640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6080a3c30 0x7fb6080a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb614005fd0 tx=0x7fb614005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:13.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.347+0000 7fb61aa00640 1 -- 172.21.15.86:0/691335153 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb60c09d1a0 con 0x7fb624101f70 2024-04-17T07:14:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:13 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2043770193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:13 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/109990293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:13.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:13 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2043770193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:13.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:13 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/109990293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:13.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.548+0000 7fb623200640 1 -- 172.21.15.86:0/691335153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb6241a2be0 con 0x7fb624101f70 2024-04-17T07:14:13.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.549+0000 7fb61aa00640 1 -- 172.21.15.86:0/691335153 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fb60c09caa0 con 0x7fb624101f70 2024-04-17T07:14:13.549 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.551+0000 7fb623200640 1 -- 172.21.15.86:0/691335153 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6080a3c30 msgr2=0x7fb6080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.551+0000 7fb623200640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6080a3c30 0x7fb6080a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb614005fd0 tx=0x7fb614005850 comp rx=0 tx=0).stop 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.552+0000 7fb623200640 1 -- 172.21.15.86:0/691335153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624101f70 msgr2=0x7fb62419a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.552+0000 7fb623200640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624101f70 0x7fb62419a840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00e9f0 tx=0x7fb60c00eeb0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.552+0000 7fb623200640 1 -- 172.21.15.86:0/691335153 shutdown_connections 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.552+0000 7fb623200640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6080a3c30 0x7fb6080a60f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.552+0000 7fb623200640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb624102930 0x7fb6241a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.552+0000 7fb623200640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624101f70 0x7fb62419a840 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.552+0000 7fb623200640 1 --2- 172.21.15.86:0/691335153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb624100d80 0x7fb62419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.552+0000 7fb623200640 1 -- 172.21.15.86:0/691335153 >> 172.21.15.86:0/691335153 conn(0x7fb6240fc310 msgr2=0x7fb624109b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:13.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.552+0000 7fb623200640 1 -- 172.21.15.86:0/691335153 shutdown_connections 2024-04-17T07:14:13.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.552+0000 7fb623200640 1 -- 172.21.15.86:0/691335153 wait complete. 2024-04-17T07:14:13.569 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:13 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2043770193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:13.569 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:13 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/109990293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:13.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:14:13.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.792+0000 7fcb1f200640 1 -- 172.21.15.86:0/1614328146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102800 msgr2=0x7fcb20104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:13.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.792+0000 7fcb1f200640 1 --2- 172.21.15.86:0/1614328146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102800 0x7fcb20104bf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009900 tx=0x7fcb1802bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:13.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.792+0000 7fcb1f200640 1 -- 172.21.15.86:0/1614328146 shutdown_connections 2024-04-17T07:14:13.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.792+0000 7fcb1f200640 1 --2- 172.21.15.86:0/1614328146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20107a60 0x7fcb20109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.792+0000 7fcb1f200640 1 --2- 172.21.15.86:0/1614328146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20105130 0x7fcb20107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.792+0000 7fcb1f200640 1 --2- 172.21.15.86:0/1614328146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102800 0x7fcb20104bf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.792+0000 7fcb1f200640 1 -- 172.21.15.86:0/1614328146 >> 172.21.15.86:0/1614328146 conn(0x7fcb200fc230 msgr2=0x7fcb200fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:13.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.792+0000 7fcb1f200640 1 -- 172.21.15.86:0/1614328146 shutdown_connections 2024-04-17T07:14:13.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.792+0000 7fcb1f200640 1 -- 172.21.15.86:0/1614328146 wait complete. 2024-04-17T07:14:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.793+0000 7fcb1f200640 1 Processor -- start 2024-04-17T07:14:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.793+0000 7fcb1f200640 1 -- start start 2024-04-17T07:14:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.793+0000 7fcb1f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102800 0x7fcb2019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.793+0000 7fcb1f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20105130 0x7fcb2019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.793+0000 7fcb1f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20107a60 0x7fcb201a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.793+0000 7fcb1f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2010d930 con 0x7fcb20102800 2024-04-17T07:14:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.793+0000 7fcb1f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2010d7b0 con 0x7fcb20107a60 2024-04-17T07:14:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.793+0000 7fcb1f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2010dab0 con 0x7fcb20105130 2024-04-17T07:14:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.794+0000 7fcb1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102800 0x7fcb2019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.794+0000 7fcb1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102800 0x7fcb2019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37784/0 (socket says 172.21.15.86:37784) 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.794+0000 7fcb1de00640 1 -- 172.21.15.86:0/3499583199 learned_addr learned my addr 172.21.15.86:0/3499583199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.794+0000 7fcb1de00640 1 -- 172.21.15.86:0/3499583199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20105130 msgr2=0x7fcb2019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.794+0000 7fcb1de00640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20105130 0x7fcb2019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.794+0000 7fcb1e800640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20107a60 0x7fcb201a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.794+0000 7fcb1de00640 1 -- 172.21.15.86:0/3499583199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20107a60 msgr2=0x7fcb201a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.794+0000 7fcb1de00640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20107a60 0x7fcb201a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.794+0000 7fcb1de00640 1 -- 172.21.15.86:0/3499583199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb201a1d80 con 0x7fcb20102800 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.795+0000 7fcb1de00640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102800 0x7fcb2019a160 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcb18000c00 tx=0x7fcb180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.795+0000 7fcb16a00640 1 -- 172.21.15.86:0/3499583199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18043020 con 0x7fcb20102800 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.795+0000 7fcb16a00640 1 -- 172.21.15.86:0/3499583199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb180054c0 con 0x7fcb20102800 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.795+0000 7fcb1f200640 1 -- 172.21.15.86:0/3499583199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb201a2010 con 0x7fcb20102800 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.795+0000 7fcb16a00640 1 -- 172.21.15.86:0/3499583199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18003d40 con 0x7fcb20102800 2024-04-17T07:14:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.795+0000 7fcb1f200640 1 -- 172.21.15.86:0/3499583199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb201a25b0 con 0x7fcb20102800 2024-04-17T07:14:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.796+0000 7fcb1f200640 1 -- 172.21.15.86:0/3499583199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb201095b0 con 0x7fcb20102800 2024-04-17T07:14:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.798+0000 7fcb16a00640 1 -- 172.21.15.86:0/3499583199 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcb18046020 con 0x7fcb20102800 2024-04-17T07:14:13.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.799+0000 7fcb16a00640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3c30 0x7fcb040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:13.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.799+0000 7fcb16a00640 1 -- 172.21.15.86:0/3499583199 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcb180f2c80 con 0x7fcb20102800 2024-04-17T07:14:13.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.800+0000 7fcb1d400640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3c30 0x7fcb040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.800+0000 7fcb1d400640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3c30 0x7fcb040a60f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb2019b670 tx=0x7fcb08009710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:13.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:13.804+0000 7fcb16a00640 1 -- 172.21.15.86:0/3499583199 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcb1804d020 con 0x7fcb20102800 2024-04-17T07:14:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.006+0000 7fcb1f200640 1 -- 172.21.15.86:0/3499583199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb20062b60 con 0x7fcb20102800 2024-04-17T07:14:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.007+0000 7fcb16a00640 1 -- 172.21.15.86:0/3499583199 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fcb180bf400 con 0x7fcb20102800 2024-04-17T07:14:14.007 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:14:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 -- 172.21.15.86:0/3499583199 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3c30 msgr2=0x7fcb040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3c30 0x7fcb040a60f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb2019b670 tx=0x7fcb08009710 comp rx=0 tx=0).stop 2024-04-17T07:14:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 -- 172.21.15.86:0/3499583199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102800 msgr2=0x7fcb2019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102800 0x7fcb2019a160 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcb18000c00 tx=0x7fcb180042f0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 -- 172.21.15.86:0/3499583199 shutdown_connections 2024-04-17T07:14:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3c30 0x7fcb040a60f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20107a60 0x7fcb201a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20105130 0x7fcb2019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 --2- 172.21.15.86:0/3499583199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102800 0x7fcb2019a160 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 -- 172.21.15.86:0/3499583199 >> 172.21.15.86:0/3499583199 conn(0x7fcb200fc230 msgr2=0x7fcb20101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 -- 172.21.15.86:0/3499583199 shutdown_connections 2024-04-17T07:14:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.010+0000 7fcb1f200640 1 -- 172.21.15.86:0/3499583199 wait complete. 2024-04-17T07:14:14.030 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:14:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.238+0000 7fe816800640 1 -- 172.21.15.86:0/879565615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810077b60 msgr2=0x7fe810077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.238+0000 7fe816800640 1 --2- 172.21.15.86:0/879565615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810077b60 0x7fe810077fc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009900 tx=0x7fe7fc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.238+0000 7fe816800640 1 -- 172.21.15.86:0/879565615 shutdown_connections 2024-04-17T07:14:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.238+0000 7fe816800640 1 --2- 172.21.15.86:0/879565615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810078500 0x7fe810078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.238+0000 7fe816800640 1 --2- 172.21.15.86:0/879565615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810077b60 0x7fe810077fc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.238+0000 7fe816800640 1 --2- 172.21.15.86:0/879565615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe810076920 0x7fe810076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.238+0000 7fe816800640 1 -- 172.21.15.86:0/879565615 >> 172.21.15.86:0/879565615 conn(0x7fe8100ff610 msgr2=0x7fe810101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.238+0000 7fe816800640 1 -- 172.21.15.86:0/879565615 shutdown_connections 2024-04-17T07:14:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.238+0000 7fe816800640 1 -- 172.21.15.86:0/879565615 wait complete. 2024-04-17T07:14:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.239+0000 7fe816800640 1 Processor -- start 2024-04-17T07:14:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.239+0000 7fe816800640 1 -- start start 2024-04-17T07:14:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe816800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810076920 0x7fe81019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe816800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810077b60 0x7fe81019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe816800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe810078500 0x7fe8101a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:14.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe816800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe810113fe0 con 0x7fe810076920 2024-04-17T07:14:14.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe816800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe810113e60 con 0x7fe810078500 2024-04-17T07:14:14.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe816800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe810114160 con 0x7fe810077b60 2024-04-17T07:14:14.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe815400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810076920 0x7fe81019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:14.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe815400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810076920 0x7fe81019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37794/0 (socket says 172.21.15.86:37794) 2024-04-17T07:14:14.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe815400640 1 -- 172.21.15.86:0/719993385 learned_addr learned my addr 172.21.15.86:0/719993385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:14.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe815400640 1 -- 172.21.15.86:0/719993385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810077b60 msgr2=0x7fe81019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe815400640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810077b60 0x7fe81019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe815400640 1 -- 172.21.15.86:0/719993385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe810078500 msgr2=0x7fe8101a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe815400640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe810078500 0x7fe8101a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.240+0000 7fe815400640 1 -- 172.21.15.86:0/719993385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8101a6260 con 0x7fe810076920 2024-04-17T07:14:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.241+0000 7fe815400640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810076920 0x7fe81019e640 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe80400d950 tx=0x7fe80400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.241+0000 7fe80e000640 1 -- 172.21.15.86:0/719993385 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804017070 con 0x7fe810076920 2024-04-17T07:14:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.241+0000 7fe80e000640 1 -- 172.21.15.86:0/719993385 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe804009c30 con 0x7fe810076920 2024-04-17T07:14:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.241+0000 7fe816800640 1 -- 172.21.15.86:0/719993385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8101a6550 con 0x7fe810076920 2024-04-17T07:14:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.241+0000 7fe80e000640 1 -- 172.21.15.86:0/719993385 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80400f620 con 0x7fe810076920 2024-04-17T07:14:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.241+0000 7fe816800640 1 -- 172.21.15.86:0/719993385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8101a6aa0 con 0x7fe810076920 2024-04-17T07:14:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.242+0000 7fe816800640 1 -- 172.21.15.86:0/719993385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe810110a10 con 0x7fe810076920 2024-04-17T07:14:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.244+0000 7fe80e000640 1 -- 172.21.15.86:0/719993385 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe8040109a0 con 0x7fe810076920 2024-04-17T07:14:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.245+0000 7fe80e000640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7f80a3bb0 0x7fe7f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.245+0000 7fe80e000640 1 -- 172.21.15.86:0/719993385 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe8040d04c0 con 0x7fe810076920 2024-04-17T07:14:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.245+0000 7fe814a00640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7f80a3bb0 0x7fe7f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.246+0000 7fe814a00640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7f80a3bb0 0x7fe7f80a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc036040 tx=0x7fe7fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.249+0000 7fe80e000640 1 -- 172.21.15.86:0/719993385 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe804009de0 con 0x7fe810076920 2024-04-17T07:14:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:14 smithi086 ceph-mon[31733]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-17T07:14:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:14 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/691335153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:14.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:14 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3499583199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:14.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.450+0000 7fe816800640 1 -- 172.21.15.86:0/719993385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe810062b60 con 0x7fe810076920 2024-04-17T07:14:14.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.451+0000 7fe80e000640 1 -- 172.21.15.86:0/719993385 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fe80409cac0 con 0x7fe810076920 2024-04-17T07:14:14.451 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:14:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.453+0000 7fe816800640 1 -- 172.21.15.86:0/719993385 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7f80a3bb0 msgr2=0x7fe7f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.453+0000 7fe816800640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7f80a3bb0 0x7fe7f80a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc036040 tx=0x7fe7fc000bc0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.453+0000 7fe816800640 1 -- 172.21.15.86:0/719993385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810076920 msgr2=0x7fe81019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.453+0000 7fe816800640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810076920 0x7fe81019e640 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe80400d950 tx=0x7fe80400de10 comp rx=0 tx=0).stop 2024-04-17T07:14:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.453+0000 7fe816800640 1 -- 172.21.15.86:0/719993385 shutdown_connections 2024-04-17T07:14:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.453+0000 7fe816800640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe7f80a3bb0 0x7fe7f80a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.453+0000 7fe816800640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe810078500 0x7fe8101a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.453+0000 7fe816800640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe810077b60 0x7fe81019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.453+0000 7fe816800640 1 --2- 172.21.15.86:0/719993385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe810076920 0x7fe81019e640 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.453+0000 7fe816800640 1 -- 172.21.15.86:0/719993385 >> 172.21.15.86:0/719993385 conn(0x7fe8100ff610 msgr2=0x7fe810101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.454+0000 7fe816800640 1 -- 172.21.15.86:0/719993385 shutdown_connections 2024-04-17T07:14:14.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.454+0000 7fe816800640 1 -- 172.21.15.86:0/719993385 wait complete. 2024-04-17T07:14:14.472 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:14:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:14 smithi098 ceph-mon[35418]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-17T07:14:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:14 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/691335153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:14.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:14 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3499583199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:14.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:14 smithi103 ceph-mon[35495]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-17T07:14:14.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:14 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/691335153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:14.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:14 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3499583199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.689+0000 7f92e7e00640 1 -- 172.21.15.86:0/367169802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8103fe0 msgr2=0x7f92e8104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.689+0000 7f92e7e00640 1 --2- 172.21.15.86:0/367169802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8103fe0 0x7f92e8104400 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009900 tx=0x7f92d802bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.689+0000 7f92e7e00640 1 -- 172.21.15.86:0/367169802 shutdown_connections 2024-04-17T07:14:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.689+0000 7f92e7e00640 1 --2- 172.21.15.86:0/367169802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e8105b90 0x7f92e810c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.689+0000 7f92e7e00640 1 --2- 172.21.15.86:0/367169802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92e81051d0 0x7f92e8105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.689+0000 7f92e7e00640 1 --2- 172.21.15.86:0/367169802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8103fe0 0x7f92e8104400 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.689+0000 7f92e7e00640 1 -- 172.21.15.86:0/367169802 >> 172.21.15.86:0/367169802 conn(0x7f92e80ff570 msgr2=0x7f92e8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.690+0000 7f92e7e00640 1 -- 172.21.15.86:0/367169802 shutdown_connections 2024-04-17T07:14:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.690+0000 7f92e7e00640 1 -- 172.21.15.86:0/367169802 wait complete. 2024-04-17T07:14:14.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7e00640 1 Processor -- start 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7e00640 1 -- start start 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92e8103fe0 0x7f92e819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e81051d0 0x7f92e819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8105b90 0x7f92e81a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92e810fae0 con 0x7f92e8105b90 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92e810f960 con 0x7f92e81051d0 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92e810fc60 con 0x7f92e8103fe0 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8105b90 0x7f92e81a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8105b90 0x7f92e81a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37812/0 (socket says 172.21.15.86:37812) 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.691+0000 7f92e7400640 1 -- 172.21.15.86:0/3747446578 learned_addr learned my addr 172.21.15.86:0/3747446578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92e6a00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92e8103fe0 0x7f92e819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:14.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92dfe00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e81051d0 0x7f92e819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92e7400640 1 -- 172.21.15.86:0/3747446578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92e8103fe0 msgr2=0x7f92e819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92e7400640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92e8103fe0 0x7f92e819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92e7400640 1 -- 172.21.15.86:0/3747446578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e81051d0 msgr2=0x7f92e819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92e7400640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e81051d0 0x7f92e819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92e7400640 1 -- 172.21.15.86:0/3747446578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e81a1de0 con 0x7f92e8105b90 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92dfe00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e81051d0 0x7f92e819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92e7400640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8105b90 0x7f92e81a1770 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f92e000cce0 tx=0x7f92e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92df400640 1 -- 172.21.15.86:0/3747446578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e000f450 con 0x7f92e8105b90 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92df400640 1 -- 172.21.15.86:0/3747446578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92e000faa0 con 0x7f92e8105b90 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92e7e00640 1 -- 172.21.15.86:0/3747446578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e81a20d0 con 0x7f92e8105b90 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92df400640 1 -- 172.21.15.86:0/3747446578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e0018b70 con 0x7f92e8105b90 2024-04-17T07:14:14.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.692+0000 7f92e7e00640 1 -- 172.21.15.86:0/3747446578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e81a25d0 con 0x7f92e8105b90 2024-04-17T07:14:14.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.693+0000 7f92e7e00640 1 -- 172.21.15.86:0/3747446578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e810b380 con 0x7f92e8105b90 2024-04-17T07:14:14.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.696+0000 7f92df400640 1 -- 172.21.15.86:0/3747446578 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f92e0018d10 con 0x7f92e8105b90 2024-04-17T07:14:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.697+0000 7f92df400640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92cc0a3bb0 0x7f92cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.697+0000 7f92df400640 1 -- 172.21.15.86:0/3747446578 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f92e00d0cd0 con 0x7f92e8105b90 2024-04-17T07:14:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.697+0000 7f92e6a00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92cc0a3bb0 0x7f92cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:14.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.698+0000 7f92e6a00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92cc0a3bb0 0x7f92cc0a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f92d8006010 tx=0x7f92d802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.701+0000 7f92df400640 1 -- 172.21.15.86:0/3747446578 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f92e0014030 con 0x7f92e8105b90 2024-04-17T07:14:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.944+0000 7f92e7e00640 1 -- 172.21.15.86:0/3747446578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92e8062b60 con 0x7f92e8105b90 2024-04-17T07:14:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.945+0000 7f92df400640 1 -- 172.21.15.86:0/3747446578 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f92e009d2d0 con 0x7f92e8105b90 2024-04-17T07:14:14.946 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:14.946 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:12:31.203694+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[34460,34490,34430,34400,24505]},"id":1}]} 2024-04-17T07:14:14.946 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-04-17T07:14:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 -- 172.21.15.86:0/3747446578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92cc0a3bb0 msgr2=0x7f92cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92cc0a3bb0 0x7f92cc0a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f92d8006010 tx=0x7f92d802c800 comp rx=0 tx=0).stop 2024-04-17T07:14:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 -- 172.21.15.86:0/3747446578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8105b90 msgr2=0x7f92e81a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8105b90 0x7f92e81a1770 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f92e000cce0 tx=0x7f92e000a620 comp rx=0 tx=0).stop 2024-04-17T07:14:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 -- 172.21.15.86:0/3747446578 shutdown_connections 2024-04-17T07:14:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92cc0a3bb0 0x7f92cc0a6070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e8105b90 0x7f92e81a1770 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92e81051d0 0x7f92e819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 --2- 172.21.15.86:0/3747446578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92e8103fe0 0x7f92e819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 -- 172.21.15.86:0/3747446578 >> 172.21.15.86:0/3747446578 conn(0x7f92e80ff570 msgr2=0x7f92e8108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 -- 172.21.15.86:0/3747446578 shutdown_connections 2024-04-17T07:14:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:14.948+0000 7f92e7e00640 1 -- 172.21.15.86:0/3747446578 wait complete. 2024-04-17T07:14:14.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.174+0000 7f0ebe000640 1 -- 172.21.15.86:0/3389208968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8103980 msgr2=0x7f0eb810a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.174+0000 7f0ebe000640 1 --2- 172.21.15.86:0/3389208968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8103980 0x7f0eb810a210 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009ca0 tx=0x7f0eb002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.174+0000 7f0ebe000640 1 -- 172.21.15.86:0/3389208968 shutdown_connections 2024-04-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.174+0000 7f0ebe000640 1 --2- 172.21.15.86:0/3389208968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8103980 0x7f0eb810a210 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.174+0000 7f0ebe000640 1 --2- 172.21.15.86:0/3389208968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eb8102fc0 0x7f0eb8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.174+0000 7f0ebe000640 1 --2- 172.21.15.86:0/3389208968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8101dd0 0x7f0eb81021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.174+0000 7f0ebe000640 1 -- 172.21.15.86:0/3389208968 >> 172.21.15.86:0/3389208968 conn(0x7f0eb80fd340 msgr2=0x7f0eb80ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.174+0000 7f0ebe000640 1 -- 172.21.15.86:0/3389208968 shutdown_connections 2024-04-17T07:14:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.174+0000 7f0ebe000640 1 -- 172.21.15.86:0/3389208968 wait complete. 2024-04-17T07:14:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.175+0000 7f0ebe000640 1 Processor -- start 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.175+0000 7f0ebe000640 1 -- start start 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.175+0000 7f0ebe000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8101dd0 0x7f0eb8075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.175+0000 7f0ebe000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eb8102fc0 0x7f0eb8074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.175+0000 7f0ebe000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8103980 0x7f0eb8074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.175+0000 7f0ebe000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810d900 con 0x7f0eb8101dd0 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.175+0000 7f0ebe000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810d780 con 0x7f0eb8102fc0 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.175+0000 7f0ebe000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810da80 con 0x7f0eb8103980 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0ebcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8101dd0 0x7f0eb8075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0ebcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8101dd0 0x7f0eb8075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37832/0 (socket says 172.21.15.86:37832) 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0ebcc00640 1 -- 172.21.15.86:0/4214863600 learned_addr learned my addr 172.21.15.86:0/4214863600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eb8102fc0 0x7f0eb8074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0ebcc00640 1 -- 172.21.15.86:0/4214863600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8103980 msgr2=0x7f0eb8074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0ebcc00640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8103980 0x7f0eb8074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0ebcc00640 1 -- 172.21.15.86:0/4214863600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eb8102fc0 msgr2=0x7f0eb8074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0ebcc00640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eb8102fc0 0x7f0eb8074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0ebcc00640 1 -- 172.21.15.86:0/4214863600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb8075120 con 0x7f0eb8101dd0 2024-04-17T07:14:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eb8102fc0 0x7f0eb8074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.176+0000 7f0ebcc00640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8101dd0 0x7f0eb8075d10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00d950 tx=0x7f0eac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.177+0000 7f0eb5600640 1 -- 172.21.15.86:0/4214863600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac017070 con 0x7f0eb8101dd0 2024-04-17T07:14:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.177+0000 7f0ebe000640 1 -- 172.21.15.86:0/4214863600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb81a20f0 con 0x7f0eb8101dd0 2024-04-17T07:14:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.177+0000 7f0eb5600640 1 -- 172.21.15.86:0/4214863600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eac009bb0 con 0x7f0eb8101dd0 2024-04-17T07:14:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.177+0000 7f0eb5600640 1 -- 172.21.15.86:0/4214863600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00f620 con 0x7f0eb8101dd0 2024-04-17T07:14:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.177+0000 7f0ebe000640 1 -- 172.21.15.86:0/4214863600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb81a25f0 con 0x7f0eb8101dd0 2024-04-17T07:14:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.178+0000 7f0ebe000640 1 -- 172.21.15.86:0/4214863600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb8109170 con 0x7f0eb8101dd0 2024-04-17T07:14:15.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.180+0000 7f0eb5600640 1 -- 172.21.15.86:0/4214863600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0eac01a070 con 0x7f0eb8101dd0 2024-04-17T07:14:15.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.181+0000 7f0eb5600640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea00a3c00 0x7f0ea00a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:15.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.182+0000 7f0eb5600640 1 -- 172.21.15.86:0/4214863600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0eac0d0200 con 0x7f0eb8101dd0 2024-04-17T07:14:15.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.182+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea00a3c00 0x7f0ea00a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:15.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.182+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea00a3c00 0x7f0ea00a60c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8075a80 tx=0x7f0ea8005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:15.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.184+0000 7f0eb5600640 1 -- 172.21.15.86:0/4214863600 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0eac09c860 con 0x7f0eb8101dd0 2024-04-17T07:14:15.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.391+0000 7f0ebe000640 1 -- 172.21.15.86:0/4214863600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0eb802d830 con 0x7f0eb8101dd0 2024-04-17T07:14:15.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.395+0000 7f0eb5600640 1 -- 172.21.15.86:0/4214863600 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7f0eac09c160 con 0x7f0eb8101dd0 2024-04-17T07:14:15.395 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:15.396 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:14:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 -- 172.21.15.86:0/4214863600 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea00a3c00 msgr2=0x7f0ea00a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea00a3c00 0x7f0ea00a60c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8075a80 tx=0x7f0ea8005d90 comp rx=0 tx=0).stop 2024-04-17T07:14:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 -- 172.21.15.86:0/4214863600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8101dd0 msgr2=0x7f0eb8075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8101dd0 0x7f0eb8075d10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00d950 tx=0x7f0eac00de10 comp rx=0 tx=0).stop 2024-04-17T07:14:15.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 -- 172.21.15.86:0/4214863600 shutdown_connections 2024-04-17T07:14:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea00a3c00 0x7f0ea00a60c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8103980 0x7f0eb8074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eb8102fc0 0x7f0eb8074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 --2- 172.21.15.86:0/4214863600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8101dd0 0x7f0eb8075d10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 -- 172.21.15.86:0/4214863600 >> 172.21.15.86:0/4214863600 conn(0x7f0eb80fd340 msgr2=0x7f0eb8106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 -- 172.21.15.86:0/4214863600 shutdown_connections 2024-04-17T07:14:15.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.398+0000 7f0ebe000640 1 -- 172.21.15.86:0/4214863600 wait complete. 2024-04-17T07:14:15.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:14:15.498 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:15 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/719993385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:15.498 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:15 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3747446578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:15.499 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:15 smithi086 ceph-mon[31733]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:15.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:15 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/719993385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:15 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3747446578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:15 smithi098 ceph-mon[35418]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:15 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/719993385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:15.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:15 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3747446578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:15.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:15 smithi103 ceph-mon[35495]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.638+0000 7f238da00640 1 -- 172.21.15.86:0/4216150359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388109c90 msgr2=0x7f238810c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.638+0000 7f238da00640 1 --2- 172.21.15.86:0/4216150359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388109c90 0x7f238810c120 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2380009ca0 tx=0x7f238002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.638+0000 7f238da00640 1 -- 172.21.15.86:0/4216150359 shutdown_connections 2024-04-17T07:14:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.638+0000 7f238da00640 1 --2- 172.21.15.86:0/4216150359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388109c90 0x7f238810c120 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.639+0000 7f238da00640 1 --2- 172.21.15.86:0/4216150359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23880749c0 0x7f2388074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.639+0000 7f238da00640 1 --2- 172.21.15.86:0/4216150359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2388075fa0 0x7f23880743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.639+0000 7f238da00640 1 -- 172.21.15.86:0/4216150359 >> 172.21.15.86:0/4216150359 conn(0x7f23880fd340 msgr2=0x7f23880ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.639+0000 7f238da00640 1 -- 172.21.15.86:0/4216150359 shutdown_connections 2024-04-17T07:14:15.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.639+0000 7f238da00640 1 -- 172.21.15.86:0/4216150359 wait complete. 2024-04-17T07:14:15.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f238da00640 1 Processor -- start 2024-04-17T07:14:15.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f238da00640 1 -- start start 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f238da00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23880749c0 0x7f238819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f238da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388075fa0 0x7f238819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f238da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2388109c90 0x7f23881a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f238da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f238810fb00 con 0x7f2388075fa0 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f238da00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f238810f980 con 0x7f2388109c90 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f238da00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f238810fc80 con 0x7f23880749c0 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f2387400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388075fa0 0x7f238819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f2387400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388075fa0 0x7f238819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37840/0 (socket says 172.21.15.86:37840) 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f2387400640 1 -- 172.21.15.86:0/2460292034 learned_addr learned my addr 172.21.15.86:0/2460292034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.640+0000 7f2387e00640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23880749c0 0x7f238819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f238d000640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2388109c90 0x7f23881a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f2387400640 1 -- 172.21.15.86:0/2460292034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23880749c0 msgr2=0x7f238819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f2387400640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23880749c0 0x7f238819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f2387400640 1 -- 172.21.15.86:0/2460292034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2388109c90 msgr2=0x7f23881a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f2387400640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2388109c90 0x7f23881a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f2387400640 1 -- 172.21.15.86:0/2460292034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23881a1d20 con 0x7f2388075fa0 2024-04-17T07:14:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f2387400640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388075fa0 0x7f238819a640 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f237400c920 tx=0x7f237400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f2384c00640 1 -- 172.21.15.86:0/2460292034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374007c70 con 0x7f2388075fa0 2024-04-17T07:14:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f2384c00640 1 -- 172.21.15.86:0/2460292034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2374007e10 con 0x7f2388075fa0 2024-04-17T07:14:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f238da00640 1 -- 172.21.15.86:0/2460292034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23881a1fb0 con 0x7f2388075fa0 2024-04-17T07:14:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f2384c00640 1 -- 172.21.15.86:0/2460292034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374018820 con 0x7f2388075fa0 2024-04-17T07:14:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.641+0000 7f238da00640 1 -- 172.21.15.86:0/2460292034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23881a2520 con 0x7f2388075fa0 2024-04-17T07:14:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.643+0000 7f238da00640 1 -- 172.21.15.86:0/2460292034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23881089c0 con 0x7f2388075fa0 2024-04-17T07:14:15.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.645+0000 7f2384c00640 1 -- 172.21.15.86:0/2460292034 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f23740189c0 con 0x7f2388075fa0 2024-04-17T07:14:15.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.646+0000 7f2384c00640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f236c0a3c30 0x7f236c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:15.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.646+0000 7f2384c00640 1 -- 172.21.15.86:0/2460292034 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f23740d0950 con 0x7f2388075fa0 2024-04-17T07:14:15.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.646+0000 7f2387e00640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f236c0a3c30 0x7f236c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:15.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.647+0000 7f2387e00640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f236c0a3c30 0x7f236c0a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2378005fd0 tx=0x7f2378005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.650+0000 7f2384c00640 1 -- 172.21.15.86:0/2460292034 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2374014030 con 0x7f2388075fa0 2024-04-17T07:14:15.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.896+0000 7f238da00640 1 -- 172.21.15.86:0/2460292034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2388062c00 con 0x7f2388075fa0 2024-04-17T07:14:15.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.897+0000 7f2384c00640 1 -- 172.21.15.86:0/2460292034 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f237400de20 con 0x7f2388075fa0 2024-04-17T07:14:15.896 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:15.896 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:12:31.203694+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[34460,34490,34430,34400,24505]},"id":1}]} 2024-04-17T07:14:15.896 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.899+0000 7f238da00640 1 -- 172.21.15.86:0/2460292034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f236c0a3c30 msgr2=0x7f236c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.899+0000 7f238da00640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f236c0a3c30 0x7f236c0a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2378005fd0 tx=0x7f2378005e20 comp rx=0 tx=0).stop 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.899+0000 7f238da00640 1 -- 172.21.15.86:0/2460292034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388075fa0 msgr2=0x7f238819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.899+0000 7f238da00640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388075fa0 0x7f238819a640 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f237400c920 tx=0x7f237400cde0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.900+0000 7f238da00640 1 -- 172.21.15.86:0/2460292034 shutdown_connections 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.900+0000 7f238da00640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f236c0a3c30 0x7f236c0a60f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.900+0000 7f238da00640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2388109c90 0x7f23881a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.900+0000 7f238da00640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388075fa0 0x7f238819a640 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.900+0000 7f238da00640 1 --2- 172.21.15.86:0/2460292034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23880749c0 0x7f238819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.900+0000 7f238da00640 1 -- 172.21.15.86:0/2460292034 >> 172.21.15.86:0/2460292034 conn(0x7f23880fd340 msgr2=0x7f2388105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.900+0000 7f238da00640 1 -- 172.21.15.86:0/2460292034 shutdown_connections 2024-04-17T07:14:15.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:15.900+0000 7f238da00640 1 -- 172.21.15.86:0/2460292034 wait complete. 2024-04-17T07:14:15.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T07:11:54.934275+0000', 'modified': '2024-04-17T07:12:31.203694+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34490, 'mds_1': 34460, 'mds_2': 34430, 'mds_3': 34400, 'mds_4': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14688': {'gid': 14688, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/2923053510', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 2923053510}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 2923053510}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14700': {'gid': 14700, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/4256331668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 4256331668}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 4256331668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6833/1707978686', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6832', 'nonce': 1707978686}, {'type': 'v1', 'addr': '172.21.15.103:6833', 'nonce': 1707978686}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6835/423518277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 423518277}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 423518277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6837/299191474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 299191474}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 299191474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.103:6839/2592108290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2592108290}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2592108290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34400': {'gid': 34400, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.98:6833/2318991416', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6832', 'nonce': 2318991416}, {'type': 'v1', 'addr': '172.21.15.98:6833', 'nonce': 2318991416}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34430': {'gid': 34430, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.98:6835/133134430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 133134430}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 133134430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34460': {'gid': 34460, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.98:6837/2794896256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 2794896256}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 2794896256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34490': {'gid': 34490, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/809350102', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 809350102}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 809350102}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34490, 'qdb_cluster': [34460, 34490, 34430, 34400, 24505]} 2024-04-17T07:14:15.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-17T07:14:15.920 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-17T07:14:16.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.130+0000 7efc4a200640 1 -- 172.21.15.86:0/3171841964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc441084a0 msgr2=0x7efc441088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.130+0000 7efc4a200640 1 --2- 172.21.15.86:0/3171841964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc441084a0 0x7efc441088c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009900 tx=0x7efc2c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.130+0000 7efc4a200640 1 -- 172.21.15.86:0/3171841964 shutdown_connections 2024-04-17T07:14:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.130+0000 7efc4a200640 1 --2- 172.21.15.86:0/3171841964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc4410a050 0x7efc441108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.130+0000 7efc4a200640 1 --2- 172.21.15.86:0/3171841964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc44109690 0x7efc44109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.130+0000 7efc4a200640 1 --2- 172.21.15.86:0/3171841964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc441084a0 0x7efc441088c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.130+0000 7efc4a200640 1 -- 172.21.15.86:0/3171841964 >> 172.21.15.86:0/3171841964 conn(0x7efc44077430 msgr2=0x7efc44079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.130+0000 7efc4a200640 1 -- 172.21.15.86:0/3171841964 shutdown_connections 2024-04-17T07:14:16.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.130+0000 7efc4a200640 1 -- 172.21.15.86:0/3171841964 wait complete. 2024-04-17T07:14:16.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.131+0000 7efc4a200640 1 Processor -- start 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.131+0000 7efc4a200640 1 -- start start 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc4a200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc441084a0 0x7efc4419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc4a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc44109690 0x7efc4419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc4a200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4410a050 0x7efc441a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc4a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc44114030 con 0x7efc44109690 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc4a200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc44113eb0 con 0x7efc441084a0 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc4a200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc441141b0 con 0x7efc4410a050 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc44109690 0x7efc4419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc44109690 0x7efc4419ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37858/0 (socket says 172.21.15.86:37858) 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc3be00640 1 -- 172.21.15.86:0/1821685551 learned_addr learned my addr 172.21.15.86:0/1821685551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc3be00640 1 -- 172.21.15.86:0/1821685551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4410a050 msgr2=0x7efc441a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc3be00640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4410a050 0x7efc441a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc48e00640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc441084a0 0x7efc4419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc3be00640 1 -- 172.21.15.86:0/1821685551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc441084a0 msgr2=0x7efc4419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc3be00640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc441084a0 0x7efc4419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc3be00640 1 -- 172.21.15.86:0/1821685551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc441a6310 con 0x7efc44109690 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.132+0000 7efc48e00640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc441084a0 0x7efc4419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.133+0000 7efc3be00640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc44109690 0x7efc4419ec30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efc3400d950 tx=0x7efc3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.133+0000 7efc42000640 1 -- 172.21.15.86:0/1821685551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34017030 con 0x7efc44109690 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.133+0000 7efc4a200640 1 -- 172.21.15.86:0/1821685551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc441a6600 con 0x7efc44109690 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.133+0000 7efc42000640 1 -- 172.21.15.86:0/1821685551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc34004020 con 0x7efc44109690 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.133+0000 7efc42000640 1 -- 172.21.15.86:0/1821685551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc340047a0 con 0x7efc44109690 2024-04-17T07:14:16.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.133+0000 7efc4a200640 1 -- 172.21.15.86:0/1821685551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc441a6ba0 con 0x7efc44109690 2024-04-17T07:14:16.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.136+0000 7efc4e3e1740 1 Processor -- start 2024-04-17T07:14:16.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.136+0000 7efc4e3e1740 1 -- start start 2024-04-17T07:14:16.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae775f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 0x5623ae77afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae77b650 0x5623ae77dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623ae6dc980 con 0x5623ae775af0 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5623ae683220 con 0x5623ae7764e0 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5623ae6d8940 con 0x5623ae77b650 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc49800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae775f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc49800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae775f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37870/0 (socket says 172.21.15.86:37870) 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc49800640 1 -- 172.21.15.86:0/800912909 learned_addr learned my addr 172.21.15.86:0/800912909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc48e00640 1 --2- 172.21.15.86:0/800912909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 0x5623ae77afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.137+0000 7efc42000640 1 -- 172.21.15.86:0/1821685551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efc340041d0 con 0x7efc44109690 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.138+0000 7efc49800640 1 -- 172.21.15.86:0/800912909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae77b650 msgr2=0x5623ae77dae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.138+0000 7efc49800640 1 --2- 172.21.15.86:0/800912909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae77b650 0x5623ae77dae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.138+0000 7efc49800640 1 -- 172.21.15.86:0/800912909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 msgr2=0x5623ae77afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.138+0000 7efc49800640 1 --2- 172.21.15.86:0/800912909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 0x5623ae77afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.138+0000 7efc49800640 1 -- 172.21.15.86:0/800912909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623ae3ef2d0 con 0x5623ae775af0 2024-04-17T07:14:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.138+0000 7efc48e00640 1 --2- 172.21.15.86:0/800912909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 0x5623ae77afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:14:16.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.138+0000 7efc49800640 1 --2- 172.21.15.86:0/800912909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae775f10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009d50 tx=0x7efc3c00d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1f56d56a8021c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.138+0000 7efc3a000640 1 -- 172.21.15.86:0/800912909 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c003970 con 0x5623ae775af0 2024-04-17T07:14:16.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.138+0000 7efc3a000640 1 -- 172.21.15.86:0/800912909 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc3c004020 con 0x5623ae775af0 2024-04-17T07:14:16.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.138+0000 7efc3a000640 1 -- 172.21.15.86:0/800912909 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0106c0 con 0x5623ae775af0 2024-04-17T07:14:16.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.139+0000 7efc42000640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc240a3bb0 0x7efc240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.139+0000 7efc4e3e1740 1 -- 172.21.15.86:0/800912909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 msgr2=0x5623ae775f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.139+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/800912909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae775f10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009d50 tx=0x7efc3c00d6b0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.140+0000 7efc4e3e1740 1 -- 172.21.15.86:0/800912909 shutdown_connections 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.140+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/800912909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae77b650 0x5623ae77dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.140+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/800912909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 0x5623ae77afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.140+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/800912909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae775f10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.140+0000 7efc4e3e1740 1 -- 172.21.15.86:0/800912909 >> 172.21.15.86:0/800912909 conn(0x5623ae771060 msgr2=0x5623ae771460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.140+0000 7efc42000640 1 -- 172.21.15.86:0/1821685551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efc340d03c0 con 0x7efc44109690 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.140+0000 7efc48e00640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc240a3bb0 0x7efc240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.140+0000 7efc4e3e1740 1 -- 172.21.15.86:0/800912909 shutdown_connections 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.140+0000 7efc4e3e1740 1 -- 172.21.15.86:0/800912909 wait complete. 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.140+0000 7efc4e3e1740 1 -- start start 2024-04-17T07:14:16.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc48e00640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc240a3bb0 0x7efc240a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc2c006010 tx=0x7efc2c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc4e3e1740 1 Processor -- start 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae780000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 0x5623ae780540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae77b650 0x5623ae787580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623ae77e600 con 0x5623ae775af0 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5623ae696590 con 0x5623ae7764e0 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5623ae77e780 con 0x5623ae77b650 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc4e3e1740 10 client.0 _ll_register_callbacks cb 0x5623ae3eeac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc48e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae780000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc48e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae780000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37882/0 (socket says 172.21.15.86:37882) 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.141+0000 7efc48e00640 1 -- 172.21.15.86:0/1835365004 learned_addr learned my addr 172.21.15.86:0/1835365004 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.142+0000 7efc3be00640 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 0x5623ae780540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.142+0000 7efc49800640 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae77b650 0x5623ae787580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.142+0000 7efc48e00640 1 -- 172.21.15.86:0/1835365004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae77b650 msgr2=0x5623ae787580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.142+0000 7efc48e00640 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae77b650 0x5623ae787580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.142+0000 7efc48e00640 1 -- 172.21.15.86:0/1835365004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 msgr2=0x5623ae780540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.142+0000 7efc48e00640 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 0x5623ae780540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.142+0000 7efc48e00640 1 -- 172.21.15.86:0/1835365004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623ae4b96b0 con 0x5623ae775af0 2024-04-17T07:14:16.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.142+0000 7efc49800640 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae77b650 0x5623ae787580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.142+0000 7efc48e00640 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae780000 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efc441094f0 tx=0x7efc2c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f72a04bbf2aa279 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.142+0000 7efc32a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:16.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.143+0000 7efc32a00640 1 -- 172.21.15.86:0/1835365004 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c04e020 con 0x5623ae775af0 2024-04-17T07:14:16.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.143+0000 7efc32a00640 1 -- 172.21.15.86:0/1835365004 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efc2c0048c0 con 0x5623ae775af0 2024-04-17T07:14:16.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.143+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1835365004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc3c004020 con 0x5623ae775af0 2024-04-17T07:14:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.143+0000 7efc32a00640 1 -- 172.21.15.86:0/1835365004 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc2c059050 con 0x5623ae775af0 2024-04-17T07:14:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.143+0000 7efc4e3e1740 10 client.15315 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.143+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1835365004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623ae78a960 con 0x5623ae775af0 2024-04-17T07:14:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.143+0000 7efc32a00640 1 -- 172.21.15.86:0/1835365004 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc3c004020 con 0x5623ae775af0 2024-04-17T07:14:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.143+0000 7efc32a00640 1 -- 172.21.15.86:0/1835365004 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7efc2c003ce0 con 0x5623ae775af0 2024-04-17T07:14:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.143+0000 7efc4e3e1740 10 client.15315 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.144+0000 7efc4e3e1740 10 client.15315 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-17T07:14:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.144+0000 7efc4e3e1740 20 client.15315 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.144+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5623ae78d010 0x5623ae78e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.144+0000 7efc4e3e1740 4 client.15315 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:14:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.144+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1835365004 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623ae6ea3a0 con 0x5623ae78d010 2024-04-17T07:14:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.144+0000 7efc3be00640 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5623ae78d010 0x5623ae78e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.145+0000 7efc3be00640 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5623ae78d010 0x5623ae78e480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e11c30f2ff363849 server_cookie=c107119fea130d17 in_seq=0 out_seq=0 2024-04-17T07:14:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.145+0000 7efc32a00640 10 client.15315 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:14:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.147+0000 7efc32a00640 1 -- 172.21.15.86:0/1835365004 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623ae6ea3a0 con 0x5623ae78d010 2024-04-17T07:14:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.147+0000 7efc32a00640 10 client.15315 handle_command_reply: tid=0 2024-04-17T07:14:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.147+0000 7efc4e3e1740 1 client.15315 shutdown 2024-04-17T07:14:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.147+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1835365004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 msgr2=0x5623ae780000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.147+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae780000 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efc441094f0 tx=0x7efc2c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.148+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1835365004 shutdown_connections 2024-04-17T07:14:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.148+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5623ae78d010 0x5623ae78e480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.148+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae77b650 0x5623ae787580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.148+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae7764e0 0x5623ae780540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.148+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/1835365004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae775af0 0x5623ae780000 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.148+0000 7efc32a00640 0 client.15315 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:14:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.148+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1835365004 >> 172.21.15.86:0/1835365004 conn(0x5623ae771060 msgr2=0x5623ae7703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.148+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1835365004 shutdown_connections 2024-04-17T07:14:16.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.148+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1835365004 wait complete. 2024-04-17T07:14:16.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.148+0000 7efc4e3e1740 20 client.15315 trim_cache size 0 max 16384 2024-04-17T07:14:16.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.170+0000 7efc4e3e1740 1 Processor -- start 2024-04-17T07:14:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.170+0000 7efc4e3e1740 1 -- start start 2024-04-17T07:14:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.170+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae775af0 0x5623ae76c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.170+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae7764e0 0x5623ae787540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.170+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae77b650 0x5623ae777a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.170+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623ae78c7c0 con 0x5623ae7764e0 2024-04-17T07:14:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.170+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5623ae78ce90 con 0x5623ae77b650 2024-04-17T07:14:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.170+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5623ae78d560 con 0x5623ae775af0 2024-04-17T07:14:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae7764e0 0x5623ae787540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae7764e0 0x5623ae787540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37896/0 (socket says 172.21.15.86:37896) 2024-04-17T07:14:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc3be00640 1 -- 172.21.15.86:0/1927079856 learned_addr learned my addr 172.21.15.86:0/1927079856 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc48e00640 1 --2- 172.21.15.86:0/1927079856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae775af0 0x5623ae76c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc3be00640 1 -- 172.21.15.86:0/1927079856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae775af0 msgr2=0x5623ae76c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc3be00640 1 --2- 172.21.15.86:0/1927079856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae775af0 0x5623ae76c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc3be00640 1 -- 172.21.15.86:0/1927079856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae77b650 msgr2=0x5623ae777a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc49800640 1 --2- 172.21.15.86:0/1927079856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae77b650 0x5623ae777a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc3be00640 1 --2- 172.21.15.86:0/1927079856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae77b650 0x5623ae777a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc3be00640 1 -- 172.21.15.86:0/1927079856 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623ae77acf0 con 0x5623ae7764e0 2024-04-17T07:14:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.171+0000 7efc49800640 1 --2- 172.21.15.86:0/1927079856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae77b650 0x5623ae777a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.172+0000 7efc3be00640 1 --2- 172.21.15.86:0/1927079856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae7764e0 0x5623ae787540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efc3409bfc0 tx=0x7efc3400b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d717edb76d22d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.172+0000 7efc39600640 1 -- 172.21.15.86:0/1927079856 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc340b5070 con 0x5623ae7764e0 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.172+0000 7efc39600640 1 -- 172.21.15.86:0/1927079856 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc340a0670 con 0x5623ae7764e0 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.172+0000 7efc39600640 1 -- 172.21.15.86:0/1927079856 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34085070 con 0x5623ae7764e0 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.172+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1927079856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae7764e0 msgr2=0x5623ae787540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.172+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/1927079856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae7764e0 0x5623ae787540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efc3409bfc0 tx=0x7efc3400b510 comp rx=0 tx=0).stop 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.173+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1927079856 shutdown_connections 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.173+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/1927079856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae77b650 0x5623ae777a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.173+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/1927079856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae7764e0 0x5623ae787540 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.173+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/1927079856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae775af0 0x5623ae76c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.173+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1927079856 >> 172.21.15.86:0/1927079856 conn(0x5623ae771060 msgr2=0x5623ae771460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.173+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1927079856 shutdown_connections 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.173+0000 7efc4e3e1740 1 -- 172.21.15.86:0/1927079856 wait complete. 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.173+0000 7efc4e3e1740 1 -- start start 2024-04-17T07:14:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.173+0000 7efc4e3e1740 1 Processor -- start 2024-04-17T07:14:16.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae775af0 0x5623ae785530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae7764e0 0x5623ae7eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc4e3e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae77b650 0x5623ae7f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5623ae78e300 con 0x5623ae77b650 2024-04-17T07:14:16.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5623ae78dc30 con 0x5623ae775af0 2024-04-17T07:14:16.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc4e3e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5623ae78e9d0 con 0x5623ae7764e0 2024-04-17T07:14:16.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc4e3e1740 10 client.0 _ll_register_callbacks cb 0x7efc2c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc49800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae77b650 0x5623ae7f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc49800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae77b650 0x5623ae7f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37906/0 (socket says 172.21.15.86:37906) 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc49800640 1 -- 172.21.15.86:0/2478965799 learned_addr learned my addr 172.21.15.86:0/2478965799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc48e00640 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae775af0 0x5623ae785530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.174+0000 7efc3be00640 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae7764e0 0x5623ae7eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc49800640 1 -- 172.21.15.86:0/2478965799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae7764e0 msgr2=0x5623ae7eefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc49800640 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae7764e0 0x5623ae7eefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc49800640 1 -- 172.21.15.86:0/2478965799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae775af0 msgr2=0x5623ae785530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc49800640 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae775af0 0x5623ae785530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc49800640 1 -- 172.21.15.86:0/2478965799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623ae7f1df0 con 0x5623ae77b650 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc3be00640 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae7764e0 0x5623ae7eefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc48e00640 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae775af0 0x5623ae785530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc49800640 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae77b650 0x5623ae7f15c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009610 tx=0x7efc3c009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecad495796a2beb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc32a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc32a00640 1 -- 172.21.15.86:0/2478965799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c003c40 con 0x5623ae77b650 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc32a00640 1 -- 172.21.15.86:0/2478965799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efc3c003de0 con 0x5623ae77b650 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc4e3e1740 1 -- 172.21.15.86:0/2478965799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc340a0670 con 0x5623ae77b650 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc32a00640 1 -- 172.21.15.86:0/2478965799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc3c004020 con 0x5623ae77b650 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc4e3e1740 10 client.15321 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.175+0000 7efc4e3e1740 1 -- 172.21.15.86:0/2478965799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623ae7f30d0 con 0x5623ae77b650 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.176+0000 7efc32a00640 1 -- 172.21.15.86:0/2478965799 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc340a0670 con 0x5623ae77b650 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.176+0000 7efc32a00640 1 -- 172.21.15.86:0/2478965799 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7efc3c0108a0 con 0x5623ae77b650 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.176+0000 7efc4e3e1740 10 client.15321 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.176+0000 7efc4e3e1740 10 client.15321 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.176+0000 7efc4e3e1740 20 client.15321 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.176+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5623ae7f7b40 0x5623ae7f6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.176+0000 7efc4e3e1740 4 client.15321 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:14:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.176+0000 7efc4e3e1740 1 -- 172.21.15.86:0/2478965799 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "status"}) -- 0x5623ae6e4210 con 0x5623ae7f7b40 2024-04-17T07:14:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.177+0000 7efc48e00640 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5623ae7f7b40 0x5623ae7f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc48e00640 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5623ae7f7b40 0x5623ae7f6850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bbf391dbf80e8ba server_cookie=fd4a4469387d98f2 in_seq=0 out_seq=0 2024-04-17T07:14:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc32a00640 10 client.15321 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:14:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc32a00640 1 -- 172.21.15.86:0/2478965799 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5623ae6e4210 con 0x5623ae7f7b40 2024-04-17T07:14:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc32a00640 10 client.15321 handle_command_reply: tid=0 2024-04-17T07:14:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc4e3e1740 1 client.15321 shutdown 2024-04-17T07:14:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc4e3e1740 1 -- 172.21.15.86:0/2478965799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae77b650 msgr2=0x5623ae7f15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae77b650 0x5623ae7f15c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009610 tx=0x7efc3c009a60 comp rx=0 tx=0).stop 2024-04-17T07:14:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc4e3e1740 1 -- 172.21.15.86:0/2478965799 shutdown_connections 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5623ae7f7b40 0x5623ae7f6850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5623ae77b650 0x5623ae7f15c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5623ae7764e0 0x5623ae7eefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc4e3e1740 1 --2- 172.21.15.86:0/2478965799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5623ae775af0 0x5623ae785530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc32a00640 0 client.15321 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.178+0000 7efc4e3e1740 1 -- 172.21.15.86:0/2478965799 >> 172.21.15.86:0/2478965799 conn(0x5623ae771060 msgr2=0x5623ae76c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4e3e1740 1 -- 172.21.15.86:0/2478965799 shutdown_connections 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4e3e1740 1 -- 172.21.15.86:0/2478965799 wait complete. 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4e3e1740 20 client.15321 trim_cache size 0 max 16384 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:16.181 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24505, 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 105.98638755100001, 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 91, 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 159.36327087999999 2024-04-17T07:14:16.182 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:16.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4a200640 1 -- 172.21.15.86:0/1821685551 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc240a3bb0 msgr2=0x7efc240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4a200640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc240a3bb0 0x7efc240a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc2c006010 tx=0x7efc2c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4a200640 1 -- 172.21.15.86:0/1821685551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc44109690 msgr2=0x7efc4419ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4a200640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc44109690 0x7efc4419ec30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efc3400d950 tx=0x7efc3400de10 comp rx=0 tx=0).stop 2024-04-17T07:14:16.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4a200640 1 -- 172.21.15.86:0/1821685551 shutdown_connections 2024-04-17T07:14:16.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4a200640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc240a3bb0 0x7efc240a6070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4a200640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc4410a050 0x7efc441a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4a200640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc44109690 0x7efc4419ec30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4a200640 1 --2- 172.21.15.86:0/1821685551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc441084a0 0x7efc4419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.179+0000 7efc4a200640 1 -- 172.21.15.86:0/1821685551 >> 172.21.15.86:0/1821685551 conn(0x7efc44077430 msgr2=0x7efc4410ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.180+0000 7efc4a200640 1 -- 172.21.15.86:0/1821685551 shutdown_connections 2024-04-17T07:14:16.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.180+0000 7efc4a200640 1 -- 172.21.15.86:0/1821685551 wait complete. 2024-04-17T07:14:16.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-17T07:14:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.403+0000 7f7906600640 1 -- 172.21.15.86:0/4294790623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7900107c00 msgr2=0x7f7900109ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.403+0000 7f7906600640 1 --2- 172.21.15.86:0/4294790623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7900107c00 0x7f7900109ff0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009ca0 tx=0x7f78f802c0f0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.403+0000 7f7906600640 1 -- 172.21.15.86:0/4294790623 shutdown_connections 2024-04-17T07:14:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.403+0000 7f7906600640 1 --2- 172.21.15.86:0/4294790623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7900107c00 0x7f7900109ff0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.403+0000 7f7906600640 1 --2- 172.21.15.86:0/4294790623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79001052d0 0x7f79001076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.403+0000 7f7906600640 1 --2- 172.21.15.86:0/4294790623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79001029a0 0x7f7900104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.403+0000 7f7906600640 1 -- 172.21.15.86:0/4294790623 >> 172.21.15.86:0/4294790623 conn(0x7f79000fc310 msgr2=0x7f79000fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.403+0000 7f7906600640 1 -- 172.21.15.86:0/4294790623 shutdown_connections 2024-04-17T07:14:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.403+0000 7f7906600640 1 -- 172.21.15.86:0/4294790623 wait complete. 2024-04-17T07:14:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.404+0000 7f7906600640 1 Processor -- start 2024-04-17T07:14:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.405+0000 7f7906600640 1 -- start start 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.405+0000 7f7906600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79001029a0 0x7f790019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.405+0000 7f7906600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79001052d0 0x7f790019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.405+0000 7f7906600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7900107c00 0x7f79001a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.405+0000 7f7906600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f790010d9d0 con 0x7f7900107c00 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.405+0000 7f7906600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f790010d850 con 0x7f79001052d0 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.405+0000 7f7906600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f790010db50 con 0x7f79001029a0 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.405+0000 7f7905c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7900107c00 0x7f79001a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.405+0000 7f7905c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7900107c00 0x7f79001a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37922/0 (socket says 172.21.15.86:37922) 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.405+0000 7f7905c00640 1 -- 172.21.15.86:0/2925811890 learned_addr learned my addr 172.21.15.86:0/2925811890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f78ffe00640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79001052d0 0x7f790019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f7905200640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79001029a0 0x7f790019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f7905c00640 1 -- 172.21.15.86:0/2925811890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79001029a0 msgr2=0x7f790019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f7905c00640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79001029a0 0x7f790019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f7905c00640 1 -- 172.21.15.86:0/2925811890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79001052d0 msgr2=0x7f790019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f7905c00640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79001052d0 0x7f790019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f7905c00640 1 -- 172.21.15.86:0/2925811890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79001a1f40 con 0x7f7900107c00 2024-04-17T07:14:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f7905c00640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7900107c00 0x7f79001a18d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f78f8032380 tx=0x7f78f8032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f78fd600640 1 -- 172.21.15.86:0/2925811890 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8041020 con 0x7f7900107c00 2024-04-17T07:14:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f78fd600640 1 -- 172.21.15.86:0/2925811890 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78f8003e20 con 0x7f7900107c00 2024-04-17T07:14:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f78fd600640 1 -- 172.21.15.86:0/2925811890 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8039040 con 0x7f7900107c00 2024-04-17T07:14:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.406+0000 7f7906600640 1 -- 172.21.15.86:0/2925811890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79001a21d0 con 0x7f7900107c00 2024-04-17T07:14:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.407+0000 7f7906600640 1 -- 172.21.15.86:0/2925811890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79001a2670 con 0x7f7900107c00 2024-04-17T07:14:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.410+0000 7f78fd600640 1 -- 172.21.15.86:0/2925811890 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f78f8044020 con 0x7f7900107c00 2024-04-17T07:14:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.410+0000 7f790a9a3740 1 Processor -- start 2024-04-17T07:14:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.410+0000 7f790a9a3740 1 -- start start 2024-04-17T07:14:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.410+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe4af0 0x55c395fe4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.410+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe54e0 0x55c395fe9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.410+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395fecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.410+0000 7f790a9a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c395f4b980 con 0x55c395fea650 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.410+0000 7f790a9a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c395ef2220 con 0x55c395fe54e0 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.410+0000 7f790a9a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c395f47940 con 0x55c395fe4af0 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395fecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395fecae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37934/0 (socket says 172.21.15.86:37934) 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78ffe00640 1 -- 172.21.15.86:0/3628858137 learned_addr learned my addr 172.21.15.86:0/3628858137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f7905200640 1 --2- 172.21.15.86:0/3628858137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe54e0 0x55c395fe9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78ffe00640 1 --2- 172.21.15.86:0/3628858137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe4af0 0x55c395fe4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78ffe00640 1 -- 172.21.15.86:0/3628858137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe4af0 msgr2=0x55c395fe4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78ffe00640 1 --2- 172.21.15.86:0/3628858137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe4af0 0x55c395fe4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78ffe00640 1 -- 172.21.15.86:0/3628858137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe54e0 msgr2=0x55c395fe9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78ffe00640 1 --2- 172.21.15.86:0/3628858137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe54e0 0x55c395fe9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78ffe00640 1 -- 172.21.15.86:0/3628858137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c395c5e2d0 con 0x55c395fea650 2024-04-17T07:14:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78fd600640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f78e80a3c30 0x7f78e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78fd600640 1 -- 172.21.15.86:0/2925811890 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f78f80f2ae0 con 0x7f7900107c00 2024-04-17T07:14:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f78ffe00640 1 --2- 172.21.15.86:0/3628858137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395fecae0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f78f000cfa0 tx=0x7f78f000c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149edc4840a51d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.411+0000 7f7905200640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f78e80a3c30 0x7f78e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.412+0000 7f78ee000640 1 -- 172.21.15.86:0/3628858137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0019030 con 0x55c395fea650 2024-04-17T07:14:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.412+0000 7f78ee000640 1 -- 172.21.15.86:0/3628858137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78f00054d0 con 0x55c395fea650 2024-04-17T07:14:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.412+0000 7f78ee000640 1 -- 172.21.15.86:0/3628858137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0011780 con 0x55c395fea650 2024-04-17T07:14:16.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.412+0000 7f790a9a3740 1 -- 172.21.15.86:0/3628858137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 msgr2=0x55c395fecae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.412+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3628858137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395fecae0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f78f000cfa0 tx=0x7f78f000c670 comp rx=0 tx=0).stop 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.412+0000 7f7905200640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f78e80a3c30 0x7f78e80a60f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f78f4005fd0 tx=0x7f78f400b650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.413+0000 7f790a9a3740 1 -- 172.21.15.86:0/3628858137 shutdown_connections 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.413+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3628858137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395fecae0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.413+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3628858137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe54e0 0x55c395fe9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.413+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3628858137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe4af0 0x55c395fe4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.413+0000 7f790a9a3740 1 -- 172.21.15.86:0/3628858137 >> 172.21.15.86:0/3628858137 conn(0x55c395fe0060 msgr2=0x55c395fe0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.413+0000 7f790a9a3740 1 -- 172.21.15.86:0/3628858137 shutdown_connections 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.413+0000 7f790a9a3740 1 -- 172.21.15.86:0/3628858137 wait complete. 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.413+0000 7f790a9a3740 1 -- start start 2024-04-17T07:14:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.413+0000 7f790a9a3740 1 Processor -- start 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe4af0 0x55c395fef070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe54e0 0x55c395fef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f790a9a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c395fed600 con 0x55c395fea650 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f790a9a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c395f05590 con 0x55c395fe54e0 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f790a9a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c395fed780 con 0x55c395fe4af0 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f790a9a3740 10 client.0 _ll_register_callbacks cb 0x55c395c5dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f7905c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f7905c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37946/0 (socket says 172.21.15.86:37946) 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f7905c00640 1 -- 172.21.15.86:0/1017634937 learned_addr learned my addr 172.21.15.86:0/1017634937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f7905200640 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe4af0 0x55c395fef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.414+0000 7f78ffe00640 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe54e0 0x55c395fef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f7905c00640 1 -- 172.21.15.86:0/1017634937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe4af0 msgr2=0x55c395fef070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f7905c00640 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe4af0 0x55c395fef070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f7905c00640 1 -- 172.21.15.86:0/1017634937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe54e0 msgr2=0x55c395fef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f7905c00640 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe54e0 0x55c395fef5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f7905c00640 1 -- 172.21.15.86:0/1017634937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c395d286b0 con 0x55c395fea650 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f7905c00640 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff65f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f78f802c630 tx=0x7f78f80be7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174f8da4f27aba42 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f78e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f78e2a00640 1 -- 172.21.15.86:0/1017634937 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8004040 con 0x55c395fea650 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f78e2a00640 1 -- 172.21.15.86:0/1017634937 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f78f8004680 con 0x55c395fea650 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f790a9a3740 1 -- 172.21.15.86:0/1017634937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f00054d0 con 0x55c395fea650 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f78e2a00640 1 -- 172.21.15.86:0/1017634937 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78f803b2f0 con 0x55c395fea650 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f790a9a3740 10 client.15333 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.415+0000 7f790a9a3740 1 -- 172.21.15.86:0/1017634937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c395ff9960 con 0x55c395fea650 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.416+0000 7f78e2a00640 1 -- 172.21.15.86:0/1017634937 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78f00054d0 con 0x55c395fea650 2024-04-17T07:14:16.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.416+0000 7f78e2a00640 1 -- 172.21.15.86:0/1017634937 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7f78f803c030 con 0x55c395fea650 2024-04-17T07:14:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.416+0000 7f790a9a3740 10 client.15333 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.416+0000 7f790a9a3740 10 client.15333 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-17T07:14:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.416+0000 7f790a9a3740 20 client.15333 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.416+0000 7f790a9a3740 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c395ffbfb0 0x55c395ffd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.416+0000 7f790a9a3740 4 client.15333 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:14:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.416+0000 7f790a9a3740 1 -- 172.21.15.86:0/1017634937 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c395f593a0 con 0x55c395ffbfb0 2024-04-17T07:14:16.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.417+0000 7f7905200640 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c395ffbfb0 0x55c395ffd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.418+0000 7f7905200640 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c395ffbfb0 0x55c395ffd420 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bfee02de464a541 server_cookie=4b91b6666876813e in_seq=0 out_seq=0 2024-04-17T07:14:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.418+0000 7f78e2a00640 10 client.15333 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:14:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.419+0000 7f78e2a00640 1 -- 172.21.15.86:0/1017634937 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c395f593a0 con 0x55c395ffbfb0 2024-04-17T07:14:16.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.419+0000 7f78e2a00640 10 client.15333 handle_command_reply: tid=0 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.419+0000 7f790a9a3740 1 client.15333 shutdown 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 1 -- 172.21.15.86:0/1017634937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 msgr2=0x55c395ff65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff65f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f78f802c630 tx=0x7f78f80be7b0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 1 -- 172.21.15.86:0/1017634937 shutdown_connections 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c395ffbfb0 0x55c395ffd420 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff65f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe54e0 0x55c395fef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 1 --2- 172.21.15.86:0/1017634937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe4af0 0x55c395fef070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 1 -- 172.21.15.86:0/1017634937 >> 172.21.15.86:0/1017634937 conn(0x55c395fe0060 msgr2=0x55c395fdf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f78e2a00640 0 client.15333 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 1 -- 172.21.15.86:0/1017634937 shutdown_connections 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 1 -- 172.21.15.86:0/1017634937 wait complete. 2024-04-17T07:14:16.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.420+0000 7f790a9a3740 20 client.15333 trim_cache size 0 max 16384 2024-04-17T07:14:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f790a9a3740 1 Processor -- start 2024-04-17T07:14:16.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f790a9a3740 1 -- start start 2024-04-17T07:14:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 0x55c395fdb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fe54e0 0x55c395ff65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fea650 0x55c395fe0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f790a9a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c395ffb760 con 0x55c395fe54e0 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f790a9a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c395ffbe30 con 0x55c395fe4af0 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f790a9a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c395ffc500 con 0x55c395fea650 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f78ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fe54e0 0x55c395ff65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f78ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fe54e0 0x55c395ff65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37950/0 (socket says 172.21.15.86:37950) 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.440+0000 7f78ffe00640 1 -- 172.21.15.86:0/3636123655 learned_addr learned my addr 172.21.15.86:0/3636123655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.441+0000 7f7905c00640 1 --2- 172.21.15.86:0/3636123655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fea650 0x55c395fe0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.441+0000 7f7905200640 1 --2- 172.21.15.86:0/3636123655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 0x55c395fdb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.441+0000 7f78ffe00640 1 -- 172.21.15.86:0/3636123655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fea650 msgr2=0x55c395fe0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.441+0000 7f78ffe00640 1 --2- 172.21.15.86:0/3636123655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fea650 0x55c395fe0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.441+0000 7f78ffe00640 1 -- 172.21.15.86:0/3636123655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 msgr2=0x55c395fdb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.441+0000 7f78ffe00640 1 --2- 172.21.15.86:0/3636123655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 0x55c395fdb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.441+0000 7f78ffe00640 1 -- 172.21.15.86:0/3636123655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c395fd1900 con 0x55c395fe54e0 2024-04-17T07:14:16.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.441+0000 7f78ffe00640 1 --2- 172.21.15.86:0/3636123655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fe54e0 0x55c395ff65b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f78f00058c0 tx=0x7f78f0005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dec1de4598eeeac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f78ed600640 1 -- 172.21.15.86:0/3636123655 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0019040 con 0x55c395fe54e0 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f78ed600640 1 -- 172.21.15.86:0/3636123655 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78f0004680 con 0x55c395fe54e0 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f78ed600640 1 -- 172.21.15.86:0/3636123655 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0011780 con 0x55c395fe54e0 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f790a9a3740 1 -- 172.21.15.86:0/3636123655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fe54e0 msgr2=0x55c395ff65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3636123655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fe54e0 0x55c395ff65b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f78f00058c0 tx=0x7f78f0005eb0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f790a9a3740 1 -- 172.21.15.86:0/3636123655 shutdown_connections 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3636123655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fea650 0x55c395fe0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3636123655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fe54e0 0x55c395ff65b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3636123655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 0x55c395fdb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f790a9a3740 1 -- 172.21.15.86:0/3636123655 >> 172.21.15.86:0/3636123655 conn(0x55c395fe0060 msgr2=0x55c395fedb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f790a9a3740 1 -- 172.21.15.86:0/3636123655 shutdown_connections 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.442+0000 7f790a9a3740 1 -- 172.21.15.86:0/3636123655 wait complete. 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.443+0000 7f790a9a3740 1 -- start start 2024-04-17T07:14:16.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.443+0000 7f790a9a3740 1 Processor -- start 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.443+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 0x55c395ff2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.443+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe54e0 0x55c395ff2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.443+0000 7f790a9a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.443+0000 7f790a9a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c395ffd2a0 con 0x55c395fea650 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.443+0000 7f790a9a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c395ffcbd0 con 0x55c395fe4af0 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.443+0000 7f790a9a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c395ffd970 con 0x55c395fe54e0 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.443+0000 7f790a9a3740 10 client.0 _ll_register_callbacks cb 0x7f78f8009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37958/0 (socket says 172.21.15.86:37958) 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905c00640 1 -- 172.21.15.86:0/3362096743 learned_addr learned my addr 172.21.15.86:0/3362096743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905200640 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 0x55c395ff2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f78ffe00640 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe54e0 0x55c395ff2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905c00640 1 -- 172.21.15.86:0/3362096743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe54e0 msgr2=0x55c395ff2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905c00640 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe54e0 0x55c395ff2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905c00640 1 -- 172.21.15.86:0/3362096743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 msgr2=0x55c395ff2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905c00640 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 0x55c395ff2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905c00640 1 -- 172.21.15.86:0/3362096743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c395ff5e50 con 0x55c395fea650 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f78ffe00640 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe54e0 0x55c395ff2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905200640 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 0x55c395ff2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f7905c00640 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff5860 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f78f80bb360 tx=0x7f78f80bcef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd0b2019d6bf3df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.444+0000 7f78e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f78e2a00640 1 -- 172.21.15.86:0/3362096743 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80c76a0 con 0x55c395fea650 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f78e2a00640 1 -- 172.21.15.86:0/3362096743 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f78f80ddda0 con 0x55c395fea650 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f790a9a3740 1 -- 172.21.15.86:0/3362096743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f0004680 con 0x55c395fea650 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f78e2a00640 1 -- 172.21.15.86:0/3362096743 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78f80be520 con 0x55c395fea650 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f790a9a3740 10 client.15339 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:16.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f790a9a3740 1 -- 172.21.15.86:0/3362096743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c3960625c0 con 0x55c395fea650 2024-04-17T07:14:16.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f78e2a00640 1 -- 172.21.15.86:0/3362096743 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78f0004680 con 0x55c395fea650 2024-04-17T07:14:16.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f78e2a00640 1 -- 172.21.15.86:0/3362096743 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7f78f8004b40 con 0x55c395fea650 2024-04-17T07:14:16.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f790a9a3740 10 client.15339 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:16.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f790a9a3740 10 client.15339 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-17T07:14:16.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f790a9a3740 20 client.15339 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:16.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c396066ff0 0x55c396065d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f790a9a3740 4 client.15339 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:14:16.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.445+0000 7f790a9a3740 1 -- 172.21.15.86:0/3362096743 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "status"}) -- 0x55c395f53210 con 0x55c396066ff0 2024-04-17T07:14:16.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.446+0000 7f7905200640 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c396066ff0 0x55c396065d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.447+0000 7f7905200640 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c396066ff0 0x55c396065d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3f9241fe38344fd server_cookie=1da69796d1ba76b7 in_seq=0 out_seq=0 2024-04-17T07:14:16.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.447+0000 7f78e2a00640 10 client.15339 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:14:16.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.447+0000 7f78e2a00640 1 -- 172.21.15.86:0/3362096743 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c395f53210 con 0x55c396066ff0 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.447+0000 7f78e2a00640 10 client.15339 handle_command_reply: tid=0 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.447+0000 7f790a9a3740 1 client.15339 shutdown 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 1 -- 172.21.15.86:0/3362096743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 msgr2=0x55c395ff5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff5860 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f78f80bb360 tx=0x7f78f80bcef0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 1 -- 172.21.15.86:0/3362096743 shutdown_connections 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c396066ff0 0x55c396065d00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c395fea650 0x55c395ff5860 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c395fe54e0 0x55c395ff2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 1 --2- 172.21.15.86:0/3362096743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c395fe4af0 0x55c395ff2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 1 -- 172.21.15.86:0/3362096743 >> 172.21.15.86:0/3362096743 conn(0x55c395fe0060 msgr2=0x55c395fdb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f78e2a00640 0 client.15339 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 1 -- 172.21.15.86:0/3362096743 shutdown_connections 2024-04-17T07:14:16.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 1 -- 172.21.15.86:0/3362096743 wait complete. 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.448+0000 7f790a9a3740 20 client.15339 trim_cache size 0 max 16384 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 34400, 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 107.272625578, 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-04-17T07:14:16.451 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-17T07:14:16.452 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 91, 2024-04-17T07:14:16.452 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 188.30321604400001 2024-04-17T07:14:16.452 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:16.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 -- 172.21.15.86:0/2925811890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f78e80a3c30 msgr2=0x7f78e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f78e80a3c30 0x7f78e80a60f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f78f4005fd0 tx=0x7f78f400b650 comp rx=0 tx=0).stop 2024-04-17T07:14:16.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 -- 172.21.15.86:0/2925811890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7900107c00 msgr2=0x7f79001a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7900107c00 0x7f79001a18d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f78f8032380 tx=0x7f78f8032820 comp rx=0 tx=0).stop 2024-04-17T07:14:16.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 -- 172.21.15.86:0/2925811890 shutdown_connections 2024-04-17T07:14:16.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f78e80a3c30 0x7f78e80a60f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7900107c00 0x7f79001a18d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79001052d0 0x7f790019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 --2- 172.21.15.86:0/2925811890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79001029a0 0x7f790019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 -- 172.21.15.86:0/2925811890 >> 172.21.15.86:0/2925811890 conn(0x7f79000fc310 msgr2=0x7f79001021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 -- 172.21.15.86:0/2925811890 shutdown_connections 2024-04-17T07:14:16.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.449+0000 7f7906600640 1 -- 172.21.15.86:0/2925811890 wait complete. 2024-04-17T07:14:16.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:16 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4214863600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:16.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:16 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2460292034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:16.464 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-17T07:14:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:16 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4214863600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:16.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:16 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2460292034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:16 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4214863600' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:16 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2460292034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:16.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.660+0000 7f921ce00640 1 -- 172.21.15.86:0/2890170508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92181053b0 msgr2=0x7f9218105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.660+0000 7f921ce00640 1 --2- 172.21.15.86:0/2890170508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92181053b0 0x7f9218105830 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9200009900 tx=0x7f920002bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.660+0000 7f921ce00640 1 -- 172.21.15.86:0/2890170508 shutdown_connections 2024-04-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.660+0000 7f921ce00640 1 --2- 172.21.15.86:0/2890170508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9218105d70 0x7f921810c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.660+0000 7f921ce00640 1 --2- 172.21.15.86:0/2890170508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92181053b0 0x7f9218105830 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.660+0000 7f921ce00640 1 --2- 172.21.15.86:0/2890170508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92181041c0 0x7f92181045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.660+0000 7f921ce00640 1 -- 172.21.15.86:0/2890170508 >> 172.21.15.86:0/2890170508 conn(0x7f92180ff750 msgr2=0x7f9218101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.660+0000 7f921ce00640 1 -- 172.21.15.86:0/2890170508 shutdown_connections 2024-04-17T07:14:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.660+0000 7f921ce00640 1 -- 172.21.15.86:0/2890170508 wait complete. 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.661+0000 7f921ce00640 1 Processor -- start 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.662+0000 7f921ce00640 1 -- start start 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.662+0000 7f921ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92181041c0 0x7f921819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.662+0000 7f921ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92181053b0 0x7f921819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.662+0000 7f921ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9218105d70 0x7f92181a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.662+0000 7f921ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f921810fd00 con 0x7f92181053b0 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.662+0000 7f921ce00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f921810fb80 con 0x7f9218105d70 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.662+0000 7f921ce00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f921810fe80 con 0x7f92181041c0 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.662+0000 7f920d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92181053b0 0x7f921819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.662+0000 7f920d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92181053b0 0x7f921819a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37986/0 (socket says 172.21.15.86:37986) 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.662+0000 7f920d400640 1 -- 172.21.15.86:0/2925497492 learned_addr learned my addr 172.21.15.86:0/2925497492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f9215e00640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92181041c0 0x7f921819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f9216800640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9218105d70 0x7f92181a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f920d400640 1 -- 172.21.15.86:0/2925497492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92181041c0 msgr2=0x7f921819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f920d400640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92181041c0 0x7f921819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f920d400640 1 -- 172.21.15.86:0/2925497492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9218105d70 msgr2=0x7f92181a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f920d400640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9218105d70 0x7f92181a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f920d400640 1 -- 172.21.15.86:0/2925497492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92181a1fa0 con 0x7f92181053b0 2024-04-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f9216800640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9218105d70 0x7f92181a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f920d400640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92181053b0 0x7f921819a8c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9200036040 tx=0x7f92000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f920f400640 1 -- 172.21.15.86:0/2925497492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200043020 con 0x7f92181053b0 2024-04-17T07:14:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f921ce00640 1 -- 172.21.15.86:0/2925497492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92181a2230 con 0x7f92181053b0 2024-04-17T07:14:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.663+0000 7f920f400640 1 -- 172.21.15.86:0/2925497492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92000054c0 con 0x7f92181053b0 2024-04-17T07:14:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.664+0000 7f921ce00640 1 -- 172.21.15.86:0/2925497492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92181a2780 con 0x7f92181053b0 2024-04-17T07:14:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.664+0000 7f920f400640 1 -- 172.21.15.86:0/2925497492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200003d10 con 0x7f92181053b0 2024-04-17T07:14:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.667+0000 7f921fbc7740 1 Processor -- start 2024-04-17T07:14:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.667+0000 7f920f400640 1 -- 172.21.15.86:0/2925497492 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9200046020 con 0x7f92181053b0 2024-04-17T07:14:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.667+0000 7f921fbc7740 1 -- start start 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.667+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e47d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.667+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 0x562ed0e4cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.667+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 0x562ed0e4f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.667+0000 7f921fbc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ed0dae980 con 0x562ed0e478e0 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.667+0000 7f921fbc7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562ed0d55220 con 0x562ed0e482d0 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.667+0000 7f921fbc7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562ed0daa940 con 0x562ed0e4d440 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.667+0000 7f9216800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e47d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f9216800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e47d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38002/0 (socket says 172.21.15.86:38002) 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f9216800640 1 -- 172.21.15.86:0/3230123593 learned_addr learned my addr 172.21.15.86:0/3230123593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f9215e00640 1 --2- 172.21.15.86:0/3230123593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 0x562ed0e4cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f920d400640 1 --2- 172.21.15.86:0/3230123593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 0x562ed0e4f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f9216800640 1 -- 172.21.15.86:0/3230123593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 msgr2=0x562ed0e4f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f9216800640 1 --2- 172.21.15.86:0/3230123593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 0x562ed0e4f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f9216800640 1 -- 172.21.15.86:0/3230123593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 msgr2=0x562ed0e4cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f9216800640 1 --2- 172.21.15.86:0/3230123593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 0x562ed0e4cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f9216800640 1 -- 172.21.15.86:0/3230123593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ed0ac12d0 con 0x562ed0e478e0 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f920d400640 1 --2- 172.21.15.86:0/3230123593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 0x562ed0e4f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f9216800640 1 --2- 172.21.15.86:0/3230123593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e47d00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9210008dd0 tx=0x7f921000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90de65a1ae4d6553 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.668+0000 7f920f400640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f91fc0a3bb0 0x7f91fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.669+0000 7f9207400640 1 -- 172.21.15.86:0/3230123593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921000f830 con 0x562ed0e478e0 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.669+0000 7f9207400640 1 -- 172.21.15.86:0/3230123593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f921000de20 con 0x562ed0e478e0 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.669+0000 7f9207400640 1 -- 172.21.15.86:0/3230123593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210018df0 con 0x562ed0e478e0 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.669+0000 7f920f400640 1 -- 172.21.15.86:0/2925497492 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f92000f2cb0 con 0x7f92181053b0 2024-04-17T07:14:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.669+0000 7f9215e00640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f91fc0a3bb0 0x7f91fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.669+0000 7f921fbc7740 1 -- 172.21.15.86:0/3230123593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 msgr2=0x562ed0e47d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.669+0000 7f921fbc7740 1 --2- 172.21.15.86:0/3230123593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e47d00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9210008dd0 tx=0x7f921000d710 comp rx=0 tx=0).stop 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.669+0000 7f9215e00640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f91fc0a3bb0 0x7f91fc0a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f92080058c0 tx=0x7f9208005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.669+0000 7f921fbc7740 1 -- 172.21.15.86:0/3230123593 shutdown_connections 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.670+0000 7f921fbc7740 1 --2- 172.21.15.86:0/3230123593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 0x562ed0e4f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.670+0000 7f921fbc7740 1 --2- 172.21.15.86:0/3230123593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 0x562ed0e4cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.670+0000 7f921fbc7740 1 --2- 172.21.15.86:0/3230123593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e47d00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.670+0000 7f921fbc7740 1 -- 172.21.15.86:0/3230123593 >> 172.21.15.86:0/3230123593 conn(0x562ed0e42e00 msgr2=0x562ed0e43200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.670+0000 7f921fbc7740 1 -- 172.21.15.86:0/3230123593 shutdown_connections 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.670+0000 7f921fbc7740 1 -- 172.21.15.86:0/3230123593 wait complete. 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.670+0000 7f921fbc7740 1 -- start start 2024-04-17T07:14:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.670+0000 7f921fbc7740 1 Processor -- start 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e51e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 0x562ed0e523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 0x562ed0e59400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f921fbc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ed0e503f0 con 0x562ed0e478e0 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f921fbc7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562ed0d68590 con 0x562ed0e482d0 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f921fbc7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562ed0e50570 con 0x562ed0e4d440 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f921fbc7740 10 client.0 _ll_register_callbacks cb 0x562ed0ac0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f9215e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e51e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f9215e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e51e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38016/0 (socket says 172.21.15.86:38016) 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f9215e00640 1 -- 172.21.15.86:0/2520503282 learned_addr learned my addr 172.21.15.86:0/2520503282 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f920d400640 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 0x562ed0e523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f9216800640 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 0x562ed0e59400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f9215e00640 1 -- 172.21.15.86:0/2520503282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 msgr2=0x562ed0e59400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f9215e00640 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 0x562ed0e59400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f9215e00640 1 -- 172.21.15.86:0/2520503282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 msgr2=0x562ed0e523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.671+0000 7f9215e00640 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 0x562ed0e523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f9215e00640 1 -- 172.21.15.86:0/2520503282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ed0b8b6b0 con 0x562ed0e478e0 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f9216800640 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 0x562ed0e59400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f920d400640 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 0x562ed0e523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f9215e00640 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e51e80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9208011010 tx=0x7f92080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f713865c6ce665 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f91fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f91fbe00640 1 -- 172.21.15.86:0/2520503282 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208022070 con 0x562ed0e478e0 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f91fbe00640 1 -- 172.21.15.86:0/2520503282 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9208004050 con 0x562ed0e478e0 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f921fbc7740 1 -- 172.21.15.86:0/2520503282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f921000de20 con 0x562ed0e478e0 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f91fbe00640 1 -- 172.21.15.86:0/2520503282 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f920802e050 con 0x562ed0e478e0 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f921fbc7740 10 client.15351 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.672+0000 7f921fbc7740 1 -- 172.21.15.86:0/2520503282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562ed0e5c800 con 0x562ed0e478e0 2024-04-17T07:14:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.673+0000 7f91fbe00640 1 -- 172.21.15.86:0/2520503282 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f921000de20 con 0x562ed0e478e0 2024-04-17T07:14:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.673+0000 7f91fbe00640 1 -- 172.21.15.86:0/2520503282 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7f920801e420 con 0x562ed0e478e0 2024-04-17T07:14:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.673+0000 7f921fbc7740 10 client.15351 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.673+0000 7f921fbc7740 10 client.15351 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-17T07:14:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.673+0000 7f921fbc7740 20 client.15351 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.673+0000 7f921fbc7740 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562ed0e5eee0 0x562ed0e60350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.673+0000 7f921fbc7740 4 client.15351 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:14:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.673+0000 7f921fbc7740 1 -- 172.21.15.86:0/2520503282 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ed0dbc3a0 con 0x562ed0e5eee0 2024-04-17T07:14:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.674+0000 7f920d400640 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562ed0e5eee0 0x562ed0e60350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.675+0000 7f920d400640 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562ed0e5eee0 0x562ed0e60350 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=390fb2ac06f8678c server_cookie=ad164cb171b3be8b in_seq=0 out_seq=0 2024-04-17T07:14:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.675+0000 7f91fbe00640 10 client.15351 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.677+0000 7f91fbe00640 1 -- 172.21.15.86:0/2520503282 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ed0dbc3a0 con 0x562ed0e5eee0 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.677+0000 7f91fbe00640 10 client.15351 handle_command_reply: tid=0 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 client.15351 shutdown 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 -- 172.21.15.86:0/2520503282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 msgr2=0x562ed0e51e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e51e80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9208011010 tx=0x7f92080046c0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 -- 172.21.15.86:0/2520503282 shutdown_connections 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562ed0e5eee0 0x562ed0e60350 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e4d440 0x562ed0e59400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e482d0 0x562ed0e523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f91fbe00640 0 client.15351 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 --2- 172.21.15.86:0/2520503282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e478e0 0x562ed0e51e80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 -- 172.21.15.86:0/2520503282 >> 172.21.15.86:0/2520503282 conn(0x562ed0e42e00 msgr2=0x562ed0e42090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 -- 172.21.15.86:0/2520503282 shutdown_connections 2024-04-17T07:14:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 1 -- 172.21.15.86:0/2520503282 wait complete. 2024-04-17T07:14:16.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.678+0000 7f921fbc7740 20 client.15351 trim_cache size 0 max 16384 2024-04-17T07:14:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.698+0000 7f921fbc7740 1 Processor -- start 2024-04-17T07:14:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.698+0000 7f921fbc7740 1 -- start start 2024-04-17T07:14:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.698+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 0x562ed0e50970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.698+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 0x562ed0e50eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.698+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.698+0000 7f921fbc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ed0e5e690 con 0x562ed0e4d440 2024-04-17T07:14:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.698+0000 7f921fbc7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562ed0e5ed60 con 0x562ed0e478e0 2024-04-17T07:14:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.698+0000 7f921fbc7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562ed0e5f430 con 0x562ed0e482d0 2024-04-17T07:14:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.698+0000 7f9216800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.698+0000 7f9216800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38026/0 (socket says 172.21.15.86:38026) 2024-04-17T07:14:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.699+0000 7f9216800640 1 -- 172.21.15.86:0/1955872677 learned_addr learned my addr 172.21.15.86:0/1955872677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.699+0000 7f9215e00640 1 --2- 172.21.15.86:0/1955872677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 0x562ed0e50970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.699+0000 7f920d400640 1 --2- 172.21.15.86:0/1955872677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 0x562ed0e50eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.699+0000 7f9216800640 1 -- 172.21.15.86:0/1955872677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 msgr2=0x562ed0e50eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.699+0000 7f9216800640 1 --2- 172.21.15.86:0/1955872677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 0x562ed0e50eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.699+0000 7f9216800640 1 -- 172.21.15.86:0/1955872677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 msgr2=0x562ed0e50970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.699+0000 7f9216800640 1 --2- 172.21.15.86:0/1955872677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 0x562ed0e50970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.699+0000 7f9216800640 1 -- 172.21.15.86:0/1955872677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ed0e60680 con 0x562ed0e4d440 2024-04-17T07:14:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.699+0000 7f9216800640 1 --2- 172.21.15.86:0/1955872677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e593c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9210017000 tx=0x7f921000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de453297a90b3ff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f9206a00640 1 -- 172.21.15.86:0/1955872677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921000f630 con 0x562ed0e4d440 2024-04-17T07:14:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f9206a00640 1 -- 172.21.15.86:0/1955872677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9210027070 con 0x562ed0e4d440 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f9206a00640 1 -- 172.21.15.86:0/1955872677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210018cb0 con 0x562ed0e4d440 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f921fbc7740 1 -- 172.21.15.86:0/1955872677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 msgr2=0x562ed0e593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f921fbc7740 1 --2- 172.21.15.86:0/1955872677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e593c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9210017000 tx=0x7f921000ded0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f921fbc7740 1 -- 172.21.15.86:0/1955872677 shutdown_connections 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f921fbc7740 1 --2- 172.21.15.86:0/1955872677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e593c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f921fbc7740 1 --2- 172.21.15.86:0/1955872677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 0x562ed0e50eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f921fbc7740 1 --2- 172.21.15.86:0/1955872677 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 0x562ed0e50970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f921fbc7740 1 -- 172.21.15.86:0/1955872677 >> 172.21.15.86:0/1955872677 conn(0x562ed0e42e00 msgr2=0x562ed0e5dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.700+0000 7f921fbc7740 1 -- 172.21.15.86:0/1955872677 shutdown_connections 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.701+0000 7f921fbc7740 1 -- 172.21.15.86:0/1955872677 wait complete. 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.701+0000 7f921fbc7740 1 -- start start 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.701+0000 7f921fbc7740 1 Processor -- start 2024-04-17T07:14:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.701+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 0x562ed0e54be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.701+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 0x562ed0e55120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.701+0000 7f921fbc7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e58140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.701+0000 7f921fbc7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ed0e601d0 con 0x562ed0e4d440 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f921fbc7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562ed0e5fb00 con 0x562ed0e478e0 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f921fbc7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562ed0e608a0 con 0x562ed0e482d0 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f921fbc7740 10 client.0 _ll_register_callbacks cb 0x7f920800c960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f9216800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e58140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f9216800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e58140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38028/0 (socket says 172.21.15.86:38028) 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f9216800640 1 -- 172.21.15.86:0/3647656979 learned_addr learned my addr 172.21.15.86:0/3647656979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f9216800640 1 -- 172.21.15.86:0/3647656979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 msgr2=0x562ed0e55120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f920d400640 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 0x562ed0e55120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f9215e00640 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 0x562ed0e54be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f9216800640 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 0x562ed0e55120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f9216800640 1 -- 172.21.15.86:0/3647656979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 msgr2=0x562ed0e54be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f9216800640 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 0x562ed0e54be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f9216800640 1 -- 172.21.15.86:0/3647656979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ed0e58730 con 0x562ed0e4d440 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f920d400640 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 0x562ed0e55120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.702+0000 7f9215e00640 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 0x562ed0e54be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.703+0000 7f9216800640 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e58140 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f921000d750 tx=0x7f9210018730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad5b3eafcd8964a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.703+0000 7f91fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.703+0000 7f91fbe00640 1 -- 172.21.15.86:0/3647656979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210031070 con 0x562ed0e4d440 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.703+0000 7f91fbe00640 1 -- 172.21.15.86:0/3647656979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f921000f630 con 0x562ed0e4d440 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.703+0000 7f921fbc7740 1 -- 172.21.15.86:0/3647656979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9210027070 con 0x562ed0e4d440 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.703+0000 7f91fbe00640 1 -- 172.21.15.86:0/3647656979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f921003d050 con 0x562ed0e4d440 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.703+0000 7f921fbc7740 10 client.15357 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.703+0000 7f921fbc7740 1 -- 172.21.15.86:0/3647656979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562ed0ecd700 con 0x562ed0e4d440 2024-04-17T07:14:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.704+0000 7f91fbe00640 1 -- 172.21.15.86:0/3647656979 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9210027070 con 0x562ed0e4d440 2024-04-17T07:14:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.704+0000 7f91fbe00640 1 -- 172.21.15.86:0/3647656979 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7f921002d8f0 con 0x562ed0e4d440 2024-04-17T07:14:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.704+0000 7f921fbc7740 10 client.15357 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.704+0000 7f921fbc7740 10 client.15357 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-17T07:14:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.704+0000 7f921fbc7740 20 client.15357 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.704+0000 7f921fbc7740 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562ed0ed1c30 0x562ed0ed0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.704+0000 7f921fbc7740 4 client.15357 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:14:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.704+0000 7f921fbc7740 1 -- 172.21.15.86:0/3647656979 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "status"}) -- 0x562ed0db6210 con 0x562ed0ed1c30 2024-04-17T07:14:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.705+0000 7f9215e00640 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562ed0ed1c30 0x562ed0ed0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.706+0000 7f9215e00640 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562ed0ed1c30 0x562ed0ed0940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9597eefdfd64cb0 server_cookie=5bf890fcc80cc512 in_seq=0 out_seq=0 2024-04-17T07:14:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.706+0000 7f91fbe00640 10 client.15357 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:14:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.706+0000 7f91fbe00640 1 -- 172.21.15.86:0/3647656979 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562ed0db6210 con 0x562ed0ed1c30 2024-04-17T07:14:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.706+0000 7f91fbe00640 10 client.15357 handle_command_reply: tid=0 2024-04-17T07:14:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.706+0000 7f921fbc7740 1 client.15357 shutdown 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 1 -- 172.21.15.86:0/3647656979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 msgr2=0x562ed0e58140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e58140 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f921000d750 tx=0x7f9210018730 comp rx=0 tx=0).stop 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 1 -- 172.21.15.86:0/3647656979 shutdown_connections 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562ed0ed1c30 0x562ed0ed0940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ed0e4d440 0x562ed0e58140 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562ed0e482d0 0x562ed0e55120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 1 --2- 172.21.15.86:0/3647656979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562ed0e478e0 0x562ed0e54be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f91fbe00640 0 client.15357 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 1 -- 172.21.15.86:0/3647656979 >> 172.21.15.86:0/3647656979 conn(0x562ed0e42e00 msgr2=0x562ed0e50970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 1 -- 172.21.15.86:0/3647656979 shutdown_connections 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 1 -- 172.21.15.86:0/3647656979 wait complete. 2024-04-17T07:14:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.707+0000 7f921fbc7740 20 client.15357 trim_cache size 0 max 16384 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 34430, 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 108.998634744, 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-17T07:14:16.711 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 91, 2024-04-17T07:14:16.712 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 176.26651486599999 2024-04-17T07:14:16.712 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:16.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 -- 172.21.15.86:0/2925497492 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f91fc0a3bb0 msgr2=0x7f91fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f91fc0a3bb0 0x7f91fc0a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f92080058c0 tx=0x7f9208005850 comp rx=0 tx=0).stop 2024-04-17T07:14:16.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 -- 172.21.15.86:0/2925497492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92181053b0 msgr2=0x7f921819a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92181053b0 0x7f921819a8c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9200036040 tx=0x7f92000042f0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 -- 172.21.15.86:0/2925497492 shutdown_connections 2024-04-17T07:14:16.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f91fc0a3bb0 0x7f91fc0a6070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9218105d70 0x7f92181a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92181053b0 0x7f921819a8c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 --2- 172.21.15.86:0/2925497492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92181041c0 0x7f921819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 -- 172.21.15.86:0/2925497492 >> 172.21.15.86:0/2925497492 conn(0x7f92180ff750 msgr2=0x7f9218108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 -- 172.21.15.86:0/2925497492 shutdown_connections 2024-04-17T07:14:16.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.708+0000 7f921ce00640 1 -- 172.21.15.86:0/2925497492 wait complete. 2024-04-17T07:14:16.724 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-17T07:14:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.928+0000 7fd404a00640 1 -- 172.21.15.86:0/769304061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4001053b0 msgr2=0x7fd400105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.928+0000 7fd404a00640 1 --2- 172.21.15.86:0/769304061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4001053b0 0x7fd400105830 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009900 tx=0x7fd3e802bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.929+0000 7fd404a00640 1 -- 172.21.15.86:0/769304061 shutdown_connections 2024-04-17T07:14:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.929+0000 7fd404a00640 1 --2- 172.21.15.86:0/769304061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd400105d70 0x7fd40010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.929+0000 7fd404a00640 1 --2- 172.21.15.86:0/769304061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4001053b0 0x7fd400105830 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.929+0000 7fd404a00640 1 --2- 172.21.15.86:0/769304061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4001041c0 0x7fd4001045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.929+0000 7fd404a00640 1 -- 172.21.15.86:0/769304061 >> 172.21.15.86:0/769304061 conn(0x7fd4000ff750 msgr2=0x7fd400101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.929+0000 7fd404a00640 1 -- 172.21.15.86:0/769304061 shutdown_connections 2024-04-17T07:14:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.929+0000 7fd404a00640 1 -- 172.21.15.86:0/769304061 wait complete. 2024-04-17T07:14:16.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.930+0000 7fd404a00640 1 Processor -- start 2024-04-17T07:14:16.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.930+0000 7fd404a00640 1 -- start start 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.930+0000 7fd404a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4001041c0 0x7fd40019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.930+0000 7fd404a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4001053b0 0x7fd40019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.930+0000 7fd404a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd400105d70 0x7fd4001a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.930+0000 7fd404a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd40010fd00 con 0x7fd4001041c0 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.930+0000 7fd404a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd40010fb80 con 0x7fd400105d70 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.930+0000 7fd404a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd40010fe80 con 0x7fd4001053b0 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4001041c0 0x7fd40019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4001041c0 0x7fd40019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54844/0 (socket says 172.21.15.86:54844) 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fde00640 1 -- 172.21.15.86:0/2790392986 learned_addr learned my addr 172.21.15.86:0/2790392986 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fe800640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd400105d70 0x7fd4001a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3f5400640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4001053b0 0x7fd40019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fde00640 1 -- 172.21.15.86:0/2790392986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4001053b0 msgr2=0x7fd40019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fde00640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4001053b0 0x7fd40019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fde00640 1 -- 172.21.15.86:0/2790392986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd400105d70 msgr2=0x7fd4001a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fde00640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd400105d70 0x7fd4001a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fde00640 1 -- 172.21.15.86:0/2790392986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4001a1fa0 con 0x7fd4001041c0 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fe800640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd400105d70 0x7fd4001a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3f5400640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4001053b0 0x7fd40019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.931+0000 7fd3fde00640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4001041c0 0x7fd40019a380 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000c970 tx=0x7fd3f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.932+0000 7fd3f7400640 1 -- 172.21.15.86:0/2790392986 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0007c10 con 0x7fd4001041c0 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.932+0000 7fd3f7400640 1 -- 172.21.15.86:0/2790392986 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3f0007db0 con 0x7fd4001041c0 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.932+0000 7fd3f7400640 1 -- 172.21.15.86:0/2790392986 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f000f870 con 0x7fd4001041c0 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.932+0000 7fd404a00640 1 -- 172.21.15.86:0/2790392986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4001a2290 con 0x7fd4001041c0 2024-04-17T07:14:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.932+0000 7fd404a00640 1 -- 172.21.15.86:0/2790392986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4001a2790 con 0x7fd4001041c0 2024-04-17T07:14:16.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.935+0000 7fd407612740 1 Processor -- start 2024-04-17T07:14:16.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.935+0000 7fd407612740 1 -- start start 2024-04-17T07:14:16.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.935+0000 7fd3f7400640 1 -- 172.21.15.86:0/2790392986 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd3f000fa10 con 0x7fd4001041c0 2024-04-17T07:14:16.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd407612740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb88e0 0x55f587cb8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd407612740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cbddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd407612740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd407612740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f587c1f980 con 0x55f587cbe440 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd407612740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f587bc6220 con 0x55f587cb92d0 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd407612740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f587c1b940 con 0x55f587cb88e0 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd3f5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd3f5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54846/0 (socket says 172.21.15.86:54846) 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd3f5400640 1 -- 172.21.15.86:0/4121141082 learned_addr learned my addr 172.21.15.86:0/4121141082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd3fe800640 1 --2- 172.21.15.86:0/4121141082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb88e0 0x55f587cb8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd3fde00640 1 --2- 172.21.15.86:0/4121141082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cbddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd3f5400640 1 -- 172.21.15.86:0/4121141082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb88e0 msgr2=0x55f587cb8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd3f5400640 1 --2- 172.21.15.86:0/4121141082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb88e0 0x55f587cb8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd3f5400640 1 -- 172.21.15.86:0/4121141082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 msgr2=0x55f587cbddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd3f5400640 1 --2- 172.21.15.86:0/4121141082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cbddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.936+0000 7fd3f5400640 1 -- 172.21.15.86:0/4121141082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5879322d0 con 0x55f587cbe440 2024-04-17T07:14:16.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.937+0000 7fd3f5400640 1 --2- 172.21.15.86:0/4121141082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc08d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8036040 tx=0x7fd3e8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47a21c22a97d4fc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.937+0000 7fd3ef000640 1 -- 172.21.15.86:0/4121141082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8043020 con 0x55f587cbe440 2024-04-17T07:14:16.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.937+0000 7fd3ef000640 1 -- 172.21.15.86:0/4121141082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3e802ca60 con 0x55f587cbe440 2024-04-17T07:14:16.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.937+0000 7fd3ef000640 1 -- 172.21.15.86:0/4121141082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80046c0 con 0x55f587cbe440 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.937+0000 7fd3f7400640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3e40a3c30 0x7fd3e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd3f7400640 1 -- 172.21.15.86:0/2790392986 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd3f00d0b50 con 0x7fd4001041c0 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd3fe800640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3e40a3c30 0x7fd3e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd407612740 1 -- 172.21.15.86:0/4121141082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 msgr2=0x55f587cc08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd407612740 1 --2- 172.21.15.86:0/4121141082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc08d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8036040 tx=0x7fd3e8005e80 comp rx=0 tx=0).stop 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd407612740 1 -- 172.21.15.86:0/4121141082 shutdown_connections 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd407612740 1 --2- 172.21.15.86:0/4121141082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc08d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd407612740 1 --2- 172.21.15.86:0/4121141082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cbddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd407612740 1 --2- 172.21.15.86:0/4121141082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb88e0 0x55f587cb8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd407612740 1 -- 172.21.15.86:0/4121141082 >> 172.21.15.86:0/4121141082 conn(0x55f587cb3e00 msgr2=0x55f587cb4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd407612740 1 -- 172.21.15.86:0/4121141082 shutdown_connections 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd3fe800640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3e40a3c30 0x7fd3e40a60f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55f587cb9b30 tx=0x7fd3f80083f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.938+0000 7fd407612740 1 -- 172.21.15.86:0/4121141082 wait complete. 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.939+0000 7fd407612740 1 -- start start 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.939+0000 7fd407612740 1 Processor -- start 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.939+0000 7fd407612740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd407612740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cc32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd407612740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cbe440 0x55f587cca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd407612740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f587cc1390 con 0x55f587cb88e0 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd407612740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f587bd9590 con 0x55f587cb92d0 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd407612740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f587cc1510 con 0x55f587cbe440 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd407612740 10 client.0 _ll_register_callbacks cb 0x55f587931ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:16.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3fde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3fde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54860/0 (socket says 172.21.15.86:54860) 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3fde00640 1 -- 172.21.15.86:0/114625299 learned_addr learned my addr 172.21.15.86:0/114625299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3fe800640 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cbe440 0x55f587cca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3fde00640 1 -- 172.21.15.86:0/114625299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cbe440 msgr2=0x55f587cca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3f5400640 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cc32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3fde00640 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cbe440 0x55f587cca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3fde00640 1 -- 172.21.15.86:0/114625299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 msgr2=0x55f587cc32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3fde00640 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cc32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3fde00640 1 -- 172.21.15.86:0/114625299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5879fc6b0 con 0x55f587cb88e0 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3fe800640 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cbe440 0x55f587cca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.940+0000 7fd3f5400640 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cc32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.941+0000 7fd3fde00640 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc2d90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd3f009cad0 tx=0x7fd3f009c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a569f77e0a49bb49 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.941+0000 7fd3e3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:16.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.941+0000 7fd3e3400640 1 -- 172.21.15.86:0/114625299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f009d240 con 0x55f587cb88e0 2024-04-17T07:14:16.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.941+0000 7fd3e3400640 1 -- 172.21.15.86:0/114625299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd3f000fd00 con 0x55f587cb88e0 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.941+0000 7fd407612740 1 -- 172.21.15.86:0/114625299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3e802ca60 con 0x55f587cb88e0 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.941+0000 7fd3e3400640 1 -- 172.21.15.86:0/114625299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd3f009cb40 con 0x55f587cb88e0 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.941+0000 7fd407612740 10 client.15369 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.941+0000 7fd407612740 1 -- 172.21.15.86:0/114625299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f587ccd6f0 con 0x55f587cb88e0 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.941+0000 7fd3e3400640 1 -- 172.21.15.86:0/114625299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3e802ca60 con 0x55f587cb88e0 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.942+0000 7fd3e3400640 1 -- 172.21.15.86:0/114625299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7fd3f00a5290 con 0x55f587cb88e0 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.942+0000 7fd407612740 10 client.15369 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.942+0000 7fd407612740 10 client.15369 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.942+0000 7fd407612740 20 client.15369 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.942+0000 7fd407612740 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f587ccfd30 0x55f587cd11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.942+0000 7fd407612740 4 client.15369 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.942+0000 7fd407612740 1 -- 172.21.15.86:0/114625299 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f587c2d3a0 con 0x55f587ccfd30 2024-04-17T07:14:16.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.943+0000 7fd3f5400640 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f587ccfd30 0x55f587cd11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.944+0000 7fd3e3400640 10 client.15369 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.944+0000 7fd3f5400640 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f587ccfd30 0x55f587cd11a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f88f9629d35cb73d server_cookie=9c7954df51835327 in_seq=0 out_seq=0 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd3e3400640 1 -- 172.21.15.86:0/114625299 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f587c2d3a0 con 0x55f587ccfd30 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd3e3400640 10 client.15369 handle_command_reply: tid=0 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd407612740 1 client.15369 shutdown 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd407612740 1 -- 172.21.15.86:0/114625299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 msgr2=0x55f587cc2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd407612740 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc2d90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd3f009cad0 tx=0x7fd3f009c680 comp rx=0 tx=0).stop 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd407612740 1 -- 172.21.15.86:0/114625299 shutdown_connections 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd407612740 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cbe440 0x55f587cca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd407612740 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f587ccfd30 0x55f587cd11a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd407612740 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cc32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd407612740 1 --2- 172.21.15.86:0/114625299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc2d90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd3e3400640 0 client.15369 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:14:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.946+0000 7fd407612740 1 -- 172.21.15.86:0/114625299 >> 172.21.15.86:0/114625299 conn(0x55f587cb3e00 msgr2=0x55f587cb3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.947+0000 7fd407612740 1 -- 172.21.15.86:0/114625299 shutdown_connections 2024-04-17T07:14:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.947+0000 7fd407612740 1 -- 172.21.15.86:0/114625299 wait complete. 2024-04-17T07:14:16.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.947+0000 7fd407612740 20 client.15369 trim_cache size 0 max 16384 2024-04-17T07:14:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.966+0000 7fd407612740 1 Processor -- start 2024-04-17T07:14:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.967+0000 7fd407612740 1 -- start start 2024-04-17T07:14:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.967+0000 7fd407612740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.967+0000 7fd407612740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb92d0 0x55f587cca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.967+0000 7fd407612740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cbe440 0x55f587cba790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.967+0000 7fd407612740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f587ccf4e0 con 0x55f587cb88e0 2024-04-17T07:14:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.967+0000 7fd407612740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f587ccfbb0 con 0x55f587cbe440 2024-04-17T07:14:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.967+0000 7fd407612740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f587cd0280 con 0x55f587cb92d0 2024-04-17T07:14:16.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.967+0000 7fd3fde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.967+0000 7fd3fde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54862/0 (socket says 172.21.15.86:54862) 2024-04-17T07:14:16.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.967+0000 7fd3fde00640 1 -- 172.21.15.86:0/2538645911 learned_addr learned my addr 172.21.15.86:0/2538645911 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3fde00640 1 -- 172.21.15.86:0/2538645911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb92d0 msgr2=0x55f587cca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3fde00640 1 --2- 172.21.15.86:0/2538645911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb92d0 0x55f587cca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3fde00640 1 -- 172.21.15.86:0/2538645911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cbe440 msgr2=0x55f587cba790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3fe800640 1 --2- 172.21.15.86:0/2538645911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cbe440 0x55f587cba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3fde00640 1 --2- 172.21.15.86:0/2538645911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cbe440 0x55f587cba790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3fde00640 1 -- 172.21.15.86:0/2538645911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f587cc3d80 con 0x55f587cb88e0 2024-04-17T07:14:16.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3fe800640 1 --2- 172.21.15.86:0/2538645911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cbe440 0x55f587cba790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:16.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3fde00640 1 --2- 172.21.15.86:0/2538645911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc0830 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0004790 tx=0x7fd3f009b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6c53c5a62ee80b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3ee600640 1 -- 172.21.15.86:0/2538645911 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f009c4a0 con 0x55f587cb88e0 2024-04-17T07:14:16.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3ee600640 1 -- 172.21.15.86:0/2538645911 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3f00d0e50 con 0x55f587cb88e0 2024-04-17T07:14:16.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.968+0000 7fd3ee600640 1 -- 172.21.15.86:0/2538645911 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0098bc0 con 0x55f587cb88e0 2024-04-17T07:14:16.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.969+0000 7fd407612740 1 -- 172.21.15.86:0/2538645911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 msgr2=0x55f587cc0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.969+0000 7fd407612740 1 --2- 172.21.15.86:0/2538645911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc0830 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0004790 tx=0x7fd3f009b050 comp rx=0 tx=0).stop 2024-04-17T07:14:16.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.969+0000 7fd407612740 1 -- 172.21.15.86:0/2538645911 shutdown_connections 2024-04-17T07:14:16.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.969+0000 7fd407612740 1 --2- 172.21.15.86:0/2538645911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cbe440 0x55f587cba790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.969+0000 7fd407612740 1 --2- 172.21.15.86:0/2538645911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb92d0 0x55f587cca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.969+0000 7fd407612740 1 --2- 172.21.15.86:0/2538645911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cb88e0 0x55f587cc0830 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.969+0000 7fd407612740 1 -- 172.21.15.86:0/2538645911 >> 172.21.15.86:0/2538645911 conn(0x55f587cb3e00 msgr2=0x55f587caf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.969+0000 7fd407612740 1 -- 172.21.15.86:0/2538645911 shutdown_connections 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.969+0000 7fd407612740 1 -- 172.21.15.86:0/2538645911 wait complete. 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.969+0000 7fd407612740 1 -- start start 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.970+0000 7fd407612740 1 Processor -- start 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.970+0000 7fd407612740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb88e0 0x55f587cc5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.970+0000 7fd407612740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cc6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.970+0000 7fd407612740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.970+0000 7fd407612740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f587cd1020 con 0x55f587cbe440 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.970+0000 7fd407612740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f587cd0950 con 0x55f587cb92d0 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.970+0000 7fd407612740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f587cd16f0 con 0x55f587cb88e0 2024-04-17T07:14:16.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.970+0000 7fd407612740 10 client.0 _ll_register_callbacks cb 0x7fd3f00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.970+0000 7fd3fe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3fe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54870/0 (socket says 172.21.15.86:54870) 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3fe800640 1 -- 172.21.15.86:0/2643598314 learned_addr learned my addr 172.21.15.86:0/2643598314 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3f5400640 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cc6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3fde00640 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb88e0 0x55f587cc5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3fe800640 1 -- 172.21.15.86:0/2643598314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb88e0 msgr2=0x55f587cc5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3fe800640 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb88e0 0x55f587cc5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3fe800640 1 -- 172.21.15.86:0/2643598314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 msgr2=0x55f587cc6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3fe800640 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cc6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3fe800640 1 -- 172.21.15.86:0/2643598314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f587cc9860 con 0x55f587cbe440 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3fe800640 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc9270 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8010e00 tx=0x7fd3f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0832982c43bc86 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.971+0000 7fd3e3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:16.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd3e3400640 1 -- 172.21.15.86:0/2643598314 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f801ddb0 con 0x55f587cbe440 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd3e3400640 1 -- 172.21.15.86:0/2643598314 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd3f801b480 con 0x55f587cbe440 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd407612740 1 -- 172.21.15.86:0/2643598314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3f00d0e50 con 0x55f587cbe440 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd3e3400640 1 -- 172.21.15.86:0/2643598314 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd3f8004b40 con 0x55f587cbe440 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd407612740 10 client.15375 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd407612740 1 -- 172.21.15.86:0/2643598314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f587d3e860 con 0x55f587cbe440 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd3e3400640 1 -- 172.21.15.86:0/2643598314 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3f00d0e50 con 0x55f587cbe440 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd3e3400640 1 -- 172.21.15.86:0/2643598314 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7fd3f8024070 con 0x55f587cbe440 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd407612740 10 client.15375 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd407612740 10 client.15375 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd407612740 20 client.15375 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd407612740 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f587d43530 0x55f587d42240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd407612740 4 client.15375 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:14:16.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.972+0000 7fd407612740 1 -- 172.21.15.86:0/2643598314 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "status"}) -- 0x55f587c27210 con 0x55f587d43530 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.973+0000 7fd3fde00640 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f587d43530 0x55f587d42240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.974+0000 7fd3fde00640 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f587d43530 0x55f587d42240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87be958dc3a076f server_cookie=2c6200d87ac3b8e8 in_seq=0 out_seq=0 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.974+0000 7fd3e3400640 10 client.15375 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd3e3400640 1 -- 172.21.15.86:0/2643598314 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f587c27210 con 0x55f587d43530 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd3e3400640 10 client.15375 handle_command_reply: tid=0 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 client.15375 shutdown 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 -- 172.21.15.86:0/2643598314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 msgr2=0x55f587cc9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc9270 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8010e00 tx=0x7fd3f8004a40 comp rx=0 tx=0).stop 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 -- 172.21.15.86:0/2643598314 shutdown_connections 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f587cbe440 0x55f587cc9270 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f587d43530 0x55f587d42240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f587cb92d0 0x55f587cc6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 --2- 172.21.15.86:0/2643598314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f587cb88e0 0x55f587cc5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd3e3400640 0 client.15375 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 -- 172.21.15.86:0/2643598314 >> 172.21.15.86:0/2643598314 conn(0x55f587cb3e00 msgr2=0x55f587cba790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 -- 172.21.15.86:0/2643598314 shutdown_connections 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 1 -- 172.21.15.86:0/2643598314 wait complete. 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.975+0000 7fd407612740 20 client.15375 trim_cache size 0 max 16384 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 34460, 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:14:16.977 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:14:16.978 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:14:16.978 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 110.28083287600001, 2024-04-17T07:14:16.978 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-04-17T07:14:16.978 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-17T07:14:16.978 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 91, 2024-04-17T07:14:16.978 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 164.26193007099999 2024-04-17T07:14:16.978 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 -- 172.21.15.86:0/2790392986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3e40a3c30 msgr2=0x7fd3e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3e40a3c30 0x7fd3e40a60f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55f587cb9b30 tx=0x7fd3f80083f0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 -- 172.21.15.86:0/2790392986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4001041c0 msgr2=0x7fd40019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4001041c0 0x7fd40019a380 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000c970 tx=0x7fd3f000ce30 comp rx=0 tx=0).stop 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 -- 172.21.15.86:0/2790392986 shutdown_connections 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3e40a3c30 0x7fd3e40a60f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd400105d70 0x7fd4001a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4001053b0 0x7fd40019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 --2- 172.21.15.86:0/2790392986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4001041c0 0x7fd40019a380 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 -- 172.21.15.86:0/2790392986 >> 172.21.15.86:0/2790392986 conn(0x7fd4000ff750 msgr2=0x7fd400108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 -- 172.21.15.86:0/2790392986 shutdown_connections 2024-04-17T07:14:16.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:16.976+0000 7fd404a00640 1 -- 172.21.15.86:0/2790392986 wait complete. 2024-04-17T07:14:16.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-17T07:14:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.206+0000 7faebb200640 1 -- 172.21.15.86:0/1273150747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc1084a0 msgr2=0x7faebc1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.206+0000 7faebb200640 1 --2- 172.21.15.86:0/1273150747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc1084a0 0x7faebc1088c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faeac009900 tx=0x7faeac02bc40 comp rx=0 tx=0).stop 2024-04-17T07:14:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.207+0000 7faebb200640 1 -- 172.21.15.86:0/1273150747 shutdown_connections 2024-04-17T07:14:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.207+0000 7faebb200640 1 --2- 172.21.15.86:0/1273150747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faebc10a050 0x7faebc1108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.207+0000 7faebb200640 1 --2- 172.21.15.86:0/1273150747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faebc109690 0x7faebc109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.207+0000 7faebb200640 1 --2- 172.21.15.86:0/1273150747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc1084a0 0x7faebc1088c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.207+0000 7faebb200640 1 -- 172.21.15.86:0/1273150747 >> 172.21.15.86:0/1273150747 conn(0x7faebc077430 msgr2=0x7faebc079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.207+0000 7faebb200640 1 -- 172.21.15.86:0/1273150747 shutdown_connections 2024-04-17T07:14:17.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.207+0000 7faebb200640 1 -- 172.21.15.86:0/1273150747 wait complete. 2024-04-17T07:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faebb200640 1 Processor -- start 2024-04-17T07:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faebb200640 1 -- start start 2024-04-17T07:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faebb200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faebc1084a0 0x7faebc19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faebb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc109690 0x7faebc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faebb200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faebc10a050 0x7faebc1a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faebb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faebc113fe0 con 0x7faebc109690 2024-04-17T07:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faebb200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faebc113e60 con 0x7faebc1084a0 2024-04-17T07:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faebb200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faebc114160 con 0x7faebc10a050 2024-04-17T07:14:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faeb1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc109690 0x7faebc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faeb1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc109690 0x7faebc19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54886/0 (socket says 172.21.15.86:54886) 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.208+0000 7faeb1400640 1 -- 172.21.15.86:0/3413069505 learned_addr learned my addr 172.21.15.86:0/3413069505 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeb9e00640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faebc1084a0 0x7faebc19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeba800640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faebc10a050 0x7faebc1a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeb1400640 1 -- 172.21.15.86:0/3413069505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faebc10a050 msgr2=0x7faebc1a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeb1400640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faebc10a050 0x7faebc1a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeb1400640 1 -- 172.21.15.86:0/3413069505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faebc1084a0 msgr2=0x7faebc19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeb1400640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faebc1084a0 0x7faebc19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeb1400640 1 -- 172.21.15.86:0/3413069505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faebc1a61d0 con 0x7faebc109690 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeba800640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faebc10a050 0x7faebc1a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeb1400640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc109690 0x7faebc19eaf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faea400c920 tx=0x7faea400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:17.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeb3400640 1 -- 172.21.15.86:0/3413069505 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4007c70 con 0x7faebc109690 2024-04-17T07:14:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeb3400640 1 -- 172.21.15.86:0/3413069505 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faea4007e10 con 0x7faebc109690 2024-04-17T07:14:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faebb200640 1 -- 172.21.15.86:0/3413069505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faebc1a64c0 con 0x7faebc109690 2024-04-17T07:14:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.209+0000 7faeb3400640 1 -- 172.21.15.86:0/3413069505 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4018820 con 0x7faebc109690 2024-04-17T07:14:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.210+0000 7faebb200640 1 -- 172.21.15.86:0/3413069505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faebc1a6a60 con 0x7faebc109690 2024-04-17T07:14:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.213+0000 7faec3852740 1 Processor -- start 2024-04-17T07:14:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.213+0000 7faec3852740 1 -- start start 2024-04-17T07:14:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.213+0000 7faeb3400640 1 -- 172.21.15.86:0/3413069505 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faea40189c0 con 0x7faebc109690 2024-04-17T07:14:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.213+0000 7faec3852740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df5af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.213+0000 7faec3852740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df5ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.213+0000 7faec3852740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df62ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.213+0000 7faec3852740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d0dec1980 con 0x563d0df5aaf0 2024-04-17T07:14:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.213+0000 7faec3852740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563d0de68220 con 0x563d0df60650 2024-04-17T07:14:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.213+0000 7faec3852740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d0debd940 con 0x563d0df5b4e0 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df5af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df5af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54900/0 (socket says 172.21.15.86:54900) 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeba800640 1 -- 172.21.15.86:0/2335739200 learned_addr learned my addr 172.21.15.86:0/2335739200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeb9e00640 1 --2- 172.21.15.86:0/2335739200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df5ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeb1400640 1 --2- 172.21.15.86:0/2335739200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df62ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeba800640 1 -- 172.21.15.86:0/2335739200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 msgr2=0x563d0df5ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeba800640 1 --2- 172.21.15.86:0/2335739200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df5ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeba800640 1 -- 172.21.15.86:0/2335739200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 msgr2=0x563d0df62ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeba800640 1 --2- 172.21.15.86:0/2335739200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df62ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeba800640 1 -- 172.21.15.86:0/2335739200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d0dbd42d0 con 0x563d0df5aaf0 2024-04-17T07:14:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.214+0000 7faeb1400640 1 --2- 172.21.15.86:0/2335739200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df62ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.215+0000 7faeba800640 1 --2- 172.21.15.86:0/2335739200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df5af10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faeb4008e60 tx=0x7faeb400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756e2e0dd36c3abf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:17.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.215+0000 7faeb3400640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faea00a3c30 0x7faea00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.215+0000 7faeab000640 1 -- 172.21.15.86:0/2335739200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb400f830 con 0x563d0df5aaf0 2024-04-17T07:14:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.215+0000 7faeab000640 1 -- 172.21.15.86:0/2335739200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faeb4018480 con 0x563d0df5aaf0 2024-04-17T07:14:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.215+0000 7faeab000640 1 -- 172.21.15.86:0/2335739200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb400e5b0 con 0x563d0df5aaf0 2024-04-17T07:14:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.215+0000 7faeb3400640 1 -- 172.21.15.86:0/3413069505 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faea40d0980 con 0x7faebc109690 2024-04-17T07:14:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.215+0000 7faeb9e00640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faea00a3c30 0x7faea00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faec3852740 1 -- 172.21.15.86:0/2335739200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 msgr2=0x563d0df5af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faec3852740 1 --2- 172.21.15.86:0/2335739200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df5af10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faeb4008e60 tx=0x7faeb400d710 comp rx=0 tx=0).stop 2024-04-17T07:14:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faec3852740 1 -- 172.21.15.86:0/2335739200 shutdown_connections 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faec3852740 1 --2- 172.21.15.86:0/2335739200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df62ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faec3852740 1 --2- 172.21.15.86:0/2335739200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df5ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faec3852740 1 --2- 172.21.15.86:0/2335739200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df5af10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faec3852740 1 -- 172.21.15.86:0/2335739200 >> 172.21.15.86:0/2335739200 conn(0x563d0df56060 msgr2=0x563d0df56460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faeb9e00640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faea00a3c30 0x7faea00a60f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faeac006010 tx=0x7faeac02c810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faec3852740 1 -- 172.21.15.86:0/2335739200 shutdown_connections 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faec3852740 1 -- 172.21.15.86:0/2335739200 wait complete. 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.216+0000 7faec3852740 1 -- start start 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faec3852740 1 Processor -- start 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faec3852740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5aaf0 0x563d0df65060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faec3852740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df5b4e0 0x563d0df655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faec3852740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df60650 0x563d0df6c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faec3852740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d0df63600 con 0x563d0df60650 2024-04-17T07:14:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faec3852740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563d0de7b590 con 0x563d0df5b4e0 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faec3852740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d0df63780 con 0x563d0df5aaf0 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faec3852740 10 client.0 _ll_register_callbacks cb 0x563d0dbd3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faeba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df60650 0x563d0df6c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faeba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df60650 0x563d0df6c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54914/0 (socket says 172.21.15.86:54914) 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.217+0000 7faeba800640 1 -- 172.21.15.86:0/2627480463 learned_addr learned my addr 172.21.15.86:0/2627480463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faeb1400640 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df5b4e0 0x563d0df655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faeb9e00640 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5aaf0 0x563d0df65060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faeba800640 1 -- 172.21.15.86:0/2627480463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5aaf0 msgr2=0x563d0df65060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faeba800640 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5aaf0 0x563d0df65060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faeba800640 1 -- 172.21.15.86:0/2627480463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df5b4e0 msgr2=0x563d0df655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faeba800640 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df5b4e0 0x563d0df655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faeba800640 1 -- 172.21.15.86:0/2627480463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d0dc9e6b0 con 0x563d0df60650 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faeb9e00640 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5aaf0 0x563d0df65060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faeb1400640 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df5b4e0 0x563d0df655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faeba800640 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df60650 0x563d0df6c5e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faeb400dc10 tx=0x7faeb4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d676dd64d1bd980 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7fae9f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7fae9f400640 1 -- 172.21.15.86:0/2627480463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4027060 con 0x563d0df60650 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7fae9f400640 1 -- 172.21.15.86:0/2627480463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faeb4018980 con 0x563d0df60650 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.218+0000 7faec3852740 1 -- 172.21.15.86:0/2627480463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeb4018480 con 0x563d0df60650 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7fae9f400640 1 -- 172.21.15.86:0/2627480463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faeb402e050 con 0x563d0df60650 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7faec3852740 10 client.15387 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7faec3852740 1 -- 172.21.15.86:0/2627480463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d0df6f980 con 0x563d0df60650 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7fae9f400640 1 -- 172.21.15.86:0/2627480463 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faeb4018480 con 0x563d0df60650 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7fae9f400640 1 -- 172.21.15.86:0/2627480463 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7faeb4005340 con 0x563d0df60650 2024-04-17T07:14:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7faec3852740 10 client.15387 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7faec3852740 10 client.15387 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:14:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7faec3852740 20 client.15387 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7faec3852740 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563d0df71c60 0x563d0df730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7faec3852740 4 client.15387 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:14:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.219+0000 7faec3852740 1 -- 172.21.15.86:0/2627480463 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d0decf3a0 con 0x563d0df71c60 2024-04-17T07:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.220+0000 7faeb9e00640 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563d0df71c60 0x563d0df730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.221+0000 7faeb9e00640 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563d0df71c60 0x563d0df730d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf8e35d664df54b server_cookie=cf1ca84f086c78c5 in_seq=0 out_seq=0 2024-04-17T07:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.221+0000 7fae9f400640 10 client.15387 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7fae9f400640 1 -- 172.21.15.86:0/2627480463 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d0decf3a0 con 0x563d0df71c60 2024-04-17T07:14:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7fae9f400640 10 client.15387 handle_command_reply: tid=0 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 client.15387 shutdown 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 -- 172.21.15.86:0/2627480463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df60650 msgr2=0x563d0df6c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df60650 0x563d0df6c5e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faeb400dc10 tx=0x7faeb4004f90 comp rx=0 tx=0).stop 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 -- 172.21.15.86:0/2627480463 shutdown_connections 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563d0df71c60 0x563d0df730d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df60650 0x563d0df6c5e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7fae9f400640 0 client.15387 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df5b4e0 0x563d0df655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 --2- 172.21.15.86:0/2627480463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5aaf0 0x563d0df65060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 -- 172.21.15.86:0/2627480463 >> 172.21.15.86:0/2627480463 conn(0x563d0df56060 msgr2=0x563d0df553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 -- 172.21.15.86:0/2627480463 shutdown_connections 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 1 -- 172.21.15.86:0/2627480463 wait complete. 2024-04-17T07:14:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.223+0000 7faec3852740 20 client.15387 trim_cache size 0 max 16384 2024-04-17T07:14:17.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.244+0000 7faec3852740 1 Processor -- start 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.244+0000 7faec3852740 1 -- start start 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faec3852740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df56440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faec3852740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df6c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faec3852740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df54b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faec3852740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d0df71410 con 0x563d0df5aaf0 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faec3852740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563d0df71ae0 con 0x563d0df60650 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faec3852740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d0df721b0 con 0x563d0df5b4e0 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faeb9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df56440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faeb9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df56440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54930/0 (socket says 172.21.15.86:54930) 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faeb9e00640 1 -- 172.21.15.86:0/3257803199 learned_addr learned my addr 172.21.15.86:0/3257803199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faeb1400640 1 --2- 172.21.15.86:0/3257803199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df6c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faeba800640 1 --2- 172.21.15.86:0/3257803199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df54b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faeb9e00640 1 -- 172.21.15.86:0/3257803199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 msgr2=0x563d0df6c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:17.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faeb9e00640 1 --2- 172.21.15.86:0/3257803199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df6c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faeb9e00640 1 -- 172.21.15.86:0/3257803199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 msgr2=0x563d0df54b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:17.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.245+0000 7faeb9e00640 1 --2- 172.21.15.86:0/3257803199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df54b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.246+0000 7faeb9e00640 1 -- 172.21.15.86:0/3257803199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d0df47900 con 0x563d0df5aaf0 2024-04-17T07:14:17.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.246+0000 7faeba800640 1 --2- 172.21.15.86:0/3257803199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df54b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:17.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.246+0000 7faeb9e00640 1 --2- 172.21.15.86:0/3257803199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df56440 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faeac000c00 tx=0x7faeac043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37bff65358b3714 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:17.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.246+0000 7faeaa600640 1 -- 172.21.15.86:0/3257803199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac051b50 con 0x563d0df5aaf0 2024-04-17T07:14:17.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.246+0000 7faeaa600640 1 -- 172.21.15.86:0/3257803199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faeac057070 con 0x563d0df5aaf0 2024-04-17T07:14:17.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.246+0000 7faeaa600640 1 -- 172.21.15.86:0/3257803199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac051730 con 0x563d0df5aaf0 2024-04-17T07:14:17.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.247+0000 7faec3852740 1 -- 172.21.15.86:0/3257803199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 msgr2=0x563d0df56440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:17.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.247+0000 7faec3852740 1 --2- 172.21.15.86:0/3257803199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df56440 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faeac000c00 tx=0x7faeac043000 comp rx=0 tx=0).stop 2024-04-17T07:14:17.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.247+0000 7faec3852740 1 -- 172.21.15.86:0/3257803199 shutdown_connections 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.247+0000 7faec3852740 1 --2- 172.21.15.86:0/3257803199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df54b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.247+0000 7faec3852740 1 --2- 172.21.15.86:0/3257803199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df6c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.247+0000 7faec3852740 1 --2- 172.21.15.86:0/3257803199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df56440 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.247+0000 7faec3852740 1 -- 172.21.15.86:0/3257803199 >> 172.21.15.86:0/3257803199 conn(0x563d0df56060 msgr2=0x563d0df51230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.247+0000 7faec3852740 1 -- 172.21.15.86:0/3257803199 shutdown_connections 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.247+0000 7faec3852740 1 -- 172.21.15.86:0/3257803199 wait complete. 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.248+0000 7faec3852740 1 -- start start 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.248+0000 7faec3852740 1 Processor -- start 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.248+0000 7faec3852740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df67920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.248+0000 7faec3852740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df67e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.248+0000 7faec3852740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df6ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.248+0000 7faec3852740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d0df72f50 con 0x563d0df5aaf0 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.248+0000 7faec3852740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563d0df72880 con 0x563d0df60650 2024-04-17T07:14:17.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.248+0000 7faec3852740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d0df73620 con 0x563d0df5b4e0 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.248+0000 7faec3852740 10 client.0 _ll_register_callbacks cb 0x7faeb4008cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.249+0000 7faeb9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df67920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.249+0000 7faeb9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df67920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54944/0 (socket says 172.21.15.86:54944) 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.249+0000 7faeb9e00640 1 -- 172.21.15.86:0/1586388684 learned_addr learned my addr 172.21.15.86:0/1586388684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.249+0000 7faeb9e00640 1 -- 172.21.15.86:0/1586388684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 msgr2=0x563d0df67e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.249+0000 7faeb9e00640 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df67e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.249+0000 7faeb9e00640 1 -- 172.21.15.86:0/1586388684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 msgr2=0x563d0df6ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.249+0000 7faeb9e00640 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df6ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.249+0000 7faeb9e00640 1 -- 172.21.15.86:0/1586388684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d0df6b470 con 0x563d0df5aaf0 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.249+0000 7faeb9e00640 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df67920 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faeac009900 tx=0x7faeac005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca0d524b38ffb20 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.249+0000 7fae9f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.250+0000 7fae9f400640 1 -- 172.21.15.86:0/1586388684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac060060 con 0x563d0df5aaf0 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.250+0000 7fae9f400640 1 -- 172.21.15.86:0/1586388684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faeac0046c0 con 0x563d0df5aaf0 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.250+0000 7faec3852740 1 -- 172.21.15.86:0/1586388684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeac057070 con 0x563d0df5aaf0 2024-04-17T07:14:17.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.250+0000 7fae9f400640 1 -- 172.21.15.86:0/1586388684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faeac0048c0 con 0x563d0df5aaf0 2024-04-17T07:14:17.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.250+0000 7faec3852740 10 client.15393 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:17.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.250+0000 7faec3852740 1 -- 172.21.15.86:0/1586388684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d0df6c6e0 con 0x563d0df5aaf0 2024-04-17T07:14:17.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.250+0000 7fae9f400640 1 -- 172.21.15.86:0/1586388684 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faeac057070 con 0x563d0df5aaf0 2024-04-17T07:14:17.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.250+0000 7fae9f400640 1 -- 172.21.15.86:0/1586388684 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7faeac003cc0 con 0x563d0df5aaf0 2024-04-17T07:14:17.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.250+0000 7faec3852740 10 client.15393 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:17.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.251+0000 7faec3852740 10 client.15393 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:14:17.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.251+0000 7faec3852740 20 client.15393 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:17.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.251+0000 7faec3852740 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563d0dfdbf10 0x563d0dfdac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:17.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.251+0000 7faec3852740 4 client.15393 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:14:17.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.251+0000 7faec3852740 1 -- 172.21.15.86:0/1586388684 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "status"}) -- 0x563d0dec9210 con 0x563d0dfdbf10 2024-04-17T07:14:17.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.251+0000 7faeb1400640 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563d0dfdbf10 0x563d0dfdac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.252+0000 7faeb1400640 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563d0dfdbf10 0x563d0dfdac20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e605340b273cb9b server_cookie=edd01a763bb81e in_seq=0 out_seq=0 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.252+0000 7fae9f400640 10 client.15393 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7fae9f400640 1 -- 172.21.15.86:0/1586388684 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563d0dec9210 con 0x563d0dfdbf10 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7fae9f400640 10 client.15393 handle_command_reply: tid=0 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7faec3852740 1 client.15393 shutdown 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7faec3852740 1 -- 172.21.15.86:0/1586388684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 msgr2=0x563d0df67920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7faec3852740 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df67920 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faeac009900 tx=0x7faeac005200 comp rx=0 tx=0).stop 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7faec3852740 1 -- 172.21.15.86:0/1586388684 shutdown_connections 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7faec3852740 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563d0dfdbf10 0x563d0dfdac20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7faec3852740 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d0df60650 0x563d0df6ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7faec3852740 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d0df5b4e0 0x563d0df67e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7fae9f400640 0 client.15393 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7faec3852740 1 --2- 172.21.15.86:0/1586388684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0df5aaf0 0x563d0df67920 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7faec3852740 1 -- 172.21.15.86:0/1586388684 >> 172.21.15.86:0/1586388684 conn(0x563d0df56060 msgr2=0x563d0df56440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.253+0000 7faec3852740 1 -- 172.21.15.86:0/1586388684 shutdown_connections 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faec3852740 1 -- 172.21.15.86:0/1586388684 wait complete. 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faec3852740 20 client.15393 trim_cache size 0 max 16384 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 34490, 2024-04-17T07:14:17.256 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:14:17.257 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:14:17.257 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:14:17.257 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 141.786588341, 2024-04-17T07:14:17.257 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-04-17T07:14:17.257 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-17T07:14:17.257 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 91, 2024-04-17T07:14:17.257 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 148.33132063100001 2024-04-17T07:14:17.257 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faebb200640 1 -- 172.21.15.86:0/3413069505 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faea00a3c30 msgr2=0x7faea00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faebb200640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faea00a3c30 0x7faea00a60f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faeac006010 tx=0x7faeac02c810 comp rx=0 tx=0).stop 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faebb200640 1 -- 172.21.15.86:0/3413069505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc109690 msgr2=0x7faebc19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faebb200640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc109690 0x7faebc19eaf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faea400c920 tx=0x7faea400cde0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faebb200640 1 -- 172.21.15.86:0/3413069505 shutdown_connections 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faebb200640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faea00a3c30 0x7faea00a60f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faebb200640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faebc10a050 0x7faebc1a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faebb200640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faebc109690 0x7faebc19eaf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faebb200640 1 --2- 172.21.15.86:0/3413069505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faebc1084a0 0x7faebc19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.254+0000 7faebb200640 1 -- 172.21.15.86:0/3413069505 >> 172.21.15.86:0/3413069505 conn(0x7faebc077430 msgr2=0x7faebc10c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:17.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.255+0000 7faebb200640 1 -- 172.21.15.86:0/3413069505 shutdown_connections 2024-04-17T07:14:17.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:17.255+0000 7faebb200640 1 -- 172.21.15.86:0/3413069505 wait complete. 2024-04-17T07:14:17.273 INFO:tasks.cephadm:Setup complete, yielding 2024-04-17T07:14:17.273 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-17T07:14:17.283 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-17T07:14:17.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- bash -c 'ceph orch status' 2024-04-17T07:14:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:17 smithi098 ceph-mon[35418]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:17 smithi103 ceph-mon[35495]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:17.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:17 smithi086 ceph-mon[31733]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:14:20.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.003+0000 7fa07fc00700 1 -- 172.21.15.86:0/371321522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0780754b0 msgr2=0x7fa078075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.003+0000 7fa07fc00700 1 --2- 172.21.15.86:0/371321522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0780754b0 0x7fa078075930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa060009990 tx=0x7fa060009ca0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.003+0000 7fa07fc00700 1 -- 172.21.15.86:0/371321522 shutdown_connections 2024-04-17T07:14:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.003+0000 7fa07fc00700 1 --2- 172.21.15.86:0/371321522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa07810f0a0 0x7fa078111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.003+0000 7fa07fc00700 1 --2- 172.21.15.86:0/371321522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa078076a80 0x7fa078074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.003+0000 7fa07fc00700 1 --2- 172.21.15.86:0/371321522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0780754b0 0x7fa078075930 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.003+0000 7fa07fc00700 1 -- 172.21.15.86:0/371321522 >> 172.21.15.86:0/371321522 conn(0x7fa0780fe020 msgr2=0x7fa0781004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.003+0000 7fa07fc00700 1 -- 172.21.15.86:0/371321522 shutdown_connections 2024-04-17T07:14:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.003+0000 7fa07fc00700 1 -- 172.21.15.86:0/371321522 wait complete. 2024-04-17T07:14:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.004+0000 7fa07fc00700 1 Processor -- start 2024-04-17T07:14:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.004+0000 7fa07fc00700 1 -- start start 2024-04-17T07:14:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.004+0000 7fa07fc00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0780754b0 0x7fa07819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.004+0000 7fa07fc00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa078076a80 0x7fa07819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07fc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07810f0a0 0x7fa0781a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07fc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa078114b70 con 0x7fa07810f0a0 2024-04-17T07:14:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07fc00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa0781149f0 con 0x7fa0780754b0 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07fc00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa078114cf0 con 0x7fa078076a80 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07810f0a0 0x7fa0781a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07dc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07810f0a0 0x7fa0781a6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54952/0 (socket says 172.21.15.86:54952) 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07dc00700 1 -- 172.21.15.86:0/3726159411 learned_addr learned my addr 172.21.15.86:0/3726159411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07d200700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0780754b0 0x7fa07819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa06fe00700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa078076a80 0x7fa07819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07dc00700 1 -- 172.21.15.86:0/3726159411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa078076a80 msgr2=0x7fa07819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07dc00700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa078076a80 0x7fa07819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07dc00700 1 -- 172.21.15.86:0/3726159411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0780754b0 msgr2=0x7fa07819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07dc00700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0780754b0 0x7fa07819f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07dc00700 1 -- 172.21.15.86:0/3726159411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0781a6ea0 con 0x7fa07810f0a0 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.005+0000 7fa07d200700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0780754b0 0x7fa07819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:14:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.006+0000 7fa07dc00700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07810f0a0 0x7fa0781a6800 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa070007f10 tx=0x7fa07000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.006+0000 7fa076000700 1 -- 172.21.15.86:0/3726159411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000ddd0 con 0x7fa07810f0a0 2024-04-17T07:14:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.006+0000 7fa076000700 1 -- 172.21.15.86:0/3726159411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa070004020 con 0x7fa07810f0a0 2024-04-17T07:14:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.006+0000 7fa076000700 1 -- 172.21.15.86:0/3726159411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070015660 con 0x7fa07810f0a0 2024-04-17T07:14:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.006+0000 7fa07fc00700 1 -- 172.21.15.86:0/3726159411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0781a7190 con 0x7fa07810f0a0 2024-04-17T07:14:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.007+0000 7fa07fc00700 1 -- 172.21.15.86:0/3726159411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0781a7680 con 0x7fa07810f0a0 2024-04-17T07:14:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.008+0000 7fa07fc00700 1 -- 172.21.15.86:0/3726159411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa078069740 con 0x7fa07810f0a0 2024-04-17T07:14:20.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.010+0000 7fa076000700 1 -- 172.21.15.86:0/3726159411 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa070003a00 con 0x7fa07810f0a0 2024-04-17T07:14:20.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.012+0000 7fa076000700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa05c0a3c40 0x7fa05c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:20.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.012+0000 7fa076000700 1 -- 172.21.15.86:0/3726159411 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa0700d0280 con 0x7fa07810f0a0 2024-04-17T07:14:20.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.012+0000 7fa07d200700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa05c0a3c40 0x7fa05c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:20.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.013+0000 7fa07d200700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa05c0a3c40 0x7fa05c0a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa078076e60 tx=0x7fa06800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:20.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.017+0000 7fa076000700 1 -- 172.21.15.86:0/3726159411 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa070010070 con 0x7fa07810f0a0 2024-04-17T07:14:20.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.259+0000 7fa07fc00700 1 -- 172.21.15.86:0/3726159411 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fa078062e90 con 0x7fa05c0a3c40 2024-04-17T07:14:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.261+0000 7fa076000700 1 -- 172.21.15.86:0/3726159411 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fa078062e90 con 0x7fa05c0a3c40 2024-04-17T07:14:20.260 INFO:teuthology.orchestra.run.smithi086.stdout:Backend: cephadm 2024-04-17T07:14:20.260 INFO:teuthology.orchestra.run.smithi086.stdout:Available: Yes 2024-04-17T07:14:20.260 INFO:teuthology.orchestra.run.smithi086.stdout:Paused: No 2024-04-17T07:14:20.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.263+0000 7fa07fc00700 1 -- 172.21.15.86:0/3726159411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa05c0a3c40 msgr2=0x7fa05c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:20.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.263+0000 7fa07fc00700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa05c0a3c40 0x7fa05c0a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa078076e60 tx=0x7fa06800b2c0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.263+0000 7fa07fc00700 1 -- 172.21.15.86:0/3726159411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07810f0a0 msgr2=0x7fa0781a6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:20.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.263+0000 7fa07fc00700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07810f0a0 0x7fa0781a6800 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa070007f10 tx=0x7fa07000d3b0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.263+0000 7fa07fc00700 1 -- 172.21.15.86:0/3726159411 shutdown_connections 2024-04-17T07:14:20.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.263+0000 7fa07fc00700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa05c0a3c40 0x7fa05c0a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.263+0000 7fa07fc00700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa07810f0a0 0x7fa0781a6800 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.264+0000 7fa07fc00700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0780754b0 0x7fa07819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.264+0000 7fa07fc00700 1 --2- 172.21.15.86:0/3726159411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa078076a80 0x7fa07819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:20.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.264+0000 7fa07fc00700 1 -- 172.21.15.86:0/3726159411 >> 172.21.15.86:0/3726159411 conn(0x7fa0780fe020 msgr2=0x7fa078102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:20.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.264+0000 7fa07fc00700 1 -- 172.21.15.86:0/3726159411 shutdown_connections 2024-04-17T07:14:20.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:20.264+0000 7fa07fc00700 1 -- 172.21.15.86:0/3726159411 wait complete. 2024-04-17T07:14:20.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:20 smithi098 ceph-mon[35418]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:20 smithi103 ceph-mon[35495]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:20 smithi086 ceph-mon[31733]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:21.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-17T07:14:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:14:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:22 smithi086 ceph-mon[31733]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:22.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:22 smithi086 ceph-mon[31733]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:22 smithi098 ceph-mon[35418]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:22 smithi098 ceph-mon[35418]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:22.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:22 smithi103 ceph-mon[35495]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:22.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:22 smithi103 ceph-mon[35495]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.833+0000 7f5ac4200700 1 -- 172.21.15.86:0/3999678521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5abc105eb0 msgr2=0x7f5abc1082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.833+0000 7f5ac4200700 1 --2- 172.21.15.86:0/3999678521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5abc105eb0 0x7f5abc1082a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009990 tx=0x7f5aa8009ca0 comp rx=0 tx=0).stop 2024-04-17T07:14:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.833+0000 7f5ac4200700 1 -- 172.21.15.86:0/3999678521 shutdown_connections 2024-04-17T07:14:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.833+0000 7f5ac4200700 1 --2- 172.21.15.86:0/3999678521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5abc1087e0 0x7f5abc10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.833+0000 7f5ac4200700 1 --2- 172.21.15.86:0/3999678521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc103580 0x7f5abc105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.833+0000 7f5ac4200700 1 --2- 172.21.15.86:0/3999678521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5abc105eb0 0x7f5abc1082a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.833+0000 7f5ac4200700 1 -- 172.21.15.86:0/3999678521 >> 172.21.15.86:0/3999678521 conn(0x7f5abc0fcf50 msgr2=0x7f5abc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.833+0000 7f5ac4200700 1 -- 172.21.15.86:0/3999678521 shutdown_connections 2024-04-17T07:14:23.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.833+0000 7f5ac4200700 1 -- 172.21.15.86:0/3999678521 wait complete. 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.835+0000 7f5ac4200700 1 Processor -- start 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.835+0000 7f5ac4200700 1 -- start start 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.835+0000 7f5ac4200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5abc103580 0x7f5abc198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.835+0000 7f5ac4200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5abc105eb0 0x7f5abc1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac4200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc1087e0 0x7f5abc1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac4200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc10e5c0 con 0x7f5abc103580 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac4200700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc10e440 con 0x7f5abc105eb0 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac4200700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc10e740 con 0x7f5abc1087e0 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5abc103580 0x7f5abc198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac1800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5abc103580 0x7f5abc198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54982/0 (socket says 172.21.15.86:54982) 2024-04-17T07:14:23.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac1800700 1 -- 172.21.15.86:0/2727181842 learned_addr learned my addr 172.21.15.86:0/2727181842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac0e00700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5abc105eb0 0x7f5abc1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac1800700 1 -- 172.21.15.86:0/2727181842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc1087e0 msgr2=0x7f5abc1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac2200700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc1087e0 0x7f5abc1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac1800700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc1087e0 0x7f5abc1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac1800700 1 -- 172.21.15.86:0/2727181842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5abc105eb0 msgr2=0x7f5abc1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac1800700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5abc105eb0 0x7f5abc1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac1800700 1 -- 172.21.15.86:0/2727181842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5abc1a0900 con 0x7f5abc103580 2024-04-17T07:14:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.836+0000 7f5ac2200700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc1087e0 0x7f5abc1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.837+0000 7f5ac1800700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5abc103580 0x7f5abc198c80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00d980 tx=0x7f5aac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:23.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.837+0000 7f5aba000700 1 -- 172.21.15.86:0/2727181842 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac017070 con 0x7f5abc103580 2024-04-17T07:14:23.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.837+0000 7f5aba000700 1 -- 172.21.15.86:0/2727181842 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aac004830 con 0x7f5abc103580 2024-04-17T07:14:23.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.837+0000 7f5ac4200700 1 -- 172.21.15.86:0/2727181842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5abc1a0bf0 con 0x7f5abc103580 2024-04-17T07:14:23.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.837+0000 7f5aba000700 1 -- 172.21.15.86:0/2727181842 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac005440 con 0x7f5abc103580 2024-04-17T07:14:23.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.837+0000 7f5ac4200700 1 -- 172.21.15.86:0/2727181842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5abc1a10e0 con 0x7f5abc103580 2024-04-17T07:14:23.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.838+0000 7f5ac4200700 1 -- 172.21.15.86:0/2727181842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5abc069740 con 0x7f5abc103580 2024-04-17T07:14:23.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.840+0000 7f5aba000700 1 -- 172.21.15.86:0/2727181842 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5aac0055e0 con 0x7f5abc103580 2024-04-17T07:14:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.842+0000 7f5aba000700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5aa00a3c40 0x7f5aa00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.842+0000 7f5aba000700 1 -- 172.21.15.86:0/2727181842 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5aac0d0730 con 0x7f5abc103580 2024-04-17T07:14:23.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.842+0000 7f5ac0e00700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5aa00a3c40 0x7f5aa00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:23.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.843+0000 7f5ac0e00700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5aa00a3c40 0x7f5aa00a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006010 tx=0x7f5aa8003870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:23.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:23.847+0000 7f5aba000700 1 -- 172.21.15.86:0/2727181842 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5aac020020 con 0x7f5abc103580 2024-04-17T07:14:24.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.084+0000 7f5ac4200700 1 -- 172.21.15.86:0/2727181842 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5abc062e90 con 0x7f5aa00a3c40 2024-04-17T07:14:24.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.100+0000 7f5aba000700 1 -- 172.21.15.86:0/2727181842 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f5abc062e90 con 0x7f5aa00a3c40 2024-04-17T07:14:24.099 INFO:teuthology.orchestra.run.smithi086.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-17T07:14:24.099 INFO:teuthology.orchestra.run.smithi086.stdout:mds.a smithi086 running (2m) 29s ago 2m 14.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 f0157b6dc715 2024-04-17T07:14:24.099 INFO:teuthology.orchestra.run.smithi086.stdout:mds.b smithi098 running (2m) 82s ago 2m 24.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 28e0e021f6c9 2024-04-17T07:14:24.099 INFO:teuthology.orchestra.run.smithi086.stdout:mds.c smithi103 running (2m) 83s ago 2m 15.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 e7940f396d36 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mds.d smithi086 running (3m) 29s ago 3m 12.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 67a4d7830c90 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mds.e smithi098 running (3m) 82s ago 3m 15.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 19cc2df441fd 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mds.f smithi103 running (3m) 83s ago 2m 14.2M - 19.0.0-3133-ga8c84f59 76bc6b91b726 7b5269347c63 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mds.g smithi086 running (2m) 29s ago 2m 14.8M - 19.0.0-3133-ga8c84f59 76bc6b91b726 ed9543eec6ff 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mds.h smithi098 running (2m) 82s ago 2m 16.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2590f5f40a5f 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mds.i smithi103 running (2m) 83s ago 2m 15.1M - 19.0.0-3133-ga8c84f59 76bc6b91b726 8e3e7493cc70 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mds.j smithi086 running (3m) 29s ago 3m 12.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 97cd7ae16a23 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mds.k smithi098 running (3m) 82s ago 3m 15.6M - 19.0.0-3133-ga8c84f59 76bc6b91b726 6b3e665830c9 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mds.l smithi103 running (3m) 83s ago 3m 14.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 8cccce955125 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.x smithi086 *:9283,8765 running (12m) 29s ago 12m 516M - 19.0.0-3133-ga8c84f59 76bc6b91b726 e4f189e0cff0 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.y smithi098 *:8443,8765 running (10m) 82s ago 10m 452M - 19.0.0-3133-ga8c84f59 76bc6b91b726 7f5ef95ce723 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.z smithi103 *:8443,8765 running (10m) 83s ago 10m 451M - 19.0.0-3133-ga8c84f59 76bc6b91b726 afa69e5a544e 2024-04-17T07:14:24.100 INFO:teuthology.orchestra.run.smithi086.stdout:mon.a smithi086 running (12m) 29s ago 13m 51.3M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 c352ceee4038 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:mon.b smithi098 running (10m) 82s ago 10m 39.9M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 35a692606e82 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:mon.c smithi103 running (10m) 83s ago 10m 39.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 819a7f1e426c 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 smithi086 running (9m) 29s ago 9m 45.2M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 d03541379307 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 smithi098 running (9m) 82s ago 9m 46.0M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 7397415dd010 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 smithi103 running (8m) 83s ago 8m 45.4M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 b000872e2c24 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 smithi086 running (8m) 29s ago 8m 53.8M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 58df7478181b 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 smithi098 running (7m) 82s ago 7m 42.2M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 8c1ea0f9bfca 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 smithi103 running (7m) 83s ago 7m 53.6M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 5d7ac68c4430 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 smithi086 running (6m) 29s ago 6m 53.1M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 1c06539ce62f 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 smithi098 running (6m) 82s ago 6m 45.3M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 f213f7017453 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 smithi103 running (5m) 83s ago 5m 54.0M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 5a4acff0b847 2024-04-17T07:14:24.101 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 smithi086 running (4m) 29s ago 4m 52.7M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 0471db46e5d9 2024-04-17T07:14:24.102 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 smithi098 running (4m) 82s ago 4m 53.8M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 6ce9930a838b 2024-04-17T07:14:24.102 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 smithi103 running (3m) 83s ago 3m 52.0M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 d3816451d8f4 2024-04-17T07:14:24.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 -- 172.21.15.86:0/2727181842 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5aa00a3c40 msgr2=0x7f5aa00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:24.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5aa00a3c40 0x7f5aa00a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006010 tx=0x7f5aa8003870 comp rx=0 tx=0).stop 2024-04-17T07:14:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 -- 172.21.15.86:0/2727181842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5abc103580 msgr2=0x7f5abc198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5abc103580 0x7f5abc198c80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00d980 tx=0x7f5aac00dd40 comp rx=0 tx=0).stop 2024-04-17T07:14:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 -- 172.21.15.86:0/2727181842 shutdown_connections 2024-04-17T07:14:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5aa00a3c40 0x7f5aa00a6100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5abc1087e0 0x7f5abc1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5abc103580 0x7f5abc198c80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 --2- 172.21.15.86:0/2727181842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5abc105eb0 0x7f5abc1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 -- 172.21.15.86:0/2727181842 >> 172.21.15.86:0/2727181842 conn(0x7f5abc0fcf50 msgr2=0x7f5abc103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 -- 172.21.15.86:0/2727181842 shutdown_connections 2024-04-17T07:14:24.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:24.103+0000 7f5ac4200700 1 -- 172.21.15.86:0/2727181842 wait complete. 2024-04-17T07:14:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:24 smithi098 ceph-mon[35418]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:24.518 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:24 smithi086 ceph-mon[31733]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:24 smithi103 ceph-mon[35495]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:25.021 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-17T07:14:25.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:25 smithi086 ceph-mon[31733]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:25.477 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:14:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:25 smithi098 ceph-mon[35418]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:25 smithi103 ceph-mon[35495]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:26.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:26 smithi086 ceph-mon[31733]: pgmap v442: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:26.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:26 smithi098 ceph-mon[35418]: pgmap v442: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:26.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:26 smithi103 ceph-mon[35495]: pgmap v442: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.773+0000 7f8a27c00700 1 -- 172.21.15.86:0/21898928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a201090f0 msgr2=0x7f8a20109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.773+0000 7f8a27c00700 1 --2- 172.21.15.86:0/21898928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a201090f0 0x7f8a20109510 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a140099c0 tx=0x7f8a14009cd0 comp rx=0 tx=0).stop 2024-04-17T07:14:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.774+0000 7f8a27c00700 1 -- 172.21.15.86:0/21898928 shutdown_connections 2024-04-17T07:14:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.774+0000 7f8a27c00700 1 --2- 172.21.15.86:0/21898928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a2010ac90 0x7f8a20111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.774+0000 7f8a27c00700 1 --2- 172.21.15.86:0/21898928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a201090f0 0x7f8a20109510 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.774+0000 7f8a27c00700 1 --2- 172.21.15.86:0/21898928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a2010a2f0 0x7f8a2010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.774+0000 7f8a27c00700 1 -- 172.21.15.86:0/21898928 >> 172.21.15.86:0/21898928 conn(0x7f8a200780c0 msgr2=0x7f8a2007a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.774+0000 7f8a27c00700 1 -- 172.21.15.86:0/21898928 shutdown_connections 2024-04-17T07:14:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.774+0000 7f8a27c00700 1 -- 172.21.15.86:0/21898928 wait complete. 2024-04-17T07:14:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.775+0000 7f8a27c00700 1 Processor -- start 2024-04-17T07:14:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.775+0000 7f8a27c00700 1 -- start start 2024-04-17T07:14:27.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.775+0000 7f8a27c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a201090f0 0x7f8a2019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:27.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.775+0000 7f8a27c00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a2010a2f0 0x7f8a2019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:27.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.775+0000 7f8a27c00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a2010ac90 0x7f8a201a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:27.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.775+0000 7f8a27c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20114b70 con 0x7f8a201090f0 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.775+0000 7f8a27c00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201149f0 con 0x7f8a2010ac90 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.775+0000 7f8a27c00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20114cf0 con 0x7f8a2010a2f0 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a25200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a201090f0 0x7f8a2019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a25200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a201090f0 0x7f8a2019f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38716/0 (socket says 172.21.15.86:38716) 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a25200700 1 -- 172.21.15.86:0/1651713906 learned_addr learned my addr 172.21.15.86:0/1651713906 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a25c00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a2010ac90 0x7f8a201a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a1fe00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a2010a2f0 0x7f8a2019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a25200700 1 -- 172.21.15.86:0/1651713906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a2010a2f0 msgr2=0x7f8a2019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a25200700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a2010a2f0 0x7f8a2019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a25200700 1 -- 172.21.15.86:0/1651713906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a2010ac90 msgr2=0x7f8a201a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a25200700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a2010ac90 0x7f8a201a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a25200700 1 -- 172.21.15.86:0/1651713906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a201a6e40 con 0x7f8a201090f0 2024-04-17T07:14:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.776+0000 7f8a25c00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a2010ac90 0x7f8a201a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:27.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.777+0000 7f8a25200700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a201090f0 0x7f8a2019f230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400fa30 tx=0x7f8a1400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:27.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.777+0000 7f8a1d600700 1 -- 172.21.15.86:0/1651713906 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14027070 con 0x7f8a201090f0 2024-04-17T07:14:27.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.777+0000 7f8a1d600700 1 -- 172.21.15.86:0/1651713906 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a14005940 con 0x7f8a201090f0 2024-04-17T07:14:27.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.777+0000 7f8a27c00700 1 -- 172.21.15.86:0/1651713906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a201a70d0 con 0x7f8a201090f0 2024-04-17T07:14:27.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.777+0000 7f8a1d600700 1 -- 172.21.15.86:0/1651713906 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14017650 con 0x7f8a201090f0 2024-04-17T07:14:27.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.777+0000 7f8a27c00700 1 -- 172.21.15.86:0/1651713906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a201a7600 con 0x7f8a201090f0 2024-04-17T07:14:27.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.778+0000 7f8a27c00700 1 -- 172.21.15.86:0/1651713906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a20069740 con 0x7f8a201090f0 2024-04-17T07:14:27.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.780+0000 7f8a1d600700 1 -- 172.21.15.86:0/1651713906 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8a1402a070 con 0x7f8a201090f0 2024-04-17T07:14:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.782+0000 7f8a1d600700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a080a3cc0 0x7f8a080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.782+0000 7f8a1d600700 1 -- 172.21.15.86:0/1651713906 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8a140d18c0 con 0x7f8a201090f0 2024-04-17T07:14:27.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.782+0000 7f8a1fe00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a080a3cc0 0x7f8a080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:27.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.783+0000 7f8a1fe00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a080a3cc0 0x7f8a080a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c005fd0 tx=0x7f8a0c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:27.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:27.787+0000 7f8a1d600700 1 -- 172.21.15.86:0/1651713906 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8a1405eff0 con 0x7f8a201090f0 2024-04-17T07:14:28.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.020+0000 7f8a27c00700 1 -- 172.21.15.86:0/1651713906 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f8a20062e90 con 0x7f8a080a3cc0 2024-04-17T07:14:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.029+0000 7f8a1d600700 1 -- 172.21.15.86:0/1651713906 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f8a20062e90 con 0x7f8a080a3cc0 2024-04-17T07:14:28.029 INFO:teuthology.orchestra.run.smithi086.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-17T07:14:28.029 INFO:teuthology.orchestra.run.smithi086.stdout:mds.all 12/12 87s ago 3m smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi103=c;smithi103=f;smithi103=i;smithi103=l;count:12 2024-04-17T07:14:28.029 INFO:teuthology.orchestra.run.smithi086.stdout:mgr 3/3 87s ago 10m smithi086=x;smithi098=y;smithi103=z;count:3 2024-04-17T07:14:28.029 INFO:teuthology.orchestra.run.smithi086.stdout:mon 3/3 87s ago 11m smithi086:172.21.15.86=a;smithi098:172.21.15.98=b;smithi103:172.21.15.103=c;count:3 2024-04-17T07:14:28.029 INFO:teuthology.orchestra.run.smithi086.stdout:osd 12 87s ago - 2024-04-17T07:14:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.032+0000 7f8a27c00700 1 -- 172.21.15.86:0/1651713906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a080a3cc0 msgr2=0x7f8a080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.032+0000 7f8a27c00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a080a3cc0 0x7f8a080a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c005fd0 tx=0x7f8a0c005f60 comp rx=0 tx=0).stop 2024-04-17T07:14:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.032+0000 7f8a27c00700 1 -- 172.21.15.86:0/1651713906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a201090f0 msgr2=0x7f8a2019f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.032+0000 7f8a27c00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a201090f0 0x7f8a2019f230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400fa30 tx=0x7f8a1400fb10 comp rx=0 tx=0).stop 2024-04-17T07:14:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.032+0000 7f8a27c00700 1 -- 172.21.15.86:0/1651713906 shutdown_connections 2024-04-17T07:14:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.032+0000 7f8a27c00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a080a3cc0 0x7f8a080a6180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.032+0000 7f8a27c00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a2010ac90 0x7f8a201a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.032+0000 7f8a27c00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a201090f0 0x7f8a2019f230 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.032+0000 7f8a27c00700 1 --2- 172.21.15.86:0/1651713906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a2010a2f0 0x7f8a2019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.032+0000 7f8a27c00700 1 -- 172.21.15.86:0/1651713906 >> 172.21.15.86:0/1651713906 conn(0x7f8a200780c0 msgr2=0x7f8a2010d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.033+0000 7f8a27c00700 1 -- 172.21.15.86:0/1651713906 shutdown_connections 2024-04-17T07:14:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:28.033+0000 7f8a27c00700 1 -- 172.21.15.86:0/1651713906 wait complete. 2024-04-17T07:14:28.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:28 smithi086 ceph-mon[31733]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:28.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:28 smithi098 ceph-mon[35418]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:28 smithi103 ceph-mon[35495]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:28.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-17T07:14:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:14:29.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:29 smithi098 ceph-mon[35418]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:29 smithi103 ceph-mon[35495]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:29.569 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:29 smithi086 ceph-mon[31733]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:30.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:30 smithi098 ceph-mon[35418]: pgmap v444: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:30 smithi103 ceph-mon[35495]: pgmap v444: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:30 smithi086 ceph-mon[31733]: pgmap v444: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:31.473 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:31 smithi086 ceph-mon[31733]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:31.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:31 smithi098 ceph-mon[35418]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:31.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:31 smithi103 ceph-mon[35495]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.819+0000 7f3197200700 1 -- 172.21.15.86:0/2086132797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198105df0 msgr2=0x7f3198106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.819+0000 7f3197200700 1 --2- 172.21.15.86:0/2086132797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198105df0 0x7f3198106250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f31840099c0 tx=0x7f3184009cd0 comp rx=0 tx=0).stop 2024-04-17T07:14:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.819+0000 7f3197200700 1 -- 172.21.15.86:0/2086132797 shutdown_connections 2024-04-17T07:14:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.819+0000 7f3197200700 1 --2- 172.21.15.86:0/2086132797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3198106790 0x7f319810d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.819+0000 7f3197200700 1 --2- 172.21.15.86:0/2086132797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3198104bf0 0x7f3198105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.819+0000 7f3197200700 1 --2- 172.21.15.86:0/2086132797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198105df0 0x7f3198106250 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.819+0000 7f3197200700 1 -- 172.21.15.86:0/2086132797 >> 172.21.15.86:0/2086132797 conn(0x7f3198100190 msgr2=0x7f31981025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.819+0000 7f3197200700 1 -- 172.21.15.86:0/2086132797 shutdown_connections 2024-04-17T07:14:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.819+0000 7f3197200700 1 -- 172.21.15.86:0/2086132797 wait complete. 2024-04-17T07:14:31.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.820+0000 7f3197200700 1 Processor -- start 2024-04-17T07:14:31.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.820+0000 7f3197200700 1 -- start start 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3197200700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3198104bf0 0x7f319819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3197200700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198105df0 0x7f319819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3197200700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3198106790 0x7f31981a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3197200700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31981106e0 con 0x7f3198105df0 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3197200700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3198110560 con 0x7f3198106790 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3197200700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3198110860 con 0x7f3198104bf0 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3195400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198105df0 0x7f319819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3195400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198105df0 0x7f319819b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38726/0 (socket says 172.21.15.86:38726) 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3195400700 1 -- 172.21.15.86:0/3618894172 learned_addr learned my addr 172.21.15.86:0/3618894172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3196800700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3198106790 0x7f31981a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.821+0000 7f3195e00700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3198104bf0 0x7f319819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f3195400700 1 -- 172.21.15.86:0/3618894172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3198104bf0 msgr2=0x7f319819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f3195400700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3198104bf0 0x7f319819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f3195400700 1 -- 172.21.15.86:0/3618894172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3198106790 msgr2=0x7f31981a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f3195400700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3198106790 0x7f31981a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f3195400700 1 -- 172.21.15.86:0/3618894172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31981a29a0 con 0x7f3198105df0 2024-04-17T07:14:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f3196800700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3198106790 0x7f31981a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:14:31.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f3195400700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198105df0 0x7f319819b280 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3184009990 tx=0x7f31840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:31.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f318ea00700 1 -- 172.21.15.86:0/3618894172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184027070 con 0x7f3198105df0 2024-04-17T07:14:31.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f318ea00700 1 -- 172.21.15.86:0/3618894172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3184022470 con 0x7f3198105df0 2024-04-17T07:14:31.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f3197200700 1 -- 172.21.15.86:0/3618894172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31981a2c30 con 0x7f3198105df0 2024-04-17T07:14:31.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.822+0000 7f318ea00700 1 -- 172.21.15.86:0/3618894172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318400f6c0 con 0x7f3198105df0 2024-04-17T07:14:31.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.823+0000 7f3197200700 1 -- 172.21.15.86:0/3618894172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31981a30e0 con 0x7f3198105df0 2024-04-17T07:14:31.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.824+0000 7f3197200700 1 -- 172.21.15.86:0/3618894172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3198069740 con 0x7f3198105df0 2024-04-17T07:14:31.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.826+0000 7f318ea00700 1 -- 172.21.15.86:0/3618894172 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f318402a070 con 0x7f3198105df0 2024-04-17T07:14:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.828+0000 7f318ea00700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f317c0a3cc0 0x7f317c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.828+0000 7f318ea00700 1 -- 172.21.15.86:0/3618894172 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f31840d17b0 con 0x7f3198105df0 2024-04-17T07:14:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.828+0000 7f3195e00700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f317c0a3cc0 0x7f317c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.829+0000 7f3195e00700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f317c0a3cc0 0x7f317c0a6180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3188005fd0 tx=0x7f3188005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:31.833+0000 7f318ea00700 1 -- 172.21.15.86:0/3618894172 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f318405eff0 con 0x7f3198105df0 2024-04-17T07:14:32.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.062+0000 7f3197200700 1 -- 172.21.15.86:0/3618894172 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3198062f50 con 0x7f317c0a3cc0 2024-04-17T07:14:32.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.066+0000 7f318ea00700 1 -- 172.21.15.86:0/3618894172 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3198062f50 con 0x7f317c0a3cc0 2024-04-17T07:14:32.066 INFO:teuthology.orchestra.run.smithi086.stdout:HOST ADDR LABELS STATUS 2024-04-17T07:14:32.066 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 172.21.15.86 2024-04-17T07:14:32.066 INFO:teuthology.orchestra.run.smithi086.stdout:smithi098 172.21.15.98 2024-04-17T07:14:32.066 INFO:teuthology.orchestra.run.smithi086.stdout:smithi103 172.21.15.103 2024-04-17T07:14:32.066 INFO:teuthology.orchestra.run.smithi086.stdout:3 hosts in cluster 2024-04-17T07:14:32.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.068+0000 7f3197200700 1 -- 172.21.15.86:0/3618894172 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f317c0a3cc0 msgr2=0x7f317c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:32.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.068+0000 7f3197200700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f317c0a3cc0 0x7f317c0a6180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3188005fd0 tx=0x7f3188005dd0 comp rx=0 tx=0).stop 2024-04-17T07:14:32.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.068+0000 7f3197200700 1 -- 172.21.15.86:0/3618894172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198105df0 msgr2=0x7f319819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:32.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.068+0000 7f3197200700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198105df0 0x7f319819b280 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3184009990 tx=0x7f31840047b0 comp rx=0 tx=0).stop 2024-04-17T07:14:32.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.069+0000 7f3197200700 1 -- 172.21.15.86:0/3618894172 shutdown_connections 2024-04-17T07:14:32.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.069+0000 7f3197200700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f317c0a3cc0 0x7f317c0a6180 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:32.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.069+0000 7f3197200700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3198106790 0x7f31981a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:32.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.069+0000 7f3197200700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3198104bf0 0x7f319819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:32.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.069+0000 7f3197200700 1 --2- 172.21.15.86:0/3618894172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3198105df0 0x7f319819b280 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:32.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.069+0000 7f3197200700 1 -- 172.21.15.86:0/3618894172 >> 172.21.15.86:0/3618894172 conn(0x7f3198100190 msgr2=0x7f3198109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:32.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.069+0000 7f3197200700 1 -- 172.21.15.86:0/3618894172 shutdown_connections 2024-04-17T07:14:32.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:32.069+0000 7f3197200700 1 -- 172.21.15.86:0/3618894172 wait complete. 2024-04-17T07:14:32.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:32 smithi086 ceph-mon[31733]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:32 smithi098 ceph-mon[35418]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:32 smithi103 ceph-mon[35495]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:32.888 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-17T07:14:33.362 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:14:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:33 smithi098 ceph-mon[35418]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:33.519 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:33 smithi086 ceph-mon[31733]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:33.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:33 smithi103 ceph-mon[35495]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.657+0000 7f0c93c00700 1 -- 172.21.15.86:0/3388716968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c106890 msgr2=0x7f0c8c10d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.657+0000 7f0c93c00700 1 --2- 172.21.15.86:0/3388716968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c106890 0x7f0c8c10d140 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400b0d0 tx=0x7f0c8400b3e0 comp rx=0 tx=0).stop 2024-04-17T07:14:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.657+0000 7f0c93c00700 1 -- 172.21.15.86:0/3388716968 shutdown_connections 2024-04-17T07:14:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.657+0000 7f0c93c00700 1 --2- 172.21.15.86:0/3388716968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c106890 0x7f0c8c10d140 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.657+0000 7f0c93c00700 1 --2- 172.21.15.86:0/3388716968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c8c104cf0 0x7f0c8c105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.657+0000 7f0c93c00700 1 --2- 172.21.15.86:0/3388716968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c8c105ef0 0x7f0c8c106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.657+0000 7f0c93c00700 1 -- 172.21.15.86:0/3388716968 >> 172.21.15.86:0/3388716968 conn(0x7f0c8c1002b0 msgr2=0x7f0c8c1026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.657+0000 7f0c93c00700 1 -- 172.21.15.86:0/3388716968 shutdown_connections 2024-04-17T07:14:35.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.657+0000 7f0c93c00700 1 -- 172.21.15.86:0/3388716968 wait complete. 2024-04-17T07:14:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.658+0000 7f0c93c00700 1 Processor -- start 2024-04-17T07:14:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c93c00700 1 -- start start 2024-04-17T07:14:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c93c00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c8c104cf0 0x7f0c8c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c93c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c105ef0 0x7f0c8c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c93c00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c8c106890 0x7f0c8c1a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c93c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c110790 con 0x7f0c8c105ef0 2024-04-17T07:14:35.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c93c00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c110610 con 0x7f0c8c104cf0 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c93c00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c110910 con 0x7f0c8c106890 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c8be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c105ef0 0x7f0c8c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c8be00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c105ef0 0x7f0c8c19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38744/0 (socket says 172.21.15.86:38744) 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c8be00700 1 -- 172.21.15.86:0/834455296 learned_addr learned my addr 172.21.15.86:0/834455296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c91200700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c8c104cf0 0x7f0c8c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.659+0000 7f0c91c00700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c8c106890 0x7f0c8c1a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c8be00700 1 -- 172.21.15.86:0/834455296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c8c106890 msgr2=0x7f0c8c1a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c8be00700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c8c106890 0x7f0c8c1a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c8be00700 1 -- 172.21.15.86:0/834455296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c8c104cf0 msgr2=0x7f0c8c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c8be00700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c8c104cf0 0x7f0c8c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c8be00700 1 -- 172.21.15.86:0/834455296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c1a29a0 con 0x7f0c8c105ef0 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c91200700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c8c104cf0 0x7f0c8c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:35.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c8be00700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c105ef0 0x7f0c8c19b280 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800d9b0 tx=0x7f0c7800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:35.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c89600700 1 -- 172.21.15.86:0/834455296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78017070 con 0x7f0c8c105ef0 2024-04-17T07:14:35.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c93c00700 1 -- 172.21.15.86:0/834455296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8c1a2c90 con 0x7f0c8c105ef0 2024-04-17T07:14:35.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c89600700 1 -- 172.21.15.86:0/834455296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c78004830 con 0x7f0c8c105ef0 2024-04-17T07:14:35.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c89600700 1 -- 172.21.15.86:0/834455296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c780053a0 con 0x7f0c8c105ef0 2024-04-17T07:14:35.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.660+0000 7f0c93c00700 1 -- 172.21.15.86:0/834455296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8c1a3180 con 0x7f0c8c105ef0 2024-04-17T07:14:35.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.662+0000 7f0c93c00700 1 -- 172.21.15.86:0/834455296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8c069740 con 0x7f0c8c105ef0 2024-04-17T07:14:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.663+0000 7f0c89600700 1 -- 172.21.15.86:0/834455296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0c780049e0 con 0x7f0c8c105ef0 2024-04-17T07:14:35.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.665+0000 7f0c89600700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c740a3c40 0x7f0c740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:35.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.665+0000 7f0c89600700 1 -- 172.21.15.86:0/834455296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0c780d0730 con 0x7f0c8c105ef0 2024-04-17T07:14:35.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.665+0000 7f0c91200700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c740a3c40 0x7f0c740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:35.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.666+0000 7f0c91200700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c740a3c40 0x7f0c740a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0c80005fd0 tx=0x7f0c80005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:35.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.669+0000 7f0c89600700 1 -- 172.21.15.86:0/834455296 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0c78020020 con 0x7f0c8c105ef0 2024-04-17T07:14:35.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.905+0000 7f0c93c00700 1 -- 172.21.15.86:0/834455296 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0c8c062e90 con 0x7f0c740a3c40 2024-04-17T07:14:35.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.909+0000 7f0c89600700 1 -- 172.21.15.86:0/834455296 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f0c8c062e90 con 0x7f0c740a3c40 2024-04-17T07:14:35.909 INFO:teuthology.orchestra.run.smithi086.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-17T07:14:35.909 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-17T07:14:35.909 INFO:teuthology.orchestra.run.smithi086.stdout:smithi098 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010H400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-17T07:14:35.909 INFO:teuthology.orchestra.run.smithi086.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-17T07:14:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.911+0000 7f0c93c00700 1 -- 172.21.15.86:0/834455296 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c740a3c40 msgr2=0x7f0c740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.911+0000 7f0c93c00700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c740a3c40 0x7f0c740a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0c80005fd0 tx=0x7f0c80005f00 comp rx=0 tx=0).stop 2024-04-17T07:14:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.911+0000 7f0c93c00700 1 -- 172.21.15.86:0/834455296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c105ef0 msgr2=0x7f0c8c19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:35.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.911+0000 7f0c93c00700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c105ef0 0x7f0c8c19b280 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0c7800d9b0 tx=0x7f0c7800dd70 comp rx=0 tx=0).stop 2024-04-17T07:14:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.911+0000 7f0c93c00700 1 -- 172.21.15.86:0/834455296 shutdown_connections 2024-04-17T07:14:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.911+0000 7f0c93c00700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c740a3c40 0x7f0c740a6100 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.911+0000 7f0c93c00700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c8c106890 0x7f0c8c1a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.911+0000 7f0c93c00700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c8c104cf0 0x7f0c8c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.911+0000 7f0c93c00700 1 --2- 172.21.15.86:0/834455296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c8c105ef0 0x7f0c8c19b280 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.911+0000 7f0c93c00700 1 -- 172.21.15.86:0/834455296 >> 172.21.15.86:0/834455296 conn(0x7f0c8c1002b0 msgr2=0x7f0c8c1091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.912+0000 7f0c93c00700 1 -- 172.21.15.86:0/834455296 shutdown_connections 2024-04-17T07:14:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:35.912+0000 7f0c93c00700 1 -- 172.21.15.86:0/834455296 wait complete. 2024-04-17T07:14:36.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:36 smithi098 ceph-mon[35418]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:36.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:36 smithi103 ceph-mon[35495]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:36 smithi086 ceph-mon[31733]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:36.763 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-17T07:14:36.773 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-17T07:14:36.773 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-17T07:14:37.236 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:14:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:37 smithi098 ceph-mon[35418]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:37 smithi103 ceph-mon[35495]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:37.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:37 smithi086 ceph-mon[31733]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:14:38.427 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:38 smithi086 ceph-mon[31733]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:38.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:38 smithi098 ceph-mon[35418]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:38 smithi103 ceph-mon[35495]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.594+0000 7f3e5cc00700 1 -- 172.21.15.86:0/602680722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e58108fe0 msgr2=0x7f3e58109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.594+0000 7f3e5cc00700 1 --2- 172.21.15.86:0/602680722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e58108fe0 0x7f3e58109400 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3e480099c0 tx=0x7f3e48009cd0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.594+0000 7f3e5cc00700 1 -- 172.21.15.86:0/602680722 shutdown_connections 2024-04-17T07:14:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.594+0000 7f3e5cc00700 1 --2- 172.21.15.86:0/602680722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5810ab80 0x7f3e58111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.594+0000 7f3e5cc00700 1 --2- 172.21.15.86:0/602680722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e58108fe0 0x7f3e58109400 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.594+0000 7f3e5cc00700 1 --2- 172.21.15.86:0/602680722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e5810a1e0 0x7f3e5810a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.594+0000 7f3e5cc00700 1 -- 172.21.15.86:0/602680722 >> 172.21.15.86:0/602680722 conn(0x7f3e58077fb0 msgr2=0x7f3e5807a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.594+0000 7f3e5cc00700 1 -- 172.21.15.86:0/602680722 shutdown_connections 2024-04-17T07:14:39.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.594+0000 7f3e5cc00700 1 -- 172.21.15.86:0/602680722 wait complete. 2024-04-17T07:14:39.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.595+0000 7f3e5cc00700 1 Processor -- start 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.595+0000 7f3e5cc00700 1 -- start start 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.595+0000 7f3e5cc00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e58108fe0 0x7f3e5819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e5cc00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5810a1e0 0x7f3e5819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e5cc00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e5810ab80 0x7f3e581a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e5cc00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58114a50 con 0x7f3e5810ab80 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e5cc00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581148d0 con 0x7f3e58108fe0 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e5cc00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58114bd0 con 0x7f3e5810a1e0 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e5810ab80 0x7f3e581a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e57e00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e5810ab80 0x7f3e581a6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46116/0 (socket says 172.21.15.86:46116) 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e57e00700 1 -- 172.21.15.86:0/1323568488 learned_addr learned my addr 172.21.15.86:0/1323568488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e57400700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e58108fe0 0x7f3e5819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e56a00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5810a1e0 0x7f3e5819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e57e00700 1 -- 172.21.15.86:0/1323568488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5810a1e0 msgr2=0x7f3e5819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e57e00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5810a1e0 0x7f3e5819f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e57e00700 1 -- 172.21.15.86:0/1323568488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e58108fe0 msgr2=0x7f3e5819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:39.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e57e00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e58108fe0 0x7f3e5819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.596+0000 7f3e57e00700 1 -- 172.21.15.86:0/1323568488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e581a6c10 con 0x7f3e5810ab80 2024-04-17T07:14:39.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.597+0000 7f3e57e00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e5810ab80 0x7f3e581a6570 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3e50007da0 tx=0x7f3e5000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.597+0000 7f3e4fe00700 1 -- 172.21.15.86:0/1323568488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50016020 con 0x7f3e5810ab80 2024-04-17T07:14:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.597+0000 7f3e4fe00700 1 -- 172.21.15.86:0/1323568488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e5000ad00 con 0x7f3e5810ab80 2024-04-17T07:14:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.597+0000 7f3e5cc00700 1 -- 172.21.15.86:0/1323568488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e581a6f00 con 0x7f3e5810ab80 2024-04-17T07:14:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.597+0000 7f3e4fe00700 1 -- 172.21.15.86:0/1323568488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e500115e0 con 0x7f3e5810ab80 2024-04-17T07:14:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.597+0000 7f3e5cc00700 1 -- 172.21.15.86:0/1323568488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e581a7360 con 0x7f3e5810ab80 2024-04-17T07:14:39.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.598+0000 7f3e5cc00700 1 -- 172.21.15.86:0/1323568488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e58069740 con 0x7f3e5810ab80 2024-04-17T07:14:39.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.600+0000 7f3e4fe00700 1 -- 172.21.15.86:0/1323568488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3e50019020 con 0x7f3e5810ab80 2024-04-17T07:14:39.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.602+0000 7f3e4fe00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e3c0a3c40 0x7f3e3c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:39.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.602+0000 7f3e4fe00700 1 -- 172.21.15.86:0/1323568488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3e500d0d00 con 0x7f3e5810ab80 2024-04-17T07:14:39.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.602+0000 7f3e57400700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e3c0a3c40 0x7f3e3c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:39.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.603+0000 7f3e57400700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e3c0a3c40 0x7f3e3c0a6100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3e48000c00 tx=0x7f3e48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.605+0000 7f3e4fe00700 1 -- 172.21.15.86:0/1323568488 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e50020020 con 0x7f3e5810ab80 2024-04-17T07:14:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.880+0000 7f3e5cc00700 1 -- 172.21.15.86:0/1323568488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3e5802e010 con 0x7f3e5810ab80 2024-04-17T07:14:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.881+0000 7f3e4fe00700 1 -- 172.21.15.86:0/1323568488 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3095 (secure 0 0 0) 0x7f3e500bbe20 con 0x7f3e5810ab80 2024-04-17T07:14:39.882 INFO:teuthology.orchestra.run.smithi086.stdout:e27 2024-04-17T07:14:39.882 INFO:teuthology.orchestra.run.smithi086.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-17T07:14:39.882 INFO:teuthology.orchestra.run.smithi086.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T07:14:39.882 INFO:teuthology.orchestra.run.smithi086.stdout:legacy client fscid: 1 2024-04-17T07:14:39.882 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:39.882 INFO:teuthology.orchestra.run.smithi086.stdout:Filesystem 'cephfs' (1) 2024-04-17T07:14:39.882 INFO:teuthology.orchestra.run.smithi086.stdout:fs_name cephfs 2024-04-17T07:14:39.882 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 27 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-04-17T07:11:54.934275+0000 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-04-17T07:12:31.203694+0000 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:tableserver 0 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:root 0 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:session_timeout 300 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:session_autoclose 300 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:max_file_size 1099511627776 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:max_xattr_size 65536 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:required_client_features {} 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure 0 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure_osd_epoch 0 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:snap_rstat 1 2024-04-17T07:14:39.883 INFO:teuthology.orchestra.run.smithi086.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:max_mds 5 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:in 0,1,2,3,4 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:up {0=34490,1=34460,2=34430,3=34400,4=24505} 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:failed 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:damaged 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:stopped 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:data_pools [3,4] 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:metadata_pool 2 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:inline_data disabled 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:balancer 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:bal_rank_mask -1 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:standby_count_wanted 0 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:qdb_cluster leader: 34490 members: 34460,34490,34430,34400,24505 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b{0:34490} state up:active seq 3 addr [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.884 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c{0:24487} state up:standby-replay seq 1 addr [v2:172.21.15.103:6836/299191474,v1:172.21.15.103:6837/299191474] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.h{1:34460} state up:active seq 15 addr [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.f{1:24469} state up:standby-replay seq 1 addr [v2:172.21.15.103:6834/423518277,v1:172.21.15.103:6835/423518277] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.e{2:34430} state up:active seq 19 addr [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.l{2:24412} state up:standby-replay seq 1 addr [v2:172.21.15.103:6832/1707978686,v1:172.21.15.103:6833/1707978686] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.k{3:34400} state up:active seq 22 addr [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a{3:14700} state up:standby-replay seq 1 addr [v2:172.21.15.86:6840/4256331668,v1:172.21.15.86:6841/4256331668] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.i{4:24505} state up:active seq 15 addr [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.g{4:14688} state up:standby-replay seq 1 addr [v2:172.21.15.86:6838/2923053510,v1:172.21.15.86:6839/2923053510] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:Standby daemons: 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.j{-1:14583} state up:standby seq 1 addr [v2:172.21.15.86:6834/22224874,v1:172.21.15.86:6835/22224874] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.885 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d{-1:14601} state up:standby seq 1 addr [v2:172.21.15.86:6836/267440661,v1:172.21.15.86:6837/267440661] compat {c=[1],r=[1],i=[1fff]}] 2024-04-17T07:14:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.885+0000 7f3e5cc00700 1 -- 172.21.15.86:0/1323568488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e3c0a3c40 msgr2=0x7f3e3c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.885+0000 7f3e5cc00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e3c0a3c40 0x7f3e3c0a6100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3e48000c00 tx=0x7f3e48005fd0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.885+0000 7f3e5cc00700 1 -- 172.21.15.86:0/1323568488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e5810ab80 msgr2=0x7f3e581a6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.885+0000 7f3e5cc00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e5810ab80 0x7f3e581a6570 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3e50007da0 tx=0x7f3e5000a340 comp rx=0 tx=0).stop 2024-04-17T07:14:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.885+0000 7f3e5cc00700 1 -- 172.21.15.86:0/1323568488 shutdown_connections 2024-04-17T07:14:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.885+0000 7f3e5cc00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e3c0a3c40 0x7f3e3c0a6100 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.885+0000 7f3e5cc00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e5810ab80 0x7f3e581a6570 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.885+0000 7f3e5cc00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e58108fe0 0x7f3e5819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.885+0000 7f3e5cc00700 1 --2- 172.21.15.86:0/1323568488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e5810a1e0 0x7f3e5819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.885+0000 7f3e5cc00700 1 -- 172.21.15.86:0/1323568488 >> 172.21.15.86:0/1323568488 conn(0x7f3e58077fb0 msgr2=0x7f3e5810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.886+0000 7f3e5cc00700 1 -- 172.21.15.86:0/1323568488 shutdown_connections 2024-04-17T07:14:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:39.886+0000 7f3e5cc00700 1 -- 172.21.15.86:0/1323568488 wait complete. 2024-04-17T07:14:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-04-17T07:14:40.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:40 smithi098 ceph-mon[35418]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:40 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1323568488' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-17T07:14:40.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:40 smithi103 ceph-mon[35495]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:40.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:40 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1323568488' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-17T07:14:40.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:40 smithi086 ceph-mon[31733]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:40.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:40 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1323568488' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-17T07:14:40.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-17T07:14:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:14:42.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:42 smithi098 ceph-mon[35418]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:42.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:42 smithi103 ceph-mon[35495]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:42.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:42 smithi086 ceph-mon[31733]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.502+0000 7f32a0c00700 1 -- 172.21.15.86:0/536712575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c105df0 msgr2=0x7f329c106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.502+0000 7f32a0c00700 1 --2- 172.21.15.86:0/536712575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c105df0 0x7f329c106250 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f32880099c0 tx=0x7f3288009cd0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.502+0000 7f32a0c00700 1 -- 172.21.15.86:0/536712575 shutdown_connections 2024-04-17T07:14:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.502+0000 7f32a0c00700 1 --2- 172.21.15.86:0/536712575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c106790 0x7f329c10d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.502+0000 7f32a0c00700 1 --2- 172.21.15.86:0/536712575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f329c104bf0 0x7f329c105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.502+0000 7f32a0c00700 1 --2- 172.21.15.86:0/536712575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c105df0 0x7f329c106250 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.502+0000 7f32a0c00700 1 -- 172.21.15.86:0/536712575 >> 172.21.15.86:0/536712575 conn(0x7f329c100170 msgr2=0x7f329c1025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.502+0000 7f32a0c00700 1 -- 172.21.15.86:0/536712575 shutdown_connections 2024-04-17T07:14:43.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.502+0000 7f32a0c00700 1 -- 172.21.15.86:0/536712575 wait complete. 2024-04-17T07:14:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.503+0000 7f32a0c00700 1 Processor -- start 2024-04-17T07:14:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.504+0000 7f32a0c00700 1 -- start start 2024-04-17T07:14:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.504+0000 7f32a0c00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c104bf0 0x7f329c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.504+0000 7f32a0c00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f329c105df0 0x7f329c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.504+0000 7f32a0c00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c106790 0x7f329c1a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.504+0000 7f32a0c00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f329c1106e0 con 0x7f329c104bf0 2024-04-17T07:14:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.504+0000 7f32a0c00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f329c110560 con 0x7f329c105df0 2024-04-17T07:14:43.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.504+0000 7f32a0c00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f329c110860 con 0x7f329c106790 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.504+0000 7f329b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c104bf0 0x7f329c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.504+0000 7f329b400700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c104bf0 0x7f329c19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46140/0 (socket says 172.21.15.86:46140) 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.504+0000 7f329b400700 1 -- 172.21.15.86:0/155083857 learned_addr learned my addr 172.21.15.86:0/155083857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.505+0000 7f329aa00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f329c105df0 0x7f329c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.505+0000 7f329be00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c106790 0x7f329c1a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.505+0000 7f329b400700 1 -- 172.21.15.86:0/155083857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c106790 msgr2=0x7f329c1a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.505+0000 7f329b400700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c106790 0x7f329c1a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.505+0000 7f329b400700 1 -- 172.21.15.86:0/155083857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f329c105df0 msgr2=0x7f329c19b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.505+0000 7f329b400700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f329c105df0 0x7f329c19b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.505+0000 7f329b400700 1 -- 172.21.15.86:0/155083857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c1a2990 con 0x7f329c104bf0 2024-04-17T07:14:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.505+0000 7f329b400700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c104bf0 0x7f329c19ad40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f328c00d980 tx=0x7f328c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.506+0000 7f3293e00700 1 -- 172.21.15.86:0/155083857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c017030 con 0x7f329c104bf0 2024-04-17T07:14:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.506+0000 7f32a0c00700 1 -- 172.21.15.86:0/155083857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c1a2c80 con 0x7f329c104bf0 2024-04-17T07:14:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.506+0000 7f3293e00700 1 -- 172.21.15.86:0/155083857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f328c0044d0 con 0x7f329c104bf0 2024-04-17T07:14:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.506+0000 7f3293e00700 1 -- 172.21.15.86:0/155083857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c003a40 con 0x7f329c104bf0 2024-04-17T07:14:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.506+0000 7f32a0c00700 1 -- 172.21.15.86:0/155083857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c1a31c0 con 0x7f329c104bf0 2024-04-17T07:14:43.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.507+0000 7f32a0c00700 1 -- 172.21.15.86:0/155083857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329c069740 con 0x7f329c104bf0 2024-04-17T07:14:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.509+0000 7f3293e00700 1 -- 172.21.15.86:0/155083857 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f328c01a070 con 0x7f329c104bf0 2024-04-17T07:14:43.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.510+0000 7f3293e00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32800a3c40 0x7f32800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.511+0000 7f3293e00700 1 -- 172.21.15.86:0/155083857 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f328c0d0000 con 0x7f329c104bf0 2024-04-17T07:14:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.511+0000 7f329aa00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32800a3c40 0x7f32800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:43.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.512+0000 7f329aa00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32800a3c40 0x7f32800a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f32880038f0 tx=0x7f3288005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:43.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.515+0000 7f3293e00700 1 -- 172.21.15.86:0/155083857 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f328c021070 con 0x7f329c104bf0 2024-04-17T07:14:43.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.743+0000 7f32a0c00700 1 -- 172.21.15.86:0/155083857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f329c063660 con 0x7f329c104bf0 2024-04-17T07:14:43.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.744+0000 7f3293e00700 1 -- 172.21.15.86:0/155083857 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5606 (secure 0 0 0) 0x7f328c0222a0 con 0x7f329c104bf0 2024-04-17T07:14:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 93 2024-04-17T07:14:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:14:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-04-17T07:01:19.941306+0000 2024-04-17T07:14:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-04-17T07:13:29.152264+0000 2024-04-17T07:14:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-17T07:14:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:crush_version 27 2024-04-17T07:14:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:full_ratio 0.95 2024-04-17T07:14:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:backfillfull_ratio 0.9 2024-04-17T07:14:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:nearfull_ratio 0.85 2024-04-17T07:14:43.745 INFO:teuthology.orchestra.run.smithi086.stdout:require_min_compat_client luminous 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:min_compat_client luminous 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:require_osd_release squid 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:stretch_mode_enabled false 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:max_osd 12 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6802/871033288,v1:172.21.15.86:6803/871033288] [v2:172.21.15.86:6804/871033288,v1:172.21.15.86:6805/871033288] exists,up 770a17ec-a9cd-4339-b3d5-1a841ac60e52 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/140628921,v1:172.21.15.98:6801/140628921] [v2:172.21.15.98:6802/140628921,v1:172.21.15.98:6803/140628921] exists,up ded7c4aa-ef6b-4065-9f7f-b26aca9c1299 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6800/3728485789,v1:172.21.15.103:6801/3728485789] [v2:172.21.15.103:6802/3728485789,v1:172.21.15.103:6803/3728485789] exists,up 29d6fda2-deeb-4a3f-aa06-16bec72d0cc7 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6810/3089932177,v1:172.21.15.86:6811/3089932177] [v2:172.21.15.86:6812/3089932177,v1:172.21.15.86:6813/3089932177] exists,up 2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 up in weight 1 up_from 33 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/2532868809,v1:172.21.15.98:6809/2532868809] [v2:172.21.15.98:6810/2532868809,v1:172.21.15.98:6811/2532868809] exists,up 0591e547-4420-43e7-9454-663746016d3a 2024-04-17T07:14:43.746 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 up in weight 1 up_from 38 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6808/3545437589,v1:172.21.15.103:6809/3545437589] [v2:172.21.15.103:6810/3545437589,v1:172.21.15.103:6811/3545437589] exists,up 3965d5b0-7cd1-40fe-9b78-287e5a584a5b 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 up in weight 1 up_from 46 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6818/994878330,v1:172.21.15.86:6819/994878330] [v2:172.21.15.86:6820/994878330,v1:172.21.15.86:6821/994878330] exists,up c695ed77-6e45-4218-88cd-40094d317983 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/3839591799,v1:172.21.15.98:6817/3839591799] [v2:172.21.15.98:6818/3839591799,v1:172.21.15.98:6819/3839591799] exists,up eacf793e-e041-433e-ba5b-374fb4f1c6d6 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6816/55757000,v1:172.21.15.103:6817/55757000] [v2:172.21.15.103:6818/55757000,v1:172.21.15.103:6819/55757000] exists,up 6c7c6330-3ffb-45e1-b9b5-77db0f63a735 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6826/3368302207,v1:172.21.15.86:6827/3368302207] [v2:172.21.15.86:6828/3368302207,v1:172.21.15.86:6829/3368302207] exists,up c4f5184c-d482-4569-9b76-78377f60226d 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 up in weight 1 up_from 72 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6824/2308079272,v1:172.21.15.98:6825/2308079272] [v2:172.21.15.98:6826/2308079272,v1:172.21.15.98:6827/2308079272] exists,up f42db156-1323-4e33-9d20-d3401ef400fd 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 up in weight 1 up_from 80 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6824/1742700108,v1:172.21.15.103:6825/1742700108] [v2:172.21.15.103:6826/1742700108,v1:172.21.15.103:6827/1742700108] exists,up 86c2d8c2-4547-437f-a8f4-d12f4a8047b3 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.10 [8,11] 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.16 [2,4] 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.1b [2,1] 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.1d [8,11] 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.1 [4,9] 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.33 [5,9] 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.34 [4,9] 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.6 [5,8] 2024-04-17T07:14:43.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.f [9,10] 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.18 [9,8] 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.1c [9,8] 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/174556107 expires 2024-04-18T07:02:26.881886+0000 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/2497304884 expires 2024-04-18T07:02:26.881886+0000 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/2497304884 expires 2024-04-18T07:02:26.881886+0000 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2537127051 expires 2024-04-18T07:01:52.772350+0000 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2158768684 expires 2024-04-18T07:02:26.881886+0000 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3172534015 expires 2024-04-18T07:02:26.881886+0000 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/4176177513 expires 2024-04-18T07:01:52.772350+0000 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3239168822 expires 2024-04-18T07:01:52.772350+0000 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/4176177513 expires 2024-04-18T07:01:52.772350+0000 2024-04-17T07:14:43.748 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3864604926 expires 2024-04-18T07:01:52.772350+0000 2024-04-17T07:14:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 -- 172.21.15.86:0/155083857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32800a3c40 msgr2=0x7f32800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32800a3c40 0x7f32800a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f32880038f0 tx=0x7f3288005fb0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 -- 172.21.15.86:0/155083857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c104bf0 msgr2=0x7f329c19ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c104bf0 0x7f329c19ad40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f328c00d980 tx=0x7f328c00dd40 comp rx=0 tx=0).stop 2024-04-17T07:14:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 -- 172.21.15.86:0/155083857 shutdown_connections 2024-04-17T07:14:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32800a3c40 0x7f32800a6100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f329c106790 0x7f329c1a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c104bf0 0x7f329c19ad40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 --2- 172.21.15.86:0/155083857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f329c105df0 0x7f329c19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 -- 172.21.15.86:0/155083857 >> 172.21.15.86:0/155083857 conn(0x7f329c100170 msgr2=0x7f329c109110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 -- 172.21.15.86:0/155083857 shutdown_connections 2024-04-17T07:14:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:43.747+0000 7f32a0c00700 1 -- 172.21.15.86:0/155083857 wait complete. 2024-04-17T07:14:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:44 smithi098 ceph-mon[35418]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:44.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:44 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/155083857' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-17T07:14:44.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:44 smithi086 ceph-mon[31733]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:44.525 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:44 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/155083857' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-17T07:14:44.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:44 smithi103 ceph-mon[35495]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:44.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:44 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/155083857' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-17T07:14:44.574 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-17T07:14:44.585 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:14:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.801+0000 7f2508a00640 1 -- 172.21.15.86:0/2594657798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2504109f90 msgr2=0x7f2504110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.801+0000 7f2508a00640 1 --2- 172.21.15.86:0/2594657798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2504109f90 0x7f2504110820 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009900 tx=0x7f24fc02bc40 comp rx=0 tx=0).stop 2024-04-17T07:14:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.801+0000 7f2508a00640 1 -- 172.21.15.86:0/2594657798 shutdown_connections 2024-04-17T07:14:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.801+0000 7f2508a00640 1 --2- 172.21.15.86:0/2594657798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2504109f90 0x7f2504110820 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.801+0000 7f2508a00640 1 --2- 172.21.15.86:0/2594657798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25041095d0 0x7f2504109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.801+0000 7f2508a00640 1 --2- 172.21.15.86:0/2594657798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25041083e0 0x7f2504108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.801+0000 7f2508a00640 1 -- 172.21.15.86:0/2594657798 >> 172.21.15.86:0/2594657798 conn(0x7f25040773a0 msgr2=0x7f25040797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.801+0000 7f2508a00640 1 -- 172.21.15.86:0/2594657798 shutdown_connections 2024-04-17T07:14:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.801+0000 7f2508a00640 1 -- 172.21.15.86:0/2594657798 wait complete. 2024-04-17T07:14:44.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.802+0000 7f2508a00640 1 Processor -- start 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.802+0000 7f2508a00640 1 -- start start 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.802+0000 7f2508a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25041083e0 0x7f250419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.802+0000 7f2508a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25041095d0 0x7f250419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.802+0000 7f2508a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2504109f90 0x7f25041a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.802+0000 7f2508a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2504113f70 con 0x7f25041083e0 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.802+0000 7f2508a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2504113df0 con 0x7f2504109f90 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.802+0000 7f2508a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25041140f0 con 0x7f25041095d0 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.803+0000 7f2502800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25041083e0 0x7f250419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.803+0000 7f2502800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25041083e0 0x7f250419e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46162/0 (socket says 172.21.15.86:46162) 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.803+0000 7f2502800640 1 -- 172.21.15.86:0/4005424077 learned_addr learned my addr 172.21.15.86:0/4005424077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.803+0000 7f2502800640 1 -- 172.21.15.86:0/4005424077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25041095d0 msgr2=0x7f250419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.803+0000 7f2502800640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25041095d0 0x7f250419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.803+0000 7f2503200640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2504109f90 0x7f25041a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.803+0000 7f2502800640 1 -- 172.21.15.86:0/4005424077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2504109f90 msgr2=0x7f25041a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.803+0000 7f2502800640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2504109f90 0x7f25041a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.803+0000 7f2502800640 1 -- 172.21.15.86:0/4005424077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25041a61e0 con 0x7f25041083e0 2024-04-17T07:14:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.803+0000 7f2502800640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25041083e0 0x7f250419e5c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f24f400eaa0 tx=0x7f24f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.804+0000 7f24fb400640 1 -- 172.21.15.86:0/4005424077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4016070 con 0x7f25041083e0 2024-04-17T07:14:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.804+0000 7f24fb400640 1 -- 172.21.15.86:0/4005424077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24f4003e20 con 0x7f25041083e0 2024-04-17T07:14:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.804+0000 7f2508a00640 1 -- 172.21.15.86:0/4005424077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25041a64d0 con 0x7f25041083e0 2024-04-17T07:14:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.804+0000 7f24fb400640 1 -- 172.21.15.86:0/4005424077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f40116d0 con 0x7f25041083e0 2024-04-17T07:14:44.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.804+0000 7f2508a00640 1 -- 172.21.15.86:0/4005424077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25041a6a20 con 0x7f25041083e0 2024-04-17T07:14:44.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.805+0000 7f2508a00640 1 -- 172.21.15.86:0/4005424077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250410f700 con 0x7f25041083e0 2024-04-17T07:14:44.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.807+0000 7f24fb400640 1 -- 172.21.15.86:0/4005424077 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f24f4019070 con 0x7f25041083e0 2024-04-17T07:14:44.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.809+0000 7f24fb400640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24e80a3c30 0x7f24e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:44.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.809+0000 7f24fb400640 1 -- 172.21.15.86:0/4005424077 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f24f40d08c0 con 0x7f25041083e0 2024-04-17T07:14:44.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.809+0000 7f2501e00640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24e80a3c30 0x7f24e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:44.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.810+0000 7f2501e00640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24e80a3c30 0x7f24e80a60f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24ec005fd0 tx=0x7f24ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:44.814+0000 7f24fb400640 1 -- 172.21.15.86:0/4005424077 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f24f409cec0 con 0x7f25041083e0 2024-04-17T07:14:45.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.017+0000 7f2508a00640 1 -- 172.21.15.86:0/4005424077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2504062b90 con 0x7f25041083e0 2024-04-17T07:14:45.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.020+0000 7f24fb400640 1 -- 172.21.15.86:0/4005424077 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7f24f409c7c0 con 0x7f25041083e0 2024-04-17T07:14:45.021 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:45.021 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:14:45.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.023+0000 7f2508a00640 1 -- 172.21.15.86:0/4005424077 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24e80a3c30 msgr2=0x7f24e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.023+0000 7f2508a00640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24e80a3c30 0x7f24e80a60f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24ec005fd0 tx=0x7f24ec005850 comp rx=0 tx=0).stop 2024-04-17T07:14:45.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.023+0000 7f2508a00640 1 -- 172.21.15.86:0/4005424077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25041083e0 msgr2=0x7f250419e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.023+0000 7f2508a00640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25041083e0 0x7f250419e5c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f24f400eaa0 tx=0x7f24f400ef60 comp rx=0 tx=0).stop 2024-04-17T07:14:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.023+0000 7f2508a00640 1 -- 172.21.15.86:0/4005424077 shutdown_connections 2024-04-17T07:14:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.023+0000 7f2508a00640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24e80a3c30 0x7f24e80a60f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.023+0000 7f2508a00640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2504109f90 0x7f25041a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.023+0000 7f2508a00640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25041095d0 0x7f250419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.023+0000 7f2508a00640 1 --2- 172.21.15.86:0/4005424077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25041083e0 0x7f250419e5c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.023+0000 7f2508a00640 1 -- 172.21.15.86:0/4005424077 >> 172.21.15.86:0/4005424077 conn(0x7f25040773a0 msgr2=0x7f250410c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.024+0000 7f2508a00640 1 -- 172.21.15.86:0/4005424077 shutdown_connections 2024-04-17T07:14:45.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.024+0000 7f2508a00640 1 -- 172.21.15.86:0/4005424077 wait complete. 2024-04-17T07:14:45.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:14:45.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:14:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.265+0000 7f7ba4200640 1 -- 172.21.15.86:0/1635584487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b9c105d70 msgr2=0x7f7b9c10c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.265+0000 7f7ba4200640 1 --2- 172.21.15.86:0/1635584487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b9c105d70 0x7f7b9c10c600 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ca0 tx=0x7f7b9402c100 comp rx=0 tx=0).stop 2024-04-17T07:14:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.265+0000 7f7ba4200640 1 -- 172.21.15.86:0/1635584487 shutdown_connections 2024-04-17T07:14:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.265+0000 7f7ba4200640 1 --2- 172.21.15.86:0/1635584487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b9c105d70 0x7f7b9c10c600 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.266+0000 7f7ba4200640 1 --2- 172.21.15.86:0/1635584487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b9c1053b0 0x7f7b9c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.266+0000 7f7ba4200640 1 --2- 172.21.15.86:0/1635584487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b9c1041c0 0x7f7b9c1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.266+0000 7f7ba4200640 1 -- 172.21.15.86:0/1635584487 >> 172.21.15.86:0/1635584487 conn(0x7f7b9c0ff750 msgr2=0x7f7b9c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.266+0000 7f7ba4200640 1 -- 172.21.15.86:0/1635584487 shutdown_connections 2024-04-17T07:14:45.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.266+0000 7f7ba4200640 1 -- 172.21.15.86:0/1635584487 wait complete. 2024-04-17T07:14:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba4200640 1 Processor -- start 2024-04-17T07:14:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba4200640 1 -- start start 2024-04-17T07:14:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba4200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b9c1041c0 0x7f7b9c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:45.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba4200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b9c1053b0 0x7f7b9c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba4200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b9c105d70 0x7f7b9c1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba4200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10fd00 con 0x7f7b9c1041c0 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba4200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10fb80 con 0x7f7b9c105d70 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba4200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c10fe80 con 0x7f7b9c1053b0 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b9c1041c0 0x7f7b9c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b9c1041c0 0x7f7b9c19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46174/0 (socket says 172.21.15.86:46174) 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.267+0000 7f7ba1800640 1 -- 172.21.15.86:0/4172030958 learned_addr learned my addr 172.21.15.86:0/4172030958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7ba2200640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b9c105d70 0x7f7b9c1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7ba0e00640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b9c1053b0 0x7f7b9c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7ba1800640 1 -- 172.21.15.86:0/4172030958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b9c1053b0 msgr2=0x7f7b9c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7ba1800640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b9c1053b0 0x7f7b9c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7ba1800640 1 -- 172.21.15.86:0/4172030958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b9c105d70 msgr2=0x7f7b9c1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7ba1800640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b9c105d70 0x7f7b9c1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7ba1800640 1 -- 172.21.15.86:0/4172030958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c1a1f50 con 0x7f7b9c1041c0 2024-04-17T07:14:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7ba1800640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b9c1041c0 0x7f7b9c19a330 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000d770 tx=0x7f7b9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7b9a000640 1 -- 172.21.15.86:0/4172030958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90017070 con 0x7f7b9c1041c0 2024-04-17T07:14:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7b9a000640 1 -- 172.21.15.86:0/4172030958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b90009c30 con 0x7f7b9c1041c0 2024-04-17T07:14:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7ba4200640 1 -- 172.21.15.86:0/4172030958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c1a2240 con 0x7f7b9c1041c0 2024-04-17T07:14:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.268+0000 7f7b9a000640 1 -- 172.21.15.86:0/4172030958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90003d00 con 0x7f7b9c1041c0 2024-04-17T07:14:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.269+0000 7f7ba4200640 1 -- 172.21.15.86:0/4172030958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c1a2790 con 0x7f7b9c1041c0 2024-04-17T07:14:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.270+0000 7f7ba4200640 1 -- 172.21.15.86:0/4172030958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9c10b4e0 con 0x7f7b9c1041c0 2024-04-17T07:14:45.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.272+0000 7f7b9a000640 1 -- 172.21.15.86:0/4172030958 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7b90003ea0 con 0x7f7b9c1041c0 2024-04-17T07:14:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.273+0000 7f7b9a000640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b840a3bb0 0x7f7b840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.273+0000 7f7b9a000640 1 -- 172.21.15.86:0/4172030958 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7b900d05a0 con 0x7f7b9c1041c0 2024-04-17T07:14:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.273+0000 7f7ba0e00640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b840a3bb0 0x7f7b840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:45.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.274+0000 7f7ba0e00640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b840a3bb0 0x7f7b840a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b88005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:45.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.277+0000 7f7b9a000640 1 -- 172.21.15.86:0/4172030958 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7b9009cba0 con 0x7f7b9c1041c0 2024-04-17T07:14:45.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:45 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4005424077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:45.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.477+0000 7f7ba4200640 1 -- 172.21.15.86:0/4172030958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b9c062b90 con 0x7f7b9c1041c0 2024-04-17T07:14:45.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.479+0000 7f7b9a000640 1 -- 172.21.15.86:0/4172030958 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f7b90009de0 con 0x7f7b9c1041c0 2024-04-17T07:14:45.479 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:14:45.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 -- 172.21.15.86:0/4172030958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b840a3bb0 msgr2=0x7f7b840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b840a3bb0 0x7f7b840a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b88005850 comp rx=0 tx=0).stop 2024-04-17T07:14:45.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 -- 172.21.15.86:0/4172030958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b9c1041c0 msgr2=0x7f7b9c19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b9c1041c0 0x7f7b9c19a330 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000d770 tx=0x7f7b9000dc30 comp rx=0 tx=0).stop 2024-04-17T07:14:45.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 -- 172.21.15.86:0/4172030958 shutdown_connections 2024-04-17T07:14:45.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7b840a3bb0 0x7f7b840a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b9c105d70 0x7f7b9c1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b9c1053b0 0x7f7b9c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 --2- 172.21.15.86:0/4172030958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b9c1041c0 0x7f7b9c19a330 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 -- 172.21.15.86:0/4172030958 >> 172.21.15.86:0/4172030958 conn(0x7f7b9c0ff750 msgr2=0x7f7b9c1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:45.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.481+0000 7f7ba4200640 1 -- 172.21.15.86:0/4172030958 shutdown_connections 2024-04-17T07:14:45.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.482+0000 7f7ba4200640 1 -- 172.21.15.86:0/4172030958 wait complete. 2024-04-17T07:14:45.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:14:45.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:45 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4005424077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:45.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:45 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4005424077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.717+0000 7fe3c4e00640 1 -- 172.21.15.86:0/3928170342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0101e70 msgr2=0x7fe3c0102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.717+0000 7fe3c4e00640 1 --2- 172.21.15.86:0/3928170342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0101e70 0x7fe3c0102290 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009900 tx=0x7fe3b002bc40 comp rx=0 tx=0).stop 2024-04-17T07:14:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.717+0000 7fe3c4e00640 1 -- 172.21.15.86:0/3928170342 shutdown_connections 2024-04-17T07:14:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.717+0000 7fe3c4e00640 1 --2- 172.21.15.86:0/3928170342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c0103980 0x7fe3c010a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.717+0000 7fe3c4e00640 1 --2- 172.21.15.86:0/3928170342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c0102fc0 0x7fe3c0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.717+0000 7fe3c4e00640 1 --2- 172.21.15.86:0/3928170342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0101e70 0x7fe3c0102290 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.717+0000 7fe3c4e00640 1 -- 172.21.15.86:0/3928170342 >> 172.21.15.86:0/3928170342 conn(0x7fe3c00fd400 msgr2=0x7fe3c00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.717+0000 7fe3c4e00640 1 -- 172.21.15.86:0/3928170342 shutdown_connections 2024-04-17T07:14:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.717+0000 7fe3c4e00640 1 -- 172.21.15.86:0/3928170342 wait complete. 2024-04-17T07:14:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.718+0000 7fe3c4e00640 1 Processor -- start 2024-04-17T07:14:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.718+0000 7fe3c4e00640 1 -- start start 2024-04-17T07:14:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.718+0000 7fe3c4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0101e70 0x7fe3c019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3c4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c0102fc0 0x7fe3c019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3c4e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c0103980 0x7fe3c01a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3c4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c010da90 con 0x7fe3c0101e70 2024-04-17T07:14:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3c4e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c010d910 con 0x7fe3c0103980 2024-04-17T07:14:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3c4e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c010dc10 con 0x7fe3c0102fc0 2024-04-17T07:14:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0101e70 0x7fe3c019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0101e70 0x7fe3c019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46196/0 (socket says 172.21.15.86:46196) 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3bf400640 1 -- 172.21.15.86:0/1368092900 learned_addr learned my addr 172.21.15.86:0/1368092900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3b6a00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c0102fc0 0x7fe3c019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3bfe00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c0103980 0x7fe3c01a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3bf400640 1 -- 172.21.15.86:0/1368092900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c0102fc0 msgr2=0x7fe3c019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3bf400640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c0102fc0 0x7fe3c019a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3bf400640 1 -- 172.21.15.86:0/1368092900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c0103980 msgr2=0x7fe3c01a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3bf400640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c0103980 0x7fe3c01a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.719+0000 7fe3bf400640 1 -- 172.21.15.86:0/1368092900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c01a1f50 con 0x7fe3c0101e70 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.720+0000 7fe3bf400640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0101e70 0x7fe3c019a330 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0000c00 tx=0x7fe3b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.720+0000 7fe3bcc00640 1 -- 172.21.15.86:0/1368092900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0043020 con 0x7fe3c0101e70 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.720+0000 7fe3bcc00640 1 -- 172.21.15.86:0/1368092900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3b00054c0 con 0x7fe3c0101e70 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.720+0000 7fe3bcc00640 1 -- 172.21.15.86:0/1368092900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0003d80 con 0x7fe3c0101e70 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.720+0000 7fe3c4e00640 1 -- 172.21.15.86:0/1368092900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c01a21e0 con 0x7fe3c0101e70 2024-04-17T07:14:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.720+0000 7fe3c4e00640 1 -- 172.21.15.86:0/1368092900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c01a2800 con 0x7fe3c0101e70 2024-04-17T07:14:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.721+0000 7fe3c4e00640 1 -- 172.21.15.86:0/1368092900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c01a2b20 con 0x7fe3c0101e70 2024-04-17T07:14:45.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.723+0000 7fe3bcc00640 1 -- 172.21.15.86:0/1368092900 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe3b0046020 con 0x7fe3c0101e70 2024-04-17T07:14:45.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.724+0000 7fe3bcc00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3a40a3c30 0x7fe3a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:45.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.725+0000 7fe3bcc00640 1 -- 172.21.15.86:0/1368092900 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe3b00f3150 con 0x7fe3c0101e70 2024-04-17T07:14:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.725+0000 7fe3b6a00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3a40a3c30 0x7fe3a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.726+0000 7fe3b6a00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3a40a3c30 0x7fe3a40a60f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe3c019b840 tx=0x7fe3a800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.727+0000 7fe3bcc00640 1 -- 172.21.15.86:0/1368092900 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe3b004d020 con 0x7fe3c0101e70 2024-04-17T07:14:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.929+0000 7fe3c4e00640 1 -- 172.21.15.86:0/1368092900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3c01a2cf0 con 0x7fe3c0101e70 2024-04-17T07:14:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.931+0000 7fe3bcc00640 1 -- 172.21.15.86:0/1368092900 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fe3b00bf870 con 0x7fe3c0101e70 2024-04-17T07:14:45.930 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:14:45.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 -- 172.21.15.86:0/1368092900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3a40a3c30 msgr2=0x7fe3a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3a40a3c30 0x7fe3a40a60f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe3c019b840 tx=0x7fe3a800a4c0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 -- 172.21.15.86:0/1368092900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0101e70 msgr2=0x7fe3c019a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:45.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0101e70 0x7fe3c019a330 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0000c00 tx=0x7fe3b00042f0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 -- 172.21.15.86:0/1368092900 shutdown_connections 2024-04-17T07:14:45.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3a40a3c30 0x7fe3a40a60f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c0103980 0x7fe3c01a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3c0102fc0 0x7fe3c019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 --2- 172.21.15.86:0/1368092900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c0101e70 0x7fe3c019a330 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:45.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 -- 172.21.15.86:0/1368092900 >> 172.21.15.86:0/1368092900 conn(0x7fe3c00fd400 msgr2=0x7fe3c0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:45.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 -- 172.21.15.86:0/1368092900 shutdown_connections 2024-04-17T07:14:45.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:45.933+0000 7fe3c4e00640 1 -- 172.21.15.86:0/1368092900 wait complete. 2024-04-17T07:14:45.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:14:46.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.168+0000 7fb086e00640 1 -- 172.21.15.86:0/2669357030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080109f70 msgr2=0x7fb080110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:46.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.168+0000 7fb086e00640 1 --2- 172.21.15.86:0/2669357030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080109f70 0x7fb080110800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb078009ca0 tx=0x7fb07802c100 comp rx=0 tx=0).stop 2024-04-17T07:14:46.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.169+0000 7fb086e00640 1 -- 172.21.15.86:0/2669357030 shutdown_connections 2024-04-17T07:14:46.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.169+0000 7fb086e00640 1 --2- 172.21.15.86:0/2669357030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb080109f70 0x7fb080110800 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.169+0000 7fb086e00640 1 --2- 172.21.15.86:0/2669357030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0801095b0 0x7fb080109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.169+0000 7fb086e00640 1 --2- 172.21.15.86:0/2669357030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb0801083c0 0x7fb0801087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.169+0000 7fb086e00640 1 -- 172.21.15.86:0/2669357030 >> 172.21.15.86:0/2669357030 conn(0x7fb080077360 msgr2=0x7fb0800797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:46.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.169+0000 7fb086e00640 1 -- 172.21.15.86:0/2669357030 shutdown_connections 2024-04-17T07:14:46.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.169+0000 7fb086e00640 1 -- 172.21.15.86:0/2669357030 wait complete. 2024-04-17T07:14:46.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb086e00640 1 Processor -- start 2024-04-17T07:14:46.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb086e00640 1 -- start start 2024-04-17T07:14:46.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb086e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0801083c0 0x7fb08019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:46.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb086e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0801095b0 0x7fb08019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:46.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb086e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb080109f70 0x7fb0801a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:46.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb086e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb080113f00 con 0x7fb0801083c0 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb086e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb080113d80 con 0x7fb0801095b0 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb086e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb080114080 con 0x7fb080109f70 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb085a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0801083c0 0x7fb08019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb085a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0801083c0 0x7fb08019e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46218/0 (socket says 172.21.15.86:46218) 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.170+0000 7fb085a00640 1 -- 172.21.15.86:0/2995886651 learned_addr learned my addr 172.21.15.86:0/2995886651 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.171+0000 7fb085a00640 1 -- 172.21.15.86:0/2995886651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb080109f70 msgr2=0x7fb0801a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.171+0000 7fb085a00640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb080109f70 0x7fb0801a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.171+0000 7fb085a00640 1 -- 172.21.15.86:0/2995886651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0801095b0 msgr2=0x7fb08019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.171+0000 7fb085a00640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0801095b0 0x7fb08019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.171+0000 7fb085a00640 1 -- 172.21.15.86:0/2995886651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0801a6180 con 0x7fb0801083c0 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.171+0000 7fb085a00640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0801083c0 0x7fb08019e560 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb07400d950 tx=0x7fb07400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.171+0000 7fb07e000640 1 -- 172.21.15.86:0/2995886651 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074017070 con 0x7fb0801083c0 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.171+0000 7fb07e000640 1 -- 172.21.15.86:0/2995886651 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb074009c30 con 0x7fb0801083c0 2024-04-17T07:14:46.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.171+0000 7fb086e00640 1 -- 172.21.15.86:0/2995886651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0801a6470 con 0x7fb0801083c0 2024-04-17T07:14:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.171+0000 7fb07e000640 1 -- 172.21.15.86:0/2995886651 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07400f620 con 0x7fb0801083c0 2024-04-17T07:14:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.172+0000 7fb086e00640 1 -- 172.21.15.86:0/2995886651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0801a6970 con 0x7fb0801083c0 2024-04-17T07:14:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.173+0000 7fb086e00640 1 -- 172.21.15.86:0/2995886651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08010f6e0 con 0x7fb0801083c0 2024-04-17T07:14:46.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.175+0000 7fb07e000640 1 -- 172.21.15.86:0/2995886651 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb07401a070 con 0x7fb0801083c0 2024-04-17T07:14:46.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.176+0000 7fb07e000640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0680a3bb0 0x7fb0680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:46.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.176+0000 7fb07e000640 1 -- 172.21.15.86:0/2995886651 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb0740d02a0 con 0x7fb0801083c0 2024-04-17T07:14:46.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.176+0000 7fb085000640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0680a3bb0 0x7fb0680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:46.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.177+0000 7fb085000640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0680a3bb0 0x7fb0680a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb08019fa70 tx=0x7fb06c005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:46.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.180+0000 7fb07e000640 1 -- 172.21.15.86:0/2995886651 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb074009de0 con 0x7fb0801083c0 2024-04-17T07:14:46.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.382+0000 7fb086e00640 1 -- 172.21.15.86:0/2995886651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb080062c00 con 0x7fb0801083c0 2024-04-17T07:14:46.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.383+0000 7fb07e000640 1 -- 172.21.15.86:0/2995886651 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fb07409c900 con 0x7fb0801083c0 2024-04-17T07:14:46.383 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:14:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.385+0000 7fb086e00640 1 -- 172.21.15.86:0/2995886651 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0680a3bb0 msgr2=0x7fb0680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:46.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.385+0000 7fb086e00640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0680a3bb0 0x7fb0680a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb08019fa70 tx=0x7fb06c005bf0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.385+0000 7fb086e00640 1 -- 172.21.15.86:0/2995886651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0801083c0 msgr2=0x7fb08019e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.385+0000 7fb086e00640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0801083c0 0x7fb08019e560 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb07400d950 tx=0x7fb07400de10 comp rx=0 tx=0).stop 2024-04-17T07:14:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.386+0000 7fb086e00640 1 -- 172.21.15.86:0/2995886651 shutdown_connections 2024-04-17T07:14:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.386+0000 7fb086e00640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0680a3bb0 0x7fb0680a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.386+0000 7fb086e00640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb080109f70 0x7fb0801a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.386+0000 7fb086e00640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0801095b0 0x7fb08019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.386+0000 7fb086e00640 1 --2- 172.21.15.86:0/2995886651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0801083c0 0x7fb08019e560 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.386+0000 7fb086e00640 1 -- 172.21.15.86:0/2995886651 >> 172.21.15.86:0/2995886651 conn(0x7fb080077360 msgr2=0x7fb08010c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.386+0000 7fb086e00640 1 -- 172.21.15.86:0/2995886651 shutdown_connections 2024-04-17T07:14:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.386+0000 7fb086e00640 1 -- 172.21.15.86:0/2995886651 wait complete. 2024-04-17T07:14:46.404 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:14:46.472 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:46 smithi086 ceph-mon[31733]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:46.472 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:46 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4172030958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:46.472 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:46 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1368092900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:46 smithi098 ceph-mon[35418]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:46.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:46 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4172030958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:46 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1368092900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:46.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:46 smithi103 ceph-mon[35495]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:46.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:46 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4172030958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:46.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:46 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1368092900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.602+0000 7fcf7fe00640 1 -- 172.21.15.86:0/2064315383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf80109d20 msgr2=0x7fcf8010c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.602+0000 7fcf7fe00640 1 --2- 172.21.15.86:0/2064315383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf80109d20 0x7fcf8010c1c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009ca0 tx=0x7fcf7802c190 comp rx=0 tx=0).stop 2024-04-17T07:14:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.602+0000 7fcf7fe00640 1 -- 172.21.15.86:0/2064315383 shutdown_connections 2024-04-17T07:14:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.602+0000 7fcf7fe00640 1 --2- 172.21.15.86:0/2064315383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf80109d20 0x7fcf8010c1c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.602+0000 7fcf7fe00640 1 --2- 172.21.15.86:0/2064315383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf80074960 0x7fcf80074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.602+0000 7fcf7fe00640 1 --2- 172.21.15.86:0/2064315383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf80075f40 0x7fcf80074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.602+0000 7fcf7fe00640 1 -- 172.21.15.86:0/2064315383 >> 172.21.15.86:0/2064315383 conn(0x7fcf800fd400 msgr2=0x7fcf800ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.602+0000 7fcf7fe00640 1 -- 172.21.15.86:0/2064315383 shutdown_connections 2024-04-17T07:14:46.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.602+0000 7fcf7fe00640 1 -- 172.21.15.86:0/2064315383 wait complete. 2024-04-17T07:14:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.603+0000 7fcf7fe00640 1 Processor -- start 2024-04-17T07:14:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.603+0000 7fcf7fe00640 1 -- start start 2024-04-17T07:14:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.603+0000 7fcf7fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf80074960 0x7fcf8019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.603+0000 7fcf7fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf80075f40 0x7fcf8019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.603+0000 7fcf7fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf80109d20 0x7fcf801a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.603+0000 7fcf7fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8010fbd0 con 0x7fcf80075f40 2024-04-17T07:14:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.603+0000 7fcf7fe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8010fa50 con 0x7fcf80109d20 2024-04-17T07:14:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.603+0000 7fcf7fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8010fd50 con 0x7fcf80074960 2024-04-17T07:14:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf76000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf80075f40 0x7fcf8019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf76000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf80075f40 0x7fcf8019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46236/0 (socket says 172.21.15.86:46236) 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf76000640 1 -- 172.21.15.86:0/3805813207 learned_addr learned my addr 172.21.15.86:0/3805813207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf76000640 1 -- 172.21.15.86:0/3805813207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf80074960 msgr2=0x7fcf8019a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf76000640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf80074960 0x7fcf8019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf7f400640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf80109d20 0x7fcf801a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf76000640 1 -- 172.21.15.86:0/3805813207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf80109d20 msgr2=0x7fcf801a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf76000640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf80109d20 0x7fcf801a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf76000640 1 -- 172.21.15.86:0/3805813207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf801a1e40 con 0x7fcf80075f40 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf76000640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf80075f40 0x7fcf8019a760 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800d770 tx=0x7fcf6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf84a00640 1 -- 172.21.15.86:0/3805813207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68017070 con 0x7fcf80075f40 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf7fe00640 1 -- 172.21.15.86:0/3805813207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf801a2130 con 0x7fcf80075f40 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.604+0000 7fcf84a00640 1 -- 172.21.15.86:0/3805813207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf680054d0 con 0x7fcf80075f40 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.605+0000 7fcf84a00640 1 -- 172.21.15.86:0/3805813207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68020060 con 0x7fcf80075f40 2024-04-17T07:14:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.605+0000 7fcf7fe00640 1 -- 172.21.15.86:0/3805813207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf801a2680 con 0x7fcf80075f40 2024-04-17T07:14:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.606+0000 7fcf7fe00640 1 -- 172.21.15.86:0/3805813207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf80108ae0 con 0x7fcf80075f40 2024-04-17T07:14:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.608+0000 7fcf84a00640 1 -- 172.21.15.86:0/3805813207 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcf68004c90 con 0x7fcf80075f40 2024-04-17T07:14:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.609+0000 7fcf84a00640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf640a3bb0 0x7fcf640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.609+0000 7fcf84a00640 1 -- 172.21.15.86:0/3805813207 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcf680d0400 con 0x7fcf80075f40 2024-04-17T07:14:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.609+0000 7fcf7ea00640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf640a3bb0 0x7fcf640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.610+0000 7fcf7ea00640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf640a3bb0 0x7fcf640a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf70005fd0 tx=0x7fcf70005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.613+0000 7fcf84a00640 1 -- 172.21.15.86:0/3805813207 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcf6809ca00 con 0x7fcf80075f40 2024-04-17T07:14:46.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.813+0000 7fcf7fe00640 1 -- 172.21.15.86:0/3805813207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcf80062b60 con 0x7fcf80075f40 2024-04-17T07:14:46.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.814+0000 7fcf84a00640 1 -- 172.21.15.86:0/3805813207 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fcf6809c300 con 0x7fcf80075f40 2024-04-17T07:14:46.814 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:14:46.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.816+0000 7fcf7fe00640 1 -- 172.21.15.86:0/3805813207 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf640a3bb0 msgr2=0x7fcf640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:46.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.816+0000 7fcf7fe00640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf640a3bb0 0x7fcf640a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf70005fd0 tx=0x7fcf70005850 comp rx=0 tx=0).stop 2024-04-17T07:14:46.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.816+0000 7fcf7fe00640 1 -- 172.21.15.86:0/3805813207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf80075f40 msgr2=0x7fcf8019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:46.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.816+0000 7fcf7fe00640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf80075f40 0x7fcf8019a760 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800d770 tx=0x7fcf6800dc30 comp rx=0 tx=0).stop 2024-04-17T07:14:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.817+0000 7fcf7fe00640 1 -- 172.21.15.86:0/3805813207 shutdown_connections 2024-04-17T07:14:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.817+0000 7fcf7fe00640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf640a3bb0 0x7fcf640a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.817+0000 7fcf7fe00640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf80109d20 0x7fcf801a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.817+0000 7fcf7fe00640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf80075f40 0x7fcf8019a760 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.817+0000 7fcf7fe00640 1 --2- 172.21.15.86:0/3805813207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf80074960 0x7fcf8019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.817+0000 7fcf7fe00640 1 -- 172.21.15.86:0/3805813207 >> 172.21.15.86:0/3805813207 conn(0x7fcf800fd400 msgr2=0x7fcf80105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.817+0000 7fcf7fe00640 1 -- 172.21.15.86:0/3805813207 shutdown_connections 2024-04-17T07:14:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:46.817+0000 7fcf7fe00640 1 -- 172.21.15.86:0/3805813207 wait complete. 2024-04-17T07:14:46.834 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:14:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.038+0000 7fd6fa000640 1 -- 172.21.15.86:0/3737807164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f4105b90 msgr2=0x7fd6f410c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.038+0000 7fd6fa000640 1 --2- 172.21.15.86:0/3737807164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f4105b90 0x7fd6f410c420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009ca0 tx=0x7fd6ec02c100 comp rx=0 tx=0).stop 2024-04-17T07:14:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.039+0000 7fd6fa000640 1 -- 172.21.15.86:0/3737807164 shutdown_connections 2024-04-17T07:14:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.039+0000 7fd6fa000640 1 --2- 172.21.15.86:0/3737807164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f4105b90 0x7fd6f410c420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.039+0000 7fd6fa000640 1 --2- 172.21.15.86:0/3737807164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f41051d0 0x7fd6f4105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.039+0000 7fd6fa000640 1 --2- 172.21.15.86:0/3737807164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4103fe0 0x7fd6f4104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.039+0000 7fd6fa000640 1 -- 172.21.15.86:0/3737807164 >> 172.21.15.86:0/3737807164 conn(0x7fd6f40ff570 msgr2=0x7fd6f4101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:47.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.039+0000 7fd6fa000640 1 -- 172.21.15.86:0/3737807164 shutdown_connections 2024-04-17T07:14:47.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.039+0000 7fd6fa000640 1 -- 172.21.15.86:0/3737807164 wait complete. 2024-04-17T07:14:47.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6fa000640 1 Processor -- start 2024-04-17T07:14:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6fa000640 1 -- start start 2024-04-17T07:14:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6fa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f4103fe0 0x7fd6f419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6fa000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f41051d0 0x7fd6f419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6fa000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4105b90 0x7fd6f41a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6fa000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f410fb20 con 0x7fd6f4103fe0 2024-04-17T07:14:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6fa000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f410f9a0 con 0x7fd6f41051d0 2024-04-17T07:14:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6fa000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f410fca0 con 0x7fd6f4105b90 2024-04-17T07:14:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6f8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f4103fe0 0x7fd6f419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6f8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f4103fe0 0x7fd6f419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53966/0 (socket says 172.21.15.86:53966) 2024-04-17T07:14:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.040+0000 7fd6f8c00640 1 -- 172.21.15.86:0/837216978 learned_addr learned my addr 172.21.15.86:0/837216978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.041+0000 7fd6f8c00640 1 -- 172.21.15.86:0/837216978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4105b90 msgr2=0x7fd6f41a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.041+0000 7fd6f8c00640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4105b90 0x7fd6f41a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.041+0000 7fd6f8c00640 1 -- 172.21.15.86:0/837216978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f41051d0 msgr2=0x7fd6f419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.041+0000 7fd6f8c00640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f41051d0 0x7fd6f419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.041+0000 7fd6f8c00640 1 -- 172.21.15.86:0/837216978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f41a1d10 con 0x7fd6f4103fe0 2024-04-17T07:14:47.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.041+0000 7fd6f8c00640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f4103fe0 0x7fd6f419a0f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000c970 tx=0x7fd6e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:47.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.041+0000 7fd6f1600640 1 -- 172.21.15.86:0/837216978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0007c10 con 0x7fd6f4103fe0 2024-04-17T07:14:47.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.041+0000 7fd6f1600640 1 -- 172.21.15.86:0/837216978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6e0007db0 con 0x7fd6f4103fe0 2024-04-17T07:14:47.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.041+0000 7fd6fa000640 1 -- 172.21.15.86:0/837216978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f41a1fa0 con 0x7fd6f4103fe0 2024-04-17T07:14:47.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.041+0000 7fd6f1600640 1 -- 172.21.15.86:0/837216978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000f7d0 con 0x7fd6f4103fe0 2024-04-17T07:14:47.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.042+0000 7fd6fa000640 1 -- 172.21.15.86:0/837216978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f41a24c0 con 0x7fd6f4103fe0 2024-04-17T07:14:47.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.043+0000 7fd6fa000640 1 -- 172.21.15.86:0/837216978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f410b300 con 0x7fd6f4103fe0 2024-04-17T07:14:47.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.045+0000 7fd6f1600640 1 -- 172.21.15.86:0/837216978 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd6e0018480 con 0x7fd6f4103fe0 2024-04-17T07:14:47.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.046+0000 7fd6f1600640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6dc0a3bb0 0x7fd6dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.046+0000 7fd6f1600640 1 -- 172.21.15.86:0/837216978 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd6e00d0b60 con 0x7fd6f4103fe0 2024-04-17T07:14:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.046+0000 7fd6f3e00640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6dc0a3bb0 0x7fd6dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.047+0000 7fd6f3e00640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6dc0a3bb0 0x7fd6dc0a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8005fd0 tx=0x7fd6e8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.051+0000 7fd6f1600640 1 -- 172.21.15.86:0/837216978 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd6e009d250 con 0x7fd6f4103fe0 2024-04-17T07:14:47.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.296+0000 7fd6fa000640 1 -- 172.21.15.86:0/837216978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6f4062c60 con 0x7fd6f4103fe0 2024-04-17T07:14:47.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.298+0000 7fd6f1600640 1 -- 172.21.15.86:0/837216978 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7fd6e009cb50 con 0x7fd6f4103fe0 2024-04-17T07:14:47.298 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:47.298 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:12:31.203694+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[34460,34490,34430,34400,24505]},"id":1}]} 2024-04-17T07:14:47.299 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-04-17T07:14:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.300+0000 7fd6fa000640 1 -- 172.21.15.86:0/837216978 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6dc0a3bb0 msgr2=0x7fd6dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.300+0000 7fd6fa000640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6dc0a3bb0 0x7fd6dc0a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8005fd0 tx=0x7fd6e8005850 comp rx=0 tx=0).stop 2024-04-17T07:14:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.301+0000 7fd6fa000640 1 -- 172.21.15.86:0/837216978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f4103fe0 msgr2=0x7fd6f419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.301+0000 7fd6fa000640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f4103fe0 0x7fd6f419a0f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000c970 tx=0x7fd6e000ce30 comp rx=0 tx=0).stop 2024-04-17T07:14:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.301+0000 7fd6fa000640 1 -- 172.21.15.86:0/837216978 shutdown_connections 2024-04-17T07:14:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.301+0000 7fd6fa000640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd6dc0a3bb0 0x7fd6dc0a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.301+0000 7fd6fa000640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4105b90 0x7fd6f41a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.301+0000 7fd6fa000640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f41051d0 0x7fd6f419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.301+0000 7fd6fa000640 1 --2- 172.21.15.86:0/837216978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6f4103fe0 0x7fd6f419a0f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.301+0000 7fd6fa000640 1 -- 172.21.15.86:0/837216978 >> 172.21.15.86:0/837216978 conn(0x7fd6f40ff570 msgr2=0x7fd6f41084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:47.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.301+0000 7fd6fa000640 1 -- 172.21.15.86:0/837216978 shutdown_connections 2024-04-17T07:14:47.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.301+0000 7fd6fa000640 1 -- 172.21.15.86:0/837216978 wait complete. 2024-04-17T07:14:47.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:14:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:47 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2995886651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:47 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3805813207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:47.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.523+0000 7f5f82a00640 1 -- 172.21.15.86:0/3864666339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7c103fe0 msgr2=0x7f5f7c104400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.523+0000 7f5f82a00640 1 --2- 172.21.15.86:0/3864666339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7c103fe0 0x7f5f7c104400 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009900 tx=0x7f5f7002bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:47.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.523+0000 7f5f82a00640 1 -- 172.21.15.86:0/3864666339 shutdown_connections 2024-04-17T07:14:47.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.523+0000 7f5f82a00640 1 --2- 172.21.15.86:0/3864666339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f7c105b90 0x7f5f7c10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.523+0000 7f5f82a00640 1 --2- 172.21.15.86:0/3864666339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f7c1051d0 0x7f5f7c105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.523+0000 7f5f82a00640 1 --2- 172.21.15.86:0/3864666339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7c103fe0 0x7f5f7c104400 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.523+0000 7f5f82a00640 1 -- 172.21.15.86:0/3864666339 >> 172.21.15.86:0/3864666339 conn(0x7f5f7c0ff570 msgr2=0x7f5f7c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:47.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.524+0000 7f5f82a00640 1 -- 172.21.15.86:0/3864666339 shutdown_connections 2024-04-17T07:14:47.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.524+0000 7f5f82a00640 1 -- 172.21.15.86:0/3864666339 wait complete. 2024-04-17T07:14:47.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f82a00640 1 Processor -- start 2024-04-17T07:14:47.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f82a00640 1 -- start start 2024-04-17T07:14:47.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f82a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f7c103fe0 0x7f5f7c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7c1051d0 0x7f5f7c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f82a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f7c105b90 0x7f5f7c1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f82a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10fb70 con 0x7f5f7c1051d0 2024-04-17T07:14:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f82a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10f9f0 con 0x7f5f7c103fe0 2024-04-17T07:14:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f82a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10fcf0 con 0x7f5f7c105b90 2024-04-17T07:14:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f80c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7c1051d0 0x7f5f7c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f80c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7c1051d0 0x7f5f7c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53988/0 (socket says 172.21.15.86:53988) 2024-04-17T07:14:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.525+0000 7f5f80c00640 1 -- 172.21.15.86:0/2553638609 learned_addr learned my addr 172.21.15.86:0/2553638609 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:47.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.526+0000 7f5f81600640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f7c103fe0 0x7f5f7c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:47.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.526+0000 7f5f80c00640 1 -- 172.21.15.86:0/2553638609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f7c105b90 msgr2=0x7f5f7c1a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.526+0000 7f5f80c00640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f7c105b90 0x7f5f7c1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.526+0000 7f5f80c00640 1 -- 172.21.15.86:0/2553638609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f7c103fe0 msgr2=0x7f5f7c19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.526+0000 7f5f80c00640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f7c103fe0 0x7f5f7c19a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.526+0000 7f5f80c00640 1 -- 172.21.15.86:0/2553638609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c1a1e10 con 0x7f5f7c1051d0 2024-04-17T07:14:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.527+0000 7f5f80c00640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7c1051d0 0x7f5f7c19a730 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800d770 tx=0x7f5f6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.527+0000 7f5f7a000640 1 -- 172.21.15.86:0/2553638609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68017070 con 0x7f5f7c1051d0 2024-04-17T07:14:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.527+0000 7f5f7a000640 1 -- 172.21.15.86:0/2553638609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f680054d0 con 0x7f5f7c1051d0 2024-04-17T07:14:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.527+0000 7f5f82a00640 1 -- 172.21.15.86:0/2553638609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7c1a2100 con 0x7f5f7c1051d0 2024-04-17T07:14:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.527+0000 7f5f7a000640 1 -- 172.21.15.86:0/2553638609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68004cf0 con 0x7f5f7c1051d0 2024-04-17T07:14:47.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.527+0000 7f5f82a00640 1 -- 172.21.15.86:0/2553638609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7c1a2650 con 0x7f5f7c1051d0 2024-04-17T07:14:47.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:47 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2995886651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:47.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:47 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3805813207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:47.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.528+0000 7f5f82a00640 1 -- 172.21.15.86:0/2553638609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f7c10b300 con 0x7f5f7c1051d0 2024-04-17T07:14:47.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.531+0000 7f5f7a000640 1 -- 172.21.15.86:0/2553638609 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5f68004e90 con 0x7f5f7c1051d0 2024-04-17T07:14:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.532+0000 7f5f7a000640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f640a3c30 0x7f5f640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.533+0000 7f5f7a000640 1 -- 172.21.15.86:0/2553638609 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5f680d0280 con 0x7f5f7c1051d0 2024-04-17T07:14:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.533+0000 7f5f81600640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f640a3c30 0x7f5f640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:47.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.534+0000 7f5f81600640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f640a3c30 0x7f5f640a60f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5f7002c240 tx=0x7f5f70005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:47.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.537+0000 7f5f7a000640 1 -- 172.21.15.86:0/2553638609 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5f6809c880 con 0x7f5f7c1051d0 2024-04-17T07:14:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:47 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2995886651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:47 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3805813207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.744+0000 7f5f82a00640 1 -- 172.21.15.86:0/2553638609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f7c062c00 con 0x7f5f7c1051d0 2024-04-17T07:14:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.746+0000 7f5f7a000640 1 -- 172.21.15.86:0/2553638609 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7f5f6809c180 con 0x7f5f7c1051d0 2024-04-17T07:14:47.746 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:47.746 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:14:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 -- 172.21.15.86:0/2553638609 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f640a3c30 msgr2=0x7f5f640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f640a3c30 0x7f5f640a60f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5f7002c240 tx=0x7f5f70005fd0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 -- 172.21.15.86:0/2553638609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7c1051d0 msgr2=0x7f5f7c19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7c1051d0 0x7f5f7c19a730 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800d770 tx=0x7f5f6800dc30 comp rx=0 tx=0).stop 2024-04-17T07:14:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 -- 172.21.15.86:0/2553638609 shutdown_connections 2024-04-17T07:14:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f640a3c30 0x7f5f640a60f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f7c105b90 0x7f5f7c1a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f7c1051d0 0x7f5f7c19a730 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 --2- 172.21.15.86:0/2553638609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f7c103fe0 0x7f5f7c19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 -- 172.21.15.86:0/2553638609 >> 172.21.15.86:0/2553638609 conn(0x7f5f7c0ff570 msgr2=0x7f5f7c1084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 -- 172.21.15.86:0/2553638609 shutdown_connections 2024-04-17T07:14:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.749+0000 7f5f82a00640 1 -- 172.21.15.86:0/2553638609 wait complete. 2024-04-17T07:14:47.768 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:14:47.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:14:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.988+0000 7faac6800640 1 -- 172.21.15.86:0/2236286995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac0109f70 msgr2=0x7faac0110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.988+0000 7faac6800640 1 --2- 172.21.15.86:0/2236286995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac0109f70 0x7faac0110800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faab8009900 tx=0x7faab802bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.988+0000 7faac6800640 1 -- 172.21.15.86:0/2236286995 shutdown_connections 2024-04-17T07:14:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.988+0000 7faac6800640 1 --2- 172.21.15.86:0/2236286995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac0109f70 0x7faac0110800 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.988+0000 7faac6800640 1 --2- 172.21.15.86:0/2236286995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac01095b0 0x7faac0109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.988+0000 7faac6800640 1 --2- 172.21.15.86:0/2236286995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac01083c0 0x7faac01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.988+0000 7faac6800640 1 -- 172.21.15.86:0/2236286995 >> 172.21.15.86:0/2236286995 conn(0x7faac0077360 msgr2=0x7faac00797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.989+0000 7faac6800640 1 -- 172.21.15.86:0/2236286995 shutdown_connections 2024-04-17T07:14:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.989+0000 7faac6800640 1 -- 172.21.15.86:0/2236286995 wait complete. 2024-04-17T07:14:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.989+0000 7faac6800640 1 Processor -- start 2024-04-17T07:14:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac6800640 1 -- start start 2024-04-17T07:14:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac01083c0 0x7faac019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac01095b0 0x7faac019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac6800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac0109f70 0x7faac01a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faac0113f20 con 0x7faac01095b0 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac6800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faac0113da0 con 0x7faac0109f70 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac6800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faac01140a0 con 0x7faac01083c0 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac01095b0 0x7faac019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac01095b0 0x7faac019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54010/0 (socket says 172.21.15.86:54010) 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac4a00640 1 -- 172.21.15.86:0/1860423108 learned_addr learned my addr 172.21.15.86:0/1860423108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.990+0000 7faac5400640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac01083c0 0x7faac019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faac4a00640 1 -- 172.21.15.86:0/1860423108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac01083c0 msgr2=0x7faac019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faac5e00640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac0109f70 0x7faac01a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faac4a00640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac01083c0 0x7faac019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faac4a00640 1 -- 172.21.15.86:0/1860423108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac0109f70 msgr2=0x7faac01a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faac4a00640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac0109f70 0x7faac01a5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faac4a00640 1 -- 172.21.15.86:0/1860423108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac01a6190 con 0x7faac01095b0 2024-04-17T07:14:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faac5400640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac01083c0 0x7faac019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faac4a00640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac01095b0 0x7faac019eab0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faab000eab0 tx=0x7faab000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faabe000640 1 -- 172.21.15.86:0/1860423108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab000ce40 con 0x7faac01095b0 2024-04-17T07:14:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faabe000640 1 -- 172.21.15.86:0/1860423108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faab0011920 con 0x7faac01095b0 2024-04-17T07:14:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faabe000640 1 -- 172.21.15.86:0/1860423108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0010a00 con 0x7faac01095b0 2024-04-17T07:14:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faac6800640 1 -- 172.21.15.86:0/1860423108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac01a6480 con 0x7faac01095b0 2024-04-17T07:14:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.991+0000 7faac6800640 1 -- 172.21.15.86:0/1860423108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac01a6a20 con 0x7faac01095b0 2024-04-17T07:14:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.993+0000 7faac6800640 1 -- 172.21.15.86:0/1860423108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac010f6e0 con 0x7faac01095b0 2024-04-17T07:14:47.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.994+0000 7faabe000640 1 -- 172.21.15.86:0/1860423108 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faab0011ad0 con 0x7faac01095b0 2024-04-17T07:14:47.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.995+0000 7faabe000640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faaa80a3c30 0x7faaa80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:47.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.996+0000 7faabe000640 1 -- 172.21.15.86:0/1860423108 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faab00d0e30 con 0x7faac01095b0 2024-04-17T07:14:47.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.996+0000 7faac5400640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faaa80a3c30 0x7faaa80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:47.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:47.997+0000 7faac5400640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faaa80a3c30 0x7faaa80a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faac0109410 tx=0x7faab4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:48.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.000+0000 7faabe000640 1 -- 172.21.15.86:0/1860423108 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faab0016030 con 0x7faac01095b0 2024-04-17T07:14:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.199+0000 7faac6800640 1 -- 172.21.15.86:0/1860423108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faac0062b60 con 0x7faac01095b0 2024-04-17T07:14:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.200+0000 7faabe000640 1 -- 172.21.15.86:0/1860423108 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7faab009d490 con 0x7faac01095b0 2024-04-17T07:14:48.200 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:14:48.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.202+0000 7faac6800640 1 -- 172.21.15.86:0/1860423108 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faaa80a3c30 msgr2=0x7faaa80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:48.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.202+0000 7faac6800640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faaa80a3c30 0x7faaa80a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faac0109410 tx=0x7faab4005d30 comp rx=0 tx=0).stop 2024-04-17T07:14:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.202+0000 7faac6800640 1 -- 172.21.15.86:0/1860423108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac01095b0 msgr2=0x7faac019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.202+0000 7faac6800640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac01095b0 0x7faac019eab0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faab000eab0 tx=0x7faab000ef70 comp rx=0 tx=0).stop 2024-04-17T07:14:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.203+0000 7faac6800640 1 -- 172.21.15.86:0/1860423108 shutdown_connections 2024-04-17T07:14:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.203+0000 7faac6800640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faaa80a3c30 0x7faaa80a60f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.203+0000 7faac6800640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac0109f70 0x7faac01a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.203+0000 7faac6800640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac01095b0 0x7faac019eab0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.203+0000 7faac6800640 1 --2- 172.21.15.86:0/1860423108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac01083c0 0x7faac019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.203+0000 7faac6800640 1 -- 172.21.15.86:0/1860423108 >> 172.21.15.86:0/1860423108 conn(0x7faac0077360 msgr2=0x7faac010c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.203+0000 7faac6800640 1 -- 172.21.15.86:0/1860423108 shutdown_connections 2024-04-17T07:14:48.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.203+0000 7faac6800640 1 -- 172.21.15.86:0/1860423108 wait complete. 2024-04-17T07:14:48.223 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:14:48.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.436+0000 7ff2a2000640 1 -- 172.21.15.86:0/2715104796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c101050 msgr2=0x7ff29c1014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:48.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.436+0000 7ff2a2000640 1 --2- 172.21.15.86:0/2715104796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c101050 0x7ff29c1014b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009900 tx=0x7ff28c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:48.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.436+0000 7ff2a2000640 1 -- 172.21.15.86:0/2715104796 shutdown_connections 2024-04-17T07:14:48.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.436+0000 7ff2a2000640 1 --2- 172.21.15.86:0/2715104796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29c1019f0 0x7ff29c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.436+0000 7ff2a2000640 1 --2- 172.21.15.86:0/2715104796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c101050 0x7ff29c1014b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.436+0000 7ff2a2000640 1 --2- 172.21.15.86:0/2715104796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff29c1006f0 0x7ff29c100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.436+0000 7ff2a2000640 1 -- 172.21.15.86:0/2715104796 >> 172.21.15.86:0/2715104796 conn(0x7ff29c0fc310 msgr2=0x7ff29c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:48.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.436+0000 7ff2a2000640 1 -- 172.21.15.86:0/2715104796 shutdown_connections 2024-04-17T07:14:48.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.436+0000 7ff2a2000640 1 -- 172.21.15.86:0/2715104796 wait complete. 2024-04-17T07:14:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.437+0000 7ff2a2000640 1 Processor -- start 2024-04-17T07:14:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.437+0000 7ff2a2000640 1 -- start start 2024-04-17T07:14:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29c1006f0 0x7ff29c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff29c101050 0x7ff29c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c1019f0 0x7ff29c1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c10fbd0 con 0x7ff29c1019f0 2024-04-17T07:14:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a2000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c10fa50 con 0x7ff29c1006f0 2024-04-17T07:14:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a2000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c10fd50 con 0x7ff29c101050 2024-04-17T07:14:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c1019f0 0x7ff29c1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:48.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c1019f0 0x7ff29c1a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54038/0 (socket says 172.21.15.86:54038) 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a1600640 1 -- 172.21.15.86:0/1853710941 learned_addr learned my addr 172.21.15.86:0/1853710941 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff29be00640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff29c101050 0x7ff29c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a1600640 1 -- 172.21.15.86:0/1853710941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff29c101050 msgr2=0x7ff29c19a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a1600640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff29c101050 0x7ff29c19a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a1600640 1 -- 172.21.15.86:0/1853710941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29c1006f0 msgr2=0x7ff29c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a0c00640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29c1006f0 0x7ff29c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a1600640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29c1006f0 0x7ff29c19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.438+0000 7ff2a1600640 1 -- 172.21.15.86:0/1853710941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c1a1eb0 con 0x7ff29c1019f0 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.439+0000 7ff2a1600640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c1019f0 0x7ff29c1a1840 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff29400dac0 tx=0x7ff29400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.439+0000 7ff299600640 1 -- 172.21.15.86:0/1853710941 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294017070 con 0x7ff29c1019f0 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.439+0000 7ff299600640 1 -- 172.21.15.86:0/1853710941 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff29400be30 con 0x7ff29c1019f0 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.439+0000 7ff299600640 1 -- 172.21.15.86:0/1853710941 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294010460 con 0x7ff29c1019f0 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.439+0000 7ff2a2000640 1 -- 172.21.15.86:0/1853710941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29c1a21a0 con 0x7ff29c1019f0 2024-04-17T07:14:48.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.439+0000 7ff2a2000640 1 -- 172.21.15.86:0/1853710941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29c1a26a0 con 0x7ff29c1019f0 2024-04-17T07:14:48.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.440+0000 7ff2a2000640 1 -- 172.21.15.86:0/1853710941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29c10b610 con 0x7ff29c1019f0 2024-04-17T07:14:48.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.442+0000 7ff299600640 1 -- 172.21.15.86:0/1853710941 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff2940039c0 con 0x7ff29c1019f0 2024-04-17T07:14:48.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.444+0000 7ff299600640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2840a3c30 0x7ff2840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:48.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.444+0000 7ff299600640 1 -- 172.21.15.86:0/1853710941 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff2940d0a20 con 0x7ff29c1019f0 2024-04-17T07:14:48.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.444+0000 7ff2a0c00640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2840a3c30 0x7ff2840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:48.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.445+0000 7ff2a0c00640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2840a3c30 0x7ff2840a60f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff290005fd0 tx=0x7ff290005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:48.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.448+0000 7ff299600640 1 -- 172.21.15.86:0/1853710941 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff294020020 con 0x7ff29c1019f0 2024-04-17T07:14:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:48 smithi098 ceph-mon[35418]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:48 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/837216978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:48 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2553638609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:48 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1860423108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:48 smithi103 ceph-mon[35495]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:48 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/837216978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:48 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2553638609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:48 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1860423108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:48 smithi086 ceph-mon[31733]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:48 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/837216978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:48 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2553638609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:48 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1860423108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:14:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.651+0000 7ff2a2000640 1 -- 172.21.15.86:0/1853710941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff29c062b60 con 0x7ff29c1019f0 2024-04-17T07:14:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.652+0000 7ff299600640 1 -- 172.21.15.86:0/1853710941 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7ff29409d080 con 0x7ff29c1019f0 2024-04-17T07:14:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:14:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 -- 172.21.15.86:0/1853710941 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2840a3c30 msgr2=0x7ff2840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2840a3c30 0x7ff2840a60f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff290005fd0 tx=0x7ff290005850 comp rx=0 tx=0).stop 2024-04-17T07:14:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 -- 172.21.15.86:0/1853710941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c1019f0 msgr2=0x7ff29c1a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c1019f0 0x7ff29c1a1840 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff29400dac0 tx=0x7ff29400df80 comp rx=0 tx=0).stop 2024-04-17T07:14:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 -- 172.21.15.86:0/1853710941 shutdown_connections 2024-04-17T07:14:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2840a3c30 0x7ff2840a60f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29c1019f0 0x7ff29c1a1840 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff29c101050 0x7ff29c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 --2- 172.21.15.86:0/1853710941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29c1006f0 0x7ff29c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 -- 172.21.15.86:0/1853710941 >> 172.21.15.86:0/1853710941 conn(0x7ff29c0fc310 msgr2=0x7ff29c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 -- 172.21.15.86:0/1853710941 shutdown_connections 2024-04-17T07:14:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.654+0000 7ff2a2000640 1 -- 172.21.15.86:0/1853710941 wait complete. 2024-04-17T07:14:48.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:14:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.884+0000 7fbfb6e00640 1 -- 172.21.15.86:0/1267512834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb0105d70 msgr2=0x7fbfb010c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.884+0000 7fbfb6e00640 1 --2- 172.21.15.86:0/1267512834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb0105d70 0x7fbfb010c600 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009900 tx=0x7fbfa002bc50 comp rx=0 tx=0).stop 2024-04-17T07:14:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.884+0000 7fbfb6e00640 1 -- 172.21.15.86:0/1267512834 shutdown_connections 2024-04-17T07:14:48.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.884+0000 7fbfb6e00640 1 --2- 172.21.15.86:0/1267512834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb0105d70 0x7fbfb010c600 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.884+0000 7fbfb6e00640 1 --2- 172.21.15.86:0/1267512834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfb01053b0 0x7fbfb0105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.884+0000 7fbfb6e00640 1 --2- 172.21.15.86:0/1267512834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfb01041c0 0x7fbfb01045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.884+0000 7fbfb6e00640 1 -- 172.21.15.86:0/1267512834 >> 172.21.15.86:0/1267512834 conn(0x7fbfb00ff750 msgr2=0x7fbfb0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.884+0000 7fbfb6e00640 1 -- 172.21.15.86:0/1267512834 shutdown_connections 2024-04-17T07:14:48.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.884+0000 7fbfb6e00640 1 -- 172.21.15.86:0/1267512834 wait complete. 2024-04-17T07:14:48.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.885+0000 7fbfb6e00640 1 Processor -- start 2024-04-17T07:14:48.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.885+0000 7fbfb6e00640 1 -- start start 2024-04-17T07:14:48.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.885+0000 7fbfb6e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfb01041c0 0x7fbfb019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:48.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.885+0000 7fbfb6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb01053b0 0x7fbfb019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:48.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfb6e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfb0105d70 0x7fbfb01a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:48.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfb6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb010fd50 con 0x7fbfb01053b0 2024-04-17T07:14:48.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfb6e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb010fbd0 con 0x7fbfb01041c0 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfb6e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb010fed0 con 0x7fbfb0105d70 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfaf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb01053b0 0x7fbfb019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfaf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb01053b0 0x7fbfb019a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54048/0 (socket says 172.21.15.86:54048) 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfaf400640 1 -- 172.21.15.86:0/3313316141 learned_addr learned my addr 172.21.15.86:0/3313316141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfafe00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfb01041c0 0x7fbfb019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfb4e00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfb0105d70 0x7fbfb01a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfaf400640 1 -- 172.21.15.86:0/3313316141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfb0105d70 msgr2=0x7fbfb01a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfaf400640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfb0105d70 0x7fbfb01a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfaf400640 1 -- 172.21.15.86:0/3313316141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfb01041c0 msgr2=0x7fbfb019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfaf400640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfb01041c0 0x7fbfb019a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.886+0000 7fbfaf400640 1 -- 172.21.15.86:0/3313316141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb01a1f00 con 0x7fbfb01053b0 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.887+0000 7fbfaf400640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb01053b0 0x7fbfb019a820 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c970 tx=0x7fbf9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:48.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.887+0000 7fbfacc00640 1 -- 172.21.15.86:0/3313316141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98007bb0 con 0x7fbfb01053b0 2024-04-17T07:14:48.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.887+0000 7fbfacc00640 1 -- 172.21.15.86:0/3313316141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf98007d50 con 0x7fbfb01053b0 2024-04-17T07:14:48.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.887+0000 7fbfacc00640 1 -- 172.21.15.86:0/3313316141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800f450 con 0x7fbfb01053b0 2024-04-17T07:14:48.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.887+0000 7fbfb6e00640 1 -- 172.21.15.86:0/3313316141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb01a21f0 con 0x7fbfb01053b0 2024-04-17T07:14:48.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.887+0000 7fbfb6e00640 1 -- 172.21.15.86:0/3313316141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb01a26f0 con 0x7fbfb01053b0 2024-04-17T07:14:48.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.888+0000 7fbfb6e00640 1 -- 172.21.15.86:0/3313316141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb010b480 con 0x7fbfb01053b0 2024-04-17T07:14:48.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.891+0000 7fbfacc00640 1 -- 172.21.15.86:0/3313316141 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbf98004f20 con 0x7fbfb01053b0 2024-04-17T07:14:48.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.893+0000 7fbfacc00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf940a3bb0 0x7fbf940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:48.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.893+0000 7fbfacc00640 1 -- 172.21.15.86:0/3313316141 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbf980d0610 con 0x7fbfb01053b0 2024-04-17T07:14:48.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.893+0000 7fbfafe00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf940a3bb0 0x7fbf940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:48.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.894+0000 7fbfafe00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf940a3bb0 0x7fbf940a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005c40 tx=0x7fbfa8005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:48.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:48.898+0000 7fbfacc00640 1 -- 172.21.15.86:0/3313316141 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbf98014030 con 0x7fbfb01053b0 2024-04-17T07:14:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.098+0000 7fbfb6e00640 1 -- 172.21.15.86:0/3313316141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbfb0062b90 con 0x7fbfb01053b0 2024-04-17T07:14:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.099+0000 7fbfacc00640 1 -- 172.21.15.86:0/3313316141 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fbf9809cc10 con 0x7fbfb01053b0 2024-04-17T07:14:49.099 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:14:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 -- 172.21.15.86:0/3313316141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf940a3bb0 msgr2=0x7fbf940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf940a3bb0 0x7fbf940a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005c40 tx=0x7fbfa8005bd0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 -- 172.21.15.86:0/3313316141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb01053b0 msgr2=0x7fbfb019a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb01053b0 0x7fbfb019a820 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c970 tx=0x7fbf9800ce30 comp rx=0 tx=0).stop 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 -- 172.21.15.86:0/3313316141 shutdown_connections 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbf940a3bb0 0x7fbf940a6070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbfb0105d70 0x7fbfb01a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb01053b0 0x7fbfb019a820 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 --2- 172.21.15.86:0/3313316141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbfb01041c0 0x7fbfb019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 -- 172.21.15.86:0/3313316141 >> 172.21.15.86:0/3313316141 conn(0x7fbfb00ff750 msgr2=0x7fbfb0108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 -- 172.21.15.86:0/3313316141 shutdown_connections 2024-04-17T07:14:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.102+0000 7fbfb6e00640 1 -- 172.21.15.86:0/3313316141 wait complete. 2024-04-17T07:14:49.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:14:49.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.342+0000 7f2f8ce00640 1 -- 172.21.15.86:0/908874576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f88105290 msgr2=0x7f2f88105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:49.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.342+0000 7f2f8ce00640 1 --2- 172.21.15.86:0/908874576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f88105290 0x7f2f88105710 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009900 tx=0x7f2f7402bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:49.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.342+0000 7f2f8ce00640 1 -- 172.21.15.86:0/908874576 shutdown_connections 2024-04-17T07:14:49.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.342+0000 7f2f8ce00640 1 --2- 172.21.15.86:0/908874576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f88105c50 0x7f2f8810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.342+0000 7f2f8ce00640 1 --2- 172.21.15.86:0/908874576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f88105290 0x7f2f88105710 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.342+0000 7f2f8ce00640 1 --2- 172.21.15.86:0/908874576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f881040a0 0x7f2f881044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.342+0000 7f2f8ce00640 1 -- 172.21.15.86:0/908874576 >> 172.21.15.86:0/908874576 conn(0x7f2f880ff610 msgr2=0x7f2f88101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:49.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.342+0000 7f2f8ce00640 1 -- 172.21.15.86:0/908874576 shutdown_connections 2024-04-17T07:14:49.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.342+0000 7f2f8ce00640 1 -- 172.21.15.86:0/908874576 wait complete. 2024-04-17T07:14:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.343+0000 7f2f8ce00640 1 Processor -- start 2024-04-17T07:14:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.343+0000 7f2f8ce00640 1 -- start start 2024-04-17T07:14:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f8ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f881040a0 0x7f2f8819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f8ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f88105290 0x7f2f8819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f8ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f88105c50 0x7f2f881a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f8ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8810fc30 con 0x7f2f88105c50 2024-04-17T07:14:49.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f8ce00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8810fab0 con 0x7f2f88105290 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f8ce00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8810fdb0 con 0x7f2f881040a0 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f88105c50 0x7f2f881a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f88105c50 0x7f2f881a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54066/0 (socket says 172.21.15.86:54066) 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f87e00640 1 -- 172.21.15.86:0/1183124580 learned_addr learned my addr 172.21.15.86:0/1183124580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f87400640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f881040a0 0x7f2f8819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f86a00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f88105290 0x7f2f8819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f87e00640 1 -- 172.21.15.86:0/1183124580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f881040a0 msgr2=0x7f2f8819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f87e00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f881040a0 0x7f2f8819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f87e00640 1 -- 172.21.15.86:0/1183124580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f88105290 msgr2=0x7f2f8819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f87e00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f88105290 0x7f2f8819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f87e00640 1 -- 172.21.15.86:0/1183124580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f881a1ed0 con 0x7f2f88105c50 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.344+0000 7f2f86a00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f88105290 0x7f2f8819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.345+0000 7f2f87e00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f88105c50 0x7f2f881a1860 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000eb10 tx=0x7f2f8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:49.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.345+0000 7f2f7fe00640 1 -- 172.21.15.86:0/1183124580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80004b20 con 0x7f2f88105c50 2024-04-17T07:14:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.345+0000 7f2f7fe00640 1 -- 172.21.15.86:0/1183124580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f80004cc0 con 0x7f2f88105c50 2024-04-17T07:14:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.345+0000 7f2f8ce00640 1 -- 172.21.15.86:0/1183124580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f881a21c0 con 0x7f2f88105c50 2024-04-17T07:14:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.345+0000 7f2f7fe00640 1 -- 172.21.15.86:0/1183124580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80010690 con 0x7f2f88105c50 2024-04-17T07:14:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.345+0000 7f2f8ce00640 1 -- 172.21.15.86:0/1183124580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f881a2760 con 0x7f2f88105c50 2024-04-17T07:14:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.347+0000 7f2f8ce00640 1 -- 172.21.15.86:0/1183124580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8810b410 con 0x7f2f88105c50 2024-04-17T07:14:49.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.348+0000 7f2f7fe00640 1 -- 172.21.15.86:0/1183124580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2f80005020 con 0x7f2f88105c50 2024-04-17T07:14:49.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.350+0000 7f2f7fe00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f6c0a3c30 0x7f2f6c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:49.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.350+0000 7f2f7fe00640 1 -- 172.21.15.86:0/1183124580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2f800d0f60 con 0x7f2f88105c50 2024-04-17T07:14:49.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.350+0000 7f2f87400640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f6c0a3c30 0x7f2f6c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:49.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.351+0000 7f2f87400640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f6c0a3c30 0x7f2f6c0a60f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f881050f0 tx=0x7f2f78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:49.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.354+0000 7f2f7fe00640 1 -- 172.21.15.86:0/1183124580 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2f8009d560 con 0x7f2f88105c50 2024-04-17T07:14:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:49 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1853710941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:49 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3313316141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:49 smithi098 ceph-mon[35418]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:49.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:49 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1853710941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:49.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:49 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3313316141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:49.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:49 smithi103 ceph-mon[35495]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.555+0000 7f2f8ce00640 1 -- 172.21.15.86:0/1183124580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2f88062b60 con 0x7f2f88105c50 2024-04-17T07:14:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.556+0000 7f2f7fe00640 1 -- 172.21.15.86:0/1183124580 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f2f8009ce60 con 0x7f2f88105c50 2024-04-17T07:14:49.556 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.558+0000 7f2f8ce00640 1 -- 172.21.15.86:0/1183124580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f6c0a3c30 msgr2=0x7f2f6c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.558+0000 7f2f8ce00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f6c0a3c30 0x7f2f6c0a60f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2f881050f0 tx=0x7f2f78005f90 comp rx=0 tx=0).stop 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.558+0000 7f2f8ce00640 1 -- 172.21.15.86:0/1183124580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f88105c50 msgr2=0x7f2f881a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.558+0000 7f2f8ce00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f88105c50 0x7f2f881a1860 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000eb10 tx=0x7f2f8000c2d0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.559+0000 7f2f8ce00640 1 -- 172.21.15.86:0/1183124580 shutdown_connections 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.559+0000 7f2f8ce00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f6c0a3c30 0x7f2f6c0a60f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.559+0000 7f2f8ce00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f88105c50 0x7f2f881a1860 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.559+0000 7f2f8ce00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f88105290 0x7f2f8819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.559+0000 7f2f8ce00640 1 --2- 172.21.15.86:0/1183124580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f881040a0 0x7f2f8819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.559+0000 7f2f8ce00640 1 -- 172.21.15.86:0/1183124580 >> 172.21.15.86:0/1183124580 conn(0x7f2f880ff610 msgr2=0x7f2f881085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.559+0000 7f2f8ce00640 1 -- 172.21.15.86:0/1183124580 shutdown_connections 2024-04-17T07:14:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.559+0000 7f2f8ce00640 1 -- 172.21.15.86:0/1183124580 wait complete. 2024-04-17T07:14:49.575 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:49 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1853710941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:14:49.575 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:49 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3313316141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:14:49.575 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:49 smithi086 ceph-mon[31733]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:14:49.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:14:49.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.793+0000 7fe668e00640 1 -- 172.21.15.86:0/411916428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664105b90 msgr2=0x7fe66410c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:49.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.793+0000 7fe668e00640 1 --2- 172.21.15.86:0/411916428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664105b90 0x7fe66410c420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009ca0 tx=0x7fe65c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.793+0000 7fe668e00640 1 -- 172.21.15.86:0/411916428 shutdown_connections 2024-04-17T07:14:49.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.793+0000 7fe668e00640 1 --2- 172.21.15.86:0/411916428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664105b90 0x7fe66410c420 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.793+0000 7fe668e00640 1 --2- 172.21.15.86:0/411916428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6641051d0 0x7fe664105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.793+0000 7fe668e00640 1 --2- 172.21.15.86:0/411916428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe664103fe0 0x7fe664104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.793+0000 7fe668e00640 1 -- 172.21.15.86:0/411916428 >> 172.21.15.86:0/411916428 conn(0x7fe6640ff570 msgr2=0x7fe664101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:49.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.794+0000 7fe668e00640 1 -- 172.21.15.86:0/411916428 shutdown_connections 2024-04-17T07:14:49.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.794+0000 7fe668e00640 1 -- 172.21.15.86:0/411916428 wait complete. 2024-04-17T07:14:49.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.794+0000 7fe668e00640 1 Processor -- start 2024-04-17T07:14:49.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.795+0000 7fe668e00640 1 -- start start 2024-04-17T07:14:49.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.795+0000 7fe668e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe664103fe0 0x7fe66419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:49.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.795+0000 7fe668e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6641051d0 0x7fe66419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:49.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.795+0000 7fe668e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664105b90 0x7fe6641a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:49.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.795+0000 7fe668e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe66410fb20 con 0x7fe664105b90 2024-04-17T07:14:49.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.795+0000 7fe668e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe66410f9a0 con 0x7fe6641051d0 2024-04-17T07:14:49.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.795+0000 7fe668e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe66410fca0 con 0x7fe664103fe0 2024-04-17T07:14:49.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.795+0000 7fe663e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664105b90 0x7fe6641a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.795+0000 7fe663e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664105b90 0x7fe6641a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54076/0 (socket says 172.21.15.86:54076) 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.795+0000 7fe663e00640 1 -- 172.21.15.86:0/1541662830 learned_addr learned my addr 172.21.15.86:0/1541662830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe663400640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe664103fe0 0x7fe66419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe65be00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6641051d0 0x7fe66419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe663e00640 1 -- 172.21.15.86:0/1541662830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe664103fe0 msgr2=0x7fe66419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe663e00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe664103fe0 0x7fe66419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe663e00640 1 -- 172.21.15.86:0/1541662830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6641051d0 msgr2=0x7fe66419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe663e00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6641051d0 0x7fe66419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe663e00640 1 -- 172.21.15.86:0/1541662830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6641a1dd0 con 0x7fe664105b90 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe663400640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe664103fe0 0x7fe66419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe663e00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664105b90 0x7fe6641a1760 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe65c032380 tx=0x7fe65c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe660c00640 1 -- 172.21.15.86:0/1541662830 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c041020 con 0x7fe664105b90 2024-04-17T07:14:49.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe660c00640 1 -- 172.21.15.86:0/1541662830 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe65c003e20 con 0x7fe664105b90 2024-04-17T07:14:49.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe660c00640 1 -- 172.21.15.86:0/1541662830 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c039040 con 0x7fe664105b90 2024-04-17T07:14:49.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe668e00640 1 -- 172.21.15.86:0/1541662830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6641a2060 con 0x7fe664105b90 2024-04-17T07:14:49.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.796+0000 7fe668e00640 1 -- 172.21.15.86:0/1541662830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6641a25a0 con 0x7fe664105b90 2024-04-17T07:14:49.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.798+0000 7fe668e00640 1 -- 172.21.15.86:0/1541662830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66410b320 con 0x7fe664105b90 2024-04-17T07:14:49.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.800+0000 7fe660c00640 1 -- 172.21.15.86:0/1541662830 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe65c044020 con 0x7fe664105b90 2024-04-17T07:14:49.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.801+0000 7fe660c00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c30 0x7fe6480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:49.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.801+0000 7fe660c00640 1 -- 172.21.15.86:0/1541662830 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe65c0f2a50 con 0x7fe664105b90 2024-04-17T07:14:49.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.801+0000 7fe663400640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c30 0x7fe6480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:49.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.802+0000 7fe663400640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c30 0x7fe6480a60f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe664105030 tx=0x7fe654005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:49.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:49.805+0000 7fe660c00640 1 -- 172.21.15.86:0/1541662830 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe65c04b020 con 0x7fe664105b90 2024-04-17T07:14:50.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.051+0000 7fe668e00640 1 -- 172.21.15.86:0/1541662830 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe664062b60 con 0x7fe664105b90 2024-04-17T07:14:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.053+0000 7fe660c00640 1 -- 172.21.15.86:0/1541662830 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7fe65c03b2f0 con 0x7fe664105b90 2024-04-17T07:14:50.053 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:50.053 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:12:31.203694+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[34460,34490,34430,34400,24505]},"id":1}]} 2024-04-17T07:14:50.053 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 27 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.055+0000 7fe668e00640 1 -- 172.21.15.86:0/1541662830 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c30 msgr2=0x7fe6480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.055+0000 7fe668e00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c30 0x7fe6480a60f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe664105030 tx=0x7fe654005e70 comp rx=0 tx=0).stop 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.055+0000 7fe668e00640 1 -- 172.21.15.86:0/1541662830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664105b90 msgr2=0x7fe6641a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.055+0000 7fe668e00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664105b90 0x7fe6641a1760 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe65c032380 tx=0x7fe65c032820 comp rx=0 tx=0).stop 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.056+0000 7fe668e00640 1 -- 172.21.15.86:0/1541662830 shutdown_connections 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.056+0000 7fe668e00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c30 0x7fe6480a60f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.056+0000 7fe668e00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664105b90 0x7fe6641a1760 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.056+0000 7fe668e00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6641051d0 0x7fe66419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.056+0000 7fe668e00640 1 --2- 172.21.15.86:0/1541662830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe664103fe0 0x7fe66419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.056+0000 7fe668e00640 1 -- 172.21.15.86:0/1541662830 >> 172.21.15.86:0/1541662830 conn(0x7fe6640ff570 msgr2=0x7fe6641084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.056+0000 7fe668e00640 1 -- 172.21.15.86:0/1541662830 shutdown_connections 2024-04-17T07:14:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.056+0000 7fe668e00640 1 -- 172.21.15.86:0/1541662830 wait complete. 2024-04-17T07:14:50.076 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:14:50.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.273+0000 7fa6e0a00640 1 -- 172.21.15.86:0/1237012674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc1095d0 msgr2=0x7fa6dc109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.273+0000 7fa6e0a00640 1 --2- 172.21.15.86:0/1237012674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc1095d0 0x7fa6dc109a50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009900 tx=0x7fa6c402bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.273+0000 7fa6e0a00640 1 -- 172.21.15.86:0/1237012674 shutdown_connections 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.273+0000 7fa6e0a00640 1 --2- 172.21.15.86:0/1237012674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6dc109f90 0x7fa6dc110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.273+0000 7fa6e0a00640 1 --2- 172.21.15.86:0/1237012674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc1095d0 0x7fa6dc109a50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.273+0000 7fa6e0a00640 1 --2- 172.21.15.86:0/1237012674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6dc1083e0 0x7fa6dc108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.273+0000 7fa6e0a00640 1 -- 172.21.15.86:0/1237012674 >> 172.21.15.86:0/1237012674 conn(0x7fa6dc0773a0 msgr2=0x7fa6dc0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.273+0000 7fa6e0a00640 1 -- 172.21.15.86:0/1237012674 shutdown_connections 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.273+0000 7fa6e0a00640 1 -- 172.21.15.86:0/1237012674 wait complete. 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.274+0000 7fa6e0a00640 1 Processor -- start 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.274+0000 7fa6e0a00640 1 -- start start 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.274+0000 7fa6e0a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6dc1083e0 0x7fa6dc19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.274+0000 7fa6e0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc1095d0 0x7fa6dc19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.274+0000 7fa6e0a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6dc109f90 0x7fa6dc1a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.274+0000 7fa6e0a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc113f70 con 0x7fa6dc1095d0 2024-04-17T07:14:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.274+0000 7fa6e0a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc113df0 con 0x7fa6dc109f90 2024-04-17T07:14:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.274+0000 7fa6e0a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc1140f0 con 0x7fa6dc1083e0 2024-04-17T07:14:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.275+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc1095d0 0x7fa6dc19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.275+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc1095d0 0x7fa6dc19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54102/0 (socket says 172.21.15.86:54102) 2024-04-17T07:14:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.275+0000 7fa6d2a00640 1 -- 172.21.15.86:0/2784559805 learned_addr learned my addr 172.21.15.86:0/2784559805 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.275+0000 7fa6db400640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6dc1083e0 0x7fa6dc19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.275+0000 7fa6d2a00640 1 -- 172.21.15.86:0/2784559805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6dc1083e0 msgr2=0x7fa6dc19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.275+0000 7fa6d2a00640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6dc1083e0 0x7fa6dc19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.275+0000 7fa6d2a00640 1 -- 172.21.15.86:0/2784559805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6dc109f90 msgr2=0x7fa6dc1a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.275+0000 7fa6d2a00640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6dc109f90 0x7fa6dc1a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.275+0000 7fa6d2a00640 1 -- 172.21.15.86:0/2784559805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc1a6220 con 0x7fa6dc1095d0 2024-04-17T07:14:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.275+0000 7fa6d2a00640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc1095d0 0x7fa6dc19eb90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4000c00 tx=0x7fa6c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.276+0000 7fa6d8c00640 1 -- 172.21.15.86:0/2784559805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4043020 con 0x7fa6dc1095d0 2024-04-17T07:14:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.276+0000 7fa6d8c00640 1 -- 172.21.15.86:0/2784559805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6c40054c0 con 0x7fa6dc1095d0 2024-04-17T07:14:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.276+0000 7fa6d8c00640 1 -- 172.21.15.86:0/2784559805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4003d80 con 0x7fa6dc1095d0 2024-04-17T07:14:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.276+0000 7fa6e0a00640 1 -- 172.21.15.86:0/2784559805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc1a64b0 con 0x7fa6dc1095d0 2024-04-17T07:14:50.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.276+0000 7fa6e0a00640 1 -- 172.21.15.86:0/2784559805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc1a6a50 con 0x7fa6dc1095d0 2024-04-17T07:14:50.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.277+0000 7fa6e0a00640 1 -- 172.21.15.86:0/2784559805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6dc10f7d0 con 0x7fa6dc1095d0 2024-04-17T07:14:50.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.279+0000 7fa6d8c00640 1 -- 172.21.15.86:0/2784559805 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa6c4046020 con 0x7fa6dc1095d0 2024-04-17T07:14:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.281+0000 7fa6d8c00640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa6c00a3c30 0x7fa6c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.281+0000 7fa6d8c00640 1 -- 172.21.15.86:0/2784559805 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa6c40f2bf0 con 0x7fa6dc1095d0 2024-04-17T07:14:50.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.281+0000 7fa6db400640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa6c00a3c30 0x7fa6c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.282+0000 7fa6db400640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa6c00a3c30 0x7fa6c00a60f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0058c0 tx=0x7fa6cc005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.286+0000 7fa6d8c00640 1 -- 172.21.15.86:0/2784559805 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa6c404d020 con 0x7fa6dc1095d0 2024-04-17T07:14:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:50 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1183124580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:50 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1541662830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.490+0000 7fa6e0a00640 1 -- 172.21.15.86:0/2784559805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6dc062b90 con 0x7fa6dc1095d0 2024-04-17T07:14:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.492+0000 7fa6d8c00640 1 -- 172.21.15.86:0/2784559805 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7fa6c40bf1f0 con 0x7fa6dc1095d0 2024-04-17T07:14:50.493 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:14:50.493 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:14:50.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 -- 172.21.15.86:0/2784559805 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa6c00a3c30 msgr2=0x7fa6c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa6c00a3c30 0x7fa6c00a60f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0058c0 tx=0x7fa6cc005c80 comp rx=0 tx=0).stop 2024-04-17T07:14:50.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 -- 172.21.15.86:0/2784559805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc1095d0 msgr2=0x7fa6dc19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc1095d0 0x7fa6dc19eb90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4000c00 tx=0x7fa6c40042f0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 -- 172.21.15.86:0/2784559805 shutdown_connections 2024-04-17T07:14:50.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa6c00a3c30 0x7fa6c00a60f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6dc109f90 0x7fa6dc1a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6dc1095d0 0x7fa6dc19eb90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 --2- 172.21.15.86:0/2784559805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6dc1083e0 0x7fa6dc19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 -- 172.21.15.86:0/2784559805 >> 172.21.15.86:0/2784559805 conn(0x7fa6dc0773a0 msgr2=0x7fa6dc10c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 -- 172.21.15.86:0/2784559805 shutdown_connections 2024-04-17T07:14:50.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.495+0000 7fa6e0a00640 1 -- 172.21.15.86:0/2784559805 wait complete. 2024-04-17T07:14:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:50 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1183124580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:50.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:50 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1541662830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:50.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-17T07:11:54.934275+0000', 'modified': '2024-04-17T07:12:31.203694+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34490, 'mds_1': 34460, 'mds_2': 34430, 'mds_3': 34400, 'mds_4': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14688': {'gid': 14688, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/2923053510', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 2923053510}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 2923053510}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14700': {'gid': 14700, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/4256331668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 4256331668}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 4256331668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6833/1707978686', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6832', 'nonce': 1707978686}, {'type': 'v1', 'addr': '172.21.15.103:6833', 'nonce': 1707978686}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6835/423518277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 423518277}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 423518277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6837/299191474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 299191474}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 299191474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.103:6839/2592108290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2592108290}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2592108290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34400': {'gid': 34400, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.98:6833/2318991416', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6832', 'nonce': 2318991416}, {'type': 'v1', 'addr': '172.21.15.98:6833', 'nonce': 2318991416}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34430': {'gid': 34430, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.98:6835/133134430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 133134430}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 133134430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34460': {'gid': 34460, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.98:6837/2794896256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 2794896256}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 2794896256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34490': {'gid': 34490, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6839/809350102', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 809350102}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 809350102}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34490, 'qdb_cluster': [34460, 34490, 34430, 34400, 24505]} 2024-04-17T07:14:50.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-17T07:14:50.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-17T07:14:50.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:50 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1183124580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:14:50.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:50 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1541662830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:14:50.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.713+0000 7f389e800640 1 -- 172.21.15.86:0/2422282279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3898103f70 msgr2=0x7f3898106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.713+0000 7f389e800640 1 --2- 172.21.15.86:0/2422282279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3898103f70 0x7f3898106360 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3884009900 tx=0x7f388402bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:50.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.713+0000 7f389e800640 1 -- 172.21.15.86:0/2422282279 shutdown_connections 2024-04-17T07:14:50.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.713+0000 7f389e800640 1 --2- 172.21.15.86:0/2422282279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3898106980 0x7f3898108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.713+0000 7f389e800640 1 --2- 172.21.15.86:0/2422282279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3898103f70 0x7f3898106360 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.713+0000 7f389e800640 1 --2- 172.21.15.86:0/2422282279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3898101640 0x7f3898103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.713+0000 7f389e800640 1 -- 172.21.15.86:0/2422282279 >> 172.21.15.86:0/2422282279 conn(0x7f38980fb620 msgr2=0x7f38980fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.713+0000 7f389e800640 1 -- 172.21.15.86:0/2422282279 shutdown_connections 2024-04-17T07:14:50.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.713+0000 7f389e800640 1 -- 172.21.15.86:0/2422282279 wait complete. 2024-04-17T07:14:50.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.714+0000 7f389e800640 1 Processor -- start 2024-04-17T07:14:50.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.714+0000 7f389e800640 1 -- start start 2024-04-17T07:14:50.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.714+0000 7f389e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3898101640 0x7f389819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.714+0000 7f389e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3898103f70 0x7f389819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.714+0000 7f389e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3898106980 0x7f38981a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.714+0000 7f389e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f389810c910 con 0x7f3898101640 2024-04-17T07:14:50.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.714+0000 7f389e800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f389810c790 con 0x7f3898103f70 2024-04-17T07:14:50.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.714+0000 7f389e800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f389810ca90 con 0x7f3898106980 2024-04-17T07:14:50.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.714+0000 7f389d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3898101640 0x7f389819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f389d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3898101640 0x7f389819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54110/0 (socket says 172.21.15.86:54110) 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f389d400640 1 -- 172.21.15.86:0/2872221663 learned_addr learned my addr 172.21.15.86:0/2872221663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f389ca00640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3898103f70 0x7f389819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f389de00640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3898106980 0x7f38981a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f389d400640 1 -- 172.21.15.86:0/2872221663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3898106980 msgr2=0x7f38981a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f389d400640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3898106980 0x7f38981a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f389d400640 1 -- 172.21.15.86:0/2872221663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3898103f70 msgr2=0x7f389819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f389d400640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3898103f70 0x7f389819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f389d400640 1 -- 172.21.15.86:0/2872221663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38981a1ea0 con 0x7f3898101640 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f389d400640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3898101640 0x7f389819a280 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c970 tx=0x7f388c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.715+0000 7f3896000640 1 -- 172.21.15.86:0/2872221663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c014070 con 0x7f3898101640 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.716+0000 7f3896000640 1 -- 172.21.15.86:0/2872221663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f388c004440 con 0x7f3898101640 2024-04-17T07:14:50.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.716+0000 7f3896000640 1 -- 172.21.15.86:0/2872221663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c003e50 con 0x7f3898101640 2024-04-17T07:14:50.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.716+0000 7f389e800640 1 -- 172.21.15.86:0/2872221663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38981a2190 con 0x7f3898101640 2024-04-17T07:14:50.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.716+0000 7f389e800640 1 -- 172.21.15.86:0/2872221663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38981a2700 con 0x7f3898101640 2024-04-17T07:14:50.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f38a2bd4740 1 Processor -- start 2024-04-17T07:14:50.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f38a2bd4740 1 -- start start 2024-04-17T07:14:50.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f3896000640 1 -- 172.21.15.86:0/2872221663 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f388c017070 con 0x7f3898101640 2024-04-17T07:14:50.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ed4e0 0x55794b4f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 0x55794b4f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55794b453980 con 0x55794b4ecaf0 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55794b3fa220 con 0x55794b4f2650 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55794b44f940 con 0x55794b4ed4e0 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f389de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f389de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4ecf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54116/0 (socket says 172.21.15.86:54116) 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.719+0000 7f389de00640 1 -- 172.21.15.86:0/863689768 learned_addr learned my addr 172.21.15.86:0/863689768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f389d400640 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ed4e0 0x55794b4f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f389ca00640 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 0x55794b4f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f389de00640 1 -- 172.21.15.86:0/863689768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ed4e0 msgr2=0x55794b4f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f389de00640 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ed4e0 0x55794b4f1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f389de00640 1 -- 172.21.15.86:0/863689768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 msgr2=0x55794b4f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f389de00640 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 0x55794b4f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f389de00640 1 -- 172.21.15.86:0/863689768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794b1662d0 con 0x55794b4ecaf0 2024-04-17T07:14:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f389d400640 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ed4e0 0x55794b4f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:50.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f389ca00640 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 0x55794b4f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:50.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f389de00640 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4ecf10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3890009df0 tx=0x7f389000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6a4f287140724f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f388a600640 1 -- 172.21.15.86:0/863689768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890003970 con 0x55794b4ecaf0 2024-04-17T07:14:50.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f3896000640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38800a3dd0 0x7f38800a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f388a600640 1 -- 172.21.15.86:0/863689768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3890004030 con 0x55794b4ecaf0 2024-04-17T07:14:50.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.720+0000 7f388a600640 1 -- 172.21.15.86:0/863689768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890011770 con 0x55794b4ecaf0 2024-04-17T07:14:50.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f3896000640 1 -- 172.21.15.86:0/2872221663 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f388c0d0600 con 0x7f3898101640 2024-04-17T07:14:50.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f389d400640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38800a3dd0 0x7f38800a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f38a2bd4740 1 -- 172.21.15.86:0/863689768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 msgr2=0x55794b4ecf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4ecf10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3890009df0 tx=0x7f389000f730 comp rx=0 tx=0).stop 2024-04-17T07:14:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f389d400640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38800a3dd0 0x7f38800a6290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f388c0ce0f0 tx=0x7f388c0045f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f38a2bd4740 1 -- 172.21.15.86:0/863689768 shutdown_connections 2024-04-17T07:14:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 0x55794b4f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ed4e0 0x55794b4f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/863689768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4ecf10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f38a2bd4740 1 -- 172.21.15.86:0/863689768 >> 172.21.15.86:0/863689768 conn(0x55794b4e8060 msgr2=0x55794b4e8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f38a2bd4740 1 -- 172.21.15.86:0/863689768 shutdown_connections 2024-04-17T07:14:50.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.721+0000 7f38a2bd4740 1 -- 172.21.15.86:0/863689768 wait complete. 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.722+0000 7f38a2bd4740 1 -- start start 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.722+0000 7f38a2bd4740 1 Processor -- start 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.722+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4ecaf0 0x55794b4f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.722+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.722+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4f2650 0x55794b4fe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.722+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55794b4f5600 con 0x55794b4ed4e0 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.722+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55794b40d590 con 0x55794b4ecaf0 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.722+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55794b4f5780 con 0x55794b4f2650 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.722+0000 7f38a2bd4740 10 client.0 _ll_register_callbacks cb 0x55794b165ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f389ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f389ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54132/0 (socket says 172.21.15.86:54132) 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f389ca00640 1 -- 172.21.15.86:0/245961956 learned_addr learned my addr 172.21.15.86:0/245961956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f389de00640 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4ecaf0 0x55794b4f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f389ca00640 1 -- 172.21.15.86:0/245961956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4f2650 msgr2=0x55794b4fe530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f389ca00640 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4f2650 0x55794b4fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f389ca00640 1 -- 172.21.15.86:0/245961956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4ecaf0 msgr2=0x55794b4f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f389ca00640 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4ecaf0 0x55794b4f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f389ca00640 1 -- 172.21.15.86:0/245961956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794b2306b0 con 0x55794b4ed4e0 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f389ca00640 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f74f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3884009900 tx=0x7f3884004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55cbcae41615ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.723+0000 7f387ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f387ea00640 1 -- 172.21.15.86:0/245961956 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884005d50 con 0x55794b4ed4e0 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f38a2bd4740 1 -- 172.21.15.86:0/245961956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3890004030 con 0x55794b4ed4e0 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f387ea00640 1 -- 172.21.15.86:0/245961956 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3884005230 con 0x55794b4ed4e0 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f387ea00640 1 -- 172.21.15.86:0/245961956 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3884005430 con 0x55794b4ed4e0 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f38a2bd4740 10 client.15525 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f38a2bd4740 1 -- 172.21.15.86:0/245961956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55794b501880 con 0x55794b4ed4e0 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f387ea00640 1 -- 172.21.15.86:0/245961956 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3890004030 con 0x55794b4ed4e0 2024-04-17T07:14:50.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f387ea00640 1 -- 172.21.15.86:0/245961956 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7f38840037b0 con 0x55794b4ed4e0 2024-04-17T07:14:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f38a2bd4740 10 client.15525 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f38a2bd4740 10 client.15525 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-17T07:14:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f38a2bd4740 20 client.15525 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55794b503f60 0x55794b5053d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f38a2bd4740 4 client.15525 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:14:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.724+0000 7f38a2bd4740 1 -- 172.21.15.86:0/245961956 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55794b4613a0 con 0x55794b503f60 2024-04-17T07:14:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.725+0000 7f389de00640 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55794b503f60 0x55794b5053d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.726+0000 7f387ea00640 10 client.15525 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:14:50.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.726+0000 7f389de00640 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55794b503f60 0x55794b5053d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f6ed4ebecff042c server_cookie=24aabfa1796bb218 in_seq=0 out_seq=0 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.728+0000 7f387ea00640 1 -- 172.21.15.86:0/245961956 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55794b4613a0 con 0x55794b503f60 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.728+0000 7f387ea00640 10 client.15525 handle_command_reply: tid=0 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.728+0000 7f38a2bd4740 1 client.15525 shutdown 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 1 -- 172.21.15.86:0/245961956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 msgr2=0x55794b4f74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f74f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3884009900 tx=0x7f3884004270 comp rx=0 tx=0).stop 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 1 -- 172.21.15.86:0/245961956 shutdown_connections 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55794b503f60 0x55794b5053d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4f2650 0x55794b4fe530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f74f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/245961956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4ecaf0 0x55794b4f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 1 -- 172.21.15.86:0/245961956 >> 172.21.15.86:0/245961956 conn(0x55794b4e8060 msgr2=0x55794b4e73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f387ea00640 0 client.15525 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:14:50.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 1 -- 172.21.15.86:0/245961956 shutdown_connections 2024-04-17T07:14:50.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 1 -- 172.21.15.86:0/245961956 wait complete. 2024-04-17T07:14:50.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.729+0000 7f38a2bd4740 20 client.15525 trim_cache size 0 max 16384 2024-04-17T07:14:50.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.749+0000 7f38a2bd4740 1 Processor -- start 2024-04-17T07:14:50.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.749+0000 7f38a2bd4740 1 -- start start 2024-04-17T07:14:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.749+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.749+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4ed4e0 0x55794b4fe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4f2650 0x55794b4e3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55794b503710 con 0x55794b4ecaf0 2024-04-17T07:14:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55794b503de0 con 0x55794b4ed4e0 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55794b5044b0 con 0x55794b4f2650 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f389de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f389de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54144/0 (socket says 172.21.15.86:54144) 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f389de00640 1 -- 172.21.15.86:0/929057428 learned_addr learned my addr 172.21.15.86:0/929057428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f389de00640 1 -- 172.21.15.86:0/929057428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4f2650 msgr2=0x55794b4e3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f389de00640 1 --2- 172.21.15.86:0/929057428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4f2650 0x55794b4e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f389de00640 1 -- 172.21.15.86:0/929057428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4ed4e0 msgr2=0x55794b4fe4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f389ca00640 1 --2- 172.21.15.86:0/929057428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4ed4e0 0x55794b4fe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f389de00640 1 --2- 172.21.15.86:0/929057428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4ed4e0 0x55794b4fe4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.750+0000 7f389de00640 1 -- 172.21.15.86:0/929057428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794b4ed8c0 con 0x55794b4ecaf0 2024-04-17T07:14:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.751+0000 7f389de00640 1 --2- 172.21.15.86:0/929057428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4f4a40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3890004920 tx=0x7f389000ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d8f3fbe192ee74 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.751+0000 7f3889c00640 1 -- 172.21.15.86:0/929057428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389002a070 con 0x55794b4ecaf0 2024-04-17T07:14:50.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.751+0000 7f3889c00640 1 -- 172.21.15.86:0/929057428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3890004e00 con 0x55794b4ecaf0 2024-04-17T07:14:50.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.751+0000 7f3889c00640 1 -- 172.21.15.86:0/929057428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890011890 con 0x55794b4ecaf0 2024-04-17T07:14:50.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 -- 172.21.15.86:0/929057428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 msgr2=0x55794b4f4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/929057428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4f4a40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3890004920 tx=0x7f389000ffd0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 -- 172.21.15.86:0/929057428 shutdown_connections 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/929057428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4f2650 0x55794b4e3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/929057428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4ed4e0 0x55794b4fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/929057428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ecaf0 0x55794b4f4a40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 -- 172.21.15.86:0/929057428 >> 172.21.15.86:0/929057428 conn(0x55794b4e8060 msgr2=0x55794b4f5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 -- 172.21.15.86:0/929057428 shutdown_connections 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 -- 172.21.15.86:0/929057428 wait complete. 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 -- start start 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.752+0000 7f38a2bd4740 1 Processor -- start 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ecaf0 0x55794b4f9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f38a2bd4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 0x55794b4fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55794b505250 con 0x55794b4ed4e0 2024-04-17T07:14:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55794b504b80 con 0x55794b4f2650 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f38a2bd4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55794b505920 con 0x55794b4ecaf0 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f38a2bd4740 10 client.0 _ll_register_callbacks cb 0x7f3884009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f389ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f389ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54146/0 (socket says 172.21.15.86:54146) 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f389ca00640 1 -- 172.21.15.86:0/3624945857 learned_addr learned my addr 172.21.15.86:0/3624945857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f389de00640 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ecaf0 0x55794b4f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f389ca00640 1 -- 172.21.15.86:0/3624945857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ecaf0 msgr2=0x55794b4f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f389ca00640 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ecaf0 0x55794b4f9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f389ca00640 1 -- 172.21.15.86:0/3624945857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 msgr2=0x55794b4fcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f389de00640 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 0x55794b4fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f389ca00640 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 0x55794b4fcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.753+0000 7f389ca00640 1 -- 172.21.15.86:0/3624945857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794b4fd340 con 0x55794b4ed4e0 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.754+0000 7f389ca00640 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f9d60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3884006010 tx=0x7f3884005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9ad438e95e4956 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.754+0000 7f387ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.754+0000 7f387ea00640 1 -- 172.21.15.86:0/3624945857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884005d50 con 0x55794b4ed4e0 2024-04-17T07:14:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.754+0000 7f38a2bd4740 1 -- 172.21.15.86:0/3624945857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3890004e00 con 0x55794b4ed4e0 2024-04-17T07:14:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.754+0000 7f387ea00640 1 -- 172.21.15.86:0/3624945857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f388403e430 con 0x55794b4ed4e0 2024-04-17T07:14:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.754+0000 7f387ea00640 1 -- 172.21.15.86:0/3624945857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3884009520 con 0x55794b4ed4e0 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.754+0000 7f38a2bd4740 10 client.15531 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.754+0000 7f38a2bd4740 1 -- 172.21.15.86:0/3624945857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55794b4fe6e0 con 0x55794b4ed4e0 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.754+0000 7f387ea00640 1 -- 172.21.15.86:0/3624945857 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3890004e00 con 0x55794b4ed4e0 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.755+0000 7f387ea00640 1 -- 172.21.15.86:0/3624945857 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7f388403e8c0 con 0x55794b4ed4e0 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.755+0000 7f38a2bd4740 10 client.15531 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.755+0000 7f38a2bd4740 10 client.15531 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.755+0000 7f38a2bd4740 20 client.15531 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.755+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55794b576ff0 0x55794b575d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.755+0000 7f38a2bd4740 4 client.15531 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.755+0000 7f38a2bd4740 1 -- 172.21.15.86:0/3624945857 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "status"}) -- 0x55794b45b210 con 0x55794b576ff0 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.755+0000 7f389de00640 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55794b576ff0 0x55794b575d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.756+0000 7f389de00640 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55794b576ff0 0x55794b575d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de48d1fb8483d9b2 server_cookie=565bcc176ffa6479 in_seq=0 out_seq=0 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.756+0000 7f387ea00640 10 client.15531 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f387ea00640 1 -- 172.21.15.86:0/3624945857 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55794b45b210 con 0x55794b576ff0 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f387ea00640 10 client.15531 handle_command_reply: tid=0 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 client.15531 shutdown 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 -- 172.21.15.86:0/3624945857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 msgr2=0x55794b4f9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f9d60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3884006010 tx=0x7f3884005800 comp rx=0 tx=0).stop 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 -- 172.21.15.86:0/3624945857 shutdown_connections 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55794b576ff0 0x55794b575d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55794b4f2650 0x55794b4fcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55794b4ed4e0 0x55794b4f9d60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f387ea00640 0 client.15531 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 --2- 172.21.15.86:0/3624945857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55794b4ecaf0 0x55794b4f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 -- 172.21.15.86:0/3624945857 >> 172.21.15.86:0/3624945857 conn(0x55794b4e8060 msgr2=0x55794b4e3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 -- 172.21.15.86:0/3624945857 shutdown_connections 2024-04-17T07:14:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.757+0000 7f38a2bd4740 1 -- 172.21.15.86:0/3624945857 wait complete. 2024-04-17T07:14:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f38a2bd4740 20 client.15531 trim_cache size 0 max 16384 2024-04-17T07:14:50.761 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:50.761 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:14:50.761 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-04-17T07:14:50.761 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24505, 2024-04-17T07:14:50.761 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:14:50.761 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:14:50.761 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:14:50.762 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 140.56616603399999, 2024-04-17T07:14:50.762 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-04-17T07:14:50.762 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-17T07:14:50.762 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 91, 2024-04-17T07:14:50.762 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 193.94305123999999 2024-04-17T07:14:50.762 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:50.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 -- 172.21.15.86:0/2872221663 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38800a3dd0 msgr2=0x7f38800a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38800a3dd0 0x7f38800a6290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f388c0ce0f0 tx=0x7f388c0045f0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 -- 172.21.15.86:0/2872221663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3898101640 msgr2=0x7f389819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3898101640 0x7f389819a280 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f388c00c970 tx=0x7f388c00ce30 comp rx=0 tx=0).stop 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 -- 172.21.15.86:0/2872221663 shutdown_connections 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38800a3dd0 0x7f38800a6290 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3898106980 0x7f38981a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3898103f70 0x7f389819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 --2- 172.21.15.86:0/2872221663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3898101640 0x7f389819a280 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 -- 172.21.15.86:0/2872221663 >> 172.21.15.86:0/2872221663 conn(0x7f38980fb620 msgr2=0x7f38981007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 -- 172.21.15.86:0/2872221663 shutdown_connections 2024-04-17T07:14:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.758+0000 7f389e800640 1 -- 172.21.15.86:0/2872221663 wait complete. 2024-04-17T07:14:50.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-17T07:14:50.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.972+0000 7fe08a600640 1 -- 172.21.15.86:0/3667297142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0840749c0 msgr2=0x7fe084074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.972+0000 7fe08a600640 1 --2- 172.21.15.86:0/3667297142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0840749c0 0x7fe084074e40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe070009900 tx=0x7fe07002bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.972+0000 7fe08a600640 1 -- 172.21.15.86:0/3667297142 shutdown_connections 2024-04-17T07:14:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.972+0000 7fe08a600640 1 --2- 172.21.15.86:0/3667297142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe084109d30 0x7fe08410c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.972+0000 7fe08a600640 1 --2- 172.21.15.86:0/3667297142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0840749c0 0x7fe084074e40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.972+0000 7fe08a600640 1 --2- 172.21.15.86:0/3667297142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe084075fa0 0x7fe0840743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.972+0000 7fe08a600640 1 -- 172.21.15.86:0/3667297142 >> 172.21.15.86:0/3667297142 conn(0x7fe0840fd400 msgr2=0x7fe0840ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.972+0000 7fe08a600640 1 -- 172.21.15.86:0/3667297142 shutdown_connections 2024-04-17T07:14:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.972+0000 7fe08a600640 1 -- 172.21.15.86:0/3667297142 wait complete. 2024-04-17T07:14:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.973+0000 7fe08a600640 1 Processor -- start 2024-04-17T07:14:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.973+0000 7fe08a600640 1 -- start start 2024-04-17T07:14:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe08a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0840749c0 0x7fe08419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe08a600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe084075fa0 0x7fe08419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe08a600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe084109d30 0x7fe0841a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe08a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe08410fbf0 con 0x7fe0840749c0 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe08a600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe08410fa70 con 0x7fe084075fa0 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe08a600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe08410fd70 con 0x7fe084109d30 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe089200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0840749c0 0x7fe08419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe089200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0840749c0 0x7fe08419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54168/0 (socket says 172.21.15.86:54168) 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe089200640 1 -- 172.21.15.86:0/1070672338 learned_addr learned my addr 172.21.15.86:0/1070672338 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe089c00640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe084109d30 0x7fe0841a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe083e00640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe084075fa0 0x7fe08419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe089200640 1 -- 172.21.15.86:0/1070672338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe084109d30 msgr2=0x7fe0841a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe089200640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe084109d30 0x7fe0841a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe089200640 1 -- 172.21.15.86:0/1070672338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe084075fa0 msgr2=0x7fe08419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe089200640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe084075fa0 0x7fe08419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.974+0000 7fe089200640 1 -- 172.21.15.86:0/1070672338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0841a1e10 con 0x7fe0840749c0 2024-04-17T07:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.975+0000 7fe089200640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0840749c0 0x7fe08419a1f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe07400eab0 tx=0x7fe07400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.975+0000 7fe081600640 1 -- 172.21.15.86:0/1070672338 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07400ce40 con 0x7fe0840749c0 2024-04-17T07:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.975+0000 7fe081600640 1 -- 172.21.15.86:0/1070672338 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe074011920 con 0x7fe0840749c0 2024-04-17T07:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.975+0000 7fe081600640 1 -- 172.21.15.86:0/1070672338 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074010a00 con 0x7fe0840749c0 2024-04-17T07:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.975+0000 7fe08a600640 1 -- 172.21.15.86:0/1070672338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0841a2100 con 0x7fe0840749c0 2024-04-17T07:14:50.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.975+0000 7fe08a600640 1 -- 172.21.15.86:0/1070672338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0841a2650 con 0x7fe0840749c0 2024-04-17T07:14:50.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.978+0000 7fe08e9b0740 1 Processor -- start 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.978+0000 7fe081600640 1 -- 172.21.15.86:0/1070672338 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe074011ad0 con 0x7fe0840749c0 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.978+0000 7fe08e9b0740 1 -- start start 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c98698f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c986994e0 0x559c9869dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c9869e650 0x559c986a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c985ff980 con 0x559c98698af0 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559c985a6220 con 0x559c9869e650 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559c985fb940 con 0x559c986994e0 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe089c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c98698f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe089c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c98698f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54174/0 (socket says 172.21.15.86:54174) 2024-04-17T07:14:50.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe089c00640 1 -- 172.21.15.86:0/42549533 learned_addr learned my addr 172.21.15.86:0/42549533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe089200640 1 --2- 172.21.15.86:0/42549533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c986994e0 0x559c9869dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe089c00640 1 -- 172.21.15.86:0/42549533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c986994e0 msgr2=0x559c9869dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe089c00640 1 --2- 172.21.15.86:0/42549533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c986994e0 0x559c9869dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe089c00640 1 -- 172.21.15.86:0/42549533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c9869e650 msgr2=0x559c986a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe089c00640 1 --2- 172.21.15.86:0/42549533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c9869e650 0x559c986a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.979+0000 7fe089c00640 1 -- 172.21.15.86:0/42549533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c983122d0 con 0x559c98698af0 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.980+0000 7fe089c00640 1 --2- 172.21.15.86:0/42549533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c98698f10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00cbe0 tx=0x7fe07c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f2bdd9515192df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.980+0000 7fe07a000640 1 -- 172.21.15.86:0/42549533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c005220 con 0x559c98698af0 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.980+0000 7fe081600640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0680a3bb0 0x7fe0680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.980+0000 7fe07a000640 1 -- 172.21.15.86:0/42549533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe07c004c50 con 0x559c98698af0 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.980+0000 7fe07a000640 1 -- 172.21.15.86:0/42549533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0107b0 con 0x559c98698af0 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.980+0000 7fe081600640 1 -- 172.21.15.86:0/1070672338 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe0740d0e70 con 0x7fe0840749c0 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.980+0000 7fe089200640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0680a3bb0 0x7fe0680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe08e9b0740 1 -- 172.21.15.86:0/42549533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 msgr2=0x559c98698f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/42549533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c98698f10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00cbe0 tx=0x7fe07c00e720 comp rx=0 tx=0).stop 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe08e9b0740 1 -- 172.21.15.86:0/42549533 shutdown_connections 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/42549533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c9869e650 0x559c986a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/42549533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c986994e0 0x559c9869dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/42549533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c98698f10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe08e9b0740 1 -- 172.21.15.86:0/42549533 >> 172.21.15.86:0/42549533 conn(0x559c98694060 msgr2=0x559c98694460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe08e9b0740 1 -- 172.21.15.86:0/42549533 shutdown_connections 2024-04-17T07:14:50.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe089200640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0680a3bb0 0x7fe0680a6070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe0740ce960 tx=0x7fe0740094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe08e9b0740 1 -- 172.21.15.86:0/42549533 wait complete. 2024-04-17T07:14:50.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.981+0000 7fe08e9b0740 1 -- start start 2024-04-17T07:14:50.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe08e9b0740 1 Processor -- start 2024-04-17T07:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c986994e0 0x559c986a3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c9869e650 0x559c986aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c986a1600 con 0x559c98698af0 2024-04-17T07:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559c985b9590 con 0x559c986994e0 2024-04-17T07:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559c986a1780 con 0x559c9869e650 2024-04-17T07:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe08e9b0740 10 client.0 _ll_register_callbacks cb 0x559c98311ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe089c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe089c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54176/0 (socket says 172.21.15.86:54176) 2024-04-17T07:14:50.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.982+0000 7fe089c00640 1 -- 172.21.15.86:0/3846712948 learned_addr learned my addr 172.21.15.86:0/3846712948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe089c00640 1 -- 172.21.15.86:0/3846712948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c9869e650 msgr2=0x559c986aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe089c00640 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c9869e650 0x559c986aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe089c00640 1 -- 172.21.15.86:0/3846712948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c986994e0 msgr2=0x559c986a3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe089c00640 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c986994e0 0x559c986a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe089c00640 1 -- 172.21.15.86:0/3846712948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c983dc6b0 con 0x559c98698af0 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe089c00640 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a2ff0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe07c000c00 tx=0x7fe07c004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244cf741e0c6a5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe06ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe06ea00640 1 -- 172.21.15.86:0/3846712948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c005490 con 0x559c98698af0 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe06ea00640 1 -- 172.21.15.86:0/3846712948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe07c010530 con 0x559c98698af0 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe08e9b0740 1 -- 172.21.15.86:0/3846712948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe07c004c50 con 0x559c98698af0 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe06ea00640 1 -- 172.21.15.86:0/3846712948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe07c010730 con 0x559c98698af0 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe08e9b0740 10 client.15543 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:50.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.983+0000 7fe08e9b0740 1 -- 172.21.15.86:0/3846712948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c986ad8e0 con 0x559c98698af0 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.984+0000 7fe06ea00640 1 -- 172.21.15.86:0/3846712948 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe07c004c50 con 0x559c98698af0 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.984+0000 7fe06ea00640 1 -- 172.21.15.86:0/3846712948 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7fe07c01a960 con 0x559c98698af0 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.984+0000 7fe08e9b0740 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.984+0000 7fe08e9b0740 10 client.15543 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.984+0000 7fe08e9b0740 20 client.15543 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.984+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559c986afb60 0x559c986b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.984+0000 7fe08e9b0740 4 client.15543 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.984+0000 7fe08e9b0740 1 -- 172.21.15.86:0/3846712948 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c9860d3a0 con 0x559c986afb60 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.985+0000 7fe083e00640 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559c986afb60 0x559c986b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.986+0000 7fe083e00640 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559c986afb60 0x559c986b0fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b83ab5274bc73e0 server_cookie=93c4474fa73cae98 in_seq=0 out_seq=0 2024-04-17T07:14:50.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.986+0000 7fe06ea00640 10 client.15543 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:14:50.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe06ea00640 1 -- 172.21.15.86:0/3846712948 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c9860d3a0 con 0x559c986afb60 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe06ea00640 10 client.15543 handle_command_reply: tid=0 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 client.15543 shutdown 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 -- 172.21.15.86:0/3846712948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 msgr2=0x559c986a2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a2ff0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe07c000c00 tx=0x7fe07c004ba0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 -- 172.21.15.86:0/3846712948 shutdown_connections 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559c986afb60 0x559c986b0fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c9869e650 0x559c986aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c986994e0 0x559c986a3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/3846712948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a2ff0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 -- 172.21.15.86:0/3846712948 >> 172.21.15.86:0/3846712948 conn(0x559c98694060 msgr2=0x559c986933a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe06ea00640 0 client.15543 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 -- 172.21.15.86:0/3846712948 shutdown_connections 2024-04-17T07:14:50.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 1 -- 172.21.15.86:0/3846712948 wait complete. 2024-04-17T07:14:50.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:50.988+0000 7fe08e9b0740 20 client.15543 trim_cache size 0 max 16384 2024-04-17T07:14:51.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.007+0000 7fe08e9b0740 1 Processor -- start 2024-04-17T07:14:51.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.008+0000 7fe08e9b0740 1 -- start start 2024-04-17T07:14:51.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.008+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c98698af0 0x559c9868f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.008+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c986994e0 0x559c986aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.008+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9869e650 0x559c98694440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.008+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c986af310 con 0x559c9869e650 2024-04-17T07:14:51.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.008+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559c986af9e0 con 0x559c98698af0 2024-04-17T07:14:51.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.008+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559c986b00b0 con 0x559c986994e0 2024-04-17T07:14:51.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe089c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9869e650 0x559c98694440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe089c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c98698af0 0x559c9868f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe089c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c98698af0 0x559c9868f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:35814/0 (socket says 172.21.15.86:35814) 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe089c00640 1 -- 172.21.15.86:0/2477525154 learned_addr learned my addr 172.21.15.86:0/2477525154 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe083e00640 1 --2- 172.21.15.86:0/2477525154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c986994e0 0x559c986aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe089c00640 1 -- 172.21.15.86:0/2477525154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c986994e0 msgr2=0x559c986aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe089c00640 1 --2- 172.21.15.86:0/2477525154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c986994e0 0x559c986aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe089c00640 1 -- 172.21.15.86:0/2477525154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c98698af0 msgr2=0x559c9868f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe089c00640 1 --2- 172.21.15.86:0/2477525154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c98698af0 0x559c9868f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe089c00640 1 -- 172.21.15.86:0/2477525154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c986998c0 con 0x559c9869e650 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.009+0000 7fe089c00640 1 --2- 172.21.15.86:0/2477525154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9869e650 0x559c98694440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00c3f0 tx=0x7fe07c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcc874c065cc940 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe079600640 1 -- 172.21.15.86:0/2477525154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c005490 con 0x559c9869e650 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe079600640 1 -- 172.21.15.86:0/2477525154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe07c022070 con 0x559c9869e650 2024-04-17T07:14:51.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe079600640 1 -- 172.21.15.86:0/2477525154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c027760 con 0x559c9869e650 2024-04-17T07:14:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe08e9b0740 1 -- 172.21.15.86:0/2477525154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9869e650 msgr2=0x559c98694440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/2477525154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9869e650 0x559c98694440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00c3f0 tx=0x7fe07c00cee0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe08e9b0740 1 -- 172.21.15.86:0/2477525154 shutdown_connections 2024-04-17T07:14:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/2477525154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c9869e650 0x559c98694440 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/2477525154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c986994e0 0x559c986aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/2477525154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c98698af0 0x559c9868f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe08e9b0740 1 -- 172.21.15.86:0/2477525154 >> 172.21.15.86:0/2477525154 conn(0x559c98694060 msgr2=0x559c98699f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.010+0000 7fe08e9b0740 1 -- 172.21.15.86:0/2477525154 shutdown_connections 2024-04-17T07:14:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.011+0000 7fe08e9b0740 1 -- 172.21.15.86:0/2477525154 wait complete. 2024-04-17T07:14:51.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.011+0000 7fe08e9b0740 1 -- start start 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.011+0000 7fe08e9b0740 1 Processor -- start 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.011+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.011+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c986994e0 0x559c986a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.011+0000 7fe08e9b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c9869e650 0x559c986a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.011+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c986b0e50 con 0x559c98698af0 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559c986b0780 con 0x559c986994e0 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe08e9b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559c986b1520 con 0x559c9869e650 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe08e9b0740 10 client.0 _ll_register_callbacks cb 0x7fe07c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe089c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe089c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54182/0 (socket says 172.21.15.86:54182) 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe089c00640 1 -- 172.21.15.86:0/1553186168 learned_addr learned my addr 172.21.15.86:0/1553186168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe089c00640 1 -- 172.21.15.86:0/1553186168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c9869e650 msgr2=0x559c986a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe089c00640 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c9869e650 0x559c986a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe089c00640 1 -- 172.21.15.86:0/1553186168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c986994e0 msgr2=0x559c986a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe089c00640 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c986994e0 0x559c986a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe089c00640 1 -- 172.21.15.86:0/1553186168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c986a9ca0 con 0x559c98698af0 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe089c00640 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a6760 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe07c000c00 tx=0x7fe07c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90650883f09261d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.012+0000 7fe06ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.013+0000 7fe06ea00640 1 -- 172.21.15.86:0/1553186168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c004a40 con 0x559c98698af0 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.013+0000 7fe06ea00640 1 -- 172.21.15.86:0/1553186168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe07c0108d0 con 0x559c98698af0 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.013+0000 7fe08e9b0740 1 -- 172.21.15.86:0/1553186168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe07c022070 con 0x559c98698af0 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.013+0000 7fe06ea00640 1 -- 172.21.15.86:0/1553186168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe07c010ad0 con 0x559c98698af0 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.013+0000 7fe08e9b0740 10 client.15549 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.013+0000 7fe08e9b0740 1 -- 172.21.15.86:0/1553186168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c98716060 con 0x559c98698af0 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.013+0000 7fe06ea00640 1 -- 172.21.15.86:0/1553186168 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe07c022070 con 0x559c98698af0 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.013+0000 7fe06ea00640 1 -- 172.21.15.86:0/1553186168 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7fe07c027a20 con 0x559c98698af0 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.014+0000 7fe08e9b0740 10 client.15549 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.014+0000 7fe08e9b0740 10 client.15549 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-17T07:14:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.014+0000 7fe08e9b0740 20 client.15549 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:51.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.014+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559c9871aac0 0x559c987197d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.014+0000 7fe08e9b0740 4 client.15549 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:14:51.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.014+0000 7fe08e9b0740 1 -- 172.21.15.86:0/1553186168 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "status"}) -- 0x559c98607210 con 0x559c9871aac0 2024-04-17T07:14:51.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.014+0000 7fe083e00640 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559c9871aac0 0x559c987197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.015+0000 7fe083e00640 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559c9871aac0 0x559c987197d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=464ccbd09e7fea7a server_cookie=da45380003ada061 in_seq=0 out_seq=0 2024-04-17T07:14:51.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.015+0000 7fe06ea00640 10 client.15549 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:14:51.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe06ea00640 1 -- 172.21.15.86:0/1553186168 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559c98607210 con 0x559c9871aac0 2024-04-17T07:14:51.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe06ea00640 10 client.15549 handle_command_reply: tid=0 2024-04-17T07:14:51.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 client.15549 shutdown 2024-04-17T07:14:51.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 -- 172.21.15.86:0/1553186168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 msgr2=0x559c986a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a6760 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe07c000c00 tx=0x7fe07c00c830 comp rx=0 tx=0).stop 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 -- 172.21.15.86:0/1553186168 shutdown_connections 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559c9871aac0 0x559c987197d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559c9869e650 0x559c986a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c986994e0 0x559c986a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe06ea00640 0 client.15549 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 --2- 172.21.15.86:0/1553186168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c98698af0 0x559c986a6760 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 -- 172.21.15.86:0/1553186168 >> 172.21.15.86:0/1553186168 conn(0x559c98694060 msgr2=0x559c9868f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 -- 172.21.15.86:0/1553186168 shutdown_connections 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.016+0000 7fe08e9b0740 1 -- 172.21.15.86:0/1553186168 wait complete. 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08e9b0740 20 client.15549 trim_cache size 0 max 16384 2024-04-17T07:14:51.019 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 34400, 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 141.84196531800001, 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 91, 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 222.87256138500001 2024-04-17T07:14:51.020 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 -- 172.21.15.86:0/1070672338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0680a3bb0 msgr2=0x7fe0680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0680a3bb0 0x7fe0680a6070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe0740ce960 tx=0x7fe0740094a0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 -- 172.21.15.86:0/1070672338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0840749c0 msgr2=0x7fe08419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0840749c0 0x7fe08419a1f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe07400eab0 tx=0x7fe07400ef70 comp rx=0 tx=0).stop 2024-04-17T07:14:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 -- 172.21.15.86:0/1070672338 shutdown_connections 2024-04-17T07:14:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe0680a3bb0 0x7fe0680a6070 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe084109d30 0x7fe0841a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe084075fa0 0x7fe08419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 --2- 172.21.15.86:0/1070672338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0840749c0 0x7fe08419a1f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 -- 172.21.15.86:0/1070672338 >> 172.21.15.86:0/1070672338 conn(0x7fe0840fd400 msgr2=0x7fe084105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 -- 172.21.15.86:0/1070672338 shutdown_connections 2024-04-17T07:14:51.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.017+0000 7fe08a600640 1 -- 172.21.15.86:0/1070672338 wait complete. 2024-04-17T07:14:51.033 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-17T07:14:51.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.230+0000 7f3e38200640 1 -- 172.21.15.86:0/3036326172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30107ca0 msgr2=0x7f3e3010a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.230+0000 7f3e38200640 1 --2- 172.21.15.86:0/3036326172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30107ca0 0x7f3e3010a0f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009ca0 tx=0x7f3e2802c0f0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.230+0000 7f3e38200640 1 -- 172.21.15.86:0/3036326172 shutdown_connections 2024-04-17T07:14:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.230+0000 7f3e38200640 1 --2- 172.21.15.86:0/3036326172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30107ca0 0x7f3e3010a0f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.230+0000 7f3e38200640 1 --2- 172.21.15.86:0/3036326172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e30105370 0x7f3e30107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.230+0000 7f3e38200640 1 --2- 172.21.15.86:0/3036326172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e30102a40 0x7f3e30104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.230+0000 7f3e38200640 1 -- 172.21.15.86:0/3036326172 >> 172.21.15.86:0/3036326172 conn(0x7f3e300fc430 msgr2=0x7f3e300fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.230+0000 7f3e38200640 1 -- 172.21.15.86:0/3036326172 shutdown_connections 2024-04-17T07:14:51.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.230+0000 7f3e38200640 1 -- 172.21.15.86:0/3036326172 wait complete. 2024-04-17T07:14:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.231+0000 7f3e38200640 1 Processor -- start 2024-04-17T07:14:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.231+0000 7f3e38200640 1 -- start start 2024-04-17T07:14:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e38200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e30102a40 0x7f3e301981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e38200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30105370 0x7f3e301986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e38200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e30107ca0 0x7f3e3019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e38200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3010db00 con 0x7f3e30105370 2024-04-17T07:14:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e38200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3010d980 con 0x7f3e30107ca0 2024-04-17T07:14:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e38200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3010dc80 con 0x7f3e30102a40 2024-04-17T07:14:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e34e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30105370 0x7f3e301986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e34e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30105370 0x7f3e301986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54200/0 (socket says 172.21.15.86:54200) 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e34e00640 1 -- 172.21.15.86:0/3022879602 learned_addr learned my addr 172.21.15.86:0/3022879602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e35800640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e30102a40 0x7f3e301981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e34e00640 1 -- 172.21.15.86:0/3022879602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e30102a40 msgr2=0x7f3e301981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e34e00640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e30102a40 0x7f3e301981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e34e00640 1 -- 172.21.15.86:0/3022879602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e30107ca0 msgr2=0x7f3e3019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e36200640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e30107ca0 0x7f3e3019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e34e00640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e30107ca0 0x7f3e3019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e34e00640 1 -- 172.21.15.86:0/3022879602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3019fdd0 con 0x7f3e30105370 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e35800640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e30102a40 0x7f3e301981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.232+0000 7f3e36200640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e30107ca0 0x7f3e3019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.233+0000 7f3e34e00640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30105370 0x7f3e301986f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00e9c0 tx=0x7f3e1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.233+0000 7f3e2e000640 1 -- 172.21.15.86:0/3022879602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c00cd80 con 0x7f3e30105370 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.233+0000 7f3e2e000640 1 -- 172.21.15.86:0/3022879602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e1c0054d0 con 0x7f3e30105370 2024-04-17T07:14:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.233+0000 7f3e38200640 1 -- 172.21.15.86:0/3022879602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e301a00c0 con 0x7f3e30105370 2024-04-17T07:14:51.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.233+0000 7f3e2e000640 1 -- 172.21.15.86:0/3022879602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c010430 con 0x7f3e30105370 2024-04-17T07:14:51.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.233+0000 7f3e38200640 1 -- 172.21.15.86:0/3022879602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e301a0610 con 0x7f3e30105370 2024-04-17T07:14:51.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.236+0000 7f3e3ae15740 1 Processor -- start 2024-04-17T07:14:51.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.236+0000 7f3e3ae15740 1 -- start start 2024-04-17T07:14:51.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c2c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c2c6e0 0x5630a8c311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c31850 0x5630a8c33ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630a8b8e980 con 0x5630a8c2bcf0 2024-04-17T07:14:51.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5630a8b35220 con 0x5630a8c31850 2024-04-17T07:14:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630a8b8a940 con 0x5630a8c2c6e0 2024-04-17T07:14:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e36200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c2c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e2e000640 1 -- 172.21.15.86:0/3022879602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3e1c004fd0 con 0x7f3e30105370 2024-04-17T07:14:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e36200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c2c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54204/0 (socket says 172.21.15.86:54204) 2024-04-17T07:14:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e36200640 1 -- 172.21.15.86:0/1587268408 learned_addr learned my addr 172.21.15.86:0/1587268408 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e36200640 1 -- 172.21.15.86:0/1587268408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c2c6e0 msgr2=0x5630a8c311e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e36200640 1 --2- 172.21.15.86:0/1587268408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c2c6e0 0x5630a8c311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e36200640 1 -- 172.21.15.86:0/1587268408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c31850 msgr2=0x5630a8c33ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e36200640 1 --2- 172.21.15.86:0/1587268408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c31850 0x5630a8c33ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.237+0000 7f3e36200640 1 -- 172.21.15.86:0/1587268408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630a88a12d0 con 0x5630a8c2bcf0 2024-04-17T07:14:51.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.238+0000 7f3e36200640 1 --2- 172.21.15.86:0/1587268408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c2c110 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3e28004af0 tx=0x7f3e28009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25788377801cd0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.238+0000 7f3e26a00640 1 -- 172.21.15.86:0/1587268408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28039050 con 0x5630a8c2bcf0 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.238+0000 7f3e26a00640 1 -- 172.21.15.86:0/1587268408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e28045070 con 0x5630a8c2bcf0 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.238+0000 7f3e26a00640 1 -- 172.21.15.86:0/1587268408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28003e60 con 0x5630a8c2bcf0 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.238+0000 7f3e2e000640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e140a3bb0 0x7f3e140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e2e000640 1 -- 172.21.15.86:0/3022879602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3e1c0d0cb0 con 0x7f3e30105370 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e35800640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e140a3bb0 0x7f3e140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e3ae15740 1 -- 172.21.15.86:0/1587268408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 msgr2=0x5630a8c2c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/1587268408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c2c110 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3e28004af0 tx=0x7f3e28009800 comp rx=0 tx=0).stop 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e3ae15740 1 -- 172.21.15.86:0/1587268408 shutdown_connections 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/1587268408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c31850 0x5630a8c33ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/1587268408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c2c6e0 0x5630a8c311e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/1587268408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c2c110 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e3ae15740 1 -- 172.21.15.86:0/1587268408 >> 172.21.15.86:0/1587268408 conn(0x5630a8c27210 msgr2=0x5630a8c27610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e3ae15740 1 -- 172.21.15.86:0/1587268408 shutdown_connections 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e3ae15740 1 -- 172.21.15.86:0/1587268408 wait complete. 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.239+0000 7f3e35800640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e140a3bb0 0x7f3e140a6070 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x5630a8c281d0 tx=0x7f3e200093b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.240+0000 7f3e3ae15740 1 -- start start 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.240+0000 7f3e3ae15740 1 Processor -- start 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.240+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c2bcf0 0x5630a8c36200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.240+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2c6e0 0x5630a8c36740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.240+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c31850 0x5630a8c3d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.240+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630a8c347a0 con 0x5630a8c2c6e0 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.240+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5630a8b48590 con 0x5630a8c2bcf0 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.240+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630a8c34920 con 0x5630a8c31850 2024-04-17T07:14:51.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.240+0000 7f3e3ae15740 10 client.0 _ll_register_callbacks cb 0x5630a88a0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e34e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2c6e0 0x5630a8c36740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e34e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2c6e0 0x5630a8c36740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54220/0 (socket says 172.21.15.86:54220) 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e34e00640 1 -- 172.21.15.86:0/3642642098 learned_addr learned my addr 172.21.15.86:0/3642642098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e35800640 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c2bcf0 0x5630a8c36200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e34e00640 1 -- 172.21.15.86:0/3642642098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c31850 msgr2=0x5630a8c3d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e34e00640 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c31850 0x5630a8c3d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e34e00640 1 -- 172.21.15.86:0/3642642098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c2bcf0 msgr2=0x5630a8c36200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e34e00640 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c2bcf0 0x5630a8c36200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e34e00640 1 -- 172.21.15.86:0/3642642098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630a896b6b0 con 0x5630a8c2c6e0 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e34e00640 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2c6e0 0x5630a8c36740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c09cba0 tx=0x7f3e1c09c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b0c553f0156a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e1b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e1b400640 1 -- 172.21.15.86:0/3642642098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c09d310 con 0x5630a8c2c6e0 2024-04-17T07:14:51.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e1b400640 1 -- 172.21.15.86:0/3642642098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3e1c0bd070 con 0x5630a8c2c6e0 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.241+0000 7f3e3ae15740 1 -- 172.21.15.86:0/3642642098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e28045070 con 0x5630a8c2c6e0 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e1b400640 1 -- 172.21.15.86:0/3642642098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e1c0bd290 con 0x5630a8c2c6e0 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e3ae15740 10 client.15561 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e3ae15740 1 -- 172.21.15.86:0/3642642098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5630a8c40c10 con 0x5630a8c2c6e0 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e1b400640 1 -- 172.21.15.86:0/3642642098 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e28045070 con 0x5630a8c2c6e0 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e1b400640 1 -- 172.21.15.86:0/3642642098 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7f3e1c0a53f0 con 0x5630a8c2c6e0 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e3ae15740 10 client.15561 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e3ae15740 10 client.15561 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e3ae15740 20 client.15561 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630a8c43230 0x5630a8c446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e3ae15740 4 client.15561 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.242+0000 7f3e3ae15740 1 -- 172.21.15.86:0/3642642098 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630a8b9c3a0 con 0x5630a8c43230 2024-04-17T07:14:51.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.243+0000 7f3e35800640 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630a8c43230 0x5630a8c446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.244+0000 7f3e35800640 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630a8c43230 0x5630a8c446a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4ea45e38d7f5b35 server_cookie=42e5a275d88461ae in_seq=0 out_seq=0 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.244+0000 7f3e1b400640 10 client.15561 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.246+0000 7f3e1b400640 1 -- 172.21.15.86:0/3642642098 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630a8b9c3a0 con 0x5630a8c43230 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.246+0000 7f3e1b400640 10 client.15561 handle_command_reply: tid=0 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.246+0000 7f3e3ae15740 1 client.15561 shutdown 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.246+0000 7f3e3ae15740 1 -- 172.21.15.86:0/3642642098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2c6e0 msgr2=0x5630a8c36740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.246+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2c6e0 0x5630a8c36740 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c09cba0 tx=0x7f3e1c09c750 comp rx=0 tx=0).stop 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.247+0000 7f3e3ae15740 1 -- 172.21.15.86:0/3642642098 shutdown_connections 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.247+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630a8c43230 0x5630a8c446a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.247+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c31850 0x5630a8c3d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.247+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2c6e0 0x5630a8c36740 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.247+0000 7f3e1b400640 0 client.15561 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:14:51.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.247+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/3642642098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c2bcf0 0x5630a8c36200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.247+0000 7f3e3ae15740 1 -- 172.21.15.86:0/3642642098 >> 172.21.15.86:0/3642642098 conn(0x5630a8c27210 msgr2=0x5630a8c22090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.247+0000 7f3e3ae15740 1 -- 172.21.15.86:0/3642642098 shutdown_connections 2024-04-17T07:14:51.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.247+0000 7f3e3ae15740 1 -- 172.21.15.86:0/3642642098 wait complete. 2024-04-17T07:14:51.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.247+0000 7f3e3ae15740 20 client.15561 trim_cache size 0 max 16384 2024-04-17T07:14:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.267+0000 7f3e3ae15740 1 Processor -- start 2024-04-17T07:14:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.267+0000 7f3e3ae15740 1 -- start start 2024-04-17T07:14:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.267+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.267+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c2c6e0 0x5630a8c3d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.267+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c31850 0x5630a8c354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.267+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630a8c429e0 con 0x5630a8c2bcf0 2024-04-17T07:14:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.267+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5630a8c430b0 con 0x5630a8c31850 2024-04-17T07:14:51.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.267+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630a8c43780 con 0x5630a8c2c6e0 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e35800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e35800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c275f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54234/0 (socket says 172.21.15.86:54234) 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e35800640 1 -- 172.21.15.86:0/71334974 learned_addr learned my addr 172.21.15.86:0/71334974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e36200640 1 --2- 172.21.15.86:0/71334974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c31850 0x5630a8c354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e34e00640 1 --2- 172.21.15.86:0/71334974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c2c6e0 0x5630a8c3d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e35800640 1 -- 172.21.15.86:0/71334974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c2c6e0 msgr2=0x5630a8c3d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e35800640 1 --2- 172.21.15.86:0/71334974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c2c6e0 0x5630a8c3d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e35800640 1 -- 172.21.15.86:0/71334974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c31850 msgr2=0x5630a8c354d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e35800640 1 --2- 172.21.15.86:0/71334974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c31850 0x5630a8c354d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e35800640 1 -- 172.21.15.86:0/71334974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630a8c2e260 con 0x5630a8c2bcf0 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e35800640 1 --2- 172.21.15.86:0/71334974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c275f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x5630a8c44de0 tx=0x7f3e200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c9cbe8a7b0b4db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.268+0000 7f3e26000640 1 -- 172.21.15.86:0/71334974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20004030 con 0x5630a8c2bcf0 2024-04-17T07:14:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.269+0000 7f3e26000640 1 -- 172.21.15.86:0/71334974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e200128a0 con 0x5630a8c2bcf0 2024-04-17T07:14:51.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.269+0000 7f3e26000640 1 -- 172.21.15.86:0/71334974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20026650 con 0x5630a8c2bcf0 2024-04-17T07:14:51.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.269+0000 7f3e3ae15740 1 -- 172.21.15.86:0/71334974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 msgr2=0x5630a8c275f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.269+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/71334974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c275f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x5630a8c44de0 tx=0x7f3e200046c0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.269+0000 7f3e3ae15740 1 -- 172.21.15.86:0/71334974 shutdown_connections 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.269+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/71334974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c31850 0x5630a8c354d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.269+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/71334974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c2c6e0 0x5630a8c3d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.269+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/71334974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c275f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.269+0000 7f3e3ae15740 1 -- 172.21.15.86:0/71334974 >> 172.21.15.86:0/71334974 conn(0x5630a8c27210 msgr2=0x5630a8c36200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.270+0000 7f3e3ae15740 1 -- 172.21.15.86:0/71334974 shutdown_connections 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.270+0000 7f3e3ae15740 1 -- 172.21.15.86:0/71334974 wait complete. 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.270+0000 7f3e3ae15740 1 -- start start 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.270+0000 7f3e3ae15740 1 Processor -- start 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.270+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c3d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.270+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c2c6e0 0x5630a8c3d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.270+0000 7f3e3ae15740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c31850 0x5630a8c39cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.270+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630a8c44520 con 0x5630a8c2bcf0 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.270+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5630a8c43e50 con 0x5630a8c2c6e0 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.270+0000 7f3e3ae15740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630a8c44bf0 con 0x5630a8c31850 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e3ae15740 10 client.0 _ll_register_callbacks cb 0x7f3e1c004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e35800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c3d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e35800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c3d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54250/0 (socket says 172.21.15.86:54250) 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e35800640 1 -- 172.21.15.86:0/2652180337 learned_addr learned my addr 172.21.15.86:0/2652180337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e35800640 1 -- 172.21.15.86:0/2652180337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c31850 msgr2=0x5630a8c39cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e35800640 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c31850 0x5630a8c39cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e35800640 1 -- 172.21.15.86:0/2652180337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c2c6e0 msgr2=0x5630a8c3d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e35800640 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c2c6e0 0x5630a8c3d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e35800640 1 -- 172.21.15.86:0/2652180337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630a8c3a520 con 0x5630a8c2bcf0 2024-04-17T07:14:51.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e35800640 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c3d1f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3e20029040 tx=0x7f3e20026050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d51547117bbf0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.271+0000 7f3e1b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.272+0000 7f3e1b400640 1 -- 172.21.15.86:0/2652180337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20034060 con 0x5630a8c2bcf0 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.272+0000 7f3e1b400640 1 -- 172.21.15.86:0/2652180337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3e20004030 con 0x5630a8c2bcf0 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.272+0000 7f3e3ae15740 1 -- 172.21.15.86:0/2652180337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e200128a0 con 0x5630a8c2bcf0 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.272+0000 7f3e1b400640 1 -- 172.21.15.86:0/2652180337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e2003b050 con 0x5630a8c2bcf0 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.272+0000 7f3e3ae15740 10 client.15567 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.272+0000 7f3e3ae15740 1 -- 172.21.15.86:0/2652180337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5630a8ca9fe0 con 0x5630a8c2bcf0 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.272+0000 7f3e1b400640 1 -- 172.21.15.86:0/2652180337 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e200128a0 con 0x5630a8c2bcf0 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.272+0000 7f3e1b400640 1 -- 172.21.15.86:0/2652180337 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7f3e2000f040 con 0x5630a8c2bcf0 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.272+0000 7f3e3ae15740 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.273+0000 7f3e3ae15740 10 client.15567 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.273+0000 7f3e3ae15740 20 client.15567 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.273+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630a8cae930 0x5630a8cad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.273+0000 7f3e3ae15740 4 client.15567 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:14:51.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.273+0000 7f3e3ae15740 1 -- 172.21.15.86:0/2652180337 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "status"}) -- 0x5630a8b96210 con 0x5630a8cae930 2024-04-17T07:14:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.273+0000 7f3e34e00640 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630a8cae930 0x5630a8cad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.274+0000 7f3e34e00640 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630a8cae930 0x5630a8cad640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=205482b0974ebcfb server_cookie=63c7b9c68bcdf579 in_seq=0 out_seq=0 2024-04-17T07:14:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.274+0000 7f3e1b400640 10 client.15567 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:14:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e1b400640 1 -- 172.21.15.86:0/2652180337 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5630a8b96210 con 0x5630a8cae930 2024-04-17T07:14:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e1b400640 10 client.15567 handle_command_reply: tid=0 2024-04-17T07:14:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e3ae15740 1 client.15567 shutdown 2024-04-17T07:14:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e3ae15740 1 -- 172.21.15.86:0/2652180337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 msgr2=0x5630a8c3d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c3d1f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3e20029040 tx=0x7f3e20026050 comp rx=0 tx=0).stop 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e3ae15740 1 -- 172.21.15.86:0/2652180337 shutdown_connections 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630a8cae930 0x5630a8cad640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5630a8c31850 0x5630a8c39cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5630a8c2c6e0 0x5630a8c3d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e3ae15740 1 --2- 172.21.15.86:0/2652180337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630a8c2bcf0 0x5630a8c3d1f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e1b400640 0 client.15567 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.275+0000 7f3e3ae15740 1 -- 172.21.15.86:0/2652180337 >> 172.21.15.86:0/2652180337 conn(0x5630a8c27210 msgr2=0x5630a8c354d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e3ae15740 1 -- 172.21.15.86:0/2652180337 shutdown_connections 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e3ae15740 1 -- 172.21.15.86:0/2652180337 wait complete. 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e3ae15740 20 client.15567 trim_cache size 0 max 16384 2024-04-17T07:14:51.278 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 34430, 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 143.568140715, 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 91, 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 210.83602656599999 2024-04-17T07:14:51.279 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:51.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e38200640 1 -- 172.21.15.86:0/3022879602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e140a3bb0 msgr2=0x7f3e140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e38200640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e140a3bb0 0x7f3e140a6070 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x5630a8c281d0 tx=0x7f3e200093b0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e38200640 1 -- 172.21.15.86:0/3022879602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30105370 msgr2=0x7f3e301986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e38200640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30105370 0x7f3e301986f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00e9c0 tx=0x7f3e1c00ee80 comp rx=0 tx=0).stop 2024-04-17T07:14:51.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e38200640 1 -- 172.21.15.86:0/3022879602 shutdown_connections 2024-04-17T07:14:51.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e38200640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e140a3bb0 0x7f3e140a6070 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e38200640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e30107ca0 0x7f3e3019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e38200640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e30105370 0x7f3e301986f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e38200640 1 --2- 172.21.15.86:0/3022879602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e30102a40 0x7f3e301981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.276+0000 7f3e38200640 1 -- 172.21.15.86:0/3022879602 >> 172.21.15.86:0/3022879602 conn(0x7f3e300fc430 msgr2=0x7f3e30103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.277+0000 7f3e38200640 1 -- 172.21.15.86:0/3022879602 shutdown_connections 2024-04-17T07:14:51.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.277+0000 7f3e38200640 1 -- 172.21.15.86:0/3022879602 wait complete. 2024-04-17T07:14:51.290 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:51 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2784559805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:51.290 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:51 smithi086 ceph-mon[31733]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:51.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-17T07:14:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:51 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2784559805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:51 smithi098 ceph-mon[35418]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.513+0000 7f98db600640 1 -- 172.21.15.86:0/4244755675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d4105d70 msgr2=0x7f98d410c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.513+0000 7f98db600640 1 --2- 172.21.15.86:0/4244755675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d4105d70 0x7f98d410c600 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009ca0 tx=0x7f98cc02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.513+0000 7f98db600640 1 -- 172.21.15.86:0/4244755675 shutdown_connections 2024-04-17T07:14:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.513+0000 7f98db600640 1 --2- 172.21.15.86:0/4244755675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d4105d70 0x7f98d410c600 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.513+0000 7f98db600640 1 --2- 172.21.15.86:0/4244755675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d41053b0 0x7f98d4105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.513+0000 7f98db600640 1 --2- 172.21.15.86:0/4244755675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d41041c0 0x7f98d41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.513+0000 7f98db600640 1 -- 172.21.15.86:0/4244755675 >> 172.21.15.86:0/4244755675 conn(0x7f98d40ff750 msgr2=0x7f98d4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.514+0000 7f98db600640 1 -- 172.21.15.86:0/4244755675 shutdown_connections 2024-04-17T07:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.514+0000 7f98db600640 1 -- 172.21.15.86:0/4244755675 wait complete. 2024-04-17T07:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.514+0000 7f98db600640 1 Processor -- start 2024-04-17T07:14:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.515+0000 7f98db600640 1 -- start start 2024-04-17T07:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.515+0000 7f98db600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d41041c0 0x7f98d419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.515+0000 7f98db600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d41053b0 0x7f98d419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.515+0000 7f98db600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d4105d70 0x7f98d41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.515+0000 7f98db600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98d410fd00 con 0x7f98d41041c0 2024-04-17T07:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.515+0000 7f98db600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98d410fb80 con 0x7f98d41053b0 2024-04-17T07:14:51.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.515+0000 7f98db600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98d410fe80 con 0x7f98d4105d70 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.515+0000 7f98d8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d41041c0 0x7f98d419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.515+0000 7f98d8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d41041c0 0x7f98d419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54272/0 (socket says 172.21.15.86:54272) 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.515+0000 7f98d8c00640 1 -- 172.21.15.86:0/3715161647 learned_addr learned my addr 172.21.15.86:0/3715161647 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d41053b0 0x7f98d419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d9600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d4105d70 0x7f98d41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d8c00640 1 -- 172.21.15.86:0/3715161647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d4105d70 msgr2=0x7f98d41a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d8c00640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d4105d70 0x7f98d41a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d8c00640 1 -- 172.21.15.86:0/3715161647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d41053b0 msgr2=0x7f98d419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d8c00640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d41053b0 0x7f98d419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d8c00640 1 -- 172.21.15.86:0/3715161647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d41a1fa0 con 0x7f98d41041c0 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d41053b0 0x7f98d419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d9600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d4105d70 0x7f98d41a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d8c00640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d41041c0 0x7f98d419a380 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f98c800d950 tx=0x7f98c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d1600640 1 -- 172.21.15.86:0/3715161647 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8017070 con 0x7f98d41041c0 2024-04-17T07:14:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d1600640 1 -- 172.21.15.86:0/3715161647 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98c8009c30 con 0x7f98d41041c0 2024-04-17T07:14:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98d1600640 1 -- 172.21.15.86:0/3715161647 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c800f620 con 0x7f98d41041c0 2024-04-17T07:14:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98db600640 1 -- 172.21.15.86:0/3715161647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d41a2290 con 0x7f98d41041c0 2024-04-17T07:14:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.516+0000 7f98db600640 1 -- 172.21.15.86:0/3715161647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d41a2790 con 0x7f98d41041c0 2024-04-17T07:14:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.521+0000 7f98de3d6740 1 Processor -- start 2024-04-17T07:14:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.521+0000 7f98de3d6740 1 -- start start 2024-04-17T07:14:51.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.521+0000 7f98d1600640 1 -- 172.21.15.86:0/3715161647 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f98c801a070 con 0x7f98d41041c0 2024-04-17T07:14:51.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.521+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d75c8e0 0x55b49d75cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.521+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d761dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.521+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d7648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.521+0000 7f98de3d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b49d6c3980 con 0x55b49d762440 2024-04-17T07:14:51.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.521+0000 7f98de3d6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b49d66a220 con 0x55b49d75c8e0 2024-04-17T07:14:51.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.521+0000 7f98de3d6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b49d6bf940 con 0x55b49d75d2d0 2024-04-17T07:14:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.522+0000 7f98d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d7648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.522+0000 7f98d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d7648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54276/0 (socket says 172.21.15.86:54276) 2024-04-17T07:14:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.522+0000 7f98d3e00640 1 -- 172.21.15.86:0/3329289351 learned_addr learned my addr 172.21.15.86:0/3329289351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.522+0000 7f98d9600640 1 --2- 172.21.15.86:0/3329289351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d75c8e0 0x55b49d75cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.522+0000 7f98d8c00640 1 --2- 172.21.15.86:0/3329289351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d761dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.522+0000 7f98d3e00640 1 -- 172.21.15.86:0/3329289351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 msgr2=0x55b49d761dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.522+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3329289351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d761dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.522+0000 7f98d3e00640 1 -- 172.21.15.86:0/3329289351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d75c8e0 msgr2=0x55b49d75cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.522+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3329289351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d75c8e0 0x55b49d75cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.522+0000 7f98d3e00640 1 -- 172.21.15.86:0/3329289351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49d3d62d0 con 0x55b49d762440 2024-04-17T07:14:51.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.523+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3329289351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d7648d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009c20 tx=0x7f98c000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6c88d82b68d584 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.523+0000 7f98c5c00640 1 -- 172.21.15.86:0/3329289351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0016070 con 0x55b49d762440 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.523+0000 7f98c5c00640 1 -- 172.21.15.86:0/3329289351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98c0011920 con 0x55b49d762440 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.523+0000 7f98c5c00640 1 -- 172.21.15.86:0/3329289351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c00107a0 con 0x55b49d762440 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98de3d6740 1 -- 172.21.15.86:0/3329289351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 msgr2=0x55b49d7648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98de3d6740 1 --2- 172.21.15.86:0/3329289351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d7648d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009c20 tx=0x7f98c000e6c0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98d1600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98bc0a3bb0 0x7f98bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98de3d6740 1 -- 172.21.15.86:0/3329289351 shutdown_connections 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98de3d6740 1 --2- 172.21.15.86:0/3329289351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d7648d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98de3d6740 1 --2- 172.21.15.86:0/3329289351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d761dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98de3d6740 1 --2- 172.21.15.86:0/3329289351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d75c8e0 0x55b49d75cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98de3d6740 1 -- 172.21.15.86:0/3329289351 >> 172.21.15.86:0/3329289351 conn(0x55b49d757e00 msgr2=0x55b49d758200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98d1600640 1 -- 172.21.15.86:0/3715161647 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f98c80d0300 con 0x7f98d41041c0 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98de3d6740 1 -- 172.21.15.86:0/3329289351 shutdown_connections 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98d9600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98bc0a3bb0 0x7f98bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.524+0000 7f98de3d6740 1 -- 172.21.15.86:0/3329289351 wait complete. 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.525+0000 7f98de3d6740 1 -- start start 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.525+0000 7f98d9600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98bc0a3bb0 0x7f98bc0a6070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f98cc032420 tx=0x7f98cc032bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.525+0000 7f98de3d6740 1 Processor -- start 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.525+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d766ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.525+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d7673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.525+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d762440 0x55b49d76e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.525+0000 7f98de3d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b49d7653f0 con 0x55b49d75c8e0 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.525+0000 7f98de3d6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b49d67d590 con 0x55b49d762440 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.525+0000 7f98de3d6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b49d765570 con 0x55b49d75d2d0 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.525+0000 7f98de3d6740 10 client.0 _ll_register_callbacks cb 0x55b49d3d5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d766ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d766ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54290/0 (socket says 172.21.15.86:54290) 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d8c00640 1 -- 172.21.15.86:0/2596771879 learned_addr learned my addr 172.21.15.86:0/2596771879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d3e00640 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d7673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d9600640 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d762440 0x55b49d76e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d8c00640 1 -- 172.21.15.86:0/2596771879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 msgr2=0x55b49d7673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d8c00640 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d7673e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d8c00640 1 -- 172.21.15.86:0/2596771879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d762440 msgr2=0x55b49d76e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d8c00640 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d762440 0x55b49d76e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d8c00640 1 -- 172.21.15.86:0/2596771879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49d4a06b0 con 0x55b49d75c8e0 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d9600640 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d762440 0x55b49d76e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98d8c00640 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d766ea0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f98c8005fd0 tx=0x7f98c8009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9783d2f941130f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98ba000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.526+0000 7f98ba000640 1 -- 172.21.15.86:0/2596771879 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8021040 con 0x55b49d75c8e0 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98ba000640 1 -- 172.21.15.86:0/2596771879 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f98c80a0150 con 0x55b49d75c8e0 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98de3d6740 1 -- 172.21.15.86:0/2596771879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c0011920 con 0x55b49d75c8e0 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98ba000640 1 -- 172.21.15.86:0/2596771879 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98c80a4e70 con 0x55b49d75c8e0 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98de3d6740 10 client.15579 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98de3d6740 1 -- 172.21.15.86:0/2596771879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b49d771800 con 0x55b49d75c8e0 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98ba000640 1 -- 172.21.15.86:0/2596771879 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98c0011920 con 0x55b49d75c8e0 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98ba000640 1 -- 172.21.15.86:0/2596771879 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7f98c809b0c0 con 0x55b49d75c8e0 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98de3d6740 10 client.15579 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98de3d6740 10 client.15579 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98de3d6740 20 client.15579 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98de3d6740 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55b49d773ec0 0x55b49d775330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98de3d6740 4 client.15579 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:14:51.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.527+0000 7f98de3d6740 1 -- 172.21.15.86:0/2596771879 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b49d6d13a0 con 0x55b49d773ec0 2024-04-17T07:14:51.532 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:51 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2784559805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:14:51.532 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:51 smithi103 ceph-mon[35495]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:51.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.528+0000 7f98d3e00640 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55b49d773ec0 0x55b49d775330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.529+0000 7f98d3e00640 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55b49d773ec0 0x55b49d775330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4542147b55962a31 server_cookie=44a88b0a97dfdd8f in_seq=0 out_seq=0 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.529+0000 7f98ba000640 10 client.15579 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.531+0000 7f98ba000640 1 -- 172.21.15.86:0/2596771879 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b49d6d13a0 con 0x55b49d773ec0 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.531+0000 7f98ba000640 10 client.15579 handle_command_reply: tid=0 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.531+0000 7f98de3d6740 1 client.15579 shutdown 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.531+0000 7f98de3d6740 1 -- 172.21.15.86:0/2596771879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 msgr2=0x55b49d766ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.531+0000 7f98de3d6740 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d766ea0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f98c8005fd0 tx=0x7f98c8009f20 comp rx=0 tx=0).stop 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.531+0000 7f98de3d6740 1 -- 172.21.15.86:0/2596771879 shutdown_connections 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.531+0000 7f98de3d6740 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55b49d773ec0 0x55b49d775330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.532+0000 7f98de3d6740 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d762440 0x55b49d76e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.532+0000 7f98de3d6740 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d7673e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.532+0000 7f98de3d6740 1 --2- 172.21.15.86:0/2596771879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d766ea0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.532+0000 7f98de3d6740 1 -- 172.21.15.86:0/2596771879 >> 172.21.15.86:0/2596771879 conn(0x55b49d757e00 msgr2=0x55b49d757090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.532+0000 7f98ba000640 0 client.15579 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:14:51.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.532+0000 7f98de3d6740 1 -- 172.21.15.86:0/2596771879 shutdown_connections 2024-04-17T07:14:51.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.532+0000 7f98de3d6740 1 -- 172.21.15.86:0/2596771879 wait complete. 2024-04-17T07:14:51.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.532+0000 7f98de3d6740 20 client.15579 trim_cache size 0 max 16384 2024-04-17T07:14:51.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.551+0000 7f98de3d6740 1 Processor -- start 2024-04-17T07:14:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.551+0000 7f98de3d6740 1 -- start start 2024-04-17T07:14:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75c8e0 0x55b49d7652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d75d2d0 0x55b49d76e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d75dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98de3d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b49d773670 con 0x55b49d762440 2024-04-17T07:14:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98de3d6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b49d773d40 con 0x55b49d75d2d0 2024-04-17T07:14:51.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98de3d6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b49d774410 con 0x55b49d75c8e0 2024-04-17T07:14:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98d9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d75dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98d9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d75dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54296/0 (socket says 172.21.15.86:54296) 2024-04-17T07:14:51.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98d9600640 1 -- 172.21.15.86:0/1215767924 learned_addr learned my addr 172.21.15.86:0/1215767924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98d8c00640 1 --2- 172.21.15.86:0/1215767924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75c8e0 0x55b49d7652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.552+0000 7f98d3e00640 1 --2- 172.21.15.86:0/1215767924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d75d2d0 0x55b49d76e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.553+0000 7f98d9600640 1 -- 172.21.15.86:0/1215767924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75c8e0 msgr2=0x55b49d7652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.553+0000 7f98d9600640 1 --2- 172.21.15.86:0/1215767924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75c8e0 0x55b49d7652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.553+0000 7f98d9600640 1 -- 172.21.15.86:0/1215767924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d75d2d0 msgr2=0x55b49d76e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.553+0000 7f98d9600640 1 --2- 172.21.15.86:0/1215767924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d75d2d0 0x55b49d76e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.553+0000 7f98d9600640 1 -- 172.21.15.86:0/1215767924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49d75d6b0 con 0x55b49d762440 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.553+0000 7f98d9600640 1 --2- 172.21.15.86:0/1215767924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d75dd40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009ca0 tx=0x7f98cc03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e34346931fd637 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98c5200640 1 -- 172.21.15.86:0/1215767924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc04f070 con 0x55b49d762440 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98c5200640 1 -- 172.21.15.86:0/1215767924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98cc045040 con 0x55b49d762440 2024-04-17T07:14:51.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98c5200640 1 -- 172.21.15.86:0/1215767924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc039040 con 0x55b49d762440 2024-04-17T07:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98de3d6740 1 -- 172.21.15.86:0/1215767924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 msgr2=0x55b49d75dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98de3d6740 1 --2- 172.21.15.86:0/1215767924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d75dd40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009ca0 tx=0x7f98cc03e000 comp rx=0 tx=0).stop 2024-04-17T07:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98de3d6740 1 -- 172.21.15.86:0/1215767924 shutdown_connections 2024-04-17T07:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98de3d6740 1 --2- 172.21.15.86:0/1215767924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d762440 0x55b49d75dd40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98de3d6740 1 --2- 172.21.15.86:0/1215767924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d75d2d0 0x55b49d76e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98de3d6740 1 --2- 172.21.15.86:0/1215767924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75c8e0 0x55b49d7652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98de3d6740 1 -- 172.21.15.86:0/1215767924 >> 172.21.15.86:0/1215767924 conn(0x55b49d757e00 msgr2=0x55b49d7659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98de3d6740 1 -- 172.21.15.86:0/1215767924 shutdown_connections 2024-04-17T07:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.554+0000 7f98de3d6740 1 -- 172.21.15.86:0/1215767924 wait complete. 2024-04-17T07:14:51.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.555+0000 7f98de3d6740 1 -- start start 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.555+0000 7f98de3d6740 1 Processor -- start 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.555+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d76c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.555+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d7d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.555+0000 7f98de3d6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d762440 0x55b49d7d8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.555+0000 7f98de3d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b49d7751b0 con 0x55b49d75c8e0 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.555+0000 7f98de3d6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b49d774ae0 con 0x55b49d762440 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.555+0000 7f98de3d6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b49d775880 con 0x55b49d75d2d0 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98de3d6740 10 client.0 _ll_register_callbacks cb 0x7f98c80984a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98d8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d76c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98d8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d76c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54308/0 (socket says 172.21.15.86:54308) 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98d8c00640 1 -- 172.21.15.86:0/4028970811 learned_addr learned my addr 172.21.15.86:0/4028970811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98d3e00640 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d7d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98d8c00640 1 -- 172.21.15.86:0/4028970811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 msgr2=0x55b49d7d5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98d8c00640 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d7d5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98d8c00640 1 -- 172.21.15.86:0/4028970811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d762440 msgr2=0x55b49d7d8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98d8c00640 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d762440 0x55b49d7d8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98d8c00640 1 -- 172.21.15.86:0/4028970811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49d7d8d30 con 0x55b49d75c8e0 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98d8c00640 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d76c3d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f98c809b460 tx=0x7f98c809b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4650192bd009d7cb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.556+0000 7f98ba000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98ba000640 1 -- 172.21.15.86:0/4028970811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8014020 con 0x55b49d75c8e0 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98ba000640 1 -- 172.21.15.86:0/4028970811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f98c809b850 con 0x55b49d75c8e0 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98de3d6740 1 -- 172.21.15.86:0/4028970811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98cc045040 con 0x55b49d75c8e0 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98ba000640 1 -- 172.21.15.86:0/4028970811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98c8021040 con 0x55b49d75c8e0 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98de3d6740 10 client.15585 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:51.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98de3d6740 1 -- 172.21.15.86:0/4028970811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b49d7da0a0 con 0x55b49d75c8e0 2024-04-17T07:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98ba000640 1 -- 172.21.15.86:0/4028970811 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98cc045040 con 0x55b49d75c8e0 2024-04-17T07:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98ba000640 1 -- 172.21.15.86:0/4028970811 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7f98c80a1960 con 0x55b49d75c8e0 2024-04-17T07:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98de3d6740 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98de3d6740 10 client.15585 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-17T07:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98de3d6740 20 client.15585 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98de3d6740 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55b49d7dedb0 0x55b49d7ddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98de3d6740 4 client.15585 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:14:51.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.557+0000 7f98de3d6740 1 -- 172.21.15.86:0/4028970811 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "status"}) -- 0x55b49d6cb210 con 0x55b49d7dedb0 2024-04-17T07:14:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.558+0000 7f98d3e00640 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55b49d7dedb0 0x55b49d7ddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.559+0000 7f98d3e00640 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55b49d7dedb0 0x55b49d7ddac0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d72a9d9680f8fe33 server_cookie=ba1ab18a7493fd9f in_seq=0 out_seq=0 2024-04-17T07:14:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.559+0000 7f98ba000640 10 client.15585 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:14:51.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98ba000640 1 -- 172.21.15.86:0/4028970811 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b49d6cb210 con 0x55b49d7dedb0 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98ba000640 10 client.15585 handle_command_reply: tid=0 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98de3d6740 1 client.15585 shutdown 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98de3d6740 1 -- 172.21.15.86:0/4028970811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 msgr2=0x55b49d76c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98de3d6740 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d76c3d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f98c809b460 tx=0x7f98c809b370 comp rx=0 tx=0).stop 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98de3d6740 1 -- 172.21.15.86:0/4028970811 shutdown_connections 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98de3d6740 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55b49d7dedb0 0x55b49d7ddac0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98de3d6740 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49d762440 0x55b49d7d8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98de3d6740 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49d75d2d0 0x55b49d7d5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98ba000640 0 client.15585 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98de3d6740 1 --2- 172.21.15.86:0/4028970811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49d75c8e0 0x55b49d76c3d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98de3d6740 1 -- 172.21.15.86:0/4028970811 >> 172.21.15.86:0/4028970811 conn(0x55b49d757e00 msgr2=0x55b49d75dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.560+0000 7f98de3d6740 1 -- 172.21.15.86:0/4028970811 shutdown_connections 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98de3d6740 1 -- 172.21.15.86:0/4028970811 wait complete. 2024-04-17T07:14:51.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98de3d6740 20 client.15585 trim_cache size 0 max 16384 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 34460, 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 144.86686378799999, 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 91, 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 198.84796149300001 2024-04-17T07:14:51.564 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98db600640 1 -- 172.21.15.86:0/3715161647 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98bc0a3bb0 msgr2=0x7f98bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98db600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98bc0a3bb0 0x7f98bc0a6070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f98cc032420 tx=0x7f98cc032bb0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98db600640 1 -- 172.21.15.86:0/3715161647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d41041c0 msgr2=0x7f98d419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98db600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d41041c0 0x7f98d419a380 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f98c800d950 tx=0x7f98c800de10 comp rx=0 tx=0).stop 2024-04-17T07:14:51.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98db600640 1 -- 172.21.15.86:0/3715161647 shutdown_connections 2024-04-17T07:14:51.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98db600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98bc0a3bb0 0x7f98bc0a6070 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98db600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d4105d70 0x7f98d41a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98db600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d41053b0 0x7f98d419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98db600640 1 --2- 172.21.15.86:0/3715161647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d41041c0 0x7f98d419a380 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.561+0000 7f98db600640 1 -- 172.21.15.86:0/3715161647 >> 172.21.15.86:0/3715161647 conn(0x7f98d40ff750 msgr2=0x7f98d41086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.562+0000 7f98db600640 1 -- 172.21.15.86:0/3715161647 shutdown_connections 2024-04-17T07:14:51.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.562+0000 7f98db600640 1 -- 172.21.15.86:0/3715161647 wait complete. 2024-04-17T07:14:51.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-17T07:14:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.797+0000 7ff6ca400640 1 -- 172.21.15.86:0/4089917414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c4101040 msgr2=0x7ff6c41014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.797+0000 7ff6ca400640 1 --2- 172.21.15.86:0/4089917414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c4101040 0x7ff6c41014a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009900 tx=0x7ff6ac02bc30 comp rx=0 tx=0).stop 2024-04-17T07:14:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.797+0000 7ff6ca400640 1 -- 172.21.15.86:0/4089917414 shutdown_connections 2024-04-17T07:14:51.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.797+0000 7ff6ca400640 1 --2- 172.21.15.86:0/4089917414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c41019e0 0x7ff6c4101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.797+0000 7ff6ca400640 1 --2- 172.21.15.86:0/4089917414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c4101040 0x7ff6c41014a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.797+0000 7ff6ca400640 1 --2- 172.21.15.86:0/4089917414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6c41006e0 0x7ff6c4100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.797+0000 7ff6ca400640 1 -- 172.21.15.86:0/4089917414 >> 172.21.15.86:0/4089917414 conn(0x7ff6c40fc300 msgr2=0x7ff6c40fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.797+0000 7ff6ca400640 1 -- 172.21.15.86:0/4089917414 shutdown_connections 2024-04-17T07:14:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.797+0000 7ff6ca400640 1 -- 172.21.15.86:0/4089917414 wait complete. 2024-04-17T07:14:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.798+0000 7ff6ca400640 1 Processor -- start 2024-04-17T07:14:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.798+0000 7ff6ca400640 1 -- start start 2024-04-17T07:14:51.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.798+0000 7ff6ca400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c41006e0 0x7ff6c419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.798+0000 7ff6ca400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6c4101040 0x7ff6c419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.798+0000 7ff6ca400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c41019e0 0x7ff6c41a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6ca400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410fc10 con 0x7ff6c41006e0 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6ca400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410fa90 con 0x7ff6c4101040 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6ca400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410fd90 con 0x7ff6c41019e0 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6c9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c41006e0 0x7ff6c419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6c9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c41006e0 0x7ff6c419a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54332/0 (socket says 172.21.15.86:54332) 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6c9000640 1 -- 172.21.15.86:0/2768714672 learned_addr learned my addr 172.21.15.86:0/2768714672 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6c4101040 0x7ff6c419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6c9a00640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c41019e0 0x7ff6c41a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6c9000640 1 -- 172.21.15.86:0/2768714672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c41019e0 msgr2=0x7ff6c41a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6c9000640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c41019e0 0x7ff6c41a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6c9000640 1 -- 172.21.15.86:0/2768714672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6c4101040 msgr2=0x7ff6c419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6c9000640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6c4101040 0x7ff6c419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6c9000640 1 -- 172.21.15.86:0/2768714672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c41a1e90 con 0x7ff6c41006e0 2024-04-17T07:14:51.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.799+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6c4101040 0x7ff6c419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:51.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.800+0000 7ff6c9000640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c41006e0 0x7ff6c419a270 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400eab0 tx=0x7ff6b400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.800+0000 7ff6c2000640 1 -- 172.21.15.86:0/2768714672 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400ce40 con 0x7ff6c41006e0 2024-04-17T07:14:51.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.800+0000 7ff6c2000640 1 -- 172.21.15.86:0/2768714672 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6b4011920 con 0x7ff6c41006e0 2024-04-17T07:14:51.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.800+0000 7ff6ca400640 1 -- 172.21.15.86:0/2768714672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c41a2180 con 0x7ff6c41006e0 2024-04-17T07:14:51.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.800+0000 7ff6c2000640 1 -- 172.21.15.86:0/2768714672 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4010a00 con 0x7ff6c41006e0 2024-04-17T07:14:51.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.800+0000 7ff6ca400640 1 -- 172.21.15.86:0/2768714672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c41a2720 con 0x7ff6c41006e0 2024-04-17T07:14:51.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.803+0000 7ff6c2000640 1 -- 172.21.15.86:0/2768714672 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff6b4011ad0 con 0x7ff6c41006e0 2024-04-17T07:14:51.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.803+0000 7ff6ce61a740 1 Processor -- start 2024-04-17T07:14:51.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.804+0000 7ff6ce61a740 1 -- start start 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.804+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107221f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.804+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107226fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.804+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 0x55e107229ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.804+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e107188980 con 0x55e1072224e0 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.804+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e10712f220 con 0x55e107227650 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.804+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e107184940 con 0x55e107221af0 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.804+0000 7ff6c9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107226fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.804+0000 7ff6c9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107226fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54336/0 (socket says 172.21.15.86:54336) 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.804+0000 7ff6c9000640 1 -- 172.21.15.86:0/27356006 learned_addr learned my addr 172.21.15.86:0/27356006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/27356006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 0x55e107229ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6c9a00640 1 --2- 172.21.15.86:0/27356006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107221f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6c9000640 1 -- 172.21.15.86:0/27356006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 msgr2=0x55e107221f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6c9000640 1 --2- 172.21.15.86:0/27356006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107221f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6c9000640 1 -- 172.21.15.86:0/27356006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 msgr2=0x55e107229ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6c9000640 1 --2- 172.21.15.86:0/27356006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 0x55e107229ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6c9000640 1 -- 172.21.15.86:0/27356006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e106e9b2d0 con 0x55e1072224e0 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/27356006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 0x55e107229ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6c2000640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff6a80a3c30 0x7ff6a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6c2000640 1 -- 172.21.15.86:0/2768714672 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff6b40d0e90 con 0x7ff6c41006e0 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6c9000640 1 --2- 172.21.15.86:0/27356006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107226fe0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40d9110 tx=0x7ff6b40d9200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5f82fb67a3527a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.805+0000 7ff6c9a00640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff6a80a3c30 0x7ff6a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.806+0000 7ff6ba000640 1 -- 172.21.15.86:0/27356006 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4099030 con 0x55e1072224e0 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.806+0000 7ff6ba000640 1 -- 172.21.15.86:0/27356006 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6b4011e00 con 0x55e1072224e0 2024-04-17T07:14:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.806+0000 7ff6ba000640 1 -- 172.21.15.86:0/27356006 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b409c6f0 con 0x55e1072224e0 2024-04-17T07:14:51.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.806+0000 7ff6ce61a740 1 -- 172.21.15.86:0/27356006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 msgr2=0x55e107226fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.806+0000 7ff6c9a00640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff6a80a3c30 0x7ff6a80a60f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55e107222d40 tx=0x7ff6bc005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.806+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/27356006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107226fe0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40d9110 tx=0x7ff6b40d9200 comp rx=0 tx=0).stop 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.807+0000 7ff6ce61a740 1 -- 172.21.15.86:0/27356006 shutdown_connections 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.807+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/27356006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 0x55e107229ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.807+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/27356006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107226fe0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.807+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/27356006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107221f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.807+0000 7ff6ce61a740 1 -- 172.21.15.86:0/27356006 >> 172.21.15.86:0/27356006 conn(0x55e10721d060 msgr2=0x55e10721d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.807+0000 7ff6ce61a740 1 -- 172.21.15.86:0/27356006 shutdown_connections 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.807+0000 7ff6ce61a740 1 -- 172.21.15.86:0/27356006 wait complete. 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.807+0000 7ff6ce61a740 1 -- start start 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.807+0000 7ff6ce61a740 1 Processor -- start 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107221af0 0x55e10722c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e1072224e0 0x55e10722c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107227650 0x55e1072335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e10722a600 con 0x55e107221af0 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e107142590 con 0x55e1072224e0 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e10722a780 con 0x55e107227650 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6ce61a740 10 client.0 _ll_register_callbacks cb 0x55e106e9aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6c9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107221af0 0x55e10722c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6c9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107221af0 0x55e10722c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54342/0 (socket says 172.21.15.86:54342) 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6c9000640 1 -- 172.21.15.86:0/368392330 learned_addr learned my addr 172.21.15.86:0/368392330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e1072224e0 0x55e10722c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6c9000640 1 -- 172.21.15.86:0/368392330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107227650 msgr2=0x55e1072335a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6c9000640 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107227650 0x55e1072335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6c9000640 1 -- 172.21.15.86:0/368392330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e1072224e0 msgr2=0x55e10722c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6c9000640 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e1072224e0 0x55e10722c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6c9000640 1 -- 172.21.15.86:0/368392330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e106f656b0 con 0x55e107221af0 2024-04-17T07:14:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.808+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e1072224e0 0x55e10722c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.809+0000 7ff6c9000640 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107221af0 0x55e10722c020 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40d90e0 tx=0x7ff6b409d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67951ec4bfdb24cb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.809+0000 7ff6b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.809+0000 7ff6b2a00640 1 -- 172.21.15.86:0/368392330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40b5070 con 0x55e107221af0 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.809+0000 7ff6b2a00640 1 -- 172.21.15.86:0/368392330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff6b40b6260 con 0x55e107221af0 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.809+0000 7ff6ce61a740 1 -- 172.21.15.86:0/368392330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1071963a0 con 0x55e107221af0 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.809+0000 7ff6b2a00640 1 -- 172.21.15.86:0/368392330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6b409c8d0 con 0x55e107221af0 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.809+0000 7ff6ce61a740 10 client.15597 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.809+0000 7ff6ce61a740 1 -- 172.21.15.86:0/368392330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e10722a600 con 0x55e107221af0 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.809+0000 7ff6b2a00640 1 -- 172.21.15.86:0/368392330 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e1071963a0 con 0x55e107221af0 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.810+0000 7ff6b2a00640 1 -- 172.21.15.86:0/368392330 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7ff6b400f040 con 0x55e107221af0 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.810+0000 7ff6ce61a740 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.810+0000 7ff6ce61a740 10 client.15597 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.810+0000 7ff6ce61a740 20 client.15597 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.810+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e1072389d0 0x55e107239e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.810+0000 7ff6ce61a740 4 client.15597 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:14:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.810+0000 7ff6ce61a740 1 -- 172.21.15.86:0/368392330 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e107190210 con 0x55e1072389d0 2024-04-17T07:14:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.811+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e1072389d0 0x55e107239e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.811+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e1072389d0 0x55e107239e40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47556214cfe02731 server_cookie=850c65c6cdac52e5 in_seq=0 out_seq=0 2024-04-17T07:14:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.811+0000 7ff6b2a00640 10 client.15597 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:14:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.813+0000 7ff6b2a00640 1 -- 172.21.15.86:0/368392330 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e107190210 con 0x55e1072389d0 2024-04-17T07:14:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.813+0000 7ff6b2a00640 10 client.15597 handle_command_reply: tid=0 2024-04-17T07:14:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.813+0000 7ff6ce61a740 1 client.15597 shutdown 2024-04-17T07:14:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 1 -- 172.21.15.86:0/368392330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107221af0 msgr2=0x55e10722c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107221af0 0x55e10722c020 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40d90e0 tx=0x7ff6b409d490 comp rx=0 tx=0).stop 2024-04-17T07:14:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 1 -- 172.21.15.86:0/368392330 shutdown_connections 2024-04-17T07:14:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e1072389d0 0x55e107239e40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107227650 0x55e1072335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e1072224e0 0x55e10722c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6b2a00640 0 client.15597 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:14:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/368392330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107221af0 0x55e10722c020 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 1 -- 172.21.15.86:0/368392330 >> 172.21.15.86:0/368392330 conn(0x55e10721d060 msgr2=0x55e10721c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 1 -- 172.21.15.86:0/368392330 shutdown_connections 2024-04-17T07:14:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 1 -- 172.21.15.86:0/368392330 wait complete. 2024-04-17T07:14:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.814+0000 7ff6ce61a740 20 client.15597 trim_cache size 0 max 16384 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6ce61a740 1 Processor -- start 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6ce61a740 1 -- start start 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107223a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107223fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 0x55e107233560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e107238180 con 0x55e1072224e0 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e107238850 con 0x55e107227650 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e107238f20 con 0x55e107221af0 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107223fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107223fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54350/0 (socket says 172.21.15.86:54350) 2024-04-17T07:14:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.834+0000 7ff6bbe00640 1 -- 172.21.15.86:0/1342938555 learned_addr learned my addr 172.21.15.86:0/1342938555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6c9000640 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107223a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6bbe00640 1 -- 172.21.15.86:0/1342938555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 msgr2=0x55e107223a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107223a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6c9a00640 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 0x55e107233560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6bbe00640 1 -- 172.21.15.86:0/1342938555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 msgr2=0x55e107233560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 0x55e107233560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6bbe00640 1 -- 172.21.15.86:0/1342938555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1072228c0 con 0x55e1072224e0 2024-04-17T07:14:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6c9000640 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107223a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6c9a00640 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 0x55e107233560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:51.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107223fb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff6c419b780 tx=0x7ff6ac005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26149eff8f88740 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6b9600640 1 -- 172.21.15.86:0/1342938555 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac046070 con 0x55e1072224e0 2024-04-17T07:14:51.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.835+0000 7ff6b9600640 1 -- 172.21.15.86:0/1342938555 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6ac0054f0 con 0x55e1072224e0 2024-04-17T07:14:51.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.836+0000 7ff6ce61a740 1 -- 172.21.15.86:0/1342938555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 msgr2=0x55e107223fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.836+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107223fb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff6c419b780 tx=0x7ff6ac005b50 comp rx=0 tx=0).stop 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.836+0000 7ff6ce61a740 1 -- 172.21.15.86:0/1342938555 shutdown_connections 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.836+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107227650 0x55e107233560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.836+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e1072224e0 0x55e107223fb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.836+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/1342938555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107223a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.836+0000 7ff6ce61a740 1 -- 172.21.15.86:0/1342938555 >> 172.21.15.86:0/1342938555 conn(0x55e10721d060 msgr2=0x55e10721d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.836+0000 7ff6ce61a740 1 -- 172.21.15.86:0/1342938555 shutdown_connections 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.836+0000 7ff6ce61a740 1 -- 172.21.15.86:0/1342938555 wait complete. 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.837+0000 7ff6ce61a740 1 -- start start 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.837+0000 7ff6ce61a740 1 Processor -- start 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.837+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107231700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.837+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107227650 0x55e107231c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.837+0000 7ff6ce61a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107232220 0x55e1072326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.837+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e107239cc0 con 0x55e107227650 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.837+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e1072395f0 con 0x55e107232220 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.837+0000 7ff6ce61a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e10723a390 con 0x55e107221af0 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.837+0000 7ff6ce61a740 10 client.0 _ll_register_callbacks cb 0x7ff6b40bbf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107227650 0x55e107231c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107227650 0x55e107231c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54366/0 (socket says 172.21.15.86:54366) 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6bbe00640 1 -- 172.21.15.86:0/3781462600 learned_addr learned my addr 172.21.15.86:0/3781462600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6bbe00640 1 -- 172.21.15.86:0/3781462600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 msgr2=0x55e107231700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107231700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6bbe00640 1 -- 172.21.15.86:0/3781462600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107232220 msgr2=0x55e1072326a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6c9a00640 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107232220 0x55e1072326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107232220 0x55e1072326a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6bbe00640 1 -- 172.21.15.86:0/3781462600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e107232ed0 con 0x55e107227650 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6bbe00640 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107227650 0x55e107231c40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac000c00 tx=0x7ff6ac0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74a1a5dc6ec3050 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.838+0000 7ff6b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.839+0000 7ff6b2a00640 1 -- 172.21.15.86:0/3781462600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac050070 con 0x55e107227650 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.839+0000 7ff6ce61a740 1 -- 172.21.15.86:0/3781462600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac0054f0 con 0x55e107227650 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.839+0000 7ff6b2a00640 1 -- 172.21.15.86:0/3781462600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff6ac004a40 con 0x55e107227650 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.839+0000 7ff6b2a00640 1 -- 172.21.15.86:0/3781462600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6ac046040 con 0x55e107227650 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.839+0000 7ff6ce61a740 10 client.15603 fetch_fsmap learned FSMap version 27 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.839+0000 7ff6ce61a740 1 -- 172.21.15.86:0/3781462600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1072a7740 con 0x55e107227650 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.839+0000 7ff6b2a00640 1 -- 172.21.15.86:0/3781462600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6ac0054f0 con 0x55e107227650 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.840+0000 7ff6b2a00640 1 -- 172.21.15.86:0/3781462600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 27) ==== 8363+0+0 (secure 0 0 0) 0x7ff6ac034c60 con 0x55e107227650 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.840+0000 7ff6ce61a740 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.840+0000 7ff6ce61a740 10 client.15603 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.840+0000 7ff6ce61a740 20 client.15603 populate_metadata read hostname 'smithi086' 2024-04-17T07:14:51.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.840+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e1072abef0 0x55e1072a8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:51.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.840+0000 7ff6ce61a740 4 client.15603 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T07:14:51.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.840+0000 7ff6ce61a740 1 -- 172.21.15.86:0/3781462600 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "status"}) -- 0x55e10710c7d0 con 0x55e1072abef0 2024-04-17T07:14:51.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.840+0000 7ff6c9000640 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e1072abef0 0x55e1072a8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:51.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.841+0000 7ff6c9000640 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e1072abef0 0x55e1072a8330 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c9a44d555d1d744 server_cookie=43d437f3b3fb130f in_seq=0 out_seq=0 2024-04-17T07:14:51.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.841+0000 7ff6b2a00640 10 client.15603 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:14:51.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6b2a00640 1 -- 172.21.15.86:0/3781462600 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e10710c7d0 con 0x55e1072abef0 2024-04-17T07:14:51.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6b2a00640 10 client.15603 handle_command_reply: tid=0 2024-04-17T07:14:51.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 client.15603 shutdown 2024-04-17T07:14:51.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 -- 172.21.15.86:0/3781462600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107227650 msgr2=0x55e107231c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107227650 0x55e107231c40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac000c00 tx=0x7ff6ac0056a0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 -- 172.21.15.86:0/3781462600 shutdown_connections 2024-04-17T07:14:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e1072abef0 0x55e1072a8330 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e107232220 0x55e1072326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6b2a00640 0 client.15603 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:14:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e107227650 0x55e107231c40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 --2- 172.21.15.86:0/3781462600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e107221af0 0x55e107231700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 -- 172.21.15.86:0/3781462600 >> 172.21.15.86:0/3781462600 conn(0x55e10721d060 msgr2=0x55e10722ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 -- 172.21.15.86:0/3781462600 shutdown_connections 2024-04-17T07:14:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 1 -- 172.21.15.86:0/3781462600 wait complete. 2024-04-17T07:14:51.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.842+0000 7ff6ce61a740 20 client.15603 trim_cache size 0 max 16384 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "1f7ed99a-fc88-11ee-bc91-c7b262605968", 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 34490, 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 176.37652547799999, 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 27, 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 91, 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 182.92130121400001 2024-04-17T07:14:51.847 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 -- 172.21.15.86:0/2768714672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff6a80a3c30 msgr2=0x7ff6a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff6a80a3c30 0x7ff6a80a60f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55e107222d40 tx=0x7ff6bc005d90 comp rx=0 tx=0).stop 2024-04-17T07:14:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 -- 172.21.15.86:0/2768714672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c41006e0 msgr2=0x7ff6c419a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c41006e0 0x7ff6c419a270 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400eab0 tx=0x7ff6b400ef70 comp rx=0 tx=0).stop 2024-04-17T07:14:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 -- 172.21.15.86:0/2768714672 shutdown_connections 2024-04-17T07:14:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff6a80a3c30 0x7ff6a80a60f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c41019e0 0x7ff6c41a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff6c4101040 0x7ff6c419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 --2- 172.21.15.86:0/2768714672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6c41006e0 0x7ff6c419a270 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:51.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 -- 172.21.15.86:0/2768714672 >> 172.21.15.86:0/2768714672 conn(0x7ff6c40fc300 msgr2=0x7ff6c40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:51.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 -- 172.21.15.86:0/2768714672 shutdown_connections 2024-04-17T07:14:51.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:51.843+0000 7ff6ca400640 1 -- 172.21.15.86:0/2768714672 wait complete. 2024-04-17T07:14:51.863 INFO:teuthology.run_tasks:Running task sequential... 2024-04-17T07:14:51.873 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-17T07:14:51.873 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-17T07:14:51.873 INFO:teuthology.task.print:Enabling mgr modules 2024-04-17T07:14:51.873 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-17T07:14:51.896 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-17T07:14:51.896 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-17T07:14:51.896 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-04-17T07:14:51.896 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T07:14:51.896 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-04-17T07:14:51.897 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-04-17T07:14:51.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:51.897 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-17T07:14:51.953 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-17T07:14:51.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:51.954 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T07:14:52.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:52.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-17T07:14:52.172 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-17T07:14:52.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph fs ls 2024-04-17T07:14:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:14:54.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:54 smithi086 ceph-mon[31733]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:54 smithi098 ceph-mon[35418]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:54 smithi103 ceph-mon[35495]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-17T07:14:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.892+0000 7fb6dac00700 1 -- 172.21.15.86:0/2635982075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d41016d0 msgr2=0x7fb6d4101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.892+0000 7fb6dac00700 1 --2- 172.21.15.86:0/2635982075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d41016d0 0x7fb6d4101ab0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009990 tx=0x7fb6c8009ca0 comp rx=0 tx=0).stop 2024-04-17T07:14:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.892+0000 7fb6dac00700 1 -- 172.21.15.86:0/2635982075 shutdown_connections 2024-04-17T07:14:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.892+0000 7fb6dac00700 1 --2- 172.21.15.86:0/2635982075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6d410f280 0x7fb6d4111670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.892+0000 7fb6dac00700 1 --2- 172.21.15.86:0/2635982075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d41016d0 0x7fb6d4101ab0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.892+0000 7fb6dac00700 1 --2- 172.21.15.86:0/2635982075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6d4102080 0x7fb6d410ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.892+0000 7fb6dac00700 1 -- 172.21.15.86:0/2635982075 >> 172.21.15.86:0/2635982075 conn(0x7fb6d40fd370 msgr2=0x7fb6d40ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.892+0000 7fb6dac00700 1 -- 172.21.15.86:0/2635982075 shutdown_connections 2024-04-17T07:14:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.892+0000 7fb6dac00700 1 -- 172.21.15.86:0/2635982075 wait complete. 2024-04-17T07:14:54.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.893+0000 7fb6dac00700 1 Processor -- start 2024-04-17T07:14:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.893+0000 7fb6dac00700 1 -- start start 2024-04-17T07:14:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6dac00700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d41016d0 0x7fb6d419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6dac00700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6d4102080 0x7fb6d419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6dac00700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6d410f280 0x7fb6d419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6dac00700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4114da0 con 0x7fb6d41016d0 2024-04-17T07:14:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6dac00700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4114c20 con 0x7fb6d4102080 2024-04-17T07:14:54.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6dac00700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4114f20 con 0x7fb6d410f280 2024-04-17T07:14:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6d9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d41016d0 0x7fb6d419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6d9800700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d41016d0 0x7fb6d419b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54392/0 (socket says 172.21.15.86:54392) 2024-04-17T07:14:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6d9800700 1 -- 172.21.15.86:0/3799003155 learned_addr learned my addr 172.21.15.86:0/3799003155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:14:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6da200700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6d410f280 0x7fb6d419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6d8e00700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6d4102080 0x7fb6d419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6d9800700 1 -- 172.21.15.86:0/3799003155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6d410f280 msgr2=0x7fb6d419fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.894+0000 7fb6d9800700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6d410f280 0x7fb6d419fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.895+0000 7fb6d9800700 1 -- 172.21.15.86:0/3799003155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6d4102080 msgr2=0x7fb6d419b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.895+0000 7fb6d9800700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6d4102080 0x7fb6d419b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.895+0000 7fb6d9800700 1 -- 172.21.15.86:0/3799003155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d41a01a0 con 0x7fb6d41016d0 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.895+0000 7fb6da200700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6d410f280 0x7fb6d419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.895+0000 7fb6d9800700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d41016d0 0x7fb6d419b190 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8003ab0 tx=0x7fb6c8003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.895+0000 7fb6d2000700 1 -- 172.21.15.86:0/3799003155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8027070 con 0x7fb6d41016d0 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.895+0000 7fb6d2000700 1 -- 172.21.15.86:0/3799003155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6c800bdc0 con 0x7fb6d41016d0 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.895+0000 7fb6dac00700 1 -- 172.21.15.86:0/3799003155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d41a0430 con 0x7fb6d41016d0 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.895+0000 7fb6d2000700 1 -- 172.21.15.86:0/3799003155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8022de0 con 0x7fb6d41016d0 2024-04-17T07:14:54.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.895+0000 7fb6dac00700 1 -- 172.21.15.86:0/3799003155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d41a7d10 con 0x7fb6d41016d0 2024-04-17T07:14:54.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.897+0000 7fb6dac00700 1 -- 172.21.15.86:0/3799003155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d4069740 con 0x7fb6d41016d0 2024-04-17T07:14:54.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.898+0000 7fb6d2000700 1 -- 172.21.15.86:0/3799003155 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6c802a070 con 0x7fb6d41016d0 2024-04-17T07:14:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.900+0000 7fb6d2000700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6bc0a3c40 0x7fb6bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:14:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.900+0000 7fb6d2000700 1 -- 172.21.15.86:0/3799003155 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb6c80d1610 con 0x7fb6d41016d0 2024-04-17T07:14:54.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.900+0000 7fb6d8e00700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6bc0a3c40 0x7fb6bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:14:54.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.901+0000 7fb6d8e00700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6bc0a3c40 0x7fb6bc0a6100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4111540 tx=0x7fb6c0005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:14:54.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:54.905+0000 7fb6d2000700 1 -- 172.21.15.86:0/3799003155 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb6c805eff0 con 0x7fb6d41016d0 2024-04-17T07:14:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.177+0000 7fb6dac00700 1 -- 172.21.15.86:0/3799003155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb6d41a7ee0 con 0x7fb6d41016d0 2024-04-17T07:14:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.178+0000 7fb6d2000700 1 -- 172.21.15.86:0/3799003155 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7fb6c8013070 con 0x7fb6d41016d0 2024-04-17T07:14:55.178 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-17T07:14:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 -- 172.21.15.86:0/3799003155 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6bc0a3c40 msgr2=0x7fb6bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6bc0a3c40 0x7fb6bc0a6100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4111540 tx=0x7fb6c0005eb0 comp rx=0 tx=0).stop 2024-04-17T07:14:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 -- 172.21.15.86:0/3799003155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d41016d0 msgr2=0x7fb6d419b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:14:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d41016d0 0x7fb6d419b190 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8003ab0 tx=0x7fb6c8003ae0 comp rx=0 tx=0).stop 2024-04-17T07:14:55.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 -- 172.21.15.86:0/3799003155 shutdown_connections 2024-04-17T07:14:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb6bc0a3c40 0x7fb6bc0a6100 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6d410f280 0x7fb6d419fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6d41016d0 0x7fb6d419b190 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 --2- 172.21.15.86:0/3799003155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6d4102080 0x7fb6d419b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:14:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 -- 172.21.15.86:0/3799003155 >> 172.21.15.86:0/3799003155 conn(0x7fb6d40fd370 msgr2=0x7fb6d4102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:14:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 -- 172.21.15.86:0/3799003155 shutdown_connections 2024-04-17T07:14:55.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:55.181+0000 7fb6dac00700 1 -- 172.21.15.86:0/3799003155 wait complete. 2024-04-17T07:14:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:55 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3799003155' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:14:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:55 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3799003155' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:14:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:55 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3799003155' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-17T07:14:56.043 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T07:14:56.043 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T07:14:56.043 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-17T07:14:56.043 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T07:14:56.043 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-04-17T07:14:56.043 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T07:14:56.043 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T07:14:56.043 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T07:14:56.043 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-04-17T07:14:56.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:56.044 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-17T07:14:56.100 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T07:14:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T07:14:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T07:14:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:14:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-17T07:14:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:14:56.100 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:14:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:14:56.100 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-17T07:14:56.100 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:14:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:14:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-17T07:14:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T07:14:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:14:56.101 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T07:14:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-17T07:14:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-17T07:14:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41801sec preferred_lft 41801sec 2024-04-17T07:14:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-17T07:14:56.101 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-17T07:14:56.102 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T07:14:56.103 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:14:56.103 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-17T07:14:56.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-17T07:14:56.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-17T07:14:56.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-17T07:14:56.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T07:14:56.103 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-17T07:14:56.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:14:56.250 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:56 smithi086 ceph-mon[31733]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:56.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:14:56.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:56.402 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T07:14:56.474 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-17T07:14:56.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:56.478 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-17T07:14:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:56 smithi098 ceph-mon[35418]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:56.523 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-17T07:14:56.523 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T07:14:56.523 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T07:14:56.523 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T07:14:56.523 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T07:14:56.524 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:14:56.524 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-17T07:14:56.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T07:14:56.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T07:14:56.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T07:14:56.524 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-17T07:14:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:56 smithi103 ceph-mon[35495]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-17T07:14:56.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:14:56.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:14:56.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:56.786 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-17T07:14:56.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:56.833 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-17T07:14:56.925 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:14:56.925 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-17T07:14:56.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--tmp-cephfs 2024-04-17T07:14:56.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-04-17T07:14:56.925 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-17T07:14:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:14:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:14:57.114 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-04-17T07:14:57.115 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:14:57.115 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-17T07:14:57.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-04-17T07:14:57.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T07:14:57.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-04-17T07:14:57.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-04-17T07:14:57.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-04-17T07:14:57.115 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-17T07:14:57.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:14:57.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:14:57.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:14:57.430 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T07:14:57.430 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-17T07:14:57.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-17T07:14:57.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T07:14:57.431 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-17T07:14:57.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:14:57.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T07:14:57.525 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:14:57.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:14:57.627 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-04-17T07:14:57.627 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T07:14:57.627 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /tmp/cephfs 2024-04-17T07:14:57.666 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/tmp/cephfs' 2024-04-17T07:14:57.666 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T07:14:57.666 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /tmp/cephfs 2024-04-17T07:14:57.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-17T07:14:57.881 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-17T07:14:57.931 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-17T07:14:57.932 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/2cd09f6d91b55586871c77946312699cfe780fb744dc7dd04fef954998dd3d6b/merged 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/daa4e0009951020f1d8e0f1a30c631b514d43a7dd9bfbcf2d1eae139c2a7eb2f/merged 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/b94ef22031a7435e285187988bebca5e4cef86fe8b13dcf0d1444c00be1fcf3c/merged 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c5a576a8d0f15fde9a8605e3a9629d20764354d21e798f72c6fed568f4a86ccf/merged 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5ca71d6a5746de4bdb3cc50a830763d933d1001eac8f4b450402954a84b1efed/merged 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c49121dbdaf6788fba7930f24e1dc568a87dcc5c8c4158e40ff110b9a4723737/merged 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/93635c477aa616f15f8542b827e2586cbbbb03a902f83386b128502d0c1009e3/merged 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5c0cc5bb9c77431ff4db506e0500c3287d3fe329e5876e767259f50d1396aa30/merged 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/3e44fcef40b275061b17516d4d6207e6e80be758f3e73e8d2710d71d42ccdd1a/merged 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/13eace86438efe164220139509f4b1079c443786d7afc6365f2862a4ea4a1f85/merged 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-17T07:14:57.933 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-17T07:14:57.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:57.934 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-17T07:14:57.973 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T07:14:57.973 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/qa/sv_0/20a7ddba-5b0c-401a-839f-309c9dcc27f5 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-04-17T07:14:58.020 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-17T07:14:58.096 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T07:14:58.138 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-17T07:14:58.138 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-17T07:14:58.138 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-17T07:14:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/2cd09f6d91b55586871c77946312699cfe780fb744dc7dd04fef954998dd3d6b/merged 2024-04-17T07:14:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/daa4e0009951020f1d8e0f1a30c631b514d43a7dd9bfbcf2d1eae139c2a7eb2f/merged 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/b94ef22031a7435e285187988bebca5e4cef86fe8b13dcf0d1444c00be1fcf3c/merged 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c5a576a8d0f15fde9a8605e3a9629d20764354d21e798f72c6fed568f4a86ccf/merged 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5ca71d6a5746de4bdb3cc50a830763d933d1001eac8f4b450402954a84b1efed/merged 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c49121dbdaf6788fba7930f24e1dc568a87dcc5c8c4158e40ff110b9a4723737/merged 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/93635c477aa616f15f8542b827e2586cbbbb03a902f83386b128502d0c1009e3/merged 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5c0cc5bb9c77431ff4db506e0500c3287d3fe329e5876e767259f50d1396aa30/merged 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/3e44fcef40b275061b17516d4d6207e6e80be758f3e73e8d2710d71d42ccdd1a/merged 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/13eace86438efe164220139509f4b1079c443786d7afc6365f2862a4ea4a1f85/merged 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-17T07:14:58.141 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-17T07:14:58.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:58.143 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-17T07:14:58.217 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-17T07:14:58.218+0000 7f7f34e42480 -1 init, newargv = 0x5556e3dc03c0 newargc=17 2024-04-17T07:14:58.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[83951]: starting ceph client 2024-04-17T07:14:58.260 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[83951]: starting fuse 2024-04-17T07:14:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:58 smithi098 ceph-mon[35418]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:58 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:14:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:14:58 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:14:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:58 smithi103 ceph-mon[35495]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:58 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:14:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:14:58 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:14:58.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:58 smithi086 ceph-mon[31733]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:14:58.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:58 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:14:58.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:14:58 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:14:59.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-17T07:14:59.423 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T07:14:59.470 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-17T07:14:59.470 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-17T07:14:59.470 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-17T07:14:59.470 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-17T07:14:59.471 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/2cd09f6d91b55586871c77946312699cfe780fb744dc7dd04fef954998dd3d6b/merged 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/daa4e0009951020f1d8e0f1a30c631b514d43a7dd9bfbcf2d1eae139c2a7eb2f/merged 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/b94ef22031a7435e285187988bebca5e4cef86fe8b13dcf0d1444c00be1fcf3c/merged 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c5a576a8d0f15fde9a8605e3a9629d20764354d21e798f72c6fed568f4a86ccf/merged 2024-04-17T07:14:59.472 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5ca71d6a5746de4bdb3cc50a830763d933d1001eac8f4b450402954a84b1efed/merged 2024-04-17T07:14:59.473 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/c49121dbdaf6788fba7930f24e1dc568a87dcc5c8c4158e40ff110b9a4723737/merged 2024-04-17T07:14:59.473 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/93635c477aa616f15f8542b827e2586cbbbb03a902f83386b128502d0c1009e3/merged 2024-04-17T07:14:59.473 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/5c0cc5bb9c77431ff4db506e0500c3287d3fe329e5876e767259f50d1396aa30/merged 2024-04-17T07:14:59.473 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/3e44fcef40b275061b17516d4d6207e6e80be758f3e73e8d2710d71d42ccdd1a/merged 2024-04-17T07:14:59.473 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/13eace86438efe164220139509f4b1079c443786d7afc6365f2862a4ea4a1f85/merged 2024-04-17T07:14:59.473 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-17T07:14:59.473 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-17T07:14:59.473 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-04-17T07:14:59.473 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs 2024-04-17T07:14:59.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:59.474 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-17T07:14:59.515 INFO:teuthology.orchestra.run.smithi086.stdout:119 2024-04-17T07:14:59.515 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/tmp/cephfs" 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-17T07:14:59.516 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-17T07:14:59.517 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-17T07:14:59.714 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.83951.asok 2024-04-17T07:14:59.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:14:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T07:14:59.730 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83951.asok 2024-04-17T07:14:59.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:59.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83951.asok status 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/tmp/cephfs", 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "83951", 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/volumes/qa/sv_0/20a7ddba-5b0c-401a-839f-309c9dcc27f5" 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:14:59.954 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 15615, 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 15615 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2446517357 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-17T07:14:59.955 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2446517357 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.15615 192.168.0.1:0/2446517357", 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/2446517357", 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 5, 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 27, 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 93, 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-17T07:14:59.956 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:14:59.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:14:59.968 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-17T07:14:59.968 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /tmp/cephfs 2024-04-17T07:15:00.009 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-17T07:15:00.009 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-04-17T07:15:00.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:15:00.009 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /tmp/cephfs 2024-04-17T07:15:00.155 INFO:teuthology.run_tasks:Running task exec... 2024-04-17T07:15:00.165 INFO:teuthology.task.exec:Executing custom commands... 2024-04-17T07:15:00.165 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-17T07:15:00.165 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-17T07:15:00.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.357+0000 7ff92a200640 1 -- 172.21.15.86:0/2740474313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9241041f0 msgr2=0x7ff924104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:00.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.357+0000 7ff92a200640 1 --2- 172.21.15.86:0/2740474313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9241041f0 0x7ff924104610 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff914009900 tx=0x7ff91402bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:00.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.357+0000 7ff92a200640 1 -- 172.21.15.86:0/2740474313 shutdown_connections 2024-04-17T07:15:00.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.357+0000 7ff92a200640 1 --2- 172.21.15.86:0/2740474313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff924105da0 0x7ff92410c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:00.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.357+0000 7ff92a200640 1 --2- 172.21.15.86:0/2740474313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9241053e0 0x7ff924105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:00.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.357+0000 7ff92a200640 1 --2- 172.21.15.86:0/2740474313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9241041f0 0x7ff924104610 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:00.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.357+0000 7ff92a200640 1 -- 172.21.15.86:0/2740474313 >> 172.21.15.86:0/2740474313 conn(0x7ff9240ff780 msgr2=0x7ff924101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:00.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.357+0000 7ff92a200640 1 -- 172.21.15.86:0/2740474313 shutdown_connections 2024-04-17T07:15:00.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.357+0000 7ff92a200640 1 -- 172.21.15.86:0/2740474313 wait complete. 2024-04-17T07:15:00.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.358+0000 7ff92a200640 1 Processor -- start 2024-04-17T07:15:00.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff92a200640 1 -- start start 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff92a200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9241041f0 0x7ff92419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff92a200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9241053e0 0x7ff92419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff92a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff924105da0 0x7ff9241a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff92a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff92410fd50 con 0x7ff924105da0 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff92a200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff92410fbd0 con 0x7ff9241053e0 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff92a200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff92410fed0 con 0x7ff9241041f0 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff923e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff924105da0 0x7ff9241a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff923400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9241041f0 0x7ff92419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff923e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff924105da0 0x7ff9241a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53042/0 (socket says 172.21.15.86:53042) 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff923400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9241041f0 0x7ff92419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38472/0 (socket says 172.21.15.86:38472) 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff923e00640 1 -- 172.21.15.86:0/1568450527 learned_addr learned my addr 172.21.15.86:0/1568450527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.359+0000 7ff922a00640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9241053e0 0x7ff92419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff923e00640 1 -- 172.21.15.86:0/1568450527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9241041f0 msgr2=0x7ff92419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff923e00640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9241041f0 0x7ff92419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff923e00640 1 -- 172.21.15.86:0/1568450527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9241053e0 msgr2=0x7ff92419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff923e00640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9241053e0 0x7ff92419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff923e00640 1 -- 172.21.15.86:0/1568450527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9241a1fc0 con 0x7ff924105da0 2024-04-17T07:15:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff923e00640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff924105da0 0x7ff9241a1950 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009fd0 tx=0x7ff91c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff91be00640 1 -- 172.21.15.86:0/1568450527 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c017070 con 0x7ff924105da0 2024-04-17T07:15:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff92a200640 1 -- 172.21.15.86:0/1568450527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9241a22b0 con 0x7ff924105da0 2024-04-17T07:15:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff91be00640 1 -- 172.21.15.86:0/1568450527 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff91c003e20 con 0x7ff924105da0 2024-04-17T07:15:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff91be00640 1 -- 172.21.15.86:0/1568450527 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c010460 con 0x7ff924105da0 2024-04-17T07:15:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.360+0000 7ff92a200640 1 -- 172.21.15.86:0/1568450527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9241a2800 con 0x7ff924105da0 2024-04-17T07:15:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.361+0000 7ff92a200640 1 -- 172.21.15.86:0/1568450527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92410b510 con 0x7ff924105da0 2024-04-17T07:15:00.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.364+0000 7ff91be00640 1 -- 172.21.15.86:0/1568450527 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff91c003970 con 0x7ff924105da0 2024-04-17T07:15:00.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.365+0000 7ff91be00640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9080a3c50 0x7ff9080a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:00.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.365+0000 7ff91be00640 1 -- 172.21.15.86:0/1568450527 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff91c0d0600 con 0x7ff924105da0 2024-04-17T07:15:00.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.365+0000 7ff923400640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9080a3c50 0x7ff9080a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:00.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.366+0000 7ff923400640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9080a3c50 0x7ff9080a6110 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff91402c240 tx=0x7ff91402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.374+0000 7ff91be00640 1 -- 172.21.15.86:0/1568450527 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff91c09cc00 con 0x7ff924105da0 2024-04-17T07:15:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:00 smithi098 ceph-mon[35418]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:15:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:00 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:00 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:00 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:00 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:00 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:00 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:00 smithi103 ceph-mon[35495]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:15:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:00 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:00 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:00 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:00 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:00 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:00 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:00 smithi086 ceph-mon[31733]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:15:00.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:00 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:00 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:00 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:00 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:00 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:00 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:00.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:00.636+0000 7ff92a200640 1 -- 172.21.15.86:0/1568450527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7ff924062b60 con 0x7ff924105da0 2024-04-17T07:15:01.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.229+0000 7ff91be00640 1 -- 172.21.15.86:0/1568450527 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7ff91c09c500 con 0x7ff924105da0 2024-04-17T07:15:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 -- 172.21.15.86:0/1568450527 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9080a3c50 msgr2=0x7ff9080a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9080a3c50 0x7ff9080a6110 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff91402c240 tx=0x7ff91402c7e0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 -- 172.21.15.86:0/1568450527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff924105da0 msgr2=0x7ff9241a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff924105da0 0x7ff9241a1950 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009fd0 tx=0x7ff91c00b260 comp rx=0 tx=0).stop 2024-04-17T07:15:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 -- 172.21.15.86:0/1568450527 shutdown_connections 2024-04-17T07:15:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9080a3c50 0x7ff9080a6110 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff924105da0 0x7ff9241a1950 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9241053e0 0x7ff92419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 --2- 172.21.15.86:0/1568450527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9241041f0 0x7ff92419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 -- 172.21.15.86:0/1568450527 >> 172.21.15.86:0/1568450527 conn(0x7ff9240ff780 msgr2=0x7ff9241086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 -- 172.21.15.86:0/1568450527 shutdown_connections 2024-04-17T07:15:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.233+0000 7ff92a200640 1 -- 172.21.15.86:0/1568450527 wait complete. 2024-04-17T07:15:01.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-17T07:15:01.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.466+0000 7f61b7600640 1 -- 172.21.15.86:0/2621780244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b01053e0 msgr2=0x7f61b0105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:01.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.466+0000 7f61b7600640 1 --2- 172.21.15.86:0/2621780244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b01053e0 0x7f61b0105860 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f619c009900 tx=0x7f619c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:01.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.466+0000 7f61b7600640 1 -- 172.21.15.86:0/2621780244 shutdown_connections 2024-04-17T07:15:01.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.466+0000 7f61b7600640 1 --2- 172.21.15.86:0/2621780244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b0105da0 0x7f61b010c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.466+0000 7f61b7600640 1 --2- 172.21.15.86:0/2621780244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b01053e0 0x7f61b0105860 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.466+0000 7f61b7600640 1 --2- 172.21.15.86:0/2621780244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b01041f0 0x7f61b0104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.466+0000 7f61b7600640 1 -- 172.21.15.86:0/2621780244 >> 172.21.15.86:0/2621780244 conn(0x7f61b00ff780 msgr2=0x7f61b0101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:01.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.466+0000 7f61b7600640 1 -- 172.21.15.86:0/2621780244 shutdown_connections 2024-04-17T07:15:01.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.466+0000 7f61b7600640 1 -- 172.21.15.86:0/2621780244 wait complete. 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.467+0000 7f61b7600640 1 Processor -- start 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.467+0000 7f61b7600640 1 -- start start 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.467+0000 7f61b7600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b01041f0 0x7f61b019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.467+0000 7f61b7600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b01053e0 0x7f61b019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.467+0000 7f61b7600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b0105da0 0x7f61b01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.467+0000 7f61b7600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61b010fd80 con 0x7f61b01053e0 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.467+0000 7f61b7600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f61b010fc00 con 0x7f61b01041f0 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.467+0000 7f61b7600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61b010ff00 con 0x7f61b0105da0 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.468+0000 7f61afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b01053e0 0x7f61b019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.468+0000 7f61afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b01053e0 0x7f61b019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53054/0 (socket says 172.21.15.86:53054) 2024-04-17T07:15:01.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.468+0000 7f61afe00640 1 -- 172.21.15.86:0/4133753478 learned_addr learned my addr 172.21.15.86:0/4133753478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.468+0000 7f61b4c00640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b01041f0 0x7f61b019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.468+0000 7f61afe00640 1 -- 172.21.15.86:0/4133753478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b0105da0 msgr2=0x7f61b01a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.468+0000 7f61afe00640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b0105da0 0x7f61b01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.468+0000 7f61afe00640 1 -- 172.21.15.86:0/4133753478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b01041f0 msgr2=0x7f61b019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.468+0000 7f61afe00640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b01041f0 0x7f61b019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.468+0000 7f61afe00640 1 -- 172.21.15.86:0/4133753478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b01a1ee0 con 0x7f61b01053e0 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.469+0000 7f61afe00640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b01053e0 0x7f61b019a800 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f619c000c00 tx=0x7f619c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.469+0000 7f61ad600640 1 -- 172.21.15.86:0/4133753478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c043020 con 0x7f61b01053e0 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.469+0000 7f61ad600640 1 -- 172.21.15.86:0/4133753478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f619c0054c0 con 0x7f61b01053e0 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.469+0000 7f61b7600640 1 -- 172.21.15.86:0/4133753478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b01a2170 con 0x7f61b01053e0 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.469+0000 7f61ad600640 1 -- 172.21.15.86:0/4133753478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c003d80 con 0x7f61b01053e0 2024-04-17T07:15:01.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.469+0000 7f61b7600640 1 -- 172.21.15.86:0/4133753478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b01a2710 con 0x7f61b01053e0 2024-04-17T07:15:01.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.470+0000 7f61b7600640 1 -- 172.21.15.86:0/4133753478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b010b4b0 con 0x7f61b01053e0 2024-04-17T07:15:01.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.472+0000 7f61ad600640 1 -- 172.21.15.86:0/4133753478 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f619c046020 con 0x7f61b01053e0 2024-04-17T07:15:01.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.474+0000 7f61ad600640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61980a3cd0 0x7f61980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:01.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.475+0000 7f61ad600640 1 -- 172.21.15.86:0/4133753478 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f619c0f2bf0 con 0x7f61b01053e0 2024-04-17T07:15:01.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.475+0000 7f61b4c00640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61980a3cd0 0x7f61980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:01.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.476+0000 7f61b4c00640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61980a3cd0 0x7f61980a6190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f61b0105240 tx=0x7f61a4005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:01.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.479+0000 7f61ad600640 1 -- 172.21.15.86:0/4133753478 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f619c04d020 con 0x7f61b01053e0 2024-04-17T07:15:01.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:01 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1568450527' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-17T07:15:01.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:01 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1568450527' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-17T07:15:01.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:01 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1568450527' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-17T07:15:01.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.713+0000 7f61b7600640 1 -- 172.21.15.86:0/4133753478 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f61b0062500 con 0x7f61980a3cd0 2024-04-17T07:15:01.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.754+0000 7f61ad600640 1 -- 172.21.15.86:0/4133753478 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f61b0062500 con 0x7f61980a3cd0 2024-04-17T07:15:01.754 INFO:teuthology.orchestra.run.smithi086.stdout:{} 2024-04-17T07:15:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 -- 172.21.15.86:0/4133753478 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61980a3cd0 msgr2=0x7f61980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61980a3cd0 0x7f61980a6190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f61b0105240 tx=0x7f61a4005c60 comp rx=0 tx=0).stop 2024-04-17T07:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 -- 172.21.15.86:0/4133753478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b01053e0 msgr2=0x7f61b019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b01053e0 0x7f61b019a800 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f619c000c00 tx=0x7f619c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 -- 172.21.15.86:0/4133753478 shutdown_connections 2024-04-17T07:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61980a3cd0 0x7f61980a6190 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b0105da0 0x7f61b01a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b01053e0 0x7f61b019a800 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 --2- 172.21.15.86:0/4133753478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61b01041f0 0x7f61b019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 -- 172.21.15.86:0/4133753478 >> 172.21.15.86:0/4133753478 conn(0x7f61b00ff780 msgr2=0x7f61b0108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 -- 172.21.15.86:0/4133753478 shutdown_connections 2024-04-17T07:15:01.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:01.756+0000 7f61b7600640 1 -- 172.21.15.86:0/4133753478 wait complete. 2024-04-17T07:15:01.775 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-17T07:15:01.792 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-17T07:15:01.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:15:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.010+0000 7f34e6000640 1 -- 172.21.15.86:0/738689532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e01095d0 msgr2=0x7f34e0109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.010+0000 7f34e6000640 1 --2- 172.21.15.86:0/738689532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e01095d0 0x7f34e0109a50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009900 tx=0x7f34cc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.011+0000 7f34e6000640 1 -- 172.21.15.86:0/738689532 shutdown_connections 2024-04-17T07:15:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.011+0000 7f34e6000640 1 --2- 172.21.15.86:0/738689532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e0109f90 0x7f34e0110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.011+0000 7f34e6000640 1 --2- 172.21.15.86:0/738689532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e01095d0 0x7f34e0109a50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.011+0000 7f34e6000640 1 --2- 172.21.15.86:0/738689532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e01083e0 0x7f34e0108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.011+0000 7f34e6000640 1 -- 172.21.15.86:0/738689532 >> 172.21.15.86:0/738689532 conn(0x7f34e00773a0 msgr2=0x7f34e00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.011+0000 7f34e6000640 1 -- 172.21.15.86:0/738689532 shutdown_connections 2024-04-17T07:15:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.011+0000 7f34e6000640 1 -- 172.21.15.86:0/738689532 wait complete. 2024-04-17T07:15:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34e6000640 1 Processor -- start 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34e6000640 1 -- start start 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34e6000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e01083e0 0x7f34e019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34e6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e01095d0 0x7f34e019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34e6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e0109f90 0x7f34e01a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34e6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0113f20 con 0x7f34e01095d0 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34e6000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0113da0 con 0x7f34e01083e0 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34e6000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01140a0 con 0x7f34e0109f90 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e01095d0 0x7f34e019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e01095d0 0x7f34e019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53090/0 (socket says 172.21.15.86:53090) 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.012+0000 7f34dfe00640 1 -- 172.21.15.86:0/157524349 learned_addr learned my addr 172.21.15.86:0/157524349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.013+0000 7f34e4c00640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e01083e0 0x7f34e019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.013+0000 7f34e5600640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e0109f90 0x7f34e01a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.013+0000 7f34dfe00640 1 -- 172.21.15.86:0/157524349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e0109f90 msgr2=0x7f34e01a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.013+0000 7f34dfe00640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e0109f90 0x7f34e01a5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.013+0000 7f34dfe00640 1 -- 172.21.15.86:0/157524349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e01083e0 msgr2=0x7f34e019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.013+0000 7f34dfe00640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e01083e0 0x7f34e019e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.013+0000 7f34dfe00640 1 -- 172.21.15.86:0/157524349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e01a6230 con 0x7f34e01095d0 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.013+0000 7f34dfe00640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e01095d0 0x7f34e019eb50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f34cc006010 tx=0x7f34cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.014+0000 7f34dd600640 1 -- 172.21.15.86:0/157524349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc043020 con 0x7f34e01095d0 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.014+0000 7f34e6000640 1 -- 172.21.15.86:0/157524349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e01a64c0 con 0x7f34e01095d0 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.014+0000 7f34dd600640 1 -- 172.21.15.86:0/157524349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34cc004aa0 con 0x7f34e01095d0 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.014+0000 7f34dd600640 1 -- 172.21.15.86:0/157524349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc003c60 con 0x7f34e01095d0 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.014+0000 7f34e6000640 1 -- 172.21.15.86:0/157524349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e01a6a60 con 0x7f34e01095d0 2024-04-17T07:15:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.015+0000 7f34e6000640 1 -- 172.21.15.86:0/157524349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e010f770 con 0x7f34e01095d0 2024-04-17T07:15:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.017+0000 7f34dd600640 1 -- 172.21.15.86:0/157524349 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f34cc046020 con 0x7f34e01095d0 2024-04-17T07:15:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.018+0000 7f34dd600640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34c80a3bb0 0x7f34c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.018+0000 7f34dd600640 1 -- 172.21.15.86:0/157524349 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f34cc0f3250 con 0x7f34e01095d0 2024-04-17T07:15:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.018+0000 7f34e4c00640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34c80a3bb0 0x7f34c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.019+0000 7f34e4c00640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34c80a3bb0 0x7f34c80a6070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f34d4005fd0 tx=0x7f34d4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.022+0000 7f34dd600640 1 -- 172.21.15.86:0/157524349 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f34cc04d020 con 0x7f34e01095d0 2024-04-17T07:15:02.080 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:15:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:15:01.725+0000 7ffa4ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:15:02.081 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:15:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:15:01.725+0000 7ffa4ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:15:02.081 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:15:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:15:01.725+0000 7ffa4ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:15:02.081 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:15:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:15:01.725+0000 7ffa4ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:15:02.081 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:15:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:15:01.725+0000 7ffa4ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-17T07:15:02.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.232+0000 7f34e6000640 1 -- 172.21.15.86:0/157524349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34e002d7d0 con 0x7f34e01095d0 2024-04-17T07:15:02.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.236+0000 7f34dd600640 1 -- 172.21.15.86:0/157524349 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7f34cc0bf8b0 con 0x7f34e01095d0 2024-04-17T07:15:02.235 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:02.235 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:15:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 -- 172.21.15.86:0/157524349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34c80a3bb0 msgr2=0x7f34c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34c80a3bb0 0x7f34c80a6070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f34d4005fd0 tx=0x7f34d4005850 comp rx=0 tx=0).stop 2024-04-17T07:15:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 -- 172.21.15.86:0/157524349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e01095d0 msgr2=0x7f34e019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e01095d0 0x7f34e019eb50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f34cc006010 tx=0x7f34cc0042f0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 -- 172.21.15.86:0/157524349 shutdown_connections 2024-04-17T07:15:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34c80a3bb0 0x7f34c80a6070 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e0109f90 0x7f34e01a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e01095d0 0x7f34e019eb50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 --2- 172.21.15.86:0/157524349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34e01083e0 0x7f34e019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 -- 172.21.15.86:0/157524349 >> 172.21.15.86:0/157524349 conn(0x7f34e00773a0 msgr2=0x7f34e010c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 -- 172.21.15.86:0/157524349 shutdown_connections 2024-04-17T07:15:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.239+0000 7f34e6000640 1 -- 172.21.15.86:0/157524349 wait complete. 2024-04-17T07:15:02.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:15:02.258 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:15:02.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.465+0000 7f0473200640 1 -- 172.21.15.86:0/1555054455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0474101d80 msgr2=0x7f04741021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.465+0000 7f0473200640 1 --2- 172.21.15.86:0/1555054455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0474101d80 0x7f04741021a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0464009900 tx=0x7f046402bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:02.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.465+0000 7f0473200640 1 -- 172.21.15.86:0/1555054455 shutdown_connections 2024-04-17T07:15:02.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.465+0000 7f0473200640 1 --2- 172.21.15.86:0/1555054455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0474103930 0x7f047410a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.465+0000 7f0473200640 1 --2- 172.21.15.86:0/1555054455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0474102f70 0x7f04741033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.465+0000 7f0473200640 1 --2- 172.21.15.86:0/1555054455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0474101d80 0x7f04741021a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.465+0000 7f0473200640 1 -- 172.21.15.86:0/1555054455 >> 172.21.15.86:0/1555054455 conn(0x7f04740fd2f0 msgr2=0x7f04740ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:02.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.465+0000 7f0473200640 1 -- 172.21.15.86:0/1555054455 shutdown_connections 2024-04-17T07:15:02.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.465+0000 7f0473200640 1 -- 172.21.15.86:0/1555054455 wait complete. 2024-04-17T07:15:02.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.466+0000 7f0473200640 1 Processor -- start 2024-04-17T07:15:02.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.466+0000 7f0473200640 1 -- start start 2024-04-17T07:15:02.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0473200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0474101d80 0x7f047419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0473200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0474102f70 0x7f047419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0473200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0474103930 0x7f04741a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0473200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f047410d8b0 con 0x7f0474101d80 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0473200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f047410d730 con 0x7f0474103930 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0473200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f047410da30 con 0x7f0474102f70 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0471e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0474101d80 0x7f047419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0471e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0474101d80 0x7f047419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53100/0 (socket says 172.21.15.86:53100) 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0471e00640 1 -- 172.21.15.86:0/4273247980 learned_addr learned my addr 172.21.15.86:0/4273247980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0471e00640 1 -- 172.21.15.86:0/4273247980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0474102f70 msgr2=0x7f047419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0471e00640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0474102f70 0x7f047419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0471e00640 1 -- 172.21.15.86:0/4273247980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0474103930 msgr2=0x7f04741a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0472800640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0474103930 0x7f04741a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0471e00640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0474103930 0x7f04741a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.467+0000 7f0471e00640 1 -- 172.21.15.86:0/4273247980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04741a1df0 con 0x7f0474101d80 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.468+0000 7f0471e00640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0474101d80 0x7f047419a1d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0464000c00 tx=0x7f04640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.468+0000 7f046aa00640 1 -- 172.21.15.86:0/4273247980 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0464043020 con 0x7f0474101d80 2024-04-17T07:15:02.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.468+0000 7f046aa00640 1 -- 172.21.15.86:0/4273247980 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04640054c0 con 0x7f0474101d80 2024-04-17T07:15:02.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.468+0000 7f0473200640 1 -- 172.21.15.86:0/4273247980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04741a2080 con 0x7f0474101d80 2024-04-17T07:15:02.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.468+0000 7f046aa00640 1 -- 172.21.15.86:0/4273247980 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046403e450 con 0x7f0474101d80 2024-04-17T07:15:02.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.468+0000 7f0473200640 1 -- 172.21.15.86:0/4273247980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04741a2620 con 0x7f0474101d80 2024-04-17T07:15:02.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.469+0000 7f0473200640 1 -- 172.21.15.86:0/4273247980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04741a2820 con 0x7f0474101d80 2024-04-17T07:15:02.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.471+0000 7f046aa00640 1 -- 172.21.15.86:0/4273247980 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0464046020 con 0x7f0474101d80 2024-04-17T07:15:02.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.472+0000 7f046aa00640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04580a3c30 0x7f04580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.473+0000 7f046aa00640 1 -- 172.21.15.86:0/4273247980 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f04640f2c10 con 0x7f0474101d80 2024-04-17T07:15:02.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.473+0000 7f0471400640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04580a3c30 0x7f04580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.473+0000 7f0471400640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04580a3c30 0x7f04580a60f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f047419b6e0 tx=0x7f0460005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:02.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.477+0000 7f046aa00640 1 -- 172.21.15.86:0/4273247980 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f046404d020 con 0x7f0474101d80 2024-04-17T07:15:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:02 smithi098 ceph-mon[35418]: pgmap v460: 129 pgs: 129 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-17T07:15:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:02 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1568450527' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-17T07:15:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:02 smithi098 ceph-mon[35418]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-17T07:15:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:02 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:15:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:02 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:15:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:02 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:02 smithi098 ceph-mon[35418]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:15:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:02 smithi103 ceph-mon[35495]: pgmap v460: 129 pgs: 129 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-17T07:15:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:02 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1568450527' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-17T07:15:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:02 smithi103 ceph-mon[35495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-17T07:15:02.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:02 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:15:02.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:02 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:15:02.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:02 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:02.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:02 smithi103 ceph-mon[35495]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:15:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:02 smithi086 ceph-mon[31733]: pgmap v460: 129 pgs: 129 active+clean; 645 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-17T07:15:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:02 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1568450527' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-17T07:15:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:02 smithi086 ceph-mon[31733]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-17T07:15:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:02 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:15:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:02 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:15:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:02 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:15:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:02 smithi086 ceph-mon[31733]: from='client.15627 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-17T07:15:02.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.680+0000 7f0473200640 1 -- 172.21.15.86:0/4273247980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04741a2bb0 con 0x7f0474101d80 2024-04-17T07:15:02.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.681+0000 7f046aa00640 1 -- 172.21.15.86:0/4273247980 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f04640bf210 con 0x7f0474101d80 2024-04-17T07:15:02.680 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:15:02.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 -- 172.21.15.86:0/4273247980 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04580a3c30 msgr2=0x7f04580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04580a3c30 0x7f04580a60f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f047419b6e0 tx=0x7f0460005c70 comp rx=0 tx=0).stop 2024-04-17T07:15:02.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 -- 172.21.15.86:0/4273247980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0474101d80 msgr2=0x7f047419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0474101d80 0x7f047419a1d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0464000c00 tx=0x7f04640042f0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 -- 172.21.15.86:0/4273247980 shutdown_connections 2024-04-17T07:15:02.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04580a3c30 0x7f04580a60f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0474103930 0x7f04741a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0474102f70 0x7f047419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 --2- 172.21.15.86:0/4273247980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0474101d80 0x7f047419a1d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 -- 172.21.15.86:0/4273247980 >> 172.21.15.86:0/4273247980 conn(0x7f04740fd2f0 msgr2=0x7f0474074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:02.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 -- 172.21.15.86:0/4273247980 shutdown_connections 2024-04-17T07:15:02.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.684+0000 7f0473200640 1 -- 172.21.15.86:0/4273247980 wait complete. 2024-04-17T07:15:02.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:15:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.911+0000 7fe919a00640 1 -- 172.21.15.86:0/2984328728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe914103980 msgr2=0x7fe91410a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.911+0000 7fe919a00640 1 --2- 172.21.15.86:0/2984328728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe914103980 0x7fe91410a2a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009ca0 tx=0x7fe90c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.911+0000 7fe919a00640 1 -- 172.21.15.86:0/2984328728 shutdown_connections 2024-04-17T07:15:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.911+0000 7fe919a00640 1 --2- 172.21.15.86:0/2984328728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe914103980 0x7fe91410a2a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.911+0000 7fe919a00640 1 --2- 172.21.15.86:0/2984328728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe914102fc0 0x7fe914103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.911+0000 7fe919a00640 1 --2- 172.21.15.86:0/2984328728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe914101e70 0x7fe914102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.911+0000 7fe919a00640 1 -- 172.21.15.86:0/2984328728 >> 172.21.15.86:0/2984328728 conn(0x7fe9140fd400 msgr2=0x7fe9140ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.911+0000 7fe919a00640 1 -- 172.21.15.86:0/2984328728 shutdown_connections 2024-04-17T07:15:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.911+0000 7fe919a00640 1 -- 172.21.15.86:0/2984328728 wait complete. 2024-04-17T07:15:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.912+0000 7fe919a00640 1 Processor -- start 2024-04-17T07:15:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.912+0000 7fe919a00640 1 -- start start 2024-04-17T07:15:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.912+0000 7fe919a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe914101e70 0x7fe914198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.912+0000 7fe919a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe914102fc0 0x7fe914198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.912+0000 7fe919a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe914103980 0x7fe91419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.912+0000 7fe919a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410d9d0 con 0x7fe914103980 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.912+0000 7fe919a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410d850 con 0x7fe914101e70 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.912+0000 7fe919a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410db50 con 0x7fe914102fc0 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe919000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe914103980 0x7fe91419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe919000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe914103980 0x7fe91419f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53110/0 (socket says 172.21.15.86:53110) 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe919000640 1 -- 172.21.15.86:0/1695476387 learned_addr learned my addr 172.21.15.86:0/1695476387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe913e00640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe914101e70 0x7fe914198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe913400640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe914102fc0 0x7fe914198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe919000640 1 -- 172.21.15.86:0/1695476387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe914102fc0 msgr2=0x7fe914198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe919000640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe914102fc0 0x7fe914198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe919000640 1 -- 172.21.15.86:0/1695476387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe914101e70 msgr2=0x7fe914198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe919000640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe914101e70 0x7fe914198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe919000640 1 -- 172.21.15.86:0/1695476387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91419fc40 con 0x7fe914103980 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe913400640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe914102fc0 0x7fe914198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.913+0000 7fe919000640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe914103980 0x7fe91419f5d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe90c032380 tx=0x7fe90c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.914+0000 7fe910c00640 1 -- 172.21.15.86:0/1695476387 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c041020 con 0x7fe914103980 2024-04-17T07:15:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.914+0000 7fe910c00640 1 -- 172.21.15.86:0/1695476387 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe90c003e20 con 0x7fe914103980 2024-04-17T07:15:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.914+0000 7fe919a00640 1 -- 172.21.15.86:0/1695476387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91419fed0 con 0x7fe914103980 2024-04-17T07:15:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.914+0000 7fe910c00640 1 -- 172.21.15.86:0/1695476387 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c039040 con 0x7fe914103980 2024-04-17T07:15:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.914+0000 7fe919a00640 1 -- 172.21.15.86:0/1695476387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9141a0330 con 0x7fe914103980 2024-04-17T07:15:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.915+0000 7fe919a00640 1 -- 172.21.15.86:0/1695476387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe914109180 con 0x7fe914103980 2024-04-17T07:15:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.917+0000 7fe910c00640 1 -- 172.21.15.86:0/1695476387 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe90c044020 con 0x7fe914103980 2024-04-17T07:15:02.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.918+0000 7fe910c00640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8f80a3bb0 0x7fe8f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:02.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.918+0000 7fe910c00640 1 -- 172.21.15.86:0/1695476387 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe90c0f2ad0 con 0x7fe914103980 2024-04-17T07:15:02.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.919+0000 7fe913e00640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8f80a3bb0 0x7fe8f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:02.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.919+0000 7fe913e00640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8f80a3bb0 0x7fe8f80a6070 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc005fd0 tx=0x7fe8fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:02.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:02.923+0000 7fe910c00640 1 -- 172.21.15.86:0/1695476387 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe90c04b020 con 0x7fe914103980 2024-04-17T07:15:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.124+0000 7fe919a00640 1 -- 172.21.15.86:0/1695476387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe914062b60 con 0x7fe914103980 2024-04-17T07:15:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.125+0000 7fe910c00640 1 -- 172.21.15.86:0/1695476387 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fe90c0bf0d0 con 0x7fe914103980 2024-04-17T07:15:03.124 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 -- 172.21.15.86:0/1695476387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8f80a3bb0 msgr2=0x7fe8f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8f80a3bb0 0x7fe8f80a6070 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc005fd0 tx=0x7fe8fc005850 comp rx=0 tx=0).stop 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 -- 172.21.15.86:0/1695476387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe914103980 msgr2=0x7fe91419f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe914103980 0x7fe91419f5d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe90c032380 tx=0x7fe90c032820 comp rx=0 tx=0).stop 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 -- 172.21.15.86:0/1695476387 shutdown_connections 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8f80a3bb0 0x7fe8f80a6070 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe914103980 0x7fe91419f5d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe914102fc0 0x7fe914198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 --2- 172.21.15.86:0/1695476387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe914101e70 0x7fe914198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 -- 172.21.15.86:0/1695476387 >> 172.21.15.86:0/1695476387 conn(0x7fe9140fd400 msgr2=0x7fe914106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 -- 172.21.15.86:0/1695476387 shutdown_connections 2024-04-17T07:15:03.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.128+0000 7fe919a00640 1 -- 172.21.15.86:0/1695476387 wait complete. 2024-04-17T07:15:03.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:15:03.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.368+0000 7f94b0800640 1 -- 172.21.15.86:0/3618177986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a81041a0 msgr2=0x7f94a81045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.368+0000 7f94b0800640 1 --2- 172.21.15.86:0/3618177986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a81041a0 0x7f94a81045c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f949c009900 tx=0x7f949c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.368+0000 7f94b0800640 1 -- 172.21.15.86:0/3618177986 shutdown_connections 2024-04-17T07:15:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.368+0000 7f94b0800640 1 --2- 172.21.15.86:0/3618177986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94a8105d50 0x7f94a810c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.368+0000 7f94b0800640 1 --2- 172.21.15.86:0/3618177986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8105390 0x7f94a8105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.368+0000 7f94b0800640 1 --2- 172.21.15.86:0/3618177986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a81041a0 0x7f94a81045c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.368+0000 7f94b0800640 1 -- 172.21.15.86:0/3618177986 >> 172.21.15.86:0/3618177986 conn(0x7f94a80ff750 msgr2=0x7f94a8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.369+0000 7f94b0800640 1 -- 172.21.15.86:0/3618177986 shutdown_connections 2024-04-17T07:15:03.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.369+0000 7f94b0800640 1 -- 172.21.15.86:0/3618177986 wait complete. 2024-04-17T07:15:03.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94b0800640 1 Processor -- start 2024-04-17T07:15:03.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94b0800640 1 -- start start 2024-04-17T07:15:03.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94b0800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a81041a0 0x7f94a819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:03.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94b0800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94a8105390 0x7f94a819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94b0800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8105d50 0x7f94a81a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94b0800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94a810fd30 con 0x7f94a81041a0 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94b0800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f94a810fbb0 con 0x7f94a8105390 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94b0800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94a810feb0 con 0x7f94a8105d50 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a81041a0 0x7f94a819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a81041a0 0x7f94a819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53132/0 (socket says 172.21.15.86:53132) 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.370+0000 7f94ade00640 1 -- 172.21.15.86:0/1248266038 learned_addr learned my addr 172.21.15.86:0/1248266038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.371+0000 7f94ad400640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94a8105390 0x7f94a819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.371+0000 7f94ade00640 1 -- 172.21.15.86:0/1248266038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8105d50 msgr2=0x7f94a81a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.371+0000 7f94ae800640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8105d50 0x7f94a81a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.371+0000 7f94ade00640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8105d50 0x7f94a81a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.371+0000 7f94ade00640 1 -- 172.21.15.86:0/1248266038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94a8105390 msgr2=0x7f94a819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:03.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.371+0000 7f94ade00640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94a8105390 0x7f94a819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.371+0000 7f94ade00640 1 -- 172.21.15.86:0/1248266038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a81a2030 con 0x7f94a81041a0 2024-04-17T07:15:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.371+0000 7f94ade00640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a81041a0 0x7f94a819a410 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f949c005ab0 tx=0x7f949c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.371+0000 7f94a2a00640 1 -- 172.21.15.86:0/1248266038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c043020 con 0x7f94a81041a0 2024-04-17T07:15:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.372+0000 7f94a2a00640 1 -- 172.21.15.86:0/1248266038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f949c0054c0 con 0x7f94a81041a0 2024-04-17T07:15:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.372+0000 7f94b0800640 1 -- 172.21.15.86:0/1248266038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81a22c0 con 0x7f94a81041a0 2024-04-17T07:15:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.372+0000 7f94a2a00640 1 -- 172.21.15.86:0/1248266038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c003d10 con 0x7f94a81041a0 2024-04-17T07:15:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.372+0000 7f94b0800640 1 -- 172.21.15.86:0/1248266038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81a2810 con 0x7f94a81041a0 2024-04-17T07:15:03.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.373+0000 7f94b0800640 1 -- 172.21.15.86:0/1248266038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a810b590 con 0x7f94a81041a0 2024-04-17T07:15:03.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.375+0000 7f94a2a00640 1 -- 172.21.15.86:0/1248266038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f949c046020 con 0x7f94a81041a0 2024-04-17T07:15:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.376+0000 7f94a2a00640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94900a3bb0 0x7f94900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:03.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.376+0000 7f94a2a00640 1 -- 172.21.15.86:0/1248266038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f949c0f2ff0 con 0x7f94a81041a0 2024-04-17T07:15:03.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.377+0000 7f94ad400640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94900a3bb0 0x7f94900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:03.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.377+0000 7f94ad400640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94900a3bb0 0x7f94900a6070 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9498005fd0 tx=0x7f9498005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:03.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.378+0000 7f94a2a00640 1 -- 172.21.15.86:0/1248266038 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f949c04d020 con 0x7f94a81041a0 2024-04-17T07:15:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:03 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/157524349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:03 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4273247980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:15:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:03 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1695476387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:15:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:03 smithi098 ceph-mon[35418]: pgmap v461: 129 pgs: 129 active+clean; 651 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-17T07:15:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:03 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/157524349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:03 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4273247980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:15:03.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:03 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1695476387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:15:03.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:03 smithi103 ceph-mon[35495]: pgmap v461: 129 pgs: 129 active+clean; 651 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-17T07:15:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:03 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/157524349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:03 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4273247980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:15:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:03 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1695476387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:15:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:03 smithi086 ceph-mon[31733]: pgmap v461: 129 pgs: 129 active+clean; 651 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-04-17T07:15:03.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.586+0000 7f94b0800640 1 -- 172.21.15.86:0/1248266038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f94a8062b60 con 0x7f94a81041a0 2024-04-17T07:15:03.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.587+0000 7f94a2a00640 1 -- 172.21.15.86:0/1248266038 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f949c0bf5f0 con 0x7f94a81041a0 2024-04-17T07:15:03.586 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:15:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.589+0000 7f94b0800640 1 -- 172.21.15.86:0/1248266038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94900a3bb0 msgr2=0x7f94900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.589+0000 7f94b0800640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94900a3bb0 0x7f94900a6070 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9498005fd0 tx=0x7f9498005850 comp rx=0 tx=0).stop 2024-04-17T07:15:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.589+0000 7f94b0800640 1 -- 172.21.15.86:0/1248266038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a81041a0 msgr2=0x7f94a819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.589+0000 7f94b0800640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a81041a0 0x7f94a819a410 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f949c005ab0 tx=0x7f949c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.590+0000 7f94b0800640 1 -- 172.21.15.86:0/1248266038 shutdown_connections 2024-04-17T07:15:03.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.590+0000 7f94b0800640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94900a3bb0 0x7f94900a6070 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.590+0000 7f94b0800640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94a8105d50 0x7f94a81a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.590+0000 7f94b0800640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94a8105390 0x7f94a819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.590+0000 7f94b0800640 1 --2- 172.21.15.86:0/1248266038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94a81041a0 0x7f94a819a410 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.590+0000 7f94b0800640 1 -- 172.21.15.86:0/1248266038 >> 172.21.15.86:0/1248266038 conn(0x7f94a80ff750 msgr2=0x7f94a8108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.590+0000 7f94b0800640 1 -- 172.21.15.86:0/1248266038 shutdown_connections 2024-04-17T07:15:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.590+0000 7f94b0800640 1 -- 172.21.15.86:0/1248266038 wait complete. 2024-04-17T07:15:03.607 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:15:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.814+0000 7f356b200640 1 -- 172.21.15.86:0/1193411655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c077b60 msgr2=0x7f356c077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.814+0000 7f356b200640 1 --2- 172.21.15.86:0/1193411655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c077b60 0x7f356c077fc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3554009900 tx=0x7f355402bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.814+0000 7f356b200640 1 -- 172.21.15.86:0/1193411655 shutdown_connections 2024-04-17T07:15:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.814+0000 7f356b200640 1 --2- 172.21.15.86:0/1193411655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c078500 0x7f356c078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.814+0000 7f356b200640 1 --2- 172.21.15.86:0/1193411655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c077b60 0x7f356c077fc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.814+0000 7f356b200640 1 --2- 172.21.15.86:0/1193411655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f356c076920 0x7f356c076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.814+0000 7f356b200640 1 -- 172.21.15.86:0/1193411655 >> 172.21.15.86:0/1193411655 conn(0x7f356c0ff610 msgr2=0x7f356c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.814+0000 7f356b200640 1 -- 172.21.15.86:0/1193411655 shutdown_connections 2024-04-17T07:15:03.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.814+0000 7f356b200640 1 -- 172.21.15.86:0/1193411655 wait complete. 2024-04-17T07:15:03.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.815+0000 7f356b200640 1 Processor -- start 2024-04-17T07:15:03.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f356b200640 1 -- start start 2024-04-17T07:15:03.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f356b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f356c076920 0x7f356c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:03.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f356b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c077b60 0x7f356c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:03.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f356b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c078500 0x7f356c1a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:03.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f356b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f356c114020 con 0x7f356c077b60 2024-04-17T07:15:03.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f356b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f356c113ea0 con 0x7f356c076920 2024-04-17T07:15:03.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f356b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1141a0 con 0x7f356c078500 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f3569400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c077b60 0x7f356c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f3569400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c077b60 0x7f356c19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53142/0 (socket says 172.21.15.86:53142) 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f3569400640 1 -- 172.21.15.86:0/1618135384 learned_addr learned my addr 172.21.15.86:0/1618135384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.816+0000 7f3569e00640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f356c076920 0x7f356c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.817+0000 7f3569400640 1 -- 172.21.15.86:0/1618135384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c078500 msgr2=0x7f356c1a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.817+0000 7f3569400640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c078500 0x7f356c1a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.817+0000 7f3569400640 1 -- 172.21.15.86:0/1618135384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f356c076920 msgr2=0x7f356c19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.817+0000 7f3569400640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f356c076920 0x7f356c19e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.817+0000 7f3569400640 1 -- 172.21.15.86:0/1618135384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356c1a62d0 con 0x7f356c077b60 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.817+0000 7f3569400640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c077b60 0x7f356c19ebf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3554000c00 tx=0x7f35540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.817+0000 7f3562a00640 1 -- 172.21.15.86:0/1618135384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554043020 con 0x7f356c077b60 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.817+0000 7f356b200640 1 -- 172.21.15.86:0/1618135384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356c1a6560 con 0x7f356c077b60 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.817+0000 7f3562a00640 1 -- 172.21.15.86:0/1618135384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35540054c0 con 0x7f356c077b60 2024-04-17T07:15:03.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.817+0000 7f3562a00640 1 -- 172.21.15.86:0/1618135384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554003d70 con 0x7f356c077b60 2024-04-17T07:15:03.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.818+0000 7f356b200640 1 -- 172.21.15.86:0/1618135384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356c1a6ab0 con 0x7f356c077b60 2024-04-17T07:15:03.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.819+0000 7f356b200640 1 -- 172.21.15.86:0/1618135384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f356c110a10 con 0x7f356c077b60 2024-04-17T07:15:03.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.821+0000 7f3562a00640 1 -- 172.21.15.86:0/1618135384 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3554046020 con 0x7f356c077b60 2024-04-17T07:15:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.823+0000 7f3562a00640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35500a3bb0 0x7f35500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.823+0000 7f3562a00640 1 -- 172.21.15.86:0/1618135384 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f35540f2ce0 con 0x7f356c077b60 2024-04-17T07:15:03.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.823+0000 7f3569e00640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35500a3bb0 0x7f35500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:03.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.824+0000 7f3569e00640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35500a3bb0 0x7f35500a6070 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f355c005d00 tx=0x7f355c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:03.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:03.826+0000 7f3562a00640 1 -- 172.21.15.86:0/1618135384 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f355404d020 con 0x7f356c077b60 2024-04-17T07:15:04.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.028+0000 7f356b200640 1 -- 172.21.15.86:0/1618135384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f356c062b60 con 0x7f356c077b60 2024-04-17T07:15:04.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.028+0000 7f3562a00640 1 -- 172.21.15.86:0/1618135384 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f35540bf400 con 0x7f356c077b60 2024-04-17T07:15:04.027 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:15:04.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.031+0000 7f356b200640 1 -- 172.21.15.86:0/1618135384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35500a3bb0 msgr2=0x7f35500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.031+0000 7f356b200640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35500a3bb0 0x7f35500a6070 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f355c005d00 tx=0x7f355c005c50 comp rx=0 tx=0).stop 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.031+0000 7f356b200640 1 -- 172.21.15.86:0/1618135384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c077b60 msgr2=0x7f356c19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.031+0000 7f356b200640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c077b60 0x7f356c19ebf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3554000c00 tx=0x7f35540042f0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.031+0000 7f356b200640 1 -- 172.21.15.86:0/1618135384 shutdown_connections 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.031+0000 7f356b200640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35500a3bb0 0x7f35500a6070 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.031+0000 7f356b200640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c078500 0x7f356c1a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.031+0000 7f356b200640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f356c077b60 0x7f356c19ebf0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.031+0000 7f356b200640 1 --2- 172.21.15.86:0/1618135384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f356c076920 0x7f356c19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.031+0000 7f356b200640 1 -- 172.21.15.86:0/1618135384 >> 172.21.15.86:0/1618135384 conn(0x7f356c0ff610 msgr2=0x7f356c101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.032+0000 7f356b200640 1 -- 172.21.15.86:0/1618135384 shutdown_connections 2024-04-17T07:15:04.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.032+0000 7f356b200640 1 -- 172.21.15.86:0/1618135384 wait complete. 2024-04-17T07:15:04.050 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:15:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.270+0000 7f08d2a00640 1 -- 172.21.15.86:0/935232071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc1083e0 msgr2=0x7f08cc108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.270+0000 7f08d2a00640 1 --2- 172.21.15.86:0/935232071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc1083e0 0x7f08cc108800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009900 tx=0x7f08c002bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.271+0000 7f08d2a00640 1 -- 172.21.15.86:0/935232071 shutdown_connections 2024-04-17T07:15:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.271+0000 7f08d2a00640 1 --2- 172.21.15.86:0/935232071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc109f90 0x7f08cc110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.271+0000 7f08d2a00640 1 --2- 172.21.15.86:0/935232071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1095d0 0x7f08cc109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.271+0000 7f08d2a00640 1 --2- 172.21.15.86:0/935232071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc1083e0 0x7f08cc108800 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.271+0000 7f08d2a00640 1 -- 172.21.15.86:0/935232071 >> 172.21.15.86:0/935232071 conn(0x7f08cc0773a0 msgr2=0x7f08cc0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.271+0000 7f08d2a00640 1 -- 172.21.15.86:0/935232071 shutdown_connections 2024-04-17T07:15:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.271+0000 7f08d2a00640 1 -- 172.21.15.86:0/935232071 wait complete. 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.272+0000 7f08d2a00640 1 Processor -- start 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.272+0000 7f08d2a00640 1 -- start start 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.272+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1083e0 0x7f08cc19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.272+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc1095d0 0x7f08cc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.272+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc109f90 0x7f08cc1a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.272+0000 7f08d2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc113f20 con 0x7f08cc1095d0 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.272+0000 7f08d2a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc113da0 con 0x7f08cc109f90 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.272+0000 7f08d2a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc1140a0 con 0x7f08cc1083e0 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc1095d0 0x7f08cc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc1095d0 0x7f08cc19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53164/0 (socket says 172.21.15.86:53164) 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d0c00640 1 -- 172.21.15.86:0/2149638344 learned_addr learned my addr 172.21.15.86:0/2149638344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d2000640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc109f90 0x7f08cc1a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d1600640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1083e0 0x7f08cc19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d0c00640 1 -- 172.21.15.86:0/2149638344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1083e0 msgr2=0x7f08cc19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d0c00640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1083e0 0x7f08cc19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d0c00640 1 -- 172.21.15.86:0/2149638344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc109f90 msgr2=0x7f08cc1a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d0c00640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc109f90 0x7f08cc1a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d0c00640 1 -- 172.21.15.86:0/2149638344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc1a6260 con 0x7f08cc1095d0 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d2000640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc109f90 0x7f08cc1a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.273+0000 7f08d0c00640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc1095d0 0x7f08cc19eb80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f08b800d770 tx=0x7f08b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.274+0000 7f08ca000640 1 -- 172.21.15.86:0/2149638344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8017070 con 0x7f08cc1095d0 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.274+0000 7f08ca000640 1 -- 172.21.15.86:0/2149638344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08b80054d0 con 0x7f08cc1095d0 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.274+0000 7f08d2a00640 1 -- 172.21.15.86:0/2149638344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc1a6550 con 0x7f08cc1095d0 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.274+0000 7f08ca000640 1 -- 172.21.15.86:0/2149638344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8004cf0 con 0x7f08cc1095d0 2024-04-17T07:15:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.274+0000 7f08d2a00640 1 -- 172.21.15.86:0/2149638344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc1a6a50 con 0x7f08cc1095d0 2024-04-17T07:15:04.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.275+0000 7f08d2a00640 1 -- 172.21.15.86:0/2149638344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08cc10f720 con 0x7f08cc1095d0 2024-04-17T07:15:04.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.278+0000 7f08ca000640 1 -- 172.21.15.86:0/2149638344 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f08b8004e90 con 0x7f08cc1095d0 2024-04-17T07:15:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.279+0000 7f08ca000640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f08b40a3c40 0x7f08b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.279+0000 7f08ca000640 1 -- 172.21.15.86:0/2149638344 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f08b80d0490 con 0x7f08cc1095d0 2024-04-17T07:15:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.279+0000 7f08d1600640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f08b40a3c40 0x7f08b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.280+0000 7f08d1600640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f08b40a3c40 0x7f08b40a6100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f08c0036040 tx=0x7f08c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.284+0000 7f08ca000640 1 -- 172.21.15.86:0/2149638344 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f08b809c930 con 0x7f08cc1095d0 2024-04-17T07:15:04.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:04 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1248266038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:15:04.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:04 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1618135384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:15:04.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.489+0000 7f08d2a00640 1 -- 172.21.15.86:0/2149638344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08cc02d820 con 0x7f08cc1095d0 2024-04-17T07:15:04.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.492+0000 7f08ca000640 1 -- 172.21.15.86:0/2149638344 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7f08b809c230 con 0x7f08cc1095d0 2024-04-17T07:15:04.491 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:04.491 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:15:04.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.494+0000 7f08d2a00640 1 -- 172.21.15.86:0/2149638344 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f08b40a3c40 msgr2=0x7f08b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.494+0000 7f08d2a00640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f08b40a3c40 0x7f08b40a6100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f08c0036040 tx=0x7f08c0000bc0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.494+0000 7f08d2a00640 1 -- 172.21.15.86:0/2149638344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc1095d0 msgr2=0x7f08cc19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.495+0000 7f08d2a00640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc1095d0 0x7f08cc19eb80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f08b800d770 tx=0x7f08b800dc30 comp rx=0 tx=0).stop 2024-04-17T07:15:04.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.495+0000 7f08d2a00640 1 -- 172.21.15.86:0/2149638344 shutdown_connections 2024-04-17T07:15:04.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.495+0000 7f08d2a00640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f08b40a3c40 0x7f08b40a6100 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.495+0000 7f08d2a00640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08cc109f90 0x7f08cc1a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.495+0000 7f08d2a00640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc1095d0 0x7f08cc19eb80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.495+0000 7f08d2a00640 1 --2- 172.21.15.86:0/2149638344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08cc1083e0 0x7f08cc19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.495+0000 7f08d2a00640 1 -- 172.21.15.86:0/2149638344 >> 172.21.15.86:0/2149638344 conn(0x7f08cc0773a0 msgr2=0x7f08cc10c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.495+0000 7f08d2a00640 1 -- 172.21.15.86:0/2149638344 shutdown_connections 2024-04-17T07:15:04.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.495+0000 7f08d2a00640 1 -- 172.21.15.86:0/2149638344 wait complete. 2024-04-17T07:15:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:04 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1248266038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:15:04.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:04 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1618135384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:15:04.513 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:15:04.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:15:04.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:04 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1248266038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:15:04.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:04 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1618135384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:15:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.731+0000 7fd05b200640 1 -- 172.21.15.86:0/4170391381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd054105350 msgr2=0x7fd054107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.731+0000 7fd05b200640 1 --2- 172.21.15.86:0/4170391381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd054105350 0x7fd054107740 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009900 tx=0x7fd03c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:04.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.731+0000 7fd05b200640 1 -- 172.21.15.86:0/4170391381 shutdown_connections 2024-04-17T07:15:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.731+0000 7fd05b200640 1 --2- 172.21.15.86:0/4170391381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd054107c80 0x7fd05410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.731+0000 7fd05b200640 1 --2- 172.21.15.86:0/4170391381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd054105350 0x7fd054107740 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.731+0000 7fd05b200640 1 --2- 172.21.15.86:0/4170391381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd054102a20 0x7fd054104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.731+0000 7fd05b200640 1 -- 172.21.15.86:0/4170391381 >> 172.21.15.86:0/4170391381 conn(0x7fd0540fc410 msgr2=0x7fd0540fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.731+0000 7fd05b200640 1 -- 172.21.15.86:0/4170391381 shutdown_connections 2024-04-17T07:15:04.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.731+0000 7fd05b200640 1 -- 172.21.15.86:0/4170391381 wait complete. 2024-04-17T07:15:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.732+0000 7fd05b200640 1 Processor -- start 2024-04-17T07:15:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.732+0000 7fd05b200640 1 -- start start 2024-04-17T07:15:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.732+0000 7fd05b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd054102a20 0x7fd05419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.732+0000 7fd05b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd054105350 0x7fd05419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.732+0000 7fd05b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd054107c80 0x7fd0541a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.732+0000 7fd05b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410db30 con 0x7fd054102a20 2024-04-17T07:15:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.732+0000 7fd05b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410d9b0 con 0x7fd054107c80 2024-04-17T07:15:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.732+0000 7fd05b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410dcb0 con 0x7fd054105350 2024-04-17T07:15:04.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.733+0000 7fd053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd054102a20 0x7fd05419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.733+0000 7fd053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd054102a20 0x7fd05419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53178/0 (socket says 172.21.15.86:53178) 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.733+0000 7fd053e00640 1 -- 172.21.15.86:0/2212357889 learned_addr learned my addr 172.21.15.86:0/2212357889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.733+0000 7fd04be00640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd054105350 0x7fd05419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.733+0000 7fd053e00640 1 -- 172.21.15.86:0/2212357889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd054105350 msgr2=0x7fd05419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.733+0000 7fd053e00640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd054105350 0x7fd05419a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.733+0000 7fd053e00640 1 -- 172.21.15.86:0/2212357889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd054107c80 msgr2=0x7fd0541a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.733+0000 7fd053e00640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd054107c80 0x7fd0541a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.733+0000 7fd053e00640 1 -- 172.21.15.86:0/2212357889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0541a2000 con 0x7fd054102a20 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.733+0000 7fd053e00640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd054102a20 0x7fd05419a3e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c890 tx=0x7fd04400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.734+0000 7fd051600640 1 -- 172.21.15.86:0/2212357889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044004220 con 0x7fd054102a20 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.734+0000 7fd051600640 1 -- 172.21.15.86:0/2212357889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0440043c0 con 0x7fd054102a20 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.734+0000 7fd051600640 1 -- 172.21.15.86:0/2212357889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044004a00 con 0x7fd054102a20 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.734+0000 7fd05b200640 1 -- 172.21.15.86:0/2212357889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0541a22f0 con 0x7fd054102a20 2024-04-17T07:15:04.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.734+0000 7fd05b200640 1 -- 172.21.15.86:0/2212357889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0541a27f0 con 0x7fd054102a20 2024-04-17T07:15:04.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.735+0000 7fd05b200640 1 -- 172.21.15.86:0/2212357889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0541097d0 con 0x7fd054102a20 2024-04-17T07:15:04.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.737+0000 7fd051600640 1 -- 172.21.15.86:0/2212357889 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fd044004ba0 con 0x7fd054102a20 2024-04-17T07:15:04.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.738+0000 7fd051600640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0380a3cc0 0x7fd0380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:04.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.738+0000 7fd051600640 1 -- 172.21.15.86:0/2212357889 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd0440d0630 con 0x7fd054102a20 2024-04-17T07:15:04.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.739+0000 7fd04be00640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0380a3cc0 0x7fd0380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:04.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.740+0000 7fd04be00640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0380a3cc0 0x7fd0380a6180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd03c036040 tx=0x7fd03c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:04.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.743+0000 7fd051600640 1 -- 172.21.15.86:0/2212357889 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd044014030 con 0x7fd054102a20 2024-04-17T07:15:04.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.944+0000 7fd05b200640 1 -- 172.21.15.86:0/2212357889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd054062bc0 con 0x7fd054102a20 2024-04-17T07:15:04.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.945+0000 7fd051600640 1 -- 172.21.15.86:0/2212357889 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fd04409cad0 con 0x7fd054102a20 2024-04-17T07:15:04.944 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:15:04.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 -- 172.21.15.86:0/2212357889 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0380a3cc0 msgr2=0x7fd0380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0380a3cc0 0x7fd0380a6180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd03c036040 tx=0x7fd03c005fd0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 -- 172.21.15.86:0/2212357889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd054102a20 msgr2=0x7fd05419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:04.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd054102a20 0x7fd05419a3e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c890 tx=0x7fd04400cd50 comp rx=0 tx=0).stop 2024-04-17T07:15:04.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 -- 172.21.15.86:0/2212357889 shutdown_connections 2024-04-17T07:15:04.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0380a3cc0 0x7fd0380a6180 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd054107c80 0x7fd0541a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd054105350 0x7fd05419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 --2- 172.21.15.86:0/2212357889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd054102a20 0x7fd05419a3e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:04.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 -- 172.21.15.86:0/2212357889 >> 172.21.15.86:0/2212357889 conn(0x7fd0540fc410 msgr2=0x7fd054102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:04.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 -- 172.21.15.86:0/2212357889 shutdown_connections 2024-04-17T07:15:04.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:04.948+0000 7fd05b200640 1 -- 172.21.15.86:0/2212357889 wait complete. 2024-04-17T07:15:04.965 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:15:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.165+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1686395005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a68103980 msgr2=0x7f2a6810a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.165+0000 7f2a6ce00640 1 --2- 172.21.15.86:0/1686395005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a68103980 0x7f2a6810a2a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009ca0 tx=0x7f2a6002c100 comp rx=0 tx=0).stop 2024-04-17T07:15:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.165+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1686395005 shutdown_connections 2024-04-17T07:15:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.165+0000 7f2a6ce00640 1 --2- 172.21.15.86:0/1686395005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a68103980 0x7f2a6810a2a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.165+0000 7f2a6ce00640 1 --2- 172.21.15.86:0/1686395005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a68102fc0 0x7f2a68103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.165+0000 7f2a6ce00640 1 --2- 172.21.15.86:0/1686395005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a68101e70 0x7f2a68102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.165+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1686395005 >> 172.21.15.86:0/1686395005 conn(0x7f2a680fd400 msgr2=0x7f2a680ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.165+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1686395005 shutdown_connections 2024-04-17T07:15:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.165+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1686395005 wait complete. 2024-04-17T07:15:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.166+0000 7f2a6ce00640 1 Processor -- start 2024-04-17T07:15:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.166+0000 7f2a6ce00640 1 -- start start 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a6ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a68101e70 0x7f2a6819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a6ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a68102fc0 0x7f2a6819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a6ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a68103980 0x7f2a681a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a6ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6810da40 con 0x7f2a68101e70 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a6ce00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6810d8c0 con 0x7f2a68102fc0 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a6ce00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6810dbc0 con 0x7f2a68103980 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a67400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a68101e70 0x7f2a6819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a67400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a68101e70 0x7f2a6819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53206/0 (socket says 172.21.15.86:53206) 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a67400640 1 -- 172.21.15.86:0/1743504218 learned_addr learned my addr 172.21.15.86:0/1743504218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a5fe00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a68102fc0 0x7f2a6819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a67e00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a68103980 0x7f2a681a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a67400640 1 -- 172.21.15.86:0/1743504218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a68103980 msgr2=0x7f2a681a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a67400640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a68103980 0x7f2a681a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a67400640 1 -- 172.21.15.86:0/1743504218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a68102fc0 msgr2=0x7f2a6819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a67400640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a68102fc0 0x7f2a6819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.167+0000 7f2a67400640 1 -- 172.21.15.86:0/1743504218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a681a1f40 con 0x7f2a68101e70 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.168+0000 7f2a5fe00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a68102fc0 0x7f2a6819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.168+0000 7f2a67e00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a68103980 0x7f2a681a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.168+0000 7f2a67400640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a68101e70 0x7f2a6819a320 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800c890 tx=0x7f2a5800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.168+0000 7f2a64c00640 1 -- 172.21.15.86:0/1743504218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58004220 con 0x7f2a68101e70 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.168+0000 7f2a64c00640 1 -- 172.21.15.86:0/1743504218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a580043c0 con 0x7f2a68101e70 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.168+0000 7f2a64c00640 1 -- 172.21.15.86:0/1743504218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58004a00 con 0x7f2a68101e70 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.168+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1743504218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a681a2230 con 0x7f2a68101e70 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.168+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1743504218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a681a27b0 con 0x7f2a68101e70 2024-04-17T07:15:05.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.169+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1743504218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a681a2a70 con 0x7f2a68101e70 2024-04-17T07:15:05.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.171+0000 7f2a64c00640 1 -- 172.21.15.86:0/1743504218 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f2a58004ba0 con 0x7f2a68101e70 2024-04-17T07:15:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.173+0000 7f2a64c00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2a4c0a3c40 0x7f2a4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.173+0000 7f2a64c00640 1 -- 172.21.15.86:0/1743504218 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2a580d06b0 con 0x7f2a68101e70 2024-04-17T07:15:05.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.173+0000 7f2a5fe00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2a4c0a3c40 0x7f2a4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:05.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.174+0000 7f2a5fe00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2a4c0a3c40 0x7f2a4c0a6100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2a50006fd0 tx=0x7f2a50008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:05.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.177+0000 7f2a64c00640 1 -- 172.21.15.86:0/1743504218 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2a58014030 con 0x7f2a68101e70 2024-04-17T07:15:05.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.370+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1743504218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2a681a2c40 con 0x7f2a68101e70 2024-04-17T07:15:05.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.371+0000 7f2a64c00640 1 -- 172.21.15.86:0/1743504218 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f2a5809cb50 con 0x7f2a68101e70 2024-04-17T07:15:05.370 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:15:05.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1743504218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2a4c0a3c40 msgr2=0x7f2a4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:05.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2a4c0a3c40 0x7f2a4c0a6100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2a50006fd0 tx=0x7f2a50008040 comp rx=0 tx=0).stop 2024-04-17T07:15:05.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1743504218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a68101e70 msgr2=0x7f2a6819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:05.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a68101e70 0x7f2a6819a320 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800c890 tx=0x7f2a5800cd50 comp rx=0 tx=0).stop 2024-04-17T07:15:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1743504218 shutdown_connections 2024-04-17T07:15:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2a4c0a3c40 0x7f2a4c0a6100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a68103980 0x7f2a681a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a68102fc0 0x7f2a6819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 --2- 172.21.15.86:0/1743504218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a68101e70 0x7f2a6819a320 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1743504218 >> 172.21.15.86:0/1743504218 conn(0x7f2a680fd400 msgr2=0x7f2a68074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1743504218 shutdown_connections 2024-04-17T07:15:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.374+0000 7f2a6ce00640 1 -- 172.21.15.86:0/1743504218 wait complete. 2024-04-17T07:15:05.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:15:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:05 smithi098 ceph-mon[35418]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-17T07:15:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:05 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2149638344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:05 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2212357889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:15:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:05 smithi098 ceph-mon[35418]: pgmap v462: 129 pgs: 129 active+clean; 651 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-17T07:15:05.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:05 smithi103 ceph-mon[35495]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-17T07:15:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:05 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2149638344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:05 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2212357889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:15:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:05 smithi103 ceph-mon[35495]: pgmap v462: 129 pgs: 129 active+clean; 651 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-17T07:15:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:05 smithi086 ceph-mon[31733]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-17T07:15:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:05 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2149638344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:05 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2212357889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:15:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:05 smithi086 ceph-mon[31733]: pgmap v462: 129 pgs: 129 active+clean; 651 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-17T07:15:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.598+0000 7fd77dc00640 1 -- 172.21.15.86:0/2498327763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778105390 msgr2=0x7fd778105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.598+0000 7fd77dc00640 1 --2- 172.21.15.86:0/2498327763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778105390 0x7fd778105810 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd760009900 tx=0x7fd76002bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.599+0000 7fd77dc00640 1 -- 172.21.15.86:0/2498327763 shutdown_connections 2024-04-17T07:15:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.599+0000 7fd77dc00640 1 --2- 172.21.15.86:0/2498327763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd778105d50 0x7fd77810c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.599+0000 7fd77dc00640 1 --2- 172.21.15.86:0/2498327763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd778105390 0x7fd778105810 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.599+0000 7fd77dc00640 1 --2- 172.21.15.86:0/2498327763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7781041a0 0x7fd7781045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.599+0000 7fd77dc00640 1 -- 172.21.15.86:0/2498327763 >> 172.21.15.86:0/2498327763 conn(0x7fd7780ff750 msgr2=0x7fd778101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.599+0000 7fd77dc00640 1 -- 172.21.15.86:0/2498327763 shutdown_connections 2024-04-17T07:15:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.599+0000 7fd77dc00640 1 -- 172.21.15.86:0/2498327763 wait complete. 2024-04-17T07:15:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd77dc00640 1 Processor -- start 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd77dc00640 1 -- start start 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd77dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7781041a0 0x7fd77819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd77dc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd778105390 0x7fd77819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd77dc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd778105d50 0x7fd7781a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd77dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd77810fd30 con 0x7fd7781041a0 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd77dc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd77810fbb0 con 0x7fd778105d50 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd77dc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd77810feb0 con 0x7fd778105390 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd776a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7781041a0 0x7fd77819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd776a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7781041a0 0x7fd77819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53220/0 (socket says 172.21.15.86:53220) 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.600+0000 7fd776a00640 1 -- 172.21.15.86:0/4244140188 learned_addr learned my addr 172.21.15.86:0/4244140188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd76e000640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd778105390 0x7fd77819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd777400640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd778105d50 0x7fd7781a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd776a00640 1 -- 172.21.15.86:0/4244140188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd778105390 msgr2=0x7fd77819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd776a00640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd778105390 0x7fd77819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd776a00640 1 -- 172.21.15.86:0/4244140188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd778105d50 msgr2=0x7fd7781a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd776a00640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd778105d50 0x7fd7781a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd776a00640 1 -- 172.21.15.86:0/4244140188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7781a1f40 con 0x7fd7781041a0 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd776a00640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7781041a0 0x7fd77819a320 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd76800d950 tx=0x7fd76800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd76fe00640 1 -- 172.21.15.86:0/4244140188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768017070 con 0x7fd7781041a0 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd76fe00640 1 -- 172.21.15.86:0/4244140188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd768009c30 con 0x7fd7781041a0 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd76fe00640 1 -- 172.21.15.86:0/4244140188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76800f620 con 0x7fd7781041a0 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.601+0000 7fd77dc00640 1 -- 172.21.15.86:0/4244140188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7781a2230 con 0x7fd7781041a0 2024-04-17T07:15:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.602+0000 7fd77dc00640 1 -- 172.21.15.86:0/4244140188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7781a2700 con 0x7fd7781041a0 2024-04-17T07:15:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.603+0000 7fd77dc00640 1 -- 172.21.15.86:0/4244140188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd77810b4c0 con 0x7fd7781041a0 2024-04-17T07:15:05.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.605+0000 7fd76fe00640 1 -- 172.21.15.86:0/4244140188 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fd76801a070 con 0x7fd7781041a0 2024-04-17T07:15:05.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.607+0000 7fd76fe00640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd75c0a3c40 0x7fd75c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:05.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.608+0000 7fd76fe00640 1 -- 172.21.15.86:0/4244140188 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd7680d0490 con 0x7fd7781041a0 2024-04-17T07:15:05.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.608+0000 7fd76e000640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd75c0a3c40 0x7fd75c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:05.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.608+0000 7fd76e000640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd75c0a3c40 0x7fd75c0a6100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd760006010 tx=0x7fd760000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.610+0000 7fd76fe00640 1 -- 172.21.15.86:0/4244140188 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd768009de0 con 0x7fd7781041a0 2024-04-17T07:15:05.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.815+0000 7fd77dc00640 1 -- 172.21.15.86:0/4244140188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd778062b90 con 0x7fd7781041a0 2024-04-17T07:15:05.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.816+0000 7fd76fe00640 1 -- 172.21.15.86:0/4244140188 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fd76809c930 con 0x7fd7781041a0 2024-04-17T07:15:05.815 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:15:05.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 -- 172.21.15.86:0/4244140188 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd75c0a3c40 msgr2=0x7fd75c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:05.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd75c0a3c40 0x7fd75c0a6100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd760006010 tx=0x7fd760000bc0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 -- 172.21.15.86:0/4244140188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7781041a0 msgr2=0x7fd77819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:05.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7781041a0 0x7fd77819a320 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd76800d950 tx=0x7fd76800de10 comp rx=0 tx=0).stop 2024-04-17T07:15:05.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 -- 172.21.15.86:0/4244140188 shutdown_connections 2024-04-17T07:15:05.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd75c0a3c40 0x7fd75c0a6100 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd778105d50 0x7fd7781a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd778105390 0x7fd77819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 --2- 172.21.15.86:0/4244140188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7781041a0 0x7fd77819a320 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:05.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 -- 172.21.15.86:0/4244140188 >> 172.21.15.86:0/4244140188 conn(0x7fd7780ff750 msgr2=0x7fd778108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:05.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 -- 172.21.15.86:0/4244140188 shutdown_connections 2024-04-17T07:15:05.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:05.818+0000 7fd77dc00640 1 -- 172.21.15.86:0/4244140188 wait complete. 2024-04-17T07:15:05.835 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:15:06.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.052+0000 7f4eea200640 1 -- 172.21.15.86:0/3145355440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee41084a0 msgr2=0x7f4ee41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.052+0000 7f4eea200640 1 --2- 172.21.15.86:0/3145355440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee41084a0 0x7f4ee41088c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009900 tx=0x7f4ed802bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:06.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.053+0000 7f4eea200640 1 -- 172.21.15.86:0/3145355440 shutdown_connections 2024-04-17T07:15:06.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.053+0000 7f4eea200640 1 --2- 172.21.15.86:0/3145355440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee410a050 0x7f4ee41108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.053+0000 7f4eea200640 1 --2- 172.21.15.86:0/3145355440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee4109690 0x7f4ee4109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.053+0000 7f4eea200640 1 --2- 172.21.15.86:0/3145355440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee41084a0 0x7f4ee41088c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.053+0000 7f4eea200640 1 -- 172.21.15.86:0/3145355440 >> 172.21.15.86:0/3145355440 conn(0x7f4ee4077430 msgr2=0x7f4ee4079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:06.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.053+0000 7f4eea200640 1 -- 172.21.15.86:0/3145355440 shutdown_connections 2024-04-17T07:15:06.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.053+0000 7f4eea200640 1 -- 172.21.15.86:0/3145355440 wait complete. 2024-04-17T07:15:06.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4eea200640 1 Processor -- start 2024-04-17T07:15:06.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4eea200640 1 -- start start 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4eea200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee41084a0 0x7f4ee419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4eea200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4109690 0x7f4ee419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4eea200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee410a050 0x7f4ee41a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4eea200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4113fe0 con 0x7f4ee4109690 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4eea200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4113e60 con 0x7f4ee410a050 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4eea200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4114160 con 0x7f4ee41084a0 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4109690 0x7f4ee419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4109690 0x7f4ee419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53238/0 (socket says 172.21.15.86:53238) 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.054+0000 7f4ee3e00640 1 -- 172.21.15.86:0/317591058 learned_addr learned my addr 172.21.15.86:0/317591058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee8e00640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee41084a0 0x7f4ee419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee9800640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee410a050 0x7f4ee41a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee3e00640 1 -- 172.21.15.86:0/317591058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee41084a0 msgr2=0x7f4ee419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee3e00640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee41084a0 0x7f4ee419e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee3e00640 1 -- 172.21.15.86:0/317591058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee410a050 msgr2=0x7f4ee41a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee3e00640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee410a050 0x7f4ee41a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee3e00640 1 -- 172.21.15.86:0/317591058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee41a6290 con 0x7f4ee4109690 2024-04-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee3e00640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4109690 0x7f4ee419ebb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400d770 tx=0x7f4ed400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee1600640 1 -- 172.21.15.86:0/317591058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4017070 con 0x7f4ee4109690 2024-04-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee1600640 1 -- 172.21.15.86:0/317591058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed40054d0 con 0x7f4ee4109690 2024-04-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4eea200640 1 -- 172.21.15.86:0/317591058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee41a6580 con 0x7f4ee4109690 2024-04-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.055+0000 7f4ee1600640 1 -- 172.21.15.86:0/317591058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4004cf0 con 0x7f4ee4109690 2024-04-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.056+0000 7f4eea200640 1 -- 172.21.15.86:0/317591058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41a6ad0 con 0x7f4ee4109690 2024-04-17T07:15:06.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.057+0000 7f4eea200640 1 -- 172.21.15.86:0/317591058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee410f7c0 con 0x7f4ee4109690 2024-04-17T07:15:06.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.059+0000 7f4ee1600640 1 -- 172.21.15.86:0/317591058 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f4ed4004e90 con 0x7f4ee4109690 2024-04-17T07:15:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.060+0000 7f4ee1600640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4ecc0a3cc0 0x7f4ecc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.060+0000 7f4ee1600640 1 -- 172.21.15.86:0/317591058 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4ed40d03b0 con 0x7f4ee4109690 2024-04-17T07:15:06.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.060+0000 7f4ee8e00640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4ecc0a3cc0 0x7f4ecc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.061+0000 7f4ee8e00640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4ecc0a3cc0 0x7f4ecc0a6180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8006010 tx=0x7f4ed8005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:06.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.065+0000 7f4ee1600640 1 -- 172.21.15.86:0/317591058 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4ed409c8b0 con 0x7f4ee4109690 2024-04-17T07:15:06.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.268+0000 7f4eea200640 1 -- 172.21.15.86:0/317591058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4ee402d820 con 0x7f4ee4109690 2024-04-17T07:15:06.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.271+0000 7f4ee1600640 1 -- 172.21.15.86:0/317591058 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f4ed409c1b0 con 0x7f4ee4109690 2024-04-17T07:15:06.270 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:15:06.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 -- 172.21.15.86:0/317591058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4ecc0a3cc0 msgr2=0x7f4ecc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4ecc0a3cc0 0x7f4ecc0a6180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8006010 tx=0x7f4ed8005be0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 -- 172.21.15.86:0/317591058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4109690 msgr2=0x7f4ee419ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4109690 0x7f4ee419ebb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400d770 tx=0x7f4ed400dc30 comp rx=0 tx=0).stop 2024-04-17T07:15:06.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 -- 172.21.15.86:0/317591058 shutdown_connections 2024-04-17T07:15:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4ecc0a3cc0 0x7f4ecc0a6180 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee410a050 0x7f4ee41a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4109690 0x7f4ee419ebb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 --2- 172.21.15.86:0/317591058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ee41084a0 0x7f4ee419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 -- 172.21.15.86:0/317591058 >> 172.21.15.86:0/317591058 conn(0x7f4ee4077430 msgr2=0x7f4ee410c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 -- 172.21.15.86:0/317591058 shutdown_connections 2024-04-17T07:15:06.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.274+0000 7f4eea200640 1 -- 172.21.15.86:0/317591058 wait complete. 2024-04-17T07:15:06.294 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-17T07:15:06.294 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:15:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.502+0000 7febcfa00640 1 -- 172.21.15.86:0/884593866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc81053d0 msgr2=0x7febc8105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.502+0000 7febcfa00640 1 --2- 172.21.15.86:0/884593866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc81053d0 0x7febc8105850 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7febb8009900 tx=0x7febb802bc50 comp rx=0 tx=0).stop 2024-04-17T07:15:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.503+0000 7febcfa00640 1 -- 172.21.15.86:0/884593866 shutdown_connections 2024-04-17T07:15:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.503+0000 7febcfa00640 1 --2- 172.21.15.86:0/884593866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc8105d90 0x7febc810c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.503+0000 7febcfa00640 1 --2- 172.21.15.86:0/884593866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc81053d0 0x7febc8105850 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.503+0000 7febcfa00640 1 --2- 172.21.15.86:0/884593866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc81041e0 0x7febc8104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.503+0000 7febcfa00640 1 -- 172.21.15.86:0/884593866 >> 172.21.15.86:0/884593866 conn(0x7febc80ff750 msgr2=0x7febc8101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.503+0000 7febcfa00640 1 -- 172.21.15.86:0/884593866 shutdown_connections 2024-04-17T07:15:06.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.503+0000 7febcfa00640 1 -- 172.21.15.86:0/884593866 wait complete. 2024-04-17T07:15:06.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.504+0000 7febcfa00640 1 Processor -- start 2024-04-17T07:15:06.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.504+0000 7febcfa00640 1 -- start start 2024-04-17T07:15:06.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.504+0000 7febcfa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc81041e0 0x7febc819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.504+0000 7febcfa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc81053d0 0x7febc819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.504+0000 7febcfa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc8105d90 0x7febc81a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.504+0000 7febcfa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febc810fd20 con 0x7febc81041e0 2024-04-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.504+0000 7febcfa00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7febc810fba0 con 0x7febc8105d90 2024-04-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.504+0000 7febcfa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7febc810fea0 con 0x7febc81053d0 2024-04-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.504+0000 7febcd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc81041e0 0x7febc819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febcd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc81041e0 0x7febc819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53260/0 (socket says 172.21.15.86:53260) 2024-04-17T07:15:06.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febcd000640 1 -- 172.21.15.86:0/2639390654 learned_addr learned my addr 172.21.15.86:0/2639390654 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febcda00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc8105d90 0x7febc81a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febc7e00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc81053d0 0x7febc819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febcd000640 1 -- 172.21.15.86:0/2639390654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc81053d0 msgr2=0x7febc819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febcd000640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc81053d0 0x7febc819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febcd000640 1 -- 172.21.15.86:0/2639390654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc8105d90 msgr2=0x7febc81a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febcd000640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc8105d90 0x7febc81a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febcd000640 1 -- 172.21.15.86:0/2639390654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc81a1fe0 con 0x7febc81041e0 2024-04-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febc7e00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc81053d0 0x7febc819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.505+0000 7febcd000640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc81041e0 0x7febc819a3c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c970 tx=0x7febbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.506+0000 7febc5600640 1 -- 172.21.15.86:0/2639390654 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc004220 con 0x7febc81041e0 2024-04-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.506+0000 7febc5600640 1 -- 172.21.15.86:0/2639390654 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febbc0043c0 con 0x7febc81041e0 2024-04-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.506+0000 7febcfa00640 1 -- 172.21.15.86:0/2639390654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc81a22d0 con 0x7febc81041e0 2024-04-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.506+0000 7febc5600640 1 -- 172.21.15.86:0/2639390654 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc00f450 con 0x7febc81041e0 2024-04-17T07:15:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.506+0000 7febcfa00640 1 -- 172.21.15.86:0/2639390654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc81a2870 con 0x7febc81041e0 2024-04-17T07:15:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.507+0000 7febcfa00640 1 -- 172.21.15.86:0/2639390654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc810b580 con 0x7febc81041e0 2024-04-17T07:15:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.509+0000 7febc5600640 1 -- 172.21.15.86:0/2639390654 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7febbc0079b0 con 0x7febc81041e0 2024-04-17T07:15:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.510+0000 7febc5600640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febb00a3cc0 0x7febb00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.510+0000 7febc5600640 1 -- 172.21.15.86:0/2639390654 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7febbc0d0730 con 0x7febc81041e0 2024-04-17T07:15:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.510+0000 7febc7e00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febb00a3cc0 0x7febb00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.511+0000 7febc7e00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febb00a3cc0 0x7febb00a6180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7febb80096c0 tx=0x7febb8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:06.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.514+0000 7febc5600640 1 -- 172.21.15.86:0/2639390654 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7febbc014030 con 0x7febc81041e0 2024-04-17T07:15:06.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:06 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1743504218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:15:06.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:06 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4244140188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:15:06.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:06 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1743504218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:15:06.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:06 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4244140188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:15:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:06 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1743504218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:15:06.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:06 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4244140188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:15:06.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.758+0000 7febcfa00640 1 -- 172.21.15.86:0/2639390654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febc802d5d0 con 0x7febc81041e0 2024-04-17T07:15:06.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.760+0000 7febc5600640 1 -- 172.21.15.86:0/2639390654 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7febbc09cbd0 con 0x7febc81041e0 2024-04-17T07:15:06.759 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:06.759 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:15:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:15:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.762+0000 7febcfa00640 1 -- 172.21.15.86:0/2639390654 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febb00a3cc0 msgr2=0x7febb00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.762+0000 7febcfa00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febb00a3cc0 0x7febb00a6180 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7febb80096c0 tx=0x7febb8000bc0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.762+0000 7febcfa00640 1 -- 172.21.15.86:0/2639390654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc81041e0 msgr2=0x7febc819a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.762+0000 7febcfa00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc81041e0 0x7febc819a3c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c970 tx=0x7febbc00ce30 comp rx=0 tx=0).stop 2024-04-17T07:15:06.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.762+0000 7febcfa00640 1 -- 172.21.15.86:0/2639390654 shutdown_connections 2024-04-17T07:15:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.762+0000 7febcfa00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7febb00a3cc0 0x7febb00a6180 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.762+0000 7febcfa00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febc8105d90 0x7febc81a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.762+0000 7febcfa00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7febc81053d0 0x7febc819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.762+0000 7febcfa00640 1 --2- 172.21.15.86:0/2639390654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc81041e0 0x7febc819a3c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.762+0000 7febcfa00640 1 -- 172.21.15.86:0/2639390654 >> 172.21.15.86:0/2639390654 conn(0x7febc80ff750 msgr2=0x7febc8108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.763+0000 7febcfa00640 1 -- 172.21.15.86:0/2639390654 shutdown_connections 2024-04-17T07:15:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.763+0000 7febcfa00640 1 -- 172.21.15.86:0/2639390654 wait complete. 2024-04-17T07:15:06.781 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-17T07:15:06.781 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-17T07:15:06.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.989+0000 7fb894e00640 1 -- 172.21.15.86:0/1140712429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb890109f90 msgr2=0x7fb890110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.989+0000 7fb894e00640 1 --2- 172.21.15.86:0/1140712429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb890109f90 0x7fb890110820 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb888009ca0 tx=0x7fb88802c100 comp rx=0 tx=0).stop 2024-04-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.989+0000 7fb894e00640 1 -- 172.21.15.86:0/1140712429 shutdown_connections 2024-04-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.989+0000 7fb894e00640 1 --2- 172.21.15.86:0/1140712429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb890109f90 0x7fb890110820 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.989+0000 7fb894e00640 1 --2- 172.21.15.86:0/1140712429 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8901095d0 0x7fb890109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.989+0000 7fb894e00640 1 --2- 172.21.15.86:0/1140712429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8901083e0 0x7fb890108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.989+0000 7fb894e00640 1 -- 172.21.15.86:0/1140712429 >> 172.21.15.86:0/1140712429 conn(0x7fb8900773a0 msgr2=0x7fb8900797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.990+0000 7fb894e00640 1 -- 172.21.15.86:0/1140712429 shutdown_connections 2024-04-17T07:15:06.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.990+0000 7fb894e00640 1 -- 172.21.15.86:0/1140712429 wait complete. 2024-04-17T07:15:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.990+0000 7fb894e00640 1 Processor -- start 2024-04-17T07:15:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.991+0000 7fb894e00640 1 -- start start 2024-04-17T07:15:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.991+0000 7fb894e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8901083e0 0x7fb89019e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.991+0000 7fb894e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8901095d0 0x7fb89019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.991+0000 7fb894e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb890109f90 0x7fb8901a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.991+0000 7fb894e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb890113f20 con 0x7fb8901083e0 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.991+0000 7fb894e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb890113da0 con 0x7fb8901095d0 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.991+0000 7fb894e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901140a0 con 0x7fb890109f90 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.991+0000 7fb88f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8901083e0 0x7fb89019e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.991+0000 7fb88f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8901083e0 0x7fb89019e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37790/0 (socket says 172.21.15.86:37790) 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.991+0000 7fb88f400640 1 -- 172.21.15.86:0/1483521740 learned_addr learned my addr 172.21.15.86:0/1483521740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb88fe00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb890109f90 0x7fb8901a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb88ea00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8901095d0 0x7fb89019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb88f400640 1 -- 172.21.15.86:0/1483521740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb890109f90 msgr2=0x7fb8901a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb88f400640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb890109f90 0x7fb8901a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb88f400640 1 -- 172.21.15.86:0/1483521740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8901095d0 msgr2=0x7fb89019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb88f400640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8901095d0 0x7fb89019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb88f400640 1 -- 172.21.15.86:0/1483521740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8901a6120 con 0x7fb8901083e0 2024-04-17T07:15:06.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb88f400640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8901083e0 0x7fb89019e500 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c970 tx=0x7fb88000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb887e00640 1 -- 172.21.15.86:0/1483521740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880007c10 con 0x7fb8901083e0 2024-04-17T07:15:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb894e00640 1 -- 172.21.15.86:0/1483521740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8901a6410 con 0x7fb8901083e0 2024-04-17T07:15:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb887e00640 1 -- 172.21.15.86:0/1483521740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb880007db0 con 0x7fb8901083e0 2024-04-17T07:15:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.992+0000 7fb887e00640 1 -- 172.21.15.86:0/1483521740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88000f870 con 0x7fb8901083e0 2024-04-17T07:15:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.993+0000 7fb894e00640 1 -- 172.21.15.86:0/1483521740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8901a6960 con 0x7fb8901083e0 2024-04-17T07:15:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.994+0000 7fb894e00640 1 -- 172.21.15.86:0/1483521740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89010f700 con 0x7fb8901083e0 2024-04-17T07:15:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.996+0000 7fb887e00640 1 -- 172.21.15.86:0/1483521740 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fb88000fa10 con 0x7fb8901083e0 2024-04-17T07:15:06.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.998+0000 7fb887e00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8740a3c40 0x7fb8740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:06.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.998+0000 7fb887e00640 1 -- 172.21.15.86:0/1483521740 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb8800d0d40 con 0x7fb8901083e0 2024-04-17T07:15:06.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.998+0000 7fb88ea00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8740a3c40 0x7fb8740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:06.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:06.999+0000 7fb88ea00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8740a3c40 0x7fb8740a6100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb89019fa10 tx=0x7fb87c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:07.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.003+0000 7fb887e00640 1 -- 172.21.15.86:0/1483521740 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb88009d2d0 con 0x7fb8901083e0 2024-04-17T07:15:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.199+0000 7fb894e00640 1 -- 172.21.15.86:0/1483521740 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8900623a0 con 0x7fb8740a3c40 2024-04-17T07:15:07.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.227+0000 7fb887e00640 1 -- 172.21.15.86:0/1483521740 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340881 (secure 0 0 0) 0x7fb8900623a0 con 0x7fb8740a3c40 2024-04-17T07:15:07.232 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:07.234 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 -- 172.21.15.86:0/1483521740 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8740a3c40 msgr2=0x7fb8740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8740a3c40 0x7fb8740a6100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb89019fa10 tx=0x7fb87c00a4c0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 -- 172.21.15.86:0/1483521740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8901083e0 msgr2=0x7fb89019e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8901083e0 0x7fb89019e500 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c970 tx=0x7fb88000ce30 comp rx=0 tx=0).stop 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 -- 172.21.15.86:0/1483521740 shutdown_connections 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8740a3c40 0x7fb8740a6100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb890109f90 0x7fb8901a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8901095d0 0x7fb89019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 --2- 172.21.15.86:0/1483521740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8901083e0 0x7fb89019e500 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 -- 172.21.15.86:0/1483521740 >> 172.21.15.86:0/1483521740 conn(0x7fb8900773a0 msgr2=0x7fb89010c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:07.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.233+0000 7fb894e00640 1 -- 172.21.15.86:0/1483521740 shutdown_connections 2024-04-17T07:15:07.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.234+0000 7fb894e00640 1 -- 172.21.15.86:0/1483521740 wait complete. 2024-04-17T07:15:07.250 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-04-17T07:15:07.131692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667287,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1702,"num_read_kb":971,"num_write":457,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181358,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":873,"ondisk_log_size":873,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365908,"kb_used_data":12596,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575916,"statfs":{"total":1151940427776,"available":1151565737984,"internally_reserved":0,"allocated":12898304,"data_stored":8217047,"data_compressed":432012,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7481,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":61,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005284"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078723+0000","last_change":"2024-04-17T07:11:55.486746+0000","last_active":"2024-04-17T07:13:57.078723+0000","last_peered":"2024-04-17T07:13:57.078723+0000","last_clean":"2024-04-17T07:13:57.078723+0000","last_became_active":"2024-04-17T07:11:55.486335+0000","last_became_peered":"2024-04-17T07:11:55.486335+0000","last_unstale":"2024-04-17T07:13:57.078723+0000","last_undegraded":"2024-04-17T07:13:57.078723+0000","last_fullsized":"2024-04-17T07:13:57.078723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:48.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082087+0000","last_change":"2024-04-17T07:11:55.481832+0000","last_active":"2024-04-17T07:13:57.082087+0000","last_peered":"2024-04-17T07:13:57.082087+0000","last_clean":"2024-04-17T07:13:57.082087+0000","last_became_active":"2024-04-17T07:11:55.476428+0000","last_became_peered":"2024-04-17T07:11:55.476428+0000","last_unstale":"2024-04-17T07:13:57.082087+0000","last_undegraded":"2024-04-17T07:13:57.082087+0000","last_fullsized":"2024-04-17T07:13:57.082087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:04.452542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083395+0000","last_change":"2024-04-17T07:11:55.483721+0000","last_active":"2024-04-17T07:13:57.083395+0000","last_peered":"2024-04-17T07:13:57.083395+0000","last_clean":"2024-04-17T07:13:57.083395+0000","last_became_active":"2024-04-17T07:11:55.483268+0000","last_became_peered":"2024-04-17T07:11:55.483268+0000","last_unstale":"2024-04-17T07:13:57.083395+0000","last_undegraded":"2024-04-17T07:13:57.083395+0000","last_fullsized":"2024-04-17T07:13:57.083395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:41.584783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067540+0000","last_change":"2024-04-17T07:11:55.485114+0000","last_active":"2024-04-17T07:13:57.067540+0000","last_peered":"2024-04-17T07:13:57.067540+0000","last_clean":"2024-04-17T07:13:57.067540+0000","last_became_active":"2024-04-17T07:11:55.483913+0000","last_became_peered":"2024-04-17T07:11:55.483913+0000","last_unstale":"2024-04-17T07:13:57.067540+0000","last_undegraded":"2024-04-17T07:13:57.067540+0000","last_fullsized":"2024-04-17T07:13:57.067540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:46.211121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.060857+0000","last_change":"2024-04-17T07:11:55.491952+0000","last_active":"2024-04-17T07:13:57.060857+0000","last_peered":"2024-04-17T07:13:57.060857+0000","last_clean":"2024-04-17T07:13:57.060857+0000","last_became_active":"2024-04-17T07:11:55.491238+0000","last_became_peered":"2024-04-17T07:11:55.491238+0000","last_unstale":"2024-04-17T07:13:57.060857+0000","last_undegraded":"2024-04-17T07:13:57.060857+0000","last_fullsized":"2024-04-17T07:13:57.060857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:06.759797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068819+0000","last_change":"2024-04-17T07:11:55.471124+0000","last_active":"2024-04-17T07:13:57.068819+0000","last_peered":"2024-04-17T07:13:57.068819+0000","last_clean":"2024-04-17T07:13:57.068819+0000","last_became_active":"2024-04-17T07:11:55.470781+0000","last_became_peered":"2024-04-17T07:11:55.470781+0000","last_unstale":"2024-04-17T07:13:57.068819+0000","last_undegraded":"2024-04-17T07:13:57.068819+0000","last_fullsized":"2024-04-17T07:13:57.068819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:45.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.072320+0000","last_change":"2024-04-17T07:11:55.469099+0000","last_active":"2024-04-17T07:13:57.072320+0000","last_peered":"2024-04-17T07:13:57.072320+0000","last_clean":"2024-04-17T07:13:57.072320+0000","last_became_active":"2024-04-17T07:11:55.468584+0000","last_became_peered":"2024-04-17T07:11:55.468584+0000","last_unstale":"2024-04-17T07:13:57.072320+0000","last_undegraded":"2024-04-17T07:13:57.072320+0000","last_fullsized":"2024-04-17T07:13:57.072320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:52.834846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071967+0000","last_change":"2024-04-17T07:11:55.473794+0000","last_active":"2024-04-17T07:13:57.071967+0000","last_peered":"2024-04-17T07:13:57.071967+0000","last_clean":"2024-04-17T07:13:57.071967+0000","last_became_active":"2024-04-17T07:11:55.471919+0000","last_became_peered":"2024-04-17T07:11:55.471919+0000","last_unstale":"2024-04-17T07:13:57.071967+0000","last_undegraded":"2024-04-17T07:13:57.071967+0000","last_fullsized":"2024-04-17T07:13:57.071967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:16.085117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.079281+0000","last_change":"2024-04-17T07:11:55.482445+0000","last_active":"2024-04-17T07:13:57.079281+0000","last_peered":"2024-04-17T07:13:57.079281+0000","last_clean":"2024-04-17T07:13:57.079281+0000","last_became_active":"2024-04-17T07:11:55.481552+0000","last_became_peered":"2024-04-17T07:11:55.481552+0000","last_unstale":"2024-04-17T07:13:57.079281+0000","last_undegraded":"2024-04-17T07:13:57.079281+0000","last_fullsized":"2024-04-17T07:13:57.079281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:48.733585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071322+0000","last_change":"2024-04-17T07:11:55.476726+0000","last_active":"2024-04-17T07:13:57.071322+0000","last_peered":"2024-04-17T07:13:57.071322+0000","last_clean":"2024-04-17T07:13:57.071322+0000","last_became_active":"2024-04-17T07:11:55.476320+0000","last_became_peered":"2024-04-17T07:11:55.476320+0000","last_unstale":"2024-04-17T07:13:57.071322+0000","last_undegraded":"2024-04-17T07:13:57.071322+0000","last_fullsized":"2024-04-17T07:13:57.071322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.230181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084097+0000","last_change":"2024-04-17T07:11:55.485798+0000","last_active":"2024-04-17T07:13:57.084097+0000","last_peered":"2024-04-17T07:13:57.084097+0000","last_clean":"2024-04-17T07:13:57.084097+0000","last_became_active":"2024-04-17T07:11:55.484015+0000","last_became_peered":"2024-04-17T07:11:55.484015+0000","last_unstale":"2024-04-17T07:13:57.084097+0000","last_undegraded":"2024-04-17T07:13:57.084097+0000","last_fullsized":"2024-04-17T07:13:57.084097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:37.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.062297+0000","last_change":"2024-04-17T07:12:28.729834+0000","last_active":"2024-04-17T07:13:57.062297+0000","last_peered":"2024-04-17T07:13:57.062297+0000","last_clean":"2024-04-17T07:13:57.062297+0000","last_became_active":"2024-04-17T07:12:28.728694+0000","last_became_peered":"2024-04-17T07:12:28.728694+0000","last_unstale":"2024-04-17T07:13:57.062297+0000","last_undegraded":"2024-04-17T07:13:57.062297+0000","last_fullsized":"2024-04-17T07:13:57.062297+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:46.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067980+0000","last_change":"2024-04-17T07:13:29.210325+0000","last_active":"2024-04-17T07:13:57.067980+0000","last_peered":"2024-04-17T07:13:57.067980+0000","last_clean":"2024-04-17T07:13:57.067980+0000","last_became_active":"2024-04-17T07:13:29.209759+0000","last_became_peered":"2024-04-17T07:13:29.209759+0000","last_unstale":"2024-04-17T07:13:57.067980+0000","last_undegraded":"2024-04-17T07:13:57.067980+0000","last_fullsized":"2024-04-17T07:13:57.067980+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:17.168931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083596+0000","last_change":"2024-04-17T07:11:55.473839+0000","last_active":"2024-04-17T07:13:57.083596+0000","last_peered":"2024-04-17T07:13:57.083596+0000","last_clean":"2024-04-17T07:13:57.083596+0000","last_became_active":"2024-04-17T07:11:55.473472+0000","last_became_peered":"2024-04-17T07:11:55.473472+0000","last_unstale":"2024-04-17T07:13:57.083596+0000","last_undegraded":"2024-04-17T07:13:57.083596+0000","last_fullsized":"2024-04-17T07:13:57.083596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:02.263272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081481+0000","last_change":"2024-04-17T07:11:55.484820+0000","last_active":"2024-04-17T07:13:57.081481+0000","last_peered":"2024-04-17T07:13:57.081481+0000","last_clean":"2024-04-17T07:13:57.081481+0000","last_became_active":"2024-04-17T07:11:55.484314+0000","last_became_peered":"2024-04-17T07:11:55.484314+0000","last_unstale":"2024-04-17T07:13:57.081481+0000","last_undegraded":"2024-04-17T07:13:57.081481+0000","last_fullsized":"2024-04-17T07:13:57.081481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.186912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084394+0000","last_change":"2024-04-17T07:11:55.492018+0000","last_active":"2024-04-17T07:13:57.084394+0000","last_peered":"2024-04-17T07:13:57.084394+0000","last_clean":"2024-04-17T07:13:57.084394+0000","last_became_active":"2024-04-17T07:11:55.490512+0000","last_became_peered":"2024-04-17T07:11:55.490512+0000","last_unstale":"2024-04-17T07:13:57.084394+0000","last_undegraded":"2024-04-17T07:13:57.084394+0000","last_fullsized":"2024-04-17T07:13:57.084394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:45.436270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081460+0000","last_change":"2024-04-17T07:11:55.487210+0000","last_active":"2024-04-17T07:13:57.081460+0000","last_peered":"2024-04-17T07:13:57.081460+0000","last_clean":"2024-04-17T07:13:57.081460+0000","last_became_active":"2024-04-17T07:11:55.485713+0000","last_became_peered":"2024-04-17T07:11:55.485713+0000","last_unstale":"2024-04-17T07:13:57.081460+0000","last_undegraded":"2024-04-17T07:13:57.081460+0000","last_fullsized":"2024-04-17T07:13:57.081460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:54.495950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082817+0000","last_change":"2024-04-17T07:11:55.482705+0000","last_active":"2024-04-17T07:13:57.082817+0000","last_peered":"2024-04-17T07:13:57.082817+0000","last_clean":"2024-04-17T07:13:57.082817+0000","last_became_active":"2024-04-17T07:11:55.482349+0000","last_became_peered":"2024-04-17T07:11:55.482349+0000","last_unstale":"2024-04-17T07:13:57.082817+0000","last_undegraded":"2024-04-17T07:13:57.082817+0000","last_fullsized":"2024-04-17T07:13:57.082817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:57.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069422+0000","last_change":"2024-04-17T07:11:55.467065+0000","last_active":"2024-04-17T07:13:57.069422+0000","last_peered":"2024-04-17T07:13:57.069422+0000","last_clean":"2024-04-17T07:13:57.069422+0000","last_became_active":"2024-04-17T07:11:55.466535+0000","last_became_peered":"2024-04-17T07:11:55.466535+0000","last_unstale":"2024-04-17T07:13:57.069422+0000","last_undegraded":"2024-04-17T07:13:57.069422+0000","last_fullsized":"2024-04-17T07:13:57.069422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:49.955453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204394+0000","last_change":"2024-04-17T07:11:59.515778+0000","last_active":"2024-04-17T07:13:29.204394+0000","last_peered":"2024-04-17T07:13:29.204394+0000","last_clean":"2024-04-17T07:13:29.204394+0000","last_became_active":"2024-04-17T07:11:59.515266+0000","last_became_peered":"2024-04-17T07:11:59.515266+0000","last_unstale":"2024-04-17T07:13:29.204394+0000","last_undegraded":"2024-04-17T07:13:29.204394+0000","last_fullsized":"2024-04-17T07:13:29.204394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:43.084724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":203,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:05.648744+0000","last_change":"2024-04-17T07:11:54.467060+0000","last_active":"2024-04-17T07:15:05.648744+0000","last_peered":"2024-04-17T07:15:05.648744+0000","last_clean":"2024-04-17T07:15:05.648744+0000","last_became_active":"2024-04-17T07:11:54.466687+0000","last_became_peered":"2024-04-17T07:11:54.466687+0000","last_unstale":"2024-04-17T07:15:05.648744+0000","last_undegraded":"2024-04-17T07:15:05.648744+0000","last_fullsized":"2024-04-17T07:15:05.648744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:57.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":157,"num_read_kb":157,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.064498+0000","last_change":"2024-04-17T07:11:55.469857+0000","last_active":"2024-04-17T07:13:57.064498+0000","last_peered":"2024-04-17T07:13:57.064498+0000","last_clean":"2024-04-17T07:13:57.064498+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:13:57.064498+0000","last_undegraded":"2024-04-17T07:13:57.064498+0000","last_fullsized":"2024-04-17T07:13:57.064498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:43.565627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174451+0000","last_change":"2024-04-17T07:12:29.125397+0000","last_active":"2024-04-17T07:13:29.174451+0000","last_peered":"2024-04-17T07:13:29.174451+0000","last_clean":"2024-04-17T07:13:29.174451+0000","last_became_active":"2024-04-17T07:12:29.124158+0000","last_became_peered":"2024-04-17T07:12:29.124158+0000","last_unstale":"2024-04-17T07:13:29.174451+0000","last_undegraded":"2024-04-17T07:13:29.174451+0000","last_fullsized":"2024-04-17T07:13:29.174451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:24.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205266+0000","last_change":"2024-04-17T07:11:54.464715+0000","last_active":"2024-04-17T07:13:29.205266+0000","last_peered":"2024-04-17T07:13:29.205266+0000","last_clean":"2024-04-17T07:13:29.205266+0000","last_became_active":"2024-04-17T07:11:54.463917+0000","last_became_peered":"2024-04-17T07:11:54.463917+0000","last_unstale":"2024-04-17T07:13:29.205266+0000","last_undegraded":"2024-04-17T07:13:29.205266+0000","last_fullsized":"2024-04-17T07:13:29.205266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:43.738471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.085059+0000","last_change":"2024-04-17T07:11:55.482876+0000","last_active":"2024-04-17T07:13:57.085059+0000","last_peered":"2024-04-17T07:13:57.085059+0000","last_clean":"2024-04-17T07:13:57.085059+0000","last_became_active":"2024-04-17T07:11:55.482451+0000","last_became_peered":"2024-04-17T07:11:55.482451+0000","last_unstale":"2024-04-17T07:13:57.085059+0000","last_undegraded":"2024-04-17T07:13:57.085059+0000","last_fullsized":"2024-04-17T07:13:57.085059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:39.394090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174912+0000","last_change":"2024-04-17T07:11:59.522634+0000","last_active":"2024-04-17T07:13:29.174912+0000","last_peered":"2024-04-17T07:13:29.174912+0000","last_clean":"2024-04-17T07:13:29.174912+0000","last_became_active":"2024-04-17T07:11:59.520797+0000","last_became_peered":"2024-04-17T07:11:59.520797+0000","last_unstale":"2024-04-17T07:13:29.174912+0000","last_undegraded":"2024-04-17T07:13:29.174912+0000","last_fullsized":"2024-04-17T07:13:29.174912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:02.316515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.334212+0000","last_change":"2024-04-17T07:12:28.756528+0000","last_active":"2024-04-17T07:13:29.334212+0000","last_peered":"2024-04-17T07:13:29.334212+0000","last_clean":"2024-04-17T07:13:29.334212+0000","last_became_active":"2024-04-17T07:12:28.737008+0000","last_became_peered":"2024-04-17T07:12:28.737008+0000","last_unstale":"2024-04-17T07:13:29.334212+0000","last_undegraded":"2024-04-17T07:13:29.334212+0000","last_fullsized":"2024-04-17T07:13:29.334212+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:56:26.719377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083022+0000","last_change":"2024-04-17T07:11:55.485999+0000","last_active":"2024-04-17T07:13:57.083022+0000","last_peered":"2024-04-17T07:13:57.083022+0000","last_clean":"2024-04-17T07:13:57.083022+0000","last_became_active":"2024-04-17T07:11:55.484048+0000","last_became_peered":"2024-04-17T07:11:55.484048+0000","last_unstale":"2024-04-17T07:13:57.083022+0000","last_undegraded":"2024-04-17T07:13:57.083022+0000","last_fullsized":"2024-04-17T07:13:57.083022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:38.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174422+0000","last_change":"2024-04-17T07:11:59.529929+0000","last_active":"2024-04-17T07:13:29.174422+0000","last_peered":"2024-04-17T07:13:29.174422+0000","last_clean":"2024-04-17T07:13:29.174422+0000","last_became_active":"2024-04-17T07:11:59.529400+0000","last_became_peered":"2024-04-17T07:11:59.529400+0000","last_unstale":"2024-04-17T07:13:29.174422+0000","last_undegraded":"2024-04-17T07:13:29.174422+0000","last_fullsized":"2024-04-17T07:13:29.174422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:54.845529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204898+0000","last_change":"2024-04-17T07:11:54.462852+0000","last_active":"2024-04-17T07:13:29.204898+0000","last_peered":"2024-04-17T07:13:29.204898+0000","last_clean":"2024-04-17T07:13:29.204898+0000","last_became_active":"2024-04-17T07:11:54.459178+0000","last_became_peered":"2024-04-17T07:11:54.459178+0000","last_unstale":"2024-04-17T07:13:29.204898+0000","last_undegraded":"2024-04-17T07:13:29.204898+0000","last_fullsized":"2024-04-17T07:13:29.204898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:07.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.075256+0000","last_change":"2024-04-17T07:11:55.480951+0000","last_active":"2024-04-17T07:13:57.075256+0000","last_peered":"2024-04-17T07:13:57.075256+0000","last_clean":"2024-04-17T07:13:57.075256+0000","last_became_active":"2024-04-17T07:11:55.480507+0000","last_became_peered":"2024-04-17T07:11:55.480507+0000","last_unstale":"2024-04-17T07:13:57.075256+0000","last_undegraded":"2024-04-17T07:13:57.075256+0000","last_fullsized":"2024-04-17T07:13:57.075256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:42.584180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205487+0000","last_change":"2024-04-17T07:11:59.520897+0000","last_active":"2024-04-17T07:13:29.205487+0000","last_peered":"2024-04-17T07:13:29.205487+0000","last_clean":"2024-04-17T07:13:29.205487+0000","last_became_active":"2024-04-17T07:11:59.520350+0000","last_became_peered":"2024-04-17T07:11:59.520350+0000","last_unstale":"2024-04-17T07:13:29.205487+0000","last_undegraded":"2024-04-17T07:13:29.205487+0000","last_fullsized":"2024-04-17T07:13:29.205487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:39.813097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":212,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:05.699123+0000","last_change":"2024-04-17T07:12:28.753717+0000","last_active":"2024-04-17T07:15:05.699123+0000","last_peered":"2024-04-17T07:15:05.699123+0000","last_clean":"2024-04-17T07:15:05.699123+0000","last_became_active":"2024-04-17T07:12:28.739556+0000","last_became_peered":"2024-04-17T07:12:28.739556+0000","last_unstale":"2024-04-17T07:15:05.699123+0000","last_undegraded":"2024-04-17T07:15:05.699123+0000","last_fullsized":"2024-04-17T07:15:05.699123+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:11.967731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1631,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":160,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.055509+0000","last_change":"2024-04-17T07:11:55.476675+0000","last_active":"2024-04-17T07:13:57.055509+0000","last_peered":"2024-04-17T07:13:57.055509+0000","last_clean":"2024-04-17T07:13:57.055509+0000","last_became_active":"2024-04-17T07:11:55.476273+0000","last_became_peered":"2024-04-17T07:11:55.476273+0000","last_unstale":"2024-04-17T07:13:57.055509+0000","last_undegraded":"2024-04-17T07:13:57.055509+0000","last_fullsized":"2024-04-17T07:13:57.055509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:09.612138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205617+0000","last_change":"2024-04-17T07:12:28.741537+0000","last_active":"2024-04-17T07:13:29.205617+0000","last_peered":"2024-04-17T07:13:29.205617+0000","last_clean":"2024-04-17T07:13:29.205617+0000","last_became_active":"2024-04-17T07:12:28.741035+0000","last_became_peered":"2024-04-17T07:12:28.741035+0000","last_unstale":"2024-04-17T07:13:29.205617+0000","last_undegraded":"2024-04-17T07:13:29.205617+0000","last_fullsized":"2024-04-17T07:13:29.205617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:15.550112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.207986+0000","last_change":"2024-04-17T07:11:54.471118+0000","last_active":"2024-04-17T07:13:29.207986+0000","last_peered":"2024-04-17T07:13:29.207986+0000","last_clean":"2024-04-17T07:13:29.207986+0000","last_became_active":"2024-04-17T07:11:54.470526+0000","last_became_peered":"2024-04-17T07:11:54.470526+0000","last_unstale":"2024-04-17T07:13:29.207986+0000","last_undegraded":"2024-04-17T07:13:29.207986+0000","last_fullsized":"2024-04-17T07:13:29.207986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:52:47.918086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071367+0000","last_change":"2024-04-17T07:11:55.472850+0000","last_active":"2024-04-17T07:13:57.071367+0000","last_peered":"2024-04-17T07:13:57.071367+0000","last_clean":"2024-04-17T07:13:57.071367+0000","last_became_active":"2024-04-17T07:11:55.472097+0000","last_became_peered":"2024-04-17T07:11:55.472097+0000","last_unstale":"2024-04-17T07:13:57.071367+0000","last_undegraded":"2024-04-17T07:13:57.071367+0000","last_fullsized":"2024-04-17T07:13:57.071367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:41.730809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.334157+0000","last_change":"2024-04-17T07:11:59.522960+0000","last_active":"2024-04-17T07:13:29.334157+0000","last_peered":"2024-04-17T07:13:29.334157+0000","last_clean":"2024-04-17T07:13:29.334157+0000","last_became_active":"2024-04-17T07:11:59.522470+0000","last_became_peered":"2024-04-17T07:11:59.522470+0000","last_unstale":"2024-04-17T07:13:29.334157+0000","last_undegraded":"2024-04-17T07:13:29.334157+0000","last_fullsized":"2024-04-17T07:13:29.334157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:36.620442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.171736+0000","last_change":"2024-04-17T07:11:54.470693+0000","last_active":"2024-04-17T07:13:29.171736+0000","last_peered":"2024-04-17T07:13:29.171736+0000","last_clean":"2024-04-17T07:13:29.171736+0000","last_became_active":"2024-04-17T07:11:54.470334+0000","last_became_peered":"2024-04-17T07:11:54.470334+0000","last_unstale":"2024-04-17T07:13:29.171736+0000","last_undegraded":"2024-04-17T07:13:29.171736+0000","last_fullsized":"2024-04-17T07:13:29.171736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:19:04.980786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070659+0000","last_change":"2024-04-17T07:11:55.474755+0000","last_active":"2024-04-17T07:13:57.070659+0000","last_peered":"2024-04-17T07:13:57.070659+0000","last_clean":"2024-04-17T07:13:57.070659+0000","last_became_active":"2024-04-17T07:11:55.474261+0000","last_became_peered":"2024-04-17T07:11:55.474261+0000","last_unstale":"2024-04-17T07:13:57.070659+0000","last_undegraded":"2024-04-17T07:13:57.070659+0000","last_fullsized":"2024-04-17T07:13:57.070659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:00.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203208+0000","last_change":"2024-04-17T07:11:59.517152+0000","last_active":"2024-04-17T07:13:29.203208+0000","last_peered":"2024-04-17T07:13:29.203208+0000","last_clean":"2024-04-17T07:13:29.203208+0000","last_became_active":"2024-04-17T07:11:59.514977+0000","last_became_peered":"2024-04-17T07:11:59.514977+0000","last_unstale":"2024-04-17T07:13:29.203208+0000","last_undegraded":"2024-04-17T07:13:29.203208+0000","last_fullsized":"2024-04-17T07:13:29.203208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:14.974136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172963+0000","last_change":"2024-04-17T07:11:54.459786+0000","last_active":"2024-04-17T07:13:29.172963+0000","last_peered":"2024-04-17T07:13:29.172963+0000","last_clean":"2024-04-17T07:13:29.172963+0000","last_became_active":"2024-04-17T07:11:54.457589+0000","last_became_peered":"2024-04-17T07:11:54.457589+0000","last_unstale":"2024-04-17T07:13:29.172963+0000","last_undegraded":"2024-04-17T07:13:29.172963+0000","last_fullsized":"2024-04-17T07:13:29.172963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:25.986539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068150+0000","last_change":"2024-04-17T07:11:55.465020+0000","last_active":"2024-04-17T07:13:57.068150+0000","last_peered":"2024-04-17T07:13:57.068150+0000","last_clean":"2024-04-17T07:13:57.068150+0000","last_became_active":"2024-04-17T07:11:55.464608+0000","last_became_peered":"2024-04-17T07:11:55.464608+0000","last_unstale":"2024-04-17T07:13:57.068150+0000","last_undegraded":"2024-04-17T07:13:57.068150+0000","last_fullsized":"2024-04-17T07:13:57.068150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204578+0000","last_change":"2024-04-17T07:11:59.519333+0000","last_active":"2024-04-17T07:13:29.204578+0000","last_peered":"2024-04-17T07:13:29.204578+0000","last_clean":"2024-04-17T07:13:29.204578+0000","last_became_active":"2024-04-17T07:11:59.518865+0000","last_became_peered":"2024-04-17T07:11:59.518865+0000","last_unstale":"2024-04-17T07:13:29.204578+0000","last_undegraded":"2024-04-17T07:13:29.204578+0000","last_fullsized":"2024-04-17T07:13:29.204578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:42.009010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":341,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:01.666653+0000","last_change":"2024-04-17T07:11:54.462283+0000","last_active":"2024-04-17T07:15:01.666653+0000","last_peered":"2024-04-17T07:15:01.666653+0000","last_clean":"2024-04-17T07:15:01.666653+0000","last_became_active":"2024-04-17T07:11:54.461818+0000","last_became_peered":"2024-04-17T07:11:54.461818+0000","last_unstale":"2024-04-17T07:15:01.666653+0000","last_undegraded":"2024-04-17T07:15:01.666653+0000","last_fullsized":"2024-04-17T07:15:01.666653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.765058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":302,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081011+0000","last_change":"2024-04-17T07:11:55.486491+0000","last_active":"2024-04-17T07:13:57.081011+0000","last_peered":"2024-04-17T07:13:57.081011+0000","last_clean":"2024-04-17T07:13:57.081011+0000","last_became_active":"2024-04-17T07:11:55.486121+0000","last_became_peered":"2024-04-17T07:11:55.486121+0000","last_unstale":"2024-04-17T07:13:57.081011+0000","last_undegraded":"2024-04-17T07:13:57.081011+0000","last_fullsized":"2024-04-17T07:13:57.081011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:12.998321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205968+0000","last_change":"2024-04-17T07:11:59.511555+0000","last_active":"2024-04-17T07:13:29.205968+0000","last_peered":"2024-04-17T07:13:29.205968+0000","last_clean":"2024-04-17T07:13:29.205968+0000","last_became_active":"2024-04-17T07:11:59.511147+0000","last_became_peered":"2024-04-17T07:11:59.511147+0000","last_unstale":"2024-04-17T07:13:29.205968+0000","last_undegraded":"2024-04-17T07:13:29.205968+0000","last_fullsized":"2024-04-17T07:13:29.205968+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:41.854815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209570+0000","last_change":"2024-04-17T07:12:28.734476+0000","last_active":"2024-04-17T07:13:29.209570+0000","last_peered":"2024-04-17T07:13:29.209570+0000","last_clean":"2024-04-17T07:13:29.209570+0000","last_became_active":"2024-04-17T07:12:28.728187+0000","last_became_peered":"2024-04-17T07:12:28.728187+0000","last_unstale":"2024-04-17T07:13:29.209570+0000","last_undegraded":"2024-04-17T07:13:29.209570+0000","last_fullsized":"2024-04-17T07:13:29.209570+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:45.749607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081552+0000","last_change":"2024-04-17T07:11:55.480589+0000","last_active":"2024-04-17T07:13:57.081552+0000","last_peered":"2024-04-17T07:13:57.081552+0000","last_clean":"2024-04-17T07:13:57.081552+0000","last_became_active":"2024-04-17T07:11:55.480224+0000","last_became_peered":"2024-04-17T07:11:55.480224+0000","last_unstale":"2024-04-17T07:13:57.081552+0000","last_undegraded":"2024-04-17T07:13:57.081552+0000","last_fullsized":"2024-04-17T07:13:57.081552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:05.892442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175186+0000","last_change":"2024-04-17T07:11:59.522021+0000","last_active":"2024-04-17T07:13:29.175186+0000","last_peered":"2024-04-17T07:13:29.175186+0000","last_clean":"2024-04-17T07:13:29.175186+0000","last_became_active":"2024-04-17T07:11:59.521493+0000","last_became_peered":"2024-04-17T07:11:59.521493+0000","last_unstale":"2024-04-17T07:13:29.175186+0000","last_undegraded":"2024-04-17T07:13:29.175186+0000","last_fullsized":"2024-04-17T07:13:29.175186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:20.386216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174471+0000","last_change":"2024-04-17T07:11:54.468225+0000","last_active":"2024-04-17T07:13:29.174471+0000","last_peered":"2024-04-17T07:13:29.174471+0000","last_clean":"2024-04-17T07:13:29.174471+0000","last_became_active":"2024-04-17T07:11:54.467046+0000","last_became_peered":"2024-04-17T07:11:54.467046+0000","last_unstale":"2024-04-17T07:13:29.174471+0000","last_undegraded":"2024-04-17T07:13:29.174471+0000","last_fullsized":"2024-04-17T07:13:29.174471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:17.725255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083563+0000","last_change":"2024-04-17T07:11:55.488980+0000","last_active":"2024-04-17T07:13:57.083563+0000","last_peered":"2024-04-17T07:13:57.083563+0000","last_clean":"2024-04-17T07:13:57.083563+0000","last_became_active":"2024-04-17T07:11:55.488226+0000","last_became_peered":"2024-04-17T07:11:55.488226+0000","last_unstale":"2024-04-17T07:13:57.083563+0000","last_undegraded":"2024-04-17T07:13:57.083563+0000","last_fullsized":"2024-04-17T07:13:57.083563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:00.135973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205610+0000","last_change":"2024-04-17T07:11:59.515134+0000","last_active":"2024-04-17T07:13:29.205610+0000","last_peered":"2024-04-17T07:13:29.205610+0000","last_clean":"2024-04-17T07:13:29.205610+0000","last_became_active":"2024-04-17T07:11:59.514560+0000","last_became_peered":"2024-04-17T07:11:59.514560+0000","last_unstale":"2024-04-17T07:13:29.205610+0000","last_undegraded":"2024-04-17T07:13:29.205610+0000","last_fullsized":"2024-04-17T07:13:29.205610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:34.669789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":381,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:01.667846+0000","last_change":"2024-04-17T07:11:54.461226+0000","last_active":"2024-04-17T07:15:01.667846+0000","last_peered":"2024-04-17T07:15:01.667846+0000","last_clean":"2024-04-17T07:15:01.667846+0000","last_became_active":"2024-04-17T07:11:54.460573+0000","last_became_peered":"2024-04-17T07:11:54.460573+0000","last_unstale":"2024-04-17T07:15:01.667846+0000","last_undegraded":"2024-04-17T07:15:01.667846+0000","last_fullsized":"2024-04-17T07:15:01.667846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67432,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":310,"num_read_kb":216,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068452+0000","last_change":"2024-04-17T07:11:55.483925+0000","last_active":"2024-04-17T07:13:57.068452+0000","last_peered":"2024-04-17T07:13:57.068452+0000","last_clean":"2024-04-17T07:13:57.068452+0000","last_became_active":"2024-04-17T07:11:55.483584+0000","last_became_peered":"2024-04-17T07:11:55.483584+0000","last_unstale":"2024-04-17T07:13:57.068452+0000","last_undegraded":"2024-04-17T07:13:57.068452+0000","last_fullsized":"2024-04-17T07:13:57.068452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:12.328876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174981+0000","last_change":"2024-04-17T07:11:59.515597+0000","last_active":"2024-04-17T07:13:29.174981+0000","last_peered":"2024-04-17T07:13:29.174981+0000","last_clean":"2024-04-17T07:13:29.174981+0000","last_became_active":"2024-04-17T07:11:59.514413+0000","last_became_peered":"2024-04-17T07:11:59.514413+0000","last_unstale":"2024-04-17T07:13:29.174981+0000","last_undegraded":"2024-04-17T07:13:29.174981+0000","last_fullsized":"2024-04-17T07:13:29.174981+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:45.824127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205257+0000","last_change":"2024-04-17T07:11:54.449984+0000","last_active":"2024-04-17T07:13:29.205257+0000","last_peered":"2024-04-17T07:13:29.205257+0000","last_clean":"2024-04-17T07:13:29.205257+0000","last_became_active":"2024-04-17T07:11:54.449628+0000","last_became_peered":"2024-04-17T07:11:54.449628+0000","last_unstale":"2024-04-17T07:13:29.205257+0000","last_undegraded":"2024-04-17T07:13:29.205257+0000","last_fullsized":"2024-04-17T07:13:29.205257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:26:03.281684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069434+0000","last_change":"2024-04-17T07:11:55.476912+0000","last_active":"2024-04-17T07:13:57.069434+0000","last_peered":"2024-04-17T07:13:57.069434+0000","last_clean":"2024-04-17T07:13:57.069434+0000","last_became_active":"2024-04-17T07:11:55.471576+0000","last_became_peered":"2024-04-17T07:11:55.471576+0000","last_unstale":"2024-04-17T07:13:57.069434+0000","last_undegraded":"2024-04-17T07:13:57.069434+0000","last_fullsized":"2024-04-17T07:13:57.069434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:35.439271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333921+0000","last_change":"2024-04-17T07:11:59.515897+0000","last_active":"2024-04-17T07:13:29.333921+0000","last_peered":"2024-04-17T07:13:29.333921+0000","last_clean":"2024-04-17T07:13:29.333921+0000","last_became_active":"2024-04-17T07:11:59.515349+0000","last_became_peered":"2024-04-17T07:11:59.515349+0000","last_unstale":"2024-04-17T07:13:29.333921+0000","last_undegraded":"2024-04-17T07:13:29.333921+0000","last_fullsized":"2024-04-17T07:13:29.333921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:00.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173683+0000","last_change":"2024-04-17T07:11:54.462369+0000","last_active":"2024-04-17T07:13:29.173683+0000","last_peered":"2024-04-17T07:13:29.173683+0000","last_clean":"2024-04-17T07:13:29.173683+0000","last_became_active":"2024-04-17T07:11:54.460357+0000","last_became_peered":"2024-04-17T07:11:54.460357+0000","last_unstale":"2024-04-17T07:13:29.173683+0000","last_undegraded":"2024-04-17T07:13:29.173683+0000","last_fullsized":"2024-04-17T07:13:29.173683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:38.029980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071612+0000","last_change":"2024-04-17T07:11:55.475874+0000","last_active":"2024-04-17T07:13:57.071612+0000","last_peered":"2024-04-17T07:13:57.071612+0000","last_clean":"2024-04-17T07:13:57.071612+0000","last_became_active":"2024-04-17T07:11:55.475550+0000","last_became_peered":"2024-04-17T07:11:55.475550+0000","last_unstale":"2024-04-17T07:13:57.071612+0000","last_undegraded":"2024-04-17T07:13:57.071612+0000","last_fullsized":"2024-04-17T07:13:57.071612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:40.096573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206848+0000","last_change":"2024-04-17T07:11:59.514661+0000","last_active":"2024-04-17T07:13:29.206848+0000","last_peered":"2024-04-17T07:13:29.206848+0000","last_clean":"2024-04-17T07:13:29.206848+0000","last_became_active":"2024-04-17T07:11:59.513030+0000","last_became_peered":"2024-04-17T07:11:59.513030+0000","last_unstale":"2024-04-17T07:13:29.206848+0000","last_undegraded":"2024-04-17T07:13:29.206848+0000","last_fullsized":"2024-04-17T07:13:29.206848+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:35.696961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":194,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:02.671857+0000","last_change":"2024-04-17T07:11:54.455279+0000","last_active":"2024-04-17T07:15:02.671857+0000","last_peered":"2024-04-17T07:15:02.671857+0000","last_clean":"2024-04-17T07:15:02.671857+0000","last_became_active":"2024-04-17T07:11:54.453620+0000","last_became_peered":"2024-04-17T07:11:54.453620+0000","last_unstale":"2024-04-17T07:15:02.671857+0000","last_undegraded":"2024-04-17T07:15:02.671857+0000","last_fullsized":"2024-04-17T07:15:02.671857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:15.227410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":157,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.072155+0000","last_change":"2024-04-17T07:11:55.469579+0000","last_active":"2024-04-17T07:13:57.072155+0000","last_peered":"2024-04-17T07:13:57.072155+0000","last_clean":"2024-04-17T07:13:57.072155+0000","last_became_active":"2024-04-17T07:11:55.468389+0000","last_became_peered":"2024-04-17T07:11:55.468389+0000","last_unstale":"2024-04-17T07:13:57.072155+0000","last_undegraded":"2024-04-17T07:13:57.072155+0000","last_fullsized":"2024-04-17T07:13:57.072155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:01.274677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173558+0000","last_change":"2024-04-17T07:11:59.512110+0000","last_active":"2024-04-17T07:13:29.173558+0000","last_peered":"2024-04-17T07:13:29.173558+0000","last_clean":"2024-04-17T07:13:29.173558+0000","last_became_active":"2024-04-17T07:11:59.510573+0000","last_became_peered":"2024-04-17T07:11:59.510573+0000","last_unstale":"2024-04-17T07:13:29.173558+0000","last_undegraded":"2024-04-17T07:13:29.173558+0000","last_fullsized":"2024-04-17T07:13:29.173558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:32.515687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333363+0000","last_change":"2024-04-17T07:12:28.767517+0000","last_active":"2024-04-17T07:13:29.333363+0000","last_peered":"2024-04-17T07:13:29.333363+0000","last_clean":"2024-04-17T07:13:29.333363+0000","last_became_active":"2024-04-17T07:12:28.738908+0000","last_became_peered":"2024-04-17T07:12:28.738908+0000","last_unstale":"2024-04-17T07:13:29.333363+0000","last_undegraded":"2024-04-17T07:13:29.333363+0000","last_fullsized":"2024-04-17T07:13:29.333363+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:47:30.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.073775+0000","last_change":"2024-04-17T07:11:55.473540+0000","last_active":"2024-04-17T07:13:57.073775+0000","last_peered":"2024-04-17T07:13:57.073775+0000","last_clean":"2024-04-17T07:13:57.073775+0000","last_became_active":"2024-04-17T07:11:55.472436+0000","last_became_peered":"2024-04-17T07:11:55.472436+0000","last_unstale":"2024-04-17T07:13:57.073775+0000","last_undegraded":"2024-04-17T07:13:57.073775+0000","last_fullsized":"2024-04-17T07:13:57.073775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:54.317052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172929+0000","last_change":"2024-04-17T07:11:59.521305+0000","last_active":"2024-04-17T07:13:29.172929+0000","last_peered":"2024-04-17T07:13:29.172929+0000","last_clean":"2024-04-17T07:13:29.172929+0000","last_became_active":"2024-04-17T07:11:59.516860+0000","last_became_peered":"2024-04-17T07:11:59.516860+0000","last_unstale":"2024-04-17T07:13:29.172929+0000","last_undegraded":"2024-04-17T07:13:29.172929+0000","last_fullsized":"2024-04-17T07:13:29.172929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:23.403218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:11.134134+0000","last_change":"2024-04-17T07:11:54.463378+0000","last_active":"2024-04-17T07:14:11.134134+0000","last_peered":"2024-04-17T07:14:11.134134+0000","last_clean":"2024-04-17T07:14:11.134134+0000","last_became_active":"2024-04-17T07:11:54.462120+0000","last_became_peered":"2024-04-17T07:11:54.462120+0000","last_unstale":"2024-04-17T07:14:11.134134+0000","last_undegraded":"2024-04-17T07:14:11.134134+0000","last_fullsized":"2024-04-17T07:14:11.134134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:42.109928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071709+0000","last_change":"2024-04-17T07:11:55.478604+0000","last_active":"2024-04-17T07:13:57.071709+0000","last_peered":"2024-04-17T07:13:57.071709+0000","last_clean":"2024-04-17T07:13:57.071709+0000","last_became_active":"2024-04-17T07:11:55.478252+0000","last_became_peered":"2024-04-17T07:11:55.478252+0000","last_unstale":"2024-04-17T07:13:57.071709+0000","last_undegraded":"2024-04-17T07:13:57.071709+0000","last_fullsized":"2024-04-17T07:13:57.071709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:08.365896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206440+0000","last_change":"2024-04-17T07:11:59.518619+0000","last_active":"2024-04-17T07:13:29.206440+0000","last_peered":"2024-04-17T07:13:29.206440+0000","last_clean":"2024-04-17T07:13:29.206440+0000","last_became_active":"2024-04-17T07:11:59.518214+0000","last_became_peered":"2024-04-17T07:11:59.518214+0000","last_unstale":"2024-04-17T07:13:29.206440+0000","last_undegraded":"2024-04-17T07:13:29.206440+0000","last_fullsized":"2024-04-17T07:13:29.206440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:46.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206316+0000","last_change":"2024-04-17T07:11:54.453038+0000","last_active":"2024-04-17T07:13:29.206316+0000","last_peered":"2024-04-17T07:13:29.206316+0000","last_clean":"2024-04-17T07:13:29.206316+0000","last_became_active":"2024-04-17T07:11:54.452619+0000","last_became_peered":"2024-04-17T07:11:54.452619+0000","last_unstale":"2024-04-17T07:13:29.206316+0000","last_undegraded":"2024-04-17T07:13:29.206316+0000","last_fullsized":"2024-04-17T07:13:29.206316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:25.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.063521+0000","last_change":"2024-04-17T07:11:55.465990+0000","last_active":"2024-04-17T07:13:57.063521+0000","last_peered":"2024-04-17T07:13:57.063521+0000","last_clean":"2024-04-17T07:13:57.063521+0000","last_became_active":"2024-04-17T07:11:55.465523+0000","last_became_peered":"2024-04-17T07:11:55.465523+0000","last_unstale":"2024-04-17T07:13:57.063521+0000","last_undegraded":"2024-04-17T07:13:57.063521+0000","last_fullsized":"2024-04-17T07:13:57.063521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:40.366027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172399+0000","last_change":"2024-04-17T07:11:59.512119+0000","last_active":"2024-04-17T07:13:29.172399+0000","last_peered":"2024-04-17T07:13:29.172399+0000","last_clean":"2024-04-17T07:13:29.172399+0000","last_became_active":"2024-04-17T07:11:59.511460+0000","last_became_peered":"2024-04-17T07:11:59.511460+0000","last_unstale":"2024-04-17T07:13:29.172399+0000","last_undegraded":"2024-04-17T07:13:29.172399+0000","last_fullsized":"2024-04-17T07:13:29.172399+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:50.888203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":196,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:05.650024+0000","last_change":"2024-04-17T07:11:54.469346+0000","last_active":"2024-04-17T07:15:05.650024+0000","last_peered":"2024-04-17T07:15:05.650024+0000","last_clean":"2024-04-17T07:15:05.650024+0000","last_became_active":"2024-04-17T07:11:54.468947+0000","last_became_peered":"2024-04-17T07:11:54.468947+0000","last_unstale":"2024-04-17T07:15:05.650024+0000","last_undegraded":"2024-04-17T07:15:05.650024+0000","last_fullsized":"2024-04-17T07:15:05.650024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:04.854625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":158,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078974+0000","last_change":"2024-04-17T07:11:55.471235+0000","last_active":"2024-04-17T07:13:57.078974+0000","last_peered":"2024-04-17T07:13:57.078974+0000","last_clean":"2024-04-17T07:13:57.078974+0000","last_became_active":"2024-04-17T07:11:55.470857+0000","last_became_peered":"2024-04-17T07:11:55.470857+0000","last_unstale":"2024-04-17T07:13:57.078974+0000","last_undegraded":"2024-04-17T07:13:57.078974+0000","last_fullsized":"2024-04-17T07:13:57.078974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:21.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172054+0000","last_change":"2024-04-17T07:11:59.524519+0000","last_active":"2024-04-17T07:13:29.172054+0000","last_peered":"2024-04-17T07:13:29.172054+0000","last_clean":"2024-04-17T07:13:29.172054+0000","last_became_active":"2024-04-17T07:11:59.519270+0000","last_became_peered":"2024-04-17T07:11:59.519270+0000","last_unstale":"2024-04-17T07:13:29.172054+0000","last_undegraded":"2024-04-17T07:13:29.172054+0000","last_fullsized":"2024-04-17T07:13:29.172054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:41.080249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070265+0000","last_change":"2024-04-17T07:11:55.473922+0000","last_active":"2024-04-17T07:13:57.070265+0000","last_peered":"2024-04-17T07:13:57.070265+0000","last_clean":"2024-04-17T07:13:57.070265+0000","last_became_active":"2024-04-17T07:11:55.468594+0000","last_became_peered":"2024-04-17T07:11:55.468594+0000","last_unstale":"2024-04-17T07:13:57.070265+0000","last_undegraded":"2024-04-17T07:13:57.070265+0000","last_fullsized":"2024-04-17T07:13:57.070265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:21.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":191,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:01.667538+0000","last_change":"2024-04-17T07:11:54.473066+0000","last_active":"2024-04-17T07:15:01.667538+0000","last_peered":"2024-04-17T07:15:01.667538+0000","last_clean":"2024-04-17T07:15:01.667538+0000","last_became_active":"2024-04-17T07:11:54.472708+0000","last_became_peered":"2024-04-17T07:11:54.472708+0000","last_unstale":"2024-04-17T07:15:01.667538+0000","last_undegraded":"2024-04-17T07:15:01.667538+0000","last_fullsized":"2024-04-17T07:15:01.667538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:27.614819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209116+0000","last_change":"2024-04-17T07:11:59.516939+0000","last_active":"2024-04-17T07:13:29.209116+0000","last_peered":"2024-04-17T07:13:29.209116+0000","last_clean":"2024-04-17T07:13:29.209116+0000","last_became_active":"2024-04-17T07:11:59.516333+0000","last_became_peered":"2024-04-17T07:11:59.516333+0000","last_unstale":"2024-04-17T07:13:29.209116+0000","last_undegraded":"2024-04-17T07:13:29.209116+0000","last_fullsized":"2024-04-17T07:13:29.209116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:15.648831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082745+0000","last_change":"2024-04-17T07:11:55.481006+0000","last_active":"2024-04-17T07:13:57.082745+0000","last_peered":"2024-04-17T07:13:57.082745+0000","last_clean":"2024-04-17T07:13:57.082745+0000","last_became_active":"2024-04-17T07:11:55.479751+0000","last_became_peered":"2024-04-17T07:11:55.479751+0000","last_unstale":"2024-04-17T07:13:57.082745+0000","last_undegraded":"2024-04-17T07:13:57.082745+0000","last_fullsized":"2024-04-17T07:13:57.082745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:13.004650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175099+0000","last_change":"2024-04-17T07:11:54.463106+0000","last_active":"2024-04-17T07:13:29.175099+0000","last_peered":"2024-04-17T07:13:29.175099+0000","last_clean":"2024-04-17T07:13:29.175099+0000","last_became_active":"2024-04-17T07:11:54.462691+0000","last_became_peered":"2024-04-17T07:11:54.462691+0000","last_unstale":"2024-04-17T07:13:29.175099+0000","last_undegraded":"2024-04-17T07:13:29.175099+0000","last_fullsized":"2024-04-17T07:13:29.175099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:34.302019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205049+0000","last_change":"2024-04-17T07:10:14.685883+0000","last_active":"2024-04-17T07:13:29.205049+0000","last_peered":"2024-04-17T07:13:29.205049+0000","last_clean":"2024-04-17T07:13:29.205049+0000","last_became_active":"2024-04-17T07:10:14.684551+0000","last_became_peered":"2024-04-17T07:10:14.684551+0000","last_unstale":"2024-04-17T07:13:29.205049+0000","last_undegraded":"2024-04-17T07:13:29.205049+0000","last_fullsized":"2024-04-17T07:13:29.205049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_clean_scrub_stamp":"2024-04-17T07:05:37.112317+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:04.265147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206602+0000","last_change":"2024-04-17T07:12:28.731807+0000","last_active":"2024-04-17T07:13:29.206602+0000","last_peered":"2024-04-17T07:13:29.206602+0000","last_clean":"2024-04-17T07:13:29.206602+0000","last_became_active":"2024-04-17T07:12:28.730656+0000","last_became_peered":"2024-04-17T07:12:28.730656+0000","last_unstale":"2024-04-17T07:13:29.206602+0000","last_undegraded":"2024-04-17T07:13:29.206602+0000","last_fullsized":"2024-04-17T07:13:29.206602+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:58.023333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068799+0000","last_change":"2024-04-17T07:12:28.727980+0000","last_active":"2024-04-17T07:13:57.068799+0000","last_peered":"2024-04-17T07:13:57.068799+0000","last_clean":"2024-04-17T07:13:57.068799+0000","last_became_active":"2024-04-17T07:12:28.727060+0000","last_became_peered":"2024-04-17T07:12:28.727060+0000","last_unstale":"2024-04-17T07:13:57.068799+0000","last_undegraded":"2024-04-17T07:13:57.068799+0000","last_fullsized":"2024-04-17T07:13:57.068799+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:33.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205324+0000","last_change":"2024-04-17T07:11:54.463180+0000","last_active":"2024-04-17T07:13:29.205324+0000","last_peered":"2024-04-17T07:13:29.205324+0000","last_clean":"2024-04-17T07:13:29.205324+0000","last_became_active":"2024-04-17T07:11:54.462793+0000","last_became_peered":"2024-04-17T07:11:54.462793+0000","last_unstale":"2024-04-17T07:13:29.205324+0000","last_undegraded":"2024-04-17T07:13:29.205324+0000","last_fullsized":"2024-04-17T07:13:29.205324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:19.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333323+0000","last_change":"2024-04-17T07:11:59.518355+0000","last_active":"2024-04-17T07:13:29.333323+0000","last_peered":"2024-04-17T07:13:29.333323+0000","last_clean":"2024-04-17T07:13:29.333323+0000","last_became_active":"2024-04-17T07:11:59.517619+0000","last_became_peered":"2024-04-17T07:11:59.517619+0000","last_unstale":"2024-04-17T07:13:29.333323+0000","last_undegraded":"2024-04-17T07:13:29.333323+0000","last_fullsized":"2024-04-17T07:13:29.333323+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:36.826021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206403+0000","last_change":"2024-04-17T07:11:54.465285+0000","last_active":"2024-04-17T07:13:29.206403+0000","last_peered":"2024-04-17T07:13:29.206403+0000","last_clean":"2024-04-17T07:13:29.206403+0000","last_became_active":"2024-04-17T07:11:54.464818+0000","last_became_peered":"2024-04-17T07:11:54.464818+0000","last_unstale":"2024-04-17T07:13:29.206403+0000","last_undegraded":"2024-04-17T07:13:29.206403+0000","last_fullsized":"2024-04-17T07:13:29.206403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069247+0000","last_change":"2024-04-17T07:11:55.470405+0000","last_active":"2024-04-17T07:13:57.069247+0000","last_peered":"2024-04-17T07:13:57.069247+0000","last_clean":"2024-04-17T07:13:57.069247+0000","last_became_active":"2024-04-17T07:11:55.470058+0000","last_became_peered":"2024-04-17T07:11:55.470058+0000","last_unstale":"2024-04-17T07:13:57.069247+0000","last_undegraded":"2024-04-17T07:13:57.069247+0000","last_fullsized":"2024-04-17T07:13:57.069247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:17.416775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.202954+0000","last_change":"2024-04-17T07:11:59.522040+0000","last_active":"2024-04-17T07:13:29.202954+0000","last_peered":"2024-04-17T07:13:29.202954+0000","last_clean":"2024-04-17T07:13:29.202954+0000","last_became_active":"2024-04-17T07:11:59.519771+0000","last_became_peered":"2024-04-17T07:11:59.519771+0000","last_unstale":"2024-04-17T07:13:29.202954+0000","last_undegraded":"2024-04-17T07:13:29.202954+0000","last_fullsized":"2024-04-17T07:13:29.202954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:58.203737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'17","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:00.514634+0000","last_change":"2024-04-17T07:11:54.457954+0000","last_active":"2024-04-17T07:15:00.514634+0000","last_peered":"2024-04-17T07:15:00.514634+0000","last_clean":"2024-04-17T07:15:00.514634+0000","last_became_active":"2024-04-17T07:11:54.457560+0000","last_became_peered":"2024-04-17T07:11:54.457560+0000","last_unstale":"2024-04-17T07:15:00.514634+0000","last_undegraded":"2024-04-17T07:15:00.514634+0000","last_fullsized":"2024-04-17T07:15:00.514634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:31.919730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068908+0000","last_change":"2024-04-17T07:11:55.480475+0000","last_active":"2024-04-17T07:13:57.068908+0000","last_peered":"2024-04-17T07:13:57.068908+0000","last_clean":"2024-04-17T07:13:57.068908+0000","last_became_active":"2024-04-17T07:11:55.480060+0000","last_became_peered":"2024-04-17T07:11:55.480060+0000","last_unstale":"2024-04-17T07:13:57.068908+0000","last_undegraded":"2024-04-17T07:13:57.068908+0000","last_fullsized":"2024-04-17T07:13:57.068908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:30:23.600244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203796+0000","last_change":"2024-04-17T07:11:59.525422+0000","last_active":"2024-04-17T07:13:29.203796+0000","last_peered":"2024-04-17T07:13:29.203796+0000","last_clean":"2024-04-17T07:13:29.203796+0000","last_became_active":"2024-04-17T07:11:59.523649+0000","last_became_peered":"2024-04-17T07:11:59.523649+0000","last_unstale":"2024-04-17T07:13:29.203796+0000","last_undegraded":"2024-04-17T07:13:29.203796+0000","last_fullsized":"2024-04-17T07:13:29.203796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:16.653752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.176127+0000","last_change":"2024-04-17T07:11:54.465824+0000","last_active":"2024-04-17T07:13:29.176127+0000","last_peered":"2024-04-17T07:13:29.176127+0000","last_clean":"2024-04-17T07:13:29.176127+0000","last_became_active":"2024-04-17T07:11:54.465348+0000","last_became_peered":"2024-04-17T07:11:54.465348+0000","last_unstale":"2024-04-17T07:13:29.176127+0000","last_undegraded":"2024-04-17T07:13:29.176127+0000","last_fullsized":"2024-04-17T07:13:29.176127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:17.729903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081497+0000","last_change":"2024-04-17T07:11:55.481666+0000","last_active":"2024-04-17T07:13:57.081497+0000","last_peered":"2024-04-17T07:13:57.081497+0000","last_clean":"2024-04-17T07:13:57.081497+0000","last_became_active":"2024-04-17T07:11:55.480332+0000","last_became_peered":"2024-04-17T07:11:55.480332+0000","last_unstale":"2024-04-17T07:13:57.081497+0000","last_undegraded":"2024-04-17T07:13:57.081497+0000","last_fullsized":"2024-04-17T07:13:57.081497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:19.825407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175613+0000","last_change":"2024-04-17T07:11:59.527850+0000","last_active":"2024-04-17T07:13:29.175613+0000","last_peered":"2024-04-17T07:13:29.175613+0000","last_clean":"2024-04-17T07:13:29.175613+0000","last_became_active":"2024-04-17T07:11:59.527306+0000","last_became_peered":"2024-04-17T07:11:59.527306+0000","last_unstale":"2024-04-17T07:13:29.175613+0000","last_undegraded":"2024-04-17T07:13:29.175613+0000","last_fullsized":"2024-04-17T07:13:29.175613+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:35.906222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204229+0000","last_change":"2024-04-17T07:11:54.455862+0000","last_active":"2024-04-17T07:13:29.204229+0000","last_peered":"2024-04-17T07:13:29.204229+0000","last_clean":"2024-04-17T07:13:29.204229+0000","last_became_active":"2024-04-17T07:11:54.453904+0000","last_became_peered":"2024-04-17T07:11:54.453904+0000","last_unstale":"2024-04-17T07:13:29.204229+0000","last_undegraded":"2024-04-17T07:13:29.204229+0000","last_fullsized":"2024-04-17T07:13:29.204229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:01.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.060326+0000","last_change":"2024-04-17T07:11:55.470315+0000","last_active":"2024-04-17T07:13:57.060326+0000","last_peered":"2024-04-17T07:13:57.060326+0000","last_clean":"2024-04-17T07:13:57.060326+0000","last_became_active":"2024-04-17T07:11:55.469829+0000","last_became_peered":"2024-04-17T07:11:55.469829+0000","last_unstale":"2024-04-17T07:13:57.060326+0000","last_undegraded":"2024-04-17T07:13:57.060326+0000","last_fullsized":"2024-04-17T07:13:57.060326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:48.197546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.210025+0000","last_change":"2024-04-17T07:12:29.125447+0000","last_active":"2024-04-17T07:13:29.210025+0000","last_peered":"2024-04-17T07:13:29.210025+0000","last_clean":"2024-04-17T07:13:29.210025+0000","last_became_active":"2024-04-17T07:12:29.124551+0000","last_became_peered":"2024-04-17T07:12:29.124551+0000","last_unstale":"2024-04-17T07:13:29.210025+0000","last_undegraded":"2024-04-17T07:13:29.210025+0000","last_fullsized":"2024-04-17T07:13:29.210025+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:09.489755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204318+0000","last_change":"2024-04-17T07:11:54.453117+0000","last_active":"2024-04-17T07:13:29.204318+0000","last_peered":"2024-04-17T07:13:29.204318+0000","last_clean":"2024-04-17T07:13:29.204318+0000","last_became_active":"2024-04-17T07:11:54.452632+0000","last_became_peered":"2024-04-17T07:11:54.452632+0000","last_unstale":"2024-04-17T07:13:29.204318+0000","last_undegraded":"2024-04-17T07:13:29.204318+0000","last_fullsized":"2024-04-17T07:13:29.204318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:09.366149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078286+0000","last_change":"2024-04-17T07:11:55.469991+0000","last_active":"2024-04-17T07:13:57.078286+0000","last_peered":"2024-04-17T07:13:57.078286+0000","last_clean":"2024-04-17T07:13:57.078286+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:13:57.078286+0000","last_undegraded":"2024-04-17T07:13:57.078286+0000","last_fullsized":"2024-04-17T07:13:57.078286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:02.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175116+0000","last_change":"2024-04-17T07:11:59.519817+0000","last_active":"2024-04-17T07:13:29.175116+0000","last_peered":"2024-04-17T07:13:29.175116+0000","last_clean":"2024-04-17T07:13:29.175116+0000","last_became_active":"2024-04-17T07:11:59.519108+0000","last_became_peered":"2024-04-17T07:11:59.519108+0000","last_unstale":"2024-04-17T07:13:29.175116+0000","last_undegraded":"2024-04-17T07:13:29.175116+0000","last_fullsized":"2024-04-17T07:13:29.175116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:58.378837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:59.206828+0000","last_change":"2024-04-17T07:11:54.463618+0000","last_active":"2024-04-17T07:14:59.206828+0000","last_peered":"2024-04-17T07:14:59.206828+0000","last_clean":"2024-04-17T07:14:59.206828+0000","last_became_active":"2024-04-17T07:11:54.463257+0000","last_became_peered":"2024-04-17T07:11:54.463257+0000","last_unstale":"2024-04-17T07:14:59.206828+0000","last_undegraded":"2024-04-17T07:14:59.206828+0000","last_fullsized":"2024-04-17T07:14:59.206828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:20.135338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077111+0000","last_change":"2024-04-17T07:11:55.487471+0000","last_active":"2024-04-17T07:13:57.077111+0000","last_peered":"2024-04-17T07:13:57.077111+0000","last_clean":"2024-04-17T07:13:57.077111+0000","last_became_active":"2024-04-17T07:11:55.487024+0000","last_became_peered":"2024-04-17T07:11:55.487024+0000","last_unstale":"2024-04-17T07:13:57.077111+0000","last_undegraded":"2024-04-17T07:13:57.077111+0000","last_fullsized":"2024-04-17T07:13:57.077111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:36.851210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175245+0000","last_change":"2024-04-17T07:11:59.527017+0000","last_active":"2024-04-17T07:13:29.175245+0000","last_peered":"2024-04-17T07:13:29.175245+0000","last_clean":"2024-04-17T07:13:29.175245+0000","last_became_active":"2024-04-17T07:11:59.526541+0000","last_became_peered":"2024-04-17T07:11:59.526541+0000","last_unstale":"2024-04-17T07:13:29.175245+0000","last_undegraded":"2024-04-17T07:13:29.175245+0000","last_fullsized":"2024-04-17T07:13:29.175245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:54.411647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204636+0000","last_change":"2024-04-17T07:11:54.465671+0000","last_active":"2024-04-17T07:13:29.204636+0000","last_peered":"2024-04-17T07:13:29.204636+0000","last_clean":"2024-04-17T07:13:29.204636+0000","last_became_active":"2024-04-17T07:11:54.464902+0000","last_became_peered":"2024-04-17T07:11:54.464902+0000","last_unstale":"2024-04-17T07:13:29.204636+0000","last_undegraded":"2024-04-17T07:13:29.204636+0000","last_fullsized":"2024-04-17T07:13:29.204636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:13:38.232965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084718+0000","last_change":"2024-04-17T07:11:55.484259+0000","last_active":"2024-04-17T07:13:57.084718+0000","last_peered":"2024-04-17T07:13:57.084718+0000","last_clean":"2024-04-17T07:13:57.084718+0000","last_became_active":"2024-04-17T07:11:55.483911+0000","last_became_peered":"2024-04-17T07:11:55.483911+0000","last_unstale":"2024-04-17T07:13:57.084718+0000","last_undegraded":"2024-04-17T07:13:57.084718+0000","last_fullsized":"2024-04-17T07:13:57.084718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:54.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204244+0000","last_change":"2024-04-17T07:11:59.525369+0000","last_active":"2024-04-17T07:13:29.204244+0000","last_peered":"2024-04-17T07:13:29.204244+0000","last_clean":"2024-04-17T07:13:29.204244+0000","last_became_active":"2024-04-17T07:11:59.524809+0000","last_became_peered":"2024-04-17T07:11:59.524809+0000","last_unstale":"2024-04-17T07:13:29.204244+0000","last_undegraded":"2024-04-17T07:13:29.204244+0000","last_fullsized":"2024-04-17T07:13:29.204244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:20.511001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'7","reported_seq":198,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:04.678820+0000","last_change":"2024-04-17T07:11:54.459079+0000","last_active":"2024-04-17T07:15:04.678820+0000","last_peered":"2024-04-17T07:15:04.678820+0000","last_clean":"2024-04-17T07:15:04.678820+0000","last_became_active":"2024-04-17T07:11:54.457008+0000","last_became_peered":"2024-04-17T07:11:54.457008+0000","last_unstale":"2024-04-17T07:15:04.678820+0000","last_undegraded":"2024-04-17T07:15:04.678820+0000","last_fullsized":"2024-04-17T07:15:04.678820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:36.084512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":159,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084193+0000","last_change":"2024-04-17T07:11:55.484453+0000","last_active":"2024-04-17T07:13:57.084193+0000","last_peered":"2024-04-17T07:13:57.084193+0000","last_clean":"2024-04-17T07:13:57.084193+0000","last_became_active":"2024-04-17T07:11:55.483152+0000","last_became_peered":"2024-04-17T07:11:55.483152+0000","last_unstale":"2024-04-17T07:13:57.084193+0000","last_undegraded":"2024-04-17T07:13:57.084193+0000","last_fullsized":"2024-04-17T07:13:57.084193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:32.224868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.202670+0000","last_change":"2024-04-17T07:11:59.521487+0000","last_active":"2024-04-17T07:13:29.202670+0000","last_peered":"2024-04-17T07:13:29.202670+0000","last_clean":"2024-04-17T07:13:29.202670+0000","last_became_active":"2024-04-17T07:11:59.521060+0000","last_became_peered":"2024-04-17T07:11:59.521060+0000","last_unstale":"2024-04-17T07:13:29.202670+0000","last_undegraded":"2024-04-17T07:13:29.202670+0000","last_fullsized":"2024-04-17T07:13:29.202670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:06.692434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174602+0000","last_change":"2024-04-17T07:11:54.459368+0000","last_active":"2024-04-17T07:13:29.174602+0000","last_peered":"2024-04-17T07:13:29.174602+0000","last_clean":"2024-04-17T07:13:29.174602+0000","last_became_active":"2024-04-17T07:11:54.458973+0000","last_became_peered":"2024-04-17T07:11:54.458973+0000","last_unstale":"2024-04-17T07:13:29.174602+0000","last_undegraded":"2024-04-17T07:13:29.174602+0000","last_fullsized":"2024-04-17T07:13:29.174602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:22.034110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.072233+0000","last_change":"2024-04-17T07:11:55.470648+0000","last_active":"2024-04-17T07:13:57.072233+0000","last_peered":"2024-04-17T07:13:57.072233+0000","last_clean":"2024-04-17T07:13:57.072233+0000","last_became_active":"2024-04-17T07:11:55.469666+0000","last_became_peered":"2024-04-17T07:11:55.469666+0000","last_unstale":"2024-04-17T07:13:57.072233+0000","last_undegraded":"2024-04-17T07:13:57.072233+0000","last_fullsized":"2024-04-17T07:13:57.072233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:59.208872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172389+0000","last_change":"2024-04-17T07:11:59.524027+0000","last_active":"2024-04-17T07:13:29.172389+0000","last_peered":"2024-04-17T07:13:29.172389+0000","last_clean":"2024-04-17T07:13:29.172389+0000","last_became_active":"2024-04-17T07:11:59.522764+0000","last_became_peered":"2024-04-17T07:11:59.522764+0000","last_unstale":"2024-04-17T07:13:29.172389+0000","last_undegraded":"2024-04-17T07:13:29.172389+0000","last_fullsized":"2024-04-17T07:13:29.172389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:40.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173260+0000","last_change":"2024-04-17T07:11:54.472884+0000","last_active":"2024-04-17T07:13:29.173260+0000","last_peered":"2024-04-17T07:13:29.173260+0000","last_clean":"2024-04-17T07:13:29.173260+0000","last_became_active":"2024-04-17T07:11:54.472305+0000","last_became_peered":"2024-04-17T07:11:54.472305+0000","last_unstale":"2024-04-17T07:13:29.173260+0000","last_undegraded":"2024-04-17T07:13:29.173260+0000","last_fullsized":"2024-04-17T07:13:29.173260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:21.031442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070727+0000","last_change":"2024-04-17T07:11:55.479982+0000","last_active":"2024-04-17T07:13:57.070727+0000","last_peered":"2024-04-17T07:13:57.070727+0000","last_clean":"2024-04-17T07:13:57.070727+0000","last_became_active":"2024-04-17T07:11:55.477594+0000","last_became_peered":"2024-04-17T07:11:55.477594+0000","last_unstale":"2024-04-17T07:13:57.070727+0000","last_undegraded":"2024-04-17T07:13:57.070727+0000","last_fullsized":"2024-04-17T07:13:57.070727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:23.507312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071666+0000","last_change":"2024-04-17T07:11:55.480941+0000","last_active":"2024-04-17T07:13:57.071666+0000","last_peered":"2024-04-17T07:13:57.071666+0000","last_clean":"2024-04-17T07:13:57.071666+0000","last_became_active":"2024-04-17T07:11:55.480180+0000","last_became_peered":"2024-04-17T07:11:55.480180+0000","last_unstale":"2024-04-17T07:13:57.071666+0000","last_undegraded":"2024-04-17T07:13:57.071666+0000","last_fullsized":"2024-04-17T07:13:57.071666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:16.321806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071075+0000","last_change":"2024-04-17T07:11:55.477663+0000","last_active":"2024-04-17T07:13:57.071075+0000","last_peered":"2024-04-17T07:13:57.071075+0000","last_clean":"2024-04-17T07:13:57.071075+0000","last_became_active":"2024-04-17T07:11:55.477304+0000","last_became_peered":"2024-04-17T07:11:55.477304+0000","last_unstale":"2024-04-17T07:13:57.071075+0000","last_undegraded":"2024-04-17T07:13:57.071075+0000","last_fullsized":"2024-04-17T07:13:57.071075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:00.796633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077531+0000","last_change":"2024-04-17T07:11:55.475536+0000","last_active":"2024-04-17T07:13:57.077531+0000","last_peered":"2024-04-17T07:13:57.077531+0000","last_clean":"2024-04-17T07:13:57.077531+0000","last_became_active":"2024-04-17T07:11:55.475138+0000","last_became_peered":"2024-04-17T07:11:55.475138+0000","last_unstale":"2024-04-17T07:13:57.077531+0000","last_undegraded":"2024-04-17T07:13:57.077531+0000","last_fullsized":"2024-04-17T07:13:57.077531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:47.849076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081347+0000","last_change":"2024-04-17T07:11:55.483742+0000","last_active":"2024-04-17T07:13:57.081347+0000","last_peered":"2024-04-17T07:13:57.081347+0000","last_clean":"2024-04-17T07:13:57.081347+0000","last_became_active":"2024-04-17T07:11:55.483044+0000","last_became_peered":"2024-04-17T07:11:55.483044+0000","last_unstale":"2024-04-17T07:13:57.081347+0000","last_undegraded":"2024-04-17T07:13:57.081347+0000","last_fullsized":"2024-04-17T07:13:57.081347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:18.972137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078285+0000","last_change":"2024-04-17T07:11:55.471266+0000","last_active":"2024-04-17T07:13:57.078285+0000","last_peered":"2024-04-17T07:13:57.078285+0000","last_clean":"2024-04-17T07:13:57.078285+0000","last_became_active":"2024-04-17T07:11:55.470897+0000","last_became_peered":"2024-04-17T07:11:55.470897+0000","last_unstale":"2024-04-17T07:13:57.078285+0000","last_undegraded":"2024-04-17T07:13:57.078285+0000","last_fullsized":"2024-04-17T07:13:57.078285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:25.811064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.066947+0000","last_change":"2024-04-17T07:11:55.475860+0000","last_active":"2024-04-17T07:13:57.066947+0000","last_peered":"2024-04-17T07:13:57.066947+0000","last_clean":"2024-04-17T07:13:57.066947+0000","last_became_active":"2024-04-17T07:11:55.475522+0000","last_became_peered":"2024-04-17T07:11:55.475522+0000","last_unstale":"2024-04-17T07:13:57.066947+0000","last_undegraded":"2024-04-17T07:13:57.066947+0000","last_fullsized":"2024-04-17T07:13:57.066947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:41.548964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081296+0000","last_change":"2024-04-17T07:11:55.478815+0000","last_active":"2024-04-17T07:13:57.081296+0000","last_peered":"2024-04-17T07:13:57.081296+0000","last_clean":"2024-04-17T07:13:57.081296+0000","last_became_active":"2024-04-17T07:11:55.478347+0000","last_became_peered":"2024-04-17T07:11:55.478347+0000","last_unstale":"2024-04-17T07:13:57.081296+0000","last_undegraded":"2024-04-17T07:13:57.081296+0000","last_fullsized":"2024-04-17T07:13:57.081296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:16.465511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069021+0000","last_change":"2024-04-17T07:11:55.474208+0000","last_active":"2024-04-17T07:13:57.069021+0000","last_peered":"2024-04-17T07:13:57.069021+0000","last_clean":"2024-04-17T07:13:57.069021+0000","last_became_active":"2024-04-17T07:11:55.472042+0000","last_became_peered":"2024-04-17T07:11:55.472042+0000","last_unstale":"2024-04-17T07:13:57.069021+0000","last_undegraded":"2024-04-17T07:13:57.069021+0000","last_fullsized":"2024-04-17T07:13:57.069021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:37:24.248506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069995+0000","last_change":"2024-04-17T07:11:55.477257+0000","last_active":"2024-04-17T07:13:57.069995+0000","last_peered":"2024-04-17T07:13:57.069995+0000","last_clean":"2024-04-17T07:13:57.069995+0000","last_became_active":"2024-04-17T07:11:55.473491+0000","last_became_peered":"2024-04-17T07:11:55.473491+0000","last_unstale":"2024-04-17T07:13:57.069995+0000","last_undegraded":"2024-04-17T07:13:57.069995+0000","last_fullsized":"2024-04-17T07:13:57.069995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:33.737485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068915+0000","last_change":"2024-04-17T07:11:55.474930+0000","last_active":"2024-04-17T07:13:57.068915+0000","last_peered":"2024-04-17T07:13:57.068915+0000","last_clean":"2024-04-17T07:13:57.068915+0000","last_became_active":"2024-04-17T07:11:55.472007+0000","last_became_peered":"2024-04-17T07:11:55.472007+0000","last_unstale":"2024-04-17T07:13:57.068915+0000","last_undegraded":"2024-04-17T07:13:57.068915+0000","last_fullsized":"2024-04-17T07:13:57.068915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:07.111267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084510+0000","last_change":"2024-04-17T07:11:55.492422+0000","last_active":"2024-04-17T07:13:57.084510+0000","last_peered":"2024-04-17T07:13:57.084510+0000","last_clean":"2024-04-17T07:13:57.084510+0000","last_became_active":"2024-04-17T07:11:55.491906+0000","last_became_peered":"2024-04-17T07:11:55.491906+0000","last_unstale":"2024-04-17T07:13:57.084510+0000","last_undegraded":"2024-04-17T07:13:57.084510+0000","last_fullsized":"2024-04-17T07:13:57.084510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:13:05.322586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077939+0000","last_change":"2024-04-17T07:11:55.479436+0000","last_active":"2024-04-17T07:13:57.077939+0000","last_peered":"2024-04-17T07:13:57.077939+0000","last_clean":"2024-04-17T07:13:57.077939+0000","last_became_active":"2024-04-17T07:11:55.478982+0000","last_became_peered":"2024-04-17T07:11:55.478982+0000","last_unstale":"2024-04-17T07:13:57.077939+0000","last_undegraded":"2024-04-17T07:13:57.077939+0000","last_fullsized":"2024-04-17T07:13:57.077939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:20.663457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070325+0000","last_change":"2024-04-17T07:11:55.476708+0000","last_active":"2024-04-17T07:13:57.070325+0000","last_peered":"2024-04-17T07:13:57.070325+0000","last_clean":"2024-04-17T07:13:57.070325+0000","last_became_active":"2024-04-17T07:11:55.476344+0000","last_became_peered":"2024-04-17T07:11:55.476344+0000","last_unstale":"2024-04-17T07:13:57.070325+0000","last_undegraded":"2024-04-17T07:13:57.070325+0000","last_fullsized":"2024-04-17T07:13:57.070325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:58.874987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279575,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76685,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1576,"num_read_kb":862,"num_write":274,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":235095,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383734,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":521108,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645374,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":1174853,"data_compressed":64060,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808901,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":524154,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070540,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":520324,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332179,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1118505,"data_compressed":62211,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":516039,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":588652,"data_compressed":28497,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738499,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":517886,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":587010,"data_compressed":28497,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":515270,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920872,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":520113,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757345,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":1113133,"data_compressed":62211,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73812,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72170,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":69645,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:15:07.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:15:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.513+0000 7f4e64c00640 1 -- 172.21.15.86:0/2362854555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60076920 msgr2=0x7f4e60076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.513+0000 7f4e64c00640 1 --2- 172.21.15.86:0/2362854555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60076920 0x7f4e60076d40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009900 tx=0x7f4e5002bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:07.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.513+0000 7f4e64c00640 1 -- 172.21.15.86:0/2362854555 shutdown_connections 2024-04-17T07:15:07.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.513+0000 7f4e64c00640 1 --2- 172.21.15.86:0/2362854555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60078500 0x7f4e60078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.513+0000 7f4e64c00640 1 --2- 172.21.15.86:0/2362854555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e60077b60 0x7f4e60077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.513+0000 7f4e64c00640 1 --2- 172.21.15.86:0/2362854555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60076920 0x7f4e60076d40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.513+0000 7f4e64c00640 1 -- 172.21.15.86:0/2362854555 >> 172.21.15.86:0/2362854555 conn(0x7f4e600ff750 msgr2=0x7f4e60101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:07.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.513+0000 7f4e64c00640 1 -- 172.21.15.86:0/2362854555 shutdown_connections 2024-04-17T07:15:07.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.513+0000 7f4e64c00640 1 -- 172.21.15.86:0/2362854555 wait complete. 2024-04-17T07:15:07.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.514+0000 7f4e64c00640 1 Processor -- start 2024-04-17T07:15:07.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.514+0000 7f4e64c00640 1 -- start start 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.514+0000 7f4e64c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60076920 0x7f4e6019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.514+0000 7f4e64c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e60077b60 0x7f4e6019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e64c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60078500 0x7f4e601a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e64c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60114160 con 0x7f4e60076920 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e64c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60113fe0 con 0x7f4e60077b60 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e64c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601142e0 con 0x7f4e60078500 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e5de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60076920 0x7f4e6019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e5de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60076920 0x7f4e6019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37806/0 (socket says 172.21.15.86:37806) 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e5de00640 1 -- 172.21.15.86:0/3859339352 learned_addr learned my addr 172.21.15.86:0/3859339352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e57e00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e60077b60 0x7f4e6019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e5e800640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60078500 0x7f4e601a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e5de00640 1 -- 172.21.15.86:0/3859339352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60078500 msgr2=0x7f4e601a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e5de00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60078500 0x7f4e601a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e5de00640 1 -- 172.21.15.86:0/3859339352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e60077b60 msgr2=0x7f4e6019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e5de00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e60077b60 0x7f4e6019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.515+0000 7f4e5de00640 1 -- 172.21.15.86:0/3859339352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e601a63e0 con 0x7f4e60076920 2024-04-17T07:15:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.516+0000 7f4e5de00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60076920 0x7f4e6019e7c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4e50000c00 tx=0x7f4e500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.516+0000 7f4e56a00640 1 -- 172.21.15.86:0/3859339352 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50043020 con 0x7f4e60076920 2024-04-17T07:15:07.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.516+0000 7f4e56a00640 1 -- 172.21.15.86:0/3859339352 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e500054c0 con 0x7f4e60076920 2024-04-17T07:15:07.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.516+0000 7f4e64c00640 1 -- 172.21.15.86:0/3859339352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e601a6670 con 0x7f4e60076920 2024-04-17T07:15:07.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.516+0000 7f4e56a00640 1 -- 172.21.15.86:0/3859339352 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50003d70 con 0x7f4e60076920 2024-04-17T07:15:07.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.516+0000 7f4e64c00640 1 -- 172.21.15.86:0/3859339352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e601a6bc0 con 0x7f4e60076920 2024-04-17T07:15:07.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.517+0000 7f4e64c00640 1 -- 172.21.15.86:0/3859339352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e60110d30 con 0x7f4e60076920 2024-04-17T07:15:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.519+0000 7f4e56a00640 1 -- 172.21.15.86:0/3859339352 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f4e50046020 con 0x7f4e60076920 2024-04-17T07:15:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.520+0000 7f4e56a00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e400a3c40 0x7f4e400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.521+0000 7f4e56a00640 1 -- 172.21.15.86:0/3859339352 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4e500f2c20 con 0x7f4e60076920 2024-04-17T07:15:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.521+0000 7f4e57e00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e400a3c40 0x7f4e400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.521+0000 7f4e57e00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e400a3c40 0x7f4e400a6100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4e6019fcd0 tx=0x7f4e4800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:07.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.524+0000 7f4e56a00640 1 -- 172.21.15.86:0/3859339352 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4e5004d020 con 0x7f4e60076920 2024-04-17T07:15:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:07 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/317591058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:15:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:07 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2639390654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:15:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:07 smithi103 ceph-mon[35495]: pgmap v463: 129 pgs: 129 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-17T07:15:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:07 smithi103 ceph-mon[35495]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:15:07.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:07 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/317591058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:15:07.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:07 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2639390654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:15:07.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:07 smithi086 ceph-mon[31733]: pgmap v463: 129 pgs: 129 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-17T07:15:07.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:07 smithi086 ceph-mon[31733]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:15:07.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.726+0000 7f4e64c00640 1 -- 172.21.15.86:0/3859339352 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e600623a0 con 0x7f4e400a3c40 2024-04-17T07:15:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:07 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/317591058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:15:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:07 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2639390654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:15:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:07 smithi098 ceph-mon[35418]: pgmap v463: 129 pgs: 129 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-17T07:15:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:07 smithi098 ceph-mon[35418]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:15:07.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.755+0000 7f4e56a00640 1 -- 172.21.15.86:0/3859339352 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340881 (secure 0 0 0) 0x7f4e600623a0 con 0x7f4e400a3c40 2024-04-17T07:15:07.757 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-17T07:15:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 -- 172.21.15.86:0/3859339352 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e400a3c40 msgr2=0x7f4e400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e400a3c40 0x7f4e400a6100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4e6019fcd0 tx=0x7f4e4800b390 comp rx=0 tx=0).stop 2024-04-17T07:15:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 -- 172.21.15.86:0/3859339352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60076920 msgr2=0x7f4e6019e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60076920 0x7f4e6019e7c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4e50000c00 tx=0x7f4e500042f0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 -- 172.21.15.86:0/3859339352 shutdown_connections 2024-04-17T07:15:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e400a3c40 0x7f4e400a6100 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60078500 0x7f4e601a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e60077b60 0x7f4e6019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 --2- 172.21.15.86:0/3859339352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e60076920 0x7f4e6019e7c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 -- 172.21.15.86:0/3859339352 >> 172.21.15.86:0/3859339352 conn(0x7f4e600ff750 msgr2=0x7f4e60100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 -- 172.21.15.86:0/3859339352 shutdown_connections 2024-04-17T07:15:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:07.764+0000 7f4e64c00640 1 -- 172.21.15.86:0/3859339352 wait complete. 2024-04-17T07:15:07.785 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-04-17T07:15:07.131692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667287,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1702,"num_read_kb":971,"num_write":457,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181358,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":873,"ondisk_log_size":873,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365908,"kb_used_data":12596,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575916,"statfs":{"total":1151940427776,"available":1151565737984,"internally_reserved":0,"allocated":12898304,"data_stored":8217047,"data_compressed":432012,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7481,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":61,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005284"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078723+0000","last_change":"2024-04-17T07:11:55.486746+0000","last_active":"2024-04-17T07:13:57.078723+0000","last_peered":"2024-04-17T07:13:57.078723+0000","last_clean":"2024-04-17T07:13:57.078723+0000","last_became_active":"2024-04-17T07:11:55.486335+0000","last_became_peered":"2024-04-17T07:11:55.486335+0000","last_unstale":"2024-04-17T07:13:57.078723+0000","last_undegraded":"2024-04-17T07:13:57.078723+0000","last_fullsized":"2024-04-17T07:13:57.078723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:48.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082087+0000","last_change":"2024-04-17T07:11:55.481832+0000","last_active":"2024-04-17T07:13:57.082087+0000","last_peered":"2024-04-17T07:13:57.082087+0000","last_clean":"2024-04-17T07:13:57.082087+0000","last_became_active":"2024-04-17T07:11:55.476428+0000","last_became_peered":"2024-04-17T07:11:55.476428+0000","last_unstale":"2024-04-17T07:13:57.082087+0000","last_undegraded":"2024-04-17T07:13:57.082087+0000","last_fullsized":"2024-04-17T07:13:57.082087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:04.452542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083395+0000","last_change":"2024-04-17T07:11:55.483721+0000","last_active":"2024-04-17T07:13:57.083395+0000","last_peered":"2024-04-17T07:13:57.083395+0000","last_clean":"2024-04-17T07:13:57.083395+0000","last_became_active":"2024-04-17T07:11:55.483268+0000","last_became_peered":"2024-04-17T07:11:55.483268+0000","last_unstale":"2024-04-17T07:13:57.083395+0000","last_undegraded":"2024-04-17T07:13:57.083395+0000","last_fullsized":"2024-04-17T07:13:57.083395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:41.584783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067540+0000","last_change":"2024-04-17T07:11:55.485114+0000","last_active":"2024-04-17T07:13:57.067540+0000","last_peered":"2024-04-17T07:13:57.067540+0000","last_clean":"2024-04-17T07:13:57.067540+0000","last_became_active":"2024-04-17T07:11:55.483913+0000","last_became_peered":"2024-04-17T07:11:55.483913+0000","last_unstale":"2024-04-17T07:13:57.067540+0000","last_undegraded":"2024-04-17T07:13:57.067540+0000","last_fullsized":"2024-04-17T07:13:57.067540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:46.211121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.060857+0000","last_change":"2024-04-17T07:11:55.491952+0000","last_active":"2024-04-17T07:13:57.060857+0000","last_peered":"2024-04-17T07:13:57.060857+0000","last_clean":"2024-04-17T07:13:57.060857+0000","last_became_active":"2024-04-17T07:11:55.491238+0000","last_became_peered":"2024-04-17T07:11:55.491238+0000","last_unstale":"2024-04-17T07:13:57.060857+0000","last_undegraded":"2024-04-17T07:13:57.060857+0000","last_fullsized":"2024-04-17T07:13:57.060857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:06.759797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068819+0000","last_change":"2024-04-17T07:11:55.471124+0000","last_active":"2024-04-17T07:13:57.068819+0000","last_peered":"2024-04-17T07:13:57.068819+0000","last_clean":"2024-04-17T07:13:57.068819+0000","last_became_active":"2024-04-17T07:11:55.470781+0000","last_became_peered":"2024-04-17T07:11:55.470781+0000","last_unstale":"2024-04-17T07:13:57.068819+0000","last_undegraded":"2024-04-17T07:13:57.068819+0000","last_fullsized":"2024-04-17T07:13:57.068819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:45.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.072320+0000","last_change":"2024-04-17T07:11:55.469099+0000","last_active":"2024-04-17T07:13:57.072320+0000","last_peered":"2024-04-17T07:13:57.072320+0000","last_clean":"2024-04-17T07:13:57.072320+0000","last_became_active":"2024-04-17T07:11:55.468584+0000","last_became_peered":"2024-04-17T07:11:55.468584+0000","last_unstale":"2024-04-17T07:13:57.072320+0000","last_undegraded":"2024-04-17T07:13:57.072320+0000","last_fullsized":"2024-04-17T07:13:57.072320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:52.834846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071967+0000","last_change":"2024-04-17T07:11:55.473794+0000","last_active":"2024-04-17T07:13:57.071967+0000","last_peered":"2024-04-17T07:13:57.071967+0000","last_clean":"2024-04-17T07:13:57.071967+0000","last_became_active":"2024-04-17T07:11:55.471919+0000","last_became_peered":"2024-04-17T07:11:55.471919+0000","last_unstale":"2024-04-17T07:13:57.071967+0000","last_undegraded":"2024-04-17T07:13:57.071967+0000","last_fullsized":"2024-04-17T07:13:57.071967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:16.085117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.079281+0000","last_change":"2024-04-17T07:11:55.482445+0000","last_active":"2024-04-17T07:13:57.079281+0000","last_peered":"2024-04-17T07:13:57.079281+0000","last_clean":"2024-04-17T07:13:57.079281+0000","last_became_active":"2024-04-17T07:11:55.481552+0000","last_became_peered":"2024-04-17T07:11:55.481552+0000","last_unstale":"2024-04-17T07:13:57.079281+0000","last_undegraded":"2024-04-17T07:13:57.079281+0000","last_fullsized":"2024-04-17T07:13:57.079281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:48.733585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071322+0000","last_change":"2024-04-17T07:11:55.476726+0000","last_active":"2024-04-17T07:13:57.071322+0000","last_peered":"2024-04-17T07:13:57.071322+0000","last_clean":"2024-04-17T07:13:57.071322+0000","last_became_active":"2024-04-17T07:11:55.476320+0000","last_became_peered":"2024-04-17T07:11:55.476320+0000","last_unstale":"2024-04-17T07:13:57.071322+0000","last_undegraded":"2024-04-17T07:13:57.071322+0000","last_fullsized":"2024-04-17T07:13:57.071322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.230181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084097+0000","last_change":"2024-04-17T07:11:55.485798+0000","last_active":"2024-04-17T07:13:57.084097+0000","last_peered":"2024-04-17T07:13:57.084097+0000","last_clean":"2024-04-17T07:13:57.084097+0000","last_became_active":"2024-04-17T07:11:55.484015+0000","last_became_peered":"2024-04-17T07:11:55.484015+0000","last_unstale":"2024-04-17T07:13:57.084097+0000","last_undegraded":"2024-04-17T07:13:57.084097+0000","last_fullsized":"2024-04-17T07:13:57.084097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:37.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.062297+0000","last_change":"2024-04-17T07:12:28.729834+0000","last_active":"2024-04-17T07:13:57.062297+0000","last_peered":"2024-04-17T07:13:57.062297+0000","last_clean":"2024-04-17T07:13:57.062297+0000","last_became_active":"2024-04-17T07:12:28.728694+0000","last_became_peered":"2024-04-17T07:12:28.728694+0000","last_unstale":"2024-04-17T07:13:57.062297+0000","last_undegraded":"2024-04-17T07:13:57.062297+0000","last_fullsized":"2024-04-17T07:13:57.062297+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:46.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067980+0000","last_change":"2024-04-17T07:13:29.210325+0000","last_active":"2024-04-17T07:13:57.067980+0000","last_peered":"2024-04-17T07:13:57.067980+0000","last_clean":"2024-04-17T07:13:57.067980+0000","last_became_active":"2024-04-17T07:13:29.209759+0000","last_became_peered":"2024-04-17T07:13:29.209759+0000","last_unstale":"2024-04-17T07:13:57.067980+0000","last_undegraded":"2024-04-17T07:13:57.067980+0000","last_fullsized":"2024-04-17T07:13:57.067980+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:17.168931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083596+0000","last_change":"2024-04-17T07:11:55.473839+0000","last_active":"2024-04-17T07:13:57.083596+0000","last_peered":"2024-04-17T07:13:57.083596+0000","last_clean":"2024-04-17T07:13:57.083596+0000","last_became_active":"2024-04-17T07:11:55.473472+0000","last_became_peered":"2024-04-17T07:11:55.473472+0000","last_unstale":"2024-04-17T07:13:57.083596+0000","last_undegraded":"2024-04-17T07:13:57.083596+0000","last_fullsized":"2024-04-17T07:13:57.083596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:02.263272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081481+0000","last_change":"2024-04-17T07:11:55.484820+0000","last_active":"2024-04-17T07:13:57.081481+0000","last_peered":"2024-04-17T07:13:57.081481+0000","last_clean":"2024-04-17T07:13:57.081481+0000","last_became_active":"2024-04-17T07:11:55.484314+0000","last_became_peered":"2024-04-17T07:11:55.484314+0000","last_unstale":"2024-04-17T07:13:57.081481+0000","last_undegraded":"2024-04-17T07:13:57.081481+0000","last_fullsized":"2024-04-17T07:13:57.081481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.186912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084394+0000","last_change":"2024-04-17T07:11:55.492018+0000","last_active":"2024-04-17T07:13:57.084394+0000","last_peered":"2024-04-17T07:13:57.084394+0000","last_clean":"2024-04-17T07:13:57.084394+0000","last_became_active":"2024-04-17T07:11:55.490512+0000","last_became_peered":"2024-04-17T07:11:55.490512+0000","last_unstale":"2024-04-17T07:13:57.084394+0000","last_undegraded":"2024-04-17T07:13:57.084394+0000","last_fullsized":"2024-04-17T07:13:57.084394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:45.436270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081460+0000","last_change":"2024-04-17T07:11:55.487210+0000","last_active":"2024-04-17T07:13:57.081460+0000","last_peered":"2024-04-17T07:13:57.081460+0000","last_clean":"2024-04-17T07:13:57.081460+0000","last_became_active":"2024-04-17T07:11:55.485713+0000","last_became_peered":"2024-04-17T07:11:55.485713+0000","last_unstale":"2024-04-17T07:13:57.081460+0000","last_undegraded":"2024-04-17T07:13:57.081460+0000","last_fullsized":"2024-04-17T07:13:57.081460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:54.495950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082817+0000","last_change":"2024-04-17T07:11:55.482705+0000","last_active":"2024-04-17T07:13:57.082817+0000","last_peered":"2024-04-17T07:13:57.082817+0000","last_clean":"2024-04-17T07:13:57.082817+0000","last_became_active":"2024-04-17T07:11:55.482349+0000","last_became_peered":"2024-04-17T07:11:55.482349+0000","last_unstale":"2024-04-17T07:13:57.082817+0000","last_undegraded":"2024-04-17T07:13:57.082817+0000","last_fullsized":"2024-04-17T07:13:57.082817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:57.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069422+0000","last_change":"2024-04-17T07:11:55.467065+0000","last_active":"2024-04-17T07:13:57.069422+0000","last_peered":"2024-04-17T07:13:57.069422+0000","last_clean":"2024-04-17T07:13:57.069422+0000","last_became_active":"2024-04-17T07:11:55.466535+0000","last_became_peered":"2024-04-17T07:11:55.466535+0000","last_unstale":"2024-04-17T07:13:57.069422+0000","last_undegraded":"2024-04-17T07:13:57.069422+0000","last_fullsized":"2024-04-17T07:13:57.069422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:49.955453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204394+0000","last_change":"2024-04-17T07:11:59.515778+0000","last_active":"2024-04-17T07:13:29.204394+0000","last_peered":"2024-04-17T07:13:29.204394+0000","last_clean":"2024-04-17T07:13:29.204394+0000","last_became_active":"2024-04-17T07:11:59.515266+0000","last_became_peered":"2024-04-17T07:11:59.515266+0000","last_unstale":"2024-04-17T07:13:29.204394+0000","last_undegraded":"2024-04-17T07:13:29.204394+0000","last_fullsized":"2024-04-17T07:13:29.204394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:43.084724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":203,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:05.648744+0000","last_change":"2024-04-17T07:11:54.467060+0000","last_active":"2024-04-17T07:15:05.648744+0000","last_peered":"2024-04-17T07:15:05.648744+0000","last_clean":"2024-04-17T07:15:05.648744+0000","last_became_active":"2024-04-17T07:11:54.466687+0000","last_became_peered":"2024-04-17T07:11:54.466687+0000","last_unstale":"2024-04-17T07:15:05.648744+0000","last_undegraded":"2024-04-17T07:15:05.648744+0000","last_fullsized":"2024-04-17T07:15:05.648744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:57.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":157,"num_read_kb":157,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.064498+0000","last_change":"2024-04-17T07:11:55.469857+0000","last_active":"2024-04-17T07:13:57.064498+0000","last_peered":"2024-04-17T07:13:57.064498+0000","last_clean":"2024-04-17T07:13:57.064498+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:13:57.064498+0000","last_undegraded":"2024-04-17T07:13:57.064498+0000","last_fullsized":"2024-04-17T07:13:57.064498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:43.565627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174451+0000","last_change":"2024-04-17T07:12:29.125397+0000","last_active":"2024-04-17T07:13:29.174451+0000","last_peered":"2024-04-17T07:13:29.174451+0000","last_clean":"2024-04-17T07:13:29.174451+0000","last_became_active":"2024-04-17T07:12:29.124158+0000","last_became_peered":"2024-04-17T07:12:29.124158+0000","last_unstale":"2024-04-17T07:13:29.174451+0000","last_undegraded":"2024-04-17T07:13:29.174451+0000","last_fullsized":"2024-04-17T07:13:29.174451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:24.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205266+0000","last_change":"2024-04-17T07:11:54.464715+0000","last_active":"2024-04-17T07:13:29.205266+0000","last_peered":"2024-04-17T07:13:29.205266+0000","last_clean":"2024-04-17T07:13:29.205266+0000","last_became_active":"2024-04-17T07:11:54.463917+0000","last_became_peered":"2024-04-17T07:11:54.463917+0000","last_unstale":"2024-04-17T07:13:29.205266+0000","last_undegraded":"2024-04-17T07:13:29.205266+0000","last_fullsized":"2024-04-17T07:13:29.205266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:43.738471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.085059+0000","last_change":"2024-04-17T07:11:55.482876+0000","last_active":"2024-04-17T07:13:57.085059+0000","last_peered":"2024-04-17T07:13:57.085059+0000","last_clean":"2024-04-17T07:13:57.085059+0000","last_became_active":"2024-04-17T07:11:55.482451+0000","last_became_peered":"2024-04-17T07:11:55.482451+0000","last_unstale":"2024-04-17T07:13:57.085059+0000","last_undegraded":"2024-04-17T07:13:57.085059+0000","last_fullsized":"2024-04-17T07:13:57.085059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:39.394090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174912+0000","last_change":"2024-04-17T07:11:59.522634+0000","last_active":"2024-04-17T07:13:29.174912+0000","last_peered":"2024-04-17T07:13:29.174912+0000","last_clean":"2024-04-17T07:13:29.174912+0000","last_became_active":"2024-04-17T07:11:59.520797+0000","last_became_peered":"2024-04-17T07:11:59.520797+0000","last_unstale":"2024-04-17T07:13:29.174912+0000","last_undegraded":"2024-04-17T07:13:29.174912+0000","last_fullsized":"2024-04-17T07:13:29.174912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:02.316515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.334212+0000","last_change":"2024-04-17T07:12:28.756528+0000","last_active":"2024-04-17T07:13:29.334212+0000","last_peered":"2024-04-17T07:13:29.334212+0000","last_clean":"2024-04-17T07:13:29.334212+0000","last_became_active":"2024-04-17T07:12:28.737008+0000","last_became_peered":"2024-04-17T07:12:28.737008+0000","last_unstale":"2024-04-17T07:13:29.334212+0000","last_undegraded":"2024-04-17T07:13:29.334212+0000","last_fullsized":"2024-04-17T07:13:29.334212+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:56:26.719377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083022+0000","last_change":"2024-04-17T07:11:55.485999+0000","last_active":"2024-04-17T07:13:57.083022+0000","last_peered":"2024-04-17T07:13:57.083022+0000","last_clean":"2024-04-17T07:13:57.083022+0000","last_became_active":"2024-04-17T07:11:55.484048+0000","last_became_peered":"2024-04-17T07:11:55.484048+0000","last_unstale":"2024-04-17T07:13:57.083022+0000","last_undegraded":"2024-04-17T07:13:57.083022+0000","last_fullsized":"2024-04-17T07:13:57.083022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:38.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174422+0000","last_change":"2024-04-17T07:11:59.529929+0000","last_active":"2024-04-17T07:13:29.174422+0000","last_peered":"2024-04-17T07:13:29.174422+0000","last_clean":"2024-04-17T07:13:29.174422+0000","last_became_active":"2024-04-17T07:11:59.529400+0000","last_became_peered":"2024-04-17T07:11:59.529400+0000","last_unstale":"2024-04-17T07:13:29.174422+0000","last_undegraded":"2024-04-17T07:13:29.174422+0000","last_fullsized":"2024-04-17T07:13:29.174422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:54.845529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204898+0000","last_change":"2024-04-17T07:11:54.462852+0000","last_active":"2024-04-17T07:13:29.204898+0000","last_peered":"2024-04-17T07:13:29.204898+0000","last_clean":"2024-04-17T07:13:29.204898+0000","last_became_active":"2024-04-17T07:11:54.459178+0000","last_became_peered":"2024-04-17T07:11:54.459178+0000","last_unstale":"2024-04-17T07:13:29.204898+0000","last_undegraded":"2024-04-17T07:13:29.204898+0000","last_fullsized":"2024-04-17T07:13:29.204898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:07.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.075256+0000","last_change":"2024-04-17T07:11:55.480951+0000","last_active":"2024-04-17T07:13:57.075256+0000","last_peered":"2024-04-17T07:13:57.075256+0000","last_clean":"2024-04-17T07:13:57.075256+0000","last_became_active":"2024-04-17T07:11:55.480507+0000","last_became_peered":"2024-04-17T07:11:55.480507+0000","last_unstale":"2024-04-17T07:13:57.075256+0000","last_undegraded":"2024-04-17T07:13:57.075256+0000","last_fullsized":"2024-04-17T07:13:57.075256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:42.584180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205487+0000","last_change":"2024-04-17T07:11:59.520897+0000","last_active":"2024-04-17T07:13:29.205487+0000","last_peered":"2024-04-17T07:13:29.205487+0000","last_clean":"2024-04-17T07:13:29.205487+0000","last_became_active":"2024-04-17T07:11:59.520350+0000","last_became_peered":"2024-04-17T07:11:59.520350+0000","last_unstale":"2024-04-17T07:13:29.205487+0000","last_undegraded":"2024-04-17T07:13:29.205487+0000","last_fullsized":"2024-04-17T07:13:29.205487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:39.813097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":212,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:05.699123+0000","last_change":"2024-04-17T07:12:28.753717+0000","last_active":"2024-04-17T07:15:05.699123+0000","last_peered":"2024-04-17T07:15:05.699123+0000","last_clean":"2024-04-17T07:15:05.699123+0000","last_became_active":"2024-04-17T07:12:28.739556+0000","last_became_peered":"2024-04-17T07:12:28.739556+0000","last_unstale":"2024-04-17T07:15:05.699123+0000","last_undegraded":"2024-04-17T07:15:05.699123+0000","last_fullsized":"2024-04-17T07:15:05.699123+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:11.967731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1631,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":160,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.055509+0000","last_change":"2024-04-17T07:11:55.476675+0000","last_active":"2024-04-17T07:13:57.055509+0000","last_peered":"2024-04-17T07:13:57.055509+0000","last_clean":"2024-04-17T07:13:57.055509+0000","last_became_active":"2024-04-17T07:11:55.476273+0000","last_became_peered":"2024-04-17T07:11:55.476273+0000","last_unstale":"2024-04-17T07:13:57.055509+0000","last_undegraded":"2024-04-17T07:13:57.055509+0000","last_fullsized":"2024-04-17T07:13:57.055509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:09.612138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205617+0000","last_change":"2024-04-17T07:12:28.741537+0000","last_active":"2024-04-17T07:13:29.205617+0000","last_peered":"2024-04-17T07:13:29.205617+0000","last_clean":"2024-04-17T07:13:29.205617+0000","last_became_active":"2024-04-17T07:12:28.741035+0000","last_became_peered":"2024-04-17T07:12:28.741035+0000","last_unstale":"2024-04-17T07:13:29.205617+0000","last_undegraded":"2024-04-17T07:13:29.205617+0000","last_fullsized":"2024-04-17T07:13:29.205617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:15.550112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.207986+0000","last_change":"2024-04-17T07:11:54.471118+0000","last_active":"2024-04-17T07:13:29.207986+0000","last_peered":"2024-04-17T07:13:29.207986+0000","last_clean":"2024-04-17T07:13:29.207986+0000","last_became_active":"2024-04-17T07:11:54.470526+0000","last_became_peered":"2024-04-17T07:11:54.470526+0000","last_unstale":"2024-04-17T07:13:29.207986+0000","last_undegraded":"2024-04-17T07:13:29.207986+0000","last_fullsized":"2024-04-17T07:13:29.207986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:52:47.918086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071367+0000","last_change":"2024-04-17T07:11:55.472850+0000","last_active":"2024-04-17T07:13:57.071367+0000","last_peered":"2024-04-17T07:13:57.071367+0000","last_clean":"2024-04-17T07:13:57.071367+0000","last_became_active":"2024-04-17T07:11:55.472097+0000","last_became_peered":"2024-04-17T07:11:55.472097+0000","last_unstale":"2024-04-17T07:13:57.071367+0000","last_undegraded":"2024-04-17T07:13:57.071367+0000","last_fullsized":"2024-04-17T07:13:57.071367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:41.730809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.334157+0000","last_change":"2024-04-17T07:11:59.522960+0000","last_active":"2024-04-17T07:13:29.334157+0000","last_peered":"2024-04-17T07:13:29.334157+0000","last_clean":"2024-04-17T07:13:29.334157+0000","last_became_active":"2024-04-17T07:11:59.522470+0000","last_became_peered":"2024-04-17T07:11:59.522470+0000","last_unstale":"2024-04-17T07:13:29.334157+0000","last_undegraded":"2024-04-17T07:13:29.334157+0000","last_fullsized":"2024-04-17T07:13:29.334157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:36.620442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.171736+0000","last_change":"2024-04-17T07:11:54.470693+0000","last_active":"2024-04-17T07:13:29.171736+0000","last_peered":"2024-04-17T07:13:29.171736+0000","last_clean":"2024-04-17T07:13:29.171736+0000","last_became_active":"2024-04-17T07:11:54.470334+0000","last_became_peered":"2024-04-17T07:11:54.470334+0000","last_unstale":"2024-04-17T07:13:29.171736+0000","last_undegraded":"2024-04-17T07:13:29.171736+0000","last_fullsized":"2024-04-17T07:13:29.171736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:19:04.980786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070659+0000","last_change":"2024-04-17T07:11:55.474755+0000","last_active":"2024-04-17T07:13:57.070659+0000","last_peered":"2024-04-17T07:13:57.070659+0000","last_clean":"2024-04-17T07:13:57.070659+0000","last_became_active":"2024-04-17T07:11:55.474261+0000","last_became_peered":"2024-04-17T07:11:55.474261+0000","last_unstale":"2024-04-17T07:13:57.070659+0000","last_undegraded":"2024-04-17T07:13:57.070659+0000","last_fullsized":"2024-04-17T07:13:57.070659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:00.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203208+0000","last_change":"2024-04-17T07:11:59.517152+0000","last_active":"2024-04-17T07:13:29.203208+0000","last_peered":"2024-04-17T07:13:29.203208+0000","last_clean":"2024-04-17T07:13:29.203208+0000","last_became_active":"2024-04-17T07:11:59.514977+0000","last_became_peered":"2024-04-17T07:11:59.514977+0000","last_unstale":"2024-04-17T07:13:29.203208+0000","last_undegraded":"2024-04-17T07:13:29.203208+0000","last_fullsized":"2024-04-17T07:13:29.203208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:14.974136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172963+0000","last_change":"2024-04-17T07:11:54.459786+0000","last_active":"2024-04-17T07:13:29.172963+0000","last_peered":"2024-04-17T07:13:29.172963+0000","last_clean":"2024-04-17T07:13:29.172963+0000","last_became_active":"2024-04-17T07:11:54.457589+0000","last_became_peered":"2024-04-17T07:11:54.457589+0000","last_unstale":"2024-04-17T07:13:29.172963+0000","last_undegraded":"2024-04-17T07:13:29.172963+0000","last_fullsized":"2024-04-17T07:13:29.172963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:25.986539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068150+0000","last_change":"2024-04-17T07:11:55.465020+0000","last_active":"2024-04-17T07:13:57.068150+0000","last_peered":"2024-04-17T07:13:57.068150+0000","last_clean":"2024-04-17T07:13:57.068150+0000","last_became_active":"2024-04-17T07:11:55.464608+0000","last_became_peered":"2024-04-17T07:11:55.464608+0000","last_unstale":"2024-04-17T07:13:57.068150+0000","last_undegraded":"2024-04-17T07:13:57.068150+0000","last_fullsized":"2024-04-17T07:13:57.068150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204578+0000","last_change":"2024-04-17T07:11:59.519333+0000","last_active":"2024-04-17T07:13:29.204578+0000","last_peered":"2024-04-17T07:13:29.204578+0000","last_clean":"2024-04-17T07:13:29.204578+0000","last_became_active":"2024-04-17T07:11:59.518865+0000","last_became_peered":"2024-04-17T07:11:59.518865+0000","last_unstale":"2024-04-17T07:13:29.204578+0000","last_undegraded":"2024-04-17T07:13:29.204578+0000","last_fullsized":"2024-04-17T07:13:29.204578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:42.009010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":341,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:01.666653+0000","last_change":"2024-04-17T07:11:54.462283+0000","last_active":"2024-04-17T07:15:01.666653+0000","last_peered":"2024-04-17T07:15:01.666653+0000","last_clean":"2024-04-17T07:15:01.666653+0000","last_became_active":"2024-04-17T07:11:54.461818+0000","last_became_peered":"2024-04-17T07:11:54.461818+0000","last_unstale":"2024-04-17T07:15:01.666653+0000","last_undegraded":"2024-04-17T07:15:01.666653+0000","last_fullsized":"2024-04-17T07:15:01.666653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.765058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":302,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081011+0000","last_change":"2024-04-17T07:11:55.486491+0000","last_active":"2024-04-17T07:13:57.081011+0000","last_peered":"2024-04-17T07:13:57.081011+0000","last_clean":"2024-04-17T07:13:57.081011+0000","last_became_active":"2024-04-17T07:11:55.486121+0000","last_became_peered":"2024-04-17T07:11:55.486121+0000","last_unstale":"2024-04-17T07:13:57.081011+0000","last_undegraded":"2024-04-17T07:13:57.081011+0000","last_fullsized":"2024-04-17T07:13:57.081011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:12.998321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205968+0000","last_change":"2024-04-17T07:11:59.511555+0000","last_active":"2024-04-17T07:13:29.205968+0000","last_peered":"2024-04-17T07:13:29.205968+0000","last_clean":"2024-04-17T07:13:29.205968+0000","last_became_active":"2024-04-17T07:11:59.511147+0000","last_became_peered":"2024-04-17T07:11:59.511147+0000","last_unstale":"2024-04-17T07:13:29.205968+0000","last_undegraded":"2024-04-17T07:13:29.205968+0000","last_fullsized":"2024-04-17T07:13:29.205968+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:41.854815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209570+0000","last_change":"2024-04-17T07:12:28.734476+0000","last_active":"2024-04-17T07:13:29.209570+0000","last_peered":"2024-04-17T07:13:29.209570+0000","last_clean":"2024-04-17T07:13:29.209570+0000","last_became_active":"2024-04-17T07:12:28.728187+0000","last_became_peered":"2024-04-17T07:12:28.728187+0000","last_unstale":"2024-04-17T07:13:29.209570+0000","last_undegraded":"2024-04-17T07:13:29.209570+0000","last_fullsized":"2024-04-17T07:13:29.209570+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:45.749607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081552+0000","last_change":"2024-04-17T07:11:55.480589+0000","last_active":"2024-04-17T07:13:57.081552+0000","last_peered":"2024-04-17T07:13:57.081552+0000","last_clean":"2024-04-17T07:13:57.081552+0000","last_became_active":"2024-04-17T07:11:55.480224+0000","last_became_peered":"2024-04-17T07:11:55.480224+0000","last_unstale":"2024-04-17T07:13:57.081552+0000","last_undegraded":"2024-04-17T07:13:57.081552+0000","last_fullsized":"2024-04-17T07:13:57.081552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:05.892442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175186+0000","last_change":"2024-04-17T07:11:59.522021+0000","last_active":"2024-04-17T07:13:29.175186+0000","last_peered":"2024-04-17T07:13:29.175186+0000","last_clean":"2024-04-17T07:13:29.175186+0000","last_became_active":"2024-04-17T07:11:59.521493+0000","last_became_peered":"2024-04-17T07:11:59.521493+0000","last_unstale":"2024-04-17T07:13:29.175186+0000","last_undegraded":"2024-04-17T07:13:29.175186+0000","last_fullsized":"2024-04-17T07:13:29.175186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:20.386216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174471+0000","last_change":"2024-04-17T07:11:54.468225+0000","last_active":"2024-04-17T07:13:29.174471+0000","last_peered":"2024-04-17T07:13:29.174471+0000","last_clean":"2024-04-17T07:13:29.174471+0000","last_became_active":"2024-04-17T07:11:54.467046+0000","last_became_peered":"2024-04-17T07:11:54.467046+0000","last_unstale":"2024-04-17T07:13:29.174471+0000","last_undegraded":"2024-04-17T07:13:29.174471+0000","last_fullsized":"2024-04-17T07:13:29.174471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:17.725255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083563+0000","last_change":"2024-04-17T07:11:55.488980+0000","last_active":"2024-04-17T07:13:57.083563+0000","last_peered":"2024-04-17T07:13:57.083563+0000","last_clean":"2024-04-17T07:13:57.083563+0000","last_became_active":"2024-04-17T07:11:55.488226+0000","last_became_peered":"2024-04-17T07:11:55.488226+0000","last_unstale":"2024-04-17T07:13:57.083563+0000","last_undegraded":"2024-04-17T07:13:57.083563+0000","last_fullsized":"2024-04-17T07:13:57.083563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:00.135973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205610+0000","last_change":"2024-04-17T07:11:59.515134+0000","last_active":"2024-04-17T07:13:29.205610+0000","last_peered":"2024-04-17T07:13:29.205610+0000","last_clean":"2024-04-17T07:13:29.205610+0000","last_became_active":"2024-04-17T07:11:59.514560+0000","last_became_peered":"2024-04-17T07:11:59.514560+0000","last_unstale":"2024-04-17T07:13:29.205610+0000","last_undegraded":"2024-04-17T07:13:29.205610+0000","last_fullsized":"2024-04-17T07:13:29.205610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:34.669789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":381,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:01.667846+0000","last_change":"2024-04-17T07:11:54.461226+0000","last_active":"2024-04-17T07:15:01.667846+0000","last_peered":"2024-04-17T07:15:01.667846+0000","last_clean":"2024-04-17T07:15:01.667846+0000","last_became_active":"2024-04-17T07:11:54.460573+0000","last_became_peered":"2024-04-17T07:11:54.460573+0000","last_unstale":"2024-04-17T07:15:01.667846+0000","last_undegraded":"2024-04-17T07:15:01.667846+0000","last_fullsized":"2024-04-17T07:15:01.667846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67432,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":310,"num_read_kb":216,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068452+0000","last_change":"2024-04-17T07:11:55.483925+0000","last_active":"2024-04-17T07:13:57.068452+0000","last_peered":"2024-04-17T07:13:57.068452+0000","last_clean":"2024-04-17T07:13:57.068452+0000","last_became_active":"2024-04-17T07:11:55.483584+0000","last_became_peered":"2024-04-17T07:11:55.483584+0000","last_unstale":"2024-04-17T07:13:57.068452+0000","last_undegraded":"2024-04-17T07:13:57.068452+0000","last_fullsized":"2024-04-17T07:13:57.068452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:12.328876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174981+0000","last_change":"2024-04-17T07:11:59.515597+0000","last_active":"2024-04-17T07:13:29.174981+0000","last_peered":"2024-04-17T07:13:29.174981+0000","last_clean":"2024-04-17T07:13:29.174981+0000","last_became_active":"2024-04-17T07:11:59.514413+0000","last_became_peered":"2024-04-17T07:11:59.514413+0000","last_unstale":"2024-04-17T07:13:29.174981+0000","last_undegraded":"2024-04-17T07:13:29.174981+0000","last_fullsized":"2024-04-17T07:13:29.174981+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:45.824127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205257+0000","last_change":"2024-04-17T07:11:54.449984+0000","last_active":"2024-04-17T07:13:29.205257+0000","last_peered":"2024-04-17T07:13:29.205257+0000","last_clean":"2024-04-17T07:13:29.205257+0000","last_became_active":"2024-04-17T07:11:54.449628+0000","last_became_peered":"2024-04-17T07:11:54.449628+0000","last_unstale":"2024-04-17T07:13:29.205257+0000","last_undegraded":"2024-04-17T07:13:29.205257+0000","last_fullsized":"2024-04-17T07:13:29.205257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:26:03.281684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069434+0000","last_change":"2024-04-17T07:11:55.476912+0000","last_active":"2024-04-17T07:13:57.069434+0000","last_peered":"2024-04-17T07:13:57.069434+0000","last_clean":"2024-04-17T07:13:57.069434+0000","last_became_active":"2024-04-17T07:11:55.471576+0000","last_became_peered":"2024-04-17T07:11:55.471576+0000","last_unstale":"2024-04-17T07:13:57.069434+0000","last_undegraded":"2024-04-17T07:13:57.069434+0000","last_fullsized":"2024-04-17T07:13:57.069434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:35.439271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333921+0000","last_change":"2024-04-17T07:11:59.515897+0000","last_active":"2024-04-17T07:13:29.333921+0000","last_peered":"2024-04-17T07:13:29.333921+0000","last_clean":"2024-04-17T07:13:29.333921+0000","last_became_active":"2024-04-17T07:11:59.515349+0000","last_became_peered":"2024-04-17T07:11:59.515349+0000","last_unstale":"2024-04-17T07:13:29.333921+0000","last_undegraded":"2024-04-17T07:13:29.333921+0000","last_fullsized":"2024-04-17T07:13:29.333921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:00.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173683+0000","last_change":"2024-04-17T07:11:54.462369+0000","last_active":"2024-04-17T07:13:29.173683+0000","last_peered":"2024-04-17T07:13:29.173683+0000","last_clean":"2024-04-17T07:13:29.173683+0000","last_became_active":"2024-04-17T07:11:54.460357+0000","last_became_peered":"2024-04-17T07:11:54.460357+0000","last_unstale":"2024-04-17T07:13:29.173683+0000","last_undegraded":"2024-04-17T07:13:29.173683+0000","last_fullsized":"2024-04-17T07:13:29.173683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:38.029980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071612+0000","last_change":"2024-04-17T07:11:55.475874+0000","last_active":"2024-04-17T07:13:57.071612+0000","last_peered":"2024-04-17T07:13:57.071612+0000","last_clean":"2024-04-17T07:13:57.071612+0000","last_became_active":"2024-04-17T07:11:55.475550+0000","last_became_peered":"2024-04-17T07:11:55.475550+0000","last_unstale":"2024-04-17T07:13:57.071612+0000","last_undegraded":"2024-04-17T07:13:57.071612+0000","last_fullsized":"2024-04-17T07:13:57.071612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:40.096573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206848+0000","last_change":"2024-04-17T07:11:59.514661+0000","last_active":"2024-04-17T07:13:29.206848+0000","last_peered":"2024-04-17T07:13:29.206848+0000","last_clean":"2024-04-17T07:13:29.206848+0000","last_became_active":"2024-04-17T07:11:59.513030+0000","last_became_peered":"2024-04-17T07:11:59.513030+0000","last_unstale":"2024-04-17T07:13:29.206848+0000","last_undegraded":"2024-04-17T07:13:29.206848+0000","last_fullsized":"2024-04-17T07:13:29.206848+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:35.696961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":194,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:02.671857+0000","last_change":"2024-04-17T07:11:54.455279+0000","last_active":"2024-04-17T07:15:02.671857+0000","last_peered":"2024-04-17T07:15:02.671857+0000","last_clean":"2024-04-17T07:15:02.671857+0000","last_became_active":"2024-04-17T07:11:54.453620+0000","last_became_peered":"2024-04-17T07:11:54.453620+0000","last_unstale":"2024-04-17T07:15:02.671857+0000","last_undegraded":"2024-04-17T07:15:02.671857+0000","last_fullsized":"2024-04-17T07:15:02.671857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:15.227410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":157,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.072155+0000","last_change":"2024-04-17T07:11:55.469579+0000","last_active":"2024-04-17T07:13:57.072155+0000","last_peered":"2024-04-17T07:13:57.072155+0000","last_clean":"2024-04-17T07:13:57.072155+0000","last_became_active":"2024-04-17T07:11:55.468389+0000","last_became_peered":"2024-04-17T07:11:55.468389+0000","last_unstale":"2024-04-17T07:13:57.072155+0000","last_undegraded":"2024-04-17T07:13:57.072155+0000","last_fullsized":"2024-04-17T07:13:57.072155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:01.274677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173558+0000","last_change":"2024-04-17T07:11:59.512110+0000","last_active":"2024-04-17T07:13:29.173558+0000","last_peered":"2024-04-17T07:13:29.173558+0000","last_clean":"2024-04-17T07:13:29.173558+0000","last_became_active":"2024-04-17T07:11:59.510573+0000","last_became_peered":"2024-04-17T07:11:59.510573+0000","last_unstale":"2024-04-17T07:13:29.173558+0000","last_undegraded":"2024-04-17T07:13:29.173558+0000","last_fullsized":"2024-04-17T07:13:29.173558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:32.515687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333363+0000","last_change":"2024-04-17T07:12:28.767517+0000","last_active":"2024-04-17T07:13:29.333363+0000","last_peered":"2024-04-17T07:13:29.333363+0000","last_clean":"2024-04-17T07:13:29.333363+0000","last_became_active":"2024-04-17T07:12:28.738908+0000","last_became_peered":"2024-04-17T07:12:28.738908+0000","last_unstale":"2024-04-17T07:13:29.333363+0000","last_undegraded":"2024-04-17T07:13:29.333363+0000","last_fullsized":"2024-04-17T07:13:29.333363+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:47:30.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.073775+0000","last_change":"2024-04-17T07:11:55.473540+0000","last_active":"2024-04-17T07:13:57.073775+0000","last_peered":"2024-04-17T07:13:57.073775+0000","last_clean":"2024-04-17T07:13:57.073775+0000","last_became_active":"2024-04-17T07:11:55.472436+0000","last_became_peered":"2024-04-17T07:11:55.472436+0000","last_unstale":"2024-04-17T07:13:57.073775+0000","last_undegraded":"2024-04-17T07:13:57.073775+0000","last_fullsized":"2024-04-17T07:13:57.073775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:54.317052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172929+0000","last_change":"2024-04-17T07:11:59.521305+0000","last_active":"2024-04-17T07:13:29.172929+0000","last_peered":"2024-04-17T07:13:29.172929+0000","last_clean":"2024-04-17T07:13:29.172929+0000","last_became_active":"2024-04-17T07:11:59.516860+0000","last_became_peered":"2024-04-17T07:11:59.516860+0000","last_unstale":"2024-04-17T07:13:29.172929+0000","last_undegraded":"2024-04-17T07:13:29.172929+0000","last_fullsized":"2024-04-17T07:13:29.172929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:23.403218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:11.134134+0000","last_change":"2024-04-17T07:11:54.463378+0000","last_active":"2024-04-17T07:14:11.134134+0000","last_peered":"2024-04-17T07:14:11.134134+0000","last_clean":"2024-04-17T07:14:11.134134+0000","last_became_active":"2024-04-17T07:11:54.462120+0000","last_became_peered":"2024-04-17T07:11:54.462120+0000","last_unstale":"2024-04-17T07:14:11.134134+0000","last_undegraded":"2024-04-17T07:14:11.134134+0000","last_fullsized":"2024-04-17T07:14:11.134134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:42.109928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071709+0000","last_change":"2024-04-17T07:11:55.478604+0000","last_active":"2024-04-17T07:13:57.071709+0000","last_peered":"2024-04-17T07:13:57.071709+0000","last_clean":"2024-04-17T07:13:57.071709+0000","last_became_active":"2024-04-17T07:11:55.478252+0000","last_became_peered":"2024-04-17T07:11:55.478252+0000","last_unstale":"2024-04-17T07:13:57.071709+0000","last_undegraded":"2024-04-17T07:13:57.071709+0000","last_fullsized":"2024-04-17T07:13:57.071709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:08.365896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206440+0000","last_change":"2024-04-17T07:11:59.518619+0000","last_active":"2024-04-17T07:13:29.206440+0000","last_peered":"2024-04-17T07:13:29.206440+0000","last_clean":"2024-04-17T07:13:29.206440+0000","last_became_active":"2024-04-17T07:11:59.518214+0000","last_became_peered":"2024-04-17T07:11:59.518214+0000","last_unstale":"2024-04-17T07:13:29.206440+0000","last_undegraded":"2024-04-17T07:13:29.206440+0000","last_fullsized":"2024-04-17T07:13:29.206440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:46.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206316+0000","last_change":"2024-04-17T07:11:54.453038+0000","last_active":"2024-04-17T07:13:29.206316+0000","last_peered":"2024-04-17T07:13:29.206316+0000","last_clean":"2024-04-17T07:13:29.206316+0000","last_became_active":"2024-04-17T07:11:54.452619+0000","last_became_peered":"2024-04-17T07:11:54.452619+0000","last_unstale":"2024-04-17T07:13:29.206316+0000","last_undegraded":"2024-04-17T07:13:29.206316+0000","last_fullsized":"2024-04-17T07:13:29.206316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:25.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.063521+0000","last_change":"2024-04-17T07:11:55.465990+0000","last_active":"2024-04-17T07:13:57.063521+0000","last_peered":"2024-04-17T07:13:57.063521+0000","last_clean":"2024-04-17T07:13:57.063521+0000","last_became_active":"2024-04-17T07:11:55.465523+0000","last_became_peered":"2024-04-17T07:11:55.465523+0000","last_unstale":"2024-04-17T07:13:57.063521+0000","last_undegraded":"2024-04-17T07:13:57.063521+0000","last_fullsized":"2024-04-17T07:13:57.063521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:40.366027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172399+0000","last_change":"2024-04-17T07:11:59.512119+0000","last_active":"2024-04-17T07:13:29.172399+0000","last_peered":"2024-04-17T07:13:29.172399+0000","last_clean":"2024-04-17T07:13:29.172399+0000","last_became_active":"2024-04-17T07:11:59.511460+0000","last_became_peered":"2024-04-17T07:11:59.511460+0000","last_unstale":"2024-04-17T07:13:29.172399+0000","last_undegraded":"2024-04-17T07:13:29.172399+0000","last_fullsized":"2024-04-17T07:13:29.172399+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:50.888203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":196,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:05.650024+0000","last_change":"2024-04-17T07:11:54.469346+0000","last_active":"2024-04-17T07:15:05.650024+0000","last_peered":"2024-04-17T07:15:05.650024+0000","last_clean":"2024-04-17T07:15:05.650024+0000","last_became_active":"2024-04-17T07:11:54.468947+0000","last_became_peered":"2024-04-17T07:11:54.468947+0000","last_unstale":"2024-04-17T07:15:05.650024+0000","last_undegraded":"2024-04-17T07:15:05.650024+0000","last_fullsized":"2024-04-17T07:15:05.650024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:04.854625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":158,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078974+0000","last_change":"2024-04-17T07:11:55.471235+0000","last_active":"2024-04-17T07:13:57.078974+0000","last_peered":"2024-04-17T07:13:57.078974+0000","last_clean":"2024-04-17T07:13:57.078974+0000","last_became_active":"2024-04-17T07:11:55.470857+0000","last_became_peered":"2024-04-17T07:11:55.470857+0000","last_unstale":"2024-04-17T07:13:57.078974+0000","last_undegraded":"2024-04-17T07:13:57.078974+0000","last_fullsized":"2024-04-17T07:13:57.078974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:21.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172054+0000","last_change":"2024-04-17T07:11:59.524519+0000","last_active":"2024-04-17T07:13:29.172054+0000","last_peered":"2024-04-17T07:13:29.172054+0000","last_clean":"2024-04-17T07:13:29.172054+0000","last_became_active":"2024-04-17T07:11:59.519270+0000","last_became_peered":"2024-04-17T07:11:59.519270+0000","last_unstale":"2024-04-17T07:13:29.172054+0000","last_undegraded":"2024-04-17T07:13:29.172054+0000","last_fullsized":"2024-04-17T07:13:29.172054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:41.080249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070265+0000","last_change":"2024-04-17T07:11:55.473922+0000","last_active":"2024-04-17T07:13:57.070265+0000","last_peered":"2024-04-17T07:13:57.070265+0000","last_clean":"2024-04-17T07:13:57.070265+0000","last_became_active":"2024-04-17T07:11:55.468594+0000","last_became_peered":"2024-04-17T07:11:55.468594+0000","last_unstale":"2024-04-17T07:13:57.070265+0000","last_undegraded":"2024-04-17T07:13:57.070265+0000","last_fullsized":"2024-04-17T07:13:57.070265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:21.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":191,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:01.667538+0000","last_change":"2024-04-17T07:11:54.473066+0000","last_active":"2024-04-17T07:15:01.667538+0000","last_peered":"2024-04-17T07:15:01.667538+0000","last_clean":"2024-04-17T07:15:01.667538+0000","last_became_active":"2024-04-17T07:11:54.472708+0000","last_became_peered":"2024-04-17T07:11:54.472708+0000","last_unstale":"2024-04-17T07:15:01.667538+0000","last_undegraded":"2024-04-17T07:15:01.667538+0000","last_fullsized":"2024-04-17T07:15:01.667538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:27.614819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209116+0000","last_change":"2024-04-17T07:11:59.516939+0000","last_active":"2024-04-17T07:13:29.209116+0000","last_peered":"2024-04-17T07:13:29.209116+0000","last_clean":"2024-04-17T07:13:29.209116+0000","last_became_active":"2024-04-17T07:11:59.516333+0000","last_became_peered":"2024-04-17T07:11:59.516333+0000","last_unstale":"2024-04-17T07:13:29.209116+0000","last_undegraded":"2024-04-17T07:13:29.209116+0000","last_fullsized":"2024-04-17T07:13:29.209116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:15.648831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082745+0000","last_change":"2024-04-17T07:11:55.481006+0000","last_active":"2024-04-17T07:13:57.082745+0000","last_peered":"2024-04-17T07:13:57.082745+0000","last_clean":"2024-04-17T07:13:57.082745+0000","last_became_active":"2024-04-17T07:11:55.479751+0000","last_became_peered":"2024-04-17T07:11:55.479751+0000","last_unstale":"2024-04-17T07:13:57.082745+0000","last_undegraded":"2024-04-17T07:13:57.082745+0000","last_fullsized":"2024-04-17T07:13:57.082745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:13.004650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175099+0000","last_change":"2024-04-17T07:11:54.463106+0000","last_active":"2024-04-17T07:13:29.175099+0000","last_peered":"2024-04-17T07:13:29.175099+0000","last_clean":"2024-04-17T07:13:29.175099+0000","last_became_active":"2024-04-17T07:11:54.462691+0000","last_became_peered":"2024-04-17T07:11:54.462691+0000","last_unstale":"2024-04-17T07:13:29.175099+0000","last_undegraded":"2024-04-17T07:13:29.175099+0000","last_fullsized":"2024-04-17T07:13:29.175099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:34.302019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205049+0000","last_change":"2024-04-17T07:10:14.685883+0000","last_active":"2024-04-17T07:13:29.205049+0000","last_peered":"2024-04-17T07:13:29.205049+0000","last_clean":"2024-04-17T07:13:29.205049+0000","last_became_active":"2024-04-17T07:10:14.684551+0000","last_became_peered":"2024-04-17T07:10:14.684551+0000","last_unstale":"2024-04-17T07:13:29.205049+0000","last_undegraded":"2024-04-17T07:13:29.205049+0000","last_fullsized":"2024-04-17T07:13:29.205049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_clean_scrub_stamp":"2024-04-17T07:05:37.112317+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:04.265147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206602+0000","last_change":"2024-04-17T07:12:28.731807+0000","last_active":"2024-04-17T07:13:29.206602+0000","last_peered":"2024-04-17T07:13:29.206602+0000","last_clean":"2024-04-17T07:13:29.206602+0000","last_became_active":"2024-04-17T07:12:28.730656+0000","last_became_peered":"2024-04-17T07:12:28.730656+0000","last_unstale":"2024-04-17T07:13:29.206602+0000","last_undegraded":"2024-04-17T07:13:29.206602+0000","last_fullsized":"2024-04-17T07:13:29.206602+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:58.023333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068799+0000","last_change":"2024-04-17T07:12:28.727980+0000","last_active":"2024-04-17T07:13:57.068799+0000","last_peered":"2024-04-17T07:13:57.068799+0000","last_clean":"2024-04-17T07:13:57.068799+0000","last_became_active":"2024-04-17T07:12:28.727060+0000","last_became_peered":"2024-04-17T07:12:28.727060+0000","last_unstale":"2024-04-17T07:13:57.068799+0000","last_undegraded":"2024-04-17T07:13:57.068799+0000","last_fullsized":"2024-04-17T07:13:57.068799+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:33.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205324+0000","last_change":"2024-04-17T07:11:54.463180+0000","last_active":"2024-04-17T07:13:29.205324+0000","last_peered":"2024-04-17T07:13:29.205324+0000","last_clean":"2024-04-17T07:13:29.205324+0000","last_became_active":"2024-04-17T07:11:54.462793+0000","last_became_peered":"2024-04-17T07:11:54.462793+0000","last_unstale":"2024-04-17T07:13:29.205324+0000","last_undegraded":"2024-04-17T07:13:29.205324+0000","last_fullsized":"2024-04-17T07:13:29.205324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:19.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333323+0000","last_change":"2024-04-17T07:11:59.518355+0000","last_active":"2024-04-17T07:13:29.333323+0000","last_peered":"2024-04-17T07:13:29.333323+0000","last_clean":"2024-04-17T07:13:29.333323+0000","last_became_active":"2024-04-17T07:11:59.517619+0000","last_became_peered":"2024-04-17T07:11:59.517619+0000","last_unstale":"2024-04-17T07:13:29.333323+0000","last_undegraded":"2024-04-17T07:13:29.333323+0000","last_fullsized":"2024-04-17T07:13:29.333323+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:36.826021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206403+0000","last_change":"2024-04-17T07:11:54.465285+0000","last_active":"2024-04-17T07:13:29.206403+0000","last_peered":"2024-04-17T07:13:29.206403+0000","last_clean":"2024-04-17T07:13:29.206403+0000","last_became_active":"2024-04-17T07:11:54.464818+0000","last_became_peered":"2024-04-17T07:11:54.464818+0000","last_unstale":"2024-04-17T07:13:29.206403+0000","last_undegraded":"2024-04-17T07:13:29.206403+0000","last_fullsized":"2024-04-17T07:13:29.206403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069247+0000","last_change":"2024-04-17T07:11:55.470405+0000","last_active":"2024-04-17T07:13:57.069247+0000","last_peered":"2024-04-17T07:13:57.069247+0000","last_clean":"2024-04-17T07:13:57.069247+0000","last_became_active":"2024-04-17T07:11:55.470058+0000","last_became_peered":"2024-04-17T07:11:55.470058+0000","last_unstale":"2024-04-17T07:13:57.069247+0000","last_undegraded":"2024-04-17T07:13:57.069247+0000","last_fullsized":"2024-04-17T07:13:57.069247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:17.416775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.202954+0000","last_change":"2024-04-17T07:11:59.522040+0000","last_active":"2024-04-17T07:13:29.202954+0000","last_peered":"2024-04-17T07:13:29.202954+0000","last_clean":"2024-04-17T07:13:29.202954+0000","last_became_active":"2024-04-17T07:11:59.519771+0000","last_became_peered":"2024-04-17T07:11:59.519771+0000","last_unstale":"2024-04-17T07:13:29.202954+0000","last_undegraded":"2024-04-17T07:13:29.202954+0000","last_fullsized":"2024-04-17T07:13:29.202954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:58.203737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'17","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:00.514634+0000","last_change":"2024-04-17T07:11:54.457954+0000","last_active":"2024-04-17T07:15:00.514634+0000","last_peered":"2024-04-17T07:15:00.514634+0000","last_clean":"2024-04-17T07:15:00.514634+0000","last_became_active":"2024-04-17T07:11:54.457560+0000","last_became_peered":"2024-04-17T07:11:54.457560+0000","last_unstale":"2024-04-17T07:15:00.514634+0000","last_undegraded":"2024-04-17T07:15:00.514634+0000","last_fullsized":"2024-04-17T07:15:00.514634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:31.919730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068908+0000","last_change":"2024-04-17T07:11:55.480475+0000","last_active":"2024-04-17T07:13:57.068908+0000","last_peered":"2024-04-17T07:13:57.068908+0000","last_clean":"2024-04-17T07:13:57.068908+0000","last_became_active":"2024-04-17T07:11:55.480060+0000","last_became_peered":"2024-04-17T07:11:55.480060+0000","last_unstale":"2024-04-17T07:13:57.068908+0000","last_undegraded":"2024-04-17T07:13:57.068908+0000","last_fullsized":"2024-04-17T07:13:57.068908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:30:23.600244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203796+0000","last_change":"2024-04-17T07:11:59.525422+0000","last_active":"2024-04-17T07:13:29.203796+0000","last_peered":"2024-04-17T07:13:29.203796+0000","last_clean":"2024-04-17T07:13:29.203796+0000","last_became_active":"2024-04-17T07:11:59.523649+0000","last_became_peered":"2024-04-17T07:11:59.523649+0000","last_unstale":"2024-04-17T07:13:29.203796+0000","last_undegraded":"2024-04-17T07:13:29.203796+0000","last_fullsized":"2024-04-17T07:13:29.203796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:16.653752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.176127+0000","last_change":"2024-04-17T07:11:54.465824+0000","last_active":"2024-04-17T07:13:29.176127+0000","last_peered":"2024-04-17T07:13:29.176127+0000","last_clean":"2024-04-17T07:13:29.176127+0000","last_became_active":"2024-04-17T07:11:54.465348+0000","last_became_peered":"2024-04-17T07:11:54.465348+0000","last_unstale":"2024-04-17T07:13:29.176127+0000","last_undegraded":"2024-04-17T07:13:29.176127+0000","last_fullsized":"2024-04-17T07:13:29.176127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:17.729903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081497+0000","last_change":"2024-04-17T07:11:55.481666+0000","last_active":"2024-04-17T07:13:57.081497+0000","last_peered":"2024-04-17T07:13:57.081497+0000","last_clean":"2024-04-17T07:13:57.081497+0000","last_became_active":"2024-04-17T07:11:55.480332+0000","last_became_peered":"2024-04-17T07:11:55.480332+0000","last_unstale":"2024-04-17T07:13:57.081497+0000","last_undegraded":"2024-04-17T07:13:57.081497+0000","last_fullsized":"2024-04-17T07:13:57.081497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:19.825407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175613+0000","last_change":"2024-04-17T07:11:59.527850+0000","last_active":"2024-04-17T07:13:29.175613+0000","last_peered":"2024-04-17T07:13:29.175613+0000","last_clean":"2024-04-17T07:13:29.175613+0000","last_became_active":"2024-04-17T07:11:59.527306+0000","last_became_peered":"2024-04-17T07:11:59.527306+0000","last_unstale":"2024-04-17T07:13:29.175613+0000","last_undegraded":"2024-04-17T07:13:29.175613+0000","last_fullsized":"2024-04-17T07:13:29.175613+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:35.906222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204229+0000","last_change":"2024-04-17T07:11:54.455862+0000","last_active":"2024-04-17T07:13:29.204229+0000","last_peered":"2024-04-17T07:13:29.204229+0000","last_clean":"2024-04-17T07:13:29.204229+0000","last_became_active":"2024-04-17T07:11:54.453904+0000","last_became_peered":"2024-04-17T07:11:54.453904+0000","last_unstale":"2024-04-17T07:13:29.204229+0000","last_undegraded":"2024-04-17T07:13:29.204229+0000","last_fullsized":"2024-04-17T07:13:29.204229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:01.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.060326+0000","last_change":"2024-04-17T07:11:55.470315+0000","last_active":"2024-04-17T07:13:57.060326+0000","last_peered":"2024-04-17T07:13:57.060326+0000","last_clean":"2024-04-17T07:13:57.060326+0000","last_became_active":"2024-04-17T07:11:55.469829+0000","last_became_peered":"2024-04-17T07:11:55.469829+0000","last_unstale":"2024-04-17T07:13:57.060326+0000","last_undegraded":"2024-04-17T07:13:57.060326+0000","last_fullsized":"2024-04-17T07:13:57.060326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:48.197546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.210025+0000","last_change":"2024-04-17T07:12:29.125447+0000","last_active":"2024-04-17T07:13:29.210025+0000","last_peered":"2024-04-17T07:13:29.210025+0000","last_clean":"2024-04-17T07:13:29.210025+0000","last_became_active":"2024-04-17T07:12:29.124551+0000","last_became_peered":"2024-04-17T07:12:29.124551+0000","last_unstale":"2024-04-17T07:13:29.210025+0000","last_undegraded":"2024-04-17T07:13:29.210025+0000","last_fullsized":"2024-04-17T07:13:29.210025+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:09.489755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204318+0000","last_change":"2024-04-17T07:11:54.453117+0000","last_active":"2024-04-17T07:13:29.204318+0000","last_peered":"2024-04-17T07:13:29.204318+0000","last_clean":"2024-04-17T07:13:29.204318+0000","last_became_active":"2024-04-17T07:11:54.452632+0000","last_became_peered":"2024-04-17T07:11:54.452632+0000","last_unstale":"2024-04-17T07:13:29.204318+0000","last_undegraded":"2024-04-17T07:13:29.204318+0000","last_fullsized":"2024-04-17T07:13:29.204318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:09.366149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078286+0000","last_change":"2024-04-17T07:11:55.469991+0000","last_active":"2024-04-17T07:13:57.078286+0000","last_peered":"2024-04-17T07:13:57.078286+0000","last_clean":"2024-04-17T07:13:57.078286+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:13:57.078286+0000","last_undegraded":"2024-04-17T07:13:57.078286+0000","last_fullsized":"2024-04-17T07:13:57.078286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:02.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175116+0000","last_change":"2024-04-17T07:11:59.519817+0000","last_active":"2024-04-17T07:13:29.175116+0000","last_peered":"2024-04-17T07:13:29.175116+0000","last_clean":"2024-04-17T07:13:29.175116+0000","last_became_active":"2024-04-17T07:11:59.519108+0000","last_became_peered":"2024-04-17T07:11:59.519108+0000","last_unstale":"2024-04-17T07:13:29.175116+0000","last_undegraded":"2024-04-17T07:13:29.175116+0000","last_fullsized":"2024-04-17T07:13:29.175116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:58.378837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:59.206828+0000","last_change":"2024-04-17T07:11:54.463618+0000","last_active":"2024-04-17T07:14:59.206828+0000","last_peered":"2024-04-17T07:14:59.206828+0000","last_clean":"2024-04-17T07:14:59.206828+0000","last_became_active":"2024-04-17T07:11:54.463257+0000","last_became_peered":"2024-04-17T07:11:54.463257+0000","last_unstale":"2024-04-17T07:14:59.206828+0000","last_undegraded":"2024-04-17T07:14:59.206828+0000","last_fullsized":"2024-04-17T07:14:59.206828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:20.135338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077111+0000","last_change":"2024-04-17T07:11:55.487471+0000","last_active":"2024-04-17T07:13:57.077111+0000","last_peered":"2024-04-17T07:13:57.077111+0000","last_clean":"2024-04-17T07:13:57.077111+0000","last_became_active":"2024-04-17T07:11:55.487024+0000","last_became_peered":"2024-04-17T07:11:55.487024+0000","last_unstale":"2024-04-17T07:13:57.077111+0000","last_undegraded":"2024-04-17T07:13:57.077111+0000","last_fullsized":"2024-04-17T07:13:57.077111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:36.851210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175245+0000","last_change":"2024-04-17T07:11:59.527017+0000","last_active":"2024-04-17T07:13:29.175245+0000","last_peered":"2024-04-17T07:13:29.175245+0000","last_clean":"2024-04-17T07:13:29.175245+0000","last_became_active":"2024-04-17T07:11:59.526541+0000","last_became_peered":"2024-04-17T07:11:59.526541+0000","last_unstale":"2024-04-17T07:13:29.175245+0000","last_undegraded":"2024-04-17T07:13:29.175245+0000","last_fullsized":"2024-04-17T07:13:29.175245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:54.411647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204636+0000","last_change":"2024-04-17T07:11:54.465671+0000","last_active":"2024-04-17T07:13:29.204636+0000","last_peered":"2024-04-17T07:13:29.204636+0000","last_clean":"2024-04-17T07:13:29.204636+0000","last_became_active":"2024-04-17T07:11:54.464902+0000","last_became_peered":"2024-04-17T07:11:54.464902+0000","last_unstale":"2024-04-17T07:13:29.204636+0000","last_undegraded":"2024-04-17T07:13:29.204636+0000","last_fullsized":"2024-04-17T07:13:29.204636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:13:38.232965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084718+0000","last_change":"2024-04-17T07:11:55.484259+0000","last_active":"2024-04-17T07:13:57.084718+0000","last_peered":"2024-04-17T07:13:57.084718+0000","last_clean":"2024-04-17T07:13:57.084718+0000","last_became_active":"2024-04-17T07:11:55.483911+0000","last_became_peered":"2024-04-17T07:11:55.483911+0000","last_unstale":"2024-04-17T07:13:57.084718+0000","last_undegraded":"2024-04-17T07:13:57.084718+0000","last_fullsized":"2024-04-17T07:13:57.084718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:54.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204244+0000","last_change":"2024-04-17T07:11:59.525369+0000","last_active":"2024-04-17T07:13:29.204244+0000","last_peered":"2024-04-17T07:13:29.204244+0000","last_clean":"2024-04-17T07:13:29.204244+0000","last_became_active":"2024-04-17T07:11:59.524809+0000","last_became_peered":"2024-04-17T07:11:59.524809+0000","last_unstale":"2024-04-17T07:13:29.204244+0000","last_undegraded":"2024-04-17T07:13:29.204244+0000","last_fullsized":"2024-04-17T07:13:29.204244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:20.511001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'7","reported_seq":198,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:04.678820+0000","last_change":"2024-04-17T07:11:54.459079+0000","last_active":"2024-04-17T07:15:04.678820+0000","last_peered":"2024-04-17T07:15:04.678820+0000","last_clean":"2024-04-17T07:15:04.678820+0000","last_became_active":"2024-04-17T07:11:54.457008+0000","last_became_peered":"2024-04-17T07:11:54.457008+0000","last_unstale":"2024-04-17T07:15:04.678820+0000","last_undegraded":"2024-04-17T07:15:04.678820+0000","last_fullsized":"2024-04-17T07:15:04.678820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:36.084512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":159,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084193+0000","last_change":"2024-04-17T07:11:55.484453+0000","last_active":"2024-04-17T07:13:57.084193+0000","last_peered":"2024-04-17T07:13:57.084193+0000","last_clean":"2024-04-17T07:13:57.084193+0000","last_became_active":"2024-04-17T07:11:55.483152+0000","last_became_peered":"2024-04-17T07:11:55.483152+0000","last_unstale":"2024-04-17T07:13:57.084193+0000","last_undegraded":"2024-04-17T07:13:57.084193+0000","last_fullsized":"2024-04-17T07:13:57.084193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:32.224868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.202670+0000","last_change":"2024-04-17T07:11:59.521487+0000","last_active":"2024-04-17T07:13:29.202670+0000","last_peered":"2024-04-17T07:13:29.202670+0000","last_clean":"2024-04-17T07:13:29.202670+0000","last_became_active":"2024-04-17T07:11:59.521060+0000","last_became_peered":"2024-04-17T07:11:59.521060+0000","last_unstale":"2024-04-17T07:13:29.202670+0000","last_undegraded":"2024-04-17T07:13:29.202670+0000","last_fullsized":"2024-04-17T07:13:29.202670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:06.692434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174602+0000","last_change":"2024-04-17T07:11:54.459368+0000","last_active":"2024-04-17T07:13:29.174602+0000","last_peered":"2024-04-17T07:13:29.174602+0000","last_clean":"2024-04-17T07:13:29.174602+0000","last_became_active":"2024-04-17T07:11:54.458973+0000","last_became_peered":"2024-04-17T07:11:54.458973+0000","last_unstale":"2024-04-17T07:13:29.174602+0000","last_undegraded":"2024-04-17T07:13:29.174602+0000","last_fullsized":"2024-04-17T07:13:29.174602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:22.034110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.072233+0000","last_change":"2024-04-17T07:11:55.470648+0000","last_active":"2024-04-17T07:13:57.072233+0000","last_peered":"2024-04-17T07:13:57.072233+0000","last_clean":"2024-04-17T07:13:57.072233+0000","last_became_active":"2024-04-17T07:11:55.469666+0000","last_became_peered":"2024-04-17T07:11:55.469666+0000","last_unstale":"2024-04-17T07:13:57.072233+0000","last_undegraded":"2024-04-17T07:13:57.072233+0000","last_fullsized":"2024-04-17T07:13:57.072233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:59.208872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172389+0000","last_change":"2024-04-17T07:11:59.524027+0000","last_active":"2024-04-17T07:13:29.172389+0000","last_peered":"2024-04-17T07:13:29.172389+0000","last_clean":"2024-04-17T07:13:29.172389+0000","last_became_active":"2024-04-17T07:11:59.522764+0000","last_became_peered":"2024-04-17T07:11:59.522764+0000","last_unstale":"2024-04-17T07:13:29.172389+0000","last_undegraded":"2024-04-17T07:13:29.172389+0000","last_fullsized":"2024-04-17T07:13:29.172389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:40.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173260+0000","last_change":"2024-04-17T07:11:54.472884+0000","last_active":"2024-04-17T07:13:29.173260+0000","last_peered":"2024-04-17T07:13:29.173260+0000","last_clean":"2024-04-17T07:13:29.173260+0000","last_became_active":"2024-04-17T07:11:54.472305+0000","last_became_peered":"2024-04-17T07:11:54.472305+0000","last_unstale":"2024-04-17T07:13:29.173260+0000","last_undegraded":"2024-04-17T07:13:29.173260+0000","last_fullsized":"2024-04-17T07:13:29.173260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:21.031442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070727+0000","last_change":"2024-04-17T07:11:55.479982+0000","last_active":"2024-04-17T07:13:57.070727+0000","last_peered":"2024-04-17T07:13:57.070727+0000","last_clean":"2024-04-17T07:13:57.070727+0000","last_became_active":"2024-04-17T07:11:55.477594+0000","last_became_peered":"2024-04-17T07:11:55.477594+0000","last_unstale":"2024-04-17T07:13:57.070727+0000","last_undegraded":"2024-04-17T07:13:57.070727+0000","last_fullsized":"2024-04-17T07:13:57.070727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:23.507312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071666+0000","last_change":"2024-04-17T07:11:55.480941+0000","last_active":"2024-04-17T07:13:57.071666+0000","last_peered":"2024-04-17T07:13:57.071666+0000","last_clean":"2024-04-17T07:13:57.071666+0000","last_became_active":"2024-04-17T07:11:55.480180+0000","last_became_peered":"2024-04-17T07:11:55.480180+0000","last_unstale":"2024-04-17T07:13:57.071666+0000","last_undegraded":"2024-04-17T07:13:57.071666+0000","last_fullsized":"2024-04-17T07:13:57.071666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:16.321806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071075+0000","last_change":"2024-04-17T07:11:55.477663+0000","last_active":"2024-04-17T07:13:57.071075+0000","last_peered":"2024-04-17T07:13:57.071075+0000","last_clean":"2024-04-17T07:13:57.071075+0000","last_became_active":"2024-04-17T07:11:55.477304+0000","last_became_peered":"2024-04-17T07:11:55.477304+0000","last_unstale":"2024-04-17T07:13:57.071075+0000","last_undegraded":"2024-04-17T07:13:57.071075+0000","last_fullsized":"2024-04-17T07:13:57.071075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:00.796633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077531+0000","last_change":"2024-04-17T07:11:55.475536+0000","last_active":"2024-04-17T07:13:57.077531+0000","last_peered":"2024-04-17T07:13:57.077531+0000","last_clean":"2024-04-17T07:13:57.077531+0000","last_became_active":"2024-04-17T07:11:55.475138+0000","last_became_peered":"2024-04-17T07:11:55.475138+0000","last_unstale":"2024-04-17T07:13:57.077531+0000","last_undegraded":"2024-04-17T07:13:57.077531+0000","last_fullsized":"2024-04-17T07:13:57.077531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:47.849076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081347+0000","last_change":"2024-04-17T07:11:55.483742+0000","last_active":"2024-04-17T07:13:57.081347+0000","last_peered":"2024-04-17T07:13:57.081347+0000","last_clean":"2024-04-17T07:13:57.081347+0000","last_became_active":"2024-04-17T07:11:55.483044+0000","last_became_peered":"2024-04-17T07:11:55.483044+0000","last_unstale":"2024-04-17T07:13:57.081347+0000","last_undegraded":"2024-04-17T07:13:57.081347+0000","last_fullsized":"2024-04-17T07:13:57.081347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:18.972137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078285+0000","last_change":"2024-04-17T07:11:55.471266+0000","last_active":"2024-04-17T07:13:57.078285+0000","last_peered":"2024-04-17T07:13:57.078285+0000","last_clean":"2024-04-17T07:13:57.078285+0000","last_became_active":"2024-04-17T07:11:55.470897+0000","last_became_peered":"2024-04-17T07:11:55.470897+0000","last_unstale":"2024-04-17T07:13:57.078285+0000","last_undegraded":"2024-04-17T07:13:57.078285+0000","last_fullsized":"2024-04-17T07:13:57.078285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:25.811064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.066947+0000","last_change":"2024-04-17T07:11:55.475860+0000","last_active":"2024-04-17T07:13:57.066947+0000","last_peered":"2024-04-17T07:13:57.066947+0000","last_clean":"2024-04-17T07:13:57.066947+0000","last_became_active":"2024-04-17T07:11:55.475522+0000","last_became_peered":"2024-04-17T07:11:55.475522+0000","last_unstale":"2024-04-17T07:13:57.066947+0000","last_undegraded":"2024-04-17T07:13:57.066947+0000","last_fullsized":"2024-04-17T07:13:57.066947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:41.548964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081296+0000","last_change":"2024-04-17T07:11:55.478815+0000","last_active":"2024-04-17T07:13:57.081296+0000","last_peered":"2024-04-17T07:13:57.081296+0000","last_clean":"2024-04-17T07:13:57.081296+0000","last_became_active":"2024-04-17T07:11:55.478347+0000","last_became_peered":"2024-04-17T07:11:55.478347+0000","last_unstale":"2024-04-17T07:13:57.081296+0000","last_undegraded":"2024-04-17T07:13:57.081296+0000","last_fullsized":"2024-04-17T07:13:57.081296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:16.465511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069021+0000","last_change":"2024-04-17T07:11:55.474208+0000","last_active":"2024-04-17T07:13:57.069021+0000","last_peered":"2024-04-17T07:13:57.069021+0000","last_clean":"2024-04-17T07:13:57.069021+0000","last_became_active":"2024-04-17T07:11:55.472042+0000","last_became_peered":"2024-04-17T07:11:55.472042+0000","last_unstale":"2024-04-17T07:13:57.069021+0000","last_undegraded":"2024-04-17T07:13:57.069021+0000","last_fullsized":"2024-04-17T07:13:57.069021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:37:24.248506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069995+0000","last_change":"2024-04-17T07:11:55.477257+0000","last_active":"2024-04-17T07:13:57.069995+0000","last_peered":"2024-04-17T07:13:57.069995+0000","last_clean":"2024-04-17T07:13:57.069995+0000","last_became_active":"2024-04-17T07:11:55.473491+0000","last_became_peered":"2024-04-17T07:11:55.473491+0000","last_unstale":"2024-04-17T07:13:57.069995+0000","last_undegraded":"2024-04-17T07:13:57.069995+0000","last_fullsized":"2024-04-17T07:13:57.069995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:33.737485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068915+0000","last_change":"2024-04-17T07:11:55.474930+0000","last_active":"2024-04-17T07:13:57.068915+0000","last_peered":"2024-04-17T07:13:57.068915+0000","last_clean":"2024-04-17T07:13:57.068915+0000","last_became_active":"2024-04-17T07:11:55.472007+0000","last_became_peered":"2024-04-17T07:11:55.472007+0000","last_unstale":"2024-04-17T07:13:57.068915+0000","last_undegraded":"2024-04-17T07:13:57.068915+0000","last_fullsized":"2024-04-17T07:13:57.068915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:07.111267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084510+0000","last_change":"2024-04-17T07:11:55.492422+0000","last_active":"2024-04-17T07:13:57.084510+0000","last_peered":"2024-04-17T07:13:57.084510+0000","last_clean":"2024-04-17T07:13:57.084510+0000","last_became_active":"2024-04-17T07:11:55.491906+0000","last_became_peered":"2024-04-17T07:11:55.491906+0000","last_unstale":"2024-04-17T07:13:57.084510+0000","last_undegraded":"2024-04-17T07:13:57.084510+0000","last_fullsized":"2024-04-17T07:13:57.084510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:13:05.322586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077939+0000","last_change":"2024-04-17T07:11:55.479436+0000","last_active":"2024-04-17T07:13:57.077939+0000","last_peered":"2024-04-17T07:13:57.077939+0000","last_clean":"2024-04-17T07:13:57.077939+0000","last_became_active":"2024-04-17T07:11:55.478982+0000","last_became_peered":"2024-04-17T07:11:55.478982+0000","last_unstale":"2024-04-17T07:13:57.077939+0000","last_undegraded":"2024-04-17T07:13:57.077939+0000","last_fullsized":"2024-04-17T07:13:57.077939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:20.663457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070325+0000","last_change":"2024-04-17T07:11:55.476708+0000","last_active":"2024-04-17T07:13:57.070325+0000","last_peered":"2024-04-17T07:13:57.070325+0000","last_clean":"2024-04-17T07:13:57.070325+0000","last_became_active":"2024-04-17T07:11:55.476344+0000","last_became_peered":"2024-04-17T07:11:55.476344+0000","last_unstale":"2024-04-17T07:13:57.070325+0000","last_undegraded":"2024-04-17T07:13:57.070325+0000","last_fullsized":"2024-04-17T07:13:57.070325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:58.874987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279575,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76685,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1576,"num_read_kb":862,"num_write":274,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":235095,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383734,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":521108,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645374,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":1174853,"data_compressed":64060,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808901,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":524154,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070540,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":520324,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332179,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1118505,"data_compressed":62211,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":516039,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":588652,"data_compressed":28497,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738499,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":517886,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":587010,"data_compressed":28497,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":515270,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920872,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":520113,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757345,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":1113133,"data_compressed":62211,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73812,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72170,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":69645,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:15:07.789 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-17T07:15:07.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T07:15:08.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.046+0000 7f7dae800640 1 -- 172.21.15.86:0/1171554508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da810a170 msgr2=0x7f7da8110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.046+0000 7f7dae800640 1 --2- 172.21.15.86:0/1171554508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da810a170 0x7f7da8110a00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009900 tx=0x7f7da002bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.046+0000 7f7dae800640 1 -- 172.21.15.86:0/1171554508 shutdown_connections 2024-04-17T07:15:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.046+0000 7f7dae800640 1 --2- 172.21.15.86:0/1171554508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da810a170 0x7f7da8110a00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.046+0000 7f7dae800640 1 --2- 172.21.15.86:0/1171554508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da81097b0 0x7f7da8109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.046+0000 7f7dae800640 1 --2- 172.21.15.86:0/1171554508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da81085c0 0x7f7da81089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.046+0000 7f7dae800640 1 -- 172.21.15.86:0/1171554508 >> 172.21.15.86:0/1171554508 conn(0x7f7da8077580 msgr2=0x7f7da80799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.046+0000 7f7dae800640 1 -- 172.21.15.86:0/1171554508 shutdown_connections 2024-04-17T07:15:08.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.047+0000 7f7dae800640 1 -- 172.21.15.86:0/1171554508 wait complete. 2024-04-17T07:15:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.047+0000 7f7dae800640 1 Processor -- start 2024-04-17T07:15:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7dae800640 1 -- start start 2024-04-17T07:15:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7dae800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da81085c0 0x7f7da819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7dae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da81097b0 0x7f7da819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7dae800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da810a170 0x7f7da81a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7dae800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8114120 con 0x7f7da81097b0 2024-04-17T07:15:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7dae800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8113fa0 con 0x7f7da81085c0 2024-04-17T07:15:08.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7dae800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81142a0 con 0x7f7da810a170 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7d9ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da81097b0 0x7f7da819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7d9ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da81097b0 0x7f7da819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37834/0 (socket says 172.21.15.86:37834) 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7d9ea00640 1 -- 172.21.15.86:0/2046018696 learned_addr learned my addr 172.21.15.86:0/2046018696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7da7400640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da81085c0 0x7f7da819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.048+0000 7f7da7e00640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da810a170 0x7f7da81a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7d9ea00640 1 -- 172.21.15.86:0/2046018696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da810a170 msgr2=0x7f7da81a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7d9ea00640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da810a170 0x7f7da81a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7d9ea00640 1 -- 172.21.15.86:0/2046018696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da81085c0 msgr2=0x7f7da819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7d9ea00640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da81085c0 0x7f7da819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7d9ea00640 1 -- 172.21.15.86:0/2046018696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da81a6330 con 0x7f7da81097b0 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7da7400640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da81085c0 0x7f7da819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7d9ea00640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da81097b0 0x7f7da819ec50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c920 tx=0x7f7d9000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7da4c00640 1 -- 172.21.15.86:0/2046018696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90007c70 con 0x7f7da81097b0 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7da4c00640 1 -- 172.21.15.86:0/2046018696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d90007e10 con 0x7f7da81097b0 2024-04-17T07:15:08.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7da4c00640 1 -- 172.21.15.86:0/2046018696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90018820 con 0x7f7da81097b0 2024-04-17T07:15:08.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7dae800640 1 -- 172.21.15.86:0/2046018696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da81a6620 con 0x7f7da81097b0 2024-04-17T07:15:08.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.049+0000 7f7dae800640 1 -- 172.21.15.86:0/2046018696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da81a6b70 con 0x7f7da81097b0 2024-04-17T07:15:08.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.051+0000 7f7dae800640 1 -- 172.21.15.86:0/2046018696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da810f880 con 0x7f7da81097b0 2024-04-17T07:15:08.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.053+0000 7f7da4c00640 1 -- 172.21.15.86:0/2046018696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f7d900189c0 con 0x7f7da81097b0 2024-04-17T07:15:08.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.055+0000 7f7da4c00640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d8c0a3c40 0x7f7d8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:08.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.055+0000 7f7da4c00640 1 -- 172.21.15.86:0/2046018696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7d900d0af0 con 0x7f7da81097b0 2024-04-17T07:15:08.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.055+0000 7f7da7400640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d8c0a3c40 0x7f7d8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:08.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.056+0000 7f7da7400640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d8c0a3c40 0x7f7d8c0a6100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7da8109610 tx=0x7f7d98009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:08.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.059+0000 7f7da4c00640 1 -- 172.21.15.86:0/2046018696 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7d90014030 con 0x7f7da81097b0 2024-04-17T07:15:08.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.275+0000 7f7dae800640 1 -- 172.21.15.86:0/2046018696 --> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7da80623a0 con 0x7f7d8c0a3c40 2024-04-17T07:15:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.287+0000 7f7da4c00640 1 -- 172.21.15.86:0/2046018696 <== mgr.14152 v2:172.21.15.86:6800/561620030 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340881 (secure 0 0 0) 0x7f7da80623a0 con 0x7f7d8c0a3c40 2024-04-17T07:15:08.291 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-17T07:15:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 -- 172.21.15.86:0/2046018696 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d8c0a3c40 msgr2=0x7f7d8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d8c0a3c40 0x7f7d8c0a6100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7da8109610 tx=0x7f7d98009400 comp rx=0 tx=0).stop 2024-04-17T07:15:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 -- 172.21.15.86:0/2046018696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da81097b0 msgr2=0x7f7da819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da81097b0 0x7f7da819ec50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c920 tx=0x7f7d9000cde0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 -- 172.21.15.86:0/2046018696 shutdown_connections 2024-04-17T07:15:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d8c0a3c40 0x7f7d8c0a6100 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7da810a170 0x7f7da81a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7da81097b0 0x7f7da819ec50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 --2- 172.21.15.86:0/2046018696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da81085c0 0x7f7da819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 -- 172.21.15.86:0/2046018696 >> 172.21.15.86:0/2046018696 conn(0x7f7da8077580 msgr2=0x7f7da810ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 -- 172.21.15.86:0/2046018696 shutdown_connections 2024-04-17T07:15:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.294+0000 7f7dae800640 1 -- 172.21.15.86:0/2046018696 wait complete. 2024-04-17T07:15:08.311 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-04-17T07:15:07.131692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667287,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1702,"num_read_kb":971,"num_write":457,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181358,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":873,"ondisk_log_size":873,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365908,"kb_used_data":12596,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575916,"statfs":{"total":1151940427776,"available":1151565737984,"internally_reserved":0,"allocated":12898304,"data_stored":8217047,"data_compressed":432012,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7481,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":61,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005284"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078723+0000","last_change":"2024-04-17T07:11:55.486746+0000","last_active":"2024-04-17T07:13:57.078723+0000","last_peered":"2024-04-17T07:13:57.078723+0000","last_clean":"2024-04-17T07:13:57.078723+0000","last_became_active":"2024-04-17T07:11:55.486335+0000","last_became_peered":"2024-04-17T07:11:55.486335+0000","last_unstale":"2024-04-17T07:13:57.078723+0000","last_undegraded":"2024-04-17T07:13:57.078723+0000","last_fullsized":"2024-04-17T07:13:57.078723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:48.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082087+0000","last_change":"2024-04-17T07:11:55.481832+0000","last_active":"2024-04-17T07:13:57.082087+0000","last_peered":"2024-04-17T07:13:57.082087+0000","last_clean":"2024-04-17T07:13:57.082087+0000","last_became_active":"2024-04-17T07:11:55.476428+0000","last_became_peered":"2024-04-17T07:11:55.476428+0000","last_unstale":"2024-04-17T07:13:57.082087+0000","last_undegraded":"2024-04-17T07:13:57.082087+0000","last_fullsized":"2024-04-17T07:13:57.082087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:10:04.452542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083395+0000","last_change":"2024-04-17T07:11:55.483721+0000","last_active":"2024-04-17T07:13:57.083395+0000","last_peered":"2024-04-17T07:13:57.083395+0000","last_clean":"2024-04-17T07:13:57.083395+0000","last_became_active":"2024-04-17T07:11:55.483268+0000","last_became_peered":"2024-04-17T07:11:55.483268+0000","last_unstale":"2024-04-17T07:13:57.083395+0000","last_undegraded":"2024-04-17T07:13:57.083395+0000","last_fullsized":"2024-04-17T07:13:57.083395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:05:41.584783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067540+0000","last_change":"2024-04-17T07:11:55.485114+0000","last_active":"2024-04-17T07:13:57.067540+0000","last_peered":"2024-04-17T07:13:57.067540+0000","last_clean":"2024-04-17T07:13:57.067540+0000","last_became_active":"2024-04-17T07:11:55.483913+0000","last_became_peered":"2024-04-17T07:11:55.483913+0000","last_unstale":"2024-04-17T07:13:57.067540+0000","last_undegraded":"2024-04-17T07:13:57.067540+0000","last_fullsized":"2024-04-17T07:13:57.067540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:46.211121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.060857+0000","last_change":"2024-04-17T07:11:55.491952+0000","last_active":"2024-04-17T07:13:57.060857+0000","last_peered":"2024-04-17T07:13:57.060857+0000","last_clean":"2024-04-17T07:13:57.060857+0000","last_became_active":"2024-04-17T07:11:55.491238+0000","last_became_peered":"2024-04-17T07:11:55.491238+0000","last_unstale":"2024-04-17T07:13:57.060857+0000","last_undegraded":"2024-04-17T07:13:57.060857+0000","last_fullsized":"2024-04-17T07:13:57.060857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:06.759797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068819+0000","last_change":"2024-04-17T07:11:55.471124+0000","last_active":"2024-04-17T07:13:57.068819+0000","last_peered":"2024-04-17T07:13:57.068819+0000","last_clean":"2024-04-17T07:13:57.068819+0000","last_became_active":"2024-04-17T07:11:55.470781+0000","last_became_peered":"2024-04-17T07:11:55.470781+0000","last_unstale":"2024-04-17T07:13:57.068819+0000","last_undegraded":"2024-04-17T07:13:57.068819+0000","last_fullsized":"2024-04-17T07:13:57.068819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:45.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.072320+0000","last_change":"2024-04-17T07:11:55.469099+0000","last_active":"2024-04-17T07:13:57.072320+0000","last_peered":"2024-04-17T07:13:57.072320+0000","last_clean":"2024-04-17T07:13:57.072320+0000","last_became_active":"2024-04-17T07:11:55.468584+0000","last_became_peered":"2024-04-17T07:11:55.468584+0000","last_unstale":"2024-04-17T07:13:57.072320+0000","last_undegraded":"2024-04-17T07:13:57.072320+0000","last_fullsized":"2024-04-17T07:13:57.072320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:57:52.834846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071967+0000","last_change":"2024-04-17T07:11:55.473794+0000","last_active":"2024-04-17T07:13:57.071967+0000","last_peered":"2024-04-17T07:13:57.071967+0000","last_clean":"2024-04-17T07:13:57.071967+0000","last_became_active":"2024-04-17T07:11:55.471919+0000","last_became_peered":"2024-04-17T07:11:55.471919+0000","last_unstale":"2024-04-17T07:13:57.071967+0000","last_undegraded":"2024-04-17T07:13:57.071967+0000","last_fullsized":"2024-04-17T07:13:57.071967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:56:16.085117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.079281+0000","last_change":"2024-04-17T07:11:55.482445+0000","last_active":"2024-04-17T07:13:57.079281+0000","last_peered":"2024-04-17T07:13:57.079281+0000","last_clean":"2024-04-17T07:13:57.079281+0000","last_became_active":"2024-04-17T07:11:55.481552+0000","last_became_peered":"2024-04-17T07:11:55.481552+0000","last_unstale":"2024-04-17T07:13:57.079281+0000","last_undegraded":"2024-04-17T07:13:57.079281+0000","last_fullsized":"2024-04-17T07:13:57.079281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:48.733585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071322+0000","last_change":"2024-04-17T07:11:55.476726+0000","last_active":"2024-04-17T07:13:57.071322+0000","last_peered":"2024-04-17T07:13:57.071322+0000","last_clean":"2024-04-17T07:13:57.071322+0000","last_became_active":"2024-04-17T07:11:55.476320+0000","last_became_peered":"2024-04-17T07:11:55.476320+0000","last_unstale":"2024-04-17T07:13:57.071322+0000","last_undegraded":"2024-04-17T07:13:57.071322+0000","last_fullsized":"2024-04-17T07:13:57.071322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:00:06.230181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084097+0000","last_change":"2024-04-17T07:11:55.485798+0000","last_active":"2024-04-17T07:13:57.084097+0000","last_peered":"2024-04-17T07:13:57.084097+0000","last_clean":"2024-04-17T07:13:57.084097+0000","last_became_active":"2024-04-17T07:11:55.484015+0000","last_became_peered":"2024-04-17T07:11:55.484015+0000","last_unstale":"2024-04-17T07:13:57.084097+0000","last_undegraded":"2024-04-17T07:13:57.084097+0000","last_fullsized":"2024-04-17T07:13:57.084097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:37.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.062297+0000","last_change":"2024-04-17T07:12:28.729834+0000","last_active":"2024-04-17T07:13:57.062297+0000","last_peered":"2024-04-17T07:13:57.062297+0000","last_clean":"2024-04-17T07:13:57.062297+0000","last_became_active":"2024-04-17T07:12:28.728694+0000","last_became_peered":"2024-04-17T07:12:28.728694+0000","last_unstale":"2024-04-17T07:13:57.062297+0000","last_undegraded":"2024-04-17T07:13:57.062297+0000","last_fullsized":"2024-04-17T07:13:57.062297+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:46.381497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.067980+0000","last_change":"2024-04-17T07:13:29.210325+0000","last_active":"2024-04-17T07:13:57.067980+0000","last_peered":"2024-04-17T07:13:57.067980+0000","last_clean":"2024-04-17T07:13:57.067980+0000","last_became_active":"2024-04-17T07:13:29.209759+0000","last_became_peered":"2024-04-17T07:13:29.209759+0000","last_unstale":"2024-04-17T07:13:57.067980+0000","last_undegraded":"2024-04-17T07:13:57.067980+0000","last_fullsized":"2024-04-17T07:13:57.067980+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:17.168931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083596+0000","last_change":"2024-04-17T07:11:55.473839+0000","last_active":"2024-04-17T07:13:57.083596+0000","last_peered":"2024-04-17T07:13:57.083596+0000","last_clean":"2024-04-17T07:13:57.083596+0000","last_became_active":"2024-04-17T07:11:55.473472+0000","last_became_peered":"2024-04-17T07:11:55.473472+0000","last_unstale":"2024-04-17T07:13:57.083596+0000","last_undegraded":"2024-04-17T07:13:57.083596+0000","last_fullsized":"2024-04-17T07:13:57.083596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:30:02.263272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081481+0000","last_change":"2024-04-17T07:11:55.484820+0000","last_active":"2024-04-17T07:13:57.081481+0000","last_peered":"2024-04-17T07:13:57.081481+0000","last_clean":"2024-04-17T07:13:57.081481+0000","last_became_active":"2024-04-17T07:11:55.484314+0000","last_became_peered":"2024-04-17T07:11:55.484314+0000","last_unstale":"2024-04-17T07:13:57.081481+0000","last_undegraded":"2024-04-17T07:13:57.081481+0000","last_fullsized":"2024-04-17T07:13:57.081481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:38.186912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084394+0000","last_change":"2024-04-17T07:11:55.492018+0000","last_active":"2024-04-17T07:13:57.084394+0000","last_peered":"2024-04-17T07:13:57.084394+0000","last_clean":"2024-04-17T07:13:57.084394+0000","last_became_active":"2024-04-17T07:11:55.490512+0000","last_became_peered":"2024-04-17T07:11:55.490512+0000","last_unstale":"2024-04-17T07:13:57.084394+0000","last_undegraded":"2024-04-17T07:13:57.084394+0000","last_fullsized":"2024-04-17T07:13:57.084394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:31:45.436270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081460+0000","last_change":"2024-04-17T07:11:55.487210+0000","last_active":"2024-04-17T07:13:57.081460+0000","last_peered":"2024-04-17T07:13:57.081460+0000","last_clean":"2024-04-17T07:13:57.081460+0000","last_became_active":"2024-04-17T07:11:55.485713+0000","last_became_peered":"2024-04-17T07:11:55.485713+0000","last_unstale":"2024-04-17T07:13:57.081460+0000","last_undegraded":"2024-04-17T07:13:57.081460+0000","last_fullsized":"2024-04-17T07:13:57.081460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:21:54.495950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082817+0000","last_change":"2024-04-17T07:11:55.482705+0000","last_active":"2024-04-17T07:13:57.082817+0000","last_peered":"2024-04-17T07:13:57.082817+0000","last_clean":"2024-04-17T07:13:57.082817+0000","last_became_active":"2024-04-17T07:11:55.482349+0000","last_became_peered":"2024-04-17T07:11:55.482349+0000","last_unstale":"2024-04-17T07:13:57.082817+0000","last_undegraded":"2024-04-17T07:13:57.082817+0000","last_fullsized":"2024-04-17T07:13:57.082817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:12:57.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069422+0000","last_change":"2024-04-17T07:11:55.467065+0000","last_active":"2024-04-17T07:13:57.069422+0000","last_peered":"2024-04-17T07:13:57.069422+0000","last_clean":"2024-04-17T07:13:57.069422+0000","last_became_active":"2024-04-17T07:11:55.466535+0000","last_became_peered":"2024-04-17T07:11:55.466535+0000","last_unstale":"2024-04-17T07:13:57.069422+0000","last_undegraded":"2024-04-17T07:13:57.069422+0000","last_fullsized":"2024-04-17T07:13:57.069422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:49.955453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204394+0000","last_change":"2024-04-17T07:11:59.515778+0000","last_active":"2024-04-17T07:13:29.204394+0000","last_peered":"2024-04-17T07:13:29.204394+0000","last_clean":"2024-04-17T07:13:29.204394+0000","last_became_active":"2024-04-17T07:11:59.515266+0000","last_became_peered":"2024-04-17T07:11:59.515266+0000","last_unstale":"2024-04-17T07:13:29.204394+0000","last_undegraded":"2024-04-17T07:13:29.204394+0000","last_fullsized":"2024-04-17T07:13:29.204394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:11:43.084724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":203,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:05.648744+0000","last_change":"2024-04-17T07:11:54.467060+0000","last_active":"2024-04-17T07:15:05.648744+0000","last_peered":"2024-04-17T07:15:05.648744+0000","last_clean":"2024-04-17T07:15:05.648744+0000","last_became_active":"2024-04-17T07:11:54.466687+0000","last_became_peered":"2024-04-17T07:11:54.466687+0000","last_unstale":"2024-04-17T07:15:05.648744+0000","last_undegraded":"2024-04-17T07:15:05.648744+0000","last_fullsized":"2024-04-17T07:15:05.648744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:28:57.747200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":157,"num_read_kb":157,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.064498+0000","last_change":"2024-04-17T07:11:55.469857+0000","last_active":"2024-04-17T07:13:57.064498+0000","last_peered":"2024-04-17T07:13:57.064498+0000","last_clean":"2024-04-17T07:13:57.064498+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:13:57.064498+0000","last_undegraded":"2024-04-17T07:13:57.064498+0000","last_fullsized":"2024-04-17T07:13:57.064498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:43.565627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174451+0000","last_change":"2024-04-17T07:12:29.125397+0000","last_active":"2024-04-17T07:13:29.174451+0000","last_peered":"2024-04-17T07:13:29.174451+0000","last_clean":"2024-04-17T07:13:29.174451+0000","last_became_active":"2024-04-17T07:12:29.124158+0000","last_became_peered":"2024-04-17T07:12:29.124158+0000","last_unstale":"2024-04-17T07:13:29.174451+0000","last_undegraded":"2024-04-17T07:13:29.174451+0000","last_fullsized":"2024-04-17T07:13:29.174451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:39:24.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205266+0000","last_change":"2024-04-17T07:11:54.464715+0000","last_active":"2024-04-17T07:13:29.205266+0000","last_peered":"2024-04-17T07:13:29.205266+0000","last_clean":"2024-04-17T07:13:29.205266+0000","last_became_active":"2024-04-17T07:11:54.463917+0000","last_became_peered":"2024-04-17T07:11:54.463917+0000","last_unstale":"2024-04-17T07:13:29.205266+0000","last_undegraded":"2024-04-17T07:13:29.205266+0000","last_fullsized":"2024-04-17T07:13:29.205266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:39:43.738471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.085059+0000","last_change":"2024-04-17T07:11:55.482876+0000","last_active":"2024-04-17T07:13:57.085059+0000","last_peered":"2024-04-17T07:13:57.085059+0000","last_clean":"2024-04-17T07:13:57.085059+0000","last_became_active":"2024-04-17T07:11:55.482451+0000","last_became_peered":"2024-04-17T07:11:55.482451+0000","last_unstale":"2024-04-17T07:13:57.085059+0000","last_undegraded":"2024-04-17T07:13:57.085059+0000","last_fullsized":"2024-04-17T07:13:57.085059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:59:39.394090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174912+0000","last_change":"2024-04-17T07:11:59.522634+0000","last_active":"2024-04-17T07:13:29.174912+0000","last_peered":"2024-04-17T07:13:29.174912+0000","last_clean":"2024-04-17T07:13:29.174912+0000","last_became_active":"2024-04-17T07:11:59.520797+0000","last_became_peered":"2024-04-17T07:11:59.520797+0000","last_unstale":"2024-04-17T07:13:29.174912+0000","last_undegraded":"2024-04-17T07:13:29.174912+0000","last_fullsized":"2024-04-17T07:13:29.174912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:32:02.316515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.334212+0000","last_change":"2024-04-17T07:12:28.756528+0000","last_active":"2024-04-17T07:13:29.334212+0000","last_peered":"2024-04-17T07:13:29.334212+0000","last_clean":"2024-04-17T07:13:29.334212+0000","last_became_active":"2024-04-17T07:12:28.737008+0000","last_became_peered":"2024-04-17T07:12:28.737008+0000","last_unstale":"2024-04-17T07:13:29.334212+0000","last_undegraded":"2024-04-17T07:13:29.334212+0000","last_fullsized":"2024-04-17T07:13:29.334212+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:56:26.719377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083022+0000","last_change":"2024-04-17T07:11:55.485999+0000","last_active":"2024-04-17T07:13:57.083022+0000","last_peered":"2024-04-17T07:13:57.083022+0000","last_clean":"2024-04-17T07:13:57.083022+0000","last_became_active":"2024-04-17T07:11:55.484048+0000","last_became_peered":"2024-04-17T07:11:55.484048+0000","last_unstale":"2024-04-17T07:13:57.083022+0000","last_undegraded":"2024-04-17T07:13:57.083022+0000","last_fullsized":"2024-04-17T07:13:57.083022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:31:38.933166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174422+0000","last_change":"2024-04-17T07:11:59.529929+0000","last_active":"2024-04-17T07:13:29.174422+0000","last_peered":"2024-04-17T07:13:29.174422+0000","last_clean":"2024-04-17T07:13:29.174422+0000","last_became_active":"2024-04-17T07:11:59.529400+0000","last_became_peered":"2024-04-17T07:11:59.529400+0000","last_unstale":"2024-04-17T07:13:29.174422+0000","last_undegraded":"2024-04-17T07:13:29.174422+0000","last_fullsized":"2024-04-17T07:13:29.174422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:31:54.845529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204898+0000","last_change":"2024-04-17T07:11:54.462852+0000","last_active":"2024-04-17T07:13:29.204898+0000","last_peered":"2024-04-17T07:13:29.204898+0000","last_clean":"2024-04-17T07:13:29.204898+0000","last_became_active":"2024-04-17T07:11:54.459178+0000","last_became_peered":"2024-04-17T07:11:54.459178+0000","last_unstale":"2024-04-17T07:13:29.204898+0000","last_undegraded":"2024-04-17T07:13:29.204898+0000","last_fullsized":"2024-04-17T07:13:29.204898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:40:07.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.075256+0000","last_change":"2024-04-17T07:11:55.480951+0000","last_active":"2024-04-17T07:13:57.075256+0000","last_peered":"2024-04-17T07:13:57.075256+0000","last_clean":"2024-04-17T07:13:57.075256+0000","last_became_active":"2024-04-17T07:11:55.480507+0000","last_became_peered":"2024-04-17T07:11:55.480507+0000","last_unstale":"2024-04-17T07:13:57.075256+0000","last_undegraded":"2024-04-17T07:13:57.075256+0000","last_fullsized":"2024-04-17T07:13:57.075256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:26:42.584180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205487+0000","last_change":"2024-04-17T07:11:59.520897+0000","last_active":"2024-04-17T07:13:29.205487+0000","last_peered":"2024-04-17T07:13:29.205487+0000","last_clean":"2024-04-17T07:13:29.205487+0000","last_became_active":"2024-04-17T07:11:59.520350+0000","last_became_peered":"2024-04-17T07:11:59.520350+0000","last_unstale":"2024-04-17T07:13:29.205487+0000","last_undegraded":"2024-04-17T07:13:29.205487+0000","last_fullsized":"2024-04-17T07:13:29.205487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:30:39.813097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":212,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:05.699123+0000","last_change":"2024-04-17T07:12:28.753717+0000","last_active":"2024-04-17T07:15:05.699123+0000","last_peered":"2024-04-17T07:15:05.699123+0000","last_clean":"2024-04-17T07:15:05.699123+0000","last_became_active":"2024-04-17T07:12:28.739556+0000","last_became_peered":"2024-04-17T07:12:28.739556+0000","last_unstale":"2024-04-17T07:15:05.699123+0000","last_undegraded":"2024-04-17T07:15:05.699123+0000","last_fullsized":"2024-04-17T07:15:05.699123+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:11.967731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1631,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":160,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.055509+0000","last_change":"2024-04-17T07:11:55.476675+0000","last_active":"2024-04-17T07:13:57.055509+0000","last_peered":"2024-04-17T07:13:57.055509+0000","last_clean":"2024-04-17T07:13:57.055509+0000","last_became_active":"2024-04-17T07:11:55.476273+0000","last_became_peered":"2024-04-17T07:11:55.476273+0000","last_unstale":"2024-04-17T07:13:57.055509+0000","last_undegraded":"2024-04-17T07:13:57.055509+0000","last_fullsized":"2024-04-17T07:13:57.055509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:41:09.612138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205617+0000","last_change":"2024-04-17T07:12:28.741537+0000","last_active":"2024-04-17T07:13:29.205617+0000","last_peered":"2024-04-17T07:13:29.205617+0000","last_clean":"2024-04-17T07:13:29.205617+0000","last_became_active":"2024-04-17T07:12:28.741035+0000","last_became_peered":"2024-04-17T07:12:28.741035+0000","last_unstale":"2024-04-17T07:13:29.205617+0000","last_undegraded":"2024-04-17T07:13:29.205617+0000","last_fullsized":"2024-04-17T07:13:29.205617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:19:15.550112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"91'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.207986+0000","last_change":"2024-04-17T07:11:54.471118+0000","last_active":"2024-04-17T07:13:29.207986+0000","last_peered":"2024-04-17T07:13:29.207986+0000","last_clean":"2024-04-17T07:13:29.207986+0000","last_became_active":"2024-04-17T07:11:54.470526+0000","last_became_peered":"2024-04-17T07:11:54.470526+0000","last_unstale":"2024-04-17T07:13:29.207986+0000","last_undegraded":"2024-04-17T07:13:29.207986+0000","last_fullsized":"2024-04-17T07:13:29.207986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:52:47.918086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071367+0000","last_change":"2024-04-17T07:11:55.472850+0000","last_active":"2024-04-17T07:13:57.071367+0000","last_peered":"2024-04-17T07:13:57.071367+0000","last_clean":"2024-04-17T07:13:57.071367+0000","last_became_active":"2024-04-17T07:11:55.472097+0000","last_became_peered":"2024-04-17T07:11:55.472097+0000","last_unstale":"2024-04-17T07:13:57.071367+0000","last_undegraded":"2024-04-17T07:13:57.071367+0000","last_fullsized":"2024-04-17T07:13:57.071367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:41.730809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.334157+0000","last_change":"2024-04-17T07:11:59.522960+0000","last_active":"2024-04-17T07:13:29.334157+0000","last_peered":"2024-04-17T07:13:29.334157+0000","last_clean":"2024-04-17T07:13:29.334157+0000","last_became_active":"2024-04-17T07:11:59.522470+0000","last_became_peered":"2024-04-17T07:11:59.522470+0000","last_unstale":"2024-04-17T07:13:29.334157+0000","last_undegraded":"2024-04-17T07:13:29.334157+0000","last_fullsized":"2024-04-17T07:13:29.334157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:29:36.620442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"91'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.171736+0000","last_change":"2024-04-17T07:11:54.470693+0000","last_active":"2024-04-17T07:13:29.171736+0000","last_peered":"2024-04-17T07:13:29.171736+0000","last_clean":"2024-04-17T07:13:29.171736+0000","last_became_active":"2024-04-17T07:11:54.470334+0000","last_became_peered":"2024-04-17T07:11:54.470334+0000","last_unstale":"2024-04-17T07:13:29.171736+0000","last_undegraded":"2024-04-17T07:13:29.171736+0000","last_fullsized":"2024-04-17T07:13:29.171736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:19:04.980786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070659+0000","last_change":"2024-04-17T07:11:55.474755+0000","last_active":"2024-04-17T07:13:57.070659+0000","last_peered":"2024-04-17T07:13:57.070659+0000","last_clean":"2024-04-17T07:13:57.070659+0000","last_became_active":"2024-04-17T07:11:55.474261+0000","last_became_peered":"2024-04-17T07:11:55.474261+0000","last_unstale":"2024-04-17T07:13:57.070659+0000","last_undegraded":"2024-04-17T07:13:57.070659+0000","last_fullsized":"2024-04-17T07:13:57.070659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:26:00.436519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203208+0000","last_change":"2024-04-17T07:11:59.517152+0000","last_active":"2024-04-17T07:13:29.203208+0000","last_peered":"2024-04-17T07:13:29.203208+0000","last_clean":"2024-04-17T07:13:29.203208+0000","last_became_active":"2024-04-17T07:11:59.514977+0000","last_became_peered":"2024-04-17T07:11:59.514977+0000","last_unstale":"2024-04-17T07:13:29.203208+0000","last_undegraded":"2024-04-17T07:13:29.203208+0000","last_fullsized":"2024-04-17T07:13:29.203208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:07:14.974136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"91'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172963+0000","last_change":"2024-04-17T07:11:54.459786+0000","last_active":"2024-04-17T07:13:29.172963+0000","last_peered":"2024-04-17T07:13:29.172963+0000","last_clean":"2024-04-17T07:13:29.172963+0000","last_became_active":"2024-04-17T07:11:54.457589+0000","last_became_peered":"2024-04-17T07:11:54.457589+0000","last_unstale":"2024-04-17T07:13:29.172963+0000","last_undegraded":"2024-04-17T07:13:29.172963+0000","last_fullsized":"2024-04-17T07:13:29.172963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:46:25.986539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068150+0000","last_change":"2024-04-17T07:11:55.465020+0000","last_active":"2024-04-17T07:13:57.068150+0000","last_peered":"2024-04-17T07:13:57.068150+0000","last_clean":"2024-04-17T07:13:57.068150+0000","last_became_active":"2024-04-17T07:11:55.464608+0000","last_became_peered":"2024-04-17T07:11:55.464608+0000","last_unstale":"2024-04-17T07:13:57.068150+0000","last_undegraded":"2024-04-17T07:13:57.068150+0000","last_fullsized":"2024-04-17T07:13:57.068150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.852199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204578+0000","last_change":"2024-04-17T07:11:59.519333+0000","last_active":"2024-04-17T07:13:29.204578+0000","last_peered":"2024-04-17T07:13:29.204578+0000","last_clean":"2024-04-17T07:13:29.204578+0000","last_became_active":"2024-04-17T07:11:59.518865+0000","last_became_peered":"2024-04-17T07:11:59.518865+0000","last_unstale":"2024-04-17T07:13:29.204578+0000","last_undegraded":"2024-04-17T07:13:29.204578+0000","last_fullsized":"2024-04-17T07:13:29.204578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:48:42.009010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":341,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:01.666653+0000","last_change":"2024-04-17T07:11:54.462283+0000","last_active":"2024-04-17T07:15:01.666653+0000","last_peered":"2024-04-17T07:15:01.666653+0000","last_clean":"2024-04-17T07:15:01.666653+0000","last_became_active":"2024-04-17T07:11:54.461818+0000","last_became_peered":"2024-04-17T07:11:54.461818+0000","last_unstale":"2024-04-17T07:15:01.666653+0000","last_undegraded":"2024-04-17T07:15:01.666653+0000","last_fullsized":"2024-04-17T07:15:01.666653+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:07:26.765058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":302,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081011+0000","last_change":"2024-04-17T07:11:55.486491+0000","last_active":"2024-04-17T07:13:57.081011+0000","last_peered":"2024-04-17T07:13:57.081011+0000","last_clean":"2024-04-17T07:13:57.081011+0000","last_became_active":"2024-04-17T07:11:55.486121+0000","last_became_peered":"2024-04-17T07:11:55.486121+0000","last_unstale":"2024-04-17T07:13:57.081011+0000","last_undegraded":"2024-04-17T07:13:57.081011+0000","last_fullsized":"2024-04-17T07:13:57.081011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:12.998321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205968+0000","last_change":"2024-04-17T07:11:59.511555+0000","last_active":"2024-04-17T07:13:29.205968+0000","last_peered":"2024-04-17T07:13:29.205968+0000","last_clean":"2024-04-17T07:13:29.205968+0000","last_became_active":"2024-04-17T07:11:59.511147+0000","last_became_peered":"2024-04-17T07:11:59.511147+0000","last_unstale":"2024-04-17T07:13:29.205968+0000","last_undegraded":"2024-04-17T07:13:29.205968+0000","last_fullsized":"2024-04-17T07:13:29.205968+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:54:41.854815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"91'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209570+0000","last_change":"2024-04-17T07:12:28.734476+0000","last_active":"2024-04-17T07:13:29.209570+0000","last_peered":"2024-04-17T07:13:29.209570+0000","last_clean":"2024-04-17T07:13:29.209570+0000","last_became_active":"2024-04-17T07:12:28.728187+0000","last_became_peered":"2024-04-17T07:12:28.728187+0000","last_unstale":"2024-04-17T07:13:29.209570+0000","last_undegraded":"2024-04-17T07:13:29.209570+0000","last_fullsized":"2024-04-17T07:13:29.209570+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:23:45.749607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081552+0000","last_change":"2024-04-17T07:11:55.480589+0000","last_active":"2024-04-17T07:13:57.081552+0000","last_peered":"2024-04-17T07:13:57.081552+0000","last_clean":"2024-04-17T07:13:57.081552+0000","last_became_active":"2024-04-17T07:11:55.480224+0000","last_became_peered":"2024-04-17T07:11:55.480224+0000","last_unstale":"2024-04-17T07:13:57.081552+0000","last_undegraded":"2024-04-17T07:13:57.081552+0000","last_fullsized":"2024-04-17T07:13:57.081552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:19:05.892442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175186+0000","last_change":"2024-04-17T07:11:59.522021+0000","last_active":"2024-04-17T07:13:29.175186+0000","last_peered":"2024-04-17T07:13:29.175186+0000","last_clean":"2024-04-17T07:13:29.175186+0000","last_became_active":"2024-04-17T07:11:59.521493+0000","last_became_peered":"2024-04-17T07:11:59.521493+0000","last_unstale":"2024-04-17T07:13:29.175186+0000","last_undegraded":"2024-04-17T07:13:29.175186+0000","last_fullsized":"2024-04-17T07:13:29.175186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:20.386216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174471+0000","last_change":"2024-04-17T07:11:54.468225+0000","last_active":"2024-04-17T07:13:29.174471+0000","last_peered":"2024-04-17T07:13:29.174471+0000","last_clean":"2024-04-17T07:13:29.174471+0000","last_became_active":"2024-04-17T07:11:54.467046+0000","last_became_peered":"2024-04-17T07:11:54.467046+0000","last_unstale":"2024-04-17T07:13:29.174471+0000","last_undegraded":"2024-04-17T07:13:29.174471+0000","last_fullsized":"2024-04-17T07:13:29.174471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:10:17.725255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.083563+0000","last_change":"2024-04-17T07:11:55.488980+0000","last_active":"2024-04-17T07:13:57.083563+0000","last_peered":"2024-04-17T07:13:57.083563+0000","last_clean":"2024-04-17T07:13:57.083563+0000","last_became_active":"2024-04-17T07:11:55.488226+0000","last_became_peered":"2024-04-17T07:11:55.488226+0000","last_unstale":"2024-04-17T07:13:57.083563+0000","last_undegraded":"2024-04-17T07:13:57.083563+0000","last_fullsized":"2024-04-17T07:13:57.083563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:22:00.135973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205610+0000","last_change":"2024-04-17T07:11:59.515134+0000","last_active":"2024-04-17T07:13:29.205610+0000","last_peered":"2024-04-17T07:13:29.205610+0000","last_clean":"2024-04-17T07:13:29.205610+0000","last_became_active":"2024-04-17T07:11:59.514560+0000","last_became_peered":"2024-04-17T07:11:59.514560+0000","last_unstale":"2024-04-17T07:13:29.205610+0000","last_undegraded":"2024-04-17T07:13:29.205610+0000","last_fullsized":"2024-04-17T07:13:29.205610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:28:34.669789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":381,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:01.667846+0000","last_change":"2024-04-17T07:11:54.461226+0000","last_active":"2024-04-17T07:15:01.667846+0000","last_peered":"2024-04-17T07:15:01.667846+0000","last_clean":"2024-04-17T07:15:01.667846+0000","last_became_active":"2024-04-17T07:11:54.460573+0000","last_became_peered":"2024-04-17T07:11:54.460573+0000","last_unstale":"2024-04-17T07:15:01.667846+0000","last_undegraded":"2024-04-17T07:15:01.667846+0000","last_fullsized":"2024-04-17T07:15:01.667846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:59:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67432,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":310,"num_read_kb":216,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068452+0000","last_change":"2024-04-17T07:11:55.483925+0000","last_active":"2024-04-17T07:13:57.068452+0000","last_peered":"2024-04-17T07:13:57.068452+0000","last_clean":"2024-04-17T07:13:57.068452+0000","last_became_active":"2024-04-17T07:11:55.483584+0000","last_became_peered":"2024-04-17T07:11:55.483584+0000","last_unstale":"2024-04-17T07:13:57.068452+0000","last_undegraded":"2024-04-17T07:13:57.068452+0000","last_fullsized":"2024-04-17T07:13:57.068452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:49:12.328876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174981+0000","last_change":"2024-04-17T07:11:59.515597+0000","last_active":"2024-04-17T07:13:29.174981+0000","last_peered":"2024-04-17T07:13:29.174981+0000","last_clean":"2024-04-17T07:13:29.174981+0000","last_became_active":"2024-04-17T07:11:59.514413+0000","last_became_peered":"2024-04-17T07:11:59.514413+0000","last_unstale":"2024-04-17T07:13:29.174981+0000","last_undegraded":"2024-04-17T07:13:29.174981+0000","last_fullsized":"2024-04-17T07:13:29.174981+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:45.824127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"91'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205257+0000","last_change":"2024-04-17T07:11:54.449984+0000","last_active":"2024-04-17T07:13:29.205257+0000","last_peered":"2024-04-17T07:13:29.205257+0000","last_clean":"2024-04-17T07:13:29.205257+0000","last_became_active":"2024-04-17T07:11:54.449628+0000","last_became_peered":"2024-04-17T07:11:54.449628+0000","last_unstale":"2024-04-17T07:13:29.205257+0000","last_undegraded":"2024-04-17T07:13:29.205257+0000","last_fullsized":"2024-04-17T07:13:29.205257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:26:03.281684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069434+0000","last_change":"2024-04-17T07:11:55.476912+0000","last_active":"2024-04-17T07:13:57.069434+0000","last_peered":"2024-04-17T07:13:57.069434+0000","last_clean":"2024-04-17T07:13:57.069434+0000","last_became_active":"2024-04-17T07:11:55.471576+0000","last_became_peered":"2024-04-17T07:11:55.471576+0000","last_unstale":"2024-04-17T07:13:57.069434+0000","last_undegraded":"2024-04-17T07:13:57.069434+0000","last_fullsized":"2024-04-17T07:13:57.069434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:44:35.439271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333921+0000","last_change":"2024-04-17T07:11:59.515897+0000","last_active":"2024-04-17T07:13:29.333921+0000","last_peered":"2024-04-17T07:13:29.333921+0000","last_clean":"2024-04-17T07:13:29.333921+0000","last_became_active":"2024-04-17T07:11:59.515349+0000","last_became_peered":"2024-04-17T07:11:59.515349+0000","last_unstale":"2024-04-17T07:13:29.333921+0000","last_undegraded":"2024-04-17T07:13:29.333921+0000","last_fullsized":"2024-04-17T07:13:29.333921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T19:10:00.138286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"91'5","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173683+0000","last_change":"2024-04-17T07:11:54.462369+0000","last_active":"2024-04-17T07:13:29.173683+0000","last_peered":"2024-04-17T07:13:29.173683+0000","last_clean":"2024-04-17T07:13:29.173683+0000","last_became_active":"2024-04-17T07:11:54.460357+0000","last_became_peered":"2024-04-17T07:11:54.460357+0000","last_unstale":"2024-04-17T07:13:29.173683+0000","last_undegraded":"2024-04-17T07:13:29.173683+0000","last_fullsized":"2024-04-17T07:13:29.173683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:38.029980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071612+0000","last_change":"2024-04-17T07:11:55.475874+0000","last_active":"2024-04-17T07:13:57.071612+0000","last_peered":"2024-04-17T07:13:57.071612+0000","last_clean":"2024-04-17T07:13:57.071612+0000","last_became_active":"2024-04-17T07:11:55.475550+0000","last_became_peered":"2024-04-17T07:11:55.475550+0000","last_unstale":"2024-04-17T07:13:57.071612+0000","last_undegraded":"2024-04-17T07:13:57.071612+0000","last_fullsized":"2024-04-17T07:13:57.071612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:51:40.096573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206848+0000","last_change":"2024-04-17T07:11:59.514661+0000","last_active":"2024-04-17T07:13:29.206848+0000","last_peered":"2024-04-17T07:13:29.206848+0000","last_clean":"2024-04-17T07:13:29.206848+0000","last_became_active":"2024-04-17T07:11:59.513030+0000","last_became_peered":"2024-04-17T07:11:59.513030+0000","last_unstale":"2024-04-17T07:13:29.206848+0000","last_undegraded":"2024-04-17T07:13:29.206848+0000","last_fullsized":"2024-04-17T07:13:29.206848+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:55:35.696961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":194,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:02.671857+0000","last_change":"2024-04-17T07:11:54.455279+0000","last_active":"2024-04-17T07:15:02.671857+0000","last_peered":"2024-04-17T07:15:02.671857+0000","last_clean":"2024-04-17T07:15:02.671857+0000","last_became_active":"2024-04-17T07:11:54.453620+0000","last_became_peered":"2024-04-17T07:11:54.453620+0000","last_unstale":"2024-04-17T07:15:02.671857+0000","last_undegraded":"2024-04-17T07:15:02.671857+0000","last_fullsized":"2024-04-17T07:15:02.671857+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:15.227410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":157,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.072155+0000","last_change":"2024-04-17T07:11:55.469579+0000","last_active":"2024-04-17T07:13:57.072155+0000","last_peered":"2024-04-17T07:13:57.072155+0000","last_clean":"2024-04-17T07:13:57.072155+0000","last_became_active":"2024-04-17T07:11:55.468389+0000","last_became_peered":"2024-04-17T07:11:55.468389+0000","last_unstale":"2024-04-17T07:13:57.072155+0000","last_undegraded":"2024-04-17T07:13:57.072155+0000","last_fullsized":"2024-04-17T07:13:57.072155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:12:01.274677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173558+0000","last_change":"2024-04-17T07:11:59.512110+0000","last_active":"2024-04-17T07:13:29.173558+0000","last_peered":"2024-04-17T07:13:29.173558+0000","last_clean":"2024-04-17T07:13:29.173558+0000","last_became_active":"2024-04-17T07:11:59.510573+0000","last_became_peered":"2024-04-17T07:11:59.510573+0000","last_unstale":"2024-04-17T07:13:29.173558+0000","last_undegraded":"2024-04-17T07:13:29.173558+0000","last_fullsized":"2024-04-17T07:13:29.173558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:54:32.515687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"91'17","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333363+0000","last_change":"2024-04-17T07:12:28.767517+0000","last_active":"2024-04-17T07:13:29.333363+0000","last_peered":"2024-04-17T07:13:29.333363+0000","last_clean":"2024-04-17T07:13:29.333363+0000","last_became_active":"2024-04-17T07:12:28.738908+0000","last_became_peered":"2024-04-17T07:12:28.738908+0000","last_unstale":"2024-04-17T07:13:29.333363+0000","last_undegraded":"2024-04-17T07:13:29.333363+0000","last_fullsized":"2024-04-17T07:13:29.333363+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:47:30.170909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.073775+0000","last_change":"2024-04-17T07:11:55.473540+0000","last_active":"2024-04-17T07:13:57.073775+0000","last_peered":"2024-04-17T07:13:57.073775+0000","last_clean":"2024-04-17T07:13:57.073775+0000","last_became_active":"2024-04-17T07:11:55.472436+0000","last_became_peered":"2024-04-17T07:11:55.472436+0000","last_unstale":"2024-04-17T07:13:57.073775+0000","last_undegraded":"2024-04-17T07:13:57.073775+0000","last_fullsized":"2024-04-17T07:13:57.073775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:17:54.317052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172929+0000","last_change":"2024-04-17T07:11:59.521305+0000","last_active":"2024-04-17T07:13:29.172929+0000","last_peered":"2024-04-17T07:13:29.172929+0000","last_clean":"2024-04-17T07:13:29.172929+0000","last_became_active":"2024-04-17T07:11:59.516860+0000","last_became_peered":"2024-04-17T07:11:59.516860+0000","last_unstale":"2024-04-17T07:13:29.172929+0000","last_undegraded":"2024-04-17T07:13:29.172929+0000","last_fullsized":"2024-04-17T07:13:29.172929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:33:23.403218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:11.134134+0000","last_change":"2024-04-17T07:11:54.463378+0000","last_active":"2024-04-17T07:14:11.134134+0000","last_peered":"2024-04-17T07:14:11.134134+0000","last_clean":"2024-04-17T07:14:11.134134+0000","last_became_active":"2024-04-17T07:11:54.462120+0000","last_became_peered":"2024-04-17T07:11:54.462120+0000","last_unstale":"2024-04-17T07:14:11.134134+0000","last_undegraded":"2024-04-17T07:14:11.134134+0000","last_fullsized":"2024-04-17T07:14:11.134134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:50:42.109928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071709+0000","last_change":"2024-04-17T07:11:55.478604+0000","last_active":"2024-04-17T07:13:57.071709+0000","last_peered":"2024-04-17T07:13:57.071709+0000","last_clean":"2024-04-17T07:13:57.071709+0000","last_became_active":"2024-04-17T07:11:55.478252+0000","last_became_peered":"2024-04-17T07:11:55.478252+0000","last_unstale":"2024-04-17T07:13:57.071709+0000","last_undegraded":"2024-04-17T07:13:57.071709+0000","last_fullsized":"2024-04-17T07:13:57.071709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:47:08.365896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206440+0000","last_change":"2024-04-17T07:11:59.518619+0000","last_active":"2024-04-17T07:13:29.206440+0000","last_peered":"2024-04-17T07:13:29.206440+0000","last_clean":"2024-04-17T07:13:29.206440+0000","last_became_active":"2024-04-17T07:11:59.518214+0000","last_became_peered":"2024-04-17T07:11:59.518214+0000","last_unstale":"2024-04-17T07:13:29.206440+0000","last_undegraded":"2024-04-17T07:13:29.206440+0000","last_fullsized":"2024-04-17T07:13:29.206440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:54:46.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"91'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206316+0000","last_change":"2024-04-17T07:11:54.453038+0000","last_active":"2024-04-17T07:13:29.206316+0000","last_peered":"2024-04-17T07:13:29.206316+0000","last_clean":"2024-04-17T07:13:29.206316+0000","last_became_active":"2024-04-17T07:11:54.452619+0000","last_became_peered":"2024-04-17T07:11:54.452619+0000","last_unstale":"2024-04-17T07:13:29.206316+0000","last_undegraded":"2024-04-17T07:13:29.206316+0000","last_fullsized":"2024-04-17T07:13:29.206316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:25.765662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.063521+0000","last_change":"2024-04-17T07:11:55.465990+0000","last_active":"2024-04-17T07:13:57.063521+0000","last_peered":"2024-04-17T07:13:57.063521+0000","last_clean":"2024-04-17T07:13:57.063521+0000","last_became_active":"2024-04-17T07:11:55.465523+0000","last_became_peered":"2024-04-17T07:11:55.465523+0000","last_unstale":"2024-04-17T07:13:57.063521+0000","last_undegraded":"2024-04-17T07:13:57.063521+0000","last_fullsized":"2024-04-17T07:13:57.063521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:39:40.366027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172399+0000","last_change":"2024-04-17T07:11:59.512119+0000","last_active":"2024-04-17T07:13:29.172399+0000","last_peered":"2024-04-17T07:13:29.172399+0000","last_clean":"2024-04-17T07:13:29.172399+0000","last_became_active":"2024-04-17T07:11:59.511460+0000","last_became_peered":"2024-04-17T07:11:59.511460+0000","last_unstale":"2024-04-17T07:13:29.172399+0000","last_undegraded":"2024-04-17T07:13:29.172399+0000","last_fullsized":"2024-04-17T07:13:29.172399+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:58:50.888203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":196,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:05.650024+0000","last_change":"2024-04-17T07:11:54.469346+0000","last_active":"2024-04-17T07:15:05.650024+0000","last_peered":"2024-04-17T07:15:05.650024+0000","last_clean":"2024-04-17T07:15:05.650024+0000","last_became_active":"2024-04-17T07:11:54.468947+0000","last_became_peered":"2024-04-17T07:11:54.468947+0000","last_unstale":"2024-04-17T07:15:05.650024+0000","last_undegraded":"2024-04-17T07:15:05.650024+0000","last_fullsized":"2024-04-17T07:15:05.650024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:41:04.854625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1065,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":158,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078974+0000","last_change":"2024-04-17T07:11:55.471235+0000","last_active":"2024-04-17T07:13:57.078974+0000","last_peered":"2024-04-17T07:13:57.078974+0000","last_clean":"2024-04-17T07:13:57.078974+0000","last_became_active":"2024-04-17T07:11:55.470857+0000","last_became_peered":"2024-04-17T07:11:55.470857+0000","last_unstale":"2024-04-17T07:13:57.078974+0000","last_undegraded":"2024-04-17T07:13:57.078974+0000","last_fullsized":"2024-04-17T07:13:57.078974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:14:21.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172054+0000","last_change":"2024-04-17T07:11:59.524519+0000","last_active":"2024-04-17T07:13:29.172054+0000","last_peered":"2024-04-17T07:13:29.172054+0000","last_clean":"2024-04-17T07:13:29.172054+0000","last_became_active":"2024-04-17T07:11:59.519270+0000","last_became_peered":"2024-04-17T07:11:59.519270+0000","last_unstale":"2024-04-17T07:13:29.172054+0000","last_undegraded":"2024-04-17T07:13:29.172054+0000","last_fullsized":"2024-04-17T07:13:29.172054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:41.080249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070265+0000","last_change":"2024-04-17T07:11:55.473922+0000","last_active":"2024-04-17T07:13:57.070265+0000","last_peered":"2024-04-17T07:13:57.070265+0000","last_clean":"2024-04-17T07:13:57.070265+0000","last_became_active":"2024-04-17T07:11:55.468594+0000","last_became_peered":"2024-04-17T07:11:55.468594+0000","last_unstale":"2024-04-17T07:13:57.070265+0000","last_undegraded":"2024-04-17T07:13:57.070265+0000","last_fullsized":"2024-04-17T07:13:57.070265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:55:21.723493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":191,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:01.667538+0000","last_change":"2024-04-17T07:11:54.473066+0000","last_active":"2024-04-17T07:15:01.667538+0000","last_peered":"2024-04-17T07:15:01.667538+0000","last_clean":"2024-04-17T07:15:01.667538+0000","last_became_active":"2024-04-17T07:11:54.472708+0000","last_became_peered":"2024-04-17T07:11:54.472708+0000","last_unstale":"2024-04-17T07:15:01.667538+0000","last_undegraded":"2024-04-17T07:15:01.667538+0000","last_fullsized":"2024-04-17T07:15:01.667538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:30:27.614819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.209116+0000","last_change":"2024-04-17T07:11:59.516939+0000","last_active":"2024-04-17T07:13:29.209116+0000","last_peered":"2024-04-17T07:13:29.209116+0000","last_clean":"2024-04-17T07:13:29.209116+0000","last_became_active":"2024-04-17T07:11:59.516333+0000","last_became_peered":"2024-04-17T07:11:59.516333+0000","last_unstale":"2024-04-17T07:13:29.209116+0000","last_undegraded":"2024-04-17T07:13:29.209116+0000","last_fullsized":"2024-04-17T07:13:29.209116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:15:15.648831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.082745+0000","last_change":"2024-04-17T07:11:55.481006+0000","last_active":"2024-04-17T07:13:57.082745+0000","last_peered":"2024-04-17T07:13:57.082745+0000","last_clean":"2024-04-17T07:13:57.082745+0000","last_became_active":"2024-04-17T07:11:55.479751+0000","last_became_peered":"2024-04-17T07:11:55.479751+0000","last_unstale":"2024-04-17T07:13:57.082745+0000","last_undegraded":"2024-04-17T07:13:57.082745+0000","last_fullsized":"2024-04-17T07:13:57.082745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:06:13.004650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175099+0000","last_change":"2024-04-17T07:11:54.463106+0000","last_active":"2024-04-17T07:13:29.175099+0000","last_peered":"2024-04-17T07:13:29.175099+0000","last_clean":"2024-04-17T07:13:29.175099+0000","last_became_active":"2024-04-17T07:11:54.462691+0000","last_became_peered":"2024-04-17T07:11:54.462691+0000","last_unstale":"2024-04-17T07:13:29.175099+0000","last_undegraded":"2024-04-17T07:13:29.175099+0000","last_fullsized":"2024-04-17T07:13:29.175099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:34.302019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205049+0000","last_change":"2024-04-17T07:10:14.685883+0000","last_active":"2024-04-17T07:13:29.205049+0000","last_peered":"2024-04-17T07:13:29.205049+0000","last_clean":"2024-04-17T07:13:29.205049+0000","last_became_active":"2024-04-17T07:10:14.684551+0000","last_became_peered":"2024-04-17T07:10:14.684551+0000","last_unstale":"2024-04-17T07:13:29.205049+0000","last_undegraded":"2024-04-17T07:13:29.205049+0000","last_fullsized":"2024-04-17T07:13:29.205049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:05:37.112317+0000","last_clean_scrub_stamp":"2024-04-17T07:05:37.112317+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:07:04.265147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206602+0000","last_change":"2024-04-17T07:12:28.731807+0000","last_active":"2024-04-17T07:13:29.206602+0000","last_peered":"2024-04-17T07:13:29.206602+0000","last_clean":"2024-04-17T07:13:29.206602+0000","last_became_active":"2024-04-17T07:12:28.730656+0000","last_became_peered":"2024-04-17T07:12:28.730656+0000","last_unstale":"2024-04-17T07:13:29.206602+0000","last_undegraded":"2024-04-17T07:13:29.206602+0000","last_fullsized":"2024-04-17T07:13:29.206602+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:39:58.023333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068799+0000","last_change":"2024-04-17T07:12:28.727980+0000","last_active":"2024-04-17T07:13:57.068799+0000","last_peered":"2024-04-17T07:13:57.068799+0000","last_clean":"2024-04-17T07:13:57.068799+0000","last_became_active":"2024-04-17T07:12:28.727060+0000","last_became_peered":"2024-04-17T07:12:28.727060+0000","last_unstale":"2024-04-17T07:13:57.068799+0000","last_undegraded":"2024-04-17T07:13:57.068799+0000","last_fullsized":"2024-04-17T07:13:57.068799+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:41:33.841790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.205324+0000","last_change":"2024-04-17T07:11:54.463180+0000","last_active":"2024-04-17T07:13:29.205324+0000","last_peered":"2024-04-17T07:13:29.205324+0000","last_clean":"2024-04-17T07:13:29.205324+0000","last_became_active":"2024-04-17T07:11:54.462793+0000","last_became_peered":"2024-04-17T07:11:54.462793+0000","last_unstale":"2024-04-17T07:13:29.205324+0000","last_undegraded":"2024-04-17T07:13:29.205324+0000","last_fullsized":"2024-04-17T07:13:29.205324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:19:19.217902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.333323+0000","last_change":"2024-04-17T07:11:59.518355+0000","last_active":"2024-04-17T07:13:29.333323+0000","last_peered":"2024-04-17T07:13:29.333323+0000","last_clean":"2024-04-17T07:13:29.333323+0000","last_became_active":"2024-04-17T07:11:59.517619+0000","last_became_peered":"2024-04-17T07:11:59.517619+0000","last_unstale":"2024-04-17T07:13:29.333323+0000","last_undegraded":"2024-04-17T07:13:29.333323+0000","last_fullsized":"2024-04-17T07:13:29.333323+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:40:36.826021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"91'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.206403+0000","last_change":"2024-04-17T07:11:54.465285+0000","last_active":"2024-04-17T07:13:29.206403+0000","last_peered":"2024-04-17T07:13:29.206403+0000","last_clean":"2024-04-17T07:13:29.206403+0000","last_became_active":"2024-04-17T07:11:54.464818+0000","last_became_peered":"2024-04-17T07:11:54.464818+0000","last_unstale":"2024-04-17T07:13:29.206403+0000","last_undegraded":"2024-04-17T07:13:29.206403+0000","last_fullsized":"2024-04-17T07:13:29.206403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:28:31.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069247+0000","last_change":"2024-04-17T07:11:55.470405+0000","last_active":"2024-04-17T07:13:57.069247+0000","last_peered":"2024-04-17T07:13:57.069247+0000","last_clean":"2024-04-17T07:13:57.069247+0000","last_became_active":"2024-04-17T07:11:55.470058+0000","last_became_peered":"2024-04-17T07:11:55.470058+0000","last_unstale":"2024-04-17T07:13:57.069247+0000","last_undegraded":"2024-04-17T07:13:57.069247+0000","last_fullsized":"2024-04-17T07:13:57.069247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:41:17.416775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.202954+0000","last_change":"2024-04-17T07:11:59.522040+0000","last_active":"2024-04-17T07:13:29.202954+0000","last_peered":"2024-04-17T07:13:29.202954+0000","last_clean":"2024-04-17T07:13:29.202954+0000","last_became_active":"2024-04-17T07:11:59.519771+0000","last_became_peered":"2024-04-17T07:11:59.519771+0000","last_unstale":"2024-04-17T07:13:29.202954+0000","last_undegraded":"2024-04-17T07:13:29.202954+0000","last_fullsized":"2024-04-17T07:13:29.202954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:32:58.203737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'17","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:00.514634+0000","last_change":"2024-04-17T07:11:54.457954+0000","last_active":"2024-04-17T07:15:00.514634+0000","last_peered":"2024-04-17T07:15:00.514634+0000","last_clean":"2024-04-17T07:15:00.514634+0000","last_became_active":"2024-04-17T07:11:54.457560+0000","last_became_peered":"2024-04-17T07:11:54.457560+0000","last_unstale":"2024-04-17T07:15:00.514634+0000","last_undegraded":"2024-04-17T07:15:00.514634+0000","last_fullsized":"2024-04-17T07:15:00.514634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:01:31.919730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068908+0000","last_change":"2024-04-17T07:11:55.480475+0000","last_active":"2024-04-17T07:13:57.068908+0000","last_peered":"2024-04-17T07:13:57.068908+0000","last_clean":"2024-04-17T07:13:57.068908+0000","last_became_active":"2024-04-17T07:11:55.480060+0000","last_became_peered":"2024-04-17T07:11:55.480060+0000","last_unstale":"2024-04-17T07:13:57.068908+0000","last_undegraded":"2024-04-17T07:13:57.068908+0000","last_fullsized":"2024-04-17T07:13:57.068908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:30:23.600244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.203796+0000","last_change":"2024-04-17T07:11:59.525422+0000","last_active":"2024-04-17T07:13:29.203796+0000","last_peered":"2024-04-17T07:13:29.203796+0000","last_clean":"2024-04-17T07:13:29.203796+0000","last_became_active":"2024-04-17T07:11:59.523649+0000","last_became_peered":"2024-04-17T07:11:59.523649+0000","last_unstale":"2024-04-17T07:13:29.203796+0000","last_undegraded":"2024-04-17T07:13:29.203796+0000","last_fullsized":"2024-04-17T07:13:29.203796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:27:16.653752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"91'7","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.176127+0000","last_change":"2024-04-17T07:11:54.465824+0000","last_active":"2024-04-17T07:13:29.176127+0000","last_peered":"2024-04-17T07:13:29.176127+0000","last_clean":"2024-04-17T07:13:29.176127+0000","last_became_active":"2024-04-17T07:11:54.465348+0000","last_became_peered":"2024-04-17T07:11:54.465348+0000","last_unstale":"2024-04-17T07:13:29.176127+0000","last_undegraded":"2024-04-17T07:13:29.176127+0000","last_fullsized":"2024-04-17T07:13:29.176127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:29:17.729903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081497+0000","last_change":"2024-04-17T07:11:55.481666+0000","last_active":"2024-04-17T07:13:57.081497+0000","last_peered":"2024-04-17T07:13:57.081497+0000","last_clean":"2024-04-17T07:13:57.081497+0000","last_became_active":"2024-04-17T07:11:55.480332+0000","last_became_peered":"2024-04-17T07:11:55.480332+0000","last_unstale":"2024-04-17T07:13:57.081497+0000","last_undegraded":"2024-04-17T07:13:57.081497+0000","last_fullsized":"2024-04-17T07:13:57.081497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:06:19.825407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175613+0000","last_change":"2024-04-17T07:11:59.527850+0000","last_active":"2024-04-17T07:13:29.175613+0000","last_peered":"2024-04-17T07:13:29.175613+0000","last_clean":"2024-04-17T07:13:29.175613+0000","last_became_active":"2024-04-17T07:11:59.527306+0000","last_became_peered":"2024-04-17T07:11:59.527306+0000","last_unstale":"2024-04-17T07:13:29.175613+0000","last_undegraded":"2024-04-17T07:13:29.175613+0000","last_fullsized":"2024-04-17T07:13:29.175613+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:53:35.906222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204229+0000","last_change":"2024-04-17T07:11:54.455862+0000","last_active":"2024-04-17T07:13:29.204229+0000","last_peered":"2024-04-17T07:13:29.204229+0000","last_clean":"2024-04-17T07:13:29.204229+0000","last_became_active":"2024-04-17T07:11:54.453904+0000","last_became_peered":"2024-04-17T07:11:54.453904+0000","last_unstale":"2024-04-17T07:13:29.204229+0000","last_undegraded":"2024-04-17T07:13:29.204229+0000","last_fullsized":"2024-04-17T07:13:29.204229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:44:01.471860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.060326+0000","last_change":"2024-04-17T07:11:55.470315+0000","last_active":"2024-04-17T07:13:57.060326+0000","last_peered":"2024-04-17T07:13:57.060326+0000","last_clean":"2024-04-17T07:13:57.060326+0000","last_became_active":"2024-04-17T07:11:55.469829+0000","last_became_peered":"2024-04-17T07:11:55.469829+0000","last_unstale":"2024-04-17T07:13:57.060326+0000","last_undegraded":"2024-04-17T07:13:57.060326+0000","last_fullsized":"2024-04-17T07:13:57.060326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:33:48.197546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.210025+0000","last_change":"2024-04-17T07:12:29.125447+0000","last_active":"2024-04-17T07:13:29.210025+0000","last_peered":"2024-04-17T07:13:29.210025+0000","last_clean":"2024-04-17T07:13:29.210025+0000","last_became_active":"2024-04-17T07:12:29.124551+0000","last_became_peered":"2024-04-17T07:12:29.124551+0000","last_unstale":"2024-04-17T07:13:29.210025+0000","last_undegraded":"2024-04-17T07:13:29.210025+0000","last_fullsized":"2024-04-17T07:13:29.210025+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:57:09.489755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,2,5],"acting":[10,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.9","version":"91'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204318+0000","last_change":"2024-04-17T07:11:54.453117+0000","last_active":"2024-04-17T07:13:29.204318+0000","last_peered":"2024-04-17T07:13:29.204318+0000","last_clean":"2024-04-17T07:13:29.204318+0000","last_became_active":"2024-04-17T07:11:54.452632+0000","last_became_peered":"2024-04-17T07:11:54.452632+0000","last_unstale":"2024-04-17T07:13:29.204318+0000","last_undegraded":"2024-04-17T07:13:29.204318+0000","last_fullsized":"2024-04-17T07:13:29.204318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:24:09.366149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078286+0000","last_change":"2024-04-17T07:11:55.469991+0000","last_active":"2024-04-17T07:13:57.078286+0000","last_peered":"2024-04-17T07:13:57.078286+0000","last_clean":"2024-04-17T07:13:57.078286+0000","last_became_active":"2024-04-17T07:11:55.469546+0000","last_became_peered":"2024-04-17T07:11:55.469546+0000","last_unstale":"2024-04-17T07:13:57.078286+0000","last_undegraded":"2024-04-17T07:13:57.078286+0000","last_fullsized":"2024-04-17T07:13:57.078286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:43:02.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175116+0000","last_change":"2024-04-17T07:11:59.519817+0000","last_active":"2024-04-17T07:13:29.175116+0000","last_peered":"2024-04-17T07:13:29.175116+0000","last_clean":"2024-04-17T07:13:29.175116+0000","last_became_active":"2024-04-17T07:11:59.519108+0000","last_became_peered":"2024-04-17T07:11:59.519108+0000","last_unstale":"2024-04-17T07:13:29.175116+0000","last_undegraded":"2024-04-17T07:13:29.175116+0000","last_fullsized":"2024-04-17T07:13:29.175116+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:05:58.378837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'6","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:14:59.206828+0000","last_change":"2024-04-17T07:11:54.463618+0000","last_active":"2024-04-17T07:14:59.206828+0000","last_peered":"2024-04-17T07:14:59.206828+0000","last_clean":"2024-04-17T07:14:59.206828+0000","last_became_active":"2024-04-17T07:11:54.463257+0000","last_became_peered":"2024-04-17T07:11:54.463257+0000","last_unstale":"2024-04-17T07:14:59.206828+0000","last_undegraded":"2024-04-17T07:14:59.206828+0000","last_fullsized":"2024-04-17T07:14:59.206828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:04:20.135338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077111+0000","last_change":"2024-04-17T07:11:55.487471+0000","last_active":"2024-04-17T07:13:57.077111+0000","last_peered":"2024-04-17T07:13:57.077111+0000","last_clean":"2024-04-17T07:13:57.077111+0000","last_became_active":"2024-04-17T07:11:55.487024+0000","last_became_peered":"2024-04-17T07:11:55.487024+0000","last_unstale":"2024-04-17T07:13:57.077111+0000","last_undegraded":"2024-04-17T07:13:57.077111+0000","last_fullsized":"2024-04-17T07:13:57.077111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:31:36.851210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.175245+0000","last_change":"2024-04-17T07:11:59.527017+0000","last_active":"2024-04-17T07:13:29.175245+0000","last_peered":"2024-04-17T07:13:29.175245+0000","last_clean":"2024-04-17T07:13:29.175245+0000","last_became_active":"2024-04-17T07:11:59.526541+0000","last_became_peered":"2024-04-17T07:11:59.526541+0000","last_unstale":"2024-04-17T07:13:29.175245+0000","last_undegraded":"2024-04-17T07:13:29.175245+0000","last_fullsized":"2024-04-17T07:13:29.175245+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:30:54.411647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204636+0000","last_change":"2024-04-17T07:11:54.465671+0000","last_active":"2024-04-17T07:13:29.204636+0000","last_peered":"2024-04-17T07:13:29.204636+0000","last_clean":"2024-04-17T07:13:29.204636+0000","last_became_active":"2024-04-17T07:11:54.464902+0000","last_became_peered":"2024-04-17T07:11:54.464902+0000","last_unstale":"2024-04-17T07:13:29.204636+0000","last_undegraded":"2024-04-17T07:13:29.204636+0000","last_fullsized":"2024-04-17T07:13:29.204636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:13:38.232965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084718+0000","last_change":"2024-04-17T07:11:55.484259+0000","last_active":"2024-04-17T07:13:57.084718+0000","last_peered":"2024-04-17T07:13:57.084718+0000","last_clean":"2024-04-17T07:13:57.084718+0000","last_became_active":"2024-04-17T07:11:55.483911+0000","last_became_peered":"2024-04-17T07:11:55.483911+0000","last_unstale":"2024-04-17T07:13:57.084718+0000","last_undegraded":"2024-04-17T07:13:57.084718+0000","last_fullsized":"2024-04-17T07:13:57.084718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:54.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.204244+0000","last_change":"2024-04-17T07:11:59.525369+0000","last_active":"2024-04-17T07:13:29.204244+0000","last_peered":"2024-04-17T07:13:29.204244+0000","last_clean":"2024-04-17T07:13:29.204244+0000","last_became_active":"2024-04-17T07:11:59.524809+0000","last_became_peered":"2024-04-17T07:11:59.524809+0000","last_unstale":"2024-04-17T07:13:29.204244+0000","last_undegraded":"2024-04-17T07:13:29.204244+0000","last_fullsized":"2024-04-17T07:13:29.204244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:36:20.511001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"93'7","reported_seq":198,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:15:04.678820+0000","last_change":"2024-04-17T07:11:54.459079+0000","last_active":"2024-04-17T07:15:04.678820+0000","last_peered":"2024-04-17T07:15:04.678820+0000","last_clean":"2024-04-17T07:15:04.678820+0000","last_became_active":"2024-04-17T07:11:54.457008+0000","last_became_peered":"2024-04-17T07:11:54.457008+0000","last_unstale":"2024-04-17T07:15:04.678820+0000","last_undegraded":"2024-04-17T07:15:04.678820+0000","last_fullsized":"2024-04-17T07:15:04.678820+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:48:36.084512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":159,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084193+0000","last_change":"2024-04-17T07:11:55.484453+0000","last_active":"2024-04-17T07:13:57.084193+0000","last_peered":"2024-04-17T07:13:57.084193+0000","last_clean":"2024-04-17T07:13:57.084193+0000","last_became_active":"2024-04-17T07:11:55.483152+0000","last_became_peered":"2024-04-17T07:11:55.483152+0000","last_unstale":"2024-04-17T07:13:57.084193+0000","last_undegraded":"2024-04-17T07:13:57.084193+0000","last_fullsized":"2024-04-17T07:13:57.084193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:26:32.224868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.202670+0000","last_change":"2024-04-17T07:11:59.521487+0000","last_active":"2024-04-17T07:13:29.202670+0000","last_peered":"2024-04-17T07:13:29.202670+0000","last_clean":"2024-04-17T07:13:29.202670+0000","last_became_active":"2024-04-17T07:11:59.521060+0000","last_became_peered":"2024-04-17T07:11:59.521060+0000","last_unstale":"2024-04-17T07:13:29.202670+0000","last_undegraded":"2024-04-17T07:13:29.202670+0000","last_fullsized":"2024-04-17T07:13:29.202670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:39:06.692434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.174602+0000","last_change":"2024-04-17T07:11:54.459368+0000","last_active":"2024-04-17T07:13:29.174602+0000","last_peered":"2024-04-17T07:13:29.174602+0000","last_clean":"2024-04-17T07:13:29.174602+0000","last_became_active":"2024-04-17T07:11:54.458973+0000","last_became_peered":"2024-04-17T07:11:54.458973+0000","last_unstale":"2024-04-17T07:13:29.174602+0000","last_undegraded":"2024-04-17T07:13:29.174602+0000","last_fullsized":"2024-04-17T07:13:29.174602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:01:22.034110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.072233+0000","last_change":"2024-04-17T07:11:55.470648+0000","last_active":"2024-04-17T07:13:57.072233+0000","last_peered":"2024-04-17T07:13:57.072233+0000","last_clean":"2024-04-17T07:13:57.072233+0000","last_became_active":"2024-04-17T07:11:55.469666+0000","last_became_peered":"2024-04-17T07:11:55.469666+0000","last_unstale":"2024-04-17T07:13:57.072233+0000","last_undegraded":"2024-04-17T07:13:57.072233+0000","last_fullsized":"2024-04-17T07:13:57.072233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:59:59.208872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.172389+0000","last_change":"2024-04-17T07:11:59.524027+0000","last_active":"2024-04-17T07:13:29.172389+0000","last_peered":"2024-04-17T07:13:29.172389+0000","last_clean":"2024-04-17T07:13:29.172389+0000","last_became_active":"2024-04-17T07:11:59.522764+0000","last_became_peered":"2024-04-17T07:11:59.522764+0000","last_unstale":"2024-04-17T07:13:29.172389+0000","last_undegraded":"2024-04-17T07:13:29.172389+0000","last_fullsized":"2024-04-17T07:13:29.172389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:58.472667+0000","last_clean_scrub_stamp":"2024-04-17T07:11:58.472667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:18:40.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"91'5","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:29.173260+0000","last_change":"2024-04-17T07:11:54.472884+0000","last_active":"2024-04-17T07:13:29.173260+0000","last_peered":"2024-04-17T07:13:29.173260+0000","last_clean":"2024-04-17T07:13:29.173260+0000","last_became_active":"2024-04-17T07:11:54.472305+0000","last_became_peered":"2024-04-17T07:11:54.472305+0000","last_unstale":"2024-04-17T07:13:29.173260+0000","last_undegraded":"2024-04-17T07:13:29.173260+0000","last_fullsized":"2024-04-17T07:13:29.173260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:53.415144+0000","last_clean_scrub_stamp":"2024-04-17T07:11:53.415144+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:23:21.031442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070727+0000","last_change":"2024-04-17T07:11:55.479982+0000","last_active":"2024-04-17T07:13:57.070727+0000","last_peered":"2024-04-17T07:13:57.070727+0000","last_clean":"2024-04-17T07:13:57.070727+0000","last_became_active":"2024-04-17T07:11:55.477594+0000","last_became_peered":"2024-04-17T07:11:55.477594+0000","last_unstale":"2024-04-17T07:13:57.070727+0000","last_undegraded":"2024-04-17T07:13:57.070727+0000","last_fullsized":"2024-04-17T07:13:57.070727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:25:23.507312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071666+0000","last_change":"2024-04-17T07:11:55.480941+0000","last_active":"2024-04-17T07:13:57.071666+0000","last_peered":"2024-04-17T07:13:57.071666+0000","last_clean":"2024-04-17T07:13:57.071666+0000","last_became_active":"2024-04-17T07:11:55.480180+0000","last_became_peered":"2024-04-17T07:11:55.480180+0000","last_unstale":"2024-04-17T07:13:57.071666+0000","last_undegraded":"2024-04-17T07:13:57.071666+0000","last_fullsized":"2024-04-17T07:13:57.071666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:36:16.321806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.071075+0000","last_change":"2024-04-17T07:11:55.477663+0000","last_active":"2024-04-17T07:13:57.071075+0000","last_peered":"2024-04-17T07:13:57.071075+0000","last_clean":"2024-04-17T07:13:57.071075+0000","last_became_active":"2024-04-17T07:11:55.477304+0000","last_became_peered":"2024-04-17T07:11:55.477304+0000","last_unstale":"2024-04-17T07:13:57.071075+0000","last_undegraded":"2024-04-17T07:13:57.071075+0000","last_fullsized":"2024-04-17T07:13:57.071075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:03:00.796633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077531+0000","last_change":"2024-04-17T07:11:55.475536+0000","last_active":"2024-04-17T07:13:57.077531+0000","last_peered":"2024-04-17T07:13:57.077531+0000","last_clean":"2024-04-17T07:13:57.077531+0000","last_became_active":"2024-04-17T07:11:55.475138+0000","last_became_peered":"2024-04-17T07:11:55.475138+0000","last_unstale":"2024-04-17T07:13:57.077531+0000","last_undegraded":"2024-04-17T07:13:57.077531+0000","last_fullsized":"2024-04-17T07:13:57.077531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:36:47.849076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081347+0000","last_change":"2024-04-17T07:11:55.483742+0000","last_active":"2024-04-17T07:13:57.081347+0000","last_peered":"2024-04-17T07:13:57.081347+0000","last_clean":"2024-04-17T07:13:57.081347+0000","last_became_active":"2024-04-17T07:11:55.483044+0000","last_became_peered":"2024-04-17T07:11:55.483044+0000","last_unstale":"2024-04-17T07:13:57.081347+0000","last_undegraded":"2024-04-17T07:13:57.081347+0000","last_fullsized":"2024-04-17T07:13:57.081347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:59:18.972137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.078285+0000","last_change":"2024-04-17T07:11:55.471266+0000","last_active":"2024-04-17T07:13:57.078285+0000","last_peered":"2024-04-17T07:13:57.078285+0000","last_clean":"2024-04-17T07:13:57.078285+0000","last_became_active":"2024-04-17T07:11:55.470897+0000","last_became_peered":"2024-04-17T07:11:55.470897+0000","last_unstale":"2024-04-17T07:13:57.078285+0000","last_undegraded":"2024-04-17T07:13:57.078285+0000","last_fullsized":"2024-04-17T07:13:57.078285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:54:25.811064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.066947+0000","last_change":"2024-04-17T07:11:55.475860+0000","last_active":"2024-04-17T07:13:57.066947+0000","last_peered":"2024-04-17T07:13:57.066947+0000","last_clean":"2024-04-17T07:13:57.066947+0000","last_became_active":"2024-04-17T07:11:55.475522+0000","last_became_peered":"2024-04-17T07:11:55.475522+0000","last_unstale":"2024-04-17T07:13:57.066947+0000","last_undegraded":"2024-04-17T07:13:57.066947+0000","last_fullsized":"2024-04-17T07:13:57.066947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:00:41.548964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.081296+0000","last_change":"2024-04-17T07:11:55.478815+0000","last_active":"2024-04-17T07:13:57.081296+0000","last_peered":"2024-04-17T07:13:57.081296+0000","last_clean":"2024-04-17T07:13:57.081296+0000","last_became_active":"2024-04-17T07:11:55.478347+0000","last_became_peered":"2024-04-17T07:11:55.478347+0000","last_unstale":"2024-04-17T07:13:57.081296+0000","last_undegraded":"2024-04-17T07:13:57.081296+0000","last_fullsized":"2024-04-17T07:13:57.081296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:16.465511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069021+0000","last_change":"2024-04-17T07:11:55.474208+0000","last_active":"2024-04-17T07:13:57.069021+0000","last_peered":"2024-04-17T07:13:57.069021+0000","last_clean":"2024-04-17T07:13:57.069021+0000","last_became_active":"2024-04-17T07:11:55.472042+0000","last_became_peered":"2024-04-17T07:11:55.472042+0000","last_unstale":"2024-04-17T07:13:57.069021+0000","last_undegraded":"2024-04-17T07:13:57.069021+0000","last_fullsized":"2024-04-17T07:13:57.069021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:37:24.248506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.069995+0000","last_change":"2024-04-17T07:11:55.477257+0000","last_active":"2024-04-17T07:13:57.069995+0000","last_peered":"2024-04-17T07:13:57.069995+0000","last_clean":"2024-04-17T07:13:57.069995+0000","last_became_active":"2024-04-17T07:11:55.473491+0000","last_became_peered":"2024-04-17T07:11:55.473491+0000","last_unstale":"2024-04-17T07:13:57.069995+0000","last_undegraded":"2024-04-17T07:13:57.069995+0000","last_fullsized":"2024-04-17T07:13:57.069995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:43:33.737485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.068915+0000","last_change":"2024-04-17T07:11:55.474930+0000","last_active":"2024-04-17T07:13:57.068915+0000","last_peered":"2024-04-17T07:13:57.068915+0000","last_clean":"2024-04-17T07:13:57.068915+0000","last_became_active":"2024-04-17T07:11:55.472007+0000","last_became_peered":"2024-04-17T07:11:55.472007+0000","last_unstale":"2024-04-17T07:13:57.068915+0000","last_undegraded":"2024-04-17T07:13:57.068915+0000","last_fullsized":"2024-04-17T07:13:57.068915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:07.111267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.084510+0000","last_change":"2024-04-17T07:11:55.492422+0000","last_active":"2024-04-17T07:13:57.084510+0000","last_peered":"2024-04-17T07:13:57.084510+0000","last_clean":"2024-04-17T07:13:57.084510+0000","last_became_active":"2024-04-17T07:11:55.491906+0000","last_became_peered":"2024-04-17T07:11:55.491906+0000","last_unstale":"2024-04-17T07:13:57.084510+0000","last_undegraded":"2024-04-17T07:13:57.084510+0000","last_fullsized":"2024-04-17T07:13:57.084510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:13:05.322586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.077939+0000","last_change":"2024-04-17T07:11:55.479436+0000","last_active":"2024-04-17T07:13:57.077939+0000","last_peered":"2024-04-17T07:13:57.077939+0000","last_clean":"2024-04-17T07:13:57.077939+0000","last_became_active":"2024-04-17T07:11:55.478982+0000","last_became_peered":"2024-04-17T07:11:55.478982+0000","last_unstale":"2024-04-17T07:13:57.077939+0000","last_undegraded":"2024-04-17T07:13:57.077939+0000","last_fullsized":"2024-04-17T07:13:57.077939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:40:20.663457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-17T07:13:57.070325+0000","last_change":"2024-04-17T07:11:55.476708+0000","last_active":"2024-04-17T07:13:57.070325+0000","last_peered":"2024-04-17T07:13:57.070325+0000","last_clean":"2024-04-17T07:13:57.070325+0000","last_became_active":"2024-04-17T07:11:55.476344+0000","last_became_peered":"2024-04-17T07:11:55.476344+0000","last_unstale":"2024-04-17T07:13:57.070325+0000","last_undegraded":"2024-04-17T07:13:57.070325+0000","last_fullsized":"2024-04-17T07:13:57.070325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T07:11:54.419611+0000","last_clean_scrub_stamp":"2024-04-17T07:11:54.419611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:16:58.874987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":279575,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76685,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1576,"num_read_kb":862,"num_write":274,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":235095,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383734,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":521108,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645374,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":1174853,"data_compressed":64060,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808901,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":524154,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070540,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":520324,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332179,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1118505,"data_compressed":62211,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":516039,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":588652,"data_compressed":28497,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738499,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":517886,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":587010,"data_compressed":28497,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":515270,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920872,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":520113,"data_compressed":26648,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757345,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":1113133,"data_compressed":62211,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33920,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35563,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73812,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72170,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":69645,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T07:15:08.316 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:15:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.569+0000 7f164ea00640 1 -- 172.21.15.86:0/1465206141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1648109f90 msgr2=0x7f1648110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.569+0000 7f164ea00640 1 --2- 172.21.15.86:0/1465206141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1648109f90 0x7f1648110820 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1640009900 tx=0x7f164002bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.569+0000 7f164ea00640 1 -- 172.21.15.86:0/1465206141 shutdown_connections 2024-04-17T07:15:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.569+0000 7f164ea00640 1 --2- 172.21.15.86:0/1465206141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1648109f90 0x7f1648110820 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.569+0000 7f164ea00640 1 --2- 172.21.15.86:0/1465206141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16481095d0 0x7f1648109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.569+0000 7f164ea00640 1 --2- 172.21.15.86:0/1465206141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16481083e0 0x7f1648108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.569+0000 7f164ea00640 1 -- 172.21.15.86:0/1465206141 >> 172.21.15.86:0/1465206141 conn(0x7f16480773a0 msgr2=0x7f16480797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.569+0000 7f164ea00640 1 -- 172.21.15.86:0/1465206141 shutdown_connections 2024-04-17T07:15:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.569+0000 7f164ea00640 1 -- 172.21.15.86:0/1465206141 wait complete. 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.570+0000 7f164ea00640 1 Processor -- start 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.570+0000 7f164ea00640 1 -- start start 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.570+0000 7f164ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16481083e0 0x7f164819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.570+0000 7f164ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16481095d0 0x7f164819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.570+0000 7f164ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1648109f90 0x7f16481a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.570+0000 7f164ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1648113f20 con 0x7f1648109f90 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.570+0000 7f164ea00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1648113da0 con 0x7f16481083e0 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.570+0000 7f164ea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16481140a0 con 0x7f16481095d0 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1648109f90 0x7f16481a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1648109f90 0x7f16481a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37850/0 (socket says 172.21.15.86:37850) 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164e000640 1 -- 172.21.15.86:0/1682149436 learned_addr learned my addr 172.21.15.86:0/1682149436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164d600640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16481083e0 0x7f164819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164e000640 1 -- 172.21.15.86:0/1682149436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16481095d0 msgr2=0x7f164819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164e000640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16481095d0 0x7f164819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164e000640 1 -- 172.21.15.86:0/1682149436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16481083e0 msgr2=0x7f164819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164e000640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16481083e0 0x7f164819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164e000640 1 -- 172.21.15.86:0/1682149436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16481a61f0 con 0x7f1648109f90 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164d600640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16481083e0 0x7f164819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.571+0000 7f164e000640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1648109f90 0x7f16481a5b80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1640009230 tx=0x7f16400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.572+0000 7f1646000640 1 -- 172.21.15.86:0/1682149436 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640043020 con 0x7f1648109f90 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.572+0000 7f1646000640 1 -- 172.21.15.86:0/1682149436 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1640004aa0 con 0x7f1648109f90 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.572+0000 7f164ea00640 1 -- 172.21.15.86:0/1682149436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16481a6480 con 0x7f1648109f90 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.572+0000 7f1646000640 1 -- 172.21.15.86:0/1682149436 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640003c60 con 0x7f1648109f90 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.572+0000 7f164ea00640 1 -- 172.21.15.86:0/1682149436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16481a69d0 con 0x7f1648109f90 2024-04-17T07:15:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.573+0000 7f164ea00640 1 -- 172.21.15.86:0/1682149436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f164810f780 con 0x7f1648109f90 2024-04-17T07:15:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.575+0000 7f1646000640 1 -- 172.21.15.86:0/1682149436 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f1640046020 con 0x7f1648109f90 2024-04-17T07:15:08.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.576+0000 7f1646000640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16300a3c40 0x7f16300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:08.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.576+0000 7f1646000640 1 -- 172.21.15.86:0/1682149436 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f16400f2ca0 con 0x7f1648109f90 2024-04-17T07:15:08.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.577+0000 7f164d600640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16300a3c40 0x7f16300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:08.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.577+0000 7f164d600640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16300a3c40 0x7f16300a6100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1648109430 tx=0x7f163c009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:08.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.580+0000 7f1646000640 1 -- 172.21.15.86:0/1682149436 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f164004d020 con 0x7f1648109f90 2024-04-17T07:15:08.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:08 smithi086 ceph-mon[31733]: from='client.15711 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:15:08.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:08 smithi098 ceph-mon[35418]: from='client.15711 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:15:08.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:08 smithi103 ceph-mon[35495]: from='client.15711 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:15:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.787+0000 7f164ea00640 1 -- 172.21.15.86:0/1682149436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f164802d820 con 0x7f1648109f90 2024-04-17T07:15:08.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.792+0000 7f1646000640 1 -- 172.21.15.86:0/1682149436 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7f16400bf250 con 0x7f1648109f90 2024-04-17T07:15:08.791 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:08.791 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:15:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 -- 172.21.15.86:0/1682149436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16300a3c40 msgr2=0x7f16300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16300a3c40 0x7f16300a6100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1648109430 tx=0x7f163c009400 comp rx=0 tx=0).stop 2024-04-17T07:15:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 -- 172.21.15.86:0/1682149436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1648109f90 msgr2=0x7f16481a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1648109f90 0x7f16481a5b80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1640009230 tx=0x7f16400042f0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 -- 172.21.15.86:0/1682149436 shutdown_connections 2024-04-17T07:15:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16300a3c40 0x7f16300a6100 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1648109f90 0x7f16481a5b80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16481095d0 0x7f164819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 --2- 172.21.15.86:0/1682149436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16481083e0 0x7f164819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 -- 172.21.15.86:0/1682149436 >> 172.21.15.86:0/1682149436 conn(0x7f16480773a0 msgr2=0x7f164810c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 -- 172.21.15.86:0/1682149436 shutdown_connections 2024-04-17T07:15:08.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:08.795+0000 7f164ea00640 1 -- 172.21.15.86:0/1682149436 wait complete. 2024-04-17T07:15:08.814 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:15:08.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:15:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.033+0000 7f6f80800640 1 -- 172.21.15.86:0/809217813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78077b60 msgr2=0x7f6f78077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.033+0000 7f6f80800640 1 --2- 172.21.15.86:0/809217813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78077b60 0x7f6f78077fc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009900 tx=0x7f6f6802bc40 comp rx=0 tx=0).stop 2024-04-17T07:15:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.033+0000 7f6f80800640 1 -- 172.21.15.86:0/809217813 shutdown_connections 2024-04-17T07:15:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.033+0000 7f6f80800640 1 --2- 172.21.15.86:0/809217813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f78078500 0x7f6f78078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.033+0000 7f6f80800640 1 --2- 172.21.15.86:0/809217813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78077b60 0x7f6f78077fc0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.033+0000 7f6f80800640 1 --2- 172.21.15.86:0/809217813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78076920 0x7f6f78076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.033+0000 7f6f80800640 1 -- 172.21.15.86:0/809217813 >> 172.21.15.86:0/809217813 conn(0x7f6f780ff750 msgr2=0x7f6f78101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:09.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.033+0000 7f6f80800640 1 -- 172.21.15.86:0/809217813 shutdown_connections 2024-04-17T07:15:09.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.033+0000 7f6f80800640 1 -- 172.21.15.86:0/809217813 wait complete. 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.034+0000 7f6f80800640 1 Processor -- start 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.034+0000 7f6f80800640 1 -- start start 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f80800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f78076920 0x7f6f7819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f80800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78077b60 0x7f6f7819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f80800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78078500 0x7f6f781a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f80800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78114140 con 0x7f6f78078500 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f80800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78113fc0 con 0x7f6f78077b60 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f80800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f781142c0 con 0x7f6f78076920 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78078500 0x7f6f781a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78078500 0x7f6f781a5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37862/0 (socket says 172.21.15.86:37862) 2024-04-17T07:15:09.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f7e800640 1 -- 172.21.15.86:0/2112576986 learned_addr learned my addr 172.21.15.86:0/2112576986 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:09.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f7de00640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f78076920 0x7f6f7819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f7d400640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78077b60 0x7f6f7819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f7e800640 1 -- 172.21.15.86:0/2112576986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f78076920 msgr2=0x7f6f7819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f7e800640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f78076920 0x7f6f7819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f7e800640 1 -- 172.21.15.86:0/2112576986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78077b60 msgr2=0x7f6f7819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.035+0000 7f6f7e800640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78077b60 0x7f6f7819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.036+0000 7f6f7e800640 1 -- 172.21.15.86:0/2112576986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f781a6340 con 0x7f6f78078500 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.036+0000 7f6f7e800640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78078500 0x7f6f781a5cd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400c970 tx=0x7f6f7400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.036+0000 7f6f6ea00640 1 -- 172.21.15.86:0/2112576986 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74007bb0 con 0x7f6f78078500 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.036+0000 7f6f6ea00640 1 -- 172.21.15.86:0/2112576986 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f74007d50 con 0x7f6f78078500 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.036+0000 7f6f6ea00640 1 -- 172.21.15.86:0/2112576986 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74018700 con 0x7f6f78078500 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.036+0000 7f6f80800640 1 -- 172.21.15.86:0/2112576986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f781a6630 con 0x7f6f78078500 2024-04-17T07:15:09.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.036+0000 7f6f80800640 1 -- 172.21.15.86:0/2112576986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f781a6b50 con 0x7f6f78078500 2024-04-17T07:15:09.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.037+0000 7f6f80800640 1 -- 172.21.15.86:0/2112576986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f78110b30 con 0x7f6f78078500 2024-04-17T07:15:09.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.040+0000 7f6f6ea00640 1 -- 172.21.15.86:0/2112576986 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f6f7400fa20 con 0x7f6f78078500 2024-04-17T07:15:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.041+0000 7f6f6ea00640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6f640a3cc0 0x7f6f640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.042+0000 7f6f6ea00640 1 -- 172.21.15.86:0/2112576986 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6f740d09f0 con 0x7f6f78078500 2024-04-17T07:15:09.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.042+0000 7f6f7de00640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6f640a3cc0 0x7f6f640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.042+0000 7f6f7de00640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6f640a3cc0 0x7f6f640a6180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6f70005fd0 tx=0x7f6f70009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:09.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.046+0000 7f6f6ea00640 1 -- 172.21.15.86:0/2112576986 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6f74014030 con 0x7f6f78078500 2024-04-17T07:15:09.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.249+0000 7f6f80800640 1 -- 172.21.15.86:0/2112576986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f78062b60 con 0x7f6f78078500 2024-04-17T07:15:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.250+0000 7f6f6ea00640 1 -- 172.21.15.86:0/2112576986 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f6f7409cef0 con 0x7f6f78078500 2024-04-17T07:15:09.249 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:15:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 -- 172.21.15.86:0/2112576986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6f640a3cc0 msgr2=0x7f6f640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6f640a3cc0 0x7f6f640a6180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6f70005fd0 tx=0x7f6f70009870 comp rx=0 tx=0).stop 2024-04-17T07:15:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 -- 172.21.15.86:0/2112576986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78078500 msgr2=0x7f6f781a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78078500 0x7f6f781a5cd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400c970 tx=0x7f6f7400ce30 comp rx=0 tx=0).stop 2024-04-17T07:15:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 -- 172.21.15.86:0/2112576986 shutdown_connections 2024-04-17T07:15:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6f640a3cc0 0x7f6f640a6180 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f78078500 0x7f6f781a5cd0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78077b60 0x7f6f7819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 --2- 172.21.15.86:0/2112576986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f78076920 0x7f6f7819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 -- 172.21.15.86:0/2112576986 >> 172.21.15.86:0/2112576986 conn(0x7f6f780ff750 msgr2=0x7f6f78101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 -- 172.21.15.86:0/2112576986 shutdown_connections 2024-04-17T07:15:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.253+0000 7f6f80800640 1 -- 172.21.15.86:0/2112576986 wait complete. 2024-04-17T07:15:09.271 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:15:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.489+0000 7f2568a00640 1 -- 172.21.15.86:0/3950796037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25641040a0 msgr2=0x7f25641044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.489+0000 7f2568a00640 1 --2- 172.21.15.86:0/3950796037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25641040a0 0x7f25641044c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2554009900 tx=0x7f255402bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.489+0000 7f2568a00640 1 -- 172.21.15.86:0/3950796037 shutdown_connections 2024-04-17T07:15:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.489+0000 7f2568a00640 1 --2- 172.21.15.86:0/3950796037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2564105c50 0x7f256410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.489+0000 7f2568a00640 1 --2- 172.21.15.86:0/3950796037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2564105290 0x7f2564105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.489+0000 7f2568a00640 1 --2- 172.21.15.86:0/3950796037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25641040a0 0x7f25641044c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.489+0000 7f2568a00640 1 -- 172.21.15.86:0/3950796037 >> 172.21.15.86:0/3950796037 conn(0x7f25640ff610 msgr2=0x7f2564101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:09.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.489+0000 7f2568a00640 1 -- 172.21.15.86:0/3950796037 shutdown_connections 2024-04-17T07:15:09.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.489+0000 7f2568a00640 1 -- 172.21.15.86:0/3950796037 wait complete. 2024-04-17T07:15:09.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.490+0000 7f2568a00640 1 Processor -- start 2024-04-17T07:15:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.490+0000 7f2568a00640 1 -- start start 2024-04-17T07:15:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.490+0000 7f2568a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25641040a0 0x7f256419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.490+0000 7f2568a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2564105290 0x7f256419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.490+0000 7f2568a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2564105c50 0x7f25641a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.490+0000 7f2568a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f256410fc30 con 0x7f2564105290 2024-04-17T07:15:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.490+0000 7f2568a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f256410fab0 con 0x7f2564105c50 2024-04-17T07:15:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.490+0000 7f2568a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f256410fdb0 con 0x7f25641040a0 2024-04-17T07:15:09.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.491+0000 7f2562a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2564105290 0x7f256419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.491+0000 7f2562a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2564105290 0x7f256419a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37884/0 (socket says 172.21.15.86:37884) 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.491+0000 7f2562a00640 1 -- 172.21.15.86:0/1035228063 learned_addr learned my addr 172.21.15.86:0/1035228063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.491+0000 7f2563e00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2564105c50 0x7f25641a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.491+0000 7f2562a00640 1 -- 172.21.15.86:0/1035228063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25641040a0 msgr2=0x7f256419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.491+0000 7f2562a00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25641040a0 0x7f256419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.491+0000 7f2562a00640 1 -- 172.21.15.86:0/1035228063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2564105c50 msgr2=0x7f25641a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.491+0000 7f2562a00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2564105c50 0x7f25641a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.491+0000 7f2562a00640 1 -- 172.21.15.86:0/1035228063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25641a1ee0 con 0x7f2564105290 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.491+0000 7f2562a00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2564105290 0x7f256419a800 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f255000d950 tx=0x7f255000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.492+0000 7f255be00640 1 -- 172.21.15.86:0/1035228063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550017070 con 0x7f2564105290 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.492+0000 7f255be00640 1 -- 172.21.15.86:0/1035228063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2550009c30 con 0x7f2564105290 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.492+0000 7f2568a00640 1 -- 172.21.15.86:0/1035228063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25641a21d0 con 0x7f2564105290 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.492+0000 7f255be00640 1 -- 172.21.15.86:0/1035228063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255000f620 con 0x7f2564105290 2024-04-17T07:15:09.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.492+0000 7f2568a00640 1 -- 172.21.15.86:0/1035228063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25641a2720 con 0x7f2564105290 2024-04-17T07:15:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.493+0000 7f2568a00640 1 -- 172.21.15.86:0/1035228063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f256410b410 con 0x7f2564105290 2024-04-17T07:15:09.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.496+0000 7f255be00640 1 -- 172.21.15.86:0/1035228063 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f25500109a0 con 0x7f2564105290 2024-04-17T07:15:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.497+0000 7f255be00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f25480a3c40 0x7f25480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.497+0000 7f255be00640 1 -- 172.21.15.86:0/1035228063 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f25500d0650 con 0x7f2564105290 2024-04-17T07:15:09.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.497+0000 7f2563400640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f25480a3c40 0x7f25480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.498+0000 7f2563400640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f25480a3c40 0x7f25480a6100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2554005c20 tx=0x7f2554000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:09.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.502+0000 7f255be00640 1 -- 172.21.15.86:0/1035228063 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2550009de0 con 0x7f2564105290 2024-04-17T07:15:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:09 smithi086 ceph-mon[31733]: from='client.15717 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:15:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:09 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1682149436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:09 smithi086 ceph-mon[31733]: pgmap v464: 129 pgs: 129 active+clean; 654 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-04-17T07:15:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:09 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2112576986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:15:09.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.706+0000 7f2568a00640 1 -- 172.21.15.86:0/1035228063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f256402d880 con 0x7f2564105290 2024-04-17T07:15:09.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.708+0000 7f255be00640 1 -- 172.21.15.86:0/1035228063 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f255009caf0 con 0x7f2564105290 2024-04-17T07:15:09.707 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:15:09.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.710+0000 7f2568a00640 1 -- 172.21.15.86:0/1035228063 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f25480a3c40 msgr2=0x7f25480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.710+0000 7f2568a00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f25480a3c40 0x7f25480a6100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2554005c20 tx=0x7f2554000bc0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.710+0000 7f2568a00640 1 -- 172.21.15.86:0/1035228063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2564105290 msgr2=0x7f256419a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.710+0000 7f2568a00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2564105290 0x7f256419a800 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f255000d950 tx=0x7f255000de10 comp rx=0 tx=0).stop 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.710+0000 7f2568a00640 1 -- 172.21.15.86:0/1035228063 shutdown_connections 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.710+0000 7f2568a00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f25480a3c40 0x7f25480a6100 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.710+0000 7f2568a00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2564105c50 0x7f25641a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.710+0000 7f2568a00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2564105290 0x7f256419a800 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.710+0000 7f2568a00640 1 --2- 172.21.15.86:0/1035228063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25641040a0 0x7f256419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.710+0000 7f2568a00640 1 -- 172.21.15.86:0/1035228063 >> 172.21.15.86:0/1035228063 conn(0x7f25640ff610 msgr2=0x7f25641085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.711+0000 7f2568a00640 1 -- 172.21.15.86:0/1035228063 shutdown_connections 2024-04-17T07:15:09.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.711+0000 7f2568a00640 1 -- 172.21.15.86:0/1035228063 wait complete. 2024-04-17T07:15:09.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:15:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:09 smithi098 ceph-mon[35418]: from='client.15717 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:15:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:09 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1682149436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:09 smithi098 ceph-mon[35418]: pgmap v464: 129 pgs: 129 active+clean; 654 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-04-17T07:15:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:09 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2112576986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:15:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:09 smithi103 ceph-mon[35495]: from='client.15717 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-17T07:15:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:09 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1682149436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:09 smithi103 ceph-mon[35495]: pgmap v464: 129 pgs: 129 active+clean; 654 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-04-17T07:15:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:09 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2112576986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:15:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.942+0000 7fa420000640 1 -- 172.21.15.86:0/2744716647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418078500 msgr2=0x7fa418078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.942+0000 7fa420000640 1 --2- 172.21.15.86:0/2744716647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418078500 0x7fa418078da0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa410009ca0 tx=0x7fa41002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.942+0000 7fa420000640 1 -- 172.21.15.86:0/2744716647 shutdown_connections 2024-04-17T07:15:09.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.942+0000 7fa420000640 1 --2- 172.21.15.86:0/2744716647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418078500 0x7fa418078da0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.942+0000 7fa420000640 1 --2- 172.21.15.86:0/2744716647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa418077b60 0x7fa418077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.942+0000 7fa420000640 1 --2- 172.21.15.86:0/2744716647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa418076920 0x7fa418076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.942+0000 7fa420000640 1 -- 172.21.15.86:0/2744716647 >> 172.21.15.86:0/2744716647 conn(0x7fa4180ff750 msgr2=0x7fa418101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:09.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.942+0000 7fa420000640 1 -- 172.21.15.86:0/2744716647 shutdown_connections 2024-04-17T07:15:09.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.942+0000 7fa420000640 1 -- 172.21.15.86:0/2744716647 wait complete. 2024-04-17T07:15:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa420000640 1 Processor -- start 2024-04-17T07:15:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa420000640 1 -- start start 2024-04-17T07:15:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa420000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa418076920 0x7fa41819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa420000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418077b60 0x7fa41819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa420000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa418078500 0x7fa4181a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa420000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa418114100 con 0x7fa418077b60 2024-04-17T07:15:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa420000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa418113f80 con 0x7fa418078500 2024-04-17T07:15:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa420000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa418114280 con 0x7fa418076920 2024-04-17T07:15:09.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa41cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418077b60 0x7fa41819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa41cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418077b60 0x7fa41819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37892/0 (socket says 172.21.15.86:37892) 2024-04-17T07:15:09.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.944+0000 7fa41cc00640 1 -- 172.21.15.86:0/646008455 learned_addr learned my addr 172.21.15.86:0/646008455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.945+0000 7fa41e000640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa418078500 0x7fa4181a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.945+0000 7fa41d600640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa418076920 0x7fa41819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.945+0000 7fa41cc00640 1 -- 172.21.15.86:0/646008455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa418076920 msgr2=0x7fa41819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.945+0000 7fa41cc00640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa418076920 0x7fa41819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.945+0000 7fa41cc00640 1 -- 172.21.15.86:0/646008455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa418078500 msgr2=0x7fa4181a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.945+0000 7fa41cc00640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa418078500 0x7fa4181a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.945+0000 7fa41cc00640 1 -- 172.21.15.86:0/646008455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4181a63a0 con 0x7fa418077b60 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.945+0000 7fa41cc00640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418077b60 0x7fa41819ecc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa40800d770 tx=0x7fa40800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.945+0000 7fa416000640 1 -- 172.21.15.86:0/646008455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408017070 con 0x7fa418077b60 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.946+0000 7fa416000640 1 -- 172.21.15.86:0/646008455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4080054d0 con 0x7fa418077b60 2024-04-17T07:15:09.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.946+0000 7fa416000640 1 -- 172.21.15.86:0/646008455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408004cf0 con 0x7fa418077b60 2024-04-17T07:15:09.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.946+0000 7fa420000640 1 -- 172.21.15.86:0/646008455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4181a6690 con 0x7fa418077b60 2024-04-17T07:15:09.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.946+0000 7fa420000640 1 -- 172.21.15.86:0/646008455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181a6be0 con 0x7fa418077b60 2024-04-17T07:15:09.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.947+0000 7fa420000640 1 -- 172.21.15.86:0/646008455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa418110d10 con 0x7fa418077b60 2024-04-17T07:15:09.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.949+0000 7fa416000640 1 -- 172.21.15.86:0/646008455 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fa408004e90 con 0x7fa418077b60 2024-04-17T07:15:09.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.950+0000 7fa416000640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4000a3cc0 0x7fa4000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:09.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.951+0000 7fa416000640 1 -- 172.21.15.86:0/646008455 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa4080d03b0 con 0x7fa418077b60 2024-04-17T07:15:09.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.951+0000 7fa41d600640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4000a3cc0 0x7fa4000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:09.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.951+0000 7fa41d600640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4000a3cc0 0x7fa4000a6180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa40c005fd0 tx=0x7fa40c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:09.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:09.956+0000 7fa416000640 1 -- 172.21.15.86:0/646008455 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa40809c8b0 con 0x7fa418077b60 2024-04-17T07:15:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.158+0000 7fa420000640 1 -- 172.21.15.86:0/646008455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa41802d840 con 0x7fa418077b60 2024-04-17T07:15:10.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.159+0000 7fa416000640 1 -- 172.21.15.86:0/646008455 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fa40809c1b0 con 0x7fa418077b60 2024-04-17T07:15:10.158 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:15:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 -- 172.21.15.86:0/646008455 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4000a3cc0 msgr2=0x7fa4000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4000a3cc0 0x7fa4000a6180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa40c005fd0 tx=0x7fa40c005850 comp rx=0 tx=0).stop 2024-04-17T07:15:10.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 -- 172.21.15.86:0/646008455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418077b60 msgr2=0x7fa41819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418077b60 0x7fa41819ecc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa40800d770 tx=0x7fa40800dc30 comp rx=0 tx=0).stop 2024-04-17T07:15:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 -- 172.21.15.86:0/646008455 shutdown_connections 2024-04-17T07:15:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4000a3cc0 0x7fa4000a6180 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa418078500 0x7fa4181a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa418077b60 0x7fa41819ecc0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 --2- 172.21.15.86:0/646008455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa418076920 0x7fa41819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 -- 172.21.15.86:0/646008455 >> 172.21.15.86:0/646008455 conn(0x7fa4180ff750 msgr2=0x7fa418100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 -- 172.21.15.86:0/646008455 shutdown_connections 2024-04-17T07:15:10.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.161+0000 7fa420000640 1 -- 172.21.15.86:0/646008455 wait complete. 2024-04-17T07:15:10.180 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:15:10.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.403+0000 7fa3aae00640 1 -- 172.21.15.86:0/3147711710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3a410a050 msgr2=0x7fa3a41108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:10.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.403+0000 7fa3aae00640 1 --2- 172.21.15.86:0/3147711710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3a410a050 0x7fa3a41108e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009ca0 tx=0x7fa39c02c100 comp rx=0 tx=0).stop 2024-04-17T07:15:10.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.403+0000 7fa3aae00640 1 -- 172.21.15.86:0/3147711710 shutdown_connections 2024-04-17T07:15:10.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.403+0000 7fa3aae00640 1 --2- 172.21.15.86:0/3147711710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3a410a050 0x7fa3a41108e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.403+0000 7fa3aae00640 1 --2- 172.21.15.86:0/3147711710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3a4109690 0x7fa3a4109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.403+0000 7fa3aae00640 1 --2- 172.21.15.86:0/3147711710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a41084a0 0x7fa3a41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.403+0000 7fa3aae00640 1 -- 172.21.15.86:0/3147711710 >> 172.21.15.86:0/3147711710 conn(0x7fa3a4077430 msgr2=0x7fa3a4079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:10.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.403+0000 7fa3aae00640 1 -- 172.21.15.86:0/3147711710 shutdown_connections 2024-04-17T07:15:10.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.403+0000 7fa3aae00640 1 -- 172.21.15.86:0/3147711710 wait complete. 2024-04-17T07:15:10.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.404+0000 7fa3aae00640 1 Processor -- start 2024-04-17T07:15:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.404+0000 7fa3aae00640 1 -- start start 2024-04-17T07:15:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.404+0000 7fa3aae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3a41084a0 0x7fa3a419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3aae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3a4109690 0x7fa3a419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3aae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a050 0x7fa3a41a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3aae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4113fe0 con 0x7fa3a41084a0 2024-04-17T07:15:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3aae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4113e60 con 0x7fa3a410a050 2024-04-17T07:15:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3aae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a4114160 con 0x7fa3a4109690 2024-04-17T07:15:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3a9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3a41084a0 0x7fa3a419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3a9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3a41084a0 0x7fa3a419e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37910/0 (socket says 172.21.15.86:37910) 2024-04-17T07:15:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3a9a00640 1 -- 172.21.15.86:0/500695447 learned_addr learned my addr 172.21.15.86:0/500695447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3a9a00640 1 -- 172.21.15.86:0/500695447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3a4109690 msgr2=0x7fa3a419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3a9000640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3a4109690 0x7fa3a419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3aa400640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a050 0x7fa3a41a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3a9a00640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3a4109690 0x7fa3a419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3a9a00640 1 -- 172.21.15.86:0/500695447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a050 msgr2=0x7fa3a41a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3a9a00640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a050 0x7fa3a41a5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.405+0000 7fa3a9a00640 1 -- 172.21.15.86:0/500695447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a41a6230 con 0x7fa3a41084a0 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.406+0000 7fa3a9a00640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3a41084a0 0x7fa3a419e610 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa39800e9c0 tx=0x7fa39800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.406+0000 7fa3a2000640 1 -- 172.21.15.86:0/500695447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39800cd80 con 0x7fa3a41084a0 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.406+0000 7fa3a2000640 1 -- 172.21.15.86:0/500695447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3980035f0 con 0x7fa3a41084a0 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.406+0000 7fa3a2000640 1 -- 172.21.15.86:0/500695447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398010690 con 0x7fa3a41084a0 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.406+0000 7fa3aae00640 1 -- 172.21.15.86:0/500695447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a41a6520 con 0x7fa3a41084a0 2024-04-17T07:15:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.406+0000 7fa3aae00640 1 -- 172.21.15.86:0/500695447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a41a6ac0 con 0x7fa3a41084a0 2024-04-17T07:15:10.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.407+0000 7fa3aae00640 1 -- 172.21.15.86:0/500695447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a410f840 con 0x7fa3a41084a0 2024-04-17T07:15:10.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.409+0000 7fa3a2000640 1 -- 172.21.15.86:0/500695447 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fa3980037a0 con 0x7fa3a41084a0 2024-04-17T07:15:10.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.411+0000 7fa3a2000640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa38c0a3cc0 0x7fa38c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:10.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.411+0000 7fa3a2000640 1 -- 172.21.15.86:0/500695447 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa3980d0de0 con 0x7fa3a41084a0 2024-04-17T07:15:10.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.411+0000 7fa3a9000640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa38c0a3cc0 0x7fa38c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:10.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.412+0000 7fa3a9000640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa38c0a3cc0 0x7fa38c0a6180 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa390005fd0 tx=0x7fa390005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:10.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.415+0000 7fa3a2000640 1 -- 172.21.15.86:0/500695447 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa39809d280 con 0x7fa3a41084a0 2024-04-17T07:15:10.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:10 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1035228063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:15:10.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:10 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/646008455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:15:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.615+0000 7fa3aae00640 1 -- 172.21.15.86:0/500695447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa3a402d840 con 0x7fa3a41084a0 2024-04-17T07:15:10.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.616+0000 7fa3a2000640 1 -- 172.21.15.86:0/500695447 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fa39809cb80 con 0x7fa3a41084a0 2024-04-17T07:15:10.615 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:15:10.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 -- 172.21.15.86:0/500695447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa38c0a3cc0 msgr2=0x7fa38c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:10.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa38c0a3cc0 0x7fa38c0a6180 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa390005fd0 tx=0x7fa390005850 comp rx=0 tx=0).stop 2024-04-17T07:15:10.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 -- 172.21.15.86:0/500695447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3a41084a0 msgr2=0x7fa3a419e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:10.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3a41084a0 0x7fa3a419e610 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa39800e9c0 tx=0x7fa39800ee80 comp rx=0 tx=0).stop 2024-04-17T07:15:10.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 -- 172.21.15.86:0/500695447 shutdown_connections 2024-04-17T07:15:10.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa38c0a3cc0 0x7fa38c0a6180 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3a410a050 0x7fa3a41a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3a4109690 0x7fa3a419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 --2- 172.21.15.86:0/500695447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3a41084a0 0x7fa3a419e610 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 -- 172.21.15.86:0/500695447 >> 172.21.15.86:0/500695447 conn(0x7fa3a4077430 msgr2=0x7fa3a410ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:10.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 -- 172.21.15.86:0/500695447 shutdown_connections 2024-04-17T07:15:10.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.619+0000 7fa3aae00640 1 -- 172.21.15.86:0/500695447 wait complete. 2024-04-17T07:15:10.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:15:10.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:10 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1035228063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:15:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:10 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/646008455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:15:10.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:10 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1035228063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:15:10.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:10 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/646008455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:15:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.831+0000 7fd441600640 1 -- 172.21.15.86:0/3278597432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c10a170 msgr2=0x7fd43c110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.831+0000 7fd441600640 1 --2- 172.21.15.86:0/3278597432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c10a170 0x7fd43c110a00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd434009ca0 tx=0x7fd43402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.832+0000 7fd441600640 1 -- 172.21.15.86:0/3278597432 shutdown_connections 2024-04-17T07:15:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.832+0000 7fd441600640 1 --2- 172.21.15.86:0/3278597432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c10a170 0x7fd43c110a00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.832+0000 7fd441600640 1 --2- 172.21.15.86:0/3278597432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd43c1097b0 0x7fd43c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.832+0000 7fd441600640 1 --2- 172.21.15.86:0/3278597432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43c1085c0 0x7fd43c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.832+0000 7fd441600640 1 -- 172.21.15.86:0/3278597432 >> 172.21.15.86:0/3278597432 conn(0x7fd43c077580 msgr2=0x7fd43c0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.832+0000 7fd441600640 1 -- 172.21.15.86:0/3278597432 shutdown_connections 2024-04-17T07:15:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.832+0000 7fd441600640 1 -- 172.21.15.86:0/3278597432 wait complete. 2024-04-17T07:15:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.833+0000 7fd441600640 1 Processor -- start 2024-04-17T07:15:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.833+0000 7fd441600640 1 -- start start 2024-04-17T07:15:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.833+0000 7fd441600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd43c1085c0 0x7fd43c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.833+0000 7fd441600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43c1097b0 0x7fd43c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.833+0000 7fd441600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c10a170 0x7fd43c1a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.833+0000 7fd441600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c1140f0 con 0x7fd43c10a170 2024-04-17T07:15:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.833+0000 7fd441600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c113f70 con 0x7fd43c1085c0 2024-04-17T07:15:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.833+0000 7fd441600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c114270 con 0x7fd43c1097b0 2024-04-17T07:15:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd43b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c10a170 0x7fd43c1a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd43b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c10a170 0x7fd43c1a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37926/0 (socket says 172.21.15.86:37926) 2024-04-17T07:15:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd43b400640 1 -- 172.21.15.86:0/3479114025 learned_addr learned my addr 172.21.15.86:0/3479114025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd43aa00640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd43c1085c0 0x7fd43c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd43b400640 1 -- 172.21.15.86:0/3479114025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43c1097b0 msgr2=0x7fd43c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd43b400640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43c1097b0 0x7fd43c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd43b400640 1 -- 172.21.15.86:0/3479114025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd43c1085c0 msgr2=0x7fd43c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd43b400640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd43c1085c0 0x7fd43c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd43b400640 1 -- 172.21.15.86:0/3479114025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c1a6400 con 0x7fd43c10a170 2024-04-17T07:15:10.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd43b400640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c10a170 0x7fd43c1a5d90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd434032380 tx=0x7fd434032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.834+0000 7fd433e00640 1 -- 172.21.15.86:0/3479114025 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434041020 con 0x7fd43c10a170 2024-04-17T07:15:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.835+0000 7fd433e00640 1 -- 172.21.15.86:0/3479114025 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd434003e20 con 0x7fd43c10a170 2024-04-17T07:15:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.835+0000 7fd433e00640 1 -- 172.21.15.86:0/3479114025 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434039040 con 0x7fd43c10a170 2024-04-17T07:15:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.835+0000 7fd441600640 1 -- 172.21.15.86:0/3479114025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c1a6690 con 0x7fd43c10a170 2024-04-17T07:15:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.835+0000 7fd441600640 1 -- 172.21.15.86:0/3479114025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c1a6b80 con 0x7fd43c10a170 2024-04-17T07:15:10.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.836+0000 7fd441600640 1 -- 172.21.15.86:0/3479114025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c10f8e0 con 0x7fd43c10a170 2024-04-17T07:15:10.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.838+0000 7fd433e00640 1 -- 172.21.15.86:0/3479114025 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fd434044020 con 0x7fd43c10a170 2024-04-17T07:15:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.840+0000 7fd433e00640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4200a3c40 0x7fd4200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.840+0000 7fd433e00640 1 -- 172.21.15.86:0/3479114025 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd4340f2b90 con 0x7fd43c10a170 2024-04-17T07:15:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.840+0000 7fd43aa00640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4200a3c40 0x7fd4200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:10.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.841+0000 7fd43aa00640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4200a3c40 0x7fd4200a6100 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd42c005fd0 tx=0x7fd42c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:10.845+0000 7fd433e00640 1 -- 172.21.15.86:0/3479114025 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd43404b020 con 0x7fd43c10a170 2024-04-17T07:15:11.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.090+0000 7fd441600640 1 -- 172.21.15.86:0/3479114025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd43c02d7b0 con 0x7fd43c10a170 2024-04-17T07:15:11.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.092+0000 7fd433e00640 1 -- 172.21.15.86:0/3479114025 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fd4340bf1a0 con 0x7fd43c10a170 2024-04-17T07:15:11.092 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:11.092 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:15:11.092 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:15:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 -- 172.21.15.86:0/3479114025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4200a3c40 msgr2=0x7fd4200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4200a3c40 0x7fd4200a6100 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd42c005fd0 tx=0x7fd42c005850 comp rx=0 tx=0).stop 2024-04-17T07:15:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 -- 172.21.15.86:0/3479114025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c10a170 msgr2=0x7fd43c1a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c10a170 0x7fd43c1a5d90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd434032380 tx=0x7fd434032820 comp rx=0 tx=0).stop 2024-04-17T07:15:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 -- 172.21.15.86:0/3479114025 shutdown_connections 2024-04-17T07:15:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd4200a3c40 0x7fd4200a6100 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd43c10a170 0x7fd43c1a5d90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd43c1097b0 0x7fd43c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 --2- 172.21.15.86:0/3479114025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd43c1085c0 0x7fd43c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 -- 172.21.15.86:0/3479114025 >> 172.21.15.86:0/3479114025 conn(0x7fd43c077580 msgr2=0x7fd43c10cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 -- 172.21.15.86:0/3479114025 shutdown_connections 2024-04-17T07:15:11.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.095+0000 7fd441600640 1 -- 172.21.15.86:0/3479114025 wait complete. 2024-04-17T07:15:11.116 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:15:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.328+0000 7f2426200640 1 -- 172.21.15.86:0/3203646782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24201041e0 msgr2=0x7f2420104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.328+0000 7f2426200640 1 --2- 172.21.15.86:0/3203646782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24201041e0 0x7f2420104600 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2410009900 tx=0x7f241002bc50 comp rx=0 tx=0).stop 2024-04-17T07:15:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.328+0000 7f2426200640 1 -- 172.21.15.86:0/3203646782 shutdown_connections 2024-04-17T07:15:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.328+0000 7f2426200640 1 --2- 172.21.15.86:0/3203646782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2420105d90 0x7f242010c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.328+0000 7f2426200640 1 --2- 172.21.15.86:0/3203646782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24201053d0 0x7f2420105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.328+0000 7f2426200640 1 --2- 172.21.15.86:0/3203646782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24201041e0 0x7f2420104600 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.328+0000 7f2426200640 1 -- 172.21.15.86:0/3203646782 >> 172.21.15.86:0/3203646782 conn(0x7f24200ff750 msgr2=0x7f2420101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.328+0000 7f2426200640 1 -- 172.21.15.86:0/3203646782 shutdown_connections 2024-04-17T07:15:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.328+0000 7f2426200640 1 -- 172.21.15.86:0/3203646782 wait complete. 2024-04-17T07:15:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.329+0000 7f2426200640 1 Processor -- start 2024-04-17T07:15:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.329+0000 7f2426200640 1 -- start start 2024-04-17T07:15:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f2426200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24201041e0 0x7f242019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f2426200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24201053d0 0x7f242019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f2426200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2420105d90 0x7f24201a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f2426200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f242010fd40 con 0x7f24201041e0 2024-04-17T07:15:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f2426200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f242010fbc0 con 0x7f24201053d0 2024-04-17T07:15:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f2426200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f242010fec0 con 0x7f2420105d90 2024-04-17T07:15:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f241f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24201041e0 0x7f242019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f241f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24201041e0 0x7f242019a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37944/0 (socket says 172.21.15.86:37944) 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f241f400640 1 -- 172.21.15.86:0/1337783797 learned_addr learned my addr 172.21.15.86:0/1337783797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f241f400640 1 -- 172.21.15.86:0/1337783797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2420105d90 msgr2=0x7f24201a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f241f400640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2420105d90 0x7f24201a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f241f400640 1 -- 172.21.15.86:0/1337783797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24201053d0 msgr2=0x7f242019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f241f400640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24201053d0 0x7f242019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.330+0000 7f241f400640 1 -- 172.21.15.86:0/1337783797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24201a1fe0 con 0x7f24201041e0 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.331+0000 7f241f400640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24201041e0 0x7f242019a3c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2410005ab0 tx=0x7f24100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.331+0000 7f2417e00640 1 -- 172.21.15.86:0/1337783797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410043020 con 0x7f24201041e0 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.331+0000 7f2417e00640 1 -- 172.21.15.86:0/1337783797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24100054c0 con 0x7f24201041e0 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.331+0000 7f2426200640 1 -- 172.21.15.86:0/1337783797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24201a2270 con 0x7f24201041e0 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.331+0000 7f2417e00640 1 -- 172.21.15.86:0/1337783797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410003de0 con 0x7f24201041e0 2024-04-17T07:15:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.331+0000 7f2426200640 1 -- 172.21.15.86:0/1337783797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24201a2790 con 0x7f24201041e0 2024-04-17T07:15:11.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.332+0000 7f2426200640 1 -- 172.21.15.86:0/1337783797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242010b500 con 0x7f24201041e0 2024-04-17T07:15:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.334+0000 7f2417e00640 1 -- 172.21.15.86:0/1337783797 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f2410046020 con 0x7f24201041e0 2024-04-17T07:15:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.336+0000 7f2417e00640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24040a3c40 0x7f24040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.337+0000 7f2417e00640 1 -- 172.21.15.86:0/1337783797 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f24100f2df0 con 0x7f24201041e0 2024-04-17T07:15:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.337+0000 7f241ea00640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24040a3c40 0x7f24040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.337+0000 7f241ea00640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24040a3c40 0x7f24040a6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f242019b8d0 tx=0x7f240c005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.341+0000 7f2417e00640 1 -- 172.21.15.86:0/1337783797 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f241004d020 con 0x7f24201041e0 2024-04-17T07:15:11.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.551+0000 7f2426200640 1 -- 172.21.15.86:0/1337783797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2420062c00 con 0x7f24201041e0 2024-04-17T07:15:11.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.554+0000 7f2417e00640 1 -- 172.21.15.86:0/1337783797 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7f24100bf3a0 con 0x7f24201041e0 2024-04-17T07:15:11.554 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:11.554 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:15:11.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 -- 172.21.15.86:0/1337783797 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24040a3c40 msgr2=0x7f24040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24040a3c40 0x7f24040a6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f242019b8d0 tx=0x7f240c005bf0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 -- 172.21.15.86:0/1337783797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24201041e0 msgr2=0x7f242019a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24201041e0 0x7f242019a3c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2410005ab0 tx=0x7f24100042f0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 -- 172.21.15.86:0/1337783797 shutdown_connections 2024-04-17T07:15:11.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24040a3c40 0x7f24040a6100 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2420105d90 0x7f24201a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24201053d0 0x7f242019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 --2- 172.21.15.86:0/1337783797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24201041e0 0x7f242019a3c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 -- 172.21.15.86:0/1337783797 >> 172.21.15.86:0/1337783797 conn(0x7f24200ff750 msgr2=0x7f24201086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:11.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 -- 172.21.15.86:0/1337783797 shutdown_connections 2024-04-17T07:15:11.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.556+0000 7f2426200640 1 -- 172.21.15.86:0/1337783797 wait complete. 2024-04-17T07:15:11.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:11 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/500695447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:15:11.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:11 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3479114025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:15:11.572 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:11 smithi086 ceph-mon[31733]: pgmap v465: 129 pgs: 129 active+clean; 654 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-17T07:15:11.576 DEBUG:tasks.fwd_scrub:Yielding 2024-04-17T07:15:11.576 INFO:teuthology.run_tasks:Running task exec... 2024-04-17T07:15:11.585 INFO:teuthology.task.exec:Executing custom commands... 2024-04-17T07:15:11.585 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi086.front.sepia.ceph.com 2024-04-17T07:15:11.585 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-04-17T07:15:11.586 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-17T07:15:11.586 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:15:11.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:15:11.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:15:11.706 INFO:teuthology.orchestra.run.smithi086.stdout:total 0 2024-04-17T07:15:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:11 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/500695447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:15:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:11 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3479114025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:15:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:11 smithi098 ceph-mon[35418]: pgmap v465: 129 pgs: 129 active+clean; 654 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-17T07:15:11.753 INFO:teuthology.orchestra.run.smithi086.stdout:Filesystem Size Used Avail Use% Mounted on 2024-04-17T07:15:11.753 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-fuse 1.1T 360M 1.1T 1% /tmp/cephfs 2024-04-17T07:15:11.760 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-04-17T07:15:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:11 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/500695447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:15:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:11 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3479114025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:15:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:11 smithi103 ceph-mon[35495]: pgmap v465: 129 pgs: 129 active+clean; 654 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-17T07:15:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.793+0000 7fe42d600640 1 -- 172.21.15.86:0/2320013117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428101fb0 msgr2=0x7fe4281023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.793+0000 7fe42d600640 1 --2- 172.21.15.86:0/2320013117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428101fb0 0x7fe4281023d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe418009900 tx=0x7fe41802bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.793+0000 7fe42d600640 1 -- 172.21.15.86:0/2320013117 shutdown_connections 2024-04-17T07:15:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.793+0000 7fe42d600640 1 --2- 172.21.15.86:0/2320013117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe428103b60 0x7fe42810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.793+0000 7fe42d600640 1 --2- 172.21.15.86:0/2320013117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4281031a0 0x7fe428103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.793+0000 7fe42d600640 1 --2- 172.21.15.86:0/2320013117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe428101fb0 0x7fe4281023d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.793+0000 7fe42d600640 1 -- 172.21.15.86:0/2320013117 >> 172.21.15.86:0/2320013117 conn(0x7fe4280fd540 msgr2=0x7fe4280ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.793+0000 7fe42d600640 1 -- 172.21.15.86:0/2320013117 shutdown_connections 2024-04-17T07:15:11.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.793+0000 7fe42d600640 1 -- 172.21.15.86:0/2320013117 wait complete. 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.794+0000 7fe42d600640 1 Processor -- start 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.794+0000 7fe42d600640 1 -- start start 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.794+0000 7fe42d600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe428101fb0 0x7fe42819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.794+0000 7fe42d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281031a0 0x7fe42819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.794+0000 7fe42d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe428103b60 0x7fe4281a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.794+0000 7fe42d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe42810db40 con 0x7fe4281031a0 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.794+0000 7fe42d600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe42810d9c0 con 0x7fe428101fb0 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.794+0000 7fe42d600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe42810dcc0 con 0x7fe428103b60 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.795+0000 7fe426000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281031a0 0x7fe42819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.795+0000 7fe426000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281031a0 0x7fe42819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37960/0 (socket says 172.21.15.86:37960) 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.795+0000 7fe426000640 1 -- 172.21.15.86:0/3222669343 learned_addr learned my addr 172.21.15.86:0/3222669343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:11.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.795+0000 7fe426a00640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe428101fb0 0x7fe42819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.795+0000 7fe426000640 1 -- 172.21.15.86:0/3222669343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe428103b60 msgr2=0x7fe4281a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.795+0000 7fe426000640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe428103b60 0x7fe4281a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.795+0000 7fe426000640 1 -- 172.21.15.86:0/3222669343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe428101fb0 msgr2=0x7fe42819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.795+0000 7fe426000640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe428101fb0 0x7fe42819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.795+0000 7fe426000640 1 -- 172.21.15.86:0/3222669343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4281a2040 con 0x7fe4281031a0 2024-04-17T07:15:11.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.795+0000 7fe426000640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281031a0 0x7fe42819a960 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c890 tx=0x7fe41400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:11.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.796+0000 7fe41f400640 1 -- 172.21.15.86:0/3222669343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414004220 con 0x7fe4281031a0 2024-04-17T07:15:11.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.796+0000 7fe41f400640 1 -- 172.21.15.86:0/3222669343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4140043c0 con 0x7fe4281031a0 2024-04-17T07:15:11.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.796+0000 7fe42d600640 1 -- 172.21.15.86:0/3222669343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4281a2330 con 0x7fe4281031a0 2024-04-17T07:15:11.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.796+0000 7fe41f400640 1 -- 172.21.15.86:0/3222669343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414004a00 con 0x7fe4281031a0 2024-04-17T07:15:11.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.796+0000 7fe42d600640 1 -- 172.21.15.86:0/3222669343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4281a28d0 con 0x7fe4281031a0 2024-04-17T07:15:11.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.799+0000 7fe41f400640 1 -- 172.21.15.86:0/3222669343 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fe414004ba0 con 0x7fe4281031a0 2024-04-17T07:15:11.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.799+0000 7fe42fffd740 1 Processor -- start 2024-04-17T07:15:11.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.799+0000 7fe42fffd740 1 -- start start 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4cec0 0x55b51fb4d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4d8b0 0x55b51fb523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb52a20 0x55b51fb54eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe42fffd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b51fab2cb0 con 0x55b51fb4d8b0 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe42fffd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b51fa59370 con 0x55b51fb52a20 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe42fffd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b51faaec70 con 0x55b51fb4cec0 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe426a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4d8b0 0x55b51fb523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe426a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4d8b0 0x55b51fb523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37962/0 (socket says 172.21.15.86:37962) 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe426a00640 1 -- 172.21.15.86:0/4219726811 learned_addr learned my addr 172.21.15.86:0/4219726811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe426000640 1 --2- 172.21.15.86:0/4219726811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb52a20 0x55b51fb54eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.800+0000 7fe427400640 1 --2- 172.21.15.86:0/4219726811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4cec0 0x55b51fb4d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe426a00640 1 -- 172.21.15.86:0/4219726811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4cec0 msgr2=0x55b51fb4d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe426a00640 1 --2- 172.21.15.86:0/4219726811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4cec0 0x55b51fb4d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe426a00640 1 -- 172.21.15.86:0/4219726811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb52a20 msgr2=0x55b51fb54eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:11.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe426a00640 1 --2- 172.21.15.86:0/4219726811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb52a20 0x55b51fb54eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe426a00640 1 -- 172.21.15.86:0/4219726811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b51fa78910 con 0x55b51fb4d8b0 2024-04-17T07:15:11.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe41f400640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe40c0a3cc0 0x7fe40c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe41f400640 1 -- 172.21.15.86:0/3222669343 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe4140d05d0 con 0x7fe4281031a0 2024-04-17T07:15:11.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe426a00640 1 --2- 172.21.15.86:0/4219726811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4d8b0 0x55b51fb523b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe418006010 tx=0x7fe418009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd23acee6dd1de3c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:11.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe427400640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe40c0a3cc0 0x7fe40c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe413400640 1 -- 172.21.15.86:0/4219726811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418044020 con 0x55b51fb4d8b0 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe413400640 1 -- 172.21.15.86:0/4219726811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe418005040 con 0x55b51fb4d8b0 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.801+0000 7fe413400640 1 -- 172.21.15.86:0/4219726811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4180049e0 con 0x55b51fb4d8b0 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe42fffd740 1 -- 172.21.15.86:0/4219726811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4d8b0 msgr2=0x55b51fb523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe42fffd740 1 --2- 172.21.15.86:0/4219726811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4d8b0 0x55b51fb523b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe418006010 tx=0x7fe418009680 comp rx=0 tx=0).stop 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe427400640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe40c0a3cc0 0x7fe40c0a6180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55b51fb4e110 tx=0x7fe420007360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe42fffd740 1 -- 172.21.15.86:0/4219726811 shutdown_connections 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe42fffd740 1 --2- 172.21.15.86:0/4219726811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb52a20 0x55b51fb54eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe42fffd740 1 --2- 172.21.15.86:0/4219726811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4d8b0 0x55b51fb523b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe42fffd740 1 --2- 172.21.15.86:0/4219726811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4cec0 0x55b51fb4d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe42fffd740 1 -- 172.21.15.86:0/4219726811 >> 172.21.15.86:0/4219726811 conn(0x55b51fb48430 msgr2=0x55b51fb48830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe42fffd740 1 -- 172.21.15.86:0/4219726811 shutdown_connections 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe42fffd740 1 -- 172.21.15.86:0/4219726811 wait complete. 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.803+0000 7fe42fffd740 1 -- start start 2024-04-17T07:15:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.804+0000 7fe42fffd740 1 Processor -- start 2024-04-17T07:15:11.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.804+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 0x55b51fb573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.804+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb578f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.804+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.804+0000 7fe42fffd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b51fb55970 con 0x55b51fb52a20 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.804+0000 7fe42fffd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b51fa6c730 con 0x55b51fb4cec0 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.804+0000 7fe42fffd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b51fb55af0 con 0x55b51fb4d8b0 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.804+0000 7fe42fffd740 10 client.0 _ll_register_callbacks cb 0x55b51fa6eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe427400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe427400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37976/0 (socket says 172.21.15.86:37976) 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe427400640 1 -- 172.21.15.86:0/572508943 learned_addr learned my addr 172.21.15.86:0/572508943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe426a00640 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 0x55b51fb573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe426000640 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe427400640 1 -- 172.21.15.86:0/572508943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 msgr2=0x55b51fb578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe427400640 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe427400640 1 -- 172.21.15.86:0/572508943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 msgr2=0x55b51fb573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe427400640 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 0x55b51fb573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe427400640 1 -- 172.21.15.86:0/572508943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b51f88d7e0 con 0x55b51fb52a20 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe426a00640 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 0x55b51fb573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe426000640 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe427400640 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5e930 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe420010a20 tx=0x7fe420010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5319684c05f5f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.805+0000 7fe407e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.806+0000 7fe407e00640 1 -- 172.21.15.86:0/572508943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420021070 con 0x55b51fb52a20 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.806+0000 7fe407e00640 1 -- 172.21.15.86:0/572508943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe42001d450 con 0x55b51fb52a20 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.806+0000 7fe42fffd740 1 -- 172.21.15.86:0/572508943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe418005040 con 0x55b51fb52a20 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.806+0000 7fe407e00640 1 -- 172.21.15.86:0/572508943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe42000fe10 con 0x55b51fb52a20 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.806+0000 7fe42fffd740 10 client.15771 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.806+0000 7fe42fffd740 1 -- 172.21.15.86:0/572508943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b51f8edf70 con 0x55b51fb52a20 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.806+0000 7fe407e00640 1 -- 172.21.15.86:0/572508943 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe418005040 con 0x55b51fb52a20 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.807+0000 7fe407e00640 1 -- 172.21.15.86:0/572508943 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe420005020 con 0x55b51fb52a20 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.807+0000 7fe42fffd740 10 client.15771 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.807+0000 7fe42fffd740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.807+0000 7fe42fffd740 20 client.15771 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.807+0000 7fe42fffd740 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b51fb632b0 0x55b51fb65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.807+0000 7fe42fffd740 4 client.15771 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:15:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.807+0000 7fe42fffd740 1 -- 172.21.15.86:0/572508943 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b51fac06d0 con 0x55b51fb632b0 2024-04-17T07:15:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.808+0000 7fe426a00640 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b51fb632b0 0x55b51fb65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.809+0000 7fe407e00640 10 client.15771 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.809+0000 7fe426a00640 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b51fb632b0 0x55b51fb65770 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d238b5fbc99e04c server_cookie=6c263886d1fb2334 in_seq=0 out_seq=0 2024-04-17T07:15:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.810+0000 7fe407e00640 1 -- 172.21.15.86:0/572508943 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b51fac06d0 con 0x55b51fb632b0 2024-04-17T07:15:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.810+0000 7fe407e00640 10 client.15771 handle_command_reply: tid=0 2024-04-17T07:15:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 client.15771 shutdown 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 -- 172.21.15.86:0/572508943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 msgr2=0x55b51fb5e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5e930 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe420010a20 tx=0x7fe420010e20 comp rx=0 tx=0).stop 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 -- 172.21.15.86:0/572508943 shutdown_connections 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b51fb632b0 0x55b51fb65770 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5e930 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe407e00640 0 client.15771 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 --2- 172.21.15.86:0/572508943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 0x55b51fb573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 -- 172.21.15.86:0/572508943 >> 172.21.15.86:0/572508943 conn(0x55b51fb48430 msgr2=0x55b51fb475e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 -- 172.21.15.86:0/572508943 shutdown_connections 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 1 -- 172.21.15.86:0/572508943 wait complete. 2024-04-17T07:15:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.811+0000 7fe42fffd740 20 client.15771 trim_cache size 0 max 16384 2024-04-17T07:15:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.837+0000 7fe42fffd740 1 Processor -- start 2024-04-17T07:15:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.837+0000 7fe42fffd740 1 -- start start 2024-04-17T07:15:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.837+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4cec0 0x55b51fb46d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.837+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb5e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.837+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb52a20 0x55b51fb47290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.837+0000 7fe42fffd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281acce0 con 0x55b51fb4cec0 2024-04-17T07:15:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.837+0000 7fe42fffd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281ae820 con 0x55b51fb52a20 2024-04-17T07:15:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.837+0000 7fe42fffd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281ad3b0 con 0x55b51fb4d8b0 2024-04-17T07:15:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.838+0000 7fe426a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4cec0 0x55b51fb46d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.838+0000 7fe426a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4cec0 0x55b51fb46d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37988/0 (socket says 172.21.15.86:37988) 2024-04-17T07:15:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.838+0000 7fe426a00640 1 -- 172.21.15.86:0/3496530059 learned_addr learned my addr 172.21.15.86:0/3496530059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.838+0000 7fe427400640 1 --2- 172.21.15.86:0/3496530059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb52a20 0x55b51fb47290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.838+0000 7fe426000640 1 --2- 172.21.15.86:0/3496530059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb5e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.838+0000 7fe426a00640 1 -- 172.21.15.86:0/3496530059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 msgr2=0x55b51fb5e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.838+0000 7fe426a00640 1 --2- 172.21.15.86:0/3496530059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb5e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.838+0000 7fe426a00640 1 -- 172.21.15.86:0/3496530059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb52a20 msgr2=0x55b51fb47290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.838+0000 7fe426a00640 1 --2- 172.21.15.86:0/3496530059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb52a20 0x55b51fb47290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.838+0000 7fe426a00640 1 -- 172.21.15.86:0/3496530059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b51f785170 con 0x55b51fb4cec0 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.839+0000 7fe426a00640 1 --2- 172.21.15.86:0/3496530059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4cec0 0x55b51fb46d50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe418004650 tx=0x7fe41802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6debe8fb7e6bc432 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.839+0000 7fe412a00640 1 -- 172.21.15.86:0/3496530059 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418044040 con 0x55b51fb4cec0 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.839+0000 7fe412a00640 1 -- 172.21.15.86:0/3496530059 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe41802cc00 con 0x55b51fb4cec0 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.839+0000 7fe412a00640 1 -- 172.21.15.86:0/3496530059 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4180406d0 con 0x55b51fb4cec0 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.840+0000 7fe42fffd740 1 -- 172.21.15.86:0/3496530059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4cec0 msgr2=0x55b51fb46d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.840+0000 7fe42fffd740 1 --2- 172.21.15.86:0/3496530059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4cec0 0x55b51fb46d50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe418004650 tx=0x7fe41802bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:11.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.840+0000 7fe42fffd740 1 -- 172.21.15.86:0/3496530059 shutdown_connections 2024-04-17T07:15:11.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.840+0000 7fe42fffd740 1 --2- 172.21.15.86:0/3496530059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb52a20 0x55b51fb47290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.840+0000 7fe42fffd740 1 --2- 172.21.15.86:0/3496530059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb5e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.840+0000 7fe42fffd740 1 --2- 172.21.15.86:0/3496530059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb4cec0 0x55b51fb46d50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.840+0000 7fe42fffd740 1 -- 172.21.15.86:0/3496530059 >> 172.21.15.86:0/3496530059 conn(0x55b51fb48430 msgr2=0x55b51fb4edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:11.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.840+0000 7fe42fffd740 1 -- 172.21.15.86:0/3496530059 shutdown_connections 2024-04-17T07:15:11.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.840+0000 7fe42fffd740 1 -- 172.21.15.86:0/3496530059 wait complete. 2024-04-17T07:15:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.841+0000 7fe42fffd740 1 -- start start 2024-04-17T07:15:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.841+0000 7fe42fffd740 1 Processor -- start 2024-04-17T07:15:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.841+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 0x55b51fb5bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.841+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb5c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.841+0000 7fe42fffd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe42fffd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281ae150 con 0x55b51fb52a20 2024-04-17T07:15:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe42fffd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281ada80 con 0x55b51fb4cec0 2024-04-17T07:15:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe42fffd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4281aacb0 con 0x55b51fb4d8b0 2024-04-17T07:15:11.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe42fffd740 10 client.0 _ll_register_callbacks cb 0x7fe420014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe427400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe427400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37990/0 (socket says 172.21.15.86:37990) 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe427400640 1 -- 172.21.15.86:0/1154837803 learned_addr learned my addr 172.21.15.86:0/1154837803 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe426a00640 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 0x55b51fb5bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe427400640 1 -- 172.21.15.86:0/1154837803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 msgr2=0x55b51fb5c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe427400640 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb5c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe427400640 1 -- 172.21.15.86:0/1154837803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 msgr2=0x55b51fb5bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe427400640 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 0x55b51fb5bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe427400640 1 -- 172.21.15.86:0/1154837803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b51fb5db00 con 0x55b51fb52a20 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.842+0000 7fe426a00640 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 0x55b51fb5bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.843+0000 7fe427400640 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5d510 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe420005680 tx=0x7fe420004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bde878a0db082c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.843+0000 7fe407e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.843+0000 7fe407e00640 1 -- 172.21.15.86:0/1154837803 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420035070 con 0x55b51fb52a20 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.843+0000 7fe407e00640 1 -- 172.21.15.86:0/1154837803 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe420005190 con 0x55b51fb52a20 2024-04-17T07:15:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.843+0000 7fe42fffd740 1 -- 172.21.15.86:0/1154837803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b51faba540 con 0x55b51fb52a20 2024-04-17T07:15:11.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.843+0000 7fe407e00640 1 -- 172.21.15.86:0/1154837803 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe42002a020 con 0x55b51fb52a20 2024-04-17T07:15:11.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.843+0000 7fe42fffd740 10 client.15777 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:11.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.843+0000 7fe42fffd740 1 -- 172.21.15.86:0/1154837803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b51fbc98b0 con 0x55b51fb52a20 2024-04-17T07:15:11.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.843+0000 7fe407e00640 1 -- 172.21.15.86:0/1154837803 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b51faba540 con 0x55b51fb52a20 2024-04-17T07:15:11.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.844+0000 7fe407e00640 1 -- 172.21.15.86:0/1154837803 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe4200044b0 con 0x55b51fb52a20 2024-04-17T07:15:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.844+0000 7fe42fffd740 10 client.15777 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.844+0000 7fe42fffd740 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.844+0000 7fe42fffd740 20 client.15777 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.844+0000 7fe42fffd740 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b51fbca230 0x55b51fbcc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.844+0000 7fe42fffd740 4 client.15777 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:15:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.844+0000 7fe42fffd740 1 -- 172.21.15.86:0/1154837803 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b51fa36aa0 con 0x55b51fbca230 2024-04-17T07:15:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.844+0000 7fe426a00640 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b51fbca230 0x55b51fbcc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.845+0000 7fe426a00640 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b51fbca230 0x55b51fbcc650 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e23d1beeab1bb8e server_cookie=a664a99f9be85e79 in_seq=0 out_seq=0 2024-04-17T07:15:11.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.845+0000 7fe407e00640 10 client.15777 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe407e00640 1 -- 172.21.15.86:0/1154837803 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b51fa36aa0 con 0x55b51fbca230 2024-04-17T07:15:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe407e00640 10 client.15777 handle_command_reply: tid=0 2024-04-17T07:15:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe42fffd740 1 client.15777 shutdown 2024-04-17T07:15:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe42fffd740 1 -- 172.21.15.86:0/1154837803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 msgr2=0x55b51fb5d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe42fffd740 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5d510 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe420005680 tx=0x7fe420004d90 comp rx=0 tx=0).stop 2024-04-17T07:15:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe42fffd740 1 -- 172.21.15.86:0/1154837803 shutdown_connections 2024-04-17T07:15:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe42fffd740 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b51fbca230 0x55b51fbcc650 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe42fffd740 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b51fb52a20 0x55b51fb5d510 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe42fffd740 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51fb4d8b0 0x55b51fb5c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe42fffd740 1 --2- 172.21.15.86:0/1154837803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b51fb4cec0 0x55b51fb5bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe42fffd740 1 -- 172.21.15.86:0/1154837803 >> 172.21.15.86:0/1154837803 conn(0x55b51fb48430 msgr2=0x55b51fb46d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.848+0000 7fe407e00640 0 client.15777 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.849+0000 7fe42fffd740 1 -- 172.21.15.86:0/1154837803 shutdown_connections 2024-04-17T07:15:11.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.849+0000 7fe42fffd740 1 -- 172.21.15.86:0/1154837803 wait complete. 2024-04-17T07:15:11.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.849+0000 7fe42fffd740 20 client.15777 trim_cache size 0 max 16384 2024-04-17T07:15:11.850 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:15:11.850 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:15:11.850 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "5e5a0747-faaf-4300-88c5-88acbf50aa1f", 2024-04-17T07:15:11.850 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:15:11.850 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:15:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 -- 172.21.15.86:0/3222669343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe40c0a3cc0 msgr2=0x7fe40c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe40c0a3cc0 0x7fe40c0a6180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55b51fb4e110 tx=0x7fe420007360 comp rx=0 tx=0).stop 2024-04-17T07:15:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 -- 172.21.15.86:0/3222669343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281031a0 msgr2=0x7fe42819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281031a0 0x7fe42819a960 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c890 tx=0x7fe41400cd50 comp rx=0 tx=0).stop 2024-04-17T07:15:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 -- 172.21.15.86:0/3222669343 shutdown_connections 2024-04-17T07:15:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe40c0a3cc0 0x7fe40c0a6180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe428103b60 0x7fe4281a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281031a0 0x7fe42819a960 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 --2- 172.21.15.86:0/3222669343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe428101fb0 0x7fe42819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 -- 172.21.15.86:0/3222669343 >> 172.21.15.86:0/3222669343 conn(0x7fe4280fd540 msgr2=0x7fe428108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 -- 172.21.15.86:0/3222669343 shutdown_connections 2024-04-17T07:15:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:11.850+0000 7fe42d600640 1 -- 172.21.15.86:0/3222669343 wait complete. 2024-04-17T07:15:11.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:15:11.867 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:15:11.931 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-04-17T07:15:12.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-04-17T07:15:12.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.103+0000 7f2855e00640 1 -- 172.21.15.86:0/3082308340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2850103050 msgr2=0x7f28501054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.103+0000 7f2855e00640 1 --2- 172.21.15.86:0/3082308340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2850103050 0x7f28501054c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f283c009900 tx=0x7f283c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.103+0000 7f2855e00640 1 -- 172.21.15.86:0/3082308340 shutdown_connections 2024-04-17T07:15:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.103+0000 7f2855e00640 1 --2- 172.21.15.86:0/3082308340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2850105a00 0x7f2850105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.103+0000 7f2855e00640 1 --2- 172.21.15.86:0/3082308340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2850103050 0x7f28501054c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.103+0000 7f2855e00640 1 --2- 172.21.15.86:0/3082308340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28501006e0 0x7f2850102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.103+0000 7f2855e00640 1 -- 172.21.15.86:0/3082308340 >> 172.21.15.86:0/3082308340 conn(0x7f28500fc300 msgr2=0x7f28500fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.103+0000 7f2855e00640 1 -- 172.21.15.86:0/3082308340 shutdown_connections 2024-04-17T07:15:12.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.103+0000 7f2855e00640 1 -- 172.21.15.86:0/3082308340 wait complete. 2024-04-17T07:15:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.104+0000 7f2855e00640 1 Processor -- start 2024-04-17T07:15:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f2855e00640 1 -- start start 2024-04-17T07:15:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f2855e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28501006e0 0x7f285019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f2855e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2850103050 0x7f285019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f2855e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2850105a00 0x7f28501a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f2855e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2850113f90 con 0x7f28501006e0 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f2855e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2850113e10 con 0x7f2850103050 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f2855e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2850114110 con 0x7f2850105a00 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f2854a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28501006e0 0x7f285019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f2854a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28501006e0 0x7f285019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38012/0 (socket says 172.21.15.86:38012) 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f2854a00640 1 -- 172.21.15.86:0/34484705 learned_addr learned my addr 172.21.15.86:0/34484705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.105+0000 7f284fe00640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2850103050 0x7f285019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f2855400640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2850105a00 0x7f28501a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f2854a00640 1 -- 172.21.15.86:0/34484705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2850105a00 msgr2=0x7f28501a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f2854a00640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2850105a00 0x7f28501a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f2854a00640 1 -- 172.21.15.86:0/34484705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2850103050 msgr2=0x7f285019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f2854a00640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2850103050 0x7f285019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f2854a00640 1 -- 172.21.15.86:0/34484705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28501a6210 con 0x7f28501006e0 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f2855400640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2850105a00 0x7f28501a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f2854a00640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28501006e0 0x7f285019e5f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f284400c920 tx=0x7f284400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f284d600640 1 -- 172.21.15.86:0/34484705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844007c70 con 0x7f28501006e0 2024-04-17T07:15:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f2855e00640 1 -- 172.21.15.86:0/34484705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28501a6500 con 0x7f28501006e0 2024-04-17T07:15:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f284d600640 1 -- 172.21.15.86:0/34484705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2844007e10 con 0x7f28501006e0 2024-04-17T07:15:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f284d600640 1 -- 172.21.15.86:0/34484705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844005220 con 0x7f28501006e0 2024-04-17T07:15:12.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.106+0000 7f2855e00640 1 -- 172.21.15.86:0/34484705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28501a6a50 con 0x7f28501006e0 2024-04-17T07:15:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.110+0000 7f285a216740 1 Processor -- start 2024-04-17T07:15:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.110+0000 7f284d600640 1 -- 172.21.15.86:0/34484705 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f28440053c0 con 0x7f28501006e0 2024-04-17T07:15:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.110+0000 7f285a216740 1 -- start start 2024-04-17T07:15:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.110+0000 7f285a216740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a98c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f285a216740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a991860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f285a216740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a994360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f285a216740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b12a8f3120 con 0x55b12a98cd60 2024-04-17T07:15:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f285a216740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b12a899340 con 0x55b12a991ed0 2024-04-17T07:15:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f285a216740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b12a8ef0e0 con 0x55b12a98c370 2024-04-17T07:15:12.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f2854a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a991860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f2854a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a991860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38018/0 (socket says 172.21.15.86:38018) 2024-04-17T07:15:12.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f2854a00640 1 -- 172.21.15.86:0/2537864585 learned_addr learned my addr 172.21.15.86:0/2537864585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:12.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f2855400640 1 --2- 172.21.15.86:0/2537864585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a98c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f284fe00640 1 --2- 172.21.15.86:0/2537864585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a994360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f2854a00640 1 -- 172.21.15.86:0/2537864585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 msgr2=0x55b12a98c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.111+0000 7f2854a00640 1 --2- 172.21.15.86:0/2537864585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a98c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.112+0000 7f2854a00640 1 -- 172.21.15.86:0/2537864585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 msgr2=0x55b12a994360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.112+0000 7f2854a00640 1 --2- 172.21.15.86:0/2537864585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a994360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.112+0000 7f2854a00640 1 -- 172.21.15.86:0/2537864585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12a6cf7d0 con 0x55b12a98cd60 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.112+0000 7f2854a00640 1 --2- 172.21.15.86:0/2537864585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a991860 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f28440ce080 tx=0x7f28440d4c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fce77d39933ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.112+0000 7f284d600640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28380a3c40 0x7f28380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.112+0000 7f284d600640 1 -- 172.21.15.86:0/34484705 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f28440d0b10 con 0x7f28501006e0 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.112+0000 7f2841c00640 1 -- 172.21.15.86:0/2537864585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28440e0020 con 0x55b12a98cd60 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.112+0000 7f2841c00640 1 -- 172.21.15.86:0/2537864585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f284400fb00 con 0x55b12a98cd60 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.112+0000 7f2855400640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28380a3c40 0x7f28380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.112+0000 7f2841c00640 1 -- 172.21.15.86:0/2537864585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284409c6e0 con 0x55b12a98cd60 2024-04-17T07:15:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.113+0000 7f285a216740 1 -- 172.21.15.86:0/2537864585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 msgr2=0x55b12a991860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.113+0000 7f2855400640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28380a3c40 0x7f28380a6100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55b12a98d5c0 tx=0x7f28480083f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.113+0000 7f285a216740 1 --2- 172.21.15.86:0/2537864585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a991860 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f28440ce080 tx=0x7f28440d4c40 comp rx=0 tx=0).stop 2024-04-17T07:15:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.113+0000 7f285a216740 1 -- 172.21.15.86:0/2537864585 shutdown_connections 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.113+0000 7f285a216740 1 --2- 172.21.15.86:0/2537864585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a994360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.113+0000 7f285a216740 1 --2- 172.21.15.86:0/2537864585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a991860 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.113+0000 7f285a216740 1 --2- 172.21.15.86:0/2537864585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a98c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.113+0000 7f285a216740 1 -- 172.21.15.86:0/2537864585 >> 172.21.15.86:0/2537864585 conn(0x55b12a9878e0 msgr2=0x55b12a987ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.113+0000 7f285a216740 1 -- 172.21.15.86:0/2537864585 shutdown_connections 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.114+0000 7f285a216740 1 -- 172.21.15.86:0/2537864585 wait complete. 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.114+0000 7f285a216740 1 -- start start 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.114+0000 7f285a216740 1 Processor -- start 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f285a216740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98c370 0x55b12a9968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f285a216740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98cd60 0x55b12a996e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f285a216740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a99de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f285a216740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b12a994e80 con 0x55b12a98c370 2024-04-17T07:15:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f285a216740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b12a8ac710 con 0x55b12a991ed0 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f285a216740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b12a995000 con 0x55b12a98cd60 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f285a216740 10 client.0 _ll_register_callbacks cb 0x55b12a8a7af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f2854a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98c370 0x55b12a9968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f2854a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98c370 0x55b12a9968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38032/0 (socket says 172.21.15.86:38032) 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f2854a00640 1 -- 172.21.15.86:0/267642491 learned_addr learned my addr 172.21.15.86:0/267642491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f2854a00640 1 -- 172.21.15.86:0/267642491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98cd60 msgr2=0x55b12a996e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f2855400640 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a99de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f284fe00640 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98cd60 0x55b12a996e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f2854a00640 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98cd60 0x55b12a996e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f2854a00640 1 -- 172.21.15.86:0/267642491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 msgr2=0x55b12a99de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f2854a00640 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a99de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.115+0000 7f2854a00640 1 -- 172.21.15.86:0/267642491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12a6053f0 con 0x55b12a98c370 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.116+0000 7f2854a00640 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98c370 0x55b12a9968c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2844098d20 tx=0x7f28440d4f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8f99dd93418fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.116+0000 7f2836000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.116+0000 7f2836000640 1 -- 172.21.15.86:0/267642491 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28440e0030 con 0x55b12a98c370 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.116+0000 7f2836000640 1 -- 172.21.15.86:0/267642491 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f28440ce390 con 0x55b12a98c370 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.116+0000 7f285a216740 1 -- 172.21.15.86:0/267642491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b12a900b40 con 0x55b12a98c370 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.116+0000 7f2836000640 1 -- 172.21.15.86:0/267642491 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f284409c270 con 0x55b12a98c370 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.116+0000 7f285a216740 10 client.15789 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.116+0000 7f285a216740 1 -- 172.21.15.86:0/267642491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b12a5c50f0 con 0x55b12a98c370 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.116+0000 7f2836000640 1 -- 172.21.15.86:0/267642491 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b12a900b40 con 0x55b12a98c370 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.117+0000 7f2836000640 1 -- 172.21.15.86:0/267642491 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2844098f80 con 0x55b12a98c370 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.117+0000 7f285a216740 10 client.15789 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.117+0000 7f285a216740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.117+0000 7f285a216740 20 client.15789 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.117+0000 7f285a216740 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b12a9a2330 0x55b12a9a4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.117+0000 7f285a216740 4 client.15789 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:15:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.117+0000 7f285a216740 1 -- 172.21.15.86:0/267642491 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b12a8fa9b0 con 0x55b12a9a2330 2024-04-17T07:15:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.117+0000 7f284fe00640 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b12a9a2330 0x55b12a9a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.118+0000 7f284fe00640 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b12a9a2330 0x55b12a9a4720 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db801ab880b79479 server_cookie=7ee8696b1d2245ff in_seq=0 out_seq=0 2024-04-17T07:15:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.118+0000 7f2836000640 10 client.15789 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f2836000640 1 -- 172.21.15.86:0/267642491 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b12a8fa9b0 con 0x55b12a9a2330 2024-04-17T07:15:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f2836000640 10 client.15789 handle_command_reply: tid=0 2024-04-17T07:15:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f285a216740 1 client.15789 shutdown 2024-04-17T07:15:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f285a216740 1 -- 172.21.15.86:0/267642491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98c370 msgr2=0x55b12a9968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f285a216740 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98c370 0x55b12a9968c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2844098d20 tx=0x7f28440d4f00 comp rx=0 tx=0).stop 2024-04-17T07:15:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f285a216740 1 -- 172.21.15.86:0/267642491 shutdown_connections 2024-04-17T07:15:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f285a216740 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b12a9a2330 0x55b12a9a4720 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f285a216740 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a99de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f285a216740 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98cd60 0x55b12a996e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f2836000640 0 client.15789 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f285a216740 1 --2- 172.21.15.86:0/267642491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98c370 0x55b12a9968c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.120+0000 7f285a216740 1 -- 172.21.15.86:0/267642491 >> 172.21.15.86:0/267642491 conn(0x55b12a9878e0 msgr2=0x55b12a986ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.121+0000 7f285a216740 1 -- 172.21.15.86:0/267642491 shutdown_connections 2024-04-17T07:15:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.121+0000 7f285a216740 1 -- 172.21.15.86:0/267642491 wait complete. 2024-04-17T07:15:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.121+0000 7f285a216740 20 client.15789 trim_cache size 0 max 16384 2024-04-17T07:15:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.142+0000 7f285a216740 1 Processor -- start 2024-04-17T07:15:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.143+0000 7f285a216740 1 -- start start 2024-04-17T07:15:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.143+0000 7f285a216740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a996be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.143+0000 7f285a216740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.143+0000 7f285a216740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a986250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.143+0000 7f285a216740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28501b0e70 con 0x55b12a98cd60 2024-04-17T07:15:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.143+0000 7f285a216740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f28501b29b0 con 0x55b12a991ed0 2024-04-17T07:15:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.143+0000 7f285a216740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28501b1540 con 0x55b12a98c370 2024-04-17T07:15:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.143+0000 7f284fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.143+0000 7f284fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38034/0 (socket says 172.21.15.86:38034) 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.143+0000 7f284fe00640 1 -- 172.21.15.86:0/287603124 learned_addr learned my addr 172.21.15.86:0/287603124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f2855400640 1 --2- 172.21.15.86:0/287603124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a986250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f2854a00640 1 --2- 172.21.15.86:0/287603124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a996be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f284fe00640 1 -- 172.21.15.86:0/287603124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 msgr2=0x55b12a996be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f284fe00640 1 --2- 172.21.15.86:0/287603124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a996be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f284fe00640 1 -- 172.21.15.86:0/287603124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 msgr2=0x55b12a986250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f284fe00640 1 --2- 172.21.15.86:0/287603124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a986250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f284fe00640 1 -- 172.21.15.86:0/287603124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12a988350 con 0x55b12a98cd60 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f284fe00640 1 --2- 172.21.15.86:0/287603124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99ddd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f283c02c170 tx=0x7f283c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b035d5ce7c0e76d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:12.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f2841200640 1 -- 172.21.15.86:0/287603124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c004040 con 0x55b12a98cd60 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f2841200640 1 -- 172.21.15.86:0/287603124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f283c0054f0 con 0x55b12a98cd60 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.144+0000 7f2841200640 1 -- 172.21.15.86:0/287603124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c004ae0 con 0x55b12a98cd60 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.145+0000 7f285a216740 1 -- 172.21.15.86:0/287603124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 msgr2=0x55b12a99ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.145+0000 7f285a216740 1 --2- 172.21.15.86:0/287603124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99ddd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f283c02c170 tx=0x7f283c005a20 comp rx=0 tx=0).stop 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.145+0000 7f285a216740 1 -- 172.21.15.86:0/287603124 shutdown_connections 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.145+0000 7f285a216740 1 --2- 172.21.15.86:0/287603124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a986250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.145+0000 7f285a216740 1 --2- 172.21.15.86:0/287603124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99ddd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.145+0000 7f285a216740 1 --2- 172.21.15.86:0/287603124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a996be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.145+0000 7f285a216740 1 -- 172.21.15.86:0/287603124 >> 172.21.15.86:0/287603124 conn(0x55b12a9878e0 msgr2=0x55b12a9964d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.145+0000 7f285a216740 1 -- 172.21.15.86:0/287603124 shutdown_connections 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.145+0000 7f285a216740 1 -- 172.21.15.86:0/287603124 wait complete. 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.146+0000 7f285a216740 1 -- start start 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.146+0000 7f285a216740 1 Processor -- start 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.146+0000 7f285a216740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a99af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.146+0000 7f285a216740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.146+0000 7f285a216740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a99c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.146+0000 7f285a216740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28501b22e0 con 0x55b12a98cd60 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.146+0000 7f285a216740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f28501b1c10 con 0x55b12a991ed0 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.146+0000 7f285a216740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28501aee40 con 0x55b12a98c370 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.146+0000 7f285a216740 10 client.0 _ll_register_callbacks cb 0x7f28440d4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.147+0000 7f284fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.147+0000 7f284fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38040/0 (socket says 172.21.15.86:38040) 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.147+0000 7f284fe00640 1 -- 172.21.15.86:0/3475197364 learned_addr learned my addr 172.21.15.86:0/3475197364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.147+0000 7f284fe00640 1 -- 172.21.15.86:0/3475197364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 msgr2=0x55b12a99af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.147+0000 7f284fe00640 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a99af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.147+0000 7f284fe00640 1 -- 172.21.15.86:0/3475197364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 msgr2=0x55b12a99c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.147+0000 7f284fe00640 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a99c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.147+0000 7f284fe00640 1 -- 172.21.15.86:0/3475197364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12a99c9a0 con 0x55b12a98cd60 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.147+0000 7f284fe00640 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99b440 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f283c005940 tx=0x7f283c02c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cca99037edd64b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.147+0000 7f2836000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:12.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.148+0000 7f2836000640 1 -- 172.21.15.86:0/3475197364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c004860 con 0x55b12a98cd60 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.148+0000 7f2836000640 1 -- 172.21.15.86:0/3475197364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f283c004a00 con 0x55b12a98cd60 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.148+0000 7f285a216740 1 -- 172.21.15.86:0/3475197364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283c0054f0 con 0x55b12a98cd60 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.148+0000 7f2836000640 1 -- 172.21.15.86:0/3475197364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f283c004c00 con 0x55b12a98cd60 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.148+0000 7f285a216740 10 client.15795 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.148+0000 7f285a216740 1 -- 172.21.15.86:0/3475197364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b12a99dd20 con 0x55b12a98cd60 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.148+0000 7f2836000640 1 -- 172.21.15.86:0/3475197364 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f283c0054f0 con 0x55b12a98cd60 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.148+0000 7f2836000640 1 -- 172.21.15.86:0/3475197364 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f283c04d420 con 0x55b12a98cd60 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.149+0000 7f285a216740 10 client.15795 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.149+0000 7f285a216740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.149+0000 7f285a216740 20 client.15795 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.149+0000 7f285a216740 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b12aa08a90 0x55b12aa0aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.149+0000 7f285a216740 4 client.15795 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.149+0000 7f285a216740 1 -- 172.21.15.86:0/3475197364 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b12a983140 con 0x55b12aa08a90 2024-04-17T07:15:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.149+0000 7f2854a00640 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b12aa08a90 0x55b12aa0aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:12.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.150+0000 7f2854a00640 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b12aa08a90 0x55b12aa0aed0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f15483be04c614 server_cookie=ff0deec0db379e8c in_seq=0 out_seq=0 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.150+0000 7f2836000640 10 client.15795 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f2836000640 1 -- 172.21.15.86:0/3475197364 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b12a983140 con 0x55b12aa08a90 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f2836000640 10 client.15795 handle_command_reply: tid=0 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 client.15795 shutdown 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 -- 172.21.15.86:0/3475197364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 msgr2=0x55b12a99b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99b440 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f283c005940 tx=0x7f283c02c330 comp rx=0 tx=0).stop 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 -- 172.21.15.86:0/3475197364 shutdown_connections 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b12aa08a90 0x55b12aa0aed0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b12a991ed0 0x55b12a99c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b12a98cd60 0x55b12a99b440 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 --2- 172.21.15.86:0/3475197364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b12a98c370 0x55b12a99af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 -- 172.21.15.86:0/3475197364 >> 172.21.15.86:0/3475197364 conn(0x55b12a9878e0 msgr2=0x55b12a98ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f2836000640 0 client.15795 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 -- 172.21.15.86:0/3475197364 shutdown_connections 2024-04-17T07:15:12.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 1 -- 172.21.15.86:0/3475197364 wait complete. 2024-04-17T07:15:12.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.151+0000 7f285a216740 20 client.15795 trim_cache size 0 max 16384 2024-04-17T07:15:12.156 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:15:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:15:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:15:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "5e5a0747-faaf-4300-88c5-88acbf50aa1f": { 2024-04-17T07:15:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:15:12.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "5e5a0747-faaf-4300-88c5-88acbf50aa1f", 2024-04-17T07:15:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:15:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:15:12.157 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:15:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:15:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 -- 172.21.15.86:0/34484705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28380a3c40 msgr2=0x7f28380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28380a3c40 0x7f28380a6100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55b12a98d5c0 tx=0x7f28480083f0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 -- 172.21.15.86:0/34484705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28501006e0 msgr2=0x7f285019e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28501006e0 0x7f285019e5f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f284400c920 tx=0x7f284400cde0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 -- 172.21.15.86:0/34484705 shutdown_connections 2024-04-17T07:15:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28380a3c40 0x7f28380a6100 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2850105a00 0x7f28501a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2850103050 0x7f285019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 --2- 172.21.15.86:0/34484705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28501006e0 0x7f285019e5f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 -- 172.21.15.86:0/34484705 >> 172.21.15.86:0/34484705 conn(0x7f28500fc300 msgr2=0x7f2850101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 -- 172.21.15.86:0/34484705 shutdown_connections 2024-04-17T07:15:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:12.152+0000 7f2855e00640 1 -- 172.21.15.86:0/34484705 wait complete. 2024-04-17T07:15:12.171 INFO:tasks.cephfs.filesystem:scrub status for tag:5e5a0747-faaf-4300-88c5-88acbf50aa1f - {'path': '/', 'tag': '5e5a0747-faaf-4300-88c5-88acbf50aa1f', 'options': 'recursive,force'} 2024-04-17T07:15:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1337783797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:12 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:15:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:12 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:12 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:15:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:12 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:12.678 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1337783797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:12.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:12 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:15:12.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:12 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:12.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:12 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:15:12.679 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:12 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1337783797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:15:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:12 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:15:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:12 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:12 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:15:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:12 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:13 smithi086 ceph-mon[31733]: pgmap v466: 129 pgs: 129 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-04-17T07:15:13.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:13 smithi098 ceph-mon[35418]: pgmap v466: 129 pgs: 129 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-04-17T07:15:13.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:13 smithi103 ceph-mon[35495]: pgmap v466: 129 pgs: 129 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-04-17T07:15:14.417 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-04-17T07:15:14.550 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-04-17T07:15:14.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-04-17T07:15:14.690 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-04-17T07:15:14.834 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/: 2024-04-17T07:15:14.834 INFO:teuthology.orchestra.run.smithi086.stdout:total 2 2024-04-17T07:15:14.834 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:15:14.834 INFO:teuthology.orchestra.run.smithi086.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 .. 2024-04-17T07:15:14.834 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 data 2024-04-17T07:15:14.835 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:14.835 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data: 2024-04-17T07:15:14.835 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:15:14.835 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:15:14.835 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 .. 2024-04-17T07:15:14.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-04-17T07:15:15.005 INFO:teuthology.orchestra.run.smithi086.stdout:[Service] 2024-04-17T07:15:15.006 INFO:teuthology.orchestra.run.smithi086.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-04-17T07:15:15.006 INFO:teuthology.orchestra.run.smithi086.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-04-17T07:15:15.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-04-17T07:15:15.255 INFO:teuthology.orchestra.run.smithi086.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-04-17T07:15:16.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:16 smithi098 ceph-mon[35418]: pgmap v467: 129 pgs: 129 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-17T07:15:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:16 smithi103 ceph-mon[35495]: pgmap v467: 129 pgs: 129 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-17T07:15:16.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:16 smithi086 ceph-mon[31733]: pgmap v467: 129 pgs: 129 active+clean; 663 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-17T07:15:18.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:18 smithi098 ceph-mon[35418]: pgmap v468: 129 pgs: 129 active+clean; 664 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-17T07:15:18.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:18 smithi103 ceph-mon[35495]: pgmap v468: 129 pgs: 129 active+clean; 664 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-17T07:15:18.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:18 smithi086 ceph-mon[31733]: pgmap v468: 129 pgs: 129 active+clean; 664 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-17T07:15:20.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:20 smithi098 ceph-mon[35418]: pgmap v469: 129 pgs: 129 active+clean; 8.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 697 KiB/s wr, 14 op/s 2024-04-17T07:15:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:20 smithi103 ceph-mon[35495]: pgmap v469: 129 pgs: 129 active+clean; 8.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 697 KiB/s wr, 14 op/s 2024-04-17T07:15:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:20 smithi086 ceph-mon[31733]: pgmap v469: 129 pgs: 129 active+clean; 8.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 697 KiB/s wr, 14 op/s 2024-04-17T07:15:22.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:22 smithi098 ceph-mon[35418]: pgmap v470: 129 pgs: 129 active+clean; 8.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 704 KiB/s wr, 12 op/s 2024-04-17T07:15:22.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:22 smithi103 ceph-mon[35495]: pgmap v470: 129 pgs: 129 active+clean; 8.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 704 KiB/s wr, 12 op/s 2024-04-17T07:15:22.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:22 smithi086 ceph-mon[31733]: pgmap v470: 129 pgs: 129 active+clean; 8.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 704 KiB/s wr, 12 op/s 2024-04-17T07:15:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:24 smithi098 ceph-mon[35418]: pgmap v471: 129 pgs: 129 active+clean; 19 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.5 MiB/s wr, 22 op/s 2024-04-17T07:15:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:24 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:15:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:24 smithi103 ceph-mon[35495]: pgmap v471: 129 pgs: 129 active+clean; 19 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.5 MiB/s wr, 22 op/s 2024-04-17T07:15:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:24 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:15:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:24 smithi086 ceph-mon[31733]: pgmap v471: 129 pgs: 129 active+clean; 19 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.5 MiB/s wr, 22 op/s 2024-04-17T07:15:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:24 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:15:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:26 smithi098 ceph-mon[35418]: pgmap v472: 129 pgs: 129 active+clean; 19 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.5 MiB/s wr, 19 op/s 2024-04-17T07:15:26.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:26 smithi103 ceph-mon[35495]: pgmap v472: 129 pgs: 129 active+clean; 19 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.5 MiB/s wr, 19 op/s 2024-04-17T07:15:26.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:26 smithi086 ceph-mon[31733]: pgmap v472: 129 pgs: 129 active+clean; 19 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.5 MiB/s wr, 19 op/s 2024-04-17T07:15:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:27 smithi098 ceph-mon[35418]: pgmap v473: 129 pgs: 129 active+clean; 20 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 26 op/s 2024-04-17T07:15:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:27 smithi103 ceph-mon[35495]: pgmap v473: 129 pgs: 129 active+clean; 20 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 26 op/s 2024-04-17T07:15:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:27 smithi086 ceph-mon[31733]: pgmap v473: 129 pgs: 129 active+clean; 20 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.6 MiB/s wr, 26 op/s 2024-04-17T07:15:30.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:30 smithi098 ceph-mon[35418]: pgmap v474: 129 pgs: 129 active+clean; 22 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.9 MiB/s wr, 44 op/s 2024-04-17T07:15:30.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:30 smithi103 ceph-mon[35495]: pgmap v474: 129 pgs: 129 active+clean; 22 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.9 MiB/s wr, 44 op/s 2024-04-17T07:15:30.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:30 smithi086 ceph-mon[31733]: pgmap v474: 129 pgs: 129 active+clean; 22 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.9 MiB/s wr, 44 op/s 2024-04-17T07:15:32.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:32 smithi098 ceph-mon[35418]: pgmap v475: 129 pgs: 129 active+clean; 22 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.2 MiB/s wr, 42 op/s 2024-04-17T07:15:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:32 smithi103 ceph-mon[35495]: pgmap v475: 129 pgs: 129 active+clean; 22 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.2 MiB/s wr, 42 op/s 2024-04-17T07:15:32.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:32 smithi086 ceph-mon[31733]: pgmap v475: 129 pgs: 129 active+clean; 22 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.2 MiB/s wr, 42 op/s 2024-04-17T07:15:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:34 smithi098 ceph-mon[35418]: pgmap v476: 129 pgs: 129 active+clean; 25 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.6 MiB/s wr, 59 op/s 2024-04-17T07:15:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:34 smithi103 ceph-mon[35495]: pgmap v476: 129 pgs: 129 active+clean; 25 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.6 MiB/s wr, 59 op/s 2024-04-17T07:15:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:34 smithi086 ceph-mon[31733]: pgmap v476: 129 pgs: 129 active+clean; 25 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.6 MiB/s wr, 59 op/s 2024-04-17T07:15:36.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:36 smithi098 ceph-mon[35418]: pgmap v477: 129 pgs: 129 active+clean; 26 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 838 KiB/s wr, 53 op/s 2024-04-17T07:15:36.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:36 smithi103 ceph-mon[35495]: pgmap v477: 129 pgs: 129 active+clean; 26 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 838 KiB/s wr, 53 op/s 2024-04-17T07:15:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:36 smithi086 ceph-mon[31733]: pgmap v477: 129 pgs: 129 active+clean; 26 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 838 KiB/s wr, 53 op/s 2024-04-17T07:15:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:37 smithi086 ceph-mon[31733]: pgmap v478: 129 pgs: 129 active+clean; 28 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.3 MiB/s wr, 72 op/s 2024-04-17T07:15:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:37 smithi098 ceph-mon[35418]: pgmap v478: 129 pgs: 129 active+clean; 28 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.3 MiB/s wr, 72 op/s 2024-04-17T07:15:37.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:37 smithi103 ceph-mon[35495]: pgmap v478: 129 pgs: 129 active+clean; 28 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.3 MiB/s wr, 72 op/s 2024-04-17T07:15:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:40 smithi098 ceph-mon[35418]: pgmap v479: 129 pgs: 129 active+clean; 30 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.9 MiB/s wr, 82 op/s 2024-04-17T07:15:40.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:40 smithi103 ceph-mon[35495]: pgmap v479: 129 pgs: 129 active+clean; 30 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.9 MiB/s wr, 82 op/s 2024-04-17T07:15:40.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:40 smithi086 ceph-mon[31733]: pgmap v479: 129 pgs: 129 active+clean; 30 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.9 MiB/s wr, 82 op/s 2024-04-17T07:15:42.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:15:42.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:15:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.397+0000 7fba1c800640 1 -- 172.21.15.86:0/231410990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14076920 msgr2=0x7fba14076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.397+0000 7fba1c800640 1 --2- 172.21.15.86:0/231410990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14076920 0x7fba14076d40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009ca0 tx=0x7fba0c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.397+0000 7fba1c800640 1 -- 172.21.15.86:0/231410990 shutdown_connections 2024-04-17T07:15:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.397+0000 7fba1c800640 1 --2- 172.21.15.86:0/231410990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba14078500 0x7fba14078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.397+0000 7fba1c800640 1 --2- 172.21.15.86:0/231410990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba14077b60 0x7fba14077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.397+0000 7fba1c800640 1 --2- 172.21.15.86:0/231410990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14076920 0x7fba14076d40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.397+0000 7fba1c800640 1 -- 172.21.15.86:0/231410990 >> 172.21.15.86:0/231410990 conn(0x7fba140ff710 msgr2=0x7fba14101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.397+0000 7fba1c800640 1 -- 172.21.15.86:0/231410990 shutdown_connections 2024-04-17T07:15:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.397+0000 7fba1c800640 1 -- 172.21.15.86:0/231410990 wait complete. 2024-04-17T07:15:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.398+0000 7fba1c800640 1 Processor -- start 2024-04-17T07:15:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba1c800640 1 -- start start 2024-04-17T07:15:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba1c800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba14076920 0x7fba1419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba1c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14077b60 0x7fba1419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba1c800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba14078500 0x7fba141a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba1c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba14114100 con 0x7fba14077b60 2024-04-17T07:15:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba1c800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fba14113f80 con 0x7fba14078500 2024-04-17T07:15:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba1c800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba14114280 con 0x7fba14076920 2024-04-17T07:15:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba19400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14077b60 0x7fba1419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba19400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14077b60 0x7fba1419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59822/0 (socket says 172.21.15.86:59822) 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba19400640 1 -- 172.21.15.86:0/219834446 learned_addr learned my addr 172.21.15.86:0/219834446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.399+0000 7fba19e00640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba14076920 0x7fba1419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba1a800640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba14078500 0x7fba141a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba19400640 1 -- 172.21.15.86:0/219834446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba14076920 msgr2=0x7fba1419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba19400640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba14076920 0x7fba1419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba19400640 1 -- 172.21.15.86:0/219834446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba14078500 msgr2=0x7fba141a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba19400640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba14078500 0x7fba141a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba19400640 1 -- 172.21.15.86:0/219834446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba141a6450 con 0x7fba14077b60 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba1a800640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba14078500 0x7fba141a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba19400640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14077b60 0x7fba1419edc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fba0400d770 tx=0x7fba0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba0aa00640 1 -- 172.21.15.86:0/219834446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04017070 con 0x7fba14077b60 2024-04-17T07:15:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba0aa00640 1 -- 172.21.15.86:0/219834446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba040054d0 con 0x7fba14077b60 2024-04-17T07:15:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba1c800640 1 -- 172.21.15.86:0/219834446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba141a6740 con 0x7fba14077b60 2024-04-17T07:15:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba0aa00640 1 -- 172.21.15.86:0/219834446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04004cf0 con 0x7fba14077b60 2024-04-17T07:15:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.400+0000 7fba1c800640 1 -- 172.21.15.86:0/219834446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba141a6c90 con 0x7fba14077b60 2024-04-17T07:15:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.403+0000 7fba1f5e7740 1 Processor -- start 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.403+0000 7fba0aa00640 1 -- 172.21.15.86:0/219834446 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fba04004e90 con 0x7fba14077b60 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.404+0000 7fba1f5e7740 1 -- start start 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.404+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 0x55d7c6eb45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.404+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4b80 0x55d7c6eb9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.404+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ebc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.404+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7c6e1b120 con 0x55d7c6eb4b80 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.404+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d7c6dc1340 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.404+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7c6e170e0 con 0x55d7c6eb4190 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.404+0000 7fba19e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4b80 0x55d7c6eb9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.404+0000 7fba19e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4b80 0x55d7c6eb9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59834/0 (socket says 172.21.15.86:59834) 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.404+0000 7fba19e00640 1 -- 172.21.15.86:0/3504233119 learned_addr learned my addr 172.21.15.86:0/3504233119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.405+0000 7fba1a800640 1 --2- 172.21.15.86:0/3504233119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 0x55d7c6eb45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.405+0000 7fba19400640 1 --2- 172.21.15.86:0/3504233119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ebc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.405+0000 7fba19e00640 1 -- 172.21.15.86:0/3504233119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 msgr2=0x55d7c6eb45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.405+0000 7fba19e00640 1 --2- 172.21.15.86:0/3504233119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 0x55d7c6eb45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.405+0000 7fba19e00640 1 -- 172.21.15.86:0/3504233119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 msgr2=0x55d7c6ebc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.405+0000 7fba19e00640 1 --2- 172.21.15.86:0/3504233119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ebc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.405+0000 7fba19e00640 1 -- 172.21.15.86:0/3504233119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c6bf77d0 con 0x55d7c6eb4b80 2024-04-17T07:15:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.405+0000 7fba1a800640 1 --2- 172.21.15.86:0/3504233119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 0x55d7c6eb45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:15:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.405+0000 7fba19e00640 1 --2- 172.21.15.86:0/3504233119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4b80 0x55d7c6eb9680 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fba0c000f80 tx=0x7fba0c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f941b5bcad25b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.405+0000 7fba0aa00640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fba000a3cc0 0x7fba000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.406+0000 7fba0aa00640 1 -- 172.21.15.86:0/219834446 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fba040d0410 con 0x7fba14077b60 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.406+0000 7fba1a800640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fba000a3cc0 0x7fba000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.406+0000 7fb9ff000640 1 -- 172.21.15.86:0/3504233119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c043020 con 0x55d7c6eb4b80 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.406+0000 7fb9ff000640 1 -- 172.21.15.86:0/3504233119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba0c032db0 con 0x55d7c6eb4b80 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.406+0000 7fb9ff000640 1 -- 172.21.15.86:0/3504233119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c039040 con 0x55d7c6eb4b80 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.406+0000 7fba1f5e7740 1 -- 172.21.15.86:0/3504233119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4b80 msgr2=0x55d7c6eb9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.406+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/3504233119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4b80 0x55d7c6eb9680 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fba0c000f80 tx=0x7fba0c0093f0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.406+0000 7fba1a800640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fba000a3cc0 0x7fba000a6180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fba10005fd0 tx=0x7fba10009330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.407+0000 7fba1f5e7740 1 -- 172.21.15.86:0/3504233119 shutdown_connections 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.407+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/3504233119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ebc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.407+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/3504233119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4b80 0x55d7c6eb9680 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.407+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/3504233119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 0x55d7c6eb45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.407+0000 7fba1f5e7740 1 -- 172.21.15.86:0/3504233119 >> 172.21.15.86:0/3504233119 conn(0x55d7c6eaf700 msgr2=0x55d7c6eafb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.407+0000 7fba1f5e7740 1 -- 172.21.15.86:0/3504233119 shutdown_connections 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.407+0000 7fba1f5e7740 1 -- 172.21.15.86:0/3504233119 wait complete. 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.407+0000 7fba1f5e7740 1 -- start start 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.407+0000 7fba1f5e7740 1 Processor -- start 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4190 0x55d7c6ebe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ebec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ec5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7c6ebcc40 con 0x55d7c6eb4190 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d7c6dd4710 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7c6ebcdc0 con 0x55d7c6eb4b80 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba1f5e7740 10 client.0 _ll_register_callbacks cb 0x55d7c6dcfaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba19e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4190 0x55d7c6ebe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba19e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4190 0x55d7c6ebe700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59836/0 (socket says 172.21.15.86:59836) 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba19e00640 1 -- 172.21.15.86:0/4093275476 learned_addr learned my addr 172.21.15.86:0/4093275476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba19400640 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ebec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba19e00640 1 -- 172.21.15.86:0/4093275476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 msgr2=0x55d7c6ebec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba1a800640 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ec5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba19e00640 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ebec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.408+0000 7fba19e00640 1 -- 172.21.15.86:0/4093275476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 msgr2=0x55d7c6ec5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.409+0000 7fba19e00640 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ec5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.409+0000 7fba19e00640 1 -- 172.21.15.86:0/4093275476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c6b2d3f0 con 0x55d7c6eb4190 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.409+0000 7fba1a800640 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ec5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.409+0000 7fba19e00640 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4190 0x55d7c6ebe700 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fba0c006010 tx=0x7fba0c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b1cbecaec24e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.409+0000 7fb9f3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.409+0000 7fb9f3400640 1 -- 172.21.15.86:0/4093275476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c04e020 con 0x55d7c6eb4190 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.409+0000 7fb9f3400640 1 -- 172.21.15.86:0/4093275476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fba0c004140 con 0x55d7c6eb4190 2024-04-17T07:15:42.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.409+0000 7fba1f5e7740 1 -- 172.21.15.86:0/4093275476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba0c032db0 con 0x55d7c6eb4190 2024-04-17T07:15:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fb9f3400640 1 -- 172.21.15.86:0/4093275476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba0c059050 con 0x55d7c6eb4190 2024-04-17T07:15:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fba1f5e7740 10 client.15807 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fba1f5e7740 1 -- 172.21.15.86:0/4093275476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7c6aed0f0 con 0x55d7c6eb4190 2024-04-17T07:15:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fb9f3400640 1 -- 172.21.15.86:0/4093275476 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba0c032db0 con 0x55d7c6eb4190 2024-04-17T07:15:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fb9f3400640 1 -- 172.21.15.86:0/4093275476 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fba0c03f980 con 0x55d7c6eb4190 2024-04-17T07:15:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fba1f5e7740 10 client.15807 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fba1f5e7740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fba1f5e7740 20 client.15807 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c6eca170 0x55d7c6ecc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fba1f5e7740 4 client.15807 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:15:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.410+0000 7fba1f5e7740 1 -- 172.21.15.86:0/4093275476 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7c6e28b40 con 0x55d7c6eca170 2024-04-17T07:15:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.411+0000 7fba19400640 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c6eca170 0x55d7c6ecc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.412+0000 7fba19400640 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c6eca170 0x55d7c6ecc560 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d04a55443f47488 server_cookie=efdab5a8403142ba in_seq=0 out_seq=0 2024-04-17T07:15:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.412+0000 7fb9f3400640 10 client.15807 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fb9f3400640 1 -- 172.21.15.86:0/4093275476 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7c6e28b40 con 0x55d7c6eca170 2024-04-17T07:15:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fb9f3400640 10 client.15807 handle_command_reply: tid=0 2024-04-17T07:15:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fba1f5e7740 1 client.15807 shutdown 2024-04-17T07:15:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fba1f5e7740 1 -- 172.21.15.86:0/4093275476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4190 msgr2=0x55d7c6ebe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4190 0x55d7c6ebe700 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fba0c006010 tx=0x7fba0c009a20 comp rx=0 tx=0).stop 2024-04-17T07:15:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fba1f5e7740 1 -- 172.21.15.86:0/4093275476 shutdown_connections 2024-04-17T07:15:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c6eca170 0x55d7c6ecc560 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ec5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ebec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/4093275476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb4190 0x55d7c6ebe700 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.415+0000 7fb9f3400640 0 client.15807 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:42.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.416+0000 7fba1f5e7740 1 -- 172.21.15.86:0/4093275476 >> 172.21.15.86:0/4093275476 conn(0x55d7c6eaf700 msgr2=0x55d7c6eae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.416+0000 7fba1f5e7740 1 -- 172.21.15.86:0/4093275476 shutdown_connections 2024-04-17T07:15:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.416+0000 7fba1f5e7740 1 -- 172.21.15.86:0/4093275476 wait complete. 2024-04-17T07:15:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.416+0000 7fba1f5e7740 20 client.15807 trim_cache size 0 max 16384 2024-04-17T07:15:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1f5e7740 1 Processor -- start 2024-04-17T07:15:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1f5e7740 1 -- start start 2024-04-17T07:15:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb4190 0x55d7c6ed0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ec5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ed12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba141b10c0 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fba141b2c00 con 0x55d7c6eb4190 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba141b1790 con 0x55d7c6eb4b80 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ed12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ed12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59852/0 (socket says 172.21.15.86:59852) 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.435+0000 7fba1a800640 1 -- 172.21.15.86:0/1211931236 learned_addr learned my addr 172.21.15.86:0/1211931236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fba19e00640 1 --2- 172.21.15.86:0/1211931236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb4190 0x55d7c6ed0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fba19400640 1 --2- 172.21.15.86:0/1211931236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ec5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fba1a800640 1 -- 172.21.15.86:0/1211931236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 msgr2=0x55d7c6ec5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fba1a800640 1 --2- 172.21.15.86:0/1211931236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ec5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fba1a800640 1 -- 172.21.15.86:0/1211931236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb4190 msgr2=0x55d7c6ed0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fba1a800640 1 --2- 172.21.15.86:0/1211931236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb4190 0x55d7c6ed0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fba1a800640 1 -- 172.21.15.86:0/1211931236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c6eccb90 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fba19400640 1 --2- 172.21.15.86:0/1211931236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ec5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fba1a800640 1 --2- 172.21.15.86:0/1211931236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ed12f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fba10009a50 tx=0x7fba10013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a983954c5571c709 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fb9fe600640 1 -- 172.21.15.86:0/1211931236 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1001f620 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fb9fe600640 1 -- 172.21.15.86:0/1211931236 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba1001fc70 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.436+0000 7fb9fe600640 1 -- 172.21.15.86:0/1211931236 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10011c40 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.437+0000 7fba1f5e7740 1 -- 172.21.15.86:0/1211931236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 msgr2=0x55d7c6ed12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.437+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/1211931236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ed12f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fba10009a50 tx=0x7fba10013000 comp rx=0 tx=0).stop 2024-04-17T07:15:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.437+0000 7fba1f5e7740 1 -- 172.21.15.86:0/1211931236 shutdown_connections 2024-04-17T07:15:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.437+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/1211931236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6ed12f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.437+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/1211931236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ec5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.437+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/1211931236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb4190 0x55d7c6ed0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.437+0000 7fba1f5e7740 1 -- 172.21.15.86:0/1211931236 >> 172.21.15.86:0/1211931236 conn(0x55d7c6eaf700 msgr2=0x55d7c6eae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.437+0000 7fba1f5e7740 1 -- 172.21.15.86:0/1211931236 shutdown_connections 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.437+0000 7fba1f5e7740 1 -- 172.21.15.86:0/1211931236 wait complete. 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.438+0000 7fba1f5e7740 1 -- start start 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.438+0000 7fba1f5e7740 1 Processor -- start 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.438+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 0x55d7c6ec5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.438+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ec58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.438+0000 7fba1f5e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6f2ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.438+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba141b2530 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.438+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fba141b1e60 con 0x55d7c6eb4b80 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.438+0000 7fba1f5e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba141af090 con 0x55d7c6eb4190 2024-04-17T07:15:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.438+0000 7fba1f5e7740 10 client.0 _ll_register_callbacks cb 0x7fba0c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.438+0000 7fba1a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6f2ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fba1a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6f2ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59866/0 (socket says 172.21.15.86:59866) 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fba1a800640 1 -- 172.21.15.86:0/500655039 learned_addr learned my addr 172.21.15.86:0/500655039 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fba19400640 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ec58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fba19e00640 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 0x55d7c6ec5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fba1a800640 1 -- 172.21.15.86:0/500655039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 msgr2=0x55d7c6ec5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fba1a800640 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 0x55d7c6ec5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fba1a800640 1 -- 172.21.15.86:0/500655039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb4b80 msgr2=0x55d7c6ec58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fba1a800640 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ec58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fba1a800640 1 -- 172.21.15.86:0/500655039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c6f2f300 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fba1a800640 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6f2ec90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fba10009970 tx=0x7fba10009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f0a30ea901395f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.439+0000 7fb9f3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fb9f3400640 1 -- 172.21.15.86:0/500655039 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10011cd0 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fb9f3400640 1 -- 172.21.15.86:0/500655039 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fba10005050 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fba1f5e7740 1 -- 172.21.15.86:0/500655039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1001fc70 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fb9f3400640 1 -- 172.21.15.86:0/500655039 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba10005270 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fba1f5e7740 10 client.15813 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fba1f5e7740 1 -- 172.21.15.86:0/500655039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7c6f30580 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fb9f3400640 1 -- 172.21.15.86:0/500655039 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba1001fc70 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fb9f3400640 1 -- 172.21.15.86:0/500655039 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fba1000f040 con 0x55d7c6eb9cf0 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fba1f5e7740 10 client.15813 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fba1f5e7740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fba1f5e7740 20 client.15813 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.440+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c6f30f00 0x55d7c6f33320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.441+0000 7fba1f5e7740 4 client.15813 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.441+0000 7fba1f5e7740 1 -- 172.21.15.86:0/500655039 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7c6e229b0 con 0x55d7c6f30f00 2024-04-17T07:15:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.441+0000 7fba19e00640 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c6f30f00 0x55d7c6f33320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.442+0000 7fb9f3400640 10 client.15813 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.442+0000 7fba19e00640 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c6f30f00 0x55d7c6f33320 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f87d0d8bd4d746 server_cookie=e6b748e1fb4a761f in_seq=0 out_seq=0 2024-04-17T07:15:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.444+0000 7fb9f3400640 1 -- 172.21.15.86:0/500655039 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d7c6e229b0 con 0x55d7c6f30f00 2024-04-17T07:15:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.444+0000 7fb9f3400640 10 client.15813 handle_command_reply: tid=0 2024-04-17T07:15:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.444+0000 7fba1f5e7740 1 client.15813 shutdown 2024-04-17T07:15:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 1 -- 172.21.15.86:0/500655039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 msgr2=0x55d7c6f2ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6f2ec90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fba10009970 tx=0x7fba10009af0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 1 -- 172.21.15.86:0/500655039 shutdown_connections 2024-04-17T07:15:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c6f30f00 0x55d7c6f33320 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c6eb9cf0 0x55d7c6f2ec90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c6eb4b80 0x55d7c6ec58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 1 --2- 172.21.15.86:0/500655039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c6eb4190 0x55d7c6ec5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fb9f3400640 0 client.15813 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 1 -- 172.21.15.86:0/500655039 >> 172.21.15.86:0/500655039 conn(0x55d7c6eaf700 msgr2=0x55d7c6eb2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 1 -- 172.21.15.86:0/500655039 shutdown_connections 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 1 -- 172.21.15.86:0/500655039 wait complete. 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.445+0000 7fba1f5e7740 20 client.15813 trim_cache size 0 max 16384 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:15:42.447 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:15:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 -- 172.21.15.86:0/219834446 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fba000a3cc0 msgr2=0x7fba000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fba000a3cc0 0x7fba000a6180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fba10005fd0 tx=0x7fba10009330 comp rx=0 tx=0).stop 2024-04-17T07:15:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 -- 172.21.15.86:0/219834446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14077b60 msgr2=0x7fba1419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14077b60 0x7fba1419edc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fba0400d770 tx=0x7fba0400dc30 comp rx=0 tx=0).stop 2024-04-17T07:15:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 -- 172.21.15.86:0/219834446 shutdown_connections 2024-04-17T07:15:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fba000a3cc0 0x7fba000a6180 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba14078500 0x7fba141a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba14077b60 0x7fba1419edc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 --2- 172.21.15.86:0/219834446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba14076920 0x7fba1419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 -- 172.21.15.86:0/219834446 >> 172.21.15.86:0/219834446 conn(0x7fba140ff710 msgr2=0x7fba14100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 -- 172.21.15.86:0/219834446 shutdown_connections 2024-04-17T07:15:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.446+0000 7fba1c800640 1 -- 172.21.15.86:0/219834446 wait complete. 2024-04-17T07:15:42.466 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:15:42.467 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:15:42.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:42 smithi098 ceph-mon[35418]: pgmap v480: 129 pgs: 129 active+clean; 30 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.7 MiB/s wr, 63 op/s 2024-04-17T07:15:42.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:42 smithi103 ceph-mon[35495]: pgmap v480: 129 pgs: 129 active+clean; 30 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.7 MiB/s wr, 63 op/s 2024-04-17T07:15:42.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:42 smithi086 ceph-mon[31733]: pgmap v480: 129 pgs: 129 active+clean; 30 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.7 MiB/s wr, 63 op/s 2024-04-17T07:15:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.692+0000 7f06e7e00640 1 -- 172.21.15.86:0/653634213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e8109c90 msgr2=0x7f06e810c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.692+0000 7f06e7e00640 1 --2- 172.21.15.86:0/653634213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e8109c90 0x7f06e810c130 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009ca0 tx=0x7f06e002c180 comp rx=0 tx=0).stop 2024-04-17T07:15:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.692+0000 7f06e7e00640 1 -- 172.21.15.86:0/653634213 shutdown_connections 2024-04-17T07:15:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.692+0000 7f06e7e00640 1 --2- 172.21.15.86:0/653634213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e8109c90 0x7f06e810c130 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.692+0000 7f06e7e00640 1 --2- 172.21.15.86:0/653634213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8074960 0x7f06e8074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.692+0000 7f06e7e00640 1 --2- 172.21.15.86:0/653634213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06e8075f40 0x7f06e8074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.692+0000 7f06e7e00640 1 -- 172.21.15.86:0/653634213 >> 172.21.15.86:0/653634213 conn(0x7f06e80fd370 msgr2=0x7f06e80ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:42.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.692+0000 7f06e7e00640 1 -- 172.21.15.86:0/653634213 shutdown_connections 2024-04-17T07:15:42.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.692+0000 7f06e7e00640 1 -- 172.21.15.86:0/653634213 wait complete. 2024-04-17T07:15:42.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.693+0000 7f06e7e00640 1 Processor -- start 2024-04-17T07:15:42.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.693+0000 7f06e7e00640 1 -- start start 2024-04-17T07:15:42.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.693+0000 7f06e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e8074960 0x7f06e819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06e8075f40 0x7f06e819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8109c90 0x7f06e81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06e810fb40 con 0x7f06e8074960 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e7e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f06e810f9c0 con 0x7f06e8075f40 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e7e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06e810fcc0 con 0x7f06e8109c90 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e8074960 0x7f06e819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e8074960 0x7f06e819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59888/0 (socket says 172.21.15.86:59888) 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e6a00640 1 -- 172.21.15.86:0/3388578970 learned_addr learned my addr 172.21.15.86:0/3388578970 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e6a00640 1 -- 172.21.15.86:0/3388578970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8109c90 msgr2=0x7f06e81a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e6a00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8109c90 0x7f06e81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e6a00640 1 -- 172.21.15.86:0/3388578970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06e8075f40 msgr2=0x7f06e819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e6a00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06e8075f40 0x7f06e819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e6a00640 1 -- 172.21.15.86:0/3388578970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e81a1dd0 con 0x7f06e8074960 2024-04-17T07:15:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.694+0000 7f06e6a00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e8074960 0x7f06e819a1b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f06d800d950 tx=0x7f06d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.695+0000 7f06df400640 1 -- 172.21.15.86:0/3388578970 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8017030 con 0x7f06e8074960 2024-04-17T07:15:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.695+0000 7f06df400640 1 -- 172.21.15.86:0/3388578970 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d8004020 con 0x7f06e8074960 2024-04-17T07:15:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.695+0000 7f06e7e00640 1 -- 172.21.15.86:0/3388578970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e81a20c0 con 0x7f06e8074960 2024-04-17T07:15:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.695+0000 7f06df400640 1 -- 172.21.15.86:0/3388578970 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80047a0 con 0x7f06e8074960 2024-04-17T07:15:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.695+0000 7f06e7e00640 1 -- 172.21.15.86:0/3388578970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e81a2610 con 0x7f06e8074960 2024-04-17T07:15:42.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.696+0000 7f06e7e00640 1 -- 172.21.15.86:0/3388578970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e8108a50 con 0x7f06e8074960 2024-04-17T07:15:42.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.698+0000 7f06df400640 1 -- 172.21.15.86:0/3388578970 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f06d80041d0 con 0x7f06e8074960 2024-04-17T07:15:42.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.700+0000 7f06df400640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06cc0a3c40 0x7f06cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:42.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.700+0000 7f06df400640 1 -- 172.21.15.86:0/3388578970 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f06d80d04c0 con 0x7f06e8074960 2024-04-17T07:15:42.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.700+0000 7f06dfe00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06cc0a3c40 0x7f06cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:42.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.701+0000 7f06dfe00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06cc0a3c40 0x7f06cc0a6100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f06d00058c0 tx=0x7f06d0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.706+0000 7f06df400640 1 -- 172.21.15.86:0/3388578970 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f06d809c960 con 0x7f06e8074960 2024-04-17T07:15:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.988+0000 7f06e7e00640 1 -- 172.21.15.86:0/3388578970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06e802d810 con 0x7f06e8074960 2024-04-17T07:15:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.989+0000 7f06df400640 1 -- 172.21.15.86:0/3388578970 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f06d809c260 con 0x7f06e8074960 2024-04-17T07:15:42.989 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:15:42.989 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:15:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:15:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.991+0000 7f06e7e00640 1 -- 172.21.15.86:0/3388578970 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06cc0a3c40 msgr2=0x7f06cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.991+0000 7f06e7e00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06cc0a3c40 0x7f06cc0a6100 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f06d00058c0 tx=0x7f06d0005850 comp rx=0 tx=0).stop 2024-04-17T07:15:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.992+0000 7f06e7e00640 1 -- 172.21.15.86:0/3388578970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e8074960 msgr2=0x7f06e819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.992+0000 7f06e7e00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e8074960 0x7f06e819a1b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f06d800d950 tx=0x7f06d800de10 comp rx=0 tx=0).stop 2024-04-17T07:15:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.992+0000 7f06e7e00640 1 -- 172.21.15.86:0/3388578970 shutdown_connections 2024-04-17T07:15:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.992+0000 7f06e7e00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06cc0a3c40 0x7f06cc0a6100 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.992+0000 7f06e7e00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8109c90 0x7f06e81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.992+0000 7f06e7e00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06e8075f40 0x7f06e819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.992+0000 7f06e7e00640 1 --2- 172.21.15.86:0/3388578970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e8074960 0x7f06e819a1b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.992+0000 7f06e7e00640 1 -- 172.21.15.86:0/3388578970 >> 172.21.15.86:0/3388578970 conn(0x7f06e80fd370 msgr2=0x7f06e8105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.992+0000 7f06e7e00640 1 -- 172.21.15.86:0/3388578970 shutdown_connections 2024-04-17T07:15:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:42.992+0000 7f06e7e00640 1 -- 172.21.15.86:0/3388578970 wait complete. 2024-04-17T07:15:43.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:15:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.236+0000 7f14b9600640 1 -- 172.21.15.86:0/4004316277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b41097b0 msgr2=0x7f14b4109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.236+0000 7f14b9600640 1 --2- 172.21.15.86:0/4004316277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b41097b0 0x7f14b4109c30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009900 tx=0x7f14a002bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.236+0000 7f14b9600640 1 -- 172.21.15.86:0/4004316277 shutdown_connections 2024-04-17T07:15:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.236+0000 7f14b9600640 1 --2- 172.21.15.86:0/4004316277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b410a170 0x7f14b4110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.236+0000 7f14b9600640 1 --2- 172.21.15.86:0/4004316277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b41097b0 0x7f14b4109c30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.236+0000 7f14b9600640 1 --2- 172.21.15.86:0/4004316277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b41085c0 0x7f14b41089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.236+0000 7f14b9600640 1 -- 172.21.15.86:0/4004316277 >> 172.21.15.86:0/4004316277 conn(0x7f14b4077560 msgr2=0x7f14b40799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.236+0000 7f14b9600640 1 -- 172.21.15.86:0/4004316277 shutdown_connections 2024-04-17T07:15:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.236+0000 7f14b9600640 1 -- 172.21.15.86:0/4004316277 wait complete. 2024-04-17T07:15:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b9600640 1 Processor -- start 2024-04-17T07:15:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b9600640 1 -- start start 2024-04-17T07:15:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b9600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b41085c0 0x7f14b419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b9600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b41097b0 0x7f14b419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b410a170 0x7f14b41a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41141e0 con 0x7f14b410a170 2024-04-17T07:15:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b9600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4114060 con 0x7f14b41097b0 2024-04-17T07:15:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b9600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4114360 con 0x7f14b41085c0 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b410a170 0x7f14b41a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b410a170 0x7f14b41a5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59914/0 (socket says 172.21.15.86:59914) 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.238+0000 7f14b3400640 1 -- 172.21.15.86:0/733886169 learned_addr learned my addr 172.21.15.86:0/733886169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.239+0000 7f14b2a00640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b41085c0 0x7f14b419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.239+0000 7f14b2000640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b41097b0 0x7f14b419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.239+0000 7f14b3400640 1 -- 172.21.15.86:0/733886169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b41085c0 msgr2=0x7f14b419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.239+0000 7f14b3400640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b41085c0 0x7f14b419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.239+0000 7f14b3400640 1 -- 172.21.15.86:0/733886169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b41097b0 msgr2=0x7f14b419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.239+0000 7f14b3400640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b41097b0 0x7f14b419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.239+0000 7f14b3400640 1 -- 172.21.15.86:0/733886169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b41a64f0 con 0x7f14b410a170 2024-04-17T07:15:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.239+0000 7f14b3400640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b410a170 0x7f14b41a5e80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00eaa0 tx=0x7f14ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.240+0000 7f14ab400640 1 -- 172.21.15.86:0/733886169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00cdb0 con 0x7f14b410a170 2024-04-17T07:15:43.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.240+0000 7f14ab400640 1 -- 172.21.15.86:0/733886169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14ac005020 con 0x7f14b410a170 2024-04-17T07:15:43.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.240+0000 7f14b9600640 1 -- 172.21.15.86:0/733886169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b41a67e0 con 0x7f14b410a170 2024-04-17T07:15:43.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.240+0000 7f14ab400640 1 -- 172.21.15.86:0/733886169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0116d0 con 0x7f14b410a170 2024-04-17T07:15:43.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.240+0000 7f14b9600640 1 -- 172.21.15.86:0/733886169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b41a6d80 con 0x7f14b410a170 2024-04-17T07:15:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.244+0000 7f14ab400640 1 -- 172.21.15.86:0/733886169 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f14ac0051d0 con 0x7f14b410a170 2024-04-17T07:15:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.244+0000 7f14bc364740 1 Processor -- start 2024-04-17T07:15:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.244+0000 7f14bc364740 1 -- start start 2024-04-17T07:15:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.244+0000 7f14bc364740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 0x55e2a46db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.244+0000 7f14bc364740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46db990 0x55e2a46e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.244+0000 7f14bc364740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46e0b00 0x55e2a46e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.244+0000 7f14bc364740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2a4641020 con 0x55e2a46e0b00 2024-04-17T07:15:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.244+0000 7f14bc364740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e2a45e7460 con 0x55e2a46dafa0 2024-04-17T07:15:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.244+0000 7f14bc364740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e2a463cfe0 con 0x55e2a46db990 2024-04-17T07:15:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.245+0000 7f14b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46e0b00 0x55e2a46e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.245+0000 7f14b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46e0b00 0x55e2a46e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59920/0 (socket says 172.21.15.86:59920) 2024-04-17T07:15:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.245+0000 7f14b2000640 1 -- 172.21.15.86:0/1425019159 learned_addr learned my addr 172.21.15.86:0/1425019159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.245+0000 7f14b2000640 1 --2- 172.21.15.86:0/1425019159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 0x55e2a46db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.245+0000 7f14b2a00640 1 --2- 172.21.15.86:0/1425019159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46db990 0x55e2a46e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.245+0000 7f14b2000640 1 -- 172.21.15.86:0/1425019159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46db990 msgr2=0x55e2a46e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.245+0000 7f14b2000640 1 --2- 172.21.15.86:0/1425019159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46db990 0x55e2a46e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.245+0000 7f14b2000640 1 -- 172.21.15.86:0/1425019159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 msgr2=0x55e2a46db3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.245+0000 7f14b2000640 1 --2- 172.21.15.86:0/1425019159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 0x55e2a46db3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.246+0000 7f14b2000640 1 -- 172.21.15.86:0/1425019159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a45ef040 con 0x55e2a46e0b00 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.246+0000 7f14ab400640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14980a3c40 0x7f14980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.246+0000 7f14b2000640 1 --2- 172.21.15.86:0/1425019159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46e0b00 0x55e2a46e2f90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f14a00096c0 tx=0x7f14a0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5ebac598f7b920 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.246+0000 7f14ab400640 1 -- 172.21.15.86:0/733886169 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f14ac0d0bc0 con 0x7f14b410a170 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.247+0000 7f149f400640 1 -- 172.21.15.86:0/1425019159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0004240 con 0x55e2a46e0b00 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.247+0000 7f149f400640 1 -- 172.21.15.86:0/1425019159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14a0048070 con 0x55e2a46e0b00 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.247+0000 7f14b2a00640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14980a3c40 0x7f14980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.247+0000 7f149f400640 1 -- 172.21.15.86:0/1425019159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0004a40 con 0x55e2a46e0b00 2024-04-17T07:15:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.248+0000 7f14bc364740 1 -- 172.21.15.86:0/1425019159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46e0b00 msgr2=0x55e2a46e2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.248+0000 7f14b2a00640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14980a3c40 0x7f14980a6100 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55e2a46dbd90 tx=0x7f14a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.248+0000 7f14bc364740 1 --2- 172.21.15.86:0/1425019159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46e0b00 0x55e2a46e2f90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f14a00096c0 tx=0x7f14a0005b50 comp rx=0 tx=0).stop 2024-04-17T07:15:43.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.248+0000 7f14bc364740 1 -- 172.21.15.86:0/1425019159 shutdown_connections 2024-04-17T07:15:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.248+0000 7f14bc364740 1 --2- 172.21.15.86:0/1425019159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46e0b00 0x55e2a46e2f90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.248+0000 7f14bc364740 1 --2- 172.21.15.86:0/1425019159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46db990 0x55e2a46e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.248+0000 7f14bc364740 1 --2- 172.21.15.86:0/1425019159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 0x55e2a46db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.248+0000 7f14bc364740 1 -- 172.21.15.86:0/1425019159 >> 172.21.15.86:0/1425019159 conn(0x55e2a46d6510 msgr2=0x55e2a46d6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.248+0000 7f14bc364740 1 -- 172.21.15.86:0/1425019159 shutdown_connections 2024-04-17T07:15:43.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.248+0000 7f14bc364740 1 -- 172.21.15.86:0/1425019159 wait complete. 2024-04-17T07:15:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.249+0000 7f14bc364740 1 -- start start 2024-04-17T07:15:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.249+0000 7f14bc364740 1 Processor -- start 2024-04-17T07:15:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14bc364740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46dafa0 0x55e2a46e5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14bc364740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46db990 0x55e2a46e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14bc364740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46eaba0 0x55e2a46ecf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14bc364740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2a46e3a50 con 0x55e2a46db990 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14bc364740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e2a45fa830 con 0x55e2a46eaba0 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14bc364740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e2a46e3bd0 con 0x55e2a46dafa0 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14bc364740 10 client.0 _ll_register_callbacks cb 0x55e2a45c8fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46db990 0x55e2a46e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46db990 0x55e2a46e5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59934/0 (socket says 172.21.15.86:59934) 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14b2000640 1 -- 172.21.15.86:0/3013433449 learned_addr learned my addr 172.21.15.86:0/3013433449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46dafa0 0x55e2a46e5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14b3400640 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46eaba0 0x55e2a46ecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14b2000640 1 -- 172.21.15.86:0/3013433449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46dafa0 msgr2=0x55e2a46e5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.250+0000 7f14b2000640 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46dafa0 0x55e2a46e5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f14b2000640 1 -- 172.21.15.86:0/3013433449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46eaba0 msgr2=0x55e2a46ecf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f14b2000640 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46eaba0 0x55e2a46ecf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f14b2000640 1 -- 172.21.15.86:0/3013433449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a45fe920 con 0x55e2a46db990 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f14b3400640 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46eaba0 0x55e2a46ecf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f14b2000640 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46db990 0x55e2a46e5b70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f14a002c240 tx=0x7f14a002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee99c86d2a077d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f1493e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f1493e00640 1 -- 172.21.15.86:0/3013433449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0003a00 con 0x55e2a46db990 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f1493e00640 1 -- 172.21.15.86:0/3013433449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f14a0003ba0 con 0x55e2a46db990 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f14bc364740 1 -- 172.21.15.86:0/3013433449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a0048070 con 0x55e2a46db990 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f1493e00640 1 -- 172.21.15.86:0/3013433449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14a0003da0 con 0x55e2a46db990 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f14bc364740 10 client.15831 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.251+0000 7f14bc364740 1 -- 172.21.15.86:0/3013433449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2a45cc7f0 con 0x55e2a46db990 2024-04-17T07:15:43.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.252+0000 7f1493e00640 1 -- 172.21.15.86:0/3013433449 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14a0048070 con 0x55e2a46db990 2024-04-17T07:15:43.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.252+0000 7f1493e00640 1 -- 172.21.15.86:0/3013433449 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f14a0040040 con 0x55e2a46db990 2024-04-17T07:15:43.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.252+0000 7f14bc364740 10 client.15831 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:43.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.252+0000 7f14bc364740 10 client.15831 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:15:43.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.252+0000 7f14bc364740 20 client.15831 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:43.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.252+0000 7f14bc364740 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e2a46f10f0 0x55e2a46f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.252+0000 7f14bc364740 4 client.15831 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:15:43.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.252+0000 7f14bc364740 1 -- 172.21.15.86:0/3013433449 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a464ea40 con 0x55e2a46f10f0 2024-04-17T07:15:43.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.253+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e2a46f10f0 0x55e2a46f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.254+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e2a46f10f0 0x55e2a46f34e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3731763843cbd1bf server_cookie=32097363ccc83d88 in_seq=0 out_seq=0 2024-04-17T07:15:43.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.254+0000 7f1493e00640 10 client.15831 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:15:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f1493e00640 1 -- 172.21.15.86:0/3013433449 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2a464ea40 con 0x55e2a46f10f0 2024-04-17T07:15:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f1493e00640 10 client.15831 handle_command_reply: tid=0 2024-04-17T07:15:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 client.15831 shutdown 2024-04-17T07:15:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 -- 172.21.15.86:0/3013433449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46db990 msgr2=0x55e2a46e5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46db990 0x55e2a46e5b70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f14a002c240 tx=0x7f14a002bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 -- 172.21.15.86:0/3013433449 shutdown_connections 2024-04-17T07:15:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e2a46f10f0 0x55e2a46f34e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46eaba0 0x55e2a46ecf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46db990 0x55e2a46e5b70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 --2- 172.21.15.86:0/3013433449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46dafa0 0x55e2a46e5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 -- 172.21.15.86:0/3013433449 >> 172.21.15.86:0/3013433449 conn(0x55e2a46d6510 msgr2=0x55e2a46d5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f1493e00640 0 client.15831 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:15:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 -- 172.21.15.86:0/3013433449 shutdown_connections 2024-04-17T07:15:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.256+0000 7f14bc364740 1 -- 172.21.15.86:0/3013433449 wait complete. 2024-04-17T07:15:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.257+0000 7f14bc364740 20 client.15831 trim_cache size 0 max 16384 2024-04-17T07:15:43.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.272+0000 7f14bc364740 1 Processor -- start 2024-04-17T07:15:43.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14bc364740 1 -- start start 2024-04-17T07:15:43.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14bc364740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 0x55e2a46d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14bc364740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a46e5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14bc364740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46d1ab0 0x55e2a46e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14bc364740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41b10c0 con 0x55e2a46eaba0 2024-04-17T07:15:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14bc364740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41b2c00 con 0x55e2a46dafa0 2024-04-17T07:15:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14bc364740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41b1790 con 0x55e2a46d1ab0 2024-04-17T07:15:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a46e5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a46e5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59936/0 (socket says 172.21.15.86:59936) 2024-04-17T07:15:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14b2000640 1 -- 172.21.15.86:0/3936931703 learned_addr learned my addr 172.21.15.86:0/3936931703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.273+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3936931703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 0x55e2a46d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f14b3400640 1 --2- 172.21.15.86:0/3936931703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46d1ab0 0x55e2a46e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f14b2000640 1 -- 172.21.15.86:0/3936931703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46d1ab0 msgr2=0x55e2a46e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f14b2000640 1 --2- 172.21.15.86:0/3936931703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46d1ab0 0x55e2a46e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f14b2000640 1 -- 172.21.15.86:0/3936931703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 msgr2=0x55e2a46d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f14b2000640 1 --2- 172.21.15.86:0/3936931703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 0x55e2a46d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f14b2000640 1 -- 172.21.15.86:0/3936931703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a43533f0 con 0x55e2a46eaba0 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3936931703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 0x55e2a46d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f14b2000640 1 --2- 172.21.15.86:0/3936931703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a46e5630 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006010 tx=0x7f14a0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c38550dfbf5545 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f149ea00640 1 -- 172.21.15.86:0/3936931703 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0034850 con 0x55e2a46eaba0 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f149ea00640 1 -- 172.21.15.86:0/3936931703 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14a0048070 con 0x55e2a46eaba0 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.274+0000 7f149ea00640 1 -- 172.21.15.86:0/3936931703 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0003a60 con 0x55e2a46eaba0 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.275+0000 7f14bc364740 1 -- 172.21.15.86:0/3936931703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 msgr2=0x55e2a46e5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.275+0000 7f14bc364740 1 --2- 172.21.15.86:0/3936931703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a46e5630 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006010 tx=0x7f14a0005b50 comp rx=0 tx=0).stop 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.275+0000 7f14bc364740 1 -- 172.21.15.86:0/3936931703 shutdown_connections 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.275+0000 7f14bc364740 1 --2- 172.21.15.86:0/3936931703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46d1ab0 0x55e2a46e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.275+0000 7f14bc364740 1 --2- 172.21.15.86:0/3936931703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a46e5630 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.275+0000 7f14bc364740 1 --2- 172.21.15.86:0/3936931703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46dafa0 0x55e2a46d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.275+0000 7f14bc364740 1 -- 172.21.15.86:0/3936931703 >> 172.21.15.86:0/3936931703 conn(0x55e2a46d6510 msgr2=0x55e2a46e4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.276+0000 7f14bc364740 1 -- 172.21.15.86:0/3936931703 shutdown_connections 2024-04-17T07:15:43.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.276+0000 7f14bc364740 1 -- 172.21.15.86:0/3936931703 wait complete. 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.276+0000 7f14bc364740 1 -- start start 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.276+0000 7f14bc364740 1 Processor -- start 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14bc364740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46d1ab0 0x55e2a47022d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14bc364740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46dafa0 0x55e2a4702810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14bc364740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a4703820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14bc364740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41b2530 con 0x55e2a46eaba0 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14bc364740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41b1e60 con 0x55e2a46d1ab0 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14bc364740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41af090 con 0x55e2a46dafa0 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14bc364740 10 client.0 _ll_register_callbacks cb 0x7f14a0004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a4703820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a4703820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59952/0 (socket says 172.21.15.86:59952) 2024-04-17T07:15:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14b3400640 1 -- 172.21.15.86:0/3518308144 learned_addr learned my addr 172.21.15.86:0/3518308144 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.277+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46d1ab0 0x55e2a47022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f14b2000640 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46dafa0 0x55e2a4702810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f14b3400640 1 -- 172.21.15.86:0/3518308144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46dafa0 msgr2=0x55e2a4702810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f14b3400640 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46dafa0 0x55e2a4702810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f14b3400640 1 -- 172.21.15.86:0/3518308144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46d1ab0 msgr2=0x55e2a47022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f14b3400640 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46d1ab0 0x55e2a47022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f14b3400640 1 -- 172.21.15.86:0/3518308144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a441d7d0 con 0x55e2a46eaba0 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f14b3400640 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a4703820 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f14ac09c660 tx=0x7f14ac09d0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d88f236bd8e3d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f1493e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f1493e00640 1 -- 172.21.15.86:0/3518308144 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac01f040 con 0x55e2a46eaba0 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f14bc364740 1 -- 172.21.15.86:0/3518308144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a0048070 con 0x55e2a46eaba0 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.278+0000 7f1493e00640 1 -- 172.21.15.86:0/3518308144 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f14ac0870c0 con 0x55e2a46eaba0 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f1493e00640 1 -- 172.21.15.86:0/3518308144 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14ac0a55d0 con 0x55e2a46eaba0 2024-04-17T07:15:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f14bc364740 10 client.15837 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f14bc364740 1 -- 172.21.15.86:0/3518308144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2a47572a0 con 0x55e2a46eaba0 2024-04-17T07:15:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f1493e00640 1 -- 172.21.15.86:0/3518308144 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14a0048070 con 0x55e2a46eaba0 2024-04-17T07:15:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f1493e00640 1 -- 172.21.15.86:0/3518308144 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f14ac00f040 con 0x55e2a46eaba0 2024-04-17T07:15:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f14bc364740 10 client.15837 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f14bc364740 10 client.15837 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:15:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f14bc364740 20 client.15837 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f14bc364740 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e2a4757ca0 0x55e2a475a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f14bc364740 4 client.15837 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:15:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.279+0000 7f14bc364740 1 -- 172.21.15.86:0/3518308144 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2a46488b0 con 0x55e2a4757ca0 2024-04-17T07:15:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.280+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e2a4757ca0 0x55e2a475a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.281+0000 7f14b2a00640 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e2a4757ca0 0x55e2a475a0c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d8c01a69f50f4f76 server_cookie=68c903d52713100 in_seq=0 out_seq=0 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.281+0000 7f1493e00640 10 client.15837 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.282+0000 7f1493e00640 1 -- 172.21.15.86:0/3518308144 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2a46488b0 con 0x55e2a4757ca0 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.282+0000 7f1493e00640 10 client.15837 handle_command_reply: tid=0 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.282+0000 7f14bc364740 1 client.15837 shutdown 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 1 -- 172.21.15.86:0/3518308144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 msgr2=0x55e2a4703820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a4703820 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f14ac09c660 tx=0x7f14ac09d0e0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 1 -- 172.21.15.86:0/3518308144 shutdown_connections 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e2a4757ca0 0x55e2a475a0c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2a46eaba0 0x55e2a4703820 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2a46dafa0 0x55e2a4702810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 1 --2- 172.21.15.86:0/3518308144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2a46d1ab0 0x55e2a47022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f1493e00640 0 client.15837 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:15:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 1 -- 172.21.15.86:0/3518308144 >> 172.21.15.86:0/3518308144 conn(0x55e2a46d6510 msgr2=0x55e2a46d4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 1 -- 172.21.15.86:0/3518308144 shutdown_connections 2024-04-17T07:15:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 1 -- 172.21.15.86:0/3518308144 wait complete. 2024-04-17T07:15:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.283+0000 7f14bc364740 20 client.15837 trim_cache size 0 max 16384 2024-04-17T07:15:43.285 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:15:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 -- 172.21.15.86:0/733886169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14980a3c40 msgr2=0x7f14980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14980a3c40 0x7f14980a6100 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55e2a46dbd90 tx=0x7f14a4008040 comp rx=0 tx=0).stop 2024-04-17T07:15:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 -- 172.21.15.86:0/733886169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b410a170 msgr2=0x7f14b41a5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b410a170 0x7f14b41a5e80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00eaa0 tx=0x7f14ac00ef60 comp rx=0 tx=0).stop 2024-04-17T07:15:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 -- 172.21.15.86:0/733886169 shutdown_connections 2024-04-17T07:15:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14980a3c40 0x7f14980a6100 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b410a170 0x7f14b41a5e80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b41097b0 0x7f14b419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 --2- 172.21.15.86:0/733886169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b41085c0 0x7f14b419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 -- 172.21.15.86:0/733886169 >> 172.21.15.86:0/733886169 conn(0x7f14b4077560 msgr2=0x7f14b410cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 -- 172.21.15.86:0/733886169 shutdown_connections 2024-04-17T07:15:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.284+0000 7f14b9600640 1 -- 172.21.15.86:0/733886169 wait complete. 2024-04-17T07:15:43.301 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:43 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3388578970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:15:43.305 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:15:43.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:43 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3388578970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:15:43.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:43 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3388578970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:15:43.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.534+0000 7f57c3600640 1 -- 172.21.15.86:0/3152143834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc104050 msgr2=0x7f57bc106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.534+0000 7f57c3600640 1 --2- 172.21.15.86:0/3152143834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc104050 0x7f57bc106440 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009900 tx=0x7f57a402bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:43.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.535+0000 7f57c3600640 1 -- 172.21.15.86:0/3152143834 shutdown_connections 2024-04-17T07:15:43.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.535+0000 7f57c3600640 1 --2- 172.21.15.86:0/3152143834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57bc106a60 0x7f57bc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.535+0000 7f57c3600640 1 --2- 172.21.15.86:0/3152143834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc104050 0x7f57bc106440 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.535+0000 7f57c3600640 1 --2- 172.21.15.86:0/3152143834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc101720 0x7f57bc103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.535+0000 7f57c3600640 1 -- 172.21.15.86:0/3152143834 >> 172.21.15.86:0/3152143834 conn(0x7f57bc0fb760 msgr2=0x7f57bc0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.535+0000 7f57c3600640 1 -- 172.21.15.86:0/3152143834 shutdown_connections 2024-04-17T07:15:43.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.535+0000 7f57c3600640 1 -- 172.21.15.86:0/3152143834 wait complete. 2024-04-17T07:15:43.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.536+0000 7f57c3600640 1 Processor -- start 2024-04-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.536+0000 7f57c3600640 1 -- start start 2024-04-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.536+0000 7f57c3600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc101720 0x7f57bc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.536+0000 7f57c3600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57bc104050 0x7f57bc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.536+0000 7f57c3600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc106a60 0x7f57bc1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.536+0000 7f57c3600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc10b910 con 0x7f57bc101720 2024-04-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.536+0000 7f57c3600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc10b790 con 0x7f57bc104050 2024-04-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.536+0000 7f57c3600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc10ba90 con 0x7f57bc106a60 2024-04-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.537+0000 7f57c0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc101720 0x7f57bc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.537+0000 7f57c0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc101720 0x7f57bc19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59976/0 (socket says 172.21.15.86:59976) 2024-04-17T07:15:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.537+0000 7f57c0c00640 1 -- 172.21.15.86:0/1537990900 learned_addr learned my addr 172.21.15.86:0/1537990900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.537+0000 7f57bbe00640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57bc104050 0x7f57bc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.537+0000 7f57c0c00640 1 -- 172.21.15.86:0/1537990900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc106a60 msgr2=0x7f57bc1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.537+0000 7f57c0c00640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc106a60 0x7f57bc1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.537+0000 7f57c0c00640 1 -- 172.21.15.86:0/1537990900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57bc104050 msgr2=0x7f57bc19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.537+0000 7f57c0c00640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57bc104050 0x7f57bc19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.537+0000 7f57c0c00640 1 -- 172.21.15.86:0/1537990900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57bc1a2090 con 0x7f57bc101720 2024-04-17T07:15:43.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.538+0000 7f57c0c00640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc101720 0x7f57bc19a4c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f57b400ea50 tx=0x7f57b400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.538+0000 7f57b9600640 1 -- 172.21.15.86:0/1537990900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b400cdc0 con 0x7f57bc101720 2024-04-17T07:15:43.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.538+0000 7f57c3600640 1 -- 172.21.15.86:0/1537990900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57bc1a2380 con 0x7f57bc101720 2024-04-17T07:15:43.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.538+0000 7f57b9600640 1 -- 172.21.15.86:0/1537990900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b4005020 con 0x7f57bc101720 2024-04-17T07:15:43.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.538+0000 7f57b9600640 1 -- 172.21.15.86:0/1537990900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b40116b0 con 0x7f57bc101720 2024-04-17T07:15:43.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.538+0000 7f57c3600640 1 -- 172.21.15.86:0/1537990900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57bc1a2880 con 0x7f57bc101720 2024-04-17T07:15:43.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.542+0000 7f57b9600640 1 -- 172.21.15.86:0/1537990900 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f57b40051d0 con 0x7f57bc101720 2024-04-17T07:15:43.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.542+0000 7f57c61f5740 1 Processor -- start 2024-04-17T07:15:43.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.543+0000 7f57c61f5740 1 -- start start 2024-04-17T07:15:43.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.543+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x5625690f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.543+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625690fd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.543+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690fdb00 0x5625690fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.543+0000 7f57c61f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56256905e020 con 0x5625690f7fa0 2024-04-17T07:15:43.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.543+0000 7f57c61f5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562569004460 con 0x5625690f8990 2024-04-17T07:15:43.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.543+0000 7f57c61f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562569059fe0 con 0x5625690fdb00 2024-04-17T07:15:43.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.543+0000 7f57c1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x5625690f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.543+0000 7f57c1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x5625690f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59982/0 (socket says 172.21.15.86:59982) 2024-04-17T07:15:43.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.543+0000 7f57c1600640 1 -- 172.21.15.86:0/4192349056 learned_addr learned my addr 172.21.15.86:0/4192349056 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.544+0000 7f57c0c00640 1 --2- 172.21.15.86:0/4192349056 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625690fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.544+0000 7f57bbe00640 1 --2- 172.21.15.86:0/4192349056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690fdb00 0x5625690fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.544+0000 7f57c1600640 1 -- 172.21.15.86:0/4192349056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690fdb00 msgr2=0x5625690fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.544+0000 7f57c1600640 1 --2- 172.21.15.86:0/4192349056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690fdb00 0x5625690fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.544+0000 7f57c1600640 1 -- 172.21.15.86:0/4192349056 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 msgr2=0x5625690fd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.544+0000 7f57c1600640 1 --2- 172.21.15.86:0/4192349056 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625690fd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.544+0000 7f57c1600640 1 -- 172.21.15.86:0/4192349056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56256900c040 con 0x5625690f7fa0 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.544+0000 7f57bbe00640 1 --2- 172.21.15.86:0/4192349056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690fdb00 0x5625690fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.544+0000 7f57b9600640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a00a3c20 0x7f57a00a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.544+0000 7f57c1600640 1 --2- 172.21.15.86:0/4192349056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x5625690f83c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00ba90 tx=0x7f57ac00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8e9eeb128ce0e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.545+0000 7f57c0c00640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a00a3c20 0x7f57a00a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.545+0000 7f57b9600640 1 -- 172.21.15.86:0/1537990900 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f57b40d0cc0 con 0x7f57bc101720 2024-04-17T07:15:43.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.545+0000 7f57b1c00640 1 -- 172.21.15.86:0/4192349056 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac016070 con 0x5625690f7fa0 2024-04-17T07:15:43.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.545+0000 7f57b1c00640 1 -- 172.21.15.86:0/4192349056 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57ac0054d0 con 0x5625690f7fa0 2024-04-17T07:15:43.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.545+0000 7f57b1c00640 1 -- 172.21.15.86:0/4192349056 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac010460 con 0x5625690f7fa0 2024-04-17T07:15:43.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.546+0000 7f57c61f5740 1 -- 172.21.15.86:0/4192349056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 msgr2=0x5625690f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.546+0000 7f57c0c00640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a00a3c20 0x7f57a00a60e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5625690f8d90 tx=0x7f57b40d4850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.546+0000 7f57c61f5740 1 --2- 172.21.15.86:0/4192349056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x5625690f83c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00ba90 tx=0x7f57ac00bf50 comp rx=0 tx=0).stop 2024-04-17T07:15:43.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.547+0000 7f57c61f5740 1 -- 172.21.15.86:0/4192349056 shutdown_connections 2024-04-17T07:15:43.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.547+0000 7f57c61f5740 1 --2- 172.21.15.86:0/4192349056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690fdb00 0x5625690fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.547+0000 7f57c61f5740 1 --2- 172.21.15.86:0/4192349056 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625690fd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.547+0000 7f57c61f5740 1 --2- 172.21.15.86:0/4192349056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x5625690f83c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.547+0000 7f57c61f5740 1 -- 172.21.15.86:0/4192349056 >> 172.21.15.86:0/4192349056 conn(0x5625690f3510 msgr2=0x5625690f3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.547+0000 7f57c61f5740 1 -- 172.21.15.86:0/4192349056 shutdown_connections 2024-04-17T07:15:43.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.548+0000 7f57c61f5740 1 -- 172.21.15.86:0/4192349056 wait complete. 2024-04-17T07:15:43.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.548+0000 7f57c61f5740 1 -- start start 2024-04-17T07:15:43.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.548+0000 7f57c61f5740 1 Processor -- start 2024-04-17T07:15:43.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.549+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690f7fa0 0x562569113520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.549+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f8990 0x562569113a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.549+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690fdb00 0x56256911aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.549+0000 7f57c61f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562569111ac0 con 0x5625690f8990 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.549+0000 7f57c61f5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562569017830 con 0x5625690fdb00 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.549+0000 7f57c61f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562569111c40 con 0x5625690f7fa0 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.549+0000 7f57c61f5740 10 client.0 _ll_register_callbacks cb 0x562568fe5fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.549+0000 7f57bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f8990 0x562569113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.550+0000 7f57bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f8990 0x562569113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59994/0 (socket says 172.21.15.86:59994) 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.550+0000 7f57bbe00640 1 -- 172.21.15.86:0/2031750248 learned_addr learned my addr 172.21.15.86:0/2031750248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.550+0000 7f57c1600640 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690f7fa0 0x562569113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.550+0000 7f57c1600640 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690fdb00 0x56256911aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.550+0000 7f57bbe00640 1 -- 172.21.15.86:0/2031750248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690f7fa0 msgr2=0x562569113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.550+0000 7f57bbe00640 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690f7fa0 0x562569113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.550+0000 7f57bbe00640 1 -- 172.21.15.86:0/2031750248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690fdb00 msgr2=0x56256911aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.550+0000 7f57bbe00640 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690fdb00 0x56256911aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.550+0000 7f57bbe00640 1 -- 172.21.15.86:0/2031750248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256901b920 con 0x5625690f8990 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.551+0000 7f57bbe00640 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f8990 0x562569113a60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009900 tx=0x7f57a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a28a10cfa459bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.551+0000 7f57aa000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.551+0000 7f57aa000640 1 -- 172.21.15.86:0/2031750248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4005d50 con 0x5625690f8990 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.551+0000 7f57aa000640 1 -- 172.21.15.86:0/2031750248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f57a4005230 con 0x5625690f8990 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.551+0000 7f57c61f5740 1 -- 172.21.15.86:0/2031750248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57ac0054d0 con 0x5625690f8990 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.551+0000 7f57aa000640 1 -- 172.21.15.86:0/2031750248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57a404d050 con 0x5625690f8990 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.551+0000 7f57c61f5740 10 client.15849 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.551+0000 7f57c61f5740 1 -- 172.21.15.86:0/2031750248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562568fe97f0 con 0x5625690f8990 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.552+0000 7f57aa000640 1 -- 172.21.15.86:0/2031750248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57ac0054d0 con 0x5625690f8990 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.552+0000 7f57aa000640 1 -- 172.21.15.86:0/2031750248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f57a4003680 con 0x5625690f8990 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.552+0000 7f57c61f5740 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:43.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.552+0000 7f57c61f5740 10 client.15849 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:15:43.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.552+0000 7f57c61f5740 20 client.15849 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:43.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.552+0000 7f57c61f5740 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56256911f020 0x562569121440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.552+0000 7f57c61f5740 4 client.15849 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:15:43.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.552+0000 7f57c61f5740 1 -- 172.21.15.86:0/2031750248 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56256906ba40 con 0x56256911f020 2024-04-17T07:15:43.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.553+0000 7f57c1600640 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56256911f020 0x562569121440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.554+0000 7f57c1600640 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56256911f020 0x562569121440 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd2a8454b00846c server_cookie=3867ea017ac92617 in_seq=0 out_seq=0 2024-04-17T07:15:43.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.554+0000 7f57aa000640 10 client.15849 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:15:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.555+0000 7f57aa000640 1 -- 172.21.15.86:0/2031750248 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56256906ba40 con 0x56256911f020 2024-04-17T07:15:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.555+0000 7f57aa000640 10 client.15849 handle_command_reply: tid=0 2024-04-17T07:15:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 client.15849 shutdown 2024-04-17T07:15:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 -- 172.21.15.86:0/2031750248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f8990 msgr2=0x562569113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f8990 0x562569113a60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009900 tx=0x7f57a4004270 comp rx=0 tx=0).stop 2024-04-17T07:15:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 -- 172.21.15.86:0/2031750248 shutdown_connections 2024-04-17T07:15:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56256911f020 0x562569121440 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690fdb00 0x56256911aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f8990 0x562569113a60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 --2- 172.21.15.86:0/2031750248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690f7fa0 0x562569113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57aa000640 0 client.15849 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:15:43.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 -- 172.21.15.86:0/2031750248 >> 172.21.15.86:0/2031750248 conn(0x5625690f3510 msgr2=0x5625690f2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 -- 172.21.15.86:0/2031750248 shutdown_connections 2024-04-17T07:15:43.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 1 -- 172.21.15.86:0/2031750248 wait complete. 2024-04-17T07:15:43.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.556+0000 7f57c61f5740 20 client.15849 trim_cache size 0 max 16384 2024-04-17T07:15:43.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.573+0000 7f57c61f5740 1 Processor -- start 2024-04-17T07:15:43.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.573+0000 7f57c61f5740 1 -- start start 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690f7fa0 0x5625690f9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625690fa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690fdb00 0x56256911aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57c61f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1a44d0 con 0x5625690fdb00 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57c61f5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1a6010 con 0x5625690f8990 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57c61f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1a4ba0 con 0x5625690f7fa0 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57c1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690fdb00 0x56256911aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57c1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690fdb00 0x56256911aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59996/0 (socket says 172.21.15.86:59996) 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57c1600640 1 -- 172.21.15.86:0/3110029031 learned_addr learned my addr 172.21.15.86:0/3110029031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57c1600640 1 --2- 172.21.15.86:0/3110029031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690f7fa0 0x5625690f9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.574+0000 7f57bbe00640 1 --2- 172.21.15.86:0/3110029031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625690fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.575+0000 7f57c1600640 1 -- 172.21.15.86:0/3110029031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690f7fa0 msgr2=0x5625690f9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.575+0000 7f57c1600640 1 --2- 172.21.15.86:0/3110029031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690f7fa0 0x5625690f9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.575+0000 7f57c1600640 1 -- 172.21.15.86:0/3110029031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 msgr2=0x5625690fa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.575+0000 7f57c1600640 1 --2- 172.21.15.86:0/3110029031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625690fa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.575+0000 7f57c1600640 1 -- 172.21.15.86:0/3110029031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562568d703f0 con 0x5625690fdb00 2024-04-17T07:15:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.575+0000 7f57c1600640 1 --2- 172.21.15.86:0/3110029031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690fdb00 0x56256911aa60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00b820 tx=0x7f57ac005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc89e4f9300afd45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.575+0000 7f57b1200640 1 -- 172.21.15.86:0/3110029031 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac016040 con 0x5625690fdb00 2024-04-17T07:15:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.575+0000 7f57b1200640 1 -- 172.21.15.86:0/3110029031 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57ac00dd20 con 0x5625690fdb00 2024-04-17T07:15:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.575+0000 7f57b1200640 1 -- 172.21.15.86:0/3110029031 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac010bc0 con 0x5625690fdb00 2024-04-17T07:15:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.576+0000 7f57c61f5740 1 -- 172.21.15.86:0/3110029031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690fdb00 msgr2=0x56256911aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.576+0000 7f57c61f5740 1 --2- 172.21.15.86:0/3110029031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690fdb00 0x56256911aa60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00b820 tx=0x7f57ac005c90 comp rx=0 tx=0).stop 2024-04-17T07:15:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.576+0000 7f57c61f5740 1 -- 172.21.15.86:0/3110029031 shutdown_connections 2024-04-17T07:15:43.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.576+0000 7f57c61f5740 1 --2- 172.21.15.86:0/3110029031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690fdb00 0x56256911aa60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.576+0000 7f57c61f5740 1 --2- 172.21.15.86:0/3110029031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625690fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.576+0000 7f57c61f5740 1 --2- 172.21.15.86:0/3110029031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690f7fa0 0x5625690f9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.576+0000 7f57c61f5740 1 -- 172.21.15.86:0/3110029031 >> 172.21.15.86:0/3110029031 conn(0x5625690f3510 msgr2=0x562569113520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.576+0000 7f57c61f5740 1 -- 172.21.15.86:0/3110029031 shutdown_connections 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.576+0000 7f57c61f5740 1 -- 172.21.15.86:0/3110029031 wait complete. 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.577+0000 7f57c61f5740 1 -- start start 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.577+0000 7f57c61f5740 1 Processor -- start 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.577+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x562569117b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.577+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625691180d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.577+0000 7f57c61f5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690fdb00 0x562569119090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.577+0000 7f57c61f5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1a5940 con 0x5625690f7fa0 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.577+0000 7f57c61f5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc1a5270 con 0x5625690f8990 2024-04-17T07:15:43.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.577+0000 7f57c61f5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc073af0 con 0x5625690fdb00 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.577+0000 7f57c61f5740 10 client.0 _ll_register_callbacks cb 0x7f57a402ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57c1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x562569117b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57c1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x562569117b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60002/0 (socket says 172.21.15.86:60002) 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57c1600640 1 -- 172.21.15.86:0/3683846549 learned_addr learned my addr 172.21.15.86:0/3683846549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57bbe00640 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625691180d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57c1600640 1 -- 172.21.15.86:0/3683846549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690fdb00 msgr2=0x562569119090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57c1600640 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690fdb00 0x562569119090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57c1600640 1 -- 172.21.15.86:0/3683846549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 msgr2=0x5625691180d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57c1600640 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625691180d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57c1600640 1 -- 172.21.15.86:0/3683846549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562568e3a7d0 con 0x5625690f7fa0 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57bbe00640 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625691180d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57c1600640 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x562569117b90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00d770 tx=0x7f57ac00daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7850926205a30c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.578+0000 7f57aa000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:43.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.579+0000 7f57aa000640 1 -- 172.21.15.86:0/3683846549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac016070 con 0x5625690f7fa0 2024-04-17T07:15:43.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.579+0000 7f57c61f5740 1 -- 172.21.15.86:0/3683846549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57ac00dd20 con 0x5625690f7fa0 2024-04-17T07:15:43.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.579+0000 7f57aa000640 1 -- 172.21.15.86:0/3683846549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f57ac010b40 con 0x5625690f7fa0 2024-04-17T07:15:43.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.579+0000 7f57aa000640 1 -- 172.21.15.86:0/3683846549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57ac03d050 con 0x5625690f7fa0 2024-04-17T07:15:43.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.579+0000 7f57c61f5740 10 client.15855 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:43.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.579+0000 7f57c61f5740 1 -- 172.21.15.86:0/3683846549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56256911a800 con 0x5625690f7fa0 2024-04-17T07:15:43.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.579+0000 7f57aa000640 1 -- 172.21.15.86:0/3683846549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57ac00dd20 con 0x5625690f7fa0 2024-04-17T07:15:43.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.579+0000 7f57aa000640 1 -- 172.21.15.86:0/3683846549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f57ac034630 con 0x5625690f7fa0 2024-04-17T07:15:43.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.580+0000 7f57c61f5740 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:43.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.580+0000 7f57c61f5740 10 client.15855 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.580+0000 7f57c61f5740 20 client.15855 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.580+0000 7f57c61f5740 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562569185580 0x562569187970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.580+0000 7f57c61f5740 4 client.15855 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.580+0000 7f57c61f5740 1 -- 172.21.15.86:0/3683846549 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625690658b0 con 0x562569185580 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.580+0000 7f57bbe00640 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562569185580 0x562569187970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57bbe00640 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562569185580 0x562569187970 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb1b0b3d6c958bfc server_cookie=c3b76a566e72181a in_seq=0 out_seq=0 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57aa000640 10 client.15855 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57aa000640 1 -- 172.21.15.86:0/3683846549 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625690658b0 con 0x562569185580 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57aa000640 10 client.15855 handle_command_reply: tid=0 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57c61f5740 1 client.15855 shutdown 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57c61f5740 1 -- 172.21.15.86:0/3683846549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 msgr2=0x562569117b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57c61f5740 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x562569117b90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00d770 tx=0x7f57ac00daa0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57c61f5740 1 -- 172.21.15.86:0/3683846549 shutdown_connections 2024-04-17T07:15:43.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57c61f5740 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562569185580 0x562569187970 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57c61f5740 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5625690fdb00 0x562569119090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57c61f5740 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5625690f8990 0x5625691180d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57c61f5740 1 --2- 172.21.15.86:0/3683846549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5625690f7fa0 0x562569117b90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57c61f5740 1 -- 172.21.15.86:0/3683846549 >> 172.21.15.86:0/3683846549 conn(0x5625690f3510 msgr2=0x562569113450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.582+0000 7f57aa000640 0 client.15855 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:15:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c61f5740 1 -- 172.21.15.86:0/3683846549 shutdown_connections 2024-04-17T07:15:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c61f5740 1 -- 172.21.15.86:0/3683846549 wait complete. 2024-04-17T07:15:43.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c61f5740 20 client.15855 trim_cache size 0 max 16384 2024-04-17T07:15:43.585 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:15:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c3600640 1 -- 172.21.15.86:0/1537990900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a00a3c20 msgr2=0x7f57a00a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c3600640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a00a3c20 0x7f57a00a60e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5625690f8d90 tx=0x7f57b40d4850 comp rx=0 tx=0).stop 2024-04-17T07:15:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c3600640 1 -- 172.21.15.86:0/1537990900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc101720 msgr2=0x7f57bc19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c3600640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc101720 0x7f57bc19a4c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f57b400ea50 tx=0x7f57b400ef10 comp rx=0 tx=0).stop 2024-04-17T07:15:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c3600640 1 -- 172.21.15.86:0/1537990900 shutdown_connections 2024-04-17T07:15:43.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c3600640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a00a3c20 0x7f57a00a60e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c3600640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57bc106a60 0x7f57bc1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c3600640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57bc104050 0x7f57bc19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.583+0000 7f57c3600640 1 --2- 172.21.15.86:0/1537990900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57bc101720 0x7f57bc19a4c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.584+0000 7f57c3600640 1 -- 172.21.15.86:0/1537990900 >> 172.21.15.86:0/1537990900 conn(0x7f57bc0fb760 msgr2=0x7f57bc1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.584+0000 7f57c3600640 1 -- 172.21.15.86:0/1537990900 shutdown_connections 2024-04-17T07:15:43.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.584+0000 7f57c3600640 1 -- 172.21.15.86:0/1537990900 wait complete. 2024-04-17T07:15:43.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:15:43.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.818+0000 7fd414e00640 1 -- 172.21.15.86:0/3645941832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd410102ae0 msgr2=0x7fd410104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.818+0000 7fd414e00640 1 --2- 172.21.15.86:0/3645941832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd410102ae0 0x7fd410104ed0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd400009900 tx=0x7fd40002bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:43.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.818+0000 7fd414e00640 1 -- 172.21.15.86:0/3645941832 shutdown_connections 2024-04-17T07:15:43.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.818+0000 7fd414e00640 1 --2- 172.21.15.86:0/3645941832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd410107d40 0x7fd41010a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.818+0000 7fd414e00640 1 --2- 172.21.15.86:0/3645941832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd410105410 0x7fd410107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.818+0000 7fd414e00640 1 --2- 172.21.15.86:0/3645941832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd410102ae0 0x7fd410104ed0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.818+0000 7fd414e00640 1 -- 172.21.15.86:0/3645941832 >> 172.21.15.86:0/3645941832 conn(0x7fd4100fc450 msgr2=0x7fd4100fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.818+0000 7fd414e00640 1 -- 172.21.15.86:0/3645941832 shutdown_connections 2024-04-17T07:15:43.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.818+0000 7fd414e00640 1 -- 172.21.15.86:0/3645941832 wait complete. 2024-04-17T07:15:43.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.819+0000 7fd414e00640 1 Processor -- start 2024-04-17T07:15:43.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.819+0000 7fd414e00640 1 -- start start 2024-04-17T07:15:43.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.819+0000 7fd414e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd410102ae0 0x7fd41019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.819+0000 7fd414e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd410105410 0x7fd41019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.819+0000 7fd414e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd410107d40 0x7fd4101a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.819+0000 7fd414e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd41010db70 con 0x7fd410107d40 2024-04-17T07:15:43.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.819+0000 7fd414e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd41010d9f0 con 0x7fd410102ae0 2024-04-17T07:15:43.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.819+0000 7fd414e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd41010dcf0 con 0x7fd410105410 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd410107d40 0x7fd4101a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd410107d40 0x7fd4101a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60018/0 (socket says 172.21.15.86:60018) 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40e800640 1 -- 172.21.15.86:0/2922835118 learned_addr learned my addr 172.21.15.86:0/2922835118 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40d400640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd410105410 0x7fd41019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40de00640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd410102ae0 0x7fd41019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40e800640 1 -- 172.21.15.86:0/2922835118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd410105410 msgr2=0x7fd41019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40e800640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd410105410 0x7fd41019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40e800640 1 -- 172.21.15.86:0/2922835118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd410102ae0 msgr2=0x7fd41019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40e800640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd410102ae0 0x7fd41019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40e800640 1 -- 172.21.15.86:0/2922835118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4101a20c0 con 0x7fd410107d40 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.820+0000 7fd40d400640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd410105410 0x7fd41019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.821+0000 7fd40e800640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd410107d40 0x7fd4101a1a50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd40800cad0 tx=0x7fd40800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.821+0000 7fd406a00640 1 -- 172.21.15.86:0/2922835118 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40800ae20 con 0x7fd410107d40 2024-04-17T07:15:43.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.821+0000 7fd406a00640 1 -- 172.21.15.86:0/2922835118 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd408005020 con 0x7fd410107d40 2024-04-17T07:15:43.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.821+0000 7fd406a00640 1 -- 172.21.15.86:0/2922835118 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40800f6b0 con 0x7fd410107d40 2024-04-17T07:15:43.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.821+0000 7fd414e00640 1 -- 172.21.15.86:0/2922835118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4101a23b0 con 0x7fd410107d40 2024-04-17T07:15:43.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.821+0000 7fd414e00640 1 -- 172.21.15.86:0/2922835118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4101a28b0 con 0x7fd410107d40 2024-04-17T07:15:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.825+0000 7fd417bb5740 1 Processor -- start 2024-04-17T07:15:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.825+0000 7fd406a00640 1 -- 172.21.15.86:0/2922835118 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fd4080051d0 con 0x7fd410107d40 2024-04-17T07:15:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.825+0000 7fd417bb5740 1 -- start start 2024-04-17T07:15:43.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.825+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.825+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e43990 0x55d919e48490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.825+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e48b00 0x55d919e4af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.825+0000 7fd417bb5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d919da9020 con 0x55d919e42fa0 2024-04-17T07:15:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.825+0000 7fd417bb5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d919d4f460 con 0x55d919e48b00 2024-04-17T07:15:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.825+0000 7fd417bb5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d919da4fe0 con 0x55d919e43990 2024-04-17T07:15:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.826+0000 7fd40d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.826+0000 7fd40d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60030/0 (socket says 172.21.15.86:60030) 2024-04-17T07:15:43.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.826+0000 7fd40d400640 1 -- 172.21.15.86:0/2582492382 learned_addr learned my addr 172.21.15.86:0/2582492382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.826+0000 7fd40de00640 1 --2- 172.21.15.86:0/2582492382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e43990 0x55d919e48490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.826+0000 7fd40d400640 1 -- 172.21.15.86:0/2582492382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e43990 msgr2=0x55d919e48490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.826+0000 7fd40d400640 1 --2- 172.21.15.86:0/2582492382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e43990 0x55d919e48490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.826+0000 7fd40d400640 1 -- 172.21.15.86:0/2582492382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e48b00 msgr2=0x55d919e4af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.826+0000 7fd40d400640 1 --2- 172.21.15.86:0/2582492382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e48b00 0x55d919e4af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.826+0000 7fd40d400640 1 -- 172.21.15.86:0/2582492382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d919d57040 con 0x55d919e42fa0 2024-04-17T07:15:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.826+0000 7fd40d400640 1 --2- 172.21.15.86:0/2582492382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e433c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00bda0 tx=0x7fd3fc011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c5bee17bd6e9ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.827+0000 7fd406a00640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3f40a3c40 0x7fd3f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.827+0000 7fd3fb400640 1 -- 172.21.15.86:0/2582492382 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc011e60 con 0x55d919e42fa0 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.827+0000 7fd406a00640 1 -- 172.21.15.86:0/2922835118 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd4080d0a40 con 0x7fd410107d40 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.827+0000 7fd3fb400640 1 -- 172.21.15.86:0/2582492382 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3fc004230 con 0x55d919e42fa0 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.827+0000 7fd3fb400640 1 -- 172.21.15.86:0/2582492382 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0126c0 con 0x55d919e42fa0 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.827+0000 7fd40de00640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3f40a3c40 0x7fd3f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.828+0000 7fd417bb5740 1 -- 172.21.15.86:0/2582492382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 msgr2=0x55d919e433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.828+0000 7fd417bb5740 1 --2- 172.21.15.86:0/2582492382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e433c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00bda0 tx=0x7fd3fc011750 comp rx=0 tx=0).stop 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.828+0000 7fd40de00640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3f40a3c40 0x7fd3f40a6100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd400006010 tx=0x7fd400000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.828+0000 7fd417bb5740 1 -- 172.21.15.86:0/2582492382 shutdown_connections 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.828+0000 7fd417bb5740 1 --2- 172.21.15.86:0/2582492382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e48b00 0x55d919e4af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.828+0000 7fd417bb5740 1 --2- 172.21.15.86:0/2582492382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e43990 0x55d919e48490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.828+0000 7fd417bb5740 1 --2- 172.21.15.86:0/2582492382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e433c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.828+0000 7fd417bb5740 1 -- 172.21.15.86:0/2582492382 >> 172.21.15.86:0/2582492382 conn(0x55d919e3e510 msgr2=0x55d919e3e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.828+0000 7fd417bb5740 1 -- 172.21.15.86:0/2582492382 shutdown_connections 2024-04-17T07:15:43.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.828+0000 7fd417bb5740 1 -- 172.21.15.86:0/2582492382 wait complete. 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.829+0000 7fd417bb5740 1 -- start start 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.829+0000 7fd417bb5740 1 Processor -- start 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.829+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e4d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.829+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e4da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.829+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e48b00 0x55d919e54a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.829+0000 7fd417bb5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d919e4bab0 con 0x55d919e42fa0 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.829+0000 7fd417bb5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d919d62830 con 0x55d919e43990 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.829+0000 7fd417bb5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d919e4bc30 con 0x55d919e48b00 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd417bb5740 10 client.0 _ll_register_callbacks cb 0x55d919d30fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd40de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e4d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd40de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e4d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60032/0 (socket says 172.21.15.86:60032) 2024-04-17T07:15:43.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd40de00640 1 -- 172.21.15.86:0/1838499808 learned_addr learned my addr 172.21.15.86:0/1838499808 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd40d400640 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e4da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd40e800640 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e48b00 0x55d919e54a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd40de00640 1 -- 172.21.15.86:0/1838499808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e48b00 msgr2=0x55d919e54a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd40de00640 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e48b00 0x55d919e54a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd40de00640 1 -- 172.21.15.86:0/1838499808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 msgr2=0x55d919e4da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd40de00640 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e4da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.830+0000 7fd40de00640 1 -- 172.21.15.86:0/1838499808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d919d66920 con 0x55d919e42fa0 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.831+0000 7fd40de00640 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e4d4e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd400009900 tx=0x7fd40002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a35046314aa966 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.831+0000 7fd3efe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.831+0000 7fd3efe00640 1 -- 172.21.15.86:0/1838499808 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40004e030 con 0x55d919e42fa0 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.831+0000 7fd3efe00640 1 -- 172.21.15.86:0/1838499808 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd4000046c0 con 0x55d919e42fa0 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.831+0000 7fd417bb5740 1 -- 172.21.15.86:0/1838499808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3fc004230 con 0x55d919e42fa0 2024-04-17T07:15:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd3efe00640 1 -- 172.21.15.86:0/1838499808 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd40005b050 con 0x55d919e42fa0 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd417bb5740 10 client.15867 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd417bb5740 1 -- 172.21.15.86:0/1838499808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d919d347f0 con 0x55d919e42fa0 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd3efe00640 1 -- 172.21.15.86:0/1838499808 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3fc004230 con 0x55d919e42fa0 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd3efe00640 1 -- 172.21.15.86:0/1838499808 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fd400003c00 con 0x55d919e42fa0 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd417bb5740 10 client.15867 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd417bb5740 10 client.15867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd417bb5740 20 client.15867 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55d919e59000 0x55d919e5b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd417bb5740 4 client.15867 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.832+0000 7fd417bb5740 1 -- 172.21.15.86:0/1838499808 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d919db6a40 con 0x55d919e59000 2024-04-17T07:15:43.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.833+0000 7fd40d400640 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55d919e59000 0x55d919e5b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.834+0000 7fd40d400640 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55d919e59000 0x55d919e5b4c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14e032d842d3d326 server_cookie=d0bf9c15e445c6f9 in_seq=0 out_seq=0 2024-04-17T07:15:43.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.834+0000 7fd3efe00640 10 client.15867 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:15:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd3efe00640 1 -- 172.21.15.86:0/1838499808 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d919db6a40 con 0x55d919e59000 2024-04-17T07:15:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd3efe00640 10 client.15867 handle_command_reply: tid=0 2024-04-17T07:15:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd417bb5740 1 client.15867 shutdown 2024-04-17T07:15:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd417bb5740 1 -- 172.21.15.86:0/1838499808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 msgr2=0x55d919e4d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e4d4e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd400009900 tx=0x7fd40002bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd417bb5740 1 -- 172.21.15.86:0/1838499808 shutdown_connections 2024-04-17T07:15:43.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55d919e59000 0x55d919e5b4c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e48b00 0x55d919e54a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e4da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1838499808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e42fa0 0x55d919e4d4e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd417bb5740 1 -- 172.21.15.86:0/1838499808 >> 172.21.15.86:0/1838499808 conn(0x55d919e3e510 msgr2=0x55d919e3d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.837+0000 7fd3efe00640 0 client.15867 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:15:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.838+0000 7fd417bb5740 1 -- 172.21.15.86:0/1838499808 shutdown_connections 2024-04-17T07:15:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.838+0000 7fd417bb5740 1 -- 172.21.15.86:0/1838499808 wait complete. 2024-04-17T07:15:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.838+0000 7fd417bb5740 20 client.15867 trim_cache size 0 max 16384 2024-04-17T07:15:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.855+0000 7fd417bb5740 1 Processor -- start 2024-04-17T07:15:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.855+0000 7fd417bb5740 1 -- start start 2024-04-17T07:15:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.855+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e42fa0 0x55d919e399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.855+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e54a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.855+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919e4d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.855+0000 7fd417bb5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101acc50 con 0x55d919e48b00 2024-04-17T07:15:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.855+0000 7fd417bb5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101ae790 con 0x55d919e43990 2024-04-17T07:15:43.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.855+0000 7fd417bb5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101ad320 con 0x55d919e42fa0 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919e4d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919e4d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60034/0 (socket says 172.21.15.86:60034) 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40e800640 1 -- 172.21.15.86:0/479519161 learned_addr learned my addr 172.21.15.86:0/479519161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40de00640 1 --2- 172.21.15.86:0/479519161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e42fa0 0x55d919e399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40d400640 1 --2- 172.21.15.86:0/479519161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e54a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40e800640 1 -- 172.21.15.86:0/479519161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e42fa0 msgr2=0x55d919e399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40e800640 1 --2- 172.21.15.86:0/479519161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e42fa0 0x55d919e399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40e800640 1 -- 172.21.15.86:0/479519161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 msgr2=0x55d919e54a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40e800640 1 --2- 172.21.15.86:0/479519161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e54a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40e800640 1 -- 172.21.15.86:0/479519161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d919abb3f0 con 0x55d919e48b00 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.856+0000 7fd40e800640 1 --2- 172.21.15.86:0/479519161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919e4d800 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd408098ca0 tx=0x7fd40809d090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28cec47b9a8efbf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.857+0000 7fd3faa00640 1 -- 172.21.15.86:0/479519161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40801d040 con 0x55d919e48b00 2024-04-17T07:15:43.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.857+0000 7fd3faa00640 1 -- 172.21.15.86:0/479519161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd408005560 con 0x55d919e48b00 2024-04-17T07:15:43.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.857+0000 7fd3faa00640 1 -- 172.21.15.86:0/479519161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080a5280 con 0x55d919e48b00 2024-04-17T07:15:43.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.857+0000 7fd417bb5740 1 -- 172.21.15.86:0/479519161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 msgr2=0x55d919e4d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.857+0000 7fd417bb5740 1 --2- 172.21.15.86:0/479519161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919e4d800 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd408098ca0 tx=0x7fd40809d090 comp rx=0 tx=0).stop 2024-04-17T07:15:43.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.858+0000 7fd417bb5740 1 -- 172.21.15.86:0/479519161 shutdown_connections 2024-04-17T07:15:43.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.858+0000 7fd417bb5740 1 --2- 172.21.15.86:0/479519161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919e4d800 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.858+0000 7fd417bb5740 1 --2- 172.21.15.86:0/479519161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e54a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.858+0000 7fd417bb5740 1 --2- 172.21.15.86:0/479519161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e42fa0 0x55d919e399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.858+0000 7fd417bb5740 1 -- 172.21.15.86:0/479519161 >> 172.21.15.86:0/479519161 conn(0x55d919e3e510 msgr2=0x55d919e4d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.858+0000 7fd417bb5740 1 -- 172.21.15.86:0/479519161 shutdown_connections 2024-04-17T07:15:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.858+0000 7fd417bb5740 1 -- 172.21.15.86:0/479519161 wait complete. 2024-04-17T07:15:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.858+0000 7fd417bb5740 1 -- start start 2024-04-17T07:15:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.858+0000 7fd417bb5740 1 Processor -- start 2024-04-17T07:15:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.859+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e42fa0 0x55d919e51830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.859+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e51d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.859+0000 7fd417bb5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919ebd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.859+0000 7fd417bb5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101ae0c0 con 0x55d919e48b00 2024-04-17T07:15:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.859+0000 7fd417bb5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101ad9f0 con 0x55d919e43990 2024-04-17T07:15:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.859+0000 7fd417bb5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd4101aac20 con 0x55d919e42fa0 2024-04-17T07:15:43.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.859+0000 7fd417bb5740 10 client.0 _ll_register_callbacks cb 0x7fd4000043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.859+0000 7fd40e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919ebd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd40e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919ebd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60048/0 (socket says 172.21.15.86:60048) 2024-04-17T07:15:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd40e800640 1 -- 172.21.15.86:0/1786129979 learned_addr learned my addr 172.21.15.86:0/1786129979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd40d400640 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e51d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd40de00640 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e42fa0 0x55d919e51830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd40e800640 1 -- 172.21.15.86:0/1786129979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e42fa0 msgr2=0x55d919e51830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd40e800640 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e42fa0 0x55d919e51830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd40e800640 1 -- 172.21.15.86:0/1786129979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 msgr2=0x55d919e51d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:43.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd40e800640 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e51d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd40e800640 1 -- 172.21.15.86:0/1786129979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d919b857d0 con 0x55d919e48b00 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd40e800640 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919ebd5f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd40809c540 tx=0x7fd408005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1d6036febaa46c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.860+0000 7fd3efe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.861+0000 7fd3efe00640 1 -- 172.21.15.86:0/1786129979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080b9030 con 0x55d919e48b00 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.861+0000 7fd3efe00640 1 -- 172.21.15.86:0/1786129979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd40809db70 con 0x55d919e48b00 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.861+0000 7fd417bb5740 1 -- 172.21.15.86:0/1786129979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd408005560 con 0x55d919e48b00 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.861+0000 7fd3efe00640 1 -- 172.21.15.86:0/1786129979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd40801d070 con 0x55d919e48b00 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.861+0000 7fd417bb5740 10 client.15873 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.861+0000 7fd417bb5740 1 -- 172.21.15.86:0/1786129979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d919ebea60 con 0x55d919e48b00 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.861+0000 7fd3efe00640 1 -- 172.21.15.86:0/1786129979 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd408005560 con 0x55d919e48b00 2024-04-17T07:15:43.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.862+0000 7fd3efe00640 1 -- 172.21.15.86:0/1786129979 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fd408098720 con 0x55d919e48b00 2024-04-17T07:15:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.863+0000 7fd417bb5740 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.863+0000 7fd417bb5740 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:15:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.863+0000 7fd417bb5740 20 client.15873 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.863+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55d919ebf330 0x55d919ec1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.863+0000 7fd417bb5740 4 client.15873 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:15:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.863+0000 7fd417bb5740 1 -- 172.21.15.86:0/1786129979 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d919db08b0 con 0x55d919ebf330 2024-04-17T07:15:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.863+0000 7fd40de00640 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55d919ebf330 0x55d919ec1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:43.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.864+0000 7fd40de00640 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55d919ebf330 0x55d919ec1750 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b32cf0d44a3d1fe8 server_cookie=764672413cd64d00 in_seq=0 out_seq=0 2024-04-17T07:15:43.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.864+0000 7fd3efe00640 10 client.15873 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:15:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd3efe00640 1 -- 172.21.15.86:0/1786129979 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd400058020 con 0x55d919ebf330 2024-04-17T07:15:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd3efe00640 10 client.15873 handle_command_reply: tid=0 2024-04-17T07:15:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 client.15873 shutdown 2024-04-17T07:15:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 -- 172.21.15.86:0/1786129979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 msgr2=0x55d919ebd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919ebd5f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd40809c540 tx=0x7fd408005790 comp rx=0 tx=0).stop 2024-04-17T07:15:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 -- 172.21.15.86:0/1786129979 shutdown_connections 2024-04-17T07:15:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55d919ebf330 0x55d919ec1750 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d919e48b00 0x55d919ebd5f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d919e43990 0x55d919e51d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 --2- 172.21.15.86:0/1786129979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d919e42fa0 0x55d919e51830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 -- 172.21.15.86:0/1786129979 >> 172.21.15.86:0/1786129979 conn(0x55d919e3e510 msgr2=0x55d919e41770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd3efe00640 0 client.15873 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:15:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 -- 172.21.15.86:0/1786129979 shutdown_connections 2024-04-17T07:15:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 1 -- 172.21.15.86:0/1786129979 wait complete. 2024-04-17T07:15:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.865+0000 7fd417bb5740 20 client.15873 trim_cache size 0 max 16384 2024-04-17T07:15:43.867 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:15:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 -- 172.21.15.86:0/2922835118 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3f40a3c40 msgr2=0x7fd3f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3f40a3c40 0x7fd3f40a6100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd400006010 tx=0x7fd400000bc0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 -- 172.21.15.86:0/2922835118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd410107d40 msgr2=0x7fd4101a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd410107d40 0x7fd4101a1a50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd40800cad0 tx=0x7fd40800cf90 comp rx=0 tx=0).stop 2024-04-17T07:15:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 -- 172.21.15.86:0/2922835118 shutdown_connections 2024-04-17T07:15:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd3f40a3c40 0x7fd3f40a6100 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd410107d40 0x7fd4101a1a50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd410105410 0x7fd41019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 --2- 172.21.15.86:0/2922835118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd410102ae0 0x7fd41019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:43.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 -- 172.21.15.86:0/2922835118 >> 172.21.15.86:0/2922835118 conn(0x7fd4100fc450 msgr2=0x7fd4101022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 -- 172.21.15.86:0/2922835118 shutdown_connections 2024-04-17T07:15:43.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:43.866+0000 7fd414e00640 1 -- 172.21.15.86:0/2922835118 wait complete. 2024-04-17T07:15:43.883 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:15:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.092+0000 7feee4e00640 1 -- 172.21.15.86:0/1634500460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0102fe0 msgr2=0x7feee0103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.092+0000 7feee4e00640 1 --2- 172.21.15.86:0/1634500460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0102fe0 0x7feee0103460 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7feecc009900 tx=0x7feecc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.092+0000 7feee4e00640 1 -- 172.21.15.86:0/1634500460 shutdown_connections 2024-04-17T07:15:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.092+0000 7feee4e00640 1 --2- 172.21.15.86:0/1634500460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee01039a0 0x7feee010a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.092+0000 7feee4e00640 1 --2- 172.21.15.86:0/1634500460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0102fe0 0x7feee0103460 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.092+0000 7feee4e00640 1 --2- 172.21.15.86:0/1634500460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feee0101e90 0x7feee01022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.092+0000 7feee4e00640 1 -- 172.21.15.86:0/1634500460 >> 172.21.15.86:0/1634500460 conn(0x7feee00fd400 msgr2=0x7feee00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.092+0000 7feee4e00640 1 -- 172.21.15.86:0/1634500460 shutdown_connections 2024-04-17T07:15:44.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.092+0000 7feee4e00640 1 -- 172.21.15.86:0/1634500460 wait complete. 2024-04-17T07:15:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.093+0000 7feee4e00640 1 Processor -- start 2024-04-17T07:15:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feee4e00640 1 -- start start 2024-04-17T07:15:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feee4e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feee0101e90 0x7feee0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feee4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee0102fe0 0x7feee01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feee4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee01039a0 0x7feee019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feee4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feee010d9e0 con 0x7feee01039a0 2024-04-17T07:15:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feee4e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feee010d860 con 0x7feee0101e90 2024-04-17T07:15:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feee4e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feee010db60 con 0x7feee0102fe0 2024-04-17T07:15:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feedfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee01039a0 0x7feee019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feedfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee01039a0 0x7feee019f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60076/0 (socket says 172.21.15.86:60076) 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feedfe00640 1 -- 172.21.15.86:0/799186449 learned_addr learned my addr 172.21.15.86:0/799186449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.094+0000 7feedf400640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feee0101e90 0x7feee0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feedea00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee0102fe0 0x7feee01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feedfe00640 1 -- 172.21.15.86:0/799186449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee0102fe0 msgr2=0x7feee01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feedfe00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee0102fe0 0x7feee01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feedfe00640 1 -- 172.21.15.86:0/799186449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feee0101e90 msgr2=0x7feee0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feedfe00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feee0101e90 0x7feee0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feedfe00640 1 -- 172.21.15.86:0/799186449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee019fca0 con 0x7feee01039a0 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feedf400640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feee0101e90 0x7feee0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feedfe00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee01039a0 0x7feee019f630 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7feed800eb10 tx=0x7feed800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feed7e00640 1 -- 172.21.15.86:0/799186449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8004b20 con 0x7feee01039a0 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feed7e00640 1 -- 172.21.15.86:0/799186449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feed8004cc0 con 0x7feee01039a0 2024-04-17T07:15:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feee4e00640 1 -- 172.21.15.86:0/799186449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee019ff90 con 0x7feee01039a0 2024-04-17T07:15:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.095+0000 7feed7e00640 1 -- 172.21.15.86:0/799186449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8010690 con 0x7feee01039a0 2024-04-17T07:15:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.096+0000 7feee4e00640 1 -- 172.21.15.86:0/799186449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee01a04a0 con 0x7feee01039a0 2024-04-17T07:15:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.099+0000 7feed7e00640 1 -- 172.21.15.86:0/799186449 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7feed8005020 con 0x7feee01039a0 2024-04-17T07:15:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.099+0000 7feee9224740 1 Processor -- start 2024-04-17T07:15:44.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.099+0000 7feee9224740 1 -- start start 2024-04-17T07:15:44.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feee9224740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d95c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feee9224740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95cf60 0x560c2d961a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feee9224740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d964560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feee9224740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560c2d8be020 con 0x560c2d9620d0 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feee9224740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560c2d864460 con 0x560c2d95c570 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feee9224740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560c2d8b9fe0 con 0x560c2d95cf60 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feedea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d964560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feedea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d964560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60080/0 (socket says 172.21.15.86:60080) 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feedea00640 1 -- 172.21.15.86:0/2656735793 learned_addr learned my addr 172.21.15.86:0/2656735793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feedea00640 1 --2- 172.21.15.86:0/2656735793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d95c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feedf400640 1 --2- 172.21.15.86:0/2656735793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95cf60 0x560c2d961a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.100+0000 7feed7e00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feec40a3cc0 0x7feec40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feedea00640 1 -- 172.21.15.86:0/2656735793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95cf60 msgr2=0x560c2d961a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feedea00640 1 --2- 172.21.15.86:0/2656735793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95cf60 0x560c2d961a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feedea00640 1 -- 172.21.15.86:0/2656735793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 msgr2=0x560c2d95c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feedea00640 1 --2- 172.21.15.86:0/2656735793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d95c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feedea00640 1 -- 172.21.15.86:0/2656735793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c2d86c040 con 0x560c2d9620d0 2024-04-17T07:15:44.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feed7e00640 1 -- 172.21.15.86:0/799186449 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feed80d10f0 con 0x7feee01039a0 2024-04-17T07:15:44.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feedfe00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feec40a3cc0 0x7feec40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feedea00640 1 --2- 172.21.15.86:0/2656735793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d964560 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feecc005ec0 tx=0x7feecc0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c3af37b73aa731 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feed4c00640 1 -- 172.21.15.86:0/2656735793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc004040 con 0x560c2d9620d0 2024-04-17T07:15:44.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feed4c00640 1 -- 172.21.15.86:0/2656735793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feecc048070 con 0x560c2d9620d0 2024-04-17T07:15:44.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.101+0000 7feed4c00640 1 -- 172.21.15.86:0/2656735793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc0049b0 con 0x560c2d9620d0 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.102+0000 7feee9224740 1 -- 172.21.15.86:0/2656735793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 msgr2=0x560c2d964560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.102+0000 7feee9224740 1 --2- 172.21.15.86:0/2656735793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d964560 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feecc005ec0 tx=0x7feecc0058b0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.102+0000 7feedfe00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feec40a3cc0 0x7feec40a6180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feed8000f80 tx=0x7feed8005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.102+0000 7feee9224740 1 -- 172.21.15.86:0/2656735793 shutdown_connections 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.102+0000 7feee9224740 1 --2- 172.21.15.86:0/2656735793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d964560 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.102+0000 7feee9224740 1 --2- 172.21.15.86:0/2656735793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95cf60 0x560c2d961a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.102+0000 7feee9224740 1 --2- 172.21.15.86:0/2656735793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d95c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.102+0000 7feee9224740 1 -- 172.21.15.86:0/2656735793 >> 172.21.15.86:0/2656735793 conn(0x560c2d9536d0 msgr2=0x560c2d953ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.102+0000 7feee9224740 1 -- 172.21.15.86:0/2656735793 shutdown_connections 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.103+0000 7feee9224740 1 -- 172.21.15.86:0/2656735793 wait complete. 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.103+0000 7feee9224740 1 -- start start 2024-04-17T07:15:44.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.103+0000 7feee9224740 1 Processor -- start 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feee9224740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d966a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feee9224740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d95cf60 0x560c2d966fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feee9224740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d9620d0 0x560c2d96dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feee9224740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560c2d965020 con 0x560c2d95cf60 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feee9224740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560c2d877830 con 0x560c2d95c570 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feee9224740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560c2d9651a0 con 0x560c2d9620d0 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feee9224740 10 client.0 _ll_register_callbacks cb 0x560c2d845fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feedf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d95cf60 0x560c2d966fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feedf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d95cf60 0x560c2d966fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60092/0 (socket says 172.21.15.86:60092) 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feedf400640 1 -- 172.21.15.86:0/2745273506 learned_addr learned my addr 172.21.15.86:0/2745273506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feedea00640 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d966a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feedea00640 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d9620d0 0x560c2d96dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feedf400640 1 -- 172.21.15.86:0/2745273506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d9620d0 msgr2=0x560c2d96dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feedf400640 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d9620d0 0x560c2d96dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feedf400640 1 -- 172.21.15.86:0/2745273506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 msgr2=0x560c2d966a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.104+0000 7feedf400640 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d966a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.105+0000 7feedf400640 1 -- 172.21.15.86:0/2745273506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c2d87b920 con 0x560c2d95cf60 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.105+0000 7feedea00640 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d9620d0 0x560c2d96dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.105+0000 7feedf400640 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d95cf60 0x560c2d966fb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feed0013790 tx=0x7feed0013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292818a3ea33b8ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.105+0000 7feec9600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.105+0000 7feec9600640 1 -- 172.21.15.86:0/2745273506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0011d70 con 0x560c2d95cf60 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.105+0000 7feec9600640 1 -- 172.21.15.86:0/2745273506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feed0004030 con 0x560c2d95cf60 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.105+0000 7feee9224740 1 -- 172.21.15.86:0/2745273506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feecc048070 con 0x560c2d95cf60 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.105+0000 7feec9600640 1 -- 172.21.15.86:0/2745273506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feed0004230 con 0x560c2d95cf60 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.105+0000 7feee9224740 10 client.15885 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.105+0000 7feee9224740 1 -- 172.21.15.86:0/2745273506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c2d8497f0 con 0x560c2d95cf60 2024-04-17T07:15:44.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.106+0000 7feec9600640 1 -- 172.21.15.86:0/2745273506 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feecc048070 con 0x560c2d95cf60 2024-04-17T07:15:44.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.106+0000 7feec9600640 1 -- 172.21.15.86:0/2745273506 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7feed001a070 con 0x560c2d95cf60 2024-04-17T07:15:44.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.106+0000 7feee9224740 10 client.15885 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.106+0000 7feee9224740 10 client.15885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.106+0000 7feee9224740 20 client.15885 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.106+0000 7feee9224740 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560c2d972590 0x560c2d974a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.106+0000 7feee9224740 4 client.15885 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.106+0000 7feee9224740 1 -- 172.21.15.86:0/2745273506 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c2d8cba40 con 0x560c2d972590 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.107+0000 7feedea00640 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560c2d972590 0x560c2d974a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.108+0000 7feedea00640 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560c2d972590 0x560c2d974a30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8306ad0a4a7dbd2 server_cookie=26271007047ee175 in_seq=0 out_seq=0 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.108+0000 7feec9600640 10 client.15885 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feec9600640 1 -- 172.21.15.86:0/2745273506 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c2d8cba40 con 0x560c2d972590 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feec9600640 10 client.15885 handle_command_reply: tid=0 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 client.15885 shutdown 2024-04-17T07:15:44.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 -- 172.21.15.86:0/2745273506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d95cf60 msgr2=0x560c2d966fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d95cf60 0x560c2d966fb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feed0013790 tx=0x7feed0013c50 comp rx=0 tx=0).stop 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 -- 172.21.15.86:0/2745273506 shutdown_connections 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d9620d0 0x560c2d96dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d95cf60 0x560c2d966fb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560c2d972590 0x560c2d974a30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 --2- 172.21.15.86:0/2745273506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d966a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 -- 172.21.15.86:0/2745273506 >> 172.21.15.86:0/2745273506 conn(0x560c2d9536d0 msgr2=0x560c2d952930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feec9600640 0 client.15885 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 -- 172.21.15.86:0/2745273506 shutdown_connections 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 1 -- 172.21.15.86:0/2745273506 wait complete. 2024-04-17T07:15:44.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.110+0000 7feee9224740 20 client.15885 trim_cache size 0 max 16384 2024-04-17T07:15:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.125+0000 7feee9224740 1 Processor -- start 2024-04-17T07:15:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.125+0000 7feee9224740 1 -- start start 2024-04-17T07:15:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.125+0000 7feee9224740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95c570 0x560c2d9520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.125+0000 7feee9224740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95cf60 0x560c2d96dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.125+0000 7feee9224740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d9525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.125+0000 7feee9224740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feee01a8600 con 0x560c2d9620d0 2024-04-17T07:15:44.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.125+0000 7feee9224740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feee01aa140 con 0x560c2d95cf60 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.125+0000 7feee9224740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feee01a8cd0 con 0x560c2d95c570 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d9525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d9525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60108/0 (socket says 172.21.15.86:60108) 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedea00640 1 -- 172.21.15.86:0/3132245063 learned_addr learned my addr 172.21.15.86:0/3132245063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedf400640 1 --2- 172.21.15.86:0/3132245063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95cf60 0x560c2d96dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedea00640 1 --2- 172.21.15.86:0/3132245063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95c570 0x560c2d9520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedea00640 1 -- 172.21.15.86:0/3132245063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95c570 msgr2=0x560c2d9520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedea00640 1 --2- 172.21.15.86:0/3132245063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95c570 0x560c2d9520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedea00640 1 -- 172.21.15.86:0/3132245063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95cf60 msgr2=0x560c2d96dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedea00640 1 --2- 172.21.15.86:0/3132245063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95cf60 0x560c2d96dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedea00640 1 -- 172.21.15.86:0/3132245063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c2d5d03f0 con 0x560c2d9620d0 2024-04-17T07:15:44.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.126+0000 7feedea00640 1 --2- 172.21.15.86:0/3132245063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d9525e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7feecc005ec0 tx=0x7feecc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4f83645f38c73e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.127+0000 7feecbe00640 1 -- 172.21.15.86:0/3132245063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc05a070 con 0x560c2d9620d0 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.127+0000 7feecbe00640 1 -- 172.21.15.86:0/3132245063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feecc048070 con 0x560c2d9620d0 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.127+0000 7feecbe00640 1 -- 172.21.15.86:0/3132245063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc005500 con 0x560c2d9620d0 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.127+0000 7feee9224740 1 -- 172.21.15.86:0/3132245063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 msgr2=0x560c2d9525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.127+0000 7feee9224740 1 --2- 172.21.15.86:0/3132245063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d9525e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7feecc005ec0 tx=0x7feecc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.127+0000 7feee9224740 1 -- 172.21.15.86:0/3132245063 shutdown_connections 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.127+0000 7feee9224740 1 --2- 172.21.15.86:0/3132245063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d9525e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.127+0000 7feee9224740 1 --2- 172.21.15.86:0/3132245063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95cf60 0x560c2d96dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.127+0000 7feee9224740 1 --2- 172.21.15.86:0/3132245063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95c570 0x560c2d9520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.127+0000 7feee9224740 1 -- 172.21.15.86:0/3132245063 >> 172.21.15.86:0/3132245063 conn(0x560c2d9536d0 msgr2=0x560c2d95e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.128+0000 7feee9224740 1 -- 172.21.15.86:0/3132245063 shutdown_connections 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.128+0000 7feee9224740 1 -- 172.21.15.86:0/3132245063 wait complete. 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.128+0000 7feee9224740 1 -- start start 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.128+0000 7feee9224740 1 Processor -- start 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.128+0000 7feee9224740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d96bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.128+0000 7feee9224740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95cf60 0x560c2d96c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.128+0000 7feee9224740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d96d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feee9224740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feee01a9a70 con 0x560c2d9620d0 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feee9224740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feee01a93a0 con 0x560c2d95c570 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feee9224740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feee01a65d0 con 0x560c2d95cf60 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feee9224740 10 client.0 _ll_register_callbacks cb 0x7feed0009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feedea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d96d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feedea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d96d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60122/0 (socket says 172.21.15.86:60122) 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feedea00640 1 -- 172.21.15.86:0/4149436617 learned_addr learned my addr 172.21.15.86:0/4149436617 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feedea00640 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d96bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feedf400640 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95cf60 0x560c2d96c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feedea00640 1 -- 172.21.15.86:0/4149436617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95cf60 msgr2=0x560c2d96c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feedea00640 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95cf60 0x560c2d96c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feedea00640 1 -- 172.21.15.86:0/4149436617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 msgr2=0x560c2d96bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feedea00640 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d96bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.129+0000 7feedea00640 1 -- 172.21.15.86:0/4149436617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c2d69a7d0 con 0x560c2d9620d0 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.130+0000 7feedea00640 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d96d010 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7feecc058040 tx=0x7feecc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9391c2723f691b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.130+0000 7feec9600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.130+0000 7feec9600640 1 -- 172.21.15.86:0/4149436617 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc05a070 con 0x560c2d9620d0 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.130+0000 7feec9600640 1 -- 172.21.15.86:0/4149436617 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feecc005580 con 0x560c2d9620d0 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.130+0000 7feee9224740 1 -- 172.21.15.86:0/4149436617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feecc048070 con 0x560c2d9620d0 2024-04-17T07:15:44.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.130+0000 7feec9600640 1 -- 172.21.15.86:0/4149436617 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feecc06b050 con 0x560c2d9620d0 2024-04-17T07:15:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.130+0000 7feee9224740 10 client.15891 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.130+0000 7feee9224740 1 -- 172.21.15.86:0/4149436617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c2d9d8d10 con 0x560c2d9620d0 2024-04-17T07:15:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.131+0000 7feec9600640 1 -- 172.21.15.86:0/4149436617 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feecc048070 con 0x560c2d9620d0 2024-04-17T07:15:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.131+0000 7feec9600640 1 -- 172.21.15.86:0/4149436617 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7feecc004ac0 con 0x560c2d9620d0 2024-04-17T07:15:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.131+0000 7feee9224740 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.131+0000 7feee9224740 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:15:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.131+0000 7feee9224740 20 client.15891 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.131+0000 7feee9224740 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560c2d9d91f0 0x560c2d9db610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.131+0000 7feee9224740 4 client.15891 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:15:44.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.131+0000 7feee9224740 1 -- 172.21.15.86:0/4149436617 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c2d8c58b0 con 0x560c2d9d91f0 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.131+0000 7feedf400640 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560c2d9d91f0 0x560c2d9db610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.132+0000 7feedf400640 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560c2d9d91f0 0x560c2d9db610 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3097e13e442accb6 server_cookie=e86e0224fae94e3f in_seq=0 out_seq=0 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.132+0000 7feec9600640 10 client.15891 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feec9600640 1 -- 172.21.15.86:0/4149436617 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c2d8c58b0 con 0x560c2d9d91f0 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feec9600640 10 client.15891 handle_command_reply: tid=0 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 client.15891 shutdown 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 -- 172.21.15.86:0/4149436617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 msgr2=0x560c2d96d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d96d010 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7feecc058040 tx=0x7feecc000bc0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 -- 172.21.15.86:0/4149436617 shutdown_connections 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c2d9620d0 0x560c2d96d010 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560c2d95cf60 0x560c2d96c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560c2d9d91f0 0x560c2d9db610 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 --2- 172.21.15.86:0/4149436617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560c2d95c570 0x560c2d96bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feec9600640 0 client.15891 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:15:44.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 -- 172.21.15.86:0/4149436617 >> 172.21.15.86:0/4149436617 conn(0x560c2d9536d0 msgr2=0x560c2d956930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 -- 172.21.15.86:0/4149436617 shutdown_connections 2024-04-17T07:15:44.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 1 -- 172.21.15.86:0/4149436617 wait complete. 2024-04-17T07:15:44.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.133+0000 7feee9224740 20 client.15891 trim_cache size 0 max 16384 2024-04-17T07:15:44.137 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:15:44.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 -- 172.21.15.86:0/799186449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feec40a3cc0 msgr2=0x7feec40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feec40a3cc0 0x7feec40a6180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feed8000f80 tx=0x7feed8005b40 comp rx=0 tx=0).stop 2024-04-17T07:15:44.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 -- 172.21.15.86:0/799186449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee01039a0 msgr2=0x7feee019f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee01039a0 0x7feee019f630 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7feed800eb10 tx=0x7feed800c2d0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 -- 172.21.15.86:0/799186449 shutdown_connections 2024-04-17T07:15:44.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feec40a3cc0 0x7feec40a6180 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee01039a0 0x7feee019f630 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee0102fe0 0x7feee01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 --2- 172.21.15.86:0/799186449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feee0101e90 0x7feee0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 -- 172.21.15.86:0/799186449 >> 172.21.15.86:0/799186449 conn(0x7feee00fd400 msgr2=0x7feee01063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 -- 172.21.15.86:0/799186449 shutdown_connections 2024-04-17T07:15:44.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.134+0000 7feee4e00640 1 -- 172.21.15.86:0/799186449 wait complete. 2024-04-17T07:15:44.154 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:15:44.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.378+0000 7f52e2200640 1 -- 172.21.15.86:0/1687839290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52dc103080 msgr2=0x7f52dc103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.378+0000 7f52e2200640 1 --2- 172.21.15.86:0/1687839290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52dc103080 0x7f52dc103500 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009900 tx=0x7f52c802bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:44.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.378+0000 7f52e2200640 1 -- 172.21.15.86:0/1687839290 shutdown_connections 2024-04-17T07:15:44.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.378+0000 7f52e2200640 1 --2- 172.21.15.86:0/1687839290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52dc103a40 0x7f52dc10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.378+0000 7f52e2200640 1 --2- 172.21.15.86:0/1687839290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52dc103080 0x7f52dc103500 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.378+0000 7f52e2200640 1 --2- 172.21.15.86:0/1687839290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52dc101ea0 0x7f52dc1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.378+0000 7f52e2200640 1 -- 172.21.15.86:0/1687839290 >> 172.21.15.86:0/1687839290 conn(0x7f52dc0fd460 msgr2=0x7f52dc0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.378+0000 7f52e2200640 1 -- 172.21.15.86:0/1687839290 shutdown_connections 2024-04-17T07:15:44.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.378+0000 7f52e2200640 1 -- 172.21.15.86:0/1687839290 wait complete. 2024-04-17T07:15:44.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.379+0000 7f52e2200640 1 Processor -- start 2024-04-17T07:15:44.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.379+0000 7f52e2200640 1 -- start start 2024-04-17T07:15:44.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52dc101ea0 0x7f52dc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e2200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52dc103080 0x7f52dc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e2200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52dc103a40 0x7f52dc1a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc10d9c0 con 0x7f52dc101ea0 2024-04-17T07:15:44.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e2200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc10d840 con 0x7f52dc103a40 2024-04-17T07:15:44.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e2200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc10db40 con 0x7f52dc103080 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52dc101ea0 0x7f52dc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52dc101ea0 0x7f52dc19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60138/0 (socket says 172.21.15.86:60138) 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e0e00640 1 -- 172.21.15.86:0/2196178883 learned_addr learned my addr 172.21.15.86:0/2196178883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52dbe00640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52dc103080 0x7f52dc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e1800640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52dc103a40 0x7f52dc1a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e0e00640 1 -- 172.21.15.86:0/2196178883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52dc103080 msgr2=0x7f52dc19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e0e00640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52dc103080 0x7f52dc19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e0e00640 1 -- 172.21.15.86:0/2196178883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52dc103a40 msgr2=0x7f52dc1a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e0e00640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52dc103a40 0x7f52dc1a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.380+0000 7f52e0e00640 1 -- 172.21.15.86:0/2196178883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc1a1ee0 con 0x7f52dc101ea0 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.381+0000 7f52e0e00640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52dc101ea0 0x7f52dc19a310 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f52d000d770 tx=0x7f52d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.381+0000 7f52d9600640 1 -- 172.21.15.86:0/2196178883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0017070 con 0x7f52dc101ea0 2024-04-17T07:15:44.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.381+0000 7f52e2200640 1 -- 172.21.15.86:0/2196178883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc1a21d0 con 0x7f52dc101ea0 2024-04-17T07:15:44.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.381+0000 7f52d9600640 1 -- 172.21.15.86:0/2196178883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52d0009c30 con 0x7f52dc101ea0 2024-04-17T07:15:44.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.381+0000 7f52d9600640 1 -- 172.21.15.86:0/2196178883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0003d00 con 0x7f52dc101ea0 2024-04-17T07:15:44.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.381+0000 7f52e2200640 1 -- 172.21.15.86:0/2196178883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc1a2770 con 0x7f52dc101ea0 2024-04-17T07:15:44.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.384+0000 7f52d9600640 1 -- 172.21.15.86:0/2196178883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f52d0003ea0 con 0x7f52dc101ea0 2024-04-17T07:15:44.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.385+0000 7f52e65cb740 1 Processor -- start 2024-04-17T07:15:44.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.385+0000 7f52e65cb740 1 -- start start 2024-04-17T07:15:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.385+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f05160 0x55d5c0f05580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.385+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05b50 0x55d5c0f0a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.385+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f0d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.385+0000 7f52e65cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d5c0e6b020 con 0x55d5c0f05160 2024-04-17T07:15:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.385+0000 7f52e65cb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d5c0e11460 con 0x55d5c0f05b50 2024-04-17T07:15:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.385+0000 7f52e65cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5c0e66fe0 con 0x55d5c0f0acc0 2024-04-17T07:15:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.385+0000 7f52e1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f05160 0x55d5c0f05580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.386+0000 7f52e1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f05160 0x55d5c0f05580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60152/0 (socket says 172.21.15.86:60152) 2024-04-17T07:15:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.386+0000 7f52e1800640 1 -- 172.21.15.86:0/981533331 learned_addr learned my addr 172.21.15.86:0/981533331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:44.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.386+0000 7f52dbe00640 1 --2- 172.21.15.86:0/981533331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f0d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.386+0000 7f52e1800640 1 -- 172.21.15.86:0/981533331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f0acc0 msgr2=0x55d5c0f0d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.386+0000 7f52e1800640 1 --2- 172.21.15.86:0/981533331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f0d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.386+0000 7f52e1800640 1 -- 172.21.15.86:0/981533331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05b50 msgr2=0x55d5c0f0a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.386+0000 7f52e1800640 1 --2- 172.21.15.86:0/981533331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05b50 0x55d5c0f0a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.386+0000 7f52e1800640 1 -- 172.21.15.86:0/981533331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5c0e19040 con 0x55d5c0f05160 2024-04-17T07:15:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.386+0000 7f52d9600640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52c40a3cc0 0x7f52c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.387+0000 7f52e1800640 1 --2- 172.21.15.86:0/981533331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f05160 0x55d5c0f05580 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009df0 tx=0x7f52d400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df2378d1a71d7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.387+0000 7f52d9600640 1 -- 172.21.15.86:0/2196178883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f52d00d0620 con 0x7f52dc101ea0 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.387+0000 7f52e0e00640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52c40a3cc0 0x7f52c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.387+0000 7f52ce000640 1 -- 172.21.15.86:0/981533331 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4003970 con 0x55d5c0f05160 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.387+0000 7f52ce000640 1 -- 172.21.15.86:0/981533331 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52d4004030 con 0x55d5c0f05160 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.387+0000 7f52ce000640 1 -- 172.21.15.86:0/981533331 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4011820 con 0x55d5c0f05160 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.388+0000 7f52e65cb740 1 -- 172.21.15.86:0/981533331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f05160 msgr2=0x55d5c0f05580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.388+0000 7f52e65cb740 1 --2- 172.21.15.86:0/981533331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f05160 0x55d5c0f05580 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009df0 tx=0x7f52d400f730 comp rx=0 tx=0).stop 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.388+0000 7f52e0e00640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52c40a3cc0 0x7f52c40a6180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55d5c0f065c0 tx=0x7f52d0096f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.388+0000 7f52e65cb740 1 -- 172.21.15.86:0/981533331 shutdown_connections 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.388+0000 7f52e65cb740 1 --2- 172.21.15.86:0/981533331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f0d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.388+0000 7f52e65cb740 1 --2- 172.21.15.86:0/981533331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05b50 0x55d5c0f0a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.388+0000 7f52e65cb740 1 --2- 172.21.15.86:0/981533331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f05160 0x55d5c0f05580 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.388+0000 7f52e65cb740 1 -- 172.21.15.86:0/981533331 >> 172.21.15.86:0/981533331 conn(0x55d5c0f006d0 msgr2=0x55d5c0f00ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.388+0000 7f52e65cb740 1 -- 172.21.15.86:0/981533331 shutdown_connections 2024-04-17T07:15:44.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.388+0000 7f52e65cb740 1 -- 172.21.15.86:0/981533331 wait complete. 2024-04-17T07:15:44.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.389+0000 7f52e65cb740 1 -- start start 2024-04-17T07:15:44.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.389+0000 7f52e65cb740 1 Processor -- start 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.389+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f0f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.389+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f0fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e65cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d5c0f0dc70 con 0x55d5c0f0acc0 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e65cb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d5c0e24830 con 0x55d5c0f05160 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e65cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d5c0f0ddf0 con 0x55d5c0f05b50 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e65cb740 10 client.0 _ll_register_callbacks cb 0x55d5c0df2fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60164/0 (socket says 172.21.15.86:60164) 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e1800640 1 -- 172.21.15.86:0/2656318497 learned_addr learned my addr 172.21.15.86:0/2656318497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52dbe00640 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f0fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e1800640 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f0f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e1800640 1 -- 172.21.15.86:0/2656318497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 msgr2=0x55d5c0f0fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e1800640 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f0fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e1800640 1 -- 172.21.15.86:0/2656318497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 msgr2=0x55d5c0f0f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e1800640 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f0f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.390+0000 7f52e1800640 1 -- 172.21.15.86:0/2656318497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5c0e28920 con 0x55d5c0f0acc0 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.391+0000 7f52e1800640 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f52d4004940 tx=0x7f52d4004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ce9a08308f016 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.391+0000 7f52c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.391+0000 7f52c2a00640 1 -- 172.21.15.86:0/2656318497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d401e070 con 0x55d5c0f0acc0 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.391+0000 7f52c2a00640 1 -- 172.21.15.86:0/2656318497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f52d4003c80 con 0x55d5c0f0acc0 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.391+0000 7f52e65cb740 1 -- 172.21.15.86:0/2656318497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52d4004030 con 0x55d5c0f0acc0 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.391+0000 7f52c2a00640 1 -- 172.21.15.86:0/2656318497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52d4030050 con 0x55d5c0f0acc0 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.391+0000 7f52e65cb740 10 client.15903 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.391+0000 7f52e65cb740 1 -- 172.21.15.86:0/2656318497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5c0df67f0 con 0x55d5c0f0acc0 2024-04-17T07:15:44.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.392+0000 7f52c2a00640 1 -- 172.21.15.86:0/2656318497 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52d4004030 con 0x55d5c0f0acc0 2024-04-17T07:15:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.392+0000 7f52c2a00640 1 -- 172.21.15.86:0/2656318497 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f52d4011b50 con 0x55d5c0f0acc0 2024-04-17T07:15:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.392+0000 7f52e65cb740 10 client.15903 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.392+0000 7f52e65cb740 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.392+0000 7f52e65cb740 20 client.15903 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.392+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d5c0f1ae80 0x55d5c0f1d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.392+0000 7f52e65cb740 4 client.15903 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:15:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.392+0000 7f52e65cb740 1 -- 172.21.15.86:0/2656318497 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5c0e78a40 con 0x55d5c0f1ae80 2024-04-17T07:15:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.393+0000 7f52dbe00640 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d5c0f1ae80 0x55d5c0f1d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.394+0000 7f52dbe00640 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d5c0f1ae80 0x55d5c0f1d290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba580e94ecf2e6b server_cookie=12295e816b51ea37 in_seq=0 out_seq=0 2024-04-17T07:15:44.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.394+0000 7f52c2a00640 10 client.15903 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.396+0000 7f52c2a00640 1 -- 172.21.15.86:0/2656318497 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5c0e78a40 con 0x55d5c0f1ae80 2024-04-17T07:15:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.396+0000 7f52c2a00640 10 client.15903 handle_command_reply: tid=0 2024-04-17T07:15:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.396+0000 7f52e65cb740 1 client.15903 shutdown 2024-04-17T07:15:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.396+0000 7f52e65cb740 1 -- 172.21.15.86:0/2656318497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 msgr2=0x55d5c0f16bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.396+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f52d4004940 tx=0x7f52d4004870 comp rx=0 tx=0).stop 2024-04-17T07:15:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.397+0000 7f52e65cb740 1 -- 172.21.15.86:0/2656318497 shutdown_connections 2024-04-17T07:15:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.397+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d5c0f1ae80 0x55d5c0f1d290 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.397+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bf0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.397+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f0fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.397+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2656318497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f0f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.397+0000 7f52e65cb740 1 -- 172.21.15.86:0/2656318497 >> 172.21.15.86:0/2656318497 conn(0x55d5c0f006d0 msgr2=0x55d5c0eff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.397+0000 7f52c2a00640 0 client.15903 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.397+0000 7f52e65cb740 1 -- 172.21.15.86:0/2656318497 shutdown_connections 2024-04-17T07:15:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.397+0000 7f52e65cb740 1 -- 172.21.15.86:0/2656318497 wait complete. 2024-04-17T07:15:44.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.397+0000 7f52e65cb740 20 client.15903 trim_cache size 0 max 16384 2024-04-17T07:15:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.415+0000 7f52e65cb740 1 Processor -- start 2024-04-17T07:15:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.415+0000 7f52e65cb740 1 -- start start 2024-04-17T07:15:44.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.415+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f07050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.415+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f07590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.416+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.416+0000 7f52e65cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1b0f50 con 0x55d5c0f0acc0 2024-04-17T07:15:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.416+0000 7f52e65cb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1b2a90 con 0x55d5c0f05160 2024-04-17T07:15:44.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.416+0000 7f52e65cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1b1620 con 0x55d5c0f05b50 2024-04-17T07:15:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.416+0000 7f52e1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.416+0000 7f52e1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60180/0 (socket says 172.21.15.86:60180) 2024-04-17T07:15:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.416+0000 7f52e1800640 1 -- 172.21.15.86:0/734497296 learned_addr learned my addr 172.21.15.86:0/734497296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.416+0000 7f52dbe00640 1 --2- 172.21.15.86:0/734497296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f07590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.416+0000 7f52e1800640 1 --2- 172.21.15.86:0/734497296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f07050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.417+0000 7f52e1800640 1 -- 172.21.15.86:0/734497296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 msgr2=0x55d5c0f07590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.417+0000 7f52e1800640 1 --2- 172.21.15.86:0/734497296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f07590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.417+0000 7f52e1800640 1 -- 172.21.15.86:0/734497296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 msgr2=0x55d5c0f07050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.417+0000 7f52e1800640 1 --2- 172.21.15.86:0/734497296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f07050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.417+0000 7f52e1800640 1 -- 172.21.15.86:0/734497296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5c0f0aa10 con 0x55d5c0f0acc0 2024-04-17T07:15:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.417+0000 7f52dbe00640 1 --2- 172.21.15.86:0/734497296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f07590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:15:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.417+0000 7f52e1800640 1 --2- 172.21.15.86:0/734497296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f52d4011f20 tx=0x7f52d4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafb651a916b8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.417+0000 7f52cd600640 1 -- 172.21.15.86:0/734497296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4038070 con 0x55d5c0f0acc0 2024-04-17T07:15:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.417+0000 7f52cd600640 1 -- 172.21.15.86:0/734497296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52d4004690 con 0x55d5c0f0acc0 2024-04-17T07:15:44.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.417+0000 7f52cd600640 1 -- 172.21.15.86:0/734497296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d401e070 con 0x55d5c0f0acc0 2024-04-17T07:15:44.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.418+0000 7f52e65cb740 1 -- 172.21.15.86:0/734497296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 msgr2=0x55d5c0f16bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.418+0000 7f52e65cb740 1 --2- 172.21.15.86:0/734497296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f52d4011f20 tx=0x7f52d4005bd0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.418+0000 7f52e65cb740 1 -- 172.21.15.86:0/734497296 shutdown_connections 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.418+0000 7f52e65cb740 1 --2- 172.21.15.86:0/734497296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f16bb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.418+0000 7f52e65cb740 1 --2- 172.21.15.86:0/734497296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f07590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.418+0000 7f52e65cb740 1 --2- 172.21.15.86:0/734497296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f07050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.418+0000 7f52e65cb740 1 -- 172.21.15.86:0/734497296 >> 172.21.15.86:0/734497296 conn(0x55d5c0f006d0 msgr2=0x55d5c0f00ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.418+0000 7f52e65cb740 1 -- 172.21.15.86:0/734497296 shutdown_connections 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.419+0000 7f52e65cb740 1 -- 172.21.15.86:0/734497296 wait complete. 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.419+0000 7f52e65cb740 1 -- start start 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.419+0000 7f52e65cb740 1 Processor -- start 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.419+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f13e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f14340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e65cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f15350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e65cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1b23c0 con 0x55d5c0f0acc0 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e65cb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1b1cf0 con 0x55d5c0f05160 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e65cb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1aef60 con 0x55d5c0f05b50 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e65cb740 10 client.0 _ll_register_callbacks cb 0x7f52d4004340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f15350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f15350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60184/0 (socket says 172.21.15.86:60184) 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e1800640 1 -- 172.21.15.86:0/2566270428 learned_addr learned my addr 172.21.15.86:0/2566270428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52dbe00640 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f14340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e1800640 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f13e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e1800640 1 -- 172.21.15.86:0/2566270428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 msgr2=0x55d5c0f14340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e1800640 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f14340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e1800640 1 -- 172.21.15.86:0/2566270428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 msgr2=0x55d5c0f13e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e1800640 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f13e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.420+0000 7f52e1800640 1 -- 172.21.15.86:0/2566270428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5c0b7d3f0 con 0x55d5c0f0acc0 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.421+0000 7f52e1800640 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f15350 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f52d40095e0 tx=0x7f52d40097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bab7d47c0ae6cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.421+0000 7f52c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.421+0000 7f52c2a00640 1 -- 172.21.15.86:0/2566270428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4038040 con 0x55d5c0f0acc0 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.421+0000 7f52e65cb740 1 -- 172.21.15.86:0/2566270428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52d4004690 con 0x55d5c0f0acc0 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.421+0000 7f52c2a00640 1 -- 172.21.15.86:0/2566270428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f52d4005880 con 0x55d5c0f0acc0 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.421+0000 7f52c2a00640 1 -- 172.21.15.86:0/2566270428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52d4047050 con 0x55d5c0f0acc0 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.421+0000 7f52e65cb740 10 client.15909 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.421+0000 7f52e65cb740 1 -- 172.21.15.86:0/2566270428 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5c0c477d0 con 0x55d5c0f0acc0 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.421+0000 7f52c2a00640 1 -- 172.21.15.86:0/2566270428 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52d4004690 con 0x55d5c0f0acc0 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.422+0000 7f52c2a00640 1 -- 172.21.15.86:0/2566270428 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f52d40117b0 con 0x55d5c0f0acc0 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.422+0000 7f52e65cb740 10 client.15909 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.422+0000 7f52e65cb740 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.422+0000 7f52e65cb740 20 client.15909 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:44.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.422+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d5c0f81440 0x55d5c0f83860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.422+0000 7f52e65cb740 4 client.15909 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:15:44.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.422+0000 7f52e65cb740 1 -- 172.21.15.86:0/2566270428 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5c0e728b0 con 0x55d5c0f81440 2024-04-17T07:15:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.422+0000 7f52dbe00640 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d5c0f81440 0x55d5c0f83860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.423+0000 7f52dbe00640 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d5c0f81440 0x55d5c0f83860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e53d9d37cb4695d2 server_cookie=8d378ababe0de5ce in_seq=0 out_seq=0 2024-04-17T07:15:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.423+0000 7f52c2a00640 10 client.15909 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52c2a00640 1 -- 172.21.15.86:0/2566270428 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5c0e728b0 con 0x55d5c0f81440 2024-04-17T07:15:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52c2a00640 10 client.15909 handle_command_reply: tid=0 2024-04-17T07:15:44.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 client.15909 shutdown 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 -- 172.21.15.86:0/2566270428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 msgr2=0x55d5c0f15350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f15350 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f52d40095e0 tx=0x7f52d40097f0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 -- 172.21.15.86:0/2566270428 shutdown_connections 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d5c0f81440 0x55d5c0f83860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5c0f0acc0 0x55d5c0f15350 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d5c0f05b50 0x55d5c0f14340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 --2- 172.21.15.86:0/2566270428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d5c0f05160 0x55d5c0f13e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52c2a00640 0 client.15909 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 -- 172.21.15.86:0/2566270428 >> 172.21.15.86:0/2566270428 conn(0x55d5c0f006d0 msgr2=0x55d5c0f0f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 -- 172.21.15.86:0/2566270428 shutdown_connections 2024-04-17T07:15:44.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.424+0000 7f52e65cb740 1 -- 172.21.15.86:0/2566270428 wait complete. 2024-04-17T07:15:44.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.425+0000 7f52e65cb740 20 client.15909 trim_cache size 0 max 16384 2024-04-17T07:15:44.428 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:15:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.425+0000 7f52e2200640 1 -- 172.21.15.86:0/2196178883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52c40a3cc0 msgr2=0x7f52c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.425+0000 7f52e2200640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52c40a3cc0 0x7f52c40a6180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55d5c0f065c0 tx=0x7f52d0096f90 comp rx=0 tx=0).stop 2024-04-17T07:15:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.425+0000 7f52e2200640 1 -- 172.21.15.86:0/2196178883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52dc101ea0 msgr2=0x7f52dc19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.425+0000 7f52e2200640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52dc101ea0 0x7f52dc19a310 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f52d000d770 tx=0x7f52d000dc30 comp rx=0 tx=0).stop 2024-04-17T07:15:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.426+0000 7f52e2200640 1 -- 172.21.15.86:0/2196178883 shutdown_connections 2024-04-17T07:15:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.426+0000 7f52e2200640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f52c40a3cc0 0x7f52c40a6180 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.426+0000 7f52e2200640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52dc103a40 0x7f52dc1a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.426+0000 7f52e2200640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52dc103080 0x7f52dc19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.426+0000 7f52e2200640 1 --2- 172.21.15.86:0/2196178883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52dc101ea0 0x7f52dc19a310 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.426+0000 7f52e2200640 1 -- 172.21.15.86:0/2196178883 >> 172.21.15.86:0/2196178883 conn(0x7f52dc0fd460 msgr2=0x7f52dc074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.426+0000 7f52e2200640 1 -- 172.21.15.86:0/2196178883 shutdown_connections 2024-04-17T07:15:44.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:44.426+0000 7f52e2200640 1 -- 172.21.15.86:0/2196178883 wait complete. 2024-04-17T07:15:44.441 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:44 smithi086 ceph-mon[31733]: pgmap v481: 129 pgs: 129 active+clean; 35 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.2 MiB/s wr, 80 op/s 2024-04-17T07:15:44.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:44 smithi098 ceph-mon[35418]: pgmap v481: 129 pgs: 129 active+clean; 35 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.2 MiB/s wr, 80 op/s 2024-04-17T07:15:44.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:44 smithi103 ceph-mon[35495]: pgmap v481: 129 pgs: 129 active+clean; 35 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.2 MiB/s wr, 80 op/s 2024-04-17T07:15:45.445 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:15:45.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:15:45.445 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:15:45.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.695+0000 7efe59400640 1 -- 172.21.15.86:0/876629508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe54075f20 msgr2=0x7efe54074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:45.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.695+0000 7efe59400640 1 --2- 172.21.15.86:0/876629508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe54075f20 0x7efe54074370 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efe44009900 tx=0x7efe4402bcd0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.696+0000 7efe59400640 1 -- 172.21.15.86:0/876629508 shutdown_connections 2024-04-17T07:15:45.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.696+0000 7efe59400640 1 --2- 172.21.15.86:0/876629508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe54109e60 0x7efe5410c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.696+0000 7efe59400640 1 --2- 172.21.15.86:0/876629508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe54074940 0x7efe54074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.696+0000 7efe59400640 1 --2- 172.21.15.86:0/876629508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe54075f20 0x7efe54074370 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.696+0000 7efe59400640 1 -- 172.21.15.86:0/876629508 >> 172.21.15.86:0/876629508 conn(0x7efe540fd540 msgr2=0x7efe540ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:45.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.696+0000 7efe59400640 1 -- 172.21.15.86:0/876629508 shutdown_connections 2024-04-17T07:15:45.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.696+0000 7efe59400640 1 -- 172.21.15.86:0/876629508 wait complete. 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe59400640 1 Processor -- start 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe59400640 1 -- start start 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe59400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe54074940 0x7efe5419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe59400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe54075f20 0x7efe5419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe59400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe54109e60 0x7efe541a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe59400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe5410fcd0 con 0x7efe54075f20 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe59400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efe5410fb50 con 0x7efe54109e60 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe59400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe5410fe50 con 0x7efe54074940 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe54075f20 0x7efe5419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe54075f20 0x7efe5419a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60202/0 (socket says 172.21.15.86:60202) 2024-04-17T07:15:45.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.697+0000 7efe51e00640 1 -- 172.21.15.86:0/80671431 learned_addr learned my addr 172.21.15.86:0/80671431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.698+0000 7efe52800640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe54074940 0x7efe5419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.698+0000 7efe53200640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe54109e60 0x7efe541a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.698+0000 7efe51e00640 1 -- 172.21.15.86:0/80671431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe54074940 msgr2=0x7efe5419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.698+0000 7efe51e00640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe54074940 0x7efe5419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.698+0000 7efe51e00640 1 -- 172.21.15.86:0/80671431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe54109e60 msgr2=0x7efe541a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.698+0000 7efe51e00640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe54109e60 0x7efe541a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.698+0000 7efe51e00640 1 -- 172.21.15.86:0/80671431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe541a2010 con 0x7efe54075f20 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.698+0000 7efe51e00640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe54075f20 0x7efe5419a930 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efe4000d950 tx=0x7efe4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.699+0000 7efe4b400640 1 -- 172.21.15.86:0/80671431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40017070 con 0x7efe54075f20 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.699+0000 7efe4b400640 1 -- 172.21.15.86:0/80671431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe40009c30 con 0x7efe54075f20 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.699+0000 7efe59400640 1 -- 172.21.15.86:0/80671431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe541a2300 con 0x7efe54075f20 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.699+0000 7efe4b400640 1 -- 172.21.15.86:0/80671431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4000f620 con 0x7efe54075f20 2024-04-17T07:15:45.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.699+0000 7efe59400640 1 -- 172.21.15.86:0/80671431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541a2850 con 0x7efe54075f20 2024-04-17T07:15:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe5c05d740 1 Processor -- start 2024-04-17T07:15:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe4b400640 1 -- 172.21.15.86:0/80671431 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7efe400109a0 con 0x7efe54075f20 2024-04-17T07:15:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe5c05d740 1 -- start start 2024-04-17T07:15:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928abb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abb8b0 0x562928ac03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 0x562928ac2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe5c05d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562928a20cb0 con 0x562928abb8b0 2024-04-17T07:15:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe5c05d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5629289c7370 con 0x562928ac0a20 2024-04-17T07:15:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe5c05d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562928a1cc70 con 0x562928abaec0 2024-04-17T07:15:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abb8b0 0x562928ac03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abb8b0 0x562928ac03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60206/0 (socket says 172.21.15.86:60206) 2024-04-17T07:15:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.703+0000 7efe52800640 1 -- 172.21.15.86:0/1453433253 learned_addr learned my addr 172.21.15.86:0/1453433253 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.704+0000 7efe51e00640 1 --2- 172.21.15.86:0/1453433253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 0x562928ac2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.704+0000 7efe53200640 1 --2- 172.21.15.86:0/1453433253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928abb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.704+0000 7efe52800640 1 -- 172.21.15.86:0/1453433253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 msgr2=0x562928abb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:45.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.704+0000 7efe52800640 1 --2- 172.21.15.86:0/1453433253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928abb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.704+0000 7efe52800640 1 -- 172.21.15.86:0/1453433253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 msgr2=0x562928ac2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:45.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.704+0000 7efe52800640 1 --2- 172.21.15.86:0/1453433253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 0x562928ac2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.704+0000 7efe52800640 1 -- 172.21.15.86:0/1453433253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629289e6910 con 0x562928abb8b0 2024-04-17T07:15:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.704+0000 7efe4b400640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efe380a3c40 0x7efe380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.705+0000 7efe52800640 1 --2- 172.21.15.86:0/1453433253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abb8b0 0x562928ac03b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efe44009900 tx=0x7efe44005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9097fd9e92b9cb2d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.705+0000 7efe4b400640 1 -- 172.21.15.86:0/80671431 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efe400d0650 con 0x7efe54075f20 2024-04-17T07:15:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.705+0000 7efe53200640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efe380a3c40 0x7efe380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.705+0000 7efe3f400640 1 -- 172.21.15.86:0/1453433253 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44004480 con 0x562928abb8b0 2024-04-17T07:15:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.705+0000 7efe3f400640 1 -- 172.21.15.86:0/1453433253 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe4402ce30 con 0x562928abb8b0 2024-04-17T07:15:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.705+0000 7efe3f400640 1 -- 172.21.15.86:0/1453433253 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe440049f0 con 0x562928abb8b0 2024-04-17T07:15:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.706+0000 7efe5c05d740 1 -- 172.21.15.86:0/1453433253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abb8b0 msgr2=0x562928ac03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.706+0000 7efe5c05d740 1 --2- 172.21.15.86:0/1453433253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abb8b0 0x562928ac03b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efe44009900 tx=0x7efe44005e80 comp rx=0 tx=0).stop 2024-04-17T07:15:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.706+0000 7efe53200640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efe380a3c40 0x7efe380a6100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x562928abc110 tx=0x7efe4c007680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.706+0000 7efe5c05d740 1 -- 172.21.15.86:0/1453433253 shutdown_connections 2024-04-17T07:15:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.706+0000 7efe5c05d740 1 --2- 172.21.15.86:0/1453433253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 0x562928ac2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.706+0000 7efe5c05d740 1 --2- 172.21.15.86:0/1453433253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abb8b0 0x562928ac03b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.706+0000 7efe5c05d740 1 --2- 172.21.15.86:0/1453433253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928abb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.706+0000 7efe5c05d740 1 -- 172.21.15.86:0/1453433253 >> 172.21.15.86:0/1453433253 conn(0x562928ab6430 msgr2=0x562928ab6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.707+0000 7efe5c05d740 1 -- 172.21.15.86:0/1453433253 shutdown_connections 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.707+0000 7efe5c05d740 1 -- 172.21.15.86:0/1453433253 wait complete. 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.707+0000 7efe5c05d740 1 -- start start 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.707+0000 7efe5c05d740 1 Processor -- start 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.708+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928acdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.708+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928abb8b0 0x562928ace140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.708+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.708+0000 7efe5c05d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562928ac3970 con 0x562928ac0a20 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.708+0000 7efe5c05d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5629289da730 con 0x562928abb8b0 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.708+0000 7efe5c05d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562928ac3af0 con 0x562928abaec0 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.708+0000 7efe5c05d740 10 client.0 _ll_register_callbacks cb 0x5629289dcb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.709+0000 7efe53200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.709+0000 7efe53200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60208/0 (socket says 172.21.15.86:60208) 2024-04-17T07:15:45.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.709+0000 7efe53200640 1 -- 172.21.15.86:0/285192248 learned_addr learned my addr 172.21.15.86:0/285192248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:45.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.709+0000 7efe51e00640 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928abb8b0 0x562928ace140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.709+0000 7efe52800640 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928acdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.709+0000 7efe53200640 1 -- 172.21.15.86:0/285192248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 msgr2=0x562928acdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.709+0000 7efe53200640 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928acdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.709+0000 7efe53200640 1 -- 172.21.15.86:0/285192248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928abb8b0 msgr2=0x562928ace140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.709+0000 7efe53200640 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928abb8b0 0x562928ace140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.709+0000 7efe53200640 1 -- 172.21.15.86:0/285192248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629287fb7e0 con 0x562928ac0a20 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.710+0000 7efe53200640 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5180 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0078f0 tx=0x7efe4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f86c77cf533773 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.710+0000 7efe33e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.710+0000 7efe33e00640 1 -- 172.21.15.86:0/285192248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c01cd30 con 0x562928ac0a20 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.710+0000 7efe33e00640 1 -- 172.21.15.86:0/285192248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efe4c028480 con 0x562928ac0a20 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.710+0000 7efe5c05d740 1 -- 172.21.15.86:0/285192248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe4402ce30 con 0x562928ac0a20 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.710+0000 7efe33e00640 1 -- 172.21.15.86:0/285192248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efe4c031050 con 0x562928ac0a20 2024-04-17T07:15:45.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.710+0000 7efe5c05d740 10 client.15921 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.710+0000 7efe5c05d740 1 -- 172.21.15.86:0/285192248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56292885bf70 con 0x562928ac0a20 2024-04-17T07:15:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.711+0000 7efe33e00640 1 -- 172.21.15.86:0/285192248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe4402ce30 con 0x562928ac0a20 2024-04-17T07:15:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.711+0000 7efe33e00640 1 -- 172.21.15.86:0/285192248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7efe4c01d5a0 con 0x562928ac0a20 2024-04-17T07:15:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.711+0000 7efe5c05d740 10 client.15921 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.711+0000 7efe5c05d740 10 client.15921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.711+0000 7efe5c05d740 20 client.15921 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.711+0000 7efe5c05d740 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x562928ad9b60 0x562928adbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.711+0000 7efe5c05d740 4 client.15921 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:15:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.711+0000 7efe5c05d740 1 -- 172.21.15.86:0/285192248 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562928a2e6d0 con 0x562928ad9b60 2024-04-17T07:15:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.712+0000 7efe52800640 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x562928ad9b60 0x562928adbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.713+0000 7efe52800640 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x562928ad9b60 0x562928adbf90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b4e51191823ebf2 server_cookie=fbfca1e19de145f1 in_seq=0 out_seq=0 2024-04-17T07:15:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.713+0000 7efe33e00640 10 client.15921 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe33e00640 1 -- 172.21.15.86:0/285192248 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562928a2e6d0 con 0x562928ad9b60 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe33e00640 10 client.15921 handle_command_reply: tid=0 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe5c05d740 1 client.15921 shutdown 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe5c05d740 1 -- 172.21.15.86:0/285192248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 msgr2=0x562928ad5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe5c05d740 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5180 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0078f0 tx=0x7efe4c004a40 comp rx=0 tx=0).stop 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe5c05d740 1 -- 172.21.15.86:0/285192248 shutdown_connections 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe5c05d740 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x562928ad9b60 0x562928adbf90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe5c05d740 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5180 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe5c05d740 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928abb8b0 0x562928ace140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe5c05d740 1 --2- 172.21.15.86:0/285192248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928acdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe5c05d740 1 -- 172.21.15.86:0/285192248 >> 172.21.15.86:0/285192248 conn(0x562928ab6430 msgr2=0x562928ab55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe33e00640 0 client.15921 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.714+0000 7efe5c05d740 1 -- 172.21.15.86:0/285192248 shutdown_connections 2024-04-17T07:15:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.715+0000 7efe5c05d740 1 -- 172.21.15.86:0/285192248 wait complete. 2024-04-17T07:15:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.715+0000 7efe5c05d740 20 client.15921 trim_cache size 0 max 16384 2024-04-17T07:15:45.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.737+0000 7efe5c05d740 1 Processor -- start 2024-04-17T07:15:45.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.737+0000 7efe5c05d740 1 -- start start 2024-04-17T07:15:45.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.737+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928abce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.737+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928abb8b0 0x562928abd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.737+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.737+0000 7efe5c05d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe541a8990 con 0x562928ac0a20 2024-04-17T07:15:45.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.737+0000 7efe5c05d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efe541aa4d0 con 0x562928abb8b0 2024-04-17T07:15:45.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.737+0000 7efe5c05d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe541a9060 con 0x562928abaec0 2024-04-17T07:15:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe53200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe53200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60214/0 (socket says 172.21.15.86:60214) 2024-04-17T07:15:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe53200640 1 -- 172.21.15.86:0/1989332599 learned_addr learned my addr 172.21.15.86:0/1989332599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe52800640 1 --2- 172.21.15.86:0/1989332599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928abce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe51e00640 1 --2- 172.21.15.86:0/1989332599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928abb8b0 0x562928abd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe53200640 1 -- 172.21.15.86:0/1989332599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 msgr2=0x562928abce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe53200640 1 --2- 172.21.15.86:0/1989332599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928abce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe53200640 1 -- 172.21.15.86:0/1989332599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928abb8b0 msgr2=0x562928abd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe53200640 1 --2- 172.21.15.86:0/1989332599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928abb8b0 0x562928abd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe53200640 1 -- 172.21.15.86:0/1989332599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629286f3170 con 0x562928ac0a20 2024-04-17T07:15:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.738+0000 7efe53200640 1 --2- 172.21.15.86:0/1989332599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5140 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00e7d0 tx=0x7efe4c01c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7a18d414004cde server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.739+0000 7efe3ea00640 1 -- 172.21.15.86:0/1989332599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c01dc00 con 0x562928ac0a20 2024-04-17T07:15:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.739+0000 7efe3ea00640 1 -- 172.21.15.86:0/1989332599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe4c01dda0 con 0x562928ac0a20 2024-04-17T07:15:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.739+0000 7efe3ea00640 1 -- 172.21.15.86:0/1989332599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c005290 con 0x562928ac0a20 2024-04-17T07:15:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.739+0000 7efe5c05d740 1 -- 172.21.15.86:0/1989332599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 msgr2=0x562928ad5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.739+0000 7efe5c05d740 1 --2- 172.21.15.86:0/1989332599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5140 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00e7d0 tx=0x7efe4c01c770 comp rx=0 tx=0).stop 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.740+0000 7efe5c05d740 1 -- 172.21.15.86:0/1989332599 shutdown_connections 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.740+0000 7efe5c05d740 1 --2- 172.21.15.86:0/1989332599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928ac0a20 0x562928ad5140 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.740+0000 7efe5c05d740 1 --2- 172.21.15.86:0/1989332599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928abb8b0 0x562928abd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.740+0000 7efe5c05d740 1 --2- 172.21.15.86:0/1989332599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abaec0 0x562928abce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.740+0000 7efe5c05d740 1 -- 172.21.15.86:0/1989332599 >> 172.21.15.86:0/1989332599 conn(0x562928ab6430 msgr2=0x562928acdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.740+0000 7efe5c05d740 1 -- 172.21.15.86:0/1989332599 shutdown_connections 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.740+0000 7efe5c05d740 1 -- 172.21.15.86:0/1989332599 wait complete. 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.740+0000 7efe5c05d740 1 -- start start 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.740+0000 7efe5c05d740 1 Processor -- start 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abaec0 0x562928ad2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abb8b0 0x562928ad2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe5c05d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 0x562928ad3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe5c05d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe541a9e00 con 0x562928abaec0 2024-04-17T07:15:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe5c05d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efe541a9730 con 0x562928ac0a20 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe5c05d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe541a6980 con 0x562928abb8b0 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe5c05d740 10 client.0 _ll_register_callbacks cb 0x7efe4c007940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abaec0 0x562928ad2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abaec0 0x562928ad2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60224/0 (socket says 172.21.15.86:60224) 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe52800640 1 -- 172.21.15.86:0/281836999 learned_addr learned my addr 172.21.15.86:0/281836999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe53200640 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 0x562928ad3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe52800640 1 -- 172.21.15.86:0/281836999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abb8b0 msgr2=0x562928ad2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe51e00640 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abb8b0 0x562928ad2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe52800640 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abb8b0 0x562928ad2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe52800640 1 -- 172.21.15.86:0/281836999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 msgr2=0x562928ad3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe52800640 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 0x562928ad3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe52800640 1 -- 172.21.15.86:0/281836999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562928ad43a0 con 0x562928abaec0 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.741+0000 7efe53200640 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 0x562928ad3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:15:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.742+0000 7efe52800640 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abaec0 0x562928ad2860 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efe4402c210 tx=0x7efe44009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de367dc11e1dd2dc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.742+0000 7efe33e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.742+0000 7efe33e00640 1 -- 172.21.15.86:0/281836999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44059060 con 0x562928abaec0 2024-04-17T07:15:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.742+0000 7efe33e00640 1 -- 172.21.15.86:0/281836999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efe4402cd90 con 0x562928abaec0 2024-04-17T07:15:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.742+0000 7efe5c05d740 1 -- 172.21.15.86:0/281836999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe4c01dda0 con 0x562928abaec0 2024-04-17T07:15:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe33e00640 1 -- 172.21.15.86:0/281836999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efe44060050 con 0x562928abaec0 2024-04-17T07:15:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe5c05d740 10 client.15927 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe5c05d740 1 -- 172.21.15.86:0/281836999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562928b40130 con 0x562928abaec0 2024-04-17T07:15:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe33e00640 1 -- 172.21.15.86:0/281836999 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe4c01dda0 con 0x562928abaec0 2024-04-17T07:15:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe33e00640 1 -- 172.21.15.86:0/281836999 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7efe4404f040 con 0x562928abaec0 2024-04-17T07:15:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe5c05d740 10 client.15927 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe5c05d740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe5c05d740 20 client.15927 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe5c05d740 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x562928b40700 0x562928b42af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe5c05d740 4 client.15927 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:15:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.743+0000 7efe5c05d740 1 -- 172.21.15.86:0/281836999 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562928acd9f0 con 0x562928b40700 2024-04-17T07:15:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.744+0000 7efe51e00640 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x562928b40700 0x562928b42af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.745+0000 7efe51e00640 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x562928b40700 0x562928b42af0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf569fe142de00da server_cookie=a0d0581b99523c98 in_seq=0 out_seq=0 2024-04-17T07:15:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.745+0000 7efe33e00640 10 client.15927 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.746+0000 7efe33e00640 1 -- 172.21.15.86:0/281836999 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562928acd9f0 con 0x562928b40700 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.746+0000 7efe33e00640 10 client.15927 handle_command_reply: tid=0 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.746+0000 7efe5c05d740 1 client.15927 shutdown 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 1 -- 172.21.15.86:0/281836999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abaec0 msgr2=0x562928ad2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abaec0 0x562928ad2860 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efe4402c210 tx=0x7efe44009560 comp rx=0 tx=0).stop 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 1 -- 172.21.15.86:0/281836999 shutdown_connections 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x562928b40700 0x562928b42af0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562928ac0a20 0x562928ad3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562928abb8b0 0x562928ad2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 1 --2- 172.21.15.86:0/281836999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562928abaec0 0x562928ad2860 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 1 -- 172.21.15.86:0/281836999 >> 172.21.15.86:0/281836999 conn(0x562928ab6430 msgr2=0x562928acdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:45.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe33e00640 0 client.15927 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:45.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 1 -- 172.21.15.86:0/281836999 shutdown_connections 2024-04-17T07:15:45.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 1 -- 172.21.15.86:0/281836999 wait complete. 2024-04-17T07:15:45.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.747+0000 7efe5c05d740 20 client.15927 trim_cache size 0 max 16384 2024-04-17T07:15:45.749 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:15:45.749 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:15:45.749 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "a5c6d131-885d-40b8-a246-b0af07a3b458", 2024-04-17T07:15:45.749 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:15:45.749 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:15:45.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 -- 172.21.15.86:0/80671431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efe380a3c40 msgr2=0x7efe380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:45.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efe380a3c40 0x7efe380a6100 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x562928abc110 tx=0x7efe4c007680 comp rx=0 tx=0).stop 2024-04-17T07:15:45.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 -- 172.21.15.86:0/80671431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe54075f20 msgr2=0x7efe5419a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:45.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe54075f20 0x7efe5419a930 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efe4000d950 tx=0x7efe4000de10 comp rx=0 tx=0).stop 2024-04-17T07:15:45.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 -- 172.21.15.86:0/80671431 shutdown_connections 2024-04-17T07:15:45.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efe380a3c40 0x7efe380a6100 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe54109e60 0x7efe541a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe54075f20 0x7efe5419a930 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 --2- 172.21.15.86:0/80671431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe54074940 0x7efe5419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 -- 172.21.15.86:0/80671431 >> 172.21.15.86:0/80671431 conn(0x7efe540fd540 msgr2=0x7efe54105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 -- 172.21.15.86:0/80671431 shutdown_connections 2024-04-17T07:15:45.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.748+0000 7efe59400640 1 -- 172.21.15.86:0/80671431 wait complete. 2024-04-17T07:15:45.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:15:45.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:15:45.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.996+0000 7f9330a00640 1 -- 172.21.15.86:0/2163663377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f932c0749c0 msgr2=0x7f932c074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:45.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.996+0000 7f9330a00640 1 --2- 172.21.15.86:0/2163663377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f932c0749c0 0x7f932c074e40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9318009900 tx=0x7f931802bc30 comp rx=0 tx=0).stop 2024-04-17T07:15:45.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.996+0000 7f9330a00640 1 -- 172.21.15.86:0/2163663377 shutdown_connections 2024-04-17T07:15:45.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.996+0000 7f9330a00640 1 --2- 172.21.15.86:0/2163663377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f932c109db0 0x7f932c10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.996+0000 7f9330a00640 1 --2- 172.21.15.86:0/2163663377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f932c0749c0 0x7f932c074e40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.996+0000 7f9330a00640 1 --2- 172.21.15.86:0/2163663377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c075fa0 0x7f932c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:45.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.996+0000 7f9330a00640 1 -- 172.21.15.86:0/2163663377 >> 172.21.15.86:0/2163663377 conn(0x7f932c0fd460 msgr2=0x7f932c0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:45.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.996+0000 7f9330a00640 1 -- 172.21.15.86:0/2163663377 shutdown_connections 2024-04-17T07:15:45.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.997+0000 7f9330a00640 1 -- 172.21.15.86:0/2163663377 wait complete. 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.997+0000 7f9330a00640 1 Processor -- start 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f9330a00640 1 -- start start 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f9330a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f932c0749c0 0x7f932c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f9330a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f932c075fa0 0x7f932c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f9330a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c109db0 0x7f932c1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f9330a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f932c10fc40 con 0x7f932c075fa0 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f9330a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f932c10fac0 con 0x7f932c109db0 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f9330a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f932c10fdc0 con 0x7f932c0749c0 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f932aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f932c075fa0 0x7f932c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f932aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f932c075fa0 0x7f932c19a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60238/0 (socket says 172.21.15.86:60238) 2024-04-17T07:15:45.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f932aa00640 1 -- 172.21.15.86:0/3266761010 learned_addr learned my addr 172.21.15.86:0/3266761010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:46.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.998+0000 7f932be00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c109db0 0x7f932c1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f932b400640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f932c0749c0 0x7f932c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f932aa00640 1 -- 172.21.15.86:0/3266761010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f932c0749c0 msgr2=0x7f932c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:46.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f932aa00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f932c0749c0 0x7f932c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f932aa00640 1 -- 172.21.15.86:0/3266761010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c109db0 msgr2=0x7f932c1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:46.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f932aa00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c109db0 0x7f932c1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f932aa00640 1 -- 172.21.15.86:0/3266761010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c1a1f50 con 0x7f932c075fa0 2024-04-17T07:15:46.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f932b400640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f932c0749c0 0x7f932c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:46.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f932aa00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f932c075fa0 0x7f932c19a870 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f93180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f9323e00640 1 -- 172.21.15.86:0/3266761010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318043020 con 0x7f932c075fa0 2024-04-17T07:15:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f9323e00640 1 -- 172.21.15.86:0/3266761010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9318004aa0 con 0x7f932c075fa0 2024-04-17T07:15:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f9330a00640 1 -- 172.21.15.86:0/3266761010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c1a21e0 con 0x7f932c075fa0 2024-04-17T07:15:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:45.999+0000 7f9323e00640 1 -- 172.21.15.86:0/3266761010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318003d00 con 0x7f932c075fa0 2024-04-17T07:15:46.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.000+0000 7f9330a00640 1 -- 172.21.15.86:0/3266761010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c1a2800 con 0x7f932c075fa0 2024-04-17T07:15:46.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.003+0000 7f9323e00640 1 -- 172.21.15.86:0/3266761010 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f9318046020 con 0x7f932c075fa0 2024-04-17T07:15:46.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.004+0000 7f9334e0c740 1 Processor -- start 2024-04-17T07:15:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.005+0000 7f9334e0c740 1 -- start start 2024-04-17T07:15:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.005+0000 7f9323e00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93100a3cc0 0x7f93100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.005+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.005+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.005+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.005+0000 7f9334e0c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55618831e120 con 0x5561883c56e0 2024-04-17T07:15:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.005+0000 7f9334e0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5561882c4340 con 0x5561883bfb80 2024-04-17T07:15:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.005+0000 7f9323e00640 1 -- 172.21.15.86:0/3266761010 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93180f2b70 con 0x7f932c075fa0 2024-04-17T07:15:46.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.005+0000 7f9334e0c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55618831a0e0 con 0x5561883c0570 2024-04-17T07:15:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.005+0000 7f932be00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93100a3cc0 0x7f93100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.006+0000 7f932be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.006+0000 7f932b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.006+0000 7f932b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883bffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54022/0 (socket says 172.21.15.86:54022) 2024-04-17T07:15:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.006+0000 7f932b400640 1 -- 172.21.15.86:0/1284275935 learned_addr learned my addr 172.21.15.86:0/1284275935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.006+0000 7f932aa00640 1 --2- 172.21.15.86:0/1284275935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.006+0000 7f932b400640 1 -- 172.21.15.86:0/1284275935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 msgr2=0x5561883c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.006+0000 7f932b400640 1 --2- 172.21.15.86:0/1284275935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.006+0000 7f932b400640 1 -- 172.21.15.86:0/1284275935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 msgr2=0x5561883c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.006+0000 7f932b400640 1 --2- 172.21.15.86:0/1284275935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.007+0000 7f932b400640 1 -- 172.21.15.86:0/1284275935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561880fa7d0 con 0x5561883bfb80 2024-04-17T07:15:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.007+0000 7f932aa00640 1 --2- 172.21.15.86:0/1284275935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:46.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.007+0000 7f932be00640 1 --2- 172.21.15.86:0/1284275935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:15:46.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.007+0000 7f932b400640 1 --2- 172.21.15.86:0/1284275935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883bffa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f931c009c30 tx=0x7f931c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d8bf518c8e21b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:46.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.007+0000 7f932be00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93100a3cc0 0x7f93100a6180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9324009fd0 tx=0x7f9324009c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:46.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.008+0000 7f9317e00640 1 -- 172.21.15.86:0/1284275935 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c004030 con 0x5561883bfb80 2024-04-17T07:15:46.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.008+0000 7f9317e00640 1 -- 172.21.15.86:0/1284275935 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f931c0041d0 con 0x5561883bfb80 2024-04-17T07:15:46.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.008+0000 7f9317e00640 1 -- 172.21.15.86:0/1284275935 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0126b0 con 0x5561883bfb80 2024-04-17T07:15:46.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.010+0000 7f9334e0c740 1 -- 172.21.15.86:0/1284275935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 msgr2=0x5561883bffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:46.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.010+0000 7f9334e0c740 1 --2- 172.21.15.86:0/1284275935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883bffa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f931c009c30 tx=0x7f931c010740 comp rx=0 tx=0).stop 2024-04-17T07:15:46.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.010+0000 7f9334e0c740 1 -- 172.21.15.86:0/1284275935 shutdown_connections 2024-04-17T07:15:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.010+0000 7f9334e0c740 1 --2- 172.21.15.86:0/1284275935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.010+0000 7f9334e0c740 1 --2- 172.21.15.86:0/1284275935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.010+0000 7f9334e0c740 1 --2- 172.21.15.86:0/1284275935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883bffa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.011+0000 7f9334e0c740 1 -- 172.21.15.86:0/1284275935 >> 172.21.15.86:0/1284275935 conn(0x5561883b28e0 msgr2=0x5561883b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.011+0000 7f9334e0c740 1 -- 172.21.15.86:0/1284275935 shutdown_connections 2024-04-17T07:15:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.011+0000 7f9334e0c740 1 -- 172.21.15.86:0/1284275935 wait complete. 2024-04-17T07:15:46.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.011+0000 7f9334e0c740 1 -- start start 2024-04-17T07:15:46.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.012+0000 7f9334e0c740 1 Processor -- start 2024-04-17T07:15:46.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.012+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883bfb80 0x5561883ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.012+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883c0570 0x5561883ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.012+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.012+0000 7f9334e0c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561883c8690 con 0x5561883c56e0 2024-04-17T07:15:46.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.012+0000 7f9334e0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5561882d7710 con 0x5561883c0570 2024-04-17T07:15:46.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.012+0000 7f9334e0c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5561883c8810 con 0x5561883bfb80 2024-04-17T07:15:46.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.012+0000 7f9334e0c740 10 client.0 _ll_register_callbacks cb 0x5561882d2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:46.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.013+0000 7f932be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.013+0000 7f932b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883bfb80 0x5561883ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.013+0000 7f932be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60248/0 (socket says 172.21.15.86:60248) 2024-04-17T07:15:46.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.013+0000 7f932be00640 1 -- 172.21.15.86:0/2355680750 learned_addr learned my addr 172.21.15.86:0/2355680750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:46.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.014+0000 7f932aa00640 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883c0570 0x5561883ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.014+0000 7f932be00640 1 -- 172.21.15.86:0/2355680750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883bfb80 msgr2=0x5561883ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:46.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.014+0000 7f932be00640 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883bfb80 0x5561883ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.014+0000 7f932be00640 1 -- 172.21.15.86:0/2355680750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883c0570 msgr2=0x5561883ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:46.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.014+0000 7f932be00640 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883c0570 0x5561883ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.014+0000 7f932be00640 1 -- 172.21.15.86:0/2355680750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561880303f0 con 0x5561883c56e0 2024-04-17T07:15:46.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.014+0000 7f932b400640 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883bfb80 0x5561883ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:15:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.015+0000 7f932be00640 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883d1640 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9324009e90 tx=0x7f9324004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261a1f1d61797fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.015+0000 7f9314c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.015+0000 7f9314c00640 1 -- 172.21.15.86:0/2355680750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932402a060 con 0x5561883c56e0 2024-04-17T07:15:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.015+0000 7f9314c00640 1 -- 172.21.15.86:0/2355680750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9324022750 con 0x5561883c56e0 2024-04-17T07:15:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.015+0000 7f9334e0c740 1 -- 172.21.15.86:0/2355680750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f931c0041d0 con 0x5561883c56e0 2024-04-17T07:15:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.015+0000 7f9314c00640 1 -- 172.21.15.86:0/2355680750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9324031050 con 0x5561883c56e0 2024-04-17T07:15:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.015+0000 7f9334e0c740 10 client.15939 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.015+0000 7f9334e0c740 1 -- 172.21.15.86:0/2355680750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556187ff00f0 con 0x5561883c56e0 2024-04-17T07:15:46.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.016+0000 7f9314c00640 1 -- 172.21.15.86:0/2355680750 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f931c0041d0 con 0x5561883c56e0 2024-04-17T07:15:46.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.016+0000 7f9314c00640 1 -- 172.21.15.86:0/2355680750 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9324024070 con 0x5561883c56e0 2024-04-17T07:15:46.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.016+0000 7f9334e0c740 10 client.15939 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:46.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.016+0000 7f9334e0c740 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:46.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.016+0000 7f9334e0c740 20 client.15939 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:46.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.016+0000 7f9334e0c740 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5561883e6f10 0x5561883e9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.017+0000 7f9334e0c740 4 client.15939 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:15:46.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.017+0000 7f9334e0c740 1 -- 172.21.15.86:0/2355680750 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55618832bb40 con 0x5561883e6f10 2024-04-17T07:15:46.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.017+0000 7f932b400640 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5561883e6f10 0x5561883e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.018+0000 7f932b400640 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5561883e6f10 0x5561883e9390 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd0ab88261b3eda server_cookie=53d761cff35112b9 in_seq=0 out_seq=0 2024-04-17T07:15:46.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.018+0000 7f9314c00640 10 client.15939 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:46.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9314c00640 1 -- 172.21.15.86:0/2355680750 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55618832bb40 con 0x5561883e6f10 2024-04-17T07:15:46.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9314c00640 10 client.15939 handle_command_reply: tid=0 2024-04-17T07:15:46.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9334e0c740 1 client.15939 shutdown 2024-04-17T07:15:46.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9334e0c740 1 -- 172.21.15.86:0/2355680750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 msgr2=0x5561883d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9334e0c740 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883d1640 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9324009e90 tx=0x7f9324004b50 comp rx=0 tx=0).stop 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9334e0c740 1 -- 172.21.15.86:0/2355680750 shutdown_connections 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9334e0c740 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5561883e6f10 0x5561883e9390 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9334e0c740 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883d1640 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9314c00640 0 client.15939 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9334e0c740 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883c0570 0x5561883ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9334e0c740 1 --2- 172.21.15.86:0/2355680750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883bfb80 0x5561883ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9334e0c740 1 -- 172.21.15.86:0/2355680750 >> 172.21.15.86:0/2355680750 conn(0x5561883b28e0 msgr2=0x5561883b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.020+0000 7f9334e0c740 1 -- 172.21.15.86:0/2355680750 shutdown_connections 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.021+0000 7f9334e0c740 1 -- 172.21.15.86:0/2355680750 wait complete. 2024-04-17T07:15:46.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.021+0000 7f9334e0c740 20 client.15939 trim_cache size 0 max 16384 2024-04-17T07:15:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.043+0000 7f9334e0c740 1 Processor -- start 2024-04-17T07:15:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.043+0000 7f9334e0c740 1 -- start start 2024-04-17T07:15:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.043+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.043+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.043+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.043+0000 7f9334e0c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f932c071740 con 0x5561883c56e0 2024-04-17T07:15:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.043+0000 7f9334e0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f932c073280 con 0x5561883bfb80 2024-04-17T07:15:46.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.043+0000 7f9334e0c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f932c071e10 con 0x5561883c0570 2024-04-17T07:15:46.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60260/0 (socket says 172.21.15.86:60260) 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932be00640 1 -- 172.21.15.86:0/442217195 learned_addr learned my addr 172.21.15.86:0/442217195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932aa00640 1 --2- 172.21.15.86:0/442217195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932be00640 1 -- 172.21.15.86:0/442217195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 msgr2=0x5561883d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932be00640 1 --2- 172.21.15.86:0/442217195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932be00640 1 -- 172.21.15.86:0/442217195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 msgr2=0x5561883b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932be00640 1 --2- 172.21.15.86:0/442217195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932be00640 1 -- 172.21.15.86:0/442217195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561883c9180 con 0x5561883c56e0 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932b400640 1 --2- 172.21.15.86:0/442217195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.044+0000 7f932be00640 1 --2- 172.21.15.86:0/442217195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883b1250 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9324009900 tx=0x7f932400bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df39c97b18d1a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:46.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.045+0000 7f9317400640 1 -- 172.21.15.86:0/442217195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932402a040 con 0x5561883c56e0 2024-04-17T07:15:46.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.045+0000 7f9317400640 1 -- 172.21.15.86:0/442217195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f932403a070 con 0x5561883c56e0 2024-04-17T07:15:46.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.045+0000 7f9317400640 1 -- 172.21.15.86:0/442217195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324014040 con 0x5561883c56e0 2024-04-17T07:15:46.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.046+0000 7f9334e0c740 1 -- 172.21.15.86:0/442217195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 msgr2=0x5561883b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.046+0000 7f9334e0c740 1 --2- 172.21.15.86:0/442217195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883b1250 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9324009900 tx=0x7f932400bd00 comp rx=0 tx=0).stop 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.046+0000 7f9334e0c740 1 -- 172.21.15.86:0/442217195 shutdown_connections 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.046+0000 7f9334e0c740 1 --2- 172.21.15.86:0/442217195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883b1250 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.046+0000 7f9334e0c740 1 --2- 172.21.15.86:0/442217195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.046+0000 7f9334e0c740 1 --2- 172.21.15.86:0/442217195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.046+0000 7f9334e0c740 1 -- 172.21.15.86:0/442217195 >> 172.21.15.86:0/442217195 conn(0x5561883b28e0 msgr2=0x5561883c1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.046+0000 7f9334e0c740 1 -- 172.21.15.86:0/442217195 shutdown_connections 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.046+0000 7f9334e0c740 1 -- 172.21.15.86:0/442217195 wait complete. 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.047+0000 7f9334e0c740 1 -- start start 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.047+0000 7f9334e0c740 1 Processor -- start 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.047+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.047+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.047+0000 7f9334e0c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.047+0000 7f9334e0c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f932c072bb0 con 0x5561883c56e0 2024-04-17T07:15:46.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.047+0000 7f9334e0c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0724e0 con 0x5561883bfb80 2024-04-17T07:15:46.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.047+0000 7f9334e0c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f932c06f710 con 0x5561883c0570 2024-04-17T07:15:46.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.047+0000 7f9334e0c740 10 client.0 _ll_register_callbacks cb 0x7f9324009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:15:46.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883cff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60268/0 (socket says 172.21.15.86:60268) 2024-04-17T07:15:46.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932be00640 1 -- 172.21.15.86:0/660804210 learned_addr learned my addr 172.21.15.86:0/660804210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:15:46.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932b400640 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932aa00640 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932be00640 1 -- 172.21.15.86:0/660804210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 msgr2=0x5561883cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932be00640 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932be00640 1 -- 172.21.15.86:0/660804210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 msgr2=0x5561883cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932be00640 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932be00640 1 -- 172.21.15.86:0/660804210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561883d0580 con 0x5561883c56e0 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.048+0000 7f932aa00640 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.049+0000 7f932be00640 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883cff90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9324009010 tx=0x7f9324022780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a75af265eab976 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.049+0000 7f9314c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.049+0000 7f9314c00640 1 -- 172.21.15.86:0/660804210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932402a040 con 0x5561883c56e0 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.049+0000 7f9314c00640 1 -- 172.21.15.86:0/660804210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9324022980 con 0x5561883c56e0 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.049+0000 7f9334e0c740 1 -- 172.21.15.86:0/660804210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f932403a070 con 0x5561883c56e0 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.049+0000 7f9314c00640 1 -- 172.21.15.86:0/660804210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9324022b80 con 0x5561883c56e0 2024-04-17T07:15:46.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.049+0000 7f9334e0c740 10 client.15945 fetch_fsmap learned FSMap version 28 2024-04-17T07:15:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.049+0000 7f9334e0c740 1 -- 172.21.15.86:0/660804210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5561883d1890 con 0x5561883c56e0 2024-04-17T07:15:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.050+0000 7f9314c00640 1 -- 172.21.15.86:0/660804210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f932403a070 con 0x5561883c56e0 2024-04-17T07:15:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.050+0000 7f9314c00640 1 -- 172.21.15.86:0/660804210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9324024040 con 0x5561883c56e0 2024-04-17T07:15:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.050+0000 7f9334e0c740 10 client.15945 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:15:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.050+0000 7f9334e0c740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:15:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.050+0000 7f9334e0c740 20 client.15945 populate_metadata read hostname 'smithi086' 2024-04-17T07:15:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.050+0000 7f9334e0c740 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55618844dbb0 0x55618844ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:15:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.050+0000 7f9334e0c740 4 client.15945 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:15:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.050+0000 7f9334e0c740 1 -- 172.21.15.86:0/660804210 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561883259b0 con 0x55618844dbb0 2024-04-17T07:15:46.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.051+0000 7f932b400640 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55618844dbb0 0x55618844ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:15:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.051+0000 7f9314c00640 10 client.15945 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.051+0000 7f932b400640 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55618844dbb0 0x55618844ffd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476a04c1e515f5ee server_cookie=f3c2db7e2ca8b898 in_seq=0 out_seq=0 2024-04-17T07:15:46.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9314c00640 1 -- 172.21.15.86:0/660804210 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x5561883259b0 con 0x55618844dbb0 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9314c00640 10 client.15945 handle_command_reply: tid=0 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 client.15945 shutdown 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 -- 172.21.15.86:0/660804210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 msgr2=0x5561883cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883cff90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9324009010 tx=0x7f9324022780 comp rx=0 tx=0).stop 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 -- 172.21.15.86:0/660804210 shutdown_connections 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55618844dbb0 0x55618844ffd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561883c56e0 0x5561883cff90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5561883c0570 0x5561883cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 --2- 172.21.15.86:0/660804210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561883bfb80 0x5561883cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 -- 172.21.15.86:0/660804210 >> 172.21.15.86:0/660804210 conn(0x5561883b28e0 msgr2=0x5561883b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9314c00640 0 client.15945 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 -- 172.21.15.86:0/660804210 shutdown_connections 2024-04-17T07:15:46.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 1 -- 172.21.15.86:0/660804210 wait complete. 2024-04-17T07:15:46.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.052+0000 7f9334e0c740 20 client.15945 trim_cache size 0 max 16384 2024-04-17T07:15:46.055 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:15:46.055 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (0 inodes in the stack)", 2024-04-17T07:15:46.055 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:15:46.055 INFO:teuthology.orchestra.run.smithi086.stdout: "a5c6d131-885d-40b8-a246-b0af07a3b458": { 2024-04-17T07:15:46.055 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:15:46.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "a5c6d131-885d-40b8-a246-b0af07a3b458", 2024-04-17T07:15:46.055 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:15:46.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:15:46.055 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:15:46.056 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:15:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 -- 172.21.15.86:0/3266761010 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93100a3cc0 msgr2=0x7f93100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:46.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93100a3cc0 0x7f93100a6180 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9324009fd0 tx=0x7f9324009c30 comp rx=0 tx=0).stop 2024-04-17T07:15:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 -- 172.21.15.86:0/3266761010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f932c075fa0 msgr2=0x7f932c19a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:15:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f932c075fa0 0x7f932c19a870 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9318000c00 tx=0x7f93180042f0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 -- 172.21.15.86:0/3266761010 shutdown_connections 2024-04-17T07:15:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93100a3cc0 0x7f93100a6180 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f932c109db0 0x7f932c1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f932c075fa0 0x7f932c19a870 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 --2- 172.21.15.86:0/3266761010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f932c0749c0 0x7f932c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:15:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 -- 172.21.15.86:0/3266761010 >> 172.21.15.86:0/3266761010 conn(0x7f932c0fd460 msgr2=0x7f932c105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:15:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.053+0000 7f9330a00640 1 -- 172.21.15.86:0/3266761010 shutdown_connections 2024-04-17T07:15:46.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:15:46.054+0000 7f9330a00640 1 -- 172.21.15.86:0/3266761010 wait complete. 2024-04-17T07:15:46.072 INFO:tasks.cephfs.filesystem:scrub status for tag:a5c6d131-885d-40b8-a246-b0af07a3b458 - {'path': '/', 'tag': 'a5c6d131-885d-40b8-a246-b0af07a3b458', 'options': 'recursive,force'} 2024-04-17T07:15:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:46 smithi098 ceph-mon[35418]: pgmap v482: 129 pgs: 129 active+clean; 37 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 2.0 MiB/s wr, 68 op/s 2024-04-17T07:15:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:46 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:15:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:46 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:46 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:15:46.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:46 smithi103 ceph-mon[35495]: pgmap v482: 129 pgs: 129 active+clean; 37 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 2.0 MiB/s wr, 68 op/s 2024-04-17T07:15:46.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:46 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:15:46.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:46 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:46.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:46 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:15:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:46 smithi086 ceph-mon[31733]: pgmap v482: 129 pgs: 129 active+clean; 37 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 2.0 MiB/s wr, 68 op/s 2024-04-17T07:15:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:46 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:15:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:46 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:46 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:15:47.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:47 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:47.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:47 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:47.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:47 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:15:48.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:48 smithi098 ceph-mon[35418]: pgmap v483: 129 pgs: 129 active+clean; 43 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 2.5 MiB/s wr, 81 op/s 2024-04-17T07:15:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:48 smithi103 ceph-mon[35495]: pgmap v483: 129 pgs: 129 active+clean; 43 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 2.5 MiB/s wr, 81 op/s 2024-04-17T07:15:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:48 smithi086 ceph-mon[31733]: pgmap v483: 129 pgs: 129 active+clean; 43 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 2.5 MiB/s wr, 81 op/s 2024-04-17T07:15:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:49 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:15:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:49 smithi098 ceph-mon[35418]: pgmap v484: 129 pgs: 129 active+clean; 45 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.1 MiB/s wr, 72 op/s 2024-04-17T07:15:49.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:49 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:15:49.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:49 smithi103 ceph-mon[35495]: pgmap v484: 129 pgs: 129 active+clean; 45 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.1 MiB/s wr, 72 op/s 2024-04-17T07:15:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:49 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:15:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:49 smithi086 ceph-mon[31733]: pgmap v484: 129 pgs: 129 active+clean; 45 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.1 MiB/s wr, 72 op/s 2024-04-17T07:15:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:52 smithi098 ceph-mon[35418]: pgmap v485: 129 pgs: 129 active+clean; 45 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.4 MiB/s wr, 53 op/s 2024-04-17T07:15:52.633 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:52 smithi086 ceph-mon[31733]: pgmap v485: 129 pgs: 129 active+clean; 45 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.4 MiB/s wr, 53 op/s 2024-04-17T07:15:52.633 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:52 smithi103 ceph-mon[35495]: pgmap v485: 129 pgs: 129 active+clean; 45 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.4 MiB/s wr, 53 op/s 2024-04-17T07:15:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:54 smithi098 ceph-mon[35418]: pgmap v486: 129 pgs: 129 active+clean; 46 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 60 op/s 2024-04-17T07:15:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:54 smithi103 ceph-mon[35495]: pgmap v486: 129 pgs: 129 active+clean; 46 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 60 op/s 2024-04-17T07:15:54.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:54 smithi086 ceph-mon[31733]: pgmap v486: 129 pgs: 129 active+clean; 46 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 60 op/s 2024-04-17T07:15:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:56 smithi098 ceph-mon[35418]: pgmap v487: 129 pgs: 129 active+clean; 46 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 977 KiB/s wr, 43 op/s 2024-04-17T07:15:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:56 smithi103 ceph-mon[35495]: pgmap v487: 129 pgs: 129 active+clean; 46 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 977 KiB/s wr, 43 op/s 2024-04-17T07:15:56.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:56 smithi086 ceph-mon[31733]: pgmap v487: 129 pgs: 129 active+clean; 46 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 977 KiB/s wr, 43 op/s 2024-04-17T07:15:58.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:58 smithi098 ceph-mon[35418]: pgmap v488: 129 pgs: 129 active+clean; 48 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 979 KiB/s wr, 66 op/s 2024-04-17T07:15:58.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:58 smithi103 ceph-mon[35495]: pgmap v488: 129 pgs: 129 active+clean; 48 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 979 KiB/s wr, 66 op/s 2024-04-17T07:15:58.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:58 smithi086 ceph-mon[31733]: pgmap v488: 129 pgs: 129 active+clean; 48 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 979 KiB/s wr, 66 op/s 2024-04-17T07:15:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:15:59 smithi098 ceph-mon[35418]: pgmap v489: 129 pgs: 129 active+clean; 48 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 409 KiB/s wr, 50 op/s 2024-04-17T07:15:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:15:59 smithi103 ceph-mon[35495]: pgmap v489: 129 pgs: 129 active+clean; 48 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 409 KiB/s wr, 50 op/s 2024-04-17T07:15:59.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:15:59 smithi086 ceph-mon[31733]: pgmap v489: 129 pgs: 129 active+clean; 48 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 409 KiB/s wr, 50 op/s 2024-04-17T07:16:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:02 smithi098 ceph-mon[35418]: pgmap v490: 129 pgs: 129 active+clean; 50 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 430 KiB/s wr, 75 op/s 2024-04-17T07:16:02.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:02 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:16:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:02 smithi103 ceph-mon[35495]: pgmap v490: 129 pgs: 129 active+clean; 50 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 430 KiB/s wr, 75 op/s 2024-04-17T07:16:02.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:02 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:16:02.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:02 smithi086 ceph-mon[31733]: pgmap v490: 129 pgs: 129 active+clean; 50 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 430 KiB/s wr, 75 op/s 2024-04-17T07:16:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:02 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:16:03.007 INFO:teuthology.orchestra.run.smithi086.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-04-17T07:16:03.017 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-04-17T07:16:03.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:16:03.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:16:03.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:03 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:16:03.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:16:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:16:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:03 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:16:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:16:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:16:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:03 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:16:04.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:04 smithi098 ceph-mon[35418]: pgmap v491: 129 pgs: 129 active+clean; 50 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 434 KiB/s wr, 81 op/s 2024-04-17T07:16:04.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:04 smithi103 ceph-mon[35495]: pgmap v491: 129 pgs: 129 active+clean; 50 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 434 KiB/s wr, 81 op/s 2024-04-17T07:16:04.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:04 smithi086 ceph-mon[31733]: pgmap v491: 129 pgs: 129 active+clean; 50 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 434 KiB/s wr, 81 op/s 2024-04-17T07:16:04.959 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/: 2024-04-17T07:16:04.959 INFO:teuthology.orchestra.run.smithi086.stdout:total 2 2024-04-17T07:16:04.959 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:15 . 2024-04-17T07:16:04.959 INFO:teuthology.orchestra.run.smithi086.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:15 .. 2024-04-17T07:16:04.959 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 data 2024-04-17T07:16:04.959 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:04.959 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data: 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:total 45 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 . 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:15 .. 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Apr 17 07:15 base 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 Apr 17 07:15 global 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:16 log 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_commit_ts 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_dynshmem 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Apr 17 07:15 pg_hba.conf 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Apr 17 07:15 pg_ident.conf 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 17 07:15 pg_logical 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 pg_multixact 2024-04-17T07:16:04.960 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_notify 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_replslot 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_serial 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_snapshots 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_stat 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_stat_tmp 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 pg_subtrans 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_tblspc 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_twophase 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 17 07:15 PG_VERSION 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:15 pg_wal 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 pg_xact 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Apr 17 07:15 postgresql.auto.conf 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Apr 17 07:15 postgresql.conf 2024-04-17T07:16:04.961 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/base: 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:total 3 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Apr 17 07:15 . 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 17 07:15 1 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 17 07:15 13493 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 17 07:15 13494 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/base/1: 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:total 7778 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 17 07:15 . 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Apr 17 07:15 .. 2024-04-17T07:16:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 112 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 113 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:15 1247 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1247_fsm 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1247_vm 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 17 07:15 1249 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1249_fsm 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1249_vm 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 17 07:15 1255 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1255_fsm 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1255_vm 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 17 07:15 1259 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1259_fsm 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1259_vm 2024-04-17T07:16:04.963 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 17 07:15 13344 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13344_fsm 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13344_vm 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13346 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13348 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13349_fsm 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349_vm 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13351 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13353 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13354_fsm 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354_vm 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13356 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13358 2024-04-17T07:16:04.964 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13359_fsm 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359_vm 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13361 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13363 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1417 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1418 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 174 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 175 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2187 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2224 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2328 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2336 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2337 2024-04-17T07:16:04.965 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2579 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2600 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2600_fsm 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2600_vm 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2601_fsm 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601_vm 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2602 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2602_fsm 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2602_vm 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2603 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2603_fsm 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2603_vm 2024-04-17T07:16:04.966 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2604 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2605 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2605_fsm 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2605_vm 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2606 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2606_fsm 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2606_vm 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2607 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2607_fsm 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2607_vm 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 17 07:15 2608 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2608_fsm 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2608_vm 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 17 07:15 2609 2024-04-17T07:16:04.967 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2609_fsm 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2609_vm 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2610 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2610_fsm 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2610_vm 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2611 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2612_fsm 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612_vm 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2613 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2615_fsm 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615_vm 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2616 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2616_fsm 2024-04-17T07:16:04.968 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2616_vm 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 17 07:15 2617 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2617_fsm 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2617_vm 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 17 07:15 2618 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2618_fsm 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2618_vm 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Apr 17 07:15 2619 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2619_fsm 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2619_vm 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2620 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2650 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2651 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2652 2024-04-17T07:16:04.969 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2653 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2654 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2655 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2656 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2657 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 17 07:15 2658 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:15 2659 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2660 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2661 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2662 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2663 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2664 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2665 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2666 2024-04-17T07:16:04.970 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2667 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2668 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2669 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2670 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 17 07:15 2673 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 17 07:15 2674 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 17 07:15 2675 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2678 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2679 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2680 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2681 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2682 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2683 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2684 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2685 2024-04-17T07:16:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2686 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2687 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2688 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2689 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 17 07:15 2690 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 17 07:15 2691 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2692 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2693 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2696 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2699 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2701 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2702 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2703 2024-04-17T07:16:04.972 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2704 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2753 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2753_fsm 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2753_vm 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2754 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2755 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2756 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2757 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2830 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2831 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2832 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2833 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2834 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2835 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2836 2024-04-17T07:16:04.973 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2837 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 17 07:15 2838 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2838_fsm 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2838_vm 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2839 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840_fsm 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2840_vm 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2841 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2995 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2996 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3079_fsm 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079_vm 2024-04-17T07:16:04.974 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3080 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3081 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3085 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3118 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3119 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 17 07:15 3164 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3256 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3257 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3258 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3350 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3351 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3379 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3380 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3381 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3394 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3394_fsm 2024-04-17T07:16:04.975 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3394_vm 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3395 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3429 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3430 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3431 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3433 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3439 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3440 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3455 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 17 07:15 3456 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3456_fsm 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3456_vm 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3466 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3467 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3468 2024-04-17T07:16:04.976 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3501 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3502 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3503 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3534 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3541_fsm 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541_vm 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3542 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3574 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3575 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3576 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3596 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3597 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3598 2024-04-17T07:16:04.977 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3599 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3600_fsm 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600_vm 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3601_fsm 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601_vm 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3602_fsm 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602_vm 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603_fsm 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3603_vm 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3604 2024-04-17T07:16:04.978 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3605 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3606 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3607 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3608 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 3609 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3712 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3764_fsm 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764_vm 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3766 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3767 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3997 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4143 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4144 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4145 2024-04-17T07:16:04.979 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4146 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4147 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4148 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4149 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4150 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4151 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4152 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4153 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4154 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4155 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4156 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4157 2024-04-17T07:16:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4158 2024-04-17T07:16:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:05 smithi098 ceph-mon[35418]: pgmap v492: 129 pgs: 129 active+clean; 50 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 384 KiB/s wr, 73 op/s 2024-04-17T07:16:05.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:05 smithi103 ceph-mon[35495]: pgmap v492: 129 pgs: 129 active+clean; 50 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 384 KiB/s wr, 73 op/s 2024-04-17T07:16:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:05 smithi086 ceph-mon[31733]: pgmap v492: 129 pgs: 129 active+clean; 50 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 384 KiB/s wr, 73 op/s 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4159 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4160 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4161 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4162 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4163 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4164 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4165 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4166 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4167 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4168 2024-04-17T07:16:06.722 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4169 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4170 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4171 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4172 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4173 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4174 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 5002 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 548 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 549 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6102 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6104 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6106 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6110 2024-04-17T07:16:06.723 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6111 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6112 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6113 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6117 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 826 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 827 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 828 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 17 07:15 pg_filenode.map 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 17 07:15 PG_VERSION 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:total 7778 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 17 07:15 . 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Apr 17 07:15 .. 2024-04-17T07:16:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 112 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 113 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:15 1247 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1247_fsm 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1247_vm 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 17 07:15 1249 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1249_fsm 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1249_vm 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 17 07:15 1255 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1255_fsm 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1255_vm 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 17 07:15 1259 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1259_fsm 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1259_vm 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 17 07:15 13344 2024-04-17T07:16:06.725 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13344_fsm 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13344_vm 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13346 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13348 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13349_fsm 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349_vm 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13351 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13353 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13354_fsm 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354_vm 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13356 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13358 2024-04-17T07:16:06.726 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13359_fsm 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359_vm 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13361 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13363 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1417 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1418 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 174 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 175 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2187 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2224 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2328 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2336 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2337 2024-04-17T07:16:06.727 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2579 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2600 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2600_fsm 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2600_vm 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2601_fsm 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601_vm 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2602 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2602_fsm 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2602_vm 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2603 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2603_fsm 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2603_vm 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2604 2024-04-17T07:16:06.728 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2605 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2605_fsm 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2605_vm 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2606 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2606_fsm 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2606_vm 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2607 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2607_fsm 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2607_vm 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 17 07:15 2608 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2608_fsm 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2608_vm 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 17 07:15 2609 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2609_fsm 2024-04-17T07:16:06.729 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2609_vm 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2610 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2610_fsm 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2610_vm 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2611 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2612_fsm 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612_vm 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2613 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2615_fsm 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615_vm 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2616 2024-04-17T07:16:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2616_fsm 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2616_vm 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 17 07:15 2617 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2617_fsm 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2617_vm 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 17 07:15 2618 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2618_fsm 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2618_vm 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Apr 17 07:15 2619 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2619_fsm 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2619_vm 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2620 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2650 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2651 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2652 2024-04-17T07:16:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2653 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2654 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2655 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2656 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2657 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 17 07:15 2658 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:15 2659 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2660 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2661 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2662 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2663 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2664 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2665 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2666 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2667 2024-04-17T07:16:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2668 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2669 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2670 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 17 07:15 2673 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 17 07:15 2674 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 17 07:15 2675 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2678 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2679 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2680 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2681 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2682 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2683 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2684 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2685 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2686 2024-04-17T07:16:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2687 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2688 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2689 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 17 07:15 2690 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 17 07:15 2691 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2692 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2693 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2696 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2699 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2701 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2702 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2703 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2704 2024-04-17T07:16:06.734 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2753 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2753_fsm 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2753_vm 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2754 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2755 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2756 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2757 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2830 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2831 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2832 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2833 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2834 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2835 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2836 2024-04-17T07:16:06.735 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2837 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 17 07:15 2838 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2838_fsm 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2838_vm 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2839 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840_fsm 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2840_vm 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2841 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2995 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2996 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3079_fsm 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079_vm 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3080 2024-04-17T07:16:06.736 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3081 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3085 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3118 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3119 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 17 07:15 3164 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3256 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3257 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3258 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3350 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3351 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3379 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3380 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3381 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3394 2024-04-17T07:16:06.737 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3394_fsm 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3394_vm 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3395 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3429 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3430 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3431 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3433 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3439 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3440 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3455 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 17 07:15 3456 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3456_fsm 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3456_vm 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3466 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3467 2024-04-17T07:16:06.738 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3468 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3501 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3502 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3503 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3534 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3541_fsm 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541_vm 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3542 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3574 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3575 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3576 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3596 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3597 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3598 2024-04-17T07:16:06.739 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3599 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3600_fsm 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600_vm 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3601_fsm 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601_vm 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3602_fsm 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602_vm 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603_fsm 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3603_vm 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3604 2024-04-17T07:16:06.740 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3605 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3606 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3607 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3608 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 3609 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3712 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3764_fsm 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764_vm 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3766 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3767 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3997 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4143 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4144 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4145 2024-04-17T07:16:06.741 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4146 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4147 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4148 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4149 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4150 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4151 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4152 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4153 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4154 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4155 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4156 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4157 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4158 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4159 2024-04-17T07:16:06.742 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4160 2024-04-17T07:16:06.743 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4161 2024-04-17T07:16:06.743 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4162 2024-04-17T07:16:08.497 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4163 2024-04-17T07:16:08.498 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4164 2024-04-17T07:16:08.498 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4165 2024-04-17T07:16:08.498 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4166 2024-04-17T07:16:08.498 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4167 2024-04-17T07:16:08.498 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4168 2024-04-17T07:16:08.498 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4169 2024-04-17T07:16:08.498 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4170 2024-04-17T07:16:08.498 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4171 2024-04-17T07:16:08.498 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4172 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4173 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4174 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 5002 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 548 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 549 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6102 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6104 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6106 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6110 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6111 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6112 2024-04-17T07:16:08.499 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6113 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6117 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 826 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 827 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 828 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 17 07:15 pg_filenode.map 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 17 07:15 PG_VERSION 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout:total 7778 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 17 07:15 . 2024-04-17T07:16:08.500 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Apr 17 07:15 .. 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 112 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 113 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:15 1247 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1247_fsm 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1247_vm 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 17 07:15 1249 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1249_fsm 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1249_vm 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 17 07:15 1255 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1255_fsm 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1255_vm 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 17 07:15 1259 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1259_fsm 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1259_vm 2024-04-17T07:16:08.501 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 17 07:15 13344 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13344_fsm 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13344_vm 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13346 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13348 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13349_fsm 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349_vm 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13351 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13353 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13354_fsm 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354_vm 2024-04-17T07:16:08.502 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13356 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13358 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13359_fsm 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359_vm 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13361 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13363 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1417 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1418 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 174 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 175 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2187 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2224 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2328 2024-04-17T07:16:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2336 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2337 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2579 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2600 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2600_fsm 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2600_vm 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2601_fsm 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601_vm 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2602 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2602_fsm 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2602_vm 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2603 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2603_fsm 2024-04-17T07:16:08.504 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2603_vm 2024-04-17T07:16:08.505 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2604 2024-04-17T07:16:08.505 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2605 2024-04-17T07:16:08.505 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2605_fsm 2024-04-17T07:16:08.505 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2605_vm 2024-04-17T07:16:08.505 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2606 2024-04-17T07:16:08.505 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2606_fsm 2024-04-17T07:16:08.506 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2606_vm 2024-04-17T07:16:08.506 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2607 2024-04-17T07:16:08.506 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2607_fsm 2024-04-17T07:16:08.506 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2607_vm 2024-04-17T07:16:08.506 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 17 07:15 2608 2024-04-17T07:16:08.507 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2608_fsm 2024-04-17T07:16:08.507 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2608_vm 2024-04-17T07:16:08.507 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 17 07:15 2609 2024-04-17T07:16:08.507 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2609_fsm 2024-04-17T07:16:08.507 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2609_vm 2024-04-17T07:16:08.507 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2610 2024-04-17T07:16:08.508 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2610_fsm 2024-04-17T07:16:08.508 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2610_vm 2024-04-17T07:16:08.508 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2611 2024-04-17T07:16:08.508 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612 2024-04-17T07:16:08.508 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2612_fsm 2024-04-17T07:16:08.508 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612_vm 2024-04-17T07:16:08.508 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2613 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2615_fsm 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615_vm 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2616 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2616_fsm 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2616_vm 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 17 07:15 2617 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2617_fsm 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2617_vm 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 17 07:15 2618 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2618_fsm 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2618_vm 2024-04-17T07:16:08.509 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Apr 17 07:15 2619 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2619_fsm 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2619_vm 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2620 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2650 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2651 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2652 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2653 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2654 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2655 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2656 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2657 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 17 07:15 2658 2024-04-17T07:16:08.510 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:15 2659 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2660 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2661 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2662 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2663 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2664 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2665 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2666 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2667 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2668 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2669 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2670 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 17 07:15 2673 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 17 07:15 2674 2024-04-17T07:16:08.511 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 17 07:15 2675 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2678 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2679 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2680 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2681 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2682 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2683 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2684 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2685 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2686 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2687 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2688 2024-04-17T07:16:08.512 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2689 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 17 07:15 2690 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 17 07:15 2691 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2692 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2693 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2696 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2699 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2701 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2702 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2703 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2704 2024-04-17T07:16:08.513 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2753 2024-04-17T07:16:08.514 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2753_fsm 2024-04-17T07:16:08.514 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2753_vm 2024-04-17T07:16:08.514 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2754 2024-04-17T07:16:08.514 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2755 2024-04-17T07:16:08.514 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2756 2024-04-17T07:16:08.514 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2757 2024-04-17T07:16:08.514 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2830 2024-04-17T07:16:08.514 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2831 2024-04-17T07:16:08.514 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2832 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2833 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2834 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2835 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2836 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2837 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 17 07:15 2838 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2838_fsm 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2838_vm 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2839 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840_fsm 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2840_vm 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2841 2024-04-17T07:16:08.515 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2995 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2996 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3079_fsm 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079_vm 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3080 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3081 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3085 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3118 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3119 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 17 07:15 3164 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3256 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3257 2024-04-17T07:16:08.516 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3258 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3350 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3351 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3379 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3380 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3381 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3394 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3394_fsm 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3394_vm 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3395 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3429 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3430 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3431 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3433 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3439 2024-04-17T07:16:08.517 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3440 2024-04-17T07:16:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3455 2024-04-17T07:16:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 17 07:15 3456 2024-04-17T07:16:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3456_fsm 2024-04-17T07:16:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3456_vm 2024-04-17T07:16:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3466 2024-04-17T07:16:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3467 2024-04-17T07:16:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3468 2024-04-17T07:16:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3501 2024-04-17T07:16:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3502 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3503 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3534 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3541_fsm 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541_vm 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3542 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3574 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3575 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3576 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3596 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3597 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3598 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3599 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600 2024-04-17T07:16:08.519 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3600_fsm 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600_vm 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3601_fsm 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601_vm 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3602_fsm 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602_vm 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603_fsm 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3603_vm 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3604 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3605 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3606 2024-04-17T07:16:08.520 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3607 2024-04-17T07:16:08.521 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3608 2024-04-17T07:16:08.521 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 3609 2024-04-17T07:16:08.522 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3712 2024-04-17T07:16:08.522 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764 2024-04-17T07:16:08.522 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3764_fsm 2024-04-17T07:16:08.522 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764_vm 2024-04-17T07:16:08.522 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3766 2024-04-17T07:16:08.522 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3767 2024-04-17T07:16:08.522 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3997 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4143 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4144 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4145 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4146 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4147 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4148 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4149 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4150 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4151 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4152 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4153 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4154 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4155 2024-04-17T07:16:08.523 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4156 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4157 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4158 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4159 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4160 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4161 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4162 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4163 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4164 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4165 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4166 2024-04-17T07:16:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4167 2024-04-17T07:16:08.525 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:08 smithi098 ceph-mon[35418]: pgmap v493: 129 pgs: 129 active+clean; 51 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 472 KiB/s wr, 93 op/s 2024-04-17T07:16:08.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:08 smithi103 ceph-mon[35495]: pgmap v493: 129 pgs: 129 active+clean; 51 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 472 KiB/s wr, 93 op/s 2024-04-17T07:16:08.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:08 smithi086 ceph-mon[31733]: pgmap v493: 129 pgs: 129 active+clean; 51 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 472 KiB/s wr, 93 op/s 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4168 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4169 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4170 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4171 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4172 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4173 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4174 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 5002 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 548 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 549 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6102 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6104 2024-04-17T07:16:08.786 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6106 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6110 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6111 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6112 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6113 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6117 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 826 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 827 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 828 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 17 07:15 pg_filenode.map 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 17 07:15 PG_VERSION 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/global: 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:total 538 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 Apr 17 07:15 . 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:08.787 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1213 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1213_fsm 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1213_vm 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1214 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1214_fsm 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1214_vm 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 1232 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 1233 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1260 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1260_fsm 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1260_vm 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1261 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1261_fsm 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1261_vm 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1262 2024-04-17T07:16:08.788 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1262_fsm 2024-04-17T07:16:08.801 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1262_vm 2024-04-17T07:16:08.801 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2396 2024-04-17T07:16:08.801 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2396_fsm 2024-04-17T07:16:08.801 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2396_vm 2024-04-17T07:16:08.801 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2397 2024-04-17T07:16:08.801 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2671 2024-04-17T07:16:08.801 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2672 2024-04-17T07:16:08.801 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2676 2024-04-17T07:16:08.801 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2677 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2694 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2695 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2697 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2698 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2846 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2847 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2964 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2965 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2966 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2967 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3592 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3593 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4060 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4061 2024-04-17T07:16:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4175 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4176 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4177 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4178 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4181 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4182 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4183 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4184 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4185 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4186 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6000 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6001 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6002 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6100 2024-04-17T07:16:08.803 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6114 2024-04-17T07:16:08.804 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6115 2024-04-17T07:16:08.804 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 pg_control 2024-04-17T07:16:08.804 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 17 07:15 pg_filenode.map 2024-04-17T07:16:08.804 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:08.804 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/log: 2024-04-17T07:16:08.804 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:08.804 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:16 . 2024-04-17T07:16:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.081 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-04-17T07:16:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.081 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-04-17T07:16:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.081 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:total 3 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 17 07:15 . 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 mappings 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 17 07:15 replorigin_checkpoint 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 snapshots 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.082 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 17 07:15 .. 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 17 07:15 .. 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:total 2 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 . 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 members 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 offsets 2024-04-17T07:16:09.083 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:total 9 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 . 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 .. 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 0000 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:total 9 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 . 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 .. 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 0000 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.084 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-04-17T07:16:09.085 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.086 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-04-17T07:16:09.164 INFO:teuthology.orchestra.run.smithi086.stdout:total 9 2024-04-17T07:16:09.164 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 . 2024-04-17T07:16:09.164 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 0000 2024-04-17T07:16:09.164 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.164 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:total 16386 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:15 . 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:15 000000010000000000000001 2024-04-17T07:16:09.166 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 archive_status 2024-04-17T07:16:09.166 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.166 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-04-17T07:16:09.166 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:16:09.166 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:16:09.166 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:15 .. 2024-04-17T07:16:09.166 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:09.166 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-04-17T07:16:09.166 INFO:teuthology.orchestra.run.smithi086.stdout:total 9 2024-04-17T07:16:09.167 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 . 2024-04-17T07:16:09.167 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 Apr 17 07:16 .. 2024-04-17T07:16:09.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 0000 2024-04-17T07:16:09.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-04-17T07:16:09.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-04-17T07:16:10.091 INFO:teuthology.orchestra.run.smithi086.stderr:dropping old tables... 2024-04-17T07:16:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-04-17T07:16:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-04-17T07:16:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-04-17T07:16:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-04-17T07:16:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:creating tables... 2024-04-17T07:16:10.326 INFO:teuthology.orchestra.run.smithi086.stderr:generating data (client-side)... 2024-04-17T07:16:10.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:10 smithi098 ceph-mon[35418]: pgmap v494: 129 pgs: 129 active+clean; 51 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 265 KiB/s wr, 65 op/s 2024-04-17T07:16:10.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:10 smithi103 ceph-mon[35495]: pgmap v494: 129 pgs: 129 active+clean; 51 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 265 KiB/s wr, 65 op/s 2024-04-17T07:16:10.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:10 smithi086 ceph-mon[31733]: pgmap v494: 129 pgs: 129 active+clean; 51 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 265 KiB/s wr, 65 op/s 2024-04-17T07:16:12.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:12 smithi098 ceph-mon[35418]: pgmap v495: 129 pgs: 129 active+clean; 51 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 283 KiB/s wr, 68 op/s 2024-04-17T07:16:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:12 smithi103 ceph-mon[35495]: pgmap v495: 129 pgs: 129 active+clean; 51 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 283 KiB/s wr, 68 op/s 2024-04-17T07:16:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:12 smithi086 ceph-mon[31733]: pgmap v495: 129 pgs: 129 active+clean; 51 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 283 KiB/s wr, 68 op/s 2024-04-17T07:16:12.864 INFO:teuthology.orchestra.run.smithi086.stderr:100000 of 3200000 tuples (3%) done (elapsed 2.41 s, remaining 74.71 s) 2024-04-17T07:16:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:13 smithi086 ceph-mon[31733]: pgmap v496: 129 pgs: 129 active+clean; 51 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 884 KiB/s wr, 39 op/s 2024-04-17T07:16:13.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:13 smithi098 ceph-mon[35418]: pgmap v496: 129 pgs: 129 active+clean; 51 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 884 KiB/s wr, 39 op/s 2024-04-17T07:16:13.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:13 smithi103 ceph-mon[35495]: pgmap v496: 129 pgs: 129 active+clean; 51 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 884 KiB/s wr, 39 op/s 2024-04-17T07:16:16.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:16:16.073 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:16:16.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.310+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2427284169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd14107b50 msgr2=0x7fdd14109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.310+0000 7fdd1aa00640 1 --2- 172.21.15.86:0/2427284169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd14107b50 0x7fdd14109fa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009ca0 tx=0x7fdd0c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.310+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2427284169 shutdown_connections 2024-04-17T07:16:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.310+0000 7fdd1aa00640 1 --2- 172.21.15.86:0/2427284169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd14107b50 0x7fdd14109fa0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.310+0000 7fdd1aa00640 1 --2- 172.21.15.86:0/2427284169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd14105220 0x7fdd14107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.310+0000 7fdd1aa00640 1 --2- 172.21.15.86:0/2427284169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd141028f0 0x7fdd14104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.310+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2427284169 >> 172.21.15.86:0/2427284169 conn(0x7fdd140fc300 msgr2=0x7fdd140fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.310+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2427284169 shutdown_connections 2024-04-17T07:16:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.310+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2427284169 wait complete. 2024-04-17T07:16:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.311+0000 7fdd1aa00640 1 Processor -- start 2024-04-17T07:16:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.311+0000 7fdd1aa00640 1 -- start start 2024-04-17T07:16:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd1aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd141028f0 0x7fdd1419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd14105220 0x7fdd1419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd1aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14107b50 0x7fdd141a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd1aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410d9c0 con 0x7fdd14105220 2024-04-17T07:16:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd1aa00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410d840 con 0x7fdd14107b50 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd1aa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1410db40 con 0x7fdd141028f0 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd14105220 0x7fdd1419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd14105220 0x7fdd1419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40738/0 (socket says 172.21.15.86:40738) 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd18c00640 1 -- 172.21.15.86:0/2763620818 learned_addr learned my addr 172.21.15.86:0/2763620818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd19600640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd141028f0 0x7fdd1419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.312+0000 7fdd1a000640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14107b50 0x7fdd141a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.313+0000 7fdd18c00640 1 -- 172.21.15.86:0/2763620818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd141028f0 msgr2=0x7fdd1419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.313+0000 7fdd18c00640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd141028f0 0x7fdd1419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.313+0000 7fdd18c00640 1 -- 172.21.15.86:0/2763620818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14107b50 msgr2=0x7fdd141a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.313+0000 7fdd18c00640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14107b50 0x7fdd141a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.313+0000 7fdd18c00640 1 -- 172.21.15.86:0/2763620818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd141a1e90 con 0x7fdd14105220 2024-04-17T07:16:16.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.313+0000 7fdd18c00640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd14105220 0x7fdd1419a7b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000d740 tx=0x7fdd0000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:16.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.314+0000 7fdd12000640 1 -- 172.21.15.86:0/2763620818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00017070 con 0x7fdd14105220 2024-04-17T07:16:16.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.314+0000 7fdd12000640 1 -- 172.21.15.86:0/2763620818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd0000de50 con 0x7fdd14105220 2024-04-17T07:16:16.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.314+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2763620818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd141a2180 con 0x7fdd14105220 2024-04-17T07:16:16.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.314+0000 7fdd12000640 1 -- 172.21.15.86:0/2763620818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00004d60 con 0x7fdd14105220 2024-04-17T07:16:16.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.314+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2763620818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd141a2680 con 0x7fdd14105220 2024-04-17T07:16:16.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.318+0000 7fdd1ec01740 1 Processor -- start 2024-04-17T07:16:16.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.318+0000 7fdd12000640 1 -- 172.21.15.86:0/2763620818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fdd00004f00 con 0x7fdd14105220 2024-04-17T07:16:16.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.319+0000 7fdd1ec01740 1 -- start start 2024-04-17T07:16:16.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.319+0000 7fdd12000640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdcfc0a3c40 0x7fdcfc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.319+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76370 0x55ed25e76790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.319+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76d60 0x55ed25e7b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.319+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e7e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.319+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed25ddd120 con 0x55ed25e76370 2024-04-17T07:16:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.319+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ed25d83340 con 0x55ed25e76d60 2024-04-17T07:16:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.319+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ed25dd90e0 con 0x55ed25e7bed0 2024-04-17T07:16:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.319+0000 7fdd12000640 1 -- 172.21.15.86:0/2763620818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdd000d0290 con 0x7fdd14105220 2024-04-17T07:16:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.319+0000 7fdd1a000640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdcfc0a3c40 0x7fdcfc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd19600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76370 0x55ed25e76790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd19600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76370 0x55ed25e76790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40742/0 (socket says 172.21.15.86:40742) 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd19600640 1 -- 172.21.15.86:0/916423876 learned_addr learned my addr 172.21.15.86:0/916423876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd1a000640 1 --2- 172.21.15.86:0/916423876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e7e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd18c00640 1 --2- 172.21.15.86:0/916423876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76d60 0x55ed25e7b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd19600640 1 -- 172.21.15.86:0/916423876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 msgr2=0x55ed25e7e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd19600640 1 --2- 172.21.15.86:0/916423876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e7e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd19600640 1 -- 172.21.15.86:0/916423876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76d60 msgr2=0x55ed25e7b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd19600640 1 --2- 172.21.15.86:0/916423876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76d60 0x55ed25e7b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd19600640 1 -- 172.21.15.86:0/916423876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed25bb97d0 con 0x55ed25e76370 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.320+0000 7fdd1a000640 1 --2- 172.21.15.86:0/916423876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e7e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.321+0000 7fdd19600640 1 --2- 172.21.15.86:0/916423876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76370 0x55ed25e76790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800bd10 tx=0x7fdd080107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d8665e8af0a003 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.321+0000 7fdd1a000640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdcfc0a3c40 0x7fdcfc0a6100 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c006010 tx=0x7fdd0c005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.321+0000 7fdd06600640 1 -- 172.21.15.86:0/916423876 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08004030 con 0x55ed25e76370 2024-04-17T07:16:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.321+0000 7fdd06600640 1 -- 172.21.15.86:0/916423876 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd080035f0 con 0x55ed25e76370 2024-04-17T07:16:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.321+0000 7fdd06600640 1 -- 172.21.15.86:0/916423876 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd080117b0 con 0x55ed25e76370 2024-04-17T07:16:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.322+0000 7fdd1ec01740 1 -- 172.21.15.86:0/916423876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76370 msgr2=0x55ed25e76790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.322+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/916423876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76370 0x55ed25e76790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800bd10 tx=0x7fdd080107a0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.322+0000 7fdd1ec01740 1 -- 172.21.15.86:0/916423876 shutdown_connections 2024-04-17T07:16:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.322+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/916423876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e7e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.322+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/916423876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76d60 0x55ed25e7b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.322+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/916423876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76370 0x55ed25e76790 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.322+0000 7fdd1ec01740 1 -- 172.21.15.86:0/916423876 >> 172.21.15.86:0/916423876 conn(0x55ed25e718e0 msgr2=0x55ed25e71ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:16.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.322+0000 7fdd1ec01740 1 -- 172.21.15.86:0/916423876 shutdown_connections 2024-04-17T07:16:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.322+0000 7fdd1ec01740 1 -- 172.21.15.86:0/916423876 wait complete. 2024-04-17T07:16:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.323+0000 7fdd1ec01740 1 -- start start 2024-04-17T07:16:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.323+0000 7fdd1ec01740 1 Processor -- start 2024-04-17T07:16:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.323+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e76370 0x55ed25e80850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.323+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76d60 0x55ed25e80d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.323+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e7bed0 0x55ed25e87dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.323+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed25e7ee80 con 0x55ed25e7bed0 2024-04-17T07:16:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.323+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ed25d96710 con 0x55ed25e76d60 2024-04-17T07:16:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.323+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ed25e7f000 con 0x55ed25e76370 2024-04-17T07:16:16.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.323+0000 7fdd1ec01740 10 client.0 _ll_register_callbacks cb 0x55ed25d91af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:16.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.324+0000 7fdd1a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e7bed0 0x55ed25e87dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.324+0000 7fdd1a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e7bed0 0x55ed25e87dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40748/0 (socket says 172.21.15.86:40748) 2024-04-17T07:16:16.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.324+0000 7fdd1a000640 1 -- 172.21.15.86:0/674684751 learned_addr learned my addr 172.21.15.86:0/674684751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:16.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.324+0000 7fdd19600640 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e76370 0x55ed25e80850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.324+0000 7fdd18c00640 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76d60 0x55ed25e80d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.324+0000 7fdd1a000640 1 -- 172.21.15.86:0/674684751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e76370 msgr2=0x55ed25e80850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:16.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.324+0000 7fdd1a000640 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e76370 0x55ed25e80850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.324+0000 7fdd1a000640 1 -- 172.21.15.86:0/674684751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76d60 msgr2=0x55ed25e80d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:16.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.324+0000 7fdd1a000640 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76d60 0x55ed25e80d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.324+0000 7fdd1a000640 1 -- 172.21.15.86:0/674684751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed25aef3f0 con 0x55ed25e7bed0 2024-04-17T07:16:16.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.325+0000 7fdd1a000640 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e7bed0 0x55ed25e87dd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005e30 tx=0x7fdd0c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31df05d70a756b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:16.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.325+0000 7fdcfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:16.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.325+0000 7fdcfaa00640 1 -- 172.21.15.86:0/674684751 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c052060 con 0x55ed25e7bed0 2024-04-17T07:16:16.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.325+0000 7fdcfaa00640 1 -- 172.21.15.86:0/674684751 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdd0c004450 con 0x55ed25e7bed0 2024-04-17T07:16:16.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.325+0000 7fdd1ec01740 1 -- 172.21.15.86:0/674684751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd080035f0 con 0x55ed25e7bed0 2024-04-17T07:16:16.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.325+0000 7fdcfaa00640 1 -- 172.21.15.86:0/674684751 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd0c059050 con 0x55ed25e7bed0 2024-04-17T07:16:16.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.325+0000 7fdd1ec01740 10 client.15957 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:16.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.325+0000 7fdd1ec01740 1 -- 172.21.15.86:0/674684751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed25aaf0f0 con 0x55ed25e7bed0 2024-04-17T07:16:16.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.326+0000 7fdcfaa00640 1 -- 172.21.15.86:0/674684751 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd080035f0 con 0x55ed25e7bed0 2024-04-17T07:16:16.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.326+0000 7fdcfaa00640 1 -- 172.21.15.86:0/674684751 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fdd0c03f040 con 0x55ed25e7bed0 2024-04-17T07:16:16.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.326+0000 7fdd1ec01740 10 client.15957 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:16.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.326+0000 7fdd1ec01740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:16.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.326+0000 7fdd1ec01740 20 client.15957 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:16.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.326+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ed25e8c6c0 0x55ed25e8eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.326+0000 7fdd1ec01740 4 client.15957 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:16.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.326+0000 7fdd1ec01740 1 -- 172.21.15.86:0/674684751 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed25deab40 con 0x55ed25e8c6c0 2024-04-17T07:16:16.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.327+0000 7fdd19600640 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ed25e8c6c0 0x55ed25e8eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.328+0000 7fdd19600640 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ed25e8c6c0 0x55ed25e8eb20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21585a44d9bbcacd server_cookie=47e82bf3979b9f3e in_seq=0 out_seq=0 2024-04-17T07:16:16.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.328+0000 7fdcfaa00640 10 client.15957 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:16.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdcfaa00640 1 -- 172.21.15.86:0/674684751 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed25deab40 con 0x55ed25e8c6c0 2024-04-17T07:16:16.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdcfaa00640 10 client.15957 handle_command_reply: tid=0 2024-04-17T07:16:16.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 client.15957 shutdown 2024-04-17T07:16:16.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 -- 172.21.15.86:0/674684751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e7bed0 msgr2=0x55ed25e87dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e7bed0 0x55ed25e87dd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005e30 tx=0x7fdd0c0095c0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 -- 172.21.15.86:0/674684751 shutdown_connections 2024-04-17T07:16:16.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ed25e8c6c0 0x55ed25e8eb20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e7bed0 0x55ed25e87dd0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76d60 0x55ed25e80d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/674684751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e76370 0x55ed25e80850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 -- 172.21.15.86:0/674684751 >> 172.21.15.86:0/674684751 conn(0x55ed25e718e0 msgr2=0x55ed25e70ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:16.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdcfaa00640 0 client.15957 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:16.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 -- 172.21.15.86:0/674684751 shutdown_connections 2024-04-17T07:16:16.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 1 -- 172.21.15.86:0/674684751 wait complete. 2024-04-17T07:16:16.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.330+0000 7fdd1ec01740 20 client.15957 trim_cache size 0 max 16384 2024-04-17T07:16:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.353+0000 7fdd1ec01740 1 Processor -- start 2024-04-17T07:16:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.353+0000 7fdd1ec01740 1 -- start start 2024-04-17T07:16:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 0x55ed25e75250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e87d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141aca40 con 0x55ed25e76d60 2024-04-17T07:16:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ae580 con 0x55ed25e76370 2024-04-17T07:16:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ad110 con 0x55ed25e7bed0 2024-04-17T07:16:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e87d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e87d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40764/0 (socket says 172.21.15.86:40764) 2024-04-17T07:16:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd18c00640 1 -- 172.21.15.86:0/4067964183 learned_addr learned my addr 172.21.15.86:0/4067964183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd19600640 1 --2- 172.21.15.86:0/4067964183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 0x55ed25e75250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.354+0000 7fdd1a000640 1 --2- 172.21.15.86:0/4067964183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.355+0000 7fdd18c00640 1 -- 172.21.15.86:0/4067964183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 msgr2=0x55ed25e933e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.355+0000 7fdd18c00640 1 --2- 172.21.15.86:0/4067964183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e933e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.355+0000 7fdd18c00640 1 -- 172.21.15.86:0/4067964183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 msgr2=0x55ed25e75250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.355+0000 7fdd18c00640 1 --2- 172.21.15.86:0/4067964183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 0x55ed25e75250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.355+0000 7fdd18c00640 1 -- 172.21.15.86:0/4067964183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed25e775f0 con 0x55ed25e76d60 2024-04-17T07:16:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.355+0000 7fdd19600640 1 --2- 172.21.15.86:0/4067964183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 0x55ed25e75250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:16:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.355+0000 7fdd18c00640 1 --2- 172.21.15.86:0/4067964183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e87d90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdd000cd890 tx=0x7fdd00009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ec3636d80ac3b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.355+0000 7fdd05c00640 1 -- 172.21.15.86:0/4067964183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd000b5070 con 0x55ed25e76d60 2024-04-17T07:16:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.355+0000 7fdd05c00640 1 -- 172.21.15.86:0/4067964183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd000a03e0 con 0x55ed25e76d60 2024-04-17T07:16:16.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.355+0000 7fdd05c00640 1 -- 172.21.15.86:0/4067964183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00020040 con 0x55ed25e76d60 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.356+0000 7fdd1ec01740 1 -- 172.21.15.86:0/4067964183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 msgr2=0x55ed25e87d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.356+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/4067964183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e87d90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdd000cd890 tx=0x7fdd00009e00 comp rx=0 tx=0).stop 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.356+0000 7fdd1ec01740 1 -- 172.21.15.86:0/4067964183 shutdown_connections 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.356+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/4067964183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e933e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.356+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/4067964183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e87d90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.356+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/4067964183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 0x55ed25e75250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.356+0000 7fdd1ec01740 1 -- 172.21.15.86:0/4067964183 >> 172.21.15.86:0/4067964183 conn(0x55ed25e718e0 msgr2=0x55ed25e74b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.356+0000 7fdd1ec01740 1 -- 172.21.15.86:0/4067964183 shutdown_connections 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.356+0000 7fdd1ec01740 1 -- 172.21.15.86:0/4067964183 wait complete. 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.357+0000 7fdd1ec01740 1 -- start start 2024-04-17T07:16:16.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.357+0000 7fdd1ec01740 1 Processor -- start 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.357+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 0x55ed25e85b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.357+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e86050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.357+0000 7fdd1ec01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e87060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.357+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141adeb0 con 0x55ed25e76d60 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.357+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141ad7e0 con 0x55ed25e76370 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.357+0000 7fdd1ec01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141aaa10 con 0x55ed25e7bed0 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd1ec01740 10 client.0 _ll_register_callbacks cb 0x7fdd0c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e86050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd18c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e86050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40766/0 (socket says 172.21.15.86:40766) 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd18c00640 1 -- 172.21.15.86:0/324589633 learned_addr learned my addr 172.21.15.86:0/324589633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd19600640 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 0x55ed25e85b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd18c00640 1 -- 172.21.15.86:0/324589633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 msgr2=0x55ed25e87060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:16.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd1a000640 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e87060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd18c00640 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e87060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd18c00640 1 -- 172.21.15.86:0/324589633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 msgr2=0x55ed25e85b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd18c00640 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 0x55ed25e85b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd18c00640 1 -- 172.21.15.86:0/324589633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed25e87650 con 0x55ed25e76d60 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd19600640 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 0x55ed25e85b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.358+0000 7fdd1a000640 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e87060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.359+0000 7fdd18c00640 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e86050 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdd000a0040 tx=0x7fdd00098440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24664fe9e0783ea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.359+0000 7fdcfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.359+0000 7fdcfaa00640 1 -- 172.21.15.86:0/324589633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd000b5040 con 0x55ed25e76d60 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.359+0000 7fdcfaa00640 1 -- 172.21.15.86:0/324589633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdd00098750 con 0x55ed25e76d60 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.359+0000 7fdd1ec01740 1 -- 172.21.15.86:0/324589633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd000a03e0 con 0x55ed25e76d60 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.359+0000 7fdcfaa00640 1 -- 172.21.15.86:0/324589633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd0009b530 con 0x55ed25e76d60 2024-04-17T07:16:16.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.359+0000 7fdd1ec01740 10 client.15963 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:16.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.359+0000 7fdd1ec01740 1 -- 172.21.15.86:0/324589633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed25ef3320 con 0x55ed25e76d60 2024-04-17T07:16:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.359+0000 7fdcfaa00640 1 -- 172.21.15.86:0/324589633 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd000a03e0 con 0x55ed25e76d60 2024-04-17T07:16:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.360+0000 7fdcfaa00640 1 -- 172.21.15.86:0/324589633 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fdd0009afa0 con 0x55ed25e76d60 2024-04-17T07:16:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.360+0000 7fdd1ec01740 10 client.15963 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:16.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.360+0000 7fdd1ec01740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.360+0000 7fdd1ec01740 20 client.15963 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.360+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ed25ef3c10 0x55ed25ef6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.360+0000 7fdd1ec01740 4 client.15963 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.360+0000 7fdd1ec01740 1 -- 172.21.15.86:0/324589633 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed25e80640 con 0x55ed25ef3c10 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.360+0000 7fdd19600640 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ed25ef3c10 0x55ed25ef6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.361+0000 7fdcfaa00640 10 client.15963 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.361+0000 7fdd19600640 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ed25ef3c10 0x55ed25ef6030 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8bce889719d96d server_cookie=e42a051374c22e50 in_seq=0 out_seq=0 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdcfaa00640 1 -- 172.21.15.86:0/324589633 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ed25e80640 con 0x55ed25ef3c10 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdcfaa00640 10 client.15963 handle_command_reply: tid=0 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 client.15963 shutdown 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 -- 172.21.15.86:0/324589633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 msgr2=0x55ed25e86050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e86050 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdd000a0040 tx=0x7fdd00098440 comp rx=0 tx=0).stop 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 -- 172.21.15.86:0/324589633 shutdown_connections 2024-04-17T07:16:16.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ed25ef3c10 0x55ed25ef6030 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ed25e7bed0 0x55ed25e87060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed25e76d60 0x55ed25e86050 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 --2- 172.21.15.86:0/324589633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed25e76370 0x55ed25e85b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 -- 172.21.15.86:0/324589633 >> 172.21.15.86:0/324589633 conn(0x55ed25e718e0 msgr2=0x55ed25e753d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdcfaa00640 0 client.15963 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 -- 172.21.15.86:0/324589633 shutdown_connections 2024-04-17T07:16:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 1 -- 172.21.15.86:0/324589633 wait complete. 2024-04-17T07:16:16.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.362+0000 7fdd1ec01740 20 client.15963 trim_cache size 0 max 16384 2024-04-17T07:16:16.365 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:16:16.365 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:16:16.366 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:16:16.366 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:16:16.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2763620818 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdcfc0a3c40 msgr2=0x7fdcfc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdcfc0a3c40 0x7fdcfc0a6100 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c006010 tx=0x7fdd0c005ef0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2763620818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd14105220 msgr2=0x7fdd1419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd14105220 0x7fdd1419a7b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000d740 tx=0x7fdd0000dc00 comp rx=0 tx=0).stop 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2763620818 shutdown_connections 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdcfc0a3c40 0x7fdcfc0a6100 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd14107b50 0x7fdd141a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd14105220 0x7fdd1419a7b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 --2- 172.21.15.86:0/2763620818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd141028f0 0x7fdd1419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2763620818 >> 172.21.15.86:0/2763620818 conn(0x7fdd140fc300 msgr2=0x7fdd14102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2763620818 shutdown_connections 2024-04-17T07:16:16.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.363+0000 7fdd1aa00640 1 -- 172.21.15.86:0/2763620818 wait complete. 2024-04-17T07:16:16.384 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:16:16.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:16:16.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:16 smithi086 ceph-mon[31733]: pgmap v497: 129 pgs: 129 active+clean; 55 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.2 MiB/s wr, 34 op/s 2024-04-17T07:16:16.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:16 smithi098 ceph-mon[35418]: pgmap v497: 129 pgs: 129 active+clean; 55 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.2 MiB/s wr, 34 op/s 2024-04-17T07:16:16.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:16 smithi103 ceph-mon[35495]: pgmap v497: 129 pgs: 129 active+clean; 55 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.2 MiB/s wr, 34 op/s 2024-04-17T07:16:16.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.596+0000 7f04bec00640 1 -- 172.21.15.86:0/105897049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04b8075fa0 msgr2=0x7f04b80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.596+0000 7f04bec00640 1 --2- 172.21.15.86:0/105897049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04b8075fa0 0x7f04b80743f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009930 tx=0x7f04a802bc30 comp rx=0 tx=0).stop 2024-04-17T07:16:16.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.597+0000 7f04bec00640 1 -- 172.21.15.86:0/105897049 shutdown_connections 2024-04-17T07:16:16.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.597+0000 7f04bec00640 1 --2- 172.21.15.86:0/105897049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04b810e580 0x7f04b8110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.597+0000 7f04bec00640 1 --2- 172.21.15.86:0/105897049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04b80749c0 0x7f04b8074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.597+0000 7f04bec00640 1 --2- 172.21.15.86:0/105897049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04b8075fa0 0x7f04b80743f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.597+0000 7f04bec00640 1 -- 172.21.15.86:0/105897049 >> 172.21.15.86:0/105897049 conn(0x7f04b80fd540 msgr2=0x7f04b80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:16.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.597+0000 7f04bec00640 1 -- 172.21.15.86:0/105897049 shutdown_connections 2024-04-17T07:16:16.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.597+0000 7f04bec00640 1 -- 172.21.15.86:0/105897049 wait complete. 2024-04-17T07:16:16.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.598+0000 7f04bec00640 1 Processor -- start 2024-04-17T07:16:16.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.598+0000 7f04bec00640 1 -- start start 2024-04-17T07:16:16.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.598+0000 7f04bec00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04b80749c0 0x7f04b819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.598+0000 7f04bec00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04b8075fa0 0x7f04b819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.598+0000 7f04bec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04b810e580 0x7f04b81a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.598+0000 7f04bec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04b81140f0 con 0x7f04b810e580 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.598+0000 7f04bec00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8113f70 con 0x7f04b8075fa0 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.598+0000 7f04bec00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8114270 con 0x7f04b80749c0 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04bcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04b810e580 0x7f04b81a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04bcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04b810e580 0x7f04b81a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40780/0 (socket says 172.21.15.86:40780) 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04bcc00640 1 -- 172.21.15.86:0/2830249409 learned_addr learned my addr 172.21.15.86:0/2830249409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04b7e00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04b80749c0 0x7f04b819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04b7400640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04b8075fa0 0x7f04b819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04bcc00640 1 -- 172.21.15.86:0/2830249409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04b80749c0 msgr2=0x7f04b819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04bcc00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04b80749c0 0x7f04b819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04bcc00640 1 -- 172.21.15.86:0/2830249409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04b8075fa0 msgr2=0x7f04b819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04bcc00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04b8075fa0 0x7f04b819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04bcc00640 1 -- 172.21.15.86:0/2830249409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b81a6450 con 0x7f04b810e580 2024-04-17T07:16:16.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.599+0000 7f04b7400640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04b8075fa0 0x7f04b819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:16.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.600+0000 7f04bcc00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04b810e580 0x7f04b81a5de0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f04b000eaa0 tx=0x7f04b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:16.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.600+0000 7f04b4c00640 1 -- 172.21.15.86:0/2830249409 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b000cdb0 con 0x7f04b810e580 2024-04-17T07:16:16.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.600+0000 7f04b4c00640 1 -- 172.21.15.86:0/2830249409 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04b0005020 con 0x7f04b810e580 2024-04-17T07:16:16.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.600+0000 7f04b4c00640 1 -- 172.21.15.86:0/2830249409 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b00116d0 con 0x7f04b810e580 2024-04-17T07:16:16.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.600+0000 7f04bec00640 1 -- 172.21.15.86:0/2830249409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b81a6740 con 0x7f04b810e580 2024-04-17T07:16:16.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.600+0000 7f04bec00640 1 -- 172.21.15.86:0/2830249409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b81a6c40 con 0x7f04b810e580 2024-04-17T07:16:16.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.601+0000 7f04bec00640 1 -- 172.21.15.86:0/2830249409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b8104750 con 0x7f04b810e580 2024-04-17T07:16:16.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.604+0000 7f04b4c00640 1 -- 172.21.15.86:0/2830249409 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f04b00051d0 con 0x7f04b810e580 2024-04-17T07:16:16.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.606+0000 7f04b4c00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f049c0a3cc0 0x7f049c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:16.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.606+0000 7f04b4c00640 1 -- 172.21.15.86:0/2830249409 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f04b00d0b40 con 0x7f04b810e580 2024-04-17T07:16:16.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.606+0000 7f04b7e00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f049c0a3cc0 0x7f049c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:16.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.607+0000 7f04b7e00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f049c0a3cc0 0x7f049c0a6180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f04a8006010 tx=0x7f04a802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:16.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.612+0000 7f04b4c00640 1 -- 172.21.15.86:0/2830249409 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f04b0016030 con 0x7f04b810e580 2024-04-17T07:16:16.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.886+0000 7f04bec00640 1 -- 172.21.15.86:0/2830249409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04b802d5d0 con 0x7f04b810e580 2024-04-17T07:16:16.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.887+0000 7f04b4c00640 1 -- 172.21.15.86:0/2830249409 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f04b009cfe0 con 0x7f04b810e580 2024-04-17T07:16:16.887 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:16.887 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:16:16.888 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:16:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 -- 172.21.15.86:0/2830249409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f049c0a3cc0 msgr2=0x7f049c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f049c0a3cc0 0x7f049c0a6180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f04a8006010 tx=0x7f04a802c800 comp rx=0 tx=0).stop 2024-04-17T07:16:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 -- 172.21.15.86:0/2830249409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04b810e580 msgr2=0x7f04b81a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:16.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04b810e580 0x7f04b81a5de0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f04b000eaa0 tx=0x7f04b000ef60 comp rx=0 tx=0).stop 2024-04-17T07:16:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 -- 172.21.15.86:0/2830249409 shutdown_connections 2024-04-17T07:16:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f049c0a3cc0 0x7f049c0a6180 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04b810e580 0x7f04b81a5de0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04b8075fa0 0x7f04b819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 --2- 172.21.15.86:0/2830249409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04b80749c0 0x7f04b819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 -- 172.21.15.86:0/2830249409 >> 172.21.15.86:0/2830249409 conn(0x7f04b80fd540 msgr2=0x7f04b8101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 -- 172.21.15.86:0/2830249409 shutdown_connections 2024-04-17T07:16:16.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:16.890+0000 7f04bec00640 1 -- 172.21.15.86:0/2830249409 wait complete. 2024-04-17T07:16:16.911 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:16:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.139+0000 7f8d4de00640 1 -- 172.21.15.86:0/2454554027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481019e0 msgr2=0x7f8d48101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.139+0000 7f8d4de00640 1 --2- 172.21.15.86:0/2454554027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481019e0 0x7f8d48101e60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009ca0 tx=0x7f8d4002c100 comp rx=0 tx=0).stop 2024-04-17T07:16:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.140+0000 7f8d4de00640 1 -- 172.21.15.86:0/2454554027 shutdown_connections 2024-04-17T07:16:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.140+0000 7f8d4de00640 1 --2- 172.21.15.86:0/2454554027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d481019e0 0x7f8d48101e60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.140+0000 7f8d4de00640 1 --2- 172.21.15.86:0/2454554027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d48101040 0x7f8d481014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.140+0000 7f8d4de00640 1 --2- 172.21.15.86:0/2454554027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d481006e0 0x7f8d48100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.140+0000 7f8d4de00640 1 -- 172.21.15.86:0/2454554027 >> 172.21.15.86:0/2454554027 conn(0x7f8d480fc300 msgr2=0x7f8d480fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.140+0000 7f8d4de00640 1 -- 172.21.15.86:0/2454554027 shutdown_connections 2024-04-17T07:16:17.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.140+0000 7f8d4de00640 1 -- 172.21.15.86:0/2454554027 wait complete. 2024-04-17T07:16:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.141+0000 7f8d4de00640 1 Processor -- start 2024-04-17T07:16:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.141+0000 7f8d4de00640 1 -- start start 2024-04-17T07:16:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.141+0000 7f8d4de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d481006e0 0x7f8d4819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d4de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d48101040 0x7f8d4819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d4de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d481019e0 0x7f8d481a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d4de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4810fbc0 con 0x7f8d48101040 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d4de00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4810fa40 con 0x7f8d481006e0 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d4de00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4810fd40 con 0x7f8d481019e0 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d48101040 0x7f8d4819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d48101040 0x7f8d4819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55312/0 (socket says 172.21.15.86:55312) 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2259574693 learned_addr learned my addr 172.21.15.86:0/2259574693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d481006e0 0x7f8d4819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d4d400640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d481019e0 0x7f8d481a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2259574693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d481019e0 msgr2=0x7f8d481a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d481019e0 0x7f8d481a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2259574693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d481006e0 msgr2=0x7f8d4819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d481006e0 0x7f8d4819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.142+0000 7f8d3fe00640 1 -- 172.21.15.86:0/2259574693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d481a1dc0 con 0x7f8d48101040 2024-04-17T07:16:17.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.143+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d48101040 0x7f8d4819a6e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000d770 tx=0x7f8d3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.143+0000 7f8d46000640 1 -- 172.21.15.86:0/2259574693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30017070 con 0x7f8d48101040 2024-04-17T07:16:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.143+0000 7f8d4de00640 1 -- 172.21.15.86:0/2259574693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d481a20b0 con 0x7f8d48101040 2024-04-17T07:16:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.143+0000 7f8d46000640 1 -- 172.21.15.86:0/2259574693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d300054d0 con 0x7f8d48101040 2024-04-17T07:16:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.143+0000 7f8d4de00640 1 -- 172.21.15.86:0/2259574693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d481a2600 con 0x7f8d48101040 2024-04-17T07:16:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.143+0000 7f8d46000640 1 -- 172.21.15.86:0/2259574693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30020060 con 0x7f8d48101040 2024-04-17T07:16:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.147+0000 7f8d46000640 1 -- 172.21.15.86:0/2259574693 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f8d30004c90 con 0x7f8d48101040 2024-04-17T07:16:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.147+0000 7f8d521d3740 1 Processor -- start 2024-04-17T07:16:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.147+0000 7f8d521d3740 1 -- start start 2024-04-17T07:16:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f83075160 0x555f83075580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f8307a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f8307acc0 0x555f8307d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d521d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f82fdb020 con 0x555f8307acc0 2024-04-17T07:16:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d521d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555f82f81460 con 0x555f83075160 2024-04-17T07:16:17.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d521d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555f82fd6fe0 con 0x555f83075b50 2024-04-17T07:16:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f8307acc0 0x555f8307d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f8307acc0 0x555f8307d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55326/0 (socket says 172.21.15.86:55326) 2024-04-17T07:16:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d3fe00640 1 -- 172.21.15.86:0/1763370940 learned_addr learned my addr 172.21.15.86:0/1763370940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/1763370940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f8307a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.148+0000 7f8d4d400640 1 --2- 172.21.15.86:0/1763370940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f83075160 0x555f83075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.149+0000 7f8d3fe00640 1 -- 172.21.15.86:0/1763370940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 msgr2=0x555f8307a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.149+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/1763370940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f8307a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.149+0000 7f8d3fe00640 1 -- 172.21.15.86:0/1763370940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f83075160 msgr2=0x555f83075580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.149+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/1763370940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f83075160 0x555f83075580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.149+0000 7f8d3fe00640 1 -- 172.21.15.86:0/1763370940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f82f89040 con 0x555f8307acc0 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.149+0000 7f8d46000640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8d2c0a3c40 0x7f8d2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.149+0000 7f8d46000640 1 -- 172.21.15.86:0/2259574693 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8d300d04f0 con 0x7f8d48101040 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.149+0000 7f8d4d400640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8d2c0a3c40 0x7f8d2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.149+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/1763370940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f8307acc0 0x555f8307d150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8d300cda60 tx=0x7f8d300d6b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d9cd5f670d9118 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.149+0000 7f8d3dc00640 1 -- 172.21.15.86:0/1763370940 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30096e60 con 0x555f8307acc0 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.150+0000 7f8d3dc00640 1 -- 172.21.15.86:0/1763370940 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d30009c30 con 0x555f8307acc0 2024-04-17T07:16:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.150+0000 7f8d3dc00640 1 -- 172.21.15.86:0/1763370940 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3009c0c0 con 0x555f8307acc0 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.150+0000 7f8d521d3740 1 -- 172.21.15.86:0/1763370940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f8307acc0 msgr2=0x555f8307d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.150+0000 7f8d521d3740 1 --2- 172.21.15.86:0/1763370940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f8307acc0 0x555f8307d150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8d300cda60 tx=0x7f8d300d6b60 comp rx=0 tx=0).stop 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.150+0000 7f8d4d400640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8d2c0a3c40 0x7f8d2c0a6100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006010 tx=0x7f8d4002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.151+0000 7f8d521d3740 1 -- 172.21.15.86:0/1763370940 shutdown_connections 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.151+0000 7f8d521d3740 1 --2- 172.21.15.86:0/1763370940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f8307acc0 0x555f8307d150 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.151+0000 7f8d521d3740 1 --2- 172.21.15.86:0/1763370940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f8307a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.151+0000 7f8d521d3740 1 --2- 172.21.15.86:0/1763370940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f83075160 0x555f83075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.151+0000 7f8d521d3740 1 -- 172.21.15.86:0/1763370940 >> 172.21.15.86:0/1763370940 conn(0x555f830706d0 msgr2=0x555f83070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.151+0000 7f8d521d3740 1 -- 172.21.15.86:0/1763370940 shutdown_connections 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.151+0000 7f8d521d3740 1 -- 172.21.15.86:0/1763370940 wait complete. 2024-04-17T07:16:17.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.151+0000 7f8d521d3740 1 -- start start 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.151+0000 7f8d521d3740 1 Processor -- start 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f8307f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f8307fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f83086c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d521d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f8307dc70 con 0x555f83075160 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d521d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555f82f94830 con 0x555f8307acc0 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d521d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555f8307ddf0 con 0x555f83075b50 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d521d3740 10 client.0 _ll_register_callbacks cb 0x555f82f62fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d4ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f8307f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d4ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f8307f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55342/0 (socket says 172.21.15.86:55342) 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d4ca00640 1 -- 172.21.15.86:0/3676503576 learned_addr learned my addr 172.21.15.86:0/3676503576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.152+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f8307fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.153+0000 7f8d4d400640 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f83086c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.153+0000 7f8d4ca00640 1 -- 172.21.15.86:0/3676503576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 msgr2=0x555f8307fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.153+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f8307fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.153+0000 7f8d4ca00640 1 -- 172.21.15.86:0/3676503576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 msgr2=0x555f83086c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.153+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f83086c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.153+0000 7f8d4ca00640 1 -- 172.21.15.86:0/3676503576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f82f98920 con 0x555f83075160 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.153+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f8307f6a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8d380107c0 tx=0x7f8d38010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9009a11b903c03bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.153+0000 7f8d36000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.153+0000 7f8d36000640 1 -- 172.21.15.86:0/3676503576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800cab0 con 0x555f83075160 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.153+0000 7f8d36000640 1 -- 172.21.15.86:0/3676503576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8d3800cc50 con 0x555f83075160 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.154+0000 7f8d521d3740 1 -- 172.21.15.86:0/3676503576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f82fe8a40 con 0x555f83075160 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.154+0000 7f8d36000640 1 -- 172.21.15.86:0/3676503576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d3800ce50 con 0x555f83075160 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.154+0000 7f8d521d3740 10 client.15981 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.154+0000 7f8d521d3740 1 -- 172.21.15.86:0/3676503576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f82f667f0 con 0x555f83075160 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.154+0000 7f8d36000640 1 -- 172.21.15.86:0/3676503576 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555f82fe8a40 con 0x555f83075160 2024-04-17T07:16:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.154+0000 7f8d36000640 1 -- 172.21.15.86:0/3676503576 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8d3801a550 con 0x555f83075160 2024-04-17T07:16:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.155+0000 7f8d521d3740 10 client.15981 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.155+0000 7f8d521d3740 10 client.15981 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:16:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.155+0000 7f8d521d3740 20 client.15981 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.155+0000 7f8d521d3740 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x555f8308b2a0 0x555f8308d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.155+0000 7f8d521d3740 4 client.15981 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.155+0000 7f8d521d3740 1 -- 172.21.15.86:0/3676503576 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f82fe28b0 con 0x555f8308b2a0 2024-04-17T07:16:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.155+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x555f8308b2a0 0x555f8308d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:200000 of 3200000 tuples (6%) done (elapsed 6.70 s, remaining 100.53 s) 2024-04-17T07:16:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.156+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x555f8308b2a0 0x555f8308d740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e79fd3edbdb242f server_cookie=eef8e81672ba6144 in_seq=0 out_seq=0 2024-04-17T07:16:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.156+0000 7f8d36000640 10 client.15981 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:16:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.158+0000 7f8d36000640 1 -- 172.21.15.86:0/3676503576 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f82fe28b0 con 0x555f8308b2a0 2024-04-17T07:16:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.158+0000 7f8d36000640 10 client.15981 handle_command_reply: tid=0 2024-04-17T07:16:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.158+0000 7f8d521d3740 1 client.15981 shutdown 2024-04-17T07:16:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 1 -- 172.21.15.86:0/3676503576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 msgr2=0x555f8307f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f8307f6a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8d380107c0 tx=0x7f8d38010c80 comp rx=0 tx=0).stop 2024-04-17T07:16:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 1 -- 172.21.15.86:0/3676503576 shutdown_connections 2024-04-17T07:16:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x555f8308b2a0 0x555f8308d740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f83086c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f8307fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 1 --2- 172.21.15.86:0/3676503576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f8307f6a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d36000640 0 client.15981 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:16:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 1 -- 172.21.15.86:0/3676503576 >> 172.21.15.86:0/3676503576 conn(0x555f830706d0 msgr2=0x555f8306f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 1 -- 172.21.15.86:0/3676503576 shutdown_connections 2024-04-17T07:16:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 1 -- 172.21.15.86:0/3676503576 wait complete. 2024-04-17T07:16:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.159+0000 7f8d521d3740 20 client.15981 trim_cache size 0 max 16384 2024-04-17T07:16:17.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d521d3740 1 Processor -- start 2024-04-17T07:16:17.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d521d3740 1 -- start start 2024-04-17T07:16:17.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f83092000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f83086be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f83092540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d521d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d481ac9c0 con 0x555f83075160 2024-04-17T07:16:17.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d521d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8d481ae500 con 0x555f8307acc0 2024-04-17T07:16:17.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d521d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d481ad090 con 0x555f83075b50 2024-04-17T07:16:17.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d4ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f83092000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d4ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f83092000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55356/0 (socket says 172.21.15.86:55356) 2024-04-17T07:16:17.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.179+0000 7f8d4ca00640 1 -- 172.21.15.86:0/2206341813 learned_addr learned my addr 172.21.15.86:0/2206341813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.180+0000 7f8d4d400640 1 --2- 172.21.15.86:0/2206341813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f83092540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.180+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/2206341813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f83086be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.180+0000 7f8d4ca00640 1 -- 172.21.15.86:0/2206341813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 msgr2=0x555f83086be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.180+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/2206341813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f83086be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.180+0000 7f8d4ca00640 1 -- 172.21.15.86:0/2206341813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 msgr2=0x555f83092540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.180+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/2206341813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f83092540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.180+0000 7f8d4ca00640 1 -- 172.21.15.86:0/2206341813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f82ced3f0 con 0x555f83075160 2024-04-17T07:16:17.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.180+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/2206341813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f83092000 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8d38000c00 tx=0x7f8d3801bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbb287b33d155bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.181+0000 7f8d3d200640 1 -- 172.21.15.86:0/2206341813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3801a630 con 0x555f83075160 2024-04-17T07:16:17.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.181+0000 7f8d3d200640 1 -- 172.21.15.86:0/2206341813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d3802d070 con 0x555f83075160 2024-04-17T07:16:17.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.181+0000 7f8d3d200640 1 -- 172.21.15.86:0/2206341813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d380035f0 con 0x555f83075160 2024-04-17T07:16:17.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.181+0000 7f8d521d3740 1 -- 172.21.15.86:0/2206341813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 msgr2=0x555f83092000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.181+0000 7f8d521d3740 1 --2- 172.21.15.86:0/2206341813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f83092000 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8d38000c00 tx=0x7f8d3801bce0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.182+0000 7f8d521d3740 1 -- 172.21.15.86:0/2206341813 shutdown_connections 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.182+0000 7f8d521d3740 1 --2- 172.21.15.86:0/2206341813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f83092540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.182+0000 7f8d521d3740 1 --2- 172.21.15.86:0/2206341813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075b50 0x555f83086be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.182+0000 7f8d521d3740 1 --2- 172.21.15.86:0/2206341813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075160 0x555f83092000 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.182+0000 7f8d521d3740 1 -- 172.21.15.86:0/2206341813 >> 172.21.15.86:0/2206341813 conn(0x555f830706d0 msgr2=0x555f830741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.182+0000 7f8d521d3740 1 -- 172.21.15.86:0/2206341813 shutdown_connections 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.182+0000 7f8d521d3740 1 -- 172.21.15.86:0/2206341813 wait complete. 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.182+0000 7f8d521d3740 1 -- start start 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.182+0000 7f8d521d3740 1 Processor -- start 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075160 0x555f83083b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075b50 0x555f830840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d521d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f830ef870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d521d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d481ade30 con 0x555f83075b50 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d521d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8d481ad760 con 0x555f8307acc0 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d521d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d481aa990 con 0x555f83075160 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d521d3740 10 client.0 _ll_register_callbacks cb 0x7f8d380103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075b50 0x555f830840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075b50 0x555f830840c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55366/0 (socket says 172.21.15.86:55366) 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d3fe00640 1 -- 172.21.15.86:0/231466544 learned_addr learned my addr 172.21.15.86:0/231466544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.183+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075160 0x555f83083b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d4d400640 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f830ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d3fe00640 1 -- 172.21.15.86:0/231466544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075160 msgr2=0x555f83083b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075160 0x555f83083b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d3fe00640 1 -- 172.21.15.86:0/231466544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 msgr2=0x555f830ef870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f830ef870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d3fe00640 1 -- 172.21.15.86:0/231466544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f82db77d0 con 0x555f83075b50 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d3fe00640 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075b50 0x555f830840c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8d300047e0 tx=0x7f8d3000de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa38689011fbad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d36000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d36000640 1 -- 172.21.15.86:0/231466544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30085070 con 0x555f83075b50 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d36000640 1 -- 172.21.15.86:0/231466544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8d300d0240 con 0x555f83075b50 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.184+0000 7f8d521d3740 1 -- 172.21.15.86:0/231466544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d3802d070 con 0x555f83075b50 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d36000640 1 -- 172.21.15.86:0/231466544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d30096e60 con 0x555f83075b50 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d521d3740 10 client.15987 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d521d3740 1 -- 172.21.15.86:0/231466544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f830f0fa0 con 0x555f83075b50 2024-04-17T07:16:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d36000640 1 -- 172.21.15.86:0/231466544 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d3802d070 con 0x555f83075b50 2024-04-17T07:16:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d36000640 1 -- 172.21.15.86:0/231466544 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8d30023020 con 0x555f83075b50 2024-04-17T07:16:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d521d3740 10 client.15987 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d521d3740 10 client.15987 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:16:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d521d3740 20 client.15987 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d521d3740 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x555f830f1340 0x555f830f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d521d3740 4 client.15987 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:16:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.185+0000 7f8d521d3740 1 -- 172.21.15.86:0/231466544 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f82f5e570 con 0x555f830f1340 2024-04-17T07:16:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.186+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x555f830f1340 0x555f830f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d36000640 10 client.15987 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d4ca00640 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x555f830f1340 0x555f830f3730 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc492f6ee0705169 server_cookie=2d8528604c27bf63 in_seq=0 out_seq=0 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d36000640 1 -- 172.21.15.86:0/231466544 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f82f5e570 con 0x555f830f1340 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d36000640 10 client.15987 handle_command_reply: tid=0 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d521d3740 1 client.15987 shutdown 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d521d3740 1 -- 172.21.15.86:0/231466544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075b50 msgr2=0x555f830840c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d521d3740 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075b50 0x555f830840c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8d300047e0 tx=0x7f8d3000de80 comp rx=0 tx=0).stop 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d521d3740 1 -- 172.21.15.86:0/231466544 shutdown_connections 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d521d3740 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x555f830f1340 0x555f830f3730 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d521d3740 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f8307acc0 0x555f830ef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d521d3740 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f83075b50 0x555f830840c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d521d3740 1 --2- 172.21.15.86:0/231466544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f83075160 0x555f83083b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d36000640 0 client.15987 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:16:17.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.187+0000 7f8d521d3740 1 -- 172.21.15.86:0/231466544 >> 172.21.15.86:0/231466544 conn(0x555f830706d0 msgr2=0x555f8306f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.188+0000 7f8d521d3740 1 -- 172.21.15.86:0/231466544 shutdown_connections 2024-04-17T07:16:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.188+0000 7f8d521d3740 1 -- 172.21.15.86:0/231466544 wait complete. 2024-04-17T07:16:17.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.188+0000 7f8d521d3740 20 client.15987 trim_cache size 0 max 16384 2024-04-17T07:16:17.191 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:16:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.188+0000 7f8d4de00640 1 -- 172.21.15.86:0/2259574693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8d2c0a3c40 msgr2=0x7f8d2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8d2c0a3c40 0x7f8d2c0a6100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006010 tx=0x7f8d4002e040 comp rx=0 tx=0).stop 2024-04-17T07:16:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 -- 172.21.15.86:0/2259574693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d48101040 msgr2=0x7f8d4819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d48101040 0x7f8d4819a6e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000d770 tx=0x7f8d3000dc30 comp rx=0 tx=0).stop 2024-04-17T07:16:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 -- 172.21.15.86:0/2259574693 shutdown_connections 2024-04-17T07:16:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8d2c0a3c40 0x7f8d2c0a6100 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d481019e0 0x7f8d481a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d48101040 0x7f8d4819a6e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 --2- 172.21.15.86:0/2259574693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d481006e0 0x7f8d4819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 -- 172.21.15.86:0/2259574693 >> 172.21.15.86:0/2259574693 conn(0x7f8d480fc300 msgr2=0x7f8d480fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 -- 172.21.15.86:0/2259574693 shutdown_connections 2024-04-17T07:16:17.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.189+0000 7f8d4de00640 1 -- 172.21.15.86:0/2259574693 wait complete. 2024-04-17T07:16:17.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:16:17.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.430+0000 7f18c1a00640 1 -- 172.21.15.86:0/1419223693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18bc109d20 msgr2=0x7f18bc10c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.430+0000 7f18c1a00640 1 --2- 172.21.15.86:0/1419223693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18bc109d20 0x7f18bc10c1c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009ca0 tx=0x7f18b402c190 comp rx=0 tx=0).stop 2024-04-17T07:16:17.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.430+0000 7f18c1a00640 1 -- 172.21.15.86:0/1419223693 shutdown_connections 2024-04-17T07:16:17.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.430+0000 7f18c1a00640 1 --2- 172.21.15.86:0/1419223693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18bc109d20 0x7f18bc10c1c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.430+0000 7f18c1a00640 1 --2- 172.21.15.86:0/1419223693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18bc074960 0x7f18bc074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.430+0000 7f18c1a00640 1 --2- 172.21.15.86:0/1419223693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc075f40 0x7f18bc074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.430+0000 7f18c1a00640 1 -- 172.21.15.86:0/1419223693 >> 172.21.15.86:0/1419223693 conn(0x7f18bc0fd400 msgr2=0x7f18bc0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.430+0000 7f18c1a00640 1 -- 172.21.15.86:0/1419223693 shutdown_connections 2024-04-17T07:16:17.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.430+0000 7f18c1a00640 1 -- 172.21.15.86:0/1419223693 wait complete. 2024-04-17T07:16:17.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.431+0000 7f18c1a00640 1 Processor -- start 2024-04-17T07:16:17.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.432+0000 7f18c1a00640 1 -- start start 2024-04-17T07:16:17.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.432+0000 7f18c1a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18bc074960 0x7f18bc19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.432+0000 7f18c1a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc075f40 0x7f18bc19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.432+0000 7f18c1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18bc109d20 0x7f18bc1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.432+0000 7f18c1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc10fbd0 con 0x7f18bc109d20 2024-04-17T07:16:17.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.432+0000 7f18c1a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc10fa50 con 0x7f18bc075f40 2024-04-17T07:16:17.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.432+0000 7f18c1a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc10fd50 con 0x7f18bc074960 2024-04-17T07:16:17.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.432+0000 7f18c1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18bc109d20 0x7f18bc1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.432+0000 7f18c1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18bc109d20 0x7f18bc1a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55390/0 (socket says 172.21.15.86:55390) 2024-04-17T07:16:17.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.432+0000 7f18c1000640 1 -- 172.21.15.86:0/2657758870 learned_addr learned my addr 172.21.15.86:0/2657758870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.433+0000 7f18bbe00640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18bc074960 0x7f18bc19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.433+0000 7f18bb400640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc075f40 0x7f18bc19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.433+0000 7f18c1000640 1 -- 172.21.15.86:0/2657758870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18bc074960 msgr2=0x7f18bc19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.433+0000 7f18c1000640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18bc074960 0x7f18bc19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.433+0000 7f18c1000640 1 -- 172.21.15.86:0/2657758870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc075f40 msgr2=0x7f18bc19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.433+0000 7f18c1000640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc075f40 0x7f18bc19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.433+0000 7f18c1000640 1 -- 172.21.15.86:0/2657758870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc1a1dc0 con 0x7f18bc109d20 2024-04-17T07:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.433+0000 7f18bbe00640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18bc074960 0x7f18bc19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.433+0000 7f18c1000640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18bc109d20 0x7f18bc1a1750 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f18b402c6d0 tx=0x7f18b4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.434+0000 7f18b8c00640 1 -- 172.21.15.86:0/2657758870 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4039040 con 0x7f18bc109d20 2024-04-17T07:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.434+0000 7f18b8c00640 1 -- 172.21.15.86:0/2657758870 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18b4044070 con 0x7f18bc109d20 2024-04-17T07:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.434+0000 7f18c1a00640 1 -- 172.21.15.86:0/2657758870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc1a2050 con 0x7f18bc109d20 2024-04-17T07:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.434+0000 7f18b8c00640 1 -- 172.21.15.86:0/2657758870 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b40044d0 con 0x7f18bc109d20 2024-04-17T07:16:17.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.434+0000 7f18c1a00640 1 -- 172.21.15.86:0/2657758870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc1a2540 con 0x7f18bc109d20 2024-04-17T07:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.438+0000 7f18c5def740 1 Processor -- start 2024-04-17T07:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.438+0000 7f18b8c00640 1 -- 172.21.15.86:0/2657758870 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f18b4032ca0 con 0x7f18bc109d20 2024-04-17T07:16:17.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.438+0000 7f18c5def740 1 -- start start 2024-04-17T07:16:17.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.438+0000 7f18c5def740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a26e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.438+0000 7f18c5def740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a273650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.438+0000 7f18c5def740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a276150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.438+0000 7f18c5def740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca7a1d4020 con 0x55ca7a26eb50 2024-04-17T07:16:17.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.438+0000 7f18c5def740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ca7a17a460 con 0x55ca7a273cc0 2024-04-17T07:16:17.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.438+0000 7f18c5def740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca7a1cffe0 con 0x55ca7a26e160 2024-04-17T07:16:17.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a273650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a273650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55398/0 (socket says 172.21.15.86:55398) 2024-04-17T07:16:17.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bbe00640 1 -- 172.21.15.86:0/4006993858 learned_addr learned my addr 172.21.15.86:0/4006993858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bb400640 1 --2- 172.21.15.86:0/4006993858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a26e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bbe00640 1 -- 172.21.15.86:0/4006993858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 msgr2=0x55ca7a26e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bbe00640 1 --2- 172.21.15.86:0/4006993858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a26e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bb400640 1 --2- 172.21.15.86:0/4006993858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a276150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bbe00640 1 -- 172.21.15.86:0/4006993858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 msgr2=0x55ca7a276150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bbe00640 1 --2- 172.21.15.86:0/4006993858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a276150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bbe00640 1 -- 172.21.15.86:0/4006993858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7a182040 con 0x55ca7a26eb50 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18bb400640 1 --2- 172.21.15.86:0/4006993858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a276150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.439+0000 7f18b8c00640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f18a00a3c40 0x7f18a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.440+0000 7f18b8c00640 1 -- 172.21.15.86:0/2657758870 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f18b40f3120 con 0x7f18bc109d20 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.440+0000 7f18bbe00640 1 --2- 172.21.15.86:0/4006993858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a273650 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f18ac008c60 tx=0x7f18ac00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50555b08e66dd64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.440+0000 7f18bb400640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f18a00a3c40 0x7f18a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.440+0000 7f18b1600640 1 -- 172.21.15.86:0/4006993858 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac014070 con 0x55ca7a26eb50 2024-04-17T07:16:17.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.440+0000 7f18b1600640 1 -- 172.21.15.86:0/4006993858 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18ac0047a0 con 0x55ca7a26eb50 2024-04-17T07:16:17.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.440+0000 7f18b1600640 1 -- 172.21.15.86:0/4006993858 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac010630 con 0x55ca7a26eb50 2024-04-17T07:16:17.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.441+0000 7f18c5def740 1 -- 172.21.15.86:0/4006993858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 msgr2=0x55ca7a273650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.441+0000 7f18bb400640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f18a00a3c40 0x7f18a00a6100 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55ca7a26f3b0 tx=0x7f18a8012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.441+0000 7f18c5def740 1 --2- 172.21.15.86:0/4006993858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a273650 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f18ac008c60 tx=0x7f18ac00e720 comp rx=0 tx=0).stop 2024-04-17T07:16:17.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.441+0000 7f18c5def740 1 -- 172.21.15.86:0/4006993858 shutdown_connections 2024-04-17T07:16:17.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.441+0000 7f18c5def740 1 --2- 172.21.15.86:0/4006993858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a276150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.441+0000 7f18c5def740 1 --2- 172.21.15.86:0/4006993858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a273650 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.441+0000 7f18c5def740 1 --2- 172.21.15.86:0/4006993858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a26e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.441+0000 7f18c5def740 1 -- 172.21.15.86:0/4006993858 >> 172.21.15.86:0/4006993858 conn(0x55ca7a2696d0 msgr2=0x55ca7a269ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.441+0000 7f18c5def740 1 -- 172.21.15.86:0/4006993858 shutdown_connections 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.441+0000 7f18c5def740 1 -- 172.21.15.86:0/4006993858 wait complete. 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.442+0000 7f18c5def740 1 -- start start 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.442+0000 7f18c5def740 1 Processor -- start 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.442+0000 7f18c5def740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a26e160 0x55ca7a2786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.442+0000 7f18c5def740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a278c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18c5def740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a273cc0 0x55ca7a27fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18c5def740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca7a276c70 con 0x55ca7a26eb50 2024-04-17T07:16:17.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18c5def740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ca7a18d830 con 0x55ca7a26e160 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18c5def740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca7a276df0 con 0x55ca7a273cc0 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18c5def740 10 client.0 _ll_register_callbacks cb 0x55ca7a15bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a278c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a278c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55404/0 (socket says 172.21.15.86:55404) 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18bb400640 1 -- 172.21.15.86:0/1661141717 learned_addr learned my addr 172.21.15.86:0/1661141717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18c1000640 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a273cc0 0x55ca7a27fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18bbe00640 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a26e160 0x55ca7a2786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18bb400640 1 -- 172.21.15.86:0/1661141717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a273cc0 msgr2=0x55ca7a27fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18bb400640 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a273cc0 0x55ca7a27fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18bb400640 1 -- 172.21.15.86:0/1661141717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a26e160 msgr2=0x55ca7a2786e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18bb400640 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a26e160 0x55ca7a2786e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.443+0000 7f18bb400640 1 -- 172.21.15.86:0/1661141717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7a191920 con 0x55ca7a26eb50 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.444+0000 7f18bb400640 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a278c20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f18a800fd10 tx=0x7f18a800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbacb00ede0f3e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.444+0000 7f18a6000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.444+0000 7f18a6000640 1 -- 172.21.15.86:0/1661141717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a801ce20 con 0x55ca7a26eb50 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.444+0000 7f18a6000640 1 -- 172.21.15.86:0/1661141717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f18a801a400 con 0x55ca7a26eb50 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.444+0000 7f18c5def740 1 -- 172.21.15.86:0/1661141717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18ac0047a0 con 0x55ca7a26eb50 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.444+0000 7f18a6000640 1 -- 172.21.15.86:0/1661141717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f18a8031050 con 0x55ca7a26eb50 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.445+0000 7f18c5def740 10 client.15999 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.445+0000 7f18c5def740 1 -- 172.21.15.86:0/1661141717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca7a15f7f0 con 0x55ca7a26eb50 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.445+0000 7f18a6000640 1 -- 172.21.15.86:0/1661141717 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18ac0047a0 con 0x55ca7a26eb50 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.445+0000 7f18a6000640 1 -- 172.21.15.86:0/1661141717 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f18a8026650 con 0x55ca7a26eb50 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.445+0000 7f18c5def740 10 client.15999 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.445+0000 7f18c5def740 10 client.15999 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.445+0000 7f18c5def740 20 client.15999 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.445+0000 7f18c5def740 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca7a284170 0x55ca7a286630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.445+0000 7f18c5def740 4 client.15999 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:17.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.445+0000 7f18c5def740 1 -- 172.21.15.86:0/1661141717 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca7a1e1a40 con 0x55ca7a284170 2024-04-17T07:16:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.446+0000 7f18bbe00640 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca7a284170 0x55ca7a286630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.447+0000 7f18bbe00640 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca7a284170 0x55ca7a286630 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83ea2e56f76a3d01 server_cookie=afdb847faa444cee in_seq=0 out_seq=0 2024-04-17T07:16:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.447+0000 7f18a6000640 10 client.15999 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:16:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18a6000640 1 -- 172.21.15.86:0/1661141717 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca7a1e1a40 con 0x55ca7a284170 2024-04-17T07:16:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18a6000640 10 client.15999 handle_command_reply: tid=0 2024-04-17T07:16:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18c5def740 1 client.15999 shutdown 2024-04-17T07:16:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18c5def740 1 -- 172.21.15.86:0/1661141717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 msgr2=0x55ca7a278c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18c5def740 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a278c20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f18a800fd10 tx=0x7f18a800a250 comp rx=0 tx=0).stop 2024-04-17T07:16:17.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18c5def740 1 -- 172.21.15.86:0/1661141717 shutdown_connections 2024-04-17T07:16:17.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18c5def740 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca7a284170 0x55ca7a286630 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18c5def740 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a273cc0 0x55ca7a27fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18c5def740 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a278c20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18c5def740 1 --2- 172.21.15.86:0/1661141717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a26e160 0x55ca7a2786e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18a6000640 0 client.15999 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:16:17.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.449+0000 7f18c5def740 1 -- 172.21.15.86:0/1661141717 >> 172.21.15.86:0/1661141717 conn(0x55ca7a2696d0 msgr2=0x55ca7a268930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.450+0000 7f18c5def740 1 -- 172.21.15.86:0/1661141717 shutdown_connections 2024-04-17T07:16:17.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.450+0000 7f18c5def740 1 -- 172.21.15.86:0/1661141717 wait complete. 2024-04-17T07:16:17.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.450+0000 7f18c5def740 20 client.15999 trim_cache size 0 max 16384 2024-04-17T07:16:17.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.468+0000 7f18c5def740 1 Processor -- start 2024-04-17T07:16:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.468+0000 7f18c5def740 1 -- start start 2024-04-17T07:16:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.468+0000 7f18c5def740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a2687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.468+0000 7f18c5def740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.468+0000 7f18c5def740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a28aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.468+0000 7f18c5def740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1aca60 con 0x55ca7a26eb50 2024-04-17T07:16:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.468+0000 7f18c5def740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1ae5a0 con 0x55ca7a273cc0 2024-04-17T07:16:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.468+0000 7f18c5def740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1ad130 con 0x55ca7a26e160 2024-04-17T07:16:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55406/0 (socket says 172.21.15.86:55406) 2024-04-17T07:16:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18bb400640 1 -- 172.21.15.86:0/3188901141 learned_addr learned my addr 172.21.15.86:0/3188901141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18bbe00640 1 --2- 172.21.15.86:0/3188901141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a2687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18bb400640 1 -- 172.21.15.86:0/3188901141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 msgr2=0x55ca7a2687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18bb400640 1 --2- 172.21.15.86:0/3188901141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a2687b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18bb400640 1 -- 172.21.15.86:0/3188901141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 msgr2=0x55ca7a28aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18c1000640 1 --2- 172.21.15.86:0/3188901141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a28aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18bb400640 1 --2- 172.21.15.86:0/3188901141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a28aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18bb400640 1 -- 172.21.15.86:0/3188901141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca79ee63f0 con 0x55ca7a26eb50 2024-04-17T07:16:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.469+0000 7f18bb400640 1 --2- 172.21.15.86:0/3188901141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27fc20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f18a800afc0 tx=0x7f18a8010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925e783826d5c9de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.470+0000 7f18b0c00640 1 -- 172.21.15.86:0/3188901141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8039070 con 0x55ca7a26eb50 2024-04-17T07:16:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.470+0000 7f18b0c00640 1 -- 172.21.15.86:0/3188901141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18a80268a0 con 0x55ca7a26eb50 2024-04-17T07:16:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.470+0000 7f18b0c00640 1 -- 172.21.15.86:0/3188901141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8004a00 con 0x55ca7a26eb50 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.471+0000 7f18c5def740 1 -- 172.21.15.86:0/3188901141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 msgr2=0x55ca7a27fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.471+0000 7f18c5def740 1 --2- 172.21.15.86:0/3188901141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27fc20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f18a800afc0 tx=0x7f18a8010ce0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.471+0000 7f18c5def740 1 -- 172.21.15.86:0/3188901141 shutdown_connections 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.471+0000 7f18c5def740 1 --2- 172.21.15.86:0/3188901141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a28aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.471+0000 7f18c5def740 1 --2- 172.21.15.86:0/3188901141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27fc20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.471+0000 7f18c5def740 1 --2- 172.21.15.86:0/3188901141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a2687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.471+0000 7f18c5def740 1 -- 172.21.15.86:0/3188901141 >> 172.21.15.86:0/3188901141 conn(0x55ca7a2696d0 msgr2=0x55ca7a2680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.471+0000 7f18c5def740 1 -- 172.21.15.86:0/3188901141 shutdown_connections 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.471+0000 7f18c5def740 1 -- 172.21.15.86:0/3188901141 wait complete. 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.471+0000 7f18c5def740 1 -- start start 2024-04-17T07:16:17.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18c5def740 1 Processor -- start 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18c5def740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a27d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18c5def740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18c5def740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a27e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18c5def740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1aded0 con 0x55ca7a26eb50 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18c5def740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1ad800 con 0x55ca7a273cc0 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18c5def740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1aaa30 con 0x55ca7a26e160 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18c5def740 10 client.0 _ll_register_callbacks cb 0x7f18a800fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55408/0 (socket says 172.21.15.86:55408) 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.472+0000 7f18bb400640 1 -- 172.21.15.86:0/3778141538 learned_addr learned my addr 172.21.15.86:0/3778141538 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18bbe00640 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a27d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18c1000640 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a27e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18bb400640 1 -- 172.21.15.86:0/3778141538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 msgr2=0x55ca7a27d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18bb400640 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a27d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18bb400640 1 -- 172.21.15.86:0/3778141538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 msgr2=0x55ca7a27e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18bb400640 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a27e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18bb400640 1 -- 172.21.15.86:0/3778141538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca79fb07d0 con 0x55ca7a26eb50 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18c1000640 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a27e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18bb400640 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27d850 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f18a8010e00 tx=0x7f18a8005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98c43ba6937d585 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18a6000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.473+0000 7f18a6000640 1 -- 172.21.15.86:0/3778141538 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8043020 con 0x55ca7a26eb50 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18c5def740 1 -- 172.21.15.86:0/3778141538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a80268a0 con 0x55ca7a26eb50 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18a6000640 1 -- 172.21.15.86:0/3778141538 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f18a80047a0 con 0x55ca7a26eb50 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18a6000640 1 -- 172.21.15.86:0/3778141538 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f18a804e050 con 0x55ca7a26eb50 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18c5def740 10 client.16005 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18c5def740 1 -- 172.21.15.86:0/3778141538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca7a27ff30 con 0x55ca7a26eb50 2024-04-17T07:16:17.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18a6000640 1 -- 172.21.15.86:0/3778141538 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18a80268a0 con 0x55ca7a26eb50 2024-04-17T07:16:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18a6000640 1 -- 172.21.15.86:0/3778141538 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f18a8035a60 con 0x55ca7a26eb50 2024-04-17T07:16:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18c5def740 10 client.16005 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18c5def740 10 client.16005 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:16:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18c5def740 20 client.16005 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:17.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18c5def740 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca7a2eae80 0x55ca7a2ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18c5def740 4 client.16005 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.474+0000 7f18c5def740 1 -- 172.21.15.86:0/3778141538 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca7a1db8b0 con 0x55ca7a2eae80 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.475+0000 7f18bbe00640 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca7a2eae80 0x55ca7a2ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.476+0000 7f18bbe00640 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca7a2eae80 0x55ca7a2ed2a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93a860d5583ba138 server_cookie=9cfd2a26edc290d9 in_seq=0 out_seq=0 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.476+0000 7f18a6000640 10 client.16005 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18a6000640 1 -- 172.21.15.86:0/3778141538 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca7a1db8b0 con 0x55ca7a2eae80 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18a6000640 10 client.16005 handle_command_reply: tid=0 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 client.16005 shutdown 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 -- 172.21.15.86:0/3778141538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 msgr2=0x55ca7a27d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27d850 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f18a8010e00 tx=0x7f18a8005250 comp rx=0 tx=0).stop 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 -- 172.21.15.86:0/3778141538 shutdown_connections 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca7a2eae80 0x55ca7a2ed2a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca7a273cc0 0x55ca7a27e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca7a26eb50 0x55ca7a27d850 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 --2- 172.21.15.86:0/3778141538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca7a26e160 0x55ca7a27d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 -- 172.21.15.86:0/3778141538 >> 172.21.15.86:0/3778141538 conn(0x55ca7a2696d0 msgr2=0x55ca7a268930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18a6000640 0 client.16005 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:16:17.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 -- 172.21.15.86:0/3778141538 shutdown_connections 2024-04-17T07:16:17.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 1 -- 172.21.15.86:0/3778141538 wait complete. 2024-04-17T07:16:17.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.477+0000 7f18c5def740 20 client.16005 trim_cache size 0 max 16384 2024-04-17T07:16:17.480 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 -- 172.21.15.86:0/2657758870 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f18a00a3c40 msgr2=0x7f18a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f18a00a3c40 0x7f18a00a6100 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55ca7a26f3b0 tx=0x7f18a8012040 comp rx=0 tx=0).stop 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 -- 172.21.15.86:0/2657758870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18bc109d20 msgr2=0x7f18bc1a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18bc109d20 0x7f18bc1a1750 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f18b402c6d0 tx=0x7f18b4003ba0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 -- 172.21.15.86:0/2657758870 shutdown_connections 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f18a00a3c40 0x7f18a00a6100 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18bc109d20 0x7f18bc1a1750 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc075f40 0x7f18bc19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 --2- 172.21.15.86:0/2657758870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18bc074960 0x7f18bc19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 -- 172.21.15.86:0/2657758870 >> 172.21.15.86:0/2657758870 conn(0x7f18bc0fd400 msgr2=0x7f18bc105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 -- 172.21.15.86:0/2657758870 shutdown_connections 2024-04-17T07:16:17.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.478+0000 7f18c1a00640 1 -- 172.21.15.86:0/2657758870 wait complete. 2024-04-17T07:16:17.492 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:17 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2830249409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:16:17.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:16:17.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:17 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2830249409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:16:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:17 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2830249409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:16:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.709+0000 7f2e8e000640 1 -- 172.21.15.86:0/2551800189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e881019e0 msgr2=0x7f2e88101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.709+0000 7f2e8e000640 1 --2- 172.21.15.86:0/2551800189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e881019e0 0x7f2e88101e60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009ca0 tx=0x7f2e8002c100 comp rx=0 tx=0).stop 2024-04-17T07:16:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.710+0000 7f2e8e000640 1 -- 172.21.15.86:0/2551800189 shutdown_connections 2024-04-17T07:16:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.710+0000 7f2e8e000640 1 --2- 172.21.15.86:0/2551800189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e881019e0 0x7f2e88101e60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.710+0000 7f2e8e000640 1 --2- 172.21.15.86:0/2551800189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e88101040 0x7f2e881014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.710+0000 7f2e8e000640 1 --2- 172.21.15.86:0/2551800189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e881006e0 0x7f2e88100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.710+0000 7f2e8e000640 1 -- 172.21.15.86:0/2551800189 >> 172.21.15.86:0/2551800189 conn(0x7f2e880fc300 msgr2=0x7f2e880fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.710+0000 7f2e8e000640 1 -- 172.21.15.86:0/2551800189 shutdown_connections 2024-04-17T07:16:17.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.710+0000 7f2e8e000640 1 -- 172.21.15.86:0/2551800189 wait complete. 2024-04-17T07:16:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.711+0000 7f2e8e000640 1 Processor -- start 2024-04-17T07:16:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.711+0000 7f2e8e000640 1 -- start start 2024-04-17T07:16:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.711+0000 7f2e8e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e881006e0 0x7f2e8819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.711+0000 7f2e8e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e88101040 0x7f2e8819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.711+0000 7f2e8e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e881019e0 0x7f2e881a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.711+0000 7f2e8e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8810fbc0 con 0x7f2e881006e0 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.711+0000 7f2e8e000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8810fa40 con 0x7f2e88101040 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.711+0000 7f2e8e000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8810fd40 con 0x7f2e881019e0 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.712+0000 7f2e8cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e881006e0 0x7f2e8819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.712+0000 7f2e8cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e881006e0 0x7f2e8819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55430/0 (socket says 172.21.15.86:55430) 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.712+0000 7f2e8cc00640 1 -- 172.21.15.86:0/876639707 learned_addr learned my addr 172.21.15.86:0/876639707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.712+0000 7f2e87e00640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e88101040 0x7f2e8819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.712+0000 7f2e8cc00640 1 -- 172.21.15.86:0/876639707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e881019e0 msgr2=0x7f2e881a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.712+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e881019e0 0x7f2e881a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.712+0000 7f2e8cc00640 1 -- 172.21.15.86:0/876639707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e88101040 msgr2=0x7f2e8819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.712+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e88101040 0x7f2e8819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.712+0000 7f2e8cc00640 1 -- 172.21.15.86:0/876639707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e881a1e50 con 0x7f2e881006e0 2024-04-17T07:16:17.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.712+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e881006e0 0x7f2e8819a230 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00d950 tx=0x7f2e7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.713+0000 7f2e85600640 1 -- 172.21.15.86:0/876639707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c017030 con 0x7f2e881006e0 2024-04-17T07:16:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.713+0000 7f2e85600640 1 -- 172.21.15.86:0/876639707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7c004020 con 0x7f2e881006e0 2024-04-17T07:16:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.713+0000 7f2e85600640 1 -- 172.21.15.86:0/876639707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c0047a0 con 0x7f2e881006e0 2024-04-17T07:16:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.713+0000 7f2e8e000640 1 -- 172.21.15.86:0/876639707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e881a2140 con 0x7f2e881006e0 2024-04-17T07:16:17.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.713+0000 7f2e8e000640 1 -- 172.21.15.86:0/876639707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881a2640 con 0x7f2e881006e0 2024-04-17T07:16:17.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.716+0000 7f2e85600640 1 -- 172.21.15.86:0/876639707 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f2e7c0041d0 con 0x7f2e881006e0 2024-04-17T07:16:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.717+0000 7f2e923ad740 1 Processor -- start 2024-04-17T07:16:17.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.717+0000 7f2e923ad740 1 -- start start 2024-04-17T07:16:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.717+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9d160 0x5620bac9d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620bac9db50 0x5620baca2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620baca2cc0 0x5620baca5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e923ad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5620bac03020 con 0x5620bac9d160 2024-04-17T07:16:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e923ad740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5620baba9460 con 0x5620baca2cc0 2024-04-17T07:16:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e923ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5620babfefe0 con 0x5620bac9db50 2024-04-17T07:16:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e8d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9d160 0x5620bac9d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e8d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9d160 0x5620bac9d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55440/0 (socket says 172.21.15.86:55440) 2024-04-17T07:16:17.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e8d600640 1 -- 172.21.15.86:0/1660484598 learned_addr learned my addr 172.21.15.86:0/1660484598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e85600640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cc0 0x7f2e700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/1660484598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620bac9db50 0x5620baca2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e8d600640 1 -- 172.21.15.86:0/1660484598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620bac9db50 msgr2=0x5620baca2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e8d600640 1 --2- 172.21.15.86:0/1660484598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620bac9db50 0x5620baca2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e8d600640 1 -- 172.21.15.86:0/1660484598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620baca2cc0 msgr2=0x5620baca5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e8d600640 1 --2- 172.21.15.86:0/1660484598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620baca2cc0 0x5620baca5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e85600640 1 -- 172.21.15.86:0/876639707 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2e7c0d04e0 con 0x7f2e881006e0 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.718+0000 7f2e8d600640 1 -- 172.21.15.86:0/1660484598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620babb1040 con 0x5620bac9d160 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.719+0000 7f2e8d600640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cc0 0x7f2e700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.719+0000 7f2e8d600640 1 --2- 172.21.15.86:0/1660484598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9d160 0x5620bac9d580 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2e80005e30 tx=0x7f2e80009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f58f2a5d3e2e0db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.719+0000 7f2e79c00640 1 -- 172.21.15.86:0/1660484598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80043040 con 0x5620bac9d160 2024-04-17T07:16:17.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.720+0000 7f2e79c00640 1 -- 172.21.15.86:0/1660484598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e80003c30 con 0x5620bac9d160 2024-04-17T07:16:17.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.720+0000 7f2e79c00640 1 -- 172.21.15.86:0/1660484598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80039040 con 0x5620bac9d160 2024-04-17T07:16:17.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.720+0000 7f2e923ad740 1 -- 172.21.15.86:0/1660484598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9d160 msgr2=0x5620bac9d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.720+0000 7f2e923ad740 1 --2- 172.21.15.86:0/1660484598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9d160 0x5620bac9d580 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2e80005e30 tx=0x7f2e80009850 comp rx=0 tx=0).stop 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.720+0000 7f2e8d600640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cc0 0x7f2e700a6180 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2e80004af0 tx=0x7f2e80046000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.721+0000 7f2e923ad740 1 -- 172.21.15.86:0/1660484598 shutdown_connections 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.721+0000 7f2e923ad740 1 --2- 172.21.15.86:0/1660484598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620baca2cc0 0x5620baca5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.721+0000 7f2e923ad740 1 --2- 172.21.15.86:0/1660484598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620bac9db50 0x5620baca2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.721+0000 7f2e923ad740 1 --2- 172.21.15.86:0/1660484598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9d160 0x5620bac9d580 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.721+0000 7f2e923ad740 1 -- 172.21.15.86:0/1660484598 >> 172.21.15.86:0/1660484598 conn(0x5620bac986d0 msgr2=0x5620bac98ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.721+0000 7f2e923ad740 1 -- 172.21.15.86:0/1660484598 shutdown_connections 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.721+0000 7f2e923ad740 1 -- 172.21.15.86:0/1660484598 wait complete. 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.721+0000 7f2e923ad740 1 -- start start 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.721+0000 7f2e923ad740 1 Processor -- start 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.722+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620bac9d160 0x5620baca76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.722+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620baca7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.722+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620baca2cc0 0x5620bacaec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.722+0000 7f2e923ad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5620baca5c70 con 0x5620bac9db50 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.722+0000 7f2e923ad740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5620babbc830 con 0x5620baca2cc0 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.722+0000 7f2e923ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5620baca5df0 con 0x5620bac9d160 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.722+0000 7f2e923ad740 10 client.0 _ll_register_callbacks cb 0x5620bab8afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.722+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620baca7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.722+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620baca7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55444/0 (socket says 172.21.15.86:55444) 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.722+0000 7f2e87e00640 1 -- 172.21.15.86:0/2872474985 learned_addr learned my addr 172.21.15.86:0/2872474985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620bac9d160 0x5620baca76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e8d600640 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620baca2cc0 0x5620bacaec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e87e00640 1 -- 172.21.15.86:0/2872474985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620bac9d160 msgr2=0x5620baca76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e87e00640 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620bac9d160 0x5620baca76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e87e00640 1 -- 172.21.15.86:0/2872474985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620baca2cc0 msgr2=0x5620bacaec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e87e00640 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620baca2cc0 0x5620bacaec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e87e00640 1 -- 172.21.15.86:0/2872474985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620babc0920 con 0x5620bac9db50 2024-04-17T07:16:17.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e8d600640 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620baca2cc0 0x5620bacaec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:17.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e87e00640 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620baca7c10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2e74004700 tx=0x7f2e74010850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b758d6b7777738 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e6e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e6e000640 1 -- 172.21.15.86:0/2872474985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74012860 con 0x5620bac9db50 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e6e000640 1 -- 172.21.15.86:0/2872474985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2e74012a00 con 0x5620bac9db50 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.723+0000 7f2e923ad740 1 -- 172.21.15.86:0/2872474985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e80003c30 con 0x5620bac9db50 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.724+0000 7f2e6e000640 1 -- 172.21.15.86:0/2872474985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e74024050 con 0x5620bac9db50 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.724+0000 7f2e923ad740 10 client.16017 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.724+0000 7f2e923ad740 1 -- 172.21.15.86:0/2872474985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620bab8e7f0 con 0x5620bac9db50 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.724+0000 7f2e6e000640 1 -- 172.21.15.86:0/2872474985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e80003c30 con 0x5620bac9db50 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.724+0000 7f2e6e000640 1 -- 172.21.15.86:0/2872474985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2e7400f040 con 0x5620bac9db50 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.724+0000 7f2e923ad740 10 client.16017 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.724+0000 7f2e923ad740 10 client.16017 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.725+0000 7f2e923ad740 20 client.16017 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:17.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.725+0000 7f2e923ad740 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5620bacb32a0 0x5620bacb5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.725+0000 7f2e923ad740 4 client.16017 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:17.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.725+0000 7f2e923ad740 1 -- 172.21.15.86:0/2872474985 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620bac10a40 con 0x5620bacb32a0 2024-04-17T07:16:17.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.725+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5620bacb32a0 0x5620bacb5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.726+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5620bacb32a0 0x5620bacb5700 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=690d0e6d0c29a682 server_cookie=9e7719c1409de486 in_seq=0 out_seq=0 2024-04-17T07:16:17.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.726+0000 7f2e6e000640 10 client.16017 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:16:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e6e000640 1 -- 172.21.15.86:0/2872474985 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620bac10a40 con 0x5620bacb32a0 2024-04-17T07:16:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e6e000640 10 client.16017 handle_command_reply: tid=0 2024-04-17T07:16:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e923ad740 1 client.16017 shutdown 2024-04-17T07:16:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e923ad740 1 -- 172.21.15.86:0/2872474985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 msgr2=0x5620baca7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e923ad740 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620baca7c10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2e74004700 tx=0x7f2e74010850 comp rx=0 tx=0).stop 2024-04-17T07:16:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e923ad740 1 -- 172.21.15.86:0/2872474985 shutdown_connections 2024-04-17T07:16:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e923ad740 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5620bacb32a0 0x5620bacb5700 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e923ad740 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620baca2cc0 0x5620bacaec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e923ad740 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620baca7c10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e6e000640 0 client.16017 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:16:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.728+0000 7f2e923ad740 1 --2- 172.21.15.86:0/2872474985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620bac9d160 0x5620baca76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.729+0000 7f2e923ad740 1 -- 172.21.15.86:0/2872474985 >> 172.21.15.86:0/2872474985 conn(0x5620bac986d0 msgr2=0x5620bac97930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.729+0000 7f2e923ad740 1 -- 172.21.15.86:0/2872474985 shutdown_connections 2024-04-17T07:16:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.729+0000 7f2e923ad740 1 -- 172.21.15.86:0/2872474985 wait complete. 2024-04-17T07:16:17.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.729+0000 7f2e923ad740 20 client.16017 trim_cache size 0 max 16384 2024-04-17T07:16:17.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.748+0000 7f2e923ad740 1 Processor -- start 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.748+0000 7f2e923ad740 1 -- start start 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bac9d160 0x5620bac970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacaec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620baca2cc0 0x5620bac975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e923ad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881aca80 con 0x5620bac9db50 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e923ad740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881ae5c0 con 0x5620bac9d160 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e923ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881ad150 con 0x5620baca2cc0 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacaec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacaec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55446/0 (socket says 172.21.15.86:55446) 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e87e00640 1 -- 172.21.15.86:0/438126191 learned_addr learned my addr 172.21.15.86:0/438126191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e8d600640 1 --2- 172.21.15.86:0/438126191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620baca2cc0 0x5620bac975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.749+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/438126191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bac9d160 0x5620bac970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.750+0000 7f2e87e00640 1 -- 172.21.15.86:0/438126191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620baca2cc0 msgr2=0x5620bac975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.750+0000 7f2e87e00640 1 --2- 172.21.15.86:0/438126191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620baca2cc0 0x5620bac975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.750+0000 7f2e87e00640 1 -- 172.21.15.86:0/438126191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bac9d160 msgr2=0x5620bac970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.750+0000 7f2e87e00640 1 --2- 172.21.15.86:0/438126191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bac9d160 0x5620bac970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.750+0000 7f2e87e00640 1 -- 172.21.15.86:0/438126191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ba9153f0 con 0x5620bac9db50 2024-04-17T07:16:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.750+0000 7f2e87e00640 1 --2- 172.21.15.86:0/438126191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacaec10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2e740046d0 tx=0x7f2e74011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6e3343a8191805 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.750+0000 7f2e79200640 1 -- 172.21.15.86:0/438126191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7402c070 con 0x5620bac9db50 2024-04-17T07:16:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.750+0000 7f2e79200640 1 -- 172.21.15.86:0/438126191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e74011dd0 con 0x5620bac9db50 2024-04-17T07:16:17.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.750+0000 7f2e79200640 1 -- 172.21.15.86:0/438126191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74021040 con 0x5620bac9db50 2024-04-17T07:16:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.751+0000 7f2e923ad740 1 -- 172.21.15.86:0/438126191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 msgr2=0x5620bacaec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.751+0000 7f2e923ad740 1 --2- 172.21.15.86:0/438126191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacaec10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2e740046d0 tx=0x7f2e74011890 comp rx=0 tx=0).stop 2024-04-17T07:16:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.751+0000 7f2e923ad740 1 -- 172.21.15.86:0/438126191 shutdown_connections 2024-04-17T07:16:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.751+0000 7f2e923ad740 1 --2- 172.21.15.86:0/438126191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620baca2cc0 0x5620bac975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.751+0000 7f2e923ad740 1 --2- 172.21.15.86:0/438126191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacaec10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.751+0000 7f2e923ad740 1 --2- 172.21.15.86:0/438126191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bac9d160 0x5620bac970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.751+0000 7f2e923ad740 1 -- 172.21.15.86:0/438126191 >> 172.21.15.86:0/438126191 conn(0x5620bac986d0 msgr2=0x5620bac9f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.751+0000 7f2e923ad740 1 -- 172.21.15.86:0/438126191 shutdown_connections 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.752+0000 7f2e923ad740 1 -- 172.21.15.86:0/438126191 wait complete. 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.752+0000 7f2e923ad740 1 -- start start 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.752+0000 7f2e923ad740 1 Processor -- start 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bac9d160 0x5620bacac070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e923ad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620baca2cc0 0x5620bacad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e923ad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881adef0 con 0x5620bac9db50 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e923ad740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881ad820 con 0x5620bac9d160 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e923ad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881aaa50 con 0x5620baca2cc0 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e923ad740 10 client.0 _ll_register_callbacks cb 0x7f2e74009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacac5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55458/0 (socket says 172.21.15.86:55458) 2024-04-17T07:16:17.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e87e00640 1 -- 172.21.15.86:0/384411892 learned_addr learned my addr 172.21.15.86:0/384411892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bac9d160 0x5620bacac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.753+0000 7f2e8d600640 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620baca2cc0 0x5620bacad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e87e00640 1 -- 172.21.15.86:0/384411892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620baca2cc0 msgr2=0x5620bacad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e87e00640 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620baca2cc0 0x5620bacad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e87e00640 1 -- 172.21.15.86:0/384411892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bac9d160 msgr2=0x5620bacac070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e87e00640 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bac9d160 0x5620bacac070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e87e00640 1 -- 172.21.15.86:0/384411892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ba9df7d0 con 0x5620bac9db50 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e87e00640 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacac5b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2e74011400 tx=0x7f2e74011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cce55734983038 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e6e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e6e000640 1 -- 172.21.15.86:0/384411892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7402c070 con 0x5620bac9db50 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e6e000640 1 -- 172.21.15.86:0/384411892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2e74005420 con 0x5620bac9db50 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e923ad740 1 -- 172.21.15.86:0/384411892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e74011dd0 con 0x5620bac9db50 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e6e000640 1 -- 172.21.15.86:0/384411892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e74041050 con 0x5620bac9db50 2024-04-17T07:16:17.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e923ad740 10 client.16023 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:17.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.754+0000 7f2e923ad740 1 -- 172.21.15.86:0/384411892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620bacaece0 con 0x5620bac9db50 2024-04-17T07:16:17.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.755+0000 7f2e6e000640 1 -- 172.21.15.86:0/384411892 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e74035020 con 0x5620bac9db50 2024-04-17T07:16:17.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.755+0000 7f2e6e000640 1 -- 172.21.15.86:0/384411892 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2e74012820 con 0x5620bac9db50 2024-04-17T07:16:17.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.755+0000 7f2e923ad740 10 client.16023 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:17.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.755+0000 7f2e923ad740 10 client.16023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:16:17.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.755+0000 7f2e923ad740 20 client.16023 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:17.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.755+0000 7f2e923ad740 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5620bad19c20 0x5620bad1c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:17.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.755+0000 7f2e923ad740 4 client.16023 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:16:17.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.755+0000 7f2e923ad740 1 -- 172.21.15.86:0/384411892 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620bac0a8b0 con 0x5620bad19c20 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.756+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5620bad19c20 0x5620bad1c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.757+0000 7f2e6e000640 10 client.16023 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.757+0000 7f2e8cc00640 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5620bad19c20 0x5620bad1c010 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2d92fd776467326 server_cookie=5bd4eb92801974a4 in_seq=0 out_seq=0 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.757+0000 7f2e6e000640 1 -- 172.21.15.86:0/384411892 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620bac0a8b0 con 0x5620bad19c20 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.757+0000 7f2e6e000640 10 client.16023 handle_command_reply: tid=0 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.757+0000 7f2e923ad740 1 client.16023 shutdown 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.757+0000 7f2e923ad740 1 -- 172.21.15.86:0/384411892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 msgr2=0x5620bacac5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.757+0000 7f2e923ad740 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacac5b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2e74011400 tx=0x7f2e74011900 comp rx=0 tx=0).stop 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.758+0000 7f2e923ad740 1 -- 172.21.15.86:0/384411892 shutdown_connections 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.758+0000 7f2e923ad740 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5620bad19c20 0x5620bad1c010 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.758+0000 7f2e923ad740 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620baca2cc0 0x5620bacad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.758+0000 7f2e923ad740 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620bac9db50 0x5620bacac5b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.758+0000 7f2e923ad740 1 --2- 172.21.15.86:0/384411892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620bac9d160 0x5620bacac070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.758+0000 7f2e6e000640 0 client.16023 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:16:17.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.758+0000 7f2e923ad740 1 -- 172.21.15.86:0/384411892 >> 172.21.15.86:0/384411892 conn(0x5620bac986d0 msgr2=0x5620bac9b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.758+0000 7f2e923ad740 1 -- 172.21.15.86:0/384411892 shutdown_connections 2024-04-17T07:16:17.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.758+0000 7f2e923ad740 1 -- 172.21.15.86:0/384411892 wait complete. 2024-04-17T07:16:17.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.758+0000 7f2e923ad740 20 client.16023 trim_cache size 0 max 16384 2024-04-17T07:16:17.761 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:16:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 -- 172.21.15.86:0/876639707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cc0 msgr2=0x7f2e700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cc0 0x7f2e700a6180 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2e80004af0 tx=0x7f2e80046000 comp rx=0 tx=0).stop 2024-04-17T07:16:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 -- 172.21.15.86:0/876639707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e881006e0 msgr2=0x7f2e8819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e881006e0 0x7f2e8819a230 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00d950 tx=0x7f2e7c00de10 comp rx=0 tx=0).stop 2024-04-17T07:16:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 -- 172.21.15.86:0/876639707 shutdown_connections 2024-04-17T07:16:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cc0 0x7f2e700a6180 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e881019e0 0x7f2e881a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e88101040 0x7f2e8819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 --2- 172.21.15.86:0/876639707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e881006e0 0x7f2e8819a230 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 -- 172.21.15.86:0/876639707 >> 172.21.15.86:0/876639707 conn(0x7f2e880fc300 msgr2=0x7f2e880fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 -- 172.21.15.86:0/876639707 shutdown_connections 2024-04-17T07:16:17.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:17.759+0000 7f2e8e000640 1 -- 172.21.15.86:0/876639707 wait complete. 2024-04-17T07:16:17.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:16:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.000+0000 7efc2c800640 1 -- 172.21.15.86:0/919090250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc241029b0 msgr2=0x7efc241092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.000+0000 7efc2c800640 1 --2- 172.21.15.86:0/919090250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc241029b0 0x7efc241092d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7efc20009ca0 tx=0x7efc2002c100 comp rx=0 tx=0).stop 2024-04-17T07:16:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.000+0000 7efc2c800640 1 -- 172.21.15.86:0/919090250 shutdown_connections 2024-04-17T07:16:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.000+0000 7efc2c800640 1 --2- 172.21.15.86:0/919090250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc241029b0 0x7efc241092d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.000+0000 7efc2c800640 1 --2- 172.21.15.86:0/919090250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc24101ff0 0x7efc24102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.000+0000 7efc2c800640 1 --2- 172.21.15.86:0/919090250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc24100ea0 0x7efc241012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.000+0000 7efc2c800640 1 -- 172.21.15.86:0/919090250 >> 172.21.15.86:0/919090250 conn(0x7efc240fc450 msgr2=0x7efc240fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.000+0000 7efc2c800640 1 -- 172.21.15.86:0/919090250 shutdown_connections 2024-04-17T07:16:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.000+0000 7efc2c800640 1 -- 172.21.15.86:0/919090250 wait complete. 2024-04-17T07:16:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.001+0000 7efc2c800640 1 Processor -- start 2024-04-17T07:16:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.001+0000 7efc2c800640 1 -- start start 2024-04-17T07:16:18.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.001+0000 7efc2c800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc24100ea0 0x7efc24106c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.001+0000 7efc2c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc24101ff0 0x7efc24107150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.001+0000 7efc2c800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc241029b0 0x7efc24103780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.001+0000 7efc2c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc2406b100 con 0x7efc24101ff0 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.001+0000 7efc2c800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc2406af80 con 0x7efc24100ea0 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.001+0000 7efc2c800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc2406b280 con 0x7efc241029b0 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.002+0000 7efc29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc24101ff0 0x7efc24107150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.002+0000 7efc29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc24101ff0 0x7efc24107150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55472/0 (socket says 172.21.15.86:55472) 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.002+0000 7efc29400640 1 -- 172.21.15.86:0/2027077901 learned_addr learned my addr 172.21.15.86:0/2027077901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.002+0000 7efc29e00640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc24100ea0 0x7efc24106c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.002+0000 7efc2a800640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc241029b0 0x7efc24103780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.002+0000 7efc29400640 1 -- 172.21.15.86:0/2027077901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc241029b0 msgr2=0x7efc24103780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.002+0000 7efc29400640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc241029b0 0x7efc24103780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.002+0000 7efc29400640 1 -- 172.21.15.86:0/2027077901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc24100ea0 msgr2=0x7efc24106c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.002+0000 7efc29400640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc24100ea0 0x7efc24106c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.002+0000 7efc29400640 1 -- 172.21.15.86:0/2027077901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc24104040 con 0x7efc24101ff0 2024-04-17T07:16:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.003+0000 7efc29400640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc24101ff0 0x7efc24107150 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c970 tx=0x7efc1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.003+0000 7efc1ea00640 1 -- 172.21.15.86:0/2027077901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14007c10 con 0x7efc24101ff0 2024-04-17T07:16:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.003+0000 7efc2c800640 1 -- 172.21.15.86:0/2027077901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc241a66e0 con 0x7efc24101ff0 2024-04-17T07:16:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.003+0000 7efc1ea00640 1 -- 172.21.15.86:0/2027077901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc14007db0 con 0x7efc24101ff0 2024-04-17T07:16:18.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.004+0000 7efc2c800640 1 -- 172.21.15.86:0/2027077901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241a6b40 con 0x7efc24101ff0 2024-04-17T07:16:18.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.004+0000 7efc1ea00640 1 -- 172.21.15.86:0/2027077901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1400f7f0 con 0x7efc24101ff0 2024-04-17T07:16:18.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.007+0000 7efc2f47e740 1 Processor -- start 2024-04-17T07:16:18.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.007+0000 7efc2f47e740 1 -- start start 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bb83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bbd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bbff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc2f47e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a118b1e020 con 0x55a118bb7fa0 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc2f47e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a118ac4460 con 0x55a118bbdb00 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc2f47e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a118b19fe0 con 0x55a118bb8990 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc1ea00640 1 -- 172.21.15.86:0/2027077901 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7efc14018480 con 0x7efc24101ff0 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc2a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bb83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc2a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bb83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55480/0 (socket says 172.21.15.86:55480) 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc2a800640 1 -- 172.21.15.86:0/1043032592 learned_addr learned my addr 172.21.15.86:0/1043032592 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:18.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.008+0000 7efc29e00640 1 --2- 172.21.15.86:0/1043032592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bbd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.009+0000 7efc29400640 1 --2- 172.21.15.86:0/1043032592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bbff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.009+0000 7efc2a800640 1 -- 172.21.15.86:0/1043032592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 msgr2=0x55a118bbd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.009+0000 7efc2a800640 1 --2- 172.21.15.86:0/1043032592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bbd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.009+0000 7efc2a800640 1 -- 172.21.15.86:0/1043032592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 msgr2=0x55a118bbff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.009+0000 7efc2a800640 1 --2- 172.21.15.86:0/1043032592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bbff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.009+0000 7efc2a800640 1 -- 172.21.15.86:0/1043032592 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a118acc040 con 0x55a118bb7fa0 2024-04-17T07:16:18.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.009+0000 7efc2a800640 1 --2- 172.21.15.86:0/1043032592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bb83c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efc20005cc0 tx=0x7efc20009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95017e2c5601a2c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.009+0000 7efc1ea00640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc0c0a3c40 0x7efc0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.010+0000 7efc13000640 1 -- 172.21.15.86:0/1043032592 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20043020 con 0x55a118bb7fa0 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.010+0000 7efc13000640 1 -- 172.21.15.86:0/1043032592 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc20032e40 con 0x55a118bb7fa0 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.010+0000 7efc13000640 1 -- 172.21.15.86:0/1043032592 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20039040 con 0x55a118bb7fa0 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.010+0000 7efc1ea00640 1 -- 172.21.15.86:0/2027077901 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efc140d0c70 con 0x7efc24101ff0 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.010+0000 7efc29e00640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc0c0a3c40 0x7efc0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.011+0000 7efc2f47e740 1 -- 172.21.15.86:0/1043032592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 msgr2=0x55a118bb83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.011+0000 7efc2f47e740 1 --2- 172.21.15.86:0/1043032592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bb83c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efc20005cc0 tx=0x7efc20009910 comp rx=0 tx=0).stop 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.011+0000 7efc29e00640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc0c0a3c40 0x7efc0c0a6100 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55a118bb8d90 tx=0x7efc1800c4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.011+0000 7efc2f47e740 1 -- 172.21.15.86:0/1043032592 shutdown_connections 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.011+0000 7efc2f47e740 1 --2- 172.21.15.86:0/1043032592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bbff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.011+0000 7efc2f47e740 1 --2- 172.21.15.86:0/1043032592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bbd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.011+0000 7efc2f47e740 1 --2- 172.21.15.86:0/1043032592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bb83c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.011+0000 7efc2f47e740 1 -- 172.21.15.86:0/1043032592 >> 172.21.15.86:0/1043032592 conn(0x55a118bb3510 msgr2=0x55a118bb3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.011+0000 7efc2f47e740 1 -- 172.21.15.86:0/1043032592 shutdown_connections 2024-04-17T07:16:18.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.011+0000 7efc2f47e740 1 -- 172.21.15.86:0/1043032592 wait complete. 2024-04-17T07:16:18.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.012+0000 7efc2f47e740 1 -- start start 2024-04-17T07:16:18.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.012+0000 7efc2f47e740 1 Processor -- start 2024-04-17T07:16:18.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.012+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bb7fa0 0x55a118bc24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.012+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bc2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.012+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bbdb00 0x55a118bc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.012+0000 7efc2f47e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a118bc0a50 con 0x55a118bbdb00 2024-04-17T07:16:18.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.012+0000 7efc2f47e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a118ad7830 con 0x55a118bb7fa0 2024-04-17T07:16:18.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.012+0000 7efc2f47e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a118bc0bd0 con 0x55a118bb8990 2024-04-17T07:16:18.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.012+0000 7efc2f47e740 10 client.0 _ll_register_callbacks cb 0x55a118aa5fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:18.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.013+0000 7efc2a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bbdb00 0x55a118bc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.013+0000 7efc2a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bbdb00 0x55a118bc9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55484/0 (socket says 172.21.15.86:55484) 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.013+0000 7efc2a800640 1 -- 172.21.15.86:0/2586533904 learned_addr learned my addr 172.21.15.86:0/2586533904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.013+0000 7efc29e00640 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bb7fa0 0x55a118bc24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.013+0000 7efc2a800640 1 -- 172.21.15.86:0/2586533904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 msgr2=0x55a118bc2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.013+0000 7efc2a800640 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bc2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.013+0000 7efc2a800640 1 -- 172.21.15.86:0/2586533904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bb7fa0 msgr2=0x55a118bc24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.013+0000 7efc2a800640 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bb7fa0 0x55a118bc24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.013+0000 7efc2a800640 1 -- 172.21.15.86:0/2586533904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a118adb920 con 0x55a118bbdb00 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.014+0000 7efc2a800640 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bbdb00 0x55a118bc9a60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efc20032380 tx=0x7efc2002c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45fc3d5c4e83622 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.014+0000 7efc07400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.014+0000 7efc07400640 1 -- 172.21.15.86:0/2586533904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20043030 con 0x55a118bbdb00 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.014+0000 7efc07400640 1 -- 172.21.15.86:0/2586533904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efc20004280 con 0x55a118bbdb00 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.014+0000 7efc2f47e740 1 -- 172.21.15.86:0/2586533904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc20032e40 con 0x55a118bbdb00 2024-04-17T07:16:18.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.014+0000 7efc07400640 1 -- 172.21.15.86:0/2586533904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efc20058050 con 0x55a118bbdb00 2024-04-17T07:16:18.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.014+0000 7efc2f47e740 10 client.16035 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:18.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.014+0000 7efc2f47e740 1 -- 172.21.15.86:0/2586533904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a118aa97f0 con 0x55a118bbdb00 2024-04-17T07:16:18.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.015+0000 7efc07400640 1 -- 172.21.15.86:0/2586533904 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc20032e40 con 0x55a118bbdb00 2024-04-17T07:16:18.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.015+0000 7efc07400640 1 -- 172.21.15.86:0/2586533904 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7efc2003fa50 con 0x55a118bbdb00 2024-04-17T07:16:18.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.015+0000 7efc2f47e740 10 client.16035 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:18.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.015+0000 7efc2f47e740 10 client.16035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:16:18.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.015+0000 7efc2f47e740 20 client.16035 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:18.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.015+0000 7efc2f47e740 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55a118bcdbc0 0x55a118bcffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.015+0000 7efc2f47e740 4 client.16035 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:18.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.015+0000 7efc2f47e740 1 -- 172.21.15.86:0/2586533904 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a118b2ba40 con 0x55a118bcdbc0 2024-04-17T07:16:18.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.016+0000 7efc29e00640 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55a118bcdbc0 0x55a118bcffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.017+0000 7efc29e00640 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55a118bcdbc0 0x55a118bcffb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=749aa3acd67696dc server_cookie=5ab5ecc9d0c33409 in_seq=0 out_seq=0 2024-04-17T07:16:18.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.017+0000 7efc07400640 10 client.16035 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:16:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.018+0000 7efc07400640 1 -- 172.21.15.86:0/2586533904 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a118b2ba40 con 0x55a118bcdbc0 2024-04-17T07:16:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.018+0000 7efc07400640 10 client.16035 handle_command_reply: tid=0 2024-04-17T07:16:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 client.16035 shutdown 2024-04-17T07:16:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 -- 172.21.15.86:0/2586533904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bbdb00 msgr2=0x55a118bc9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bbdb00 0x55a118bc9a60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efc20032380 tx=0x7efc2002c880 comp rx=0 tx=0).stop 2024-04-17T07:16:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 -- 172.21.15.86:0/2586533904 shutdown_connections 2024-04-17T07:16:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bbdb00 0x55a118bc9a60 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bc2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55a118bcdbc0 0x55a118bcffb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 --2- 172.21.15.86:0/2586533904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bb7fa0 0x55a118bc24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 -- 172.21.15.86:0/2586533904 >> 172.21.15.86:0/2586533904 conn(0x55a118bb3510 msgr2=0x55a118bb2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc07400640 0 client.16035 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:16:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 -- 172.21.15.86:0/2586533904 shutdown_connections 2024-04-17T07:16:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 1 -- 172.21.15.86:0/2586533904 wait complete. 2024-04-17T07:16:18.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.019+0000 7efc2f47e740 20 client.16035 trim_cache size 0 max 16384 2024-04-17T07:16:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.037+0000 7efc2f47e740 1 Processor -- start 2024-04-17T07:16:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.037+0000 7efc2f47e740 1 -- start start 2024-04-17T07:16:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.037+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bc24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.037+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bc9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bc2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc2f47e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc241b0f70 con 0x55a118bb7fa0 2024-04-17T07:16:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc2f47e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc241b2ab0 con 0x55a118bbdb00 2024-04-17T07:16:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc2f47e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc241b1640 con 0x55a118bb8990 2024-04-17T07:16:18.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc29e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bc24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc29e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bc24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55488/0 (socket says 172.21.15.86:55488) 2024-04-17T07:16:18.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc29e00640 1 -- 172.21.15.86:0/739258200 learned_addr learned my addr 172.21.15.86:0/739258200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:18.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc29400640 1 --2- 172.21.15.86:0/739258200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bc9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc2a800640 1 --2- 172.21.15.86:0/739258200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bc2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc29e00640 1 -- 172.21.15.86:0/739258200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 msgr2=0x55a118bc9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc29e00640 1 --2- 172.21.15.86:0/739258200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bc9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc29e00640 1 -- 172.21.15.86:0/739258200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 msgr2=0x55a118bc2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc29e00640 1 --2- 172.21.15.86:0/739258200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bc2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.038+0000 7efc29e00640 1 -- 172.21.15.86:0/739258200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1188303f0 con 0x55a118bb7fa0 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.039+0000 7efc29e00640 1 --2- 172.21.15.86:0/739258200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bc24e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7efc18004650 tx=0x7efc18014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d86670d4fcf005 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.039+0000 7efc12600640 1 -- 172.21.15.86:0/739258200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18011c90 con 0x55a118bb7fa0 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.039+0000 7efc12600640 1 -- 172.21.15.86:0/739258200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc18011e30 con 0x55a118bb7fa0 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.039+0000 7efc12600640 1 -- 172.21.15.86:0/739258200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc180270d0 con 0x55a118bb7fa0 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.040+0000 7efc2f47e740 1 -- 172.21.15.86:0/739258200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 msgr2=0x55a118bc24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.040+0000 7efc2f47e740 1 --2- 172.21.15.86:0/739258200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bc24e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7efc18004650 tx=0x7efc18014000 comp rx=0 tx=0).stop 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.040+0000 7efc2f47e740 1 -- 172.21.15.86:0/739258200 shutdown_connections 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.040+0000 7efc2f47e740 1 --2- 172.21.15.86:0/739258200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bc2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.040+0000 7efc2f47e740 1 --2- 172.21.15.86:0/739258200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb8990 0x55a118bc9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.040+0000 7efc2f47e740 1 --2- 172.21.15.86:0/739258200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb7fa0 0x55a118bc24e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.040+0000 7efc2f47e740 1 -- 172.21.15.86:0/739258200 >> 172.21.15.86:0/739258200 conn(0x55a118bb3510 msgr2=0x55a118bc0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.040+0000 7efc2f47e740 1 -- 172.21.15.86:0/739258200 shutdown_connections 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.040+0000 7efc2f47e740 1 -- 172.21.15.86:0/739258200 wait complete. 2024-04-17T07:16:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.041+0000 7efc2f47e740 1 -- start start 2024-04-17T07:16:18.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.041+0000 7efc2f47e740 1 Processor -- start 2024-04-17T07:16:18.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.041+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb7fa0 0x55a118bc6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.041+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb8990 0x55a118bc70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.041+0000 7efc2f47e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bc80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.041+0000 7efc2f47e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc241b23e0 con 0x55a118bb8990 2024-04-17T07:16:18.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.041+0000 7efc2f47e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc241b1d10 con 0x55a118bbdb00 2024-04-17T07:16:18.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.041+0000 7efc2f47e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc241aef40 con 0x55a118bb7fa0 2024-04-17T07:16:18.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.041+0000 7efc2f47e740 10 client.0 _ll_register_callbacks cb 0x7efc20032910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:18.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb8990 0x55a118bc70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb8990 0x55a118bc70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55502/0 (socket says 172.21.15.86:55502) 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29400640 1 -- 172.21.15.86:0/3951943486 learned_addr learned my addr 172.21.15.86:0/3951943486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29e00640 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb7fa0 0x55a118bc6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc2a800640 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bc80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29400640 1 -- 172.21.15.86:0/3951943486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb7fa0 msgr2=0x55a118bc6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29400640 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb7fa0 0x55a118bc6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29400640 1 -- 172.21.15.86:0/3951943486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 msgr2=0x55a118bc80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29400640 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bc80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29400640 1 -- 172.21.15.86:0/3951943486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1188fa7d0 con 0x55a118bb8990 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29e00640 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb7fa0 0x55a118bc6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc2a800640 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bc80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc29400640 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb8990 0x55a118bc70c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7efc140ce2d0 tx=0x7efc1409c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185b764e57fab8d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.042+0000 7efc07400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.043+0000 7efc07400640 1 -- 172.21.15.86:0/3951943486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1409cb00 con 0x55a118bb8990 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.043+0000 7efc07400640 1 -- 172.21.15.86:0/3951943486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efc140ce4f0 con 0x55a118bb8990 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.043+0000 7efc2f47e740 1 -- 172.21.15.86:0/3951943486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc18011e30 con 0x55a118bb8990 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.043+0000 7efc07400640 1 -- 172.21.15.86:0/3951943486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efc140a0eb0 con 0x55a118bb8990 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.043+0000 7efc2f47e740 10 client.16041 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.043+0000 7efc2f47e740 1 -- 172.21.15.86:0/3951943486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a118bc9860 con 0x55a118bb8990 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.043+0000 7efc07400640 1 -- 172.21.15.86:0/3951943486 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc18011e30 con 0x55a118bb8990 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.043+0000 7efc07400640 1 -- 172.21.15.86:0/3951943486 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7efc140b9050 con 0x55a118bb8990 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.044+0000 7efc2f47e740 10 client.16041 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.044+0000 7efc2f47e740 10 client.16041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.044+0000 7efc2f47e740 20 client.16041 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.044+0000 7efc2f47e740 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55a118c34410 0x55a118c36800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.044+0000 7efc2f47e740 4 client.16041 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:16:18.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.044+0000 7efc2f47e740 1 -- 172.21.15.86:0/3951943486 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a118bc22d0 con 0x55a118c34410 2024-04-17T07:16:18.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.044+0000 7efc29e00640 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55a118c34410 0x55a118c36800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.045+0000 7efc29e00640 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55a118c34410 0x55a118c36800 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c51fe0704b257e8 server_cookie=a9ad651746dcb8e2 in_seq=0 out_seq=0 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.045+0000 7efc07400640 10 client.16041 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc07400640 1 -- 172.21.15.86:0/3951943486 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a118bc22d0 con 0x55a118c34410 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc07400640 10 client.16041 handle_command_reply: tid=0 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 client.16041 shutdown 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 -- 172.21.15.86:0/3951943486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb8990 msgr2=0x55a118bc70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb8990 0x55a118bc70c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7efc140ce2d0 tx=0x7efc1409c740 comp rx=0 tx=0).stop 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 -- 172.21.15.86:0/3951943486 shutdown_connections 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55a118c34410 0x55a118c36800 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a118bbdb00 0x55a118bc80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a118bb8990 0x55a118bc70c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 --2- 172.21.15.86:0/3951943486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a118bb7fa0 0x55a118bc6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 -- 172.21.15.86:0/3951943486 >> 172.21.15.86:0/3951943486 conn(0x55a118bb3510 msgr2=0x55a118bc0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc07400640 0 client.16041 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:16:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 -- 172.21.15.86:0/3951943486 shutdown_connections 2024-04-17T07:16:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.046+0000 7efc2f47e740 1 -- 172.21.15.86:0/3951943486 wait complete. 2024-04-17T07:16:18.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2f47e740 20 client.16041 trim_cache size 0 max 16384 2024-04-17T07:16:18.049 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:16:18.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2c800640 1 -- 172.21.15.86:0/2027077901 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc0c0a3c40 msgr2=0x7efc0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2c800640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc0c0a3c40 0x7efc0c0a6100 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55a118bb8d90 tx=0x7efc1800c4b0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2c800640 1 -- 172.21.15.86:0/2027077901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc24101ff0 msgr2=0x7efc24107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2c800640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc24101ff0 0x7efc24107150 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c970 tx=0x7efc1400ce30 comp rx=0 tx=0).stop 2024-04-17T07:16:18.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2c800640 1 -- 172.21.15.86:0/2027077901 shutdown_connections 2024-04-17T07:16:18.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2c800640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efc0c0a3c40 0x7efc0c0a6100 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2c800640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc241029b0 0x7efc24103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2c800640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc24101ff0 0x7efc24107150 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2c800640 1 --2- 172.21.15.86:0/2027077901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc24100ea0 0x7efc24106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.047+0000 7efc2c800640 1 -- 172.21.15.86:0/2027077901 >> 172.21.15.86:0/2027077901 conn(0x7efc240fc450 msgr2=0x7efc2410be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.048+0000 7efc2c800640 1 -- 172.21.15.86:0/2027077901 shutdown_connections 2024-04-17T07:16:18.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.048+0000 7efc2c800640 1 -- 172.21.15.86:0/2027077901 wait complete. 2024-04-17T07:16:18.066 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:16:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.278+0000 7fee1ee00640 1 -- 172.21.15.86:0/1587937795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee18100740 msgr2=0x7fee18100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.279+0000 7fee1ee00640 1 --2- 172.21.15.86:0/1587937795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee18100740 0x7fee18100b60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009900 tx=0x7fee0c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:16:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.279+0000 7fee1ee00640 1 -- 172.21.15.86:0/1587937795 shutdown_connections 2024-04-17T07:16:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.279+0000 7fee1ee00640 1 --2- 172.21.15.86:0/1587937795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee18101a40 0x7fee18101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.279+0000 7fee1ee00640 1 --2- 172.21.15.86:0/1587937795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee181010a0 0x7fee18101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.279+0000 7fee1ee00640 1 --2- 172.21.15.86:0/1587937795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee18100740 0x7fee18100b60 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.279+0000 7fee1ee00640 1 -- 172.21.15.86:0/1587937795 >> 172.21.15.86:0/1587937795 conn(0x7fee180fc300 msgr2=0x7fee180fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.279+0000 7fee1ee00640 1 -- 172.21.15.86:0/1587937795 shutdown_connections 2024-04-17T07:16:18.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.279+0000 7fee1ee00640 1 -- 172.21.15.86:0/1587937795 wait complete. 2024-04-17T07:16:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.280+0000 7fee1ee00640 1 Processor -- start 2024-04-17T07:16:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.280+0000 7fee1ee00640 1 -- start start 2024-04-17T07:16:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1ee00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee18100740 0x7fee1819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1ee00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee181010a0 0x7fee1819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee18101a40 0x7fee181a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee1810fc10 con 0x7fee18101a40 2024-04-17T07:16:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1ee00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee1810fa90 con 0x7fee18100740 2024-04-17T07:16:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1ee00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee1810fd90 con 0x7fee181010a0 2024-04-17T07:16:18.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee18101a40 0x7fee181a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee18101a40 0x7fee181a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55518/0 (socket says 172.21.15.86:55518) 2024-04-17T07:16:18.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1e400640 1 -- 172.21.15.86:0/1514910082 learned_addr learned my addr 172.21.15.86:0/1514910082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:18.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1d000640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee181010a0 0x7fee1819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.281+0000 7fee1da00640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee18100740 0x7fee1819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee1e400640 1 -- 172.21.15.86:0/1514910082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee181010a0 msgr2=0x7fee1819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee1e400640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee181010a0 0x7fee1819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee1e400640 1 -- 172.21.15.86:0/1514910082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee18100740 msgr2=0x7fee1819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee1e400640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee18100740 0x7fee1819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee1e400640 1 -- 172.21.15.86:0/1514910082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee181a1ea0 con 0x7fee18101a40 2024-04-17T07:16:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee1e400640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee18101a40 0x7fee181a1830 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fee1000eaa0 tx=0x7fee1000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee16000640 1 -- 172.21.15.86:0/1514910082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1000cdb0 con 0x7fee18101a40 2024-04-17T07:16:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee16000640 1 -- 172.21.15.86:0/1514910082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee10005020 con 0x7fee18101a40 2024-04-17T07:16:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee1ee00640 1 -- 172.21.15.86:0/1514910082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee181a2190 con 0x7fee18101a40 2024-04-17T07:16:18.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee16000640 1 -- 172.21.15.86:0/1514910082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100116d0 con 0x7fee18101a40 2024-04-17T07:16:18.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.282+0000 7fee1ee00640 1 -- 172.21.15.86:0/1514910082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee181a2690 con 0x7fee18101a40 2024-04-17T07:16:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.286+0000 7fee23012740 1 Processor -- start 2024-04-17T07:16:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.286+0000 7fee16000640 1 -- 172.21.15.86:0/1514910082 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7fee100051d0 con 0x7fee18101a40 2024-04-17T07:16:18.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.287+0000 7fee23012740 1 -- start start 2024-04-17T07:16:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.287+0000 7fee23012740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.287+0000 7fee23012740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80ccb50 0x55b8f80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.287+0000 7fee23012740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 0x55b8f80d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.287+0000 7fee23012740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8f8032020 con 0x55b8f80ccb50 2024-04-17T07:16:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.287+0000 7fee23012740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b8f7fd8460 con 0x55b8f80d1cc0 2024-04-17T07:16:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.287+0000 7fee23012740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b8f802dfe0 con 0x55b8f80cc160 2024-04-17T07:16:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.287+0000 7fee1da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80ccb50 0x55b8f80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.287+0000 7fee1da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80ccb50 0x55b8f80d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55532/0 (socket says 172.21.15.86:55532) 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.287+0000 7fee1da00640 1 -- 172.21.15.86:0/299094260 learned_addr learned my addr 172.21.15.86:0/299094260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee1d000640 1 --2- 172.21.15.86:0/299094260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee1d000640 1 --2- 172.21.15.86:0/299094260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 0x55b8f80d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee1da00640 1 -- 172.21.15.86:0/299094260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 msgr2=0x55b8f80cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee1da00640 1 --2- 172.21.15.86:0/299094260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee1da00640 1 -- 172.21.15.86:0/299094260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 msgr2=0x55b8f80d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee1da00640 1 --2- 172.21.15.86:0/299094260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 0x55b8f80d4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee1da00640 1 -- 172.21.15.86:0/299094260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f7fe0040 con 0x55b8f80ccb50 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee16000640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee000a3c40 0x7fee000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee16000640 1 -- 172.21.15.86:0/1514910082 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fee100d0c20 con 0x7fee18101a40 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee1da00640 1 --2- 172.21.15.86:0/299094260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80ccb50 0x55b8f80d1650 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fee0c02c840 tx=0x7fee0c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592012feacff5cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.288+0000 7fee1d000640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee000a3c40 0x7fee000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.289+0000 7fee0aa00640 1 -- 172.21.15.86:0/299094260 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c044020 con 0x55b8f80ccb50 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.289+0000 7fee0aa00640 1 -- 172.21.15.86:0/299094260 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee0c02cbe0 con 0x55b8f80ccb50 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.289+0000 7fee0aa00640 1 -- 172.21.15.86:0/299094260 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0048c0 con 0x55b8f80ccb50 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.290+0000 7fee23012740 1 -- 172.21.15.86:0/299094260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80ccb50 msgr2=0x55b8f80d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.290+0000 7fee23012740 1 --2- 172.21.15.86:0/299094260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80ccb50 0x55b8f80d1650 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fee0c02c840 tx=0x7fee0c005dd0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.290+0000 7fee23012740 1 -- 172.21.15.86:0/299094260 shutdown_connections 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.290+0000 7fee23012740 1 --2- 172.21.15.86:0/299094260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 0x55b8f80d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.290+0000 7fee23012740 1 --2- 172.21.15.86:0/299094260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80ccb50 0x55b8f80d1650 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.290+0000 7fee23012740 1 --2- 172.21.15.86:0/299094260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.290+0000 7fee23012740 1 -- 172.21.15.86:0/299094260 >> 172.21.15.86:0/299094260 conn(0x55b8f80c76d0 msgr2=0x55b8f80c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.290+0000 7fee23012740 1 -- 172.21.15.86:0/299094260 shutdown_connections 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.290+0000 7fee1d000640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee000a3c40 0x7fee000a6100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55b8f80cd3b0 tx=0x7fee040046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.290+0000 7fee23012740 1 -- 172.21.15.86:0/299094260 wait complete. 2024-04-17T07:16:18.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.291+0000 7fee23012740 1 -- start start 2024-04-17T07:16:18.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.291+0000 7fee23012740 1 Processor -- start 2024-04-17T07:16:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.291+0000 7fee23012740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80d6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.291+0000 7fee23012740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 0x55b8f80d6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.291+0000 7fee23012740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.291+0000 7fee23012740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8f80d4c10 con 0x55b8f80d1cc0 2024-04-17T07:16:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee23012740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b8f7feb830 con 0x55b8f80ccb50 2024-04-17T07:16:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee23012740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b8f80d4d90 con 0x55b8f80cc160 2024-04-17T07:16:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee23012740 10 client.0 _ll_register_callbacks cb 0x55b8f7fb9fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee1e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee1e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55534/0 (socket says 172.21.15.86:55534) 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee1e400640 1 -- 172.21.15.86:0/2443689970 learned_addr learned my addr 172.21.15.86:0/2443689970 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee1da00640 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80d6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee1d000640 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 0x55b8f80d6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee1e400640 1 -- 172.21.15.86:0/2443689970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 msgr2=0x55b8f80d6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee1e400640 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80d6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee1e400640 1 -- 172.21.15.86:0/2443689970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 msgr2=0x55b8f80d6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee1e400640 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 0x55b8f80d6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.292+0000 7fee1e400640 1 -- 172.21.15.86:0/2443689970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f7fef920 con 0x55b8f80d1cc0 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.293+0000 7fee1e400640 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fee10099390 tx=0x7fee1009c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593f8f9f52df84c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.293+0000 7fedff400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.293+0000 7fedff400640 1 -- 172.21.15.86:0/2443689970 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1001f040 con 0x55b8f80d1cc0 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.293+0000 7fedff400640 1 -- 172.21.15.86:0/2443689970 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fee100bbb80 con 0x55b8f80d1cc0 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.293+0000 7fee23012740 1 -- 172.21.15.86:0/2443689970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee0c02cbe0 con 0x55b8f80d1cc0 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.293+0000 7fedff400640 1 -- 172.21.15.86:0/2443689970 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee100a5630 con 0x55b8f80d1cc0 2024-04-17T07:16:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.293+0000 7fee23012740 10 client.16053 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.293+0000 7fee23012740 1 -- 172.21.15.86:0/2443689970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b8f7fbd7f0 con 0x55b8f80d1cc0 2024-04-17T07:16:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.294+0000 7fedff400640 1 -- 172.21.15.86:0/2443689970 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee0c02cbe0 con 0x55b8f80d1cc0 2024-04-17T07:16:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.294+0000 7fedff400640 1 -- 172.21.15.86:0/2443689970 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fee1000f040 con 0x55b8f80d1cc0 2024-04-17T07:16:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.294+0000 7fee23012740 10 client.16053 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.294+0000 7fee23012740 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.294+0000 7fee23012740 20 client.16053 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.294+0000 7fee23012740 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b8f80e2170 0x55b8f80e4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.294+0000 7fee23012740 4 client.16053 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.294+0000 7fee23012740 1 -- 172.21.15.86:0/2443689970 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8f803fa40 con 0x55b8f80e2170 2024-04-17T07:16:18.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.295+0000 7fee1da00640 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b8f80e2170 0x55b8f80e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.296+0000 7fee1da00640 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b8f80e2170 0x55b8f80e4610 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d203ba491d6a9c03 server_cookie=14510a32d7a5623c in_seq=0 out_seq=0 2024-04-17T07:16:18.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.296+0000 7fedff400640 10 client.16053 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:18.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.297+0000 7fedff400640 1 -- 172.21.15.86:0/2443689970 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8f803fa40 con 0x55b8f80e2170 2024-04-17T07:16:18.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.297+0000 7fedff400640 10 client.16053 handle_command_reply: tid=0 2024-04-17T07:16:18.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.297+0000 7fee23012740 1 client.16053 shutdown 2024-04-17T07:16:18.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 1 -- 172.21.15.86:0/2443689970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 msgr2=0x55b8f80ddb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fee10099390 tx=0x7fee1009c9c0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 1 -- 172.21.15.86:0/2443689970 shutdown_connections 2024-04-17T07:16:18.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b8f80e2170 0x55b8f80e4610 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 0x55b8f80d6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 1 --2- 172.21.15.86:0/2443689970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80d6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 1 -- 172.21.15.86:0/2443689970 >> 172.21.15.86:0/2443689970 conn(0x55b8f80c76d0 msgr2=0x55b8f80c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fedff400640 0 client.16053 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:18.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 1 -- 172.21.15.86:0/2443689970 shutdown_connections 2024-04-17T07:16:18.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 1 -- 172.21.15.86:0/2443689970 wait complete. 2024-04-17T07:16:18.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.298+0000 7fee23012740 20 client.16053 trim_cache size 0 max 16384 2024-04-17T07:16:18.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.318+0000 7fee23012740 1 Processor -- start 2024-04-17T07:16:18.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.318+0000 7fee23012740 1 -- start start 2024-04-17T07:16:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.318+0000 7fee23012740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.318+0000 7fee23012740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 0x55b8f80ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.318+0000 7fee23012740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.318+0000 7fee23012740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee181aca80 con 0x55b8f80d1cc0 2024-04-17T07:16:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.318+0000 7fee23012740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee181ae5c0 con 0x55b8f80ccb50 2024-04-17T07:16:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.318+0000 7fee23012740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee181ad150 con 0x55b8f80cc160 2024-04-17T07:16:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55548/0 (socket says 172.21.15.86:55548) 2024-04-17T07:16:18.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1e400640 1 -- 172.21.15.86:0/3469067512 learned_addr learned my addr 172.21.15.86:0/3469067512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:18.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1d000640 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 0x55b8f80ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1da00640 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1e400640 1 -- 172.21.15.86:0/3469067512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 msgr2=0x55b8f80ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1e400640 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1e400640 1 -- 172.21.15.86:0/3469067512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 msgr2=0x55b8f80ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1e400640 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 0x55b8f80ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1e400640 1 -- 172.21.15.86:0/3469067512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f7d443f0 con 0x55b8f80d1cc0 2024-04-17T07:16:18.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1da00640 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:18.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.319+0000 7fee1d000640 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 0x55b8f80ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:16:18.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.320+0000 7fee1e400640 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee10098e20 tx=0x7fee10005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754b6c1a80208fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.320+0000 7fee0a000640 1 -- 172.21.15.86:0/3469067512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1001f020 con 0x55b8f80d1cc0 2024-04-17T07:16:18.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.320+0000 7fee0a000640 1 -- 172.21.15.86:0/3469067512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee10005620 con 0x55b8f80d1cc0 2024-04-17T07:16:18.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.320+0000 7fee0a000640 1 -- 172.21.15.86:0/3469067512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1009b880 con 0x55b8f80d1cc0 2024-04-17T07:16:18.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.321+0000 7fee23012740 1 -- 172.21.15.86:0/3469067512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 msgr2=0x55b8f80ddb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.321+0000 7fee23012740 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee10098e20 tx=0x7fee10005870 comp rx=0 tx=0).stop 2024-04-17T07:16:18.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.321+0000 7fee23012740 1 -- 172.21.15.86:0/3469067512 shutdown_connections 2024-04-17T07:16:18.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.321+0000 7fee23012740 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80d1cc0 0x55b8f80ddb50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.321+0000 7fee23012740 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80ccb50 0x55b8f80ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.321+0000 7fee23012740 1 --2- 172.21.15.86:0/3469067512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80cc160 0x55b8f80ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.321+0000 7fee23012740 1 -- 172.21.15.86:0/3469067512 >> 172.21.15.86:0/3469067512 conn(0x55b8f80c76d0 msgr2=0x55b8f80d6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.322+0000 7fee23012740 1 -- 172.21.15.86:0/3469067512 shutdown_connections 2024-04-17T07:16:18.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.322+0000 7fee23012740 1 -- 172.21.15.86:0/3469067512 wait complete. 2024-04-17T07:16:18.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.322+0000 7fee23012740 1 -- start start 2024-04-17T07:16:18.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.322+0000 7fee23012740 1 Processor -- start 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee23012740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80cc160 0x55b8f80dad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee23012740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80ccb50 0x55b8f80db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee23012740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 0x55b8f80dc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee23012740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee181adef0 con 0x55b8f80cc160 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee23012740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee181ad820 con 0x55b8f80d1cc0 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee23012740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee181aaa50 con 0x55b8f80ccb50 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee23012740 10 client.0 _ll_register_callbacks cb 0x7fee1009c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee1da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80cc160 0x55b8f80dad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee1da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80cc160 0x55b8f80dad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55554/0 (socket says 172.21.15.86:55554) 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee1da00640 1 -- 172.21.15.86:0/2214908501 learned_addr learned my addr 172.21.15.86:0/2214908501 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.323+0000 7fee1d000640 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80ccb50 0x55b8f80db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fee1e400640 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 0x55b8f80dc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fee1da00640 1 -- 172.21.15.86:0/2214908501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80ccb50 msgr2=0x55b8f80db2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fee1da00640 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80ccb50 0x55b8f80db2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fee1da00640 1 -- 172.21.15.86:0/2214908501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 msgr2=0x55b8f80dc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fee1da00640 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 0x55b8f80dc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fee1da00640 1 -- 172.21.15.86:0/2214908501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f7e0e7d0 con 0x55b8f80cc160 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fee1e400640 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 0x55b8f80dc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fee1da00640 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80cc160 0x55b8f80dad90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009230 tx=0x7fee0c02c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936008cfbe475ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fedff400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fedff400640 1 -- 172.21.15.86:0/2214908501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0c050030 con 0x55b8f80cc160 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fedff400640 1 -- 172.21.15.86:0/2214908501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fee0c0041c0 con 0x55b8f80cc160 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.324+0000 7fee23012740 1 -- 172.21.15.86:0/2214908501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee10005620 con 0x55b8f80cc160 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fedff400640 1 -- 172.21.15.86:0/2214908501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee0c05c050 con 0x55b8f80cc160 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fee23012740 10 client.16059 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fee23012740 1 -- 172.21.15.86:0/2214908501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b8f80dd9e0 con 0x55b8f80cc160 2024-04-17T07:16:18.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fedff400640 1 -- 172.21.15.86:0/2214908501 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee10005620 con 0x55b8f80cc160 2024-04-17T07:16:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fedff400640 1 -- 172.21.15.86:0/2214908501 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fee0c004750 con 0x55b8f80cc160 2024-04-17T07:16:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fee23012740 10 client.16059 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fee23012740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fee23012740 20 client.16059 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fee23012740 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b8f8148750 0x55b8f814ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fee23012740 4 client.16059 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:16:18.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.325+0000 7fee23012740 1 -- 172.21.15.86:0/2214908501 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8f80398b0 con 0x55b8f8148750 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.326+0000 7fee1d000640 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b8f8148750 0x55b8f814ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.327+0000 7fee1d000640 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b8f8148750 0x55b8f814ab40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17af9da5b6a1df7 server_cookie=2c2070375acd871b in_seq=0 out_seq=0 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.327+0000 7fedff400640 10 client.16059 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.327+0000 7fedff400640 1 -- 172.21.15.86:0/2214908501 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8f80398b0 con 0x55b8f8148750 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.327+0000 7fedff400640 10 client.16059 handle_command_reply: tid=0 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.327+0000 7fee23012740 1 client.16059 shutdown 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 1 -- 172.21.15.86:0/2214908501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80cc160 msgr2=0x55b8f80dad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80cc160 0x55b8f80dad90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009230 tx=0x7fee0c02c660 comp rx=0 tx=0).stop 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 1 -- 172.21.15.86:0/2214908501 shutdown_connections 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b8f8148750 0x55b8f814ab40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b8f80d1cc0 0x55b8f80dc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b8f80ccb50 0x55b8f80db2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 1 --2- 172.21.15.86:0/2214908501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b8f80cc160 0x55b8f80dad90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fedff400640 0 client.16059 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 1 -- 172.21.15.86:0/2214908501 >> 172.21.15.86:0/2214908501 conn(0x55b8f80c76d0 msgr2=0x55b8f80d65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 1 -- 172.21.15.86:0/2214908501 shutdown_connections 2024-04-17T07:16:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 1 -- 172.21.15.86:0/2214908501 wait complete. 2024-04-17T07:16:18.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.328+0000 7fee23012740 20 client.16059 trim_cache size 0 max 16384 2024-04-17T07:16:18.330 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:16:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 -- 172.21.15.86:0/1514910082 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee000a3c40 msgr2=0x7fee000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee000a3c40 0x7fee000a6100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55b8f80cd3b0 tx=0x7fee040046c0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 -- 172.21.15.86:0/1514910082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee18101a40 msgr2=0x7fee181a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee18101a40 0x7fee181a1830 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fee1000eaa0 tx=0x7fee1000ef60 comp rx=0 tx=0).stop 2024-04-17T07:16:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 -- 172.21.15.86:0/1514910082 shutdown_connections 2024-04-17T07:16:18.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fee000a3c40 0x7fee000a6100 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee18101a40 0x7fee181a1830 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee181010a0 0x7fee1819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 --2- 172.21.15.86:0/1514910082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee18100740 0x7fee1819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:18.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 -- 172.21.15.86:0/1514910082 >> 172.21.15.86:0/1514910082 conn(0x7fee180fc300 msgr2=0x7fee180fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:18.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 -- 172.21.15.86:0/1514910082 shutdown_connections 2024-04-17T07:16:18.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:18.329+0000 7fee1ee00640 1 -- 172.21.15.86:0/1514910082 wait complete. 2024-04-17T07:16:18.332 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:18 smithi086 ceph-mon[31733]: pgmap v498: 129 pgs: 129 active+clean; 75 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 3.2 MiB/s wr, 39 op/s 2024-04-17T07:16:18.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:18 smithi098 ceph-mon[35418]: pgmap v498: 129 pgs: 129 active+clean; 75 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 3.2 MiB/s wr, 39 op/s 2024-04-17T07:16:18.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:18 smithi103 ceph-mon[35495]: pgmap v498: 129 pgs: 129 active+clean; 75 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 3.2 MiB/s wr, 39 op/s 2024-04-17T07:16:19.350 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:16:19.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:16:19.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:16:19.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:19 smithi103 ceph-mon[35495]: pgmap v499: 129 pgs: 129 active+clean; 83 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 22 op/s 2024-04-17T07:16:19.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:19 smithi086 ceph-mon[31733]: pgmap v499: 129 pgs: 129 active+clean; 83 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 22 op/s 2024-04-17T07:16:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.596+0000 7f9582800640 1 -- 172.21.15.86:0/3376543050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c102fe0 msgr2=0x7f957c103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.596+0000 7f9582800640 1 --2- 172.21.15.86:0/3376543050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c102fe0 0x7f957c103460 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9568009900 tx=0x7f956802bc30 comp rx=0 tx=0).stop 2024-04-17T07:16:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.596+0000 7f9582800640 1 -- 172.21.15.86:0/3376543050 shutdown_connections 2024-04-17T07:16:19.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.596+0000 7f9582800640 1 --2- 172.21.15.86:0/3376543050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957c1039a0 0x7f957c10a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.596+0000 7f9582800640 1 --2- 172.21.15.86:0/3376543050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c102fe0 0x7f957c103460 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.596+0000 7f9582800640 1 --2- 172.21.15.86:0/3376543050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f957c101e90 0x7f957c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.596+0000 7f9582800640 1 -- 172.21.15.86:0/3376543050 >> 172.21.15.86:0/3376543050 conn(0x7f957c0fd400 msgr2=0x7f957c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.596+0000 7f9582800640 1 -- 172.21.15.86:0/3376543050 shutdown_connections 2024-04-17T07:16:19.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.596+0000 7f9582800640 1 -- 172.21.15.86:0/3376543050 wait complete. 2024-04-17T07:16:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.597+0000 7f9582800640 1 Processor -- start 2024-04-17T07:16:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9582800640 1 -- start start 2024-04-17T07:16:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9582800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957c101e90 0x7f957c075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9582800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c102fe0 0x7f957c074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9582800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f957c1039a0 0x7f957c074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9582800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10d9f0 con 0x7f957c102fe0 2024-04-17T07:16:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9582800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10d870 con 0x7f957c101e90 2024-04-17T07:16:19.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9582800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f957c10db70 con 0x7f957c1039a0 2024-04-17T07:16:19.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9580a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c102fe0 0x7f957c074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9580a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c102fe0 0x7f957c074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55578/0 (socket says 172.21.15.86:55578) 2024-04-17T07:16:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9580a00640 1 -- 172.21.15.86:0/4056578676 learned_addr learned my addr 172.21.15.86:0/4056578676 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.598+0000 7f9581400640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957c101e90 0x7f957c075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.599+0000 7f9581e00640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f957c1039a0 0x7f957c074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.599+0000 7f9580a00640 1 -- 172.21.15.86:0/4056578676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f957c1039a0 msgr2=0x7f957c074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.599+0000 7f9580a00640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f957c1039a0 0x7f957c074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.599+0000 7f9580a00640 1 -- 172.21.15.86:0/4056578676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957c101e90 msgr2=0x7f957c075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.599+0000 7f9580a00640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957c101e90 0x7f957c075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.599+0000 7f9580a00640 1 -- 172.21.15.86:0/4056578676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c075120 con 0x7f957c102fe0 2024-04-17T07:16:19.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.599+0000 7f9581400640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957c101e90 0x7f957c075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:16:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.599+0000 7f9580a00640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c102fe0 0x7f957c074350 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9568000c00 tx=0x7f9568004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.600+0000 7f957a000640 1 -- 172.21.15.86:0/4056578676 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568043020 con 0x7f957c102fe0 2024-04-17T07:16:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.600+0000 7f957a000640 1 -- 172.21.15.86:0/4056578676 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9568004aa0 con 0x7f957c102fe0 2024-04-17T07:16:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.600+0000 7f9582800640 1 -- 172.21.15.86:0/4056578676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c1a2230 con 0x7f957c102fe0 2024-04-17T07:16:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.600+0000 7f957a000640 1 -- 172.21.15.86:0/4056578676 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568003d00 con 0x7f957c102fe0 2024-04-17T07:16:19.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.600+0000 7f9582800640 1 -- 172.21.15.86:0/4056578676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c1a2780 con 0x7f957c102fe0 2024-04-17T07:16:19.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.604+0000 7f9586bb1740 1 Processor -- start 2024-04-17T07:16:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.605+0000 7f9586bb1740 1 -- start start 2024-04-17T07:16:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.605+0000 7f957a000640 1 -- 172.21.15.86:0/4056578676 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f9568046020 con 0x7f957c102fe0 2024-04-17T07:16:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.605+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cd0a0 0x55b49c1cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.605+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.605+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.605+0000 7f9586bb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b49c132cb0 con 0x55b49c1cda90 2024-04-17T07:16:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.605+0000 7f9586bb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b49c0d9370 con 0x55b49c1cd0a0 2024-04-17T07:16:19.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.605+0000 7f9586bb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b49c12ec70 con 0x55b49c1d2c00 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.605+0000 7f9581400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.606+0000 7f9581400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55594/0 (socket says 172.21.15.86:55594) 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.606+0000 7f9581400640 1 -- 172.21.15.86:0/2031196027 learned_addr learned my addr 172.21.15.86:0/2031196027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.606+0000 7f9581e00640 1 --2- 172.21.15.86:0/2031196027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cd0a0 0x55b49c1cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.606+0000 7f9581400640 1 -- 172.21.15.86:0/2031196027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 msgr2=0x55b49c1d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.606+0000 7f9581400640 1 --2- 172.21.15.86:0/2031196027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.606+0000 7f9581400640 1 -- 172.21.15.86:0/2031196027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cd0a0 msgr2=0x55b49c1cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.606+0000 7f9581400640 1 --2- 172.21.15.86:0/2031196027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cd0a0 0x55b49c1cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.606+0000 7f9581400640 1 -- 172.21.15.86:0/2031196027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49c0f8910 con 0x55b49c1cda90 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.606+0000 7f9581400640 1 --2- 172.21.15.86:0/2031196027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d2590 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f957000bd40 tx=0x7f957000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbb0b2784c9e2d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.607+0000 7f956e600640 1 -- 172.21.15.86:0/2031196027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570015070 con 0x55b49c1cda90 2024-04-17T07:16:19.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.607+0000 7f957a000640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f95640a3c40 0x7f95640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.607+0000 7f956e600640 1 -- 172.21.15.86:0/2031196027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f957000be30 con 0x55b49c1cda90 2024-04-17T07:16:19.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.607+0000 7f956e600640 1 -- 172.21.15.86:0/2031196027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570011990 con 0x55b49c1cda90 2024-04-17T07:16:19.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.607+0000 7f957a000640 1 -- 172.21.15.86:0/4056578676 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f95680f2e60 con 0x7f957c102fe0 2024-04-17T07:16:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.607+0000 7f9581e00640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f95640a3c40 0x7f95640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.608+0000 7f9586bb1740 1 -- 172.21.15.86:0/2031196027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 msgr2=0x55b49c1d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.608+0000 7f9586bb1740 1 --2- 172.21.15.86:0/2031196027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d2590 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f957000bd40 tx=0x7f957000f730 comp rx=0 tx=0).stop 2024-04-17T07:16:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.608+0000 7f9586bb1740 1 -- 172.21.15.86:0/2031196027 shutdown_connections 2024-04-17T07:16:19.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.608+0000 7f9586bb1740 1 --2- 172.21.15.86:0/2031196027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.608+0000 7f9586bb1740 1 --2- 172.21.15.86:0/2031196027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d2590 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.608+0000 7f9586bb1740 1 --2- 172.21.15.86:0/2031196027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cd0a0 0x55b49c1cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.608+0000 7f9586bb1740 1 -- 172.21.15.86:0/2031196027 >> 172.21.15.86:0/2031196027 conn(0x55b49c1c8610 msgr2=0x55b49c1c8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.608+0000 7f9586bb1740 1 -- 172.21.15.86:0/2031196027 shutdown_connections 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.608+0000 7f9586bb1740 1 -- 172.21.15.86:0/2031196027 wait complete. 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.608+0000 7f9581e00640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f95640a3c40 0x7f95640a6100 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55b49c1ce2f0 tx=0x7f9574009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.609+0000 7f9586bb1740 1 -- start start 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.609+0000 7f9586bb1740 1 Processor -- start 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.609+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1cd0a0 0x55b49c1d7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.609+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.609+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1d2c00 0x55b49c1debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.609+0000 7f9586bb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b49c1d5bb0 con 0x55b49c1cda90 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.609+0000 7f9586bb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b49c0ec730 con 0x55b49c1d2c00 2024-04-17T07:16:19.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.609+0000 7f9586bb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b49c1d5d30 con 0x55b49c1cd0a0 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9586bb1740 10 client.0 _ll_register_callbacks cb 0x55b49c0eeb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9580a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9580a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55610/0 (socket says 172.21.15.86:55610) 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9580a00640 1 -- 172.21.15.86:0/1860908803 learned_addr learned my addr 172.21.15.86:0/1860908803 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9581400640 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1cd0a0 0x55b49c1d7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9581e00640 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1d2c00 0x55b49c1debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9580a00640 1 -- 172.21.15.86:0/1860908803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1cd0a0 msgr2=0x55b49c1d7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9580a00640 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1cd0a0 0x55b49c1d7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9580a00640 1 -- 172.21.15.86:0/1860908803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1d2c00 msgr2=0x55b49c1debf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9580a00640 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1d2c00 0x55b49c1debf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.610+0000 7f9580a00640 1 -- 172.21.15.86:0/1860908803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49bf0d7e0 con 0x55b49c1cda90 2024-04-17T07:16:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.611+0000 7f9580a00640 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d7bb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f95680bb1d0 tx=0x7f95680be8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26f805742ed035 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.611+0000 7f9562a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.611+0000 7f9562a00640 1 -- 172.21.15.86:0/1860908803 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95680d8060 con 0x55b49c1cda90 2024-04-17T07:16:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.611+0000 7f9562a00640 1 -- 172.21.15.86:0/1860908803 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f95680f06e0 con 0x55b49c1cda90 2024-04-17T07:16:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.611+0000 7f9586bb1740 1 -- 172.21.15.86:0/1860908803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f957000be30 con 0x55b49c1cda90 2024-04-17T07:16:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.611+0000 7f9562a00640 1 -- 172.21.15.86:0/1860908803 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95680bed70 con 0x55b49c1cda90 2024-04-17T07:16:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.611+0000 7f9586bb1740 10 client.16071 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.611+0000 7f9586bb1740 1 -- 172.21.15.86:0/1860908803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b49bf6df70 con 0x55b49c1cda90 2024-04-17T07:16:19.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.612+0000 7f9562a00640 1 -- 172.21.15.86:0/1860908803 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f957000be30 con 0x55b49c1cda90 2024-04-17T07:16:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.612+0000 7f9562a00640 1 -- 172.21.15.86:0/1860908803 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f95680df0b0 con 0x55b49c1cda90 2024-04-17T07:16:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.612+0000 7f9586bb1740 10 client.16071 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.612+0000 7f9586bb1740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.612+0000 7f9586bb1740 20 client.16071 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.612+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b49c1e3470 0x55b49c1e5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.612+0000 7f9586bb1740 4 client.16071 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.612+0000 7f9586bb1740 1 -- 172.21.15.86:0/1860908803 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b49c1406d0 con 0x55b49c1e3470 2024-04-17T07:16:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.613+0000 7f9581400640 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b49c1e3470 0x55b49c1e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.614+0000 7f9581400640 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b49c1e3470 0x55b49c1e5860 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a6e48fd137315f server_cookie=3ca03c0485086849 in_seq=0 out_seq=0 2024-04-17T07:16:19.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.614+0000 7f9562a00640 10 client.16071 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:19.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9562a00640 1 -- 172.21.15.86:0/1860908803 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b49c1406d0 con 0x55b49c1e3470 2024-04-17T07:16:19.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9562a00640 10 client.16071 handle_command_reply: tid=0 2024-04-17T07:16:19.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9586bb1740 1 client.16071 shutdown 2024-04-17T07:16:19.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9586bb1740 1 -- 172.21.15.86:0/1860908803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 msgr2=0x55b49c1d7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d7bb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f95680bb1d0 tx=0x7f95680be8b0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9586bb1740 1 -- 172.21.15.86:0/1860908803 shutdown_connections 2024-04-17T07:16:19.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b49c1e3470 0x55b49c1e5860 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1d2c00 0x55b49c1debf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cda90 0x55b49c1d7bb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1860908803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1cd0a0 0x55b49c1d7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.617+0000 7f9586bb1740 1 -- 172.21.15.86:0/1860908803 >> 172.21.15.86:0/1860908803 conn(0x55b49c1c8610 msgr2=0x55b49c1c7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.618+0000 7f9562a00640 0 client.16071 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:19.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.618+0000 7f9586bb1740 1 -- 172.21.15.86:0/1860908803 shutdown_connections 2024-04-17T07:16:19.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.618+0000 7f9586bb1740 1 -- 172.21.15.86:0/1860908803 wait complete. 2024-04-17T07:16:19.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.618+0000 7f9586bb1740 20 client.16071 trim_cache size 0 max 16384 2024-04-17T07:16:19.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.644+0000 7f9586bb1740 1 Processor -- start 2024-04-17T07:16:19.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.644+0000 7f9586bb1740 1 -- start start 2024-04-17T07:16:19.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.645+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.645+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cda90 0x55b49c1debb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.645+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.645+0000 7f9586bb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1ac9c0 con 0x55b49c1cd0a0 2024-04-17T07:16:19.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.645+0000 7f9586bb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1ae500 con 0x55b49c1cda90 2024-04-17T07:16:19.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.645+0000 7f9586bb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1ad090 con 0x55b49c1d2c00 2024-04-17T07:16:19.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.645+0000 7f9581400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.645+0000 7f9581400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55620/0 (socket says 172.21.15.86:55620) 2024-04-17T07:16:19.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.645+0000 7f9581400640 1 -- 172.21.15.86:0/2480725817 learned_addr learned my addr 172.21.15.86:0/2480725817 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.646+0000 7f9580a00640 1 --2- 172.21.15.86:0/2480725817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cda90 0x55b49c1debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.646+0000 7f9581e00640 1 --2- 172.21.15.86:0/2480725817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.646+0000 7f9581400640 1 -- 172.21.15.86:0/2480725817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 msgr2=0x55b49c1c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.646+0000 7f9581400640 1 --2- 172.21.15.86:0/2480725817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.646+0000 7f9581400640 1 -- 172.21.15.86:0/2480725817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cda90 msgr2=0x55b49c1debb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.646+0000 7f9581400640 1 --2- 172.21.15.86:0/2480725817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cda90 0x55b49c1debb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.646+0000 7f9581400640 1 -- 172.21.15.86:0/2480725817 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49be05170 con 0x55b49c1cd0a0 2024-04-17T07:16:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.646+0000 7f9581400640 1 --2- 172.21.15.86:0/2480725817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1c7100 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f957000bd40 tx=0x7f957000b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75361f9793d168c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.647+0000 7f956dc00640 1 -- 172.21.15.86:0/2480725817 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957002e070 con 0x55b49c1cd0a0 2024-04-17T07:16:19.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.647+0000 7f956dc00640 1 -- 172.21.15.86:0/2480725817 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9570005570 con 0x55b49c1cd0a0 2024-04-17T07:16:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.647+0000 7f956dc00640 1 -- 172.21.15.86:0/2480725817 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570015040 con 0x55b49c1cd0a0 2024-04-17T07:16:19.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.648+0000 7f9586bb1740 1 -- 172.21.15.86:0/2480725817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 msgr2=0x55b49c1c7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.648+0000 7f9586bb1740 1 --2- 172.21.15.86:0/2480725817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1c7100 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f957000bd40 tx=0x7f957000b640 comp rx=0 tx=0).stop 2024-04-17T07:16:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.648+0000 7f9586bb1740 1 -- 172.21.15.86:0/2480725817 shutdown_connections 2024-04-17T07:16:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.648+0000 7f9586bb1740 1 --2- 172.21.15.86:0/2480725817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.648+0000 7f9586bb1740 1 --2- 172.21.15.86:0/2480725817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cda90 0x55b49c1debb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.648+0000 7f9586bb1740 1 --2- 172.21.15.86:0/2480725817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1c7100 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.648+0000 7f9586bb1740 1 -- 172.21.15.86:0/2480725817 >> 172.21.15.86:0/2480725817 conn(0x55b49c1c8610 msgr2=0x55b49c1d7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.648+0000 7f9586bb1740 1 -- 172.21.15.86:0/2480725817 shutdown_connections 2024-04-17T07:16:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.648+0000 7f9586bb1740 1 -- 172.21.15.86:0/2480725817 wait complete. 2024-04-17T07:16:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.648+0000 7f9586bb1740 1 -- start start 2024-04-17T07:16:19.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.649+0000 7f9586bb1740 1 Processor -- start 2024-04-17T07:16:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.649+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1dc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.649+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cda90 0x55b49c1dcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.649+0000 7f9586bb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1ddca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.649+0000 7f9586bb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1ade30 con 0x55b49c1cd0a0 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.649+0000 7f9586bb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1ad760 con 0x55b49c1cda90 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.649+0000 7f9586bb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f957c1aa990 con 0x55b49c1d2c00 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.649+0000 7f9586bb1740 10 client.0 _ll_register_callbacks cb 0x7f95680057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.650+0000 7f9581400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1dc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.650+0000 7f9581400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1dc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55634/0 (socket says 172.21.15.86:55634) 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.650+0000 7f9581400640 1 -- 172.21.15.86:0/1927886714 learned_addr learned my addr 172.21.15.86:0/1927886714 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.650+0000 7f9580a00640 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cda90 0x55b49c1dcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.650+0000 7f9581e00640 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1ddca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.650+0000 7f9581400640 1 -- 172.21.15.86:0/1927886714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 msgr2=0x55b49c1ddca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.650+0000 7f9581400640 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1ddca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.650+0000 7f9581400640 1 -- 172.21.15.86:0/1927886714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cda90 msgr2=0x55b49c1dcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.650+0000 7f9581400640 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cda90 0x55b49c1dcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.650+0000 7f9581400640 1 -- 172.21.15.86:0/1927886714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49c1de290 con 0x55b49c1cd0a0 2024-04-17T07:16:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.651+0000 7f9581400640 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1dc750 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f957000fc30 tx=0x7f957000fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfc6bf5c2398940 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.651+0000 7f9562a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.651+0000 7f9562a00640 1 -- 172.21.15.86:0/1927886714 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570038060 con 0x55b49c1cd0a0 2024-04-17T07:16:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.651+0000 7f9562a00640 1 -- 172.21.15.86:0/1927886714 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9570028170 con 0x55b49c1cd0a0 2024-04-17T07:16:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.651+0000 7f9586bb1740 1 -- 172.21.15.86:0/1927886714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9570005570 con 0x55b49c1cd0a0 2024-04-17T07:16:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.651+0000 7f9562a00640 1 -- 172.21.15.86:0/1927886714 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f957002e030 con 0x55b49c1cd0a0 2024-04-17T07:16:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.651+0000 7f9586bb1740 10 client.16077 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.651+0000 7f9586bb1740 1 -- 172.21.15.86:0/1927886714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b49c249ec0 con 0x55b49c1cd0a0 2024-04-17T07:16:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.652+0000 7f9562a00640 1 -- 172.21.15.86:0/1927886714 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9570005570 con 0x55b49c1cd0a0 2024-04-17T07:16:19.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.652+0000 7f9562a00640 1 -- 172.21.15.86:0/1927886714 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9570004bb0 con 0x55b49c1cd0a0 2024-04-17T07:16:19.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.652+0000 7f9586bb1740 10 client.16077 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:19.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.652+0000 7f9586bb1740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:19.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.652+0000 7f9586bb1740 20 client.16077 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:19.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.652+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b49c24a510 0x55b49c24c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.652+0000 7f9586bb1740 4 client.16077 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:16:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.652+0000 7f9586bb1740 1 -- 172.21.15.86:0/1927886714 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b49c1d7460 con 0x55b49c24a510 2024-04-17T07:16:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.653+0000 7f9580a00640 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b49c24a510 0x55b49c24c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.654+0000 7f9562a00640 10 client.16077 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.654+0000 7f9580a00640 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b49c24a510 0x55b49c24c900 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9700393dc35dac5b server_cookie=f1e43e9ab806f26a in_seq=0 out_seq=0 2024-04-17T07:16:19.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.656+0000 7f9562a00640 1 -- 172.21.15.86:0/1927886714 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b49c1d7460 con 0x55b49c24a510 2024-04-17T07:16:19.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.656+0000 7f9562a00640 10 client.16077 handle_command_reply: tid=0 2024-04-17T07:16:19.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.656+0000 7f9586bb1740 1 client.16077 shutdown 2024-04-17T07:16:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9586bb1740 1 -- 172.21.15.86:0/1927886714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 msgr2=0x55b49c1dc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1dc750 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f957000fc30 tx=0x7f957000fd60 comp rx=0 tx=0).stop 2024-04-17T07:16:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9586bb1740 1 -- 172.21.15.86:0/1927886714 shutdown_connections 2024-04-17T07:16:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b49c24a510 0x55b49c24c900 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b49c1d2c00 0x55b49c1ddca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9562a00640 0 client.16077 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b49c1cda90 0x55b49c1dcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9586bb1740 1 --2- 172.21.15.86:0/1927886714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b49c1cd0a0 0x55b49c1dc750 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9586bb1740 1 -- 172.21.15.86:0/1927886714 >> 172.21.15.86:0/1927886714 conn(0x55b49c1c8610 msgr2=0x55b49c1cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9586bb1740 1 -- 172.21.15.86:0/1927886714 shutdown_connections 2024-04-17T07:16:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.657+0000 7f9586bb1740 1 -- 172.21.15.86:0/1927886714 wait complete. 2024-04-17T07:16:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.658+0000 7f9586bb1740 20 client.16077 trim_cache size 0 max 16384 2024-04-17T07:16:19.658 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:16:19.659 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:16:19.659 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "2b89e546-d35c-4086-8047-17242c94bf59", 2024-04-17T07:16:19.659 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:16:19.659 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:16:19.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.658+0000 7f9582800640 1 -- 172.21.15.86:0/4056578676 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f95640a3c40 msgr2=0x7f95640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.658+0000 7f9582800640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f95640a3c40 0x7f95640a6100 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55b49c1ce2f0 tx=0x7f9574009580 comp rx=0 tx=0).stop 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.659+0000 7f9582800640 1 -- 172.21.15.86:0/4056578676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c102fe0 msgr2=0x7f957c074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.659+0000 7f9582800640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c102fe0 0x7f957c074350 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9568000c00 tx=0x7f9568004270 comp rx=0 tx=0).stop 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.659+0000 7f9582800640 1 -- 172.21.15.86:0/4056578676 shutdown_connections 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.659+0000 7f9582800640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f95640a3c40 0x7f95640a6100 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.659+0000 7f9582800640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f957c1039a0 0x7f957c074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.659+0000 7f9582800640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c102fe0 0x7f957c074350 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.659+0000 7f9582800640 1 --2- 172.21.15.86:0/4056578676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957c101e90 0x7f957c075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.659+0000 7f9582800640 1 -- 172.21.15.86:0/4056578676 >> 172.21.15.86:0/4056578676 conn(0x7f957c0fd400 msgr2=0x7f957c106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.659+0000 7f9582800640 1 -- 172.21.15.86:0/4056578676 shutdown_connections 2024-04-17T07:16:19.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.659+0000 7f9582800640 1 -- 172.21.15.86:0/4056578676 wait complete. 2024-04-17T07:16:19.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:16:19.679 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:16:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:19 smithi098 ceph-mon[35418]: pgmap v499: 129 pgs: 129 active+clean; 83 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 22 op/s 2024-04-17T07:16:19.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.897+0000 7f3a35000640 1 -- 172.21.15.86:0/3067786615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a30101040 msgr2=0x7f3a301014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.897+0000 7f3a35000640 1 --2- 172.21.15.86:0/3067786615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a30101040 0x7f3a301014a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009900 tx=0x7f3a1c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:16:19.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.897+0000 7f3a35000640 1 -- 172.21.15.86:0/3067786615 shutdown_connections 2024-04-17T07:16:19.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.897+0000 7f3a35000640 1 --2- 172.21.15.86:0/3067786615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a301019e0 0x7f3a30101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.897+0000 7f3a35000640 1 --2- 172.21.15.86:0/3067786615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a30101040 0x7f3a301014a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.897+0000 7f3a35000640 1 --2- 172.21.15.86:0/3067786615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a301006e0 0x7f3a30100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.897+0000 7f3a35000640 1 -- 172.21.15.86:0/3067786615 >> 172.21.15.86:0/3067786615 conn(0x7f3a300fc300 msgr2=0x7f3a300fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.898+0000 7f3a35000640 1 -- 172.21.15.86:0/3067786615 shutdown_connections 2024-04-17T07:16:19.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.898+0000 7f3a35000640 1 -- 172.21.15.86:0/3067786615 wait complete. 2024-04-17T07:16:19.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.899+0000 7f3a35000640 1 Processor -- start 2024-04-17T07:16:19.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.899+0000 7f3a35000640 1 -- start start 2024-04-17T07:16:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.899+0000 7f3a35000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a301006e0 0x7f3a3019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.899+0000 7f3a35000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a30101040 0x7f3a3019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.899+0000 7f3a35000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a301019e0 0x7f3a301a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.899+0000 7f3a35000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010fc10 con 0x7f3a30101040 2024-04-17T07:16:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.899+0000 7f3a35000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010fa90 con 0x7f3a301019e0 2024-04-17T07:16:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.899+0000 7f3a35000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010fd90 con 0x7f3a301006e0 2024-04-17T07:16:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.900+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a30101040 0x7f3a3019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.900+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a30101040 0x7f3a3019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55648/0 (socket says 172.21.15.86:55648) 2024-04-17T07:16:19.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.900+0000 7f3a2ea00640 1 -- 172.21.15.86:0/2674214703 learned_addr learned my addr 172.21.15.86:0/2674214703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:19.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.900+0000 7f3a2f400640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a301006e0 0x7f3a3019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.900+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a301019e0 0x7f3a301a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.901+0000 7f3a2ea00640 1 -- 172.21.15.86:0/2674214703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a301006e0 msgr2=0x7f3a3019a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.901+0000 7f3a2ea00640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a301006e0 0x7f3a3019a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.901+0000 7f3a2ea00640 1 -- 172.21.15.86:0/2674214703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a301019e0 msgr2=0x7f3a301a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.901+0000 7f3a2ea00640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a301019e0 0x7f3a301a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.901+0000 7f3a2ea00640 1 -- 172.21.15.86:0/2674214703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a301a1df0 con 0x7f3a30101040 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.901+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a301019e0 0x7f3a301a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.901+0000 7f3a2ea00640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a30101040 0x7f3a3019a760 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c000c00 tx=0x7f3a1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.901+0000 7f3a27e00640 1 -- 172.21.15.86:0/2674214703 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c043020 con 0x7f3a30101040 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.902+0000 7f3a35000640 1 -- 172.21.15.86:0/2674214703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a301a2080 con 0x7f3a30101040 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.902+0000 7f3a27e00640 1 -- 172.21.15.86:0/2674214703 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a1c004aa0 con 0x7f3a30101040 2024-04-17T07:16:19.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.902+0000 7f3a27e00640 1 -- 172.21.15.86:0/2674214703 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c003d70 con 0x7f3a30101040 2024-04-17T07:16:19.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.902+0000 7f3a35000640 1 -- 172.21.15.86:0/2674214703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a301a2620 con 0x7f3a30101040 2024-04-17T07:16:19.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.906+0000 7f3a39249740 1 Processor -- start 2024-04-17T07:16:19.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.906+0000 7f3a39249740 1 -- start start 2024-04-17T07:16:19.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.906+0000 7f3a39249740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7370 0x55c86d0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.906+0000 7f3a39249740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 0x55c86d0bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.906+0000 7f3a39249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0bced0 0x55c86d0bf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.906+0000 7f3a39249740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c86d01e120 con 0x55c86d0b7d60 2024-04-17T07:16:19.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.906+0000 7f3a39249740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c86cfc4340 con 0x55c86d0b7370 2024-04-17T07:16:19.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.906+0000 7f3a39249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c86d01a0e0 con 0x55c86d0bced0 2024-04-17T07:16:19.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.906+0000 7f3a27e00640 1 -- 172.21.15.86:0/2674214703 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138697+0+0 (secure 0 0 0) 0x7f3a1c046020 con 0x7f3a30101040 2024-04-17T07:16:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.907+0000 7f3a2f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 0x55c86d0bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.907+0000 7f3a2f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 0x55c86d0bc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55662/0 (socket says 172.21.15.86:55662) 2024-04-17T07:16:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.907+0000 7f3a2f400640 1 -- 172.21.15.86:0/4186184087 learned_addr learned my addr 172.21.15.86:0/4186184087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.907+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/4186184087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7370 0x55c86d0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.907+0000 7f3a2ea00640 1 --2- 172.21.15.86:0/4186184087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0bced0 0x55c86d0bf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.908+0000 7f3a2f400640 1 -- 172.21.15.86:0/4186184087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0bced0 msgr2=0x55c86d0bf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.908+0000 7f3a2f400640 1 --2- 172.21.15.86:0/4186184087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0bced0 0x55c86d0bf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.908+0000 7f3a2f400640 1 -- 172.21.15.86:0/4186184087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7370 msgr2=0x55c86d0b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.908+0000 7f3a2f400640 1 --2- 172.21.15.86:0/4186184087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7370 0x55c86d0b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.908+0000 7f3a2f400640 1 -- 172.21.15.86:0/4186184087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86cdfa7d0 con 0x55c86d0b7d60 2024-04-17T07:16:19.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.908+0000 7f3a2f400640 1 --2- 172.21.15.86:0/4186184087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 0x55c86d0bc860 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009c80 tx=0x7f3a200117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa100251d9d35d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.909+0000 7f3a27e00640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a140a3cc0 0x7f3a140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.909+0000 7f3a27e00640 1 -- 172.21.15.86:0/2674214703 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3a1c0f2bb0 con 0x7f3a30101040 2024-04-17T07:16:19.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.909+0000 7f3a24c00640 1 -- 172.21.15.86:0/4186184087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20012450 con 0x55c86d0b7d60 2024-04-17T07:16:19.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.909+0000 7f3a24c00640 1 -- 172.21.15.86:0/4186184087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a20012aa0 con 0x55c86d0b7d60 2024-04-17T07:16:19.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.909+0000 7f3a24c00640 1 -- 172.21.15.86:0/4186184087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000fb70 con 0x55c86d0b7d60 2024-04-17T07:16:19.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.909+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a140a3cc0 0x7f3a140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.910+0000 7f3a39249740 1 -- 172.21.15.86:0/4186184087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 msgr2=0x55c86d0bc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.910+0000 7f3a39249740 1 --2- 172.21.15.86:0/4186184087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 0x55c86d0bc860 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009c80 tx=0x7f3a200117b0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.910+0000 7f3a39249740 1 -- 172.21.15.86:0/4186184087 shutdown_connections 2024-04-17T07:16:19.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.910+0000 7f3a39249740 1 --2- 172.21.15.86:0/4186184087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0bced0 0x55c86d0bf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.910+0000 7f3a39249740 1 --2- 172.21.15.86:0/4186184087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 0x55c86d0bc860 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.910+0000 7f3a39249740 1 --2- 172.21.15.86:0/4186184087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7370 0x55c86d0b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.910+0000 7f3a39249740 1 -- 172.21.15.86:0/4186184087 >> 172.21.15.86:0/4186184087 conn(0x55c86d0b28e0 msgr2=0x55c86d0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.910+0000 7f3a39249740 1 -- 172.21.15.86:0/4186184087 shutdown_connections 2024-04-17T07:16:19.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.911+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a140a3cc0 0x7f3a140a6180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55c86d0b85c0 tx=0x7f3a28007720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.911+0000 7f3a39249740 1 -- 172.21.15.86:0/4186184087 wait complete. 2024-04-17T07:16:19.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.911+0000 7f3a39249740 1 -- start start 2024-04-17T07:16:19.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.912+0000 7f3a39249740 1 Processor -- start 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.912+0000 7f3a39249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7370 0x55c86d0ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.912+0000 7f3a39249740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 0x55c86d0ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.912+0000 7f3a39249740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d0d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.912+0000 7f3a39249740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c86d0c8690 con 0x55c86d0b7d60 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.912+0000 7f3a39249740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c86cfd7710 con 0x55c86d0bced0 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.912+0000 7f3a39249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c86d0c8810 con 0x55c86d0b7370 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a39249740 10 client.0 _ll_register_callbacks cb 0x55c86cfd2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a2ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 0x55c86d0ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d0d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d0d1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52328/0 (socket says 172.21.15.86:52328) 2024-04-17T07:16:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a2fe00640 1 -- 172.21.15.86:0/1676139830 learned_addr learned my addr 172.21.15.86:0/1676139830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a2f400640 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7370 0x55c86d0ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a2fe00640 1 -- 172.21.15.86:0/1676139830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7370 msgr2=0x55c86d0ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7370 0x55c86d0ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a2fe00640 1 -- 172.21.15.86:0/1676139830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 msgr2=0x55c86d0ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 0x55c86d0ca5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.913+0000 7f3a2fe00640 1 -- 172.21.15.86:0/1676139830 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86cd303f0 con 0x55c86d0bced0 2024-04-17T07:16:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.914+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d0d1620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a280043d0 tx=0x7f3a28004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ff3d1df00e17221 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.914+0000 7f3a19600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.914+0000 7f3a19600640 1 -- 172.21.15.86:0/1676139830 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28007c00 con 0x55c86d0bced0 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.914+0000 7f3a39249740 1 -- 172.21.15.86:0/1676139830 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a20012aa0 con 0x55c86d0bced0 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.914+0000 7f3a19600640 1 -- 172.21.15.86:0/1676139830 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3a2800fe10 con 0x55c86d0bced0 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.915+0000 7f3a19600640 1 -- 172.21.15.86:0/1676139830 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a2802e050 con 0x55c86d0bced0 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.915+0000 7f3a39249740 10 client.35276 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.915+0000 7f3a39249740 1 -- 172.21.15.86:0/1676139830 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c86ccf00f0 con 0x55c86d0bced0 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.915+0000 7f3a19600640 1 -- 172.21.15.86:0/1676139830 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a20012aa0 con 0x55c86d0bced0 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.915+0000 7f3a19600640 1 -- 172.21.15.86:0/1676139830 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3a2801c620 con 0x55c86d0bced0 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.915+0000 7f3a39249740 10 client.35276 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.915+0000 7f3a39249740 10 client.35276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.915+0000 7f3a39249740 20 client.35276 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.915+0000 7f3a39249740 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c86d0d63a0 0x55c86d0d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.916+0000 7f3a39249740 4 client.35276 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.916+0000 7f3a39249740 1 -- 172.21.15.86:0/1676139830 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c86d02bb40 con 0x55c86d0d63a0 2024-04-17T07:16:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.916+0000 7f3a2f400640 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c86d0d63a0 0x55c86d0d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.917+0000 7f3a2f400640 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c86d0d63a0 0x55c86d0d8800 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=951dcd2bda9d3738 server_cookie=dfb2df88b21b2e8d in_seq=0 out_seq=0 2024-04-17T07:16:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.917+0000 7f3a19600640 10 client.35276 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.918+0000 7f3a19600640 1 -- 172.21.15.86:0/1676139830 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c86d02bb40 con 0x55c86d0d63a0 2024-04-17T07:16:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.918+0000 7f3a19600640 10 client.35276 handle_command_reply: tid=0 2024-04-17T07:16:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.918+0000 7f3a39249740 1 client.35276 shutdown 2024-04-17T07:16:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 1 -- 172.21.15.86:0/1676139830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 msgr2=0x55c86d0d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d0d1620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a280043d0 tx=0x7f3a28004a40 comp rx=0 tx=0).stop 2024-04-17T07:16:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 1 -- 172.21.15.86:0/1676139830 shutdown_connections 2024-04-17T07:16:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c86d0d63a0 0x55c86d0d8800 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d0d1620 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7d60 0x55c86d0ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 1 --2- 172.21.15.86:0/1676139830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7370 0x55c86d0ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 1 -- 172.21.15.86:0/1676139830 >> 172.21.15.86:0/1676139830 conn(0x55c86d0b28e0 msgr2=0x55c86d0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a19600640 0 client.35276 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 1 -- 172.21.15.86:0/1676139830 shutdown_connections 2024-04-17T07:16:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 1 -- 172.21.15.86:0/1676139830 wait complete. 2024-04-17T07:16:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.919+0000 7f3a39249740 20 client.35276 trim_cache size 0 max 16384 2024-04-17T07:16:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.941+0000 7f3a39249740 1 Processor -- start 2024-04-17T07:16:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.941+0000 7f3a39249740 1 -- start start 2024-04-17T07:16:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.941+0000 7f3a39249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7370 0x55c86d0b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.941+0000 7f3a39249740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7d60 0x55c86d0b9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.941+0000 7f3a39249740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0bced0 0x55c86d0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.941+0000 7f3a39249740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301a8610 con 0x55c86d0bced0 2024-04-17T07:16:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.941+0000 7f3a39249740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301aa150 con 0x55c86d0b7d60 2024-04-17T07:16:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.941+0000 7f3a39249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301a8ce0 con 0x55c86d0b7370 2024-04-17T07:16:19.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0bced0 0x55c86d0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0bced0 0x55c86d0b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55682/0 (socket says 172.21.15.86:55682) 2024-04-17T07:16:19.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2fe00640 1 -- 172.21.15.86:0/1626603525 learned_addr learned my addr 172.21.15.86:0/1626603525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:19.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2ea00640 1 --2- 172.21.15.86:0/1626603525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7d60 0x55c86d0b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2f400640 1 --2- 172.21.15.86:0/1626603525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7370 0x55c86d0b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2fe00640 1 -- 172.21.15.86:0/1626603525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7370 msgr2=0x55c86d0b92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/1626603525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7370 0x55c86d0b92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2fe00640 1 -- 172.21.15.86:0/1626603525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7d60 msgr2=0x55c86d0b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/1626603525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7d60 0x55c86d0b9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2fe00640 1 -- 172.21.15.86:0/1626603525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86d0d1a40 con 0x55c86d0bced0 2024-04-17T07:16:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2ea00640 1 --2- 172.21.15.86:0/1626603525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7d60 0x55c86d0b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:16:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.942+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/1626603525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0bced0 0x55c86d0b1250 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3a28011d40 tx=0x7f3a28007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18178843018a55a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.943+0000 7f3a1be00640 1 -- 172.21.15.86:0/1626603525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800fc10 con 0x55c86d0bced0 2024-04-17T07:16:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.943+0000 7f3a1be00640 1 -- 172.21.15.86:0/1626603525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a28036030 con 0x55c86d0bced0 2024-04-17T07:16:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.943+0000 7f3a1be00640 1 -- 172.21.15.86:0/1626603525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2801d8d0 con 0x55c86d0bced0 2024-04-17T07:16:19.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.944+0000 7f3a39249740 1 -- 172.21.15.86:0/1626603525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0bced0 msgr2=0x55c86d0b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.944+0000 7f3a39249740 1 --2- 172.21.15.86:0/1626603525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0bced0 0x55c86d0b1250 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3a28011d40 tx=0x7f3a28007c80 comp rx=0 tx=0).stop 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.944+0000 7f3a39249740 1 -- 172.21.15.86:0/1626603525 shutdown_connections 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.944+0000 7f3a39249740 1 --2- 172.21.15.86:0/1626603525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0bced0 0x55c86d0b1250 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.944+0000 7f3a39249740 1 --2- 172.21.15.86:0/1626603525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0b7d60 0x55c86d0b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.944+0000 7f3a39249740 1 --2- 172.21.15.86:0/1626603525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7370 0x55c86d0b92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.944+0000 7f3a39249740 1 -- 172.21.15.86:0/1626603525 >> 172.21.15.86:0/1626603525 conn(0x55c86d0b28e0 msgr2=0x55c86d0ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.944+0000 7f3a39249740 1 -- 172.21.15.86:0/1626603525 shutdown_connections 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.944+0000 7f3a39249740 1 -- 172.21.15.86:0/1626603525 wait complete. 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.944+0000 7f3a39249740 1 -- start start 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a39249740 1 Processor -- start 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a39249740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7370 0x55c86d0dd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a39249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7d60 0x55c86d0dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a39249740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d13ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a39249740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301a9a80 con 0x55c86d0b7370 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a39249740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301a93b0 con 0x55c86d0bced0 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a39249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a301a6600 con 0x55c86d0b7d60 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a39249740 10 client.0 _ll_register_callbacks cb 0x7f3a28011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a2f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7370 0x55c86d0dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a2f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7370 0x55c86d0dd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55698/0 (socket says 172.21.15.86:55698) 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.945+0000 7f3a2f400640 1 -- 172.21.15.86:0/547195532 learned_addr learned my addr 172.21.15.86:0/547195532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d13ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a2ea00640 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7d60 0x55c86d0dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a2f400640 1 -- 172.21.15.86:0/547195532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7d60 msgr2=0x55c86d0dded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a2f400640 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7d60 0x55c86d0dded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a2f400640 1 -- 172.21.15.86:0/547195532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 msgr2=0x55c86d13ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:19.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a2f400640 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d13ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a2f400640 1 -- 172.21.15.86:0/547195532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86d13b280 con 0x55c86d0b7370 2024-04-17T07:16:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a2fe00640 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d13ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:16:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a2f400640 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7370 0x55c86d0dd990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3a2001c040 tx=0x7f3a20005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37a3d394eac925a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a19600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a19600640 1 -- 172.21.15.86:0/547195532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20030060 con 0x55c86d0b7370 2024-04-17T07:16:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a39249740 1 -- 172.21.15.86:0/547195532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a28036030 con 0x55c86d0b7370 2024-04-17T07:16:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.946+0000 7f3a19600640 1 -- 172.21.15.86:0/547195532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3a2002a1d0 con 0x55c86d0b7370 2024-04-17T07:16:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a19600640 1 -- 172.21.15.86:0/547195532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a20037050 con 0x55c86d0b7370 2024-04-17T07:16:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a39249740 10 client.16095 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a39249740 1 -- 172.21.15.86:0/547195532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c86d13c600 con 0x55c86d0b7370 2024-04-17T07:16:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a19600640 1 -- 172.21.15.86:0/547195532 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a28036030 con 0x55c86d0b7370 2024-04-17T07:16:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a19600640 1 -- 172.21.15.86:0/547195532 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3a20003960 con 0x55c86d0b7370 2024-04-17T07:16:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a39249740 10 client.16095 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a39249740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a39249740 20 client.16095 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a39249740 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c86d13cea0 0x55c86d13f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a39249740 4 client.16095 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.947+0000 7f3a39249740 1 -- 172.21.15.86:0/547195532 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c86d0ae140 con 0x55c86d13cea0 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.948+0000 7f3a2ea00640 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c86d13cea0 0x55c86d13f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.949+0000 7f3a19600640 10 client.16095 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.949+0000 7f3a2ea00640 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c86d13cea0 0x55c86d13f2c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62adb27ce88cfaf3 server_cookie=33397ce9b98a917c in_seq=0 out_seq=0 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.949+0000 7f3a19600640 1 -- 172.21.15.86:0/547195532 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c86d0ae140 con 0x55c86d13cea0 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.949+0000 7f3a19600640 10 client.16095 handle_command_reply: tid=0 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.949+0000 7f3a39249740 1 client.16095 shutdown 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 1 -- 172.21.15.86:0/547195532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7370 msgr2=0x55c86d0dd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7370 0x55c86d0dd990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3a2001c040 tx=0x7f3a20005f80 comp rx=0 tx=0).stop 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 1 -- 172.21.15.86:0/547195532 shutdown_connections 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c86d13cea0 0x55c86d13f2c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c86d0bced0 0x55c86d13ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a19600640 0 client.16095 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c86d0b7d60 0x55c86d0dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 1 --2- 172.21.15.86:0/547195532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86d0b7370 0x55c86d0dd990 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 1 -- 172.21.15.86:0/547195532 >> 172.21.15.86:0/547195532 conn(0x55c86d0b28e0 msgr2=0x55c86d0b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 1 -- 172.21.15.86:0/547195532 shutdown_connections 2024-04-17T07:16:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 1 -- 172.21.15.86:0/547195532 wait complete. 2024-04-17T07:16:19.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.950+0000 7f3a39249740 20 client.16095 trim_cache size 0 max 16384 2024-04-17T07:16:19.953 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:16:19.953 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:16:19.953 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:16:19.954 INFO:teuthology.orchestra.run.smithi086.stdout: "2b89e546-d35c-4086-8047-17242c94bf59": { 2024-04-17T07:16:19.954 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:16:19.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "2b89e546-d35c-4086-8047-17242c94bf59", 2024-04-17T07:16:19.954 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:16:19.954 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:16:19.954 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:16:19.954 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:16:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 -- 172.21.15.86:0/2674214703 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a140a3cc0 msgr2=0x7f3a140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a140a3cc0 0x7f3a140a6180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55c86d0b85c0 tx=0x7f3a28007720 comp rx=0 tx=0).stop 2024-04-17T07:16:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 -- 172.21.15.86:0/2674214703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a30101040 msgr2=0x7f3a3019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a30101040 0x7f3a3019a760 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c000c00 tx=0x7f3a1c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 -- 172.21.15.86:0/2674214703 shutdown_connections 2024-04-17T07:16:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a140a3cc0 0x7f3a140a6180 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a301019e0 0x7f3a301a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a30101040 0x7f3a3019a760 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 --2- 172.21.15.86:0/2674214703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a301006e0 0x7f3a3019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:19.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 -- 172.21.15.86:0/2674214703 >> 172.21.15.86:0/2674214703 conn(0x7f3a300fc300 msgr2=0x7f3a300fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:19.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.951+0000 7f3a35000640 1 -- 172.21.15.86:0/2674214703 shutdown_connections 2024-04-17T07:16:19.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:19.952+0000 7f3a35000640 1 -- 172.21.15.86:0/2674214703 wait complete. 2024-04-17T07:16:19.969 INFO:tasks.cephfs.filesystem:scrub status for tag:2b89e546-d35c-4086-8047-17242c94bf59 - {'path': '/', 'tag': '2b89e546-d35c-4086-8047-17242c94bf59', 'options': 'recursive,force'} 2024-04-17T07:16:20.221 INFO:teuthology.orchestra.run.smithi086.stderr:300000 of 3200000 tuples (9%) done (elapsed 9.77 s, remaining 94.41 s) 2024-04-17T07:16:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:20 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:16:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:20 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:20 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:16:20.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:20 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:20 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:16:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:20 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:20 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:16:20.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:20 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:20 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:16:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:20 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:20 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:16:20.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:20 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:21.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:21 smithi103 ceph-mon[35495]: pgmap v500: 129 pgs: 129 active+clean; 91 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 23 op/s 2024-04-17T07:16:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:21 smithi086 ceph-mon[31733]: pgmap v500: 129 pgs: 129 active+clean; 91 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 23 op/s 2024-04-17T07:16:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:21 smithi098 ceph-mon[35418]: pgmap v500: 129 pgs: 129 active+clean; 91 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 23 op/s 2024-04-17T07:16:23.990 INFO:teuthology.orchestra.run.smithi086.stderr:400000 of 3200000 tuples (12%) done (elapsed 13.54 s, remaining 94.75 s) 2024-04-17T07:16:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:24 smithi098 ceph-mon[35418]: pgmap v501: 129 pgs: 129 active+clean; 115 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.9 MiB/s wr, 23 op/s 2024-04-17T07:16:24.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:24 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:16:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:24 smithi103 ceph-mon[35495]: pgmap v501: 129 pgs: 129 active+clean; 115 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.9 MiB/s wr, 23 op/s 2024-04-17T07:16:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:24 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:16:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:24 smithi086 ceph-mon[31733]: pgmap v501: 129 pgs: 129 active+clean; 115 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.9 MiB/s wr, 23 op/s 2024-04-17T07:16:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:24 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:16:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:26 smithi098 ceph-mon[35418]: pgmap v502: 129 pgs: 129 active+clean; 127 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-04-17T07:16:26.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:26 smithi103 ceph-mon[35495]: pgmap v502: 129 pgs: 129 active+clean; 127 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-04-17T07:16:26.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:26 smithi086 ceph-mon[31733]: pgmap v502: 129 pgs: 129 active+clean; 127 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-04-17T07:16:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:27 smithi086 ceph-mon[31733]: pgmap v503: 129 pgs: 129 active+clean; 135 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:16:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:27 smithi098 ceph-mon[35418]: pgmap v503: 129 pgs: 129 active+clean; 135 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:16:27.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:27 smithi103 ceph-mon[35495]: pgmap v503: 129 pgs: 129 active+clean; 135 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:16:29.081 INFO:teuthology.orchestra.run.smithi086.stderr:500000 of 3200000 tuples (15%) done (elapsed 18.63 s, remaining 100.58 s) 2024-04-17T07:16:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:29 smithi086 ceph-mon[31733]: mgrmap e20: x(active, since 14m), standbys: z, y 2024-04-17T07:16:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:29 smithi086 ceph-mon[31733]: pgmap v504: 129 pgs: 129 active+clean; 151 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-04-17T07:16:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:29 smithi098 ceph-mon[35418]: mgrmap e20: x(active, since 14m), standbys: z, y 2024-04-17T07:16:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:29 smithi098 ceph-mon[35418]: pgmap v504: 129 pgs: 129 active+clean; 151 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-04-17T07:16:29.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:29 smithi103 ceph-mon[35495]: mgrmap e20: x(active, since 14m), standbys: z, y 2024-04-17T07:16:29.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:29 smithi103 ceph-mon[35495]: pgmap v504: 129 pgs: 129 active+clean; 151 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-04-17T07:16:32.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:32 smithi098 ceph-mon[35418]: pgmap v505: 129 pgs: 129 active+clean; 151 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:16:32.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:32 smithi103 ceph-mon[35495]: pgmap v505: 129 pgs: 129 active+clean; 151 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:16:32.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:32 smithi086 ceph-mon[31733]: pgmap v505: 129 pgs: 129 active+clean; 151 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:16:33.296 INFO:teuthology.orchestra.run.smithi086.stderr:600000 of 3200000 tuples (18%) done (elapsed 22.84 s, remaining 98.98 s) 2024-04-17T07:16:34.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:34 smithi098 ceph-mon[35418]: pgmap v506: 129 pgs: 129 active+clean; 167 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:16:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:34 smithi103 ceph-mon[35495]: pgmap v506: 129 pgs: 129 active+clean; 167 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:16:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:34 smithi086 ceph-mon[31733]: pgmap v506: 129 pgs: 129 active+clean; 167 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:16:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:35 smithi103 ceph-mon[35495]: pgmap v507: 129 pgs: 129 active+clean; 171 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.7 MiB/s wr, 17 op/s 2024-04-17T07:16:35.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:35 smithi086 ceph-mon[31733]: pgmap v507: 129 pgs: 129 active+clean; 171 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.7 MiB/s wr, 17 op/s 2024-04-17T07:16:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:35 smithi098 ceph-mon[35418]: pgmap v507: 129 pgs: 129 active+clean; 171 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.7 MiB/s wr, 17 op/s 2024-04-17T07:16:38.081 INFO:teuthology.orchestra.run.smithi086.stderr:700000 of 3200000 tuples (21%) done (elapsed 27.63 s, remaining 98.67 s) 2024-04-17T07:16:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:38 smithi098 ceph-mon[35418]: pgmap v508: 129 pgs: 129 active+clean; 187 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:16:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:38 smithi103 ceph-mon[35495]: pgmap v508: 129 pgs: 129 active+clean; 187 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:16:38.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:38 smithi086 ceph-mon[31733]: pgmap v508: 129 pgs: 129 active+clean; 187 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:16:39.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:39 smithi086 ceph-mon[31733]: pgmap v509: 129 pgs: 129 active+clean; 195 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:16:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:39 smithi098 ceph-mon[35418]: pgmap v509: 129 pgs: 129 active+clean; 195 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:16:39.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:39 smithi103 ceph-mon[35495]: pgmap v509: 129 pgs: 129 active+clean; 195 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:16:42.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:42 smithi098 ceph-mon[35418]: pgmap v510: 129 pgs: 129 active+clean; 199 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 18 op/s 2024-04-17T07:16:42.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:42 smithi103 ceph-mon[35495]: pgmap v510: 129 pgs: 129 active+clean; 199 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 18 op/s 2024-04-17T07:16:42.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:42 smithi086 ceph-mon[31733]: pgmap v510: 129 pgs: 129 active+clean; 199 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.1 MiB/s wr, 18 op/s 2024-04-17T07:16:43.026 INFO:teuthology.orchestra.run.smithi086.stderr:800000 of 3200000 tuples (25%) done (elapsed 32.57 s, remaining 97.72 s) 2024-04-17T07:16:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:43 smithi098 ceph-mon[35418]: pgmap v511: 129 pgs: 129 active+clean; 213 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:16:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:43 smithi103 ceph-mon[35495]: pgmap v511: 129 pgs: 129 active+clean; 213 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:16:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:43 smithi086 ceph-mon[31733]: pgmap v511: 129 pgs: 129 active+clean; 213 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:16:46.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:46 smithi098 ceph-mon[35418]: pgmap v512: 129 pgs: 129 active+clean; 225 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 MiB/s wr, 16 op/s 2024-04-17T07:16:46.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:46 smithi103 ceph-mon[35495]: pgmap v512: 129 pgs: 129 active+clean; 225 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 MiB/s wr, 16 op/s 2024-04-17T07:16:46.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:46 smithi086 ceph-mon[31733]: pgmap v512: 129 pgs: 129 active+clean; 225 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 MiB/s wr, 16 op/s 2024-04-17T07:16:47.227 INFO:teuthology.orchestra.run.smithi086.stderr:900000 of 3200000 tuples (28%) done (elapsed 36.77 s, remaining 93.97 s) 2024-04-17T07:16:47.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:47 smithi103 ceph-mon[35495]: pgmap v513: 129 pgs: 129 active+clean; 235 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:16:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:47 smithi086 ceph-mon[31733]: pgmap v513: 129 pgs: 129 active+clean; 235 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:16:47.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:47 smithi098 ceph-mon[35418]: pgmap v513: 129 pgs: 129 active+clean; 235 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:16:49.555 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:49 smithi098 ceph-mon[35418]: pgmap v514: 129 pgs: 129 active+clean; 247 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-04-17T07:16:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:49 smithi086 ceph-mon[31733]: pgmap v514: 129 pgs: 129 active+clean; 247 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-04-17T07:16:49.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:49 smithi103 ceph-mon[35495]: pgmap v514: 129 pgs: 129 active+clean; 247 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 18 op/s 2024-04-17T07:16:49.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:16:49.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:16:50.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.214+0000 7fbebce00640 1 -- 172.21.15.86:0/53581485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb8075f40 msgr2=0x7fbeb8074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.214+0000 7fbebce00640 1 --2- 172.21.15.86:0/53581485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb8075f40 0x7fbeb8074390 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009900 tx=0x7fbea802bcd0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.215+0000 7fbebce00640 1 -- 172.21.15.86:0/53581485 shutdown_connections 2024-04-17T07:16:50.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.215+0000 7fbebce00640 1 --2- 172.21.15.86:0/53581485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeb8109d00 0x7fbeb810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.215+0000 7fbebce00640 1 --2- 172.21.15.86:0/53581485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeb8074960 0x7fbeb8074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.215+0000 7fbebce00640 1 --2- 172.21.15.86:0/53581485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb8075f40 0x7fbeb8074390 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.215+0000 7fbebce00640 1 -- 172.21.15.86:0/53581485 >> 172.21.15.86:0/53581485 conn(0x7fbeb80fd400 msgr2=0x7fbeb80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:50.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.215+0000 7fbebce00640 1 -- 172.21.15.86:0/53581485 shutdown_connections 2024-04-17T07:16:50.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.215+0000 7fbebce00640 1 -- 172.21.15.86:0/53581485 wait complete. 2024-04-17T07:16:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.216+0000 7fbebce00640 1 Processor -- start 2024-04-17T07:16:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.216+0000 7fbebce00640 1 -- start start 2024-04-17T07:16:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.216+0000 7fbebce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeb8074960 0x7fbeb819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.216+0000 7fbebce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeb8075f40 0x7fbeb819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.216+0000 7fbebce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb8109d00 0x7fbeb81a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.216+0000 7fbebce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb810fc40 con 0x7fbeb8109d00 2024-04-17T07:16:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.216+0000 7fbebce00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb810fac0 con 0x7fbeb8075f40 2024-04-17T07:16:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.216+0000 7fbebce00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb810fdc0 con 0x7fbeb8074960 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb8109d00 0x7fbeb81a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb8109d00 0x7fbeb81a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34752/0 (socket says 172.21.15.86:34752) 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb7e00640 1 -- 172.21.15.86:0/3706347822 learned_addr learned my addr 172.21.15.86:0/3706347822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb7400640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeb8074960 0x7fbeb819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeb8075f40 0x7fbeb819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb7e00640 1 -- 172.21.15.86:0/3706347822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeb8074960 msgr2=0x7fbeb819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeb8074960 0x7fbeb819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb7e00640 1 -- 172.21.15.86:0/3706347822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeb8075f40 msgr2=0x7fbeb819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeb8075f40 0x7fbeb819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb7e00640 1 -- 172.21.15.86:0/3706347822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb81a1f30 con 0x7fbeb8109d00 2024-04-17T07:16:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.217+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeb8075f40 0x7fbeb819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:50.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.218+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb8109d00 0x7fbeb81a18c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000cad0 tx=0x7fbeb000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:50.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.218+0000 7fbeafe00640 1 -- 172.21.15.86:0/3706347822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb000ae20 con 0x7fbeb8109d00 2024-04-17T07:16:50.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.218+0000 7fbeafe00640 1 -- 172.21.15.86:0/3706347822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbeb0005020 con 0x7fbeb8109d00 2024-04-17T07:16:50.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.218+0000 7fbebce00640 1 -- 172.21.15.86:0/3706347822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb81a2220 con 0x7fbeb8109d00 2024-04-17T07:16:50.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.218+0000 7fbeafe00640 1 -- 172.21.15.86:0/3706347822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb000f6b0 con 0x7fbeb8109d00 2024-04-17T07:16:50.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.218+0000 7fbebce00640 1 -- 172.21.15.86:0/3706347822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb81a27f0 con 0x7fbeb8109d00 2024-04-17T07:16:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.222+0000 7fbec1212740 1 Processor -- start 2024-04-17T07:16:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.222+0000 7fbeafe00640 1 -- 172.21.15.86:0/3706347822 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fbeb00051d0 con 0x7fbeb8109d00 2024-04-17T07:16:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.222+0000 7fbec1212740 1 -- start start 2024-04-17T07:16:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.223+0000 7fbec1212740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f760b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.223+0000 7fbec1212740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f7610860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.223+0000 7fbec1212740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7613360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.223+0000 7fbec1212740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c7f7572120 con 0x55c7f7610ed0 2024-04-17T07:16:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.223+0000 7fbec1212740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c7f7518340 con 0x55c7f760bd60 2024-04-17T07:16:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.223+0000 7fbec1212740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c7f756e0e0 con 0x55c7f760b370 2024-04-17T07:16:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.223+0000 7fbeb6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7613360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.223+0000 7fbeb6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7613360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34754/0 (socket says 172.21.15.86:34754) 2024-04-17T07:16:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.223+0000 7fbeb6a00640 1 -- 172.21.15.86:0/3610007569 learned_addr learned my addr 172.21.15.86:0/3610007569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.223+0000 7fbeb7400640 1 --2- 172.21.15.86:0/3610007569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f7610860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.224+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/3610007569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f760b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.224+0000 7fbeb6a00640 1 -- 172.21.15.86:0/3610007569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 msgr2=0x55c7f760b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.224+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/3610007569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f760b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.224+0000 7fbeb6a00640 1 -- 172.21.15.86:0/3610007569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 msgr2=0x55c7f7610860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.224+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/3610007569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f7610860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.224+0000 7fbeb6a00640 1 -- 172.21.15.86:0/3610007569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f734e7d0 con 0x55c7f7610ed0 2024-04-17T07:16:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.224+0000 7fbeafe00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbe9c0a3c30 0x7fbe9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.224+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/3610007569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7613360 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbea000cfd0 tx=0x7fbea000c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad4ddaaeef75413 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.225+0000 7fbeafe00640 1 -- 172.21.15.86:0/3706347822 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbeb00d0830 con 0x7fbeb8109d00 2024-04-17T07:16:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.225+0000 7fbeacc00640 1 -- 172.21.15.86:0/3610007569 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea001c070 con 0x55c7f7610ed0 2024-04-17T07:16:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.225+0000 7fbeb7400640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbe9c0a3c30 0x7fbe9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.225+0000 7fbeacc00640 1 -- 172.21.15.86:0/3610007569 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbea0017ac0 con 0x55c7f7610ed0 2024-04-17T07:16:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.225+0000 7fbeacc00640 1 -- 172.21.15.86:0/3610007569 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea000f040 con 0x55c7f7610ed0 2024-04-17T07:16:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.226+0000 7fbec1212740 1 -- 172.21.15.86:0/3610007569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 msgr2=0x55c7f7613360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.226+0000 7fbeb7400640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbe9c0a3c30 0x7fbe9c0a60f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbea8006010 tx=0x7fbea802c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.226+0000 7fbec1212740 1 --2- 172.21.15.86:0/3610007569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7613360 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbea000cfd0 tx=0x7fbea000c230 comp rx=0 tx=0).stop 2024-04-17T07:16:50.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.226+0000 7fbec1212740 1 -- 172.21.15.86:0/3610007569 shutdown_connections 2024-04-17T07:16:50.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.226+0000 7fbec1212740 1 --2- 172.21.15.86:0/3610007569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7613360 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.226+0000 7fbec1212740 1 --2- 172.21.15.86:0/3610007569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f7610860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.226+0000 7fbec1212740 1 --2- 172.21.15.86:0/3610007569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f760b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.226+0000 7fbec1212740 1 -- 172.21.15.86:0/3610007569 >> 172.21.15.86:0/3610007569 conn(0x55c7f76068e0 msgr2=0x55c7f7606ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:50.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.226+0000 7fbec1212740 1 -- 172.21.15.86:0/3610007569 shutdown_connections 2024-04-17T07:16:50.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.226+0000 7fbec1212740 1 -- 172.21.15.86:0/3610007569 wait complete. 2024-04-17T07:16:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.227+0000 7fbec1212740 1 -- start start 2024-04-17T07:16:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.227+0000 7fbec1212740 1 Processor -- start 2024-04-17T07:16:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbec1212740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760b370 0x55c7f76158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbec1212740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760bd60 0x55c7f7615e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbec1212740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbec1212740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c7f7613e80 con 0x55c7f7610ed0 2024-04-17T07:16:50.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbec1212740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c7f752b710 con 0x55c7f760b370 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbec1212740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c7f7614000 con 0x55c7f760bd60 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbec1212740 10 client.0 _ll_register_callbacks cb 0x55c7f7526af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbeb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbeb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34764/0 (socket says 172.21.15.86:34764) 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbeb7e00640 1 -- 172.21.15.86:0/2600749782 learned_addr learned my addr 172.21.15.86:0/2600749782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760bd60 0x55c7f7615e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.228+0000 7fbeb7400640 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760b370 0x55c7f76158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbeb7e00640 1 -- 172.21.15.86:0/2600749782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760bd60 msgr2=0x55c7f7615e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760bd60 0x55c7f7615e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbeb7e00640 1 -- 172.21.15.86:0/2600749782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760b370 msgr2=0x55c7f76158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760b370 0x55c7f76158f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbeb7e00640 1 -- 172.21.15.86:0/2600749782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f72843f0 con 0x55c7f7610ed0 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761ce70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00a4ed0 tx=0x7fbeb009cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98699fb81b700b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbea5600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbea5600640 1 -- 172.21.15.86:0/2600749782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb001d040 con 0x55c7f7610ed0 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbea5600640 1 -- 172.21.15.86:0/2600749782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbeb0098e00 con 0x55c7f7610ed0 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbec1212740 1 -- 172.21.15.86:0/2600749782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea0017ac0 con 0x55c7f7610ed0 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.229+0000 7fbea5600640 1 -- 172.21.15.86:0/2600749782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbeb00a5400 con 0x55c7f7610ed0 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.230+0000 7fbec1212740 10 client.16107 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.230+0000 7fbec1212740 1 -- 172.21.15.86:0/2600749782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7f72440f0 con 0x55c7f7610ed0 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.230+0000 7fbea5600640 1 -- 172.21.15.86:0/2600749782 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbea0017ac0 con 0x55c7f7610ed0 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.230+0000 7fbea5600640 1 -- 172.21.15.86:0/2600749782 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fbeb0098550 con 0x55c7f7610ed0 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.230+0000 7fbec1212740 10 client.16107 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.230+0000 7fbec1212740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.230+0000 7fbec1212740 20 client.16107 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.230+0000 7fbec1212740 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c7f76217f0 0x55c7f7623c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.230+0000 7fbec1212740 4 client.16107 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.230+0000 7fbec1212740 1 -- 172.21.15.86:0/2600749782 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7f757fb40 con 0x55c7f76217f0 2024-04-17T07:16:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.231+0000 7fbeb7400640 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c7f76217f0 0x55c7f7623c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.232+0000 7fbeb7400640 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c7f76217f0 0x55c7f7623c20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99927c3559e9b109 server_cookie=edd7347ee015efd0 in_seq=0 out_seq=0 2024-04-17T07:16:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.232+0000 7fbea5600640 10 client.16107 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbea5600640 1 -- 172.21.15.86:0/2600749782 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7f757fb40 con 0x55c7f76217f0 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbea5600640 10 client.16107 handle_command_reply: tid=0 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 client.16107 shutdown 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 -- 172.21.15.86:0/2600749782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 msgr2=0x55c7f761ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761ce70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00a4ed0 tx=0x7fbeb009cf50 comp rx=0 tx=0).stop 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 -- 172.21.15.86:0/2600749782 shutdown_connections 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c7f76217f0 0x55c7f7623c20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761ce70 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbea5600640 0 client.16107 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760bd60 0x55c7f7615e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 --2- 172.21.15.86:0/2600749782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760b370 0x55c7f76158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 -- 172.21.15.86:0/2600749782 >> 172.21.15.86:0/2600749782 conn(0x55c7f76068e0 msgr2=0x55c7f7605ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 -- 172.21.15.86:0/2600749782 shutdown_connections 2024-04-17T07:16:50.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.234+0000 7fbec1212740 1 -- 172.21.15.86:0/2600749782 wait complete. 2024-04-17T07:16:50.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.235+0000 7fbec1212740 20 client.16107 trim_cache size 0 max 16384 2024-04-17T07:16:50.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.259+0000 7fbec1212740 1 Processor -- start 2024-04-17T07:16:50.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.259+0000 7fbec1212740 1 -- start start 2024-04-17T07:16:50.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.259+0000 7fbec1212740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f760c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.259+0000 7fbec1212740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f761ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.259+0000 7fbec1212740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7605250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.260+0000 7fbec1212740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81acb40 con 0x55c7f7610ed0 2024-04-17T07:16:50.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.260+0000 7fbec1212740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ae680 con 0x55c7f760bd60 2024-04-17T07:16:50.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.260+0000 7fbec1212740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ad210 con 0x55c7f760b370 2024-04-17T07:16:50.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.260+0000 7fbeb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7605250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.260+0000 7fbeb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7605250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34776/0 (socket says 172.21.15.86:34776) 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.260+0000 7fbeb7e00640 1 -- 172.21.15.86:0/2323213880 learned_addr learned my addr 172.21.15.86:0/2323213880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.260+0000 7fbeb7400640 1 --2- 172.21.15.86:0/2323213880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f760c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.260+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/2323213880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f761ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.261+0000 7fbeb7e00640 1 -- 172.21.15.86:0/2323213880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 msgr2=0x55c7f760c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.261+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/2323213880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f760c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.261+0000 7fbeb7e00640 1 -- 172.21.15.86:0/2323213880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 msgr2=0x55c7f761ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.261+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/2323213880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f761ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.261+0000 7fbeb7e00640 1 -- 172.21.15.86:0/2323213880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f7607650 con 0x55c7f7610ed0 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.261+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/2323213880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f761ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.261+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/2323213880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7605250 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00ce000 tx=0x7fbeb009aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e843b059c9fd64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.261+0000 7fbea7e00640 1 -- 172.21.15.86:0/2323213880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb001d070 con 0x55c7f7610ed0 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.261+0000 7fbea7e00640 1 -- 172.21.15.86:0/2323213880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbeb007f070 con 0x55c7f7610ed0 2024-04-17T07:16:50.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.261+0000 7fbea7e00640 1 -- 172.21.15.86:0/2323213880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0098740 con 0x55c7f7610ed0 2024-04-17T07:16:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.262+0000 7fbec1212740 1 -- 172.21.15.86:0/2323213880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 msgr2=0x55c7f7605250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.262+0000 7fbec1212740 1 --2- 172.21.15.86:0/2323213880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7605250 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00ce000 tx=0x7fbeb009aca0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.262+0000 7fbec1212740 1 -- 172.21.15.86:0/2323213880 shutdown_connections 2024-04-17T07:16:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.262+0000 7fbec1212740 1 --2- 172.21.15.86:0/2323213880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f7605250 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.262+0000 7fbec1212740 1 --2- 172.21.15.86:0/2323213880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f761ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.262+0000 7fbec1212740 1 --2- 172.21.15.86:0/2323213880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f760c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.262+0000 7fbec1212740 1 -- 172.21.15.86:0/2323213880 >> 172.21.15.86:0/2323213880 conn(0x55c7f76068e0 msgr2=0x55c7f76143a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:50.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.262+0000 7fbec1212740 1 -- 172.21.15.86:0/2323213880 shutdown_connections 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.262+0000 7fbec1212740 1 -- 172.21.15.86:0/2323213880 wait complete. 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.263+0000 7fbec1212740 1 -- start start 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.263+0000 7fbec1212740 1 Processor -- start 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbec1212740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f761a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbec1212740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f761ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbec1212740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbec1212740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81adfb0 con 0x55c7f7610ed0 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbec1212740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81ad8e0 con 0x55c7f760bd60 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbec1212740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb81aab10 con 0x55c7f760b370 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbec1212740 10 client.0 _ll_register_callbacks cb 0x7fbeb0098370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34782/0 (socket says 172.21.15.86:34782) 2024-04-17T07:16:50.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb7e00640 1 -- 172.21.15.86:0/658129631 learned_addr learned my addr 172.21.15.86:0/658129631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb7400640 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f761a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f761ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb7e00640 1 -- 172.21.15.86:0/658129631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 msgr2=0x55c7f761a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f761a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb7e00640 1 -- 172.21.15.86:0/658129631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 msgr2=0x55c7f761ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f761ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb7e00640 1 -- 172.21.15.86:0/658129631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f761c440 con 0x55c7f7610ed0 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.264+0000 7fbeb6a00640 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f761ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.265+0000 7fbeb7e00640 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761be50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00a55c0 tx=0x7fbeb0098990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac583236f33c1ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.265+0000 7fbea5600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.265+0000 7fbea5600640 1 -- 172.21.15.86:0/658129631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb001d070 con 0x55c7f7610ed0 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.265+0000 7fbea5600640 1 -- 172.21.15.86:0/658129631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbeb00bb640 con 0x55c7f7610ed0 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.265+0000 7fbec1212740 1 -- 172.21.15.86:0/658129631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb007f070 con 0x55c7f7610ed0 2024-04-17T07:16:50.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.265+0000 7fbea5600640 1 -- 172.21.15.86:0/658129631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbeb00b0050 con 0x55c7f7610ed0 2024-04-17T07:16:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.265+0000 7fbec1212740 10 client.16113 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:50.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.265+0000 7fbec1212740 1 -- 172.21.15.86:0/658129631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7f7688140 con 0x55c7f7610ed0 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.266+0000 7fbea5600640 1 -- 172.21.15.86:0/658129631 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbeb007f070 con 0x55c7f7610ed0 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.266+0000 7fbea5600640 1 -- 172.21.15.86:0/658129631 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fbeb00a1df0 con 0x55c7f7610ed0 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.266+0000 7fbec1212740 10 client.16113 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.266+0000 7fbec1212740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.266+0000 7fbec1212740 20 client.16113 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.266+0000 7fbec1212740 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c7f7688a00 0x55c7f768ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.266+0000 7fbec1212740 4 client.16113 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.266+0000 7fbec1212740 1 -- 172.21.15.86:0/658129631 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c7f75799b0 con 0x55c7f7688a00 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.267+0000 7fbeb7400640 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c7f7688a00 0x55c7f768ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.267+0000 7fbeb7400640 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c7f7688a00 0x55c7f768ae20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f3790ed1567016 server_cookie=dbe698069076f914 in_seq=0 out_seq=0 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.267+0000 7fbea5600640 10 client.16113 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:50.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbea5600640 1 -- 172.21.15.86:0/658129631 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c7f75799b0 con 0x55c7f7688a00 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbea5600640 10 client.16113 handle_command_reply: tid=0 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 client.16113 shutdown 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 -- 172.21.15.86:0/658129631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 msgr2=0x55c7f761be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761be50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00a55c0 tx=0x7fbeb0098990 comp rx=0 tx=0).stop 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 -- 172.21.15.86:0/658129631 shutdown_connections 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c7f7688a00 0x55c7f768ae20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7f7610ed0 0x55c7f761be50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c7f760bd60 0x55c7f761ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 --2- 172.21.15.86:0/658129631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c7f760b370 0x55c7f761a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 -- 172.21.15.86:0/658129631 >> 172.21.15.86:0/658129631 conn(0x55c7f76068e0 msgr2=0x55c7f7609b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbea5600640 0 client.16113 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 -- 172.21.15.86:0/658129631 shutdown_connections 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.268+0000 7fbec1212740 1 -- 172.21.15.86:0/658129631 wait complete. 2024-04-17T07:16:50.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbec1212740 20 client.16113 trim_cache size 0 max 16384 2024-04-17T07:16:50.272 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:16:50.272 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:16:50.272 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:16:50.272 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:16:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbebce00640 1 -- 172.21.15.86:0/3706347822 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbe9c0a3c30 msgr2=0x7fbe9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbebce00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbe9c0a3c30 0x7fbe9c0a60f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbea8006010 tx=0x7fbea802c880 comp rx=0 tx=0).stop 2024-04-17T07:16:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbebce00640 1 -- 172.21.15.86:0/3706347822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb8109d00 msgr2=0x7fbeb81a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbebce00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb8109d00 0x7fbeb81a18c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbeb000cad0 tx=0x7fbeb000cf90 comp rx=0 tx=0).stop 2024-04-17T07:16:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbebce00640 1 -- 172.21.15.86:0/3706347822 shutdown_connections 2024-04-17T07:16:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbebce00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbe9c0a3c30 0x7fbe9c0a60f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbebce00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbeb8109d00 0x7fbeb81a18c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbebce00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbeb8075f40 0x7fbeb819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbebce00640 1 --2- 172.21.15.86:0/3706347822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbeb8074960 0x7fbeb819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.269+0000 7fbebce00640 1 -- 172.21.15.86:0/3706347822 >> 172.21.15.86:0/3706347822 conn(0x7fbeb80fd400 msgr2=0x7fbeb8105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:50.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.270+0000 7fbebce00640 1 -- 172.21.15.86:0/3706347822 shutdown_connections 2024-04-17T07:16:50.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.270+0000 7fbebce00640 1 -- 172.21.15.86:0/3706347822 wait complete. 2024-04-17T07:16:50.288 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:16:50.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:16:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.509+0000 7f9509000640 1 -- 172.21.15.86:0/1634237284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9504105360 msgr2=0x7f9504107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.509+0000 7f9509000640 1 --2- 172.21.15.86:0/1634237284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9504105360 0x7f9504107750 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009900 tx=0x7f94ec02bc30 comp rx=0 tx=0).stop 2024-04-17T07:16:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.509+0000 7f9509000640 1 -- 172.21.15.86:0/1634237284 shutdown_connections 2024-04-17T07:16:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.509+0000 7f9509000640 1 --2- 172.21.15.86:0/1634237284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9504107c90 0x7f950410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.509+0000 7f9509000640 1 --2- 172.21.15.86:0/1634237284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9504105360 0x7f9504107750 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.509+0000 7f9509000640 1 --2- 172.21.15.86:0/1634237284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9504102a30 0x7f9504104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.509+0000 7f9509000640 1 -- 172.21.15.86:0/1634237284 >> 172.21.15.86:0/1634237284 conn(0x7f95040fc440 msgr2=0x7f95040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.509+0000 7f9509000640 1 -- 172.21.15.86:0/1634237284 shutdown_connections 2024-04-17T07:16:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.509+0000 7f9509000640 1 -- 172.21.15.86:0/1634237284 wait complete. 2024-04-17T07:16:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.510+0000 7f9509000640 1 Processor -- start 2024-04-17T07:16:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.510+0000 7f9509000640 1 -- start start 2024-04-17T07:16:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f9509000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9504102a30 0x7f95041023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f9509000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9504105360 0x7f95041009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f9509000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9504107c90 0x7f9504100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f9509000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f950410db40 con 0x7f9504105360 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f9509000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f950410d9c0 con 0x7f9504102a30 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f9509000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f950410dcc0 con 0x7f9504107c90 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9504105360 0x7f95041009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f94fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9504105360 0x7f95041009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34802/0 (socket says 172.21.15.86:34802) 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f94fbe00640 1 -- 172.21.15.86:0/2118114857 learned_addr learned my addr 172.21.15.86:0/2118114857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f9501e00640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9504102a30 0x7f95041023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.511+0000 7f9502800640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9504107c90 0x7f9504100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.512+0000 7f94fbe00640 1 -- 172.21.15.86:0/2118114857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9504107c90 msgr2=0x7f9504100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.512+0000 7f94fbe00640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9504107c90 0x7f9504100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.512+0000 7f94fbe00640 1 -- 172.21.15.86:0/2118114857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9504102a30 msgr2=0x7f95041023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.512+0000 7f94fbe00640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9504102a30 0x7f95041023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.512+0000 7f94fbe00640 1 -- 172.21.15.86:0/2118114857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95041017b0 con 0x7f9504105360 2024-04-17T07:16:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.512+0000 7f94fbe00640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9504105360 0x7f95041009e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f94ec000c00 tx=0x7f94ec0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.512+0000 7f94faa00640 1 -- 172.21.15.86:0/2118114857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec043020 con 0x7f9504105360 2024-04-17T07:16:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.512+0000 7f9509000640 1 -- 172.21.15.86:0/2118114857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95041a2340 con 0x7f9504105360 2024-04-17T07:16:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.512+0000 7f94faa00640 1 -- 172.21.15.86:0/2118114857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94ec0044c0 con 0x7f9504105360 2024-04-17T07:16:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.513+0000 7f9509000640 1 -- 172.21.15.86:0/2118114857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95041a28e0 con 0x7f9504105360 2024-04-17T07:16:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.513+0000 7f94faa00640 1 -- 172.21.15.86:0/2118114857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec003c30 con 0x7f9504105360 2024-04-17T07:16:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.514+0000 7f9509000640 1 -- 172.21.15.86:0/2118114857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95040fdf30 con 0x7f9504105360 2024-04-17T07:16:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.516+0000 7f94faa00640 1 -- 172.21.15.86:0/2118114857 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f94ec046020 con 0x7f9504105360 2024-04-17T07:16:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.518+0000 7f94faa00640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94e80a3bb0 0x7f94e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.518+0000 7f94faa00640 1 -- 172.21.15.86:0/2118114857 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f94ec0f29e0 con 0x7f9504105360 2024-04-17T07:16:50.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.518+0000 7f9501e00640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94e80a3bb0 0x7f94e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:50.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.519+0000 7f9501e00640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94e80a3bb0 0x7f94e80a6070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009bd0 tx=0x7f94f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:50.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.524+0000 7f94faa00640 1 -- 172.21.15.86:0/2118114857 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f94ec04d020 con 0x7f9504105360 2024-04-17T07:16:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.798+0000 7f9509000640 1 -- 172.21.15.86:0/2118114857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f950402d5d0 con 0x7f9504105360 2024-04-17T07:16:50.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.800+0000 7f94faa00640 1 -- 172.21.15.86:0/2118114857 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f94ec0befe0 con 0x7f9504105360 2024-04-17T07:16:50.800 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:16:50.800 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:16:50.801 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:16:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 -- 172.21.15.86:0/2118114857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94e80a3bb0 msgr2=0x7f94e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94e80a3bb0 0x7f94e80a6070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009bd0 tx=0x7f94f4005f90 comp rx=0 tx=0).stop 2024-04-17T07:16:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 -- 172.21.15.86:0/2118114857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9504105360 msgr2=0x7f95041009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9504105360 0x7f95041009e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f94ec000c00 tx=0x7f94ec0040c0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 -- 172.21.15.86:0/2118114857 shutdown_connections 2024-04-17T07:16:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94e80a3bb0 0x7f94e80a6070 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9504107c90 0x7f9504100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9504105360 0x7f95041009e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 --2- 172.21.15.86:0/2118114857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9504102a30 0x7f95041023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 -- 172.21.15.86:0/2118114857 >> 172.21.15.86:0/2118114857 conn(0x7f95040fc440 msgr2=0x7f95041035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 -- 172.21.15.86:0/2118114857 shutdown_connections 2024-04-17T07:16:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:50.803+0000 7f9509000640 1 -- 172.21.15.86:0/2118114857 wait complete. 2024-04-17T07:16:50.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:16:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.056+0000 7f29c1c00640 1 -- 172.21.15.86:0/3831929778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29bc105d90 msgr2=0x7f29bc10c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.056+0000 7f29c1c00640 1 --2- 172.21.15.86:0/3831929778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29bc105d90 0x7f29bc10c620 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009ca0 tx=0x7f29b402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.056+0000 7f29c1c00640 1 -- 172.21.15.86:0/3831929778 shutdown_connections 2024-04-17T07:16:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.056+0000 7f29c1c00640 1 --2- 172.21.15.86:0/3831929778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29bc105d90 0x7f29bc10c620 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.056+0000 7f29c1c00640 1 --2- 172.21.15.86:0/3831929778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc1053d0 0x7f29bc105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.056+0000 7f29c1c00640 1 --2- 172.21.15.86:0/3831929778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29bc1041e0 0x7f29bc104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.056+0000 7f29c1c00640 1 -- 172.21.15.86:0/3831929778 >> 172.21.15.86:0/3831929778 conn(0x7f29bc0ff750 msgr2=0x7f29bc101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.056+0000 7f29c1c00640 1 -- 172.21.15.86:0/3831929778 shutdown_connections 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.056+0000 7f29c1c00640 1 -- 172.21.15.86:0/3831929778 wait complete. 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.057+0000 7f29c1c00640 1 Processor -- start 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29c1c00640 1 -- start start 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29c1c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc1041e0 0x7f29bc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29c1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29bc1053d0 0x7f29bc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29c1c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29bc105d90 0x7f29bc1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29c1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc10fd20 con 0x7f29bc1053d0 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29c1c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc10fba0 con 0x7f29bc1041e0 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29c1c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc10fea0 con 0x7f29bc105d90 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29bc1053d0 0x7f29bc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29bc1053d0 0x7f29bc19a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34808/0 (socket says 172.21.15.86:34808) 2024-04-17T07:16:51.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29ba000640 1 -- 172.21.15.86:0/526552034 learned_addr learned my addr 172.21.15.86:0/526552034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29baa00640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc1041e0 0x7f29bc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.058+0000 7f29bb400640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29bc105d90 0x7f29bc1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.059+0000 7f29ba000640 1 -- 172.21.15.86:0/526552034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29bc105d90 msgr2=0x7f29bc1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.059+0000 7f29ba000640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29bc105d90 0x7f29bc1a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.059+0000 7f29ba000640 1 -- 172.21.15.86:0/526552034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc1041e0 msgr2=0x7f29bc19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.059+0000 7f29ba000640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc1041e0 0x7f29bc19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.059+0000 7f29ba000640 1 -- 172.21.15.86:0/526552034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc1a1fc0 con 0x7f29bc1053d0 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.059+0000 7f29ba000640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29bc1053d0 0x7f29bc19a8e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e9f0 tx=0x7f29a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.059+0000 7f29b3400640 1 -- 172.21.15.86:0/526552034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a800cd80 con 0x7f29bc1053d0 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.059+0000 7f29b3400640 1 -- 172.21.15.86:0/526552034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29a80108e0 con 0x7f29bc1053d0 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.059+0000 7f29c1c00640 1 -- 172.21.15.86:0/526552034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc1a22b0 con 0x7f29bc1053d0 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.059+0000 7f29b3400640 1 -- 172.21.15.86:0/526552034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8018920 con 0x7f29bc1053d0 2024-04-17T07:16:51.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.060+0000 7f29c1c00640 1 -- 172.21.15.86:0/526552034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc1a2800 con 0x7f29bc1053d0 2024-04-17T07:16:51.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.063+0000 7f29c47f8740 1 Processor -- start 2024-04-17T07:16:51.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.064+0000 7f29c47f8740 1 -- start start 2024-04-17T07:16:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.064+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367846ffa0 0x5636784703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.064+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563678470990 0x563678475490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.064+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678475b00 0x563678477f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.064+0000 7f29c47f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5636783d6020 con 0x56367846ffa0 2024-04-17T07:16:51.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.064+0000 7f29c47f8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56367837c460 con 0x563678475b00 2024-04-17T07:16:51.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.064+0000 7f29c47f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5636783d1fe0 con 0x563678470990 2024-04-17T07:16:51.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.064+0000 7f29b3400640 1 -- 172.21.15.86:0/526552034 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f29a8010a90 con 0x7f29bc1053d0 2024-04-17T07:16:51.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.065+0000 7f29bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367846ffa0 0x5636784703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.065+0000 7f29bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367846ffa0 0x5636784703c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34820/0 (socket says 172.21.15.86:34820) 2024-04-17T07:16:51.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.065+0000 7f29bb400640 1 -- 172.21.15.86:0/4116211188 learned_addr learned my addr 172.21.15.86:0/4116211188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.065+0000 7f29baa00640 1 --2- 172.21.15.86:0/4116211188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563678470990 0x563678475490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.065+0000 7f29bb400640 1 -- 172.21.15.86:0/4116211188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563678470990 msgr2=0x563678475490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.065+0000 7f29bb400640 1 --2- 172.21.15.86:0/4116211188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563678470990 0x563678475490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.065+0000 7f29bb400640 1 -- 172.21.15.86:0/4116211188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678475b00 msgr2=0x563678477f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.065+0000 7f29bb400640 1 --2- 172.21.15.86:0/4116211188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678475b00 0x563678477f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.065+0000 7f29bb400640 1 -- 172.21.15.86:0/4116211188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563678384040 con 0x56367846ffa0 2024-04-17T07:16:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.065+0000 7f29bb400640 1 --2- 172.21.15.86:0/4116211188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367846ffa0 0x5636784703c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f29b4005e30 tx=0x7f29b4005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb049d8b4c0dbdd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.066+0000 7f29a7e00640 1 -- 172.21.15.86:0/4116211188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4043020 con 0x56367846ffa0 2024-04-17T07:16:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.066+0000 7f29a7e00640 1 -- 172.21.15.86:0/4116211188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29b4032db0 con 0x56367846ffa0 2024-04-17T07:16:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.066+0000 7f29a7e00640 1 -- 172.21.15.86:0/4116211188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4039040 con 0x56367846ffa0 2024-04-17T07:16:51.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.066+0000 7f29b3400640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29a00a3d00 0x7f29a00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29b3400640 1 -- 172.21.15.86:0/526552034 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f29a80d0ab0 con 0x7f29bc1053d0 2024-04-17T07:16:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29baa00640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29a00a3d00 0x7f29a00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29c47f8740 1 -- 172.21.15.86:0/4116211188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367846ffa0 msgr2=0x5636784703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29c47f8740 1 --2- 172.21.15.86:0/4116211188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367846ffa0 0x5636784703c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f29b4005e30 tx=0x7f29b4005f80 comp rx=0 tx=0).stop 2024-04-17T07:16:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29c47f8740 1 -- 172.21.15.86:0/4116211188 shutdown_connections 2024-04-17T07:16:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29c47f8740 1 --2- 172.21.15.86:0/4116211188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678475b00 0x563678477f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29c47f8740 1 --2- 172.21.15.86:0/4116211188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563678470990 0x563678475490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29c47f8740 1 --2- 172.21.15.86:0/4116211188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367846ffa0 0x5636784703c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29c47f8740 1 -- 172.21.15.86:0/4116211188 >> 172.21.15.86:0/4116211188 conn(0x56367846b510 msgr2=0x56367846b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29c47f8740 1 -- 172.21.15.86:0/4116211188 shutdown_connections 2024-04-17T07:16:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.067+0000 7f29c47f8740 1 -- 172.21.15.86:0/4116211188 wait complete. 2024-04-17T07:16:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.068+0000 7f29baa00640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29a00a3d00 0x7f29a00a61c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x56367846c480 tx=0x7f29ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.068+0000 7f29c47f8740 1 -- start start 2024-04-17T07:16:51.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.068+0000 7f29c47f8740 1 Processor -- start 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678470990 0x56367847aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x563678481a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29c47f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563678478a50 con 0x563678475b00 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29c47f8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56367838f830 con 0x563678470990 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29c47f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563678478bd0 con 0x56367846ffa0 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29c47f8740 10 client.0 _ll_register_callbacks cb 0x56367835dfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x563678481a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x563678481a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34828/0 (socket says 172.21.15.86:34828) 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29bb400640 1 -- 172.21.15.86:0/3955354223 learned_addr learned my addr 172.21.15.86:0/3955354223 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.069+0000 7f29ba000640 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678470990 0x56367847aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29baa00640 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29bb400640 1 -- 172.21.15.86:0/3955354223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 msgr2=0x56367847a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29bb400640 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29bb400640 1 -- 172.21.15.86:0/3955354223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678470990 msgr2=0x56367847aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29bb400640 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678470990 0x56367847aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29bb400640 1 -- 172.21.15.86:0/3955354223 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563678393920 con 0x563678475b00 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29baa00640 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29bb400640 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x563678481a30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009bf0 tx=0x7f29b4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffcaeb38c70fe7f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29a4c00640 1 -- 172.21.15.86:0/3955354223 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4043030 con 0x563678475b00 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29a4c00640 1 -- 172.21.15.86:0/3955354223 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f29b4003fd0 con 0x563678475b00 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.070+0000 7f29c47f8740 1 -- 172.21.15.86:0/3955354223 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b4032db0 con 0x563678475b00 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.071+0000 7f29a4c00640 1 -- 172.21.15.86:0/3955354223 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29b4058050 con 0x563678475b00 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.071+0000 7f29c47f8740 10 client.16131 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.071+0000 7f29c47f8740 1 -- 172.21.15.86:0/3955354223 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636783617f0 con 0x563678475b00 2024-04-17T07:16:51.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.071+0000 7f29a4c00640 1 -- 172.21.15.86:0/3955354223 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29b4032db0 con 0x563678475b00 2024-04-17T07:16:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.072+0000 7f29a4c00640 1 -- 172.21.15.86:0/3955354223 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f29b403f870 con 0x563678475b00 2024-04-17T07:16:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.072+0000 7f29c47f8740 10 client.16131 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.072+0000 7f29c47f8740 10 client.16131 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:16:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.072+0000 7f29c47f8740 20 client.16131 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.072+0000 7f29c47f8740 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x563678485c30 0x563678488060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.072+0000 7f29c47f8740 4 client.16131 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.072+0000 7f29c47f8740 1 -- 172.21.15.86:0/3955354223 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636783e3a40 con 0x563678485c30 2024-04-17T07:16:51.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.072+0000 7f29baa00640 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x563678485c30 0x563678488060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.074+0000 7f29a4c00640 10 client.16131 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:16:51.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.074+0000 7f29baa00640 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x563678485c30 0x563678488060 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=440a0cd9cf7c4fc2 server_cookie=9d186cc1e8b922c3 in_seq=0 out_seq=0 2024-04-17T07:16:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29a4c00640 1 -- 172.21.15.86:0/3955354223 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636783e3a40 con 0x563678485c30 2024-04-17T07:16:51.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29a4c00640 10 client.16131 handle_command_reply: tid=0 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29c47f8740 1 client.16131 shutdown 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29c47f8740 1 -- 172.21.15.86:0/3955354223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 msgr2=0x563678481a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29c47f8740 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x563678481a30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009bf0 tx=0x7f29b4005c50 comp rx=0 tx=0).stop 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29c47f8740 1 -- 172.21.15.86:0/3955354223 shutdown_connections 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29c47f8740 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x563678485c30 0x563678488060 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29c47f8740 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x563678481a30 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29c47f8740 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678470990 0x56367847aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29c47f8740 1 --2- 172.21.15.86:0/3955354223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29c47f8740 1 -- 172.21.15.86:0/3955354223 >> 172.21.15.86:0/3955354223 conn(0x56367846b510 msgr2=0x56367846a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.076+0000 7f29a4c00640 0 client.16131 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.077+0000 7f29c47f8740 1 -- 172.21.15.86:0/3955354223 shutdown_connections 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.077+0000 7f29c47f8740 1 -- 172.21.15.86:0/3955354223 wait complete. 2024-04-17T07:16:51.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.077+0000 7f29c47f8740 20 client.16131 trim_cache size 0 max 16384 2024-04-17T07:16:51.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:50 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2118114857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:16:51.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.092+0000 7f29c47f8740 1 Processor -- start 2024-04-17T07:16:51.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.093+0000 7f29c47f8740 1 -- start start 2024-04-17T07:16:51.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.093+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.093+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678470990 0x5636784819f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.093+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x56367847aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.093+0000 7f29c47f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc1acc80 con 0x563678475b00 2024-04-17T07:16:51.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.093+0000 7f29c47f8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc1ae7c0 con 0x563678470990 2024-04-17T07:16:51.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.093+0000 7f29c47f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc1ad350 con 0x56367846ffa0 2024-04-17T07:16:51.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.094+0000 7f29bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x56367847aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.094+0000 7f29bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x56367847aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34840/0 (socket says 172.21.15.86:34840) 2024-04-17T07:16:51.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.094+0000 7f29bb400640 1 -- 172.21.15.86:0/2095332339 learned_addr learned my addr 172.21.15.86:0/2095332339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.094+0000 7f29baa00640 1 --2- 172.21.15.86:0/2095332339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.094+0000 7f29bb400640 1 -- 172.21.15.86:0/2095332339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 msgr2=0x56367847a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.094+0000 7f29bb400640 1 --2- 172.21.15.86:0/2095332339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.094+0000 7f29bb400640 1 -- 172.21.15.86:0/2095332339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678470990 msgr2=0x5636784819f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.094+0000 7f29bb400640 1 --2- 172.21.15.86:0/2095332339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678470990 0x5636784819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.094+0000 7f29bb400640 1 -- 172.21.15.86:0/2095332339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636780e83f0 con 0x563678475b00 2024-04-17T07:16:51.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.095+0000 7f29bb400640 1 --2- 172.21.15.86:0/2095332339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x56367847aa60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f29b402c630 tx=0x7f29b4005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f073187266d0926 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.095+0000 7f29a7400640 1 -- 172.21.15.86:0/2095332339 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4043040 con 0x563678475b00 2024-04-17T07:16:51.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.095+0000 7f29a7400640 1 -- 172.21.15.86:0/2095332339 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29b4032800 con 0x563678475b00 2024-04-17T07:16:51.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.095+0000 7f29a7400640 1 -- 172.21.15.86:0/2095332339 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4039050 con 0x563678475b00 2024-04-17T07:16:51.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.096+0000 7f29c47f8740 1 -- 172.21.15.86:0/2095332339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 msgr2=0x56367847aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.096+0000 7f29c47f8740 1 --2- 172.21.15.86:0/2095332339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x56367847aa60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f29b402c630 tx=0x7f29b4005f80 comp rx=0 tx=0).stop 2024-04-17T07:16:51.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.096+0000 7f29c47f8740 1 -- 172.21.15.86:0/2095332339 shutdown_connections 2024-04-17T07:16:51.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.096+0000 7f29c47f8740 1 --2- 172.21.15.86:0/2095332339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678475b00 0x56367847aa60 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.096+0000 7f29c47f8740 1 --2- 172.21.15.86:0/2095332339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678470990 0x5636784819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.096+0000 7f29c47f8740 1 --2- 172.21.15.86:0/2095332339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.096+0000 7f29c47f8740 1 -- 172.21.15.86:0/2095332339 >> 172.21.15.86:0/2095332339 conn(0x56367846b510 msgr2=0x563678471f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.096+0000 7f29c47f8740 1 -- 172.21.15.86:0/2095332339 shutdown_connections 2024-04-17T07:16:51.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.096+0000 7f29c47f8740 1 -- 172.21.15.86:0/2095332339 wait complete. 2024-04-17T07:16:51.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.097+0000 7f29c47f8740 1 -- start start 2024-04-17T07:16:51.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.097+0000 7f29c47f8740 1 Processor -- start 2024-04-17T07:16:51.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678470990 0x56367847ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29c47f8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678475b00 0x5636784ea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29c47f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc1ae0f0 con 0x563678470990 2024-04-17T07:16:51.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29c47f8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc1ada20 con 0x563678475b00 2024-04-17T07:16:51.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29c47f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc1aac50 con 0x56367846ffa0 2024-04-17T07:16:51.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29c47f8740 10 client.0 _ll_register_callbacks cb 0x7f29b4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:51.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678470990 0x56367847ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678470990 0x56367847ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34848/0 (socket says 172.21.15.86:34848) 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29ba000640 1 -- 172.21.15.86:0/1823147546 learned_addr learned my addr 172.21.15.86:0/1823147546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.098+0000 7f29baa00640 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.099+0000 7f29bb400640 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678475b00 0x5636784ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.099+0000 7f29ba000640 1 -- 172.21.15.86:0/1823147546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 msgr2=0x56367847e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.099+0000 7f29ba000640 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.099+0000 7f29ba000640 1 -- 172.21.15.86:0/1823147546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678475b00 msgr2=0x5636784ea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.099+0000 7f29ba000640 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678475b00 0x5636784ea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.099+0000 7f29ba000640 1 -- 172.21.15.86:0/1823147546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636781b27d0 con 0x563678470990 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.099+0000 7f29ba000640 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678470990 0x56367847ebb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f29a80ce230 tx=0x7f29a809c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11deb65f21b3a871 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.099+0000 7f29a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.099+0000 7f29a4c00640 1 -- 172.21.15.86:0/1823147546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a80a53f0 con 0x563678470990 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.099+0000 7f29a4c00640 1 -- 172.21.15.86:0/1823147546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f29a8010430 con 0x563678470990 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29c47f8740 1 -- 172.21.15.86:0/1823147546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b4032800 con 0x563678470990 2024-04-17T07:16:51.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29a4c00640 1 -- 172.21.15.86:0/1823147546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29a809ca10 con 0x563678470990 2024-04-17T07:16:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29c47f8740 10 client.16137 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29c47f8740 1 -- 172.21.15.86:0/1823147546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636784eb620 con 0x563678470990 2024-04-17T07:16:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29a4c00640 1 -- 172.21.15.86:0/1823147546 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29b4032800 con 0x563678470990 2024-04-17T07:16:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29a4c00640 1 -- 172.21.15.86:0/1823147546 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f29a80988d0 con 0x563678470990 2024-04-17T07:16:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29c47f8740 10 client.16137 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29c47f8740 10 client.16137 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:16:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29c47f8740 20 client.16137 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29c47f8740 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5636784ebfb0 0x5636784ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29c47f8740 4 client.16137 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:16:51.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.100+0000 7f29c47f8740 1 -- 172.21.15.86:0/1823147546 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636783dd8b0 con 0x5636784ebfb0 2024-04-17T07:16:51.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.101+0000 7f29baa00640 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5636784ebfb0 0x5636784ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.102+0000 7f29baa00640 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5636784ebfb0 0x5636784ee3d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=617a0bf0a1653140 server_cookie=1aab240c48eb2e1f in_seq=0 out_seq=0 2024-04-17T07:16:51.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.102+0000 7f29a4c00640 10 client.16137 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29a4c00640 1 -- 172.21.15.86:0/1823147546 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636783dd8b0 con 0x5636784ebfb0 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29a4c00640 10 client.16137 handle_command_reply: tid=0 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29c47f8740 1 client.16137 shutdown 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29c47f8740 1 -- 172.21.15.86:0/1823147546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678470990 msgr2=0x56367847ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29c47f8740 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678470990 0x56367847ebb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f29a80ce230 tx=0x7f29a809c430 comp rx=0 tx=0).stop 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29c47f8740 1 -- 172.21.15.86:0/1823147546 shutdown_connections 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29c47f8740 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5636784ebfb0 0x5636784ee3d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29c47f8740 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563678475b00 0x5636784ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29c47f8740 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563678470990 0x56367847ebb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29c47f8740 1 --2- 172.21.15.86:0/1823147546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56367846ffa0 0x56367847e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29c47f8740 1 -- 172.21.15.86:0/1823147546 >> 172.21.15.86:0/1823147546 conn(0x56367846b510 msgr2=0x56367847a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.103+0000 7f29a4c00640 0 client.16137 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.104+0000 7f29c47f8740 1 -- 172.21.15.86:0/1823147546 shutdown_connections 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.104+0000 7f29c47f8740 1 -- 172.21.15.86:0/1823147546 wait complete. 2024-04-17T07:16:51.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.104+0000 7f29c47f8740 20 client.16137 trim_cache size 0 max 16384 2024-04-17T07:16:51.105 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.104+0000 7f29c1c00640 1 -- 172.21.15.86:0/526552034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29a00a3d00 msgr2=0x7f29a00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.104+0000 7f29c1c00640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29a00a3d00 0x7f29a00a61c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x56367846c480 tx=0x7f29ac005f90 comp rx=0 tx=0).stop 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.104+0000 7f29c1c00640 1 -- 172.21.15.86:0/526552034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29bc1053d0 msgr2=0x7f29bc19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.104+0000 7f29c1c00640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29bc1053d0 0x7f29bc19a8e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e9f0 tx=0x7f29a800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.105+0000 7f29c1c00640 1 -- 172.21.15.86:0/526552034 shutdown_connections 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.105+0000 7f29c1c00640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29a00a3d00 0x7f29a00a61c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.105+0000 7f29c1c00640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29bc105d90 0x7f29bc1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.105+0000 7f29c1c00640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29bc1053d0 0x7f29bc19a8e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.105+0000 7f29c1c00640 1 --2- 172.21.15.86:0/526552034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc1041e0 0x7f29bc19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.105+0000 7f29c1c00640 1 -- 172.21.15.86:0/526552034 >> 172.21.15.86:0/526552034 conn(0x7f29bc0ff750 msgr2=0x7f29bc1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.105+0000 7f29c1c00640 1 -- 172.21.15.86:0/526552034 shutdown_connections 2024-04-17T07:16:51.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.105+0000 7f29c1c00640 1 -- 172.21.15.86:0/526552034 wait complete. 2024-04-17T07:16:51.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:16:51.207 INFO:teuthology.orchestra.run.smithi086.stderr:1000000 of 3200000 tuples (31%) done (elapsed 40.75 s, remaining 89.66 s) 2024-04-17T07:16:51.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:50 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2118114857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:16:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:50 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2118114857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:16:51.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.360+0000 7f35f3e00640 1 -- 172.21.15.86:0/1848951305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f4101f60 msgr2=0x7f35f41023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.360+0000 7f35f3e00640 1 --2- 172.21.15.86:0/1848951305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f4101f60 0x7f35f41023e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009980 tx=0x7f35dc02de90 comp rx=0 tx=0).stop 2024-04-17T07:16:51.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.360+0000 7f35f3e00640 1 -- 172.21.15.86:0/1848951305 shutdown_connections 2024-04-17T07:16:51.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.360+0000 7f35f3e00640 1 --2- 172.21.15.86:0/1848951305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35f4102920 0x7f35f41091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.360+0000 7f35f3e00640 1 --2- 172.21.15.86:0/1848951305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f4101f60 0x7f35f41023e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.360+0000 7f35f3e00640 1 --2- 172.21.15.86:0/1848951305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f4100d70 0x7f35f4101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.360+0000 7f35f3e00640 1 -- 172.21.15.86:0/1848951305 >> 172.21.15.86:0/1848951305 conn(0x7f35f40fc300 msgr2=0x7f35f40fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.360+0000 7f35f3e00640 1 -- 172.21.15.86:0/1848951305 shutdown_connections 2024-04-17T07:16:51.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.360+0000 7f35f3e00640 1 -- 172.21.15.86:0/1848951305 wait complete. 2024-04-17T07:16:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.361+0000 7f35f3e00640 1 Processor -- start 2024-04-17T07:16:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35f3e00640 1 -- start start 2024-04-17T07:16:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35f3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f4100d70 0x7f35f419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f4101f60 0x7f35f419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35f3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35f4102920 0x7f35f41a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35f3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35f410b7f0 con 0x7f35f4101f60 2024-04-17T07:16:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35f3e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f35f410b670 con 0x7f35f4100d70 2024-04-17T07:16:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35f3e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f35f410b970 con 0x7f35f4102920 2024-04-17T07:16:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f4101f60 0x7f35f419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f4101f60 0x7f35f419a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34878/0 (socket says 172.21.15.86:34878) 2024-04-17T07:16:51.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35ea000640 1 -- 172.21.15.86:0/533037433 learned_addr learned my addr 172.21.15.86:0/533037433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35f2a00640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f4100d70 0x7f35f419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.362+0000 7f35f3400640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35f4102920 0x7f35f41a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.363+0000 7f35ea000640 1 -- 172.21.15.86:0/533037433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35f4102920 msgr2=0x7f35f41a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.363+0000 7f35ea000640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35f4102920 0x7f35f41a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.363+0000 7f35ea000640 1 -- 172.21.15.86:0/533037433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f4100d70 msgr2=0x7f35f419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.363+0000 7f35ea000640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f4100d70 0x7f35f419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.363+0000 7f35ea000640 1 -- 172.21.15.86:0/533037433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f41a1ee0 con 0x7f35f4101f60 2024-04-17T07:16:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.363+0000 7f35ea000640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f4101f60 0x7f35f419a800 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02e4a0 tx=0x7f35dc005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.364+0000 7f35f8a00640 1 -- 172.21.15.86:0/533037433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc004cb0 con 0x7f35f4101f60 2024-04-17T07:16:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.364+0000 7f35f3e00640 1 -- 172.21.15.86:0/533037433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f41a2170 con 0x7f35f4101f60 2024-04-17T07:16:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.364+0000 7f35f8a00640 1 -- 172.21.15.86:0/533037433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35dc043070 con 0x7f35f4101f60 2024-04-17T07:16:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.364+0000 7f35f8a00640 1 -- 172.21.15.86:0/533037433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc03e410 con 0x7f35f4101f60 2024-04-17T07:16:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.365+0000 7f35f3e00640 1 -- 172.21.15.86:0/533037433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f41a26b0 con 0x7f35f4101f60 2024-04-17T07:16:51.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.369+0000 7f35fca74740 1 Processor -- start 2024-04-17T07:16:51.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.369+0000 7f35fca74740 1 -- start start 2024-04-17T07:16:51.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.369+0000 7f35fca74740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2dd160 0x5602db2dd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.369+0000 7f35fca74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2ddb50 0x5602db2e2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.369+0000 7f35fca74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2e2cc0 0x5602db2e5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.369+0000 7f35fca74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5602db243020 con 0x5602db2ddb50 2024-04-17T07:16:51.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.369+0000 7f35fca74740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5602db1e9460 con 0x5602db2dd160 2024-04-17T07:16:51.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.369+0000 7f35fca74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602db23efe0 con 0x5602db2e2cc0 2024-04-17T07:16:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35f8a00640 1 -- 172.21.15.86:0/533037433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f35dc036440 con 0x7f35f4101f60 2024-04-17T07:16:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35f2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2ddb50 0x5602db2e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35f2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2ddb50 0x5602db2e2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34894/0 (socket says 172.21.15.86:34894) 2024-04-17T07:16:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35f2a00640 1 -- 172.21.15.86:0/2698069928 learned_addr learned my addr 172.21.15.86:0/2698069928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35f3400640 1 --2- 172.21.15.86:0/2698069928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2dd160 0x5602db2dd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35ea000640 1 --2- 172.21.15.86:0/2698069928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2e2cc0 0x5602db2e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35f2a00640 1 -- 172.21.15.86:0/2698069928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2e2cc0 msgr2=0x5602db2e5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35f2a00640 1 --2- 172.21.15.86:0/2698069928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2e2cc0 0x5602db2e5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35f2a00640 1 -- 172.21.15.86:0/2698069928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2dd160 msgr2=0x5602db2dd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35f2a00640 1 --2- 172.21.15.86:0/2698069928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2dd160 0x5602db2dd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.370+0000 7f35f2a00640 1 -- 172.21.15.86:0/2698069928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602db1f1040 con 0x5602db2ddb50 2024-04-17T07:16:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.371+0000 7f35f2a00640 1 --2- 172.21.15.86:0/2698069928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2ddb50 0x5602db2e2650 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009f40 tx=0x7f35e4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88ca84f32ccd482 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.371+0000 7f35e3e00640 1 -- 172.21.15.86:0/2698069928 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4016070 con 0x5602db2ddb50 2024-04-17T07:16:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.371+0000 7f35e3e00640 1 -- 172.21.15.86:0/2698069928 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e4004d20 con 0x5602db2ddb50 2024-04-17T07:16:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.371+0000 7f35e3e00640 1 -- 172.21.15.86:0/2698069928 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4012700 con 0x5602db2ddb50 2024-04-17T07:16:51.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35f8a00640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35d80a3c80 0x7f35d80a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35f8a00640 1 -- 172.21.15.86:0/533037433 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f35dc0f2910 con 0x7f35f4101f60 2024-04-17T07:16:51.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35f3400640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35d80a3c80 0x7f35d80a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35fca74740 1 -- 172.21.15.86:0/2698069928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2ddb50 msgr2=0x5602db2e2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35fca74740 1 --2- 172.21.15.86:0/2698069928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2ddb50 0x5602db2e2650 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009f40 tx=0x7f35e4010800 comp rx=0 tx=0).stop 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35fca74740 1 -- 172.21.15.86:0/2698069928 shutdown_connections 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35fca74740 1 --2- 172.21.15.86:0/2698069928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2e2cc0 0x5602db2e5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35fca74740 1 --2- 172.21.15.86:0/2698069928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2ddb50 0x5602db2e2650 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35fca74740 1 --2- 172.21.15.86:0/2698069928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2dd160 0x5602db2dd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35fca74740 1 -- 172.21.15.86:0/2698069928 >> 172.21.15.86:0/2698069928 conn(0x5602db2d86d0 msgr2=0x5602db2d8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.372+0000 7f35fca74740 1 -- 172.21.15.86:0/2698069928 shutdown_connections 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.373+0000 7f35fca74740 1 -- 172.21.15.86:0/2698069928 wait complete. 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.373+0000 7f35f3400640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35d80a3c80 0x7f35d80a6140 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f35ec005fd0 tx=0x7f35ec005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.373+0000 7f35fca74740 1 -- start start 2024-04-17T07:16:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.373+0000 7f35fca74740 1 Processor -- start 2024-04-17T07:16:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35fca74740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2dd160 0x5602db2e7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35fca74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2ddb50 0x5602db2e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35fca74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2eec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35fca74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5602db2e5c70 con 0x5602db2e2cc0 2024-04-17T07:16:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35fca74740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5602db1fc830 con 0x5602db2dd160 2024-04-17T07:16:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35fca74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602db2e5df0 con 0x5602db2ddb50 2024-04-17T07:16:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35fca74740 10 client.0 _ll_register_callbacks cb 0x5602db1cafb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2eec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34900/0 (socket says 172.21.15.86:34900) 2024-04-17T07:16:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35f3400640 1 -- 172.21.15.86:0/2494269936 learned_addr learned my addr 172.21.15.86:0/2494269936 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35f2a00640 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2dd160 0x5602db2e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.374+0000 7f35ea000640 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2ddb50 0x5602db2e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.375+0000 7f35f3400640 1 -- 172.21.15.86:0/2494269936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2ddb50 msgr2=0x5602db2e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.375+0000 7f35f3400640 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2ddb50 0x5602db2e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.375+0000 7f35f3400640 1 -- 172.21.15.86:0/2494269936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2dd160 msgr2=0x5602db2e7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.375+0000 7f35f3400640 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2dd160 0x5602db2e7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.375+0000 7f35f3400640 1 -- 172.21.15.86:0/2494269936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602db200920 con 0x5602db2e2cc0 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.375+0000 7f35f3400640 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2eec00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00c580 tx=0x7f35ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5944e3dd8e8598b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.375+0000 7f35e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.375+0000 7f35e0c00640 1 -- 172.21.15.86:0/2494269936 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec022070 con 0x5602db2e2cc0 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.375+0000 7f35e0c00640 1 -- 172.21.15.86:0/2494269936 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f35ec00ea00 con 0x5602db2e2cc0 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.376+0000 7f35fca74740 1 -- 172.21.15.86:0/2494269936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e4004d20 con 0x5602db2e2cc0 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.376+0000 7f35e0c00640 1 -- 172.21.15.86:0/2494269936 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35ec02e050 con 0x5602db2e2cc0 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.376+0000 7f35fca74740 10 client.16149 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.376+0000 7f35fca74740 1 -- 172.21.15.86:0/2494269936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602db1ce7f0 con 0x5602db2e2cc0 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.376+0000 7f35e0c00640 1 -- 172.21.15.86:0/2494269936 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e4004d20 con 0x5602db2e2cc0 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.376+0000 7f35e0c00640 1 -- 172.21.15.86:0/2494269936 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f35ec01e420 con 0x5602db2e2cc0 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.376+0000 7f35fca74740 10 client.16149 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.376+0000 7f35fca74740 10 client.16149 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:16:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.376+0000 7f35fca74740 20 client.16149 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.377+0000 7f35fca74740 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5602db2f31a0 0x5602db2f5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.377+0000 7f35fca74740 4 client.16149 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.377+0000 7f35fca74740 1 -- 172.21.15.86:0/2494269936 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602db250a40 con 0x5602db2f31a0 2024-04-17T07:16:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.377+0000 7f35f2a00640 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5602db2f31a0 0x5602db2f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.378+0000 7f35f2a00640 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5602db2f31a0 0x5602db2f5640 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d47226fa52f4de51 server_cookie=6426c2ecc8f50cd6 in_seq=0 out_seq=0 2024-04-17T07:16:51.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.378+0000 7f35e0c00640 10 client.16149 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:16:51.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.380+0000 7f35e0c00640 1 -- 172.21.15.86:0/2494269936 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602db250a40 con 0x5602db2f31a0 2024-04-17T07:16:51.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.380+0000 7f35e0c00640 10 client.16149 handle_command_reply: tid=0 2024-04-17T07:16:51.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.380+0000 7f35fca74740 1 client.16149 shutdown 2024-04-17T07:16:51.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.380+0000 7f35fca74740 1 -- 172.21.15.86:0/2494269936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 msgr2=0x5602db2eec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.380+0000 7f35fca74740 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2eec00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00c580 tx=0x7f35ec0046c0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.381+0000 7f35fca74740 1 -- 172.21.15.86:0/2494269936 shutdown_connections 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.381+0000 7f35fca74740 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5602db2f31a0 0x5602db2f5640 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.381+0000 7f35fca74740 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2eec00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.381+0000 7f35fca74740 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2ddb50 0x5602db2e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.381+0000 7f35fca74740 1 --2- 172.21.15.86:0/2494269936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2dd160 0x5602db2e7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.381+0000 7f35fca74740 1 -- 172.21.15.86:0/2494269936 >> 172.21.15.86:0/2494269936 conn(0x5602db2d86d0 msgr2=0x5602db2d7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.381+0000 7f35e0c00640 0 client.16149 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.381+0000 7f35fca74740 1 -- 172.21.15.86:0/2494269936 shutdown_connections 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.381+0000 7f35fca74740 1 -- 172.21.15.86:0/2494269936 wait complete. 2024-04-17T07:16:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.381+0000 7f35fca74740 20 client.16149 trim_cache size 0 max 16384 2024-04-17T07:16:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.398+0000 7f35fca74740 1 Processor -- start 2024-04-17T07:16:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.398+0000 7f35fca74740 1 -- start start 2024-04-17T07:16:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35fca74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2dd160 0x5602db2d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35fca74740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 0x5602db2eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35fca74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2e2cc0 0x5602db2d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35fca74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35f41acae0 con 0x5602db2dd160 2024-04-17T07:16:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35fca74740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f35f41ae620 con 0x5602db2ddb50 2024-04-17T07:16:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35fca74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f35f41ad1b0 con 0x5602db2e2cc0 2024-04-17T07:16:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35f2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2dd160 0x5602db2d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35f2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2dd160 0x5602db2d70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34908/0 (socket says 172.21.15.86:34908) 2024-04-17T07:16:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35f2a00640 1 -- 172.21.15.86:0/3409822490 learned_addr learned my addr 172.21.15.86:0/3409822490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35f3400640 1 --2- 172.21.15.86:0/3409822490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2e2cc0 0x5602db2d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.399+0000 7f35ea000640 1 --2- 172.21.15.86:0/3409822490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 0x5602db2eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.400+0000 7f35f2a00640 1 -- 172.21.15.86:0/3409822490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2e2cc0 msgr2=0x5602db2d75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.400+0000 7f35f2a00640 1 --2- 172.21.15.86:0/3409822490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2e2cc0 0x5602db2d75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.400+0000 7f35f2a00640 1 -- 172.21.15.86:0/3409822490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 msgr2=0x5602db2eebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.400+0000 7f35f2a00640 1 --2- 172.21.15.86:0/3409822490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 0x5602db2eebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.400+0000 7f35f2a00640 1 -- 172.21.15.86:0/3409822490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602db2e6760 con 0x5602db2dd160 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.400+0000 7f35ea000640 1 --2- 172.21.15.86:0/3409822490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 0x5602db2eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.400+0000 7f35f2a00640 1 --2- 172.21.15.86:0/3409822490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2dd160 0x5602db2d70a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f35e4011010 tx=0x7f35e4009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e5639ad447d977 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.400+0000 7f35e3400640 1 -- 172.21.15.86:0/3409822490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4016050 con 0x5602db2dd160 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.400+0000 7f35e3400640 1 -- 172.21.15.86:0/3409822490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e40274d0 con 0x5602db2dd160 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.400+0000 7f35e3400640 1 -- 172.21.15.86:0/3409822490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40053b0 con 0x5602db2dd160 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.401+0000 7f35fca74740 1 -- 172.21.15.86:0/3409822490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2dd160 msgr2=0x5602db2d70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.401+0000 7f35fca74740 1 --2- 172.21.15.86:0/3409822490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2dd160 0x5602db2d70a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f35e4011010 tx=0x7f35e4009860 comp rx=0 tx=0).stop 2024-04-17T07:16:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.401+0000 7f35fca74740 1 -- 172.21.15.86:0/3409822490 shutdown_connections 2024-04-17T07:16:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.401+0000 7f35fca74740 1 --2- 172.21.15.86:0/3409822490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2e2cc0 0x5602db2d75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.401+0000 7f35fca74740 1 --2- 172.21.15.86:0/3409822490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 0x5602db2eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.401+0000 7f35fca74740 1 --2- 172.21.15.86:0/3409822490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2dd160 0x5602db2d70a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.401+0000 7f35fca74740 1 -- 172.21.15.86:0/3409822490 >> 172.21.15.86:0/3409822490 conn(0x5602db2d86d0 msgr2=0x5602db2df050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.401+0000 7f35fca74740 1 -- 172.21.15.86:0/3409822490 shutdown_connections 2024-04-17T07:16:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.401+0000 7f35fca74740 1 -- 172.21.15.86:0/3409822490 wait complete. 2024-04-17T07:16:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.402+0000 7f35fca74740 1 -- start start 2024-04-17T07:16:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.402+0000 7f35fca74740 1 Processor -- start 2024-04-17T07:16:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.402+0000 7f35fca74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2dd160 0x5602db2ecab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.402+0000 7f35fca74740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 0x5602db2ecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.402+0000 7f35fca74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2ee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.402+0000 7f35fca74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35f41adf50 con 0x5602db2e2cc0 2024-04-17T07:16:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.402+0000 7f35fca74740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f35f41ad880 con 0x5602db2ddb50 2024-04-17T07:16:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.402+0000 7f35fca74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f35f41aaab0 con 0x5602db2dd160 2024-04-17T07:16:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35fca74740 10 client.0 _ll_register_callbacks cb 0x7f35ec0117b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2ee000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34918/0 (socket says 172.21.15.86:34918) 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35f3400640 1 -- 172.21.15.86:0/998327131 learned_addr learned my addr 172.21.15.86:0/998327131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35f2a00640 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2dd160 0x5602db2ecab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35ea000640 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 0x5602db2ecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35f3400640 1 -- 172.21.15.86:0/998327131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2dd160 msgr2=0x5602db2ecab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35f3400640 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2dd160 0x5602db2ecab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35f3400640 1 -- 172.21.15.86:0/998327131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 msgr2=0x5602db2ecff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35f3400640 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 0x5602db2ecff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.403+0000 7f35f3400640 1 -- 172.21.15.86:0/998327131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602daf553f0 con 0x5602db2e2cc0 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.404+0000 7f35f3400640 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2ee000 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00c850 tx=0x7f35ec010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0667752ccb06711 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.404+0000 7f35e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.404+0000 7f35e0c00640 1 -- 172.21.15.86:0/998327131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec022040 con 0x5602db2e2cc0 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.404+0000 7f35fca74740 1 -- 172.21.15.86:0/998327131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e40274d0 con 0x5602db2e2cc0 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.404+0000 7f35e0c00640 1 -- 172.21.15.86:0/998327131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f35ec00ea00 con 0x5602db2e2cc0 2024-04-17T07:16:51.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.404+0000 7f35e0c00640 1 -- 172.21.15.86:0/998327131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35ec042050 con 0x5602db2e2cc0 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.404+0000 7f35fca74740 10 client.16155 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.404+0000 7f35fca74740 1 -- 172.21.15.86:0/998327131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602db01f7d0 con 0x5602db2e2cc0 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.405+0000 7f35e0c00640 1 -- 172.21.15.86:0/998327131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e40274d0 con 0x5602db2e2cc0 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.405+0000 7f35e0c00640 1 -- 172.21.15.86:0/998327131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f35ec00ec00 con 0x5602db2e2cc0 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.405+0000 7f35fca74740 10 client.16155 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.405+0000 7f35fca74740 10 client.16155 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.405+0000 7f35fca74740 20 client.16155 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.405+0000 7f35fca74740 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5602db35a490 0x5602db35c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.405+0000 7f35fca74740 4 client.16155 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.405+0000 7f35fca74740 1 -- 172.21.15.86:0/998327131 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602db24a8b0 con 0x5602db35a490 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.406+0000 7f35f2a00640 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5602db35a490 0x5602db35c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.406+0000 7f35f2a00640 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5602db35a490 0x5602db35c8b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=78dfeeb707e46a21 server_cookie=34599971fb6c69f3 in_seq=0 out_seq=0 2024-04-17T07:16:51.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.406+0000 7f35e0c00640 10 client.16155 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:16:51.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35e0c00640 1 -- 172.21.15.86:0/998327131 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602db24a8b0 con 0x5602db35a490 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35e0c00640 10 client.16155 handle_command_reply: tid=0 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 client.16155 shutdown 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 -- 172.21.15.86:0/998327131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 msgr2=0x5602db2ee000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2ee000 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00c850 tx=0x7f35ec010d00 comp rx=0 tx=0).stop 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 -- 172.21.15.86:0/998327131 shutdown_connections 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5602db35a490 0x5602db35c8b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602db2e2cc0 0x5602db2ee000 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602db2ddb50 0x5602db2ecff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 --2- 172.21.15.86:0/998327131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602db2dd160 0x5602db2ecab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35e0c00640 0 client.16155 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 -- 172.21.15.86:0/998327131 >> 172.21.15.86:0/998327131 conn(0x5602db2d86d0 msgr2=0x5602db2d70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 -- 172.21.15.86:0/998327131 shutdown_connections 2024-04-17T07:16:51.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 1 -- 172.21.15.86:0/998327131 wait complete. 2024-04-17T07:16:51.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.407+0000 7f35fca74740 20 client.16155 trim_cache size 0 max 16384 2024-04-17T07:16:51.411 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:16:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.408+0000 7f35f3e00640 1 -- 172.21.15.86:0/533037433 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35d80a3c80 msgr2=0x7f35d80a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.408+0000 7f35f3e00640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35d80a3c80 0x7f35d80a6140 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f35ec005fd0 tx=0x7f35ec005ec0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.408+0000 7f35f3e00640 1 -- 172.21.15.86:0/533037433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f4101f60 msgr2=0x7f35f419a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.408+0000 7f35f3e00640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f4101f60 0x7f35f419a800 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02e4a0 tx=0x7f35dc005a70 comp rx=0 tx=0).stop 2024-04-17T07:16:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.408+0000 7f35f3e00640 1 -- 172.21.15.86:0/533037433 shutdown_connections 2024-04-17T07:16:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.408+0000 7f35f3e00640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f35d80a3c80 0x7f35d80a6140 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.408+0000 7f35f3e00640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35f4102920 0x7f35f41a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.409+0000 7f35f3e00640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f4101f60 0x7f35f419a800 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.409+0000 7f35f3e00640 1 --2- 172.21.15.86:0/533037433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f4100d70 0x7f35f419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.409+0000 7f35f3e00640 1 -- 172.21.15.86:0/533037433 >> 172.21.15.86:0/533037433 conn(0x7f35f40fc300 msgr2=0x7f35f4190e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.409+0000 7f35f3e00640 1 -- 172.21.15.86:0/533037433 shutdown_connections 2024-04-17T07:16:51.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.409+0000 7f35f3e00640 1 -- 172.21.15.86:0/533037433 wait complete. 2024-04-17T07:16:51.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:16:51.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.632+0000 7efd61800640 1 -- 172.21.15.86:0/182211065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd5c105230 msgr2=0x7efd5c107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.632+0000 7efd61800640 1 --2- 172.21.15.86:0/182211065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd5c105230 0x7efd5c107620 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efd48009900 tx=0x7efd4802bc30 comp rx=0 tx=0).stop 2024-04-17T07:16:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.632+0000 7efd61800640 1 -- 172.21.15.86:0/182211065 shutdown_connections 2024-04-17T07:16:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.632+0000 7efd61800640 1 --2- 172.21.15.86:0/182211065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c107b60 0x7efd5c109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.632+0000 7efd61800640 1 --2- 172.21.15.86:0/182211065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd5c105230 0x7efd5c107620 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.632+0000 7efd61800640 1 --2- 172.21.15.86:0/182211065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c102900 0x7efd5c104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.632+0000 7efd61800640 1 -- 172.21.15.86:0/182211065 >> 172.21.15.86:0/182211065 conn(0x7efd5c0fc310 msgr2=0x7efd5c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.633+0000 7efd61800640 1 -- 172.21.15.86:0/182211065 shutdown_connections 2024-04-17T07:16:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.633+0000 7efd61800640 1 -- 172.21.15.86:0/182211065 wait complete. 2024-04-17T07:16:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.633+0000 7efd61800640 1 Processor -- start 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.634+0000 7efd61800640 1 -- start start 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.634+0000 7efd61800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd5c102900 0x7efd5c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.634+0000 7efd61800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c105230 0x7efd5c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.634+0000 7efd61800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c107b60 0x7efd5c1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.634+0000 7efd61800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c10da10 con 0x7efd5c102900 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.634+0000 7efd61800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c10d890 con 0x7efd5c105230 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.634+0000 7efd61800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c10db90 con 0x7efd5c107b60 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.634+0000 7efd5be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd5c102900 0x7efd5c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.634+0000 7efd5be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd5c102900 0x7efd5c19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34934/0 (socket says 172.21.15.86:34934) 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.634+0000 7efd5be00640 1 -- 172.21.15.86:0/2073641350 learned_addr learned my addr 172.21.15.86:0/2073641350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd5b400640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c105230 0x7efd5c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd60e00640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c107b60 0x7efd5c1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd5be00640 1 -- 172.21.15.86:0/2073641350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c107b60 msgr2=0x7efd5c1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd5be00640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c107b60 0x7efd5c1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd5be00640 1 -- 172.21.15.86:0/2073641350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c105230 msgr2=0x7efd5c19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd5be00640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c105230 0x7efd5c19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd5be00640 1 -- 172.21.15.86:0/2073641350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c1a1f00 con 0x7efd5c102900 2024-04-17T07:16:51.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd5be00640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd5c102900 0x7efd5c19a2e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d770 tx=0x7efd4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd58c00640 1 -- 172.21.15.86:0/2073641350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c017070 con 0x7efd5c102900 2024-04-17T07:16:51.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd58c00640 1 -- 172.21.15.86:0/2073641350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd4c0054d0 con 0x7efd5c102900 2024-04-17T07:16:51.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd61800640 1 -- 172.21.15.86:0/2073641350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c1a21f0 con 0x7efd5c102900 2024-04-17T07:16:51.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.635+0000 7efd58c00640 1 -- 172.21.15.86:0/2073641350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c004cf0 con 0x7efd5c102900 2024-04-17T07:16:51.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.636+0000 7efd61800640 1 -- 172.21.15.86:0/2073641350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c1a2710 con 0x7efd5c102900 2024-04-17T07:16:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.639+0000 7efd65722740 1 Processor -- start 2024-04-17T07:16:51.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd65722740 1 -- start start 2024-04-17T07:16:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd65722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472beda580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd65722740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedab50 0x56472bedf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd65722740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedfcc0 0x56472bee2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd65722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56472be40020 con 0x56472beda160 2024-04-17T07:16:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd65722740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56472bde6460 con 0x56472bedfcc0 2024-04-17T07:16:51.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd65722740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56472be3bfe0 con 0x56472bedab50 2024-04-17T07:16:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd58c00640 1 -- 172.21.15.86:0/2073641350 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7efd4c004e90 con 0x7efd5c102900 2024-04-17T07:16:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd60e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472beda580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd60e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472beda580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34946/0 (socket says 172.21.15.86:34946) 2024-04-17T07:16:51.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.640+0000 7efd60e00640 1 -- 172.21.15.86:0/4232945659 learned_addr learned my addr 172.21.15.86:0/4232945659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.641+0000 7efd60e00640 1 -- 172.21.15.86:0/4232945659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedab50 msgr2=0x56472bedf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.641+0000 7efd60e00640 1 --2- 172.21.15.86:0/4232945659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedab50 0x56472bedf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.641+0000 7efd60e00640 1 -- 172.21.15.86:0/4232945659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedfcc0 msgr2=0x56472bee2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.641+0000 7efd60e00640 1 --2- 172.21.15.86:0/4232945659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedfcc0 0x56472bee2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.641+0000 7efd60e00640 1 -- 172.21.15.86:0/4232945659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472bdee040 con 0x56472beda160 2024-04-17T07:16:51.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.641+0000 7efd60e00640 1 --2- 172.21.15.86:0/4232945659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472beda580 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efd54009ee0 tx=0x7efd54010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbc38023f11e7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.641+0000 7efd51600640 1 -- 172.21.15.86:0/4232945659 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54016070 con 0x56472beda160 2024-04-17T07:16:51.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.642+0000 7efd51600640 1 -- 172.21.15.86:0/4232945659 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd54004d20 con 0x56472beda160 2024-04-17T07:16:51.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.642+0000 7efd51600640 1 -- 172.21.15.86:0/4232945659 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd540126e0 con 0x56472beda160 2024-04-17T07:16:51.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.642+0000 7efd58c00640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd400a3c30 0x7efd400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.642+0000 7efd58c00640 1 -- 172.21.15.86:0/2073641350 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efd4c0d0220 con 0x7efd5c102900 2024-04-17T07:16:51.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.642+0000 7efd65722740 1 -- 172.21.15.86:0/4232945659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 msgr2=0x56472beda580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.642+0000 7efd65722740 1 --2- 172.21.15.86:0/4232945659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472beda580 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efd54009ee0 tx=0x7efd54010740 comp rx=0 tx=0).stop 2024-04-17T07:16:51.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.642+0000 7efd5be00640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd400a3c30 0x7efd400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.643+0000 7efd65722740 1 -- 172.21.15.86:0/4232945659 shutdown_connections 2024-04-17T07:16:51.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.643+0000 7efd65722740 1 --2- 172.21.15.86:0/4232945659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedfcc0 0x56472bee2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.643+0000 7efd65722740 1 --2- 172.21.15.86:0/4232945659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedab50 0x56472bedf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.643+0000 7efd65722740 1 --2- 172.21.15.86:0/4232945659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472beda580 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.643+0000 7efd65722740 1 -- 172.21.15.86:0/4232945659 >> 172.21.15.86:0/4232945659 conn(0x56472bed56d0 msgr2=0x56472bed5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.643+0000 7efd65722740 1 -- 172.21.15.86:0/4232945659 shutdown_connections 2024-04-17T07:16:51.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.643+0000 7efd65722740 1 -- 172.21.15.86:0/4232945659 wait complete. 2024-04-17T07:16:51.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.643+0000 7efd65722740 1 -- start start 2024-04-17T07:16:51.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.643+0000 7efd5be00640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd400a3c30 0x7efd400a60f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0cd9f0 tx=0x7efd4c005bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd65722740 1 Processor -- start 2024-04-17T07:16:51.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd65722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472bee4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd65722740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedab50 0x56472bee4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd65722740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedfcc0 0x56472beebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd65722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56472bee2cc0 con 0x56472beda160 2024-04-17T07:16:51.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd65722740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56472bdf9830 con 0x56472bedab50 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd65722740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56472bee2e40 con 0x56472bedfcc0 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd65722740 10 client.0 _ll_register_callbacks cb 0x56472bdc7fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd60e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472bee4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd60e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472bee4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34962/0 (socket says 172.21.15.86:34962) 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.644+0000 7efd60e00640 1 -- 172.21.15.86:0/3082709309 learned_addr learned my addr 172.21.15.86:0/3082709309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.645+0000 7efd5b400640 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedab50 0x56472bee4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.645+0000 7efd60e00640 1 -- 172.21.15.86:0/3082709309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedfcc0 msgr2=0x56472beebbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.645+0000 7efd60e00640 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedfcc0 0x56472beebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.645+0000 7efd60e00640 1 -- 172.21.15.86:0/3082709309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedab50 msgr2=0x56472bee4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.645+0000 7efd60e00640 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedab50 0x56472bee4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.645+0000 7efd60e00640 1 -- 172.21.15.86:0/3082709309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472bdfd920 con 0x56472beda160 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.645+0000 7efd60e00640 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472bee4650 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efd54005b40 tx=0x7efd54004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89be1dc81a5d961 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.645+0000 7efd46000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:51.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.646+0000 7efd46000640 1 -- 172.21.15.86:0/3082709309 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54016070 con 0x56472beda160 2024-04-17T07:16:51.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.646+0000 7efd46000640 1 -- 172.21.15.86:0/3082709309 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efd540054f0 con 0x56472beda160 2024-04-17T07:16:51.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.646+0000 7efd65722740 1 -- 172.21.15.86:0/3082709309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd54004d20 con 0x56472beda160 2024-04-17T07:16:51.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.646+0000 7efd46000640 1 -- 172.21.15.86:0/3082709309 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd540124e0 con 0x56472beda160 2024-04-17T07:16:51.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.646+0000 7efd65722740 10 client.16167 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:51.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.646+0000 7efd65722740 1 -- 172.21.15.86:0/3082709309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56472bdcb7f0 con 0x56472beda160 2024-04-17T07:16:51.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.646+0000 7efd46000640 1 -- 172.21.15.86:0/3082709309 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd54004d20 con 0x56472beda160 2024-04-17T07:16:51.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.647+0000 7efd46000640 1 -- 172.21.15.86:0/3082709309 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7efd5400f040 con 0x56472beda160 2024-04-17T07:16:51.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.647+0000 7efd65722740 10 client.16167 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:51.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.647+0000 7efd65722740 10 client.16167 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:16:51.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.647+0000 7efd65722740 20 client.16167 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:51.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.647+0000 7efd65722740 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56472beefde0 0x56472bef21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.647+0000 7efd65722740 4 client.16167 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:51.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.647+0000 7efd65722740 1 -- 172.21.15.86:0/3082709309 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56472be4da40 con 0x56472beefde0 2024-04-17T07:16:51.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.648+0000 7efd5b400640 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56472beefde0 0x56472bef21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.649+0000 7efd46000640 10 client.16167 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:16:51.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.649+0000 7efd5b400640 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56472beefde0 0x56472bef21d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d82ac1845d65672 server_cookie=c24e6df4ccf33df9 in_seq=0 out_seq=0 2024-04-17T07:16:51.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd46000640 1 -- 172.21.15.86:0/3082709309 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56472be4da40 con 0x56472beefde0 2024-04-17T07:16:51.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd46000640 10 client.16167 handle_command_reply: tid=0 2024-04-17T07:16:51.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd65722740 1 client.16167 shutdown 2024-04-17T07:16:51.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd65722740 1 -- 172.21.15.86:0/3082709309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 msgr2=0x56472bee4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd65722740 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472bee4650 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efd54005b40 tx=0x7efd54004ef0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd65722740 1 -- 172.21.15.86:0/3082709309 shutdown_connections 2024-04-17T07:16:51.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd65722740 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56472beefde0 0x56472bef21d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd65722740 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedfcc0 0x56472beebbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd65722740 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedab50 0x56472bee4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd65722740 1 --2- 172.21.15.86:0/3082709309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472beda160 0x56472bee4650 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd46000640 0 client.16167 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:16:51.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd65722740 1 -- 172.21.15.86:0/3082709309 >> 172.21.15.86:0/3082709309 conn(0x56472bed56d0 msgr2=0x56472bed4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.651+0000 7efd65722740 1 -- 172.21.15.86:0/3082709309 shutdown_connections 2024-04-17T07:16:51.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.652+0000 7efd65722740 1 -- 172.21.15.86:0/3082709309 wait complete. 2024-04-17T07:16:51.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.652+0000 7efd65722740 20 client.16167 trim_cache size 0 max 16384 2024-04-17T07:16:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.669+0000 7efd65722740 1 Processor -- start 2024-04-17T07:16:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.669+0000 7efd65722740 1 -- start start 2024-04-17T07:16:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.669+0000 7efd65722740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472beda160 0x56472bee48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.669+0000 7efd65722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedab50 0x56472beebb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.669+0000 7efd65722740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedfcc0 0x56472bee3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.669+0000 7efd65722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1acb90 con 0x56472bedab50 2024-04-17T07:16:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.669+0000 7efd65722740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1ae6d0 con 0x56472beda160 2024-04-17T07:16:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.669+0000 7efd65722740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1ad260 con 0x56472bedfcc0 2024-04-17T07:16:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.670+0000 7efd5b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedab50 0x56472beebb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.670+0000 7efd5b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedab50 0x56472beebb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34964/0 (socket says 172.21.15.86:34964) 2024-04-17T07:16:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.670+0000 7efd5b400640 1 -- 172.21.15.86:0/2511778860 learned_addr learned my addr 172.21.15.86:0/2511778860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.670+0000 7efd60e00640 1 --2- 172.21.15.86:0/2511778860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472beda160 0x56472bee48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.670+0000 7efd60e00640 1 --2- 172.21.15.86:0/2511778860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedfcc0 0x56472bee3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.670+0000 7efd5b400640 1 -- 172.21.15.86:0/2511778860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedfcc0 msgr2=0x56472bee3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.670+0000 7efd5b400640 1 --2- 172.21.15.86:0/2511778860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedfcc0 0x56472bee3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.670+0000 7efd5b400640 1 -- 172.21.15.86:0/2511778860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472beda160 msgr2=0x56472bee48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.670+0000 7efd5b400640 1 --2- 172.21.15.86:0/2511778860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472beda160 0x56472bee48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.671+0000 7efd5b400640 1 -- 172.21.15.86:0/2511778860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56472bb523f0 con 0x56472bedab50 2024-04-17T07:16:51.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.671+0000 7efd5b400640 1 --2- 172.21.15.86:0/2511778860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedab50 0x56472beebb90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efd480096c0 tx=0x7efd48009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ca1541f54cac25 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.671+0000 7efd50c00640 1 -- 172.21.15.86:0/2511778860 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4804c020 con 0x56472bedab50 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.671+0000 7efd50c00640 1 -- 172.21.15.86:0/2511778860 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd4802ca00 con 0x56472bedab50 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.671+0000 7efd50c00640 1 -- 172.21.15.86:0/2511778860 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd480046c0 con 0x56472bedab50 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 -- 172.21.15.86:0/2511778860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedab50 msgr2=0x56472beebb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 --2- 172.21.15.86:0/2511778860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedab50 0x56472beebb90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efd480096c0 tx=0x7efd48009260 comp rx=0 tx=0).stop 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 -- 172.21.15.86:0/2511778860 shutdown_connections 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 --2- 172.21.15.86:0/2511778860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472bedfcc0 0x56472bee3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 --2- 172.21.15.86:0/2511778860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedab50 0x56472beebb90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 --2- 172.21.15.86:0/2511778860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472beda160 0x56472bee48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 -- 172.21.15.86:0/2511778860 >> 172.21.15.86:0/2511778860 conn(0x56472bed56d0 msgr2=0x56472bed4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 -- 172.21.15.86:0/2511778860 shutdown_connections 2024-04-17T07:16:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 -- 172.21.15.86:0/2511778860 wait complete. 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 -- start start 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.673+0000 7efd65722740 1 Processor -- start 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.674+0000 7efd65722740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472beda160 0x56472bee8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.674+0000 7efd65722740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedab50 0x56472bee9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.674+0000 7efd65722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedfcc0 0x56472beea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.674+0000 7efd65722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1ae000 con 0x56472bedfcc0 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.674+0000 7efd65722740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1ad930 con 0x56472bedab50 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.674+0000 7efd65722740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c1aab60 con 0x56472beda160 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.674+0000 7efd65722740 10 client.0 _ll_register_callbacks cb 0x7efd54004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.674+0000 7efd60e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedfcc0 0x56472beea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.674+0000 7efd60e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedfcc0 0x56472beea320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34966/0 (socket says 172.21.15.86:34966) 2024-04-17T07:16:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.674+0000 7efd60e00640 1 -- 172.21.15.86:0/1129914186 learned_addr learned my addr 172.21.15.86:0/1129914186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.675+0000 7efd60e00640 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472beda160 0x56472bee8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.675+0000 7efd5b400640 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedab50 0x56472bee9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.675+0000 7efd60e00640 1 -- 172.21.15.86:0/1129914186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472beda160 msgr2=0x56472bee8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.675+0000 7efd60e00640 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472beda160 0x56472bee8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.675+0000 7efd60e00640 1 -- 172.21.15.86:0/1129914186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedab50 msgr2=0x56472bee9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.675+0000 7efd60e00640 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedab50 0x56472bee9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.675+0000 7efd60e00640 1 -- 172.21.15.86:0/1129914186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56472bc1c7d0 con 0x56472bedfcc0 2024-04-17T07:16:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.675+0000 7efd60e00640 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedfcc0 0x56472beea320 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efd5401a9c0 tx=0x7efd54010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e3f106c63e0a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.675+0000 7efd46000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:51.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.675+0000 7efd46000640 1 -- 172.21.15.86:0/1129914186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5403e060 con 0x56472bedfcc0 2024-04-17T07:16:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd46000640 1 -- 172.21.15.86:0/1129914186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efd540054f0 con 0x56472bedfcc0 2024-04-17T07:16:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd65722740 1 -- 172.21.15.86:0/1129914186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd4802ca00 con 0x56472bedfcc0 2024-04-17T07:16:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd46000640 1 -- 172.21.15.86:0/1129914186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd54045050 con 0x56472bedfcc0 2024-04-17T07:16:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd65722740 10 client.16173 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd65722740 1 -- 172.21.15.86:0/1129914186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56472beeba40 con 0x56472bedfcc0 2024-04-17T07:16:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd46000640 1 -- 172.21.15.86:0/1129914186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd4802ca00 con 0x56472bedfcc0 2024-04-17T07:16:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd46000640 1 -- 172.21.15.86:0/1129914186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7efd54033040 con 0x56472bedfcc0 2024-04-17T07:16:51.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd65722740 10 client.16173 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:51.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd65722740 10 client.16173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:16:51.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd65722740 20 client.16173 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:51.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.676+0000 7efd65722740 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56472bf56430 0x56472bf58850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.677+0000 7efd65722740 4 client.16173 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:16:51.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.677+0000 7efd65722740 1 -- 172.21.15.86:0/1129914186 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56472be478b0 con 0x56472bf56430 2024-04-17T07:16:51.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.677+0000 7efd5b400640 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56472bf56430 0x56472bf58850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.679+0000 7efd5b400640 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56472bf56430 0x56472bf58850 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b8e22fd32fe33b2 server_cookie=668aff7ac4eb3a56 in_seq=0 out_seq=0 2024-04-17T07:16:51.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.679+0000 7efd46000640 10 client.16173 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:16:51.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.679+0000 7efd46000640 1 -- 172.21.15.86:0/1129914186 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56472be478b0 con 0x56472bf56430 2024-04-17T07:16:51.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.679+0000 7efd46000640 10 client.16173 handle_command_reply: tid=0 2024-04-17T07:16:51.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.679+0000 7efd65722740 1 client.16173 shutdown 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 1 -- 172.21.15.86:0/1129914186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedfcc0 msgr2=0x56472beea320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedfcc0 0x56472beea320 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efd5401a9c0 tx=0x7efd54010f80 comp rx=0 tx=0).stop 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 1 -- 172.21.15.86:0/1129914186 shutdown_connections 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56472bf56430 0x56472bf58850 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56472bedfcc0 0x56472beea320 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56472bedab50 0x56472bee9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 1 --2- 172.21.15.86:0/1129914186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56472beda160 0x56472bee8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd46000640 0 client.16173 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 1 -- 172.21.15.86:0/1129914186 >> 172.21.15.86:0/1129914186 conn(0x56472bed56d0 msgr2=0x56472bed4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 1 -- 172.21.15.86:0/1129914186 shutdown_connections 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 1 -- 172.21.15.86:0/1129914186 wait complete. 2024-04-17T07:16:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.680+0000 7efd65722740 20 client.16173 trim_cache size 0 max 16384 2024-04-17T07:16:51.682 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:16:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 -- 172.21.15.86:0/2073641350 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd400a3c30 msgr2=0x7efd400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd400a3c30 0x7efd400a60f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0cd9f0 tx=0x7efd4c005bb0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 -- 172.21.15.86:0/2073641350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd5c102900 msgr2=0x7efd5c19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd5c102900 0x7efd5c19a2e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d770 tx=0x7efd4c00dc30 comp rx=0 tx=0).stop 2024-04-17T07:16:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 -- 172.21.15.86:0/2073641350 shutdown_connections 2024-04-17T07:16:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efd400a3c30 0x7efd400a60f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efd5c107b60 0x7efd5c1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd5c105230 0x7efd5c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 --2- 172.21.15.86:0/2073641350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd5c102900 0x7efd5c19a2e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 -- 172.21.15.86:0/2073641350 >> 172.21.15.86:0/2073641350 conn(0x7efd5c0fc310 msgr2=0x7efd5c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 -- 172.21.15.86:0/2073641350 shutdown_connections 2024-04-17T07:16:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.681+0000 7efd61800640 1 -- 172.21.15.86:0/2073641350 wait complete. 2024-04-17T07:16:51.701 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:16:51.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.915+0000 7f5f28c00640 1 -- 172.21.15.86:0/2311780460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f24101fb0 msgr2=0x7f5f241023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.915+0000 7f5f28c00640 1 --2- 172.21.15.86:0/2311780460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f24101fb0 0x7f5f241023d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009ca0 tx=0x7f5f1402c100 comp rx=0 tx=0).stop 2024-04-17T07:16:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.915+0000 7f5f28c00640 1 -- 172.21.15.86:0/2311780460 shutdown_connections 2024-04-17T07:16:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.915+0000 7f5f28c00640 1 --2- 172.21.15.86:0/2311780460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f24103b60 0x7f5f2410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.915+0000 7f5f28c00640 1 --2- 172.21.15.86:0/2311780460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f241031a0 0x7f5f24103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.915+0000 7f5f28c00640 1 --2- 172.21.15.86:0/2311780460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f24101fb0 0x7f5f241023d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.915+0000 7f5f28c00640 1 -- 172.21.15.86:0/2311780460 >> 172.21.15.86:0/2311780460 conn(0x7f5f240fd540 msgr2=0x7f5f240ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.916+0000 7f5f28c00640 1 -- 172.21.15.86:0/2311780460 shutdown_connections 2024-04-17T07:16:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.916+0000 7f5f28c00640 1 -- 172.21.15.86:0/2311780460 wait complete. 2024-04-17T07:16:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.917+0000 7f5f28c00640 1 Processor -- start 2024-04-17T07:16:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.917+0000 7f5f28c00640 1 -- start start 2024-04-17T07:16:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.917+0000 7f5f28c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f24101fb0 0x7f5f24075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.917+0000 7f5f28c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f241031a0 0x7f5f24074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.917+0000 7f5f28c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f24103b60 0x7f5f24074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.917+0000 7f5f28c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2410daf0 con 0x7f5f241031a0 2024-04-17T07:16:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.917+0000 7f5f28c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2410d970 con 0x7f5f24103b60 2024-04-17T07:16:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.917+0000 7f5f28c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2410dc70 con 0x7f5f24101fb0 2024-04-17T07:16:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f241031a0 0x7f5f24074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f241031a0 0x7f5f24074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34976/0 (socket says 172.21.15.86:34976) 2024-04-17T07:16:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f21400640 1 -- 172.21.15.86:0/3889675890 learned_addr learned my addr 172.21.15.86:0/3889675890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f21e00640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f24101fb0 0x7f5f24075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f22800640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f24103b60 0x7f5f24074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f21400640 1 -- 172.21.15.86:0/3889675890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f24101fb0 msgr2=0x7f5f24075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f21400640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f24101fb0 0x7f5f24075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f21400640 1 -- 172.21.15.86:0/3889675890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f24103b60 msgr2=0x7f5f24074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f21400640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f24103b60 0x7f5f24074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f21400640 1 -- 172.21.15.86:0/3889675890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f24075120 con 0x7f5f241031a0 2024-04-17T07:16:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f22800640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f24103b60 0x7f5f24074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.918+0000 7f5f21400640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f241031a0 0x7f5f24074350 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000d950 tx=0x7f5f1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.919+0000 7f5f1aa00640 1 -- 172.21.15.86:0/3889675890 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10017070 con 0x7f5f241031a0 2024-04-17T07:16:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.919+0000 7f5f1aa00640 1 -- 172.21.15.86:0/3889675890 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f10009c30 con 0x7f5f241031a0 2024-04-17T07:16:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.919+0000 7f5f28c00640 1 -- 172.21.15.86:0/3889675890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f241a2340 con 0x7f5f241031a0 2024-04-17T07:16:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.919+0000 7f5f1aa00640 1 -- 172.21.15.86:0/3889675890 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1000f620 con 0x7f5f241031a0 2024-04-17T07:16:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.919+0000 7f5f28c00640 1 -- 172.21.15.86:0/3889675890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f241a28e0 con 0x7f5f241031a0 2024-04-17T07:16:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.922+0000 7f5f1aa00640 1 -- 172.21.15.86:0/3889675890 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5f100109a0 con 0x7f5f241031a0 2024-04-17T07:16:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.923+0000 7f5f2b995740 1 Processor -- start 2024-04-17T07:16:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.923+0000 7f5f2b995740 1 -- start start 2024-04-17T07:16:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.923+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.923+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8b8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.923+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b8b00 0x56307f8baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.923+0000 7f5f2b995740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56307f819020 con 0x56307f8b2fa0 2024-04-17T07:16:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.923+0000 7f5f2b995740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56307f7bf460 con 0x56307f8b8b00 2024-04-17T07:16:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.923+0000 7f5f2b995740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56307f814fe0 con 0x56307f8b3990 2024-04-17T07:16:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f22800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f22800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34978/0 (socket says 172.21.15.86:34978) 2024-04-17T07:16:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f22800640 1 -- 172.21.15.86:0/595058771 learned_addr learned my addr 172.21.15.86:0/595058771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f21e00640 1 --2- 172.21.15.86:0/595058771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f21400640 1 --2- 172.21.15.86:0/595058771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b8b00 0x56307f8baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f22800640 1 -- 172.21.15.86:0/595058771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 msgr2=0x56307f8b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f22800640 1 --2- 172.21.15.86:0/595058771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f22800640 1 -- 172.21.15.86:0/595058771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b8b00 msgr2=0x56307f8baf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f22800640 1 --2- 172.21.15.86:0/595058771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b8b00 0x56307f8baf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f22800640 1 -- 172.21.15.86:0/595058771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307f7c7040 con 0x56307f8b2fa0 2024-04-17T07:16:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.924+0000 7f5f1aa00640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f080a3c30 0x7f5f080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.925+0000 7f5f22800640 1 --2- 172.21.15.86:0/595058771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8b33c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009bd0 tx=0x7f5f1c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65612585044997ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.925+0000 7f5f1aa00640 1 -- 172.21.15.86:0/3889675890 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5f100d0340 con 0x7f5f241031a0 2024-04-17T07:16:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.925+0000 7f5f21e00640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f080a3c30 0x7f5f080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.925+0000 7f5f0f000640 1 -- 172.21.15.86:0/595058771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c016070 con 0x56307f8b2fa0 2024-04-17T07:16:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.925+0000 7f5f0f000640 1 -- 172.21.15.86:0/595058771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1c011920 con 0x56307f8b2fa0 2024-04-17T07:16:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.925+0000 7f5f0f000640 1 -- 172.21.15.86:0/595058771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c010730 con 0x56307f8b2fa0 2024-04-17T07:16:51.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.926+0000 7f5f2b995740 1 -- 172.21.15.86:0/595058771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 msgr2=0x56307f8b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.926+0000 7f5f21e00640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f080a3c30 0x7f5f080a60f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5f1402c710 tx=0x7f5f14005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.926+0000 7f5f2b995740 1 --2- 172.21.15.86:0/595058771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8b33c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009bd0 tx=0x7f5f1c00e6c0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.926+0000 7f5f2b995740 1 -- 172.21.15.86:0/595058771 shutdown_connections 2024-04-17T07:16:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.926+0000 7f5f2b995740 1 --2- 172.21.15.86:0/595058771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b8b00 0x56307f8baf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.926+0000 7f5f2b995740 1 --2- 172.21.15.86:0/595058771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.926+0000 7f5f2b995740 1 --2- 172.21.15.86:0/595058771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8b33c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.926+0000 7f5f2b995740 1 -- 172.21.15.86:0/595058771 >> 172.21.15.86:0/595058771 conn(0x56307f8ae510 msgr2=0x56307f8ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.926+0000 7f5f2b995740 1 -- 172.21.15.86:0/595058771 shutdown_connections 2024-04-17T07:16:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.926+0000 7f5f2b995740 1 -- 172.21.15.86:0/595058771 wait complete. 2024-04-17T07:16:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.927+0000 7f5f2b995740 1 -- start start 2024-04-17T07:16:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.927+0000 7f5f2b995740 1 Processor -- start 2024-04-17T07:16:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b2fa0 0x56307f8bd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b3990 0x56307f8bda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f2b995740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56307f8bbb00 con 0x56307f8b8b00 2024-04-17T07:16:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f2b995740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56307f7d2830 con 0x56307f8b3990 2024-04-17T07:16:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f2b995740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56307f8bbc80 con 0x56307f8b2fa0 2024-04-17T07:16:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f2b995740 10 client.0 _ll_register_callbacks cb 0x56307f7a0fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f22800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f22800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34986/0 (socket says 172.21.15.86:34986) 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f22800640 1 -- 172.21.15.86:0/4027014703 learned_addr learned my addr 172.21.15.86:0/4027014703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.928+0000 7f5f21e00640 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b2fa0 0x56307f8bd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.929+0000 7f5f21400640 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b3990 0x56307f8bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.929+0000 7f5f22800640 1 -- 172.21.15.86:0/4027014703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b2fa0 msgr2=0x56307f8bd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.929+0000 7f5f22800640 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b2fa0 0x56307f8bd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.929+0000 7f5f22800640 1 -- 172.21.15.86:0/4027014703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b3990 msgr2=0x56307f8bda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.929+0000 7f5f22800640 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b3990 0x56307f8bda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.929+0000 7f5f22800640 1 -- 172.21.15.86:0/4027014703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307f7d6920 con 0x56307f8b8b00 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.929+0000 7f5f21400640 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b3990 0x56307f8bda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.929+0000 7f5f22800640 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c4a60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00ebc0 tx=0x7f5f1c011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773a62e3d77d0473 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.929+0000 7f5f03400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.930+0000 7f5f03400640 1 -- 172.21.15.86:0/4027014703 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c028060 con 0x56307f8b8b00 2024-04-17T07:16:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.930+0000 7f5f03400640 1 -- 172.21.15.86:0/4027014703 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5f1c005240 con 0x56307f8b8b00 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.930+0000 7f5f2b995740 1 -- 172.21.15.86:0/4027014703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1c011920 con 0x56307f8b8b00 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.930+0000 7f5f03400640 1 -- 172.21.15.86:0/4027014703 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c011e60 con 0x56307f8b8b00 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.930+0000 7f5f2b995740 10 client.16185 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.930+0000 7f5f2b995740 1 -- 172.21.15.86:0/4027014703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56307f7a47f0 con 0x56307f8b8b00 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.931+0000 7f5f03400640 1 -- 172.21.15.86:0/4027014703 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1c011920 con 0x56307f8b8b00 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.931+0000 7f5f03400640 1 -- 172.21.15.86:0/4027014703 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5f1c00f040 con 0x56307f8b8b00 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.931+0000 7f5f2b995740 10 client.16185 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.931+0000 7f5f2b995740 10 client.16185 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.931+0000 7f5f2b995740 20 client.16185 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.931+0000 7f5f2b995740 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56307f8c8ce0 0x56307f8cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.931+0000 7f5f2b995740 4 client.16185 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.931+0000 7f5f2b995740 1 -- 172.21.15.86:0/4027014703 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56307f826a40 con 0x56307f8c8ce0 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.932+0000 7f5f21e00640 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56307f8c8ce0 0x56307f8cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.933+0000 7f5f21e00640 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56307f8c8ce0 0x56307f8cb120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8a9a19b2ebf1aa7 server_cookie=7a6b5ebbd4855c29 in_seq=0 out_seq=0 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.933+0000 7f5f03400640 10 client.16185 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.934+0000 7f5f03400640 1 -- 172.21.15.86:0/4027014703 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56307f826a40 con 0x56307f8c8ce0 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.934+0000 7f5f03400640 10 client.16185 handle_command_reply: tid=0 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.934+0000 7f5f2b995740 1 client.16185 shutdown 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 1 -- 172.21.15.86:0/4027014703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 msgr2=0x56307f8c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c4a60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00ebc0 tx=0x7f5f1c011860 comp rx=0 tx=0).stop 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 1 -- 172.21.15.86:0/4027014703 shutdown_connections 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c4a60 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56307f8c8ce0 0x56307f8cb120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b3990 0x56307f8bda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 1 --2- 172.21.15.86:0/4027014703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b2fa0 0x56307f8bd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f03400640 0 client.16185 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:16:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 1 -- 172.21.15.86:0/4027014703 >> 172.21.15.86:0/4027014703 conn(0x56307f8ae510 msgr2=0x56307f8ad710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 1 -- 172.21.15.86:0/4027014703 shutdown_connections 2024-04-17T07:16:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 1 -- 172.21.15.86:0/4027014703 wait complete. 2024-04-17T07:16:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.935+0000 7f5f2b995740 20 client.16185 trim_cache size 0 max 16384 2024-04-17T07:16:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.952+0000 7f5f2b995740 1 Processor -- start 2024-04-17T07:16:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f2b995740 1 -- start start 2024-04-17T07:16:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8bd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b8b00 0x56307f8bda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f2b995740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241acda0 con 0x56307f8b2fa0 2024-04-17T07:16:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f2b995740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241ae8e0 con 0x56307f8b8b00 2024-04-17T07:16:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f2b995740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241ad470 con 0x56307f8b3990 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f21e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8bd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f21e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8bd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34992/0 (socket says 172.21.15.86:34992) 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f21e00640 1 -- 172.21.15.86:0/4032190618 learned_addr learned my addr 172.21.15.86:0/4032190618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.953+0000 7f5f21400640 1 --2- 172.21.15.86:0/4032190618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.954+0000 7f5f22800640 1 --2- 172.21.15.86:0/4032190618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b8b00 0x56307f8bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.954+0000 7f5f21e00640 1 -- 172.21.15.86:0/4032190618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 msgr2=0x56307f8c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.954+0000 7f5f21e00640 1 --2- 172.21.15.86:0/4032190618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.954+0000 7f5f21e00640 1 -- 172.21.15.86:0/4032190618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b8b00 msgr2=0x56307f8bda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.954+0000 7f5f21e00640 1 --2- 172.21.15.86:0/4032190618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b8b00 0x56307f8bda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.954+0000 7f5f21e00640 1 -- 172.21.15.86:0/4032190618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307f52b3f0 con 0x56307f8b2fa0 2024-04-17T07:16:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.954+0000 7f5f21e00640 1 --2- 172.21.15.86:0/4032190618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8bd4e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5f24103000 tx=0x7f5f14043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1461cc1befb7aaa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.954+0000 7f5f0e600640 1 -- 172.21.15.86:0/4032190618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14039040 con 0x56307f8b2fa0 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.955+0000 7f5f0e600640 1 -- 172.21.15.86:0/4032190618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f14056070 con 0x56307f8b2fa0 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.955+0000 7f5f0e600640 1 -- 172.21.15.86:0/4032190618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14040be0 con 0x56307f8b2fa0 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.955+0000 7f5f2b995740 1 -- 172.21.15.86:0/4032190618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 msgr2=0x56307f8bd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.955+0000 7f5f2b995740 1 --2- 172.21.15.86:0/4032190618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8bd4e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5f24103000 tx=0x7f5f14043000 comp rx=0 tx=0).stop 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.955+0000 7f5f2b995740 1 -- 172.21.15.86:0/4032190618 shutdown_connections 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.955+0000 7f5f2b995740 1 --2- 172.21.15.86:0/4032190618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f8b8b00 0x56307f8bda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.955+0000 7f5f2b995740 1 --2- 172.21.15.86:0/4032190618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.955+0000 7f5f2b995740 1 --2- 172.21.15.86:0/4032190618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b2fa0 0x56307f8bd4e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.955+0000 7f5f2b995740 1 -- 172.21.15.86:0/4032190618 >> 172.21.15.86:0/4032190618 conn(0x56307f8ae510 msgr2=0x56307f8b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.956+0000 7f5f2b995740 1 -- 172.21.15.86:0/4032190618 shutdown_connections 2024-04-17T07:16:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.956+0000 7f5f2b995740 1 -- 172.21.15.86:0/4032190618 wait complete. 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.956+0000 7f5f2b995740 1 -- start start 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.956+0000 7f5f2b995740 1 Processor -- start 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f2b995740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f92b240 0x56307f92d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f2b995740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241ae210 con 0x56307f8b8b00 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f2b995740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241adb40 con 0x56307f92b240 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f2b995740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241aad70 con 0x56307f8b3990 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f2b995740 10 client.0 _ll_register_callbacks cb 0x7f5f1c011470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f21400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35008/0 (socket says 172.21.15.86:35008) 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f21400640 1 -- 172.21.15.86:0/92897691 learned_addr learned my addr 172.21.15.86:0/92897691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f21e00640 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f21400640 1 -- 172.21.15.86:0/92897691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 msgr2=0x56307f8c19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f21400640 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8c19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f21400640 1 -- 172.21.15.86:0/92897691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f92b240 msgr2=0x56307f92d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f21400640 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f92b240 0x56307f92d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.957+0000 7f5f21400640 1 -- 172.21.15.86:0/92897691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307f5f57d0 con 0x56307f8b8b00 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.958+0000 7f5f21400640 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c1ef0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5f100987a0 tx=0x7f5f1009c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae8090bc3b587c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.958+0000 7f5f03400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.958+0000 7f5f03400640 1 -- 172.21.15.86:0/92897691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10020040 con 0x56307f8b8b00 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.958+0000 7f5f03400640 1 -- 172.21.15.86:0/92897691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5f100d3ba0 con 0x56307f8b8b00 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.958+0000 7f5f2b995740 1 -- 172.21.15.86:0/92897691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f14056070 con 0x56307f8b8b00 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.958+0000 7f5f03400640 1 -- 172.21.15.86:0/92897691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f100d0170 con 0x56307f8b8b00 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.958+0000 7f5f2b995740 10 client.16191 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.958+0000 7f5f2b995740 1 -- 172.21.15.86:0/92897691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56307f92ed20 con 0x56307f8b8b00 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.959+0000 7f5f03400640 1 -- 172.21.15.86:0/92897691 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f14056070 con 0x56307f8b8b00 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.959+0000 7f5f03400640 1 -- 172.21.15.86:0/92897691 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5f10098090 con 0x56307f8b8b00 2024-04-17T07:16:51.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.959+0000 7f5f2b995740 10 client.16191 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:51.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.959+0000 7f5f2b995740 10 client.16191 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:16:51.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.959+0000 7f5f2b995740 20 client.16191 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:51.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.959+0000 7f5f2b995740 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56307f92f6a0 0x56307f931ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:51.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.959+0000 7f5f2b995740 4 client.16191 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:16:51.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.959+0000 7f5f2b995740 1 -- 172.21.15.86:0/92897691 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56307f8208b0 con 0x56307f92f6a0 2024-04-17T07:16:51.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.960+0000 7f5f21e00640 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56307f92f6a0 0x56307f931ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:51.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f21e00640 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56307f92f6a0 0x56307f931ac0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23cf4294e8fbe474 server_cookie=417afe0b5d8e59f1 in_seq=0 out_seq=0 2024-04-17T07:16:51.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f03400640 10 client.16191 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:16:51.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f03400640 1 -- 172.21.15.86:0/92897691 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56307f8208b0 con 0x56307f92f6a0 2024-04-17T07:16:51.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f03400640 10 client.16191 handle_command_reply: tid=0 2024-04-17T07:16:51.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f2b995740 1 client.16191 shutdown 2024-04-17T07:16:51.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f2b995740 1 -- 172.21.15.86:0/92897691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 msgr2=0x56307f8c1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f2b995740 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c1ef0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5f100987a0 tx=0x7f5f1009c210 comp rx=0 tx=0).stop 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f2b995740 1 -- 172.21.15.86:0/92897691 shutdown_connections 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f2b995740 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56307f92f6a0 0x56307f931ac0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f2b995740 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56307f92b240 0x56307f92d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f2b995740 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56307f8b8b00 0x56307f8c1ef0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f03400640 0 client.16191 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f2b995740 1 --2- 172.21.15.86:0/92897691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56307f8b3990 0x56307f8c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f2b995740 1 -- 172.21.15.86:0/92897691 >> 172.21.15.86:0/92897691 conn(0x56307f8ae510 msgr2=0x56307f8bd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.961+0000 7f5f2b995740 1 -- 172.21.15.86:0/92897691 shutdown_connections 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f2b995740 1 -- 172.21.15.86:0/92897691 wait complete. 2024-04-17T07:16:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f2b995740 20 client.16191 trim_cache size 0 max 16384 2024-04-17T07:16:51.965 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:16:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f28c00640 1 -- 172.21.15.86:0/3889675890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f080a3c30 msgr2=0x7f5f080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f28c00640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f080a3c30 0x7f5f080a60f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5f1402c710 tx=0x7f5f14005c30 comp rx=0 tx=0).stop 2024-04-17T07:16:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f28c00640 1 -- 172.21.15.86:0/3889675890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f241031a0 msgr2=0x7f5f24074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f28c00640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f241031a0 0x7f5f24074350 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000d950 tx=0x7f5f1000de10 comp rx=0 tx=0).stop 2024-04-17T07:16:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f28c00640 1 -- 172.21.15.86:0/3889675890 shutdown_connections 2024-04-17T07:16:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f28c00640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f080a3c30 0x7f5f080a60f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f28c00640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f24103b60 0x7f5f24074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f28c00640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f241031a0 0x7f5f24074350 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.962+0000 7f5f28c00640 1 --2- 172.21.15.86:0/3889675890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f24101fb0 0x7f5f24075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.963+0000 7f5f28c00640 1 -- 172.21.15.86:0/3889675890 >> 172.21.15.86:0/3889675890 conn(0x7f5f240fd540 msgr2=0x7f5f24106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.963+0000 7f5f28c00640 1 -- 172.21.15.86:0/3889675890 shutdown_connections 2024-04-17T07:16:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:51.963+0000 7f5f28c00640 1 -- 172.21.15.86:0/3889675890 wait complete. 2024-04-17T07:16:51.978 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:51 smithi086 ceph-mon[31733]: pgmap v515: 129 pgs: 129 active+clean; 259 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.5 MiB/s wr, 17 op/s 2024-04-17T07:16:51.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:16:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.204+0000 7f5f36e00640 1 -- 172.21.15.86:0/1987872634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f301028f0 msgr2=0x7f5f30104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.204+0000 7f5f36e00640 1 --2- 172.21.15.86:0/1987872634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f301028f0 0x7f5f30104ce0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009900 tx=0x7f5f2402bc30 comp rx=0 tx=0).stop 2024-04-17T07:16:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.204+0000 7f5f36e00640 1 -- 172.21.15.86:0/1987872634 shutdown_connections 2024-04-17T07:16:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.204+0000 7f5f36e00640 1 --2- 172.21.15.86:0/1987872634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f30107b50 0x7f5f30109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.204+0000 7f5f36e00640 1 --2- 172.21.15.86:0/1987872634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f30105220 0x7f5f30107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.204+0000 7f5f36e00640 1 --2- 172.21.15.86:0/1987872634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f301028f0 0x7f5f30104ce0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.204+0000 7f5f36e00640 1 -- 172.21.15.86:0/1987872634 >> 172.21.15.86:0/1987872634 conn(0x7f5f300fc300 msgr2=0x7f5f300fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.204+0000 7f5f36e00640 1 -- 172.21.15.86:0/1987872634 shutdown_connections 2024-04-17T07:16:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.204+0000 7f5f36e00640 1 -- 172.21.15.86:0/1987872634 wait complete. 2024-04-17T07:16:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.205+0000 7f5f36e00640 1 Processor -- start 2024-04-17T07:16:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.206+0000 7f5f36e00640 1 -- start start 2024-04-17T07:16:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.206+0000 7f5f36e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f301028f0 0x7f5f301980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.206+0000 7f5f36e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f30105220 0x7f5f30198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.206+0000 7f5f36e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f30107b50 0x7f5f3019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.206+0000 7f5f36e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3010d9f0 con 0x7f5f30105220 2024-04-17T07:16:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.206+0000 7f5f36e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3010d870 con 0x7f5f301028f0 2024-04-17T07:16:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.206+0000 7f5f36e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3010db70 con 0x7f5f30107b50 2024-04-17T07:16:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.206+0000 7f5f35000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f30105220 0x7f5f30198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.206+0000 7f5f35000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f30105220 0x7f5f30198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35028/0 (socket says 172.21.15.86:35028) 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.207+0000 7f5f35000640 1 -- 172.21.15.86:0/3832401085 learned_addr learned my addr 172.21.15.86:0/3832401085 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.207+0000 7f5f35a00640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f301028f0 0x7f5f301980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.207+0000 7f5f36400640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f30107b50 0x7f5f3019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.207+0000 7f5f35000640 1 -- 172.21.15.86:0/3832401085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f30107b50 msgr2=0x7f5f3019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.207+0000 7f5f35000640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f30107b50 0x7f5f3019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.207+0000 7f5f35000640 1 -- 172.21.15.86:0/3832401085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f301028f0 msgr2=0x7f5f301980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.207+0000 7f5f35000640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f301028f0 0x7f5f301980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.207+0000 7f5f35000640 1 -- 172.21.15.86:0/3832401085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3019fce0 con 0x7f5f30105220 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.207+0000 7f5f35000640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f30105220 0x7f5f30198600 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c005fd0 tx=0x7f5f1c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.208+0000 7f5f2e000640 1 -- 172.21.15.86:0/3832401085 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c017070 con 0x7f5f30105220 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.208+0000 7f5f2e000640 1 -- 172.21.15.86:0/3832401085 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1c009c30 con 0x7f5f30105220 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.208+0000 7f5f36e00640 1 -- 172.21.15.86:0/3832401085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3019ffd0 con 0x7f5f30105220 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.208+0000 7f5f2e000640 1 -- 172.21.15.86:0/3832401085 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c003d40 con 0x7f5f30105220 2024-04-17T07:16:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.208+0000 7f5f36e00640 1 -- 172.21.15.86:0/3832401085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f301a0490 con 0x7f5f30105220 2024-04-17T07:16:52.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.212+0000 7f5f3affd740 1 Processor -- start 2024-04-17T07:16:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.212+0000 7f5f2e000640 1 -- 172.21.15.86:0/3832401085 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5f1c003ee0 con 0x7f5f30105220 2024-04-17T07:16:52.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.212+0000 7f5f3affd740 1 -- start start 2024-04-17T07:16:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.212+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c8e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.212+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c93650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.212+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c96150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.212+0000 7f5f3affd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0bf4020 con 0x55d7c0c8e160 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.212+0000 7f5f3affd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0b9a460 con 0x55d7c0c8eb50 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.212+0000 7f5f3affd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0beffe0 con 0x55d7c0c93cc0 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.213+0000 7f5f36400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.213+0000 7f5f36400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c8e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35044/0 (socket says 172.21.15.86:35044) 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.213+0000 7f5f36400640 1 -- 172.21.15.86:0/3130063347 learned_addr learned my addr 172.21.15.86:0/3130063347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.213+0000 7f5f35a00640 1 --2- 172.21.15.86:0/3130063347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c93650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.213+0000 7f5f35000640 1 --2- 172.21.15.86:0/3130063347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c96150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.213+0000 7f5f36400640 1 -- 172.21.15.86:0/3130063347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 msgr2=0x55d7c0c96150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.213+0000 7f5f36400640 1 --2- 172.21.15.86:0/3130063347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c96150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.213+0000 7f5f36400640 1 -- 172.21.15.86:0/3130063347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8eb50 msgr2=0x55d7c0c93650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.213+0000 7f5f36400640 1 --2- 172.21.15.86:0/3130063347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c93650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.213+0000 7f5f36400640 1 -- 172.21.15.86:0/3130063347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c0ba2040 con 0x55d7c0c8e160 2024-04-17T07:16:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.214+0000 7f5f36400640 1 --2- 172.21.15.86:0/3130063347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c8e580 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009e60 tx=0x7f5f28010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816c04a9af9e65cb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:52.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.214+0000 7f5f2e000640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3bb0 0x7f5f180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.214+0000 7f5f22a00640 1 -- 172.21.15.86:0/3130063347 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28016070 con 0x55d7c0c8e160 2024-04-17T07:16:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.214+0000 7f5f22a00640 1 -- 172.21.15.86:0/3130063347 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f28004d20 con 0x55d7c0c8e160 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.214+0000 7f5f22a00640 1 -- 172.21.15.86:0/3130063347 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f280126b0 con 0x55d7c0c8e160 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.214+0000 7f5f2e000640 1 -- 172.21.15.86:0/3832401085 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5f1c0d0510 con 0x7f5f30105220 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.215+0000 7f5f3affd740 1 -- 172.21.15.86:0/3130063347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 msgr2=0x55d7c0c8e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.215+0000 7f5f3affd740 1 --2- 172.21.15.86:0/3130063347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c8e580 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009e60 tx=0x7f5f28010740 comp rx=0 tx=0).stop 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.215+0000 7f5f35a00640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3bb0 0x7f5f180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.215+0000 7f5f3affd740 1 -- 172.21.15.86:0/3130063347 shutdown_connections 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.216+0000 7f5f3affd740 1 --2- 172.21.15.86:0/3130063347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c96150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.216+0000 7f5f3affd740 1 --2- 172.21.15.86:0/3130063347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c93650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.216+0000 7f5f3affd740 1 --2- 172.21.15.86:0/3130063347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c8e580 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.216+0000 7f5f3affd740 1 -- 172.21.15.86:0/3130063347 >> 172.21.15.86:0/3130063347 conn(0x55d7c0c896d0 msgr2=0x55d7c0c89ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:52.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.216+0000 7f5f3affd740 1 -- 172.21.15.86:0/3130063347 shutdown_connections 2024-04-17T07:16:52.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.216+0000 7f5f3affd740 1 -- 172.21.15.86:0/3130063347 wait complete. 2024-04-17T07:16:52.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.216+0000 7f5f35a00640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3bb0 0x7f5f180a6070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006010 tx=0x7f5f2402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:52.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.216+0000 7f5f3affd740 1 -- start start 2024-04-17T07:16:52.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.217+0000 7f5f3affd740 1 Processor -- start 2024-04-17T07:16:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.217+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.217+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c98b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.217+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c9fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.217+0000 7f5f3affd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0c96c70 con 0x55d7c0c8e160 2024-04-17T07:16:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.217+0000 7f5f3affd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0bad830 con 0x55d7c0c93cc0 2024-04-17T07:16:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.217+0000 7f5f3affd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7c0c96df0 con 0x55d7c0c8eb50 2024-04-17T07:16:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.217+0000 7f5f3affd740 10 client.0 _ll_register_callbacks cb 0x55d7c0b7bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.217+0000 7f5f35a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.218+0000 7f5f35a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35052/0 (socket says 172.21.15.86:35052) 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.218+0000 7f5f35a00640 1 -- 172.21.15.86:0/1238899097 learned_addr learned my addr 172.21.15.86:0/1238899097 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.218+0000 7f5f35000640 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c98b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.218+0000 7f5f36400640 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c9fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.218+0000 7f5f35a00640 1 -- 172.21.15.86:0/1238899097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c8eb50 msgr2=0x55d7c0c98b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.218+0000 7f5f35a00640 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c98b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.218+0000 7f5f35a00640 1 -- 172.21.15.86:0/1238899097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c93cc0 msgr2=0x55d7c0c9fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.218+0000 7f5f35a00640 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c9fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.218+0000 7f5f35a00640 1 -- 172.21.15.86:0/1238899097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c0bb1920 con 0x55d7c0c8e160 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.219+0000 7f5f35a00640 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98650 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5f24035000 tx=0x7f5f2402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995c1d86f7bf06e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.219+0000 7f5f17400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:52.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.219+0000 7f5f17400640 1 -- 172.21.15.86:0/1238899097 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2404e030 con 0x55d7c0c8e160 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.219+0000 7f5f17400640 1 -- 172.21.15.86:0/1238899097 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5f240046c0 con 0x55d7c0c8e160 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.219+0000 7f5f3affd740 1 -- 172.21.15.86:0/1238899097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f28004d20 con 0x55d7c0c8e160 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.219+0000 7f5f17400640 1 -- 172.21.15.86:0/1238899097 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f2405b050 con 0x55d7c0c8e160 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.219+0000 7f5f3affd740 10 client.16203 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.219+0000 7f5f3affd740 1 -- 172.21.15.86:0/1238899097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7c0b7f7f0 con 0x55d7c0c8e160 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.220+0000 7f5f17400640 1 -- 172.21.15.86:0/1238899097 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f28004d20 con 0x55d7c0c8e160 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.220+0000 7f5f17400640 1 -- 172.21.15.86:0/1238899097 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5f24003ba0 con 0x55d7c0c8e160 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.220+0000 7f5f3affd740 10 client.16203 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.220+0000 7f5f3affd740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.220+0000 7f5f3affd740 20 client.16203 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.220+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c0ca4120 0x55d7c0ca65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.220+0000 7f5f3affd740 4 client.16203 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:52.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.220+0000 7f5f3affd740 1 -- 172.21.15.86:0/1238899097 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7c0c01a40 con 0x55d7c0ca4120 2024-04-17T07:16:52.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.221+0000 7f5f35000640 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c0ca4120 0x55d7c0ca65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.222+0000 7f5f35000640 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c0ca4120 0x55d7c0ca65e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43b098e06f481cc7 server_cookie=13db4686f27a0fc in_seq=0 out_seq=0 2024-04-17T07:16:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.222+0000 7f5f17400640 10 client.16203 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f17400640 1 -- 172.21.15.86:0/1238899097 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7c0c01a40 con 0x55d7c0ca4120 2024-04-17T07:16:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f17400640 10 client.16203 handle_command_reply: tid=0 2024-04-17T07:16:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 client.16203 shutdown 2024-04-17T07:16:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 -- 172.21.15.86:0/1238899097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 msgr2=0x55d7c0c98650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98650 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5f24035000 tx=0x7f5f2402bc30 comp rx=0 tx=0).stop 2024-04-17T07:16:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 -- 172.21.15.86:0/1238899097 shutdown_connections 2024-04-17T07:16:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c0ca4120 0x55d7c0ca65e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c9fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c98b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f17400640 0 client.16203 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:52.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1238899097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98650 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 -- 172.21.15.86:0/1238899097 >> 172.21.15.86:0/1238899097 conn(0x55d7c0c896d0 msgr2=0x55d7c0c88930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:52.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 -- 172.21.15.86:0/1238899097 shutdown_connections 2024-04-17T07:16:52.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 1 -- 172.21.15.86:0/1238899097 wait complete. 2024-04-17T07:16:52.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.224+0000 7f5f3affd740 20 client.16203 trim_cache size 0 max 16384 2024-04-17T07:16:52.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.243+0000 7f5f3affd740 1 Processor -- start 2024-04-17T07:16:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.243+0000 7f5f3affd740 1 -- start start 2024-04-17T07:16:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0caaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f3affd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301aa8e0 con 0x55d7c0c8eb50 2024-04-17T07:16:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f3affd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301ac420 con 0x55d7c0c8e160 2024-04-17T07:16:52.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f3affd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301aafb0 con 0x55d7c0c93cc0 2024-04-17T07:16:52.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f35000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f35000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c887b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35064/0 (socket says 172.21.15.86:35064) 2024-04-17T07:16:52.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f35000640 1 -- 172.21.15.86:0/1595192422 learned_addr learned my addr 172.21.15.86:0/1595192422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f35a00640 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f36400640 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0caaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.244+0000 7f5f35000640 1 -- 172.21.15.86:0/1595192422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 msgr2=0x55d7c0caaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.245+0000 7f5f35000640 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0caaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.245+0000 7f5f35000640 1 -- 172.21.15.86:0/1595192422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 msgr2=0x55d7c0c98af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.245+0000 7f5f35000640 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.245+0000 7f5f35000640 1 -- 172.21.15.86:0/1595192422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c09063f0 con 0x55d7c0c8eb50 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.245+0000 7f5f36400640 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0caaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.245+0000 7f5f35a00640 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.245+0000 7f5f35000640 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c887b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c000c00 tx=0x7f5f1c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204625fa39b68cab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.245+0000 7f5f22000640 1 -- 172.21.15.86:0/1595192422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c09cb70 con 0x55d7c0c8eb50 2024-04-17T07:16:52.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.245+0000 7f5f22000640 1 -- 172.21.15.86:0/1595192422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1c009de0 con 0x55d7c0c8eb50 2024-04-17T07:16:52.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.245+0000 7f5f22000640 1 -- 172.21.15.86:0/1595192422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0d0250 con 0x55d7c0c8eb50 2024-04-17T07:16:52.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.246+0000 7f5f3affd740 1 -- 172.21.15.86:0/1595192422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 msgr2=0x55d7c0c887b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:52.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.246+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c887b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c000c00 tx=0x7f5f1c005bc0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.246+0000 7f5f3affd740 1 -- 172.21.15.86:0/1595192422 shutdown_connections 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.246+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0caaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.246+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c887b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.246+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1595192422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 0x55d7c0c98af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.246+0000 7f5f3affd740 1 -- 172.21.15.86:0/1595192422 >> 172.21.15.86:0/1595192422 conn(0x55d7c0c896d0 msgr2=0x55d7c0c880a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.246+0000 7f5f3affd740 1 -- 172.21.15.86:0/1595192422 shutdown_connections 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.246+0000 7f5f3affd740 1 -- 172.21.15.86:0/1595192422 wait complete. 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.247+0000 7f5f3affd740 1 -- start start 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.247+0000 7f5f3affd740 1 Processor -- start 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 0x55d7c0c9d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c9d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f3affd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c9e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f3affd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301abd50 con 0x55d7c0c8eb50 2024-04-17T07:16:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f3affd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301ab680 con 0x55d7c0c8e160 2024-04-17T07:16:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f3affd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301a88b0 con 0x55d7c0c93cc0 2024-04-17T07:16:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f3affd740 10 client.0 _ll_register_callbacks cb 0x7f5f240040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f35000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c9d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f35000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c9d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35078/0 (socket says 172.21.15.86:35078) 2024-04-17T07:16:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f35000640 1 -- 172.21.15.86:0/1453259880 learned_addr learned my addr 172.21.15.86:0/1453259880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.248+0000 7f5f36400640 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c9e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.249+0000 7f5f35a00640 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 0x55d7c0c9d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.249+0000 7f5f35000640 1 -- 172.21.15.86:0/1453259880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 msgr2=0x55d7c0c9e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:52.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.249+0000 7f5f35000640 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c9e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.249+0000 7f5f35000640 1 -- 172.21.15.86:0/1453259880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 msgr2=0x55d7c0c9d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:52.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.249+0000 7f5f35000640 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 0x55d7c0c9d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.249+0000 7f5f35000640 1 -- 172.21.15.86:0/1453259880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c09d07d0 con 0x55d7c0c8eb50 2024-04-17T07:16:52.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.249+0000 7f5f35000640 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c9d6c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c09c170 tx=0x7f5f1c0a5160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5f63929787624d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:52.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.249+0000 7f5f17400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.250+0000 7f5f17400640 1 -- 172.21.15.86:0/1453259880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c09cb70 con 0x55d7c0c8eb50 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.250+0000 7f5f17400640 1 -- 172.21.15.86:0/1453259880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5f1c0d0250 con 0x55d7c0c8eb50 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.250+0000 7f5f3affd740 1 -- 172.21.15.86:0/1453259880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1c009de0 con 0x55d7c0c8eb50 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.250+0000 7f5f17400640 1 -- 172.21.15.86:0/1453259880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c098ae0 con 0x55d7c0c8eb50 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.250+0000 7f5f3affd740 10 client.16209 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.250+0000 7f5f3affd740 1 -- 172.21.15.86:0/1453259880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7c0c9fcc0 con 0x55d7c0c8eb50 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.251+0000 7f5f17400640 1 -- 172.21.15.86:0/1453259880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1c009de0 con 0x55d7c0c8eb50 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.251+0000 7f5f17400640 1 -- 172.21.15.86:0/1453259880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5f1c098380 con 0x55d7c0c8eb50 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.251+0000 7f5f3affd740 10 client.16209 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.251+0000 7f5f3affd740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.251+0000 7f5f3affd740 20 client.16209 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.251+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c0d13440 0x55d7c0d15860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:52.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.251+0000 7f5f3affd740 4 client.16209 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:16:52.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.251+0000 7f5f3affd740 1 -- 172.21.15.86:0/1453259880 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7c0bfb8b0 con 0x55d7c0d13440 2024-04-17T07:16:52.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.252+0000 7f5f35a00640 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c0d13440 0x55d7c0d15860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:52.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:51 smithi098 ceph-mon[35418]: pgmap v515: 129 pgs: 129 active+clean; 259 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.5 MiB/s wr, 17 op/s 2024-04-17T07:16:52.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.253+0000 7f5f35a00640 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c0d13440 0x55d7c0d15860 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d0bf51f642008c server_cookie=47f613ccb944af06 in_seq=0 out_seq=0 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.253+0000 7f5f17400640 10 client.16209 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f17400640 1 -- 172.21.15.86:0/1453259880 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7c0bfb8b0 con 0x55d7c0d13440 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f17400640 10 client.16209 handle_command_reply: tid=0 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f3affd740 1 client.16209 shutdown 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f3affd740 1 -- 172.21.15.86:0/1453259880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 msgr2=0x55d7c0c9d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c9d6c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c09c170 tx=0x7f5f1c0a5160 comp rx=0 tx=0).stop 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f3affd740 1 -- 172.21.15.86:0/1453259880 shutdown_connections 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7c0d13440 0x55d7c0d15860 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7c0c93cc0 0x55d7c0c9e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7c0c8eb50 0x55d7c0c9d6c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f3affd740 1 --2- 172.21.15.86:0/1453259880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7c0c8e160 0x55d7c0c9d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f3affd740 1 -- 172.21.15.86:0/1453259880 >> 172.21.15.86:0/1453259880 conn(0x55d7c0c896d0 msgr2=0x55d7c0c98970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:52.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f17400640 0 client.16209 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:52.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.254+0000 7f5f3affd740 1 -- 172.21.15.86:0/1453259880 shutdown_connections 2024-04-17T07:16:52.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.255+0000 7f5f3affd740 1 -- 172.21.15.86:0/1453259880 wait complete. 2024-04-17T07:16:52.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.255+0000 7f5f3affd740 20 client.16209 trim_cache size 0 max 16384 2024-04-17T07:16:52.257 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.255+0000 7f5f36e00640 1 -- 172.21.15.86:0/3832401085 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3bb0 msgr2=0x7f5f180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.255+0000 7f5f36e00640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3bb0 0x7f5f180a6070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006010 tx=0x7f5f2402c7e0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.255+0000 7f5f36e00640 1 -- 172.21.15.86:0/3832401085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f30105220 msgr2=0x7f5f30198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.255+0000 7f5f36e00640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f30105220 0x7f5f30198600 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c005fd0 tx=0x7f5f1c00dde0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.256+0000 7f5f36e00640 1 -- 172.21.15.86:0/3832401085 shutdown_connections 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.256+0000 7f5f36e00640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3bb0 0x7f5f180a6070 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.256+0000 7f5f36e00640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f30107b50 0x7f5f3019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.256+0000 7f5f36e00640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f30105220 0x7f5f30198600 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.256+0000 7f5f36e00640 1 --2- 172.21.15.86:0/3832401085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f301028f0 0x7f5f301980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.256+0000 7f5f36e00640 1 -- 172.21.15.86:0/3832401085 >> 172.21.15.86:0/3832401085 conn(0x7f5f300fc300 msgr2=0x7f5f30103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.256+0000 7f5f36e00640 1 -- 172.21.15.86:0/3832401085 shutdown_connections 2024-04-17T07:16:52.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:52.256+0000 7f5f36e00640 1 -- 172.21.15.86:0/3832401085 wait complete. 2024-04-17T07:16:52.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:51 smithi103 ceph-mon[35495]: pgmap v515: 129 pgs: 129 active+clean; 259 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.5 MiB/s wr, 17 op/s 2024-04-17T07:16:53.275 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:16:53.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:16:53.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:16:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.523+0000 7fa307200640 1 -- 172.21.15.86:0/3466158651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa308078500 msgr2=0x7fa308078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.523+0000 7fa307200640 1 --2- 172.21.15.86:0/3466158651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa308078500 0x7fa308078da0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa300009ca0 tx=0x7fa30002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.523+0000 7fa307200640 1 -- 172.21.15.86:0/3466158651 shutdown_connections 2024-04-17T07:16:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.523+0000 7fa307200640 1 --2- 172.21.15.86:0/3466158651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa308078500 0x7fa308078da0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.523+0000 7fa307200640 1 --2- 172.21.15.86:0/3466158651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308077b60 0x7fa308077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.523+0000 7fa307200640 1 --2- 172.21.15.86:0/3466158651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa308076920 0x7fa308076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.523+0000 7fa307200640 1 -- 172.21.15.86:0/3466158651 >> 172.21.15.86:0/3466158651 conn(0x7fa3080ff610 msgr2=0x7fa308101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.523+0000 7fa307200640 1 -- 172.21.15.86:0/3466158651 shutdown_connections 2024-04-17T07:16:53.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.523+0000 7fa307200640 1 -- 172.21.15.86:0/3466158651 wait complete. 2024-04-17T07:16:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.524+0000 7fa307200640 1 Processor -- start 2024-04-17T07:16:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.525+0000 7fa307200640 1 -- start start 2024-04-17T07:16:53.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.525+0000 7fa307200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa308076920 0x7fa30819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.525+0000 7fa307200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308077b60 0x7fa30819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.525+0000 7fa307200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa308078500 0x7fa3081a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.525+0000 7fa307200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa308113fe0 con 0x7fa308078500 2024-04-17T07:16:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.525+0000 7fa307200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa308113e60 con 0x7fa308077b60 2024-04-17T07:16:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.525+0000 7fa307200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa308114160 con 0x7fa308076920 2024-04-17T07:16:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.525+0000 7fa306800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa308078500 0x7fa3081a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.525+0000 7fa306800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa308078500 0x7fa3081a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35090/0 (socket says 172.21.15.86:35090) 2024-04-17T07:16:53.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.525+0000 7fa306800640 1 -- 172.21.15.86:0/2101784076 learned_addr learned my addr 172.21.15.86:0/2101784076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.526+0000 7fa305e00640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa308076920 0x7fa30819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.526+0000 7fa305400640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308077b60 0x7fa30819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.526+0000 7fa306800640 1 -- 172.21.15.86:0/2101784076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa308076920 msgr2=0x7fa30819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.526+0000 7fa306800640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa308076920 0x7fa30819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.526+0000 7fa306800640 1 -- 172.21.15.86:0/2101784076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308077b60 msgr2=0x7fa30819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.526+0000 7fa306800640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308077b60 0x7fa30819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.526+0000 7fa306800640 1 -- 172.21.15.86:0/2101784076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3081a6260 con 0x7fa308078500 2024-04-17T07:16:53.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.526+0000 7fa306800640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa308078500 0x7fa3081a5bf0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa300032380 tx=0x7fa300032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.527+0000 7fa2fea00640 1 -- 172.21.15.86:0/2101784076 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300041020 con 0x7fa308078500 2024-04-17T07:16:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.527+0000 7fa2fea00640 1 -- 172.21.15.86:0/2101784076 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa300003e20 con 0x7fa308078500 2024-04-17T07:16:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.527+0000 7fa2fea00640 1 -- 172.21.15.86:0/2101784076 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300039040 con 0x7fa308078500 2024-04-17T07:16:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.527+0000 7fa307200640 1 -- 172.21.15.86:0/2101784076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3081a64f0 con 0x7fa308078500 2024-04-17T07:16:53.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.527+0000 7fa307200640 1 -- 172.21.15.86:0/2101784076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3081a6990 con 0x7fa308078500 2024-04-17T07:16:53.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.531+0000 7fa30f7e3740 1 Processor -- start 2024-04-17T07:16:53.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.531+0000 7fa2fea00640 1 -- 172.21.15.86:0/2101784076 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa300044020 con 0x7fa308078500 2024-04-17T07:16:53.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.532+0000 7fa30f7e3740 1 -- start start 2024-04-17T07:16:53.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.532+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a390a0 0x561fb1a394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.532+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a3e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.532+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 0x561fb1a41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.532+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fb199ecb0 con 0x561fb1a39a90 2024-04-17T07:16:53.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.532+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fb1945370 con 0x561fb1a390a0 2024-04-17T07:16:53.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.532+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fb199ac70 con 0x561fb1a3ec00 2024-04-17T07:16:53.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.532+0000 7fa305e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a3e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.532+0000 7fa305e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a3e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35092/0 (socket says 172.21.15.86:35092) 2024-04-17T07:16:53.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.532+0000 7fa305e00640 1 -- 172.21.15.86:0/4228281605 learned_addr learned my addr 172.21.15.86:0/4228281605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:53.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa305400640 1 --2- 172.21.15.86:0/4228281605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a390a0 0x561fb1a394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa305400640 1 --2- 172.21.15.86:0/4228281605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 0x561fb1a41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa305e00640 1 -- 172.21.15.86:0/4228281605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 msgr2=0x561fb1a41090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa305e00640 1 --2- 172.21.15.86:0/4228281605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 0x561fb1a41090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa305e00640 1 -- 172.21.15.86:0/4228281605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a390a0 msgr2=0x561fb1a394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa305e00640 1 --2- 172.21.15.86:0/4228281605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a390a0 0x561fb1a394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa305e00640 1 -- 172.21.15.86:0/4228281605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fb1964910 con 0x561fb1a39a90 2024-04-17T07:16:53.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa305400640 1 --2- 172.21.15.86:0/4228281605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 0x561fb1a41090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:53.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa2fea00640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa2ec0a3d00 0x7fa2ec0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa2fea00640 1 -- 172.21.15.86:0/2101784076 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa3000f1ec0 con 0x7fa308078500 2024-04-17T07:16:53.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa305e00640 1 --2- 172.21.15.86:0/4228281605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a3e590 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009c90 tx=0x7fa2f80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b61ef5d67ce640b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.533+0000 7fa305400640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa2ec0a3d00 0x7fa2ec0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.534+0000 7fa2f3000640 1 -- 172.21.15.86:0/4228281605 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8012450 con 0x561fb1a39a90 2024-04-17T07:16:53.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.534+0000 7fa2f3000640 1 -- 172.21.15.86:0/4228281605 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2f8012aa0 con 0x561fb1a39a90 2024-04-17T07:16:53.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.534+0000 7fa2f3000640 1 -- 172.21.15.86:0/4228281605 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f800fb50 con 0x561fb1a39a90 2024-04-17T07:16:53.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa30f7e3740 1 -- 172.21.15.86:0/4228281605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 msgr2=0x561fb1a3e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/4228281605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a3e590 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009c90 tx=0x7fa2f80117b0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa305400640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa2ec0a3d00 0x7fa2ec0a61c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x561fb1a3a2f0 tx=0x7fa2f4014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa30f7e3740 1 -- 172.21.15.86:0/4228281605 shutdown_connections 2024-04-17T07:16:53.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/4228281605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 0x561fb1a41090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/4228281605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a3e590 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/4228281605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a390a0 0x561fb1a394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa30f7e3740 1 -- 172.21.15.86:0/4228281605 >> 172.21.15.86:0/4228281605 conn(0x561fb1a34610 msgr2=0x561fb1a34a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa30f7e3740 1 -- 172.21.15.86:0/4228281605 shutdown_connections 2024-04-17T07:16:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa30f7e3740 1 -- 172.21.15.86:0/4228281605 wait complete. 2024-04-17T07:16:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.535+0000 7fa30f7e3740 1 -- start start 2024-04-17T07:16:53.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.536+0000 7fa30f7e3740 1 Processor -- start 2024-04-17T07:16:53.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.536+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 0x561fb1a435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.536+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a43b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.536+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a3ec00 0x561fb1a4ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.536+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561fb1a41bb0 con 0x561fb1a39a90 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.536+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561fb1958730 con 0x561fb1a3ec00 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.536+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561fb1a41d30 con 0x561fb1a390a0 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.536+0000 7fa30f7e3740 10 client.0 _ll_register_callbacks cb 0x561fb195ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.537+0000 7fa305400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a43b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.537+0000 7fa305400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a43b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35104/0 (socket says 172.21.15.86:35104) 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.537+0000 7fa305400640 1 -- 172.21.15.86:0/2551468195 learned_addr learned my addr 172.21.15.86:0/2551468195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.537+0000 7fa305e00640 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 0x561fb1a435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.537+0000 7fa306800640 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a3ec00 0x561fb1a4ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.537+0000 7fa305400640 1 -- 172.21.15.86:0/2551468195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 msgr2=0x561fb1a435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.537+0000 7fa305400640 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 0x561fb1a435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.537+0000 7fa305400640 1 -- 172.21.15.86:0/2551468195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a3ec00 msgr2=0x561fb1a4ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.537+0000 7fa305400640 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a3ec00 0x561fb1a4ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.537+0000 7fa305400640 1 -- 172.21.15.86:0/2551468195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fb17797e0 con 0x561fb1a39a90 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.538+0000 7fa305400640 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a43b20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009fd0 tx=0x7fa2f4009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772c252689c7331d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.538+0000 7fa2e7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.538+0000 7fa2e7400640 1 -- 172.21.15.86:0/2551468195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4010930 con 0x561fb1a39a90 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.538+0000 7fa2e7400640 1 -- 172.21.15.86:0/2551468195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa2f4010ad0 con 0x561fb1a39a90 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.538+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2551468195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f8012aa0 con 0x561fb1a39a90 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.538+0000 7fa2e7400640 1 -- 172.21.15.86:0/2551468195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2f4010d10 con 0x561fb1a39a90 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.538+0000 7fa30f7e3740 10 client.16221 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.538+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2551468195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fb17d9f70 con 0x561fb1a39a90 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.539+0000 7fa2e7400640 1 -- 172.21.15.86:0/2551468195 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2f8012aa0 con 0x561fb1a39a90 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.539+0000 7fa2e7400640 1 -- 172.21.15.86:0/2551468195 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa2f401b070 con 0x561fb1a39a90 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.539+0000 7fa30f7e3740 10 client.16221 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:53.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.539+0000 7fa30f7e3740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.539+0000 7fa30f7e3740 20 client.16221 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.539+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561fb1a4f550 0x561fb1a519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.539+0000 7fa30f7e3740 4 client.16221 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.539+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2551468195 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fb19ac6d0 con 0x561fb1a4f550 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.540+0000 7fa305e00640 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561fb1a4f550 0x561fb1a519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.541+0000 7fa2e7400640 10 client.16221 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.541+0000 7fa305e00640 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561fb1a4f550 0x561fb1a519d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a818af9f392844b server_cookie=3dfbda3fcb7c9961 in_seq=0 out_seq=0 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.542+0000 7fa2e7400640 1 -- 172.21.15.86:0/2551468195 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fb19ac6d0 con 0x561fb1a4f550 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.542+0000 7fa2e7400640 10 client.16221 handle_command_reply: tid=0 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.542+0000 7fa30f7e3740 1 client.16221 shutdown 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2551468195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 msgr2=0x561fb1a43b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a43b20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009fd0 tx=0x7fa2f4009640 comp rx=0 tx=0).stop 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2551468195 shutdown_connections 2024-04-17T07:16:53.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561fb1a4f550 0x561fb1a519d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a3ec00 0x561fb1a4ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa2e7400640 0 client.16221 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a43b20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2551468195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 0x561fb1a435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2551468195 >> 172.21.15.86:0/2551468195 conn(0x561fb1a34610 msgr2=0x561fb1a33990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2551468195 shutdown_connections 2024-04-17T07:16:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2551468195 wait complete. 2024-04-17T07:16:53.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.543+0000 7fa30f7e3740 20 client.16221 trim_cache size 0 max 16384 2024-04-17T07:16:53.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa30f7e3740 1 Processor -- start 2024-04-17T07:16:53.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa30f7e3740 1 -- start start 2024-04-17T07:16:53.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a390a0 0x561fb1a33100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a4ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 0x561fb1a33640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b0dc0 con 0x561fb1a39a90 2024-04-17T07:16:53.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b2900 con 0x561fb1a390a0 2024-04-17T07:16:53.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b1490 con 0x561fb1a3ec00 2024-04-17T07:16:53.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa305400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a4ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa305400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a4ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35108/0 (socket says 172.21.15.86:35108) 2024-04-17T07:16:53.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.565+0000 7fa305400640 1 -- 172.21.15.86:0/567188260 learned_addr learned my addr 172.21.15.86:0/567188260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:53.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.566+0000 7fa306800640 1 --2- 172.21.15.86:0/567188260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 0x561fb1a33640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.566+0000 7fa305e00640 1 --2- 172.21.15.86:0/567188260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a390a0 0x561fb1a33100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.566+0000 7fa305400640 1 -- 172.21.15.86:0/567188260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 msgr2=0x561fb1a33640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.566+0000 7fa305400640 1 --2- 172.21.15.86:0/567188260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 0x561fb1a33640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.566+0000 7fa305400640 1 -- 172.21.15.86:0/567188260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a390a0 msgr2=0x561fb1a33100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.566+0000 7fa305400640 1 --2- 172.21.15.86:0/567188260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a390a0 0x561fb1a33100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.566+0000 7fa305400640 1 -- 172.21.15.86:0/567188260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fb1671170 con 0x561fb1a39a90 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.566+0000 7fa305400640 1 --2- 172.21.15.86:0/567188260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a4ab20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa2f401e930 tx=0x7fa2f4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672ffbe961c8be70 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.567+0000 7fa2f2600640 1 -- 172.21.15.86:0/567188260 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4010a00 con 0x561fb1a39a90 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.567+0000 7fa2f2600640 1 -- 172.21.15.86:0/567188260 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2f40054f0 con 0x561fb1a39a90 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.567+0000 7fa2f2600640 1 -- 172.21.15.86:0/567188260 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40286e0 con 0x561fb1a39a90 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.567+0000 7fa30f7e3740 1 -- 172.21.15.86:0/567188260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 msgr2=0x561fb1a4ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.567+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/567188260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a4ab20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa2f401e930 tx=0x7fa2f4009b20 comp rx=0 tx=0).stop 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.568+0000 7fa30f7e3740 1 -- 172.21.15.86:0/567188260 shutdown_connections 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.568+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/567188260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a3ec00 0x561fb1a33640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.568+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/567188260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a4ab20 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.568+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/567188260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a390a0 0x561fb1a33100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.568+0000 7fa30f7e3740 1 -- 172.21.15.86:0/567188260 >> 172.21.15.86:0/567188260 conn(0x561fb1a34610 msgr2=0x561fb1a3af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.568+0000 7fa30f7e3740 1 -- 172.21.15.86:0/567188260 shutdown_connections 2024-04-17T07:16:53.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.568+0000 7fa30f7e3740 1 -- 172.21.15.86:0/567188260 wait complete. 2024-04-17T07:16:53.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.568+0000 7fa30f7e3740 1 -- start start 2024-04-17T07:16:53.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.568+0000 7fa30f7e3740 1 Processor -- start 2024-04-17T07:16:53.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 0x561fb1a48280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa30f7e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a3ec00 0x561fb1a497d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b2230 con 0x561fb1a39a90 2024-04-17T07:16:53.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081b1b60 con 0x561fb1a3ec00 2024-04-17T07:16:53.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa30f7e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3081aed90 con 0x561fb1a390a0 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa30f7e3740 10 client.0 _ll_register_callbacks cb 0x7fa2f4009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa305e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 0x561fb1a48280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa305400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa305e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 0x561fb1a48280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40592/0 (socket says 172.21.15.86:40592) 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.569+0000 7fa305e00640 1 -- 172.21.15.86:0/2977458292 learned_addr learned my addr 172.21.15.86:0/2977458292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.570+0000 7fa306800640 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a3ec00 0x561fb1a497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.570+0000 7fa305400640 1 -- 172.21.15.86:0/2977458292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 msgr2=0x561fb1a48280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.570+0000 7fa305400640 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 0x561fb1a48280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.570+0000 7fa305400640 1 -- 172.21.15.86:0/2977458292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a3ec00 msgr2=0x561fb1a497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.570+0000 7fa305400640 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a3ec00 0x561fb1a497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.570+0000 7fa305400640 1 -- 172.21.15.86:0/2977458292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fb1a49dc0 con 0x561fb1a39a90 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.570+0000 7fa305400640 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a487c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x561fb1a2fb60 tx=0x7fa2f4004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2060aeded9524a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.570+0000 7fa2e7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:53.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.571+0000 7fa2e7400640 1 -- 172.21.15.86:0/2977458292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f403c070 con 0x561fb1a39a90 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.571+0000 7fa2e7400640 1 -- 172.21.15.86:0/2977458292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa2f4005150 con 0x561fb1a39a90 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.571+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2977458292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f40054f0 con 0x561fb1a39a90 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.571+0000 7fa2e7400640 1 -- 172.21.15.86:0/2977458292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2f4005350 con 0x561fb1a39a90 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.571+0000 7fa30f7e3740 10 client.16227 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.571+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2977458292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fb1ab5bd0 con 0x561fb1a39a90 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.571+0000 7fa2e7400640 1 -- 172.21.15.86:0/2977458292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2f40054f0 con 0x561fb1a39a90 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.572+0000 7fa2e7400640 1 -- 172.21.15.86:0/2977458292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa2f401b040 con 0x561fb1a39a90 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.572+0000 7fa30f7e3740 10 client.16227 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.572+0000 7fa30f7e3740 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.572+0000 7fa30f7e3740 20 client.16227 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.572+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561fb1ab63c0 0x561fb1ab87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.572+0000 7fa30f7e3740 4 client.16227 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.572+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2977458292 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561fb19a6540 con 0x561fb1ab63c0 2024-04-17T07:16:53.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.572+0000 7fa305e00640 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561fb1ab63c0 0x561fb1ab87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.573+0000 7fa305e00640 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561fb1ab63c0 0x561fb1ab87e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6356119941e40f5d server_cookie=ae602b361debdfaa in_seq=0 out_seq=0 2024-04-17T07:16:53.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.573+0000 7fa2e7400640 10 client.16227 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:53.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa2e7400640 1 -- 172.21.15.86:0/2977458292 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561fb19a6540 con 0x561fb1ab63c0 2024-04-17T07:16:53.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa2e7400640 10 client.16227 handle_command_reply: tid=0 2024-04-17T07:16:53.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 client.16227 shutdown 2024-04-17T07:16:53.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2977458292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 msgr2=0x561fb1a487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a487c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x561fb1a2fb60 tx=0x7fa2f4004960 comp rx=0 tx=0).stop 2024-04-17T07:16:53.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2977458292 shutdown_connections 2024-04-17T07:16:53.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561fb1ab63c0 0x561fb1ab87e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561fb1a3ec00 0x561fb1a497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561fb1a39a90 0x561fb1a487c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 --2- 172.21.15.86:0/2977458292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561fb1a390a0 0x561fb1a48280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2977458292 >> 172.21.15.86:0/2977458292 conn(0x561fb1a34610 msgr2=0x561fb1a33100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa2e7400640 0 client.16227 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:53.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2977458292 shutdown_connections 2024-04-17T07:16:53.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.576+0000 7fa30f7e3740 1 -- 172.21.15.86:0/2977458292 wait complete. 2024-04-17T07:16:53.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa30f7e3740 20 client.16227 trim_cache size 0 max 16384 2024-04-17T07:16:53.580 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:16:53.581 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:16:53.581 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "a808bde0-bbb5-461c-afbd-de5af4ad66fb", 2024-04-17T07:16:53.581 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:16:53.581 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa307200640 1 -- 172.21.15.86:0/2101784076 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa2ec0a3d00 msgr2=0x7fa2ec0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa307200640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa2ec0a3d00 0x7fa2ec0a61c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x561fb1a3a2f0 tx=0x7fa2f4014040 comp rx=0 tx=0).stop 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa307200640 1 -- 172.21.15.86:0/2101784076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa308078500 msgr2=0x7fa3081a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa307200640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa308078500 0x7fa3081a5bf0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa300032380 tx=0x7fa300032820 comp rx=0 tx=0).stop 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa307200640 1 -- 172.21.15.86:0/2101784076 shutdown_connections 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa307200640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa2ec0a3d00 0x7fa2ec0a61c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa307200640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa308078500 0x7fa3081a5bf0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa307200640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa308077b60 0x7fa30819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa307200640 1 --2- 172.21.15.86:0/2101784076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa308076920 0x7fa30819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.577+0000 7fa307200640 1 -- 172.21.15.86:0/2101784076 >> 172.21.15.86:0/2101784076 conn(0x7fa3080ff610 msgr2=0x7fa308101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.578+0000 7fa307200640 1 -- 172.21.15.86:0/2101784076 shutdown_connections 2024-04-17T07:16:53.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.578+0000 7fa307200640 1 -- 172.21.15.86:0/2101784076 wait complete. 2024-04-17T07:16:53.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:53 smithi086 ceph-mon[31733]: pgmap v516: 129 pgs: 129 active+clean; 275 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:16:53.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:16:53.597 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:16:53.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:53 smithi098 ceph-mon[35418]: pgmap v516: 129 pgs: 129 active+clean; 275 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:16:53.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:53 smithi103 ceph-mon[35495]: pgmap v516: 129 pgs: 129 active+clean; 275 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:16:53.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.827+0000 7fda57c00640 1 -- 172.21.15.86:0/2285688513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda50077b60 msgr2=0x7fda50077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.827+0000 7fda57c00640 1 --2- 172.21.15.86:0/2285688513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda50077b60 0x7fda50077fc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009900 tx=0x7fda3c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:16:53.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.827+0000 7fda57c00640 1 -- 172.21.15.86:0/2285688513 shutdown_connections 2024-04-17T07:16:53.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.827+0000 7fda57c00640 1 --2- 172.21.15.86:0/2285688513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda50078500 0x7fda50078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.827+0000 7fda57c00640 1 --2- 172.21.15.86:0/2285688513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda50077b60 0x7fda50077fc0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.827+0000 7fda57c00640 1 --2- 172.21.15.86:0/2285688513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda50076920 0x7fda50076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.827+0000 7fda57c00640 1 -- 172.21.15.86:0/2285688513 >> 172.21.15.86:0/2285688513 conn(0x7fda500ff710 msgr2=0x7fda50101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.827+0000 7fda57c00640 1 -- 172.21.15.86:0/2285688513 shutdown_connections 2024-04-17T07:16:53.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.827+0000 7fda57c00640 1 -- 172.21.15.86:0/2285688513 wait complete. 2024-04-17T07:16:53.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.828+0000 7fda57c00640 1 Processor -- start 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.828+0000 7fda57c00640 1 -- start start 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda57c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda50076920 0x7fda5019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda57c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda50077b60 0x7fda5019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda57c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda50078500 0x7fda501a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda57c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda50114180 con 0x7fda50076920 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda57c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda50114000 con 0x7fda50077b60 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda57c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda50114300 con 0x7fda50078500 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda55200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda50076920 0x7fda5019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda55200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda50076920 0x7fda5019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35142/0 (socket says 172.21.15.86:35142) 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda55200640 1 -- 172.21.15.86:0/2080421090 learned_addr learned my addr 172.21.15.86:0/2080421090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda4fe00640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda50077b60 0x7fda5019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.829+0000 7fda55c00640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda50078500 0x7fda501a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.830+0000 7fda55200640 1 -- 172.21.15.86:0/2080421090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda50078500 msgr2=0x7fda501a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.830+0000 7fda55200640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda50078500 0x7fda501a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.830+0000 7fda55200640 1 -- 172.21.15.86:0/2080421090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda50077b60 msgr2=0x7fda5019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.830+0000 7fda55200640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda50077b60 0x7fda5019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.830+0000 7fda55200640 1 -- 172.21.15.86:0/2080421090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda501a6480 con 0x7fda50076920 2024-04-17T07:16:53.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.830+0000 7fda55200640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda50076920 0x7fda5019e860 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fda4400b4a0 tx=0x7fda4400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.830+0000 7fda4d600640 1 -- 172.21.15.86:0/2080421090 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44014070 con 0x7fda50076920 2024-04-17T07:16:53.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.831+0000 7fda4d600640 1 -- 172.21.15.86:0/2080421090 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda44010ac0 con 0x7fda50076920 2024-04-17T07:16:53.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.831+0000 7fda57c00640 1 -- 172.21.15.86:0/2080421090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda501a6770 con 0x7fda50076920 2024-04-17T07:16:53.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.831+0000 7fda4d600640 1 -- 172.21.15.86:0/2080421090 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44017c30 con 0x7fda50076920 2024-04-17T07:16:53.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.831+0000 7fda57c00640 1 -- 172.21.15.86:0/2080421090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda501a6d60 con 0x7fda50076920 2024-04-17T07:16:53.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.835+0000 7fda5a9ba740 1 Processor -- start 2024-04-17T07:16:53.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.835+0000 7fda5a9ba740 1 -- start start 2024-04-17T07:16:53.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.835+0000 7fda4d600640 1 -- 172.21.15.86:0/2080421090 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fda44017070 con 0x7fda50076920 2024-04-17T07:16:53.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.835+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 0x560f76d235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.836+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23b80 0x560f76d28680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.836+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d2b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.836+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f76c8a120 con 0x560f76d23190 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.836+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560f76c30340 con 0x560f76d28cf0 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.836+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560f76c860e0 con 0x560f76d23b80 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.836+0000 7fda4fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d2b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.836+0000 7fda4fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d2b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:35662/0 (socket says 172.21.15.86:35662) 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.836+0000 7fda4fe00640 1 -- 172.21.15.86:0/2887282188 learned_addr learned my addr 172.21.15.86:0/2887282188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.836+0000 7fda55c00640 1 --2- 172.21.15.86:0/2887282188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 0x560f76d235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.836+0000 7fda55200640 1 --2- 172.21.15.86:0/2887282188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23b80 0x560f76d28680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda4fe00640 1 -- 172.21.15.86:0/2887282188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23b80 msgr2=0x560f76d28680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda4fe00640 1 --2- 172.21.15.86:0/2887282188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23b80 0x560f76d28680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda4fe00640 1 -- 172.21.15.86:0/2887282188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 msgr2=0x560f76d235b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda4fe00640 1 --2- 172.21.15.86:0/2887282188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 0x560f76d235b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda4fe00640 1 -- 172.21.15.86:0/2887282188 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f76a667d0 con 0x560f76d28cf0 2024-04-17T07:16:53.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda4fe00640 1 --2- 172.21.15.86:0/2887282188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d2b180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009230 tx=0x7fda3c02c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f792f7af453e3751 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda4d600640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fda380a3c30 0x7fda380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda4d600640 1 -- 172.21.15.86:0/2080421090 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fda440cfda0 con 0x7fda50076920 2024-04-17T07:16:53.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda42000640 1 -- 172.21.15.86:0/2887282188 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c045020 con 0x560f76d28cf0 2024-04-17T07:16:53.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda55c00640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fda380a3c30 0x7fda380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.837+0000 7fda42000640 1 -- 172.21.15.86:0/2887282188 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda3c02cb90 con 0x560f76d28cf0 2024-04-17T07:16:53.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.838+0000 7fda42000640 1 -- 172.21.15.86:0/2887282188 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0048c0 con 0x560f76d28cf0 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.838+0000 7fda5a9ba740 1 -- 172.21.15.86:0/2887282188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 msgr2=0x560f76d2b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.838+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/2887282188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d2b180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009230 tx=0x7fda3c02c8e0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.838+0000 7fda55c00640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fda380a3c30 0x7fda380a60f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x560f76d243e0 tx=0x7fda48009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.839+0000 7fda5a9ba740 1 -- 172.21.15.86:0/2887282188 shutdown_connections 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.839+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/2887282188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d2b180 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.839+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/2887282188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23b80 0x560f76d28680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.839+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/2887282188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 0x560f76d235b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.839+0000 7fda5a9ba740 1 -- 172.21.15.86:0/2887282188 >> 172.21.15.86:0/2887282188 conn(0x560f76d1e700 msgr2=0x560f76d1eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.839+0000 7fda5a9ba740 1 -- 172.21.15.86:0/2887282188 shutdown_connections 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.839+0000 7fda5a9ba740 1 -- 172.21.15.86:0/2887282188 wait complete. 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.839+0000 7fda5a9ba740 1 -- start start 2024-04-17T07:16:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.839+0000 7fda5a9ba740 1 Processor -- start 2024-04-17T07:16:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23190 0x560f76d2d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d34b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f76d2bc40 con 0x560f76d23b80 2024-04-17T07:16:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560f76c43710 con 0x560f76d28cf0 2024-04-17T07:16:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560f76d2bdc0 con 0x560f76d23190 2024-04-17T07:16:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda5a9ba740 10 client.0 _ll_register_callbacks cb 0x560f76c3eaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda55c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d34b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35154/0 (socket says 172.21.15.86:35154) 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda4fe00640 1 -- 172.21.15.86:0/3430907216 learned_addr learned my addr 172.21.15.86:0/3430907216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.840+0000 7fda55200640 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23190 0x560f76d2d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.841+0000 7fda4fe00640 1 -- 172.21.15.86:0/3430907216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23190 msgr2=0x560f76d2d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.841+0000 7fda4fe00640 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23190 0x560f76d2d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.841+0000 7fda4fe00640 1 -- 172.21.15.86:0/3430907216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 msgr2=0x560f76d34b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.841+0000 7fda4fe00640 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d34b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.841+0000 7fda4fe00640 1 -- 172.21.15.86:0/3430907216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f7699c3f0 con 0x560f76d23b80 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.841+0000 7fda4fe00640 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2dbb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fda3c02c170 tx=0x7fda3c02cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22350fa38a57d626 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.841+0000 7fda36a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.841+0000 7fda36a00640 1 -- 172.21.15.86:0/3430907216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c045070 con 0x560f76d23b80 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.842+0000 7fda36a00640 1 -- 172.21.15.86:0/3430907216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fda3c003a00 con 0x560f76d23b80 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.842+0000 7fda5a9ba740 1 -- 172.21.15.86:0/3430907216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda3c02cb90 con 0x560f76d23b80 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.842+0000 7fda36a00640 1 -- 172.21.15.86:0/3430907216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda3c05a050 con 0x560f76d23b80 2024-04-17T07:16:53.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.842+0000 7fda5a9ba740 10 client.16239 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.842+0000 7fda5a9ba740 1 -- 172.21.15.86:0/3430907216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f7695c0f0 con 0x560f76d23b80 2024-04-17T07:16:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.843+0000 7fda36a00640 1 -- 172.21.15.86:0/3430907216 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda3c02cb90 con 0x560f76d23b80 2024-04-17T07:16:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.843+0000 7fda36a00640 1 -- 172.21.15.86:0/3430907216 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fda3c04a450 con 0x560f76d23b80 2024-04-17T07:16:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.843+0000 7fda5a9ba740 10 client.16239 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.843+0000 7fda5a9ba740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.843+0000 7fda5a9ba740 20 client.16239 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.843+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f76d390c0 0x560f76d3b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.843+0000 7fda5a9ba740 4 client.16239 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:16:53.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.843+0000 7fda5a9ba740 1 -- 172.21.15.86:0/3430907216 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f76c97b40 con 0x560f76d390c0 2024-04-17T07:16:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.843+0000 7fda55200640 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f76d390c0 0x560f76d3b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.844+0000 7fda55200640 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f76d390c0 0x560f76d3b4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23774950cb000a72 server_cookie=16c7f58c68743922 in_seq=0 out_seq=0 2024-04-17T07:16:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.844+0000 7fda36a00640 10 client.16239 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:53.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.847+0000 7fda36a00640 1 -- 172.21.15.86:0/3430907216 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f76c97b40 con 0x560f76d390c0 2024-04-17T07:16:53.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.847+0000 7fda36a00640 10 client.16239 handle_command_reply: tid=0 2024-04-17T07:16:53.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.847+0000 7fda5a9ba740 1 client.16239 shutdown 2024-04-17T07:16:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 1 -- 172.21.15.86:0/3430907216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 msgr2=0x560f76d2dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2dbb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fda3c02c170 tx=0x7fda3c02cdc0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 1 -- 172.21.15.86:0/3430907216 shutdown_connections 2024-04-17T07:16:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f76d390c0 0x560f76d3b4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d34b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2dbb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/3430907216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23190 0x560f76d2d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda36a00640 0 client.16239 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 1 -- 172.21.15.86:0/3430907216 >> 172.21.15.86:0/3430907216 conn(0x560f76d1e700 msgr2=0x560f76d1d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 1 -- 172.21.15.86:0/3430907216 shutdown_connections 2024-04-17T07:16:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 1 -- 172.21.15.86:0/3430907216 wait complete. 2024-04-17T07:16:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.848+0000 7fda5a9ba740 20 client.16239 trim_cache size 0 max 16384 2024-04-17T07:16:53.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.871+0000 7fda5a9ba740 1 Processor -- start 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.871+0000 7fda5a9ba740 1 -- start start 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.871+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d23190 0x560f76d2c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.871+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.871+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d28cf0 0x560f76d34b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.871+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b10c0 con 0x560f76d23b80 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.871+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b2c00 con 0x560f76d23190 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.871+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b1790 con 0x560f76d28cf0 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35158/0 (socket says 172.21.15.86:35158) 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda4fe00640 1 -- 172.21.15.86:0/2791042828 learned_addr learned my addr 172.21.15.86:0/2791042828 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:53.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda55200640 1 --2- 172.21.15.86:0/2791042828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d23190 0x560f76d2c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda4fe00640 1 -- 172.21.15.86:0/2791042828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d28cf0 msgr2=0x560f76d34b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda4fe00640 1 --2- 172.21.15.86:0/2791042828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d28cf0 0x560f76d34b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda4fe00640 1 -- 172.21.15.86:0/2791042828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d23190 msgr2=0x560f76d2c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda4fe00640 1 --2- 172.21.15.86:0/2791042828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d23190 0x560f76d2c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda4fe00640 1 -- 172.21.15.86:0/2791042828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f76d23f60 con 0x560f76d23b80 2024-04-17T07:16:53.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda55200640 1 --2- 172.21.15.86:0/2791042828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d23190 0x560f76d2c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:16:53.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.872+0000 7fda4fe00640 1 --2- 172.21.15.86:0/2791042828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2c6a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fda3c04a8f0 tx=0x7fda3c004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1ebde23c0ede01 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.873+0000 7fda41600640 1 -- 172.21.15.86:0/2791042828 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c045040 con 0x560f76d23b80 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.873+0000 7fda41600640 1 -- 172.21.15.86:0/2791042828 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda3c04a450 con 0x560f76d23b80 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.873+0000 7fda41600640 1 -- 172.21.15.86:0/2791042828 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c005030 con 0x560f76d23b80 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.873+0000 7fda5a9ba740 1 -- 172.21.15.86:0/2791042828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 msgr2=0x560f76d2c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.873+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/2791042828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2c6a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fda3c04a8f0 tx=0x7fda3c004390 comp rx=0 tx=0).stop 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.873+0000 7fda5a9ba740 1 -- 172.21.15.86:0/2791042828 shutdown_connections 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.873+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/2791042828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d28cf0 0x560f76d34b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.873+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/2791042828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23b80 0x560f76d2c6a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.873+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/2791042828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d23190 0x560f76d2c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.873+0000 7fda5a9ba740 1 -- 172.21.15.86:0/2791042828 >> 172.21.15.86:0/2791042828 conn(0x560f76d1e700 msgr2=0x560f76d2d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.874+0000 7fda5a9ba740 1 -- 172.21.15.86:0/2791042828 shutdown_connections 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.874+0000 7fda5a9ba740 1 -- 172.21.15.86:0/2791042828 wait complete. 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.874+0000 7fda5a9ba740 1 -- start start 2024-04-17T07:16:53.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.874+0000 7fda5a9ba740 1 Processor -- start 2024-04-17T07:16:53.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 0x560f76d31a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23b80 0x560f76d31f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda5a9ba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d9d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b2530 con 0x560f76d23190 2024-04-17T07:16:53.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda501b1e60 con 0x560f76d28cf0 2024-04-17T07:16:53.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda5a9ba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda501af090 con 0x560f76d23b80 2024-04-17T07:16:53.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda5a9ba740 10 client.0 _ll_register_callbacks cb 0x7fda480095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:16:53.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda55200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 0x560f76d31a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda55200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 0x560f76d31a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35164/0 (socket says 172.21.15.86:35164) 2024-04-17T07:16:53.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda55200640 1 -- 172.21.15.86:0/1621400596 learned_addr learned my addr 172.21.15.86:0/1621400596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda55c00640 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d9d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda55200640 1 -- 172.21.15.86:0/1621400596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23b80 msgr2=0x560f76d31f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda55200640 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23b80 0x560f76d31f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda55200640 1 -- 172.21.15.86:0/1621400596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 msgr2=0x560f76d9d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda55200640 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d9d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda55200640 1 -- 172.21.15.86:0/1621400596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f76d9db40 con 0x560f76d23190 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.875+0000 7fda55c00640 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d9d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.876+0000 7fda55200640 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 0x560f76d31a40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fda44098160 tx=0x7fda440d35a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334dacea93f0b335 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.876+0000 7fda36a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.876+0000 7fda36a00640 1 -- 172.21.15.86:0/1621400596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4401e040 con 0x560f76d23190 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.876+0000 7fda36a00640 1 -- 172.21.15.86:0/1621400596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fda440cd630 con 0x560f76d23190 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.876+0000 7fda5a9ba740 1 -- 172.21.15.86:0/1621400596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda3c04a450 con 0x560f76d23190 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.876+0000 7fda36a00640 1 -- 172.21.15.86:0/1621400596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda440a00b0 con 0x560f76d23190 2024-04-17T07:16:53.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.876+0000 7fda5a9ba740 10 client.16245 fetch_fsmap learned FSMap version 28 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.876+0000 7fda5a9ba740 1 -- 172.21.15.86:0/1621400596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f76d9edd0 con 0x560f76d23190 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.876+0000 7fda36a00640 1 -- 172.21.15.86:0/1621400596 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda3c04a450 con 0x560f76d23190 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.877+0000 7fda36a00640 1 -- 172.21.15.86:0/1621400596 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fda44010c70 con 0x560f76d23190 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.877+0000 7fda5a9ba740 10 client.16245 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.877+0000 7fda5a9ba740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.877+0000 7fda5a9ba740 20 client.16245 populate_metadata read hostname 'smithi086' 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.877+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f76d9f730 0x560f76da1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.877+0000 7fda5a9ba740 4 client.16245 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.877+0000 7fda5a9ba740 1 -- 172.21.15.86:0/1621400596 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f76c919b0 con 0x560f76d9f730 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.877+0000 7fda4fe00640 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f76d9f730 0x560f76da1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.878+0000 7fda4fe00640 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f76d9f730 0x560f76da1b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5246f66618d22a9 server_cookie=617b895064fb4d5c in_seq=0 out_seq=0 2024-04-17T07:16:53.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.878+0000 7fda36a00640 10 client.16245 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.878+0000 7fda36a00640 1 -- 172.21.15.86:0/1621400596 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560f76c919b0 con 0x560f76d9f730 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.878+0000 7fda36a00640 10 client.16245 handle_command_reply: tid=0 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.878+0000 7fda5a9ba740 1 client.16245 shutdown 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 1 -- 172.21.15.86:0/1621400596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 msgr2=0x560f76d31a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 0x560f76d31a40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fda44098160 tx=0x7fda440d35a0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 1 -- 172.21.15.86:0/1621400596 shutdown_connections 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f76d9f730 0x560f76da1b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f76d28cf0 0x560f76d9d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f76d23b80 0x560f76d31f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 1 --2- 172.21.15.86:0/1621400596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f76d23190 0x560f76d31a40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 1 -- 172.21.15.86:0/1621400596 >> 172.21.15.86:0/1621400596 conn(0x560f76d1e700 msgr2=0x560f76d2d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda36a00640 0 client.16245 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:16:53.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 1 -- 172.21.15.86:0/1621400596 shutdown_connections 2024-04-17T07:16:53.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 1 -- 172.21.15.86:0/1621400596 wait complete. 2024-04-17T07:16:53.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.879+0000 7fda5a9ba740 20 client.16245 trim_cache size 0 max 16384 2024-04-17T07:16:53.883 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:16:53.883 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:16:53.883 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:16:53.883 INFO:teuthology.orchestra.run.smithi086.stdout: "a808bde0-bbb5-461c-afbd-de5af4ad66fb": { 2024-04-17T07:16:53.883 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:16:53.883 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "a808bde0-bbb5-461c-afbd-de5af4ad66fb", 2024-04-17T07:16:53.883 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:16:53.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:16:53.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:16:53.884 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:16:53.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 -- 172.21.15.86:0/2080421090 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fda380a3c30 msgr2=0x7fda380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fda380a3c30 0x7fda380a60f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x560f76d243e0 tx=0x7fda48009740 comp rx=0 tx=0).stop 2024-04-17T07:16:53.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 -- 172.21.15.86:0/2080421090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda50076920 msgr2=0x7fda5019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:16:53.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda50076920 0x7fda5019e860 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fda4400b4a0 tx=0x7fda4400b960 comp rx=0 tx=0).stop 2024-04-17T07:16:53.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 -- 172.21.15.86:0/2080421090 shutdown_connections 2024-04-17T07:16:53.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fda380a3c30 0x7fda380a60f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda50078500 0x7fda501a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda50077b60 0x7fda5019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 --2- 172.21.15.86:0/2080421090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda50076920 0x7fda5019e860 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:16:53.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 -- 172.21.15.86:0/2080421090 >> 172.21.15.86:0/2080421090 conn(0x7fda500ff710 msgr2=0x7fda50100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:16:53.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 -- 172.21.15.86:0/2080421090 shutdown_connections 2024-04-17T07:16:53.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:16:53.880+0000 7fda57c00640 1 -- 172.21.15.86:0/2080421090 wait complete. 2024-04-17T07:16:53.899 INFO:tasks.cephfs.filesystem:scrub status for tag:a808bde0-bbb5-461c-afbd-de5af4ad66fb - {'path': '/', 'tag': 'a808bde0-bbb5-461c-afbd-de5af4ad66fb', 'options': 'recursive,force'} 2024-04-17T07:16:54.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:54 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:16:54.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:54 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:54.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:54 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:16:54.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:54 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:54.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:54 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:16:54.625 INFO:teuthology.orchestra.run.smithi086.stderr:1100000 of 3200000 tuples (34%) done (elapsed 44.17 s, remaining 84.33 s) 2024-04-17T07:16:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:54 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:16:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:54 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:54 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:16:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:54 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:54 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:16:54.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:54 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:16:54.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:54 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:54.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:54 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:16:54.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:54 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:16:54.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:54 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:16:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:55 smithi086 ceph-mon[31733]: pgmap v517: 129 pgs: 129 active+clean; 283 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 17 op/s 2024-04-17T07:16:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:55 smithi098 ceph-mon[35418]: pgmap v517: 129 pgs: 129 active+clean; 283 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 17 op/s 2024-04-17T07:16:55.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:55 smithi103 ceph-mon[35495]: pgmap v517: 129 pgs: 129 active+clean; 283 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 17 op/s 2024-04-17T07:16:57.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:57 smithi086 ceph-mon[31733]: pgmap v518: 129 pgs: 129 active+clean; 299 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:16:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:57 smithi098 ceph-mon[35418]: pgmap v518: 129 pgs: 129 active+clean; 299 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:16:57.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:57 smithi103 ceph-mon[35495]: pgmap v518: 129 pgs: 129 active+clean; 299 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:16:58.683 INFO:teuthology.orchestra.run.smithi086.stderr:1200000 of 3200000 tuples (37%) done (elapsed 48.23 s, remaining 80.38 s) 2024-04-17T07:16:59.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:16:59 smithi086 ceph-mon[31733]: pgmap v519: 129 pgs: 129 active+clean; 307 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-04-17T07:16:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:16:59 smithi098 ceph-mon[35418]: pgmap v519: 129 pgs: 129 active+clean; 307 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-04-17T07:16:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:16:59 smithi103 ceph-mon[35495]: pgmap v519: 129 pgs: 129 active+clean; 307 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-04-17T07:17:01.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:01 smithi086 ceph-mon[31733]: pgmap v520: 129 pgs: 129 active+clean; 311 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:17:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:01 smithi098 ceph-mon[35418]: pgmap v520: 129 pgs: 129 active+clean; 311 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:17:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:01 smithi103 ceph-mon[35495]: pgmap v520: 129 pgs: 129 active+clean; 311 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:17:03.256 INFO:teuthology.orchestra.run.smithi086.stderr:1300000 of 3200000 tuples (40%) done (elapsed 52.80 s, remaining 77.17 s) 2024-04-17T07:17:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:03 smithi086 ceph-mon[31733]: pgmap v521: 129 pgs: 129 active+clean; 331 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:17:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:03 smithi098 ceph-mon[35418]: pgmap v521: 129 pgs: 129 active+clean; 331 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:17:03.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:03 smithi103 ceph-mon[35495]: pgmap v521: 129 pgs: 129 active+clean; 331 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:17:04.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:04 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:17:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:04 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:17:04.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:04 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:17:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:17:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:17:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:17:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:05 smithi086 ceph-mon[31733]: pgmap v522: 129 pgs: 129 active+clean; 339 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:17:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:17:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:17:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:17:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:05 smithi098 ceph-mon[35418]: pgmap v522: 129 pgs: 129 active+clean; 339 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:17:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:17:05.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:17:05.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:17:05.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:05 smithi103 ceph-mon[35495]: pgmap v522: 129 pgs: 129 active+clean; 339 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:17:07.527 INFO:teuthology.orchestra.run.smithi086.stderr:1400000 of 3200000 tuples (43%) done (elapsed 57.07 s, remaining 73.38 s) 2024-04-17T07:17:07.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:07 smithi086 ceph-mon[31733]: pgmap v523: 129 pgs: 129 active+clean; 355 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:17:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:07 smithi098 ceph-mon[35418]: pgmap v523: 129 pgs: 129 active+clean; 355 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:17:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:07 smithi103 ceph-mon[35495]: pgmap v523: 129 pgs: 129 active+clean; 355 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:17:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:09 smithi086 ceph-mon[31733]: pgmap v524: 129 pgs: 129 active+clean; 367 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:17:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:09 smithi098 ceph-mon[35418]: pgmap v524: 129 pgs: 129 active+clean; 367 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:17:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:09 smithi103 ceph-mon[35495]: pgmap v524: 129 pgs: 129 active+clean; 367 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:17:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:11 smithi086 ceph-mon[31733]: pgmap v525: 129 pgs: 129 active+clean; 368 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.8 MiB/s wr, 18 op/s 2024-04-17T07:17:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:1500000 of 3200000 tuples (46%) done (elapsed 61.18 s, remaining 69.34 s) 2024-04-17T07:17:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:11 smithi098 ceph-mon[35418]: pgmap v525: 129 pgs: 129 active+clean; 368 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.8 MiB/s wr, 18 op/s 2024-04-17T07:17:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:11 smithi103 ceph-mon[35495]: pgmap v525: 129 pgs: 129 active+clean; 368 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.8 MiB/s wr, 18 op/s 2024-04-17T07:17:13.579 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:13 smithi103 ceph-mon[35495]: pgmap v526: 129 pgs: 129 active+clean; 383 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-17T07:17:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:13 smithi086 ceph-mon[31733]: pgmap v526: 129 pgs: 129 active+clean; 383 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-17T07:17:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:13 smithi098 ceph-mon[35418]: pgmap v526: 129 pgs: 129 active+clean; 383 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-17T07:17:15.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:15 smithi086 ceph-mon[31733]: pgmap v527: 129 pgs: 129 active+clean; 383 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 17 op/s 2024-04-17T07:17:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:15 smithi098 ceph-mon[35418]: pgmap v527: 129 pgs: 129 active+clean; 383 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 17 op/s 2024-04-17T07:17:15.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:15 smithi103 ceph-mon[35495]: pgmap v527: 129 pgs: 129 active+clean; 383 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 17 op/s 2024-04-17T07:17:16.611 INFO:teuthology.orchestra.run.smithi086.stderr:1600000 of 3200000 tuples (50%) done (elapsed 66.16 s, remaining 66.16 s) 2024-04-17T07:17:17.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:17 smithi086 ceph-mon[31733]: pgmap v528: 129 pgs: 129 active+clean; 403 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 22 op/s 2024-04-17T07:17:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:17 smithi098 ceph-mon[35418]: pgmap v528: 129 pgs: 129 active+clean; 403 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 22 op/s 2024-04-17T07:17:17.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:17 smithi103 ceph-mon[35495]: pgmap v528: 129 pgs: 129 active+clean; 403 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 22 op/s 2024-04-17T07:17:19.555 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:19 smithi098 ceph-mon[35418]: pgmap v529: 129 pgs: 129 active+clean; 416 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 22 op/s 2024-04-17T07:17:19.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:19 smithi086 ceph-mon[31733]: pgmap v529: 129 pgs: 129 active+clean; 416 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 22 op/s 2024-04-17T07:17:19.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:19 smithi103 ceph-mon[35495]: pgmap v529: 129 pgs: 129 active+clean; 416 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 22 op/s 2024-04-17T07:17:20.512 INFO:teuthology.orchestra.run.smithi086.stderr:1700000 of 3200000 tuples (53%) done (elapsed 70.06 s, remaining 61.82 s) 2024-04-17T07:17:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:21 smithi086 ceph-mon[31733]: pgmap v530: 129 pgs: 129 active+clean; 416 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 20 op/s 2024-04-17T07:17:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:21 smithi098 ceph-mon[35418]: pgmap v530: 129 pgs: 129 active+clean; 416 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 20 op/s 2024-04-17T07:17:21.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:21 smithi103 ceph-mon[35495]: pgmap v530: 129 pgs: 129 active+clean; 416 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 20 op/s 2024-04-17T07:17:23.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:23 smithi086 ceph-mon[31733]: pgmap v531: 129 pgs: 129 active+clean; 432 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 23 op/s 2024-04-17T07:17:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:23 smithi098 ceph-mon[35418]: pgmap v531: 129 pgs: 129 active+clean; 432 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 23 op/s 2024-04-17T07:17:23.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:23 smithi103 ceph-mon[35495]: pgmap v531: 129 pgs: 129 active+clean; 432 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 23 op/s 2024-04-17T07:17:23.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:17:23.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:17:24.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.151+0000 7fc83c200640 1 -- 172.21.15.86:0/1695816975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8341053b0 msgr2=0x7fc834105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.151+0000 7fc83c200640 1 --2- 172.21.15.86:0/1695816975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8341053b0 0x7fc834105830 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc820009900 tx=0x7fc82002bc30 comp rx=0 tx=0).stop 2024-04-17T07:17:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.151+0000 7fc83c200640 1 -- 172.21.15.86:0/1695816975 shutdown_connections 2024-04-17T07:17:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.151+0000 7fc83c200640 1 --2- 172.21.15.86:0/1695816975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834105d70 0x7fc83410c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.151+0000 7fc83c200640 1 --2- 172.21.15.86:0/1695816975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8341053b0 0x7fc834105830 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.151+0000 7fc83c200640 1 --2- 172.21.15.86:0/1695816975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8341041c0 0x7fc8341045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.151+0000 7fc83c200640 1 -- 172.21.15.86:0/1695816975 >> 172.21.15.86:0/1695816975 conn(0x7fc8340ff710 msgr2=0x7fc834101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.152+0000 7fc83c200640 1 -- 172.21.15.86:0/1695816975 shutdown_connections 2024-04-17T07:17:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.152+0000 7fc83c200640 1 -- 172.21.15.86:0/1695816975 wait complete. 2024-04-17T07:17:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.153+0000 7fc83c200640 1 Processor -- start 2024-04-17T07:17:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.153+0000 7fc83c200640 1 -- start start 2024-04-17T07:17:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.153+0000 7fc83c200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8341041c0 0x7fc83419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.153+0000 7fc83c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8341053b0 0x7fc83419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.153+0000 7fc83c200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834105d70 0x7fc8341a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.153+0000 7fc83c200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc83410fd50 con 0x7fc8341053b0 2024-04-17T07:17:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.153+0000 7fc83c200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc83410fbd0 con 0x7fc834105d70 2024-04-17T07:17:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.153+0000 7fc83c200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc83410fed0 con 0x7fc8341041c0 2024-04-17T07:17:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.154+0000 7fc838e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8341053b0 0x7fc83419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.154+0000 7fc838e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8341053b0 0x7fc83419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38164/0 (socket says 172.21.15.86:38164) 2024-04-17T07:17:24.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.154+0000 7fc838e00640 1 -- 172.21.15.86:0/1099223096 learned_addr learned my addr 172.21.15.86:0/1099223096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.154+0000 7fc839800640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8341041c0 0x7fc83419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.154+0000 7fc83a200640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834105d70 0x7fc8341a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.154+0000 7fc838e00640 1 -- 172.21.15.86:0/1099223096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8341041c0 msgr2=0x7fc83419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.154+0000 7fc838e00640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8341041c0 0x7fc83419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.154+0000 7fc838e00640 1 -- 172.21.15.86:0/1099223096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834105d70 msgr2=0x7fc8341a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.154+0000 7fc838e00640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834105d70 0x7fc8341a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.154+0000 7fc838e00640 1 -- 172.21.15.86:0/1099223096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8341a1fe0 con 0x7fc8341053b0 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.155+0000 7fc838e00640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8341053b0 0x7fc83419a900 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc820000c00 tx=0x7fc8200040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.155+0000 7fc832000640 1 -- 172.21.15.86:0/1099223096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820043020 con 0x7fc8341053b0 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.155+0000 7fc832000640 1 -- 172.21.15.86:0/1099223096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8200044c0 con 0x7fc8341053b0 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.155+0000 7fc83c200640 1 -- 172.21.15.86:0/1099223096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8341a2270 con 0x7fc8341053b0 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.155+0000 7fc832000640 1 -- 172.21.15.86:0/1099223096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820003c30 con 0x7fc8341053b0 2024-04-17T07:17:24.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.155+0000 7fc83c200640 1 -- 172.21.15.86:0/1099223096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8341a2810 con 0x7fc8341053b0 2024-04-17T07:17:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.160+0000 7fc83ebfd740 1 Processor -- start 2024-04-17T07:17:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.160+0000 7fc83ebfd740 1 -- start start 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.160+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2190 0x5578a49a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.160+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 0x5578a49a7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.160+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a7cf0 0x5578a49aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.160+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5578a4909120 con 0x5578a49a2190 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.160+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5578a48af340 con 0x5578a49a7cf0 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.160+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5578a49050e0 con 0x5578a49a2b80 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.160+0000 7fc83a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2190 0x5578a49a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.161+0000 7fc832000640 1 -- 172.21.15.86:0/1099223096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc820046020 con 0x7fc8341053b0 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.161+0000 7fc83a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2190 0x5578a49a25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38176/0 (socket says 172.21.15.86:38176) 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.161+0000 7fc83a200640 1 -- 172.21.15.86:0/3627139396 learned_addr learned my addr 172.21.15.86:0/3627139396 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.161+0000 7fc839800640 1 --2- 172.21.15.86:0/3627139396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 0x5578a49a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.161+0000 7fc83a200640 1 -- 172.21.15.86:0/3627139396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 msgr2=0x5578a49a7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.161+0000 7fc83a200640 1 --2- 172.21.15.86:0/3627139396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 0x5578a49a7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.161+0000 7fc83a200640 1 -- 172.21.15.86:0/3627139396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a7cf0 msgr2=0x5578a49aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.161+0000 7fc83a200640 1 --2- 172.21.15.86:0/3627139396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a7cf0 0x5578a49aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.161+0000 7fc83a200640 1 -- 172.21.15.86:0/3627139396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a46e57d0 con 0x5578a49a2190 2024-04-17T07:17:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.162+0000 7fc83a200640 1 --2- 172.21.15.86:0/3627139396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2190 0x5578a49a25b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009fc0 tx=0x7fc82c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e01f1f84b86f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.162+0000 7fc826a00640 1 -- 172.21.15.86:0/3627139396 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c003b70 con 0x5578a49a2190 2024-04-17T07:17:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.162+0000 7fc826a00640 1 -- 172.21.15.86:0/3627139396 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc82c003d10 con 0x5578a49a2190 2024-04-17T07:17:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.162+0000 7fc826a00640 1 -- 172.21.15.86:0/3627139396 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0119f0 con 0x5578a49a2190 2024-04-17T07:17:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.162+0000 7fc832000640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc81c0a3c30 0x7fc81c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc832000640 1 -- 172.21.15.86:0/1099223096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc8200f2a50 con 0x7fc8341053b0 2024-04-17T07:17:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3627139396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2190 msgr2=0x5578a49a25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/3627139396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2190 0x5578a49a25b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009fc0 tx=0x7fc82c010860 comp rx=0 tx=0).stop 2024-04-17T07:17:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc839800640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc81c0a3c30 0x7fc81c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3627139396 shutdown_connections 2024-04-17T07:17:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/3627139396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a7cf0 0x5578a49aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/3627139396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 0x5578a49a7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/3627139396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2190 0x5578a49a25b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3627139396 >> 172.21.15.86:0/3627139396 conn(0x5578a499d700 msgr2=0x5578a499db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3627139396 shutdown_connections 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.163+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3627139396 wait complete. 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.164+0000 7fc83ebfd740 1 -- start start 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.164+0000 7fc839800640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc81c0a3c30 0x7fc81c0a60f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc8280095f0 tx=0x7fc828005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.164+0000 7fc83ebfd740 1 Processor -- start 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 0x5578a49ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 0x5578a49acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a7cf0 0x5578a49b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5578a49aaca0 con 0x5578a49a7cf0 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5578a48c2710 con 0x5578a49a2190 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5578a49aae20 con 0x5578a49a2b80 2024-04-17T07:17:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc83ebfd740 10 client.0 _ll_register_callbacks cb 0x5578a48bdaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc83a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a7cf0 0x5578a49b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc83a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a7cf0 0x5578a49b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38186/0 (socket says 172.21.15.86:38186) 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc83a200640 1 -- 172.21.15.86:0/4018356497 learned_addr learned my addr 172.21.15.86:0/4018356497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc839800640 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 0x5578a49ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.165+0000 7fc838e00640 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 0x5578a49acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc83a200640 1 -- 172.21.15.86:0/4018356497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 msgr2=0x5578a49acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc83a200640 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 0x5578a49acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc83a200640 1 -- 172.21.15.86:0/4018356497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 msgr2=0x5578a49ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc83a200640 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 0x5578a49ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc83a200640 1 -- 172.21.15.86:0/4018356497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a461b3f0 con 0x5578a49a7cf0 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc839800640 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 0x5578a49ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc838e00640 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 0x5578a49acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc83a200640 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a7cf0 0x5578a49b3cb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc82c011560 tx=0x7fc82c004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09ee3cddc840c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc81b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc81b400640 1 -- 172.21.15.86:0/4018356497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c011a00 con 0x5578a49a7cf0 2024-04-17T07:17:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc81b400640 1 -- 172.21.15.86:0/4018356497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc82c011ba0 con 0x5578a49a7cf0 2024-04-17T07:17:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.166+0000 7fc83ebfd740 1 -- 172.21.15.86:0/4018356497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82c003d10 con 0x5578a49a7cf0 2024-04-17T07:17:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.167+0000 7fc81b400640 1 -- 172.21.15.86:0/4018356497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc82c011dc0 con 0x5578a49a7cf0 2024-04-17T07:17:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.167+0000 7fc83ebfd740 10 client.16257 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.167+0000 7fc83ebfd740 1 -- 172.21.15.86:0/4018356497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578a45db0f0 con 0x5578a49a7cf0 2024-04-17T07:17:24.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.167+0000 7fc81b400640 1 -- 172.21.15.86:0/4018356497 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc82c003d10 con 0x5578a49a7cf0 2024-04-17T07:17:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.167+0000 7fc81b400640 1 -- 172.21.15.86:0/4018356497 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc82c005340 con 0x5578a49a7cf0 2024-04-17T07:17:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.167+0000 7fc83ebfd740 10 client.16257 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.167+0000 7fc83ebfd740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:17:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.168+0000 7fc83ebfd740 20 client.16257 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.168+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5578a49b8330 0x5578a49ba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.168+0000 7fc83ebfd740 4 client.16257 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:24.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.168+0000 7fc83ebfd740 1 -- 172.21.15.86:0/4018356497 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a4916b40 con 0x5578a49b8330 2024-04-17T07:17:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.168+0000 7fc839800640 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5578a49b8330 0x5578a49ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.169+0000 7fc839800640 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5578a49b8330 0x5578a49ba720 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ecf8b50dc497c24 server_cookie=59cde2e650a0ab4 in_seq=0 out_seq=0 2024-04-17T07:17:24.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.169+0000 7fc81b400640 10 client.16257 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:24.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc81b400640 1 -- 172.21.15.86:0/4018356497 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578a4916b40 con 0x5578a49b8330 2024-04-17T07:17:24.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc81b400640 10 client.16257 handle_command_reply: tid=0 2024-04-17T07:17:24.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc83ebfd740 1 client.16257 shutdown 2024-04-17T07:17:24.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc83ebfd740 1 -- 172.21.15.86:0/4018356497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a7cf0 msgr2=0x5578a49b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a7cf0 0x5578a49b3cb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc82c011560 tx=0x7fc82c004c70 comp rx=0 tx=0).stop 2024-04-17T07:17:24.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc83ebfd740 1 -- 172.21.15.86:0/4018356497 shutdown_connections 2024-04-17T07:17:24.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5578a49b8330 0x5578a49ba720 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a7cf0 0x5578a49b3cb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2b80 0x5578a49acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc81b400640 0 client.16257 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:24.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/4018356497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 0x5578a49ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.171+0000 7fc83ebfd740 1 -- 172.21.15.86:0/4018356497 >> 172.21.15.86:0/4018356497 conn(0x5578a499d700 msgr2=0x5578a499c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.172+0000 7fc83ebfd740 1 -- 172.21.15.86:0/4018356497 shutdown_connections 2024-04-17T07:17:24.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.172+0000 7fc83ebfd740 1 -- 172.21.15.86:0/4018356497 wait complete. 2024-04-17T07:17:24.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.172+0000 7fc83ebfd740 20 client.16257 trim_cache size 0 max 16384 2024-04-17T07:17:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.196+0000 7fc83ebfd740 1 Processor -- start 2024-04-17T07:17:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.196+0000 7fc83ebfd740 1 -- start start 2024-04-17T07:17:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2190 0x5578a499c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a7cf0 0x5578a49befe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341acd30 con 0x5578a49a2b80 2024-04-17T07:17:24.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341ae870 con 0x5578a49a7cf0 2024-04-17T07:17:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341ad400 con 0x5578a49a2190 2024-04-17T07:17:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc838e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc838e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38194/0 (socket says 172.21.15.86:38194) 2024-04-17T07:17:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc838e00640 1 -- 172.21.15.86:0/527486293 learned_addr learned my addr 172.21.15.86:0/527486293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc839800640 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2190 0x5578a499c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.197+0000 7fc83a200640 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a7cf0 0x5578a49befe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc838e00640 1 -- 172.21.15.86:0/527486293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2190 msgr2=0x5578a499c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc838e00640 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2190 0x5578a499c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc838e00640 1 -- 172.21.15.86:0/527486293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a7cf0 msgr2=0x5578a49befe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc838e00640 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a7cf0 0x5578a49befe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc838e00640 1 -- 172.21.15.86:0/527486293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a499e470 con 0x5578a49a2b80 2024-04-17T07:17:24.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc839800640 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2190 0x5578a499c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:24.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc83a200640 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a7cf0 0x5578a49befe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:24.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc838e00640 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b3c70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc8200bb380 tx=0x7fc8200df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d82cd7276f367b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc826000640 1 -- 172.21.15.86:0/527486293 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820005600 con 0x5578a49a2b80 2024-04-17T07:17:24.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc826000640 1 -- 172.21.15.86:0/527486293 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc82003c350 con 0x5578a49a2b80 2024-04-17T07:17:24.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.198+0000 7fc826000640 1 -- 172.21.15.86:0/527486293 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200f02e0 con 0x5578a49a2b80 2024-04-17T07:17:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.199+0000 7fc83ebfd740 1 -- 172.21.15.86:0/527486293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 msgr2=0x5578a49b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.199+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b3c70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc8200bb380 tx=0x7fc8200df040 comp rx=0 tx=0).stop 2024-04-17T07:17:24.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.199+0000 7fc83ebfd740 1 -- 172.21.15.86:0/527486293 shutdown_connections 2024-04-17T07:17:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.199+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a7cf0 0x5578a49befe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.199+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b3c70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.199+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/527486293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a2190 0x5578a499c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.199+0000 7fc83ebfd740 1 -- 172.21.15.86:0/527486293 >> 172.21.15.86:0/527486293 conn(0x5578a499d700 msgr2=0x5578a499c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.199+0000 7fc83ebfd740 1 -- 172.21.15.86:0/527486293 shutdown_connections 2024-04-17T07:17:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.199+0000 7fc83ebfd740 1 -- 172.21.15.86:0/527486293 wait complete. 2024-04-17T07:17:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.200+0000 7fc83ebfd740 1 -- start start 2024-04-17T07:17:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.200+0000 7fc83ebfd740 1 Processor -- start 2024-04-17T07:17:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.200+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 0x5578a49b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.200+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.200+0000 7fc83ebfd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a7cf0 0x5578a49b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.200+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341ae1a0 con 0x5578a49a2b80 2024-04-17T07:17:24.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.200+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341adad0 con 0x5578a49a2190 2024-04-17T07:17:24.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.200+0000 7fc83ebfd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341aad00 con 0x5578a49a7cf0 2024-04-17T07:17:24.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc83ebfd740 10 client.0 _ll_register_callbacks cb 0x7fc82c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:24.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc838e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc838e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38200/0 (socket says 172.21.15.86:38200) 2024-04-17T07:17:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc838e00640 1 -- 172.21.15.86:0/3382889768 learned_addr learned my addr 172.21.15.86:0/3382889768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc839800640 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 0x5578a49b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc838e00640 1 -- 172.21.15.86:0/3382889768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a7cf0 msgr2=0x5578a49b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc838e00640 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a7cf0 0x5578a49b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc838e00640 1 -- 172.21.15.86:0/3382889768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 msgr2=0x5578a49b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc838e00640 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 0x5578a49b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc838e00640 1 -- 172.21.15.86:0/3382889768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a49b31c0 con 0x5578a49a2b80 2024-04-17T07:17:24.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.201+0000 7fc838e00640 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b1bc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc82002c6a0 tx=0x7fc820009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7cbb6dbbf83717 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.202+0000 7fc81b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:24.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.202+0000 7fc81b400640 1 -- 172.21.15.86:0/3382889768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200c2d60 con 0x5578a49a2b80 2024-04-17T07:17:24.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.202+0000 7fc81b400640 1 -- 172.21.15.86:0/3382889768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc8200f02e0 con 0x5578a49a2b80 2024-04-17T07:17:24.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.202+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3382889768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82003c350 con 0x5578a49a2b80 2024-04-17T07:17:24.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.202+0000 7fc81b400640 1 -- 172.21.15.86:0/3382889768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8200bf0b0 con 0x5578a49a2b80 2024-04-17T07:17:24.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.202+0000 7fc83ebfd740 10 client.16263 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:24.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.202+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3382889768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578a4a1eac0 con 0x5578a49a2b80 2024-04-17T07:17:24.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.202+0000 7fc81b400640 1 -- 172.21.15.86:0/3382889768 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc82003c350 con 0x5578a49a2b80 2024-04-17T07:17:24.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.202+0000 7fc81b400640 1 -- 172.21.15.86:0/3382889768 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc8200bfce0 con 0x5578a49a2b80 2024-04-17T07:17:24.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.203+0000 7fc83ebfd740 10 client.16263 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.203+0000 7fc83ebfd740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:17:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.203+0000 7fc83ebfd740 20 client.16263 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.203+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5578a4a1f4a0 0x5578a4a218c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.203+0000 7fc83ebfd740 4 client.16263 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:17:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.203+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3382889768 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578a49109b0 con 0x5578a4a1f4a0 2024-04-17T07:17:24.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.203+0000 7fc839800640 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5578a4a1f4a0 0x5578a4a218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.204+0000 7fc839800640 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5578a4a1f4a0 0x5578a4a218c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=369bfa891a4b41ea server_cookie=e060153786a540d5 in_seq=0 out_seq=0 2024-04-17T07:17:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.204+0000 7fc81b400640 10 client.16263 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc81b400640 1 -- 172.21.15.86:0/3382889768 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5578a49109b0 con 0x5578a4a1f4a0 2024-04-17T07:17:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc81b400640 10 client.16263 handle_command_reply: tid=0 2024-04-17T07:17:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 client.16263 shutdown 2024-04-17T07:17:24.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3382889768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 msgr2=0x5578a49b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b1bc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc82002c6a0 tx=0x7fc820009260 comp rx=0 tx=0).stop 2024-04-17T07:17:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3382889768 shutdown_connections 2024-04-17T07:17:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5578a4a1f4a0 0x5578a4a218c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5578a49a7cf0 0x5578a49b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5578a49a2b80 0x5578a49b1bc0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 --2- 172.21.15.86:0/3382889768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5578a49a2190 0x5578a49b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3382889768 >> 172.21.15.86:0/3382889768 conn(0x5578a499d700 msgr2=0x5578a499c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc81b400640 0 client.16263 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3382889768 shutdown_connections 2024-04-17T07:17:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 1 -- 172.21.15.86:0/3382889768 wait complete. 2024-04-17T07:17:24.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.205+0000 7fc83ebfd740 20 client.16263 trim_cache size 0 max 16384 2024-04-17T07:17:24.220 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:17:24.220 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:17:24.220 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:17:24.220 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:17:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 -- 172.21.15.86:0/1099223096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc81c0a3c30 msgr2=0x7fc81c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc81c0a3c30 0x7fc81c0a60f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc8280095f0 tx=0x7fc828005f90 comp rx=0 tx=0).stop 2024-04-17T07:17:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 -- 172.21.15.86:0/1099223096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8341053b0 msgr2=0x7fc83419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8341053b0 0x7fc83419a900 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc820000c00 tx=0x7fc8200040c0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 -- 172.21.15.86:0/1099223096 shutdown_connections 2024-04-17T07:17:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc81c0a3c30 0x7fc81c0a60f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834105d70 0x7fc8341a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8341053b0 0x7fc83419a900 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 --2- 172.21.15.86:0/1099223096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8341041c0 0x7fc83419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 -- 172.21.15.86:0/1099223096 >> 172.21.15.86:0/1099223096 conn(0x7fc8340ff710 msgr2=0x7fc8341086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 -- 172.21.15.86:0/1099223096 shutdown_connections 2024-04-17T07:17:24.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.206+0000 7fc83c200640 1 -- 172.21.15.86:0/1099223096 wait complete. 2024-04-17T07:17:24.225 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:17:24.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:17:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.444+0000 7fb731200640 1 -- 172.21.15.86:0/678057767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c109cc0 msgr2=0x7fb72c10c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.444+0000 7fb731200640 1 --2- 172.21.15.86:0/678057767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c109cc0 0x7fb72c10c150 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb724009ca0 tx=0x7fb72402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.444+0000 7fb731200640 1 -- 172.21.15.86:0/678057767 shutdown_connections 2024-04-17T07:17:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.444+0000 7fb731200640 1 --2- 172.21.15.86:0/678057767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c109cc0 0x7fb72c10c150 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.444+0000 7fb731200640 1 --2- 172.21.15.86:0/678057767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c0749c0 0x7fb72c074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.444+0000 7fb731200640 1 --2- 172.21.15.86:0/678057767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c075fa0 0x7fb72c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.444+0000 7fb731200640 1 -- 172.21.15.86:0/678057767 >> 172.21.15.86:0/678057767 conn(0x7fb72c0fd370 msgr2=0x7fb72c0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.444+0000 7fb731200640 1 -- 172.21.15.86:0/678057767 shutdown_connections 2024-04-17T07:17:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.444+0000 7fb731200640 1 -- 172.21.15.86:0/678057767 wait complete. 2024-04-17T07:17:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.445+0000 7fb731200640 1 Processor -- start 2024-04-17T07:17:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.445+0000 7fb731200640 1 -- start start 2024-04-17T07:17:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb731200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0749c0 0x7fb72c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb731200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c075fa0 0x7fb72c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb731200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c109cc0 0x7fb72c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb731200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c10faf0 con 0x7fb72c0749c0 2024-04-17T07:17:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb731200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c10f970 con 0x7fb72c075fa0 2024-04-17T07:17:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb731200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c10fc70 con 0x7fb72c109cc0 2024-04-17T07:17:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb72b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0749c0 0x7fb72c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb72b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0749c0 0x7fb72c19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38218/0 (socket says 172.21.15.86:38218) 2024-04-17T07:17:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb72b400640 1 -- 172.21.15.86:0/2505683788 learned_addr learned my addr 172.21.15.86:0/2505683788 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb72aa00640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c075fa0 0x7fb72c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.446+0000 7fb72be00640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c109cc0 0x7fb72c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.447+0000 7fb72b400640 1 -- 172.21.15.86:0/2505683788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c109cc0 msgr2=0x7fb72c1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.447+0000 7fb72b400640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c109cc0 0x7fb72c1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.447+0000 7fb72b400640 1 -- 172.21.15.86:0/2505683788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c075fa0 msgr2=0x7fb72c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.447+0000 7fb72b400640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c075fa0 0x7fb72c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.447+0000 7fb72b400640 1 -- 172.21.15.86:0/2505683788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c1a1dd0 con 0x7fb72c0749c0 2024-04-17T07:17:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.447+0000 7fb72b400640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0749c0 0x7fb72c19a1b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00d950 tx=0x7fb71c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.447+0000 7fb723e00640 1 -- 172.21.15.86:0/2505683788 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c017070 con 0x7fb72c0749c0 2024-04-17T07:17:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.448+0000 7fb731200640 1 -- 172.21.15.86:0/2505683788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c1a20c0 con 0x7fb72c0749c0 2024-04-17T07:17:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.448+0000 7fb723e00640 1 -- 172.21.15.86:0/2505683788 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb71c009c30 con 0x7fb72c0749c0 2024-04-17T07:17:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.448+0000 7fb723e00640 1 -- 172.21.15.86:0/2505683788 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c020060 con 0x7fb72c0749c0 2024-04-17T07:17:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.448+0000 7fb731200640 1 -- 172.21.15.86:0/2505683788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c1a2610 con 0x7fb72c0749c0 2024-04-17T07:17:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.449+0000 7fb731200640 1 -- 172.21.15.86:0/2505683788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c1089f0 con 0x7fb72c0749c0 2024-04-17T07:17:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.451+0000 7fb723e00640 1 -- 172.21.15.86:0/2505683788 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb71c0109a0 con 0x7fb72c0749c0 2024-04-17T07:17:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.453+0000 7fb723e00640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3bb0 0x7fb7100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.453+0000 7fb723e00640 1 -- 172.21.15.86:0/2505683788 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb71c0d03f0 con 0x7fb72c0749c0 2024-04-17T07:17:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.453+0000 7fb72aa00640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3bb0 0x7fb7100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.454+0000 7fb72aa00640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3bb0 0x7fb7100a6070 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb72c19b6c0 tx=0x7fb71800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.459+0000 7fb723e00640 1 -- 172.21.15.86:0/2505683788 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb71c009de0 con 0x7fb72c0749c0 2024-04-17T07:17:24.720 INFO:teuthology.orchestra.run.smithi086.stderr:1800000 of 3200000 tuples (56%) done (elapsed 74.27 s, remaining 57.76 s) 2024-04-17T07:17:24.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.732+0000 7fb731200640 1 -- 172.21.15.86:0/2505683788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb72c02d5d0 con 0x7fb72c0749c0 2024-04-17T07:17:24.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.733+0000 7fb723e00640 1 -- 172.21.15.86:0/2505683788 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fb71c09c400 con 0x7fb72c0749c0 2024-04-17T07:17:24.732 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:17:24.732 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:17:24.732 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:17:24.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 -- 172.21.15.86:0/2505683788 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3bb0 msgr2=0x7fb7100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3bb0 0x7fb7100a6070 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb72c19b6c0 tx=0x7fb71800b390 comp rx=0 tx=0).stop 2024-04-17T07:17:24.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 -- 172.21.15.86:0/2505683788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0749c0 msgr2=0x7fb72c19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0749c0 0x7fb72c19a1b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00d950 tx=0x7fb71c00de10 comp rx=0 tx=0).stop 2024-04-17T07:17:24.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 -- 172.21.15.86:0/2505683788 shutdown_connections 2024-04-17T07:17:24.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3bb0 0x7fb7100a6070 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c109cc0 0x7fb72c1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c075fa0 0x7fb72c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 --2- 172.21.15.86:0/2505683788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0749c0 0x7fb72c19a1b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 -- 172.21.15.86:0/2505683788 >> 172.21.15.86:0/2505683788 conn(0x7fb72c0fd370 msgr2=0x7fb72c105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.736+0000 7fb731200640 1 -- 172.21.15.86:0/2505683788 shutdown_connections 2024-04-17T07:17:24.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.737+0000 7fb731200640 1 -- 172.21.15.86:0/2505683788 wait complete. 2024-04-17T07:17:24.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:17:24.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.974+0000 7fe229600640 1 -- 172.21.15.86:0/418241956 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21802cbe0 con 0x7fe22c105370 2024-04-17T07:17:24.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.974+0000 7fe231800640 1 -- 172.21.15.86:0/418241956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22c105370 msgr2=0x7fe22c107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.974+0000 7fe231800640 1 --2- 172.21.15.86:0/418241956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22c105370 0x7fe22c107760 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe218009900 tx=0x7fe21802bc30 comp rx=0 tx=0).stop 2024-04-17T07:17:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.974+0000 7fe231800640 1 -- 172.21.15.86:0/418241956 shutdown_connections 2024-04-17T07:17:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.974+0000 7fe231800640 1 --2- 172.21.15.86:0/418241956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe22c107ca0 0x7fe22c10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.974+0000 7fe231800640 1 --2- 172.21.15.86:0/418241956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22c105370 0x7fe22c107760 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.974+0000 7fe231800640 1 --2- 172.21.15.86:0/418241956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe22c102a40 0x7fe22c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.974+0000 7fe231800640 1 -- 172.21.15.86:0/418241956 >> 172.21.15.86:0/418241956 conn(0x7fe22c0fc450 msgr2=0x7fe22c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.974+0000 7fe231800640 1 -- 172.21.15.86:0/418241956 shutdown_connections 2024-04-17T07:17:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.975+0000 7fe231800640 1 -- 172.21.15.86:0/418241956 wait complete. 2024-04-17T07:17:24.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.975+0000 7fe231800640 1 Processor -- start 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.976+0000 7fe231800640 1 -- start start 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.976+0000 7fe231800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe22c102a40 0x7fe22c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.976+0000 7fe231800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22c105370 0x7fe22c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.976+0000 7fe231800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe22c107ca0 0x7fe22c1a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.976+0000 7fe231800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c10db00 con 0x7fe22c105370 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.976+0000 7fe231800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c10d980 con 0x7fe22c107ca0 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.976+0000 7fe231800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c10dc80 con 0x7fe22c102a40 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.976+0000 7fe22a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22c105370 0x7fe22c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.976+0000 7fe22a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22c105370 0x7fe22c19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38234/0 (socket says 172.21.15.86:38234) 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.976+0000 7fe22a000640 1 -- 172.21.15.86:0/1303957162 learned_addr learned my addr 172.21.15.86:0/1303957162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe22b400640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe22c107ca0 0x7fe22c1a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe22aa00640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe22c102a40 0x7fe22c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe22a000640 1 -- 172.21.15.86:0/1303957162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe22c102a40 msgr2=0x7fe22c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe22a000640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe22c102a40 0x7fe22c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe22a000640 1 -- 172.21.15.86:0/1303957162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe22c107ca0 msgr2=0x7fe22c1a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe22a000640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe22c107ca0 0x7fe22c1a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe22a000640 1 -- 172.21.15.86:0/1303957162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c1a2080 con 0x7fe22c105370 2024-04-17T07:17:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe22a000640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22c105370 0x7fe22c19a9a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe218000c00 tx=0x7fe218004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe223400640 1 -- 172.21.15.86:0/1303957162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218043020 con 0x7fe22c105370 2024-04-17T07:17:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe223400640 1 -- 172.21.15.86:0/1303957162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe218004aa0 con 0x7fe22c105370 2024-04-17T07:17:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.977+0000 7fe223400640 1 -- 172.21.15.86:0/1303957162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218003d00 con 0x7fe22c105370 2024-04-17T07:17:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.978+0000 7fe231800640 1 -- 172.21.15.86:0/1303957162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c1a2310 con 0x7fe22c105370 2024-04-17T07:17:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.978+0000 7fe231800640 1 -- 172.21.15.86:0/1303957162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c1a28b0 con 0x7fe22c105370 2024-04-17T07:17:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.981+0000 7fe223400640 1 -- 172.21.15.86:0/1303957162 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fe218046020 con 0x7fe22c105370 2024-04-17T07:17:24.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.982+0000 7fe234589740 1 Processor -- start 2024-04-17T07:17:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.982+0000 7fe234589740 1 -- start start 2024-04-17T07:17:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.982+0000 7fe234589740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f22c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.982+0000 7fe234589740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f22c990 0x56313f231490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.982+0000 7fe234589740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f231b00 0x56313f233f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.982+0000 7fe234589740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56313f192020 con 0x56313f22c990 2024-04-17T07:17:24.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.982+0000 7fe234589740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56313f138460 con 0x56313f231b00 2024-04-17T07:17:24.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.982+0000 7fe234589740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56313f18dfe0 con 0x56313f22bfa0 2024-04-17T07:17:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.983+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f22c990 0x56313f231490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.983+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f22c990 0x56313f231490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38240/0 (socket says 172.21.15.86:38240) 2024-04-17T07:17:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.983+0000 7fe22aa00640 1 -- 172.21.15.86:0/2134994635 learned_addr learned my addr 172.21.15.86:0/2134994635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.983+0000 7fe223400640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2100a3c30 0x7fe2100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.983+0000 7fe22b400640 1 --2- 172.21.15.86:0/2134994635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f22c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.983+0000 7fe223400640 1 -- 172.21.15.86:0/1303957162 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe2180f2da0 con 0x7fe22c105370 2024-04-17T07:17:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.983+0000 7fe22a000640 1 --2- 172.21.15.86:0/2134994635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f231b00 0x56313f233f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.983+0000 7fe22b400640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2100a3c30 0x7fe2100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.984+0000 7fe22aa00640 1 -- 172.21.15.86:0/2134994635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 msgr2=0x56313f22c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.984+0000 7fe22aa00640 1 --2- 172.21.15.86:0/2134994635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f22c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.984+0000 7fe22aa00640 1 -- 172.21.15.86:0/2134994635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f231b00 msgr2=0x56313f233f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.984+0000 7fe22aa00640 1 --2- 172.21.15.86:0/2134994635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f231b00 0x56313f233f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.984+0000 7fe22aa00640 1 -- 172.21.15.86:0/2134994635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56313f140040 con 0x56313f22c990 2024-04-17T07:17:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.984+0000 7fe22a000640 1 --2- 172.21.15.86:0/2134994635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f231b00 0x56313f233f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.984+0000 7fe22aa00640 1 --2- 172.21.15.86:0/2134994635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f22c990 0x56313f231490 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009f60 tx=0x7fe21c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91957929123761ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.984+0000 7fe22b400640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2100a3c30 0x7fe2100a60f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe2240058c0 tx=0x7fe224004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.985+0000 7fe217e00640 1 -- 172.21.15.86:0/2134994635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c016070 con 0x56313f22c990 2024-04-17T07:17:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.985+0000 7fe217e00640 1 -- 172.21.15.86:0/2134994635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21c004d20 con 0x56313f22c990 2024-04-17T07:17:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.985+0000 7fe217e00640 1 -- 172.21.15.86:0/2134994635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c012770 con 0x56313f22c990 2024-04-17T07:17:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.985+0000 7fe234589740 1 -- 172.21.15.86:0/2134994635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f22c990 msgr2=0x56313f231490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.985+0000 7fe234589740 1 --2- 172.21.15.86:0/2134994635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f22c990 0x56313f231490 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009f60 tx=0x7fe21c009f90 comp rx=0 tx=0).stop 2024-04-17T07:17:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.986+0000 7fe234589740 1 -- 172.21.15.86:0/2134994635 shutdown_connections 2024-04-17T07:17:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.986+0000 7fe234589740 1 --2- 172.21.15.86:0/2134994635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f231b00 0x56313f233f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.986+0000 7fe234589740 1 --2- 172.21.15.86:0/2134994635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f22c990 0x56313f231490 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.986+0000 7fe234589740 1 --2- 172.21.15.86:0/2134994635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f22c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.986+0000 7fe234589740 1 -- 172.21.15.86:0/2134994635 >> 172.21.15.86:0/2134994635 conn(0x56313f227510 msgr2=0x56313f227910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.986+0000 7fe234589740 1 -- 172.21.15.86:0/2134994635 shutdown_connections 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.986+0000 7fe234589740 1 -- 172.21.15.86:0/2134994635 wait complete. 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.986+0000 7fe234589740 1 -- start start 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.986+0000 7fe234589740 1 Processor -- start 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe234589740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f23ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe234589740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 0x56313f23f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe234589740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f2462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe234589740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56313f23d2c0 con 0x56313f231b00 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe234589740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56313f14b830 con 0x56313f22c990 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe234589740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56313f23d440 con 0x56313f22bfa0 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe234589740 10 client.0 _ll_register_callbacks cb 0x56313f119fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe22b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f2462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe22b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f2462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38248/0 (socket says 172.21.15.86:38248) 2024-04-17T07:17:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe22b400640 1 -- 172.21.15.86:0/3762786103 learned_addr learned my addr 172.21.15.86:0/3762786103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.987+0000 7fe22aa00640 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f23ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.988+0000 7fe22a000640 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 0x56313f23f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.988+0000 7fe22b400640 1 -- 172.21.15.86:0/3762786103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 msgr2=0x56313f23ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.988+0000 7fe22b400640 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f23ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.988+0000 7fe22b400640 1 -- 172.21.15.86:0/3762786103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 msgr2=0x56313f23f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.988+0000 7fe22b400640 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 0x56313f23f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.988+0000 7fe22b400640 1 -- 172.21.15.86:0/3762786103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56313f14f920 con 0x56313f231b00 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.988+0000 7fe22aa00640 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f23ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.988+0000 7fe22a000640 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 0x56313f23f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.988+0000 7fe22b400640 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f2462a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe22400f740 tx=0x7fe2240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1c667552a61ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.988+0000 7fe214c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.989+0000 7fe214c00640 1 -- 172.21.15.86:0/3762786103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224024030 con 0x56313f231b00 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.989+0000 7fe234589740 1 -- 172.21.15.86:0/3762786103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe21c004d20 con 0x56313f231b00 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.989+0000 7fe214c00640 1 -- 172.21.15.86:0/3762786103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe2240117c0 con 0x56313f231b00 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.989+0000 7fe214c00640 1 -- 172.21.15.86:0/3762786103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe224031050 con 0x56313f231b00 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.989+0000 7fe234589740 10 client.16281 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.989+0000 7fe234589740 1 -- 172.21.15.86:0/3762786103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56313f11d7f0 con 0x56313f231b00 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.989+0000 7fe214c00640 1 -- 172.21.15.86:0/3762786103 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21c004d20 con 0x56313f231b00 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.989+0000 7fe214c00640 1 -- 172.21.15.86:0/3762786103 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe22401d050 con 0x56313f231b00 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.990+0000 7fe234589740 10 client.16281 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.990+0000 7fe234589740 10 client.16281 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.990+0000 7fe234589740 20 client.16281 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.990+0000 7fe234589740 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56313f24a7e0 0x56313f24cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.990+0000 7fe234589740 4 client.16281 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:24.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.990+0000 7fe234589740 1 -- 172.21.15.86:0/3762786103 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56313f19fa40 con 0x56313f24a7e0 2024-04-17T07:17:24.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.990+0000 7fe22aa00640 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56313f24a7e0 0x56313f24cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.991+0000 7fe214c00640 10 client.16281 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:17:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.991+0000 7fe22aa00640 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56313f24a7e0 0x56313f24cca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51ee1fe30927bb41 server_cookie=50bbeabcff71eb0b in_seq=0 out_seq=0 2024-04-17T07:17:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe214c00640 1 -- 172.21.15.86:0/3762786103 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56313f19fa40 con 0x56313f24a7e0 2024-04-17T07:17:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe214c00640 10 client.16281 handle_command_reply: tid=0 2024-04-17T07:17:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe234589740 1 client.16281 shutdown 2024-04-17T07:17:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe234589740 1 -- 172.21.15.86:0/3762786103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 msgr2=0x56313f2462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:24.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe234589740 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f2462a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe22400f740 tx=0x7fe2240046c0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe234589740 1 -- 172.21.15.86:0/3762786103 shutdown_connections 2024-04-17T07:17:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe234589740 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56313f24a7e0 0x56313f24cca0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe234589740 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f2462a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe234589740 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 0x56313f23f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe234589740 1 --2- 172.21.15.86:0/3762786103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f23ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe234589740 1 -- 172.21.15.86:0/3762786103 >> 172.21.15.86:0/3762786103 conn(0x56313f227510 msgr2=0x56313f226710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.993+0000 7fe214c00640 0 client.16281 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:17:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.994+0000 7fe234589740 1 -- 172.21.15.86:0/3762786103 shutdown_connections 2024-04-17T07:17:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.994+0000 7fe234589740 1 -- 172.21.15.86:0/3762786103 wait complete. 2024-04-17T07:17:24.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:24.994+0000 7fe234589740 20 client.16281 trim_cache size 0 max 16384 2024-04-17T07:17:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe234589740 1 Processor -- start 2024-04-17T07:17:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe234589740 1 -- start start 2024-04-17T07:17:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe234589740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f24c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe234589740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 0x56313f246260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe234589740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f225e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe234589740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1a8890 con 0x56313f231b00 2024-04-17T07:17:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe234589740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1aa3d0 con 0x56313f22c990 2024-04-17T07:17:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe234589740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1a8f60 con 0x56313f22bfa0 2024-04-17T07:17:25.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe22b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f225e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe22b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f225e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38262/0 (socket says 172.21.15.86:38262) 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe22b400640 1 -- 172.21.15.86:0/3620415192 learned_addr learned my addr 172.21.15.86:0/3620415192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.010+0000 7fe22aa00640 1 --2- 172.21.15.86:0/3620415192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f24c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.011+0000 7fe22a000640 1 --2- 172.21.15.86:0/3620415192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 0x56313f246260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.011+0000 7fe22b400640 1 -- 172.21.15.86:0/3620415192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 msgr2=0x56313f24c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.011+0000 7fe22b400640 1 --2- 172.21.15.86:0/3620415192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f24c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.011+0000 7fe22b400640 1 -- 172.21.15.86:0/3620415192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 msgr2=0x56313f246260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.011+0000 7fe22b400640 1 --2- 172.21.15.86:0/3620415192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 0x56313f246260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.011+0000 7fe22b400640 1 -- 172.21.15.86:0/3620415192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56313eea43f0 con 0x56313f231b00 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.011+0000 7fe22b400640 1 --2- 172.21.15.86:0/3620415192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f225e80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe224014f20 tx=0x7fe224011450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2edd771ee09d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.011+0000 7fe217400640 1 -- 172.21.15.86:0/3620415192 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224024040 con 0x56313f231b00 2024-04-17T07:17:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.012+0000 7fe217400640 1 -- 172.21.15.86:0/3620415192 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe224005040 con 0x56313f231b00 2024-04-17T07:17:25.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.012+0000 7fe217400640 1 -- 172.21.15.86:0/3620415192 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224004a70 con 0x56313f231b00 2024-04-17T07:17:25.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.012+0000 7fe234589740 1 -- 172.21.15.86:0/3620415192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 msgr2=0x56313f225e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.012+0000 7fe234589740 1 --2- 172.21.15.86:0/3620415192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f225e80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe224014f20 tx=0x7fe224011450 comp rx=0 tx=0).stop 2024-04-17T07:17:25.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.012+0000 7fe234589740 1 -- 172.21.15.86:0/3620415192 shutdown_connections 2024-04-17T07:17:25.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.012+0000 7fe234589740 1 --2- 172.21.15.86:0/3620415192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f225e80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.012+0000 7fe234589740 1 --2- 172.21.15.86:0/3620415192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22c990 0x56313f246260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.013+0000 7fe234589740 1 --2- 172.21.15.86:0/3620415192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22bfa0 0x56313f24c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.013+0000 7fe234589740 1 -- 172.21.15.86:0/3620415192 >> 172.21.15.86:0/3620415192 conn(0x56313f227510 msgr2=0x56313f22de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.013+0000 7fe234589740 1 -- 172.21.15.86:0/3620415192 shutdown_connections 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.013+0000 7fe234589740 1 -- 172.21.15.86:0/3620415192 wait complete. 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.013+0000 7fe234589740 1 -- start start 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.013+0000 7fe234589740 1 Processor -- start 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe234589740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22bfa0 0x56313f243660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe234589740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22c990 0x56313f243ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe234589740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f244b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe234589740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1a9d00 con 0x56313f231b00 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe234589740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1a9630 con 0x56313f22bfa0 2024-04-17T07:17:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe234589740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c1a6860 con 0x56313f22c990 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe234589740 10 client.0 _ll_register_callbacks cb 0x7fe22400f3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe22b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f244b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe22b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f244b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38266/0 (socket says 172.21.15.86:38266) 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe22b400640 1 -- 172.21.15.86:0/1575568735 learned_addr learned my addr 172.21.15.86:0/1575568735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe22aa00640 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22bfa0 0x56313f243660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.014+0000 7fe22a000640 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22c990 0x56313f243ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.015+0000 7fe22b400640 1 -- 172.21.15.86:0/1575568735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22c990 msgr2=0x56313f243ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.015+0000 7fe22b400640 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22c990 0x56313f243ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.015+0000 7fe22b400640 1 -- 172.21.15.86:0/1575568735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22bfa0 msgr2=0x56313f243660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.015+0000 7fe22b400640 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22bfa0 0x56313f243660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.015+0000 7fe22b400640 1 -- 172.21.15.86:0/1575568735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56313ef6e7d0 con 0x56313f231b00 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.015+0000 7fe22b400640 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f244b60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe22400bfd0 tx=0x7fe224005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6059fd07667f6419 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.015+0000 7fe214c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.015+0000 7fe214c00640 1 -- 172.21.15.86:0/1575568735 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224043060 con 0x56313f231b00 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.015+0000 7fe214c00640 1 -- 172.21.15.86:0/1575568735 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe224020680 con 0x56313f231b00 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.015+0000 7fe234589740 1 -- 172.21.15.86:0/1575568735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe224005040 con 0x56313f231b00 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe214c00640 1 -- 172.21.15.86:0/1575568735 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe22404a050 con 0x56313f231b00 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe234589740 10 client.16287 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe234589740 1 -- 172.21.15.86:0/1575568735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56313f246280 con 0x56313f231b00 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe214c00640 1 -- 172.21.15.86:0/1575568735 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe224005040 con 0x56313f231b00 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe214c00640 1 -- 172.21.15.86:0/1575568735 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe22401d070 con 0x56313f231b00 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe234589740 10 client.16287 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe234589740 10 client.16287 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:17:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe234589740 20 client.16287 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe234589740 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56313f2b11f0 0x56313f2b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe234589740 4 client.16287 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:17:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.016+0000 7fe234589740 1 -- 172.21.15.86:0/1575568735 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56313f1998b0 con 0x56313f2b11f0 2024-04-17T07:17:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.017+0000 7fe22aa00640 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56313f2b11f0 0x56313f2b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.018+0000 7fe214c00640 10 client.16287 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:17:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.018+0000 7fe22aa00640 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56313f2b11f0 0x56313f2b3610 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e680670e55ddb29e server_cookie=1c8c19a4f19f37a1 in_seq=0 out_seq=0 2024-04-17T07:17:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.018+0000 7fe214c00640 1 -- 172.21.15.86:0/1575568735 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56313f1998b0 con 0x56313f2b11f0 2024-04-17T07:17:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.018+0000 7fe214c00640 10 client.16287 handle_command_reply: tid=0 2024-04-17T07:17:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.018+0000 7fe234589740 1 client.16287 shutdown 2024-04-17T07:17:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.018+0000 7fe234589740 1 -- 172.21.15.86:0/1575568735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 msgr2=0x56313f244b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.018+0000 7fe234589740 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f244b60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe22400bfd0 tx=0x7fe224005260 comp rx=0 tx=0).stop 2024-04-17T07:17:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.019+0000 7fe234589740 1 -- 172.21.15.86:0/1575568735 shutdown_connections 2024-04-17T07:17:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.019+0000 7fe234589740 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56313f2b11f0 0x56313f2b3610 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.019+0000 7fe234589740 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56313f231b00 0x56313f244b60 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.019+0000 7fe234589740 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56313f22c990 0x56313f243ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.019+0000 7fe234589740 1 --2- 172.21.15.86:0/1575568735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56313f22bfa0 0x56313f243660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.019+0000 7fe234589740 1 -- 172.21.15.86:0/1575568735 >> 172.21.15.86:0/1575568735 conn(0x56313f227510 msgr2=0x56313f22a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.019+0000 7fe214c00640 0 client.16287 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:17:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.019+0000 7fe234589740 1 -- 172.21.15.86:0/1575568735 shutdown_connections 2024-04-17T07:17:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.019+0000 7fe234589740 1 -- 172.21.15.86:0/1575568735 wait complete. 2024-04-17T07:17:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.019+0000 7fe234589740 20 client.16287 trim_cache size 0 max 16384 2024-04-17T07:17:25.020 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:17:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 -- 172.21.15.86:0/1303957162 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2100a3c30 msgr2=0x7fe2100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2100a3c30 0x7fe2100a60f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe2240058c0 tx=0x7fe224004730 comp rx=0 tx=0).stop 2024-04-17T07:17:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 -- 172.21.15.86:0/1303957162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22c105370 msgr2=0x7fe22c19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22c105370 0x7fe22c19a9a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe218000c00 tx=0x7fe218004270 comp rx=0 tx=0).stop 2024-04-17T07:17:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 -- 172.21.15.86:0/1303957162 shutdown_connections 2024-04-17T07:17:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe2100a3c30 0x7fe2100a60f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe22c107ca0 0x7fe22c1a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22c105370 0x7fe22c19a9a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 --2- 172.21.15.86:0/1303957162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe22c102a40 0x7fe22c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 -- 172.21.15.86:0/1303957162 >> 172.21.15.86:0/1303957162 conn(0x7fe22c0fc450 msgr2=0x7fe22c102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 -- 172.21.15.86:0/1303957162 shutdown_connections 2024-04-17T07:17:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.020+0000 7fe231800640 1 -- 172.21.15.86:0/1303957162 wait complete. 2024-04-17T07:17:25.034 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:24 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2505683788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:17:25.037 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:17:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.248+0000 7f27a3a00640 1 -- 172.21.15.86:0/1998708831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279c101fb0 msgr2=0x7f279c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.248+0000 7f27a3a00640 1 --2- 172.21.15.86:0/1998708831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279c101fb0 0x7f279c1023d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2790009900 tx=0x7f279002bc30 comp rx=0 tx=0).stop 2024-04-17T07:17:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.248+0000 7f27a3a00640 1 -- 172.21.15.86:0/1998708831 shutdown_connections 2024-04-17T07:17:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.248+0000 7f27a3a00640 1 --2- 172.21.15.86:0/1998708831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f279c103b60 0x7f279c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.248+0000 7f27a3a00640 1 --2- 172.21.15.86:0/1998708831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279c1031a0 0x7f279c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.248+0000 7f27a3a00640 1 --2- 172.21.15.86:0/1998708831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279c101fb0 0x7f279c1023d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.248+0000 7f27a3a00640 1 -- 172.21.15.86:0/1998708831 >> 172.21.15.86:0/1998708831 conn(0x7f279c0fd540 msgr2=0x7f279c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.248+0000 7f27a3a00640 1 -- 172.21.15.86:0/1998708831 shutdown_connections 2024-04-17T07:17:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.248+0000 7f27a3a00640 1 -- 172.21.15.86:0/1998708831 wait complete. 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.249+0000 7f27a3a00640 1 Processor -- start 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.249+0000 7f27a3a00640 1 -- start start 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f27a3a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279c101fb0 0x7f279c075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f27a3a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f279c1031a0 0x7f279c074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f27a3a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279c103b60 0x7f279c074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f27a3a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f279c10db40 con 0x7f279c101fb0 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f27a3a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f279c10d9c0 con 0x7f279c103b60 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f27a3a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f279c10dcc0 con 0x7f279c1031a0 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f27a1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279c101fb0 0x7f279c075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f27a1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279c101fb0 0x7f279c075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38290/0 (socket says 172.21.15.86:38290) 2024-04-17T07:17:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f27a1000640 1 -- 172.21.15.86:0/3004209431 learned_addr learned my addr 172.21.15.86:0/3004209431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f27a1a00640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279c103b60 0x7f279c074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.250+0000 7f279be00640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f279c1031a0 0x7f279c074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.251+0000 7f27a1000640 1 -- 172.21.15.86:0/3004209431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f279c1031a0 msgr2=0x7f279c074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.251+0000 7f27a1000640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f279c1031a0 0x7f279c074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.251+0000 7f27a1000640 1 -- 172.21.15.86:0/3004209431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279c103b60 msgr2=0x7f279c074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.251+0000 7f27a1000640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279c103b60 0x7f279c074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.251+0000 7f27a1000640 1 -- 172.21.15.86:0/3004209431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c075120 con 0x7f279c101fb0 2024-04-17T07:17:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.251+0000 7f27a1000640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279c101fb0 0x7f279c075d10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2790005ab0 tx=0x7f27900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.251+0000 7f2799600640 1 -- 172.21.15.86:0/3004209431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790043020 con 0x7f279c101fb0 2024-04-17T07:17:25.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.251+0000 7f2799600640 1 -- 172.21.15.86:0/3004209431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27900054c0 con 0x7f279c101fb0 2024-04-17T07:17:25.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.251+0000 7f27a3a00640 1 -- 172.21.15.86:0/3004209431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c1a22e0 con 0x7f279c101fb0 2024-04-17T07:17:25.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.252+0000 7f2799600640 1 -- 172.21.15.86:0/3004209431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790003d10 con 0x7f279c101fb0 2024-04-17T07:17:25.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.252+0000 7f27a3a00640 1 -- 172.21.15.86:0/3004209431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c1a2880 con 0x7f279c101fb0 2024-04-17T07:17:25.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:24 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2505683788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:17:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.256+0000 7f2799600640 1 -- 172.21.15.86:0/3004209431 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2790046020 con 0x7f279c101fb0 2024-04-17T07:17:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.256+0000 7f27a65f6740 1 Processor -- start 2024-04-17T07:17:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.257+0000 7f27a65f6740 1 -- start start 2024-04-17T07:17:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.257+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 0x55a57310d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.257+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573112490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.257+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 0x55a573114f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.257+0000 7f27a65f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a573073020 con 0x55a57310d990 2024-04-17T07:17:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.257+0000 7f27a65f6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a573019460 con 0x55a57310cfa0 2024-04-17T07:17:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.257+0000 7f27a65f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a57306efe0 con 0x55a573112b00 2024-04-17T07:17:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.257+0000 7f27a1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573112490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.257+0000 7f27a1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573112490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38292/0 (socket says 172.21.15.86:38292) 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.257+0000 7f27a1000640 1 -- 172.21.15.86:0/4189730943 learned_addr learned my addr 172.21.15.86:0/4189730943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.258+0000 7f27a1a00640 1 --2- 172.21.15.86:0/4189730943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 0x55a57310d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.258+0000 7f279be00640 1 --2- 172.21.15.86:0/4189730943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 0x55a573114f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.258+0000 7f27a1000640 1 -- 172.21.15.86:0/4189730943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 msgr2=0x55a573114f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.258+0000 7f27a1000640 1 --2- 172.21.15.86:0/4189730943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 0x55a573114f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.258+0000 7f27a1000640 1 -- 172.21.15.86:0/4189730943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 msgr2=0x55a57310d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.258+0000 7f27a1000640 1 --2- 172.21.15.86:0/4189730943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 0x55a57310d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.258+0000 7f27a1000640 1 -- 172.21.15.86:0/4189730943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a573021040 con 0x55a57310d990 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.258+0000 7f27a1a00640 1 --2- 172.21.15.86:0/4189730943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 0x55a57310d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.258+0000 7f2799600640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f27840a3c30 0x7f27840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.258+0000 7f27a1000640 1 --2- 172.21.15.86:0/4189730943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573112490 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f27900f8a80 tx=0x7f27900f8b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab04cf98e5ca8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.259+0000 7f27a1a00640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f27840a3c30 0x7f27840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.259+0000 7f2799600640 1 -- 172.21.15.86:0/3004209431 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f27900f27a0 con 0x7f279c101fb0 2024-04-17T07:17:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.259+0000 7f278e000640 1 -- 172.21.15.86:0/4189730943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27900ba9a0 con 0x55a57310d990 2024-04-17T07:17:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.259+0000 7f278e000640 1 -- 172.21.15.86:0/4189730943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2790004c50 con 0x55a57310d990 2024-04-17T07:17:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.259+0000 7f278e000640 1 -- 172.21.15.86:0/4189730943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27900be060 con 0x55a57310d990 2024-04-17T07:17:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.260+0000 7f27a65f6740 1 -- 172.21.15.86:0/4189730943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 msgr2=0x55a573112490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.260+0000 7f27a1a00640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f27840a3c30 0x7f27840a60f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55a57310e1f0 tx=0x7f27940078f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.260+0000 7f27a65f6740 1 --2- 172.21.15.86:0/4189730943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573112490 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f27900f8a80 tx=0x7f27900f8b70 comp rx=0 tx=0).stop 2024-04-17T07:17:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.260+0000 7f27a65f6740 1 -- 172.21.15.86:0/4189730943 shutdown_connections 2024-04-17T07:17:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.260+0000 7f27a65f6740 1 --2- 172.21.15.86:0/4189730943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 0x55a573114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.260+0000 7f27a65f6740 1 --2- 172.21.15.86:0/4189730943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573112490 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.260+0000 7f27a65f6740 1 --2- 172.21.15.86:0/4189730943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 0x55a57310d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.260+0000 7f27a65f6740 1 -- 172.21.15.86:0/4189730943 >> 172.21.15.86:0/4189730943 conn(0x55a573108510 msgr2=0x55a573108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.260+0000 7f27a65f6740 1 -- 172.21.15.86:0/4189730943 shutdown_connections 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.260+0000 7f27a65f6740 1 -- 172.21.15.86:0/4189730943 wait complete. 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.261+0000 7f27a65f6740 1 -- start start 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.261+0000 7f27a65f6740 1 Processor -- start 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.261+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310cfa0 0x55a5731171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.261+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573117710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.261+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a573112b00 0x55a57311e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.261+0000 7f27a65f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a573115870 con 0x55a57310d990 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.261+0000 7f27a65f6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a57302c830 con 0x55a573112b00 2024-04-17T07:17:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.261+0000 7f27a65f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5731159f0 con 0x55a57310cfa0 2024-04-17T07:17:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.261+0000 7f27a65f6740 10 client.0 _ll_register_callbacks cb 0x55a572ffafb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.262+0000 7f279be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573117710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.262+0000 7f279be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573117710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38302/0 (socket says 172.21.15.86:38302) 2024-04-17T07:17:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.262+0000 7f279be00640 1 -- 172.21.15.86:0/13594088 learned_addr learned my addr 172.21.15.86:0/13594088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.262+0000 7f27a1000640 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310cfa0 0x55a5731171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.262+0000 7f279be00640 1 -- 172.21.15.86:0/13594088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310cfa0 msgr2=0x55a5731171d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.262+0000 7f279be00640 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310cfa0 0x55a5731171d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.262+0000 7f279be00640 1 -- 172.21.15.86:0/13594088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a573112b00 msgr2=0x55a57311e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.262+0000 7f279be00640 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a573112b00 0x55a57311e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.262+0000 7f279be00640 1 -- 172.21.15.86:0/13594088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a573030920 con 0x55a57310d990 2024-04-17T07:17:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.263+0000 7f279be00640 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573117710 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f278800be10 tx=0x7f27880117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b099701790f3ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.263+0000 7f2782a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.263+0000 7f2782a00640 1 -- 172.21.15.86:0/13594088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788004230 con 0x55a57310d990 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.263+0000 7f2782a00640 1 -- 172.21.15.86:0/13594088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f27880043d0 con 0x55a57310d990 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.263+0000 7f27a65f6740 1 -- 172.21.15.86:0/13594088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2790004c50 con 0x55a57310d990 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.263+0000 7f2782a00640 1 -- 172.21.15.86:0/13594088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27880035f0 con 0x55a57310d990 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.263+0000 7f27a65f6740 10 client.16299 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.263+0000 7f27a65f6740 1 -- 172.21.15.86:0/13594088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a572ffe7f0 con 0x55a57310d990 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.264+0000 7f2782a00640 1 -- 172.21.15.86:0/13594088 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2790004c50 con 0x55a57310d990 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.264+0000 7f2782a00640 1 -- 172.21.15.86:0/13594088 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f278801b7e0 con 0x55a57310d990 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.264+0000 7f27a65f6740 10 client.16299 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.264+0000 7f27a65f6740 10 client.16299 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.264+0000 7f27a65f6740 20 client.16299 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.264+0000 7f27a65f6740 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55a573122cf0 0x55a573125190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.264+0000 7f27a65f6740 4 client.16299 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.264+0000 7f27a65f6740 1 -- 172.21.15.86:0/13594088 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a573080a40 con 0x55a573122cf0 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.265+0000 7f27a1000640 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55a573122cf0 0x55a573125190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.266+0000 7f27a1000640 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55a573122cf0 0x55a573125190 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd5b3d256cadb35b server_cookie=4bcfa859ad2dcabb in_seq=0 out_seq=0 2024-04-17T07:17:25.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.266+0000 7f2782a00640 10 client.16299 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.267+0000 7f2782a00640 1 -- 172.21.15.86:0/13594088 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a573080a40 con 0x55a573122cf0 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.267+0000 7f2782a00640 10 client.16299 handle_command_reply: tid=0 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.267+0000 7f27a65f6740 1 client.16299 shutdown 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 1 -- 172.21.15.86:0/13594088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 msgr2=0x55a573117710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573117710 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f278800be10 tx=0x7f27880117b0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 1 -- 172.21.15.86:0/13594088 shutdown_connections 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55a573122cf0 0x55a573125190 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a573112b00 0x55a57311e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a573117710 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 1 --2- 172.21.15.86:0/13594088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310cfa0 0x55a5731171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 1 -- 172.21.15.86:0/13594088 >> 172.21.15.86:0/13594088 conn(0x55a573108510 msgr2=0x55a573107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f2782a00640 0 client.16299 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:17:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 1 -- 172.21.15.86:0/13594088 shutdown_connections 2024-04-17T07:17:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 1 -- 172.21.15.86:0/13594088 wait complete. 2024-04-17T07:17:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.268+0000 7f27a65f6740 20 client.16299 trim_cache size 0 max 16384 2024-04-17T07:17:25.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:24 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2505683788' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:17:25.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.285+0000 7f27a65f6740 1 Processor -- start 2024-04-17T07:17:25.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.285+0000 7f27a65f6740 1 -- start start 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310cfa0 0x55a57310ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310d990 0x55a57310f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a573112b00 0x55a57311e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a65f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1acce0 con 0x55a57310cfa0 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a65f6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1ae820 con 0x55a573112b00 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a65f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1ad3b0 con 0x55a57310d990 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310cfa0 0x55a57310ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310cfa0 0x55a57310ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38318/0 (socket says 172.21.15.86:38318) 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a1000640 1 -- 172.21.15.86:0/2240358427 learned_addr learned my addr 172.21.15.86:0/2240358427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a1000640 1 -- 172.21.15.86:0/2240358427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310d990 msgr2=0x55a57310f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f27a1a00640 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a573112b00 0x55a57311e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.286+0000 7f279be00640 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310d990 0x55a57310f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.287+0000 7f27a1000640 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310d990 0x55a57310f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.287+0000 7f27a1000640 1 -- 172.21.15.86:0/2240358427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a573112b00 msgr2=0x55a57311e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.287+0000 7f27a1000640 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a573112b00 0x55a57311e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.287+0000 7f27a1000640 1 -- 172.21.15.86:0/2240358427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a572d853f0 con 0x55a57310cfa0 2024-04-17T07:17:25.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.287+0000 7f279be00640 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310d990 0x55a57310f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:25.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.287+0000 7f27a1a00640 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a573112b00 0x55a57311e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:25.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.287+0000 7f27a1000640 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310cfa0 0x55a57310ee50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f27900f8fd0 tx=0x7f2790000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd9ee4a51efc1ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.287+0000 7f278d600640 1 -- 172.21.15.86:0/2240358427 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27900be240 con 0x55a57310cfa0 2024-04-17T07:17:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.287+0000 7f278d600640 1 -- 172.21.15.86:0/2240358427 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2790004c50 con 0x55a57310cfa0 2024-04-17T07:17:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.287+0000 7f278d600640 1 -- 172.21.15.86:0/2240358427 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27900be4d0 con 0x55a57310cfa0 2024-04-17T07:17:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.288+0000 7f27a65f6740 1 -- 172.21.15.86:0/2240358427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310cfa0 msgr2=0x55a57310ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.288+0000 7f27a65f6740 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310cfa0 0x55a57310ee50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f27900f8fd0 tx=0x7f2790000bc0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.288+0000 7f27a65f6740 1 -- 172.21.15.86:0/2240358427 shutdown_connections 2024-04-17T07:17:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.288+0000 7f27a65f6740 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a573112b00 0x55a57311e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.288+0000 7f27a65f6740 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a57310d990 0x55a57310f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.288+0000 7f27a65f6740 1 --2- 172.21.15.86:0/2240358427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310cfa0 0x55a57310ee50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.288+0000 7f27a65f6740 1 -- 172.21.15.86:0/2240358427 >> 172.21.15.86:0/2240358427 conn(0x55a573108510 msgr2=0x55a5731171d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.288+0000 7f27a65f6740 1 -- 172.21.15.86:0/2240358427 shutdown_connections 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.288+0000 7f27a65f6740 1 -- 172.21.15.86:0/2240358427 wait complete. 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.289+0000 7f27a65f6740 1 -- start start 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.289+0000 7f27a65f6740 1 Processor -- start 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.289+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 0x55a57311bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.289+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a57311c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.289+0000 7f27a65f6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 0x55a57311d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.289+0000 7f27a65f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1ae150 con 0x55a57310d990 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.289+0000 7f27a65f6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1ada80 con 0x55a57310cfa0 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.289+0000 7f27a65f6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1aacb0 con 0x55a573112b00 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.289+0000 7f27a65f6740 10 client.0 _ll_register_callbacks cb 0x7f278800b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f279be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a57311c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f279be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a57311c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38326/0 (socket says 172.21.15.86:38326) 2024-04-17T07:17:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f279be00640 1 -- 172.21.15.86:0/1619276295 learned_addr learned my addr 172.21.15.86:0/1619276295 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f27a1000640 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 0x55a57311bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f279be00640 1 -- 172.21.15.86:0/1619276295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 msgr2=0x55a57311d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f27a1a00640 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 0x55a57311d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f279be00640 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 0x55a57311d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f279be00640 1 -- 172.21.15.86:0/1619276295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 msgr2=0x55a57311bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f279be00640 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 0x55a57311bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f279be00640 1 -- 172.21.15.86:0/1619276295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a572e4f7d0 con 0x55a57310d990 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.290+0000 7f27a1a00640 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 0x55a57311d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.291+0000 7f279be00640 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a57311c120 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2788012900 tx=0x7f2788012930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf3b7d9f58b15bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.291+0000 7f2782a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.291+0000 7f2782a00640 1 -- 172.21.15.86:0/1619276295 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788011cb0 con 0x55a57310d990 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.291+0000 7f2782a00640 1 -- 172.21.15.86:0/1619276295 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2788005070 con 0x55a57310d990 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.291+0000 7f27a65f6740 1 -- 172.21.15.86:0/1619276295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2790004c50 con 0x55a57310d990 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.291+0000 7f2782a00640 1 -- 172.21.15.86:0/1619276295 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f278800b540 con 0x55a57310d990 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.291+0000 7f27a65f6740 10 client.16305 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.291+0000 7f27a65f6740 1 -- 172.21.15.86:0/1619276295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a57311e810 con 0x55a57310d990 2024-04-17T07:17:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.291+0000 7f2782a00640 1 -- 172.21.15.86:0/1619276295 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2790004c50 con 0x55a57310d990 2024-04-17T07:17:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.292+0000 7f2782a00640 1 -- 172.21.15.86:0/1619276295 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2788030030 con 0x55a57310d990 2024-04-17T07:17:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.292+0000 7f27a65f6740 10 client.16305 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.292+0000 7f27a65f6740 10 client.16305 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:17:25.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.292+0000 7f27a65f6740 20 client.16305 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.292+0000 7f27a65f6740 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55a5731897b0 0x55a57318bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.292+0000 7f27a65f6740 4 client.16305 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.292+0000 7f27a65f6740 1 -- 172.21.15.86:0/1619276295 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a57307a8b0 con 0x55a5731897b0 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.292+0000 7f27a1000640 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55a5731897b0 0x55a57318bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.293+0000 7f27a1000640 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55a5731897b0 0x55a57318bbd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd078d160164e00a server_cookie=ead4370f79889bfe in_seq=0 out_seq=0 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.293+0000 7f2782a00640 10 client.16305 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f2782a00640 1 -- 172.21.15.86:0/1619276295 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a57307a8b0 con 0x55a5731897b0 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f2782a00640 10 client.16305 handle_command_reply: tid=0 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 client.16305 shutdown 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 -- 172.21.15.86:0/1619276295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 msgr2=0x55a57311c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a57311c120 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2788012900 tx=0x7f2788012930 comp rx=0 tx=0).stop 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 -- 172.21.15.86:0/1619276295 shutdown_connections 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55a5731897b0 0x55a57318bbd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a573112b00 0x55a57311d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a57310d990 0x55a57311c120 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 --2- 172.21.15.86:0/1619276295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a57310cfa0 0x55a57311bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 -- 172.21.15.86:0/1619276295 >> 172.21.15.86:0/1619276295 conn(0x55a573108510 msgr2=0x55a5731171b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f2782a00640 0 client.16305 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:17:25.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 -- 172.21.15.86:0/1619276295 shutdown_connections 2024-04-17T07:17:25.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.294+0000 7f27a65f6740 1 -- 172.21.15.86:0/1619276295 wait complete. 2024-04-17T07:17:25.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a65f6740 20 client.16305 trim_cache size 0 max 16384 2024-04-17T07:17:25.296 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a3a00640 1 -- 172.21.15.86:0/3004209431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f27840a3c30 msgr2=0x7f27840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a3a00640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f27840a3c30 0x7f27840a60f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55a57310e1f0 tx=0x7f27940078f0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a3a00640 1 -- 172.21.15.86:0/3004209431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279c101fb0 msgr2=0x7f279c075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a3a00640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279c101fb0 0x7f279c075d10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2790005ab0 tx=0x7f27900042f0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a3a00640 1 -- 172.21.15.86:0/3004209431 shutdown_connections 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a3a00640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f27840a3c30 0x7f27840a60f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a3a00640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f279c103b60 0x7f279c074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a3a00640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f279c1031a0 0x7f279c074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a3a00640 1 --2- 172.21.15.86:0/3004209431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f279c101fb0 0x7f279c075d10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.295+0000 7f27a3a00640 1 -- 172.21.15.86:0/3004209431 >> 172.21.15.86:0/3004209431 conn(0x7f279c0fd540 msgr2=0x7f279c106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.296+0000 7f27a3a00640 1 -- 172.21.15.86:0/3004209431 shutdown_connections 2024-04-17T07:17:25.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.296+0000 7f27a3a00640 1 -- 172.21.15.86:0/3004209431 wait complete. 2024-04-17T07:17:25.311 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:17:25.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.530+0000 7f3fbe800640 1 -- 172.21.15.86:0/1422097011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fb8105c50 msgr2=0x7f3fb810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.530+0000 7f3fbe800640 1 --2- 172.21.15.86:0/1422097011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fb8105c50 0x7f3fb810c4e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009ca0 tx=0x7f3fb002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.530+0000 7f3fbe800640 1 -- 172.21.15.86:0/1422097011 shutdown_connections 2024-04-17T07:17:25.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.530+0000 7f3fbe800640 1 --2- 172.21.15.86:0/1422097011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fb8105c50 0x7f3fb810c4e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.530+0000 7f3fbe800640 1 --2- 172.21.15.86:0/1422097011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb8105290 0x7f3fb8105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.530+0000 7f3fbe800640 1 --2- 172.21.15.86:0/1422097011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb81040a0 0x7f3fb81044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.530+0000 7f3fbe800640 1 -- 172.21.15.86:0/1422097011 >> 172.21.15.86:0/1422097011 conn(0x7f3fb80ff610 msgr2=0x7f3fb8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.530+0000 7f3fbe800640 1 -- 172.21.15.86:0/1422097011 shutdown_connections 2024-04-17T07:17:25.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.530+0000 7f3fbe800640 1 -- 172.21.15.86:0/1422097011 wait complete. 2024-04-17T07:17:25.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.531+0000 7f3fbe800640 1 Processor -- start 2024-04-17T07:17:25.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.531+0000 7f3fbe800640 1 -- start start 2024-04-17T07:17:25.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbe800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb81040a0 0x7f3fb819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbe800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8105290 0x7f3fb819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fb8105c50 0x7f3fb81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbe800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb810fba0 con 0x7f3fb8105c50 2024-04-17T07:17:25.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbe800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb810fa20 con 0x7f3fb8105290 2024-04-17T07:17:25.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbe800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb810fd20 con 0x7f3fb81040a0 2024-04-17T07:17:25.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fb8105c50 0x7f3fb81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fb8105c50 0x7f3fb81a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38354/0 (socket says 172.21.15.86:38354) 2024-04-17T07:17:25.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbde00640 1 -- 172.21.15.86:0/1359776973 learned_addr learned my addr 172.21.15.86:0/1359776973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbd400640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb81040a0 0x7f3fb819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbca00640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8105290 0x7f3fb819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbde00640 1 -- 172.21.15.86:0/1359776973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb81040a0 msgr2=0x7f3fb819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbde00640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb81040a0 0x7f3fb819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbde00640 1 -- 172.21.15.86:0/1359776973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8105290 msgr2=0x7f3fb819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbde00640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8105290 0x7f3fb819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbde00640 1 -- 172.21.15.86:0/1359776973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb81a1dd0 con 0x7f3fb8105c50 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.532+0000 7f3fbca00640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8105290 0x7f3fb819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.533+0000 7f3fbde00640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fb8105c50 0x7f3fb81a1760 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0032380 tx=0x7f3fb0032960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.533+0000 7f3fb6000640 1 -- 172.21.15.86:0/1359776973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0041020 con 0x7f3fb8105c50 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.533+0000 7f3fb6000640 1 -- 172.21.15.86:0/1359776973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb0003e20 con 0x7f3fb8105c50 2024-04-17T07:17:25.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.533+0000 7f3fbe800640 1 -- 172.21.15.86:0/1359776973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb81a2060 con 0x7f3fb8105c50 2024-04-17T07:17:25.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.533+0000 7f3fb6000640 1 -- 172.21.15.86:0/1359776973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0039040 con 0x7f3fb8105c50 2024-04-17T07:17:25.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.533+0000 7f3fbe800640 1 -- 172.21.15.86:0/1359776973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb81a2550 con 0x7f3fb8105c50 2024-04-17T07:17:25.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.537+0000 7f3fc2b92740 1 Processor -- start 2024-04-17T07:17:25.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.537+0000 7f3fb6000640 1 -- 172.21.15.86:0/1359776973 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3fb0044020 con 0x7f3fb8105c50 2024-04-17T07:17:25.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.538+0000 7f3fc2b92740 1 -- start start 2024-04-17T07:17:25.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.538+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064160 0x55b09d064580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.538+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064b50 0x55b09d069650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.538+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d06c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.538+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b09cfca020 con 0x55b09d064b50 2024-04-17T07:17:25.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.538+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b09cf70460 con 0x55b09d064160 2024-04-17T07:17:25.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.538+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b09cfc5fe0 con 0x55b09d069cc0 2024-04-17T07:17:25.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.538+0000 7f3fbd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064b50 0x55b09d069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.539+0000 7f3fbd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064b50 0x55b09d069650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38370/0 (socket says 172.21.15.86:38370) 2024-04-17T07:17:25.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.539+0000 7f3fbd400640 1 -- 172.21.15.86:0/2305340293 learned_addr learned my addr 172.21.15.86:0/2305340293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.539+0000 7f3fbca00640 1 --2- 172.21.15.86:0/2305340293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064160 0x55b09d064580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.539+0000 7f3fbca00640 1 --2- 172.21.15.86:0/2305340293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d06c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.539+0000 7f3fbd400640 1 -- 172.21.15.86:0/2305340293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 msgr2=0x55b09d06c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.539+0000 7f3fbd400640 1 --2- 172.21.15.86:0/2305340293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d06c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.539+0000 7f3fbd400640 1 -- 172.21.15.86:0/2305340293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064160 msgr2=0x55b09d064580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.539+0000 7f3fbd400640 1 --2- 172.21.15.86:0/2305340293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064160 0x55b09d064580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.539+0000 7f3fbd400640 1 -- 172.21.15.86:0/2305340293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b09cf78040 con 0x55b09d064b50 2024-04-17T07:17:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.539+0000 7f3fb6000640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa00a3bb0 0x7f3fa00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.540+0000 7f3fbd400640 1 --2- 172.21.15.86:0/2305340293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064b50 0x55b09d069650 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00cc40 tx=0x7f3fac00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7b427784a6c812 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.540+0000 7f3fb6000640 1 -- 172.21.15.86:0/1359776973 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3fb00f2c10 con 0x7f3fb8105c50 2024-04-17T07:17:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.540+0000 7f3fbca00640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa00a3bb0 0x7f3fa00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.540+0000 7f3faa600640 1 -- 172.21.15.86:0/2305340293 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac014070 con 0x55b09d064b50 2024-04-17T07:17:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.540+0000 7f3faa600640 1 -- 172.21.15.86:0/2305340293 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fac0054d0 con 0x55b09d064b50 2024-04-17T07:17:25.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.540+0000 7f3faa600640 1 -- 172.21.15.86:0/2305340293 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac004cf0 con 0x55b09d064b50 2024-04-17T07:17:25.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.541+0000 7f3fbca00640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa00a3bb0 0x7f3fa00a6070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4005fd0 tx=0x7f3fa400d360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.541+0000 7f3fc2b92740 1 -- 172.21.15.86:0/2305340293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064b50 msgr2=0x55b09d069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.541+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/2305340293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064b50 0x55b09d069650 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00cc40 tx=0x7f3fac00e720 comp rx=0 tx=0).stop 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.541+0000 7f3fc2b92740 1 -- 172.21.15.86:0/2305340293 shutdown_connections 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.541+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/2305340293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d06c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.541+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/2305340293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064b50 0x55b09d069650 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.541+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/2305340293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064160 0x55b09d064580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.541+0000 7f3fc2b92740 1 -- 172.21.15.86:0/2305340293 >> 172.21.15.86:0/2305340293 conn(0x55b09d05f6d0 msgr2=0x55b09d05fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.541+0000 7f3fc2b92740 1 -- 172.21.15.86:0/2305340293 shutdown_connections 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.541+0000 7f3fc2b92740 1 -- 172.21.15.86:0/2305340293 wait complete. 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.542+0000 7f3fc2b92740 1 -- start start 2024-04-17T07:17:25.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.542+0000 7f3fc2b92740 1 Processor -- start 2024-04-17T07:17:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d06e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 0x55b09d06ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d075c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b09d06cc70 con 0x55b09d064160 2024-04-17T07:17:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b09cf83830 con 0x55b09d064b50 2024-04-17T07:17:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b09d06cdf0 con 0x55b09d069cc0 2024-04-17T07:17:25.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fc2b92740 10 client.0 _ll_register_callbacks cb 0x55b09cf51fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fbd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d06e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fbd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d06e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38378/0 (socket says 172.21.15.86:38378) 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fbd400640 1 -- 172.21.15.86:0/752148985 learned_addr learned my addr 172.21.15.86:0/752148985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.543+0000 7f3fbde00640 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d075c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.544+0000 7f3fbca00640 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 0x55b09d06ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.544+0000 7f3fbd400640 1 -- 172.21.15.86:0/752148985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 msgr2=0x55b09d075c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.544+0000 7f3fbd400640 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d075c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.544+0000 7f3fbd400640 1 -- 172.21.15.86:0/752148985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 msgr2=0x55b09d06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.544+0000 7f3fbd400640 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 0x55b09d06ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.544+0000 7f3fbd400640 1 -- 172.21.15.86:0/752148985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09cf87920 con 0x55b09d064160 2024-04-17T07:17:25.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.544+0000 7f3fbd400640 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d06e700 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3fac017000 tx=0x7f3fac005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d5f82a39d7d1c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.544+0000 7f3f9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.544+0000 7f3f9ea00640 1 -- 172.21.15.86:0/752148985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac014040 con 0x55b09d064160 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.545+0000 7f3f9ea00640 1 -- 172.21.15.86:0/752148985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3fac010430 con 0x55b09d064160 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.545+0000 7f3fc2b92740 1 -- 172.21.15.86:0/752148985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fac0054d0 con 0x55b09d064160 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.545+0000 7f3f9ea00640 1 -- 172.21.15.86:0/752148985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fac010630 con 0x55b09d064160 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.545+0000 7f3fc2b92740 10 client.16317 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.545+0000 7f3fc2b92740 1 -- 172.21.15.86:0/752148985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b09cf557f0 con 0x55b09d064160 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.545+0000 7f3f9ea00640 1 -- 172.21.15.86:0/752148985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fac0054d0 con 0x55b09d064160 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.546+0000 7f3f9ea00640 1 -- 172.21.15.86:0/752148985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3fac024420 con 0x55b09d064160 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.546+0000 7f3fc2b92740 10 client.16317 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.546+0000 7f3fc2b92740 10 client.16317 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.546+0000 7f3fc2b92740 20 client.16317 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.546+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b09d079f40 0x55b09d07c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.546+0000 7f3fc2b92740 4 client.16317 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:25.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.546+0000 7f3fc2b92740 1 -- 172.21.15.86:0/752148985 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b09cfd7a40 con 0x55b09d079f40 2024-04-17T07:17:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.547+0000 7f3fbca00640 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b09d079f40 0x55b09d07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.548+0000 7f3fbca00640 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b09d079f40 0x55b09d07c390 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75b6e35f9d395918 server_cookie=e29806164b12beec in_seq=0 out_seq=0 2024-04-17T07:17:25.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.548+0000 7f3f9ea00640 10 client.16317 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:17:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.549+0000 7f3f9ea00640 1 -- 172.21.15.86:0/752148985 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b09cfd7a40 con 0x55b09d079f40 2024-04-17T07:17:25.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.549+0000 7f3f9ea00640 10 client.16317 handle_command_reply: tid=0 2024-04-17T07:17:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.550+0000 7f3fc2b92740 1 client.16317 shutdown 2024-04-17T07:17:25.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.550+0000 7f3fc2b92740 1 -- 172.21.15.86:0/752148985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 msgr2=0x55b09d06e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.550+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d06e700 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3fac017000 tx=0x7f3fac005230 comp rx=0 tx=0).stop 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.550+0000 7f3fc2b92740 1 -- 172.21.15.86:0/752148985 shutdown_connections 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.550+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b09d079f40 0x55b09d07c390 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.550+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d075c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.550+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 0x55b09d06ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.550+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/752148985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d06e700 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.550+0000 7f3fc2b92740 1 -- 172.21.15.86:0/752148985 >> 172.21.15.86:0/752148985 conn(0x55b09d05f6d0 msgr2=0x55b09d05e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.550+0000 7f3f9ea00640 0 client.16317 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.551+0000 7f3fc2b92740 1 -- 172.21.15.86:0/752148985 shutdown_connections 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.551+0000 7f3fc2b92740 1 -- 172.21.15.86:0/752148985 wait complete. 2024-04-17T07:17:25.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.551+0000 7f3fc2b92740 20 client.16317 trim_cache size 0 max 16384 2024-04-17T07:17:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fc2b92740 1 Processor -- start 2024-04-17T07:17:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fc2b92740 1 -- start start 2024-04-17T07:17:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d063040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 0x55b09d075ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81aca20 con 0x55b09d064160 2024-04-17T07:17:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81ae560 con 0x55b09d064b50 2024-04-17T07:17:25.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81ad0f0 con 0x55b09d069cc0 2024-04-17T07:17:25.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fbd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d063040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fbd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d063040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38386/0 (socket says 172.21.15.86:38386) 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.570+0000 7f3fbd400640 1 -- 172.21.15.86:0/2849116013 learned_addr learned my addr 172.21.15.86:0/2849116013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.571+0000 7f3fbca00640 1 --2- 172.21.15.86:0/2849116013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 0x55b09d075ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.571+0000 7f3fbde00640 1 --2- 172.21.15.86:0/2849116013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.571+0000 7f3fbd400640 1 -- 172.21.15.86:0/2849116013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 msgr2=0x55b09d06dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.571+0000 7f3fbd400640 1 --2- 172.21.15.86:0/2849116013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d06dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.571+0000 7f3fbd400640 1 -- 172.21.15.86:0/2849116013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 msgr2=0x55b09d075ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.571+0000 7f3fbd400640 1 --2- 172.21.15.86:0/2849116013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 0x55b09d075ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.571+0000 7f3fbd400640 1 -- 172.21.15.86:0/2849116013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b09ccdc3f0 con 0x55b09d064160 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.571+0000 7f3fbca00640 1 --2- 172.21.15.86:0/2849116013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 0x55b09d075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.571+0000 7f3fbd400640 1 --2- 172.21.15.86:0/2849116013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d063040 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c720 tx=0x7f3fac00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f3febfcbe49a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.572+0000 7f3fa9c00640 1 -- 172.21.15.86:0/2849116013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac014030 con 0x55b09d064160 2024-04-17T07:17:25.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.572+0000 7f3fa9c00640 1 -- 172.21.15.86:0/2849116013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fac010430 con 0x55b09d064160 2024-04-17T07:17:25.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.572+0000 7f3fa9c00640 1 -- 172.21.15.86:0/2849116013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac0048a0 con 0x55b09d064160 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.573+0000 7f3fc2b92740 1 -- 172.21.15.86:0/2849116013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 msgr2=0x55b09d063040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.573+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/2849116013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d063040 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c720 tx=0x7f3fac00ec20 comp rx=0 tx=0).stop 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.573+0000 7f3fc2b92740 1 -- 172.21.15.86:0/2849116013 shutdown_connections 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.573+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/2849116013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d069cc0 0x55b09d06dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.573+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/2849116013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d064b50 0x55b09d075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.573+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/2849116013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d063040 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.573+0000 7f3fc2b92740 1 -- 172.21.15.86:0/2849116013 >> 172.21.15.86:0/2849116013 conn(0x55b09d05f6d0 msgr2=0x55b09d062930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.573+0000 7f3fc2b92740 1 -- 172.21.15.86:0/2849116013 shutdown_connections 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.573+0000 7f3fc2b92740 1 -- 172.21.15.86:0/2849116013 wait complete. 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.574+0000 7f3fc2b92740 1 -- start start 2024-04-17T07:17:25.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.574+0000 7f3fc2b92740 1 Processor -- start 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.574+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d072a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.574+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d064b50 0x55b09d072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.574+0000 7f3fc2b92740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d069cc0 0x55b09d0de500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.574+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81ade90 con 0x55b09d064160 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.574+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81ad7c0 con 0x55b09d069cc0 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.574+0000 7f3fc2b92740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb81aa9f0 con 0x55b09d064b50 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.574+0000 7f3fc2b92740 10 client.0 _ll_register_callbacks cb 0x7f3fac004ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.575+0000 7f3fbd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.575+0000 7f3fbd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d072a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38400/0 (socket says 172.21.15.86:38400) 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.575+0000 7f3fbd400640 1 -- 172.21.15.86:0/1974776916 learned_addr learned my addr 172.21.15.86:0/1974776916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.575+0000 7f3fbca00640 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d064b50 0x55b09d072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.575+0000 7f3fbd400640 1 -- 172.21.15.86:0/1974776916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d064b50 msgr2=0x55b09d072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.575+0000 7f3fbd400640 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d064b50 0x55b09d072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.575+0000 7f3fbd400640 1 -- 172.21.15.86:0/1974776916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d069cc0 msgr2=0x55b09d0de500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.575+0000 7f3fbd400640 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d069cc0 0x55b09d0de500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.575+0000 7f3fbd400640 1 -- 172.21.15.86:0/1974776916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09cda67d0 con 0x55b09d064160 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.575+0000 7f3fbca00640 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d064b50 0x55b09d072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.576+0000 7f3fbd400640 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d072a70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c9a0 tx=0x7f3fac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed056d4cca8382df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.576+0000 7f3f9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.576+0000 7f3f9ea00640 1 -- 172.21.15.86:0/1974776916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac014040 con 0x55b09d064160 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.576+0000 7f3f9ea00640 1 -- 172.21.15.86:0/1974776916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3fac005620 con 0x55b09d064160 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.576+0000 7f3fc2b92740 1 -- 172.21.15.86:0/1974776916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fac010430 con 0x55b09d064160 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.576+0000 7f3f9ea00640 1 -- 172.21.15.86:0/1974776916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fac0107d0 con 0x55b09d064160 2024-04-17T07:17:25.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.576+0000 7f3fc2b92740 10 client.16323 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.576+0000 7f3fc2b92740 1 -- 172.21.15.86:0/1974776916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b09d0df8c0 con 0x55b09d064160 2024-04-17T07:17:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.577+0000 7f3f9ea00640 1 -- 172.21.15.86:0/1974776916 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fac010430 con 0x55b09d064160 2024-04-17T07:17:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.577+0000 7f3f9ea00640 1 -- 172.21.15.86:0/1974776916 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3fac018940 con 0x55b09d064160 2024-04-17T07:17:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.577+0000 7f3fc2b92740 10 client.16323 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.577+0000 7f3fc2b92740 10 client.16323 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:17:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.577+0000 7f3fc2b92740 20 client.16323 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.577+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b09d0e0080 0x55b09d0e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.577+0000 7f3fc2b92740 4 client.16323 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:17:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.577+0000 7f3fc2b92740 1 -- 172.21.15.86:0/1974776916 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b09cfd18b0 con 0x55b09d0e0080 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.578+0000 7f3fbca00640 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b09d0e0080 0x55b09d0e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.579+0000 7f3f9ea00640 10 client.16323 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.579+0000 7f3fbca00640 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b09d0e0080 0x55b09d0e24a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cccab09eeafddac server_cookie=2f8d08c2ea410571 in_seq=0 out_seq=0 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.579+0000 7f3f9ea00640 1 -- 172.21.15.86:0/1974776916 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b09cfd18b0 con 0x55b09d0e0080 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.579+0000 7f3f9ea00640 10 client.16323 handle_command_reply: tid=0 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.579+0000 7f3fc2b92740 1 client.16323 shutdown 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 1 -- 172.21.15.86:0/1974776916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 msgr2=0x55b09d072a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d072a70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c9a0 tx=0x7f3fac0046c0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 1 -- 172.21.15.86:0/1974776916 shutdown_connections 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b09d0e0080 0x55b09d0e24a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b09d069cc0 0x55b09d0de500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b09d064b50 0x55b09d072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 1 --2- 172.21.15.86:0/1974776916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b09d064160 0x55b09d072a70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 1 -- 172.21.15.86:0/1974776916 >> 172.21.15.86:0/1974776916 conn(0x55b09d05f6d0 msgr2=0x55b09d0631c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3f9ea00640 0 client.16323 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:17:25.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 1 -- 172.21.15.86:0/1974776916 shutdown_connections 2024-04-17T07:17:25.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 1 -- 172.21.15.86:0/1974776916 wait complete. 2024-04-17T07:17:25.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.580+0000 7f3fc2b92740 20 client.16323 trim_cache size 0 max 16384 2024-04-17T07:17:25.581 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:17:25.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 -- 172.21.15.86:0/1359776973 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa00a3bb0 msgr2=0x7f3fa00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa00a3bb0 0x7f3fa00a6070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4005fd0 tx=0x7f3fa400d360 comp rx=0 tx=0).stop 2024-04-17T07:17:25.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 -- 172.21.15.86:0/1359776973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fb8105c50 msgr2=0x7f3fb81a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fb8105c50 0x7f3fb81a1760 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0032380 tx=0x7f3fb0032960 comp rx=0 tx=0).stop 2024-04-17T07:17:25.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 -- 172.21.15.86:0/1359776973 shutdown_connections 2024-04-17T07:17:25.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fa00a3bb0 0x7f3fa00a6070 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fb8105c50 0x7f3fb81a1760 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fb8105290 0x7f3fb819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 --2- 172.21.15.86:0/1359776973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fb81040a0 0x7f3fb819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 -- 172.21.15.86:0/1359776973 >> 172.21.15.86:0/1359776973 conn(0x7f3fb80ff610 msgr2=0x7f3fb8108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.581+0000 7f3fbe800640 1 -- 172.21.15.86:0/1359776973 shutdown_connections 2024-04-17T07:17:25.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.582+0000 7f3fbe800640 1 -- 172.21.15.86:0/1359776973 wait complete. 2024-04-17T07:17:25.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:17:25.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.831+0000 7f50c1600640 1 -- 172.21.15.86:0/2813225563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50bc1053b0 msgr2=0x7f50bc105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.831+0000 7f50c1600640 1 --2- 172.21.15.86:0/2813225563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50bc1053b0 0x7f50bc105830 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009900 tx=0x7f50a802bc30 comp rx=0 tx=0).stop 2024-04-17T07:17:25.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.831+0000 7f50c1600640 1 -- 172.21.15.86:0/2813225563 shutdown_connections 2024-04-17T07:17:25.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.831+0000 7f50c1600640 1 --2- 172.21.15.86:0/2813225563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50bc105d70 0x7f50bc10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.831+0000 7f50c1600640 1 --2- 172.21.15.86:0/2813225563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50bc1053b0 0x7f50bc105830 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.831+0000 7f50c1600640 1 --2- 172.21.15.86:0/2813225563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1041c0 0x7f50bc1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.831+0000 7f50c1600640 1 -- 172.21.15.86:0/2813225563 >> 172.21.15.86:0/2813225563 conn(0x7f50bc0ff710 msgr2=0x7f50bc101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.831+0000 7f50c1600640 1 -- 172.21.15.86:0/2813225563 shutdown_connections 2024-04-17T07:17:25.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.831+0000 7f50c1600640 1 -- 172.21.15.86:0/2813225563 wait complete. 2024-04-17T07:17:25.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.832+0000 7f50c1600640 1 Processor -- start 2024-04-17T07:17:25.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.833+0000 7f50c1600640 1 -- start start 2024-04-17T07:17:25.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.833+0000 7f50c1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50bc1041c0 0x7f50bc19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.833+0000 7f50c1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1053b0 0x7f50bc19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.833+0000 7f50c1600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50bc105d70 0x7f50bc1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.833+0000 7f50c1600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc10fd00 con 0x7f50bc1041c0 2024-04-17T07:17:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.833+0000 7f50c1600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc10fb80 con 0x7f50bc105d70 2024-04-17T07:17:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.833+0000 7f50c1600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc10fe80 con 0x7f50bc1053b0 2024-04-17T07:17:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.833+0000 7f50baa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50bc1041c0 0x7f50bc19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.833+0000 7f50baa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50bc1041c0 0x7f50bc19a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38424/0 (socket says 172.21.15.86:38424) 2024-04-17T07:17:25.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.833+0000 7f50baa00640 1 -- 172.21.15.86:0/3540516557 learned_addr learned my addr 172.21.15.86:0/3540516557 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.834+0000 7f50ba000640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1053b0 0x7f50bc19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.834+0000 7f50bb400640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50bc105d70 0x7f50bc1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.834+0000 7f50baa00640 1 -- 172.21.15.86:0/3540516557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1053b0 msgr2=0x7f50bc19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.834+0000 7f50baa00640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1053b0 0x7f50bc19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.834+0000 7f50baa00640 1 -- 172.21.15.86:0/3540516557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50bc105d70 msgr2=0x7f50bc1a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.834+0000 7f50baa00640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50bc105d70 0x7f50bc1a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.834+0000 7f50baa00640 1 -- 172.21.15.86:0/3540516557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc1a1fe0 con 0x7f50bc1041c0 2024-04-17T07:17:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.834+0000 7f50baa00640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50bc1041c0 0x7f50bc19a3c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00d950 tx=0x7f50ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.834+0000 7f50b3400640 1 -- 172.21.15.86:0/3540516557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac017070 con 0x7f50bc1041c0 2024-04-17T07:17:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.835+0000 7f50b3400640 1 -- 172.21.15.86:0/3540516557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50ac009c30 con 0x7f50bc1041c0 2024-04-17T07:17:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.835+0000 7f50c1600640 1 -- 172.21.15.86:0/3540516557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc1a22d0 con 0x7f50bc1041c0 2024-04-17T07:17:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.835+0000 7f50b3400640 1 -- 172.21.15.86:0/3540516557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac00f620 con 0x7f50bc1041c0 2024-04-17T07:17:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.835+0000 7f50c1600640 1 -- 172.21.15.86:0/3540516557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc1a2820 con 0x7f50bc1041c0 2024-04-17T07:17:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.839+0000 7f50c4386740 1 Processor -- start 2024-04-17T07:17:25.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.839+0000 7f50c4386740 1 -- start start 2024-04-17T07:17:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.839+0000 7f50c4386740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451dfa0 0x55819451e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.839+0000 7f50c4386740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451e990 0x558194523490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.839+0000 7f50c4386740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 0x558194525f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.839+0000 7f50c4386740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558194484020 con 0x55819451dfa0 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.839+0000 7f50c4386740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55819442a460 con 0x558194523b00 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.839+0000 7f50c4386740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55819447ffe0 con 0x55819451e990 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50b3400640 1 -- 172.21.15.86:0/3540516557 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f50ac01a070 con 0x7f50bc1041c0 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451dfa0 0x55819451e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451dfa0 0x55819451e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38432/0 (socket says 172.21.15.86:38432) 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50bb400640 1 -- 172.21.15.86:0/206179132 learned_addr learned my addr 172.21.15.86:0/206179132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50baa00640 1 --2- 172.21.15.86:0/206179132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451e990 0x558194523490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50bb400640 1 -- 172.21.15.86:0/206179132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451e990 msgr2=0x558194523490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50bb400640 1 --2- 172.21.15.86:0/206179132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451e990 0x558194523490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50ba000640 1 --2- 172.21.15.86:0/206179132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 0x558194525f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50bb400640 1 -- 172.21.15.86:0/206179132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 msgr2=0x558194525f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50bb400640 1 --2- 172.21.15.86:0/206179132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 0x558194525f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50bb400640 1 -- 172.21.15.86:0/206179132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558194432040 con 0x55819451dfa0 2024-04-17T07:17:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.840+0000 7f50ba000640 1 --2- 172.21.15.86:0/206179132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 0x558194525f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.841+0000 7f50bb400640 1 --2- 172.21.15.86:0/206179132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451dfa0 0x55819451e3c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009ee0 tx=0x7f50b4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce39de2b398e708 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.841+0000 7f50a7400640 1 -- 172.21.15.86:0/206179132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4016070 con 0x55819451dfa0 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.841+0000 7f50a7400640 1 -- 172.21.15.86:0/206179132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50b4004d20 con 0x55819451dfa0 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.841+0000 7f50b3400640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50a00a3cb0 0x7f50a00a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.841+0000 7f50a7400640 1 -- 172.21.15.86:0/206179132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4012720 con 0x55819451dfa0 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.842+0000 7f50b3400640 1 -- 172.21.15.86:0/3540516557 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f50ac0d02b0 con 0x7f50bc1041c0 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.842+0000 7f50baa00640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50a00a3cb0 0x7f50a00a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.842+0000 7f50c4386740 1 -- 172.21.15.86:0/206179132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451dfa0 msgr2=0x55819451e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.842+0000 7f50c4386740 1 --2- 172.21.15.86:0/206179132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451dfa0 0x55819451e3c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009ee0 tx=0x7f50b4010740 comp rx=0 tx=0).stop 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.842+0000 7f50baa00640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50a00a3cb0 0x7f50a00a6170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f50ac09beb0 tx=0x7f50ac0d37b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.843+0000 7f50c4386740 1 -- 172.21.15.86:0/206179132 shutdown_connections 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.843+0000 7f50c4386740 1 --2- 172.21.15.86:0/206179132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 0x558194525f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.843+0000 7f50c4386740 1 --2- 172.21.15.86:0/206179132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451e990 0x558194523490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.843+0000 7f50c4386740 1 --2- 172.21.15.86:0/206179132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451dfa0 0x55819451e3c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.843+0000 7f50c4386740 1 -- 172.21.15.86:0/206179132 >> 172.21.15.86:0/206179132 conn(0x558194519510 msgr2=0x558194519910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.843+0000 7f50c4386740 1 -- 172.21.15.86:0/206179132 shutdown_connections 2024-04-17T07:17:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.843+0000 7f50c4386740 1 -- 172.21.15.86:0/206179132 wait complete. 2024-04-17T07:17:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.843+0000 7f50c4386740 1 -- start start 2024-04-17T07:17:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.844+0000 7f50c4386740 1 Processor -- start 2024-04-17T07:17:25.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.844+0000 7f50c4386740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 0x558194528540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.844+0000 7f50c4386740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x558194528a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.844+0000 7f50c4386740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 0x55819452fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.844+0000 7f50c4386740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558194526b00 con 0x55819451e990 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.844+0000 7f50c4386740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55819443d830 con 0x558194523b00 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.844+0000 7f50c4386740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558194526c80 con 0x55819451dfa0 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.844+0000 7f50c4386740 10 client.0 _ll_register_callbacks cb 0x55819440bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.844+0000 7f50ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x558194528a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.845+0000 7f50ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x558194528a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38442/0 (socket says 172.21.15.86:38442) 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.845+0000 7f50ba000640 1 -- 172.21.15.86:0/3574066456 learned_addr learned my addr 172.21.15.86:0/3574066456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.845+0000 7f50bb400640 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 0x558194528540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.845+0000 7f50bb400640 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 0x55819452fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.845+0000 7f50ba000640 1 -- 172.21.15.86:0/3574066456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 msgr2=0x558194528540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.845+0000 7f50ba000640 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 0x558194528540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.845+0000 7f50ba000640 1 -- 172.21.15.86:0/3574066456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 msgr2=0x55819452fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.845+0000 7f50ba000640 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 0x55819452fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.845+0000 7f50ba000640 1 -- 172.21.15.86:0/3574066456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558194441920 con 0x55819451e990 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.845+0000 7f50bb400640 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 0x558194528540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.846+0000 7f50ba000640 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x558194528a80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f50a8005ab0 tx=0x7f50a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f98bfbc0c45a686 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.846+0000 7f509be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.846+0000 7f509be00640 1 -- 172.21.15.86:0/3574066456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8044070 con 0x55819451e990 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.846+0000 7f509be00640 1 -- 172.21.15.86:0/3574066456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f50a8005450 con 0x55819451e990 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.846+0000 7f50c4386740 1 -- 172.21.15.86:0/3574066456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b4004d20 con 0x55819451e990 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f509be00640 1 -- 172.21.15.86:0/3574066456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50a8005d50 con 0x55819451e990 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f50c4386740 10 client.16335 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f50c4386740 1 -- 172.21.15.86:0/3574066456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55819440f7f0 con 0x55819451e990 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f509be00640 1 -- 172.21.15.86:0/3574066456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50b4004d20 con 0x55819451e990 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f509be00640 1 -- 172.21.15.86:0/3574066456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f50a80037a0 con 0x55819451e990 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f50c4386740 10 client.16335 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f50c4386740 10 client.16335 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:17:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f50c4386740 20 client.16335 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f50c4386740 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558194533f20 0x558194536340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f50c4386740 4 client.16335 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.847+0000 7f50c4386740 1 -- 172.21.15.86:0/3574066456 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558194491a40 con 0x558194533f20 2024-04-17T07:17:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.848+0000 7f50bb400640 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558194533f20 0x558194536340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.849+0000 7f50bb400640 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558194533f20 0x558194536340 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4e6420a7e5d2cdf server_cookie=e370137f98639cc9 in_seq=0 out_seq=0 2024-04-17T07:17:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.849+0000 7f509be00640 10 client.16335 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:17:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.851+0000 7f509be00640 1 -- 172.21.15.86:0/3574066456 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558194491a40 con 0x558194533f20 2024-04-17T07:17:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.851+0000 7f509be00640 10 client.16335 handle_command_reply: tid=0 2024-04-17T07:17:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.851+0000 7f50c4386740 1 client.16335 shutdown 2024-04-17T07:17:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.852+0000 7f50c4386740 1 -- 172.21.15.86:0/3574066456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 msgr2=0x558194528a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.852+0000 7f50c4386740 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x558194528a80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f50a8005ab0 tx=0x7f50a80042f0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.852+0000 7f50c4386740 1 -- 172.21.15.86:0/3574066456 shutdown_connections 2024-04-17T07:17:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.852+0000 7f50c4386740 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558194533f20 0x558194536340 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.852+0000 7f50c4386740 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558194523b00 0x55819452fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.852+0000 7f50c4386740 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x558194528a80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.852+0000 7f50c4386740 1 --2- 172.21.15.86:0/3574066456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 0x558194528540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.852+0000 7f50c4386740 1 -- 172.21.15.86:0/3574066456 >> 172.21.15.86:0/3574066456 conn(0x558194519510 msgr2=0x558194518710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.852+0000 7f509be00640 0 client.16335 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:17:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.852+0000 7f50c4386740 1 -- 172.21.15.86:0/3574066456 shutdown_connections 2024-04-17T07:17:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.853+0000 7f50c4386740 1 -- 172.21.15.86:0/3574066456 wait complete. 2024-04-17T07:17:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.853+0000 7f50c4386740 20 client.16335 trim_cache size 0 max 16384 2024-04-17T07:17:25.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.874+0000 7f50c4386740 1 Processor -- start 2024-04-17T07:17:25.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.874+0000 7f50c4386740 1 -- start start 2024-04-17T07:17:25.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.874+0000 7f50c4386740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 0x55819451ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.874+0000 7f50c4386740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55819451e990 0x558194520460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.874+0000 7f50c4386740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558194523b00 0x55819452fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.874+0000 7f50c4386740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1acca0 con 0x558194523b00 2024-04-17T07:17:25.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.874+0000 7f50c4386740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1ae7e0 con 0x55819451e990 2024-04-17T07:17:25.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.874+0000 7f50c4386740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1ad370 con 0x55819451dfa0 2024-04-17T07:17:25.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.875+0000 7f50bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558194523b00 0x55819452fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.875+0000 7f50bb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558194523b00 0x55819452fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38450/0 (socket says 172.21.15.86:38450) 2024-04-17T07:17:25.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.875+0000 7f50bb400640 1 -- 172.21.15.86:0/4208108696 learned_addr learned my addr 172.21.15.86:0/4208108696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.875+0000 7f50bb400640 1 --2- 172.21.15.86:0/4208108696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 0x55819451ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.875+0000 7f50ba000640 1 --2- 172.21.15.86:0/4208108696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55819451e990 0x558194520460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.875+0000 7f50bb400640 1 -- 172.21.15.86:0/4208108696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 msgr2=0x55819451ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.875+0000 7f50bb400640 1 --2- 172.21.15.86:0/4208108696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 0x55819451ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.875+0000 7f50bb400640 1 -- 172.21.15.86:0/4208108696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55819451e990 msgr2=0x558194520460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.875+0000 7f50bb400640 1 --2- 172.21.15.86:0/4208108696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55819451e990 0x558194520460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.875+0000 7f50bb400640 1 -- 172.21.15.86:0/4208108696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581941963f0 con 0x558194523b00 2024-04-17T07:17:25.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.876+0000 7f50bb400640 1 --2- 172.21.15.86:0/4208108696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558194523b00 0x55819452fa10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009fd0 tx=0x7f50b4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12b71acf1bd25ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.876+0000 7f50a6a00640 1 -- 172.21.15.86:0/4208108696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b402e070 con 0x558194523b00 2024-04-17T07:17:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.876+0000 7f50a6a00640 1 -- 172.21.15.86:0/4208108696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50b4028790 con 0x558194523b00 2024-04-17T07:17:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.876+0000 7f50a6a00640 1 -- 172.21.15.86:0/4208108696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4016040 con 0x558194523b00 2024-04-17T07:17:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.877+0000 7f50c4386740 1 -- 172.21.15.86:0/4208108696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558194523b00 msgr2=0x55819452fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.877+0000 7f50c4386740 1 --2- 172.21.15.86:0/4208108696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558194523b00 0x55819452fa10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009fd0 tx=0x7f50b4000f40 comp rx=0 tx=0).stop 2024-04-17T07:17:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.877+0000 7f50c4386740 1 -- 172.21.15.86:0/4208108696 shutdown_connections 2024-04-17T07:17:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.877+0000 7f50c4386740 1 --2- 172.21.15.86:0/4208108696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558194523b00 0x55819452fa10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.877+0000 7f50c4386740 1 --2- 172.21.15.86:0/4208108696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55819451e990 0x558194520460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.877+0000 7f50c4386740 1 --2- 172.21.15.86:0/4208108696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55819451dfa0 0x55819451ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.877+0000 7f50c4386740 1 -- 172.21.15.86:0/4208108696 >> 172.21.15.86:0/4208108696 conn(0x558194519510 msgr2=0x5581945198f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.877+0000 7f50c4386740 1 -- 172.21.15.86:0/4208108696 shutdown_connections 2024-04-17T07:17:25.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.877+0000 7f50c4386740 1 -- 172.21.15.86:0/4208108696 wait complete. 2024-04-17T07:17:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.878+0000 7f50c4386740 1 -- start start 2024-04-17T07:17:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.878+0000 7f50c4386740 1 Processor -- start 2024-04-17T07:17:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.878+0000 7f50c4386740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55819451dfa0 0x55819452ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.878+0000 7f50c4386740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x55819452cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.878+0000 7f50c4386740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558194523b00 0x558194598470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.878+0000 7f50c4386740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1ae110 con 0x55819451e990 2024-04-17T07:17:25.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.878+0000 7f50c4386740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1ada40 con 0x55819451dfa0 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.878+0000 7f50c4386740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1aac70 con 0x558194523b00 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.878+0000 7f50c4386740 10 client.0 _ll_register_callbacks cb 0x7f50a802cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x55819452cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50ba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x55819452cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38452/0 (socket says 172.21.15.86:38452) 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50ba000640 1 -- 172.21.15.86:0/11891191 learned_addr learned my addr 172.21.15.86:0/11891191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50bb400640 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55819451dfa0 0x55819452ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50bb400640 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558194523b00 0x558194598470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50ba000640 1 -- 172.21.15.86:0/11891191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558194523b00 msgr2=0x558194598470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50ba000640 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558194523b00 0x558194598470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50ba000640 1 -- 172.21.15.86:0/11891191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55819451dfa0 msgr2=0x55819452ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50ba000640 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55819451dfa0 0x55819452ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50ba000640 1 -- 172.21.15.86:0/11891191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581942607d0 con 0x55819451e990 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.879+0000 7f50bb400640 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558194523b00 0x558194598470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:25.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.880+0000 7f50ba000640 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x55819452cf70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009900 tx=0x7f50a802c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4541ed6141888ead server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.880+0000 7f509be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.880+0000 7f509be00640 1 -- 172.21.15.86:0/11891191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8044040 con 0x55819451e990 2024-04-17T07:17:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.880+0000 7f509be00640 1 -- 172.21.15.86:0/11891191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f50a8005460 con 0x55819451e990 2024-04-17T07:17:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.880+0000 7f50c4386740 1 -- 172.21.15.86:0/11891191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b4028790 con 0x55819451e990 2024-04-17T07:17:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.880+0000 7f509be00640 1 -- 172.21.15.86:0/11891191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50a80044e0 con 0x55819451e990 2024-04-17T07:17:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.880+0000 7f50c4386740 10 client.16341 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:25.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.880+0000 7f50c4386740 1 -- 172.21.15.86:0/11891191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558194599b60 con 0x55819451e990 2024-04-17T07:17:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.881+0000 7f509be00640 1 -- 172.21.15.86:0/11891191 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50b4028790 con 0x55819451e990 2024-04-17T07:17:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.881+0000 7f509be00640 1 -- 172.21.15.86:0/11891191 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f50a80046c0 con 0x55819451e990 2024-04-17T07:17:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.881+0000 7f50c4386740 10 client.16341 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.881+0000 7f50c4386740 10 client.16341 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:17:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.881+0000 7f50c4386740 20 client.16341 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:25.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.881+0000 7f50c4386740 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55819459a590 0x55819459c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.881+0000 7f50c4386740 4 client.16341 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:17:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.881+0000 7f50c4386740 1 -- 172.21.15.86:0/11891191 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55819448b8b0 con 0x55819459a590 2024-04-17T07:17:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.882+0000 7f50bb400640 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55819459a590 0x55819459c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.883+0000 7f509be00640 10 client.16341 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:17:25.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.883+0000 7f50bb400640 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55819459a590 0x55819459c9b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=499c1ba7a9480705 server_cookie=170b53bd75208691 in_seq=0 out_seq=0 2024-04-17T07:17:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f509be00640 1 -- 172.21.15.86:0/11891191 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55819448b8b0 con 0x55819459a590 2024-04-17T07:17:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f509be00640 10 client.16341 handle_command_reply: tid=0 2024-04-17T07:17:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f50c4386740 1 client.16341 shutdown 2024-04-17T07:17:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f50c4386740 1 -- 172.21.15.86:0/11891191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 msgr2=0x55819452cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f50c4386740 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x55819452cf70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009900 tx=0x7f50a802c2b0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f50c4386740 1 -- 172.21.15.86:0/11891191 shutdown_connections 2024-04-17T07:17:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f50c4386740 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558194523b00 0x558194598470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f50c4386740 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55819451e990 0x55819452cf70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f50c4386740 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55819459a590 0x55819459c9b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f50c4386740 1 --2- 172.21.15.86:0/11891191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55819451dfa0 0x55819452ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f50c4386740 1 -- 172.21.15.86:0/11891191 >> 172.21.15.86:0/11891191 conn(0x558194519510 msgr2=0x55819451c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f509be00640 0 client.16341 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:17:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.884+0000 7f50c4386740 1 -- 172.21.15.86:0/11891191 shutdown_connections 2024-04-17T07:17:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.885+0000 7f50c4386740 1 -- 172.21.15.86:0/11891191 wait complete. 2024-04-17T07:17:25.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.885+0000 7f50c4386740 20 client.16341 trim_cache size 0 max 16384 2024-04-17T07:17:25.885 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:17:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.885+0000 7f50c1600640 1 -- 172.21.15.86:0/3540516557 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50a00a3cb0 msgr2=0x7f50a00a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.885+0000 7f50c1600640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50a00a3cb0 0x7f50a00a6170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f50ac09beb0 tx=0x7f50ac0d37b0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.885+0000 7f50c1600640 1 -- 172.21.15.86:0/3540516557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50bc1041c0 msgr2=0x7f50bc19a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.885+0000 7f50c1600640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50bc1041c0 0x7f50bc19a3c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00d950 tx=0x7f50ac00de10 comp rx=0 tx=0).stop 2024-04-17T07:17:25.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.886+0000 7f50c1600640 1 -- 172.21.15.86:0/3540516557 shutdown_connections 2024-04-17T07:17:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.886+0000 7f50c1600640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f50a00a3cb0 0x7f50a00a6170 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.886+0000 7f50c1600640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50bc105d70 0x7f50bc1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.886+0000 7f50c1600640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50bc1053b0 0x7f50bc19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.886+0000 7f50c1600640 1 --2- 172.21.15.86:0/3540516557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50bc1041c0 0x7f50bc19a3c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.886+0000 7f50c1600640 1 -- 172.21.15.86:0/3540516557 >> 172.21.15.86:0/3540516557 conn(0x7f50bc0ff710 msgr2=0x7f50bc1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.886+0000 7f50c1600640 1 -- 172.21.15.86:0/3540516557 shutdown_connections 2024-04-17T07:17:25.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:25.886+0000 7f50c1600640 1 -- 172.21.15.86:0/3540516557 wait complete. 2024-04-17T07:17:25.901 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:25 smithi086 ceph-mon[31733]: pgmap v532: 129 pgs: 129 active+clean; 436 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 18 op/s 2024-04-17T07:17:25.904 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:17:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.113+0000 7fe473200640 1 -- 172.21.15.86:0/3447361503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c101ea0 msgr2=0x7fe46c1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.113+0000 7fe473200640 1 --2- 172.21.15.86:0/3447361503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c101ea0 0x7fe46c1022c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe460009900 tx=0x7fe46002bc50 comp rx=0 tx=0).stop 2024-04-17T07:17:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.113+0000 7fe473200640 1 -- 172.21.15.86:0/3447361503 shutdown_connections 2024-04-17T07:17:26.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.113+0000 7fe473200640 1 --2- 172.21.15.86:0/3447361503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe46c103a40 0x7fe46c10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.113+0000 7fe473200640 1 --2- 172.21.15.86:0/3447361503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe46c103080 0x7fe46c103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.113+0000 7fe473200640 1 --2- 172.21.15.86:0/3447361503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c101ea0 0x7fe46c1022c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.113+0000 7fe473200640 1 -- 172.21.15.86:0/3447361503 >> 172.21.15.86:0/3447361503 conn(0x7fe46c0fd460 msgr2=0x7fe46c0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.113+0000 7fe473200640 1 -- 172.21.15.86:0/3447361503 shutdown_connections 2024-04-17T07:17:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.113+0000 7fe473200640 1 -- 172.21.15.86:0/3447361503 wait complete. 2024-04-17T07:17:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.114+0000 7fe473200640 1 Processor -- start 2024-04-17T07:17:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.114+0000 7fe473200640 1 -- start start 2024-04-17T07:17:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.114+0000 7fe473200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe46c101ea0 0x7fe46c075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.114+0000 7fe473200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe46c103080 0x7fe46c074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.114+0000 7fe473200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c103a40 0x7fe46c074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.114+0000 7fe473200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c10d9c0 con 0x7fe46c103a40 2024-04-17T07:17:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.114+0000 7fe473200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c10d840 con 0x7fe46c101ea0 2024-04-17T07:17:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.114+0000 7fe473200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c10db40 con 0x7fe46c103080 2024-04-17T07:17:26.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.115+0000 7fe472800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c103a40 0x7fe46c074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.115+0000 7fe472800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c103a40 0x7fe46c074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38464/0 (socket says 172.21.15.86:38464) 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.115+0000 7fe472800640 1 -- 172.21.15.86:0/4000738785 learned_addr learned my addr 172.21.15.86:0/4000738785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.115+0000 7fe471e00640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe46c101ea0 0x7fe46c075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.115+0000 7fe471400640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe46c103080 0x7fe46c074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.115+0000 7fe472800640 1 -- 172.21.15.86:0/4000738785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe46c103080 msgr2=0x7fe46c074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.115+0000 7fe472800640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe46c103080 0x7fe46c074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.115+0000 7fe472800640 1 -- 172.21.15.86:0/4000738785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe46c101ea0 msgr2=0x7fe46c075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.115+0000 7fe472800640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe46c101ea0 0x7fe46c075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.115+0000 7fe472800640 1 -- 172.21.15.86:0/4000738785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c075120 con 0x7fe46c103a40 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.116+0000 7fe472800640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c103a40 0x7fe46c074890 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe46800cce0 tx=0x7fe46800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.116+0000 7fe466a00640 1 -- 172.21.15.86:0/4000738785 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46800f450 con 0x7fe46c103a40 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.116+0000 7fe466a00640 1 -- 172.21.15.86:0/4000738785 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe46800faa0 con 0x7fe46c103a40 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.116+0000 7fe473200640 1 -- 172.21.15.86:0/4000738785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c1a2180 con 0x7fe46c103a40 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.116+0000 7fe466a00640 1 -- 172.21.15.86:0/4000738785 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468018b70 con 0x7fe46c103a40 2024-04-17T07:17:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.116+0000 7fe473200640 1 -- 172.21.15.86:0/4000738785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c1a2680 con 0x7fe46c103a40 2024-04-17T07:17:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.120+0000 7fe4775a2740 1 Processor -- start 2024-04-17T07:17:26.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.121+0000 7fe466a00640 1 -- 172.21.15.86:0/4000738785 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fe468018d10 con 0x7fe46c103a40 2024-04-17T07:17:26.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.122+0000 7fe4775a2740 1 -- start start 2024-04-17T07:17:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.122+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6ed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.122+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6edb50 0x56354c6f2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.122+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6f2cc0 0x56354c6f5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.122+0000 7fe4775a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56354c653020 con 0x56354c6edb50 2024-04-17T07:17:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.122+0000 7fe4775a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56354c5f9460 con 0x56354c6f2cc0 2024-04-17T07:17:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.122+0000 7fe4775a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56354c64efe0 con 0x56354c6ed160 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe471e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6edb50 0x56354c6f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe471e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6edb50 0x56354c6f2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38480/0 (socket says 172.21.15.86:38480) 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe471e00640 1 -- 172.21.15.86:0/440204221 learned_addr learned my addr 172.21.15.86:0/440204221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe471400640 1 --2- 172.21.15.86:0/440204221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe471400640 1 --2- 172.21.15.86:0/440204221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6f2cc0 0x56354c6f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe466a00640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3bb0 0x7fe4540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe471e00640 1 -- 172.21.15.86:0/440204221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 msgr2=0x56354c6ed580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe471e00640 1 --2- 172.21.15.86:0/440204221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6ed580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe471e00640 1 -- 172.21.15.86:0/440204221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6f2cc0 msgr2=0x56354c6f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe471e00640 1 --2- 172.21.15.86:0/440204221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6f2cc0 0x56354c6f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe471e00640 1 -- 172.21.15.86:0/440204221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354c601040 con 0x56354c6edb50 2024-04-17T07:17:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.123+0000 7fe466a00640 1 -- 172.21.15.86:0/4000738785 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe4680d0ce0 con 0x7fe46c103a40 2024-04-17T07:17:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.124+0000 7fe472800640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3bb0 0x7fe4540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.124+0000 7fe471e00640 1 --2- 172.21.15.86:0/440204221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6edb50 0x56354c6f2650 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe460009900 tx=0x7fe460005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8445d2f8f726ba33 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.124+0000 7fe472800640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3bb0 0x7fe4540a6070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe46809c940 tx=0x7fe46800af30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.124+0000 7fe45b000640 1 -- 172.21.15.86:0/440204221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460044020 con 0x56354c6edb50 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.124+0000 7fe45b000640 1 -- 172.21.15.86:0/440204221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe460005040 con 0x56354c6edb50 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.124+0000 7fe45b000640 1 -- 172.21.15.86:0/440204221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460004900 con 0x56354c6edb50 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.125+0000 7fe4775a2740 1 -- 172.21.15.86:0/440204221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6edb50 msgr2=0x56354c6f2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.125+0000 7fe4775a2740 1 --2- 172.21.15.86:0/440204221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6edb50 0x56354c6f2650 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe460009900 tx=0x7fe460005e80 comp rx=0 tx=0).stop 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.125+0000 7fe4775a2740 1 -- 172.21.15.86:0/440204221 shutdown_connections 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.125+0000 7fe4775a2740 1 --2- 172.21.15.86:0/440204221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6f2cc0 0x56354c6f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.125+0000 7fe4775a2740 1 --2- 172.21.15.86:0/440204221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6edb50 0x56354c6f2650 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.125+0000 7fe4775a2740 1 --2- 172.21.15.86:0/440204221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6ed580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.125+0000 7fe4775a2740 1 -- 172.21.15.86:0/440204221 >> 172.21.15.86:0/440204221 conn(0x56354c6e86d0 msgr2=0x56354c6e8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.126+0000 7fe4775a2740 1 -- 172.21.15.86:0/440204221 shutdown_connections 2024-04-17T07:17:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.126+0000 7fe4775a2740 1 -- 172.21.15.86:0/440204221 wait complete. 2024-04-17T07:17:26.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.126+0000 7fe4775a2740 1 -- start start 2024-04-17T07:17:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.126+0000 7fe4775a2740 1 Processor -- start 2024-04-17T07:17:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.127+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6ed160 0x56354c6f7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.127+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.127+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6f2cc0 0x56354c6fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.127+0000 7fe4775a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56354c6f5c10 con 0x56354c6ed160 2024-04-17T07:17:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.127+0000 7fe4775a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56354c60c830 con 0x56354c6edb50 2024-04-17T07:17:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.127+0000 7fe4775a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56354c6f5d90 con 0x56354c6f2cc0 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.127+0000 7fe4775a2740 10 client.0 _ll_register_callbacks cb 0x56354c5dafb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.127+0000 7fe471400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6ed160 0x56354c6f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.127+0000 7fe471400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6ed160 0x56354c6f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38490/0 (socket says 172.21.15.86:38490) 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.127+0000 7fe471400640 1 -- 172.21.15.86:0/4141765964 learned_addr learned my addr 172.21.15.86:0/4141765964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe471e00640 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe471400640 1 -- 172.21.15.86:0/4141765964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6f2cc0 msgr2=0x56354c6fec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe471400640 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6f2cc0 0x56354c6fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe471400640 1 -- 172.21.15.86:0/4141765964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 msgr2=0x56354c6f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe471400640 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6f7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe471400640 1 -- 172.21.15.86:0/4141765964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354c610920 con 0x56354c6ed160 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe471400640 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6ed160 0x56354c6f7680 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe45c014920 tx=0x7fe45c014de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc66e6ed1df79b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe44f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe44f400640 1 -- 172.21.15.86:0/4141765964 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c0142d0 con 0x56354c6ed160 2024-04-17T07:17:26.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe44f400640 1 -- 172.21.15.86:0/4141765964 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe45c011bd0 con 0x56354c6ed160 2024-04-17T07:17:26.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.128+0000 7fe4775a2740 1 -- 172.21.15.86:0/4141765964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe460005040 con 0x56354c6ed160 2024-04-17T07:17:26.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.129+0000 7fe44f400640 1 -- 172.21.15.86:0/4141765964 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe45c025050 con 0x56354c6ed160 2024-04-17T07:17:26.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.129+0000 7fe4775a2740 10 client.16353 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:26.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.129+0000 7fe4775a2740 1 -- 172.21.15.86:0/4141765964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56354c5de7f0 con 0x56354c6ed160 2024-04-17T07:17:26.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.129+0000 7fe44f400640 1 -- 172.21.15.86:0/4141765964 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe460005040 con 0x56354c6ed160 2024-04-17T07:17:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.130+0000 7fe44f400640 1 -- 172.21.15.86:0/4141765964 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe45c01d420 con 0x56354c6ed160 2024-04-17T07:17:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.130+0000 7fe4775a2740 10 client.16353 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.130+0000 7fe4775a2740 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:17:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.130+0000 7fe4775a2740 20 client.16353 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.130+0000 7fe4775a2740 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56354c703230 0x56354c7056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.130+0000 7fe4775a2740 4 client.16353 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.130+0000 7fe4775a2740 1 -- 172.21.15.86:0/4141765964 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56354c660a40 con 0x56354c703230 2024-04-17T07:17:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.131+0000 7fe471e00640 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56354c703230 0x56354c7056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.132+0000 7fe471e00640 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56354c703230 0x56354c7056d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f632ab4f186b1d2 server_cookie=f7dc04f83c2869b6 in_seq=0 out_seq=0 2024-04-17T07:17:26.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.132+0000 7fe44f400640 10 client.16353 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:26.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.134+0000 7fe44f400640 1 -- 172.21.15.86:0/4141765964 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56354c660a40 con 0x56354c703230 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.134+0000 7fe44f400640 10 client.16353 handle_command_reply: tid=0 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.134+0000 7fe4775a2740 1 client.16353 shutdown 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 1 -- 172.21.15.86:0/4141765964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6ed160 msgr2=0x56354c6f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6ed160 0x56354c6f7680 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe45c014920 tx=0x7fe45c014de0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 1 -- 172.21.15.86:0/4141765964 shutdown_connections 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56354c703230 0x56354c7056d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6f2cc0 0x56354c6fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 1 --2- 172.21.15.86:0/4141765964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6ed160 0x56354c6f7680 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 1 -- 172.21.15.86:0/4141765964 >> 172.21.15.86:0/4141765964 conn(0x56354c6e86d0 msgr2=0x56354c6e7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:26.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe44f400640 0 client.16353 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:26.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 1 -- 172.21.15.86:0/4141765964 shutdown_connections 2024-04-17T07:17:26.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 1 -- 172.21.15.86:0/4141765964 wait complete. 2024-04-17T07:17:26.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.135+0000 7fe4775a2740 20 client.16353 trim_cache size 0 max 16384 2024-04-17T07:17:26.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.153+0000 7fe4775a2740 1 Processor -- start 2024-04-17T07:17:26.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.153+0000 7fe4775a2740 1 -- start start 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.153+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.153+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.153+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.153+0000 7fe4775a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1aca40 con 0x56354c6f2cc0 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.153+0000 7fe4775a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1ae580 con 0x56354c6edb50 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.153+0000 7fe4775a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1ad110 con 0x56354c6ed160 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.154+0000 7fe471400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.154+0000 7fe471400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38496/0 (socket says 172.21.15.86:38496) 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.154+0000 7fe471400640 1 -- 172.21.15.86:0/1373959636 learned_addr learned my addr 172.21.15.86:0/1373959636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.154+0000 7fe471e00640 1 --2- 172.21.15.86:0/1373959636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.154+0000 7fe471400640 1 --2- 172.21.15.86:0/1373959636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.154+0000 7fe471400640 1 -- 172.21.15.86:0/1373959636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 msgr2=0x56354c6f79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:26.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.154+0000 7fe471400640 1 --2- 172.21.15.86:0/1373959636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6f79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.154+0000 7fe471400640 1 -- 172.21.15.86:0/1373959636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 msgr2=0x56354c6febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:26.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.154+0000 7fe471400640 1 --2- 172.21.15.86:0/1373959636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.154+0000 7fe471400640 1 -- 172.21.15.86:0/1373959636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354c3653f0 con 0x56354c6f2cc0 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.155+0000 7fe471400640 1 --2- 172.21.15.86:0/1373959636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6e70a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe45c005fd0 tx=0x7fe45c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b87cafef221889a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.155+0000 7fe45a600640 1 -- 172.21.15.86:0/1373959636 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c022070 con 0x56354c6f2cc0 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.155+0000 7fe45a600640 1 -- 172.21.15.86:0/1373959636 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe45c018030 con 0x56354c6f2cc0 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.155+0000 7fe45a600640 1 -- 172.21.15.86:0/1373959636 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c011e30 con 0x56354c6f2cc0 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.156+0000 7fe4775a2740 1 -- 172.21.15.86:0/1373959636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 msgr2=0x56354c6e70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.156+0000 7fe4775a2740 1 --2- 172.21.15.86:0/1373959636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6e70a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe45c005fd0 tx=0x7fe45c016040 comp rx=0 tx=0).stop 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.156+0000 7fe4775a2740 1 -- 172.21.15.86:0/1373959636 shutdown_connections 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.156+0000 7fe4775a2740 1 --2- 172.21.15.86:0/1373959636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6e70a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.156+0000 7fe4775a2740 1 --2- 172.21.15.86:0/1373959636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.156+0000 7fe4775a2740 1 --2- 172.21.15.86:0/1373959636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6f79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.156+0000 7fe4775a2740 1 -- 172.21.15.86:0/1373959636 >> 172.21.15.86:0/1373959636 conn(0x56354c6e86d0 msgr2=0x56354c6f7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.156+0000 7fe4775a2740 1 -- 172.21.15.86:0/1373959636 shutdown_connections 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.156+0000 7fe4775a2740 1 -- 172.21.15.86:0/1373959636 wait complete. 2024-04-17T07:17:26.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.157+0000 7fe4775a2740 1 -- start start 2024-04-17T07:17:26.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.157+0000 7fe4775a2740 1 Processor -- start 2024-04-17T07:17:26.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.157+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.157+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6fc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.157+0000 7fe4775a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.157+0000 7fe4775a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1adeb0 con 0x56354c6f2cc0 2024-04-17T07:17:26.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.157+0000 7fe4775a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1ad7e0 con 0x56354c6edb50 2024-04-17T07:17:26.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.157+0000 7fe4775a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c1aaa10 con 0x56354c6ed160 2024-04-17T07:17:26.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.157+0000 7fe4775a2740 10 client.0 _ll_register_callbacks cb 0x7fe45c014560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:26.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6fd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38498/0 (socket says 172.21.15.86:38498) 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471400640 1 -- 172.21.15.86:0/3623116853 learned_addr learned my addr 172.21.15.86:0/3623116853 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471e00640 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6fc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471400640 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471400640 1 -- 172.21.15.86:0/3623116853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 msgr2=0x56354c6fbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471400640 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6fbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471400640 1 -- 172.21.15.86:0/3623116853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 msgr2=0x56354c6fc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471400640 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6fc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471400640 1 -- 172.21.15.86:0/3623116853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354c42f7d0 con 0x56354c6f2cc0 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe471400640 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6fd3c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe45c01d8e0 tx=0x7fe45c014de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef8586c890abe86 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.158+0000 7fe44f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.159+0000 7fe44f400640 1 -- 172.21.15.86:0/3623116853 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c022070 con 0x56354c6f2cc0 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.159+0000 7fe44f400640 1 -- 172.21.15.86:0/3623116853 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe45c01d420 con 0x56354c6f2cc0 2024-04-17T07:17:26.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.159+0000 7fe4775a2740 1 -- 172.21.15.86:0/3623116853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe45c018030 con 0x56354c6f2cc0 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.159+0000 7fe44f400640 1 -- 172.21.15.86:0/3623116853 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe45c03f050 con 0x56354c6f2cc0 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.159+0000 7fe4775a2740 10 client.16359 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.159+0000 7fe4775a2740 1 -- 172.21.15.86:0/3623116853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56354c6feae0 con 0x56354c6f2cc0 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.159+0000 7fe44f400640 1 -- 172.21.15.86:0/3623116853 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe45c018030 con 0x56354c6f2cc0 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.160+0000 7fe44f400640 1 -- 172.21.15.86:0/3623116853 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe45c036020 con 0x56354c6f2cc0 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.160+0000 7fe4775a2740 10 client.16359 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.160+0000 7fe4775a2740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.160+0000 7fe4775a2740 20 client.16359 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.160+0000 7fe4775a2740 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56354c769a90 0x56354c76beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.160+0000 7fe4775a2740 4 client.16359 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:17:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.160+0000 7fe4775a2740 1 -- 172.21.15.86:0/3623116853 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56354c65a8b0 con 0x56354c769a90 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.160+0000 7fe471e00640 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56354c769a90 0x56354c76beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.161+0000 7fe471e00640 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56354c769a90 0x56354c76beb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=791935c367ce5979 server_cookie=b9b5924b985bb801 in_seq=0 out_seq=0 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.161+0000 7fe44f400640 10 client.16359 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe44f400640 1 -- 172.21.15.86:0/3623116853 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56354c65a8b0 con 0x56354c769a90 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe44f400640 10 client.16359 handle_command_reply: tid=0 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 client.16359 shutdown 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 -- 172.21.15.86:0/3623116853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 msgr2=0x56354c6fd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6fd3c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe45c01d8e0 tx=0x7fe45c014de0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 -- 172.21.15.86:0/3623116853 shutdown_connections 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56354c769a90 0x56354c76beb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56354c6f2cc0 0x56354c6fd3c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56354c6edb50 0x56354c6fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 --2- 172.21.15.86:0/3623116853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56354c6ed160 0x56354c6fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe44f400640 0 client.16359 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:26.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 -- 172.21.15.86:0/3623116853 >> 172.21.15.86:0/3623116853 conn(0x56354c6e86d0 msgr2=0x56354c6e70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:26.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 -- 172.21.15.86:0/3623116853 shutdown_connections 2024-04-17T07:17:26.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 1 -- 172.21.15.86:0/3623116853 wait complete. 2024-04-17T07:17:26.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.162+0000 7fe4775a2740 20 client.16359 trim_cache size 0 max 16384 2024-04-17T07:17:26.164 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:17:26.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 -- 172.21.15.86:0/4000738785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3bb0 msgr2=0x7fe4540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:26.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3bb0 0x7fe4540a6070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe46809c940 tx=0x7fe46800af30 comp rx=0 tx=0).stop 2024-04-17T07:17:26.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 -- 172.21.15.86:0/4000738785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c103a40 msgr2=0x7fe46c074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:26.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c103a40 0x7fe46c074890 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe46800cce0 tx=0x7fe46800a620 comp rx=0 tx=0).stop 2024-04-17T07:17:26.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 -- 172.21.15.86:0/4000738785 shutdown_connections 2024-04-17T07:17:26.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe4540a3bb0 0x7fe4540a6070 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe46c103a40 0x7fe46c074890 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe46c103080 0x7fe46c074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 --2- 172.21.15.86:0/4000738785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe46c101ea0 0x7fe46c075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:26.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 -- 172.21.15.86:0/4000738785 >> 172.21.15.86:0/4000738785 conn(0x7fe46c0fd460 msgr2=0x7fe46c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:26.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 -- 172.21.15.86:0/4000738785 shutdown_connections 2024-04-17T07:17:26.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:26.163+0000 7fe473200640 1 -- 172.21.15.86:0/4000738785 wait complete. 2024-04-17T07:17:26.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:25 smithi098 ceph-mon[35418]: pgmap v532: 129 pgs: 129 active+clean; 436 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 18 op/s 2024-04-17T07:17:26.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:25 smithi103 ceph-mon[35495]: pgmap v532: 129 pgs: 129 active+clean; 436 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 18 op/s 2024-04-17T07:17:27.181 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:17:27.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:17:27.181 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:17:27.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.423+0000 7f8ddf200640 1 -- 172.21.15.86:0/4141959054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0102960 msgr2=0x7f8de0104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.423+0000 7f8ddf200640 1 --2- 172.21.15.86:0/4141959054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0102960 0x7f8de0104d50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009900 tx=0x7f8dd002bc40 comp rx=0 tx=0).stop 2024-04-17T07:17:27.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.423+0000 7f8ddf200640 1 -- 172.21.15.86:0/4141959054 shutdown_connections 2024-04-17T07:17:27.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.423+0000 7f8ddf200640 1 --2- 172.21.15.86:0/4141959054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de0107bc0 0x7f8de0109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.423+0000 7f8ddf200640 1 --2- 172.21.15.86:0/4141959054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de0105290 0x7f8de0107680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.423+0000 7f8ddf200640 1 --2- 172.21.15.86:0/4141959054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0102960 0x7f8de0104d50 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.423+0000 7f8ddf200640 1 -- 172.21.15.86:0/4141959054 >> 172.21.15.86:0/4141959054 conn(0x7f8de00fc310 msgr2=0x7f8de00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.423+0000 7f8ddf200640 1 -- 172.21.15.86:0/4141959054 shutdown_connections 2024-04-17T07:17:27.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.423+0000 7f8ddf200640 1 -- 172.21.15.86:0/4141959054 wait complete. 2024-04-17T07:17:27.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.424+0000 7f8ddf200640 1 Processor -- start 2024-04-17T07:17:27.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.424+0000 7f8ddf200640 1 -- start start 2024-04-17T07:17:27.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0102960 0x7f8de019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddf200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de0105290 0x7f8de019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddf200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de0107bc0 0x7f8de01a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8de010d9f0 con 0x7f8de0102960 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddf200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8de010d870 con 0x7f8de0105290 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddf200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8de010db70 con 0x7f8de0107bc0 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0102960 0x7f8de019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0102960 0x7f8de019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57964/0 (socket says 172.21.15.86:57964) 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddde00640 1 -- 172.21.15.86:0/2538809279 learned_addr learned my addr 172.21.15.86:0/2538809279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddde00640 1 -- 172.21.15.86:0/2538809279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de0107bc0 msgr2=0x7f8de01a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8dde800640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de0107bc0 0x7f8de01a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddd400640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de0105290 0x7f8de019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddde00640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de0107bc0 0x7f8de01a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddde00640 1 -- 172.21.15.86:0/2538809279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de0105290 msgr2=0x7f8de019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddde00640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de0105290 0x7f8de019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.425+0000 7f8ddde00640 1 -- 172.21.15.86:0/2538809279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de01a1e20 con 0x7f8de0102960 2024-04-17T07:17:27.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.426+0000 7f8ddd400640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de0105290 0x7f8de019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:27.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.426+0000 7f8dde800640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de0107bc0 0x7f8de01a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:27.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.426+0000 7f8ddde00640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0102960 0x7f8de019a200 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0000c00 tx=0x7f8dd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.426+0000 7f8dd6a00640 1 -- 172.21.15.86:0/2538809279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0043020 con 0x7f8de0102960 2024-04-17T07:17:27.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.426+0000 7f8dd6a00640 1 -- 172.21.15.86:0/2538809279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8dd00054c0 con 0x7f8de0102960 2024-04-17T07:17:27.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.426+0000 7f8dd6a00640 1 -- 172.21.15.86:0/2538809279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0003d80 con 0x7f8de0102960 2024-04-17T07:17:27.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.426+0000 7f8ddf200640 1 -- 172.21.15.86:0/2538809279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de01a20b0 con 0x7f8de0102960 2024-04-17T07:17:27.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.426+0000 7f8ddf200640 1 -- 172.21.15.86:0/2538809279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de01a2600 con 0x7f8de0102960 2024-04-17T07:17:27.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.430+0000 7f8dd6a00640 1 -- 172.21.15.86:0/2538809279 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8dd0046020 con 0x7f8de0102960 2024-04-17T07:17:27.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.430+0000 7f8de7be6740 1 Processor -- start 2024-04-17T07:17:27.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.430+0000 7f8de7be6740 1 -- start start 2024-04-17T07:17:27.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.430+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x5635871144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.430+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587119590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.430+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 0x56358711c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.430+0000 7f8de7be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563587079cb0 con 0x5635871140a0 2024-04-17T07:17:27.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.430+0000 7f8de7be6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563587020370 con 0x563587114a90 2024-04-17T07:17:27.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.430+0000 7f8de7be6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563587075c70 con 0x563587119c00 2024-04-17T07:17:27.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8dde800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x5635871144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8dde800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x5635871144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57972/0 (socket says 172.21.15.86:57972) 2024-04-17T07:17:27.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8dde800640 1 -- 172.21.15.86:0/156889231 learned_addr learned my addr 172.21.15.86:0/156889231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:27.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8ddde00640 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587119590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8ddd400640 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 0x56358711c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8dde800640 1 -- 172.21.15.86:0/156889231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 msgr2=0x56358711c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8dde800640 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 0x56358711c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8dde800640 1 -- 172.21.15.86:0/156889231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 msgr2=0x563587119590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8dde800640 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587119590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8dde800640 1 -- 172.21.15.86:0/156889231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56358703f910 con 0x5635871140a0 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8ddd400640 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 0x56358711c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8ddde00640 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587119590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.431+0000 7f8dd6a00640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dc40a3bb0 0x7f8dc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.432+0000 7f8dde800640 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x5635871144c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8008e60 tx=0x7f8dd800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa1566101da6e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.432+0000 7f8ddde00640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dc40a3bb0 0x7f8dc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.432+0000 7f8dd6a00640 1 -- 172.21.15.86:0/2538809279 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8dd00f2cc0 con 0x7f8de0102960 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.432+0000 7f8dcb400640 1 -- 172.21.15.86:0/156889231 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800f830 con 0x5635871140a0 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.432+0000 7f8dcb400640 1 -- 172.21.15.86:0/156889231 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8dd8018480 con 0x5635871140a0 2024-04-17T07:17:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.432+0000 7f8dcb400640 1 -- 172.21.15.86:0/156889231 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800e610 con 0x5635871140a0 2024-04-17T07:17:27.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.433+0000 7f8de7be6740 1 -- 172.21.15.86:0/156889231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 msgr2=0x5635871144c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.433+0000 7f8de7be6740 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x5635871144c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8008e60 tx=0x7f8dd800d710 comp rx=0 tx=0).stop 2024-04-17T07:17:27.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.433+0000 7f8ddde00640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dc40a3bb0 0x7f8dc40a6070 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00bb020 tx=0x7f8dd00df040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.433+0000 7f8de7be6740 1 -- 172.21.15.86:0/156889231 shutdown_connections 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.433+0000 7f8de7be6740 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 0x56358711c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.433+0000 7f8de7be6740 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587119590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.433+0000 7f8de7be6740 1 --2- 172.21.15.86:0/156889231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x5635871144c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.433+0000 7f8de7be6740 1 -- 172.21.15.86:0/156889231 >> 172.21.15.86:0/156889231 conn(0x56358710f610 msgr2=0x56358710fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.433+0000 7f8de7be6740 1 -- 172.21.15.86:0/156889231 shutdown_connections 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.433+0000 7f8de7be6740 1 -- 172.21.15.86:0/156889231 wait complete. 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.434+0000 7f8de7be6740 1 -- start start 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.434+0000 7f8de7be6740 1 Processor -- start 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587114a90 0x56358711eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587119c00 0x563587125ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8de7be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56358711cbb0 con 0x5635871140a0 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8de7be6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563587033730 con 0x563587119c00 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8de7be6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56358711cd30 con 0x563587114a90 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8de7be6740 10 client.0 _ll_register_callbacks cb 0x563587035b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8dde800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8dde800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57974/0 (socket says 172.21.15.86:57974) 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8dde800640 1 -- 172.21.15.86:0/2396198356 learned_addr learned my addr 172.21.15.86:0/2396198356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8ddd400640 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587114a90 0x56358711eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8dde800640 1 -- 172.21.15.86:0/2396198356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587114a90 msgr2=0x56358711eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8dde800640 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587114a90 0x56358711eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.435+0000 7f8dde800640 1 -- 172.21.15.86:0/2396198356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587119c00 msgr2=0x563587125ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.436+0000 7f8dde800640 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587119c00 0x563587125ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.436+0000 7f8dde800640 1 -- 172.21.15.86:0/2396198356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563586e547e0 con 0x5635871140a0 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.436+0000 7f8ddd400640 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587114a90 0x56358711eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.436+0000 7f8dde800640 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e560 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8006010 tx=0x7f8dd8008ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e67fe81e176766 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.436+0000 7f8dbfe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:27.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.436+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2396198356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800fb80 con 0x5635871140a0 2024-04-17T07:17:27.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.436+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2396198356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8dd800fd20 con 0x5635871140a0 2024-04-17T07:17:27.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.436+0000 7f8de7be6740 1 -- 172.21.15.86:0/2396198356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dd8018480 con 0x5635871140a0 2024-04-17T07:17:27.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2396198356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8dd8030050 con 0x5635871140a0 2024-04-17T07:17:27.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8de7be6740 10 client.16371 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:27.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8de7be6740 1 -- 172.21.15.86:0/2396198356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563586eb4f70 con 0x5635871140a0 2024-04-17T07:17:27.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2396198356 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8dd8018480 con 0x5635871140a0 2024-04-17T07:17:27.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2396198356 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8dd802b030 con 0x5635871140a0 2024-04-17T07:17:27.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8de7be6740 10 client.16371 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:27.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8de7be6740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:17:27.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8de7be6740 20 client.16371 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:27.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563587129ff0 0x56358712c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8de7be6740 4 client.16371 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:27.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.437+0000 7f8de7be6740 1 -- 172.21.15.86:0/2396198356 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635870876d0 con 0x563587129ff0 2024-04-17T07:17:27.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.438+0000 7f8ddd400640 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563587129ff0 0x56358712c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.439+0000 7f8dbfe00640 10 client.16371 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:27.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.439+0000 7f8ddd400640 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563587129ff0 0x56358712c3e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce6be03cdf1553b server_cookie=5c5d997b1899f58b in_seq=0 out_seq=0 2024-04-17T07:17:27.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2396198356 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635870876d0 con 0x563587129ff0 2024-04-17T07:17:27.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8dbfe00640 10 client.16371 handle_command_reply: tid=0 2024-04-17T07:17:27.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8de7be6740 1 client.16371 shutdown 2024-04-17T07:17:27.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8de7be6740 1 -- 172.21.15.86:0/2396198356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 msgr2=0x56358711e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e560 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8006010 tx=0x7f8dd8008ba0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8de7be6740 1 -- 172.21.15.86:0/2396198356 shutdown_connections 2024-04-17T07:17:27.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563587129ff0 0x56358712c3e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587119c00 0x563587125ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587114a90 0x56358711eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2396198356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e560 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8de7be6740 1 -- 172.21.15.86:0/2396198356 >> 172.21.15.86:0/2396198356 conn(0x56358710f610 msgr2=0x56358710e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.441+0000 7f8dbfe00640 0 client.16371 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:27.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.442+0000 7f8de7be6740 1 -- 172.21.15.86:0/2396198356 shutdown_connections 2024-04-17T07:17:27.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.442+0000 7f8de7be6740 1 -- 172.21.15.86:0/2396198356 wait complete. 2024-04-17T07:17:27.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.442+0000 7f8de7be6740 20 client.16371 trim_cache size 0 max 16384 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.467+0000 7f8de7be6740 1 Processor -- start 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.467+0000 7f8de7be6740 1 -- start start 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.467+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.467+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587125a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.467+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 0x56358710e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.467+0000 7f8de7be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8de01acad0 con 0x5635871140a0 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.467+0000 7f8de7be6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8de01ae610 con 0x563587114a90 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.468+0000 7f8de7be6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8de01ad1a0 con 0x563587119c00 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.468+0000 7f8dde800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.468+0000 7f8dde800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57986/0 (socket says 172.21.15.86:57986) 2024-04-17T07:17:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.468+0000 7f8dde800640 1 -- 172.21.15.86:0/1104476229 learned_addr learned my addr 172.21.15.86:0/1104476229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.468+0000 7f8ddd400640 1 --2- 172.21.15.86:0/1104476229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587125a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.468+0000 7f8dde800640 1 --2- 172.21.15.86:0/1104476229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 0x56358710e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.468+0000 7f8dde800640 1 -- 172.21.15.86:0/1104476229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 msgr2=0x56358710e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.468+0000 7f8dde800640 1 --2- 172.21.15.86:0/1104476229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 0x56358710e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.468+0000 7f8dde800640 1 -- 172.21.15.86:0/1104476229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 msgr2=0x563587125a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.468+0000 7f8dde800640 1 --2- 172.21.15.86:0/1104476229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587125a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.469+0000 7f8dde800640 1 -- 172.21.15.86:0/1104476229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56358712c6e0 con 0x5635871140a0 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.469+0000 7f8ddd400640 1 --2- 172.21.15.86:0/1104476229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587125a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.469+0000 7f8dde800640 1 --2- 172.21.15.86:0/1104476229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e880 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800dc90 tx=0x7f8dd8005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d402758c8af50fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.469+0000 7f8dcaa00640 1 -- 172.21.15.86:0/1104476229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8038020 con 0x5635871140a0 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.469+0000 7f8dcaa00640 1 -- 172.21.15.86:0/1104476229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8dd800fe30 con 0x5635871140a0 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.469+0000 7f8dcaa00640 1 -- 172.21.15.86:0/1104476229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8023420 con 0x5635871140a0 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.470+0000 7f8de7be6740 1 -- 172.21.15.86:0/1104476229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 msgr2=0x56358711e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.470+0000 7f8de7be6740 1 --2- 172.21.15.86:0/1104476229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e880 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800dc90 tx=0x7f8dd8005ac0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.470+0000 7f8de7be6740 1 -- 172.21.15.86:0/1104476229 shutdown_connections 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.470+0000 7f8de7be6740 1 --2- 172.21.15.86:0/1104476229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563587119c00 0x56358710e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.470+0000 7f8de7be6740 1 --2- 172.21.15.86:0/1104476229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587125a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.470+0000 7f8de7be6740 1 --2- 172.21.15.86:0/1104476229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635871140a0 0x56358711e880 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.470+0000 7f8de7be6740 1 -- 172.21.15.86:0/1104476229 >> 172.21.15.86:0/1104476229 conn(0x56358710f610 msgr2=0x56358711e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.470+0000 7f8de7be6740 1 -- 172.21.15.86:0/1104476229 shutdown_connections 2024-04-17T07:17:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.470+0000 7f8de7be6740 1 -- 172.21.15.86:0/1104476229 wait complete. 2024-04-17T07:17:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.471+0000 7f8de7be6740 1 -- start start 2024-04-17T07:17:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.471+0000 7f8de7be6740 1 Processor -- start 2024-04-17T07:17:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.471+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635871140a0 0x563587122fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.471+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587123520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.471+0000 7f8de7be6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563587119c00 0x563587124530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8de7be6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8de01adf40 con 0x563587119c00 2024-04-17T07:17:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8de7be6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8de01ad870 con 0x563587114a90 2024-04-17T07:17:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8de7be6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8de01aaaa0 con 0x5635871140a0 2024-04-17T07:17:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8de7be6740 10 client.0 _ll_register_callbacks cb 0x7f8dd8008cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8dde800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563587119c00 0x563587124530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8dde800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563587119c00 0x563587124530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57998/0 (socket says 172.21.15.86:57998) 2024-04-17T07:17:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8dde800640 1 -- 172.21.15.86:0/2870473477 learned_addr learned my addr 172.21.15.86:0/2870473477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8dde800640 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635871140a0 0x563587122fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8ddd400640 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587123520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8dde800640 1 -- 172.21.15.86:0/2870473477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635871140a0 msgr2=0x563587122fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8dde800640 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635871140a0 0x563587122fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8dde800640 1 -- 172.21.15.86:0/2870473477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 msgr2=0x563587123520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8dde800640 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587123520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.472+0000 7f8dde800640 1 -- 172.21.15.86:0/2870473477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563586d4c170 con 0x563587119c00 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.473+0000 7f8dde800640 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563587119c00 0x563587124530 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8008e30 tx=0x7f8dd8018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbe35c30481b5df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.473+0000 7f8dbfe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.473+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2870473477 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8038070 con 0x563587119c00 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.473+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2870473477 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8dd8022e30 con 0x563587119c00 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.473+0000 7f8de7be6740 1 -- 172.21.15.86:0/2870473477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dd800fe30 con 0x563587119c00 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.473+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2870473477 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8dd800f980 con 0x563587119c00 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.473+0000 7f8de7be6740 10 client.16377 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.473+0000 7f8de7be6740 1 -- 172.21.15.86:0/2870473477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563587125c90 con 0x563587119c00 2024-04-17T07:17:27.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.474+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2870473477 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8dd800fe30 con 0x563587119c00 2024-04-17T07:17:27.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.474+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2870473477 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8dd802b040 con 0x563587119c00 2024-04-17T07:17:27.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.474+0000 7f8de7be6740 10 client.16377 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:27.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.474+0000 7f8de7be6740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:17:27.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.474+0000 7f8de7be6740 20 client.16377 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:27.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.474+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5635871992b0 0x56358719b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.474+0000 7f8de7be6740 4 client.16377 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:17:27.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.474+0000 7f8de7be6740 1 -- 172.21.15.86:0/2870473477 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563587081540 con 0x5635871992b0 2024-04-17T07:17:27.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.475+0000 7f8ddd400640 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5635871992b0 0x56358719b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.475+0000 7f8dbfe00640 10 client.16377 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:27.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.475+0000 7f8ddd400640 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5635871992b0 0x56358719b6d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4917dd0857024de2 server_cookie=bcf8ab065f689268 in_seq=0 out_seq=0 2024-04-17T07:17:27.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8dbfe00640 1 -- 172.21.15.86:0/2870473477 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563587081540 con 0x5635871992b0 2024-04-17T07:17:27.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8dbfe00640 10 client.16377 handle_command_reply: tid=0 2024-04-17T07:17:27.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8de7be6740 1 client.16377 shutdown 2024-04-17T07:17:27.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8de7be6740 1 -- 172.21.15.86:0/2870473477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563587119c00 msgr2=0x563587124530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563587119c00 0x563587124530 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8008e30 tx=0x7f8dd8018f90 comp rx=0 tx=0).stop 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8de7be6740 1 -- 172.21.15.86:0/2870473477 shutdown_connections 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5635871992b0 0x56358719b6d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563587119c00 0x563587124530 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563587114a90 0x563587123520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8de7be6740 1 --2- 172.21.15.86:0/2870473477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635871140a0 0x563587122fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8dbfe00640 0 client.16377 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8de7be6740 1 -- 172.21.15.86:0/2870473477 >> 172.21.15.86:0/2870473477 conn(0x56358710f610 msgr2=0x563587112870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.478+0000 7f8de7be6740 1 -- 172.21.15.86:0/2870473477 shutdown_connections 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.479+0000 7f8de7be6740 1 -- 172.21.15.86:0/2870473477 wait complete. 2024-04-17T07:17:27.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.479+0000 7f8de7be6740 20 client.16377 trim_cache size 0 max 16384 2024-04-17T07:17:27.480 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:17:27.480 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:17:27.480 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "6ac6bb36-7f1a-4db8-9f64-fedfdb06f93a", 2024-04-17T07:17:27.480 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:17:27.480 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:17:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.479+0000 7f8ddf200640 1 -- 172.21.15.86:0/2538809279 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dc40a3bb0 msgr2=0x7f8dc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.479+0000 7f8ddf200640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dc40a3bb0 0x7f8dc40a6070 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00bb020 tx=0x7f8dd00df040 comp rx=0 tx=0).stop 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.479+0000 7f8ddf200640 1 -- 172.21.15.86:0/2538809279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0102960 msgr2=0x7f8de019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.479+0000 7f8ddf200640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0102960 0x7f8de019a200 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0000c00 tx=0x7f8dd00042f0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.479+0000 7f8ddf200640 1 -- 172.21.15.86:0/2538809279 shutdown_connections 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.479+0000 7f8ddf200640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8dc40a3bb0 0x7f8dc40a6070 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.480+0000 7f8ddf200640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de0107bc0 0x7f8de01a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.480+0000 7f8ddf200640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de0105290 0x7f8de019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.480+0000 7f8ddf200640 1 --2- 172.21.15.86:0/2538809279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8de0102960 0x7f8de019a200 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.480+0000 7f8ddf200640 1 -- 172.21.15.86:0/2538809279 >> 172.21.15.86:0/2538809279 conn(0x7f8de00fc310 msgr2=0x7f8de0101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.480+0000 7f8ddf200640 1 -- 172.21.15.86:0/2538809279 shutdown_connections 2024-04-17T07:17:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.480+0000 7f8ddf200640 1 -- 172.21.15.86:0/2538809279 wait complete. 2024-04-17T07:17:27.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:17:27.497 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:17:27.574 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:27 smithi086 ceph-mon[31733]: pgmap v533: 129 pgs: 129 active+clean; 468 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:17:27.624 INFO:teuthology.orchestra.run.smithi086.stderr:1900000 of 3200000 tuples (59%) done (elapsed 77.17 s, remaining 52.80 s) 2024-04-17T07:17:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.725+0000 7f705a200640 1 -- 172.21.15.86:0/2829561675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7054075fa0 msgr2=0x7f70540743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.725+0000 7f705a200640 1 --2- 172.21.15.86:0/2829561675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7054075fa0 0x7f70540743f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7044009900 tx=0x7f704402bc30 comp rx=0 tx=0).stop 2024-04-17T07:17:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.726+0000 7f705a200640 1 -- 172.21.15.86:0/2829561675 shutdown_connections 2024-04-17T07:17:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.726+0000 7f705a200640 1 --2- 172.21.15.86:0/2829561675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7054109e60 0x7f705410c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.726+0000 7f705a200640 1 --2- 172.21.15.86:0/2829561675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70540749c0 0x7f7054074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.726+0000 7f705a200640 1 --2- 172.21.15.86:0/2829561675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7054075fa0 0x7f70540743f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.726+0000 7f705a200640 1 -- 172.21.15.86:0/2829561675 >> 172.21.15.86:0/2829561675 conn(0x7f70540fd510 msgr2=0x7f70540ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.726+0000 7f705a200640 1 -- 172.21.15.86:0/2829561675 shutdown_connections 2024-04-17T07:17:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.726+0000 7f705a200640 1 -- 172.21.15.86:0/2829561675 wait complete. 2024-04-17T07:17:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.727+0000 7f705a200640 1 Processor -- start 2024-04-17T07:17:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.727+0000 7f705a200640 1 -- start start 2024-04-17T07:17:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f705a200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70540749c0 0x7f705419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f705a200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7054075fa0 0x7f705419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f705a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7054109e60 0x7f70541a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f705a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f705410fcf0 con 0x7f7054109e60 2024-04-17T07:17:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f705a200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f705410fb70 con 0x7f7054075fa0 2024-04-17T07:17:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f705a200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f705410fe70 con 0x7f70540749c0 2024-04-17T07:17:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f7053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7054109e60 0x7f70541a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f7053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7054109e60 0x7f70541a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58022/0 (socket says 172.21.15.86:58022) 2024-04-17T07:17:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f7053e00640 1 -- 172.21.15.86:0/3286977966 learned_addr learned my addr 172.21.15.86:0/3286977966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f7052a00640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7054075fa0 0x7f705419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.728+0000 7f7053400640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70540749c0 0x7f705419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.729+0000 7f7053e00640 1 -- 172.21.15.86:0/3286977966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70540749c0 msgr2=0x7f705419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.729+0000 7f7053e00640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70540749c0 0x7f705419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.729+0000 7f7053e00640 1 -- 172.21.15.86:0/3286977966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7054075fa0 msgr2=0x7f705419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.729+0000 7f7053e00640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7054075fa0 0x7f705419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.729+0000 7f7053e00640 1 -- 172.21.15.86:0/3286977966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70541a6400 con 0x7f7054109e60 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.729+0000 7f7053e00640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7054109e60 0x7f70541a5d90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f704c00cce0 tx=0x7f704c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.729+0000 7f704be00640 1 -- 172.21.15.86:0/3286977966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c00f450 con 0x7f7054109e60 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.730+0000 7f704be00640 1 -- 172.21.15.86:0/3286977966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f704c00faa0 con 0x7f7054109e60 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.730+0000 7f705a200640 1 -- 172.21.15.86:0/3286977966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70541a66f0 con 0x7f7054109e60 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.730+0000 7f704be00640 1 -- 172.21.15.86:0/3286977966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c018b70 con 0x7f7054109e60 2024-04-17T07:17:27.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.730+0000 7f705a200640 1 -- 172.21.15.86:0/3286977966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70541a6bf0 con 0x7f7054109e60 2024-04-17T07:17:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.734+0000 7f704be00640 1 -- 172.21.15.86:0/3286977966 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f704c018d10 con 0x7f7054109e60 2024-04-17T07:17:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.734+0000 7f705d00c740 1 Processor -- start 2024-04-17T07:17:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.734+0000 7f705d00c740 1 -- start start 2024-04-17T07:17:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.735+0000 7f705d00c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f775a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.735+0000 7f705d00c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775ab80 0x55c2f775f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.735+0000 7f705d00c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775fcf0 0x55c2f7762180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.735+0000 7f705d00c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c2f76c1120 con 0x55c2f775ab80 2024-04-17T07:17:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.735+0000 7f705d00c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c2f7667340 con 0x55c2f775a190 2024-04-17T07:17:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.735+0000 7f705d00c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2f76bd0e0 con 0x55c2f775fcf0 2024-04-17T07:17:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.735+0000 7f7053400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775ab80 0x55c2f775f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.735+0000 7f7053400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775ab80 0x55c2f775f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58024/0 (socket says 172.21.15.86:58024) 2024-04-17T07:17:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.735+0000 7f7053400640 1 -- 172.21.15.86:0/1617191413 learned_addr learned my addr 172.21.15.86:0/1617191413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.736+0000 7f7053400640 1 -- 172.21.15.86:0/1617191413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775fcf0 msgr2=0x55c2f7762180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.736+0000 7f7053400640 1 --2- 172.21.15.86:0/1617191413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775fcf0 0x55c2f7762180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.736+0000 7f7053400640 1 -- 172.21.15.86:0/1617191413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 msgr2=0x55c2f775a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.736+0000 7f7053400640 1 --2- 172.21.15.86:0/1617191413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f775a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.736+0000 7f7053400640 1 -- 172.21.15.86:0/1617191413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f749d7d0 con 0x55c2f775ab80 2024-04-17T07:17:27.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.736+0000 7f7053400640 1 --2- 172.21.15.86:0/1617191413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775ab80 0x55c2f775f680 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7044005ab0 tx=0x7f7044005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27a3123fe7a5442 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.737+0000 7f703fe00640 1 -- 172.21.15.86:0/1617191413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044044020 con 0x55c2f775ab80 2024-04-17T07:17:27.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.737+0000 7f703fe00640 1 -- 172.21.15.86:0/1617191413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f704402cc20 con 0x55c2f775ab80 2024-04-17T07:17:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.737+0000 7f703fe00640 1 -- 172.21.15.86:0/1617191413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044004940 con 0x55c2f775ab80 2024-04-17T07:17:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.737+0000 7f704be00640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70380a3bb0 0x7f70380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.737+0000 7f704be00640 1 -- 172.21.15.86:0/3286977966 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f704c0d0bf0 con 0x7f7054109e60 2024-04-17T07:17:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.738+0000 7f705d00c740 1 -- 172.21.15.86:0/1617191413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775ab80 msgr2=0x55c2f775f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.738+0000 7f705d00c740 1 --2- 172.21.15.86:0/1617191413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775ab80 0x55c2f775f680 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7044005ab0 tx=0x7f7044005d50 comp rx=0 tx=0).stop 2024-04-17T07:17:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.738+0000 7f705d00c740 1 -- 172.21.15.86:0/1617191413 shutdown_connections 2024-04-17T07:17:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.738+0000 7f705d00c740 1 --2- 172.21.15.86:0/1617191413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775fcf0 0x55c2f7762180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.738+0000 7f705d00c740 1 --2- 172.21.15.86:0/1617191413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775ab80 0x55c2f775f680 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.738+0000 7f705d00c740 1 --2- 172.21.15.86:0/1617191413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f775a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.738+0000 7f705d00c740 1 -- 172.21.15.86:0/1617191413 >> 172.21.15.86:0/1617191413 conn(0x55c2f7755700 msgr2=0x55c2f7755b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.738+0000 7f7052a00640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70380a3bb0 0x7f70380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.739+0000 7f7052a00640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70380a3bb0 0x7f70380a6070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55c2f775b3e0 tx=0x7f7040004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.739+0000 7f705d00c740 1 -- 172.21.15.86:0/1617191413 shutdown_connections 2024-04-17T07:17:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.739+0000 7f705d00c740 1 -- 172.21.15.86:0/1617191413 wait complete. 2024-04-17T07:17:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.740+0000 7f705d00c740 1 -- start start 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.740+0000 7f705d00c740 1 Processor -- start 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f705d00c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f7764620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f705d00c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f7764b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f705d00c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f776bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f705d00c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c2f7762c40 con 0x55c2f775fcf0 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f705d00c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c2f767a710 con 0x55c2f775a190 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f705d00c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2f7762dc0 con 0x55c2f775ab80 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f705d00c740 10 client.0 _ll_register_callbacks cb 0x55c2f7675af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f7053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f776bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f7053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f776bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58034/0 (socket says 172.21.15.86:58034) 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f7053e00640 1 -- 172.21.15.86:0/2442277566 learned_addr learned my addr 172.21.15.86:0/2442277566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f7053400640 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f7764620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.741+0000 7f7052a00640 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f7764b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f7053e00640 1 -- 172.21.15.86:0/2442277566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 msgr2=0x55c2f7764b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f7053e00640 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f7764b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f7053e00640 1 -- 172.21.15.86:0/2442277566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 msgr2=0x55c2f7764620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f7053e00640 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f7764620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f7053e00640 1 -- 172.21.15.86:0/2442277566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f73d33f0 con 0x55c2f775fcf0 2024-04-17T07:17:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f7052a00640 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f7764b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f7053e00640 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f776bba0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f704c09cae0 tx=0x7f704c09d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae62c06971045b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f703cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f703cc00640 1 -- 172.21.15.86:0/2442277566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0b6060 con 0x55c2f775fcf0 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f703cc00640 1 -- 172.21.15.86:0/2442277566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f704c00fc50 con 0x55c2f775fcf0 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.742+0000 7f705d00c740 1 -- 172.21.15.86:0/2442277566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704402cc20 con 0x55c2f775fcf0 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f703cc00640 1 -- 172.21.15.86:0/2442277566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f704c0a0f00 con 0x55c2f775fcf0 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f705d00c740 10 client.16389 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f705d00c740 1 -- 172.21.15.86:0/2442277566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c2f73930f0 con 0x55c2f775fcf0 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f703cc00640 1 -- 172.21.15.86:0/2442277566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f704402cc20 con 0x55c2f775fcf0 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f703cc00640 1 -- 172.21.15.86:0/2442277566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f704c09c130 con 0x55c2f775fcf0 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f705d00c740 10 client.16389 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f705d00c740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f705d00c740 20 client.16389 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f705d00c740 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c2f77704d0 0x55c2f7772990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f705d00c740 4 client.16389 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.743+0000 7f705d00c740 1 -- 172.21.15.86:0/2442277566 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f76ceb40 con 0x55c2f77704d0 2024-04-17T07:17:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.744+0000 7f7053400640 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c2f77704d0 0x55c2f7772990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.745+0000 7f7053400640 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c2f77704d0 0x55c2f7772990 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=180c0202187ff4e5 server_cookie=643b5dcd5a2ab350 in_seq=0 out_seq=0 2024-04-17T07:17:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.745+0000 7f703cc00640 10 client.16389 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f703cc00640 1 -- 172.21.15.86:0/2442277566 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2f76ceb40 con 0x55c2f77704d0 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f703cc00640 10 client.16389 handle_command_reply: tid=0 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f705d00c740 1 client.16389 shutdown 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f705d00c740 1 -- 172.21.15.86:0/2442277566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 msgr2=0x55c2f776bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f705d00c740 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f776bba0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f704c09cae0 tx=0x7f704c09d250 comp rx=0 tx=0).stop 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f705d00c740 1 -- 172.21.15.86:0/2442277566 shutdown_connections 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f705d00c740 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c2f77704d0 0x55c2f7772990 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f705d00c740 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f776bba0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f703cc00640 0 client.16389 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f705d00c740 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f7764b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f705d00c740 1 --2- 172.21.15.86:0/2442277566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f7764620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.746+0000 7f705d00c740 1 -- 172.21.15.86:0/2442277566 >> 172.21.15.86:0/2442277566 conn(0x55c2f7755700 msgr2=0x55c2f7754890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.747+0000 7f705d00c740 1 -- 172.21.15.86:0/2442277566 shutdown_connections 2024-04-17T07:17:27.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.747+0000 7f705d00c740 1 -- 172.21.15.86:0/2442277566 wait complete. 2024-04-17T07:17:27.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.747+0000 7f705d00c740 20 client.16389 trim_cache size 0 max 16384 2024-04-17T07:17:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:27 smithi098 ceph-mon[35418]: pgmap v533: 129 pgs: 129 active+clean; 468 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:17:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.767+0000 7f705d00c740 1 Processor -- start 2024-04-17T07:17:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f705d00c740 1 -- start start 2024-04-17T07:17:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f705d00c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f7754000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f705d00c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f776bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f705d00c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f7754540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f705d00c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b1060 con 0x55c2f775fcf0 2024-04-17T07:17:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f705d00c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b2ba0 con 0x55c2f775a190 2024-04-17T07:17:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f705d00c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b1730 con 0x55c2f775ab80 2024-04-17T07:17:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f7053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f7754540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f7053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f7754540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58046/0 (socket says 172.21.15.86:58046) 2024-04-17T07:17:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f7053e00640 1 -- 172.21.15.86:0/3464964004 learned_addr learned my addr 172.21.15.86:0/3464964004 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.768+0000 7f7053400640 1 --2- 172.21.15.86:0/3464964004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f7754000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f7052a00640 1 --2- 172.21.15.86:0/3464964004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f776bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f7053e00640 1 -- 172.21.15.86:0/3464964004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 msgr2=0x55c2f776bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f7053e00640 1 --2- 172.21.15.86:0/3464964004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f776bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f7053e00640 1 -- 172.21.15.86:0/3464964004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 msgr2=0x55c2f7754000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f7053e00640 1 --2- 172.21.15.86:0/3464964004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f7754000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f7053e00640 1 -- 172.21.15.86:0/3464964004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f77552a0 con 0x55c2f775fcf0 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f7053400640 1 --2- 172.21.15.86:0/3464964004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f7754000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f7053e00640 1 --2- 172.21.15.86:0/3464964004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f7754540 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f704c09c850 tx=0x7f704c09b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c3854e2bbf8479 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f703f400640 1 -- 172.21.15.86:0/3464964004 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0b6040 con 0x55c2f775fcf0 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f703f400640 1 -- 172.21.15.86:0/3464964004 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f704c0989e0 con 0x55c2f775fcf0 2024-04-17T07:17:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.769+0000 7f703f400640 1 -- 172.21.15.86:0/3464964004 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c09bfa0 con 0x55c2f775fcf0 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.770+0000 7f705d00c740 1 -- 172.21.15.86:0/3464964004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 msgr2=0x55c2f7754540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.770+0000 7f705d00c740 1 --2- 172.21.15.86:0/3464964004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f7754540 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f704c09c850 tx=0x7f704c09b060 comp rx=0 tx=0).stop 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.770+0000 7f705d00c740 1 -- 172.21.15.86:0/3464964004 shutdown_connections 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.770+0000 7f705d00c740 1 --2- 172.21.15.86:0/3464964004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775fcf0 0x55c2f7754540 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.770+0000 7f705d00c740 1 --2- 172.21.15.86:0/3464964004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f776bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.770+0000 7f705d00c740 1 --2- 172.21.15.86:0/3464964004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775a190 0x55c2f7754000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.770+0000 7f705d00c740 1 -- 172.21.15.86:0/3464964004 >> 172.21.15.86:0/3464964004 conn(0x55c2f7755700 msgr2=0x55c2f7764620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.770+0000 7f705d00c740 1 -- 172.21.15.86:0/3464964004 shutdown_connections 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.770+0000 7f705d00c740 1 -- 172.21.15.86:0/3464964004 wait complete. 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.771+0000 7f705d00c740 1 -- start start 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.771+0000 7f705d00c740 1 Processor -- start 2024-04-17T07:17:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f705d00c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775a190 0x55c2f7769500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f705d00c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f7769a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f705d00c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775fcf0 0x55c2f776aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f705d00c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b24d0 con 0x55c2f775a190 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f705d00c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b1e00 con 0x55c2f775fcf0 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f705d00c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70541af030 con 0x55c2f775ab80 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f705d00c740 10 client.0 _ll_register_callbacks cb 0x7f704c0077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7053400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775a190 0x55c2f7769500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7053400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775a190 0x55c2f7769500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58050/0 (socket says 172.21.15.86:58050) 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7053400640 1 -- 172.21.15.86:0/697833096 learned_addr learned my addr 172.21.15.86:0/697833096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7052a00640 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f7769a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7053400640 1 -- 172.21.15.86:0/697833096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 msgr2=0x55c2f7769a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7053e00640 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775fcf0 0x55c2f776aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7053400640 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f7769a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7053400640 1 -- 172.21.15.86:0/697833096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775fcf0 msgr2=0x55c2f776aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7053400640 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775fcf0 0x55c2f776aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7053400640 1 -- 172.21.15.86:0/697833096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f776b040 con 0x55c2f775a190 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7052a00640 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f7769a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.772+0000 7f7053e00640 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775fcf0 0x55c2f776aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.773+0000 7f7053400640 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775a190 0x55c2f7769500 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f704402c240 tx=0x7f704402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8badad3041c3a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.773+0000 7f703cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.773+0000 7f703cc00640 1 -- 172.21.15.86:0/697833096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044044040 con 0x55c2f775a190 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.773+0000 7f705d00c740 1 -- 172.21.15.86:0/697833096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704c0989e0 con 0x55c2f775a190 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.773+0000 7f703cc00640 1 -- 172.21.15.86:0/697833096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7044053bd0 con 0x55c2f775a190 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.773+0000 7f703cc00640 1 -- 172.21.15.86:0/697833096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f704402cca0 con 0x55c2f775a190 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.773+0000 7f705d00c740 10 client.16395 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.773+0000 7f705d00c740 1 -- 172.21.15.86:0/697833096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c2f77d6d80 con 0x55c2f775a190 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.774+0000 7f703cc00640 1 -- 172.21.15.86:0/697833096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f704c0989e0 con 0x55c2f775a190 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.774+0000 7f703cc00640 1 -- 172.21.15.86:0/697833096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f70440054d0 con 0x55c2f775a190 2024-04-17T07:17:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.774+0000 7f705d00c740 10 client.16395 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.774+0000 7f705d00c740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:17:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.774+0000 7f705d00c740 20 client.16395 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.774+0000 7f705d00c740 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c2f77d7340 0x55c2f77d9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.774+0000 7f705d00c740 4 client.16395 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:17:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.774+0000 7f705d00c740 1 -- 172.21.15.86:0/697833096 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2f76c89b0 con 0x55c2f77d7340 2024-04-17T07:17:27.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:27 smithi103 ceph-mon[35495]: pgmap v533: 129 pgs: 129 active+clean; 468 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:17:27.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.775+0000 7f7052a00640 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c2f77d7340 0x55c2f77d9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:27.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.775+0000 7f703cc00640 10 client.16395 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.775+0000 7f7052a00640 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c2f77d7340 0x55c2f77d9730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54cf06af74451ad2 server_cookie=76e4e2a2fe24ae51 in_seq=0 out_seq=0 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f703cc00640 1 -- 172.21.15.86:0/697833096 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c2f76c89b0 con 0x55c2f77d7340 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f703cc00640 10 client.16395 handle_command_reply: tid=0 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 client.16395 shutdown 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 -- 172.21.15.86:0/697833096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775a190 msgr2=0x55c2f7769500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775a190 0x55c2f7769500 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f704402c240 tx=0x7f704402bc30 comp rx=0 tx=0).stop 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 -- 172.21.15.86:0/697833096 shutdown_connections 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c2f77d7340 0x55c2f77d9730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c2f775fcf0 0x55c2f776aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2f775ab80 0x55c2f7769a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 --2- 172.21.15.86:0/697833096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2f775a190 0x55c2f7769500 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 -- 172.21.15.86:0/697833096 >> 172.21.15.86:0/697833096 conn(0x55c2f7755700 msgr2=0x55c2f7754000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f703cc00640 0 client.16395 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 -- 172.21.15.86:0/697833096 shutdown_connections 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 1 -- 172.21.15.86:0/697833096 wait complete. 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.776+0000 7f705d00c740 20 client.16395 trim_cache size 0 max 16384 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stdout: "6ac6bb36-7f1a-4db8-9f64-fedfdb06f93a": { 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "6ac6bb36-7f1a-4db8-9f64-fedfdb06f93a", 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:17:27.779 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:17:27.780 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:17:27.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 -- 172.21.15.86:0/3286977966 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70380a3bb0 msgr2=0x7f70380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70380a3bb0 0x7f70380a6070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55c2f775b3e0 tx=0x7f7040004730 comp rx=0 tx=0).stop 2024-04-17T07:17:27.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 -- 172.21.15.86:0/3286977966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7054109e60 msgr2=0x7f70541a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:27.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7054109e60 0x7f70541a5d90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f704c00cce0 tx=0x7f704c00a620 comp rx=0 tx=0).stop 2024-04-17T07:17:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 -- 172.21.15.86:0/3286977966 shutdown_connections 2024-04-17T07:17:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70380a3bb0 0x7f70380a6070 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7054109e60 0x7f70541a5d90 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7054075fa0 0x7f705419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 --2- 172.21.15.86:0/3286977966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70540749c0 0x7f705419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 -- 172.21.15.86:0/3286977966 >> 172.21.15.86:0/3286977966 conn(0x7f70540fd510 msgr2=0x7f7054101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 -- 172.21.15.86:0/3286977966 shutdown_connections 2024-04-17T07:17:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:27.777+0000 7f705a200640 1 -- 172.21.15.86:0/3286977966 wait complete. 2024-04-17T07:17:27.794 INFO:tasks.cephfs.filesystem:scrub status for tag:6ac6bb36-7f1a-4db8-9f64-fedfdb06f93a - {'path': '/', 'tag': '6ac6bb36-7f1a-4db8-9f64-fedfdb06f93a', 'options': 'recursive,force'} 2024-04-17T07:17:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:28 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:17:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:28 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:17:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:28 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:17:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:28 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:17:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:28 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:17:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:28 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:17:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:28 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:17:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:28 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:17:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:28 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:17:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:28 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:17:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:28 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:17:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:28 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:17:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:29 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:17:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:29 smithi086 ceph-mon[31733]: pgmap v534: 129 pgs: 129 active+clean; 484 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-04-17T07:17:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:29 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:17:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:29 smithi098 ceph-mon[35418]: pgmap v534: 129 pgs: 129 active+clean; 484 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-04-17T07:17:29.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:29 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:17:29.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:29 smithi103 ceph-mon[35495]: pgmap v534: 129 pgs: 129 active+clean; 484 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-04-17T07:17:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:31 smithi086 ceph-mon[31733]: pgmap v535: 129 pgs: 129 active+clean; 488 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:17:31.628 INFO:teuthology.orchestra.run.smithi086.stderr:2000000 of 3200000 tuples (62%) done (elapsed 81.17 s, remaining 48.70 s) 2024-04-17T07:17:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:31 smithi098 ceph-mon[35418]: pgmap v535: 129 pgs: 129 active+clean; 488 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:17:31.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:31 smithi103 ceph-mon[35495]: pgmap v535: 129 pgs: 129 active+clean; 488 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:17:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:33 smithi086 ceph-mon[31733]: pgmap v536: 129 pgs: 129 active+clean; 502 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:17:33.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:33 smithi098 ceph-mon[35418]: pgmap v536: 129 pgs: 129 active+clean; 502 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:17:33.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:33 smithi103 ceph-mon[35495]: pgmap v536: 129 pgs: 129 active+clean; 502 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-04-17T07:17:35.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:35 smithi086 ceph-mon[31733]: pgmap v537: 129 pgs: 129 active+clean; 506 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 18 op/s 2024-04-17T07:17:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:35 smithi098 ceph-mon[35418]: pgmap v537: 129 pgs: 129 active+clean; 506 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 18 op/s 2024-04-17T07:17:35.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:35 smithi103 ceph-mon[35495]: pgmap v537: 129 pgs: 129 active+clean; 506 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 18 op/s 2024-04-17T07:17:36.194 INFO:teuthology.orchestra.run.smithi086.stderr:2100000 of 3200000 tuples (65%) done (elapsed 85.74 s, remaining 44.91 s) 2024-04-17T07:17:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:37 smithi086 ceph-mon[31733]: pgmap v538: 129 pgs: 129 active+clean; 526 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 21 op/s 2024-04-17T07:17:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:37 smithi098 ceph-mon[35418]: pgmap v538: 129 pgs: 129 active+clean; 526 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 21 op/s 2024-04-17T07:17:37.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:37 smithi103 ceph-mon[35495]: pgmap v538: 129 pgs: 129 active+clean; 526 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 21 op/s 2024-04-17T07:17:39.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:39 smithi086 ceph-mon[31733]: pgmap v539: 129 pgs: 129 active+clean; 531 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:17:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:39 smithi098 ceph-mon[35418]: pgmap v539: 129 pgs: 129 active+clean; 531 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:17:39.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:39 smithi103 ceph-mon[35495]: pgmap v539: 129 pgs: 129 active+clean; 531 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:17:41.106 INFO:teuthology.orchestra.run.smithi086.stderr:2200000 of 3200000 tuples (68%) done (elapsed 90.65 s, remaining 41.21 s) 2024-04-17T07:17:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:41 smithi086 ceph-mon[31733]: pgmap v540: 129 pgs: 129 active+clean; 539 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.8 MiB/s wr, 18 op/s 2024-04-17T07:17:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:41 smithi098 ceph-mon[35418]: pgmap v540: 129 pgs: 129 active+clean; 539 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.8 MiB/s wr, 18 op/s 2024-04-17T07:17:41.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:41 smithi103 ceph-mon[35495]: pgmap v540: 129 pgs: 129 active+clean; 539 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.8 MiB/s wr, 18 op/s 2024-04-17T07:17:43.579 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:43 smithi103 ceph-mon[35495]: pgmap v541: 129 pgs: 129 active+clean; 555 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-17T07:17:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:43 smithi086 ceph-mon[31733]: pgmap v541: 129 pgs: 129 active+clean; 555 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-17T07:17:43.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:43 smithi098 ceph-mon[35418]: pgmap v541: 129 pgs: 129 active+clean; 555 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-17T07:17:45.226 INFO:teuthology.orchestra.run.smithi086.stderr:2300000 of 3200000 tuples (71%) done (elapsed 94.77 s, remaining 37.08 s) 2024-04-17T07:17:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:45 smithi086 ceph-mon[31733]: pgmap v542: 129 pgs: 129 active+clean; 560 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 17 op/s 2024-04-17T07:17:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:45 smithi098 ceph-mon[35418]: pgmap v542: 129 pgs: 129 active+clean; 560 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 17 op/s 2024-04-17T07:17:45.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:45 smithi103 ceph-mon[35495]: pgmap v542: 129 pgs: 129 active+clean; 560 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 17 op/s 2024-04-17T07:17:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:47 smithi086 ceph-mon[31733]: pgmap v543: 129 pgs: 129 active+clean; 572 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:17:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:47 smithi098 ceph-mon[35418]: pgmap v543: 129 pgs: 129 active+clean; 572 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:17:47.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:47 smithi103 ceph-mon[35495]: pgmap v543: 129 pgs: 129 active+clean; 572 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:17:49.557 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:49 smithi098 ceph-mon[35418]: pgmap v544: 129 pgs: 129 active+clean; 582 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 18 op/s 2024-04-17T07:17:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:49 smithi086 ceph-mon[31733]: pgmap v544: 129 pgs: 129 active+clean; 582 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 18 op/s 2024-04-17T07:17:49.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:49 smithi103 ceph-mon[35495]: pgmap v544: 129 pgs: 129 active+clean; 582 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 18 op/s 2024-04-17T07:17:50.109 INFO:teuthology.orchestra.run.smithi086.stderr:2400000 of 3200000 tuples (75%) done (elapsed 99.66 s, remaining 33.22 s) 2024-04-17T07:17:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:51 smithi086 ceph-mon[31733]: pgmap v545: 129 pgs: 129 active+clean; 582 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-04-17T07:17:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:51 smithi098 ceph-mon[35418]: pgmap v545: 129 pgs: 129 active+clean; 582 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-04-17T07:17:51.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:51 smithi103 ceph-mon[35495]: pgmap v545: 129 pgs: 129 active+clean; 582 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 17 op/s 2024-04-17T07:17:53.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:53 smithi086 ceph-mon[31733]: pgmap v546: 129 pgs: 129 active+clean; 596 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:17:53.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:53 smithi098 ceph-mon[35418]: pgmap v546: 129 pgs: 129 active+clean; 596 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:17:53.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:53 smithi103 ceph-mon[35495]: pgmap v546: 129 pgs: 129 active+clean; 596 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:17:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:2500000 of 3200000 tuples (78%) done (elapsed 104.79 s, remaining 29.34 s) 2024-04-17T07:17:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:55 smithi086 ceph-mon[31733]: pgmap v547: 129 pgs: 129 active+clean; 600 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 MiB/s wr, 15 op/s 2024-04-17T07:17:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:55 smithi098 ceph-mon[35418]: pgmap v547: 129 pgs: 129 active+clean; 600 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 MiB/s wr, 15 op/s 2024-04-17T07:17:55.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:55 smithi103 ceph-mon[35495]: pgmap v547: 129 pgs: 129 active+clean; 600 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.1 MiB/s wr, 15 op/s 2024-04-17T07:17:57.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:57 smithi086 ceph-mon[31733]: pgmap v548: 129 pgs: 129 active+clean; 616 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:17:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:57 smithi098 ceph-mon[35418]: pgmap v548: 129 pgs: 129 active+clean; 616 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:17:57.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:57 smithi103 ceph-mon[35495]: pgmap v548: 129 pgs: 129 active+clean; 616 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:17:57.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:17:57.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:17:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.035+0000 7fcbcca00640 1 -- 172.21.15.86:0/1286542333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc8100d70 msgr2=0x7fcbc8101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.035+0000 7fcbcca00640 1 --2- 172.21.15.86:0/1286542333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc8100d70 0x7fcbc8101190 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009900 tx=0x7fcbb802bc30 comp rx=0 tx=0).stop 2024-04-17T07:17:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.035+0000 7fcbcca00640 1 -- 172.21.15.86:0/1286542333 shutdown_connections 2024-04-17T07:17:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.035+0000 7fcbcca00640 1 --2- 172.21.15.86:0/1286542333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbc8102920 0x7fcbc81091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.035+0000 7fcbcca00640 1 --2- 172.21.15.86:0/1286542333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbc8101f60 0x7fcbc81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.035+0000 7fcbcca00640 1 --2- 172.21.15.86:0/1286542333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc8100d70 0x7fcbc8101190 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.035+0000 7fcbcca00640 1 -- 172.21.15.86:0/1286542333 >> 172.21.15.86:0/1286542333 conn(0x7fcbc80fc300 msgr2=0x7fcbc80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.035+0000 7fcbcca00640 1 -- 172.21.15.86:0/1286542333 shutdown_connections 2024-04-17T07:17:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.035+0000 7fcbcca00640 1 -- 172.21.15.86:0/1286542333 wait complete. 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbcca00640 1 Processor -- start 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbcca00640 1 -- start start 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbcca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbc8100d70 0x7fcbc819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbcca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbc8101f60 0x7fcbc819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbcca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc8102920 0x7fcbc81a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbcca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc810c940 con 0x7fcbc8102920 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbcca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc810c7c0 con 0x7fcbc8101f60 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbcca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc810cac0 con 0x7fcbc8100d70 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc8102920 0x7fcbc81a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc8102920 0x7fcbc81a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56182/0 (socket says 172.21.15.86:56182) 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.037+0000 7fcbc7e00640 1 -- 172.21.15.86:0/1096038722 learned_addr learned my addr 172.21.15.86:0/1096038722 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.038+0000 7fcbc6a00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbc8101f60 0x7fcbc819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.038+0000 7fcbc7400640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbc8100d70 0x7fcbc819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.038+0000 7fcbc7e00640 1 -- 172.21.15.86:0/1096038722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbc8100d70 msgr2=0x7fcbc819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.038+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbc8100d70 0x7fcbc819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.038+0000 7fcbc7e00640 1 -- 172.21.15.86:0/1096038722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbc8101f60 msgr2=0x7fcbc819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.038+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbc8101f60 0x7fcbc819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.038+0000 7fcbc7e00640 1 -- 172.21.15.86:0/1096038722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc81a1fb0 con 0x7fcbc8102920 2024-04-17T07:17:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.038+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc8102920 0x7fcbc81a1940 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000cad0 tx=0x7fcbc000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.039+0000 7fcbbfe00640 1 -- 172.21.15.86:0/1096038722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000ae20 con 0x7fcbc8102920 2024-04-17T07:17:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.039+0000 7fcbcca00640 1 -- 172.21.15.86:0/1096038722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc81a22a0 con 0x7fcbc8102920 2024-04-17T07:17:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.039+0000 7fcbbfe00640 1 -- 172.21.15.86:0/1096038722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbc0005020 con 0x7fcbc8102920 2024-04-17T07:17:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.039+0000 7fcbbfe00640 1 -- 172.21.15.86:0/1096038722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000f6b0 con 0x7fcbc8102920 2024-04-17T07:17:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.039+0000 7fcbcca00640 1 -- 172.21.15.86:0/1096038722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc81a2870 con 0x7fcbc8102920 2024-04-17T07:17:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.042+0000 7fcbd0c26740 1 Processor -- start 2024-04-17T07:17:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.043+0000 7fcbbfe00640 1 -- 172.21.15.86:0/1096038722 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fcbc00051d0 con 0x7fcbc8102920 2024-04-17T07:17:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.043+0000 7fcbd0c26740 1 -- start start 2024-04-17T07:17:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.043+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880370 0x556cbb880790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.043+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb880d60 0x556cbb885860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.043+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb888360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.044+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556cbb7e7120 con 0x556cbb885ed0 2024-04-17T07:17:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.044+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556cbb78d340 con 0x556cbb880d60 2024-04-17T07:17:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.044+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556cbb7e30e0 con 0x556cbb880370 2024-04-17T07:17:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.044+0000 7fcbc6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb888360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.044+0000 7fcbc6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb888360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56190/0 (socket says 172.21.15.86:56190) 2024-04-17T07:17:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.044+0000 7fcbc6a00640 1 -- 172.21.15.86:0/2050054254 learned_addr learned my addr 172.21.15.86:0/2050054254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.044+0000 7fcbc6a00640 1 --2- 172.21.15.86:0/2050054254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880370 0x556cbb880790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.044+0000 7fcbc7400640 1 --2- 172.21.15.86:0/2050054254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb880d60 0x556cbb885860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.044+0000 7fcbc6a00640 1 -- 172.21.15.86:0/2050054254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880370 msgr2=0x556cbb880790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbc6a00640 1 --2- 172.21.15.86:0/2050054254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880370 0x556cbb880790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbc6a00640 1 -- 172.21.15.86:0/2050054254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb880d60 msgr2=0x556cbb885860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbc6a00640 1 --2- 172.21.15.86:0/2050054254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb880d60 0x556cbb885860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbc6a00640 1 -- 172.21.15.86:0/2050054254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cbb5c37d0 con 0x556cbb885ed0 2024-04-17T07:17:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbbfe00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcbac0a3c30 0x7fcbac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbc6a00640 1 --2- 172.21.15.86:0/2050054254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb888360 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400ad50 tx=0x7fcbb400a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf73ec7c4988b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbbfe00640 1 -- 172.21.15.86:0/1096038722 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcbc00d0890 con 0x7fcbc8102920 2024-04-17T07:17:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbc7400640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcbac0a3c30 0x7fcbac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbb3e00640 1 -- 172.21.15.86:0/2050054254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb401b070 con 0x556cbb885ed0 2024-04-17T07:17:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbb3e00640 1 -- 172.21.15.86:0/2050054254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbb4011c70 con 0x556cbb885ed0 2024-04-17T07:17:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.045+0000 7fcbb3e00640 1 -- 172.21.15.86:0/2050054254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40145f0 con 0x556cbb885ed0 2024-04-17T07:17:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.046+0000 7fcbd0c26740 1 -- 172.21.15.86:0/2050054254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 msgr2=0x556cbb888360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.046+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/2050054254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb888360 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400ad50 tx=0x7fcbb400a2b0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.046+0000 7fcbc7400640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcbac0a3c30 0x7fcbac0a60f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcbb802c240 tx=0x7fcbb8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.046+0000 7fcbd0c26740 1 -- 172.21.15.86:0/2050054254 shutdown_connections 2024-04-17T07:17:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.046+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/2050054254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb888360 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.046+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/2050054254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb880d60 0x556cbb885860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.046+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/2050054254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880370 0x556cbb880790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.046+0000 7fcbd0c26740 1 -- 172.21.15.86:0/2050054254 >> 172.21.15.86:0/2050054254 conn(0x556cbb87b8e0 msgr2=0x556cbb87bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.046+0000 7fcbd0c26740 1 -- 172.21.15.86:0/2050054254 shutdown_connections 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.047+0000 7fcbd0c26740 1 -- 172.21.15.86:0/2050054254 wait complete. 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.047+0000 7fcbd0c26740 1 -- start start 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.047+0000 7fcbd0c26740 1 Processor -- start 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb880370 0x556cbb88a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb88ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb891e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556cbb888e80 con 0x556cbb885ed0 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556cbb7a0710 con 0x556cbb880370 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556cbb889000 con 0x556cbb880d60 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbd0c26740 10 client.0 _ll_register_callbacks cb 0x556cbb79baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb891e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb891e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56194/0 (socket says 172.21.15.86:56194) 2024-04-17T07:17:58.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbc7e00640 1 -- 172.21.15.86:0/1143522107 learned_addr learned my addr 172.21.15.86:0/1143522107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbc7400640 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb880370 0x556cbb88a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbc6a00640 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb88ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.048+0000 7fcbc7e00640 1 -- 172.21.15.86:0/1143522107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 msgr2=0x556cbb88ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb88ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbc7e00640 1 -- 172.21.15.86:0/1143522107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb880370 msgr2=0x556cbb88a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb880370 0x556cbb88a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbc7e00640 1 -- 172.21.15.86:0/1143522107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cbb4f93f0 con 0x556cbb885ed0 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb891e40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0099160 tx=0x7fcbc009cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80bcaa8dc202be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbb0c00640 1 -- 172.21.15.86:0/1143522107 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc001d040 con 0x556cbb885ed0 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbb0c00640 1 -- 172.21.15.86:0/1143522107 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcbc0098e00 con 0x556cbb885ed0 2024-04-17T07:17:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbd0c26740 1 -- 172.21.15.86:0/1143522107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb4011c70 con 0x556cbb885ed0 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbb0c00640 1 -- 172.21.15.86:0/1143522107 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcbc009bbe0 con 0x556cbb885ed0 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.049+0000 7fcbd0c26740 10 client.16407 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.050+0000 7fcbd0c26740 1 -- 172.21.15.86:0/1143522107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556cbb4b90f0 con 0x556cbb885ed0 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.050+0000 7fcbb0c00640 1 -- 172.21.15.86:0/1143522107 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbb4011c70 con 0x556cbb885ed0 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.050+0000 7fcbb0c00640 1 -- 172.21.15.86:0/1143522107 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcbc009db50 con 0x556cbb885ed0 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.050+0000 7fcbd0c26740 10 client.16407 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.050+0000 7fcbd0c26740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.050+0000 7fcbd0c26740 20 client.16407 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.050+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556cbb896770 0x556cbb898c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.050+0000 7fcbd0c26740 4 client.16407 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.050+0000 7fcbd0c26740 1 -- 172.21.15.86:0/1143522107 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cbb7f4b40 con 0x556cbb896770 2024-04-17T07:17:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.051+0000 7fcbc7400640 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556cbb896770 0x556cbb898c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.052+0000 7fcbc7400640 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556cbb896770 0x556cbb898c10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d427f135c400564 server_cookie=fe701607e7b04d70 in_seq=0 out_seq=0 2024-04-17T07:17:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.052+0000 7fcbb0c00640 10 client.16407 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbb0c00640 1 -- 172.21.15.86:0/1143522107 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cbb7f4b40 con 0x556cbb896770 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbb0c00640 10 client.16407 handle_command_reply: tid=0 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 client.16407 shutdown 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 -- 172.21.15.86:0/1143522107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 msgr2=0x556cbb891e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb891e40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0099160 tx=0x7fcbc009cfd0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 -- 172.21.15.86:0/1143522107 shutdown_connections 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556cbb896770 0x556cbb898c10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb885ed0 0x556cbb891e40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb88ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/1143522107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb880370 0x556cbb88a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbb0c00640 0 client.16407 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 -- 172.21.15.86:0/1143522107 >> 172.21.15.86:0/1143522107 conn(0x556cbb87b8e0 msgr2=0x556cbb87aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 -- 172.21.15.86:0/1143522107 shutdown_connections 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 1 -- 172.21.15.86:0/1143522107 wait complete. 2024-04-17T07:17:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.054+0000 7fcbd0c26740 20 client.16407 trim_cache size 0 max 16384 2024-04-17T07:17:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.076+0000 7fcbd0c26740 1 Processor -- start 2024-04-17T07:17:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.076+0000 7fcbd0c26740 1 -- start start 2024-04-17T07:17:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.076+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb880370 0x556cbb881780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.076+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb891e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.076+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 0x556cbb87a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.076+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81acc40 con 0x556cbb880370 2024-04-17T07:17:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.076+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81ae780 con 0x556cbb885ed0 2024-04-17T07:17:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.076+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81ad310 con 0x556cbb880d60 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb880370 0x556cbb881780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb880370 0x556cbb881780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56198/0 (socket says 172.21.15.86:56198) 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc7400640 1 -- 172.21.15.86:0/434429044 learned_addr learned my addr 172.21.15.86:0/434429044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc6a00640 1 --2- 172.21.15.86:0/434429044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb891e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/434429044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 0x556cbb87a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc7400640 1 -- 172.21.15.86:0/434429044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 msgr2=0x556cbb891e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc7400640 1 --2- 172.21.15.86:0/434429044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb891e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc7400640 1 -- 172.21.15.86:0/434429044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 msgr2=0x556cbb87a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc7400640 1 --2- 172.21.15.86:0/434429044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 0x556cbb87a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc7400640 1 -- 172.21.15.86:0/434429044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cbb881140 con 0x556cbb880370 2024-04-17T07:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.077+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/434429044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 0x556cbb87a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.078+0000 7fcbc7400640 1 --2- 172.21.15.86:0/434429044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb880370 0x556cbb881780 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8035000 tx=0x7fcbb803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e989912ce711bfb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.078+0000 7fcbb3400640 1 -- 172.21.15.86:0/434429044 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8052b50 con 0x556cbb880370 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.078+0000 7fcbb3400640 1 -- 172.21.15.86:0/434429044 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbb8057070 con 0x556cbb880370 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.078+0000 7fcbb3400640 1 -- 172.21.15.86:0/434429044 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8052850 con 0x556cbb880370 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.079+0000 7fcbd0c26740 1 -- 172.21.15.86:0/434429044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb880370 msgr2=0x556cbb881780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.079+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/434429044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb880370 0x556cbb881780 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8035000 tx=0x7fcbb803f000 comp rx=0 tx=0).stop 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.079+0000 7fcbd0c26740 1 -- 172.21.15.86:0/434429044 shutdown_connections 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.079+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/434429044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 0x556cbb87a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.079+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/434429044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb891e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.079+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/434429044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb880370 0x556cbb881780 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.079+0000 7fcbd0c26740 1 -- 172.21.15.86:0/434429044 >> 172.21.15.86:0/434429044 conn(0x556cbb87b8e0 msgr2=0x556cbb8822d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.079+0000 7fcbd0c26740 1 -- 172.21.15.86:0/434429044 shutdown_connections 2024-04-17T07:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.079+0000 7fcbd0c26740 1 -- 172.21.15.86:0/434429044 wait complete. 2024-04-17T07:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.080+0000 7fcbd0c26740 1 -- start start 2024-04-17T07:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.080+0000 7fcbd0c26740 1 Processor -- start 2024-04-17T07:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.080+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb88f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.080+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 0x556cbb88fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.080+0000 7fcbd0c26740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb890b90 0x556cbb891010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.080+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81ae0b0 con 0x556cbb890b90 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.080+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81ad9e0 con 0x556cbb885ed0 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.080+0000 7fcbd0c26740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81aac10 con 0x556cbb880d60 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.080+0000 7fcbd0c26740 10 client.0 _ll_register_callbacks cb 0x7fcbc0098370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb890b90 0x556cbb891010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb890b90 0x556cbb891010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56200/0 (socket says 172.21.15.86:56200) 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc7e00640 1 -- 172.21.15.86:0/14806782 learned_addr learned my addr 172.21.15.86:0/14806782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc6a00640 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 0x556cbb88fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc7400640 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb88f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc7e00640 1 -- 172.21.15.86:0/14806782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 msgr2=0x556cbb88f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb88f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc7e00640 1 -- 172.21.15.86:0/14806782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 msgr2=0x556cbb88fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 0x556cbb88fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc7e00640 1 -- 172.21.15.86:0/14806782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cbb891550 con 0x556cbb890b90 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.081+0000 7fcbc7e00640 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb890b90 0x556cbb891010 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0098bf0 tx=0x7fcbc00a5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ac6158e1a673c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.082+0000 7fcbb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.082+0000 7fcbb0c00640 1 -- 172.21.15.86:0/14806782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc001d040 con 0x556cbb890b90 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.082+0000 7fcbb0c00640 1 -- 172.21.15.86:0/14806782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcbc0098e00 con 0x556cbb890b90 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.082+0000 7fcbd0c26740 1 -- 172.21.15.86:0/14806782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb8057070 con 0x556cbb890b90 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.082+0000 7fcbb0c00640 1 -- 172.21.15.86:0/14806782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcbc009c2d0 con 0x556cbb890b90 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.082+0000 7fcbd0c26740 10 client.16413 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.082+0000 7fcbd0c26740 1 -- 172.21.15.86:0/14806782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556cbb8fd1b0 con 0x556cbb890b90 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.083+0000 7fcbb0c00640 1 -- 172.21.15.86:0/14806782 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbb8057070 con 0x556cbb890b90 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.083+0000 7fcbb0c00640 1 -- 172.21.15.86:0/14806782 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcbc0098550 con 0x556cbb890b90 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.083+0000 7fcbd0c26740 10 client.16413 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.083+0000 7fcbd0c26740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:17:58.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.083+0000 7fcbd0c26740 20 client.16413 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.083+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556cbb8fd980 0x556cbb8ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.083+0000 7fcbd0c26740 4 client.16413 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:17:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.083+0000 7fcbd0c26740 1 -- 172.21.15.86:0/14806782 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556cbb881de0 con 0x556cbb8fd980 2024-04-17T07:17:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.084+0000 7fcbc7400640 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556cbb8fd980 0x556cbb8ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbc7400640 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556cbb8fd980 0x556cbb8ffda0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb9ccb9557b7ea9 server_cookie=4fbb6fcff6671a82 in_seq=0 out_seq=0 2024-04-17T07:17:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbb0c00640 10 client.16413 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbb0c00640 1 -- 172.21.15.86:0/14806782 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556cbb881de0 con 0x556cbb8fd980 2024-04-17T07:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbb0c00640 10 client.16413 handle_command_reply: tid=0 2024-04-17T07:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbd0c26740 1 client.16413 shutdown 2024-04-17T07:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbd0c26740 1 -- 172.21.15.86:0/14806782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb890b90 msgr2=0x556cbb891010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb890b90 0x556cbb891010 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0098bf0 tx=0x7fcbc00a5530 comp rx=0 tx=0).stop 2024-04-17T07:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbd0c26740 1 -- 172.21.15.86:0/14806782 shutdown_connections 2024-04-17T07:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556cbb8fd980 0x556cbb8ffda0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556cbb890b90 0x556cbb891010 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556cbb885ed0 0x556cbb88fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbd0c26740 1 --2- 172.21.15.86:0/14806782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cbb880d60 0x556cbb88f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.085+0000 7fcbd0c26740 1 -- 172.21.15.86:0/14806782 >> 172.21.15.86:0/14806782 conn(0x556cbb87b8e0 msgr2=0x556cbb88a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.086+0000 7fcbb0c00640 0 client.16413 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.086+0000 7fcbd0c26740 1 -- 172.21.15.86:0/14806782 shutdown_connections 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.086+0000 7fcbd0c26740 1 -- 172.21.15.86:0/14806782 wait complete. 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.086+0000 7fcbd0c26740 20 client.16413 trim_cache size 0 max 16384 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:17:58.089 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:17:58.090 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:17:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.086+0000 7fcbcca00640 1 -- 172.21.15.86:0/1096038722 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcbac0a3c30 msgr2=0x7fcbac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.086+0000 7fcbcca00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcbac0a3c30 0x7fcbac0a60f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcbb802c240 tx=0x7fcbb8005fd0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.087+0000 7fcbcca00640 1 -- 172.21.15.86:0/1096038722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc8102920 msgr2=0x7fcbc81a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.087+0000 7fcbcca00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc8102920 0x7fcbc81a1940 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000cad0 tx=0x7fcbc000cf90 comp rx=0 tx=0).stop 2024-04-17T07:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.087+0000 7fcbcca00640 1 -- 172.21.15.86:0/1096038722 shutdown_connections 2024-04-17T07:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.087+0000 7fcbcca00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcbac0a3c30 0x7fcbac0a60f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.087+0000 7fcbcca00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbc8102920 0x7fcbc81a1940 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.087+0000 7fcbcca00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbc8101f60 0x7fcbc819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.087+0000 7fcbcca00640 1 --2- 172.21.15.86:0/1096038722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcbc8100d70 0x7fcbc819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.087+0000 7fcbcca00640 1 -- 172.21.15.86:0/1096038722 >> 172.21.15.86:0/1096038722 conn(0x7fcbc80fc300 msgr2=0x7fcbc8069140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.087+0000 7fcbcca00640 1 -- 172.21.15.86:0/1096038722 shutdown_connections 2024-04-17T07:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.087+0000 7fcbcca00640 1 -- 172.21.15.86:0/1096038722 wait complete. 2024-04-17T07:17:58.110 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:17:58.110 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:17:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.336+0000 7fc904e00640 1 -- 172.21.15.86:0/2648026264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc900101fc0 msgr2=0x7fc9001023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.336+0000 7fc904e00640 1 --2- 172.21.15.86:0/2648026264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc900101fc0 0x7fc9001023e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009900 tx=0x7fc8f002bc30 comp rx=0 tx=0).stop 2024-04-17T07:17:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.336+0000 7fc904e00640 1 -- 172.21.15.86:0/2648026264 shutdown_connections 2024-04-17T07:17:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.336+0000 7fc904e00640 1 --2- 172.21.15.86:0/2648026264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc900103b70 0x7fc90010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.336+0000 7fc904e00640 1 --2- 172.21.15.86:0/2648026264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9001031b0 0x7fc900103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.336+0000 7fc904e00640 1 --2- 172.21.15.86:0/2648026264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc900101fc0 0x7fc9001023e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.336+0000 7fc904e00640 1 -- 172.21.15.86:0/2648026264 >> 172.21.15.86:0/2648026264 conn(0x7fc9000fd510 msgr2=0x7fc9000ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.336+0000 7fc904e00640 1 -- 172.21.15.86:0/2648026264 shutdown_connections 2024-04-17T07:17:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.336+0000 7fc904e00640 1 -- 172.21.15.86:0/2648026264 wait complete. 2024-04-17T07:17:58.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.337+0000 7fc904e00640 1 Processor -- start 2024-04-17T07:17:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.337+0000 7fc904e00640 1 -- start start 2024-04-17T07:17:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.338+0000 7fc904e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc900101fc0 0x7fc900075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.338+0000 7fc904e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9001031b0 0x7fc900074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.338+0000 7fc904e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc900103b70 0x7fc900074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.338+0000 7fc904e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc90010db50 con 0x7fc900103b70 2024-04-17T07:17:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.338+0000 7fc904e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc90010d9d0 con 0x7fc900101fc0 2024-04-17T07:17:58.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.338+0000 7fc904e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc90010dcd0 con 0x7fc9001031b0 2024-04-17T07:17:58.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.339+0000 7fc8fe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc900103b70 0x7fc900074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.339+0000 7fc8fd400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9001031b0 0x7fc900074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.339+0000 7fc8fe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc900103b70 0x7fc900074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56220/0 (socket says 172.21.15.86:56220) 2024-04-17T07:17:58.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.339+0000 7fc8fe800640 1 -- 172.21.15.86:0/3891574540 learned_addr learned my addr 172.21.15.86:0/3891574540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.339+0000 7fc8fde00640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc900101fc0 0x7fc900075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.339+0000 7fc8fe800640 1 -- 172.21.15.86:0/3891574540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9001031b0 msgr2=0x7fc900074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.339+0000 7fc8fe800640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9001031b0 0x7fc900074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.339+0000 7fc8fe800640 1 -- 172.21.15.86:0/3891574540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc900101fc0 msgr2=0x7fc900075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.339+0000 7fc8fe800640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc900101fc0 0x7fc900075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.340+0000 7fc8fe800640 1 -- 172.21.15.86:0/3891574540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc900075120 con 0x7fc900103b70 2024-04-17T07:17:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.340+0000 7fc8fe800640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc900103b70 0x7fc900074890 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800eaa0 tx=0x7fc8f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.340+0000 7fc8f6a00640 1 -- 172.21.15.86:0/3891574540 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800cdb0 con 0x7fc900103b70 2024-04-17T07:17:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.340+0000 7fc904e00640 1 -- 172.21.15.86:0/3891574540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9001a2300 con 0x7fc900103b70 2024-04-17T07:17:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.340+0000 7fc8f6a00640 1 -- 172.21.15.86:0/3891574540 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8f8005020 con 0x7fc900103b70 2024-04-17T07:17:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.340+0000 7fc8f6a00640 1 -- 172.21.15.86:0/3891574540 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80116d0 con 0x7fc900103b70 2024-04-17T07:17:58.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.340+0000 7fc904e00640 1 -- 172.21.15.86:0/3891574540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a2800 con 0x7fc900103b70 2024-04-17T07:17:58.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.341+0000 7fc904e00640 1 -- 172.21.15.86:0/3891574540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9001093b0 con 0x7fc900103b70 2024-04-17T07:17:58.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.344+0000 7fc8f6a00640 1 -- 172.21.15.86:0/3891574540 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc8f80051d0 con 0x7fc900103b70 2024-04-17T07:17:58.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.346+0000 7fc8f6a00640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8e40a3bb0 0x7fc8e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.346+0000 7fc8f6a00640 1 -- 172.21.15.86:0/3891574540 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc8f80d08f0 con 0x7fc900103b70 2024-04-17T07:17:58.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.346+0000 7fc8fde00640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8e40a3bb0 0x7fc8e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.347+0000 7fc8fde00640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8e40a3bb0 0x7fc8e40a6070 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc8f002c240 tx=0x7fc8f002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.352+0000 7fc8f6a00640 1 -- 172.21.15.86:0/3891574540 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc8f8016030 con 0x7fc900103b70 2024-04-17T07:17:58.528 INFO:teuthology.orchestra.run.smithi086.stderr:2600000 of 3200000 tuples (81%) done (elapsed 108.07 s, remaining 24.94 s) 2024-04-17T07:17:58.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.631+0000 7fc904e00640 1 -- 172.21.15.86:0/3891574540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc900062b60 con 0x7fc900103b70 2024-04-17T07:17:58.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.632+0000 7fc8f6a00640 1 -- 172.21.15.86:0/3891574540 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fc8f809c900 con 0x7fc900103b70 2024-04-17T07:17:58.632 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:17:58.632 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:17:58.632 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:17:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 -- 172.21.15.86:0/3891574540 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8e40a3bb0 msgr2=0x7fc8e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8e40a3bb0 0x7fc8e40a6070 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc8f002c240 tx=0x7fc8f002c7e0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 -- 172.21.15.86:0/3891574540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc900103b70 msgr2=0x7fc900074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc900103b70 0x7fc900074890 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800eaa0 tx=0x7fc8f800ef60 comp rx=0 tx=0).stop 2024-04-17T07:17:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 -- 172.21.15.86:0/3891574540 shutdown_connections 2024-04-17T07:17:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc8e40a3bb0 0x7fc8e40a6070 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc900103b70 0x7fc900074890 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9001031b0 0x7fc900074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 --2- 172.21.15.86:0/3891574540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc900101fc0 0x7fc900075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 -- 172.21.15.86:0/3891574540 >> 172.21.15.86:0/3891574540 conn(0x7fc9000fd510 msgr2=0x7fc900106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 -- 172.21.15.86:0/3891574540 shutdown_connections 2024-04-17T07:17:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.635+0000 7fc904e00640 1 -- 172.21.15.86:0/3891574540 wait complete. 2024-04-17T07:17:58.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:17:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.875+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3818997572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c0105220 msgr2=0x7ff9c0107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.875+0000 7ff9c5a00640 1 --2- 172.21.15.86:0/3818997572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c0105220 0x7ff9c0107610 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009900 tx=0x7ff9ac02bc30 comp rx=0 tx=0).stop 2024-04-17T07:17:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.876+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3818997572 shutdown_connections 2024-04-17T07:17:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.876+0000 7ff9c5a00640 1 --2- 172.21.15.86:0/3818997572 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c0107b50 0x7ff9c0109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.876+0000 7ff9c5a00640 1 --2- 172.21.15.86:0/3818997572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c0105220 0x7ff9c0107610 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.876+0000 7ff9c5a00640 1 --2- 172.21.15.86:0/3818997572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c01028f0 0x7ff9c0104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.876+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3818997572 >> 172.21.15.86:0/3818997572 conn(0x7ff9c00fc300 msgr2=0x7ff9c00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.876+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3818997572 shutdown_connections 2024-04-17T07:17:58.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.876+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3818997572 wait complete. 2024-04-17T07:17:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.877+0000 7ff9c5a00640 1 Processor -- start 2024-04-17T07:17:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.877+0000 7ff9c5a00640 1 -- start start 2024-04-17T07:17:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.877+0000 7ff9c5a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c01028f0 0x7ff9c019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.877+0000 7ff9c5a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c0105220 0x7ff9c019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.877+0000 7ff9c5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c0107b50 0x7ff9c01a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.877+0000 7ff9c5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c010d9b0 con 0x7ff9c0107b50 2024-04-17T07:17:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.877+0000 7ff9c5a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c010d830 con 0x7ff9c01028f0 2024-04-17T07:17:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.877+0000 7ff9c5a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c010db30 con 0x7ff9c0105220 2024-04-17T07:17:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9c5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c0107b50 0x7ff9c01a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9c5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c0107b50 0x7ff9c01a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56240/0 (socket says 172.21.15.86:56240) 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9c5000640 1 -- 172.21.15.86:0/3310240956 learned_addr learned my addr 172.21.15.86:0/3310240956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9bf400640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c0105220 0x7ff9c019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c01028f0 0x7ff9c019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9c5000640 1 -- 172.21.15.86:0/3310240956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c0105220 msgr2=0x7ff9c019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9c5000640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c0105220 0x7ff9c019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9c5000640 1 -- 172.21.15.86:0/3310240956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c01028f0 msgr2=0x7ff9c019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9c5000640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c01028f0 0x7ff9c019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9c5000640 1 -- 172.21.15.86:0/3310240956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c01a1f20 con 0x7ff9c0107b50 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.878+0000 7ff9bf400640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c0105220 0x7ff9c019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.879+0000 7ff9c5000640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c0107b50 0x7ff9c01a18b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800eaa0 tx=0x7ff9b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.879+0000 7ff9bcc00640 1 -- 172.21.15.86:0/3310240956 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800cdb0 con 0x7ff9c0107b50 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.879+0000 7ff9bcc00640 1 -- 172.21.15.86:0/3310240956 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9b8005020 con 0x7ff9c0107b50 2024-04-17T07:17:58.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.879+0000 7ff9bcc00640 1 -- 172.21.15.86:0/3310240956 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80116d0 con 0x7ff9c0107b50 2024-04-17T07:17:58.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.879+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3310240956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c01a2210 con 0x7ff9c0107b50 2024-04-17T07:17:58.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.879+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3310240956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c01a26e0 con 0x7ff9c0107b50 2024-04-17T07:17:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.883+0000 7ff9bcc00640 1 -- 172.21.15.86:0/3310240956 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff9b80051d0 con 0x7ff9c0107b50 2024-04-17T07:17:58.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.883+0000 7ff9c9c2c740 1 Processor -- start 2024-04-17T07:17:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.883+0000 7ff9c9c2c740 1 -- start start 2024-04-17T07:17:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829ceed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cef2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cef5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55829ce53020 con 0x55829cef2cc0 2024-04-17T07:17:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55829cdf9460 con 0x55829ceedb50 2024-04-17T07:17:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55829ce4efe0 con 0x55829ceed160 2024-04-17T07:17:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cef5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cef5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56250/0 (socket says 172.21.15.86:56250) 2024-04-17T07:17:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9bf400640 1 -- 172.21.15.86:0/2906821484 learned_addr learned my addr 172.21.15.86:0/2906821484 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2906821484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829ceed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/2906821484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cef2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.884+0000 7ff9bcc00640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9a40a3c30 0x7ff9a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.885+0000 7ff9bf400640 1 -- 172.21.15.86:0/2906821484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 msgr2=0x55829ceed580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.885+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2906821484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829ceed580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.885+0000 7ff9bf400640 1 -- 172.21.15.86:0/2906821484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 msgr2=0x55829cef2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.885+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2906821484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cef2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.885+0000 7ff9bf400640 1 -- 172.21.15.86:0/2906821484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829ce01040 con 0x55829cef2cc0 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.885+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/2906821484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cef2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.885+0000 7ff9bcc00640 1 -- 172.21.15.86:0/3310240956 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff9b80d09b0 con 0x7ff9c0107b50 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.885+0000 7ff9c5000640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9a40a3c30 0x7ff9a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.885+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2906821484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cef5150 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac005940 tx=0x7ff9ac0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6d08104ca839e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.885+0000 7ff9b5600640 1 -- 172.21.15.86:0/2906821484 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac049070 con 0x55829cef2cc0 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.886+0000 7ff9b5600640 1 -- 172.21.15.86:0/2906821484 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9ac02c980 con 0x55829cef2cc0 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.886+0000 7ff9b5600640 1 -- 172.21.15.86:0/2906821484 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac0436b0 con 0x55829cef2cc0 2024-04-17T07:17:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.886+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2906821484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 msgr2=0x55829cef5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.886+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2906821484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cef5150 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac005940 tx=0x7ff9ac0058b0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.886+0000 7ff9c5000640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9a40a3c30 0x7ff9a40a60f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff9b809c8a0 tx=0x7ff9b800cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.887+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2906821484 shutdown_connections 2024-04-17T07:17:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.887+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2906821484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cef5150 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.887+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2906821484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cef2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.887+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2906821484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829ceed580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.887+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2906821484 >> 172.21.15.86:0/2906821484 conn(0x55829cee86d0 msgr2=0x55829cee8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.887+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2906821484 shutdown_connections 2024-04-17T07:17:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.887+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2906821484 wait complete. 2024-04-17T07:17:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.887+0000 7ff9c9c2c740 1 -- start start 2024-04-17T07:17:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.887+0000 7ff9c9c2c740 1 Processor -- start 2024-04-17T07:17:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829cef76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cef7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cefec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55829cef5c10 con 0x55829cef2cc0 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55829ce0c830 con 0x55829ceedb50 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55829cef5d90 con 0x55829ceed160 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9c9c2c740 10 client.0 _ll_register_callbacks cb 0x55829cddafb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cefec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cefec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56264/0 (socket says 172.21.15.86:56264) 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9bf400640 1 -- 172.21.15.86:0/2043655807 learned_addr learned my addr 172.21.15.86:0/2043655807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.888+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829cef76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cef7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9bf400640 1 -- 172.21.15.86:0/2043655807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 msgr2=0x55829cef76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829cef76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9bf400640 1 -- 172.21.15.86:0/2043655807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 msgr2=0x55829cef7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cef7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9bf400640 1 -- 172.21.15.86:0/2043655807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55829ce10920 con 0x55829cef2cc0 2024-04-17T07:17:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cefec60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac02c840 tx=0x7ff9ac02cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d7804bc2f3b90a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9aa000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9aa000640 1 -- 172.21.15.86:0/2043655807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac057060 con 0x55829cef2cc0 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9aa000640 1 -- 172.21.15.86:0/2043655807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff9ac003a80 con 0x55829cef2cc0 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.889+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2043655807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9ac02c980 con 0x55829cef2cc0 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.890+0000 7ff9aa000640 1 -- 172.21.15.86:0/2043655807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9ac05e050 con 0x55829cef2cc0 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.890+0000 7ff9c9c2c740 10 client.16431 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.890+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2043655807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55829cdde7f0 con 0x55829cef2cc0 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.890+0000 7ff9aa000640 1 -- 172.21.15.86:0/2043655807 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9ac02c980 con 0x55829cef2cc0 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.890+0000 7ff9aa000640 1 -- 172.21.15.86:0/2043655807 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff9ac04f050 con 0x55829cef2cc0 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.890+0000 7ff9c9c2c740 10 client.16431 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.890+0000 7ff9c9c2c740 10 client.16431 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.891+0000 7ff9c9c2c740 20 client.16431 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.891+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55829cf02dc0 0x55829cf051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.891+0000 7ff9c9c2c740 4 client.16431 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.891+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2043655807 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55829ce60a40 con 0x55829cf02dc0 2024-04-17T07:17:58.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.891+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55829cf02dc0 0x55829cf051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.892+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55829cf02dc0 0x55829cf051b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1300f42efb6c75f server_cookie=cedd7958a120c1c3 in_seq=0 out_seq=0 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.892+0000 7ff9aa000640 10 client.16431 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.894+0000 7ff9aa000640 1 -- 172.21.15.86:0/2043655807 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55829ce60a40 con 0x55829cf02dc0 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.894+0000 7ff9aa000640 10 client.16431 handle_command_reply: tid=0 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.894+0000 7ff9c9c2c740 1 client.16431 shutdown 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2043655807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 msgr2=0x55829cefec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cefec60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac02c840 tx=0x7ff9ac02cd90 comp rx=0 tx=0).stop 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2043655807 shutdown_connections 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55829cf02dc0 0x55829cf051b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cefec60 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cef7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2043655807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829cef76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9aa000640 0 client.16431 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:17:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2043655807 >> 172.21.15.86:0/2043655807 conn(0x55829cee86d0 msgr2=0x55829cee7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2043655807 shutdown_connections 2024-04-17T07:17:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2043655807 wait complete. 2024-04-17T07:17:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.895+0000 7ff9c9c2c740 20 client.16431 trim_cache size 0 max 16384 2024-04-17T07:17:58.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.910+0000 7ff9c9c2c740 1 Processor -- start 2024-04-17T07:17:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.910+0000 7ff9c9c2c740 1 -- start start 2024-04-17T07:17:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829ceec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cefec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cee70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01aca70 con 0x55829cef2cc0 2024-04-17T07:17:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01ae5b0 con 0x55829ceedb50 2024-04-17T07:17:58.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01ad140 con 0x55829ceed160 2024-04-17T07:17:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cee70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cee70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56268/0 (socket says 172.21.15.86:56268) 2024-04-17T07:17:58.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9bf400640 1 -- 172.21.15.86:0/2083818731 learned_addr learned my addr 172.21.15.86:0/2083818731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2083818731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829ceec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.911+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/2083818731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cefec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.912+0000 7ff9bf400640 1 -- 172.21.15.86:0/2083818731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 msgr2=0x55829ceec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.912+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2083818731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829ceec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.912+0000 7ff9bf400640 1 -- 172.21.15.86:0/2083818731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 msgr2=0x55829cefec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.912+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2083818731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cefec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.912+0000 7ff9bf400640 1 -- 172.21.15.86:0/2083818731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829cb653f0 con 0x55829cef2cc0 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.912+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/2083818731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cefec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.912+0000 7ff9bf400640 1 --2- 172.21.15.86:0/2083818731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cee70a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac044fd0 tx=0x7ff9ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d36e7df6a923e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.912+0000 7ff9b4c00640 1 -- 172.21.15.86:0/2083818731 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac057070 con 0x55829cef2cc0 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.912+0000 7ff9b4c00640 1 -- 172.21.15.86:0/2083818731 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9ac02c960 con 0x55829cef2cc0 2024-04-17T07:17:58.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.912+0000 7ff9b4c00640 1 -- 172.21.15.86:0/2083818731 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac049040 con 0x55829cef2cc0 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.913+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2083818731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 msgr2=0x55829cee70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.913+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2083818731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cee70a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac044fd0 tx=0x7ff9ac000bc0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.913+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2083818731 shutdown_connections 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.913+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2083818731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829cef2cc0 0x55829cee70a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.913+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2083818731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cefec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.913+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/2083818731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829ceed160 0x55829ceec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.913+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2083818731 >> 172.21.15.86:0/2083818731 conn(0x55829cee86d0 msgr2=0x55829ceeb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.913+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2083818731 shutdown_connections 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.913+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/2083818731 wait complete. 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.914+0000 7ff9c9c2c740 1 -- start start 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.914+0000 7ff9c9c2c740 1 Processor -- start 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.914+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829ceed160 0x55829cefbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.914+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cefc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.914+0000 7ff9c9c2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829cef2cc0 0x55829cefd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.914+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01adee0 con 0x55829ceed160 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.914+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01ad810 con 0x55829ceedb50 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.914+0000 7ff9c9c2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01aaa40 con 0x55829cef2cc0 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.915+0000 7ff9c9c2c740 10 client.0 _ll_register_callbacks cb 0x7ff9ac003920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.915+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829ceed160 0x55829cefbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.915+0000 7ff9bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829ceed160 0x55829cefbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56278/0 (socket says 172.21.15.86:56278) 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.915+0000 7ff9bf400640 1 -- 172.21.15.86:0/690720365 learned_addr learned my addr 172.21.15.86:0/690720365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.915+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cefc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.915+0000 7ff9bf400640 1 -- 172.21.15.86:0/690720365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829cef2cc0 msgr2=0x55829cefd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.915+0000 7ff9bf400640 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829cef2cc0 0x55829cefd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.915+0000 7ff9bf400640 1 -- 172.21.15.86:0/690720365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 msgr2=0x55829cefc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.915+0000 7ff9bf400640 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cefc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.915+0000 7ff9bf400640 1 -- 172.21.15.86:0/690720365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55829cc2f7d0 con 0x55829ceed160 2024-04-17T07:17:58.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.916+0000 7ff9bf400640 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829ceed160 0x55829cefbdb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac047040 tx=0x7ff9ac009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750e7995be68a7ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.916+0000 7ff9aa000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.916+0000 7ff9aa000640 1 -- 172.21.15.86:0/690720365 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac066070 con 0x55829ceed160 2024-04-17T07:17:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.916+0000 7ff9aa000640 1 -- 172.21.15.86:0/690720365 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff9ac0042f0 con 0x55829ceed160 2024-04-17T07:17:58.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.916+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/690720365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9ac02c960 con 0x55829ceed160 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.916+0000 7ff9aa000640 1 -- 172.21.15.86:0/690720365 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9ac049030 con 0x55829ceed160 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.916+0000 7ff9c9c2c740 10 client.16437 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.916+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/690720365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55829cefe990 con 0x55829ceed160 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.917+0000 7ff9aa000640 1 -- 172.21.15.86:0/690720365 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9ac02c960 con 0x55829ceed160 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.917+0000 7ff9aa000640 1 -- 172.21.15.86:0/690720365 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff9ac04f040 con 0x55829ceed160 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.917+0000 7ff9c9c2c740 10 client.16437 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.917+0000 7ff9c9c2c740 10 client.16437 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.917+0000 7ff9c9c2c740 20 client.16437 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.917+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55829cf691a0 0x55829cf6b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.917+0000 7ff9c9c2c740 4 client.16437 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.917+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/690720365 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55829ce5a8b0 con 0x55829cf691a0 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.917+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55829cf691a0 0x55829cf6b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.918+0000 7ff9bfe00640 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55829cf691a0 0x55829cf6b5c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a16c0a52f29c359 server_cookie=556c25420854e94c in_seq=0 out_seq=0 2024-04-17T07:17:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.918+0000 7ff9aa000640 10 client.16437 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9aa000640 1 -- 172.21.15.86:0/690720365 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55829ce5a8b0 con 0x55829cf691a0 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9aa000640 10 client.16437 handle_command_reply: tid=0 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 client.16437 shutdown 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/690720365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829ceed160 msgr2=0x55829cefbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829ceed160 0x55829cefbdb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac047040 tx=0x7ff9ac009640 comp rx=0 tx=0).stop 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/690720365 shutdown_connections 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55829cf691a0 0x55829cf6b5c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55829cef2cc0 0x55829cefd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55829ceedb50 0x55829cefc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 --2- 172.21.15.86:0/690720365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55829ceed160 0x55829cefbdb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9aa000640 0 client.16437 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/690720365 >> 172.21.15.86:0/690720365 conn(0x55829cee86d0 msgr2=0x55829ceec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/690720365 shutdown_connections 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.919+0000 7ff9c9c2c740 1 -- 172.21.15.86:0/690720365 wait complete. 2024-04-17T07:17:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c9c2c740 20 client.16437 trim_cache size 0 max 16384 2024-04-17T07:17:58.922 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:17:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3310240956 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9a40a3c30 msgr2=0x7ff9a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c5a00640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9a40a3c30 0x7ff9a40a60f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff9b809c8a0 tx=0x7ff9b800cf50 comp rx=0 tx=0).stop 2024-04-17T07:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3310240956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c0107b50 msgr2=0x7ff9c01a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c5a00640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c0107b50 0x7ff9c01a18b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800eaa0 tx=0x7ff9b800ef60 comp rx=0 tx=0).stop 2024-04-17T07:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3310240956 shutdown_connections 2024-04-17T07:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c5a00640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9a40a3c30 0x7ff9a40a60f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c5a00640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9c0107b50 0x7ff9c01a18b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c5a00640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9c0105220 0x7ff9c019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c5a00640 1 --2- 172.21.15.86:0/3310240956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c01028f0 0x7ff9c019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.920+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3310240956 >> 172.21.15.86:0/3310240956 conn(0x7ff9c00fc300 msgr2=0x7ff9c0102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.921+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3310240956 shutdown_connections 2024-04-17T07:17:58.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:58.921+0000 7ff9c5a00640 1 -- 172.21.15.86:0/3310240956 wait complete. 2024-04-17T07:17:58.936 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:58 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3891574540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:17:58.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:17:59.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:58 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3891574540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:17:59.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:58 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3891574540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:17:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.168+0000 7f414ac00640 1 -- 172.21.15.86:0/2300456180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441040a0 msgr2=0x7f41441044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.168+0000 7f414ac00640 1 --2- 172.21.15.86:0/2300456180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441040a0 0x7f41441044c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4138009900 tx=0x7f413802bc30 comp rx=0 tx=0).stop 2024-04-17T07:17:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.168+0000 7f414ac00640 1 -- 172.21.15.86:0/2300456180 shutdown_connections 2024-04-17T07:17:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.168+0000 7f414ac00640 1 --2- 172.21.15.86:0/2300456180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4144105c50 0x7f414410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.168+0000 7f414ac00640 1 --2- 172.21.15.86:0/2300456180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4144105290 0x7f4144105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.168+0000 7f414ac00640 1 --2- 172.21.15.86:0/2300456180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441040a0 0x7f41441044c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.168+0000 7f414ac00640 1 -- 172.21.15.86:0/2300456180 >> 172.21.15.86:0/2300456180 conn(0x7f41440ff610 msgr2=0x7f4144101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.168+0000 7f414ac00640 1 -- 172.21.15.86:0/2300456180 shutdown_connections 2024-04-17T07:17:59.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.168+0000 7f414ac00640 1 -- 172.21.15.86:0/2300456180 wait complete. 2024-04-17T07:17:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.169+0000 7f414ac00640 1 Processor -- start 2024-04-17T07:17:59.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.169+0000 7f414ac00640 1 -- start start 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.169+0000 7f414ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441040a0 0x7f414419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.169+0000 7f414ac00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4144105290 0x7f414419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f414ac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4144105c50 0x7f41441a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f414ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f414410fc30 con 0x7f41441040a0 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f414ac00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f414410fab0 con 0x7f4144105290 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f414ac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f414410fdb0 con 0x7f4144105c50 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f4149800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441040a0 0x7f414419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f4149800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441040a0 0x7f414419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56296/0 (socket says 172.21.15.86:56296) 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f4149800640 1 -- 172.21.15.86:0/1264550131 learned_addr learned my addr 172.21.15.86:0/1264550131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f414a200640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4144105c50 0x7f41441a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f4148e00640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4144105290 0x7f414419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f4149800640 1 -- 172.21.15.86:0/1264550131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4144105c50 msgr2=0x7f41441a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f4149800640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4144105c50 0x7f41441a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f4149800640 1 -- 172.21.15.86:0/1264550131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4144105290 msgr2=0x7f414419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f4149800640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4144105290 0x7f414419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f4149800640 1 -- 172.21.15.86:0/1264550131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41441a1e60 con 0x7f41441040a0 2024-04-17T07:17:59.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.170+0000 7f414a200640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4144105c50 0x7f41441a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:59.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.171+0000 7f4149800640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441040a0 0x7f414419a240 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4138005ab0 tx=0x7f41380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.171+0000 7f4142000640 1 -- 172.21.15.86:0/1264550131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138043020 con 0x7f41441040a0 2024-04-17T07:17:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.171+0000 7f414ac00640 1 -- 172.21.15.86:0/1264550131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41441a20f0 con 0x7f41441040a0 2024-04-17T07:17:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.171+0000 7f4142000640 1 -- 172.21.15.86:0/1264550131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41380054c0 con 0x7f41441040a0 2024-04-17T07:17:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.171+0000 7f4142000640 1 -- 172.21.15.86:0/1264550131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138003de0 con 0x7f41441040a0 2024-04-17T07:17:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.171+0000 7f414ac00640 1 -- 172.21.15.86:0/1264550131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41441a2660 con 0x7f41441040a0 2024-04-17T07:17:59.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.175+0000 7f414efba740 1 Processor -- start 2024-04-17T07:17:59.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.175+0000 7f414efba740 1 -- start start 2024-04-17T07:17:59.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.175+0000 7f414efba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f35416580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.175+0000 7f414efba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f35416b50 0x560f3541b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.175+0000 7f414efba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f3541e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.175+0000 7f414efba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f3537c020 con 0x560f3541bcc0 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.175+0000 7f414efba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560f35322460 con 0x560f35416160 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.175+0000 7f414efba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560f35377fe0 con 0x560f35416b50 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.176+0000 7f4148e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f3541e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.176+0000 7f4148e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f3541e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56302/0 (socket says 172.21.15.86:56302) 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.176+0000 7f4148e00640 1 -- 172.21.15.86:0/2772920193 learned_addr learned my addr 172.21.15.86:0/2772920193 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.176+0000 7f4142000640 1 -- 172.21.15.86:0/1264550131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4138046020 con 0x7f41441040a0 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.176+0000 7f414a200640 1 --2- 172.21.15.86:0/2772920193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f35416580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.176+0000 7f4148e00640 1 -- 172.21.15.86:0/2772920193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f35416b50 msgr2=0x560f3541b650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.176+0000 7f4148e00640 1 --2- 172.21.15.86:0/2772920193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f35416b50 0x560f3541b650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.176+0000 7f4148e00640 1 -- 172.21.15.86:0/2772920193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 msgr2=0x560f35416580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.176+0000 7f4148e00640 1 --2- 172.21.15.86:0/2772920193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f35416580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.176+0000 7f4148e00640 1 -- 172.21.15.86:0/2772920193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3532a040 con 0x560f3541bcc0 2024-04-17T07:17:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.177+0000 7f4148e00640 1 --2- 172.21.15.86:0/2772920193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f3541e150 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f413400bb10 tx=0x7f413400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5b82b11a58d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.177+0000 7f4132a00640 1 -- 172.21.15.86:0/2772920193 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134017420 con 0x560f3541bcc0 2024-04-17T07:17:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.177+0000 7f4132a00640 1 -- 172.21.15.86:0/2772920193 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4134017a70 con 0x560f3541bcc0 2024-04-17T07:17:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.177+0000 7f4132a00640 1 -- 172.21.15.86:0/2772920193 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413400fa60 con 0x560f3541bcc0 2024-04-17T07:17:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.178+0000 7f4142000640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f412c0a3c30 0x7f412c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.178+0000 7f414efba740 1 -- 172.21.15.86:0/2772920193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 msgr2=0x560f3541e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.178+0000 7f414efba740 1 --2- 172.21.15.86:0/2772920193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f3541e150 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f413400bb10 tx=0x7f413400d340 comp rx=0 tx=0).stop 2024-04-17T07:17:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.178+0000 7f4142000640 1 -- 172.21.15.86:0/1264550131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f41380f2ce0 con 0x7f41441040a0 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.178+0000 7f414a200640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f412c0a3c30 0x7f412c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.178+0000 7f414efba740 1 -- 172.21.15.86:0/2772920193 shutdown_connections 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.178+0000 7f414efba740 1 --2- 172.21.15.86:0/2772920193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f3541e150 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.178+0000 7f414efba740 1 --2- 172.21.15.86:0/2772920193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f35416b50 0x560f3541b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.178+0000 7f414efba740 1 --2- 172.21.15.86:0/2772920193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f35416580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.178+0000 7f414efba740 1 -- 172.21.15.86:0/2772920193 >> 172.21.15.86:0/2772920193 conn(0x560f354116d0 msgr2=0x560f35411ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.179+0000 7f414efba740 1 -- 172.21.15.86:0/2772920193 shutdown_connections 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.179+0000 7f414efba740 1 -- 172.21.15.86:0/2772920193 wait complete. 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.179+0000 7f414a200640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f412c0a3c30 0x7f412c0a60f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x560f354173b0 tx=0x7f413c0083f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.179+0000 7f414efba740 1 -- start start 2024-04-17T07:17:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.179+0000 7f414efba740 1 Processor -- start 2024-04-17T07:17:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f414efba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f35420700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f414efba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f35416b50 0x560f35420c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f414efba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f35427c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f414efba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f3541ec70 con 0x560f3541bcc0 2024-04-17T07:17:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f414efba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560f35335830 con 0x560f35416160 2024-04-17T07:17:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f414efba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560f3541edf0 con 0x560f35416b50 2024-04-17T07:17:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f414efba740 10 client.0 _ll_register_callbacks cb 0x560f35303fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f414a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f35427c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f414a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f35427c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56304/0 (socket says 172.21.15.86:56304) 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f414a200640 1 -- 172.21.15.86:0/568553377 learned_addr learned my addr 172.21.15.86:0/568553377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f4148e00640 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f35416b50 0x560f35420c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.180+0000 7f4149800640 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f35420700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.181+0000 7f414a200640 1 -- 172.21.15.86:0/568553377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f35416b50 msgr2=0x560f35420c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.181+0000 7f414a200640 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f35416b50 0x560f35420c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.181+0000 7f414a200640 1 -- 172.21.15.86:0/568553377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 msgr2=0x560f35420700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.181+0000 7f414a200640 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f35420700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.181+0000 7f414a200640 1 -- 172.21.15.86:0/568553377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f35339920 con 0x560f3541bcc0 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.181+0000 7f414a200640 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f35427c80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f413c010e30 tx=0x7f413c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9b4b565a73dc3c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.181+0000 7f4127400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.181+0000 7f4127400640 1 -- 172.21.15.86:0/568553377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c01d420 con 0x560f3541bcc0 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.181+0000 7f4127400640 1 -- 172.21.15.86:0/568553377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f413c01d5c0 con 0x560f3541bcc0 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.181+0000 7f414efba740 1 -- 172.21.15.86:0/568553377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4134017a70 con 0x560f3541bcc0 2024-04-17T07:17:59.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f4127400640 1 -- 172.21.15.86:0/568553377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f413c02e050 con 0x560f3541bcc0 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f414efba740 10 client.16449 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f414efba740 1 -- 172.21.15.86:0/568553377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f353077f0 con 0x560f3541bcc0 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f4127400640 1 -- 172.21.15.86:0/568553377 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4134017a70 con 0x560f3541bcc0 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f4127400640 1 -- 172.21.15.86:0/568553377 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f413c025930 con 0x560f3541bcc0 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f414efba740 10 client.16449 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f414efba740 10 client.16449 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f414efba740 20 client.16449 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f414efba740 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560f3542c280 0x560f3542e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f414efba740 4 client.16449 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.182+0000 7f414efba740 1 -- 172.21.15.86:0/568553377 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f35389a40 con 0x560f3542c280 2024-04-17T07:17:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.183+0000 7f4149800640 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560f3542c280 0x560f3542e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.184+0000 7f4149800640 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560f3542c280 0x560f3542e6e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=270ddff2e39c7ef4 server_cookie=9457ee9b6086ef28 in_seq=0 out_seq=0 2024-04-17T07:17:59.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.184+0000 7f4127400640 10 client.16449 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f4127400640 1 -- 172.21.15.86:0/568553377 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f35389a40 con 0x560f3542c280 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f4127400640 10 client.16449 handle_command_reply: tid=0 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 client.16449 shutdown 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 -- 172.21.15.86:0/568553377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 msgr2=0x560f35427c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f35427c80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f413c010e30 tx=0x7f413c004a40 comp rx=0 tx=0).stop 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 -- 172.21.15.86:0/568553377 shutdown_connections 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560f3542c280 0x560f3542e6e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f3541bcc0 0x560f35427c80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f35416b50 0x560f35420c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 --2- 172.21.15.86:0/568553377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f35420700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f4127400640 0 client.16449 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 -- 172.21.15.86:0/568553377 >> 172.21.15.86:0/568553377 conn(0x560f354116d0 msgr2=0x560f35410930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 -- 172.21.15.86:0/568553377 shutdown_connections 2024-04-17T07:17:59.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.186+0000 7f414efba740 1 -- 172.21.15.86:0/568553377 wait complete. 2024-04-17T07:17:59.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.187+0000 7f414efba740 20 client.16449 trim_cache size 0 max 16384 2024-04-17T07:17:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.207+0000 7f414efba740 1 Processor -- start 2024-04-17T07:17:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.207+0000 7f414efba740 1 -- start start 2024-04-17T07:17:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.207+0000 7f414efba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f354100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.207+0000 7f414efba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35427c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.207+0000 7f414efba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f3541bcc0 0x560f354105e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.207+0000 7f414efba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41441aca20 con 0x560f35416b50 2024-04-17T07:17:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.207+0000 7f414efba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f41441ae560 con 0x560f35416160 2024-04-17T07:17:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.207+0000 7f414efba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41441ad0f0 con 0x560f3541bcc0 2024-04-17T07:17:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.208+0000 7f4148e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35427c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.208+0000 7f4148e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35427c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56318/0 (socket says 172.21.15.86:56318) 2024-04-17T07:17:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.208+0000 7f4148e00640 1 -- 172.21.15.86:0/3989066575 learned_addr learned my addr 172.21.15.86:0/3989066575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.208+0000 7f4149800640 1 --2- 172.21.15.86:0/3989066575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f354100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.208+0000 7f414a200640 1 --2- 172.21.15.86:0/3989066575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f3541bcc0 0x560f354105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.208+0000 7f4148e00640 1 -- 172.21.15.86:0/3989066575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f3541bcc0 msgr2=0x560f354105e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.208+0000 7f4148e00640 1 --2- 172.21.15.86:0/3989066575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f3541bcc0 0x560f354105e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.208+0000 7f4148e00640 1 -- 172.21.15.86:0/3989066575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 msgr2=0x560f354100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.208+0000 7f4148e00640 1 --2- 172.21.15.86:0/3989066575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f354100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.208+0000 7f4148e00640 1 -- 172.21.15.86:0/3989066575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3508e3f0 con 0x560f35416b50 2024-04-17T07:17:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.209+0000 7f4148e00640 1 --2- 172.21.15.86:0/3989066575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35427c40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f413400ba30 tx=0x7f413400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905cebc647851a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.209+0000 7f4132000640 1 -- 172.21.15.86:0/3989066575 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134027020 con 0x560f35416b50 2024-04-17T07:17:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.209+0000 7f4132000640 1 -- 172.21.15.86:0/3989066575 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4134004640 con 0x560f35416b50 2024-04-17T07:17:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.209+0000 7f4132000640 1 -- 172.21.15.86:0/3989066575 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41340236d0 con 0x560f35416b50 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.210+0000 7f414efba740 1 -- 172.21.15.86:0/3989066575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 msgr2=0x560f35427c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.210+0000 7f414efba740 1 --2- 172.21.15.86:0/3989066575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35427c40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f413400ba30 tx=0x7f413400b670 comp rx=0 tx=0).stop 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.210+0000 7f414efba740 1 -- 172.21.15.86:0/3989066575 shutdown_connections 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.210+0000 7f414efba740 1 --2- 172.21.15.86:0/3989066575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f3541bcc0 0x560f354105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.210+0000 7f414efba740 1 --2- 172.21.15.86:0/3989066575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35427c40 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.210+0000 7f414efba740 1 --2- 172.21.15.86:0/3989066575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f354100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.210+0000 7f414efba740 1 -- 172.21.15.86:0/3989066575 >> 172.21.15.86:0/3989066575 conn(0x560f354116d0 msgr2=0x560f35418050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.210+0000 7f414efba740 1 -- 172.21.15.86:0/3989066575 shutdown_connections 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.210+0000 7f414efba740 1 -- 172.21.15.86:0/3989066575 wait complete. 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.211+0000 7f414efba740 1 -- start start 2024-04-17T07:17:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.211+0000 7f414efba740 1 Processor -- start 2024-04-17T07:17:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.211+0000 7f414efba740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f354254f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.211+0000 7f414efba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35425a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.211+0000 7f414efba740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f3541bcc0 0x560f354269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.211+0000 7f414efba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41441ade90 con 0x560f35416b50 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.211+0000 7f414efba740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f41441ad7c0 con 0x560f35416160 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.211+0000 7f414efba740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41441aa9f0 con 0x560f3541bcc0 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.212+0000 7f414efba740 10 client.0 _ll_register_callbacks cb 0x7f413c010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.212+0000 7f4148e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35425a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.212+0000 7f4148e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35425a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56322/0 (socket says 172.21.15.86:56322) 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.212+0000 7f4148e00640 1 -- 172.21.15.86:0/1273562634 learned_addr learned my addr 172.21.15.86:0/1273562634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.212+0000 7f4148e00640 1 -- 172.21.15.86:0/1273562634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f3541bcc0 msgr2=0x560f354269a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.212+0000 7f4148e00640 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f3541bcc0 0x560f354269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.212+0000 7f4149800640 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f354254f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.212+0000 7f4148e00640 1 -- 172.21.15.86:0/1273562634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 msgr2=0x560f354254f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.212+0000 7f4148e00640 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f354254f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.212+0000 7f4148e00640 1 -- 172.21.15.86:0/1273562634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f351587d0 con 0x560f35416b50 2024-04-17T07:17:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.213+0000 7f4148e00640 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35425a30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f413400c010 tx=0x7f4134005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ba487c4ef21a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.213+0000 7f4127400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.213+0000 7f4127400640 1 -- 172.21.15.86:0/1273562634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134023670 con 0x560f35416b50 2024-04-17T07:17:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.213+0000 7f414efba740 1 -- 172.21.15.86:0/1273562634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4134004640 con 0x560f35416b50 2024-04-17T07:17:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.213+0000 7f4127400640 1 -- 172.21.15.86:0/1273562634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4134023810 con 0x560f35416b50 2024-04-17T07:17:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.213+0000 7f4127400640 1 -- 172.21.15.86:0/1273562634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f413403f050 con 0x560f35416b50 2024-04-17T07:17:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.213+0000 7f414efba740 10 client.16455 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.213+0000 7f414efba740 1 -- 172.21.15.86:0/1273562634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f35492820 con 0x560f35416b50 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.214+0000 7f4127400640 1 -- 172.21.15.86:0/1273562634 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4134004640 con 0x560f35416b50 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.214+0000 7f4127400640 1 -- 172.21.15.86:0/1273562634 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4134021720 con 0x560f35416b50 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.214+0000 7f414efba740 10 client.16455 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.214+0000 7f414efba740 10 client.16455 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.214+0000 7f414efba740 20 client.16455 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.214+0000 7f414efba740 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560f35492ff0 0x560f35495410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.214+0000 7f414efba740 4 client.16455 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.214+0000 7f414efba740 1 -- 172.21.15.86:0/1273562634 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f3541ba30 con 0x560f35492ff0 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.215+0000 7f4149800640 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560f35492ff0 0x560f35495410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.216+0000 7f4149800640 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560f35492ff0 0x560f35495410 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1cd61ebfe7cd5ca server_cookie=769be9c491b2e235 in_seq=0 out_seq=0 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.216+0000 7f4127400640 10 client.16455 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:17:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.216+0000 7f4127400640 1 -- 172.21.15.86:0/1273562634 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f3541ba30 con 0x560f35492ff0 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.216+0000 7f4127400640 10 client.16455 handle_command_reply: tid=0 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.216+0000 7f414efba740 1 client.16455 shutdown 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.216+0000 7f414efba740 1 -- 172.21.15.86:0/1273562634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 msgr2=0x560f35425a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.216+0000 7f414efba740 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35425a30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f413400c010 tx=0x7f4134005230 comp rx=0 tx=0).stop 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.217+0000 7f414efba740 1 -- 172.21.15.86:0/1273562634 shutdown_connections 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.217+0000 7f414efba740 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560f35492ff0 0x560f35495410 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.217+0000 7f414efba740 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f3541bcc0 0x560f354269a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.217+0000 7f414efba740 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f35416b50 0x560f35425a30 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.217+0000 7f414efba740 1 --2- 172.21.15.86:0/1273562634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f35416160 0x560f354254f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.217+0000 7f414efba740 1 -- 172.21.15.86:0/1273562634 >> 172.21.15.86:0/1273562634 conn(0x560f354116d0 msgr2=0x560f35414930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.217+0000 7f4127400640 0 client.16455 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.217+0000 7f414efba740 1 -- 172.21.15.86:0/1273562634 shutdown_connections 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.217+0000 7f414efba740 1 -- 172.21.15.86:0/1273562634 wait complete. 2024-04-17T07:17:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.217+0000 7f414efba740 20 client.16455 trim_cache size 0 max 16384 2024-04-17T07:17:59.219 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:17:59.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 -- 172.21.15.86:0/1264550131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f412c0a3c30 msgr2=0x7f412c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f412c0a3c30 0x7f412c0a60f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x560f354173b0 tx=0x7f413c0083f0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 -- 172.21.15.86:0/1264550131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441040a0 msgr2=0x7f414419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441040a0 0x7f414419a240 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4138005ab0 tx=0x7f41380042f0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 -- 172.21.15.86:0/1264550131 shutdown_connections 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f412c0a3c30 0x7f412c0a60f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4144105c50 0x7f41441a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4144105290 0x7f414419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 --2- 172.21.15.86:0/1264550131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41441040a0 0x7f414419a240 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 -- 172.21.15.86:0/1264550131 >> 172.21.15.86:0/1264550131 conn(0x7f41440ff610 msgr2=0x7f4144108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 -- 172.21.15.86:0/1264550131 shutdown_connections 2024-04-17T07:17:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.218+0000 7f414ac00640 1 -- 172.21.15.86:0/1264550131 wait complete. 2024-04-17T07:17:59.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:17:59.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.458+0000 7ff3c5800640 1 -- 172.21.15.86:0/2189387422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c0109d40 msgr2=0x7ff3c010c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.458+0000 7ff3c5800640 1 --2- 172.21.15.86:0/2189387422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c0109d40 0x7ff3c010c1d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009ca0 tx=0x7ff3b802c190 comp rx=0 tx=0).stop 2024-04-17T07:17:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.458+0000 7ff3c5800640 1 -- 172.21.15.86:0/2189387422 shutdown_connections 2024-04-17T07:17:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.458+0000 7ff3c5800640 1 --2- 172.21.15.86:0/2189387422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c0109d40 0x7ff3c010c1d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.458+0000 7ff3c5800640 1 --2- 172.21.15.86:0/2189387422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3c0074940 0x7ff3c0074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.458+0000 7ff3c5800640 1 --2- 172.21.15.86:0/2189387422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3c0075f20 0x7ff3c0074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.458+0000 7ff3c5800640 1 -- 172.21.15.86:0/2189387422 >> 172.21.15.86:0/2189387422 conn(0x7ff3c00fd400 msgr2=0x7ff3c00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.458+0000 7ff3c5800640 1 -- 172.21.15.86:0/2189387422 shutdown_connections 2024-04-17T07:17:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.458+0000 7ff3c5800640 1 -- 172.21.15.86:0/2189387422 wait complete. 2024-04-17T07:17:59.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.460+0000 7ff3c5800640 1 Processor -- start 2024-04-17T07:17:59.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.460+0000 7ff3c5800640 1 -- start start 2024-04-17T07:17:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.460+0000 7ff3c5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c0074940 0x7ff3c019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.460+0000 7ff3c5800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3c0075f20 0x7ff3c019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.460+0000 7ff3c5800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3c0109d40 0x7ff3c01a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.460+0000 7ff3c5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c010fbe0 con 0x7ff3c0074940 2024-04-17T07:17:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.460+0000 7ff3c5800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c010fa60 con 0x7ff3c0075f20 2024-04-17T07:17:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.460+0000 7ff3c5800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c010fd60 con 0x7ff3c0109d40 2024-04-17T07:17:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.461+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c0074940 0x7ff3c019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.461+0000 7ff3bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c0074940 0x7ff3c019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56338/0 (socket says 172.21.15.86:56338) 2024-04-17T07:17:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.461+0000 7ff3bfe00640 1 -- 172.21.15.86:0/2455104695 learned_addr learned my addr 172.21.15.86:0/2455104695 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.461+0000 7ff3bf400640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3c0075f20 0x7ff3c019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.461+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3c0109d40 0x7ff3c01a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.461+0000 7ff3bfe00640 1 -- 172.21.15.86:0/2455104695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3c0109d40 msgr2=0x7ff3c01a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.461+0000 7ff3bfe00640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3c0109d40 0x7ff3c01a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.461+0000 7ff3bfe00640 1 -- 172.21.15.86:0/2455104695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3c0075f20 msgr2=0x7ff3c019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.461+0000 7ff3bfe00640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3c0075f20 0x7ff3c019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.461+0000 7ff3bfe00640 1 -- 172.21.15.86:0/2455104695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c01a6230 con 0x7ff3c0074940 2024-04-17T07:17:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.462+0000 7ff3bfe00640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c0074940 0x7ff3c019e610 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000d770 tx=0x7ff3b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.462+0000 7ff3bcc00640 1 -- 172.21.15.86:0/2455104695 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0017070 con 0x7ff3c0074940 2024-04-17T07:17:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.462+0000 7ff3bcc00640 1 -- 172.21.15.86:0/2455104695 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3b00054d0 con 0x7ff3c0074940 2024-04-17T07:17:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.462+0000 7ff3bcc00640 1 -- 172.21.15.86:0/2455104695 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0004cf0 con 0x7ff3c0074940 2024-04-17T07:17:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.462+0000 7ff3c5800640 1 -- 172.21.15.86:0/2455104695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c01a6520 con 0x7ff3c0074940 2024-04-17T07:17:59.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.462+0000 7ff3c5800640 1 -- 172.21.15.86:0/2455104695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c01a6a20 con 0x7ff3c0074940 2024-04-17T07:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.466+0000 7ff3c9bda740 1 Processor -- start 2024-04-17T07:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.467+0000 7ff3c9bda740 1 -- start start 2024-04-17T07:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.467+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 0x55f1f8699580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.467+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f869e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.467+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f869ecc0 0x55f1f86a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.467+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f1f85ff020 con 0x55f1f8699b50 2024-04-17T07:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.467+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f1f85a5460 con 0x55f1f869ecc0 2024-04-17T07:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.467+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f1f85fafe0 con 0x55f1f8699160 2024-04-17T07:17:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.467+0000 7ff3bcc00640 1 -- 172.21.15.86:0/2455104695 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff3b0004e90 con 0x7ff3c0074940 2024-04-17T07:17:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.468+0000 7ff3c4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 0x55f1f8699580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.468+0000 7ff3c4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 0x55f1f8699580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:53016/0 (socket says 172.21.15.86:53016) 2024-04-17T07:17:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.468+0000 7ff3c4e00640 1 -- 172.21.15.86:0/2035822531 learned_addr learned my addr 172.21.15.86:0/2035822531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.468+0000 7ff3bf400640 1 --2- 172.21.15.86:0/2035822531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f869ecc0 0x55f1f86a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.468+0000 7ff3c4e00640 1 -- 172.21.15.86:0/2035822531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f869ecc0 msgr2=0x55f1f86a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.468+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/2035822531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f869ecc0 0x55f1f86a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.468+0000 7ff3c4e00640 1 -- 172.21.15.86:0/2035822531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 msgr2=0x55f1f869e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.468+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/2035822531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f869e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.468+0000 7ff3c4e00640 1 -- 172.21.15.86:0/2035822531 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f85ad040 con 0x55f1f8699160 2024-04-17T07:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.469+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/2035822531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 0x55f1f8699580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005e30 tx=0x7ff3b8005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=128671f9a72ff346 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.469+0000 7ff3b5600640 1 -- 172.21.15.86:0/2035822531 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8043020 con 0x55f1f8699160 2024-04-17T07:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.469+0000 7ff3bcc00640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff3a40a3c30 0x7ff3a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.469+0000 7ff3b5600640 1 -- 172.21.15.86:0/2035822531 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3b8003e20 con 0x55f1f8699160 2024-04-17T07:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.469+0000 7ff3b5600640 1 -- 172.21.15.86:0/2035822531 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8039040 con 0x55f1f8699160 2024-04-17T07:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.470+0000 7ff3bcc00640 1 -- 172.21.15.86:0/2455104695 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff3b00d0300 con 0x7ff3c0074940 2024-04-17T07:17:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.470+0000 7ff3bfe00640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff3a40a3c30 0x7ff3a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.470+0000 7ff3c9bda740 1 -- 172.21.15.86:0/2035822531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 msgr2=0x55f1f8699580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.470+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/2035822531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 0x55f1f8699580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005e30 tx=0x7ff3b8005f80 comp rx=0 tx=0).stop 2024-04-17T07:17:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.471+0000 7ff3bfe00640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff3a40a3c30 0x7ff3a40a60f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009610 tx=0x7ff3b00d0550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.471+0000 7ff3c9bda740 1 -- 172.21.15.86:0/2035822531 shutdown_connections 2024-04-17T07:17:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.471+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/2035822531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f869ecc0 0x55f1f86a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.471+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/2035822531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f869e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.471+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/2035822531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 0x55f1f8699580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.471+0000 7ff3c9bda740 1 -- 172.21.15.86:0/2035822531 >> 172.21.15.86:0/2035822531 conn(0x55f1f86946d0 msgr2=0x55f1f8694ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.471+0000 7ff3c9bda740 1 -- 172.21.15.86:0/2035822531 shutdown_connections 2024-04-17T07:17:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.471+0000 7ff3c9bda740 1 -- 172.21.15.86:0/2035822531 wait complete. 2024-04-17T07:17:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.471+0000 7ff3c9bda740 1 -- start start 2024-04-17T07:17:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.472+0000 7ff3c9bda740 1 Processor -- start 2024-04-17T07:17:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.472+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f8699160 0x55f1f86a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.472+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.472+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f869ecc0 0x55f1f86aab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.472+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f1f86a1c10 con 0x55f1f8699b50 2024-04-17T07:17:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.472+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f1f85b8830 con 0x55f1f8699160 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.472+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f1f86a1d90 con 0x55f1f869ecc0 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.472+0000 7ff3c9bda740 10 client.0 _ll_register_callbacks cb 0x55f1f8586fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56354/0 (socket says 172.21.15.86:56354) 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3bf400640 1 -- 172.21.15.86:0/384240167 learned_addr learned my addr 172.21.15.86:0/384240167 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f8699160 0x55f1f86a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f869ecc0 0x55f1f86aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3bf400640 1 -- 172.21.15.86:0/384240167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f869ecc0 msgr2=0x55f1f86aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3bf400640 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f869ecc0 0x55f1f86aab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3bf400640 1 -- 172.21.15.86:0/384240167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f8699160 msgr2=0x55f1f86a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3bf400640 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f8699160 0x55f1f86a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3bf400640 1 -- 172.21.15.86:0/384240167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f85bc920 con 0x55f1f8699b50 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f869ecc0 0x55f1f86aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:17:59.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.473+0000 7ff3bf400640 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a3bb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00beb0 tx=0x7ff3ac012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e72f67e215f638c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.474+0000 7ff3aa000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:59.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.474+0000 7ff3aa000640 1 -- 172.21.15.86:0/384240167 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac004230 con 0x55f1f8699b50 2024-04-17T07:17:59.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.474+0000 7ff3aa000640 1 -- 172.21.15.86:0/384240167 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff3ac0043d0 con 0x55f1f8699b50 2024-04-17T07:17:59.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.474+0000 7ff3c9bda740 1 -- 172.21.15.86:0/384240167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b8003e20 con 0x55f1f8699b50 2024-04-17T07:17:59.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.474+0000 7ff3aa000640 1 -- 172.21.15.86:0/384240167 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff3ac024050 con 0x55f1f8699b50 2024-04-17T07:17:59.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.475+0000 7ff3c9bda740 10 client.16464 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:59.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.475+0000 7ff3c9bda740 1 -- 172.21.15.86:0/384240167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1f858a7f0 con 0x55f1f8699b50 2024-04-17T07:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.475+0000 7ff3aa000640 1 -- 172.21.15.86:0/384240167 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3b8003e20 con 0x55f1f8699b50 2024-04-17T07:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.475+0000 7ff3aa000640 1 -- 172.21.15.86:0/384240167 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff3ac0109b0 con 0x55f1f8699b50 2024-04-17T07:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.475+0000 7ff3c9bda740 10 client.16464 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.475+0000 7ff3c9bda740 10 client.16464 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.475+0000 7ff3c9bda740 20 client.16464 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.476+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f1f86af110 0x55f1f86b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.476+0000 7ff3c9bda740 4 client.16464 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:59.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.476+0000 7ff3c9bda740 1 -- 172.21.15.86:0/384240167 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1f860ca40 con 0x55f1f86af110 2024-04-17T07:17:59.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.476+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f1f86af110 0x55f1f86b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.477+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f1f86af110 0x55f1f86b1590 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=503fff9df24f1696 server_cookie=787a208c51f711f0 in_seq=0 out_seq=0 2024-04-17T07:17:59.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.477+0000 7ff3aa000640 10 client.16464 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:17:59.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.479+0000 7ff3aa000640 1 -- 172.21.15.86:0/384240167 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1f860ca40 con 0x55f1f86af110 2024-04-17T07:17:59.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.480+0000 7ff3aa000640 10 client.16464 handle_command_reply: tid=0 2024-04-17T07:17:59.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.480+0000 7ff3c9bda740 1 client.16464 shutdown 2024-04-17T07:17:59.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.480+0000 7ff3c9bda740 1 -- 172.21.15.86:0/384240167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 msgr2=0x55f1f86a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.480+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a3bb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00beb0 tx=0x7ff3ac012820 comp rx=0 tx=0).stop 2024-04-17T07:17:59.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.481+0000 7ff3c9bda740 1 -- 172.21.15.86:0/384240167 shutdown_connections 2024-04-17T07:17:59.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.481+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f1f86af110 0x55f1f86b1590 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.481+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f869ecc0 0x55f1f86aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.481+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a3bb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.481+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/384240167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f8699160 0x55f1f86a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.481+0000 7ff3c9bda740 1 -- 172.21.15.86:0/384240167 >> 172.21.15.86:0/384240167 conn(0x55f1f86946d0 msgr2=0x55f1f8693930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.481+0000 7ff3aa000640 0 client.16464 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.481+0000 7ff3c9bda740 1 -- 172.21.15.86:0/384240167 shutdown_connections 2024-04-17T07:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.481+0000 7ff3c9bda740 1 -- 172.21.15.86:0/384240167 wait complete. 2024-04-17T07:17:59.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.482+0000 7ff3c9bda740 20 client.16464 trim_cache size 0 max 16384 2024-04-17T07:17:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.501+0000 7ff3c9bda740 1 Processor -- start 2024-04-17T07:17:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.501+0000 7ff3c9bda740 1 -- start start 2024-04-17T07:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f8699160 0x55f1f86b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699b50 0x55f1f86aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f869ecc0 0x55f1f86930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b0e30 con 0x55f1f869ecc0 2024-04-17T07:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b2970 con 0x55f1f8699160 2024-04-17T07:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b1500 con 0x55f1f8699b50 2024-04-17T07:17:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3c4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f869ecc0 0x55f1f86930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3c4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f869ecc0 0x55f1f86930a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56358/0 (socket says 172.21.15.86:56358) 2024-04-17T07:17:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3c4e00640 1 -- 172.21.15.86:0/2927963369 learned_addr learned my addr 172.21.15.86:0/2927963369 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3bf400640 1 --2- 172.21.15.86:0/2927963369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699b50 0x55f1f86aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.502+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/2927963369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f8699160 0x55f1f86b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.503+0000 7ff3c4e00640 1 -- 172.21.15.86:0/2927963369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699b50 msgr2=0x55f1f86aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.503+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/2927963369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699b50 0x55f1f86aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.503+0000 7ff3c4e00640 1 -- 172.21.15.86:0/2927963369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f8699160 msgr2=0x55f1f86b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.503+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/2927963369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f8699160 0x55f1f86b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.503+0000 7ff3c4e00640 1 -- 172.21.15.86:0/2927963369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1f83113f0 con 0x55f1f869ecc0 2024-04-17T07:17:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.503+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/2927963369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f869ecc0 0x55f1f86930a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005e30 tx=0x7ff3b8036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410041136b60d1c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.504+0000 7ff3b4c00640 1 -- 172.21.15.86:0/2927963369 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8039050 con 0x55f1f869ecc0 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.504+0000 7ff3b4c00640 1 -- 172.21.15.86:0/2927963369 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3b8032c20 con 0x55f1f869ecc0 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.504+0000 7ff3b4c00640 1 -- 172.21.15.86:0/2927963369 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8043040 con 0x55f1f869ecc0 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.504+0000 7ff3c9bda740 1 -- 172.21.15.86:0/2927963369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f869ecc0 msgr2=0x55f1f86930a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.504+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/2927963369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f869ecc0 0x55f1f86930a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005e30 tx=0x7ff3b8036040 comp rx=0 tx=0).stop 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.505+0000 7ff3c9bda740 1 -- 172.21.15.86:0/2927963369 shutdown_connections 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.505+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/2927963369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f869ecc0 0x55f1f86930a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.505+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/2927963369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699b50 0x55f1f86aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.505+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/2927963369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f8699160 0x55f1f86b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.505+0000 7ff3c9bda740 1 -- 172.21.15.86:0/2927963369 >> 172.21.15.86:0/2927963369 conn(0x55f1f86946d0 msgr2=0x55f1f869b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.505+0000 7ff3c9bda740 1 -- 172.21.15.86:0/2927963369 shutdown_connections 2024-04-17T07:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.505+0000 7ff3c9bda740 1 -- 172.21.15.86:0/2927963369 wait complete. 2024-04-17T07:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.505+0000 7ff3c9bda740 1 -- start start 2024-04-17T07:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.505+0000 7ff3c9bda740 1 Processor -- start 2024-04-17T07:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 0x55f1f86a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3c9bda740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f869ecc0 0x55f1f86a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b22a0 con 0x55f1f8699b50 2024-04-17T07:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b1bd0 con 0x55f1f869ecc0 2024-04-17T07:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3c9bda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01aee00 con 0x55f1f8699160 2024-04-17T07:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3c9bda740 10 client.0 _ll_register_callbacks cb 0x7ff3ac00baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56370/0 (socket says 172.21.15.86:56370) 2024-04-17T07:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3bf400640 1 -- 172.21.15.86:0/3166293134 learned_addr learned my addr 172.21.15.86:0/3166293134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.506+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 0x55f1f86a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.507+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f869ecc0 0x55f1f86a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.507+0000 7ff3bf400640 1 -- 172.21.15.86:0/3166293134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 msgr2=0x55f1f86a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.507+0000 7ff3bf400640 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 0x55f1f86a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.507+0000 7ff3bf400640 1 -- 172.21.15.86:0/3166293134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f869ecc0 msgr2=0x55f1f86a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.507+0000 7ff3bf400640 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f869ecc0 0x55f1f86a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.507+0000 7ff3bf400640 1 -- 172.21.15.86:0/3166293134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1f83db7d0 con 0x55f1f8699b50 2024-04-17T07:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.507+0000 7ff3bf400640 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a8110 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x55f1f8695610 tx=0x7ff3ac0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7a49d87b26f525 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.507+0000 7ff3aa000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.508+0000 7ff3aa000640 1 -- 172.21.15.86:0/3166293134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0105e0 con 0x55f1f8699b50 2024-04-17T07:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.508+0000 7ff3aa000640 1 -- 172.21.15.86:0/3166293134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff3ac010780 con 0x55f1f8699b50 2024-04-17T07:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.508+0000 7ff3c9bda740 1 -- 172.21.15.86:0/3166293134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b8032c20 con 0x55f1f8699b50 2024-04-17T07:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.508+0000 7ff3aa000640 1 -- 172.21.15.86:0/3166293134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff3ac039050 con 0x55f1f8699b50 2024-04-17T07:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.508+0000 7ff3c9bda740 10 client.16470 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.508+0000 7ff3c9bda740 1 -- 172.21.15.86:0/3166293134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1f86aa8e0 con 0x55f1f8699b50 2024-04-17T07:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.509+0000 7ff3aa000640 1 -- 172.21.15.86:0/3166293134 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3b8032c20 con 0x55f1f8699b50 2024-04-17T07:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.509+0000 7ff3aa000640 1 -- 172.21.15.86:0/3166293134 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff3ac01b480 con 0x55f1f8699b50 2024-04-17T07:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.509+0000 7ff3c9bda740 10 client.16470 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.509+0000 7ff3c9bda740 10 client.16470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.509+0000 7ff3c9bda740 20 client.16470 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.509+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f1f87158b0 0x55f1f8717cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.509+0000 7ff3c9bda740 4 client.16470 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.509+0000 7ff3c9bda740 1 -- 172.21.15.86:0/3166293134 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1f86068b0 con 0x55f1f87158b0 2024-04-17T07:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.510+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f1f87158b0 0x55f1f8717cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.511+0000 7ff3aa000640 10 client.16470 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.511+0000 7ff3c4e00640 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f1f87158b0 0x55f1f8717cd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a05843bc842fbba7 server_cookie=6ea726ea853cf908 in_seq=0 out_seq=0 2024-04-17T07:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3aa000640 1 -- 172.21.15.86:0/3166293134 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1f86068b0 con 0x55f1f87158b0 2024-04-17T07:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3aa000640 10 client.16470 handle_command_reply: tid=0 2024-04-17T07:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 client.16470 shutdown 2024-04-17T07:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 -- 172.21.15.86:0/3166293134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 msgr2=0x55f1f86a8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a8110 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x55f1f8695610 tx=0x7ff3ac0052f0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 -- 172.21.15.86:0/3166293134 shutdown_connections 2024-04-17T07:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f1f87158b0 0x55f1f8717cd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1f869ecc0 0x55f1f86a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1f8699b50 0x55f1f86a8110 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 --2- 172.21.15.86:0/3166293134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1f8699160 0x55f1f86a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 -- 172.21.15.86:0/3166293134 >> 172.21.15.86:0/3166293134 conn(0x55f1f86946d0 msgr2=0x55f1f8697930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3aa000640 0 client.16470 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 -- 172.21.15.86:0/3166293134 shutdown_connections 2024-04-17T07:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 1 -- 172.21.15.86:0/3166293134 wait complete. 2024-04-17T07:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.512+0000 7ff3c9bda740 20 client.16470 trim_cache size 0 max 16384 2024-04-17T07:17:59.513 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:17:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 -- 172.21.15.86:0/2455104695 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff3a40a3c30 msgr2=0x7ff3a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff3a40a3c30 0x7ff3a40a60f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009610 tx=0x7ff3b00d0550 comp rx=0 tx=0).stop 2024-04-17T07:17:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 -- 172.21.15.86:0/2455104695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c0074940 msgr2=0x7ff3c019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c0074940 0x7ff3c019e610 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000d770 tx=0x7ff3b000dc30 comp rx=0 tx=0).stop 2024-04-17T07:17:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 -- 172.21.15.86:0/2455104695 shutdown_connections 2024-04-17T07:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff3a40a3c30 0x7ff3a40a60f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3c0109d40 0x7ff3c01a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3c0075f20 0x7ff3c019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 --2- 172.21.15.86:0/2455104695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3c0074940 0x7ff3c019e610 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 -- 172.21.15.86:0/2455104695 >> 172.21.15.86:0/2455104695 conn(0x7ff3c00fd400 msgr2=0x7ff3c01018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.513+0000 7ff3c5800640 1 -- 172.21.15.86:0/2455104695 shutdown_connections 2024-04-17T07:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.514+0000 7ff3c5800640 1 -- 172.21.15.86:0/2455104695 wait complete. 2024-04-17T07:17:59.533 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:17:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.749+0000 7f0efde00640 1 -- 172.21.15.86:0/397540082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef8102920 msgr2=0x7f0ef81091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.749+0000 7f0efde00640 1 --2- 172.21.15.86:0/397540082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef8102920 0x7f0ef81091b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009ca0 tx=0x7f0ef002c100 comp rx=0 tx=0).stop 2024-04-17T07:17:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.749+0000 7f0efde00640 1 -- 172.21.15.86:0/397540082 shutdown_connections 2024-04-17T07:17:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.749+0000 7f0efde00640 1 --2- 172.21.15.86:0/397540082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef8102920 0x7f0ef81091b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.749+0000 7f0efde00640 1 --2- 172.21.15.86:0/397540082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef8101f60 0x7f0ef81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.749+0000 7f0efde00640 1 --2- 172.21.15.86:0/397540082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef8100d70 0x7f0ef8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.749+0000 7f0efde00640 1 -- 172.21.15.86:0/397540082 >> 172.21.15.86:0/397540082 conn(0x7f0ef80fc300 msgr2=0x7f0ef80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.749+0000 7f0efde00640 1 -- 172.21.15.86:0/397540082 shutdown_connections 2024-04-17T07:17:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.749+0000 7f0efde00640 1 -- 172.21.15.86:0/397540082 wait complete. 2024-04-17T07:17:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.750+0000 7f0efde00640 1 Processor -- start 2024-04-17T07:17:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.750+0000 7f0efde00640 1 -- start start 2024-04-17T07:17:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef8100d70 0x7f0ef8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef8101f60 0x7f0ef81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef8102920 0x7f0ef819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efde00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef810d9c0 con 0x7f0ef8100d70 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efde00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef810d840 con 0x7f0ef8102920 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efde00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef810db40 con 0x7f0ef8101f60 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef8100d70 0x7f0ef8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef8100d70 0x7f0ef8197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56390/0 (socket says 172.21.15.86:56390) 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efca00640 1 -- 172.21.15.86:0/3119954619 learned_addr learned my addr 172.21.15.86:0/3119954619 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0ef7e00640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef8101f60 0x7f0ef81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efca00640 1 -- 172.21.15.86:0/3119954619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef8101f60 msgr2=0x7f0ef81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efca00640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef8101f60 0x7f0ef81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efd400640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef8102920 0x7f0ef819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efca00640 1 -- 172.21.15.86:0/3119954619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef8102920 msgr2=0x7f0ef819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.751+0000 7f0efca00640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef8102920 0x7f0ef819f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.752+0000 7f0efca00640 1 -- 172.21.15.86:0/3119954619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef819fbb0 con 0x7f0ef8100d70 2024-04-17T07:17:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.752+0000 7f0efca00640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef8100d70 0x7f0ef8197f90 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00d770 tx=0x7f0eec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.752+0000 7f0ef5600640 1 -- 172.21.15.86:0/3119954619 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec017070 con 0x7f0ef8100d70 2024-04-17T07:17:59.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.752+0000 7f0ef5600640 1 -- 172.21.15.86:0/3119954619 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eec009c30 con 0x7f0ef8100d70 2024-04-17T07:17:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.752+0000 7f0ef5600640 1 -- 172.21.15.86:0/3119954619 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec003d00 con 0x7f0ef8100d70 2024-04-17T07:17:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.752+0000 7f0efde00640 1 -- 172.21.15.86:0/3119954619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef819fea0 con 0x7f0ef8100d70 2024-04-17T07:17:59.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.752+0000 7f0efde00640 1 -- 172.21.15.86:0/3119954619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef81a0300 con 0x7f0ef8100d70 2024-04-17T07:17:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.756+0000 7f0ef5600640 1 -- 172.21.15.86:0/3119954619 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0eec003ea0 con 0x7f0ef8100d70 2024-04-17T07:17:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.756+0000 7f0f0218f740 1 Processor -- start 2024-04-17T07:17:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.757+0000 7f0f0218f740 1 -- start start 2024-04-17T07:17:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.757+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b2b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.757+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b2bf60 0x560a60b30a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.757+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b310d0 0x560a60b33560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.757+0000 7f0f0218f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a60a8d020 con 0x560a60b2bf60 2024-04-17T07:17:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.757+0000 7f0f0218f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560a60a33460 con 0x560a60b310d0 2024-04-17T07:17:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.757+0000 7f0f0218f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a60a88fe0 con 0x560a60b2b570 2024-04-17T07:17:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.757+0000 7f0efca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b2bf60 0x560a60b30a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.757+0000 7f0efca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b2bf60 0x560a60b30a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56404/0 (socket says 172.21.15.86:56404) 2024-04-17T07:17:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.757+0000 7f0efca00640 1 -- 172.21.15.86:0/1287696786 learned_addr learned my addr 172.21.15.86:0/1287696786 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.758+0000 7f0efd400640 1 --2- 172.21.15.86:0/1287696786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b2b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.758+0000 7f0ef7e00640 1 --2- 172.21.15.86:0/1287696786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b310d0 0x560a60b33560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.758+0000 7f0efca00640 1 -- 172.21.15.86:0/1287696786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 msgr2=0x560a60b2b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.758+0000 7f0efca00640 1 --2- 172.21.15.86:0/1287696786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b2b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.758+0000 7f0efca00640 1 -- 172.21.15.86:0/1287696786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b310d0 msgr2=0x560a60b33560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.758+0000 7f0efca00640 1 --2- 172.21.15.86:0/1287696786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b310d0 0x560a60b33560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.758+0000 7f0efca00640 1 -- 172.21.15.86:0/1287696786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a60a3b040 con 0x560a60b2bf60 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.758+0000 7f0ef5600640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ee00a3bb0 0x7f0ee00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.758+0000 7f0efca00640 1 --2- 172.21.15.86:0/1287696786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b2bf60 0x560a60b30a60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0d66b0 tx=0x7f0eec0d67a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163896b0e1787a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.759+0000 7f0ef5600640 1 -- 172.21.15.86:0/3119954619 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0eec0d0540 con 0x7f0ef8100d70 2024-04-17T07:17:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.759+0000 7f0efd400640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ee00a3bb0 0x7f0ee00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.759+0000 7f0ee9c00640 1 -- 172.21.15.86:0/1287696786 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec098740 con 0x560a60b2bf60 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.759+0000 7f0ee9c00640 1 -- 172.21.15.86:0/1287696786 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eec09cba0 con 0x560a60b2bf60 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.759+0000 7f0ee9c00640 1 -- 172.21.15.86:0/1287696786 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec09be00 con 0x560a60b2bf60 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.760+0000 7f0f0218f740 1 -- 172.21.15.86:0/1287696786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b2bf60 msgr2=0x560a60b30a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.760+0000 7f0efd400640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ee00a3bb0 0x7f0ee00a6070 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0ef002c710 tx=0x7f0ef00324c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.760+0000 7f0f0218f740 1 --2- 172.21.15.86:0/1287696786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b2bf60 0x560a60b30a60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0d66b0 tx=0x7f0eec0d67a0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.760+0000 7f0f0218f740 1 -- 172.21.15.86:0/1287696786 shutdown_connections 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.760+0000 7f0f0218f740 1 --2- 172.21.15.86:0/1287696786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b310d0 0x560a60b33560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.760+0000 7f0f0218f740 1 --2- 172.21.15.86:0/1287696786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b2bf60 0x560a60b30a60 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.760+0000 7f0f0218f740 1 --2- 172.21.15.86:0/1287696786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b2b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.760+0000 7f0f0218f740 1 -- 172.21.15.86:0/1287696786 >> 172.21.15.86:0/1287696786 conn(0x560a60b226d0 msgr2=0x560a60b22ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.760+0000 7f0f0218f740 1 -- 172.21.15.86:0/1287696786 shutdown_connections 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.760+0000 7f0f0218f740 1 -- 172.21.15.86:0/1287696786 wait complete. 2024-04-17T07:17:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.761+0000 7f0f0218f740 1 -- start start 2024-04-17T07:17:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.761+0000 7f0f0218f740 1 Processor -- start 2024-04-17T07:17:59.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.761+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2b570 0x560a60b35ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.761+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2bf60 0x560a60b35ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.761+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.761+0000 7f0f0218f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a60b34080 con 0x560a60b310d0 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.761+0000 7f0f0218f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560a60a46830 con 0x560a60b2b570 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.761+0000 7f0f0218f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a60b34200 con 0x560a60b2bf60 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0f0218f740 10 client.0 _ll_register_callbacks cb 0x560a60a14fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0efd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0efd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56414/0 (socket says 172.21.15.86:56414) 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0efd400640 1 -- 172.21.15.86:0/2259516939 learned_addr learned my addr 172.21.15.86:0/2259516939 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0efca00640 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2b570 0x560a60b35ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0ef7e00640 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2bf60 0x560a60b35ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0efd400640 1 -- 172.21.15.86:0/2259516939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2bf60 msgr2=0x560a60b35ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0efd400640 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2bf60 0x560a60b35ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0efd400640 1 -- 172.21.15.86:0/2259516939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2b570 msgr2=0x560a60b35ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0efd400640 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2b570 0x560a60b35ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.762+0000 7f0efd400640 1 -- 172.21.15.86:0/2259516939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a60a4a920 con 0x560a60b310d0 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.763+0000 7f0efd400640 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3d030 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0036000 tx=0x7f0ef002c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d9f97e53b15239 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.763+0000 7f0ede000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.763+0000 7f0ede000640 1 -- 172.21.15.86:0/2259516939 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef004f070 con 0x560a60b310d0 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.763+0000 7f0ede000640 1 -- 172.21.15.86:0/2259516939 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0ef00041b0 con 0x560a60b310d0 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.763+0000 7f0f0218f740 1 -- 172.21.15.86:0/2259516939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eec09cba0 con 0x560a60b310d0 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.763+0000 7f0ede000640 1 -- 172.21.15.86:0/2259516939 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0ef005b050 con 0x560a60b310d0 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.763+0000 7f0f0218f740 10 client.16482 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.764+0000 7f0f0218f740 1 -- 172.21.15.86:0/2259516939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a60a187f0 con 0x560a60b310d0 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.764+0000 7f0ede000640 1 -- 172.21.15.86:0/2259516939 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eec09cba0 con 0x560a60b310d0 2024-04-17T07:17:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.764+0000 7f0ede000640 1 -- 172.21.15.86:0/2259516939 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0ef004a070 con 0x560a60b310d0 2024-04-17T07:17:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.764+0000 7f0f0218f740 10 client.16482 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.764+0000 7f0f0218f740 10 client.16482 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:17:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.764+0000 7f0f0218f740 20 client.16482 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.764+0000 7f0f0218f740 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560a60b41590 0x560a60b43a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.764+0000 7f0f0218f740 4 client.16482 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:17:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.764+0000 7f0f0218f740 1 -- 172.21.15.86:0/2259516939 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a60a9aa40 con 0x560a60b41590 2024-04-17T07:17:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.765+0000 7f0efca00640 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560a60b41590 0x560a60b43a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.766+0000 7f0efca00640 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560a60b41590 0x560a60b43a10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38f7344ef73a49a server_cookie=d3193425eafbee02 in_seq=0 out_seq=0 2024-04-17T07:17:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.766+0000 7f0ede000640 10 client.16482 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:17:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0ede000640 1 -- 172.21.15.86:0/2259516939 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a60a9aa40 con 0x560a60b41590 2024-04-17T07:17:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0ede000640 10 client.16482 handle_command_reply: tid=0 2024-04-17T07:17:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 client.16482 shutdown 2024-04-17T07:17:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 -- 172.21.15.86:0/2259516939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 msgr2=0x560a60b3d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3d030 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0036000 tx=0x7f0ef002c100 comp rx=0 tx=0).stop 2024-04-17T07:17:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 -- 172.21.15.86:0/2259516939 shutdown_connections 2024-04-17T07:17:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3d030 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2bf60 0x560a60b35ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560a60b41590 0x560a60b43a10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 --2- 172.21.15.86:0/2259516939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2b570 0x560a60b35ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0ede000640 0 client.16482 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:17:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 -- 172.21.15.86:0/2259516939 >> 172.21.15.86:0/2259516939 conn(0x560a60b226d0 msgr2=0x560a60b21930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 -- 172.21.15.86:0/2259516939 shutdown_connections 2024-04-17T07:17:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.768+0000 7f0f0218f740 1 -- 172.21.15.86:0/2259516939 wait complete. 2024-04-17T07:17:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.769+0000 7f0f0218f740 20 client.16482 trim_cache size 0 max 16384 2024-04-17T07:17:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.783+0000 7f0f0218f740 1 Processor -- start 2024-04-17T07:17:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.783+0000 7f0f0218f740 1 -- start start 2024-04-17T07:17:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.783+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b26040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.783+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 0x560a60b3cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.783+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.783+0000 7f0f0218f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef81a8540 con 0x560a60b310d0 2024-04-17T07:17:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.783+0000 7f0f0218f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef81aa080 con 0x560a60b2bf60 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.783+0000 7f0f0218f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef81a8c10 con 0x560a60b2b570 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0efd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0efd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b482d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56424/0 (socket says 172.21.15.86:56424) 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0efd400640 1 -- 172.21.15.86:0/3970758533 learned_addr learned my addr 172.21.15.86:0/3970758533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0efca00640 1 --2- 172.21.15.86:0/3970758533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b26040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0ef7e00640 1 --2- 172.21.15.86:0/3970758533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 0x560a60b3cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0efd400640 1 -- 172.21.15.86:0/3970758533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 msgr2=0x560a60b26040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0efd400640 1 --2- 172.21.15.86:0/3970758533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b26040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0efd400640 1 -- 172.21.15.86:0/3970758533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 msgr2=0x560a60b3cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0efd400640 1 --2- 172.21.15.86:0/3970758533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 0x560a60b3cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0efd400640 1 -- 172.21.15.86:0/3970758533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6079f3f0 con 0x560a60b310d0 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0ef7e00640 1 --2- 172.21.15.86:0/3970758533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 0x560a60b3cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:17:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.784+0000 7f0efd400640 1 --2- 172.21.15.86:0/3970758533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b482d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0032c30 tx=0x7f0ef0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4db08979cf3a35 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.785+0000 7f0ee9200640 1 -- 172.21.15.86:0/3970758533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef004f070 con 0x560a60b310d0 2024-04-17T07:17:59.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.785+0000 7f0ee9200640 1 -- 172.21.15.86:0/3970758533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ef0045070 con 0x560a60b310d0 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.785+0000 7f0ee9200640 1 -- 172.21.15.86:0/3970758533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0039040 con 0x560a60b310d0 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 -- 172.21.15.86:0/3970758533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 msgr2=0x560a60b482d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 --2- 172.21.15.86:0/3970758533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b482d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0032c30 tx=0x7f0ef0009ba0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 -- 172.21.15.86:0/3970758533 shutdown_connections 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 --2- 172.21.15.86:0/3970758533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b482d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 --2- 172.21.15.86:0/3970758533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 0x560a60b3cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 --2- 172.21.15.86:0/3970758533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b26040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 -- 172.21.15.86:0/3970758533 >> 172.21.15.86:0/3970758533 conn(0x560a60b226d0 msgr2=0x560a60b25930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 -- 172.21.15.86:0/3970758533 shutdown_connections 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 -- 172.21.15.86:0/3970758533 wait complete. 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 -- start start 2024-04-17T07:17:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.786+0000 7f0f0218f740 1 Processor -- start 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.787+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.787+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 0x560a60b3a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.787+0000 7f0f0218f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.787+0000 7f0f0218f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef81a99b0 con 0x560a60b310d0 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.787+0000 7f0f0218f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef81a92e0 con 0x560a60b2bf60 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.787+0000 7f0f0218f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef81a6510 con 0x560a60b2b570 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.787+0000 7f0f0218f740 10 client.0 _ll_register_callbacks cb 0x7f0ef0032980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.787+0000 7f0efd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.787+0000 7f0efd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56428/0 (socket says 172.21.15.86:56428) 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.787+0000 7f0efd400640 1 -- 172.21.15.86:0/1425330405 learned_addr learned my addr 172.21.15.86:0/1425330405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.788+0000 7f0ef7e00640 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 0x560a60b3a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.788+0000 7f0efca00640 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.788+0000 7f0efd400640 1 -- 172.21.15.86:0/1425330405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 msgr2=0x560a60b3a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.788+0000 7f0efd400640 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b3a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.788+0000 7f0efd400640 1 -- 172.21.15.86:0/1425330405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 msgr2=0x560a60b3a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:17:59.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.788+0000 7f0efd400640 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 0x560a60b3a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.788+0000 7f0efd400640 1 -- 172.21.15.86:0/1425330405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a608697d0 con 0x560a60b310d0 2024-04-17T07:17:59.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.788+0000 7f0efd400640 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3b760 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0004210 tx=0x7f0ef0004240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42aa83fe8712f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:17:59.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.788+0000 7f0ede000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:17:59.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.788+0000 7f0ede000640 1 -- 172.21.15.86:0/1425330405 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0039030 con 0x560a60b310d0 2024-04-17T07:17:59.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0ede000640 1 -- 172.21.15.86:0/1425330405 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0ef003f910 con 0x560a60b310d0 2024-04-17T07:17:59.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0f0218f740 1 -- 172.21.15.86:0/1425330405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef0045070 con 0x560a60b310d0 2024-04-17T07:17:59.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0ede000640 1 -- 172.21.15.86:0/1425330405 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0ef0070050 con 0x560a60b310d0 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0f0218f740 10 client.16488 fetch_fsmap learned FSMap version 28 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0f0218f740 1 -- 172.21.15.86:0/1425330405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a60b3cdf0 con 0x560a60b310d0 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0ede000640 1 -- 172.21.15.86:0/1425330405 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ef0045070 con 0x560a60b310d0 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0ede000640 1 -- 172.21.15.86:0/1425330405 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0ef004a040 con 0x560a60b310d0 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0f0218f740 10 client.16488 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0f0218f740 10 client.16488 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0f0218f740 20 client.16488 populate_metadata read hostname 'smithi086' 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.789+0000 7f0f0218f740 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560a60ba7d10 0x560a60baa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.790+0000 7f0f0218f740 4 client.16488 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.790+0000 7f0f0218f740 1 -- 172.21.15.86:0/1425330405 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a60a948b0 con 0x560a60ba7d10 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.790+0000 7f0efca00640 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560a60ba7d10 0x560a60baa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.791+0000 7f0efca00640 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560a60ba7d10 0x560a60baa130 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f0ec3d5287b4e23 server_cookie=69a8eea97cf76e1f in_seq=0 out_seq=0 2024-04-17T07:17:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.791+0000 7f0ede000640 10 client.16488 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:17:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0ede000640 1 -- 172.21.15.86:0/1425330405 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a60a948b0 con 0x560a60ba7d10 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0ede000640 10 client.16488 handle_command_reply: tid=0 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0f0218f740 1 client.16488 shutdown 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0f0218f740 1 -- 172.21.15.86:0/1425330405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 msgr2=0x560a60b3b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0f0218f740 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3b760 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0004210 tx=0x7f0ef0004240 comp rx=0 tx=0).stop 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0f0218f740 1 -- 172.21.15.86:0/1425330405 shutdown_connections 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0f0218f740 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a60b310d0 0x560a60b3b760 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0f0218f740 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560a60ba7d10 0x560a60baa130 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0f0218f740 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a60b2bf60 0x560a60b3a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0f0218f740 1 --2- 172.21.15.86:0/1425330405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a60b2b570 0x560a60b3a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0ede000640 0 client.16488 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.792+0000 7f0f0218f740 1 -- 172.21.15.86:0/1425330405 >> 172.21.15.86:0/1425330405 conn(0x560a60b226d0 msgr2=0x560a60b261c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.793+0000 7f0f0218f740 1 -- 172.21.15.86:0/1425330405 shutdown_connections 2024-04-17T07:17:59.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.793+0000 7f0f0218f740 1 -- 172.21.15.86:0/1425330405 wait complete. 2024-04-17T07:17:59.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.793+0000 7f0f0218f740 20 client.16488 trim_cache size 0 max 16384 2024-04-17T07:17:59.795 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 -- 172.21.15.86:0/3119954619 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ee00a3bb0 msgr2=0x7f0ee00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ee00a3bb0 0x7f0ee00a6070 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0ef002c710 tx=0x7f0ef00324c0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 -- 172.21.15.86:0/3119954619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef8100d70 msgr2=0x7f0ef8197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef8100d70 0x7f0ef8197f90 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00d770 tx=0x7f0eec00dc30 comp rx=0 tx=0).stop 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 -- 172.21.15.86:0/3119954619 shutdown_connections 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ee00a3bb0 0x7f0ee00a6070 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ef8102920 0x7f0ef819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ef8101f60 0x7f0ef81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 --2- 172.21.15.86:0/3119954619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef8100d70 0x7f0ef8197f90 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 -- 172.21.15.86:0/3119954619 >> 172.21.15.86:0/3119954619 conn(0x7f0ef80fc300 msgr2=0x7f0ef8105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 -- 172.21.15.86:0/3119954619 shutdown_connections 2024-04-17T07:17:59.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:17:59.794+0000 7f0efde00640 1 -- 172.21.15.86:0/3119954619 wait complete. 2024-04-17T07:17:59.810 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:17:59 smithi086 ceph-mon[31733]: pgmap v549: 129 pgs: 129 active+clean; 628 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 19 op/s 2024-04-17T07:17:59.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:18:00.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:17:59 smithi098 ceph-mon[35418]: pgmap v549: 129 pgs: 129 active+clean; 628 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 19 op/s 2024-04-17T07:18:00.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:17:59 smithi103 ceph-mon[35495]: pgmap v549: 129 pgs: 129 active+clean; 628 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 19 op/s 2024-04-17T07:18:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.054+0000 7f9a6b800640 1 -- 172.21.15.86:0/373888251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a641031a0 msgr2=0x7f9a64103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.054+0000 7f9a6b800640 1 --2- 172.21.15.86:0/373888251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a641031a0 0x7f9a64103620 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009900 tx=0x7f9a4c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:18:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.054+0000 7f9a6b800640 1 -- 172.21.15.86:0/373888251 shutdown_connections 2024-04-17T07:18:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.054+0000 7f9a6b800640 1 --2- 172.21.15.86:0/373888251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a64103b60 0x7f9a6410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.054+0000 7f9a6b800640 1 --2- 172.21.15.86:0/373888251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a641031a0 0x7f9a64103620 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.054+0000 7f9a6b800640 1 --2- 172.21.15.86:0/373888251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a64101fb0 0x7f9a641023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.054+0000 7f9a6b800640 1 -- 172.21.15.86:0/373888251 >> 172.21.15.86:0/373888251 conn(0x7f9a640fd540 msgr2=0x7f9a640ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.054+0000 7f9a6b800640 1 -- 172.21.15.86:0/373888251 shutdown_connections 2024-04-17T07:18:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.054+0000 7f9a6b800640 1 -- 172.21.15.86:0/373888251 wait complete. 2024-04-17T07:18:00.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.055+0000 7f9a6b800640 1 Processor -- start 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.056+0000 7f9a6b800640 1 -- start start 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.056+0000 7f9a6b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a64101fb0 0x7f9a6419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.056+0000 7f9a6b800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a641031a0 0x7f9a6419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.056+0000 7f9a6b800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a64103b60 0x7f9a641a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.056+0000 7f9a6b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6410db40 con 0x7f9a64101fb0 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.056+0000 7f9a6b800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6410d9c0 con 0x7f9a641031a0 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.056+0000 7f9a6b800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6410dcc0 con 0x7f9a64103b60 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.056+0000 7f9a68e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a64101fb0 0x7f9a6419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.056+0000 7f9a68e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a64101fb0 0x7f9a6419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56450/0 (socket says 172.21.15.86:56450) 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.056+0000 7f9a68e00640 1 -- 172.21.15.86:0/3573027218 learned_addr learned my addr 172.21.15.86:0/3573027218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a63e00640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a641031a0 0x7f9a6419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a69800640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a64103b60 0x7f9a641a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a68e00640 1 -- 172.21.15.86:0/3573027218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a64103b60 msgr2=0x7f9a641a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a68e00640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a64103b60 0x7f9a641a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a68e00640 1 -- 172.21.15.86:0/3573027218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a641031a0 msgr2=0x7f9a6419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a68e00640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a641031a0 0x7f9a6419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a68e00640 1 -- 172.21.15.86:0/3573027218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a641a1fd0 con 0x7f9a64101fb0 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a63e00640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a641031a0 0x7f9a6419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a68e00640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a64101fb0 0x7f9a6419a3b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400d770 tx=0x7f9a5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a61600640 1 -- 172.21.15.86:0/3573027218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54017070 con 0x7f9a64101fb0 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a61600640 1 -- 172.21.15.86:0/3573027218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a54009c30 con 0x7f9a64101fb0 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a6b800640 1 -- 172.21.15.86:0/3573027218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a641a22c0 con 0x7f9a64101fb0 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.057+0000 7f9a61600640 1 -- 172.21.15.86:0/3573027218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54003d00 con 0x7f9a64101fb0 2024-04-17T07:18:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.058+0000 7f9a6b800640 1 -- 172.21.15.86:0/3573027218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a641a2860 con 0x7f9a64101fb0 2024-04-17T07:18:00.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.061+0000 7f9a6e614740 1 Processor -- start 2024-04-17T07:18:00.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.061+0000 7f9a6e614740 1 -- start start 2024-04-17T07:18:00.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.061+0000 7f9a61600640 1 -- 172.21.15.86:0/3573027218 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9a54003ea0 con 0x7f9a64101fb0 2024-04-17T07:18:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.061+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.061+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a5990 0x555fc29aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.061+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.061+0000 7f9a6e614740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555fc290b020 con 0x555fc29aab00 2024-04-17T07:18:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.061+0000 7f9a6e614740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555fc28b1460 con 0x555fc29a5990 2024-04-17T07:18:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.061+0000 7f9a6e614740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555fc2906fe0 con 0x555fc29a4fa0 2024-04-17T07:18:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.062+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.062+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29acf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56466/0 (socket says 172.21.15.86:56466) 2024-04-17T07:18:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.062+0000 7f9a63e00640 1 -- 172.21.15.86:0/4093217719 learned_addr learned my addr 172.21.15.86:0/4093217719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:00.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.062+0000 7f9a68e00640 1 --2- 172.21.15.86:0/4093217719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a5990 0x555fc29aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.062+0000 7f9a69800640 1 --2- 172.21.15.86:0/4093217719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.062+0000 7f9a63e00640 1 -- 172.21.15.86:0/4093217719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 msgr2=0x555fc29a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.062+0000 7f9a63e00640 1 --2- 172.21.15.86:0/4093217719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.062+0000 7f9a63e00640 1 -- 172.21.15.86:0/4093217719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a5990 msgr2=0x555fc29aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.062+0000 7f9a63e00640 1 --2- 172.21.15.86:0/4093217719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a5990 0x555fc29aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.062+0000 7f9a63e00640 1 -- 172.21.15.86:0/4093217719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fc28b9040 con 0x555fc29aab00 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.063+0000 7f9a63e00640 1 --2- 172.21.15.86:0/4093217719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29acf90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009a20 tx=0x7f9a4c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73708352f1bb2da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.063+0000 7f9a61600640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a480a3c30 0x7f9a480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.063+0000 7f9a5a000640 1 -- 172.21.15.86:0/4093217719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c043020 con 0x555fc29aab00 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.063+0000 7f9a5a000640 1 -- 172.21.15.86:0/4093217719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a4c02ca60 con 0x555fc29aab00 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.063+0000 7f9a5a000640 1 -- 172.21.15.86:0/4093217719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0046c0 con 0x555fc29aab00 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.063+0000 7f9a61600640 1 -- 172.21.15.86:0/3573027218 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9a540d0540 con 0x7f9a64101fb0 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.063+0000 7f9a69800640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a480a3c30 0x7f9a480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.064+0000 7f9a6e614740 1 -- 172.21.15.86:0/4093217719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 msgr2=0x555fc29acf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:00.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.064+0000 7f9a6e614740 1 --2- 172.21.15.86:0/4093217719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29acf90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009a20 tx=0x7f9a4c009650 comp rx=0 tx=0).stop 2024-04-17T07:18:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.064+0000 7f9a69800640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a480a3c30 0x7f9a480a60f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x555fc29a61f0 tx=0x7f9a5c009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.064+0000 7f9a6e614740 1 -- 172.21.15.86:0/4093217719 shutdown_connections 2024-04-17T07:18:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.064+0000 7f9a6e614740 1 --2- 172.21.15.86:0/4093217719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29acf90 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.064+0000 7f9a6e614740 1 --2- 172.21.15.86:0/4093217719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a5990 0x555fc29aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.064+0000 7f9a6e614740 1 --2- 172.21.15.86:0/4093217719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.064+0000 7f9a6e614740 1 -- 172.21.15.86:0/4093217719 >> 172.21.15.86:0/4093217719 conn(0x555fc29a0510 msgr2=0x555fc29a0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.064+0000 7f9a6e614740 1 -- 172.21.15.86:0/4093217719 shutdown_connections 2024-04-17T07:18:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.064+0000 7f9a6e614740 1 -- 172.21.15.86:0/4093217719 wait complete. 2024-04-17T07:18:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.065+0000 7f9a6e614740 1 -- start start 2024-04-17T07:18:00.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.065+0000 7f9a6e614740 1 Processor -- start 2024-04-17T07:18:00.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.065+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.065+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29a5990 0x555fc29af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.065+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29aab00 0x555fc29b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.065+0000 7f9a6e614740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555fc29ada50 con 0x555fc29a5990 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.065+0000 7f9a6e614740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555fc28c4830 con 0x555fc29aab00 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.065+0000 7f9a6e614740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555fc29adbd0 con 0x555fc29a4fa0 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.065+0000 7f9a6e614740 10 client.0 _ll_register_callbacks cb 0x555fc2892fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.065+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29a5990 0x555fc29af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29a5990 0x555fc29af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56474/0 (socket says 172.21.15.86:56474) 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a63e00640 1 -- 172.21.15.86:0/3915637413 learned_addr learned my addr 172.21.15.86:0/3915637413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a63e00640 1 -- 172.21.15.86:0/3915637413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 msgr2=0x555fc29af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a63e00640 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a63e00640 1 -- 172.21.15.86:0/3915637413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29aab00 msgr2=0x555fc29b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a63e00640 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29aab00 0x555fc29b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a63e00640 1 -- 172.21.15.86:0/3915637413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fc28c8920 con 0x555fc29a5990 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a63e00640 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29a5990 0x555fc29af960 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c005c20 tx=0x7f9a4c02c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87053518b1af389f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a52a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a52a00640 1 -- 172.21.15.86:0/3915637413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c043030 con 0x555fc29a5990 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.066+0000 7f9a52a00640 1 -- 172.21.15.86:0/3915637413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9a4c0037f0 con 0x555fc29a5990 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.067+0000 7f9a6e614740 1 -- 172.21.15.86:0/3915637413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a4c02ca60 con 0x555fc29a5990 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.067+0000 7f9a52a00640 1 -- 172.21.15.86:0/3915637413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a4c058050 con 0x555fc29a5990 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.067+0000 7f9a6e614740 10 client.16500 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.067+0000 7f9a6e614740 1 -- 172.21.15.86:0/3915637413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555fc28967f0 con 0x555fc29a5990 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.067+0000 7f9a52a00640 1 -- 172.21.15.86:0/3915637413 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a4c02ca60 con 0x555fc29a5990 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.067+0000 7f9a52a00640 1 -- 172.21.15.86:0/3915637413 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9a4c04ac40 con 0x555fc29a5990 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.067+0000 7f9a6e614740 10 client.16500 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.067+0000 7f9a6e614740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.067+0000 7f9a6e614740 20 client.16500 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.067+0000 7f9a6e614740 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555fc29bab00 0x555fc29bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.068+0000 7f9a6e614740 4 client.16500 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.068+0000 7f9a6e614740 1 -- 172.21.15.86:0/3915637413 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fc2918a40 con 0x555fc29bab00 2024-04-17T07:18:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.068+0000 7f9a68e00640 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555fc29bab00 0x555fc29bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.069+0000 7f9a68e00640 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555fc29bab00 0x555fc29bcef0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4555cb06f1fbeb server_cookie=f05fde7d8a94517a in_seq=0 out_seq=0 2024-04-17T07:18:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.069+0000 7f9a52a00640 10 client.16500 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.072+0000 7f9a52a00640 1 -- 172.21.15.86:0/3915637413 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fc2918a40 con 0x555fc29bab00 2024-04-17T07:18:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.072+0000 7f9a52a00640 10 client.16500 handle_command_reply: tid=0 2024-04-17T07:18:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.072+0000 7f9a6e614740 1 client.16500 shutdown 2024-04-17T07:18:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.072+0000 7f9a6e614740 1 -- 172.21.15.86:0/3915637413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29a5990 msgr2=0x555fc29af960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.072+0000 7f9a6e614740 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29a5990 0x555fc29af960 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c005c20 tx=0x7f9a4c02c3b0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.073+0000 7f9a6e614740 1 -- 172.21.15.86:0/3915637413 shutdown_connections 2024-04-17T07:18:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.073+0000 7f9a6e614740 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555fc29bab00 0x555fc29bcef0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.073+0000 7f9a6e614740 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29aab00 0x555fc29b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.073+0000 7f9a6e614740 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29a5990 0x555fc29af960 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.073+0000 7f9a52a00640 0 client.16500 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.073+0000 7f9a6e614740 1 --2- 172.21.15.86:0/3915637413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.073+0000 7f9a6e614740 1 -- 172.21.15.86:0/3915637413 >> 172.21.15.86:0/3915637413 conn(0x555fc29a0510 msgr2=0x555fc299f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.073+0000 7f9a6e614740 1 -- 172.21.15.86:0/3915637413 shutdown_connections 2024-04-17T07:18:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.073+0000 7f9a6e614740 1 -- 172.21.15.86:0/3915637413 wait complete. 2024-04-17T07:18:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.073+0000 7f9a6e614740 20 client.16500 trim_cache size 0 max 16384 2024-04-17T07:18:00.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.090+0000 7f9a6e614740 1 Processor -- start 2024-04-17T07:18:00.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.090+0000 7f9a6e614740 1 -- start start 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.090+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a4fa0 0x555fc29a6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.090+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a5990 0x555fc29a7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.090+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.090+0000 7f9a6e614740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a641acd00 con 0x555fc29aab00 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.090+0000 7f9a6e614740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a641ae840 con 0x555fc29a4fa0 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.090+0000 7f9a6e614740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a641ad3d0 con 0x555fc29a5990 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.091+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a5990 0x555fc29a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.091+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a5990 0x555fc29a7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:53126/0 (socket says 172.21.15.86:53126) 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.091+0000 7f9a63e00640 1 -- 172.21.15.86:0/56629360 learned_addr learned my addr 172.21.15.86:0/56629360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.091+0000 7f9a69800640 1 --2- 172.21.15.86:0/56629360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.092+0000 7f9a63e00640 1 -- 172.21.15.86:0/56629360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a4fa0 msgr2=0x555fc29a6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.092+0000 7f9a63e00640 1 --2- 172.21.15.86:0/56629360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a4fa0 0x555fc29a6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.092+0000 7f9a63e00640 1 -- 172.21.15.86:0/56629360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 msgr2=0x555fc29b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.092+0000 7f9a63e00640 1 --2- 172.21.15.86:0/56629360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.092+0000 7f9a63e00640 1 -- 172.21.15.86:0/56629360 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fc261d3f0 con 0x555fc29a5990 2024-04-17T07:18:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.092+0000 7f9a69800640 1 --2- 172.21.15.86:0/56629360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.092+0000 7f9a63e00640 1 --2- 172.21.15.86:0/56629360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a5990 0x555fc29a7390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c02c170 tx=0x7f9a4c02c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d783b47db17213fa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.093+0000 7f9a59600640 1 -- 172.21.15.86:0/56629360 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c061070 con 0x555fc29a5990 2024-04-17T07:18:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.093+0000 7f9a59600640 1 -- 172.21.15.86:0/56629360 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a4c02ca60 con 0x555fc29a5990 2024-04-17T07:18:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.093+0000 7f9a59600640 1 -- 172.21.15.86:0/56629360 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c043040 con 0x555fc29a5990 2024-04-17T07:18:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.094+0000 7f9a6e614740 1 -- 172.21.15.86:0/56629360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a5990 msgr2=0x555fc29a7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.094+0000 7f9a6e614740 1 --2- 172.21.15.86:0/56629360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a5990 0x555fc29a7390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c02c170 tx=0x7f9a4c02c5d0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.094+0000 7f9a6e614740 1 -- 172.21.15.86:0/56629360 shutdown_connections 2024-04-17T07:18:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.094+0000 7f9a6e614740 1 --2- 172.21.15.86:0/56629360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.094+0000 7f9a6e614740 1 --2- 172.21.15.86:0/56629360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a5990 0x555fc29a7390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.094+0000 7f9a6e614740 1 --2- 172.21.15.86:0/56629360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a4fa0 0x555fc29a6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.094+0000 7f9a6e614740 1 -- 172.21.15.86:0/56629360 >> 172.21.15.86:0/56629360 conn(0x555fc29a0510 msgr2=0x555fc29af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.095+0000 7f9a6e614740 1 -- 172.21.15.86:0/56629360 shutdown_connections 2024-04-17T07:18:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.095+0000 7f9a6e614740 1 -- 172.21.15.86:0/56629360 wait complete. 2024-04-17T07:18:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.095+0000 7f9a6e614740 1 -- start start 2024-04-17T07:18:00.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.095+0000 7f9a6e614740 1 Processor -- start 2024-04-17T07:18:00.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.096+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.096+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a5990 0x555fc29b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.096+0000 7f9a6e614740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.096+0000 7f9a6e614740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a641ae170 con 0x555fc29aab00 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.096+0000 7f9a6e614740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a641adaa0 con 0x555fc29a5990 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.096+0000 7f9a6e614740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a641aacd0 con 0x555fc29a4fa0 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.096+0000 7f9a6e614740 10 client.0 _ll_register_callbacks cb 0x7f9a4c004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.096+0000 7f9a69800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.096+0000 7f9a69800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56504/0 (socket says 172.21.15.86:56504) 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.096+0000 7f9a69800640 1 -- 172.21.15.86:0/42483696 learned_addr learned my addr 172.21.15.86:0/42483696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.097+0000 7f9a68e00640 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.097+0000 7f9a63e00640 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a5990 0x555fc29b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.097+0000 7f9a69800640 1 -- 172.21.15.86:0/42483696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 msgr2=0x555fc29b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.097+0000 7f9a69800640 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.097+0000 7f9a69800640 1 -- 172.21.15.86:0/42483696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a5990 msgr2=0x555fc29b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.097+0000 7f9a69800640 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a5990 0x555fc29b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.097+0000 7f9a69800640 1 -- 172.21.15.86:0/42483696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fc26e77d0 con 0x555fc29aab00 2024-04-17T07:18:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.097+0000 7f9a68e00640 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.097+0000 7f9a69800640 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b5330 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0127c0 tx=0x7f9a5c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d04ded661a7af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.097+0000 7f9a52a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a52a00640 1 -- 172.21.15.86:0/42483696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c01f940 con 0x555fc29aab00 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a6e614740 1 -- 172.21.15.86:0/42483696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a4c02ca60 con 0x555fc29aab00 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a52a00640 1 -- 172.21.15.86:0/42483696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9a5c01fae0 con 0x555fc29aab00 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a52a00640 1 -- 172.21.15.86:0/42483696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a5c01fce0 con 0x555fc29aab00 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a6e614740 10 client.16506 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a6e614740 1 -- 172.21.15.86:0/42483696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555fc29b6b50 con 0x555fc29aab00 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a52a00640 1 -- 172.21.15.86:0/42483696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a4c02ca60 con 0x555fc29aab00 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a52a00640 1 -- 172.21.15.86:0/42483696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9a5c00f040 con 0x555fc29aab00 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a6e614740 10 client.16506 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a6e614740 10 client.16506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a6e614740 20 client.16506 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:00.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a6e614740 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555fc2a21670 0x555fc2a23a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a6e614740 4 client.16506 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:18:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.098+0000 7f9a6e614740 1 -- 172.21.15.86:0/42483696 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fc29128b0 con 0x555fc2a21670 2024-04-17T07:18:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.099+0000 7f9a68e00640 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555fc2a21670 0x555fc2a23a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.100+0000 7f9a68e00640 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555fc2a21670 0x555fc2a23a90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c07238b2141bbc3 server_cookie=5009759479e8531f in_seq=0 out_seq=0 2024-04-17T07:18:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.100+0000 7f9a52a00640 10 client.16506 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.100+0000 7f9a52a00640 1 -- 172.21.15.86:0/42483696 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fc29128b0 con 0x555fc2a21670 2024-04-17T07:18:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.100+0000 7f9a52a00640 10 client.16506 handle_command_reply: tid=0 2024-04-17T07:18:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.100+0000 7f9a6e614740 1 client.16506 shutdown 2024-04-17T07:18:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 1 -- 172.21.15.86:0/42483696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 msgr2=0x555fc29b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b5330 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0127c0 tx=0x7f9a5c004ab0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 1 -- 172.21.15.86:0/42483696 shutdown_connections 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555fc2a21670 0x555fc2a23a90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc29aab00 0x555fc29b5330 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555fc29a5990 0x555fc29b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 1 --2- 172.21.15.86:0/42483696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555fc29a4fa0 0x555fc29b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a52a00640 0 client.16506 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 1 -- 172.21.15.86:0/42483696 >> 172.21.15.86:0/42483696 conn(0x555fc29a0510 msgr2=0x555fc29af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 1 -- 172.21.15.86:0/42483696 shutdown_connections 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 1 -- 172.21.15.86:0/42483696 wait complete. 2024-04-17T07:18:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.101+0000 7f9a6e614740 20 client.16506 trim_cache size 0 max 16384 2024-04-17T07:18:00.104 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:18:00.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.102+0000 7f9a6b800640 1 -- 172.21.15.86:0/3573027218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a480a3c30 msgr2=0x7f9a480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:00.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.102+0000 7f9a6b800640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a480a3c30 0x7f9a480a60f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x555fc29a61f0 tx=0x7f9a5c009580 comp rx=0 tx=0).stop 2024-04-17T07:18:00.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.102+0000 7f9a6b800640 1 -- 172.21.15.86:0/3573027218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a64101fb0 msgr2=0x7f9a6419a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:00.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.102+0000 7f9a6b800640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a64101fb0 0x7f9a6419a3b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400d770 tx=0x7f9a5400dc30 comp rx=0 tx=0).stop 2024-04-17T07:18:00.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.102+0000 7f9a6b800640 1 -- 172.21.15.86:0/3573027218 shutdown_connections 2024-04-17T07:18:00.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.102+0000 7f9a6b800640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a480a3c30 0x7f9a480a60f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.102+0000 7f9a6b800640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a64103b60 0x7f9a641a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.102+0000 7f9a6b800640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a641031a0 0x7f9a6419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.102+0000 7f9a6b800640 1 --2- 172.21.15.86:0/3573027218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a64101fb0 0x7f9a6419a3b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:00.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.102+0000 7f9a6b800640 1 -- 172.21.15.86:0/3573027218 >> 172.21.15.86:0/3573027218 conn(0x7f9a640fd540 msgr2=0x7f9a64074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:00.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.103+0000 7f9a6b800640 1 -- 172.21.15.86:0/3573027218 shutdown_connections 2024-04-17T07:18:00.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:00.103+0000 7f9a6b800640 1 -- 172.21.15.86:0/3573027218 wait complete. 2024-04-17T07:18:01.122 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:18:01.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:18:01.122 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:18:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.358+0000 7f9c6aa00640 1 -- 172.21.15.86:0/618357899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c64103080 msgr2=0x7f9c64103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.358+0000 7f9c6aa00640 1 --2- 172.21.15.86:0/618357899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c64103080 0x7f9c64103500 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009900 tx=0x7f9c5002bc30 comp rx=0 tx=0).stop 2024-04-17T07:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.358+0000 7f9c6aa00640 1 -- 172.21.15.86:0/618357899 shutdown_connections 2024-04-17T07:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.358+0000 7f9c6aa00640 1 --2- 172.21.15.86:0/618357899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c64103a40 0x7f9c6410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.358+0000 7f9c6aa00640 1 --2- 172.21.15.86:0/618357899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c64103080 0x7f9c64103500 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.358+0000 7f9c6aa00640 1 --2- 172.21.15.86:0/618357899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c64101ea0 0x7f9c641022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.358+0000 7f9c6aa00640 1 -- 172.21.15.86:0/618357899 >> 172.21.15.86:0/618357899 conn(0x7f9c640fd460 msgr2=0x7f9c640ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.358+0000 7f9c6aa00640 1 -- 172.21.15.86:0/618357899 shutdown_connections 2024-04-17T07:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.358+0000 7f9c6aa00640 1 -- 172.21.15.86:0/618357899 wait complete. 2024-04-17T07:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.359+0000 7f9c6aa00640 1 Processor -- start 2024-04-17T07:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.360+0000 7f9c6aa00640 1 -- start start 2024-04-17T07:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.360+0000 7f9c6aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c64101ea0 0x7f9c6419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.360+0000 7f9c6aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c64103080 0x7f9c6419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.360+0000 7f9c6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c64103a40 0x7f9c641a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.360+0000 7f9c6aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6410d9e0 con 0x7f9c64103a40 2024-04-17T07:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.360+0000 7f9c6aa00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6410d860 con 0x7f9c64101ea0 2024-04-17T07:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.360+0000 7f9c6aa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6410db60 con 0x7f9c64103080 2024-04-17T07:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.360+0000 7f9c6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c64103a40 0x7f9c641a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.360+0000 7f9c6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c64103a40 0x7f9c641a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56512/0 (socket says 172.21.15.86:56512) 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.360+0000 7f9c6a000640 1 -- 172.21.15.86:0/2135732993 learned_addr learned my addr 172.21.15.86:0/2135732993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.361+0000 7f9c69600640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c64101ea0 0x7f9c6419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.361+0000 7f9c68c00640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c64103080 0x7f9c6419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.361+0000 7f9c6a000640 1 -- 172.21.15.86:0/2135732993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c64103080 msgr2=0x7f9c6419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.361+0000 7f9c6a000640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c64103080 0x7f9c6419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.361+0000 7f9c6a000640 1 -- 172.21.15.86:0/2135732993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c64101ea0 msgr2=0x7f9c6419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.361+0000 7f9c6a000640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c64101ea0 0x7f9c6419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.361+0000 7f9c6a000640 1 -- 172.21.15.86:0/2135732993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c641a1ee0 con 0x7f9c64103a40 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.361+0000 7f9c69600640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c64101ea0 0x7f9c6419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.361+0000 7f9c6a000640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c64103a40 0x7f9c641a1870 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00cce0 tx=0x7f9c5c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.362+0000 7f9c62000640 1 -- 172.21.15.86:0/2135732993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c00f450 con 0x7f9c64103a40 2024-04-17T07:18:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.362+0000 7f9c6aa00640 1 -- 172.21.15.86:0/2135732993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c641a21d0 con 0x7f9c64103a40 2024-04-17T07:18:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.362+0000 7f9c62000640 1 -- 172.21.15.86:0/2135732993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c5c00faa0 con 0x7f9c64103a40 2024-04-17T07:18:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.362+0000 7f9c62000640 1 -- 172.21.15.86:0/2135732993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c018b70 con 0x7f9c64103a40 2024-04-17T07:18:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.362+0000 7f9c6aa00640 1 -- 172.21.15.86:0/2135732993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c641a26d0 con 0x7f9c64103a40 2024-04-17T07:18:01.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.365+0000 7f9c6ed74740 1 Processor -- start 2024-04-17T07:18:01.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.366+0000 7f9c62000640 1 -- 172.21.15.86:0/2135732993 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9c5c018d10 con 0x7f9c64103a40 2024-04-17T07:18:01.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.366+0000 7f9c6ed74740 1 -- start start 2024-04-17T07:18:01.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.366+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe140a0 0x556a1fe144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.366+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 0x556a1fe19590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.366+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe1c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.366+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a1fd79cb0 con 0x556a1fe19c00 2024-04-17T07:18:01.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.366+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556a1fd20370 con 0x556a1fe14a90 2024-04-17T07:18:01.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.366+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a1fd75c70 con 0x556a1fe140a0 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.366+0000 7f9c68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe1c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.367+0000 7f9c68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe1c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56522/0 (socket says 172.21.15.86:56522) 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.367+0000 7f9c68c00640 1 -- 172.21.15.86:0/3667089792 learned_addr learned my addr 172.21.15.86:0/3667089792 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.367+0000 7f9c68c00640 1 --2- 172.21.15.86:0/3667089792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe140a0 0x556a1fe144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.367+0000 7f9c69600640 1 --2- 172.21.15.86:0/3667089792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 0x556a1fe19590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.367+0000 7f9c68c00640 1 -- 172.21.15.86:0/3667089792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe140a0 msgr2=0x556a1fe144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.367+0000 7f9c68c00640 1 --2- 172.21.15.86:0/3667089792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe140a0 0x556a1fe144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.367+0000 7f9c68c00640 1 -- 172.21.15.86:0/3667089792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 msgr2=0x556a1fe19590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.367+0000 7f9c68c00640 1 --2- 172.21.15.86:0/3667089792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 0x556a1fe19590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.367+0000 7f9c68c00640 1 -- 172.21.15.86:0/3667089792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1fd3f910 con 0x556a1fe19c00 2024-04-17T07:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.367+0000 7f9c68c00640 1 --2- 172.21.15.86:0/3667089792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe1c090 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9c50005940 tx=0x7f9c500058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc264d827cf6618 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.368+0000 7f9c56600640 1 -- 172.21.15.86:0/3667089792 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50049070 con 0x556a1fe19c00 2024-04-17T07:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.368+0000 7f9c56600640 1 -- 172.21.15.86:0/3667089792 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c500043e0 con 0x556a1fe19c00 2024-04-17T07:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.368+0000 7f9c62000640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9c4c0a3bb0 0x7f9c4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.368+0000 7f9c56600640 1 -- 172.21.15.86:0/3667089792 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50004bf0 con 0x556a1fe19c00 2024-04-17T07:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.368+0000 7f9c62000640 1 -- 172.21.15.86:0/2135732993 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9c5c0d0cd0 con 0x7f9c64103a40 2024-04-17T07:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.368+0000 7f9c69600640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9c4c0a3bb0 0x7f9c4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.369+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3667089792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 msgr2=0x556a1fe1c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.369+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3667089792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe1c090 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9c50005940 tx=0x7f9c500058b0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.369+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3667089792 shutdown_connections 2024-04-17T07:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.369+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3667089792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe1c090 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.369+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3667089792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 0x556a1fe19590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.369+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3667089792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe140a0 0x556a1fe144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.369+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3667089792 >> 172.21.15.86:0/3667089792 conn(0x556a1fe0f610 msgr2=0x556a1fe0fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.369+0000 7f9c69600640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9c4c0a3bb0 0x7f9c4c0a6070 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x556a1fe14e90 tx=0x7f9c5800b710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.369+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3667089792 shutdown_connections 2024-04-17T07:18:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.369+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3667089792 wait complete. 2024-04-17T07:18:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.370+0000 7f9c6ed74740 1 -- start start 2024-04-17T07:18:01.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.370+0000 7f9c6ed74740 1 Processor -- start 2024-04-17T07:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.370+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe140a0 0x556a1fe1e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.370+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe14a90 0x556a1fe1eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.370+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe19c00 0x556a1fe25b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a1fe1cb50 con 0x556a1fe14a90 2024-04-17T07:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556a1fd33730 con 0x556a1fe140a0 2024-04-17T07:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a1fe1ccd0 con 0x556a1fe19c00 2024-04-17T07:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c6ed74740 10 client.0 _ll_register_callbacks cb 0x556a1fd35b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe14a90 0x556a1fe1eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe14a90 0x556a1fe1eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56534/0 (socket says 172.21.15.86:56534) 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c68c00640 1 -- 172.21.15.86:0/3218076991 learned_addr learned my addr 172.21.15.86:0/3218076991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c69600640 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe140a0 0x556a1fe1e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c6a000640 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe19c00 0x556a1fe25b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c68c00640 1 -- 172.21.15.86:0/3218076991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe19c00 msgr2=0x556a1fe25b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c68c00640 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe19c00 0x556a1fe25b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c68c00640 1 -- 172.21.15.86:0/3218076991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe140a0 msgr2=0x556a1fe1e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c68c00640 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe140a0 0x556a1fe1e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c68c00640 1 -- 172.21.15.86:0/3218076991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a1fb547e0 con 0x556a1fe14a90 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.371+0000 7f9c6a000640 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe19c00 0x556a1fe25b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.372+0000 7f9c68c00640 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe14a90 0x556a1fe1eb50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000c00 tx=0x7f9c50005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b2d151a6d09156 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.372+0000 7f9c4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.372+0000 7f9c4aa00640 1 -- 172.21.15.86:0/3218076991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50049050 con 0x556a1fe14a90 2024-04-17T07:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.372+0000 7f9c4aa00640 1 -- 172.21.15.86:0/3218076991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9c50003db0 con 0x556a1fe14a90 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.372+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3218076991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c500043e0 con 0x556a1fe14a90 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.372+0000 7f9c4aa00640 1 -- 172.21.15.86:0/3218076991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9c5005e050 con 0x556a1fe14a90 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.372+0000 7f9c6ed74740 10 client.16518 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.372+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3218076991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a1fbb4f70 con 0x556a1fe14a90 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.373+0000 7f9c4aa00640 1 -- 172.21.15.86:0/3218076991 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c500043e0 con 0x556a1fe14a90 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.373+0000 7f9c4aa00640 1 -- 172.21.15.86:0/3218076991 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9c50040040 con 0x556a1fe14a90 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.373+0000 7f9c6ed74740 10 client.16518 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.373+0000 7f9c6ed74740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.373+0000 7f9c6ed74740 20 client.16518 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.373+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a1fe2a080 0x556a1fe2c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.373+0000 7f9c6ed74740 4 client.16518 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.373+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3218076991 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a1fd876d0 con 0x556a1fe2a080 2024-04-17T07:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.374+0000 7f9c69600640 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a1fe2a080 0x556a1fe2c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.375+0000 7f9c69600640 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a1fe2a080 0x556a1fe2c470 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=507fb1031c7f0d09 server_cookie=9de91d2a6f519b4e in_seq=0 out_seq=0 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.375+0000 7f9c4aa00640 10 client.16518 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c4aa00640 1 -- 172.21.15.86:0/3218076991 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a1fd876d0 con 0x556a1fe2a080 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c4aa00640 10 client.16518 handle_command_reply: tid=0 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 client.16518 shutdown 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3218076991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe14a90 msgr2=0x556a1fe1eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe14a90 0x556a1fe1eb50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000c00 tx=0x7f9c50005c00 comp rx=0 tx=0).stop 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3218076991 shutdown_connections 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a1fe2a080 0x556a1fe2c470 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe19c00 0x556a1fe25b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe14a90 0x556a1fe1eb50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c4aa00640 0 client.16518 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:01.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3218076991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe140a0 0x556a1fe1e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3218076991 >> 172.21.15.86:0/3218076991 conn(0x556a1fe0f610 msgr2=0x556a1fe0e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3218076991 shutdown_connections 2024-04-17T07:18:01.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3218076991 wait complete. 2024-04-17T07:18:01.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.377+0000 7f9c6ed74740 20 client.16518 trim_cache size 0 max 16384 2024-04-17T07:18:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.399+0000 7f9c6ed74740 1 Processor -- start 2024-04-17T07:18:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.399+0000 7f9c6ed74740 1 -- start start 2024-04-17T07:18:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.399+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe140a0 0x556a1fe0e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.399+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 0x556a1fe25b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.399+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe0e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.399+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641ac8e0 con 0x556a1fe19c00 2024-04-17T07:18:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.399+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641ae420 con 0x556a1fe14a90 2024-04-17T07:18:01.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.399+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641acfb0 con 0x556a1fe140a0 2024-04-17T07:18:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.400+0000 7f9c6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe0e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.400+0000 7f9c6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe0e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56548/0 (socket says 172.21.15.86:56548) 2024-04-17T07:18:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.400+0000 7f9c6a000640 1 -- 172.21.15.86:0/3410494947 learned_addr learned my addr 172.21.15.86:0/3410494947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.400+0000 7f9c69600640 1 --2- 172.21.15.86:0/3410494947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe140a0 0x556a1fe0e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.400+0000 7f9c68c00640 1 --2- 172.21.15.86:0/3410494947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 0x556a1fe25b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.400+0000 7f9c6a000640 1 -- 172.21.15.86:0/3410494947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe140a0 msgr2=0x556a1fe0e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.401+0000 7f9c6a000640 1 --2- 172.21.15.86:0/3410494947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe140a0 0x556a1fe0e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.401+0000 7f9c6a000640 1 -- 172.21.15.86:0/3410494947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 msgr2=0x556a1fe25b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.401+0000 7f9c6a000640 1 --2- 172.21.15.86:0/3410494947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 0x556a1fe25b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.401+0000 7f9c6a000640 1 -- 172.21.15.86:0/3410494947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a1fe166d0 con 0x556a1fe19c00 2024-04-17T07:18:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.401+0000 7f9c68c00640 1 --2- 172.21.15.86:0/3410494947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 0x556a1fe25b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.401+0000 7f9c6a000640 1 --2- 172.21.15.86:0/3410494947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe0e640 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c018550 tx=0x7f9c5c00fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee27a72c22059d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.401+0000 7f9c55c00640 1 -- 172.21.15.86:0/3410494947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0a0f80 con 0x556a1fe19c00 2024-04-17T07:18:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.401+0000 7f9c55c00640 1 -- 172.21.15.86:0/3410494947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c5c0a5840 con 0x556a1fe19c00 2024-04-17T07:18:01.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.401+0000 7f9c55c00640 1 -- 172.21.15.86:0/3410494947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c09ce30 con 0x556a1fe19c00 2024-04-17T07:18:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.402+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3410494947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 msgr2=0x556a1fe0e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.402+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3410494947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe0e640 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c018550 tx=0x7f9c5c00fc50 comp rx=0 tx=0).stop 2024-04-17T07:18:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.402+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3410494947 shutdown_connections 2024-04-17T07:18:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.402+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3410494947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe19c00 0x556a1fe0e640 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.402+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3410494947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe14a90 0x556a1fe25b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.402+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/3410494947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe140a0 0x556a1fe0e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.402+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3410494947 >> 172.21.15.86:0/3410494947 conn(0x556a1fe0f610 msgr2=0x556a1fe1d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.403+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3410494947 shutdown_connections 2024-04-17T07:18:01.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.403+0000 7f9c6ed74740 1 -- 172.21.15.86:0/3410494947 wait complete. 2024-04-17T07:18:01.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.403+0000 7f9c6ed74740 1 -- start start 2024-04-17T07:18:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.404+0000 7f9c6ed74740 1 Processor -- start 2024-04-17T07:18:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.404+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe140a0 0x556a1fe25550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.404+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe14a90 0x556a1fe25a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.404+0000 7f9c6ed74740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe19c00 0x556a1fe23c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.404+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641add50 con 0x556a1fe140a0 2024-04-17T07:18:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.404+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641ad680 con 0x556a1fe19c00 2024-04-17T07:18:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.404+0000 7f9c6ed74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641aa8b0 con 0x556a1fe14a90 2024-04-17T07:18:01.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.404+0000 7f9c6ed74740 10 client.0 _ll_register_callbacks cb 0x7f9c50047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.405+0000 7f9c69600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe140a0 0x556a1fe25550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.405+0000 7f9c69600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe140a0 0x556a1fe25550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56558/0 (socket says 172.21.15.86:56558) 2024-04-17T07:18:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.405+0000 7f9c69600640 1 -- 172.21.15.86:0/1586637879 learned_addr learned my addr 172.21.15.86:0/1586637879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.405+0000 7f9c6a000640 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe19c00 0x556a1fe23c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.405+0000 7f9c68c00640 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe14a90 0x556a1fe25a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.406+0000 7f9c69600640 1 -- 172.21.15.86:0/1586637879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe14a90 msgr2=0x556a1fe25a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.406+0000 7f9c69600640 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe14a90 0x556a1fe25a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.406+0000 7f9c69600640 1 -- 172.21.15.86:0/1586637879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe19c00 msgr2=0x556a1fe23c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.406+0000 7f9c69600640 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe19c00 0x556a1fe23c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.406+0000 7f9c69600640 1 -- 172.21.15.86:0/1586637879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a1fa4c170 con 0x556a1fe140a0 2024-04-17T07:18:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.406+0000 7f9c69600640 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe140a0 0x556a1fe25550 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x556a1fe15500 tx=0x7f9c580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af58fa5cb7d8754a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.406+0000 7f9c4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.406+0000 7f9c4aa00640 1 -- 172.21.15.86:0/1586637879 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c580129b0 con 0x556a1fe140a0 2024-04-17T07:18:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.407+0000 7f9c4aa00640 1 -- 172.21.15.86:0/1586637879 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9c58012b50 con 0x556a1fe140a0 2024-04-17T07:18:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.407+0000 7f9c6ed74740 1 -- 172.21.15.86:0/1586637879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c5c0a5840 con 0x556a1fe140a0 2024-04-17T07:18:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.407+0000 7f9c4aa00640 1 -- 172.21.15.86:0/1586637879 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9c5803b050 con 0x556a1fe140a0 2024-04-17T07:18:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.407+0000 7f9c6ed74740 10 client.16524 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:01.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.407+0000 7f9c6ed74740 1 -- 172.21.15.86:0/1586637879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a1fe90ea0 con 0x556a1fe140a0 2024-04-17T07:18:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.407+0000 7f9c4aa00640 1 -- 172.21.15.86:0/1586637879 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c5c0a5840 con 0x556a1fe140a0 2024-04-17T07:18:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.408+0000 7f9c4aa00640 1 -- 172.21.15.86:0/1586637879 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9c5801f420 con 0x556a1fe140a0 2024-04-17T07:18:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.408+0000 7f9c6ed74740 10 client.16524 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.408+0000 7f9c6ed74740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.408+0000 7f9c6ed74740 20 client.16524 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.408+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a1fe91830 0x556a1fe93c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.408+0000 7f9c6ed74740 4 client.16524 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:18:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.408+0000 7f9c6ed74740 1 -- 172.21.15.86:0/1586637879 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556a1fd81540 con 0x556a1fe91830 2024-04-17T07:18:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.408+0000 7f9c68c00640 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a1fe91830 0x556a1fe93c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.409+0000 7f9c68c00640 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a1fe91830 0x556a1fe93c50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a370ddee0033a5a3 server_cookie=4d681263a17f9142 in_seq=0 out_seq=0 2024-04-17T07:18:01.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.409+0000 7f9c4aa00640 10 client.16524 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.412+0000 7f9c4aa00640 1 -- 172.21.15.86:0/1586637879 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556a1fd81540 con 0x556a1fe91830 2024-04-17T07:18:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.412+0000 7f9c4aa00640 10 client.16524 handle_command_reply: tid=0 2024-04-17T07:18:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.412+0000 7f9c6ed74740 1 client.16524 shutdown 2024-04-17T07:18:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.412+0000 7f9c6ed74740 1 -- 172.21.15.86:0/1586637879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe140a0 msgr2=0x556a1fe25550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.412+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe140a0 0x556a1fe25550 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x556a1fe15500 tx=0x7f9c580046c0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.413+0000 7f9c6ed74740 1 -- 172.21.15.86:0/1586637879 shutdown_connections 2024-04-17T07:18:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.413+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a1fe91830 0x556a1fe93c50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.413+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a1fe19c00 0x556a1fe23c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.413+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a1fe14a90 0x556a1fe25a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.413+0000 7f9c6ed74740 1 --2- 172.21.15.86:0/1586637879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a1fe140a0 0x556a1fe25550 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.413+0000 7f9c6ed74740 1 -- 172.21.15.86:0/1586637879 >> 172.21.15.86:0/1586637879 conn(0x556a1fe0f610 msgr2=0x556a1fe12870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.413+0000 7f9c4aa00640 0 client.16524 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:01.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.413+0000 7f9c6ed74740 1 -- 172.21.15.86:0/1586637879 shutdown_connections 2024-04-17T07:18:01.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.413+0000 7f9c6ed74740 1 -- 172.21.15.86:0/1586637879 wait complete. 2024-04-17T07:18:01.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.414+0000 7f9c6ed74740 20 client.16524 trim_cache size 0 max 16384 2024-04-17T07:18:01.414 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:18:01.414 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:18:01.415 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "3ad1778a-e745-47c0-b700-0865fb464243", 2024-04-17T07:18:01.415 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:18:01.415 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:18:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.414+0000 7f9c6aa00640 1 -- 172.21.15.86:0/2135732993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9c4c0a3bb0 msgr2=0x7f9c4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.414+0000 7f9c6aa00640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9c4c0a3bb0 0x7f9c4c0a6070 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x556a1fe14e90 tx=0x7f9c5800b710 comp rx=0 tx=0).stop 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.414+0000 7f9c6aa00640 1 -- 172.21.15.86:0/2135732993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c64103a40 msgr2=0x7f9c641a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.414+0000 7f9c6aa00640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c64103a40 0x7f9c641a1870 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00cce0 tx=0x7f9c5c00a620 comp rx=0 tx=0).stop 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.415+0000 7f9c6aa00640 1 -- 172.21.15.86:0/2135732993 shutdown_connections 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.415+0000 7f9c6aa00640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9c4c0a3bb0 0x7f9c4c0a6070 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.415+0000 7f9c6aa00640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c64103a40 0x7f9c641a1870 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.415+0000 7f9c6aa00640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c64103080 0x7f9c6419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.415+0000 7f9c6aa00640 1 --2- 172.21.15.86:0/2135732993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c64101ea0 0x7f9c6419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.415+0000 7f9c6aa00640 1 -- 172.21.15.86:0/2135732993 >> 172.21.15.86:0/2135732993 conn(0x7f9c640fd460 msgr2=0x7f9c64074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.415+0000 7f9c6aa00640 1 -- 172.21.15.86:0/2135732993 shutdown_connections 2024-04-17T07:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.415+0000 7f9c6aa00640 1 -- 172.21.15.86:0/2135732993 wait complete. 2024-04-17T07:18:01.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:18:01.434 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:18:01.506 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:01 smithi086 ceph-mon[31733]: pgmap v550: 129 pgs: 129 active+clean; 636 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.4 MiB/s wr, 18 op/s 2024-04-17T07:18:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2700000 of 3200000 tuples (84%) done (elapsed 111.19 s, remaining 20.59 s) 2024-04-17T07:18:01.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.661+0000 7f8aad600640 1 -- 172.21.15.86:0/4130125403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa81041c0 msgr2=0x7f8aa81045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.661+0000 7f8aad600640 1 --2- 172.21.15.86:0/4130125403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa81041c0 0x7f8aa81045e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009900 tx=0x7f8a9802bc30 comp rx=0 tx=0).stop 2024-04-17T07:18:01.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.662+0000 7f8aad600640 1 -- 172.21.15.86:0/4130125403 shutdown_connections 2024-04-17T07:18:01.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.662+0000 7f8aad600640 1 --2- 172.21.15.86:0/4130125403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa8105d70 0x7f8aa810c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.662+0000 7f8aad600640 1 --2- 172.21.15.86:0/4130125403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa81053b0 0x7f8aa8105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.662+0000 7f8aad600640 1 --2- 172.21.15.86:0/4130125403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa81041c0 0x7f8aa81045e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.662+0000 7f8aad600640 1 -- 172.21.15.86:0/4130125403 >> 172.21.15.86:0/4130125403 conn(0x7f8aa80ff750 msgr2=0x7f8aa8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.662+0000 7f8aad600640 1 -- 172.21.15.86:0/4130125403 shutdown_connections 2024-04-17T07:18:01.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.662+0000 7f8aad600640 1 -- 172.21.15.86:0/4130125403 wait complete. 2024-04-17T07:18:01.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.663+0000 7f8aad600640 1 Processor -- start 2024-04-17T07:18:01.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.664+0000 7f8aad600640 1 -- start start 2024-04-17T07:18:01.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.664+0000 7f8aad600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa81041c0 0x7f8aa819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.664+0000 7f8aad600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa81053b0 0x7f8aa819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.664+0000 7f8aad600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa8105d70 0x7f8aa81a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.664+0000 7f8aad600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa810fd50 con 0x7f8aa8105d70 2024-04-17T07:18:01.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.664+0000 7f8aad600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa810fbd0 con 0x7f8aa81041c0 2024-04-17T07:18:01.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.664+0000 7f8aad600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa810fed0 con 0x7f8aa81053b0 2024-04-17T07:18:01.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.665+0000 7f8aa6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa81041c0 0x7f8aa819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.665+0000 7f8aa6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa81041c0 0x7f8aa819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:55322/0 (socket says 172.21.15.86:55322) 2024-04-17T07:18:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.665+0000 7f8aa6a00640 1 -- 172.21.15.86:0/3758142127 learned_addr learned my addr 172.21.15.86:0/3758142127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.665+0000 7f8aa7400640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa8105d70 0x7f8aa81a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.665+0000 7f8aa6000640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa81053b0 0x7f8aa819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.666+0000 7f8aa6a00640 1 -- 172.21.15.86:0/3758142127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa81053b0 msgr2=0x7f8aa819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.666+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa81053b0 0x7f8aa819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.666+0000 7f8aa6a00640 1 -- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa8105d70 msgr2=0x7f8aa81a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.666+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa8105d70 0x7f8aa81a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.666+0000 7f8aa6a00640 1 -- 172.21.15.86:0/3758142127 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa81a2020 con 0x7f8aa81041c0 2024-04-17T07:18:01.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.666+0000 7f8aa7400640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa8105d70 0x7f8aa81a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:18:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.666+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa81041c0 0x7f8aa819a400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a98000c00 tx=0x7f8a980040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.666+0000 7f8a9f400640 1 -- 172.21.15.86:0/3758142127 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98043020 con 0x7f8aa81041c0 2024-04-17T07:18:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.667+0000 7f8a9f400640 1 -- 172.21.15.86:0/3758142127 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a980044c0 con 0x7f8aa81041c0 2024-04-17T07:18:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.667+0000 7f8aad600640 1 -- 172.21.15.86:0/3758142127 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa81a22b0 con 0x7f8aa81041c0 2024-04-17T07:18:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.667+0000 7f8a9f400640 1 -- 172.21.15.86:0/3758142127 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98003c60 con 0x7f8aa81041c0 2024-04-17T07:18:01.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.667+0000 7f8aad600640 1 -- 172.21.15.86:0/3758142127 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa81a2800 con 0x7f8aa81041c0 2024-04-17T07:18:01.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.670+0000 7f8ab0375740 1 Processor -- start 2024-04-17T07:18:01.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.671+0000 7f8ab0375740 1 -- start start 2024-04-17T07:18:01.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.671+0000 7f8a9f400640 1 -- 172.21.15.86:0/3758142127 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8a98046020 con 0x7f8aa81041c0 2024-04-17T07:18:01.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.671+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 0x5632a006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.671+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0074680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.671+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0077180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.671+0000 7f8ab0375740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56329ffd6120 con 0x5632a006fb80 2024-04-17T07:18:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.671+0000 7f8ab0375740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56329ff7c340 con 0x5632a0074cf0 2024-04-17T07:18:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.671+0000 7f8ab0375740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56329ffd20e0 con 0x5632a006f190 2024-04-17T07:18:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.671+0000 7f8aa6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.672+0000 7f8aa6000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.672+0000 7f8aa6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0074680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56590/0 (socket says 172.21.15.86:56590) 2024-04-17T07:18:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.672+0000 7f8aa6a00640 1 -- 172.21.15.86:0/3584337005 learned_addr learned my addr 172.21.15.86:0/3584337005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:01.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.672+0000 7f8aa7400640 1 --2- 172.21.15.86:0/3584337005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 0x5632a006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.672+0000 7f8aa6a00640 1 -- 172.21.15.86:0/3584337005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 msgr2=0x5632a006f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.672+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/3584337005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 0x5632a006f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.672+0000 7f8aa6a00640 1 -- 172.21.15.86:0/3584337005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 msgr2=0x5632a0077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.672+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/3584337005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.672+0000 7f8aa6a00640 1 -- 172.21.15.86:0/3584337005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329fdb27d0 con 0x5632a006fb80 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.672+0000 7f8aa6000640 1 --2- 172.21.15.86:0/3584337005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.673+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/3584337005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0074680 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8a980f8a90 tx=0x7f8a980f8b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb5b8df54249ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.673+0000 7f8a9f400640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a8c0a3bb0 0x7f8a8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.673+0000 7f8a93400640 1 -- 172.21.15.86:0/3584337005 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98101070 con 0x5632a006fb80 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.673+0000 7f8a93400640 1 -- 172.21.15.86:0/3584337005 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a980035f0 con 0x5632a006fb80 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.673+0000 7f8a93400640 1 -- 172.21.15.86:0/3584337005 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a980be3f0 con 0x5632a006fb80 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.673+0000 7f8a9f400640 1 -- 172.21.15.86:0/3758142127 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8a980f29b0 con 0x7f8aa81041c0 2024-04-17T07:18:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.673+0000 7f8aa7400640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a8c0a3bb0 0x7f8a8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.674+0000 7f8ab0375740 1 -- 172.21.15.86:0/3584337005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 msgr2=0x5632a0074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.674+0000 7f8aa7400640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a8c0a3bb0 0x7f8a8c0a6070 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5632a00703e0 tx=0x7f8aa0009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.674+0000 7f8ab0375740 1 --2- 172.21.15.86:0/3584337005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0074680 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8a980f8a90 tx=0x7f8a980f8b80 comp rx=0 tx=0).stop 2024-04-17T07:18:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.675+0000 7f8ab0375740 1 -- 172.21.15.86:0/3584337005 shutdown_connections 2024-04-17T07:18:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.675+0000 7f8ab0375740 1 --2- 172.21.15.86:0/3584337005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.675+0000 7f8ab0375740 1 --2- 172.21.15.86:0/3584337005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0074680 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.675+0000 7f8ab0375740 1 --2- 172.21.15.86:0/3584337005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 0x5632a006f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.675+0000 7f8ab0375740 1 -- 172.21.15.86:0/3584337005 >> 172.21.15.86:0/3584337005 conn(0x5632a006a700 msgr2=0x5632a006ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.675+0000 7f8ab0375740 1 -- 172.21.15.86:0/3584337005 shutdown_connections 2024-04-17T07:18:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.675+0000 7f8ab0375740 1 -- 172.21.15.86:0/3584337005 wait complete. 2024-04-17T07:18:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.675+0000 7f8ab0375740 1 -- start start 2024-04-17T07:18:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.676+0000 7f8ab0375740 1 Processor -- start 2024-04-17T07:18:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.676+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 0x5632a0081ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.676+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a00821e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.676+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0089220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.676+0000 7f8ab0375740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5632a0080270 con 0x5632a006fb80 2024-04-17T07:18:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.676+0000 7f8ab0375740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56329ff8f710 con 0x5632a0074cf0 2024-04-17T07:18:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.676+0000 7f8ab0375740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5632a00803f0 con 0x5632a006f190 2024-04-17T07:18:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.676+0000 7f8ab0375740 10 client.0 _ll_register_callbacks cb 0x56329ff8aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.677+0000 7f8aa7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0089220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.677+0000 7f8aa7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0089220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:55340/0 (socket says 172.21.15.86:55340) 2024-04-17T07:18:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.677+0000 7f8aa7400640 1 -- 172.21.15.86:0/492564405 learned_addr learned my addr 172.21.15.86:0/492564405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.677+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 0x5632a0081ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.677+0000 7f8aa6000640 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a00821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.677+0000 7f8aa7400640 1 -- 172.21.15.86:0/492564405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 msgr2=0x5632a0081ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.677+0000 7f8aa7400640 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 0x5632a0081ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.677+0000 7f8aa7400640 1 -- 172.21.15.86:0/492564405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 msgr2=0x5632a00821e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.677+0000 7f8aa7400640 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a00821e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.677+0000 7f8aa7400640 1 -- 172.21.15.86:0/492564405 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56329fce83f0 con 0x5632a0074cf0 2024-04-17T07:18:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.678+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 0x5632a0081ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.678+0000 7f8aa7400640 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0089220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8aa001e680 tx=0x7f8aa0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a0367e6432965a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.678+0000 7f8a87e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.678+0000 7f8a87e00640 1 -- 172.21.15.86:0/492564405 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa002b060 con 0x5632a0074cf0 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.678+0000 7f8ab0375740 1 -- 172.21.15.86:0/492564405 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56329ffe3b40 con 0x5632a0074cf0 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.678+0000 7f8a87e00640 1 -- 172.21.15.86:0/492564405 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8aa001c670 con 0x5632a0074cf0 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.679+0000 7f8a87e00640 1 -- 172.21.15.86:0/492564405 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8aa0032050 con 0x5632a0074cf0 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.679+0000 7f8ab0375740 10 client.35597 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.679+0000 7f8ab0375740 1 -- 172.21.15.86:0/492564405 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56329fca80f0 con 0x5632a0074cf0 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.679+0000 7f8a87e00640 1 -- 172.21.15.86:0/492564405 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56329ffe3b40 con 0x5632a0074cf0 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.680+0000 7f8a87e00640 1 -- 172.21.15.86:0/492564405 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8aa001a040 con 0x5632a0074cf0 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.680+0000 7f8ab0375740 10 client.35597 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.680+0000 7f8ab0375740 10 client.35597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.680+0000 7f8ab0375740 20 client.35597 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.680+0000 7f8ab0375740 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5632a008db10 0x5632a008ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.680+0000 7f8ab0375740 4 client.35597 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.680+0000 7f8ab0375740 1 -- 172.21.15.86:0/492564405 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56329ffdd9b0 con 0x5632a008db10 2024-04-17T07:18:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.680+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5632a008db10 0x5632a008ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.682+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5632a008db10 0x5632a008ff30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b13b977a39bffa40 server_cookie=caa38fed45f141da in_seq=0 out_seq=0 2024-04-17T07:18:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.682+0000 7f8a87e00640 10 client.35597 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8a87e00640 1 -- 172.21.15.86:0/492564405 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56329ffdd9b0 con 0x5632a008db10 2024-04-17T07:18:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8a87e00640 10 client.35597 handle_command_reply: tid=0 2024-04-17T07:18:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8ab0375740 1 client.35597 shutdown 2024-04-17T07:18:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8ab0375740 1 -- 172.21.15.86:0/492564405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 msgr2=0x5632a0089220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8ab0375740 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0089220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8aa001e680 tx=0x7f8aa0004a40 comp rx=0 tx=0).stop 2024-04-17T07:18:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8ab0375740 1 -- 172.21.15.86:0/492564405 shutdown_connections 2024-04-17T07:18:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8ab0375740 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5632a008db10 0x5632a008ff30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8a87e00640 0 client.35597 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8ab0375740 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a0074cf0 0x5632a0089220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8ab0375740 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a00821e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8ab0375740 1 --2- 172.21.15.86:0/492564405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a006f190 0x5632a0081ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.683+0000 7f8ab0375740 1 -- 172.21.15.86:0/492564405 >> 172.21.15.86:0/492564405 conn(0x5632a006a700 msgr2=0x5632a0069890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.684+0000 7f8ab0375740 1 -- 172.21.15.86:0/492564405 shutdown_connections 2024-04-17T07:18:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.684+0000 7f8ab0375740 1 -- 172.21.15.86:0/492564405 wait complete. 2024-04-17T07:18:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.684+0000 7f8ab0375740 20 client.35597 trim_cache size 0 max 16384 2024-04-17T07:18:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.705+0000 7f8ab0375740 1 Processor -- start 2024-04-17T07:18:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.705+0000 7f8ab0375740 1 -- start start 2024-04-17T07:18:01.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.705+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006f190 0x5632a0069000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.705+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a006fb80 0x5632a00891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a0074cf0 0x5632a0069540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8ab0375740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81a8970 con 0x5632a006f190 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8ab0375740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81aa4b0 con 0x5632a006fb80 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8ab0375740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81a9040 con 0x5632a0074cf0 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8aa6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006f190 0x5632a0069000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8aa6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006f190 0x5632a0069000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56610/0 (socket says 172.21.15.86:56610) 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8aa6a00640 1 -- 172.21.15.86:0/661232362 learned_addr learned my addr 172.21.15.86:0/661232362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8aa7400640 1 --2- 172.21.15.86:0/661232362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a0074cf0 0x5632a0069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8aa6000640 1 --2- 172.21.15.86:0/661232362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a006fb80 0x5632a00891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8aa6a00640 1 -- 172.21.15.86:0/661232362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a0074cf0 msgr2=0x5632a0069540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/661232362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a0074cf0 0x5632a0069540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8aa6a00640 1 -- 172.21.15.86:0/661232362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a006fb80 msgr2=0x5632a00891e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/661232362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a006fb80 0x5632a00891e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.706+0000 7f8aa6a00640 1 -- 172.21.15.86:0/661232362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632a0070410 con 0x5632a006f190 2024-04-17T07:18:01.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.707+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/661232362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006f190 0x5632a0069000 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8a98006010 tx=0x7f8a980f8b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ae94052e36fab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.707+0000 7f8a92a00640 1 -- 172.21.15.86:0/661232362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98101070 con 0x5632a006f190 2024-04-17T07:18:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.707+0000 7f8a92a00640 1 -- 172.21.15.86:0/661232362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a98005600 con 0x5632a006f190 2024-04-17T07:18:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.707+0000 7f8a92a00640 1 -- 172.21.15.86:0/661232362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9807d970 con 0x5632a006f190 2024-04-17T07:18:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.708+0000 7f8ab0375740 1 -- 172.21.15.86:0/661232362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006f190 msgr2=0x5632a0069000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.708+0000 7f8ab0375740 1 --2- 172.21.15.86:0/661232362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006f190 0x5632a0069000 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8a98006010 tx=0x7f8a980f8b80 comp rx=0 tx=0).stop 2024-04-17T07:18:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.708+0000 7f8ab0375740 1 -- 172.21.15.86:0/661232362 shutdown_connections 2024-04-17T07:18:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.708+0000 7f8ab0375740 1 --2- 172.21.15.86:0/661232362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a0074cf0 0x5632a0069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.708+0000 7f8ab0375740 1 --2- 172.21.15.86:0/661232362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a006fb80 0x5632a00891e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.708+0000 7f8ab0375740 1 --2- 172.21.15.86:0/661232362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006f190 0x5632a0069000 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.708+0000 7f8ab0375740 1 -- 172.21.15.86:0/661232362 >> 172.21.15.86:0/661232362 conn(0x5632a006a700 msgr2=0x5632a0071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.708+0000 7f8ab0375740 1 -- 172.21.15.86:0/661232362 shutdown_connections 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.708+0000 7f8ab0375740 1 -- 172.21.15.86:0/661232362 wait complete. 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.709+0000 7f8ab0375740 1 -- start start 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.709+0000 7f8ab0375740 1 Processor -- start 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.709+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a006f190 0x5632a0086e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.709+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0087390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.709+0000 7f8ab0375740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a0074cf0 0x5632a0088300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.709+0000 7f8ab0375740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81a9de0 con 0x5632a006fb80 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.709+0000 7f8ab0375740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81a9710 con 0x5632a006f190 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.709+0000 7f8ab0375740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81a6940 con 0x5632a0074cf0 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.709+0000 7f8ab0375740 10 client.0 _ll_register_callbacks cb 0x7f8aa0009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:01.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0087390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0087390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56626/0 (socket says 172.21.15.86:56626) 2024-04-17T07:18:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa6000640 1 -- 172.21.15.86:0/336652926 learned_addr learned my addr 172.21.15.86:0/336652926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a006f190 0x5632a0086e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa7400640 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a0074cf0 0x5632a0088300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa6000640 1 -- 172.21.15.86:0/336652926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a0074cf0 msgr2=0x5632a0088300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa6000640 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a0074cf0 0x5632a0088300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa6000640 1 -- 172.21.15.86:0/336652926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a006f190 msgr2=0x5632a0086e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa6000640 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a006f190 0x5632a0086e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa6000640 1 -- 172.21.15.86:0/336652926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632a00888f0 con 0x5632a006fb80 2024-04-17T07:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.710+0000 7f8aa6000640 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0087390 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8a94010800 tx=0x7f8a94010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138dd08b38b60089 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.711+0000 7f8a87e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.711+0000 7f8a87e00640 1 -- 172.21.15.86:0/336652926 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94022060 con 0x5632a006fb80 2024-04-17T07:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.711+0000 7f8a87e00640 1 -- 172.21.15.86:0/336652926 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8a94004460 con 0x5632a006fb80 2024-04-17T07:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.711+0000 7f8ab0375740 1 -- 172.21.15.86:0/336652926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56329ff59420 con 0x5632a006fb80 2024-04-17T07:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.711+0000 7f8a87e00640 1 -- 172.21.15.86:0/336652926 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a94029050 con 0x5632a006fb80 2024-04-17T07:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.711+0000 7f8ab0375740 10 client.16542 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.711+0000 7f8ab0375740 1 -- 172.21.15.86:0/336652926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632a00f45c0 con 0x5632a006fb80 2024-04-17T07:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.712+0000 7f8a87e00640 1 -- 172.21.15.86:0/336652926 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56329ff59420 con 0x5632a006fb80 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.712+0000 7f8a87e00640 1 -- 172.21.15.86:0/336652926 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8a94003c40 con 0x5632a006fb80 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.712+0000 7f8ab0375740 10 client.16542 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.712+0000 7f8ab0375740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.712+0000 7f8ab0375740 20 client.16542 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.712+0000 7f8ab0375740 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5632a00f5020 0x5632a00f7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.712+0000 7f8ab0375740 4 client.16542 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.712+0000 7f8ab0375740 1 -- 172.21.15.86:0/336652926 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56329ffa6a40 con 0x5632a00f5020 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.713+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5632a00f5020 0x5632a00f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8aa6a00640 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5632a00f5020 0x5632a00f7440 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=420dc55823fb71a server_cookie=a8770f8e9f6324af in_seq=0 out_seq=0 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8a87e00640 10 client.16542 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8a87e00640 1 -- 172.21.15.86:0/336652926 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56329ffa6a40 con 0x5632a00f5020 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8a87e00640 10 client.16542 handle_command_reply: tid=0 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8ab0375740 1 client.16542 shutdown 2024-04-17T07:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8ab0375740 1 -- 172.21.15.86:0/336652926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 msgr2=0x5632a0087390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8ab0375740 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0087390 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8a94010800 tx=0x7f8a94010cc0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8ab0375740 1 -- 172.21.15.86:0/336652926 shutdown_connections 2024-04-17T07:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8ab0375740 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5632a00f5020 0x5632a00f7440 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8ab0375740 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632a0074cf0 0x5632a0088300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8ab0375740 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632a006fb80 0x5632a0087390 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8ab0375740 1 --2- 172.21.15.86:0/336652926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632a006f190 0x5632a0086e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8a87e00640 0 client.16542 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.714+0000 7f8ab0375740 1 -- 172.21.15.86:0/336652926 >> 172.21.15.86:0/336652926 conn(0x5632a006a700 msgr2=0x5632a0069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.715+0000 7f8ab0375740 1 -- 172.21.15.86:0/336652926 shutdown_connections 2024-04-17T07:18:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.715+0000 7f8ab0375740 1 -- 172.21.15.86:0/336652926 wait complete. 2024-04-17T07:18:01.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.715+0000 7f8ab0375740 20 client.16542 trim_cache size 0 max 16384 2024-04-17T07:18:01.717 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:18:01.717 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:18:01.718 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:18:01.718 INFO:teuthology.orchestra.run.smithi086.stdout: "3ad1778a-e745-47c0-b700-0865fb464243": { 2024-04-17T07:18:01.718 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:18:01.718 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "3ad1778a-e745-47c0-b700-0865fb464243", 2024-04-17T07:18:01.718 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:18:01.718 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:18:01.718 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:18:01.718 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:18:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 -- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a8c0a3bb0 msgr2=0x7f8a8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a8c0a3bb0 0x7f8a8c0a6070 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5632a00703e0 tx=0x7f8aa0009600 comp rx=0 tx=0).stop 2024-04-17T07:18:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 -- 172.21.15.86:0/3758142127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa81041c0 msgr2=0x7f8aa819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa81041c0 0x7f8aa819a400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a98000c00 tx=0x7f8a980040c0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 -- 172.21.15.86:0/3758142127 shutdown_connections 2024-04-17T07:18:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8a8c0a3bb0 0x7f8a8c0a6070 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa8105d70 0x7f8aa81a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8aa81053b0 0x7f8aa819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 --2- 172.21.15.86:0/3758142127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa81041c0 0x7f8aa819a400 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 -- 172.21.15.86:0/3758142127 >> 172.21.15.86:0/3758142127 conn(0x7f8aa80ff750 msgr2=0x7f8aa8108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 -- 172.21.15.86:0/3758142127 shutdown_connections 2024-04-17T07:18:01.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:01.716+0000 7f8aad600640 1 -- 172.21.15.86:0/3758142127 wait complete. 2024-04-17T07:18:01.735 INFO:tasks.cephfs.filesystem:scrub status for tag:3ad1778a-e745-47c0-b700-0865fb464243 - {'path': '/', 'tag': '3ad1778a-e745-47c0-b700-0865fb464243', 'options': 'recursive,force'} 2024-04-17T07:18:01.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:01 smithi098 ceph-mon[35418]: pgmap v550: 129 pgs: 129 active+clean; 636 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.4 MiB/s wr, 18 op/s 2024-04-17T07:18:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:01 smithi103 ceph-mon[35495]: pgmap v550: 129 pgs: 129 active+clean; 636 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.4 MiB/s wr, 18 op/s 2024-04-17T07:18:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:02 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:18:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:02 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:02 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:18:02.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:02 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:02 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:18:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:02 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:02 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:18:02.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:02 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:02.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:02 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:18:02.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:02 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:02.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:02 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:18:02.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:02 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:03 smithi086 ceph-mon[31733]: pgmap v551: 129 pgs: 129 active+clean; 660 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-17T07:18:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:03 smithi098 ceph-mon[35418]: pgmap v551: 129 pgs: 129 active+clean; 660 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-17T07:18:03.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:03 smithi103 ceph-mon[35495]: pgmap v551: 129 pgs: 129 active+clean; 660 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 22 op/s 2024-04-17T07:18:04.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:04 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:18:04.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:04 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:18:04.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:04 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:18:05.624 INFO:teuthology.orchestra.run.smithi086.stderr:2800000 of 3200000 tuples (87%) done (elapsed 115.17 s, remaining 16.45 s) 2024-04-17T07:18:05.781 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:05 smithi103 ceph-mon[35495]: pgmap v552: 129 pgs: 129 active+clean; 668 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:18:05.782 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:05 smithi098 ceph-mon[35418]: pgmap v552: 129 pgs: 129 active+clean; 668 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:18:05.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:05 smithi086 ceph-mon[31733]: pgmap v552: 129 pgs: 129 active+clean; 668 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:18:06.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:06 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:18:06.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:06 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:18:06.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:06 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:18:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:18:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:18:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:18:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:07 smithi098 ceph-mon[35418]: pgmap v553: 129 pgs: 129 active+clean; 678 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:18:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:18:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:18:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:18:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:07 smithi103 ceph-mon[35495]: pgmap v553: 129 pgs: 129 active+clean; 678 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:18:07.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:18:07.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:18:07.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:18:07.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:07 smithi086 ceph-mon[31733]: pgmap v553: 129 pgs: 129 active+clean; 678 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:18:09.411 INFO:teuthology.orchestra.run.smithi086.stderr:2900000 of 3200000 tuples (90%) done (elapsed 118.96 s, remaining 12.31 s) 2024-04-17T07:18:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:09 smithi086 ceph-mon[31733]: pgmap v554: 129 pgs: 129 active+clean; 698 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-04-17T07:18:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:09 smithi098 ceph-mon[35418]: pgmap v554: 129 pgs: 129 active+clean; 698 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-04-17T07:18:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:09 smithi103 ceph-mon[35495]: pgmap v554: 129 pgs: 129 active+clean; 698 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-04-17T07:18:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:11 smithi086 ceph-mon[31733]: pgmap v555: 129 pgs: 129 active+clean; 702 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 18 op/s 2024-04-17T07:18:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:11 smithi098 ceph-mon[35418]: pgmap v555: 129 pgs: 129 active+clean; 702 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 18 op/s 2024-04-17T07:18:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:11 smithi103 ceph-mon[35495]: pgmap v555: 129 pgs: 129 active+clean; 702 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 18 op/s 2024-04-17T07:18:13.579 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:13 smithi103 ceph-mon[35495]: pgmap v556: 129 pgs: 129 active+clean; 712 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 23 op/s 2024-04-17T07:18:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:13 smithi086 ceph-mon[31733]: pgmap v556: 129 pgs: 129 active+clean; 712 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 23 op/s 2024-04-17T07:18:13.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:13 smithi098 ceph-mon[35418]: pgmap v556: 129 pgs: 129 active+clean; 712 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 23 op/s 2024-04-17T07:18:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:3000000 of 3200000 tuples (93%) done (elapsed 123.85 s, remaining 8.26 s) 2024-04-17T07:18:15.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:15 smithi086 ceph-mon[31733]: pgmap v557: 129 pgs: 129 active+clean; 716 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:18:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:15 smithi098 ceph-mon[35418]: pgmap v557: 129 pgs: 129 active+clean; 716 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:18:15.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:15 smithi103 ceph-mon[35495]: pgmap v557: 129 pgs: 129 active+clean; 716 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-04-17T07:18:17.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:17 smithi086 ceph-mon[31733]: pgmap v558: 129 pgs: 129 active+clean; 736 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 23 op/s 2024-04-17T07:18:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:17 smithi098 ceph-mon[35418]: pgmap v558: 129 pgs: 129 active+clean; 736 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 23 op/s 2024-04-17T07:18:17.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:17 smithi103 ceph-mon[35495]: pgmap v558: 129 pgs: 129 active+clean; 736 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 23 op/s 2024-04-17T07:18:18.476 INFO:teuthology.orchestra.run.smithi086.stderr:3100000 of 3200000 tuples (96%) done (elapsed 128.02 s, remaining 4.13 s) 2024-04-17T07:18:19.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:19 smithi086 ceph-mon[31733]: pgmap v559: 129 pgs: 129 active+clean; 752 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:18:19.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:19 smithi098 ceph-mon[35418]: pgmap v559: 129 pgs: 129 active+clean; 752 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:18:19.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:19 smithi103 ceph-mon[35495]: pgmap v559: 129 pgs: 129 active+clean; 752 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:18:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:21 smithi086 ceph-mon[31733]: pgmap v560: 129 pgs: 129 active+clean; 762 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.3 MiB/s wr, 19 op/s 2024-04-17T07:18:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:21 smithi098 ceph-mon[35418]: pgmap v560: 129 pgs: 129 active+clean; 762 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.3 MiB/s wr, 19 op/s 2024-04-17T07:18:21.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:21 smithi103 ceph-mon[35495]: pgmap v560: 129 pgs: 129 active+clean; 762 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.3 MiB/s wr, 19 op/s 2024-04-17T07:18:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:3200000 of 3200000 tuples (100%) done (elapsed 132.98 s, remaining 0.00 s) 2024-04-17T07:18:23.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:23 smithi086 ceph-mon[31733]: pgmap v561: 129 pgs: 129 active+clean; 770 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-04-17T07:18:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:23 smithi098 ceph-mon[35418]: pgmap v561: 129 pgs: 129 active+clean; 770 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-04-17T07:18:23.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:23 smithi103 ceph-mon[35495]: pgmap v561: 129 pgs: 129 active+clean; 770 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-04-17T07:18:23.787 INFO:teuthology.orchestra.run.smithi086.stderr:vacuuming... 2024-04-17T07:18:25.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:25 smithi086 ceph-mon[31733]: pgmap v562: 129 pgs: 129 active+clean; 770 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 MiB/s wr, 18 op/s 2024-04-17T07:18:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:25 smithi098 ceph-mon[35418]: pgmap v562: 129 pgs: 129 active+clean; 770 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 MiB/s wr, 18 op/s 2024-04-17T07:18:25.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:25 smithi103 ceph-mon[35495]: pgmap v562: 129 pgs: 129 active+clean; 770 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.4 MiB/s wr, 18 op/s 2024-04-17T07:18:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:27 smithi086 ceph-mon[31733]: pgmap v563: 129 pgs: 129 active+clean; 782 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 25 op/s 2024-04-17T07:18:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:27 smithi098 ceph-mon[35418]: pgmap v563: 129 pgs: 129 active+clean; 782 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 25 op/s 2024-04-17T07:18:27.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:27 smithi103 ceph-mon[35495]: pgmap v563: 129 pgs: 129 active+clean; 782 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 25 op/s 2024-04-17T07:18:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:29 smithi086 ceph-mon[31733]: pgmap v564: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 22 op/s 2024-04-17T07:18:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:29 smithi098 ceph-mon[35418]: pgmap v564: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 22 op/s 2024-04-17T07:18:29.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:29 smithi103 ceph-mon[35495]: pgmap v564: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 22 op/s 2024-04-17T07:18:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:31 smithi086 ceph-mon[31733]: pgmap v565: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.5 MiB/s wr, 20 op/s 2024-04-17T07:18:31.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:18:31.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:18:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:31 smithi098 ceph-mon[35418]: pgmap v565: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.5 MiB/s wr, 20 op/s 2024-04-17T07:18:31.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:31 smithi103 ceph-mon[35495]: pgmap v565: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.5 MiB/s wr, 20 op/s 2024-04-17T07:18:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.965+0000 7f93cb600640 1 -- 172.21.15.86:0/3741932441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c4102050 msgr2=0x7f93c41024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.965+0000 7f93cb600640 1 --2- 172.21.15.86:0/3741932441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c4102050 0x7f93c41024d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009980 tx=0x7f93b002de70 comp rx=0 tx=0).stop 2024-04-17T07:18:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.965+0000 7f93cb600640 1 -- 172.21.15.86:0/3741932441 shutdown_connections 2024-04-17T07:18:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.965+0000 7f93cb600640 1 --2- 172.21.15.86:0/3741932441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93c4102a10 0x7f93c4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.965+0000 7f93cb600640 1 --2- 172.21.15.86:0/3741932441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c4102050 0x7f93c41024d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.965+0000 7f93cb600640 1 --2- 172.21.15.86:0/3741932441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93c4100e70 0x7f93c4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.965+0000 7f93cb600640 1 -- 172.21.15.86:0/3741932441 >> 172.21.15.86:0/3741932441 conn(0x7f93c40fc450 msgr2=0x7f93c40fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.966+0000 7f93cb600640 1 -- 172.21.15.86:0/3741932441 shutdown_connections 2024-04-17T07:18:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.966+0000 7f93cb600640 1 -- 172.21.15.86:0/3741932441 wait complete. 2024-04-17T07:18:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.967+0000 7f93cb600640 1 Processor -- start 2024-04-17T07:18:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.967+0000 7f93cb600640 1 -- start start 2024-04-17T07:18:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.967+0000 7f93cb600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93c4100e70 0x7f93c419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.967+0000 7f93cb600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93c4102050 0x7f93c419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.967+0000 7f93cb600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c4102a10 0x7f93c41a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.967+0000 7f93cb600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93c410b960 con 0x7f93c4102a10 2024-04-17T07:18:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.967+0000 7f93cb600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93c410b7e0 con 0x7f93c4100e70 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.967+0000 7f93cb600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93c410bae0 con 0x7f93c4102050 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c4102a10 0x7f93c41a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c4102a10 0x7f93c41a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40300/0 (socket says 172.21.15.86:40300) 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c9600640 1 -- 172.21.15.86:0/1436687156 learned_addr learned my addr 172.21.15.86:0/1436687156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c8c00640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93c4100e70 0x7f93c419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c3e00640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93c4102050 0x7f93c419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c9600640 1 -- 172.21.15.86:0/1436687156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93c4102050 msgr2=0x7f93c419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c9600640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93c4102050 0x7f93c419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c9600640 1 -- 172.21.15.86:0/1436687156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93c4100e70 msgr2=0x7f93c419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c9600640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93c4100e70 0x7f93c419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c9600640 1 -- 172.21.15.86:0/1436687156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c41a20b0 con 0x7f93c4102a10 2024-04-17T07:18:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.968+0000 7f93c8c00640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93c4100e70 0x7f93c419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.969+0000 7f93c9600640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c4102a10 0x7f93c41a1a40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00eaa0 tx=0x7f93bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.969+0000 7f93c1600640 1 -- 172.21.15.86:0/1436687156 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc00cdb0 con 0x7f93c4102a10 2024-04-17T07:18:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.969+0000 7f93c1600640 1 -- 172.21.15.86:0/1436687156 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93bc005020 con 0x7f93c4102a10 2024-04-17T07:18:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.969+0000 7f93cb600640 1 -- 172.21.15.86:0/1436687156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c41a23a0 con 0x7f93c4102a10 2024-04-17T07:18:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.969+0000 7f93c1600640 1 -- 172.21.15.86:0/1436687156 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0116d0 con 0x7f93c4102a10 2024-04-17T07:18:31.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.969+0000 7f93cb600640 1 -- 172.21.15.86:0/1436687156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c41a28c0 con 0x7f93c4102a10 2024-04-17T07:18:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.973+0000 7f93c1600640 1 -- 172.21.15.86:0/1436687156 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f93bc0051d0 con 0x7f93c4102a10 2024-04-17T07:18:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.973+0000 7f93ce27e740 1 Processor -- start 2024-04-17T07:18:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.973+0000 7f93ce27e740 1 -- start start 2024-04-17T07:18:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.973+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64d95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.973+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64de680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.973+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64decf0 0x5580c64e1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.973+0000 7f93ce27e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5580c6440120 con 0x5580c64d9190 2024-04-17T07:18:31.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.973+0000 7f93ce27e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5580c63e6340 con 0x5580c64decf0 2024-04-17T07:18:31.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.973+0000 7f93ce27e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5580c643c0e0 con 0x5580c64d9b80 2024-04-17T07:18:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.974+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64d95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.974+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64d95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40304/0 (socket says 172.21.15.86:40304) 2024-04-17T07:18:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.974+0000 7f93c3e00640 1 -- 172.21.15.86:0/1540578279 learned_addr learned my addr 172.21.15.86:0/1540578279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.974+0000 7f93c8c00640 1 --2- 172.21.15.86:0/1540578279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.974+0000 7f93c3e00640 1 -- 172.21.15.86:0/1540578279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 msgr2=0x5580c64de680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:31.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.974+0000 7f93c3e00640 1 --2- 172.21.15.86:0/1540578279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64de680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.974+0000 7f93c3e00640 1 -- 172.21.15.86:0/1540578279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64decf0 msgr2=0x5580c64e1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.974+0000 7f93c3e00640 1 --2- 172.21.15.86:0/1540578279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64decf0 0x5580c64e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.974+0000 7f93c3e00640 1 -- 172.21.15.86:0/1540578279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c621c7d0 con 0x5580c64d9190 2024-04-17T07:18:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.975+0000 7f93c3e00640 1 --2- 172.21.15.86:0/1540578279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64d95b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f93b0004a40 tx=0x7f93b0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6c704d12d28dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.975+0000 7f93b5c00640 1 -- 172.21.15.86:0/1540578279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004030 con 0x5580c64d9190 2024-04-17T07:18:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.975+0000 7f93b5c00640 1 -- 172.21.15.86:0/1540578279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b002e810 con 0x5580c64d9190 2024-04-17T07:18:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.975+0000 7f93b5c00640 1 -- 172.21.15.86:0/1540578279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0003ae0 con 0x5580c64d9190 2024-04-17T07:18:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93c1600640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93ac0a3bb0 0x7f93ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93c1600640 1 -- 172.21.15.86:0/1436687156 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93bc0d0a30 con 0x7f93c4102a10 2024-04-17T07:18:31.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93c8c00640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93ac0a3bb0 0x7f93ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:31.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93ce27e740 1 -- 172.21.15.86:0/1540578279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 msgr2=0x5580c64d95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:31.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93ce27e740 1 --2- 172.21.15.86:0/1540578279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64d95b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f93b0004a40 tx=0x7f93b0005d50 comp rx=0 tx=0).stop 2024-04-17T07:18:31.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93ce27e740 1 -- 172.21.15.86:0/1540578279 shutdown_connections 2024-04-17T07:18:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93ce27e740 1 --2- 172.21.15.86:0/1540578279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64decf0 0x5580c64e1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93ce27e740 1 --2- 172.21.15.86:0/1540578279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64de680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93ce27e740 1 --2- 172.21.15.86:0/1540578279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64d95b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93ce27e740 1 -- 172.21.15.86:0/1540578279 >> 172.21.15.86:0/1540578279 conn(0x5580c64d4700 msgr2=0x5580c64d4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93ce27e740 1 -- 172.21.15.86:0/1540578279 shutdown_connections 2024-04-17T07:18:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.976+0000 7f93ce27e740 1 -- 172.21.15.86:0/1540578279 wait complete. 2024-04-17T07:18:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.977+0000 7f93ce27e740 1 -- start start 2024-04-17T07:18:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.977+0000 7f93c8c00640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93ac0a3bb0 0x7f93ac0a6070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5580c64d5670 tx=0x7f93b800b710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.977+0000 7f93ce27e740 1 Processor -- start 2024-04-17T07:18:31.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64e3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64decf0 0x5580c64eac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93ce27e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5580c64e1c40 con 0x5580c64d9190 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93ce27e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5580c63f9710 con 0x5580c64decf0 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93ce27e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5580c64e1dc0 con 0x5580c64d9b80 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93ce27e740 10 client.0 _ll_register_callbacks cb 0x5580c63f4af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93c8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93c8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64e3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40308/0 (socket says 172.21.15.86:40308) 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93c8c00640 1 -- 172.21.15.86:0/1200027284 learned_addr learned my addr 172.21.15.86:0/1200027284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93c3e00640 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.978+0000 7f93c9600640 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64decf0 0x5580c64eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93c8c00640 1 -- 172.21.15.86:0/1200027284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 msgr2=0x5580c64e3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:31.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93c8c00640 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64e3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93c8c00640 1 -- 172.21.15.86:0/1200027284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64decf0 msgr2=0x5580c64eac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93c8c00640 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64decf0 0x5580c64eac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93c8c00640 1 -- 172.21.15.86:0/1200027284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c61523f0 con 0x5580c64d9190 2024-04-17T07:18:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93c3e00640 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64e3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93c8c00640 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64e3760 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f93b8012700 tx=0x7f93b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55a2d587149227a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93aa000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93aa000640 1 -- 172.21.15.86:0/1200027284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b800fae0 con 0x5580c64d9190 2024-04-17T07:18:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93aa000640 1 -- 172.21.15.86:0/1200027284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93b800fc80 con 0x5580c64d9190 2024-04-17T07:18:31.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.979+0000 7f93ce27e740 1 -- 172.21.15.86:0/1200027284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b002e810 con 0x5580c64d9190 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.980+0000 7f93aa000640 1 -- 172.21.15.86:0/1200027284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93b800fe80 con 0x5580c64d9190 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.980+0000 7f93ce27e740 10 client.16554 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.980+0000 7f93ce27e740 1 -- 172.21.15.86:0/1200027284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5580c61120f0 con 0x5580c64d9190 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.980+0000 7f93aa000640 1 -- 172.21.15.86:0/1200027284 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b002e810 con 0x5580c64d9190 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.980+0000 7f93aa000640 1 -- 172.21.15.86:0/1200027284 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f93b80275c0 con 0x5580c64d9190 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.980+0000 7f93ce27e740 10 client.16554 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.980+0000 7f93ce27e740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.980+0000 7f93ce27e740 20 client.16554 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.980+0000 7f93ce27e740 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5580c64ef5c0 0x5580c64f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.981+0000 7f93ce27e740 4 client.16554 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:31.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.981+0000 7f93ce27e740 1 -- 172.21.15.86:0/1200027284 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580c644db40 con 0x5580c64ef5c0 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.981+0000 7f93c3e00640 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5580c64ef5c0 0x5580c64f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.982+0000 7f93c3e00640 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5580c64ef5c0 0x5580c64f1a60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0beb9300fe8d08b server_cookie=5c672a90809ba4a6 in_seq=0 out_seq=0 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.982+0000 7f93aa000640 10 client.16554 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93aa000640 1 -- 172.21.15.86:0/1200027284 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580c644db40 con 0x5580c64ef5c0 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93aa000640 10 client.16554 handle_command_reply: tid=0 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93ce27e740 1 client.16554 shutdown 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93ce27e740 1 -- 172.21.15.86:0/1200027284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 msgr2=0x5580c64e3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93ce27e740 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64e3760 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f93b8012700 tx=0x7f93b80046c0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93ce27e740 1 -- 172.21.15.86:0/1200027284 shutdown_connections 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93ce27e740 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5580c64ef5c0 0x5580c64f1a60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93ce27e740 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64decf0 0x5580c64eac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93ce27e740 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64e3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93ce27e740 1 --2- 172.21.15.86:0/1200027284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9190 0x5580c64e3760 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:31.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93ce27e740 1 -- 172.21.15.86:0/1200027284 >> 172.21.15.86:0/1200027284 conn(0x5580c64d4700 msgr2=0x5580c64d3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:31.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.984+0000 7f93aa000640 0 client.16554 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:31.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.985+0000 7f93ce27e740 1 -- 172.21.15.86:0/1200027284 shutdown_connections 2024-04-17T07:18:31.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.985+0000 7f93ce27e740 1 -- 172.21.15.86:0/1200027284 wait complete. 2024-04-17T07:18:31.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:31.985+0000 7f93ce27e740 20 client.16554 trim_cache size 0 max 16384 2024-04-17T07:18:32.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93ce27e740 1 Processor -- start 2024-04-17T07:18:32.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93ce27e740 1 -- start start 2024-04-17T07:18:32.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 0x5580c64e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64eac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64decf0 0x5580c64db110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93ce27e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41acd70 con 0x5580c64decf0 2024-04-17T07:18:32.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93ce27e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41ae8b0 con 0x5580c64d9190 2024-04-17T07:18:32.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93ce27e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41ad440 con 0x5580c64d9b80 2024-04-17T07:18:32.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93c9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64decf0 0x5580c64db110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93c9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64decf0 0x5580c64db110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40310/0 (socket says 172.21.15.86:40310) 2024-04-17T07:18:32.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.004+0000 7f93c9600640 1 -- 172.21.15.86:0/2224833237 learned_addr learned my addr 172.21.15.86:0/2224833237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93c8c00640 1 --2- 172.21.15.86:0/2224833237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 0x5580c64e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93c3e00640 1 --2- 172.21.15.86:0/2224833237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64eac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93c9600640 1 -- 172.21.15.86:0/2224833237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 msgr2=0x5580c64eac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93c9600640 1 --2- 172.21.15.86:0/2224833237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64eac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93c9600640 1 -- 172.21.15.86:0/2224833237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 msgr2=0x5580c64e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93c9600640 1 --2- 172.21.15.86:0/2224833237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 0x5580c64e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93c9600640 1 -- 172.21.15.86:0/2224833237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c64ef3a0 con 0x5580c64decf0 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93c9600640 1 --2- 172.21.15.86:0/2224833237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64decf0 0x5580c64db110 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0ce150 tx=0x7f93bc09c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe874edc507b7875 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93b5200640 1 -- 172.21.15.86:0/2224833237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc01f040 con 0x5580c64decf0 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93b5200640 1 -- 172.21.15.86:0/2224833237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93bc0a55a0 con 0x5580c64decf0 2024-04-17T07:18:32.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.005+0000 7f93b5200640 1 -- 172.21.15.86:0/2224833237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc087310 con 0x5580c64decf0 2024-04-17T07:18:32.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.006+0000 7f93ce27e740 1 -- 172.21.15.86:0/2224833237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64decf0 msgr2=0x5580c64db110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.006+0000 7f93ce27e740 1 --2- 172.21.15.86:0/2224833237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64decf0 0x5580c64db110 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0ce150 tx=0x7f93bc09c930 comp rx=0 tx=0).stop 2024-04-17T07:18:32.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.006+0000 7f93ce27e740 1 -- 172.21.15.86:0/2224833237 shutdown_connections 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.006+0000 7f93ce27e740 1 --2- 172.21.15.86:0/2224833237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64decf0 0x5580c64db110 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.006+0000 7f93ce27e740 1 --2- 172.21.15.86:0/2224833237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64d9b80 0x5580c64eac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.006+0000 7f93ce27e740 1 --2- 172.21.15.86:0/2224833237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 0x5580c64e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.006+0000 7f93ce27e740 1 -- 172.21.15.86:0/2224833237 >> 172.21.15.86:0/2224833237 conn(0x5580c64d4700 msgr2=0x5580c64d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.007+0000 7f93ce27e740 1 -- 172.21.15.86:0/2224833237 shutdown_connections 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.007+0000 7f93ce27e740 1 -- 172.21.15.86:0/2224833237 wait complete. 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.007+0000 7f93ce27e740 1 -- start start 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.007+0000 7f93ce27e740 1 Processor -- start 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 0x5580c64e8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9b80 0x5580c64e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93ce27e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64decf0 0x5580c64e9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93ce27e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41ae1e0 con 0x5580c64d9b80 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93ce27e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41adb10 con 0x5580c64d9190 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93ce27e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41aad40 con 0x5580c64decf0 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93ce27e740 10 client.0 _ll_register_callbacks cb 0x7f93b800b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9b80 0x5580c64e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9b80 0x5580c64e8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40324/0 (socket says 172.21.15.86:40324) 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c3e00640 1 -- 172.21.15.86:0/2925716857 learned_addr learned my addr 172.21.15.86:0/2925716857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c9600640 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64decf0 0x5580c64e9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c8c00640 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 0x5580c64e8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c3e00640 1 -- 172.21.15.86:0/2925716857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64decf0 msgr2=0x5580c64e9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c3e00640 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64decf0 0x5580c64e9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c3e00640 1 -- 172.21.15.86:0/2925716857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 msgr2=0x5580c64e8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c3e00640 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 0x5580c64e8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c3e00640 1 -- 172.21.15.86:0/2925716857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c64d5170 con 0x5580c64d9b80 2024-04-17T07:18:32.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c9600640 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64decf0 0x5580c64e9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.008+0000 7f93c8c00640 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 0x5580c64e8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.009+0000 7f93c3e00640 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9b80 0x5580c64e8a70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f93b0004a40 tx=0x7f93b002de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612f0261a5d0409d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.009+0000 7f93aa000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.009+0000 7f93aa000640 1 -- 172.21.15.86:0/2925716857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b002e810 con 0x5580c64d9b80 2024-04-17T07:18:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.009+0000 7f93aa000640 1 -- 172.21.15.86:0/2925716857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93b0004030 con 0x5580c64d9b80 2024-04-17T07:18:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.009+0000 7f93ce27e740 1 -- 172.21.15.86:0/2925716857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93bc0a55a0 con 0x5580c64d9b80 2024-04-17T07:18:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.009+0000 7f93aa000640 1 -- 172.21.15.86:0/2925716857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93b0064050 con 0x5580c64d9b80 2024-04-17T07:18:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.009+0000 7f93ce27e740 10 client.16560 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.009+0000 7f93ce27e740 1 -- 172.21.15.86:0/2925716857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5580c6555c10 con 0x5580c64d9b80 2024-04-17T07:18:32.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.010+0000 7f93aa000640 1 -- 172.21.15.86:0/2925716857 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93bc0a55a0 con 0x5580c64d9b80 2024-04-17T07:18:32.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.010+0000 7f93aa000640 1 -- 172.21.15.86:0/2925716857 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f93b00035f0 con 0x5580c64d9b80 2024-04-17T07:18:32.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.010+0000 7f93ce27e740 10 client.16560 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:32.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.010+0000 7f93ce27e740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:32.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.010+0000 7f93ce27e740 20 client.16560 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:32.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.010+0000 7f93ce27e740 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5580c6556240 0x5580c6558630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.010+0000 7f93ce27e740 4 client.16560 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:18:32.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.010+0000 7f93ce27e740 1 -- 172.21.15.86:0/2925716857 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5580c64479b0 con 0x5580c6556240 2024-04-17T07:18:32.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.011+0000 7f93c8c00640 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5580c6556240 0x5580c6558630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93c8c00640 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5580c6556240 0x5580c6558630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6a5b57d97ccd36 server_cookie=c898fc5cc55fa67d in_seq=0 out_seq=0 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93aa000640 10 client.16560 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93aa000640 1 -- 172.21.15.86:0/2925716857 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5580c64479b0 con 0x5580c6556240 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93aa000640 10 client.16560 handle_command_reply: tid=0 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93ce27e740 1 client.16560 shutdown 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93ce27e740 1 -- 172.21.15.86:0/2925716857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9b80 msgr2=0x5580c64e8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93ce27e740 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9b80 0x5580c64e8a70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f93b0004a40 tx=0x7f93b002de70 comp rx=0 tx=0).stop 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93ce27e740 1 -- 172.21.15.86:0/2925716857 shutdown_connections 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93ce27e740 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5580c6556240 0x5580c6558630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93ce27e740 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580c64decf0 0x5580c64e9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93ce27e740 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5580c64d9b80 0x5580c64e8a70 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93ce27e740 1 --2- 172.21.15.86:0/2925716857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5580c64d9190 0x5580c64e8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93aa000640 0 client.16560 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.012+0000 7f93ce27e740 1 -- 172.21.15.86:0/2925716857 >> 172.21.15.86:0/2925716857 conn(0x5580c64d4700 msgr2=0x5580c64d3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.013+0000 7f93ce27e740 1 -- 172.21.15.86:0/2925716857 shutdown_connections 2024-04-17T07:18:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.013+0000 7f93ce27e740 1 -- 172.21.15.86:0/2925716857 wait complete. 2024-04-17T07:18:32.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.013+0000 7f93ce27e740 20 client.16560 trim_cache size 0 max 16384 2024-04-17T07:18:32.016 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:18:32.016 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:18:32.016 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:18:32.016 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:18:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.013+0000 7f93cb600640 1 -- 172.21.15.86:0/1436687156 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93ac0a3bb0 msgr2=0x7f93ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.013+0000 7f93cb600640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93ac0a3bb0 0x7f93ac0a6070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5580c64d5670 tx=0x7f93b800b710 comp rx=0 tx=0).stop 2024-04-17T07:18:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.013+0000 7f93cb600640 1 -- 172.21.15.86:0/1436687156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c4102a10 msgr2=0x7f93c41a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.013+0000 7f93cb600640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c4102a10 0x7f93c41a1a40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00eaa0 tx=0x7f93bc00ef60 comp rx=0 tx=0).stop 2024-04-17T07:18:32.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.014+0000 7f93cb600640 1 -- 172.21.15.86:0/1436687156 shutdown_connections 2024-04-17T07:18:32.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.014+0000 7f93cb600640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93ac0a3bb0 0x7f93ac0a6070 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.014+0000 7f93cb600640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93c4102a10 0x7f93c41a1a40 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.014+0000 7f93cb600640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93c4102050 0x7f93c419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.014+0000 7f93cb600640 1 --2- 172.21.15.86:0/1436687156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93c4100e70 0x7f93c419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.014+0000 7f93cb600640 1 -- 172.21.15.86:0/1436687156 >> 172.21.15.86:0/1436687156 conn(0x7f93c40fc450 msgr2=0x7f93c4191020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.014+0000 7f93cb600640 1 -- 172.21.15.86:0/1436687156 shutdown_connections 2024-04-17T07:18:32.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.014+0000 7f93cb600640 1 -- 172.21.15.86:0/1436687156 wait complete. 2024-04-17T07:18:32.033 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:18:32.033 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:18:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.255+0000 7f28b5200640 1 -- 172.21.15.86:0/2849517528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b01051f0 msgr2=0x7f28b0105670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.255+0000 7f28b5200640 1 --2- 172.21.15.86:0/2849517528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b01051f0 0x7f28b0105670 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f289c009ca0 tx=0x7f289c02c100 comp rx=0 tx=0).stop 2024-04-17T07:18:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.255+0000 7f28b5200640 1 -- 172.21.15.86:0/2849517528 shutdown_connections 2024-04-17T07:18:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.255+0000 7f28b5200640 1 --2- 172.21.15.86:0/2849517528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b0105bb0 0x7f28b010c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.255+0000 7f28b5200640 1 --2- 172.21.15.86:0/2849517528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b01051f0 0x7f28b0105670 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.256+0000 7f28b5200640 1 --2- 172.21.15.86:0/2849517528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b0104000 0x7f28b0104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.256+0000 7f28b5200640 1 -- 172.21.15.86:0/2849517528 >> 172.21.15.86:0/2849517528 conn(0x7f28b00ff570 msgr2=0x7f28b01019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.256+0000 7f28b5200640 1 -- 172.21.15.86:0/2849517528 shutdown_connections 2024-04-17T07:18:32.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.256+0000 7f28b5200640 1 -- 172.21.15.86:0/2849517528 wait complete. 2024-04-17T07:18:32.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28b5200640 1 Processor -- start 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28b5200640 1 -- start start 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28b5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b0104000 0x7f28b019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28b5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b01051f0 0x7f28b019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28b5200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b0105bb0 0x7f28b01a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28b5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28b010fb40 con 0x7f28b0104000 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28b5200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f28b010f9c0 con 0x7f28b0105bb0 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28b5200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28b010fcc0 con 0x7f28b01051f0 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b0104000 0x7f28b019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b0104000 0x7f28b019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40346/0 (socket says 172.21.15.86:40346) 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.257+0000 7f28af400640 1 -- 172.21.15.86:0/716452027 learned_addr learned my addr 172.21.15.86:0/716452027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.258+0000 7f28aea00640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b01051f0 0x7f28b019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.258+0000 7f28afe00640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b0105bb0 0x7f28b01a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.258+0000 7f28af400640 1 -- 172.21.15.86:0/716452027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b01051f0 msgr2=0x7f28b019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.258+0000 7f28af400640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b01051f0 0x7f28b019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.258+0000 7f28af400640 1 -- 172.21.15.86:0/716452027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b0105bb0 msgr2=0x7f28b01a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.258+0000 7f28af400640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b0105bb0 0x7f28b01a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.258+0000 7f28af400640 1 -- 172.21.15.86:0/716452027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b01a1dc0 con 0x7f28b0104000 2024-04-17T07:18:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.258+0000 7f28af400640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b0104000 0x7f28b019a1a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d770 tx=0x7f28a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.258+0000 7f28a7e00640 1 -- 172.21.15.86:0/716452027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0017070 con 0x7f28b0104000 2024-04-17T07:18:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.258+0000 7f28a7e00640 1 -- 172.21.15.86:0/716452027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28a0009c30 con 0x7f28b0104000 2024-04-17T07:18:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.259+0000 7f28a7e00640 1 -- 172.21.15.86:0/716452027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0003d00 con 0x7f28b0104000 2024-04-17T07:18:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.259+0000 7f28b5200640 1 -- 172.21.15.86:0/716452027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b01a20b0 con 0x7f28b0104000 2024-04-17T07:18:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.259+0000 7f28b5200640 1 -- 172.21.15.86:0/716452027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b01a25d0 con 0x7f28b0104000 2024-04-17T07:18:32.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.260+0000 7f28b5200640 1 -- 172.21.15.86:0/716452027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b010b370 con 0x7f28b0104000 2024-04-17T07:18:32.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.263+0000 7f28a7e00640 1 -- 172.21.15.86:0/716452027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f28a0003ea0 con 0x7f28b0104000 2024-04-17T07:18:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.264+0000 7f28a7e00640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28940a3bb0 0x7f28940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.265+0000 7f28a7e00640 1 -- 172.21.15.86:0/716452027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f28a00d0540 con 0x7f28b0104000 2024-04-17T07:18:32.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.265+0000 7f28aea00640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28940a3bb0 0x7f28940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.266+0000 7f28aea00640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28940a3bb0 0x7f28940a6070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f289c000f80 tx=0x7f289c009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:32.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.271+0000 7f28a7e00640 1 -- 172.21.15.86:0/716452027 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f28a009cba0 con 0x7f28b0104000 2024-04-17T07:18:32.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.573+0000 7f28b5200640 1 -- 172.21.15.86:0/716452027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28b002d5d0 con 0x7f28b0104000 2024-04-17T07:18:32.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.575+0000 7f28a7e00640 1 -- 172.21.15.86:0/716452027 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f28a0009de0 con 0x7f28b0104000 2024-04-17T07:18:32.575 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:18:32.575 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:18:32.576 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:18:32.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 -- 172.21.15.86:0/716452027 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28940a3bb0 msgr2=0x7f28940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28940a3bb0 0x7f28940a6070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f289c000f80 tx=0x7f289c009360 comp rx=0 tx=0).stop 2024-04-17T07:18:32.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 -- 172.21.15.86:0/716452027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b0104000 msgr2=0x7f28b019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b0104000 0x7f28b019a1a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d770 tx=0x7f28a000dc30 comp rx=0 tx=0).stop 2024-04-17T07:18:32.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 -- 172.21.15.86:0/716452027 shutdown_connections 2024-04-17T07:18:32.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28940a3bb0 0x7f28940a6070 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28b0105bb0 0x7f28b01a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b01051f0 0x7f28b019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 --2- 172.21.15.86:0/716452027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b0104000 0x7f28b019a1a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 -- 172.21.15.86:0/716452027 >> 172.21.15.86:0/716452027 conn(0x7f28b00ff570 msgr2=0x7f28b0108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.578+0000 7f28b5200640 1 -- 172.21.15.86:0/716452027 shutdown_connections 2024-04-17T07:18:32.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.579+0000 7f28b5200640 1 -- 172.21.15.86:0/716452027 wait complete. 2024-04-17T07:18:32.600 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:18:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.839+0000 7fc173200640 1 -- 172.21.15.86:0/311962322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc174101f70 msgr2=0x7fc1741023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.839+0000 7fc173200640 1 --2- 172.21.15.86:0/311962322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc174101f70 0x7fc1741023f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc160009900 tx=0x7fc16002bc30 comp rx=0 tx=0).stop 2024-04-17T07:18:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.839+0000 7fc173200640 1 -- 172.21.15.86:0/311962322 shutdown_connections 2024-04-17T07:18:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.839+0000 7fc173200640 1 --2- 172.21.15.86:0/311962322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc174102930 0x7fc1741091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.839+0000 7fc173200640 1 --2- 172.21.15.86:0/311962322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc174101f70 0x7fc1741023f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.839+0000 7fc173200640 1 --2- 172.21.15.86:0/311962322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc174100d80 0x7fc1741011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.839+0000 7fc173200640 1 -- 172.21.15.86:0/311962322 >> 172.21.15.86:0/311962322 conn(0x7fc1740fc2d0 msgr2=0x7fc1740fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.839+0000 7fc173200640 1 -- 172.21.15.86:0/311962322 shutdown_connections 2024-04-17T07:18:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.839+0000 7fc173200640 1 -- 172.21.15.86:0/311962322 wait complete. 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.840+0000 7fc173200640 1 Processor -- start 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.840+0000 7fc173200640 1 -- start start 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.840+0000 7fc173200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc174100d80 0x7fc174198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc173200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc174101f70 0x7fc174198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc173200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc174102930 0x7fc17419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc173200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc17410c890 con 0x7fc174100d80 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc173200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc17410c710 con 0x7fc174102930 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc173200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc17410ca10 con 0x7fc174101f70 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc171e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc174100d80 0x7fc174198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc171e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc174100d80 0x7fc174198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40356/0 (socket says 172.21.15.86:40356) 2024-04-17T07:18:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc171e00640 1 -- 172.21.15.86:0/154437105 learned_addr learned my addr 172.21.15.86:0/154437105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc171400640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc174101f70 0x7fc174198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc172800640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc174102930 0x7fc17419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc171e00640 1 -- 172.21.15.86:0/154437105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc174101f70 msgr2=0x7fc174198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc171e00640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc174101f70 0x7fc174198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc171e00640 1 -- 172.21.15.86:0/154437105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc174102930 msgr2=0x7fc17419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc171e00640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc174102930 0x7fc17419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc171e00640 1 -- 172.21.15.86:0/154437105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17419fc50 con 0x7fc174100d80 2024-04-17T07:18:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.841+0000 7fc172800640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc174102930 0x7fc17419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.842+0000 7fc171e00640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc174100d80 0x7fc174198030 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc16400c970 tx=0x7fc16400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.842+0000 7fc16aa00640 1 -- 172.21.15.86:0/154437105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164007c10 con 0x7fc174100d80 2024-04-17T07:18:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.842+0000 7fc16aa00640 1 -- 172.21.15.86:0/154437105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc164007db0 con 0x7fc174100d80 2024-04-17T07:18:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.842+0000 7fc16aa00640 1 -- 172.21.15.86:0/154437105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16400f870 con 0x7fc174100d80 2024-04-17T07:18:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.842+0000 7fc173200640 1 -- 172.21.15.86:0/154437105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17419ff40 con 0x7fc174100d80 2024-04-17T07:18:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.842+0000 7fc173200640 1 -- 172.21.15.86:0/154437105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1741a0350 con 0x7fc174100d80 2024-04-17T07:18:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.846+0000 7fc17bb68740 1 Processor -- start 2024-04-17T07:18:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.846+0000 7fc17bb68740 1 -- start start 2024-04-17T07:18:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.846+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127160 0x55767b127580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.846+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127b50 0x55767b12c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.846+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b12f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.846+0000 7fc17bb68740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55767b08d020 con 0x55767b12ccc0 2024-04-17T07:18:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.846+0000 7fc17bb68740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55767b033460 con 0x55767b127160 2024-04-17T07:18:32.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.846+0000 7fc17bb68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55767b088fe0 con 0x55767b127b50 2024-04-17T07:18:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc171400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b12f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc171400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b12f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40368/0 (socket says 172.21.15.86:40368) 2024-04-17T07:18:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc171400640 1 -- 172.21.15.86:0/1850141998 learned_addr learned my addr 172.21.15.86:0/1850141998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc16aa00640 1 -- 172.21.15.86:0/154437105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc16400fa10 con 0x7fc174100d80 2024-04-17T07:18:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc172800640 1 --2- 172.21.15.86:0/1850141998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127160 0x55767b127580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc171400640 1 -- 172.21.15.86:0/1850141998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127b50 msgr2=0x55767b12c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc171400640 1 --2- 172.21.15.86:0/1850141998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127b50 0x55767b12c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc171400640 1 -- 172.21.15.86:0/1850141998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127160 msgr2=0x55767b127580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc171400640 1 --2- 172.21.15.86:0/1850141998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127160 0x55767b127580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc171400640 1 -- 172.21.15.86:0/1850141998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767b03b040 con 0x55767b12ccc0 2024-04-17T07:18:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.847+0000 7fc172800640 1 --2- 172.21.15.86:0/1850141998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127160 0x55767b127580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.848+0000 7fc171400640 1 --2- 172.21.15.86:0/1850141998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b12f150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc16002c170 tx=0x7fc160005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3bd802e8d7b122 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.848+0000 7fc15f400640 1 -- 172.21.15.86:0/1850141998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160044020 con 0x55767b12ccc0 2024-04-17T07:18:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.848+0000 7fc15f400640 1 -- 172.21.15.86:0/1850141998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc16002cc60 con 0x55767b12ccc0 2024-04-17T07:18:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.848+0000 7fc15f400640 1 -- 172.21.15.86:0/1850141998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160004900 con 0x55767b12ccc0 2024-04-17T07:18:32.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.848+0000 7fc16aa00640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc1580a3c30 0x7fc1580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc16aa00640 1 -- 172.21.15.86:0/154437105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc1640d0b90 con 0x7fc174100d80 2024-04-17T07:18:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc172800640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc1580a3c30 0x7fc1580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc17bb68740 1 -- 172.21.15.86:0/1850141998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 msgr2=0x55767b12f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1850141998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b12f150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc16002c170 tx=0x7fc160005e80 comp rx=0 tx=0).stop 2024-04-17T07:18:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc17bb68740 1 -- 172.21.15.86:0/1850141998 shutdown_connections 2024-04-17T07:18:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1850141998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b12f150 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1850141998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127b50 0x55767b12c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1850141998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127160 0x55767b127580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc17bb68740 1 -- 172.21.15.86:0/1850141998 >> 172.21.15.86:0/1850141998 conn(0x55767b1226d0 msgr2=0x55767b122ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc17bb68740 1 -- 172.21.15.86:0/1850141998 shutdown_connections 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.849+0000 7fc17bb68740 1 -- 172.21.15.86:0/1850141998 wait complete. 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.850+0000 7fc172800640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc1580a3c30 0x7fc1580a60f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55767b1283b0 tx=0x7fc16c0084d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.850+0000 7fc17bb68740 1 -- start start 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.850+0000 7fc17bb68740 1 Processor -- start 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127160 0x55767b1316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127b50 0x55767b131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 0x55767b138c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc17bb68740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55767b12fc10 con 0x55767b127160 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc17bb68740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55767b046830 con 0x55767b12ccc0 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc17bb68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55767b12fd90 con 0x55767b127b50 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc17bb68740 10 client.0 _ll_register_callbacks cb 0x55767b014fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc171e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127160 0x55767b1316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc171e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127160 0x55767b1316a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40376/0 (socket says 172.21.15.86:40376) 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc171e00640 1 -- 172.21.15.86:0/512253350 learned_addr learned my addr 172.21.15.86:0/512253350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc171400640 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127b50 0x55767b131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc172800640 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 0x55767b138c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc171e00640 1 -- 172.21.15.86:0/512253350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127b50 msgr2=0x55767b131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc171e00640 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127b50 0x55767b131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc171e00640 1 -- 172.21.15.86:0/512253350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 msgr2=0x55767b138c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc171e00640 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 0x55767b138c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.851+0000 7fc171e00640 1 -- 172.21.15.86:0/512253350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767b04a920 con 0x55767b127160 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.852+0000 7fc172800640 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 0x55767b138c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.852+0000 7fc171e00640 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127160 0x55767b1316a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc1640994f0 tx=0x7fc16409c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5ab377520e64c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.852+0000 7fc153e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.852+0000 7fc153e00640 1 -- 172.21.15.86:0/512253350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1640bd0e0 con 0x55767b127160 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.852+0000 7fc153e00640 1 -- 172.21.15.86:0/512253350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc1640a5270 con 0x55767b127160 2024-04-17T07:18:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.852+0000 7fc17bb68740 1 -- 172.21.15.86:0/512253350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc16002cc60 con 0x55767b127160 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.852+0000 7fc153e00640 1 -- 172.21.15.86:0/512253350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc16409cb20 con 0x55767b127160 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.852+0000 7fc17bb68740 10 client.16578 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.853+0000 7fc17bb68740 1 -- 172.21.15.86:0/512253350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55767b0187f0 con 0x55767b127160 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.853+0000 7fc153e00640 1 -- 172.21.15.86:0/512253350 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc16002cc60 con 0x55767b127160 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.853+0000 7fc153e00640 1 -- 172.21.15.86:0/512253350 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc1640a31f0 con 0x55767b127160 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.853+0000 7fc17bb68740 10 client.16578 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.853+0000 7fc17bb68740 10 client.16578 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.853+0000 7fc17bb68740 20 client.16578 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.853+0000 7fc17bb68740 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55767b13d110 0x55767b13f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.853+0000 7fc17bb68740 4 client.16578 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.853+0000 7fc17bb68740 1 -- 172.21.15.86:0/512253350 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55767b09aa40 con 0x55767b13d110 2024-04-17T07:18:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.854+0000 7fc171400640 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55767b13d110 0x55767b13f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.855+0000 7fc171400640 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55767b13d110 0x55767b13f580 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c194805ef2a0cdde server_cookie=b5dcf689a40a67fb in_seq=0 out_seq=0 2024-04-17T07:18:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.855+0000 7fc153e00640 10 client.16578 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:18:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc153e00640 1 -- 172.21.15.86:0/512253350 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55767b09aa40 con 0x55767b13d110 2024-04-17T07:18:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc153e00640 10 client.16578 handle_command_reply: tid=0 2024-04-17T07:18:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 client.16578 shutdown 2024-04-17T07:18:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 -- 172.21.15.86:0/512253350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127160 msgr2=0x55767b1316a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127160 0x55767b1316a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc1640994f0 tx=0x7fc16409c540 comp rx=0 tx=0).stop 2024-04-17T07:18:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 -- 172.21.15.86:0/512253350 shutdown_connections 2024-04-17T07:18:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55767b13d110 0x55767b13f580 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 0x55767b138c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127b50 0x55767b131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 --2- 172.21.15.86:0/512253350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127160 0x55767b1316a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc153e00640 0 client.16578 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:18:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 -- 172.21.15.86:0/512253350 >> 172.21.15.86:0/512253350 conn(0x55767b1226d0 msgr2=0x55767b121930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 -- 172.21.15.86:0/512253350 shutdown_connections 2024-04-17T07:18:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 1 -- 172.21.15.86:0/512253350 wait complete. 2024-04-17T07:18:32.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.857+0000 7fc17bb68740 20 client.16578 trim_cache size 0 max 16384 2024-04-17T07:18:32.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.874+0000 7fc17bb68740 1 Processor -- start 2024-04-17T07:18:32.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.874+0000 7fc17bb68740 1 -- start start 2024-04-17T07:18:32.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.874+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127160 0x55767b1312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.874+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127b50 0x55767b138bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.874+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 0x55767b1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.874+0000 7fc17bb68740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741aa760 con 0x55767b127b50 2024-04-17T07:18:32.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.874+0000 7fc17bb68740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741ac2a0 con 0x55767b12ccc0 2024-04-17T07:18:32.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.874+0000 7fc17bb68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741aae30 con 0x55767b127160 2024-04-17T07:18:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc171400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127b50 0x55767b138bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc171400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127b50 0x55767b138bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40392/0 (socket says 172.21.15.86:40392) 2024-04-17T07:18:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc171400640 1 -- 172.21.15.86:0/1753699158 learned_addr learned my addr 172.21.15.86:0/1753699158 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc171e00640 1 --2- 172.21.15.86:0/1753699158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127160 0x55767b1312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc171400640 1 -- 172.21.15.86:0/1753699158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127160 msgr2=0x55767b1312b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc172800640 1 --2- 172.21.15.86:0/1753699158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 0x55767b1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc171400640 1 --2- 172.21.15.86:0/1753699158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127160 0x55767b1312b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc171400640 1 -- 172.21.15.86:0/1753699158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 msgr2=0x55767b1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc171400640 1 --2- 172.21.15.86:0/1753699158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 0x55767b1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc171400640 1 -- 172.21.15.86:0/1753699158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767b13f8f0 con 0x55767b127b50 2024-04-17T07:18:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.875+0000 7fc172800640 1 --2- 172.21.15.86:0/1753699158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 0x55767b1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.876+0000 7fc171400640 1 --2- 172.21.15.86:0/1753699158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127b50 0x55767b138bb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55767b123640 tx=0x7fc16002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8010e2a328dd3b06 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.876+0000 7fc15ea00640 1 -- 172.21.15.86:0/1753699158 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160044040 con 0x55767b127b50 2024-04-17T07:18:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.876+0000 7fc15ea00640 1 -- 172.21.15.86:0/1753699158 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc16004f070 con 0x55767b127b50 2024-04-17T07:18:32.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.876+0000 7fc15ea00640 1 -- 172.21.15.86:0/1753699158 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160003e20 con 0x55767b127b50 2024-04-17T07:18:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 -- 172.21.15.86:0/1753699158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127b50 msgr2=0x55767b138bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1753699158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127b50 0x55767b138bb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55767b123640 tx=0x7fc16002bc30 comp rx=0 tx=0).stop 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 -- 172.21.15.86:0/1753699158 shutdown_connections 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1753699158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b12ccc0 0x55767b1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1753699158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b127b50 0x55767b138bb0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1753699158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127160 0x55767b1312b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 -- 172.21.15.86:0/1753699158 >> 172.21.15.86:0/1753699158 conn(0x55767b1226d0 msgr2=0x55767b1290b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 -- 172.21.15.86:0/1753699158 shutdown_connections 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 -- 172.21.15.86:0/1753699158 wait complete. 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 -- start start 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.877+0000 7fc17bb68740 1 Processor -- start 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127160 0x55767b1359c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127b50 0x55767b135f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc17bb68740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc17bb68740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741abbd0 con 0x55767b12ccc0 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc17bb68740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741ab500 con 0x55767b127b50 2024-04-17T07:18:32.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc17bb68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741a8730 con 0x55767b127160 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc17bb68740 10 client.0 _ll_register_callbacks cb 0x7fc16409c660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc172800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc172800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b1a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40408/0 (socket says 172.21.15.86:40408) 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc172800640 1 -- 172.21.15.86:0/1054155259 learned_addr learned my addr 172.21.15.86:0/1054155259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc171e00640 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127160 0x55767b1359c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.878+0000 7fc171400640 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127b50 0x55767b135f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.879+0000 7fc172800640 1 -- 172.21.15.86:0/1054155259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127160 msgr2=0x55767b1359c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.879+0000 7fc172800640 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127160 0x55767b1359c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.879+0000 7fc172800640 1 -- 172.21.15.86:0/1054155259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127b50 msgr2=0x55767b135f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.879+0000 7fc172800640 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127b50 0x55767b135f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.879+0000 7fc172800640 1 -- 172.21.15.86:0/1054155259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767ad9f3f0 con 0x55767b12ccc0 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.879+0000 7fc172800640 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b1a16b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc16c010e40 tx=0x7fc16c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1b8954f53f9680 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.879+0000 7fc153e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.879+0000 7fc153e00640 1 -- 172.21.15.86:0/1054155259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c026440 con 0x55767b12ccc0 2024-04-17T07:18:32.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.879+0000 7fc153e00640 1 -- 172.21.15.86:0/1054155259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc16c01b480 con 0x55767b12ccc0 2024-04-17T07:18:32.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.879+0000 7fc17bb68740 1 -- 172.21.15.86:0/1054155259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc16004f070 con 0x55767b12ccc0 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc153e00640 1 -- 172.21.15.86:0/1054155259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc16c01b680 con 0x55767b12ccc0 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc17bb68740 10 client.16584 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc17bb68740 1 -- 172.21.15.86:0/1054155259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55767ae697d0 con 0x55767b12ccc0 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc153e00640 1 -- 172.21.15.86:0/1054155259 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc16004f070 con 0x55767b12ccc0 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc153e00640 1 -- 172.21.15.86:0/1054155259 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc16c024070 con 0x55767b12ccc0 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc17bb68740 10 client.16584 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc17bb68740 10 client.16584 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc17bb68740 20 client.16584 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55767b1a3470 0x55767b1a5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc17bb68740 4 client.16584 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:18:32.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.880+0000 7fc17bb68740 1 -- 172.21.15.86:0/1054155259 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55767b0948b0 con 0x55767b1a3470 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.881+0000 7fc171e00640 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55767b1a3470 0x55767b1a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.882+0000 7fc171e00640 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55767b1a3470 0x55767b1a5890 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7fea07eb3bb59807 server_cookie=76854056f272dff7 in_seq=0 out_seq=0 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.882+0000 7fc153e00640 10 client.16584 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc153e00640 1 -- 172.21.15.86:0/1054155259 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55767b0948b0 con 0x55767b1a3470 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc153e00640 10 client.16584 handle_command_reply: tid=0 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 client.16584 shutdown 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 -- 172.21.15.86:0/1054155259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 msgr2=0x55767b1a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b1a16b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc16c010e40 tx=0x7fc16c004a40 comp rx=0 tx=0).stop 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 -- 172.21.15.86:0/1054155259 shutdown_connections 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55767b1a3470 0x55767b1a5890 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55767b12ccc0 0x55767b1a16b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55767b127b50 0x55767b135f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 --2- 172.21.15.86:0/1054155259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55767b127160 0x55767b1359c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 -- 172.21.15.86:0/1054155259 >> 172.21.15.86:0/1054155259 conn(0x55767b1226d0 msgr2=0x55767b131220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc153e00640 0 client.16584 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:18:32.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 -- 172.21.15.86:0/1054155259 shutdown_connections 2024-04-17T07:18:32.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 1 -- 172.21.15.86:0/1054155259 wait complete. 2024-04-17T07:18:32.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.883+0000 7fc17bb68740 20 client.16584 trim_cache size 0 max 16384 2024-04-17T07:18:32.886 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 -- 172.21.15.86:0/154437105 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc1580a3c30 msgr2=0x7fc1580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc1580a3c30 0x7fc1580a60f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55767b1283b0 tx=0x7fc16c0084d0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 -- 172.21.15.86:0/154437105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc174100d80 msgr2=0x7fc174198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc174100d80 0x7fc174198030 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc16400c970 tx=0x7fc16400ce30 comp rx=0 tx=0).stop 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 -- 172.21.15.86:0/154437105 shutdown_connections 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc1580a3c30 0x7fc1580a60f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc174102930 0x7fc17419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc174101f70 0x7fc174198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 --2- 172.21.15.86:0/154437105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc174100d80 0x7fc174198030 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 -- 172.21.15.86:0/154437105 >> 172.21.15.86:0/154437105 conn(0x7fc1740fc2d0 msgr2=0x7fc1740fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 -- 172.21.15.86:0/154437105 shutdown_connections 2024-04-17T07:18:32.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:32.884+0000 7fc173200640 1 -- 172.21.15.86:0/154437105 wait complete. 2024-04-17T07:18:32.900 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:32 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/716452027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:18:32.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:18:33.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:32 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/716452027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:18:33.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:32 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/716452027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:18:33.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.108+0000 7f1de7e00640 1 -- 172.21.15.86:0/4117121246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1de8103070 msgr2=0x7f1de81054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.108+0000 7f1de7e00640 1 --2- 172.21.15.86:0/4117121246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1de8103070 0x7f1de81054e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009900 tx=0x7f1dd402bc30 comp rx=0 tx=0).stop 2024-04-17T07:18:33.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.108+0000 7f1de7e00640 1 -- 172.21.15.86:0/4117121246 shutdown_connections 2024-04-17T07:18:33.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.108+0000 7f1de7e00640 1 --2- 172.21.15.86:0/4117121246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de8105a20 0x7f1de8105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.108+0000 7f1de7e00640 1 --2- 172.21.15.86:0/4117121246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1de8103070 0x7f1de81054e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.108+0000 7f1de7e00640 1 --2- 172.21.15.86:0/4117121246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de8100700 0x7f1de8102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.108+0000 7f1de7e00640 1 -- 172.21.15.86:0/4117121246 >> 172.21.15.86:0/4117121246 conn(0x7f1de80fc320 msgr2=0x7f1de80fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.109+0000 7f1de7e00640 1 -- 172.21.15.86:0/4117121246 shutdown_connections 2024-04-17T07:18:33.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.109+0000 7f1de7e00640 1 -- 172.21.15.86:0/4117121246 wait complete. 2024-04-17T07:18:33.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de7e00640 1 Processor -- start 2024-04-17T07:18:33.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de7e00640 1 -- start start 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de8100700 0x7f1de819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1de8103070 0x7f1de819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de8105a20 0x7f1de81a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1de8114030 con 0x7f1de8103070 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de7e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1de8113eb0 con 0x7f1de8100700 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de7e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81141b0 con 0x7f1de8105a20 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1de8103070 0x7f1de819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1de8103070 0x7f1de819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40438/0 (socket says 172.21.15.86:40438) 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.110+0000 7f1de6000640 1 -- 172.21.15.86:0/3243047128 learned_addr learned my addr 172.21.15.86:0/3243047128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.111+0000 7f1de6a00640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de8100700 0x7f1de819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.111+0000 7f1de7400640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de8105a20 0x7f1de81a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.111+0000 7f1de6000640 1 -- 172.21.15.86:0/3243047128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de8105a20 msgr2=0x7f1de81a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.111+0000 7f1de6000640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de8105a20 0x7f1de81a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.111+0000 7f1de6000640 1 -- 172.21.15.86:0/3243047128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de8100700 msgr2=0x7f1de819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.111+0000 7f1de6000640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de8100700 0x7f1de819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.111+0000 7f1de6000640 1 -- 172.21.15.86:0/3243047128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de81a62a0 con 0x7f1de8103070 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.111+0000 7f1de6000640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1de8103070 0x7f1de819ebc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4000c00 tx=0x7f1dd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.112+0000 7f1ddf400640 1 -- 172.21.15.86:0/3243047128 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4043020 con 0x7f1de8103070 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.112+0000 7f1ddf400640 1 -- 172.21.15.86:0/3243047128 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dd4004aa0 con 0x7f1de8103070 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.112+0000 7f1de7e00640 1 -- 172.21.15.86:0/3243047128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de81a6530 con 0x7f1de8103070 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.112+0000 7f1ddf400640 1 -- 172.21.15.86:0/3243047128 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4003d00 con 0x7f1de8103070 2024-04-17T07:18:33.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.112+0000 7f1de7e00640 1 -- 172.21.15.86:0/3243047128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de81a6a80 con 0x7f1de8103070 2024-04-17T07:18:33.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.116+0000 7f1df09f8740 1 Processor -- start 2024-04-17T07:18:33.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.116+0000 7f1df09f8740 1 -- start start 2024-04-17T07:18:33.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.116+0000 7f1ddf400640 1 -- 172.21.15.86:0/3243047128 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1dd4046020 con 0x7f1de8103070 2024-04-17T07:18:33.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.116+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5160 0x55b6ccdd5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.116+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccdda650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.116+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 0x55b6ccddd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.116+0000 7f1df09f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6ccd3b020 con 0x55b6ccdd5b50 2024-04-17T07:18:33.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.116+0000 7f1df09f8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b6ccce1460 con 0x55b6ccddacc0 2024-04-17T07:18:33.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.116+0000 7f1df09f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b6ccd36fe0 con 0x55b6ccdd5160 2024-04-17T07:18:33.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.117+0000 7f1de6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccdda650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.117+0000 7f1de6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccdda650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40444/0 (socket says 172.21.15.86:40444) 2024-04-17T07:18:33.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.117+0000 7f1de6a00640 1 -- 172.21.15.86:0/2561246937 learned_addr learned my addr 172.21.15.86:0/2561246937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.117+0000 7f1de7400640 1 --2- 172.21.15.86:0/2561246937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5160 0x55b6ccdd5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.117+0000 7f1de6000640 1 --2- 172.21.15.86:0/2561246937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 0x55b6ccddd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.117+0000 7f1de6a00640 1 -- 172.21.15.86:0/2561246937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5160 msgr2=0x55b6ccdd5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.117+0000 7f1de6a00640 1 --2- 172.21.15.86:0/2561246937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5160 0x55b6ccdd5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.117+0000 7f1de6a00640 1 -- 172.21.15.86:0/2561246937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 msgr2=0x55b6ccddd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.117+0000 7f1de6a00640 1 --2- 172.21.15.86:0/2561246937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 0x55b6ccddd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.117+0000 7f1de6a00640 1 -- 172.21.15.86:0/2561246937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ccce9040 con 0x55b6ccdd5b50 2024-04-17T07:18:33.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.118+0000 7f1de6a00640 1 --2- 172.21.15.86:0/2561246937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccdda650 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009c80 tx=0x7f1dd80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34249cffcf22f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.118+0000 7f1dd3e00640 1 -- 172.21.15.86:0/2561246937 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8012450 con 0x55b6ccdd5b50 2024-04-17T07:18:33.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.118+0000 7f1dd3e00640 1 -- 172.21.15.86:0/2561246937 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dd8012aa0 con 0x55b6ccdd5b50 2024-04-17T07:18:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.118+0000 7f1dd3e00640 1 -- 172.21.15.86:0/2561246937 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800fb70 con 0x55b6ccdd5b50 2024-04-17T07:18:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.119+0000 7f1ddf400640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dcc0a3bb0 0x7f1dcc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.119+0000 7f1ddf400640 1 -- 172.21.15.86:0/3243047128 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1dd40f2c10 con 0x7f1de8103070 2024-04-17T07:18:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.119+0000 7f1de7400640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dcc0a3bb0 0x7f1dcc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.119+0000 7f1df09f8740 1 -- 172.21.15.86:0/2561246937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 msgr2=0x55b6ccdda650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.119+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2561246937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccdda650 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009c80 tx=0x7f1dd80117b0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.120+0000 7f1df09f8740 1 -- 172.21.15.86:0/2561246937 shutdown_connections 2024-04-17T07:18:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.120+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2561246937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 0x55b6ccddd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.120+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2561246937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccdda650 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.120+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2561246937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5160 0x55b6ccdd5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.120+0000 7f1df09f8740 1 -- 172.21.15.86:0/2561246937 >> 172.21.15.86:0/2561246937 conn(0x55b6ccdd06d0 msgr2=0x55b6ccdd0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.120+0000 7f1de7400640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dcc0a3bb0 0x7f1dcc0a6070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55b6ccdd63b0 tx=0x7f1de0009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.120+0000 7f1df09f8740 1 -- 172.21.15.86:0/2561246937 shutdown_connections 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.120+0000 7f1df09f8740 1 -- 172.21.15.86:0/2561246937 wait complete. 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.120+0000 7f1df09f8740 1 -- start start 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.121+0000 7f1df09f8740 1 Processor -- start 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.121+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 0x55b6ccddf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.121+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 0x55b6ccddfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.121+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.121+0000 7f1df09f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6ccdddc70 con 0x55b6ccddacc0 2024-04-17T07:18:33.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.121+0000 7f1df09f8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b6cccf4830 con 0x55b6ccdd5160 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.121+0000 7f1df09f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b6ccddddf0 con 0x55b6ccdd5b50 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.121+0000 7f1df09f8740 10 client.0 _ll_register_callbacks cb 0x55b6cccc2fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.121+0000 7f1de7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40452/0 (socket says 172.21.15.86:40452) 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de7400640 1 -- 172.21.15.86:0/204935679 learned_addr learned my addr 172.21.15.86:0/204935679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de6a00640 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 0x55b6ccddf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de6000640 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 0x55b6ccddfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de7400640 1 -- 172.21.15.86:0/204935679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 msgr2=0x55b6ccddfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de7400640 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 0x55b6ccddfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de7400640 1 -- 172.21.15.86:0/204935679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 msgr2=0x55b6ccddf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de7400640 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 0x55b6ccddf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de7400640 1 -- 172.21.15.86:0/204935679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6cccf8920 con 0x55b6ccddacc0 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de6000640 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 0x55b6ccddfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1de7400640 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6c30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009880 tx=0x7f1de0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbacf6c9c282e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.122+0000 7f1dd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.123+0000 7f1dd0c00640 1 -- 172.21.15.86:0/204935679 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de000fd30 con 0x55b6ccddacc0 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.123+0000 7f1dd0c00640 1 -- 172.21.15.86:0/204935679 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1de001f420 con 0x55b6ccddacc0 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.123+0000 7f1df09f8740 1 -- 172.21.15.86:0/204935679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd8012aa0 con 0x55b6ccddacc0 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.123+0000 7f1dd0c00640 1 -- 172.21.15.86:0/204935679 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1de0030050 con 0x55b6ccddacc0 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.123+0000 7f1df09f8740 10 client.16596 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.123+0000 7f1df09f8740 1 -- 172.21.15.86:0/204935679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6cccc67f0 con 0x55b6ccddacc0 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.123+0000 7f1dd0c00640 1 -- 172.21.15.86:0/204935679 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dd8012aa0 con 0x55b6ccddacc0 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.124+0000 7f1dd0c00640 1 -- 172.21.15.86:0/204935679 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1de001d4d0 con 0x55b6ccddacc0 2024-04-17T07:18:33.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.124+0000 7f1df09f8740 10 client.16596 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.124+0000 7f1df09f8740 10 client.16596 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:18:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.124+0000 7f1df09f8740 20 client.16596 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.124+0000 7f1df09f8740 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55b6ccdeb2d0 0x55b6ccded700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.124+0000 7f1df09f8740 4 client.16596 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.124+0000 7f1df09f8740 1 -- 172.21.15.86:0/204935679 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ccd48a40 con 0x55b6ccdeb2d0 2024-04-17T07:18:33.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.125+0000 7f1de6a00640 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55b6ccdeb2d0 0x55b6ccded700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.126+0000 7f1de6a00640 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55b6ccdeb2d0 0x55b6ccded700 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73e8e0be2c53f00f server_cookie=e21b0010acda3055 in_seq=0 out_seq=0 2024-04-17T07:18:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.126+0000 7f1dd0c00640 10 client.16596 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:18:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.127+0000 7f1dd0c00640 1 -- 172.21.15.86:0/204935679 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6ccd48a40 con 0x55b6ccdeb2d0 2024-04-17T07:18:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.127+0000 7f1dd0c00640 10 client.16596 handle_command_reply: tid=0 2024-04-17T07:18:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 client.16596 shutdown 2024-04-17T07:18:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 -- 172.21.15.86:0/204935679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccddacc0 msgr2=0x55b6ccde6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6c30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009880 tx=0x7f1de0004a40 comp rx=0 tx=0).stop 2024-04-17T07:18:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 -- 172.21.15.86:0/204935679 shutdown_connections 2024-04-17T07:18:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55b6ccdeb2d0 0x55b6ccded700 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6c30 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 0x55b6ccddfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 --2- 172.21.15.86:0/204935679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 0x55b6ccddf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 -- 172.21.15.86:0/204935679 >> 172.21.15.86:0/204935679 conn(0x55b6ccdd06d0 msgr2=0x55b6ccdcf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1dd0c00640 0 client.16596 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:18:33.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 -- 172.21.15.86:0/204935679 shutdown_connections 2024-04-17T07:18:33.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 1 -- 172.21.15.86:0/204935679 wait complete. 2024-04-17T07:18:33.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.128+0000 7f1df09f8740 20 client.16596 trim_cache size 0 max 16384 2024-04-17T07:18:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.145+0000 7f1df09f8740 1 Processor -- start 2024-04-17T07:18:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.145+0000 7f1df09f8740 1 -- start start 2024-04-17T07:18:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.146+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5160 0x55b6ccdd7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.146+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 0x55b6ccdd7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.146+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.146+0000 7f1df09f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81b0e60 con 0x55b6ccdd5160 2024-04-17T07:18:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.146+0000 7f1df09f8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81b29a0 con 0x55b6ccddacc0 2024-04-17T07:18:33.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.146+0000 7f1df09f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81b1530 con 0x55b6ccdd5b50 2024-04-17T07:18:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.146+0000 7f1de6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5160 0x55b6ccdd7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.146+0000 7f1de6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5160 0x55b6ccdd7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40454/0 (socket says 172.21.15.86:40454) 2024-04-17T07:18:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.146+0000 7f1de6a00640 1 -- 172.21.15.86:0/2681868343 learned_addr learned my addr 172.21.15.86:0/2681868343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.146+0000 7f1de7400640 1 --2- 172.21.15.86:0/2681868343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.147+0000 7f1de6000640 1 --2- 172.21.15.86:0/2681868343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 0x55b6ccdd7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.147+0000 7f1de6a00640 1 -- 172.21.15.86:0/2681868343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 msgr2=0x55b6ccdd7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.147+0000 7f1de6a00640 1 --2- 172.21.15.86:0/2681868343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 0x55b6ccdd7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.147+0000 7f1de6a00640 1 -- 172.21.15.86:0/2681868343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 msgr2=0x55b6ccde6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.147+0000 7f1de6a00640 1 --2- 172.21.15.86:0/2681868343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.147+0000 7f1de6a00640 1 -- 172.21.15.86:0/2681868343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6cca4d3f0 con 0x55b6ccdd5160 2024-04-17T07:18:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.147+0000 7f1de7400640 1 --2- 172.21.15.86:0/2681868343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.147+0000 7f1de6a00640 1 --2- 172.21.15.86:0/2681868343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5160 0x55b6ccdd7010 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8011cb0 tx=0x7f1dd8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d82229f3f826e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.148+0000 7f1dd3400640 1 -- 172.21.15.86:0/2681868343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd80052d0 con 0x55b6ccdd5160 2024-04-17T07:18:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.148+0000 7f1dd3400640 1 -- 172.21.15.86:0/2681868343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dd8029070 con 0x55b6ccdd5160 2024-04-17T07:18:33.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.148+0000 7f1dd3400640 1 -- 172.21.15.86:0/2681868343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800f500 con 0x55b6ccdd5160 2024-04-17T07:18:33.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.149+0000 7f1df09f8740 1 -- 172.21.15.86:0/2681868343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5160 msgr2=0x55b6ccdd7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.149+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2681868343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5160 0x55b6ccdd7010 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8011cb0 tx=0x7f1dd8009750 comp rx=0 tx=0).stop 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.149+0000 7f1df09f8740 1 -- 172.21.15.86:0/2681868343 shutdown_connections 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.149+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2681868343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccddacc0 0x55b6ccde6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.149+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2681868343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccdd5b50 0x55b6ccdd7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.149+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2681868343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5160 0x55b6ccdd7010 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.149+0000 7f1df09f8740 1 -- 172.21.15.86:0/2681868343 >> 172.21.15.86:0/2681868343 conn(0x55b6ccdd06d0 msgr2=0x55b6ccddf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.149+0000 7f1df09f8740 1 -- 172.21.15.86:0/2681868343 shutdown_connections 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.149+0000 7f1df09f8740 1 -- 172.21.15.86:0/2681868343 wait complete. 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.149+0000 7f1df09f8740 1 -- start start 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.150+0000 7f1df09f8740 1 Processor -- start 2024-04-17T07:18:33.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.150+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 0x55b6ccde44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.150+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccde49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.150+0000 7f1df09f8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccddacc0 0x55b6ccde59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.150+0000 7f1df09f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81b22d0 con 0x55b6ccdd5b50 2024-04-17T07:18:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.150+0000 7f1df09f8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81b1c00 con 0x55b6ccdd5160 2024-04-17T07:18:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.150+0000 7f1df09f8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1de81aee30 con 0x55b6ccddacc0 2024-04-17T07:18:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.150+0000 7f1df09f8740 10 client.0 _ll_register_callbacks cb 0x7f1de00125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccde49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccde49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40466/0 (socket says 172.21.15.86:40466) 2024-04-17T07:18:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de6000640 1 -- 172.21.15.86:0/2883688855 learned_addr learned my addr 172.21.15.86:0/2883688855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de6a00640 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 0x55b6ccde44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de7400640 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccddacc0 0x55b6ccde59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de6000640 1 -- 172.21.15.86:0/2883688855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccddacc0 msgr2=0x55b6ccde59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de6000640 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccddacc0 0x55b6ccde59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de6000640 1 -- 172.21.15.86:0/2883688855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 msgr2=0x55b6ccde44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de6000640 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 0x55b6ccde44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de6000640 1 -- 172.21.15.86:0/2883688855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ccb177d0 con 0x55b6ccdd5b50 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de7400640 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccddacc0 0x55b6ccde59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.151+0000 7f1de6a00640 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 0x55b6ccde44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.152+0000 7f1de6000640 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccde49e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40beaa0 tx=0x7f1dd40badb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1f84a518035c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.152+0000 7f1dd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:33.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.152+0000 7f1dd0c00640 1 -- 172.21.15.86:0/2883688855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd40be470 con 0x55b6ccdd5b50 2024-04-17T07:18:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.152+0000 7f1dd0c00640 1 -- 172.21.15.86:0/2883688855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1dd40f0440 con 0x55b6ccdd5b50 2024-04-17T07:18:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.152+0000 7f1df09f8740 1 -- 172.21.15.86:0/2883688855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd8029070 con 0x55b6ccdd5b50 2024-04-17T07:18:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.153+0000 7f1dd0c00640 1 -- 172.21.15.86:0/2883688855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1dd40044e0 con 0x55b6ccdd5b50 2024-04-17T07:18:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.153+0000 7f1df09f8740 10 client.16602 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:33.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.153+0000 7f1df09f8740 1 -- 172.21.15.86:0/2883688855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6cce518a0 con 0x55b6ccdd5b50 2024-04-17T07:18:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.153+0000 7f1dd0c00640 1 -- 172.21.15.86:0/2883688855 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dd8029070 con 0x55b6ccdd5b50 2024-04-17T07:18:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.153+0000 7f1dd0c00640 1 -- 172.21.15.86:0/2883688855 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1dd40bed70 con 0x55b6ccdd5b50 2024-04-17T07:18:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.153+0000 7f1df09f8740 10 client.16602 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.153+0000 7f1df09f8740 10 client.16602 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:18:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.153+0000 7f1df09f8740 20 client.16602 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.153+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55b6cce52270 0x55b6cce54690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.154+0000 7f1df09f8740 4 client.16602 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:18:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.154+0000 7f1df09f8740 1 -- 172.21.15.86:0/2883688855 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6ccd428b0 con 0x55b6cce52270 2024-04-17T07:18:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.154+0000 7f1de6a00640 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55b6cce52270 0x55b6cce54690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.155+0000 7f1de6a00640 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55b6cce52270 0x55b6cce54690 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ad365a85620b39f4 server_cookie=7848ea5014ff40f in_seq=0 out_seq=0 2024-04-17T07:18:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.155+0000 7f1dd0c00640 10 client.16602 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:18:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1dd0c00640 1 -- 172.21.15.86:0/2883688855 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6ccd428b0 con 0x55b6cce52270 2024-04-17T07:18:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1dd0c00640 10 client.16602 handle_command_reply: tid=0 2024-04-17T07:18:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 client.16602 shutdown 2024-04-17T07:18:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 -- 172.21.15.86:0/2883688855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 msgr2=0x55b6ccde49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccde49e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40beaa0 tx=0x7f1dd40badb0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 -- 172.21.15.86:0/2883688855 shutdown_connections 2024-04-17T07:18:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55b6cce52270 0x55b6cce54690 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6ccddacc0 0x55b6ccde59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6ccdd5b50 0x55b6ccde49e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 --2- 172.21.15.86:0/2883688855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6ccdd5160 0x55b6ccde44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 -- 172.21.15.86:0/2883688855 >> 172.21.15.86:0/2883688855 conn(0x55b6ccdd06d0 msgr2=0x55b6ccdd7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1dd0c00640 0 client.16602 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:18:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 -- 172.21.15.86:0/2883688855 shutdown_connections 2024-04-17T07:18:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 1 -- 172.21.15.86:0/2883688855 wait complete. 2024-04-17T07:18:33.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.156+0000 7f1df09f8740 20 client.16602 trim_cache size 0 max 16384 2024-04-17T07:18:33.160 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:18:33.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 -- 172.21.15.86:0/3243047128 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dcc0a3bb0 msgr2=0x7f1dcc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dcc0a3bb0 0x7f1dcc0a6070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55b6ccdd63b0 tx=0x7f1de0009600 comp rx=0 tx=0).stop 2024-04-17T07:18:33.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 -- 172.21.15.86:0/3243047128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1de8103070 msgr2=0x7f1de819ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1de8103070 0x7f1de819ebc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4000c00 tx=0x7f1dd40042f0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 -- 172.21.15.86:0/3243047128 shutdown_connections 2024-04-17T07:18:33.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dcc0a3bb0 0x7f1dcc0a6070 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de8105a20 0x7f1de81a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1de8103070 0x7f1de819ebc0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 --2- 172.21.15.86:0/3243047128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1de8100700 0x7f1de819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 -- 172.21.15.86:0/3243047128 >> 172.21.15.86:0/3243047128 conn(0x7f1de80fc320 msgr2=0x7f1de8100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.157+0000 7f1de7e00640 1 -- 172.21.15.86:0/3243047128 shutdown_connections 2024-04-17T07:18:33.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.158+0000 7f1de7e00640 1 -- 172.21.15.86:0/3243047128 wait complete. 2024-04-17T07:18:33.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:18:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.391+0000 7fe8cce00640 1 -- 172.21.15.86:0/866209000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8101e90 msgr2=0x7fe8c81022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.391+0000 7fe8cce00640 1 --2- 172.21.15.86:0/866209000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8101e90 0x7fe8c81022b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009900 tx=0x7fe8b802bc30 comp rx=0 tx=0).stop 2024-04-17T07:18:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.392+0000 7fe8cce00640 1 -- 172.21.15.86:0/866209000 shutdown_connections 2024-04-17T07:18:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.392+0000 7fe8cce00640 1 --2- 172.21.15.86:0/866209000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8c81039a0 0x7fe8c810a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.392+0000 7fe8cce00640 1 --2- 172.21.15.86:0/866209000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8c8102fe0 0x7fe8c8103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.392+0000 7fe8cce00640 1 --2- 172.21.15.86:0/866209000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8101e90 0x7fe8c81022b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.392+0000 7fe8cce00640 1 -- 172.21.15.86:0/866209000 >> 172.21.15.86:0/866209000 conn(0x7fe8c80fd400 msgr2=0x7fe8c80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.392+0000 7fe8cce00640 1 -- 172.21.15.86:0/866209000 shutdown_connections 2024-04-17T07:18:33.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.392+0000 7fe8cce00640 1 -- 172.21.15.86:0/866209000 wait complete. 2024-04-17T07:18:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.393+0000 7fe8cce00640 1 Processor -- start 2024-04-17T07:18:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.393+0000 7fe8cce00640 1 -- start start 2024-04-17T07:18:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.393+0000 7fe8cce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8c8101e90 0x7fe8c819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.393+0000 7fe8cce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8102fe0 0x7fe8c819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.393+0000 7fe8cce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8c81039a0 0x7fe8c81a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.393+0000 7fe8cce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c810da40 con 0x7fe8c8102fe0 2024-04-17T07:18:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.393+0000 7fe8cce00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c810d8c0 con 0x7fe8c8101e90 2024-04-17T07:18:33.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.393+0000 7fe8cce00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c810dbc0 con 0x7fe8c81039a0 2024-04-17T07:18:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.394+0000 7fe8c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8102fe0 0x7fe8c819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.394+0000 7fe8c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8102fe0 0x7fe8c819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40498/0 (socket says 172.21.15.86:40498) 2024-04-17T07:18:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.394+0000 7fe8c6a00640 1 -- 172.21.15.86:0/1525504425 learned_addr learned my addr 172.21.15.86:0/1525504425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.394+0000 7fe8c7400640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8c8101e90 0x7fe8c819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.394+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8c81039a0 0x7fe8c81a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.394+0000 7fe8c6a00640 1 -- 172.21.15.86:0/1525504425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8c81039a0 msgr2=0x7fe8c81a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.394+0000 7fe8c6a00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8c81039a0 0x7fe8c81a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.394+0000 7fe8c6a00640 1 -- 172.21.15.86:0/1525504425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8c8101e90 msgr2=0x7fe8c819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.394+0000 7fe8c6a00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8c8101e90 0x7fe8c819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.394+0000 7fe8c6a00640 1 -- 172.21.15.86:0/1525504425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c81a1f10 con 0x7fe8c8102fe0 2024-04-17T07:18:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.395+0000 7fe8c6a00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8102fe0 0x7fe8c819a830 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400d770 tx=0x7fe8b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.395+0000 7fe8bfe00640 1 -- 172.21.15.86:0/1525504425 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4017070 con 0x7fe8c8102fe0 2024-04-17T07:18:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.395+0000 7fe8bfe00640 1 -- 172.21.15.86:0/1525504425 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8b4009c30 con 0x7fe8c8102fe0 2024-04-17T07:18:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.395+0000 7fe8cce00640 1 -- 172.21.15.86:0/1525504425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c81a2200 con 0x7fe8c8102fe0 2024-04-17T07:18:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.395+0000 7fe8bfe00640 1 -- 172.21.15.86:0/1525504425 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4003d00 con 0x7fe8c8102fe0 2024-04-17T07:18:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.395+0000 7fe8cce00640 1 -- 172.21.15.86:0/1525504425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c81a2820 con 0x7fe8c8102fe0 2024-04-17T07:18:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.399+0000 7fe8d11f2740 1 Processor -- start 2024-04-17T07:18:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.399+0000 7fe8bfe00640 1 -- 172.21.15.86:0/1525504425 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fe8b4003ea0 con 0x7fe8c8102fe0 2024-04-17T07:18:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.399+0000 7fe8d11f2740 1 -- start start 2024-04-17T07:18:33.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97006580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f9700b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f9700e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f96f6c020 con 0x563f9700bcc0 2024-04-17T07:18:33.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563f96f12460 con 0x563f97006160 2024-04-17T07:18:33.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f96f67fe0 con 0x563f97006b50 2024-04-17T07:18:33.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f9700e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f9700e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40500/0 (socket says 172.21.15.86:40500) 2024-04-17T07:18:33.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8c6a00640 1 -- 172.21.15.86:0/2957744256 learned_addr learned my addr 172.21.15.86:0/2957744256 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8c7400640 1 --2- 172.21.15.86:0/2957744256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f9700b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.400+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/2957744256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97006580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.401+0000 7fe8c6a00640 1 -- 172.21.15.86:0/2957744256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 msgr2=0x563f9700b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.401+0000 7fe8c6a00640 1 --2- 172.21.15.86:0/2957744256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f9700b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.401+0000 7fe8c6a00640 1 -- 172.21.15.86:0/2957744256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 msgr2=0x563f97006580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.401+0000 7fe8c6a00640 1 --2- 172.21.15.86:0/2957744256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97006580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.401+0000 7fe8c6a00640 1 -- 172.21.15.86:0/2957744256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f96f1a040 con 0x563f9700bcc0 2024-04-17T07:18:33.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.401+0000 7fe8c6a00640 1 --2- 172.21.15.86:0/2957744256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f9700e150 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40d6650 tx=0x7fe8b40d6740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a6c1ef57756bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.401+0000 7fe8bfe00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8ac0a3c30 0x7fe8ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.402+0000 7fe8bfe00640 1 -- 172.21.15.86:0/1525504425 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe8b40d04e0 con 0x7fe8c8102fe0 2024-04-17T07:18:33.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.402+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8ac0a3c30 0x7fe8ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.402+0000 7fe8bcc00640 1 -- 172.21.15.86:0/2957744256 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40e4070 con 0x563f9700bcc0 2024-04-17T07:18:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.402+0000 7fe8bcc00640 1 -- 172.21.15.86:0/2957744256 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8b409cb40 con 0x563f9700bcc0 2024-04-17T07:18:33.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.402+0000 7fe8bcc00640 1 -- 172.21.15.86:0/2957744256 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b409bda0 con 0x563f9700bcc0 2024-04-17T07:18:33.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.403+0000 7fe8d11f2740 1 -- 172.21.15.86:0/2957744256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 msgr2=0x563f9700e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.403+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8ac0a3c30 0x7fe8ac0a60f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x563f970073b0 tx=0x7fe8c000e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.403+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/2957744256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f9700e150 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40d6650 tx=0x7fe8b40d6740 comp rx=0 tx=0).stop 2024-04-17T07:18:33.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.403+0000 7fe8d11f2740 1 -- 172.21.15.86:0/2957744256 shutdown_connections 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.403+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/2957744256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f9700e150 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.403+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/2957744256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f9700b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.403+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/2957744256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97006580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.403+0000 7fe8d11f2740 1 -- 172.21.15.86:0/2957744256 >> 172.21.15.86:0/2957744256 conn(0x563f970016d0 msgr2=0x563f97001ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.403+0000 7fe8d11f2740 1 -- 172.21.15.86:0/2957744256 shutdown_connections 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.403+0000 7fe8d11f2740 1 -- 172.21.15.86:0/2957744256 wait complete. 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.404+0000 7fe8d11f2740 1 -- start start 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.404+0000 7fe8d11f2740 1 Processor -- start 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.404+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97010710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.404+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97010c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.404+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97017c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.404+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f9700ec70 con 0x563f9700bcc0 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.404+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563f96f25830 con 0x563f97006160 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.404+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f9700edf0 con 0x563f97006b50 2024-04-17T07:18:33.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.404+0000 7fe8d11f2740 10 client.0 _ll_register_callbacks cb 0x563f96ef3fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:33.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97017c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97017c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40512/0 (socket says 172.21.15.86:40512) 2024-04-17T07:18:33.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c7e00640 1 -- 172.21.15.86:0/1031965248 learned_addr learned my addr 172.21.15.86:0/1031965248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c7400640 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97010710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c6a00640 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97010c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c7e00640 1 -- 172.21.15.86:0/1031965248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 msgr2=0x563f97010c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97010c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c7e00640 1 -- 172.21.15.86:0/1031965248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 msgr2=0x563f97010710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97010710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c7e00640 1 -- 172.21.15.86:0/1031965248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f96f29920 con 0x563f9700bcc0 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.405+0000 7fe8c7400640 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97010710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.406+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97017c90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009540 tx=0x7fe8c0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec29831d823f26f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.406+0000 7fe8b1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.406+0000 7fe8b1600640 1 -- 172.21.15.86:0/1031965248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0022400 con 0x563f9700bcc0 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.406+0000 7fe8d11f2740 1 -- 172.21.15.86:0/1031965248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b409cb40 con 0x563f9700bcc0 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.406+0000 7fe8b1600640 1 -- 172.21.15.86:0/1031965248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe8c00225a0 con 0x563f9700bcc0 2024-04-17T07:18:33.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.406+0000 7fe8b1600640 1 -- 172.21.15.86:0/1031965248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8c00227a0 con 0x563f9700bcc0 2024-04-17T07:18:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.406+0000 7fe8d11f2740 10 client.16614 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.406+0000 7fe8d11f2740 1 -- 172.21.15.86:0/1031965248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f96ef77f0 con 0x563f9700bcc0 2024-04-17T07:18:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.407+0000 7fe8b1600640 1 -- 172.21.15.86:0/1031965248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8b409cb40 con 0x563f9700bcc0 2024-04-17T07:18:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.407+0000 7fe8b1600640 1 -- 172.21.15.86:0/1031965248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe8c002c990 con 0x563f9700bcc0 2024-04-17T07:18:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.407+0000 7fe8d11f2740 10 client.16614 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.407+0000 7fe8d11f2740 10 client.16614 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:18:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.407+0000 7fe8d11f2740 20 client.16614 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.407+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f9701c280 0x563f9701e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.407+0000 7fe8d11f2740 4 client.16614 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:33.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.407+0000 7fe8d11f2740 1 -- 172.21.15.86:0/1031965248 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f96f79a40 con 0x563f9701c280 2024-04-17T07:18:33.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.408+0000 7fe8c7400640 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f9701c280 0x563f9701e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.409+0000 7fe8b1600640 10 client.16614 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:18:33.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.409+0000 7fe8c7400640 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f9701c280 0x563f9701e710 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d81dbfc031bc5d51 server_cookie=e0de60764ba7fce7 in_seq=0 out_seq=0 2024-04-17T07:18:33.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8b1600640 1 -- 172.21.15.86:0/1031965248 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f96f79a40 con 0x563f9701c280 2024-04-17T07:18:33.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8b1600640 10 client.16614 handle_command_reply: tid=0 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8d11f2740 1 client.16614 shutdown 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8d11f2740 1 -- 172.21.15.86:0/1031965248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 msgr2=0x563f97017c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97017c90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009540 tx=0x7fe8c0004b50 comp rx=0 tx=0).stop 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8d11f2740 1 -- 172.21.15.86:0/1031965248 shutdown_connections 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f9701c280 0x563f9701e710 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97017c90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97010c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/1031965248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97010710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8d11f2740 1 -- 172.21.15.86:0/1031965248 >> 172.21.15.86:0/1031965248 conn(0x563f970016d0 msgr2=0x563f97000930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.411+0000 7fe8b1600640 0 client.16614 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.412+0000 7fe8d11f2740 1 -- 172.21.15.86:0/1031965248 shutdown_connections 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.412+0000 7fe8d11f2740 1 -- 172.21.15.86:0/1031965248 wait complete. 2024-04-17T07:18:33.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.412+0000 7fe8d11f2740 20 client.16614 trim_cache size 0 max 16384 2024-04-17T07:18:33.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.429+0000 7fe8d11f2740 1 Processor -- start 2024-04-17T07:18:33.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.429+0000 7fe8d11f2740 1 -- start start 2024-04-17T07:18:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.429+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f970000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.429+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97017bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.429+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f970005e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.429+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81acbf0 con 0x563f9700bcc0 2024-04-17T07:18:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.429+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81ae730 con 0x563f97006160 2024-04-17T07:18:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.429+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81ad2c0 con 0x563f97006b50 2024-04-17T07:18:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.430+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f970005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.430+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f970005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40518/0 (socket says 172.21.15.86:40518) 2024-04-17T07:18:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.430+0000 7fe8c7e00640 1 -- 172.21.15.86:0/3194207884 learned_addr learned my addr 172.21.15.86:0/3194207884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.430+0000 7fe8c7400640 1 --2- 172.21.15.86:0/3194207884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f970000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.430+0000 7fe8c6a00640 1 --2- 172.21.15.86:0/3194207884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97017bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.430+0000 7fe8c7e00640 1 -- 172.21.15.86:0/3194207884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 msgr2=0x563f97017bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.430+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/3194207884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97017bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.430+0000 7fe8c7e00640 1 -- 172.21.15.86:0/3194207884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 msgr2=0x563f970000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.430+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/3194207884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f970000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.430+0000 7fe8c7e00640 1 -- 172.21.15.86:0/3194207884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f96c7e3f0 con 0x563f9700bcc0 2024-04-17T07:18:33.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.431+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/3194207884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f970005e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000bdd0 tx=0x7fe8c00097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be32981d5e79e150 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.431+0000 7fe8b3e00640 1 -- 172.21.15.86:0/3194207884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c003a020 con 0x563f9700bcc0 2024-04-17T07:18:33.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.431+0000 7fe8b3e00640 1 -- 172.21.15.86:0/3194207884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8c0027070 con 0x563f9700bcc0 2024-04-17T07:18:33.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.431+0000 7fe8b3e00640 1 -- 172.21.15.86:0/3194207884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c002c450 con 0x563f9700bcc0 2024-04-17T07:18:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.432+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3194207884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 msgr2=0x563f970005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.432+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/3194207884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f970005e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000bdd0 tx=0x7fe8c00097b0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.432+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3194207884 shutdown_connections 2024-04-17T07:18:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.432+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/3194207884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f970005e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.432+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/3194207884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97017bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.432+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/3194207884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f970000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.432+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3194207884 >> 172.21.15.86:0/3194207884 conn(0x563f970016d0 msgr2=0x563f970080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.432+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3194207884 shutdown_connections 2024-04-17T07:18:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.432+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3194207884 wait complete. 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.433+0000 7fe8d11f2740 1 -- start start 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.433+0000 7fe8d11f2740 1 Processor -- start 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.433+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97014e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.433+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97015340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.433+0000 7fe8d11f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97016350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.433+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81ae060 con 0x563f9700bcc0 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.433+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81ad990 con 0x563f97006160 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.433+0000 7fe8d11f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81aabc0 con 0x563f97006b50 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.433+0000 7fe8d11f2740 10 client.0 _ll_register_callbacks cb 0x7fe8c0018850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97016350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97016350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40522/0 (socket says 172.21.15.86:40522) 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7e00640 1 -- 172.21.15.86:0/3847608010 learned_addr learned my addr 172.21.15.86:0/3847608010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7400640 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97014e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7e00640 1 -- 172.21.15.86:0/3847608010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 msgr2=0x563f97015340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c6a00640 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97015340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97015340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7e00640 1 -- 172.21.15.86:0/3847608010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 msgr2=0x563f97014e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97014e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7e00640 1 -- 172.21.15.86:0/3847608010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f96d487d0 con 0x563f9700bcc0 2024-04-17T07:18:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7400640 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97014e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8c7e00640 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97016350 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000bd30 tx=0x7fe8c0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d181a1e365539da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.434+0000 7fe8b1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.435+0000 7fe8b1600640 1 -- 172.21.15.86:0/3847608010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c003a030 con 0x563f9700bcc0 2024-04-17T07:18:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.435+0000 7fe8b1600640 1 -- 172.21.15.86:0/3847608010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe8c0022d50 con 0x563f9700bcc0 2024-04-17T07:18:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.435+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3847608010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c0027070 con 0x563f9700bcc0 2024-04-17T07:18:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.435+0000 7fe8b1600640 1 -- 172.21.15.86:0/3847608010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8c004f050 con 0x563f9700bcc0 2024-04-17T07:18:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.435+0000 7fe8d11f2740 10 client.16620 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.435+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3847608010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f97017a70 con 0x563f9700bcc0 2024-04-17T07:18:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.436+0000 7fe8b1600640 1 -- 172.21.15.86:0/3847608010 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8c0027070 con 0x563f9700bcc0 2024-04-17T07:18:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.436+0000 7fe8b1600640 1 -- 172.21.15.86:0/3847608010 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe8c0015570 con 0x563f9700bcc0 2024-04-17T07:18:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.436+0000 7fe8d11f2740 10 client.16620 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.436+0000 7fe8d11f2740 10 client.16620 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:18:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.436+0000 7fe8d11f2740 20 client.16620 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.436+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f97082bb0 0x563f97084fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.436+0000 7fe8d11f2740 4 client.16620 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:18:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.436+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3847608010 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f96f738b0 con 0x563f97082bb0 2024-04-17T07:18:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.437+0000 7fe8c7400640 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f97082bb0 0x563f97084fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8c7400640 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f97082bb0 0x563f97084fa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6122cfe36472ec7f server_cookie=7784b89ff6d90117 in_seq=0 out_seq=0 2024-04-17T07:18:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8b1600640 10 client.16620 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:18:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8b1600640 1 -- 172.21.15.86:0/3847608010 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f96f738b0 con 0x563f97082bb0 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8b1600640 10 client.16620 handle_command_reply: tid=0 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8d11f2740 1 client.16620 shutdown 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3847608010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 msgr2=0x563f97016350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97016350 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000bd30 tx=0x7fe8c0009920 comp rx=0 tx=0).stop 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3847608010 shutdown_connections 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f97082bb0 0x563f97084fa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f9700bcc0 0x563f97016350 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f97006b50 0x563f97015340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8d11f2740 1 --2- 172.21.15.86:0/3847608010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f97006160 0x563f97014e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3847608010 >> 172.21.15.86:0/3847608010 conn(0x563f970016d0 msgr2=0x563f97004930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.438+0000 7fe8b1600640 0 client.16620 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.439+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3847608010 shutdown_connections 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.439+0000 7fe8d11f2740 1 -- 172.21.15.86:0/3847608010 wait complete. 2024-04-17T07:18:33.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.439+0000 7fe8d11f2740 20 client.16620 trim_cache size 0 max 16384 2024-04-17T07:18:33.441 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:18:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 -- 172.21.15.86:0/1525504425 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8ac0a3c30 msgr2=0x7fe8ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8ac0a3c30 0x7fe8ac0a60f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x563f970073b0 tx=0x7fe8c000e040 comp rx=0 tx=0).stop 2024-04-17T07:18:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 -- 172.21.15.86:0/1525504425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8102fe0 msgr2=0x7fe8c819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8102fe0 0x7fe8c819a830 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400d770 tx=0x7fe8b400dc30 comp rx=0 tx=0).stop 2024-04-17T07:18:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 -- 172.21.15.86:0/1525504425 shutdown_connections 2024-04-17T07:18:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe8ac0a3c30 0x7fe8ac0a60f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8c81039a0 0x7fe8c81a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8c8102fe0 0x7fe8c819a830 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 --2- 172.21.15.86:0/1525504425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8c8101e90 0x7fe8c819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 -- 172.21.15.86:0/1525504425 >> 172.21.15.86:0/1525504425 conn(0x7fe8c80fd400 msgr2=0x7fe8c8074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 -- 172.21.15.86:0/1525504425 shutdown_connections 2024-04-17T07:18:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.440+0000 7fe8cce00640 1 -- 172.21.15.86:0/1525504425 wait complete. 2024-04-17T07:18:33.459 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:18:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.685+0000 7feae8800640 1 -- 172.21.15.86:0/1541093147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae01011d0 msgr2=0x7feae0101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.685+0000 7feae8800640 1 --2- 172.21.15.86:0/1541093147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae01011d0 0x7feae0101630 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fead0009900 tx=0x7fead002bc30 comp rx=0 tx=0).stop 2024-04-17T07:18:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.686+0000 7feae8800640 1 -- 172.21.15.86:0/1541093147 shutdown_connections 2024-04-17T07:18:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.686+0000 7feae8800640 1 --2- 172.21.15.86:0/1541093147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0101b70 0x7feae0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.686+0000 7feae8800640 1 --2- 172.21.15.86:0/1541093147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae01011d0 0x7feae0101630 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.686+0000 7feae8800640 1 --2- 172.21.15.86:0/1541093147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feae0100870 0x7feae0100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.686+0000 7feae8800640 1 -- 172.21.15.86:0/1541093147 >> 172.21.15.86:0/1541093147 conn(0x7feae00fc410 msgr2=0x7feae00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.686+0000 7feae8800640 1 -- 172.21.15.86:0/1541093147 shutdown_connections 2024-04-17T07:18:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.686+0000 7feae8800640 1 -- 172.21.15.86:0/1541093147 wait complete. 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.687+0000 7feae8800640 1 Processor -- start 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.687+0000 7feae8800640 1 -- start start 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.687+0000 7feae8800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feae0100870 0x7feae019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.687+0000 7feae8800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae01011d0 0x7feae019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.687+0000 7feae8800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0101b70 0x7feae01a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.687+0000 7feae8800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feae010fcf0 con 0x7feae01011d0 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.687+0000 7feae8800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feae010fb70 con 0x7feae0100870 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.687+0000 7feae8800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feae010fe70 con 0x7feae0101b70 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.688+0000 7feae5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae01011d0 0x7feae019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.688+0000 7feae5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae01011d0 0x7feae019a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40534/0 (socket says 172.21.15.86:40534) 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.688+0000 7feae5400640 1 -- 172.21.15.86:0/3301569001 learned_addr learned my addr 172.21.15.86:0/3301569001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.688+0000 7feae5e00640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feae0100870 0x7feae019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.688+0000 7feae6800640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0101b70 0x7feae01a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.688+0000 7feae5400640 1 -- 172.21.15.86:0/3301569001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0101b70 msgr2=0x7feae01a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.688+0000 7feae5400640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0101b70 0x7feae01a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.688+0000 7feae5400640 1 -- 172.21.15.86:0/3301569001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feae0100870 msgr2=0x7feae019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.688+0000 7feae5400640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feae0100870 0x7feae019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.688+0000 7feae5400640 1 -- 172.21.15.86:0/3301569001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae01a2000 con 0x7feae01011d0 2024-04-17T07:18:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.689+0000 7feae5400640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae01011d0 0x7feae019a920 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fead0006010 tx=0x7fead0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.689+0000 7feadaa00640 1 -- 172.21.15.86:0/3301569001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0043020 con 0x7feae01011d0 2024-04-17T07:18:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.689+0000 7feadaa00640 1 -- 172.21.15.86:0/3301569001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead0004440 con 0x7feae01011d0 2024-04-17T07:18:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.689+0000 7feae8800640 1 -- 172.21.15.86:0/3301569001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae01a2290 con 0x7feae01011d0 2024-04-17T07:18:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.689+0000 7feadaa00640 1 -- 172.21.15.86:0/3301569001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0003bf0 con 0x7feae01011d0 2024-04-17T07:18:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.689+0000 7feae8800640 1 -- 172.21.15.86:0/3301569001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae01a27e0 con 0x7feae01011d0 2024-04-17T07:18:33.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.693+0000 7feaeb58c740 1 Processor -- start 2024-04-17T07:18:33.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.693+0000 7feadaa00640 1 -- 172.21.15.86:0/3301569001 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fead0046020 con 0x7feae01011d0 2024-04-17T07:18:33.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.693+0000 7feaeb58c740 1 -- start start 2024-04-17T07:18:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.693+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e382fa0 0x56463e3833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.694+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e383990 0x56463e388490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.694+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e388b00 0x56463e38af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.694+0000 7feaeb58c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56463e2e9020 con 0x56463e388b00 2024-04-17T07:18:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.694+0000 7feaeb58c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56463e28f460 con 0x56463e382fa0 2024-04-17T07:18:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.694+0000 7feaeb58c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56463e2e4fe0 con 0x56463e383990 2024-04-17T07:18:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.694+0000 7feae5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e388b00 0x56463e38af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.694+0000 7feae5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e388b00 0x56463e38af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40542/0 (socket says 172.21.15.86:40542) 2024-04-17T07:18:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.694+0000 7feae5400640 1 -- 172.21.15.86:0/1248785522 learned_addr learned my addr 172.21.15.86:0/1248785522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.694+0000 7feae5e00640 1 --2- 172.21.15.86:0/1248785522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e383990 0x56463e388490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.694+0000 7feae6800640 1 --2- 172.21.15.86:0/1248785522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e382fa0 0x56463e3833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.695+0000 7feae5400640 1 -- 172.21.15.86:0/1248785522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e383990 msgr2=0x56463e388490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.695+0000 7feae5400640 1 --2- 172.21.15.86:0/1248785522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e383990 0x56463e388490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.695+0000 7feae5400640 1 -- 172.21.15.86:0/1248785522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e382fa0 msgr2=0x56463e3833c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.695+0000 7feae5400640 1 --2- 172.21.15.86:0/1248785522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e382fa0 0x56463e3833c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.695+0000 7feae5400640 1 -- 172.21.15.86:0/1248785522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463e297040 con 0x56463e388b00 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.695+0000 7feadaa00640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feac80a3bb0 0x7feac80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.695+0000 7feadaa00640 1 -- 172.21.15.86:0/3301569001 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fead00f2c30 con 0x7feae01011d0 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.695+0000 7feae6800640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feac80a3bb0 0x7feac80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.695+0000 7feae5400640 1 --2- 172.21.15.86:0/1248785522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e388b00 0x56463e38af90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fead00f6bd0 tx=0x7fead00f6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66618cee835279b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.696+0000 7feacf000640 1 -- 172.21.15.86:0/1248785522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead00bfe60 con 0x56463e388b00 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.696+0000 7feacf000640 1 -- 172.21.15.86:0/1248785522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead00f6e50 con 0x56463e388b00 2024-04-17T07:18:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.696+0000 7feacf000640 1 -- 172.21.15.86:0/1248785522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead00be670 con 0x56463e388b00 2024-04-17T07:18:33.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feaeb58c740 1 -- 172.21.15.86:0/1248785522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e388b00 msgr2=0x56463e38af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feae6800640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feac80a3bb0 0x7feac80a6070 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x56463e3841f0 tx=0x7feadc009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feaeb58c740 1 --2- 172.21.15.86:0/1248785522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e388b00 0x56463e38af90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fead00f6bd0 tx=0x7fead00f6cc0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feaeb58c740 1 -- 172.21.15.86:0/1248785522 shutdown_connections 2024-04-17T07:18:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feaeb58c740 1 --2- 172.21.15.86:0/1248785522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e388b00 0x56463e38af90 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feaeb58c740 1 --2- 172.21.15.86:0/1248785522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e383990 0x56463e388490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feaeb58c740 1 --2- 172.21.15.86:0/1248785522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e382fa0 0x56463e3833c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feaeb58c740 1 -- 172.21.15.86:0/1248785522 >> 172.21.15.86:0/1248785522 conn(0x56463e37e510 msgr2=0x56463e37e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feaeb58c740 1 -- 172.21.15.86:0/1248785522 shutdown_connections 2024-04-17T07:18:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feaeb58c740 1 -- 172.21.15.86:0/1248785522 wait complete. 2024-04-17T07:18:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.697+0000 7feaeb58c740 1 -- start start 2024-04-17T07:18:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feaeb58c740 1 Processor -- start 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e382fa0 0x56463e38d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e383990 0x56463e38d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e3947b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feaeb58c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56463e38b870 con 0x56463e382fa0 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feaeb58c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56463e2a2830 con 0x56463e388b00 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feaeb58c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56463e38b9f0 con 0x56463e383990 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feaeb58c740 10 client.0 _ll_register_callbacks cb 0x56463e270fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feae5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e382fa0 0x56463e38d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feae5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e382fa0 0x56463e38d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40548/0 (socket says 172.21.15.86:40548) 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.698+0000 7feae5e00640 1 -- 172.21.15.86:0/316215972 learned_addr learned my addr 172.21.15.86:0/316215972 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.699+0000 7feae5400640 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e383990 0x56463e38d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.699+0000 7feae6800640 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e3947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.699+0000 7feae5e00640 1 -- 172.21.15.86:0/316215972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e383990 msgr2=0x56463e38d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.699+0000 7feae5e00640 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e383990 0x56463e38d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.699+0000 7feae5e00640 1 -- 172.21.15.86:0/316215972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 msgr2=0x56463e3947b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.699+0000 7feae5e00640 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e3947b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.699+0000 7feae5e00640 1 -- 172.21.15.86:0/316215972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463e2a6920 con 0x56463e382fa0 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.699+0000 7feae5e00640 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e382fa0 0x56463e38d230 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fead4012700 tx=0x7fead4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6e7d906231ee25 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.699+0000 7feac3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.699+0000 7feac3400640 1 -- 172.21.15.86:0/316215972 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4004260 con 0x56463e382fa0 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.700+0000 7feac3400640 1 -- 172.21.15.86:0/316215972 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fead4004400 con 0x56463e382fa0 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.700+0000 7feaeb58c740 1 -- 172.21.15.86:0/316215972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fead00f6e50 con 0x56463e382fa0 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.700+0000 7feac3400640 1 -- 172.21.15.86:0/316215972 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fead4025050 con 0x56463e382fa0 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.700+0000 7feaeb58c740 10 client.16632 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.700+0000 7feaeb58c740 1 -- 172.21.15.86:0/316215972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56463e2747f0 con 0x56463e382fa0 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.700+0000 7feac3400640 1 -- 172.21.15.86:0/316215972 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead00f6e50 con 0x56463e382fa0 2024-04-17T07:18:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.700+0000 7feac3400640 1 -- 172.21.15.86:0/316215972 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fead4003ca0 con 0x56463e382fa0 2024-04-17T07:18:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.701+0000 7feaeb58c740 10 client.16632 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.701+0000 7feaeb58c740 10 client.16632 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:18:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.701+0000 7feaeb58c740 20 client.16632 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.701+0000 7feaeb58c740 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56463e398d90 0x56463e39b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.701+0000 7feaeb58c740 4 client.16632 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.701+0000 7feaeb58c740 1 -- 172.21.15.86:0/316215972 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56463e2f6a40 con 0x56463e398d90 2024-04-17T07:18:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.701+0000 7feae5400640 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56463e398d90 0x56463e39b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.703+0000 7feae5400640 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56463e398d90 0x56463e39b230 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c10577b017d6ff5b server_cookie=7d5830fd5c91aa98 in_seq=0 out_seq=0 2024-04-17T07:18:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.703+0000 7feac3400640 10 client.16632 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:18:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feac3400640 1 -- 172.21.15.86:0/316215972 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56463e2f6a40 con 0x56463e398d90 2024-04-17T07:18:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feac3400640 10 client.16632 handle_command_reply: tid=0 2024-04-17T07:18:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feaeb58c740 1 client.16632 shutdown 2024-04-17T07:18:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feaeb58c740 1 -- 172.21.15.86:0/316215972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e382fa0 msgr2=0x56463e38d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feaeb58c740 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e382fa0 0x56463e38d230 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fead4012700 tx=0x7fead4012bc0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feaeb58c740 1 -- 172.21.15.86:0/316215972 shutdown_connections 2024-04-17T07:18:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feaeb58c740 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56463e398d90 0x56463e39b230 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feaeb58c740 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e3947b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feaeb58c740 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e383990 0x56463e38d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feac3400640 0 client.16632 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:18:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.705+0000 7feaeb58c740 1 --2- 172.21.15.86:0/316215972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e382fa0 0x56463e38d230 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.706+0000 7feaeb58c740 1 -- 172.21.15.86:0/316215972 >> 172.21.15.86:0/316215972 conn(0x56463e37e510 msgr2=0x56463e37d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.706+0000 7feaeb58c740 1 -- 172.21.15.86:0/316215972 shutdown_connections 2024-04-17T07:18:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.706+0000 7feaeb58c740 1 -- 172.21.15.86:0/316215972 wait complete. 2024-04-17T07:18:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.706+0000 7feaeb58c740 20 client.16632 trim_cache size 0 max 16384 2024-04-17T07:18:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.721+0000 7feaeb58c740 1 Processor -- start 2024-04-17T07:18:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.721+0000 7feaeb58c740 1 -- start start 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 0x56463e39faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e394770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e3a0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feaeb58c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feae01accf0 con 0x56463e383990 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feaeb58c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feae01ae830 con 0x56463e388b00 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feaeb58c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feae01ad3c0 con 0x56463e382fa0 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feae5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e394770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feae5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e394770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40556/0 (socket says 172.21.15.86:40556) 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feae5400640 1 -- 172.21.15.86:0/1301005101 learned_addr learned my addr 172.21.15.86:0/1301005101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feae6800640 1 --2- 172.21.15.86:0/1301005101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e3a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.722+0000 7feae5e00640 1 --2- 172.21.15.86:0/1301005101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 0x56463e39faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.723+0000 7feae5400640 1 -- 172.21.15.86:0/1301005101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 msgr2=0x56463e39faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.723+0000 7feae5400640 1 --2- 172.21.15.86:0/1301005101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 0x56463e39faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.723+0000 7feae5400640 1 -- 172.21.15.86:0/1301005101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 msgr2=0x56463e3a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.723+0000 7feae5400640 1 --2- 172.21.15.86:0/1301005101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e3a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.723+0000 7feae5400640 1 -- 172.21.15.86:0/1301005101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463dffb3f0 con 0x56463e383990 2024-04-17T07:18:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.723+0000 7feae5e00640 1 --2- 172.21.15.86:0/1301005101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 0x56463e39faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.723+0000 7feae5400640 1 --2- 172.21.15.86:0/1301005101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e394770 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fead00bcde0 tx=0x7fead00f6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b72fc0a26554dc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.723+0000 7feace600640 1 -- 172.21.15.86:0/1301005101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead00d8070 con 0x56463e383990 2024-04-17T07:18:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.724+0000 7feace600640 1 -- 172.21.15.86:0/1301005101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead00f6e50 con 0x56463e383990 2024-04-17T07:18:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.724+0000 7feace600640 1 -- 172.21.15.86:0/1301005101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead00bfe60 con 0x56463e383990 2024-04-17T07:18:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.724+0000 7feaeb58c740 1 -- 172.21.15.86:0/1301005101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 msgr2=0x56463e394770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.724+0000 7feaeb58c740 1 --2- 172.21.15.86:0/1301005101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e394770 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fead00bcde0 tx=0x7fead00f6670 comp rx=0 tx=0).stop 2024-04-17T07:18:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.724+0000 7feaeb58c740 1 -- 172.21.15.86:0/1301005101 shutdown_connections 2024-04-17T07:18:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.724+0000 7feaeb58c740 1 --2- 172.21.15.86:0/1301005101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e3a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.724+0000 7feaeb58c740 1 --2- 172.21.15.86:0/1301005101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e394770 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.724+0000 7feaeb58c740 1 --2- 172.21.15.86:0/1301005101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 0x56463e39faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.724+0000 7feaeb58c740 1 -- 172.21.15.86:0/1301005101 >> 172.21.15.86:0/1301005101 conn(0x56463e37e510 msgr2=0x56463e382000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.725+0000 7feaeb58c740 1 -- 172.21.15.86:0/1301005101 shutdown_connections 2024-04-17T07:18:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.725+0000 7feaeb58c740 1 -- 172.21.15.86:0/1301005101 wait complete. 2024-04-17T07:18:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.725+0000 7feaeb58c740 1 -- start start 2024-04-17T07:18:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.725+0000 7feaeb58c740 1 Processor -- start 2024-04-17T07:18:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 0x56463e391960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e391ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feaeb58c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e392e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feaeb58c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feae01ae160 con 0x56463e383990 2024-04-17T07:18:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feaeb58c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feae01ada90 con 0x56463e388b00 2024-04-17T07:18:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feaeb58c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feae01aacc0 con 0x56463e382fa0 2024-04-17T07:18:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feaeb58c740 10 client.0 _ll_register_callbacks cb 0x7fead4009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:33.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feae5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e391ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feae5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e391ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40572/0 (socket says 172.21.15.86:40572) 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feae5400640 1 -- 172.21.15.86:0/2066810267 learned_addr learned my addr 172.21.15.86:0/2066810267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.726+0000 7feae5e00640 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 0x56463e391960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feae6800640 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e392e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feae5400640 1 -- 172.21.15.86:0/2066810267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 msgr2=0x56463e391960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feae5400640 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 0x56463e391960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feae5400640 1 -- 172.21.15.86:0/2066810267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 msgr2=0x56463e392e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feae5400640 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e392e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feae5400640 1 -- 172.21.15.86:0/2066810267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463e0c57d0 con 0x56463e383990 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feae5400640 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e391ea0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fead00beac0 tx=0x7fead0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cd55165de6cfc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feac3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feac3400640 1 -- 172.21.15.86:0/2066810267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead00bfe60 con 0x56463e383990 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feac3400640 1 -- 172.21.15.86:0/2066810267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fead00bce30 con 0x56463e383990 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.727+0000 7feaeb58c740 1 -- 172.21.15.86:0/2066810267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fead00f6e50 con 0x56463e383990 2024-04-17T07:18:33.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feac3400640 1 -- 172.21.15.86:0/2066810267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fead00d8040 con 0x56463e383990 2024-04-17T07:18:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feaeb58c740 10 client.16638 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feaeb58c740 1 -- 172.21.15.86:0/2066810267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56463e0c57d0 con 0x56463e383990 2024-04-17T07:18:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feac3400640 1 -- 172.21.15.86:0/2066810267 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fead00f6e50 con 0x56463e383990 2024-04-17T07:18:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feac3400640 1 -- 172.21.15.86:0/2066810267 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fead00f03d0 con 0x56463e383990 2024-04-17T07:18:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feaeb58c740 10 client.16638 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feaeb58c740 10 client.16638 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:18:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feaeb58c740 20 client.16638 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feaeb58c740 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56463e394970 0x56463e401380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feaeb58c740 4 client.16638 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:18:33.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.728+0000 7feaeb58c740 1 -- 172.21.15.86:0/2066810267 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56463e388870 con 0x56463e394970 2024-04-17T07:18:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.729+0000 7feae5e00640 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56463e394970 0x56463e401380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.730+0000 7feac3400640 10 client.16638 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.730+0000 7feae5e00640 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56463e394970 0x56463e401380 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da9b52732c1a609d server_cookie=26638b1888f46942 in_seq=0 out_seq=0 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.730+0000 7feac3400640 1 -- 172.21.15.86:0/2066810267 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56463e388870 con 0x56463e394970 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.730+0000 7feac3400640 10 client.16638 handle_command_reply: tid=0 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.730+0000 7feaeb58c740 1 client.16638 shutdown 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 1 -- 172.21.15.86:0/2066810267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 msgr2=0x56463e391ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e391ea0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fead00beac0 tx=0x7fead0004610 comp rx=0 tx=0).stop 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 1 -- 172.21.15.86:0/2066810267 shutdown_connections 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56463e394970 0x56463e401380 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56463e388b00 0x56463e392e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56463e383990 0x56463e391ea0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 1 --2- 172.21.15.86:0/2066810267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56463e382fa0 0x56463e391960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feac3400640 0 client.16638 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 1 -- 172.21.15.86:0/2066810267 >> 172.21.15.86:0/2066810267 conn(0x56463e37e510 msgr2=0x56463e37ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 1 -- 172.21.15.86:0/2066810267 shutdown_connections 2024-04-17T07:18:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 1 -- 172.21.15.86:0/2066810267 wait complete. 2024-04-17T07:18:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.731+0000 7feaeb58c740 20 client.16638 trim_cache size 0 max 16384 2024-04-17T07:18:33.733 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:18:33.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 -- 172.21.15.86:0/3301569001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feac80a3bb0 msgr2=0x7feac80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feac80a3bb0 0x7feac80a6070 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x56463e3841f0 tx=0x7feadc009740 comp rx=0 tx=0).stop 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 -- 172.21.15.86:0/3301569001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae01011d0 msgr2=0x7feae019a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae01011d0 0x7feae019a920 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fead0006010 tx=0x7fead0004040 comp rx=0 tx=0).stop 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 -- 172.21.15.86:0/3301569001 shutdown_connections 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feac80a3bb0 0x7feac80a6070 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feae0101b70 0x7feae01a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feae01011d0 0x7feae019a920 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 --2- 172.21.15.86:0/3301569001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feae0100870 0x7feae019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 -- 172.21.15.86:0/3301569001 >> 172.21.15.86:0/3301569001 conn(0x7feae00fc410 msgr2=0x7feae00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 -- 172.21.15.86:0/3301569001 shutdown_connections 2024-04-17T07:18:33.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.732+0000 7feae8800640 1 -- 172.21.15.86:0/3301569001 wait complete. 2024-04-17T07:18:33.753 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:33 smithi086 ceph-mon[31733]: pgmap v566: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 26 op/s 2024-04-17T07:18:33.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:18:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.975+0000 7fbdf1800640 1 -- 172.21.15.86:0/3255776894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdec105d90 msgr2=0x7fbdec10c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.975+0000 7fbdf1800640 1 --2- 172.21.15.86:0/3255776894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdec105d90 0x7fbdec10c620 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009ca0 tx=0x7fbde402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.975+0000 7fbdf1800640 1 -- 172.21.15.86:0/3255776894 shutdown_connections 2024-04-17T07:18:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.975+0000 7fbdf1800640 1 --2- 172.21.15.86:0/3255776894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdec105d90 0x7fbdec10c620 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.975+0000 7fbdf1800640 1 --2- 172.21.15.86:0/3255776894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdec1053d0 0x7fbdec105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.975+0000 7fbdf1800640 1 --2- 172.21.15.86:0/3255776894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdec1041e0 0x7fbdec104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.975+0000 7fbdf1800640 1 -- 172.21.15.86:0/3255776894 >> 172.21.15.86:0/3255776894 conn(0x7fbdec0ff750 msgr2=0x7fbdec101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.975+0000 7fbdf1800640 1 -- 172.21.15.86:0/3255776894 shutdown_connections 2024-04-17T07:18:33.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.975+0000 7fbdf1800640 1 -- 172.21.15.86:0/3255776894 wait complete. 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.976+0000 7fbdf1800640 1 Processor -- start 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.976+0000 7fbdf1800640 1 -- start start 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdf1800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdec1041e0 0x7fbdec19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdf1800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdec1053d0 0x7fbdec19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdf1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdec105d90 0x7fbdec1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdf1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec10fd20 con 0x7fbdec105d90 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdf1800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec10fba0 con 0x7fbdec1053d0 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdf1800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec10fea0 con 0x7fbdec1041e0 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdeb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdec105d90 0x7fbdec1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdeb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdec105d90 0x7fbdec1a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40592/0 (socket says 172.21.15.86:40592) 2024-04-17T07:18:33.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdeb400640 1 -- 172.21.15.86:0/4120485715 learned_addr learned my addr 172.21.15.86:0/4120485715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdec1041e0 0x7fbdec19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdea000640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdec1053d0 0x7fbdec19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdeb400640 1 -- 172.21.15.86:0/4120485715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdec1041e0 msgr2=0x7fbdec19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdeb400640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdec1041e0 0x7fbdec19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdeb400640 1 -- 172.21.15.86:0/4120485715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdec1053d0 msgr2=0x7fbdec19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdeb400640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdec1053d0 0x7fbdec19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.977+0000 7fbdeb400640 1 -- 172.21.15.86:0/4120485715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec1a1f10 con 0x7fbdec105d90 2024-04-17T07:18:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.978+0000 7fbdeb400640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdec105d90 0x7fbdec1a18a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbde402c700 tx=0x7fbde40328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.978+0000 7fbde3400640 1 -- 172.21.15.86:0/4120485715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4041020 con 0x7fbdec105d90 2024-04-17T07:18:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.978+0000 7fbdf1800640 1 -- 172.21.15.86:0/4120485715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdec1a21a0 con 0x7fbdec105d90 2024-04-17T07:18:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.978+0000 7fbde3400640 1 -- 172.21.15.86:0/4120485715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbde4003e20 con 0x7fbdec105d90 2024-04-17T07:18:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.978+0000 7fbde3400640 1 -- 172.21.15.86:0/4120485715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4039040 con 0x7fbdec105d90 2024-04-17T07:18:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.978+0000 7fbdf1800640 1 -- 172.21.15.86:0/4120485715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdec1a2690 con 0x7fbdec105d90 2024-04-17T07:18:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.982+0000 7fbde3400640 1 -- 172.21.15.86:0/4120485715 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fbde4044020 con 0x7fbdec105d90 2024-04-17T07:18:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.982+0000 7fbdf4599740 1 Processor -- start 2024-04-17T07:18:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.982+0000 7fbdf4599740 1 -- start start 2024-04-17T07:18:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9bc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9c1b00 0x55d19f9c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdf4599740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d19f922020 con 0x55d19f9bc990 2024-04-17T07:18:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdf4599740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d19f8c8460 con 0x55d19f9bbfa0 2024-04-17T07:18:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdf4599740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d19f91dfe0 con 0x55d19f9c1b00 2024-04-17T07:18:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdeaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdeaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40596/0 (socket says 172.21.15.86:40596) 2024-04-17T07:18:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdeaa00640 1 -- 172.21.15.86:0/212483098 learned_addr learned my addr 172.21.15.86:0/212483098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdea000640 1 --2- 172.21.15.86:0/212483098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9bc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.983+0000 7fbdea000640 1 --2- 172.21.15.86:0/212483098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9c1b00 0x55d19f9c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdeaa00640 1 -- 172.21.15.86:0/212483098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9c1b00 msgr2=0x55d19f9c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/212483098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9c1b00 0x55d19f9c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdeaa00640 1 -- 172.21.15.86:0/212483098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 msgr2=0x55d19f9bc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/212483098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9bc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdeaa00640 1 -- 172.21.15.86:0/212483098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d19f8d0040 con 0x55d19f9bc990 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdea000640 1 --2- 172.21.15.86:0/212483098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9bc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbde3400640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbdd00a3bb0 0x7fbdd00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbde3400640 1 -- 172.21.15.86:0/4120485715 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbde40f2c70 con 0x7fbdec105d90 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/212483098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c1490 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00bc90 tx=0x7fbddc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26371c143f4d0447 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdea000640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbdd00a3bb0 0x7fbdd00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdd7e00640 1 -- 172.21.15.86:0/212483098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc015070 con 0x55d19f9bc990 2024-04-17T07:18:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdd7e00640 1 -- 172.21.15.86:0/212483098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbddc0035f0 con 0x55d19f9bc990 2024-04-17T07:18:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.984+0000 7fbdd7e00640 1 -- 172.21.15.86:0/212483098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0116a0 con 0x55d19f9bc990 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.985+0000 7fbdf4599740 1 -- 172.21.15.86:0/212483098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 msgr2=0x55d19f9c1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.985+0000 7fbdea000640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbdd00a3bb0 0x7fbdd00a6070 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55d19f9bd1f0 tx=0x7fbdd8004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.985+0000 7fbdf4599740 1 --2- 172.21.15.86:0/212483098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c1490 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00bc90 tx=0x7fbddc00f730 comp rx=0 tx=0).stop 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.985+0000 7fbdf4599740 1 -- 172.21.15.86:0/212483098 shutdown_connections 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.985+0000 7fbdf4599740 1 --2- 172.21.15.86:0/212483098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9c1b00 0x55d19f9c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.986+0000 7fbdf4599740 1 --2- 172.21.15.86:0/212483098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c1490 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.986+0000 7fbdf4599740 1 --2- 172.21.15.86:0/212483098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9bc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.986+0000 7fbdf4599740 1 -- 172.21.15.86:0/212483098 >> 172.21.15.86:0/212483098 conn(0x55d19f9b7510 msgr2=0x55d19f9b7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.986+0000 7fbdf4599740 1 -- 172.21.15.86:0/212483098 shutdown_connections 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.986+0000 7fbdf4599740 1 -- 172.21.15.86:0/212483098 wait complete. 2024-04-17T07:18:33.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.986+0000 7fbdf4599740 1 -- start start 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.986+0000 7fbdf4599740 1 Processor -- start 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bbfa0 0x55d19f9c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9c1b00 0x55d19f9cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdf4599740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d19f9c4ab0 con 0x55d19f9bc990 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdf4599740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d19f8db830 con 0x55d19f9c1b00 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdf4599740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d19f9c4c30 con 0x55d19f9bbfa0 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdf4599740 10 client.0 _ll_register_callbacks cb 0x55d19f8a9fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40612/0 (socket says 172.21.15.86:40612) 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdea000640 1 -- 172.21.15.86:0/4133564407 learned_addr learned my addr 172.21.15.86:0/4133564407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.987+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bbfa0 0x55d19f9c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.988+0000 7fbdeb400640 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9c1b00 0x55d19f9cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.988+0000 7fbdea000640 1 -- 172.21.15.86:0/4133564407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bbfa0 msgr2=0x55d19f9c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.988+0000 7fbdea000640 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bbfa0 0x55d19f9c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.988+0000 7fbdea000640 1 -- 172.21.15.86:0/4133564407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9c1b00 msgr2=0x55d19f9cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.988+0000 7fbdea000640 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9c1b00 0x55d19f9cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.988+0000 7fbdea000640 1 -- 172.21.15.86:0/4133564407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d19f8df920 con 0x55d19f9bc990 2024-04-17T07:18:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.988+0000 7fbdeb400640 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9c1b00 0x55d19f9cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.988+0000 7fbdea000640 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c6a20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800e400 tx=0x7fbdd800e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611fc7d2d6722637 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:33.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.988+0000 7fbdd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:33.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.988+0000 7fbdd4c00640 1 -- 172.21.15.86:0/4133564407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8020620 con 0x55d19f9bc990 2024-04-17T07:18:33.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.989+0000 7fbdd4c00640 1 -- 172.21.15.86:0/4133564407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbdd80207c0 con 0x55d19f9bc990 2024-04-17T07:18:33.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.989+0000 7fbdf4599740 1 -- 172.21.15.86:0/4133564407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbddc0035f0 con 0x55d19f9bc990 2024-04-17T07:18:33.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.989+0000 7fbdd4c00640 1 -- 172.21.15.86:0/4133564407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbdd8030050 con 0x55d19f9bc990 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.989+0000 7fbdf4599740 10 client.16650 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.989+0000 7fbdf4599740 1 -- 172.21.15.86:0/4133564407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d19f8ad7f0 con 0x55d19f9bc990 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.989+0000 7fbdd4c00640 1 -- 172.21.15.86:0/4133564407 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbddc0035f0 con 0x55d19f9bc990 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.989+0000 7fbdd4c00640 1 -- 172.21.15.86:0/4133564407 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fbdd80289a0 con 0x55d19f9bc990 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.989+0000 7fbdf4599740 10 client.16650 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.989+0000 7fbdf4599740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.989+0000 7fbdf4599740 20 client.16650 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.990+0000 7fbdf4599740 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d19f9d20d0 0x55d19f9d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.990+0000 7fbdf4599740 4 client.16650 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.990+0000 7fbdf4599740 1 -- 172.21.15.86:0/4133564407 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d19f92fa40 con 0x55d19f9d20d0 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.990+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d19f9d20d0 0x55d19f9d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.991+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d19f9d20d0 0x55d19f9d4530 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7815402396e672 server_cookie=e48487835faaf167 in_seq=0 out_seq=0 2024-04-17T07:18:33.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.991+0000 7fbdd4c00640 10 client.16650 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:33.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdd4c00640 1 -- 172.21.15.86:0/4133564407 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d19f92fa40 con 0x55d19f9d20d0 2024-04-17T07:18:33.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdd4c00640 10 client.16650 handle_command_reply: tid=0 2024-04-17T07:18:33.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdf4599740 1 client.16650 shutdown 2024-04-17T07:18:33.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdf4599740 1 -- 172.21.15.86:0/4133564407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 msgr2=0x55d19f9c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:33.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdf4599740 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c6a20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800e400 tx=0x7fbdd800e430 comp rx=0 tx=0).stop 2024-04-17T07:18:33.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdf4599740 1 -- 172.21.15.86:0/4133564407 shutdown_connections 2024-04-17T07:18:33.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdf4599740 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d19f9d20d0 0x55d19f9d4530 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdf4599740 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9c1b00 0x55d19f9cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdf4599740 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9bc990 0x55d19f9c6a20 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdf4599740 1 --2- 172.21.15.86:0/4133564407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bbfa0 0x55d19f9c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:33.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdf4599740 1 -- 172.21.15.86:0/4133564407 >> 172.21.15.86:0/4133564407 conn(0x55d19f9b7510 msgr2=0x55d19f9b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:33.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.993+0000 7fbdd4c00640 0 client.16650 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:33.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.994+0000 7fbdf4599740 1 -- 172.21.15.86:0/4133564407 shutdown_connections 2024-04-17T07:18:33.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.994+0000 7fbdf4599740 1 -- 172.21.15.86:0/4133564407 wait complete. 2024-04-17T07:18:33.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:33.994+0000 7fbdf4599740 20 client.16650 trim_cache size 0 max 16384 2024-04-17T07:18:34.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:33 smithi098 ceph-mon[35418]: pgmap v566: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 26 op/s 2024-04-17T07:18:34.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.010+0000 7fbdf4599740 1 Processor -- start 2024-04-17T07:18:34.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.010+0000 7fbdf4599740 1 -- start start 2024-04-17T07:18:34.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.010+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:34.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.010+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 0x55d19f9cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:34.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.010+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:34.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.010+0000 7fbdf4599740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1acc80 con 0x55d19f9c1b00 2024-04-17T07:18:34.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.010+0000 7fbdf4599740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1ae7c0 con 0x55d19f9bbfa0 2024-04-17T07:18:34.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.010+0000 7fbdf4599740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1ad350 con 0x55d19f9bc990 2024-04-17T07:18:34.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.010+0000 7fbdeb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.011+0000 7fbdeb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40624/0 (socket says 172.21.15.86:40624) 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.011+0000 7fbdeb400640 1 -- 172.21.15.86:0/2740015635 learned_addr learned my addr 172.21.15.86:0/2740015635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.011+0000 7fbdea000640 1 --2- 172.21.15.86:0/2740015635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 0x55d19f9cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.011+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/2740015635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.011+0000 7fbdeb400640 1 -- 172.21.15.86:0/2740015635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 msgr2=0x55d19f9cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.011+0000 7fbdeb400640 1 --2- 172.21.15.86:0/2740015635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 0x55d19f9cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.011+0000 7fbdeb400640 1 -- 172.21.15.86:0/2740015635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 msgr2=0x55d19f9b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.011+0000 7fbdeb400640 1 --2- 172.21.15.86:0/2740015635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.011+0000 7fbdeb400640 1 -- 172.21.15.86:0/2740015635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d19f6343f0 con 0x55d19f9c1b00 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.011+0000 7fbdeb400640 1 --2- 172.21.15.86:0/2740015635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9b5e80 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbde40f03f0 tx=0x7fbde40be710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfaaefd1e051bca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.012+0000 7fbdd7400640 1 -- 172.21.15.86:0/2740015635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde40bf2d0 con 0x55d19f9c1b00 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.012+0000 7fbdd7400640 1 -- 172.21.15.86:0/2740015635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbde40bebd0 con 0x55d19f9c1b00 2024-04-17T07:18:34.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.012+0000 7fbdd7400640 1 -- 172.21.15.86:0/2740015635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde40bb240 con 0x55d19f9c1b00 2024-04-17T07:18:34.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.012+0000 7fbdf4599740 1 -- 172.21.15.86:0/2740015635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 msgr2=0x55d19f9b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:34.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.012+0000 7fbdf4599740 1 --2- 172.21.15.86:0/2740015635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9b5e80 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbde40f03f0 tx=0x7fbde40be710 comp rx=0 tx=0).stop 2024-04-17T07:18:34.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.013+0000 7fbdf4599740 1 -- 172.21.15.86:0/2740015635 shutdown_connections 2024-04-17T07:18:34.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.013+0000 7fbdf4599740 1 --2- 172.21.15.86:0/2740015635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9b5e80 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.013+0000 7fbdf4599740 1 --2- 172.21.15.86:0/2740015635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 0x55d19f9cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.013+0000 7fbdf4599740 1 --2- 172.21.15.86:0/2740015635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.013+0000 7fbdf4599740 1 -- 172.21.15.86:0/2740015635 >> 172.21.15.86:0/2740015635 conn(0x55d19f9b7510 msgr2=0x55d19f9bde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.013+0000 7fbdf4599740 1 -- 172.21.15.86:0/2740015635 shutdown_connections 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.013+0000 7fbdf4599740 1 -- 172.21.15.86:0/2740015635 wait complete. 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.013+0000 7fbdf4599740 1 -- start start 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.013+0000 7fbdf4599740 1 Processor -- start 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 0x55d19f9cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdf4599740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdf4599740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1ae0f0 con 0x55d19f9c1b00 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdf4599740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1ada20 con 0x55d19f9bbfa0 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdf4599740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1aac50 con 0x55d19f9bc990 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdf4599740 10 client.0 _ll_register_callbacks cb 0x7fbdd80107d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdeb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdeb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40636/0 (socket says 172.21.15.86:40636) 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdeb400640 1 -- 172.21.15.86:0/3562162103 learned_addr learned my addr 172.21.15.86:0/3562162103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:34.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.014+0000 7fbdea000640 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 0x55d19f9cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:34.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdeb400640 1 -- 172.21.15.86:0/3562162103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 msgr2=0x55d19f9cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:34.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdeb400640 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 0x55d19f9cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdeb400640 1 -- 172.21.15.86:0/3562162103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 msgr2=0x55d19f9cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:34.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdeb400640 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdeb400640 1 -- 172.21.15.86:0/3562162103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d19f6fe7d0 con 0x55d19f9c1b00 2024-04-17T07:18:34.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdea000640 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 0x55d19f9cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdeb400640 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9ccb10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbde40beb60 tx=0x7fbde40bf340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd4fbb7b0625aae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdd4c00640 1 -- 172.21.15.86:0/3562162103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde40df250 con 0x55d19f9c1b00 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdd4c00640 1 -- 172.21.15.86:0/3562162103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbde40bee30 con 0x55d19f9c1b00 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdf4599740 1 -- 172.21.15.86:0/3562162103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde40bebd0 con 0x55d19f9c1b00 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdd4c00640 1 -- 172.21.15.86:0/3562162103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbde40c2f80 con 0x55d19f9c1b00 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdf4599740 10 client.16656 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.015+0000 7fbdf4599740 1 -- 172.21.15.86:0/3562162103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d19fa41140 con 0x55d19f9c1b00 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.016+0000 7fbdd4c00640 1 -- 172.21.15.86:0/3562162103 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbde40bebd0 con 0x55d19f9c1b00 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.016+0000 7fbdd4c00640 1 -- 172.21.15.86:0/3562162103 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fbde403c030 con 0x55d19f9c1b00 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.016+0000 7fbdf4599740 10 client.16656 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.016+0000 7fbdf4599740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.016+0000 7fbdf4599740 20 client.16656 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:34.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.016+0000 7fbdf4599740 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d19fa41970 0x55d19fa43d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:34.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.016+0000 7fbdf4599740 4 client.16656 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:18:34.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.016+0000 7fbdf4599740 1 -- 172.21.15.86:0/3562162103 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d19f9298b0 con 0x55d19fa41970 2024-04-17T07:18:34.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.017+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d19fa41970 0x55d19fa43d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:34.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.017+0000 7fbdeaa00640 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d19fa41970 0x55d19fa43d90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae78d613c47c9c4a server_cookie=bea56865c03f303b in_seq=0 out_seq=0 2024-04-17T07:18:34.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.017+0000 7fbdd4c00640 10 client.16656 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:34.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdd4c00640 1 -- 172.21.15.86:0/3562162103 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d19f9298b0 con 0x55d19fa41970 2024-04-17T07:18:34.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdd4c00640 10 client.16656 handle_command_reply: tid=0 2024-04-17T07:18:34.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdf4599740 1 client.16656 shutdown 2024-04-17T07:18:34.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdf4599740 1 -- 172.21.15.86:0/3562162103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 msgr2=0x55d19f9ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:34.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdf4599740 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9ccb10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbde40beb60 tx=0x7fbde40bf340 comp rx=0 tx=0).stop 2024-04-17T07:18:34.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdf4599740 1 -- 172.21.15.86:0/3562162103 shutdown_connections 2024-04-17T07:18:34.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdf4599740 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d19fa41970 0x55d19fa43d90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdf4599740 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d19f9c1b00 0x55d19f9ccb10 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdf4599740 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d19f9bc990 0x55d19f9cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdd4c00640 0 client.16656 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:34.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdf4599740 1 --2- 172.21.15.86:0/3562162103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d19f9bbfa0 0x55d19f9cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdf4599740 1 -- 172.21.15.86:0/3562162103 >> 172.21.15.86:0/3562162103 conn(0x55d19f9b7510 msgr2=0x55d19f9ba770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:34.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.018+0000 7fbdf4599740 1 -- 172.21.15.86:0/3562162103 shutdown_connections 2024-04-17T07:18:34.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.019+0000 7fbdf4599740 1 -- 172.21.15.86:0/3562162103 wait complete. 2024-04-17T07:18:34.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.019+0000 7fbdf4599740 20 client.16656 trim_cache size 0 max 16384 2024-04-17T07:18:34.021 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.019+0000 7fbdf1800640 1 -- 172.21.15.86:0/4120485715 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbdd00a3bb0 msgr2=0x7fbdd00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.019+0000 7fbdf1800640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbdd00a3bb0 0x7fbdd00a6070 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55d19f9bd1f0 tx=0x7fbdd8004730 comp rx=0 tx=0).stop 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.019+0000 7fbdf1800640 1 -- 172.21.15.86:0/4120485715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdec105d90 msgr2=0x7fbdec1a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.019+0000 7fbdf1800640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdec105d90 0x7fbdec1a18a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbde402c700 tx=0x7fbde40328a0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.019+0000 7fbdf1800640 1 -- 172.21.15.86:0/4120485715 shutdown_connections 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.019+0000 7fbdf1800640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbdd00a3bb0 0x7fbdd00a6070 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.019+0000 7fbdf1800640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbdec105d90 0x7fbdec1a18a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.020+0000 7fbdf1800640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdec1053d0 0x7fbdec19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.020+0000 7fbdf1800640 1 --2- 172.21.15.86:0/4120485715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdec1041e0 0x7fbdec19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.020+0000 7fbdf1800640 1 -- 172.21.15.86:0/4120485715 >> 172.21.15.86:0/4120485715 conn(0x7fbdec0ff750 msgr2=0x7fbdec1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:34.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.020+0000 7fbdf1800640 1 -- 172.21.15.86:0/4120485715 shutdown_connections 2024-04-17T07:18:34.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:34.020+0000 7fbdf1800640 1 -- 172.21.15.86:0/4120485715 wait complete. 2024-04-17T07:18:34.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:33 smithi103 ceph-mon[35495]: pgmap v566: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 MiB/s wr, 26 op/s 2024-04-17T07:18:35.040 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:18:35.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:18:35.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:18:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.296+0000 7f7c29600640 1 -- 172.21.15.86:0/2844626303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c241041e0 msgr2=0x7f7c24104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.296+0000 7f7c29600640 1 --2- 172.21.15.86:0/2844626303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c241041e0 0x7f7c24104600 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009900 tx=0x7f7c1402bc30 comp rx=0 tx=0).stop 2024-04-17T07:18:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.296+0000 7f7c29600640 1 -- 172.21.15.86:0/2844626303 shutdown_connections 2024-04-17T07:18:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.296+0000 7f7c29600640 1 --2- 172.21.15.86:0/2844626303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c24105d90 0x7f7c2410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.296+0000 7f7c29600640 1 --2- 172.21.15.86:0/2844626303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c241053d0 0x7f7c24105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.296+0000 7f7c29600640 1 --2- 172.21.15.86:0/2844626303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c241041e0 0x7f7c24104600 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.296+0000 7f7c29600640 1 -- 172.21.15.86:0/2844626303 >> 172.21.15.86:0/2844626303 conn(0x7f7c240ff750 msgr2=0x7f7c24101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.296+0000 7f7c29600640 1 -- 172.21.15.86:0/2844626303 shutdown_connections 2024-04-17T07:18:35.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.296+0000 7f7c29600640 1 -- 172.21.15.86:0/2844626303 wait complete. 2024-04-17T07:18:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.298+0000 7f7c29600640 1 Processor -- start 2024-04-17T07:18:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.298+0000 7f7c29600640 1 -- start start 2024-04-17T07:18:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.298+0000 7f7c29600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c241041e0 0x7f7c2419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.298+0000 7f7c29600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c241053d0 0x7f7c2419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.298+0000 7f7c29600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24105d90 0x7f7c241a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.298+0000 7f7c29600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2410fd70 con 0x7f7c24105d90 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.298+0000 7f7c29600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2410fbf0 con 0x7f7c241041e0 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.298+0000 7f7c29600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2410fef0 con 0x7f7c241053d0 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.299+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24105d90 0x7f7c241a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.299+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24105d90 0x7f7c241a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40668/0 (socket says 172.21.15.86:40668) 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.299+0000 7f7c23400640 1 -- 172.21.15.86:0/4013005665 learned_addr learned my addr 172.21.15.86:0/4013005665 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.299+0000 7f7c22a00640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c241041e0 0x7f7c2419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.299+0000 7f7c22000640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c241053d0 0x7f7c2419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.299+0000 7f7c23400640 1 -- 172.21.15.86:0/4013005665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c241053d0 msgr2=0x7f7c2419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.299+0000 7f7c23400640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c241053d0 0x7f7c2419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.299+0000 7f7c23400640 1 -- 172.21.15.86:0/4013005665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c241041e0 msgr2=0x7f7c2419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.299+0000 7f7c23400640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c241041e0 0x7f7c2419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.299+0000 7f7c23400640 1 -- 172.21.15.86:0/4013005665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c241a2070 con 0x7f7c24105d90 2024-04-17T07:18:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.300+0000 7f7c23400640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24105d90 0x7f7c241a1a00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00eaa0 tx=0x7f7c1c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.302+0000 7f7c1b400640 1 -- 172.21.15.86:0/4013005665 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00cdb0 con 0x7f7c24105d90 2024-04-17T07:18:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.302+0000 7f7c1b400640 1 -- 172.21.15.86:0/4013005665 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c1c005020 con 0x7f7c24105d90 2024-04-17T07:18:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.303+0000 7f7c29600640 1 -- 172.21.15.86:0/4013005665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c241a2360 con 0x7f7c24105d90 2024-04-17T07:18:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.303+0000 7f7c29600640 1 -- 172.21.15.86:0/4013005665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c241a28b0 con 0x7f7c24105d90 2024-04-17T07:18:35.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.303+0000 7f7c1b400640 1 -- 172.21.15.86:0/4013005665 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0116d0 con 0x7f7c24105d90 2024-04-17T07:18:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.306+0000 7f7c1b400640 1 -- 172.21.15.86:0/4013005665 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7c1c0051d0 con 0x7f7c24105d90 2024-04-17T07:18:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.306+0000 7f7c2c3e9740 1 Processor -- start 2024-04-17T07:18:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.307+0000 7f7c2c3e9740 1 -- start start 2024-04-17T07:18:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.307+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d133ec0 0x557f1d1342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.307+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d1348b0 0x557f1d1393b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.307+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 0x557f1d13beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.307+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f1d099cb0 con 0x557f1d1348b0 2024-04-17T07:18:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.307+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557f1d040370 con 0x557f1d139a20 2024-04-17T07:18:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.307+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557f1d095c70 con 0x557f1d133ec0 2024-04-17T07:18:35.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.307+0000 7f7c22a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d1348b0 0x557f1d1393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.307+0000 7f7c22a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d1348b0 0x557f1d1393b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40672/0 (socket says 172.21.15.86:40672) 2024-04-17T07:18:35.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22a00640 1 -- 172.21.15.86:0/3080029374 learned_addr learned my addr 172.21.15.86:0/3080029374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:35.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22000640 1 --2- 172.21.15.86:0/3080029374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d133ec0 0x557f1d1342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22000640 1 --2- 172.21.15.86:0/3080029374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 0x557f1d13beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22a00640 1 -- 172.21.15.86:0/3080029374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d133ec0 msgr2=0x557f1d1342e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22a00640 1 --2- 172.21.15.86:0/3080029374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d133ec0 0x557f1d1342e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22a00640 1 -- 172.21.15.86:0/3080029374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 msgr2=0x557f1d13beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22a00640 1 --2- 172.21.15.86:0/3080029374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 0x557f1d13beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22a00640 1 -- 172.21.15.86:0/3080029374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f1d05f910 con 0x557f1d1348b0 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c1b400640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7c080a3c30 0x7f7c080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22000640 1 --2- 172.21.15.86:0/3080029374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 0x557f1d13beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c1b400640 1 -- 172.21.15.86:0/4013005665 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7c1c0d0860 con 0x7f7c24105d90 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22a00640 1 --2- 172.21.15.86:0/3080029374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d1348b0 0x557f1d1393b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7c14000c00 tx=0x7f7c14005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d64a9fc022e569 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.308+0000 7f7c22000640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7c080a3c30 0x7f7c080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.309+0000 7f7c0f400640 1 -- 172.21.15.86:0/3080029374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14045020 con 0x557f1d1348b0 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.309+0000 7f7c0f400640 1 -- 172.21.15.86:0/3080029374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c1402c9e0 con 0x557f1d1348b0 2024-04-17T07:18:35.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.309+0000 7f7c0f400640 1 -- 172.21.15.86:0/3080029374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140046c0 con 0x557f1d1348b0 2024-04-17T07:18:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.310+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/3080029374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d1348b0 msgr2=0x557f1d1393b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.310+0000 7f7c22000640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7c080a3c30 0x7f7c080a60f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x557f1d135110 tx=0x7f7c1000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.310+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/3080029374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d1348b0 0x557f1d1393b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7c14000c00 tx=0x7f7c14005dd0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.310+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/3080029374 shutdown_connections 2024-04-17T07:18:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.310+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/3080029374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 0x557f1d13beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.310+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/3080029374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d1348b0 0x557f1d1393b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.310+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/3080029374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d133ec0 0x557f1d1342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.310+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/3080029374 >> 172.21.15.86:0/3080029374 conn(0x557f1d12f430 msgr2=0x557f1d12f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.310+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/3080029374 shutdown_connections 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.310+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/3080029374 wait complete. 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.311+0000 7f7c2c3e9740 1 -- start start 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.311+0000 7f7c2c3e9740 1 Processor -- start 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.311+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d13e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.311+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d1348b0 0x557f1d13e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.311+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 0x557f1d1458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.311+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f1d13c970 con 0x557f1d133ec0 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.312+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557f1d053730 con 0x557f1d139a20 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.312+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557f1d13caf0 con 0x557f1d1348b0 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.312+0000 7f7c2c3e9740 10 client.0 _ll_register_callbacks cb 0x557f1d055b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:35.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.312+0000 7f7c22a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d13e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.312+0000 7f7c22a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d13e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40680/0 (socket says 172.21.15.86:40680) 2024-04-17T07:18:35.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.312+0000 7f7c22a00640 1 -- 172.21.15.86:0/2003520615 learned_addr learned my addr 172.21.15.86:0/2003520615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:35.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.312+0000 7f7c22000640 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d1348b0 0x557f1d13e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.312+0000 7f7c23400640 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 0x557f1d1458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.313+0000 7f7c22a00640 1 -- 172.21.15.86:0/2003520615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d1348b0 msgr2=0x557f1d13e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.313+0000 7f7c22a00640 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d1348b0 0x557f1d13e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.313+0000 7f7c22a00640 1 -- 172.21.15.86:0/2003520615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 msgr2=0x557f1d1458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.313+0000 7f7c22a00640 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 0x557f1d1458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.313+0000 7f7c22a00640 1 -- 172.21.15.86:0/2003520615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f1ce747e0 con 0x557f1d133ec0 2024-04-17T07:18:35.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.313+0000 7f7c23400640 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 0x557f1d1458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:35.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.314+0000 7f7c22a00640 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d13e370 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7c1402c240 tx=0x7f7c140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2011a7fa4e43ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.314+0000 7f7c03e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.318+0000 7f7c03e00640 1 -- 172.21.15.86:0/2003520615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14045070 con 0x557f1d133ec0 2024-04-17T07:18:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.318+0000 7f7c03e00640 1 -- 172.21.15.86:0/2003520615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7c14003a00 con 0x557f1d133ec0 2024-04-17T07:18:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.319+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2003520615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1402c9e0 con 0x557f1d133ec0 2024-04-17T07:18:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.319+0000 7f7c03e00640 1 -- 172.21.15.86:0/2003520615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c14059050 con 0x557f1d133ec0 2024-04-17T07:18:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.319+0000 7f7c2c3e9740 10 client.16668 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.319+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2003520615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f1ced4f70 con 0x557f1d133ec0 2024-04-17T07:18:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.319+0000 7f7c03e00640 1 -- 172.21.15.86:0/2003520615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c1402c9e0 con 0x557f1d133ec0 2024-04-17T07:18:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.319+0000 7f7c03e00640 1 -- 172.21.15.86:0/2003520615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7c14054030 con 0x557f1d133ec0 2024-04-17T07:18:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.320+0000 7f7c2c3e9740 10 client.16668 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.320+0000 7f7c2c3e9740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.320+0000 7f7c2c3e9740 20 client.16668 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.320+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557f1d149d50 0x557f1d14c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.320+0000 7f7c2c3e9740 4 client.16668 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.320+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2003520615 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f1d0a76d0 con 0x557f1d149d50 2024-04-17T07:18:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.320+0000 7f7c22000640 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557f1d149d50 0x557f1d14c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.321+0000 7f7c22000640 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557f1d149d50 0x557f1d14c140 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=508198531669da2b server_cookie=2a84ed8b4ffb7a17 in_seq=0 out_seq=0 2024-04-17T07:18:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.321+0000 7f7c03e00640 10 client.16668 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.323+0000 7f7c03e00640 1 -- 172.21.15.86:0/2003520615 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f1d0a76d0 con 0x557f1d149d50 2024-04-17T07:18:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.323+0000 7f7c03e00640 10 client.16668 handle_command_reply: tid=0 2024-04-17T07:18:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.323+0000 7f7c2c3e9740 1 client.16668 shutdown 2024-04-17T07:18:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2003520615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 msgr2=0x557f1d13e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d13e370 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7c1402c240 tx=0x7f7c140042f0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2003520615 shutdown_connections 2024-04-17T07:18:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557f1d149d50 0x557f1d14c140 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d139a20 0x557f1d1458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d1348b0 0x557f1d13e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c03e00640 0 client.16668 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2003520615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d13e370 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2003520615 >> 172.21.15.86:0/2003520615 conn(0x557f1d12f430 msgr2=0x557f1d12e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2003520615 shutdown_connections 2024-04-17T07:18:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2003520615 wait complete. 2024-04-17T07:18:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.324+0000 7f7c2c3e9740 20 client.16668 trim_cache size 0 max 16384 2024-04-17T07:18:35.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.349+0000 7f7c2c3e9740 1 Processor -- start 2024-04-17T07:18:35.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.349+0000 7f7c2c3e9740 1 -- start start 2024-04-17T07:18:35.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.349+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d133ec0 0x557f1d13d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.349+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d1348b0 0x557f1d145810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.349+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d139a20 0x557f1d13ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.349+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241acca0 con 0x557f1d139a20 2024-04-17T07:18:35.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.349+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241ae7e0 con 0x557f1d133ec0 2024-04-17T07:18:35.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.349+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241ad370 con 0x557f1d1348b0 2024-04-17T07:18:35.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.350+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d139a20 0x557f1d13ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.350+0000 7f7c23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d139a20 0x557f1d13ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40690/0 (socket says 172.21.15.86:40690) 2024-04-17T07:18:35.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.350+0000 7f7c23400640 1 -- 172.21.15.86:0/3290436436 learned_addr learned my addr 172.21.15.86:0/3290436436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.350+0000 7f7c22000640 1 --2- 172.21.15.86:0/3290436436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d1348b0 0x557f1d145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.351+0000 7f7c23400640 1 -- 172.21.15.86:0/3290436436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d1348b0 msgr2=0x557f1d145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.351+0000 7f7c23400640 1 --2- 172.21.15.86:0/3290436436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d1348b0 0x557f1d145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.351+0000 7f7c23400640 1 -- 172.21.15.86:0/3290436436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d133ec0 msgr2=0x557f1d13d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.351+0000 7f7c23400640 1 --2- 172.21.15.86:0/3290436436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d133ec0 0x557f1d13d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.351+0000 7f7c23400640 1 -- 172.21.15.86:0/3290436436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f1d14c4b0 con 0x557f1d139a20 2024-04-17T07:18:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.351+0000 7f7c23400640 1 --2- 172.21.15.86:0/3290436436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d139a20 0x557f1d13ded0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0ce030 tx=0x7f7c1c09c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e63a894684b111 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.352+0000 7f7c0ea00640 1 -- 172.21.15.86:0/3290436436 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c01f040 con 0x557f1d139a20 2024-04-17T07:18:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.352+0000 7f7c0ea00640 1 -- 172.21.15.86:0/3290436436 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c1c09a910 con 0x557f1d139a20 2024-04-17T07:18:35.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.352+0000 7f7c0ea00640 1 -- 172.21.15.86:0/3290436436 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c099070 con 0x557f1d139a20 2024-04-17T07:18:35.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.353+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/3290436436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d139a20 msgr2=0x557f1d13ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.353+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/3290436436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d139a20 0x557f1d13ded0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0ce030 tx=0x7f7c1c09c930 comp rx=0 tx=0).stop 2024-04-17T07:18:35.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.353+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/3290436436 shutdown_connections 2024-04-17T07:18:35.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.353+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/3290436436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d139a20 0x557f1d13ded0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.353+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/3290436436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d1348b0 0x557f1d145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.353+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/3290436436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d133ec0 0x557f1d13d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.353+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/3290436436 >> 172.21.15.86:0/3290436436 conn(0x557f1d12f430 msgr2=0x557f1d12e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.353+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/3290436436 shutdown_connections 2024-04-17T07:18:35.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.353+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/3290436436 wait complete. 2024-04-17T07:18:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.354+0000 7f7c2c3e9740 1 -- start start 2024-04-17T07:18:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.354+0000 7f7c2c3e9740 1 Processor -- start 2024-04-17T07:18:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.355+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d145210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.355+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d1348b0 0x557f1d145750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.355+0000 7f7c2c3e9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d139a20 0x557f1d143820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.355+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241ae110 con 0x557f1d133ec0 2024-04-17T07:18:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.355+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241ada40 con 0x557f1d1348b0 2024-04-17T07:18:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.355+0000 7f7c2c3e9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c241aac70 con 0x557f1d139a20 2024-04-17T07:18:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.355+0000 7f7c2c3e9740 10 client.0 _ll_register_callbacks cb 0x7f7c14005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:35.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.356+0000 7f7c22a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.356+0000 7f7c22a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d145210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40700/0 (socket says 172.21.15.86:40700) 2024-04-17T07:18:35.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.356+0000 7f7c22a00640 1 -- 172.21.15.86:0/2403551455 learned_addr learned my addr 172.21.15.86:0/2403551455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.356+0000 7f7c22000640 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d1348b0 0x557f1d145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.356+0000 7f7c23400640 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d139a20 0x557f1d143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.357+0000 7f7c22a00640 1 -- 172.21.15.86:0/2403551455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d139a20 msgr2=0x557f1d143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.357+0000 7f7c22a00640 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d139a20 0x557f1d143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.357+0000 7f7c22a00640 1 -- 172.21.15.86:0/2403551455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d1348b0 msgr2=0x557f1d145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.357+0000 7f7c22a00640 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d1348b0 0x557f1d145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.357+0000 7f7c22a00640 1 -- 172.21.15.86:0/2403551455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f1cd6c170 con 0x557f1d133ec0 2024-04-17T07:18:35.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.357+0000 7f7c22a00640 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d145210 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7c14041d70 tx=0x7f7c1404c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e876aa6a6b8582df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.357+0000 7f7c03e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.357+0000 7f7c03e00640 1 -- 172.21.15.86:0/2403551455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14045040 con 0x557f1d133ec0 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.357+0000 7f7c03e00640 1 -- 172.21.15.86:0/2403551455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7c1402cae0 con 0x557f1d133ec0 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.357+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2403551455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1c09a910 con 0x557f1d133ec0 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c03e00640 1 -- 172.21.15.86:0/2403551455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c14066050 con 0x557f1d133ec0 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c2c3e9740 10 client.16674 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2403551455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f1d1b0ab0 con 0x557f1d133ec0 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c03e00640 1 -- 172.21.15.86:0/2403551455 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c1c09a910 con 0x557f1d133ec0 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c03e00640 1 -- 172.21.15.86:0/2403551455 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7c14054070 con 0x557f1d133ec0 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c2c3e9740 10 client.16674 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c2c3e9740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c2c3e9740 20 client.16674 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557f1d1b1490 0x557f1d1b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c2c3e9740 4 client.16674 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:18:35.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.358+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2403551455 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557f1d0a1540 con 0x557f1d1b1490 2024-04-17T07:18:35.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.359+0000 7f7c22000640 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557f1d1b1490 0x557f1d1b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.360+0000 7f7c03e00640 10 client.16674 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:35.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.360+0000 7f7c22000640 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557f1d1b1490 0x557f1d1b38b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0960d60d43c49f5 server_cookie=49a829e4d80d5bb3 in_seq=0 out_seq=0 2024-04-17T07:18:35.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c03e00640 1 -- 172.21.15.86:0/2403551455 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557f1d0a1540 con 0x557f1d1b1490 2024-04-17T07:18:35.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c03e00640 10 client.16674 handle_command_reply: tid=0 2024-04-17T07:18:35.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c2c3e9740 1 client.16674 shutdown 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2403551455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 msgr2=0x557f1d145210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d145210 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7c14041d70 tx=0x7f7c1404c710 comp rx=0 tx=0).stop 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2403551455 shutdown_connections 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557f1d1b1490 0x557f1d1b38b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f1d139a20 0x557f1d143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f1d1348b0 0x557f1d145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c2c3e9740 1 --2- 172.21.15.86:0/2403551455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f1d133ec0 0x557f1d145210 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2403551455 >> 172.21.15.86:0/2403551455 conn(0x557f1d12f430 msgr2=0x557f1d132690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.363+0000 7f7c03e00640 0 client.16674 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.364+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2403551455 shutdown_connections 2024-04-17T07:18:35.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.364+0000 7f7c2c3e9740 1 -- 172.21.15.86:0/2403551455 wait complete. 2024-04-17T07:18:35.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.364+0000 7f7c2c3e9740 20 client.16674 trim_cache size 0 max 16384 2024-04-17T07:18:35.365 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:18:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:18:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "f90350d8-70ad-4cb2-a96d-da9bfe62d80e", 2024-04-17T07:18:35.365 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:18:35.365 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:18:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 -- 172.21.15.86:0/4013005665 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7c080a3c30 msgr2=0x7f7c080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7c080a3c30 0x7f7c080a60f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x557f1d135110 tx=0x7f7c1000b390 comp rx=0 tx=0).stop 2024-04-17T07:18:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 -- 172.21.15.86:0/4013005665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24105d90 msgr2=0x7f7c241a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24105d90 0x7f7c241a1a00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00eaa0 tx=0x7f7c1c00ef60 comp rx=0 tx=0).stop 2024-04-17T07:18:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 -- 172.21.15.86:0/4013005665 shutdown_connections 2024-04-17T07:18:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7c080a3c30 0x7f7c080a60f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c24105d90 0x7f7c241a1a00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c241053d0 0x7f7c2419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 --2- 172.21.15.86:0/4013005665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c241041e0 0x7f7c2419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 -- 172.21.15.86:0/4013005665 >> 172.21.15.86:0/4013005665 conn(0x7f7c240ff750 msgr2=0x7f7c24108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 -- 172.21.15.86:0/4013005665 shutdown_connections 2024-04-17T07:18:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.365+0000 7f7c29600640 1 -- 172.21.15.86:0/4013005665 wait complete. 2024-04-17T07:18:35.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:18:35.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:18:35.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:35 smithi086 ceph-mon[31733]: pgmap v567: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 20 op/s 2024-04-17T07:18:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.621+0000 7f3a6ae00640 1 -- 172.21.15.86:0/3771894823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a64075f20 msgr2=0x7f3a64074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.621+0000 7f3a6ae00640 1 --2- 172.21.15.86:0/3771894823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a64075f20 0x7f3a64074370 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009900 tx=0x7f3a5402bcc0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.621+0000 7f3a6ae00640 1 -- 172.21.15.86:0/3771894823 shutdown_connections 2024-04-17T07:18:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.621+0000 7f3a6ae00640 1 --2- 172.21.15.86:0/3771894823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64109e60 0x7f3a6410c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.621+0000 7f3a6ae00640 1 --2- 172.21.15.86:0/3771894823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a64074940 0x7f3a64074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.621+0000 7f3a6ae00640 1 --2- 172.21.15.86:0/3771894823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a64075f20 0x7f3a64074370 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.621+0000 7f3a6ae00640 1 -- 172.21.15.86:0/3771894823 >> 172.21.15.86:0/3771894823 conn(0x7f3a640fd540 msgr2=0x7f3a640ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.621+0000 7f3a6ae00640 1 -- 172.21.15.86:0/3771894823 shutdown_connections 2024-04-17T07:18:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.621+0000 7f3a6ae00640 1 -- 172.21.15.86:0/3771894823 wait complete. 2024-04-17T07:18:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.622+0000 7f3a6ae00640 1 Processor -- start 2024-04-17T07:18:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.623+0000 7f3a6ae00640 1 -- start start 2024-04-17T07:18:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.623+0000 7f3a6ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a64074940 0x7f3a6419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.623+0000 7f3a6ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64075f20 0x7f3a6419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.623+0000 7f3a6ae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a64109e60 0x7f3a641a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.623+0000 7f3a6ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410fd90 con 0x7f3a64074940 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.623+0000 7f3a6ae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410fc10 con 0x7f3a64109e60 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.623+0000 7f3a6ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6410ff10 con 0x7f3a64075f20 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.623+0000 7f3a63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a64074940 0x7f3a6419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.623+0000 7f3a63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a64074940 0x7f3a6419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40712/0 (socket says 172.21.15.86:40712) 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.623+0000 7f3a63e00640 1 -- 172.21.15.86:0/2139476848 learned_addr learned my addr 172.21.15.86:0/2139476848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.624+0000 7f3a68e00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a64109e60 0x7f3a641a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.624+0000 7f3a63400640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64075f20 0x7f3a6419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.624+0000 7f3a63e00640 1 -- 172.21.15.86:0/2139476848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64075f20 msgr2=0x7f3a6419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.624+0000 7f3a63e00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64075f20 0x7f3a6419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.624+0000 7f3a63e00640 1 -- 172.21.15.86:0/2139476848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a64109e60 msgr2=0x7f3a641a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.624+0000 7f3a63e00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a64109e60 0x7f3a641a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.624+0000 7f3a63e00640 1 -- 172.21.15.86:0/2139476848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a641a20a0 con 0x7f3a64074940 2024-04-17T07:18:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.624+0000 7f3a63e00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a64074940 0x7f3a6419a480 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3a54000c00 tx=0x7f3a5402d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.625+0000 7f3a60c00640 1 -- 172.21.15.86:0/2139476848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54043020 con 0x7f3a64074940 2024-04-17T07:18:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.625+0000 7f3a60c00640 1 -- 172.21.15.86:0/2139476848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a5402dd80 con 0x7f3a64074940 2024-04-17T07:18:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.625+0000 7f3a6ae00640 1 -- 172.21.15.86:0/2139476848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a641a2330 con 0x7f3a64074940 2024-04-17T07:18:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.625+0000 7f3a60c00640 1 -- 172.21.15.86:0/2139476848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54003bb0 con 0x7f3a64074940 2024-04-17T07:18:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.625+0000 7f3a6ae00640 1 -- 172.21.15.86:0/2139476848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a641a2900 con 0x7f3a64074940 2024-04-17T07:18:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.629+0000 7f3a6da7e740 1 Processor -- start 2024-04-17T07:18:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.630+0000 7f3a60c00640 1 -- 172.21.15.86:0/2139476848 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3a54046020 con 0x7f3a64074940 2024-04-17T07:18:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.630+0000 7f3a6da7e740 1 -- start start 2024-04-17T07:18:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.630+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.630+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 0x564c594f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.630+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c594f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.630+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564c59458120 con 0x564c594f6cf0 2024-04-17T07:18:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.630+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564c593fe340 con 0x564c594f1190 2024-04-17T07:18:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.630+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564c594540e0 con 0x564c594f1b80 2024-04-17T07:18:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c594f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a68e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a68e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594f15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:32856/0 (socket says 172.21.15.86:32856) 2024-04-17T07:18:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a68e00640 1 -- 172.21.15.86:0/544345909 learned_addr learned my addr 172.21.15.86:0/544345909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a63e00640 1 --2- 172.21.15.86:0/544345909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 0x564c594f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a68e00640 1 -- 172.21.15.86:0/544345909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 msgr2=0x564c594f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a68e00640 1 --2- 172.21.15.86:0/544345909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 0x564c594f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a68e00640 1 -- 172.21.15.86:0/544345909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 msgr2=0x564c594f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a68e00640 1 --2- 172.21.15.86:0/544345909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c594f9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a68e00640 1 -- 172.21.15.86:0/544345909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c592347d0 con 0x564c594f1190 2024-04-17T07:18:35.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a63e00640 1 --2- 172.21.15.86:0/544345909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 0x564c594f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:18:35.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.631+0000 7f3a63400640 1 --2- 172.21.15.86:0/544345909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c594f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:18:35.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.632+0000 7f3a68e00640 1 --2- 172.21.15.86:0/544345909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594f15b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00a430 tx=0x7f3a5c009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e98b2318b5317046 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.632+0000 7f3a60c00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a480a3c30 0x7f3a480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.632+0000 7f3a59600640 1 -- 172.21.15.86:0/544345909 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c013790 con 0x564c594f1190 2024-04-17T07:18:35.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.633+0000 7f3a59600640 1 -- 172.21.15.86:0/544345909 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a5c019070 con 0x564c594f1190 2024-04-17T07:18:35.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.633+0000 7f3a60c00640 1 -- 172.21.15.86:0/2139476848 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3a540f2a40 con 0x7f3a64074940 2024-04-17T07:18:35.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.633+0000 7f3a63e00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a480a3c30 0x7f3a480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.633+0000 7f3a59600640 1 -- 172.21.15.86:0/544345909 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c014ce0 con 0x564c594f1190 2024-04-17T07:18:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.634+0000 7f3a63e00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a480a3c30 0x7f3a480a60f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3a540bb310 tx=0x7f3a540df040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.634+0000 7f3a6da7e740 1 -- 172.21.15.86:0/544345909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 msgr2=0x564c594f15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.634+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/544345909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594f15b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00a430 tx=0x7f3a5c009650 comp rx=0 tx=0).stop 2024-04-17T07:18:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.634+0000 7f3a6da7e740 1 -- 172.21.15.86:0/544345909 shutdown_connections 2024-04-17T07:18:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.634+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/544345909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c594f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.634+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/544345909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 0x564c594f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.634+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/544345909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594f15b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.634+0000 7f3a6da7e740 1 -- 172.21.15.86:0/544345909 >> 172.21.15.86:0/544345909 conn(0x564c594ec700 msgr2=0x564c594ecb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.634+0000 7f3a6da7e740 1 -- 172.21.15.86:0/544345909 shutdown_connections 2024-04-17T07:18:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.634+0000 7f3a6da7e740 1 -- 172.21.15.86:0/544345909 wait complete. 2024-04-17T07:18:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.635+0000 7f3a6da7e740 1 -- start start 2024-04-17T07:18:35.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.635+0000 7f3a6da7e740 1 Processor -- start 2024-04-17T07:18:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.635+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.635+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c594fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.635+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f6cf0 0x564c59502c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.635+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564c594f9cf0 con 0x564c594f1b80 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.635+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564c59411710 con 0x564c594f1190 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.635+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564c594f9e70 con 0x564c594f6cf0 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a6da7e740 10 client.0 _ll_register_callbacks cb 0x564c5940caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c594fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c594fbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40734/0 (socket says 172.21.15.86:40734) 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a63400640 1 -- 172.21.15.86:0/3158325413 learned_addr learned my addr 172.21.15.86:0/3158325413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a68e00640 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a68e00640 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f6cf0 0x564c59502c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a63400640 1 -- 172.21.15.86:0/3158325413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f6cf0 msgr2=0x564c59502c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a63400640 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f6cf0 0x564c59502c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a63400640 1 -- 172.21.15.86:0/3158325413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 msgr2=0x564c594fb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a63400640 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594fb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a63400640 1 -- 172.21.15.86:0/3158325413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c5916a3f0 con 0x564c594f1b80 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.636+0000 7f3a68e00640 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594fb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.637+0000 7f3a63400640 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c594fbc20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3a50012700 tx=0x7f3a50012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b618322d04b2edb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.637+0000 7f3a4e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.637+0000 7f3a4e000640 1 -- 172.21.15.86:0/3158325413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50004320 con 0x564c594f1b80 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.637+0000 7f3a4e000640 1 -- 172.21.15.86:0/3158325413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3a50011940 con 0x564c594f1b80 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.637+0000 7f3a6da7e740 1 -- 172.21.15.86:0/3158325413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a5c019070 con 0x564c594f1b80 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.637+0000 7f3a4e000640 1 -- 172.21.15.86:0/3158325413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a50025050 con 0x564c594f1b80 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.637+0000 7f3a6da7e740 10 client.16686 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.637+0000 7f3a6da7e740 1 -- 172.21.15.86:0/3158325413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c5912a0f0 con 0x564c594f1b80 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.638+0000 7f3a4e000640 1 -- 172.21.15.86:0/3158325413 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a5c019070 con 0x564c594f1b80 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.638+0000 7f3a4e000640 1 -- 172.21.15.86:0/3158325413 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3a50003ce0 con 0x564c594f1b80 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.638+0000 7f3a6da7e740 10 client.16686 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:35.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.638+0000 7f3a6da7e740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.638+0000 7f3a6da7e740 20 client.16686 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.638+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564c595074f0 0x564c59509970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.638+0000 7f3a6da7e740 4 client.16686 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:18:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.638+0000 7f3a6da7e740 1 -- 172.21.15.86:0/3158325413 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c59465b40 con 0x564c595074f0 2024-04-17T07:18:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.639+0000 7f3a68e00640 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564c595074f0 0x564c59509970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.639+0000 7f3a68e00640 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564c595074f0 0x564c59509970 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbece7f50d3039c6 server_cookie=c71fb3ac481e5075 in_seq=0 out_seq=0 2024-04-17T07:18:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.639+0000 7f3a4e000640 10 client.16686 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.640+0000 7f3a4e000640 1 -- 172.21.15.86:0/3158325413 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c59465b40 con 0x564c595074f0 2024-04-17T07:18:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.640+0000 7f3a4e000640 10 client.16686 handle_command_reply: tid=0 2024-04-17T07:18:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.640+0000 7f3a6da7e740 1 client.16686 shutdown 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.640+0000 7f3a6da7e740 1 -- 172.21.15.86:0/3158325413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 msgr2=0x564c594fbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.640+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c594fbc20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3a50012700 tx=0x7f3a50012bc0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.640+0000 7f3a6da7e740 1 -- 172.21.15.86:0/3158325413 shutdown_connections 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.641+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564c595074f0 0x564c59509970 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.641+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f6cf0 0x564c59502c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.641+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c594fbc20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.641+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/3158325413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594fb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.641+0000 7f3a6da7e740 1 -- 172.21.15.86:0/3158325413 >> 172.21.15.86:0/3158325413 conn(0x564c594ec700 msgr2=0x564c594eb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.641+0000 7f3a4e000640 0 client.16686 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.641+0000 7f3a6da7e740 1 -- 172.21.15.86:0/3158325413 shutdown_connections 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.641+0000 7f3a6da7e740 1 -- 172.21.15.86:0/3158325413 wait complete. 2024-04-17T07:18:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.641+0000 7f3a6da7e740 20 client.16686 trim_cache size 0 max 16384 2024-04-17T07:18:35.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.662+0000 7f3a6da7e740 1 Processor -- start 2024-04-17T07:18:35.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.662+0000 7f3a6da7e740 1 -- start start 2024-04-17T07:18:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594e7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c59502c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f6cf0 0x564c594eb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641acd90 con 0x564c594f1b80 2024-04-17T07:18:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641ae8d0 con 0x564c594f1190 2024-04-17T07:18:35.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641ad460 con 0x564c594f6cf0 2024-04-17T07:18:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c59502c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a68e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a68e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594e7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:32878/0 (socket says 172.21.15.86:32878) 2024-04-17T07:18:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a68e00640 1 -- 172.21.15.86:0/4289360498 learned_addr learned my addr 172.21.15.86:0/4289360498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.663+0000 7f3a63400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c59502c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40748/0 (socket says 172.21.15.86:40748) 2024-04-17T07:18:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.664+0000 7f3a68e00640 1 --2- 172.21.15.86:0/4289360498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f6cf0 0x564c594eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.664+0000 7f3a68e00640 1 -- 172.21.15.86:0/4289360498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f6cf0 msgr2=0x564c594eb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.664+0000 7f3a68e00640 1 --2- 172.21.15.86:0/4289360498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f6cf0 0x564c594eb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.664+0000 7f3a68e00640 1 -- 172.21.15.86:0/4289360498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 msgr2=0x564c59502c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.664+0000 7f3a68e00640 1 --2- 172.21.15.86:0/4289360498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c59502c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.664+0000 7f3a68e00640 1 -- 172.21.15.86:0/4289360498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c594e6ce0 con 0x564c594f1190 2024-04-17T07:18:35.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.664+0000 7f3a68e00640 1 --2- 172.21.15.86:0/4289360498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594e7720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c014600 tx=0x7f3a5c004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1edd71653d0a1927 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.665+0000 7f3a58c00640 1 -- 172.21.15.86:0/4289360498 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c029e60 con 0x564c594f1190 2024-04-17T07:18:35.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.665+0000 7f3a58c00640 1 -- 172.21.15.86:0/4289360498 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a5c019030 con 0x564c594f1190 2024-04-17T07:18:35.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.665+0000 7f3a58c00640 1 -- 172.21.15.86:0/4289360498 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0137f0 con 0x564c594f1190 2024-04-17T07:18:35.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 -- 172.21.15.86:0/4289360498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 msgr2=0x564c594e7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/4289360498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594e7720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c014600 tx=0x7f3a5c004b50 comp rx=0 tx=0).stop 2024-04-17T07:18:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 -- 172.21.15.86:0/4289360498 shutdown_connections 2024-04-17T07:18:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/4289360498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f6cf0 0x564c594eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/4289360498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f1b80 0x564c59502c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/4289360498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594e7720 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 -- 172.21.15.86:0/4289360498 >> 172.21.15.86:0/4289360498 conn(0x564c594ec700 msgr2=0x564c594f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 -- 172.21.15.86:0/4289360498 shutdown_connections 2024-04-17T07:18:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 -- 172.21.15.86:0/4289360498 wait complete. 2024-04-17T07:18:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 -- start start 2024-04-17T07:18:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.666+0000 7f3a6da7e740 1 Processor -- start 2024-04-17T07:18:35.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.667+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.667+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 0x564c59500500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.667+0000 7f3a6da7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c59501510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.667+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641ae200 con 0x564c594f6cf0 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.667+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641adb30 con 0x564c594f1190 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.667+0000 7f3a6da7e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641aad60 con 0x564c594f1b80 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.667+0000 7f3a6da7e740 10 client.0 _ll_register_callbacks cb 0x7f3a50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.667+0000 7f3a68e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c59501510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.667+0000 7f3a68e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c59501510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40758/0 (socket says 172.21.15.86:40758) 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.667+0000 7f3a68e00640 1 -- 172.21.15.86:0/2270095518 learned_addr learned my addr 172.21.15.86:0/2270095518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.668+0000 7f3a63400640 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 0x564c59500500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.668+0000 7f3a68e00640 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.668+0000 7f3a68e00640 1 -- 172.21.15.86:0/2270095518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 msgr2=0x564c59500500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.668+0000 7f3a68e00640 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 0x564c59500500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.668+0000 7f3a68e00640 1 -- 172.21.15.86:0/2270095518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 msgr2=0x564c594fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.668+0000 7f3a68e00640 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.668+0000 7f3a68e00640 1 -- 172.21.15.86:0/2270095518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c59501b00 con 0x564c594f6cf0 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.668+0000 7f3a68e00640 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c59501510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c004a80 tx=0x7f3a5c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88fd3a5f2e4f115 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.668+0000 7f3a4e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.669+0000 7f3a4e000640 1 -- 172.21.15.86:0/2270095518 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c029e60 con 0x564c594f6cf0 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.669+0000 7f3a4e000640 1 -- 172.21.15.86:0/2270095518 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3a5c014830 con 0x564c594f6cf0 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.669+0000 7f3a6da7e740 1 -- 172.21.15.86:0/2270095518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a5c019030 con 0x564c594f6cf0 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.669+0000 7f3a4e000640 1 -- 172.21.15.86:0/2270095518 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a5c03d050 con 0x564c594f6cf0 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.669+0000 7f3a6da7e740 10 client.16692 fetch_fsmap learned FSMap version 28 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.669+0000 7f3a6da7e740 1 -- 172.21.15.86:0/2270095518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c59502e10 con 0x564c594f6cf0 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.669+0000 7f3a4e000640 1 -- 172.21.15.86:0/2270095518 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a5c019030 con 0x564c594f6cf0 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.670+0000 7f3a4e000640 1 -- 172.21.15.86:0/2270095518 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3a5c013ca0 con 0x564c594f6cf0 2024-04-17T07:18:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.670+0000 7f3a6da7e740 10 client.16692 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:18:35.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.670+0000 7f3a6da7e740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:18:35.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.670+0000 7f3a6da7e740 20 client.16692 populate_metadata read hostname 'smithi086' 2024-04-17T07:18:35.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.670+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564c5956df80 0x564c595703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:18:35.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.670+0000 7f3a6da7e740 4 client.16692 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:18:35.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.670+0000 7f3a6da7e740 1 -- 172.21.15.86:0/2270095518 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c5945f9b0 con 0x564c5956df80 2024-04-17T07:18:35.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.670+0000 7f3a63400640 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564c5956df80 0x564c595703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:18:35.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.671+0000 7f3a63400640 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564c5956df80 0x564c595703a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d01aaa4dcf04dd server_cookie=96c005cfd69458 in_seq=0 out_seq=0 2024-04-17T07:18:35.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.671+0000 7f3a4e000640 10 client.16692 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:35.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.671+0000 7f3a4e000640 1 -- 172.21.15.86:0/2270095518 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564c5945f9b0 con 0x564c5956df80 2024-04-17T07:18:35.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.671+0000 7f3a4e000640 10 client.16692 handle_command_reply: tid=0 2024-04-17T07:18:35.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.671+0000 7f3a6da7e740 1 client.16692 shutdown 2024-04-17T07:18:35.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 1 -- 172.21.15.86:0/2270095518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 msgr2=0x564c59501510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c59501510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c004a80 tx=0x7f3a5c00af90 comp rx=0 tx=0).stop 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 1 -- 172.21.15.86:0/2270095518 shutdown_connections 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564c5956df80 0x564c595703a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564c594f6cf0 0x564c59501510 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564c594f1b80 0x564c59500500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 1 --2- 172.21.15.86:0/2270095518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564c594f1190 0x564c594fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a4e000640 0 client.16692 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 1 -- 172.21.15.86:0/2270095518 >> 172.21.15.86:0/2270095518 conn(0x564c594ec700 msgr2=0x564c594eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 1 -- 172.21.15.86:0/2270095518 shutdown_connections 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 1 -- 172.21.15.86:0/2270095518 wait complete. 2024-04-17T07:18:35.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.672+0000 7f3a6da7e740 20 client.16692 trim_cache size 0 max 16384 2024-04-17T07:18:35.675 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:18:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:18:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:18:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "f90350d8-70ad-4cb2-a96d-da9bfe62d80e": { 2024-04-17T07:18:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:18:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "f90350d8-70ad-4cb2-a96d-da9bfe62d80e", 2024-04-17T07:18:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:18:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:18:35.675 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:18:35.675 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 -- 172.21.15.86:0/2139476848 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a480a3c30 msgr2=0x7f3a480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a480a3c30 0x7f3a480a60f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3a540bb310 tx=0x7f3a540df040 comp rx=0 tx=0).stop 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 -- 172.21.15.86:0/2139476848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a64074940 msgr2=0x7f3a6419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a64074940 0x7f3a6419a480 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3a54000c00 tx=0x7f3a5402d980 comp rx=0 tx=0).stop 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 -- 172.21.15.86:0/2139476848 shutdown_connections 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a480a3c30 0x7f3a480a60f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a64109e60 0x7f3a641a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a64075f20 0x7f3a6419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 --2- 172.21.15.86:0/2139476848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a64074940 0x7f3a6419a480 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 -- 172.21.15.86:0/2139476848 >> 172.21.15.86:0/2139476848 conn(0x7f3a640fd540 msgr2=0x7f3a64105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:18:35.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 -- 172.21.15.86:0/2139476848 shutdown_connections 2024-04-17T07:18:35.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:18:35.673+0000 7f3a6ae00640 1 -- 172.21.15.86:0/2139476848 wait complete. 2024-04-17T07:18:35.692 INFO:tasks.cephfs.filesystem:scrub status for tag:f90350d8-70ad-4cb2-a96d-da9bfe62d80e - {'path': '/', 'tag': 'f90350d8-70ad-4cb2-a96d-da9bfe62d80e', 'options': 'recursive,force'} 2024-04-17T07:18:35.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:35 smithi098 ceph-mon[35418]: pgmap v567: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 20 op/s 2024-04-17T07:18:35.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:35 smithi103 ceph-mon[35495]: pgmap v567: 129 pgs: 129 active+clean; 782 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 20 op/s 2024-04-17T07:18:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:36 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:18:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:36 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:36 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:18:36.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:36 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:36 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:18:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:36 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:36 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:18:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:36 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:36.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:36 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:18:36.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:36 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:36.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:36 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:18:36.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:36 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:18:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:37 smithi086 ceph-mon[31733]: pgmap v568: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-04-17T07:18:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:37 smithi098 ceph-mon[35418]: pgmap v568: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-04-17T07:18:37.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:37 smithi103 ceph-mon[35495]: pgmap v568: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-04-17T07:18:39.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:39 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:18:39.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:39 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:18:39.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:39 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:18:40.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:40 smithi086 ceph-mon[31733]: pgmap v569: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:18:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:40 smithi098 ceph-mon[35418]: pgmap v569: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:18:40.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:40 smithi103 ceph-mon[35495]: pgmap v569: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 19 op/s 2024-04-17T07:18:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:41 smithi086 ceph-mon[31733]: pgmap v570: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 17 op/s 2024-04-17T07:18:41.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:41 smithi098 ceph-mon[35418]: pgmap v570: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 17 op/s 2024-04-17T07:18:41.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:41 smithi103 ceph-mon[35495]: pgmap v570: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 17 op/s 2024-04-17T07:18:43.579 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:43 smithi103 ceph-mon[35495]: pgmap v571: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-04-17T07:18:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:43 smithi086 ceph-mon[31733]: pgmap v571: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-04-17T07:18:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:43 smithi098 ceph-mon[35418]: pgmap v571: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-04-17T07:18:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:45 smithi086 ceph-mon[31733]: pgmap v572: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 16 op/s 2024-04-17T07:18:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:45 smithi098 ceph-mon[35418]: pgmap v572: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 16 op/s 2024-04-17T07:18:45.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:45 smithi103 ceph-mon[35495]: pgmap v572: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 16 op/s 2024-04-17T07:18:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:47 smithi086 ceph-mon[31733]: pgmap v573: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:18:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:47 smithi098 ceph-mon[35418]: pgmap v573: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:18:47.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:47 smithi103 ceph-mon[35495]: pgmap v573: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 21 op/s 2024-04-17T07:18:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:49 smithi086 ceph-mon[31733]: pgmap v574: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.7 MiB/s wr, 18 op/s 2024-04-17T07:18:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:49 smithi098 ceph-mon[35418]: pgmap v574: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.7 MiB/s wr, 18 op/s 2024-04-17T07:18:49.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:49 smithi103 ceph-mon[35495]: pgmap v574: 129 pgs: 129 active+clean; 782 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.7 MiB/s wr, 18 op/s 2024-04-17T07:18:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:51 smithi086 ceph-mon[31733]: pgmap v575: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-04-17T07:18:51.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:51 smithi098 ceph-mon[35418]: pgmap v575: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-04-17T07:18:51.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:51 smithi103 ceph-mon[35495]: pgmap v575: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-04-17T07:18:53.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:53 smithi086 ceph-mon[31733]: pgmap v576: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 22 op/s 2024-04-17T07:18:53.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:53 smithi098 ceph-mon[35418]: pgmap v576: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 22 op/s 2024-04-17T07:18:53.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:53 smithi103 ceph-mon[35495]: pgmap v576: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 22 op/s 2024-04-17T07:18:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:55 smithi086 ceph-mon[31733]: pgmap v577: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.7 MiB/s wr, 15 op/s 2024-04-17T07:18:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:55 smithi098 ceph-mon[35418]: pgmap v577: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.7 MiB/s wr, 15 op/s 2024-04-17T07:18:55.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:55 smithi103 ceph-mon[35495]: pgmap v577: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.7 MiB/s wr, 15 op/s 2024-04-17T07:18:57.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:57 smithi086 ceph-mon[31733]: pgmap v578: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:18:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:57 smithi098 ceph-mon[35418]: pgmap v578: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:18:57.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:57 smithi103 ceph-mon[35495]: pgmap v578: 129 pgs: 129 active+clean; 782 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 MiB/s wr, 20 op/s 2024-04-17T07:18:59.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:18:59 smithi086 ceph-mon[31733]: pgmap v579: 129 pgs: 129 active+clean; 790 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.4 MiB/s wr, 17 op/s 2024-04-17T07:18:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:18:59 smithi098 ceph-mon[35418]: pgmap v579: 129 pgs: 129 active+clean; 790 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.4 MiB/s wr, 17 op/s 2024-04-17T07:18:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:18:59 smithi103 ceph-mon[35495]: pgmap v579: 129 pgs: 129 active+clean; 790 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.4 MiB/s wr, 17 op/s 2024-04-17T07:19:01.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:01 smithi086 ceph-mon[31733]: pgmap v580: 129 pgs: 129 active+clean; 794 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.4 MiB/s wr, 17 op/s 2024-04-17T07:19:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:01 smithi098 ceph-mon[35418]: pgmap v580: 129 pgs: 129 active+clean; 794 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.4 MiB/s wr, 17 op/s 2024-04-17T07:19:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:01 smithi103 ceph-mon[35495]: pgmap v580: 129 pgs: 129 active+clean; 794 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.4 MiB/s wr, 17 op/s 2024-04-17T07:19:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:03 smithi086 ceph-mon[31733]: pgmap v581: 129 pgs: 129 active+clean; 799 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:19:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:03 smithi098 ceph-mon[35418]: pgmap v581: 129 pgs: 129 active+clean; 799 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:19:03.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:03 smithi103 ceph-mon[35495]: pgmap v581: 129 pgs: 129 active+clean; 799 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-04-17T07:19:04.771 INFO:teuthology.orchestra.run.smithi086.stderr:creating primary keys... 2024-04-17T07:19:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:05 smithi086 ceph-mon[31733]: pgmap v582: 129 pgs: 129 active+clean; 799 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 MiB/s wr, 15 op/s 2024-04-17T07:19:05.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:19:05.694 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:19:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:05 smithi098 ceph-mon[35418]: pgmap v582: 129 pgs: 129 active+clean; 799 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 MiB/s wr, 15 op/s 2024-04-17T07:19:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:05 smithi103 ceph-mon[35495]: pgmap v582: 129 pgs: 129 active+clean; 799 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 MiB/s wr, 15 op/s 2024-04-17T07:19:05.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.941+0000 7f4068c00640 1 -- 172.21.15.86:0/3781922294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4064101ea0 msgr2=0x7f40641022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:05.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.941+0000 7f4068c00640 1 --2- 172.21.15.86:0/3781922294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4064101ea0 0x7f40641022c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4054009900 tx=0x7f405402bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.942+0000 7f4068c00640 1 -- 172.21.15.86:0/3781922294 shutdown_connections 2024-04-17T07:19:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.942+0000 7f4068c00640 1 --2- 172.21.15.86:0/3781922294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064103a40 0x7f406410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.942+0000 7f4068c00640 1 --2- 172.21.15.86:0/3781922294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4064103080 0x7f4064103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.942+0000 7f4068c00640 1 --2- 172.21.15.86:0/3781922294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4064101ea0 0x7f40641022c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.942+0000 7f4068c00640 1 -- 172.21.15.86:0/3781922294 >> 172.21.15.86:0/3781922294 conn(0x7f40640fd460 msgr2=0x7f40640ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.942+0000 7f4068c00640 1 -- 172.21.15.86:0/3781922294 shutdown_connections 2024-04-17T07:19:05.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.942+0000 7f4068c00640 1 -- 172.21.15.86:0/3781922294 wait complete. 2024-04-17T07:19:05.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.943+0000 7f4068c00640 1 Processor -- start 2024-04-17T07:19:05.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.943+0000 7f4068c00640 1 -- start start 2024-04-17T07:19:05.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.943+0000 7f4068c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4064101ea0 0x7f406419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.944+0000 7f4068c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064103080 0x7f406419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.944+0000 7f4068c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4064103a40 0x7f40641a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.944+0000 7f4068c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f406410d9c0 con 0x7f4064103a40 2024-04-17T07:19:05.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.944+0000 7f4068c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f406410d840 con 0x7f4064101ea0 2024-04-17T07:19:05.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.944+0000 7f4068c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f406410db40 con 0x7f4064103080 2024-04-17T07:19:05.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.944+0000 7f4063e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4064103a40 0x7f40641a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.944+0000 7f4063e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4064103a40 0x7f40641a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41304/0 (socket says 172.21.15.86:41304) 2024-04-17T07:19:05.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.944+0000 7f4063e00640 1 -- 172.21.15.86:0/3118206150 learned_addr learned my addr 172.21.15.86:0/3118206150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:05.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.944+0000 7f4062a00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064103080 0x7f406419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.944+0000 7f4063400640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4064101ea0 0x7f406419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.945+0000 7f4063e00640 1 -- 172.21.15.86:0/3118206150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064103080 msgr2=0x7f406419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.945+0000 7f4063e00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064103080 0x7f406419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.945+0000 7f4063e00640 1 -- 172.21.15.86:0/3118206150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4064101ea0 msgr2=0x7f406419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.945+0000 7f4063e00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4064101ea0 0x7f406419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.945+0000 7f4063e00640 1 -- 172.21.15.86:0/3118206150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40641a1f50 con 0x7f4064103a40 2024-04-17T07:19:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.945+0000 7f4062a00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064103080 0x7f406419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.945+0000 7f4063e00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4064103a40 0x7f40641a18e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f405c00cad0 tx=0x7f405c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.945+0000 7f405be00640 1 -- 172.21.15.86:0/3118206150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c00ae20 con 0x7f4064103a40 2024-04-17T07:19:05.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.945+0000 7f405be00640 1 -- 172.21.15.86:0/3118206150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f405c005020 con 0x7f4064103a40 2024-04-17T07:19:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.946+0000 7f4068c00640 1 -- 172.21.15.86:0/3118206150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4064104390 con 0x7f4064103a40 2024-04-17T07:19:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.946+0000 7f405be00640 1 -- 172.21.15.86:0/3118206150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c00f6b0 con 0x7f4064103a40 2024-04-17T07:19:05.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.946+0000 7f4068c00640 1 -- 172.21.15.86:0/3118206150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4064104930 con 0x7f4064103a40 2024-04-17T07:19:05.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.950+0000 7f406ce11740 1 Processor -- start 2024-04-17T07:19:05.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.950+0000 7f405be00640 1 -- 172.21.15.86:0/3118206150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f405c0051d0 con 0x7f4064103a40 2024-04-17T07:19:05.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.951+0000 7f406ce11740 1 -- start start 2024-04-17T07:19:05.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.951+0000 7f406ce11740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.951+0000 7f406ce11740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997ed60 0x5582c9983860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.951+0000 7f406ce11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 0x5582c9986360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.951+0000 7f406ce11740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5582c98e5120 con 0x5582c997e370 2024-04-17T07:19:05.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.951+0000 7f406ce11740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5582c988b340 con 0x5582c997ed60 2024-04-17T07:19:05.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.951+0000 7f406ce11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5582c98e10e0 con 0x5582c9983ed0 2024-04-17T07:19:05.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.952+0000 7f4062a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.952+0000 7f4062a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41316/0 (socket says 172.21.15.86:41316) 2024-04-17T07:19:05.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.952+0000 7f4062a00640 1 -- 172.21.15.86:0/58779765 learned_addr learned my addr 172.21.15.86:0/58779765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:05.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.952+0000 7f4062a00640 1 --2- 172.21.15.86:0/58779765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 0x5582c9986360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.952+0000 7f4062a00640 1 -- 172.21.15.86:0/58779765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 msgr2=0x5582c9986360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.952+0000 7f4062a00640 1 --2- 172.21.15.86:0/58779765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 0x5582c9986360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.953+0000 7f4062a00640 1 -- 172.21.15.86:0/58779765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997ed60 msgr2=0x5582c9983860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.953+0000 7f4062a00640 1 --2- 172.21.15.86:0/58779765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997ed60 0x5582c9983860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.953+0000 7f4062a00640 1 -- 172.21.15.86:0/58779765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c96c17d0 con 0x5582c997e370 2024-04-17T07:19:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.953+0000 7f405be00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40480a3c30 0x7f40480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.953+0000 7f405be00640 1 -- 172.21.15.86:0/3118206150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f405c0d07f0 con 0x7f4064103a40 2024-04-17T07:19:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.953+0000 7f4062a00640 1 --2- 172.21.15.86:0/58779765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997e790 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f405000be80 tx=0x7f405000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76d7beccc1386f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.953+0000 7f4062a00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40480a3c30 0x7f40480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.954+0000 7f404fe00640 1 -- 172.21.15.86:0/58779765 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050019030 con 0x5582c997e370 2024-04-17T07:19:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.954+0000 7f404fe00640 1 -- 172.21.15.86:0/58779765 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40500119f0 con 0x5582c997e370 2024-04-17T07:19:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.954+0000 7f404fe00640 1 -- 172.21.15.86:0/58779765 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405001ec20 con 0x5582c997e370 2024-04-17T07:19:05.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.955+0000 7f406ce11740 1 -- 172.21.15.86:0/58779765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 msgr2=0x5582c997e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:05.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.955+0000 7f406ce11740 1 --2- 172.21.15.86:0/58779765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997e790 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f405000be80 tx=0x7f405000b730 comp rx=0 tx=0).stop 2024-04-17T07:19:05.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.955+0000 7f4062a00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40480a3c30 0x7f40480a60f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4050012010 tx=0x7f405001edc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:05.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.955+0000 7f406ce11740 1 -- 172.21.15.86:0/58779765 shutdown_connections 2024-04-17T07:19:05.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.955+0000 7f406ce11740 1 --2- 172.21.15.86:0/58779765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 0x5582c9986360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.955+0000 7f406ce11740 1 --2- 172.21.15.86:0/58779765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997ed60 0x5582c9983860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.955+0000 7f406ce11740 1 --2- 172.21.15.86:0/58779765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997e790 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.955+0000 7f406ce11740 1 -- 172.21.15.86:0/58779765 >> 172.21.15.86:0/58779765 conn(0x5582c99798e0 msgr2=0x5582c9979ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:05.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.955+0000 7f406ce11740 1 -- 172.21.15.86:0/58779765 shutdown_connections 2024-04-17T07:19:05.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.955+0000 7f406ce11740 1 -- 172.21.15.86:0/58779765 wait complete. 2024-04-17T07:19:05.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.956+0000 7f406ce11740 1 -- start start 2024-04-17T07:19:05.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.956+0000 7f406ce11740 1 Processor -- start 2024-04-17T07:19:05.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.957+0000 7f406ce11740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c9988850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.957+0000 7f406ce11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 0x5582c9988d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.957+0000 7f406ce11740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c9983ed0 0x5582c998fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.957+0000 7f406ce11740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5582c9986e80 con 0x5582c997e370 2024-04-17T07:19:05.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.957+0000 7f406ce11740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5582c989e710 con 0x5582c9983ed0 2024-04-17T07:19:05.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.957+0000 7f406ce11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5582c9987000 con 0x5582c997ed60 2024-04-17T07:19:05.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.957+0000 7f406ce11740 10 client.0 _ll_register_callbacks cb 0x5582c9899af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:05.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.957+0000 7f4063400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c9988850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.957+0000 7f4063400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c9988850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41324/0 (socket says 172.21.15.86:41324) 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.957+0000 7f4063400640 1 -- 172.21.15.86:0/1632436966 learned_addr learned my addr 172.21.15.86:0/1632436966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.958+0000 7f4062a00640 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 0x5582c9988d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.958+0000 7f4063e00640 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c9983ed0 0x5582c998fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.958+0000 7f4063400640 1 -- 172.21.15.86:0/1632436966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 msgr2=0x5582c9988d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.958+0000 7f4063400640 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 0x5582c9988d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.958+0000 7f4063400640 1 -- 172.21.15.86:0/1632436966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c9983ed0 msgr2=0x5582c998fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.958+0000 7f4063400640 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c9983ed0 0x5582c998fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.958+0000 7f4063400640 1 -- 172.21.15.86:0/1632436966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582c95f73f0 con 0x5582c997e370 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.958+0000 7f4062a00640 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 0x5582c9988d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.958+0000 7f4063e00640 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c9983ed0 0x5582c998fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:05.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.959+0000 7f4063400640 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c9988850 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f405402c840 tx=0x7f4054004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6e9c9fbc1f9681 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:05.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.959+0000 7f404cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:05.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.959+0000 7f404cc00640 1 -- 172.21.15.86:0/1632436966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054044020 con 0x5582c997e370 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.959+0000 7f404cc00640 1 -- 172.21.15.86:0/1632436966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4054005240 con 0x5582c997e370 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.959+0000 7f406ce11740 1 -- 172.21.15.86:0/1632436966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40500119f0 con 0x5582c997e370 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f404cc00640 1 -- 172.21.15.86:0/1632436966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f405404f050 con 0x5582c997e370 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f406ce11740 10 client.16704 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f406ce11740 1 -- 172.21.15.86:0/1632436966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582c95b70f0 con 0x5582c997e370 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f404cc00640 1 -- 172.21.15.86:0/1632436966 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40500119f0 con 0x5582c997e370 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f404cc00640 1 -- 172.21.15.86:0/1632436966 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4054003850 con 0x5582c997e370 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f406ce11740 10 client.16704 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f406ce11740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f406ce11740 20 client.16704 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f406ce11740 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5582c9994700 0x5582c9996b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f406ce11740 4 client.16704 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:05.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.960+0000 7f406ce11740 1 -- 172.21.15.86:0/1632436966 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582c98f2b40 con 0x5582c9994700 2024-04-17T07:19:05.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.961+0000 7f4062a00640 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5582c9994700 0x5582c9996b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.962+0000 7f4062a00640 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5582c9994700 0x5582c9996b70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b5860faa1555bc server_cookie=8d88b567de5b04c9 in_seq=0 out_seq=0 2024-04-17T07:19:05.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.962+0000 7f404cc00640 10 client.16704 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f404cc00640 1 -- 172.21.15.86:0/1632436966 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582c98f2b40 con 0x5582c9994700 2024-04-17T07:19:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f404cc00640 10 client.16704 handle_command_reply: tid=0 2024-04-17T07:19:05.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 client.16704 shutdown 2024-04-17T07:19:05.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 -- 172.21.15.86:0/1632436966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 msgr2=0x5582c9988850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:05.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c9988850 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f405402c840 tx=0x7f4054004270 comp rx=0 tx=0).stop 2024-04-17T07:19:05.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 -- 172.21.15.86:0/1632436966 shutdown_connections 2024-04-17T07:19:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5582c9994700 0x5582c9996b70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c9983ed0 0x5582c998fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 0x5582c9988d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 --2- 172.21.15.86:0/1632436966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c9988850 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 -- 172.21.15.86:0/1632436966 >> 172.21.15.86:0/1632436966 conn(0x5582c99798e0 msgr2=0x5582c9978ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f404cc00640 0 client.16704 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 -- 172.21.15.86:0/1632436966 shutdown_connections 2024-04-17T07:19:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.965+0000 7f406ce11740 1 -- 172.21.15.86:0/1632436966 wait complete. 2024-04-17T07:19:05.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.966+0000 7f406ce11740 20 client.16704 trim_cache size 0 max 16384 2024-04-17T07:19:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.988+0000 7f406ce11740 1 Processor -- start 2024-04-17T07:19:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.988+0000 7f406ce11740 1 -- start start 2024-04-17T07:19:05.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.988+0000 7f406ce11740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.988+0000 7f406ce11740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997ed60 0x5582c998fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f406ce11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 0x5582c9988b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f406ce11740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b1040 con 0x5582c997e370 2024-04-17T07:19:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f406ce11740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b2b80 con 0x5582c997ed60 2024-04-17T07:19:05.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f406ce11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b1710 con 0x5582c9983ed0 2024-04-17T07:19:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f4063400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f4063400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41340/0 (socket says 172.21.15.86:41340) 2024-04-17T07:19:05.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f4063400640 1 -- 172.21.15.86:0/2317216442 learned_addr learned my addr 172.21.15.86:0/2317216442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f4062a00640 1 --2- 172.21.15.86:0/2317216442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997ed60 0x5582c998fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f4063e00640 1 --2- 172.21.15.86:0/2317216442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 0x5582c9988b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f4063400640 1 -- 172.21.15.86:0/2317216442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 msgr2=0x5582c9988b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f4063400640 1 --2- 172.21.15.86:0/2317216442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 0x5582c9988b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f4063400640 1 -- 172.21.15.86:0/2317216442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997ed60 msgr2=0x5582c998fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f4063400640 1 --2- 172.21.15.86:0/2317216442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997ed60 0x5582c998fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.989+0000 7f4063400640 1 -- 172.21.15.86:0/2317216442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c9975340 con 0x5582c997e370 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.990+0000 7f4063e00640 1 --2- 172.21.15.86:0/2317216442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 0x5582c9988b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.990+0000 7f4063400640 1 --2- 172.21.15.86:0/2317216442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997d3d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f40540096c0 tx=0x7f405402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7380ab7a3af2ffa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.990+0000 7f404f400640 1 -- 172.21.15.86:0/2317216442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054000e60 con 0x5582c997e370 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.990+0000 7f404f400640 1 -- 172.21.15.86:0/2317216442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4054040660 con 0x5582c997e370 2024-04-17T07:19:05.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.990+0000 7f404f400640 1 -- 172.21.15.86:0/2317216442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054044040 con 0x5582c997e370 2024-04-17T07:19:05.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.991+0000 7f406ce11740 1 -- 172.21.15.86:0/2317216442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 msgr2=0x5582c997d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:05.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.991+0000 7f406ce11740 1 --2- 172.21.15.86:0/2317216442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997d3d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f40540096c0 tx=0x7f405402bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:05.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.991+0000 7f406ce11740 1 -- 172.21.15.86:0/2317216442 shutdown_connections 2024-04-17T07:19:05.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.991+0000 7f406ce11740 1 --2- 172.21.15.86:0/2317216442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c9983ed0 0x5582c9988b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.991+0000 7f406ce11740 1 --2- 172.21.15.86:0/2317216442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997ed60 0x5582c998fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.991+0000 7f406ce11740 1 --2- 172.21.15.86:0/2317216442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c997e370 0x5582c997d3d0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.991+0000 7f406ce11740 1 -- 172.21.15.86:0/2317216442 >> 172.21.15.86:0/2317216442 conn(0x5582c99798e0 msgr2=0x5582c9978250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.991+0000 7f406ce11740 1 -- 172.21.15.86:0/2317216442 shutdown_connections 2024-04-17T07:19:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.991+0000 7f406ce11740 1 -- 172.21.15.86:0/2317216442 wait complete. 2024-04-17T07:19:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.992+0000 7f406ce11740 1 -- start start 2024-04-17T07:19:05.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.992+0000 7f406ce11740 1 Processor -- start 2024-04-17T07:19:05.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f406ce11740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997e370 0x5582c998d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f406ce11740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 0x5582c998da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f406ce11740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c9983ed0 0x5582c998ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f406ce11740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b24b0 con 0x5582c9983ed0 2024-04-17T07:19:05.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f406ce11740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40641b1de0 con 0x5582c997e370 2024-04-17T07:19:05.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f406ce11740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f40641af010 con 0x5582c997ed60 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f406ce11740 10 client.0 _ll_register_callbacks cb 0x7f4054009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f4063e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c9983ed0 0x5582c998ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f4063e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c9983ed0 0x5582c998ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41356/0 (socket says 172.21.15.86:41356) 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f4063e00640 1 -- 172.21.15.86:0/1593063256 learned_addr learned my addr 172.21.15.86:0/1593063256 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.993+0000 7f4063400640 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997e370 0x5582c998d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f4062a00640 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 0x5582c998da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f4063e00640 1 -- 172.21.15.86:0/1593063256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 msgr2=0x5582c998da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f4063e00640 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 0x5582c998da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f4063e00640 1 -- 172.21.15.86:0/1593063256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997e370 msgr2=0x5582c998d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f4063e00640 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997e370 0x5582c998d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f4063e00640 1 -- 172.21.15.86:0/1593063256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582c998f080 con 0x5582c9983ed0 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f4062a00640 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 0x5582c998da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f4063e00640 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c9983ed0 0x5582c998ea90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f405c00a6e0 tx=0x7f405c09c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c2a8b75c9e78be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f404cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f404cc00640 1 -- 172.21.15.86:0/1593063256 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c01d030 con 0x5582c9983ed0 2024-04-17T07:19:05.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f404cc00640 1 -- 172.21.15.86:0/1593063256 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f405c005560 con 0x5582c9983ed0 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.994+0000 7f406ce11740 1 -- 172.21.15.86:0/1593063256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4054040660 con 0x5582c9983ed0 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f404cc00640 1 -- 172.21.15.86:0/1593063256 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f405c0ba050 con 0x5582c9983ed0 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f406ce11740 10 client.16710 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f406ce11740 1 -- 172.21.15.86:0/1593063256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582c99facb0 con 0x5582c9983ed0 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f404cc00640 1 -- 172.21.15.86:0/1593063256 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4054040660 con 0x5582c9983ed0 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f404cc00640 1 -- 172.21.15.86:0/1593063256 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f405c085ce0 con 0x5582c9983ed0 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f406ce11740 10 client.16710 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f406ce11740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f406ce11740 20 client.16710 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f406ce11740 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5582c99fb630 0x5582c99fda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f406ce11740 4 client.16710 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:19:05.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.995+0000 7f406ce11740 1 -- 172.21.15.86:0/1593063256 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582c98ec9b0 con 0x5582c99fb630 2024-04-17T07:19:05.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.996+0000 7f4063400640 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5582c99fb630 0x5582c99fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:05.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.997+0000 7f4063400640 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5582c99fb630 0x5582c99fda50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ea977bdf56f84d server_cookie=fd9022bc0d745638 in_seq=0 out_seq=0 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.997+0000 7f404cc00640 10 client.16710 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.997+0000 7f404cc00640 1 -- 172.21.15.86:0/1593063256 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5582c98ec9b0 con 0x5582c99fb630 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.997+0000 7f404cc00640 10 client.16710 handle_command_reply: tid=0 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.997+0000 7f406ce11740 1 client.16710 shutdown 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 1 -- 172.21.15.86:0/1593063256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c9983ed0 msgr2=0x5582c998ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c9983ed0 0x5582c998ea90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f405c00a6e0 tx=0x7f405c09c750 comp rx=0 tx=0).stop 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 1 -- 172.21.15.86:0/1593063256 shutdown_connections 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5582c99fb630 0x5582c99fda50 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582c9983ed0 0x5582c998ea90 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5582c997ed60 0x5582c998da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 1 --2- 172.21.15.86:0/1593063256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5582c997e370 0x5582c998d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 1 -- 172.21.15.86:0/1593063256 >> 172.21.15.86:0/1593063256 conn(0x5582c99798e0 msgr2=0x5582c997d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f404cc00640 0 client.16710 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:05.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 1 -- 172.21.15.86:0/1593063256 shutdown_connections 2024-04-17T07:19:06.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 1 -- 172.21.15.86:0/1593063256 wait complete. 2024-04-17T07:19:06.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.998+0000 7f406ce11740 20 client.16710 trim_cache size 0 max 16384 2024-04-17T07:19:06.000 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:19:06.000 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:19:06.000 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:19:06.000 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:19:06.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.999+0000 7f4068c00640 1 -- 172.21.15.86:0/3118206150 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40480a3c30 msgr2=0x7f40480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.999+0000 7f4068c00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40480a3c30 0x7f40480a60f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4050012010 tx=0x7f405001edc0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.999+0000 7f4068c00640 1 -- 172.21.15.86:0/3118206150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4064103a40 msgr2=0x7f40641a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.999+0000 7f4068c00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4064103a40 0x7f40641a18e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f405c00cad0 tx=0x7f405c00cf90 comp rx=0 tx=0).stop 2024-04-17T07:19:06.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.999+0000 7f4068c00640 1 -- 172.21.15.86:0/3118206150 shutdown_connections 2024-04-17T07:19:06.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.999+0000 7f4068c00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40480a3c30 0x7f40480a60f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.999+0000 7f4068c00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4064103a40 0x7f40641a18e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.999+0000 7f4068c00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4064103080 0x7f406419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.999+0000 7f4068c00640 1 --2- 172.21.15.86:0/3118206150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4064101ea0 0x7f406419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:05.999+0000 7f4068c00640 1 -- 172.21.15.86:0/3118206150 >> 172.21.15.86:0/3118206150 conn(0x7f40640fd460 msgr2=0x7f4064074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:06.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.000+0000 7f4068c00640 1 -- 172.21.15.86:0/3118206150 shutdown_connections 2024-04-17T07:19:06.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.000+0000 7f4068c00640 1 -- 172.21.15.86:0/3118206150 wait complete. 2024-04-17T07:19:06.018 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:19:06.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:19:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.232+0000 7f53ada00640 1 -- 172.21.15.86:0/2121669773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a8103b60 msgr2=0x7f53a810a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.232+0000 7f53ada00640 1 --2- 172.21.15.86:0/2121669773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a8103b60 0x7f53a810a3f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009ca0 tx=0x7f53a002c0f0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.232+0000 7f53ada00640 1 -- 172.21.15.86:0/2121669773 shutdown_connections 2024-04-17T07:19:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.232+0000 7f53ada00640 1 --2- 172.21.15.86:0/2121669773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a8103b60 0x7f53a810a3f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.232+0000 7f53ada00640 1 --2- 172.21.15.86:0/2121669773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53a81031a0 0x7f53a8103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.232+0000 7f53ada00640 1 --2- 172.21.15.86:0/2121669773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a8101fb0 0x7f53a81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.232+0000 7f53ada00640 1 -- 172.21.15.86:0/2121669773 >> 172.21.15.86:0/2121669773 conn(0x7f53a80fd540 msgr2=0x7f53a80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.233+0000 7f53ada00640 1 -- 172.21.15.86:0/2121669773 shutdown_connections 2024-04-17T07:19:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.233+0000 7f53ada00640 1 -- 172.21.15.86:0/2121669773 wait complete. 2024-04-17T07:19:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.234+0000 7f53ada00640 1 Processor -- start 2024-04-17T07:19:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.234+0000 7f53ada00640 1 -- start start 2024-04-17T07:19:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.234+0000 7f53ada00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53a8101fb0 0x7f53a819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.234+0000 7f53ada00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a81031a0 0x7f53a819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.234+0000 7f53ada00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a8103b60 0x7f53a81a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.234+0000 7f53ada00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810dab0 con 0x7f53a81031a0 2024-04-17T07:19:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.234+0000 7f53ada00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810d930 con 0x7f53a8103b60 2024-04-17T07:19:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.234+0000 7f53ada00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810dc30 con 0x7f53a8101fb0 2024-04-17T07:19:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.235+0000 7f53a6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a81031a0 0x7f53a819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.235+0000 7f53a6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a81031a0 0x7f53a819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41380/0 (socket says 172.21.15.86:41380) 2024-04-17T07:19:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.235+0000 7f53a6000640 1 -- 172.21.15.86:0/2330123221 learned_addr learned my addr 172.21.15.86:0/2330123221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.235+0000 7f53a6a00640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53a8101fb0 0x7f53a819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.235+0000 7f53a7400640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a8103b60 0x7f53a81a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.235+0000 7f53a6000640 1 -- 172.21.15.86:0/2330123221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53a8101fb0 msgr2=0x7f53a819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.235+0000 7f53a6000640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53a8101fb0 0x7f53a819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.235+0000 7f53a6000640 1 -- 172.21.15.86:0/2330123221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a8103b60 msgr2=0x7f53a81a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.235+0000 7f53a6000640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a8103b60 0x7f53a81a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.235+0000 7f53a6000640 1 -- 172.21.15.86:0/2330123221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a81a1fe0 con 0x7f53a81031a0 2024-04-17T07:19:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.236+0000 7f53a6000640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a81031a0 0x7f53a819a950 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f539400d950 tx=0x7f539400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.236+0000 7f539f400640 1 -- 172.21.15.86:0/2330123221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394017070 con 0x7f53a81031a0 2024-04-17T07:19:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.236+0000 7f53ada00640 1 -- 172.21.15.86:0/2330123221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a81a22d0 con 0x7f53a81031a0 2024-04-17T07:19:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.236+0000 7f539f400640 1 -- 172.21.15.86:0/2330123221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5394009c30 con 0x7f53a81031a0 2024-04-17T07:19:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.236+0000 7f539f400640 1 -- 172.21.15.86:0/2330123221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539400f620 con 0x7f53a81031a0 2024-04-17T07:19:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.236+0000 7f53ada00640 1 -- 172.21.15.86:0/2330123221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a81a2870 con 0x7f53a81031a0 2024-04-17T07:19:06.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.238+0000 7f53ada00640 1 -- 172.21.15.86:0/2330123221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a81a2b90 con 0x7f53a81031a0 2024-04-17T07:19:06.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.242+0000 7f539f400640 1 -- 172.21.15.86:0/2330123221 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f53940109a0 con 0x7f53a81031a0 2024-04-17T07:19:06.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.243+0000 7f539f400640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f538c0a3bb0 0x7f538c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.244+0000 7f539f400640 1 -- 172.21.15.86:0/2330123221 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f53940d0340 con 0x7f53a81031a0 2024-04-17T07:19:06.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.244+0000 7f53a6a00640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f538c0a3bb0 0x7f538c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.245+0000 7f53a6a00640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f538c0a3bb0 0x7f538c0a6070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5398005d00 tx=0x7f5398005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:06.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.249+0000 7f539f400640 1 -- 172.21.15.86:0/2330123221 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5394009de0 con 0x7f53a81031a0 2024-04-17T07:19:06.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.581+0000 7f53ada00640 1 -- 172.21.15.86:0/2330123221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53a81a2d60 con 0x7f53a81031a0 2024-04-17T07:19:06.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.582+0000 7f539f400640 1 -- 172.21.15.86:0/2330123221 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f539409ca40 con 0x7f53a81031a0 2024-04-17T07:19:06.582 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:19:06.583 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:19:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:19:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 -- 172.21.15.86:0/2330123221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f538c0a3bb0 msgr2=0x7f538c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f538c0a3bb0 0x7f538c0a6070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5398005d00 tx=0x7f5398005c50 comp rx=0 tx=0).stop 2024-04-17T07:19:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 -- 172.21.15.86:0/2330123221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a81031a0 msgr2=0x7f53a819a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a81031a0 0x7f53a819a950 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f539400d950 tx=0x7f539400de10 comp rx=0 tx=0).stop 2024-04-17T07:19:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 -- 172.21.15.86:0/2330123221 shutdown_connections 2024-04-17T07:19:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f538c0a3bb0 0x7f538c0a6070 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a8103b60 0x7f53a81a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a81031a0 0x7f53a819a950 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 --2- 172.21.15.86:0/2330123221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53a8101fb0 0x7f53a819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 -- 172.21.15.86:0/2330123221 >> 172.21.15.86:0/2330123221 conn(0x7f53a80fd540 msgr2=0x7f53a8074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:06.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 -- 172.21.15.86:0/2330123221 shutdown_connections 2024-04-17T07:19:06.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.586+0000 7f53ada00640 1 -- 172.21.15.86:0/2330123221 wait complete. 2024-04-17T07:19:06.606 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:19:06.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.831+0000 7f23e0c00640 1 -- 172.21.15.86:0/248674871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc075fa0 msgr2=0x7f23dc0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.831+0000 7f23e0c00640 1 --2- 172.21.15.86:0/248674871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc075fa0 0x7f23dc0743f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009930 tx=0x7f23cc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:06.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.831+0000 7f23e0c00640 1 -- 172.21.15.86:0/248674871 shutdown_connections 2024-04-17T07:19:06.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.831+0000 7f23e0c00640 1 --2- 172.21.15.86:0/248674871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23dc10e4c0 0x7f23dc1108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.831+0000 7f23e0c00640 1 --2- 172.21.15.86:0/248674871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc0749c0 0x7f23dc074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.831+0000 7f23e0c00640 1 --2- 172.21.15.86:0/248674871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc075fa0 0x7f23dc0743f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.831+0000 7f23e0c00640 1 -- 172.21.15.86:0/248674871 >> 172.21.15.86:0/248674871 conn(0x7f23dc0fd460 msgr2=0x7f23dc0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.831+0000 7f23e0c00640 1 -- 172.21.15.86:0/248674871 shutdown_connections 2024-04-17T07:19:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.831+0000 7f23e0c00640 1 -- 172.21.15.86:0/248674871 wait complete. 2024-04-17T07:19:06.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.833+0000 7f23e0c00640 1 Processor -- start 2024-04-17T07:19:06.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.833+0000 7f23e0c00640 1 -- start start 2024-04-17T07:19:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.833+0000 7f23e0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc0749c0 0x7f23dc19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.834+0000 7f23e0c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23dc075fa0 0x7f23dc19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.834+0000 7f23e0c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc10e4c0 0x7f23dc1a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.834+0000 7f23e0c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc114010 con 0x7f23dc0749c0 2024-04-17T07:19:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.834+0000 7f23e0c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc113e90 con 0x7f23dc10e4c0 2024-04-17T07:19:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.834+0000 7f23e0c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc114190 con 0x7f23dc075fa0 2024-04-17T07:19:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.834+0000 7f23db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc0749c0 0x7f23dc19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.834+0000 7f23db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc0749c0 0x7f23dc19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48296/0 (socket says 172.21.15.86:48296) 2024-04-17T07:19:06.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.834+0000 7f23db400640 1 -- 172.21.15.86:0/527715580 learned_addr learned my addr 172.21.15.86:0/527715580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.834+0000 7f23dbe00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc10e4c0 0x7f23dc1a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.834+0000 7f23daa00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23dc075fa0 0x7f23dc19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.835+0000 7f23db400640 1 -- 172.21.15.86:0/527715580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23dc075fa0 msgr2=0x7f23dc19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.835+0000 7f23db400640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23dc075fa0 0x7f23dc19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.835+0000 7f23db400640 1 -- 172.21.15.86:0/527715580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc10e4c0 msgr2=0x7f23dc1a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.835+0000 7f23db400640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc10e4c0 0x7f23dc1a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.835+0000 7f23db400640 1 -- 172.21.15.86:0/527715580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc1a6310 con 0x7f23dc0749c0 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.835+0000 7f23dbe00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc10e4c0 0x7f23dc1a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.835+0000 7f23db400640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc0749c0 0x7f23dc19e6f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f23cc005ab0 tx=0x7f23cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.835+0000 7f23d3e00640 1 -- 172.21.15.86:0/527715580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc043020 con 0x7f23dc0749c0 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.836+0000 7f23d3e00640 1 -- 172.21.15.86:0/527715580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23cc0054c0 con 0x7f23dc0749c0 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.836+0000 7f23d3e00640 1 -- 172.21.15.86:0/527715580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc003d10 con 0x7f23dc0749c0 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.836+0000 7f23e0c00640 1 -- 172.21.15.86:0/527715580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc1a65a0 con 0x7f23dc0749c0 2024-04-17T07:19:06.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.836+0000 7f23e0c00640 1 -- 172.21.15.86:0/527715580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc1a6b40 con 0x7f23dc0749c0 2024-04-17T07:19:06.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.839+0000 7f23d3e00640 1 -- 172.21.15.86:0/527715580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f23cc046020 con 0x7f23dc0749c0 2024-04-17T07:19:06.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.840+0000 7f23e4e3a740 1 Processor -- start 2024-04-17T07:19:06.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.840+0000 7f23e4e3a740 1 -- start start 2024-04-17T07:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.840+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0ee160 0x56004d0ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.840+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 0x56004d0f3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.840+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0f6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.840+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56004d054020 con 0x56004d0f3cc0 2024-04-17T07:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.840+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56004cffa460 con 0x56004d0ee160 2024-04-17T07:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.840+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56004d04ffe0 con 0x56004d0eeb50 2024-04-17T07:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.841+0000 7f23daa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.841+0000 7f23daa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48308/0 (socket says 172.21.15.86:48308) 2024-04-17T07:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.841+0000 7f23daa00640 1 -- 172.21.15.86:0/2845915918 learned_addr learned my addr 172.21.15.86:0/2845915918 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.841+0000 7f23dbe00640 1 --2- 172.21.15.86:0/2845915918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0ee160 0x56004d0ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.841+0000 7f23db400640 1 --2- 172.21.15.86:0/2845915918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 0x56004d0f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.841+0000 7f23daa00640 1 -- 172.21.15.86:0/2845915918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 msgr2=0x56004d0f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23daa00640 1 --2- 172.21.15.86:0/2845915918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 0x56004d0f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23daa00640 1 -- 172.21.15.86:0/2845915918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0ee160 msgr2=0x56004d0ee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23daa00640 1 --2- 172.21.15.86:0/2845915918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0ee160 0x56004d0ee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23daa00640 1 -- 172.21.15.86:0/2845915918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56004d002040 con 0x56004d0f3cc0 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23d3e00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23c00a3c30 0x7f23c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23d3e00640 1 -- 172.21.15.86:0/527715580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f23cc0f2b30 con 0x7f23dc0749c0 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23daa00640 1 --2- 172.21.15.86:0/2845915918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0f6150 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f23c800bc80 tx=0x7f23c800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efbb5cfc2ce6cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23dbe00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23c00a3c30 0x7f23c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23c7e00640 1 -- 172.21.15.86:0/2845915918 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8015070 con 0x56004d0f3cc0 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23c7e00640 1 -- 172.21.15.86:0/2845915918 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23c80118f0 con 0x56004d0f3cc0 2024-04-17T07:19:06.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.842+0000 7f23c7e00640 1 -- 172.21.15.86:0/2845915918 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c80107f0 con 0x56004d0f3cc0 2024-04-17T07:19:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.843+0000 7f23e4e3a740 1 -- 172.21.15.86:0/2845915918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 msgr2=0x56004d0f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.843+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/2845915918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0f6150 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f23c800bc80 tx=0x7f23c800f730 comp rx=0 tx=0).stop 2024-04-17T07:19:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.843+0000 7f23dbe00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23c00a3c30 0x7f23c00a60f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x56004d0ef3b0 tx=0x7f23d4009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.843+0000 7f23e4e3a740 1 -- 172.21.15.86:0/2845915918 shutdown_connections 2024-04-17T07:19:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.843+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/2845915918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0f6150 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.844+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/2845915918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 0x56004d0f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.844+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/2845915918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0ee160 0x56004d0ee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.844+0000 7f23e4e3a740 1 -- 172.21.15.86:0/2845915918 >> 172.21.15.86:0/2845915918 conn(0x56004d0e96d0 msgr2=0x56004d0e9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.844+0000 7f23e4e3a740 1 -- 172.21.15.86:0/2845915918 shutdown_connections 2024-04-17T07:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.844+0000 7f23e4e3a740 1 -- 172.21.15.86:0/2845915918 wait complete. 2024-04-17T07:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.844+0000 7f23e4e3a740 1 -- start start 2024-04-17T07:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.844+0000 7f23e4e3a740 1 Processor -- start 2024-04-17T07:19:06.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.845+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0ee160 0x56004d0f8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.845+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0eeb50 0x56004d0f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.845+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.845+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56004d0f6c70 con 0x56004d0f3cc0 2024-04-17T07:19:06.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.845+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56004d00d830 con 0x56004d0eeb50 2024-04-17T07:19:06.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.845+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56004d0f6df0 con 0x56004d0ee160 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.845+0000 7f23e4e3a740 10 client.0 _ll_register_callbacks cb 0x56004cfdbfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.845+0000 7f23dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.845+0000 7f23dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48316/0 (socket says 172.21.15.86:48316) 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.845+0000 7f23dbe00640 1 -- 172.21.15.86:0/1440018375 learned_addr learned my addr 172.21.15.86:0/1440018375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.846+0000 7f23daa00640 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0eeb50 0x56004d0f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.846+0000 7f23db400640 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0ee160 0x56004d0f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.846+0000 7f23dbe00640 1 -- 172.21.15.86:0/1440018375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0ee160 msgr2=0x56004d0f8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.846+0000 7f23dbe00640 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0ee160 0x56004d0f8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.846+0000 7f23dbe00640 1 -- 172.21.15.86:0/1440018375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0eeb50 msgr2=0x56004d0f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.846+0000 7f23dbe00640 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0eeb50 0x56004d0f8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.846+0000 7f23dbe00640 1 -- 172.21.15.86:0/1440018375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56004d011920 con 0x56004d0f3cc0 2024-04-17T07:19:06.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.847+0000 7f23dbe00640 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0ffc80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f23d4011d10 tx=0x7f23d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c56bc68737d449c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:06.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.847+0000 7f23c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:06.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.847+0000 7f23c4c00640 1 -- 172.21.15.86:0/1440018375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4020620 con 0x56004d0f3cc0 2024-04-17T07:19:06.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.847+0000 7f23c4c00640 1 -- 172.21.15.86:0/1440018375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f23d40207c0 con 0x56004d0f3cc0 2024-04-17T07:19:06.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.847+0000 7f23e4e3a740 1 -- 172.21.15.86:0/1440018375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23c80118f0 con 0x56004d0f3cc0 2024-04-17T07:19:06.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23c4c00640 1 -- 172.21.15.86:0/1440018375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23d4033050 con 0x56004d0f3cc0 2024-04-17T07:19:06.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23e4e3a740 10 client.16728 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23e4e3a740 1 -- 172.21.15.86:0/1440018375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56004cfdf7f0 con 0x56004d0f3cc0 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23c4c00640 1 -- 172.21.15.86:0/1440018375 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23c80118f0 con 0x56004d0f3cc0 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23c4c00640 1 -- 172.21.15.86:0/1440018375 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f23d400f040 con 0x56004d0f3cc0 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23e4e3a740 10 client.16728 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23e4e3a740 10 client.16728 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23e4e3a740 20 client.16728 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56004d104340 0x56004d1067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23e4e3a740 4 client.16728 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.848+0000 7f23e4e3a740 1 -- 172.21.15.86:0/1440018375 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56004d061a40 con 0x56004d104340 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.849+0000 7f23db400640 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56004d104340 0x56004d1067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.850+0000 7f23db400640 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56004d104340 0x56004d1067a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eada4f6d19c37f8b server_cookie=c6f0fb0a52220b71 in_seq=0 out_seq=0 2024-04-17T07:19:06.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.850+0000 7f23c4c00640 10 client.16728 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:19:06.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23c4c00640 1 -- 172.21.15.86:0/1440018375 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56004d061a40 con 0x56004d104340 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23c4c00640 10 client.16728 handle_command_reply: tid=0 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 client.16728 shutdown 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 -- 172.21.15.86:0/1440018375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 msgr2=0x56004d0ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0ffc80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f23d4011d10 tx=0x7f23d4004a40 comp rx=0 tx=0).stop 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 -- 172.21.15.86:0/1440018375 shutdown_connections 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56004d104340 0x56004d1067a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0f3cc0 0x56004d0ffc80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0eeb50 0x56004d0f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/1440018375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0ee160 0x56004d0f8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 -- 172.21.15.86:0/1440018375 >> 172.21.15.86:0/1440018375 conn(0x56004d0e96d0 msgr2=0x56004d0e8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23c4c00640 0 client.16728 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 -- 172.21.15.86:0/1440018375 shutdown_connections 2024-04-17T07:19:06.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.852+0000 7f23e4e3a740 1 -- 172.21.15.86:0/1440018375 wait complete. 2024-04-17T07:19:06.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.853+0000 7f23e4e3a740 20 client.16728 trim_cache size 0 max 16384 2024-04-17T07:19:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.867+0000 7f23e4e3a740 1 Processor -- start 2024-04-17T07:19:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.868+0000 7f23e4e3a740 1 -- start start 2024-04-17T07:19:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.868+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d106350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.868+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 0x56004d0ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.868+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0f3cc0 0x56004d0e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.868+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1b0e70 con 0x56004d0ee160 2024-04-17T07:19:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.868+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1b29b0 con 0x56004d0f3cc0 2024-04-17T07:19:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.868+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1b1540 con 0x56004d0eeb50 2024-04-17T07:19:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.868+0000 7f23db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d106350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.868+0000 7f23db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d106350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48318/0 (socket says 172.21.15.86:48318) 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.868+0000 7f23db400640 1 -- 172.21.15.86:0/3548909915 learned_addr learned my addr 172.21.15.86:0/3548909915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.869+0000 7f23daa00640 1 --2- 172.21.15.86:0/3548909915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 0x56004d0ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.869+0000 7f23dbe00640 1 --2- 172.21.15.86:0/3548909915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0f3cc0 0x56004d0e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.869+0000 7f23db400640 1 -- 172.21.15.86:0/3548909915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 msgr2=0x56004d0ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.869+0000 7f23db400640 1 --2- 172.21.15.86:0/3548909915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 0x56004d0ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.869+0000 7f23db400640 1 -- 172.21.15.86:0/3548909915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0f3cc0 msgr2=0x56004d0e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.869+0000 7f23db400640 1 --2- 172.21.15.86:0/3548909915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0f3cc0 0x56004d0e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.869+0000 7f23db400640 1 -- 172.21.15.86:0/3548909915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56004cd663f0 con 0x56004d0ee160 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.869+0000 7f23daa00640 1 --2- 172.21.15.86:0/3548909915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 0x56004d0ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.869+0000 7f23db400640 1 --2- 172.21.15.86:0/3548909915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d106350 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0be9c0 tx=0x7f23cc0df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19414463b2dd4c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.869+0000 7f23c7400640 1 -- 172.21.15.86:0/3548909915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0bacd0 con 0x56004d0ee160 2024-04-17T07:19:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.870+0000 7f23c7400640 1 -- 172.21.15.86:0/3548909915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23cc0bf130 con 0x56004d0ee160 2024-04-17T07:19:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.870+0000 7f23c7400640 1 -- 172.21.15.86:0/3548909915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0df0b0 con 0x56004d0ee160 2024-04-17T07:19:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.870+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3548909915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 msgr2=0x56004d106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.870+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/3548909915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d106350 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0be9c0 tx=0x7f23cc0df040 comp rx=0 tx=0).stop 2024-04-17T07:19:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.870+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3548909915 shutdown_connections 2024-04-17T07:19:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.870+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/3548909915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0f3cc0 0x56004d0e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.871+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/3548909915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0eeb50 0x56004d0ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.871+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/3548909915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d106350 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.871+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3548909915 >> 172.21.15.86:0/3548909915 conn(0x56004d0e96d0 msgr2=0x56004d0f0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.871+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3548909915 shutdown_connections 2024-04-17T07:19:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.871+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3548909915 wait complete. 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.871+0000 7f23e4e3a740 1 -- start start 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.871+0000 7f23e4e3a740 1 Processor -- start 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d0fd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0eeb50 0x56004d0fd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23e4e3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0f3cc0 0x56004d0fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1b22e0 con 0x56004d0ee160 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1b1c10 con 0x56004d0eeb50 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23e4e3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc1aee40 con 0x56004d0f3cc0 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23e4e3a740 10 client.0 _ll_register_callbacks cb 0x7f23d4011990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d0fd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d0fd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48324/0 (socket says 172.21.15.86:48324) 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23db400640 1 -- 172.21.15.86:0/3523815760 learned_addr learned my addr 172.21.15.86:0/3523815760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23dbe00640 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0f3cc0 0x56004d0fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23daa00640 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0eeb50 0x56004d0fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23db400640 1 -- 172.21.15.86:0/3523815760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0f3cc0 msgr2=0x56004d0fe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23db400640 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0f3cc0 0x56004d0fe540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23db400640 1 -- 172.21.15.86:0/3523815760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0eeb50 msgr2=0x56004d0fd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23db400640 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0eeb50 0x56004d0fd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.872+0000 7f23db400640 1 -- 172.21.15.86:0/3523815760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56004ce307d0 con 0x56004d0ee160 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.873+0000 7f23db400640 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d0fd040 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0f2ea0 tx=0x7f23cc0f6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1da4b0b5d9a4b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.873+0000 7f23c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.873+0000 7f23c4c00640 1 -- 172.21.15.86:0/3523815760 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0044e0 con 0x56004d0ee160 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.873+0000 7f23c4c00640 1 -- 172.21.15.86:0/3523815760 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f23cc0bb0a0 con 0x56004d0ee160 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.873+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3523815760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23cc0bf130 con 0x56004d0ee160 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.873+0000 7f23c4c00640 1 -- 172.21.15.86:0/3523815760 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23cc0be390 con 0x56004d0ee160 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.873+0000 7f23e4e3a740 10 client.16734 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.873+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3523815760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56004d0ffcd0 con 0x56004d0ee160 2024-04-17T07:19:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.874+0000 7f23c4c00640 1 -- 172.21.15.86:0/3523815760 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23cc0bf130 con 0x56004d0ee160 2024-04-17T07:19:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.874+0000 7f23c4c00640 1 -- 172.21.15.86:0/3523815760 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f23cc0c7210 con 0x56004d0ee160 2024-04-17T07:19:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.874+0000 7f23e4e3a740 10 client.16734 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.874+0000 7f23e4e3a740 10 client.16734 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:19:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.874+0000 7f23e4e3a740 20 client.16734 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.874+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56004d16abd0 0x56004d16cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.874+0000 7f23e4e3a740 4 client.16734 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:19:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.874+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3523815760 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56004d05b8b0 con 0x56004d16abd0 2024-04-17T07:19:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.874+0000 7f23daa00640 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56004d16abd0 0x56004d16cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.875+0000 7f23daa00640 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56004d16abd0 0x56004d16cfc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7641f03ced006a2e server_cookie=38fe192e00568f0e in_seq=0 out_seq=0 2024-04-17T07:19:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.875+0000 7f23c4c00640 10 client.16734 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:19:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23c4c00640 1 -- 172.21.15.86:0/3523815760 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56004d05b8b0 con 0x56004d16abd0 2024-04-17T07:19:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23c4c00640 10 client.16734 handle_command_reply: tid=0 2024-04-17T07:19:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 client.16734 shutdown 2024-04-17T07:19:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3523815760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 msgr2=0x56004d0fd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d0fd040 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0f2ea0 tx=0x7f23cc0f6570 comp rx=0 tx=0).stop 2024-04-17T07:19:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3523815760 shutdown_connections 2024-04-17T07:19:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56004d16abd0 0x56004d16cfc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56004d0f3cc0 0x56004d0fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56004d0eeb50 0x56004d0fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 --2- 172.21.15.86:0/3523815760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56004d0ee160 0x56004d0fd040 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23c4c00640 0 client.16734 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:19:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3523815760 >> 172.21.15.86:0/3523815760 conn(0x56004d0e96d0 msgr2=0x56004d0ec930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3523815760 shutdown_connections 2024-04-17T07:19:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 1 -- 172.21.15.86:0/3523815760 wait complete. 2024-04-17T07:19:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.876+0000 7f23e4e3a740 20 client.16734 trim_cache size 0 max 16384 2024-04-17T07:19:06.880 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:19:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.877+0000 7f23e0c00640 1 -- 172.21.15.86:0/527715580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23c00a3c30 msgr2=0x7f23c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.877+0000 7f23e0c00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23c00a3c30 0x7f23c00a60f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x56004d0ef3b0 tx=0x7f23d4009800 comp rx=0 tx=0).stop 2024-04-17T07:19:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.877+0000 7f23e0c00640 1 -- 172.21.15.86:0/527715580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc0749c0 msgr2=0x7f23dc19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.877+0000 7f23e0c00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc0749c0 0x7f23dc19e6f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f23cc005ab0 tx=0x7f23cc0042f0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.877+0000 7f23e0c00640 1 -- 172.21.15.86:0/527715580 shutdown_connections 2024-04-17T07:19:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.877+0000 7f23e0c00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23c00a3c30 0x7f23c00a60f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.877+0000 7f23e0c00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc10e4c0 0x7f23dc1a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.877+0000 7f23e0c00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23dc075fa0 0x7f23dc19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.877+0000 7f23e0c00640 1 --2- 172.21.15.86:0/527715580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23dc0749c0 0x7f23dc19e6f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:06.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.877+0000 7f23e0c00640 1 -- 172.21.15.86:0/527715580 >> 172.21.15.86:0/527715580 conn(0x7f23dc0fd460 msgr2=0x7f23dc101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:06.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.878+0000 7f23e0c00640 1 -- 172.21.15.86:0/527715580 shutdown_connections 2024-04-17T07:19:06.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:06.878+0000 7f23e0c00640 1 -- 172.21.15.86:0/527715580 wait complete. 2024-04-17T07:19:06.899 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:06 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2330123221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:19:06.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:19:07.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:06 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2330123221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:19:07.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:06 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2330123221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:19:07.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.133+0000 7f5d53200640 1 -- 172.21.15.86:0/543678379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541039a0 msgr2=0x7f5d5410a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.133+0000 7f5d53200640 1 --2- 172.21.15.86:0/543678379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541039a0 0x7f5d5410a2c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009ca0 tx=0x7f5d4c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.134+0000 7f5d53200640 1 -- 172.21.15.86:0/543678379 shutdown_connections 2024-04-17T07:19:07.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.134+0000 7f5d53200640 1 --2- 172.21.15.86:0/543678379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541039a0 0x7f5d5410a2c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.134+0000 7f5d53200640 1 --2- 172.21.15.86:0/543678379 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d54102fe0 0x7f5d54103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.134+0000 7f5d53200640 1 --2- 172.21.15.86:0/543678379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d54101e90 0x7f5d541022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.134+0000 7f5d53200640 1 -- 172.21.15.86:0/543678379 >> 172.21.15.86:0/543678379 conn(0x7f5d540fd400 msgr2=0x7f5d540ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.134+0000 7f5d53200640 1 -- 172.21.15.86:0/543678379 shutdown_connections 2024-04-17T07:19:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.134+0000 7f5d53200640 1 -- 172.21.15.86:0/543678379 wait complete. 2024-04-17T07:19:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.135+0000 7f5d53200640 1 Processor -- start 2024-04-17T07:19:07.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.135+0000 7f5d53200640 1 -- start start 2024-04-17T07:19:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.136+0000 7f5d53200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d54101e90 0x7f5d54195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.136+0000 7f5d53200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d54102fe0 0x7f5d54196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.136+0000 7f5d53200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541039a0 0x7f5d5419d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.136+0000 7f5d53200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5410b850 con 0x7f5d541039a0 2024-04-17T07:19:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.136+0000 7f5d53200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5410b6d0 con 0x7f5d54102fe0 2024-04-17T07:19:07.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.136+0000 7f5d53200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5410b9d0 con 0x7f5d54101e90 2024-04-17T07:19:07.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.136+0000 7f5d52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541039a0 0x7f5d5419d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.137+0000 7f5d52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541039a0 0x7f5d5419d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48334/0 (socket says 172.21.15.86:48334) 2024-04-17T07:19:07.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.137+0000 7f5d52800640 1 -- 172.21.15.86:0/480086602 learned_addr learned my addr 172.21.15.86:0/480086602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.137+0000 7f5d51400640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d54102fe0 0x7f5d54196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.137+0000 7f5d51e00640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d54101e90 0x7f5d54195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.137+0000 7f5d52800640 1 -- 172.21.15.86:0/480086602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d54101e90 msgr2=0x7f5d54195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.137+0000 7f5d52800640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d54101e90 0x7f5d54195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.137+0000 7f5d52800640 1 -- 172.21.15.86:0/480086602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d54102fe0 msgr2=0x7f5d54196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.137+0000 7f5d52800640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d54102fe0 0x7f5d54196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.137+0000 7f5d52800640 1 -- 172.21.15.86:0/480086602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5419da10 con 0x7f5d541039a0 2024-04-17T07:19:07.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.138+0000 7f5d52800640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541039a0 0x7f5d5419d3a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c032380 tx=0x7f5d4c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.138+0000 7f5d4aa00640 1 -- 172.21.15.86:0/480086602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c041020 con 0x7f5d541039a0 2024-04-17T07:19:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.138+0000 7f5d53200640 1 -- 172.21.15.86:0/480086602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5419dca0 con 0x7f5d541039a0 2024-04-17T07:19:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.139+0000 7f5d4aa00640 1 -- 172.21.15.86:0/480086602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d4c003e20 con 0x7f5d541039a0 2024-04-17T07:19:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.139+0000 7f5d4aa00640 1 -- 172.21.15.86:0/480086602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c039040 con 0x7f5d541039a0 2024-04-17T07:19:07.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.139+0000 7f5d53200640 1 -- 172.21.15.86:0/480086602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5419e190 con 0x7f5d541039a0 2024-04-17T07:19:07.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.143+0000 7f5d5ba35740 1 Processor -- start 2024-04-17T07:19:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.143+0000 7f5d4aa00640 1 -- 172.21.15.86:0/480086602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5d4c044020 con 0x7f5d541039a0 2024-04-17T07:19:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.144+0000 7f5d5ba35740 1 -- start start 2024-04-17T07:19:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.144+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x557081773580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.144+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773b50 0x557081778650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.144+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081778cc0 0x55708177b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.144+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5570816d9020 con 0x557081773160 2024-04-17T07:19:07.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.144+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55708167f460 con 0x557081773b50 2024-04-17T07:19:07.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.144+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570816d4fe0 con 0x557081778cc0 2024-04-17T07:19:07.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.144+0000 7f5d51400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x557081773580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.144+0000 7f5d51400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x557081773580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48336/0 (socket says 172.21.15.86:48336) 2024-04-17T07:19:07.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.144+0000 7f5d51400640 1 -- 172.21.15.86:0/4249437650 learned_addr learned my addr 172.21.15.86:0/4249437650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.145+0000 7f5d51e00640 1 --2- 172.21.15.86:0/4249437650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773b50 0x557081778650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.145+0000 7f5d51400640 1 --2- 172.21.15.86:0/4249437650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081778cc0 0x55708177b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.145+0000 7f5d51400640 1 -- 172.21.15.86:0/4249437650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081778cc0 msgr2=0x55708177b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.145+0000 7f5d51400640 1 --2- 172.21.15.86:0/4249437650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081778cc0 0x55708177b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.145+0000 7f5d51400640 1 -- 172.21.15.86:0/4249437650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773b50 msgr2=0x557081778650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.145+0000 7f5d51400640 1 --2- 172.21.15.86:0/4249437650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773b50 0x557081778650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.145+0000 7f5d51400640 1 -- 172.21.15.86:0/4249437650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557081687040 con 0x557081773160 2024-04-17T07:19:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.145+0000 7f5d4aa00640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d380a7fc0 0x7f5d380aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.145+0000 7f5d4aa00640 1 -- 172.21.15.86:0/480086602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5d4c0f28d0 con 0x7f5d541039a0 2024-04-17T07:19:07.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.145+0000 7f5d51400640 1 --2- 172.21.15.86:0/4249437650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x557081773580 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000b620 tx=0x7f5d4000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61344021d620a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.146+0000 7f5d51e00640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d380a7fc0 0x7f5d380aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.146+0000 7f5d3f000640 1 -- 172.21.15.86:0/4249437650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40004020 con 0x557081773160 2024-04-17T07:19:07.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.146+0000 7f5d3f000640 1 -- 172.21.15.86:0/4249437650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d400054d0 con 0x557081773160 2024-04-17T07:19:07.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.146+0000 7f5d3f000640 1 -- 172.21.15.86:0/4249437650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40004b60 con 0x557081773160 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.147+0000 7f5d51e00640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d380a7fc0 0x7f5d380aa480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x557081773f50 tx=0x7f5d44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.147+0000 7f5d5ba35740 1 -- 172.21.15.86:0/4249437650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 msgr2=0x557081773580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.147+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/4249437650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x557081773580 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000b620 tx=0x7f5d4000b820 comp rx=0 tx=0).stop 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.147+0000 7f5d5ba35740 1 -- 172.21.15.86:0/4249437650 shutdown_connections 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.147+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/4249437650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081778cc0 0x55708177b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.147+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/4249437650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773b50 0x557081778650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.147+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/4249437650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x557081773580 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.147+0000 7f5d5ba35740 1 -- 172.21.15.86:0/4249437650 >> 172.21.15.86:0/4249437650 conn(0x55708176e6d0 msgr2=0x55708176ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.147+0000 7f5d5ba35740 1 -- 172.21.15.86:0/4249437650 shutdown_connections 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.148+0000 7f5d5ba35740 1 -- 172.21.15.86:0/4249437650 wait complete. 2024-04-17T07:19:07.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.148+0000 7f5d5ba35740 1 -- start start 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.148+0000 7f5d5ba35740 1 Processor -- start 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773b50 0x55708177dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081778cc0 0x557081784c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55708177bc70 con 0x557081773160 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557081692830 con 0x557081773b50 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55708177bdf0 con 0x557081778cc0 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d5ba35740 10 client.0 _ll_register_callbacks cb 0x557081660fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d51400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773b50 0x55708177dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48342/0 (socket says 172.21.15.86:48342) 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.149+0000 7f5d51e00640 1 -- 172.21.15.86:0/2608530149 learned_addr learned my addr 172.21.15.86:0/2608530149 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.150+0000 7f5d52800640 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081778cc0 0x557081784c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.150+0000 7f5d51e00640 1 -- 172.21.15.86:0/2608530149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081778cc0 msgr2=0x557081784c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.150+0000 7f5d51e00640 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081778cc0 0x557081784c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.150+0000 7f5d51e00640 1 -- 172.21.15.86:0/2608530149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773b50 msgr2=0x55708177dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.150+0000 7f5d51e00640 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773b50 0x55708177dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.150+0000 7f5d51e00640 1 -- 172.21.15.86:0/2608530149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557081696920 con 0x557081773160 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.150+0000 7f5d51e00640 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d700 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5d44004600 tx=0x7f5d44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468bcd1eee6176ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.150+0000 7f5d33400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.151+0000 7f5d33400640 1 -- 172.21.15.86:0/2608530149 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44005020 con 0x557081773160 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.151+0000 7f5d33400640 1 -- 172.21.15.86:0/2608530149 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5d440051c0 con 0x557081773160 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.151+0000 7f5d5ba35740 1 -- 172.21.15.86:0/2608530149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d400054d0 con 0x557081773160 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.151+0000 7f5d33400640 1 -- 172.21.15.86:0/2608530149 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d440053c0 con 0x557081773160 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.151+0000 7f5d5ba35740 10 client.16746 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:07.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.151+0000 7f5d5ba35740 1 -- 172.21.15.86:0/2608530149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570816647f0 con 0x557081773160 2024-04-17T07:19:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.151+0000 7f5d33400640 1 -- 172.21.15.86:0/2608530149 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d400054d0 con 0x557081773160 2024-04-17T07:19:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.152+0000 7f5d33400640 1 -- 172.21.15.86:0/2608530149 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5d44027070 con 0x557081773160 2024-04-17T07:19:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.152+0000 7f5d5ba35740 10 client.16746 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:07.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.152+0000 7f5d5ba35740 10 client.16746 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:19:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.152+0000 7f5d5ba35740 20 client.16746 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.152+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5570817892b0 0x55708178b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.152+0000 7f5d5ba35740 4 client.16746 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.152+0000 7f5d5ba35740 1 -- 172.21.15.86:0/2608530149 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570816e6a40 con 0x5570817892b0 2024-04-17T07:19:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.153+0000 7f5d51400640 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5570817892b0 0x55708178b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.154+0000 7f5d51400640 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5570817892b0 0x55708178b750 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f3c1d926a843e31 server_cookie=5af795d5fd3f35c4 in_seq=0 out_seq=0 2024-04-17T07:19:07.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.154+0000 7f5d33400640 10 client.16746 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d33400640 1 -- 172.21.15.86:0/2608530149 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570816e6a40 con 0x5570817892b0 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d33400640 10 client.16746 handle_command_reply: tid=0 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 client.16746 shutdown 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 -- 172.21.15.86:0/2608530149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 msgr2=0x55708177d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d700 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5d44004600 tx=0x7f5d44004730 comp rx=0 tx=0).stop 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 -- 172.21.15.86:0/2608530149 shutdown_connections 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5570817892b0 0x55708178b750 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081778cc0 0x557081784c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773b50 0x55708177dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/2608530149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d700 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 -- 172.21.15.86:0/2608530149 >> 172.21.15.86:0/2608530149 conn(0x55708176e6d0 msgr2=0x55708176d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d33400640 0 client.16746 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:19:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 -- 172.21.15.86:0/2608530149 shutdown_connections 2024-04-17T07:19:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.156+0000 7f5d5ba35740 1 -- 172.21.15.86:0/2608530149 wait complete. 2024-04-17T07:19:07.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.157+0000 7f5d5ba35740 20 client.16746 trim_cache size 0 max 16384 2024-04-17T07:19:07.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.178+0000 7f5d5ba35740 1 Processor -- start 2024-04-17T07:19:07.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.178+0000 7f5d5ba35740 1 -- start start 2024-04-17T07:19:07.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.179+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.179+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081773b50 0x557081784c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.179+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081778cc0 0x55708177d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.179+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541a8650 con 0x557081773160 2024-04-17T07:19:07.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.179+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541aa190 con 0x557081778cc0 2024-04-17T07:19:07.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.179+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541a8d20 con 0x557081773b50 2024-04-17T07:19:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.179+0000 7f5d51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.179+0000 7f5d51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48346/0 (socket says 172.21.15.86:48346) 2024-04-17T07:19:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.179+0000 7f5d51e00640 1 -- 172.21.15.86:0/861926454 learned_addr learned my addr 172.21.15.86:0/861926454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.179+0000 7f5d51400640 1 --2- 172.21.15.86:0/861926454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081773b50 0x557081784c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.180+0000 7f5d52800640 1 --2- 172.21.15.86:0/861926454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081778cc0 0x55708177d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.180+0000 7f5d51e00640 1 -- 172.21.15.86:0/861926454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081773b50 msgr2=0x557081784c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.180+0000 7f5d51e00640 1 --2- 172.21.15.86:0/861926454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081773b50 0x557081784c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.180+0000 7f5d51e00640 1 -- 172.21.15.86:0/861926454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081778cc0 msgr2=0x55708177d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.180+0000 7f5d51e00640 1 --2- 172.21.15.86:0/861926454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081778cc0 0x55708177d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.180+0000 7f5d51e00640 1 -- 172.21.15.86:0/861926454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570813eb3f0 con 0x557081773160 2024-04-17T07:19:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.180+0000 7f5d51e00640 1 --2- 172.21.15.86:0/861926454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d310 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009b60 tx=0x7f5d44009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41eb7c228ed2035 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.180+0000 7f5d3e600640 1 -- 172.21.15.86:0/861926454 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4401e680 con 0x557081773160 2024-04-17T07:19:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.180+0000 7f5d3e600640 1 -- 172.21.15.86:0/861926454 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d4401ec20 con 0x557081773160 2024-04-17T07:19:07.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.180+0000 7f5d3e600640 1 -- 172.21.15.86:0/861926454 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44036650 con 0x557081773160 2024-04-17T07:19:07.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.181+0000 7f5d5ba35740 1 -- 172.21.15.86:0/861926454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 msgr2=0x55708177d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.181+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/861926454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d310 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009b60 tx=0x7f5d44009b90 comp rx=0 tx=0).stop 2024-04-17T07:19:07.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.181+0000 7f5d5ba35740 1 -- 172.21.15.86:0/861926454 shutdown_connections 2024-04-17T07:19:07.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.181+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/861926454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081778cc0 0x55708177d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.181+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/861926454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081773b50 0x557081784c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.181+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/861926454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081773160 0x55708177d310 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.181+0000 7f5d5ba35740 1 -- 172.21.15.86:0/861926454 >> 172.21.15.86:0/861926454 conn(0x55708176e6d0 msgr2=0x55708177c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.182+0000 7f5d5ba35740 1 -- 172.21.15.86:0/861926454 shutdown_connections 2024-04-17T07:19:07.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.182+0000 7f5d5ba35740 1 -- 172.21.15.86:0/861926454 wait complete. 2024-04-17T07:19:07.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.182+0000 7f5d5ba35740 1 -- start start 2024-04-17T07:19:07.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.182+0000 7f5d5ba35740 1 Processor -- start 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773160 0x557081781b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081773b50 0x557081782050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d5ba35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081778cc0 0x5570817f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541a9ac0 con 0x557081778cc0 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541a93f0 con 0x557081773160 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d5ba35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541a6620 con 0x557081773b50 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d5ba35740 10 client.0 _ll_register_callbacks cb 0x7f5d440044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081778cc0 0x5570817f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081778cc0 0x5570817f6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48348/0 (socket says 172.21.15.86:48348) 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d52800640 1 -- 172.21.15.86:0/3016394142 learned_addr learned my addr 172.21.15.86:0/3016394142 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.183+0000 7f5d51400640 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081773b50 0x557081782050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.184+0000 7f5d52800640 1 -- 172.21.15.86:0/3016394142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081773b50 msgr2=0x557081782050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.184+0000 7f5d52800640 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081773b50 0x557081782050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.184+0000 7f5d52800640 1 -- 172.21.15.86:0/3016394142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773160 msgr2=0x557081781b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.184+0000 7f5d52800640 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773160 0x557081781b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.184+0000 7f5d52800640 1 -- 172.21.15.86:0/3016394142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570814b57d0 con 0x557081778cc0 2024-04-17T07:19:07.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.184+0000 7f5d52800640 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081778cc0 0x5570817f6090 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0bac30 tx=0x7f5d4c0be690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea8910d1ef74b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.184+0000 7f5d33400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:07.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.184+0000 7f5d33400640 1 -- 172.21.15.86:0/3016394142 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0be7d0 con 0x557081778cc0 2024-04-17T07:19:07.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.184+0000 7f5d33400640 1 -- 172.21.15.86:0/3016394142 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5d4c034d30 con 0x557081778cc0 2024-04-17T07:19:07.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.184+0000 7f5d5ba35740 1 -- 172.21.15.86:0/3016394142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d4401ec20 con 0x557081778cc0 2024-04-17T07:19:07.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d33400640 1 -- 172.21.15.86:0/3016394142 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d4c0c2b80 con 0x557081778cc0 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d5ba35740 10 client.16752 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d5ba35740 1 -- 172.21.15.86:0/3016394142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570817f77a0 con 0x557081778cc0 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d33400640 1 -- 172.21.15.86:0/3016394142 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d4401ec20 con 0x557081778cc0 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d33400640 1 -- 172.21.15.86:0/3016394142 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5d4c03c030 con 0x557081778cc0 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d5ba35740 10 client.16752 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d5ba35740 10 client.16752 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d5ba35740 20 client.16752 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5570817f8150 0x5570817fa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d5ba35740 4 client.16752 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.185+0000 7f5d5ba35740 1 -- 172.21.15.86:0/3016394142 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570816e08b0 con 0x5570817f8150 2024-04-17T07:19:07.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.186+0000 7f5d51e00640 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5570817f8150 0x5570817fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.187+0000 7f5d33400640 10 client.16752 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:19:07.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.187+0000 7f5d51e00640 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5570817f8150 0x5570817fa570 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd7eb3e25cb05aa9 server_cookie=16af5d48cb6dab06 in_seq=0 out_seq=0 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.187+0000 7f5d33400640 1 -- 172.21.15.86:0/3016394142 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570816e08b0 con 0x5570817f8150 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d33400640 10 client.16752 handle_command_reply: tid=0 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 client.16752 shutdown 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 -- 172.21.15.86:0/3016394142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081778cc0 msgr2=0x5570817f6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081778cc0 0x5570817f6090 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0bac30 tx=0x7f5d4c0be690 comp rx=0 tx=0).stop 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 -- 172.21.15.86:0/3016394142 shutdown_connections 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5570817f8150 0x5570817fa570 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557081778cc0 0x5570817f6090 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557081773b50 0x557081782050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 --2- 172.21.15.86:0/3016394142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557081773160 0x557081781b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 -- 172.21.15.86:0/3016394142 >> 172.21.15.86:0/3016394142 conn(0x55708176e6d0 msgr2=0x55708177d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d33400640 0 client.16752 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:19:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 -- 172.21.15.86:0/3016394142 shutdown_connections 2024-04-17T07:19:07.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.188+0000 7f5d5ba35740 1 -- 172.21.15.86:0/3016394142 wait complete. 2024-04-17T07:19:07.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.189+0000 7f5d5ba35740 20 client.16752 trim_cache size 0 max 16384 2024-04-17T07:19:07.190 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.189+0000 7f5d53200640 1 -- 172.21.15.86:0/480086602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d380a7fc0 msgr2=0x7f5d380aa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.189+0000 7f5d53200640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d380a7fc0 0x7f5d380aa480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x557081773f50 tx=0x7f5d44005f90 comp rx=0 tx=0).stop 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.189+0000 7f5d53200640 1 -- 172.21.15.86:0/480086602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541039a0 msgr2=0x7f5d5419d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.189+0000 7f5d53200640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541039a0 0x7f5d5419d3a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c032380 tx=0x7f5d4c032820 comp rx=0 tx=0).stop 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.190+0000 7f5d53200640 1 -- 172.21.15.86:0/480086602 shutdown_connections 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.190+0000 7f5d53200640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d380a7fc0 0x7f5d380aa480 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.190+0000 7f5d53200640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d541039a0 0x7f5d5419d3a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.190+0000 7f5d53200640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d54102fe0 0x7f5d54196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.190+0000 7f5d53200640 1 --2- 172.21.15.86:0/480086602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d54101e90 0x7f5d54195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.190+0000 7f5d53200640 1 -- 172.21.15.86:0/480086602 >> 172.21.15.86:0/480086602 conn(0x7f5d540fd400 msgr2=0x7f5d54106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.190+0000 7f5d53200640 1 -- 172.21.15.86:0/480086602 shutdown_connections 2024-04-17T07:19:07.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.190+0000 7f5d53200640 1 -- 172.21.15.86:0/480086602 wait complete. 2024-04-17T07:19:07.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:19:07.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.435+0000 7f4645200640 1 -- 172.21.15.86:0/2137105413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4640074960 msgr2=0x7f4640074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.435+0000 7f4645200640 1 --2- 172.21.15.86:0/2137105413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4640074960 0x7f4640074dc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f462c009900 tx=0x7f462c02bcd0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.435+0000 7f4645200640 1 -- 172.21.15.86:0/2137105413 shutdown_connections 2024-04-17T07:19:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.435+0000 7f4645200640 1 --2- 172.21.15.86:0/2137105413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4640109e40 0x7f464010c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.435+0000 7f4645200640 1 --2- 172.21.15.86:0/2137105413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4640074960 0x7f4640074dc0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.435+0000 7f4645200640 1 --2- 172.21.15.86:0/2137105413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4640075f40 0x7f4640074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.435+0000 7f4645200640 1 -- 172.21.15.86:0/2137105413 >> 172.21.15.86:0/2137105413 conn(0x7f46400fd540 msgr2=0x7f46400ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.435+0000 7f4645200640 1 -- 172.21.15.86:0/2137105413 shutdown_connections 2024-04-17T07:19:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.435+0000 7f4645200640 1 -- 172.21.15.86:0/2137105413 wait complete. 2024-04-17T07:19:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.436+0000 7f4645200640 1 Processor -- start 2024-04-17T07:19:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.436+0000 7f4645200640 1 -- start start 2024-04-17T07:19:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.437+0000 7f4645200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4640074960 0x7f464019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.437+0000 7f4645200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4640075f40 0x7f464019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.437+0000 7f4645200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4640109e40 0x7f46401a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.437+0000 7f4645200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f464010fd40 con 0x7f4640109e40 2024-04-17T07:19:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.437+0000 7f4645200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f464010fbc0 con 0x7f4640074960 2024-04-17T07:19:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.437+0000 7f4645200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f464010fec0 con 0x7f4640075f40 2024-04-17T07:19:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.437+0000 7f463e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4640109e40 0x7f46401a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.437+0000 7f463e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4640109e40 0x7f46401a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48374/0 (socket says 172.21.15.86:48374) 2024-04-17T07:19:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.437+0000 7f463e800640 1 -- 172.21.15.86:0/1762084900 learned_addr learned my addr 172.21.15.86:0/1762084900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.437+0000 7f463d400640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4640075f40 0x7f464019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.438+0000 7f463de00640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4640074960 0x7f464019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.438+0000 7f463e800640 1 -- 172.21.15.86:0/1762084900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4640075f40 msgr2=0x7f464019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.438+0000 7f463e800640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4640075f40 0x7f464019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.438+0000 7f463e800640 1 -- 172.21.15.86:0/1762084900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4640074960 msgr2=0x7f464019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.438+0000 7f463e800640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4640074960 0x7f464019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.438+0000 7f463e800640 1 -- 172.21.15.86:0/1762084900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46401a2050 con 0x7f4640109e40 2024-04-17T07:19:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.438+0000 7f463e800640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4640109e40 0x7f46401a19e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f463800eb10 tx=0x7f463800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.439+0000 7f4636a00640 1 -- 172.21.15.86:0/1762084900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638004b20 con 0x7f4640109e40 2024-04-17T07:19:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.439+0000 7f4636a00640 1 -- 172.21.15.86:0/1762084900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4638004cc0 con 0x7f4640109e40 2024-04-17T07:19:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.439+0000 7f4636a00640 1 -- 172.21.15.86:0/1762084900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638010690 con 0x7f4640109e40 2024-04-17T07:19:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.439+0000 7f4645200640 1 -- 172.21.15.86:0/1762084900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46401a2340 con 0x7f4640109e40 2024-04-17T07:19:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.439+0000 7f4645200640 1 -- 172.21.15.86:0/1762084900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46401a2810 con 0x7f4640109e40 2024-04-17T07:19:07.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.443+0000 7f4648029740 1 Processor -- start 2024-04-17T07:19:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.444+0000 7f4636a00640 1 -- 172.21.15.86:0/1762084900 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4638005020 con 0x7f4640109e40 2024-04-17T07:19:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.444+0000 7f4648029740 1 -- start start 2024-04-17T07:19:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.445+0000 7f4648029740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.445+0000 7f4648029740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 0x55c98fb3e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.445+0000 7f4648029740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb3eb00 0x55c98fb40f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.445+0000 7f4648029740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c98fa9f020 con 0x55c98fb39990 2024-04-17T07:19:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.445+0000 7f4648029740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c98fa45460 con 0x55c98fb38fa0 2024-04-17T07:19:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.445+0000 7f4648029740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c98fa9afe0 con 0x55c98fb3eb00 2024-04-17T07:19:07.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.445+0000 7f463de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 0x55c98fb3e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.446+0000 7f463de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 0x55c98fb3e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48378/0 (socket says 172.21.15.86:48378) 2024-04-17T07:19:07.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.446+0000 7f463de00640 1 -- 172.21.15.86:0/3250946558 learned_addr learned my addr 172.21.15.86:0/3250946558 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.446+0000 7f463d400640 1 --2- 172.21.15.86:0/3250946558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.446+0000 7f463d400640 1 --2- 172.21.15.86:0/3250946558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb3eb00 0x55c98fb40f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.446+0000 7f463de00640 1 -- 172.21.15.86:0/3250946558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb3eb00 msgr2=0x55c98fb40f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.446+0000 7f463de00640 1 --2- 172.21.15.86:0/3250946558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb3eb00 0x55c98fb40f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.446+0000 7f463de00640 1 -- 172.21.15.86:0/3250946558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 msgr2=0x55c98fb393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.446+0000 7f463de00640 1 --2- 172.21.15.86:0/3250946558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.446+0000 7f463de00640 1 -- 172.21.15.86:0/3250946558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98fa4d040 con 0x55c98fb39990 2024-04-17T07:19:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.446+0000 7f4636a00640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46240a3c30 0x7f46240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.447+0000 7f4636a00640 1 -- 172.21.15.86:0/1762084900 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f46380d0dc0 con 0x7f4640109e40 2024-04-17T07:19:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.447+0000 7f463d400640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46240a3c30 0x7f46240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.447+0000 7f463de00640 1 --2- 172.21.15.86:0/3250946558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 0x55c98fb3e490 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4630009d20 tx=0x7f4630011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658d36141d7f8e31 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.448+0000 7f462b400640 1 -- 172.21.15.86:0/3250946558 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463001e070 con 0x55c98fb39990 2024-04-17T07:19:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.448+0000 7f463d400640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46240a3c30 0x7f46240a60f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f462c000c00 tx=0x7f462c02c6b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.448+0000 7f462b400640 1 -- 172.21.15.86:0/3250946558 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4630019ae0 con 0x55c98fb39990 2024-04-17T07:19:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.448+0000 7f462b400640 1 -- 172.21.15.86:0/3250946558 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463000ea10 con 0x55c98fb39990 2024-04-17T07:19:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.449+0000 7f4648029740 1 -- 172.21.15.86:0/3250946558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 msgr2=0x55c98fb3e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.449+0000 7f4648029740 1 --2- 172.21.15.86:0/3250946558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 0x55c98fb3e490 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4630009d20 tx=0x7f4630011810 comp rx=0 tx=0).stop 2024-04-17T07:19:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.449+0000 7f4648029740 1 -- 172.21.15.86:0/3250946558 shutdown_connections 2024-04-17T07:19:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.449+0000 7f4648029740 1 --2- 172.21.15.86:0/3250946558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb3eb00 0x55c98fb40f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.449+0000 7f4648029740 1 --2- 172.21.15.86:0/3250946558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 0x55c98fb3e490 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.449+0000 7f4648029740 1 --2- 172.21.15.86:0/3250946558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.449+0000 7f4648029740 1 -- 172.21.15.86:0/3250946558 >> 172.21.15.86:0/3250946558 conn(0x55c98fb34510 msgr2=0x55c98fb34910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.449+0000 7f4648029740 1 -- 172.21.15.86:0/3250946558 shutdown_connections 2024-04-17T07:19:07.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.449+0000 7f4648029740 1 -- 172.21.15.86:0/3250946558 wait complete. 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.449+0000 7f4648029740 1 -- start start 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.450+0000 7f4648029740 1 Processor -- start 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.450+0000 7f4648029740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb4bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.450+0000 7f4648029740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 0x55c98fb4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.450+0000 7f4648029740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb3eb00 0x55c98fb53290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.450+0000 7f4648029740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c98fb41ab0 con 0x55c98fb39990 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.450+0000 7f4648029740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c98fa58830 con 0x55c98fb38fa0 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.450+0000 7f4648029740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c98fb41c30 con 0x55c98fb3eb00 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f4648029740 10 client.0 _ll_register_callbacks cb 0x55c98fa26fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f463de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb4bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f463d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 0x55c98fb4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f463de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb4bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43200/0 (socket says 172.21.15.86:43200) 2024-04-17T07:19:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f463de00640 1 -- 172.21.15.86:0/2425815384 learned_addr learned my addr 172.21.15.86:0/2425815384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f463e800640 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb3eb00 0x55c98fb53290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f463de00640 1 -- 172.21.15.86:0/2425815384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb3eb00 msgr2=0x55c98fb53290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f463de00640 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb3eb00 0x55c98fb53290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f463de00640 1 -- 172.21.15.86:0/2425815384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 msgr2=0x55c98fb4c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f463de00640 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 0x55c98fb4c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.451+0000 7f463de00640 1 -- 172.21.15.86:0/2425815384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98fa5c920 con 0x55c98fb38fa0 2024-04-17T07:19:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.452+0000 7f463de00640 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb4bd80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4630011d10 tx=0x7f4630009980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6724be979e285056 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.452+0000 7f461fe00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:19:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.452+0000 7f461fe00640 1 -- 172.21.15.86:0/2425815384 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630026020 con 0x55c98fb38fa0 2024-04-17T07:19:07.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.453+0000 7f461fe00640 1 -- 172.21.15.86:0/2425815384 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4630005030 con 0x55c98fb38fa0 2024-04-17T07:19:07.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.453+0000 7f4648029740 1 -- 172.21.15.86:0/2425815384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4630019ae0 con 0x55c98fb38fa0 2024-04-17T07:19:07.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.453+0000 7f461fe00640 1 -- 172.21.15.86:0/2425815384 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4630031050 con 0x55c98fb38fa0 2024-04-17T07:19:07.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.453+0000 7f4648029740 10 client.35777 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:07.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.453+0000 7f4648029740 1 -- 172.21.15.86:0/2425815384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c98fa2a7f0 con 0x55c98fb38fa0 2024-04-17T07:19:07.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.454+0000 7f461fe00640 1 -- 172.21.15.86:0/2425815384 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f463000e750 con 0x55c98fb38fa0 2024-04-17T07:19:07.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.454+0000 7f461fe00640 1 -- 172.21.15.86:0/2425815384 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f46300046c0 con 0x55c98fb38fa0 2024-04-17T07:19:07.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.454+0000 7f4648029740 10 client.35777 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:07.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.454+0000 7f4648029740 10 client.35777 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:19:07.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.454+0000 7f4648029740 20 client.35777 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:07.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.454+0000 7f4648029740 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55c98fb57560 0x55c98fb599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.455+0000 7f4648029740 4 client.35777 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:07.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.455+0000 7f4648029740 1 -- 172.21.15.86:0/2425815384 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4630019ae0 con 0x55c98fb57560 2024-04-17T07:19:07.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.455+0000 7f463d400640 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55c98fb57560 0x55c98fb599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.456+0000 7f461fe00640 10 client.35777 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:19:07.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.456+0000 7f463d400640 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55c98fb57560 0x55c98fb599b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74e726951391263a server_cookie=82063b59c62e4549 in_seq=0 out_seq=0 2024-04-17T07:19:07.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.457+0000 7f461fe00640 1 -- 172.21.15.86:0/2425815384 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f4630019ae0 con 0x55c98fb57560 2024-04-17T07:19:07.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.457+0000 7f461fe00640 10 client.35777 handle_command_reply: tid=0 2024-04-17T07:19:07.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.457+0000 7f4648029740 1 client.35777 shutdown 2024-04-17T07:19:07.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 1 -- 172.21.15.86:0/2425815384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 msgr2=0x55c98fb4bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb4bd80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4630011d10 tx=0x7f4630009980 comp rx=0 tx=0).stop 2024-04-17T07:19:07.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 1 -- 172.21.15.86:0/2425815384 shutdown_connections 2024-04-17T07:19:07.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55c98fb57560 0x55c98fb599b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb3eb00 0x55c98fb53290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb39990 0x55c98fb4c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f461fe00640 0 client.35777 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:19:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 1 --2- 172.21.15.86:0/2425815384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb4bd80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 1 -- 172.21.15.86:0/2425815384 >> 172.21.15.86:0/2425815384 conn(0x55c98fb34510 msgr2=0x55c98fb33710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 1 -- 172.21.15.86:0/2425815384 shutdown_connections 2024-04-17T07:19:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 1 -- 172.21.15.86:0/2425815384 wait complete. 2024-04-17T07:19:07.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.458+0000 7f4648029740 20 client.35777 trim_cache size 0 max 16384 2024-04-17T07:19:07.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.479+0000 7f4648029740 1 Processor -- start 2024-04-17T07:19:07.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.479+0000 7f4648029740 1 -- start start 2024-04-17T07:19:07.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.479+0000 7f4648029740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb38fa0 0x55c98fb4b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.479+0000 7f4648029740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb39990 0x55c98fb53250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.479+0000 7f4648029740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 0x55c98fb4b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.479+0000 7f4648029740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a8960 con 0x55c98fb3eb00 2024-04-17T07:19:07.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.479+0000 7f4648029740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f46401aa4a0 con 0x55c98fb39990 2024-04-17T07:19:07.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.479+0000 7f4648029740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a9030 con 0x55c98fb38fa0 2024-04-17T07:19:07.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 0x55c98fb4b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb38fa0 0x55c98fb4b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb39990 0x55c98fb53250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb38fa0 0x55c98fb4b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:57356/0 (socket says 172.21.15.86:57356) 2024-04-17T07:19:07.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463de00640 1 -- 172.21.15.86:0/1028448010 learned_addr learned my addr 172.21.15.86:0/1028448010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463e800640 1 -- 172.21.15.86:0/1028448010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb38fa0 msgr2=0x55c98fb4b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463e800640 1 --2- 172.21.15.86:0/1028448010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb38fa0 0x55c98fb4b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463e800640 1 -- 172.21.15.86:0/1028448010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb39990 msgr2=0x55c98fb53250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463e800640 1 --2- 172.21.15.86:0/1028448010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb39990 0x55c98fb53250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463e800640 1 -- 172.21.15.86:0/1028448010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98f7b13f0 con 0x55c98fb3eb00 2024-04-17T07:19:07.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.480+0000 7f463d400640 1 --2- 172.21.15.86:0/1028448010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb39990 0x55c98fb53250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:07.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.481+0000 7f463e800640 1 --2- 172.21.15.86:0/1028448010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 0x55c98fb4b8e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f46380049d0 tx=0x7f46380990e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955666d6619ef61 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.481+0000 7f462aa00640 1 -- 172.21.15.86:0/1028448010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463809d420 con 0x55c98fb3eb00 2024-04-17T07:19:07.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.481+0000 7f462aa00640 1 -- 172.21.15.86:0/1028448010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46380b6260 con 0x55c98fb3eb00 2024-04-17T07:19:07.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.481+0000 7f462aa00640 1 -- 172.21.15.86:0/1028448010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638005310 con 0x55c98fb3eb00 2024-04-17T07:19:07.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.482+0000 7f4648029740 1 -- 172.21.15.86:0/1028448010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 msgr2=0x55c98fb4b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.482+0000 7f4648029740 1 --2- 172.21.15.86:0/1028448010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 0x55c98fb4b8e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f46380049d0 tx=0x7f46380990e0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.483+0000 7f4648029740 1 -- 172.21.15.86:0/1028448010 shutdown_connections 2024-04-17T07:19:07.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.483+0000 7f4648029740 1 --2- 172.21.15.86:0/1028448010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 0x55c98fb4b8e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.483+0000 7f4648029740 1 --2- 172.21.15.86:0/1028448010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb39990 0x55c98fb53250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.483+0000 7f4648029740 1 --2- 172.21.15.86:0/1028448010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb38fa0 0x55c98fb4b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.483+0000 7f4648029740 1 -- 172.21.15.86:0/1028448010 >> 172.21.15.86:0/1028448010 conn(0x55c98fb34510 msgr2=0x55c98fb38000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.483+0000 7f4648029740 1 -- 172.21.15.86:0/1028448010 shutdown_connections 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.483+0000 7f4648029740 1 -- 172.21.15.86:0/1028448010 wait complete. 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.484+0000 7f4648029740 1 -- start start 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.484+0000 7f4648029740 1 Processor -- start 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.484+0000 7f4648029740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.484+0000 7f4648029740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb39990 0x55c98fb50ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.485+0000 7f4648029740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 0x55c98fb51cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.485+0000 7f4648029740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a9dd0 con 0x55c98fb3eb00 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.485+0000 7f4648029740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a9700 con 0x55c98fb38fa0 2024-04-17T07:19:07.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.485+0000 7f4648029740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46401a6930 con 0x55c98fb39990 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.485+0000 7f4648029740 10 client.0 _ll_register_callbacks cb 0x7f462c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.485+0000 7f463de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.485+0000 7f463d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb39990 0x55c98fb50ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.485+0000 7f463de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43216/0 (socket says 172.21.15.86:43216) 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.485+0000 7f463de00640 1 -- 172.21.15.86:0/2960126353 learned_addr learned my addr 172.21.15.86:0/2960126353 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.486+0000 7f463e800640 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 0x55c98fb51cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.486+0000 7f463de00640 1 -- 172.21.15.86:0/2960126353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb39990 msgr2=0x55c98fb50ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.486+0000 7f463de00640 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb39990 0x55c98fb50ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.486+0000 7f463de00640 1 -- 172.21.15.86:0/2960126353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 msgr2=0x55c98fb51cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.486+0000 7f463de00640 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 0x55c98fb51cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.486+0000 7f463de00640 1 -- 172.21.15.86:0/2960126353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98f87b7d0 con 0x55c98fb38fa0 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.486+0000 7f463e800640 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 0x55c98fb51cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:07.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.487+0000 7f463de00640 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb507a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4630009c40 tx=0x7f4630009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c6e1a768f95896 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.487+0000 7f461fe00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:19:07.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.487+0000 7f461fe00640 1 -- 172.21.15.86:0/2960126353 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463001e040 con 0x55c98fb38fa0 2024-04-17T07:19:07.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.487+0000 7f461fe00640 1 -- 172.21.15.86:0/2960126353 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4630019da0 con 0x55c98fb38fa0 2024-04-17T07:19:07.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.487+0000 7f4648029740 1 -- 172.21.15.86:0/2960126353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46380b6260 con 0x55c98fb38fa0 2024-04-17T07:19:07.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.488+0000 7f461fe00640 1 -- 172.21.15.86:0/2960126353 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46300035f0 con 0x55c98fb38fa0 2024-04-17T07:19:07.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.488+0000 7f4648029740 10 client.35783 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:07.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.488+0000 7f4648029740 1 -- 172.21.15.86:0/2960126353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c98fb53410 con 0x55c98fb38fa0 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.489+0000 7f461fe00640 1 -- 172.21.15.86:0/2960126353 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46380b6260 con 0x55c98fb38fa0 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.489+0000 7f461fe00640 1 -- 172.21.15.86:0/2960126353 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4630004110 con 0x55c98fb38fa0 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.489+0000 7f4648029740 10 client.35783 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.489+0000 7f4648029740 10 client.35783 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.489+0000 7f4648029740 20 client.35783 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.489+0000 7f4648029740 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55c98fbbe0f0 0x55c98fbc0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.489+0000 7f4648029740 4 client.35783 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.489+0000 7f4648029740 1 -- 172.21.15.86:0/2960126353 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c98faaca40 con 0x55c98fbbe0f0 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.490+0000 7f463d400640 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55c98fbbe0f0 0x55c98fbc0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.491+0000 7f463d400640 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55c98fbbe0f0 0x55c98fbc0510 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6582fdcf5fbffb34 server_cookie=58e50ed668cf6f49 in_seq=0 out_seq=0 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.491+0000 7f461fe00640 10 client.35783 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:19:07.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.491+0000 7f461fe00640 1 -- 172.21.15.86:0/2960126353 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c98faaca40 con 0x55c98fbbe0f0 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.491+0000 7f461fe00640 10 client.35783 handle_command_reply: tid=0 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.491+0000 7f4648029740 1 client.35783 shutdown 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.491+0000 7f4648029740 1 -- 172.21.15.86:0/2960126353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 msgr2=0x55c98fb507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.491+0000 7f4648029740 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb507a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4630009c40 tx=0x7f4630009f30 comp rx=0 tx=0).stop 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.492+0000 7f4648029740 1 -- 172.21.15.86:0/2960126353 shutdown_connections 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.492+0000 7f4648029740 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55c98fbbe0f0 0x55c98fbc0510 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.492+0000 7f4648029740 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c98fb3eb00 0x55c98fb51cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.492+0000 7f4648029740 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c98fb39990 0x55c98fb50ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.492+0000 7f4648029740 1 --2- 172.21.15.86:0/2960126353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c98fb38fa0 0x55c98fb507a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.492+0000 7f4648029740 1 -- 172.21.15.86:0/2960126353 >> 172.21.15.86:0/2960126353 conn(0x55c98fb34510 msgr2=0x55c98fb32e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.492+0000 7f461fe00640 0 client.35783 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.492+0000 7f4648029740 1 -- 172.21.15.86:0/2960126353 shutdown_connections 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.492+0000 7f4648029740 1 -- 172.21.15.86:0/2960126353 wait complete. 2024-04-17T07:19:07.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.493+0000 7f4648029740 20 client.35783 trim_cache size 0 max 16384 2024-04-17T07:19:07.495 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.494+0000 7f4645200640 1 -- 172.21.15.86:0/1762084900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46240a3c30 msgr2=0x7f46240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.494+0000 7f4645200640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46240a3c30 0x7f46240a60f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f462c000c00 tx=0x7f462c02c6b0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.494+0000 7f4645200640 1 -- 172.21.15.86:0/1762084900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4640109e40 msgr2=0x7f46401a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.494+0000 7f4645200640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4640109e40 0x7f46401a19e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f463800eb10 tx=0x7f463800c2d0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.495+0000 7f4645200640 1 -- 172.21.15.86:0/1762084900 shutdown_connections 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.495+0000 7f4645200640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46240a3c30 0x7f46240a60f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.495+0000 7f4645200640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4640109e40 0x7f46401a19e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.495+0000 7f4645200640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4640075f40 0x7f464019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.495+0000 7f4645200640 1 --2- 172.21.15.86:0/1762084900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4640074960 0x7f464019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.495+0000 7f4645200640 1 -- 172.21.15.86:0/1762084900 >> 172.21.15.86:0/1762084900 conn(0x7f46400fd540 msgr2=0x7f4640105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.495+0000 7f4645200640 1 -- 172.21.15.86:0/1762084900 shutdown_connections 2024-04-17T07:19:07.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.495+0000 7f4645200640 1 -- 172.21.15.86:0/1762084900 wait complete. 2024-04-17T07:19:07.521 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:19:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.759+0000 7fa7dd200640 1 -- 172.21.15.86:0/2389134070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d8102fe0 msgr2=0x7fa7d8103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.759+0000 7fa7dd200640 1 --2- 172.21.15.86:0/2389134070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d8102fe0 0x7fa7d8103460 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009900 tx=0x7fa7c402bc50 comp rx=0 tx=0).stop 2024-04-17T07:19:07.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.759+0000 7fa7dd200640 1 -- 172.21.15.86:0/2389134070 shutdown_connections 2024-04-17T07:19:07.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.759+0000 7fa7dd200640 1 --2- 172.21.15.86:0/2389134070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d81039a0 0x7fa7d810a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.759+0000 7fa7dd200640 1 --2- 172.21.15.86:0/2389134070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d8102fe0 0x7fa7d8103460 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.759+0000 7fa7dd200640 1 --2- 172.21.15.86:0/2389134070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d8101e90 0x7fa7d81022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.759+0000 7fa7dd200640 1 -- 172.21.15.86:0/2389134070 >> 172.21.15.86:0/2389134070 conn(0x7fa7d80fd400 msgr2=0x7fa7d80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.759+0000 7fa7dd200640 1 -- 172.21.15.86:0/2389134070 shutdown_connections 2024-04-17T07:19:07.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.759+0000 7fa7dd200640 1 -- 172.21.15.86:0/2389134070 wait complete. 2024-04-17T07:19:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.760+0000 7fa7dd200640 1 Processor -- start 2024-04-17T07:19:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.761+0000 7fa7dd200640 1 -- start start 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.761+0000 7fa7dd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d8101e90 0x7fa7d81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.761+0000 7fa7dd200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d8102fe0 0x7fa7d81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.761+0000 7fa7dd200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d81039a0 0x7fa7d819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.761+0000 7fa7dd200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d810da30 con 0x7fa7d8101e90 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.761+0000 7fa7dd200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d810d8b0 con 0x7fa7d81039a0 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.761+0000 7fa7dd200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d810dbb0 con 0x7fa7d8102fe0 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.761+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d8101e90 0x7fa7d81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.761+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d8101e90 0x7fa7d81980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48444/0 (socket says 172.21.15.86:48444) 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.762+0000 7fa7d7400640 1 -- 172.21.15.86:0/3092900038 learned_addr learned my addr 172.21.15.86:0/3092900038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.762+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d81039a0 0x7fa7d819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.762+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d8102fe0 0x7fa7d81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.762+0000 7fa7d7400640 1 -- 172.21.15.86:0/3092900038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d8102fe0 msgr2=0x7fa7d81985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.762+0000 7fa7d7400640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d8102fe0 0x7fa7d81985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.762+0000 7fa7d7400640 1 -- 172.21.15.86:0/3092900038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d81039a0 msgr2=0x7fa7d819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.762+0000 7fa7d7400640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d81039a0 0x7fa7d819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.762+0000 7fa7d7400640 1 -- 172.21.15.86:0/3092900038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d819fcc0 con 0x7fa7d8101e90 2024-04-17T07:19:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.763+0000 7fa7d7400640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d8101e90 0x7fa7d81980a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800e9f0 tx=0x7fa7c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.763+0000 7fa7cfe00640 1 -- 172.21.15.86:0/3092900038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c800cd80 con 0x7fa7d8101e90 2024-04-17T07:19:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.763+0000 7fa7cfe00640 1 -- 172.21.15.86:0/3092900038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7c80108e0 con 0x7fa7d8101e90 2024-04-17T07:19:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.763+0000 7fa7dd200640 1 -- 172.21.15.86:0/3092900038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d819ffb0 con 0x7fa7d8101e90 2024-04-17T07:19:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.763+0000 7fa7cfe00640 1 -- 172.21.15.86:0/3092900038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8018920 con 0x7fa7d8101e90 2024-04-17T07:19:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.763+0000 7fa7dd200640 1 -- 172.21.15.86:0/3092900038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d81a0420 con 0x7fa7d8101e90 2024-04-17T07:19:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.767+0000 7fa7e15f9740 1 Processor -- start 2024-04-17T07:19:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.767+0000 7fa7e15f9740 1 -- start start 2024-04-17T07:19:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.767+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 0x55e2c3534990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.767+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 0x55e2c3539a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.767+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c353c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.767+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2c3496020 con 0x55e2c353a0d0 2024-04-17T07:19:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.767+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e2c343c460 con 0x55e2c3534570 2024-04-17T07:19:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.767+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e2c3491fe0 con 0x55e2c3534f60 2024-04-17T07:19:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.768+0000 7fa7cfe00640 1 -- 172.21.15.86:0/3092900038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa7c8010a90 con 0x7fa7d8101e90 2024-04-17T07:19:07.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.768+0000 7fa7d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c353c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.768+0000 7fa7d7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 0x55e2c3534990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.768+0000 7fa7d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c353c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48448/0 (socket says 172.21.15.86:48448) 2024-04-17T07:19:07.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.768+0000 7fa7d6a00640 1 -- 172.21.15.86:0/1170214244 learned_addr learned my addr 172.21.15.86:0/1170214244 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.768+0000 7fa7d7400640 1 --2- 172.21.15.86:0/1170214244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 0x55e2c3539a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.769+0000 7fa7d7e00640 1 -- 172.21.15.86:0/1170214244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 msgr2=0x55e2c3539a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.769+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/1170214244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 0x55e2c3539a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.769+0000 7fa7d7e00640 1 -- 172.21.15.86:0/1170214244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 msgr2=0x55e2c353c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.769+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/1170214244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c353c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.769+0000 7fa7d7e00640 1 -- 172.21.15.86:0/1170214244 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2c3444040 con 0x55e2c3534570 2024-04-17T07:19:07.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.769+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/1170214244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c353c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.770+0000 7fa7cfe00640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7bc0a3c30 0x7fa7bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.770+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/1170214244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 0x55e2c3534990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009f00 tx=0x7fa7d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa4e0dbecb9c8919 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.770+0000 7fa7cfe00640 1 -- 172.21.15.86:0/3092900038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa7c80d09a0 con 0x7fa7d8101e90 2024-04-17T07:19:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.770+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1170214244 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0004d20 con 0x55e2c3534570 2024-04-17T07:19:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.770+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1170214244 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7d0004030 con 0x55e2c3534570 2024-04-17T07:19:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.770+0000 7fa7ccc00640 1 -- 172.21.15.86:0/1170214244 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0005280 con 0x55e2c3534570 2024-04-17T07:19:07.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.770+0000 7fa7d7400640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7bc0a3c30 0x7fa7bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.771+0000 7fa7e15f9740 1 -- 172.21.15.86:0/1170214244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 msgr2=0x55e2c3534990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.771+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/1170214244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 0x55e2c3534990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009f00 tx=0x7fa7d0011810 comp rx=0 tx=0).stop 2024-04-17T07:19:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.771+0000 7fa7d7400640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7bc0a3c30 0x7fa7bc0a60f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80ce490 tx=0x7fa7c80d3f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.772+0000 7fa7e15f9740 1 -- 172.21.15.86:0/1170214244 shutdown_connections 2024-04-17T07:19:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.772+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/1170214244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c353c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.772+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/1170214244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 0x55e2c3539a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.772+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/1170214244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 0x55e2c3534990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.772+0000 7fa7e15f9740 1 -- 172.21.15.86:0/1170214244 >> 172.21.15.86:0/1170214244 conn(0x55e2c352fae0 msgr2=0x55e2c352fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.772+0000 7fa7e15f9740 1 -- 172.21.15.86:0/1170214244 shutdown_connections 2024-04-17T07:19:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.772+0000 7fa7e15f9740 1 -- 172.21.15.86:0/1170214244 wait complete. 2024-04-17T07:19:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.772+0000 7fa7e15f9740 1 -- start start 2024-04-17T07:19:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.772+0000 7fa7e15f9740 1 Processor -- start 2024-04-17T07:19:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.773+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 0x55e2c353eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.773+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 0x55e2c353f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.773+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c3546090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.774+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2c353d080 con 0x55e2c353a0d0 2024-04-17T07:19:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.774+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e2c344f830 con 0x55e2c3534570 2024-04-17T07:19:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.774+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e2c353d200 con 0x55e2c3534f60 2024-04-17T07:19:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.774+0000 7fa7d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c3546090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.774+0000 7fa7d6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 0x55e2c353f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.774+0000 7fa7e15f9740 10 client.0 _ll_register_callbacks cb 0x55e2c341dfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.774+0000 7fa7d6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 0x55e2c353f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:57398/0 (socket says 172.21.15.86:57398) 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.774+0000 7fa7d6a00640 1 -- 172.21.15.86:0/2928365797 learned_addr learned my addr 172.21.15.86:0/2928365797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.774+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 0x55e2c353eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.775+0000 7fa7d7e00640 1 -- 172.21.15.86:0/2928365797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 msgr2=0x55e2c353f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.775+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 0x55e2c353f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.775+0000 7fa7d7e00640 1 -- 172.21.15.86:0/2928365797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 msgr2=0x55e2c353eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.775+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 0x55e2c353eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.775+0000 7fa7d7e00640 1 -- 172.21.15.86:0/2928365797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2c3453920 con 0x55e2c353a0d0 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.775+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c3546090 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0005b40 tx=0x7fa7d0005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e5e9bf7313fadf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.775+0000 7fa7c1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:07.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.776+0000 7fa7c1600640 1 -- 172.21.15.86:0/2928365797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d001e040 con 0x55e2c353a0d0 2024-04-17T07:19:07.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.776+0000 7fa7c1600640 1 -- 172.21.15.86:0/2928365797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa7d0004620 con 0x55e2c353a0d0 2024-04-17T07:19:07.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.776+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2928365797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d0004030 con 0x55e2c353a0d0 2024-04-17T07:19:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.776+0000 7fa7c1600640 1 -- 172.21.15.86:0/2928365797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7d003a050 con 0x55e2c353a0d0 2024-04-17T07:19:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.776+0000 7fa7e15f9740 10 client.16782 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.776+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2928365797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2c34217f0 con 0x55e2c353a0d0 2024-04-17T07:19:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.777+0000 7fa7c1600640 1 -- 172.21.15.86:0/2928365797 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7d0004030 con 0x55e2c353a0d0 2024-04-17T07:19:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.777+0000 7fa7c1600640 1 -- 172.21.15.86:0/2928365797 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa7d0018070 con 0x55e2c353a0d0 2024-04-17T07:19:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.777+0000 7fa7e15f9740 10 client.16782 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.777+0000 7fa7e15f9740 10 client.16782 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:19:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.777+0000 7fa7e15f9740 20 client.16782 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:07.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.777+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2c352c470 0x55e2c352e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.777+0000 7fa7e15f9740 4 client.16782 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:07.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.777+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2928365797 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2c34a3a40 con 0x55e2c352c470 2024-04-17T07:19:07.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.778+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2c352c470 0x55e2c352e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.779+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2c352c470 0x55e2c352e860 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f53a5e2fc11d076 server_cookie=4812491630b0b2e2 in_seq=0 out_seq=0 2024-04-17T07:19:07.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.779+0000 7fa7c1600640 10 client.16782 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:19:07.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7c1600640 1 -- 172.21.15.86:0/2928365797 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2c34a3a40 con 0x55e2c352c470 2024-04-17T07:19:07.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7c1600640 10 client.16782 handle_command_reply: tid=0 2024-04-17T07:19:07.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7e15f9740 1 client.16782 shutdown 2024-04-17T07:19:07.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2928365797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 msgr2=0x55e2c3546090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c3546090 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0005b40 tx=0x7fa7d0005a50 comp rx=0 tx=0).stop 2024-04-17T07:19:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2928365797 shutdown_connections 2024-04-17T07:19:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c353a0d0 0x55e2c3546090 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534f60 0x55e2c353f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2c352c470 0x55e2c352e860 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/2928365797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534570 0x55e2c353eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2928365797 >> 172.21.15.86:0/2928365797 conn(0x55e2c352fae0 msgr2=0x55e2c352a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.781+0000 7fa7c1600640 0 client.16782 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:19:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.782+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2928365797 shutdown_connections 2024-04-17T07:19:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.782+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2928365797 wait complete. 2024-04-17T07:19:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.782+0000 7fa7e15f9740 20 client.16782 trim_cache size 0 max 16384 2024-04-17T07:19:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.812+0000 7fa7e15f9740 1 Processor -- start 2024-04-17T07:19:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.812+0000 7fa7e15f9740 1 -- start start 2024-04-17T07:19:07.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.813+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534570 0x55e2c353d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.813+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534f60 0x55e2c353dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.813+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c353a0d0 0x55e2c352a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.813+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81a8680 con 0x55e2c3534570 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.813+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81aa1c0 con 0x55e2c3534f60 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.813+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81a8d50 con 0x55e2c353a0d0 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.813+0000 7fa7d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534570 0x55e2c353d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.813+0000 7fa7d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534570 0x55e2c353d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48464/0 (socket says 172.21.15.86:48464) 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.813+0000 7fa7d7e00640 1 -- 172.21.15.86:0/2701240252 learned_addr learned my addr 172.21.15.86:0/2701240252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.813+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/2701240252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534f60 0x55e2c353dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.814+0000 7fa7d7e00640 1 -- 172.21.15.86:0/2701240252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c353a0d0 msgr2=0x55e2c352a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.814+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/2701240252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c353a0d0 0x55e2c352a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.814+0000 7fa7d7e00640 1 -- 172.21.15.86:0/2701240252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534f60 msgr2=0x55e2c353dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.814+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/2701240252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534f60 0x55e2c353dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.814+0000 7fa7d7e00640 1 -- 172.21.15.86:0/2701240252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2c31a83f0 con 0x55e2c3534570 2024-04-17T07:19:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.814+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/2701240252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534570 0x55e2c353d5a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0000f80 tx=0x7fa7d0009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea5ef62057a7151 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.815+0000 7fa7c3e00640 1 -- 172.21.15.86:0/2701240252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0003ff0 con 0x55e2c3534570 2024-04-17T07:19:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.815+0000 7fa7c3e00640 1 -- 172.21.15.86:0/2701240252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7d000f480 con 0x55e2c3534570 2024-04-17T07:19:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.815+0000 7fa7c3e00640 1 -- 172.21.15.86:0/2701240252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0004880 con 0x55e2c3534570 2024-04-17T07:19:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.815+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2701240252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534570 msgr2=0x55e2c353d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.815+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/2701240252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534570 0x55e2c353d5a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0000f80 tx=0x7fa7d0009e00 comp rx=0 tx=0).stop 2024-04-17T07:19:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.816+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2701240252 shutdown_connections 2024-04-17T07:19:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.816+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/2701240252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c353a0d0 0x55e2c352a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.816+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/2701240252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c3534f60 0x55e2c353dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.816+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/2701240252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534570 0x55e2c353d5a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.816+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2701240252 >> 172.21.15.86:0/2701240252 conn(0x55e2c352fae0 msgr2=0x55e2c35364f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.816+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2701240252 shutdown_connections 2024-04-17T07:19:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.816+0000 7fa7e15f9740 1 -- 172.21.15.86:0/2701240252 wait complete. 2024-04-17T07:19:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.816+0000 7fa7e15f9740 1 -- start start 2024-04-17T07:19:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.817+0000 7fa7e15f9740 1 Processor -- start 2024-04-17T07:19:07.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.817+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534570 0x55e2c354ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.817+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534f60 0x55e2c354d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.817+0000 7fa7e15f9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c353a0d0 0x55e2c354d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.817+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81a9af0 con 0x55e2c3534f60 2024-04-17T07:19:07.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.817+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81a9420 con 0x55e2c353a0d0 2024-04-17T07:19:07.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.817+0000 7fa7e15f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81a6650 con 0x55e2c3534570 2024-04-17T07:19:07.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.817+0000 7fa7e15f9740 10 client.0 _ll_register_callbacks cb 0x7fa7d0003df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:07.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.818+0000 7fa7d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534f60 0x55e2c354d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.818+0000 7fa7d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534f60 0x55e2c354d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48480/0 (socket says 172.21.15.86:48480) 2024-04-17T07:19:07.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.818+0000 7fa7d6a00640 1 -- 172.21.15.86:0/3542092812 learned_addr learned my addr 172.21.15.86:0/3542092812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.818+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c353a0d0 0x55e2c354d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.818+0000 7fa7d6a00640 1 -- 172.21.15.86:0/3542092812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534570 msgr2=0x55e2c354ce20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:19:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.818+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534570 0x55e2c354ce20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.818+0000 7fa7d6a00640 1 -- 172.21.15.86:0/3542092812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c353a0d0 msgr2=0x55e2c354d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:07.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.818+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c353a0d0 0x55e2c354d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.818+0000 7fa7d6a00640 1 -- 172.21.15.86:0/3542092812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2c32727d0 con 0x55e2c3534f60 2024-04-17T07:19:07.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.818+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c353a0d0 0x55e2c354d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:19:07.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.819+0000 7fa7d6a00640 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534f60 0x55e2c354d360 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4000c00 tx=0x7fa7c402cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90cde8a939d0522 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:07.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.819+0000 7fa7c1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:07.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.819+0000 7fa7c1600640 1 -- 172.21.15.86:0/3542092812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4004390 con 0x55e2c3534f60 2024-04-17T07:19:07.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.820+0000 7fa7c1600640 1 -- 172.21.15.86:0/3542092812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa7c4005030 con 0x55e2c3534f60 2024-04-17T07:19:07.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.820+0000 7fa7e15f9740 1 -- 172.21.15.86:0/3542092812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d000f480 con 0x55e2c3534f60 2024-04-17T07:19:07.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.820+0000 7fa7c1600640 1 -- 172.21.15.86:0/3542092812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7c4055050 con 0x55e2c3534f60 2024-04-17T07:19:07.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.820+0000 7fa7e15f9740 10 client.16788 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:07.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.820+0000 7fa7e15f9740 1 -- 172.21.15.86:0/3542092812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2c35abb00 con 0x55e2c3534f60 2024-04-17T07:19:07.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.821+0000 7fa7c1600640 1 -- 172.21.15.86:0/3542092812 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7d000f480 con 0x55e2c3534f60 2024-04-17T07:19:07.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.821+0000 7fa7c1600640 1 -- 172.21.15.86:0/3542092812 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa7c40035f0 con 0x55e2c3534f60 2024-04-17T07:19:07.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.823+0000 7fa7e15f9740 10 client.16788 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:07.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.823+0000 7fa7e15f9740 10 client.16788 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:19:07.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.823+0000 7fa7e15f9740 20 client.16788 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:07.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.823+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2c35ac460 0x55e2c35ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:07.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.823+0000 7fa7e15f9740 4 client.16788 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:19:07.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.823+0000 7fa7e15f9740 1 -- 172.21.15.86:0/3542092812 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2c349d8b0 con 0x55e2c35ac460 2024-04-17T07:19:07.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.824+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2c35ac460 0x55e2c35ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.826+0000 7fa7d7e00640 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2c35ac460 0x55e2c35ae8a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b43abcec27c5d85 server_cookie=9739379cfe01d10b in_seq=0 out_seq=0 2024-04-17T07:19:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.826+0000 7fa7c1600640 10 client.16788 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:19:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.826+0000 7fa7c1600640 1 -- 172.21.15.86:0/3542092812 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2c349d8b0 con 0x55e2c35ac460 2024-04-17T07:19:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.827+0000 7fa7c1600640 10 client.16788 handle_command_reply: tid=0 2024-04-17T07:19:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.827+0000 7fa7e15f9740 1 client.16788 shutdown 2024-04-17T07:19:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.828+0000 7fa7e15f9740 1 -- 172.21.15.86:0/3542092812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534f60 msgr2=0x55e2c354d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.828+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534f60 0x55e2c354d360 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4000c00 tx=0x7fa7c402cd10 comp rx=0 tx=0).stop 2024-04-17T07:19:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.829+0000 7fa7e15f9740 1 -- 172.21.15.86:0/3542092812 shutdown_connections 2024-04-17T07:19:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.829+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2c35ac460 0x55e2c35ae8a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.829+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2c353a0d0 0x55e2c354d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.829+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2c3534f60 0x55e2c354d360 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.829+0000 7fa7e15f9740 1 --2- 172.21.15.86:0/3542092812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2c3534570 0x55e2c354ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.829+0000 7fa7c1600640 0 client.16788 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:19:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.829+0000 7fa7e15f9740 1 -- 172.21.15.86:0/3542092812 >> 172.21.15.86:0/3542092812 conn(0x55e2c352fae0 msgr2=0x55e2c35364f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.829+0000 7fa7e15f9740 1 -- 172.21.15.86:0/3542092812 shutdown_connections 2024-04-17T07:19:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.829+0000 7fa7e15f9740 1 -- 172.21.15.86:0/3542092812 wait complete. 2024-04-17T07:19:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.830+0000 7fa7e15f9740 20 client.16788 trim_cache size 0 max 16384 2024-04-17T07:19:07.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:07 smithi086 ceph-mon[31733]: pgmap v583: 129 pgs: 129 active+clean; 811 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 23 op/s 2024-04-17T07:19:07.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:07 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:19:07.831 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:19:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.831+0000 7fa7dd200640 1 -- 172.21.15.86:0/3092900038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7bc0a3c30 msgr2=0x7fa7bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.832+0000 7fa7dd200640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7bc0a3c30 0x7fa7bc0a60f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80ce490 tx=0x7fa7c80d3f90 comp rx=0 tx=0).stop 2024-04-17T07:19:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.832+0000 7fa7dd200640 1 -- 172.21.15.86:0/3092900038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d8101e90 msgr2=0x7fa7d81980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.832+0000 7fa7dd200640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d8101e90 0x7fa7d81980a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800e9f0 tx=0x7fa7c800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.832+0000 7fa7dd200640 1 -- 172.21.15.86:0/3092900038 shutdown_connections 2024-04-17T07:19:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.832+0000 7fa7dd200640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7bc0a3c30 0x7fa7bc0a60f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.832+0000 7fa7dd200640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7d81039a0 0x7fa7d819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.832+0000 7fa7dd200640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7d8102fe0 0x7fa7d81985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.832+0000 7fa7dd200640 1 --2- 172.21.15.86:0/3092900038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7d8101e90 0x7fa7d81980a0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.832+0000 7fa7dd200640 1 -- 172.21.15.86:0/3092900038 >> 172.21.15.86:0/3092900038 conn(0x7fa7d80fd400 msgr2=0x7fa7d81063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:07.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.833+0000 7fa7dd200640 1 -- 172.21.15.86:0/3092900038 shutdown_connections 2024-04-17T07:19:07.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:07.833+0000 7fa7dd200640 1 -- 172.21.15.86:0/3092900038 wait complete. 2024-04-17T07:19:07.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:19:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:07 smithi098 ceph-mon[35418]: pgmap v583: 129 pgs: 129 active+clean; 811 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 23 op/s 2024-04-17T07:19:08.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:07 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:19:08.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:07 smithi103 ceph-mon[35495]: pgmap v583: 129 pgs: 129 active+clean; 811 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 23 op/s 2024-04-17T07:19:08.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:07 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:19:08.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.089+0000 7f6eb2000640 1 -- 172.21.15.86:0/2182936040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eac1041e0 msgr2=0x7f6eac104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:08.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.089+0000 7f6eb2000640 1 --2- 172.21.15.86:0/2182936040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eac1041e0 0x7f6eac104600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009860 tx=0x7f6e9c02bb90 comp rx=0 tx=0).stop 2024-04-17T07:19:08.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.089+0000 7f6eb2000640 1 -- 172.21.15.86:0/2182936040 shutdown_connections 2024-04-17T07:19:08.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.089+0000 7f6eb2000640 1 --2- 172.21.15.86:0/2182936040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6eac105d90 0x7f6eac10c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.089+0000 7f6eb2000640 1 --2- 172.21.15.86:0/2182936040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eac1053d0 0x7f6eac105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.089+0000 7f6eb2000640 1 --2- 172.21.15.86:0/2182936040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eac1041e0 0x7f6eac104600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.089+0000 7f6eb2000640 1 -- 172.21.15.86:0/2182936040 >> 172.21.15.86:0/2182936040 conn(0x7f6eac0ff750 msgr2=0x7f6eac101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:08.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.089+0000 7f6eb2000640 1 -- 172.21.15.86:0/2182936040 shutdown_connections 2024-04-17T07:19:08.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.089+0000 7f6eb2000640 1 -- 172.21.15.86:0/2182936040 wait complete. 2024-04-17T07:19:08.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.090+0000 7f6eb2000640 1 Processor -- start 2024-04-17T07:19:08.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eb2000640 1 -- start start 2024-04-17T07:19:08.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eb2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eac1041e0 0x7f6eac19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eb2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eac1053d0 0x7f6eac19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eb2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6eac105d90 0x7f6eac1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eb2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac10fcf0 con 0x7f6eac1041e0 2024-04-17T07:19:08.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eb2000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac10fb70 con 0x7f6eac1053d0 2024-04-17T07:19:08.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eb2000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac10fe70 con 0x7f6eac105d90 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eac1041e0 0x7f6eac19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6eac105d90 0x7f6eac1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6eac105d90 0x7f6eac1a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:57436/0 (socket says 172.21.15.86:57436) 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eabe00640 1 -- 172.21.15.86:0/2699119447 learned_addr learned my addr 172.21.15.86:0/2699119447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eac1041e0 0x7f6eac19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48496/0 (socket says 172.21.15.86:48496) 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.091+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eac1053d0 0x7f6eac19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.092+0000 7f6eabe00640 1 -- 172.21.15.86:0/2699119447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eac1053d0 msgr2=0x7f6eac19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.092+0000 7f6eabe00640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eac1053d0 0x7f6eac19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.092+0000 7f6eabe00640 1 -- 172.21.15.86:0/2699119447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eac1041e0 msgr2=0x7f6eac19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.092+0000 7f6eabe00640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eac1041e0 0x7f6eac19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.092+0000 7f6eabe00640 1 -- 172.21.15.86:0/2699119447 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac1a20a0 con 0x7f6eac105d90 2024-04-17T07:19:08.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.092+0000 7f6eabe00640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6eac105d90 0x7f6eac1a1a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400efd0 tx=0x7f6ea400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:08.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.093+0000 7f6ea3e00640 1 -- 172.21.15.86:0/2699119447 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4003b70 con 0x7f6eac105d90 2024-04-17T07:19:08.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.093+0000 7f6ea3e00640 1 -- 172.21.15.86:0/2699119447 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea4003d10 con 0x7f6eac105d90 2024-04-17T07:19:08.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.093+0000 7f6ea3e00640 1 -- 172.21.15.86:0/2699119447 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4010850 con 0x7f6eac105d90 2024-04-17T07:19:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.093+0000 7f6eb2000640 1 -- 172.21.15.86:0/2699119447 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac1a2390 con 0x7f6eac105d90 2024-04-17T07:19:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.093+0000 7f6eb2000640 1 -- 172.21.15.86:0/2699119447 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac1a2890 con 0x7f6eac105d90 2024-04-17T07:19:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.096+0000 7f6eb4c08740 1 Processor -- start 2024-04-17T07:19:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.097+0000 7f6ea3e00640 1 -- 172.21.15.86:0/2699119447 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f6ea4004000 con 0x7f6eac105d90 2024-04-17T07:19:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.097+0000 7f6eb4c08740 1 -- start start 2024-04-17T07:19:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fceefa0 0x55f61fcef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcef990 0x55f61fcf4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fcf4b00 0x55f61fcf6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f61fc55020 con 0x55f61fcef990 2024-04-17T07:19:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f61fbfb460 con 0x55f61fcf4b00 2024-04-17T07:19:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f61fc50fe0 con 0x55f61fceefa0 2024-04-17T07:19:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eaaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fceefa0 0x55f61fcef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcef990 0x55f61fcf4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eaaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fceefa0 0x55f61fcef3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:57438/0 (socket says 172.21.15.86:57438) 2024-04-17T07:19:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eaaa00640 1 -- 172.21.15.86:0/4096459637 learned_addr learned my addr 172.21.15.86:0/4096459637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:08.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.098+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/4096459637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fcf4b00 0x55f61fcf6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.099+0000 7f6eab400640 1 -- 172.21.15.86:0/4096459637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fceefa0 msgr2=0x55f61fcef3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.099+0000 7f6eab400640 1 --2- 172.21.15.86:0/4096459637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fceefa0 0x55f61fcef3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.099+0000 7f6eab400640 1 -- 172.21.15.86:0/4096459637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fcf4b00 msgr2=0x55f61fcf6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.099+0000 7f6eab400640 1 --2- 172.21.15.86:0/4096459637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fcf4b00 0x55f61fcf6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.099+0000 7f6eab400640 1 -- 172.21.15.86:0/4096459637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61fc03040 con 0x55f61fcef990 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.099+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/4096459637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fceefa0 0x55f61fcef3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.099+0000 7f6eab400640 1 --2- 172.21.15.86:0/4096459637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcef990 0x55f61fcf4490 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c02c0d0 tx=0x7f6e9c009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e2ac6a33c1d2ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.099+0000 7f6ea3e00640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e900a3bb0 0x7f6e900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.099+0000 7f6e97e00640 1 -- 172.21.15.86:0/4096459637 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c043020 con 0x55f61fcef990 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.099+0000 7f6ea3e00640 1 -- 172.21.15.86:0/2699119447 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6ea40d0b80 con 0x7f6eac105d90 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.100+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e900a3bb0 0x7f6e900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.100+0000 7f6e97e00640 1 -- 172.21.15.86:0/4096459637 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e9c004040 con 0x55f61fcef990 2024-04-17T07:19:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.100+0000 7f6e97e00640 1 -- 172.21.15.86:0/4096459637 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c039040 con 0x55f61fcef990 2024-04-17T07:19:08.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.101+0000 7f6eb4c08740 1 -- 172.21.15.86:0/4096459637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcef990 msgr2=0x55f61fcf4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:08.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.101+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/4096459637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcef990 0x55f61fcf4490 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c02c0d0 tx=0x7f6e9c009490 comp rx=0 tx=0).stop 2024-04-17T07:19:08.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.101+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e900a3bb0 0x7f6e900a6070 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55f61fcf01f0 tx=0x7f6e98013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:08.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.101+0000 7f6eb4c08740 1 -- 172.21.15.86:0/4096459637 shutdown_connections 2024-04-17T07:19:08.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.101+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/4096459637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fcf4b00 0x55f61fcf6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.101+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/4096459637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcef990 0x55f61fcf4490 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.101+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/4096459637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fceefa0 0x55f61fcef3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.101+0000 7f6eb4c08740 1 -- 172.21.15.86:0/4096459637 >> 172.21.15.86:0/4096459637 conn(0x55f61fcea510 msgr2=0x55f61fcea910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:08.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.101+0000 7f6eb4c08740 1 -- 172.21.15.86:0/4096459637 shutdown_connections 2024-04-17T07:19:08.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.101+0000 7f6eb4c08740 1 -- 172.21.15.86:0/4096459637 wait complete. 2024-04-17T07:19:08.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.102+0000 7f6eb4c08740 1 -- start start 2024-04-17T07:19:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.102+0000 7f6eb4c08740 1 Processor -- start 2024-04-17T07:19:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.102+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fcf9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.102+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fcf99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.102+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fd00a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.102+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f61fcf7a50 con 0x55f61fcf4b00 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.102+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f61fc0e830 con 0x55f61fceefa0 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.102+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f61fcf7bd0 con 0x55f61fcef990 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.102+0000 7f6eb4c08740 10 client.0 _ll_register_callbacks cb 0x55f61fbdcfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.103+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fd00a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.103+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fd00a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48502/0 (socket says 172.21.15.86:48502) 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.103+0000 7f6eaaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fcf99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.103+0000 7f6eabe00640 1 -- 172.21.15.86:0/1304634742 learned_addr learned my addr 172.21.15.86:0/1304634742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.103+0000 7f6eab400640 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fcf9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.103+0000 7f6eaaa00640 1 -- 172.21.15.86:0/1304634742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 msgr2=0x55f61fcf9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.103+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fcf9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.103+0000 7f6eaaa00640 1 -- 172.21.15.86:0/1304634742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 msgr2=0x55f61fd00a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.103+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fd00a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.103+0000 7f6eaaa00640 1 -- 172.21.15.86:0/1304634742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61fc12920 con 0x55f61fcef990 2024-04-17T07:19:08.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.104+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fcf99c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e980058c0 tx=0x7f6e98009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b829f56c34ba40f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:08.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.104+0000 7f6e94c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.104+0000 7f6e94c00640 1 -- 172.21.15.86:0/1304634742 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98026070 con 0x55f61fcef990 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.104+0000 7f6eb4c08740 1 -- 172.21.15.86:0/1304634742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e9c004040 con 0x55f61fcef990 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.104+0000 7f6e94c00640 1 -- 172.21.15.86:0/1304634742 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6e9801fae0 con 0x55f61fcef990 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6e94c00640 1 -- 172.21.15.86:0/1304634742 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e98035050 con 0x55f61fcef990 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6eb4c08740 10 client.25837 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6eb4c08740 1 -- 172.21.15.86:0/1304634742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f61fbe07f0 con 0x55f61fcef990 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6e94c00640 1 -- 172.21.15.86:0/1304634742 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e9c004040 con 0x55f61fcef990 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6e94c00640 1 -- 172.21.15.86:0/1304634742 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6e9802a400 con 0x55f61fcef990 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6eb4c08740 10 client.25837 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6eb4c08740 10 client.25837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6eb4c08740 20 client.25837 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f61fd04f10 0x55f61fd073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6eb4c08740 4 client.25837 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.105+0000 7f6eb4c08740 1 -- 172.21.15.86:0/1304634742 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f61fc62a40 con 0x55f61fd04f10 2024-04-17T07:19:08.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.106+0000 7f6eab400640 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f61fd04f10 0x55f61fd073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.107+0000 7f6e94c00640 10 client.25837 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:08.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.107+0000 7f6eab400640 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f61fd04f10 0x55f61fd073d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f056a9a651e1064b server_cookie=decdc2f732427713 in_seq=0 out_seq=0 2024-04-17T07:19:08.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.108+0000 7f6e94c00640 1 -- 172.21.15.86:0/1304634742 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f61fc62a40 con 0x55f61fd04f10 2024-04-17T07:19:08.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.108+0000 7f6e94c00640 10 client.25837 handle_command_reply: tid=0 2024-04-17T07:19:08.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.108+0000 7f6eb4c08740 1 client.25837 shutdown 2024-04-17T07:19:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 1 -- 172.21.15.86:0/1304634742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 msgr2=0x55f61fcf99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fcf99c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e980058c0 tx=0x7f6e98009260 comp rx=0 tx=0).stop 2024-04-17T07:19:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 1 -- 172.21.15.86:0/1304634742 shutdown_connections 2024-04-17T07:19:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f61fd04f10 0x55f61fd073d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fd00a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fcf99c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/1304634742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fcf9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6e94c00640 0 client.25837 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 1 -- 172.21.15.86:0/1304634742 >> 172.21.15.86:0/1304634742 conn(0x55f61fcea510 msgr2=0x55f61fce9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 1 -- 172.21.15.86:0/1304634742 shutdown_connections 2024-04-17T07:19:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 1 -- 172.21.15.86:0/1304634742 wait complete. 2024-04-17T07:19:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.109+0000 7f6eb4c08740 20 client.25837 trim_cache size 0 max 16384 2024-04-17T07:19:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eb4c08740 1 Processor -- start 2024-04-17T07:19:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eb4c08740 1 -- start start 2024-04-17T07:19:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fce8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fd009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fce93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1accf0 con 0x55f61fcf4b00 2024-04-17T07:19:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1ae830 con 0x55f61fceefa0 2024-04-17T07:19:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1ad3c0 con 0x55f61fcef990 2024-04-17T07:19:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fce93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fce93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48506/0 (socket says 172.21.15.86:48506) 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.131+0000 7f6eabe00640 1 -- 172.21.15.86:0/2762752310 learned_addr learned my addr 172.21.15.86:0/2762752310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6eab400640 1 --2- 172.21.15.86:0/2762752310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fce8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/2762752310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fd009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6eabe00640 1 -- 172.21.15.86:0/2762752310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 msgr2=0x55f61fd009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6eabe00640 1 --2- 172.21.15.86:0/2762752310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fd009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6eabe00640 1 -- 172.21.15.86:0/2762752310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 msgr2=0x55f61fce8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6eabe00640 1 --2- 172.21.15.86:0/2762752310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fce8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6eabe00640 1 -- 172.21.15.86:0/2762752310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61f9673f0 con 0x55f61fcf4b00 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6eabe00640 1 --2- 172.21.15.86:0/2762752310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fce93c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40ce670 tx=0x7f6ea4009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0173d2eab092c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6e97400640 1 -- 172.21.15.86:0/2762752310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40b5020 con 0x55f61fcf4b00 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6e97400640 1 -- 172.21.15.86:0/2762752310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea409d1e0 con 0x55f61fcf4b00 2024-04-17T07:19:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.132+0000 7f6e97400640 1 -- 172.21.15.86:0/2762752310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4099150 con 0x55f61fcf4b00 2024-04-17T07:19:08.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.133+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2762752310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 msgr2=0x55f61fce93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.133+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/2762752310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fce93c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40ce670 tx=0x7f6ea4009f00 comp rx=0 tx=0).stop 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.133+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2762752310 shutdown_connections 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.133+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/2762752310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fce93c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.133+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/2762752310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fd009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.133+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/2762752310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fce8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.133+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2762752310 >> 172.21.15.86:0/2762752310 conn(0x55f61fcea510 msgr2=0x55f61fcf0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.134+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2762752310 shutdown_connections 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.134+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2762752310 wait complete. 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.134+0000 7f6eb4c08740 1 -- start start 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.134+0000 7f6eb4c08740 1 Processor -- start 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fcfe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fcfe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eb4c08740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fcff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1ae160 con 0x55f61fcf4b00 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1ada90 con 0x55f61fceefa0 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eb4c08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1aacc0 con 0x55f61fcef990 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eb4c08740 10 client.0 _ll_register_callbacks cb 0x7f6e9800f790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fcff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fcff7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48522/0 (socket says 172.21.15.86:48522) 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eabe00640 1 -- 172.21.15.86:0/2213977853 learned_addr learned my addr 172.21.15.86:0/2213977853 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eab400640 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fcfe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.135+0000 7f6eaaa00640 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fcfe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.136+0000 7f6eabe00640 1 -- 172.21.15.86:0/2213977853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 msgr2=0x55f61fcfe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.136+0000 7f6eabe00640 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fcfe830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.136+0000 7f6eabe00640 1 -- 172.21.15.86:0/2213977853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 msgr2=0x55f61fcfe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.136+0000 7f6eabe00640 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fcfe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.136+0000 7f6eabe00640 1 -- 172.21.15.86:0/2213977853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61fa317d0 con 0x55f61fcf4b00 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.136+0000 7f6eabe00640 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fcff7f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6ea409c7e0 tx=0x7f6ea40bd230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b481788a6e1e7de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.136+0000 7f6e94c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.136+0000 7f6e94c00640 1 -- 172.21.15.86:0/2213977853 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4080060 con 0x55f61fcf4b00 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.136+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2213977853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea409d1e0 con 0x55f61fcf4b00 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6e94c00640 1 -- 172.21.15.86:0/2213977853 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6ea4004b40 con 0x55f61fcf4b00 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6e94c00640 1 -- 172.21.15.86:0/2213977853 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ea40b5030 con 0x55f61fcf4b00 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6eb4c08740 10 client.16806 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2213977853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f61fd73dc0 con 0x55f61fcf4b00 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6e94c00640 1 -- 172.21.15.86:0/2213977853 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea409d1e0 con 0x55f61fcf4b00 2024-04-17T07:19:08.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6e94c00640 1 -- 172.21.15.86:0/2213977853 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6ea401f020 con 0x55f61fcf4b00 2024-04-17T07:19:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6eb4c08740 10 client.16806 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6eb4c08740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6eb4c08740 20 client.16806 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f61fd746e0 0x55f61fd76b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.137+0000 7f6eb4c08740 4 client.16806 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:19:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.138+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2213977853 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f61fc5c8b0 con 0x55f61fd746e0 2024-04-17T07:19:08.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.138+0000 7f6eab400640 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f61fd746e0 0x55f61fd76b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.139+0000 7f6eab400640 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f61fd746e0 0x55f61fd76b00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced8da52c26ab802 server_cookie=164577517c238a8e in_seq=0 out_seq=0 2024-04-17T07:19:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.139+0000 7f6e94c00640 10 client.16806 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:08.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.139+0000 7f6e94c00640 1 -- 172.21.15.86:0/2213977853 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f61fc5c8b0 con 0x55f61fd746e0 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.139+0000 7f6e94c00640 10 client.16806 handle_command_reply: tid=0 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.139+0000 7f6eb4c08740 1 client.16806 shutdown 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.139+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2213977853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 msgr2=0x55f61fcff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.139+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fcff7f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6ea409c7e0 tx=0x7f6ea40bd230 comp rx=0 tx=0).stop 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.139+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2213977853 shutdown_connections 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.140+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f61fd746e0 0x55f61fd76b00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.140+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f61fcf4b00 0x55f61fcff7f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.140+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f61fcef990 0x55f61fcfe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.140+0000 7f6eb4c08740 1 --2- 172.21.15.86:0/2213977853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f61fceefa0 0x55f61fcfe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.140+0000 7f6e94c00640 0 client.16806 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.140+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2213977853 >> 172.21.15.86:0/2213977853 conn(0x55f61fcea510 msgr2=0x55f61fced770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.140+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2213977853 shutdown_connections 2024-04-17T07:19:08.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.140+0000 7f6eb4c08740 1 -- 172.21.15.86:0/2213977853 wait complete. 2024-04-17T07:19:08.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.140+0000 7f6eb4c08740 20 client.16806 trim_cache size 0 max 16384 2024-04-17T07:19:08.143 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.141+0000 7f6eb2000640 1 -- 172.21.15.86:0/2699119447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e900a3bb0 msgr2=0x7f6e900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.141+0000 7f6eb2000640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e900a3bb0 0x7f6e900a6070 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55f61fcf01f0 tx=0x7f6e98013040 comp rx=0 tx=0).stop 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.141+0000 7f6eb2000640 1 -- 172.21.15.86:0/2699119447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6eac105d90 msgr2=0x7f6eac1a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.141+0000 7f6eb2000640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6eac105d90 0x7f6eac1a1a30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400efd0 tx=0x7f6ea400c6a0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.142+0000 7f6eb2000640 1 -- 172.21.15.86:0/2699119447 shutdown_connections 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.142+0000 7f6eb2000640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e900a3bb0 0x7f6e900a6070 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.142+0000 7f6eb2000640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6eac105d90 0x7f6eac1a1a30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.142+0000 7f6eb2000640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6eac1053d0 0x7f6eac19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.142+0000 7f6eb2000640 1 --2- 172.21.15.86:0/2699119447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eac1041e0 0x7f6eac19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.142+0000 7f6eb2000640 1 -- 172.21.15.86:0/2699119447 >> 172.21.15.86:0/2699119447 conn(0x7f6eac0ff750 msgr2=0x7f6eac1087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.142+0000 7f6eb2000640 1 -- 172.21.15.86:0/2699119447 shutdown_connections 2024-04-17T07:19:08.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:08.142+0000 7f6eb2000640 1 -- 172.21.15.86:0/2699119447 wait complete. 2024-04-17T07:19:09.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:19:09.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:19:09.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:08 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:19:09.161 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:19:09.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:19:09.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:19:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:19:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:19:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:08 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:19:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:19:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:19:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:08 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:19:09.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.409+0000 7fc271400640 1 -- 172.21.15.86:0/4277081856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c105290 msgr2=0x7fc26c105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.409+0000 7fc271400640 1 --2- 172.21.15.86:0/4277081856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c105290 0x7fc26c105710 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc258009900 tx=0x7fc25802bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:09.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.409+0000 7fc271400640 1 -- 172.21.15.86:0/4277081856 shutdown_connections 2024-04-17T07:19:09.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.409+0000 7fc271400640 1 --2- 172.21.15.86:0/4277081856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26c105c50 0x7fc26c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.409+0000 7fc271400640 1 --2- 172.21.15.86:0/4277081856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c105290 0x7fc26c105710 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.409+0000 7fc271400640 1 --2- 172.21.15.86:0/4277081856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc26c1040a0 0x7fc26c1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.409+0000 7fc271400640 1 -- 172.21.15.86:0/4277081856 >> 172.21.15.86:0/4277081856 conn(0x7fc26c0ff610 msgr2=0x7fc26c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.409+0000 7fc271400640 1 -- 172.21.15.86:0/4277081856 shutdown_connections 2024-04-17T07:19:09.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.410+0000 7fc271400640 1 -- 172.21.15.86:0/4277081856 wait complete. 2024-04-17T07:19:09.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.411+0000 7fc271400640 1 Processor -- start 2024-04-17T07:19:09.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.411+0000 7fc271400640 1 -- start start 2024-04-17T07:19:09.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.411+0000 7fc271400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc26c1040a0 0x7fc26c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.411+0000 7fc271400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c105290 0x7fc26c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.411+0000 7fc271400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26c105c50 0x7fc26c1a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.411+0000 7fc271400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c10fc30 con 0x7fc26c105290 2024-04-17T07:19:09.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.411+0000 7fc271400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c10fab0 con 0x7fc26c105c50 2024-04-17T07:19:09.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.411+0000 7fc271400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c10fdb0 con 0x7fc26c1040a0 2024-04-17T07:19:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.411+0000 7fc26b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c105290 0x7fc26c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.411+0000 7fc26b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c105290 0x7fc26c19a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48530/0 (socket says 172.21.15.86:48530) 2024-04-17T07:19:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.412+0000 7fc26b400640 1 -- 172.21.15.86:0/1220143202 learned_addr learned my addr 172.21.15.86:0/1220143202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.412+0000 7fc26be00640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc26c1040a0 0x7fc26c19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.412+0000 7fc270a00640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26c105c50 0x7fc26c1a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.412+0000 7fc26b400640 1 -- 172.21.15.86:0/1220143202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc26c1040a0 msgr2=0x7fc26c19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.412+0000 7fc26b400640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc26c1040a0 0x7fc26c19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.412+0000 7fc26b400640 1 -- 172.21.15.86:0/1220143202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26c105c50 msgr2=0x7fc26c1a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.412+0000 7fc26b400640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26c105c50 0x7fc26c1a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.412+0000 7fc26b400640 1 -- 172.21.15.86:0/1220143202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c1a1ee0 con 0x7fc26c105290 2024-04-17T07:19:09.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.412+0000 7fc26be00640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc26c1040a0 0x7fc26c19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:09.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.413+0000 7fc26b400640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c105290 0x7fc26c19a800 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc258000c00 tx=0x7fc2580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.413+0000 7fc268c00640 1 -- 172.21.15.86:0/1220143202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258043020 con 0x7fc26c105290 2024-04-17T07:19:09.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.413+0000 7fc271400640 1 -- 172.21.15.86:0/1220143202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c1a2170 con 0x7fc26c105290 2024-04-17T07:19:09.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.413+0000 7fc271400640 1 -- 172.21.15.86:0/1220143202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c1a2710 con 0x7fc26c105290 2024-04-17T07:19:09.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.413+0000 7fc268c00640 1 -- 172.21.15.86:0/1220143202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2580054c0 con 0x7fc26c105290 2024-04-17T07:19:09.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.413+0000 7fc268c00640 1 -- 172.21.15.86:0/1220143202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258003d40 con 0x7fc26c105290 2024-04-17T07:19:09.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.417+0000 7fc268c00640 1 -- 172.21.15.86:0/1220143202 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc258046020 con 0x7fc26c105290 2024-04-17T07:19:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.417+0000 7fc275609740 1 Processor -- start 2024-04-17T07:19:09.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.418+0000 7fc275609740 1 -- start start 2024-04-17T07:19:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.418+0000 7fc275609740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 0x5560a4e1b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.418+0000 7fc275609740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 0x5560a4e20590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.418+0000 7fc275609740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.418+0000 7fc275609740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560a4d80cb0 con 0x5560a4e20c00 2024-04-17T07:19:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.418+0000 7fc275609740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5560a4d27370 con 0x5560a4e1b0a0 2024-04-17T07:19:09.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.418+0000 7fc275609740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560a4d7cc70 con 0x5560a4e1ba90 2024-04-17T07:19:09.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.419+0000 7fc270a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 0x5560a4e1b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.419+0000 7fc270a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 0x5560a4e1b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43358/0 (socket says 172.21.15.86:43358) 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.419+0000 7fc270a00640 1 -- 172.21.15.86:0/2439569517 learned_addr learned my addr 172.21.15.86:0/2439569517 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.419+0000 7fc26b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.419+0000 7fc26be00640 1 --2- 172.21.15.86:0/2439569517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 0x5560a4e20590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.419+0000 7fc268c00640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc2500a3bb0 0x7fc2500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.419+0000 7fc270a00640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc2500a3bb0 0x7fc2500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.419+0000 7fc268c00640 1 -- 172.21.15.86:0/1220143202 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc2580f2b90 con 0x7fc26c105290 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.420+0000 7fc26b400640 1 -- 172.21.15.86:0/2439569517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 msgr2=0x5560a4e20590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.420+0000 7fc26b400640 1 --2- 172.21.15.86:0/2439569517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 0x5560a4e20590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.420+0000 7fc26b400640 1 -- 172.21.15.86:0/2439569517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 msgr2=0x5560a4e1b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.420+0000 7fc26b400640 1 --2- 172.21.15.86:0/2439569517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 0x5560a4e1b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.420+0000 7fc26b400640 1 -- 172.21.15.86:0/2439569517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a4d46910 con 0x5560a4e20c00 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.420+0000 7fc26b400640 1 --2- 172.21.15.86:0/2439569517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e23090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc258005ec0 tx=0x7fc25802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8066637184ef56 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.420+0000 7fc261200640 1 -- 172.21.15.86:0/2439569517 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580f8020 con 0x5560a4e20c00 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.420+0000 7fc270a00640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc2500a3bb0 0x7fc2500a6070 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc264005c40 tx=0x7fc264009e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.421+0000 7fc261200640 1 -- 172.21.15.86:0/2439569517 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc258004c50 con 0x5560a4e20c00 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.421+0000 7fc261200640 1 -- 172.21.15.86:0/2439569517 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580c2f60 con 0x5560a4e20c00 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.421+0000 7fc275609740 1 -- 172.21.15.86:0/2439569517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 msgr2=0x5560a4e23090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.421+0000 7fc275609740 1 --2- 172.21.15.86:0/2439569517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e23090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc258005ec0 tx=0x7fc25802bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.422+0000 7fc275609740 1 -- 172.21.15.86:0/2439569517 shutdown_connections 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.422+0000 7fc275609740 1 --2- 172.21.15.86:0/2439569517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e23090 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.422+0000 7fc275609740 1 --2- 172.21.15.86:0/2439569517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 0x5560a4e20590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.422+0000 7fc275609740 1 --2- 172.21.15.86:0/2439569517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 0x5560a4e1b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.422+0000 7fc275609740 1 -- 172.21.15.86:0/2439569517 >> 172.21.15.86:0/2439569517 conn(0x5560a4e16610 msgr2=0x5560a4e16a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.422+0000 7fc275609740 1 -- 172.21.15.86:0/2439569517 shutdown_connections 2024-04-17T07:19:09.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.422+0000 7fc275609740 1 -- 172.21.15.86:0/2439569517 wait complete. 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.422+0000 7fc275609740 1 -- start start 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.423+0000 7fc275609740 1 Processor -- start 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.423+0000 7fc275609740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 0x5560a4e25290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.423+0000 7fc275609740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 0x5560a4e257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.423+0000 7fc275609740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e2c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.423+0000 7fc275609740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560a4d80cb0 con 0x5560a4e20c00 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.423+0000 7fc275609740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5560a4d3a730 con 0x5560a4e1b0a0 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.423+0000 7fc275609740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560a4e23970 con 0x5560a4e1ba90 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.423+0000 7fc275609740 10 client.0 _ll_register_callbacks cb 0x5560a4d3cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.424+0000 7fc270a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e2c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.424+0000 7fc270a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e2c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48558/0 (socket says 172.21.15.86:48558) 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.424+0000 7fc270a00640 1 -- 172.21.15.86:0/760023176 learned_addr learned my addr 172.21.15.86:0/760023176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.424+0000 7fc26b400640 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 0x5560a4e257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.424+0000 7fc26be00640 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 0x5560a4e25290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.424+0000 7fc270a00640 1 -- 172.21.15.86:0/760023176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 msgr2=0x5560a4e257d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.424+0000 7fc270a00640 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 0x5560a4e257d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.424+0000 7fc270a00640 1 -- 172.21.15.86:0/760023176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 msgr2=0x5560a4e25290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.424+0000 7fc270a00640 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 0x5560a4e25290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.424+0000 7fc270a00640 1 -- 172.21.15.86:0/760023176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a4b5b7e0 con 0x5560a4e20c00 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.425+0000 7fc26b400640 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 0x5560a4e257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.425+0000 7fc270a00640 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e2c810 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc264005fd0 tx=0x7fc264009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c9cab1b90ebb04 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.425+0000 7fc255600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.425+0000 7fc255600640 1 -- 172.21.15.86:0/760023176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264004d20 con 0x5560a4e20c00 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.425+0000 7fc255600640 1 -- 172.21.15.86:0/760023176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc2640139c0 con 0x5560a4e20c00 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.425+0000 7fc275609740 1 -- 172.21.15.86:0/760023176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc258004c50 con 0x5560a4e20c00 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.426+0000 7fc255600640 1 -- 172.21.15.86:0/760023176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc264013bc0 con 0x5560a4e20c00 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.426+0000 7fc275609740 10 client.16818 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:09.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.426+0000 7fc275609740 1 -- 172.21.15.86:0/760023176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560a4bbbf70 con 0x5560a4e20c00 2024-04-17T07:19:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.426+0000 7fc255600640 1 -- 172.21.15.86:0/760023176 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc258004c50 con 0x5560a4e20c00 2024-04-17T07:19:09.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.426+0000 7fc255600640 1 -- 172.21.15.86:0/760023176 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc26400f040 con 0x5560a4e20c00 2024-04-17T07:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.426+0000 7fc275609740 10 client.16818 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.426+0000 7fc275609740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.426+0000 7fc275609740 20 client.16818 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.427+0000 7fc275609740 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5560a4e311b0 0x5560a4e33610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.427+0000 7fc275609740 4 client.16818 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.427+0000 7fc275609740 1 -- 172.21.15.86:0/760023176 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560a4d8e6d0 con 0x5560a4e311b0 2024-04-17T07:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.427+0000 7fc26be00640 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5560a4e311b0 0x5560a4e33610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.428+0000 7fc255600640 10 client.16818 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:09.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.428+0000 7fc26be00640 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5560a4e311b0 0x5560a4e33610 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe7021706db2d94 server_cookie=697f4b60225f9e35 in_seq=0 out_seq=0 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.430+0000 7fc255600640 1 -- 172.21.15.86:0/760023176 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560a4d8e6d0 con 0x5560a4e311b0 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.430+0000 7fc255600640 10 client.16818 handle_command_reply: tid=0 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.430+0000 7fc275609740 1 client.16818 shutdown 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.430+0000 7fc275609740 1 -- 172.21.15.86:0/760023176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 msgr2=0x5560a4e2c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.430+0000 7fc275609740 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e2c810 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc264005fd0 tx=0x7fc264009900 comp rx=0 tx=0).stop 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.431+0000 7fc275609740 1 -- 172.21.15.86:0/760023176 shutdown_connections 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.431+0000 7fc275609740 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5560a4e311b0 0x5560a4e33610 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.431+0000 7fc275609740 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e20c00 0x5560a4e2c810 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.431+0000 7fc275609740 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1ba90 0x5560a4e257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.431+0000 7fc275609740 1 --2- 172.21.15.86:0/760023176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1b0a0 0x5560a4e25290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.431+0000 7fc275609740 1 -- 172.21.15.86:0/760023176 >> 172.21.15.86:0/760023176 conn(0x5560a4e16610 msgr2=0x5560a4e15990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.431+0000 7fc255600640 0 client.16818 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.431+0000 7fc275609740 1 -- 172.21.15.86:0/760023176 shutdown_connections 2024-04-17T07:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.431+0000 7fc275609740 1 -- 172.21.15.86:0/760023176 wait complete. 2024-04-17T07:19:09.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.431+0000 7fc275609740 20 client.16818 trim_cache size 0 max 16384 2024-04-17T07:19:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.458+0000 7fc275609740 1 Processor -- start 2024-04-17T07:19:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.458+0000 7fc275609740 1 -- start start 2024-04-17T07:19:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.458+0000 7fc275609740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1b0a0 0x5560a4e23d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.458+0000 7fc275609740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1ba90 0x5560a4e2c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.458+0000 7fc275609740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e20c00 0x5560a4e242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.458+0000 7fc275609740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00fb60 con 0x5560a4e1b0a0 2024-04-17T07:19:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.458+0000 7fc275609740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c0116a0 con 0x5560a4e1ba90 2024-04-17T07:19:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.458+0000 7fc275609740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c010230 con 0x5560a4e20c00 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.458+0000 7fc26be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1b0a0 0x5560a4e23d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.459+0000 7fc26be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1b0a0 0x5560a4e23d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48568/0 (socket says 172.21.15.86:48568) 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.459+0000 7fc26be00640 1 -- 172.21.15.86:0/2670482924 learned_addr learned my addr 172.21.15.86:0/2670482924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.459+0000 7fc26b400640 1 --2- 172.21.15.86:0/2670482924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1ba90 0x5560a4e2c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.459+0000 7fc270a00640 1 --2- 172.21.15.86:0/2670482924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e20c00 0x5560a4e242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.459+0000 7fc26be00640 1 -- 172.21.15.86:0/2670482924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e20c00 msgr2=0x5560a4e242b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.459+0000 7fc26be00640 1 --2- 172.21.15.86:0/2670482924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e20c00 0x5560a4e242b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.459+0000 7fc26be00640 1 -- 172.21.15.86:0/2670482924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1ba90 msgr2=0x5560a4e2c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.459+0000 7fc26be00640 1 --2- 172.21.15.86:0/2670482924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1ba90 0x5560a4e2c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.459+0000 7fc26be00640 1 -- 172.21.15.86:0/2670482924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a4a53170 con 0x5560a4e1b0a0 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.459+0000 7fc26be00640 1 --2- 172.21.15.86:0/2670482924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1b0a0 0x5560a4e23d70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x5560a4e1c500 tx=0x7fc25c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0b3b8c0eb88cff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.460+0000 7fc257e00640 1 -- 172.21.15.86:0/2670482924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c005020 con 0x5560a4e1b0a0 2024-04-17T07:19:09.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.460+0000 7fc257e00640 1 -- 172.21.15.86:0/2670482924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc25c0047a0 con 0x5560a4e1b0a0 2024-04-17T07:19:09.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.460+0000 7fc257e00640 1 -- 172.21.15.86:0/2670482924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c01d290 con 0x5560a4e1b0a0 2024-04-17T07:19:09.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.461+0000 7fc275609740 1 -- 172.21.15.86:0/2670482924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1b0a0 msgr2=0x5560a4e23d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.461+0000 7fc275609740 1 --2- 172.21.15.86:0/2670482924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1b0a0 0x5560a4e23d70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x5560a4e1c500 tx=0x7fc25c0046c0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.462+0000 7fc275609740 1 -- 172.21.15.86:0/2670482924 shutdown_connections 2024-04-17T07:19:09.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.462+0000 7fc275609740 1 --2- 172.21.15.86:0/2670482924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e20c00 0x5560a4e242b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.462+0000 7fc275609740 1 --2- 172.21.15.86:0/2670482924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e1ba90 0x5560a4e2c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.462+0000 7fc275609740 1 --2- 172.21.15.86:0/2670482924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1b0a0 0x5560a4e23d70 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.462+0000 7fc275609740 1 -- 172.21.15.86:0/2670482924 >> 172.21.15.86:0/2670482924 conn(0x5560a4e16610 msgr2=0x5560a4e1a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.462+0000 7fc275609740 1 -- 172.21.15.86:0/2670482924 shutdown_connections 2024-04-17T07:19:09.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.462+0000 7fc275609740 1 -- 172.21.15.86:0/2670482924 wait complete. 2024-04-17T07:19:09.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.462+0000 7fc275609740 1 -- start start 2024-04-17T07:19:09.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.463+0000 7fc275609740 1 Processor -- start 2024-04-17T07:19:09.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.464+0000 7fc275609740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1b0a0 0x5560a4e29d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.464+0000 7fc275609740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1ba90 0x5560a4e2a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.464+0000 7fc275609740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e20c00 0x5560a4e2b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.464+0000 7fc275609740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c010fd0 con 0x5560a4e1ba90 2024-04-17T07:19:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.464+0000 7fc275609740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c010900 con 0x5560a4e20c00 2024-04-17T07:19:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.464+0000 7fc275609740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00db30 con 0x5560a4e1b0a0 2024-04-17T07:19:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.464+0000 7fc275609740 10 client.0 _ll_register_callbacks cb 0x7fc264014910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.464+0000 7fc26b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1ba90 0x5560a4e2a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.464+0000 7fc26b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1ba90 0x5560a4e2a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48576/0 (socket says 172.21.15.86:48576) 2024-04-17T07:19:09.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.464+0000 7fc26b400640 1 -- 172.21.15.86:0/1919197303 learned_addr learned my addr 172.21.15.86:0/1919197303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc26be00640 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1b0a0 0x5560a4e29d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc26b400640 1 -- 172.21.15.86:0/1919197303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1b0a0 msgr2=0x5560a4e29d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc26b400640 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1b0a0 0x5560a4e29d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc270a00640 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e20c00 0x5560a4e2b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc26b400640 1 -- 172.21.15.86:0/1919197303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e20c00 msgr2=0x5560a4e2b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc26b400640 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e20c00 0x5560a4e2b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc26b400640 1 -- 172.21.15.86:0/1919197303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a4e2b8d0 con 0x5560a4e1ba90 2024-04-17T07:19:09.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc26be00640 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1b0a0 0x5560a4e29d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc26b400640 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1ba90 0x5560a4e2a2d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc2580bb580 tx=0x7fc258009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28322e61cc3df936 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc255600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.465+0000 7fc255600640 1 -- 172.21.15.86:0/1919197303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580d8020 con 0x5560a4e1ba90 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.466+0000 7fc255600640 1 -- 172.21.15.86:0/1919197303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc2580035f0 con 0x5560a4e1ba90 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.466+0000 7fc275609740 1 -- 172.21.15.86:0/1919197303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc25c0047a0 con 0x5560a4e1ba90 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.466+0000 7fc255600640 1 -- 172.21.15.86:0/1919197303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc2580f8040 con 0x5560a4e1ba90 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.466+0000 7fc275609740 10 client.16824 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.466+0000 7fc275609740 1 -- 172.21.15.86:0/1919197303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560a4e97800 con 0x5560a4e1ba90 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.467+0000 7fc255600640 1 -- 172.21.15.86:0/1919197303 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc25c0047a0 con 0x5560a4e1ba90 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.467+0000 7fc255600640 1 -- 172.21.15.86:0/1919197303 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc2580c7390 con 0x5560a4e1ba90 2024-04-17T07:19:09.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.467+0000 7fc275609740 10 client.16824 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:09.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.467+0000 7fc275609740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:09.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.467+0000 7fc275609740 20 client.16824 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:09.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.467+0000 7fc275609740 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5560a4e97ff0 0x5560a4e9a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.467+0000 7fc275609740 4 client.16824 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:19:09.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.467+0000 7fc275609740 1 -- 172.21.15.86:0/1919197303 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5560a4d88540 con 0x5560a4e97ff0 2024-04-17T07:19:09.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.468+0000 7fc26be00640 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5560a4e97ff0 0x5560a4e9a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.469+0000 7fc26be00640 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5560a4e97ff0 0x5560a4e9a410 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=281db56a4c6c5b16 server_cookie=96ad6f77ce2f06ca in_seq=0 out_seq=0 2024-04-17T07:19:09.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.469+0000 7fc255600640 10 client.16824 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc255600640 1 -- 172.21.15.86:0/1919197303 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5560a4d88540 con 0x5560a4e97ff0 2024-04-17T07:19:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc255600640 10 client.16824 handle_command_reply: tid=0 2024-04-17T07:19:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 client.16824 shutdown 2024-04-17T07:19:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 -- 172.21.15.86:0/1919197303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1ba90 msgr2=0x5560a4e2a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1ba90 0x5560a4e2a2d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc2580bb580 tx=0x7fc258009260 comp rx=0 tx=0).stop 2024-04-17T07:19:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 -- 172.21.15.86:0/1919197303 shutdown_connections 2024-04-17T07:19:09.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5560a4e97ff0 0x5560a4e9a410 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560a4e20c00 0x5560a4e2b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560a4e1ba90 0x5560a4e2a2d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 --2- 172.21.15.86:0/1919197303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560a4e1b0a0 0x5560a4e29d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc255600640 0 client.16824 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 -- 172.21.15.86:0/1919197303 >> 172.21.15.86:0/1919197303 conn(0x5560a4e16610 msgr2=0x5560a4e15100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 -- 172.21.15.86:0/1919197303 shutdown_connections 2024-04-17T07:19:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 1 -- 172.21.15.86:0/1919197303 wait complete. 2024-04-17T07:19:09.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.474+0000 7fc275609740 20 client.16824 trim_cache size 0 max 16384 2024-04-17T07:19:09.475 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:19:09.476 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:19:09.476 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "7daffc82-9fe9-4f59-b475-d5aa2432985a", 2024-04-17T07:19:09.476 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:19:09.476 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:19:09.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.475+0000 7fc271400640 1 -- 172.21.15.86:0/1220143202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc2500a3bb0 msgr2=0x7fc2500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.475+0000 7fc271400640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc2500a3bb0 0x7fc2500a6070 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc264005c40 tx=0x7fc264009e10 comp rx=0 tx=0).stop 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.475+0000 7fc271400640 1 -- 172.21.15.86:0/1220143202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c105290 msgr2=0x7fc26c19a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.475+0000 7fc271400640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c105290 0x7fc26c19a800 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc258000c00 tx=0x7fc2580042f0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.475+0000 7fc271400640 1 -- 172.21.15.86:0/1220143202 shutdown_connections 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.475+0000 7fc271400640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc2500a3bb0 0x7fc2500a6070 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.475+0000 7fc271400640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26c105c50 0x7fc26c1a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.475+0000 7fc271400640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c105290 0x7fc26c19a800 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.475+0000 7fc271400640 1 --2- 172.21.15.86:0/1220143202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc26c1040a0 0x7fc26c19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.475+0000 7fc271400640 1 -- 172.21.15.86:0/1220143202 >> 172.21.15.86:0/1220143202 conn(0x7fc26c0ff610 msgr2=0x7fc26c108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.476+0000 7fc271400640 1 -- 172.21.15.86:0/1220143202 shutdown_connections 2024-04-17T07:19:09.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.476+0000 7fc271400640 1 -- 172.21.15.86:0/1220143202 wait complete. 2024-04-17T07:19:09.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:19:09.498 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:19:09.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.715+0000 7f93b7200640 1 -- 172.21.15.86:0/179433897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b8105220 msgr2=0x7f93b8107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.715+0000 7f93b7200640 1 --2- 172.21.15.86:0/179433897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b8105220 0x7f93b8107610 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009900 tx=0x7f93a402bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.715+0000 7f93b7200640 1 -- 172.21.15.86:0/179433897 shutdown_connections 2024-04-17T07:19:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.715+0000 7f93b7200640 1 --2- 172.21.15.86:0/179433897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b8107b50 0x7f93b8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.715+0000 7f93b7200640 1 --2- 172.21.15.86:0/179433897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b8105220 0x7f93b8107610 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.715+0000 7f93b7200640 1 --2- 172.21.15.86:0/179433897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93b81028f0 0x7f93b8104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.715+0000 7f93b7200640 1 -- 172.21.15.86:0/179433897 >> 172.21.15.86:0/179433897 conn(0x7f93b80fc300 msgr2=0x7f93b80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.715+0000 7f93b7200640 1 -- 172.21.15.86:0/179433897 shutdown_connections 2024-04-17T07:19:09.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.715+0000 7f93b7200640 1 -- 172.21.15.86:0/179433897 wait complete. 2024-04-17T07:19:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.716+0000 7f93b7200640 1 Processor -- start 2024-04-17T07:19:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b7200640 1 -- start start 2024-04-17T07:19:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b81028f0 0x7f93b819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b8105220 0x7f93b819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b7200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93b8107b50 0x7f93b81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93b8101390 con 0x7f93b81028f0 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b7200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93b8101210 con 0x7f93b8107b50 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93b8101510 con 0x7f93b8105220 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b81028f0 0x7f93b819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b81028f0 0x7f93b819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48592/0 (socket says 172.21.15.86:48592) 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b5e00640 1 -- 172.21.15.86:0/475271580 learned_addr learned my addr 172.21.15.86:0/475271580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.717+0000 7f93b6800640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93b8107b50 0x7f93b81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.718+0000 7f93b5400640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b8105220 0x7f93b819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.718+0000 7f93b6800640 1 -- 172.21.15.86:0/475271580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b8105220 msgr2=0x7f93b819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.718+0000 7f93b6800640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b8105220 0x7f93b819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.718+0000 7f93b6800640 1 -- 172.21.15.86:0/475271580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b81028f0 msgr2=0x7f93b819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.718+0000 7f93b6800640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b81028f0 0x7f93b819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.718+0000 7f93b6800640 1 -- 172.21.15.86:0/475271580 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b81a1ec0 con 0x7f93b8107b50 2024-04-17T07:19:09.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.718+0000 7f93b6800640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93b8107b50 0x7f93b81a1850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f93b000e940 tx=0x7f93b000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.719+0000 7f93aea00640 1 -- 172.21.15.86:0/475271580 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004d20 con 0x7f93b8107b50 2024-04-17T07:19:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.719+0000 7f93b7200640 1 -- 172.21.15.86:0/475271580 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b81a21b0 con 0x7f93b8107b50 2024-04-17T07:19:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.719+0000 7f93aea00640 1 -- 172.21.15.86:0/475271580 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b0005020 con 0x7f93b8107b50 2024-04-17T07:19:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.719+0000 7f93aea00640 1 -- 172.21.15.86:0/475271580 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b00116d0 con 0x7f93b8107b50 2024-04-17T07:19:09.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.719+0000 7f93b7200640 1 -- 172.21.15.86:0/475271580 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b81a26d0 con 0x7f93b8107b50 2024-04-17T07:19:09.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.723+0000 7f93bf98e740 1 Processor -- start 2024-04-17T07:19:09.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.725+0000 7f93aea00640 1 -- 172.21.15.86:0/475271580 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f93b000cb20 con 0x7f93b8107b50 2024-04-17T07:19:09.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.725+0000 7f93bf98e740 1 -- start start 2024-04-17T07:19:09.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.725+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042370 0x560025042790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.725+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025047860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.725+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x56002504a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.725+0000 7f93bf98e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560024fa9120 con 0x560025042d60 2024-04-17T07:19:09.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.725+0000 7f93bf98e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560024f4f340 con 0x560025047ed0 2024-04-17T07:19:09.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.725+0000 7f93bf98e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560024fa50e0 con 0x560025042370 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.726+0000 7f93b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025047860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.726+0000 7f93b5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x56002504a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.726+0000 7f93b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025047860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48604/0 (socket says 172.21.15.86:48604) 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.726+0000 7f93b5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x56002504a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43412/0 (socket says 172.21.15.86:43412) 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.726+0000 7f93b5e00640 1 -- 172.21.15.86:0/2730272857 learned_addr learned my addr 172.21.15.86:0/2730272857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.726+0000 7f93b5400640 1 --2- 172.21.15.86:0/2730272857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042370 0x560025042790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93b5e00640 1 -- 172.21.15.86:0/2730272857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042370 msgr2=0x560025042790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93b5e00640 1 --2- 172.21.15.86:0/2730272857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042370 0x560025042790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93b5e00640 1 -- 172.21.15.86:0/2730272857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 msgr2=0x56002504a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93b5e00640 1 --2- 172.21.15.86:0/2730272857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x56002504a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93b5e00640 1 -- 172.21.15.86:0/2730272857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560024d857d0 con 0x560025042d60 2024-04-17T07:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93b5400640 1 --2- 172.21.15.86:0/2730272857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x56002504a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93b5e00640 1 --2- 172.21.15.86:0/2730272857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025047860 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009dc0 tx=0x7f93a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0f51d4694f2b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93aea00640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f939c0a3bb0 0x7f939c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93a3000640 1 -- 172.21.15.86:0/2730272857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8005220 con 0x560025042d60 2024-04-17T07:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93aea00640 1 -- 172.21.15.86:0/475271580 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93b00d08f0 con 0x7f93b8107b50 2024-04-17T07:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93b5400640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f939c0a3bb0 0x7f939c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93a3000640 1 -- 172.21.15.86:0/2730272857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a8004c50 con 0x560025042d60 2024-04-17T07:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.727+0000 7f93a3000640 1 -- 172.21.15.86:0/2730272857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8019760 con 0x560025042d60 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.729+0000 7f93bf98e740 1 -- 172.21.15.86:0/2730272857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 msgr2=0x560025047860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.729+0000 7f93bf98e740 1 --2- 172.21.15.86:0/2730272857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025047860 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009dc0 tx=0x7f93a8011810 comp rx=0 tx=0).stop 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.729+0000 7f93b5400640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f939c0a3bb0 0x7f939c0a6070 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f93a4006010 tx=0x7f93a40058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.729+0000 7f93bf98e740 1 -- 172.21.15.86:0/2730272857 shutdown_connections 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.729+0000 7f93bf98e740 1 --2- 172.21.15.86:0/2730272857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x56002504a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.729+0000 7f93bf98e740 1 --2- 172.21.15.86:0/2730272857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025047860 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.729+0000 7f93bf98e740 1 --2- 172.21.15.86:0/2730272857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042370 0x560025042790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.729+0000 7f93bf98e740 1 -- 172.21.15.86:0/2730272857 >> 172.21.15.86:0/2730272857 conn(0x56002503d8e0 msgr2=0x56002503dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.729+0000 7f93bf98e740 1 -- 172.21.15.86:0/2730272857 shutdown_connections 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.729+0000 7f93bf98e740 1 -- 172.21.15.86:0/2730272857 wait complete. 2024-04-17T07:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.730+0000 7f93bf98e740 1 -- start start 2024-04-17T07:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.730+0000 7f93bf98e740 1 Processor -- start 2024-04-17T07:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x56002504c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x56002504ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025047ed0 0x560025053e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93bf98e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56002504ae80 con 0x560025042d60 2024-04-17T07:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93bf98e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560024f62710 con 0x560025042370 2024-04-17T07:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93bf98e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56002504b000 con 0x560025047ed0 2024-04-17T07:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93bf98e740 10 client.0 _ll_register_callbacks cb 0x560024f5daf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93b5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x56002504c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x56002504ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93b5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x56002504c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43418/0 (socket says 172.21.15.86:43418) 2024-04-17T07:19:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.731+0000 7f93b5e00640 1 -- 172.21.15.86:0/3231107400 learned_addr learned my addr 172.21.15.86:0/3231107400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.732+0000 7f93b6800640 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025047ed0 0x560025053e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.732+0000 7f93b5400640 1 -- 172.21.15.86:0/3231107400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025047ed0 msgr2=0x560025053e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.732+0000 7f93b5400640 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025047ed0 0x560025053e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.732+0000 7f93b5400640 1 -- 172.21.15.86:0/3231107400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 msgr2=0x56002504c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.732+0000 7f93b5400640 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x56002504c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.732+0000 7f93b5400640 1 -- 172.21.15.86:0/3231107400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560024cbb3f0 con 0x560025042d60 2024-04-17T07:19:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.732+0000 7f93b5e00640 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x56002504c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.733+0000 7f93b5400640 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x56002504ce20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f93a4036040 tx=0x7f93a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51899e60fdd755d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.733+0000 7f9397400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.733+0000 7f9397400640 1 -- 172.21.15.86:0/3231107400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4053060 con 0x560025042d60 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.733+0000 7f93bf98e740 1 -- 172.21.15.86:0/3231107400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a8004c50 con 0x560025042d60 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.733+0000 7f9397400640 1 -- 172.21.15.86:0/3231107400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93a4004990 con 0x560025042d60 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.733+0000 7f9397400640 1 -- 172.21.15.86:0/3231107400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93a405a050 con 0x560025042d60 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.733+0000 7f93bf98e740 10 client.16836 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.733+0000 7f93bf98e740 1 -- 172.21.15.86:0/3231107400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560024c7b0f0 con 0x560025042d60 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.734+0000 7f9397400640 1 -- 172.21.15.86:0/3231107400 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a8004c50 con 0x560025042d60 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.734+0000 7f9397400640 1 -- 172.21.15.86:0/3231107400 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f93a4040040 con 0x560025042d60 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.734+0000 7f93bf98e740 10 client.16836 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.734+0000 7f93bf98e740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.734+0000 7f93bf98e740 20 client.16836 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.734+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600250586e0 0x56002505ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.734+0000 7f93bf98e740 4 client.16836 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.734+0000 7f93bf98e740 1 -- 172.21.15.86:0/3231107400 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560024fb6b40 con 0x5600250586e0 2024-04-17T07:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.735+0000 7f93b5e00640 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600250586e0 0x56002505ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.736+0000 7f93b5e00640 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600250586e0 0x56002505ab70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=553e76df9ae76ea server_cookie=b30d44b58c0181d1 in_seq=0 out_seq=0 2024-04-17T07:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.736+0000 7f9397400640 10 client.16836 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.737+0000 7f9397400640 1 -- 172.21.15.86:0/3231107400 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560024fb6b40 con 0x5600250586e0 2024-04-17T07:19:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.737+0000 7f9397400640 10 client.16836 handle_command_reply: tid=0 2024-04-17T07:19:09.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.737+0000 7f93bf98e740 1 client.16836 shutdown 2024-04-17T07:19:09.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.737+0000 7f93bf98e740 1 -- 172.21.15.86:0/3231107400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 msgr2=0x56002504ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.737+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x56002504ce20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f93a4036040 tx=0x7f93a4009260 comp rx=0 tx=0).stop 2024-04-17T07:19:09.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.738+0000 7f93bf98e740 1 -- 172.21.15.86:0/3231107400 shutdown_connections 2024-04-17T07:19:09.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.738+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600250586e0 0x56002505ab70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.738+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025047ed0 0x560025053e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.738+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x56002504ce20 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.738+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3231107400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x56002504c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.738+0000 7f9397400640 0 client.16836 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:09.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.738+0000 7f93bf98e740 1 -- 172.21.15.86:0/3231107400 >> 172.21.15.86:0/3231107400 conn(0x56002503d8e0 msgr2=0x56002503cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.738+0000 7f93bf98e740 1 -- 172.21.15.86:0/3231107400 shutdown_connections 2024-04-17T07:19:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.738+0000 7f93bf98e740 1 -- 172.21.15.86:0/3231107400 wait complete. 2024-04-17T07:19:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.738+0000 7f93bf98e740 20 client.16836 trim_cache size 0 max 16384 2024-04-17T07:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.758+0000 7f93bf98e740 1 Processor -- start 2024-04-17T07:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.758+0000 7f93bf98e740 1 -- start start 2024-04-17T07:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.758+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x5600250442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.758+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042d60 0x560025053e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.758+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025047ed0 0x5600250447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.758+0000 7f93bf98e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81acaa0 con 0x560025047ed0 2024-04-17T07:19:09.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.758+0000 7f93bf98e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81ae5e0 con 0x560025042370 2024-04-17T07:19:09.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.758+0000 7f93bf98e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81ad170 con 0x560025042d60 2024-04-17T07:19:09.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025047ed0 0x5600250447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025047ed0 0x5600250447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48622/0 (socket says 172.21.15.86:48622) 2024-04-17T07:19:09.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b6800640 1 -- 172.21.15.86:0/4026782813 learned_addr learned my addr 172.21.15.86:0/4026782813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:09.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b5e00640 1 --2- 172.21.15.86:0/4026782813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x5600250442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b5400640 1 --2- 172.21.15.86:0/4026782813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042d60 0x560025053e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b6800640 1 -- 172.21.15.86:0/4026782813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042d60 msgr2=0x560025053e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b6800640 1 --2- 172.21.15.86:0/4026782813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042d60 0x560025053e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b6800640 1 -- 172.21.15.86:0/4026782813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 msgr2=0x5600250442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b6800640 1 --2- 172.21.15.86:0/4026782813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x5600250442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b6800640 1 -- 172.21.15.86:0/4026782813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560025057110 con 0x560025047ed0 2024-04-17T07:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.759+0000 7f93b5e00640 1 --2- 172.21.15.86:0/4026782813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x5600250442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.760+0000 7f93b6800640 1 --2- 172.21.15.86:0/4026782813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025047ed0 0x5600250447e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f93b00cdfb0 tx=0x7f93b00983d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297f66f1b082042c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.760+0000 7f93a2600640 1 -- 172.21.15.86:0/4026782813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b001f040 con 0x560025047ed0 2024-04-17T07:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.760+0000 7f93a2600640 1 -- 172.21.15.86:0/4026782813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b009c7f0 con 0x560025047ed0 2024-04-17T07:19:09.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.760+0000 7f93a2600640 1 -- 172.21.15.86:0/4026782813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b00870b0 con 0x560025047ed0 2024-04-17T07:19:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.761+0000 7f93bf98e740 1 -- 172.21.15.86:0/4026782813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025047ed0 msgr2=0x5600250447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.761+0000 7f93bf98e740 1 --2- 172.21.15.86:0/4026782813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025047ed0 0x5600250447e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f93b00cdfb0 tx=0x7f93b00983d0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.761+0000 7f93bf98e740 1 -- 172.21.15.86:0/4026782813 shutdown_connections 2024-04-17T07:19:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.761+0000 7f93bf98e740 1 --2- 172.21.15.86:0/4026782813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025047ed0 0x5600250447e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.761+0000 7f93bf98e740 1 --2- 172.21.15.86:0/4026782813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042d60 0x560025053e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.761+0000 7f93bf98e740 1 --2- 172.21.15.86:0/4026782813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025042370 0x5600250442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.761+0000 7f93bf98e740 1 -- 172.21.15.86:0/4026782813 >> 172.21.15.86:0/4026782813 conn(0x56002503d8e0 msgr2=0x56002504b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.761+0000 7f93bf98e740 1 -- 172.21.15.86:0/4026782813 shutdown_connections 2024-04-17T07:19:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.761+0000 7f93bf98e740 1 -- 172.21.15.86:0/4026782813 wait complete. 2024-04-17T07:19:09.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.762+0000 7f93bf98e740 1 -- start start 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.762+0000 7f93bf98e740 1 Processor -- start 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.762+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042370 0x560025050ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.762+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025051420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.762+0000 7f93bf98e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x560025052430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.762+0000 7f93bf98e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81adf10 con 0x560025042d60 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.762+0000 7f93bf98e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81ad840 con 0x560025047ed0 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.762+0000 7f93bf98e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93b81aaa70 con 0x560025042370 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.762+0000 7f93bf98e740 10 client.0 _ll_register_callbacks cb 0x7f93a4004110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.762+0000 7f93b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025051420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025051420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48624/0 (socket says 172.21.15.86:48624) 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b5400640 1 -- 172.21.15.86:0/3825623273 learned_addr learned my addr 172.21.15.86:0/3825623273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b5e00640 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042370 0x560025050ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b6800640 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x560025052430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b5400640 1 -- 172.21.15.86:0/3825623273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042370 msgr2=0x560025050ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b5400640 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042370 0x560025050ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b5400640 1 -- 172.21.15.86:0/3825623273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 msgr2=0x560025052430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b5400640 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x560025052430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b5400640 1 -- 172.21.15.86:0/3825623273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560025052a20 con 0x560025042d60 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b6800640 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x560025052430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f93b5400640 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025051420 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f93a4000c00 tx=0x7f93a40346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cb2c97328e784d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.763+0000 7f9397400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.764+0000 7f9397400640 1 -- 172.21.15.86:0/3825623273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4053040 con 0x560025042d60 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.764+0000 7f93bf98e740 1 -- 172.21.15.86:0/3825623273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b009c7f0 con 0x560025042d60 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.764+0000 7f9397400640 1 -- 172.21.15.86:0/3825623273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93a4004ba0 con 0x560025042d60 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.764+0000 7f9397400640 1 -- 172.21.15.86:0/3825623273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93a40044d0 con 0x560025042d60 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.764+0000 7f93bf98e740 10 client.16842 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:09.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.764+0000 7f93bf98e740 1 -- 172.21.15.86:0/3825623273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560025053d30 con 0x560025042d60 2024-04-17T07:19:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.764+0000 7f9397400640 1 -- 172.21.15.86:0/3825623273 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b009c7f0 con 0x560025042d60 2024-04-17T07:19:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.764+0000 7f9397400640 1 -- 172.21.15.86:0/3825623273 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f93a4034910 con 0x560025042d60 2024-04-17T07:19:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.765+0000 7f93bf98e740 10 client.16842 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:09.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.765+0000 7f93bf98e740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.765+0000 7f93bf98e740 20 client.16842 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.765+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600250bf070 0x5600250c1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.765+0000 7f93bf98e740 4 client.16842 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.765+0000 7f93bf98e740 1 -- 172.21.15.86:0/3825623273 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560024fb09b0 con 0x5600250bf070 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.765+0000 7f93b5e00640 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600250bf070 0x5600250c1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.766+0000 7f93b5e00640 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600250bf070 0x5600250c1490 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60628afb499c48aa server_cookie=f5c398f621d3e756 in_seq=0 out_seq=0 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.766+0000 7f9397400640 10 client.16842 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.766+0000 7f9397400640 1 -- 172.21.15.86:0/3825623273 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560024fb09b0 con 0x5600250bf070 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.766+0000 7f9397400640 10 client.16842 handle_command_reply: tid=0 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.766+0000 7f93bf98e740 1 client.16842 shutdown 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 1 -- 172.21.15.86:0/3825623273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 msgr2=0x560025051420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025051420 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f93a4000c00 tx=0x7f93a40346f0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 1 -- 172.21.15.86:0/3825623273 shutdown_connections 2024-04-17T07:19:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600250bf070 0x5600250c1490 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560025047ed0 0x560025052430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560025042d60 0x560025051420 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 1 --2- 172.21.15.86:0/3825623273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560025042370 0x560025050ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 1 -- 172.21.15.86:0/3825623273 >> 172.21.15.86:0/3825623273 conn(0x56002503d8e0 msgr2=0x56002504b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f9397400640 0 client.16842 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 1 -- 172.21.15.86:0/3825623273 shutdown_connections 2024-04-17T07:19:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 1 -- 172.21.15.86:0/3825623273 wait complete. 2024-04-17T07:19:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.767+0000 7f93bf98e740 20 client.16842 trim_cache size 0 max 16384 2024-04-17T07:19:09.770 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:19:09.771 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:19:09.771 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:19:09.771 INFO:teuthology.orchestra.run.smithi086.stdout: "7daffc82-9fe9-4f59-b475-d5aa2432985a": { 2024-04-17T07:19:09.771 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:19:09.771 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "7daffc82-9fe9-4f59-b475-d5aa2432985a", 2024-04-17T07:19:09.771 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:19:09.771 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:19:09.771 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:19:09.771 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:19:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 -- 172.21.15.86:0/475271580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f939c0a3bb0 msgr2=0x7f939c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f939c0a3bb0 0x7f939c0a6070 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f93a4006010 tx=0x7f93a40058b0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 -- 172.21.15.86:0/475271580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93b8107b50 msgr2=0x7f93b81a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93b8107b50 0x7f93b81a1850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f93b000e940 tx=0x7f93b000ee00 comp rx=0 tx=0).stop 2024-04-17T07:19:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 -- 172.21.15.86:0/475271580 shutdown_connections 2024-04-17T07:19:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f939c0a3bb0 0x7f939c0a6070 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93b8107b50 0x7f93b81a1850 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93b8105220 0x7f93b819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 --2- 172.21.15.86:0/475271580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b81028f0 0x7f93b819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 -- 172.21.15.86:0/475271580 >> 172.21.15.86:0/475271580 conn(0x7f93b80fc300 msgr2=0x7f93b8107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 -- 172.21.15.86:0/475271580 shutdown_connections 2024-04-17T07:19:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:09.768+0000 7f93b7200640 1 -- 172.21.15.86:0/475271580 wait complete. 2024-04-17T07:19:09.788 INFO:tasks.cephfs.filesystem:scrub status for tag:7daffc82-9fe9-4f59-b475-d5aa2432985a - {'path': '/', 'tag': '7daffc82-9fe9-4f59-b475-d5aa2432985a', 'options': 'recursive,force'} 2024-04-17T07:19:09.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:09 smithi086 ceph-mon[31733]: pgmap v584: 129 pgs: 129 active+clean; 839 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 22 op/s 2024-04-17T07:19:10.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:09 smithi098 ceph-mon[35418]: pgmap v584: 129 pgs: 129 active+clean; 839 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 22 op/s 2024-04-17T07:19:10.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:09 smithi103 ceph-mon[35495]: pgmap v584: 129 pgs: 129 active+clean; 839 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 22 op/s 2024-04-17T07:19:11.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:10 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:19:11.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:10 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:11.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:10 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:19:11.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:10 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:10 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:19:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:10 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:10 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:19:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:10 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:10 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:19:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:10 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:10 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:19:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:10 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:11 smithi098 ceph-mon[35418]: pgmap v585: 129 pgs: 129 active+clean; 854 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-04-17T07:19:12.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:11 smithi103 ceph-mon[35495]: pgmap v585: 129 pgs: 129 active+clean; 854 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-04-17T07:19:12.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:11 smithi086 ceph-mon[31733]: pgmap v585: 129 pgs: 129 active+clean; 854 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-04-17T07:19:13.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:13 smithi098 ceph-mon[35418]: pgmap v586: 129 pgs: 129 active+clean; 868 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 27 op/s 2024-04-17T07:19:13.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:13 smithi103 ceph-mon[35495]: pgmap v586: 129 pgs: 129 active+clean; 868 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 27 op/s 2024-04-17T07:19:13.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:13 smithi086 ceph-mon[31733]: pgmap v586: 129 pgs: 129 active+clean; 868 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 27 op/s 2024-04-17T07:19:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:14 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:19:14.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:14 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:19:14.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:14 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:19:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:15 smithi098 ceph-mon[35418]: pgmap v587: 129 pgs: 129 active+clean; 889 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 23 op/s 2024-04-17T07:19:15.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:15 smithi103 ceph-mon[35495]: pgmap v587: 129 pgs: 129 active+clean; 889 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 23 op/s 2024-04-17T07:19:15.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:15 smithi086 ceph-mon[31733]: pgmap v587: 129 pgs: 129 active+clean; 889 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 MiB/s wr, 23 op/s 2024-04-17T07:19:18.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:18 smithi086 ceph-mon[31733]: pgmap v588: 129 pgs: 129 active+clean; 909 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 28 op/s 2024-04-17T07:19:18.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:18 smithi098 ceph-mon[35418]: pgmap v588: 129 pgs: 129 active+clean; 909 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 28 op/s 2024-04-17T07:19:18.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:18 smithi103 ceph-mon[35495]: pgmap v588: 129 pgs: 129 active+clean; 909 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 28 op/s 2024-04-17T07:19:20.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:20 smithi098 ceph-mon[35418]: pgmap v589: 129 pgs: 129 active+clean; 916 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 23 op/s 2024-04-17T07:19:20.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:20 smithi103 ceph-mon[35495]: pgmap v589: 129 pgs: 129 active+clean; 916 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 23 op/s 2024-04-17T07:19:20.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:20 smithi086 ceph-mon[31733]: pgmap v589: 129 pgs: 129 active+clean; 916 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 MiB/s wr, 23 op/s 2024-04-17T07:19:22.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:22 smithi098 ceph-mon[35418]: pgmap v590: 129 pgs: 129 active+clean; 936 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 21 op/s 2024-04-17T07:19:22.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:22 smithi103 ceph-mon[35495]: pgmap v590: 129 pgs: 129 active+clean; 936 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 21 op/s 2024-04-17T07:19:22.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:22 smithi086 ceph-mon[31733]: pgmap v590: 129 pgs: 129 active+clean; 936 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 21 op/s 2024-04-17T07:19:23.889 INFO:teuthology.orchestra.run.smithi086.stderr:done in 193.80 s (drop tables 0.03 s, create tables 0.21 s, client-side generate 133.46 s, vacuum 40.98 s, primary keys 19.12 s). 2024-04-17T07:19:23.899 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-04-17T07:19:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:starting vacuum...end. 2024-04-17T07:19:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:24 smithi098 ceph-mon[35418]: pgmap v591: 129 pgs: 129 active+clean; 952 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 25 op/s 2024-04-17T07:19:24.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:24 smithi103 ceph-mon[35495]: pgmap v591: 129 pgs: 129 active+clean; 952 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 25 op/s 2024-04-17T07:19:24.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:24 smithi086 ceph-mon[31733]: pgmap v591: 129 pgs: 129 active+clean; 952 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 25 op/s 2024-04-17T07:19:26.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:26 smithi098 ceph-mon[35418]: pgmap v592: 129 pgs: 129 active+clean; 964 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 18 op/s 2024-04-17T07:19:26.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:26 smithi103 ceph-mon[35495]: pgmap v592: 129 pgs: 129 active+clean; 964 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 18 op/s 2024-04-17T07:19:26.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:26 smithi086 ceph-mon[31733]: pgmap v592: 129 pgs: 129 active+clean; 964 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 18 op/s 2024-04-17T07:19:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:28 smithi098 ceph-mon[35418]: pgmap v593: 129 pgs: 129 active+clean; 956 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-04-17T07:19:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:28 smithi103 ceph-mon[35495]: pgmap v593: 129 pgs: 129 active+clean; 956 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-04-17T07:19:28.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:28 smithi086 ceph-mon[31733]: pgmap v593: 129 pgs: 129 active+clean; 956 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-04-17T07:19:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 5.0 s, 582.8 tps, lat 151.938 ms stddev 342.779 2024-04-17T07:19:30.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:30 smithi098 ceph-mon[35418]: pgmap v594: 129 pgs: 129 active+clean; 936 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 41 op/s 2024-04-17T07:19:30.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:30 smithi103 ceph-mon[35495]: pgmap v594: 129 pgs: 129 active+clean; 936 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 41 op/s 2024-04-17T07:19:30.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:30 smithi086 ceph-mon[31733]: pgmap v594: 129 pgs: 129 active+clean; 936 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 10 MiB/s wr, 41 op/s 2024-04-17T07:19:32.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:32 smithi098 ceph-mon[35418]: pgmap v595: 129 pgs: 129 active+clean; 934 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.4 MiB/s wr, 63 op/s 2024-04-17T07:19:32.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:32 smithi103 ceph-mon[35495]: pgmap v595: 129 pgs: 129 active+clean; 934 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.4 MiB/s wr, 63 op/s 2024-04-17T07:19:32.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:32 smithi086 ceph-mon[31733]: pgmap v595: 129 pgs: 129 active+clean; 934 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.4 MiB/s wr, 63 op/s 2024-04-17T07:19:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 10.0 s, 1355.8 tps, lat 73.760 ms stddev 59.410 2024-04-17T07:19:34.634 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:34 smithi086 ceph-mon[31733]: pgmap v596: 129 pgs: 129 active+clean; 939 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.1 MiB/s wr, 67 op/s 2024-04-17T07:19:34.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:34 smithi098 ceph-mon[35418]: pgmap v596: 129 pgs: 129 active+clean; 939 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.1 MiB/s wr, 67 op/s 2024-04-17T07:19:34.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:34 smithi103 ceph-mon[35495]: pgmap v596: 129 pgs: 129 active+clean; 939 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.1 MiB/s wr, 67 op/s 2024-04-17T07:19:36.576 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:36 smithi086 ceph-mon[31733]: pgmap v597: 129 pgs: 129 active+clean; 939 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.6 MiB/s wr, 74 op/s 2024-04-17T07:19:36.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:36 smithi098 ceph-mon[35418]: pgmap v597: 129 pgs: 129 active+clean; 939 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.6 MiB/s wr, 74 op/s 2024-04-17T07:19:36.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:36 smithi103 ceph-mon[35495]: pgmap v597: 129 pgs: 129 active+clean; 939 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.6 MiB/s wr, 74 op/s 2024-04-17T07:19:38.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:38 smithi098 ceph-mon[35418]: pgmap v598: 129 pgs: 129 active+clean; 939 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.0 MiB/s wr, 149 op/s 2024-04-17T07:19:38.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:38 smithi103 ceph-mon[35495]: pgmap v598: 129 pgs: 129 active+clean; 939 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.0 MiB/s wr, 149 op/s 2024-04-17T07:19:38.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:38 smithi086 ceph-mon[31733]: pgmap v598: 129 pgs: 129 active+clean; 939 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.0 MiB/s wr, 149 op/s 2024-04-17T07:19:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 15.0 s, 783.4 tps, lat 127.000 ms stddev 45.296 2024-04-17T07:19:39.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:19:39.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:19:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.078+0000 7fefb9400640 1 -- 172.21.15.86:0/255755465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb4075fa0 msgr2=0x7fefb40743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.078+0000 7fefb9400640 1 --2- 172.21.15.86:0/255755465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb4075fa0 0x7fefb40743f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009930 tx=0x7fefa402bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.079+0000 7fefb9400640 1 -- 172.21.15.86:0/255755465 shutdown_connections 2024-04-17T07:19:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.079+0000 7fefb9400640 1 --2- 172.21.15.86:0/255755465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb410e580 0x7fefb4110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.079+0000 7fefb9400640 1 --2- 172.21.15.86:0/255755465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefb40749c0 0x7fefb4074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.079+0000 7fefb9400640 1 --2- 172.21.15.86:0/255755465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb4075fa0 0x7fefb40743f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.079+0000 7fefb9400640 1 -- 172.21.15.86:0/255755465 >> 172.21.15.86:0/255755465 conn(0x7fefb40fd540 msgr2=0x7fefb40ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.079+0000 7fefb9400640 1 -- 172.21.15.86:0/255755465 shutdown_connections 2024-04-17T07:19:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.079+0000 7fefb9400640 1 -- 172.21.15.86:0/255755465 wait complete. 2024-04-17T07:19:40.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.081+0000 7fefb9400640 1 Processor -- start 2024-04-17T07:19:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.082+0000 7fefb9400640 1 -- start start 2024-04-17T07:19:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.082+0000 7fefb9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb40749c0 0x7fefb419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.082+0000 7fefb9400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefb4075fa0 0x7fefb419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.083+0000 7fefb9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb410e580 0x7fefb41a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.083+0000 7fefb9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41140f0 con 0x7fefb40749c0 2024-04-17T07:19:40.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.083+0000 7fefb9400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4113f70 con 0x7fefb4075fa0 2024-04-17T07:19:40.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.083+0000 7fefb9400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4114270 con 0x7fefb410e580 2024-04-17T07:19:40.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.083+0000 7fefb2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb40749c0 0x7fefb419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.084+0000 7fefb2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb40749c0 0x7fefb419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50568/0 (socket says 172.21.15.86:50568) 2024-04-17T07:19:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.084+0000 7fefb1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefb4075fa0 0x7fefb419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.084+0000 7fefb2800640 1 -- 172.21.15.86:0/3675625187 learned_addr learned my addr 172.21.15.86:0/3675625187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:40.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.084+0000 7fefb3200640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb410e580 0x7fefb41a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.085+0000 7fefb2800640 1 -- 172.21.15.86:0/3675625187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb410e580 msgr2=0x7fefb41a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.085+0000 7fefb2800640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb410e580 0x7fefb41a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.085+0000 7fefb2800640 1 -- 172.21.15.86:0/3675625187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefb4075fa0 msgr2=0x7fefb419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.085+0000 7fefb2800640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefb4075fa0 0x7fefb419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.085+0000 7fefb2800640 1 -- 172.21.15.86:0/3675625187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb41a6450 con 0x7fefb40749c0 2024-04-17T07:19:40.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.085+0000 7fefb2800640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb40749c0 0x7fefb419e830 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009230 tx=0x7fefa4005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.087+0000 7fefab400640 1 -- 172.21.15.86:0/3675625187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4043020 con 0x7fefb40749c0 2024-04-17T07:19:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.087+0000 7fefab400640 1 -- 172.21.15.86:0/3675625187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa40043d0 con 0x7fefb40749c0 2024-04-17T07:19:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.087+0000 7fefab400640 1 -- 172.21.15.86:0/3675625187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4003af0 con 0x7fefb40749c0 2024-04-17T07:19:40.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.087+0000 7fefb9400640 1 -- 172.21.15.86:0/3675625187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb41a66e0 con 0x7fefb40749c0 2024-04-17T07:19:40.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.087+0000 7fefb9400640 1 -- 172.21.15.86:0/3675625187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb41a6be0 con 0x7fefb40749c0 2024-04-17T07:19:40.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.092+0000 7fefbc1e8740 1 Processor -- start 2024-04-17T07:19:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.092+0000 7fefbc1e8740 1 -- start start 2024-04-17T07:19:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.092+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 0x55f067de35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.093+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 0x55f067de8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.093+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067deb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.093+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f067d4a120 con 0x55f067de3b80 2024-04-17T07:19:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.093+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f067cf0340 con 0x55f067de3190 2024-04-17T07:19:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.093+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f067d460e0 con 0x55f067de8cf0 2024-04-17T07:19:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.093+0000 7fefb1e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067deb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.094+0000 7fefab400640 1 -- 172.21.15.86:0/3675625187 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fefa4046020 con 0x7fefb40749c0 2024-04-17T07:19:40.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.094+0000 7fefb1e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067deb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41628/0 (socket says 172.21.15.86:41628) 2024-04-17T07:19:40.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.094+0000 7fefb1e00640 1 -- 172.21.15.86:0/2831784451 learned_addr learned my addr 172.21.15.86:0/2831784451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:40.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.094+0000 7fefb3200640 1 --2- 172.21.15.86:0/2831784451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 0x55f067de35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.095+0000 7fefb3200640 1 -- 172.21.15.86:0/2831784451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 msgr2=0x55f067deb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:40.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.095+0000 7fefb3200640 1 --2- 172.21.15.86:0/2831784451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067deb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.095+0000 7fefb3200640 1 -- 172.21.15.86:0/2831784451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 msgr2=0x55f067de8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:40.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.095+0000 7fefb3200640 1 --2- 172.21.15.86:0/2831784451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 0x55f067de8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.095+0000 7fefb3200640 1 -- 172.21.15.86:0/2831784451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f067b267d0 con 0x55f067de3190 2024-04-17T07:19:40.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.095+0000 7fefb3200640 1 --2- 172.21.15.86:0/2831784451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 0x55f067de35b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fefac009ed0 tx=0x7fefac0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a45cc24f7a418f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:40.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.096+0000 7fef9f400640 1 -- 172.21.15.86:0/2831784451 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac003b70 con 0x55f067de3190 2024-04-17T07:19:40.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.096+0000 7fef9f400640 1 -- 172.21.15.86:0/2831784451 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefac003d10 con 0x55f067de3190 2024-04-17T07:19:40.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.096+0000 7fef9f400640 1 -- 172.21.15.86:0/2831784451 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac003b70 con 0x55f067de3190 2024-04-17T07:19:40.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.097+0000 7fefab400640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef980a3d80 0x7fef980a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.097+0000 7fefab400640 1 -- 172.21.15.86:0/3675625187 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fefa40f2900 con 0x7fefb40749c0 2024-04-17T07:19:40.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.098+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2831784451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 msgr2=0x55f067de35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.098+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/2831784451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 0x55f067de35b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fefac009ed0 tx=0x7fefac0107a0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.098+0000 7fefb2800640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef980a3d80 0x7fef980a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.098+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2831784451 shutdown_connections 2024-04-17T07:19:40.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.098+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/2831784451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067deb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.098+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/2831784451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 0x55f067de8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.098+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/2831784451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 0x55f067de35b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.098+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2831784451 >> 172.21.15.86:0/2831784451 conn(0x55f067dde700 msgr2=0x55f067ddeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:40.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.099+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2831784451 shutdown_connections 2024-04-17T07:19:40.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.099+0000 7fefb2800640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef980a3d80 0x7fef980a6240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009900 tx=0x7fefa402bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:40.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.099+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2831784451 wait complete. 2024-04-17T07:19:40.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.100+0000 7fefbc1e8740 1 -- start start 2024-04-17T07:19:40.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.100+0000 7fefbc1e8740 1 Processor -- start 2024-04-17T07:19:40.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.101+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3190 0x55f067ded6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.101+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3b80 0x55f067dedc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.102+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067df4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.102+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f067debca0 con 0x55f067de3190 2024-04-17T07:19:40.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.102+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f067d03710 con 0x55f067de3b80 2024-04-17T07:19:40.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.102+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f067debe20 con 0x55f067de8cf0 2024-04-17T07:19:40.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.102+0000 7fefbc1e8740 10 client.0 _ll_register_callbacks cb 0x55f067cfeaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:40.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.102+0000 7fefb3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3190 0x55f067ded6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.102+0000 7fefb1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3b80 0x55f067dedc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.102+0000 7fefb3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3190 0x55f067ded6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50582/0 (socket says 172.21.15.86:50582) 2024-04-17T07:19:40.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.102+0000 7fefb3200640 1 -- 172.21.15.86:0/2042489632 learned_addr learned my addr 172.21.15.86:0/2042489632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:40.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.103+0000 7fefb3200640 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067df4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.104+0000 7fefb3200640 1 -- 172.21.15.86:0/2042489632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 msgr2=0x55f067df4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:40.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.104+0000 7fefb3200640 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067df4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.104+0000 7fefb3200640 1 -- 172.21.15.86:0/2042489632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3b80 msgr2=0x55f067dedc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:40.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.104+0000 7fefb3200640 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3b80 0x55f067dedc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.104+0000 7fefb3200640 1 -- 172.21.15.86:0/2042489632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f067a5c3f0 con 0x55f067de3190 2024-04-17T07:19:40.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.104+0000 7fefb3200640 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3190 0x55f067ded6e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fefac0049d0 tx=0x7fefac0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182a7bda39a8f71a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:40.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.105+0000 7fef93e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:40.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.106+0000 7fef93e00640 1 -- 172.21.15.86:0/2042489632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac01f070 con 0x55f067de3190 2024-04-17T07:19:40.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.106+0000 7fef93e00640 1 -- 172.21.15.86:0/2042489632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fefac003ec0 con 0x55f067de3190 2024-04-17T07:19:40.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.107+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2042489632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefac003d10 con 0x55f067de3190 2024-04-17T07:19:40.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.107+0000 7fef93e00640 1 -- 172.21.15.86:0/2042489632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fefac025070 con 0x55f067de3190 2024-04-17T07:19:40.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.107+0000 7fefbc1e8740 10 client.16851 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:40.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.107+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2042489632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f067a1c0f0 con 0x55f067de3190 2024-04-17T07:19:40.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.108+0000 7fef93e00640 1 -- 172.21.15.86:0/2042489632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefac003d10 con 0x55f067de3190 2024-04-17T07:19:40.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.108+0000 7fef93e00640 1 -- 172.21.15.86:0/2042489632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fefac0052c0 con 0x55f067de3190 2024-04-17T07:19:40.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.109+0000 7fefbc1e8740 10 client.16851 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:40.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.109+0000 7fefbc1e8740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:40.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.109+0000 7fefbc1e8740 20 client.16851 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:40.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.109+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f067df9250 0x55f067dfb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.109+0000 7fefbc1e8740 4 client.16851 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:40.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.109+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2042489632 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f067d57b40 con 0x55f067df9250 2024-04-17T07:19:40.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.111+0000 7fefb1e00640 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f067df9250 0x55f067dfb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.113+0000 7fefb1e00640 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f067df9250 0x55f067dfb660 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feb2e2ff29e233c0 server_cookie=e311836a99533756 in_seq=0 out_seq=0 2024-04-17T07:19:40.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.113+0000 7fef93e00640 10 client.16851 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:40.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.115+0000 7fef93e00640 1 -- 172.21.15.86:0/2042489632 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f067d57b40 con 0x55f067df9250 2024-04-17T07:19:40.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.115+0000 7fef93e00640 10 client.16851 handle_command_reply: tid=0 2024-04-17T07:19:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.115+0000 7fefbc1e8740 1 client.16851 shutdown 2024-04-17T07:19:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2042489632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3190 msgr2=0x55f067ded6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3190 0x55f067ded6e0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fefac0049d0 tx=0x7fefac0092a0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2042489632 shutdown_connections 2024-04-17T07:19:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f067df9250 0x55f067dfb660 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067df4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3b80 0x55f067dedc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/2042489632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3190 0x55f067ded6e0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fef93e00640 0 client.16851 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:40.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2042489632 >> 172.21.15.86:0/2042489632 conn(0x55f067dde700 msgr2=0x55f067ddd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:40.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2042489632 shutdown_connections 2024-04-17T07:19:40.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.116+0000 7fefbc1e8740 1 -- 172.21.15.86:0/2042489632 wait complete. 2024-04-17T07:19:40.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.117+0000 7fefbc1e8740 20 client.16851 trim_cache size 0 max 16384 2024-04-17T07:19:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.158+0000 7fefbc1e8740 1 Processor -- start 2024-04-17T07:19:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.158+0000 7fefbc1e8740 1 -- start start 2024-04-17T07:19:40.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.159+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 0x55f067dedb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.159+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 0x55f067df4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.159+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067ddd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.159+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41b1150 con 0x55f067de3b80 2024-04-17T07:19:40.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.159+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41b2c90 con 0x55f067de3190 2024-04-17T07:19:40.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.159+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41b1820 con 0x55f067de8cf0 2024-04-17T07:19:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.159+0000 7fefb1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 0x55f067df4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.159+0000 7fefb1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 0x55f067df4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50594/0 (socket says 172.21.15.86:50594) 2024-04-17T07:19:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.159+0000 7fefb1e00640 1 -- 172.21.15.86:0/4152681303 learned_addr learned my addr 172.21.15.86:0/4152681303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.159+0000 7fefb3200640 1 --2- 172.21.15.86:0/4152681303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 0x55f067dedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.160+0000 7fefb3200640 1 --2- 172.21.15.86:0/4152681303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067ddd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.160+0000 7fefb1e00640 1 -- 172.21.15.86:0/4152681303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 msgr2=0x55f067ddd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.160+0000 7fefb1e00640 1 --2- 172.21.15.86:0/4152681303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067ddd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.160+0000 7fefb1e00640 1 -- 172.21.15.86:0/4152681303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 msgr2=0x55f067dedb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.160+0000 7fefb1e00640 1 --2- 172.21.15.86:0/4152681303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 0x55f067dedb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.160+0000 7fefb1e00640 1 -- 172.21.15.86:0/4152681303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f067ddf470 con 0x55f067de3b80 2024-04-17T07:19:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.160+0000 7fefb1e00640 1 --2- 172.21.15.86:0/4152681303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 0x55f067df4bf0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009530 tx=0x7fefa0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f66349d807fb02 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.161+0000 7fef9ea00640 1 -- 172.21.15.86:0/4152681303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0024070 con 0x55f067de3b80 2024-04-17T07:19:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.161+0000 7fef9ea00640 1 -- 172.21.15.86:0/4152681303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa00047a0 con 0x55f067de3b80 2024-04-17T07:19:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.161+0000 7fef9ea00640 1 -- 172.21.15.86:0/4152681303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0020450 con 0x55f067de3b80 2024-04-17T07:19:40.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.163+0000 7fefbc1e8740 1 -- 172.21.15.86:0/4152681303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 msgr2=0x55f067df4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.163+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/4152681303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 0x55f067df4bf0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009530 tx=0x7fefa0005f90 comp rx=0 tx=0).stop 2024-04-17T07:19:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.163+0000 7fefbc1e8740 1 -- 172.21.15.86:0/4152681303 shutdown_connections 2024-04-17T07:19:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.163+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/4152681303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de8cf0 0x55f067ddd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.163+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/4152681303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de3b80 0x55f067df4bf0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.163+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/4152681303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3190 0x55f067dedb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.163+0000 7fefbc1e8740 1 -- 172.21.15.86:0/4152681303 >> 172.21.15.86:0/4152681303 conn(0x55f067dde700 msgr2=0x55f067ddd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:40.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.164+0000 7fefbc1e8740 1 -- 172.21.15.86:0/4152681303 shutdown_connections 2024-04-17T07:19:40.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.164+0000 7fefbc1e8740 1 -- 172.21.15.86:0/4152681303 wait complete. 2024-04-17T07:19:40.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.164+0000 7fefbc1e8740 1 -- start start 2024-04-17T07:19:40.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.164+0000 7fefbc1e8740 1 Processor -- start 2024-04-17T07:19:40.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.165+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de3190 0x55f067df2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.165+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3b80 0x55f067df29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.165+0000 7fefbc1e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de8cf0 0x55f067df3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.165+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41b25c0 con 0x55f067de8cf0 2024-04-17T07:19:40.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.165+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41b1ef0 con 0x55f067de3b80 2024-04-17T07:19:40.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.165+0000 7fefbc1e8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41af120 con 0x55f067de3190 2024-04-17T07:19:40.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.165+0000 7fefbc1e8740 10 client.0 _ll_register_callbacks cb 0x7fefa00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:40.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.166+0000 7fefb3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de8cf0 0x55f067df3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.166+0000 7fefb3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de3190 0x55f067df2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.166+0000 7fefb3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de3190 0x55f067df2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41650/0 (socket says 172.21.15.86:41650) 2024-04-17T07:19:40.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.166+0000 7fefb3200640 1 -- 172.21.15.86:0/3419922633 learned_addr learned my addr 172.21.15.86:0/3419922633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:40.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.167+0000 7fefb1e00640 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3b80 0x55f067df29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.167+0000 7fefb3200640 1 -- 172.21.15.86:0/3419922633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de3190 msgr2=0x55f067df2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:40.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.167+0000 7fefb3200640 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de3190 0x55f067df2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.167+0000 7fefb3200640 1 -- 172.21.15.86:0/3419922633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3b80 msgr2=0x55f067df29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:40.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.167+0000 7fefb3200640 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3b80 0x55f067df29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.167+0000 7fefb3200640 1 -- 172.21.15.86:0/3419922633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f067df3f70 con 0x55f067de8cf0 2024-04-17T07:19:40.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.168+0000 7fefb3200640 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de8cf0 0x55f067df3980 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fefac005b40 tx=0x7fefac010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f15d7784994ac86 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:40.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.168+0000 7fef93e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:40.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.168+0000 7fef93e00640 1 -- 172.21.15.86:0/3419922633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac040060 con 0x55f067de8cf0 2024-04-17T07:19:40.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.169+0000 7fef93e00640 1 -- 172.21.15.86:0/3419922633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fefac005150 con 0x55f067de8cf0 2024-04-17T07:19:40.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.169+0000 7fefbc1e8740 1 -- 172.21.15.86:0/3419922633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa00047a0 con 0x55f067de8cf0 2024-04-17T07:19:40.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.169+0000 7fef93e00640 1 -- 172.21.15.86:0/3419922633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fefac047050 con 0x55f067de8cf0 2024-04-17T07:19:40.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.169+0000 7fefbc1e8740 10 client.16857 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:40.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.169+0000 7fefbc1e8740 1 -- 172.21.15.86:0/3419922633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f067e5f8e0 con 0x55f067de8cf0 2024-04-17T07:19:40.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.170+0000 7fef93e00640 1 -- 172.21.15.86:0/3419922633 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa00047a0 con 0x55f067de8cf0 2024-04-17T07:19:40.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.171+0000 7fef93e00640 1 -- 172.21.15.86:0/3419922633 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fefac02c450 con 0x55f067de8cf0 2024-04-17T07:19:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.171+0000 7fefbc1e8740 10 client.16857 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.171+0000 7fefbc1e8740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.171+0000 7fefbc1e8740 20 client.16857 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.171+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f067e601d0 0x55f067e625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.171+0000 7fefbc1e8740 4 client.16857 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:19:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.171+0000 7fefbc1e8740 1 -- 172.21.15.86:0/3419922633 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f067d519b0 con 0x55f067e601d0 2024-04-17T07:19:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.171+0000 7fefb1e00640 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f067e601d0 0x55f067e625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.172+0000 7fefb1e00640 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f067e601d0 0x55f067e625f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351ff116b39b2f8 server_cookie=35434ef0d6dade99 in_seq=0 out_seq=0 2024-04-17T07:19:40.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.172+0000 7fef93e00640 10 client.16857 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:40.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.173+0000 7fef93e00640 1 -- 172.21.15.86:0/3419922633 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f067d519b0 con 0x55f067e601d0 2024-04-17T07:19:40.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.173+0000 7fef93e00640 10 client.16857 handle_command_reply: tid=0 2024-04-17T07:19:40.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.173+0000 7fefbc1e8740 1 client.16857 shutdown 2024-04-17T07:19:40.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.173+0000 7fefbc1e8740 1 -- 172.21.15.86:0/3419922633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de8cf0 msgr2=0x55f067df3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.173+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de8cf0 0x55f067df3980 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fefac005b40 tx=0x7fefac010fd0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.174+0000 7fefbc1e8740 1 -- 172.21.15.86:0/3419922633 shutdown_connections 2024-04-17T07:19:40.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.174+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f067e601d0 0x55f067e625f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.174+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f067de8cf0 0x55f067df3980 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.174+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f067de3b80 0x55f067df29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.174+0000 7fef93e00640 0 client.16857 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:40.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.174+0000 7fefbc1e8740 1 --2- 172.21.15.86:0/3419922633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f067de3190 0x55f067df2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.174+0000 7fefbc1e8740 1 -- 172.21.15.86:0/3419922633 >> 172.21.15.86:0/3419922633 conn(0x55f067dde700 msgr2=0x55f067ddd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.174+0000 7fefbc1e8740 1 -- 172.21.15.86:0/3419922633 shutdown_connections 2024-04-17T07:19:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.175+0000 7fefbc1e8740 1 -- 172.21.15.86:0/3419922633 wait complete. 2024-04-17T07:19:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.175+0000 7fefbc1e8740 20 client.16857 trim_cache size 0 max 16384 2024-04-17T07:19:40.174 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:19:40.175 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:19:40.175 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:19:40.175 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:19:40.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.176+0000 7fefb9400640 1 -- 172.21.15.86:0/3675625187 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef980a3d80 msgr2=0x7fef980a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.176+0000 7fefb9400640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef980a3d80 0x7fef980a6240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009900 tx=0x7fefa402bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:40.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.176+0000 7fefb9400640 1 -- 172.21.15.86:0/3675625187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb40749c0 msgr2=0x7fefb419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.176+0000 7fefb9400640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb40749c0 0x7fefb419e830 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009230 tx=0x7fefa4005e40 comp rx=0 tx=0).stop 2024-04-17T07:19:40.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.177+0000 7fefb9400640 1 -- 172.21.15.86:0/3675625187 shutdown_connections 2024-04-17T07:19:40.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.177+0000 7fefb9400640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef980a3d80 0x7fef980a6240 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.177+0000 7fefb9400640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb410e580 0x7fefb41a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.177+0000 7fefb9400640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefb4075fa0 0x7fefb419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.177+0000 7fefb9400640 1 --2- 172.21.15.86:0/3675625187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb40749c0 0x7fefb419e830 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.177+0000 7fefb9400640 1 -- 172.21.15.86:0/3675625187 >> 172.21.15.86:0/3675625187 conn(0x7fefb40fd540 msgr2=0x7fefb4101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:40.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.177+0000 7fefb9400640 1 -- 172.21.15.86:0/3675625187 shutdown_connections 2024-04-17T07:19:40.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.178+0000 7fefb9400640 1 -- 172.21.15.86:0/3675625187 wait complete. 2024-04-17T07:19:40.205 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:19:40.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:19:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.482+0000 7f73ec400640 1 -- 172.21.15.86:0/2566482535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e41053d0 msgr2=0x7f73e4105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.482+0000 7f73ec400640 1 --2- 172.21.15.86:0/2566482535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e41053d0 0x7f73e4105850 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009950 tx=0x7f73d002bc90 comp rx=0 tx=0).stop 2024-04-17T07:19:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.483+0000 7f73ec400640 1 -- 172.21.15.86:0/2566482535 shutdown_connections 2024-04-17T07:19:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.483+0000 7f73ec400640 1 --2- 172.21.15.86:0/2566482535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e4105d90 0x7f73e410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.483+0000 7f73ec400640 1 --2- 172.21.15.86:0/2566482535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e41053d0 0x7f73e4105850 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.483+0000 7f73ec400640 1 --2- 172.21.15.86:0/2566482535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73e41041e0 0x7f73e4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.483+0000 7f73ec400640 1 -- 172.21.15.86:0/2566482535 >> 172.21.15.86:0/2566482535 conn(0x7f73e40ff750 msgr2=0x7f73e4101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.483+0000 7f73ec400640 1 -- 172.21.15.86:0/2566482535 shutdown_connections 2024-04-17T07:19:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.483+0000 7f73ec400640 1 -- 172.21.15.86:0/2566482535 wait complete. 2024-04-17T07:19:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.484+0000 7f73ec400640 1 Processor -- start 2024-04-17T07:19:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.485+0000 7f73ec400640 1 -- start start 2024-04-17T07:19:40.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.485+0000 7f73ec400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e41041e0 0x7f73e419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.486+0000 7f73ec400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73e41053d0 0x7f73e419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.486+0000 7f73ec400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e4105d90 0x7f73e41a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.486+0000 7f73ec400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73e410fd70 con 0x7f73e4105d90 2024-04-17T07:19:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.486+0000 7f73ec400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f73e410fbf0 con 0x7f73e41041e0 2024-04-17T07:19:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.486+0000 7f73ec400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73e410fef0 con 0x7f73e41053d0 2024-04-17T07:19:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.486+0000 7f73e9a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e41041e0 0x7f73e419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.487+0000 7f73e9000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73e41053d0 0x7f73e419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.487+0000 7f73e9000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73e41053d0 0x7f73e419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41666/0 (socket says 172.21.15.86:41666) 2024-04-17T07:19:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.487+0000 7f73e9000640 1 -- 172.21.15.86:0/1025130079 learned_addr learned my addr 172.21.15.86:0/1025130079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.487+0000 7f73ea400640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e4105d90 0x7f73e41a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.487+0000 7f73e9000640 1 -- 172.21.15.86:0/1025130079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e41041e0 msgr2=0x7f73e419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.487+0000 7f73e9000640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e41041e0 0x7f73e419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.487+0000 7f73e9000640 1 -- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e4105d90 msgr2=0x7f73e41a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.487+0000 7f73e9000640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e4105d90 0x7f73e41a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.487+0000 7f73e9000640 1 -- 172.21.15.86:0/1025130079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e41a6400 con 0x7f73e41053d0 2024-04-17T07:19:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.487+0000 7f73ea400640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e4105d90 0x7f73e41a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.488+0000 7f73e9a00640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e41041e0 0x7f73e419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:19:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.488+0000 7f73e9000640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73e41053d0 0x7f73e419ed20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73d0005ab0 tx=0x7f73d00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.489+0000 7f73e2000640 1 -- 172.21.15.86:0/1025130079 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0043020 con 0x7f73e41053d0 2024-04-17T07:19:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.489+0000 7f73ec400640 1 -- 172.21.15.86:0/1025130079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e41a6690 con 0x7f73e41053d0 2024-04-17T07:19:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.489+0000 7f73ec400640 1 -- 172.21.15.86:0/1025130079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e41a6be0 con 0x7f73e41053d0 2024-04-17T07:19:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.490+0000 7f73e2000640 1 -- 172.21.15.86:0/1025130079 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73d00044c0 con 0x7f73e41053d0 2024-04-17T07:19:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.490+0000 7f73e2000640 1 -- 172.21.15.86:0/1025130079 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0003e20 con 0x7f73e41053d0 2024-04-17T07:19:40.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.491+0000 7f73ec400640 1 -- 172.21.15.86:0/1025130079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e41a6f50 con 0x7f73e41053d0 2024-04-17T07:19:40.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.497+0000 7f73e2000640 1 -- 172.21.15.86:0/1025130079 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f73d0046020 con 0x7f73e41053d0 2024-04-17T07:19:40.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.499+0000 7f73e2000640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73cc0a3c30 0x7f73cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:40.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.500+0000 7f73e2000640 1 -- 172.21.15.86:0/1025130079 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f73d00f3580 con 0x7f73e41053d0 2024-04-17T07:19:40.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.500+0000 7f73e9a00640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73cc0a3c30 0x7f73cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:40.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.501+0000 7f73e9a00640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73cc0a3c30 0x7f73cc0a60f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f73e4105230 tx=0x7f73d8005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:40.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.503+0000 7f73e2000640 1 -- 172.21.15.86:0/1025130079 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f73d004d020 con 0x7f73e41053d0 2024-04-17T07:19:40.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:40 smithi086 ceph-mon[31733]: pgmap v599: 129 pgs: 129 active+clean; 939 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 296 op/s 2024-04-17T07:19:40.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:40 smithi098 ceph-mon[35418]: pgmap v599: 129 pgs: 129 active+clean; 939 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 296 op/s 2024-04-17T07:19:40.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:40 smithi103 ceph-mon[35495]: pgmap v599: 129 pgs: 129 active+clean; 939 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 296 op/s 2024-04-17T07:19:40.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.969+0000 7f73ec400640 1 -- 172.21.15.86:0/1025130079 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73e41a7120 con 0x7f73e41053d0 2024-04-17T07:19:40.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.971+0000 7f73e2000640 1 -- 172.21.15.86:0/1025130079 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f73d00f3090 con 0x7f73e41053d0 2024-04-17T07:19:40.970 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:19:40.971 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:19:40.974 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:19:40.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.975+0000 7f73ec400640 1 -- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73cc0a3c30 msgr2=0x7f73cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.975+0000 7f73ec400640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73cc0a3c30 0x7f73cc0a60f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f73e4105230 tx=0x7f73d8005e70 comp rx=0 tx=0).stop 2024-04-17T07:19:40.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.975+0000 7f73ec400640 1 -- 172.21.15.86:0/1025130079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73e41053d0 msgr2=0x7f73e419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:40.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.975+0000 7f73ec400640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73e41053d0 0x7f73e419ed20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73d0005ab0 tx=0x7f73d00040c0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.975+0000 7f73ec400640 1 -- 172.21.15.86:0/1025130079 shutdown_connections 2024-04-17T07:19:40.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.975+0000 7f73ec400640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73cc0a3c30 0x7f73cc0a60f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.975+0000 7f73ec400640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73e4105d90 0x7f73e41a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.975+0000 7f73ec400640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73e41053d0 0x7f73e419ed20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.975+0000 7f73ec400640 1 --2- 172.21.15.86:0/1025130079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e41041e0 0x7f73e419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:40.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.975+0000 7f73ec400640 1 -- 172.21.15.86:0/1025130079 >> 172.21.15.86:0/1025130079 conn(0x7f73e40ff750 msgr2=0x7f73e40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:40.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.976+0000 7f73ec400640 1 -- 172.21.15.86:0/1025130079 shutdown_connections 2024-04-17T07:19:40.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:40.977+0000 7f73ec400640 1 -- 172.21.15.86:0/1025130079 wait complete. 2024-04-17T07:19:41.006 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:19:41.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.317+0000 7f278e200640 1 -- 172.21.15.86:0/963488084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27881085c0 msgr2=0x7f27881089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.317+0000 7f278e200640 1 --2- 172.21.15.86:0/963488084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27881085c0 0x7f27881089e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2770009900 tx=0x7f277002bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:41.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.318+0000 7f278e200640 1 -- 172.21.15.86:0/963488084 shutdown_connections 2024-04-17T07:19:41.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.318+0000 7f278e200640 1 --2- 172.21.15.86:0/963488084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f278810a170 0x7f2788110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.318+0000 7f278e200640 1 --2- 172.21.15.86:0/963488084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27881097b0 0x7f2788109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.318+0000 7f278e200640 1 --2- 172.21.15.86:0/963488084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27881085c0 0x7f27881089e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.318+0000 7f278e200640 1 -- 172.21.15.86:0/963488084 >> 172.21.15.86:0/963488084 conn(0x7f2788077560 msgr2=0x7f27880799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.318+0000 7f278e200640 1 -- 172.21.15.86:0/963488084 shutdown_connections 2024-04-17T07:19:41.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.318+0000 7f278e200640 1 -- 172.21.15.86:0/963488084 wait complete. 2024-04-17T07:19:41.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.319+0000 7f278e200640 1 Processor -- start 2024-04-17T07:19:41.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.320+0000 7f278e200640 1 -- start start 2024-04-17T07:19:41.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.320+0000 7f278e200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27881085c0 0x7f278819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.320+0000 7f278e200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27881097b0 0x7f278819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.320+0000 7f278e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278810a170 0x7f27881a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.320+0000 7f278e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2788114140 con 0x7f278810a170 2024-04-17T07:19:41.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.320+0000 7f278e200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2788113fc0 con 0x7f27881097b0 2024-04-17T07:19:41.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.320+0000 7f278e200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27881142c0 con 0x7f27881085c0 2024-04-17T07:19:41.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.321+0000 7f2787e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278810a170 0x7f27881a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.321+0000 7f2787400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27881085c0 0x7f278819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.321+0000 7f2787400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27881085c0 0x7f278819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41672/0 (socket says 172.21.15.86:41672) 2024-04-17T07:19:41.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.321+0000 7f2787400640 1 -- 172.21.15.86:0/1209936812 learned_addr learned my addr 172.21.15.86:0/1209936812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:41.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.321+0000 7f277ea00640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27881097b0 0x7f278819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.321+0000 7f2787e00640 1 -- 172.21.15.86:0/1209936812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27881085c0 msgr2=0x7f278819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.321+0000 7f2787e00640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27881085c0 0x7f278819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.321+0000 7f2787e00640 1 -- 172.21.15.86:0/1209936812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27881097b0 msgr2=0x7f278819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.321+0000 7f2787e00640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27881097b0 0x7f278819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.321+0000 7f2787e00640 1 -- 172.21.15.86:0/1209936812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27881a64b0 con 0x7f278810a170 2024-04-17T07:19:41.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.322+0000 7f2787e00640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278810a170 0x7f27881a5e40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f278000e970 tx=0x7f278000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.323+0000 7f2784c00640 1 -- 172.21.15.86:0/1209936812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278000ccc0 con 0x7f278810a170 2024-04-17T07:19:41.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.323+0000 7f278e200640 1 -- 172.21.15.86:0/1209936812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27881a67a0 con 0x7f278810a170 2024-04-17T07:19:41.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.323+0000 7f2784c00640 1 -- 172.21.15.86:0/1209936812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27800035f0 con 0x7f278810a170 2024-04-17T07:19:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.324+0000 7f278e200640 1 -- 172.21.15.86:0/1209936812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27881a6d40 con 0x7f278810a170 2024-04-17T07:19:41.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.324+0000 7f2784c00640 1 -- 172.21.15.86:0/1209936812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278000ce60 con 0x7f278810a170 2024-04-17T07:19:41.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.328+0000 7f2784c00640 1 -- 172.21.15.86:0/1209936812 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2780004020 con 0x7f278810a170 2024-04-17T07:19:41.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.329+0000 7f2790f82740 1 Processor -- start 2024-04-17T07:19:41.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.330+0000 7f2790f82740 1 -- start start 2024-04-17T07:19:41.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.330+0000 7f2790f82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.330+0000 7f2790f82740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e50990 0x559145e55490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.330+0000 7f2790f82740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e57f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.330+0000 7f2790f82740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559145db6020 con 0x559145e50990 2024-04-17T07:19:41.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.330+0000 7f2790f82740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559145d5c460 con 0x559145e55b00 2024-04-17T07:19:41.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.330+0000 7f2790f82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559145db1fe0 con 0x559145e4ffa0 2024-04-17T07:19:41.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.331+0000 7f2787400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e50990 0x559145e55490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.331+0000 7f2784c00640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f276c0a3c00 0x7f276c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.331+0000 7f277ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e57f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.331+0000 7f2787400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e50990 0x559145e55490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50660/0 (socket says 172.21.15.86:50660) 2024-04-17T07:19:41.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.331+0000 7f2787400640 1 -- 172.21.15.86:0/3341891867 learned_addr learned my addr 172.21.15.86:0/3341891867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:41.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.331+0000 7f2784c00640 1 -- 172.21.15.86:0/1209936812 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f27800d0a60 con 0x7f278810a170 2024-04-17T07:19:41.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.331+0000 7f277ea00640 1 --2- 172.21.15.86:0/3341891867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.332+0000 7f2787e00640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f276c0a3c00 0x7f276c0a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.332+0000 7f2787400640 1 -- 172.21.15.86:0/3341891867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 msgr2=0x559145e503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.332+0000 7f2787400640 1 --2- 172.21.15.86:0/3341891867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.332+0000 7f2787400640 1 -- 172.21.15.86:0/3341891867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 msgr2=0x559145e57f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.332+0000 7f2787400640 1 --2- 172.21.15.86:0/3341891867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e57f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.332+0000 7f2787400640 1 -- 172.21.15.86:0/3341891867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559145d64040 con 0x559145e50990 2024-04-17T07:19:41.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.332+0000 7f277ea00640 1 --2- 172.21.15.86:0/3341891867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:41.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.332+0000 7f2787400640 1 --2- 172.21.15.86:0/3341891867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e50990 0x559145e55490 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2770005a90 tx=0x7f277002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf6dc5ba8208dff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.333+0000 7f2787e00640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f276c0a3c00 0x7f276c0a60c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2780000c00 tx=0x7f2780003e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.333+0000 7f2777e00640 1 -- 172.21.15.86:0/3341891867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770045020 con 0x559145e50990 2024-04-17T07:19:41.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.333+0000 7f2777e00640 1 -- 172.21.15.86:0/3341891867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f277002ca50 con 0x559145e50990 2024-04-17T07:19:41.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.333+0000 7f2777e00640 1 -- 172.21.15.86:0/3341891867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27700046c0 con 0x559145e50990 2024-04-17T07:19:41.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.335+0000 7f2790f82740 1 -- 172.21.15.86:0/3341891867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e50990 msgr2=0x559145e55490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.335+0000 7f2790f82740 1 --2- 172.21.15.86:0/3341891867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e50990 0x559145e55490 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2770005a90 tx=0x7f277002c8e0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.335+0000 7f2790f82740 1 -- 172.21.15.86:0/3341891867 shutdown_connections 2024-04-17T07:19:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.335+0000 7f2790f82740 1 --2- 172.21.15.86:0/3341891867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e57f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.336+0000 7f2790f82740 1 --2- 172.21.15.86:0/3341891867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e50990 0x559145e55490 secure :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2770005a90 tx=0x7f277002c8e0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.336+0000 7f2790f82740 1 --2- 172.21.15.86:0/3341891867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.336+0000 7f2790f82740 1 -- 172.21.15.86:0/3341891867 >> 172.21.15.86:0/3341891867 conn(0x559145e4b510 msgr2=0x559145e4b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.336+0000 7f2790f82740 1 -- 172.21.15.86:0/3341891867 shutdown_connections 2024-04-17T07:19:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.336+0000 7f2790f82740 1 -- 172.21.15.86:0/3341891867 wait complete. 2024-04-17T07:19:41.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.337+0000 7f2790f82740 1 -- start start 2024-04-17T07:19:41.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.337+0000 7f2790f82740 1 Processor -- start 2024-04-17T07:19:41.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.338+0000 7f2790f82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e5a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.338+0000 7f2790f82740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e5ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.338+0000 7f2790f82740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e61f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.338+0000 7f2790f82740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559145e58a50 con 0x559145e5fb80 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.338+0000 7f2790f82740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559145d6f830 con 0x559145e55b00 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.338+0000 7f2790f82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559145e58bd0 con 0x559145e4ffa0 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.338+0000 7f2790f82740 10 client.0 _ll_register_callbacks cb 0x559145d3dfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.339+0000 7f277ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.339+0000 7f277ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e61f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50676/0 (socket says 172.21.15.86:50676) 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.339+0000 7f277ea00640 1 -- 172.21.15.86:0/3995723084 learned_addr learned my addr 172.21.15.86:0/3995723084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.339+0000 7f277ea00640 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e5a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.339+0000 7f2787400640 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e5ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.339+0000 7f2787400640 1 -- 172.21.15.86:0/3995723084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 msgr2=0x559145e5a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.339+0000 7f2787400640 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e5a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.339+0000 7f2787400640 1 -- 172.21.15.86:0/3995723084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 msgr2=0x559145e61f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.339+0000 7f2787400640 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e61f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.340+0000 7f2787400640 1 -- 172.21.15.86:0/3995723084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559145d73920 con 0x559145e55b00 2024-04-17T07:19:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.340+0000 7f277ea00640 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e61f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.340+0000 7f2787400640 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e5ab50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x559145e50d90 tx=0x7f2770005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=605e25a3887b36db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.340+0000 7f2774c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:19:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.340+0000 7f2774c00640 1 -- 172.21.15.86:0/3995723084 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770051020 con 0x559145e55b00 2024-04-17T07:19:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.340+0000 7f2790f82740 1 -- 172.21.15.86:0/3995723084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277002ca50 con 0x559145e55b00 2024-04-17T07:19:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.340+0000 7f2774c00640 1 -- 172.21.15.86:0/3995723084 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2770003800 con 0x559145e55b00 2024-04-17T07:19:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.341+0000 7f2774c00640 1 -- 172.21.15.86:0/3995723084 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2770000e60 con 0x559145e55b00 2024-04-17T07:19:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.341+0000 7f2790f82740 10 client.35882 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.341+0000 7f2790f82740 1 -- 172.21.15.86:0/3995723084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559145d417f0 con 0x559145e55b00 2024-04-17T07:19:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.342+0000 7f2774c00640 1 -- 172.21.15.86:0/3995723084 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f277002ca50 con 0x559145e55b00 2024-04-17T07:19:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.342+0000 7f2774c00640 1 -- 172.21.15.86:0/3995723084 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2770041b90 con 0x559145e55b00 2024-04-17T07:19:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.342+0000 7f2790f82740 10 client.35882 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.342+0000 7f2790f82740 10 client.35882 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:19:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.342+0000 7f2790f82740 20 client.35882 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.342+0000 7f2790f82740 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559145e660d0 0x559145e684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.342+0000 7f2790f82740 4 client.35882 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.342+0000 7f2790f82740 1 -- 172.21.15.86:0/3995723084 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559145dc3a40 con 0x559145e660d0 2024-04-17T07:19:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.343+0000 7f277ea00640 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559145e660d0 0x559145e684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.345+0000 7f277ea00640 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559145e660d0 0x559145e684c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a416c18b13b7dd1 server_cookie=41658337d97e9e38 in_seq=0 out_seq=0 2024-04-17T07:19:41.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.345+0000 7f2774c00640 10 client.35882 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:19:41.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.346+0000 7f2774c00640 1 -- 172.21.15.86:0/3995723084 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559145dc3a40 con 0x559145e660d0 2024-04-17T07:19:41.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.346+0000 7f2774c00640 10 client.35882 handle_command_reply: tid=0 2024-04-17T07:19:41.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.346+0000 7f2790f82740 1 client.35882 shutdown 2024-04-17T07:19:41.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.347+0000 7f2790f82740 1 -- 172.21.15.86:0/3995723084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 msgr2=0x559145e5ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.347+0000 7f2790f82740 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e5ab50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x559145e50d90 tx=0x7f2770005fd0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.347+0000 7f2790f82740 1 -- 172.21.15.86:0/3995723084 shutdown_connections 2024-04-17T07:19:41.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.347+0000 7f2790f82740 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559145e660d0 0x559145e684c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.347+0000 7f2790f82740 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e61f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.347+0000 7f2790f82740 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e5ab50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.347+0000 7f2790f82740 1 --2- 172.21.15.86:0/3995723084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e5a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.347+0000 7f2790f82740 1 -- 172.21.15.86:0/3995723084 >> 172.21.15.86:0/3995723084 conn(0x559145e4b510 msgr2=0x559145e4a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.347+0000 7f2774c00640 0 client.35882 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:19:41.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.348+0000 7f2790f82740 1 -- 172.21.15.86:0/3995723084 shutdown_connections 2024-04-17T07:19:41.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.348+0000 7f2790f82740 1 -- 172.21.15.86:0/3995723084 wait complete. 2024-04-17T07:19:41.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.348+0000 7f2790f82740 20 client.35882 trim_cache size 0 max 16384 2024-04-17T07:19:41.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.376+0000 7f2790f82740 1 Processor -- start 2024-04-17T07:19:41.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.377+0000 7f2790f82740 1 -- start start 2024-04-17T07:19:41.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.377+0000 7f2790f82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e51530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.377+0000 7f2790f82740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e58f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.377+0000 7f2790f82740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.377+0000 7f2790f82740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27881b1110 con 0x559145e5fb80 2024-04-17T07:19:41.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.377+0000 7f2790f82740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27881b2c50 con 0x559145e55b00 2024-04-17T07:19:41.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.377+0000 7f2790f82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27881b17e0 con 0x559145e4ffa0 2024-04-17T07:19:41.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.378+0000 7f277ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.378+0000 7f277ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e51530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.379+0000 7f277ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e51530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41706/0 (socket says 172.21.15.86:41706) 2024-04-17T07:19:41.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.379+0000 7f277ea00640 1 -- 172.21.15.86:0/2669556764 learned_addr learned my addr 172.21.15.86:0/2669556764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:41.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.379+0000 7f2787400640 1 --2- 172.21.15.86:0/2669556764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e58f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.379+0000 7f277ea00640 1 -- 172.21.15.86:0/2669556764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 msgr2=0x559145e51530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.379+0000 7f277ea00640 1 --2- 172.21.15.86:0/2669556764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e51530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.379+0000 7f277ea00640 1 -- 172.21.15.86:0/2669556764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 msgr2=0x559145e58f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.379+0000 7f277ea00640 1 --2- 172.21.15.86:0/2669556764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e58f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.379+0000 7f277ea00640 1 -- 172.21.15.86:0/2669556764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559145ac83f0 con 0x559145e5fb80 2024-04-17T07:19:41.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.379+0000 7f2787400640 1 --2- 172.21.15.86:0/2669556764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e58f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:41.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.380+0000 7f277ea00640 1 --2- 172.21.15.86:0/2669556764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e594b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2778013ac0 tx=0x7f2778004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17b75329aac0f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.380+0000 7f2777400640 1 -- 172.21.15.86:0/2669556764 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778023930 con 0x559145e5fb80 2024-04-17T07:19:41.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.380+0000 7f2777400640 1 -- 172.21.15.86:0/2669556764 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27780134e0 con 0x559145e5fb80 2024-04-17T07:19:41.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.380+0000 7f2777400640 1 -- 172.21.15.86:0/2669556764 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778012880 con 0x559145e5fb80 2024-04-17T07:19:41.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.381+0000 7f2790f82740 1 -- 172.21.15.86:0/2669556764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 msgr2=0x559145e594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.381+0000 7f2790f82740 1 --2- 172.21.15.86:0/2669556764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e594b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2778013ac0 tx=0x7f2778004ab0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.382+0000 7f2790f82740 1 -- 172.21.15.86:0/2669556764 shutdown_connections 2024-04-17T07:19:41.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.382+0000 7f2790f82740 1 --2- 172.21.15.86:0/2669556764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e5fb80 0x559145e594b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.382+0000 7f2790f82740 1 --2- 172.21.15.86:0/2669556764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e55b00 0x559145e58f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.382+0000 7f2790f82740 1 --2- 172.21.15.86:0/2669556764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e4ffa0 0x559145e51530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.382+0000 7f2790f82740 1 -- 172.21.15.86:0/2669556764 >> 172.21.15.86:0/2669556764 conn(0x559145e4b510 msgr2=0x559145e5a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.382+0000 7f2790f82740 1 -- 172.21.15.86:0/2669556764 shutdown_connections 2024-04-17T07:19:41.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.382+0000 7f2790f82740 1 -- 172.21.15.86:0/2669556764 wait complete. 2024-04-17T07:19:41.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.382+0000 7f2790f82740 1 -- start start 2024-04-17T07:19:41.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.383+0000 7f2790f82740 1 Processor -- start 2024-04-17T07:19:41.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.384+0000 7f2790f82740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e4ffa0 0x559145e77820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.384+0000 7f2790f82740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e55b00 0x559145e77d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.384+0000 7f2790f82740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e5fb80 0x559145e78d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.384+0000 7f2790f82740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27881b2580 con 0x559145e4ffa0 2024-04-17T07:19:41.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.384+0000 7f2790f82740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27881b1eb0 con 0x559145e5fb80 2024-04-17T07:19:41.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.384+0000 7f2790f82740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27881af0e0 con 0x559145e55b00 2024-04-17T07:19:41.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.384+0000 7f2790f82740 10 client.0 _ll_register_callbacks cb 0x7f27700043b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:41.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.385+0000 7f277ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e4ffa0 0x559145e77820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.385+0000 7f2787400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e55b00 0x559145e77d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.385+0000 7f2787400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e55b00 0x559145e77d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41716/0 (socket says 172.21.15.86:41716) 2024-04-17T07:19:41.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.385+0000 7f2787400640 1 -- 172.21.15.86:0/1102377046 learned_addr learned my addr 172.21.15.86:0/1102377046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:41.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.386+0000 7f2787400640 1 -- 172.21.15.86:0/1102377046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e5fb80 msgr2=0x559145e78d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.386+0000 7f2787400640 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e5fb80 0x559145e78d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.386+0000 7f2787400640 1 -- 172.21.15.86:0/1102377046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e4ffa0 msgr2=0x559145e77820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.386+0000 7f2787400640 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e4ffa0 0x559145e77820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.386+0000 7f2787400640 1 -- 172.21.15.86:0/1102377046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559145b927d0 con 0x559145e55b00 2024-04-17T07:19:41.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.386+0000 7f277ea00640 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e4ffa0 0x559145e77820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:41.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.387+0000 7f2787400640 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e55b00 0x559145e77d60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2770041f80 tx=0x7f277002c2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4579f807ff2f182d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.388+0000 7f2774c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:19:41.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.388+0000 7f2774c00640 1 -- 172.21.15.86:0/1102377046 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770063020 con 0x559145e55b00 2024-04-17T07:19:41.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.388+0000 7f2774c00640 1 -- 172.21.15.86:0/1102377046 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f277004ac30 con 0x559145e55b00 2024-04-17T07:19:41.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.388+0000 7f2790f82740 1 -- 172.21.15.86:0/1102377046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27780134e0 con 0x559145e55b00 2024-04-17T07:19:41.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.389+0000 7f2774c00640 1 -- 172.21.15.86:0/1102377046 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2770051040 con 0x559145e55b00 2024-04-17T07:19:41.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.390+0000 7f2790f82740 10 client.25915 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:41.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.390+0000 7f2790f82740 1 -- 172.21.15.86:0/1102377046 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559145ecc6c0 con 0x559145e55b00 2024-04-17T07:19:41.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.391+0000 7f2774c00640 1 -- 172.21.15.86:0/1102377046 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27780134e0 con 0x559145e55b00 2024-04-17T07:19:41.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.391+0000 7f2774c00640 1 -- 172.21.15.86:0/1102377046 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2770041700 con 0x559145e55b00 2024-04-17T07:19:41.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.391+0000 7f2790f82740 10 client.25915 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:41.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.391+0000 7f2790f82740 10 client.25915 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:19:41.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.391+0000 7f2790f82740 20 client.25915 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:41.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.391+0000 7f2790f82740 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559145eccf50 0x559145ecf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.391+0000 7f2790f82740 4 client.25915 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:19:41.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.391+0000 7f2790f82740 1 -- 172.21.15.86:0/1102377046 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559145dbd8b0 con 0x559145eccf50 2024-04-17T07:19:41.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.392+0000 7f277ea00640 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559145eccf50 0x559145ecf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.394+0000 7f2774c00640 10 client.25915 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:19:41.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.394+0000 7f277ea00640 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559145eccf50 0x559145ecf370 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=45ef347c1f01ee2d server_cookie=b2bb579487fb12f3 in_seq=0 out_seq=0 2024-04-17T07:19:41.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.395+0000 7f2774c00640 1 -- 172.21.15.86:0/1102377046 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559145dbd8b0 con 0x559145eccf50 2024-04-17T07:19:41.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.395+0000 7f2774c00640 10 client.25915 handle_command_reply: tid=0 2024-04-17T07:19:41.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.396+0000 7f2790f82740 1 client.25915 shutdown 2024-04-17T07:19:41.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2790f82740 1 -- 172.21.15.86:0/1102377046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e55b00 msgr2=0x559145e77d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2790f82740 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e55b00 0x559145e77d60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2770041f80 tx=0x7f277002c2b0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2790f82740 1 -- 172.21.15.86:0/1102377046 shutdown_connections 2024-04-17T07:19:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2790f82740 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559145eccf50 0x559145ecf370 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2790f82740 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559145e5fb80 0x559145e78d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2790f82740 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559145e55b00 0x559145e77d60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2790f82740 1 --2- 172.21.15.86:0/1102377046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559145e4ffa0 0x559145e77820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2774c00640 0 client.25915 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:19:41.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2790f82740 1 -- 172.21.15.86:0/1102377046 >> 172.21.15.86:0/1102377046 conn(0x559145e4b510 msgr2=0x559145e5a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2790f82740 1 -- 172.21.15.86:0/1102377046 shutdown_connections 2024-04-17T07:19:41.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.397+0000 7f2790f82740 1 -- 172.21.15.86:0/1102377046 wait complete. 2024-04-17T07:19:41.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.398+0000 7f2790f82740 20 client.25915 trim_cache size 0 max 16384 2024-04-17T07:19:41.397 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:19:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.400+0000 7f278e200640 1 -- 172.21.15.86:0/1209936812 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f276c0a3c00 msgr2=0x7f276c0a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.400+0000 7f278e200640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f276c0a3c00 0x7f276c0a60c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2780000c00 tx=0x7f2780003e30 comp rx=0 tx=0).stop 2024-04-17T07:19:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.400+0000 7f278e200640 1 -- 172.21.15.86:0/1209936812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278810a170 msgr2=0x7f27881a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.400+0000 7f278e200640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278810a170 0x7f27881a5e40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f278000e970 tx=0x7f278000ee30 comp rx=0 tx=0).stop 2024-04-17T07:19:41.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.400+0000 7f278e200640 1 -- 172.21.15.86:0/1209936812 shutdown_connections 2024-04-17T07:19:41.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.401+0000 7f278e200640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f276c0a3c00 0x7f276c0a60c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.401+0000 7f278e200640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278810a170 0x7f27881a5e40 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.401+0000 7f278e200640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27881097b0 0x7f278819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.401+0000 7f278e200640 1 --2- 172.21.15.86:0/1209936812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27881085c0 0x7f278819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.401+0000 7f278e200640 1 -- 172.21.15.86:0/1209936812 >> 172.21.15.86:0/1209936812 conn(0x7f2788077560 msgr2=0x7f278810cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.401+0000 7f278e200640 1 -- 172.21.15.86:0/1209936812 shutdown_connections 2024-04-17T07:19:41.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.402+0000 7f278e200640 1 -- 172.21.15.86:0/1209936812 wait complete. 2024-04-17T07:19:41.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:19:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.710+0000 7f1f4ae00640 1 -- 172.21.15.86:0/3239893810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f44074960 msgr2=0x7f1f44074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.710+0000 7f1f4ae00640 1 --2- 172.21.15.86:0/3239893810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f44074960 0x7f1f44074dc0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009860 tx=0x7f1f3002bc20 comp rx=0 tx=0).stop 2024-04-17T07:19:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.710+0000 7f1f4ae00640 1 -- 172.21.15.86:0/3239893810 shutdown_connections 2024-04-17T07:19:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.710+0000 7f1f4ae00640 1 --2- 172.21.15.86:0/3239893810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f44109e30 0x7f1f4410c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.710+0000 7f1f4ae00640 1 --2- 172.21.15.86:0/3239893810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f44074960 0x7f1f44074dc0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.710+0000 7f1f4ae00640 1 --2- 172.21.15.86:0/3239893810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f44075f40 0x7f1f44074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.710+0000 7f1f4ae00640 1 -- 172.21.15.86:0/3239893810 >> 172.21.15.86:0/3239893810 conn(0x7f1f440fd510 msgr2=0x7f1f440ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.710+0000 7f1f4ae00640 1 -- 172.21.15.86:0/3239893810 shutdown_connections 2024-04-17T07:19:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.710+0000 7f1f4ae00640 1 -- 172.21.15.86:0/3239893810 wait complete. 2024-04-17T07:19:41.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.712+0000 7f1f4ae00640 1 Processor -- start 2024-04-17T07:19:41.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.712+0000 7f1f4ae00640 1 -- start start 2024-04-17T07:19:41.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.713+0000 7f1f4ae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f44075f40 0x7f1f4419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.713+0000 7f1f4ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f44109e30 0x7f1f4419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.713+0000 7f1f4ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4419fc10 0x7f1f441a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.713+0000 7f1f4ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410fd30 con 0x7f1f4419fc10 2024-04-17T07:19:41.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.713+0000 7f1f4ae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410fbb0 con 0x7f1f44075f40 2024-04-17T07:19:41.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.713+0000 7f1f4ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410feb0 con 0x7f1f44109e30 2024-04-17T07:19:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.713+0000 7f1f48e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4419fc10 0x7f1f441a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.713+0000 7f1f43e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f44075f40 0x7f1f4419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.713+0000 7f1f48e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4419fc10 0x7f1f441a2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50720/0 (socket says 172.21.15.86:50720) 2024-04-17T07:19:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.713+0000 7f1f48e00640 1 -- 172.21.15.86:0/4114423625 learned_addr learned my addr 172.21.15.86:0/4114423625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.714+0000 7f1f43400640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f44109e30 0x7f1f4419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.714+0000 7f1f48e00640 1 -- 172.21.15.86:0/4114423625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f44109e30 msgr2=0x7f1f4419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.715+0000 7f1f48e00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f44109e30 0x7f1f4419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.715+0000 7f1f48e00640 1 -- 172.21.15.86:0/4114423625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f44075f40 msgr2=0x7f1f4419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.715+0000 7f1f48e00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f44075f40 0x7f1f4419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.715+0000 7f1f48e00640 1 -- 172.21.15.86:0/4114423625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f441a2670 con 0x7f1f4419fc10 2024-04-17T07:19:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.715+0000 7f1f43e00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f44075f40 0x7f1f4419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:19:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.715+0000 7f1f43400640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f44109e30 0x7f1f4419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:19:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.715+0000 7f1f48e00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4419fc10 0x7f1f441a2000 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400d750 tx=0x7f1f3400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.716+0000 7f1f40c00640 1 -- 172.21.15.86:0/4114423625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34017070 con 0x7f1f4419fc10 2024-04-17T07:19:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.716+0000 7f1f4ae00640 1 -- 172.21.15.86:0/4114423625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f441a2960 con 0x7f1f4419fc10 2024-04-17T07:19:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.716+0000 7f1f4ae00640 1 -- 172.21.15.86:0/4114423625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f441a2e80 con 0x7f1f4419fc10 2024-04-17T07:19:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.716+0000 7f1f40c00640 1 -- 172.21.15.86:0/4114423625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f340054d0 con 0x7f1f4419fc10 2024-04-17T07:19:41.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.716+0000 7f1f40c00640 1 -- 172.21.15.86:0/4114423625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34004c60 con 0x7f1f4419fc10 2024-04-17T07:19:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.724+0000 7f1f4da1a740 1 Processor -- start 2024-04-17T07:19:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.724+0000 7f1f40c00640 1 -- 172.21.15.86:0/4114423625 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1f34010460 con 0x7f1f4419fc10 2024-04-17T07:19:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.724+0000 7f1f4da1a740 1 -- start start 2024-04-17T07:19:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.724+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09a7b0 0x562c3e09abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.725+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09b1a0 0x562c3e09fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.725+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e0a0310 0x562c3e0a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.725+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562c3dff8020 con 0x562c3e09a7b0 2024-04-17T07:19:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.725+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562c3df9e460 con 0x562c3e0a0310 2024-04-17T07:19:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.725+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562c3dff3fe0 con 0x562c3e09b1a0 2024-04-17T07:19:41.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.726+0000 7f1f43400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e0a0310 0x562c3e0a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.726+0000 7f1f43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09a7b0 0x562c3e09abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.726+0000 7f1f43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09a7b0 0x562c3e09abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50726/0 (socket says 172.21.15.86:50726) 2024-04-17T07:19:41.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.726+0000 7f1f43400640 1 -- 172.21.15.86:0/75169698 learned_addr learned my addr 172.21.15.86:0/75169698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:41.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.726+0000 7f1f43e00640 1 --2- 172.21.15.86:0/75169698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09b1a0 0x562c3e09fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.727+0000 7f1f40c00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1f280a3d00 0x7f1f280a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.727+0000 7f1f43e00640 1 -- 172.21.15.86:0/75169698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e0a0310 msgr2=0x562c3e0a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.727+0000 7f1f43e00640 1 --2- 172.21.15.86:0/75169698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e0a0310 0x562c3e0a27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.727+0000 7f1f43e00640 1 -- 172.21.15.86:0/75169698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09a7b0 msgr2=0x562c3e09abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.727+0000 7f1f40c00640 1 -- 172.21.15.86:0/4114423625 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1f340d01c0 con 0x7f1f4419fc10 2024-04-17T07:19:41.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.727+0000 7f1f43e00640 1 --2- 172.21.15.86:0/75169698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09a7b0 0x562c3e09abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.727+0000 7f1f43e00640 1 -- 172.21.15.86:0/75169698 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c3dfa6040 con 0x562c3e09b1a0 2024-04-17T07:19:41.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.727+0000 7f1f43400640 1 --2- 172.21.15.86:0/75169698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09a7b0 0x562c3e09abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:41.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.728+0000 7f1f43e00640 1 --2- 172.21.15.86:0/75169698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09b1a0 0x562c3e09fca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c010590 tx=0x7f1f3c010a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f87a97d88f4e79b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.728+0000 7f1f39600640 1 -- 172.21.15.86:0/75169698 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c01d070 con 0x562c3e09b1a0 2024-04-17T07:19:41.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.728+0000 7f1f48e00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1f280a3d00 0x7f1f280a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.728+0000 7f1f39600640 1 -- 172.21.15.86:0/75169698 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f3c003e20 con 0x562c3e09b1a0 2024-04-17T07:19:41.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.729+0000 7f1f39600640 1 -- 172.21.15.86:0/75169698 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c01d070 con 0x562c3e09b1a0 2024-04-17T07:19:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.730+0000 7f1f4da1a740 1 -- 172.21.15.86:0/75169698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09b1a0 msgr2=0x562c3e09fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.730+0000 7f1f48e00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1f280a3d00 0x7f1f280a61c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1f3409bdc0 tx=0x7f1f340d0410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.730+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/75169698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09b1a0 0x562c3e09fca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c010590 tx=0x7f1f3c010a50 comp rx=0 tx=0).stop 2024-04-17T07:19:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.730+0000 7f1f4da1a740 1 -- 172.21.15.86:0/75169698 shutdown_connections 2024-04-17T07:19:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.730+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/75169698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e0a0310 0x562c3e0a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.730+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/75169698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09b1a0 0x562c3e09fca0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.730+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/75169698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09a7b0 0x562c3e09abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.730+0000 7f1f4da1a740 1 -- 172.21.15.86:0/75169698 >> 172.21.15.86:0/75169698 conn(0x562c3e095d20 msgr2=0x562c3e096120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.730+0000 7f1f4da1a740 1 -- 172.21.15.86:0/75169698 shutdown_connections 2024-04-17T07:19:41.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.730+0000 7f1f4da1a740 1 -- 172.21.15.86:0/75169698 wait complete. 2024-04-17T07:19:41.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.731+0000 7f1f4da1a740 1 -- start start 2024-04-17T07:19:41.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.731+0000 7f1f4da1a740 1 Processor -- start 2024-04-17T07:19:41.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.732+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 0x562c3e0a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.732+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09b1a0 0x562c3e0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.732+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e0a0310 0x562c3e0ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.732+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562c3e0a32c0 con 0x562c3e0a0310 2024-04-17T07:19:41.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.732+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562c3dfb1830 con 0x562c3e09b1a0 2024-04-17T07:19:41.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.732+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562c3e0a3440 con 0x562c3e09a7b0 2024-04-17T07:19:41.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.732+0000 7f1f43e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09b1a0 0x562c3e0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.733+0000 7f1f43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e0a0310 0x562c3e0ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.733+0000 7f1f4da1a740 10 client.0 _ll_register_callbacks cb 0x562c3df7ffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:41.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.733+0000 7f1f43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e0a0310 0x562c3e0ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50740/0 (socket says 172.21.15.86:50740) 2024-04-17T07:19:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.733+0000 7f1f43400640 1 -- 172.21.15.86:0/2787688980 learned_addr learned my addr 172.21.15.86:0/2787688980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.733+0000 7f1f43400640 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 0x562c3e0a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.733+0000 7f1f43400640 1 -- 172.21.15.86:0/2787688980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 msgr2=0x562c3e0a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.733+0000 7f1f43400640 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 0x562c3e0a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.733+0000 7f1f43400640 1 -- 172.21.15.86:0/2787688980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09b1a0 msgr2=0x562c3e0a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.733+0000 7f1f43400640 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09b1a0 0x562c3e0a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.733+0000 7f1f43400640 1 -- 172.21.15.86:0/2787688980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c3dfb5920 con 0x562c3e0a0310 2024-04-17T07:19:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.734+0000 7f1f43400640 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e0a0310 0x562c3e0ac090 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009860 tx=0x7f1f300094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7b6b37578dc8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.734+0000 7f1f2e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:41.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.735+0000 7f1f2e000640 1 -- 172.21.15.86:0/2787688980 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30039040 con 0x562c3e0a0310 2024-04-17T07:19:41.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.735+0000 7f1f2e000640 1 -- 172.21.15.86:0/2787688980 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1f300043e0 con 0x562c3e0a0310 2024-04-17T07:19:41.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.736+0000 7f1f4da1a740 1 -- 172.21.15.86:0/2787688980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3c003e20 con 0x562c3e0a0310 2024-04-17T07:19:41.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.736+0000 7f1f2e000640 1 -- 172.21.15.86:0/2787688980 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f30053050 con 0x562c3e0a0310 2024-04-17T07:19:41.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.737+0000 7f1f4da1a740 10 client.16893 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:41.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.737+0000 7f1f4da1a740 1 -- 172.21.15.86:0/2787688980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c3df837f0 con 0x562c3e0a0310 2024-04-17T07:19:41.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.738+0000 7f1f2e000640 1 -- 172.21.15.86:0/2787688980 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f3c003e20 con 0x562c3e0a0310 2024-04-17T07:19:41.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.738+0000 7f1f2e000640 1 -- 172.21.15.86:0/2787688980 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1f3002c890 con 0x562c3e0a0310 2024-04-17T07:19:41.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.738+0000 7f1f4da1a740 10 client.16893 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:41.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.738+0000 7f1f4da1a740 10 client.16893 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:19:41.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.738+0000 7f1f4da1a740 20 client.16893 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.738+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562c3e0b0650 0x562c3e0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.739+0000 7f1f4da1a740 4 client.16893 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.739+0000 7f1f4da1a740 1 -- 172.21.15.86:0/2787688980 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c3e005a40 con 0x562c3e0b0650 2024-04-17T07:19:41.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.740+0000 7f1f43e00640 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562c3e0b0650 0x562c3e0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.741+0000 7f1f43e00640 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562c3e0b0650 0x562c3e0b2af0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=13fedb47b1e190fa server_cookie=de02089736ff7682 in_seq=0 out_seq=0 2024-04-17T07:19:41.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.741+0000 7f1f2e000640 10 client.16893 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:19:41.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.742+0000 7f1f2e000640 1 -- 172.21.15.86:0/2787688980 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c3e005a40 con 0x562c3e0b0650 2024-04-17T07:19:41.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.742+0000 7f1f2e000640 10 client.16893 handle_command_reply: tid=0 2024-04-17T07:19:41.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.743+0000 7f1f4da1a740 1 client.16893 shutdown 2024-04-17T07:19:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.743+0000 7f1f4da1a740 1 -- 172.21.15.86:0/2787688980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e0a0310 msgr2=0x562c3e0ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.743+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e0a0310 0x562c3e0ac090 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009860 tx=0x7f1f300094d0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.744+0000 7f1f4da1a740 1 -- 172.21.15.86:0/2787688980 shutdown_connections 2024-04-17T07:19:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.744+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562c3e0b0650 0x562c3e0b2af0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.744+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e0a0310 0x562c3e0ac090 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.744+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09b1a0 0x562c3e0a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.744+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/2787688980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 0x562c3e0a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.744+0000 7f1f4da1a740 1 -- 172.21.15.86:0/2787688980 >> 172.21.15.86:0/2787688980 conn(0x562c3e095d20 msgr2=0x562c3e08c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.744+0000 7f1f2e000640 0 client.16893 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:19:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.744+0000 7f1f4da1a740 1 -- 172.21.15.86:0/2787688980 shutdown_connections 2024-04-17T07:19:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.745+0000 7f1f4da1a740 1 -- 172.21.15.86:0/2787688980 wait complete. 2024-04-17T07:19:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.745+0000 7f1f4da1a740 20 client.16893 trim_cache size 0 max 16384 2024-04-17T07:19:41.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:41 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1025130079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:19:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.770+0000 7f1f4da1a740 1 Processor -- start 2024-04-17T07:19:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.770+0000 7f1f4da1a740 1 -- start start 2024-04-17T07:19:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.771+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e095d20 0x562c3e0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.771+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 0x562c3e0ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.771+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 0x562c3e08be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.771+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2400fb60 con 0x562c3e09b1a0 2024-04-17T07:19:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.771+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f240116a0 con 0x562c3e095d20 2024-04-17T07:19:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.771+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24010230 con 0x562c3e09a7b0 2024-04-17T07:19:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.772+0000 7f1f43e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 0x562c3e0ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.772+0000 7f1f43e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 0x562c3e0ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41766/0 (socket says 172.21.15.86:41766) 2024-04-17T07:19:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.772+0000 7f1f43400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e095d20 0x562c3e0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.772+0000 7f1f43400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e095d20 0x562c3e0a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54012/0 (socket says 172.21.15.86:54012) 2024-04-17T07:19:41.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.772+0000 7f1f43400640 1 -- 172.21.15.86:0/1815009538 learned_addr learned my addr 172.21.15.86:0/1815009538 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:41.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.772+0000 7f1f43400640 1 --2- 172.21.15.86:0/1815009538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 0x562c3e08be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.773+0000 7f1f43400640 1 -- 172.21.15.86:0/1815009538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 msgr2=0x562c3e0ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.773+0000 7f1f43400640 1 --2- 172.21.15.86:0/1815009538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 0x562c3e0ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.773+0000 7f1f43400640 1 -- 172.21.15.86:0/1815009538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e095d20 msgr2=0x562c3e0a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.773+0000 7f1f43400640 1 --2- 172.21.15.86:0/1815009538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e095d20 0x562c3e0a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.773+0000 7f1f43400640 1 -- 172.21.15.86:0/1815009538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c3dd0a3f0 con 0x562c3e09b1a0 2024-04-17T07:19:41.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.773+0000 7f1f43400640 1 --2- 172.21.15.86:0/1815009538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 0x562c3e08be80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1f30037010 tx=0x7f1f3002bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cade2ed45638039 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.774+0000 7f1f38c00640 1 -- 172.21.15.86:0/1815009538 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30039040 con 0x562c3e09b1a0 2024-04-17T07:19:41.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.774+0000 7f1f38c00640 1 -- 172.21.15.86:0/1815009538 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f30048020 con 0x562c3e09b1a0 2024-04-17T07:19:41.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.774+0000 7f1f38c00640 1 -- 172.21.15.86:0/1815009538 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30003a50 con 0x562c3e09b1a0 2024-04-17T07:19:41.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.775+0000 7f1f4da1a740 1 -- 172.21.15.86:0/1815009538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 msgr2=0x562c3e08be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.775+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/1815009538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 0x562c3e08be80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1f30037010 tx=0x7f1f3002bc20 comp rx=0 tx=0).stop 2024-04-17T07:19:41.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.776+0000 7f1f4da1a740 1 -- 172.21.15.86:0/1815009538 shutdown_connections 2024-04-17T07:19:41.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.776+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/1815009538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 0x562c3e08be80 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.776+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/1815009538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e09a7b0 0x562c3e0ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.776+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/1815009538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e095d20 0x562c3e0a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.776+0000 7f1f4da1a740 1 -- 172.21.15.86:0/1815009538 >> 172.21.15.86:0/1815009538 conn(0x562c3e09fe10 msgr2=0x562c3e0a3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.776+0000 7f1f4da1a740 1 -- 172.21.15.86:0/1815009538 shutdown_connections 2024-04-17T07:19:41.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.776+0000 7f1f4da1a740 1 -- 172.21.15.86:0/1815009538 wait complete. 2024-04-17T07:19:41.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.777+0000 7f1f4da1a740 1 -- start start 2024-04-17T07:19:41.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:41 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1025130079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:19:41.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.777+0000 7f1f4da1a740 1 Processor -- start 2024-04-17T07:19:41.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.778+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e095d20 0x562c3e0a9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.778+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09a7b0 0x562c3e0a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.778+0000 7f1f4da1a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 0x562c3e0aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.778+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24010fd0 con 0x562c3e09b1a0 2024-04-17T07:19:41.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.778+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24010900 con 0x562c3e09a7b0 2024-04-17T07:19:41.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.778+0000 7f1f4da1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2400db30 con 0x562c3e095d20 2024-04-17T07:19:41.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.778+0000 7f1f4da1a740 10 client.0 _ll_register_callbacks cb 0x7f1f30035ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:41.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.779+0000 7f1f43400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e095d20 0x562c3e0a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.779+0000 7f1f43e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09a7b0 0x562c3e0a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.779+0000 7f1f43400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e095d20 0x562c3e0a9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41776/0 (socket says 172.21.15.86:41776) 2024-04-17T07:19:41.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.779+0000 7f1f43400640 1 -- 172.21.15.86:0/363083429 learned_addr learned my addr 172.21.15.86:0/363083429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:41.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.779+0000 7f1f43400640 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 0x562c3e0aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.780+0000 7f1f43e00640 1 -- 172.21.15.86:0/363083429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e095d20 msgr2=0x562c3e0a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.780+0000 7f1f43e00640 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e095d20 0x562c3e0a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.780+0000 7f1f43e00640 1 -- 172.21.15.86:0/363083429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 msgr2=0x562c3e0aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:41.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.780+0000 7f1f43e00640 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 0x562c3e0aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.780+0000 7f1f43e00640 1 -- 172.21.15.86:0/363083429 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c3ddd47d0 con 0x562c3e09a7b0 2024-04-17T07:19:41.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.781+0000 7f1f43e00640 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09a7b0 0x562c3e0a9ba0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x562c3e09b5a0 tx=0x7f1f3c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945c6c002dfd9bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:41.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.781+0000 7f1f2e000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:19:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.782+0000 7f1f2e000640 1 -- 172.21.15.86:0/363083429 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c01d070 con 0x562c3e09a7b0 2024-04-17T07:19:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.782+0000 7f1f2e000640 1 -- 172.21.15.86:0/363083429 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1f3c003a90 con 0x562c3e09a7b0 2024-04-17T07:19:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.782+0000 7f1f4da1a740 1 -- 172.21.15.86:0/363083429 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f30048020 con 0x562c3e09a7b0 2024-04-17T07:19:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.783+0000 7f1f2e000640 1 -- 172.21.15.86:0/363083429 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f3c02ad60 con 0x562c3e09a7b0 2024-04-17T07:19:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.783+0000 7f1f4da1a740 10 client.35900 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.783+0000 7f1f4da1a740 1 -- 172.21.15.86:0/363083429 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c3e0ac2d0 con 0x562c3e09a7b0 2024-04-17T07:19:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.784+0000 7f1f2e000640 1 -- 172.21.15.86:0/363083429 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f30048020 con 0x562c3e09a7b0 2024-04-17T07:19:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.784+0000 7f1f2e000640 1 -- 172.21.15.86:0/363083429 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1f3c00b040 con 0x562c3e09a7b0 2024-04-17T07:19:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.784+0000 7f1f4da1a740 10 client.35900 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.784+0000 7f1f4da1a740 10 client.35900 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:19:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.784+0000 7f1f4da1a740 20 client.35900 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.784+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562c3e1170d0 0x562c3e1194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.784+0000 7f1f4da1a740 4 client.35900 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:19:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.784+0000 7f1f4da1a740 1 -- 172.21.15.86:0/363083429 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c3dfff8b0 con 0x562c3e1170d0 2024-04-17T07:19:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.785+0000 7f1f43400640 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562c3e1170d0 0x562c3e1194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.786+0000 7f1f43400640 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562c3e1170d0 0x562c3e1194f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d7dfabea5ba41f35 server_cookie=4d220bd037e42fd7 in_seq=0 out_seq=0 2024-04-17T07:19:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.786+0000 7f1f2e000640 10 client.35900 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:19:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.786+0000 7f1f2e000640 1 -- 172.21.15.86:0/363083429 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c3dfff8b0 con 0x562c3e1170d0 2024-04-17T07:19:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f2e000640 10 client.35900 handle_command_reply: tid=0 2024-04-17T07:19:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f4da1a740 1 client.35900 shutdown 2024-04-17T07:19:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f4da1a740 1 -- 172.21.15.86:0/363083429 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09a7b0 msgr2=0x562c3e0a9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09a7b0 0x562c3e0a9ba0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x562c3e09b5a0 tx=0x7f1f3c000f40 comp rx=0 tx=0).stop 2024-04-17T07:19:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f4da1a740 1 -- 172.21.15.86:0/363083429 shutdown_connections 2024-04-17T07:19:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x562c3e1170d0 0x562c3e1194f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c3e09b1a0 0x562c3e0aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c3e09a7b0 0x562c3e0a9ba0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f4da1a740 1 --2- 172.21.15.86:0/363083429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c3e095d20 0x562c3e0a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f4da1a740 1 -- 172.21.15.86:0/363083429 >> 172.21.15.86:0/363083429 conn(0x562c3e09fe10 msgr2=0x562c3e098f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.787+0000 7f1f2e000640 0 client.35900 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:19:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.788+0000 7f1f4da1a740 1 -- 172.21.15.86:0/363083429 shutdown_connections 2024-04-17T07:19:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.788+0000 7f1f4da1a740 1 -- 172.21.15.86:0/363083429 wait complete. 2024-04-17T07:19:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.788+0000 7f1f4da1a740 20 client.35900 trim_cache size 0 max 16384 2024-04-17T07:19:41.790 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:19:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.789+0000 7f1f4ae00640 1 -- 172.21.15.86:0/4114423625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1f280a3d00 msgr2=0x7f1f280a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.789+0000 7f1f4ae00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1f280a3d00 0x7f1f280a61c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1f3409bdc0 tx=0x7f1f340d0410 comp rx=0 tx=0).stop 2024-04-17T07:19:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.790+0000 7f1f4ae00640 1 -- 172.21.15.86:0/4114423625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4419fc10 msgr2=0x7f1f441a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.790+0000 7f1f4ae00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4419fc10 0x7f1f441a2000 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400d750 tx=0x7f1f3400dc10 comp rx=0 tx=0).stop 2024-04-17T07:19:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.790+0000 7f1f4ae00640 1 -- 172.21.15.86:0/4114423625 shutdown_connections 2024-04-17T07:19:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.790+0000 7f1f4ae00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1f280a3d00 0x7f1f280a61c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.790+0000 7f1f4ae00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4419fc10 0x7f1f441a2000 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.790+0000 7f1f4ae00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f44109e30 0x7f1f4419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.790+0000 7f1f4ae00640 1 --2- 172.21.15.86:0/4114423625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f44075f40 0x7f1f4419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:41.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.790+0000 7f1f4ae00640 1 -- 172.21.15.86:0/4114423625 >> 172.21.15.86:0/4114423625 conn(0x7f1f440fd510 msgr2=0x7f1f44105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:41.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.790+0000 7f1f4ae00640 1 -- 172.21.15.86:0/4114423625 shutdown_connections 2024-04-17T07:19:41.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:41.790+0000 7f1f4ae00640 1 -- 172.21.15.86:0/4114423625 wait complete. 2024-04-17T07:19:41.810 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:41 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1025130079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:19:41.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:19:42.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.117+0000 7fa4f4200640 1 -- 172.21.15.86:0/1009692539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ec075fa0 msgr2=0x7fa4ec0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.118+0000 7fa4f4200640 1 --2- 172.21.15.86:0/1009692539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ec075fa0 0x7fa4ec0743f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009930 tx=0x7fa4e002bc20 comp rx=0 tx=0).stop 2024-04-17T07:19:42.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.118+0000 7fa4f4200640 1 -- 172.21.15.86:0/1009692539 shutdown_connections 2024-04-17T07:19:42.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.118+0000 7fa4f4200640 1 --2- 172.21.15.86:0/1009692539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec10e570 0x7fa4ec110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.118+0000 7fa4f4200640 1 --2- 172.21.15.86:0/1009692539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ec0749c0 0x7fa4ec074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.118+0000 7fa4f4200640 1 --2- 172.21.15.86:0/1009692539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ec075fa0 0x7fa4ec0743f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.118+0000 7fa4f4200640 1 -- 172.21.15.86:0/1009692539 >> 172.21.15.86:0/1009692539 conn(0x7fa4ec0fd510 msgr2=0x7fa4ec0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.118+0000 7fa4f4200640 1 -- 172.21.15.86:0/1009692539 shutdown_connections 2024-04-17T07:19:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.118+0000 7fa4f4200640 1 -- 172.21.15.86:0/1009692539 wait complete. 2024-04-17T07:19:42.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.119+0000 7fa4f4200640 1 Processor -- start 2024-04-17T07:19:42.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.120+0000 7fa4f4200640 1 -- start start 2024-04-17T07:19:42.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.120+0000 7fa4f4200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ec0749c0 0x7fa4ec19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.120+0000 7fa4f4200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ec075fa0 0x7fa4ec19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.120+0000 7fa4f4200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec10e570 0x7fa4ec1a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.120+0000 7fa4f4200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec114110 con 0x7fa4ec10e570 2024-04-17T07:19:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.120+0000 7fa4f4200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec113f90 con 0x7fa4ec0749c0 2024-04-17T07:19:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.120+0000 7fa4f4200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec114290 con 0x7fa4ec075fa0 2024-04-17T07:19:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.121+0000 7fa4f1800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ec0749c0 0x7fa4ec19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.121+0000 7fa4f1800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ec0749c0 0x7fa4ec19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54034/0 (socket says 172.21.15.86:54034) 2024-04-17T07:19:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.121+0000 7fa4f1800640 1 -- 172.21.15.86:0/1439977225 learned_addr learned my addr 172.21.15.86:0/1439977225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.121+0000 7fa4f2200640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec10e570 0x7fa4ec1a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.121+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ec075fa0 0x7fa4ec19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.121+0000 7fa4f1800640 1 -- 172.21.15.86:0/1439977225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ec075fa0 msgr2=0x7fa4ec19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.121+0000 7fa4f1800640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ec075fa0 0x7fa4ec19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.121+0000 7fa4f1800640 1 -- 172.21.15.86:0/1439977225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec10e570 msgr2=0x7fa4ec1a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.121+0000 7fa4f1800640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec10e570 0x7fa4ec1a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.121+0000 7fa4f1800640 1 -- 172.21.15.86:0/1439977225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec1a6490 con 0x7fa4ec0749c0 2024-04-17T07:19:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.122+0000 7fa4f1800640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ec0749c0 0x7fa4ec19e870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000c00 tx=0x7fa4e00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.123+0000 7fa4ea000640 1 -- 172.21.15.86:0/1439977225 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0043020 con 0x7fa4ec0749c0 2024-04-17T07:19:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.123+0000 7fa4ea000640 1 -- 172.21.15.86:0/1439977225 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4e00044c0 con 0x7fa4ec0749c0 2024-04-17T07:19:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.123+0000 7fa4ea000640 1 -- 172.21.15.86:0/1439977225 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0003c60 con 0x7fa4ec0749c0 2024-04-17T07:19:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.123+0000 7fa4f4200640 1 -- 172.21.15.86:0/1439977225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec1a6720 con 0x7fa4ec0749c0 2024-04-17T07:19:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.123+0000 7fa4f4200640 1 -- 172.21.15.86:0/1439977225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec1a6c20 con 0x7fa4ec0749c0 2024-04-17T07:19:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.126+0000 7fa4f6fcd740 1 Processor -- start 2024-04-17T07:19:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.127+0000 7fa4f6fcd740 1 -- start start 2024-04-17T07:19:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.127+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 0x560b9da403d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.127+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da409a0 0x560b9da454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.127+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da45b10 0x560b9da47fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.127+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b9d995020 con 0x560b9da45b10 2024-04-17T07:19:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.127+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560b9d93b460 con 0x560b9da409a0 2024-04-17T07:19:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.127+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b9d990fe0 con 0x560b9da3ffb0 2024-04-17T07:19:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.127+0000 7fa4f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da45b10 0x560b9da47fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.128+0000 7fa4f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da45b10 0x560b9da47fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50790/0 (socket says 172.21.15.86:50790) 2024-04-17T07:19:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.128+0000 7fa4f0e00640 1 -- 172.21.15.86:0/2220917553 learned_addr learned my addr 172.21.15.86:0/2220917553 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.128+0000 7fa4f2200640 1 --2- 172.21.15.86:0/2220917553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 0x560b9da403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.128+0000 7fa4f0e00640 1 -- 172.21.15.86:0/2220917553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 msgr2=0x560b9da403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.128+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/2220917553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 0x560b9da403d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.128+0000 7fa4f0e00640 1 -- 172.21.15.86:0/2220917553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da409a0 msgr2=0x560b9da454a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:19:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.128+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/2220917553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da409a0 0x560b9da454a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.128+0000 7fa4f0e00640 1 -- 172.21.15.86:0/2220917553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9d943040 con 0x560b9da45b10 2024-04-17T07:19:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.128+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/2220917553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da45b10 0x560b9da47fa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009ca0 tx=0x7fa4d8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aafb753a19c5de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.129+0000 7fa4dea00640 1 -- 172.21.15.86:0/2220917553 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8005220 con 0x560b9da45b10 2024-04-17T07:19:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.129+0000 7fa4dea00640 1 -- 172.21.15.86:0/2220917553 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d8004c50 con 0x560b9da45b10 2024-04-17T07:19:42.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.129+0000 7fa4dea00640 1 -- 172.21.15.86:0/2220917553 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8019830 con 0x560b9da45b10 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.129+0000 7fa4ea000640 1 -- 172.21.15.86:0/1439977225 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa4e0046020 con 0x7fa4ec0749c0 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.130+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/2220917553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da45b10 msgr2=0x560b9da47fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.130+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/2220917553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da45b10 0x560b9da47fa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009ca0 tx=0x7fa4d8011810 comp rx=0 tx=0).stop 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.130+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/2220917553 shutdown_connections 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.130+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/2220917553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da45b10 0x560b9da47fa0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.130+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/2220917553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da409a0 0x560b9da454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.130+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/2220917553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 0x560b9da403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.130+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/2220917553 >> 172.21.15.86:0/2220917553 conn(0x560b9da2a510 msgr2=0x560b9da2a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.130+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/2220917553 shutdown_connections 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.131+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/2220917553 wait complete. 2024-04-17T07:19:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.131+0000 7fa4f6fcd740 1 -- start start 2024-04-17T07:19:42.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.132+0000 7fa4f6fcd740 1 Processor -- start 2024-04-17T07:19:42.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.132+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 0x560b9da4a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4ea000640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4d40a3b50 0x7fa4d40a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da409a0 0x560b9da4ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da4fc90 0x560b9da52080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b9da48ac0 con 0x560b9da4fc90 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560b9d94e830 con 0x560b9da409a0 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560b9da48c40 con 0x560b9da3ffb0 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4f6fcd740 10 client.0 _ll_register_callbacks cb 0x560b9d91cfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4f2200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 0x560b9da4a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4f2200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 0x560b9da4a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41814/0 (socket says 172.21.15.86:41814) 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4f2200640 1 -- 172.21.15.86:0/1437821596 learned_addr learned my addr 172.21.15.86:0/1437821596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.133+0000 7fa4ea000640 1 -- 172.21.15.86:0/1439977225 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa4e00f27a0 con 0x7fa4ec0749c0 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.134+0000 7fa4f2200640 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da4fc90 0x560b9da52080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.134+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da409a0 0x560b9da4ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.134+0000 7fa4f1800640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4d40a3b50 0x7fa4d40a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.134+0000 7fa4f2200640 1 -- 172.21.15.86:0/1437821596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da409a0 msgr2=0x560b9da4ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.134+0000 7fa4f2200640 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da409a0 0x560b9da4ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.134+0000 7fa4f2200640 1 -- 172.21.15.86:0/1437821596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da4fc90 msgr2=0x560b9da52080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.134+0000 7fa4f2200640 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da4fc90 0x560b9da52080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.134+0000 7fa4f2200640 1 -- 172.21.15.86:0/1437821596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9d952920 con 0x560b9da3ffb0 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.135+0000 7fa4f2200640 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 0x560b9da4a720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4005c40 tx=0x7fa4e4009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30573b409c89d707 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.135+0000 7fa4d3400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.135+0000 7fa4d3400640 1 -- 172.21.15.86:0/1437821596 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4014120 con 0x560b9da3ffb0 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.135+0000 7fa4d3400640 1 -- 172.21.15.86:0/1437821596 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa4e4011440 con 0x560b9da3ffb0 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.135+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1437821596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d8004c50 con 0x560b9da3ffb0 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.135+0000 7fa4f1800640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4d40a3b50 0x7fa4d40a6010 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00eff70 tx=0x7fa4e00baa80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.136+0000 7fa4d3400640 1 -- 172.21.15.86:0/1437821596 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4e40142c0 con 0x560b9da3ffb0 2024-04-17T07:19:42.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.136+0000 7fa4f6fcd740 10 client.25942 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:42.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.136+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1437821596 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b9d9207f0 con 0x560b9da3ffb0 2024-04-17T07:19:42.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.136+0000 7fa4d3400640 1 -- 172.21.15.86:0/1437821596 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d8004c50 con 0x560b9da3ffb0 2024-04-17T07:19:42.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.137+0000 7fa4d3400640 1 -- 172.21.15.86:0/1437821596 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa4e400f3c0 con 0x560b9da3ffb0 2024-04-17T07:19:42.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.137+0000 7fa4f6fcd740 10 client.25942 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:42.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.137+0000 7fa4f6fcd740 10 client.25942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:19:42.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.137+0000 7fa4f6fcd740 20 client.25942 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:42.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.137+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x560b9da565b0 0x560b9da58a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.137+0000 7fa4f6fcd740 4 client.25942 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:42.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.137+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1437821596 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b9d9a2a40 con 0x560b9da565b0 2024-04-17T07:19:42.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.139+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x560b9da565b0 0x560b9da58a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.141+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x560b9da565b0 0x560b9da58a30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81201b10c1239443 server_cookie=5011e62a4df5a084 in_seq=0 out_seq=0 2024-04-17T07:19:42.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.141+0000 7fa4d3400640 10 client.25942 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:19:42.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.142+0000 7fa4d3400640 1 -- 172.21.15.86:0/1437821596 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b9d9a2a40 con 0x560b9da565b0 2024-04-17T07:19:42.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.142+0000 7fa4d3400640 10 client.25942 handle_command_reply: tid=0 2024-04-17T07:19:42.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.142+0000 7fa4f6fcd740 1 client.25942 shutdown 2024-04-17T07:19:42.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1437821596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 msgr2=0x560b9da4a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 0x560b9da4a720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4005c40 tx=0x7fa4e4009c70 comp rx=0 tx=0).stop 2024-04-17T07:19:42.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1437821596 shutdown_connections 2024-04-17T07:19:42.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x560b9da565b0 0x560b9da58a30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da4fc90 0x560b9da52080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da409a0 0x560b9da4ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/1437821596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da3ffb0 0x560b9da4a720 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4d3400640 0 client.25942 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:19:42.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1437821596 >> 172.21.15.86:0/1437821596 conn(0x560b9da2a510 msgr2=0x560b9da29710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1437821596 shutdown_connections 2024-04-17T07:19:42.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.143+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1437821596 wait complete. 2024-04-17T07:19:42.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.144+0000 7fa4f6fcd740 20 client.25942 trim_cache size 0 max 16384 2024-04-17T07:19:42.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.175+0000 7fa4f6fcd740 1 Processor -- start 2024-04-17T07:19:42.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.175+0000 7fa4f6fcd740 1 -- start start 2024-04-17T07:19:42.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.176+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 0x560b9da4aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.176+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da409a0 0x560b9da28e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.176+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da4fc90 0x560b9da293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.176+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1a88b0 con 0x560b9da4fc90 2024-04-17T07:19:42.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.176+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1aa3f0 con 0x560b9da3ffb0 2024-04-17T07:19:42.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.176+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1a8f80 con 0x560b9da409a0 2024-04-17T07:19:42.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.177+0000 7fa4f2200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 0x560b9da4aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.177+0000 7fa4f2200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 0x560b9da4aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54052/0 (socket says 172.21.15.86:54052) 2024-04-17T07:19:42.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.177+0000 7fa4f2200640 1 -- 172.21.15.86:0/1572036309 learned_addr learned my addr 172.21.15.86:0/1572036309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.177+0000 7fa4f2200640 1 --2- 172.21.15.86:0/1572036309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da4fc90 0x560b9da293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.177+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/1572036309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da409a0 0x560b9da28e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.177+0000 7fa4f2200640 1 -- 172.21.15.86:0/1572036309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da409a0 msgr2=0x560b9da28e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.177+0000 7fa4f2200640 1 --2- 172.21.15.86:0/1572036309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da409a0 0x560b9da28e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.177+0000 7fa4f2200640 1 -- 172.21.15.86:0/1572036309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da4fc90 msgr2=0x560b9da293c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.177+0000 7fa4f2200640 1 --2- 172.21.15.86:0/1572036309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da4fc90 0x560b9da293c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.177+0000 7fa4f2200640 1 -- 172.21.15.86:0/1572036309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9d6a73f0 con 0x560b9da3ffb0 2024-04-17T07:19:42.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.178+0000 7fa4f2200640 1 --2- 172.21.15.86:0/1572036309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 0x560b9da4aa40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4002230 tx=0x7fa4e4005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd5e63fa8aac497 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.179+0000 7fa4de000640 1 -- 172.21.15.86:0/1572036309 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4025030 con 0x560b9da3ffb0 2024-04-17T07:19:42.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.179+0000 7fa4de000640 1 -- 172.21.15.86:0/1572036309 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4e4005920 con 0x560b9da3ffb0 2024-04-17T07:19:42.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.179+0000 7fa4de000640 1 -- 172.21.15.86:0/1572036309 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4011440 con 0x560b9da3ffb0 2024-04-17T07:19:42.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.180+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1572036309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 msgr2=0x560b9da4aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.180+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/1572036309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 0x560b9da4aa40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4002230 tx=0x7fa4e4005f10 comp rx=0 tx=0).stop 2024-04-17T07:19:42.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.180+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1572036309 shutdown_connections 2024-04-17T07:19:42.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.180+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/1572036309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da4fc90 0x560b9da293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.180+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/1572036309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da409a0 0x560b9da28e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.180+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/1572036309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 0x560b9da4aa40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.180+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1572036309 >> 172.21.15.86:0/1572036309 conn(0x560b9da2a510 msgr2=0x560b9da4a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.180+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1572036309 shutdown_connections 2024-04-17T07:19:42.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.180+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/1572036309 wait complete. 2024-04-17T07:19:42.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.181+0000 7fa4f6fcd740 1 -- start start 2024-04-17T07:19:42.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.181+0000 7fa4f6fcd740 1 Processor -- start 2024-04-17T07:19:42.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 0x560b9da67ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da409a0 0x560b9da68020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f6fcd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da4fc90 0x560b9da68fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1a9d20 con 0x560b9da409a0 2024-04-17T07:19:42.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1a9650 con 0x560b9da3ffb0 2024-04-17T07:19:42.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f6fcd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec10e000 con 0x560b9da4fc90 2024-04-17T07:19:42.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f6fcd740 10 client.0 _ll_register_callbacks cb 0x7fa4e40041f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da409a0 0x560b9da68020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da409a0 0x560b9da68020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50824/0 (socket says 172.21.15.86:50824) 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f0e00640 1 -- 172.21.15.86:0/447068711 learned_addr learned my addr 172.21.15.86:0/447068711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f2200640 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 0x560b9da67ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f0e00640 1 -- 172.21.15.86:0/447068711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da4fc90 msgr2=0x560b9da68fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da4fc90 0x560b9da68fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f0e00640 1 -- 172.21.15.86:0/447068711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 msgr2=0x560b9da67ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 0x560b9da67ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.182+0000 7fa4f0e00640 1 -- 172.21.15.86:0/447068711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9d7717d0 con 0x560b9da409a0 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.183+0000 7fa4f0e00640 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da409a0 0x560b9da68020 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa4d801b040 tx=0x7fa4d80056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e21fc0f39d9bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.183+0000 7fa4d3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:42.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.184+0000 7fa4d3400640 1 -- 172.21.15.86:0/447068711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8019840 con 0x560b9da409a0 2024-04-17T07:19:42.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.184+0000 7fa4d3400640 1 -- 172.21.15.86:0/447068711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa4d80199e0 con 0x560b9da409a0 2024-04-17T07:19:42.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.184+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/447068711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e4005920 con 0x560b9da409a0 2024-04-17T07:19:42.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.185+0000 7fa4d3400640 1 -- 172.21.15.86:0/447068711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4d8019c00 con 0x560b9da409a0 2024-04-17T07:19:42.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.186+0000 7fa4f6fcd740 10 client.16917 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:42.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.186+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/447068711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b9dabca30 con 0x560b9da409a0 2024-04-17T07:19:42.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.186+0000 7fa4d3400640 1 -- 172.21.15.86:0/447068711 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4e4005920 con 0x560b9da409a0 2024-04-17T07:19:42.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.186+0000 7fa4d3400640 1 -- 172.21.15.86:0/447068711 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa4d8025cd0 con 0x560b9da409a0 2024-04-17T07:19:42.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.187+0000 7fa4f6fcd740 10 client.16917 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:42.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.187+0000 7fa4f6fcd740 10 client.16917 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:19:42.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.187+0000 7fa4f6fcd740 20 client.16917 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:42.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.187+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x560b9dabd370 0x560b9dabf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.187+0000 7fa4f6fcd740 4 client.16917 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:19:42.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.187+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/447068711 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b9d99c8b0 con 0x560b9dabd370 2024-04-17T07:19:42.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.187+0000 7fa4f2200640 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x560b9dabd370 0x560b9dabf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.189+0000 7fa4d3400640 10 client.16917 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:19:42.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.189+0000 7fa4f2200640 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x560b9dabd370 0x560b9dabf790 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=977b2b3f50be65c8 server_cookie=dc21d40a714c1cd3 in_seq=0 out_seq=0 2024-04-17T07:19:42.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.189+0000 7fa4d3400640 1 -- 172.21.15.86:0/447068711 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b9d99c8b0 con 0x560b9dabd370 2024-04-17T07:19:42.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4d3400640 10 client.16917 handle_command_reply: tid=0 2024-04-17T07:19:42.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4f6fcd740 1 client.16917 shutdown 2024-04-17T07:19:42.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/447068711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da409a0 msgr2=0x560b9da68020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da409a0 0x560b9da68020 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa4d801b040 tx=0x7fa4d80056b0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/447068711 shutdown_connections 2024-04-17T07:19:42.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x560b9dabd370 0x560b9dabf790 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560b9da4fc90 0x560b9da68fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b9da409a0 0x560b9da68020 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4f6fcd740 1 --2- 172.21.15.86:0/447068711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b9da3ffb0 0x560b9da67ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4d3400640 0 client.16917 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:19:42.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.190+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/447068711 >> 172.21.15.86:0/447068711 conn(0x560b9da2a510 msgr2=0x560b9da2d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.191+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/447068711 shutdown_connections 2024-04-17T07:19:42.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.191+0000 7fa4f6fcd740 1 -- 172.21.15.86:0/447068711 wait complete. 2024-04-17T07:19:42.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.191+0000 7fa4f6fcd740 20 client.16917 trim_cache size 0 max 16384 2024-04-17T07:19:42.191 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:19:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.192+0000 7fa4f4200640 1 -- 172.21.15.86:0/1439977225 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4d40a3b50 msgr2=0x7fa4d40a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.192+0000 7fa4f4200640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4d40a3b50 0x7fa4d40a6010 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00eff70 tx=0x7fa4e00baa80 comp rx=0 tx=0).stop 2024-04-17T07:19:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.192+0000 7fa4f4200640 1 -- 172.21.15.86:0/1439977225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ec0749c0 msgr2=0x7fa4ec19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.192+0000 7fa4f4200640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ec0749c0 0x7fa4ec19e870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000c00 tx=0x7fa4e00040c0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.192+0000 7fa4f4200640 1 -- 172.21.15.86:0/1439977225 shutdown_connections 2024-04-17T07:19:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.192+0000 7fa4f4200640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4d40a3b50 0x7fa4d40a6010 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.192+0000 7fa4f4200640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4ec10e570 0x7fa4ec1a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.192+0000 7fa4f4200640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4ec075fa0 0x7fa4ec19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.192+0000 7fa4f4200640 1 --2- 172.21.15.86:0/1439977225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4ec0749c0 0x7fa4ec19e870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.192+0000 7fa4f4200640 1 -- 172.21.15.86:0/1439977225 >> 172.21.15.86:0/1439977225 conn(0x7fa4ec0fd510 msgr2=0x7fa4ec101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.193+0000 7fa4f4200640 1 -- 172.21.15.86:0/1439977225 shutdown_connections 2024-04-17T07:19:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.193+0000 7fa4f4200640 1 -- 172.21.15.86:0/1439977225 wait complete. 2024-04-17T07:19:42.223 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:19:42.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.518+0000 7fb070c00640 1 -- 172.21.15.86:0/2435473119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb06c0ffe40 msgr2=0x7fb06c100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.518+0000 7fb070c00640 1 --2- 172.21.15.86:0/2435473119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb06c0ffe40 0x7fb06c100260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009900 tx=0x7fb05c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.518+0000 7fb070c00640 1 -- 172.21.15.86:0/2435473119 shutdown_connections 2024-04-17T07:19:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.518+0000 7fb070c00640 1 --2- 172.21.15.86:0/2435473119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c101910 0x7fb06c1061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.518+0000 7fb070c00640 1 --2- 172.21.15.86:0/2435473119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c100f70 0x7fb06c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.518+0000 7fb070c00640 1 --2- 172.21.15.86:0/2435473119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb06c0ffe40 0x7fb06c100260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.518+0000 7fb070c00640 1 -- 172.21.15.86:0/2435473119 >> 172.21.15.86:0/2435473119 conn(0x7fb06c0fb760 msgr2=0x7fb06c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.518+0000 7fb070c00640 1 -- 172.21.15.86:0/2435473119 shutdown_connections 2024-04-17T07:19:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.518+0000 7fb070c00640 1 -- 172.21.15.86:0/2435473119 wait complete. 2024-04-17T07:19:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.520+0000 7fb070c00640 1 Processor -- start 2024-04-17T07:19:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.520+0000 7fb070c00640 1 -- start start 2024-04-17T07:19:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.520+0000 7fb070c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb06c0ffe40 0x7fb06c0733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.520+0000 7fb070c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c100f70 0x7fb06c073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.520+0000 7fb070c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c101910 0x7fb06c071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.520+0000 7fb070c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c06c240 con 0x7fb06c101910 2024-04-17T07:19:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.520+0000 7fb070c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c06c0c0 con 0x7fb06c0ffe40 2024-04-17T07:19:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.520+0000 7fb070c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c06c3c0 con 0x7fb06c100f70 2024-04-17T07:19:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.521+0000 7fb06a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c101910 0x7fb06c071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.521+0000 7fb06a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c101910 0x7fb06c071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50842/0 (socket says 172.21.15.86:50842) 2024-04-17T07:19:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.521+0000 7fb06a800640 1 -- 172.21.15.86:0/2626982987 learned_addr learned my addr 172.21.15.86:0/2626982987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.521+0000 7fb069400640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c100f70 0x7fb06c073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.522+0000 7fb069e00640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb06c0ffe40 0x7fb06c0733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.522+0000 7fb06a800640 1 -- 172.21.15.86:0/2626982987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c100f70 msgr2=0x7fb06c073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.522+0000 7fb06a800640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c100f70 0x7fb06c073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.522+0000 7fb06a800640 1 -- 172.21.15.86:0/2626982987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb06c0ffe40 msgr2=0x7fb06c0733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.522+0000 7fb06a800640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb06c0ffe40 0x7fb06c0733d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.522+0000 7fb06a800640 1 -- 172.21.15.86:0/2626982987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c0720b0 con 0x7fb06c101910 2024-04-17T07:19:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.522+0000 7fb06a800640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c101910 0x7fb06c071a10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb06400eb10 tx=0x7fb06400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.522+0000 7fb062a00640 1 -- 172.21.15.86:0/2626982987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064004b20 con 0x7fb06c101910 2024-04-17T07:19:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.523+0000 7fb070c00640 1 -- 172.21.15.86:0/2626982987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c0723a0 con 0x7fb06c101910 2024-04-17T07:19:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.523+0000 7fb070c00640 1 -- 172.21.15.86:0/2626982987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1af580 con 0x7fb06c101910 2024-04-17T07:19:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.523+0000 7fb062a00640 1 -- 172.21.15.86:0/2626982987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb064004cc0 con 0x7fb06c101910 2024-04-17T07:19:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.523+0000 7fb062a00640 1 -- 172.21.15.86:0/2626982987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064010690 con 0x7fb06c101910 2024-04-17T07:19:42.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.527+0000 7fb0739aa740 1 Processor -- start 2024-04-17T07:19:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.528+0000 7fb062a00640 1 -- 172.21.15.86:0/2626982987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb064010830 con 0x7fb06c101910 2024-04-17T07:19:42.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.528+0000 7fb0739aa740 1 -- start start 2024-04-17T07:19:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.529+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8965ffa0 0x55de896603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.529+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de89665490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.529+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89665b00 0x55de89667f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.529+0000 7fb0739aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de895c6020 con 0x55de8965ffa0 2024-04-17T07:19:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.529+0000 7fb0739aa740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55de8956c460 con 0x55de89665b00 2024-04-17T07:19:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.529+0000 7fb0739aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55de895c1fe0 con 0x55de89660990 2024-04-17T07:19:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.530+0000 7fb069400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8965ffa0 0x55de896603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.530+0000 7fb069400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8965ffa0 0x55de896603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50848/0 (socket says 172.21.15.86:50848) 2024-04-17T07:19:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.530+0000 7fb069400640 1 -- 172.21.15.86:0/1569197431 learned_addr learned my addr 172.21.15.86:0/1569197431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.530+0000 7fb069e00640 1 --2- 172.21.15.86:0/1569197431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de89665490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.531+0000 7fb069e00640 1 -- 172.21.15.86:0/1569197431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89665b00 msgr2=0x55de89667f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:19:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.531+0000 7fb069e00640 1 --2- 172.21.15.86:0/1569197431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89665b00 0x55de89667f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.531+0000 7fb069e00640 1 -- 172.21.15.86:0/1569197431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8965ffa0 msgr2=0x55de896603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.531+0000 7fb069e00640 1 --2- 172.21.15.86:0/1569197431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8965ffa0 0x55de896603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.531+0000 7fb069e00640 1 -- 172.21.15.86:0/1569197431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de89574040 con 0x55de89660990 2024-04-17T07:19:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.532+0000 7fb062a00640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0500a3c80 0x7fb0500a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.532+0000 7fb069e00640 1 --2- 172.21.15.86:0/1569197431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de89665490 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009850 tx=0x7fb05c005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5981e07f55cb20 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.533+0000 7fb057000640 1 -- 172.21.15.86:0/1569197431 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c044020 con 0x55de89660990 2024-04-17T07:19:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.533+0000 7fb057000640 1 -- 172.21.15.86:0/1569197431 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb05c02cba0 con 0x55de89660990 2024-04-17T07:19:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.533+0000 7fb057000640 1 -- 172.21.15.86:0/1569197431 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0046c0 con 0x55de89660990 2024-04-17T07:19:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.533+0000 7fb062a00640 1 -- 172.21.15.86:0/2626982987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb0640d0c50 con 0x7fb06c101910 2024-04-17T07:19:42.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.534+0000 7fb069400640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0500a3c80 0x7fb0500a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.535+0000 7fb0739aa740 1 -- 172.21.15.86:0/1569197431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 msgr2=0x55de89665490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.535+0000 7fb0739aa740 1 --2- 172.21.15.86:0/1569197431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de89665490 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009850 tx=0x7fb05c005b50 comp rx=0 tx=0).stop 2024-04-17T07:19:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.535+0000 7fb0739aa740 1 -- 172.21.15.86:0/1569197431 shutdown_connections 2024-04-17T07:19:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.535+0000 7fb0739aa740 1 --2- 172.21.15.86:0/1569197431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89665b00 0x55de89667f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.535+0000 7fb0739aa740 1 --2- 172.21.15.86:0/1569197431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de89665490 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.535+0000 7fb0739aa740 1 --2- 172.21.15.86:0/1569197431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de8965ffa0 0x55de896603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.535+0000 7fb0739aa740 1 -- 172.21.15.86:0/1569197431 >> 172.21.15.86:0/1569197431 conn(0x55de8965b510 msgr2=0x55de8965b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.536+0000 7fb0739aa740 1 -- 172.21.15.86:0/1569197431 shutdown_connections 2024-04-17T07:19:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.536+0000 7fb069400640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0500a3c80 0x7fb0500a6140 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb0580058c0 tx=0x7fb058005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.537+0000 7fb0739aa740 1 -- 172.21.15.86:0/1569197431 wait complete. 2024-04-17T07:19:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.537+0000 7fb0739aa740 1 -- start start 2024-04-17T07:19:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.538+0000 7fb0739aa740 1 Processor -- start 2024-04-17T07:19:42.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.539+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8966a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.539+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de8966a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.539+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de89671790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.540+0000 7fb0739aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de89668a50 con 0x55de89665b00 2024-04-17T07:19:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.540+0000 7fb0739aa740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55de8957f830 con 0x55de8965ffa0 2024-04-17T07:19:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.540+0000 7fb0739aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55de89668bd0 con 0x55de89660990 2024-04-17T07:19:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.540+0000 7fb0739aa740 10 client.0 _ll_register_callbacks cb 0x55de8954dfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.540+0000 7fb06a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de89671790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.540+0000 7fb069e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8966a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.540+0000 7fb069400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de8966a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.540+0000 7fb069e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8966a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54110/0 (socket says 172.21.15.86:54110) 2024-04-17T07:19:42.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.540+0000 7fb069e00640 1 -- 172.21.15.86:0/320267479 learned_addr learned my addr 172.21.15.86:0/320267479 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.541+0000 7fb069400640 1 -- 172.21.15.86:0/320267479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 msgr2=0x55de8966a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.541+0000 7fb069400640 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8966a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.541+0000 7fb069400640 1 -- 172.21.15.86:0/320267479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 msgr2=0x55de89671790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.541+0000 7fb069400640 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de89671790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.541+0000 7fb069400640 1 -- 172.21.15.86:0/320267479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de89583920 con 0x55de89660990 2024-04-17T07:19:42.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.542+0000 7fb06a800640 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de89671790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.543+0000 7fb069400640 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de8966a750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55de896611f0 tx=0x7fb058004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217249dd26d6167d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.543+0000 7fb04b400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:19:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.544+0000 7fb04b400640 1 -- 172.21.15.86:0/320267479 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058029440 con 0x55de89660990 2024-04-17T07:19:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.544+0000 7fb04b400640 1 -- 172.21.15.86:0/320267479 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb05801f430 con 0x55de89660990 2024-04-17T07:19:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.544+0000 7fb0739aa740 1 -- 172.21.15.86:0/320267479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c02cba0 con 0x55de89660990 2024-04-17T07:19:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.545+0000 7fb04b400640 1 -- 172.21.15.86:0/320267479 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb05801f630 con 0x55de89660990 2024-04-17T07:19:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.545+0000 7fb0739aa740 10 client.25954 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.545+0000 7fb0739aa740 1 -- 172.21.15.86:0/320267479 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de895517f0 con 0x55de89660990 2024-04-17T07:19:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.546+0000 7fb04b400640 1 -- 172.21.15.86:0/320267479 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb05c02cba0 con 0x55de89660990 2024-04-17T07:19:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.546+0000 7fb04b400640 1 -- 172.21.15.86:0/320267479 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb058027070 con 0x55de89660990 2024-04-17T07:19:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.546+0000 7fb0739aa740 10 client.25954 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.546+0000 7fb0739aa740 10 client.25954 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:19:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.546+0000 7fb0739aa740 20 client.25954 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.546+0000 7fb0739aa740 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55de89675f20 0x55de89678380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.546+0000 7fb0739aa740 4 client.25954 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:42.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.546+0000 7fb0739aa740 1 -- 172.21.15.86:0/320267479 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de895d3a40 con 0x55de89675f20 2024-04-17T07:19:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.547+0000 7fb069e00640 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55de89675f20 0x55de89678380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.548+0000 7fb069e00640 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55de89675f20 0x55de89678380 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c022ad5948cf6c7 server_cookie=c6df66c24e2c568f in_seq=0 out_seq=0 2024-04-17T07:19:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.548+0000 7fb04b400640 10 client.25954 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:19:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.549+0000 7fb04b400640 1 -- 172.21.15.86:0/320267479 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de895d3a40 con 0x55de89675f20 2024-04-17T07:19:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.549+0000 7fb04b400640 10 client.25954 handle_command_reply: tid=0 2024-04-17T07:19:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.550+0000 7fb0739aa740 1 client.25954 shutdown 2024-04-17T07:19:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.550+0000 7fb0739aa740 1 -- 172.21.15.86:0/320267479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 msgr2=0x55de8966a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.550+0000 7fb0739aa740 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de8966a750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55de896611f0 tx=0x7fb058004730 comp rx=0 tx=0).stop 2024-04-17T07:19:42.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.551+0000 7fb0739aa740 1 -- 172.21.15.86:0/320267479 shutdown_connections 2024-04-17T07:19:42.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.551+0000 7fb0739aa740 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de89671790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.551+0000 7fb0739aa740 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de8966a750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.551+0000 7fb0739aa740 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55de89675f20 0x55de89678380 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.551+0000 7fb04b400640 0 client.25954 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:19:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.551+0000 7fb0739aa740 1 --2- 172.21.15.86:0/320267479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8966a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.551+0000 7fb0739aa740 1 -- 172.21.15.86:0/320267479 >> 172.21.15.86:0/320267479 conn(0x55de8965b510 msgr2=0x55de8965a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.551+0000 7fb0739aa740 1 -- 172.21.15.86:0/320267479 shutdown_connections 2024-04-17T07:19:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.551+0000 7fb0739aa740 1 -- 172.21.15.86:0/320267479 wait complete. 2024-04-17T07:19:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.552+0000 7fb0739aa740 20 client.25954 trim_cache size 0 max 16384 2024-04-17T07:19:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.577+0000 7fb0739aa740 1 Processor -- start 2024-04-17T07:19:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.578+0000 7fb0739aa740 1 -- start start 2024-04-17T07:19:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.580+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8965e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.580+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de89671750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.580+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de8965ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.580+0000 7fb0739aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c00fb50 con 0x55de89665b00 2024-04-17T07:19:42.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.580+0000 7fb0739aa740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c011690 con 0x55de8965ffa0 2024-04-17T07:19:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.580+0000 7fb0739aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c010220 con 0x55de89660990 2024-04-17T07:19:42.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.580+0000 7fb06a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de8965ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.580+0000 7fb069e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8965e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.580+0000 7fb069e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8965e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54122/0 (socket says 172.21.15.86:54122) 2024-04-17T07:19:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.581+0000 7fb069e00640 1 -- 172.21.15.86:0/316056753 learned_addr learned my addr 172.21.15.86:0/316056753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.581+0000 7fb069400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de89671750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.580+0000 7fb06a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de8965ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50864/0 (socket says 172.21.15.86:50864) 2024-04-17T07:19:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.581+0000 7fb069e00640 1 -- 172.21.15.86:0/316056753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 msgr2=0x55de89671750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.581+0000 7fb069e00640 1 --2- 172.21.15.86:0/316056753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de89671750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.581+0000 7fb069e00640 1 -- 172.21.15.86:0/316056753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 msgr2=0x55de8965ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.581+0000 7fb069e00640 1 --2- 172.21.15.86:0/316056753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de8965ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.581+0000 7fb069e00640 1 -- 172.21.15.86:0/316056753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de892d83f0 con 0x55de8965ffa0 2024-04-17T07:19:42.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.582+0000 7fb069e00640 1 --2- 172.21.15.86:0/316056753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8965e770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb05c02c240 tx=0x7fb05c02cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aeba9f8c01a9ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.583+0000 7fb056600640 1 -- 172.21.15.86:0/316056753 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c051070 con 0x55de8965ffa0 2024-04-17T07:19:42.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.583+0000 7fb056600640 1 -- 172.21.15.86:0/316056753 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb05c02cba0 con 0x55de8965ffa0 2024-04-17T07:19:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.583+0000 7fb056600640 1 -- 172.21.15.86:0/316056753 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c044040 con 0x55de8965ffa0 2024-04-17T07:19:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.584+0000 7fb0739aa740 1 -- 172.21.15.86:0/316056753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 msgr2=0x55de8965e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.584+0000 7fb0739aa740 1 --2- 172.21.15.86:0/316056753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8965e770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb05c02c240 tx=0x7fb05c02cd50 comp rx=0 tx=0).stop 2024-04-17T07:19:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.584+0000 7fb0739aa740 1 -- 172.21.15.86:0/316056753 shutdown_connections 2024-04-17T07:19:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.584+0000 7fb0739aa740 1 --2- 172.21.15.86:0/316056753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de8965ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.584+0000 7fb0739aa740 1 --2- 172.21.15.86:0/316056753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de89660990 0x55de89671750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.584+0000 7fb0739aa740 1 --2- 172.21.15.86:0/316056753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de8965ffa0 0x55de8965e770 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.584+0000 7fb0739aa740 1 -- 172.21.15.86:0/316056753 >> 172.21.15.86:0/316056753 conn(0x55de8965b510 msgr2=0x55de89661c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.585+0000 7fb0739aa740 1 -- 172.21.15.86:0/316056753 shutdown_connections 2024-04-17T07:19:42.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.586+0000 7fb0739aa740 1 -- 172.21.15.86:0/316056753 wait complete. 2024-04-17T07:19:42.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.586+0000 7fb0739aa740 1 -- start start 2024-04-17T07:19:42.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.587+0000 7fb0739aa740 1 Processor -- start 2024-04-17T07:19:42.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.587+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8965ffa0 0x55de8966eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.587+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89660990 0x55de8966f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.587+0000 7fb0739aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de896700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.587+0000 7fb0739aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c010fc0 con 0x55de89665b00 2024-04-17T07:19:42.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.587+0000 7fb0739aa740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c0108f0 con 0x55de89660990 2024-04-17T07:19:42.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.587+0000 7fb0739aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c00db20 con 0x55de8965ffa0 2024-04-17T07:19:42.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.587+0000 7fb0739aa740 10 client.0 _ll_register_callbacks cb 0x7fb0580044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:42.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.588+0000 7fb069400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89660990 0x55de8966f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.588+0000 7fb069400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89660990 0x55de8966f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54130/0 (socket says 172.21.15.86:54130) 2024-04-17T07:19:42.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.588+0000 7fb069400640 1 -- 172.21.15.86:0/2515284020 learned_addr learned my addr 172.21.15.86:0/2515284020 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.588+0000 7fb06a800640 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de896700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.588+0000 7fb069e00640 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8965ffa0 0x55de8966eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.588+0000 7fb069400640 1 -- 172.21.15.86:0/2515284020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8965ffa0 msgr2=0x55de8966eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.588+0000 7fb069400640 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8965ffa0 0x55de8966eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.588+0000 7fb069400640 1 -- 172.21.15.86:0/2515284020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 msgr2=0x55de896700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.589+0000 7fb069400640 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de896700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.589+0000 7fb069400640 1 -- 172.21.15.86:0/2515284020 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de893a27d0 con 0x55de89660990 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.589+0000 7fb06a800640 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de896700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.589+0000 7fb069400640 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89660990 0x55de8966f090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb058013010 tx=0x7fb058003fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4eb6c06e457e7db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.589+0000 7fb04b400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.590+0000 7fb04b400640 1 -- 172.21.15.86:0/2515284020 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05801f430 con 0x55de89660990 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.590+0000 7fb04b400640 1 -- 172.21.15.86:0/2515284020 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb058029440 con 0x55de89660990 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.590+0000 7fb0739aa740 1 -- 172.21.15.86:0/2515284020 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c02cba0 con 0x55de89660990 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.590+0000 7fb04b400640 1 -- 172.21.15.86:0/2515284020 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb058043050 con 0x55de89660990 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.590+0000 7fb0739aa740 10 client.35927 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:42.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.591+0000 7fb0739aa740 1 -- 172.21.15.86:0/2515284020 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de89671860 con 0x55de89660990 2024-04-17T07:19:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.591+0000 7fb04b400640 1 -- 172.21.15.86:0/2515284020 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb05c02cba0 con 0x55de89660990 2024-04-17T07:19:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.591+0000 7fb04b400640 1 -- 172.21.15.86:0/2515284020 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb058027070 con 0x55de89660990 2024-04-17T07:19:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.591+0000 7fb0739aa740 10 client.35927 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.591+0000 7fb0739aa740 10 client.35927 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:19:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.591+0000 7fb0739aa740 20 client.35927 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.591+0000 7fb0739aa740 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55de896dcac0 0x55de896deee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.592+0000 7fb0739aa740 4 client.35927 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:19:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.592+0000 7fb0739aa740 1 -- 172.21.15.86:0/2515284020 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de895cd8b0 con 0x55de896dcac0 2024-04-17T07:19:42.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.592+0000 7fb069e00640 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55de896dcac0 0x55de896deee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.593+0000 7fb069e00640 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55de896dcac0 0x55de896deee0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd0b2e9ade189f80 server_cookie=1af1bfb6fefc6d30 in_seq=0 out_seq=0 2024-04-17T07:19:42.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.593+0000 7fb04b400640 10 client.35927 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:19:42.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.593+0000 7fb04b400640 1 -- 172.21.15.86:0/2515284020 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de895cd8b0 con 0x55de896dcac0 2024-04-17T07:19:42.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.593+0000 7fb04b400640 10 client.35927 handle_command_reply: tid=0 2024-04-17T07:19:42.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.593+0000 7fb0739aa740 1 client.35927 shutdown 2024-04-17T07:19:42.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.594+0000 7fb0739aa740 1 -- 172.21.15.86:0/2515284020 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89660990 msgr2=0x55de8966f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.594+0000 7fb0739aa740 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89660990 0x55de8966f090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb058013010 tx=0x7fb058003fd0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.594+0000 7fb0739aa740 1 -- 172.21.15.86:0/2515284020 shutdown_connections 2024-04-17T07:19:42.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.594+0000 7fb0739aa740 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de89665b00 0x55de896700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.594+0000 7fb0739aa740 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55de896dcac0 0x55de896deee0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.594+0000 7fb0739aa740 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de89660990 0x55de8966f090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.594+0000 7fb0739aa740 1 --2- 172.21.15.86:0/2515284020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de8965ffa0 0x55de8966eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.594+0000 7fb0739aa740 1 -- 172.21.15.86:0/2515284020 >> 172.21.15.86:0/2515284020 conn(0x55de8965b510 msgr2=0x55de89661c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.594+0000 7fb04b400640 0 client.35927 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:19:42.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.594+0000 7fb0739aa740 1 -- 172.21.15.86:0/2515284020 shutdown_connections 2024-04-17T07:19:42.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.595+0000 7fb0739aa740 1 -- 172.21.15.86:0/2515284020 wait complete. 2024-04-17T07:19:42.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.595+0000 7fb0739aa740 20 client.35927 trim_cache size 0 max 16384 2024-04-17T07:19:42.595 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 -- 172.21.15.86:0/2626982987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0500a3c80 msgr2=0x7fb0500a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0500a3c80 0x7fb0500a6140 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb0580058c0 tx=0x7fb058005f90 comp rx=0 tx=0).stop 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 -- 172.21.15.86:0/2626982987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c101910 msgr2=0x7fb06c071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c101910 0x7fb06c071a10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb06400eb10 tx=0x7fb06400c2d0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 -- 172.21.15.86:0/2626982987 shutdown_connections 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0500a3c80 0x7fb0500a6140 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb06c101910 0x7fb06c071a10 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c100f70 0x7fb06c073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 --2- 172.21.15.86:0/2626982987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb06c0ffe40 0x7fb06c0733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 -- 172.21.15.86:0/2626982987 >> 172.21.15.86:0/2626982987 conn(0x7fb06c0fb760 msgr2=0x7fb06c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.596+0000 7fb070c00640 1 -- 172.21.15.86:0/2626982987 shutdown_connections 2024-04-17T07:19:42.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.597+0000 7fb070c00640 1 -- 172.21.15.86:0/2626982987 wait complete. 2024-04-17T07:19:42.623 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:19:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.915+0000 7f7faaa00640 1 -- 172.21.15.86:0/3539143439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa41053d0 msgr2=0x7f7fa4105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.915+0000 7f7faaa00640 1 --2- 172.21.15.86:0/3539143439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa41053d0 0x7f7fa4105850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009900 tx=0x7f7f9002bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.916+0000 7f7faaa00640 1 -- 172.21.15.86:0/3539143439 shutdown_connections 2024-04-17T07:19:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.916+0000 7f7faaa00640 1 --2- 172.21.15.86:0/3539143439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa4105d90 0x7f7fa410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.916+0000 7f7faaa00640 1 --2- 172.21.15.86:0/3539143439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa41053d0 0x7f7fa4105850 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.916+0000 7f7faaa00640 1 --2- 172.21.15.86:0/3539143439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa41041e0 0x7f7fa4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.916+0000 7f7faaa00640 1 -- 172.21.15.86:0/3539143439 >> 172.21.15.86:0/3539143439 conn(0x7f7fa40ff750 msgr2=0x7f7fa4101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.917+0000 7f7faaa00640 1 -- 172.21.15.86:0/3539143439 shutdown_connections 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.917+0000 7f7faaa00640 1 -- 172.21.15.86:0/3539143439 wait complete. 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.918+0000 7f7faaa00640 1 Processor -- start 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.918+0000 7f7faaa00640 1 -- start start 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7faaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa41041e0 0x7f7fa419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7faaa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa41053d0 0x7f7fa419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7faaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa4105d90 0x7f7fa41a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7faaa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa4076d80 con 0x7f7fa4105d90 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7faaa00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa4076c00 con 0x7f7fa41053d0 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7faaa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa4076f00 con 0x7f7fa41041e0 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7fa3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa41053d0 0x7f7fa419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7fa3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa41053d0 0x7f7fa419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54146/0 (socket says 172.21.15.86:54146) 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7fa3400640 1 -- 172.21.15.86:0/4169464946 learned_addr learned my addr 172.21.15.86:0/4169464946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa41041e0 0x7f7fa419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.919+0000 7f7fa8a00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa4105d90 0x7f7fa41a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.920+0000 7f7fa3400640 1 -- 172.21.15.86:0/4169464946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa41041e0 msgr2=0x7f7fa419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.920+0000 7f7fa3400640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa41041e0 0x7f7fa419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.920+0000 7f7fa3400640 1 -- 172.21.15.86:0/4169464946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa4105d90 msgr2=0x7f7fa41a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.920+0000 7f7fa3400640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa4105d90 0x7f7fa41a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.920+0000 7f7fa3400640 1 -- 172.21.15.86:0/4169464946 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa41a6420 con 0x7f7fa41053d0 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.921+0000 7f7fa3400640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa41053d0 0x7f7fa419ed40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f90000c00 tx=0x7f7f900042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.921+0000 7f7fa0c00640 1 -- 172.21.15.86:0/4169464946 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90043020 con 0x7f7fa41053d0 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.922+0000 7f7fa0c00640 1 -- 172.21.15.86:0/4169464946 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f900054c0 con 0x7f7fa41053d0 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.922+0000 7f7fa0c00640 1 -- 172.21.15.86:0/4169464946 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90003d40 con 0x7f7fa41053d0 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.922+0000 7f7faaa00640 1 -- 172.21.15.86:0/4169464946 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa41a66b0 con 0x7f7fa41053d0 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.922+0000 7f7faaa00640 1 -- 172.21.15.86:0/4169464946 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa41a6bb0 con 0x7f7fa41053d0 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.927+0000 7f7fa0c00640 1 -- 172.21.15.86:0/4169464946 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7f90046020 con 0x7f7fa41053d0 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.929+0000 7f7fad79f740 1 Processor -- start 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.929+0000 7f7fad79f740 1 -- start start 2024-04-17T07:19:42.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.929+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8befa0 0x55f77b8bf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.929+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8bf990 0x55f77b8c4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.929+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.929+0000 7f7fa0c00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f880a3c30 0x7f7f880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.929+0000 7f7fad79f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f77b825020 con 0x55f77b8c4b00 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.929+0000 7f7fad79f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f77b7cb460 con 0x55f77b8befa0 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.929+0000 7f7fad79f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f77b820fe0 con 0x55f77b8bf990 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.930+0000 7f7fa0c00640 1 -- 172.21.15.86:0/4169464946 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7f900f2a50 con 0x7f7fa41053d0 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.930+0000 7f7fa3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.930+0000 7f7fa8a00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f880a3c30 0x7f7f880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.930+0000 7f7fa3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50900/0 (socket says 172.21.15.86:50900) 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.930+0000 7f7fa3400640 1 -- 172.21.15.86:0/3506604833 learned_addr learned my addr 172.21.15.86:0/3506604833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.930+0000 7f7fa8a00640 1 --2- 172.21.15.86:0/3506604833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8befa0 0x55f77b8bf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.931+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/3506604833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8bf990 0x55f77b8c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.931+0000 7f7fa3400640 1 -- 172.21.15.86:0/3506604833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8bf990 msgr2=0x55f77b8c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.931+0000 7f7fa3400640 1 --2- 172.21.15.86:0/3506604833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8bf990 0x55f77b8c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.931+0000 7f7fa3400640 1 -- 172.21.15.86:0/3506604833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8befa0 msgr2=0x55f77b8bf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.931+0000 7f7fa3400640 1 --2- 172.21.15.86:0/3506604833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8befa0 0x55f77b8bf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.931+0000 7f7fa3400640 1 -- 172.21.15.86:0/3506604833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77b7d3040 con 0x55f77b8c4b00 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.932+0000 7f7fa8a00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f880a3c30 0x7f7f880a60f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009fd0 tx=0x7f7f9c009ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.933+0000 7f7fa3400640 1 --2- 172.21.15.86:0/3506604833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c6f90 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7f900bb4a0 tx=0x7f7f900044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b7f2f33c82a51f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.933+0000 7f7f99200640 1 -- 172.21.15.86:0/3506604833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9009c080 con 0x55f77b8c4b00 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.933+0000 7f7f99200640 1 -- 172.21.15.86:0/3506604833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f90004c50 con 0x55f77b8c4b00 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.933+0000 7f7f99200640 1 -- 172.21.15.86:0/3506604833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9009d800 con 0x55f77b8c4b00 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.934+0000 7f7fad79f740 1 -- 172.21.15.86:0/3506604833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 msgr2=0x55f77b8c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.934+0000 7f7fad79f740 1 --2- 172.21.15.86:0/3506604833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c6f90 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7f900bb4a0 tx=0x7f7f900044e0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.935+0000 7f7fad79f740 1 -- 172.21.15.86:0/3506604833 shutdown_connections 2024-04-17T07:19:42.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.935+0000 7f7fad79f740 1 --2- 172.21.15.86:0/3506604833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c6f90 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.935+0000 7f7fad79f740 1 --2- 172.21.15.86:0/3506604833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8bf990 0x55f77b8c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.935+0000 7f7fad79f740 1 --2- 172.21.15.86:0/3506604833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8befa0 0x55f77b8bf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.935+0000 7f7fad79f740 1 -- 172.21.15.86:0/3506604833 >> 172.21.15.86:0/3506604833 conn(0x55f77b8ba510 msgr2=0x55f77b8ba910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.935+0000 7f7fad79f740 1 -- 172.21.15.86:0/3506604833 shutdown_connections 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.935+0000 7f7fad79f740 1 -- 172.21.15.86:0/3506604833 wait complete. 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.936+0000 7f7fad79f740 1 -- start start 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.936+0000 7f7fad79f740 1 Processor -- start 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.937+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.937+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.937+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8c4b00 0x55f77b8d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.937+0000 7f7fad79f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f77b8c7870 con 0x55f77b8befa0 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.937+0000 7f7fad79f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f77b7de830 con 0x55f77b8bf990 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.937+0000 7f7fad79f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f77b8c79f0 con 0x55f77b8c4b00 2024-04-17T07:19:42.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.937+0000 7f7fad79f740 10 client.0 _ll_register_callbacks cb 0x55f77b7acfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.938+0000 7f7fa3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.938+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.938+0000 7f7fa8a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8c4b00 0x55f77b8d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.938+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50902/0 (socket says 172.21.15.86:50902) 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.938+0000 7f7fa3e00640 1 -- 172.21.15.86:0/2396578334 learned_addr learned my addr 172.21.15.86:0/2396578334 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.938+0000 7f7fa3e00640 1 -- 172.21.15.86:0/2396578334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8c4b00 msgr2=0x55f77b8d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.938+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8c4b00 0x55f77b8d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.938+0000 7f7fa3e00640 1 -- 172.21.15.86:0/2396578334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 msgr2=0x55f77b8c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.938+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.938+0000 7f7fa3e00640 1 -- 172.21.15.86:0/2396578334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77b7e2920 con 0x55f77b8befa0 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.939+0000 7f7fa3400640 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.939+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8c9230 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400b9d0 tx=0x7f7f9400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e38cc842d54e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.939+0000 7f7f8d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:42.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.939+0000 7f7f8d600640 1 -- 172.21.15.86:0/2396578334 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94011440 con 0x55f77b8befa0 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.939+0000 7f7f8d600640 1 -- 172.21.15.86:0/2396578334 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7f940115e0 con 0x55f77b8befa0 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.940+0000 7f7fad79f740 1 -- 172.21.15.86:0/2396578334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f90004c50 con 0x55f77b8befa0 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.941+0000 7f7f8d600640 1 -- 172.21.15.86:0/2396578334 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f940117e0 con 0x55f77b8befa0 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.941+0000 7f7fad79f740 10 client.16947 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.941+0000 7f7fad79f740 1 -- 172.21.15.86:0/2396578334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f77b7b07f0 con 0x55f77b8befa0 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.942+0000 7f7f8d600640 1 -- 172.21.15.86:0/2396578334 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f90004c50 con 0x55f77b8befa0 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.942+0000 7f7f8d600640 1 -- 172.21.15.86:0/2396578334 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7f94010b20 con 0x55f77b8befa0 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.942+0000 7f7fad79f740 10 client.16947 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.942+0000 7f7fad79f740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.942+0000 7f7fad79f740 20 client.16947 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.942+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f77b8d4e00 0x55f77b8d7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.942+0000 7f7fad79f740 4 client.16947 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.942+0000 7f7fad79f740 1 -- 172.21.15.86:0/2396578334 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f77b832a40 con 0x55f77b8d4e00 2024-04-17T07:19:42.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.943+0000 7f7fa3400640 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f77b8d4e00 0x55f77b8d7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.944+0000 7f7fa3400640 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f77b8d4e00 0x55f77b8d7280 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd58aec6dd15ce0f server_cookie=36556c7deb369174 in_seq=0 out_seq=0 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.944+0000 7f7f8d600640 10 client.16947 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.945+0000 7f7f8d600640 1 -- 172.21.15.86:0/2396578334 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f77b832a40 con 0x55f77b8d4e00 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.945+0000 7f7f8d600640 10 client.16947 handle_command_reply: tid=0 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.945+0000 7f7fad79f740 1 client.16947 shutdown 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.945+0000 7f7fad79f740 1 -- 172.21.15.86:0/2396578334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 msgr2=0x55f77b8c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.946+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8c9230 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400b9d0 tx=0x7f7f9400be90 comp rx=0 tx=0).stop 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.946+0000 7f7fad79f740 1 -- 172.21.15.86:0/2396578334 shutdown_connections 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.946+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f77b8d4e00 0x55f77b8d7280 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.946+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8c4b00 0x55f77b8d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.946+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.946+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2396578334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8c9230 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.946+0000 7f7fad79f740 1 -- 172.21.15.86:0/2396578334 >> 172.21.15.86:0/2396578334 conn(0x55f77b8ba510 msgr2=0x55f77b8b9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.946+0000 7f7f8d600640 0 client.16947 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:42.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.946+0000 7f7fad79f740 1 -- 172.21.15.86:0/2396578334 shutdown_connections 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.946+0000 7f7fad79f740 1 -- 172.21.15.86:0/2396578334 wait complete. 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.947+0000 7f7fad79f740 20 client.16947 trim_cache size 0 max 16384 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.972+0000 7f7fad79f740 1 Processor -- start 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.972+0000 7f7fad79f740 1 -- start start 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.973+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8befa0 0x55f77b8bde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.973+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.973+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.973+0000 7f7fad79f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa41b1180 con 0x55f77b8c4b00 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.973+0000 7f7fad79f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa41b2cc0 con 0x55f77b8bf990 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.973+0000 7f7fad79f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa41b1850 con 0x55f77b8befa0 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50906/0 (socket says 172.21.15.86:50906) 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa8a00640 1 -- 172.21.15.86:0/1490512613 learned_addr learned my addr 172.21.15.86:0/1490512613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa3400640 1 --2- 172.21.15.86:0/1490512613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/1490512613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8befa0 0x55f77b8bde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa8a00640 1 -- 172.21.15.86:0/1490512613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8befa0 msgr2=0x55f77b8bde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa8a00640 1 --2- 172.21.15.86:0/1490512613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8befa0 0x55f77b8bde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa8a00640 1 -- 172.21.15.86:0/1490512613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 msgr2=0x55f77b8d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa8a00640 1 --2- 172.21.15.86:0/1490512613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa8a00640 1 -- 172.21.15.86:0/1490512613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77b5373f0 con 0x55f77b8c4b00 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.974+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/1490512613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8befa0 0x55f77b8bde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.975+0000 7f7fa8a00640 1 --2- 172.21.15.86:0/1490512613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c8850 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c012420 tx=0x7f7f9c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f150115dba24c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.975+0000 7f7f8fe00640 1 -- 172.21.15.86:0/1490512613 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c004b20 con 0x55f77b8c4b00 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.975+0000 7f7f8fe00640 1 -- 172.21.15.86:0/1490512613 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f9c004cc0 con 0x55f77b8c4b00 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.975+0000 7f7f8fe00640 1 -- 172.21.15.86:0/1490512613 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c005330 con 0x55f77b8c4b00 2024-04-17T07:19:42.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.976+0000 7f7fad79f740 1 -- 172.21.15.86:0/1490512613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 msgr2=0x55f77b8c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.976+0000 7f7fad79f740 1 --2- 172.21.15.86:0/1490512613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c8850 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c012420 tx=0x7f7f9c01e000 comp rx=0 tx=0).stop 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.977+0000 7f7fad79f740 1 -- 172.21.15.86:0/1490512613 shutdown_connections 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.977+0000 7f7fad79f740 1 --2- 172.21.15.86:0/1490512613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8c4b00 0x55f77b8c8850 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.977+0000 7f7fad79f740 1 --2- 172.21.15.86:0/1490512613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.977+0000 7f7fad79f740 1 --2- 172.21.15.86:0/1490512613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8befa0 0x55f77b8bde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.977+0000 7f7fad79f740 1 -- 172.21.15.86:0/1490512613 >> 172.21.15.86:0/1490512613 conn(0x55f77b8ba510 msgr2=0x55f77b8bd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.977+0000 7f7fad79f740 1 -- 172.21.15.86:0/1490512613 shutdown_connections 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.977+0000 7f7fad79f740 1 -- 172.21.15.86:0/1490512613 wait complete. 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.978+0000 7f7fad79f740 1 -- start start 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.978+0000 7f7fad79f740 1 Processor -- start 2024-04-17T07:19:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.979+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.979+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.979+0000 7f7fad79f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8c4b00 0x55f77b9393b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.979+0000 7f7fad79f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa41b25f0 con 0x55f77b8befa0 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.979+0000 7f7fad79f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa41b1f20 con 0x55f77b8bf990 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.979+0000 7f7fad79f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa41af150 con 0x55f77b8c4b00 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.979+0000 7f7fad79f740 10 client.0 _ll_register_callbacks cb 0x7f7f940043a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54174/0 (socket says 172.21.15.86:54174) 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa3400640 1 -- 172.21.15.86:0/2341764841 learned_addr learned my addr 172.21.15.86:0/2341764841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa8a00640 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8c4b00 0x55f77b9393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa3400640 1 -- 172.21.15.86:0/2341764841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8c4b00 msgr2=0x55f77b9393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa3400640 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8c4b00 0x55f77b9393b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa3400640 1 -- 172.21.15.86:0/2341764841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 msgr2=0x55f77b8cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa3400640 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa3400640 1 -- 172.21.15.86:0/2341764841 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77b6017d0 con 0x55f77b8bf990 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.980+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.981+0000 7f7fa3400640 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8cdd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7f900f0540 tx=0x7f7f90009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7adbef258ef5179b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.981+0000 7f7f8d600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.981+0000 7f7f8d600640 1 -- 172.21.15.86:0/2341764841 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9009de10 con 0x55f77b8bf990 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.982+0000 7f7f8d600640 1 -- 172.21.15.86:0/2341764841 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7f900bcdd0 con 0x55f77b8bf990 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.982+0000 7f7fad79f740 1 -- 172.21.15.86:0/2341764841 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f9c004cc0 con 0x55f77b8bf990 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.982+0000 7f7f8d600640 1 -- 172.21.15.86:0/2341764841 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f900a6050 con 0x55f77b8bf990 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.982+0000 7f7fad79f740 10 client.35945 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.982+0000 7f7fad79f740 1 -- 172.21.15.86:0/2341764841 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f77b93a970 con 0x55f77b8bf990 2024-04-17T07:19:42.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.983+0000 7f7f8d600640 1 -- 172.21.15.86:0/2341764841 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f90004c50 con 0x55f77b8bf990 2024-04-17T07:19:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.984+0000 7f7f8d600640 1 -- 172.21.15.86:0/2341764841 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7f900da020 con 0x55f77b8bf990 2024-04-17T07:19:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.984+0000 7f7fad79f740 10 client.35945 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.984+0000 7f7fad79f740 10 client.35945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.984+0000 7f7fad79f740 20 client.35945 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.984+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f77b93b200 0x55f77b93d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.984+0000 7f7fad79f740 4 client.35945 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:19:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.984+0000 7f7fad79f740 1 -- 172.21.15.86:0/2341764841 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7f9c004cc0 con 0x55f77b93b200 2024-04-17T07:19:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.985+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f77b93b200 0x55f77b93d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.985+0000 7f7fa3e00640 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f77b93b200 0x55f77b93d620 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53137200f731ce4 server_cookie=cb255480530b4ddd in_seq=0 out_seq=0 2024-04-17T07:19:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.986+0000 7f7f8d600640 10 client.35945 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.986+0000 7f7f8d600640 1 -- 172.21.15.86:0/2341764841 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7f9c004cc0 con 0x55f77b93b200 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.986+0000 7f7f8d600640 10 client.35945 handle_command_reply: tid=0 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.986+0000 7f7fad79f740 1 client.35945 shutdown 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7fad79f740 1 -- 172.21.15.86:0/2341764841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 msgr2=0x55f77b8cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8cdd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7f900f0540 tx=0x7f7f90009260 comp rx=0 tx=0).stop 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7fad79f740 1 -- 172.21.15.86:0/2341764841 shutdown_connections 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f77b93b200 0x55f77b93d620 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f77b8c4b00 0x55f77b9393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77b8bf990 0x55f77b8cdd20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7fad79f740 1 --2- 172.21.15.86:0/2341764841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77b8befa0 0x55f77b8cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7fad79f740 1 -- 172.21.15.86:0/2341764841 >> 172.21.15.86:0/2341764841 conn(0x55f77b8ba510 msgr2=0x55f77b8be000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7f8d600640 0 client.35945 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7fad79f740 1 -- 172.21.15.86:0/2341764841 shutdown_connections 2024-04-17T07:19:42.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.987+0000 7f7fad79f740 1 -- 172.21.15.86:0/2341764841 wait complete. 2024-04-17T07:19:42.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.988+0000 7f7fad79f740 20 client.35945 trim_cache size 0 max 16384 2024-04-17T07:19:42.999 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.989+0000 7f7faaa00640 1 -- 172.21.15.86:0/4169464946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f880a3c30 msgr2=0x7f7f880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.989+0000 7f7faaa00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f880a3c30 0x7f7f880a60f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009fd0 tx=0x7f7f9c009ec0 comp rx=0 tx=0).stop 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.989+0000 7f7faaa00640 1 -- 172.21.15.86:0/4169464946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa41053d0 msgr2=0x7f7fa419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.989+0000 7f7faaa00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa41053d0 0x7f7fa419ed40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f90000c00 tx=0x7f7f900042f0 comp rx=0 tx=0).stop 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.990+0000 7f7faaa00640 1 -- 172.21.15.86:0/4169464946 shutdown_connections 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.990+0000 7f7faaa00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f880a3c30 0x7f7f880a60f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.990+0000 7f7faaa00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa4105d90 0x7f7fa41a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.990+0000 7f7faaa00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa41053d0 0x7f7fa419ed40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.990+0000 7f7faaa00640 1 --2- 172.21.15.86:0/4169464946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa41041e0 0x7f7fa419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.990+0000 7f7faaa00640 1 -- 172.21.15.86:0/4169464946 >> 172.21.15.86:0/4169464946 conn(0x7f7fa40ff750 msgr2=0x7f7fa4079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:43.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.990+0000 7f7faaa00640 1 -- 172.21.15.86:0/4169464946 shutdown_connections 2024-04-17T07:19:43.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:42.990+0000 7f7faaa00640 1 -- 172.21.15.86:0/4169464946 wait complete. 2024-04-17T07:19:43.008 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:42 smithi086 ceph-mon[31733]: pgmap v600: 129 pgs: 129 active+clean; 939 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 377 op/s 2024-04-17T07:19:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:42 smithi098 ceph-mon[35418]: pgmap v600: 129 pgs: 129 active+clean; 939 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 377 op/s 2024-04-17T07:19:43.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:42 smithi103 ceph-mon[35495]: pgmap v600: 129 pgs: 129 active+clean; 939 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 377 op/s 2024-04-17T07:19:44.012 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:19:44.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:19:44.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:19:44.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.195+0000 7fdd53200640 1 -- 172.21.15.86:0/2163532441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd54100d70 msgr2=0x7fdd54101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.195+0000 7fdd53200640 1 --2- 172.21.15.86:0/2163532441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd54100d70 0x7fdd54101190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009900 tx=0x7fdd4402bc30 comp rx=0 tx=0).stop 2024-04-17T07:19:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.195+0000 7fdd53200640 1 -- 172.21.15.86:0/2163532441 shutdown_connections 2024-04-17T07:19:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.195+0000 7fdd53200640 1 --2- 172.21.15.86:0/2163532441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd54102920 0x7fdd541091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.195+0000 7fdd53200640 1 --2- 172.21.15.86:0/2163532441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd54101f60 0x7fdd541023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.195+0000 7fdd53200640 1 --2- 172.21.15.86:0/2163532441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd54100d70 0x7fdd54101190 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.195+0000 7fdd53200640 1 -- 172.21.15.86:0/2163532441 >> 172.21.15.86:0/2163532441 conn(0x7fdd540fc300 msgr2=0x7fdd540fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.195+0000 7fdd53200640 1 -- 172.21.15.86:0/2163532441 shutdown_connections 2024-04-17T07:19:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.195+0000 7fdd53200640 1 -- 172.21.15.86:0/2163532441 wait complete. 2024-04-17T07:19:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.196+0000 7fdd53200640 1 Processor -- start 2024-04-17T07:19:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.196+0000 7fdd53200640 1 -- start start 2024-04-17T07:19:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd53200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd54100d70 0x7fdd5419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd53200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd54101f60 0x7fdd5419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd53200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd54102920 0x7fdd541a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd53200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5410d9e0 con 0x7fdd54100d70 2024-04-17T07:19:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd53200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5410d860 con 0x7fdd54101f60 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd53200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5410db60 con 0x7fdd54102920 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd54100d70 0x7fdd5419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd54100d70 0x7fdd5419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50934/0 (socket says 172.21.15.86:50934) 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd51e00640 1 -- 172.21.15.86:0/3495200983 learned_addr learned my addr 172.21.15.86:0/3495200983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd51e00640 1 -- 172.21.15.86:0/3495200983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd54102920 msgr2=0x7fdd541a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd51e00640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd54102920 0x7fdd541a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd51e00640 1 -- 172.21.15.86:0/3495200983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd54101f60 msgr2=0x7fdd5419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd51400640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd54101f60 0x7fdd5419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd51e00640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd54101f60 0x7fdd5419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.197+0000 7fdd51e00640 1 -- 172.21.15.86:0/3495200983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd541a1f50 con 0x7fdd54100d70 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.198+0000 7fdd51e00640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd54100d70 0x7fdd5419a330 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdd44000c00 tx=0x7fdd440040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.198+0000 7fdd4aa00640 1 -- 172.21.15.86:0/3495200983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44043020 con 0x7fdd54100d70 2024-04-17T07:19:44.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.198+0000 7fdd53200640 1 -- 172.21.15.86:0/3495200983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd541a21e0 con 0x7fdd54100d70 2024-04-17T07:19:44.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.198+0000 7fdd4aa00640 1 -- 172.21.15.86:0/3495200983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd440044c0 con 0x7fdd54100d70 2024-04-17T07:19:44.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.198+0000 7fdd4aa00640 1 -- 172.21.15.86:0/3495200983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44003c90 con 0x7fdd54100d70 2024-04-17T07:19:44.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.198+0000 7fdd53200640 1 -- 172.21.15.86:0/3495200983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd541a2780 con 0x7fdd54100d70 2024-04-17T07:19:44.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.201+0000 7fdd4aa00640 1 -- 172.21.15.86:0/3495200983 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fdd44046020 con 0x7fdd54100d70 2024-04-17T07:19:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.202+0000 7fdd5b823740 1 Processor -- start 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd5b823740 1 -- start start 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd4aa00640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd380a3c30 0x7fdd380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd4aa00640 1 -- 172.21.15.86:0/3495200983 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdd440f2a80 con 0x7fdd54100d70 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 0x56021220c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56021220ca90 0x560212211590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x560212214090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd52800640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd380a3c30 0x7fdd380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd5b823740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560212171cb0 con 0x560212211c00 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd5b823740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560212118370 con 0x56021220ca90 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd5b823740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56021216dc70 con 0x56021220c0a0 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.203+0000 7fdd52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x560212214090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x560212214090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50942/0 (socket says 172.21.15.86:50942) 2024-04-17T07:19:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd52800640 1 -- 172.21.15.86:0/2389599583 learned_addr learned my addr 172.21.15.86:0/2389599583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd51400640 1 --2- 172.21.15.86:0/2389599583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56021220ca90 0x560212211590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd51e00640 1 --2- 172.21.15.86:0/2389599583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 0x56021220c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd52800640 1 -- 172.21.15.86:0/2389599583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 msgr2=0x56021220c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd52800640 1 --2- 172.21.15.86:0/2389599583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 0x56021220c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd52800640 1 -- 172.21.15.86:0/2389599583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56021220ca90 msgr2=0x560212211590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd52800640 1 --2- 172.21.15.86:0/2389599583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56021220ca90 0x560212211590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd52800640 1 -- 172.21.15.86:0/2389599583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560212137910 con 0x560212211c00 2024-04-17T07:19:44.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd52800640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd380a3c30 0x7fdd380a60f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c015b10 tx=0x7fdd4c009cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd52800640 1 --2- 172.21.15.86:0/2389599583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x560212214090 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005c40 tx=0x7fdd4c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba41193026b7e5c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.204+0000 7fdd3f000640 1 -- 172.21.15.86:0/2389599583 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c013dd0 con 0x560212211c00 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd3f000640 1 -- 172.21.15.86:0/2389599583 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd4c0106e0 con 0x560212211c00 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd3f000640 1 -- 172.21.15.86:0/2389599583 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c005020 con 0x560212211c00 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd5b823740 1 -- 172.21.15.86:0/2389599583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 msgr2=0x560212214090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd5b823740 1 --2- 172.21.15.86:0/2389599583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x560212214090 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005c40 tx=0x7fdd4c01a000 comp rx=0 tx=0).stop 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd5b823740 1 -- 172.21.15.86:0/2389599583 shutdown_connections 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd5b823740 1 --2- 172.21.15.86:0/2389599583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x560212214090 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd5b823740 1 --2- 172.21.15.86:0/2389599583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56021220ca90 0x560212211590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd5b823740 1 --2- 172.21.15.86:0/2389599583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 0x56021220c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd5b823740 1 -- 172.21.15.86:0/2389599583 >> 172.21.15.86:0/2389599583 conn(0x560212207610 msgr2=0x560212207a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd5b823740 1 -- 172.21.15.86:0/2389599583 shutdown_connections 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.205+0000 7fdd5b823740 1 -- 172.21.15.86:0/2389599583 wait complete. 2024-04-17T07:19:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.206+0000 7fdd5b823740 1 -- start start 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.206+0000 7fdd5b823740 1 Processor -- start 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.206+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220c0a0 0x560212216600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.206+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220ca90 0x560212216b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.206+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 0x56021221db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.206+0000 7fdd5b823740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560212214b50 con 0x56021220c0a0 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.206+0000 7fdd5b823740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56021212b730 con 0x560212211c00 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.206+0000 7fdd5b823740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560212214cd0 con 0x56021220ca90 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.206+0000 7fdd5b823740 10 client.0 _ll_register_callbacks cb 0x56021212db10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220c0a0 0x560212216600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd51e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220c0a0 0x560212216600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50950/0 (socket says 172.21.15.86:50950) 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd51e00640 1 -- 172.21.15.86:0/3539711810 learned_addr learned my addr 172.21.15.86:0/3539711810 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd52800640 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 0x56021221db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd51e00640 1 -- 172.21.15.86:0/3539711810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220ca90 msgr2=0x560212216b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd51e00640 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220ca90 0x560212216b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd51e00640 1 -- 172.21.15.86:0/3539711810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 msgr2=0x56021221db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd51e00640 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 0x56021221db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd51e00640 1 -- 172.21.15.86:0/3539711810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560211f4c7e0 con 0x56021220c0a0 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd51e00640 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220c0a0 0x560212216600 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fdd44006010 tx=0x7fdd440be3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79849f66fde98daa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.207+0000 7fdd33400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd33400640 1 -- 172.21.15.86:0/3539711810 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd440bf080 con 0x56021220c0a0 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd33400640 1 -- 172.21.15.86:0/3539711810 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdd440c7160 con 0x56021220c0a0 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd5b823740 1 -- 172.21.15.86:0/3539711810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd4c0106e0 con 0x56021220c0a0 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd33400640 1 -- 172.21.15.86:0/3539711810 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd440be980 con 0x56021220c0a0 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd5b823740 10 client.16962 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd5b823740 1 -- 172.21.15.86:0/3539711810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560211facf70 con 0x56021220c0a0 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd33400640 1 -- 172.21.15.86:0/3539711810 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd4c0106e0 con 0x56021220c0a0 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd33400640 1 -- 172.21.15.86:0/3539711810 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fdd440ddb40 con 0x56021220c0a0 2024-04-17T07:19:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd5b823740 10 client.16962 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd5b823740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd5b823740 20 client.16962 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd5b823740 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560212222480 0x560212224940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd5b823740 4 client.16962 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.208+0000 7fdd5b823740 1 -- 172.21.15.86:0/3539711810 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56021217f6d0 con 0x560212222480 2024-04-17T07:19:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.209+0000 7fdd51400640 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560212222480 0x560212224940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.210+0000 7fdd51400640 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560212222480 0x560212224940 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8955c39a961d5d4 server_cookie=2c0081d1fa94ad23 in_seq=0 out_seq=0 2024-04-17T07:19:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.210+0000 7fdd33400640 10 client.16962 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd33400640 1 -- 172.21.15.86:0/3539711810 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56021217f6d0 con 0x560212222480 2024-04-17T07:19:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd33400640 10 client.16962 handle_command_reply: tid=0 2024-04-17T07:19:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 client.16962 shutdown 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 -- 172.21.15.86:0/3539711810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220c0a0 msgr2=0x560212216600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220c0a0 0x560212216600 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fdd44006010 tx=0x7fdd440be3a0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 -- 172.21.15.86:0/3539711810 shutdown_connections 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560212222480 0x560212224940 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 0x56021221db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd33400640 0 client.16962 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220ca90 0x560212216b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 --2- 172.21.15.86:0/3539711810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220c0a0 0x560212216600 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 -- 172.21.15.86:0/3539711810 >> 172.21.15.86:0/3539711810 conn(0x560212207610 msgr2=0x560212206990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 -- 172.21.15.86:0/3539711810 shutdown_connections 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.212+0000 7fdd5b823740 1 -- 172.21.15.86:0/3539711810 wait complete. 2024-04-17T07:19:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.213+0000 7fdd5b823740 20 client.16962 trim_cache size 0 max 16384 2024-04-17T07:19:44.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.231+0000 7fdd5b823740 1 Processor -- start 2024-04-17T07:19:44.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.231+0000 7fdd5b823740 1 -- start start 2024-04-17T07:19:44.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.231+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56021220c0a0 0x560212216aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.231+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220ca90 0x56021221db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.231+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x56021220af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.231+0000 7fdd5b823740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541acb50 con 0x560212211c00 2024-04-17T07:19:44.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.231+0000 7fdd5b823740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ae690 con 0x56021220c0a0 2024-04-17T07:19:44.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.231+0000 7fdd5b823740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ad220 con 0x56021220ca90 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.231+0000 7fdd52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x56021220af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.232+0000 7fdd52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x56021220af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50962/0 (socket says 172.21.15.86:50962) 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.232+0000 7fdd52800640 1 -- 172.21.15.86:0/3263227187 learned_addr learned my addr 172.21.15.86:0/3263227187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.232+0000 7fdd51e00640 1 --2- 172.21.15.86:0/3263227187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56021220c0a0 0x560212216aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.232+0000 7fdd51400640 1 --2- 172.21.15.86:0/3263227187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220ca90 0x56021221db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.232+0000 7fdd52800640 1 -- 172.21.15.86:0/3263227187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220ca90 msgr2=0x56021221db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.232+0000 7fdd52800640 1 --2- 172.21.15.86:0/3263227187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220ca90 0x56021221db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.232+0000 7fdd52800640 1 -- 172.21.15.86:0/3263227187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56021220c0a0 msgr2=0x560212216aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.232+0000 7fdd52800640 1 --2- 172.21.15.86:0/3263227187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56021220c0a0 0x560212216aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.232+0000 7fdd52800640 1 -- 172.21.15.86:0/3263227187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560211e44170 con 0x560212211c00 2024-04-17T07:19:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.232+0000 7fdd52800640 1 --2- 172.21.15.86:0/3263227187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x56021220af80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009f70 tx=0x7fdd4c009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30559ced480d9a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.233+0000 7fdd3e600640 1 -- 172.21.15.86:0/3263227187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c02e020 con 0x560212211c00 2024-04-17T07:19:44.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.233+0000 7fdd3e600640 1 -- 172.21.15.86:0/3263227187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd4c013600 con 0x560212211c00 2024-04-17T07:19:44.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.233+0000 7fdd3e600640 1 -- 172.21.15.86:0/3263227187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c005020 con 0x560212211c00 2024-04-17T07:19:44.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.233+0000 7fdd5b823740 1 -- 172.21.15.86:0/3263227187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 msgr2=0x56021220af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.233+0000 7fdd5b823740 1 --2- 172.21.15.86:0/3263227187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x56021220af80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009f70 tx=0x7fdd4c009670 comp rx=0 tx=0).stop 2024-04-17T07:19:44.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.233+0000 7fdd5b823740 1 -- 172.21.15.86:0/3263227187 shutdown_connections 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.233+0000 7fdd5b823740 1 --2- 172.21.15.86:0/3263227187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560212211c00 0x56021220af80 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.233+0000 7fdd5b823740 1 --2- 172.21.15.86:0/3263227187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220ca90 0x56021221db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.233+0000 7fdd5b823740 1 --2- 172.21.15.86:0/3263227187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56021220c0a0 0x560212216aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.233+0000 7fdd5b823740 1 -- 172.21.15.86:0/3263227187 >> 172.21.15.86:0/3263227187 conn(0x560212207610 msgr2=0x56021220a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.234+0000 7fdd5b823740 1 -- 172.21.15.86:0/3263227187 shutdown_connections 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.234+0000 7fdd5b823740 1 -- 172.21.15.86:0/3263227187 wait complete. 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.234+0000 7fdd5b823740 1 -- start start 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.234+0000 7fdd5b823740 1 Processor -- start 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 0x56021221ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220ca90 0x56021221bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd5b823740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 0x56021221cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd5b823740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541adfc0 con 0x56021220ca90 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd5b823740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ad8f0 con 0x560212211c00 2024-04-17T07:19:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd5b823740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541aab20 con 0x56021220c0a0 2024-04-17T07:19:44.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd5b823740 10 client.0 _ll_register_callbacks cb 0x7fdd44005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:44.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd51400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220ca90 0x56021221bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd51e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 0x56021221ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd51e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 0x56021221ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41960/0 (socket says 172.21.15.86:41960) 2024-04-17T07:19:44.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.235+0000 7fdd51e00640 1 -- 172.21.15.86:0/4164060547 learned_addr learned my addr 172.21.15.86:0/4164060547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:44.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.236+0000 7fdd52800640 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 0x56021221cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.236+0000 7fdd52800640 1 -- 172.21.15.86:0/4164060547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 msgr2=0x56021221ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.236+0000 7fdd52800640 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 0x56021221ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.236+0000 7fdd52800640 1 -- 172.21.15.86:0/4164060547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220ca90 msgr2=0x56021221bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.236+0000 7fdd52800640 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220ca90 0x56021221bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.236+0000 7fdd52800640 1 -- 172.21.15.86:0/4164060547 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56021221d570 con 0x560212211c00 2024-04-17T07:19:44.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.236+0000 7fdd51e00640 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 0x56021221ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:44.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.236+0000 7fdd51400640 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220ca90 0x56021221bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:19:44.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.237+0000 7fdd52800640 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 0x56021221cf80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c015fd0 tx=0x7fdd4c013e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d89e997d46670577 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.237+0000 7fdd33400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:19:44.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.237+0000 7fdd33400640 1 -- 172.21.15.86:0/4164060547 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c039020 con 0x560212211c00 2024-04-17T07:19:44.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.237+0000 7fdd5b823740 1 -- 172.21.15.86:0/4164060547 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd4c013600 con 0x560212211c00 2024-04-17T07:19:44.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.237+0000 7fdd33400640 1 -- 172.21.15.86:0/4164060547 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdd4c010970 con 0x560212211c00 2024-04-17T07:19:44.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.238+0000 7fdd33400640 1 -- 172.21.15.86:0/4164060547 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd4c044050 con 0x560212211c00 2024-04-17T07:19:44.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.238+0000 7fdd5b823740 10 client.35957 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:44.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.238+0000 7fdd5b823740 1 -- 172.21.15.86:0/4164060547 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560212291b30 con 0x560212211c00 2024-04-17T07:19:44.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.239+0000 7fdd33400640 1 -- 172.21.15.86:0/4164060547 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd4c013600 con 0x560212211c00 2024-04-17T07:19:44.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.239+0000 7fdd33400640 1 -- 172.21.15.86:0/4164060547 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fdd4c035460 con 0x560212211c00 2024-04-17T07:19:44.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.240+0000 7fdd5b823740 10 client.35957 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:44.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.240+0000 7fdd5b823740 10 client.35957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:44.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.240+0000 7fdd5b823740 20 client.35957 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:44.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.240+0000 7fdd5b823740 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5602122922b0 0x5602122946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.240+0000 7fdd5b823740 4 client.35957 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:19:44.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.240+0000 7fdd5b823740 1 -- 172.21.15.86:0/4164060547 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560212179540 con 0x5602122922b0 2024-04-17T07:19:44.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.241+0000 7fdd51e00640 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5602122922b0 0x5602122946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.242+0000 7fdd51e00640 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5602122922b0 0x5602122946d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4070e63466706fba server_cookie=32bc555d0a1ffb39 in_seq=0 out_seq=0 2024-04-17T07:19:44.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.242+0000 7fdd33400640 10 client.35957 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:44.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.245+0000 7fdd33400640 1 -- 172.21.15.86:0/4164060547 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560212179540 con 0x5602122922b0 2024-04-17T07:19:44.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.245+0000 7fdd33400640 10 client.35957 handle_command_reply: tid=0 2024-04-17T07:19:44.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.246+0000 7fdd5b823740 1 client.35957 shutdown 2024-04-17T07:19:44.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.246+0000 7fdd5b823740 1 -- 172.21.15.86:0/4164060547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 msgr2=0x56021221cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.246+0000 7fdd5b823740 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 0x56021221cf80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c015fd0 tx=0x7fdd4c013e60 comp rx=0 tx=0).stop 2024-04-17T07:19:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.247+0000 7fdd5b823740 1 -- 172.21.15.86:0/4164060547 shutdown_connections 2024-04-17T07:19:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.247+0000 7fdd5b823740 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5602122922b0 0x5602122946d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.247+0000 7fdd5b823740 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560212211c00 0x56021221cf80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.247+0000 7fdd5b823740 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56021220ca90 0x56021221bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.247+0000 7fdd5b823740 1 --2- 172.21.15.86:0/4164060547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56021220c0a0 0x56021221ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.248+0000 7fdd5b823740 1 -- 172.21.15.86:0/4164060547 >> 172.21.15.86:0/4164060547 conn(0x560212207610 msgr2=0x56021220b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.248+0000 7fdd33400640 0 client.35957 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.248+0000 7fdd5b823740 1 -- 172.21.15.86:0/4164060547 shutdown_connections 2024-04-17T07:19:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.248+0000 7fdd5b823740 1 -- 172.21.15.86:0/4164060547 wait complete. 2024-04-17T07:19:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.248+0000 7fdd5b823740 20 client.35957 trim_cache size 0 max 16384 2024-04-17T07:19:44.248 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:19:44.248 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:19:44.248 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "610fc5e1-3e3f-4c73-b1ce-919dd4ffdef4", 2024-04-17T07:19:44.248 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:19:44.248 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:19:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 -- 172.21.15.86:0/3495200983 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd380a3c30 msgr2=0x7fdd380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd380a3c30 0x7fdd380a60f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c015b10 tx=0x7fdd4c009cb0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 -- 172.21.15.86:0/3495200983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd54100d70 msgr2=0x7fdd5419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd54100d70 0x7fdd5419a330 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdd44000c00 tx=0x7fdd440040c0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 -- 172.21.15.86:0/3495200983 shutdown_connections 2024-04-17T07:19:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd380a3c30 0x7fdd380a60f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd54102920 0x7fdd541a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd54101f60 0x7fdd5419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 --2- 172.21.15.86:0/3495200983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd54100d70 0x7fdd5419a330 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 -- 172.21.15.86:0/3495200983 >> 172.21.15.86:0/3495200983 conn(0x7fdd540fc300 msgr2=0x7fdd54109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 -- 172.21.15.86:0/3495200983 shutdown_connections 2024-04-17T07:19:44.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.249+0000 7fdd53200640 1 -- 172.21.15.86:0/3495200983 wait complete. 2024-04-17T07:19:44.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:19:44.269 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:19:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 20.0 s, 571.4 tps, lat 159.848 ms stddev 38.274 2024-04-17T07:19:44.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.500+0000 7f3f9ac00640 1 -- 172.21.15.86:0/3170154048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f94105290 msgr2=0x7f3f94105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.500+0000 7f3f9ac00640 1 --2- 172.21.15.86:0/3170154048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f94105290 0x7f3f94105710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009900 tx=0x7f3f8802bc60 comp rx=0 tx=0).stop 2024-04-17T07:19:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.500+0000 7f3f9ac00640 1 -- 172.21.15.86:0/3170154048 shutdown_connections 2024-04-17T07:19:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.500+0000 7f3f9ac00640 1 --2- 172.21.15.86:0/3170154048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f94105c50 0x7f3f9410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.500+0000 7f3f9ac00640 1 --2- 172.21.15.86:0/3170154048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f94105290 0x7f3f94105710 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.500+0000 7f3f9ac00640 1 --2- 172.21.15.86:0/3170154048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f941040a0 0x7f3f941044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.500+0000 7f3f9ac00640 1 -- 172.21.15.86:0/3170154048 >> 172.21.15.86:0/3170154048 conn(0x7f3f940ff610 msgr2=0x7f3f94101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.500+0000 7f3f9ac00640 1 -- 172.21.15.86:0/3170154048 shutdown_connections 2024-04-17T07:19:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.500+0000 7f3f9ac00640 1 -- 172.21.15.86:0/3170154048 wait complete. 2024-04-17T07:19:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.502+0000 7f3f9ac00640 1 Processor -- start 2024-04-17T07:19:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.502+0000 7f3f9ac00640 1 -- start start 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.502+0000 7f3f9ac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f941040a0 0x7f3f9419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.502+0000 7f3f9ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f94105290 0x7f3f9419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.502+0000 7f3f9ac00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f94105c50 0x7f3f941a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f9ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94077450 con 0x7f3f94105290 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f9ac00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f940772d0 con 0x7f3f94105c50 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f9ac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f940775d0 con 0x7f3f941040a0 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f98e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f94105290 0x7f3f9419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f98e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f94105290 0x7f3f9419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50996/0 (socket says 172.21.15.86:50996) 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f98e00640 1 -- 172.21.15.86:0/2820283625 learned_addr learned my addr 172.21.15.86:0/2820283625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f99800640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f941040a0 0x7f3f9419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f9a200640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f94105c50 0x7f3f941a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f98e00640 1 -- 172.21.15.86:0/2820283625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f941040a0 msgr2=0x7f3f9419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f98e00640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f941040a0 0x7f3f9419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f98e00640 1 -- 172.21.15.86:0/2820283625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f94105c50 msgr2=0x7f3f941a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f98e00640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f94105c50 0x7f3f941a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.503+0000 7f3f98e00640 1 -- 172.21.15.86:0/2820283625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f941a6130 con 0x7f3f94105290 2024-04-17T07:19:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.504+0000 7f3f99800640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f941040a0 0x7f3f9419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:19:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.504+0000 7f3f98e00640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f94105290 0x7f3f9419ea50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3f8802c1a0 tx=0x7f3f8802ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.505+0000 7f3f92000640 1 -- 172.21.15.86:0/2820283625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88041020 con 0x7f3f94105290 2024-04-17T07:19:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.506+0000 7f3f92000640 1 -- 172.21.15.86:0/2820283625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f8802e2e0 con 0x7f3f94105290 2024-04-17T07:19:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.506+0000 7f3f92000640 1 -- 172.21.15.86:0/2820283625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88034b50 con 0x7f3f94105290 2024-04-17T07:19:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.506+0000 7f3f9ac00640 1 -- 172.21.15.86:0/2820283625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f941a63c0 con 0x7f3f94105290 2024-04-17T07:19:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.506+0000 7f3f9ac00640 1 -- 172.21.15.86:0/2820283625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f941a68e0 con 0x7f3f94105290 2024-04-17T07:19:44.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.511+0000 7f3f92000640 1 -- 172.21.15.86:0/2820283625 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3f88044020 con 0x7f3f94105290 2024-04-17T07:19:44.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.516+0000 7f3f92000640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3f7c0a3d80 0x7f3f7c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.517+0000 7f3f99800640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3f7c0a3d80 0x7f3f7c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.517+0000 7f3f9efe6740 1 Processor -- start 2024-04-17T07:19:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.518+0000 7f3f9efe6740 1 -- start start 2024-04-17T07:19:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.518+0000 7f3f92000640 1 -- 172.21.15.86:0/2820283625 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3f880f33e0 con 0x7f3f94105290 2024-04-17T07:19:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.518+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fceda790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.518+0000 7f3f99800640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3f7c0a3d80 0x7f3f7c0a6240 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3f941050f0 tx=0x7f3f80008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.518+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 0x55d7fcedf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.518+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedfed0 0x55d7fcee2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.519+0000 7f3f99800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fceda790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.519+0000 7f3f99800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fceda790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54254/0 (socket says 172.21.15.86:54254) 2024-04-17T07:19:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.519+0000 7f3f98e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 0x55d7fcedf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.519+0000 7f3f98e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 0x55d7fcedf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51000/0 (socket says 172.21.15.86:51000) 2024-04-17T07:19:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.520+0000 7f3f99800640 1 -- 172.21.15.86:0/2827043206 learned_addr learned my addr 172.21.15.86:0/2827043206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.520+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2827043206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7fce41120 con 0x55d7fcedad60 2024-04-17T07:19:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.520+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2827043206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d7fcde7340 con 0x55d7fceda370 2024-04-17T07:19:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.520+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2827043206 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7fce3d0e0 con 0x55d7fcedfed0 2024-04-17T07:19:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.521+0000 7f3f99800640 1 -- 172.21.15.86:0/2827043206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedfed0 msgr2=0x55d7fcee2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.521+0000 7f3f99800640 1 --2- 172.21.15.86:0/2827043206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedfed0 0x55d7fcee2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.521+0000 7f3f99800640 1 -- 172.21.15.86:0/2827043206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 msgr2=0x55d7fcedf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.521+0000 7f3f99800640 1 --2- 172.21.15.86:0/2827043206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 0x55d7fcedf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.521+0000 7f3f99800640 1 -- 172.21.15.86:0/2827043206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7fcc1d7d0 con 0x55d7fceda370 2024-04-17T07:19:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.522+0000 7f3f99800640 1 --2- 172.21.15.86:0/2827043206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fceda790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009fd0 tx=0x7f3f80010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b887a10e3ea1c42a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.522+0000 7f3f86a00640 1 -- 172.21.15.86:0/2827043206 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80024070 con 0x55d7fceda370 2024-04-17T07:19:44.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.523+0000 7f3f86a00640 1 -- 172.21.15.86:0/2827043206 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f8000de50 con 0x55d7fceda370 2024-04-17T07:19:44.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.523+0000 7f3f86a00640 1 -- 172.21.15.86:0/2827043206 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f800123e0 con 0x55d7fceda370 2024-04-17T07:19:44.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.524+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2827043206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 msgr2=0x55d7fceda790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.524+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/2827043206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fceda790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009fd0 tx=0x7f3f80010000 comp rx=0 tx=0).stop 2024-04-17T07:19:44.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.524+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2827043206 shutdown_connections 2024-04-17T07:19:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.524+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/2827043206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedfed0 0x55d7fcee2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.524+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/2827043206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 0x55d7fcedf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.524+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/2827043206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fceda790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.524+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2827043206 >> 172.21.15.86:0/2827043206 conn(0x55d7fced58e0 msgr2=0x55d7fced5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.524+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2827043206 shutdown_connections 2024-04-17T07:19:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.525+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2827043206 wait complete. 2024-04-17T07:19:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.525+0000 7f3f9efe6740 1 -- start start 2024-04-17T07:19:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.525+0000 7f3f9efe6740 1 Processor -- start 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.526+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fceda370 0x55d7fcee4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.526+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedad60 0x55d7fcee4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.526+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedfed0 0x55d7fceebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.526+0000 7f3f9efe6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7fcee2e80 con 0x55d7fceda370 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.526+0000 7f3f9efe6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d7fcdfa710 con 0x55d7fcedad60 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.526+0000 7f3f9efe6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d7fcee3000 con 0x55d7fcedfed0 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.526+0000 7f3f9efe6740 10 client.0 _ll_register_callbacks cb 0x55d7fcdf5af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.526+0000 7f3f98e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedad60 0x55d7fcee4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.526+0000 7f3f98e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedad60 0x55d7fcee4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54260/0 (socket says 172.21.15.86:54260) 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.526+0000 7f3f98e00640 1 -- 172.21.15.86:0/2831947100 learned_addr learned my addr 172.21.15.86:0/2831947100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.527+0000 7f3f99800640 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fceda370 0x55d7fcee4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.527+0000 7f3f98e00640 1 -- 172.21.15.86:0/2831947100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedfed0 msgr2=0x55d7fceebf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.527+0000 7f3f98e00640 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedfed0 0x55d7fceebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.527+0000 7f3f98e00640 1 -- 172.21.15.86:0/2831947100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fceda370 msgr2=0x55d7fcee4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.527+0000 7f3f98e00640 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fceda370 0x55d7fcee4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.527+0000 7f3f98e00640 1 -- 172.21.15.86:0/2831947100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7fcb533f0 con 0x55d7fcedad60 2024-04-17T07:19:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.528+0000 7f3f98e00640 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedad60 0x55d7fcee4ec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f880f0b00 tx=0x7f3f880bec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1728ee8f23e8bd64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.528+0000 7f3f7b400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:19:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.528+0000 7f3f7b400640 1 -- 172.21.15.86:0/2831947100 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880bf2e0 con 0x55d7fcedad60 2024-04-17T07:19:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.528+0000 7f3f7b400640 1 -- 172.21.15.86:0/2831947100 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3f880f30c0 con 0x55d7fcedad60 2024-04-17T07:19:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.528+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2831947100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f8000de50 con 0x55d7fcedad60 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.529+0000 7f3f7b400640 1 -- 172.21.15.86:0/2831947100 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f880c3690 con 0x55d7fcedad60 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.529+0000 7f3f9efe6740 10 client.35969 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.529+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2831947100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7fcb130f0 con 0x55d7fcedad60 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.529+0000 7f3f7b400640 1 -- 172.21.15.86:0/2831947100 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f8000de50 con 0x55d7fcedad60 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.530+0000 7f3f7b400640 1 -- 172.21.15.86:0/2831947100 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3f880be730 con 0x55d7fcedad60 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.530+0000 7f3f9efe6740 10 client.35969 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.530+0000 7f3f9efe6740 10 client.35969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.530+0000 7f3f9efe6740 20 client.35969 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.530+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7fcef0710 0x55d7fcef2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.530+0000 7f3f9efe6740 4 client.35969 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:19:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.530+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2831947100 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7fce4eb40 con 0x55d7fcef0710 2024-04-17T07:19:44.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.531+0000 7f3f99800640 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7fcef0710 0x55d7fcef2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.532+0000 7f3f99800640 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7fcef0710 0x55d7fcef2bd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad7fa9167a89866 server_cookie=f78792726d24bca3 in_seq=0 out_seq=0 2024-04-17T07:19:44.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.532+0000 7f3f7b400640 10 client.35969 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.533+0000 7f3f7b400640 1 -- 172.21.15.86:0/2831947100 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7fce4eb40 con 0x55d7fcef0710 2024-04-17T07:19:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.533+0000 7f3f7b400640 10 client.35969 handle_command_reply: tid=0 2024-04-17T07:19:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.533+0000 7f3f9efe6740 1 client.35969 shutdown 2024-04-17T07:19:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.533+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2831947100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedad60 msgr2=0x55d7fcee4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.533+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedad60 0x55d7fcee4ec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f880f0b00 tx=0x7f3f880bec40 comp rx=0 tx=0).stop 2024-04-17T07:19:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.534+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2831947100 shutdown_connections 2024-04-17T07:19:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.534+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7fcef0710 0x55d7fcef2bd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.534+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedfed0 0x55d7fceebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.534+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedad60 0x55d7fcee4ec0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.534+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/2831947100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fceda370 0x55d7fcee4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.534+0000 7f3f7b400640 0 client.35969 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.534+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2831947100 >> 172.21.15.86:0/2831947100 conn(0x55d7fced58e0 msgr2=0x55d7fced4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.534+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2831947100 shutdown_connections 2024-04-17T07:19:44.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.534+0000 7f3f9efe6740 1 -- 172.21.15.86:0/2831947100 wait complete. 2024-04-17T07:19:44.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.535+0000 7f3f9efe6740 20 client.35969 trim_cache size 0 max 16384 2024-04-17T07:19:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.570+0000 7f3f9efe6740 1 Processor -- start 2024-04-17T07:19:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.570+0000 7f3f9efe6740 1 -- start start 2024-04-17T07:19:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.570+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fceda370 0x55d7fcedb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.570+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 0x55d7fceebec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.570+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedfed0 0x55d7fced4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.570+0000 7f3f9efe6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941b0ce0 con 0x55d7fcedad60 2024-04-17T07:19:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.570+0000 7f3f9efe6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941b2820 con 0x55d7fcedfed0 2024-04-17T07:19:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.570+0000 7f3f9efe6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941b13b0 con 0x55d7fceda370 2024-04-17T07:19:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.571+0000 7f3f9a200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedfed0 0x55d7fced4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.571+0000 7f3f9a200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedfed0 0x55d7fced4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54264/0 (socket says 172.21.15.86:54264) 2024-04-17T07:19:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.571+0000 7f3f9a200640 1 -- 172.21.15.86:0/1338164580 learned_addr learned my addr 172.21.15.86:0/1338164580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.571+0000 7f3f99800640 1 --2- 172.21.15.86:0/1338164580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fceda370 0x55d7fcedb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.571+0000 7f3f98e00640 1 --2- 172.21.15.86:0/1338164580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 0x55d7fceebec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.572+0000 7f3f98e00640 1 -- 172.21.15.86:0/1338164580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fceda370 msgr2=0x55d7fcedb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.572+0000 7f3f98e00640 1 --2- 172.21.15.86:0/1338164580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fceda370 0x55d7fcedb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.572+0000 7f3f98e00640 1 -- 172.21.15.86:0/1338164580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedfed0 msgr2=0x55d7fced4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.572+0000 7f3f98e00640 1 --2- 172.21.15.86:0/1338164580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedfed0 0x55d7fced4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.572+0000 7f3f98e00640 1 -- 172.21.15.86:0/1338164580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7fced5650 con 0x55d7fcedad60 2024-04-17T07:19:44.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.574+0000 7f3f98e00640 1 --2- 172.21.15.86:0/1338164580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 0x55d7fceebec0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3f8802cee0 tx=0x7f3f880bed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188b43265b9efad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.574+0000 7f3f86000640 1 -- 172.21.15.86:0/1338164580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880f3170 con 0x55d7fcedad60 2024-04-17T07:19:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.575+0000 7f3f86000640 1 -- 172.21.15.86:0/1338164580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f880b9e50 con 0x55d7fcedad60 2024-04-17T07:19:44.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.575+0000 7f3f86000640 1 -- 172.21.15.86:0/1338164580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880bb950 con 0x55d7fcedad60 2024-04-17T07:19:44.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.576+0000 7f3f9efe6740 1 -- 172.21.15.86:0/1338164580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 msgr2=0x55d7fceebec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.576+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/1338164580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 0x55d7fceebec0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3f8802cee0 tx=0x7f3f880bed70 comp rx=0 tx=0).stop 2024-04-17T07:19:44.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.576+0000 7f3f9efe6740 1 -- 172.21.15.86:0/1338164580 shutdown_connections 2024-04-17T07:19:44.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.577+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/1338164580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fcedfed0 0x55d7fced4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.577+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/1338164580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedad60 0x55d7fceebec0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.577+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/1338164580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fceda370 0x55d7fcedb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.577+0000 7f3f9efe6740 1 -- 172.21.15.86:0/1338164580 >> 172.21.15.86:0/1338164580 conn(0x55d7fced58e0 msgr2=0x55d7fcedc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.577+0000 7f3f9efe6740 1 -- 172.21.15.86:0/1338164580 shutdown_connections 2024-04-17T07:19:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.577+0000 7f3f9efe6740 1 -- 172.21.15.86:0/1338164580 wait complete. 2024-04-17T07:19:44.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.577+0000 7f3f9efe6740 1 -- start start 2024-04-17T07:19:44.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.578+0000 7f3f9efe6740 1 Processor -- start 2024-04-17T07:19:44.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.578+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fcee9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.578+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedad60 0x55d7fcee9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.578+0000 7f3f9efe6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedfed0 0x55d7fceeab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.578+0000 7f3f9efe6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941b2150 con 0x55d7fcedfed0 2024-04-17T07:19:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.578+0000 7f3f9efe6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941b1a80 con 0x55d7fceda370 2024-04-17T07:19:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.578+0000 7f3f9efe6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941aecb0 con 0x55d7fcedad60 2024-04-17T07:19:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.578+0000 7f3f9efe6740 10 client.0 _ll_register_callbacks cb 0x7f3f880095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:19:44.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.579+0000 7f3f99800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fcee9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.579+0000 7f3f9a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedfed0 0x55d7fceeab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.579+0000 7f3f99800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fcee9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54274/0 (socket says 172.21.15.86:54274) 2024-04-17T07:19:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.579+0000 7f3f99800640 1 -- 172.21.15.86:0/3779667971 learned_addr learned my addr 172.21.15.86:0/3779667971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:19:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.580+0000 7f3f98e00640 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedad60 0x55d7fcee9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.580+0000 7f3f9a200640 1 -- 172.21.15.86:0/3779667971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedad60 msgr2=0x55d7fcee9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.580+0000 7f3f9a200640 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedad60 0x55d7fcee9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.580+0000 7f3f9a200640 1 -- 172.21.15.86:0/3779667971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 msgr2=0x55d7fcee9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:19:44.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.580+0000 7f3f9a200640 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fcee9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.580+0000 7f3f9a200640 1 -- 172.21.15.86:0/3779667971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7fceeb130 con 0x55d7fcedfed0 2024-04-17T07:19:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.582+0000 7f3f9a200640 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedfed0 0x55d7fceeab40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x55d7fcedb2f0 tx=0x7f3f8c010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ab6e4d7f00d5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:19:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.582+0000 7f3f7b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:19:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.583+0000 7f3f7b400640 1 -- 172.21.15.86:0/3779667971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00fb40 con 0x55d7fcedfed0 2024-04-17T07:19:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.583+0000 7f3f7b400640 1 -- 172.21.15.86:0/3779667971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3f8c00fce0 con 0x55d7fcedfed0 2024-04-17T07:19:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.583+0000 7f3f9efe6740 1 -- 172.21.15.86:0/3779667971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f880b9e50 con 0x55d7fcedfed0 2024-04-17T07:19:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.583+0000 7f3f7b400640 1 -- 172.21.15.86:0/3779667971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f8c01a420 con 0x55d7fcedfed0 2024-04-17T07:19:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.583+0000 7f3f9efe6740 10 client.16983 fetch_fsmap learned FSMap version 28 2024-04-17T07:19:44.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.583+0000 7f3f9efe6740 1 -- 172.21.15.86:0/3779667971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7fcf56e80 con 0x55d7fcedfed0 2024-04-17T07:19:44.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.584+0000 7f3f7b400640 1 -- 172.21.15.86:0/3779667971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f880b9e50 con 0x55d7fcedfed0 2024-04-17T07:19:44.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.584+0000 7f3f7b400640 1 -- 172.21.15.86:0/3779667971 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3f8c019560 con 0x55d7fcedfed0 2024-04-17T07:19:44.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.585+0000 7f3f9efe6740 10 client.16983 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.585+0000 7f3f9efe6740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.585+0000 7f3f9efe6740 20 client.16983 populate_metadata read hostname 'smithi086' 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.585+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7fcf57930 0x55d7fcf59d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.585+0000 7f3f9efe6740 4 client.16983 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.585+0000 7f3f9efe6740 1 -- 172.21.15.86:0/3779667971 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7fcee3a80 con 0x55d7fcf57930 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.585+0000 7f3f99800640 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7fcf57930 0x55d7fcf59d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.586+0000 7f3f99800640 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7fcf57930 0x55d7fcf59d50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3897da8f90bd5954 server_cookie=5724a7a5f263c7f8 in_seq=0 out_seq=0 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.587+0000 7f3f7b400640 10 client.16983 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.587+0000 7f3f7b400640 1 -- 172.21.15.86:0/3779667971 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55d7fcee3a80 con 0x55d7fcf57930 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.587+0000 7f3f7b400640 10 client.16983 handle_command_reply: tid=0 2024-04-17T07:19:44.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.587+0000 7f3f9efe6740 1 client.16983 shutdown 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.587+0000 7f3f9efe6740 1 -- 172.21.15.86:0/3779667971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedfed0 msgr2=0x55d7fceeab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.587+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedfed0 0x55d7fceeab40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x55d7fcedb2f0 tx=0x7f3f8c010eb0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.588+0000 7f3f9efe6740 1 -- 172.21.15.86:0/3779667971 shutdown_connections 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.588+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d7fcf57930 0x55d7fcf59d50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.588+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7fcedfed0 0x55d7fceeab40 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.588+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d7fcedad60 0x55d7fcee9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.588+0000 7f3f9efe6740 1 --2- 172.21.15.86:0/3779667971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d7fceda370 0x55d7fcee9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.588+0000 7f3f9efe6740 1 -- 172.21.15.86:0/3779667971 >> 172.21.15.86:0/3779667971 conn(0x55d7fced58e0 msgr2=0x55d7fcee48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.588+0000 7f3f7b400640 0 client.16983 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.589+0000 7f3f9efe6740 1 -- 172.21.15.86:0/3779667971 shutdown_connections 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.589+0000 7f3f9efe6740 1 -- 172.21.15.86:0/3779667971 wait complete. 2024-04-17T07:19:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.589+0000 7f3f9efe6740 20 client.16983 trim_cache size 0 max 16384 2024-04-17T07:19:44.589 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:19:44.589 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (198 inodes in the stack)", 2024-04-17T07:19:44.589 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:19:44.589 INFO:teuthology.orchestra.run.smithi086.stdout: "610fc5e1-3e3f-4c73-b1ce-919dd4ffdef4": { 2024-04-17T07:19:44.589 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:19:44.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "610fc5e1-3e3f-4c73-b1ce-919dd4ffdef4", 2024-04-17T07:19:44.589 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:19:44.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:19:44.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:19:44.590 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:19:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.590+0000 7f3f9ac00640 1 -- 172.21.15.86:0/2820283625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3f7c0a3d80 msgr2=0x7f3f7c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.590+0000 7f3f9ac00640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3f7c0a3d80 0x7f3f7c0a6240 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3f941050f0 tx=0x7f3f80008040 comp rx=0 tx=0).stop 2024-04-17T07:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.590+0000 7f3f9ac00640 1 -- 172.21.15.86:0/2820283625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f94105290 msgr2=0x7f3f9419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.590+0000 7f3f9ac00640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f94105290 0x7f3f9419ea50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3f8802c1a0 tx=0x7f3f8802ef70 comp rx=0 tx=0).stop 2024-04-17T07:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.590+0000 7f3f9ac00640 1 -- 172.21.15.86:0/2820283625 shutdown_connections 2024-04-17T07:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.590+0000 7f3f9ac00640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3f7c0a3d80 0x7f3f7c0a6240 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.590+0000 7f3f9ac00640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f94105c50 0x7f3f941a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.590+0000 7f3f9ac00640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f94105290 0x7f3f9419ea50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.590+0000 7f3f9ac00640 1 --2- 172.21.15.86:0/2820283625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f941040a0 0x7f3f9419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.590+0000 7f3f9ac00640 1 -- 172.21.15.86:0/2820283625 >> 172.21.15.86:0/2820283625 conn(0x7f3f940ff610 msgr2=0x7f3f94079e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.591+0000 7f3f9ac00640 1 -- 172.21.15.86:0/2820283625 shutdown_connections 2024-04-17T07:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:19:44.591+0000 7f3f9ac00640 1 -- 172.21.15.86:0/2820283625 wait complete. 2024-04-17T07:19:44.619 INFO:tasks.cephfs.filesystem:scrub status for tag:610fc5e1-3e3f-4c73-b1ce-919dd4ffdef4 - {'path': '/', 'tag': '610fc5e1-3e3f-4c73-b1ce-919dd4ffdef4', 'options': 'recursive,force'} 2024-04-17T07:19:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:44 smithi098 ceph-mon[35418]: pgmap v601: 129 pgs: 129 active+clean; 939 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.8 MiB/s wr, 519 op/s 2024-04-17T07:19:45.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:44 smithi103 ceph-mon[35495]: pgmap v601: 129 pgs: 129 active+clean; 939 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.8 MiB/s wr, 519 op/s 2024-04-17T07:19:45.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:44 smithi086 ceph-mon[31733]: pgmap v601: 129 pgs: 129 active+clean; 939 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.8 MiB/s wr, 519 op/s 2024-04-17T07:19:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:45 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:19:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:45 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:45 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:19:46.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:45 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:46.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:45 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:19:46.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:45 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:45 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:19:46.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:45 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:46.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:45 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:19:46.332 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:45 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:46.332 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:45 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:19:46.332 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:45 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:19:47.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:46 smithi098 ceph-mon[35418]: pgmap v602: 129 pgs: 129 active+clean; 939 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.5 MiB/s wr, 640 op/s 2024-04-17T07:19:47.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:46 smithi103 ceph-mon[35495]: pgmap v602: 129 pgs: 129 active+clean; 939 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.5 MiB/s wr, 640 op/s 2024-04-17T07:19:47.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:46 smithi086 ceph-mon[31733]: pgmap v602: 129 pgs: 129 active+clean; 939 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.5 MiB/s wr, 640 op/s 2024-04-17T07:19:49.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:48 smithi098 ceph-mon[35418]: pgmap v603: 129 pgs: 129 active+clean; 939 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.9 MiB/s wr, 808 op/s 2024-04-17T07:19:49.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:48 smithi103 ceph-mon[35495]: pgmap v603: 129 pgs: 129 active+clean; 939 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.9 MiB/s wr, 808 op/s 2024-04-17T07:19:49.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:48 smithi086 ceph-mon[31733]: pgmap v603: 129 pgs: 129 active+clean; 939 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.9 MiB/s wr, 808 op/s 2024-04-17T07:19:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 25.0 s, 691.8 tps, lat 156.809 ms stddev 88.706 2024-04-17T07:19:50.158 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:49 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:19:50.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:49 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:19:50.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:49 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:19:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:50 smithi098 ceph-mon[35418]: pgmap v604: 129 pgs: 129 active+clean; 939 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 779 op/s 2024-04-17T07:19:51.278 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:50 smithi103 ceph-mon[35495]: pgmap v604: 129 pgs: 129 active+clean; 939 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 779 op/s 2024-04-17T07:19:51.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:50 smithi086 ceph-mon[31733]: pgmap v604: 129 pgs: 129 active+clean; 939 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 779 op/s 2024-04-17T07:19:53.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:52 smithi098 ceph-mon[35418]: pgmap v605: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.3 MiB/s wr, 767 op/s 2024-04-17T07:19:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:52 smithi103 ceph-mon[35495]: pgmap v605: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.3 MiB/s wr, 767 op/s 2024-04-17T07:19:53.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:52 smithi086 ceph-mon[31733]: pgmap v605: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.3 MiB/s wr, 767 op/s 2024-04-17T07:19:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 30.0 s, 674.6 tps, lat 148.460 ms stddev 39.092 2024-04-17T07:19:55.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:54 smithi098 ceph-mon[35418]: pgmap v606: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 841 op/s 2024-04-17T07:19:55.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:54 smithi103 ceph-mon[35495]: pgmap v606: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 841 op/s 2024-04-17T07:19:55.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:54 smithi086 ceph-mon[31733]: pgmap v606: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 841 op/s 2024-04-17T07:19:57.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:56 smithi098 ceph-mon[35418]: pgmap v607: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 728 op/s 2024-04-17T07:19:57.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:56 smithi103 ceph-mon[35495]: pgmap v607: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 728 op/s 2024-04-17T07:19:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:56 smithi086 ceph-mon[31733]: pgmap v607: 129 pgs: 129 active+clean; 939 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 728 op/s 2024-04-17T07:19:59.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:19:58 smithi098 ceph-mon[35418]: pgmap v608: 129 pgs: 129 active+clean; 939 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 806 op/s 2024-04-17T07:19:59.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:19:58 smithi103 ceph-mon[35495]: pgmap v608: 129 pgs: 129 active+clean; 939 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 806 op/s 2024-04-17T07:19:59.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:19:58 smithi086 ceph-mon[31733]: pgmap v608: 129 pgs: 129 active+clean; 939 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 806 op/s 2024-04-17T07:19:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 35.0 s, 750.2 tps, lat 133.208 ms stddev 35.766 2024-04-17T07:20:01.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:00 smithi098 ceph-mon[35418]: pgmap v609: 129 pgs: 129 active+clean; 939 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 696 op/s 2024-04-17T07:20:01.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:00 smithi098 ceph-mon[35418]: overall HEALTH_OK 2024-04-17T07:20:01.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:00 smithi103 ceph-mon[35495]: pgmap v609: 129 pgs: 129 active+clean; 939 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 696 op/s 2024-04-17T07:20:01.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:00 smithi103 ceph-mon[35495]: overall HEALTH_OK 2024-04-17T07:20:01.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:00 smithi086 ceph-mon[31733]: pgmap v609: 129 pgs: 129 active+clean; 939 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 696 op/s 2024-04-17T07:20:01.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:00 smithi086 ceph-mon[31733]: overall HEALTH_OK 2024-04-17T07:20:03.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:02 smithi098 ceph-mon[35418]: pgmap v610: 129 pgs: 129 active+clean; 939 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 887 op/s 2024-04-17T07:20:03.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:02 smithi103 ceph-mon[35495]: pgmap v610: 129 pgs: 129 active+clean; 939 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 887 op/s 2024-04-17T07:20:03.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:02 smithi086 ceph-mon[31733]: pgmap v610: 129 pgs: 129 active+clean; 939 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 887 op/s 2024-04-17T07:20:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 40.0 s, 759.4 tps, lat 131.925 ms stddev 34.035 2024-04-17T07:20:05.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:04 smithi098 ceph-mon[35418]: pgmap v611: 129 pgs: 129 active+clean; 939 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.4 MiB/s wr, 885 op/s 2024-04-17T07:20:05.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:04 smithi103 ceph-mon[35495]: pgmap v611: 129 pgs: 129 active+clean; 939 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.4 MiB/s wr, 885 op/s 2024-04-17T07:20:05.332 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:04 smithi086 ceph-mon[31733]: pgmap v611: 129 pgs: 129 active+clean; 939 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.4 MiB/s wr, 885 op/s 2024-04-17T07:20:07.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:06 smithi098 ceph-mon[35418]: pgmap v612: 129 pgs: 129 active+clean; 939 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 857 op/s 2024-04-17T07:20:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:06 smithi103 ceph-mon[35495]: pgmap v612: 129 pgs: 129 active+clean; 939 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 857 op/s 2024-04-17T07:20:07.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:06 smithi086 ceph-mon[31733]: pgmap v612: 129 pgs: 129 active+clean; 939 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 MiB/s wr, 857 op/s 2024-04-17T07:20:09.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:08 smithi098 ceph-mon[35418]: pgmap v613: 129 pgs: 129 active+clean; 951 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.5 MiB/s wr, 1.00k op/s 2024-04-17T07:20:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:08 smithi103 ceph-mon[35495]: pgmap v613: 129 pgs: 129 active+clean; 951 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.5 MiB/s wr, 1.00k op/s 2024-04-17T07:20:09.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:08 smithi086 ceph-mon[31733]: pgmap v613: 129 pgs: 129 active+clean; 951 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.5 MiB/s wr, 1.00k op/s 2024-04-17T07:20:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 45.0 s, 612.8 tps, lat 153.676 ms stddev 221.588 2024-04-17T07:20:10.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:09 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:20:10.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:09 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:20:10.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:09 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:20:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:10 smithi098 ceph-mon[35418]: pgmap v614: 129 pgs: 129 active+clean; 951 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 865 op/s 2024-04-17T07:20:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:20:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:20:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:10 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:20:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:10 smithi103 ceph-mon[35495]: pgmap v614: 129 pgs: 129 active+clean; 951 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 865 op/s 2024-04-17T07:20:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:20:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:20:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:10 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:20:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:10 smithi086 ceph-mon[31733]: pgmap v614: 129 pgs: 129 active+clean; 951 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 MiB/s wr, 865 op/s 2024-04-17T07:20:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:20:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:20:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:20:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:10 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:20:12.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:11 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-17T07:20:12.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:11 smithi098 ceph-mon[35418]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:20:12.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:11 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-17T07:20:12.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:11 smithi103 ceph-mon[35495]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:20:12.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:11 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-17T07:20:12.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:11 smithi086 ceph-mon[31733]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:20:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:13 smithi098 ceph-mon[35418]: pgmap v615: 129 pgs: 129 active+clean; 955 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 942 op/s 2024-04-17T07:20:13.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:13 smithi103 ceph-mon[35495]: pgmap v615: 129 pgs: 129 active+clean; 955 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 942 op/s 2024-04-17T07:20:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:13 smithi086 ceph-mon[31733]: pgmap v615: 129 pgs: 129 active+clean; 955 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 942 op/s 2024-04-17T07:20:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 50.0 s, 611.0 tps, lat 172.465 ms stddev 80.164 2024-04-17T07:20:14.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:20:14.620 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:20:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.933+0000 7fe384600640 1 -- 172.21.15.86:0/3388177433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe37c10a170 msgr2=0x7fe37c110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.933+0000 7fe384600640 1 --2- 172.21.15.86:0/3388177433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe37c10a170 0x7fe37c110a00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe374009be0 tx=0x7fe37402bef0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.933+0000 7fe384600640 1 -- 172.21.15.86:0/3388177433 shutdown_connections 2024-04-17T07:20:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.933+0000 7fe384600640 1 --2- 172.21.15.86:0/3388177433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe37c10a170 0x7fe37c110a00 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.933+0000 7fe384600640 1 --2- 172.21.15.86:0/3388177433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c1097b0 0x7fe37c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.933+0000 7fe384600640 1 --2- 172.21.15.86:0/3388177433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe37c1085c0 0x7fe37c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.933+0000 7fe384600640 1 -- 172.21.15.86:0/3388177433 >> 172.21.15.86:0/3388177433 conn(0x7fe37c077560 msgr2=0x7fe37c0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.934+0000 7fe384600640 1 -- 172.21.15.86:0/3388177433 shutdown_connections 2024-04-17T07:20:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.934+0000 7fe384600640 1 -- 172.21.15.86:0/3388177433 wait complete. 2024-04-17T07:20:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.936+0000 7fe384600640 1 Processor -- start 2024-04-17T07:20:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.937+0000 7fe384600640 1 -- start start 2024-04-17T07:20:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.937+0000 7fe384600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c1085c0 0x7fe37c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.937+0000 7fe384600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe37c1097b0 0x7fe37c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.937+0000 7fe384600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe37c10a170 0x7fe37c1a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.937+0000 7fe384600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1140d0 con 0x7fe37c1097b0 2024-04-17T07:20:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.937+0000 7fe384600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c113f50 con 0x7fe37c1085c0 2024-04-17T07:20:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.937+0000 7fe384600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c114250 con 0x7fe37c10a170 2024-04-17T07:20:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.938+0000 7fe381c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c1085c0 0x7fe37c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.938+0000 7fe381c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c1085c0 0x7fe37c19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38898/0 (socket says 172.21.15.86:38898) 2024-04-17T07:20:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.938+0000 7fe381c00640 1 -- 172.21.15.86:0/820926472 learned_addr learned my addr 172.21.15.86:0/820926472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.938+0000 7fe381200640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe37c1097b0 0x7fe37c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.938+0000 7fe382600640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe37c10a170 0x7fe37c1a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.940+0000 7fe381200640 1 -- 172.21.15.86:0/820926472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe37c10a170 msgr2=0x7fe37c1a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.940+0000 7fe381200640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe37c10a170 0x7fe37c1a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.940+0000 7fe381200640 1 -- 172.21.15.86:0/820926472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c1085c0 msgr2=0x7fe37c19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.940+0000 7fe381200640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c1085c0 0x7fe37c19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.940+0000 7fe381200640 1 -- 172.21.15.86:0/820926472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c1a63a0 con 0x7fe37c1097b0 2024-04-17T07:20:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.941+0000 7fe381200640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe37c1097b0 0x7fe37c19ed10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe36800e9f0 tx=0x7fe36800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.942+0000 7fe37a000640 1 -- 172.21.15.86:0/820926472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36800cd80 con 0x7fe37c1097b0 2024-04-17T07:20:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.942+0000 7fe37a000640 1 -- 172.21.15.86:0/820926472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3680035f0 con 0x7fe37c1097b0 2024-04-17T07:20:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.942+0000 7fe37a000640 1 -- 172.21.15.86:0/820926472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368010670 con 0x7fe37c1097b0 2024-04-17T07:20:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.942+0000 7fe384600640 1 -- 172.21.15.86:0/820926472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c1a6690 con 0x7fe37c1097b0 2024-04-17T07:20:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.943+0000 7fe384600640 1 -- 172.21.15.86:0/820926472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a6be0 con 0x7fe37c1097b0 2024-04-17T07:20:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.948+0000 7fe387377740 1 Processor -- start 2024-04-17T07:20:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.948+0000 7fe387377740 1 -- start start 2024-04-17T07:20:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.949+0000 7fe387377740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa24d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.949+0000 7fe387377740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 0x55b6fa252680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.949+0000 7fe387377740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 0x55b6fa255180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.949+0000 7fe387377740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa1b4120 con 0x55b6fa24db80 2024-04-17T07:20:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.949+0000 7fe387377740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa15a340 con 0x55b6fa24d190 2024-04-17T07:20:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.949+0000 7fe387377740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa1b00e0 con 0x55b6fa252cf0 2024-04-17T07:20:14.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.950+0000 7fe382600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa24d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:14.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.950+0000 7fe382600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa24d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38908/0 (socket says 172.21.15.86:38908) 2024-04-17T07:20:14.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.950+0000 7fe382600640 1 -- 172.21.15.86:0/502253550 learned_addr learned my addr 172.21.15.86:0/502253550 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:14.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.951+0000 7fe382600640 1 -- 172.21.15.86:0/502253550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 msgr2=0x55b6fa255180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:20:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.951+0000 7fe382600640 1 --2- 172.21.15.86:0/502253550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 0x55b6fa255180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.951+0000 7fe382600640 1 -- 172.21.15.86:0/502253550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 msgr2=0x55b6fa252680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.951+0000 7fe381c00640 1 --2- 172.21.15.86:0/502253550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 0x55b6fa252680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.951+0000 7fe382600640 1 --2- 172.21.15.86:0/502253550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 0x55b6fa252680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.951+0000 7fe382600640 1 -- 172.21.15.86:0/502253550 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f9f907d0 con 0x55b6fa24d190 2024-04-17T07:20:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.951+0000 7fe381c00640 1 --2- 172.21.15.86:0/502253550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 0x55b6fa252680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:14.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.952+0000 7fe37a000640 1 -- 172.21.15.86:0/820926472 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fe3680037a0 con 0x7fe37c1097b0 2024-04-17T07:20:14.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.953+0000 7fe382600640 1 --2- 172.21.15.86:0/502253550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa24d5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe37402c500 tx=0x7fe3740098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=411d304dc228e84a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.953+0000 7fe36ea00640 1 -- 172.21.15.86:0/502253550 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374043030 con 0x55b6fa24d190 2024-04-17T07:20:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.954+0000 7fe36ea00640 1 -- 172.21.15.86:0/502253550 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3740334d0 con 0x55b6fa24d190 2024-04-17T07:20:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.954+0000 7fe36ea00640 1 -- 172.21.15.86:0/502253550 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37402c890 con 0x55b6fa24d190 2024-04-17T07:20:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.954+0000 7fe37a000640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3640a3db0 0x7fe3640a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.955+0000 7fe387377740 1 -- 172.21.15.86:0/502253550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 msgr2=0x55b6fa24d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.955+0000 7fe381c00640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3640a3db0 0x7fe3640a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.955+0000 7fe387377740 1 --2- 172.21.15.86:0/502253550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa24d5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe37402c500 tx=0x7fe3740098d0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.955+0000 7fe37a000640 1 -- 172.21.15.86:0/820926472 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe3680d0930 con 0x7fe37c1097b0 2024-04-17T07:20:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.956+0000 7fe387377740 1 -- 172.21.15.86:0/502253550 shutdown_connections 2024-04-17T07:20:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.956+0000 7fe387377740 1 --2- 172.21.15.86:0/502253550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 0x55b6fa255180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.956+0000 7fe387377740 1 --2- 172.21.15.86:0/502253550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 0x55b6fa252680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.956+0000 7fe387377740 1 --2- 172.21.15.86:0/502253550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa24d5b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.956+0000 7fe387377740 1 -- 172.21.15.86:0/502253550 >> 172.21.15.86:0/502253550 conn(0x55b6fa248700 msgr2=0x55b6fa248b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.957+0000 7fe381c00640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3640a3db0 0x7fe3640a6270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55b6fa24df80 tx=0x7fe370009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.958+0000 7fe387377740 1 -- 172.21.15.86:0/502253550 shutdown_connections 2024-04-17T07:20:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.958+0000 7fe387377740 1 -- 172.21.15.86:0/502253550 wait complete. 2024-04-17T07:20:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.958+0000 7fe387377740 1 -- start start 2024-04-17T07:20:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.959+0000 7fe387377740 1 Processor -- start 2024-04-17T07:20:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.960+0000 7fe387377740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa2576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.960+0000 7fe387377740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 0x55b6fa257c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.960+0000 7fe387377740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 0x55b6fa25ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.960+0000 7fe387377740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa255c40 con 0x55b6fa24db80 2024-04-17T07:20:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.960+0000 7fe387377740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa16d710 con 0x55b6fa24d190 2024-04-17T07:20:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.960+0000 7fe387377740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b6fa255dc0 con 0x55b6fa252cf0 2024-04-17T07:20:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.960+0000 7fe387377740 10 client.0 _ll_register_callbacks cb 0x55b6fa168af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.961+0000 7fe382600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 0x55b6fa25ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.961+0000 7fe382600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 0x55b6fa25ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56800/0 (socket says 172.21.15.86:56800) 2024-04-17T07:20:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.961+0000 7fe382600640 1 -- 172.21.15.86:0/1242543374 learned_addr learned my addr 172.21.15.86:0/1242543374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.961+0000 7fe381200640 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 0x55b6fa257c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.961+0000 7fe382600640 1 -- 172.21.15.86:0/1242543374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 msgr2=0x55b6fa2576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.961+0000 7fe382600640 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa2576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.961+0000 7fe382600640 1 -- 172.21.15.86:0/1242543374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 msgr2=0x55b6fa257c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.962+0000 7fe382600640 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 0x55b6fa257c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.962+0000 7fe382600640 1 -- 172.21.15.86:0/1242543374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f9ec63f0 con 0x55b6fa252cf0 2024-04-17T07:20:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.962+0000 7fe382600640 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 0x55b6fa25ec50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe374004af0 tx=0x7fe3740039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc10505075d3dafd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.962+0000 7fe363400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.963+0000 7fe363400640 1 -- 172.21.15.86:0/1242543374 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37404c070 con 0x55b6fa252cf0 2024-04-17T07:20:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.963+0000 7fe363400640 1 -- 172.21.15.86:0/1242543374 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe37402cca0 con 0x55b6fa252cf0 2024-04-17T07:20:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.963+0000 7fe387377740 1 -- 172.21.15.86:0/1242543374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3740334d0 con 0x55b6fa252cf0 2024-04-17T07:20:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.964+0000 7fe363400640 1 -- 172.21.15.86:0/1242543374 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe374056050 con 0x55b6fa252cf0 2024-04-17T07:20:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.964+0000 7fe387377740 10 client.26008 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.964+0000 7fe387377740 1 -- 172.21.15.86:0/1242543374 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6f9e860f0 con 0x55b6fa252cf0 2024-04-17T07:20:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.965+0000 7fe363400640 1 -- 172.21.15.86:0/1242543374 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3740334d0 con 0x55b6fa252cf0 2024-04-17T07:20:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.966+0000 7fe363400640 1 -- 172.21.15.86:0/1242543374 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe374003fc0 con 0x55b6fa252cf0 2024-04-17T07:20:14.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.966+0000 7fe387377740 10 client.26008 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.966+0000 7fe387377740 10 client.26008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.967+0000 7fe387377740 20 client.26008 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.967+0000 7fe387377740 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b6fa263150 0x55b6fa265580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.967+0000 7fe387377740 4 client.26008 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.967+0000 7fe387377740 1 -- 172.21.15.86:0/1242543374 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6fa1c1b40 con 0x55b6fa263150 2024-04-17T07:20:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.968+0000 7fe381c00640 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b6fa263150 0x55b6fa265580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.968+0000 7fe381c00640 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b6fa263150 0x55b6fa265580 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba45ea1a1f04076c server_cookie=19a7873a24a06261 in_seq=0 out_seq=0 2024-04-17T07:20:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.969+0000 7fe363400640 10 client.26008 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.970+0000 7fe363400640 1 -- 172.21.15.86:0/1242543374 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6fa1c1b40 con 0x55b6fa263150 2024-04-17T07:20:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.970+0000 7fe363400640 10 client.26008 handle_command_reply: tid=0 2024-04-17T07:20:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.970+0000 7fe387377740 1 client.26008 shutdown 2024-04-17T07:20:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.971+0000 7fe387377740 1 -- 172.21.15.86:0/1242543374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 msgr2=0x55b6fa25ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.971+0000 7fe387377740 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 0x55b6fa25ec50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe374004af0 tx=0x7fe3740039d0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.971+0000 7fe387377740 1 -- 172.21.15.86:0/1242543374 shutdown_connections 2024-04-17T07:20:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.971+0000 7fe387377740 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b6fa263150 0x55b6fa265580 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.971+0000 7fe387377740 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa252cf0 0x55b6fa25ec50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.972+0000 7fe363400640 0 client.26008 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.972+0000 7fe387377740 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa24db80 0x55b6fa257c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.972+0000 7fe387377740 1 --2- 172.21.15.86:0/1242543374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa2576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.972+0000 7fe387377740 1 -- 172.21.15.86:0/1242543374 >> 172.21.15.86:0/1242543374 conn(0x55b6fa248700 msgr2=0x55b6fa247890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.972+0000 7fe387377740 1 -- 172.21.15.86:0/1242543374 shutdown_connections 2024-04-17T07:20:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.972+0000 7fe387377740 1 -- 172.21.15.86:0/1242543374 wait complete. 2024-04-17T07:20:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:14.973+0000 7fe387377740 20 client.26008 trim_cache size 0 max 16384 2024-04-17T07:20:15.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.006+0000 7fe387377740 1 Processor -- start 2024-04-17T07:20:15.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.006+0000 7fe387377740 1 -- start start 2024-04-17T07:20:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.007+0000 7fe387377740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa247710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.007+0000 7fe387377740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24db80 0x55b6fa25ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.007+0000 7fe387377740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 0x55b6fa269e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.007+0000 7fe387377740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1b1010 con 0x55b6fa252cf0 2024-04-17T07:20:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.007+0000 7fe387377740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1b2b50 con 0x55b6fa24d190 2024-04-17T07:20:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.007+0000 7fe387377740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1b16e0 con 0x55b6fa24db80 2024-04-17T07:20:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.007+0000 7fe382600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 0x55b6fa269e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.007+0000 7fe381200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24db80 0x55b6fa25ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.007+0000 7fe381200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24db80 0x55b6fa25ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56802/0 (socket says 172.21.15.86:56802) 2024-04-17T07:20:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.007+0000 7fe381200640 1 -- 172.21.15.86:0/2498245934 learned_addr learned my addr 172.21.15.86:0/2498245934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.008+0000 7fe381c00640 1 --2- 172.21.15.86:0/2498245934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa247710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.008+0000 7fe382600640 1 -- 172.21.15.86:0/2498245934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24db80 msgr2=0x55b6fa25ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.008+0000 7fe382600640 1 --2- 172.21.15.86:0/2498245934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24db80 0x55b6fa25ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.008+0000 7fe382600640 1 -- 172.21.15.86:0/2498245934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 msgr2=0x55b6fa247710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.008+0000 7fe382600640 1 --2- 172.21.15.86:0/2498245934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa247710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.008+0000 7fe382600640 1 -- 172.21.15.86:0/2498245934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6fa249170 con 0x55b6fa252cf0 2024-04-17T07:20:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.008+0000 7fe382600640 1 --2- 172.21.15.86:0/2498245934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 0x55b6fa269e40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe374004430 tx=0x7fe37402bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e2f2aa75c8c37b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:15.010 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:15 smithi086 ceph-mon[31733]: pgmap v616: 129 pgs: 129 active+clean; 955 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.6 MiB/s wr, 858 op/s 2024-04-17T07:20:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.013+0000 7fe36e000640 1 -- 172.21.15.86:0/2498245934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374056050 con 0x55b6fa252cf0 2024-04-17T07:20:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.013+0000 7fe36e000640 1 -- 172.21.15.86:0/2498245934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe374004620 con 0x55b6fa252cf0 2024-04-17T07:20:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.013+0000 7fe36e000640 1 -- 172.21.15.86:0/2498245934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374043070 con 0x55b6fa252cf0 2024-04-17T07:20:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.014+0000 7fe387377740 1 -- 172.21.15.86:0/2498245934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 msgr2=0x55b6fa269e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.014+0000 7fe387377740 1 --2- 172.21.15.86:0/2498245934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 0x55b6fa269e40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe374004430 tx=0x7fe37402bef0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.015+0000 7fe387377740 1 -- 172.21.15.86:0/2498245934 shutdown_connections 2024-04-17T07:20:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.015+0000 7fe387377740 1 --2- 172.21.15.86:0/2498245934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 0x55b6fa269e40 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.015+0000 7fe387377740 1 --2- 172.21.15.86:0/2498245934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24db80 0x55b6fa25ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.015+0000 7fe387377740 1 --2- 172.21.15.86:0/2498245934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24d190 0x55b6fa247710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.015+0000 7fe387377740 1 -- 172.21.15.86:0/2498245934 >> 172.21.15.86:0/2498245934 conn(0x55b6fa248700 msgr2=0x55b6fa247000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.016+0000 7fe387377740 1 -- 172.21.15.86:0/2498245934 shutdown_connections 2024-04-17T07:20:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.016+0000 7fe387377740 1 -- 172.21.15.86:0/2498245934 wait complete. 2024-04-17T07:20:15.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.017+0000 7fe387377740 1 -- start start 2024-04-17T07:20:15.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.017+0000 7fe387377740 1 Processor -- start 2024-04-17T07:20:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.018+0000 7fe387377740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24d190 0x55b6fa25c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.018+0000 7fe387377740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24db80 0x55b6fa25cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.018+0000 7fe387377740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 0x55b6fa25dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.018+0000 7fe387377740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1b2480 con 0x55b6fa252cf0 2024-04-17T07:20:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.018+0000 7fe387377740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1b1db0 con 0x55b6fa24db80 2024-04-17T07:20:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.019+0000 7fe382600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 0x55b6fa25dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.019+0000 7fe387377740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1aefa0 con 0x55b6fa24d190 2024-04-17T07:20:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.019+0000 7fe381c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24d190 0x55b6fa25c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.020+0000 7fe381c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24d190 0x55b6fa25c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56808/0 (socket says 172.21.15.86:56808) 2024-04-17T07:20:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.020+0000 7fe381c00640 1 -- 172.21.15.86:0/1322175083 learned_addr learned my addr 172.21.15.86:0/1322175083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.020+0000 7fe387377740 10 client.0 _ll_register_callbacks cb 0x7fe374009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.019+0000 7fe381200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24db80 0x55b6fa25cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.019+0000 7fe382600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 0x55b6fa25dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33576/0 (socket says 172.21.15.86:33576) 2024-04-17T07:20:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.021+0000 7fe381c00640 1 -- 172.21.15.86:0/1322175083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24db80 msgr2=0x55b6fa25cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.021+0000 7fe381c00640 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24db80 0x55b6fa25cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.021+0000 7fe381c00640 1 -- 172.21.15.86:0/1322175083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 msgr2=0x55b6fa25dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.021+0000 7fe381c00640 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 0x55b6fa25dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.021+0000 7fe381c00640 1 -- 172.21.15.86:0/1322175083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6fa25e190 con 0x55b6fa24d190 2024-04-17T07:20:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.021+0000 7fe381c00640 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24d190 0x55b6fa25c650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55b6fa24e5f0 tx=0x7fe3700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cec43e44309a303c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.021+0000 7fe363400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.022+0000 7fe363400640 1 -- 172.21.15.86:0/1322175083 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3700287c0 con 0x55b6fa24d190 2024-04-17T07:20:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.022+0000 7fe363400640 1 -- 172.21.15.86:0/1322175083 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe3700035f0 con 0x55b6fa24d190 2024-04-17T07:20:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.022+0000 7fe387377740 1 -- 172.21.15.86:0/1322175083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe374004620 con 0x55b6fa24d190 2024-04-17T07:20:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.022+0000 7fe363400640 1 -- 172.21.15.86:0/1322175083 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3700037f0 con 0x55b6fa24d190 2024-04-17T07:20:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.023+0000 7fe387377740 10 client.26014 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.023+0000 7fe387377740 1 -- 172.21.15.86:0/1322175083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6fa2daaa0 con 0x55b6fa24d190 2024-04-17T07:20:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.023+0000 7fe363400640 1 -- 172.21.15.86:0/1322175083 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe374004620 con 0x55b6fa24d190 2024-04-17T07:20:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.023+0000 7fe363400640 1 -- 172.21.15.86:0/1322175083 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe37002f070 con 0x55b6fa24d190 2024-04-17T07:20:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.023+0000 7fe387377740 10 client.26014 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.023+0000 7fe387377740 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.023+0000 7fe387377740 20 client.26014 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.023+0000 7fe387377740 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b6fa2db480 0x55b6fa2dd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.023+0000 7fe387377740 4 client.26014 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:20:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.023+0000 7fe387377740 1 -- 172.21.15.86:0/1322175083 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6fa1bb9b0 con 0x55b6fa2db480 2024-04-17T07:20:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.024+0000 7fe381200640 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b6fa2db480 0x55b6fa2dd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.025+0000 7fe363400640 10 client.26014 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.026+0000 7fe381200640 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b6fa2db480 0x55b6fa2dd8a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17cb6409f4fc62f3 server_cookie=ac6cba91fae96242 in_seq=0 out_seq=0 2024-04-17T07:20:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.026+0000 7fe363400640 1 -- 172.21.15.86:0/1322175083 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b6fa1bb9b0 con 0x55b6fa2db480 2024-04-17T07:20:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.026+0000 7fe363400640 10 client.26014 handle_command_reply: tid=0 2024-04-17T07:20:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.027+0000 7fe387377740 1 client.26014 shutdown 2024-04-17T07:20:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.028+0000 7fe387377740 1 -- 172.21.15.86:0/1322175083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24d190 msgr2=0x55b6fa25c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.028+0000 7fe387377740 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24d190 0x55b6fa25c650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55b6fa24e5f0 tx=0x7fe3700046c0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.028+0000 7fe387377740 1 -- 172.21.15.86:0/1322175083 shutdown_connections 2024-04-17T07:20:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.028+0000 7fe387377740 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b6fa2db480 0x55b6fa2dd8a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.028+0000 7fe387377740 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6fa252cf0 0x55b6fa25dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.028+0000 7fe387377740 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6fa24db80 0x55b6fa25cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.028+0000 7fe387377740 1 --2- 172.21.15.86:0/1322175083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6fa24d190 0x55b6fa25c650 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55b6fa24e5f0 tx=0x7fe3700046c0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.028+0000 7fe387377740 1 -- 172.21.15.86:0/1322175083 >> 172.21.15.86:0/1322175083 conn(0x55b6fa248700 msgr2=0x55b6fa247890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:15.030 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:20:15.030 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:20:15.031 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:20:15.031 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:20:15.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.029+0000 7fe363400640 0 client.26014 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:15.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.030+0000 7fe387377740 1 -- 172.21.15.86:0/1322175083 shutdown_connections 2024-04-17T07:20:15.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.030+0000 7fe387377740 1 -- 172.21.15.86:0/1322175083 wait complete. 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.030+0000 7fe387377740 20 client.26014 trim_cache size 0 max 16384 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.031+0000 7fe384600640 1 -- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3640a3db0 msgr2=0x7fe3640a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.031+0000 7fe384600640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3640a3db0 0x7fe3640a6270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55b6fa24df80 tx=0x7fe370009260 comp rx=0 tx=0).stop 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.031+0000 7fe384600640 1 -- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe37c1097b0 msgr2=0x7fe37c19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.031+0000 7fe384600640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe37c1097b0 0x7fe37c19ed10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe36800e9f0 tx=0x7fe36800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.031+0000 7fe384600640 1 -- 172.21.15.86:0/820926472 shutdown_connections 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.031+0000 7fe384600640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe3640a3db0 0x7fe3640a6270 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.031+0000 7fe384600640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe37c10a170 0x7fe37c1a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.031+0000 7fe384600640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe37c1097b0 0x7fe37c19ed10 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.031+0000 7fe384600640 1 --2- 172.21.15.86:0/820926472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c1085c0 0x7fe37c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.031+0000 7fe384600640 1 -- 172.21.15.86:0/820926472 >> 172.21.15.86:0/820926472 conn(0x7fe37c077560 msgr2=0x7fe37c10cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.032+0000 7fe384600640 1 -- 172.21.15.86:0/820926472 shutdown_connections 2024-04-17T07:20:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.032+0000 7fe384600640 1 -- 172.21.15.86:0/820926472 wait complete. 2024-04-17T07:20:15.078 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:20:15.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:20:15.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:15 smithi103 ceph-mon[35495]: pgmap v616: 129 pgs: 129 active+clean; 955 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.6 MiB/s wr, 858 op/s 2024-04-17T07:20:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.422+0000 7fd047800640 1 -- 172.21.15.86:0/1213009993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04010a170 msgr2=0x7fd040110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.422+0000 7fd047800640 1 --2- 172.21.15.86:0/1213009993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04010a170 0x7fd040110a00 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd038009ca0 tx=0x7fd03802c120 comp rx=0 tx=0).stop 2024-04-17T07:20:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.423+0000 7fd047800640 1 -- 172.21.15.86:0/1213009993 shutdown_connections 2024-04-17T07:20:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.423+0000 7fd047800640 1 --2- 172.21.15.86:0/1213009993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04010a170 0x7fd040110a00 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.423+0000 7fd047800640 1 --2- 172.21.15.86:0/1213009993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0401097b0 0x7fd040109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.423+0000 7fd047800640 1 --2- 172.21.15.86:0/1213009993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0401085c0 0x7fd0401089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.423+0000 7fd047800640 1 -- 172.21.15.86:0/1213009993 >> 172.21.15.86:0/1213009993 conn(0x7fd040077580 msgr2=0x7fd0400799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:15.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.423+0000 7fd047800640 1 -- 172.21.15.86:0/1213009993 shutdown_connections 2024-04-17T07:20:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.423+0000 7fd047800640 1 -- 172.21.15.86:0/1213009993 wait complete. 2024-04-17T07:20:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.424+0000 7fd047800640 1 Processor -- start 2024-04-17T07:20:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.424+0000 7fd047800640 1 -- start start 2024-04-17T07:20:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.424+0000 7fd047800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0401085c0 0x7fd04019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.425+0000 7fd047800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0401097b0 0x7fd04019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.425+0000 7fd047800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04010a170 0x7fd0401a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.425+0000 7fd047800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0401140d0 con 0x7fd04010a170 2024-04-17T07:20:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.425+0000 7fd047800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd040113f50 con 0x7fd0401097b0 2024-04-17T07:20:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.425+0000 7fd047800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd040114250 con 0x7fd0401085c0 2024-04-17T07:20:15.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.425+0000 7fd044e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0401085c0 0x7fd04019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.425+0000 7fd044e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0401085c0 0x7fd04019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56830/0 (socket says 172.21.15.86:56830) 2024-04-17T07:20:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.425+0000 7fd044e00640 1 -- 172.21.15.86:0/2212996460 learned_addr learned my addr 172.21.15.86:0/2212996460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.425+0000 7fd045800640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04010a170 0x7fd0401a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.426+0000 7fd03fe00640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0401097b0 0x7fd04019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.426+0000 7fd045800640 1 -- 172.21.15.86:0/2212996460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0401085c0 msgr2=0x7fd04019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.426+0000 7fd045800640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0401085c0 0x7fd04019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.426+0000 7fd045800640 1 -- 172.21.15.86:0/2212996460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0401097b0 msgr2=0x7fd04019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.426+0000 7fd045800640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0401097b0 0x7fd04019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.426+0000 7fd045800640 1 -- 172.21.15.86:0/2212996460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0401a63a0 con 0x7fd04010a170 2024-04-17T07:20:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.427+0000 7fd03fe00640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0401097b0 0x7fd04019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:15.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.427+0000 7fd045800640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04010a170 0x7fd0401a5d30 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd03802c730 tx=0x7fd038032890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.428+0000 7fd03d600640 1 -- 172.21.15.86:0/2212996460 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038039050 con 0x7fd04010a170 2024-04-17T07:20:15.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.428+0000 7fd03d600640 1 -- 172.21.15.86:0/2212996460 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd038045070 con 0x7fd04010a170 2024-04-17T07:20:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.428+0000 7fd047800640 1 -- 172.21.15.86:0/2212996460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0401a6630 con 0x7fd04010a170 2024-04-17T07:20:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.428+0000 7fd03d600640 1 -- 172.21.15.86:0/2212996460 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03803e400 con 0x7fd04010a170 2024-04-17T07:20:15.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.428+0000 7fd047800640 1 -- 172.21.15.86:0/2212996460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0401a6b70 con 0x7fd04010a170 2024-04-17T07:20:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.431+0000 7fd047800640 1 -- 172.21.15.86:0/2212996460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04010f8e0 con 0x7fd04010a170 2024-04-17T07:20:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.433+0000 7fd03d600640 1 -- 172.21.15.86:0/2212996460 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fd038040450 con 0x7fd04010a170 2024-04-17T07:20:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.437+0000 7fd03d600640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0280a3c30 0x7fd0280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.438+0000 7fd03d600640 1 -- 172.21.15.86:0/2212996460 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd0380f3460 con 0x7fd04010a170 2024-04-17T07:20:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.438+0000 7fd044e00640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0280a3c30 0x7fd0280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:15.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.439+0000 7fd044e00640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0280a3c30 0x7fd0280a60f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0340058c0 tx=0x7fd034005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:15.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.445+0000 7fd03d600640 1 -- 172.21.15.86:0/2212996460 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd0380bfac0 con 0x7fd04010a170 2024-04-17T07:20:15.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:15 smithi098 ceph-mon[35418]: pgmap v616: 129 pgs: 129 active+clean; 955 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.6 MiB/s wr, 858 op/s 2024-04-17T07:20:15.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.895+0000 7fd047800640 1 -- 172.21.15.86:0/2212996460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd04002d5d0 con 0x7fd04010a170 2024-04-17T07:20:15.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.896+0000 7fd03d600640 1 -- 172.21.15.86:0/2212996460 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fd0380bf3c0 con 0x7fd04010a170 2024-04-17T07:20:15.896 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:20:15.897 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:20:15.897 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:20:15.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.900+0000 7fd047800640 1 -- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0280a3c30 msgr2=0x7fd0280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:15.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.900+0000 7fd047800640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0280a3c30 0x7fd0280a60f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0340058c0 tx=0x7fd034005f90 comp rx=0 tx=0).stop 2024-04-17T07:20:15.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.900+0000 7fd047800640 1 -- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04010a170 msgr2=0x7fd0401a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:15.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.900+0000 7fd047800640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04010a170 0x7fd0401a5d30 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd03802c730 tx=0x7fd038032890 comp rx=0 tx=0).stop 2024-04-17T07:20:15.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.901+0000 7fd047800640 1 -- 172.21.15.86:0/2212996460 shutdown_connections 2024-04-17T07:20:15.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.901+0000 7fd047800640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd0280a3c30 0x7fd0280a60f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.901+0000 7fd047800640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd04010a170 0x7fd0401a5d30 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.901+0000 7fd047800640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0401097b0 0x7fd04019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.901+0000 7fd047800640 1 --2- 172.21.15.86:0/2212996460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0401085c0 0x7fd04019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:15.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.901+0000 7fd047800640 1 -- 172.21.15.86:0/2212996460 >> 172.21.15.86:0/2212996460 conn(0x7fd040077580 msgr2=0x7fd04010cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:15.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.901+0000 7fd047800640 1 -- 172.21.15.86:0/2212996460 shutdown_connections 2024-04-17T07:20:15.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:15.901+0000 7fd047800640 1 -- 172.21.15.86:0/2212996460 wait complete. 2024-04-17T07:20:15.931 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:20:16.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.243+0000 7fb469600640 1 -- 172.21.15.86:0/4181284107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45c0945f0 msgr2=0x7fb45c096a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.243+0000 7fb469600640 1 --2- 172.21.15.86:0/4181284107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45c0945f0 0x7fb45c096a20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb454009900 tx=0x7fb45402bc30 comp rx=0 tx=0).stop 2024-04-17T07:20:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.244+0000 7fb469600640 1 -- 172.21.15.86:0/4181284107 shutdown_connections 2024-04-17T07:20:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.244+0000 7fb469600640 1 --2- 172.21.15.86:0/4181284107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c099910 0x7fb45c099d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.244+0000 7fb469600640 1 --2- 172.21.15.86:0/4181284107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c096f60 0x7fb45c0993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.244+0000 7fb469600640 1 --2- 172.21.15.86:0/4181284107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45c0945f0 0x7fb45c096a20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.244+0000 7fb469600640 1 -- 172.21.15.86:0/4181284107 >> 172.21.15.86:0/4181284107 conn(0x7fb45c0901d0 msgr2=0x7fb45c092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.244+0000 7fb469600640 1 -- 172.21.15.86:0/4181284107 shutdown_connections 2024-04-17T07:20:16.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.244+0000 7fb469600640 1 -- 172.21.15.86:0/4181284107 wait complete. 2024-04-17T07:20:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.247+0000 7fb469600640 1 Processor -- start 2024-04-17T07:20:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.247+0000 7fb469600640 1 -- start start 2024-04-17T07:20:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.248+0000 7fb469600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c0945f0 0x7fb45c1325a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.248+0000 7fb469600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45c096f60 0x7fb45c132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.248+0000 7fb469600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c099910 0x7fb45c139b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.248+0000 7fb469600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0a7ef0 con 0x7fb45c096f60 2024-04-17T07:20:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.248+0000 7fb469600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0a7d70 con 0x7fb45c099910 2024-04-17T07:20:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.248+0000 7fb469600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0a8070 con 0x7fb45c0945f0 2024-04-17T07:20:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.249+0000 7fb463400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c099910 0x7fb45c139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.249+0000 7fb463400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c099910 0x7fb45c139b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38972/0 (socket says 172.21.15.86:38972) 2024-04-17T07:20:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.249+0000 7fb463400640 1 -- 172.21.15.86:0/1795909442 learned_addr learned my addr 172.21.15.86:0/1795909442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.249+0000 7fb462a00640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c0945f0 0x7fb45c1325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.250+0000 7fb463400640 1 -- 172.21.15.86:0/1795909442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c0945f0 msgr2=0x7fb45c1325a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.250+0000 7fb463400640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c0945f0 0x7fb45c1325a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.250+0000 7fb463400640 1 -- 172.21.15.86:0/1795909442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45c096f60 msgr2=0x7fb45c132ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:20:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.250+0000 7fb463400640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45c096f60 0x7fb45c132ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.250+0000 7fb463400640 1 -- 172.21.15.86:0/1795909442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c13a1c0 con 0x7fb45c099910 2024-04-17T07:20:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.250+0000 7fb463400640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c099910 0x7fb45c139b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb464050aa0 tx=0x7fb46407bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.251+0000 7fb45b400640 1 -- 172.21.15.86:0/1795909442 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46406aca0 con 0x7fb45c099910 2024-04-17T07:20:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.251+0000 7fb45b400640 1 -- 172.21.15.86:0/1795909442 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb46406ae40 con 0x7fb45c099910 2024-04-17T07:20:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.251+0000 7fb45b400640 1 -- 172.21.15.86:0/1795909442 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464079680 con 0x7fb45c099910 2024-04-17T07:20:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.251+0000 7fb469600640 1 -- 172.21.15.86:0/1795909442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c13a4b0 con 0x7fb45c099910 2024-04-17T07:20:16.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.252+0000 7fb469600640 1 -- 172.21.15.86:0/1795909442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c13a980 con 0x7fb45c099910 2024-04-17T07:20:16.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.257+0000 7fb46c232740 1 Processor -- start 2024-04-17T07:20:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.260+0000 7fb45b400640 1 -- 172.21.15.86:0/1795909442 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb4640779f0 con 0x7fb45c099910 2024-04-17T07:20:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.261+0000 7fb46c232740 1 -- start start 2024-04-17T07:20:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.262+0000 7fb46c232740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298160 0x558ca0298580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.262+0000 7fb46c232740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 0x558ca029d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.262+0000 7fb46c232740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca029dcc0 0x558ca02a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.262+0000 7fb46c232740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558ca01fe020 con 0x558ca0298160 2024-04-17T07:20:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.262+0000 7fb46c232740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558ca01a4460 con 0x558ca0298b50 2024-04-17T07:20:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.262+0000 7fb46c232740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558ca01f9fe0 con 0x558ca029dcc0 2024-04-17T07:20:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.263+0000 7fb462000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298160 0x558ca0298580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.263+0000 7fb462000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298160 0x558ca0298580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33606/0 (socket says 172.21.15.86:33606) 2024-04-17T07:20:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.263+0000 7fb462000640 1 -- 172.21.15.86:0/3032919070 learned_addr learned my addr 172.21.15.86:0/3032919070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.263+0000 7fb462a00640 1 --2- 172.21.15.86:0/3032919070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 0x558ca029d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.263+0000 7fb462000640 1 --2- 172.21.15.86:0/3032919070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca029dcc0 0x558ca02a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.265+0000 7fb462a00640 1 -- 172.21.15.86:0/3032919070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca029dcc0 msgr2=0x558ca02a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.265+0000 7fb462a00640 1 --2- 172.21.15.86:0/3032919070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca029dcc0 0x558ca02a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.265+0000 7fb462a00640 1 -- 172.21.15.86:0/3032919070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298160 msgr2=0x558ca0298580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.265+0000 7fb462a00640 1 --2- 172.21.15.86:0/3032919070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298160 0x558ca0298580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.265+0000 7fb462a00640 1 -- 172.21.15.86:0/3032919070 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca01ac040 con 0x558ca0298b50 2024-04-17T07:20:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.265+0000 7fb45b400640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4480a3c30 0x7fb4480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.265+0000 7fb45b400640 1 -- 172.21.15.86:0/1795909442 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb46413cb40 con 0x7fb45c099910 2024-04-17T07:20:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.265+0000 7fb462000640 1 --2- 172.21.15.86:0/3032919070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298160 0x558ca0298580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:16.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.266+0000 7fb462a00640 1 --2- 172.21.15.86:0/3032919070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 0x558ca029d650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb45402c170 tx=0x7fb454009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5b1c4c8fd198b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.266+0000 7fb462000640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4480a3c30 0x7fb4480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.267+0000 7fb44f400640 1 -- 172.21.15.86:0/3032919070 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454045020 con 0x558ca0298b50 2024-04-17T07:20:16.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.267+0000 7fb44f400640 1 -- 172.21.15.86:0/3032919070 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb45402ca20 con 0x558ca0298b50 2024-04-17T07:20:16.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.267+0000 7fb44f400640 1 -- 172.21.15.86:0/3032919070 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4540046c0 con 0x558ca0298b50 2024-04-17T07:20:16.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.268+0000 7fb46c232740 1 -- 172.21.15.86:0/3032919070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 msgr2=0x558ca029d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.268+0000 7fb462000640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4480a3c30 0x7fb4480a60f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x558ca02993b0 tx=0x7fb450009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.268+0000 7fb46c232740 1 --2- 172.21.15.86:0/3032919070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 0x558ca029d650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb45402c170 tx=0x7fb454009560 comp rx=0 tx=0).stop 2024-04-17T07:20:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.268+0000 7fb46c232740 1 -- 172.21.15.86:0/3032919070 shutdown_connections 2024-04-17T07:20:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.268+0000 7fb46c232740 1 --2- 172.21.15.86:0/3032919070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca029dcc0 0x558ca02a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.268+0000 7fb46c232740 1 --2- 172.21.15.86:0/3032919070 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 0x558ca029d650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.268+0000 7fb46c232740 1 --2- 172.21.15.86:0/3032919070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298160 0x558ca0298580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.268+0000 7fb46c232740 1 -- 172.21.15.86:0/3032919070 >> 172.21.15.86:0/3032919070 conn(0x558ca02936d0 msgr2=0x558ca0293ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.268+0000 7fb46c232740 1 -- 172.21.15.86:0/3032919070 shutdown_connections 2024-04-17T07:20:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.268+0000 7fb46c232740 1 -- 172.21.15.86:0/3032919070 wait complete. 2024-04-17T07:20:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.269+0000 7fb46c232740 1 -- start start 2024-04-17T07:20:16.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.269+0000 7fb46c232740 1 Processor -- start 2024-04-17T07:20:16.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.270+0000 7fb46c232740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298160 0x558ca02a2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.270+0000 7fb46c232740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298b50 0x558ca02a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.270+0000 7fb46c232740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca029dcc0 0x558ca02a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.270+0000 7fb46c232740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558ca02a0c10 con 0x558ca029dcc0 2024-04-17T07:20:16.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.270+0000 7fb46c232740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558ca01b7830 con 0x558ca0298160 2024-04-17T07:20:16.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.270+0000 7fb46c232740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558ca02a0d90 con 0x558ca0298b50 2024-04-17T07:20:16.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.270+0000 7fb46c232740 10 client.0 _ll_register_callbacks cb 0x558ca0185fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:16.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.270+0000 7fb462000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298b50 0x558ca02a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.270+0000 7fb462000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298b50 0x558ca02a2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56872/0 (socket says 172.21.15.86:56872) 2024-04-17T07:20:16.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.270+0000 7fb462000640 1 -- 172.21.15.86:0/3318245787 learned_addr learned my addr 172.21.15.86:0/3318245787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:16.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.271+0000 7fb463400640 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca029dcc0 0x558ca02a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.271+0000 7fb462000640 1 -- 172.21.15.86:0/3318245787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298160 msgr2=0x558ca02a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.271+0000 7fb462000640 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298160 0x558ca02a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.271+0000 7fb462000640 1 -- 172.21.15.86:0/3318245787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca029dcc0 msgr2=0x558ca02a9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.271+0000 7fb462000640 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca029dcc0 0x558ca02a9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.271+0000 7fb462000640 1 -- 172.21.15.86:0/3318245787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca01bb920 con 0x558ca0298b50 2024-04-17T07:20:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.271+0000 7fb462000640 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298b50 0x558ca02a2bc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb450012bb0 tx=0x7fb4500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4efe13e8d9de4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.272+0000 7fb443e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.272+0000 7fb443e00640 1 -- 172.21.15.86:0/3318245787 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45001fcc0 con 0x558ca0298b50 2024-04-17T07:20:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.272+0000 7fb443e00640 1 -- 172.21.15.86:0/3318245787 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb450020420 con 0x558ca0298b50 2024-04-17T07:20:16.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.272+0000 7fb46c232740 1 -- 172.21.15.86:0/3318245787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb45402ca20 con 0x558ca0298b50 2024-04-17T07:20:16.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.272+0000 7fb443e00640 1 -- 172.21.15.86:0/3318245787 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb45001fe60 con 0x558ca0298b50 2024-04-17T07:20:16.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.272+0000 7fb46c232740 10 client.26032 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:16.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.272+0000 7fb46c232740 1 -- 172.21.15.86:0/3318245787 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ca01897f0 con 0x558ca0298b50 2024-04-17T07:20:16.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.273+0000 7fb443e00640 1 -- 172.21.15.86:0/3318245787 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb45402ca20 con 0x558ca0298b50 2024-04-17T07:20:16.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.273+0000 7fb443e00640 1 -- 172.21.15.86:0/3318245787 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb45001d7e0 con 0x558ca0298b50 2024-04-17T07:20:16.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.274+0000 7fb46c232740 10 client.26032 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:16.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.274+0000 7fb46c232740 10 client.26032 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:20:16.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.274+0000 7fb46c232740 20 client.26032 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:16.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.274+0000 7fb46c232740 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x558ca02bf380 0x558ca02c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.274+0000 7fb46c232740 4 client.26032 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:16.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.274+0000 7fb46c232740 1 -- 172.21.15.86:0/3318245787 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ca020ba40 con 0x558ca02bf380 2024-04-17T07:20:16.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.275+0000 7fb462a00640 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x558ca02bf380 0x558ca02c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.275+0000 7fb462a00640 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x558ca02bf380 0x558ca02c1840 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8d2df490d3714146 server_cookie=74b831267e66a899 in_seq=0 out_seq=0 2024-04-17T07:20:16.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.275+0000 7fb443e00640 10 client.26032 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:20:16.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:16 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2212996460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:20:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb443e00640 1 -- 172.21.15.86:0/3318245787 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ca020ba40 con 0x558ca02bf380 2024-04-17T07:20:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb443e00640 10 client.26032 handle_command_reply: tid=0 2024-04-17T07:20:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb46c232740 1 client.26032 shutdown 2024-04-17T07:20:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb46c232740 1 -- 172.21.15.86:0/3318245787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298b50 msgr2=0x558ca02a2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb46c232740 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298b50 0x558ca02a2bc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb450012bb0 tx=0x7fb4500046c0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb46c232740 1 -- 172.21.15.86:0/3318245787 shutdown_connections 2024-04-17T07:20:16.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb46c232740 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x558ca02bf380 0x558ca02c1840 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb46c232740 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca029dcc0 0x558ca02a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb46c232740 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298b50 0x558ca02a2bc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb46c232740 1 --2- 172.21.15.86:0/3318245787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298160 0x558ca02a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb443e00640 0 client.26032 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:20:16.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.277+0000 7fb46c232740 1 -- 172.21.15.86:0/3318245787 >> 172.21.15.86:0/3318245787 conn(0x558ca02936d0 msgr2=0x558ca0292930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.278+0000 7fb46c232740 1 -- 172.21.15.86:0/3318245787 shutdown_connections 2024-04-17T07:20:16.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.278+0000 7fb46c232740 1 -- 172.21.15.86:0/3318245787 wait complete. 2024-04-17T07:20:16.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.279+0000 7fb46c232740 20 client.26032 trim_cache size 0 max 16384 2024-04-17T07:20:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.305+0000 7fb46c232740 1 Processor -- start 2024-04-17T07:20:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.305+0000 7fb46c232740 1 -- start start 2024-04-17T07:20:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.305+0000 7fb46c232740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca0297040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.305+0000 7fb46c232740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298b50 0x558ca02a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.305+0000 7fb46c232740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca029dcc0 0x558ca02a1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.305+0000 7fb46c232740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c144c50 con 0x558ca0298b50 2024-04-17T07:20:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.305+0000 7fb46c232740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c146790 con 0x558ca029dcc0 2024-04-17T07:20:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.305+0000 7fb46c232740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c145320 con 0x558ca0298160 2024-04-17T07:20:16.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.306+0000 7fb462000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298b50 0x558ca02a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.306+0000 7fb462a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca0297040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.306+0000 7fb462a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca0297040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56884/0 (socket says 172.21.15.86:56884) 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.306+0000 7fb462a00640 1 -- 172.21.15.86:0/3118991032 learned_addr learned my addr 172.21.15.86:0/3118991032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.306+0000 7fb463400640 1 --2- 172.21.15.86:0/3118991032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca029dcc0 0x558ca02a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.307+0000 7fb462a00640 1 -- 172.21.15.86:0/3118991032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca029dcc0 msgr2=0x558ca02a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.307+0000 7fb462a00640 1 --2- 172.21.15.86:0/3118991032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca029dcc0 0x558ca02a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.307+0000 7fb462a00640 1 -- 172.21.15.86:0/3118991032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298b50 msgr2=0x558ca02a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.307+0000 7fb462a00640 1 --2- 172.21.15.86:0/3118991032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298b50 0x558ca02a9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.307+0000 7fb462a00640 1 -- 172.21.15.86:0/3118991032 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c9ff103f0 con 0x558ca0298160 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.307+0000 7fb462000640 1 --2- 172.21.15.86:0/3118991032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298b50 0x558ca02a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:16.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.308+0000 7fb462a00640 1 --2- 172.21.15.86:0/3118991032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca0297040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb454005ab0 tx=0x7fb454009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=469225f8afae7251 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.308+0000 7fb44ea00640 1 -- 172.21.15.86:0/3118991032 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454045040 con 0x558ca0298160 2024-04-17T07:20:16.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.308+0000 7fb44ea00640 1 -- 172.21.15.86:0/3118991032 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb454005550 con 0x558ca0298160 2024-04-17T07:20:16.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.308+0000 7fb44ea00640 1 -- 172.21.15.86:0/3118991032 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454003b40 con 0x558ca0298160 2024-04-17T07:20:16.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.309+0000 7fb46c232740 1 -- 172.21.15.86:0/3118991032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 msgr2=0x558ca0297040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.309+0000 7fb46c232740 1 --2- 172.21.15.86:0/3118991032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca0297040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb454005ab0 tx=0x7fb454009260 comp rx=0 tx=0).stop 2024-04-17T07:20:16.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.310+0000 7fb46c232740 1 -- 172.21.15.86:0/3118991032 shutdown_connections 2024-04-17T07:20:16.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.310+0000 7fb46c232740 1 --2- 172.21.15.86:0/3118991032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca029dcc0 0x558ca02a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.310+0000 7fb46c232740 1 --2- 172.21.15.86:0/3118991032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca0298b50 0x558ca02a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.310+0000 7fb46c232740 1 --2- 172.21.15.86:0/3118991032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca0297040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.310+0000 7fb46c232740 1 -- 172.21.15.86:0/3118991032 >> 172.21.15.86:0/3118991032 conn(0x558ca02936d0 msgr2=0x558ca0296930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.310+0000 7fb46c232740 1 -- 172.21.15.86:0/3118991032 shutdown_connections 2024-04-17T07:20:16.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.310+0000 7fb46c232740 1 -- 172.21.15.86:0/3118991032 wait complete. 2024-04-17T07:20:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.310+0000 7fb46c232740 1 -- start start 2024-04-17T07:20:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.311+0000 7fb46c232740 1 Processor -- start 2024-04-17T07:20:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.311+0000 7fb46c232740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca02a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.311+0000 7fb46c232740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 0x558ca02a7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.311+0000 7fb46c232740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca029dcc0 0x558ca02a8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.311+0000 7fb46c232740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c1460c0 con 0x558ca029dcc0 2024-04-17T07:20:16.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.311+0000 7fb46c232740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c1459f0 con 0x558ca0298b50 2024-04-17T07:20:16.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.311+0000 7fb46c232740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c142c20 con 0x558ca0298160 2024-04-17T07:20:16.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.311+0000 7fb46c232740 10 client.0 _ll_register_callbacks cb 0x7fb450012860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:16.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.312+0000 7fb462a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca02a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.312+0000 7fb462a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca02a6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56888/0 (socket says 172.21.15.86:56888) 2024-04-17T07:20:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.312+0000 7fb462a00640 1 -- 172.21.15.86:0/4085802443 learned_addr learned my addr 172.21.15.86:0/4085802443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.312+0000 7fb462000640 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 0x558ca02a7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.313+0000 7fb463400640 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca029dcc0 0x558ca02a8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.313+0000 7fb462a00640 1 -- 172.21.15.86:0/4085802443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 msgr2=0x558ca02a7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.313+0000 7fb462a00640 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 0x558ca02a7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.313+0000 7fb462a00640 1 -- 172.21.15.86:0/4085802443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca029dcc0 msgr2=0x558ca02a8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.313+0000 7fb462a00640 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca029dcc0 0x558ca02a8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.313+0000 7fb462a00640 1 -- 172.21.15.86:0/4085802443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c9ffda7d0 con 0x558ca0298160 2024-04-17T07:20:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.314+0000 7fb462a00640 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca02a6c10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb454050000 tx=0x7fb45402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca7da34152ded3fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.314+0000 7fb443e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.314+0000 7fb443e00640 1 -- 172.21.15.86:0/4085802443 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454045030 con 0x558ca0298160 2024-04-17T07:20:16.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.314+0000 7fb443e00640 1 -- 172.21.15.86:0/4085802443 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb454041680 con 0x558ca0298160 2024-04-17T07:20:16.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.315+0000 7fb46c232740 1 -- 172.21.15.86:0/4085802443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb454005550 con 0x558ca0298160 2024-04-17T07:20:16.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.315+0000 7fb443e00640 1 -- 172.21.15.86:0/4085802443 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4540418c0 con 0x558ca0298160 2024-04-17T07:20:16.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.315+0000 7fb46c232740 10 client.26038 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:16.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.315+0000 7fb46c232740 1 -- 172.21.15.86:0/4085802443 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ca02a9880 con 0x558ca0298160 2024-04-17T07:20:16.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.316+0000 7fb443e00640 1 -- 172.21.15.86:0/4085802443 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb454005550 con 0x558ca0298160 2024-04-17T07:20:16.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.317+0000 7fb443e00640 1 -- 172.21.15.86:0/4085802443 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb454056a60 con 0x558ca0298160 2024-04-17T07:20:16.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.317+0000 7fb46c232740 10 client.26038 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:16.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.317+0000 7fb46c232740 10 client.26038 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:20:16.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.317+0000 7fb46c232740 20 client.26038 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:16.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.317+0000 7fb46c232740 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x558ca02a9d00 0x558ca0327990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.317+0000 7fb46c232740 4 client.26038 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:20:16.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.317+0000 7fb46c232740 1 -- 172.21.15.86:0/4085802443 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ca02058b0 con 0x558ca02a9d00 2024-04-17T07:20:16.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.318+0000 7fb462000640 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x558ca02a9d00 0x558ca0327990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.320+0000 7fb462000640 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x558ca02a9d00 0x558ca0327990 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e757c4c4c916419 server_cookie=4da9d9915d6faed2 in_seq=0 out_seq=0 2024-04-17T07:20:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.320+0000 7fb443e00640 10 client.26038 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:20:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.321+0000 7fb443e00640 1 -- 172.21.15.86:0/4085802443 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ca02058b0 con 0x558ca02a9d00 2024-04-17T07:20:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.321+0000 7fb443e00640 10 client.26038 handle_command_reply: tid=0 2024-04-17T07:20:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.321+0000 7fb46c232740 1 client.26038 shutdown 2024-04-17T07:20:16.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.322+0000 7fb46c232740 1 -- 172.21.15.86:0/4085802443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 msgr2=0x558ca02a6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.322+0000 7fb46c232740 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca02a6c10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb454050000 tx=0x7fb45402bc30 comp rx=0 tx=0).stop 2024-04-17T07:20:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.322+0000 7fb46c232740 1 -- 172.21.15.86:0/4085802443 shutdown_connections 2024-04-17T07:20:16.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.322+0000 7fb46c232740 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x558ca02a9d00 0x558ca0327990 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.322+0000 7fb46c232740 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ca029dcc0 0x558ca02a8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.322+0000 7fb46c232740 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ca0298b50 0x558ca02a7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.322+0000 7fb46c232740 1 --2- 172.21.15.86:0/4085802443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ca0298160 0x558ca02a6c10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.322+0000 7fb46c232740 1 -- 172.21.15.86:0/4085802443 >> 172.21.15.86:0/4085802443 conn(0x558ca02936d0 msgr2=0x558ca02971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.322+0000 7fb443e00640 0 client.26038 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:20:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.323+0000 7fb46c232740 1 -- 172.21.15.86:0/4085802443 shutdown_connections 2024-04-17T07:20:16.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.323+0000 7fb46c232740 1 -- 172.21.15.86:0/4085802443 wait complete. 2024-04-17T07:20:16.324 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.323+0000 7fb46c232740 20 client.26038 trim_cache size 0 max 16384 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 -- 172.21.15.86:0/1795909442 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4480a3c30 msgr2=0x7fb4480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4480a3c30 0x7fb4480a60f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x558ca02993b0 tx=0x7fb450009670 comp rx=0 tx=0).stop 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 -- 172.21.15.86:0/1795909442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c099910 msgr2=0x7fb45c139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c099910 0x7fb45c139b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb464050aa0 tx=0x7fb46407bee0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 -- 172.21.15.86:0/1795909442 shutdown_connections 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4480a3c30 0x7fb4480a60f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c099910 0x7fb45c139b50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45c096f60 0x7fb45c132ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 --2- 172.21.15.86:0/1795909442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c0945f0 0x7fb45c1325a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 -- 172.21.15.86:0/1795909442 >> 172.21.15.86:0/1795909442 conn(0x7fb45c0901d0 msgr2=0x7fb45c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.324+0000 7fb469600640 1 -- 172.21.15.86:0/1795909442 shutdown_connections 2024-04-17T07:20:16.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.325+0000 7fb469600640 1 -- 172.21.15.86:0/1795909442 wait complete. 2024-04-17T07:20:16.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:16 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2212996460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:20:16.351 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:20:16.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:16 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2212996460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:20:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.664+0000 7f45aca00640 1 -- 172.21.15.86:0/3423376783 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598005220 con 0x7f45b0103b60 2024-04-17T07:20:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.664+0000 7f45b4c00640 1 -- 172.21.15.86:0/3423376783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0103b60 msgr2=0x7f45b010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.664+0000 7f45b4c00640 1 --2- 172.21.15.86:0/3423376783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0103b60 0x7f45b010a3f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4598009900 tx=0x7f459802bc30 comp rx=0 tx=0).stop 2024-04-17T07:20:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.665+0000 7f45b4c00640 1 -- 172.21.15.86:0/3423376783 shutdown_connections 2024-04-17T07:20:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.665+0000 7f45b4c00640 1 --2- 172.21.15.86:0/3423376783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0103b60 0x7f45b010a3f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.665+0000 7f45b4c00640 1 --2- 172.21.15.86:0/3423376783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45b01031a0 0x7f45b0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.665+0000 7f45b4c00640 1 --2- 172.21.15.86:0/3423376783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b0101fb0 0x7f45b01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.665+0000 7f45b4c00640 1 -- 172.21.15.86:0/3423376783 >> 172.21.15.86:0/3423376783 conn(0x7f45b00fd540 msgr2=0x7f45b00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.666+0000 7f45b4c00640 1 -- 172.21.15.86:0/3423376783 shutdown_connections 2024-04-17T07:20:16.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.667+0000 7f45b4c00640 1 -- 172.21.15.86:0/3423376783 wait complete. 2024-04-17T07:20:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.669+0000 7f45b4c00640 1 Processor -- start 2024-04-17T07:20:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.670+0000 7f45b4c00640 1 -- start start 2024-04-17T07:20:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.670+0000 7f45b4c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45b0101fb0 0x7f45b019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.670+0000 7f45b4c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b01031a0 0x7f45b019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.671+0000 7f45b4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0103b60 0x7f45b01a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.671+0000 7f45b4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45b010fd40 con 0x7f45b0103b60 2024-04-17T07:20:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.671+0000 7f45b4c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45b010fbc0 con 0x7f45b0101fb0 2024-04-17T07:20:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.671+0000 7f45b4c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45b010fec0 con 0x7f45b01031a0 2024-04-17T07:20:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.671+0000 7f45ae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0103b60 0x7f45b01a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.671+0000 7f45ad400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b01031a0 0x7f45b019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.672+0000 7f45ad400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b01031a0 0x7f45b019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56906/0 (socket says 172.21.15.86:56906) 2024-04-17T07:20:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.672+0000 7f45ad400640 1 -- 172.21.15.86:0/1770544641 learned_addr learned my addr 172.21.15.86:0/1770544641 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.672+0000 7f45ade00640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45b0101fb0 0x7f45b019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.672+0000 7f45ad400640 1 -- 172.21.15.86:0/1770544641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45b0101fb0 msgr2=0x7f45b019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.672+0000 7f45ad400640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45b0101fb0 0x7f45b019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.672+0000 7f45ad400640 1 -- 172.21.15.86:0/1770544641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0103b60 msgr2=0x7f45b01a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.672+0000 7f45ad400640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0103b60 0x7f45b01a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.672+0000 7f45ad400640 1 -- 172.21.15.86:0/1770544641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b01a1db0 con 0x7f45b01031a0 2024-04-17T07:20:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.672+0000 7f45ae800640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0103b60 0x7f45b01a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.673+0000 7f45ad400640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b01031a0 0x7f45b019a6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e9f0 tx=0x7f45a800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.673+0000 7f45a6a00640 1 -- 172.21.15.86:0/1770544641 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a800cd80 con 0x7f45b01031a0 2024-04-17T07:20:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.673+0000 7f45a6a00640 1 -- 172.21.15.86:0/1770544641 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45a80108e0 con 0x7f45b01031a0 2024-04-17T07:20:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.673+0000 7f45a6a00640 1 -- 172.21.15.86:0/1770544641 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8018920 con 0x7f45b01031a0 2024-04-17T07:20:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.673+0000 7f45b4c00640 1 -- 172.21.15.86:0/1770544641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b01a20a0 con 0x7f45b01031a0 2024-04-17T07:20:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.674+0000 7f45b4c00640 1 -- 172.21.15.86:0/1770544641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b01a25a0 con 0x7f45b01031a0 2024-04-17T07:20:16.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.679+0000 7f45b7989740 1 Processor -- start 2024-04-17T07:20:16.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.679+0000 7f45b7989740 1 -- start start 2024-04-17T07:20:16.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.680+0000 7f45a6a00640 1 -- 172.21.15.86:0/1770544641 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f45a8010a90 con 0x7f45b01031a0 2024-04-17T07:20:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.680+0000 7f45b7989740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3842bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.680+0000 7f45b7989740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 0x55d8e3847ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.680+0000 7f45b7989740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e3848310 0x55d8e384a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.680+0000 7f45b7989740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8e37a0020 con 0x55d8e38431a0 2024-04-17T07:20:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.680+0000 7f45b7989740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d8e3746460 con 0x55d8e38427b0 2024-04-17T07:20:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.680+0000 7f45b7989740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d8e379bfe0 con 0x55d8e3848310 2024-04-17T07:20:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.681+0000 7f45ae800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3842bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.681+0000 7f45ae800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3842bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39040/0 (socket says 172.21.15.86:39040) 2024-04-17T07:20:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.681+0000 7f45ae800640 1 -- 172.21.15.86:0/3484204744 learned_addr learned my addr 172.21.15.86:0/3484204744 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.682+0000 7f45ade00640 1 --2- 172.21.15.86:0/3484204744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 0x55d8e3847ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.682+0000 7f45ade00640 1 -- 172.21.15.86:0/3484204744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e3848310 msgr2=0x55d8e384a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.682+0000 7f45ade00640 1 --2- 172.21.15.86:0/3484204744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e3848310 0x55d8e384a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.682+0000 7f45ade00640 1 -- 172.21.15.86:0/3484204744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 msgr2=0x55d8e3842bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.683+0000 7f45ade00640 1 --2- 172.21.15.86:0/3484204744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3842bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.683+0000 7f45ade00640 1 -- 172.21.15.86:0/3484204744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e374e040 con 0x55d8e38431a0 2024-04-17T07:20:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.683+0000 7f45ae800640 1 --2- 172.21.15.86:0/3484204744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3842bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.683+0000 7f45ade00640 1 --2- 172.21.15.86:0/3484204744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 0x55d8e3847ca0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005c40 tx=0x7f45a000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3feb011b3b802950 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.683+0000 7f45a6a00640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45940a3cb0 0x7f45940a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.683+0000 7f45a6a00640 1 -- 172.21.15.86:0/1770544641 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f45a80d0a30 con 0x7f45b01031a0 2024-04-17T07:20:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.683+0000 7f459f000640 1 -- 172.21.15.86:0/3484204744 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0004d20 con 0x55d8e38431a0 2024-04-17T07:20:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.684+0000 7f459f000640 1 -- 172.21.15.86:0/3484204744 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45a0003e20 con 0x55d8e38431a0 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.684+0000 7f459f000640 1 -- 172.21.15.86:0/3484204744 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0010640 con 0x55d8e38431a0 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.684+0000 7f45ae800640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45940a3cb0 0x7f45940a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.685+0000 7f45b7989740 1 -- 172.21.15.86:0/3484204744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 msgr2=0x55d8e3847ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.685+0000 7f45b7989740 1 --2- 172.21.15.86:0/3484204744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 0x55d8e3847ca0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005c40 tx=0x7f45a000f790 comp rx=0 tx=0).stop 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.686+0000 7f45b7989740 1 -- 172.21.15.86:0/3484204744 shutdown_connections 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.686+0000 7f45b7989740 1 --2- 172.21.15.86:0/3484204744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e3848310 0x55d8e384a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.686+0000 7f45b7989740 1 --2- 172.21.15.86:0/3484204744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 0x55d8e3847ca0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.686+0000 7f45b7989740 1 --2- 172.21.15.86:0/3484204744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3842bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.686+0000 7f45b7989740 1 -- 172.21.15.86:0/3484204744 >> 172.21.15.86:0/3484204744 conn(0x55d8e383dd20 msgr2=0x55d8e383e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.687+0000 7f45b7989740 1 -- 172.21.15.86:0/3484204744 shutdown_connections 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.687+0000 7f45ae800640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45940a3cb0 0x7f45940a6170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55d8e3843a00 tx=0x7f4598005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.687+0000 7f45b7989740 1 -- 172.21.15.86:0/3484204744 wait complete. 2024-04-17T07:20:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.688+0000 7f45b7989740 1 -- start start 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.688+0000 7f45b7989740 1 Processor -- start 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45b7989740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38427b0 0x55d8e3836d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45b7989740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38431a0 0x55d8e3837240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45b7989740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e3848310 0x55d8e383c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45b7989740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8e38351c0 con 0x55d8e38427b0 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45b7989740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d8e3759830 con 0x55d8e3848310 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45b7989740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d8e3835340 con 0x55d8e38431a0 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45b7989740 10 client.0 _ll_register_callbacks cb 0x55d8e3727fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38427b0 0x55d8e3836d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38427b0 0x55d8e3836d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33656/0 (socket says 172.21.15.86:33656) 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45ade00640 1 -- 172.21.15.86:0/1246642098 learned_addr learned my addr 172.21.15.86:0/1246642098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45ad400640 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38431a0 0x55d8e3837240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.689+0000 7f45ae800640 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e3848310 0x55d8e383c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.690+0000 7f45ade00640 1 -- 172.21.15.86:0/1246642098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38431a0 msgr2=0x55d8e3837240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.690+0000 7f45ade00640 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38431a0 0x55d8e3837240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.690+0000 7f45ade00640 1 -- 172.21.15.86:0/1246642098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e3848310 msgr2=0x55d8e383c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.690+0000 7f45ade00640 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e3848310 0x55d8e383c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.690+0000 7f45ade00640 1 -- 172.21.15.86:0/1246642098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e375d920 con 0x55d8e38427b0 2024-04-17T07:20:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.691+0000 7f45ade00640 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38427b0 0x55d8e3836d00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f45a000b010 tx=0x7f45a0004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8134f7c2741a3427 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.691+0000 7f4593400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:20:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.692+0000 7f4593400640 1 -- 172.21.15.86:0/1246642098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0026020 con 0x55d8e38427b0 2024-04-17T07:20:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.692+0000 7f4593400640 1 -- 172.21.15.86:0/1246642098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f45a0004a40 con 0x55d8e38427b0 2024-04-17T07:20:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.692+0000 7f45b7989740 1 -- 172.21.15.86:0/1246642098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a0003e20 con 0x55d8e38427b0 2024-04-17T07:20:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.692+0000 7f4593400640 1 -- 172.21.15.86:0/1246642098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45a001c070 con 0x55d8e38427b0 2024-04-17T07:20:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.693+0000 7f45b7989740 10 client.17028 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.693+0000 7f45b7989740 1 -- 172.21.15.86:0/1246642098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8e372b7f0 con 0x55d8e38427b0 2024-04-17T07:20:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.693+0000 7f4593400640 1 -- 172.21.15.86:0/1246642098 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45a0003e20 con 0x55d8e38427b0 2024-04-17T07:20:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.693+0000 7f4593400640 1 -- 172.21.15.86:0/1246642098 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f45a0005040 con 0x55d8e38427b0 2024-04-17T07:20:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.693+0000 7f45b7989740 10 client.17028 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.693+0000 7f45b7989740 10 client.17028 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:20:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.694+0000 7f45b7989740 20 client.17028 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.694+0000 7f45b7989740 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55d8e384fc80 0x55d8e3852070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.694+0000 7f45b7989740 4 client.17028 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.694+0000 7f45b7989740 1 -- 172.21.15.86:0/1246642098 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8e37ada40 con 0x55d8e384fc80 2024-04-17T07:20:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.694+0000 7f45ad400640 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55d8e384fc80 0x55d8e3852070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.698+0000 7f45ad400640 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55d8e384fc80 0x55d8e3852070 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c23f7b96229ad319 server_cookie=af3354afe92f4bc7 in_seq=0 out_seq=0 2024-04-17T07:20:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.698+0000 7f4593400640 10 client.17028 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:20:16.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.700+0000 7f4593400640 1 -- 172.21.15.86:0/1246642098 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8e37ada40 con 0x55d8e384fc80 2024-04-17T07:20:16.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.700+0000 7f4593400640 10 client.17028 handle_command_reply: tid=0 2024-04-17T07:20:16.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.701+0000 7f45b7989740 1 client.17028 shutdown 2024-04-17T07:20:16.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.701+0000 7f45b7989740 1 -- 172.21.15.86:0/1246642098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38427b0 msgr2=0x55d8e3836d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.701+0000 7f45b7989740 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38427b0 0x55d8e3836d00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f45a000b010 tx=0x7f45a0004200 comp rx=0 tx=0).stop 2024-04-17T07:20:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.701+0000 7f45b7989740 1 -- 172.21.15.86:0/1246642098 shutdown_connections 2024-04-17T07:20:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.702+0000 7f45b7989740 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55d8e384fc80 0x55d8e3852070 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.702+0000 7f45b7989740 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e3848310 0x55d8e383c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.702+0000 7f45b7989740 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38431a0 0x55d8e3837240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.702+0000 7f45b7989740 1 --2- 172.21.15.86:0/1246642098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38427b0 0x55d8e3836d00 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.702+0000 7f45b7989740 1 -- 172.21.15.86:0/1246642098 >> 172.21.15.86:0/1246642098 conn(0x55d8e383dd20 msgr2=0x55d8e3834710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.702+0000 7f4593400640 0 client.17028 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:20:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.702+0000 7f45b7989740 1 -- 172.21.15.86:0/1246642098 shutdown_connections 2024-04-17T07:20:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.702+0000 7f45b7989740 1 -- 172.21.15.86:0/1246642098 wait complete. 2024-04-17T07:20:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.703+0000 7f45b7989740 20 client.17028 trim_cache size 0 max 16384 2024-04-17T07:20:16.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.739+0000 7f45b7989740 1 Processor -- start 2024-04-17T07:20:16.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.740+0000 7f45b7989740 1 -- start start 2024-04-17T07:20:16.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.740+0000 7f45b7989740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e383dd20 0x55d8e3841690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.740+0000 7f45b7989740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38427b0 0x55d8e3836320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.740+0000 7f45b7989740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38431a0 0x55d8e3836860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.740+0000 7f45b7989740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01b9710 con 0x55d8e383dd20 2024-04-17T07:20:16.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.740+0000 7f45b7989740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01bb250 con 0x55d8e38431a0 2024-04-17T07:20:16.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.740+0000 7f45b7989740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01b9de0 con 0x55d8e38427b0 2024-04-17T07:20:16.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.741+0000 7f45ad400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38427b0 0x55d8e3836320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.741+0000 7f45ad400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38427b0 0x55d8e3836320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56930/0 (socket says 172.21.15.86:56930) 2024-04-17T07:20:16.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.741+0000 7f45ad400640 1 -- 172.21.15.86:0/3003486052 learned_addr learned my addr 172.21.15.86:0/3003486052 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:16.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.741+0000 7f45ae800640 1 --2- 172.21.15.86:0/3003486052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38431a0 0x55d8e3836860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.741+0000 7f45ade00640 1 --2- 172.21.15.86:0/3003486052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e383dd20 0x55d8e3841690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.742+0000 7f45ad400640 1 -- 172.21.15.86:0/3003486052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38431a0 msgr2=0x55d8e3836860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.742+0000 7f45ad400640 1 --2- 172.21.15.86:0/3003486052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38431a0 0x55d8e3836860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.742+0000 7f45ad400640 1 -- 172.21.15.86:0/3003486052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e383dd20 msgr2=0x55d8e3841690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.742+0000 7f45ad400640 1 --2- 172.21.15.86:0/3003486052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e383dd20 0x55d8e3841690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.742+0000 7f45ad400640 1 -- 172.21.15.86:0/3003486052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e34b23f0 con 0x55d8e38427b0 2024-04-17T07:20:16.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.743+0000 7f45ad400640 1 --2- 172.21.15.86:0/3003486052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38427b0 0x55d8e3836320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45a809c970 tx=0x7f45a8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1504d91f0e81f72c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.743+0000 7f459e600640 1 -- 172.21.15.86:0/3003486052 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a80a55a0 con 0x55d8e38427b0 2024-04-17T07:20:16.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.743+0000 7f459e600640 1 -- 172.21.15.86:0/3003486052 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d8e37ada40 con 0x55d8e38427b0 2024-04-17T07:20:16.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.743+0000 7f459e600640 1 -- 172.21.15.86:0/3003486052 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a809c120 con 0x55d8e38427b0 2024-04-17T07:20:16.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.744+0000 7f45b7989740 1 -- 172.21.15.86:0/3003486052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38427b0 msgr2=0x55d8e3836320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.744+0000 7f45b7989740 1 --2- 172.21.15.86:0/3003486052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38427b0 0x55d8e3836320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45a809c970 tx=0x7f45a8009310 comp rx=0 tx=0).stop 2024-04-17T07:20:16.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.744+0000 7f45b7989740 1 -- 172.21.15.86:0/3003486052 shutdown_connections 2024-04-17T07:20:16.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.744+0000 7f45b7989740 1 --2- 172.21.15.86:0/3003486052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38431a0 0x55d8e3836860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.744+0000 7f45b7989740 1 --2- 172.21.15.86:0/3003486052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e38427b0 0x55d8e3836320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.744+0000 7f45b7989740 1 --2- 172.21.15.86:0/3003486052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e383dd20 0x55d8e3841690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.744+0000 7f45b7989740 1 -- 172.21.15.86:0/3003486052 >> 172.21.15.86:0/3003486052 conn(0x55d8e3852460 msgr2=0x55d8e3840f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.745+0000 7f45b7989740 1 -- 172.21.15.86:0/3003486052 shutdown_connections 2024-04-17T07:20:16.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.745+0000 7f45b7989740 1 -- 172.21.15.86:0/3003486052 wait complete. 2024-04-17T07:20:16.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.745+0000 7f45b7989740 1 -- start start 2024-04-17T07:20:16.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.745+0000 7f45b7989740 1 Processor -- start 2024-04-17T07:20:16.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.746+0000 7f45b7989740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e383dd20 0x55d8e3860d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.746+0000 7f45b7989740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3861290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.746+0000 7f45b7989740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 0x55d8e38622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.746+0000 7f45b7989740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01bab80 con 0x55d8e38431a0 2024-04-17T07:20:16.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.746+0000 7f45b7989740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01ba4b0 con 0x55d8e38427b0 2024-04-17T07:20:16.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.746+0000 7f45b7989740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01b7700 con 0x55d8e383dd20 2024-04-17T07:20:16.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.746+0000 7f45b7989740 10 client.0 _ll_register_callbacks cb 0x7f45a000aab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:16.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.747+0000 7f45ad400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3861290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.747+0000 7f45ade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e383dd20 0x55d8e3860d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.747+0000 7f45ade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e383dd20 0x55d8e3860d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56932/0 (socket says 172.21.15.86:56932) 2024-04-17T07:20:16.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.747+0000 7f45ad400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3861290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39064/0 (socket says 172.21.15.86:39064) 2024-04-17T07:20:16.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.747+0000 7f45ade00640 1 -- 172.21.15.86:0/3303934241 learned_addr learned my addr 172.21.15.86:0/3303934241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:16.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.747+0000 7f45ae800640 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 0x55d8e38622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.748+0000 7f45ade00640 1 -- 172.21.15.86:0/3303934241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 msgr2=0x55d8e3861290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.748+0000 7f45ade00640 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3861290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.748+0000 7f45ade00640 1 -- 172.21.15.86:0/3303934241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 msgr2=0x55d8e38622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:16.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.748+0000 7f45ade00640 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 0x55d8e38622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.748+0000 7f45ade00640 1 -- 172.21.15.86:0/3303934241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e357c7d0 con 0x55d8e383dd20 2024-04-17T07:20:16.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.748+0000 7f45ae800640 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 0x55d8e38622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:16.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.748+0000 7f45ad400640 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3861290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:16.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.749+0000 7f45ade00640 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e383dd20 0x55d8e3860d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f45a000ac50 tx=0x7f45a0003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab97abe836d22643 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:16.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.749+0000 7f4593400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.750+0000 7f4593400640 1 -- 172.21.15.86:0/3303934241 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0039020 con 0x55d8e383dd20 2024-04-17T07:20:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.751+0000 7f4593400640 1 -- 172.21.15.86:0/3303934241 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f45a0004480 con 0x55d8e383dd20 2024-04-17T07:20:16.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.751+0000 7f45b7989740 1 -- 172.21.15.86:0/3303934241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8e37ada40 con 0x55d8e383dd20 2024-04-17T07:20:16.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.752+0000 7f4593400640 1 -- 172.21.15.86:0/3303934241 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45a0026040 con 0x55d8e383dd20 2024-04-17T07:20:16.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.753+0000 7f45b7989740 10 client.26050 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:16.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.753+0000 7f45b7989740 1 -- 172.21.15.86:0/3303934241 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8e38b5c50 con 0x55d8e383dd20 2024-04-17T07:20:16.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.754+0000 7f4593400640 1 -- 172.21.15.86:0/3303934241 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d8e37ada40 con 0x55d8e383dd20 2024-04-17T07:20:16.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.756+0000 7f4593400640 1 -- 172.21.15.86:0/3303934241 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f45a0004a00 con 0x55d8e383dd20 2024-04-17T07:20:16.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.757+0000 7f45b7989740 10 client.26050 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:16.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.757+0000 7f45b7989740 10 client.26050 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:20:16.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.757+0000 7f45b7989740 20 client.26050 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:16.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.757+0000 7f45b7989740 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55d8e3848310 0x55d8e38b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:16.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.757+0000 7f45b7989740 4 client.26050 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:20:16.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.757+0000 7f45b7989740 1 -- 172.21.15.86:0/3303934241 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8e37a78b0 con 0x55d8e3848310 2024-04-17T07:20:16.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.758+0000 7f45ad400640 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55d8e3848310 0x55d8e38b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:16.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.759+0000 7f45ad400640 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55d8e3848310 0x55d8e38b8670 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a05a33d682c11cf server_cookie=8d6901379bfc1d23 in_seq=0 out_seq=0 2024-04-17T07:20:16.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.759+0000 7f4593400640 10 client.26050 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:20:16.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.760+0000 7f4593400640 1 -- 172.21.15.86:0/3303934241 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8e37a78b0 con 0x55d8e3848310 2024-04-17T07:20:16.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.760+0000 7f4593400640 10 client.26050 handle_command_reply: tid=0 2024-04-17T07:20:16.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.760+0000 7f45b7989740 1 client.26050 shutdown 2024-04-17T07:20:16.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f45b7989740 1 -- 172.21.15.86:0/3303934241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e383dd20 msgr2=0x55d8e3860d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f45b7989740 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e383dd20 0x55d8e3860d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f45a000ac50 tx=0x7f45a0003e30 comp rx=0 tx=0).stop 2024-04-17T07:20:16.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f45b7989740 1 -- 172.21.15.86:0/3303934241 shutdown_connections 2024-04-17T07:20:16.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f45b7989740 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55d8e3848310 0x55d8e38b8670 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f45b7989740 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8e38431a0 0x55d8e38622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f45b7989740 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d8e38427b0 0x55d8e3861290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f4593400640 0 client.26050 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:20:16.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f45b7989740 1 --2- 172.21.15.86:0/3303934241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d8e383dd20 0x55d8e3860d50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f45b7989740 1 -- 172.21.15.86:0/3303934241 >> 172.21.15.86:0/3303934241 conn(0x55d8e3852460 msgr2=0x55d8e3833e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f45b7989740 1 -- 172.21.15.86:0/3303934241 shutdown_connections 2024-04-17T07:20:16.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.761+0000 7f45b7989740 1 -- 172.21.15.86:0/3303934241 wait complete. 2024-04-17T07:20:16.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.762+0000 7f45b7989740 20 client.26050 trim_cache size 0 max 16384 2024-04-17T07:20:16.762 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:20:16.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.763+0000 7f45b4c00640 1 -- 172.21.15.86:0/1770544641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45940a3cb0 msgr2=0x7f45940a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.763+0000 7f45b4c00640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45940a3cb0 0x7f45940a6170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55d8e3843a00 tx=0x7f4598005a20 comp rx=0 tx=0).stop 2024-04-17T07:20:16.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.763+0000 7f45b4c00640 1 -- 172.21.15.86:0/1770544641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b01031a0 msgr2=0x7f45b019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:16.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.763+0000 7f45b4c00640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b01031a0 0x7f45b019a6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e9f0 tx=0x7f45a800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.764+0000 7f45b4c00640 1 -- 172.21.15.86:0/1770544641 shutdown_connections 2024-04-17T07:20:16.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.764+0000 7f45b4c00640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45940a3cb0 0x7f45940a6170 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.764+0000 7f45b4c00640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0103b60 0x7f45b01a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.764+0000 7f45b4c00640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b01031a0 0x7f45b019a6d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.764+0000 7f45b4c00640 1 --2- 172.21.15.86:0/1770544641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45b0101fb0 0x7f45b019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:16.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.764+0000 7f45b4c00640 1 -- 172.21.15.86:0/1770544641 >> 172.21.15.86:0/1770544641 conn(0x7f45b00fd540 msgr2=0x7f45b01063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:16.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.764+0000 7f45b4c00640 1 -- 172.21.15.86:0/1770544641 shutdown_connections 2024-04-17T07:20:16.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:16.764+0000 7f45b4c00640 1 -- 172.21.15.86:0/1770544641 wait complete. 2024-04-17T07:20:16.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:20:17.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:17 smithi086 ceph-mon[31733]: pgmap v617: 129 pgs: 129 active+clean; 955 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 775 op/s 2024-04-17T07:20:17.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.133+0000 7ffb49800640 1 -- 172.21.15.86:0/3840874052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb4410bb90 msgr2=0x7ffb4410dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.133+0000 7ffb49800640 1 --2- 172.21.15.86:0/3840874052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb4410bb90 0x7ffb4410dfc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffb300074e0 tx=0x7ffb3002c8d0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.133+0000 7ffb49800640 1 -- 172.21.15.86:0/3840874052 shutdown_connections 2024-04-17T07:20:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.133+0000 7ffb49800640 1 --2- 172.21.15.86:0/3840874052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4410e650 0x7ffb44110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.133+0000 7ffb49800640 1 --2- 172.21.15.86:0/3840874052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb4410bb90 0x7ffb4410dfc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.133+0000 7ffb49800640 1 --2- 172.21.15.86:0/3840874052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb441091d0 0x7ffb4410b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.134+0000 7ffb49800640 1 -- 172.21.15.86:0/3840874052 >> 172.21.15.86:0/3840874052 conn(0x7ffb4406edf0 msgr2=0x7ffb44071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.134+0000 7ffb49800640 1 -- 172.21.15.86:0/3840874052 shutdown_connections 2024-04-17T07:20:17.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.134+0000 7ffb49800640 1 -- 172.21.15.86:0/3840874052 wait complete. 2024-04-17T07:20:17.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.135+0000 7ffb49800640 1 Processor -- start 2024-04-17T07:20:17.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.135+0000 7ffb49800640 1 -- start start 2024-04-17T07:20:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.136+0000 7ffb49800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb441091d0 0x7ffb441a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.136+0000 7ffb49800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4410e650 0x7ffb441a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.136+0000 7ffb49800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb441ac740 0x7ffb441aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.136+0000 7ffb49800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441140e0 con 0x7ffb4410e650 2024-04-17T07:20:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.136+0000 7ffb49800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffb44113f60 con 0x7ffb441ac740 2024-04-17T07:20:17.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.136+0000 7ffb49800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb44114260 con 0x7ffb441091d0 2024-04-17T07:20:17.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.136+0000 7ffb43400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb441ac740 0x7ffb441aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.136+0000 7ffb43400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb441ac740 0x7ffb441aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:55910/0 (socket says 172.21.15.86:55910) 2024-04-17T07:20:17.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.137+0000 7ffb43400640 1 -- 172.21.15.86:0/3279551624 learned_addr learned my addr 172.21.15.86:0/3279551624 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.137+0000 7ffb42a00640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb441091d0 0x7ffb441a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.137+0000 7ffb43400640 1 -- 172.21.15.86:0/3279551624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb441091d0 msgr2=0x7ffb441a71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.137+0000 7ffb42000640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4410e650 0x7ffb441a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.138+0000 7ffb43400640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb441091d0 0x7ffb441a71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.138+0000 7ffb43400640 1 -- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4410e650 msgr2=0x7ffb441a76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.138+0000 7ffb43400640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4410e650 0x7ffb441a76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.138+0000 7ffb43400640 1 -- 172.21.15.86:0/3279551624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb441af1a0 con 0x7ffb441ac740 2024-04-17T07:20:17.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.138+0000 7ffb42000640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4410e650 0x7ffb441a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.138+0000 7ffb42a00640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb441091d0 0x7ffb441a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.138+0000 7ffb43400640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb441ac740 0x7ffb441aeb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009c90 tx=0x7ffb3c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.139+0000 7ffb3b400640 1 -- 172.21.15.86:0/3279551624 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c003b70 con 0x7ffb441ac740 2024-04-17T07:20:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.139+0000 7ffb49800640 1 -- 172.21.15.86:0/3279551624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb441af490 con 0x7ffb441ac740 2024-04-17T07:20:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.139+0000 7ffb49800640 1 -- 172.21.15.86:0/3279551624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb441af9e0 con 0x7ffb441ac740 2024-04-17T07:20:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.140+0000 7ffb3b400640 1 -- 172.21.15.86:0/3279551624 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb3c009820 con 0x7ffb441ac740 2024-04-17T07:20:17.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.140+0000 7ffb3b400640 1 -- 172.21.15.86:0/3279551624 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c010690 con 0x7ffb441ac740 2024-04-17T07:20:17.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.147+0000 7ffb4c601740 1 Processor -- start 2024-04-17T07:20:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.149+0000 7ffb3b400640 1 -- 172.21.15.86:0/3279551624 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ffb3c003d10 con 0x7ffb441ac740 2024-04-17T07:20:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.150+0000 7ffb4c601740 1 -- start start 2024-04-17T07:20:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.150+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e6fa0 0x55accd1e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.150+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e7990 0x55accd1ec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.151+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1ecb00 0x55accd1eef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.151+0000 7ffb4c601740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55accd14d020 con 0x55accd1ecb00 2024-04-17T07:20:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.151+0000 7ffb4c601740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55accd0f3460 con 0x55accd1e7990 2024-04-17T07:20:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.151+0000 7ffb4c601740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55accd148fe0 con 0x55accd1e6fa0 2024-04-17T07:20:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.151+0000 7ffb42000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1ecb00 0x55accd1eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.151+0000 7ffb42000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1ecb00 0x55accd1eef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50326/0 (socket says 172.21.15.86:50326) 2024-04-17T07:20:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.151+0000 7ffb42000640 1 -- 172.21.15.86:0/2368495034 learned_addr learned my addr 172.21.15.86:0/2368495034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.151+0000 7ffb42000640 1 --2- 172.21.15.86:0/2368495034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e6fa0 0x55accd1e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.152+0000 7ffb42000640 1 -- 172.21.15.86:0/2368495034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e6fa0 msgr2=0x55accd1e73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.152+0000 7ffb42000640 1 --2- 172.21.15.86:0/2368495034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e6fa0 0x55accd1e73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.152+0000 7ffb42000640 1 -- 172.21.15.86:0/2368495034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e7990 msgr2=0x55accd1ec490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:20:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.152+0000 7ffb42000640 1 --2- 172.21.15.86:0/2368495034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e7990 0x55accd1ec490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.152+0000 7ffb42000640 1 -- 172.21.15.86:0/2368495034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55accd0fb040 con 0x55accd1ecb00 2024-04-17T07:20:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.153+0000 7ffb3b400640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb280a3c30 0x7ffb280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.154+0000 7ffb42000640 1 --2- 172.21.15.86:0/2368495034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1ecb00 0x55accd1eef90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x55accd1e81f0 tx=0x7ffb30005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f89b1aaf3d6b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.154+0000 7ffb3b400640 1 -- 172.21.15.86:0/3279551624 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ffb3c0d0d30 con 0x7ffb441ac740 2024-04-17T07:20:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.156+0000 7ffb2fe00640 1 -- 172.21.15.86:0/2368495034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30043020 con 0x55accd1ecb00 2024-04-17T07:20:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.156+0000 7ffb42a00640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb280a3c30 0x7ffb280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.156+0000 7ffb2fe00640 1 -- 172.21.15.86:0/2368495034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb30007a50 con 0x55accd1ecb00 2024-04-17T07:20:17.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.156+0000 7ffb2fe00640 1 -- 172.21.15.86:0/2368495034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30004f90 con 0x55accd1ecb00 2024-04-17T07:20:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.157+0000 7ffb4c601740 1 -- 172.21.15.86:0/2368495034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1ecb00 msgr2=0x55accd1eef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.157+0000 7ffb4c601740 1 --2- 172.21.15.86:0/2368495034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1ecb00 0x55accd1eef90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x55accd1e81f0 tx=0x7ffb30005a20 comp rx=0 tx=0).stop 2024-04-17T07:20:17.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.157+0000 7ffb42a00640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb280a3c30 0x7ffb280a60f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55accd1e7d90 tx=0x7ffb34005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.157+0000 7ffb4c601740 1 -- 172.21.15.86:0/2368495034 shutdown_connections 2024-04-17T07:20:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.157+0000 7ffb4c601740 1 --2- 172.21.15.86:0/2368495034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1ecb00 0x55accd1eef90 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.157+0000 7ffb4c601740 1 --2- 172.21.15.86:0/2368495034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e7990 0x55accd1ec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.157+0000 7ffb4c601740 1 --2- 172.21.15.86:0/2368495034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e6fa0 0x55accd1e73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.157+0000 7ffb4c601740 1 -- 172.21.15.86:0/2368495034 >> 172.21.15.86:0/2368495034 conn(0x55accd1e2510 msgr2=0x55accd1e2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.158+0000 7ffb4c601740 1 -- 172.21.15.86:0/2368495034 shutdown_connections 2024-04-17T07:20:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.158+0000 7ffb4c601740 1 -- 172.21.15.86:0/2368495034 wait complete. 2024-04-17T07:20:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.158+0000 7ffb4c601740 1 -- start start 2024-04-17T07:20:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.159+0000 7ffb4c601740 1 Processor -- start 2024-04-17T07:20:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.159+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 0x55accd1f1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.159+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e7990 0x55accd1f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.159+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1f6be0 0x55accd1f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.159+0000 7ffb4c601740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55accd1efa50 con 0x55accd1e7990 2024-04-17T07:20:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.159+0000 7ffb4c601740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55accd106830 con 0x55accd1e6fa0 2024-04-17T07:20:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.159+0000 7ffb4c601740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55accd1efbd0 con 0x55accd1f6be0 2024-04-17T07:20:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.159+0000 7ffb4c601740 10 client.0 _ll_register_callbacks cb 0x55accd0d4fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.160+0000 7ffb43400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1f6be0 0x55accd1f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.160+0000 7ffb43400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1f6be0 0x55accd1f8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41086/0 (socket says 172.21.15.86:41086) 2024-04-17T07:20:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.160+0000 7ffb43400640 1 -- 172.21.15.86:0/1183551161 learned_addr learned my addr 172.21.15.86:0/1183551161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.160+0000 7ffb42000640 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e7990 0x55accd1f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.160+0000 7ffb42a00640 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 0x55accd1f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.161+0000 7ffb43400640 1 -- 172.21.15.86:0/1183551161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 msgr2=0x55accd1f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.161+0000 7ffb43400640 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 0x55accd1f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.161+0000 7ffb43400640 1 -- 172.21.15.86:0/1183551161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e7990 msgr2=0x55accd1f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.161+0000 7ffb43400640 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e7990 0x55accd1f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.161+0000 7ffb43400640 1 -- 172.21.15.86:0/1183551161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55accd10a920 con 0x55accd1f6be0 2024-04-17T07:20:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.161+0000 7ffb42000640 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e7990 0x55accd1f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.162+0000 7ffb43400640 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1f6be0 0x55accd1f8fd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c09c990 tx=0x7ffb3c09c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=933873a07180fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.162+0000 7ffb2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.162+0000 7ffb2cc00640 1 -- 172.21.15.86:0/1183551161 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c0a5900 con 0x55accd1f6be0 2024-04-17T07:20:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.162+0000 7ffb2cc00640 1 -- 172.21.15.86:0/1183551161 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ffb3c0bd0c0 con 0x55accd1f6be0 2024-04-17T07:20:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.162+0000 7ffb4c601740 1 -- 172.21.15.86:0/1183551161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb30007a50 con 0x55accd1f6be0 2024-04-17T07:20:17.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.163+0000 7ffb2cc00640 1 -- 172.21.15.86:0/1183551161 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb3c0b6200 con 0x55accd1f6be0 2024-04-17T07:20:17.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.163+0000 7ffb4c601740 10 client.26056 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:17.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.163+0000 7ffb4c601740 1 -- 172.21.15.86:0/1183551161 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55accd0d87f0 con 0x55accd1f6be0 2024-04-17T07:20:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.163+0000 7ffb2cc00640 1 -- 172.21.15.86:0/1183551161 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb30007a50 con 0x55accd1f6be0 2024-04-17T07:20:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.164+0000 7ffb2cc00640 1 -- 172.21.15.86:0/1183551161 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ffb3c098a70 con 0x55accd1f6be0 2024-04-17T07:20:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.165+0000 7ffb4c601740 10 client.26056 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:17.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.165+0000 7ffb4c601740 10 client.26056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:20:17.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.165+0000 7ffb4c601740 20 client.26056 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:17.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.165+0000 7ffb4c601740 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55accd1fd4e0 0x55accd1ff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.165+0000 7ffb4c601740 4 client.26056 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:17.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.165+0000 7ffb4c601740 1 -- 172.21.15.86:0/1183551161 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55accd15aa40 con 0x55accd1fd4e0 2024-04-17T07:20:17.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.166+0000 7ffb42a00640 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55accd1fd4e0 0x55accd1ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.167+0000 7ffb2cc00640 10 client.26056 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:20:17.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.167+0000 7ffb42a00640 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55accd1fd4e0 0x55accd1ff900 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c793c2dccd708671 server_cookie=2203bd923f172864 in_seq=0 out_seq=0 2024-04-17T07:20:17.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.168+0000 7ffb2cc00640 1 -- 172.21.15.86:0/1183551161 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55accd15aa40 con 0x55accd1fd4e0 2024-04-17T07:20:17.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.168+0000 7ffb2cc00640 10 client.26056 handle_command_reply: tid=0 2024-04-17T07:20:17.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.168+0000 7ffb4c601740 1 client.26056 shutdown 2024-04-17T07:20:17.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 1 -- 172.21.15.86:0/1183551161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1f6be0 msgr2=0x55accd1f8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1f6be0 0x55accd1f8fd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c09c990 tx=0x7ffb3c09c6e0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 1 -- 172.21.15.86:0/1183551161 shutdown_connections 2024-04-17T07:20:17.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55accd1fd4e0 0x55accd1ff900 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb2cc00640 0 client.26056 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:20:17.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1f6be0 0x55accd1f8fd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e7990 0x55accd1f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 1 --2- 172.21.15.86:0/1183551161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 0x55accd1f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 1 -- 172.21.15.86:0/1183551161 >> 172.21.15.86:0/1183551161 conn(0x55accd1e2510 msgr2=0x55accd1e1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 1 -- 172.21.15.86:0/1183551161 shutdown_connections 2024-04-17T07:20:17.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 1 -- 172.21.15.86:0/1183551161 wait complete. 2024-04-17T07:20:17.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.169+0000 7ffb4c601740 20 client.26056 trim_cache size 0 max 16384 2024-04-17T07:20:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.201+0000 7ffb4c601740 1 Processor -- start 2024-04-17T07:20:17.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.202+0000 7ffb4c601740 1 -- start start 2024-04-17T07:20:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.202+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e6fa0 0x55accd1e8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.202+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 0x55accd1f0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.202+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1f6be0 0x55accd1f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.202+0000 7ffb4c601740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2400fc30 con 0x55accd1e6fa0 2024-04-17T07:20:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.202+0000 7ffb4c601740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffb24011770 con 0x55accd1f6be0 2024-04-17T07:20:17.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.202+0000 7ffb4c601740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb24010300 con 0x55accd1e7990 2024-04-17T07:20:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.203+0000 7ffb42000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 0x55accd1f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.203+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e6fa0 0x55accd1e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.203+0000 7ffb43400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1f6be0 0x55accd1f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.203+0000 7ffb43400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1f6be0 0x55accd1f0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:55936/0 (socket says 172.21.15.86:55936) 2024-04-17T07:20:17.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.203+0000 7ffb43400640 1 -- 172.21.15.86:0/397997644 learned_addr learned my addr 172.21.15.86:0/397997644 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.205+0000 7ffb42000640 1 -- 172.21.15.86:0/397997644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1f6be0 msgr2=0x55accd1f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.205+0000 7ffb42000640 1 --2- 172.21.15.86:0/397997644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1f6be0 0x55accd1f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.205+0000 7ffb42000640 1 -- 172.21.15.86:0/397997644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e6fa0 msgr2=0x55accd1e8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.205+0000 7ffb42000640 1 --2- 172.21.15.86:0/397997644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e6fa0 0x55accd1e8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.205+0000 7ffb42000640 1 -- 172.21.15.86:0/397997644 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55accce5f3f0 con 0x55accd1e7990 2024-04-17T07:20:17.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.205+0000 7ffb42a00640 1 --2- 172.21.15.86:0/397997644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e6fa0 0x55accd1e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.206+0000 7ffb42000640 1 --2- 172.21.15.86:0/397997644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 0x55accd1f0960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb30005c40 tx=0x7ffb30005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b8601180750114 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.206+0000 7ffb2f400640 1 -- 172.21.15.86:0/397997644 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30043040 con 0x55accd1e7990 2024-04-17T07:20:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.206+0000 7ffb2f400640 1 -- 172.21.15.86:0/397997644 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb30004040 con 0x55accd1e7990 2024-04-17T07:20:17.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.206+0000 7ffb2f400640 1 -- 172.21.15.86:0/397997644 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30005310 con 0x55accd1e7990 2024-04-17T07:20:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.207+0000 7ffb4c601740 1 -- 172.21.15.86:0/397997644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 msgr2=0x55accd1f0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.207+0000 7ffb4c601740 1 --2- 172.21.15.86:0/397997644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 0x55accd1f0960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb30005c40 tx=0x7ffb30005fd0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.208+0000 7ffb4c601740 1 -- 172.21.15.86:0/397997644 shutdown_connections 2024-04-17T07:20:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.208+0000 7ffb4c601740 1 --2- 172.21.15.86:0/397997644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1f6be0 0x55accd1f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.208+0000 7ffb4c601740 1 --2- 172.21.15.86:0/397997644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 0x55accd1f0960 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.208+0000 7ffb4c601740 1 --2- 172.21.15.86:0/397997644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1e6fa0 0x55accd1e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.208+0000 7ffb4c601740 1 -- 172.21.15.86:0/397997644 >> 172.21.15.86:0/397997644 conn(0x55accd1e2510 msgr2=0x55accd1e9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.208+0000 7ffb4c601740 1 -- 172.21.15.86:0/397997644 shutdown_connections 2024-04-17T07:20:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.208+0000 7ffb4c601740 1 -- 172.21.15.86:0/397997644 wait complete. 2024-04-17T07:20:17.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.209+0000 7ffb4c601740 1 -- start start 2024-04-17T07:20:17.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.209+0000 7ffb4c601740 1 Processor -- start 2024-04-17T07:20:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.210+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 0x55accd20e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.210+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 0x55accd20ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.210+0000 7ffb4c601740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1f6be0 0x55accd20fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.210+0000 7ffb4c601740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb240110a0 con 0x55accd1f6be0 2024-04-17T07:20:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.210+0000 7ffb4c601740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffb240109d0 con 0x55accd1e6fa0 2024-04-17T07:20:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.210+0000 7ffb4c601740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2400dc00 con 0x55accd1e7990 2024-04-17T07:20:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.210+0000 7ffb4c601740 10 client.0 _ll_register_callbacks cb 0x7ffb3c09c860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.211+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 0x55accd20e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.211+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 0x55accd20e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:55938/0 (socket says 172.21.15.86:55938) 2024-04-17T07:20:17.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.211+0000 7ffb42a00640 1 -- 172.21.15.86:0/3732856746 learned_addr learned my addr 172.21.15.86:0/3732856746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.211+0000 7ffb42000640 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 0x55accd20ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.211+0000 7ffb43400640 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1f6be0 0x55accd20fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.212+0000 7ffb42a00640 1 -- 172.21.15.86:0/3732856746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 msgr2=0x55accd20ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.212+0000 7ffb42a00640 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 0x55accd20ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.212+0000 7ffb42a00640 1 -- 172.21.15.86:0/3732856746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1f6be0 msgr2=0x55accd20fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.212+0000 7ffb42a00640 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1f6be0 0x55accd20fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.212+0000 7ffb42a00640 1 -- 172.21.15.86:0/3732856746 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acccf297d0 con 0x55accd1e6fa0 2024-04-17T07:20:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.212+0000 7ffb43400640 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1f6be0 0x55accd20fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.212+0000 7ffb42a00640 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 0x55accd20e670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb34005f30 tx=0x7ffb34004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=594569f9d34cf26f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.213+0000 7ffb2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:20:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.213+0000 7ffb2cc00640 1 -- 172.21.15.86:0/3732856746 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb34026500 con 0x55accd1e6fa0 2024-04-17T07:20:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.213+0000 7ffb2cc00640 1 -- 172.21.15.86:0/3732856746 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ffb34026300 con 0x55accd1e6fa0 2024-04-17T07:20:17.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.213+0000 7ffb4c601740 1 -- 172.21.15.86:0/3732856746 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb30004040 con 0x55accd1e6fa0 2024-04-17T07:20:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.215+0000 7ffb2cc00640 1 -- 172.21.15.86:0/3732856746 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb34034050 con 0x55accd1e6fa0 2024-04-17T07:20:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.215+0000 7ffb4c601740 10 client.36038 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:17.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.215+0000 7ffb4c601740 1 -- 172.21.15.86:0/3732856746 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55accd2635f0 con 0x55accd1e6fa0 2024-04-17T07:20:17.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.216+0000 7ffb2cc00640 1 -- 172.21.15.86:0/3732856746 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb30004040 con 0x55accd1e6fa0 2024-04-17T07:20:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.217+0000 7ffb2cc00640 1 -- 172.21.15.86:0/3732856746 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ffb34004940 con 0x55accd1e6fa0 2024-04-17T07:20:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.217+0000 7ffb4c601740 10 client.36038 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.217+0000 7ffb4c601740 10 client.36038 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:20:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.217+0000 7ffb4c601740 20 client.36038 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.217+0000 7ffb4c601740 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55accd1fd4e0 0x55accd266060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.217+0000 7ffb4c601740 4 client.36038 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:20:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.217+0000 7ffb4c601740 1 -- 172.21.15.86:0/3732856746 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55accd1548b0 con 0x55accd1fd4e0 2024-04-17T07:20:17.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.218+0000 7ffb42000640 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55accd1fd4e0 0x55accd266060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.219+0000 7ffb2cc00640 10 client.36038 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:20:17.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.219+0000 7ffb42000640 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55accd1fd4e0 0x55accd266060 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3870579fcd7c89b server_cookie=1653131076c54cd2 in_seq=0 out_seq=0 2024-04-17T07:20:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.220+0000 7ffb2cc00640 1 -- 172.21.15.86:0/3732856746 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55accd1548b0 con 0x55accd1fd4e0 2024-04-17T07:20:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.220+0000 7ffb2cc00640 10 client.36038 handle_command_reply: tid=0 2024-04-17T07:20:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.220+0000 7ffb4c601740 1 client.36038 shutdown 2024-04-17T07:20:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.220+0000 7ffb4c601740 1 -- 172.21.15.86:0/3732856746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 msgr2=0x55accd20e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.220+0000 7ffb4c601740 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 0x55accd20e670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb34005f30 tx=0x7ffb34004730 comp rx=0 tx=0).stop 2024-04-17T07:20:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.221+0000 7ffb4c601740 1 -- 172.21.15.86:0/3732856746 shutdown_connections 2024-04-17T07:20:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.221+0000 7ffb4c601740 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55accd1fd4e0 0x55accd266060 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.221+0000 7ffb4c601740 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55accd1f6be0 0x55accd20fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.221+0000 7ffb4c601740 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55accd1e7990 0x55accd20ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.221+0000 7ffb4c601740 1 --2- 172.21.15.86:0/3732856746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55accd1e6fa0 0x55accd20e670 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.221+0000 7ffb4c601740 1 -- 172.21.15.86:0/3732856746 >> 172.21.15.86:0/3732856746 conn(0x55accd1e2510 msgr2=0x55accd1f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.221+0000 7ffb2cc00640 0 client.36038 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:20:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.222+0000 7ffb4c601740 1 -- 172.21.15.86:0/3732856746 shutdown_connections 2024-04-17T07:20:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.222+0000 7ffb4c601740 1 -- 172.21.15.86:0/3732856746 wait complete. 2024-04-17T07:20:17.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.222+0000 7ffb4c601740 20 client.36038 trim_cache size 0 max 16384 2024-04-17T07:20:17.223 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:20:17.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.224+0000 7ffb49800640 1 -- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb280a3c30 msgr2=0x7ffb280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.224+0000 7ffb49800640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb280a3c30 0x7ffb280a60f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55accd1e7d90 tx=0x7ffb34005cf0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.224+0000 7ffb49800640 1 -- 172.21.15.86:0/3279551624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb441ac740 msgr2=0x7ffb441aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.224+0000 7ffb49800640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb441ac740 0x7ffb441aeb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009c90 tx=0x7ffb3c00c6a0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.225+0000 7ffb49800640 1 -- 172.21.15.86:0/3279551624 shutdown_connections 2024-04-17T07:20:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.225+0000 7ffb49800640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffb280a3c30 0x7ffb280a60f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.225+0000 7ffb49800640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb441ac740 0x7ffb441aeb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.225+0000 7ffb49800640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4410e650 0x7ffb441a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.225+0000 7ffb49800640 1 --2- 172.21.15.86:0/3279551624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb441091d0 0x7ffb441a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.225+0000 7ffb49800640 1 -- 172.21.15.86:0/3279551624 >> 172.21.15.86:0/3279551624 conn(0x7ffb4406edf0 msgr2=0x7ffb4410c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.225+0000 7ffb49800640 1 -- 172.21.15.86:0/3279551624 shutdown_connections 2024-04-17T07:20:17.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.225+0000 7ffb49800640 1 -- 172.21.15.86:0/3279551624 wait complete. 2024-04-17T07:20:17.257 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:20:17.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:17 smithi103 ceph-mon[35495]: pgmap v617: 129 pgs: 129 active+clean; 955 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 775 op/s 2024-04-17T07:20:17.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:17 smithi098 ceph-mon[35418]: pgmap v617: 129 pgs: 129 active+clean; 955 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 775 op/s 2024-04-17T07:20:17.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.585+0000 7f3315a00640 1 -- 172.21.15.86:0/3432027160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3310103b60 msgr2=0x7f331010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.585+0000 7f3315a00640 1 --2- 172.21.15.86:0/3432027160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3310103b60 0x7f331010a3f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3308009be0 tx=0x7f330802bef0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.585+0000 7f3315a00640 1 -- 172.21.15.86:0/3432027160 shutdown_connections 2024-04-17T07:20:17.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.585+0000 7f3315a00640 1 --2- 172.21.15.86:0/3432027160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3310103b60 0x7f331010a3f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.585+0000 7f3315a00640 1 --2- 172.21.15.86:0/3432027160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33101031a0 0x7f3310103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.585+0000 7f3315a00640 1 --2- 172.21.15.86:0/3432027160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3310101fb0 0x7f33101023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.585+0000 7f3315a00640 1 -- 172.21.15.86:0/3432027160 >> 172.21.15.86:0/3432027160 conn(0x7f33100fd540 msgr2=0x7f33100ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.585+0000 7f3315a00640 1 -- 172.21.15.86:0/3432027160 shutdown_connections 2024-04-17T07:20:17.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.585+0000 7f3315a00640 1 -- 172.21.15.86:0/3432027160 wait complete. 2024-04-17T07:20:17.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.586+0000 7f3315a00640 1 Processor -- start 2024-04-17T07:20:17.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.587+0000 7f3315a00640 1 -- start start 2024-04-17T07:20:17.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.587+0000 7f3315a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3310101fb0 0x7f331019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.587+0000 7f3315a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101031a0 0x7f331019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.587+0000 7f3315a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3310103b60 0x7f33101a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.587+0000 7f3315a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33101140b0 con 0x7f33101031a0 2024-04-17T07:20:17.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.587+0000 7f3315a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3310113f30 con 0x7f3310103b60 2024-04-17T07:20:17.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.587+0000 7f3315a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3310114230 con 0x7f3310101fb0 2024-04-17T07:20:17.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.588+0000 7f330ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3310101fb0 0x7f331019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.588+0000 7f330e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101031a0 0x7f331019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.588+0000 7f330e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101031a0 0x7f331019ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50386/0 (socket says 172.21.15.86:50386) 2024-04-17T07:20:17.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.588+0000 7f330e000640 1 -- 172.21.15.86:0/4180379693 learned_addr learned my addr 172.21.15.86:0/4180379693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.588+0000 7f330f400640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3310103b60 0x7f33101a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.589+0000 7f330e000640 1 -- 172.21.15.86:0/4180379693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3310101fb0 msgr2=0x7f331019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.589+0000 7f330e000640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3310101fb0 0x7f331019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.589+0000 7f330e000640 1 -- 172.21.15.86:0/4180379693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3310103b60 msgr2=0x7f33101a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.590+0000 7f330e000640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3310103b60 0x7f33101a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.590+0000 7f330e000640 1 -- 172.21.15.86:0/4180379693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33101a6370 con 0x7f33101031a0 2024-04-17T07:20:17.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.590+0000 7f330f400640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3310103b60 0x7f33101a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:17.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.591+0000 7f330e000640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101031a0 0x7f331019ec90 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009ad0 tx=0x7f32fc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.591+0000 7f3307400640 1 -- 172.21.15.86:0/4180379693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00cde0 con 0x7f33101031a0 2024-04-17T07:20:17.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.591+0000 7f3307400640 1 -- 172.21.15.86:0/4180379693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32fc0035f0 con 0x7f33101031a0 2024-04-17T07:20:17.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.591+0000 7f3307400640 1 -- 172.21.15.86:0/4180379693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc017640 con 0x7f33101031a0 2024-04-17T07:20:17.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.591+0000 7f3315a00640 1 -- 172.21.15.86:0/4180379693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33101a6660 con 0x7f33101031a0 2024-04-17T07:20:17.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.591+0000 7f3315a00640 1 -- 172.21.15.86:0/4180379693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33101a6b60 con 0x7f33101031a0 2024-04-17T07:20:17.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.597+0000 7f3307400640 1 -- 172.21.15.86:0/4180379693 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f32fc004000 con 0x7f33101031a0 2024-04-17T07:20:17.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.598+0000 7f3318766740 1 Processor -- start 2024-04-17T07:20:17.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.598+0000 7f3318766740 1 -- start start 2024-04-17T07:20:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.599+0000 7f3318766740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e0113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.599+0000 7f3318766740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e011990 0x564f2e016490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.599+0000 7f3318766740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 0x564f2e018f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.599+0000 7f3318766740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f2df77020 con 0x564f2e016b00 2024-04-17T07:20:17.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.599+0000 7f3318766740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564f2df1d460 con 0x564f2e010fa0 2024-04-17T07:20:17.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.599+0000 7f3318766740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f2df72fe0 con 0x564f2e011990 2024-04-17T07:20:17.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.600+0000 7f330ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e011990 0x564f2e016490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.600+0000 7f330f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e0113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.600+0000 7f330e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 0x564f2e018f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.600+0000 7f330ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e011990 0x564f2e016490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41136/0 (socket says 172.21.15.86:41136) 2024-04-17T07:20:17.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.600+0000 7f330ea00640 1 -- 172.21.15.86:0/2762231094 learned_addr learned my addr 172.21.15.86:0/2762231094 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.600+0000 7f3307400640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32f40a3c30 0x7f32f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.601+0000 7f3307400640 1 -- 172.21.15.86:0/4180379693 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f32fc0d0610 con 0x7f33101031a0 2024-04-17T07:20:17.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.601+0000 7f330ea00640 1 -- 172.21.15.86:0/2762231094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 msgr2=0x564f2e0113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.601+0000 7f330ea00640 1 --2- 172.21.15.86:0/2762231094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e0113c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.601+0000 7f330ea00640 1 -- 172.21.15.86:0/2762231094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 msgr2=0x564f2e018f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.601+0000 7f330ea00640 1 --2- 172.21.15.86:0/2762231094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 0x564f2e018f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.601+0000 7f330ea00640 1 -- 172.21.15.86:0/2762231094 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2df25040 con 0x564f2e011990 2024-04-17T07:20:17.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.601+0000 7f330f400640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32f40a3c30 0x7f32f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.602+0000 7f330ea00640 1 --2- 172.21.15.86:0/2762231094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e011990 0x564f2e016490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f330000cbb0 tx=0x7f330000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75e766d57e84bb66 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.602+0000 7f32fbe00640 1 -- 172.21.15.86:0/2762231094 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300005220 con 0x564f2e011990 2024-04-17T07:20:17.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.602+0000 7f32fbe00640 1 -- 172.21.15.86:0/2762231094 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3300004c50 con 0x564f2e011990 2024-04-17T07:20:17.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.602+0000 7f32fbe00640 1 -- 172.21.15.86:0/2762231094 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300010720 con 0x564f2e011990 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.603+0000 7f3318766740 1 -- 172.21.15.86:0/2762231094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e011990 msgr2=0x564f2e016490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.603+0000 7f3318766740 1 --2- 172.21.15.86:0/2762231094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e011990 0x564f2e016490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f330000cbb0 tx=0x7f330000e720 comp rx=0 tx=0).stop 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.603+0000 7f330f400640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32f40a3c30 0x7f32f40a60f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x564f2e0121f0 tx=0x7f3308009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.603+0000 7f3318766740 1 -- 172.21.15.86:0/2762231094 shutdown_connections 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.603+0000 7f3318766740 1 --2- 172.21.15.86:0/2762231094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 0x564f2e018f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.603+0000 7f3318766740 1 --2- 172.21.15.86:0/2762231094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e011990 0x564f2e016490 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.603+0000 7f3318766740 1 --2- 172.21.15.86:0/2762231094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e0113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.603+0000 7f3318766740 1 -- 172.21.15.86:0/2762231094 >> 172.21.15.86:0/2762231094 conn(0x564f2e00c510 msgr2=0x564f2e00c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.604+0000 7f3318766740 1 -- 172.21.15.86:0/2762231094 shutdown_connections 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.604+0000 7f3318766740 1 -- 172.21.15.86:0/2762231094 wait complete. 2024-04-17T07:20:17.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.604+0000 7f3318766740 1 -- start start 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.604+0000 7f3318766740 1 Processor -- start 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.605+0000 7f3318766740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e01b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.605+0000 7f3318766740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e011990 0x564f2e01ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.605+0000 7f3318766740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 0x564f2e022ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.605+0000 7f3318766740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f2e019ab0 con 0x564f2e011990 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.605+0000 7f3318766740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564f2df30830 con 0x564f2e010fa0 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.605+0000 7f3318766740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f2e019c30 con 0x564f2e016b00 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.605+0000 7f3318766740 10 client.0 _ll_register_callbacks cb 0x564f2defefb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.605+0000 7f330e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e011990 0x564f2e01ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.605+0000 7f330e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e011990 0x564f2e01ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50412/0 (socket says 172.21.15.86:50412) 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.605+0000 7f330e000640 1 -- 172.21.15.86:0/3060012342 learned_addr learned my addr 172.21.15.86:0/3060012342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.606+0000 7f330ea00640 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e01b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.606+0000 7f330f400640 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 0x564f2e022ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.606+0000 7f330e000640 1 -- 172.21.15.86:0/3060012342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 msgr2=0x564f2e022ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.606+0000 7f330e000640 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 0x564f2e022ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.606+0000 7f330e000640 1 -- 172.21.15.86:0/3060012342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 msgr2=0x564f2e01b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.606+0000 7f330e000640 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e01b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.606+0000 7f330e000640 1 -- 172.21.15.86:0/3060012342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2df34920 con 0x564f2e011990 2024-04-17T07:20:17.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.606+0000 7f330f400640 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 0x564f2e022ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.606+0000 7f330e000640 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e011990 0x564f2e01ba80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0a4cb0 tx=0x7f32fc09c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d526f2f68bb3b06 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.606+0000 7f32f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.607+0000 7f32f8c00640 1 -- 172.21.15.86:0/3060012342 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc0a0920 con 0x564f2e011990 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.607+0000 7f32f8c00640 1 -- 172.21.15.86:0/3060012342 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f32fc00f660 con 0x564f2e011990 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.607+0000 7f3318766740 1 -- 172.21.15.86:0/3060012342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3300004c50 con 0x564f2e011990 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.607+0000 7f32f8c00640 1 -- 172.21.15.86:0/3060012342 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32fc0a51e0 con 0x564f2e011990 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.608+0000 7f3318766740 10 client.17061 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.608+0000 7f3318766740 1 -- 172.21.15.86:0/3060012342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f2df027f0 con 0x564f2e011990 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.608+0000 7f32f8c00640 1 -- 172.21.15.86:0/3060012342 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3300004c50 con 0x564f2e011990 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.608+0000 7f32f8c00640 1 -- 172.21.15.86:0/3060012342 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f32fc0d0070 con 0x564f2e011990 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.609+0000 7f3318766740 10 client.17061 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:17.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.609+0000 7f3318766740 10 client.17061 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:20:17.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.609+0000 7f3318766740 20 client.17061 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:17.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.609+0000 7f3318766740 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f2e026fd0 0x564f2e029470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.609+0000 7f3318766740 4 client.17061 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:17.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.609+0000 7f3318766740 1 -- 172.21.15.86:0/3060012342 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f2df84a40 con 0x564f2e026fd0 2024-04-17T07:20:17.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.610+0000 7f330ea00640 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f2e026fd0 0x564f2e029470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.611+0000 7f330ea00640 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f2e026fd0 0x564f2e029470 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=582f5bee39bcff7f server_cookie=b443465288715171 in_seq=0 out_seq=0 2024-04-17T07:20:17.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.611+0000 7f32f8c00640 10 client.17061 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:20:17.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.615+0000 7f32f8c00640 1 -- 172.21.15.86:0/3060012342 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f2df84a40 con 0x564f2e026fd0 2024-04-17T07:20:17.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.615+0000 7f32f8c00640 10 client.17061 handle_command_reply: tid=0 2024-04-17T07:20:17.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.615+0000 7f3318766740 1 client.17061 shutdown 2024-04-17T07:20:17.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.615+0000 7f3318766740 1 -- 172.21.15.86:0/3060012342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e011990 msgr2=0x564f2e01ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.615+0000 7f3318766740 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e011990 0x564f2e01ba80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0a4cb0 tx=0x7f32fc09c170 comp rx=0 tx=0).stop 2024-04-17T07:20:17.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.616+0000 7f3318766740 1 -- 172.21.15.86:0/3060012342 shutdown_connections 2024-04-17T07:20:17.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.616+0000 7f3318766740 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 0x564f2e022ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.616+0000 7f3318766740 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e011990 0x564f2e01ba80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.616+0000 7f3318766740 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f2e026fd0 0x564f2e029470 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.616+0000 7f3318766740 1 --2- 172.21.15.86:0/3060012342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e01b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.616+0000 7f3318766740 1 -- 172.21.15.86:0/3060012342 >> 172.21.15.86:0/3060012342 conn(0x564f2e00c510 msgr2=0x564f2e00b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.616+0000 7f32f8c00640 0 client.17061 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:20:17.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.616+0000 7f3318766740 1 -- 172.21.15.86:0/3060012342 shutdown_connections 2024-04-17T07:20:17.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.616+0000 7f3318766740 1 -- 172.21.15.86:0/3060012342 wait complete. 2024-04-17T07:20:17.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.617+0000 7f3318766740 20 client.17061 trim_cache size 0 max 16384 2024-04-17T07:20:17.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.641+0000 7f3318766740 1 Processor -- start 2024-04-17T07:20:17.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.641+0000 7f3318766740 1 -- start start 2024-04-17T07:20:17.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.641+0000 7f3318766740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e00fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.642+0000 7f3318766740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 0x564f2e022a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.642+0000 7f3318766740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e013560 0x564f2e02dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.642+0000 7f3318766740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33101b1100 con 0x564f2e016b00 2024-04-17T07:20:17.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.642+0000 7f3318766740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f33101b2c40 con 0x564f2e010fa0 2024-04-17T07:20:17.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.642+0000 7f3318766740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33101b17d0 con 0x564f2e013560 2024-04-17T07:20:17.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.642+0000 7f330f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e013560 0x564f2e02dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.642+0000 7f330e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 0x564f2e022a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.642+0000 7f330e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 0x564f2e022a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50420/0 (socket says 172.21.15.86:50420) 2024-04-17T07:20:17.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.642+0000 7f330e000640 1 -- 172.21.15.86:0/3196029063 learned_addr learned my addr 172.21.15.86:0/3196029063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.643+0000 7f330e000640 1 -- 172.21.15.86:0/3196029063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e013560 msgr2=0x564f2e02dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.643+0000 7f330e000640 1 --2- 172.21.15.86:0/3196029063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e013560 0x564f2e02dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.643+0000 7f330e000640 1 -- 172.21.15.86:0/3196029063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 msgr2=0x564f2e00fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.643+0000 7f330ea00640 1 --2- 172.21.15.86:0/3196029063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e00fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.643+0000 7f330e000640 1 --2- 172.21.15.86:0/3196029063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e00fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.643+0000 7f330e000640 1 -- 172.21.15.86:0/3196029063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2dc893f0 con 0x564f2e016b00 2024-04-17T07:20:17.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.643+0000 7f330e000640 1 --2- 172.21.15.86:0/3196029063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 0x564f2e022a80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0cdde0 tx=0x7f32fc09aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b431700364f32b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.644+0000 7f32fb400640 1 -- 172.21.15.86:0/3196029063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc0a0920 con 0x564f2e016b00 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.644+0000 7f32fb400640 1 -- 172.21.15.86:0/3196029063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32fc0042f0 con 0x564f2e016b00 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.644+0000 7f32fb400640 1 -- 172.21.15.86:0/3196029063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc003b10 con 0x564f2e016b00 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.644+0000 7f3318766740 1 -- 172.21.15.86:0/3196029063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 msgr2=0x564f2e022a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.644+0000 7f3318766740 1 --2- 172.21.15.86:0/3196029063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 0x564f2e022a80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0cdde0 tx=0x7f32fc09aa80 comp rx=0 tx=0).stop 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.645+0000 7f3318766740 1 -- 172.21.15.86:0/3196029063 shutdown_connections 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.645+0000 7f3318766740 1 --2- 172.21.15.86:0/3196029063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e013560 0x564f2e02dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.645+0000 7f3318766740 1 --2- 172.21.15.86:0/3196029063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e016b00 0x564f2e022a80 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.645+0000 7f3318766740 1 --2- 172.21.15.86:0/3196029063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e00fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.645+0000 7f3318766740 1 -- 172.21.15.86:0/3196029063 >> 172.21.15.86:0/3196029063 conn(0x564f2e00c510 msgr2=0x564f2e00f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.645+0000 7f3318766740 1 -- 172.21.15.86:0/3196029063 shutdown_connections 2024-04-17T07:20:17.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.645+0000 7f3318766740 1 -- 172.21.15.86:0/3196029063 wait complete. 2024-04-17T07:20:17.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.646+0000 7f3318766740 1 -- start start 2024-04-17T07:20:17.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.646+0000 7f3318766740 1 Processor -- start 2024-04-17T07:20:17.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.647+0000 7f3318766740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e022510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.647+0000 7f3318766740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e013560 0x564f2e022a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.647+0000 7f3318766740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 0x564f2e020b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.647+0000 7f3318766740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33101b2570 con 0x564f2e013560 2024-04-17T07:20:17.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.647+0000 7f3318766740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f33101b1ea0 con 0x564f2e010fa0 2024-04-17T07:20:17.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.647+0000 7f3318766740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33101af0d0 con 0x564f2e016b00 2024-04-17T07:20:17.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.647+0000 7f3318766740 10 client.0 _ll_register_callbacks cb 0x7f32fc098900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:17.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.648+0000 7f330f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 0x564f2e020b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.648+0000 7f330e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e013560 0x564f2e022a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.648+0000 7f330f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 0x564f2e020b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41162/0 (socket says 172.21.15.86:41162) 2024-04-17T07:20:17.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.648+0000 7f330f400640 1 -- 172.21.15.86:0/1296152191 learned_addr learned my addr 172.21.15.86:0/1296152191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.649+0000 7f330e000640 1 -- 172.21.15.86:0/1296152191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 msgr2=0x564f2e020b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.649+0000 7f330e000640 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 0x564f2e020b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.649+0000 7f330e000640 1 -- 172.21.15.86:0/1296152191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 msgr2=0x564f2e022510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.649+0000 7f330e000640 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e022510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.649+0000 7f330e000640 1 -- 172.21.15.86:0/1296152191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f2dd537d0 con 0x564f2e013560 2024-04-17T07:20:17.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.649+0000 7f330e000640 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e013560 0x564f2e022a50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x564f2e011f20 tx=0x7f32fc0039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbec7d66d6c4f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.649+0000 7f32f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:20:17.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.649+0000 7f32f8c00640 1 -- 172.21.15.86:0/1296152191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc0d0460 con 0x564f2e013560 2024-04-17T07:20:17.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.649+0000 7f32f8c00640 1 -- 172.21.15.86:0/1296152191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f32fc0bb310 con 0x564f2e013560 2024-04-17T07:20:17.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.649+0000 7f3318766740 1 -- 172.21.15.86:0/1296152191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32fc0042f0 con 0x564f2e013560 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.650+0000 7f32f8c00640 1 -- 172.21.15.86:0/1296152191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32fc0bb510 con 0x564f2e013560 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.650+0000 7f3318766740 10 client.17067 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.650+0000 7f3318766740 1 -- 172.21.15.86:0/1296152191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f2e08ded0 con 0x564f2e013560 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.650+0000 7f32f8c00640 1 -- 172.21.15.86:0/1296152191 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32fc0042f0 con 0x564f2e013560 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.650+0000 7f32f8c00640 1 -- 172.21.15.86:0/1296152191 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f32fc0d0040 con 0x564f2e013560 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.650+0000 7f3318766740 10 client.17067 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.650+0000 7f3318766740 10 client.17067 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.650+0000 7f3318766740 20 client.17067 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.650+0000 7f3318766740 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f2e08e620 0x564f2e090a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.651+0000 7f3318766740 4 client.17067 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:20:17.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.651+0000 7f3318766740 1 -- 172.21.15.86:0/1296152191 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f2df7e8b0 con 0x564f2e08e620 2024-04-17T07:20:17.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.651+0000 7f330ea00640 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f2e08e620 0x564f2e090a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.653+0000 7f330ea00640 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f2e08e620 0x564f2e090a40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd7814943741c68f server_cookie=b209cef67bc0ba91 in_seq=0 out_seq=0 2024-04-17T07:20:17.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.653+0000 7f32f8c00640 10 client.17067 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:20:17.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.656+0000 7f32f8c00640 1 -- 172.21.15.86:0/1296152191 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f2df7e8b0 con 0x564f2e08e620 2024-04-17T07:20:17.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.657+0000 7f32f8c00640 10 client.17067 handle_command_reply: tid=0 2024-04-17T07:20:17.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.657+0000 7f3318766740 1 client.17067 shutdown 2024-04-17T07:20:17.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.658+0000 7f3318766740 1 -- 172.21.15.86:0/1296152191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e013560 msgr2=0x564f2e022a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.658+0000 7f3318766740 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e013560 0x564f2e022a50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x564f2e011f20 tx=0x7f32fc0039b0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.658+0000 7f3318766740 1 -- 172.21.15.86:0/1296152191 shutdown_connections 2024-04-17T07:20:17.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.658+0000 7f3318766740 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f2e016b00 0x564f2e020b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.658+0000 7f3318766740 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f2e013560 0x564f2e022a50 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.658+0000 7f3318766740 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f2e08e620 0x564f2e090a40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.658+0000 7f3318766740 1 --2- 172.21.15.86:0/1296152191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f2e010fa0 0x564f2e022510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.658+0000 7f3318766740 1 -- 172.21.15.86:0/1296152191 >> 172.21.15.86:0/1296152191 conn(0x564f2e00c510 msgr2=0x564f2e010000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.658+0000 7f32f8c00640 0 client.17067 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:20:17.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.659+0000 7f3318766740 1 -- 172.21.15.86:0/1296152191 shutdown_connections 2024-04-17T07:20:17.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.659+0000 7f3318766740 1 -- 172.21.15.86:0/1296152191 wait complete. 2024-04-17T07:20:17.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.660+0000 7f3318766740 20 client.17067 trim_cache size 0 max 16384 2024-04-17T07:20:17.661 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:20:17.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.662+0000 7f3315a00640 1 -- 172.21.15.86:0/4180379693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32f40a3c30 msgr2=0x7f32f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.662+0000 7f3315a00640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32f40a3c30 0x7f32f40a60f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x564f2e0121f0 tx=0x7f3308009500 comp rx=0 tx=0).stop 2024-04-17T07:20:17.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.662+0000 7f3315a00640 1 -- 172.21.15.86:0/4180379693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101031a0 msgr2=0x7f331019ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.662+0000 7f3315a00640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101031a0 0x7f331019ec90 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009ad0 tx=0x7f32fc009f90 comp rx=0 tx=0).stop 2024-04-17T07:20:17.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.662+0000 7f3315a00640 1 -- 172.21.15.86:0/4180379693 shutdown_connections 2024-04-17T07:20:17.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.662+0000 7f3315a00640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32f40a3c30 0x7f32f40a60f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.662+0000 7f3315a00640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3310103b60 0x7f33101a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.662+0000 7f3315a00640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101031a0 0x7f331019ec90 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.662+0000 7f3315a00640 1 --2- 172.21.15.86:0/4180379693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3310101fb0 0x7f331019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.662+0000 7f3315a00640 1 -- 172.21.15.86:0/4180379693 >> 172.21.15.86:0/4180379693 conn(0x7f33100fd540 msgr2=0x7f3310106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.663+0000 7f3315a00640 1 -- 172.21.15.86:0/4180379693 shutdown_connections 2024-04-17T07:20:17.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.663+0000 7f3315a00640 1 -- 172.21.15.86:0/4180379693 wait complete. 2024-04-17T07:20:17.680 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:20:17.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.905+0000 7f7ecec00640 1 -- 172.21.15.86:0/2220308751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec81029b0 msgr2=0x7f7ec81092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.905+0000 7f7ecec00640 1 --2- 172.21.15.86:0/2220308751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec81029b0 0x7f7ec81092d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009ca0 tx=0x7f7ec002c100 comp rx=0 tx=0).stop 2024-04-17T07:20:17.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.907+0000 7f7ecec00640 1 -- 172.21.15.86:0/2220308751 shutdown_connections 2024-04-17T07:20:17.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.907+0000 7f7ecec00640 1 --2- 172.21.15.86:0/2220308751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec81029b0 0x7f7ec81092d0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.907+0000 7f7ecec00640 1 --2- 172.21.15.86:0/2220308751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec8101ff0 0x7f7ec8102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.907+0000 7f7ecec00640 1 --2- 172.21.15.86:0/2220308751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec8100ea0 0x7f7ec81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.907+0000 7f7ecec00640 1 -- 172.21.15.86:0/2220308751 >> 172.21.15.86:0/2220308751 conn(0x7f7ec80fc410 msgr2=0x7f7ec80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.908+0000 7f7ecec00640 1 -- 172.21.15.86:0/2220308751 shutdown_connections 2024-04-17T07:20:17.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.908+0000 7f7ecec00640 1 -- 172.21.15.86:0/2220308751 wait complete. 2024-04-17T07:20:17.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.909+0000 7f7ecec00640 1 Processor -- start 2024-04-17T07:20:17.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.910+0000 7f7ecec00640 1 -- start start 2024-04-17T07:20:17.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.910+0000 7f7ecec00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec8100ea0 0x7f7ec810ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.910+0000 7f7ecec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec8101ff0 0x7f7ec8110500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.910+0000 7f7ecec00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec81029b0 0x7f7ec8111540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.910+0000 7f7ecec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec810a6b0 con 0x7f7ec8101ff0 2024-04-17T07:20:17.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.910+0000 7f7ecec00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec810a530 con 0x7f7ec8100ea0 2024-04-17T07:20:17.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.910+0000 7f7ecec00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec810a830 con 0x7f7ec81029b0 2024-04-17T07:20:17.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.911+0000 7f7eccc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec81029b0 0x7f7ec8111540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.911+0000 7f7eccc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec81029b0 0x7f7ec8111540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41178/0 (socket says 172.21.15.86:41178) 2024-04-17T07:20:17.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.911+0000 7f7eccc00640 1 -- 172.21.15.86:0/1895303681 learned_addr learned my addr 172.21.15.86:0/1895303681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.911+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec8100ea0 0x7f7ec810ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.911+0000 7f7eccc00640 1 -- 172.21.15.86:0/1895303681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec8100ea0 msgr2=0x7f7ec810ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.911+0000 7f7eccc00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec8100ea0 0x7f7ec810ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.912+0000 7f7eccc00640 1 -- 172.21.15.86:0/1895303681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec8101ff0 msgr2=0x7f7ec8110500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:20:17.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.912+0000 7f7eccc00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec8101ff0 0x7f7ec8110500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.912+0000 7f7eccc00640 1 -- 172.21.15.86:0/1895303681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec8111bb0 con 0x7f7ec81029b0 2024-04-17T07:20:17.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.912+0000 7f7eccc00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec81029b0 0x7f7ec8111540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0037040 tx=0x7f7ec0003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.913+0000 7f7ec4c00640 1 -- 172.21.15.86:0/1895303681 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0039040 con 0x7f7ec81029b0 2024-04-17T07:20:17.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.913+0000 7f7ec4c00640 1 -- 172.21.15.86:0/1895303681 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ec0044070 con 0x7f7ec81029b0 2024-04-17T07:20:17.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.913+0000 7f7ec4c00640 1 -- 172.21.15.86:0/1895303681 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec00043b0 con 0x7f7ec81029b0 2024-04-17T07:20:17.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.913+0000 7f7ecec00640 1 -- 172.21.15.86:0/1895303681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec8111e40 con 0x7f7ec81029b0 2024-04-17T07:20:17.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.913+0000 7f7ecec00640 1 -- 172.21.15.86:0/1895303681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec81122e0 con 0x7f7ec81029b0 2024-04-17T07:20:17.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.919+0000 7f7ec4c00640 1 -- 172.21.15.86:0/1895303681 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7ec0004550 con 0x7f7ec81029b0 2024-04-17T07:20:17.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.921+0000 7f7ec4c00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7eac0a3c20 0x7f7eac0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.921+0000 7f7ec4c00640 1 -- 172.21.15.86:0/1895303681 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7ec00f2d20 con 0x7f7ec81029b0 2024-04-17T07:20:17.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.922+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7eac0a3c20 0x7f7eac0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.923+0000 7f7ed1a3a740 1 Processor -- start 2024-04-17T07:20:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.924+0000 7f7ed1a3a740 1 -- start start 2024-04-17T07:20:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.924+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dc53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.924+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dc5990 0x55c6e5dca490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.924+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dcab00 0x55c6e5dccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.924+0000 7f7ed1a3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6e5d2b020 con 0x55c6e5dc5990 2024-04-17T07:20:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.924+0000 7f7ed1a3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c6e5cd1460 con 0x55c6e5dcab00 2024-04-17T07:20:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.924+0000 7f7ed1a3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c6e5d26fe0 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.925+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7eac0a3c20 0x7f7eac0a60e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8005fd0 tx=0x7f7eb8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.925+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dc53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.925+0000 7f7eccc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dcab00 0x55c6e5dccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.925+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dc53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41190/0 (socket says 172.21.15.86:41190) 2024-04-17T07:20:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.925+0000 7f7ec7e00640 1 -- 172.21.15.86:0/2509521026 learned_addr learned my addr 172.21.15.86:0/2509521026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.925+0000 7f7ec7400640 1 --2- 172.21.15.86:0/2509521026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dc5990 0x55c6e5dca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.926+0000 7f7ec7e00640 1 -- 172.21.15.86:0/2509521026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dcab00 msgr2=0x55c6e5dccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.926+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/2509521026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dcab00 0x55c6e5dccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.926+0000 7f7ec7e00640 1 -- 172.21.15.86:0/2509521026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dc5990 msgr2=0x55c6e5dca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.926+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/2509521026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dc5990 0x55c6e5dca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.926+0000 7f7ec7e00640 1 -- 172.21.15.86:0/2509521026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e5cd9040 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.926+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/2509521026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dc53c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800edf0 tx=0x7f7eb8011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9588b736af487fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.927+0000 7f7ebd200640 1 -- 172.21.15.86:0/2509521026 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb801e440 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.927+0000 7f7ebd200640 1 -- 172.21.15.86:0/2509521026 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eb8023070 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.927+0000 7f7ebd200640 1 -- 172.21.15.86:0/2509521026 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8010930 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.928+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/2509521026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc4fa0 msgr2=0x55c6e5dc53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.928+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/2509521026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dc53c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800edf0 tx=0x7f7eb8011000 comp rx=0 tx=0).stop 2024-04-17T07:20:17.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.928+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/2509521026 shutdown_connections 2024-04-17T07:20:17.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.928+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/2509521026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dcab00 0x55c6e5dccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.928+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/2509521026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dc5990 0x55c6e5dca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.928+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/2509521026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dc53c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.928+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/2509521026 >> 172.21.15.86:0/2509521026 conn(0x55c6e5dc0510 msgr2=0x55c6e5dc0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.929+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/2509521026 shutdown_connections 2024-04-17T07:20:17.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.929+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/2509521026 wait complete. 2024-04-17T07:20:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.930+0000 7f7ed1a3a740 1 -- start start 2024-04-17T07:20:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.931+0000 7f7ed1a3a740 1 Processor -- start 2024-04-17T07:20:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.931+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dcf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.931+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dc5990 0x55c6e5dcfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.932+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.932+0000 7f7ed1a3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6e5dcda50 con 0x55c6e5dc5990 2024-04-17T07:20:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.932+0000 7f7ed1a3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c6e5ce4830 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.932+0000 7f7ed1a3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c6e5dcdbd0 con 0x55c6e5dcab00 2024-04-17T07:20:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.932+0000 7f7ed1a3a740 10 client.0 _ll_register_callbacks cb 0x55c6e5cb2fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:17.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.932+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dcf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.932+0000 7f7ec7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dc5990 0x55c6e5dcfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.932+0000 7f7eccc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.932+0000 7f7eccc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41202/0 (socket says 172.21.15.86:41202) 2024-04-17T07:20:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.932+0000 7f7eccc00640 1 -- 172.21.15.86:0/3972602385 learned_addr learned my addr 172.21.15.86:0/3972602385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.933+0000 7f7eccc00640 1 -- 172.21.15.86:0/3972602385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 msgr2=0x55c6e5dcf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.933+0000 7f7eccc00640 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dcf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.933+0000 7f7eccc00640 1 -- 172.21.15.86:0/3972602385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dc5990 msgr2=0x55c6e5dcfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.933+0000 7f7eccc00640 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dc5990 0x55c6e5dcfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.933+0000 7f7eccc00640 1 -- 172.21.15.86:0/3972602385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e5ce8920 con 0x55c6e5dcab00 2024-04-17T07:20:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.933+0000 7f7eccc00640 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd6ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009620 tx=0x7f7ec0004900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f097e775475dd164 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.933+0000 7f7eb1600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.933+0000 7f7eb1600640 1 -- 172.21.15.86:0/3972602385 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec00335a0 con 0x55c6e5dcab00 2024-04-17T07:20:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.933+0000 7f7eb1600640 1 -- 172.21.15.86:0/3972602385 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7ec00beee0 con 0x55c6e5dcab00 2024-04-17T07:20:17.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.933+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/3972602385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb8023070 con 0x55c6e5dcab00 2024-04-17T07:20:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.934+0000 7f7eb1600640 1 -- 172.21.15.86:0/3972602385 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ec00bec80 con 0x55c6e5dcab00 2024-04-17T07:20:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.935+0000 7f7ed1a3a740 10 client.26086 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.935+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/3972602385 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6e5cb67f0 con 0x55c6e5dcab00 2024-04-17T07:20:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.935+0000 7f7eb1600640 1 -- 172.21.15.86:0/3972602385 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eb8023070 con 0x55c6e5dcab00 2024-04-17T07:20:17.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.935+0000 7f7eb1600640 1 -- 172.21.15.86:0/3972602385 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7ec003c030 con 0x55c6e5dcab00 2024-04-17T07:20:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.936+0000 7f7ed1a3a740 10 client.26086 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.936+0000 7f7ed1a3a740 10 client.26086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.936+0000 7f7ed1a3a740 20 client.26086 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.936+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6e5ddb060 0x55c6e5ddd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.936+0000 7f7ed1a3a740 4 client.26086 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:17.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.936+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/3972602385 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6e5d38a40 con 0x55c6e5ddb060 2024-04-17T07:20:17.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.937+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6e5ddb060 0x55c6e5ddd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.939+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6e5ddb060 0x55c6e5ddd490 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f3e0c5b2856339 server_cookie=9909e95bcf49f11e in_seq=0 out_seq=0 2024-04-17T07:20:17.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.939+0000 7f7eb1600640 10 client.26086 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:17.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.941+0000 7f7eb1600640 1 -- 172.21.15.86:0/3972602385 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6e5d38a40 con 0x55c6e5ddb060 2024-04-17T07:20:17.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.941+0000 7f7eb1600640 10 client.26086 handle_command_reply: tid=0 2024-04-17T07:20:17.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.942+0000 7f7ed1a3a740 1 client.26086 shutdown 2024-04-17T07:20:17.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.943+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/3972602385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dcab00 msgr2=0x55c6e5dd6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.943+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd6ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009620 tx=0x7f7ec0004900 comp rx=0 tx=0).stop 2024-04-17T07:20:17.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.943+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/3972602385 shutdown_connections 2024-04-17T07:20:17.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.943+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6e5ddb060 0x55c6e5ddd490 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.943+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd6ad0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.943+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dc5990 0x55c6e5dcfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.943+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/3972602385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dcf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.943+0000 7f7eb1600640 0 client.26086 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:17.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.943+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/3972602385 >> 172.21.15.86:0/3972602385 conn(0x55c6e5dc0510 msgr2=0x55c6e5dbf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.943+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/3972602385 shutdown_connections 2024-04-17T07:20:17.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.944+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/3972602385 wait complete. 2024-04-17T07:20:17.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.944+0000 7f7ed1a3a740 20 client.26086 trim_cache size 0 max 16384 2024-04-17T07:20:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.971+0000 7f7ed1a3a740 1 Processor -- start 2024-04-17T07:20:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.972+0000 7f7ed1a3a740 1 -- start start 2024-04-17T07:20:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.972+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dcf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.972+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 0x55c6e5dcfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.972+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.972+0000 7f7ed1a3a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81b9800 con 0x55c6e5dcab00 2024-04-17T07:20:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.972+0000 7f7ed1a3a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81bb340 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.972+0000 7f7ed1a3a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81b9ed0 con 0x55c6e5dc5990 2024-04-17T07:20:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.973+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dcf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.973+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dcf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:56036/0 (socket says 172.21.15.86:56036) 2024-04-17T07:20:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.973+0000 7f7ec7e00640 1 -- 172.21.15.86:0/1581842046 learned_addr learned my addr 172.21.15.86:0/1581842046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.973+0000 7f7eccc00640 1 --2- 172.21.15.86:0/1581842046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.974+0000 7f7ec7400640 1 --2- 172.21.15.86:0/1581842046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 0x55c6e5dcfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.974+0000 7f7ec7e00640 1 -- 172.21.15.86:0/1581842046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 msgr2=0x55c6e5dcfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.974+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/1581842046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 0x55c6e5dcfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.974+0000 7f7ec7e00640 1 -- 172.21.15.86:0/1581842046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dcab00 msgr2=0x55c6e5dd6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.974+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/1581842046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.974+0000 7f7ec7e00640 1 -- 172.21.15.86:0/1581842046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e5a3d3f0 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.974+0000 7f7ec7400640 1 --2- 172.21.15.86:0/1581842046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 0x55c6e5dcfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.975+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/1581842046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dcf550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8011cd0 tx=0x7f7eb8011f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94582dcf80f0d030 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.976+0000 7f7eb3e00640 1 -- 172.21.15.86:0/1581842046 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8034620 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.976+0000 7f7eb3e00640 1 -- 172.21.15.86:0/1581842046 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eb803a070 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.977+0000 7f7eb3e00640 1 -- 172.21.15.86:0/1581842046 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80048c0 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.978+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1581842046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 msgr2=0x55c6e5dcf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.978+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/1581842046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dcf550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8011cd0 tx=0x7f7eb8011f80 comp rx=0 tx=0).stop 2024-04-17T07:20:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.979+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1581842046 shutdown_connections 2024-04-17T07:20:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.979+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/1581842046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.979+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/1581842046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 0x55c6e5dcfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.979+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/1581842046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dcf550 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.979+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1581842046 >> 172.21.15.86:0/1581842046 conn(0x55c6e5dc0510 msgr2=0x55c6e5dc7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.979+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1581842046 shutdown_connections 2024-04-17T07:20:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.979+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1581842046 wait complete. 2024-04-17T07:20:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.980+0000 7f7ed1a3a740 1 -- start start 2024-04-17T07:20:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.980+0000 7f7ed1a3a740 1 Processor -- start 2024-04-17T07:20:17.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.981+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dd1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.982+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 0x55c6e5dd2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.982+0000 7f7ed1a3a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.983+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dd1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.983+0000 7f7ec7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dd1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:56046/0 (socket says 172.21.15.86:56046) 2024-04-17T07:20:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.983+0000 7f7ec7e00640 1 -- 172.21.15.86:0/1160163692 learned_addr learned my addr 172.21.15.86:0/1160163692 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.983+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81bac70 con 0x55c6e5dcab00 2024-04-17T07:20:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.983+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81ba5a0 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.983+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81b77d0 con 0x55c6e5dc5990 2024-04-17T07:20:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.983+0000 7f7ed1a3a740 10 client.0 _ll_register_callbacks cb 0x7f7ec00be850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.984+0000 7f7ec7400640 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 0x55c6e5dd2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.984+0000 7f7eccc00640 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.985+0000 7f7ec7e00640 1 -- 172.21.15.86:0/1160163692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 msgr2=0x55c6e5dd2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.985+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 0x55c6e5dd2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.985+0000 7f7ec7e00640 1 -- 172.21.15.86:0/1160163692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dcab00 msgr2=0x55c6e5dd5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.985+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.985+0000 7f7ec7e00640 1 -- 172.21.15.86:0/1160163692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e5b077d0 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.985+0000 7f7ec7e00640 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dd1b30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8005fa0 tx=0x7f7eb800ea00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8d92745ce15683 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.985+0000 7f7eb1600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:20:17.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.986+0000 7f7eb1600640 1 -- 172.21.15.86:0/1160163692 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8005140 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.986+0000 7f7eb1600640 1 -- 172.21.15.86:0/1160163692 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7eb80052e0 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.986+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb803a070 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.987+0000 7f7eb1600640 1 -- 172.21.15.86:0/1160163692 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7eb804e050 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.987+0000 7f7ed1a3a740 10 client.36062 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.987+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6e5dd6720 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.987+0000 7f7eb1600640 1 -- 172.21.15.86:0/1160163692 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eb8023040 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.988+0000 7f7eb1600640 1 -- 172.21.15.86:0/1160163692 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7eb8031070 con 0x55c6e5dc4fa0 2024-04-17T07:20:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.988+0000 7f7ed1a3a740 10 client.36062 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.988+0000 7f7ed1a3a740 10 client.36062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.988+0000 7f7ed1a3a740 20 client.36062 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.988+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6e5e525f0 0x55c6e5e549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.988+0000 7f7ed1a3a740 4 client.36062 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:20:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.988+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7eb803a070 con 0x55c6e5e525f0 2024-04-17T07:20:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.989+0000 7f7ec7400640 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6e5e525f0 0x55c6e5e549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.991+0000 7f7eb1600640 10 client.36062 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.991+0000 7f7ec7400640 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6e5e525f0 0x55c6e5e549e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57b3dd68644bda44 server_cookie=f91a85c9142458c1 in_seq=0 out_seq=0 2024-04-17T07:20:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.992+0000 7f7eb1600640 1 -- 172.21.15.86:0/1160163692 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7eb803a070 con 0x55c6e5e525f0 2024-04-17T07:20:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.992+0000 7f7eb1600640 10 client.36062 handle_command_reply: tid=0 2024-04-17T07:20:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.992+0000 7f7ed1a3a740 1 client.36062 shutdown 2024-04-17T07:20:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.992+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 msgr2=0x55c6e5dd1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.992+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dd1b30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8005fa0 tx=0x7f7eb800ea00 comp rx=0 tx=0).stop 2024-04-17T07:20:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.993+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 shutdown_connections 2024-04-17T07:20:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.993+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6e5e525f0 0x55c6e5e549e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.993+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e5dcab00 0x55c6e5dd5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.993+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6e5dc5990 0x55c6e5dd2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.993+0000 7f7ed1a3a740 1 --2- 172.21.15.86:0/1160163692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6e5dc4fa0 0x55c6e5dd1b30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.993+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 >> 172.21.15.86:0/1160163692 conn(0x55c6e5dc0510 msgr2=0x55c6e5dcf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.993+0000 7f7eb1600640 0 client.36062 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.993+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 shutdown_connections 2024-04-17T07:20:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.993+0000 7f7ed1a3a740 1 -- 172.21.15.86:0/1160163692 wait complete. 2024-04-17T07:20:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.994+0000 7f7ed1a3a740 20 client.36062 trim_cache size 0 max 16384 2024-04-17T07:20:17.994 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:20:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.997+0000 7f7ecec00640 1 -- 172.21.15.86:0/1895303681 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7eac0a3c20 msgr2=0x7f7eac0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.997+0000 7f7ecec00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7eac0a3c20 0x7f7eac0a60e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8005fd0 tx=0x7f7eb8005850 comp rx=0 tx=0).stop 2024-04-17T07:20:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.997+0000 7f7ecec00640 1 -- 172.21.15.86:0/1895303681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec81029b0 msgr2=0x7f7ec8111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.997+0000 7f7ecec00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec81029b0 0x7f7ec8111540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0037040 tx=0x7f7ec0003ba0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.997+0000 7f7ecec00640 1 -- 172.21.15.86:0/1895303681 shutdown_connections 2024-04-17T07:20:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.997+0000 7f7ecec00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7eac0a3c20 0x7f7eac0a60e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.997+0000 7f7ecec00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec81029b0 0x7f7ec8111540 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.997+0000 7f7ecec00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec8101ff0 0x7f7ec8110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.997+0000 7f7ecec00640 1 --2- 172.21.15.86:0/1895303681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec8100ea0 0x7f7ec810ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:17.997+0000 7f7ecec00640 1 -- 172.21.15.86:0/1895303681 >> 172.21.15.86:0/1895303681 conn(0x7f7ec80fc410 msgr2=0x7f7ec8105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:18.000+0000 7f7ecec00640 1 -- 172.21.15.86:0/1895303681 shutdown_connections 2024-04-17T07:20:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:18.000+0000 7f7ecec00640 1 -- 172.21.15.86:0/1895303681 wait complete. 2024-04-17T07:20:19.035 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:20:19.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:20:19.035 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:20:19.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:19 smithi086 ceph-mon[31733]: pgmap v618: 129 pgs: 129 active+clean; 955 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.4 MiB/s wr, 841 op/s 2024-04-17T07:20:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 55.0 s, 560.6 tps, lat 178.307 ms stddev 71.955 2024-04-17T07:20:19.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.366+0000 7fcba1000640 1 -- 172.21.15.86:0/3187266113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb9c10e650 msgr2=0x7fcb9c110b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.366+0000 7fcba1000640 1 --2- 172.21.15.86:0/3187266113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb9c10e650 0x7fcb9c110b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009be0 tx=0x7fcb9402bef0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.366+0000 7fcba1000640 1 -- 172.21.15.86:0/3187266113 shutdown_connections 2024-04-17T07:20:19.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.366+0000 7fcba1000640 1 --2- 172.21.15.86:0/3187266113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb9c10e650 0x7fcb9c110b10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.366+0000 7fcba1000640 1 --2- 172.21.15.86:0/3187266113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb9c10bb90 0x7fcb9c10dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.366+0000 7fcba1000640 1 --2- 172.21.15.86:0/3187266113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb9c1091d0 0x7fcb9c10b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.366+0000 7fcba1000640 1 -- 172.21.15.86:0/3187266113 >> 172.21.15.86:0/3187266113 conn(0x7fcb9c06f190 msgr2=0x7fcb9c0715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.367+0000 7fcba1000640 1 -- 172.21.15.86:0/3187266113 shutdown_connections 2024-04-17T07:20:19.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.367+0000 7fcba1000640 1 -- 172.21.15.86:0/3187266113 wait complete. 2024-04-17T07:20:19.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.369+0000 7fcba1000640 1 Processor -- start 2024-04-17T07:20:19.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.369+0000 7fcba1000640 1 -- start start 2024-04-17T07:20:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.370+0000 7fcba1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb9c1091d0 0x7fcb9c1a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.370+0000 7fcba1000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb9c10bb90 0x7fcb9c1a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.370+0000 7fcba1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb9c10e650 0x7fcb9c1aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.370+0000 7fcba1000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1140e0 con 0x7fcb9c1091d0 2024-04-17T07:20:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.370+0000 7fcba1000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c113f60 con 0x7fcb9c10bb90 2024-04-17T07:20:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.370+0000 7fcba1000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c114260 con 0x7fcb9c10e650 2024-04-17T07:20:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.370+0000 7fcb99e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb9c1091d0 0x7fcb9c1a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.370+0000 7fcb99e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb9c1091d0 0x7fcb9c1a2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50490/0 (socket says 172.21.15.86:50490) 2024-04-17T07:20:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.370+0000 7fcb99e00640 1 -- 172.21.15.86:0/1409182364 learned_addr learned my addr 172.21.15.86:0/1409182364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:19.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.371+0000 7fcb9a800640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb9c10e650 0x7fcb9c1aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.371+0000 7fcb99400640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb9c10bb90 0x7fcb9c1a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.371+0000 7fcb99e00640 1 -- 172.21.15.86:0/1409182364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb9c10e650 msgr2=0x7fcb9c1aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.371+0000 7fcb99e00640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb9c10e650 0x7fcb9c1aa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.371+0000 7fcb99e00640 1 -- 172.21.15.86:0/1409182364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb9c10bb90 msgr2=0x7fcb9c1a3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.371+0000 7fcb99e00640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb9c10bb90 0x7fcb9c1a3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.371+0000 7fcb99e00640 1 -- 172.21.15.86:0/1409182364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c1aa810 con 0x7fcb9c1091d0 2024-04-17T07:20:19.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.372+0000 7fcb99e00640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb9c1091d0 0x7fcb9c1a2bf0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c007ba0 tx=0x7fcb8c00d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.374+0000 7fcb92a00640 1 -- 172.21.15.86:0/1409182364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c017020 con 0x7fcb9c1091d0 2024-04-17T07:20:19.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.374+0000 7fcb92a00640 1 -- 172.21.15.86:0/1409182364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb8c013400 con 0x7fcb9c1091d0 2024-04-17T07:20:19.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.374+0000 7fcba1000640 1 -- 172.21.15.86:0/1409182364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c199820 con 0x7fcb9c1091d0 2024-04-17T07:20:19.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.374+0000 7fcb92a00640 1 -- 172.21.15.86:0/1409182364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c0125c0 con 0x7fcb9c1091d0 2024-04-17T07:20:19.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.374+0000 7fcba1000640 1 -- 172.21.15.86:0/1409182364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c199d70 con 0x7fcb9c1091d0 2024-04-17T07:20:19.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.380+0000 7fcb92a00640 1 -- 172.21.15.86:0/1409182364 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fcb8c01a020 con 0x7fcb9c1091d0 2024-04-17T07:20:19.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.381+0000 7fcba3da6740 1 Processor -- start 2024-04-17T07:20:19.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.382+0000 7fcba3da6740 1 -- start start 2024-04-17T07:20:19.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.382+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f7eec0 0x555f64f7f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.382+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 0x555f64f843b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.382+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f86eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.382+0000 7fcba3da6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f64ee4cb0 con 0x555f64f84a20 2024-04-17T07:20:19.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.382+0000 7fcba3da6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555f64e8b370 con 0x555f64f7eec0 2024-04-17T07:20:19.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.382+0000 7fcba3da6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555f64ee0c70 con 0x555f64f7f8b0 2024-04-17T07:20:19.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.382+0000 7fcb92a00640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb800a3db0 0x7fcb800a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.383+0000 7fcb92a00640 1 -- 172.21.15.86:0/1409182364 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcb8c0d1440 con 0x7fcb9c1091d0 2024-04-17T07:20:19.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.383+0000 7fcb99e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 0x555f64f843b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.383+0000 7fcb99e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 0x555f64f843b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41230/0 (socket says 172.21.15.86:41230) 2024-04-17T07:20:19.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.383+0000 7fcb99e00640 1 -- 172.21.15.86:0/1027326519 learned_addr learned my addr 172.21.15.86:0/1027326519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:19.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.383+0000 7fcb99400640 1 --2- 172.21.15.86:0/1027326519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f86eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.383+0000 7fcb9a800640 1 --2- 172.21.15.86:0/1027326519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f7eec0 0x555f64f7f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.383+0000 7fcb9a800640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb800a3db0 0x7fcb800a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.384+0000 7fcb99400640 1 -- 172.21.15.86:0/1027326519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 msgr2=0x555f64f843b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.384+0000 7fcb99400640 1 --2- 172.21.15.86:0/1027326519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 0x555f64f843b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.384+0000 7fcb99400640 1 -- 172.21.15.86:0/1027326519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f7eec0 msgr2=0x555f64f7f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.384+0000 7fcb99400640 1 --2- 172.21.15.86:0/1027326519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f7eec0 0x555f64f7f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.384+0000 7fcb99400640 1 -- 172.21.15.86:0/1027326519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f64eaa910 con 0x555f64f84a20 2024-04-17T07:20:19.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.385+0000 7fcb99400640 1 --2- 172.21.15.86:0/1027326519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f86eb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0740d0 tx=0x7fcb880113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dfd222633bb7d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.386+0000 7fcb9a800640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb800a3db0 0x7fcb800a6270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcb94034fd0 tx=0x7fcb94005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.387+0000 7fcb87400640 1 -- 172.21.15.86:0/1027326519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88011de0 con 0x555f64f84a20 2024-04-17T07:20:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.387+0000 7fcb87400640 1 -- 172.21.15.86:0/1027326519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb88012900 con 0x555f64f84a20 2024-04-17T07:20:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.387+0000 7fcb87400640 1 -- 172.21.15.86:0/1027326519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8800f8a0 con 0x555f64f84a20 2024-04-17T07:20:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.388+0000 7fcba3da6740 1 -- 172.21.15.86:0/1027326519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 msgr2=0x555f64f86eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.388+0000 7fcba3da6740 1 --2- 172.21.15.86:0/1027326519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f86eb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0740d0 tx=0x7fcb880113e0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.389+0000 7fcba3da6740 1 -- 172.21.15.86:0/1027326519 shutdown_connections 2024-04-17T07:20:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.389+0000 7fcba3da6740 1 --2- 172.21.15.86:0/1027326519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f86eb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.389+0000 7fcba3da6740 1 --2- 172.21.15.86:0/1027326519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 0x555f64f843b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.389+0000 7fcba3da6740 1 --2- 172.21.15.86:0/1027326519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f7eec0 0x555f64f7f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.389+0000 7fcba3da6740 1 -- 172.21.15.86:0/1027326519 >> 172.21.15.86:0/1027326519 conn(0x555f64f7a430 msgr2=0x555f64f7a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.389+0000 7fcba3da6740 1 -- 172.21.15.86:0/1027326519 shutdown_connections 2024-04-17T07:20:19.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.389+0000 7fcba3da6740 1 -- 172.21.15.86:0/1027326519 wait complete. 2024-04-17T07:20:19.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.390+0000 7fcba3da6740 1 -- start start 2024-04-17T07:20:19.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.390+0000 7fcba3da6740 1 Processor -- start 2024-04-17T07:20:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.391+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f89460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.391+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.391+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f84a20 0x555f64f909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.391+0000 7fcba3da6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f64f879d0 con 0x555f64f7f8b0 2024-04-17T07:20:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.391+0000 7fcba3da6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555f64e9e730 con 0x555f64f84a20 2024-04-17T07:20:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.391+0000 7fcba3da6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555f64f87b50 con 0x555f64f7eec0 2024-04-17T07:20:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.391+0000 7fcba3da6740 10 client.0 _ll_register_callbacks cb 0x555f64ea0b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.392+0000 7fcb99e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f89460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.392+0000 7fcb99e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f89460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41240/0 (socket says 172.21.15.86:41240) 2024-04-17T07:20:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.392+0000 7fcb99e00640 1 -- 172.21.15.86:0/116859820 learned_addr learned my addr 172.21.15.86:0/116859820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.392+0000 7fcb99400640 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.392+0000 7fcb9a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f84a20 0x555f64f909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.393+0000 7fcb99e00640 1 -- 172.21.15.86:0/116859820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f84a20 msgr2=0x555f64f909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.393+0000 7fcb99e00640 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f84a20 0x555f64f909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.393+0000 7fcb99e00640 1 -- 172.21.15.86:0/116859820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 msgr2=0x555f64f899a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.393+0000 7fcb99e00640 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f899a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.393+0000 7fcb99e00640 1 -- 172.21.15.86:0/116859820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f64cbf7e0 con 0x555f64f7eec0 2024-04-17T07:20:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.393+0000 7fcb99400640 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:19.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.394+0000 7fcb99e00640 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f89460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c09d2d0 tx=0x7fcb8c09cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3788fb48a62fba5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.394+0000 7fcb7be00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:19.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.394+0000 7fcb7be00640 1 -- 172.21.15.86:0/116859820 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c0b5030 con 0x555f64f7eec0 2024-04-17T07:20:19.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.394+0000 7fcb7be00640 1 -- 172.21.15.86:0/116859820 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcb8c0138d0 con 0x555f64f7eec0 2024-04-17T07:20:19.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.395+0000 7fcba3da6740 1 -- 172.21.15.86:0/116859820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb88012900 con 0x555f64f7eec0 2024-04-17T07:20:19.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.396+0000 7fcb7be00640 1 -- 172.21.15.86:0/116859820 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcb8c09d340 con 0x555f64f7eec0 2024-04-17T07:20:19.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.398+0000 7fcba3da6740 10 client.26101 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:19.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.398+0000 7fcba3da6740 1 -- 172.21.15.86:0/116859820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f64d1ff70 con 0x555f64f7eec0 2024-04-17T07:20:19.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.399+0000 7fcb7be00640 1 -- 172.21.15.86:0/116859820 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb88012900 con 0x555f64f7eec0 2024-04-17T07:20:19.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.399+0000 7fcb7be00640 1 -- 172.21.15.86:0/116859820 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcb8c0709d0 con 0x555f64f7eec0 2024-04-17T07:20:19.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.399+0000 7fcba3da6740 10 client.26101 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:19.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.399+0000 7fcba3da6740 10 client.26101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:19.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.399+0000 7fcba3da6740 20 client.26101 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:19.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.399+0000 7fcba3da6740 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555f64fa6210 0x555f64fa86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.399+0000 7fcba3da6740 4 client.26101 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:19.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.399+0000 7fcba3da6740 1 -- 172.21.15.86:0/116859820 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f64ef26d0 con 0x555f64fa6210 2024-04-17T07:20:19.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.400+0000 7fcb99400640 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555f64fa6210 0x555f64fa86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.402+0000 7fcb7be00640 10 client.26101 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:19.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.402+0000 7fcb99400640 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555f64fa6210 0x555f64fa86a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5538e74140cabb73 server_cookie=cc3902adb4dcf09f in_seq=0 out_seq=0 2024-04-17T07:20:19.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.404+0000 7fcb7be00640 1 -- 172.21.15.86:0/116859820 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f64ef26d0 con 0x555f64fa6210 2024-04-17T07:20:19.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.404+0000 7fcb7be00640 10 client.26101 handle_command_reply: tid=0 2024-04-17T07:20:19.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.404+0000 7fcba3da6740 1 client.26101 shutdown 2024-04-17T07:20:19.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.405+0000 7fcba3da6740 1 -- 172.21.15.86:0/116859820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 msgr2=0x555f64f89460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.405+0000 7fcba3da6740 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f89460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c09d2d0 tx=0x7fcb8c09cda0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.406+0000 7fcba3da6740 1 -- 172.21.15.86:0/116859820 shutdown_connections 2024-04-17T07:20:19.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.406+0000 7fcba3da6740 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555f64fa6210 0x555f64fa86a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.406+0000 7fcb7be00640 0 client.26101 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:19.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.406+0000 7fcba3da6740 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f84a20 0x555f64f909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.406+0000 7fcba3da6740 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.406+0000 7fcba3da6740 1 --2- 172.21.15.86:0/116859820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f89460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.406+0000 7fcba3da6740 1 -- 172.21.15.86:0/116859820 >> 172.21.15.86:0/116859820 conn(0x555f64f7a430 msgr2=0x555f64f795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.406+0000 7fcba3da6740 1 -- 172.21.15.86:0/116859820 shutdown_connections 2024-04-17T07:20:19.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.406+0000 7fcba3da6740 1 -- 172.21.15.86:0/116859820 wait complete. 2024-04-17T07:20:19.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.407+0000 7fcba3da6740 20 client.26101 trim_cache size 0 max 16384 2024-04-17T07:20:19.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.450+0000 7fcba3da6740 1 Processor -- start 2024-04-17T07:20:19.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.450+0000 7fcba3da6740 1 -- start start 2024-04-17T07:20:19.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.451+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f7eec0 0x555f64f89900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.451+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 0x555f64f909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.451+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f79460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.451+0000 7fcba3da6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1b1010 con 0x555f64f84a20 2024-04-17T07:20:19.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.451+0000 7fcba3da6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1b2b50 con 0x555f64f7eec0 2024-04-17T07:20:19.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.451+0000 7fcba3da6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1b16e0 con 0x555f64f7f8b0 2024-04-17T07:20:19.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.452+0000 7fcb9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f79460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.452+0000 7fcb9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f79460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50510/0 (socket says 172.21.15.86:50510) 2024-04-17T07:20:19.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.452+0000 7fcb99400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 0x555f64f909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.452+0000 7fcb99e00640 1 --2- 172.21.15.86:0/578274261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f7eec0 0x555f64f89900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.453+0000 7fcb99400640 1 -- 172.21.15.86:0/578274261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f7eec0 msgr2=0x555f64f89900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.453+0000 7fcb99400640 1 --2- 172.21.15.86:0/578274261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f7eec0 0x555f64f89900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.453+0000 7fcb99400640 1 -- 172.21.15.86:0/578274261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 msgr2=0x555f64f79460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.453+0000 7fcb99400640 1 --2- 172.21.15.86:0/578274261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f79460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.453+0000 7fcb99400640 1 -- 172.21.15.86:0/578274261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f64bb7170 con 0x555f64f7f8b0 2024-04-17T07:20:19.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.452+0000 7fcb9a800640 1 -- 172.21.15.86:0/578274261 learned_addr learned my addr 172.21.15.86:0/578274261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:19.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.453+0000 7fcb9a800640 1 --2- 172.21.15.86:0/578274261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f79460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-17T07:20:19.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.453+0000 7fcb99400640 1 --2- 172.21.15.86:0/578274261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 0x555f64f909a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009740 tx=0x7fcb880058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d30e4d0554e6dd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.454+0000 7fcb86a00640 1 -- 172.21.15.86:0/578274261 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88011e20 con 0x555f64f7f8b0 2024-04-17T07:20:19.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.454+0000 7fcb86a00640 1 -- 172.21.15.86:0/578274261 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb88028030 con 0x555f64f7f8b0 2024-04-17T07:20:19.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.454+0000 7fcb86a00640 1 -- 172.21.15.86:0/578274261 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb880125f0 con 0x555f64f7f8b0 2024-04-17T07:20:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.455+0000 7fcba3da6740 1 -- 172.21.15.86:0/578274261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 msgr2=0x555f64f909a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.455+0000 7fcba3da6740 1 --2- 172.21.15.86:0/578274261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 0x555f64f909a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009740 tx=0x7fcb880058d0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.456+0000 7fcba3da6740 1 -- 172.21.15.86:0/578274261 shutdown_connections 2024-04-17T07:20:19.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.456+0000 7fcba3da6740 1 --2- 172.21.15.86:0/578274261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f84a20 0x555f64f79460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.456+0000 7fcba3da6740 1 --2- 172.21.15.86:0/578274261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7f8b0 0x555f64f909a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.456+0000 7fcba3da6740 1 --2- 172.21.15.86:0/578274261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f7eec0 0x555f64f89900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.456+0000 7fcba3da6740 1 -- 172.21.15.86:0/578274261 >> 172.21.15.86:0/578274261 conn(0x555f64f7a430 msgr2=0x555f64f78d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.457+0000 7fcba3da6740 1 -- 172.21.15.86:0/578274261 shutdown_connections 2024-04-17T07:20:19.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.457+0000 7fcba3da6740 1 -- 172.21.15.86:0/578274261 wait complete. 2024-04-17T07:20:19.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.457+0000 7fcba3da6740 1 -- start start 2024-04-17T07:20:19.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.458+0000 7fcba3da6740 1 Processor -- start 2024-04-17T07:20:19.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.459+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f8eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.459+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f8eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.459+0000 7fcba3da6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f84a20 0x555f64f8fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.459+0000 7fcba3da6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1b2480 con 0x555f64f7f8b0 2024-04-17T07:20:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.459+0000 7fcba3da6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1b1db0 con 0x555f64f84a20 2024-04-17T07:20:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.459+0000 7fcba3da6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1aefe0 con 0x555f64f7eec0 2024-04-17T07:20:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.459+0000 7fcba3da6740 10 client.0 _ll_register_callbacks cb 0x7fcb8c0ceb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.460+0000 7fcb99e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f8eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.460+0000 7fcb99400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f8eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.460+0000 7fcb99400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f8eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50514/0 (socket says 172.21.15.86:50514) 2024-04-17T07:20:19.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.460+0000 7fcb99400640 1 -- 172.21.15.86:0/3316561761 learned_addr learned my addr 172.21.15.86:0/3316561761 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:19.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.461+0000 7fcb99e00640 1 -- 172.21.15.86:0/3316561761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f84a20 msgr2=0x555f64f8fe90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:20:19.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.462+0000 7fcb99e00640 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f84a20 0x555f64f8fe90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.462+0000 7fcb99e00640 1 -- 172.21.15.86:0/3316561761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 msgr2=0x555f64f8eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.462+0000 7fcb99e00640 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f8eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.462+0000 7fcb99e00640 1 -- 172.21.15.86:0/3316561761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f64f90530 con 0x555f64f7eec0 2024-04-17T07:20:19.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.462+0000 7fcb99400640 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f8eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:19.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.463+0000 7fcb99e00640 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f8eab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c09b5f0 tx=0x7fcb8c09d110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4114fff68f3a440 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.463+0000 7fcb7be00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.464+0000 7fcb7be00640 1 -- 172.21.15.86:0/3316561761 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c0b5070 con 0x555f64f7eec0 2024-04-17T07:20:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.464+0000 7fcb7be00640 1 -- 172.21.15.86:0/3316561761 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcb8c0138d0 con 0x555f64f7eec0 2024-04-17T07:20:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.464+0000 7fcba3da6740 1 -- 172.21.15.86:0/3316561761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb88028030 con 0x555f64f7eec0 2024-04-17T07:20:19.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.465+0000 7fcb7be00640 1 -- 172.21.15.86:0/3316561761 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcb8c0b5070 con 0x555f64f7eec0 2024-04-17T07:20:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.465+0000 7fcba3da6740 10 client.26107 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.465+0000 7fcba3da6740 1 -- 172.21.15.86:0/3316561761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f64f90530 con 0x555f64f7eec0 2024-04-17T07:20:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.465+0000 7fcb7be00640 1 -- 172.21.15.86:0/3316561761 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb8c074070 con 0x555f64f7eec0 2024-04-17T07:20:19.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.466+0000 7fcb7be00640 1 -- 172.21.15.86:0/3316561761 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcb8c0703c0 con 0x555f64f7eec0 2024-04-17T07:20:19.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.466+0000 7fcba3da6740 10 client.26107 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:19.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.466+0000 7fcba3da6740 10 client.26107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:19.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.466+0000 7fcba3da6740 20 client.26107 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:19.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.466+0000 7fcba3da6740 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555f64fa6210 0x555f6500f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.466+0000 7fcba3da6740 4 client.26107 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:20:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.466+0000 7fcba3da6740 1 -- 172.21.15.86:0/3316561761 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f64eec540 con 0x555f64fa6210 2024-04-17T07:20:19.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.467+0000 7fcb99400640 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555f64fa6210 0x555f6500f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.469+0000 7fcb7be00640 10 client.26107 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:19.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.469+0000 7fcb99400640 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555f64fa6210 0x555f6500f770 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b7f29b7651c4995 server_cookie=e20c86974081e6a7 in_seq=0 out_seq=0 2024-04-17T07:20:19.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.473+0000 7fcb7be00640 1 -- 172.21.15.86:0/3316561761 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f64eec540 con 0x555f64fa6210 2024-04-17T07:20:19.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.473+0000 7fcb7be00640 10 client.26107 handle_command_reply: tid=0 2024-04-17T07:20:19.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.473+0000 7fcba3da6740 1 client.26107 shutdown 2024-04-17T07:20:19.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.473+0000 7fcba3da6740 1 -- 172.21.15.86:0/3316561761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 msgr2=0x555f64f8eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.473+0000 7fcba3da6740 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f8eab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c09b5f0 tx=0x7fcb8c09d110 comp rx=0 tx=0).stop 2024-04-17T07:20:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.474+0000 7fcba3da6740 1 -- 172.21.15.86:0/3316561761 shutdown_connections 2024-04-17T07:20:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.474+0000 7fcba3da6740 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555f64fa6210 0x555f6500f770 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.474+0000 7fcba3da6740 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555f64f84a20 0x555f64f8fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.474+0000 7fcba3da6740 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f64f7f8b0 0x555f64f8eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.474+0000 7fcba3da6740 1 --2- 172.21.15.86:0/3316561761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555f64f7eec0 0x555f64f8eab0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.474+0000 7fcb7be00640 0 client.26107 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.474+0000 7fcba3da6740 1 -- 172.21.15.86:0/3316561761 >> 172.21.15.86:0/3316561761 conn(0x555f64f7a430 msgr2=0x555f64f795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.475+0000 7fcba3da6740 1 -- 172.21.15.86:0/3316561761 shutdown_connections 2024-04-17T07:20:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.475+0000 7fcba3da6740 1 -- 172.21.15.86:0/3316561761 wait complete. 2024-04-17T07:20:19.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.475+0000 7fcba3da6740 20 client.26107 trim_cache size 0 max 16384 2024-04-17T07:20:19.476 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:20:19.476 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:20:19.476 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "cefb0d75-16c1-4d4c-9092-b5aeb7d0006d", 2024-04-17T07:20:19.476 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:20:19.476 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:20:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.477+0000 7fcba1000640 1 -- 172.21.15.86:0/1409182364 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb800a3db0 msgr2=0x7fcb800a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.477+0000 7fcba1000640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb800a3db0 0x7fcb800a6270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcb94034fd0 tx=0x7fcb94005ea0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.477+0000 7fcba1000640 1 -- 172.21.15.86:0/1409182364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb9c1091d0 msgr2=0x7fcb9c1a2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.477+0000 7fcba1000640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb9c1091d0 0x7fcb9c1a2bf0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c007ba0 tx=0x7fcb8c00d490 comp rx=0 tx=0).stop 2024-04-17T07:20:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.477+0000 7fcba1000640 1 -- 172.21.15.86:0/1409182364 shutdown_connections 2024-04-17T07:20:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.477+0000 7fcba1000640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb800a3db0 0x7fcb800a6270 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.477+0000 7fcba1000640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb9c10e650 0x7fcb9c1aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.477+0000 7fcba1000640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb9c10bb90 0x7fcb9c1a3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.477+0000 7fcba1000640 1 --2- 172.21.15.86:0/1409182364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb9c1091d0 0x7fcb9c1a2bf0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.477+0000 7fcba1000640 1 -- 172.21.15.86:0/1409182364 >> 172.21.15.86:0/1409182364 conn(0x7fcb9c06f190 msgr2=0x7fcb9c109bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.478+0000 7fcba1000640 1 -- 172.21.15.86:0/1409182364 shutdown_connections 2024-04-17T07:20:19.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.478+0000 7fcba1000640 1 -- 172.21.15.86:0/1409182364 wait complete. 2024-04-17T07:20:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:19 smithi098 ceph-mon[35418]: pgmap v618: 129 pgs: 129 active+clean; 955 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.4 MiB/s wr, 841 op/s 2024-04-17T07:20:19.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:20:19.520 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:20:19.525 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:19 smithi103 ceph-mon[35495]: pgmap v618: 129 pgs: 129 active+clean; 955 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.4 MiB/s wr, 841 op/s 2024-04-17T07:20:19.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.838+0000 7f0ffdc00640 1 -- 172.21.15.86:0/787224490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff81053d0 msgr2=0x7f0ff8105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.838+0000 7f0ffdc00640 1 --2- 172.21.15.86:0/787224490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff81053d0 0x7f0ff8105850 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009900 tx=0x7f0fe402bc40 comp rx=0 tx=0).stop 2024-04-17T07:20:19.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.839+0000 7f0ffdc00640 1 -- 172.21.15.86:0/787224490 shutdown_connections 2024-04-17T07:20:19.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.839+0000 7f0ffdc00640 1 --2- 172.21.15.86:0/787224490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ff8105d90 0x7f0ff810c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.839+0000 7f0ffdc00640 1 --2- 172.21.15.86:0/787224490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff81053d0 0x7f0ff8105850 secure :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009900 tx=0x7f0fe402bc40 comp rx=0 tx=0).stop 2024-04-17T07:20:19.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.839+0000 7f0ffdc00640 1 --2- 172.21.15.86:0/787224490 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ff81041e0 0x7f0ff8104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.839+0000 7f0ffdc00640 1 -- 172.21.15.86:0/787224490 >> 172.21.15.86:0/787224490 conn(0x7f0ff80ff750 msgr2=0x7f0ff8101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.840+0000 7f0ffdc00640 1 -- 172.21.15.86:0/787224490 shutdown_connections 2024-04-17T07:20:19.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.840+0000 7f0ffdc00640 1 -- 172.21.15.86:0/787224490 wait complete. 2024-04-17T07:20:19.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.841+0000 7f0ffdc00640 1 Processor -- start 2024-04-17T07:20:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.842+0000 7f0ffdc00640 1 -- start start 2024-04-17T07:20:19.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.842+0000 7f0ffdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff81041e0 0x7f0ff819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.842+0000 7f0ffdc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ff8105d90 0x7f0ff819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.842+0000 7f0ffdc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ff81a4010 0x7f0ff81a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.842+0000 7f0ffdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff810fd70 con 0x7f0ff81041e0 2024-04-17T07:20:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.842+0000 7f0ffdc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff810fbf0 con 0x7f0ff81a4010 2024-04-17T07:20:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.842+0000 7f0ffdc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff810fef0 con 0x7f0ff8105d90 2024-04-17T07:20:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.843+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff81041e0 0x7f0ff819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.843+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff81041e0 0x7f0ff819eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50532/0 (socket says 172.21.15.86:50532) 2024-04-17T07:20:19.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.843+0000 7f0ff6a00640 1 -- 172.21.15.86:0/3496627586 learned_addr learned my addr 172.21.15.86:0/3496627586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.843+0000 7f0ff7400640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ff81a4010 0x7f0ff81a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.844+0000 7f0ff6000640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ff8105d90 0x7f0ff819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.844+0000 7f0ff7400640 1 -- 172.21.15.86:0/3496627586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ff8105d90 msgr2=0x7f0ff819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.844+0000 7f0ff7400640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ff8105d90 0x7f0ff819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.844+0000 7f0ff7400640 1 -- 172.21.15.86:0/3496627586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff81041e0 msgr2=0x7f0ff819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.844+0000 7f0ff7400640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff81041e0 0x7f0ff819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.844+0000 7f0ff7400640 1 -- 172.21.15.86:0/3496627586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff8195510 con 0x7f0ff81a4010 2024-04-17T07:20:19.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.844+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff81041e0 0x7f0ff819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:19.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.845+0000 7f0ff7400640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ff81a4010 0x7f0ff81a6400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000da10 tx=0x7f0ff000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.846+0000 7f0fef400640 1 -- 172.21.15.86:0/3496627586 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000bd70 con 0x7f0ff81a4010 2024-04-17T07:20:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.846+0000 7f0fef400640 1 -- 172.21.15.86:0/3496627586 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ff0003e20 con 0x7f0ff81a4010 2024-04-17T07:20:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.846+0000 7f0fef400640 1 -- 172.21.15.86:0/3496627586 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00106c0 con 0x7f0ff81a4010 2024-04-17T07:20:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.846+0000 7f0ffdc00640 1 -- 172.21.15.86:0/3496627586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff8195800 con 0x7f0ff81a4010 2024-04-17T07:20:19.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.846+0000 7f0ffdc00640 1 -- 172.21.15.86:0/3496627586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff8195d20 con 0x7f0ff81a4010 2024-04-17T07:20:19.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.852+0000 7f100099a740 1 Processor -- start 2024-04-17T07:20:19.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.853+0000 7f0fef400640 1 -- 172.21.15.86:0/3496627586 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0ff0003970 con 0x7f0ff81a4010 2024-04-17T07:20:19.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.854+0000 7f100099a740 1 -- start start 2024-04-17T07:20:19.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.854+0000 7f100099a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95190 0x558c03c955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.854+0000 7f100099a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 0x558c03c9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.854+0000 7f100099a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03c9d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.854+0000 7f100099a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c03bfc120 con 0x558c03c9acf0 2024-04-17T07:20:19.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.854+0000 7f100099a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558c03ba2340 con 0x558c03c95b80 2024-04-17T07:20:19.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.854+0000 7f100099a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558c03bf80e0 con 0x558c03c95190 2024-04-17T07:20:19.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.855+0000 7f0ff6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95190 0x558c03c955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.855+0000 7f0ff6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95190 0x558c03c955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41286/0 (socket says 172.21.15.86:41286) 2024-04-17T07:20:19.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.855+0000 7f0ff6000640 1 -- 172.21.15.86:0/362498132 learned_addr learned my addr 172.21.15.86:0/362498132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:19.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.855+0000 7f0ff6000640 1 --2- 172.21.15.86:0/362498132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03c9d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.855+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/362498132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 0x558c03c9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.856+0000 7f0ff6000640 1 -- 172.21.15.86:0/362498132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 msgr2=0x558c03c9a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.856+0000 7f0ff6000640 1 --2- 172.21.15.86:0/362498132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 0x558c03c9a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.856+0000 7f0ff6000640 1 -- 172.21.15.86:0/362498132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 msgr2=0x558c03c9d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.856+0000 7f0ff6000640 1 --2- 172.21.15.86:0/362498132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03c9d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.856+0000 7f0ff6000640 1 -- 172.21.15.86:0/362498132 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c039d87d0 con 0x558c03c95190 2024-04-17T07:20:19.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.856+0000 7f0ff6000640 1 --2- 172.21.15.86:0/362498132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95190 0x558c03c955b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0fe402c250 tx=0x7f0fe4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78c838af5ee3f55e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.857+0000 7f0fe3e00640 1 -- 172.21.15.86:0/362498132 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4004230 con 0x558c03c95190 2024-04-17T07:20:19.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.857+0000 7f0fef400640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0fdc0a3d30 0x7f0fdc0a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.857+0000 7f0fe3e00640 1 -- 172.21.15.86:0/362498132 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe40043d0 con 0x558c03c95190 2024-04-17T07:20:19.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.857+0000 7f0fe3e00640 1 -- 172.21.15.86:0/362498132 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4043690 con 0x558c03c95190 2024-04-17T07:20:19.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.857+0000 7f0fef400640 1 -- 172.21.15.86:0/3496627586 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0ff00d09d0 con 0x7f0ff81a4010 2024-04-17T07:20:19.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.857+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0fdc0a3d30 0x7f0fdc0a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.858+0000 7f100099a740 1 -- 172.21.15.86:0/362498132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95190 msgr2=0x558c03c955b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.858+0000 7f100099a740 1 --2- 172.21.15.86:0/362498132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95190 0x558c03c955b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0fe402c250 tx=0x7f0fe4005fd0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.858+0000 7f100099a740 1 -- 172.21.15.86:0/362498132 shutdown_connections 2024-04-17T07:20:19.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.858+0000 7f100099a740 1 --2- 172.21.15.86:0/362498132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03c9d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.858+0000 7f100099a740 1 --2- 172.21.15.86:0/362498132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 0x558c03c9a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.858+0000 7f100099a740 1 --2- 172.21.15.86:0/362498132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95190 0x558c03c955b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.858+0000 7f100099a740 1 -- 172.21.15.86:0/362498132 >> 172.21.15.86:0/362498132 conn(0x558c03c90700 msgr2=0x558c03c90b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.858+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0fdc0a3d30 0x7f0fdc0a61f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800bfd0 tx=0x7f0fe800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.859+0000 7f100099a740 1 -- 172.21.15.86:0/362498132 shutdown_connections 2024-04-17T07:20:19.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.859+0000 7f100099a740 1 -- 172.21.15.86:0/362498132 wait complete. 2024-04-17T07:20:19.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.859+0000 7f100099a740 1 -- start start 2024-04-17T07:20:19.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.860+0000 7f100099a740 1 Processor -- start 2024-04-17T07:20:19.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.860+0000 7f100099a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.861+0000 7f100099a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 0x558c03cb0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.861+0000 7f100099a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03cb7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.861+0000 7f100099a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c03caebf0 con 0x558c03c9acf0 2024-04-17T07:20:19.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.861+0000 7f100099a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558c03bb5710 con 0x558c03c95190 2024-04-17T07:20:19.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.861+0000 7f100099a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558c03caed70 con 0x558c03c95b80 2024-04-17T07:20:19.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.861+0000 7f100099a740 10 client.0 _ll_register_callbacks cb 0x558c03bb0af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:19.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.861+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.862+0000 7f0ff6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 0x558c03cb0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.862+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:56128/0 (socket says 172.21.15.86:56128) 2024-04-17T07:20:19.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.862+0000 7f0ff6a00640 1 -- 172.21.15.86:0/1760817667 learned_addr learned my addr 172.21.15.86:0/1760817667 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:19.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.862+0000 7f0ff6000640 1 -- 172.21.15.86:0/1760817667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 msgr2=0x558c03cb0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.862+0000 7f0ff6000640 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.862+0000 7f0ff6000640 1 -- 172.21.15.86:0/1760817667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 msgr2=0x558c03cb7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.862+0000 7f0ff7400640 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03cb7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.862+0000 7f0ff6000640 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03cb7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.862+0000 7f0ff6000640 1 -- 172.21.15.86:0/1760817667 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c0390e3f0 con 0x558c03c95b80 2024-04-17T07:20:19.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.862+0000 7f0ff7400640 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03cb7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:19.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.863+0000 7f0ff6000640 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 0x558c03cb0b60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0fe402c1d0 tx=0x7f0fe402cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43522348fdcdd9c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.863+0000 7f0fe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:19.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.864+0000 7f0fe0c00640 1 -- 172.21.15.86:0/1760817667 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4003800 con 0x558c03c95b80 2024-04-17T07:20:19.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.864+0000 7f0fe0c00640 1 -- 172.21.15.86:0/1760817667 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0fe40039a0 con 0x558c03c95b80 2024-04-17T07:20:19.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.864+0000 7f100099a740 1 -- 172.21.15.86:0/1760817667 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe40043d0 con 0x558c03c95b80 2024-04-17T07:20:19.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.864+0000 7f0fe0c00640 1 -- 172.21.15.86:0/1760817667 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0fe4003ba0 con 0x558c03c95b80 2024-04-17T07:20:19.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.865+0000 7f100099a740 10 client.26119 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:19.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.865+0000 7f100099a740 1 -- 172.21.15.86:0/1760817667 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c038ce0f0 con 0x558c03c95b80 2024-04-17T07:20:19.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.865+0000 7f0fe0c00640 1 -- 172.21.15.86:0/1760817667 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe40043d0 con 0x558c03c95b80 2024-04-17T07:20:19.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.865+0000 7f0fe0c00640 1 -- 172.21.15.86:0/1760817667 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0fe4040040 con 0x558c03c95b80 2024-04-17T07:20:19.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.865+0000 7f100099a740 10 client.26119 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.865+0000 7f100099a740 10 client.26119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.865+0000 7f100099a740 20 client.26119 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.865+0000 7f100099a740 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558c03cbc0e0 0x558c03cbe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.865+0000 7f100099a740 4 client.26119 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.865+0000 7f100099a740 1 -- 172.21.15.86:0/1760817667 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c03c09b40 con 0x558c03cbc0e0 2024-04-17T07:20:19.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.867+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558c03cbc0e0 0x558c03cbe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.869+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558c03cbc0e0 0x558c03cbe4d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a60e9b717c2f78 server_cookie=5bbc84c4fc7094a4 in_seq=0 out_seq=0 2024-04-17T07:20:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.869+0000 7f0fe0c00640 10 client.26119 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.871+0000 7f0fe0c00640 1 -- 172.21.15.86:0/1760817667 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c03c09b40 con 0x558c03cbc0e0 2024-04-17T07:20:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.871+0000 7f0fe0c00640 10 client.26119 handle_command_reply: tid=0 2024-04-17T07:20:19.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.872+0000 7f100099a740 1 client.26119 shutdown 2024-04-17T07:20:19.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.872+0000 7f100099a740 1 -- 172.21.15.86:0/1760817667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 msgr2=0x558c03cb0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.872+0000 7f100099a740 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 0x558c03cb0b60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0fe402c1d0 tx=0x7f0fe402cc20 comp rx=0 tx=0).stop 2024-04-17T07:20:19.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.873+0000 7f100099a740 1 -- 172.21.15.86:0/1760817667 shutdown_connections 2024-04-17T07:20:19.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.873+0000 7f100099a740 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558c03cbc0e0 0x558c03cbe4d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.873+0000 7f100099a740 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03cb7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.873+0000 7f100099a740 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 0x558c03cb0b60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.873+0000 7f100099a740 1 --2- 172.21.15.86:0/1760817667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.873+0000 7f100099a740 1 -- 172.21.15.86:0/1760817667 >> 172.21.15.86:0/1760817667 conn(0x558c03c90700 msgr2=0x558c03c8f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.873+0000 7f0fe0c00640 0 client.26119 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:19.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.873+0000 7f100099a740 1 -- 172.21.15.86:0/1760817667 shutdown_connections 2024-04-17T07:20:19.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.873+0000 7f100099a740 1 -- 172.21.15.86:0/1760817667 wait complete. 2024-04-17T07:20:19.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.874+0000 7f100099a740 20 client.26119 trim_cache size 0 max 16384 2024-04-17T07:20:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.923+0000 7f100099a740 1 Processor -- start 2024-04-17T07:20:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.923+0000 7f100099a740 1 -- start start 2024-04-17T07:20:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.923+0000 7f100099a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c95190 0x558c03c90ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.923+0000 7f100099a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 0x558c03cb7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.923+0000 7f100099a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c9acf0 0x558c03caf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.923+0000 7f100099a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd800fb60 con 0x558c03c95190 2024-04-17T07:20:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.923+0000 7f100099a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd80116a0 con 0x558c03c95b80 2024-04-17T07:20:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.923+0000 7f100099a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd8010230 con 0x558c03c9acf0 2024-04-17T07:20:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.924+0000 7f0ff6000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 0x558c03cb7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.924+0000 7f0ff6000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 0x558c03cb7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:56142/0 (socket says 172.21.15.86:56142) 2024-04-17T07:20:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.924+0000 7f0ff6000640 1 -- 172.21.15.86:0/143043455 learned_addr learned my addr 172.21.15.86:0/143043455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.924+0000 7f0ff7400640 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c9acf0 0x558c03caf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.925+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c95190 0x558c03c90ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.925+0000 7f0ff6000640 1 -- 172.21.15.86:0/143043455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c9acf0 msgr2=0x558c03caf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.926+0000 7f0ff6000640 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c9acf0 0x558c03caf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.926+0000 7f0ff6000640 1 -- 172.21.15.86:0/143043455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c95190 msgr2=0x558c03c90ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.926+0000 7f0ff6000640 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c95190 0x558c03c90ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.926+0000 7f0ff6000640 1 -- 172.21.15.86:0/143043455 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c03c8ace0 con 0x558c03c95b80 2024-04-17T07:20:19.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.927+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c95190 0x558c03c90ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:19.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.927+0000 7f0ff6000640 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 0x558c03cb7b60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4004ad0 tx=0x7f0fe4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403a69dd750ff95c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.927+0000 7f0ff7400640 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c9acf0 0x558c03caf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.928+0000 7f0fe3400640 1 -- 172.21.15.86:0/143043455 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4003d40 con 0x558c03c95b80 2024-04-17T07:20:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.928+0000 7f0fe3400640 1 -- 172.21.15.86:0/143043455 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe4003800 con 0x558c03c95b80 2024-04-17T07:20:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.928+0000 7f0fe3400640 1 -- 172.21.15.86:0/143043455 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40044b0 con 0x558c03c95b80 2024-04-17T07:20:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.928+0000 7f100099a740 1 -- 172.21.15.86:0/143043455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 msgr2=0x558c03cb7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.929+0000 7f100099a740 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 0x558c03cb7b60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4004ad0 tx=0x7f0fe4005fd0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.929+0000 7f100099a740 1 -- 172.21.15.86:0/143043455 shutdown_connections 2024-04-17T07:20:19.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.929+0000 7f100099a740 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c9acf0 0x558c03caf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.929+0000 7f100099a740 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95b80 0x558c03cb7b60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.929+0000 7f100099a740 1 --2- 172.21.15.86:0/143043455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c95190 0x558c03c90ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.929+0000 7f100099a740 1 -- 172.21.15.86:0/143043455 >> 172.21.15.86:0/143043455 conn(0x558c03c90700 msgr2=0x558c03c97020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.929+0000 7f100099a740 1 -- 172.21.15.86:0/143043455 shutdown_connections 2024-04-17T07:20:19.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.929+0000 7f100099a740 1 -- 172.21.15.86:0/143043455 wait complete. 2024-04-17T07:20:19.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.929+0000 7f100099a740 1 -- start start 2024-04-17T07:20:19.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.930+0000 7f100099a740 1 Processor -- start 2024-04-17T07:20:19.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.930+0000 7f100099a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.930+0000 7f100099a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 0x558c03cb55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.930+0000 7f100099a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03cb65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.930+0000 7f100099a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd8010fd0 con 0x558c03c9acf0 2024-04-17T07:20:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.930+0000 7f100099a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd8010900 con 0x558c03c95190 2024-04-17T07:20:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.930+0000 7f100099a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd800db30 con 0x558c03c95b80 2024-04-17T07:20:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.931+0000 7f100099a740 10 client.0 _ll_register_callbacks cb 0x7f0fe402c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.931+0000 7f0ff7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03cb65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.931+0000 7f0ff6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 0x558c03cb55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.931+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.932+0000 7f0ff6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:56144/0 (socket says 172.21.15.86:56144) 2024-04-17T07:20:19.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.932+0000 7f0ff7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03cb65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50558/0 (socket says 172.21.15.86:50558) 2024-04-17T07:20:19.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.932+0000 7f0ff6a00640 1 -- 172.21.15.86:0/2844752752 learned_addr learned my addr 172.21.15.86:0/2844752752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:19.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.933+0000 7f0ff6a00640 1 -- 172.21.15.86:0/2844752752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 msgr2=0x558c03cb55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.933+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 0x558c03cb55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.933+0000 7f0ff6a00640 1 -- 172.21.15.86:0/2844752752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 msgr2=0x558c03cb65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:19.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.933+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03cb65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.934+0000 7f0ff6a00640 1 -- 172.21.15.86:0/2844752752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c03cb6be0 con 0x558c03c95190 2024-04-17T07:20:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.934+0000 7f0ff6000640 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 0x558c03cb55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.936+0000 7f0ff6a00640 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb50a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8005c40 tx=0x7f0fe80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26751b458acf1d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.936+0000 7f0fe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:20:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.936+0000 7f0fe0c00640 1 -- 172.21.15.86:0/2844752752 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe802a4e0 con 0x558c03c95190 2024-04-17T07:20:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.937+0000 7f0fe0c00640 1 -- 172.21.15.86:0/2844752752 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0fe802a680 con 0x558c03c95190 2024-04-17T07:20:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.937+0000 7f100099a740 1 -- 172.21.15.86:0/2844752752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe4003800 con 0x558c03c95190 2024-04-17T07:20:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.938+0000 7f0fe0c00640 1 -- 172.21.15.86:0/2844752752 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0fe8038050 con 0x558c03c95190 2024-04-17T07:20:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.938+0000 7f100099a740 10 client.36095 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.938+0000 7f100099a740 1 -- 172.21.15.86:0/2844752752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c03cb7ed0 con 0x558c03c95190 2024-04-17T07:20:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.939+0000 7f0fe0c00640 1 -- 172.21.15.86:0/2844752752 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fe4003800 con 0x558c03c95190 2024-04-17T07:20:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.939+0000 7f0fe0c00640 1 -- 172.21.15.86:0/2844752752 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0fe801d5d0 con 0x558c03c95190 2024-04-17T07:20:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.941+0000 7f100099a740 10 client.36095 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.941+0000 7f100099a740 10 client.36095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.941+0000 7f100099a740 20 client.36095 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.941+0000 7f100099a740 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558c03d22e50 0x558c03d25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.941+0000 7f100099a740 4 client.36095 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:20:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.941+0000 7f100099a740 1 -- 172.21.15.86:0/2844752752 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c03c039b0 con 0x558c03d22e50 2024-04-17T07:20:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.942+0000 7f0ff6000640 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558c03d22e50 0x558c03d25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.944+0000 7f0ff6000640 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558c03d22e50 0x558c03d25270 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1914d3e4e873ca93 server_cookie=7b9447a799897592 in_seq=0 out_seq=0 2024-04-17T07:20:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.944+0000 7f0fe0c00640 10 client.36095 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.944+0000 7f0fe0c00640 1 -- 172.21.15.86:0/2844752752 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558c03c039b0 con 0x558c03d22e50 2024-04-17T07:20:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.944+0000 7f0fe0c00640 10 client.36095 handle_command_reply: tid=0 2024-04-17T07:20:19.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.944+0000 7f100099a740 1 client.36095 shutdown 2024-04-17T07:20:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.945+0000 7f100099a740 1 -- 172.21.15.86:0/2844752752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 msgr2=0x558c03cb50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.945+0000 7f100099a740 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb50a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8005c40 tx=0x7f0fe80046c0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.945+0000 7f100099a740 1 -- 172.21.15.86:0/2844752752 shutdown_connections 2024-04-17T07:20:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.945+0000 7f100099a740 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558c03d22e50 0x558c03d25270 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.945+0000 7f100099a740 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c03c9acf0 0x558c03cb65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.945+0000 7f100099a740 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c03c95b80 0x558c03cb55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.945+0000 7f100099a740 1 --2- 172.21.15.86:0/2844752752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c03c95190 0x558c03cb50a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.945+0000 7f100099a740 1 -- 172.21.15.86:0/2844752752 >> 172.21.15.86:0/2844752752 conn(0x558c03c90700 msgr2=0x558c03c8f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.945+0000 7f0fe0c00640 0 client.36095 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:19.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.946+0000 7f100099a740 1 -- 172.21.15.86:0/2844752752 shutdown_connections 2024-04-17T07:20:19.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.946+0000 7f100099a740 1 -- 172.21.15.86:0/2844752752 wait complete. 2024-04-17T07:20:19.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.946+0000 7f100099a740 20 client.36095 trim_cache size 0 max 16384 2024-04-17T07:20:19.947 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:20:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:20:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:20:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: "cefb0d75-16c1-4d4c-9092-b5aeb7d0006d": { 2024-04-17T07:20:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:20:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "cefb0d75-16c1-4d4c-9092-b5aeb7d0006d", 2024-04-17T07:20:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:20:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:20:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:20:19.947 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:20:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.948+0000 7f0ffdc00640 1 -- 172.21.15.86:0/3496627586 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0fdc0a3d30 msgr2=0x7f0fdc0a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.948+0000 7f0ffdc00640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0fdc0a3d30 0x7f0fdc0a61f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800bfd0 tx=0x7f0fe800b410 comp rx=0 tx=0).stop 2024-04-17T07:20:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.948+0000 7f0ffdc00640 1 -- 172.21.15.86:0/3496627586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ff81a4010 msgr2=0x7f0ff81a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.948+0000 7f0ffdc00640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ff81a4010 0x7f0ff81a6400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000da10 tx=0x7f0ff000ded0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.949+0000 7f0ffdc00640 1 -- 172.21.15.86:0/3496627586 shutdown_connections 2024-04-17T07:20:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.949+0000 7f0ffdc00640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0fdc0a3d30 0x7f0fdc0a61f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.949+0000 7f0ffdc00640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ff81a4010 0x7f0ff81a6400 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.949+0000 7f0ffdc00640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ff8105d90 0x7f0ff819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.949+0000 7f0ffdc00640 1 --2- 172.21.15.86:0/3496627586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff81041e0 0x7f0ff819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:19.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.949+0000 7f0ffdc00640 1 -- 172.21.15.86:0/3496627586 >> 172.21.15.86:0/3496627586 conn(0x7f0ff80ff750 msgr2=0x7f0ff80783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:19.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.950+0000 7f0ffdc00640 1 -- 172.21.15.86:0/3496627586 shutdown_connections 2024-04-17T07:20:19.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:19.951+0000 7f0ffdc00640 1 -- 172.21.15.86:0/3496627586 wait complete. 2024-04-17T07:20:19.978 INFO:tasks.cephfs.filesystem:scrub status for tag:cefb0d75-16c1-4d4c-9092-b5aeb7d0006d - {'path': '/', 'tag': 'cefb0d75-16c1-4d4c-9092-b5aeb7d0006d', 'options': 'recursive,force'} 2024-04-17T07:20:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:20 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:20:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:20 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:20 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:20:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:20 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:20.404 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:20 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:20:20.404 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:20 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:20.404 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:20 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:20:20.404 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:20 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:20.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:20 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:20:20.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:20 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:20.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:20 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:20:20.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:20 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:21.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:21 smithi086 ceph-mon[31733]: pgmap v619: 129 pgs: 129 active+clean; 955 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.2 MiB/s wr, 698 op/s 2024-04-17T07:20:21.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:21 smithi098 ceph-mon[35418]: pgmap v619: 129 pgs: 129 active+clean; 955 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.2 MiB/s wr, 698 op/s 2024-04-17T07:20:21.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:21 smithi103 ceph-mon[35495]: pgmap v619: 129 pgs: 129 active+clean; 955 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.2 MiB/s wr, 698 op/s 2024-04-17T07:20:23.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:23 smithi086 ceph-mon[31733]: pgmap v620: 129 pgs: 129 active+clean; 955 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.8 MiB/s wr, 772 op/s 2024-04-17T07:20:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:23 smithi098 ceph-mon[35418]: pgmap v620: 129 pgs: 129 active+clean; 955 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.8 MiB/s wr, 772 op/s 2024-04-17T07:20:23.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:23 smithi103 ceph-mon[35495]: pgmap v620: 129 pgs: 129 active+clean; 955 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.8 MiB/s wr, 772 op/s 2024-04-17T07:20:24.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:24 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:20:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 60.0 s, 731.2 tps, lat 134.454 ms stddev 38.045 2024-04-17T07:20:24.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:24 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:20:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:24 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:20:25.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:25 smithi086 ceph-mon[31733]: pgmap v621: 129 pgs: 129 active+clean; 956 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.0 MiB/s wr, 817 op/s 2024-04-17T07:20:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:25 smithi098 ceph-mon[35418]: pgmap v621: 129 pgs: 129 active+clean; 956 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.0 MiB/s wr, 817 op/s 2024-04-17T07:20:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:25 smithi103 ceph-mon[35495]: pgmap v621: 129 pgs: 129 active+clean; 956 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.0 MiB/s wr, 817 op/s 2024-04-17T07:20:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:27 smithi086 ceph-mon[31733]: pgmap v622: 129 pgs: 129 active+clean; 956 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 721 op/s 2024-04-17T07:20:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:27 smithi098 ceph-mon[35418]: pgmap v622: 129 pgs: 129 active+clean; 956 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 721 op/s 2024-04-17T07:20:27.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:27 smithi103 ceph-mon[35495]: pgmap v622: 129 pgs: 129 active+clean; 956 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 721 op/s 2024-04-17T07:20:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 65.0 s, 715.6 tps, lat 133.028 ms stddev 52.101 2024-04-17T07:20:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:29 smithi098 ceph-mon[35418]: pgmap v623: 129 pgs: 129 active+clean; 956 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.8 MiB/s wr, 919 op/s 2024-04-17T07:20:29.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:29 smithi103 ceph-mon[35495]: pgmap v623: 129 pgs: 129 active+clean; 956 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.8 MiB/s wr, 919 op/s 2024-04-17T07:20:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:29 smithi086 ceph-mon[31733]: pgmap v623: 129 pgs: 129 active+clean; 956 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.8 MiB/s wr, 919 op/s 2024-04-17T07:20:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:31 smithi098 ceph-mon[35418]: pgmap v624: 129 pgs: 129 active+clean; 956 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 768 op/s 2024-04-17T07:20:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:31 smithi103 ceph-mon[35495]: pgmap v624: 129 pgs: 129 active+clean; 956 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 768 op/s 2024-04-17T07:20:31.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:31 smithi086 ceph-mon[31733]: pgmap v624: 129 pgs: 129 active+clean; 956 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 768 op/s 2024-04-17T07:20:33.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:33 smithi098 ceph-mon[35418]: pgmap v625: 129 pgs: 129 active+clean; 956 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 885 op/s 2024-04-17T07:20:33.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:33 smithi103 ceph-mon[35495]: pgmap v625: 129 pgs: 129 active+clean; 956 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 885 op/s 2024-04-17T07:20:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:33 smithi086 ceph-mon[31733]: pgmap v625: 129 pgs: 129 active+clean; 956 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 885 op/s 2024-04-17T07:20:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 70.0 s, 618.0 tps, lat 80.767 ms stddev 73.073 2024-04-17T07:20:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:35 smithi086 ceph-mon[31733]: pgmap v626: 129 pgs: 129 active+clean; 958 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.1 MiB/s wr, 938 op/s 2024-04-17T07:20:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:35 smithi098 ceph-mon[35418]: pgmap v626: 129 pgs: 129 active+clean; 958 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.1 MiB/s wr, 938 op/s 2024-04-17T07:20:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:35 smithi103 ceph-mon[35495]: pgmap v626: 129 pgs: 129 active+clean; 958 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.1 MiB/s wr, 938 op/s 2024-04-17T07:20:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:37 smithi098 ceph-mon[35418]: pgmap v627: 129 pgs: 129 active+clean; 958 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 816 op/s 2024-04-17T07:20:37.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:37 smithi103 ceph-mon[35495]: pgmap v627: 129 pgs: 129 active+clean; 958 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 816 op/s 2024-04-17T07:20:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:37 smithi086 ceph-mon[31733]: pgmap v627: 129 pgs: 129 active+clean; 958 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 816 op/s 2024-04-17T07:20:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 75.0 s, 651.2 tps, lat 240.599 ms stddev 902.091 2024-04-17T07:20:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:39 smithi098 ceph-mon[35418]: pgmap v628: 129 pgs: 129 active+clean; 960 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.7 MiB/s wr, 997 op/s 2024-04-17T07:20:39.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:39 smithi103 ceph-mon[35495]: pgmap v628: 129 pgs: 129 active+clean; 960 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.7 MiB/s wr, 997 op/s 2024-04-17T07:20:39.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:39 smithi086 ceph-mon[31733]: pgmap v628: 129 pgs: 129 active+clean; 960 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.7 MiB/s wr, 997 op/s 2024-04-17T07:20:41.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:41 smithi098 ceph-mon[35418]: pgmap v629: 129 pgs: 129 active+clean; 962 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 808 op/s 2024-04-17T07:20:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:41 smithi103 ceph-mon[35495]: pgmap v629: 129 pgs: 129 active+clean; 962 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 808 op/s 2024-04-17T07:20:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:41 smithi086 ceph-mon[31733]: pgmap v629: 129 pgs: 129 active+clean; 962 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.2 MiB/s wr, 808 op/s 2024-04-17T07:20:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:43 smithi098 ceph-mon[35418]: pgmap v630: 129 pgs: 129 active+clean; 964 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 894 op/s 2024-04-17T07:20:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:43 smithi103 ceph-mon[35495]: pgmap v630: 129 pgs: 129 active+clean; 964 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 894 op/s 2024-04-17T07:20:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:43 smithi086 ceph-mon[31733]: pgmap v630: 129 pgs: 129 active+clean; 964 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 MiB/s wr, 894 op/s 2024-04-17T07:20:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 80.0 s, 740.4 tps, lat 135.828 ms stddev 39.256 2024-04-17T07:20:45.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:45 smithi098 ceph-mon[35418]: pgmap v631: 129 pgs: 129 active+clean; 964 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 890 op/s 2024-04-17T07:20:45.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:45 smithi103 ceph-mon[35495]: pgmap v631: 129 pgs: 129 active+clean; 964 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 890 op/s 2024-04-17T07:20:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:45 smithi086 ceph-mon[31733]: pgmap v631: 129 pgs: 129 active+clean; 964 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.2 MiB/s wr, 890 op/s 2024-04-17T07:20:47.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:47 smithi098 ceph-mon[35418]: pgmap v632: 129 pgs: 129 active+clean; 964 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 752 op/s 2024-04-17T07:20:47.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:47 smithi103 ceph-mon[35495]: pgmap v632: 129 pgs: 129 active+clean; 964 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 752 op/s 2024-04-17T07:20:47.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:47 smithi086 ceph-mon[31733]: pgmap v632: 129 pgs: 129 active+clean; 964 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 752 op/s 2024-04-17T07:20:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 85.0 s, 668.2 tps, lat 145.498 ms stddev 48.199 2024-04-17T07:20:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:49 smithi098 ceph-mon[35418]: pgmap v633: 129 pgs: 129 active+clean; 964 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 853 op/s 2024-04-17T07:20:49.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:49 smithi103 ceph-mon[35495]: pgmap v633: 129 pgs: 129 active+clean; 964 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 853 op/s 2024-04-17T07:20:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:49 smithi086 ceph-mon[31733]: pgmap v633: 129 pgs: 129 active+clean; 964 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 853 op/s 2024-04-17T07:20:49.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:20:49.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:20:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.303+0000 7f6926400640 1 -- 172.21.15.86:0/3126903155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69201085c0 msgr2=0x7f69201089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.303+0000 7f6926400640 1 --2- 172.21.15.86:0/3126903155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69201085c0 0x7f69201089e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6910009ca0 tx=0x7f691002c100 comp rx=0 tx=0).stop 2024-04-17T07:20:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.304+0000 7f6926400640 1 -- 172.21.15.86:0/3126903155 shutdown_connections 2024-04-17T07:20:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.304+0000 7f6926400640 1 --2- 172.21.15.86:0/3126903155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f692010a170 0x7f6920110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.304+0000 7f6926400640 1 --2- 172.21.15.86:0/3126903155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69201097b0 0x7f6920109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.304+0000 7f6926400640 1 --2- 172.21.15.86:0/3126903155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69201085c0 0x7f69201089e0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.304+0000 7f6926400640 1 -- 172.21.15.86:0/3126903155 >> 172.21.15.86:0/3126903155 conn(0x7f6920077560 msgr2=0x7f69200799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:50.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.305+0000 7f6926400640 1 -- 172.21.15.86:0/3126903155 shutdown_connections 2024-04-17T07:20:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.305+0000 7f6926400640 1 -- 172.21.15.86:0/3126903155 wait complete. 2024-04-17T07:20:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.307+0000 7f6926400640 1 Processor -- start 2024-04-17T07:20:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.308+0000 7f6926400640 1 -- start start 2024-04-17T07:20:50.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.308+0000 7f6926400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69201097b0 0x7f692019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.308+0000 7f6926400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f692010a170 0x7f692019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.308+0000 7f6926400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69201a4040 0x7f69201a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.308+0000 7f6926400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6920114130 con 0x7f69201a4040 2024-04-17T07:20:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.308+0000 7f6926400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6920113fb0 con 0x7f69201097b0 2024-04-17T07:20:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.308+0000 7f6926400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69201142b0 con 0x7f692010a170 2024-04-17T07:20:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.309+0000 7f691fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69201a4040 0x7f69201a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.309+0000 7f691f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69201097b0 0x7f692019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.309+0000 7f691f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69201097b0 0x7f692019eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41230/0 (socket says 172.21.15.86:41230) 2024-04-17T07:20:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.309+0000 7f691f400640 1 -- 172.21.15.86:0/2899935521 learned_addr learned my addr 172.21.15.86:0/2899935521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.309+0000 7f6916a00640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f692010a170 0x7f692019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.309+0000 7f691f400640 1 -- 172.21.15.86:0/2899935521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f692010a170 msgr2=0x7f692019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.309+0000 7f691f400640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f692010a170 0x7f692019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.309+0000 7f691f400640 1 -- 172.21.15.86:0/2899935521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69201a4040 msgr2=0x7f69201a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.309+0000 7f691f400640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69201a4040 0x7f69201a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.310+0000 7f691f400640 1 -- 172.21.15.86:0/2899935521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69201a6aa0 con 0x7f69201097b0 2024-04-17T07:20:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.310+0000 7f691fe00640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69201a4040 0x7f69201a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.311+0000 7f691f400640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69201097b0 0x7f692019eaa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f691002c690 tx=0x7f6910035ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.312+0000 7f691cc00640 1 -- 172.21.15.86:0/2899935521 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910039040 con 0x7f69201097b0 2024-04-17T07:20:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.312+0000 7f6926400640 1 -- 172.21.15.86:0/2899935521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69201a6d30 con 0x7f69201097b0 2024-04-17T07:20:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.313+0000 7f691cc00640 1 -- 172.21.15.86:0/2899935521 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6910044070 con 0x7f69201097b0 2024-04-17T07:20:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.313+0000 7f6926400640 1 -- 172.21.15.86:0/2899935521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69201a71a0 con 0x7f69201097b0 2024-04-17T07:20:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.313+0000 7f691cc00640 1 -- 172.21.15.86:0/2899935521 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910039040 con 0x7f69201097b0 2024-04-17T07:20:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.320+0000 7f692916e740 1 Processor -- start 2024-04-17T07:20:50.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.321+0000 7f692916e740 1 -- start start 2024-04-17T07:20:50.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.321+0000 7f692916e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 0x55607685c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.321+0000 7f692916e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076861680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.321+0000 7f692916e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076861cf0 0x556076864180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.321+0000 7f692916e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560767c3120 con 0x55607685cb80 2024-04-17T07:20:50.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.321+0000 7f692916e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556076769340 con 0x556076861cf0 2024-04-17T07:20:50.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.321+0000 7f692916e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560767bf0e0 con 0x55607685c190 2024-04-17T07:20:50.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.322+0000 7f691cc00640 1 -- 172.21.15.86:0/2899935521 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f6910003970 con 0x7f69201097b0 2024-04-17T07:20:50.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.322+0000 7f6916a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076861cf0 0x556076864180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.322+0000 7f6916a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076861cf0 0x556076864180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41238/0 (socket says 172.21.15.86:41238) 2024-04-17T07:20:50.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.322+0000 7f6916a00640 1 -- 172.21.15.86:0/607507146 learned_addr learned my addr 172.21.15.86:0/607507146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:50.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.322+0000 7f691f400640 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076861680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.322+0000 7f691fe00640 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 0x55607685c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.323+0000 7f6916a00640 1 -- 172.21.15.86:0/607507146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 msgr2=0x55607685c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:50.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.323+0000 7f6916a00640 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 0x55607685c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.323+0000 7f6916a00640 1 -- 172.21.15.86:0/607507146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 msgr2=0x556076861680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.324+0000 7f6916a00640 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076861680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.324+0000 7f6916a00640 1 -- 172.21.15.86:0/607507146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55607659f7d0 con 0x556076861cf0 2024-04-17T07:20:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.324+0000 7f691fe00640 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 0x55607685c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.324+0000 7f691f400640 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076861680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.324+0000 7f691cc00640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f69040a3e70 0x7f69040a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.325+0000 7f691cc00640 1 -- 172.21.15.86:0/2899935521 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f69100f2550 con 0x7f69201097b0 2024-04-17T07:20:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.325+0000 7f691fe00640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f69040a3e70 0x7f69040a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.325+0000 7f6916a00640 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076861cf0 0x556076864180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f690800bc80 tx=0x7f690800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8475d17a94939a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:50.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.325+0000 7f6914c00640 1 -- 172.21.15.86:0/607507146 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908015070 con 0x556076861cf0 2024-04-17T07:20:50.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.326+0000 7f6914c00640 1 -- 172.21.15.86:0/607507146 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f690801a910 con 0x556076861cf0 2024-04-17T07:20:50.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.326+0000 7f6914c00640 1 -- 172.21.15.86:0/607507146 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69080118d0 con 0x556076861cf0 2024-04-17T07:20:50.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.327+0000 7f692916e740 1 -- 172.21.15.86:0/607507146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076861cf0 msgr2=0x556076864180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.327+0000 7f692916e740 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076861cf0 0x556076864180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f690800bc80 tx=0x7f690800f730 comp rx=0 tx=0).stop 2024-04-17T07:20:50.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.327+0000 7f691fe00640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f69040a3e70 0x7f69040a6330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55607685d3e0 tx=0x7f6918005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:50.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.327+0000 7f692916e740 1 -- 172.21.15.86:0/607507146 shutdown_connections 2024-04-17T07:20:50.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.327+0000 7f692916e740 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076861cf0 0x556076864180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.327+0000 7f692916e740 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076861680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.327+0000 7f692916e740 1 --2- 172.21.15.86:0/607507146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 0x55607685c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.327+0000 7f692916e740 1 -- 172.21.15.86:0/607507146 >> 172.21.15.86:0/607507146 conn(0x556076857700 msgr2=0x556076857b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:50.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.327+0000 7f692916e740 1 -- 172.21.15.86:0/607507146 shutdown_connections 2024-04-17T07:20:50.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.327+0000 7f692916e740 1 -- 172.21.15.86:0/607507146 wait complete. 2024-04-17T07:20:50.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.328+0000 7f692916e740 1 -- start start 2024-04-17T07:20:50.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.329+0000 7f692916e740 1 Processor -- start 2024-04-17T07:20:50.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.330+0000 7f692916e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 0x556076866730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.330+0000 7f692916e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607685cb80 0x556076866c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.330+0000 7f692916e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556076861cf0 0x55607686dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.330+0000 7f692916e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556076864ca0 con 0x55607685c190 2024-04-17T07:20:50.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.330+0000 7f692916e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55607677c710 con 0x55607685cb80 2024-04-17T07:20:50.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.330+0000 7f692916e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556076864e20 con 0x556076861cf0 2024-04-17T07:20:50.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.330+0000 7f692916e740 10 client.0 _ll_register_callbacks cb 0x556076777af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.331+0000 7f691f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 0x556076866730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.331+0000 7f691f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 0x556076866730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47494/0 (socket says 172.21.15.86:47494) 2024-04-17T07:20:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.331+0000 7f691f400640 1 -- 172.21.15.86:0/1073479480 learned_addr learned my addr 172.21.15.86:0/1073479480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.331+0000 7f6916a00640 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607685cb80 0x556076866c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.331+0000 7f691f400640 1 -- 172.21.15.86:0/1073479480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556076861cf0 msgr2=0x55607686dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.331+0000 7f691f400640 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556076861cf0 0x55607686dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.331+0000 7f691f400640 1 -- 172.21.15.86:0/1073479480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607685cb80 msgr2=0x556076866c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.331+0000 7f691f400640 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607685cb80 0x556076866c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.331+0000 7f691f400640 1 -- 172.21.15.86:0/1073479480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560764d53f0 con 0x55607685c190 2024-04-17T07:20:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.332+0000 7f691f400640 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 0x556076866730 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6910037040 tx=0x7f69100ba120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b191052b099342c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:50.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.332+0000 7f690d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:20:50.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.332+0000 7f690d600640 1 -- 172.21.15.86:0/1073479480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69100bde40 con 0x55607685c190 2024-04-17T07:20:50.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.332+0000 7f690d600640 1 -- 172.21.15.86:0/1073479480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f69100bc580 con 0x55607685c190 2024-04-17T07:20:50.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.333+0000 7f692916e740 1 -- 172.21.15.86:0/1073479480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f690801a910 con 0x55607685c190 2024-04-17T07:20:50.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.334+0000 7f690d600640 1 -- 172.21.15.86:0/1073479480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f69100327c0 con 0x55607685c190 2024-04-17T07:20:50.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.335+0000 7f692916e740 10 client.17118 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:50.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.335+0000 7f692916e740 1 -- 172.21.15.86:0/1073479480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560764950f0 con 0x55607685c190 2024-04-17T07:20:50.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.336+0000 7f690d600640 1 -- 172.21.15.86:0/1073479480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f690801a910 con 0x55607685c190 2024-04-17T07:20:50.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.336+0000 7f690d600640 1 -- 172.21.15.86:0/1073479480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f691003c030 con 0x55607685c190 2024-04-17T07:20:50.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.336+0000 7f692916e740 10 client.17118 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.336+0000 7f692916e740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.337+0000 7f692916e740 20 client.17118 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.337+0000 7f692916e740 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556076872490 0x556076874910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.337+0000 7f692916e740 4 client.17118 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.337+0000 7f692916e740 1 -- 172.21.15.86:0/1073479480 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560767d0b40 con 0x556076872490 2024-04-17T07:20:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.337+0000 7f6916a00640 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556076872490 0x556076874910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.338+0000 7f6916a00640 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556076872490 0x556076874910 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1eaa0d4de22f4cb server_cookie=69195823596f172a in_seq=0 out_seq=0 2024-04-17T07:20:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.338+0000 7f690d600640 10 client.17118 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.340+0000 7f690d600640 1 -- 172.21.15.86:0/1073479480 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560767d0b40 con 0x556076872490 2024-04-17T07:20:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.340+0000 7f690d600640 10 client.17118 handle_command_reply: tid=0 2024-04-17T07:20:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.340+0000 7f692916e740 1 client.17118 shutdown 2024-04-17T07:20:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.341+0000 7f692916e740 1 -- 172.21.15.86:0/1073479480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 msgr2=0x556076866730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.341+0000 7f692916e740 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 0x556076866730 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6910037040 tx=0x7f69100ba120 comp rx=0 tx=0).stop 2024-04-17T07:20:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.341+0000 7f692916e740 1 -- 172.21.15.86:0/1073479480 shutdown_connections 2024-04-17T07:20:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.341+0000 7f692916e740 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556076872490 0x556076874910 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.341+0000 7f692916e740 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556076861cf0 0x55607686dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.341+0000 7f692916e740 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607685cb80 0x556076866c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.341+0000 7f690d600640 0 client.17118 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.341+0000 7f692916e740 1 --2- 172.21.15.86:0/1073479480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 0x556076866730 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.341+0000 7f692916e740 1 -- 172.21.15.86:0/1073479480 >> 172.21.15.86:0/1073479480 conn(0x556076857700 msgr2=0x556076856890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.342+0000 7f692916e740 1 -- 172.21.15.86:0/1073479480 shutdown_connections 2024-04-17T07:20:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.342+0000 7f692916e740 1 -- 172.21.15.86:0/1073479480 wait complete. 2024-04-17T07:20:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.343+0000 7f692916e740 20 client.17118 trim_cache size 0 max 16384 2024-04-17T07:20:50.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.383+0000 7f692916e740 1 Processor -- start 2024-04-17T07:20:50.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.383+0000 7f692916e740 1 -- start start 2024-04-17T07:20:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.384+0000 7f692916e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x556076856000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.384+0000 7f692916e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 0x55607686dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.384+0000 7f692916e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076856540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.384+0000 7f692916e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc00fc20 con 0x55607685cb80 2024-04-17T07:20:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.384+0000 7f692916e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc011760 con 0x556076857700 2024-04-17T07:20:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.384+0000 7f692916e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc0102f0 con 0x55607685c190 2024-04-17T07:20:50.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.385+0000 7f691f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x556076856000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.385+0000 7f691fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076856540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.385+0000 7f691fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076856540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47496/0 (socket says 172.21.15.86:47496) 2024-04-17T07:20:50.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.385+0000 7f691f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x556076856000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41252/0 (socket says 172.21.15.86:41252) 2024-04-17T07:20:50.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.385+0000 7f691fe00640 1 -- 172.21.15.86:0/2269110483 learned_addr learned my addr 172.21.15.86:0/2269110483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:50.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.385+0000 7f6916a00640 1 --2- 172.21.15.86:0/2269110483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 0x55607686dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.386+0000 7f691fe00640 1 -- 172.21.15.86:0/2269110483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 msgr2=0x55607686dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:50.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.386+0000 7f691fe00640 1 --2- 172.21.15.86:0/2269110483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 0x55607686dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.386+0000 7f691fe00640 1 -- 172.21.15.86:0/2269110483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 msgr2=0x556076856000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:50.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.386+0000 7f691fe00640 1 --2- 172.21.15.86:0/2269110483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x556076856000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.386+0000 7f691fe00640 1 -- 172.21.15.86:0/2269110483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560768657f0 con 0x55607685cb80 2024-04-17T07:20:50.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.386+0000 7f691fe00640 1 --2- 172.21.15.86:0/2269110483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076856540 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6918012480 tx=0x7f6918014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09ed51fd4fbbd9c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:50.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.387+0000 7f690fe00640 1 -- 172.21.15.86:0/2269110483 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918023070 con 0x55607685cb80 2024-04-17T07:20:50.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.387+0000 7f690fe00640 1 -- 172.21.15.86:0/2269110483 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6918004250 con 0x55607685cb80 2024-04-17T07:20:50.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.388+0000 7f692916e740 1 -- 172.21.15.86:0/2269110483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 msgr2=0x556076856540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.388+0000 7f690fe00640 1 -- 172.21.15.86:0/2269110483 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918023070 con 0x55607685cb80 2024-04-17T07:20:50.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.388+0000 7f692916e740 1 --2- 172.21.15.86:0/2269110483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076856540 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6918012480 tx=0x7f6918014070 comp rx=0 tx=0).stop 2024-04-17T07:20:50.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.389+0000 7f692916e740 1 -- 172.21.15.86:0/2269110483 shutdown_connections 2024-04-17T07:20:50.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.389+0000 7f692916e740 1 --2- 172.21.15.86:0/2269110483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685cb80 0x556076856540 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.389+0000 7f692916e740 1 --2- 172.21.15.86:0/2269110483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685c190 0x55607686dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.389+0000 7f692916e740 1 --2- 172.21.15.86:0/2269110483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x556076856000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.389+0000 7f692916e740 1 -- 172.21.15.86:0/2269110483 >> 172.21.15.86:0/2269110483 conn(0x556076857e50 msgr2=0x5560768666f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:50.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.390+0000 7f692916e740 1 -- 172.21.15.86:0/2269110483 shutdown_connections 2024-04-17T07:20:50.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.390+0000 7f692916e740 1 -- 172.21.15.86:0/2269110483 wait complete. 2024-04-17T07:20:50.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.391+0000 7f692916e740 1 -- start start 2024-04-17T07:20:50.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.391+0000 7f692916e740 1 Processor -- start 2024-04-17T07:20:50.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.392+0000 7f692916e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x55607686b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.393+0000 7f692916e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 0x55607686beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.393+0000 7f692916e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685cb80 0x55607686ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.393+0000 7f692916e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc011090 con 0x55607685c190 2024-04-17T07:20:50.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.393+0000 7f692916e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc0109c0 con 0x556076857700 2024-04-17T07:20:50.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.393+0000 7f692916e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc00dbf0 con 0x55607685cb80 2024-04-17T07:20:50.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.393+0000 7f692916e740 10 client.0 _ll_register_callbacks cb 0x7f69100347b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:50.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.393+0000 7f691f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x55607686b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.393+0000 7f691f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x55607686b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41260/0 (socket says 172.21.15.86:41260) 2024-04-17T07:20:50.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.393+0000 7f691f400640 1 -- 172.21.15.86:0/53403542 learned_addr learned my addr 172.21.15.86:0/53403542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:50.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.394+0000 7f6916a00640 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 0x55607686beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.394+0000 7f691fe00640 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685cb80 0x55607686ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.394+0000 7f691f400640 1 -- 172.21.15.86:0/53403542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685cb80 msgr2=0x55607686ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:50.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.394+0000 7f691f400640 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685cb80 0x55607686ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.394+0000 7f691f400640 1 -- 172.21.15.86:0/53403542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 msgr2=0x55607686beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:50.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.394+0000 7f691f400640 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 0x55607686beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.394+0000 7f691f400640 1 -- 172.21.15.86:0/53403542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55607686d410 con 0x556076857700 2024-04-17T07:20:50.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.395+0000 7f691f400640 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x55607686b970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f69100be1e0 tx=0x7f691002c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c723c6a35f0030df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:50.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.395+0000 7f690d600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:20:50.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.396+0000 7f690d600640 1 -- 172.21.15.86:0/53403542 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69100bde40 con 0x556076857700 2024-04-17T07:20:50.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.397+0000 7f690d600640 1 -- 172.21.15.86:0/53403542 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f69100d7bd0 con 0x556076857700 2024-04-17T07:20:50.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.397+0000 7f692916e740 1 -- 172.21.15.86:0/53403542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6918004250 con 0x556076857700 2024-04-17T07:20:50.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.397+0000 7f690d600640 1 -- 172.21.15.86:0/53403542 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f69100be2b0 con 0x556076857700 2024-04-17T07:20:50.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.398+0000 7f692916e740 10 client.36113 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:50.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.398+0000 7f692916e740 1 -- 172.21.15.86:0/53403542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560768d9080 con 0x556076857700 2024-04-17T07:20:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.399+0000 7f690d600640 1 -- 172.21.15.86:0/53403542 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6918004250 con 0x556076857700 2024-04-17T07:20:50.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.399+0000 7f690d600640 1 -- 172.21.15.86:0/53403542 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f69100dd120 con 0x556076857700 2024-04-17T07:20:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.400+0000 7f692916e740 10 client.36113 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.400+0000 7f692916e740 10 client.36113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.400+0000 7f692916e740 20 client.36113 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.400+0000 7f692916e740 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556076861cf0 0x5560768db850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.400+0000 7f692916e740 4 client.36113 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:20:50.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.400+0000 7f692916e740 1 -- 172.21.15.86:0/53403542 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5560767ca9b0 con 0x556076861cf0 2024-04-17T07:20:50.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.401+0000 7f6916a00640 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556076861cf0 0x5560768db850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.404+0000 7f6916a00640 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556076861cf0 0x5560768db850 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9adaba9f30bbfafe server_cookie=3fed3c0ce04ec7a1 in_seq=0 out_seq=0 2024-04-17T07:20:50.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.404+0000 7f690d600640 10 client.36113 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:50.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.405+0000 7f690d600640 1 -- 172.21.15.86:0/53403542 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5560767ca9b0 con 0x556076861cf0 2024-04-17T07:20:50.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.405+0000 7f690d600640 10 client.36113 handle_command_reply: tid=0 2024-04-17T07:20:50.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.405+0000 7f692916e740 1 client.36113 shutdown 2024-04-17T07:20:50.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f692916e740 1 -- 172.21.15.86:0/53403542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 msgr2=0x55607686b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f692916e740 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x55607686b970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f69100be1e0 tx=0x7f691002c780 comp rx=0 tx=0).stop 2024-04-17T07:20:50.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f692916e740 1 -- 172.21.15.86:0/53403542 shutdown_connections 2024-04-17T07:20:50.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f692916e740 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556076861cf0 0x5560768db850 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f692916e740 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607685cb80 0x55607686ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f692916e740 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607685c190 0x55607686beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f692916e740 1 --2- 172.21.15.86:0/53403542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556076857700 0x55607686b970 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f69100be1e0 tx=0x7f691002c780 comp rx=0 tx=0).stop 2024-04-17T07:20:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f692916e740 1 -- 172.21.15.86:0/53403542 >> 172.21.15.86:0/53403542 conn(0x556076857e50 msgr2=0x556076856000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f690d600640 0 client.36113 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f692916e740 1 -- 172.21.15.86:0/53403542 shutdown_connections 2024-04-17T07:20:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.406+0000 7f692916e740 1 -- 172.21.15.86:0/53403542 wait complete. 2024-04-17T07:20:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.407+0000 7f692916e740 20 client.36113 trim_cache size 0 max 16384 2024-04-17T07:20:50.408 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:20:50.408 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:20:50.408 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:20:50.408 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:20:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.409+0000 7f6926400640 1 -- 172.21.15.86:0/2899935521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f69040a3e70 msgr2=0x7f69040a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.409+0000 7f6926400640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f69040a3e70 0x7f69040a6330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55607685d3e0 tx=0x7f6918005f90 comp rx=0 tx=0).stop 2024-04-17T07:20:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.409+0000 7f6926400640 1 -- 172.21.15.86:0/2899935521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69201097b0 msgr2=0x7f692019eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.409+0000 7f6926400640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69201097b0 0x7f692019eaa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f691002c690 tx=0x7f6910035ce0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.410+0000 7f6926400640 1 -- 172.21.15.86:0/2899935521 shutdown_connections 2024-04-17T07:20:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.410+0000 7f6926400640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f69040a3e70 0x7f69040a6330 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.410+0000 7f6926400640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69201a4040 0x7f69201a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.410+0000 7f6926400640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f692010a170 0x7f692019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.410+0000 7f6926400640 1 --2- 172.21.15.86:0/2899935521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69201097b0 0x7f692019eaa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.410+0000 7f6926400640 1 -- 172.21.15.86:0/2899935521 >> 172.21.15.86:0/2899935521 conn(0x7f6920077560 msgr2=0x7f692010cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.411+0000 7f6926400640 1 -- 172.21.15.86:0/2899935521 shutdown_connections 2024-04-17T07:20:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.411+0000 7f6926400640 1 -- 172.21.15.86:0/2899935521 wait complete. 2024-04-17T07:20:50.444 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:20:50.445 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:20:50.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.778+0000 7ff4ae800640 1 -- 172.21.15.86:0/417626340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a8105b70 msgr2=0x7ff4a8105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.778+0000 7ff4ae800640 1 --2- 172.21.15.86:0/417626340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a8105b70 0x7ff4a8105ff0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009ca0 tx=0x7ff4a002c100 comp rx=0 tx=0).stop 2024-04-17T07:20:50.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.778+0000 7ff4ae800640 1 -- 172.21.15.86:0/417626340 shutdown_connections 2024-04-17T07:20:50.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.778+0000 7ff4ae800640 1 --2- 172.21.15.86:0/417626340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a8105b70 0x7ff4a8105ff0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.778+0000 7ff4ae800640 1 --2- 172.21.15.86:0/417626340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4a81031c0 0x7ff4a8105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.778+0000 7ff4ae800640 1 --2- 172.21.15.86:0/417626340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a8100850 0x7ff4a8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.778+0000 7ff4ae800640 1 -- 172.21.15.86:0/417626340 >> 172.21.15.86:0/417626340 conn(0x7ff4a80fc450 msgr2=0x7ff4a80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:50.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.778+0000 7ff4ae800640 1 -- 172.21.15.86:0/417626340 shutdown_connections 2024-04-17T07:20:50.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.778+0000 7ff4ae800640 1 -- 172.21.15.86:0/417626340 wait complete. 2024-04-17T07:20:50.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.780+0000 7ff4ae800640 1 Processor -- start 2024-04-17T07:20:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.780+0000 7ff4ae800640 1 -- start start 2024-04-17T07:20:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.781+0000 7ff4ae800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a8100850 0x7ff4a810be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.781+0000 7ff4ae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a81031c0 0x7ff4a810c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.782+0000 7ff4ae800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4a8105b70 0x7ff4a81ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.782+0000 7ff4a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a81031c0 0x7ff4a810c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.782+0000 7ff4ae800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a8114100 con 0x7ff4a81031c0 2024-04-17T07:20:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.782+0000 7ff4ae800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a8113f80 con 0x7ff4a8105b70 2024-04-17T07:20:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.782+0000 7ff4ae800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a8114280 con 0x7ff4a8100850 2024-04-17T07:20:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.782+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4a8105b70 0x7ff4a81ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.782+0000 7ff4a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a81031c0 0x7ff4a810c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47522/0 (socket says 172.21.15.86:47522) 2024-04-17T07:20:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.782+0000 7ff4a6a00640 1 -- 172.21.15.86:0/1463771774 learned_addr learned my addr 172.21.15.86:0/1463771774 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:50.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.782+0000 7ff4a7e00640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4a8105b70 0x7ff4a81ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41276/0 (socket says 172.21.15.86:41276) 2024-04-17T07:20:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.782+0000 7ff4a7400640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a8100850 0x7ff4a810be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.783+0000 7ff4a6a00640 1 -- 172.21.15.86:0/1463771774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a8100850 msgr2=0x7ff4a810be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.783+0000 7ff4a6a00640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a8100850 0x7ff4a810be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.783+0000 7ff4a6a00640 1 -- 172.21.15.86:0/1463771774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4a8105b70 msgr2=0x7ff4a81ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.783+0000 7ff4a6a00640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4a8105b70 0x7ff4a81ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:50.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.783+0000 7ff4a6a00640 1 -- 172.21.15.86:0/1463771774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a81aed50 con 0x7ff4a81031c0 2024-04-17T07:20:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.785+0000 7ff4a6a00640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a81031c0 0x7ff4a810c370 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff49400d950 tx=0x7ff49400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.785+0000 7ff49fe00640 1 -- 172.21.15.86:0/1463771774 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494017030 con 0x7ff4a81031c0 2024-04-17T07:20:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.785+0000 7ff4ae800640 1 -- 172.21.15.86:0/1463771774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a81aef20 con 0x7ff4a81031c0 2024-04-17T07:20:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.785+0000 7ff49fe00640 1 -- 172.21.15.86:0/1463771774 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff494004020 con 0x7ff4a81031c0 2024-04-17T07:20:50.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.785+0000 7ff49fe00640 1 -- 172.21.15.86:0/1463771774 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4940047a0 con 0x7ff4a81031c0 2024-04-17T07:20:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.786+0000 7ff4ae800640 1 -- 172.21.15.86:0/1463771774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a81af380 con 0x7ff4a81031c0 2024-04-17T07:20:50.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.787+0000 7ff4ae800640 1 -- 172.21.15.86:0/1463771774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4880052c0 con 0x7ff4a81031c0 2024-04-17T07:20:50.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.792+0000 7ff49fe00640 1 -- 172.21.15.86:0/1463771774 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff4940041d0 con 0x7ff4a81031c0 2024-04-17T07:20:50.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.797+0000 7ff49fe00640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff48c0a3c30 0x7ff48c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.798+0000 7ff49fe00640 1 -- 172.21.15.86:0/1463771774 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff4940d0110 con 0x7ff4a81031c0 2024-04-17T07:20:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.799+0000 7ff4a7400640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff48c0a3c30 0x7ff48c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:50.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.801+0000 7ff4a7400640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff48c0a3c30 0x7ff48c0a60f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff498009bd0 tx=0x7ff498005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:50.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:50.802+0000 7ff49fe00640 1 -- 172.21.15.86:0/1463771774 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff49409c710 con 0x7ff4a81031c0 2024-04-17T07:20:51.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.310+0000 7ff4ae800640 1 -- 172.21.15.86:0/1463771774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4880055b0 con 0x7ff4a81031c0 2024-04-17T07:20:51.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.312+0000 7ff49fe00640 1 -- 172.21.15.86:0/1463771774 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7ff49409c010 con 0x7ff4a81031c0 2024-04-17T07:20:51.313 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:20:51.313 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:20:51.313 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:20:51.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.316+0000 7ff4ae800640 1 -- 172.21.15.86:0/1463771774 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff48c0a3c30 msgr2=0x7ff48c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.316+0000 7ff4ae800640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff48c0a3c30 0x7ff48c0a60f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff498009bd0 tx=0x7ff498005f90 comp rx=0 tx=0).stop 2024-04-17T07:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.316+0000 7ff4ae800640 1 -- 172.21.15.86:0/1463771774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a81031c0 msgr2=0x7ff4a810c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.316+0000 7ff4ae800640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a81031c0 0x7ff4a810c370 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff49400d950 tx=0x7ff49400de10 comp rx=0 tx=0).stop 2024-04-17T07:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.317+0000 7ff4ae800640 1 -- 172.21.15.86:0/1463771774 shutdown_connections 2024-04-17T07:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.317+0000 7ff4ae800640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff48c0a3c30 0x7ff48c0a60f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.317+0000 7ff4ae800640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4a8105b70 0x7ff4a81ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.317+0000 7ff4ae800640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a81031c0 0x7ff4a810c370 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.317+0000 7ff4ae800640 1 --2- 172.21.15.86:0/1463771774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a8100850 0x7ff4a810be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.317+0000 7ff4ae800640 1 -- 172.21.15.86:0/1463771774 >> 172.21.15.86:0/1463771774 conn(0x7ff4a80fc450 msgr2=0x7ff4a8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:51.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.317+0000 7ff4ae800640 1 -- 172.21.15.86:0/1463771774 shutdown_connections 2024-04-17T07:20:51.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.318+0000 7ff4ae800640 1 -- 172.21.15.86:0/1463771774 wait complete. 2024-04-17T07:20:51.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:20:51.449 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:51 smithi086 ceph-mon[31733]: pgmap v634: 129 pgs: 129 active+clean; 964 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 672 op/s 2024-04-17T07:20:51.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:51 smithi098 ceph-mon[35418]: pgmap v634: 129 pgs: 129 active+clean; 964 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 672 op/s 2024-04-17T07:20:51.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:51 smithi103 ceph-mon[35495]: pgmap v634: 129 pgs: 129 active+clean; 964 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 672 op/s 2024-04-17T07:20:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.686+0000 7f546c600640 1 -- 172.21.15.86:0/2814954459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5464077b60 msgr2=0x7f5464077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.686+0000 7f546c600640 1 --2- 172.21.15.86:0/2814954459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5464077b60 0x7f5464077fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5450009860 tx=0x7f545002bb70 comp rx=0 tx=0).stop 2024-04-17T07:20:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.686+0000 7f546c600640 1 -- 172.21.15.86:0/2814954459 shutdown_connections 2024-04-17T07:20:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.686+0000 7f546c600640 1 --2- 172.21.15.86:0/2814954459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5464078500 0x7f5464078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.686+0000 7f546c600640 1 --2- 172.21.15.86:0/2814954459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5464077b60 0x7f5464077fc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.686+0000 7f546c600640 1 --2- 172.21.15.86:0/2814954459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5464076920 0x7f5464076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.686+0000 7f546c600640 1 -- 172.21.15.86:0/2814954459 >> 172.21.15.86:0/2814954459 conn(0x7f54640ff750 msgr2=0x7f5464101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.686+0000 7f546c600640 1 -- 172.21.15.86:0/2814954459 shutdown_connections 2024-04-17T07:20:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.686+0000 7f546c600640 1 -- 172.21.15.86:0/2814954459 wait complete. 2024-04-17T07:20:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.687+0000 7f546c600640 1 Processor -- start 2024-04-17T07:20:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.687+0000 7f546c600640 1 -- start start 2024-04-17T07:20:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f546c600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5464076920 0x7f546419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f546c600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5464077b60 0x7f546419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f546c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5464078500 0x7f54641a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f546c600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54641140e0 con 0x7f5464078500 2024-04-17T07:20:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f546c600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5464113f60 con 0x7f5464076920 2024-04-17T07:20:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f546c600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5464114260 con 0x7f5464077b60 2024-04-17T07:20:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f546a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5464078500 0x7f54641a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f546a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5464078500 0x7f54641a5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47534/0 (socket says 172.21.15.86:47534) 2024-04-17T07:20:51.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f546a600640 1 -- 172.21.15.86:0/3655141202 learned_addr learned my addr 172.21.15.86:0/3655141202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:51.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f5469200640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5464077b60 0x7f546419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.688+0000 7f5469c00640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5464076920 0x7f546419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.689+0000 7f546a600640 1 -- 172.21.15.86:0/3655141202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5464077b60 msgr2=0x7f546419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.689+0000 7f546a600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5464077b60 0x7f546419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.689+0000 7f546a600640 1 -- 172.21.15.86:0/3655141202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5464076920 msgr2=0x7f546419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.689+0000 7f546a600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5464076920 0x7f546419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.689+0000 7f546a600640 1 -- 172.21.15.86:0/3655141202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54641a6440 con 0x7f5464078500 2024-04-17T07:20:51.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.689+0000 7f5469c00640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5464076920 0x7f546419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:51.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.690+0000 7f546a600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5464078500 0x7f54641a5dd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f545400e970 tx=0x7f545400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:51.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.691+0000 7f5462000640 1 -- 172.21.15.86:0/3655141202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545400ccc0 con 0x7f5464078500 2024-04-17T07:20:51.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.692+0000 7f546c600640 1 -- 172.21.15.86:0/3655141202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54641a6730 con 0x7f5464078500 2024-04-17T07:20:51.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.692+0000 7f546c600640 1 -- 172.21.15.86:0/3655141202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54641a6c50 con 0x7f5464078500 2024-04-17T07:20:51.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.693+0000 7f5462000640 1 -- 172.21.15.86:0/3655141202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f545400ce60 con 0x7f5464078500 2024-04-17T07:20:51.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.693+0000 7f5462000640 1 -- 172.21.15.86:0/3655141202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454010430 con 0x7f5464078500 2024-04-17T07:20:51.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.697+0000 7f546f40a740 1 Processor -- start 2024-04-17T07:20:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.700+0000 7f5462000640 1 -- 172.21.15.86:0/3655141202 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5454004ff0 con 0x7f5464078500 2024-04-17T07:20:51.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.700+0000 7f546f40a740 1 -- start start 2024-04-17T07:20:51.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.701+0000 7f546f40a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.702+0000 7f546f40a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f65990 0x55e548f6a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.702+0000 7f546f40a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f6cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.702+0000 7f546f40a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e548ecb020 con 0x55e548f64fa0 2024-04-17T07:20:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.702+0000 7f546f40a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e548e71460 con 0x55e548f6ab00 2024-04-17T07:20:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.702+0000 7f546f40a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e548ec6fe0 con 0x55e548f65990 2024-04-17T07:20:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.702+0000 7f5469200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.702+0000 7f5469200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f653c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47548/0 (socket says 172.21.15.86:47548) 2024-04-17T07:20:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.703+0000 7f5469200640 1 -- 172.21.15.86:0/27380694 learned_addr learned my addr 172.21.15.86:0/27380694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.703+0000 7f5462000640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54480a3cb0 0x7f54480a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.702+0000 7f5469c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f65990 0x55e548f6a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.703+0000 7f5469200640 1 --2- 172.21.15.86:0/27380694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f6cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.704+0000 7f5469200640 1 -- 172.21.15.86:0/27380694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f65990 msgr2=0x55e548f6a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:51.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.704+0000 7f546a600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54480a3cb0 0x7f54480a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.704+0000 7f5469200640 1 --2- 172.21.15.86:0/27380694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f65990 0x55e548f6a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.704+0000 7f5469200640 1 -- 172.21.15.86:0/27380694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 msgr2=0x55e548f6cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.704+0000 7f5469200640 1 --2- 172.21.15.86:0/27380694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f6cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.704+0000 7f5469200640 1 -- 172.21.15.86:0/27380694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e548e79040 con 0x55e548f64fa0 2024-04-17T07:20:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.704+0000 7f5469200640 1 --2- 172.21.15.86:0/27380694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f653c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5450000c00 tx=0x7f5450005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65aefe9f70e2e927 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.705+0000 7f546a600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54480a3cb0 0x7f54480a6170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f54540a51d0 tx=0x7f5454009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:51.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.705+0000 7f545aa00640 1 -- 172.21.15.86:0/27380694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450048070 con 0x55e548f64fa0 2024-04-17T07:20:51.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.705+0000 7f545aa00640 1 -- 172.21.15.86:0/27380694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5450034dc0 con 0x55e548f64fa0 2024-04-17T07:20:51.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.703+0000 7f5462000640 1 -- 172.21.15.86:0/3655141202 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f54540d0b90 con 0x7f5464078500 2024-04-17T07:20:51.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.706+0000 7f545aa00640 1 -- 172.21.15.86:0/27380694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450048070 con 0x55e548f64fa0 2024-04-17T07:20:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.706+0000 7f546f40a740 1 -- 172.21.15.86:0/27380694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 msgr2=0x55e548f653c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.706+0000 7f546f40a740 1 --2- 172.21.15.86:0/27380694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f653c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5450000c00 tx=0x7f5450005a90 comp rx=0 tx=0).stop 2024-04-17T07:20:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.707+0000 7f546f40a740 1 -- 172.21.15.86:0/27380694 shutdown_connections 2024-04-17T07:20:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.707+0000 7f546f40a740 1 --2- 172.21.15.86:0/27380694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f6cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.707+0000 7f546f40a740 1 --2- 172.21.15.86:0/27380694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f65990 0x55e548f6a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.707+0000 7f546f40a740 1 --2- 172.21.15.86:0/27380694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f653c0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.707+0000 7f546f40a740 1 -- 172.21.15.86:0/27380694 >> 172.21.15.86:0/27380694 conn(0x55e548f60510 msgr2=0x55e548f60910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:51.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.707+0000 7f546f40a740 1 -- 172.21.15.86:0/27380694 shutdown_connections 2024-04-17T07:20:51.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.707+0000 7f546f40a740 1 -- 172.21.15.86:0/27380694 wait complete. 2024-04-17T07:20:51.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.709+0000 7f546f40a740 1 -- start start 2024-04-17T07:20:51.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.709+0000 7f546f40a740 1 Processor -- start 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f546f40a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f64fa0 0x55e548f6f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f546f40a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f65990 0x55e548f6fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f546f40a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f76a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f546f40a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e548f6daa0 con 0x55e548f65990 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f546f40a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e548e84830 con 0x55e548f6ab00 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f546f40a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e548f6dc20 con 0x55e548f64fa0 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f546f40a740 10 client.0 _ll_register_callbacks cb 0x55e548e52fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f5469200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f76a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f5469200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f76a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41316/0 (socket says 172.21.15.86:41316) 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f5469200640 1 -- 172.21.15.86:0/2912392188 learned_addr learned my addr 172.21.15.86:0/2912392188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.711+0000 7f5469200640 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f64fa0 0x55e548f6f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.712+0000 7f5469200640 1 -- 172.21.15.86:0/2912392188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f64fa0 msgr2=0x55e548f6f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.712+0000 7f5469200640 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f64fa0 0x55e548f6f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.712+0000 7f5469200640 1 -- 172.21.15.86:0/2912392188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f65990 msgr2=0x55e548f6fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.712+0000 7f5469200640 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f65990 0x55e548f6fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.712+0000 7f5469200640 1 -- 172.21.15.86:0/2912392188 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e548e88920 con 0x55e548f6ab00 2024-04-17T07:20:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.712+0000 7f5469200640 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f76a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f545002c0b0 tx=0x7f5450034440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=920250f1e4199d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.712+0000 7f544f400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:20:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.713+0000 7f544f400640 1 -- 172.21.15.86:0/2912392188 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450048040 con 0x55e548f6ab00 2024-04-17T07:20:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.713+0000 7f546f40a740 1 -- 172.21.15.86:0/2912392188 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5450034dc0 con 0x55e548f6ab00 2024-04-17T07:20:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.713+0000 7f544f400640 1 -- 172.21.15.86:0/2912392188 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5450004030 con 0x55e548f6ab00 2024-04-17T07:20:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.714+0000 7f544f400640 1 -- 172.21.15.86:0/2912392188 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5450039040 con 0x55e548f6ab00 2024-04-17T07:20:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.714+0000 7f546f40a740 10 client.36131 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:51.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.714+0000 7f546f40a740 1 -- 172.21.15.86:0/2912392188 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e548e567f0 con 0x55e548f6ab00 2024-04-17T07:20:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.714+0000 7f544f400640 1 -- 172.21.15.86:0/2912392188 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5450034dc0 con 0x55e548f6ab00 2024-04-17T07:20:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.715+0000 7f544f400640 1 -- 172.21.15.86:0/2912392188 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f545004e040 con 0x55e548f6ab00 2024-04-17T07:20:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.715+0000 7f546f40a740 10 client.36131 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.715+0000 7f546f40a740 10 client.36131 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:20:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.715+0000 7f546f40a740 20 client.36131 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.715+0000 7f546f40a740 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e548f7abc0 0x55e548f7cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.715+0000 7f546f40a740 4 client.36131 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:51.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.715+0000 7f546f40a740 1 -- 172.21.15.86:0/2912392188 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e548ed8a40 con 0x55e548f7abc0 2024-04-17T07:20:51.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.716+0000 7f5469c00640 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e548f7abc0 0x55e548f7cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.717+0000 7f544f400640 10 client.36131 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:20:51.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.717+0000 7f5469c00640 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e548f7abc0 0x55e548f7cfb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66ad3c5853f62b00 server_cookie=5764d357db4c7fae in_seq=0 out_seq=0 2024-04-17T07:20:51.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.719+0000 7f544f400640 1 -- 172.21.15.86:0/2912392188 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e548ed8a40 con 0x55e548f7abc0 2024-04-17T07:20:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.719+0000 7f544f400640 10 client.36131 handle_command_reply: tid=0 2024-04-17T07:20:51.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.719+0000 7f546f40a740 1 client.36131 shutdown 2024-04-17T07:20:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.720+0000 7f546f40a740 1 -- 172.21.15.86:0/2912392188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 msgr2=0x55e548f76a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.720+0000 7f546f40a740 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f76a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f545002c0b0 tx=0x7f5450034440 comp rx=0 tx=0).stop 2024-04-17T07:20:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.720+0000 7f546f40a740 1 -- 172.21.15.86:0/2912392188 shutdown_connections 2024-04-17T07:20:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.720+0000 7f546f40a740 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e548f7abc0 0x55e548f7cfb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.720+0000 7f546f40a740 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f76a60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.720+0000 7f546f40a740 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f65990 0x55e548f6fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.720+0000 7f546f40a740 1 --2- 172.21.15.86:0/2912392188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f64fa0 0x55e548f6f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.720+0000 7f546f40a740 1 -- 172.21.15.86:0/2912392188 >> 172.21.15.86:0/2912392188 conn(0x55e548f60510 msgr2=0x55e548f5f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:51.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.720+0000 7f544f400640 0 client.36131 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:20:51.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.721+0000 7f546f40a740 1 -- 172.21.15.86:0/2912392188 shutdown_connections 2024-04-17T07:20:51.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.721+0000 7f546f40a740 1 -- 172.21.15.86:0/2912392188 wait complete. 2024-04-17T07:20:51.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.721+0000 7f546f40a740 20 client.36131 trim_cache size 0 max 16384 2024-04-17T07:20:51.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.757+0000 7f546f40a740 1 Processor -- start 2024-04-17T07:20:51.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.758+0000 7f546f40a740 1 -- start start 2024-04-17T07:20:51.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.758+0000 7f546f40a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f64fa0 0x55e548f5b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.758+0000 7f546f40a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f65990 0x55e548f76980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.758+0000 7f546f40a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f63e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.758+0000 7f546f40a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f544400fb60 con 0x55e548f65990 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.758+0000 7f546f40a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f54440116a0 con 0x55e548f6ab00 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.758+0000 7f546f40a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5444010230 con 0x55e548f64fa0 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f63e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f63e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41332/0 (socket says 172.21.15.86:41332) 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469200640 1 -- 172.21.15.86:0/2706643807 learned_addr learned my addr 172.21.15.86:0/2706643807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469200640 1 --2- 172.21.15.86:0/2706643807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f64fa0 0x55e548f5b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469c00640 1 --2- 172.21.15.86:0/2706643807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f65990 0x55e548f76980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469200640 1 -- 172.21.15.86:0/2706643807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f64fa0 msgr2=0x55e548f5b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469200640 1 --2- 172.21.15.86:0/2706643807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f64fa0 0x55e548f5b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469200640 1 -- 172.21.15.86:0/2706643807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f65990 msgr2=0x55e548f76980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469200640 1 --2- 172.21.15.86:0/2706643807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f65990 0x55e548f76980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469200640 1 -- 172.21.15.86:0/2706643807 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e548bdd3f0 con 0x55e548f6ab00 2024-04-17T07:20:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.759+0000 7f5469c00640 1 --2- 172.21.15.86:0/2706643807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f65990 0x55e548f76980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:51.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.760+0000 7f5469200640 1 --2- 172.21.15.86:0/2706643807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f63e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54500095f0 tx=0x7f5450005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a79a22a8526553f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:51.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.760+0000 7f545a000640 1 -- 172.21.15.86:0/2706643807 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450009450 con 0x55e548f6ab00 2024-04-17T07:20:51.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.760+0000 7f545a000640 1 -- 172.21.15.86:0/2706643807 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f545002c9c0 con 0x55e548f6ab00 2024-04-17T07:20:51.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.760+0000 7f545a000640 1 -- 172.21.15.86:0/2706643807 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450051a90 con 0x55e548f6ab00 2024-04-17T07:20:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.761+0000 7f546f40a740 1 -- 172.21.15.86:0/2706643807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 msgr2=0x55e548f63e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.761+0000 7f546f40a740 1 --2- 172.21.15.86:0/2706643807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f63e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54500095f0 tx=0x7f5450005a90 comp rx=0 tx=0).stop 2024-04-17T07:20:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.762+0000 7f546f40a740 1 -- 172.21.15.86:0/2706643807 shutdown_connections 2024-04-17T07:20:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.762+0000 7f546f40a740 1 --2- 172.21.15.86:0/2706643807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f63e80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.762+0000 7f546f40a740 1 --2- 172.21.15.86:0/2706643807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f65990 0x55e548f76980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.762+0000 7f546f40a740 1 --2- 172.21.15.86:0/2706643807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f64fa0 0x55e548f5b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.762+0000 7f546f40a740 1 -- 172.21.15.86:0/2706643807 >> 172.21.15.86:0/2706643807 conn(0x55e548f60510 msgr2=0x55e548f63770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.762+0000 7f546f40a740 1 -- 172.21.15.86:0/2706643807 shutdown_connections 2024-04-17T07:20:51.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.762+0000 7f546f40a740 1 -- 172.21.15.86:0/2706643807 wait complete. 2024-04-17T07:20:51.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.763+0000 7f546f40a740 1 -- start start 2024-04-17T07:20:51.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.763+0000 7f546f40a740 1 Processor -- start 2024-04-17T07:20:51.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.764+0000 7f546f40a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.764+0000 7f546f40a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f65990 0x55e548f74b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.764+0000 7f546f40a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f75b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.764+0000 7f546f40a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5444010fd0 con 0x55e548f64fa0 2024-04-17T07:20:51.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.764+0000 7f546f40a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5444010900 con 0x55e548f6ab00 2024-04-17T07:20:51.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.764+0000 7f546f40a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f544400db30 con 0x55e548f65990 2024-04-17T07:20:51.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.764+0000 7f546f40a740 10 client.0 _ll_register_callbacks cb 0x7f545002cb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:51.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.764+0000 7f5469200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.764+0000 7f5469200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47590/0 (socket says 172.21.15.86:47590) 2024-04-17T07:20:51.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.764+0000 7f5469200640 1 -- 172.21.15.86:0/2886681215 learned_addr learned my addr 172.21.15.86:0/2886681215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:51.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.765+0000 7f5469c00640 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f65990 0x55e548f74b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.765+0000 7f5469200640 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f75b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.765+0000 7f5469200640 1 -- 172.21.15.86:0/2886681215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f65990 msgr2=0x55e548f74b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.765+0000 7f5469200640 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f65990 0x55e548f74b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.765+0000 7f5469200640 1 -- 172.21.15.86:0/2886681215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 msgr2=0x55e548f75b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.765+0000 7f5469200640 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f75b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.765+0000 7f5469200640 1 -- 172.21.15.86:0/2886681215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e548ca77d0 con 0x55e548f64fa0 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.765+0000 7f5469200640 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f745e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f54500346e0 tx=0x7f5450009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2077ffba4a8cbe5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.765+0000 7f544f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.766+0000 7f544f400640 1 -- 172.21.15.86:0/2886681215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450048070 con 0x55e548f64fa0 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.766+0000 7f544f400640 1 -- 172.21.15.86:0/2886681215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5450034dc0 con 0x55e548f64fa0 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.766+0000 7f546f40a740 1 -- 172.21.15.86:0/2886681215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f545002c9c0 con 0x55e548f64fa0 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.766+0000 7f544f400640 1 -- 172.21.15.86:0/2886681215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5450051810 con 0x55e548f64fa0 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.767+0000 7f546f40a740 10 client.17139 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:51.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.767+0000 7f546f40a740 1 -- 172.21.15.86:0/2886681215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e548fe1180 con 0x55e548f64fa0 2024-04-17T07:20:51.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.767+0000 7f544f400640 1 -- 172.21.15.86:0/2886681215 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f545002c9c0 con 0x55e548f64fa0 2024-04-17T07:20:51.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.767+0000 7f544f400640 1 -- 172.21.15.86:0/2886681215 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f545004e070 con 0x55e548f64fa0 2024-04-17T07:20:51.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.768+0000 7f546f40a740 10 client.17139 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:51.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.768+0000 7f546f40a740 10 client.17139 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:20:51.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.768+0000 7f546f40a740 20 client.17139 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:51.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.768+0000 7f546f40a740 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e548fe1ab0 0x55e548fe3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:51.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.768+0000 7f546f40a740 4 client.17139 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:20:51.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.768+0000 7f546f40a740 1 -- 172.21.15.86:0/2886681215 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e548ed28b0 con 0x55e548fe1ab0 2024-04-17T07:20:51.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.768+0000 7f5469c00640 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e548fe1ab0 0x55e548fe3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:51.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.769+0000 7f544f400640 10 client.17139 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:20:51.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.769+0000 7f5469c00640 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e548fe1ab0 0x55e548fe3ed0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4fabba4b67f9d4a5 server_cookie=3f122bde07b44dd0 in_seq=0 out_seq=0 2024-04-17T07:20:51.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.770+0000 7f544f400640 1 -- 172.21.15.86:0/2886681215 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e548ed28b0 con 0x55e548fe1ab0 2024-04-17T07:20:51.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.770+0000 7f544f400640 10 client.17139 handle_command_reply: tid=0 2024-04-17T07:20:51.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.770+0000 7f546f40a740 1 client.17139 shutdown 2024-04-17T07:20:51.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.771+0000 7f546f40a740 1 -- 172.21.15.86:0/2886681215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 msgr2=0x55e548f745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.771+0000 7f546f40a740 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f745e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f54500346e0 tx=0x7f5450009820 comp rx=0 tx=0).stop 2024-04-17T07:20:51.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.772+0000 7f546f40a740 1 -- 172.21.15.86:0/2886681215 shutdown_connections 2024-04-17T07:20:51.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.772+0000 7f546f40a740 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55e548fe1ab0 0x55e548fe3ed0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.772+0000 7f546f40a740 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e548f6ab00 0x55e548f75b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.772+0000 7f546f40a740 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e548f65990 0x55e548f74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.772+0000 7f546f40a740 1 --2- 172.21.15.86:0/2886681215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e548f64fa0 0x55e548f745e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.772+0000 7f546f40a740 1 -- 172.21.15.86:0/2886681215 >> 172.21.15.86:0/2886681215 conn(0x55e548f60510 msgr2=0x55e548f64000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:51.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.772+0000 7f544f400640 0 client.17139 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:20:51.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.774+0000 7f546f40a740 1 -- 172.21.15.86:0/2886681215 shutdown_connections 2024-04-17T07:20:51.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.774+0000 7f546f40a740 1 -- 172.21.15.86:0/2886681215 wait complete. 2024-04-17T07:20:51.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.775+0000 7f546f40a740 20 client.17139 trim_cache size 0 max 16384 2024-04-17T07:20:51.775 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:20:51.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.777+0000 7f546c600640 1 -- 172.21.15.86:0/3655141202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54480a3cb0 msgr2=0x7f54480a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.777+0000 7f546c600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54480a3cb0 0x7f54480a6170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f54540a51d0 tx=0x7f5454009720 comp rx=0 tx=0).stop 2024-04-17T07:20:51.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.777+0000 7f546c600640 1 -- 172.21.15.86:0/3655141202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5464078500 msgr2=0x7f54641a5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:51.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.777+0000 7f546c600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5464078500 0x7f54641a5dd0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f545400e970 tx=0x7f545400ed70 comp rx=0 tx=0).stop 2024-04-17T07:20:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.777+0000 7f546c600640 1 -- 172.21.15.86:0/3655141202 shutdown_connections 2024-04-17T07:20:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.777+0000 7f546c600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54480a3cb0 0x7f54480a6170 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.777+0000 7f546c600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5464078500 0x7f54641a5dd0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.777+0000 7f546c600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5464077b60 0x7f546419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.777+0000 7f546c600640 1 --2- 172.21.15.86:0/3655141202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5464076920 0x7f546419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.777+0000 7f546c600640 1 -- 172.21.15.86:0/3655141202 >> 172.21.15.86:0/3655141202 conn(0x7f54640ff750 msgr2=0x7f5464102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.778+0000 7f546c600640 1 -- 172.21.15.86:0/3655141202 shutdown_connections 2024-04-17T07:20:51.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:51.778+0000 7f546c600640 1 -- 172.21.15.86:0/3655141202 wait complete. 2024-04-17T07:20:51.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:20:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.111+0000 7f7a73200640 1 -- 172.21.15.86:0/478195187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a741040a0 msgr2=0x7f7a741044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.111+0000 7f7a73200640 1 --2- 172.21.15.86:0/478195187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a741040a0 0x7f7a741044c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009900 tx=0x7f7a6002bc60 comp rx=0 tx=0).stop 2024-04-17T07:20:52.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.111+0000 7f7a73200640 1 -- 172.21.15.86:0/478195187 shutdown_connections 2024-04-17T07:20:52.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.111+0000 7f7a73200640 1 --2- 172.21.15.86:0/478195187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74105c50 0x7f7a7410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.111+0000 7f7a73200640 1 --2- 172.21.15.86:0/478195187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74105290 0x7f7a74105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.111+0000 7f7a73200640 1 --2- 172.21.15.86:0/478195187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a741040a0 0x7f7a741044c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.111+0000 7f7a73200640 1 -- 172.21.15.86:0/478195187 >> 172.21.15.86:0/478195187 conn(0x7f7a740ff610 msgr2=0x7f7a74101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.112+0000 7f7a73200640 1 -- 172.21.15.86:0/478195187 shutdown_connections 2024-04-17T07:20:52.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.112+0000 7f7a73200640 1 -- 172.21.15.86:0/478195187 wait complete. 2024-04-17T07:20:52.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.113+0000 7f7a73200640 1 Processor -- start 2024-04-17T07:20:52.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.113+0000 7f7a73200640 1 -- start start 2024-04-17T07:20:52.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.114+0000 7f7a73200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a741040a0 0x7f7a7419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.114+0000 7f7a73200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74105290 0x7f7a7419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.114+0000 7f7a73200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74105c50 0x7f7a741a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.114+0000 7f7a73200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74076b00 con 0x7f7a74105c50 2024-04-17T07:20:52.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.114+0000 7f7a73200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74076980 con 0x7f7a741040a0 2024-04-17T07:20:52.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.114+0000 7f7a73200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74076c80 con 0x7f7a74105290 2024-04-17T07:20:52.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.115+0000 7f7a71e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a741040a0 0x7f7a7419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.115+0000 7f7a71e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a741040a0 0x7f7a7419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41356/0 (socket says 172.21.15.86:41356) 2024-04-17T07:20:52.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.115+0000 7f7a71e00640 1 -- 172.21.15.86:0/3317435456 learned_addr learned my addr 172.21.15.86:0/3317435456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.115+0000 7f7a72800640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74105c50 0x7f7a741a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.115+0000 7f7a71400640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74105290 0x7f7a7419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.115+0000 7f7a72800640 1 -- 172.21.15.86:0/3317435456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74105290 msgr2=0x7f7a7419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.115+0000 7f7a72800640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74105290 0x7f7a7419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.116+0000 7f7a72800640 1 -- 172.21.15.86:0/3317435456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a741040a0 msgr2=0x7f7a7419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.116+0000 7f7a72800640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a741040a0 0x7f7a7419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.116+0000 7f7a72800640 1 -- 172.21.15.86:0/3317435456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a741a6380 con 0x7f7a74105c50 2024-04-17T07:20:52.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.116+0000 7f7a72800640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74105c50 0x7f7a741a5d10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00be20 tx=0x7f7a6c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.117+0000 7f7a6aa00640 1 -- 172.21.15.86:0/3317435456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c019070 con 0x7f7a74105c50 2024-04-17T07:20:52.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.117+0000 7f7a6aa00640 1 -- 172.21.15.86:0/3317435456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6c0148f0 con 0x7f7a74105c50 2024-04-17T07:20:52.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.117+0000 7f7a6aa00640 1 -- 172.21.15.86:0/3317435456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c012810 con 0x7f7a74105c50 2024-04-17T07:20:52.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.117+0000 7f7a73200640 1 -- 172.21.15.86:0/3317435456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a741a6670 con 0x7f7a74105c50 2024-04-17T07:20:52.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.118+0000 7f7a73200640 1 -- 172.21.15.86:0/3317435456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a741a6b70 con 0x7f7a74105c50 2024-04-17T07:20:52.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.122+0000 7f7a7ba4a740 1 Processor -- start 2024-04-17T07:20:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.123+0000 7f7a6aa00640 1 -- 172.21.15.86:0/3317435456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7a6c01c070 con 0x7f7a74105c50 2024-04-17T07:20:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.124+0000 7f7a7ba4a740 1 -- start start 2024-04-17T07:20:52.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.124+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9160 0x55c50b9e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.125+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 0x55c50b9ee650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.125+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9eecc0 0x55c50b9f1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.125+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c50b94f020 con 0x55c50b9e9160 2024-04-17T07:20:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.125+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c50b8f5460 con 0x55c50b9eecc0 2024-04-17T07:20:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.125+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c50b94afe0 con 0x55c50b9e9b50 2024-04-17T07:20:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.126+0000 7f7a71e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 0x55c50b9ee650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.126+0000 7f7a71e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 0x55c50b9ee650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51192/0 (socket says 172.21.15.86:51192) 2024-04-17T07:20:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.126+0000 7f7a71e00640 1 -- 172.21.15.86:0/3339325984 learned_addr learned my addr 172.21.15.86:0/3339325984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.126+0000 7f7a71400640 1 --2- 172.21.15.86:0/3339325984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9160 0x55c50b9e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.126+0000 7f7a71400640 1 --2- 172.21.15.86:0/3339325984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9eecc0 0x55c50b9f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.127+0000 7f7a71400640 1 -- 172.21.15.86:0/3339325984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 msgr2=0x55c50b9ee650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.127+0000 7f7a71400640 1 --2- 172.21.15.86:0/3339325984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 0x55c50b9ee650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.127+0000 7f7a71400640 1 -- 172.21.15.86:0/3339325984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9eecc0 msgr2=0x55c50b9f1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.127+0000 7f7a71400640 1 --2- 172.21.15.86:0/3339325984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9eecc0 0x55c50b9f1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.127+0000 7f7a71400640 1 -- 172.21.15.86:0/3339325984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50b8fd040 con 0x55c50b9e9160 2024-04-17T07:20:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.127+0000 7f7a71400640 1 --2- 172.21.15.86:0/3339325984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9160 0x55c50b9e9580 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7a640058c0 tx=0x7f7a6400c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281fdaf2e6404fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.127+0000 7f7a6aa00640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a580a3d30 0x7f7a580a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.128+0000 7f7a5f400640 1 -- 172.21.15.86:0/3339325984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a640142d0 con 0x55c50b9e9160 2024-04-17T07:20:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.128+0000 7f7a5f400640 1 -- 172.21.15.86:0/3339325984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6400f730 con 0x55c50b9e9160 2024-04-17T07:20:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.128+0000 7f7a71e00640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a580a3d30 0x7f7a580a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.128+0000 7f7a5f400640 1 -- 172.21.15.86:0/3339325984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a640037f0 con 0x55c50b9e9160 2024-04-17T07:20:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.128+0000 7f7a6aa00640 1 -- 172.21.15.86:0/3317435456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7a6c0d0080 con 0x7f7a74105c50 2024-04-17T07:20:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.130+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3339325984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9160 msgr2=0x55c50b9e9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.130+0000 7f7a71e00640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a580a3d30 0x7f7a580a61f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55c50b9e9f50 tx=0x7f7a60000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.130+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3339325984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9160 0x55c50b9e9580 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7a640058c0 tx=0x7f7a6400c470 comp rx=0 tx=0).stop 2024-04-17T07:20:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.130+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3339325984 shutdown_connections 2024-04-17T07:20:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.130+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3339325984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9eecc0 0x55c50b9f1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.130+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3339325984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 0x55c50b9ee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.130+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3339325984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9160 0x55c50b9e9580 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.130+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3339325984 >> 172.21.15.86:0/3339325984 conn(0x55c50b9e46d0 msgr2=0x55c50b9e4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.130+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3339325984 shutdown_connections 2024-04-17T07:20:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.131+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3339325984 wait complete. 2024-04-17T07:20:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.131+0000 7f7a7ba4a740 1 -- start start 2024-04-17T07:20:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.132+0000 7f7a7ba4a740 1 Processor -- start 2024-04-17T07:20:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.133+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9160 0x55c50b9f3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.133+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9b50 0x55c50b9f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.133+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 0x55c50b9fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.133+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c50b9f1c60 con 0x55c50b9eecc0 2024-04-17T07:20:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.133+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c50b908830 con 0x55c50b9e9b50 2024-04-17T07:20:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.133+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c50b9f1de0 con 0x55c50b9e9160 2024-04-17T07:20:52.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.133+0000 7f7a71400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9b50 0x55c50b9f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.133+0000 7f7a71400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9b50 0x55c50b9f3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41374/0 (socket says 172.21.15.86:41374) 2024-04-17T07:20:52.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.133+0000 7f7a71400640 1 -- 172.21.15.86:0/3791015420 learned_addr learned my addr 172.21.15.86:0/3791015420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.133+0000 7f7a71e00640 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9160 0x55c50b9f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.134+0000 7f7a72800640 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 0x55c50b9fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.134+0000 7f7a71400640 1 -- 172.21.15.86:0/3791015420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9160 msgr2=0x55c50b9f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.134+0000 7f7a71400640 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9160 0x55c50b9f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.134+0000 7f7a71400640 1 -- 172.21.15.86:0/3791015420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 msgr2=0x55c50b9fac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.134+0000 7f7a71400640 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 0x55c50b9fac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.134+0000 7f7a71400640 1 -- 172.21.15.86:0/3791015420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50b90c920 con 0x55c50b9e9b50 2024-04-17T07:20:52.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.134+0000 7f7a72800640 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 0x55c50b9fac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:52.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.134+0000 7f7a7ba4a740 10 client.0 _ll_register_callbacks cb 0x55c50b8d6fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:52.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.134+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3791015420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a6400f730 con 0x55c50b9e9b50 2024-04-17T07:20:52.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.135+0000 7f7a71400640 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9b50 0x55c50b9f3c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c680 tx=0x7f7a64014300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dcf05528d916339 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.135+0000 7f7a53e00640 10 client.36149 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:20:52.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.136+0000 7f7a53e00640 1 -- 172.21.15.86:0/3791015420 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a640145e0 con 0x55c50b9e9b50 2024-04-17T07:20:52.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.136+0000 7f7a53e00640 1 -- 172.21.15.86:0/3791015420 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7a64004030 con 0x55c50b9e9b50 2024-04-17T07:20:52.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.137+0000 7f7a53e00640 1 -- 172.21.15.86:0/3791015420 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a64004290 con 0x55c50b9e9b50 2024-04-17T07:20:52.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.137+0000 7f7a7ba4a740 10 client.36149 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:52.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.137+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3791015420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55c50b8da7f0 con 0x55c50b9e9b50 2024-04-17T07:20:52.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.138+0000 7f7a53e00640 1 -- 172.21.15.86:0/3791015420 <== mon.2 v2:172.21.15.98:3300/0 4 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7a640035f0 con 0x55c50b9e9b50 2024-04-17T07:20:52.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.138+0000 7f7a7ba4a740 10 client.36149 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:52.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.138+0000 7f7a7ba4a740 10 client.36149 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:20:52.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.138+0000 7f7a7ba4a740 20 client.36149 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:52.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.138+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c50b9fecc0 0x55c50ba010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.138+0000 7f7a7ba4a740 4 client.36149 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:52.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.138+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3791015420 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c50b95ca40 con 0x55c50b9fecc0 2024-04-17T07:20:52.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.138+0000 7f7a71e00640 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c50b9fecc0 0x55c50ba010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.139+0000 7f7a71e00640 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c50b9fecc0 0x55c50ba010b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=99120f5cfd177070 server_cookie=92d58287b38573d3 in_seq=0 out_seq=0 2024-04-17T07:20:52.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.139+0000 7f7a53e00640 10 client.36149 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:20:52.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.141+0000 7f7a53e00640 1 -- 172.21.15.86:0/3791015420 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c50b95ca40 con 0x55c50b9fecc0 2024-04-17T07:20:52.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.141+0000 7f7a53e00640 10 client.36149 handle_command_reply: tid=0 2024-04-17T07:20:52.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.142+0000 7f7a7ba4a740 1 client.36149 shutdown 2024-04-17T07:20:52.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.142+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3791015420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9b50 msgr2=0x55c50b9f3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.142+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9b50 0x55c50b9f3c40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c680 tx=0x7f7a64014300 comp rx=0 tx=0).stop 2024-04-17T07:20:52.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.143+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3791015420 shutdown_connections 2024-04-17T07:20:52.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.143+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c50b9fecc0 0x55c50ba010b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.143+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 0x55c50b9fac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.143+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9b50 0x55c50b9f3c40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.143+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3791015420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9160 0x55c50b9f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.143+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3791015420 >> 172.21.15.86:0/3791015420 conn(0x55c50b9e46d0 msgr2=0x55c50b9e3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.143+0000 7f7a53e00640 0 client.36149 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:20:52.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.143+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3791015420 shutdown_connections 2024-04-17T07:20:52.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.143+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3791015420 wait complete. 2024-04-17T07:20:52.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.144+0000 7f7a7ba4a740 20 client.36149 trim_cache size 0 max 16384 2024-04-17T07:20:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.173+0000 7f7a7ba4a740 1 Processor -- start 2024-04-17T07:20:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.173+0000 7f7a7ba4a740 1 -- start start 2024-04-17T07:20:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.173+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.173+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9b50 0x55c50b9fac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.173+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9eecc0 0x55c50ba00cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.173+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b0e80 con 0x55c50b9e9b50 2024-04-17T07:20:52.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.173+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b29c0 con 0x55c50b9e9160 2024-04-17T07:20:52.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.173+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b1550 con 0x55c50b9eecc0 2024-04-17T07:20:52.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.174+0000 7f7a71e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.174+0000 7f7a71e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41376/0 (socket says 172.21.15.86:41376) 2024-04-17T07:20:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.174+0000 7f7a71e00640 1 -- 172.21.15.86:0/3883027001 learned_addr learned my addr 172.21.15.86:0/3883027001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.174+0000 7f7a71400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9b50 0x55c50b9fac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.174+0000 7f7a72800640 1 --2- 172.21.15.86:0/3883027001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9eecc0 0x55c50ba00cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.175+0000 7f7a71400640 1 -- 172.21.15.86:0/3883027001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9eecc0 msgr2=0x55c50ba00cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.175+0000 7f7a71400640 1 --2- 172.21.15.86:0/3883027001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9eecc0 0x55c50ba00cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.175+0000 7f7a71400640 1 -- 172.21.15.86:0/3883027001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 msgr2=0x55c50b9f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.175+0000 7f7a71400640 1 --2- 172.21.15.86:0/3883027001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.175+0000 7f7a71400640 1 -- 172.21.15.86:0/3883027001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50b6613f0 con 0x55c50b9e9b50 2024-04-17T07:20:52.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.175+0000 7f7a71e00640 1 --2- 172.21.15.86:0/3883027001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:52.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.175+0000 7f7a71400640 1 --2- 172.21.15.86:0/3883027001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9b50 0x55c50b9fac40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c650 tx=0x7f7a64021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc12608a64e05f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.176+0000 7f7a5ea00640 1 -- 172.21.15.86:0/3883027001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6400c930 con 0x55c50b9e9b50 2024-04-17T07:20:52.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.176+0000 7f7a5ea00640 1 -- 172.21.15.86:0/3883027001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6400f730 con 0x55c50b9e9b50 2024-04-17T07:20:52.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.176+0000 7f7a5ea00640 1 -- 172.21.15.86:0/3883027001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64003750 con 0x55c50b9e9b50 2024-04-17T07:20:52.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.177+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3883027001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9b50 msgr2=0x55c50b9fac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.177+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3883027001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9b50 0x55c50b9fac40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400c650 tx=0x7f7a64021000 comp rx=0 tx=0).stop 2024-04-17T07:20:52.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.177+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3883027001 shutdown_connections 2024-04-17T07:20:52.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.177+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3883027001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9eecc0 0x55c50ba00cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.177+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3883027001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9e9b50 0x55c50b9fac40 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.177+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/3883027001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.177+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3883027001 >> 172.21.15.86:0/3883027001 conn(0x55c50b9e46d0 msgr2=0x55c50b9f21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.177+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3883027001 shutdown_connections 2024-04-17T07:20:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.177+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/3883027001 wait complete. 2024-04-17T07:20:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.178+0000 7f7a7ba4a740 1 -- start start 2024-04-17T07:20:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.178+0000 7f7a7ba4a740 1 Processor -- start 2024-04-17T07:20:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 0x55c50b9f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a7ba4a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 0x55c50ba74130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b22f0 con 0x55c50b9eecc0 2024-04-17T07:20:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b1c20 con 0x55c50b9e9160 2024-04-17T07:20:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a7ba4a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741aee50 con 0x55c50b9e9b50 2024-04-17T07:20:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a7ba4a740 10 client.0 _ll_register_callbacks cb 0x7f7a64014840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a71e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a71e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41378/0 (socket says 172.21.15.86:41378) 2024-04-17T07:20:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a71e00640 1 -- 172.21.15.86:0/199573082 learned_addr learned my addr 172.21.15.86:0/199573082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.179+0000 7f7a72800640 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 0x55c50ba74130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.180+0000 7f7a71400640 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 0x55c50b9f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.180+0000 7f7a71e00640 1 -- 172.21.15.86:0/199573082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 msgr2=0x55c50b9f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.180+0000 7f7a71e00640 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 0x55c50b9f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.180+0000 7f7a71e00640 1 -- 172.21.15.86:0/199573082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 msgr2=0x55c50ba74130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.180+0000 7f7a71e00640 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 0x55c50ba74130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.180+0000 7f7a71e00640 1 -- 172.21.15.86:0/199573082 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50b72b7d0 con 0x55c50b9e9160 2024-04-17T07:20:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.180+0000 7f7a71e00640 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f5a30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009850 tx=0x7f7a60009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c2130e27d9d5da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.180+0000 7f7a53e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:20:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.181+0000 7f7a53e00640 1 -- 172.21.15.86:0/199573082 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600565c0 con 0x55c50b9e9160 2024-04-17T07:20:52.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.181+0000 7f7a53e00640 1 -- 172.21.15.86:0/199573082 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7a60056760 con 0x55c50b9e9160 2024-04-17T07:20:52.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.181+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/199573082 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a6400f730 con 0x55c50b9e9160 2024-04-17T07:20:52.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.182+0000 7f7a53e00640 1 -- 172.21.15.86:0/199573082 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a60064050 con 0x55c50b9e9160 2024-04-17T07:20:52.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.182+0000 7f7a7ba4a740 10 client.36155 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:52.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.182+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/199573082 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c50ba75690 con 0x55c50b9e9160 2024-04-17T07:20:52.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.183+0000 7f7a53e00640 1 -- 172.21.15.86:0/199573082 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6400f730 con 0x55c50b9e9160 2024-04-17T07:20:52.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.183+0000 7f7a53e00640 1 -- 172.21.15.86:0/199573082 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7a60004b50 con 0x55c50b9e9160 2024-04-17T07:20:52.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.183+0000 7f7a7ba4a740 10 client.36155 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:52.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.183+0000 7f7a7ba4a740 10 client.36155 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:20:52.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.183+0000 7f7a7ba4a740 20 client.36155 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:52.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.183+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c50ba76010 0x55c50ba78430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.183+0000 7f7a7ba4a740 4 client.36155 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:20:52.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.183+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/199573082 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c50b9568b0 con 0x55c50ba76010 2024-04-17T07:20:52.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.184+0000 7f7a71400640 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c50ba76010 0x55c50ba78430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.185+0000 7f7a71400640 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c50ba76010 0x55c50ba78430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a3040bb43130537 server_cookie=e7931d27b8756397 in_seq=0 out_seq=0 2024-04-17T07:20:52.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.185+0000 7f7a53e00640 10 client.36155 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:20:52.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.185+0000 7f7a53e00640 1 -- 172.21.15.86:0/199573082 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c50b9568b0 con 0x55c50ba76010 2024-04-17T07:20:52.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.185+0000 7f7a53e00640 10 client.36155 handle_command_reply: tid=0 2024-04-17T07:20:52.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.185+0000 7f7a7ba4a740 1 client.36155 shutdown 2024-04-17T07:20:52.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.186+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/199573082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 msgr2=0x55c50b9f5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.186+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f5a30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009850 tx=0x7f7a60009260 comp rx=0 tx=0).stop 2024-04-17T07:20:52.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.186+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/199573082 shutdown_connections 2024-04-17T07:20:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.186+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c50ba76010 0x55c50ba78430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.186+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50b9eecc0 0x55c50ba74130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.186+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50b9e9b50 0x55c50b9f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.186+0000 7f7a7ba4a740 1 --2- 172.21.15.86:0/199573082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50b9e9160 0x55c50b9f5a30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.186+0000 7f7a53e00640 0 client.36155 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:20:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.186+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/199573082 >> 172.21.15.86:0/199573082 conn(0x55c50b9e46d0 msgr2=0x55c50b9e7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.187+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/199573082 shutdown_connections 2024-04-17T07:20:52.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.187+0000 7f7a7ba4a740 1 -- 172.21.15.86:0/199573082 wait complete. 2024-04-17T07:20:52.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.187+0000 7f7a7ba4a740 20 client.36155 trim_cache size 0 max 16384 2024-04-17T07:20:52.188 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:20:52.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.189+0000 7f7a73200640 1 -- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a580a3d30 msgr2=0x7f7a580a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.189+0000 7f7a73200640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a580a3d30 0x7f7a580a61f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55c50b9e9f50 tx=0x7f7a60000bc0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.189+0000 7f7a73200640 1 -- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74105c50 msgr2=0x7f7a741a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.189+0000 7f7a73200640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74105c50 0x7f7a741a5d10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00be20 tx=0x7f7a6c00bf10 comp rx=0 tx=0).stop 2024-04-17T07:20:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.189+0000 7f7a73200640 1 -- 172.21.15.86:0/3317435456 shutdown_connections 2024-04-17T07:20:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.189+0000 7f7a73200640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a580a3d30 0x7f7a580a61f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.189+0000 7f7a73200640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a74105c50 0x7f7a741a5d10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.189+0000 7f7a73200640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a74105290 0x7f7a7419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.189+0000 7f7a73200640 1 --2- 172.21.15.86:0/3317435456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a741040a0 0x7f7a7419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.189+0000 7f7a73200640 1 -- 172.21.15.86:0/3317435456 >> 172.21.15.86:0/3317435456 conn(0x7f7a740ff610 msgr2=0x7f7a74079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.190+0000 7f7a73200640 1 -- 172.21.15.86:0/3317435456 shutdown_connections 2024-04-17T07:20:52.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.190+0000 7f7a73200640 1 -- 172.21.15.86:0/3317435456 wait complete. 2024-04-17T07:20:52.218 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:20:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:52 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1463771774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:20:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.520+0000 7f6693a00640 1 -- 172.21.15.86:0/3687084825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f668c101fb0 msgr2=0x7f668c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.520+0000 7f6693a00640 1 --2- 172.21.15.86:0/3687084825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f668c101fb0 0x7f668c1023d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f667c009ca0 tx=0x7f667c02c100 comp rx=0 tx=0).stop 2024-04-17T07:20:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.521+0000 7f6693a00640 1 -- 172.21.15.86:0/3687084825 shutdown_connections 2024-04-17T07:20:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.521+0000 7f6693a00640 1 --2- 172.21.15.86:0/3687084825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c103b60 0x7f668c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.521+0000 7f6693a00640 1 --2- 172.21.15.86:0/3687084825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c1031a0 0x7f668c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.521+0000 7f6693a00640 1 --2- 172.21.15.86:0/3687084825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f668c101fb0 0x7f668c1023d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.521+0000 7f6693a00640 1 -- 172.21.15.86:0/3687084825 >> 172.21.15.86:0/3687084825 conn(0x7f668c0fd540 msgr2=0x7f668c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.521+0000 7f6693a00640 1 -- 172.21.15.86:0/3687084825 shutdown_connections 2024-04-17T07:20:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.521+0000 7f6693a00640 1 -- 172.21.15.86:0/3687084825 wait complete. 2024-04-17T07:20:52.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.522+0000 7f6693a00640 1 Processor -- start 2024-04-17T07:20:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.523+0000 7f6693a00640 1 -- start start 2024-04-17T07:20:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.523+0000 7f6693a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f668c101fb0 0x7f668c110960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.523+0000 7f6693a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c1031a0 0x7f668c110ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.523+0000 7f6693a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c103b60 0x7f668c10d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.523+0000 7f6693a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f668c074cd0 con 0x7f668c103b60 2024-04-17T07:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.523+0000 7f6693a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f668c074b50 con 0x7f668c1031a0 2024-04-17T07:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.523+0000 7f6693a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f668c074e50 con 0x7f668c101fb0 2024-04-17T07:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.523+0000 7f6691000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f668c101fb0 0x7f668c110960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.524+0000 7f6691000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f668c101fb0 0x7f668c110960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51232/0 (socket says 172.21.15.86:51232) 2024-04-17T07:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.524+0000 7f6691000640 1 -- 172.21.15.86:0/2383062882 learned_addr learned my addr 172.21.15.86:0/2383062882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.524+0000 7f668be00640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c1031a0 0x7f668c110ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.524+0000 7f6691a00640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c103b60 0x7f668c10d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.524+0000 7f6691000640 1 -- 172.21.15.86:0/2383062882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c1031a0 msgr2=0x7f668c110ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.524+0000 7f6691000640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c1031a0 0x7f668c110ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.524+0000 7f6691000640 1 -- 172.21.15.86:0/2383062882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c103b60 msgr2=0x7f668c10d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.524+0000 7f6691000640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c103b60 0x7f668c10d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.524+0000 7f6691000640 1 -- 172.21.15.86:0/2383062882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c10dd90 con 0x7f668c101fb0 2024-04-17T07:20:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:52 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1463771774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:20:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.524+0000 7f6691000640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f668c101fb0 0x7f668c110960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f667c000f80 tx=0x7f667c035d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.525+0000 7f6689600640 1 -- 172.21.15.86:0/2383062882 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c039040 con 0x7f668c101fb0 2024-04-17T07:20:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.525+0000 7f6689600640 1 -- 172.21.15.86:0/2383062882 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f667c044070 con 0x7f668c101fb0 2024-04-17T07:20:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.525+0000 7f6689600640 1 -- 172.21.15.86:0/2383062882 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c004640 con 0x7f668c101fb0 2024-04-17T07:20:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.525+0000 7f6693a00640 1 -- 172.21.15.86:0/2383062882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c1a6930 con 0x7f668c101fb0 2024-04-17T07:20:52.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.525+0000 7f6693a00640 1 -- 172.21.15.86:0/2383062882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c1a6ce0 con 0x7f668c101fb0 2024-04-17T07:20:52.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.530+0000 7f6696782740 1 Processor -- start 2024-04-17T07:20:52.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.530+0000 7f6696782740 1 -- start start 2024-04-17T07:20:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.531+0000 7f6696782740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dc23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.531+0000 7f6696782740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc2990 0x5608f1dc7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.531+0000 7f6696782740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc7b00 0x5608f1dc9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.531+0000 7f6696782740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5608f1d28020 con 0x5608f1dc7b00 2024-04-17T07:20:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.531+0000 7f6696782740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5608f1cce460 con 0x5608f1dc1fa0 2024-04-17T07:20:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.531+0000 7f6696782740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5608f1d23fe0 con 0x5608f1dc2990 2024-04-17T07:20:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.532+0000 7f668be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc7b00 0x5608f1dc9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.532+0000 7f6691a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dc23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.532+0000 7f6691a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dc23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41388/0 (socket says 172.21.15.86:41388) 2024-04-17T07:20:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.532+0000 7f6691a00640 1 -- 172.21.15.86:0/3261812961 learned_addr learned my addr 172.21.15.86:0/3261812961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.533+0000 7f6691a00640 1 -- 172.21.15.86:0/3261812961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc2990 msgr2=0x5608f1dc7490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:20:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.533+0000 7f6691a00640 1 --2- 172.21.15.86:0/3261812961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc2990 0x5608f1dc7490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.533+0000 7f6691a00640 1 -- 172.21.15.86:0/3261812961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc7b00 msgr2=0x5608f1dc9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.533+0000 7f6691a00640 1 --2- 172.21.15.86:0/3261812961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc7b00 0x5608f1dc9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.533+0000 7f6691a00640 1 -- 172.21.15.86:0/3261812961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f1cd6040 con 0x5608f1dc1fa0 2024-04-17T07:20:52.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.533+0000 7f668be00640 1 --2- 172.21.15.86:0/3261812961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc7b00 0x5608f1dc9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:52.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.534+0000 7f6691a00640 1 --2- 172.21.15.86:0/3261812961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dc23c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f668400ba10 tx=0x7f668400bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54375324a78b1308 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.535+0000 7f6682000640 1 -- 172.21.15.86:0/3261812961 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668400de50 con 0x5608f1dc1fa0 2024-04-17T07:20:52.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.535+0000 7f6682000640 1 -- 172.21.15.86:0/3261812961 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66840178d0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.535+0000 7f6682000640 1 -- 172.21.15.86:0/3261812961 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668400f990 con 0x5608f1dc1fa0 2024-04-17T07:20:52.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.535+0000 7f6689600640 1 -- 172.21.15.86:0/2383062882 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f667c0327c0 con 0x7f668c101fb0 2024-04-17T07:20:52.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.536+0000 7f6696782740 1 -- 172.21.15.86:0/3261812961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 msgr2=0x5608f1dc23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.536+0000 7f6696782740 1 --2- 172.21.15.86:0/3261812961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dc23c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f668400ba10 tx=0x7f668400bed0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.538+0000 7f6689600640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66700a3c30 0x7f66700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.540+0000 7f6696782740 1 -- 172.21.15.86:0/3261812961 shutdown_connections 2024-04-17T07:20:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.540+0000 7f6696782740 1 --2- 172.21.15.86:0/3261812961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc7b00 0x5608f1dc9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.540+0000 7f6696782740 1 --2- 172.21.15.86:0/3261812961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc2990 0x5608f1dc7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.540+0000 7f6696782740 1 --2- 172.21.15.86:0/3261812961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dc23c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.540+0000 7f6696782740 1 -- 172.21.15.86:0/3261812961 >> 172.21.15.86:0/3261812961 conn(0x5608f1dbd510 msgr2=0x5608f1dbd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.540+0000 7f6691a00640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66700a3c30 0x7f66700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.542+0000 7f6689600640 1 -- 172.21.15.86:0/2383062882 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f667c0f2ac0 con 0x7f668c101fb0 2024-04-17T07:20:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.542+0000 7f6691a00640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66700a3c30 0x7f66700a60f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f668400b7a0 tx=0x7f668400f420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.543+0000 7f6696782740 1 -- 172.21.15.86:0/3261812961 shutdown_connections 2024-04-17T07:20:52.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.543+0000 7f6696782740 1 -- 172.21.15.86:0/3261812961 wait complete. 2024-04-17T07:20:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.544+0000 7f6696782740 1 -- start start 2024-04-17T07:20:52.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.544+0000 7f6696782740 1 Processor -- start 2024-04-17T07:20:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.546+0000 7f6696782740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.546+0000 7f6696782740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 0x5608f1dccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.546+0000 7f6696782740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 0x5608f1dd3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.546+0000 7f6696782740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5608f1dcaab0 con 0x5608f1dd1c00 2024-04-17T07:20:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.546+0000 7f6696782740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5608f1ce1830 con 0x5608f1dc7b00 2024-04-17T07:20:52.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.546+0000 7f6696782740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5608f1dcac30 con 0x5608f1dc1fa0 2024-04-17T07:20:52.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.546+0000 7f6696782740 10 client.0 _ll_register_callbacks cb 0x5608f1caffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.547+0000 7f6691000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.547+0000 7f6691000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51238/0 (socket says 172.21.15.86:51238) 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.547+0000 7f6691000640 1 -- 172.21.15.86:0/3774988160 learned_addr learned my addr 172.21.15.86:0/3774988160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.547+0000 7f6691a00640 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 0x5608f1dd3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.547+0000 7f668be00640 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 0x5608f1dccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.547+0000 7f6691000640 1 -- 172.21.15.86:0/3774988160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 msgr2=0x5608f1dccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.547+0000 7f6691000640 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 0x5608f1dccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.547+0000 7f6691000640 1 -- 172.21.15.86:0/3774988160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 msgr2=0x5608f1dd3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.547+0000 7f6691000640 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 0x5608f1dd3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.547+0000 7f6691000640 1 -- 172.21.15.86:0/3774988160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f1ce5920 con 0x5608f1dc1fa0 2024-04-17T07:20:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.548+0000 7f6691000640 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcc690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f667c033000 tx=0x7f667c0be500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97c99d480bc804f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.548+0000 7f667aa00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.548+0000 7f667aa00640 1 -- 172.21.15.86:0/3774988160 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c0098c0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.548+0000 7f667aa00640 1 -- 172.21.15.86:0/3774988160 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f667c03e3a0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.548+0000 7f6696782740 1 -- 172.21.15.86:0/3774988160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66840178d0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.549+0000 7f667aa00640 1 -- 172.21.15.86:0/3774988160 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f667c03e5a0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.549+0000 7f6696782740 10 client.26188 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.549+0000 7f6696782740 1 -- 172.21.15.86:0/3774988160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5608f1cb37f0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.550+0000 7f667aa00640 1 -- 172.21.15.86:0/3774988160 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66840178d0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.550+0000 7f667aa00640 1 -- 172.21.15.86:0/3774988160 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f667c03c030 con 0x5608f1dc1fa0 2024-04-17T07:20:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.550+0000 7f6696782740 10 client.26188 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.550+0000 7f6696782740 10 client.26188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:20:52.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.550+0000 7f6696782740 20 client.26188 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:52.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.550+0000 7f6696782740 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5608f1dd8360 0x5608f1dda790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.550+0000 7f6696782740 4 client.26188 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:52.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.550+0000 7f6696782740 1 -- 172.21.15.86:0/3774988160 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608f1d35a40 con 0x5608f1dd8360 2024-04-17T07:20:52.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.551+0000 7f668be00640 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5608f1dd8360 0x5608f1dda790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.552+0000 7f667aa00640 10 client.26188 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:20:52.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.552+0000 7f668be00640 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5608f1dd8360 0x5608f1dda790 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5bf753a0a54af479 server_cookie=8f3fec9c1e909793 in_seq=0 out_seq=0 2024-04-17T07:20:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.554+0000 7f667aa00640 1 -- 172.21.15.86:0/3774988160 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608f1d35a40 con 0x5608f1dd8360 2024-04-17T07:20:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.554+0000 7f667aa00640 10 client.26188 handle_command_reply: tid=0 2024-04-17T07:20:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.554+0000 7f6696782740 1 client.26188 shutdown 2024-04-17T07:20:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.555+0000 7f6696782740 1 -- 172.21.15.86:0/3774988160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc1fa0 msgr2=0x5608f1dcc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.555+0000 7f6696782740 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcc690 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f667c033000 tx=0x7f667c0be500 comp rx=0 tx=0).stop 2024-04-17T07:20:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.555+0000 7f6696782740 1 -- 172.21.15.86:0/3774988160 shutdown_connections 2024-04-17T07:20:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.555+0000 7f6696782740 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5608f1dd8360 0x5608f1dda790 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.555+0000 7f6696782740 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 0x5608f1dd3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.555+0000 7f6696782740 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 0x5608f1dccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.555+0000 7f6696782740 1 --2- 172.21.15.86:0/3774988160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcc690 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.555+0000 7f6696782740 1 -- 172.21.15.86:0/3774988160 >> 172.21.15.86:0/3774988160 conn(0x5608f1dbd510 msgr2=0x5608f1dbc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.555+0000 7f667aa00640 0 client.26188 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:20:52.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.555+0000 7f6696782740 1 -- 172.21.15.86:0/3774988160 shutdown_connections 2024-04-17T07:20:52.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.556+0000 7f6696782740 1 -- 172.21.15.86:0/3774988160 wait complete. 2024-04-17T07:20:52.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.556+0000 7f6696782740 20 client.26188 trim_cache size 0 max 16384 2024-04-17T07:20:52.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:52 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1463771774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:20:52.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.582+0000 7f6696782740 1 Processor -- start 2024-04-17T07:20:52.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.582+0000 7f6696782740 1 -- start start 2024-04-17T07:20:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.583+0000 7f6696782740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.583+0000 7f6696782740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc7b00 0x5608f1dcc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.584+0000 7f6691000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.584+0000 7f6691000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41416/0 (socket says 172.21.15.86:41416) 2024-04-17T07:20:52.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.584+0000 7f6691000640 1 -- 172.21.15.86:0/3742144994 learned_addr learned my addr 172.21.15.86:0/3742144994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.584+0000 7f6696782740 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 0x5608f1dcc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.584+0000 7f668be00640 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc7b00 0x5608f1dcc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.585+0000 7f6696782740 1 -- 172.21.15.86:0/3742144994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b1340 con 0x5608f1dd1c00 2024-04-17T07:20:52.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.585+0000 7f6696782740 1 -- 172.21.15.86:0/3742144994 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b2e80 con 0x5608f1dc1fa0 2024-04-17T07:20:52.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.585+0000 7f6696782740 1 -- 172.21.15.86:0/3742144994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b1a10 con 0x5608f1dc7b00 2024-04-17T07:20:52.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.585+0000 7f6691a00640 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 0x5608f1dcc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.586+0000 7f6691000640 1 -- 172.21.15.86:0/3742144994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc7b00 msgr2=0x5608f1dcc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.586+0000 7f6691000640 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc7b00 0x5608f1dcc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.586+0000 7f6691000640 1 -- 172.21.15.86:0/3742144994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 msgr2=0x5608f1dcc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.586+0000 7f6691000640 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 0x5608f1dcc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.586+0000 7f6691000640 1 -- 172.21.15.86:0/3742144994 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f1a3a3f0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.586+0000 7f6691a00640 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 0x5608f1dcc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.586+0000 7f6691000640 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcb6e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f667c0bae20 tx=0x7f667c0be630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed695415867d1460 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.587+0000 7f6681600640 1 -- 172.21.15.86:0/3742144994 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c0098c0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.587+0000 7f6681600640 1 -- 172.21.15.86:0/3742144994 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f667c0bf0c0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.587+0000 7f6681600640 1 -- 172.21.15.86:0/3742144994 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c0ddb80 con 0x5608f1dc1fa0 2024-04-17T07:20:52.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.588+0000 7f6696782740 1 -- 172.21.15.86:0/3742144994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 msgr2=0x5608f1dcb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.588+0000 7f6696782740 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcb6e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f667c0bae20 tx=0x7f667c0be630 comp rx=0 tx=0).stop 2024-04-17T07:20:52.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.588+0000 7f6696782740 1 -- 172.21.15.86:0/3742144994 shutdown_connections 2024-04-17T07:20:52.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.588+0000 7f6696782740 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dd1c00 0x5608f1dcc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.588+0000 7f6696782740 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dc7b00 0x5608f1dcc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.588+0000 7f6696782740 1 --2- 172.21.15.86:0/3742144994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc1fa0 0x5608f1dcb6e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.588+0000 7f6696782740 1 -- 172.21.15.86:0/3742144994 >> 172.21.15.86:0/3742144994 conn(0x5608f1dbd510 msgr2=0x5608f1dcafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.589+0000 7f6696782740 1 -- 172.21.15.86:0/3742144994 shutdown_connections 2024-04-17T07:20:52.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.589+0000 7f6696782740 1 -- 172.21.15.86:0/3742144994 wait complete. 2024-04-17T07:20:52.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.590+0000 7f6696782740 1 -- start start 2024-04-17T07:20:52.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.590+0000 7f6696782740 1 Processor -- start 2024-04-17T07:20:52.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.591+0000 7f6696782740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc1fa0 0x5608f1de91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.591+0000 7f6696782740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 0x5608f1de9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.591+0000 7f6696782740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dd1c00 0x5608f1dea680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.592+0000 7f6696782740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b27b0 con 0x5608f1dc1fa0 2024-04-17T07:20:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.592+0000 7f668be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 0x5608f1de9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.592+0000 7f668be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 0x5608f1de9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41430/0 (socket says 172.21.15.86:41430) 2024-04-17T07:20:52.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.592+0000 7f668be00640 1 -- 172.21.15.86:0/1197911155 learned_addr learned my addr 172.21.15.86:0/1197911155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.592+0000 7f6696782740 1 -- 172.21.15.86:0/1197911155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1b20e0 con 0x5608f1dc7b00 2024-04-17T07:20:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.592+0000 7f6691a00640 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dd1c00 0x5608f1dea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.592+0000 7f6691000640 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc1fa0 0x5608f1de91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.592+0000 7f6696782740 1 -- 172.21.15.86:0/1197911155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f668c1af310 con 0x5608f1dd1c00 2024-04-17T07:20:52.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.593+0000 7f6696782740 10 client.0 _ll_register_callbacks cb 0x7f667c0df130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.593+0000 7f6691a00640 1 -- 172.21.15.86:0/1197911155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 msgr2=0x5608f1de9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.593+0000 7f6691a00640 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 0x5608f1de9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.593+0000 7f6691a00640 1 -- 172.21.15.86:0/1197911155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc1fa0 msgr2=0x5608f1de91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.593+0000 7f6691a00640 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc1fa0 0x5608f1de91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.593+0000 7f6691a00640 1 -- 172.21.15.86:0/1197911155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f1b047d0 con 0x5608f1dd1c00 2024-04-17T07:20:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.594+0000 7f6691a00640 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dd1c00 0x5608f1dea680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f668400b9e0 tx=0x7f668400bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a35f5bc9058b17df server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.594+0000 7f667aa00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.594+0000 7f667aa00640 1 -- 172.21.15.86:0/1197911155 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684034070 con 0x5608f1dd1c00 2024-04-17T07:20:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.595+0000 7f6696782740 1 -- 172.21.15.86:0/1197911155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f667c0bf0c0 con 0x5608f1dd1c00 2024-04-17T07:20:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.595+0000 7f667aa00640 1 -- 172.21.15.86:0/1197911155 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f668400fd40 con 0x5608f1dd1c00 2024-04-17T07:20:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.595+0000 7f667aa00640 1 -- 172.21.15.86:0/1197911155 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f668400b600 con 0x5608f1dd1c00 2024-04-17T07:20:52.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.595+0000 7f6696782740 10 client.26194 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.595+0000 7f6696782740 1 -- 172.21.15.86:0/1197911155 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5608f1debe40 con 0x5608f1dd1c00 2024-04-17T07:20:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.596+0000 7f667aa00640 1 -- 172.21.15.86:0/1197911155 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f667c0bf0c0 con 0x5608f1dd1c00 2024-04-17T07:20:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.596+0000 7f667aa00640 1 -- 172.21.15.86:0/1197911155 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f66840049f0 con 0x5608f1dd1c00 2024-04-17T07:20:52.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.596+0000 7f6696782740 10 client.26194 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.596+0000 7f6696782740 10 client.26194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:20:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.596+0000 7f6696782740 20 client.26194 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.596+0000 7f6696782740 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5608f1e3e790 0x5608f1e40bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.596+0000 7f6696782740 4 client.26194 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:20:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.596+0000 7f6696782740 1 -- 172.21.15.86:0/1197911155 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608f1d2f8b0 con 0x5608f1e3e790 2024-04-17T07:20:52.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.598+0000 7f6691000640 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5608f1e3e790 0x5608f1e40bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.599+0000 7f6691000640 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5608f1e3e790 0x5608f1e40bb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71c8fb42bcada754 server_cookie=c26cbd7ad867ccd4 in_seq=0 out_seq=0 2024-04-17T07:20:52.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.599+0000 7f667aa00640 10 client.26194 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:20:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.599+0000 7f667aa00640 1 -- 172.21.15.86:0/1197911155 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608f1d2f8b0 con 0x5608f1e3e790 2024-04-17T07:20:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.599+0000 7f667aa00640 10 client.26194 handle_command_reply: tid=0 2024-04-17T07:20:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.599+0000 7f6696782740 1 client.26194 shutdown 2024-04-17T07:20:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.599+0000 7f6696782740 1 -- 172.21.15.86:0/1197911155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dd1c00 msgr2=0x5608f1dea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.599+0000 7f6696782740 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dd1c00 0x5608f1dea680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f668400b9e0 tx=0x7f668400bed0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.599+0000 7f6696782740 1 -- 172.21.15.86:0/1197911155 shutdown_connections 2024-04-17T07:20:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.599+0000 7f6696782740 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5608f1e3e790 0x5608f1e40bb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.600+0000 7f667aa00640 0 client.26194 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:20:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.600+0000 7f6696782740 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5608f1dd1c00 0x5608f1dea680 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.600+0000 7f6696782740 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5608f1dc7b00 0x5608f1de9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.600+0000 7f6696782740 1 --2- 172.21.15.86:0/1197911155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608f1dc1fa0 0x5608f1de91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.600+0000 7f6696782740 1 -- 172.21.15.86:0/1197911155 >> 172.21.15.86:0/1197911155 conn(0x5608f1dbd510 msgr2=0x5608f1dcc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.600+0000 7f6696782740 1 -- 172.21.15.86:0/1197911155 shutdown_connections 2024-04-17T07:20:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.600+0000 7f6696782740 1 -- 172.21.15.86:0/1197911155 wait complete. 2024-04-17T07:20:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.601+0000 7f6696782740 20 client.26194 trim_cache size 0 max 16384 2024-04-17T07:20:52.602 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:20:52.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.602+0000 7f6693a00640 1 -- 172.21.15.86:0/2383062882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66700a3c30 msgr2=0x7f66700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.602+0000 7f6693a00640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66700a3c30 0x7f66700a60f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f668400b7a0 tx=0x7f668400f420 comp rx=0 tx=0).stop 2024-04-17T07:20:52.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.603+0000 7f6693a00640 1 -- 172.21.15.86:0/2383062882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f668c101fb0 msgr2=0x7f668c110960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.603+0000 7f6693a00640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f668c101fb0 0x7f668c110960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f667c000f80 tx=0x7f667c035d60 comp rx=0 tx=0).stop 2024-04-17T07:20:52.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.603+0000 7f6693a00640 1 -- 172.21.15.86:0/2383062882 shutdown_connections 2024-04-17T07:20:52.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.603+0000 7f6693a00640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f66700a3c30 0x7f66700a60f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.603+0000 7f6693a00640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f668c103b60 0x7f668c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.603+0000 7f6693a00640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c1031a0 0x7f668c110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.603+0000 7f6693a00640 1 --2- 172.21.15.86:0/2383062882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f668c101fb0 0x7f668c110960 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.603+0000 7f6693a00640 1 -- 172.21.15.86:0/2383062882 >> 172.21.15.86:0/2383062882 conn(0x7f668c0fd540 msgr2=0x7f668c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.603+0000 7f6693a00640 1 -- 172.21.15.86:0/2383062882 shutdown_connections 2024-04-17T07:20:52.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.603+0000 7f6693a00640 1 -- 172.21.15.86:0/2383062882 wait complete. 2024-04-17T07:20:52.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:20:52.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.956+0000 7f0235e00640 1 -- 172.21.15.86:0/2030880573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230075fa0 msgr2=0x7f02300743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.956+0000 7f0235e00640 1 --2- 172.21.15.86:0/2030880573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230075fa0 0x7f02300743f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f0220009860 tx=0x7f022002bb70 comp rx=0 tx=0).stop 2024-04-17T07:20:52.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.956+0000 7f0235e00640 1 -- 172.21.15.86:0/2030880573 shutdown_connections 2024-04-17T07:20:52.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.956+0000 7f0235e00640 1 --2- 172.21.15.86:0/2030880573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0230109e70 0x7f023010c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.956+0000 7f0235e00640 1 --2- 172.21.15.86:0/2030880573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02300749c0 0x7f0230074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.956+0000 7f0235e00640 1 --2- 172.21.15.86:0/2030880573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0230075fa0 0x7f02300743f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.956+0000 7f0235e00640 1 -- 172.21.15.86:0/2030880573 >> 172.21.15.86:0/2030880573 conn(0x7f02300fd540 msgr2=0x7f02300ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.958+0000 7f0235e00640 1 -- 172.21.15.86:0/2030880573 shutdown_connections 2024-04-17T07:20:52.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.958+0000 7f0235e00640 1 -- 172.21.15.86:0/2030880573 wait complete. 2024-04-17T07:20:52.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.959+0000 7f0235e00640 1 Processor -- start 2024-04-17T07:20:52.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.960+0000 7f0235e00640 1 -- start start 2024-04-17T07:20:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.961+0000 7f0235e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02300749c0 0x7f02301112e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.961+0000 7f0235e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0230075fa0 0x7f0230111820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.961+0000 7f0235e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230109e70 0x7f0230116880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.961+0000 7f0235e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0230102080 con 0x7f02300749c0 2024-04-17T07:20:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.961+0000 7f0235e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0230101f00 con 0x7f0230109e70 2024-04-17T07:20:52.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.961+0000 7f0235e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0230102200 con 0x7f0230075fa0 2024-04-17T07:20:52.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.961+0000 7f022f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230109e70 0x7f0230116880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.961+0000 7f022f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230109e70 0x7f0230116880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41448/0 (socket says 172.21.15.86:41448) 2024-04-17T07:20:52.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.961+0000 7f022f400640 1 -- 172.21.15.86:0/2555694637 learned_addr learned my addr 172.21.15.86:0/2555694637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.961+0000 7f022e000640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0230075fa0 0x7f0230111820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.962+0000 7f022f400640 1 -- 172.21.15.86:0/2555694637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0230075fa0 msgr2=0x7f0230111820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.962+0000 7f022f400640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0230075fa0 0x7f0230111820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.962+0000 7f022f400640 1 -- 172.21.15.86:0/2555694637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02300749c0 msgr2=0x7f02301112e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.962+0000 7f022f400640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02300749c0 0x7f02301112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.962+0000 7f022f400640 1 -- 172.21.15.86:0/2555694637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0230116ef0 con 0x7f0230109e70 2024-04-17T07:20:52.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.962+0000 7f022f400640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230109e70 0x7f0230116880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f022800eaa0 tx=0x7f022800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.963+0000 7f0234a00640 1 -- 172.21.15.86:0/2555694637 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022800cdb0 con 0x7f0230109e70 2024-04-17T07:20:52.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.963+0000 7f0235e00640 1 -- 172.21.15.86:0/2555694637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02301171e0 con 0x7f0230109e70 2024-04-17T07:20:52.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.963+0000 7f0234a00640 1 -- 172.21.15.86:0/2555694637 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0228005020 con 0x7f0230109e70 2024-04-17T07:20:52.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.963+0000 7f0234a00640 1 -- 172.21.15.86:0/2555694637 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280116d0 con 0x7f0230109e70 2024-04-17T07:20:52.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.964+0000 7f0235e00640 1 -- 172.21.15.86:0/2555694637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0230117730 con 0x7f0230109e70 2024-04-17T07:20:52.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.970+0000 7f0234a00640 1 -- 172.21.15.86:0/2555694637 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f02280051d0 con 0x7f0230109e70 2024-04-17T07:20:52.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.972+0000 7f0238b66740 1 Processor -- start 2024-04-17T07:20:52.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.973+0000 7f0238b66740 1 -- start start 2024-04-17T07:20:52.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.973+0000 7f0234a00640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02140a3c30 0x7f02140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.973+0000 7f0238b66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f56fa0 0x560479f573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.973+0000 7f0238b66740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f5c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.973+0000 7f0238b66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 0x560479f5ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.973+0000 7f0238b66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560479ebd020 con 0x560479f5cb00 2024-04-17T07:20:52.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.973+0000 7f0238b66740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560479e63460 con 0x560479f57990 2024-04-17T07:20:52.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.973+0000 7f0238b66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560479eb8fe0 con 0x560479f56fa0 2024-04-17T07:20:52.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.973+0000 7f0234a00640 1 -- 172.21.15.86:0/2555694637 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f02280d0a20 con 0x7f0230109e70 2024-04-17T07:20:52.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.974+0000 7f022ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f56fa0 0x560479f573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.974+0000 7f022f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 0x560479f5ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.974+0000 7f022ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f56fa0 0x560479f573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51282/0 (socket says 172.21.15.86:51282) 2024-04-17T07:20:52.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.974+0000 7f022ea00640 1 -- 172.21.15.86:0/3483644647 learned_addr learned my addr 172.21.15.86:0/3483644647 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.975+0000 7f022e000640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02140a3c30 0x7f02140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.975+0000 7f022e000640 1 --2- 172.21.15.86:0/3483644647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f5c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.975+0000 7f022ea00640 1 -- 172.21.15.86:0/3483644647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 msgr2=0x560479f5c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.976+0000 7f022ea00640 1 --2- 172.21.15.86:0/3483644647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f5c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.976+0000 7f022ea00640 1 -- 172.21.15.86:0/3483644647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 msgr2=0x560479f5ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.976+0000 7f022ea00640 1 --2- 172.21.15.86:0/3483644647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 0x560479f5ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.976+0000 7f022ea00640 1 -- 172.21.15.86:0/3483644647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560479e6b040 con 0x560479f56fa0 2024-04-17T07:20:52.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.977+0000 7f022e000640 1 --2- 172.21.15.86:0/3483644647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f5c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:52.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.977+0000 7f022f400640 1 --2- 172.21.15.86:0/3483644647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 0x560479f5ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:52.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.977+0000 7f022ea00640 1 --2- 172.21.15.86:0/3483644647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f56fa0 0x560479f573c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02200349e0 tx=0x7f0220009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80a148f736f7cbe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.978+0000 7f022e000640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02140a3c30 0x7f02140a60f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x560479f57d90 tx=0x7f021c009460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.978+0000 7f021be00640 1 -- 172.21.15.86:0/3483644647 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220039040 con 0x560479f56fa0 2024-04-17T07:20:52.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.979+0000 7f021be00640 1 -- 172.21.15.86:0/3483644647 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0220042070 con 0x560479f56fa0 2024-04-17T07:20:52.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.979+0000 7f021be00640 1 -- 172.21.15.86:0/3483644647 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220039040 con 0x560479f56fa0 2024-04-17T07:20:52.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.980+0000 7f0238b66740 1 -- 172.21.15.86:0/3483644647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f56fa0 msgr2=0x560479f573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.980+0000 7f0238b66740 1 --2- 172.21.15.86:0/3483644647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f56fa0 0x560479f573c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02200349e0 tx=0x7f0220009560 comp rx=0 tx=0).stop 2024-04-17T07:20:52.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.980+0000 7f0238b66740 1 -- 172.21.15.86:0/3483644647 shutdown_connections 2024-04-17T07:20:52.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.980+0000 7f0238b66740 1 --2- 172.21.15.86:0/3483644647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 0x560479f5ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.980+0000 7f0238b66740 1 --2- 172.21.15.86:0/3483644647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f5c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.980+0000 7f0238b66740 1 --2- 172.21.15.86:0/3483644647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f56fa0 0x560479f573c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.980+0000 7f0238b66740 1 -- 172.21.15.86:0/3483644647 >> 172.21.15.86:0/3483644647 conn(0x560479f52510 msgr2=0x560479f52910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.981+0000 7f0238b66740 1 -- 172.21.15.86:0/3483644647 shutdown_connections 2024-04-17T07:20:52.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.981+0000 7f0238b66740 1 -- 172.21.15.86:0/3483644647 wait complete. 2024-04-17T07:20:52.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.981+0000 7f0238b66740 1 -- start start 2024-04-17T07:20:52.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.982+0000 7f0238b66740 1 Processor -- start 2024-04-17T07:20:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.983+0000 7f0238b66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f56fa0 0x560479f614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.983+0000 7f0238b66740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.983+0000 7f0238b66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f5cb00 0x560479f68a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.983+0000 7f0238b66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560479f5fa50 con 0x560479f56fa0 2024-04-17T07:20:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.983+0000 7f0238b66740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560479e76830 con 0x560479f57990 2024-04-17T07:20:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.983+0000 7f0238b66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560479f5fbd0 con 0x560479f5cb00 2024-04-17T07:20:52.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.983+0000 7f0238b66740 10 client.0 _ll_register_callbacks cb 0x560479e44fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:52.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.983+0000 7f022ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f56fa0 0x560479f614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.984+0000 7f022e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.984+0000 7f022ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f56fa0 0x560479f614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47710/0 (socket says 172.21.15.86:47710) 2024-04-17T07:20:52.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.984+0000 7f022ea00640 1 -- 172.21.15.86:0/1235571814 learned_addr learned my addr 172.21.15.86:0/1235571814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:52.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.984+0000 7f022f400640 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f5cb00 0x560479f68a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.986+0000 7f022ea00640 1 -- 172.21.15.86:0/1235571814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f5cb00 msgr2=0x560479f68a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.986+0000 7f022ea00640 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f5cb00 0x560479f68a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.986+0000 7f022ea00640 1 -- 172.21.15.86:0/1235571814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 msgr2=0x560479f619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:52.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.986+0000 7f022ea00640 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.986+0000 7f022ea00640 1 -- 172.21.15.86:0/1235571814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560479e7a920 con 0x560479f56fa0 2024-04-17T07:20:52.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.986+0000 7f022e000640 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:52.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.987+0000 7f022ea00640 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f56fa0 0x560479f614b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f022003f040 tx=0x7f0220005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2f8011684656f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:52.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.987+0000 7f0218c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:20:52.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.988+0000 7f0218c00640 1 -- 172.21.15.86:0/1235571814 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022004a030 con 0x560479f56fa0 2024-04-17T07:20:52.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.988+0000 7f0218c00640 1 -- 172.21.15.86:0/1235571814 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0220034c90 con 0x560479f56fa0 2024-04-17T07:20:52.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.988+0000 7f0238b66740 1 -- 172.21.15.86:0/1235571814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0220042070 con 0x560479f56fa0 2024-04-17T07:20:52.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.989+0000 7f0218c00640 1 -- 172.21.15.86:0/1235571814 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0220039040 con 0x560479f56fa0 2024-04-17T07:20:52.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.989+0000 7f0238b66740 10 client.17172 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:52.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.989+0000 7f0238b66740 1 -- 172.21.15.86:0/1235571814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560479e487f0 con 0x560479f56fa0 2024-04-17T07:20:52.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.990+0000 7f0218c00640 1 -- 172.21.15.86:0/1235571814 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0220042070 con 0x560479f56fa0 2024-04-17T07:20:52.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.990+0000 7f0218c00640 1 -- 172.21.15.86:0/1235571814 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f022003c070 con 0x560479f56fa0 2024-04-17T07:20:52.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.991+0000 7f0238b66740 10 client.17172 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:52.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.991+0000 7f0238b66740 10 client.17172 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:20:52.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.991+0000 7f0238b66740 20 client.17172 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:52.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.991+0000 7f0238b66740 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560479f6cb00 0x560479f6eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:52.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.991+0000 7f0238b66740 4 client.17172 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:52.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.991+0000 7f0238b66740 1 -- 172.21.15.86:0/1235571814 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560479ecaa40 con 0x560479f6cb00 2024-04-17T07:20:52.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.992+0000 7f022e000640 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560479f6cb00 0x560479f6eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:52.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.994+0000 7f022e000640 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560479f6cb00 0x560479f6eef0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35be27fd576f0234 server_cookie=6cf91600f6449750 in_seq=0 out_seq=0 2024-04-17T07:20:52.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.994+0000 7f0218c00640 10 client.17172 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:20:52.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.997+0000 7f0218c00640 1 -- 172.21.15.86:0/1235571814 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560479ecaa40 con 0x560479f6cb00 2024-04-17T07:20:52.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.997+0000 7f0218c00640 10 client.17172 handle_command_reply: tid=0 2024-04-17T07:20:52.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.997+0000 7f0238b66740 1 client.17172 shutdown 2024-04-17T07:20:52.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.997+0000 7f0238b66740 1 -- 172.21.15.86:0/1235571814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f56fa0 msgr2=0x560479f614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.997+0000 7f0238b66740 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f56fa0 0x560479f614b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f022003f040 tx=0x7f0220005890 comp rx=0 tx=0).stop 2024-04-17T07:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.998+0000 7f0238b66740 1 -- 172.21.15.86:0/1235571814 shutdown_connections 2024-04-17T07:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.998+0000 7f0238b66740 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f5cb00 0x560479f68a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.998+0000 7f0238b66740 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560479f6cb00 0x560479f6eef0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.998+0000 7f0238b66740 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.998+0000 7f0238b66740 1 --2- 172.21.15.86:0/1235571814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f56fa0 0x560479f614b0 secure :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f022003f040 tx=0x7f0220005890 comp rx=0 tx=0).stop 2024-04-17T07:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.998+0000 7f0238b66740 1 -- 172.21.15.86:0/1235571814 >> 172.21.15.86:0/1235571814 conn(0x560479f52510 msgr2=0x560479f51710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.998+0000 7f0218c00640 0 client.17172 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:20:52.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.999+0000 7f0238b66740 1 -- 172.21.15.86:0/1235571814 shutdown_connections 2024-04-17T07:20:52.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.999+0000 7f0238b66740 1 -- 172.21.15.86:0/1235571814 wait complete. 2024-04-17T07:20:52.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:52.999+0000 7f0238b66740 20 client.17172 trim_cache size 0 max 16384 2024-04-17T07:20:53.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.031+0000 7f0238b66740 1 Processor -- start 2024-04-17T07:20:53.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.031+0000 7f0238b66740 1 -- start start 2024-04-17T07:20:53.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.031+0000 7f0238b66740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f60b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.032+0000 7f0238b66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 0x560479f689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.032+0000 7f0238b66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 0x560479f61040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.032+0000 7f0238b66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02301b9980 con 0x560479f5cb00 2024-04-17T07:20:53.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.032+0000 7f0238b66740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f02301bb4c0 con 0x560479f57990 2024-04-17T07:20:53.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.032+0000 7f0238b66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ba050 con 0x560479f59500 2024-04-17T07:20:53.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.032+0000 7f022f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 0x560479f61040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.032+0000 7f022f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 0x560479f61040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51292/0 (socket says 172.21.15.86:51292) 2024-04-17T07:20:53.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.032+0000 7f022f400640 1 -- 172.21.15.86:0/169521788 learned_addr learned my addr 172.21.15.86:0/169521788 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:53.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.033+0000 7f022ea00640 1 --2- 172.21.15.86:0/169521788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f60b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.033+0000 7f022e000640 1 --2- 172.21.15.86:0/169521788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 0x560479f689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.034+0000 7f022ea00640 1 -- 172.21.15.86:0/169521788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 msgr2=0x560479f61040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:53.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.034+0000 7f022ea00640 1 --2- 172.21.15.86:0/169521788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 0x560479f61040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.034+0000 7f022ea00640 1 -- 172.21.15.86:0/169521788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 msgr2=0x560479f689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:53.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.034+0000 7f022ea00640 1 --2- 172.21.15.86:0/169521788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 0x560479f689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.034+0000 7f022ea00640 1 -- 172.21.15.86:0/169521788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560479bcf3f0 con 0x560479f57990 2024-04-17T07:20:53.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.035+0000 7f022ea00640 1 --2- 172.21.15.86:0/169521788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f60b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0220047000 tx=0x7f02200095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33cdf3ac5d3a6dcd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:53.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.035+0000 7f021b400640 1 -- 172.21.15.86:0/169521788 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022004a070 con 0x560479f57990 2024-04-17T07:20:53.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.036+0000 7f021b400640 1 -- 172.21.15.86:0/169521788 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0220042070 con 0x560479f57990 2024-04-17T07:20:53.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.036+0000 7f021b400640 1 -- 172.21.15.86:0/169521788 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022003de60 con 0x560479f57990 2024-04-17T07:20:53.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.037+0000 7f0238b66740 1 -- 172.21.15.86:0/169521788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 msgr2=0x560479f60b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.037+0000 7f0238b66740 1 --2- 172.21.15.86:0/169521788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f60b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0220047000 tx=0x7f02200095b0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.037+0000 7f0238b66740 1 -- 172.21.15.86:0/169521788 shutdown_connections 2024-04-17T07:20:53.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.037+0000 7f0238b66740 1 --2- 172.21.15.86:0/169521788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 0x560479f61040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.037+0000 7f0238b66740 1 --2- 172.21.15.86:0/169521788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f5cb00 0x560479f689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.037+0000 7f0238b66740 1 --2- 172.21.15.86:0/169521788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f57990 0x560479f60b00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.037+0000 7f0238b66740 1 -- 172.21.15.86:0/169521788 >> 172.21.15.86:0/169521788 conn(0x560479f52510 msgr2=0x560479f56000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:53.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.038+0000 7f0238b66740 1 -- 172.21.15.86:0/169521788 shutdown_connections 2024-04-17T07:20:53.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.038+0000 7f0238b66740 1 -- 172.21.15.86:0/169521788 wait complete. 2024-04-17T07:20:53.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.038+0000 7f0238b66740 1 -- start start 2024-04-17T07:20:53.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.038+0000 7f0238b66740 1 Processor -- start 2024-04-17T07:20:53.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.039+0000 7f0238b66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f57990 0x560479f663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.039+0000 7f0238b66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 0x560479f668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.039+0000 7f0238b66740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f5cb00 0x560479f678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.039+0000 7f0238b66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02301badf0 con 0x560479f57990 2024-04-17T07:20:53.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.039+0000 7f0238b66740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f02301ba720 con 0x560479f5cb00 2024-04-17T07:20:53.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.039+0000 7f0238b66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f02301b7950 con 0x560479f59500 2024-04-17T07:20:53.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.039+0000 7f0238b66740 10 client.0 _ll_register_callbacks cb 0x7f02200096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:53.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.040+0000 7f022ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f57990 0x560479f663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.040+0000 7f022ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f57990 0x560479f663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47718/0 (socket says 172.21.15.86:47718) 2024-04-17T07:20:53.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.040+0000 7f022ea00640 1 -- 172.21.15.86:0/1566820131 learned_addr learned my addr 172.21.15.86:0/1566820131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:53.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.040+0000 7f022f400640 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f5cb00 0x560479f678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.040+0000 7f022e000640 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 0x560479f668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.040+0000 7f022ea00640 1 -- 172.21.15.86:0/1566820131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 msgr2=0x560479f668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.040+0000 7f022ea00640 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 0x560479f668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.040+0000 7f022ea00640 1 -- 172.21.15.86:0/1566820131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f5cb00 msgr2=0x560479f678f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.041+0000 7f022ea00640 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f5cb00 0x560479f678f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.041+0000 7f022ea00640 1 -- 172.21.15.86:0/1566820131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560479c997d0 con 0x560479f57990 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.041+0000 7f022f400640 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f5cb00 0x560479f678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.041+0000 7f022ea00640 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f57990 0x560479f663a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f02200349e0 tx=0x7f02200056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4771de87ec1203 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.041+0000 7f0218c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.041+0000 7f0218c00640 1 -- 172.21.15.86:0/1566820131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220060060 con 0x560479f57990 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.042+0000 7f0238b66740 1 -- 172.21.15.86:0/1566820131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0220042070 con 0x560479f57990 2024-04-17T07:20:53.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.042+0000 7f0218c00640 1 -- 172.21.15.86:0/1566820131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f022002c9b0 con 0x560479f57990 2024-04-17T07:20:53.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.042+0000 7f0218c00640 1 -- 172.21.15.86:0/1566820131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0220067050 con 0x560479f57990 2024-04-17T07:20:53.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.042+0000 7f0238b66740 10 client.17178 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:53.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.042+0000 7f0238b66740 1 -- 172.21.15.86:0/1566820131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560479fd2e80 con 0x560479f57990 2024-04-17T07:20:53.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.043+0000 7f0218c00640 1 -- 172.21.15.86:0/1566820131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0220042070 con 0x560479f57990 2024-04-17T07:20:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.043+0000 7f0218c00640 1 -- 172.21.15.86:0/1566820131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f022003c040 con 0x560479f57990 2024-04-17T07:20:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.044+0000 7f0238b66740 10 client.17178 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.044+0000 7f0238b66740 10 client.17178 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:20:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.044+0000 7f0238b66740 20 client.17178 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.044+0000 7f0238b66740 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560479fd3640 0x560479fd5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.044+0000 7f0238b66740 4 client.17178 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:20:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.044+0000 7f0238b66740 1 -- 172.21.15.86:0/1566820131 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560479ec48b0 con 0x560479fd3640 2024-04-17T07:20:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.044+0000 7f022e000640 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560479fd3640 0x560479fd5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.045+0000 7f022e000640 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560479fd3640 0x560479fd5a60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c597155b6db8ca8 server_cookie=d8c5b2b8d39dc352 in_seq=0 out_seq=0 2024-04-17T07:20:53.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.045+0000 7f0218c00640 10 client.17178 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:20:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.045+0000 7f0218c00640 1 -- 172.21.15.86:0/1566820131 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560479ec48b0 con 0x560479fd3640 2024-04-17T07:20:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.045+0000 7f0218c00640 10 client.17178 handle_command_reply: tid=0 2024-04-17T07:20:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.045+0000 7f0238b66740 1 client.17178 shutdown 2024-04-17T07:20:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.046+0000 7f0238b66740 1 -- 172.21.15.86:0/1566820131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f57990 msgr2=0x560479f663a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.046+0000 7f0238b66740 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f57990 0x560479f663a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f02200349e0 tx=0x7f02200056b0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.046+0000 7f0238b66740 1 -- 172.21.15.86:0/1566820131 shutdown_connections 2024-04-17T07:20:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.046+0000 7f0238b66740 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x560479fd3640 0x560479fd5a60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.046+0000 7f0238b66740 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560479f5cb00 0x560479f678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.046+0000 7f0238b66740 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560479f59500 0x560479f668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.046+0000 7f0238b66740 1 --2- 172.21.15.86:0/1566820131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560479f57990 0x560479f663a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.046+0000 7f0238b66740 1 -- 172.21.15.86:0/1566820131 >> 172.21.15.86:0/1566820131 conn(0x560479f52510 msgr2=0x560479f50e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.046+0000 7f0218c00640 0 client.17178 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:20:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.046+0000 7f0238b66740 1 -- 172.21.15.86:0/1566820131 shutdown_connections 2024-04-17T07:20:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.047+0000 7f0238b66740 1 -- 172.21.15.86:0/1566820131 wait complete. 2024-04-17T07:20:53.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.048+0000 7f0238b66740 20 client.17178 trim_cache size 0 max 16384 2024-04-17T07:20:53.048 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:20:53.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.050+0000 7f0235e00640 1 -- 172.21.15.86:0/2555694637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02140a3c30 msgr2=0x7f02140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.050+0000 7f0235e00640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02140a3c30 0x7f02140a60f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x560479f57d90 tx=0x7f021c009460 comp rx=0 tx=0).stop 2024-04-17T07:20:53.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.050+0000 7f0235e00640 1 -- 172.21.15.86:0/2555694637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230109e70 msgr2=0x7f0230116880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.050+0000 7f0235e00640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230109e70 0x7f0230116880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f022800eaa0 tx=0x7f022800ef60 comp rx=0 tx=0).stop 2024-04-17T07:20:53.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.050+0000 7f0235e00640 1 -- 172.21.15.86:0/2555694637 shutdown_connections 2024-04-17T07:20:53.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.050+0000 7f0235e00640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02140a3c30 0x7f02140a60f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.050+0000 7f0235e00640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0230109e70 0x7f0230116880 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.050+0000 7f0235e00640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0230075fa0 0x7f0230111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.050+0000 7f0235e00640 1 --2- 172.21.15.86:0/2555694637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02300749c0 0x7f02301112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.050+0000 7f0235e00640 1 -- 172.21.15.86:0/2555694637 >> 172.21.15.86:0/2555694637 conn(0x7f02300fd540 msgr2=0x7f0230102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:53.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.051+0000 7f0235e00640 1 -- 172.21.15.86:0/2555694637 shutdown_connections 2024-04-17T07:20:53.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.052+0000 7f0235e00640 1 -- 172.21.15.86:0/2555694637 wait complete. 2024-04-17T07:20:53.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:20:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:53 smithi086 ceph-mon[31733]: pgmap v635: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 MiB/s wr, 800 op/s 2024-04-17T07:20:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.389+0000 7f2851000640 1 -- 172.21.15.86:0/1421204469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f284c1085c0 msgr2=0x7f284c1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.389+0000 7f2851000640 1 --2- 172.21.15.86:0/1421204469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f284c1085c0 0x7f284c1089e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f283c01a910 tx=0x7f283c03cc60 comp rx=0 tx=0).stop 2024-04-17T07:20:53.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.390+0000 7f2851000640 1 -- 172.21.15.86:0/1421204469 shutdown_connections 2024-04-17T07:20:53.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.390+0000 7f2851000640 1 --2- 172.21.15.86:0/1421204469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f284c10a170 0x7f284c110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.390+0000 7f2851000640 1 --2- 172.21.15.86:0/1421204469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f284c1097b0 0x7f284c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.390+0000 7f2851000640 1 --2- 172.21.15.86:0/1421204469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f284c1085c0 0x7f284c1089e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.390+0000 7f2851000640 1 -- 172.21.15.86:0/1421204469 >> 172.21.15.86:0/1421204469 conn(0x7f284c077560 msgr2=0x7f284c0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:53.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.393+0000 7f2851000640 1 -- 172.21.15.86:0/1421204469 shutdown_connections 2024-04-17T07:20:53.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.393+0000 7f2851000640 1 -- 172.21.15.86:0/1421204469 wait complete. 2024-04-17T07:20:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.394+0000 7f2851000640 1 Processor -- start 2024-04-17T07:20:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.395+0000 7f2851000640 1 -- start start 2024-04-17T07:20:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.395+0000 7f2851000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f284c1085c0 0x7f284c07a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.395+0000 7f2851000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f284c1097b0 0x7f284c07af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.395+0000 7f2851000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f284c10a170 0x7f284c0819f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.395+0000 7f2851000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f284c06e480 con 0x7f284c1097b0 2024-04-17T07:20:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.395+0000 7f2851000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f284c06e300 con 0x7f284c1085c0 2024-04-17T07:20:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.395+0000 7f2851000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f284c06e600 con 0x7f284c10a170 2024-04-17T07:20:53.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.396+0000 7f2849400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f284c1097b0 0x7f284c07af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.396+0000 7f2849400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f284c1097b0 0x7f284c07af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47744/0 (socket says 172.21.15.86:47744) 2024-04-17T07:20:53.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.396+0000 7f2849400640 1 -- 172.21.15.86:0/1939861096 learned_addr learned my addr 172.21.15.86:0/1939861096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:53.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.397+0000 7f2849e00640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f284c1085c0 0x7f284c07a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.397+0000 7f2849400640 1 -- 172.21.15.86:0/1939861096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f284c10a170 msgr2=0x7f284c0819f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.397+0000 7f2849400640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f284c10a170 0x7f284c0819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.397+0000 7f2849400640 1 -- 172.21.15.86:0/1939861096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f284c1085c0 msgr2=0x7f284c07a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.397+0000 7f2849400640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f284c1085c0 0x7f284c07a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.397+0000 7f2849400640 1 -- 172.21.15.86:0/1939861096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c082090 con 0x7f284c1097b0 2024-04-17T07:20:53.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.398+0000 7f2849400640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f284c1097b0 0x7f284c07af30 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f283800e9f0 tx=0x7f283800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:53.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.398+0000 7f2842a00640 1 -- 172.21.15.86:0/1939861096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283800cd80 con 0x7f284c1097b0 2024-04-17T07:20:53.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.398+0000 7f2842a00640 1 -- 172.21.15.86:0/1939861096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28380108e0 con 0x7f284c1097b0 2024-04-17T07:20:53.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.398+0000 7f2842a00640 1 -- 172.21.15.86:0/1939861096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838018920 con 0x7f284c1097b0 2024-04-17T07:20:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.399+0000 7f2851000640 1 -- 172.21.15.86:0/1939861096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c082380 con 0x7f284c1097b0 2024-04-17T07:20:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.399+0000 7f2851000640 1 -- 172.21.15.86:0/1939861096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c082880 con 0x7f284c1097b0 2024-04-17T07:20:53.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.403+0000 7f2853df9740 1 Processor -- start 2024-04-17T07:20:53.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.404+0000 7f2842a00640 1 -- 172.21.15.86:0/1939861096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2838010a90 con 0x7f284c1097b0 2024-04-17T07:20:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.404+0000 7f2853df9740 1 -- start start 2024-04-17T07:20:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.405+0000 7f2853df9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea719fa0 0x5564ea71a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.405+0000 7f2853df9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea71f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.405+0000 7f2853df9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71fb00 0x5564ea721f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.405+0000 7f2853df9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564ea680020 con 0x5564ea719fa0 2024-04-17T07:20:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.405+0000 7f2853df9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5564ea626460 con 0x5564ea71fb00 2024-04-17T07:20:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.405+0000 7f2853df9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5564ea67bfe0 con 0x5564ea71a990 2024-04-17T07:20:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.406+0000 7f2849e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea71f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.406+0000 7f2849e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea71f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51342/0 (socket says 172.21.15.86:51342) 2024-04-17T07:20:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.406+0000 7f2849e00640 1 -- 172.21.15.86:0/4140879263 learned_addr learned my addr 172.21.15.86:0/4140879263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:53.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.406+0000 7f284a800640 1 --2- 172.21.15.86:0/4140879263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea719fa0 0x5564ea71a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.406+0000 7f2849e00640 1 -- 172.21.15.86:0/4140879263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71fb00 msgr2=0x5564ea721f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:20:53.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.406+0000 7f2849e00640 1 --2- 172.21.15.86:0/4140879263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71fb00 0x5564ea721f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.406+0000 7f2849e00640 1 -- 172.21.15.86:0/4140879263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea719fa0 msgr2=0x5564ea71a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.406+0000 7f2849e00640 1 --2- 172.21.15.86:0/4140879263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea719fa0 0x5564ea71a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.406+0000 7f2849e00640 1 -- 172.21.15.86:0/4140879263 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ea62e040 con 0x5564ea71a990 2024-04-17T07:20:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.406+0000 7f284a800640 1 --2- 172.21.15.86:0/4140879263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea719fa0 0x5564ea71a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.407+0000 7f2849e00640 1 --2- 172.21.15.86:0/4140879263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea71f490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f283c0057c0 tx=0x7f283c01a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c0ea7c77c18eac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.407+0000 7f2837400640 1 -- 172.21.15.86:0/4140879263 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c012070 con 0x5564ea71a990 2024-04-17T07:20:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.407+0000 7f2842a00640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28300c5cc0 0x7f28300c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.407+0000 7f2837400640 1 -- 172.21.15.86:0/4140879263 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f283c004b40 con 0x5564ea71a990 2024-04-17T07:20:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.407+0000 7f2837400640 1 -- 172.21.15.86:0/4140879263 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c004e80 con 0x5564ea71a990 2024-04-17T07:20:53.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.408+0000 7f2853df9740 1 -- 172.21.15.86:0/4140879263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 msgr2=0x5564ea71f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.408+0000 7f2853df9740 1 --2- 172.21.15.86:0/4140879263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea71f490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f283c0057c0 tx=0x7f283c01a5f0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.408+0000 7f2842a00640 1 -- 172.21.15.86:0/1939861096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f28380d0c30 con 0x7f284c1097b0 2024-04-17T07:20:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.408+0000 7f284a800640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28300c5cc0 0x7f28300c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.409+0000 7f2853df9740 1 -- 172.21.15.86:0/4140879263 shutdown_connections 2024-04-17T07:20:53.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.409+0000 7f2853df9740 1 --2- 172.21.15.86:0/4140879263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71fb00 0x5564ea721f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.409+0000 7f2853df9740 1 --2- 172.21.15.86:0/4140879263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea71f490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.409+0000 7f2853df9740 1 --2- 172.21.15.86:0/4140879263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea719fa0 0x5564ea71a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.409+0000 7f2853df9740 1 -- 172.21.15.86:0/4140879263 >> 172.21.15.86:0/4140879263 conn(0x5564ea715510 msgr2=0x5564ea715910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:53.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.410+0000 7f2853df9740 1 -- 172.21.15.86:0/4140879263 shutdown_connections 2024-04-17T07:20:53.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.410+0000 7f284a800640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28300c5cc0 0x7f28300c8180 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5564ea71b1f0 tx=0x7f2844009ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:53.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.410+0000 7f2853df9740 1 -- 172.21.15.86:0/4140879263 wait complete. 2024-04-17T07:20:53.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.411+0000 7f2853df9740 1 -- start start 2024-04-17T07:20:53.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.411+0000 7f2853df9740 1 Processor -- start 2024-04-17T07:20:53.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.412+0000 7f2853df9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.412+0000 7f2853df9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71a990 0x5564ea7249f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.412+0000 7f2849e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.412+0000 7f2853df9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea72ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.412+0000 7f2853df9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5564ea722a50 con 0x5564ea71fb00 2024-04-17T07:20:53.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.412+0000 7f2849e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7244b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51352/0 (socket says 172.21.15.86:51352) 2024-04-17T07:20:53.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.412+0000 7f2849e00640 1 -- 172.21.15.86:0/4040648352 learned_addr learned my addr 172.21.15.86:0/4040648352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:53.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.412+0000 7f2853df9740 1 -- 172.21.15.86:0/4040648352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5564ea639830 con 0x5564ea71a990 2024-04-17T07:20:53.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.412+0000 7f2853df9740 1 -- 172.21.15.86:0/4040648352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5564ea722bd0 con 0x5564ea719fa0 2024-04-17T07:20:53.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.412+0000 7f2853df9740 10 client.0 _ll_register_callbacks cb 0x5564ea607fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:53.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.413+0000 7f2849400640 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71a990 0x5564ea7249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.413+0000 7f284a800640 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea72ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.413+0000 7f2849e00640 1 -- 172.21.15.86:0/4040648352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71a990 msgr2=0x5564ea7249f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:53.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.413+0000 7f2849e00640 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71a990 0x5564ea7249f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.413+0000 7f2849e00640 1 -- 172.21.15.86:0/4040648352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 msgr2=0x5564ea72ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:53.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.413+0000 7f2849e00640 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea72ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.413+0000 7f2849e00640 1 -- 172.21.15.86:0/4040648352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ea63d920 con 0x5564ea719fa0 2024-04-17T07:20:53.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.413+0000 7f284a800640 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea72ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:53.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.414+0000 7f2849e00640 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7244b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f283c047040 tx=0x7f283c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb4f3ca6311d975 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:53.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.414+0000 7f282be00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:53.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.415+0000 7f282be00640 1 -- 172.21.15.86:0/4040648352 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c052060 con 0x5564ea719fa0 2024-04-17T07:20:53.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.415+0000 7f282be00640 1 -- 172.21.15.86:0/4040648352 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f283c005290 con 0x5564ea719fa0 2024-04-17T07:20:53.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.415+0000 7f2853df9740 1 -- 172.21.15.86:0/4040648352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283c004b40 con 0x5564ea719fa0 2024-04-17T07:20:53.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.416+0000 7f282be00640 1 -- 172.21.15.86:0/4040648352 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f283c012030 con 0x5564ea719fa0 2024-04-17T07:20:53.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.416+0000 7f2853df9740 10 client.26221 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:53.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.416+0000 7f2853df9740 1 -- 172.21.15.86:0/4040648352 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564ea60b7f0 con 0x5564ea719fa0 2024-04-17T07:20:53.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.417+0000 7f282be00640 1 -- 172.21.15.86:0/4040648352 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f283c004b40 con 0x5564ea719fa0 2024-04-17T07:20:53.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.417+0000 7f282be00640 1 -- 172.21.15.86:0/4040648352 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f283c017450 con 0x5564ea719fa0 2024-04-17T07:20:53.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.417+0000 7f2853df9740 10 client.26221 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.417+0000 7f2853df9740 10 client.26221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.417+0000 7f2853df9740 20 client.26221 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.417+0000 7f2853df9740 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5564ea72fb90 0x5564ea731f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.417+0000 7f2853df9740 4 client.26221 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.417+0000 7f2853df9740 1 -- 172.21.15.86:0/4040648352 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564ea68da40 con 0x5564ea72fb90 2024-04-17T07:20:53.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.418+0000 7f2849400640 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5564ea72fb90 0x5564ea731f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.419+0000 7f282be00640 10 client.26221 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:53.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.421+0000 7f2849400640 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5564ea72fb90 0x5564ea731f80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bc82efd37ea0e66 server_cookie=32c0f8666ae4f87d in_seq=0 out_seq=0 2024-04-17T07:20:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.422+0000 7f282be00640 1 -- 172.21.15.86:0/4040648352 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564ea68da40 con 0x5564ea72fb90 2024-04-17T07:20:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.422+0000 7f282be00640 10 client.26221 handle_command_reply: tid=0 2024-04-17T07:20:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.423+0000 7f2853df9740 1 client.26221 shutdown 2024-04-17T07:20:53.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.423+0000 7f2853df9740 1 -- 172.21.15.86:0/4040648352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 msgr2=0x5564ea7244b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.423+0000 7f2853df9740 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7244b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f283c047040 tx=0x7f283c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.423+0000 7f2853df9740 1 -- 172.21.15.86:0/4040648352 shutdown_connections 2024-04-17T07:20:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.423+0000 7f2853df9740 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5564ea72fb90 0x5564ea731f80 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.423+0000 7f2853df9740 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea72ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.424+0000 7f2853df9740 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71a990 0x5564ea7249f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.423+0000 7f282be00640 0 client.26221 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.424+0000 7f2853df9740 1 --2- 172.21.15.86:0/4040648352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7244b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.424+0000 7f2853df9740 1 -- 172.21.15.86:0/4040648352 >> 172.21.15.86:0/4040648352 conn(0x5564ea715510 msgr2=0x5564ea714710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.424+0000 7f2853df9740 1 -- 172.21.15.86:0/4040648352 shutdown_connections 2024-04-17T07:20:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.424+0000 7f2853df9740 1 -- 172.21.15.86:0/4040648352 wait complete. 2024-04-17T07:20:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.425+0000 7f2853df9740 20 client.26221 trim_cache size 0 max 16384 2024-04-17T07:20:53.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.464+0000 7f2853df9740 1 Processor -- start 2024-04-17T07:20:53.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.465+0000 7f2853df9740 1 -- start start 2024-04-17T07:20:53.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.466+0000 7f2853df9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea719fa0 0x5564ea713e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.466+0000 7f2853df9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea72b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.466+0000 7f2853df9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea7143c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.466+0000 7f2853df9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b11a0 con 0x5564ea71fb00 2024-04-17T07:20:53.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.466+0000 7f2853df9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b2ce0 con 0x5564ea719fa0 2024-04-17T07:20:53.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.466+0000 7f2853df9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b1870 con 0x5564ea71a990 2024-04-17T07:20:53.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.467+0000 7f284a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea7143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.467+0000 7f284a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea7143c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47768/0 (socket says 172.21.15.86:47768) 2024-04-17T07:20:53.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.467+0000 7f284a800640 1 -- 172.21.15.86:0/257061623 learned_addr learned my addr 172.21.15.86:0/257061623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:53.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.468+0000 7f2849400640 1 --2- 172.21.15.86:0/257061623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea72b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.468+0000 7f2849e00640 1 --2- 172.21.15.86:0/257061623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea719fa0 0x5564ea713e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.468+0000 7f2849400640 1 -- 172.21.15.86:0/257061623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea719fa0 msgr2=0x5564ea713e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:53.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.468+0000 7f2849400640 1 --2- 172.21.15.86:0/257061623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea719fa0 0x5564ea713e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.468+0000 7f2849400640 1 -- 172.21.15.86:0/257061623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 msgr2=0x5564ea7143c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:53.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.468+0000 7f2849400640 1 --2- 172.21.15.86:0/257061623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea7143c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.468+0000 7f2849400640 1 -- 172.21.15.86:0/257061623 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ea7322f0 con 0x5564ea71a990 2024-04-17T07:20:53.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.469+0000 7f2849400640 1 --2- 172.21.15.86:0/257061623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea72b9f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f28380ce350 tx=0x7f28380ce220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9776da519a4e24f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:53.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.471+0000 7f2836a00640 1 -- 172.21.15.86:0/257061623 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283809de60 con 0x5564ea71a990 2024-04-17T07:20:53.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.471+0000 7f2836a00640 1 -- 172.21.15.86:0/257061623 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f283809cb30 con 0x5564ea71a990 2024-04-17T07:20:53.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.471+0000 7f2836a00640 1 -- 172.21.15.86:0/257061623 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283809bf80 con 0x5564ea71a990 2024-04-17T07:20:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.472+0000 7f2853df9740 1 -- 172.21.15.86:0/257061623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 msgr2=0x5564ea72b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.472+0000 7f2853df9740 1 --2- 172.21.15.86:0/257061623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea72b9f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f28380ce350 tx=0x7f28380ce220 comp rx=0 tx=0).stop 2024-04-17T07:20:53.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.475+0000 7f2853df9740 1 -- 172.21.15.86:0/257061623 shutdown_connections 2024-04-17T07:20:53.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.475+0000 7f2853df9740 1 --2- 172.21.15.86:0/257061623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea7143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.475+0000 7f2853df9740 1 --2- 172.21.15.86:0/257061623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea71a990 0x5564ea72b9f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.475+0000 7f2853df9740 1 --2- 172.21.15.86:0/257061623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea719fa0 0x5564ea713e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.475+0000 7f2853df9740 1 -- 172.21.15.86:0/257061623 >> 172.21.15.86:0/257061623 conn(0x5564ea715510 msgr2=0x5564ea71bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:53.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.475+0000 7f2853df9740 1 -- 172.21.15.86:0/257061623 shutdown_connections 2024-04-17T07:20:53.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.475+0000 7f2853df9740 1 -- 172.21.15.86:0/257061623 wait complete. 2024-04-17T07:20:53.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.476+0000 7f2853df9740 1 -- start start 2024-04-17T07:20:53.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.476+0000 7f2853df9740 1 Processor -- start 2024-04-17T07:20:53.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.477+0000 7f2853df9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.477+0000 7f2853df9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71a990 0x5564ea728ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.477+0000 7f2853df9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea794040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.477+0000 7f2853df9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b2610 con 0x5564ea71fb00 2024-04-17T07:20:53.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.477+0000 7f2853df9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b1f40 con 0x5564ea71a990 2024-04-17T07:20:53.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.478+0000 7f2853df9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1af170 con 0x5564ea719fa0 2024-04-17T07:20:53.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.478+0000 7f2853df9740 10 client.0 _ll_register_callbacks cb 0x7f283c004940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:53.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.479+0000 7f2849e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.479+0000 7f284a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea794040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.479+0000 7f2849e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7287a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51372/0 (socket says 172.21.15.86:51372) 2024-04-17T07:20:53.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.479+0000 7f2849e00640 1 -- 172.21.15.86:0/1487919606 learned_addr learned my addr 172.21.15.86:0/1487919606 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:53.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.480+0000 7f2849e00640 1 -- 172.21.15.86:0/1487919606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71a990 msgr2=0x5564ea728ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:20:53.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.480+0000 7f2849e00640 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71a990 0x5564ea728ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.480+0000 7f2849e00640 1 -- 172.21.15.86:0/1487919606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 msgr2=0x5564ea794040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:53.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.480+0000 7f2849e00640 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea794040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.480+0000 7f2849e00640 1 -- 172.21.15.86:0/1487919606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ea3923f0 con 0x5564ea719fa0 2024-04-17T07:20:53.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.481+0000 7f2849e00640 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7287a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f283c0048a0 tx=0x7f283c01a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=201ca4a57088aaee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:53.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.482+0000 7f282be00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:20:53.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.482+0000 7f282be00640 1 -- 172.21.15.86:0/1487919606 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c052070 con 0x5564ea719fa0 2024-04-17T07:20:53.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.482+0000 7f282be00640 1 -- 172.21.15.86:0/1487919606 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f283c017a60 con 0x5564ea719fa0 2024-04-17T07:20:53.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.482+0000 7f2853df9740 1 -- 172.21.15.86:0/1487919606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f283809cb30 con 0x5564ea719fa0 2024-04-17T07:20:53.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.482+0000 7f282be00640 1 -- 172.21.15.86:0/1487919606 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f283c05c080 con 0x5564ea719fa0 2024-04-17T07:20:53.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.483+0000 7f2853df9740 10 client.26227 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:53.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.483+0000 7f2853df9740 1 -- 172.21.15.86:0/1487919606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564ea45c7d0 con 0x5564ea719fa0 2024-04-17T07:20:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.483+0000 7f282be00640 1 -- 172.21.15.86:0/1487919606 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f283809cb30 con 0x5564ea719fa0 2024-04-17T07:20:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.484+0000 7f282be00640 1 -- 172.21.15.86:0/1487919606 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f283c005350 con 0x5564ea719fa0 2024-04-17T07:20:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.484+0000 7f2853df9740 10 client.26227 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.484+0000 7f2853df9740 10 client.26227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.484+0000 7f2853df9740 20 client.26227 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.484+0000 7f2853df9740 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5564ea72fb90 0x5564ea797b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.484+0000 7f2853df9740 4 client.26227 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:20:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.484+0000 7f2853df9740 1 -- 172.21.15.86:0/1487919606 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564ea6878b0 con 0x5564ea72fb90 2024-04-17T07:20:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.484+0000 7f2849400640 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5564ea72fb90 0x5564ea797b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.486+0000 7f2849400640 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5564ea72fb90 0x5564ea797b60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293499c9d3e73e4f server_cookie=b4ce2870424c08ac in_seq=0 out_seq=0 2024-04-17T07:20:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.486+0000 7f282be00640 10 client.26227 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:53.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f282be00640 1 -- 172.21.15.86:0/1487919606 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564ea6878b0 con 0x5564ea72fb90 2024-04-17T07:20:53.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f282be00640 10 client.26227 handle_command_reply: tid=0 2024-04-17T07:20:53.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f2853df9740 1 client.26227 shutdown 2024-04-17T07:20:53.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f2853df9740 1 -- 172.21.15.86:0/1487919606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 msgr2=0x5564ea7287a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f2853df9740 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7287a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f283c0048a0 tx=0x7f283c01a660 comp rx=0 tx=0).stop 2024-04-17T07:20:53.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f2853df9740 1 -- 172.21.15.86:0/1487919606 shutdown_connections 2024-04-17T07:20:53.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f2853df9740 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5564ea72fb90 0x5564ea797b60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f2853df9740 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5564ea71fb00 0x5564ea794040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f2853df9740 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564ea71a990 0x5564ea728ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f2853df9740 1 --2- 172.21.15.86:0/1487919606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5564ea719fa0 0x5564ea7287a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f2853df9740 1 -- 172.21.15.86:0/1487919606 >> 172.21.15.86:0/1487919606 conn(0x5564ea715510 msgr2=0x5564ea718770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:53.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.487+0000 7f282be00640 0 client.26227 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:53.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.488+0000 7f2853df9740 1 -- 172.21.15.86:0/1487919606 shutdown_connections 2024-04-17T07:20:53.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.488+0000 7f2853df9740 1 -- 172.21.15.86:0/1487919606 wait complete. 2024-04-17T07:20:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.488+0000 7f2853df9740 20 client.26227 trim_cache size 0 max 16384 2024-04-17T07:20:53.489 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:20:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.489+0000 7f2851000640 1 -- 172.21.15.86:0/1939861096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28300c5cc0 msgr2=0x7f28300c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.489+0000 7f2851000640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28300c5cc0 0x7f28300c8180 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5564ea71b1f0 tx=0x7f2844009ba0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.489+0000 7f2851000640 1 -- 172.21.15.86:0/1939861096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f284c1097b0 msgr2=0x7f284c07af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.489+0000 7f2851000640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f284c1097b0 0x7f284c07af30 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f283800e9f0 tx=0x7f283800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.489+0000 7f2851000640 1 -- 172.21.15.86:0/1939861096 shutdown_connections 2024-04-17T07:20:53.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.489+0000 7f2851000640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28300c5cc0 0x7f28300c8180 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.489+0000 7f2851000640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f284c10a170 0x7f284c0819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.489+0000 7f2851000640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f284c1097b0 0x7f284c07af30 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.489+0000 7f2851000640 1 --2- 172.21.15.86:0/1939861096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f284c1085c0 0x7f284c07a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:53.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.489+0000 7f2851000640 1 -- 172.21.15.86:0/1939861096 >> 172.21.15.86:0/1939861096 conn(0x7f284c077560 msgr2=0x7f284c070fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:53.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.490+0000 7f2851000640 1 -- 172.21.15.86:0/1939861096 shutdown_connections 2024-04-17T07:20:53.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:53.490+0000 7f2851000640 1 -- 172.21.15.86:0/1939861096 wait complete. 2024-04-17T07:20:53.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:53 smithi098 ceph-mon[35418]: pgmap v635: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 MiB/s wr, 800 op/s 2024-04-17T07:20:53.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:53 smithi103 ceph-mon[35495]: pgmap v635: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.9 MiB/s wr, 800 op/s 2024-04-17T07:20:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 90.0 s, 553.0 tps, lat 184.654 ms stddev 144.300 2024-04-17T07:20:54.516 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:20:54.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:20:54.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:20:54.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.840+0000 7f924f000640 1 -- 172.21.15.86:0/47759025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92480ffb80 msgr2=0x7f92480fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:54.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.840+0000 7f924f000640 1 --2- 172.21.15.86:0/47759025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92480ffb80 0x7f92480fffa0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9238009860 tx=0x7f923802bb70 comp rx=0 tx=0).stop 2024-04-17T07:20:54.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.840+0000 7f924f000640 1 -- 172.21.15.86:0/47759025 shutdown_connections 2024-04-17T07:20:54.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.840+0000 7f924f000640 1 --2- 172.21.15.86:0/47759025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248107c90 0x7f924810a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.840+0000 7f924f000640 1 --2- 172.21.15.86:0/47759025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92481004e0 0x7f9248107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.840+0000 7f924f000640 1 --2- 172.21.15.86:0/47759025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92480ffb80 0x7f92480fffa0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.840+0000 7f924f000640 1 -- 172.21.15.86:0/47759025 >> 172.21.15.86:0/47759025 conn(0x7f92480fb760 msgr2=0x7f92480fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:54.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.841+0000 7f924f000640 1 -- 172.21.15.86:0/47759025 shutdown_connections 2024-04-17T07:20:54.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.841+0000 7f924f000640 1 -- 172.21.15.86:0/47759025 wait complete. 2024-04-17T07:20:54.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.842+0000 7f924f000640 1 Processor -- start 2024-04-17T07:20:54.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.842+0000 7f924f000640 1 -- start start 2024-04-17T07:20:54.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.843+0000 7f924f000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92481004e0 0x7f9248112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.843+0000 7f924f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9248107c90 0x7f9248113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.844+0000 7f924f000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248111330 0x7f92481117e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.844+0000 7f9247e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92481004e0 0x7f9248112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.844+0000 7f9247e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92481004e0 0x7f9248112cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41546/0 (socket says 172.21.15.86:41546) 2024-04-17T07:20:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.844+0000 7f9247400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9248107c90 0x7f9248113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.844+0000 7f9247400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9248107c90 0x7f9248113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47800/0 (socket says 172.21.15.86:47800) 2024-04-17T07:20:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.844+0000 7f9247e00640 1 -- 172.21.15.86:0/1600335418 learned_addr learned my addr 172.21.15.86:0/1600335418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:54.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.844+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f924810b900 con 0x7f9248107c90 2024-04-17T07:20:54.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.845+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f924810b780 con 0x7f92481004e0 2024-04-17T07:20:54.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.845+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f924810ba80 con 0x7f9248111330 2024-04-17T07:20:54.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.845+0000 7f924d000640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248111330 0x7f92481117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.846+0000 7f9247e00640 1 -- 172.21.15.86:0/1600335418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248111330 msgr2=0x7f92481117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:54.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.846+0000 7f9247e00640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248111330 0x7f92481117e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.846+0000 7f9247e00640 1 -- 172.21.15.86:0/1600335418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9248107c90 msgr2=0x7f9248113230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:54.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.846+0000 7f9247e00640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9248107c90 0x7f9248113230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.846+0000 7f9247e00640 1 -- 172.21.15.86:0/1600335418 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9248112050 con 0x7f92481004e0 2024-04-17T07:20:54.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.846+0000 7f924d000640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248111330 0x7f92481117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:54.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.847+0000 7f9247e00640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92481004e0 0x7f9248112cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9238037040 tx=0x7f92380042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:54.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.848+0000 7f9244c00640 1 -- 172.21.15.86:0/1600335418 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238041020 con 0x7f92481004e0 2024-04-17T07:20:54.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.848+0000 7f9244c00640 1 -- 172.21.15.86:0/1600335418 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f923802c9c0 con 0x7f92481004e0 2024-04-17T07:20:54.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.848+0000 7f9244c00640 1 -- 172.21.15.86:0/1600335418 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238039040 con 0x7f92481004e0 2024-04-17T07:20:54.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.848+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92481af5c0 con 0x7f92481004e0 2024-04-17T07:20:54.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.848+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92481afab0 con 0x7f92481004e0 2024-04-17T07:20:54.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.854+0000 7f9251e33740 1 Processor -- start 2024-04-17T07:20:54.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.854+0000 7f9251e33740 1 -- start start 2024-04-17T07:20:54.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.855+0000 7f9251e33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d4ed0 0x559e542d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.855+0000 7f9251e33740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 0x559e542da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.855+0000 7f9251e33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542dcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.855+0000 7f9251e33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e54229cb0 con 0x559e542d4ed0 2024-04-17T07:20:54.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.855+0000 7f9251e33740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559e541d0370 con 0x559e542d58c0 2024-04-17T07:20:54.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.855+0000 7f9251e33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559e54225c70 con 0x559e542daa30 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.855+0000 7f9244c00640 1 -- 172.21.15.86:0/1600335418 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9238044020 con 0x7f92481004e0 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.856+0000 7f924d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d4ed0 0x559e542d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.856+0000 7f924d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d4ed0 0x559e542d52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47810/0 (socket says 172.21.15.86:47810) 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.856+0000 7f924d000640 1 -- 172.21.15.86:0/1088899589 learned_addr learned my addr 172.21.15.86:0/1088899589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.856+0000 7f9247400640 1 --2- 172.21.15.86:0/1088899589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542dcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.857+0000 7f924d000640 1 -- 172.21.15.86:0/1088899589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 msgr2=0x559e542dcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.857+0000 7f924d000640 1 --2- 172.21.15.86:0/1088899589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542dcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.857+0000 7f924d000640 1 -- 172.21.15.86:0/1088899589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 msgr2=0x559e542da3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.857+0000 7f9247e00640 1 --2- 172.21.15.86:0/1088899589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 0x559e542da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.857+0000 7f924d000640 1 --2- 172.21.15.86:0/1088899589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 0x559e542da3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.857+0000 7f924d000640 1 -- 172.21.15.86:0/1088899589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e541ef910 con 0x559e542d4ed0 2024-04-17T07:20:54.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.858+0000 7f924d000640 1 --2- 172.21.15.86:0/1088899589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d4ed0 0x559e542d52f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f924000c450 tx=0x7f924000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30352ed526573d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:54.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.858+0000 7f923d600640 1 -- 172.21.15.86:0/1088899589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240007430 con 0x559e542d4ed0 2024-04-17T07:20:54.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.858+0000 7f923d600640 1 -- 172.21.15.86:0/1088899589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9240007a80 con 0x559e542d4ed0 2024-04-17T07:20:54.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.858+0000 7f923d600640 1 -- 172.21.15.86:0/1088899589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240007430 con 0x559e542d4ed0 2024-04-17T07:20:54.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.859+0000 7f9244c00640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f922c0a3c70 0x7f922c0a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.859+0000 7f9251e33740 1 -- 172.21.15.86:0/1088899589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d4ed0 msgr2=0x559e542d52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:54.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.859+0000 7f9251e33740 1 --2- 172.21.15.86:0/1088899589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d4ed0 0x559e542d52f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f924000c450 tx=0x7f924000c910 comp rx=0 tx=0).stop 2024-04-17T07:20:54.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.860+0000 7f9244c00640 1 -- 172.21.15.86:0/1600335418 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f92380f2040 con 0x7f92481004e0 2024-04-17T07:20:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.860+0000 7f9247e00640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f922c0a3c70 0x7f922c0a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.860+0000 7f9251e33740 1 -- 172.21.15.86:0/1088899589 shutdown_connections 2024-04-17T07:20:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.860+0000 7f9251e33740 1 --2- 172.21.15.86:0/1088899589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542dcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.860+0000 7f9251e33740 1 --2- 172.21.15.86:0/1088899589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 0x559e542da3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.860+0000 7f9251e33740 1 --2- 172.21.15.86:0/1088899589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d4ed0 0x559e542d52f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.860+0000 7f9251e33740 1 -- 172.21.15.86:0/1088899589 >> 172.21.15.86:0/1088899589 conn(0x559e542bf430 msgr2=0x559e542bf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:54.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.861+0000 7f9251e33740 1 -- 172.21.15.86:0/1088899589 shutdown_connections 2024-04-17T07:20:54.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.861+0000 7f9251e33740 1 -- 172.21.15.86:0/1088899589 wait complete. 2024-04-17T07:20:54.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.861+0000 7f9247e00640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f922c0a3c70 0x7f922c0a6130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f92380efb30 tx=0x7f923802bb70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:54.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.861+0000 7f9251e33740 1 -- start start 2024-04-17T07:20:54.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.862+0000 7f9251e33740 1 Processor -- start 2024-04-17T07:20:54.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.863+0000 7f9251e33740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 0x559e542df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.863+0000 7f9251e33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.863+0000 7f9251e33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.863+0000 7f9251e33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559e542dda30 con 0x559e542d58c0 2024-04-17T07:20:54.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.863+0000 7f9251e33740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559e541e3730 con 0x559e542d4ed0 2024-04-17T07:20:54.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.863+0000 7f9251e33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559e542ddbb0 con 0x559e542daa30 2024-04-17T07:20:54.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.863+0000 7f9251e33740 10 client.0 _ll_register_callbacks cb 0x559e541e5b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f924d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 0x559e542df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f924d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 0x559e542df350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41562/0 (socket says 172.21.15.86:41562) 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f924d000640 1 -- 172.21.15.86:0/2830403318 learned_addr learned my addr 172.21.15.86:0/2830403318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f924d000640 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f9247400640 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f924d000640 1 -- 172.21.15.86:0/2830403318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 msgr2=0x559e542e68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f924d000640 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542e68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f924d000640 1 -- 172.21.15.86:0/2830403318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 msgr2=0x559e542df890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f924d000640 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542df890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f924d000640 1 -- 172.21.15.86:0/2830403318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e540047e0 con 0x559e542d4ed0 2024-04-17T07:20:54.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.864+0000 7f9247400640 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:54.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.865+0000 7f924d000640 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 0x559e542df350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9240007ec0 tx=0x7f9240009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e22dc16acb37226c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:54.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.865+0000 7f9232000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:20:54.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.866+0000 7f9232000640 1 -- 172.21.15.86:0/2830403318 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924001b040 con 0x559e542d4ed0 2024-04-17T07:20:54.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.866+0000 7f9232000640 1 -- 172.21.15.86:0/2830403318 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f924000baf0 con 0x559e542d4ed0 2024-04-17T07:20:54.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.866+0000 7f9251e33740 1 -- 172.21.15.86:0/2830403318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9240007a80 con 0x559e542d4ed0 2024-04-17T07:20:54.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.866+0000 7f9232000640 1 -- 172.21.15.86:0/2830403318 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9240007430 con 0x559e542d4ed0 2024-04-17T07:20:54.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.867+0000 7f9251e33740 10 client.36206 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:54.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.867+0000 7f9251e33740 1 -- 172.21.15.86:0/2830403318 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e54064f70 con 0x559e542d4ed0 2024-04-17T07:20:54.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.867+0000 7f9232000640 1 -- 172.21.15.86:0/2830403318 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9240007a80 con 0x559e542d4ed0 2024-04-17T07:20:54.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.868+0000 7f9232000640 1 -- 172.21.15.86:0/2830403318 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9240021070 con 0x559e542d4ed0 2024-04-17T07:20:54.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.868+0000 7f9251e33740 10 client.36206 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:54.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.868+0000 7f9251e33740 10 client.36206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:54.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.868+0000 7f9251e33740 20 client.36206 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:54.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.868+0000 7f9251e33740 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559e542eaed0 0x559e542ed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.869+0000 7f9251e33740 4 client.36206 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:54.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.869+0000 7f9251e33740 1 -- 172.21.15.86:0/2830403318 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e542376d0 con 0x559e542eaed0 2024-04-17T07:20:54.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.869+0000 7f9247400640 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559e542eaed0 0x559e542ed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.871+0000 7f9247400640 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559e542eaed0 0x559e542ed2c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf57b632b1999534 server_cookie=a9d421f924b6f85e in_seq=0 out_seq=0 2024-04-17T07:20:54.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.871+0000 7f9232000640 10 client.36206 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:54.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.873+0000 7f9232000640 1 -- 172.21.15.86:0/2830403318 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e542376d0 con 0x559e542eaed0 2024-04-17T07:20:54.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.873+0000 7f9232000640 10 client.36206 handle_command_reply: tid=0 2024-04-17T07:20:54.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.874+0000 7f9251e33740 1 client.36206 shutdown 2024-04-17T07:20:54.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 1 -- 172.21.15.86:0/2830403318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 msgr2=0x559e542df350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:54.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 0x559e542df350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9240007ec0 tx=0x7f9240009b20 comp rx=0 tx=0).stop 2024-04-17T07:20:54.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 1 -- 172.21.15.86:0/2830403318 shutdown_connections 2024-04-17T07:20:54.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559e542eaed0 0x559e542ed2c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 1 --2- 172.21.15.86:0/2830403318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 0x559e542df350 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 1 -- 172.21.15.86:0/2830403318 >> 172.21.15.86:0/2830403318 conn(0x559e542bf430 msgr2=0x559e542be5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9232000640 0 client.36206 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 1 -- 172.21.15.86:0/2830403318 shutdown_connections 2024-04-17T07:20:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 1 -- 172.21.15.86:0/2830403318 wait complete. 2024-04-17T07:20:54.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.875+0000 7f9251e33740 20 client.36206 trim_cache size 0 max 16384 2024-04-17T07:20:54.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.918+0000 7f9251e33740 1 Processor -- start 2024-04-17T07:20:54.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.918+0000 7f9251e33740 1 -- start start 2024-04-17T07:20:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.919+0000 7f9251e33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542d4ed0 0x559e542df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.919+0000 7f9251e33740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 0x559e542e6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.919+0000 7f9251e33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542daa30 0x559e542be460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.919+0000 7f9251e33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92481b1880 con 0x559e542daa30 2024-04-17T07:20:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.919+0000 7f9251e33740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92481b33c0 con 0x559e542d58c0 2024-04-17T07:20:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.919+0000 7f9251e33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92481b1f50 con 0x559e542d4ed0 2024-04-17T07:20:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.920+0000 7f924d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542daa30 0x559e542be460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.920+0000 7f924d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542daa30 0x559e542be460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47834/0 (socket says 172.21.15.86:47834) 2024-04-17T07:20:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.920+0000 7f924d000640 1 -- 172.21.15.86:0/4100927618 learned_addr learned my addr 172.21.15.86:0/4100927618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.921+0000 7f924d000640 1 --2- 172.21.15.86:0/4100927618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542d4ed0 0x559e542df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.921+0000 7f9247400640 1 --2- 172.21.15.86:0/4100927618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 0x559e542e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.921+0000 7f9247400640 1 -- 172.21.15.86:0/4100927618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542d4ed0 msgr2=0x559e542df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.921+0000 7f9247400640 1 --2- 172.21.15.86:0/4100927618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542d4ed0 0x559e542df7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.921+0000 7f9247400640 1 -- 172.21.15.86:0/4100927618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542daa30 msgr2=0x559e542be460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.921+0000 7f9247400640 1 --2- 172.21.15.86:0/4100927618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542daa30 0x559e542be460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.921+0000 7f9247400640 1 -- 172.21.15.86:0/4100927618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e53efc170 con 0x559e542d58c0 2024-04-17T07:20:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.921+0000 7f924d000640 1 --2- 172.21.15.86:0/4100927618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542daa30 0x559e542be460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.922+0000 7f9247400640 1 --2- 172.21.15.86:0/4100927618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 0x559e542e6860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x559e542c03a0 tx=0x7f9234009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e09ffc72ccd7dd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.923+0000 7f923cc00640 1 -- 172.21.15.86:0/4100927618 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234003d60 con 0x559e542d58c0 2024-04-17T07:20:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.923+0000 7f923cc00640 1 -- 172.21.15.86:0/4100927618 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9234023020 con 0x559e542d58c0 2024-04-17T07:20:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.923+0000 7f923cc00640 1 -- 172.21.15.86:0/4100927618 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92340108a0 con 0x559e542d58c0 2024-04-17T07:20:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.925+0000 7f9251e33740 1 -- 172.21.15.86:0/4100927618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 msgr2=0x559e542e6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.925+0000 7f9251e33740 1 --2- 172.21.15.86:0/4100927618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 0x559e542e6860 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x559e542c03a0 tx=0x7f9234009260 comp rx=0 tx=0).stop 2024-04-17T07:20:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.925+0000 7f9251e33740 1 -- 172.21.15.86:0/4100927618 shutdown_connections 2024-04-17T07:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.925+0000 7f9251e33740 1 --2- 172.21.15.86:0/4100927618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542daa30 0x559e542be460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.925+0000 7f9251e33740 1 --2- 172.21.15.86:0/4100927618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d58c0 0x559e542e6860 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.925+0000 7f9251e33740 1 --2- 172.21.15.86:0/4100927618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542d4ed0 0x559e542df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.925+0000 7f9251e33740 1 -- 172.21.15.86:0/4100927618 >> 172.21.15.86:0/4100927618 conn(0x559e542bf430 msgr2=0x559e542bdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.926+0000 7f9251e33740 1 -- 172.21.15.86:0/4100927618 shutdown_connections 2024-04-17T07:20:54.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.926+0000 7f9251e33740 1 -- 172.21.15.86:0/4100927618 wait complete. 2024-04-17T07:20:54.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.927+0000 7f9251e33740 1 -- start start 2024-04-17T07:20:54.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.927+0000 7f9251e33740 1 Processor -- start 2024-04-17T07:20:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.929+0000 7f9251e33740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 0x559e542e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.929+0000 7f9251e33740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.929+0000 7f9251e33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542e5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.929+0000 7f9251e33740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92481b2cf0 con 0x559e542d58c0 2024-04-17T07:20:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.929+0000 7f9251e33740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92481b2620 con 0x559e542d4ed0 2024-04-17T07:20:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.929+0000 7f9251e33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9248073c30 con 0x559e542daa30 2024-04-17T07:20:54.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.929+0000 7f9251e33740 10 client.0 _ll_register_callbacks cb 0x7f92340097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f924d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 0x559e542e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f9247400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f9247400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542e4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47838/0 (socket says 172.21.15.86:47838) 2024-04-17T07:20:54.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f9247400640 1 -- 172.21.15.86:0/3883000238 learned_addr learned my addr 172.21.15.86:0/3883000238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f924d000640 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542e5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f9247400640 1 -- 172.21.15.86:0/3883000238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 msgr2=0x559e542e5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f9247400640 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542e5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f9247400640 1 -- 172.21.15.86:0/3883000238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 msgr2=0x559e542e47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f9247400640 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 0x559e542e47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f9247400640 1 -- 172.21.15.86:0/3883000238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e542e6330 con 0x559e542d58c0 2024-04-17T07:20:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.931+0000 7f924d000640 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542e5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:20:54.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.932+0000 7f9247400640 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542e4d30 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9234005890 tx=0x7f9234003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadb5559a0173b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:54.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.932+0000 7f9232000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:20:54.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.933+0000 7f9232000640 1 -- 172.21.15.86:0/3883000238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234004020 con 0x559e542d58c0 2024-04-17T07:20:54.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.933+0000 7f9232000640 1 -- 172.21.15.86:0/3883000238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9234005260 con 0x559e542d58c0 2024-04-17T07:20:54.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.933+0000 7f9251e33740 1 -- 172.21.15.86:0/3883000238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9234023020 con 0x559e542d58c0 2024-04-17T07:20:54.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.933+0000 7f9232000640 1 -- 172.21.15.86:0/3883000238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9234036050 con 0x559e542d58c0 2024-04-17T07:20:54.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.933+0000 7f9251e33740 10 client.17214 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:54.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.933+0000 7f9251e33740 1 -- 172.21.15.86:0/3883000238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e54351b80 con 0x559e542d58c0 2024-04-17T07:20:54.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.934+0000 7f9232000640 1 -- 172.21.15.86:0/3883000238 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9234023020 con 0x559e542d58c0 2024-04-17T07:20:54.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.934+0000 7f9232000640 1 -- 172.21.15.86:0/3883000238 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f923401f3d0 con 0x559e542d58c0 2024-04-17T07:20:54.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.935+0000 7f9251e33740 10 client.17214 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:54.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.935+0000 7f9251e33740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:54.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.935+0000 7f9251e33740 20 client.17214 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:54.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.935+0000 7f9251e33740 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559e54352470 0x559e54354890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:54.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.935+0000 7f9251e33740 4 client.17214 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:20:54.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.935+0000 7f9251e33740 1 -- 172.21.15.86:0/3883000238 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559e54231540 con 0x559e54352470 2024-04-17T07:20:54.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.936+0000 7f924d000640 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559e54352470 0x559e54354890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:54.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.937+0000 7f924d000640 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559e54352470 0x559e54354890 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b717d77622e50b server_cookie=27d497904d032bc7 in_seq=0 out_seq=0 2024-04-17T07:20:54.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.937+0000 7f9232000640 10 client.17214 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:54.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.940+0000 7f9232000640 1 -- 172.21.15.86:0/3883000238 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559e54231540 con 0x559e54352470 2024-04-17T07:20:54.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.940+0000 7f9232000640 10 client.17214 handle_command_reply: tid=0 2024-04-17T07:20:54.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.940+0000 7f9251e33740 1 client.17214 shutdown 2024-04-17T07:20:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.940+0000 7f9251e33740 1 -- 172.21.15.86:0/3883000238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 msgr2=0x559e542e4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.941+0000 7f9251e33740 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542e4d30 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f9234005890 tx=0x7f9234003df0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.941+0000 7f9251e33740 1 -- 172.21.15.86:0/3883000238 shutdown_connections 2024-04-17T07:20:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.941+0000 7f9251e33740 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559e54352470 0x559e54354890 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.941+0000 7f9251e33740 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e542daa30 0x559e542e5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.941+0000 7f9251e33740 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559e542d58c0 0x559e542e4d30 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.941+0000 7f9251e33740 1 --2- 172.21.15.86:0/3883000238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559e542d4ed0 0x559e542e47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.941+0000 7f9251e33740 1 -- 172.21.15.86:0/3883000238 >> 172.21.15.86:0/3883000238 conn(0x559e542bf430 msgr2=0x559e542be5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:54.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.941+0000 7f9232000640 0 client.17214 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.942+0000 7f9251e33740 1 -- 172.21.15.86:0/3883000238 shutdown_connections 2024-04-17T07:20:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.942+0000 7f9251e33740 1 -- 172.21.15.86:0/3883000238 wait complete. 2024-04-17T07:20:54.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.942+0000 7f9251e33740 20 client.17214 trim_cache size 0 max 16384 2024-04-17T07:20:54.943 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:20:54.943 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:20:54.943 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "ce50af59-7e36-43c9-ad62-af100dbf29fc", 2024-04-17T07:20:54.943 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:20:54.943 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:20:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.943+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f922c0a3c70 msgr2=0x7f922c0a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.943+0000 7f924f000640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f922c0a3c70 0x7f922c0a6130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f92380efb30 tx=0x7f923802bb70 comp rx=0 tx=0).stop 2024-04-17T07:20:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.944+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92481004e0 msgr2=0x7f9248112cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:54.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.944+0000 7f924f000640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92481004e0 0x7f9248112cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9238037040 tx=0x7f92380042e0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.944+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 shutdown_connections 2024-04-17T07:20:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.944+0000 7f924f000640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f922c0a3c70 0x7f922c0a6130 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.944+0000 7f924f000640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248111330 0x7f92481117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.944+0000 7f924f000640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9248107c90 0x7f9248113230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.944+0000 7f924f000640 1 --2- 172.21.15.86:0/1600335418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92481004e0 0x7f9248112cf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.944+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 >> 172.21.15.86:0/1600335418 conn(0x7f92480fb760 msgr2=0x7f9248103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.945+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 shutdown_connections 2024-04-17T07:20:54.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:54.945+0000 7f924f000640 1 -- 172.21.15.86:0/1600335418 wait complete. 2024-04-17T07:20:54.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:20:54.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:20:55.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:55 smithi086 ceph-mon[31733]: pgmap v636: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 761 op/s 2024-04-17T07:20:55.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:55 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:20:55.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:55 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:55.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:55 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:20:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.335+0000 7f140d600640 1 -- 172.21.15.86:0/2463619441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14081028f0 msgr2=0x7f1408104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:55.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.335+0000 7f140d600640 1 --2- 172.21.15.86:0/2463619441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14081028f0 0x7f1408104ce0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009900 tx=0x7f13f802bc40 comp rx=0 tx=0).stop 2024-04-17T07:20:55.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.336+0000 7f140d600640 1 -- 172.21.15.86:0/2463619441 shutdown_connections 2024-04-17T07:20:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.336+0000 7f140d600640 1 --2- 172.21.15.86:0/2463619441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1408107b50 0x7f1408109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.336+0000 7f140d600640 1 --2- 172.21.15.86:0/2463619441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1408105220 0x7f1408107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.336+0000 7f140d600640 1 --2- 172.21.15.86:0/2463619441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14081028f0 0x7f1408104ce0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.336+0000 7f140d600640 1 -- 172.21.15.86:0/2463619441 >> 172.21.15.86:0/2463619441 conn(0x7f14080fc300 msgr2=0x7f14080fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.336+0000 7f140d600640 1 -- 172.21.15.86:0/2463619441 shutdown_connections 2024-04-17T07:20:55.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.337+0000 7f140d600640 1 -- 172.21.15.86:0/2463619441 wait complete. 2024-04-17T07:20:55.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.339+0000 7f140d600640 1 Processor -- start 2024-04-17T07:20:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.340+0000 7f140d600640 1 -- start start 2024-04-17T07:20:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.340+0000 7f140d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14081028f0 0x7f1408196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.340+0000 7f140d600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1408105220 0x7f1408196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.341+0000 7f140d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408107b50 0x7f14081935b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.341+0000 7f140d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f140810d9f0 con 0x7f1408107b50 2024-04-17T07:20:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.341+0000 7f140d600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f140810d870 con 0x7f1408105220 2024-04-17T07:20:55.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.341+0000 7f140d600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f140810db70 con 0x7f14081028f0 2024-04-17T07:20:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.341+0000 7f1407400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1408105220 0x7f1408196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.341+0000 7f140cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408107b50 0x7f14081935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.341+0000 7f1407e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14081028f0 0x7f1408196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.341+0000 7f1407e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14081028f0 0x7f1408196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51422/0 (socket says 172.21.15.86:51422) 2024-04-17T07:20:55.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.341+0000 7f1407e00640 1 -- 172.21.15.86:0/2086370588 learned_addr learned my addr 172.21.15.86:0/2086370588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.342+0000 7f140cc00640 1 -- 172.21.15.86:0/2086370588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14081028f0 msgr2=0x7f1408196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.342+0000 7f140cc00640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14081028f0 0x7f1408196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.342+0000 7f140cc00640 1 -- 172.21.15.86:0/2086370588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1408105220 msgr2=0x7f1408196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.342+0000 7f140cc00640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1408105220 0x7f1408196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.342+0000 7f140cc00640 1 -- 172.21.15.86:0/2086370588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1408193e70 con 0x7f1408107b50 2024-04-17T07:20:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.342+0000 7f1407400640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1408105220 0x7f1408196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.343+0000 7f140cc00640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408107b50 0x7f14081935b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1400009fd0 tx=0x7f140000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:55.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.343+0000 7f1404c00640 1 -- 172.21.15.86:0/2086370588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140000ce20 con 0x7f1408107b50 2024-04-17T07:20:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.343+0000 7f140d600640 1 -- 172.21.15.86:0/2086370588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1408199ec0 con 0x7f1408107b50 2024-04-17T07:20:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.343+0000 7f1404c00640 1 -- 172.21.15.86:0/2086370588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1400005020 con 0x7f1408107b50 2024-04-17T07:20:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.343+0000 7f1404c00640 1 -- 172.21.15.86:0/2086370588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400010690 con 0x7f1408107b50 2024-04-17T07:20:55.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.343+0000 7f140d600640 1 -- 172.21.15.86:0/2086370588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140819a410 con 0x7f1408107b50 2024-04-17T07:20:55.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.350+0000 7f1404c00640 1 -- 172.21.15.86:0/2086370588 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f14000051d0 con 0x7f1408107b50 2024-04-17T07:20:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.351+0000 7f141183b740 1 Processor -- start 2024-04-17T07:20:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.352+0000 7f141183b740 1 -- start start 2024-04-17T07:20:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.352+0000 7f141183b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 0x5615a2502790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.352+0000 7f141183b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a2507860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.352+0000 7f141183b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2507ed0 0x5615a250a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.352+0000 7f141183b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5615a2469120 con 0x5615a2502370 2024-04-17T07:20:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.352+0000 7f141183b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5615a240f340 con 0x5615a2502d60 2024-04-17T07:20:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.352+0000 7f141183b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615a24650e0 con 0x5615a2507ed0 2024-04-17T07:20:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.352+0000 7f1404c00640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13ec0a3c30 0x7f13ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.352+0000 7f1404c00640 1 -- 172.21.15.86:0/2086370588 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f14000d0dc0 con 0x7f1408107b50 2024-04-17T07:20:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.353+0000 7f1407e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a2507860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.353+0000 7f1407e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a2507860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41584/0 (socket says 172.21.15.86:41584) 2024-04-17T07:20:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.353+0000 7f1407e00640 1 -- 172.21.15.86:0/2876639083 learned_addr learned my addr 172.21.15.86:0/2876639083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.354+0000 7f140cc00640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13ec0a3c30 0x7f13ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.354+0000 7f1407400640 1 --2- 172.21.15.86:0/2876639083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 0x5615a2502790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.354+0000 7f1407400640 1 --2- 172.21.15.86:0/2876639083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2507ed0 0x5615a250a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.355+0000 7f1407e00640 1 -- 172.21.15.86:0/2876639083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2507ed0 msgr2=0x5615a250a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.355+0000 7f1407e00640 1 --2- 172.21.15.86:0/2876639083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2507ed0 0x5615a250a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.355+0000 7f1407e00640 1 -- 172.21.15.86:0/2876639083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 msgr2=0x5615a2502790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.355+0000 7f1407e00640 1 --2- 172.21.15.86:0/2876639083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 0x5615a2502790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.355+0000 7f1407e00640 1 -- 172.21.15.86:0/2876639083 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615a22457d0 con 0x5615a2502d60 2024-04-17T07:20:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.355+0000 7f140cc00640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13ec0a3c30 0x7f13ec0a60f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f140009ccb0 tx=0x7f1400005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.355+0000 7f1407e00640 1 --2- 172.21.15.86:0/2876639083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a2507860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f13f8005e90 tx=0x7f13f8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eafd7df5a4d2cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.357+0000 7f13fd200640 1 -- 172.21.15.86:0/2876639083 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8045020 con 0x5615a2502d60 2024-04-17T07:20:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.357+0000 7f13fd200640 1 -- 172.21.15.86:0/2876639083 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f802ca30 con 0x5615a2502d60 2024-04-17T07:20:55.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.357+0000 7f13fd200640 1 -- 172.21.15.86:0/2876639083 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f80046c0 con 0x5615a2502d60 2024-04-17T07:20:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.358+0000 7f141183b740 1 -- 172.21.15.86:0/2876639083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 msgr2=0x5615a2507860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.358+0000 7f141183b740 1 --2- 172.21.15.86:0/2876639083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a2507860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f13f8005e90 tx=0x7f13f8009560 comp rx=0 tx=0).stop 2024-04-17T07:20:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.359+0000 7f141183b740 1 -- 172.21.15.86:0/2876639083 shutdown_connections 2024-04-17T07:20:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.359+0000 7f141183b740 1 --2- 172.21.15.86:0/2876639083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2507ed0 0x5615a250a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.359+0000 7f141183b740 1 --2- 172.21.15.86:0/2876639083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a2507860 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.359+0000 7f141183b740 1 --2- 172.21.15.86:0/2876639083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 0x5615a2502790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.359+0000 7f141183b740 1 -- 172.21.15.86:0/2876639083 >> 172.21.15.86:0/2876639083 conn(0x5615a24fd8e0 msgr2=0x5615a24fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.360+0000 7f141183b740 1 -- 172.21.15.86:0/2876639083 shutdown_connections 2024-04-17T07:20:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.360+0000 7f141183b740 1 -- 172.21.15.86:0/2876639083 wait complete. 2024-04-17T07:20:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.360+0000 7f141183b740 1 -- start start 2024-04-17T07:20:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.361+0000 7f141183b740 1 Processor -- start 2024-04-17T07:20:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.361+0000 7f141183b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 0x5615a250c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.362+0000 7f141183b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a250cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.362+0000 7f141183b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2507ed0 0x5615a2513e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.362+0000 7f141183b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5615a250ae20 con 0x5615a2502370 2024-04-17T07:20:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.362+0000 7f141183b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5615a2422710 con 0x5615a2502d60 2024-04-17T07:20:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.362+0000 7f141183b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615a250afa0 con 0x5615a2507ed0 2024-04-17T07:20:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.362+0000 7f141183b740 10 client.0 _ll_register_callbacks cb 0x5615a241daf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.362+0000 7f1407400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 0x5615a250c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.362+0000 7f1407e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a250cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.362+0000 7f1407e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a250cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:41590/0 (socket says 172.21.15.86:41590) 2024-04-17T07:20:55.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.362+0000 7f1407e00640 1 -- 172.21.15.86:0/1894561536 learned_addr learned my addr 172.21.15.86:0/1894561536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.363+0000 7f1407e00640 1 -- 172.21.15.86:0/1894561536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2507ed0 msgr2=0x5615a2513e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.363+0000 7f1407e00640 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2507ed0 0x5615a2513e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.363+0000 7f1407e00640 1 -- 172.21.15.86:0/1894561536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 msgr2=0x5615a250c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.363+0000 7f1407e00640 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 0x5615a250c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.364+0000 7f1407e00640 1 -- 172.21.15.86:0/1894561536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615a217b3f0 con 0x5615a2502d60 2024-04-17T07:20:55.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.364+0000 7f1407400640 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 0x5615a250c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:20:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.364+0000 7f1407e00640 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a250cdf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13f8000c00 tx=0x7f13f8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0e323dfdd417ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.365+0000 7f13f1600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:20:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.365+0000 7f13f1600640 1 -- 172.21.15.86:0/1894561536 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8051070 con 0x5615a2502d60 2024-04-17T07:20:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.365+0000 7f13f1600640 1 -- 172.21.15.86:0/1894561536 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f13f80048c0 con 0x5615a2502d60 2024-04-17T07:20:55.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.365+0000 7f141183b740 1 -- 172.21.15.86:0/1894561536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f802ca30 con 0x5615a2502d60 2024-04-17T07:20:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.366+0000 7f13f1600640 1 -- 172.21.15.86:0/1894561536 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f13f8004ac0 con 0x5615a2502d60 2024-04-17T07:20:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.366+0000 7f141183b740 10 client.36224 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:55.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.366+0000 7f141183b740 1 -- 172.21.15.86:0/1894561536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615a213b0f0 con 0x5615a2502d60 2024-04-17T07:20:55.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.367+0000 7f13f1600640 1 -- 172.21.15.86:0/1894561536 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f8004c60 con 0x5615a2502d60 2024-04-17T07:20:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.368+0000 7f13f1600640 1 -- 172.21.15.86:0/1894561536 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f13f8054020 con 0x5615a2502d60 2024-04-17T07:20:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.369+0000 7f141183b740 10 client.36224 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.369+0000 7f141183b740 10 client.36224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.369+0000 7f141183b740 20 client.36224 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.369+0000 7f141183b740 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615a2518320 0x5615a251a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.369+0000 7f141183b740 4 client.36224 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:20:55.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.369+0000 7f141183b740 1 -- 172.21.15.86:0/1894561536 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f13f802ca30 con 0x5615a2518320 2024-04-17T07:20:55.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.370+0000 7f1407400640 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615a2518320 0x5615a251a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.374+0000 7f13f1600640 10 client.36224 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.374+0000 7f1407400640 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615a2518320 0x5615a251a710 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e03c7b77c4af192 server_cookie=efc97f486c64ac8e in_seq=0 out_seq=0 2024-04-17T07:20:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.376+0000 7f13f1600640 1 -- 172.21.15.86:0/1894561536 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f13f802ca30 con 0x5615a2518320 2024-04-17T07:20:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.376+0000 7f13f1600640 10 client.36224 handle_command_reply: tid=0 2024-04-17T07:20:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.376+0000 7f141183b740 1 client.36224 shutdown 2024-04-17T07:20:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.377+0000 7f141183b740 1 -- 172.21.15.86:0/1894561536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 msgr2=0x5615a250cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:55.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.377+0000 7f141183b740 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a250cdf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13f8000c00 tx=0x7f13f8005d50 comp rx=0 tx=0).stop 2024-04-17T07:20:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.377+0000 7f141183b740 1 -- 172.21.15.86:0/1894561536 shutdown_connections 2024-04-17T07:20:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.377+0000 7f141183b740 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615a2518320 0x5615a251a710 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.377+0000 7f141183b740 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2507ed0 0x5615a2513e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.377+0000 7f141183b740 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2502d60 0x5615a250cdf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.377+0000 7f13f1600640 0 client.36224 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.377+0000 7f141183b740 1 --2- 172.21.15.86:0/1894561536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2502370 0x5615a250c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.377+0000 7f141183b740 1 -- 172.21.15.86:0/1894561536 >> 172.21.15.86:0/1894561536 conn(0x5615a24fd8e0 msgr2=0x5615a24fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.378+0000 7f141183b740 1 -- 172.21.15.86:0/1894561536 shutdown_connections 2024-04-17T07:20:55.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.378+0000 7f141183b740 1 -- 172.21.15.86:0/1894561536 wait complete. 2024-04-17T07:20:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.378+0000 7f141183b740 20 client.36224 trim_cache size 0 max 16384 2024-04-17T07:20:55.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.420+0000 7f141183b740 1 Processor -- start 2024-04-17T07:20:55.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.421+0000 7f141183b740 1 -- start start 2024-04-17T07:20:55.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.422+0000 7f141183b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a250c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.422+0000 7f141183b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2507ed0 0x5615a2513df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.422+0000 7f141183b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a250ba80 0x5615a250cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.422+0000 7f141183b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14081b9610 con 0x5615a250ba80 2024-04-17T07:20:55.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.422+0000 7f141183b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14081bb150 con 0x5615a2507ed0 2024-04-17T07:20:55.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.422+0000 7f141183b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14081b9ce0 con 0x5615a2502370 2024-04-17T07:20:55.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.422+0000 7f1407400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a250c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.423+0000 7f1407400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a250c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51454/0 (socket says 172.21.15.86:51454) 2024-04-17T07:20:55.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.423+0000 7f1407400640 1 -- 172.21.15.86:0/679434629 learned_addr learned my addr 172.21.15.86:0/679434629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:55.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.423+0000 7f1407400640 1 --2- 172.21.15.86:0/679434629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a250ba80 0x5615a250cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.423+0000 7f1407e00640 1 --2- 172.21.15.86:0/679434629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2507ed0 0x5615a2513df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.424+0000 7f1407400640 1 -- 172.21.15.86:0/679434629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 msgr2=0x5615a250c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:55.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.424+0000 7f1407400640 1 --2- 172.21.15.86:0/679434629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a250c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.424+0000 7f1407400640 1 -- 172.21.15.86:0/679434629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2507ed0 msgr2=0x5615a2513df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:55.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.424+0000 7f1407400640 1 --2- 172.21.15.86:0/679434629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2507ed0 0x5615a2513df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.424+0000 7f1407400640 1 -- 172.21.15.86:0/679434629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615a24fe330 con 0x5615a250ba80 2024-04-17T07:20:55.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.425+0000 7f1407400640 1 --2- 172.21.15.86:0/679434629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a250ba80 0x5615a250cdf0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f13f4005890 tx=0x7f13f4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae13e9668257501 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:55.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.425+0000 7f13f3e00640 1 -- 172.21.15.86:0/679434629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4021860 con 0x5615a250ba80 2024-04-17T07:20:55.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.425+0000 7f13f3e00640 1 -- 172.21.15.86:0/679434629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f401fc70 con 0x5615a250ba80 2024-04-17T07:20:55.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.425+0000 7f13f3e00640 1 -- 172.21.15.86:0/679434629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4005020 con 0x5615a250ba80 2024-04-17T07:20:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.427+0000 7f141183b740 1 -- 172.21.15.86:0/679434629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a250ba80 msgr2=0x5615a250cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.427+0000 7f141183b740 1 --2- 172.21.15.86:0/679434629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a250ba80 0x5615a250cdf0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f13f4005890 tx=0x7f13f4015040 comp rx=0 tx=0).stop 2024-04-17T07:20:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.427+0000 7f141183b740 1 -- 172.21.15.86:0/679434629 shutdown_connections 2024-04-17T07:20:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.427+0000 7f141183b740 1 --2- 172.21.15.86:0/679434629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a250ba80 0x5615a250cdf0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.427+0000 7f141183b740 1 --2- 172.21.15.86:0/679434629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a2507ed0 0x5615a2513df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.427+0000 7f141183b740 1 --2- 172.21.15.86:0/679434629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a250c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.427+0000 7f141183b740 1 -- 172.21.15.86:0/679434629 >> 172.21.15.86:0/679434629 conn(0x5615a24fd8e0 msgr2=0x5615a250b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.429+0000 7f141183b740 1 -- 172.21.15.86:0/679434629 shutdown_connections 2024-04-17T07:20:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.429+0000 7f141183b740 1 -- 172.21.15.86:0/679434629 wait complete. 2024-04-17T07:20:55.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.430+0000 7f141183b740 1 -- start start 2024-04-17T07:20:55.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.430+0000 7f141183b740 1 Processor -- start 2024-04-17T07:20:55.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.434+0000 7f141183b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a2511ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.434+0000 7f141183b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2507ed0 0x5615a2512010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.434+0000 7f141183b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a250ba80 0x5615a2512fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.434+0000 7f141183b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14081baa80 con 0x5615a2507ed0 2024-04-17T07:20:55.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.434+0000 7f141183b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14081ba3b0 con 0x5615a250ba80 2024-04-17T07:20:55.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.434+0000 7f141183b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14081b7700 con 0x5615a2502370 2024-04-17T07:20:55.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.435+0000 7f141183b740 10 client.0 _ll_register_callbacks cb 0x7f13f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:20:55.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.435+0000 7f1407400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a2511ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.435+0000 7f1407e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2507ed0 0x5615a2512010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.435+0000 7f1407400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a2511ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:51464/0 (socket says 172.21.15.86:51464) 2024-04-17T07:20:55.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.435+0000 7f1407400640 1 -- 172.21.15.86:0/2003900066 learned_addr learned my addr 172.21.15.86:0/2003900066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:20:55.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.436+0000 7f1407400640 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a250ba80 0x5615a2512fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.437+0000 7f1407e00640 1 -- 172.21.15.86:0/2003900066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 msgr2=0x5615a2511ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:55.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.437+0000 7f1407e00640 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a2511ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.437+0000 7f1407e00640 1 -- 172.21.15.86:0/2003900066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a250ba80 msgr2=0x5615a2512fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:20:55.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.437+0000 7f1407e00640 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a250ba80 0x5615a2512fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.437+0000 7f1407e00640 1 -- 172.21.15.86:0/2003900066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615a25135c0 con 0x5615a2507ed0 2024-04-17T07:20:55.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.437+0000 7f1407400640 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a2511ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:20:55.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.438+0000 7f1407e00640 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2507ed0 0x5615a2512010 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f13f804b000 tx=0x7f13f80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6361cb19c93d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:20:55.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.438+0000 7f13f1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:20:55.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.439+0000 7f13f1600640 1 -- 172.21.15.86:0/2003900066 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f805e070 con 0x5615a2507ed0 2024-04-17T07:20:55.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.439+0000 7f13f1600640 1 -- 172.21.15.86:0/2003900066 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f13f8005070 con 0x5615a2507ed0 2024-04-17T07:20:55.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.440+0000 7f141183b740 1 -- 172.21.15.86:0/2003900066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f401fc70 con 0x5615a2507ed0 2024-04-17T07:20:55.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.441+0000 7f13f1600640 1 -- 172.21.15.86:0/2003900066 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f13f8067050 con 0x5615a2507ed0 2024-04-17T07:20:55.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.441+0000 7f141183b740 10 client.17232 fetch_fsmap learned FSMap version 28 2024-04-17T07:20:55.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.441+0000 7f141183b740 1 -- 172.21.15.86:0/2003900066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615a257ec90 con 0x5615a2507ed0 2024-04-17T07:20:55.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.444+0000 7f13f1600640 1 -- 172.21.15.86:0/2003900066 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13f401fc70 con 0x5615a2507ed0 2024-04-17T07:20:55.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.444+0000 7f13f1600640 1 -- 172.21.15.86:0/2003900066 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f13f8054040 con 0x5615a2507ed0 2024-04-17T07:20:55.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.444+0000 7f141183b740 10 client.17232 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:20:55.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.444+0000 7f141183b740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:20:55.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.444+0000 7f141183b740 20 client.17232 populate_metadata read hostname 'smithi086' 2024-04-17T07:20:55.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.444+0000 7f141183b740 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615a257f650 0x5615a2581a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:20:55.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.444+0000 7f141183b740 4 client.17232 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:20:55.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.444+0000 7f141183b740 1 -- 172.21.15.86:0/2003900066 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615a2476b40 con 0x5615a257f650 2024-04-17T07:20:55.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.445+0000 7f1407400640 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615a257f650 0x5615a2581a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:20:55.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.446+0000 7f1407400640 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615a257f650 0x5615a2581a70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1909b6db37dd82 server_cookie=594686786533ea81 in_seq=0 out_seq=0 2024-04-17T07:20:55.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.446+0000 7f13f1600640 10 client.17232 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:55.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.447+0000 7f13f1600640 1 -- 172.21.15.86:0/2003900066 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5615a2476b40 con 0x5615a257f650 2024-04-17T07:20:55.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.447+0000 7f13f1600640 10 client.17232 handle_command_reply: tid=0 2024-04-17T07:20:55.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.447+0000 7f141183b740 1 client.17232 shutdown 2024-04-17T07:20:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.448+0000 7f141183b740 1 -- 172.21.15.86:0/2003900066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2507ed0 msgr2=0x5615a2512010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.448+0000 7f141183b740 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2507ed0 0x5615a2512010 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f13f804b000 tx=0x7f13f80043c0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.448+0000 7f141183b740 1 -- 172.21.15.86:0/2003900066 shutdown_connections 2024-04-17T07:20:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.448+0000 7f141183b740 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615a257f650 0x5615a2581a70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.448+0000 7f141183b740 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615a250ba80 0x5615a2512fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.448+0000 7f141183b740 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615a2507ed0 0x5615a2512010 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.448+0000 7f13f1600640 0 client.17232 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:20:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.448+0000 7f141183b740 1 --2- 172.21.15.86:0/2003900066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615a2502370 0x5615a2511ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.448+0000 7f141183b740 1 -- 172.21.15.86:0/2003900066 >> 172.21.15.86:0/2003900066 conn(0x5615a24fd8e0 msgr2=0x5615a250c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.450+0000 7f141183b740 1 -- 172.21.15.86:0/2003900066 shutdown_connections 2024-04-17T07:20:55.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.450+0000 7f141183b740 1 -- 172.21.15.86:0/2003900066 wait complete. 2024-04-17T07:20:55.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.450+0000 7f141183b740 20 client.17232 trim_cache size 0 max 16384 2024-04-17T07:20:55.450 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:20:55.450 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:20:55.450 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:20:55.451 INFO:teuthology.orchestra.run.smithi086.stdout: "ce50af59-7e36-43c9-ad62-af100dbf29fc": { 2024-04-17T07:20:55.451 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:20:55.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "ce50af59-7e36-43c9-ad62-af100dbf29fc", 2024-04-17T07:20:55.451 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:20:55.451 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:20:55.451 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:20:55.451 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:20:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.451+0000 7f140d600640 1 -- 172.21.15.86:0/2086370588 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13ec0a3c30 msgr2=0x7f13ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.451+0000 7f140d600640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13ec0a3c30 0x7f13ec0a60f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f140009ccb0 tx=0x7f1400005f70 comp rx=0 tx=0).stop 2024-04-17T07:20:55.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.451+0000 7f140d600640 1 -- 172.21.15.86:0/2086370588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408107b50 msgr2=0x7f14081935b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:20:55.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.451+0000 7f140d600640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408107b50 0x7f14081935b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1400009fd0 tx=0x7f140000ef10 comp rx=0 tx=0).stop 2024-04-17T07:20:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.453+0000 7f140d600640 1 -- 172.21.15.86:0/2086370588 shutdown_connections 2024-04-17T07:20:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.453+0000 7f140d600640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13ec0a3c30 0x7f13ec0a60f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.453+0000 7f140d600640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1408107b50 0x7f14081935b0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.453+0000 7f140d600640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1408105220 0x7f1408196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.453+0000 7f140d600640 1 --2- 172.21.15.86:0/2086370588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14081028f0 0x7f1408196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:20:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.453+0000 7f140d600640 1 -- 172.21.15.86:0/2086370588 >> 172.21.15.86:0/2086370588 conn(0x7f14080fc300 msgr2=0x7f1408101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:20:55.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.453+0000 7f140d600640 1 -- 172.21.15.86:0/2086370588 shutdown_connections 2024-04-17T07:20:55.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:20:55.453+0000 7f140d600640 1 -- 172.21.15.86:0/2086370588 wait complete. 2024-04-17T07:20:55.481 INFO:tasks.cephfs.filesystem:scrub status for tag:ce50af59-7e36-43c9-ad62-af100dbf29fc - {'path': '/', 'tag': 'ce50af59-7e36-43c9-ad62-af100dbf29fc', 'options': 'recursive,force'} 2024-04-17T07:20:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:55 smithi098 ceph-mon[35418]: pgmap v636: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 761 op/s 2024-04-17T07:20:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:55 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:20:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:55 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:55.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:55 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:20:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:55 smithi103 ceph-mon[35495]: pgmap v636: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 761 op/s 2024-04-17T07:20:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:55 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:20:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:55 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:55 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:20:56.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:56 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:56.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:56 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:56.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:56 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:20:57.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:57 smithi098 ceph-mon[35418]: pgmap v637: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 694 op/s 2024-04-17T07:20:57.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:57 smithi103 ceph-mon[35495]: pgmap v637: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 694 op/s 2024-04-17T07:20:57.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:57 smithi086 ceph-mon[31733]: pgmap v637: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 694 op/s 2024-04-17T07:20:59.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 95.0 s, 553.9 tps, lat 147.983 ms stddev 38.350 2024-04-17T07:20:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:59 smithi098 ceph-mon[35418]: pgmap v638: 129 pgs: 129 active+clean; 964 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 786 op/s 2024-04-17T07:20:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:20:59 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:20:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:59 smithi103 ceph-mon[35495]: pgmap v638: 129 pgs: 129 active+clean; 964 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 786 op/s 2024-04-17T07:20:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:20:59 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:20:59.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:59 smithi086 ceph-mon[31733]: pgmap v638: 129 pgs: 129 active+clean; 964 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 786 op/s 2024-04-17T07:20:59.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:20:59 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:21:01.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:01 smithi098 ceph-mon[35418]: pgmap v639: 129 pgs: 129 active+clean; 964 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 673 op/s 2024-04-17T07:21:01.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:01 smithi103 ceph-mon[35495]: pgmap v639: 129 pgs: 129 active+clean; 964 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 673 op/s 2024-04-17T07:21:01.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:01 smithi086 ceph-mon[31733]: pgmap v639: 129 pgs: 129 active+clean; 964 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 673 op/s 2024-04-17T07:21:03.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:03 smithi098 ceph-mon[35418]: pgmap v640: 129 pgs: 129 active+clean; 976 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.8 MiB/s wr, 813 op/s 2024-04-17T07:21:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:03 smithi103 ceph-mon[35495]: pgmap v640: 129 pgs: 129 active+clean; 976 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.8 MiB/s wr, 813 op/s 2024-04-17T07:21:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:03 smithi086 ceph-mon[31733]: pgmap v640: 129 pgs: 129 active+clean; 976 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.8 MiB/s wr, 813 op/s 2024-04-17T07:21:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 100.0 s, 805.7 tps, lat 146.881 ms stddev 177.978 2024-04-17T07:21:05.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:05 smithi098 ceph-mon[35418]: pgmap v641: 129 pgs: 129 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 756 op/s 2024-04-17T07:21:05.525 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:05 smithi103 ceph-mon[35495]: pgmap v641: 129 pgs: 129 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 756 op/s 2024-04-17T07:21:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:05 smithi086 ceph-mon[31733]: pgmap v641: 129 pgs: 129 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 756 op/s 2024-04-17T07:21:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:07 smithi098 ceph-mon[35418]: pgmap v642: 129 pgs: 129 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 768 op/s 2024-04-17T07:21:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:07 smithi103 ceph-mon[35495]: pgmap v642: 129 pgs: 129 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 768 op/s 2024-04-17T07:21:07.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:07 smithi086 ceph-mon[31733]: pgmap v642: 129 pgs: 129 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 768 op/s 2024-04-17T07:21:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 105.0 s, 731.0 tps, lat 136.785 ms stddev 35.765 2024-04-17T07:21:09.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:09 smithi098 ceph-mon[35418]: pgmap v643: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.3 MiB/s wr, 832 op/s 2024-04-17T07:21:09.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:09 smithi103 ceph-mon[35495]: pgmap v643: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.3 MiB/s wr, 832 op/s 2024-04-17T07:21:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:09 smithi086 ceph-mon[31733]: pgmap v643: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.3 MiB/s wr, 832 op/s 2024-04-17T07:21:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:11 smithi098 ceph-mon[35418]: pgmap v644: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.5 MiB/s wr, 748 op/s 2024-04-17T07:21:11.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:11 smithi103 ceph-mon[35495]: pgmap v644: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.5 MiB/s wr, 748 op/s 2024-04-17T07:21:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:11 smithi086 ceph-mon[31733]: pgmap v644: 129 pgs: 129 active+clean; 981 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.5 MiB/s wr, 748 op/s 2024-04-17T07:21:12.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:12 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:21:12.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:12 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:21:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:12 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:21:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: pgmap v645: 129 pgs: 129 active+clean; 981 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.4 MiB/s wr, 922 op/s 2024-04-17T07:21:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:21:13.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:21:13.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: pgmap v645: 129 pgs: 129 active+clean; 981 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.4 MiB/s wr, 922 op/s 2024-04-17T07:21:13.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:21:13.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:21:13.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:21:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: pgmap v645: 129 pgs: 129 active+clean; 981 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.4 MiB/s wr, 922 op/s 2024-04-17T07:21:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:21:13.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:21:13.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:21:13.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:21:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 110.0 s, 546.6 tps, lat 182.707 ms stddev 242.191 2024-04-17T07:21:14.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:14 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi103 to 176.4M 2024-04-17T07:21:14.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:14 smithi103 ceph-mon[35495]: Unable to set osd_memory_target on smithi103 to 185066291: error parsing value: Value '185066291' is below minimum 939524096 2024-04-17T07:21:14.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:14 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi098 to 176.4M 2024-04-17T07:21:14.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:14 smithi103 ceph-mon[35495]: Unable to set osd_memory_target on smithi098 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:21:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:14 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi103 to 176.4M 2024-04-17T07:21:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:14 smithi086 ceph-mon[31733]: Unable to set osd_memory_target on smithi103 to 185066291: error parsing value: Value '185066291' is below minimum 939524096 2024-04-17T07:21:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:14 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi098 to 176.4M 2024-04-17T07:21:14.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:14 smithi086 ceph-mon[31733]: Unable to set osd_memory_target on smithi098 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:21:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:14 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi103 to 176.4M 2024-04-17T07:21:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:14 smithi098 ceph-mon[35418]: Unable to set osd_memory_target on smithi103 to 185066291: error parsing value: Value '185066291' is below minimum 939524096 2024-04-17T07:21:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:14 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi098 to 176.4M 2024-04-17T07:21:14.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:14 smithi098 ceph-mon[35418]: Unable to set osd_memory_target on smithi098 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:21:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:15 smithi103 ceph-mon[35495]: pgmap v646: 129 pgs: 129 active+clean; 989 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.1 MiB/s wr, 893 op/s 2024-04-17T07:21:15.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:15 smithi086 ceph-mon[31733]: pgmap v646: 129 pgs: 129 active+clean; 989 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.1 MiB/s wr, 893 op/s 2024-04-17T07:21:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:15 smithi098 ceph-mon[35418]: pgmap v646: 129 pgs: 129 active+clean; 989 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.1 MiB/s wr, 893 op/s 2024-04-17T07:21:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:17 smithi103 ceph-mon[35495]: pgmap v647: 129 pgs: 129 active+clean; 993 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.9 MiB/s wr, 821 op/s 2024-04-17T07:21:17.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:17 smithi086 ceph-mon[31733]: pgmap v647: 129 pgs: 129 active+clean; 993 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.9 MiB/s wr, 821 op/s 2024-04-17T07:21:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:17 smithi098 ceph-mon[35418]: pgmap v647: 129 pgs: 129 active+clean; 993 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.9 MiB/s wr, 821 op/s 2024-04-17T07:21:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 115.0 s, 457.4 tps, lat 219.366 ms stddev 353.119 2024-04-17T07:21:19.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:19 smithi086 ceph-mon[31733]: pgmap v648: 129 pgs: 129 active+clean; 1006 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-04-17T07:21:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:19 smithi098 ceph-mon[35418]: pgmap v648: 129 pgs: 129 active+clean; 1006 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-04-17T07:21:19.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:19 smithi103 ceph-mon[35495]: pgmap v648: 129 pgs: 129 active+clean; 1006 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-04-17T07:21:21.555 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:21 smithi098 ceph-mon[35418]: pgmap v649: 129 pgs: 129 active+clean; 1010 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 780 op/s 2024-04-17T07:21:21.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:21 smithi086 ceph-mon[31733]: pgmap v649: 129 pgs: 129 active+clean; 1010 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 780 op/s 2024-04-17T07:21:21.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:21 smithi103 ceph-mon[35495]: pgmap v649: 129 pgs: 129 active+clean; 1010 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 780 op/s 2024-04-17T07:21:23.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:23 smithi086 ceph-mon[31733]: pgmap v650: 129 pgs: 129 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 866 op/s 2024-04-17T07:21:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:23 smithi098 ceph-mon[35418]: pgmap v650: 129 pgs: 129 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 866 op/s 2024-04-17T07:21:23.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:23 smithi103 ceph-mon[35495]: pgmap v650: 129 pgs: 129 active+clean; 1.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 866 op/s 2024-04-17T07:21:24.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 120.0 s, 677.7 tps, lat 116.614 ms stddev 201.707 2024-04-17T07:21:25.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:21:25.482 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:21:25.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:25 smithi086 ceph-mon[31733]: pgmap v651: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 16 MiB/s wr, 782 op/s 2024-04-17T07:21:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:25 smithi098 ceph-mon[35418]: pgmap v651: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 16 MiB/s wr, 782 op/s 2024-04-17T07:21:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.757+0000 7f73a8200640 1 -- 172.21.15.86:0/1268849373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a01041c0 msgr2=0x7f73a01045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.757+0000 7f73a8200640 1 --2- 172.21.15.86:0/1268849373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a01041c0 0x7f73a01045e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7394009900 tx=0x7f739402bc30 comp rx=0 tx=0).stop 2024-04-17T07:21:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.758+0000 7f73a8200640 1 -- 172.21.15.86:0/1268849373 shutdown_connections 2024-04-17T07:21:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.758+0000 7f73a8200640 1 --2- 172.21.15.86:0/1268849373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73a0105d70 0x7f73a010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.758+0000 7f73a8200640 1 --2- 172.21.15.86:0/1268849373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a01053b0 0x7f73a0105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.758+0000 7f73a8200640 1 --2- 172.21.15.86:0/1268849373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a01041c0 0x7f73a01045e0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.758+0000 7f73a8200640 1 -- 172.21.15.86:0/1268849373 >> 172.21.15.86:0/1268849373 conn(0x7f73a00ff750 msgr2=0x7f73a0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.759+0000 7f73a8200640 1 -- 172.21.15.86:0/1268849373 shutdown_connections 2024-04-17T07:21:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.759+0000 7f73a8200640 1 -- 172.21.15.86:0/1268849373 wait complete. 2024-04-17T07:21:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.760+0000 7f73a8200640 1 Processor -- start 2024-04-17T07:21:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.761+0000 7f73a8200640 1 -- start start 2024-04-17T07:21:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.761+0000 7f73a8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a01041c0 0x7f73a0079d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.761+0000 7f73a8200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73a01053b0 0x7f73a007a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.761+0000 7f73a8200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a0105d70 0x7f73a0076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.761+0000 7f73a8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f73a010fd20 con 0x7f73a01041c0 2024-04-17T07:21:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.761+0000 7f73a8200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f73a010fba0 con 0x7f73a01053b0 2024-04-17T07:21:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.761+0000 7f73a8200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73a010fea0 con 0x7f73a0105d70 2024-04-17T07:21:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.762+0000 7f73a5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a01041c0 0x7f73a0079d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.762+0000 7f73a5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a01041c0 0x7f73a0079d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45796/0 (socket says 172.21.15.86:45796) 2024-04-17T07:21:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.762+0000 7f73a5800640 1 -- 172.21.15.86:0/2128096931 learned_addr learned my addr 172.21.15.86:0/2128096931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.762+0000 7f73a4e00640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73a01053b0 0x7f73a007a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.763+0000 7f73a6200640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a0105d70 0x7f73a0076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.764+0000 7f73a5800640 1 -- 172.21.15.86:0/2128096931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a0105d70 msgr2=0x7f73a0076880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.764+0000 7f73a5800640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a0105d70 0x7f73a0076880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.765+0000 7f73a5800640 1 -- 172.21.15.86:0/2128096931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73a01053b0 msgr2=0x7f73a007a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.765+0000 7f73a5800640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73a01053b0 0x7f73a007a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.765+0000 7f73a5800640 1 -- 172.21.15.86:0/2128096931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a0077140 con 0x7f73a01041c0 2024-04-17T07:21:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.765+0000 7f73a6200640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a0105d70 0x7f73a0076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:21:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.765+0000 7f73a4e00640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73a01053b0 0x7f73a007a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:21:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.766+0000 7f73a5800640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a01041c0 0x7f73a0079d10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f739402c840 tx=0x7f7394004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.766+0000 7f739e000640 1 -- 172.21.15.86:0/2128096931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394043020 con 0x7f73a01041c0 2024-04-17T07:21:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.767+0000 7f73a8200640 1 -- 172.21.15.86:0/2128096931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a019a060 con 0x7f73a01041c0 2024-04-17T07:21:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.767+0000 7f73a8200640 1 -- 172.21.15.86:0/2128096931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a019a600 con 0x7f73a01041c0 2024-04-17T07:21:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.767+0000 7f739e000640 1 -- 172.21.15.86:0/2128096931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73940052b0 con 0x7f73a01041c0 2024-04-17T07:21:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.767+0000 7f739e000640 1 -- 172.21.15.86:0/2128096931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394003d90 con 0x7f73a01041c0 2024-04-17T07:21:25.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.772+0000 7f739e000640 1 -- 172.21.15.86:0/2128096931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7394046020 con 0x7f73a01041c0 2024-04-17T07:21:25.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.773+0000 7f73aaf83740 1 Processor -- start 2024-04-17T07:21:25.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.773+0000 7f73aaf83740 1 -- start start 2024-04-17T07:21:25.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.773+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c519190 0x55aa0c5195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.773+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519b80 0x55aa0c51e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.773+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c51ecf0 0x55aa0c521180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.773+0000 7f73aaf83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa0c480120 con 0x55aa0c519190 2024-04-17T07:21:25.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.773+0000 7f73aaf83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55aa0c426340 con 0x55aa0c51ecf0 2024-04-17T07:21:25.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.773+0000 7f73aaf83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aa0c47c0e0 con 0x55aa0c519b80 2024-04-17T07:21:25.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.775+0000 7f73a6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c519190 0x55aa0c5195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.775+0000 7f73a5800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519b80 0x55aa0c51e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.775+0000 7f73a5800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519b80 0x55aa0c51e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:50292/0 (socket says 172.21.15.86:50292) 2024-04-17T07:21:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.775+0000 7f73a5800640 1 -- 172.21.15.86:0/93901979 learned_addr learned my addr 172.21.15.86:0/93901979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:25.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.776+0000 7f739e000640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73880a3d00 0x7f73880a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.776+0000 7f73a5800640 1 -- 172.21.15.86:0/93901979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c51ecf0 msgr2=0x55aa0c521180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:21:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.776+0000 7f73a5800640 1 --2- 172.21.15.86:0/93901979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c51ecf0 0x55aa0c521180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.776+0000 7f73a5800640 1 -- 172.21.15.86:0/93901979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c519190 msgr2=0x55aa0c5195b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.776+0000 7f73a5800640 1 --2- 172.21.15.86:0/93901979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c519190 0x55aa0c5195b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.776+0000 7f73a5800640 1 -- 172.21.15.86:0/93901979 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa0c25c7d0 con 0x55aa0c519b80 2024-04-17T07:21:25.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.776+0000 7f73a6200640 1 --2- 172.21.15.86:0/93901979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c519190 0x55aa0c5195b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:21:25.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:25 smithi103 ceph-mon[35495]: pgmap v651: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 16 MiB/s wr, 782 op/s 2024-04-17T07:21:25.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.776+0000 7f739e000640 1 -- 172.21.15.86:0/2128096931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f73940f2a60 con 0x7f73a01041c0 2024-04-17T07:21:25.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.777+0000 7f73a6200640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73880a3d00 0x7f73880a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.777+0000 7f73a5800640 1 --2- 172.21.15.86:0/93901979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519b80 0x55aa0c51e680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f73940f89e0 tx=0x7f73940f8ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e100d2c27635deb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:25.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.778+0000 7f7392a00640 1 -- 172.21.15.86:0/93901979 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940a7e60 con 0x55aa0c519b80 2024-04-17T07:21:25.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.778+0000 7f7392a00640 1 -- 172.21.15.86:0/93901979 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73940bf060 con 0x55aa0c519b80 2024-04-17T07:21:25.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.778+0000 7f7392a00640 1 -- 172.21.15.86:0/93901979 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940be730 con 0x55aa0c519b80 2024-04-17T07:21:25.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.779+0000 7f73aaf83740 1 -- 172.21.15.86:0/93901979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519b80 msgr2=0x55aa0c51e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:25.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.779+0000 7f73aaf83740 1 --2- 172.21.15.86:0/93901979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519b80 0x55aa0c51e680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f73940f89e0 tx=0x7f73940f8ad0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.779+0000 7f73a6200640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73880a3d00 0x7f73880a61c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f73980095f0 tx=0x7f7398009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:25.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.779+0000 7f73aaf83740 1 -- 172.21.15.86:0/93901979 shutdown_connections 2024-04-17T07:21:25.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.779+0000 7f73aaf83740 1 --2- 172.21.15.86:0/93901979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c51ecf0 0x55aa0c521180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.779+0000 7f73aaf83740 1 --2- 172.21.15.86:0/93901979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519b80 0x55aa0c51e680 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f73940f89e0 tx=0x7f73940f8ad0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.779+0000 7f73aaf83740 1 --2- 172.21.15.86:0/93901979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c519190 0x55aa0c5195b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.779+0000 7f73aaf83740 1 -- 172.21.15.86:0/93901979 >> 172.21.15.86:0/93901979 conn(0x55aa0c514700 msgr2=0x55aa0c514b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:25.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.780+0000 7f73aaf83740 1 -- 172.21.15.86:0/93901979 shutdown_connections 2024-04-17T07:21:25.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.780+0000 7f73aaf83740 1 -- 172.21.15.86:0/93901979 wait complete. 2024-04-17T07:21:25.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.781+0000 7f73aaf83740 1 -- start start 2024-04-17T07:21:25.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.781+0000 7f73aaf83740 1 Processor -- start 2024-04-17T07:21:25.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.782+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519190 0x55aa0c523510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.782+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c51ecf0 0x55aa0c523a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.782+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c52ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.782+0000 7f73aaf83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa0c521a60 con 0x55aa0c528a80 2024-04-17T07:21:25.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.782+0000 7f73aaf83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55aa0c439710 con 0x55aa0c51ecf0 2024-04-17T07:21:25.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.782+0000 7f73aaf83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aa0c521be0 con 0x55aa0c519190 2024-04-17T07:21:25.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.782+0000 7f73aaf83740 10 client.0 _ll_register_callbacks cb 0x55aa0c434af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:25.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.782+0000 7f73a6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c52ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.783+0000 7f73a6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c52ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45806/0 (socket says 172.21.15.86:45806) 2024-04-17T07:21:25.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.783+0000 7f73a6200640 1 -- 172.21.15.86:0/733238801 learned_addr learned my addr 172.21.15.86:0/733238801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:25.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.783+0000 7f73a5800640 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519190 0x55aa0c523510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.783+0000 7f73a4e00640 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c51ecf0 0x55aa0c523a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.784+0000 7f73a6200640 1 -- 172.21.15.86:0/733238801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519190 msgr2=0x55aa0c523510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:25.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.784+0000 7f73a6200640 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519190 0x55aa0c523510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.784+0000 7f73a6200640 1 -- 172.21.15.86:0/733238801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c51ecf0 msgr2=0x55aa0c523a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:25.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.784+0000 7f73a6200640 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c51ecf0 0x55aa0c523a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.784+0000 7f73a6200640 1 -- 172.21.15.86:0/733238801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa0c1923f0 con 0x55aa0c528a80 2024-04-17T07:21:25.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.785+0000 7f73a6200640 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c52ae70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f739801d6d0 tx=0x7f7398004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771963cc6b6ac46 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:25.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.785+0000 7f7387400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:25.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.785+0000 7f7387400640 1 -- 172.21.15.86:0/733238801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739802c060 con 0x55aa0c528a80 2024-04-17T07:21:25.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.785+0000 7f7387400640 1 -- 172.21.15.86:0/733238801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7398010870 con 0x55aa0c528a80 2024-04-17T07:21:25.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.785+0000 7f73aaf83740 1 -- 172.21.15.86:0/733238801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73940bf060 con 0x55aa0c528a80 2024-04-17T07:21:25.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.787+0000 7f7387400640 1 -- 172.21.15.86:0/733238801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7398033050 con 0x55aa0c528a80 2024-04-17T07:21:25.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.787+0000 7f73aaf83740 10 client.17244 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:25.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.787+0000 7f73aaf83740 1 -- 172.21.15.86:0/733238801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa0c1520f0 con 0x55aa0c528a80 2024-04-17T07:21:25.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.788+0000 7f7387400640 1 -- 172.21.15.86:0/733238801 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73940bf060 con 0x55aa0c528a80 2024-04-17T07:21:25.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.788+0000 7f7387400640 1 -- 172.21.15.86:0/733238801 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f739800f040 con 0x55aa0c528a80 2024-04-17T07:21:25.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.789+0000 7f73aaf83740 10 client.17244 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:25.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.789+0000 7f73aaf83740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:21:25.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.789+0000 7f73aaf83740 20 client.17244 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:25.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.789+0000 7f73aaf83740 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55aa0c52f880 0x55aa0c531d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.789+0000 7f73aaf83740 4 client.17244 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:21:25.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.789+0000 7f73aaf83740 1 -- 172.21.15.86:0/733238801 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa0c48db40 con 0x55aa0c52f880 2024-04-17T07:21:25.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.790+0000 7f73a5800640 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55aa0c52f880 0x55aa0c531d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.791+0000 7f73a5800640 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55aa0c52f880 0x55aa0c531d00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=367895d1f3867fe0 server_cookie=a428cc105439e233 in_seq=0 out_seq=0 2024-04-17T07:21:25.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.791+0000 7f7387400640 10 client.17244 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.793+0000 7f7387400640 1 -- 172.21.15.86:0/733238801 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa0c48db40 con 0x55aa0c52f880 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f7387400640 10 client.17244 handle_command_reply: tid=0 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f73aaf83740 1 client.17244 shutdown 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f73aaf83740 1 -- 172.21.15.86:0/733238801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 msgr2=0x55aa0c52ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f73aaf83740 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c52ae70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f739801d6d0 tx=0x7f7398004a40 comp rx=0 tx=0).stop 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f73aaf83740 1 -- 172.21.15.86:0/733238801 shutdown_connections 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f73aaf83740 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55aa0c52f880 0x55aa0c531d00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f73aaf83740 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c52ae70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f73aaf83740 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c51ecf0 0x55aa0c523a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f73aaf83740 1 --2- 172.21.15.86:0/733238801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c519190 0x55aa0c523510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f73aaf83740 1 -- 172.21.15.86:0/733238801 >> 172.21.15.86:0/733238801 conn(0x55aa0c514700 msgr2=0x55aa0c513890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f7387400640 0 client.17244 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.794+0000 7f73aaf83740 1 -- 172.21.15.86:0/733238801 shutdown_connections 2024-04-17T07:21:25.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.795+0000 7f73aaf83740 1 -- 172.21.15.86:0/733238801 wait complete. 2024-04-17T07:21:25.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.795+0000 7f73aaf83740 20 client.17244 trim_cache size 0 max 16384 2024-04-17T07:21:25.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.832+0000 7f73aaf83740 1 Processor -- start 2024-04-17T07:21:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.833+0000 7f73aaf83740 1 -- start start 2024-04-17T07:21:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.834+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c519190 0x55aa0c523510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.834+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 0x55aa0c521f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.834+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c522450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.834+0000 7f73aaf83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f738000fc20 con 0x55aa0c528a80 2024-04-17T07:21:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.834+0000 7f73aaf83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7380011760 con 0x55aa0c519190 2024-04-17T07:21:25.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.834+0000 7f73aaf83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73800102f0 con 0x55aa0c51ecf0 2024-04-17T07:21:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.834+0000 7f73a4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 0x55aa0c521f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.835+0000 7f73a6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c522450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.835+0000 7f73a5800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c519190 0x55aa0c523510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.835+0000 7f73a6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c522450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45820/0 (socket says 172.21.15.86:45820) 2024-04-17T07:21:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.835+0000 7f73a4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 0x55aa0c521f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:50300/0 (socket says 172.21.15.86:50300) 2024-04-17T07:21:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.835+0000 7f73a6200640 1 -- 172.21.15.86:0/3645306779 learned_addr learned my addr 172.21.15.86:0/3645306779 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:25.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.835+0000 7f73a6200640 1 -- 172.21.15.86:0/3645306779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 msgr2=0x55aa0c521f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.835+0000 7f73a6200640 1 --2- 172.21.15.86:0/3645306779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 0x55aa0c521f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.835+0000 7f73a6200640 1 -- 172.21.15.86:0/3645306779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c519190 msgr2=0x55aa0c523510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.835+0000 7f73a6200640 1 --2- 172.21.15.86:0/3645306779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c519190 0x55aa0c523510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.835+0000 7f73a6200640 1 -- 172.21.15.86:0/3645306779 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa0c519f60 con 0x55aa0c528a80 2024-04-17T07:21:25.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.836+0000 7f73a6200640 1 --2- 172.21.15.86:0/3645306779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c522450 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7398017040 tx=0x7f739801d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c899d83c36229399 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.837+0000 7f7392000640 1 -- 172.21.15.86:0/3645306779 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739803c070 con 0x55aa0c528a80 2024-04-17T07:21:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.837+0000 7f7392000640 1 -- 172.21.15.86:0/3645306779 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73980100b0 con 0x55aa0c528a80 2024-04-17T07:21:25.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.837+0000 7f7392000640 1 -- 172.21.15.86:0/3645306779 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980279b0 con 0x55aa0c528a80 2024-04-17T07:21:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.839+0000 7f73aaf83740 1 -- 172.21.15.86:0/3645306779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 msgr2=0x55aa0c522450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.839+0000 7f73aaf83740 1 --2- 172.21.15.86:0/3645306779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c522450 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7398017040 tx=0x7f739801d520 comp rx=0 tx=0).stop 2024-04-17T07:21:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.839+0000 7f73aaf83740 1 -- 172.21.15.86:0/3645306779 shutdown_connections 2024-04-17T07:21:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.839+0000 7f73aaf83740 1 --2- 172.21.15.86:0/3645306779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c522450 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.839+0000 7f73aaf83740 1 --2- 172.21.15.86:0/3645306779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 0x55aa0c521f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.839+0000 7f73aaf83740 1 --2- 172.21.15.86:0/3645306779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c519190 0x55aa0c523510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.839+0000 7f73aaf83740 1 -- 172.21.15.86:0/3645306779 >> 172.21.15.86:0/3645306779 conn(0x55aa0c514700 msgr2=0x55aa0c5181f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.840+0000 7f73aaf83740 1 -- 172.21.15.86:0/3645306779 shutdown_connections 2024-04-17T07:21:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.840+0000 7f73aaf83740 1 -- 172.21.15.86:0/3645306779 wait complete. 2024-04-17T07:21:25.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.840+0000 7f73aaf83740 1 -- start start 2024-04-17T07:21:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.840+0000 7f73aaf83740 1 Processor -- start 2024-04-17T07:21:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.841+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c519190 0x55aa0c540bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.841+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 0x55aa0c5410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.841+0000 7f73aaf83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c542100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.842+0000 7f73aaf83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7380011090 con 0x55aa0c528a80 2024-04-17T07:21:25.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.842+0000 7f73aaf83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f73800109c0 con 0x55aa0c519190 2024-04-17T07:21:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.842+0000 7f73aaf83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f738000dbf0 con 0x55aa0c51ecf0 2024-04-17T07:21:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.842+0000 7f73aaf83740 10 client.0 _ll_register_callbacks cb 0x7f739801d590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.842+0000 7f73a6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c542100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.842+0000 7f73a6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c542100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45830/0 (socket says 172.21.15.86:45830) 2024-04-17T07:21:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.842+0000 7f73a6200640 1 -- 172.21.15.86:0/335756132 learned_addr learned my addr 172.21.15.86:0/335756132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.842+0000 7f73a5800640 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c519190 0x55aa0c540bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.843+0000 7f73a4e00640 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 0x55aa0c5410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.843+0000 7f73a6200640 1 -- 172.21.15.86:0/335756132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 msgr2=0x55aa0c5410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.843+0000 7f73a6200640 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 0x55aa0c5410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.843+0000 7f73a6200640 1 -- 172.21.15.86:0/335756132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c519190 msgr2=0x55aa0c540bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.843+0000 7f73a6200640 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c519190 0x55aa0c540bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.843+0000 7f73a6200640 1 -- 172.21.15.86:0/335756132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa0c5426f0 con 0x55aa0c528a80 2024-04-17T07:21:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.844+0000 7f73a6200640 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c542100 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7398027d70 tx=0x7f7398027920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cab22571c41913 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.844+0000 7f7387400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:25.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.844+0000 7f7387400640 1 -- 172.21.15.86:0/335756132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739803c030 con 0x55aa0c528a80 2024-04-17T07:21:25.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.844+0000 7f7387400640 1 -- 172.21.15.86:0/335756132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7398004c10 con 0x55aa0c528a80 2024-04-17T07:21:25.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.844+0000 7f73aaf83740 1 -- 172.21.15.86:0/335756132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73980100b0 con 0x55aa0c528a80 2024-04-17T07:21:25.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.845+0000 7f7387400640 1 -- 172.21.15.86:0/335756132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7398004e10 con 0x55aa0c528a80 2024-04-17T07:21:25.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.845+0000 7f73aaf83740 10 client.17250 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:25.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.845+0000 7f73aaf83740 1 -- 172.21.15.86:0/335756132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa0c595c90 con 0x55aa0c528a80 2024-04-17T07:21:25.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.846+0000 7f7387400640 1 -- 172.21.15.86:0/335756132 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73980100b0 con 0x55aa0c528a80 2024-04-17T07:21:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.846+0000 7f7387400640 1 -- 172.21.15.86:0/335756132 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7398005040 con 0x55aa0c528a80 2024-04-17T07:21:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.847+0000 7f73aaf83740 10 client.17250 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.847+0000 7f73aaf83740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:21:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.847+0000 7f73aaf83740 20 client.17250 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.847+0000 7f73aaf83740 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55aa0c596560 0x55aa0c598980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.847+0000 7f73aaf83740 4 client.17250 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:21:25.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.847+0000 7f73aaf83740 1 -- 172.21.15.86:0/335756132 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa0c4879b0 con 0x55aa0c596560 2024-04-17T07:21:25.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.848+0000 7f73a5800640 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55aa0c596560 0x55aa0c598980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.849+0000 7f73a5800640 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55aa0c596560 0x55aa0c598980 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=635a74e901fab482 server_cookie=9fbc53360b0760a6 in_seq=0 out_seq=0 2024-04-17T07:21:25.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.849+0000 7f7387400640 10 client.17250 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.850+0000 7f7387400640 1 -- 172.21.15.86:0/335756132 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aa0c4879b0 con 0x55aa0c596560 2024-04-17T07:21:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.850+0000 7f7387400640 10 client.17250 handle_command_reply: tid=0 2024-04-17T07:21:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.850+0000 7f73aaf83740 1 client.17250 shutdown 2024-04-17T07:21:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.850+0000 7f73aaf83740 1 -- 172.21.15.86:0/335756132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 msgr2=0x55aa0c542100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:25.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.850+0000 7f73aaf83740 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c542100 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7398027d70 tx=0x7f7398027920 comp rx=0 tx=0).stop 2024-04-17T07:21:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.851+0000 7f73aaf83740 1 -- 172.21.15.86:0/335756132 shutdown_connections 2024-04-17T07:21:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.851+0000 7f73aaf83740 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55aa0c596560 0x55aa0c598980 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.851+0000 7f73aaf83740 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa0c528a80 0x55aa0c542100 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.851+0000 7f7387400640 0 client.17250 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.851+0000 7f73aaf83740 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa0c51ecf0 0x55aa0c5410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.851+0000 7f73aaf83740 1 --2- 172.21.15.86:0/335756132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa0c519190 0x55aa0c540bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.851+0000 7f73aaf83740 1 -- 172.21.15.86:0/335756132 >> 172.21.15.86:0/335756132 conn(0x55aa0c514700 msgr2=0x55aa0c513000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.852+0000 7f73aaf83740 1 -- 172.21.15.86:0/335756132 shutdown_connections 2024-04-17T07:21:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.852+0000 7f73aaf83740 1 -- 172.21.15.86:0/335756132 wait complete. 2024-04-17T07:21:25.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.852+0000 7f73aaf83740 20 client.17250 trim_cache size 0 max 16384 2024-04-17T07:21:25.852 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:21:25.852 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:21:25.853 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:21:25.853 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:21:25.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.853+0000 7f73a8200640 1 -- 172.21.15.86:0/2128096931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73880a3d00 msgr2=0x7f73880a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:25.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.853+0000 7f73a8200640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73880a3d00 0x7f73880a61c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f73980095f0 tx=0x7f7398009580 comp rx=0 tx=0).stop 2024-04-17T07:21:25.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.853+0000 7f73a8200640 1 -- 172.21.15.86:0/2128096931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a01041c0 msgr2=0x7f73a0079d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:25.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.853+0000 7f73a8200640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a01041c0 0x7f73a0079d10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f739402c840 tx=0x7f7394004270 comp rx=0 tx=0).stop 2024-04-17T07:21:25.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.854+0000 7f73a8200640 1 -- 172.21.15.86:0/2128096931 shutdown_connections 2024-04-17T07:21:25.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.854+0000 7f73a8200640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f73880a3d00 0x7f73880a61c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.854+0000 7f73a8200640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a0105d70 0x7f73a0076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.854+0000 7f73a8200640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73a01053b0 0x7f73a007a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.854+0000 7f73a8200640 1 --2- 172.21.15.86:0/2128096931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73a01041c0 0x7f73a0079d10 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:25.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.854+0000 7f73a8200640 1 -- 172.21.15.86:0/2128096931 >> 172.21.15.86:0/2128096931 conn(0x7f73a00ff750 msgr2=0x7f73a01087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:25.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.856+0000 7f73a8200640 1 -- 172.21.15.86:0/2128096931 shutdown_connections 2024-04-17T07:21:25.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:25.856+0000 7f73a8200640 1 -- 172.21.15.86:0/2128096931 wait complete. 2024-04-17T07:21:25.888 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:21:25.888 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:21:26.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.199+0000 7f1ae1600640 1 -- 172.21.15.86:0/360745036 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0034de0 con 0x7f1ae4105190 2024-04-17T07:21:26.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.199+0000 7f1ae3e00640 1 -- 172.21.15.86:0/360745036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae4105190 msgr2=0x7f1ae4107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:26.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.199+0000 7f1ae3e00640 1 --2- 172.21.15.86:0/360745036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae4105190 0x7f1ae4107580 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009860 tx=0x7f1ad002bb90 comp rx=0 tx=0).stop 2024-04-17T07:21:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.200+0000 7f1ae3e00640 1 -- 172.21.15.86:0/360745036 shutdown_connections 2024-04-17T07:21:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.200+0000 7f1ae3e00640 1 --2- 172.21.15.86:0/360745036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4107ac0 0x7f1ae4109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.200+0000 7f1ae3e00640 1 --2- 172.21.15.86:0/360745036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae4105190 0x7f1ae4107580 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.200+0000 7f1ae3e00640 1 --2- 172.21.15.86:0/360745036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae4102860 0x7f1ae4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.200+0000 7f1ae3e00640 1 -- 172.21.15.86:0/360745036 >> 172.21.15.86:0/360745036 conn(0x7f1ae40fc230 msgr2=0x7f1ae40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.200+0000 7f1ae3e00640 1 -- 172.21.15.86:0/360745036 shutdown_connections 2024-04-17T07:21:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.200+0000 7f1ae3e00640 1 -- 172.21.15.86:0/360745036 wait complete. 2024-04-17T07:21:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.202+0000 7f1ae3e00640 1 Processor -- start 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.202+0000 7f1ae3e00640 1 -- start start 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.202+0000 7f1ae3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4102860 0x7f1ae4195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.202+0000 7f1ae3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae4105190 0x7f1ae4196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.203+0000 7f1ae3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae4107ac0 0x7f1ae41a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.203+0000 7f1ae3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae410d9c0 con 0x7f1ae4105190 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.203+0000 7f1ae3e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae410d840 con 0x7f1ae4102860 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.203+0000 7f1ae3e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae410db40 con 0x7f1ae4107ac0 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.203+0000 7f1ae2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae4105190 0x7f1ae4196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.203+0000 7f1ae2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae4105190 0x7f1ae4196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45844/0 (socket says 172.21.15.86:45844) 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.203+0000 7f1ae2000640 1 -- 172.21.15.86:0/92921852 learned_addr learned my addr 172.21.15.86:0/92921852 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.203+0000 7f1ae2a00640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4102860 0x7f1ae4195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.203+0000 7f1ae3400640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae4107ac0 0x7f1ae41a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.204+0000 7f1ae2000640 1 -- 172.21.15.86:0/92921852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae4107ac0 msgr2=0x7f1ae41a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.204+0000 7f1ae2000640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae4107ac0 0x7f1ae41a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.204+0000 7f1ae2000640 1 -- 172.21.15.86:0/92921852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4102860 msgr2=0x7f1ae4195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.204+0000 7f1ae2000640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4102860 0x7f1ae4195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.204+0000 7f1ae2000640 1 -- 172.21.15.86:0/92921852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae41a6370 con 0x7f1ae4105190 2024-04-17T07:21:26.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.204+0000 7f1ae3400640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae4107ac0 0x7f1ae41a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:21:26.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.204+0000 7f1ae2000640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae4105190 0x7f1ae4196480 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0034910 tx=0x7f1ad00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:26.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.205+0000 7f1adb400640 1 -- 172.21.15.86:0/92921852 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0041020 con 0x7f1ae4105190 2024-04-17T07:21:26.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.205+0000 7f1adb400640 1 -- 172.21.15.86:0/92921852 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ad00035f0 con 0x7f1ae4105190 2024-04-17T07:21:26.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.205+0000 7f1ae3e00640 1 -- 172.21.15.86:0/92921852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae41a6600 con 0x7f1ae4105190 2024-04-17T07:21:26.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.205+0000 7f1ae3e00640 1 -- 172.21.15.86:0/92921852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae41a6b70 con 0x7f1ae4105190 2024-04-17T07:21:26.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.206+0000 7f1adb400640 1 -- 172.21.15.86:0/92921852 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0039040 con 0x7f1ae4105190 2024-04-17T07:21:26.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.208+0000 7f1ae3e00640 1 -- 172.21.15.86:0/92921852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac4005290 con 0x7f1ae4105190 2024-04-17T07:21:26.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.216+0000 7f1adb400640 1 -- 172.21.15.86:0/92921852 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1ad0044020 con 0x7f1ae4105190 2024-04-17T07:21:26.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.219+0000 7f1adb400640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ac80a3d00 0x7f1ac80a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:26.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.219+0000 7f1adb400640 1 -- 172.21.15.86:0/92921852 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1ad00f2180 con 0x7f1ae4105190 2024-04-17T07:21:26.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.220+0000 7f1ae2a00640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ac80a3d00 0x7f1ac80a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:26.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.222+0000 7f1ae2a00640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ac80a3d00 0x7f1ac80a61c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009890 tx=0x7f1ad4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:26.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.228+0000 7f1adb400640 1 -- 172.21.15.86:0/92921852 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1ad004b020 con 0x7f1ae4105190 2024-04-17T07:21:26.302 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:26 smithi086 ceph-mon[31733]: pgmap v652: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 678 op/s 2024-04-17T07:21:26.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.656+0000 7f1ae3e00640 1 -- 172.21.15.86:0/92921852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ac4005580 con 0x7f1ae4105190 2024-04-17T07:21:26.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.660+0000 7f1adb400640 1 -- 172.21.15.86:0/92921852 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f1ad00be780 con 0x7f1ae4105190 2024-04-17T07:21:26.662 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:21:26.662 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:21:26.663 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:21:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.671+0000 7f1ae3e00640 1 -- 172.21.15.86:0/92921852 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ac80a3d00 msgr2=0x7f1ac80a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.671+0000 7f1ae3e00640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ac80a3d00 0x7f1ac80a61c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009890 tx=0x7f1ad4008040 comp rx=0 tx=0).stop 2024-04-17T07:21:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.671+0000 7f1ae3e00640 1 -- 172.21.15.86:0/92921852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae4105190 msgr2=0x7f1ae4196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.671+0000 7f1ae3e00640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae4105190 0x7f1ae4196480 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0034910 tx=0x7f1ad00042f0 comp rx=0 tx=0).stop 2024-04-17T07:21:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.671+0000 7f1ae3e00640 1 -- 172.21.15.86:0/92921852 shutdown_connections 2024-04-17T07:21:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.671+0000 7f1ae3e00640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ac80a3d00 0x7f1ac80a61c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.671+0000 7f1ae3e00640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ae4107ac0 0x7f1ae41a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.671+0000 7f1ae3e00640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ae4105190 0x7f1ae4196480 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.671+0000 7f1ae3e00640 1 --2- 172.21.15.86:0/92921852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae4102860 0x7f1ae4195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.671+0000 7f1ae3e00640 1 -- 172.21.15.86:0/92921852 >> 172.21.15.86:0/92921852 conn(0x7f1ae40fc230 msgr2=0x7f1ae4102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.672+0000 7f1ae3e00640 1 -- 172.21.15.86:0/92921852 shutdown_connections 2024-04-17T07:21:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:26.672+0000 7f1ae3e00640 1 -- 172.21.15.86:0/92921852 wait complete. 2024-04-17T07:21:26.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:21:26.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:26 smithi098 ceph-mon[35418]: pgmap v652: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 678 op/s 2024-04-17T07:21:26.775 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:26 smithi103 ceph-mon[35495]: pgmap v652: 129 pgs: 129 active+clean; 1.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 15 MiB/s wr, 678 op/s 2024-04-17T07:21:27.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.006+0000 7fadb2a00640 1 -- 172.21.15.86:0/4215302305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada40945f0 msgr2=0x7fada4094a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.006+0000 7fadb2a00640 1 --2- 172.21.15.86:0/4215302305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada40945f0 0x7fada4094a10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fada0009900 tx=0x7fada002bc80 comp rx=0 tx=0).stop 2024-04-17T07:21:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.008+0000 7fadb2a00640 1 -- 172.21.15.86:0/4215302305 shutdown_connections 2024-04-17T07:21:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.008+0000 7fadb2a00640 1 --2- 172.21.15.86:0/4215302305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fada40958f0 0x7fada4095d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.008+0000 7fadb2a00640 1 --2- 172.21.15.86:0/4215302305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada4094f50 0x7fada40953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.008+0000 7fadb2a00640 1 --2- 172.21.15.86:0/4215302305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada40945f0 0x7fada4094a10 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fada0009900 tx=0x7fada002bc80 comp rx=0 tx=0).stop 2024-04-17T07:21:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.008+0000 7fadb2a00640 1 -- 172.21.15.86:0/4215302305 >> 172.21.15.86:0/4215302305 conn(0x7fada40901d0 msgr2=0x7fada4092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.008+0000 7fadb2a00640 1 -- 172.21.15.86:0/4215302305 shutdown_connections 2024-04-17T07:21:27.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.008+0000 7fadb2a00640 1 -- 172.21.15.86:0/4215302305 wait complete. 2024-04-17T07:21:27.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.009+0000 7fadb2a00640 1 Processor -- start 2024-04-17T07:21:27.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.010+0000 7fadb2a00640 1 -- start start 2024-04-17T07:21:27.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.010+0000 7fadb2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fada4094f50 0x7fada412e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.010+0000 7fadb2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada40958f0 0x7fada412e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.010+0000 7fadb2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada4133900 0x7fada4135cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.010+0000 7fadb2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fada40a3ad0 con 0x7fada40958f0 2024-04-17T07:21:27.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.010+0000 7fadb2a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fada40a3950 con 0x7fada4133900 2024-04-17T07:21:27.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.010+0000 7fadb2a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fada40a3c50 con 0x7fada4094f50 2024-04-17T07:21:27.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.011+0000 7fadb1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fada4094f50 0x7fada412e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.011+0000 7fadb2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada4133900 0x7fada4135cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.011+0000 7fadb0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada40958f0 0x7fada412e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.011+0000 7fadb2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada4133900 0x7fada4135cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36894/0 (socket says 172.21.15.86:36894) 2024-04-17T07:21:27.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.011+0000 7fadb2000640 1 -- 172.21.15.86:0/15746815 learned_addr learned my addr 172.21.15.86:0/15746815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.012+0000 7fadb0c00640 1 -- 172.21.15.86:0/15746815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fada4094f50 msgr2=0x7fada412e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.012+0000 7fadb0c00640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fada4094f50 0x7fada412e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.012+0000 7fadb0c00640 1 -- 172.21.15.86:0/15746815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada4133900 msgr2=0x7fada4135cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.012+0000 7fadb0c00640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada4133900 0x7fada4135cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.012+0000 7fadb0c00640 1 -- 172.21.15.86:0/15746815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada4136360 con 0x7fada40958f0 2024-04-17T07:21:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.012+0000 7fadb2000640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada4133900 0x7fada4135cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:21:27.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.013+0000 7fadb0c00640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada40958f0 0x7fada412e8a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fad9800d6d0 tx=0x7fad9800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.014+0000 7fadaa000640 1 -- 172.21.15.86:0/15746815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98017070 con 0x7fada40958f0 2024-04-17T07:21:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.014+0000 7fadb2a00640 1 -- 172.21.15.86:0/15746815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada4136650 con 0x7fada40958f0 2024-04-17T07:21:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.014+0000 7fadb2a00640 1 -- 172.21.15.86:0/15746815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada4136bf0 con 0x7fada40958f0 2024-04-17T07:21:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.014+0000 7fadaa000640 1 -- 172.21.15.86:0/15746815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad98009820 con 0x7fada40958f0 2024-04-17T07:21:27.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.014+0000 7fadaa000640 1 -- 172.21.15.86:0/15746815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98004ba0 con 0x7fada40958f0 2024-04-17T07:21:27.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.020+0000 7fadb6de8740 1 Processor -- start 2024-04-17T07:21:27.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.021+0000 7fadaa000640 1 -- 172.21.15.86:0/15746815 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fad980099d0 con 0x7fada40958f0 2024-04-17T07:21:27.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.021+0000 7fadb6de8740 1 -- start start 2024-04-17T07:21:27.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.024+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ec3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.024+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ec8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.024+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec94d0 0x557f97ecb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.024+0000 7fadb6de8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f97e21020 con 0x557f97ec4360 2024-04-17T07:21:27.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.024+0000 7fadb6de8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557f97dc7460 con 0x557f97ec3970 2024-04-17T07:21:27.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.024+0000 7fadb6de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557f97e1cfe0 con 0x557f97ec94d0 2024-04-17T07:21:27.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.024+0000 7fadb2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ec3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.024+0000 7fadb2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ec3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36906/0 (socket says 172.21.15.86:36906) 2024-04-17T07:21:27.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.024+0000 7fadb2000640 1 -- 172.21.15.86:0/2678430674 learned_addr learned my addr 172.21.15.86:0/2678430674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.025+0000 7fadb1600640 1 --2- 172.21.15.86:0/2678430674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ec8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.025+0000 7fadb2000640 1 -- 172.21.15.86:0/2678430674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec94d0 msgr2=0x557f97ecb960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:21:27.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.025+0000 7fadb2000640 1 --2- 172.21.15.86:0/2678430674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec94d0 0x557f97ecb960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.025+0000 7fadb2000640 1 -- 172.21.15.86:0/2678430674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 msgr2=0x557f97ec8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.025+0000 7fadaa000640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fad940a3d00 0x7fad940a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.025+0000 7fadaa000640 1 -- 172.21.15.86:0/15746815 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fad980d00a0 con 0x7fada40958f0 2024-04-17T07:21:27.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.025+0000 7fadb2000640 1 --2- 172.21.15.86:0/2678430674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ec8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.025+0000 7fadb2000640 1 -- 172.21.15.86:0/2678430674 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f97dcf040 con 0x557f97ec3970 2024-04-17T07:21:27.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.025+0000 7fadb1600640 1 --2- 172.21.15.86:0/2678430674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ec8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:21:27.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.026+0000 7fadb2000640 1 --2- 172.21.15.86:0/2678430674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ec3d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fadac067850 tx=0x7fadac06b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e401c51b506386d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.026+0000 7fadb2000640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fad940a3d00 0x7fad940a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.027+0000 7fad9e600640 1 -- 172.21.15.86:0/2678430674 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac086850 con 0x557f97ec3970 2024-04-17T07:21:27.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.027+0000 7fad9e600640 1 -- 172.21.15.86:0/2678430674 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadac08c040 con 0x557f97ec3970 2024-04-17T07:21:27.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.027+0000 7fad9e600640 1 -- 172.21.15.86:0/2678430674 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac079420 con 0x557f97ec3970 2024-04-17T07:21:27.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.029+0000 7fadb6de8740 1 -- 172.21.15.86:0/2678430674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 msgr2=0x557f97ec3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.029+0000 7fadb6de8740 1 --2- 172.21.15.86:0/2678430674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ec3d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fadac067850 tx=0x7fadac06b860 comp rx=0 tx=0).stop 2024-04-17T07:21:27.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.029+0000 7fadb2000640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fad940a3d00 0x7fad940a61c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fadac050430 tx=0x7fadac079760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.029+0000 7fadb6de8740 1 -- 172.21.15.86:0/2678430674 shutdown_connections 2024-04-17T07:21:27.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.029+0000 7fadb6de8740 1 --2- 172.21.15.86:0/2678430674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec94d0 0x557f97ecb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.029+0000 7fadb6de8740 1 --2- 172.21.15.86:0/2678430674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ec8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.029+0000 7fadb6de8740 1 --2- 172.21.15.86:0/2678430674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ec3d90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.029+0000 7fadb6de8740 1 -- 172.21.15.86:0/2678430674 >> 172.21.15.86:0/2678430674 conn(0x557f97ebeee0 msgr2=0x557f97ebf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.030+0000 7fadb6de8740 1 -- 172.21.15.86:0/2678430674 shutdown_connections 2024-04-17T07:21:27.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.030+0000 7fadb6de8740 1 -- 172.21.15.86:0/2678430674 wait complete. 2024-04-17T07:21:27.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.032+0000 7fadb6de8740 1 -- start start 2024-04-17T07:21:27.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.034+0000 7fadb6de8740 1 Processor -- start 2024-04-17T07:21:27.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.034+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ecde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.034+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ece340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.034+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec94d0 0x557f97ed5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.034+0000 7fadb6de8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557f97ecc480 con 0x557f97ec4360 2024-04-17T07:21:27.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.035+0000 7fadb6de8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557f97dda830 con 0x557f97ec3970 2024-04-17T07:21:27.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.035+0000 7fadb6de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557f97ecc600 con 0x557f97ec94d0 2024-04-17T07:21:27.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.035+0000 7fadb6de8740 10 client.0 _ll_register_callbacks cb 0x557f97da8fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:27.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.035+0000 7fadb1600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ecde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.035+0000 7fadb1600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ecde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36922/0 (socket says 172.21.15.86:36922) 2024-04-17T07:21:27.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.035+0000 7fadb1600640 1 -- 172.21.15.86:0/2092300965 learned_addr learned my addr 172.21.15.86:0/2092300965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.035+0000 7fadb2000640 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec94d0 0x557f97ed5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.036+0000 7fadb0c00640 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ece340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.036+0000 7fadb1600640 1 -- 172.21.15.86:0/2092300965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec94d0 msgr2=0x557f97ed5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.036+0000 7fadb1600640 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec94d0 0x557f97ed5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.036+0000 7fadb1600640 1 -- 172.21.15.86:0/2092300965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 msgr2=0x557f97ece340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.036+0000 7fadb1600640 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ece340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.036+0000 7fadb1600640 1 -- 172.21.15.86:0/2092300965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f97dde920 con 0x557f97ec3970 2024-04-17T07:21:27.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.036+0000 7fadb0c00640 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ece340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:21:27.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.036+0000 7fadb1600640 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ecde00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fada002c290 tx=0x7fada00041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777bb1a6d5aee79 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.037+0000 7fad92a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:21:27.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.037+0000 7fad92a00640 1 -- 172.21.15.86:0/2092300965 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0043070 con 0x557f97ec3970 2024-04-17T07:21:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.037+0000 7fad92a00640 1 -- 172.21.15.86:0/2092300965 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fada002c620 con 0x557f97ec3970 2024-04-17T07:21:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.037+0000 7fadb6de8740 1 -- 172.21.15.86:0/2092300965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadac08c040 con 0x557f97ec3970 2024-04-17T07:21:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.038+0000 7fad92a00640 1 -- 172.21.15.86:0/2092300965 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fada004f050 con 0x557f97ec3970 2024-04-17T07:21:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.038+0000 7fadb6de8740 10 client.36263 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.038+0000 7fadb6de8740 1 -- 172.21.15.86:0/2092300965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f97dac7f0 con 0x557f97ec3970 2024-04-17T07:21:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.038+0000 7fad92a00640 1 -- 172.21.15.86:0/2092300965 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadac08c040 con 0x557f97ec3970 2024-04-17T07:21:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.038+0000 7fad92a00640 1 -- 172.21.15.86:0/2092300965 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fada0003a70 con 0x557f97ec3970 2024-04-17T07:21:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.039+0000 7fadb6de8740 10 client.36263 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.039+0000 7fadb6de8740 10 client.36263 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:21:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.039+0000 7fadb6de8740 20 client.36263 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.039+0000 7fadb6de8740 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557f97ed97a0 0x557f97edbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.039+0000 7fadb6de8740 4 client.36263 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:21:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.039+0000 7fadb6de8740 1 -- 172.21.15.86:0/2092300965 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f97e2ea40 con 0x557f97ed97a0 2024-04-17T07:21:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.039+0000 7fadb0c00640 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557f97ed97a0 0x557f97edbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.041+0000 7fadb0c00640 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557f97ed97a0 0x557f97edbc40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ecc7f140c249aac server_cookie=c84dfc7b4ec14a81 in_seq=0 out_seq=0 2024-04-17T07:21:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.041+0000 7fad92a00640 10 client.36263 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:21:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.042+0000 7fad92a00640 1 -- 172.21.15.86:0/2092300965 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f97e2ea40 con 0x557f97ed97a0 2024-04-17T07:21:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.042+0000 7fad92a00640 10 client.36263 handle_command_reply: tid=0 2024-04-17T07:21:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.042+0000 7fadb6de8740 1 client.36263 shutdown 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.043+0000 7fadb6de8740 1 -- 172.21.15.86:0/2092300965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 msgr2=0x557f97ecde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.043+0000 7fadb6de8740 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ecde00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fada002c290 tx=0x7fada00041a0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.043+0000 7fadb6de8740 1 -- 172.21.15.86:0/2092300965 shutdown_connections 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.043+0000 7fadb6de8740 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557f97ed97a0 0x557f97edbc40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.043+0000 7fadb6de8740 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec94d0 0x557f97ed5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.043+0000 7fadb6de8740 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ece340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.043+0000 7fadb6de8740 1 --2- 172.21.15.86:0/2092300965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec3970 0x557f97ecde00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.043+0000 7fadb6de8740 1 -- 172.21.15.86:0/2092300965 >> 172.21.15.86:0/2092300965 conn(0x557f97ebeee0 msgr2=0x557f97eb5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.044+0000 7fad92a00640 0 client.36263 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.044+0000 7fadb6de8740 1 -- 172.21.15.86:0/2092300965 shutdown_connections 2024-04-17T07:21:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.044+0000 7fadb6de8740 1 -- 172.21.15.86:0/2092300965 wait complete. 2024-04-17T07:21:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.044+0000 7fadb6de8740 20 client.36263 trim_cache size 0 max 16384 2024-04-17T07:21:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.068+0000 7fadb6de8740 1 Processor -- start 2024-04-17T07:21:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.068+0000 7fadb6de8740 1 -- start start 2024-04-17T07:21:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.069+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ec2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.069+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ed5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.070+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 0x557f97ecd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.070+0000 7fadb6de8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fada413cc90 con 0x557f97ec4360 2024-04-17T07:21:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.070+0000 7fadb1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ec2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.070+0000 7fadb6de8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fada413e7d0 con 0x557f97ec94d0 2024-04-17T07:21:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.070+0000 7fadb6de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fada413d360 con 0x557f97ec3970 2024-04-17T07:21:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.070+0000 7fadb1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ec2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52316/0 (socket says 172.21.15.86:52316) 2024-04-17T07:21:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.070+0000 7fadb1600640 1 -- 172.21.15.86:0/3121848756 learned_addr learned my addr 172.21.15.86:0/3121848756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.070+0000 7fadb0c00640 1 --2- 172.21.15.86:0/3121848756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ed5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.070+0000 7fadb2000640 1 --2- 172.21.15.86:0/3121848756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 0x557f97ecd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.071+0000 7fadb1600640 1 -- 172.21.15.86:0/3121848756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 msgr2=0x557f97ecd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.071+0000 7fadb1600640 1 --2- 172.21.15.86:0/3121848756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 0x557f97ecd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.071+0000 7fadb1600640 1 -- 172.21.15.86:0/3121848756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 msgr2=0x557f97ed5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.071+0000 7fadb1600640 1 --2- 172.21.15.86:0/3121848756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ed5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.071+0000 7fadb1600640 1 -- 172.21.15.86:0/3121848756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f97b333f0 con 0x557f97ec3970 2024-04-17T07:21:27.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.072+0000 7fadb1600640 1 --2- 172.21.15.86:0/3121848756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ec2850 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fada0001eb0 tx=0x7fada0004290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a323c260f4d08792 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.073+0000 7fad9dc00640 1 -- 172.21.15.86:0/3121848756 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0043040 con 0x557f97ec3970 2024-04-17T07:21:27.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.073+0000 7fad9dc00640 1 -- 172.21.15.86:0/3121848756 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fada004c030 con 0x557f97ec3970 2024-04-17T07:21:27.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.073+0000 7fad9dc00640 1 -- 172.21.15.86:0/3121848756 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0003a90 con 0x557f97ec3970 2024-04-17T07:21:27.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.074+0000 7fadb6de8740 1 -- 172.21.15.86:0/3121848756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 msgr2=0x557f97ec2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.074+0000 7fadb6de8740 1 --2- 172.21.15.86:0/3121848756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ec2850 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fada0001eb0 tx=0x7fada0004290 comp rx=0 tx=0).stop 2024-04-17T07:21:27.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.074+0000 7fadb6de8740 1 -- 172.21.15.86:0/3121848756 shutdown_connections 2024-04-17T07:21:27.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.074+0000 7fadb6de8740 1 --2- 172.21.15.86:0/3121848756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 0x557f97ecd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.074+0000 7fadb6de8740 1 --2- 172.21.15.86:0/3121848756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ed5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.075+0000 7fadb6de8740 1 --2- 172.21.15.86:0/3121848756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ec2850 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.075+0000 7fadb6de8740 1 -- 172.21.15.86:0/3121848756 >> 172.21.15.86:0/3121848756 conn(0x557f97ebeee0 msgr2=0x557f97ec2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.075+0000 7fadb6de8740 1 -- 172.21.15.86:0/3121848756 shutdown_connections 2024-04-17T07:21:27.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.075+0000 7fadb6de8740 1 -- 172.21.15.86:0/3121848756 wait complete. 2024-04-17T07:21:27.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.076+0000 7fadb6de8740 1 -- start start 2024-04-17T07:21:27.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.076+0000 7fadb6de8740 1 Processor -- start 2024-04-17T07:21:27.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.077+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ed21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.077+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ed2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.077+0000 7fadb6de8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 0x557f97f3dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.077+0000 7fadb6de8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fada413e100 con 0x557f97ec4360 2024-04-17T07:21:27.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.077+0000 7fadb6de8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fada413da30 con 0x557f97ec94d0 2024-04-17T07:21:27.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.077+0000 7fadb6de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fada413aca0 con 0x557f97ec3970 2024-04-17T07:21:27.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.077+0000 7fadb6de8740 10 client.0 _ll_register_callbacks cb 0x7fada0009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:27.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.078+0000 7fadb2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 0x557f97f3dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.078+0000 7fadb1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ed21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.078+0000 7fadb1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ed21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52322/0 (socket says 172.21.15.86:52322) 2024-04-17T07:21:27.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.078+0000 7fadb1600640 1 -- 172.21.15.86:0/3361865146 learned_addr learned my addr 172.21.15.86:0/3361865146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.078+0000 7fadb0c00640 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ed2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.078+0000 7fadb2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 0x557f97f3dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36946/0 (socket says 172.21.15.86:36946) 2024-04-17T07:21:27.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.079+0000 7fadb0c00640 1 -- 172.21.15.86:0/3361865146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 msgr2=0x557f97ed21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.079+0000 7fadb0c00640 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ed21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.079+0000 7fadb0c00640 1 -- 172.21.15.86:0/3361865146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 msgr2=0x557f97f3dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.079+0000 7fadb0c00640 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 0x557f97f3dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.079+0000 7fadb0c00640 1 -- 172.21.15.86:0/3361865146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f97bfd7d0 con 0x557f97ec4360 2024-04-17T07:21:27.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.080+0000 7fadb0c00640 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ed2710 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fad980cd7c0 tx=0x7fad9809be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1378c6ec6eb6657 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.081+0000 7fad92a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:27.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.081+0000 7fad92a00640 1 -- 172.21.15.86:0/3361865146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad980ba060 con 0x557f97ec4360 2024-04-17T07:21:27.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.081+0000 7fad92a00640 1 -- 172.21.15.86:0/3361865146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fad98004d40 con 0x557f97ec4360 2024-04-17T07:21:27.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.081+0000 7fadb6de8740 1 -- 172.21.15.86:0/3361865146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada004c030 con 0x557f97ec4360 2024-04-17T07:21:27.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.082+0000 7fad92a00640 1 -- 172.21.15.86:0/3361865146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad9807f050 con 0x557f97ec4360 2024-04-17T07:21:27.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.082+0000 7fadb6de8740 10 client.17268 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:27.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.082+0000 7fadb6de8740 1 -- 172.21.15.86:0/3361865146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f97f3f370 con 0x557f97ec4360 2024-04-17T07:21:27.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.083+0000 7fad92a00640 1 -- 172.21.15.86:0/3361865146 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fada004c030 con 0x557f97ec4360 2024-04-17T07:21:27.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.083+0000 7fad92a00640 1 -- 172.21.15.86:0/3361865146 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fad98096b10 con 0x557f97ec4360 2024-04-17T07:21:27.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.083+0000 7fadb6de8740 10 client.17268 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:27.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.083+0000 7fadb6de8740 10 client.17268 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:21:27.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.083+0000 7fadb6de8740 20 client.17268 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:27.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.084+0000 7fadb6de8740 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557f97f3fca0 0x557f97f420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.084+0000 7fadb6de8740 4 client.17268 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:21:27.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.084+0000 7fadb6de8740 1 -- 172.21.15.86:0/3361865146 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f97e288b0 con 0x557f97f3fca0 2024-04-17T07:21:27.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.084+0000 7fadb1600640 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557f97f3fca0 0x557f97f420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.085+0000 7fadb1600640 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557f97f3fca0 0x557f97f420c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25906ff24509dbc8 server_cookie=c69af3d633945a11 in_seq=0 out_seq=0 2024-04-17T07:21:27.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.085+0000 7fad92a00640 10 client.17268 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:21:27.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fad92a00640 1 -- 172.21.15.86:0/3361865146 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f97e288b0 con 0x557f97f3fca0 2024-04-17T07:21:27.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fad92a00640 10 client.17268 handle_command_reply: tid=0 2024-04-17T07:21:27.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fadb6de8740 1 client.17268 shutdown 2024-04-17T07:21:27.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fadb6de8740 1 -- 172.21.15.86:0/3361865146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 msgr2=0x557f97ed2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fadb6de8740 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ed2710 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fad980cd7c0 tx=0x7fad9809be40 comp rx=0 tx=0).stop 2024-04-17T07:21:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fadb6de8740 1 -- 172.21.15.86:0/3361865146 shutdown_connections 2024-04-17T07:21:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fadb6de8740 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557f97f3fca0 0x557f97f420c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fadb6de8740 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f97ec94d0 0x557f97f3dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fadb6de8740 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557f97ec4360 0x557f97ed2710 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fadb6de8740 1 --2- 172.21.15.86:0/3361865146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557f97ec3970 0x557f97ed21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.086+0000 7fadb6de8740 1 -- 172.21.15.86:0/3361865146 >> 172.21.15.86:0/3361865146 conn(0x557f97ebeee0 msgr2=0x557f97ec29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.087+0000 7fad92a00640 0 client.17268 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:21:27.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.087+0000 7fadb6de8740 1 -- 172.21.15.86:0/3361865146 shutdown_connections 2024-04-17T07:21:27.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.087+0000 7fadb6de8740 1 -- 172.21.15.86:0/3361865146 wait complete. 2024-04-17T07:21:27.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.088+0000 7fadb6de8740 20 client.17268 trim_cache size 0 max 16384 2024-04-17T07:21:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:21:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.089+0000 7fadb2a00640 1 -- 172.21.15.86:0/15746815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fad940a3d00 msgr2=0x7fad940a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.089+0000 7fadb2a00640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fad940a3d00 0x7fad940a61c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fadac050430 tx=0x7fadac079760 comp rx=0 tx=0).stop 2024-04-17T07:21:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.089+0000 7fadb2a00640 1 -- 172.21.15.86:0/15746815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada40958f0 msgr2=0x7fada412e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.089+0000 7fadb2a00640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada40958f0 0x7fada412e8a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fad9800d6d0 tx=0x7fad9800db90 comp rx=0 tx=0).stop 2024-04-17T07:21:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.089+0000 7fadb2a00640 1 -- 172.21.15.86:0/15746815 shutdown_connections 2024-04-17T07:21:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.089+0000 7fadb2a00640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fad940a3d00 0x7fad940a61c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.089+0000 7fadb2a00640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada4133900 0x7fada4135cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.089+0000 7fadb2a00640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fada40958f0 0x7fada412e8a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.089+0000 7fadb2a00640 1 --2- 172.21.15.86:0/15746815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fada4094f50 0x7fada412e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.089+0000 7fadb2a00640 1 -- 172.21.15.86:0/15746815 >> 172.21.15.86:0/15746815 conn(0x7fada40901d0 msgr2=0x7fada4091cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.090+0000 7fadb2a00640 1 -- 172.21.15.86:0/15746815 shutdown_connections 2024-04-17T07:21:27.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.090+0000 7fadb2a00640 1 -- 172.21.15.86:0/15746815 wait complete. 2024-04-17T07:21:27.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:21:27.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.386+0000 7f9730a00640 1 -- 172.21.15.86:0/1514590166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972c1052c0 msgr2=0x7f972c1076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.386+0000 7f9730a00640 1 --2- 172.21.15.86:0/1514590166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972c1052c0 0x7f972c1076b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9714009900 tx=0x7f971402bc30 comp rx=0 tx=0).stop 2024-04-17T07:21:27.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.386+0000 7f9730a00640 1 -- 172.21.15.86:0/1514590166 shutdown_connections 2024-04-17T07:21:27.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.386+0000 7f9730a00640 1 --2- 172.21.15.86:0/1514590166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c107bf0 0x7f972c109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.386+0000 7f9730a00640 1 --2- 172.21.15.86:0/1514590166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972c1052c0 0x7f972c1076b0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.386+0000 7f9730a00640 1 --2- 172.21.15.86:0/1514590166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c102990 0x7f972c104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.386+0000 7f9730a00640 1 -- 172.21.15.86:0/1514590166 >> 172.21.15.86:0/1514590166 conn(0x7f972c0fc300 msgr2=0x7f972c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.387+0000 7f9730a00640 1 -- 172.21.15.86:0/1514590166 shutdown_connections 2024-04-17T07:21:27.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.387+0000 7f9730a00640 1 -- 172.21.15.86:0/1514590166 wait complete. 2024-04-17T07:21:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.388+0000 7f9730a00640 1 Processor -- start 2024-04-17T07:21:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.388+0000 7f9730a00640 1 -- start start 2024-04-17T07:21:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.388+0000 7f9730a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972c102990 0x7f972c1022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.388+0000 7f9730a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c1052c0 0x7f972c100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.388+0000 7f9730a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c107bf0 0x7f972c100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.388+0000 7f9730a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f972c10d970 con 0x7f972c102990 2024-04-17T07:21:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.388+0000 7f9730a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f972c10d7f0 con 0x7f972c107bf0 2024-04-17T07:21:27.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.388+0000 7f9730a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f972c10daf0 con 0x7f972c1052c0 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.388+0000 7f972a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972c102990 0x7f972c1022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f972a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972c102990 0x7f972c1022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36644/0 (socket says 172.21.15.86:36644) 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f972a800640 1 -- 172.21.15.86:0/2281057823 learned_addr learned my addr 172.21.15.86:0/2281057823 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f9723e00640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c1052c0 0x7f972c100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f972b200640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c107bf0 0x7f972c100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f972a800640 1 -- 172.21.15.86:0/2281057823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c1052c0 msgr2=0x7f972c100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f972a800640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c1052c0 0x7f972c100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f972a800640 1 -- 172.21.15.86:0/2281057823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c107bf0 msgr2=0x7f972c100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f972a800640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c107bf0 0x7f972c100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f972a800640 1 -- 172.21.15.86:0/2281057823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c101740 con 0x7f972c102990 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f972b200640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c107bf0 0x7f972c100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.389+0000 7f972a800640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972c102990 0x7f972c1022b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f971c00d950 tx=0x7f971c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.390+0000 7f9723400640 1 -- 172.21.15.86:0/2281057823 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c017070 con 0x7f972c102990 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.390+0000 7f9730a00640 1 -- 172.21.15.86:0/2281057823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972c1a2230 con 0x7f972c102990 2024-04-17T07:21:27.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.390+0000 7f9723400640 1 -- 172.21.15.86:0/2281057823 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f971c009c30 con 0x7f972c102990 2024-04-17T07:21:27.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.390+0000 7f9723400640 1 -- 172.21.15.86:0/2281057823 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c00f620 con 0x7f972c102990 2024-04-17T07:21:27.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.390+0000 7f9730a00640 1 -- 172.21.15.86:0/2281057823 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972c1a27a0 con 0x7f972c102990 2024-04-17T07:21:27.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.394+0000 7f97341e7740 1 Processor -- start 2024-04-17T07:21:27.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.394+0000 7f97341e7740 1 -- start start 2024-04-17T07:21:27.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.395+0000 7f97341e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251d160 0x55ca3251d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.395+0000 7f97341e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32522650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.395+0000 7f97341e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca32522cc0 0x55ca32525150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.395+0000 7f97341e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca32483020 con 0x55ca3251db50 2024-04-17T07:21:27.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.395+0000 7f97341e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ca32429460 con 0x55ca32522cc0 2024-04-17T07:21:27.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.395+0000 7f97341e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca3247efe0 con 0x55ca3251d160 2024-04-17T07:21:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.395+0000 7f9723400640 1 -- 172.21.15.86:0/2281057823 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f971c0109a0 con 0x7f972c102990 2024-04-17T07:21:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.395+0000 7f972b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251d160 0x55ca3251d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.395+0000 7f972b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251d160 0x55ca3251d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52348/0 (socket says 172.21.15.86:52348) 2024-04-17T07:21:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.395+0000 7f972b200640 1 -- 172.21.15.86:0/392493071 learned_addr learned my addr 172.21.15.86:0/392493071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.396+0000 7f9723e00640 1 --2- 172.21.15.86:0/392493071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca32522cc0 0x55ca32525150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.396+0000 7f972a800640 1 --2- 172.21.15.86:0/392493071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32522650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.396+0000 7f972b200640 1 -- 172.21.15.86:0/392493071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca32522cc0 msgr2=0x55ca32525150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.396+0000 7f972b200640 1 --2- 172.21.15.86:0/392493071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca32522cc0 0x55ca32525150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.396+0000 7f972b200640 1 -- 172.21.15.86:0/392493071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 msgr2=0x55ca32522650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.398+0000 7f9723400640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97100a3bb0 0x7f97100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.398+0000 7f972b200640 1 --2- 172.21.15.86:0/392493071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32522650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.398+0000 7f9723e00640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97100a3bb0 0x7f97100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.398+0000 7f972b200640 1 -- 172.21.15.86:0/392493071 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca32431040 con 0x55ca3251d160 2024-04-17T07:21:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.398+0000 7f972a800640 1 --2- 172.21.15.86:0/392493071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32522650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:21:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.399+0000 7f9723400640 1 -- 172.21.15.86:0/2281057823 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f971c0d0420 con 0x7f972c102990 2024-04-17T07:21:27.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.399+0000 7f9723e00640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97100a3bb0 0x7f97100a6070 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9714006010 tx=0x7f971402ca00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.399+0000 7f972b200640 1 --2- 172.21.15.86:0/392493071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251d160 0x55ca3251d580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9724008dd0 tx=0x7f972400d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a6224c9d9346a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.400+0000 7f971b400640 1 -- 172.21.15.86:0/392493071 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972400f830 con 0x55ca3251d160 2024-04-17T07:21:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.400+0000 7f971b400640 1 -- 172.21.15.86:0/392493071 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f972400de20 con 0x55ca3251d160 2024-04-17T07:21:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.400+0000 7f971b400640 1 -- 172.21.15.86:0/392493071 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972400e3b0 con 0x55ca3251d160 2024-04-17T07:21:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.401+0000 7f97341e7740 1 -- 172.21.15.86:0/392493071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251d160 msgr2=0x55ca3251d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.401+0000 7f97341e7740 1 --2- 172.21.15.86:0/392493071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251d160 0x55ca3251d580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9724008dd0 tx=0x7f972400d710 comp rx=0 tx=0).stop 2024-04-17T07:21:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.401+0000 7f97341e7740 1 -- 172.21.15.86:0/392493071 shutdown_connections 2024-04-17T07:21:27.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.401+0000 7f97341e7740 1 --2- 172.21.15.86:0/392493071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca32522cc0 0x55ca32525150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.401+0000 7f97341e7740 1 --2- 172.21.15.86:0/392493071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32522650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.401+0000 7f97341e7740 1 --2- 172.21.15.86:0/392493071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251d160 0x55ca3251d580 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.401+0000 7f97341e7740 1 -- 172.21.15.86:0/392493071 >> 172.21.15.86:0/392493071 conn(0x55ca325186d0 msgr2=0x55ca32518ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.401+0000 7f97341e7740 1 -- 172.21.15.86:0/392493071 shutdown_connections 2024-04-17T07:21:27.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.402+0000 7f97341e7740 1 -- 172.21.15.86:0/392493071 wait complete. 2024-04-17T07:21:27.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.402+0000 7f97341e7740 1 -- start start 2024-04-17T07:21:27.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.402+0000 7f97341e7740 1 Processor -- start 2024-04-17T07:21:27.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.403+0000 7f97341e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 0x55ca325276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.403+0000 7f97341e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32527c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.403+0000 7f97341e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca32522cc0 0x55ca3252ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.403+0000 7f97341e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca32525c70 con 0x55ca3251db50 2024-04-17T07:21:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.403+0000 7f97341e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ca3243c830 con 0x55ca3251d160 2024-04-17T07:21:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.403+0000 7f97341e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca32525df0 con 0x55ca32522cc0 2024-04-17T07:21:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.403+0000 7f97341e7740 10 client.0 _ll_register_callbacks cb 0x55ca3240afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.403+0000 7f9723e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32527c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.403+0000 7f9723e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32527c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36654/0 (socket says 172.21.15.86:36654) 2024-04-17T07:21:27.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.403+0000 7f9723e00640 1 -- 172.21.15.86:0/1704669394 learned_addr learned my addr 172.21.15.86:0/1704669394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f9723e00640 1 -- 172.21.15.86:0/1704669394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca32522cc0 msgr2=0x55ca3252ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f972b200640 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca32522cc0 0x55ca3252ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f9723e00640 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca32522cc0 0x55ca3252ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f9723e00640 1 -- 172.21.15.86:0/1704669394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 msgr2=0x55ca325276d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f9723e00640 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 0x55ca325276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f9723e00640 1 -- 172.21.15.86:0/1704669394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca32440920 con 0x55ca3251db50 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f9723e00640 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32527c10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f972c069bb0 tx=0x7f971402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04cfb221ac663fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f970fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f970fe00640 1 -- 172.21.15.86:0/1704669394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971404e030 con 0x55ca3251db50 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f970fe00640 1 -- 172.21.15.86:0/1704669394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f97140048d0 con 0x55ca3251db50 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.404+0000 7f97341e7740 1 -- 172.21.15.86:0/1704669394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f972400de20 con 0x55ca3251db50 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.405+0000 7f970fe00640 1 -- 172.21.15.86:0/1704669394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9714004ad0 con 0x55ca3251db50 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.405+0000 7f97341e7740 10 client.17277 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:27.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.405+0000 7f97341e7740 1 -- 172.21.15.86:0/1704669394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca3240e7f0 con 0x55ca3251db50 2024-04-17T07:21:27.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.407+0000 7f970fe00640 1 -- 172.21.15.86:0/1704669394 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f972400de20 con 0x55ca3251db50 2024-04-17T07:21:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.408+0000 7f970fe00640 1 -- 172.21.15.86:0/1704669394 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f971404a450 con 0x55ca3251db50 2024-04-17T07:21:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.408+0000 7f97341e7740 10 client.17277 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.408+0000 7f97341e7740 10 client.17277 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:21:27.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.408+0000 7f97341e7740 20 client.17277 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.408+0000 7f97341e7740 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca32533200 0x55ca32535690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.408+0000 7f97341e7740 4 client.17277 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:21:27.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.408+0000 7f97341e7740 1 -- 172.21.15.86:0/1704669394 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca32490a40 con 0x55ca32533200 2024-04-17T07:21:27.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.411+0000 7f972a800640 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca32533200 0x55ca32535690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.415+0000 7f972a800640 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca32533200 0x55ca32535690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dee706ff2760397d server_cookie=cd739a0b8b41cd0e in_seq=0 out_seq=0 2024-04-17T07:21:27.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.415+0000 7f970fe00640 10 client.17277 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:21:27.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.418+0000 7f970fe00640 1 -- 172.21.15.86:0/1704669394 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca32490a40 con 0x55ca32533200 2024-04-17T07:21:27.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.418+0000 7f970fe00640 10 client.17277 handle_command_reply: tid=0 2024-04-17T07:21:27.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.418+0000 7f97341e7740 1 client.17277 shutdown 2024-04-17T07:21:27.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.418+0000 7f97341e7740 1 -- 172.21.15.86:0/1704669394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 msgr2=0x55ca32527c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.418+0000 7f97341e7740 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32527c10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f972c069bb0 tx=0x7f971402bc30 comp rx=0 tx=0).stop 2024-04-17T07:21:27.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.419+0000 7f97341e7740 1 -- 172.21.15.86:0/1704669394 shutdown_connections 2024-04-17T07:21:27.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.419+0000 7f97341e7740 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca32533200 0x55ca32535690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.419+0000 7f970fe00640 0 client.17277 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:21:27.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.419+0000 7f97341e7740 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca32522cc0 0x55ca3252ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.419+0000 7f97341e7740 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251db50 0x55ca32527c10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.419+0000 7f97341e7740 1 --2- 172.21.15.86:0/1704669394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 0x55ca325276d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.419+0000 7f97341e7740 1 -- 172.21.15.86:0/1704669394 >> 172.21.15.86:0/1704669394 conn(0x55ca325186d0 msgr2=0x55ca32517930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.420+0000 7f97341e7740 1 -- 172.21.15.86:0/1704669394 shutdown_connections 2024-04-17T07:21:27.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.421+0000 7f97341e7740 1 -- 172.21.15.86:0/1704669394 wait complete. 2024-04-17T07:21:27.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.421+0000 7f97341e7740 20 client.17277 trim_cache size 0 max 16384 2024-04-17T07:21:27.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.451+0000 7f97341e7740 1 Processor -- start 2024-04-17T07:21:27.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.451+0000 7f97341e7740 1 -- start start 2024-04-17T07:21:27.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.452+0000 7f97341e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 0x55ca325170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.452+0000 7f97341e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251db50 0x55ca3252ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.452+0000 7f97341e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32522cc0 0x55ca325175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.452+0000 7f97341e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1acac0 con 0x55ca32522cc0 2024-04-17T07:21:27.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.452+0000 7f97341e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1ae600 con 0x55ca3251d160 2024-04-17T07:21:27.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.452+0000 7f97341e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1ad190 con 0x55ca3251db50 2024-04-17T07:21:27.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.452+0000 7f972a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 0x55ca325170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.453+0000 7f9723e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251db50 0x55ca3252ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.453+0000 7f972b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32522cc0 0x55ca325175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.453+0000 7f9723e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251db50 0x55ca3252ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52358/0 (socket says 172.21.15.86:52358) 2024-04-17T07:21:27.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.453+0000 7f972a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 0x55ca325170a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36984/0 (socket says 172.21.15.86:36984) 2024-04-17T07:21:27.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.453+0000 7f9723e00640 1 -- 172.21.15.86:0/2175585564 learned_addr learned my addr 172.21.15.86:0/2175585564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.454+0000 7f972a800640 1 -- 172.21.15.86:0/2175585564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251db50 msgr2=0x55ca3252ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.454+0000 7f972a800640 1 --2- 172.21.15.86:0/2175585564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251db50 0x55ca3252ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.454+0000 7f972a800640 1 -- 172.21.15.86:0/2175585564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32522cc0 msgr2=0x55ca325175e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.454+0000 7f972a800640 1 --2- 172.21.15.86:0/2175585564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32522cc0 0x55ca325175e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.454+0000 7f972a800640 1 -- 172.21.15.86:0/2175585564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca321953f0 con 0x55ca3251d160 2024-04-17T07:21:27.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.456+0000 7f972a800640 1 --2- 172.21.15.86:0/2175585564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 0x55ca325170a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f971c0cdb70 tx=0x7f971c009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7960c9db9d47b258 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.456+0000 7f971aa00640 1 -- 172.21.15.86:0/2175585564 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c020040 con 0x55ca3251d160 2024-04-17T07:21:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.456+0000 7f971aa00640 1 -- 172.21.15.86:0/2175585564 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f971c0a0730 con 0x55ca3251d160 2024-04-17T07:21:27.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.456+0000 7f971aa00640 1 -- 172.21.15.86:0/2175585564 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c0989f0 con 0x55ca3251d160 2024-04-17T07:21:27.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.458+0000 7f97341e7740 1 -- 172.21.15.86:0/2175585564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 msgr2=0x55ca325170a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.458+0000 7f97341e7740 1 --2- 172.21.15.86:0/2175585564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 0x55ca325170a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f971c0cdb70 tx=0x7f971c009740 comp rx=0 tx=0).stop 2024-04-17T07:21:27.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.458+0000 7f97341e7740 1 -- 172.21.15.86:0/2175585564 shutdown_connections 2024-04-17T07:21:27.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.458+0000 7f97341e7740 1 --2- 172.21.15.86:0/2175585564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca32522cc0 0x55ca325175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.458+0000 7f97341e7740 1 --2- 172.21.15.86:0/2175585564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca3251db50 0x55ca3252ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.458+0000 7f97341e7740 1 --2- 172.21.15.86:0/2175585564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251d160 0x55ca325170a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.458+0000 7f97341e7740 1 -- 172.21.15.86:0/2175585564 >> 172.21.15.86:0/2175585564 conn(0x55ca325186d0 msgr2=0x55ca3251f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.458+0000 7f97341e7740 1 -- 172.21.15.86:0/2175585564 shutdown_connections 2024-04-17T07:21:27.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.458+0000 7f97341e7740 1 -- 172.21.15.86:0/2175585564 wait complete. 2024-04-17T07:21:27.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.458+0000 7f97341e7740 1 -- start start 2024-04-17T07:21:27.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.459+0000 7f97341e7740 1 Processor -- start 2024-04-17T07:21:27.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.459+0000 7f97341e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251d160 0x55ca3252bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.459+0000 7f97341e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251db50 0x55ca3252c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.459+0000 7f97341e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca32522cc0 0x55ca3252d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.459+0000 7f97341e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1adf30 con 0x55ca3251d160 2024-04-17T07:21:27.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.459+0000 7f97341e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1ad860 con 0x55ca3251db50 2024-04-17T07:21:27.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.459+0000 7f97341e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f972c1aaa90 con 0x55ca32522cc0 2024-04-17T07:21:27.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.459+0000 7f97341e7740 10 client.0 _ll_register_callbacks cb 0x7f9714004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:27.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.460+0000 7f9723e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251db50 0x55ca3252c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.460+0000 7f972a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251d160 0x55ca3252bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.460+0000 7f9723e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251db50 0x55ca3252c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36998/0 (socket says 172.21.15.86:36998) 2024-04-17T07:21:27.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.460+0000 7f972b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca32522cc0 0x55ca3252d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.460+0000 7f9723e00640 1 -- 172.21.15.86:0/3779240136 learned_addr learned my addr 172.21.15.86:0/3779240136 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.461+0000 7f9723e00640 1 -- 172.21.15.86:0/3779240136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca32522cc0 msgr2=0x55ca3252d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.461+0000 7f9723e00640 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca32522cc0 0x55ca3252d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.461+0000 7f9723e00640 1 -- 172.21.15.86:0/3779240136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251d160 msgr2=0x55ca3252bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.461+0000 7f9723e00640 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251d160 0x55ca3252bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.461+0000 7f9723e00640 1 -- 172.21.15.86:0/3779240136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca3225f7d0 con 0x55ca3251db50 2024-04-17T07:21:27.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.461+0000 7f9723e00640 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251db50 0x55ca3252c3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9714009900 tx=0x7f971404add0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c65bfaf86bd3be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.461+0000 7f970fe00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:21:27.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.462+0000 7f970fe00640 1 -- 172.21.15.86:0/3779240136 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971404e040 con 0x55ca3251db50 2024-04-17T07:21:27.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.462+0000 7f970fe00640 1 -- 172.21.15.86:0/3779240136 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f971404a450 con 0x55ca3251db50 2024-04-17T07:21:27.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.462+0000 7f97341e7740 1 -- 172.21.15.86:0/3779240136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f971c0a0730 con 0x55ca3251db50 2024-04-17T07:21:27.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.462+0000 7f970fe00640 1 -- 172.21.15.86:0/3779240136 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f971404a650 con 0x55ca3251db50 2024-04-17T07:21:27.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.462+0000 7f97341e7740 10 client.36281 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:27.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.462+0000 7f97341e7740 1 -- 172.21.15.86:0/3779240136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca3252eb10 con 0x55ca3251db50 2024-04-17T07:21:27.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.463+0000 7f970fe00640 1 -- 172.21.15.86:0/3779240136 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f971c0a0730 con 0x55ca3251db50 2024-04-17T07:21:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.463+0000 7f970fe00640 1 -- 172.21.15.86:0/3779240136 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9714061560 con 0x55ca3251db50 2024-04-17T07:21:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.463+0000 7f97341e7740 10 client.36281 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.463+0000 7f97341e7740 10 client.36281 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:21:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.463+0000 7f97341e7740 20 client.36281 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.463+0000 7f97341e7740 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca325999e0 0x55ca3259be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.463+0000 7f97341e7740 4 client.36281 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:21:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.463+0000 7f97341e7740 1 -- 172.21.15.86:0/3779240136 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca3248a8b0 con 0x55ca325999e0 2024-04-17T07:21:27.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.464+0000 7f972a800640 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca325999e0 0x55ca3259be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.464+0000 7f972a800640 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca325999e0 0x55ca3259be00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b4fdaf96224aef1 server_cookie=3f89b23e5404460a in_seq=0 out_seq=0 2024-04-17T07:21:27.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.464+0000 7f970fe00640 10 client.36281 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:21:27.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f970fe00640 1 -- 172.21.15.86:0/3779240136 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca3248a8b0 con 0x55ca325999e0 2024-04-17T07:21:27.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f970fe00640 10 client.36281 handle_command_reply: tid=0 2024-04-17T07:21:27.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 client.36281 shutdown 2024-04-17T07:21:27.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 -- 172.21.15.86:0/3779240136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251db50 msgr2=0x55ca3252c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251db50 0x55ca3252c3e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9714009900 tx=0x7f971404add0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 -- 172.21.15.86:0/3779240136 shutdown_connections 2024-04-17T07:21:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ca325999e0 0x55ca3259be00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca32522cc0 0x55ca3252d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca3251db50 0x55ca3252c3e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 --2- 172.21.15.86:0/3779240136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca3251d160 0x55ca3252bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 -- 172.21.15.86:0/3779240136 >> 172.21.15.86:0/3779240136 conn(0x55ca325186d0 msgr2=0x55ca3251b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f970fe00640 0 client.36281 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:21:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 -- 172.21.15.86:0/3779240136 shutdown_connections 2024-04-17T07:21:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.465+0000 7f97341e7740 1 -- 172.21.15.86:0/3779240136 wait complete. 2024-04-17T07:21:27.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.466+0000 7f97341e7740 20 client.36281 trim_cache size 0 max 16384 2024-04-17T07:21:27.467 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.466+0000 7f9730a00640 1 -- 172.21.15.86:0/2281057823 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97100a3bb0 msgr2=0x7f97100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.466+0000 7f9730a00640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97100a3bb0 0x7f97100a6070 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9714006010 tx=0x7f971402ca00 comp rx=0 tx=0).stop 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.466+0000 7f9730a00640 1 -- 172.21.15.86:0/2281057823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972c102990 msgr2=0x7f972c1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.466+0000 7f9730a00640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972c102990 0x7f972c1022b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f971c00d950 tx=0x7f971c00de10 comp rx=0 tx=0).stop 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.467+0000 7f9730a00640 1 -- 172.21.15.86:0/2281057823 shutdown_connections 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.467+0000 7f9730a00640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97100a3bb0 0x7f97100a6070 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.467+0000 7f9730a00640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f972c107bf0 0x7f972c100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.467+0000 7f9730a00640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c1052c0 0x7f972c100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.467+0000 7f9730a00640 1 --2- 172.21.15.86:0/2281057823 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f972c102990 0x7f972c1022b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.467+0000 7f9730a00640 1 -- 172.21.15.86:0/2281057823 >> 172.21.15.86:0/2281057823 conn(0x7f972c0fc300 msgr2=0x7f972c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.467+0000 7f9730a00640 1 -- 172.21.15.86:0/2281057823 shutdown_connections 2024-04-17T07:21:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.467+0000 7f9730a00640 1 -- 172.21.15.86:0/2281057823 wait complete. 2024-04-17T07:21:27.484 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:27 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/92921852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:21:27.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:21:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.729+0000 7f33e1400640 1 -- 172.21.15.86:0/4242005462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33dc101ea0 msgr2=0x7f33dc1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.729+0000 7f33e1400640 1 --2- 172.21.15.86:0/4242005462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33dc101ea0 0x7f33dc1022c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009900 tx=0x7f33cc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:21:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.729+0000 7f33e1400640 1 -- 172.21.15.86:0/4242005462 shutdown_connections 2024-04-17T07:21:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.729+0000 7f33e1400640 1 --2- 172.21.15.86:0/4242005462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33dc103a40 0x7f33dc10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.729+0000 7f33e1400640 1 --2- 172.21.15.86:0/4242005462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33dc103080 0x7f33dc103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.729+0000 7f33e1400640 1 --2- 172.21.15.86:0/4242005462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33dc101ea0 0x7f33dc1022c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.729+0000 7f33e1400640 1 -- 172.21.15.86:0/4242005462 >> 172.21.15.86:0/4242005462 conn(0x7f33dc0fd460 msgr2=0x7f33dc0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.729+0000 7f33e1400640 1 -- 172.21.15.86:0/4242005462 shutdown_connections 2024-04-17T07:21:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.729+0000 7f33e1400640 1 -- 172.21.15.86:0/4242005462 wait complete. 2024-04-17T07:21:27.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.730+0000 7f33e1400640 1 Processor -- start 2024-04-17T07:21:27.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.730+0000 7f33e1400640 1 -- start start 2024-04-17T07:21:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33e1400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33dc101ea0 0x7f33dc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33dc103080 0x7f33dc198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33e1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33dc103a40 0x7f33dc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33e1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc10d9a0 con 0x7f33dc103080 2024-04-17T07:21:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33e1400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc10d820 con 0x7f33dc101ea0 2024-04-17T07:21:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33e1400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc10db20 con 0x7f33dc103a40 2024-04-17T07:21:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33dc103080 0x7f33dc198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33dc103080 0x7f33dc198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36692/0 (socket says 172.21.15.86:36692) 2024-04-17T07:21:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33d3e00640 1 -- 172.21.15.86:0/3437659218 learned_addr learned my addr 172.21.15.86:0/3437659218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33e0a00640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33dc103a40 0x7f33dc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33d3e00640 1 -- 172.21.15.86:0/3437659218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33dc103a40 msgr2=0x7f33dc19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33d3e00640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33dc103a40 0x7f33dc19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33d3e00640 1 -- 172.21.15.86:0/3437659218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33dc101ea0 msgr2=0x7f33dc1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.731+0000 7f33dbe00640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33dc101ea0 0x7f33dc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.732+0000 7f33d3e00640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33dc101ea0 0x7f33dc1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.732+0000 7f33d3e00640 1 -- 172.21.15.86:0/3437659218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33dc19fcf0 con 0x7f33dc103080 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.732+0000 7f33e0a00640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33dc103a40 0x7f33dc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.732+0000 7f33dbe00640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33dc101ea0 0x7f33dc1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.732+0000 7f33d3e00640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33dc103080 0x7f33dc198610 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f33c400d950 tx=0x7f33c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.733+0000 7f33d9600640 1 -- 172.21.15.86:0/3437659218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4017070 con 0x7f33dc103080 2024-04-17T07:21:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.733+0000 7f33d9600640 1 -- 172.21.15.86:0/3437659218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33c4009c30 con 0x7f33dc103080 2024-04-17T07:21:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.733+0000 7f33d9600640 1 -- 172.21.15.86:0/3437659218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c400f620 con 0x7f33dc103080 2024-04-17T07:21:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.733+0000 7f33e1400640 1 -- 172.21.15.86:0/3437659218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33dc19ffe0 con 0x7f33dc103080 2024-04-17T07:21:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.733+0000 7f33e1400640 1 -- 172.21.15.86:0/3437659218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33dc074430 con 0x7f33dc103080 2024-04-17T07:21:27.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.736+0000 7f33e5678740 1 Processor -- start 2024-04-17T07:21:27.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.736+0000 7f33e5678740 1 -- start start 2024-04-17T07:21:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33d9600640 1 -- 172.21.15.86:0/3437659218 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f33c40109a0 con 0x7f33dc103080 2024-04-17T07:21:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33e5678740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595557160 0x55e595557580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33e5678740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595557b50 0x55e59555c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33e5678740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e59555ccc0 0x55e59555f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33e5678740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5954bd020 con 0x55e595557b50 2024-04-17T07:21:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33e5678740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e595463460 con 0x55e59555ccc0 2024-04-17T07:21:27.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33e5678740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e5954b8fe0 con 0x55e595557160 2024-04-17T07:21:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33e0a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595557160 0x55e595557580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33e0a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595557160 0x55e595557580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52388/0 (socket says 172.21.15.86:52388) 2024-04-17T07:21:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33e0a00640 1 -- 172.21.15.86:0/2275719198 learned_addr learned my addr 172.21.15.86:0/2275719198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33d3e00640 1 --2- 172.21.15.86:0/2275719198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e59555ccc0 0x55e59555f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.737+0000 7f33dbe00640 1 --2- 172.21.15.86:0/2275719198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595557b50 0x55e59555c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.738+0000 7f33dbe00640 1 -- 172.21.15.86:0/2275719198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595557160 msgr2=0x55e595557580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.738+0000 7f33dbe00640 1 --2- 172.21.15.86:0/2275719198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595557160 0x55e595557580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.738+0000 7f33dbe00640 1 -- 172.21.15.86:0/2275719198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e59555ccc0 msgr2=0x55e59555f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.738+0000 7f33dbe00640 1 --2- 172.21.15.86:0/2275719198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e59555ccc0 0x55e59555f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.738+0000 7f33dbe00640 1 -- 172.21.15.86:0/2275719198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59546b040 con 0x55e595557b50 2024-04-17T07:21:27.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.738+0000 7f33d3e00640 1 --2- 172.21.15.86:0/2275719198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e59555ccc0 0x55e59555f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:21:27.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.739+0000 7f33dbe00640 1 --2- 172.21.15.86:0/2275719198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595557b50 0x55e59555c650 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f33cc036040 tx=0x7f33cc005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815e0b8d051ace00 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.739+0000 7f33d9600640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f33c00a3c30 0x7f33c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.739+0000 7f33d9600640 1 -- 172.21.15.86:0/3437659218 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f33c40d03d0 con 0x7f33dc103080 2024-04-17T07:21:27.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.739+0000 7f33d1200640 1 -- 172.21.15.86:0/2275719198 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc043020 con 0x55e595557b50 2024-04-17T07:21:27.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.739+0000 7f33d1200640 1 -- 172.21.15.86:0/2275719198 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33cc004040 con 0x55e595557b50 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.739+0000 7f33d1200640 1 -- 172.21.15.86:0/2275719198 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0046c0 con 0x55e595557b50 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.739+0000 7f33e0a00640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f33c00a3c30 0x7f33c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.740+0000 7f33e0a00640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f33c00a3c30 0x7f33c00a60f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55e5955583b0 tx=0x7f33d40098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.740+0000 7f33e5678740 1 -- 172.21.15.86:0/2275719198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595557b50 msgr2=0x55e59555c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.740+0000 7f33e5678740 1 --2- 172.21.15.86:0/2275719198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595557b50 0x55e59555c650 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f33cc036040 tx=0x7f33cc005e00 comp rx=0 tx=0).stop 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.740+0000 7f33e5678740 1 -- 172.21.15.86:0/2275719198 shutdown_connections 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.740+0000 7f33e5678740 1 --2- 172.21.15.86:0/2275719198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e59555ccc0 0x55e59555f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.740+0000 7f33e5678740 1 --2- 172.21.15.86:0/2275719198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595557b50 0x55e59555c650 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.740+0000 7f33e5678740 1 --2- 172.21.15.86:0/2275719198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595557160 0x55e595557580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.740+0000 7f33e5678740 1 -- 172.21.15.86:0/2275719198 >> 172.21.15.86:0/2275719198 conn(0x55e5955526d0 msgr2=0x55e595552ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.740+0000 7f33e5678740 1 -- 172.21.15.86:0/2275719198 shutdown_connections 2024-04-17T07:21:27.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.740+0000 7f33e5678740 1 -- 172.21.15.86:0/2275719198 wait complete. 2024-04-17T07:21:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.741+0000 7f33e5678740 1 -- start start 2024-04-17T07:21:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.741+0000 7f33e5678740 1 Processor -- start 2024-04-17T07:21:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.741+0000 7f33e5678740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e595561870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.741+0000 7f33e5678740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 0x55e595561db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.741+0000 7f33e5678740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e5678740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e59555fc10 con 0x55e595566de0 2024-04-17T07:21:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e5678740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e595476830 con 0x55e595557160 2024-04-17T07:21:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e5678740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e59555fd90 con 0x55e59555ccc0 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36696/0 (socket says 172.21.15.86:36696) 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e0a00640 1 -- 172.21.15.86:0/2832535166 learned_addr learned my addr 172.21.15.86:0/2832535166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33d3e00640 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 0x55e595561db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33dbe00640 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e595561870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e5678740 10 client.0 _ll_register_callbacks cb 0x55e595444fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e0a00640 1 -- 172.21.15.86:0/2832535166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 msgr2=0x55e595561db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e0a00640 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 0x55e595561db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e0a00640 1 -- 172.21.15.86:0/2832535166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 msgr2=0x55e595561870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e0a00640 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e595561870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.742+0000 7f33e0a00640 1 -- 172.21.15.86:0/2832535166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e59547a920 con 0x55e595566de0 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.743+0000 7f33e0a00640 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955691d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f33d4011d40 tx=0x7f33d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7bb340a67cd835 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.743+0000 7f33c9600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.743+0000 7f33c9600640 1 -- 172.21.15.86:0/2832535166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4020620 con 0x55e595566de0 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.743+0000 7f33e5678740 1 -- 172.21.15.86:0/2832535166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33cc004040 con 0x55e595566de0 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.743+0000 7f33c9600640 1 -- 172.21.15.86:0/2832535166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f33d40207c0 con 0x55e595566de0 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.743+0000 7f33c9600640 1 -- 172.21.15.86:0/2832535166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33d4033050 con 0x55e595566de0 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.743+0000 7f33e5678740 10 client.17295 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.744+0000 7f33e5678740 1 -- 172.21.15.86:0/2832535166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5954487f0 con 0x55e595566de0 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.744+0000 7f33c9600640 1 -- 172.21.15.86:0/2832535166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33cc004040 con 0x55e595566de0 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.744+0000 7f33c9600640 1 -- 172.21.15.86:0/2832535166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f33d400f040 con 0x55e595566de0 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.744+0000 7f33e5678740 10 client.17295 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.744+0000 7f33e5678740 10 client.17295 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.744+0000 7f33e5678740 20 client.17295 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:27.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.744+0000 7f33e5678740 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55e59556d780 0x55e59556fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.744+0000 7f33e5678740 4 client.17295 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:21:27.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.744+0000 7f33e5678740 1 -- 172.21.15.86:0/2832535166 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5954caa40 con 0x55e59556d780 2024-04-17T07:21:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.745+0000 7f33dbe00640 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55e59556d780 0x55e59556fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.745+0000 7f33dbe00640 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55e59556d780 0x55e59556fc20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=170895f20d36873b server_cookie=b68b573191e8513e in_seq=0 out_seq=0 2024-04-17T07:21:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.745+0000 7f33c9600640 10 client.17295 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:21:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33c9600640 1 -- 172.21.15.86:0/2832535166 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5954caa40 con 0x55e59556d780 2024-04-17T07:21:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33c9600640 10 client.17295 handle_command_reply: tid=0 2024-04-17T07:21:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33e5678740 1 client.17295 shutdown 2024-04-17T07:21:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33e5678740 1 -- 172.21.15.86:0/2832535166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 msgr2=0x55e5955691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33e5678740 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955691d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f33d4011d40 tx=0x7f33d4004a40 comp rx=0 tx=0).stop 2024-04-17T07:21:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33e5678740 1 -- 172.21.15.86:0/2832535166 shutdown_connections 2024-04-17T07:21:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33e5678740 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55e59556d780 0x55e59556fc20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33e5678740 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955691d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33e5678740 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 0x55e595561db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33e5678740 1 --2- 172.21.15.86:0/2832535166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e595561870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33e5678740 1 -- 172.21.15.86:0/2832535166 >> 172.21.15.86:0/2832535166 conn(0x55e5955526d0 msgr2=0x55e595551930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.746+0000 7f33c9600640 0 client.17295 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:21:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.747+0000 7f33e5678740 1 -- 172.21.15.86:0/2832535166 shutdown_connections 2024-04-17T07:21:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.747+0000 7f33e5678740 1 -- 172.21.15.86:0/2832535166 wait complete. 2024-04-17T07:21:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.747+0000 7f33e5678740 20 client.17295 trim_cache size 0 max 16384 2024-04-17T07:21:27.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:27 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/92921852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:21:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.764+0000 7f33e5678740 1 Processor -- start 2024-04-17T07:21:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33e5678740 1 -- start start 2024-04-17T07:21:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33e5678740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e595559230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33e5678740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 0x55e595559770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33e5678740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33e5678740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc1a8840 con 0x55e595566de0 2024-04-17T07:21:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33e5678740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc1aa380 con 0x55e595557160 2024-04-17T07:21:27.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33e5678740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc1a8f10 con 0x55e59555ccc0 2024-04-17T07:21:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33e0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33e0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36704/0 (socket says 172.21.15.86:36704) 2024-04-17T07:21:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33e0a00640 1 -- 172.21.15.86:0/1003682238 learned_addr learned my addr 172.21.15.86:0/1003682238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.765+0000 7f33dbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e595559230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33d3e00640 1 --2- 172.21.15.86:0/1003682238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 0x55e595559770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33e0a00640 1 -- 172.21.15.86:0/1003682238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 msgr2=0x55e595559770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33e0a00640 1 --2- 172.21.15.86:0/1003682238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 0x55e595559770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33e0a00640 1 -- 172.21.15.86:0/1003682238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 msgr2=0x55e595559230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33e0a00640 1 --2- 172.21.15.86:0/1003682238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e595559230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33e0a00640 1 -- 172.21.15.86:0/1003682238 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5951cf3f0 con 0x55e595566de0 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33d3e00640 1 --2- 172.21.15.86:0/1003682238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 0x55e595559770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33e0a00640 1 --2- 172.21.15.86:0/1003682238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955510a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f33d401db10 tx=0x7f33d4005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bcb50fd735ff40 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33cbe00640 1 -- 172.21.15.86:0/1003682238 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d403b070 con 0x55e595566de0 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33cbe00640 1 -- 172.21.15.86:0/1003682238 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33d401fe80 con 0x55e595566de0 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.766+0000 7f33cbe00640 1 -- 172.21.15.86:0/1003682238 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d40052a0 con 0x55e595566de0 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.767+0000 7f33e5678740 1 -- 172.21.15.86:0/1003682238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 msgr2=0x55e5955510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.767+0000 7f33e5678740 1 --2- 172.21.15.86:0/1003682238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955510a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f33d401db10 tx=0x7f33d4005f50 comp rx=0 tx=0).stop 2024-04-17T07:21:27.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.767+0000 7f33e5678740 1 -- 172.21.15.86:0/1003682238 shutdown_connections 2024-04-17T07:21:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.767+0000 7f33e5678740 1 --2- 172.21.15.86:0/1003682238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e595566de0 0x55e5955510a0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.767+0000 7f33e5678740 1 --2- 172.21.15.86:0/1003682238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e59555ccc0 0x55e595559770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.767+0000 7f33e5678740 1 --2- 172.21.15.86:0/1003682238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e595559230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.767+0000 7f33e5678740 1 -- 172.21.15.86:0/1003682238 >> 172.21.15.86:0/1003682238 conn(0x55e5955526d0 msgr2=0x55e595561870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.767+0000 7f33e5678740 1 -- 172.21.15.86:0/1003682238 shutdown_connections 2024-04-17T07:21:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.767+0000 7f33e5678740 1 -- 172.21.15.86:0/1003682238 wait complete. 2024-04-17T07:21:27.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.767+0000 7f33e5678740 1 -- start start 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.768+0000 7f33e5678740 1 Processor -- start 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.768+0000 7f33e5678740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e59557e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.768+0000 7f33e5678740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e59555ccc0 0x55e59557ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.768+0000 7f33e5678740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595566de0 0x55e59557fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.768+0000 7f33e5678740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc1a9cb0 con 0x55e59555ccc0 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.768+0000 7f33e5678740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc1a95e0 con 0x55e595557160 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.768+0000 7f33e5678740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc1a6810 con 0x55e595566de0 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.768+0000 7f33e5678740 10 client.0 _ll_register_callbacks cb 0x7f33d4011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.768+0000 7f33d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e59555ccc0 0x55e59557ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.768+0000 7f33d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e59555ccc0 0x55e59557ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36706/0 (socket says 172.21.15.86:36706) 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33d3e00640 1 -- 172.21.15.86:0/1151448028 learned_addr learned my addr 172.21.15.86:0/1151448028 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:27.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33dbe00640 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e59557e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33e0a00640 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595566de0 0x55e59557fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33d3e00640 1 -- 172.21.15.86:0/1151448028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595566de0 msgr2=0x55e59557fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33d3e00640 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595566de0 0x55e59557fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33d3e00640 1 -- 172.21.15.86:0/1151448028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 msgr2=0x55e59557e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33d3e00640 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e59557e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33d3e00640 1 -- 172.21.15.86:0/1151448028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5952997d0 con 0x55e59555ccc0 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33d3e00640 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e59555ccc0 0x55e59557ec40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f33c40058c0 tx=0x7f33c40d07b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e21781ef6254894 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33c9600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33c9600640 1 -- 172.21.15.86:0/1151448028 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4020040 con 0x55e59555ccc0 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33c9600640 1 -- 172.21.15.86:0/1151448028 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f33c40989a0 con 0x55e59555ccc0 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.769+0000 7f33e5678740 1 -- 172.21.15.86:0/1151448028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e5954c48b0 con 0x55e59555ccc0 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33c9600640 1 -- 172.21.15.86:0/1151448028 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33c4009de0 con 0x55e59555ccc0 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33e5678740 10 client.17301 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:27.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33e5678740 1 -- 172.21.15.86:0/1151448028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5955813a0 con 0x55e59555ccc0 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33c9600640 1 -- 172.21.15.86:0/1151448028 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e5954c48b0 con 0x55e59555ccc0 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33c9600640 1 -- 172.21.15.86:0/1151448028 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f33c4010460 con 0x55e59555ccc0 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33e5678740 10 client.17301 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33e5678740 10 client.17301 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33e5678740 20 client.17301 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33e5678740 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55e5955d4000 0x55e5955d6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33e5678740 4 client.17301 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.770+0000 7f33e5678740 1 -- 172.21.15.86:0/1151448028 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e595440570 con 0x55e5955d4000 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.771+0000 7f33dbe00640 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55e5955d4000 0x55e5955d6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.772+0000 7f33dbe00640 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55e5955d4000 0x55e5955d6420 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc4d5e4b45a156ca server_cookie=be1d1797d780776 in_seq=0 out_seq=0 2024-04-17T07:21:27.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.772+0000 7f33c9600640 10 client.17301 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:21:27.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.772+0000 7f33c9600640 1 -- 172.21.15.86:0/1151448028 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e595440570 con 0x55e5955d4000 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.772+0000 7f33c9600640 10 client.17301 handle_command_reply: tid=0 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.772+0000 7f33e5678740 1 client.17301 shutdown 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.772+0000 7f33e5678740 1 -- 172.21.15.86:0/1151448028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e59555ccc0 msgr2=0x55e59557ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33e5678740 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e59555ccc0 0x55e59557ec40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f33c40058c0 tx=0x7f33c40d07b0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33e5678740 1 -- 172.21.15.86:0/1151448028 shutdown_connections 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33e5678740 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55e5955d4000 0x55e5955d6420 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33c9600640 0 client.17301 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33e5678740 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e595566de0 0x55e59557fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33e5678740 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e59555ccc0 0x55e59557ec40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33e5678740 1 --2- 172.21.15.86:0/1151448028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e595557160 0x55e59557e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33e5678740 1 -- 172.21.15.86:0/1151448028 >> 172.21.15.86:0/1151448028 conn(0x55e5955526d0 msgr2=0x55e595561830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33e5678740 1 -- 172.21.15.86:0/1151448028 shutdown_connections 2024-04-17T07:21:27.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33e5678740 1 -- 172.21.15.86:0/1151448028 wait complete. 2024-04-17T07:21:27.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.773+0000 7f33e5678740 20 client.17301 trim_cache size 0 max 16384 2024-04-17T07:21:27.777 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 -- 172.21.15.86:0/3437659218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f33c00a3c30 msgr2=0x7f33c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f33c00a3c30 0x7f33c00a60f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55e5955583b0 tx=0x7f33d40098a0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 -- 172.21.15.86:0/3437659218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33dc103080 msgr2=0x7f33dc198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33dc103080 0x7f33dc198610 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f33c400d950 tx=0x7f33c400de10 comp rx=0 tx=0).stop 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 -- 172.21.15.86:0/3437659218 shutdown_connections 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f33c00a3c30 0x7f33c00a60f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f33dc103a40 0x7f33dc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33dc103080 0x7f33dc198610 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 --2- 172.21.15.86:0/3437659218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33dc101ea0 0x7f33dc1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 -- 172.21.15.86:0/3437659218 >> 172.21.15.86:0/3437659218 conn(0x7f33dc0fd460 msgr2=0x7f33dc106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 -- 172.21.15.86:0/3437659218 shutdown_connections 2024-04-17T07:21:27.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:27.774+0000 7f33e1400640 1 -- 172.21.15.86:0/3437659218 wait complete. 2024-04-17T07:21:27.779 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:27 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/92921852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:21:27.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:21:28.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.072+0000 7ff491400640 1 -- 172.21.15.86:0/1047904541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff48c107c80 msgr2=0x7ff48c10a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.072+0000 7ff491400640 1 --2- 172.21.15.86:0/1047904541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff48c107c80 0x7ff48c10a0d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff484009c30 tx=0x7ff4840300c0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.073+0000 7ff491400640 1 -- 172.21.15.86:0/1047904541 shutdown_connections 2024-04-17T07:21:28.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.073+0000 7ff491400640 1 --2- 172.21.15.86:0/1047904541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff48c107c80 0x7ff48c10a0d0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.073+0000 7ff491400640 1 --2- 172.21.15.86:0/1047904541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c105350 0x7ff48c107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.074+0000 7ff491400640 1 --2- 172.21.15.86:0/1047904541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff48c102a20 0x7ff48c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.074+0000 7ff491400640 1 -- 172.21.15.86:0/1047904541 >> 172.21.15.86:0/1047904541 conn(0x7ff48c0fc410 msgr2=0x7ff48c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.074+0000 7ff491400640 1 -- 172.21.15.86:0/1047904541 shutdown_connections 2024-04-17T07:21:28.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.074+0000 7ff491400640 1 -- 172.21.15.86:0/1047904541 wait complete. 2024-04-17T07:21:28.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.076+0000 7ff491400640 1 Processor -- start 2024-04-17T07:21:28.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.076+0000 7ff491400640 1 -- start start 2024-04-17T07:21:28.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.076+0000 7ff491400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff48c102a20 0x7ff48c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.076+0000 7ff491400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c105350 0x7ff48c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.076+0000 7ff491400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff48c107c80 0x7ff48c1a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.076+0000 7ff491400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10b920 con 0x7ff48c102a20 2024-04-17T07:21:28.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.076+0000 7ff491400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10b7a0 con 0x7ff48c107c80 2024-04-17T07:21:28.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.076+0000 7ff491400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10baa0 con 0x7ff48c105350 2024-04-17T07:21:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.077+0000 7ff489e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c105350 0x7ff48c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.077+0000 7ff489e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c105350 0x7ff48c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52418/0 (socket says 172.21.15.86:52418) 2024-04-17T07:21:28.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.077+0000 7ff489e00640 1 -- 172.21.15.86:0/3385160274 learned_addr learned my addr 172.21.15.86:0/3385160274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.078+0000 7ff48a800640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff48c102a20 0x7ff48c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.078+0000 7ff48b200640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff48c107c80 0x7ff48c1a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.078+0000 7ff489e00640 1 -- 172.21.15.86:0/3385160274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff48c107c80 msgr2=0x7ff48c1a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.078+0000 7ff489e00640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff48c107c80 0x7ff48c1a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.078+0000 7ff489e00640 1 -- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff48c102a20 msgr2=0x7ff48c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.078+0000 7ff489e00640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff48c102a20 0x7ff48c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.078+0000 7ff489e00640 1 -- 172.21.15.86:0/3385160274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c195d20 con 0x7ff48c105350 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.079+0000 7ff48a800640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff48c102a20 0x7ff48c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.079+0000 7ff489e00640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c105350 0x7ff48c19ed20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00e9c0 tx=0x7ff47c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.079+0000 7ff483400640 1 -- 172.21.15.86:0/3385160274 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00cd80 con 0x7ff48c105350 2024-04-17T07:21:28.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.080+0000 7ff491400640 1 -- 172.21.15.86:0/3385160274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48c196010 con 0x7ff48c105350 2024-04-17T07:21:28.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.080+0000 7ff491400640 1 -- 172.21.15.86:0/3385160274 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48c1965b0 con 0x7ff48c105350 2024-04-17T07:21:28.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.080+0000 7ff483400640 1 -- 172.21.15.86:0/3385160274 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff47c0054d0 con 0x7ff48c105350 2024-04-17T07:21:28.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.080+0000 7ff483400640 1 -- 172.21.15.86:0/3385160274 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c010430 con 0x7ff48c105350 2024-04-17T07:21:28.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.084+0000 7ff494192740 1 Processor -- start 2024-04-17T07:21:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.084+0000 7ff494192740 1 -- start start 2024-04-17T07:21:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.085+0000 7ff494192740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fbfa0 0x56049c5fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.085+0000 7ff494192740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fc990 0x56049c601490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.085+0000 7ff494192740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c601b00 0x56049c603f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.085+0000 7ff494192740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56049c562020 con 0x56049c601b00 2024-04-17T07:21:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.085+0000 7ff494192740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56049c508460 con 0x56049c5fbfa0 2024-04-17T07:21:28.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.085+0000 7ff494192740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56049c55dfe0 con 0x56049c5fc990 2024-04-17T07:21:28.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.085+0000 7ff48b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fbfa0 0x56049c5fc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.085+0000 7ff48b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fbfa0 0x56049c5fc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37100/0 (socket says 172.21.15.86:37100) 2024-04-17T07:21:28.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.085+0000 7ff48b200640 1 -- 172.21.15.86:0/917600077 learned_addr learned my addr 172.21.15.86:0/917600077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:28.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.086+0000 7ff48a800640 1 --2- 172.21.15.86:0/917600077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fc990 0x56049c601490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.086+0000 7ff48b200640 1 -- 172.21.15.86:0/917600077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fc990 msgr2=0x56049c601490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.086+0000 7ff48b200640 1 --2- 172.21.15.86:0/917600077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fc990 0x56049c601490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.086+0000 7ff48b200640 1 -- 172.21.15.86:0/917600077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c601b00 msgr2=0x56049c603f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:21:28.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.086+0000 7ff48b200640 1 --2- 172.21.15.86:0/917600077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c601b00 0x56049c603f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.086+0000 7ff48b200640 1 -- 172.21.15.86:0/917600077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56049c510040 con 0x56049c5fbfa0 2024-04-17T07:21:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.087+0000 7ff483400640 1 -- 172.21.15.86:0/3385160274 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff47c004fd0 con 0x7ff48c105350 2024-04-17T07:21:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.087+0000 7ff48b200640 1 --2- 172.21.15.86:0/917600077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fbfa0 0x56049c5fc3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff484037420 tx=0x7ff484009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c25caa6319fd4179 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.087+0000 7ff477400640 1 -- 172.21.15.86:0/917600077 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48400f480 con 0x56049c5fbfa0 2024-04-17T07:21:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.088+0000 7ff477400640 1 -- 172.21.15.86:0/917600077 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff484045070 con 0x56049c5fbfa0 2024-04-17T07:21:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.088+0000 7ff477400640 1 -- 172.21.15.86:0/917600077 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48400f480 con 0x56049c5fbfa0 2024-04-17T07:21:28.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.088+0000 7ff494192740 1 -- 172.21.15.86:0/917600077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fbfa0 msgr2=0x56049c5fc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.089+0000 7ff494192740 1 --2- 172.21.15.86:0/917600077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fbfa0 0x56049c5fc3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff484037420 tx=0x7ff484009310 comp rx=0 tx=0).stop 2024-04-17T07:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.089+0000 7ff483400640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff4700a3c30 0x7ff4700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.089+0000 7ff494192740 1 -- 172.21.15.86:0/917600077 shutdown_connections 2024-04-17T07:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.089+0000 7ff494192740 1 --2- 172.21.15.86:0/917600077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c601b00 0x56049c603f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.089+0000 7ff494192740 1 --2- 172.21.15.86:0/917600077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fc990 0x56049c601490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.089+0000 7ff494192740 1 --2- 172.21.15.86:0/917600077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fbfa0 0x56049c5fc3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.089+0000 7ff483400640 1 -- 172.21.15.86:0/3385160274 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff47c0d0bd0 con 0x7ff48c105350 2024-04-17T07:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.089+0000 7ff494192740 1 -- 172.21.15.86:0/917600077 >> 172.21.15.86:0/917600077 conn(0x56049c5f7510 msgr2=0x56049c5f7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.089+0000 7ff48b200640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff4700a3c30 0x7ff4700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.090+0000 7ff494192740 1 -- 172.21.15.86:0/917600077 shutdown_connections 2024-04-17T07:21:28.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.090+0000 7ff48b200640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff4700a3c30 0x7ff4700a60f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff4840306a0 tx=0x7ff484005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.090+0000 7ff494192740 1 -- 172.21.15.86:0/917600077 wait complete. 2024-04-17T07:21:28.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.091+0000 7ff494192740 1 -- start start 2024-04-17T07:21:28.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.092+0000 7ff494192740 1 Processor -- start 2024-04-17T07:21:28.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.092+0000 7ff494192740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fbfa0 0x56049c6064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.092+0000 7ff494192740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 0x56049c606a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.092+0000 7ff494192740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c601b00 0x56049c60da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.092+0000 7ff494192740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56049c604af0 con 0x56049c5fc990 2024-04-17T07:21:28.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.092+0000 7ff494192740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56049c51b830 con 0x56049c601b00 2024-04-17T07:21:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.092+0000 7ff494192740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56049c604c70 con 0x56049c5fbfa0 2024-04-17T07:21:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.093+0000 7ff494192740 10 client.0 _ll_register_callbacks cb 0x56049c4e9fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.093+0000 7ff489e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 0x56049c606a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.093+0000 7ff48a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fbfa0 0x56049c6064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.093+0000 7ff48b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c601b00 0x56049c60da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.093+0000 7ff48b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c601b00 0x56049c60da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37112/0 (socket says 172.21.15.86:37112) 2024-04-17T07:21:28.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.093+0000 7ff48b200640 1 -- 172.21.15.86:0/1318001846 learned_addr learned my addr 172.21.15.86:0/1318001846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:28.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.094+0000 7ff489e00640 1 -- 172.21.15.86:0/1318001846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fbfa0 msgr2=0x56049c6064f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.094+0000 7ff489e00640 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fbfa0 0x56049c6064f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.094+0000 7ff489e00640 1 -- 172.21.15.86:0/1318001846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c601b00 msgr2=0x56049c60da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.094+0000 7ff489e00640 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c601b00 0x56049c60da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.094+0000 7ff489e00640 1 -- 172.21.15.86:0/1318001846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56049c51f920 con 0x56049c5fc990 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.095+0000 7ff48b200640 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c601b00 0x56049c60da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.096+0000 7ff489e00640 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 0x56049c606a30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0ce3a0 tx=0x7ff47c09c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf0a8332fa6cced server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.096+0000 7ff46be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.096+0000 7ff46be00640 1 -- 172.21.15.86:0/1318001846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c09de60 con 0x56049c5fc990 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.096+0000 7ff46be00640 1 -- 172.21.15.86:0/1318001846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff47c0bd070 con 0x56049c5fc990 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.096+0000 7ff494192740 1 -- 172.21.15.86:0/1318001846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff484045070 con 0x56049c5fc990 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.096+0000 7ff46be00640 1 -- 172.21.15.86:0/1318001846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff47c09d230 con 0x56049c5fc990 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.096+0000 7ff494192740 10 client.17310 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.096+0000 7ff494192740 1 -- 172.21.15.86:0/1318001846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56049c4ed7f0 con 0x56049c5fc990 2024-04-17T07:21:28.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.097+0000 7ff46be00640 1 -- 172.21.15.86:0/1318001846 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff484045070 con 0x56049c5fc990 2024-04-17T07:21:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.097+0000 7ff46be00640 1 -- 172.21.15.86:0/1318001846 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff47c098880 con 0x56049c5fc990 2024-04-17T07:21:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.097+0000 7ff494192740 10 client.17310 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.097+0000 7ff494192740 10 client.17310 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:21:28.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.098+0000 7ff494192740 20 client.17310 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:28.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.098+0000 7ff494192740 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56049c612010 0x56049c614400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.098+0000 7ff494192740 4 client.17310 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:21:28.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.098+0000 7ff494192740 1 -- 172.21.15.86:0/1318001846 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56049c56fa40 con 0x56049c612010 2024-04-17T07:21:28.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.098+0000 7ff48a800640 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56049c612010 0x56049c614400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.099+0000 7ff48a800640 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56049c612010 0x56049c614400 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad44236173cfdd9a server_cookie=3d49807ad3fed6b8 in_seq=0 out_seq=0 2024-04-17T07:21:28.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.100+0000 7ff46be00640 10 client.17310 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:21:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.102+0000 7ff46be00640 1 -- 172.21.15.86:0/1318001846 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56049c56fa40 con 0x56049c612010 2024-04-17T07:21:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.102+0000 7ff46be00640 10 client.17310 handle_command_reply: tid=0 2024-04-17T07:21:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.102+0000 7ff494192740 1 client.17310 shutdown 2024-04-17T07:21:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.102+0000 7ff494192740 1 -- 172.21.15.86:0/1318001846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 msgr2=0x56049c606a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.102+0000 7ff494192740 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 0x56049c606a30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0ce3a0 tx=0x7ff47c09c730 comp rx=0 tx=0).stop 2024-04-17T07:21:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.103+0000 7ff494192740 1 -- 172.21.15.86:0/1318001846 shutdown_connections 2024-04-17T07:21:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.103+0000 7ff494192740 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56049c612010 0x56049c614400 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.103+0000 7ff494192740 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c601b00 0x56049c60da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.103+0000 7ff494192740 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 0x56049c606a30 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.103+0000 7ff494192740 1 --2- 172.21.15.86:0/1318001846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fbfa0 0x56049c6064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.103+0000 7ff494192740 1 -- 172.21.15.86:0/1318001846 >> 172.21.15.86:0/1318001846 conn(0x56049c5f7510 msgr2=0x56049c5f6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.104+0000 7ff46be00640 0 client.17310 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:21:28.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.104+0000 7ff494192740 1 -- 172.21.15.86:0/1318001846 shutdown_connections 2024-04-17T07:21:28.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.104+0000 7ff494192740 1 -- 172.21.15.86:0/1318001846 wait complete. 2024-04-17T07:21:28.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.104+0000 7ff494192740 20 client.17310 trim_cache size 0 max 16384 2024-04-17T07:21:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.135+0000 7ff494192740 1 Processor -- start 2024-04-17T07:21:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.135+0000 7ff494192740 1 -- start start 2024-04-17T07:21:28.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.136+0000 7ff494192740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5f7510 0x56049c5f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.136+0000 7ff494192740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fbfa0 0x56049c60da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.136+0000 7ff494192740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 0x56049c5fae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.136+0000 7ff494192740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1b9980 con 0x56049c5fc990 2024-04-17T07:21:28.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.136+0000 7ff494192740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1bb4c0 con 0x56049c5f7510 2024-04-17T07:21:28.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.136+0000 7ff494192740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1ba050 con 0x56049c5fbfa0 2024-04-17T07:21:28.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.137+0000 7ff48a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5f7510 0x56049c5f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.137+0000 7ff48b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 0x56049c5fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.137+0000 7ff48a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5f7510 0x56049c5f29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37114/0 (socket says 172.21.15.86:37114) 2024-04-17T07:21:28.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.137+0000 7ff48a800640 1 -- 172.21.15.86:0/3721130781 learned_addr learned my addr 172.21.15.86:0/3721130781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:28.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.137+0000 7ff489e00640 1 --2- 172.21.15.86:0/3721130781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fbfa0 0x56049c60da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.138+0000 7ff48a800640 1 -- 172.21.15.86:0/3721130781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fbfa0 msgr2=0x56049c60da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.138+0000 7ff48a800640 1 --2- 172.21.15.86:0/3721130781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fbfa0 0x56049c60da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.138+0000 7ff48a800640 1 -- 172.21.15.86:0/3721130781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 msgr2=0x56049c5fae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.138+0000 7ff48a800640 1 --2- 172.21.15.86:0/3721130781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 0x56049c5fae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.138+0000 7ff48a800640 1 -- 172.21.15.86:0/3721130781 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56049c2743f0 con 0x56049c5f7510 2024-04-17T07:21:28.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.138+0000 7ff48a800640 1 --2- 172.21.15.86:0/3721130781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5f7510 0x56049c5f29d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff478005ab0 tx=0x7ff478003930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37b145614fbfaf1c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.139+0000 7ff476a00640 1 -- 172.21.15.86:0/3721130781 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478014b80 con 0x56049c5f7510 2024-04-17T07:21:28.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.140+0000 7ff476a00640 1 -- 172.21.15.86:0/3721130781 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff478014d20 con 0x56049c5f7510 2024-04-17T07:21:28.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.140+0000 7ff476a00640 1 -- 172.21.15.86:0/3721130781 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478004e10 con 0x56049c5f7510 2024-04-17T07:21:28.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.141+0000 7ff494192740 1 -- 172.21.15.86:0/3721130781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5f7510 msgr2=0x56049c5f29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.141+0000 7ff494192740 1 --2- 172.21.15.86:0/3721130781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5f7510 0x56049c5f29d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff478005ab0 tx=0x7ff478003930 comp rx=0 tx=0).stop 2024-04-17T07:21:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.142+0000 7ff494192740 1 -- 172.21.15.86:0/3721130781 shutdown_connections 2024-04-17T07:21:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.142+0000 7ff494192740 1 --2- 172.21.15.86:0/3721130781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fc990 0x56049c5fae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.142+0000 7ff494192740 1 --2- 172.21.15.86:0/3721130781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5fbfa0 0x56049c60da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.142+0000 7ff494192740 1 --2- 172.21.15.86:0/3721130781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5f7510 0x56049c5f29d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.142+0000 7ff494192740 1 -- 172.21.15.86:0/3721130781 >> 172.21.15.86:0/3721130781 conn(0x56049c6060b0 msgr2=0x56049c5fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.142+0000 7ff494192740 1 -- 172.21.15.86:0/3721130781 shutdown_connections 2024-04-17T07:21:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.142+0000 7ff494192740 1 -- 172.21.15.86:0/3721130781 wait complete. 2024-04-17T07:21:28.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.143+0000 7ff494192740 1 -- start start 2024-04-17T07:21:28.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.144+0000 7ff494192740 1 Processor -- start 2024-04-17T07:21:28.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.144+0000 7ff494192740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5f7510 0x56049c60b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.144+0000 7ff494192740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fbfa0 0x56049c60b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.144+0000 7ff494192740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fc990 0x56049c60c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.145+0000 7ff494192740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1badf0 con 0x56049c5fbfa0 2024-04-17T07:21:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.145+0000 7ff48a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5f7510 0x56049c60b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.145+0000 7ff489e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fbfa0 0x56049c60b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.145+0000 7ff48a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5f7510 0x56049c60b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52452/0 (socket says 172.21.15.86:52452) 2024-04-17T07:21:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.145+0000 7ff48a800640 1 -- 172.21.15.86:0/1470896039 learned_addr learned my addr 172.21.15.86:0/1470896039 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.145+0000 7ff494192740 1 -- 172.21.15.86:0/1470896039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1ba720 con 0x56049c5fc990 2024-04-17T07:21:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.145+0000 7ff494192740 1 -- 172.21.15.86:0/1470896039 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c19e160 con 0x56049c5f7510 2024-04-17T07:21:28.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.146+0000 7ff48b200640 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fc990 0x56049c60c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.147+0000 7ff494192740 10 client.0 _ll_register_callbacks cb 0x7ff47c098e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.148+0000 7ff489e00640 1 -- 172.21.15.86:0/1470896039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5f7510 msgr2=0x56049c60b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.148+0000 7ff489e00640 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5f7510 0x56049c60b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.148+0000 7ff489e00640 1 -- 172.21.15.86:0/1470896039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fc990 msgr2=0x56049c60c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.148+0000 7ff489e00640 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fc990 0x56049c60c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.148+0000 7ff489e00640 1 -- 172.21.15.86:0/1470896039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56049c33e7d0 con 0x56049c5fbfa0 2024-04-17T07:21:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.149+0000 7ff489e00640 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fbfa0 0x56049c60b720 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff47c09ade0 tx=0x7ff47c09c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8c0b83c86148c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.149+0000 7ff46be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.149+0000 7ff46be00640 1 -- 172.21.15.86:0/1470896039 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c09de60 con 0x56049c5fbfa0 2024-04-17T07:21:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.149+0000 7ff46be00640 1 -- 172.21.15.86:0/1470896039 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff47c0a5310 con 0x56049c5fbfa0 2024-04-17T07:21:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.149+0000 7ff494192740 1 -- 172.21.15.86:0/1470896039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff478014d20 con 0x56049c5fbfa0 2024-04-17T07:21:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.150+0000 7ff46be00640 1 -- 172.21.15.86:0/1470896039 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff47c09cb30 con 0x56049c5fbfa0 2024-04-17T07:21:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.150+0000 7ff494192740 10 client.17316 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.150+0000 7ff494192740 1 -- 172.21.15.86:0/1470896039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56049c678520 con 0x56049c5fbfa0 2024-04-17T07:21:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.150+0000 7ff46be00640 1 -- 172.21.15.86:0/1470896039 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff478014d20 con 0x56049c5fbfa0 2024-04-17T07:21:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.151+0000 7ff46be00640 1 -- 172.21.15.86:0/1470896039 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff47c09b9f0 con 0x56049c5fbfa0 2024-04-17T07:21:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.151+0000 7ff494192740 10 client.17316 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.151+0000 7ff494192740 10 client.17316 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:21:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.151+0000 7ff494192740 20 client.17316 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.151+0000 7ff494192740 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56049c601b00 0x56049c67ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.151+0000 7ff494192740 4 client.17316 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:21:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.151+0000 7ff494192740 1 -- 172.21.15.86:0/1470896039 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56049c5698b0 con 0x56049c601b00 2024-04-17T07:21:28.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.152+0000 7ff48a800640 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56049c601b00 0x56049c67ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.153+0000 7ff48a800640 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56049c601b00 0x56049c67ac20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3605d2e4989be69 server_cookie=1c944b50a5689762 in_seq=0 out_seq=0 2024-04-17T07:21:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.153+0000 7ff46be00640 10 client.17316 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:21:28.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.154+0000 7ff46be00640 1 -- 172.21.15.86:0/1470896039 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56049c5698b0 con 0x56049c601b00 2024-04-17T07:21:28.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.154+0000 7ff46be00640 10 client.17316 handle_command_reply: tid=0 2024-04-17T07:21:28.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.154+0000 7ff494192740 1 client.17316 shutdown 2024-04-17T07:21:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.155+0000 7ff494192740 1 -- 172.21.15.86:0/1470896039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fbfa0 msgr2=0x56049c60b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.155+0000 7ff494192740 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fbfa0 0x56049c60b720 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff47c09ade0 tx=0x7ff47c09c7a0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.155+0000 7ff494192740 1 -- 172.21.15.86:0/1470896039 shutdown_connections 2024-04-17T07:21:28.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.155+0000 7ff494192740 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56049c601b00 0x56049c67ac20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.155+0000 7ff494192740 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049c5fc990 0x56049c60c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.155+0000 7ff494192740 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049c5fbfa0 0x56049c60b720 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.155+0000 7ff494192740 1 --2- 172.21.15.86:0/1470896039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56049c5f7510 0x56049c60b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.155+0000 7ff494192740 1 -- 172.21.15.86:0/1470896039 >> 172.21.15.86:0/1470896039 conn(0x56049c6060b0 msgr2=0x56049c5fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.155+0000 7ff46be00640 0 client.17316 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:21:28.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.156+0000 7ff494192740 1 -- 172.21.15.86:0/1470896039 shutdown_connections 2024-04-17T07:21:28.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.156+0000 7ff494192740 1 -- 172.21.15.86:0/1470896039 wait complete. 2024-04-17T07:21:28.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.156+0000 7ff494192740 20 client.17316 trim_cache size 0 max 16384 2024-04-17T07:21:28.157 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:21:28.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.157+0000 7ff491400640 1 -- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff4700a3c30 msgr2=0x7ff4700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.157+0000 7ff491400640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff4700a3c30 0x7ff4700a60f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff4840306a0 tx=0x7ff484005bd0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.157+0000 7ff491400640 1 -- 172.21.15.86:0/3385160274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c105350 msgr2=0x7ff48c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.157+0000 7ff491400640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c105350 0x7ff48c19ed20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00e9c0 tx=0x7ff47c00ee80 comp rx=0 tx=0).stop 2024-04-17T07:21:28.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.158+0000 7ff491400640 1 -- 172.21.15.86:0/3385160274 shutdown_connections 2024-04-17T07:21:28.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.158+0000 7ff491400640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff4700a3c30 0x7ff4700a60f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.158+0000 7ff491400640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff48c107c80 0x7ff48c1a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.158+0000 7ff491400640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c105350 0x7ff48c19ed20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.158+0000 7ff491400640 1 --2- 172.21.15.86:0/3385160274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff48c102a20 0x7ff48c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.158+0000 7ff491400640 1 -- 172.21.15.86:0/3385160274 >> 172.21.15.86:0/3385160274 conn(0x7ff48c0fc410 msgr2=0x7ff48c103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.158+0000 7ff491400640 1 -- 172.21.15.86:0/3385160274 shutdown_connections 2024-04-17T07:21:28.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.158+0000 7ff491400640 1 -- 172.21.15.86:0/3385160274 wait complete. 2024-04-17T07:21:28.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:21:28.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.458+0000 7f8ffe800640 1 -- 172.21.15.86:0/1549139249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81107e0 msgr2=0x7f8ff8112bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.458+0000 7f8ffe800640 1 --2- 172.21.15.86:0/1549139249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81107e0 0x7f8ff8112bd0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009c10 tx=0x7f8ff002bee0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.459+0000 7f8ffe800640 1 -- 172.21.15.86:0/1549139249 shutdown_connections 2024-04-17T07:21:28.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.459+0000 7f8ffe800640 1 --2- 172.21.15.86:0/1549139249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81107e0 0x7f8ff8112bd0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.459+0000 7f8ffe800640 1 --2- 172.21.15.86:0/1549139249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff81098c0 0x7f8ff81102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.459+0000 7f8ffe800640 1 --2- 172.21.15.86:0/1549139249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ff8108f60 0x7f8ff8109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.459+0000 7f8ffe800640 1 -- 172.21.15.86:0/1549139249 >> 172.21.15.86:0/1549139249 conn(0x7f8ff806f190 msgr2=0x7f8ff80715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.459+0000 7f8ffe800640 1 -- 172.21.15.86:0/1549139249 shutdown_connections 2024-04-17T07:21:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.459+0000 7f8ffe800640 1 -- 172.21.15.86:0/1549139249 wait complete. 2024-04-17T07:21:28.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.460+0000 7f8ffe800640 1 Processor -- start 2024-04-17T07:21:28.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.461+0000 7f8ffe800640 1 -- start start 2024-04-17T07:21:28.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.461+0000 7f8ffe800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ff8108f60 0x7f8ff810fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.461+0000 7f8ffe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81098c0 0x7f8ff81100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.461+0000 7f8ffe800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff81107e0 0x7f8ff810c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.461+0000 7f8ffe800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8116310 con 0x7f8ff81098c0 2024-04-17T07:21:28.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.461+0000 7f8ffe800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8116190 con 0x7f8ff8108f60 2024-04-17T07:21:28.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.461+0000 7f8ffe800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8116490 con 0x7f8ff81107e0 2024-04-17T07:21:28.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.462+0000 7f8ff7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ff8108f60 0x7f8ff810fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.462+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff81107e0 0x7f8ff810c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.462+0000 7f8ff7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ff8108f60 0x7f8ff810fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37138/0 (socket says 172.21.15.86:37138) 2024-04-17T07:21:28.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.462+0000 7f8ff7400640 1 -- 172.21.15.86:0/3934157565 learned_addr learned my addr 172.21.15.86:0/3934157565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:28.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.462+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81098c0 0x7f8ff81100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.463+0000 7f8ff7400640 1 -- 172.21.15.86:0/3934157565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff81107e0 msgr2=0x7f8ff810c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.463+0000 7f8ff7400640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff81107e0 0x7f8ff810c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.463+0000 7f8ff7400640 1 -- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81098c0 msgr2=0x7f8ff81100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.463+0000 7f8ff7400640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81098c0 0x7f8ff81100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.463+0000 7f8ff7400640 1 -- 172.21.15.86:0/3934157565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff810cfd0 con 0x7f8ff8108f60 2024-04-17T07:21:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.463+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81098c0 0x7f8ff81100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:21:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.464+0000 7f8ff7400640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ff8108f60 0x7f8ff810fba0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800e9c0 tx=0x7f8fe800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.464+0000 7f8fefe00640 1 -- 172.21.15.86:0/3934157565 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800cde0 con 0x7f8ff8108f60 2024-04-17T07:21:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.464+0000 7f8ffe800640 1 -- 172.21.15.86:0/3934157565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff81a2880 con 0x7f8ff8108f60 2024-04-17T07:21:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.465+0000 7f8ffe800640 1 -- 172.21.15.86:0/3934157565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff81a2e20 con 0x7f8ff8108f60 2024-04-17T07:21:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.465+0000 7f8fefe00640 1 -- 172.21.15.86:0/3934157565 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fe80035f0 con 0x7f8ff8108f60 2024-04-17T07:21:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.465+0000 7f8fefe00640 1 -- 172.21.15.86:0/3934157565 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8010690 con 0x7f8ff8108f60 2024-04-17T07:21:28.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.468+0000 7f9001590740 1 Processor -- start 2024-04-17T07:21:28.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.468+0000 7f9001590740 1 -- start start 2024-04-17T07:21:28.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.469+0000 7f9001590740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c2614fa0 0x5610c26153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.469+0000 7f9001590740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c2615990 0x5610c261a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.469+0000 7f9001590740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c261ab00 0x5610c261cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.469+0000 7f9001590740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5610c257b020 con 0x5610c2615990 2024-04-17T07:21:28.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.469+0000 7f9001590740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5610c2521460 con 0x5610c261ab00 2024-04-17T07:21:28.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.469+0000 7f9001590740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5610c2576fe0 con 0x5610c2614fa0 2024-04-17T07:21:28.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.469+0000 7f8ff6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c261ab00 0x5610c261cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.470+0000 7f8ff6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c261ab00 0x5610c261cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37140/0 (socket says 172.21.15.86:37140) 2024-04-17T07:21:28.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.470+0000 7f8ff6a00640 1 -- 172.21.15.86:0/1422380245 learned_addr learned my addr 172.21.15.86:0/1422380245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:28.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.470+0000 7f8ff7e00640 1 --2- 172.21.15.86:0/1422380245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c2614fa0 0x5610c26153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.470+0000 7f8ff6a00640 1 -- 172.21.15.86:0/1422380245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c2614fa0 msgr2=0x5610c26153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.470+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/1422380245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c2614fa0 0x5610c26153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.470+0000 7f8ff6a00640 1 -- 172.21.15.86:0/1422380245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c2615990 msgr2=0x5610c261a490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:21:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.470+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/1422380245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c2615990 0x5610c261a490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.470+0000 7f8ff6a00640 1 -- 172.21.15.86:0/1422380245 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c2529040 con 0x5610c261ab00 2024-04-17T07:21:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.471+0000 7f8ff7e00640 1 --2- 172.21.15.86:0/1422380245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c2614fa0 0x5610c26153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:21:28.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.471+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/1422380245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c261ab00 0x5610c261cf90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009c80 tx=0x7f8fe40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15d2b78a22d503ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.472+0000 7f8fecc00640 1 -- 172.21.15.86:0/1422380245 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4005020 con 0x5610c261ab00 2024-04-17T07:21:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.472+0000 7f8fecc00640 1 -- 172.21.15.86:0/1422380245 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fe40047a0 con 0x5610c261ab00 2024-04-17T07:21:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.472+0000 7f8fecc00640 1 -- 172.21.15.86:0/1422380245 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40126b0 con 0x5610c261ab00 2024-04-17T07:21:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.474+0000 7f9001590740 1 -- 172.21.15.86:0/1422380245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c261ab00 msgr2=0x5610c261cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.474+0000 7f9001590740 1 --2- 172.21.15.86:0/1422380245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c261ab00 0x5610c261cf90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009c80 tx=0x7f8fe40117b0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.474+0000 7f9001590740 1 -- 172.21.15.86:0/1422380245 shutdown_connections 2024-04-17T07:21:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.474+0000 7f9001590740 1 --2- 172.21.15.86:0/1422380245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c261ab00 0x5610c261cf90 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009c80 tx=0x7f8fe40117b0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.474+0000 7f9001590740 1 --2- 172.21.15.86:0/1422380245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c2615990 0x5610c261a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.474+0000 7f9001590740 1 --2- 172.21.15.86:0/1422380245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c2614fa0 0x5610c26153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.474+0000 7f9001590740 1 -- 172.21.15.86:0/1422380245 >> 172.21.15.86:0/1422380245 conn(0x5610c2610510 msgr2=0x5610c2610910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.475+0000 7f8fefe00640 1 -- 172.21.15.86:0/3934157565 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8fe8004020 con 0x7f8ff8108f60 2024-04-17T07:21:28.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.475+0000 7f9001590740 1 -- 172.21.15.86:0/1422380245 shutdown_connections 2024-04-17T07:21:28.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.475+0000 7f9001590740 1 -- 172.21.15.86:0/1422380245 wait complete. 2024-04-17T07:21:28.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.475+0000 7f9001590740 1 -- start start 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.476+0000 7f9001590740 1 Processor -- start 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.476+0000 7f9001590740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c261f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.476+0000 7f9001590740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c261fd40 0x5610c2624d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.476+0000 7f9001590740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c2627300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.476+0000 7f9001590740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5610c261dab0 con 0x5610c26201c0 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.476+0000 7f9001590740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5610c2534830 con 0x5610c2614fa0 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.476+0000 7f9001590740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5610c261dc30 con 0x5610c261fd40 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.476+0000 7f9001590740 10 client.0 _ll_register_callbacks cb 0x5610c2502fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.477+0000 7f8ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c2627300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.477+0000 7f8ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c2627300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36782/0 (socket says 172.21.15.86:36782) 2024-04-17T07:21:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.477+0000 7f8ff6a00640 1 -- 172.21.15.86:0/3760291284 learned_addr learned my addr 172.21.15.86:0/3760291284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:28.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.477+0000 7f8ff7e00640 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c261f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.478+0000 7f8ff7400640 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c261fd40 0x5610c2624d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.478+0000 7f8ff6a00640 1 -- 172.21.15.86:0/3760291284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c261fd40 msgr2=0x5610c2624d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.478+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c261fd40 0x5610c2624d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.478+0000 7f8ff6a00640 1 -- 172.21.15.86:0/3760291284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 msgr2=0x5610c261f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.478+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c261f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.478+0000 7f8ff6a00640 1 -- 172.21.15.86:0/3760291284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c2538920 con 0x5610c26201c0 2024-04-17T07:21:28.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.478+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c2627300 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4000c00 tx=0x7f8fe4009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f198806ee3a408 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.478+0000 7f8fe1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:28.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.479+0000 7f8fefe00640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fdc0a3bc0 0x7f8fdc0a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.479+0000 7f8fefe00640 1 -- 172.21.15.86:0/3934157565 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8fe80d0b10 con 0x7f8ff8108f60 2024-04-17T07:21:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.479+0000 7f8fe1600640 1 -- 172.21.15.86:0/3760291284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4004250 con 0x5610c26201c0 2024-04-17T07:21:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.479+0000 7f8fe1600640 1 -- 172.21.15.86:0/3760291284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8fe401f070 con 0x5610c26201c0 2024-04-17T07:21:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.479+0000 7f8ff7e00640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fdc0a3bc0 0x7f8fdc0a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.479+0000 7f9001590740 1 -- 172.21.15.86:0/3760291284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe40047a0 con 0x5610c26201c0 2024-04-17T07:21:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.480+0000 7f8fe1600640 1 -- 172.21.15.86:0/3760291284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fe40043f0 con 0x5610c26201c0 2024-04-17T07:21:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.480+0000 7f9001590740 10 client.17325 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.480+0000 7f9001590740 1 -- 172.21.15.86:0/3760291284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610c25067f0 con 0x5610c26201c0 2024-04-17T07:21:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.481+0000 7f8fe1600640 1 -- 172.21.15.86:0/3760291284 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fe40047a0 con 0x5610c26201c0 2024-04-17T07:21:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.481+0000 7f8fe1600640 1 -- 172.21.15.86:0/3760291284 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8fe4003b40 con 0x5610c26201c0 2024-04-17T07:21:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.481+0000 7f8ff7e00640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fdc0a3bc0 0x7f8fdc0a6080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8ff002c4f0 tx=0x7f8ff0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.482+0000 7f9001590740 10 client.17325 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.482+0000 7f9001590740 10 client.17325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:21:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.482+0000 7f9001590740 20 client.17325 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.482+0000 7f9001590740 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5610c262bc50 0x5610c262e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.482+0000 7f9001590740 4 client.17325 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:21:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.482+0000 7f9001590740 1 -- 172.21.15.86:0/3760291284 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610c2588a40 con 0x5610c262bc50 2024-04-17T07:21:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.482+0000 7f8ff7400640 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5610c262bc50 0x5610c262e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.484+0000 7f8fe1600640 10 client.17325 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.484+0000 7f8ff7400640 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5610c262bc50 0x5610c262e070 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232f590e66428f6f server_cookie=65b5f986054b634f in_seq=0 out_seq=0 2024-04-17T07:21:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f8fe1600640 1 -- 172.21.15.86:0/3760291284 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610c2588a40 con 0x5610c262bc50 2024-04-17T07:21:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f8fe1600640 10 client.17325 handle_command_reply: tid=0 2024-04-17T07:21:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f9001590740 1 client.17325 shutdown 2024-04-17T07:21:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f9001590740 1 -- 172.21.15.86:0/3760291284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 msgr2=0x5610c2627300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f9001590740 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c2627300 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4000c00 tx=0x7f8fe4009e80 comp rx=0 tx=0).stop 2024-04-17T07:21:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f9001590740 1 -- 172.21.15.86:0/3760291284 shutdown_connections 2024-04-17T07:21:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f9001590740 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5610c262bc50 0x5610c262e070 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f8fe1600640 0 client.17325 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f9001590740 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c2627300 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f9001590740 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c261fd40 0x5610c2624d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f9001590740 1 --2- 172.21.15.86:0/3760291284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c261f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f9001590740 1 -- 172.21.15.86:0/3760291284 >> 172.21.15.86:0/3760291284 conn(0x5610c2610510 msgr2=0x5610c260f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.486+0000 7f9001590740 1 -- 172.21.15.86:0/3760291284 shutdown_connections 2024-04-17T07:21:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.487+0000 7f9001590740 1 -- 172.21.15.86:0/3760291284 wait complete. 2024-04-17T07:21:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.487+0000 7f9001590740 20 client.17325 trim_cache size 0 max 16384 2024-04-17T07:21:28.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.509+0000 7f9001590740 1 Processor -- start 2024-04-17T07:21:28.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.509+0000 7f9001590740 1 -- start start 2024-04-17T07:21:28.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.510+0000 7f9001590740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c260ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.510+0000 7f9001590740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c261fd40 0x5610c260f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.510+0000 7f9001590740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c26201c0 0x5610c2626d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.511+0000 7f8ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c261fd40 0x5610c260f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.511+0000 7f8ff7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c260ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.511+0000 7f9001590740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd800fc00 con 0x5610c261fd40 2024-04-17T07:21:28.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.511+0000 7f8ff7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c260ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37148/0 (socket says 172.21.15.86:37148) 2024-04-17T07:21:28.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.511+0000 7f8ff7400640 1 -- 172.21.15.86:0/1942952889 learned_addr learned my addr 172.21.15.86:0/1942952889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:28.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.511+0000 7f9001590740 1 -- 172.21.15.86:0/1942952889 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8011740 con 0x5610c2614fa0 2024-04-17T07:21:28.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.512+0000 7f9001590740 1 -- 172.21.15.86:0/1942952889 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd80102d0 con 0x5610c26201c0 2024-04-17T07:21:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.512+0000 7f8ff7e00640 1 --2- 172.21.15.86:0/1942952889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c26201c0 0x5610c2626d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.512+0000 7f8ff6a00640 1 -- 172.21.15.86:0/1942952889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c26201c0 msgr2=0x5610c2626d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.512+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/1942952889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c26201c0 0x5610c2626d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.512+0000 7f8ff6a00640 1 -- 172.21.15.86:0/1942952889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 msgr2=0x5610c260ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.512+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/1942952889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c260ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.512+0000 7f8ff6a00640 1 -- 172.21.15.86:0/1942952889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c261fb70 con 0x5610c261fd40 2024-04-17T07:21:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.513+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/1942952889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c261fd40 0x5610c260f3c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4012540 tx=0x7f8fe4009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89673c068b9105dd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.513+0000 7f8fe3e00640 1 -- 172.21.15.86:0/1942952889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4004250 con 0x5610c261fd40 2024-04-17T07:21:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.513+0000 7f8fe3e00640 1 -- 172.21.15.86:0/1942952889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fe4012740 con 0x5610c261fd40 2024-04-17T07:21:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.513+0000 7f8fe3e00640 1 -- 172.21.15.86:0/1942952889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4004f90 con 0x5610c261fd40 2024-04-17T07:21:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.514+0000 7f9001590740 1 -- 172.21.15.86:0/1942952889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c261fd40 msgr2=0x5610c260f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.514+0000 7f9001590740 1 --2- 172.21.15.86:0/1942952889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c261fd40 0x5610c260f3c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4012540 tx=0x7f8fe4009970 comp rx=0 tx=0).stop 2024-04-17T07:21:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.515+0000 7f9001590740 1 -- 172.21.15.86:0/1942952889 shutdown_connections 2024-04-17T07:21:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.515+0000 7f9001590740 1 --2- 172.21.15.86:0/1942952889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c26201c0 0x5610c2626d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.515+0000 7f9001590740 1 --2- 172.21.15.86:0/1942952889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c261fd40 0x5610c260f3c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.515+0000 7f9001590740 1 --2- 172.21.15.86:0/1942952889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c260ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.515+0000 7f9001590740 1 -- 172.21.15.86:0/1942952889 >> 172.21.15.86:0/1942952889 conn(0x5610c2610510 msgr2=0x5610c26171b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.515+0000 7f9001590740 1 -- 172.21.15.86:0/1942952889 shutdown_connections 2024-04-17T07:21:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.515+0000 7f9001590740 1 -- 172.21.15.86:0/1942952889 wait complete. 2024-04-17T07:21:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.515+0000 7f9001590740 1 -- start start 2024-04-17T07:21:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.516+0000 7f9001590740 1 Processor -- start 2024-04-17T07:21:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.516+0000 7f9001590740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c2627210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.516+0000 7f9001590740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c261fd40 0x5610c2623cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.517+0000 7f9001590740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c26241f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.517+0000 7f9001590740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8011070 con 0x5610c26201c0 2024-04-17T07:21:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.517+0000 7f9001590740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd80109a0 con 0x5610c2614fa0 2024-04-17T07:21:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.517+0000 7f9001590740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd800dbd0 con 0x5610c261fd40 2024-04-17T07:21:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.517+0000 7f9001590740 10 client.0 _ll_register_callbacks cb 0x7f8fe4004da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.517+0000 7f8ff7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c2627210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.517+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c26241f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.517+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c26241f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36804/0 (socket says 172.21.15.86:36804) 2024-04-17T07:21:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.517+0000 7f8ff7e00640 1 -- 172.21.15.86:0/2749128730 learned_addr learned my addr 172.21.15.86:0/2749128730 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.517+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c261fd40 0x5610c2623cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.518+0000 7f8ff7400640 1 -- 172.21.15.86:0/2749128730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c261fd40 msgr2=0x5610c2623cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.518+0000 7f8ff7400640 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c261fd40 0x5610c2623cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.518+0000 7f8ff7400640 1 -- 172.21.15.86:0/2749128730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 msgr2=0x5610c26241f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.518+0000 7f8ff7400640 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c26241f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.518+0000 7f8ff7400640 1 -- 172.21.15.86:0/2749128730 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c228d3f0 con 0x5610c2614fa0 2024-04-17T07:21:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.518+0000 7f8ff7e00640 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c26241f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:21:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.519+0000 7f8ff7400640 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c2627210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8098e70 tx=0x7f8fe809d1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91e1b89facbb5dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.519+0000 7f8fe1600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:21:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.519+0000 7f8fe1600640 1 -- 172.21.15.86:0/2749128730 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80a5680 con 0x5610c2614fa0 2024-04-17T07:21:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.519+0000 7f8fe1600640 1 -- 172.21.15.86:0/2749128730 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8fe80094a0 con 0x5610c2614fa0 2024-04-17T07:21:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.520+0000 7f9001590740 1 -- 172.21.15.86:0/2749128730 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe4012740 con 0x5610c2614fa0 2024-04-17T07:21:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.520+0000 7f8fe1600640 1 -- 172.21.15.86:0/2749128730 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fe809ad10 con 0x5610c2614fa0 2024-04-17T07:21:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.520+0000 7f9001590740 10 client.36332 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.520+0000 7f9001590740 1 -- 172.21.15.86:0/2749128730 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610c23577d0 con 0x5610c2614fa0 2024-04-17T07:21:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.521+0000 7f8fe1600640 1 -- 172.21.15.86:0/2749128730 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fe4012740 con 0x5610c2614fa0 2024-04-17T07:21:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.521+0000 7f8fe1600640 1 -- 172.21.15.86:0/2749128730 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8fe8098760 con 0x5610c2614fa0 2024-04-17T07:21:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.521+0000 7f9001590740 10 client.36332 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.521+0000 7f9001590740 10 client.36332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:21:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.521+0000 7f9001590740 20 client.36332 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.521+0000 7f9001590740 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5610c262bc50 0x5610c2693b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.521+0000 7f9001590740 4 client.36332 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:21:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.521+0000 7f9001590740 1 -- 172.21.15.86:0/2749128730 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610c25828b0 con 0x5610c262bc50 2024-04-17T07:21:28.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.523+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5610c262bc50 0x5610c2693b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.525+0000 7f8fe1600640 10 client.36332 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:28.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.525+0000 7f8ff6a00640 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5610c262bc50 0x5610c2693b00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20195ca2c1bc92b2 server_cookie=d85629582013ab7b in_seq=0 out_seq=0 2024-04-17T07:21:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.525+0000 7f8fe1600640 1 -- 172.21.15.86:0/2749128730 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610c25828b0 con 0x5610c262bc50 2024-04-17T07:21:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.525+0000 7f8fe1600640 10 client.36332 handle_command_reply: tid=0 2024-04-17T07:21:28.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.525+0000 7f9001590740 1 client.36332 shutdown 2024-04-17T07:21:28.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.525+0000 7f9001590740 1 -- 172.21.15.86:0/2749128730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 msgr2=0x5610c2627210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f9001590740 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c2627210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8098e70 tx=0x7f8fe809d1d0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f9001590740 1 -- 172.21.15.86:0/2749128730 shutdown_connections 2024-04-17T07:21:28.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f9001590740 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5610c262bc50 0x5610c2693b00 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f8fe1600640 0 client.36332 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f9001590740 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610c26201c0 0x5610c26241f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f9001590740 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610c261fd40 0x5610c2623cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f9001590740 1 --2- 172.21.15.86:0/2749128730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610c2614fa0 0x5610c2627210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f9001590740 1 -- 172.21.15.86:0/2749128730 >> 172.21.15.86:0/2749128730 conn(0x5610c2610510 msgr2=0x5610c260ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f9001590740 1 -- 172.21.15.86:0/2749128730 shutdown_connections 2024-04-17T07:21:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f9001590740 1 -- 172.21.15.86:0/2749128730 wait complete. 2024-04-17T07:21:28.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.526+0000 7f9001590740 20 client.36332 trim_cache size 0 max 16384 2024-04-17T07:21:28.527 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:21:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.528+0000 7f8ffe800640 1 -- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fdc0a3bc0 msgr2=0x7f8fdc0a6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.528+0000 7f8ffe800640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fdc0a3bc0 0x7f8fdc0a6080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8ff002c4f0 tx=0x7f8ff0004a40 comp rx=0 tx=0).stop 2024-04-17T07:21:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.528+0000 7f8ffe800640 1 -- 172.21.15.86:0/3934157565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ff8108f60 msgr2=0x7f8ff810fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.528+0000 7f8ffe800640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ff8108f60 0x7f8ff810fba0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800e9c0 tx=0x7f8fe800ee80 comp rx=0 tx=0).stop 2024-04-17T07:21:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.529+0000 7f8ffe800640 1 -- 172.21.15.86:0/3934157565 shutdown_connections 2024-04-17T07:21:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.529+0000 7f8ffe800640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8fdc0a3bc0 0x7f8fdc0a6080 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.529+0000 7f8ffe800640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff81107e0 0x7f8ff810c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.529+0000 7f8ffe800640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff81098c0 0x7f8ff81100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.529+0000 7f8ffe800640 1 --2- 172.21.15.86:0/3934157565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ff8108f60 0x7f8ff810fba0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.529+0000 7f8ffe800640 1 -- 172.21.15.86:0/3934157565 >> 172.21.15.86:0/3934157565 conn(0x7f8ff806f190 msgr2=0x7f8ff8071580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.529+0000 7f8ffe800640 1 -- 172.21.15.86:0/3934157565 shutdown_connections 2024-04-17T07:21:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:28.529+0000 7f8ffe800640 1 -- 172.21.15.86:0/3934157565 wait complete. 2024-04-17T07:21:28.552 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:28 smithi086 ceph-mon[31733]: pgmap v653: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 17 MiB/s wr, 744 op/s 2024-04-17T07:21:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:28 smithi098 ceph-mon[35418]: pgmap v653: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 17 MiB/s wr, 744 op/s 2024-04-17T07:21:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:28 smithi103 ceph-mon[35495]: pgmap v653: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 17 MiB/s wr, 744 op/s 2024-04-17T07:21:29.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 125.0 s, 737.6 tps, lat 163.833 ms stddev 278.293 2024-04-17T07:21:29.556 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:21:29.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:21:29.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:21:29.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.850+0000 7fcb26800640 1 -- 172.21.15.86:0/2902641516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb201039a0 msgr2=0x7fcb2010a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:29.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.850+0000 7fcb26800640 1 --2- 172.21.15.86:0/2902641516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb201039a0 0x7fcb2010a2c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009be0 tx=0x7fcb1002bef0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.851+0000 7fcb26800640 1 -- 172.21.15.86:0/2902641516 shutdown_connections 2024-04-17T07:21:29.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.851+0000 7fcb26800640 1 --2- 172.21.15.86:0/2902641516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb201039a0 0x7fcb2010a2c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.851+0000 7fcb26800640 1 --2- 172.21.15.86:0/2902641516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb20102fe0 0x7fcb20103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.851+0000 7fcb26800640 1 --2- 172.21.15.86:0/2902641516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20101e90 0x7fcb201022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.851+0000 7fcb26800640 1 -- 172.21.15.86:0/2902641516 >> 172.21.15.86:0/2902641516 conn(0x7fcb200fd400 msgr2=0x7fcb200ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:29.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.851+0000 7fcb26800640 1 -- 172.21.15.86:0/2902641516 shutdown_connections 2024-04-17T07:21:29.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.851+0000 7fcb26800640 1 -- 172.21.15.86:0/2902641516 wait complete. 2024-04-17T07:21:29.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.852+0000 7fcb26800640 1 Processor -- start 2024-04-17T07:21:29.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.853+0000 7fcb26800640 1 -- start start 2024-04-17T07:21:29.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.853+0000 7fcb26800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20101e90 0x7fcb20075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.853+0000 7fcb26800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20102fe0 0x7fcb20074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.853+0000 7fcb26800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb201039a0 0x7fcb20074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.853+0000 7fcb26800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2010d9d0 con 0x7fcb201039a0 2024-04-17T07:21:29.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.853+0000 7fcb26800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2010d850 con 0x7fcb20101e90 2024-04-17T07:21:29.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.853+0000 7fcb26800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2010db50 con 0x7fcb20102fe0 2024-04-17T07:21:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.854+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20102fe0 0x7fcb20074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.854+0000 7fcb25400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20101e90 0x7fcb20075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.854+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20102fe0 0x7fcb20074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52506/0 (socket says 172.21.15.86:52506) 2024-04-17T07:21:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.854+0000 7fcb24a00640 1 -- 172.21.15.86:0/2915867198 learned_addr learned my addr 172.21.15.86:0/2915867198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.854+0000 7fcb25400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20101e90 0x7fcb20075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37172/0 (socket says 172.21.15.86:37172) 2024-04-17T07:21:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.854+0000 7fcb25e00640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb201039a0 0x7fcb20074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.855+0000 7fcb25400640 1 -- 172.21.15.86:0/2915867198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20102fe0 msgr2=0x7fcb20074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.855+0000 7fcb25400640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20102fe0 0x7fcb20074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.855+0000 7fcb25400640 1 -- 172.21.15.86:0/2915867198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb201039a0 msgr2=0x7fcb20074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.855+0000 7fcb25400640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb201039a0 0x7fcb20074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.855+0000 7fcb25400640 1 -- 172.21.15.86:0/2915867198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb20074dd0 con 0x7fcb20101e90 2024-04-17T07:21:29.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.855+0000 7fcb24a00640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20102fe0 0x7fcb20074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:21:29.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.856+0000 7fcb25400640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20101e90 0x7fcb20075d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9f0 tx=0x7fcb1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:29.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.856+0000 7fcb1e000640 1 -- 172.21.15.86:0/2915867198 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800cc40 con 0x7fcb20101e90 2024-04-17T07:21:29.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.856+0000 7fcb26800640 1 -- 172.21.15.86:0/2915867198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb200750c0 con 0x7fcb20101e90 2024-04-17T07:21:29.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.856+0000 7fcb1e000640 1 -- 172.21.15.86:0/2915867198 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb1800cde0 con 0x7fcb20101e90 2024-04-17T07:21:29.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.856+0000 7fcb1e000640 1 -- 172.21.15.86:0/2915867198 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18018640 con 0x7fcb20101e90 2024-04-17T07:21:29.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.856+0000 7fcb26800640 1 -- 172.21.15.86:0/2915867198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb201a6b00 con 0x7fcb20101e90 2024-04-17T07:21:29.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.861+0000 7fcb1e000640 1 -- 172.21.15.86:0/2915867198 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fcb18010a00 con 0x7fcb20101e90 2024-04-17T07:21:29.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.861+0000 7fcb2ac00740 1 Processor -- start 2024-04-17T07:21:29.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.862+0000 7fcb2ac00740 1 -- start start 2024-04-17T07:21:29.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.863+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x56456294f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.863+0000 7fcb1e000640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3d00 0x7fcb040a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.863+0000 7fcb1e000640 1 -- 172.21.15.86:0/2915867198 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcb180d0a30 con 0x7fcb20101e90 2024-04-17T07:21:29.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.864+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562954590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.864+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564562954c00 0x564562957090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.864+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5645628b4cb0 con 0x56456294fa90 2024-04-17T07:21:29.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.864+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56456285b370 con 0x564562954c00 2024-04-17T07:21:29.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.864+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5645628b0c70 con 0x56456294f0a0 2024-04-17T07:21:29.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.864+0000 7fcb25400640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3d00 0x7fcb040a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.864+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562954590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.864+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562954590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36844/0 (socket says 172.21.15.86:36844) 2024-04-17T07:21:29.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.864+0000 7fcb24a00640 1 -- 172.21.15.86:0/3240977202 learned_addr learned my addr 172.21.15.86:0/3240977202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.865+0000 7fcb24a00640 1 -- 172.21.15.86:0/3240977202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 msgr2=0x56456294f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.865+0000 7fcb24a00640 1 --2- 172.21.15.86:0/3240977202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x56456294f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.865+0000 7fcb24a00640 1 -- 172.21.15.86:0/3240977202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564562954c00 msgr2=0x564562957090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:21:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.865+0000 7fcb24a00640 1 --2- 172.21.15.86:0/3240977202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564562954c00 0x564562957090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.865+0000 7fcb24a00640 1 -- 172.21.15.86:0/3240977202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56456287a910 con 0x56456294fa90 2024-04-17T07:21:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.865+0000 7fcb25400640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3d00 0x7fcb040a61c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcb180986c0 tx=0x7fcb18009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.865+0000 7fcb24a00640 1 --2- 172.21.15.86:0/3240977202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562954590 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0115f0 tx=0x7fcb0c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c475365cc580bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.865+0000 7fcb16600640 1 -- 172.21.15.86:0/3240977202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c005020 con 0x56456294fa90 2024-04-17T07:21:29.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.866+0000 7fcb16600640 1 -- 172.21.15.86:0/3240977202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c0047a0 con 0x56456294fa90 2024-04-17T07:21:29.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.866+0000 7fcb16600640 1 -- 172.21.15.86:0/3240977202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00f820 con 0x56456294fa90 2024-04-17T07:21:29.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.866+0000 7fcb2ac00740 1 -- 172.21.15.86:0/3240977202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 msgr2=0x564562954590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:29.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.866+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/3240977202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562954590 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0115f0 tx=0x7fcb0c011ab0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.867+0000 7fcb2ac00740 1 -- 172.21.15.86:0/3240977202 shutdown_connections 2024-04-17T07:21:29.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.867+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/3240977202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564562954c00 0x564562957090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.867+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/3240977202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562954590 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.867+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/3240977202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x56456294f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.867+0000 7fcb2ac00740 1 -- 172.21.15.86:0/3240977202 >> 172.21.15.86:0/3240977202 conn(0x56456294a610 msgr2=0x56456294aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:29.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.867+0000 7fcb2ac00740 1 -- 172.21.15.86:0/3240977202 shutdown_connections 2024-04-17T07:21:29.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.867+0000 7fcb2ac00740 1 -- 172.21.15.86:0/3240977202 wait complete. 2024-04-17T07:21:29.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.867+0000 7fcb2ac00740 1 -- start start 2024-04-17T07:21:29.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.868+0000 7fcb2ac00740 1 Processor -- start 2024-04-17T07:21:29.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.869+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294f0a0 0x5645629595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.869+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562959b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.869+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564562954c00 0x564562960b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.869+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564562957bb0 con 0x56456294fa90 2024-04-17T07:21:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.869+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56456286e730 con 0x56456294f0a0 2024-04-17T07:21:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.869+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564562957d30 con 0x564562954c00 2024-04-17T07:21:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.869+0000 7fcb2ac00740 10 client.0 _ll_register_callbacks cb 0x564562870b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.870+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562959b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.870+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562959b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36860/0 (socket says 172.21.15.86:36860) 2024-04-17T07:21:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.870+0000 7fcb24a00640 1 -- 172.21.15.86:0/1930855616 learned_addr learned my addr 172.21.15.86:0/1930855616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.870+0000 7fcb25e00640 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294f0a0 0x5645629595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.871+0000 7fcb24a00640 1 -- 172.21.15.86:0/1930855616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564562954c00 msgr2=0x564562960b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:21:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.871+0000 7fcb24a00640 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564562954c00 0x564562960b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.871+0000 7fcb24a00640 1 -- 172.21.15.86:0/1930855616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294f0a0 msgr2=0x5645629595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:29.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.871+0000 7fcb24a00640 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294f0a0 0x5645629595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.871+0000 7fcb24a00640 1 -- 172.21.15.86:0/1930855616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56456268f7e0 con 0x56456294fa90 2024-04-17T07:21:29.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.871+0000 7fcb25e00640 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294f0a0 0x5645629595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:21:29.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.871+0000 7fcb24a00640 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562959b10 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c01b040 tx=0x7fcb0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3297bed0468290f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.872+0000 7fcb0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.872+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1930855616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c028020 con 0x56456294fa90 2024-04-17T07:21:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.872+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1930855616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcb0c016040 con 0x56456294fa90 2024-04-17T07:21:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.872+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1930855616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb0c0047a0 con 0x56456294fa90 2024-04-17T07:21:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.873+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1930855616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcb0c0051b0 con 0x56456294fa90 2024-04-17T07:21:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.873+0000 7fcb2ac00740 10 client.17343 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.873+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1930855616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645626eff70 con 0x56456294fa90 2024-04-17T07:21:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.874+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1930855616 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c0047a0 con 0x56456294fa90 2024-04-17T07:21:29.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.874+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1930855616 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcb0c00f5c0 con 0x56456294fa90 2024-04-17T07:21:29.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.874+0000 7fcb2ac00740 10 client.17343 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:29.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.874+0000 7fcb2ac00740 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:21:29.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.874+0000 7fcb2ac00740 20 client.17343 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:29.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.875+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564562965290 0x564562967710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.875+0000 7fcb2ac00740 4 client.17343 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:21:29.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.875+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1930855616 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645628c26d0 con 0x564562965290 2024-04-17T07:21:29.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.876+0000 7fcb25e00640 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564562965290 0x564562967710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.877+0000 7fcb25e00640 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564562965290 0x564562967710 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39fc06663a210be8 server_cookie=677a2514ea4a2975 in_seq=0 out_seq=0 2024-04-17T07:21:29.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.877+0000 7fcb0aa00640 10 client.17343 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:29.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.879+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1930855616 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645628c26d0 con 0x564562965290 2024-04-17T07:21:29.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.879+0000 7fcb0aa00640 10 client.17343 handle_command_reply: tid=0 2024-04-17T07:21:29.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.879+0000 7fcb2ac00740 1 client.17343 shutdown 2024-04-17T07:21:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.879+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1930855616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 msgr2=0x564562959b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.879+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562959b10 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c01b040 tx=0x7fcb0c004270 comp rx=0 tx=0).stop 2024-04-17T07:21:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.880+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1930855616 shutdown_connections 2024-04-17T07:21:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.880+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564562965290 0x564562967710 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.880+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564562954c00 0x564562960b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.880+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56456294fa90 0x564562959b10 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.880+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1930855616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294f0a0 0x5645629595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.880+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1930855616 >> 172.21.15.86:0/1930855616 conn(0x56456294a610 msgr2=0x564562949990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:29.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.880+0000 7fcb0aa00640 0 client.17343 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:29.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.880+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1930855616 shutdown_connections 2024-04-17T07:21:29.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.881+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1930855616 wait complete. 2024-04-17T07:21:29.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.881+0000 7fcb2ac00740 20 client.17343 trim_cache size 0 max 16384 2024-04-17T07:21:29.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.937+0000 7fcb2ac00740 1 Processor -- start 2024-04-17T07:21:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.938+0000 7fcb2ac00740 1 -- start start 2024-04-17T07:21:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.938+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x5645629595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.938+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x564562960b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.938+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562954c00 0x564562959b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.938+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb201b1000 con 0x564562954c00 2024-04-17T07:21:29.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.938+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb201b2b40 con 0x56456294fa90 2024-04-17T07:21:29.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.939+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x564562960b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.939+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x564562960b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37198/0 (socket says 172.21.15.86:37198) 2024-04-17T07:21:29.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.939+0000 7fcb24a00640 1 -- 172.21.15.86:0/2443916930 learned_addr learned my addr 172.21.15.86:0/2443916930 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:29.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.938+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb201b16d0 con 0x56456294f0a0 2024-04-17T07:21:29.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.939+0000 7fcb25e00640 1 --2- 172.21.15.86:0/2443916930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562954c00 0x564562959b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.939+0000 7fcb25e00640 1 --2- 172.21.15.86:0/2443916930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x5645629595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.940+0000 7fcb25e00640 1 -- 172.21.15.86:0/2443916930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 msgr2=0x564562960b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:29.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.940+0000 7fcb25e00640 1 --2- 172.21.15.86:0/2443916930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x564562960b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.940+0000 7fcb25e00640 1 -- 172.21.15.86:0/2443916930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562954c00 msgr2=0x564562959b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:29.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.940+0000 7fcb25e00640 1 --2- 172.21.15.86:0/2443916930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562954c00 0x564562959b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.940+0000 7fcb25e00640 1 -- 172.21.15.86:0/2443916930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564562587170 con 0x56456294f0a0 2024-04-17T07:21:29.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.940+0000 7fcb25e00640 1 --2- 172.21.15.86:0/2443916930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x5645629595d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcb10036040 tx=0x7fcb100098d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c4abbfd0bd9ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:29.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.941+0000 7fcb15c00640 1 -- 172.21.15.86:0/2443916930 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10049040 con 0x56456294f0a0 2024-04-17T07:21:29.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.942+0000 7fcb15c00640 1 -- 172.21.15.86:0/2443916930 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb10003e20 con 0x56456294f0a0 2024-04-17T07:21:29.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.942+0000 7fcb15c00640 1 -- 172.21.15.86:0/2443916930 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb100045d0 con 0x56456294f0a0 2024-04-17T07:21:29.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.943+0000 7fcb2ac00740 1 -- 172.21.15.86:0/2443916930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 msgr2=0x5645629595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:29.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.943+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/2443916930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x5645629595d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcb10036040 tx=0x7fcb100098d0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.944+0000 7fcb2ac00740 1 -- 172.21.15.86:0/2443916930 shutdown_connections 2024-04-17T07:21:29.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.944+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/2443916930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562954c00 0x564562959b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.944+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/2443916930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x564562960b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.944+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/2443916930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x5645629595d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.944+0000 7fcb2ac00740 1 -- 172.21.15.86:0/2443916930 >> 172.21.15.86:0/2443916930 conn(0x56456294a610 msgr2=0x564562951020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:29.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.946+0000 7fcb2ac00740 1 -- 172.21.15.86:0/2443916930 shutdown_connections 2024-04-17T07:21:29.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.947+0000 7fcb2ac00740 1 -- 172.21.15.86:0/2443916930 wait complete. 2024-04-17T07:21:29.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.947+0000 7fcb2ac00740 1 -- start start 2024-04-17T07:21:29.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.947+0000 7fcb2ac00740 1 Processor -- start 2024-04-17T07:21:29.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.953+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x56456295bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.953+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x56456295c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.954+0000 7fcb2ac00740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562954c00 0x5645629da850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.954+0000 7fcb25e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x56456295bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.954+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x56456295c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.954+0000 7fcb25e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x56456295bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52526/0 (socket says 172.21.15.86:52526) 2024-04-17T07:21:29.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.955+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb201b2470 con 0x564562954c00 2024-04-17T07:21:29.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.955+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb201b1da0 con 0x56456294fa90 2024-04-17T07:21:29.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.955+0000 7fcb2ac00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb201aefd0 con 0x56456294f0a0 2024-04-17T07:21:29.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.955+0000 7fcb2ac00740 10 client.0 _ll_register_callbacks cb 0x7fcb0c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:29.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.955+0000 7fcb25e00640 1 -- 172.21.15.86:0/1620261905 learned_addr learned my addr 172.21.15.86:0/1620261905 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:29.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.954+0000 7fcb24a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x56456295c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37212/0 (socket says 172.21.15.86:37212) 2024-04-17T07:21:29.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.957+0000 7fcb25e00640 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562954c00 0x5645629da850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.957+0000 7fcb24a00640 1 -- 172.21.15.86:0/1620261905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 msgr2=0x56456295bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:29.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.958+0000 7fcb24a00640 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x56456295bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.958+0000 7fcb24a00640 1 -- 172.21.15.86:0/1620261905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562954c00 msgr2=0x5645629da850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:29.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.958+0000 7fcb24a00640 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562954c00 0x5645629da850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.959+0000 7fcb24a00640 1 -- 172.21.15.86:0/1620261905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645629dae10 con 0x56456294fa90 2024-04-17T07:21:29.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.960+0000 7fcb25e00640 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x56456295bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:21:29.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.960+0000 7fcb24a00640 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x56456295c140 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000c00 tx=0x7fcb0c011560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed0861f8ce928240 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:29.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.960+0000 7fcb0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:21:29.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.961+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1620261905 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c03b060 con 0x56456294fa90 2024-04-17T07:21:29.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.961+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1620261905 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcb0c00fc50 con 0x56456294fa90 2024-04-17T07:21:29.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.961+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1620261905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb10003e20 con 0x56456294fa90 2024-04-17T07:21:29.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.962+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1620261905 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcb0c028070 con 0x56456294fa90 2024-04-17T07:21:29.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.963+0000 7fcb2ac00740 10 client.36344 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:29.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.963+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1620261905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645629dc150 con 0x56456294fa90 2024-04-17T07:21:29.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.963+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1620261905 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb10003e20 con 0x56456294fa90 2024-04-17T07:21:29.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.964+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1620261905 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcb0c003aa0 con 0x56456294fa90 2024-04-17T07:21:29.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.964+0000 7fcb2ac00740 10 client.36344 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:29.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.964+0000 7fcb2ac00740 10 client.36344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:21:29.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.964+0000 7fcb2ac00740 20 client.36344 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:29.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.964+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5645629dcae0 0x5645629def00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:29.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.964+0000 7fcb2ac00740 4 client.36344 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:21:29.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.964+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1620261905 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5645628bc540 con 0x5645629dcae0 2024-04-17T07:21:29.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.966+0000 7fcb25e00640 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5645629dcae0 0x5645629def00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:29.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.967+0000 7fcb0aa00640 10 client.36344 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:29.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.967+0000 7fcb25e00640 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5645629dcae0 0x5645629def00 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396e9d243463d843 server_cookie=a971d31539ed7e11 in_seq=0 out_seq=0 2024-04-17T07:21:29.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.969+0000 7fcb0aa00640 1 -- 172.21.15.86:0/1620261905 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5645628bc540 con 0x5645629dcae0 2024-04-17T07:21:29.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.969+0000 7fcb0aa00640 10 client.36344 handle_command_reply: tid=0 2024-04-17T07:21:29.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.970+0000 7fcb2ac00740 1 client.36344 shutdown 2024-04-17T07:21:29.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.970+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1620261905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 msgr2=0x56456295c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:29.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.970+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x56456295c140 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000c00 tx=0x7fcb0c011560 comp rx=0 tx=0).stop 2024-04-17T07:21:29.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.971+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1620261905 shutdown_connections 2024-04-17T07:21:29.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.971+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5645629dcae0 0x5645629def00 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.971+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562954c00 0x5645629da850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.971+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56456294fa90 0x56456295c140 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.971+0000 7fcb2ac00740 1 --2- 172.21.15.86:0/1620261905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56456294f0a0 0x56456295bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.971+0000 7fcb0aa00640 0 client.36344 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:29.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.971+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1620261905 >> 172.21.15.86:0/1620261905 conn(0x56456294a610 msgr2=0x5645629595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:29.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.972+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1620261905 shutdown_connections 2024-04-17T07:21:29.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.972+0000 7fcb2ac00740 1 -- 172.21.15.86:0/1620261905 wait complete. 2024-04-17T07:21:29.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.973+0000 7fcb2ac00740 20 client.36344 trim_cache size 0 max 16384 2024-04-17T07:21:29.973 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:21:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:21:29.973 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "21d5c251-22f0-4d8d-bcea-73b9fac2aab2", 2024-04-17T07:21:29.974 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:21:29.974 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:21:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.974+0000 7fcb26800640 1 -- 172.21.15.86:0/2915867198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3d00 msgr2=0x7fcb040a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.974+0000 7fcb26800640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3d00 0x7fcb040a61c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcb180986c0 tx=0x7fcb18009700 comp rx=0 tx=0).stop 2024-04-17T07:21:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.974+0000 7fcb26800640 1 -- 172.21.15.86:0/2915867198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20101e90 msgr2=0x7fcb20075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.974+0000 7fcb26800640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20101e90 0x7fcb20075d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9f0 tx=0x7fcb1800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.975+0000 7fcb26800640 1 -- 172.21.15.86:0/2915867198 shutdown_connections 2024-04-17T07:21:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.975+0000 7fcb26800640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcb040a3d00 0x7fcb040a61c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.975+0000 7fcb26800640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb201039a0 0x7fcb20074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.975+0000 7fcb26800640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb20102fe0 0x7fcb20074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.975+0000 7fcb26800640 1 --2- 172.21.15.86:0/2915867198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb20101e90 0x7fcb20075d10 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9f0 tx=0x7fcb1800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:21:29.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.975+0000 7fcb26800640 1 -- 172.21.15.86:0/2915867198 >> 172.21.15.86:0/2915867198 conn(0x7fcb200fd400 msgr2=0x7fcb201063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.976+0000 7fcb26800640 1 -- 172.21.15.86:0/2915867198 shutdown_connections 2024-04-17T07:21:29.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:29.976+0000 7fcb26800640 1 -- 172.21.15.86:0/2915867198 wait complete. 2024-04-17T07:21:30.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:21:30.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:21:30.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.342+0000 7f933aa00640 1 -- 172.21.15.86:0/2073906643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933410a170 msgr2=0x7f9334110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:30.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.342+0000 7f933aa00640 1 --2- 172.21.15.86:0/2073906643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933410a170 0x7f9334110a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f932c009ca0 tx=0x7f932c02c100 comp rx=0 tx=0).stop 2024-04-17T07:21:30.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.342+0000 7f933aa00640 1 -- 172.21.15.86:0/2073906643 shutdown_connections 2024-04-17T07:21:30.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.342+0000 7f933aa00640 1 --2- 172.21.15.86:0/2073906643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f933410a170 0x7f9334110a00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.342+0000 7f933aa00640 1 --2- 172.21.15.86:0/2073906643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93341097b0 0x7f9334109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.342+0000 7f933aa00640 1 --2- 172.21.15.86:0/2073906643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93341085c0 0x7f93341089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.342+0000 7f933aa00640 1 -- 172.21.15.86:0/2073906643 >> 172.21.15.86:0/2073906643 conn(0x7f9334077570 msgr2=0x7f9334079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:30.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.343+0000 7f933aa00640 1 -- 172.21.15.86:0/2073906643 shutdown_connections 2024-04-17T07:21:30.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.343+0000 7f933aa00640 1 -- 172.21.15.86:0/2073906643 wait complete. 2024-04-17T07:21:30.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.344+0000 7f933aa00640 1 Processor -- start 2024-04-17T07:21:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.344+0000 7f933aa00640 1 -- start start 2024-04-17T07:21:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.345+0000 7f933aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93341085c0 0x7f9334070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.345+0000 7f933aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93341097b0 0x7f9334070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.345+0000 7f933aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f933410a170 0x7f93341ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.345+0000 7f933aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9334114100 con 0x7f933410a170 2024-04-17T07:21:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.345+0000 7f933aa00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9334113f80 con 0x7f93341097b0 2024-04-17T07:21:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.345+0000 7f933aa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9334114280 con 0x7f93341085c0 2024-04-17T07:21:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.346+0000 7f9333400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93341097b0 0x7f9334070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.346+0000 7f9333400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93341097b0 0x7f9334070eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37230/0 (socket says 172.21.15.86:37230) 2024-04-17T07:21:30.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.346+0000 7f9333400640 1 -- 172.21.15.86:0/1401883378 learned_addr learned my addr 172.21.15.86:0/1401883378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:30.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.347+0000 7f9333400640 1 -- 172.21.15.86:0/1401883378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93341085c0 msgr2=0x7f9334070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:30.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.347+0000 7f9333400640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93341085c0 0x7f9334070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.347+0000 7f9333400640 1 -- 172.21.15.86:0/1401883378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f933410a170 msgr2=0x7f93341ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:30.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.347+0000 7f9333400640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f933410a170 0x7f93341ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.347+0000 7f9333400640 1 -- 172.21.15.86:0/1401883378 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93341aed50 con 0x7f93341097b0 2024-04-17T07:21:30.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.348+0000 7f9333400640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93341097b0 0x7f9334070eb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f932000d770 tx=0x7f932000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:30.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.348+0000 7f9330c00640 1 -- 172.21.15.86:0/1401883378 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320017070 con 0x7f93341097b0 2024-04-17T07:21:30.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.348+0000 7f933aa00640 1 -- 172.21.15.86:0/1401883378 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93341aef20 con 0x7f93341097b0 2024-04-17T07:21:30.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.349+0000 7f933aa00640 1 -- 172.21.15.86:0/1401883378 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93341af3e0 con 0x7f93341097b0 2024-04-17T07:21:30.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.349+0000 7f9330c00640 1 -- 172.21.15.86:0/1401883378 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93200054d0 con 0x7f93341097b0 2024-04-17T07:21:30.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.349+0000 7f9330c00640 1 -- 172.21.15.86:0/1401883378 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320004cf0 con 0x7f93341097b0 2024-04-17T07:21:30.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.355+0000 7f9330c00640 1 -- 172.21.15.86:0/1401883378 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9320010460 con 0x7f93341097b0 2024-04-17T07:21:30.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.359+0000 7f9330c00640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93180a3c30 0x7f93180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.359+0000 7f9330c00640 1 -- 172.21.15.86:0/1401883378 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93200d1120 con 0x7f93341097b0 2024-04-17T07:21:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.360+0000 7f9333e00640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93180a3c30 0x7f93180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.361+0000 7f9333e00640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93180a3c30 0x7f93180a60f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9324005fd0 tx=0x7f932400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:30.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.362+0000 7f933d672740 1 Processor -- start 2024-04-17T07:21:30.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.363+0000 7f933d672740 1 -- start start 2024-04-17T07:21:30.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.363+0000 7f933d672740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c14186c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.363+0000 7f933d672740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186cb80 0x55c141871680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.363+0000 7f933d672740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141874180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.364+0000 7f9338a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141874180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.364+0000 7f9338a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141874180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52560/0 (socket says 172.21.15.86:52560) 2024-04-17T07:21:30.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.364+0000 7f9333e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c14186c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.365+0000 7f9333e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c14186c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37236/0 (socket says 172.21.15.86:37236) 2024-04-17T07:21:30.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.365+0000 7f9338a00640 1 -- 172.21.15.86:0/3539343743 learned_addr learned my addr 172.21.15.86:0/3539343743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:30.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.365+0000 7f933d672740 1 -- 172.21.15.86:0/3539343743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c1417d3120 con 0x55c14186cb80 2024-04-17T07:21:30.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.365+0000 7f933d672740 1 -- 172.21.15.86:0/3539343743 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c141779340 con 0x55c14186c190 2024-04-17T07:21:30.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.365+0000 7f933d672740 1 -- 172.21.15.86:0/3539343743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c1417cf0e0 con 0x55c141871cf0 2024-04-17T07:21:30.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.365+0000 7f9333400640 1 --2- 172.21.15.86:0/3539343743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186cb80 0x55c141871680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.366+0000 7f9333e00640 1 -- 172.21.15.86:0/3539343743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 msgr2=0x55c141874180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:30.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.366+0000 7f9333e00640 1 --2- 172.21.15.86:0/3539343743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141874180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.366+0000 7f9333e00640 1 -- 172.21.15.86:0/3539343743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186cb80 msgr2=0x55c141871680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:30.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.366+0000 7f9333e00640 1 --2- 172.21.15.86:0/3539343743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186cb80 0x55c141871680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.366+0000 7f9333e00640 1 -- 172.21.15.86:0/3539343743 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1415af7d0 con 0x55c14186c190 2024-04-17T07:21:30.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.366+0000 7f9338a00640 1 --2- 172.21.15.86:0/3539343743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141874180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:21:30.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.367+0000 7f9333e00640 1 --2- 172.21.15.86:0/3539343743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c14186c5b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9334109610 tx=0x7f9324010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ccd49173ffa363 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:30.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.367+0000 7f9329200640 1 -- 172.21.15.86:0/3539343743 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324021070 con 0x55c14186c190 2024-04-17T07:21:30.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.368+0000 7f9329200640 1 -- 172.21.15.86:0/3539343743 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9324017070 con 0x55c14186c190 2024-04-17T07:21:30.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.368+0000 7f9329200640 1 -- 172.21.15.86:0/3539343743 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932400f620 con 0x55c14186c190 2024-04-17T07:21:30.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.369+0000 7f933d672740 1 -- 172.21.15.86:0/3539343743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 msgr2=0x55c14186c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:30.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.369+0000 7f933d672740 1 --2- 172.21.15.86:0/3539343743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c14186c5b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9334109610 tx=0x7f9324010000 comp rx=0 tx=0).stop 2024-04-17T07:21:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.369+0000 7f933d672740 1 -- 172.21.15.86:0/3539343743 shutdown_connections 2024-04-17T07:21:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.370+0000 7f933d672740 1 --2- 172.21.15.86:0/3539343743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141874180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.370+0000 7f933d672740 1 --2- 172.21.15.86:0/3539343743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186cb80 0x55c141871680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.370+0000 7f933d672740 1 --2- 172.21.15.86:0/3539343743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c14186c5b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.370+0000 7f933d672740 1 -- 172.21.15.86:0/3539343743 >> 172.21.15.86:0/3539343743 conn(0x55c141867700 msgr2=0x55c141867b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.370+0000 7f933d672740 1 -- 172.21.15.86:0/3539343743 shutdown_connections 2024-04-17T07:21:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.371+0000 7f933d672740 1 -- 172.21.15.86:0/3539343743 wait complete. 2024-04-17T07:21:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.371+0000 7f933d672740 1 -- start start 2024-04-17T07:21:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.372+0000 7f933d672740 1 Processor -- start 2024-04-17T07:21:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.372+0000 7f933d672740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c141876750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.372+0000 7f933d672740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c14186cb80 0x55c141876c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.372+0000 7f933d672740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c141871cf0 0x55c14187dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.372+0000 7f933d672740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c141874cf0 con 0x55c141871cf0 2024-04-17T07:21:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.372+0000 7f933d672740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c14178c710 con 0x55c14186c190 2024-04-17T07:21:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.372+0000 7f933d672740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c141874e70 con 0x55c14186cb80 2024-04-17T07:21:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.372+0000 7f933d672740 10 client.0 _ll_register_callbacks cb 0x55c141787af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.373+0000 7f9333e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c141876750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.373+0000 7f9333e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c141876750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37248/0 (socket says 172.21.15.86:37248) 2024-04-17T07:21:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.373+0000 7f9333e00640 1 -- 172.21.15.86:0/4180052414 learned_addr learned my addr 172.21.15.86:0/4180052414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.373+0000 7f9333400640 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c14186cb80 0x55c141876c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.373+0000 7f9333e00640 1 -- 172.21.15.86:0/4180052414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c14186cb80 msgr2=0x55c141876c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.373+0000 7f9333e00640 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c14186cb80 0x55c141876c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.373+0000 7f9333e00640 1 -- 172.21.15.86:0/4180052414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c141871cf0 msgr2=0x55c14187dcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:21:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.373+0000 7f9333e00640 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c141871cf0 0x55c14187dcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.373+0000 7f9333e00640 1 -- 172.21.15.86:0/4180052414 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1414e53f0 con 0x55c14186c190 2024-04-17T07:21:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.374+0000 7f9333e00640 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c141876750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f932401dd60 tx=0x7f932400dec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eb1bb20d22468dd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.374+0000 7f931d600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:21:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.375+0000 7f931d600640 1 -- 172.21.15.86:0/4180052414 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932402c070 con 0x55c14186c190 2024-04-17T07:21:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.375+0000 7f931d600640 1 -- 172.21.15.86:0/4180052414 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f932401d8c0 con 0x55c14186c190 2024-04-17T07:21:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.375+0000 7f933d672740 1 -- 172.21.15.86:0/4180052414 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9324017070 con 0x55c14186c190 2024-04-17T07:21:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.375+0000 7f931d600640 1 -- 172.21.15.86:0/4180052414 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9324021040 con 0x55c14186c190 2024-04-17T07:21:30.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.378+0000 7f933d672740 10 client.36356 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:30.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.378+0000 7f933d672740 1 -- 172.21.15.86:0/4180052414 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1414a50f0 con 0x55c14186c190 2024-04-17T07:21:30.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.378+0000 7f931d600640 1 -- 172.21.15.86:0/4180052414 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9324017070 con 0x55c14186c190 2024-04-17T07:21:30.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.379+0000 7f931d600640 1 -- 172.21.15.86:0/4180052414 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f932401c070 con 0x55c14186c190 2024-04-17T07:21:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.380+0000 7f933d672740 10 client.36356 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.380+0000 7f933d672740 10 client.36356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:21:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.380+0000 7f933d672740 20 client.36356 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.380+0000 7f933d672740 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c141882290 0x55c1418846e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.380+0000 7f933d672740 4 client.36356 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:21:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.380+0000 7f933d672740 1 -- 172.21.15.86:0/4180052414 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1417e0b40 con 0x55c141882290 2024-04-17T07:21:30.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.380+0000 7f9333400640 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c141882290 0x55c1418846e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.381+0000 7f931d600640 10 client.36356 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:30.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.381+0000 7f9333400640 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c141882290 0x55c1418846e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1b154c91bba140 server_cookie=e2384009ad0b196d in_seq=0 out_seq=0 2024-04-17T07:21:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.382+0000 7f931d600640 1 -- 172.21.15.86:0/4180052414 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1417e0b40 con 0x55c141882290 2024-04-17T07:21:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.382+0000 7f931d600640 10 client.36356 handle_command_reply: tid=0 2024-04-17T07:21:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.382+0000 7f933d672740 1 client.36356 shutdown 2024-04-17T07:21:30.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.385+0000 7f933d672740 1 -- 172.21.15.86:0/4180052414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 msgr2=0x55c141876750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.385+0000 7f933d672740 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c141876750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f932401dd60 tx=0x7f932400dec0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.386+0000 7f933d672740 1 -- 172.21.15.86:0/4180052414 shutdown_connections 2024-04-17T07:21:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.386+0000 7f933d672740 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c141882290 0x55c1418846e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.386+0000 7f933d672740 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c141871cf0 0x55c14187dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.386+0000 7f933d672740 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c14186cb80 0x55c141876c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.386+0000 7f933d672740 1 --2- 172.21.15.86:0/4180052414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186c190 0x55c141876750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.386+0000 7f933d672740 1 -- 172.21.15.86:0/4180052414 >> 172.21.15.86:0/4180052414 conn(0x55c141867700 msgr2=0x55c141866890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:30.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.389+0000 7f931d600640 0 client.36356 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:30.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.389+0000 7f933d672740 1 -- 172.21.15.86:0/4180052414 shutdown_connections 2024-04-17T07:21:30.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.390+0000 7f933d672740 1 -- 172.21.15.86:0/4180052414 wait complete. 2024-04-17T07:21:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.390+0000 7f933d672740 20 client.36356 trim_cache size 0 max 16384 2024-04-17T07:21:30.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.429+0000 7f933d672740 1 Processor -- start 2024-04-17T07:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.429+0000 7f933d672740 1 -- start start 2024-04-17T07:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.430+0000 7f933d672740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 0x55c14187ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.430+0000 7f933d672740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 0x55c14187dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.430+0000 7f933d672740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141876bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.430+0000 7f933d672740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f931400fba0 con 0x55c14186c190 2024-04-17T07:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.430+0000 7f933d672740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93140116e0 con 0x55c14186cb80 2024-04-17T07:21:30.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.430+0000 7f933d672740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9314010270 con 0x55c141871cf0 2024-04-17T07:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.430+0000 7f9338a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141876bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.430+0000 7f9338a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141876bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52586/0 (socket says 172.21.15.86:52586) 2024-04-17T07:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.431+0000 7f9338a00640 1 -- 172.21.15.86:0/2158935222 learned_addr learned my addr 172.21.15.86:0/2158935222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.431+0000 7f9333400640 1 --2- 172.21.15.86:0/2158935222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 0x55c14187dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.431+0000 7f9333e00640 1 --2- 172.21.15.86:0/2158935222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 0x55c14187ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.432+0000 7f9333400640 1 -- 172.21.15.86:0/2158935222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 msgr2=0x55c141876bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.432+0000 7f9333400640 1 --2- 172.21.15.86:0/2158935222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141876bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.432+0000 7f9333400640 1 -- 172.21.15.86:0/2158935222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 msgr2=0x55c14187ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.432+0000 7f9333400640 1 --2- 172.21.15.86:0/2158935222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 0x55c14187ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.432+0000 7f9333400640 1 -- 172.21.15.86:0/2158935222 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14186db80 con 0x55c14186cb80 2024-04-17T07:21:30.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.433+0000 7f9333400640 1 --2- 172.21.15.86:0/2158935222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 0x55c14187dc90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f93200ce8f0 tx=0x7f93200047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6396709ec51f57b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.434+0000 7f931fe00640 1 -- 172.21.15.86:0/2158935222 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320020040 con 0x55c14186cb80 2024-04-17T07:21:30.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.435+0000 7f931fe00640 1 -- 172.21.15.86:0/2158935222 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f932009b120 con 0x55c14186cb80 2024-04-17T07:21:30.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.435+0000 7f931fe00640 1 -- 172.21.15.86:0/2158935222 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93200a5860 con 0x55c14186cb80 2024-04-17T07:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.436+0000 7f933d672740 1 -- 172.21.15.86:0/2158935222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 msgr2=0x55c14187dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.437+0000 7f933d672740 1 --2- 172.21.15.86:0/2158935222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 0x55c14187dc90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f93200ce8f0 tx=0x7f93200047a0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.438+0000 7f933d672740 1 -- 172.21.15.86:0/2158935222 shutdown_connections 2024-04-17T07:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.438+0000 7f933d672740 1 --2- 172.21.15.86:0/2158935222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c141876bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.438+0000 7f933d672740 1 --2- 172.21.15.86:0/2158935222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 0x55c14187dc90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.438+0000 7f933d672740 1 --2- 172.21.15.86:0/2158935222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 0x55c14187ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.438+0000 7f933d672740 1 -- 172.21.15.86:0/2158935222 >> 172.21.15.86:0/2158935222 conn(0x55c141867700 msgr2=0x55c141866000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.438+0000 7f933d672740 1 -- 172.21.15.86:0/2158935222 shutdown_connections 2024-04-17T07:21:30.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.438+0000 7f933d672740 1 -- 172.21.15.86:0/2158935222 wait complete. 2024-04-17T07:21:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.438+0000 7f933d672740 1 -- start start 2024-04-17T07:21:30.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.439+0000 7f933d672740 1 Processor -- start 2024-04-17T07:21:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.440+0000 7f933d672740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 0x55c141878cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.441+0000 7f933d672740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 0x55c141879200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.441+0000 7f933d672740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c14187c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.441+0000 7f9333400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 0x55c141879200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.442+0000 7f933d672740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9314011010 con 0x55c14186c190 2024-04-17T07:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.442+0000 7f933d672740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9314010940 con 0x55c14186cb80 2024-04-17T07:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.442+0000 7f933d672740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f931400db70 con 0x55c141871cf0 2024-04-17T07:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.442+0000 7f933d672740 10 client.0 _ll_register_callbacks cb 0x7f9320099380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.442+0000 7f9338a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c14187c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.442+0000 7f9338a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c14187c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52594/0 (socket says 172.21.15.86:52594) 2024-04-17T07:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.442+0000 7f9338a00640 1 -- 172.21.15.86:0/3759598811 learned_addr learned my addr 172.21.15.86:0/3759598811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:21:30.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.441+0000 7f9333e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 0x55c141878cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.443+0000 7f9333e00640 1 -- 172.21.15.86:0/3759598811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 msgr2=0x55c14187c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:30.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.443+0000 7f9333e00640 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c14187c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.443+0000 7f9333e00640 1 -- 172.21.15.86:0/3759598811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 msgr2=0x55c141879200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.443+0000 7f9333e00640 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 0x55c141879200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.443+0000 7f9333e00640 1 -- 172.21.15.86:0/3759598811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14187c810 con 0x55c14186c190 2024-04-17T07:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.444+0000 7f9333e00640 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 0x55c141878cc0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f932400b3e0 tx=0x7f932400da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4637f808f056976 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:21:30.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.444+0000 7f931d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:21:30.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.446+0000 7f931d600640 1 -- 172.21.15.86:0/3759598811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932402c040 con 0x55c14186c190 2024-04-17T07:21:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.446+0000 7f931d600640 1 -- 172.21.15.86:0/3759598811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f932401d6c0 con 0x55c14186c190 2024-04-17T07:21:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.447+0000 7f933d672740 1 -- 172.21.15.86:0/3759598811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f932009b120 con 0x55c14186c190 2024-04-17T07:21:30.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.448+0000 7f931d600640 1 -- 172.21.15.86:0/3759598811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f932401d8c0 con 0x55c14186c190 2024-04-17T07:21:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.448+0000 7f933d672740 10 client.17358 fetch_fsmap learned FSMap version 28 2024-04-17T07:21:30.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.448+0000 7f933d672740 1 -- 172.21.15.86:0/3759598811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c14187db20 con 0x55c14186c190 2024-04-17T07:21:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.449+0000 7f931d600640 1 -- 172.21.15.86:0/3759598811 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f932009b120 con 0x55c14186c190 2024-04-17T07:21:30.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.449+0000 7f931d600640 1 -- 172.21.15.86:0/3759598811 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f932401c040 con 0x55c14186c190 2024-04-17T07:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.450+0000 7f933d672740 10 client.17358 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.450+0000 7f933d672740 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.450+0000 7f933d672740 20 client.17358 populate_metadata read hostname 'smithi086' 2024-04-17T07:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.450+0000 7f933d672740 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c1418f9b60 0x55c1418fbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.450+0000 7f933d672740 4 client.17358 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:21:30.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.450+0000 7f933d672740 1 -- 172.21.15.86:0/3759598811 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c1417da9b0 con 0x55c1418f9b60 2024-04-17T07:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.451+0000 7f9333400640 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c1418f9b60 0x55c1418fbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.452+0000 7f9333400640 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c1418f9b60 0x55c1418fbf50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf0555aaf264158f server_cookie=62602ea727ae9e6d in_seq=0 out_seq=0 2024-04-17T07:21:30.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.452+0000 7f931d600640 10 client.17358 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:30.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.452+0000 7f931d600640 1 -- 172.21.15.86:0/3759598811 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55c1417da9b0 con 0x55c1418f9b60 2024-04-17T07:21:30.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.452+0000 7f931d600640 10 client.17358 handle_command_reply: tid=0 2024-04-17T07:21:30.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.452+0000 7f933d672740 1 client.17358 shutdown 2024-04-17T07:21:30.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.453+0000 7f933d672740 1 -- 172.21.15.86:0/3759598811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 msgr2=0x55c141878cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:30.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.453+0000 7f933d672740 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 0x55c141878cc0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f932400b3e0 tx=0x7f932400da70 comp rx=0 tx=0).stop 2024-04-17T07:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.453+0000 7f933d672740 1 -- 172.21.15.86:0/3759598811 shutdown_connections 2024-04-17T07:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.453+0000 7f933d672740 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c1418f9b60 0x55c1418fbf50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.453+0000 7f933d672740 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c141871cf0 0x55c14187c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.453+0000 7f933d672740 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c14186cb80 0x55c141879200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.453+0000 7f931d600640 0 client.17358 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.453+0000 7f933d672740 1 --2- 172.21.15.86:0/3759598811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c14186c190 0x55c141878cc0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.453+0000 7f933d672740 1 -- 172.21.15.86:0/3759598811 >> 172.21.15.86:0/3759598811 conn(0x55c141867700 msgr2=0x55c14186a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.454+0000 7f933d672740 1 -- 172.21.15.86:0/3759598811 shutdown_connections 2024-04-17T07:21:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.454+0000 7f933d672740 1 -- 172.21.15.86:0/3759598811 wait complete. 2024-04-17T07:21:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.454+0000 7f933d672740 20 client.17358 trim_cache size 0 max 16384 2024-04-17T07:21:30.454 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:21:30.454 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (3 inodes in the stack)", 2024-04-17T07:21:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:21:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "21d5c251-22f0-4d8d-bcea-73b9fac2aab2": { 2024-04-17T07:21:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:21:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "21d5c251-22f0-4d8d-bcea-73b9fac2aab2", 2024-04-17T07:21:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:21:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:21:30.455 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:21:30.455 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:21:30.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.455+0000 7f933aa00640 1 -- 172.21.15.86:0/1401883378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93180a3c30 msgr2=0x7f93180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.455+0000 7f933aa00640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93180a3c30 0x7f93180a60f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9324005fd0 tx=0x7f932400b410 comp rx=0 tx=0).stop 2024-04-17T07:21:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.455+0000 7f933aa00640 1 -- 172.21.15.86:0/1401883378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93341097b0 msgr2=0x7f9334070eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:21:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.455+0000 7f933aa00640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93341097b0 0x7f9334070eb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f932000d770 tx=0x7f932000dc30 comp rx=0 tx=0).stop 2024-04-17T07:21:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.456+0000 7f933aa00640 1 -- 172.21.15.86:0/1401883378 shutdown_connections 2024-04-17T07:21:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.456+0000 7f933aa00640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93180a3c30 0x7f93180a60f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.456+0000 7f933aa00640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f933410a170 0x7f93341ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.456+0000 7f933aa00640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93341097b0 0x7f9334070eb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.456+0000 7f933aa00640 1 --2- 172.21.15.86:0/1401883378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93341085c0 0x7f9334070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:21:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.456+0000 7f933aa00640 1 -- 172.21.15.86:0/1401883378 >> 172.21.15.86:0/1401883378 conn(0x7f9334077570 msgr2=0x7f933410cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:21:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.457+0000 7f933aa00640 1 -- 172.21.15.86:0/1401883378 shutdown_connections 2024-04-17T07:21:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:21:30.457+0000 7f933aa00640 1 -- 172.21.15.86:0/1401883378 wait complete. 2024-04-17T07:21:30.486 INFO:tasks.cephfs.filesystem:scrub status for tag:21d5c251-22f0-4d8d-bcea-73b9fac2aab2 - {'path': '/', 'tag': '21d5c251-22f0-4d8d-bcea-73b9fac2aab2', 'options': 'recursive,force'} 2024-04-17T07:21:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:30 smithi098 ceph-mon[35418]: pgmap v654: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 648 op/s 2024-04-17T07:21:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:30 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:21:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:30 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:21:30.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:30 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:21:30.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:30 smithi103 ceph-mon[35495]: pgmap v654: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 648 op/s 2024-04-17T07:21:30.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:30 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:21:30.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:30 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:21:30.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:30 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:21:30.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:30 smithi086 ceph-mon[31733]: pgmap v654: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 648 op/s 2024-04-17T07:21:30.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:30 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:21:30.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:30 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:21:30.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:30 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:21:31.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:31 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:21:31.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:31 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:21:31.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:31 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:21:32.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:32 smithi098 ceph-mon[35418]: pgmap v655: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 16 MiB/s wr, 722 op/s 2024-04-17T07:21:32.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:32 smithi103 ceph-mon[35495]: pgmap v655: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 16 MiB/s wr, 722 op/s 2024-04-17T07:21:32.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:32 smithi086 ceph-mon[31733]: pgmap v655: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 16 MiB/s wr, 722 op/s 2024-04-17T07:21:34.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 130.0 s, 498.2 tps, lat 162.093 ms stddev 279.120 2024-04-17T07:21:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:34 smithi098 ceph-mon[35418]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 15 MiB/s wr, 682 op/s 2024-04-17T07:21:34.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:34 smithi103 ceph-mon[35495]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 15 MiB/s wr, 682 op/s 2024-04-17T07:21:34.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:34 smithi086 ceph-mon[31733]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 15 MiB/s wr, 682 op/s 2024-04-17T07:21:35.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:35 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:21:35.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:35 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:21:35.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:35 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:21:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:36 smithi098 ceph-mon[35418]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 591 op/s 2024-04-17T07:21:36.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:36 smithi103 ceph-mon[35495]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 591 op/s 2024-04-17T07:21:36.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:36 smithi086 ceph-mon[31733]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 13 MiB/s wr, 591 op/s 2024-04-17T07:21:38.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:38 smithi098 ceph-mon[35418]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 657 op/s 2024-04-17T07:21:38.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:38 smithi103 ceph-mon[35495]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 657 op/s 2024-04-17T07:21:38.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:38 smithi086 ceph-mon[31733]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 16 MiB/s wr, 657 op/s 2024-04-17T07:21:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 135.0 s, 750.9 tps, lat 157.670 ms stddev 271.271 2024-04-17T07:21:40.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:40 smithi098 ceph-mon[35418]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 629 op/s 2024-04-17T07:21:40.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:40 smithi103 ceph-mon[35495]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 629 op/s 2024-04-17T07:21:40.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:40 smithi086 ceph-mon[31733]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 629 op/s 2024-04-17T07:21:42.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:42 smithi098 ceph-mon[35418]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 14 MiB/s wr, 712 op/s 2024-04-17T07:21:42.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:42 smithi103 ceph-mon[35495]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 14 MiB/s wr, 712 op/s 2024-04-17T07:21:42.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:42 smithi086 ceph-mon[31733]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 14 MiB/s wr, 712 op/s 2024-04-17T07:21:44.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 140.0 s, 659.3 tps, lat 147.897 ms stddev 211.978 2024-04-17T07:21:44.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:44 smithi098 ceph-mon[35418]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 16 MiB/s wr, 697 op/s 2024-04-17T07:21:44.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:44 smithi103 ceph-mon[35495]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 16 MiB/s wr, 697 op/s 2024-04-17T07:21:44.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:44 smithi086 ceph-mon[31733]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 16 MiB/s wr, 697 op/s 2024-04-17T07:21:46.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:46 smithi098 ceph-mon[35418]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 655 op/s 2024-04-17T07:21:46.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:46 smithi103 ceph-mon[35495]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 655 op/s 2024-04-17T07:21:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:46 smithi086 ceph-mon[31733]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 655 op/s 2024-04-17T07:21:48.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:48 smithi098 ceph-mon[35418]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 755 op/s 2024-04-17T07:21:48.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:48 smithi103 ceph-mon[35495]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 755 op/s 2024-04-17T07:21:48.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:48 smithi086 ceph-mon[31733]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 755 op/s 2024-04-17T07:21:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 145.0 s, 513.8 tps, lat 159.461 ms stddev 226.793 2024-04-17T07:21:50.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:50 smithi098 ceph-mon[35418]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 13 MiB/s wr, 710 op/s 2024-04-17T07:21:50.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:50 smithi103 ceph-mon[35495]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 13 MiB/s wr, 710 op/s 2024-04-17T07:21:50.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:50 smithi086 ceph-mon[31733]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 13 MiB/s wr, 710 op/s 2024-04-17T07:21:52.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:52 smithi098 ceph-mon[35418]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 774 op/s 2024-04-17T07:21:52.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:52 smithi103 ceph-mon[35495]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 774 op/s 2024-04-17T07:21:52.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:52 smithi086 ceph-mon[31733]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 774 op/s 2024-04-17T07:21:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 150.0 s, 771.3 tps, lat 157.358 ms stddev 253.462 2024-04-17T07:21:54.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:54 smithi098 ceph-mon[35418]: pgmap v666: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 756 op/s 2024-04-17T07:21:54.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:54 smithi103 ceph-mon[35495]: pgmap v666: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 756 op/s 2024-04-17T07:21:54.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:54 smithi086 ceph-mon[31733]: pgmap v666: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 756 op/s 2024-04-17T07:21:56.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:56 smithi098 ceph-mon[35418]: pgmap v667: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 683 op/s 2024-04-17T07:21:56.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:56 smithi103 ceph-mon[35495]: pgmap v667: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 683 op/s 2024-04-17T07:21:56.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:56 smithi086 ceph-mon[31733]: pgmap v667: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 13 MiB/s wr, 683 op/s 2024-04-17T07:21:58.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:21:58 smithi098 ceph-mon[35418]: pgmap v668: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 798 op/s 2024-04-17T07:21:58.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:21:58 smithi103 ceph-mon[35495]: pgmap v668: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 798 op/s 2024-04-17T07:21:58.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:21:58 smithi086 ceph-mon[31733]: pgmap v668: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 15 MiB/s wr, 798 op/s 2024-04-17T07:21:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 155.0 s, 679.8 tps, lat 145.872 ms stddev 203.023 2024-04-17T07:22:00.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:22:00.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:22:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:00 smithi098 ceph-mon[35418]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 14 MiB/s wr, 749 op/s 2024-04-17T07:22:00.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:00 smithi103 ceph-mon[35495]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 14 MiB/s wr, 749 op/s 2024-04-17T07:22:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.803+0000 7f8e03000640 1 -- 172.21.15.86:0/3064787483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc0749c0 msgr2=0x7f8dfc074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.803+0000 7f8e03000640 1 --2- 172.21.15.86:0/3064787483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc0749c0 0x7f8dfc074e40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009890 tx=0x7f8de402bb70 comp rx=0 tx=0).stop 2024-04-17T07:22:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.803+0000 7f8e03000640 1 -- 172.21.15.86:0/3064787483 shutdown_connections 2024-04-17T07:22:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.803+0000 7f8e03000640 1 --2- 172.21.15.86:0/3064787483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc10e570 0x7f8dfc110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.803+0000 7f8e03000640 1 --2- 172.21.15.86:0/3064787483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc0749c0 0x7f8dfc074e40 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.803+0000 7f8e03000640 1 --2- 172.21.15.86:0/3064787483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc075fa0 0x7f8dfc0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.803+0000 7f8e03000640 1 -- 172.21.15.86:0/3064787483 >> 172.21.15.86:0/3064787483 conn(0x7f8dfc0fd510 msgr2=0x7f8dfc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.803+0000 7f8e03000640 1 -- 172.21.15.86:0/3064787483 shutdown_connections 2024-04-17T07:22:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.804+0000 7f8e03000640 1 -- 172.21.15.86:0/3064787483 wait complete. 2024-04-17T07:22:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.805+0000 7f8e03000640 1 Processor -- start 2024-04-17T07:22:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.805+0000 7f8e03000640 1 -- start start 2024-04-17T07:22:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.805+0000 7f8e03000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc0749c0 0x7f8dfc19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.805+0000 7f8e03000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc075fa0 0x7f8dfc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.805+0000 7f8e03000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc10e570 0x7f8dfc1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.805+0000 7f8e03000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1140e0 con 0x7f8dfc0749c0 2024-04-17T07:22:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.805+0000 7f8e03000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc113f60 con 0x7f8dfc10e570 2024-04-17T07:22:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.805+0000 7f8e03000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc114260 con 0x7f8dfc075fa0 2024-04-17T07:22:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.806+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc0749c0 0x7f8dfc19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.806+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc0749c0 0x7f8dfc19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51758/0 (socket says 172.21.15.86:51758) 2024-04-17T07:22:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.806+0000 7f8dfbe00640 1 -- 172.21.15.86:0/105290576 learned_addr learned my addr 172.21.15.86:0/105290576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.806+0000 7f8dfb400640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc075fa0 0x7f8dfc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.806+0000 7f8e01000640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc10e570 0x7f8dfc1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.807+0000 7f8dfb400640 1 -- 172.21.15.86:0/105290576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc10e570 msgr2=0x7f8dfc1a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.807+0000 7f8dfb400640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc10e570 0x7f8dfc1a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.807+0000 7f8dfb400640 1 -- 172.21.15.86:0/105290576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc0749c0 msgr2=0x7f8dfc19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.807+0000 7f8dfb400640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc0749c0 0x7f8dfc19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.807+0000 7f8dfb400640 1 -- 172.21.15.86:0/105290576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc06bab0 con 0x7f8dfc075fa0 2024-04-17T07:22:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.808+0000 7f8dfb400640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc075fa0 0x7f8dfc19edc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8de402c180 tx=0x7f8de40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:00.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.808+0000 7f8df8c00640 1 -- 172.21.15.86:0/105290576 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4041020 con 0x7f8dfc075fa0 2024-04-17T07:22:00.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.808+0000 7f8df8c00640 1 -- 172.21.15.86:0/105290576 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8de40035f0 con 0x7f8dfc075fa0 2024-04-17T07:22:00.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.808+0000 7f8df8c00640 1 -- 172.21.15.86:0/105290576 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4039040 con 0x7f8dfc075fa0 2024-04-17T07:22:00.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.809+0000 7f8e03000640 1 -- 172.21.15.86:0/105290576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dfc06bd40 con 0x7f8dfc075fa0 2024-04-17T07:22:00.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.809+0000 7f8e03000640 1 -- 172.21.15.86:0/105290576 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dfc06c1e0 con 0x7f8dfc075fa0 2024-04-17T07:22:00.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.814+0000 7f8e05be1740 1 Processor -- start 2024-04-17T07:22:00.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.814+0000 7f8e05be1740 1 -- start start 2024-04-17T07:22:00.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.815+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8190 0x560f09dd85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.815+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8b80 0x560f09ddd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.815+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09de0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.815+0000 7f8e05be1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f09d3f120 con 0x560f09dd8b80 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.815+0000 7f8e05be1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560f09ce5340 con 0x560f09dddcf0 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.815+0000 7f8e05be1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560f09d3b0e0 con 0x560f09dd8190 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.815+0000 7f8df8c00640 1 -- 172.21.15.86:0/105290576 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8de4044020 con 0x7f8dfc075fa0 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.815+0000 7f8e01000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8190 0x560f09dd85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.815+0000 7f8e01000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8190 0x560f09dd85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44848/0 (socket says 172.21.15.86:44848) 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.815+0000 7f8e01000640 1 -- 172.21.15.86:0/4193274606 learned_addr learned my addr 172.21.15.86:0/4193274606 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.816+0000 7f8dfb400640 1 --2- 172.21.15.86:0/4193274606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09de0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.816+0000 7f8e01000640 1 -- 172.21.15.86:0/4193274606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 msgr2=0x560f09de0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.816+0000 7f8e01000640 1 --2- 172.21.15.86:0/4193274606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09de0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.816+0000 7f8e01000640 1 -- 172.21.15.86:0/4193274606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8b80 msgr2=0x560f09ddd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.816+0000 7f8e01000640 1 --2- 172.21.15.86:0/4193274606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8b80 0x560f09ddd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.816+0000 7f8e01000640 1 -- 172.21.15.86:0/4193274606 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f09b1b7d0 con 0x560f09dd8190 2024-04-17T07:22:00.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.817+0000 7f8e01000640 1 --2- 172.21.15.86:0/4193274606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8190 0x560f09dd85b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009ee0 tx=0x7f8df4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c09d4370a6fb6615 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:00.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.817+0000 7f8df1600640 1 -- 172.21.15.86:0/4193274606 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4016070 con 0x560f09dd8190 2024-04-17T07:22:00.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.818+0000 7f8df1600640 1 -- 172.21.15.86:0/4193274606 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df4004d20 con 0x560f09dd8190 2024-04-17T07:22:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.818+0000 7f8df1600640 1 -- 172.21.15.86:0/4193274606 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4012740 con 0x560f09dd8190 2024-04-17T07:22:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.818+0000 7f8df8c00640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 0x7f8de00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.819+0000 7f8e05be1740 1 -- 172.21.15.86:0/4193274606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8190 msgr2=0x560f09dd85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.819+0000 7f8e05be1740 1 --2- 172.21.15.86:0/4193274606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8190 0x560f09dd85b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009ee0 tx=0x7f8df4010740 comp rx=0 tx=0).stop 2024-04-17T07:22:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.819+0000 7f8e05be1740 1 -- 172.21.15.86:0/4193274606 shutdown_connections 2024-04-17T07:22:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.819+0000 7f8e05be1740 1 --2- 172.21.15.86:0/4193274606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09de0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.819+0000 7f8e05be1740 1 --2- 172.21.15.86:0/4193274606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8b80 0x560f09ddd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.819+0000 7f8e05be1740 1 --2- 172.21.15.86:0/4193274606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8190 0x560f09dd85b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.819+0000 7f8e05be1740 1 -- 172.21.15.86:0/4193274606 >> 172.21.15.86:0/4193274606 conn(0x560f09dd3700 msgr2=0x560f09dd3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.819+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 0x7f8de00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.819+0000 7f8df8c00640 1 -- 172.21.15.86:0/105290576 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8de40f2050 con 0x7f8dfc075fa0 2024-04-17T07:22:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.821+0000 7f8e05be1740 1 -- 172.21.15.86:0/4193274606 shutdown_connections 2024-04-17T07:22:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.822+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 0x7f8de00a6240 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x560f09dd4670 tx=0x7f8dec0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.822+0000 7f8e05be1740 1 -- 172.21.15.86:0/4193274606 wait complete. 2024-04-17T07:22:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.823+0000 7f8e05be1740 1 -- start start 2024-04-17T07:22:00.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.823+0000 7f8e05be1740 1 Processor -- start 2024-04-17T07:22:00.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.824+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09de26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09de9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8e05be1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f09de0ca0 con 0x560f09dd8190 2024-04-17T07:22:00.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8e05be1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560f09cf8710 con 0x560f09dddcf0 2024-04-17T07:22:00.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8e05be1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560f09de0e20 con 0x560f09dd8b80 2024-04-17T07:22:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09de26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8e05be1740 10 client.0 _ll_register_callbacks cb 0x560f09cf3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09de26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51782/0 (socket says 172.21.15.86:51782) 2024-04-17T07:22:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8dfbe00640 1 -- 172.21.15.86:0/1495635873 learned_addr learned my addr 172.21.15.86:0/1495635873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8dfb400640 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.825+0000 7f8e01000640 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09de9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.826+0000 7f8dfb400640 1 -- 172.21.15.86:0/1495635873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 msgr2=0x560f09de9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.826+0000 7f8dfb400640 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09de9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.826+0000 7f8dfb400640 1 -- 172.21.15.86:0/1495635873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 msgr2=0x560f09de26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:00.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.826+0000 7f8dfb400640 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09de26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.826+0000 7f8dfb400640 1 -- 172.21.15.86:0/1495635873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f09a513f0 con 0x560f09dd8b80 2024-04-17T07:22:00.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.826+0000 7f8e01000640 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09de9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:00.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.826+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09de26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:00.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.827+0000 7f8dfb400640 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de2c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009860 tx=0x7f8de4009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9365f80a49697546 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:00.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.827+0000 7f8dea000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:22:00.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.827+0000 7f8dea000640 1 -- 172.21.15.86:0/1495635873 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de40bda90 con 0x560f09dd8b80 2024-04-17T07:22:00.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.827+0000 7f8dea000640 1 -- 172.21.15.86:0/1495635873 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8de40ba5c0 con 0x560f09dd8b80 2024-04-17T07:22:00.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.828+0000 7f8e05be1740 1 -- 172.21.15.86:0/1495635873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df4004d20 con 0x560f09dd8b80 2024-04-17T07:22:00.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.828+0000 7f8dea000640 1 -- 172.21.15.86:0/1495635873 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8de40044e0 con 0x560f09dd8b80 2024-04-17T07:22:00.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.828+0000 7f8e05be1740 10 client.26380 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:00.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.828+0000 7f8e05be1740 1 -- 172.21.15.86:0/1495635873 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f09a110f0 con 0x560f09dd8b80 2024-04-17T07:22:00.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.828+0000 7f8dea000640 1 -- 172.21.15.86:0/1495635873 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df4004d20 con 0x560f09dd8b80 2024-04-17T07:22:00.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.829+0000 7f8dea000640 1 -- 172.21.15.86:0/1495635873 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8de403c030 con 0x560f09dd8b80 2024-04-17T07:22:00.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.829+0000 7f8e05be1740 10 client.26380 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.829+0000 7f8e05be1740 10 client.26380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.829+0000 7f8e05be1740 20 client.26380 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.829+0000 7f8e05be1740 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f09dee4e0 0x560f09df0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.829+0000 7f8e05be1740 4 client.26380 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.829+0000 7f8e05be1740 1 -- 172.21.15.86:0/1495635873 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f09d4cb40 con 0x560f09dee4e0 2024-04-17T07:22:00.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.830+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f09dee4e0 0x560f09df0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:00 smithi086 ceph-mon[31733]: pgmap v669: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 14 MiB/s wr, 749 op/s 2024-04-17T07:22:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.832+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f09dee4e0 0x560f09df0970 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cff6d5796cc1006 server_cookie=78613d048e56b486 in_seq=0 out_seq=0 2024-04-17T07:22:00.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.832+0000 7f8dea000640 10 client.26380 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.833+0000 7f8dea000640 1 -- 172.21.15.86:0/1495635873 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f09d4cb40 con 0x560f09dee4e0 2024-04-17T07:22:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.833+0000 7f8dea000640 10 client.26380 handle_command_reply: tid=0 2024-04-17T07:22:00.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.833+0000 7f8e05be1740 1 client.26380 shutdown 2024-04-17T07:22:00.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.834+0000 7f8e05be1740 1 -- 172.21.15.86:0/1495635873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 msgr2=0x560f09de2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:00.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.834+0000 7f8e05be1740 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de2c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009860 tx=0x7f8de4009490 comp rx=0 tx=0).stop 2024-04-17T07:22:00.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.834+0000 7f8e05be1740 1 -- 172.21.15.86:0/1495635873 shutdown_connections 2024-04-17T07:22:00.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.834+0000 7f8e05be1740 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f09dee4e0 0x560f09df0970 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.834+0000 7f8e05be1740 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09de9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.834+0000 7f8e05be1740 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de2c20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.834+0000 7f8e05be1740 1 --2- 172.21.15.86:0/1495635873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09de26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.834+0000 7f8e05be1740 1 -- 172.21.15.86:0/1495635873 >> 172.21.15.86:0/1495635873 conn(0x560f09dd3700 msgr2=0x560f09dd2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:00.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.834+0000 7f8dea000640 0 client.26380 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:00.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.835+0000 7f8e05be1740 1 -- 172.21.15.86:0/1495635873 shutdown_connections 2024-04-17T07:22:00.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.835+0000 7f8e05be1740 1 -- 172.21.15.86:0/1495635873 wait complete. 2024-04-17T07:22:00.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.835+0000 7f8e05be1740 20 client.26380 trim_cache size 0 max 16384 2024-04-17T07:22:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.878+0000 7f8e05be1740 1 Processor -- start 2024-04-17T07:22:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.878+0000 7f8e05be1740 1 -- start start 2024-04-17T07:22:00.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.878+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09dce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.878+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.878+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09dd2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.878+0000 7f8e05be1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b9910 con 0x560f09dd8190 2024-04-17T07:22:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.878+0000 7f8e05be1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1bb450 con 0x560f09dddcf0 2024-04-17T07:22:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.878+0000 7f8e05be1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b9fe0 con 0x560f09dd8b80 2024-04-17T07:22:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.879+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09dce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.879+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09dce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51788/0 (socket says 172.21.15.86:51788) 2024-04-17T07:22:00.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.879+0000 7f8dfbe00640 1 -- 172.21.15.86:0/940260340 learned_addr learned my addr 172.21.15.86:0/940260340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:00.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.879+0000 7f8dfb400640 1 --2- 172.21.15.86:0/940260340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.879+0000 7f8e01000640 1 --2- 172.21.15.86:0/940260340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09dd2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.880+0000 7f8dfb400640 1 -- 172.21.15.86:0/940260340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 msgr2=0x560f09dd2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:00.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.880+0000 7f8dfb400640 1 --2- 172.21.15.86:0/940260340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09dd2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.880+0000 7f8dfb400640 1 -- 172.21.15.86:0/940260340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 msgr2=0x560f09dce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:00.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.880+0000 7f8dfb400640 1 --2- 172.21.15.86:0/940260340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09dce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.880+0000 7f8dfb400640 1 -- 172.21.15.86:0/940260340 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f09df0fc0 con 0x560f09dd8b80 2024-04-17T07:22:00.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.881+0000 7f8dfb400640 1 --2- 172.21.15.86:0/940260340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de9c20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009230 tx=0x7f8de40bc460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=917ade78ed5e9abf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:00.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.882+0000 7f8df0c00640 1 -- 172.21.15.86:0/940260340 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de40bd8b0 con 0x560f09dd8b80 2024-04-17T07:22:00.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.882+0000 7f8df0c00640 1 -- 172.21.15.86:0/940260340 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8de40be650 con 0x560f09dd8b80 2024-04-17T07:22:00.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.882+0000 7f8df0c00640 1 -- 172.21.15.86:0/940260340 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de40c2300 con 0x560f09dd8b80 2024-04-17T07:22:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.883+0000 7f8e05be1740 1 -- 172.21.15.86:0/940260340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 msgr2=0x560f09de9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.883+0000 7f8e05be1740 1 --2- 172.21.15.86:0/940260340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de9c20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009230 tx=0x7f8de40bc460 comp rx=0 tx=0).stop 2024-04-17T07:22:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.883+0000 7f8e05be1740 1 -- 172.21.15.86:0/940260340 shutdown_connections 2024-04-17T07:22:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.883+0000 7f8e05be1740 1 --2- 172.21.15.86:0/940260340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dddcf0 0x560f09dd2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.883+0000 7f8e05be1740 1 --2- 172.21.15.86:0/940260340 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de9c20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.883+0000 7f8e05be1740 1 --2- 172.21.15.86:0/940260340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dd8190 0x560f09dce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.883+0000 7f8e05be1740 1 -- 172.21.15.86:0/940260340 >> 172.21.15.86:0/940260340 conn(0x560f09dd3700 msgr2=0x560f09dd2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:00.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.884+0000 7f8e05be1740 1 -- 172.21.15.86:0/940260340 shutdown_connections 2024-04-17T07:22:00.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.884+0000 7f8e05be1740 1 -- 172.21.15.86:0/940260340 wait complete. 2024-04-17T07:22:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.885+0000 7f8e05be1740 1 -- start start 2024-04-17T07:22:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.885+0000 7f8e05be1740 1 Processor -- start 2024-04-17T07:22:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.886+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dd8190 0x560f09de9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.886+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.886+0000 7f8e05be1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dddcf0 0x560f09de7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.886+0000 7f8e05be1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1bad80 con 0x560f09dddcf0 2024-04-17T07:22:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.886+0000 7f8e05be1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1ba6b0 con 0x560f09dd8190 2024-04-17T07:22:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.886+0000 7f8e05be1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc073e30 con 0x560f09dd8b80 2024-04-17T07:22:00.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.886+0000 7f8e05be1740 10 client.0 _ll_register_callbacks cb 0x7f8de4035ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:00.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.887+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dd8190 0x560f09de9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.887+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dd8190 0x560f09de9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52918/0 (socket says 172.21.15.86:52918) 2024-04-17T07:22:00.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.887+0000 7f8dfbe00640 1 -- 172.21.15.86:0/2904393450 learned_addr learned my addr 172.21.15.86:0/2904393450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:00.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.887+0000 7f8dfb400640 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.887+0000 7f8e01000640 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dddcf0 0x560f09de7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.887+0000 7f8dfbe00640 1 -- 172.21.15.86:0/2904393450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 msgr2=0x560f09de99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:00.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.887+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.889+0000 7f8dfbe00640 1 -- 172.21.15.86:0/2904393450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dddcf0 msgr2=0x560f09de7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:00.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.889+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dddcf0 0x560f09de7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.889+0000 7f8dfbe00640 1 -- 172.21.15.86:0/2904393450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f09dcf270 con 0x560f09dd8190 2024-04-17T07:22:00.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.890+0000 7f8e01000640 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dddcf0 0x560f09de7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.891+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dd8190 0x560f09de9490 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x560f09dd47b0 tx=0x7f8dec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8629c13da5f6a970 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.891+0000 7f8dea000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:22:00.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.892+0000 7f8dea000640 1 -- 172.21.15.86:0/2904393450 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec02a450 con 0x560f09dd8190 2024-04-17T07:22:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.892+0000 7f8dea000640 1 -- 172.21.15.86:0/2904393450 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8dec02a5f0 con 0x560f09dd8190 2024-04-17T07:22:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.892+0000 7f8e05be1740 1 -- 172.21.15.86:0/2904393450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de40be650 con 0x560f09dd8190 2024-04-17T07:22:00.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.893+0000 7f8dea000640 1 -- 172.21.15.86:0/2904393450 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8dec02a830 con 0x560f09dd8190 2024-04-17T07:22:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.893+0000 7f8e05be1740 10 client.36377 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.893+0000 7f8e05be1740 1 -- 172.21.15.86:0/2904393450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f09e553a0 con 0x560f09dd8190 2024-04-17T07:22:00.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.894+0000 7f8dea000640 1 -- 172.21.15.86:0/2904393450 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8de40be650 con 0x560f09dd8190 2024-04-17T07:22:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.894+0000 7f8dea000640 1 -- 172.21.15.86:0/2904393450 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8dec033430 con 0x560f09dd8190 2024-04-17T07:22:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.894+0000 7f8e05be1740 10 client.36377 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.894+0000 7f8e05be1740 10 client.36377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.894+0000 7f8e05be1740 20 client.36377 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.894+0000 7f8e05be1740 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f09e55d80 0x560f09e581a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.894+0000 7f8e05be1740 4 client.36377 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:22:00.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.894+0000 7f8e05be1740 1 -- 172.21.15.86:0/2904393450 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f09d469b0 con 0x560f09e55d80 2024-04-17T07:22:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.895+0000 7f8dfb400640 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f09e55d80 0x560f09e581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.897+0000 7f8dfb400640 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f09e55d80 0x560f09e581a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=878ea6802e71f797 server_cookie=ba641a431a62ee65 in_seq=0 out_seq=0 2024-04-17T07:22:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.897+0000 7f8dea000640 10 client.36377 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.898+0000 7f8dea000640 1 -- 172.21.15.86:0/2904393450 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560f09d469b0 con 0x560f09e55d80 2024-04-17T07:22:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.898+0000 7f8dea000640 10 client.36377 handle_command_reply: tid=0 2024-04-17T07:22:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.898+0000 7f8e05be1740 1 client.36377 shutdown 2024-04-17T07:22:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.899+0000 7f8e05be1740 1 -- 172.21.15.86:0/2904393450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dd8190 msgr2=0x560f09de9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:00.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.899+0000 7f8e05be1740 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dd8190 0x560f09de9490 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x560f09dd47b0 tx=0x7f8dec009260 comp rx=0 tx=0).stop 2024-04-17T07:22:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.899+0000 7f8e05be1740 1 -- 172.21.15.86:0/2904393450 shutdown_connections 2024-04-17T07:22:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.899+0000 7f8e05be1740 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560f09e55d80 0x560f09e581a0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.899+0000 7f8e05be1740 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f09dddcf0 0x560f09de7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.900+0000 7f8e05be1740 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560f09dd8b80 0x560f09de99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.900+0000 7f8e05be1740 1 --2- 172.21.15.86:0/2904393450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560f09dd8190 0x560f09de9490 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.900+0000 7f8e05be1740 1 -- 172.21.15.86:0/2904393450 >> 172.21.15.86:0/2904393450 conn(0x560f09dd3700 msgr2=0x560f09dd2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:00.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.900+0000 7f8dea000640 0 client.36377 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.900+0000 7f8e05be1740 1 -- 172.21.15.86:0/2904393450 shutdown_connections 2024-04-17T07:22:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.901+0000 7f8e05be1740 1 -- 172.21.15.86:0/2904393450 wait complete. 2024-04-17T07:22:00.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.901+0000 7f8e05be1740 20 client.36377 trim_cache size 0 max 16384 2024-04-17T07:22:00.901 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:22:00.901 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:22:00.901 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:22:00.902 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:22:00.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 -- 172.21.15.86:0/105290576 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 msgr2=0x7f8de00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 0x7f8de00a6240 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x560f09dd4670 tx=0x7f8dec0046c0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 -- 172.21.15.86:0/105290576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc075fa0 msgr2=0x7f8dfc19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc075fa0 0x7f8dfc19edc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8de402c180 tx=0x7f8de40042f0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 -- 172.21.15.86:0/105290576 shutdown_connections 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 0x7f8de00a6240 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc10e570 0x7f8dfc1a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc075fa0 0x7f8dfc19edc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 --2- 172.21.15.86:0/105290576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc0749c0 0x7f8dfc19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 -- 172.21.15.86:0/105290576 >> 172.21.15.86:0/105290576 conn(0x7f8dfc0fd510 msgr2=0x7f8dfc101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.903+0000 7f8e03000640 1 -- 172.21.15.86:0/105290576 shutdown_connections 2024-04-17T07:22:00.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:00.904+0000 7f8e03000640 1 -- 172.21.15.86:0/105290576 wait complete. 2024-04-17T07:22:00.928 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:22:00.928 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:22:01.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.231+0000 7f4043800640 1 -- 172.21.15.86:0/1996879846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c1107e0 msgr2=0x7f403c112bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:01.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.231+0000 7f4043800640 1 --2- 172.21.15.86:0/1996879846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c1107e0 0x7f403c112bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4034009ca0 tx=0x7f403402c100 comp rx=0 tx=0).stop 2024-04-17T07:22:01.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.231+0000 7f4043800640 1 -- 172.21.15.86:0/1996879846 shutdown_connections 2024-04-17T07:22:01.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.231+0000 7f4043800640 1 --2- 172.21.15.86:0/1996879846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c1107e0 0x7f403c112bd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:01.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.231+0000 7f4043800640 1 --2- 172.21.15.86:0/1996879846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c1098c0 0x7f403c1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:01.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.231+0000 7f4043800640 1 --2- 172.21.15.86:0/1996879846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f403c108f60 0x7f403c109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:01.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.231+0000 7f4043800640 1 -- 172.21.15.86:0/1996879846 >> 172.21.15.86:0/1996879846 conn(0x7f403c06ee00 msgr2=0x7f403c071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:01.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.232+0000 7f4043800640 1 -- 172.21.15.86:0/1996879846 shutdown_connections 2024-04-17T07:22:01.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.232+0000 7f4043800640 1 -- 172.21.15.86:0/1996879846 wait complete. 2024-04-17T07:22:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.234+0000 7f4043800640 1 Processor -- start 2024-04-17T07:22:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.235+0000 7f4043800640 1 -- start start 2024-04-17T07:22:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.235+0000 7f4043800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c108f60 0x7f403c10bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.235+0000 7f4043800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f403c1098c0 0x7f403c10a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.235+0000 7f4043800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c1107e0 0x7f403c10a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.235+0000 7f4043800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1162e0 con 0x7f403c1098c0 2024-04-17T07:22:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.235+0000 7f4043800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f403c116160 con 0x7f403c108f60 2024-04-17T07:22:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.235+0000 7f4043800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f403c116460 con 0x7f403c1107e0 2024-04-17T07:22:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.236+0000 7f4040e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c108f60 0x7f403c10bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.236+0000 7f403be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f403c1098c0 0x7f403c10a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.236+0000 7f403be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f403c1098c0 0x7f403c10a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51810/0 (socket says 172.21.15.86:51810) 2024-04-17T07:22:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.236+0000 7f403be00640 1 -- 172.21.15.86:0/2001290347 learned_addr learned my addr 172.21.15.86:0/2001290347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:01.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.237+0000 7f403be00640 1 -- 172.21.15.86:0/2001290347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c1107e0 msgr2=0x7f403c10a770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:22:01.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.237+0000 7f403be00640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c1107e0 0x7f403c10a770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:01.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.237+0000 7f403be00640 1 -- 172.21.15.86:0/2001290347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c108f60 msgr2=0x7f403c10bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:01.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.237+0000 7f403be00640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c108f60 0x7f403c10bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:01.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.237+0000 7f403be00640 1 -- 172.21.15.86:0/2001290347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c10acb0 con 0x7f403c1098c0 2024-04-17T07:22:01.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.238+0000 7f403be00640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f403c1098c0 0x7f403c10a230 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f402800e9c0 tx=0x7f402800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:01.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.239+0000 7f4039600640 1 -- 172.21.15.86:0/2001290347 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402800cd80 con 0x7f403c1098c0 2024-04-17T07:22:01.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.239+0000 7f4039600640 1 -- 172.21.15.86:0/2001290347 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40280035f0 con 0x7f403c1098c0 2024-04-17T07:22:01.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.239+0000 7f4039600640 1 -- 172.21.15.86:0/2001290347 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028010690 con 0x7f403c1098c0 2024-04-17T07:22:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.239+0000 7f4043800640 1 -- 172.21.15.86:0/2001290347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c10afa0 con 0x7f403c1098c0 2024-04-17T07:22:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.240+0000 7f4043800640 1 -- 172.21.15.86:0/2001290347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c1af420 con 0x7f403c1098c0 2024-04-17T07:22:01.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.241+0000 7f4043800640 1 -- 172.21.15.86:0/2001290347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c1af710 con 0x7f403c1098c0 2024-04-17T07:22:01.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.250+0000 7f4039600640 1 -- 172.21.15.86:0/2001290347 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f40280037a0 con 0x7f403c1098c0 2024-04-17T07:22:01.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.252+0000 7f4039600640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40240a3cb0 0x7f40240a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:01.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.252+0000 7f4039600640 1 -- 172.21.15.86:0/2001290347 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f40280d0d40 con 0x7f403c1098c0 2024-04-17T07:22:01.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.255+0000 7f4040e00640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40240a3cb0 0x7f40240a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:01.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.258+0000 7f4039600640 1 -- 172.21.15.86:0/2001290347 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4028053ff0 con 0x7f403c1098c0 2024-04-17T07:22:01.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.258+0000 7f4040e00640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40240a3cb0 0x7f40240a6170 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4030009bd0 tx=0x7f4030005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:01.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.755+0000 7f4043800640 1 -- 172.21.15.86:0/2001290347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f403c1af8e0 con 0x7f403c1098c0 2024-04-17T07:22:01.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.757+0000 7f4039600640 1 -- 172.21.15.86:0/2001290347 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f402809d340 con 0x7f403c1098c0 2024-04-17T07:22:01.757 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:22:01.758 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:22:01.758 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:22:01.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.761+0000 7f4043800640 1 -- 172.21.15.86:0/2001290347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40240a3cb0 msgr2=0x7f40240a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:01.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.761+0000 7f4043800640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40240a3cb0 0x7f40240a6170 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4030009bd0 tx=0x7f4030005f90 comp rx=0 tx=0).stop 2024-04-17T07:22:01.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.761+0000 7f4043800640 1 -- 172.21.15.86:0/2001290347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f403c1098c0 msgr2=0x7f403c10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.761+0000 7f4043800640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f403c1098c0 0x7f403c10a230 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f402800e9c0 tx=0x7f402800ee80 comp rx=0 tx=0).stop 2024-04-17T07:22:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.762+0000 7f4043800640 1 -- 172.21.15.86:0/2001290347 shutdown_connections 2024-04-17T07:22:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.762+0000 7f4043800640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40240a3cb0 0x7f40240a6170 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.762+0000 7f4043800640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f403c1107e0 0x7f403c10a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.762+0000 7f4043800640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f403c1098c0 0x7f403c10a230 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.762+0000 7f4043800640 1 --2- 172.21.15.86:0/2001290347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c108f60 0x7f403c10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:01.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.762+0000 7f4043800640 1 -- 172.21.15.86:0/2001290347 >> 172.21.15.86:0/2001290347 conn(0x7f403c06ee00 msgr2=0x7f403c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.763+0000 7f4043800640 1 -- 172.21.15.86:0/2001290347 shutdown_connections 2024-04-17T07:22:01.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:01.763+0000 7f4043800640 1 -- 172.21.15.86:0/2001290347 wait complete. 2024-04-17T07:22:01.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:22:02.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.049+0000 7faa0a600640 1 -- 172.21.15.86:0/2106645279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa04105220 msgr2=0x7faa04107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.049+0000 7faa0a600640 1 --2- 172.21.15.86:0/2106645279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa04105220 0x7faa04107610 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009900 tx=0x7fa9f002bc30 comp rx=0 tx=0).stop 2024-04-17T07:22:02.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.049+0000 7faa0a600640 1 -- 172.21.15.86:0/2106645279 shutdown_connections 2024-04-17T07:22:02.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.049+0000 7faa0a600640 1 --2- 172.21.15.86:0/2106645279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa04107b50 0x7faa04109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.049+0000 7faa0a600640 1 --2- 172.21.15.86:0/2106645279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa04105220 0x7faa04107610 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.049+0000 7faa0a600640 1 --2- 172.21.15.86:0/2106645279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa041028f0 0x7faa04104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.049+0000 7faa0a600640 1 -- 172.21.15.86:0/2106645279 >> 172.21.15.86:0/2106645279 conn(0x7faa040fc300 msgr2=0x7faa040fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.049+0000 7faa0a600640 1 -- 172.21.15.86:0/2106645279 shutdown_connections 2024-04-17T07:22:02.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.049+0000 7faa0a600640 1 -- 172.21.15.86:0/2106645279 wait complete. 2024-04-17T07:22:02.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.050+0000 7faa0a600640 1 Processor -- start 2024-04-17T07:22:02.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.050+0000 7faa0a600640 1 -- start start 2024-04-17T07:22:02.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.050+0000 7faa0a600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa041028f0 0x7faa0419a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.050+0000 7faa0a600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa04105220 0x7faa0419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.050+0000 7faa0a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa04107b50 0x7faa041a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.050+0000 7faa0a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410d9e0 con 0x7faa04107b50 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.050+0000 7faa0a600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410d860 con 0x7faa041028f0 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.050+0000 7faa0a600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa0410db60 con 0x7faa04105220 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa09c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa04107b50 0x7faa041a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa09c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa04107b50 0x7faa041a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51840/0 (socket says 172.21.15.86:51840) 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa09c00640 1 -- 172.21.15.86:0/2648679367 learned_addr learned my addr 172.21.15.86:0/2648679367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa09200640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa041028f0 0x7faa0419a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa03e00640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa04105220 0x7faa0419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa09c00640 1 -- 172.21.15.86:0/2648679367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa04105220 msgr2=0x7faa0419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa09c00640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa04105220 0x7faa0419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa09c00640 1 -- 172.21.15.86:0/2648679367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa041028f0 msgr2=0x7faa0419a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa09c00640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa041028f0 0x7faa0419a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa09c00640 1 -- 172.21.15.86:0/2648679367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa041a1e30 con 0x7faa04107b50 2024-04-17T07:22:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.051+0000 7faa09c00640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa04107b50 0x7faa041a17c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00cad0 tx=0x7fa9fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.052+0000 7faa01600640 1 -- 172.21.15.86:0/2648679367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00ae20 con 0x7faa04107b50 2024-04-17T07:22:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.052+0000 7faa01600640 1 -- 172.21.15.86:0/2648679367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9fc005020 con 0x7faa04107b50 2024-04-17T07:22:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.052+0000 7faa0a600640 1 -- 172.21.15.86:0/2648679367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa041a2120 con 0x7faa04107b50 2024-04-17T07:22:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.052+0000 7faa01600640 1 -- 172.21.15.86:0/2648679367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00f6b0 con 0x7faa04107b50 2024-04-17T07:22:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.052+0000 7faa0a600640 1 -- 172.21.15.86:0/2648679367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa041a25f0 con 0x7faa04107b50 2024-04-17T07:22:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.055+0000 7faa0ea0d740 1 Processor -- start 2024-04-17T07:22:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.055+0000 7faa01600640 1 -- 172.21.15.86:0/2648679367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa9fc0051d0 con 0x7faa04107b50 2024-04-17T07:22:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.055+0000 7faa0ea0d740 1 -- start start 2024-04-17T07:22:02.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f1160 0x55765f0f1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1b50 0x55765f0f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f6cc0 0x55765f0f9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55765f057020 con 0x55765f0f6cc0 2024-04-17T07:22:02.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55765effd460 con 0x55765f0f1b50 2024-04-17T07:22:02.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55765f052fe0 con 0x55765f0f1160 2024-04-17T07:22:02.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f6cc0 0x55765f0f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f6cc0 0x55765f0f9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51850/0 (socket says 172.21.15.86:51850) 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa03e00640 1 -- 172.21.15.86:0/2534513083 learned_addr learned my addr 172.21.15.86:0/2534513083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa03e00640 1 --2- 172.21.15.86:0/2534513083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f1160 0x55765f0f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.056+0000 7faa09200640 1 --2- 172.21.15.86:0/2534513083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1b50 0x55765f0f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.057+0000 7faa01600640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.057+0000 7faa03e00640 1 -- 172.21.15.86:0/2534513083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f1160 msgr2=0x55765f0f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.057+0000 7faa03e00640 1 --2- 172.21.15.86:0/2534513083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f1160 0x55765f0f1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.057+0000 7faa03e00640 1 -- 172.21.15.86:0/2534513083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1b50 msgr2=0x55765f0f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.057+0000 7faa03e00640 1 --2- 172.21.15.86:0/2534513083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1b50 0x55765f0f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.057+0000 7faa03e00640 1 -- 172.21.15.86:0/2534513083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55765f005040 con 0x55765f0f6cc0 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.057+0000 7faa01600640 1 -- 172.21.15.86:0/2648679367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa9fc0d0910 con 0x7faa04107b50 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.057+0000 7faa03e00640 1 --2- 172.21.15.86:0/2534513083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f6cc0 0x55765f0f9150 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7faa04069c50 tx=0x7fa9f0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b303d52cb873cc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.057+0000 7faa09c00640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.058+0000 7fa9f6000640 1 -- 172.21.15.86:0/2534513083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0049070 con 0x55765f0f6cc0 2024-04-17T07:22:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.058+0000 7fa9f6000640 1 -- 172.21.15.86:0/2534513083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9f0004420 con 0x55765f0f6cc0 2024-04-17T07:22:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.058+0000 7fa9f6000640 1 -- 172.21.15.86:0/2534513083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0004c00 con 0x55765f0f6cc0 2024-04-17T07:22:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.058+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2534513083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f6cc0 msgr2=0x55765f0f9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.058+0000 7faa09c00640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0058e0 tx=0x7fa9fc005830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.058+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2534513083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f6cc0 0x55765f0f9150 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7faa04069c50 tx=0x7fa9f0005fd0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.059+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2534513083 shutdown_connections 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.059+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2534513083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f6cc0 0x55765f0f9150 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.059+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2534513083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1b50 0x55765f0f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.059+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2534513083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f1160 0x55765f0f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.059+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2534513083 >> 172.21.15.86:0/2534513083 conn(0x55765f0ec6d0 msgr2=0x55765f0ecad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.059+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2534513083 shutdown_connections 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.059+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2534513083 wait complete. 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.059+0000 7faa0ea0d740 1 -- start start 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.059+0000 7faa0ea0d740 1 Processor -- start 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f1160 0x55765f0fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f0fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f6cc0 0x55765f102c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55765f0f9c10 con 0x55765f0f1b50 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55765f010830 con 0x55765f0f6cc0 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55765f0f9d90 con 0x55765f0f1160 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa0ea0d740 10 client.0 _ll_register_callbacks cb 0x55765efdefb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa09200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f0fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa09200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f0fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51862/0 (socket says 172.21.15.86:51862) 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa09200640 1 -- 172.21.15.86:0/759483704 learned_addr learned my addr 172.21.15.86:0/759483704 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa03e00640 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f1160 0x55765f0fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa09200640 1 -- 172.21.15.86:0/759483704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f1160 msgr2=0x55765f0fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa09200640 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f1160 0x55765f0fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa03e00640 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f6cc0 0x55765f102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.060+0000 7faa09200640 1 -- 172.21.15.86:0/759483704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f6cc0 msgr2=0x55765f102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.061+0000 7faa09200640 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f6cc0 0x55765f102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.061+0000 7faa09200640 1 -- 172.21.15.86:0/759483704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55765f014920 con 0x55765f0f1b50 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.061+0000 7faa03e00640 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f6cc0 0x55765f102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.061+0000 7faa09200640 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f0fbca0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800cec0 tx=0x7fa9f80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e7225ea9669dce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.061+0000 7fa9eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.061+0000 7fa9eaa00640 1 -- 172.21.15.86:0/759483704 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80119f0 con 0x55765f0f1b50 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.061+0000 7fa9eaa00640 1 -- 172.21.15.86:0/759483704 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa9f8011b90 con 0x55765f0f1b50 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.061+0000 7faa0ea0d740 1 -- 172.21.15.86:0/759483704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f0004420 con 0x55765f0f1b50 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.061+0000 7fa9eaa00640 1 -- 172.21.15.86:0/759483704 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa9f8011d90 con 0x55765f0f1b50 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.062+0000 7faa0ea0d740 10 client.17391 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.062+0000 7faa0ea0d740 1 -- 172.21.15.86:0/759483704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55765efe27f0 con 0x55765f0f1b50 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.062+0000 7fa9eaa00640 1 -- 172.21.15.86:0/759483704 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9f0004420 con 0x55765f0f1b50 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.062+0000 7fa9eaa00640 1 -- 172.21.15.86:0/759483704 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa9f8018070 con 0x55765f0f1b50 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.062+0000 7faa0ea0d740 10 client.17391 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.062+0000 7faa0ea0d740 10 client.17391 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.062+0000 7faa0ea0d740 20 client.17391 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.062+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55765f107490 0x55765f1098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.062+0000 7faa0ea0d740 4 client.17391 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:02.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.062+0000 7faa0ea0d740 1 -- 172.21.15.86:0/759483704 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55765f064a40 con 0x55765f107490 2024-04-17T07:22:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.063+0000 7faa03e00640 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55765f107490 0x55765f1098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.064+0000 7fa9eaa00640 10 client.17391 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:22:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.064+0000 7faa03e00640 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55765f107490 0x55765f1098f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3a50f7a44a5a2e8 server_cookie=d8c732c198911f48 in_seq=0 out_seq=0 2024-04-17T07:22:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7fa9eaa00640 1 -- 172.21.15.86:0/759483704 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55765f064a40 con 0x55765f107490 2024-04-17T07:22:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7fa9eaa00640 10 client.17391 handle_command_reply: tid=0 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 client.17391 shutdown 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 -- 172.21.15.86:0/759483704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 msgr2=0x55765f0fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f0fbca0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800cec0 tx=0x7fa9f80039c0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 -- 172.21.15.86:0/759483704 shutdown_connections 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55765f107490 0x55765f1098f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f6cc0 0x55765f102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f0fbca0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/759483704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f1160 0x55765f0fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7fa9eaa00640 0 client.17391 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 -- 172.21.15.86:0/759483704 >> 172.21.15.86:0/759483704 conn(0x55765f0ec6d0 msgr2=0x55765f0eb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 -- 172.21.15.86:0/759483704 shutdown_connections 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 1 -- 172.21.15.86:0/759483704 wait complete. 2024-04-17T07:22:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.066+0000 7faa0ea0d740 20 client.17391 trim_cache size 0 max 16384 2024-04-17T07:22:02.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.087+0000 7faa0ea0d740 1 Processor -- start 2024-04-17T07:22:02.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.087+0000 7faa0ea0d740 1 -- start start 2024-04-17T07:22:02.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.087+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1160 0x55765f0fad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.087+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1b50 0x55765f102c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.087+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f6cc0 0x55765f0eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.087+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa041aca80 con 0x55765f0f1160 2024-04-17T07:22:02.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.087+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa041ae5c0 con 0x55765f0f1b50 2024-04-17T07:22:02.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.087+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa041ad150 con 0x55765f0f6cc0 2024-04-17T07:22:02.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.088+0000 7faa03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1160 0x55765f0fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.088+0000 7faa03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1160 0x55765f0fad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51874/0 (socket says 172.21.15.86:51874) 2024-04-17T07:22:02.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.088+0000 7faa03e00640 1 -- 172.21.15.86:0/2426862748 learned_addr learned my addr 172.21.15.86:0/2426862748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.088+0000 7faa03e00640 1 -- 172.21.15.86:0/2426862748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f6cc0 msgr2=0x55765f0eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.088+0000 7faa03e00640 1 --2- 172.21.15.86:0/2426862748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f6cc0 0x55765f0eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.088+0000 7faa03e00640 1 -- 172.21.15.86:0/2426862748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1b50 msgr2=0x55765f102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.088+0000 7faa03e00640 1 --2- 172.21.15.86:0/2426862748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1b50 0x55765f102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.088+0000 7faa03e00640 1 -- 172.21.15.86:0/2426862748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55765ed693f0 con 0x55765f0f1160 2024-04-17T07:22:02.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.089+0000 7faa03e00640 1 --2- 172.21.15.86:0/2426862748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1160 0x55765f0fad80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009900 tx=0x7fa9f002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d8cee52e8b5334 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.089+0000 7fa9f5600640 1 -- 172.21.15.86:0/2426862748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f005a070 con 0x55765f0f1160 2024-04-17T07:22:02.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.089+0000 7fa9f5600640 1 -- 172.21.15.86:0/2426862748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9f00054e0 con 0x55765f0f1160 2024-04-17T07:22:02.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.089+0000 7fa9f5600640 1 -- 172.21.15.86:0/2426862748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0049040 con 0x55765f0f1160 2024-04-17T07:22:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.090+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2426862748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1160 msgr2=0x55765f0fad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.090+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2426862748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1160 0x55765f0fad80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009900 tx=0x7fa9f002bc30 comp rx=0 tx=0).stop 2024-04-17T07:22:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.090+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2426862748 shutdown_connections 2024-04-17T07:22:02.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.090+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2426862748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f6cc0 0x55765f0eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.090+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2426862748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1b50 0x55765f102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.090+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2426862748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1160 0x55765f0fad80 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.090+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2426862748 >> 172.21.15.86:0/2426862748 conn(0x55765f0ec6d0 msgr2=0x55765f0fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.090+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2426862748 shutdown_connections 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.090+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2426862748 wait complete. 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.091+0000 7faa0ea0d740 1 -- start start 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.091+0000 7faa0ea0d740 1 Processor -- start 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.091+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1160 0x55765f1005e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.091+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f100b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.091+0000 7faa0ea0d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f6cc0 0x55765f101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.091+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa041adef0 con 0x55765f0f1b50 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.091+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa041ad820 con 0x55765f0f1160 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.091+0000 7faa0ea0d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa041aaa50 con 0x55765f0f6cc0 2024-04-17T07:22:02.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.091+0000 7faa0ea0d740 10 client.0 _ll_register_callbacks cb 0x7fa9f800cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.092+0000 7faa09200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.092+0000 7faa09200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51886/0 (socket says 172.21.15.86:51886) 2024-04-17T07:22:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.092+0000 7faa09200640 1 -- 172.21.15.86:0/2083114540 learned_addr learned my addr 172.21.15.86:0/2083114540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.092+0000 7faa09200640 1 -- 172.21.15.86:0/2083114540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f6cc0 msgr2=0x55765f101ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.092+0000 7faa09200640 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f6cc0 0x55765f101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.092+0000 7faa09200640 1 -- 172.21.15.86:0/2083114540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1160 msgr2=0x55765f1005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.092+0000 7faa09200640 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1160 0x55765f1005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.092+0000 7faa09200640 1 -- 172.21.15.86:0/2083114540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55765ee337d0 con 0x55765f0f1b50 2024-04-17T07:22:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.092+0000 7faa09200640 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f100b20 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800ce90 tx=0x7fa9f800c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f598431d685f4d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.092+0000 7fa9eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.093+0000 7fa9eaa00640 1 -- 172.21.15.86:0/2083114540 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f802d060 con 0x55765f0f1b50 2024-04-17T07:22:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.093+0000 7fa9eaa00640 1 -- 172.21.15.86:0/2083114540 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa9f8011230 con 0x55765f0f1b50 2024-04-17T07:22:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.093+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2083114540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f00054e0 con 0x55765f0f1b50 2024-04-17T07:22:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.093+0000 7fa9eaa00640 1 -- 172.21.15.86:0/2083114540 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa9f8034050 con 0x55765f0f1b50 2024-04-17T07:22:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.093+0000 7faa0ea0d740 10 client.17397 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.093+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2083114540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55765f16db60 con 0x55765f0f1b50 2024-04-17T07:22:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.094+0000 7fa9eaa00640 1 -- 172.21.15.86:0/2083114540 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9f00054e0 con 0x55765f0f1b50 2024-04-17T07:22:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.096+0000 7fa9eaa00640 1 -- 172.21.15.86:0/2083114540 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa9f8018070 con 0x55765f0f1b50 2024-04-17T07:22:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.097+0000 7faa0ea0d740 10 client.17397 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.097+0000 7faa0ea0d740 10 client.17397 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:22:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.097+0000 7faa0ea0d740 20 client.17397 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.097+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55765f16e4c0 0x55765f1708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.097+0000 7faa0ea0d740 4 client.17397 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:22:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.097+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2083114540 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55765f0fb550 con 0x55765f16e4c0 2024-04-17T07:22:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.099+0000 7faa03e00640 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55765f16e4c0 0x55765f1708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.100+0000 7faa03e00640 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55765f16e4c0 0x55765f1708e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82d728766be865dc server_cookie=f6a95fd6f904884c in_seq=0 out_seq=0 2024-04-17T07:22:02.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.100+0000 7fa9eaa00640 10 client.17397 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:22:02.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.105+0000 7fa9eaa00640 1 -- 172.21.15.86:0/2083114540 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55765f0fb550 con 0x55765f16e4c0 2024-04-17T07:22:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.105+0000 7fa9eaa00640 10 client.17397 handle_command_reply: tid=0 2024-04-17T07:22:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.105+0000 7faa0ea0d740 1 client.17397 shutdown 2024-04-17T07:22:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.105+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2083114540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 msgr2=0x55765f100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.105+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f100b20 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800ce90 tx=0x7fa9f800c6d0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.106+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2083114540 shutdown_connections 2024-04-17T07:22:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.106+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55765f16e4c0 0x55765f1708e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.106+0000 7fa9eaa00640 0 client.17397 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:22:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.106+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55765f0f6cc0 0x55765f101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.106+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55765f0f1b50 0x55765f100b20 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.106+0000 7faa0ea0d740 1 --2- 172.21.15.86:0/2083114540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55765f0f1160 0x55765f1005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.106+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2083114540 >> 172.21.15.86:0/2083114540 conn(0x55765f0ec6d0 msgr2=0x55765f0f01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.107+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2083114540 shutdown_connections 2024-04-17T07:22:02.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.107+0000 7faa0ea0d740 1 -- 172.21.15.86:0/2083114540 wait complete. 2024-04-17T07:22:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.108+0000 7faa0ea0d740 20 client.17397 trim_cache size 0 max 16384 2024-04-17T07:22:02.108 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:22:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.110+0000 7faa0a600640 1 -- 172.21.15.86:0/2648679367 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9ec0a3bb0 msgr2=0x7fa9ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.110+0000 7faa0a600640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0058e0 tx=0x7fa9fc005830 comp rx=0 tx=0).stop 2024-04-17T07:22:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.110+0000 7faa0a600640 1 -- 172.21.15.86:0/2648679367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa04107b50 msgr2=0x7faa041a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.110+0000 7faa0a600640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa04107b50 0x7faa041a17c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00cad0 tx=0x7fa9fc00cf90 comp rx=0 tx=0).stop 2024-04-17T07:22:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.111+0000 7faa0a600640 1 -- 172.21.15.86:0/2648679367 shutdown_connections 2024-04-17T07:22:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.111+0000 7faa0a600640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.111+0000 7faa0a600640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa04107b50 0x7faa041a17c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.111+0000 7faa0a600640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa04105220 0x7faa0419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.111+0000 7faa0a600640 1 --2- 172.21.15.86:0/2648679367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa041028f0 0x7faa0419a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.111+0000 7faa0a600640 1 -- 172.21.15.86:0/2648679367 >> 172.21.15.86:0/2648679367 conn(0x7faa040fc300 msgr2=0x7faa04101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.111+0000 7faa0a600640 1 -- 172.21.15.86:0/2648679367 shutdown_connections 2024-04-17T07:22:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.112+0000 7faa0a600640 1 -- 172.21.15.86:0/2648679367 wait complete. 2024-04-17T07:22:02.137 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:22:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.372+0000 7f0d3be00640 1 -- 172.21.15.86:0/3972215935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c075f40 msgr2=0x7f0d3c074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.372+0000 7f0d3be00640 1 --2- 172.21.15.86:0/3972215935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c075f40 0x7f0d3c074390 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009900 tx=0x7f0d2c02bcd0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.373+0000 7f0d3be00640 1 -- 172.21.15.86:0/3972215935 shutdown_connections 2024-04-17T07:22:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.373+0000 7f0d3be00640 1 --2- 172.21.15.86:0/3972215935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c109d20 0x7f0d3c10c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.373+0000 7f0d3be00640 1 --2- 172.21.15.86:0/3972215935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d3c074960 0x7f0d3c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.373+0000 7f0d3be00640 1 --2- 172.21.15.86:0/3972215935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c075f40 0x7f0d3c074390 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.373+0000 7f0d3be00640 1 -- 172.21.15.86:0/3972215935 >> 172.21.15.86:0/3972215935 conn(0x7f0d3c0fd400 msgr2=0x7f0d3c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.373+0000 7f0d3be00640 1 -- 172.21.15.86:0/3972215935 shutdown_connections 2024-04-17T07:22:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.373+0000 7f0d3be00640 1 -- 172.21.15.86:0/3972215935 wait complete. 2024-04-17T07:22:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.374+0000 7f0d3be00640 1 Processor -- start 2024-04-17T07:22:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.374+0000 7f0d3be00640 1 -- start start 2024-04-17T07:22:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.374+0000 7f0d3be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c074960 0x7f0d3c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.374+0000 7f0d3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c075f40 0x7f0d3c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.374+0000 7f0d3be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d3c109d20 0x7f0d3c1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.374+0000 7f0d3be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c10fbd0 con 0x7f0d3c075f40 2024-04-17T07:22:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.374+0000 7f0d3be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c10fa50 con 0x7f0d3c109d20 2024-04-17T07:22:02.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.374+0000 7f0d3be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c10fd50 con 0x7f0d3c074960 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c075f40 0x7f0d3c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c075f40 0x7f0d3c19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51898/0 (socket says 172.21.15.86:51898) 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3a000640 1 -- 172.21.15.86:0/4216536452 learned_addr learned my addr 172.21.15.86:0/4216536452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3b400640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d3c109d20 0x7f0d3c1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3aa00640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c074960 0x7f0d3c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3a000640 1 -- 172.21.15.86:0/4216536452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c074960 msgr2=0x7f0d3c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3a000640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c074960 0x7f0d3c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3a000640 1 -- 172.21.15.86:0/4216536452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d3c109d20 msgr2=0x7f0d3c1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3a000640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d3c109d20 0x7f0d3c1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3a000640 1 -- 172.21.15.86:0/4216536452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c1a1ea0 con 0x7f0d3c075f40 2024-04-17T07:22:02.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.375+0000 7f0d3aa00640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c074960 0x7f0d3c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:02.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.376+0000 7f0d3a000640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c075f40 0x7f0d3c19a7c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800d950 tx=0x7f0d2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.376+0000 7f0d33400640 1 -- 172.21.15.86:0/4216536452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28017070 con 0x7f0d3c075f40 2024-04-17T07:22:02.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.376+0000 7f0d33400640 1 -- 172.21.15.86:0/4216536452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d28009c30 con 0x7f0d3c075f40 2024-04-17T07:22:02.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.376+0000 7f0d3be00640 1 -- 172.21.15.86:0/4216536452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c1a2190 con 0x7f0d3c075f40 2024-04-17T07:22:02.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.376+0000 7f0d33400640 1 -- 172.21.15.86:0/4216536452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800f620 con 0x7f0d3c075f40 2024-04-17T07:22:02.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.376+0000 7f0d3be00640 1 -- 172.21.15.86:0/4216536452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a2730 con 0x7f0d3c075f40 2024-04-17T07:22:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d44228740 1 Processor -- start 2024-04-17T07:22:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d44228740 1 -- start start 2024-04-17T07:22:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d33400640 1 -- 172.21.15.86:0/4216536452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0d280109a0 con 0x7f0d3c075f40 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d44228740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae5b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d44228740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae5bb50 0x55c95ae60650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d44228740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae63150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d44228740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c95adc1020 con 0x55c95ae60cc0 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d44228740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c95ad67460 con 0x55c95ae5b160 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d44228740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c95adbcfe0 con 0x55c95ae5bb50 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d3a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae63150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d3a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae63150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51912/0 (socket says 172.21.15.86:51912) 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d3a000640 1 -- 172.21.15.86:0/1883489497 learned_addr learned my addr 172.21.15.86:0/1883489497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.380+0000 7f0d3b400640 1 --2- 172.21.15.86:0/1883489497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae5b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.381+0000 7f0d3aa00640 1 --2- 172.21.15.86:0/1883489497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae5bb50 0x55c95ae60650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.381+0000 7f0d3a000640 1 -- 172.21.15.86:0/1883489497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae5bb50 msgr2=0x55c95ae60650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.381+0000 7f0d3a000640 1 --2- 172.21.15.86:0/1883489497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae5bb50 0x55c95ae60650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.381+0000 7f0d3a000640 1 -- 172.21.15.86:0/1883489497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 msgr2=0x55c95ae5b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.381+0000 7f0d3a000640 1 --2- 172.21.15.86:0/1883489497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae5b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.381+0000 7f0d3a000640 1 -- 172.21.15.86:0/1883489497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c95ad6f040 con 0x55c95ae60cc0 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.381+0000 7f0d3a000640 1 --2- 172.21.15.86:0/1883489497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae63150 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0d280d85d0 tx=0x7f0d280d8a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b8e7089daf7928 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.382+0000 7f0d27400640 1 -- 172.21.15.86:0/1883489497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d280e4070 con 0x55c95ae60cc0 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.382+0000 7f0d27400640 1 -- 172.21.15.86:0/1883489497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d280d0ce0 con 0x55c95ae60cc0 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.382+0000 7f0d33400640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d200a3c30 0x7f0d200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.382+0000 7f0d27400640 1 -- 172.21.15.86:0/1883489497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d280d6a50 con 0x55c95ae60cc0 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.382+0000 7f0d33400640 1 -- 172.21.15.86:0/4216536452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0d280d0400 con 0x7f0d3c075f40 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.382+0000 7f0d3b400640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d200a3c30 0x7f0d200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.383+0000 7f0d44228740 1 -- 172.21.15.86:0/1883489497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 msgr2=0x55c95ae63150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.383+0000 7f0d44228740 1 --2- 172.21.15.86:0/1883489497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae63150 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0d280d85d0 tx=0x7f0d280d8a90 comp rx=0 tx=0).stop 2024-04-17T07:22:02.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.383+0000 7f0d3b400640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d200a3c30 0x7f0d200a60f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55c95ae5c3b0 tx=0x7f0d34009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.383+0000 7f0d44228740 1 -- 172.21.15.86:0/1883489497 shutdown_connections 2024-04-17T07:22:02.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.383+0000 7f0d44228740 1 --2- 172.21.15.86:0/1883489497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae63150 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.383+0000 7f0d44228740 1 --2- 172.21.15.86:0/1883489497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae5bb50 0x55c95ae60650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.383+0000 7f0d44228740 1 --2- 172.21.15.86:0/1883489497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae5b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.383+0000 7f0d44228740 1 -- 172.21.15.86:0/1883489497 >> 172.21.15.86:0/1883489497 conn(0x55c95ae566d0 msgr2=0x55c95ae56ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.383+0000 7f0d44228740 1 -- 172.21.15.86:0/1883489497 shutdown_connections 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.383+0000 7f0d44228740 1 -- 172.21.15.86:0/1883489497 wait complete. 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.384+0000 7f0d44228740 1 -- start start 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.384+0000 7f0d44228740 1 Processor -- start 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d44228740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae65710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d44228740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5bb50 0x55c95ae65c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d44228740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 0x55c95ae6cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d44228740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c95ae63c70 con 0x55c95ae5bb50 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d44228740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c95ad7a830 con 0x55c95ae5b160 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d44228740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c95ae63df0 con 0x55c95ae60cc0 2024-04-17T07:22:02.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d44228740 10 client.0 _ll_register_callbacks cb 0x55c95ad48fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d3a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5bb50 0x55c95ae65c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d3a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5bb50 0x55c95ae65c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51928/0 (socket says 172.21.15.86:51928) 2024-04-17T07:22:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d3a000640 1 -- 172.21.15.86:0/3598308240 learned_addr learned my addr 172.21.15.86:0/3598308240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d3aa00640 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae65710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.385+0000 7f0d3b400640 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 0x55c95ae6cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.386+0000 7f0d3a000640 1 -- 172.21.15.86:0/3598308240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 msgr2=0x55c95ae6cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.386+0000 7f0d3a000640 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 0x55c95ae6cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.386+0000 7f0d3a000640 1 -- 172.21.15.86:0/3598308240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 msgr2=0x55c95ae65710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.386+0000 7f0d3a000640 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae65710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.386+0000 7f0d3a000640 1 -- 172.21.15.86:0/3598308240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c95ad7e920 con 0x55c95ae5bb50 2024-04-17T07:22:02.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.386+0000 7f0d3a000640 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5bb50 0x55c95ae65c50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0d280058c0 tx=0x7f0d28005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf1d3ded352b9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.386+0000 7f0d1be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.387+0000 7f0d1be00640 1 -- 172.21.15.86:0/3598308240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28020070 con 0x55c95ae5bb50 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.387+0000 7f0d1be00640 1 -- 172.21.15.86:0/3598308240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0d280a0420 con 0x55c95ae5bb50 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.387+0000 7f0d44228740 1 -- 172.21.15.86:0/3598308240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d280d0ce0 con 0x55c95ae5bb50 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.387+0000 7f0d1be00640 1 -- 172.21.15.86:0/3598308240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d280985e0 con 0x55c95ae5bb50 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.387+0000 7f0d44228740 10 client.17409 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.387+0000 7f0d44228740 1 -- 172.21.15.86:0/3598308240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c95ad4c7f0 con 0x55c95ae5bb50 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.387+0000 7f0d1be00640 1 -- 172.21.15.86:0/3598308240 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d280d0ce0 con 0x55c95ae5bb50 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.387+0000 7f0d1be00640 1 -- 172.21.15.86:0/3598308240 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0d280989b0 con 0x55c95ae5bb50 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.387+0000 7f0d44228740 10 client.17409 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.387+0000 7f0d44228740 10 client.17409 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.388+0000 7f0d44228740 20 client.17409 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.388+0000 7f0d44228740 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c95ae70d80 0x55c95ae73170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.388+0000 7f0d44228740 4 client.17409 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.388+0000 7f0d44228740 1 -- 172.21.15.86:0/3598308240 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c95adcea40 con 0x55c95ae70d80 2024-04-17T07:22:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.388+0000 7f0d3aa00640 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c95ae70d80 0x55c95ae73170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.389+0000 7f0d1be00640 10 client.17409 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:22:02.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.389+0000 7f0d3aa00640 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c95ae70d80 0x55c95ae73170 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=745111c39b87ec91 server_cookie=ae1b42ca471272d in_seq=0 out_seq=0 2024-04-17T07:22:02.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.391+0000 7f0d1be00640 1 -- 172.21.15.86:0/3598308240 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c95adcea40 con 0x55c95ae70d80 2024-04-17T07:22:02.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.391+0000 7f0d1be00640 10 client.17409 handle_command_reply: tid=0 2024-04-17T07:22:02.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.391+0000 7f0d44228740 1 client.17409 shutdown 2024-04-17T07:22:02.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.391+0000 7f0d44228740 1 -- 172.21.15.86:0/3598308240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5bb50 msgr2=0x55c95ae65c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.391+0000 7f0d44228740 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5bb50 0x55c95ae65c50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0d280058c0 tx=0x7f0d28005f70 comp rx=0 tx=0).stop 2024-04-17T07:22:02.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.392+0000 7f0d44228740 1 -- 172.21.15.86:0/3598308240 shutdown_connections 2024-04-17T07:22:02.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.392+0000 7f0d44228740 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c95ae70d80 0x55c95ae73170 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.392+0000 7f0d44228740 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 0x55c95ae6cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.392+0000 7f0d44228740 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5bb50 0x55c95ae65c50 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.392+0000 7f0d44228740 1 --2- 172.21.15.86:0/3598308240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae65710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.392+0000 7f0d44228740 1 -- 172.21.15.86:0/3598308240 >> 172.21.15.86:0/3598308240 conn(0x55c95ae566d0 msgr2=0x55c95ae55930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.392+0000 7f0d1be00640 0 client.17409 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:22:02.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.392+0000 7f0d44228740 1 -- 172.21.15.86:0/3598308240 shutdown_connections 2024-04-17T07:22:02.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.392+0000 7f0d44228740 1 -- 172.21.15.86:0/3598308240 wait complete. 2024-04-17T07:22:02.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.392+0000 7f0d44228740 20 client.17409 trim_cache size 0 max 16384 2024-04-17T07:22:02.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.408+0000 7f0d44228740 1 Processor -- start 2024-04-17T07:22:02.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.408+0000 7f0d44228740 1 -- start start 2024-04-17T07:22:02.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.408+0000 7f0d44228740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae5d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.408+0000 7f0d44228740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae5bb50 0x55c95ae6cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.408+0000 7f0d44228740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae5d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.408+0000 7f0d44228740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1acb80 con 0x55c95ae60cc0 2024-04-17T07:22:02.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.408+0000 7f0d44228740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ae6c0 con 0x55c95ae5b160 2024-04-17T07:22:02.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.408+0000 7f0d44228740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ad250 con 0x55c95ae5bb50 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae5d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae5d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51936/0 (socket says 172.21.15.86:51936) 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3b400640 1 -- 172.21.15.86:0/1140383200 learned_addr learned my addr 172.21.15.86:0/1140383200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3aa00640 1 --2- 172.21.15.86:0/1140383200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae5d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3a000640 1 --2- 172.21.15.86:0/1140383200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae5bb50 0x55c95ae6cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3b400640 1 -- 172.21.15.86:0/1140383200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae5bb50 msgr2=0x55c95ae6cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3b400640 1 --2- 172.21.15.86:0/1140383200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae5bb50 0x55c95ae6cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3b400640 1 -- 172.21.15.86:0/1140383200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 msgr2=0x55c95ae5d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3b400640 1 --2- 172.21.15.86:0/1140383200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae5d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3b400640 1 -- 172.21.15.86:0/1140383200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c95aad33f0 con 0x55c95ae60cc0 2024-04-17T07:22:02.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.409+0000 7f0d3b400640 1 --2- 172.21.15.86:0/1140383200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae5d630 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0d3401e450 tx=0x7f0d34019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663f3c0d6a03f587 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.410+0000 7f0d26a00640 1 -- 172.21.15.86:0/1140383200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34029430 con 0x55c95ae60cc0 2024-04-17T07:22:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.410+0000 7f0d26a00640 1 -- 172.21.15.86:0/1140383200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d34029a80 con 0x55c95ae60cc0 2024-04-17T07:22:02.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.410+0000 7f0d26a00640 1 -- 172.21.15.86:0/1140383200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34011bc0 con 0x55c95ae60cc0 2024-04-17T07:22:02.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.410+0000 7f0d44228740 1 -- 172.21.15.86:0/1140383200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 msgr2=0x55c95ae5d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.410+0000 7f0d44228740 1 --2- 172.21.15.86:0/1140383200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae5d630 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0d3401e450 tx=0x7f0d34019000 comp rx=0 tx=0).stop 2024-04-17T07:22:02.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.411+0000 7f0d44228740 1 -- 172.21.15.86:0/1140383200 shutdown_connections 2024-04-17T07:22:02.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.411+0000 7f0d44228740 1 --2- 172.21.15.86:0/1140383200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae60cc0 0x55c95ae5d630 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.411+0000 7f0d44228740 1 --2- 172.21.15.86:0/1140383200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae5bb50 0x55c95ae6cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.411+0000 7f0d44228740 1 --2- 172.21.15.86:0/1140383200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5b160 0x55c95ae5d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.411+0000 7f0d44228740 1 -- 172.21.15.86:0/1140383200 >> 172.21.15.86:0/1140383200 conn(0x55c95ae566d0 msgr2=0x55c95ae65710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.411+0000 7f0d44228740 1 -- 172.21.15.86:0/1140383200 shutdown_connections 2024-04-17T07:22:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.411+0000 7f0d44228740 1 -- 172.21.15.86:0/1140383200 wait complete. 2024-04-17T07:22:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.411+0000 7f0d44228740 1 -- start start 2024-04-17T07:22:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.411+0000 7f0d44228740 1 Processor -- start 2024-04-17T07:22:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d44228740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5b160 0x55c95ae69f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d44228740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5bb50 0x55c95ae6a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d44228740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 0x55c95ae6b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d44228740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1adff0 con 0x55c95ae5b160 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d44228740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ad920 con 0x55c95ae5bb50 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d44228740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1aab50 con 0x55c95ae60cc0 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d44228740 10 client.0 _ll_register_callbacks cb 0x7f0d280d8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5b160 0x55c95ae69f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5b160 0x55c95ae69f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51944/0 (socket says 172.21.15.86:51944) 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d3aa00640 1 -- 172.21.15.86:0/4187903378 learned_addr learned my addr 172.21.15.86:0/4187903378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d3a000640 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5bb50 0x55c95ae6a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d3aa00640 1 -- 172.21.15.86:0/4187903378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 msgr2=0x55c95ae6b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d3b400640 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 0x55c95ae6b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.412+0000 7f0d3aa00640 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 0x55c95ae6b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.413+0000 7f0d3aa00640 1 -- 172.21.15.86:0/4187903378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5bb50 msgr2=0x55c95ae6a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.413+0000 7f0d3aa00640 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5bb50 0x55c95ae6a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.413+0000 7f0d3aa00640 1 -- 172.21.15.86:0/4187903378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c95ab9d7d0 con 0x55c95ae5b160 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.413+0000 7f0d3a000640 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5bb50 0x55c95ae6a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.413+0000 7f0d3b400640 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 0x55c95ae6b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.413+0000 7f0d3aa00640 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5b160 0x55c95ae69f80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c005c40 tx=0x7f0d2c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1494752f6ae184fe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.413+0000 7f0d1be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.413+0000 7f0d1be00640 1 -- 172.21.15.86:0/4187903378 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c02cb90 con 0x55c95ae5b160 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.413+0000 7f0d1be00640 1 -- 172.21.15.86:0/4187903378 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0d2c02cd30 con 0x55c95ae5b160 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.413+0000 7f0d44228740 1 -- 172.21.15.86:0/4187903378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d34029a80 con 0x55c95ae5b160 2024-04-17T07:22:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d1be00640 1 -- 172.21.15.86:0/4187903378 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d2c054050 con 0x55c95ae5b160 2024-04-17T07:22:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d44228740 10 client.17415 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d44228740 1 -- 172.21.15.86:0/4187903378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c95ae6cc10 con 0x55c95ae5b160 2024-04-17T07:22:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d1be00640 1 -- 172.21.15.86:0/4187903378 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d34029a80 con 0x55c95ae5b160 2024-04-17T07:22:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d1be00640 1 -- 172.21.15.86:0/4187903378 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0d2c004bb0 con 0x55c95ae5b160 2024-04-17T07:22:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d44228740 10 client.17415 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d44228740 10 client.17415 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:22:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d44228740 20 client.17415 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d44228740 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c95aed7520 0x55c95aed9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d44228740 4 client.17415 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.414+0000 7f0d44228740 1 -- 172.21.15.86:0/4187903378 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c95adc88b0 con 0x55c95aed7520 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.415+0000 7f0d3a000640 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c95aed7520 0x55c95aed9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.416+0000 7f0d1be00640 10 client.17415 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.416+0000 7f0d3a000640 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c95aed7520 0x55c95aed9940 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32b53f0ce7a89812 server_cookie=52c3daa5a5813fd5 in_seq=0 out_seq=0 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.416+0000 7f0d1be00640 1 -- 172.21.15.86:0/4187903378 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c95adc88b0 con 0x55c95aed7520 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.416+0000 7f0d1be00640 10 client.17415 handle_command_reply: tid=0 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.416+0000 7f0d44228740 1 client.17415 shutdown 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 1 -- 172.21.15.86:0/4187903378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5b160 msgr2=0x55c95ae69f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5b160 0x55c95ae69f80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c005c40 tx=0x7f0d2c004340 comp rx=0 tx=0).stop 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 1 -- 172.21.15.86:0/4187903378 shutdown_connections 2024-04-17T07:22:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55c95aed7520 0x55c95aed9940 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c95ae60cc0 0x55c95ae6b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c95ae5bb50 0x55c95ae6a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 1 --2- 172.21.15.86:0/4187903378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c95ae5b160 0x55c95ae69f80 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d1be00640 0 client.17415 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:22:02.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 1 -- 172.21.15.86:0/4187903378 >> 172.21.15.86:0/4187903378 conn(0x55c95ae566d0 msgr2=0x55c95ae65710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 1 -- 172.21.15.86:0/4187903378 shutdown_connections 2024-04-17T07:22:02.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 1 -- 172.21.15.86:0/4187903378 wait complete. 2024-04-17T07:22:02.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.417+0000 7f0d44228740 20 client.17415 trim_cache size 0 max 16384 2024-04-17T07:22:02.419 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:22:02.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.418+0000 7f0d3be00640 1 -- 172.21.15.86:0/4216536452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d200a3c30 msgr2=0x7f0d200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.418+0000 7f0d3be00640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d200a3c30 0x7f0d200a60f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55c95ae5c3b0 tx=0x7f0d34009500 comp rx=0 tx=0).stop 2024-04-17T07:22:02.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.418+0000 7f0d3be00640 1 -- 172.21.15.86:0/4216536452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c075f40 msgr2=0x7f0d3c19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.418+0000 7f0d3be00640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c075f40 0x7f0d3c19a7c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800d950 tx=0x7f0d2800de10 comp rx=0 tx=0).stop 2024-04-17T07:22:02.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.418+0000 7f0d3be00640 1 -- 172.21.15.86:0/4216536452 shutdown_connections 2024-04-17T07:22:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.418+0000 7f0d3be00640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d200a3c30 0x7f0d200a60f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.418+0000 7f0d3be00640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d3c109d20 0x7f0d3c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.418+0000 7f0d3be00640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d3c075f40 0x7f0d3c19a7c0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.418+0000 7f0d3be00640 1 --2- 172.21.15.86:0/4216536452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c074960 0x7f0d3c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.418+0000 7f0d3be00640 1 -- 172.21.15.86:0/4216536452 >> 172.21.15.86:0/4216536452 conn(0x7f0d3c0fd400 msgr2=0x7f0d3c105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.419+0000 7f0d3be00640 1 -- 172.21.15.86:0/4216536452 shutdown_connections 2024-04-17T07:22:02.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.419+0000 7f0d3be00640 1 -- 172.21.15.86:0/4216536452 wait complete. 2024-04-17T07:22:02.436 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:22:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.707+0000 7f92d3200640 1 -- 172.21.15.86:0/42074629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92cc101e90 msgr2=0x7f92cc1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.707+0000 7f92d3200640 1 --2- 172.21.15.86:0/42074629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92cc101e90 0x7f92cc1022b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009ca0 tx=0x7f92c802c100 comp rx=0 tx=0).stop 2024-04-17T07:22:02.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.708+0000 7f92d3200640 1 -- 172.21.15.86:0/42074629 shutdown_connections 2024-04-17T07:22:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.708+0000 7f92d3200640 1 --2- 172.21.15.86:0/42074629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92cc1039a0 0x7f92cc10a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.708+0000 7f92d3200640 1 --2- 172.21.15.86:0/42074629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92cc102fe0 0x7f92cc103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.708+0000 7f92d3200640 1 --2- 172.21.15.86:0/42074629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92cc101e90 0x7f92cc1022b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.708+0000 7f92d3200640 1 -- 172.21.15.86:0/42074629 >> 172.21.15.86:0/42074629 conn(0x7f92cc0fd400 msgr2=0x7f92cc0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.709+0000 7f92d3200640 1 -- 172.21.15.86:0/42074629 shutdown_connections 2024-04-17T07:22:02.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.709+0000 7f92d3200640 1 -- 172.21.15.86:0/42074629 wait complete. 2024-04-17T07:22:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.710+0000 7f92d3200640 1 Processor -- start 2024-04-17T07:22:02.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.710+0000 7f92d3200640 1 -- start start 2024-04-17T07:22:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.711+0000 7f92d3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92cc101e90 0x7f92cc19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.711+0000 7f92d3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92cc102fe0 0x7f92cc19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.711+0000 7f92d3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92cc1039a0 0x7f92cc1a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.711+0000 7f92d3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc10b860 con 0x7f92cc1039a0 2024-04-17T07:22:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.711+0000 7f92d3200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc10b6e0 con 0x7f92cc101e90 2024-04-17T07:22:02.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.711+0000 7f92d3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc10b9e0 con 0x7f92cc102fe0 2024-04-17T07:22:02.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.711+0000 7f92d2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92cc1039a0 0x7f92cc1a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.711+0000 7f92d2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92cc1039a0 0x7f92cc1a5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51950/0 (socket says 172.21.15.86:51950) 2024-04-17T07:22:02.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.711+0000 7f92d2800640 1 -- 172.21.15.86:0/2245684362 learned_addr learned my addr 172.21.15.86:0/2245684362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.712+0000 7f92d1e00640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92cc101e90 0x7f92cc19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.712+0000 7f92d1400640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92cc102fe0 0x7f92cc19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.712+0000 7f92d2800640 1 -- 172.21.15.86:0/2245684362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92cc102fe0 msgr2=0x7f92cc19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.712+0000 7f92d2800640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92cc102fe0 0x7f92cc19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.712+0000 7f92d2800640 1 -- 172.21.15.86:0/2245684362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92cc101e90 msgr2=0x7f92cc19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.712+0000 7f92d2800640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92cc101e90 0x7f92cc19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.712+0000 7f92d2800640 1 -- 172.21.15.86:0/2245684362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc1a62c0 con 0x7f92cc1039a0 2024-04-17T07:22:02.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.712+0000 7f92d1400640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92cc102fe0 0x7f92cc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:02.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.713+0000 7f92d2800640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92cc1039a0 0x7f92cc1a5c50 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f92c000c970 tx=0x7f92c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.714+0000 7f92c6a00640 1 -- 172.21.15.86:0/2245684362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0007bb0 con 0x7f92cc1039a0 2024-04-17T07:22:02.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.714+0000 7f92c6a00640 1 -- 172.21.15.86:0/2245684362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92c0007d50 con 0x7f92cc1039a0 2024-04-17T07:22:02.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.714+0000 7f92c6a00640 1 -- 172.21.15.86:0/2245684362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0004e10 con 0x7f92cc1039a0 2024-04-17T07:22:02.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.715+0000 7f92d3200640 1 -- 172.21.15.86:0/2245684362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92cc1a65b0 con 0x7f92cc1039a0 2024-04-17T07:22:02.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.715+0000 7f92d3200640 1 -- 172.21.15.86:0/2245684362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92cc1a6ab0 con 0x7f92cc1039a0 2024-04-17T07:22:02.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.720+0000 7f92d75a1740 1 Processor -- start 2024-04-17T07:22:02.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.720+0000 7f92c6a00640 1 -- 172.21.15.86:0/2245684362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f92c0005020 con 0x7f92cc1039a0 2024-04-17T07:22:02.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.720+0000 7f92d75a1740 1 -- start start 2024-04-17T07:22:02.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.721+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4160 0x5563cd9f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.721+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.721+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f9cc0 0x5563cd9fc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.721+0000 7f92d75a1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5563cd95a020 con 0x5563cd9f4b50 2024-04-17T07:22:02.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.721+0000 7f92d75a1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5563cd900460 con 0x5563cd9f4160 2024-04-17T07:22:02.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.721+0000 7f92d75a1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563cd955fe0 con 0x5563cd9f9cc0 2024-04-17T07:22:02.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.721+0000 7f92d1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.721+0000 7f92d1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9f9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51960/0 (socket says 172.21.15.86:51960) 2024-04-17T07:22:02.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.721+0000 7f92d1e00640 1 -- 172.21.15.86:0/4189473115 learned_addr learned my addr 172.21.15.86:0/4189473115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.722+0000 7f92d1400640 1 --2- 172.21.15.86:0/4189473115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4160 0x5563cd9f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.722+0000 7f92d1e00640 1 -- 172.21.15.86:0/4189473115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f9cc0 msgr2=0x5563cd9fc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.722+0000 7f92d1e00640 1 --2- 172.21.15.86:0/4189473115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f9cc0 0x5563cd9fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.722+0000 7f92d1e00640 1 -- 172.21.15.86:0/4189473115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4160 msgr2=0x5563cd9f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.722+0000 7f92d1e00640 1 --2- 172.21.15.86:0/4189473115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4160 0x5563cd9f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.722+0000 7f92d1e00640 1 -- 172.21.15.86:0/4189473115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563cd908040 con 0x5563cd9f4b50 2024-04-17T07:22:02.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.722+0000 7f92d1400640 1 --2- 172.21.15.86:0/4189473115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4160 0x5563cd9f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:02.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.723+0000 7f92d1e00640 1 --2- 172.21.15.86:0/4189473115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9f9650 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f92c8037040 tx=0x7f92c80099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe81edc98e7d76a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.723+0000 7f92c6a00640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92b40a3d40 0x7f92b40a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.723+0000 7f92c6a00640 1 -- 172.21.15.86:0/2245684362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f92c00d06f0 con 0x7f92cc1039a0 2024-04-17T07:22:02.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.723+0000 7f92bb000640 1 -- 172.21.15.86:0/4189473115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8043020 con 0x5563cd9f4b50 2024-04-17T07:22:02.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.723+0000 7f92bb000640 1 -- 172.21.15.86:0/4189473115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92c8003970 con 0x5563cd9f4b50 2024-04-17T07:22:02.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.723+0000 7f92bb000640 1 -- 172.21.15.86:0/4189473115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8039040 con 0x5563cd9f4b50 2024-04-17T07:22:02.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.724+0000 7f92d1400640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92b40a3d40 0x7f92b40a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.724+0000 7f92d75a1740 1 -- 172.21.15.86:0/4189473115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 msgr2=0x5563cd9f9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.724+0000 7f92d75a1740 1 --2- 172.21.15.86:0/4189473115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9f9650 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f92c8037040 tx=0x7f92c80099a0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.725+0000 7f92d75a1740 1 -- 172.21.15.86:0/4189473115 shutdown_connections 2024-04-17T07:22:02.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.725+0000 7f92d75a1740 1 --2- 172.21.15.86:0/4189473115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f9cc0 0x5563cd9fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.725+0000 7f92d75a1740 1 --2- 172.21.15.86:0/4189473115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9f9650 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.725+0000 7f92d75a1740 1 --2- 172.21.15.86:0/4189473115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4160 0x5563cd9f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.725+0000 7f92d75a1740 1 -- 172.21.15.86:0/4189473115 >> 172.21.15.86:0/4189473115 conn(0x5563cd9ef6d0 msgr2=0x5563cd9efad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.726+0000 7f92d1400640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92b40a3d40 0x7f92b40a6200 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f92bc005fd0 tx=0x7f92bc00b370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.726+0000 7f92d75a1740 1 -- 172.21.15.86:0/4189473115 shutdown_connections 2024-04-17T07:22:02.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.726+0000 7f92d75a1740 1 -- 172.21.15.86:0/4189473115 wait complete. 2024-04-17T07:22:02.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.727+0000 7f92d75a1740 1 -- start start 2024-04-17T07:22:02.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.727+0000 7f92d75a1740 1 Processor -- start 2024-04-17T07:22:02.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.727+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4160 0x5563cd9fe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.727+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9febb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.727+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 0x5563cda05bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.727+0000 7f92d75a1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5563cd9fcc10 con 0x5563cd9f4b50 2024-04-17T07:22:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.728+0000 7f92d75a1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5563cd913830 con 0x5563cd9f9cc0 2024-04-17T07:22:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.728+0000 7f92d75a1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563cd9fcd90 con 0x5563cd9f4160 2024-04-17T07:22:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.728+0000 7f92d75a1740 10 client.0 _ll_register_callbacks cb 0x5563cd8e1fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.728+0000 7f92d1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9febb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.728+0000 7f92d1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9febb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51976/0 (socket says 172.21.15.86:51976) 2024-04-17T07:22:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.728+0000 7f92d1400640 1 -- 172.21.15.86:0/3383029673 learned_addr learned my addr 172.21.15.86:0/3383029673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.728+0000 7f92d1e00640 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4160 0x5563cd9fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.728+0000 7f92d2800640 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 0x5563cda05bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.729+0000 7f92d1400640 1 -- 172.21.15.86:0/3383029673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4160 msgr2=0x5563cd9fe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.729+0000 7f92d1400640 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4160 0x5563cd9fe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.729+0000 7f92d1400640 1 -- 172.21.15.86:0/3383029673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 msgr2=0x5563cda05bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.729+0000 7f92d1400640 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 0x5563cda05bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.729+0000 7f92d1400640 1 -- 172.21.15.86:0/3383029673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563cd917920 con 0x5563cd9f4b50 2024-04-17T07:22:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.729+0000 7f92d1400640 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9febb0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00b530 tx=0x7f92bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678a748e47ac0d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.729+0000 7f92af400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.730+0000 7f92af400640 1 -- 172.21.15.86:0/3383029673 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc00dd40 con 0x5563cd9f4b50 2024-04-17T07:22:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.730+0000 7f92af400640 1 -- 172.21.15.86:0/3383029673 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f92bc01d420 con 0x5563cd9f4b50 2024-04-17T07:22:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.730+0000 7f92d75a1740 1 -- 172.21.15.86:0/3383029673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c8003970 con 0x5563cd9f4b50 2024-04-17T07:22:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.730+0000 7f92af400640 1 -- 172.21.15.86:0/3383029673 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92bc005020 con 0x5563cd9f4b50 2024-04-17T07:22:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.730+0000 7f92d75a1740 10 client.17427 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.730+0000 7f92d75a1740 1 -- 172.21.15.86:0/3383029673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5563cd8e57f0 con 0x5563cd9f4b50 2024-04-17T07:22:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.731+0000 7f92af400640 1 -- 172.21.15.86:0/3383029673 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92c8003970 con 0x5563cd9f4b50 2024-04-17T07:22:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.731+0000 7f92af400640 1 -- 172.21.15.86:0/3383029673 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f92bc01d620 con 0x5563cd9f4b50 2024-04-17T07:22:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.732+0000 7f92d75a1740 10 client.17427 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.732+0000 7f92d75a1740 10 client.17427 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:22:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.732+0000 7f92d75a1740 20 client.17427 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.732+0000 7f92d75a1740 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5563cda0a190 0x5563cda0c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.732+0000 7f92d75a1740 4 client.17427 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.732+0000 7f92d75a1740 1 -- 172.21.15.86:0/3383029673 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563cd967a40 con 0x5563cda0a190 2024-04-17T07:22:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.733+0000 7f92d1e00640 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5563cda0a190 0x5563cda0c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.734+0000 7f92d1e00640 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5563cda0a190 0x5563cda0c650 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ec36d2d255da1ed server_cookie=5258f267d1c2a521 in_seq=0 out_seq=0 2024-04-17T07:22:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.734+0000 7f92af400640 10 client.17427 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:22:02.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.737+0000 7f92af400640 1 -- 172.21.15.86:0/3383029673 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5563cd967a40 con 0x5563cda0a190 2024-04-17T07:22:02.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.737+0000 7f92af400640 10 client.17427 handle_command_reply: tid=0 2024-04-17T07:22:02.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.737+0000 7f92d75a1740 1 client.17427 shutdown 2024-04-17T07:22:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.737+0000 7f92d75a1740 1 -- 172.21.15.86:0/3383029673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 msgr2=0x5563cd9febb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.737+0000 7f92d75a1740 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9febb0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00b530 tx=0x7f92bc0046c0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.737+0000 7f92d75a1740 1 -- 172.21.15.86:0/3383029673 shutdown_connections 2024-04-17T07:22:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.738+0000 7f92d75a1740 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5563cda0a190 0x5563cda0c650 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.738+0000 7f92d75a1740 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 0x5563cda05bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.738+0000 7f92d75a1740 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4b50 0x5563cd9febb0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.738+0000 7f92d75a1740 1 --2- 172.21.15.86:0/3383029673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4160 0x5563cd9fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.738+0000 7f92d75a1740 1 -- 172.21.15.86:0/3383029673 >> 172.21.15.86:0/3383029673 conn(0x5563cd9ef6d0 msgr2=0x5563cd9ee930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.738+0000 7f92af400640 0 client.17427 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:22:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.738+0000 7f92d75a1740 1 -- 172.21.15.86:0/3383029673 shutdown_connections 2024-04-17T07:22:02.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.738+0000 7f92d75a1740 1 -- 172.21.15.86:0/3383029673 wait complete. 2024-04-17T07:22:02.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.738+0000 7f92d75a1740 20 client.17427 trim_cache size 0 max 16384 2024-04-17T07:22:02.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.763+0000 7f92d75a1740 1 Processor -- start 2024-04-17T07:22:02.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.763+0000 7f92d75a1740 1 -- start start 2024-04-17T07:22:02.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.764+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 0x5563cd9fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.764+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4b50 0x5563cd9fd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.764+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f9cc0 0x5563cda05bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.764+0000 7f92d75a1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc1b0ff0 con 0x5563cd9f4160 2024-04-17T07:22:02.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.764+0000 7f92d75a1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc1b2b30 con 0x5563cd9f4b50 2024-04-17T07:22:02.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.764+0000 7f92d75a1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc1b16c0 con 0x5563cd9f9cc0 2024-04-17T07:22:02.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.764+0000 7f92d1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 0x5563cd9fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.764+0000 7f92d1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 0x5563cd9fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51988/0 (socket says 172.21.15.86:51988) 2024-04-17T07:22:02.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.764+0000 7f92d1e00640 1 -- 172.21.15.86:0/3380992583 learned_addr learned my addr 172.21.15.86:0/3380992583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.764+0000 7f92d1400640 1 --2- 172.21.15.86:0/3380992583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4b50 0x5563cd9fd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.765+0000 7f92d2800640 1 --2- 172.21.15.86:0/3380992583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f9cc0 0x5563cda05bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.765+0000 7f92d1400640 1 -- 172.21.15.86:0/3380992583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f9cc0 msgr2=0x5563cda05bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.765+0000 7f92d1400640 1 --2- 172.21.15.86:0/3380992583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f9cc0 0x5563cda05bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.765+0000 7f92d1400640 1 -- 172.21.15.86:0/3380992583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 msgr2=0x5563cd9fd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.765+0000 7f92d1400640 1 --2- 172.21.15.86:0/3380992583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 0x5563cd9fd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.765+0000 7f92d1400640 1 -- 172.21.15.86:0/3380992583 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563cd66c3f0 con 0x5563cd9f4b50 2024-04-17T07:22:02.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.766+0000 7f92d1e00640 1 --2- 172.21.15.86:0/3380992583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 0x5563cd9fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:02.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.766+0000 7f92d1400640 1 --2- 172.21.15.86:0/3380992583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4b50 0x5563cd9fd670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00d780 tx=0x7f92bc010470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700587f7362e6d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.767+0000 7f92ba600640 1 -- 172.21.15.86:0/3380992583 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc038020 con 0x5563cd9f4b50 2024-04-17T07:22:02.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.767+0000 7f92ba600640 1 -- 172.21.15.86:0/3380992583 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92bc004c70 con 0x5563cd9f4b50 2024-04-17T07:22:02.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.767+0000 7f92ba600640 1 -- 172.21.15.86:0/3380992583 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc005020 con 0x5563cd9f4b50 2024-04-17T07:22:02.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.769+0000 7f92d75a1740 1 -- 172.21.15.86:0/3380992583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4b50 msgr2=0x5563cd9fd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.769+0000 7f92d75a1740 1 --2- 172.21.15.86:0/3380992583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4b50 0x5563cd9fd670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00d780 tx=0x7f92bc010470 comp rx=0 tx=0).stop 2024-04-17T07:22:02.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.769+0000 7f92d75a1740 1 -- 172.21.15.86:0/3380992583 shutdown_connections 2024-04-17T07:22:02.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.769+0000 7f92d75a1740 1 --2- 172.21.15.86:0/3380992583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f9cc0 0x5563cda05bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.769+0000 7f92d75a1740 1 --2- 172.21.15.86:0/3380992583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f4b50 0x5563cd9fd670 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.770+0000 7f92d75a1740 1 --2- 172.21.15.86:0/3380992583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 0x5563cd9fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.770+0000 7f92d75a1740 1 -- 172.21.15.86:0/3380992583 >> 172.21.15.86:0/3380992583 conn(0x5563cd9ef6d0 msgr2=0x5563cd9f60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.771+0000 7f92d75a1740 1 -- 172.21.15.86:0/3380992583 shutdown_connections 2024-04-17T07:22:02.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.771+0000 7f92d75a1740 1 -- 172.21.15.86:0/3380992583 wait complete. 2024-04-17T07:22:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.772+0000 7f92d75a1740 1 -- start start 2024-04-17T07:22:02.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.772+0000 7f92d75a1740 1 Processor -- start 2024-04-17T07:22:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.773+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 0x5563cda031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.773+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4b50 0x5563cda03720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.773+0000 7f92d75a1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 0x5563cda04730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.773+0000 7f92d75a1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc1b2460 con 0x5563cd9f4160 2024-04-17T07:22:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.773+0000 7f92d75a1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc1b1d90 con 0x5563cd9f9cc0 2024-04-17T07:22:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.773+0000 7f92d75a1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc1aef80 con 0x5563cd9f4b50 2024-04-17T07:22:02.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.773+0000 7f92d75a1740 10 client.0 _ll_register_callbacks cb 0x7f92bc010730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4b50 0x5563cda03720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d2800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 0x5563cda04730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4b50 0x5563cda03720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45082/0 (socket says 172.21.15.86:45082) 2024-04-17T07:22:02.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d1400640 1 -- 172.21.15.86:0/4234601629 learned_addr learned my addr 172.21.15.86:0/4234601629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d1e00640 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 0x5563cda031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d2800640 1 -- 172.21.15.86:0/4234601629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4b50 msgr2=0x5563cda03720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d2800640 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4b50 0x5563cda03720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d2800640 1 -- 172.21.15.86:0/4234601629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 msgr2=0x5563cda031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d2800640 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 0x5563cda031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d2800640 1 -- 172.21.15.86:0/4234601629 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563cd7367d0 con 0x5563cd9f9cc0 2024-04-17T07:22:02.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.774+0000 7f92d1400640 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4b50 0x5563cda03720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.775+0000 7f92d2800640 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 0x5563cda04730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92c0000c00 tx=0x7f92c0005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f1f624fee63b97a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.775+0000 7f92af400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:22:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.776+0000 7f92af400640 1 -- 172.21.15.86:0/4234601629 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c001d040 con 0x5563cd9f9cc0 2024-04-17T07:22:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.776+0000 7f92af400640 1 -- 172.21.15.86:0/4234601629 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f92c009a950 con 0x5563cd9f9cc0 2024-04-17T07:22:02.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.776+0000 7f92d75a1740 1 -- 172.21.15.86:0/4234601629 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92bc004c70 con 0x5563cd9f9cc0 2024-04-17T07:22:02.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:02 smithi103 ceph-mon[35495]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 816 op/s 2024-04-17T07:22:02.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:02 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2001290347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:22:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.777+0000 7f92af400640 1 -- 172.21.15.86:0/4234601629 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92c00a0a00 con 0x5563cd9f9cc0 2024-04-17T07:22:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.777+0000 7f92d75a1740 10 client.36416 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.777+0000 7f92d75a1740 1 -- 172.21.15.86:0/4234601629 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5563cda05ec0 con 0x5563cd9f9cc0 2024-04-17T07:22:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.778+0000 7f92af400640 1 -- 172.21.15.86:0/4234601629 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92bc004c70 con 0x5563cd9f9cc0 2024-04-17T07:22:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.778+0000 7f92af400640 1 -- 172.21.15.86:0/4234601629 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f92c00a4e30 con 0x5563cd9f9cc0 2024-04-17T07:22:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.778+0000 7f92d75a1740 10 client.36416 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.778+0000 7f92d75a1740 10 client.36416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:22:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.778+0000 7f92d75a1740 20 client.36416 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:02.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.778+0000 7f92d75a1740 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5563cda70ec0 0x5563cda732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.778+0000 7f92d75a1740 4 client.36416 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:22:02.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.778+0000 7f92d75a1740 1 -- 172.21.15.86:0/4234601629 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563cd9618b0 con 0x5563cda70ec0 2024-04-17T07:22:02.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.779+0000 7f92d1e00640 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5563cda70ec0 0x5563cda732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:02.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.781+0000 7f92d1e00640 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5563cda70ec0 0x5563cda732e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86bb267f36ce7374 server_cookie=cd7514254d2a6694 in_seq=0 out_seq=0 2024-04-17T07:22:02.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.781+0000 7f92af400640 10 client.36416 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:22:02.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.781+0000 7f92af400640 1 -- 172.21.15.86:0/4234601629 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563cd9618b0 con 0x5563cda70ec0 2024-04-17T07:22:02.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.781+0000 7f92af400640 10 client.36416 handle_command_reply: tid=0 2024-04-17T07:22:02.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.781+0000 7f92d75a1740 1 client.36416 shutdown 2024-04-17T07:22:02.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.782+0000 7f92d75a1740 1 -- 172.21.15.86:0/4234601629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 msgr2=0x5563cda04730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.782+0000 7f92d75a1740 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 0x5563cda04730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f92c0000c00 tx=0x7f92c0005620 comp rx=0 tx=0).stop 2024-04-17T07:22:02.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.782+0000 7f92d75a1740 1 -- 172.21.15.86:0/4234601629 shutdown_connections 2024-04-17T07:22:02.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.782+0000 7f92d75a1740 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5563cda70ec0 0x5563cda732e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.782+0000 7f92d75a1740 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5563cd9f9cc0 0x5563cda04730 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.782+0000 7f92af400640 0 client.36416 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:22:02.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.782+0000 7f92d75a1740 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563cd9f4b50 0x5563cda03720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.782+0000 7f92d75a1740 1 --2- 172.21.15.86:0/4234601629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563cd9f4160 0x5563cda031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.782+0000 7f92d75a1740 1 -- 172.21.15.86:0/4234601629 >> 172.21.15.86:0/4234601629 conn(0x5563cd9ef6d0 msgr2=0x5563cd9fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.782+0000 7f92d75a1740 1 -- 172.21.15.86:0/4234601629 shutdown_connections 2024-04-17T07:22:02.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.783+0000 7f92d75a1740 1 -- 172.21.15.86:0/4234601629 wait complete. 2024-04-17T07:22:02.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.783+0000 7f92d75a1740 20 client.36416 trim_cache size 0 max 16384 2024-04-17T07:22:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:22:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.784+0000 7f92d3200640 1 -- 172.21.15.86:0/2245684362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92b40a3d40 msgr2=0x7f92b40a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.784+0000 7f92d3200640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92b40a3d40 0x7f92b40a6200 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f92bc005fd0 tx=0x7f92bc00b370 comp rx=0 tx=0).stop 2024-04-17T07:22:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.784+0000 7f92d3200640 1 -- 172.21.15.86:0/2245684362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92cc1039a0 msgr2=0x7f92cc1a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.784+0000 7f92d3200640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92cc1039a0 0x7f92cc1a5c50 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f92c000c970 tx=0x7f92c000ce30 comp rx=0 tx=0).stop 2024-04-17T07:22:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.784+0000 7f92d3200640 1 -- 172.21.15.86:0/2245684362 shutdown_connections 2024-04-17T07:22:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.784+0000 7f92d3200640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f92b40a3d40 0x7f92b40a6200 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.784+0000 7f92d3200640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92cc1039a0 0x7f92cc1a5c50 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.784+0000 7f92d3200640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92cc102fe0 0x7f92cc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.784+0000 7f92d3200640 1 --2- 172.21.15.86:0/2245684362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92cc101e90 0x7f92cc19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:02.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.784+0000 7f92d3200640 1 -- 172.21.15.86:0/2245684362 >> 172.21.15.86:0/2245684362 conn(0x7f92cc0fd400 msgr2=0x7f92cc106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:02.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.785+0000 7f92d3200640 1 -- 172.21.15.86:0/2245684362 shutdown_connections 2024-04-17T07:22:02.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:02.785+0000 7f92d3200640 1 -- 172.21.15.86:0/2245684362 wait complete. 2024-04-17T07:22:02.806 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:02 smithi086 ceph-mon[31733]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 816 op/s 2024-04-17T07:22:02.806 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:02 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2001290347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:22:02.809 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:22:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:02 smithi098 ceph-mon[35418]: pgmap v670: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 16 MiB/s wr, 816 op/s 2024-04-17T07:22:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:02 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2001290347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:22:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.091+0000 7fc811400640 1 -- 172.21.15.86:0/2362597241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc80c1085c0 msgr2=0x7fc80c1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.091+0000 7fc811400640 1 --2- 172.21.15.86:0/2362597241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc80c1085c0 0x7fc80c1089e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc804009ca0 tx=0x7fc80402c120 comp rx=0 tx=0).stop 2024-04-17T07:22:03.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.091+0000 7fc811400640 1 -- 172.21.15.86:0/2362597241 shutdown_connections 2024-04-17T07:22:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.091+0000 7fc811400640 1 --2- 172.21.15.86:0/2362597241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80c10a170 0x7fc80c110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.091+0000 7fc811400640 1 --2- 172.21.15.86:0/2362597241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc80c1097b0 0x7fc80c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.091+0000 7fc811400640 1 --2- 172.21.15.86:0/2362597241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc80c1085c0 0x7fc80c1089e0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.091+0000 7fc811400640 1 -- 172.21.15.86:0/2362597241 >> 172.21.15.86:0/2362597241 conn(0x7fc80c077560 msgr2=0x7fc80c0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.091+0000 7fc811400640 1 -- 172.21.15.86:0/2362597241 shutdown_connections 2024-04-17T07:22:03.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.091+0000 7fc811400640 1 -- 172.21.15.86:0/2362597241 wait complete. 2024-04-17T07:22:03.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.092+0000 7fc811400640 1 Processor -- start 2024-04-17T07:22:03.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.093+0000 7fc811400640 1 -- start start 2024-04-17T07:22:03.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.093+0000 7fc811400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc80c1085c0 0x7fc80c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.093+0000 7fc811400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc80c1097b0 0x7fc80c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.093+0000 7fc811400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80c10a170 0x7fc80c1a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.093+0000 7fc811400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c114130 con 0x7fc80c1085c0 2024-04-17T07:22:03.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.093+0000 7fc811400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c113fb0 con 0x7fc80c10a170 2024-04-17T07:22:03.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.093+0000 7fc811400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1142b0 con 0x7fc80c1097b0 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.093+0000 7fc80a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc80c1085c0 0x7fc80c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.093+0000 7fc80a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc80c1085c0 0x7fc80c19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52016/0 (socket says 172.21.15.86:52016) 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.093+0000 7fc80a800640 1 -- 172.21.15.86:0/410691276 learned_addr learned my addr 172.21.15.86:0/410691276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.094+0000 7fc809e00640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc80c1097b0 0x7fc80c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.094+0000 7fc80b200640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80c10a170 0x7fc80c1a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.094+0000 7fc80a800640 1 -- 172.21.15.86:0/410691276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc80c1097b0 msgr2=0x7fc80c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.094+0000 7fc80a800640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc80c1097b0 0x7fc80c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.094+0000 7fc80a800640 1 -- 172.21.15.86:0/410691276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80c10a170 msgr2=0x7fc80c1a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.094+0000 7fc80a800640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80c10a170 0x7fc80c1a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.094+0000 7fc80a800640 1 -- 172.21.15.86:0/410691276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c1a64b0 con 0x7fc80c1085c0 2024-04-17T07:22:03.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.094+0000 7fc80a800640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc80c1085c0 0x7fc80c19e8e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc804037040 tx=0x7fc804034cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.095+0000 7fc803400640 1 -- 172.21.15.86:0/410691276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804041020 con 0x7fc80c1085c0 2024-04-17T07:22:03.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.095+0000 7fc803400640 1 -- 172.21.15.86:0/410691276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc804034e40 con 0x7fc80c1085c0 2024-04-17T07:22:03.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.095+0000 7fc803400640 1 -- 172.21.15.86:0/410691276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804039040 con 0x7fc80c1085c0 2024-04-17T07:22:03.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.095+0000 7fc811400640 1 -- 172.21.15.86:0/410691276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c1a6740 con 0x7fc80c1085c0 2024-04-17T07:22:03.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.096+0000 7fc811400640 1 -- 172.21.15.86:0/410691276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c1a6be0 con 0x7fc80c1085c0 2024-04-17T07:22:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.100+0000 7fc803400640 1 -- 172.21.15.86:0/410691276 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc804044020 con 0x7fc80c1085c0 2024-04-17T07:22:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.102+0000 7fc803400640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc7f00a3db0 0x7fc7f00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.102+0000 7fc803400640 1 -- 172.21.15.86:0/410691276 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc8040f2a10 con 0x7fc80c1085c0 2024-04-17T07:22:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.103+0000 7fc814018740 1 Processor -- start 2024-04-17T07:22:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.103+0000 7fc809e00640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc7f00a3db0 0x7fc7f00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.103+0000 7fc814018740 1 -- start start 2024-04-17T07:22:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.103+0000 7fc814018740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d83163c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.103+0000 7fc814018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d8316990 0x5567d831b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.103+0000 7fc814018740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 0x5567d831df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.103+0000 7fc814018740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5567d827c020 con 0x5567d8315fa0 2024-04-17T07:22:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.103+0000 7fc814018740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5567d8222460 con 0x5567d831bb00 2024-04-17T07:22:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.103+0000 7fc814018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5567d8277fe0 con 0x5567d8316990 2024-04-17T07:22:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc80a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d83163c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc80a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d83163c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52020/0 (socket says 172.21.15.86:52020) 2024-04-17T07:22:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc80a800640 1 -- 172.21.15.86:0/1269922268 learned_addr learned my addr 172.21.15.86:0/1269922268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc809e00640 1 --2- 172.21.15.86:0/1269922268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d8316990 0x5567d831b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc80b200640 1 --2- 172.21.15.86:0/1269922268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 0x5567d831df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc809e00640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc7f00a3db0 0x7fc7f00a6270 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc005fd0 tx=0x7fc7fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc80a800640 1 -- 172.21.15.86:0/1269922268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d8316990 msgr2=0x5567d831b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc80a800640 1 --2- 172.21.15.86:0/1269922268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d8316990 0x5567d831b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc80a800640 1 -- 172.21.15.86:0/1269922268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 msgr2=0x5567d831df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc80a800640 1 --2- 172.21.15.86:0/1269922268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 0x5567d831df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.104+0000 7fc80a800640 1 -- 172.21.15.86:0/1269922268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567d822a040 con 0x5567d8315fa0 2024-04-17T07:22:03.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.105+0000 7fc80a800640 1 --2- 172.21.15.86:0/1269922268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d83163c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc8040c7050 tx=0x7fc8040095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dbddc1cc09cb0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.106+0000 7fc7fb400640 1 -- 172.21.15.86:0/1269922268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8040be910 con 0x5567d8315fa0 2024-04-17T07:22:03.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.106+0000 7fc7fb400640 1 -- 172.21.15.86:0/1269922268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8040c2cc0 con 0x5567d8315fa0 2024-04-17T07:22:03.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.106+0000 7fc7fb400640 1 -- 172.21.15.86:0/1269922268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8040c70f0 con 0x5567d8315fa0 2024-04-17T07:22:03.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.107+0000 7fc814018740 1 -- 172.21.15.86:0/1269922268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 msgr2=0x5567d83163c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.107+0000 7fc814018740 1 --2- 172.21.15.86:0/1269922268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d83163c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc8040c7050 tx=0x7fc8040095c0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.107+0000 7fc814018740 1 -- 172.21.15.86:0/1269922268 shutdown_connections 2024-04-17T07:22:03.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.107+0000 7fc814018740 1 --2- 172.21.15.86:0/1269922268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 0x5567d831df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.107+0000 7fc814018740 1 --2- 172.21.15.86:0/1269922268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d8316990 0x5567d831b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.107+0000 7fc814018740 1 --2- 172.21.15.86:0/1269922268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d83163c0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.107+0000 7fc814018740 1 -- 172.21.15.86:0/1269922268 >> 172.21.15.86:0/1269922268 conn(0x5567d8311510 msgr2=0x5567d8311910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.108+0000 7fc814018740 1 -- 172.21.15.86:0/1269922268 shutdown_connections 2024-04-17T07:22:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.109+0000 7fc814018740 1 -- 172.21.15.86:0/1269922268 wait complete. 2024-04-17T07:22:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.109+0000 7fc814018740 1 -- start start 2024-04-17T07:22:03.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.110+0000 7fc814018740 1 Processor -- start 2024-04-17T07:22:03.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.110+0000 7fc814018740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d8320230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.110+0000 7fc814018740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d8316990 0x5567d8320770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.110+0000 7fc814018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d831bb00 0x5567d83277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.111+0000 7fc814018740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5567d831e870 con 0x5567d8315fa0 2024-04-17T07:22:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.111+0000 7fc814018740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5567d8235830 con 0x5567d8316990 2024-04-17T07:22:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.111+0000 7fc814018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5567d831e9f0 con 0x5567d831bb00 2024-04-17T07:22:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.111+0000 7fc814018740 10 client.0 _ll_register_callbacks cb 0x5567d8203fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:03.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.111+0000 7fc80a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d8320230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.111+0000 7fc80a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d8320230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52022/0 (socket says 172.21.15.86:52022) 2024-04-17T07:22:03.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.111+0000 7fc80a800640 1 -- 172.21.15.86:0/2021115836 learned_addr learned my addr 172.21.15.86:0/2021115836 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:03.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.111+0000 7fc809e00640 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d8316990 0x5567d8320770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.111+0000 7fc80b200640 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d831bb00 0x5567d83277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.112+0000 7fc80a800640 1 -- 172.21.15.86:0/2021115836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d831bb00 msgr2=0x5567d83277b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.112+0000 7fc80a800640 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d831bb00 0x5567d83277b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.112+0000 7fc80a800640 1 -- 172.21.15.86:0/2021115836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d8316990 msgr2=0x5567d8320770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.112+0000 7fc80a800640 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d8316990 0x5567d8320770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.112+0000 7fc80a800640 1 -- 172.21.15.86:0/2021115836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567d8239920 con 0x5567d8315fa0 2024-04-17T07:22:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.112+0000 7fc80a800640 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d8320230 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc8040bcbc0 tx=0x7fc804005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a37f31d55aca55 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.112+0000 7fc7efe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.113+0000 7fc7efe00640 1 -- 172.21.15.86:0/2021115836 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804004b40 con 0x5567d8315fa0 2024-04-17T07:22:03.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.113+0000 7fc7efe00640 1 -- 172.21.15.86:0/2021115836 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc804004ce0 con 0x5567d8315fa0 2024-04-17T07:22:03.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.113+0000 7fc814018740 1 -- 172.21.15.86:0/2021115836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8040c2cc0 con 0x5567d8315fa0 2024-04-17T07:22:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.113+0000 7fc7efe00640 1 -- 172.21.15.86:0/2021115836 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8040bdd60 con 0x5567d8315fa0 2024-04-17T07:22:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.113+0000 7fc814018740 10 client.17445 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.114+0000 7fc814018740 1 -- 172.21.15.86:0/2021115836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567d82077f0 con 0x5567d8315fa0 2024-04-17T07:22:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.114+0000 7fc7efe00640 1 -- 172.21.15.86:0/2021115836 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8040c2cc0 con 0x5567d8315fa0 2024-04-17T07:22:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.114+0000 7fc7efe00640 1 -- 172.21.15.86:0/2021115836 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc80403c030 con 0x5567d8315fa0 2024-04-17T07:22:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.114+0000 7fc814018740 10 client.17445 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.114+0000 7fc814018740 10 client.17445 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:22:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.114+0000 7fc814018740 20 client.17445 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.114+0000 7fc814018740 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5567d832ba90 0x5567d832df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.114+0000 7fc814018740 4 client.17445 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:03.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.114+0000 7fc814018740 1 -- 172.21.15.86:0/2021115836 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567d8289a40 con 0x5567d832ba90 2024-04-17T07:22:03.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.115+0000 7fc809e00640 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5567d832ba90 0x5567d832df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.117+0000 7fc809e00640 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5567d832ba90 0x5567d832df10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4290fb406e7bab60 server_cookie=5929552b6c894321 in_seq=0 out_seq=0 2024-04-17T07:22:03.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.117+0000 7fc7efe00640 10 client.17445 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:22:03.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.119+0000 7fc7efe00640 1 -- 172.21.15.86:0/2021115836 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567d8289a40 con 0x5567d832ba90 2024-04-17T07:22:03.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.119+0000 7fc7efe00640 10 client.17445 handle_command_reply: tid=0 2024-04-17T07:22:03.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.119+0000 7fc814018740 1 client.17445 shutdown 2024-04-17T07:22:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 1 -- 172.21.15.86:0/2021115836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 msgr2=0x5567d8320230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d8320230 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc8040bcbc0 tx=0x7fc804005df0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 1 -- 172.21.15.86:0/2021115836 shutdown_connections 2024-04-17T07:22:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d831bb00 0x5567d83277b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5567d832ba90 0x5567d832df10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d8316990 0x5567d8320770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 1 --2- 172.21.15.86:0/2021115836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8315fa0 0x5567d8320230 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 1 -- 172.21.15.86:0/2021115836 >> 172.21.15.86:0/2021115836 conn(0x5567d8311510 msgr2=0x5567d8310710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc7efe00640 0 client.17445 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:22:03.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 1 -- 172.21.15.86:0/2021115836 shutdown_connections 2024-04-17T07:22:03.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 1 -- 172.21.15.86:0/2021115836 wait complete. 2024-04-17T07:22:03.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.120+0000 7fc814018740 20 client.17445 trim_cache size 0 max 16384 2024-04-17T07:22:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.142+0000 7fc814018740 1 Processor -- start 2024-04-17T07:22:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.142+0000 7fc814018740 1 -- start start 2024-04-17T07:22:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.142+0000 7fc814018740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8316990 0x5567d831fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.143+0000 7fc814018740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 0x5567d8327740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.143+0000 7fc814018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d83185a0 0x5567d8320380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.143+0000 7fc814018740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b11e0 con 0x5567d8316990 2024-04-17T07:22:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.143+0000 7fc814018740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b2d20 con 0x5567d831bb00 2024-04-17T07:22:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.143+0000 7fc814018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b18b0 con 0x5567d83185a0 2024-04-17T07:22:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.143+0000 7fc80a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8316990 0x5567d831fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.143+0000 7fc80a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8316990 0x5567d831fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52038/0 (socket says 172.21.15.86:52038) 2024-04-17T07:22:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.143+0000 7fc80a800640 1 -- 172.21.15.86:0/3945797534 learned_addr learned my addr 172.21.15.86:0/3945797534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.144+0000 7fc80b200640 1 --2- 172.21.15.86:0/3945797534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d83185a0 0x5567d8320380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.144+0000 7fc809e00640 1 --2- 172.21.15.86:0/3945797534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 0x5567d8327740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.144+0000 7fc80a800640 1 -- 172.21.15.86:0/3945797534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d83185a0 msgr2=0x5567d8320380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.144+0000 7fc80a800640 1 --2- 172.21.15.86:0/3945797534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d83185a0 0x5567d8320380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.144+0000 7fc80a800640 1 -- 172.21.15.86:0/3945797534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 msgr2=0x5567d8327740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.144+0000 7fc80a800640 1 --2- 172.21.15.86:0/3945797534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 0x5567d8327740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.144+0000 7fc80a800640 1 -- 172.21.15.86:0/3945797534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567d7f8e3f0 con 0x5567d8316990 2024-04-17T07:22:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.144+0000 7fc809e00640 1 --2- 172.21.15.86:0/3945797534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 0x5567d8327740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.145+0000 7fc80a800640 1 --2- 172.21.15.86:0/3945797534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8316990 0x5567d831fe40 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc8040bad70 tx=0x7fc8040bce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbeeb86d7b8e762 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.145+0000 7fc7faa00640 1 -- 172.21.15.86:0/3945797534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8040c2cc0 con 0x5567d8316990 2024-04-17T07:22:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.145+0000 7fc7faa00640 1 -- 172.21.15.86:0/3945797534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8040bdf00 con 0x5567d8316990 2024-04-17T07:22:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.145+0000 7fc7faa00640 1 -- 172.21.15.86:0/3945797534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8040bd7d0 con 0x5567d8316990 2024-04-17T07:22:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.146+0000 7fc814018740 1 -- 172.21.15.86:0/3945797534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8316990 msgr2=0x5567d831fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.146+0000 7fc814018740 1 --2- 172.21.15.86:0/3945797534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8316990 0x5567d831fe40 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc8040bad70 tx=0x7fc8040bce20 comp rx=0 tx=0).stop 2024-04-17T07:22:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.147+0000 7fc814018740 1 -- 172.21.15.86:0/3945797534 shutdown_connections 2024-04-17T07:22:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.147+0000 7fc814018740 1 --2- 172.21.15.86:0/3945797534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d83185a0 0x5567d8320380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.147+0000 7fc814018740 1 --2- 172.21.15.86:0/3945797534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d831bb00 0x5567d8327740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.147+0000 7fc814018740 1 --2- 172.21.15.86:0/3945797534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d8316990 0x5567d831fe40 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.147+0000 7fc814018740 1 -- 172.21.15.86:0/3945797534 >> 172.21.15.86:0/3945797534 conn(0x5567d8311510 msgr2=0x5567d8317e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.148+0000 7fc814018740 1 -- 172.21.15.86:0/3945797534 shutdown_connections 2024-04-17T07:22:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.148+0000 7fc814018740 1 -- 172.21.15.86:0/3945797534 wait complete. 2024-04-17T07:22:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.148+0000 7fc814018740 1 -- start start 2024-04-17T07:22:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.148+0000 7fc814018740 1 Processor -- start 2024-04-17T07:22:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.149+0000 7fc814018740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d8316990 0x5567d8324460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.149+0000 7fc814018740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d83185a0 0x5567d83249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.149+0000 7fc814018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d831bb00 0x5567d8390080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.149+0000 7fc814018740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b2650 con 0x5567d83185a0 2024-04-17T07:22:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.149+0000 7fc814018740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b1f80 con 0x5567d8316990 2024-04-17T07:22:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.149+0000 7fc814018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1af1b0 con 0x5567d831bb00 2024-04-17T07:22:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.149+0000 7fc814018740 10 client.0 _ll_register_callbacks cb 0x7fc8040c7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.150+0000 7fc809e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d83185a0 0x5567d83249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.150+0000 7fc809e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d83185a0 0x5567d83249a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52046/0 (socket says 172.21.15.86:52046) 2024-04-17T07:22:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.150+0000 7fc809e00640 1 -- 172.21.15.86:0/4249071589 learned_addr learned my addr 172.21.15.86:0/4249071589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.150+0000 7fc80b200640 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d831bb00 0x5567d8390080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.150+0000 7fc809e00640 1 -- 172.21.15.86:0/4249071589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d831bb00 msgr2=0x5567d8390080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.150+0000 7fc809e00640 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d831bb00 0x5567d8390080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.150+0000 7fc809e00640 1 -- 172.21.15.86:0/4249071589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d8316990 msgr2=0x5567d8324460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.150+0000 7fc809e00640 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d8316990 0x5567d8324460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.150+0000 7fc809e00640 1 -- 172.21.15.86:0/4249071589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567d80587d0 con 0x5567d83185a0 2024-04-17T07:22:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.151+0000 7fc809e00640 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d83185a0 0x5567d83249a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00fc30 tx=0x7fc7fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe922dcc63cf655 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.151+0000 7fc7efe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.151+0000 7fc7efe00640 1 -- 172.21.15.86:0/4249071589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0293a0 con 0x5567d83185a0 2024-04-17T07:22:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.151+0000 7fc7efe00640 1 -- 172.21.15.86:0/4249071589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc7fc029540 con 0x5567d83185a0 2024-04-17T07:22:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.151+0000 7fc814018740 1 -- 172.21.15.86:0/4249071589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8040bdf00 con 0x5567d83185a0 2024-04-17T07:22:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.152+0000 7fc7efe00640 1 -- 172.21.15.86:0/4249071589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc037050 con 0x5567d83185a0 2024-04-17T07:22:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.152+0000 7fc814018740 10 client.17451 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.152+0000 7fc814018740 1 -- 172.21.15.86:0/4249071589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567d83a2560 con 0x5567d83185a0 2024-04-17T07:22:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.152+0000 7fc7efe00640 1 -- 172.21.15.86:0/4249071589 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8040bdf00 con 0x5567d83185a0 2024-04-17T07:22:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.152+0000 7fc7efe00640 1 -- 172.21.15.86:0/4249071589 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc7fc020c60 con 0x5567d83185a0 2024-04-17T07:22:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.152+0000 7fc814018740 10 client.17451 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.152+0000 7fc814018740 10 client.17451 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:22:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.152+0000 7fc814018740 20 client.17451 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:03.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.153+0000 7fc814018740 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5567d83a2f40 0x5567d83a5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.153+0000 7fc814018740 4 client.17451 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:22:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.153+0000 7fc814018740 1 -- 172.21.15.86:0/4249071589 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567d82838b0 con 0x5567d83a2f40 2024-04-17T07:22:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.153+0000 7fc80a800640 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5567d83a2f40 0x5567d83a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.154+0000 7fc80a800640 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5567d83a2f40 0x5567d83a5360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d39b057f7ec6e41 server_cookie=902936edc3080a37 in_seq=0 out_seq=0 2024-04-17T07:22:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.154+0000 7fc7efe00640 10 client.17451 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:22:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc7efe00640 1 -- 172.21.15.86:0/4249071589 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567d82838b0 con 0x5567d83a2f40 2024-04-17T07:22:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc7efe00640 10 client.17451 handle_command_reply: tid=0 2024-04-17T07:22:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 client.17451 shutdown 2024-04-17T07:22:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 -- 172.21.15.86:0/4249071589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d83185a0 msgr2=0x5567d83249a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d83185a0 0x5567d83249a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00fc30 tx=0x7fc7fc004730 comp rx=0 tx=0).stop 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 -- 172.21.15.86:0/4249071589 shutdown_connections 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5567d831bb00 0x5567d8390080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567d83185a0 0x5567d83249a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5567d83a2f40 0x5567d83a5360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 --2- 172.21.15.86:0/4249071589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567d8316990 0x5567d8324460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 -- 172.21.15.86:0/4249071589 >> 172.21.15.86:0/4249071589 conn(0x5567d8311510 msgr2=0x5567d831fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc7efe00640 0 client.17451 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 -- 172.21.15.86:0/4249071589 shutdown_connections 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 1 -- 172.21.15.86:0/4249071589 wait complete. 2024-04-17T07:22:03.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.155+0000 7fc814018740 20 client.17451 trim_cache size 0 max 16384 2024-04-17T07:22:03.156 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:22:03.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.156+0000 7fc811400640 1 -- 172.21.15.86:0/410691276 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc7f00a3db0 msgr2=0x7fc7f00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.156+0000 7fc811400640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc7f00a3db0 0x7fc7f00a6270 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc005fd0 tx=0x7fc7fc005850 comp rx=0 tx=0).stop 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.156+0000 7fc811400640 1 -- 172.21.15.86:0/410691276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc80c1085c0 msgr2=0x7fc80c19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.156+0000 7fc811400640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc80c1085c0 0x7fc80c19e8e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc804037040 tx=0x7fc804034cd0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.157+0000 7fc811400640 1 -- 172.21.15.86:0/410691276 shutdown_connections 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.157+0000 7fc811400640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc7f00a3db0 0x7fc7f00a6270 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.157+0000 7fc811400640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80c10a170 0x7fc80c1a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.157+0000 7fc811400640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc80c1097b0 0x7fc80c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.157+0000 7fc811400640 1 --2- 172.21.15.86:0/410691276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc80c1085c0 0x7fc80c19e8e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.157+0000 7fc811400640 1 -- 172.21.15.86:0/410691276 >> 172.21.15.86:0/410691276 conn(0x7fc80c077560 msgr2=0x7fc80c10cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.157+0000 7fc811400640 1 -- 172.21.15.86:0/410691276 shutdown_connections 2024-04-17T07:22:03.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.157+0000 7fc811400640 1 -- 172.21.15.86:0/410691276 wait complete. 2024-04-17T07:22:03.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:22:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.443+0000 7fb5fcc00640 1 -- 172.21.15.86:0/3650760616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f8103580 msgr2=0x7fb5f81039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.443+0000 7fb5fcc00640 1 --2- 172.21.15.86:0/3650760616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f8103580 0x7fb5f81039a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009900 tx=0x7fb5e802bc30 comp rx=0 tx=0).stop 2024-04-17T07:22:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.444+0000 7fb5fcc00640 1 -- 172.21.15.86:0/3650760616 shutdown_connections 2024-04-17T07:22:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.444+0000 7fb5fcc00640 1 --2- 172.21.15.86:0/3650760616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f81009b0 0x7fb5f810a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.444+0000 7fb5fcc00640 1 --2- 172.21.15.86:0/3650760616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f80ffff0 0x7fb5f8100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.444+0000 7fb5fcc00640 1 --2- 172.21.15.86:0/3650760616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f8103580 0x7fb5f81039a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.444+0000 7fb5fcc00640 1 -- 172.21.15.86:0/3650760616 >> 172.21.15.86:0/3650760616 conn(0x7fb5f80fbb90 msgr2=0x7fb5f80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.444+0000 7fb5fcc00640 1 -- 172.21.15.86:0/3650760616 shutdown_connections 2024-04-17T07:22:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.444+0000 7fb5fcc00640 1 -- 172.21.15.86:0/3650760616 wait complete. 2024-04-17T07:22:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.445+0000 7fb5fcc00640 1 Processor -- start 2024-04-17T07:22:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.446+0000 7fb5fcc00640 1 -- start start 2024-04-17T07:22:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.446+0000 7fb5fcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f80ffff0 0x7fb5f8196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.446+0000 7fb5fcc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f81009b0 0x7fb5f8196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.446+0000 7fb5fcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f8103580 0x7fb5f819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.446+0000 7fb5fcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810b950 con 0x7fb5f80ffff0 2024-04-17T07:22:03.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.446+0000 7fb5fcc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810b7d0 con 0x7fb5f81009b0 2024-04-17T07:22:03.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.446+0000 7fb5fcc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810bad0 con 0x7fb5f8103580 2024-04-17T07:22:03.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.447+0000 7fb5f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f80ffff0 0x7fb5f8196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.447+0000 7fb5f5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f81009b0 0x7fb5f8196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.447+0000 7fb5f5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f81009b0 0x7fb5f8196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53196/0 (socket says 172.21.15.86:53196) 2024-04-17T07:22:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.447+0000 7fb5f5400640 1 -- 172.21.15.86:0/2524448946 learned_addr learned my addr 172.21.15.86:0/2524448946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.447+0000 7fb5f6800640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f8103580 0x7fb5f819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.448+0000 7fb5f5e00640 1 -- 172.21.15.86:0/2524448946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f8103580 msgr2=0x7fb5f819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.448+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f8103580 0x7fb5f819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.448+0000 7fb5f5e00640 1 -- 172.21.15.86:0/2524448946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f81009b0 msgr2=0x7fb5f8196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.448+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f81009b0 0x7fb5f8196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.448+0000 7fb5f5e00640 1 -- 172.21.15.86:0/2524448946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f819dc60 con 0x7fb5f80ffff0 2024-04-17T07:22:03.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.448+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f80ffff0 0x7fb5f8196040 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8036040 tx=0x7fb5e80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.449+0000 7fb5eea00640 1 -- 172.21.15.86:0/2524448946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8043020 con 0x7fb5f80ffff0 2024-04-17T07:22:03.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.449+0000 7fb5eea00640 1 -- 172.21.15.86:0/2524448946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5e80044c0 con 0x7fb5f80ffff0 2024-04-17T07:22:03.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.449+0000 7fb5fcc00640 1 -- 172.21.15.86:0/2524448946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f819def0 con 0x7fb5f80ffff0 2024-04-17T07:22:03.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.449+0000 7fb5eea00640 1 -- 172.21.15.86:0/2524448946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8003bf0 con 0x7fb5f80ffff0 2024-04-17T07:22:03.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.449+0000 7fb5fcc00640 1 -- 172.21.15.86:0/2524448946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f819e360 con 0x7fb5f80ffff0 2024-04-17T07:22:03.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.455+0000 7fb5ffa2e740 1 Processor -- start 2024-04-17T07:22:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.455+0000 7fb5ffa2e740 1 -- start start 2024-04-17T07:22:03.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.456+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a730c7b0 0x5638a730cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.456+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 0x5638a7311ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.456+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a7312310 0x5638a73147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.456+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5638a726a020 con 0x5638a7312310 2024-04-17T07:22:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.456+0000 7fb5f6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a730c7b0 0x5638a730cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.457+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5638a7210460 con 0x5638a730d1a0 2024-04-17T07:22:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.457+0000 7fb5f6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a730c7b0 0x5638a730cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45162/0 (socket says 172.21.15.86:45162) 2024-04-17T07:22:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.457+0000 7fb5f6800640 1 -- 172.21.15.86:0/3657352514 learned_addr learned my addr 172.21.15.86:0/3657352514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.457+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3657352514 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5638a7265fe0 con 0x5638a730c7b0 2024-04-17T07:22:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.457+0000 7fb5f5400640 1 --2- 172.21.15.86:0/3657352514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a7312310 0x5638a73147a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.458+0000 7fb5f5400640 1 -- 172.21.15.86:0/3657352514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a730c7b0 msgr2=0x5638a730cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.458+0000 7fb5f5400640 1 --2- 172.21.15.86:0/3657352514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a730c7b0 0x5638a730cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.458+0000 7fb5f5400640 1 -- 172.21.15.86:0/3657352514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 msgr2=0x5638a7311ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:22:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.458+0000 7fb5f5400640 1 --2- 172.21.15.86:0/3657352514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 0x5638a7311ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.458+0000 7fb5f5400640 1 -- 172.21.15.86:0/3657352514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a7218040 con 0x5638a7312310 2024-04-17T07:22:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.458+0000 7fb5f5400640 1 --2- 172.21.15.86:0/3657352514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a7312310 0x5638a73147a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009cc0 tx=0x7fb5e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adee8c7558dbd62 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.459+0000 7fb5e3000640 1 -- 172.21.15.86:0/3657352514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4005220 con 0x5638a7312310 2024-04-17T07:22:03.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.459+0000 7fb5e3000640 1 -- 172.21.15.86:0/3657352514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5e4004c50 con 0x5638a7312310 2024-04-17T07:22:03.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.459+0000 7fb5e3000640 1 -- 172.21.15.86:0/3657352514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40197a0 con 0x5638a7312310 2024-04-17T07:22:03.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.460+0000 7fb5eea00640 1 -- 172.21.15.86:0/2524448946 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb5e8046020 con 0x7fb5f80ffff0 2024-04-17T07:22:03.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.460+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3657352514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a7312310 msgr2=0x5638a73147a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.460+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3657352514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a7312310 0x5638a73147a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009cc0 tx=0x7fb5e4011810 comp rx=0 tx=0).stop 2024-04-17T07:22:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.461+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3657352514 shutdown_connections 2024-04-17T07:22:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.461+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3657352514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a7312310 0x5638a73147a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.461+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3657352514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 0x5638a7311ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.461+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3657352514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a730c7b0 0x5638a730cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.461+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3657352514 >> 172.21.15.86:0/3657352514 conn(0x5638a7307d20 msgr2=0x5638a7308120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.461+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3657352514 shutdown_connections 2024-04-17T07:22:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.461+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3657352514 wait complete. 2024-04-17T07:22:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.461+0000 7fb5eea00640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5dc0a3bd0 0x7fb5dc0a6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.462+0000 7fb5eea00640 1 -- 172.21.15.86:0/2524448946 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb5e80f2510 con 0x7fb5f80ffff0 2024-04-17T07:22:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.462+0000 7fb5f5400640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5dc0a3bd0 0x7fb5dc0a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.462+0000 7fb5ffa2e740 1 -- start start 2024-04-17T07:22:03.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.463+0000 7fb5ffa2e740 1 Processor -- start 2024-04-17T07:22:03.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.463+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a7316d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.463+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 0x5638a7317260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.463+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a731e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.463+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5638a73152c0 con 0x5638a730c7b0 2024-04-17T07:22:03.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.463+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5638a7223830 con 0x5638a730d1a0 2024-04-17T07:22:03.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.463+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5638a7315440 con 0x5638a7312310 2024-04-17T07:22:03.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.463+0000 7fb5ffa2e740 10 client.0 _ll_register_callbacks cb 0x5638a71f1fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.464+0000 7fb5f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a7316d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.464+0000 7fb5f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a7316d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52082/0 (socket says 172.21.15.86:52082) 2024-04-17T07:22:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.464+0000 7fb5f5e00640 1 -- 172.21.15.86:0/3601205949 learned_addr learned my addr 172.21.15.86:0/3601205949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.464+0000 7fb5f6800640 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a731e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.464+0000 7fb5f5400640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5dc0a3bd0 0x7fb5dc0a6090 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400a010 tx=0x7fb5e4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.464+0000 7fb5f5e00640 1 -- 172.21.15.86:0/3601205949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 msgr2=0x5638a731e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.464+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a731e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.465+0000 7fb5f5e00640 1 -- 172.21.15.86:0/3601205949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 msgr2=0x5638a7317260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:22:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.465+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 0x5638a7317260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.465+0000 7fb5f5e00640 1 -- 172.21.15.86:0/3601205949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a7227920 con 0x5638a730c7b0 2024-04-17T07:22:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.465+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a7316d20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80bade0 tx=0x7fb5e80bdf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d943b67073f92d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.465+0000 7fb5d7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.465+0000 7fb5d7400640 1 -- 172.21.15.86:0/3601205949 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80d7060 con 0x5638a730c7b0 2024-04-17T07:22:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.465+0000 7fb5d7400640 1 -- 172.21.15.86:0/3601205949 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb5e8005570 con 0x5638a730c7b0 2024-04-17T07:22:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.466+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3601205949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e4004c50 con 0x5638a730c7b0 2024-04-17T07:22:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.467+0000 7fb5d7400640 1 -- 172.21.15.86:0/3601205949 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb5e80dc050 con 0x5638a730c7b0 2024-04-17T07:22:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.467+0000 7fb5ffa2e740 10 client.17463 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.467+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3601205949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638a71f57f0 con 0x5638a730c7b0 2024-04-17T07:22:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.468+0000 7fb5d7400640 1 -- 172.21.15.86:0/3601205949 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5e4004c50 con 0x5638a730c7b0 2024-04-17T07:22:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.468+0000 7fb5d7400640 1 -- 172.21.15.86:0/3601205949 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb5e80c6bf0 con 0x5638a730c7b0 2024-04-17T07:22:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.468+0000 7fb5ffa2e740 10 client.17463 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.469+0000 7fb5ffa2e740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.469+0000 7fb5ffa2e740 20 client.17463 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.469+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5638a73226f0 0x5638a7324b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.469+0000 7fb5ffa2e740 4 client.17463 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.469+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3601205949 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638a7277a40 con 0x5638a73226f0 2024-04-17T07:22:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.470+0000 7fb5f5400640 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5638a73226f0 0x5638a7324b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.472+0000 7fb5f5400640 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5638a73226f0 0x5638a7324b70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69f6f117fef6bc9 server_cookie=e6755b1b2b8ee555 in_seq=0 out_seq=0 2024-04-17T07:22:03.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.472+0000 7fb5d7400640 10 client.17463 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.474+0000 7fb5d7400640 1 -- 172.21.15.86:0/3601205949 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638a7277a40 con 0x5638a73226f0 2024-04-17T07:22:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.474+0000 7fb5d7400640 10 client.17463 handle_command_reply: tid=0 2024-04-17T07:22:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.474+0000 7fb5ffa2e740 1 client.17463 shutdown 2024-04-17T07:22:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3601205949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 msgr2=0x5638a7316d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a7316d20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80bade0 tx=0x7fb5e80bdf50 comp rx=0 tx=0).stop 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3601205949 shutdown_connections 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5638a73226f0 0x5638a7324b70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5d7400640 0 client.17463 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a731e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 0x5638a7317260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3601205949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a7316d20 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3601205949 >> 172.21.15.86:0/3601205949 conn(0x5638a7307d20 msgr2=0x5638a72fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3601205949 shutdown_connections 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.475+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3601205949 wait complete. 2024-04-17T07:22:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.476+0000 7fb5ffa2e740 20 client.17463 trim_cache size 0 max 16384 2024-04-17T07:22:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.500+0000 7fb5ffa2e740 1 Processor -- start 2024-04-17T07:22:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.500+0000 7fb5ffa2e740 1 -- start start 2024-04-17T07:22:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.500+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a73171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.500+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 0x5638a731e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.500+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a730b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.500+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f81a8810 con 0x5638a730c7b0 2024-04-17T07:22:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.500+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f81aa350 con 0x5638a730d1a0 2024-04-17T07:22:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.501+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f81a8ee0 con 0x5638a7312310 2024-04-17T07:22:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.501+0000 7fb5f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a73171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.501+0000 7fb5f6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a730b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.502+0000 7fb5f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a73171c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52098/0 (socket says 172.21.15.86:52098) 2024-04-17T07:22:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.502+0000 7fb5f5e00640 1 -- 172.21.15.86:0/3927236867 learned_addr learned my addr 172.21.15.86:0/3927236867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.502+0000 7fb5f5e00640 1 -- 172.21.15.86:0/3927236867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 msgr2=0x5638a730b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.502+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/3927236867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a730b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.502+0000 7fb5f5e00640 1 -- 172.21.15.86:0/3927236867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 msgr2=0x5638a731e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.502+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/3927236867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 0x5638a731e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.502+0000 7fb5f5e00640 1 -- 172.21.15.86:0/3927236867 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638a6f7c3f0 con 0x5638a730c7b0 2024-04-17T07:22:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.503+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/3927236867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a73171c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb5e802c170 tx=0x7fb5e80bc920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cfd711ff261e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.503+0000 7fb5e2600640 1 -- 172.21.15.86:0/3927236867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80be410 con 0x5638a730c7b0 2024-04-17T07:22:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.503+0000 7fb5e2600640 1 -- 172.21.15.86:0/3927236867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5e80c27c0 con 0x5638a730c7b0 2024-04-17T07:22:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.503+0000 7fb5e2600640 1 -- 172.21.15.86:0/3927236867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80d7b90 con 0x5638a730c7b0 2024-04-17T07:22:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.504+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3927236867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 msgr2=0x5638a73171c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.504+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3927236867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a73171c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb5e802c170 tx=0x7fb5e80bc920 comp rx=0 tx=0).stop 2024-04-17T07:22:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.504+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3927236867 shutdown_connections 2024-04-17T07:22:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.505+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3927236867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a730b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.505+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3927236867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730d1a0 0x5638a731e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.505+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/3927236867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730c7b0 0x5638a73171c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.505+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3927236867 >> 172.21.15.86:0/3927236867 conn(0x5638a7307d20 msgr2=0x5638a730af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.506+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3927236867 shutdown_connections 2024-04-17T07:22:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.506+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/3927236867 wait complete. 2024-04-17T07:22:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.507+0000 7fb5ffa2e740 1 -- start start 2024-04-17T07:22:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.507+0000 7fb5ffa2e740 1 Processor -- start 2024-04-17T07:22:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730c7b0 0x5638a7319aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730d1a0 0x5638a7319fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5ffa2e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a731d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f81a9c80 con 0x5638a730d1a0 2024-04-17T07:22:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f81a95b0 con 0x5638a730c7b0 2024-04-17T07:22:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5ffa2e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f81a67e0 con 0x5638a7312310 2024-04-17T07:22:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5ffa2e740 10 client.0 _ll_register_callbacks cb 0x7fb5e80ba6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5f5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730d1a0 0x5638a7319fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5f5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730c7b0 0x5638a7319aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5f5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730c7b0 0x5638a7319aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53218/0 (socket says 172.21.15.86:53218) 2024-04-17T07:22:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.508+0000 7fb5f5e00640 1 -- 172.21.15.86:0/2584124131 learned_addr learned my addr 172.21.15.86:0/2584124131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.509+0000 7fb5f6800640 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a731d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.509+0000 7fb5f5400640 1 -- 172.21.15.86:0/2584124131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 msgr2=0x5638a731d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.509+0000 7fb5f5400640 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a731d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.509+0000 7fb5f5400640 1 -- 172.21.15.86:0/2584124131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730c7b0 msgr2=0x5638a7319aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.509+0000 7fb5f5400640 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730c7b0 0x5638a7319aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.509+0000 7fb5f5400640 1 -- 172.21.15.86:0/2584124131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a70467d0 con 0x5638a730d1a0 2024-04-17T07:22:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.511+0000 7fb5f5400640 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730d1a0 0x5638a7319fe0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40057c0 tx=0x7fb5e4005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220f0a340a923544 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.511+0000 7fb5d7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.512+0000 7fb5d7400640 1 -- 172.21.15.86:0/2584124131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4009e70 con 0x5638a730d1a0 2024-04-17T07:22:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.512+0000 7fb5d7400640 1 -- 172.21.15.86:0/2584124131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb5e4004aa0 con 0x5638a730d1a0 2024-04-17T07:22:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.512+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/2584124131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e80c27c0 con 0x5638a730d1a0 2024-04-17T07:22:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.513+0000 7fb5d7400640 1 -- 172.21.15.86:0/2584124131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb5e4045050 con 0x5638a730d1a0 2024-04-17T07:22:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.513+0000 7fb5ffa2e740 10 client.17469 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.513+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/2584124131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638a7399d00 con 0x5638a730d1a0 2024-04-17T07:22:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.514+0000 7fb5d7400640 1 -- 172.21.15.86:0/2584124131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5e80c27c0 con 0x5638a730d1a0 2024-04-17T07:22:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.515+0000 7fb5d7400640 1 -- 172.21.15.86:0/2584124131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb5e402f040 con 0x5638a730d1a0 2024-04-17T07:22:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.515+0000 7fb5ffa2e740 10 client.17469 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.515+0000 7fb5ffa2e740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.515+0000 7fb5ffa2e740 20 client.17469 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.515+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5638a739a400 0x5638a739c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.515+0000 7fb5ffa2e740 4 client.17469 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:22:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.515+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/2584124131 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638a72718b0 con 0x5638a739a400 2024-04-17T07:22:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.516+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5638a739a400 0x5638a739c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.518+0000 7fb5d7400640 10 client.17469 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.518+0000 7fb5f5e00640 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5638a739a400 0x5638a739c820 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce30c5388f084fd2 server_cookie=b5124bdd341f0d74 in_seq=0 out_seq=0 2024-04-17T07:22:03.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5d7400640 1 -- 172.21.15.86:0/2584124131 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638a72718b0 con 0x5638a739a400 2024-04-17T07:22:03.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5d7400640 10 client.17469 handle_command_reply: tid=0 2024-04-17T07:22:03.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5ffa2e740 1 client.17469 shutdown 2024-04-17T07:22:03.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/2584124131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730d1a0 msgr2=0x5638a7319fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730d1a0 0x5638a7319fe0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40057c0 tx=0x7fb5e4005370 comp rx=0 tx=0).stop 2024-04-17T07:22:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/2584124131 shutdown_connections 2024-04-17T07:22:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5638a739a400 0x5638a739c820 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5638a7312310 0x5638a731d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5d7400640 0 client.17469 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638a730d1a0 0x5638a7319fe0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5ffa2e740 1 --2- 172.21.15.86:0/2584124131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5638a730c7b0 0x5638a7319aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.519+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/2584124131 >> 172.21.15.86:0/2584124131 conn(0x5638a7307d20 msgr2=0x5638a7317040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.520+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/2584124131 shutdown_connections 2024-04-17T07:22:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.520+0000 7fb5ffa2e740 1 -- 172.21.15.86:0/2584124131 wait complete. 2024-04-17T07:22:03.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.520+0000 7fb5ffa2e740 20 client.17469 trim_cache size 0 max 16384 2024-04-17T07:22:03.520 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:22:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.521+0000 7fb5fcc00640 1 -- 172.21.15.86:0/2524448946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5dc0a3bd0 msgr2=0x7fb5dc0a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.521+0000 7fb5fcc00640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5dc0a3bd0 0x7fb5dc0a6090 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400a010 tx=0x7fb5e4009310 comp rx=0 tx=0).stop 2024-04-17T07:22:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.521+0000 7fb5fcc00640 1 -- 172.21.15.86:0/2524448946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f80ffff0 msgr2=0x7fb5f8196040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.521+0000 7fb5fcc00640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f80ffff0 0x7fb5f8196040 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8036040 tx=0x7fb5e80040c0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.522+0000 7fb5fcc00640 1 -- 172.21.15.86:0/2524448946 shutdown_connections 2024-04-17T07:22:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.522+0000 7fb5fcc00640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb5dc0a3bd0 0x7fb5dc0a6090 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.522+0000 7fb5fcc00640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5f8103580 0x7fb5f819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.522+0000 7fb5fcc00640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5f81009b0 0x7fb5f8196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.522+0000 7fb5fcc00640 1 --2- 172.21.15.86:0/2524448946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5f80ffff0 0x7fb5f8196040 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:03.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.522+0000 7fb5fcc00640 1 -- 172.21.15.86:0/2524448946 >> 172.21.15.86:0/2524448946 conn(0x7fb5f80fbb90 msgr2=0x7fb5f81066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:03.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.522+0000 7fb5fcc00640 1 -- 172.21.15.86:0/2524448946 shutdown_connections 2024-04-17T07:22:03.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:03.522+0000 7fb5fcc00640 1 -- 172.21.15.86:0/2524448946 wait complete. 2024-04-17T07:22:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 160.0 s, 615.4 tps, lat 162.800 ms stddev 226.490 2024-04-17T07:22:04.546 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:22:04.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:22:04.547 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:22:04.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:04 smithi103 ceph-mon[35495]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 14 MiB/s wr, 789 op/s 2024-04-17T07:22:04.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:04 smithi086 ceph-mon[31733]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 14 MiB/s wr, 789 op/s 2024-04-17T07:22:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.877+0000 7fa61ec00640 1 -- 172.21.15.86:0/2431609961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6181053b0 msgr2=0x7fa618105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:04.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.877+0000 7fa61ec00640 1 --2- 172.21.15.86:0/2431609961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6181053b0 0x7fa618105830 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa600009900 tx=0x7fa60002bc40 comp rx=0 tx=0).stop 2024-04-17T07:22:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.878+0000 7fa61ec00640 1 -- 172.21.15.86:0/2431609961 shutdown_connections 2024-04-17T07:22:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.878+0000 7fa61ec00640 1 --2- 172.21.15.86:0/2431609961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa618105d70 0x7fa61810c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.878+0000 7fa61ec00640 1 --2- 172.21.15.86:0/2431609961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6181053b0 0x7fa618105830 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.878+0000 7fa61ec00640 1 --2- 172.21.15.86:0/2431609961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6181041c0 0x7fa6181045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.878+0000 7fa61ec00640 1 -- 172.21.15.86:0/2431609961 >> 172.21.15.86:0/2431609961 conn(0x7fa6180ff710 msgr2=0x7fa618101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:04.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.878+0000 7fa61ec00640 1 -- 172.21.15.86:0/2431609961 shutdown_connections 2024-04-17T07:22:04.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.878+0000 7fa61ec00640 1 -- 172.21.15.86:0/2431609961 wait complete. 2024-04-17T07:22:04.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.880+0000 7fa61ec00640 1 Processor -- start 2024-04-17T07:22:04.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.880+0000 7fa61ec00640 1 -- start start 2024-04-17T07:22:04.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.881+0000 7fa61ec00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6181041c0 0x7fa61819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.881+0000 7fa61ec00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa618105d70 0x7fa61819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.881+0000 7fa61ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61819fb20 0x7fa6181a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.881+0000 7fa61ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa61810fd50 con 0x7fa61819fb20 2024-04-17T07:22:04.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.881+0000 7fa61ec00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa61810fbd0 con 0x7fa618105d70 2024-04-17T07:22:04.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.881+0000 7fa61ec00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa61810fed0 con 0x7fa6181041c0 2024-04-17T07:22:04.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.881+0000 7fa617e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6181041c0 0x7fa61819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.881+0000 7fa617e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6181041c0 0x7fa61819a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45200/0 (socket says 172.21.15.86:45200) 2024-04-17T07:22:04.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.881+0000 7fa617e00640 1 -- 172.21.15.86:0/3795121612 learned_addr learned my addr 172.21.15.86:0/3795121612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:04.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.882+0000 7fa617400640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa618105d70 0x7fa61819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.882+0000 7fa61cc00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61819fb20 0x7fa6181a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.882+0000 7fa617e00640 1 -- 172.21.15.86:0/3795121612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa618105d70 msgr2=0x7fa61819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:04.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.883+0000 7fa617e00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa618105d70 0x7fa61819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.883+0000 7fa617e00640 1 -- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61819fb20 msgr2=0x7fa6181a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:04.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.883+0000 7fa617e00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61819fb20 0x7fa6181a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.883+0000 7fa617e00640 1 -- 172.21.15.86:0/3795121612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61806bab0 con 0x7fa6181041c0 2024-04-17T07:22:04.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.883+0000 7fa61cc00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61819fb20 0x7fa6181a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:04.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.883+0000 7fa617400640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa618105d70 0x7fa61819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:04.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.883+0000 7fa617e00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6181041c0 0x7fa61819a580 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa61000d770 tx=0x7fa61000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:04.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.884+0000 7fa614c00640 1 -- 172.21.15.86:0/3795121612 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610017070 con 0x7fa6181041c0 2024-04-17T07:22:04.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.884+0000 7fa614c00640 1 -- 172.21.15.86:0/3795121612 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa610009c30 con 0x7fa6181041c0 2024-04-17T07:22:04.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.884+0000 7fa614c00640 1 -- 172.21.15.86:0/3795121612 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610003d00 con 0x7fa6181041c0 2024-04-17T07:22:04.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.884+0000 7fa61ec00640 1 -- 172.21.15.86:0/3795121612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61806bda0 con 0x7fa6181041c0 2024-04-17T07:22:04.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.884+0000 7fa61ec00640 1 -- 172.21.15.86:0/3795121612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61806c270 con 0x7fa6181041c0 2024-04-17T07:22:04.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.889+0000 7fa621827740 1 Processor -- start 2024-04-17T07:22:04.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.890+0000 7fa621827740 1 -- start start 2024-04-17T07:22:04.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.890+0000 7fa621827740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a156d0 0x55f6c0a15af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.890+0000 7fa614c00640 1 -- 172.21.15.86:0/3795121612 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa610003ea0 con 0x7fa6181041c0 2024-04-17T07:22:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.891+0000 7fa621827740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a1abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.891+0000 7fa621827740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a1b230 0x55f6c0a1d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.891+0000 7fa621827740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f6c0972cb0 con 0x55f6c0a160c0 2024-04-17T07:22:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.891+0000 7fa621827740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f6c0919370 con 0x55f6c0a1b230 2024-04-17T07:22:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.891+0000 7fa621827740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f6c096ec70 con 0x55f6c0a156d0 2024-04-17T07:22:04.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.892+0000 7fa61cc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a156d0 0x55f6c0a15af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.892+0000 7fa617400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a1b230 0x55f6c0a1d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.892+0000 7fa617400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a1b230 0x55f6c0a1d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53262/0 (socket says 172.21.15.86:53262) 2024-04-17T07:22:04.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.892+0000 7fa617400640 1 -- 172.21.15.86:0/3446149952 learned_addr learned my addr 172.21.15.86:0/3446149952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:04.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.892+0000 7fa617e00640 1 --2- 172.21.15.86:0/3446149952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a1abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.893+0000 7fa617e00640 1 -- 172.21.15.86:0/3446149952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a156d0 msgr2=0x55f6c0a15af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.893+0000 7fa617e00640 1 --2- 172.21.15.86:0/3446149952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a156d0 0x55f6c0a15af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.893+0000 7fa617e00640 1 -- 172.21.15.86:0/3446149952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a1b230 msgr2=0x55f6c0a1d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.893+0000 7fa617e00640 1 --2- 172.21.15.86:0/3446149952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a1b230 0x55f6c0a1d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.893+0000 7fa617e00640 1 -- 172.21.15.86:0/3446149952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c0938910 con 0x55f6c0a160c0 2024-04-17T07:22:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.893+0000 7fa61cc00640 1 --2- 172.21.15.86:0/3446149952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a156d0 0x55f6c0a15af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.893+0000 7fa617400640 1 --2- 172.21.15.86:0/3446149952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a1b230 0x55f6c0a1d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.894+0000 7fa614c00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5fc0a3c80 0x7fa5fc0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.894+0000 7fa617e00640 1 --2- 172.21.15.86:0/3446149952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a1abc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa6100cd710 tx=0x7fa6100d8580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dde03afbe1187c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:04.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.894+0000 7fa614c00640 1 -- 172.21.15.86:0/3795121612 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa6100cffc0 con 0x7fa6181041c0 2024-04-17T07:22:04.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.894+0000 7fa61cc00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5fc0a3c80 0x7fa5fc0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.895+0000 7fa60d200640 1 -- 172.21.15.86:0/3446149952 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610098270 con 0x55f6c0a160c0 2024-04-17T07:22:04.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.895+0000 7fa60d200640 1 -- 172.21.15.86:0/3446149952 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61009bfd0 con 0x55f6c0a160c0 2024-04-17T07:22:04.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.895+0000 7fa60d200640 1 -- 172.21.15.86:0/3446149952 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61009b930 con 0x55f6c0a160c0 2024-04-17T07:22:04.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.897+0000 7fa621827740 1 -- 172.21.15.86:0/3446149952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 msgr2=0x55f6c0a1abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:04.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.897+0000 7fa621827740 1 --2- 172.21.15.86:0/3446149952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a1abc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa6100cd710 tx=0x7fa6100d8580 comp rx=0 tx=0).stop 2024-04-17T07:22:04.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.897+0000 7fa61cc00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5fc0a3c80 0x7fa5fc0a6140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55f6c0a16920 tx=0x7fa608007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:04.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.897+0000 7fa621827740 1 -- 172.21.15.86:0/3446149952 shutdown_connections 2024-04-17T07:22:04.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.897+0000 7fa621827740 1 --2- 172.21.15.86:0/3446149952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a1b230 0x55f6c0a1d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.897+0000 7fa621827740 1 --2- 172.21.15.86:0/3446149952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a1abc0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.897+0000 7fa621827740 1 --2- 172.21.15.86:0/3446149952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a156d0 0x55f6c0a15af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.897+0000 7fa621827740 1 -- 172.21.15.86:0/3446149952 >> 172.21.15.86:0/3446149952 conn(0x55f6c0a08430 msgr2=0x55f6c0a08830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:04.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.898+0000 7fa621827740 1 -- 172.21.15.86:0/3446149952 shutdown_connections 2024-04-17T07:22:04.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.898+0000 7fa621827740 1 -- 172.21.15.86:0/3446149952 wait complete. 2024-04-17T07:22:04.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.899+0000 7fa621827740 1 -- start start 2024-04-17T07:22:04.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.900+0000 7fa621827740 1 Processor -- start 2024-04-17T07:22:04.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.900+0000 7fa621827740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a156d0 0x55f6c0a1fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.901+0000 7fa621827740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a160c0 0x55f6c0a20160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.901+0000 7fa617e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a156d0 0x55f6c0a1fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.901+0000 7fa617e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a156d0 0x55f6c0a1fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53278/0 (socket says 172.21.15.86:53278) 2024-04-17T07:22:04.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.901+0000 7fa617e00640 1 -- 172.21.15.86:0/1077392014 learned_addr learned my addr 172.21.15.86:0/1077392014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:04.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.901+0000 7fa621827740 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a1b230 0x55f6c0a271a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.901+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f6c0a1e1e0 con 0x55f6c0a1b230 2024-04-17T07:22:04.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.901+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f6c092c730 con 0x55f6c0a156d0 2024-04-17T07:22:04.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.901+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f6c0a1e360 con 0x55f6c0a160c0 2024-04-17T07:22:04.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.901+0000 7fa621827740 10 client.0 _ll_register_callbacks cb 0x55f6c092eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:04.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.902+0000 7fa617400640 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a160c0 0x55f6c0a20160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.902+0000 7fa61cc00640 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a1b230 0x55f6c0a271a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.902+0000 7fa617e00640 1 -- 172.21.15.86:0/1077392014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a160c0 msgr2=0x55f6c0a20160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:04.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.902+0000 7fa617e00640 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a160c0 0x55f6c0a20160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.902+0000 7fa617e00640 1 -- 172.21.15.86:0/1077392014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a1b230 msgr2=0x55f6c0a271a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:04.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.903+0000 7fa617e00640 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a1b230 0x55f6c0a271a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.903+0000 7fa617e00640 1 -- 172.21.15.86:0/1077392014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c074d7e0 con 0x55f6c0a156d0 2024-04-17T07:22:04.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.903+0000 7fa61cc00640 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a1b230 0x55f6c0a271a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:04.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.903+0000 7fa617e00640 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a156d0 0x55f6c0a1fc20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa6100cdab0 tx=0x7fa6100d35f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47520eb59b4070b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:04.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.903+0000 7fa605600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:22:04.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.904+0000 7fa605600640 1 -- 172.21.15.86:0/1077392014 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610020070 con 0x55f6c0a156d0 2024-04-17T07:22:04.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.905+0000 7fa605600640 1 -- 172.21.15.86:0/1077392014 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa61009a070 con 0x55f6c0a156d0 2024-04-17T07:22:04.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.905+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61009bfd0 con 0x55f6c0a156d0 2024-04-17T07:22:04.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.905+0000 7fa605600640 1 -- 172.21.15.86:0/1077392014 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa61009bb10 con 0x55f6c0a156d0 2024-04-17T07:22:04.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.905+0000 7fa621827740 10 client.36443 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:04.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.905+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6c07adf70 con 0x55f6c0a156d0 2024-04-17T07:22:04.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.907+0000 7fa605600640 1 -- 172.21.15.86:0/1077392014 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa61009bfd0 con 0x55f6c0a156d0 2024-04-17T07:22:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.908+0000 7fa605600640 1 -- 172.21.15.86:0/1077392014 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa61009f020 con 0x55f6c0a156d0 2024-04-17T07:22:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.908+0000 7fa621827740 10 client.36443 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.908+0000 7fa621827740 10 client.36443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.908+0000 7fa621827740 20 client.36443 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.908+0000 7fa621827740 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f6c0a2b670 0x55f6c0a2da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.908+0000 7fa621827740 4 client.36443 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:04.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.908+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6c09806d0 con 0x55f6c0a2b670 2024-04-17T07:22:04.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.910+0000 7fa617400640 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f6c0a2b670 0x55f6c0a2da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.912+0000 7fa605600640 10 client.36443 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.913+0000 7fa617400640 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f6c0a2b670 0x55f6c0a2da60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09ad89fa99ff521 server_cookie=942a47e7f706e496 in_seq=0 out_seq=0 2024-04-17T07:22:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.914+0000 7fa605600640 1 -- 172.21.15.86:0/1077392014 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6c09806d0 con 0x55f6c0a2b670 2024-04-17T07:22:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.914+0000 7fa605600640 10 client.36443 handle_command_reply: tid=0 2024-04-17T07:22:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.914+0000 7fa621827740 1 client.36443 shutdown 2024-04-17T07:22:04.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.915+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a156d0 msgr2=0x55f6c0a1fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:04.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.915+0000 7fa621827740 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a156d0 0x55f6c0a1fc20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa6100cdab0 tx=0x7fa6100d35f0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.916+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 shutdown_connections 2024-04-17T07:22:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.916+0000 7fa621827740 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f6c0a2b670 0x55f6c0a2da60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.916+0000 7fa621827740 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a1b230 0x55f6c0a271a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.916+0000 7fa621827740 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a160c0 0x55f6c0a20160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.916+0000 7fa605600640 0 client.36443 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.916+0000 7fa621827740 1 --2- 172.21.15.86:0/1077392014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a156d0 0x55f6c0a1fc20 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa6100cdab0 tx=0x7fa6100d35f0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.916+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 >> 172.21.15.86:0/1077392014 conn(0x55f6c0a08430 msgr2=0x55f6c0a075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:04.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.916+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 shutdown_connections 2024-04-17T07:22:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.916+0000 7fa621827740 1 -- 172.21.15.86:0/1077392014 wait complete. 2024-04-17T07:22:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.917+0000 7fa621827740 20 client.36443 trim_cache size 0 max 16384 2024-04-17T07:22:04.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.959+0000 7fa621827740 1 Processor -- start 2024-04-17T07:22:04.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.959+0000 7fa621827740 1 -- start start 2024-04-17T07:22:04.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.960+0000 7fa621827740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.960+0000 7fa621827740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 0x55f6c0a27160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.960+0000 7fa621827740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 0x55f6c0a32020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.960+0000 7fa621827740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa618072330 con 0x55f6c0a160c0 2024-04-17T07:22:04.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.960+0000 7fa621827740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa618073e70 con 0x55f6c0a07460 2024-04-17T07:22:04.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.960+0000 7fa621827740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa618072a00 con 0x55f6c0a1b230 2024-04-17T07:22:04.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.961+0000 7fa617e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.961+0000 7fa617e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52168/0 (socket says 172.21.15.86:52168) 2024-04-17T07:22:04.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.961+0000 7fa617e00640 1 -- 172.21.15.86:0/98690976 learned_addr learned my addr 172.21.15.86:0/98690976 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:04.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.961+0000 7fa61cc00640 1 --2- 172.21.15.86:0/98690976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 0x55f6c0a32020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.962+0000 7fa617400640 1 --2- 172.21.15.86:0/98690976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 0x55f6c0a27160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.963+0000 7fa617e00640 1 -- 172.21.15.86:0/98690976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 msgr2=0x55f6c0a27160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:04.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.963+0000 7fa617e00640 1 --2- 172.21.15.86:0/98690976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 0x55f6c0a27160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.963+0000 7fa617e00640 1 -- 172.21.15.86:0/98690976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 msgr2=0x55f6c0a32020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:04.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.963+0000 7fa617e00640 1 --2- 172.21.15.86:0/98690976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 0x55f6c0a32020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.963+0000 7fa617e00640 1 -- 172.21.15.86:0/98690976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c0645170 con 0x55f6c0a160c0 2024-04-17T07:22:04.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.963+0000 7fa617400640 1 --2- 172.21.15.86:0/98690976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 0x55f6c0a27160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:04.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.963+0000 7fa617e00640 1 --2- 172.21.15.86:0/98690976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a200c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa61009bcd0 tx=0x7fa61009a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528e87514c732d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:04.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.964+0000 7fa607e00640 1 -- 172.21.15.86:0/98690976 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610020070 con 0x55f6c0a160c0 2024-04-17T07:22:04.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.964+0000 7fa607e00640 1 -- 172.21.15.86:0/98690976 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6100bc350 con 0x55f6c0a160c0 2024-04-17T07:22:04.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.964+0000 7fa607e00640 1 -- 172.21.15.86:0/98690976 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61009bda0 con 0x55f6c0a160c0 2024-04-17T07:22:04.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.965+0000 7fa621827740 1 -- 172.21.15.86:0/98690976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 msgr2=0x55f6c0a200c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:04.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.965+0000 7fa621827740 1 --2- 172.21.15.86:0/98690976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a200c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa61009bcd0 tx=0x7fa61009a4e0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.966+0000 7fa621827740 1 -- 172.21.15.86:0/98690976 shutdown_connections 2024-04-17T07:22:04.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.966+0000 7fa621827740 1 --2- 172.21.15.86:0/98690976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 0x55f6c0a32020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.966+0000 7fa621827740 1 --2- 172.21.15.86:0/98690976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 0x55f6c0a27160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.966+0000 7fa621827740 1 --2- 172.21.15.86:0/98690976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a200c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.966+0000 7fa621827740 1 -- 172.21.15.86:0/98690976 >> 172.21.15.86:0/98690976 conn(0x55f6c0a08430 msgr2=0x55f6c0a06d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:04.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.966+0000 7fa621827740 1 -- 172.21.15.86:0/98690976 shutdown_connections 2024-04-17T07:22:04.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.966+0000 7fa621827740 1 -- 172.21.15.86:0/98690976 wait complete. 2024-04-17T07:22:04.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.967+0000 7fa621827740 1 -- start start 2024-04-17T07:22:04.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.967+0000 7fa621827740 1 Processor -- start 2024-04-17T07:22:04.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.968+0000 7fa621827740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 0x55f6c0a26f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.968+0000 7fa621827740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.968+0000 7fa621827740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 0x55f6c0a25ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.968+0000 7fa621827740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6180737a0 con 0x55f6c0a160c0 2024-04-17T07:22:04.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.968+0000 7fa617e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 0x55f6c0a26f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.968+0000 7fa617e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 0x55f6c0a26f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53298/0 (socket says 172.21.15.86:53298) 2024-04-17T07:22:04.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.968+0000 7fa617e00640 1 -- 172.21.15.86:0/4070239806 learned_addr learned my addr 172.21.15.86:0/4070239806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:04.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.968+0000 7fa61cc00640 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 0x55f6c0a25ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.969+0000 7fa617400640 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.969+0000 7fa621827740 1 -- 172.21.15.86:0/4070239806 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6180730d0 con 0x55f6c0a07460 2024-04-17T07:22:04.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.969+0000 7fa621827740 1 -- 172.21.15.86:0/4070239806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6180703b0 con 0x55f6c0a1b230 2024-04-17T07:22:04.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.969+0000 7fa621827740 10 client.0 _ll_register_callbacks cb 0x7fa608007cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:04.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.970+0000 7fa61cc00640 1 -- 172.21.15.86:0/4070239806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 msgr2=0x55f6c0a26f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:04.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.970+0000 7fa61cc00640 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 0x55f6c0a26f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.970+0000 7fa61cc00640 1 -- 172.21.15.86:0/4070239806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 msgr2=0x55f6c0a255a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:04.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.970+0000 7fa61cc00640 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a255a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.970+0000 7fa61cc00640 1 -- 172.21.15.86:0/4070239806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c0a26250 con 0x55f6c0a1b230 2024-04-17T07:22:04.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.970+0000 7fa61cc00640 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 0x55f6c0a25ae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa608007fa0 tx=0x7fa608015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d878aa6e521b3dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:04.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.970+0000 7fa605600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:22:04.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.971+0000 7fa605600640 1 -- 172.21.15.86:0/4070239806 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60802d060 con 0x55f6c0a1b230 2024-04-17T07:22:04.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.971+0000 7fa621827740 1 -- 172.21.15.86:0/4070239806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f6c097a540 con 0x55f6c0a1b230 2024-04-17T07:22:04.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.971+0000 7fa605600640 1 -- 172.21.15.86:0/4070239806 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa60801a450 con 0x55f6c0a1b230 2024-04-17T07:22:04.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.972+0000 7fa605600640 1 -- 172.21.15.86:0/4070239806 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa608034050 con 0x55f6c0a1b230 2024-04-17T07:22:04.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.972+0000 7fa621827740 10 client.26473 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:04.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.972+0000 7fa621827740 1 -- 172.21.15.86:0/4070239806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6c0a92980 con 0x55f6c0a1b230 2024-04-17T07:22:04.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.973+0000 7fa605600640 1 -- 172.21.15.86:0/4070239806 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f6c097a540 con 0x55f6c0a1b230 2024-04-17T07:22:04.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.973+0000 7fa605600640 1 -- 172.21.15.86:0/4070239806 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa6080287d0 con 0x55f6c0a1b230 2024-04-17T07:22:04.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.973+0000 7fa621827740 10 client.26473 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:04.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.973+0000 7fa621827740 10 client.26473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:04.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.973+0000 7fa621827740 20 client.26473 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:04.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.973+0000 7fa621827740 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f6c0a93310 0x55f6c0a95730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:04.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.973+0000 7fa621827740 4 client.26473 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:22:04.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.973+0000 7fa621827740 1 -- 172.21.15.86:0/4070239806 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f6c08f6aa0 con 0x55f6c0a93310 2024-04-17T07:22:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.974+0000 7fa617e00640 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f6c0a93310 0x55f6c0a95730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.975+0000 7fa617e00640 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f6c0a93310 0x55f6c0a95730 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e9f6d9caf2ed083 server_cookie=17e560e648e4504b in_seq=0 out_seq=0 2024-04-17T07:22:04.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.975+0000 7fa605600640 10 client.26473 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa605600640 1 -- 172.21.15.86:0/4070239806 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f6c08f6aa0 con 0x55f6c0a93310 2024-04-17T07:22:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa605600640 10 client.26473 handle_command_reply: tid=0 2024-04-17T07:22:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa621827740 1 client.26473 shutdown 2024-04-17T07:22:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa621827740 1 -- 172.21.15.86:0/4070239806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 msgr2=0x55f6c0a25ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa621827740 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 0x55f6c0a25ae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa608007fa0 tx=0x7fa608015000 comp rx=0 tx=0).stop 2024-04-17T07:22:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa621827740 1 -- 172.21.15.86:0/4070239806 shutdown_connections 2024-04-17T07:22:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa621827740 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f6c0a93310 0x55f6c0a95730 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa621827740 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6c0a1b230 0x55f6c0a25ae0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa621827740 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6c0a160c0 0x55f6c0a255a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa621827740 1 --2- 172.21.15.86:0/4070239806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f6c0a07460 0x55f6c0a26f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa621827740 1 -- 172.21.15.86:0/4070239806 >> 172.21.15.86:0/4070239806 conn(0x55f6c0a08430 msgr2=0x55f6c0a0b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:04.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.979+0000 7fa605600640 0 client.26473 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.980+0000 7fa621827740 1 -- 172.21.15.86:0/4070239806 shutdown_connections 2024-04-17T07:22:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.980+0000 7fa621827740 1 -- 172.21.15.86:0/4070239806 wait complete. 2024-04-17T07:22:04.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.980+0000 7fa621827740 20 client.26473 trim_cache size 0 max 16384 2024-04-17T07:22:04.980 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:22:04.980 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:22:04.981 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "82e23a98-4a53-4f13-b75d-07c5678fe875", 2024-04-17T07:22:04.981 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:22:04.981 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:22:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 -- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5fc0a3c80 msgr2=0x7fa5fc0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:04.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5fc0a3c80 0x7fa5fc0a6140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55f6c0a16920 tx=0x7fa608007520 comp rx=0 tx=0).stop 2024-04-17T07:22:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 -- 172.21.15.86:0/3795121612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6181041c0 msgr2=0x7fa61819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6181041c0 0x7fa61819a580 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa61000d770 tx=0x7fa61000dc30 comp rx=0 tx=0).stop 2024-04-17T07:22:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 -- 172.21.15.86:0/3795121612 shutdown_connections 2024-04-17T07:22:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa5fc0a3c80 0x7fa5fc0a6140 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa61819fb20 0x7fa6181a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa618105d70 0x7fa61819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 --2- 172.21.15.86:0/3795121612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6181041c0 0x7fa61819a580 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 -- 172.21.15.86:0/3795121612 >> 172.21.15.86:0/3795121612 conn(0x7fa6180ff710 msgr2=0x7fa618108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 -- 172.21.15.86:0/3795121612 shutdown_connections 2024-04-17T07:22:04.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:04.982+0000 7fa61ec00640 1 -- 172.21.15.86:0/3795121612 wait complete. 2024-04-17T07:22:05.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:04 smithi098 ceph-mon[35418]: pgmap v671: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 14 MiB/s wr, 789 op/s 2024-04-17T07:22:05.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:22:05.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:22:05.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.314+0000 7fa746800640 1 -- 172.21.15.86:0/941049050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7401097b0 msgr2=0x7fa740109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.314+0000 7fa746800640 1 --2- 172.21.15.86:0/941049050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7401097b0 0x7fa740109c30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009900 tx=0x7fa72c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:22:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.315+0000 7fa746800640 1 -- 172.21.15.86:0/941049050 shutdown_connections 2024-04-17T07:22:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.315+0000 7fa746800640 1 --2- 172.21.15.86:0/941049050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa74010a170 0x7fa740110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.315+0000 7fa746800640 1 --2- 172.21.15.86:0/941049050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7401097b0 0x7fa740109c30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.315+0000 7fa746800640 1 --2- 172.21.15.86:0/941049050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7401085c0 0x7fa7401089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.315+0000 7fa746800640 1 -- 172.21.15.86:0/941049050 >> 172.21.15.86:0/941049050 conn(0x7fa740077560 msgr2=0x7fa7400799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.315+0000 7fa746800640 1 -- 172.21.15.86:0/941049050 shutdown_connections 2024-04-17T07:22:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.316+0000 7fa746800640 1 -- 172.21.15.86:0/941049050 wait complete. 2024-04-17T07:22:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.317+0000 7fa746800640 1 Processor -- start 2024-04-17T07:22:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.318+0000 7fa746800640 1 -- start start 2024-04-17T07:22:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.318+0000 7fa746800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7401085c0 0x7fa74019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.318+0000 7fa746800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7401097b0 0x7fa74019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.318+0000 7fa746800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa74010a170 0x7fa7401a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.318+0000 7fa746800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa740114130 con 0x7fa7401085c0 2024-04-17T07:22:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.318+0000 7fa746800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa740113fb0 con 0x7fa7401097b0 2024-04-17T07:22:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.318+0000 7fa746800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401142b0 con 0x7fa74010a170 2024-04-17T07:22:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.318+0000 7fa73ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7401097b0 0x7fa74019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.318+0000 7fa73ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7401097b0 0x7fa74019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53316/0 (socket says 172.21.15.86:53316) 2024-04-17T07:22:05.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.318+0000 7fa73ea00640 1 -- 172.21.15.86:0/103305610 learned_addr learned my addr 172.21.15.86:0/103305610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.319+0000 7fa73f400640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7401085c0 0x7fa74019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.319+0000 7fa73ea00640 1 -- 172.21.15.86:0/103305610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa74010a170 msgr2=0x7fa7401a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.319+0000 7fa73ea00640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa74010a170 0x7fa7401a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.319+0000 7fa73ea00640 1 -- 172.21.15.86:0/103305610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7401085c0 msgr2=0x7fa74019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.319+0000 7fa73ea00640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7401085c0 0x7fa74019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.319+0000 7fa73ea00640 1 -- 172.21.15.86:0/103305610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74007a8e0 con 0x7fa7401097b0 2024-04-17T07:22:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.320+0000 7fa73ea00640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7401097b0 0x7fa74019ed40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa72c000c00 tx=0x7fa72c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.320+0000 7fa737e00640 1 -- 172.21.15.86:0/103305610 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c043020 con 0x7fa7401097b0 2024-04-17T07:22:05.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.320+0000 7fa746800640 1 -- 172.21.15.86:0/103305610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74007ab70 con 0x7fa7401097b0 2024-04-17T07:22:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.320+0000 7fa746800640 1 -- 172.21.15.86:0/103305610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74007b110 con 0x7fa7401097b0 2024-04-17T07:22:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.320+0000 7fa737e00640 1 -- 172.21.15.86:0/103305610 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa72c004aa0 con 0x7fa7401097b0 2024-04-17T07:22:05.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.320+0000 7fa737e00640 1 -- 172.21.15.86:0/103305610 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c003d00 con 0x7fa7401097b0 2024-04-17T07:22:05.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.327+0000 7fa737e00640 1 -- 172.21.15.86:0/103305610 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa72c046020 con 0x7fa7401097b0 2024-04-17T07:22:05.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.329+0000 7fa737e00640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7240a3cb0 0x7fa7240a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.329+0000 7fa737e00640 1 -- 172.21.15.86:0/103305610 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa72c0f2790 con 0x7fa7401097b0 2024-04-17T07:22:05.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.327+0000 7fa749572740 1 Processor -- start 2024-04-17T07:22:05.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.330+0000 7fa749572740 1 -- start start 2024-04-17T07:22:05.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.330+0000 7fa73fe00640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7240a3cb0 0x7fa7240a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.331+0000 7fa749572740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce531b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.331+0000 7fa749572740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531bb80 0x555ce5320680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.331+0000 7fa749572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce5320cf0 0x555ce5323180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.331+0000 7fa749572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ce5282120 con 0x555ce5320cf0 2024-04-17T07:22:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.331+0000 7fa749572740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555ce5228340 con 0x555ce531b190 2024-04-17T07:22:05.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.331+0000 7fa749572740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555ce527e0e0 con 0x555ce531bb80 2024-04-17T07:22:05.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.332+0000 7fa73ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531bb80 0x555ce5320680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.332+0000 7fa73ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531bb80 0x555ce5320680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45274/0 (socket says 172.21.15.86:45274) 2024-04-17T07:22:05.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.332+0000 7fa73ea00640 1 -- 172.21.15.86:0/3395868839 learned_addr learned my addr 172.21.15.86:0/3395868839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:05.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.333+0000 7fa73f400640 1 --2- 172.21.15.86:0/3395868839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce531b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.333+0000 7fa73ea00640 1 -- 172.21.15.86:0/3395868839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 msgr2=0x555ce531b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:05.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.333+0000 7fa73ea00640 1 --2- 172.21.15.86:0/3395868839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce531b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.333+0000 7fa73ea00640 1 -- 172.21.15.86:0/3395868839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce5320cf0 msgr2=0x555ce5323180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:22:05.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.333+0000 7fa73ea00640 1 --2- 172.21.15.86:0/3395868839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce5320cf0 0x555ce5323180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.333+0000 7fa73ea00640 1 -- 172.21.15.86:0/3395868839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce505e7d0 con 0x555ce531bb80 2024-04-17T07:22:05.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.334+0000 7fa73fe00640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7240a3cb0 0x7fa7240a6170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa738005c40 tx=0x7fa738005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:05.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.335+0000 7fa73ea00640 1 --2- 172.21.15.86:0/3395868839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531bb80 0x555ce5320680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0be620 tx=0x7fa72c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a98234b820a02040 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:05.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.335+0000 7fa734c00640 1 -- 172.21.15.86:0/3395868839 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0044e0 con 0x555ce531bb80 2024-04-17T07:22:05.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.335+0000 7fa734c00640 1 -- 172.21.15.86:0/3395868839 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa72c004c50 con 0x555ce531bb80 2024-04-17T07:22:05.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.336+0000 7fa734c00640 1 -- 172.21.15.86:0/3395868839 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0bc990 con 0x555ce531bb80 2024-04-17T07:22:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.337+0000 7fa749572740 1 -- 172.21.15.86:0/3395868839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531bb80 msgr2=0x555ce5320680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.337+0000 7fa749572740 1 --2- 172.21.15.86:0/3395868839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531bb80 0x555ce5320680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0be620 tx=0x7fa72c009260 comp rx=0 tx=0).stop 2024-04-17T07:22:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.337+0000 7fa749572740 1 -- 172.21.15.86:0/3395868839 shutdown_connections 2024-04-17T07:22:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.337+0000 7fa749572740 1 --2- 172.21.15.86:0/3395868839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce5320cf0 0x555ce5323180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.337+0000 7fa749572740 1 --2- 172.21.15.86:0/3395868839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531bb80 0x555ce5320680 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.337+0000 7fa749572740 1 --2- 172.21.15.86:0/3395868839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce531b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.337+0000 7fa749572740 1 -- 172.21.15.86:0/3395868839 >> 172.21.15.86:0/3395868839 conn(0x555ce5316700 msgr2=0x555ce5316b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:05.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.337+0000 7fa749572740 1 -- 172.21.15.86:0/3395868839 shutdown_connections 2024-04-17T07:22:05.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.337+0000 7fa749572740 1 -- 172.21.15.86:0/3395868839 wait complete. 2024-04-17T07:22:05.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.338+0000 7fa749572740 1 -- start start 2024-04-17T07:22:05.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.338+0000 7fa749572740 1 Processor -- start 2024-04-17T07:22:05.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.340+0000 7fa749572740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531b190 0x555ce53254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.340+0000 7fa749572740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531bb80 0x555ce53259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.340+0000 7fa749572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce5320cf0 0x555ce532ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.340+0000 7fa749572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ce5323a60 con 0x555ce5320cf0 2024-04-17T07:22:05.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.340+0000 7fa749572740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555ce523b710 con 0x555ce531bb80 2024-04-17T07:22:05.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.340+0000 7fa749572740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555ce5323be0 con 0x555ce531b190 2024-04-17T07:22:05.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.340+0000 7fa749572740 10 client.0 _ll_register_callbacks cb 0x555ce5236af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:05.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.340+0000 7fa73f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531b190 0x555ce53254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.340+0000 7fa73f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531b190 0x555ce53254a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45284/0 (socket says 172.21.15.86:45284) 2024-04-17T07:22:05.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.340+0000 7fa73f400640 1 -- 172.21.15.86:0/3663526708 learned_addr learned my addr 172.21.15.86:0/3663526708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:05.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.341+0000 7fa73fe00640 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce5320cf0 0x555ce532ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.341+0000 7fa73ea00640 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531bb80 0x555ce53259e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.341+0000 7fa73f400640 1 -- 172.21.15.86:0/3663526708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531bb80 msgr2=0x555ce53259e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:05.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.341+0000 7fa73f400640 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531bb80 0x555ce53259e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.341+0000 7fa73f400640 1 -- 172.21.15.86:0/3663526708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce5320cf0 msgr2=0x555ce532ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:05.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.341+0000 7fa73f400640 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce5320cf0 0x555ce532ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.341+0000 7fa73f400640 1 -- 172.21.15.86:0/3663526708 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce4f943f0 con 0x555ce531b190 2024-04-17T07:22:05.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.341+0000 7fa73ea00640 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531bb80 0x555ce53259e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:05.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.342+0000 7fa73f400640 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531b190 0x555ce53254a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa73000be50 tx=0x7fa7300117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd4875618b0687cd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:05.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.342+0000 7fa729600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:22:05.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.342+0000 7fa729600640 1 -- 172.21.15.86:0/3663526708 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730004220 con 0x555ce531b190 2024-04-17T07:22:05.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.342+0000 7fa729600640 1 -- 172.21.15.86:0/3663526708 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa7300043c0 con 0x555ce531b190 2024-04-17T07:22:05.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.343+0000 7fa749572740 1 -- 172.21.15.86:0/3663526708 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa72c004c50 con 0x555ce531b190 2024-04-17T07:22:05.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.343+0000 7fa729600640 1 -- 172.21.15.86:0/3663526708 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7300035f0 con 0x555ce531b190 2024-04-17T07:22:05.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.343+0000 7fa749572740 10 client.26482 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:05.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.344+0000 7fa749572740 1 -- 172.21.15.86:0/3663526708 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ce4f540f0 con 0x555ce531b190 2024-04-17T07:22:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.345+0000 7fa729600640 1 -- 172.21.15.86:0/3663526708 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa72c004c50 con 0x555ce531b190 2024-04-17T07:22:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.345+0000 7fa729600640 1 -- 172.21.15.86:0/3663526708 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa73001b920 con 0x555ce531b190 2024-04-17T07:22:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.345+0000 7fa749572740 10 client.26482 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.345+0000 7fa749572740 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.345+0000 7fa749572740 20 client.26482 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.345+0000 7fa749572740 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555ce5331330 0x555ce53337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.345+0000 7fa749572740 4 client.26482 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.345+0000 7fa749572740 1 -- 172.21.15.86:0/3663526708 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ce528fb40 con 0x555ce5331330 2024-04-17T07:22:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.346+0000 7fa73ea00640 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555ce5331330 0x555ce53337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.347+0000 7fa73ea00640 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555ce5331330 0x555ce53337b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1c2e5657dcfb1f server_cookie=d5b9aaaaca044226 in_seq=0 out_seq=0 2024-04-17T07:22:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.347+0000 7fa729600640 10 client.26482 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:05.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.349+0000 7fa729600640 1 -- 172.21.15.86:0/3663526708 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ce528fb40 con 0x555ce5331330 2024-04-17T07:22:05.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.349+0000 7fa729600640 10 client.26482 handle_command_reply: tid=0 2024-04-17T07:22:05.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.349+0000 7fa749572740 1 client.26482 shutdown 2024-04-17T07:22:05.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.350+0000 7fa749572740 1 -- 172.21.15.86:0/3663526708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531b190 msgr2=0x555ce53254a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.350+0000 7fa749572740 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531b190 0x555ce53254a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa73000be50 tx=0x7fa7300117b0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.350+0000 7fa749572740 1 -- 172.21.15.86:0/3663526708 shutdown_connections 2024-04-17T07:22:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.350+0000 7fa749572740 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555ce5331330 0x555ce53337b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.350+0000 7fa749572740 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce5320cf0 0x555ce532ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.350+0000 7fa749572740 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531bb80 0x555ce53259e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.350+0000 7fa749572740 1 --2- 172.21.15.86:0/3663526708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce531b190 0x555ce53254a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.350+0000 7fa749572740 1 -- 172.21.15.86:0/3663526708 >> 172.21.15.86:0/3663526708 conn(0x555ce5316700 msgr2=0x555ce5315890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.350+0000 7fa729600640 0 client.26482 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.351+0000 7fa749572740 1 -- 172.21.15.86:0/3663526708 shutdown_connections 2024-04-17T07:22:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.351+0000 7fa749572740 1 -- 172.21.15.86:0/3663526708 wait complete. 2024-04-17T07:22:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.351+0000 7fa749572740 20 client.26482 trim_cache size 0 max 16384 2024-04-17T07:22:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.379+0000 7fa749572740 1 Processor -- start 2024-04-17T07:22:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.379+0000 7fa749572740 1 -- start start 2024-04-17T07:22:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.379+0000 7fa749572740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce5315670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.379+0000 7fa749572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.379+0000 7fa749572740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce5320cf0 0x555ce531d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.379+0000 7fa749572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa72000fb60 con 0x555ce531bb80 2024-04-17T07:22:05.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.379+0000 7fa749572740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa7200116a0 con 0x555ce531b190 2024-04-17T07:22:05.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.379+0000 7fa749572740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa720010230 con 0x555ce5320cf0 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.380+0000 7fa73ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.380+0000 7fa73ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52220/0 (socket says 172.21.15.86:52220) 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.380+0000 7fa73ea00640 1 -- 172.21.15.86:0/3175622063 learned_addr learned my addr 172.21.15.86:0/3175622063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.380+0000 7fa73f400640 1 --2- 172.21.15.86:0/3175622063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce5315670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.380+0000 7fa73ea00640 1 -- 172.21.15.86:0/3175622063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce5320cf0 msgr2=0x555ce531d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.380+0000 7fa73ea00640 1 --2- 172.21.15.86:0/3175622063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce5320cf0 0x555ce531d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.380+0000 7fa73ea00640 1 -- 172.21.15.86:0/3175622063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 msgr2=0x555ce5315670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.380+0000 7fa73ea00640 1 --2- 172.21.15.86:0/3175622063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce5315670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.380+0000 7fa73ea00640 1 -- 172.21.15.86:0/3175622063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce5310ce0 con 0x555ce531bb80 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.380+0000 7fa73f400640 1 --2- 172.21.15.86:0/3175622063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce5315670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.381+0000 7fa73ea00640 1 --2- 172.21.15.86:0/3175622063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532c9e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x555ce531c5f0 tx=0x7fa72c02bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e811239fda32aac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:05.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.381+0000 7fa72be00640 1 -- 172.21.15.86:0/3175622063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0dd070 con 0x555ce531bb80 2024-04-17T07:22:05.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.381+0000 7fa72be00640 1 -- 172.21.15.86:0/3175622063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa72c0a2070 con 0x555ce531bb80 2024-04-17T07:22:05.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.381+0000 7fa72be00640 1 -- 172.21.15.86:0/3175622063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c003660 con 0x555ce531bb80 2024-04-17T07:22:05.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.382+0000 7fa749572740 1 -- 172.21.15.86:0/3175622063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 msgr2=0x555ce532c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:05.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.382+0000 7fa749572740 1 --2- 172.21.15.86:0/3175622063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532c9e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x555ce531c5f0 tx=0x7fa72c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:22:05.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.382+0000 7fa749572740 1 -- 172.21.15.86:0/3175622063 shutdown_connections 2024-04-17T07:22:05.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.382+0000 7fa749572740 1 --2- 172.21.15.86:0/3175622063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce5320cf0 0x555ce531d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.382+0000 7fa749572740 1 --2- 172.21.15.86:0/3175622063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532c9e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.382+0000 7fa749572740 1 --2- 172.21.15.86:0/3175622063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce5315670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.382+0000 7fa749572740 1 -- 172.21.15.86:0/3175622063 >> 172.21.15.86:0/3175622063 conn(0x555ce5316700 msgr2=0x555ce531a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:05.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.382+0000 7fa749572740 1 -- 172.21.15.86:0/3175622063 shutdown_connections 2024-04-17T07:22:05.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.383+0000 7fa749572740 1 -- 172.21.15.86:0/3175622063 wait complete. 2024-04-17T07:22:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.383+0000 7fa749572740 1 -- start start 2024-04-17T07:22:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.383+0000 7fa749572740 1 Processor -- start 2024-04-17T07:22:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.384+0000 7fa749572740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce532a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.384+0000 7fa749572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.384+0000 7fa749572740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce5320cf0 0x555ce532b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.384+0000 7fa749572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa720010fd0 con 0x555ce531bb80 2024-04-17T07:22:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.384+0000 7fa749572740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa720010900 con 0x555ce531b190 2024-04-17T07:22:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.384+0000 7fa749572740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa72000db30 con 0x555ce5320cf0 2024-04-17T07:22:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.384+0000 7fa749572740 10 client.0 _ll_register_callbacks cb 0x7fa73000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.384+0000 7fa73ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.384+0000 7fa73fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce5320cf0 0x555ce532b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.384+0000 7fa73f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce532a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.385+0000 7fa73fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce5320cf0 0x555ce532b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45308/0 (socket says 172.21.15.86:45308) 2024-04-17T07:22:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.385+0000 7fa73fe00640 1 -- 172.21.15.86:0/1227416757 learned_addr learned my addr 172.21.15.86:0/1227416757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.385+0000 7fa73ea00640 1 -- 172.21.15.86:0/1227416757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce5320cf0 msgr2=0x555ce532b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.385+0000 7fa73ea00640 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce5320cf0 0x555ce532b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.385+0000 7fa73ea00640 1 -- 172.21.15.86:0/1227416757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 msgr2=0x555ce532a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.385+0000 7fa73ea00640 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce532a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.386+0000 7fa73ea00640 1 -- 172.21.15.86:0/1227416757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce532bfd0 con 0x555ce531bb80 2024-04-17T07:22:05.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.386+0000 7fa73f400640 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce532a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.386+0000 7fa73ea00640 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532a9d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x555ce532d8f0 tx=0x7fa72c02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c965ef6ae0fe5501 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.387+0000 7fa729600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.387+0000 7fa729600640 1 -- 172.21.15.86:0/1227416757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0dd040 con 0x555ce531bb80 2024-04-17T07:22:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.387+0000 7fa729600640 1 -- 172.21.15.86:0/1227416757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa72c0bad00 con 0x555ce531bb80 2024-04-17T07:22:05.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.388+0000 7fa749572740 1 -- 172.21.15.86:0/1227416757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa72c0a2070 con 0x555ce531bb80 2024-04-17T07:22:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.388+0000 7fa729600640 1 -- 172.21.15.86:0/1227416757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa72c004c50 con 0x555ce531bb80 2024-04-17T07:22:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.388+0000 7fa749572740 10 client.17499 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:05.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.388+0000 7fa749572740 1 -- 172.21.15.86:0/1227416757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ce5397cc0 con 0x555ce531bb80 2024-04-17T07:22:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.389+0000 7fa729600640 1 -- 172.21.15.86:0/1227416757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa72c0a2070 con 0x555ce531bb80 2024-04-17T07:22:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.389+0000 7fa729600640 1 -- 172.21.15.86:0/1227416757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa72c0c6f60 con 0x555ce531bb80 2024-04-17T07:22:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.390+0000 7fa749572740 10 client.17499 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.390+0000 7fa749572740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.390+0000 7fa749572740 20 client.17499 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.390+0000 7fa749572740 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555ce5398310 0x555ce539a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.390+0000 7fa749572740 4 client.17499 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:22:05.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.390+0000 7fa749572740 1 -- 172.21.15.86:0/1227416757 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ce52899b0 con 0x555ce5398310 2024-04-17T07:22:05.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.391+0000 7fa73f400640 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555ce5398310 0x555ce539a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:05.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.393+0000 7fa73f400640 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555ce5398310 0x555ce539a700 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f60bef7f3b27a4d server_cookie=4e5ddf40b061e120 in_seq=0 out_seq=0 2024-04-17T07:22:05.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.393+0000 7fa729600640 10 client.17499 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:05.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.394+0000 7fa729600640 1 -- 172.21.15.86:0/1227416757 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555ce52899b0 con 0x555ce5398310 2024-04-17T07:22:05.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.394+0000 7fa729600640 10 client.17499 handle_command_reply: tid=0 2024-04-17T07:22:05.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.394+0000 7fa749572740 1 client.17499 shutdown 2024-04-17T07:22:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.395+0000 7fa749572740 1 -- 172.21.15.86:0/1227416757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 msgr2=0x555ce532a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.395+0000 7fa749572740 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532a9d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x555ce532d8f0 tx=0x7fa72c02c670 comp rx=0 tx=0).stop 2024-04-17T07:22:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.396+0000 7fa749572740 1 -- 172.21.15.86:0/1227416757 shutdown_connections 2024-04-17T07:22:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.396+0000 7fa749572740 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555ce5398310 0x555ce539a700 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.396+0000 7fa749572740 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555ce5320cf0 0x555ce532b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.396+0000 7fa749572740 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ce531bb80 0x555ce532a9d0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.396+0000 7fa749572740 1 --2- 172.21.15.86:0/1227416757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ce531b190 0x555ce532a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.396+0000 7fa749572740 1 -- 172.21.15.86:0/1227416757 >> 172.21.15.86:0/1227416757 conn(0x555ce5316700 msgr2=0x555ce53380b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.396+0000 7fa729600640 0 client.17499 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.396+0000 7fa749572740 1 -- 172.21.15.86:0/1227416757 shutdown_connections 2024-04-17T07:22:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.396+0000 7fa749572740 1 -- 172.21.15.86:0/1227416757 wait complete. 2024-04-17T07:22:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.397+0000 7fa749572740 20 client.17499 trim_cache size 0 max 16384 2024-04-17T07:22:05.397 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:22:05.397 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:22:05.397 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:22:05.397 INFO:teuthology.orchestra.run.smithi086.stdout: "82e23a98-4a53-4f13-b75d-07c5678fe875": { 2024-04-17T07:22:05.397 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:22:05.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "82e23a98-4a53-4f13-b75d-07c5678fe875", 2024-04-17T07:22:05.398 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:22:05.398 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:22:05.398 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:22:05.398 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:22:05.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 -- 172.21.15.86:0/103305610 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7240a3cb0 msgr2=0x7fa7240a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7240a3cb0 0x7fa7240a6170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa738005c40 tx=0x7fa738005bd0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 -- 172.21.15.86:0/103305610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7401097b0 msgr2=0x7fa74019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7401097b0 0x7fa74019ed40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa72c000c00 tx=0x7fa72c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 -- 172.21.15.86:0/103305610 shutdown_connections 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa7240a3cb0 0x7fa7240a6170 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa74010a170 0x7fa7401a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7401097b0 0x7fa74019ed40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 --2- 172.21.15.86:0/103305610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7401085c0 0x7fa74019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 -- 172.21.15.86:0/103305610 >> 172.21.15.86:0/103305610 conn(0x7fa740077560 msgr2=0x7fa74010cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 -- 172.21.15.86:0/103305610 shutdown_connections 2024-04-17T07:22:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:05.399+0000 7fa746800640 1 -- 172.21.15.86:0/103305610 wait complete. 2024-04-17T07:22:05.423 INFO:tasks.cephfs.filesystem:scrub status for tag:82e23a98-4a53-4f13-b75d-07c5678fe875 - {'path': '/', 'tag': '82e23a98-4a53-4f13-b75d-07c5678fe875', 'options': 'recursive,force'} 2024-04-17T07:22:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:05 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:22:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:05 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:05 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:22:05.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:05 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:22:05.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:05 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:05.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:05 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:22:06.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:05 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:22:06.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:05 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:06.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:05 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:22:06.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:06 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:06.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:06 smithi103 ceph-mon[35495]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 12 MiB/s wr, 703 op/s 2024-04-17T07:22:06.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:06 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:06.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:06 smithi086 ceph-mon[31733]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 12 MiB/s wr, 703 op/s 2024-04-17T07:22:07.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:06 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:07.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:06 smithi098 ceph-mon[35418]: pgmap v672: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 12 MiB/s wr, 703 op/s 2024-04-17T07:22:08.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:08 smithi086 ceph-mon[31733]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 14 MiB/s wr, 794 op/s 2024-04-17T07:22:09.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:08 smithi098 ceph-mon[35418]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 14 MiB/s wr, 794 op/s 2024-04-17T07:22:09.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:08 smithi103 ceph-mon[35495]: pgmap v673: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 14 MiB/s wr, 794 op/s 2024-04-17T07:22:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 165.0 s, 659.6 tps, lat 151.694 ms stddev 198.286 2024-04-17T07:22:10.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:10 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:22:10.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:10 smithi086 ceph-mon[31733]: pgmap v674: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 12 MiB/s wr, 703 op/s 2024-04-17T07:22:11.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:10 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:22:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:10 smithi098 ceph-mon[35418]: pgmap v674: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 12 MiB/s wr, 703 op/s 2024-04-17T07:22:11.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:10 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:22:11.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:10 smithi103 ceph-mon[35495]: pgmap v674: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 12 MiB/s wr, 703 op/s 2024-04-17T07:22:12.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:12 smithi086 ceph-mon[31733]: pgmap v675: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-04-17T07:22:13.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:12 smithi098 ceph-mon[35418]: pgmap v675: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-04-17T07:22:13.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:12 smithi103 ceph-mon[35495]: pgmap v675: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-04-17T07:22:13.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:13 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:22:14.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:13 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:22:14.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:13 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:22:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 170.0 s, 578.8 tps, lat 171.539 ms stddev 217.453 2024-04-17T07:22:14.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:14 smithi086 ceph-mon[31733]: pgmap v676: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 12 MiB/s wr, 771 op/s 2024-04-17T07:22:14.928 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:14 smithi103 ceph-mon[35495]: pgmap v676: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 12 MiB/s wr, 771 op/s 2024-04-17T07:22:15.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:14 smithi098 ceph-mon[35418]: pgmap v676: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 12 MiB/s wr, 771 op/s 2024-04-17T07:22:15.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:22:15.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:22:15.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:15 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:22:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:22:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:22:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:15 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:22:16.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:22:16.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:22:16.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:15 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:22:17.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:17 smithi098 ceph-mon[35418]: pgmap v677: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 686 op/s 2024-04-17T07:22:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:17 smithi103 ceph-mon[35495]: pgmap v677: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 686 op/s 2024-04-17T07:22:17.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:17 smithi086 ceph-mon[31733]: pgmap v677: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 686 op/s 2024-04-17T07:22:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 175.0 s, 618.0 tps, lat 163.431 ms stddev 201.287 2024-04-17T07:22:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:19 smithi098 ceph-mon[35418]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-04-17T07:22:19.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:19 smithi103 ceph-mon[35495]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-04-17T07:22:19.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:19 smithi086 ceph-mon[31733]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-04-17T07:22:21.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:21 smithi098 ceph-mon[35418]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-17T07:22:21.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:21 smithi103 ceph-mon[35495]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-17T07:22:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:21 smithi086 ceph-mon[31733]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-17T07:22:23.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:23 smithi098 ceph-mon[35418]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 754 op/s 2024-04-17T07:22:23.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:23 smithi103 ceph-mon[35495]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 754 op/s 2024-04-17T07:22:23.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:23 smithi086 ceph-mon[31733]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 754 op/s 2024-04-17T07:22:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 180.0 s, 640.8 tps, lat 141.934 ms stddev 68.057 2024-04-17T07:22:25.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:25 smithi098 ceph-mon[35418]: pgmap v681: 129 pgs: 129 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-17T07:22:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:25 smithi103 ceph-mon[35495]: pgmap v681: 129 pgs: 129 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-17T07:22:25.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:25 smithi086 ceph-mon[31733]: pgmap v681: 129 pgs: 129 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-17T07:22:27.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:27 smithi098 ceph-mon[35418]: pgmap v682: 129 pgs: 129 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.9 MiB/s wr, 626 op/s 2024-04-17T07:22:27.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:27 smithi103 ceph-mon[35495]: pgmap v682: 129 pgs: 129 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.9 MiB/s wr, 626 op/s 2024-04-17T07:22:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:27 smithi086 ceph-mon[31733]: pgmap v682: 129 pgs: 129 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.9 MiB/s wr, 626 op/s 2024-04-17T07:22:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 185.0 s, 753.4 tps, lat 144.241 ms stddev 202.012 2024-04-17T07:22:29.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:29 smithi098 ceph-mon[35418]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 11 MiB/s wr, 716 op/s 2024-04-17T07:22:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:29 smithi103 ceph-mon[35495]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 11 MiB/s wr, 716 op/s 2024-04-17T07:22:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:29 smithi086 ceph-mon[31733]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 11 MiB/s wr, 716 op/s 2024-04-17T07:22:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:31 smithi098 ceph-mon[35418]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 692 op/s 2024-04-17T07:22:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:31 smithi103 ceph-mon[35495]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 692 op/s 2024-04-17T07:22:31.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:31 smithi086 ceph-mon[31733]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 11 MiB/s wr, 692 op/s 2024-04-17T07:22:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:33 smithi098 ceph-mon[35418]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 755 op/s 2024-04-17T07:22:33.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:33 smithi103 ceph-mon[35495]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 755 op/s 2024-04-17T07:22:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:33 smithi086 ceph-mon[31733]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 12 MiB/s wr, 755 op/s 2024-04-17T07:22:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 190.0 s, 613.6 tps, lat 162.237 ms stddev 224.794 2024-04-17T07:22:35.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:22:35.424 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:22:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:35 smithi098 ceph-mon[35418]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 760 op/s 2024-04-17T07:22:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:35 smithi103 ceph-mon[35495]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 760 op/s 2024-04-17T07:22:35.529 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:35 smithi086 ceph-mon[31733]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 760 op/s 2024-04-17T07:22:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.730+0000 7f5b17800640 1 -- 172.21.15.86:0/3201914606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08095e60 msgr2=0x7f5b080962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.730+0000 7f5b17800640 1 --2- 172.21.15.86:0/3201914606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08095e60 0x7f5b080962e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009900 tx=0x7f5afc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:22:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.731+0000 7f5b17800640 1 -- 172.21.15.86:0/3201914606 shutdown_connections 2024-04-17T07:22:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.731+0000 7f5b17800640 1 --2- 172.21.15.86:0/3201914606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b08096820 0x7f5b0809d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.731+0000 7f5b17800640 1 --2- 172.21.15.86:0/3201914606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08095e60 0x7f5b080962e0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.731+0000 7f5b17800640 1 --2- 172.21.15.86:0/3201914606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b08094c70 0x7f5b08095090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.731+0000 7f5b17800640 1 -- 172.21.15.86:0/3201914606 >> 172.21.15.86:0/3201914606 conn(0x7f5b080901c0 msgr2=0x7f5b08092620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.731+0000 7f5b17800640 1 -- 172.21.15.86:0/3201914606 shutdown_connections 2024-04-17T07:22:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.731+0000 7f5b17800640 1 -- 172.21.15.86:0/3201914606 wait complete. 2024-04-17T07:22:35.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.732+0000 7f5b17800640 1 Processor -- start 2024-04-17T07:22:35.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.733+0000 7f5b17800640 1 -- start start 2024-04-17T07:22:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.733+0000 7f5b17800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08094c70 0x7f5b0812a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.733+0000 7f5b17800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b08096820 0x7f5b0812a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.733+0000 7f5b17800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0812f5e0 0x7f5b08131aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.733+0000 7f5b17800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b080a07d0 con 0x7f5b08094c70 2024-04-17T07:22:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.733+0000 7f5b17800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5b080a0650 con 0x7f5b08096820 2024-04-17T07:22:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.733+0000 7f5b17800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b080a0950 con 0x7f5b0812f5e0 2024-04-17T07:22:35.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.734+0000 7f5b15800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0812f5e0 0x7f5b08131aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.734+0000 7f5b15800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0812f5e0 0x7f5b08131aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52376/0 (socket says 172.21.15.86:52376) 2024-04-17T07:22:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.734+0000 7f5b15800640 1 -- 172.21.15.86:0/2139757501 learned_addr learned my addr 172.21.15.86:0/2139757501 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.734+0000 7f5b14e00640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08094c70 0x7f5b0812a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.734+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b08096820 0x7f5b0812a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.735+0000 7f5b14e00640 1 -- 172.21.15.86:0/2139757501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0812f5e0 msgr2=0x7f5b08131aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.735+0000 7f5b14e00640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0812f5e0 0x7f5b08131aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.735+0000 7f5b14e00640 1 -- 172.21.15.86:0/2139757501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b08096820 msgr2=0x7f5b0812a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.735+0000 7f5b14e00640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b08096820 0x7f5b0812a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.735+0000 7f5b14e00640 1 -- 172.21.15.86:0/2139757501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b08132060 con 0x7f5b08094c70 2024-04-17T07:22:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.735+0000 7f5b15800640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0812f5e0 0x7f5b08131aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.735+0000 7f5b14e00640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08094c70 0x7f5b0812a090 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400e9c0 tx=0x7f5b0400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:35.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.736+0000 7f5b0d600640 1 -- 172.21.15.86:0/2139757501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400ccc0 con 0x7f5b08094c70 2024-04-17T07:22:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.736+0000 7f5b17800640 1 -- 172.21.15.86:0/2139757501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b08132350 con 0x7f5b08094c70 2024-04-17T07:22:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.736+0000 7f5b17800640 1 -- 172.21.15.86:0/2139757501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b081328c0 con 0x7f5b08094c70 2024-04-17T07:22:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.736+0000 7f5b0d600640 1 -- 172.21.15.86:0/2139757501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b040035f0 con 0x7f5b08094c70 2024-04-17T07:22:35.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.736+0000 7f5b0d600640 1 -- 172.21.15.86:0/2139757501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400ce60 con 0x7f5b08094c70 2024-04-17T07:22:35.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.743+0000 7f5b1a61a740 1 Processor -- start 2024-04-17T07:22:35.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.744+0000 7f5b1a61a740 1 -- start start 2024-04-17T07:22:35.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.744+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d4370 0x556e332d4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.744+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.744+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 0x556e332dc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.744+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e3323b120 con 0x556e332d4d60 2024-04-17T07:22:35.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.744+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556e331e1340 con 0x556e332d4370 2024-04-17T07:22:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.744+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556e332370e0 con 0x556e332d9ed0 2024-04-17T07:22:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.745+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 0x556e332dc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.745+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 0x556e332dc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52378/0 (socket says 172.21.15.86:52378) 2024-04-17T07:22:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.745+0000 7f5b0fe00640 1 -- 172.21.15.86:0/3139048664 learned_addr learned my addr 172.21.15.86:0/3139048664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:35.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.745+0000 7f5b0d600640 1 -- 172.21.15.86:0/2139757501 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5b040037a0 con 0x7f5b08094c70 2024-04-17T07:22:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.746+0000 7f5b0fe00640 1 -- 172.21.15.86:0/3139048664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d4370 msgr2=0x556e332d4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.746+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/3139048664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d4370 0x556e332d4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.746+0000 7f5b0fe00640 1 -- 172.21.15.86:0/3139048664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 msgr2=0x556e332d9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.746+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/3139048664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.746+0000 7f5b0fe00640 1 -- 172.21.15.86:0/3139048664 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e330177d0 con 0x556e332d9ed0 2024-04-17T07:22:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.747+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/3139048664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 0x556e332dc360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5afc005ab0 tx=0x7f5afc005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7703b468cc65a3ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:35.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.748+0000 7f5b0d600640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5af80a3c30 0x7f5af80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.748+0000 7f5b02000640 1 -- 172.21.15.86:0/3139048664 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc045020 con 0x556e332d9ed0 2024-04-17T07:22:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.748+0000 7f5b02000640 1 -- 172.21.15.86:0/3139048664 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5afc004040 con 0x556e332d9ed0 2024-04-17T07:22:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.748+0000 7f5b02000640 1 -- 172.21.15.86:0/3139048664 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0046c0 con 0x556e332d9ed0 2024-04-17T07:22:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.748+0000 7f5b0d600640 1 -- 172.21.15.86:0/2139757501 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5b040d0bf0 con 0x7f5b08094c70 2024-04-17T07:22:35.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.748+0000 7f5b15800640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5af80a3c30 0x7f5af80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.749+0000 7f5b1a61a740 1 -- 172.21.15.86:0/3139048664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 msgr2=0x556e332dc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:35.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.749+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/3139048664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 0x556e332dc360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5afc005ab0 tx=0x7f5afc005d50 comp rx=0 tx=0).stop 2024-04-17T07:22:35.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.750+0000 7f5b1a61a740 1 -- 172.21.15.86:0/3139048664 shutdown_connections 2024-04-17T07:22:35.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.750+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/3139048664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 0x556e332dc360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.750+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/3139048664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332d9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.750+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/3139048664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d4370 0x556e332d4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.750+0000 7f5b1a61a740 1 -- 172.21.15.86:0/3139048664 >> 172.21.15.86:0/3139048664 conn(0x556e332cf8e0 msgr2=0x556e332cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:35.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.751+0000 7f5b1a61a740 1 -- 172.21.15.86:0/3139048664 shutdown_connections 2024-04-17T07:22:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.752+0000 7f5b1a61a740 1 -- 172.21.15.86:0/3139048664 wait complete. 2024-04-17T07:22:35.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.752+0000 7f5b15800640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5af80a3c30 0x7f5af80a60f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5b100681f0 tx=0x7f5b1007d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.753+0000 7f5b1a61a740 1 -- start start 2024-04-17T07:22:35.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.753+0000 7f5b1a61a740 1 Processor -- start 2024-04-17T07:22:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.754+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 0x556e332de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.754+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332ded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.754+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 0x556e332e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.754+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e332dce20 con 0x556e332d4d60 2024-04-17T07:22:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.754+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556e331f4710 con 0x556e332d9ed0 2024-04-17T07:22:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.754+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556e332dcfa0 con 0x556e332d4370 2024-04-17T07:22:35.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.754+0000 7f5b1a61a740 10 client.0 _ll_register_callbacks cb 0x556e331efaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.755+0000 7f5b15800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 0x556e332e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.755+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.755+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332ded70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45060/0 (socket says 172.21.15.86:45060) 2024-04-17T07:22:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.755+0000 7f5b0fe00640 1 -- 172.21.15.86:0/2416321355 learned_addr learned my addr 172.21.15.86:0/2416321355 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.755+0000 7f5b15800640 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 0x556e332e5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:46778/0 (socket says 172.21.15.86:46778) 2024-04-17T07:22:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.755+0000 7f5b14e00640 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 0x556e332de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.756+0000 7f5b0fe00640 1 -- 172.21.15.86:0/2416321355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 msgr2=0x556e332de830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.756+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 0x556e332de830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.756+0000 7f5b0fe00640 1 -- 172.21.15.86:0/2416321355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 msgr2=0x556e332e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.756+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 0x556e332e5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.756+0000 7f5b0fe00640 1 -- 172.21.15.86:0/2416321355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e32f4d3f0 con 0x556e332d4d60 2024-04-17T07:22:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.756+0000 7f5b15800640 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 0x556e332e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.756+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332ded70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5afc02bc70 tx=0x7f5afc02c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9bd4c640f2879 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.756+0000 7f5af6a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.757+0000 7f5af6a00640 1 -- 172.21.15.86:0/2416321355 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc051020 con 0x556e332d4d60 2024-04-17T07:22:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.757+0000 7f5af6a00640 1 -- 172.21.15.86:0/2416321355 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5afc0037f0 con 0x556e332d4d60 2024-04-17T07:22:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.757+0000 7f5b1a61a740 1 -- 172.21.15.86:0/2416321355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5afc004040 con 0x556e332d4d60 2024-04-17T07:22:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.758+0000 7f5af6a00640 1 -- 172.21.15.86:0/2416321355 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5afc045070 con 0x556e332d4d60 2024-04-17T07:22:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.758+0000 7f5b1a61a740 10 client.17508 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.758+0000 7f5b1a61a740 1 -- 172.21.15.86:0/2416321355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556e32f0d0f0 con 0x556e332d4d60 2024-04-17T07:22:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.758+0000 7f5af6a00640 1 -- 172.21.15.86:0/2416321355 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5afc004040 con 0x556e332d4d60 2024-04-17T07:22:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.759+0000 7f5af6a00640 1 -- 172.21.15.86:0/2416321355 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5afc004dc0 con 0x556e332d4d60 2024-04-17T07:22:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.759+0000 7f5b1a61a740 10 client.17508 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.759+0000 7f5b1a61a740 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.759+0000 7f5b1a61a740 20 client.17508 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.759+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556e332ea800 0x556e332ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.759+0000 7f5b1a61a740 4 client.17508 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.759+0000 7f5b1a61a740 1 -- 172.21.15.86:0/2416321355 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e33248b40 con 0x556e332ea800 2024-04-17T07:22:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.760+0000 7f5b14e00640 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556e332ea800 0x556e332ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.762+0000 7f5b14e00640 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556e332ea800 0x556e332ecc20 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c726dc0d73d9e12c server_cookie=835348d4945a1d2d in_seq=0 out_seq=0 2024-04-17T07:22:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.762+0000 7f5af6a00640 10 client.17508 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:35.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.765+0000 7f5af6a00640 1 -- 172.21.15.86:0/2416321355 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e33248b40 con 0x556e332ea800 2024-04-17T07:22:35.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.765+0000 7f5af6a00640 10 client.17508 handle_command_reply: tid=0 2024-04-17T07:22:35.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.766+0000 7f5b1a61a740 1 client.17508 shutdown 2024-04-17T07:22:35.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.766+0000 7f5b1a61a740 1 -- 172.21.15.86:0/2416321355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 msgr2=0x556e332ded70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:35.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.766+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332ded70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5afc02bc70 tx=0x7f5afc02c960 comp rx=0 tx=0).stop 2024-04-17T07:22:35.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.766+0000 7f5b1a61a740 1 -- 172.21.15.86:0/2416321355 shutdown_connections 2024-04-17T07:22:35.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.766+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556e332ea800 0x556e332ecc20 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.766+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 0x556e332e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.766+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332ded70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.766+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/2416321355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 0x556e332de830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.766+0000 7f5af6a00640 0 client.17508 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:35.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.766+0000 7f5b1a61a740 1 -- 172.21.15.86:0/2416321355 >> 172.21.15.86:0/2416321355 conn(0x556e332cf8e0 msgr2=0x556e332ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:35.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.767+0000 7f5b1a61a740 1 -- 172.21.15.86:0/2416321355 shutdown_connections 2024-04-17T07:22:35.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.769+0000 7f5b1a61a740 1 -- 172.21.15.86:0/2416321355 wait complete. 2024-04-17T07:22:35.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.769+0000 7f5b1a61a740 20 client.17508 trim_cache size 0 max 16384 2024-04-17T07:22:35.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.814+0000 7f5b1a61a740 1 Processor -- start 2024-04-17T07:22:35.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.815+0000 7f5b1a61a740 1 -- start start 2024-04-17T07:22:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.816+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d4370 0x556e332de230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.816+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.816+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 0x556e332ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.816+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5af000fc40 con 0x556e332d4d60 2024-04-17T07:22:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.816+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0011780 con 0x556e332d4370 2024-04-17T07:22:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.816+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0010310 con 0x556e332d9ed0 2024-04-17T07:22:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.816+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.816+0000 7f5b14e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d4370 0x556e332de230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.817+0000 7f5b15800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 0x556e332ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.817+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332e5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45070/0 (socket says 172.21.15.86:45070) 2024-04-17T07:22:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.817+0000 7f5b0fe00640 1 -- 172.21.15.86:0/940345155 learned_addr learned my addr 172.21.15.86:0/940345155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.818+0000 7f5b0fe00640 1 -- 172.21.15.86:0/940345155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 msgr2=0x556e332ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.818+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/940345155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 0x556e332ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.818+0000 7f5b0fe00640 1 -- 172.21.15.86:0/940345155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d4370 msgr2=0x556e332de230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.818+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/940345155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d4370 0x556e332de230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.818+0000 7f5b0fe00640 1 -- 172.21.15.86:0/940345155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e332d6af0 con 0x556e332d4d60 2024-04-17T07:22:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.818+0000 7f5b14e00640 1 --2- 172.21.15.86:0/940345155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d4370 0x556e332de230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.819+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/940345155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332e5cd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009900 tx=0x7f5afc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14929c84f8c4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.819+0000 7f5b01600640 1 -- 172.21.15.86:0/940345155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc04e040 con 0x556e332d4d60 2024-04-17T07:22:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.819+0000 7f5b01600640 1 -- 172.21.15.86:0/940345155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5afc02c510 con 0x556e332d4d60 2024-04-17T07:22:35.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.819+0000 7f5b01600640 1 -- 172.21.15.86:0/940345155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc045070 con 0x556e332d4d60 2024-04-17T07:22:35.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.820+0000 7f5b1a61a740 1 -- 172.21.15.86:0/940345155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 msgr2=0x556e332e5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:35.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.820+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/940345155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332e5cd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009900 tx=0x7f5afc005fd0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.821+0000 7f5b1a61a740 1 -- 172.21.15.86:0/940345155 shutdown_connections 2024-04-17T07:22:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.821+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/940345155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d9ed0 0x556e332ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.821+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/940345155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332d4d60 0x556e332e5cd0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.821+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/940345155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d4370 0x556e332de230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.821+0000 7f5b1a61a740 1 -- 172.21.15.86:0/940345155 >> 172.21.15.86:0/940345155 conn(0x556e332cf8e0 msgr2=0x556e332ddb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.822+0000 7f5b1a61a740 1 -- 172.21.15.86:0/940345155 shutdown_connections 2024-04-17T07:22:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.822+0000 7f5b1a61a740 1 -- 172.21.15.86:0/940345155 wait complete. 2024-04-17T07:22:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.822+0000 7f5b1a61a740 1 -- start start 2024-04-17T07:22:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.823+0000 7f5b1a61a740 1 Processor -- start 2024-04-17T07:22:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.823+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 0x556e332e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.823+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 0x556e332e4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.823+0000 7f5b1a61a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332e5280 0x556e332e5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.823+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5af00110b0 con 0x556e332e5280 2024-04-17T07:22:35.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.823+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5af00109e0 con 0x556e332d9ed0 2024-04-17T07:22:35.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.823+0000 7f5b1a61a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5af000dbd0 con 0x556e332d4370 2024-04-17T07:22:35.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.823+0000 7f5b1a61a740 10 client.0 _ll_register_callbacks cb 0x7f5afc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:35.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.824+0000 7f5b14e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 0x556e332e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.824+0000 7f5b14e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 0x556e332e3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:52402/0 (socket says 172.21.15.86:52402) 2024-04-17T07:22:35.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.824+0000 7f5b14e00640 1 -- 172.21.15.86:0/1039883802 learned_addr learned my addr 172.21.15.86:0/1039883802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.824+0000 7f5b15800640 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332e5280 0x556e332e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.824+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 0x556e332e4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.825+0000 7f5b14e00640 1 -- 172.21.15.86:0/1039883802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 msgr2=0x556e332e4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.825+0000 7f5b14e00640 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 0x556e332e4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.825+0000 7f5b14e00640 1 -- 172.21.15.86:0/1039883802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332e5280 msgr2=0x556e332e5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.825+0000 7f5b14e00640 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332e5280 0x556e332e5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.825+0000 7f5b14e00640 1 -- 172.21.15.86:0/1039883802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e332e5c40 con 0x556e332d4370 2024-04-17T07:22:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.826+0000 7f5b14e00640 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 0x556e332e3d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b0409c7f0 tx=0x7f5b04003d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87cdd9cb97f66e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.826+0000 7f5af6a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:22:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.826+0000 7f5af6a00640 1 -- 172.21.15.86:0/1039883802 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b040a0ea0 con 0x556e332d4370 2024-04-17T07:22:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.826+0000 7f5af6a00640 1 -- 172.21.15.86:0/1039883802 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5b040ce420 con 0x556e332d4370 2024-04-17T07:22:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.827+0000 7f5b1a61a740 1 -- 172.21.15.86:0/1039883802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e332429b0 con 0x556e332d4370 2024-04-17T07:22:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.827+0000 7f5af6a00640 1 -- 172.21.15.86:0/1039883802 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b040a5760 con 0x556e332d4370 2024-04-17T07:22:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.827+0000 7f5b1a61a740 10 client.26503 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.827+0000 7f5b1a61a740 1 -- 172.21.15.86:0/1039883802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556e333519a0 con 0x556e332d4370 2024-04-17T07:22:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.828+0000 7f5af6a00640 1 -- 172.21.15.86:0/1039883802 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556e332429b0 con 0x556e332d4370 2024-04-17T07:22:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.828+0000 7f5af6a00640 1 -- 172.21.15.86:0/1039883802 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5b0409b9b0 con 0x556e332d4370 2024-04-17T07:22:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.828+0000 7f5b1a61a740 10 client.26503 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.828+0000 7f5b1a61a740 10 client.26503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.828+0000 7f5b1a61a740 20 client.26503 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.828+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556e33352280 0x556e333546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.828+0000 7f5b1a61a740 4 client.26503 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:22:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.828+0000 7f5b1a61a740 1 -- 172.21.15.86:0/1039883802 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e331be420 con 0x556e33352280 2024-04-17T07:22:35.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.829+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556e33352280 0x556e333546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.830+0000 7f5b0fe00640 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556e33352280 0x556e333546a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c74d86be485f89d8 server_cookie=90aca6872b2197f7 in_seq=0 out_seq=0 2024-04-17T07:22:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.830+0000 7f5af6a00640 10 client.26503 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.831+0000 7f5af6a00640 1 -- 172.21.15.86:0/1039883802 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556e331be420 con 0x556e33352280 2024-04-17T07:22:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.831+0000 7f5af6a00640 10 client.26503 handle_command_reply: tid=0 2024-04-17T07:22:35.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.831+0000 7f5b1a61a740 1 client.26503 shutdown 2024-04-17T07:22:35.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.832+0000 7f5b1a61a740 1 -- 172.21.15.86:0/1039883802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 msgr2=0x556e332e3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.832+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 0x556e332e3d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b0409c7f0 tx=0x7f5b04003d00 comp rx=0 tx=0).stop 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.832+0000 7f5b1a61a740 1 -- 172.21.15.86:0/1039883802 shutdown_connections 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.832+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556e33352280 0x556e333546a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.832+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e332e5280 0x556e332e5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.832+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e332d9ed0 0x556e332e4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.832+0000 7f5b1a61a740 1 --2- 172.21.15.86:0/1039883802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e332d4370 0x556e332e3d30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.832+0000 7f5b1a61a740 1 -- 172.21.15.86:0/1039883802 >> 172.21.15.86:0/1039883802 conn(0x556e332cf8e0 msgr2=0x556e332d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.832+0000 7f5af6a00640 0 client.26503 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.832+0000 7f5b1a61a740 1 -- 172.21.15.86:0/1039883802 shutdown_connections 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.833+0000 7f5b1a61a740 1 -- 172.21.15.86:0/1039883802 wait complete. 2024-04-17T07:22:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.833+0000 7f5b1a61a740 20 client.26503 trim_cache size 0 max 16384 2024-04-17T07:22:35.834 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:22:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:22:35.834 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:22:35.834 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:22:35.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.834+0000 7f5b17800640 1 -- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5af80a3c30 msgr2=0x7f5af80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.834+0000 7f5b17800640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5af80a3c30 0x7f5af80a60f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5b100681f0 tx=0x7f5b1007d040 comp rx=0 tx=0).stop 2024-04-17T07:22:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.834+0000 7f5b17800640 1 -- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08094c70 msgr2=0x7f5b0812a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.834+0000 7f5b17800640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08094c70 0x7f5b0812a090 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400e9c0 tx=0x7f5b0400ee80 comp rx=0 tx=0).stop 2024-04-17T07:22:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.834+0000 7f5b17800640 1 -- 172.21.15.86:0/2139757501 shutdown_connections 2024-04-17T07:22:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.834+0000 7f5b17800640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5af80a3c30 0x7f5af80a60f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.834+0000 7f5b17800640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0812f5e0 0x7f5b08131aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.834+0000 7f5b17800640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b08096820 0x7f5b0812a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.834+0000 7f5b17800640 1 --2- 172.21.15.86:0/2139757501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b08094c70 0x7f5b0812a090 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:35.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.834+0000 7f5b17800640 1 -- 172.21.15.86:0/2139757501 >> 172.21.15.86:0/2139757501 conn(0x7f5b080901c0 msgr2=0x7f5b08091a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.835+0000 7f5b17800640 1 -- 172.21.15.86:0/2139757501 shutdown_connections 2024-04-17T07:22:35.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:35.835+0000 7f5b17800640 1 -- 172.21.15.86:0/2139757501 wait complete. 2024-04-17T07:22:35.862 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:22:35.862 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:22:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.165+0000 7fe66a800640 1 -- 172.21.15.86:0/911572916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6640749c0 msgr2=0x7fe664074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.165+0000 7fe66a800640 1 --2- 172.21.15.86:0/911572916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6640749c0 0x7fe664074e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe654009930 tx=0x7fe65402bc60 comp rx=0 tx=0).stop 2024-04-17T07:22:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.165+0000 7fe66a800640 1 -- 172.21.15.86:0/911572916 shutdown_connections 2024-04-17T07:22:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.165+0000 7fe66a800640 1 --2- 172.21.15.86:0/911572916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe66410e5a0 0x7fe664110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.165+0000 7fe66a800640 1 --2- 172.21.15.86:0/911572916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6640749c0 0x7fe664074e40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.165+0000 7fe66a800640 1 --2- 172.21.15.86:0/911572916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe664075fa0 0x7fe6640743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.165+0000 7fe66a800640 1 -- 172.21.15.86:0/911572916 >> 172.21.15.86:0/911572916 conn(0x7fe6640fd540 msgr2=0x7fe6640ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.165+0000 7fe66a800640 1 -- 172.21.15.86:0/911572916 shutdown_connections 2024-04-17T07:22:36.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.165+0000 7fe66a800640 1 -- 172.21.15.86:0/911572916 wait complete. 2024-04-17T07:22:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.167+0000 7fe66a800640 1 Processor -- start 2024-04-17T07:22:36.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.167+0000 7fe66a800640 1 -- start start 2024-04-17T07:22:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.168+0000 7fe66a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6640749c0 0x7fe66419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.168+0000 7fe66a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe664075fa0 0x7fe66419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.168+0000 7fe66a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66410e5a0 0x7fe6641a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.168+0000 7fe66a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe664114120 con 0x7fe66410e5a0 2024-04-17T07:22:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.168+0000 7fe66a800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe664113fa0 con 0x7fe664075fa0 2024-04-17T07:22:36.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.168+0000 7fe66a800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6641142a0 con 0x7fe6640749c0 2024-04-17T07:22:36.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.169+0000 7fe662a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe664075fa0 0x7fe66419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:36.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.169+0000 7fe662a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe664075fa0 0x7fe66419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:46826/0 (socket says 172.21.15.86:46826) 2024-04-17T07:22:36.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.169+0000 7fe662a00640 1 -- 172.21.15.86:0/3407600030 learned_addr learned my addr 172.21.15.86:0/3407600030 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:36.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.169+0000 7fe663400640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6640749c0 0x7fe66419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:36.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.169+0000 7fe663e00640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66410e5a0 0x7fe6641a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:36.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.169+0000 7fe662a00640 1 -- 172.21.15.86:0/3407600030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6640749c0 msgr2=0x7fe66419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.169+0000 7fe662a00640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6640749c0 0x7fe66419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.169+0000 7fe662a00640 1 -- 172.21.15.86:0/3407600030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66410e5a0 msgr2=0x7fe6641a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.169+0000 7fe662a00640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66410e5a0 0x7fe6641a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.170+0000 7fe662a00640 1 -- 172.21.15.86:0/3407600030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6641a6350 con 0x7fe664075fa0 2024-04-17T07:22:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.170+0000 7fe663400640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6640749c0 0x7fe66419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:36.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.170+0000 7fe662a00640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe664075fa0 0x7fe66419ec70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe654004270 tx=0x7fe6540042a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:36.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.174+0000 7fe65be00640 1 -- 172.21.15.86:0/3407600030 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654043020 con 0x7fe664075fa0 2024-04-17T07:22:36.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.174+0000 7fe65be00640 1 -- 172.21.15.86:0/3407600030 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6540044c0 con 0x7fe664075fa0 2024-04-17T07:22:36.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.174+0000 7fe65be00640 1 -- 172.21.15.86:0/3407600030 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654003db0 con 0x7fe664075fa0 2024-04-17T07:22:36.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.174+0000 7fe66a800640 1 -- 172.21.15.86:0/3407600030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6641a65e0 con 0x7fe664075fa0 2024-04-17T07:22:36.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.175+0000 7fe66a800640 1 -- 172.21.15.86:0/3407600030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6641a6ae0 con 0x7fe664075fa0 2024-04-17T07:22:36.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.177+0000 7fe66a800640 1 -- 172.21.15.86:0/3407600030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe664104770 con 0x7fe664075fa0 2024-04-17T07:22:36.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.179+0000 7fe65be00640 1 -- 172.21.15.86:0/3407600030 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fe654046020 con 0x7fe664075fa0 2024-04-17T07:22:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.181+0000 7fe65be00640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c80 0x7fe6480a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.181+0000 7fe65be00640 1 -- 172.21.15.86:0/3407600030 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe6540f3340 con 0x7fe664075fa0 2024-04-17T07:22:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.182+0000 7fe663400640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c80 0x7fe6480a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:36.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.183+0000 7fe663400640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c80 0x7fe6480a6140 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe65000baa0 tx=0x7fe650005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:36.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.192+0000 7fe65be00640 1 -- 172.21.15.86:0/3407600030 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe65404d020 con 0x7fe664075fa0 2024-04-17T07:22:36.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.662+0000 7fe66a800640 1 -- 172.21.15.86:0/3407600030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe66402d820 con 0x7fe664075fa0 2024-04-17T07:22:36.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.665+0000 7fe65be00640 1 -- 172.21.15.86:0/3407600030 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fe6540bf940 con 0x7fe664075fa0 2024-04-17T07:22:36.667 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:22:36.667 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:22:36.667 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:22:36.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.669+0000 7fe66a800640 1 -- 172.21.15.86:0/3407600030 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c80 msgr2=0x7fe6480a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:36.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.670+0000 7fe66a800640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c80 0x7fe6480a6140 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe65000baa0 tx=0x7fe650005d90 comp rx=0 tx=0).stop 2024-04-17T07:22:36.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.670+0000 7fe66a800640 1 -- 172.21.15.86:0/3407600030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe664075fa0 msgr2=0x7fe66419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:36.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.670+0000 7fe66a800640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe664075fa0 0x7fe66419ec70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe654004270 tx=0x7fe6540042a0 comp rx=0 tx=0).stop 2024-04-17T07:22:36.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.670+0000 7fe66a800640 1 -- 172.21.15.86:0/3407600030 shutdown_connections 2024-04-17T07:22:36.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.670+0000 7fe66a800640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe6480a3c80 0x7fe6480a6140 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:36.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.670+0000 7fe66a800640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66410e5a0 0x7fe6641a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:36.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.670+0000 7fe66a800640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe664075fa0 0x7fe66419ec70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:36.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.670+0000 7fe66a800640 1 --2- 172.21.15.86:0/3407600030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6640749c0 0x7fe66419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:36.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.670+0000 7fe66a800640 1 -- 172.21.15.86:0/3407600030 >> 172.21.15.86:0/3407600030 conn(0x7fe6640fd540 msgr2=0x7fe664101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:36.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.671+0000 7fe66a800640 1 -- 172.21.15.86:0/3407600030 shutdown_connections 2024-04-17T07:22:36.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:36.671+0000 7fe66a800640 1 -- 172.21.15.86:0/3407600030 wait complete. 2024-04-17T07:22:36.704 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:22:37.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.010+0000 7fa49fa00640 1 -- 172.21.15.86:0/1273197020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa498077b60 msgr2=0x7fa498077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.011+0000 7fa49fa00640 1 --2- 172.21.15.86:0/1273197020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa498077b60 0x7fa498077fc0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa484009900 tx=0x7fa48402bc60 comp rx=0 tx=0).stop 2024-04-17T07:22:37.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.014+0000 7fa49fa00640 1 -- 172.21.15.86:0/1273197020 shutdown_connections 2024-04-17T07:22:37.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.014+0000 7fa49fa00640 1 --2- 172.21.15.86:0/1273197020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa498078500 0x7fa498078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.014+0000 7fa49fa00640 1 --2- 172.21.15.86:0/1273197020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa498077b60 0x7fa498077fc0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.014+0000 7fa49fa00640 1 --2- 172.21.15.86:0/1273197020 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa498076920 0x7fa498076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.015+0000 7fa49fa00640 1 -- 172.21.15.86:0/1273197020 >> 172.21.15.86:0/1273197020 conn(0x7fa4980ff750 msgr2=0x7fa498101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.018+0000 7fa49fa00640 1 -- 172.21.15.86:0/1273197020 shutdown_connections 2024-04-17T07:22:37.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.018+0000 7fa49fa00640 1 -- 172.21.15.86:0/1273197020 wait complete. 2024-04-17T07:22:37.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.019+0000 7fa49fa00640 1 Processor -- start 2024-04-17T07:22:37.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.019+0000 7fa49fa00640 1 -- start start 2024-04-17T07:22:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.020+0000 7fa49fa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa498076920 0x7fa49819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.020+0000 7fa49fa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa498077b60 0x7fa49819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.020+0000 7fa49fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa498078500 0x7fa4981a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.021+0000 7fa49d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa498076920 0x7fa49819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.021+0000 7fa49fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa498114100 con 0x7fa498078500 2024-04-17T07:22:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.021+0000 7fa49fa00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa498113f80 con 0x7fa498077b60 2024-04-17T07:22:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.021+0000 7fa49fa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa498114280 con 0x7fa498076920 2024-04-17T07:22:37.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.021+0000 7fa49da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa498078500 0x7fa4981a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.021+0000 7fa497e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa498077b60 0x7fa49819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.021+0000 7fa497e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa498077b60 0x7fa49819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53414/0 (socket says 172.21.15.86:53414) 2024-04-17T07:22:37.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.021+0000 7fa497e00640 1 -- 172.21.15.86:0/7828707 learned_addr learned my addr 172.21.15.86:0/7828707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.022+0000 7fa497e00640 1 -- 172.21.15.86:0/7828707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa498076920 msgr2=0x7fa49819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.022+0000 7fa497e00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa498076920 0x7fa49819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.022+0000 7fa497e00640 1 -- 172.21.15.86:0/7828707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa498078500 msgr2=0x7fa4981a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.022+0000 7fa497e00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa498078500 0x7fa4981a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.022+0000 7fa497e00640 1 -- 172.21.15.86:0/7828707 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4981a6410 con 0x7fa498077b60 2024-04-17T07:22:37.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.022+0000 7fa49da00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa498078500 0x7fa4981a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:37.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.024+0000 7fa497e00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa498077b60 0x7fa49819ed30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa4840058c0 tx=0x7fa48402df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.025+0000 7fa495600640 1 -- 172.21.15.86:0/7828707 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484043020 con 0x7fa498077b60 2024-04-17T07:22:37.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.025+0000 7fa495600640 1 -- 172.21.15.86:0/7828707 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4840043d0 con 0x7fa498077b60 2024-04-17T07:22:37.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.025+0000 7fa495600640 1 -- 172.21.15.86:0/7828707 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484003b90 con 0x7fa498077b60 2024-04-17T07:22:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.026+0000 7fa49fa00640 1 -- 172.21.15.86:0/7828707 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4981a66a0 con 0x7fa498077b60 2024-04-17T07:22:37.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.026+0000 7fa49fa00640 1 -- 172.21.15.86:0/7828707 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4981a6ba0 con 0x7fa498077b60 2024-04-17T07:22:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.031+0000 7fa4a2811740 1 Processor -- start 2024-04-17T07:22:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.032+0000 7fa4a2811740 1 -- start start 2024-04-17T07:22:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.032+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.032+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b0990 0x5583e95b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.032+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95b5b00 0x5583e95b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.032+0000 7fa4a2811740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5583e9516020 con 0x5583e95affa0 2024-04-17T07:22:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.032+0000 7fa4a2811740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5583e94bc460 con 0x5583e95b0990 2024-04-17T07:22:37.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.032+0000 7fa4a2811740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5583e9511fe0 con 0x5583e95b5b00 2024-04-17T07:22:37.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.033+0000 7fa49da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.033+0000 7fa49da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56566/0 (socket says 172.21.15.86:56566) 2024-04-17T07:22:37.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.033+0000 7fa49da00640 1 -- 172.21.15.86:0/2041056689 learned_addr learned my addr 172.21.15.86:0/2041056689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.033+0000 7fa49d000640 1 --2- 172.21.15.86:0/2041056689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b0990 0x5583e95b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.033+0000 7fa49da00640 1 -- 172.21.15.86:0/2041056689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95b5b00 msgr2=0x5583e95b7f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:22:37.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.033+0000 7fa49da00640 1 --2- 172.21.15.86:0/2041056689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95b5b00 0x5583e95b7f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.033+0000 7fa49da00640 1 -- 172.21.15.86:0/2041056689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b0990 msgr2=0x5583e95b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.033+0000 7fa49da00640 1 --2- 172.21.15.86:0/2041056689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b0990 0x5583e95b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.033+0000 7fa49da00640 1 -- 172.21.15.86:0/2041056689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e94c4040 con 0x5583e95affa0 2024-04-17T07:22:37.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.034+0000 7fa49da00640 1 --2- 172.21.15.86:0/2041056689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95b03c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa490009fd0 tx=0x7fa4900108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffedb9e8a0de960 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.034+0000 7fa48a000640 1 -- 172.21.15.86:0/2041056689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490011760 con 0x5583e95affa0 2024-04-17T07:22:37.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.034+0000 7fa48a000640 1 -- 172.21.15.86:0/2041056689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa490011db0 con 0x5583e95affa0 2024-04-17T07:22:37.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.034+0000 7fa48a000640 1 -- 172.21.15.86:0/2041056689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49000e3c0 con 0x5583e95affa0 2024-04-17T07:22:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.035+0000 7fa495600640 1 -- 172.21.15.86:0/7828707 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa484046020 con 0x7fa498077b60 2024-04-17T07:22:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.036+0000 7fa4a2811740 1 -- 172.21.15.86:0/2041056689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 msgr2=0x5583e95b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.036+0000 7fa4a2811740 1 --2- 172.21.15.86:0/2041056689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95b03c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa490009fd0 tx=0x7fa4900108e0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.036+0000 7fa4a2811740 1 -- 172.21.15.86:0/2041056689 shutdown_connections 2024-04-17T07:22:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.036+0000 7fa4a2811740 1 --2- 172.21.15.86:0/2041056689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95b5b00 0x5583e95b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.036+0000 7fa4a2811740 1 --2- 172.21.15.86:0/2041056689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b0990 0x5583e95b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.036+0000 7fa4a2811740 1 --2- 172.21.15.86:0/2041056689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95b03c0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.036+0000 7fa4a2811740 1 -- 172.21.15.86:0/2041056689 >> 172.21.15.86:0/2041056689 conn(0x5583e95ab510 msgr2=0x5583e95ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.036+0000 7fa4a2811740 1 -- 172.21.15.86:0/2041056689 shutdown_connections 2024-04-17T07:22:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.036+0000 7fa4a2811740 1 -- 172.21.15.86:0/2041056689 wait complete. 2024-04-17T07:22:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.037+0000 7fa4a2811740 1 -- start start 2024-04-17T07:22:37.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.037+0000 7fa4a2811740 1 Processor -- start 2024-04-17T07:22:37.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.037+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95ba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.037+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95b0990 0x5583e95baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.038+0000 7fa495600640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4800a3bc0 0x7fa4800a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.038+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.038+0000 7fa4a2811740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5583e95b8ab0 con 0x5583e95affa0 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.038+0000 7fa4a2811740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5583e94cf830 con 0x5583e95b5b00 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.038+0000 7fa4a2811740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5583e95b8c30 con 0x5583e95b0990 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.038+0000 7fa4a2811740 10 client.0 _ll_register_callbacks cb 0x5583e949dfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.038+0000 7fa49da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.038+0000 7fa49da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95ba570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56574/0 (socket says 172.21.15.86:56574) 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.038+0000 7fa49da00640 1 -- 172.21.15.86:0/3814130465 learned_addr learned my addr 172.21.15.86:0/3814130465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.038+0000 7fa495600640 1 -- 172.21.15.86:0/7828707 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa4840f25c0 con 0x7fa498077b60 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.039+0000 7fa49da00640 1 -- 172.21.15.86:0/3814130465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95b0990 msgr2=0x5583e95baab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.039+0000 7fa49da00640 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95b0990 0x5583e95baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.039+0000 7fa49da00640 1 -- 172.21.15.86:0/3814130465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 msgr2=0x5583e95c1af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:22:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.039+0000 7fa49da00640 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95c1af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.039+0000 7fa49da00640 1 -- 172.21.15.86:0/3814130465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e94d3920 con 0x5583e95affa0 2024-04-17T07:22:37.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.039+0000 7fa497e00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4800a3bc0 0x7fa4800a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.040+0000 7fa49da00640 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95ba570 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa490009c50 tx=0x7fa49001b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935b958775fff25d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.040+0000 7fa47ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:37.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.040+0000 7fa47ea00640 1 -- 172.21.15.86:0/3814130465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490011560 con 0x5583e95affa0 2024-04-17T07:22:37.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.040+0000 7fa47ea00640 1 -- 172.21.15.86:0/3814130465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa490011700 con 0x5583e95affa0 2024-04-17T07:22:37.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.041+0000 7fa4a2811740 1 -- 172.21.15.86:0/3814130465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa490011db0 con 0x5583e95affa0 2024-04-17T07:22:37.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.041+0000 7fa497e00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4800a3bc0 0x7fa4800a6080 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa484006010 tx=0x7fa48402bc60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.041+0000 7fa47ea00640 1 -- 172.21.15.86:0/3814130465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa490011900 con 0x5583e95affa0 2024-04-17T07:22:37.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.041+0000 7fa4a2811740 10 client.17529 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:37.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.041+0000 7fa4a2811740 1 -- 172.21.15.86:0/3814130465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583e94a17f0 con 0x5583e95affa0 2024-04-17T07:22:37.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.041+0000 7fa47ea00640 1 -- 172.21.15.86:0/3814130465 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa490011db0 con 0x5583e95affa0 2024-04-17T07:22:37.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.042+0000 7fa47ea00640 1 -- 172.21.15.86:0/3814130465 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa490025460 con 0x5583e95affa0 2024-04-17T07:22:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.042+0000 7fa4a2811740 10 client.17529 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.042+0000 7fa4a2811740 10 client.17529 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:22:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.042+0000 7fa4a2811740 20 client.17529 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.042+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5583e95c5e10 0x5583e95c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.042+0000 7fa4a2811740 4 client.17529 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:37.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.042+0000 7fa4a2811740 1 -- 172.21.15.86:0/3814130465 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583e9523a40 con 0x5583e95c5e10 2024-04-17T07:22:37.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.043+0000 7fa49d000640 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5583e95c5e10 0x5583e95c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.045+0000 7fa49d000640 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5583e95c5e10 0x5583e95c8230 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6705620908baa385 server_cookie=92fae3075eb4d8fa in_seq=0 out_seq=0 2024-04-17T07:22:37.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.045+0000 7fa47ea00640 10 client.17529 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:22:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.047+0000 7fa47ea00640 1 -- 172.21.15.86:0/3814130465 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583e9523a40 con 0x5583e95c5e10 2024-04-17T07:22:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.047+0000 7fa47ea00640 10 client.17529 handle_command_reply: tid=0 2024-04-17T07:22:37.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.048+0000 7fa4a2811740 1 client.17529 shutdown 2024-04-17T07:22:37.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa4a2811740 1 -- 172.21.15.86:0/3814130465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 msgr2=0x5583e95ba570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95ba570 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa490009c50 tx=0x7fa49001b660 comp rx=0 tx=0).stop 2024-04-17T07:22:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa4a2811740 1 -- 172.21.15.86:0/3814130465 shutdown_connections 2024-04-17T07:22:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5583e95c5e10 0x5583e95c8230 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95b0990 0x5583e95baab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3814130465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95affa0 0x5583e95ba570 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa4a2811740 1 -- 172.21.15.86:0/3814130465 >> 172.21.15.86:0/3814130465 conn(0x5583e95ab510 msgr2=0x5583e95aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa47ea00640 0 client.17529 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:22:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa4a2811740 1 -- 172.21.15.86:0/3814130465 shutdown_connections 2024-04-17T07:22:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.049+0000 7fa4a2811740 1 -- 172.21.15.86:0/3814130465 wait complete. 2024-04-17T07:22:37.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.050+0000 7fa4a2811740 20 client.17529 trim_cache size 0 max 16384 2024-04-17T07:22:37.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.081+0000 7fa4a2811740 1 Processor -- start 2024-04-17T07:22:37.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.081+0000 7fa4a2811740 1 -- start start 2024-04-17T07:22:37.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.082+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 0x5583e95ba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.082+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 0x5583e95c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.082+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95ba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.082+0000 7fa4a2811740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b1130 con 0x5583e95b0990 2024-04-17T07:22:37.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.082+0000 7fa4a2811740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b2c70 con 0x5583e95b5b00 2024-04-17T07:22:37.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.082+0000 7fa4a2811740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b1800 con 0x5583e95affa0 2024-04-17T07:22:37.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.083+0000 7fa49da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95ba6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.083+0000 7fa49da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95ba6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53440/0 (socket says 172.21.15.86:53440) 2024-04-17T07:22:37.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.083+0000 7fa49da00640 1 -- 172.21.15.86:0/1263919455 learned_addr learned my addr 172.21.15.86:0/1263919455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.083+0000 7fa49da00640 1 --2- 172.21.15.86:0/1263919455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 0x5583e95ba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.083+0000 7fa49d000640 1 --2- 172.21.15.86:0/1263919455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 0x5583e95c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.083+0000 7fa49da00640 1 -- 172.21.15.86:0/1263919455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 msgr2=0x5583e95ba180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.083+0000 7fa49da00640 1 --2- 172.21.15.86:0/1263919455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 0x5583e95ba180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.083+0000 7fa49da00640 1 -- 172.21.15.86:0/1263919455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 msgr2=0x5583e95c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.083+0000 7fa49da00640 1 --2- 172.21.15.86:0/1263919455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 0x5583e95c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.083+0000 7fa49da00640 1 -- 172.21.15.86:0/1263919455 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e92283f0 con 0x5583e95b5b00 2024-04-17T07:22:37.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.084+0000 7fa49da00640 1 --2- 172.21.15.86:0/1263919455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95ba6c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa4900096f0 tx=0x7fa490010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1a7b8ae0e089c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.084+0000 7fa489600640 1 -- 172.21.15.86:0/1263919455 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490025a10 con 0x5583e95b5b00 2024-04-17T07:22:37.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.084+0000 7fa489600640 1 -- 172.21.15.86:0/1263919455 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa49002a070 con 0x5583e95b5b00 2024-04-17T07:22:37.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.086+0000 7fa489600640 1 -- 172.21.15.86:0/1263919455 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49001bc90 con 0x5583e95b5b00 2024-04-17T07:22:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.087+0000 7fa4a2811740 1 -- 172.21.15.86:0/1263919455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 msgr2=0x5583e95ba6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.087+0000 7fa4a2811740 1 --2- 172.21.15.86:0/1263919455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95ba6c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa4900096f0 tx=0x7fa490010f90 comp rx=0 tx=0).stop 2024-04-17T07:22:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.087+0000 7fa4a2811740 1 -- 172.21.15.86:0/1263919455 shutdown_connections 2024-04-17T07:22:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.087+0000 7fa4a2811740 1 --2- 172.21.15.86:0/1263919455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95ba6c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.087+0000 7fa4a2811740 1 --2- 172.21.15.86:0/1263919455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 0x5583e95c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.087+0000 7fa4a2811740 1 --2- 172.21.15.86:0/1263919455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 0x5583e95ba180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.087+0000 7fa4a2811740 1 -- 172.21.15.86:0/1263919455 >> 172.21.15.86:0/1263919455 conn(0x5583e95ab510 msgr2=0x5583e95b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.088+0000 7fa4a2811740 1 -- 172.21.15.86:0/1263919455 shutdown_connections 2024-04-17T07:22:37.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.088+0000 7fa4a2811740 1 -- 172.21.15.86:0/1263919455 wait complete. 2024-04-17T07:22:37.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.089+0000 7fa4a2811740 1 -- start start 2024-04-17T07:22:37.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.089+0000 7fa4a2811740 1 Processor -- start 2024-04-17T07:22:37.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.090+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 0x5583e95c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.090+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 0x5583e95be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.090+0000 7fa4a2811740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.090+0000 7fa4a2811740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b25a0 con 0x5583e95b0990 2024-04-17T07:22:37.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.090+0000 7fa4a2811740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b1ed0 con 0x5583e95b5b00 2024-04-17T07:22:37.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.090+0000 7fa4a2811740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981af100 con 0x5583e95affa0 2024-04-17T07:22:37.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.090+0000 7fa4a2811740 10 client.0 _ll_register_callbacks cb 0x7fa490004db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:37.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.091+0000 7fa49da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.091+0000 7fa49d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 0x5583e95be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.091+0000 7fa49da00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 0x5583e95c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.091+0000 7fa49da00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 0x5583e95c18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46308/0 (socket says 172.21.15.86:46308) 2024-04-17T07:22:37.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.091+0000 7fa49da00640 1 -- 172.21.15.86:0/3765704545 learned_addr learned my addr 172.21.15.86:0/3765704545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.092+0000 7fa49d000640 1 -- 172.21.15.86:0/3765704545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 msgr2=0x5583e95c18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.092+0000 7fa49d000640 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 0x5583e95c18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.093+0000 7fa49d000640 1 -- 172.21.15.86:0/3765704545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 msgr2=0x5583e95be880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.093+0000 7fa49d000640 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95be880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.093+0000 7fa49d000640 1 -- 172.21.15.86:0/3765704545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e92f27d0 con 0x5583e95b0990 2024-04-17T07:22:37.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.093+0000 7fa49da00640 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 0x5583e95c18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:37.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.094+0000 7fa49d000640 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 0x5583e95be340 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x5583e95c8a70 tx=0x7fa48c00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5684a152e10dfed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.095+0000 7fa47ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:37.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.095+0000 7fa47ea00640 1 -- 172.21.15.86:0/3765704545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c01b890 con 0x5583e95b0990 2024-04-17T07:22:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.095+0000 7fa4a2811740 1 -- 172.21.15.86:0/3765704545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa49002a070 con 0x5583e95b0990 2024-04-17T07:22:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.096+0000 7fa47ea00640 1 -- 172.21.15.86:0/3765704545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa48c0035f0 con 0x5583e95b0990 2024-04-17T07:22:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.096+0000 7fa47ea00640 1 -- 172.21.15.86:0/3765704545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa48c02c050 con 0x5583e95b0990 2024-04-17T07:22:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.096+0000 7fa4a2811740 10 client.17535 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:37.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.096+0000 7fa4a2811740 1 -- 172.21.15.86:0/3765704545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583e962b470 con 0x5583e95b0990 2024-04-17T07:22:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.096+0000 7fa47ea00640 1 -- 172.21.15.86:0/3765704545 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa49002a070 con 0x5583e95b0990 2024-04-17T07:22:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.097+0000 7fa47ea00640 1 -- 172.21.15.86:0/3765704545 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa48c012750 con 0x5583e95b0990 2024-04-17T07:22:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.097+0000 7fa4a2811740 10 client.17535 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.097+0000 7fa4a2811740 10 client.17535 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:22:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.097+0000 7fa4a2811740 20 client.17535 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.097+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5583e962bdd0 0x5583e962e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.097+0000 7fa4a2811740 4 client.17535 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:22:37.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.097+0000 7fa4a2811740 1 -- 172.21.15.86:0/3765704545 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583e951d8b0 con 0x5583e962bdd0 2024-04-17T07:22:37.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.097+0000 7fa49da00640 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5583e962bdd0 0x5583e962e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.099+0000 7fa49da00640 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5583e962bdd0 0x5583e962e1f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1356dccb8a184ad0 server_cookie=321ebc6c374ffbe in_seq=0 out_seq=0 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.099+0000 7fa47ea00640 10 client.17535 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa47ea00640 1 -- 172.21.15.86:0/3765704545 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583e951d8b0 con 0x5583e962bdd0 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa47ea00640 10 client.17535 handle_command_reply: tid=0 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa4a2811740 1 client.17535 shutdown 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa4a2811740 1 -- 172.21.15.86:0/3765704545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 msgr2=0x5583e95be340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 0x5583e95be340 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x5583e95c8a70 tx=0x7fa48c00b930 comp rx=0 tx=0).stop 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa4a2811740 1 -- 172.21.15.86:0/3765704545 shutdown_connections 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5583e962bdd0 0x5583e962e1f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e95b5b00 0x5583e95be880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583e95b0990 0x5583e95be340 secure :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x5583e95c8a70 tx=0x7fa48c00b930 comp rx=0 tx=0).stop 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa4a2811740 1 --2- 172.21.15.86:0/3765704545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583e95affa0 0x5583e95c18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.100+0000 7fa4a2811740 1 -- 172.21.15.86:0/3765704545 >> 172.21.15.86:0/3765704545 conn(0x5583e95ab510 msgr2=0x5583e95ba180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.101+0000 7fa47ea00640 0 client.17535 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:22:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.101+0000 7fa4a2811740 1 -- 172.21.15.86:0/3765704545 shutdown_connections 2024-04-17T07:22:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.101+0000 7fa4a2811740 1 -- 172.21.15.86:0/3765704545 wait complete. 2024-04-17T07:22:37.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.102+0000 7fa4a2811740 20 client.17535 trim_cache size 0 max 16384 2024-04-17T07:22:37.103 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:22:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 -- 172.21.15.86:0/7828707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4800a3bc0 msgr2=0x7fa4800a6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4800a3bc0 0x7fa4800a6080 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa484006010 tx=0x7fa48402bc60 comp rx=0 tx=0).stop 2024-04-17T07:22:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 -- 172.21.15.86:0/7828707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa498077b60 msgr2=0x7fa49819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa498077b60 0x7fa49819ed30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa4840058c0 tx=0x7fa48402df70 comp rx=0 tx=0).stop 2024-04-17T07:22:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 -- 172.21.15.86:0/7828707 shutdown_connections 2024-04-17T07:22:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa4800a3bc0 0x7fa4800a6080 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa498078500 0x7fa4981a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa498077b60 0x7fa49819ed30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 --2- 172.21.15.86:0/7828707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa498076920 0x7fa49819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 -- 172.21.15.86:0/7828707 >> 172.21.15.86:0/7828707 conn(0x7fa4980ff750 msgr2=0x7fa498101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 -- 172.21.15.86:0/7828707 shutdown_connections 2024-04-17T07:22:37.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.104+0000 7fa49fa00640 1 -- 172.21.15.86:0/7828707 wait complete. 2024-04-17T07:22:37.131 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:22:37.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.445+0000 7f8e03600640 1 -- 172.21.15.86:0/695345154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc1041c0 msgr2=0x7f8dfc1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.445+0000 7f8e03600640 1 --2- 172.21.15.86:0/695345154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc1041c0 0x7f8dfc1045e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009ca0 tx=0x7f8df402c120 comp rx=0 tx=0).stop 2024-04-17T07:22:37.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.446+0000 7f8e03600640 1 -- 172.21.15.86:0/695345154 shutdown_connections 2024-04-17T07:22:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.446+0000 7f8e03600640 1 --2- 172.21.15.86:0/695345154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc105d70 0x7f8dfc10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.446+0000 7f8e03600640 1 --2- 172.21.15.86:0/695345154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc1053b0 0x7f8dfc105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.446+0000 7f8e03600640 1 --2- 172.21.15.86:0/695345154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc1041c0 0x7f8dfc1045e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.446+0000 7f8e03600640 1 -- 172.21.15.86:0/695345154 >> 172.21.15.86:0/695345154 conn(0x7f8dfc0ff750 msgr2=0x7f8dfc101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.446+0000 7f8e03600640 1 -- 172.21.15.86:0/695345154 shutdown_connections 2024-04-17T07:22:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.446+0000 7f8e03600640 1 -- 172.21.15.86:0/695345154 wait complete. 2024-04-17T07:22:37.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.448+0000 7f8e03600640 1 Processor -- start 2024-04-17T07:22:37.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.448+0000 7f8e03600640 1 -- start start 2024-04-17T07:22:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.449+0000 7f8e03600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc1041c0 0x7f8dfc195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.449+0000 7f8e03600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc1053b0 0x7f8dfc1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.449+0000 7f8e03600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc105d70 0x7f8dfc19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.450+0000 7f8e03600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10fd40 con 0x7f8dfc1041c0 2024-04-17T07:22:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.450+0000 7f8e03600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10fbc0 con 0x7f8dfc105d70 2024-04-17T07:22:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.450+0000 7f8e03600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10fec0 con 0x7f8dfc1053b0 2024-04-17T07:22:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.450+0000 7f8e01600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc105d70 0x7f8dfc19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.451+0000 7f8e01600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc105d70 0x7f8dfc19d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53448/0 (socket says 172.21.15.86:53448) 2024-04-17T07:22:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.451+0000 7f8e00c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc1041c0 0x7f8dfc195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.451+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc1053b0 0x7f8dfc1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.451+0000 7f8e01600640 1 -- 172.21.15.86:0/895172109 learned_addr learned my addr 172.21.15.86:0/895172109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.451+0000 7f8e01600640 1 -- 172.21.15.86:0/895172109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc1053b0 msgr2=0x7f8dfc1963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.451+0000 7f8e01600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc1053b0 0x7f8dfc1963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.451+0000 7f8e01600640 1 -- 172.21.15.86:0/895172109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc1041c0 msgr2=0x7f8dfc195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.452+0000 7f8e01600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc1041c0 0x7f8dfc195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.452+0000 7f8e01600640 1 -- 172.21.15.86:0/895172109 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc1aec40 con 0x7f8dfc105d70 2024-04-17T07:22:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.452+0000 7f8e00c00640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc1041c0 0x7f8dfc195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:37.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.453+0000 7f8e01600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc105d70 0x7f8dfc19d410 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8de400c970 tx=0x7f8de400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.454+0000 7f8df9600640 1 -- 172.21.15.86:0/895172109 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4007bb0 con 0x7f8dfc105d70 2024-04-17T07:22:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.454+0000 7f8df9600640 1 -- 172.21.15.86:0/895172109 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8de4007d50 con 0x7f8dfc105d70 2024-04-17T07:22:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.454+0000 7f8df9600640 1 -- 172.21.15.86:0/895172109 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4018660 con 0x7f8dfc105d70 2024-04-17T07:22:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.454+0000 7f8e03600640 1 -- 172.21.15.86:0/895172109 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dfc1aee10 con 0x7f8dfc105d70 2024-04-17T07:22:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.454+0000 7f8e03600640 1 -- 172.21.15.86:0/895172109 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dfc1af2f0 con 0x7f8dfc105d70 2024-04-17T07:22:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.458+0000 7f8e06391740 1 Processor -- start 2024-04-17T07:22:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.461+0000 7f8e06391740 1 -- start start 2024-04-17T07:22:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.461+0000 7f8df9600640 1 -- 172.21.15.86:0/895172109 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8de400f450 con 0x7f8dfc105d70 2024-04-17T07:22:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.461+0000 7f8e06391740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 0x55cf606273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.461+0000 7f8e06391740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf6062c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.462+0000 7f8e06391740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf6062ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.462+0000 7f8e06391740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf6058d020 con 0x55cf60626fa0 2024-04-17T07:22:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.462+0000 7f8e06391740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cf60533460 con 0x55cf60627990 2024-04-17T07:22:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.462+0000 7f8e06391740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf60588fe0 con 0x55cf6062cb00 2024-04-17T07:22:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.462+0000 7f8e00c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf6062c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.462+0000 7f8e00c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf6062c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53456/0 (socket says 172.21.15.86:53456) 2024-04-17T07:22:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.462+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 0x55cf606273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.462+0000 7f8e00c00640 1 -- 172.21.15.86:0/2212657333 learned_addr learned my addr 172.21.15.86:0/2212657333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.462+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/2212657333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf6062ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.463+0000 7f8dfbe00640 1 -- 172.21.15.86:0/2212657333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 msgr2=0x55cf6062ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.463+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/2212657333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf6062ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.463+0000 7f8dfbe00640 1 -- 172.21.15.86:0/2212657333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 msgr2=0x55cf6062c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.463+0000 7f8df9600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 0x7f8de00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.463+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/2212657333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf6062c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.463+0000 7f8dfbe00640 1 -- 172.21.15.86:0/2212657333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf6053b040 con 0x55cf60626fa0 2024-04-17T07:22:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.463+0000 7f8df9600640 1 -- 172.21.15.86:0/895172109 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8de40d06f0 con 0x7f8dfc105d70 2024-04-17T07:22:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.464+0000 7f8e01600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 0x7f8de00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.464+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/2212657333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 0x55cf606273c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009fa0 tx=0x7f8dec009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd6c9d41d7486 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.465+0000 7f8df1c00640 1 -- 172.21.15.86:0/2212657333 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec013780 con 0x55cf60626fa0 2024-04-17T07:22:37.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.465+0000 7f8e01600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 0x7f8de00a6240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8de409c2f0 tx=0x7f8de40d0940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.465+0000 7f8df1c00640 1 -- 172.21.15.86:0/2212657333 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8dec01d070 con 0x55cf60626fa0 2024-04-17T07:22:37.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.465+0000 7f8df1c00640 1 -- 172.21.15.86:0/2212657333 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec018a30 con 0x55cf60626fa0 2024-04-17T07:22:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.466+0000 7f8e06391740 1 -- 172.21.15.86:0/2212657333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 msgr2=0x55cf606273c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.466+0000 7f8e06391740 1 --2- 172.21.15.86:0/2212657333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 0x55cf606273c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009fa0 tx=0x7f8dec009420 comp rx=0 tx=0).stop 2024-04-17T07:22:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.467+0000 7f8e06391740 1 -- 172.21.15.86:0/2212657333 shutdown_connections 2024-04-17T07:22:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.467+0000 7f8e06391740 1 --2- 172.21.15.86:0/2212657333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf6062ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.467+0000 7f8e06391740 1 --2- 172.21.15.86:0/2212657333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf6062c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.467+0000 7f8e06391740 1 --2- 172.21.15.86:0/2212657333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 0x55cf606273c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.467+0000 7f8e06391740 1 -- 172.21.15.86:0/2212657333 >> 172.21.15.86:0/2212657333 conn(0x55cf60622510 msgr2=0x55cf60622910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.467+0000 7f8e06391740 1 -- 172.21.15.86:0/2212657333 shutdown_connections 2024-04-17T07:22:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.467+0000 7f8e06391740 1 -- 172.21.15.86:0/2212657333 wait complete. 2024-04-17T07:22:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.468+0000 7f8e06391740 1 -- start start 2024-04-17T07:22:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.468+0000 7f8e06391740 1 Processor -- start 2024-04-17T07:22:37.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.469+0000 7f8e06391740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60626fa0 0x55cf606312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.469+0000 7f8e06391740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf606317e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.470+0000 7f8e06391740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf6062cb00 0x55cf60638820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.470+0000 7f8e06391740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cf6062fa50 con 0x55cf6062cb00 2024-04-17T07:22:37.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.470+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60626fa0 0x55cf606312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.470+0000 7f8e00c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf606317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.470+0000 7f8e00c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf606317e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53466/0 (socket says 172.21.15.86:53466) 2024-04-17T07:22:37.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.470+0000 7f8e00c00640 1 -- 172.21.15.86:0/3692449259 learned_addr learned my addr 172.21.15.86:0/3692449259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.471+0000 7f8e06391740 1 -- 172.21.15.86:0/3692449259 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cf60546830 con 0x55cf60627990 2024-04-17T07:22:37.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.471+0000 7f8e06391740 1 -- 172.21.15.86:0/3692449259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf6062fbd0 con 0x55cf60626fa0 2024-04-17T07:22:37.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.471+0000 7f8e06391740 10 client.0 _ll_register_callbacks cb 0x55cf60514fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:37.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.472+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf6062cb00 0x55cf60638820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.472+0000 7f8e00c00640 1 -- 172.21.15.86:0/3692449259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60626fa0 msgr2=0x55cf606312a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.472+0000 7f8e00c00640 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60626fa0 0x55cf606312a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.472+0000 7f8e00c00640 1 -- 172.21.15.86:0/3692449259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf6062cb00 msgr2=0x55cf60638820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.472+0000 7f8e00c00640 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf6062cb00 0x55cf60638820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.472+0000 7f8e00c00640 1 -- 172.21.15.86:0/3692449259 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf6054a920 con 0x55cf60627990 2024-04-17T07:22:37.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.472+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf6062cb00 0x55cf60638820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:37.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.473+0000 7f8e00c00640 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf606317e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8df4037040 tx=0x7f8df4003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=300dee0804d59ae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.473+0000 7f8dea000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:22:37.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.473+0000 7f8dea000640 1 -- 172.21.15.86:0/3692449259 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4044070 con 0x55cf60627990 2024-04-17T07:22:37.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.474+0000 7f8dea000640 1 -- 172.21.15.86:0/3692449259 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8df4004240 con 0x55cf60627990 2024-04-17T07:22:37.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.474+0000 7f8e06391740 1 -- 172.21.15.86:0/3692449259 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dec01d070 con 0x55cf60627990 2024-04-17T07:22:37.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.474+0000 7f8dea000640 1 -- 172.21.15.86:0/3692449259 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8df4039040 con 0x55cf60627990 2024-04-17T07:22:37.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.475+0000 7f8e06391740 10 client.36509 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:37.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.475+0000 7f8e06391740 1 -- 172.21.15.86:0/3692449259 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf605187f0 con 0x55cf60627990 2024-04-17T07:22:37.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.475+0000 7f8dea000640 1 -- 172.21.15.86:0/3692449259 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8dec01d070 con 0x55cf60627990 2024-04-17T07:22:37.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.475+0000 7f8dea000640 1 -- 172.21.15.86:0/3692449259 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8df4034b20 con 0x55cf60627990 2024-04-17T07:22:37.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.476+0000 7f8e06391740 10 client.36509 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:37.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.476+0000 7f8e06391740 10 client.36509 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:22:37.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.476+0000 7f8e06391740 20 client.36509 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:37.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.476+0000 7f8e06391740 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cf6063cd70 0x55cf6063f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.476+0000 7f8e06391740 4 client.36509 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:37.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.476+0000 7f8e06391740 1 -- 172.21.15.86:0/3692449259 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf6059aa40 con 0x55cf6063cd70 2024-04-17T07:22:37.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.476+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cf6063cd70 0x55cf6063f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.478+0000 7f8dea000640 10 client.36509 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:22:37.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.480+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cf6063cd70 0x55cf6063f1f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=568aca7e09be61ca server_cookie=5b3309bcfabdebc7 in_seq=0 out_seq=0 2024-04-17T07:22:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.482+0000 7f8dea000640 1 -- 172.21.15.86:0/3692449259 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf6059aa40 con 0x55cf6063cd70 2024-04-17T07:22:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.482+0000 7f8dea000640 10 client.36509 handle_command_reply: tid=0 2024-04-17T07:22:37.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.482+0000 7f8e06391740 1 client.36509 shutdown 2024-04-17T07:22:37.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.483+0000 7f8e06391740 1 -- 172.21.15.86:0/3692449259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 msgr2=0x55cf606317e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.483+0000 7f8e06391740 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf606317e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8df4037040 tx=0x7f8df4003b80 comp rx=0 tx=0).stop 2024-04-17T07:22:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.483+0000 7f8e06391740 1 -- 172.21.15.86:0/3692449259 shutdown_connections 2024-04-17T07:22:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.483+0000 7f8e06391740 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cf6063cd70 0x55cf6063f1f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.484+0000 7f8e06391740 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf6062cb00 0x55cf60638820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.484+0000 7f8e06391740 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf606317e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.484+0000 7f8e06391740 1 --2- 172.21.15.86:0/3692449259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf60626fa0 0x55cf606312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.484+0000 7f8e06391740 1 -- 172.21.15.86:0/3692449259 >> 172.21.15.86:0/3692449259 conn(0x55cf60622510 msgr2=0x55cf60621710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.484+0000 7f8dea000640 0 client.36509 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:22:37.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.484+0000 7f8e06391740 1 -- 172.21.15.86:0/3692449259 shutdown_connections 2024-04-17T07:22:37.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.484+0000 7f8e06391740 1 -- 172.21.15.86:0/3692449259 wait complete. 2024-04-17T07:22:37.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.484+0000 7f8e06391740 20 client.36509 trim_cache size 0 max 16384 2024-04-17T07:22:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:37 smithi098 ceph-mon[35418]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.9 MiB/s wr, 677 op/s 2024-04-17T07:22:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:37 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3407600030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:22:37.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.514+0000 7f8e06391740 1 Processor -- start 2024-04-17T07:22:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.515+0000 7f8e06391740 1 -- start start 2024-04-17T07:22:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.515+0000 7f8e06391740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 0x55cf6061d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.515+0000 7f8e06391740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf60625e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.516+0000 7f8e06391740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf606387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.516+0000 7f8e06391740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b9600 con 0x55cf60626fa0 2024-04-17T07:22:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.516+0000 7f8e06391740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1bb140 con 0x55cf60627990 2024-04-17T07:22:37.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.516+0000 7f8e06391740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b9cd0 con 0x55cf6062cb00 2024-04-17T07:22:37.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.516+0000 7f8e00c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf60625e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.516+0000 7f8e00c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf60625e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53482/0 (socket says 172.21.15.86:53482) 2024-04-17T07:22:37.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.516+0000 7f8e00c00640 1 -- 172.21.15.86:0/3701935713 learned_addr learned my addr 172.21.15.86:0/3701935713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.516+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 0x55cf6061d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.517+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/3701935713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf606387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.517+0000 7f8e00c00640 1 -- 172.21.15.86:0/3701935713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 msgr2=0x55cf606387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.517+0000 7f8e00c00640 1 --2- 172.21.15.86:0/3701935713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf606387e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.517+0000 7f8e00c00640 1 -- 172.21.15.86:0/3701935713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 msgr2=0x55cf6061d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.517+0000 7f8e00c00640 1 --2- 172.21.15.86:0/3701935713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 0x55cf6061d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.517+0000 7f8e00c00640 1 -- 172.21.15.86:0/3701935713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf6029f3f0 con 0x55cf60627990 2024-04-17T07:22:37.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.518+0000 7f8e00c00640 1 --2- 172.21.15.86:0/3701935713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf60625e80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8df402f020 tx=0x7f8df4034f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fbbf7db7c2cb2ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.518+0000 7f8df1200640 1 -- 172.21.15.86:0/3701935713 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4044050 con 0x55cf60627990 2024-04-17T07:22:37.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.519+0000 7f8df1200640 1 -- 172.21.15.86:0/3701935713 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df4056070 con 0x55cf60627990 2024-04-17T07:22:37.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.519+0000 7f8df1200640 1 -- 172.21.15.86:0/3701935713 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004290 con 0x55cf60627990 2024-04-17T07:22:37.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.520+0000 7f8e06391740 1 -- 172.21.15.86:0/3701935713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 msgr2=0x55cf60625e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.520+0000 7f8e06391740 1 --2- 172.21.15.86:0/3701935713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf60625e80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8df402f020 tx=0x7f8df4034f90 comp rx=0 tx=0).stop 2024-04-17T07:22:37.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.520+0000 7f8e06391740 1 -- 172.21.15.86:0/3701935713 shutdown_connections 2024-04-17T07:22:37.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.520+0000 7f8e06391740 1 --2- 172.21.15.86:0/3701935713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf606387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.520+0000 7f8e06391740 1 --2- 172.21.15.86:0/3701935713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60627990 0x55cf60625e80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.520+0000 7f8e06391740 1 --2- 172.21.15.86:0/3701935713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60626fa0 0x55cf6061d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.520+0000 7f8e06391740 1 -- 172.21.15.86:0/3701935713 >> 172.21.15.86:0/3701935713 conn(0x55cf60622510 msgr2=0x55cf60625770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.521+0000 7f8e06391740 1 -- 172.21.15.86:0/3701935713 shutdown_connections 2024-04-17T07:22:37.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.522+0000 7f8e06391740 1 -- 172.21.15.86:0/3701935713 wait complete. 2024-04-17T07:22:37.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.522+0000 7f8e06391740 1 -- start start 2024-04-17T07:22:37.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.523+0000 7f8e06391740 1 Processor -- start 2024-04-17T07:22:37.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.523+0000 7f8e06391740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60626fa0 0x55cf60633fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.523+0000 7f8e06391740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60627990 0x55cf60634510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.523+0000 7f8e06391740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf606372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.523+0000 7f8e06391740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1baa70 con 0x55cf60627990 2024-04-17T07:22:37.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.523+0000 7f8e06391740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1ba3a0 con 0x55cf60626fa0 2024-04-17T07:22:37.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.523+0000 7f8e06391740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b75d0 con 0x55cf6062cb00 2024-04-17T07:22:37.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.523+0000 7f8e06391740 10 client.0 _ll_register_callbacks cb 0x7f8df4032a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.524+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60626fa0 0x55cf60633fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.524+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60626fa0 0x55cf60633fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53488/0 (socket says 172.21.15.86:53488) 2024-04-17T07:22:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.524+0000 7f8e00c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60627990 0x55cf60634510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.524+0000 7f8dfbe00640 1 -- 172.21.15.86:0/4079894852 learned_addr learned my addr 172.21.15.86:0/4079894852 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.524+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf606372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.525+0000 7f8e00c00640 1 -- 172.21.15.86:0/4079894852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 msgr2=0x55cf606372c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.525+0000 7f8e00c00640 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf606372c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.525+0000 7f8e00c00640 1 -- 172.21.15.86:0/4079894852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60626fa0 msgr2=0x55cf60633fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.525+0000 7f8e00c00640 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60626fa0 0x55cf60633fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.525+0000 7f8e00c00640 1 -- 172.21.15.86:0/4079894852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf603697d0 con 0x55cf60627990 2024-04-17T07:22:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:37 smithi103 ceph-mon[35495]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.9 MiB/s wr, 677 op/s 2024-04-17T07:22:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:37 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3407600030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:22:37.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.526+0000 7f8e00c00640 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60627990 0x55cf60634510 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8df402f020 tx=0x7f8df4009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97cb7286fcb424 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.526+0000 7f8dea000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:37.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.526+0000 7f8dea000640 1 -- 172.21.15.86:0/4079894852 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df405f030 con 0x55cf60627990 2024-04-17T07:22:37.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.527+0000 7f8dea000640 1 -- 172.21.15.86:0/4079894852 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8df4004200 con 0x55cf60627990 2024-04-17T07:22:37.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.527+0000 7f8e06391740 1 -- 172.21.15.86:0/4079894852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df4056070 con 0x55cf60627990 2024-04-17T07:22:37.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.528+0000 7f8dea000640 1 -- 172.21.15.86:0/4079894852 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8df406a050 con 0x55cf60627990 2024-04-17T07:22:37.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.528+0000 7f8e06391740 10 client.17550 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:37.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.528+0000 7f8e06391740 1 -- 172.21.15.86:0/4079894852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf606389e0 con 0x55cf60627990 2024-04-17T07:22:37.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.529+0000 7f8dea000640 1 -- 172.21.15.86:0/4079894852 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df4056070 con 0x55cf60627990 2024-04-17T07:22:37.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.530+0000 7f8dea000640 1 -- 172.21.15.86:0/4079894852 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8df4034740 con 0x55cf60627990 2024-04-17T07:22:37.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.530+0000 7f8e06391740 10 client.17550 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:37.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.530+0000 7f8e06391740 10 client.17550 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:22:37.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.530+0000 7f8e06391740 20 client.17550 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:37.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.530+0000 7f8e06391740 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cf606b47a0 0x55cf606b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.530+0000 7f8e06391740 4 client.17550 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:22:37.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.530+0000 7f8e06391740 1 -- 172.21.15.86:0/4079894852 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf605948b0 con 0x55cf606b47a0 2024-04-17T07:22:37.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.531+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cf606b47a0 0x55cf606b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.532+0000 7f8dfbe00640 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cf606b47a0 0x55cf606b6bc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=28570f2635eb7fa9 server_cookie=7541bc52886067de in_seq=0 out_seq=0 2024-04-17T07:22:37.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.532+0000 7f8dea000640 10 client.17550 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:22:37.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.533+0000 7f8dea000640 1 -- 172.21.15.86:0/4079894852 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf605948b0 con 0x55cf606b47a0 2024-04-17T07:22:37.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.533+0000 7f8dea000640 10 client.17550 handle_command_reply: tid=0 2024-04-17T07:22:37.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.533+0000 7f8e06391740 1 client.17550 shutdown 2024-04-17T07:22:37.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.533+0000 7f8e06391740 1 -- 172.21.15.86:0/4079894852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60627990 msgr2=0x55cf60634510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.533+0000 7f8e06391740 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60627990 0x55cf60634510 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8df402f020 tx=0x7f8df4009890 comp rx=0 tx=0).stop 2024-04-17T07:22:37.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.533+0000 7f8e06391740 1 -- 172.21.15.86:0/4079894852 shutdown_connections 2024-04-17T07:22:37.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.533+0000 7f8e06391740 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cf606b47a0 0x55cf606b6bc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.534+0000 7f8e06391740 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf6062cb00 0x55cf606372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.534+0000 7f8e06391740 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cf60627990 0x55cf60634510 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.534+0000 7f8dea000640 0 client.17550 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:22:37.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.534+0000 7f8e06391740 1 --2- 172.21.15.86:0/4079894852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cf60626fa0 0x55cf60633fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.534+0000 7f8e06391740 1 -- 172.21.15.86:0/4079894852 >> 172.21.15.86:0/4079894852 conn(0x55cf60622510 msgr2=0x55cf60620e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.534+0000 7f8e06391740 1 -- 172.21.15.86:0/4079894852 shutdown_connections 2024-04-17T07:22:37.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.534+0000 7f8e06391740 1 -- 172.21.15.86:0/4079894852 wait complete. 2024-04-17T07:22:37.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.535+0000 7f8e06391740 20 client.17550 trim_cache size 0 max 16384 2024-04-17T07:22:37.535 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:22:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.536+0000 7f8e03600640 1 -- 172.21.15.86:0/895172109 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 msgr2=0x7f8de00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.536+0000 7f8e03600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 0x7f8de00a6240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8de409c2f0 tx=0x7f8de40d0940 comp rx=0 tx=0).stop 2024-04-17T07:22:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.536+0000 7f8e03600640 1 -- 172.21.15.86:0/895172109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc105d70 msgr2=0x7f8dfc19d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.536+0000 7f8e03600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc105d70 0x7f8dfc19d410 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8de400c970 tx=0x7f8de400cd70 comp rx=0 tx=0).stop 2024-04-17T07:22:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.537+0000 7f8e03600640 1 -- 172.21.15.86:0/895172109 shutdown_connections 2024-04-17T07:22:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.537+0000 7f8e03600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8de00a3d80 0x7f8de00a6240 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.537+0000 7f8e03600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dfc105d70 0x7f8dfc19d410 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.537+0000 7f8e03600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8dfc1053b0 0x7f8dfc1963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.537+0000 7f8e03600640 1 --2- 172.21.15.86:0/895172109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc1041c0 0x7f8dfc195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.537+0000 7f8e03600640 1 -- 172.21.15.86:0/895172109 >> 172.21.15.86:0/895172109 conn(0x7f8dfc0ff750 msgr2=0x7f8dfc0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.537+0000 7f8e03600640 1 -- 172.21.15.86:0/895172109 shutdown_connections 2024-04-17T07:22:37.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.537+0000 7f8e03600640 1 -- 172.21.15.86:0/895172109 wait complete. 2024-04-17T07:22:37.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:37 smithi086 ceph-mon[31733]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.9 MiB/s wr, 677 op/s 2024-04-17T07:22:37.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:37 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3407600030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:22:37.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:22:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.925+0000 7f71ed600640 1 -- 172.21.15.86:0/2532846597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71e8075fa0 msgr2=0x7f71e80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.925+0000 7f71ed600640 1 --2- 172.21.15.86:0/2532846597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71e8075fa0 0x7f71e80743f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009930 tx=0x7f71d802bc20 comp rx=0 tx=0).stop 2024-04-17T07:22:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.926+0000 7f71ed600640 1 -- 172.21.15.86:0/2532846597 shutdown_connections 2024-04-17T07:22:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.926+0000 7f71ed600640 1 --2- 172.21.15.86:0/2532846597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71e810e580 0x7f71e8110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.926+0000 7f71ed600640 1 --2- 172.21.15.86:0/2532846597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e80749c0 0x7f71e8074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.926+0000 7f71ed600640 1 --2- 172.21.15.86:0/2532846597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71e8075fa0 0x7f71e80743f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.926+0000 7f71ed600640 1 -- 172.21.15.86:0/2532846597 >> 172.21.15.86:0/2532846597 conn(0x7f71e80fd540 msgr2=0x7f71e80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.926+0000 7f71ed600640 1 -- 172.21.15.86:0/2532846597 shutdown_connections 2024-04-17T07:22:37.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.926+0000 7f71ed600640 1 -- 172.21.15.86:0/2532846597 wait complete. 2024-04-17T07:22:37.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.928+0000 7f71ed600640 1 Processor -- start 2024-04-17T07:22:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.928+0000 7f71ed600640 1 -- start start 2024-04-17T07:22:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.928+0000 7f71ed600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e80749c0 0x7f71e819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.928+0000 7f71ed600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71e8075fa0 0x7f71e819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.928+0000 7f71ed600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71e810e580 0x7f71e81a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.928+0000 7f71ed600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8114100 con 0x7f71e8075fa0 2024-04-17T07:22:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.929+0000 7f71ed600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8113f80 con 0x7f71e810e580 2024-04-17T07:22:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.929+0000 7f71ed600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8114280 con 0x7f71e80749c0 2024-04-17T07:22:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.929+0000 7f71e6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e80749c0 0x7f71e819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.930+0000 7f71e6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e80749c0 0x7f71e819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46382/0 (socket says 172.21.15.86:46382) 2024-04-17T07:22:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.930+0000 7f71e6a00640 1 -- 172.21.15.86:0/994599645 learned_addr learned my addr 172.21.15.86:0/994599645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.930+0000 7f71e7400640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71e810e580 0x7f71e81a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.930+0000 7f71e6000640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71e8075fa0 0x7f71e819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.930+0000 7f71e6a00640 1 -- 172.21.15.86:0/994599645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71e810e580 msgr2=0x7f71e81a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.930+0000 7f71e6a00640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71e810e580 0x7f71e81a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.930+0000 7f71e6a00640 1 -- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71e8075fa0 msgr2=0x7f71e819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.930+0000 7f71e6a00640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71e8075fa0 0x7f71e819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.930+0000 7f71e6a00640 1 -- 172.21.15.86:0/994599645 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e8105ea0 con 0x7f71e80749c0 2024-04-17T07:22:37.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.930+0000 7f71e6000640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71e8075fa0 0x7f71e819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:37.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.931+0000 7f71e6a00640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e80749c0 0x7f71e819e750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009900 tx=0x7f71d8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.931+0000 7f71df400640 1 -- 172.21.15.86:0/994599645 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8043020 con 0x7f71e80749c0 2024-04-17T07:22:37.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.931+0000 7f71df400640 1 -- 172.21.15.86:0/994599645 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71d80052b0 con 0x7f71e80749c0 2024-04-17T07:22:37.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.931+0000 7f71df400640 1 -- 172.21.15.86:0/994599645 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8003c80 con 0x7f71e80749c0 2024-04-17T07:22:37.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.931+0000 7f71ed600640 1 -- 172.21.15.86:0/994599645 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e8106130 con 0x7f71e80749c0 2024-04-17T07:22:37.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.932+0000 7f71ed600640 1 -- 172.21.15.86:0/994599645 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e8106630 con 0x7f71e80749c0 2024-04-17T07:22:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.936+0000 7f71df400640 1 -- 172.21.15.86:0/994599645 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f71d8046020 con 0x7f71e80749c0 2024-04-17T07:22:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.937+0000 7f71f0363740 1 Processor -- start 2024-04-17T07:22:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.938+0000 7f71f0363740 1 -- start start 2024-04-17T07:22:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.940+0000 7f71f0363740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d16fa0 0x55ec74d173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.940+0000 7f71f0363740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d17990 0x55ec74d1c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.940+0000 7f71f0363740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d1ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.940+0000 7f71f0363740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec74c7d020 con 0x55ec74d16fa0 2024-04-17T07:22:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.940+0000 7f71f0363740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ec74c23460 con 0x55ec74d17990 2024-04-17T07:22:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.940+0000 7f71f0363740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ec74c78fe0 con 0x55ec74d1cb00 2024-04-17T07:22:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.941+0000 7f71e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d16fa0 0x55ec74d173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.941+0000 7f71e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d16fa0 0x55ec74d173c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56662/0 (socket says 172.21.15.86:56662) 2024-04-17T07:22:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.942+0000 7f71e6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d17990 0x55ec74d1c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.942+0000 7f71e7400640 1 -- 172.21.15.86:0/1828799890 learned_addr learned my addr 172.21.15.86:0/1828799890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.942+0000 7f71e6000640 1 --2- 172.21.15.86:0/1828799890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d1ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.942+0000 7f71df400640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71cc0a3c80 0x7f71cc0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.942+0000 7f71df400640 1 -- 172.21.15.86:0/994599645 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f71d80f2c40 con 0x7f71e80749c0 2024-04-17T07:22:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.943+0000 7f71e6000640 1 -- 172.21.15.86:0/1828799890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d17990 msgr2=0x55ec74d1c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.943+0000 7f71e6000640 1 --2- 172.21.15.86:0/1828799890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d17990 0x55ec74d1c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.943+0000 7f71e6000640 1 -- 172.21.15.86:0/1828799890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d16fa0 msgr2=0x55ec74d173c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.943+0000 7f71e6000640 1 --2- 172.21.15.86:0/1828799890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d16fa0 0x55ec74d173c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.943+0000 7f71e6000640 1 -- 172.21.15.86:0/1828799890 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec74c2b040 con 0x55ec74d1cb00 2024-04-17T07:22:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.943+0000 7f71e7400640 1 --2- 172.21.15.86:0/1828799890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d16fa0 0x55ec74d173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.944+0000 7f71e6000640 1 --2- 172.21.15.86:0/1828799890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d1ef90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71e819fc60 tx=0x7f71d400bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=431347504111d886 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.944+0000 7f71e7400640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71cc0a3c80 0x7f71cc0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.945+0000 7f71d3400640 1 -- 172.21.15.86:0/1828799890 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4017420 con 0x55ec74d1cb00 2024-04-17T07:22:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.946+0000 7f71e7400640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71cc0a3c80 0x7f71cc0a6140 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55ec74d181f0 tx=0x7f71e0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.946+0000 7f71d3400640 1 -- 172.21.15.86:0/1828799890 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71d4017a70 con 0x55ec74d1cb00 2024-04-17T07:22:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.946+0000 7f71d3400640 1 -- 172.21.15.86:0/1828799890 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d400faf0 con 0x55ec74d1cb00 2024-04-17T07:22:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.947+0000 7f71f0363740 1 -- 172.21.15.86:0/1828799890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 msgr2=0x55ec74d1ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.947+0000 7f71f0363740 1 --2- 172.21.15.86:0/1828799890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d1ef90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f71e819fc60 tx=0x7f71d400bf00 comp rx=0 tx=0).stop 2024-04-17T07:22:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.948+0000 7f71f0363740 1 -- 172.21.15.86:0/1828799890 shutdown_connections 2024-04-17T07:22:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.948+0000 7f71f0363740 1 --2- 172.21.15.86:0/1828799890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d1ef90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.948+0000 7f71f0363740 1 --2- 172.21.15.86:0/1828799890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d17990 0x55ec74d1c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.948+0000 7f71f0363740 1 --2- 172.21.15.86:0/1828799890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d16fa0 0x55ec74d173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.948+0000 7f71f0363740 1 -- 172.21.15.86:0/1828799890 >> 172.21.15.86:0/1828799890 conn(0x55ec74d12510 msgr2=0x55ec74d12910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.949+0000 7f71f0363740 1 -- 172.21.15.86:0/1828799890 shutdown_connections 2024-04-17T07:22:37.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.949+0000 7f71f0363740 1 -- 172.21.15.86:0/1828799890 wait complete. 2024-04-17T07:22:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.950+0000 7f71f0363740 1 -- start start 2024-04-17T07:22:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.950+0000 7f71f0363740 1 Processor -- start 2024-04-17T07:22:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.951+0000 7f71f0363740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 0x55ec74d214c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.951+0000 7f71f0363740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d21a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.951+0000 7f71f0363740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d28a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.951+0000 7f71f0363740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec74d1fab0 con 0x55ec74d17990 2024-04-17T07:22:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.951+0000 7f71f0363740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ec74c36830 con 0x55ec74d16fa0 2024-04-17T07:22:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.951+0000 7f71f0363740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ec74d1fc30 con 0x55ec74d1cb00 2024-04-17T07:22:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.951+0000 7f71f0363740 10 client.0 _ll_register_callbacks cb 0x55ec74c04fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.952+0000 7f71e6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 0x55ec74d214c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.953+0000 7f71e6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d21a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.953+0000 7f71e6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 0x55ec74d214c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53520/0 (socket says 172.21.15.86:53520) 2024-04-17T07:22:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.953+0000 7f71e6a00640 1 -- 172.21.15.86:0/3441301063 learned_addr learned my addr 172.21.15.86:0/3441301063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.953+0000 7f71e7400640 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d28a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.954+0000 7f71e7400640 1 -- 172.21.15.86:0/3441301063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 msgr2=0x55ec74d214c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.954+0000 7f71e7400640 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 0x55ec74d214c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.954+0000 7f71e7400640 1 -- 172.21.15.86:0/3441301063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 msgr2=0x55ec74d21a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.954+0000 7f71e7400640 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d21a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.954+0000 7f71e7400640 1 -- 172.21.15.86:0/3441301063 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec74c3a920 con 0x55ec74d1cb00 2024-04-17T07:22:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.954+0000 7f71e6000640 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d21a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.955+0000 7f71e7400640 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d28a40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f71e0010c90 tx=0x7f71e0007630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3f9f17c4d20e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.955+0000 7f71c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:22:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.955+0000 7f71c7e00640 1 -- 172.21.15.86:0/3441301063 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e001de40 con 0x55ec74d1cb00 2024-04-17T07:22:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.955+0000 7f71c7e00640 1 -- 172.21.15.86:0/3441301063 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f71e001b450 con 0x55ec74d1cb00 2024-04-17T07:22:37.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.956+0000 7f71f0363740 1 -- 172.21.15.86:0/3441301063 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d4017a70 con 0x55ec74d1cb00 2024-04-17T07:22:37.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.956+0000 7f71c7e00640 1 -- 172.21.15.86:0/3441301063 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71e0004b20 con 0x55ec74d1cb00 2024-04-17T07:22:37.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.957+0000 7f71f0363740 10 client.26542 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:37.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.957+0000 7f71f0363740 1 -- 172.21.15.86:0/3441301063 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec74c087f0 con 0x55ec74d1cb00 2024-04-17T07:22:37.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.957+0000 7f71c7e00640 1 -- 172.21.15.86:0/3441301063 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71d4017a70 con 0x55ec74d1cb00 2024-04-17T07:22:37.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.958+0000 7f71c7e00640 1 -- 172.21.15.86:0/3441301063 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f71e0024070 con 0x55ec74d1cb00 2024-04-17T07:22:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.958+0000 7f71f0363740 10 client.26542 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.958+0000 7f71f0363740 10 client.26542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:22:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.958+0000 7f71f0363740 20 client.26542 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.958+0000 7f71f0363740 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55ec74d3e010 0x55ec74d404b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.958+0000 7f71f0363740 4 client.26542 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.958+0000 7f71f0363740 1 -- 172.21.15.86:0/3441301063 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec74c8aa40 con 0x55ec74d3e010 2024-04-17T07:22:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.958+0000 7f71e6a00640 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55ec74d3e010 0x55ec74d404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.959+0000 7f71e6a00640 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55ec74d3e010 0x55ec74d404b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d03671340bb35f6 server_cookie=476ac7f9c933ca79 in_seq=0 out_seq=0 2024-04-17T07:22:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.959+0000 7f71c7e00640 10 client.26542 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:22:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.960+0000 7f71c7e00640 1 -- 172.21.15.86:0/3441301063 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec74c8aa40 con 0x55ec74d3e010 2024-04-17T07:22:37.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.960+0000 7f71c7e00640 10 client.26542 handle_command_reply: tid=0 2024-04-17T07:22:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.961+0000 7f71f0363740 1 client.26542 shutdown 2024-04-17T07:22:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.962+0000 7f71f0363740 1 -- 172.21.15.86:0/3441301063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 msgr2=0x55ec74d28a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.962+0000 7f71f0363740 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d28a40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f71e0010c90 tx=0x7f71e0007630 comp rx=0 tx=0).stop 2024-04-17T07:22:37.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.962+0000 7f71f0363740 1 -- 172.21.15.86:0/3441301063 shutdown_connections 2024-04-17T07:22:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.962+0000 7f71f0363740 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55ec74d3e010 0x55ec74d404b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.962+0000 7f71f0363740 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d28a40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.962+0000 7f71f0363740 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d21a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.962+0000 7f71f0363740 1 --2- 172.21.15.86:0/3441301063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 0x55ec74d214c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.962+0000 7f71c7e00640 0 client.26542 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:22:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.962+0000 7f71f0363740 1 -- 172.21.15.86:0/3441301063 >> 172.21.15.86:0/3441301063 conn(0x55ec74d12510 msgr2=0x55ec74d11710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.963+0000 7f71f0363740 1 -- 172.21.15.86:0/3441301063 shutdown_connections 2024-04-17T07:22:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.963+0000 7f71f0363740 1 -- 172.21.15.86:0/3441301063 wait complete. 2024-04-17T07:22:37.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.963+0000 7f71f0363740 20 client.26542 trim_cache size 0 max 16384 2024-04-17T07:22:37.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.985+0000 7f71f0363740 1 Processor -- start 2024-04-17T07:22:37.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.985+0000 7f71f0363740 1 -- start start 2024-04-17T07:22:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.985+0000 7f71f0363740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d16fa0 0x55ec74d20ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.985+0000 7f71f0363740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d28a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.985+0000 7f71f0363740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d1cb00 0x55ec74d10e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.985+0000 7f71f0363740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71e806df30 con 0x55ec74d17990 2024-04-17T07:22:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.985+0000 7f71f0363740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f71e806fa70 con 0x55ec74d1cb00 2024-04-17T07:22:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.985+0000 7f71f0363740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71e806e600 con 0x55ec74d16fa0 2024-04-17T07:22:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.986+0000 7f71e7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d1cb00 0x55ec74d10e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.986+0000 7f71e7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d1cb00 0x55ec74d10e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53532/0 (socket says 172.21.15.86:53532) 2024-04-17T07:22:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.986+0000 7f71e7400640 1 -- 172.21.15.86:0/3078166053 learned_addr learned my addr 172.21.15.86:0/3078166053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.986+0000 7f71e6a00640 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d16fa0 0x55ec74d20ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.986+0000 7f71e7400640 1 -- 172.21.15.86:0/3078166053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d16fa0 msgr2=0x55ec74d20ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.986+0000 7f71e7400640 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d16fa0 0x55ec74d20ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.986+0000 7f71e7400640 1 -- 172.21.15.86:0/3078166053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 msgr2=0x55ec74d28a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:22:37.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.986+0000 7f71e6000640 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d28a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.987+0000 7f71e7400640 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d28a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.987+0000 7f71e7400640 1 -- 172.21.15.86:0/3078166053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7498f3f0 con 0x55ec74d1cb00 2024-04-17T07:22:37.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.987+0000 7f71e6a00640 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d16fa0 0x55ec74d20ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:37.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.987+0000 7f71e6000640 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d28a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:37.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.987+0000 7f71e7400640 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d1cb00 0x55ec74d10e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f71e0010d30 tx=0x7f71e0007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60ada108cbc31c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.989+0000 7f71d2a00640 1 -- 172.21.15.86:0/3078166053 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0004ec0 con 0x55ec74d1cb00 2024-04-17T07:22:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.990+0000 7f71d2a00640 1 -- 172.21.15.86:0/3078166053 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e001b450 con 0x55ec74d1cb00 2024-04-17T07:22:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.990+0000 7f71d2a00640 1 -- 172.21.15.86:0/3078166053 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e00059d0 con 0x55ec74d1cb00 2024-04-17T07:22:37.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.991+0000 7f71f0363740 1 -- 172.21.15.86:0/3078166053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d1cb00 msgr2=0x55ec74d10e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.991+0000 7f71f0363740 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d1cb00 0x55ec74d10e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f71e0010d30 tx=0x7f71e0007dd0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.992+0000 7f71f0363740 1 -- 172.21.15.86:0/3078166053 shutdown_connections 2024-04-17T07:22:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.992+0000 7f71f0363740 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d1cb00 0x55ec74d10e80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.992+0000 7f71f0363740 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d28a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.992+0000 7f71f0363740 1 --2- 172.21.15.86:0/3078166053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d16fa0 0x55ec74d20ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.992+0000 7f71f0363740 1 -- 172.21.15.86:0/3078166053 >> 172.21.15.86:0/3078166053 conn(0x55ec74d12510 msgr2=0x55ec74d18ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.992+0000 7f71f0363740 1 -- 172.21.15.86:0/3078166053 shutdown_connections 2024-04-17T07:22:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.992+0000 7f71f0363740 1 -- 172.21.15.86:0/3078166053 wait complete. 2024-04-17T07:22:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.992+0000 7f71f0363740 1 -- start start 2024-04-17T07:22:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.993+0000 7f71f0363740 1 Processor -- start 2024-04-17T07:22:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.993+0000 7f71f0363740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 0x55ec74d25bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.993+0000 7f71f0363740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d26110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.993+0000 7f71f0363740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d27080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.993+0000 7f71f0363740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71e806f3a0 con 0x55ec74d17990 2024-04-17T07:22:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.993+0000 7f71f0363740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f71e806ecd0 con 0x55ec74d16fa0 2024-04-17T07:22:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.993+0000 7f71f0363740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71e810e1f0 con 0x55ec74d1cb00 2024-04-17T07:22:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.993+0000 7f71f0363740 10 client.0 _ll_register_callbacks cb 0x7f71e0007d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.994+0000 7f71e6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.994+0000 7f71e6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d26110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56694/0 (socket says 172.21.15.86:56694) 2024-04-17T07:22:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.994+0000 7f71e6000640 1 -- 172.21.15.86:0/4107704352 learned_addr learned my addr 172.21.15.86:0/4107704352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.994+0000 7f71e6a00640 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 0x55ec74d25bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.994+0000 7f71e7400640 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d27080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.995+0000 7f71e6a00640 1 -- 172.21.15.86:0/4107704352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 msgr2=0x55ec74d27080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.995+0000 7f71e6a00640 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d27080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.995+0000 7f71e6a00640 1 -- 172.21.15.86:0/4107704352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 msgr2=0x55ec74d26110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.995+0000 7f71e6a00640 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d26110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.995+0000 7f71e6a00640 1 -- 172.21.15.86:0/4107704352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec74a597d0 con 0x55ec74d16fa0 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.995+0000 7f71e7400640 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d27080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.995+0000 7f71e6000640 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d26110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.996+0000 7f71e6a00640 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 0x55ec74d25bd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f71d80f0730 tx=0x7f71d8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c2b73de4117f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.996+0000 7f71c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.996+0000 7f71c7e00640 1 -- 172.21.15.86:0/4107704352 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d80c2f50 con 0x55ec74d16fa0 2024-04-17T07:22:37.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.996+0000 7f71f0363740 1 -- 172.21.15.86:0/4107704352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e001b450 con 0x55ec74d16fa0 2024-04-17T07:22:37.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.996+0000 7f71c7e00640 1 -- 172.21.15.86:0/4107704352 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f71d80bcea0 con 0x55ec74d16fa0 2024-04-17T07:22:37.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.997+0000 7f71c7e00640 1 -- 172.21.15.86:0/4107704352 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71d80c7810 con 0x55ec74d16fa0 2024-04-17T07:22:37.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.997+0000 7f71f0363740 10 client.36530 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:37.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.997+0000 7f71f0363740 1 -- 172.21.15.86:0/4107704352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec74d28780 con 0x55ec74d16fa0 2024-04-17T07:22:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.997+0000 7f71c7e00640 1 -- 172.21.15.86:0/4107704352 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e001b450 con 0x55ec74d16fa0 2024-04-17T07:22:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.997+0000 7f71c7e00640 1 -- 172.21.15.86:0/4107704352 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f71d8005460 con 0x55ec74d16fa0 2024-04-17T07:22:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.998+0000 7f71f0363740 10 client.36530 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.998+0000 7f71f0363740 10 client.36530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:22:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.998+0000 7f71f0363740 20 client.36530 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.998+0000 7f71f0363740 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55ec74da4790 0x55ec74da6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.998+0000 7f71f0363740 4 client.36530 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:22:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.998+0000 7f71f0363740 1 -- 172.21.15.86:0/4107704352 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec74c848b0 con 0x55ec74da4790 2024-04-17T07:22:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.998+0000 7f71e6000640 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55ec74da4790 0x55ec74da6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.999+0000 7f71e6000640 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55ec74da4790 0x55ec74da6bb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48a1bbe7c1ec6e0 server_cookie=829f8f42e870c240 in_seq=0 out_seq=0 2024-04-17T07:22:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:37.999+0000 7f71c7e00640 10 client.36530 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:22:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.000+0000 7f71c7e00640 1 -- 172.21.15.86:0/4107704352 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec74c848b0 con 0x55ec74da4790 2024-04-17T07:22:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.000+0000 7f71c7e00640 10 client.36530 handle_command_reply: tid=0 2024-04-17T07:22:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.000+0000 7f71f0363740 1 client.36530 shutdown 2024-04-17T07:22:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.001+0000 7f71f0363740 1 -- 172.21.15.86:0/4107704352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 msgr2=0x55ec74d25bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.001+0000 7f71f0363740 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 0x55ec74d25bd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f71d80f0730 tx=0x7f71d8005d50 comp rx=0 tx=0).stop 2024-04-17T07:22:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.001+0000 7f71f0363740 1 -- 172.21.15.86:0/4107704352 shutdown_connections 2024-04-17T07:22:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.001+0000 7f71f0363740 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55ec74da4790 0x55ec74da6bb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.001+0000 7f71f0363740 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ec74d1cb00 0x55ec74d27080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.001+0000 7f71f0363740 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec74d17990 0x55ec74d26110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.001+0000 7f71f0363740 1 --2- 172.21.15.86:0/4107704352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ec74d16fa0 0x55ec74d25bd0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f71d80f0730 tx=0x7f71d8005d50 comp rx=0 tx=0).stop 2024-04-17T07:22:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.001+0000 7f71f0363740 1 -- 172.21.15.86:0/4107704352 >> 172.21.15.86:0/4107704352 conn(0x55ec74d12510 msgr2=0x55ec74d16000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.001+0000 7f71c7e00640 0 client.36530 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:22:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.001+0000 7f71f0363740 1 -- 172.21.15.86:0/4107704352 shutdown_connections 2024-04-17T07:22:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.002+0000 7f71f0363740 1 -- 172.21.15.86:0/4107704352 wait complete. 2024-04-17T07:22:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.002+0000 7f71f0363740 20 client.36530 trim_cache size 0 max 16384 2024-04-17T07:22:38.003 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:22:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.003+0000 7f71ed600640 1 -- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71cc0a3c80 msgr2=0x7f71cc0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.003+0000 7f71ed600640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71cc0a3c80 0x7f71cc0a6140 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55ec74d181f0 tx=0x7f71e0004a40 comp rx=0 tx=0).stop 2024-04-17T07:22:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.003+0000 7f71ed600640 1 -- 172.21.15.86:0/994599645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e80749c0 msgr2=0x7f71e819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.003+0000 7f71ed600640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e80749c0 0x7f71e819e750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009900 tx=0x7f71d8004270 comp rx=0 tx=0).stop 2024-04-17T07:22:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.003+0000 7f71ed600640 1 -- 172.21.15.86:0/994599645 shutdown_connections 2024-04-17T07:22:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.003+0000 7f71ed600640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f71cc0a3c80 0x7f71cc0a6140 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.003+0000 7f71ed600640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71e810e580 0x7f71e81a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.003+0000 7f71ed600640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71e8075fa0 0x7f71e819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.003+0000 7f71ed600640 1 --2- 172.21.15.86:0/994599645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71e80749c0 0x7f71e819e750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.003+0000 7f71ed600640 1 -- 172.21.15.86:0/994599645 >> 172.21.15.86:0/994599645 conn(0x7f71e80fd540 msgr2=0x7f71e8101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.004+0000 7f71ed600640 1 -- 172.21.15.86:0/994599645 shutdown_connections 2024-04-17T07:22:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.004+0000 7f71ed600640 1 -- 172.21.15.86:0/994599645 wait complete. 2024-04-17T07:22:38.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:22:38.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.359+0000 7feb48e00640 1 -- 172.21.15.86:0/246477375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb441098c0 msgr2=0x7feb441102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.359+0000 7feb48e00640 1 --2- 172.21.15.86:0/246477375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb441098c0 0x7feb441102a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feb30009860 tx=0x7feb3002bb70 comp rx=0 tx=0).stop 2024-04-17T07:22:38.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.359+0000 7feb48e00640 1 -- 172.21.15.86:0/246477375 shutdown_connections 2024-04-17T07:22:38.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.359+0000 7feb48e00640 1 --2- 172.21.15.86:0/246477375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb441107e0 0x7feb44112bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.359+0000 7feb48e00640 1 --2- 172.21.15.86:0/246477375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb441098c0 0x7feb441102a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.359+0000 7feb48e00640 1 --2- 172.21.15.86:0/246477375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb44108f60 0x7feb44109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.359+0000 7feb48e00640 1 -- 172.21.15.86:0/246477375 >> 172.21.15.86:0/246477375 conn(0x7feb4406f190 msgr2=0x7feb440715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.361+0000 7feb48e00640 1 -- 172.21.15.86:0/246477375 shutdown_connections 2024-04-17T07:22:38.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.362+0000 7feb48e00640 1 -- 172.21.15.86:0/246477375 wait complete. 2024-04-17T07:22:38.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.364+0000 7feb48e00640 1 Processor -- start 2024-04-17T07:22:38.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.365+0000 7feb48e00640 1 -- start start 2024-04-17T07:22:38.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.365+0000 7feb48e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb44108f60 0x7feb4410bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.365+0000 7feb48e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb441098c0 0x7feb4410a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.365+0000 7feb48e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb441107e0 0x7feb4410a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.365+0000 7feb48e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb441162c0 con 0x7feb441107e0 2024-04-17T07:22:38.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.365+0000 7feb48e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb44116140 con 0x7feb44108f60 2024-04-17T07:22:38.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.365+0000 7feb48e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb44116440 con 0x7feb441098c0 2024-04-17T07:22:38.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.366+0000 7feb41400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb441098c0 0x7feb4410a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.366+0000 7feb41400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb441098c0 0x7feb4410a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46440/0 (socket says 172.21.15.86:46440) 2024-04-17T07:22:38.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.366+0000 7feb41400640 1 -- 172.21.15.86:0/2279390196 learned_addr learned my addr 172.21.15.86:0/2279390196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:38.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.366+0000 7feb41e00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb44108f60 0x7feb4410bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.366+0000 7feb42800640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb441107e0 0x7feb4410a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.367+0000 7feb41e00640 1 -- 172.21.15.86:0/2279390196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb441098c0 msgr2=0x7feb4410a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.367+0000 7feb41e00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb441098c0 0x7feb4410a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.367+0000 7feb41e00640 1 -- 172.21.15.86:0/2279390196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb441107e0 msgr2=0x7feb4410a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.367+0000 7feb41e00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb441107e0 0x7feb4410a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.367+0000 7feb41e00640 1 -- 172.21.15.86:0/2279390196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4410b030 con 0x7feb44108f60 2024-04-17T07:22:38.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.367+0000 7feb41400640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb441098c0 0x7feb4410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:38.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.368+0000 7feb41e00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb44108f60 0x7feb4410bba0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e9f0 tx=0x7feb3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.368+0000 7feb3aa00640 1 -- 172.21.15.86:0/2279390196 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3400ccc0 con 0x7feb44108f60 2024-04-17T07:22:38.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.368+0000 7feb3aa00640 1 -- 172.21.15.86:0/2279390196 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb340108e0 con 0x7feb44108f60 2024-04-17T07:22:38.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.368+0000 7feb3aa00640 1 -- 172.21.15.86:0/2279390196 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34018840 con 0x7feb44108f60 2024-04-17T07:22:38.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.369+0000 7feb48e00640 1 -- 172.21.15.86:0/2279390196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb441a0730 con 0x7feb44108f60 2024-04-17T07:22:38.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.369+0000 7feb48e00640 1 -- 172.21.15.86:0/2279390196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb441a0c00 con 0x7feb44108f60 2024-04-17T07:22:38.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.373+0000 7feb4bb98740 1 Processor -- start 2024-04-17T07:22:38.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.373+0000 7feb3aa00640 1 -- 172.21.15.86:0/2279390196 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7feb34010a90 con 0x7feb44108f60 2024-04-17T07:22:38.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.375+0000 7feb4bb98740 1 -- start start 2024-04-17T07:22:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.376+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667438fa0 0x55f6674393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.376+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667439990 0x55f66743e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.376+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 0x55f667440f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.376+0000 7feb4bb98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f66739f020 con 0x55f667439990 2024-04-17T07:22:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.376+0000 7feb4bb98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f667345460 con 0x55f667438fa0 2024-04-17T07:22:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.376+0000 7feb4bb98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f66739afe0 con 0x55f66743eb00 2024-04-17T07:22:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.376+0000 7feb41400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 0x55f667440f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.377+0000 7feb41e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667439990 0x55f66743e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.377+0000 7feb41400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 0x55f667440f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46446/0 (socket says 172.21.15.86:46446) 2024-04-17T07:22:38.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.377+0000 7feb41400640 1 -- 172.21.15.86:0/2096783858 learned_addr learned my addr 172.21.15.86:0/2096783858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:38.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.377+0000 7feb3aa00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb280a3d00 0x7feb280a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.377+0000 7feb3aa00640 1 -- 172.21.15.86:0/2279390196 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feb340d0a80 con 0x7feb44108f60 2024-04-17T07:22:38.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.377+0000 7feb41400640 1 -- 172.21.15.86:0/2096783858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667438fa0 msgr2=0x55f6674393c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:22:38.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.377+0000 7feb41400640 1 --2- 172.21.15.86:0/2096783858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667438fa0 0x55f6674393c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.377+0000 7feb41400640 1 -- 172.21.15.86:0/2096783858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667439990 msgr2=0x55f66743e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.377+0000 7feb41400640 1 --2- 172.21.15.86:0/2096783858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667439990 0x55f66743e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.377+0000 7feb41400640 1 -- 172.21.15.86:0/2096783858 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66734d040 con 0x55f66743eb00 2024-04-17T07:22:38.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.378+0000 7feb41400640 1 --2- 172.21.15.86:0/2096783858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 0x55f667440f90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feb3002c0b0 tx=0x7feb30005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2c32bec8be88303 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.379+0000 7feb42800640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb280a3d00 0x7feb280a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.379+0000 7feb2f400640 1 -- 172.21.15.86:0/2096783858 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30043020 con 0x55f66743eb00 2024-04-17T07:22:38.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.379+0000 7feb2f400640 1 -- 172.21.15.86:0/2096783858 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb30034df0 con 0x55f66743eb00 2024-04-17T07:22:38.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.379+0000 7feb2f400640 1 -- 172.21.15.86:0/2096783858 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30039040 con 0x55f66743eb00 2024-04-17T07:22:38.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.380+0000 7feb4bb98740 1 -- 172.21.15.86:0/2096783858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 msgr2=0x55f667440f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.380+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2096783858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 0x55f667440f90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feb3002c0b0 tx=0x7feb30005dc0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.381+0000 7feb42800640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb280a3d00 0x7feb280a61c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55f66743a1f0 tx=0x7feb3c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.381+0000 7feb4bb98740 1 -- 172.21.15.86:0/2096783858 shutdown_connections 2024-04-17T07:22:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.381+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2096783858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 0x55f667440f90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.381+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2096783858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667439990 0x55f66743e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.381+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2096783858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667438fa0 0x55f6674393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.381+0000 7feb4bb98740 1 -- 172.21.15.86:0/2096783858 >> 172.21.15.86:0/2096783858 conn(0x55f667434510 msgr2=0x55f667434910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.381+0000 7feb4bb98740 1 -- 172.21.15.86:0/2096783858 shutdown_connections 2024-04-17T07:22:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.381+0000 7feb4bb98740 1 -- 172.21.15.86:0/2096783858 wait complete. 2024-04-17T07:22:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.382+0000 7feb4bb98740 1 -- start start 2024-04-17T07:22:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.382+0000 7feb4bb98740 1 Processor -- start 2024-04-17T07:22:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.382+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 0x55f6674434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.382+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667443a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.382+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f66743eb00 0x55f66744aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.383+0000 7feb4bb98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f667441a50 con 0x55f667438fa0 2024-04-17T07:22:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.383+0000 7feb4bb98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f667358830 con 0x55f66743eb00 2024-04-17T07:22:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.383+0000 7feb4bb98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f667441bd0 con 0x55f667439990 2024-04-17T07:22:38.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.383+0000 7feb4bb98740 10 client.0 _ll_register_callbacks cb 0x55f667326fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:38.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.383+0000 7feb41400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667443a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.383+0000 7feb42800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f66743eb00 0x55f66744aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.383+0000 7feb41e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 0x55f6674434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.383+0000 7feb41400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667443a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46454/0 (socket says 172.21.15.86:46454) 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.383+0000 7feb41400640 1 -- 172.21.15.86:0/2236349912 learned_addr learned my addr 172.21.15.86:0/2236349912 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.384+0000 7feb41400640 1 -- 172.21.15.86:0/2236349912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f66743eb00 msgr2=0x55f66744aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.384+0000 7feb41400640 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f66743eb00 0x55f66744aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.384+0000 7feb41400640 1 -- 172.21.15.86:0/2236349912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 msgr2=0x55f6674434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.384+0000 7feb41400640 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 0x55f6674434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.384+0000 7feb41400640 1 -- 172.21.15.86:0/2236349912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66735c920 con 0x55f667439990 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.384+0000 7feb41400640 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667443a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb30000c00 tx=0x7feb30004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa78a1bc21a6090c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.384+0000 7feb23e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:22:38.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.385+0000 7feb23e00640 1 -- 172.21.15.86:0/2236349912 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30039040 con 0x55f667439990 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.385+0000 7feb23e00640 1 -- 172.21.15.86:0/2236349912 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feb3002c510 con 0x55f667439990 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.385+0000 7feb4bb98740 1 -- 172.21.15.86:0/2236349912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb30034df0 con 0x55f667439990 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.385+0000 7feb23e00640 1 -- 172.21.15.86:0/2236349912 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb30058050 con 0x55f667439990 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.385+0000 7feb4bb98740 10 client.26557 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.385+0000 7feb4bb98740 1 -- 172.21.15.86:0/2236349912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f66732a7f0 con 0x55f667439990 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.386+0000 7feb23e00640 1 -- 172.21.15.86:0/2236349912 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb30034df0 con 0x55f667439990 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.386+0000 7feb23e00640 1 -- 172.21.15.86:0/2236349912 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7feb30051070 con 0x55f667439990 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.386+0000 7feb4bb98740 10 client.26557 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.386+0000 7feb4bb98740 10 client.26557 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.386+0000 7feb4bb98740 20 client.26557 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.386+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f66744ebd0 0x55f667450fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.386+0000 7feb4bb98740 4 client.26557 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.386+0000 7feb4bb98740 1 -- 172.21.15.86:0/2236349912 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6673aca40 con 0x55f66744ebd0 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.386+0000 7feb41e00640 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f66744ebd0 0x55f667450fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.387+0000 7feb41e00640 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f66744ebd0 0x55f667450fc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0b437752c7f71a server_cookie=a76a1b7eb0c066fb in_seq=0 out_seq=0 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.387+0000 7feb23e00640 10 client.26557 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.387+0000 7feb23e00640 1 -- 172.21.15.86:0/2236349912 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6673aca40 con 0x55f66744ebd0 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb23e00640 10 client.26557 handle_command_reply: tid=0 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 client.26557 shutdown 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 -- 172.21.15.86:0/2236349912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 msgr2=0x55f667443a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667443a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feb30000c00 tx=0x7feb30004540 comp rx=0 tx=0).stop 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 -- 172.21.15.86:0/2236349912 shutdown_connections 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f66744ebd0 0x55f667450fc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f66743eb00 0x55f66744aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667443a30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2236349912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 0x55f6674434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 -- 172.21.15.86:0/2236349912 >> 172.21.15.86:0/2236349912 conn(0x55f667434510 msgr2=0x55f667433710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb23e00640 0 client.26557 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:22:38.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 -- 172.21.15.86:0/2236349912 shutdown_connections 2024-04-17T07:22:38.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 1 -- 172.21.15.86:0/2236349912 wait complete. 2024-04-17T07:22:38.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.388+0000 7feb4bb98740 20 client.26557 trim_cache size 0 max 16384 2024-04-17T07:22:38.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.402+0000 7feb4bb98740 1 Processor -- start 2024-04-17T07:22:38.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.402+0000 7feb4bb98740 1 -- start start 2024-04-17T07:22:38.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 0x55f66743a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667439990 0x55f66744aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 0x55f667432e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb4bb98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb441b9910 con 0x55f667438fa0 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb4bb98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb441bb450 con 0x55f667439990 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb4bb98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb441b9fe0 con 0x55f66743eb00 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb41e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 0x55f66743a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb41e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 0x55f66743a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56746/0 (socket says 172.21.15.86:56746) 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb41e00640 1 -- 172.21.15.86:0/2714338323 learned_addr learned my addr 172.21.15.86:0/2714338323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb42800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 0x55f667432e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb41400640 1 --2- 172.21.15.86:0/2714338323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667439990 0x55f66744aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb41e00640 1 -- 172.21.15.86:0/2714338323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 msgr2=0x55f667432e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb41e00640 1 --2- 172.21.15.86:0/2714338323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 0x55f667432e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb41e00640 1 -- 172.21.15.86:0/2714338323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667439990 msgr2=0x55f66744aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb41e00640 1 --2- 172.21.15.86:0/2714338323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667439990 0x55f66744aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.403+0000 7feb41e00640 1 -- 172.21.15.86:0/2714338323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6670b13f0 con 0x55f667438fa0 2024-04-17T07:22:38.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.404+0000 7feb41e00640 1 --2- 172.21.15.86:0/2714338323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 0x55f66743a220 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7feb3400cf10 tx=0x7feb340ce070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8241d0a554ccce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.404+0000 7feb2ea00640 1 -- 172.21.15.86:0/2714338323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3409c980 con 0x55f667438fa0 2024-04-17T07:22:38.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.404+0000 7feb2ea00640 1 -- 172.21.15.86:0/2714338323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb340a0d30 con 0x55f667438fa0 2024-04-17T07:22:38.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.404+0000 7feb2ea00640 1 -- 172.21.15.86:0/2714338323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb340bbb40 con 0x55f667438fa0 2024-04-17T07:22:38.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.405+0000 7feb4bb98740 1 -- 172.21.15.86:0/2714338323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 msgr2=0x55f66743a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.405+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2714338323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 0x55f66743a220 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7feb3400cf10 tx=0x7feb340ce070 comp rx=0 tx=0).stop 2024-04-17T07:22:38.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.405+0000 7feb4bb98740 1 -- 172.21.15.86:0/2714338323 shutdown_connections 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.405+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2714338323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f66743eb00 0x55f667432e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.405+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2714338323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667439990 0x55f66744aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.405+0000 7feb4bb98740 1 --2- 172.21.15.86:0/2714338323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f667438fa0 0x55f66743a220 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.405+0000 7feb4bb98740 1 -- 172.21.15.86:0/2714338323 >> 172.21.15.86:0/2714338323 conn(0x55f667434510 msgr2=0x55f66743af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.405+0000 7feb4bb98740 1 -- 172.21.15.86:0/2714338323 shutdown_connections 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.405+0000 7feb4bb98740 1 -- 172.21.15.86:0/2714338323 wait complete. 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.405+0000 7feb4bb98740 1 -- start start 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb4bb98740 1 Processor -- start 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667438fa0 0x55f667448420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667448960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb4bb98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66743eb00 0x55f6674498d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb4bb98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb441bad80 con 0x55f66743eb00 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb4bb98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb441ba6b0 con 0x55f667438fa0 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb4bb98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb441b78e0 con 0x55f667439990 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb4bb98740 10 client.0 _ll_register_callbacks cb 0x7feb300043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb42800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66743eb00 0x55f6674498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb41400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667448960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb41e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667438fa0 0x55f667448420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb41400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667448960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46462/0 (socket says 172.21.15.86:46462) 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb42800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66743eb00 0x55f6674498d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56754/0 (socket says 172.21.15.86:56754) 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.406+0000 7feb41400640 1 -- 172.21.15.86:0/1637802757 learned_addr learned my addr 172.21.15.86:0/1637802757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.407+0000 7feb41400640 1 -- 172.21.15.86:0/1637802757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667438fa0 msgr2=0x55f667448420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.407+0000 7feb41400640 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667438fa0 0x55f667448420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.407+0000 7feb41400640 1 -- 172.21.15.86:0/1637802757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66743eb00 msgr2=0x55f6674498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.407+0000 7feb41400640 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66743eb00 0x55f6674498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.407+0000 7feb41400640 1 -- 172.21.15.86:0/1637802757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66717b7d0 con 0x55f667439990 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.407+0000 7feb42800640 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66743eb00 0x55f6674498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.407+0000 7feb41400640 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667448960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feb3002c0b0 tx=0x7feb3002c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148fddf3902a19a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.407+0000 7feb23e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.408+0000 7feb23e00640 1 -- 172.21.15.86:0/1637802757 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30061070 con 0x55f667439990 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.408+0000 7feb4bb98740 1 -- 172.21.15.86:0/1637802757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb340a0d30 con 0x55f667439990 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.408+0000 7feb23e00640 1 -- 172.21.15.86:0/1637802757 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feb30034cb0 con 0x55f667439990 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.408+0000 7feb23e00640 1 -- 172.21.15.86:0/1637802757 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb3006d050 con 0x55f667439990 2024-04-17T07:22:38.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.408+0000 7feb4bb98740 10 client.26563 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:38.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.408+0000 7feb4bb98740 1 -- 172.21.15.86:0/1637802757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6674b5000 con 0x55f667439990 2024-04-17T07:22:38.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.408+0000 7feb23e00640 1 -- 172.21.15.86:0/1637802757 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb340a0d30 con 0x55f667439990 2024-04-17T07:22:38.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.409+0000 7feb23e00640 1 -- 172.21.15.86:0/1637802757 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7feb30051040 con 0x55f667439990 2024-04-17T07:22:38.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.409+0000 7feb4bb98740 10 client.26563 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:38.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.409+0000 7feb4bb98740 10 client.26563 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:22:38.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.409+0000 7feb4bb98740 20 client.26563 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:38.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.409+0000 7feb4bb98740 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f6674b5920 0x55f6674b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.409+0000 7feb4bb98740 4 client.26563 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:22:38.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.409+0000 7feb4bb98740 1 -- 172.21.15.86:0/1637802757 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6673a68b0 con 0x55f6674b5920 2024-04-17T07:22:38.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.409+0000 7feb41e00640 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f6674b5920 0x55f6674b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb41e00640 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f6674b5920 0x55f6674b7d40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72f8f726a271e2ce server_cookie=8c36500252a6c397 in_seq=0 out_seq=0 2024-04-17T07:22:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb23e00640 10 client.26563 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:22:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb23e00640 1 -- 172.21.15.86:0/1637802757 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6673a68b0 con 0x55f6674b5920 2024-04-17T07:22:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb23e00640 10 client.26563 handle_command_reply: tid=0 2024-04-17T07:22:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb4bb98740 1 client.26563 shutdown 2024-04-17T07:22:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb4bb98740 1 -- 172.21.15.86:0/1637802757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 msgr2=0x55f667448960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb4bb98740 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667448960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feb3002c0b0 tx=0x7feb3002c310 comp rx=0 tx=0).stop 2024-04-17T07:22:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb4bb98740 1 -- 172.21.15.86:0/1637802757 shutdown_connections 2024-04-17T07:22:38.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb4bb98740 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f66743eb00 0x55f6674498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb4bb98740 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f667439990 0x55f667448960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb4bb98740 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55f6674b5920 0x55f6674b7d40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb4bb98740 1 --2- 172.21.15.86:0/1637802757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f667438fa0 0x55f667448420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb23e00640 0 client.26563 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:22:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.410+0000 7feb4bb98740 1 -- 172.21.15.86:0/1637802757 >> 172.21.15.86:0/1637802757 conn(0x55f667434510 msgr2=0x55f667437770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.411+0000 7feb4bb98740 1 -- 172.21.15.86:0/1637802757 shutdown_connections 2024-04-17T07:22:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.411+0000 7feb4bb98740 1 -- 172.21.15.86:0/1637802757 wait complete. 2024-04-17T07:22:38.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.411+0000 7feb4bb98740 20 client.26563 trim_cache size 0 max 16384 2024-04-17T07:22:38.416 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:22:38.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.411+0000 7feb48e00640 1 -- 172.21.15.86:0/2279390196 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb280a3d00 msgr2=0x7feb280a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.411+0000 7feb48e00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb280a3d00 0x7feb280a61c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55f66743a1f0 tx=0x7feb3c0095c0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.411+0000 7feb48e00640 1 -- 172.21.15.86:0/2279390196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb44108f60 msgr2=0x7feb4410bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.411+0000 7feb48e00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb44108f60 0x7feb4410bba0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e9f0 tx=0x7feb3400eeb0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.412+0000 7feb48e00640 1 -- 172.21.15.86:0/2279390196 shutdown_connections 2024-04-17T07:22:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.412+0000 7feb48e00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb280a3d00 0x7feb280a61c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.412+0000 7feb48e00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb441107e0 0x7feb4410a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.412+0000 7feb48e00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb441098c0 0x7feb4410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.412+0000 7feb48e00640 1 --2- 172.21.15.86:0/2279390196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb44108f60 0x7feb4410bba0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.412+0000 7feb48e00640 1 -- 172.21.15.86:0/2279390196 >> 172.21.15.86:0/2279390196 conn(0x7feb4406f190 msgr2=0x7feb4410c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.412+0000 7feb48e00640 1 -- 172.21.15.86:0/2279390196 shutdown_connections 2024-04-17T07:22:38.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.412+0000 7feb48e00640 1 -- 172.21.15.86:0/2279390196 wait complete. 2024-04-17T07:22:38.428 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:22:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.628+0000 7fc8aac00640 1 -- 172.21.15.86:0/143107427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4105c50 msgr2=0x7fc8a410c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.628+0000 7fc8aac00640 1 --2- 172.21.15.86:0/143107427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4105c50 0x7fc8a410c4e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009ca0 tx=0x7fc89c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.628+0000 7fc8aac00640 1 -- 172.21.15.86:0/143107427 shutdown_connections 2024-04-17T07:22:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.628+0000 7fc8aac00640 1 --2- 172.21.15.86:0/143107427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4105c50 0x7fc8a410c4e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.628+0000 7fc8aac00640 1 --2- 172.21.15.86:0/143107427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8a4105290 0x7fc8a4105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.628+0000 7fc8aac00640 1 --2- 172.21.15.86:0/143107427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a41040a0 0x7fc8a41044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.628+0000 7fc8aac00640 1 -- 172.21.15.86:0/143107427 >> 172.21.15.86:0/143107427 conn(0x7fc8a40ff610 msgr2=0x7fc8a4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.628+0000 7fc8aac00640 1 -- 172.21.15.86:0/143107427 shutdown_connections 2024-04-17T07:22:38.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.628+0000 7fc8aac00640 1 -- 172.21.15.86:0/143107427 wait complete. 2024-04-17T07:22:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.629+0000 7fc8aac00640 1 Processor -- start 2024-04-17T07:22:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8aac00640 1 -- start start 2024-04-17T07:22:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8aac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a41040a0 0x7fc8a419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8aac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a4105290 0x7fc8a419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8aac00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8a4105c50 0x7fc8a41a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8aac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a410fbe0 con 0x7fc8a41040a0 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8aac00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a410fa60 con 0x7fc8a4105c50 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8aac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a410fd60 con 0x7fc8a4105290 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8a9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a41040a0 0x7fc8a419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8a9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a41040a0 0x7fc8a419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56758/0 (socket says 172.21.15.86:56758) 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8a9800640 1 -- 172.21.15.86:0/1591389621 learned_addr learned my addr 172.21.15.86:0/1591389621 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8a8e00640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a4105290 0x7fc8a419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.630+0000 7fc8aa200640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8a4105c50 0x7fc8a41a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8a9800640 1 -- 172.21.15.86:0/1591389621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a4105290 msgr2=0x7fc8a419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8a9800640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a4105290 0x7fc8a419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8a9800640 1 -- 172.21.15.86:0/1591389621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8a4105c50 msgr2=0x7fc8a41a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8a9800640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8a4105c50 0x7fc8a41a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8a9800640 1 -- 172.21.15.86:0/1591389621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a41a1da0 con 0x7fc8a41040a0 2024-04-17T07:22:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8aa200640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8a4105c50 0x7fc8a41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8a8e00640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a4105290 0x7fc8a419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8a9800640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a41040a0 0x7fc8a419a180 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc89800c970 tx=0x7fc89800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8a2000640 1 -- 172.21.15.86:0/1591389621 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc898004220 con 0x7fc8a41040a0 2024-04-17T07:22:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8a2000640 1 -- 172.21.15.86:0/1591389621 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8980043c0 con 0x7fc8a41040a0 2024-04-17T07:22:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8aac00640 1 -- 172.21.15.86:0/1591389621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a41a2090 con 0x7fc8a41040a0 2024-04-17T07:22:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8a2000640 1 -- 172.21.15.86:0/1591389621 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89800f450 con 0x7fc8a41040a0 2024-04-17T07:22:38.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.631+0000 7fc8aac00640 1 -- 172.21.15.86:0/1591389621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a41a2630 con 0x7fc8a41040a0 2024-04-17T07:22:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.635+0000 7fc8a2000640 1 -- 172.21.15.86:0/1591389621 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc8980079b0 con 0x7fc8a41040a0 2024-04-17T07:22:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.635+0000 7fc8aefa7740 1 Processor -- start 2024-04-17T07:22:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.635+0000 7fc8aefa7740 1 -- start start 2024-04-17T07:22:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.635+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354763580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.635+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354763b50 0x561354768650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.635+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354768cc0 0x56135476b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.635+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5613546c9020 con 0x561354763160 2024-04-17T07:22:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.635+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56135466f460 con 0x561354768cc0 2024-04-17T07:22:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.635+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5613546c4fe0 con 0x561354763b50 2024-04-17T07:22:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.636+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354763580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.636+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354763580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56766/0 (socket says 172.21.15.86:56766) 2024-04-17T07:22:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.636+0000 7fc8aa200640 1 -- 172.21.15.86:0/661522455 learned_addr learned my addr 172.21.15.86:0/661522455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.636+0000 7fc8a9800640 1 --2- 172.21.15.86:0/661522455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354763b50 0x561354768650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.636+0000 7fc8a8e00640 1 --2- 172.21.15.86:0/661522455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354768cc0 0x56135476b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.636+0000 7fc8aa200640 1 -- 172.21.15.86:0/661522455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354763b50 msgr2=0x561354768650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.636+0000 7fc8aa200640 1 --2- 172.21.15.86:0/661522455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354763b50 0x561354768650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.636+0000 7fc8aa200640 1 -- 172.21.15.86:0/661522455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354768cc0 msgr2=0x56135476b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.636+0000 7fc8aa200640 1 --2- 172.21.15.86:0/661522455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354768cc0 0x56135476b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.636+0000 7fc8aa200640 1 -- 172.21.15.86:0/661522455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561354677040 con 0x561354763160 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.637+0000 7fc8a2000640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc88c0a3c30 0x7fc88c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.637+0000 7fc8a2000640 1 -- 172.21.15.86:0/1591389621 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc8980d05a0 con 0x7fc8a41040a0 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.637+0000 7fc8aa200640 1 --2- 172.21.15.86:0/661522455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354763580 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc89c037040 tx=0x7fc89c009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c711d452aad0f408 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.637+0000 7fc8a9800640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc88c0a3c30 0x7fc88c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.637+0000 7fc896a00640 1 -- 172.21.15.86:0/661522455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c039050 con 0x561354763160 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.637+0000 7fc896a00640 1 -- 172.21.15.86:0/661522455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc89c045070 con 0x561354763160 2024-04-17T07:22:38.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.637+0000 7fc896a00640 1 -- 172.21.15.86:0/661522455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c004350 con 0x561354763160 2024-04-17T07:22:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.638+0000 7fc8aefa7740 1 -- 172.21.15.86:0/661522455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 msgr2=0x561354763580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.638+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/661522455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354763580 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc89c037040 tx=0x7fc89c009950 comp rx=0 tx=0).stop 2024-04-17T07:22:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.638+0000 7fc8a9800640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc88c0a3c30 0x7fc88c0a60f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc89809c430 tx=0x7fc898004570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.638+0000 7fc8aefa7740 1 -- 172.21.15.86:0/661522455 shutdown_connections 2024-04-17T07:22:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.638+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/661522455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354768cc0 0x56135476b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.638+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/661522455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354763b50 0x561354768650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.638+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/661522455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354763580 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.638+0000 7fc8aefa7740 1 -- 172.21.15.86:0/661522455 >> 172.21.15.86:0/661522455 conn(0x56135475e6d0 msgr2=0x56135475ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.638+0000 7fc8aefa7740 1 -- 172.21.15.86:0/661522455 shutdown_connections 2024-04-17T07:22:38.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.638+0000 7fc8aefa7740 1 -- 172.21.15.86:0/661522455 wait complete. 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.639+0000 7fc8aefa7740 1 -- start start 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.639+0000 7fc8aefa7740 1 Processor -- start 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x56135476d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x56135476dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354768cc0 0x561354774c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56135476bc10 con 0x561354763160 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561354682830 con 0x561354763b50 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56135476bd90 con 0x561354768cc0 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aefa7740 10 client.0 _ll_register_callbacks cb 0x561354650fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x56135476d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x56135476d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56770/0 (socket says 172.21.15.86:56770) 2024-04-17T07:22:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aa200640 1 -- 172.21.15.86:0/2658937626 learned_addr learned my addr 172.21.15.86:0/2658937626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8a8e00640 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x56135476dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aa200640 1 -- 172.21.15.86:0/2658937626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354768cc0 msgr2=0x561354774c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aa200640 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354768cc0 0x561354774c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aa200640 1 -- 172.21.15.86:0/2658937626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 msgr2=0x56135476dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aa200640 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x56135476dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.640+0000 7fc8aa200640 1 -- 172.21.15.86:0/2658937626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561354686920 con 0x561354763160 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.641+0000 7fc8aa200640 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x56135476d6d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc89c032fc0 tx=0x7fc89c003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac63f6e17c3d7fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.641+0000 7fc88b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.641+0000 7fc88b400640 1 -- 172.21.15.86:0/2658937626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c050070 con 0x561354763160 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.641+0000 7fc88b400640 1 -- 172.21.15.86:0/2658937626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc89c0042b0 con 0x561354763160 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.641+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2658937626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc89c045070 con 0x561354763160 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc88b400640 1 -- 172.21.15.86:0/2658937626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc89c0044b0 con 0x561354763160 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc8aefa7740 10 client.17592 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2658937626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613546547f0 con 0x561354763160 2024-04-17T07:22:38.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc88b400640 1 -- 172.21.15.86:0/2658937626 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc89c045070 con 0x561354763160 2024-04-17T07:22:38.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc88b400640 1 -- 172.21.15.86:0/2658937626 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc89c040cd0 con 0x561354763160 2024-04-17T07:22:38.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc8aefa7740 10 client.17592 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc8aefa7740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc8aefa7740 20 client.17592 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561354778db0 0x56135477b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc8aefa7740 4 client.17592 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.642+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2658937626 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613546d6a40 con 0x561354778db0 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.643+0000 7fc8a8e00640 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561354778db0 0x56135477b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.644+0000 7fc8a8e00640 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561354778db0 0x56135477b1a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c8a5358e8c75b6 server_cookie=c90d0823c8f1a1e in_seq=0 out_seq=0 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.644+0000 7fc88b400640 10 client.17592 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc88b400640 1 -- 172.21.15.86:0/2658937626 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613546d6a40 con 0x561354778db0 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc88b400640 10 client.17592 handle_command_reply: tid=0 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 client.17592 shutdown 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2658937626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 msgr2=0x56135476d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x56135476d6d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc89c032fc0 tx=0x7fc89c003d70 comp rx=0 tx=0).stop 2024-04-17T07:22:38.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2658937626 shutdown_connections 2024-04-17T07:22:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561354778db0 0x56135477b1a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354768cc0 0x561354774c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x56135476dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2658937626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x56135476d6d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2658937626 >> 172.21.15.86:0/2658937626 conn(0x56135475e6d0 msgr2=0x56135475d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc88b400640 0 client.17592 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2658937626 shutdown_connections 2024-04-17T07:22:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2658937626 wait complete. 2024-04-17T07:22:38.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.646+0000 7fc8aefa7740 20 client.17592 trim_cache size 0 max 16384 2024-04-17T07:22:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.665+0000 7fc8aefa7740 1 Processor -- start 2024-04-17T07:22:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.665+0000 7fc8aefa7740 1 -- start start 2024-04-17T07:22:38.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.665+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354763160 0x56135476db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.665+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x561354774be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.665+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354768cc0 0x561354762040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41ac9b0 con 0x561354768cc0 2024-04-17T07:22:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41ae4f0 con 0x561354763b50 2024-04-17T07:22:38.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41ad080 con 0x561354763160 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354768cc0 0x561354762040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354768cc0 0x561354762040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56782/0 (socket says 172.21.15.86:56782) 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aa200640 1 -- 172.21.15.86:0/836997082 learned_addr learned my addr 172.21.15.86:0/836997082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aa200640 1 --2- 172.21.15.86:0/836997082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354763160 0x56135476db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8a8e00640 1 --2- 172.21.15.86:0/836997082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x561354774be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aa200640 1 -- 172.21.15.86:0/836997082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354763160 msgr2=0x56135476db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aa200640 1 --2- 172.21.15.86:0/836997082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354763160 0x56135476db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aa200640 1 -- 172.21.15.86:0/836997082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 msgr2=0x561354774be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aa200640 1 --2- 172.21.15.86:0/836997082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x561354774be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.666+0000 7fc8aa200640 1 -- 172.21.15.86:0/836997082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613543db3f0 con 0x561354768cc0 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.667+0000 7fc8aa200640 1 --2- 172.21.15.86:0/836997082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354768cc0 0x561354762040 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc89c02c6d0 tx=0x7fc89c037040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57786ba2e7e4e670 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.667+0000 7fc896000640 1 -- 172.21.15.86:0/836997082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c039070 con 0x561354768cc0 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.667+0000 7fc896000640 1 -- 172.21.15.86:0/836997082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc89c045070 con 0x561354768cc0 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.667+0000 7fc896000640 1 -- 172.21.15.86:0/836997082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c050070 con 0x561354768cc0 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 -- 172.21.15.86:0/836997082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354768cc0 msgr2=0x561354762040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/836997082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354768cc0 0x561354762040 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc89c02c6d0 tx=0x7fc89c037040 comp rx=0 tx=0).stop 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 -- 172.21.15.86:0/836997082 shutdown_connections 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/836997082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354768cc0 0x561354762040 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/836997082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x561354774be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/836997082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354763160 0x56135476db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 -- 172.21.15.86:0/836997082 >> 172.21.15.86:0/836997082 conn(0x56135475e6d0 msgr2=0x561354761930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 -- 172.21.15.86:0/836997082 shutdown_connections 2024-04-17T07:22:38.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 -- 172.21.15.86:0/836997082 wait complete. 2024-04-17T07:22:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 -- start start 2024-04-17T07:22:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.668+0000 7fc8aefa7740 1 Processor -- start 2024-04-17T07:22:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354771ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x561354772420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aefa7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354768cc0 0x561354773430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41ade20 con 0x561354763160 2024-04-17T07:22:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41ad750 con 0x561354763b50 2024-04-17T07:22:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aefa7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41aa980 con 0x561354768cc0 2024-04-17T07:22:38.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aefa7740 10 client.0 _ll_register_callbacks cb 0x7fc89c003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354771ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354771ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56788/0 (socket says 172.21.15.86:56788) 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aa200640 1 -- 172.21.15.86:0/2764414292 learned_addr learned my addr 172.21.15.86:0/2764414292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8a8e00640 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x561354772420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aa200640 1 -- 172.21.15.86:0/2764414292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354768cc0 msgr2=0x561354773430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aa200640 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354768cc0 0x561354773430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.669+0000 7fc8aa200640 1 -- 172.21.15.86:0/2764414292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 msgr2=0x561354772420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.670+0000 7fc8aa200640 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x561354772420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.670+0000 7fc8aa200640 1 -- 172.21.15.86:0/2764414292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613544a57d0 con 0x561354763160 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.670+0000 7fc8aa200640 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354771ee0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc89c02c630 tx=0x7fc89c032980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793e074a49aaffa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.670+0000 7fc88b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.670+0000 7fc88b400640 1 -- 172.21.15.86:0/2764414292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c039040 con 0x561354763160 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.670+0000 7fc88b400640 1 -- 172.21.15.86:0/2764414292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc89c004d40 con 0x561354763160 2024-04-17T07:22:38.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.670+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2764414292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc89c045070 con 0x561354763160 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.670+0000 7fc88b400640 1 -- 172.21.15.86:0/2764414292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc89c050040 con 0x561354763160 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.671+0000 7fc8aefa7740 10 client.17598 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.671+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2764414292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561354774b50 con 0x561354763160 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.671+0000 7fc88b400640 1 -- 172.21.15.86:0/2764414292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc89c045070 con 0x561354763160 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.671+0000 7fc88b400640 1 -- 172.21.15.86:0/2764414292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc89c040ab0 con 0x561354763160 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.671+0000 7fc8aefa7740 10 client.17598 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.671+0000 7fc8aefa7740 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.671+0000 7fc8aefa7740 20 client.17598 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.671+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5613547df210 0x5613547e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.671+0000 7fc8aefa7740 4 client.17598 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.671+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2764414292 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613546d08b0 con 0x5613547df210 2024-04-17T07:22:38.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.672+0000 7fc8a8e00640 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5613547df210 0x5613547e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:38.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.673+0000 7fc88b400640 10 client.17598 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.673+0000 7fc8a8e00640 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5613547df210 0x5613547e1600 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6a6d0de891dfa8b server_cookie=b9a090fcd29e2bbb in_seq=0 out_seq=0 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.673+0000 7fc88b400640 1 -- 172.21.15.86:0/2764414292 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613546d08b0 con 0x5613547df210 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.673+0000 7fc88b400640 10 client.17598 handle_command_reply: tid=0 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.673+0000 7fc8aefa7740 1 client.17598 shutdown 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.673+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2764414292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 msgr2=0x561354771ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.673+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354771ee0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc89c02c630 tx=0x7fc89c032980 comp rx=0 tx=0).stop 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.674+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2764414292 shutdown_connections 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.674+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5613547df210 0x5613547e1600 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.674+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561354768cc0 0x561354773430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.674+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561354763b50 0x561354772420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.674+0000 7fc8aefa7740 1 --2- 172.21.15.86:0/2764414292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561354763160 0x561354771ee0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.674+0000 7fc88b400640 0 client.17598 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:38.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.674+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2764414292 >> 172.21.15.86:0/2764414292 conn(0x56135475e6d0 msgr2=0x56135476d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.674+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2764414292 shutdown_connections 2024-04-17T07:22:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.674+0000 7fc8aefa7740 1 -- 172.21.15.86:0/2764414292 wait complete. 2024-04-17T07:22:38.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.674+0000 7fc8aefa7740 20 client.17598 trim_cache size 0 max 16384 2024-04-17T07:22:38.677 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:22:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 -- 172.21.15.86:0/1591389621 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc88c0a3c30 msgr2=0x7fc88c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc88c0a3c30 0x7fc88c0a60f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc89809c430 tx=0x7fc898004570 comp rx=0 tx=0).stop 2024-04-17T07:22:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 -- 172.21.15.86:0/1591389621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a41040a0 msgr2=0x7fc8a419a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a41040a0 0x7fc8a419a180 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc89800c970 tx=0x7fc89800ce30 comp rx=0 tx=0).stop 2024-04-17T07:22:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 -- 172.21.15.86:0/1591389621 shutdown_connections 2024-04-17T07:22:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc88c0a3c30 0x7fc88c0a60f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8a4105c50 0x7fc8a41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a4105290 0x7fc8a419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 --2- 172.21.15.86:0/1591389621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a41040a0 0x7fc8a419a180 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 -- 172.21.15.86:0/1591389621 >> 172.21.15.86:0/1591389621 conn(0x7fc8a40ff610 msgr2=0x7fc8a4108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 -- 172.21.15.86:0/1591389621 shutdown_connections 2024-04-17T07:22:38.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:38.675+0000 7fc8aac00640 1 -- 172.21.15.86:0/1591389621 wait complete. 2024-04-17T07:22:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 195.0 s, 520.6 tps, lat 193.690 ms stddev 259.198 2024-04-17T07:22:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:39 smithi098 ceph-mon[35418]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-04-17T07:22:39.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:39 smithi103 ceph-mon[35495]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-04-17T07:22:39.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:39 smithi086 ceph-mon[31733]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-04-17T07:22:39.700 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:22:39.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:22:39.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:22:39.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.979+0000 7fb4a8c00640 1 -- 172.21.15.86:0/4152636516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a4101ff0 msgr2=0x7fb4a4102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:39.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.979+0000 7fb4a8c00640 1 --2- 172.21.15.86:0/4152636516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a4101ff0 0x7fb4a4102470 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb490009860 tx=0x7fb49002bb70 comp rx=0 tx=0).stop 2024-04-17T07:22:39.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.980+0000 7fb4a8c00640 1 -- 172.21.15.86:0/4152636516 shutdown_connections 2024-04-17T07:22:39.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.980+0000 7fb4a8c00640 1 --2- 172.21.15.86:0/4152636516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4a41029b0 0x7fb4a40690f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:39.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.980+0000 7fb4a8c00640 1 --2- 172.21.15.86:0/4152636516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a4101ff0 0x7fb4a4102470 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:39.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.980+0000 7fb4a8c00640 1 --2- 172.21.15.86:0/4152636516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4a4100ea0 0x7fb4a41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:39.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.980+0000 7fb4a8c00640 1 -- 172.21.15.86:0/4152636516 >> 172.21.15.86:0/4152636516 conn(0x7fb4a40fc410 msgr2=0x7fb4a40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:39.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.980+0000 7fb4a8c00640 1 -- 172.21.15.86:0/4152636516 shutdown_connections 2024-04-17T07:22:39.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.982+0000 7fb4a8c00640 1 -- 172.21.15.86:0/4152636516 wait complete. 2024-04-17T07:22:39.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.984+0000 7fb4a8c00640 1 Processor -- start 2024-04-17T07:22:39.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.985+0000 7fb4a8c00640 1 -- start start 2024-04-17T07:22:39.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.985+0000 7fb4a8c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4a4100ea0 0x7fb4a419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:39.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.985+0000 7fb4a8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a4101ff0 0x7fb4a419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:39.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.985+0000 7fb4a8c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4a41029b0 0x7fb4a41a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:39.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.985+0000 7fb4a8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a410fd30 con 0x7fb4a4101ff0 2024-04-17T07:22:39.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.985+0000 7fb4a8c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a410fbb0 con 0x7fb4a4100ea0 2024-04-17T07:22:39.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.985+0000 7fb4a8c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a410feb0 con 0x7fb4a41029b0 2024-04-17T07:22:39.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.986+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a4101ff0 0x7fb4a419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:39.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.986+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4a4100ea0 0x7fb4a419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:39.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.986+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4a4100ea0 0x7fb4a419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53672/0 (socket says 172.21.15.86:53672) 2024-04-17T07:22:39.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.986+0000 7fb4a1e00640 1 -- 172.21.15.86:0/1162551229 learned_addr learned my addr 172.21.15.86:0/1162551229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:39.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.986+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a4101ff0 0x7fb4a419a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56804/0 (socket says 172.21.15.86:56804) 2024-04-17T07:22:39.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.986+0000 7fb4a2800640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4a41029b0 0x7fb4a41a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:39.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.987+0000 7fb4a1400640 1 -- 172.21.15.86:0/1162551229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4a41029b0 msgr2=0x7fb4a41a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:39.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.987+0000 7fb4a1400640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4a41029b0 0x7fb4a41a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:39.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.987+0000 7fb4a1400640 1 -- 172.21.15.86:0/1162551229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4a4100ea0 msgr2=0x7fb4a419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:39.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.987+0000 7fb4a1400640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4a4100ea0 0x7fb4a419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:39.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.987+0000 7fb4a1400640 1 -- 172.21.15.86:0/1162551229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a406ba00 con 0x7fb4a4101ff0 2024-04-17T07:22:39.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.987+0000 7fb4a1e00640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4a4100ea0 0x7fb4a419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:39.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.988+0000 7fb4a1400640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a4101ff0 0x7fb4a419a8f0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb49002c0b0 tx=0x7fb4900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:39.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.989+0000 7fb49aa00640 1 -- 172.21.15.86:0/1162551229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490041020 con 0x7fb4a4101ff0 2024-04-17T07:22:39.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.989+0000 7fb49aa00640 1 -- 172.21.15.86:0/1162551229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4900035f0 con 0x7fb4a4101ff0 2024-04-17T07:22:39.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.989+0000 7fb49aa00640 1 -- 172.21.15.86:0/1162551229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490039040 con 0x7fb4a4101ff0 2024-04-17T07:22:39.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.989+0000 7fb4a8c00640 1 -- 172.21.15.86:0/1162551229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a406bc90 con 0x7fb4a4101ff0 2024-04-17T07:22:39.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.989+0000 7fb4a8c00640 1 -- 172.21.15.86:0/1162551229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a406c130 con 0x7fb4a4101ff0 2024-04-17T07:22:39.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.994+0000 7fb49aa00640 1 -- 172.21.15.86:0/1162551229 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb490044020 con 0x7fb4a4101ff0 2024-04-17T07:22:39.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.995+0000 7fb4ab969740 1 Processor -- start 2024-04-17T07:22:39.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.995+0000 7fb4ab969740 1 -- start start 2024-04-17T07:22:39.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.996+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 0x55c031343af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:39.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.996+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313440c0 0x55c031348bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:39.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.996+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c031349230 0x55c03134b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:39.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.996+0000 7fb4ab969740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c0312a0cb0 con 0x55c0313436d0 2024-04-17T07:22:39.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.996+0000 7fb4a2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 0x55c031343af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:39.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.996+0000 7fb4ab969740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c031247370 con 0x55c0313440c0 2024-04-17T07:22:39.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.996+0000 7fb4ab969740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c03129cc70 con 0x55c031349230 2024-04-17T07:22:39.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.996+0000 7fb4a2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 0x55c031343af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56820/0 (socket says 172.21.15.86:56820) 2024-04-17T07:22:39.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.996+0000 7fb4a2800640 1 -- 172.21.15.86:0/462764414 learned_addr learned my addr 172.21.15.86:0/462764414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:39.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.997+0000 7fb49aa00640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4880a3d00 0x7fb4880a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.997+0000 7fb4a1e00640 1 --2- 172.21.15.86:0/462764414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313440c0 0x55c031348bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.997+0000 7fb49aa00640 1 -- 172.21.15.86:0/1162551229 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb4900f2060 con 0x7fb4a4101ff0 2024-04-17T07:22:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.997+0000 7fb4a2800640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4880a3d00 0x7fb4880a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.997+0000 7fb4a1400640 1 --2- 172.21.15.86:0/462764414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c031349230 0x55c03134b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.998+0000 7fb4a1400640 1 -- 172.21.15.86:0/462764414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313440c0 msgr2=0x55c031348bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.998+0000 7fb4a1400640 1 --2- 172.21.15.86:0/462764414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313440c0 0x55c031348bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.998+0000 7fb4a1400640 1 -- 172.21.15.86:0/462764414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 msgr2=0x55c031343af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.998+0000 7fb4a1400640 1 --2- 172.21.15.86:0/462764414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 0x55c031343af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.998+0000 7fb4a1400640 1 -- 172.21.15.86:0/462764414 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c031266910 con 0x55c031349230 2024-04-17T07:22:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.999+0000 7fb4a1400640 1 --2- 172.21.15.86:0/462764414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c031349230 0x55c03134b6c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4900ef830 tx=0x7fb4900bde50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=529456dec75694c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.999+0000 7fb48f000640 1 -- 172.21.15.86:0/462764414 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4900d7070 con 0x55c031349230 2024-04-17T07:22:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.999+0000 7fb48f000640 1 -- 172.21.15.86:0/462764414 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4900de040 con 0x55c031349230 2024-04-17T07:22:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:39.999+0000 7fb48f000640 1 -- 172.21.15.86:0/462764414 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4900ba630 con 0x55c031349230 2024-04-17T07:22:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.000+0000 7fb4ab969740 1 -- 172.21.15.86:0/462764414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c031349230 msgr2=0x55c03134b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.000+0000 7fb4a2800640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4880a3d00 0x7fb4880a61c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55c031344920 tx=0x7fb49c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.000+0000 7fb4ab969740 1 --2- 172.21.15.86:0/462764414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c031349230 0x55c03134b6c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4900ef830 tx=0x7fb4900bde50 comp rx=0 tx=0).stop 2024-04-17T07:22:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.001+0000 7fb4ab969740 1 -- 172.21.15.86:0/462764414 shutdown_connections 2024-04-17T07:22:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.001+0000 7fb4ab969740 1 --2- 172.21.15.86:0/462764414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c031349230 0x55c03134b6c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.001+0000 7fb4ab969740 1 --2- 172.21.15.86:0/462764414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313440c0 0x55c031348bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.001+0000 7fb4ab969740 1 --2- 172.21.15.86:0/462764414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 0x55c031343af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.001+0000 7fb4ab969740 1 -- 172.21.15.86:0/462764414 >> 172.21.15.86:0/462764414 conn(0x55c03133ec40 msgr2=0x55c03133f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.002+0000 7fb4ab969740 1 -- 172.21.15.86:0/462764414 shutdown_connections 2024-04-17T07:22:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.002+0000 7fb4ab969740 1 -- 172.21.15.86:0/462764414 wait complete. 2024-04-17T07:22:40.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.003+0000 7fb4ab969740 1 -- start start 2024-04-17T07:22:40.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.003+0000 7fb4ab969740 1 Processor -- start 2024-04-17T07:22:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.004+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313436d0 0x55c03134d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.004+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313440c0 0x55c03134de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.004+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031354ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.004+0000 7fb4ab969740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c03134bfa0 con 0x55c031349230 2024-04-17T07:22:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.004+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313440c0 0x55c03134de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.004+0000 7fb4ab969740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c03125a730 con 0x55c0313440c0 2024-04-17T07:22:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.004+0000 7fb4ab969740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c03134c120 con 0x55c0313436d0 2024-04-17T07:22:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.004+0000 7fb4a2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031354ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.004+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313436d0 0x55c03134d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.005+0000 7fb4a2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031354ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56834/0 (socket says 172.21.15.86:56834) 2024-04-17T07:22:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.005+0000 7fb4a2800640 1 -- 172.21.15.86:0/1255875339 learned_addr learned my addr 172.21.15.86:0/1255875339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.005+0000 7fb4ab969740 10 client.0 _ll_register_callbacks cb 0x55c03125cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.005+0000 7fb4a2800640 1 -- 172.21.15.86:0/1255875339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313436d0 msgr2=0x55c03134d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.005+0000 7fb4a2800640 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313436d0 0x55c03134d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.005+0000 7fb4a2800640 1 -- 172.21.15.86:0/1255875339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313440c0 msgr2=0x55c03134de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.005+0000 7fb4a2800640 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313440c0 0x55c03134de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.005+0000 7fb4a2800640 1 -- 172.21.15.86:0/1255875339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c03107b7e0 con 0x55c031349230 2024-04-17T07:22:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.005+0000 7fb4a1e00640 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313436d0 0x55c03134d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:22:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.006+0000 7fb4a2800640 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031354ec0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb49c003ed0 tx=0x7fb49c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e583795c9f430ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.006+0000 7fb483400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.006+0000 7fb483400640 1 -- 172.21.15.86:0/1255875339 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c025070 con 0x55c031349230 2024-04-17T07:22:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.006+0000 7fb483400640 1 -- 172.21.15.86:0/1255875339 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb49c004d20 con 0x55c031349230 2024-04-17T07:22:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.006+0000 7fb4ab969740 1 -- 172.21.15.86:0/1255875339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4900de040 con 0x55c031349230 2024-04-17T07:22:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.007+0000 7fb483400640 1 -- 172.21.15.86:0/1255875339 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb49c030050 con 0x55c031349230 2024-04-17T07:22:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.007+0000 7fb4ab969740 10 client.17610 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.007+0000 7fb4ab969740 1 -- 172.21.15.86:0/1255875339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c0310dbf70 con 0x55c031349230 2024-04-17T07:22:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.007+0000 7fb483400640 1 -- 172.21.15.86:0/1255875339 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4900de040 con 0x55c031349230 2024-04-17T07:22:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.007+0000 7fb483400640 1 -- 172.21.15.86:0/1255875339 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb49c01e050 con 0x55c031349230 2024-04-17T07:22:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.008+0000 7fb4ab969740 10 client.17610 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.008+0000 7fb4ab969740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.008+0000 7fb4ab969740 20 client.17610 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.008+0000 7fb4ab969740 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c031359740 0x55c03135bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.008+0000 7fb4ab969740 4 client.17610 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.008+0000 7fb4ab969740 1 -- 172.21.15.86:0/1255875339 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0312ae6d0 con 0x55c031359740 2024-04-17T07:22:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.008+0000 7fb4a1e00640 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c031359740 0x55c03135bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.009+0000 7fb4a1e00640 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c031359740 0x55c03135bbb0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d645476263917640 server_cookie=e8160d3e5f2c9d0 in_seq=0 out_seq=0 2024-04-17T07:22:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.009+0000 7fb483400640 10 client.17610 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.010+0000 7fb483400640 1 -- 172.21.15.86:0/1255875339 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0312ae6d0 con 0x55c031359740 2024-04-17T07:22:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.010+0000 7fb483400640 10 client.17610 handle_command_reply: tid=0 2024-04-17T07:22:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.010+0000 7fb4ab969740 1 client.17610 shutdown 2024-04-17T07:22:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.011+0000 7fb4ab969740 1 -- 172.21.15.86:0/1255875339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 msgr2=0x55c031354ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.011+0000 7fb4ab969740 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031354ec0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb49c003ed0 tx=0x7fb49c004a40 comp rx=0 tx=0).stop 2024-04-17T07:22:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.011+0000 7fb4ab969740 1 -- 172.21.15.86:0/1255875339 shutdown_connections 2024-04-17T07:22:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.011+0000 7fb4ab969740 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c031359740 0x55c03135bbb0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.011+0000 7fb4ab969740 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031354ec0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.012+0000 7fb4ab969740 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313440c0 0x55c03134de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.012+0000 7fb4ab969740 1 --2- 172.21.15.86:0/1255875339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313436d0 0x55c03134d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.012+0000 7fb4ab969740 1 -- 172.21.15.86:0/1255875339 >> 172.21.15.86:0/1255875339 conn(0x55c03133ec40 msgr2=0x55c0313355e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.012+0000 7fb483400640 0 client.17610 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.012+0000 7fb4ab969740 1 -- 172.21.15.86:0/1255875339 shutdown_connections 2024-04-17T07:22:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.012+0000 7fb4ab969740 1 -- 172.21.15.86:0/1255875339 wait complete. 2024-04-17T07:22:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.013+0000 7fb4ab969740 20 client.17610 trim_cache size 0 max 16384 2024-04-17T07:22:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.058+0000 7fb4ab969740 1 Processor -- start 2024-04-17T07:22:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.058+0000 7fb4ab969740 1 -- start start 2024-04-17T07:22:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.058+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313436d0 0x55c031334d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.058+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 0x55c031354de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.058+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031335290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.058+0000 7fb4ab969740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4072230 con 0x55c031349230 2024-04-17T07:22:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.058+0000 7fb4ab969740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4073d70 con 0x55c0313436d0 2024-04-17T07:22:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.058+0000 7fb4ab969740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4072900 con 0x55c0313440c0 2024-04-17T07:22:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.059+0000 7fb4a2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031335290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.059+0000 7fb4a2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031335290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56838/0 (socket says 172.21.15.86:56838) 2024-04-17T07:22:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.059+0000 7fb4a2800640 1 -- 172.21.15.86:0/2923678050 learned_addr learned my addr 172.21.15.86:0/2923678050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.059+0000 7fb4a2800640 1 -- 172.21.15.86:0/2923678050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 msgr2=0x55c031354de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.059+0000 7fb4a1400640 1 --2- 172.21.15.86:0/2923678050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 0x55c031354de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.059+0000 7fb4a2800640 1 --2- 172.21.15.86:0/2923678050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 0x55c031354de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.059+0000 7fb4a2800640 1 -- 172.21.15.86:0/2923678050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313436d0 msgr2=0x55c031334d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.059+0000 7fb4a2800640 1 --2- 172.21.15.86:0/2923678050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313436d0 0x55c031334d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.059+0000 7fb4a2800640 1 -- 172.21.15.86:0/2923678050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c030f73170 con 0x55c031349230 2024-04-17T07:22:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.060+0000 7fb4a1400640 1 --2- 172.21.15.86:0/2923678050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 0x55c031354de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:22:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.060+0000 7fb4a2800640 1 --2- 172.21.15.86:0/2923678050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031335290 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb49c000f80 tx=0x7fb49c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b081282d1cf74c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.061+0000 7fb48e600640 1 -- 172.21.15.86:0/2923678050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c025070 con 0x55c031349230 2024-04-17T07:22:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.061+0000 7fb48e600640 1 -- 172.21.15.86:0/2923678050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb49c019040 con 0x55c031349230 2024-04-17T07:22:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.061+0000 7fb48e600640 1 -- 172.21.15.86:0/2923678050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c025070 con 0x55c031349230 2024-04-17T07:22:40.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.062+0000 7fb4ab969740 1 -- 172.21.15.86:0/2923678050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 msgr2=0x55c031335290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.062+0000 7fb4ab969740 1 --2- 172.21.15.86:0/2923678050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031335290 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb49c000f80 tx=0x7fb49c009fd0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.063+0000 7fb4ab969740 1 -- 172.21.15.86:0/2923678050 shutdown_connections 2024-04-17T07:22:40.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.063+0000 7fb4ab969740 1 --2- 172.21.15.86:0/2923678050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c031349230 0x55c031335290 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.063+0000 7fb4ab969740 1 --2- 172.21.15.86:0/2923678050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 0x55c031354de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.063+0000 7fb4ab969740 1 --2- 172.21.15.86:0/2923678050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c0313436d0 0x55c031334d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.063+0000 7fb4ab969740 1 -- 172.21.15.86:0/2923678050 >> 172.21.15.86:0/2923678050 conn(0x55c03133ec40 msgr2=0x55c03134c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.063+0000 7fb4ab969740 1 -- 172.21.15.86:0/2923678050 shutdown_connections 2024-04-17T07:22:40.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.063+0000 7fb4ab969740 1 -- 172.21.15.86:0/2923678050 wait complete. 2024-04-17T07:22:40.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.064+0000 7fb4ab969740 1 -- start start 2024-04-17T07:22:40.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.064+0000 7fb4ab969740 1 Processor -- start 2024-04-17T07:22:40.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.064+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 0x55c0313523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.064+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 0x55c031352920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.065+0000 7fb4ab969740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c031349230 0x55c031353930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.065+0000 7fb4a1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 0x55c0313523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.065+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 0x55c031352920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 0x55c031352920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46584/0 (socket says 172.21.15.86:46584) 2024-04-17T07:22:40.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4ab969740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a40736a0 con 0x55c0313436d0 2024-04-17T07:22:40.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4a1400640 1 -- 172.21.15.86:0/2977103742 learned_addr learned my addr 172.21.15.86:0/2977103742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4ab969740 1 -- 172.21.15.86:0/2977103742 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4072fd0 con 0x55c031349230 2024-04-17T07:22:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4ab969740 1 -- 172.21.15.86:0/2977103742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4070200 con 0x55c0313440c0 2024-04-17T07:22:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4ab969740 10 client.0 _ll_register_callbacks cb 0x7fb49c003b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4a2800640 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c031349230 0x55c031353930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4a1e00640 1 -- 172.21.15.86:0/2977103742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 msgr2=0x55c031352920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4a1e00640 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 0x55c031352920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4a1e00640 1 -- 172.21.15.86:0/2977103742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c031349230 msgr2=0x55c031353930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4a1e00640 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c031349230 0x55c031353930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.066+0000 7fb4a1e00640 1 -- 172.21.15.86:0/2977103742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c031353f20 con 0x55c0313436d0 2024-04-17T07:22:40.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.067+0000 7fb4a1e00640 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 0x55c0313523e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x55c03135c3f0 tx=0x7fb494013750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2458cf05f3d3732 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.067+0000 7fb483400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:22:40.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.068+0000 7fb483400640 1 -- 172.21.15.86:0/2977103742 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494004460 con 0x55c0313436d0 2024-04-17T07:22:40.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.068+0000 7fb483400640 1 -- 172.21.15.86:0/2977103742 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb49401b3f0 con 0x55c0313436d0 2024-04-17T07:22:40.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.068+0000 7fb4ab969740 1 -- 172.21.15.86:0/2977103742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49c019040 con 0x55c0313436d0 2024-04-17T07:22:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.069+0000 7fb483400640 1 -- 172.21.15.86:0/2977103742 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb49401b5f0 con 0x55c0313436d0 2024-04-17T07:22:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.069+0000 7fb4ab969740 10 client.17616 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.069+0000 7fb4ab969740 1 -- 172.21.15.86:0/2977103742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c0313552a0 con 0x55c0313436d0 2024-04-17T07:22:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.069+0000 7fb483400640 1 -- 172.21.15.86:0/2977103742 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb49c019040 con 0x55c0313436d0 2024-04-17T07:22:40.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.069+0000 7fb483400640 1 -- 172.21.15.86:0/2977103742 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb4940048c0 con 0x55c0313436d0 2024-04-17T07:22:40.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.069+0000 7fb4ab969740 10 client.17616 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:40.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.069+0000 7fb4ab969740 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:40.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.069+0000 7fb4ab969740 20 client.17616 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:40.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.070+0000 7fb4ab969740 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c0313c04f0 0x55c0313c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.070+0000 7fb4ab969740 4 client.17616 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:22:40.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.070+0000 7fb4ab969740 1 -- 172.21.15.86:0/2977103742 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c0312a8540 con 0x55c0313c04f0 2024-04-17T07:22:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.070+0000 7fb4a1400640 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c0313c04f0 0x55c0313c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.071+0000 7fb4a1400640 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c0313c04f0 0x55c0313c2910 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a58e2ae59e831 server_cookie=5c51f1299e199970 in_seq=0 out_seq=0 2024-04-17T07:22:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.071+0000 7fb483400640 10 client.17616 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:40.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb483400640 1 -- 172.21.15.86:0/2977103742 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c0312a8540 con 0x55c0313c04f0 2024-04-17T07:22:40.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb483400640 10 client.17616 handle_command_reply: tid=0 2024-04-17T07:22:40.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb4ab969740 1 client.17616 shutdown 2024-04-17T07:22:40.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb4ab969740 1 -- 172.21.15.86:0/2977103742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 msgr2=0x55c0313523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb4ab969740 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 0x55c0313523e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x55c03135c3f0 tx=0x7fb494013750 comp rx=0 tx=0).stop 2024-04-17T07:22:40.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb4ab969740 1 -- 172.21.15.86:0/2977103742 shutdown_connections 2024-04-17T07:22:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb4ab969740 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c0313c04f0 0x55c0313c2910 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb4ab969740 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c031349230 0x55c031353930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb4ab969740 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c0313440c0 0x55c031352920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb483400640 0 client.17616 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb4ab969740 1 --2- 172.21.15.86:0/2977103742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c0313436d0 0x55c0313523e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.073+0000 7fb4ab969740 1 -- 172.21.15.86:0/2977103742 >> 172.21.15.86:0/2977103742 conn(0x55c03133ec40 msgr2=0x55c031341ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.074+0000 7fb4ab969740 1 -- 172.21.15.86:0/2977103742 shutdown_connections 2024-04-17T07:22:40.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.074+0000 7fb4ab969740 1 -- 172.21.15.86:0/2977103742 wait complete. 2024-04-17T07:22:40.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.074+0000 7fb4ab969740 20 client.17616 trim_cache size 0 max 16384 2024-04-17T07:22:40.076 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:22:40.076 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:22:40.076 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "d8d7ab09-dda9-4bca-a48b-8a65e933b7fe", 2024-04-17T07:22:40.076 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:22:40.076 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:22:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.075+0000 7fb4a8c00640 1 -- 172.21.15.86:0/1162551229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4880a3d00 msgr2=0x7fb4880a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.075+0000 7fb4a8c00640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4880a3d00 0x7fb4880a61c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55c031344920 tx=0x7fb49c004ab0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.075+0000 7fb4a8c00640 1 -- 172.21.15.86:0/1162551229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a4101ff0 msgr2=0x7fb4a419a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.075+0000 7fb4a8c00640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a4101ff0 0x7fb4a419a8f0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb49002c0b0 tx=0x7fb4900042f0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.075+0000 7fb4a8c00640 1 -- 172.21.15.86:0/1162551229 shutdown_connections 2024-04-17T07:22:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.075+0000 7fb4a8c00640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4880a3d00 0x7fb4880a61c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.075+0000 7fb4a8c00640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4a41029b0 0x7fb4a41a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.075+0000 7fb4a8c00640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4a4101ff0 0x7fb4a419a8f0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.075+0000 7fb4a8c00640 1 --2- 172.21.15.86:0/1162551229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4a4100ea0 0x7fb4a419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.075+0000 7fb4a8c00640 1 -- 172.21.15.86:0/1162551229 >> 172.21.15.86:0/1162551229 conn(0x7fb4a40fc410 msgr2=0x7fb4a40fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.076+0000 7fb4a8c00640 1 -- 172.21.15.86:0/1162551229 shutdown_connections 2024-04-17T07:22:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.077+0000 7fb4a8c00640 1 -- 172.21.15.86:0/1162551229 wait complete. 2024-04-17T07:22:40.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:22:40.101 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:22:40.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.408+0000 7fde82000640 1 -- 172.21.15.86:0/2578807251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde7c103190 msgr2=0x7fde7c105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.408+0000 7fde82000640 1 --2- 172.21.15.86:0/2578807251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde7c103190 0x7fde7c105600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fde68009900 tx=0x7fde6802bc40 comp rx=0 tx=0).stop 2024-04-17T07:22:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.408+0000 7fde82000640 1 -- 172.21.15.86:0/2578807251 shutdown_connections 2024-04-17T07:22:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.408+0000 7fde82000640 1 --2- 172.21.15.86:0/2578807251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c105b40 0x7fde7c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.408+0000 7fde82000640 1 --2- 172.21.15.86:0/2578807251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde7c103190 0x7fde7c105600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.408+0000 7fde82000640 1 --2- 172.21.15.86:0/2578807251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde7c100820 0x7fde7c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.408+0000 7fde82000640 1 -- 172.21.15.86:0/2578807251 >> 172.21.15.86:0/2578807251 conn(0x7fde7c0fc420 msgr2=0x7fde7c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.408+0000 7fde82000640 1 -- 172.21.15.86:0/2578807251 shutdown_connections 2024-04-17T07:22:40.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.408+0000 7fde82000640 1 -- 172.21.15.86:0/2578807251 wait complete. 2024-04-17T07:22:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.410+0000 7fde82000640 1 Processor -- start 2024-04-17T07:22:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.410+0000 7fde82000640 1 -- start start 2024-04-17T07:22:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.410+0000 7fde82000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c100820 0x7fde7c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.410+0000 7fde82000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde7c103190 0x7fde7c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.410+0000 7fde82000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde7c105b40 0x7fde7c1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.410+0000 7fde82000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c114100 con 0x7fde7c100820 2024-04-17T07:22:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.410+0000 7fde82000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c113f80 con 0x7fde7c105b40 2024-04-17T07:22:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.410+0000 7fde82000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c114280 con 0x7fde7c103190 2024-04-17T07:22:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.411+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde7c103190 0x7fde7c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.411+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde7c103190 0x7fde7c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46590/0 (socket says 172.21.15.86:46590) 2024-04-17T07:22:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.411+0000 7fde7aa00640 1 -- 172.21.15.86:0/2608526274 learned_addr learned my addr 172.21.15.86:0/2608526274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:40.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.411+0000 7fde7be00640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde7c105b40 0x7fde7c1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.411+0000 7fde7b400640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c100820 0x7fde7c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.412+0000 7fde7be00640 1 -- 172.21.15.86:0/2608526274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde7c103190 msgr2=0x7fde7c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.412+0000 7fde7be00640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde7c103190 0x7fde7c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.412+0000 7fde7be00640 1 -- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c100820 msgr2=0x7fde7c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.412+0000 7fde7be00640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c100820 0x7fde7c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.412+0000 7fde7be00640 1 -- 172.21.15.86:0/2608526274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c1a6450 con 0x7fde7c105b40 2024-04-17T07:22:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.412+0000 7fde7b400640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c100820 0x7fde7c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:22:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.413+0000 7fde7be00640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde7c105b40 0x7fde7c1a5de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde7400ec50 tx=0x7fde7400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.414+0000 7fde73e00640 1 -- 172.21.15.86:0/2608526274 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74003b70 con 0x7fde7c105b40 2024-04-17T07:22:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.414+0000 7fde73e00640 1 -- 172.21.15.86:0/2608526274 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde74003d10 con 0x7fde7c105b40 2024-04-17T07:22:40.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.414+0000 7fde73e00640 1 -- 172.21.15.86:0/2608526274 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740107d0 con 0x7fde7c105b40 2024-04-17T07:22:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.414+0000 7fde82000640 1 -- 172.21.15.86:0/2608526274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c1a6740 con 0x7fde7c105b40 2024-04-17T07:22:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.414+0000 7fde82000640 1 -- 172.21.15.86:0/2608526274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c1a6c40 con 0x7fde7c105b40 2024-04-17T07:22:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.420+0000 7fde73e00640 1 -- 172.21.15.86:0/2608526274 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fde74003fe0 con 0x7fde7c105b40 2024-04-17T07:22:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.420+0000 7fde84d67740 1 Processor -- start 2024-04-17T07:22:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.420+0000 7fde84d67740 1 -- start start 2024-04-17T07:22:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.420+0000 7fde84d67740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.420+0000 7fde84d67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 0x555aa4c8e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.420+0000 7fde84d67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c8ecf0 0x555aa4c91180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.420+0000 7fde84d67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555aa4bf0120 con 0x555aa4c8ecf0 2024-04-17T07:22:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.420+0000 7fde84d67740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555aa4b96340 con 0x555aa4c89190 2024-04-17T07:22:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.420+0000 7fde84d67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555aa4bec0e0 con 0x555aa4c89b80 2024-04-17T07:22:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.421+0000 7fde7b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 0x555aa4c8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.421+0000 7fde7b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 0x555aa4c8e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46596/0 (socket says 172.21.15.86:46596) 2024-04-17T07:22:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.421+0000 7fde7b400640 1 -- 172.21.15.86:0/786317266 learned_addr learned my addr 172.21.15.86:0/786317266 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.421+0000 7fde7aa00640 1 --2- 172.21.15.86:0/786317266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.422+0000 7fde7aa00640 1 --2- 172.21.15.86:0/786317266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c8ecf0 0x555aa4c91180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.422+0000 7fde7b400640 1 -- 172.21.15.86:0/786317266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 msgr2=0x555aa4c895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.422+0000 7fde7b400640 1 --2- 172.21.15.86:0/786317266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.422+0000 7fde7b400640 1 -- 172.21.15.86:0/786317266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c8ecf0 msgr2=0x555aa4c91180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.422+0000 7fde7b400640 1 --2- 172.21.15.86:0/786317266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c8ecf0 0x555aa4c91180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.422+0000 7fde7b400640 1 -- 172.21.15.86:0/786317266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa49cc7d0 con 0x555aa4c89b80 2024-04-17T07:22:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.424+0000 7fde7b400640 1 --2- 172.21.15.86:0/786317266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 0x555aa4c8e680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fde6c011570 tx=0x7fde6c011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa649a61bc1740a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.424+0000 7fde73e00640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde600a3c30 0x7fde600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.424+0000 7fde73e00640 1 -- 172.21.15.86:0/2608526274 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fde740d0cc0 con 0x7fde7c105b40 2024-04-17T07:22:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.425+0000 7fde7aa00640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde600a3c30 0x7fde600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.425+0000 7fde67e00640 1 -- 172.21.15.86:0/786317266 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00f840 con 0x555aa4c89b80 2024-04-17T07:22:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.425+0000 7fde67e00640 1 -- 172.21.15.86:0/786317266 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde6c00fe90 con 0x555aa4c89b80 2024-04-17T07:22:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.425+0000 7fde67e00640 1 -- 172.21.15.86:0/786317266 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c03a3c0 con 0x555aa4c89b80 2024-04-17T07:22:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.426+0000 7fde84d67740 1 -- 172.21.15.86:0/786317266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 msgr2=0x555aa4c8e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.426+0000 7fde84d67740 1 --2- 172.21.15.86:0/786317266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 0x555aa4c8e680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fde6c011570 tx=0x7fde6c011a30 comp rx=0 tx=0).stop 2024-04-17T07:22:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.426+0000 7fde84d67740 1 -- 172.21.15.86:0/786317266 shutdown_connections 2024-04-17T07:22:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.426+0000 7fde84d67740 1 --2- 172.21.15.86:0/786317266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c8ecf0 0x555aa4c91180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.426+0000 7fde84d67740 1 --2- 172.21.15.86:0/786317266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 0x555aa4c8e680 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.426+0000 7fde84d67740 1 --2- 172.21.15.86:0/786317266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.426+0000 7fde84d67740 1 -- 172.21.15.86:0/786317266 >> 172.21.15.86:0/786317266 conn(0x555aa4c84700 msgr2=0x555aa4c84b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.427+0000 7fde7aa00640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde600a3c30 0x7fde600a60f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fde68005dc0 tx=0x7fde6802c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.427+0000 7fde84d67740 1 -- 172.21.15.86:0/786317266 shutdown_connections 2024-04-17T07:22:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.427+0000 7fde84d67740 1 -- 172.21.15.86:0/786317266 wait complete. 2024-04-17T07:22:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.428+0000 7fde84d67740 1 -- start start 2024-04-17T07:22:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.428+0000 7fde84d67740 1 Processor -- start 2024-04-17T07:22:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde84d67740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c93540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde84d67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 0x555aa4c93a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde84d67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c8ecf0 0x555aa4c9aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde84d67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555aa4c91ae0 con 0x555aa4c8ecf0 2024-04-17T07:22:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde84d67740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555aa4ba9710 con 0x555aa4c89190 2024-04-17T07:22:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde84d67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555aa4c91c60 con 0x555aa4c89b80 2024-04-17T07:22:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde84d67740 10 client.0 _ll_register_callbacks cb 0x555aa4ba4af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde7b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c93540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde7b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c93540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:53732/0 (socket says 172.21.15.86:53732) 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde7b400640 1 -- 172.21.15.86:0/1791453181 learned_addr learned my addr 172.21.15.86:0/1791453181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.429+0000 7fde7be00640 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c8ecf0 0x555aa4c9aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.430+0000 7fde7b400640 1 -- 172.21.15.86:0/1791453181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 msgr2=0x555aa4c93a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.430+0000 7fde7b400640 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 0x555aa4c93a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.430+0000 7fde7b400640 1 -- 172.21.15.86:0/1791453181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c8ecf0 msgr2=0x555aa4c9aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.430+0000 7fde7b400640 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c8ecf0 0x555aa4c9aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.430+0000 7fde7b400640 1 -- 172.21.15.86:0/1791453181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa49023f0 con 0x555aa4c89190 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.430+0000 7fde7b400640 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c93540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fde6c03e000 tx=0x7fde6c009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67c330045fff9fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.430+0000 7fde64c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.431+0000 7fde64c00640 1 -- 172.21.15.86:0/1791453181 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c005050 con 0x555aa4c89190 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.431+0000 7fde64c00640 1 -- 172.21.15.86:0/1791453181 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fde6c0051f0 con 0x555aa4c89190 2024-04-17T07:22:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.431+0000 7fde84d67740 1 -- 172.21.15.86:0/1791453181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6c00fe90 con 0x555aa4c89190 2024-04-17T07:22:40.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.431+0000 7fde64c00640 1 -- 172.21.15.86:0/1791453181 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde6c0053f0 con 0x555aa4c89190 2024-04-17T07:22:40.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.432+0000 7fde84d67740 10 client.36572 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:40.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.432+0000 7fde84d67740 1 -- 172.21.15.86:0/1791453181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555aa48c20f0 con 0x555aa4c89190 2024-04-17T07:22:40.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.432+0000 7fde64c00640 1 -- 172.21.15.86:0/1791453181 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde6c00fe90 con 0x555aa4c89190 2024-04-17T07:22:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.433+0000 7fde64c00640 1 -- 172.21.15.86:0/1791453181 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fde6c004a30 con 0x555aa4c89190 2024-04-17T07:22:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.433+0000 7fde84d67740 10 client.36572 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.433+0000 7fde84d67740 10 client.36572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.433+0000 7fde84d67740 20 client.36572 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.433+0000 7fde84d67740 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555aa4c9f060 0x555aa4ca14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.433+0000 7fde84d67740 4 client.36572 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:22:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.433+0000 7fde84d67740 1 -- 172.21.15.86:0/1791453181 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aa4bfdb40 con 0x555aa4c9f060 2024-04-17T07:22:40.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.433+0000 7fde7aa00640 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555aa4c9f060 0x555aa4ca14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.434+0000 7fde7aa00640 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555aa4c9f060 0x555aa4ca14a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769ec26d8763fe2 server_cookie=bbb4ecf95792fa5a in_seq=0 out_seq=0 2024-04-17T07:22:40.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.434+0000 7fde64c00640 10 client.36572 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.437+0000 7fde64c00640 1 -- 172.21.15.86:0/1791453181 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555aa4bfdb40 con 0x555aa4c9f060 2024-04-17T07:22:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.437+0000 7fde64c00640 10 client.36572 handle_command_reply: tid=0 2024-04-17T07:22:40.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.438+0000 7fde84d67740 1 client.36572 shutdown 2024-04-17T07:22:40.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.438+0000 7fde84d67740 1 -- 172.21.15.86:0/1791453181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 msgr2=0x555aa4c93540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.438+0000 7fde84d67740 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c93540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fde6c03e000 tx=0x7fde6c009dd0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.439+0000 7fde84d67740 1 -- 172.21.15.86:0/1791453181 shutdown_connections 2024-04-17T07:22:40.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.439+0000 7fde84d67740 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555aa4c9f060 0x555aa4ca14a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.439+0000 7fde84d67740 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c8ecf0 0x555aa4c9aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.439+0000 7fde84d67740 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89b80 0x555aa4c93a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.439+0000 7fde84d67740 1 --2- 172.21.15.86:0/1791453181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c93540 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.439+0000 7fde84d67740 1 -- 172.21.15.86:0/1791453181 >> 172.21.15.86:0/1791453181 conn(0x555aa4c84700 msgr2=0x555aa4c83890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.439+0000 7fde64c00640 0 client.36572 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:40.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.439+0000 7fde84d67740 1 -- 172.21.15.86:0/1791453181 shutdown_connections 2024-04-17T07:22:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.439+0000 7fde84d67740 1 -- 172.21.15.86:0/1791453181 wait complete. 2024-04-17T07:22:40.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.440+0000 7fde84d67740 20 client.36572 trim_cache size 0 max 16384 2024-04-17T07:22:40.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.473+0000 7fde84d67740 1 Processor -- start 2024-04-17T07:22:40.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde84d67740 1 -- start start 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde84d67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89190 0x555aa4c83710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde84d67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c9a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde84d67740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c8ecf0 0x555aa4c92b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde84d67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1a8a10 con 0x555aa4c89b80 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde84d67740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1aa550 con 0x555aa4c8ecf0 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde84d67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1a90e0 con 0x555aa4c89190 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c9a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c9a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56886/0 (socket says 172.21.15.86:56886) 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde7aa00640 1 -- 172.21.15.86:0/486135069 learned_addr learned my addr 172.21.15.86:0/486135069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.474+0000 7fde7be00640 1 --2- 172.21.15.86:0/486135069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c8ecf0 0x555aa4c92b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.475+0000 7fde7b400640 1 --2- 172.21.15.86:0/486135069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89190 0x555aa4c83710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.475+0000 7fde7aa00640 1 -- 172.21.15.86:0/486135069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89190 msgr2=0x555aa4c83710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.475+0000 7fde7aa00640 1 --2- 172.21.15.86:0/486135069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89190 0x555aa4c83710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.475+0000 7fde7aa00640 1 -- 172.21.15.86:0/486135069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c8ecf0 msgr2=0x555aa4c92b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.475+0000 7fde7aa00640 1 --2- 172.21.15.86:0/486135069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c8ecf0 0x555aa4c92b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.475+0000 7fde7aa00640 1 -- 172.21.15.86:0/486135069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa4c89f60 con 0x555aa4c89b80 2024-04-17T07:22:40.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.475+0000 7fde7aa00640 1 --2- 172.21.15.86:0/486135069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c9a990 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fde68009340 tx=0x7fde68005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6345c0e16541be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.476+0000 7fde67400640 1 -- 172.21.15.86:0/486135069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68005490 con 0x555aa4c89b80 2024-04-17T07:22:40.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.476+0000 7fde67400640 1 -- 172.21.15.86:0/486135069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde6805b070 con 0x555aa4c89b80 2024-04-17T07:22:40.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.476+0000 7fde67400640 1 -- 172.21.15.86:0/486135069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680049b0 con 0x555aa4c89b80 2024-04-17T07:22:40.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.477+0000 7fde84d67740 1 -- 172.21.15.86:0/486135069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 msgr2=0x555aa4c9a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.477+0000 7fde84d67740 1 --2- 172.21.15.86:0/486135069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c9a990 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fde68009340 tx=0x7fde68005d50 comp rx=0 tx=0).stop 2024-04-17T07:22:40.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.477+0000 7fde84d67740 1 -- 172.21.15.86:0/486135069 shutdown_connections 2024-04-17T07:22:40.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.477+0000 7fde84d67740 1 --2- 172.21.15.86:0/486135069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c8ecf0 0x555aa4c92b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.477+0000 7fde84d67740 1 --2- 172.21.15.86:0/486135069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c9a990 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.477+0000 7fde84d67740 1 --2- 172.21.15.86:0/486135069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c89190 0x555aa4c83710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.477+0000 7fde84d67740 1 -- 172.21.15.86:0/486135069 >> 172.21.15.86:0/486135069 conn(0x555aa4c84700 msgr2=0x555aa4c83000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.477+0000 7fde84d67740 1 -- 172.21.15.86:0/486135069 shutdown_connections 2024-04-17T07:22:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.478+0000 7fde84d67740 1 -- 172.21.15.86:0/486135069 wait complete. 2024-04-17T07:22:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.478+0000 7fde84d67740 1 -- start start 2024-04-17T07:22:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.478+0000 7fde84d67740 1 Processor -- start 2024-04-17T07:22:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde84d67740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde84d67740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c97b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde84d67740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c8ecf0 0x555aa4d04300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde84d67740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1a9e80 con 0x555aa4c89b80 2024-04-17T07:22:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde84d67740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1a97b0 con 0x555aa4c89190 2024-04-17T07:22:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde84d67740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c10f0d0 con 0x555aa4c8ecf0 2024-04-17T07:22:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde84d67740 10 client.0 _ll_register_callbacks cb 0x7fde68009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:22:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c97b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c97b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56888/0 (socket says 172.21.15.86:56888) 2024-04-17T07:22:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde7aa00640 1 -- 172.21.15.86:0/4050146631 learned_addr learned my addr 172.21.15.86:0/4050146631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:22:40.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde7b400640 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.479+0000 7fde7be00640 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c8ecf0 0x555aa4d04300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.480+0000 7fde7b400640 1 -- 172.21.15.86:0/4050146631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c8ecf0 msgr2=0x555aa4d04300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.480+0000 7fde7b400640 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c8ecf0 0x555aa4d04300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.480+0000 7fde7b400640 1 -- 172.21.15.86:0/4050146631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 msgr2=0x555aa4c97b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:22:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.480+0000 7fde7b400640 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c97b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.480+0000 7fde7b400640 1 -- 172.21.15.86:0/4050146631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa4c9aba0 con 0x555aa4c89190 2024-04-17T07:22:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.480+0000 7fde7b400640 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c975e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009e40 tx=0x7fde6c005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4424f1109a35401 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:22:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.480+0000 7fde64c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:22:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.481+0000 7fde64c00640 1 -- 172.21.15.86:0/4050146631 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c011240 con 0x555aa4c89190 2024-04-17T07:22:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.481+0000 7fde64c00640 1 -- 172.21.15.86:0/4050146631 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fde6c046860 con 0x555aa4c89190 2024-04-17T07:22:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.481+0000 7fde84d67740 1 -- 172.21.15.86:0/4050146631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6805b070 con 0x555aa4c89190 2024-04-17T07:22:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.481+0000 7fde64c00640 1 -- 172.21.15.86:0/4050146631 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde6c0113e0 con 0x555aa4c89190 2024-04-17T07:22:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.482+0000 7fde84d67740 10 client.36578 fetch_fsmap learned FSMap version 28 2024-04-17T07:22:40.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.482+0000 7fde84d67740 1 -- 172.21.15.86:0/4050146631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555aa4d05820 con 0x555aa4c89190 2024-04-17T07:22:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.482+0000 7fde64c00640 1 -- 172.21.15.86:0/4050146631 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde6805b070 con 0x555aa4c89190 2024-04-17T07:22:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.482+0000 7fde64c00640 1 -- 172.21.15.86:0/4050146631 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fde6c063030 con 0x555aa4c89190 2024-04-17T07:22:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.483+0000 7fde84d67740 10 client.36578 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:22:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.483+0000 7fde84d67740 10 client.36578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:22:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.483+0000 7fde84d67740 20 client.36578 populate_metadata read hostname 'smithi086' 2024-04-17T07:22:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.483+0000 7fde84d67740 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555aa4d061c0 0x555aa4d085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:22:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.483+0000 7fde84d67740 4 client.36578 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:22:40.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.483+0000 7fde84d67740 1 -- 172.21.15.86:0/4050146631 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555aa4c8ab90 con 0x555aa4d061c0 2024-04-17T07:22:40.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.484+0000 7fde7aa00640 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555aa4d061c0 0x555aa4d085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:22:40.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.484+0000 7fde64c00640 10 client.36578 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:40.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.484+0000 7fde7aa00640 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555aa4d061c0 0x555aa4d085e0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e425ccef4ae8a97 server_cookie=bdd0157ad926f672 in_seq=0 out_seq=0 2024-04-17T07:22:40.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde64c00640 1 -- 172.21.15.86:0/4050146631 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555aa4c8ab90 con 0x555aa4d061c0 2024-04-17T07:22:40.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde64c00640 10 client.36578 handle_command_reply: tid=0 2024-04-17T07:22:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde84d67740 1 client.36578 shutdown 2024-04-17T07:22:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde84d67740 1 -- 172.21.15.86:0/4050146631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 msgr2=0x555aa4c975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde84d67740 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c975e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009e40 tx=0x7fde6c005df0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde84d67740 1 -- 172.21.15.86:0/4050146631 shutdown_connections 2024-04-17T07:22:40.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde84d67740 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555aa4d061c0 0x555aa4d085e0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde84d67740 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555aa4c8ecf0 0x555aa4d04300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde84d67740 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555aa4c89b80 0x555aa4c97b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde84d67740 1 --2- 172.21.15.86:0/4050146631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555aa4c89190 0x555aa4c975e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde84d67740 1 -- 172.21.15.86:0/4050146631 >> 172.21.15.86:0/4050146631 conn(0x555aa4c84700 msgr2=0x555aa4c83890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.485+0000 7fde64c00640 0 client.36578 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:22:40.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.486+0000 7fde84d67740 1 -- 172.21.15.86:0/4050146631 shutdown_connections 2024-04-17T07:22:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.486+0000 7fde84d67740 1 -- 172.21.15.86:0/4050146631 wait complete. 2024-04-17T07:22:40.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.486+0000 7fde84d67740 20 client.36578 trim_cache size 0 max 16384 2024-04-17T07:22:40.487 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:22:40.487 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:22:40.487 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:22:40.487 INFO:teuthology.orchestra.run.smithi086.stdout: "d8d7ab09-dda9-4bca-a48b-8a65e933b7fe": { 2024-04-17T07:22:40.487 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:22:40.487 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "d8d7ab09-dda9-4bca-a48b-8a65e933b7fe", 2024-04-17T07:22:40.487 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:22:40.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:22:40.488 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:22:40.488 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:22:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 -- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde600a3c30 msgr2=0x7fde600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde600a3c30 0x7fde600a60f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fde68005dc0 tx=0x7fde6802c7f0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 -- 172.21.15.86:0/2608526274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde7c105b40 msgr2=0x7fde7c1a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:22:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde7c105b40 0x7fde7c1a5de0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde7400ec50 tx=0x7fde7400c6a0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 -- 172.21.15.86:0/2608526274 shutdown_connections 2024-04-17T07:22:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde600a3c30 0x7fde600a60f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde7c105b40 0x7fde7c1a5de0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde7c103190 0x7fde7c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 --2- 172.21.15.86:0/2608526274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c100820 0x7fde7c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:22:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 -- 172.21.15.86:0/2608526274 >> 172.21.15.86:0/2608526274 conn(0x7fde7c0fc420 msgr2=0x7fde7c101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:22:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.487+0000 7fde82000640 1 -- 172.21.15.86:0/2608526274 shutdown_connections 2024-04-17T07:22:40.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:22:40.488+0000 7fde82000640 1 -- 172.21.15.86:0/2608526274 wait complete. 2024-04-17T07:22:40.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:40 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:22:40.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:40 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:40.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:40 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:22:40.504 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:40 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:22:40.504 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:40 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:40.504 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:40 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:22:40.507 INFO:tasks.cephfs.filesystem:scrub status for tag:d8d7ab09-dda9-4bca-a48b-8a65e933b7fe - {'path': '/', 'tag': 'd8d7ab09-dda9-4bca-a48b-8a65e933b7fe', 'options': 'recursive,force'} 2024-04-17T07:22:40.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:40 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:22:40.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:40 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:40.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:40 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:22:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:41 smithi098 ceph-mon[35418]: pgmap v689: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.8 MiB/s wr, 699 op/s 2024-04-17T07:22:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:41 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:41 smithi103 ceph-mon[35495]: pgmap v689: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.8 MiB/s wr, 699 op/s 2024-04-17T07:22:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:41 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:41 smithi086 ceph-mon[31733]: pgmap v689: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.8 MiB/s wr, 699 op/s 2024-04-17T07:22:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:41 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:22:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:43 smithi098 ceph-mon[35418]: pgmap v690: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 737 op/s 2024-04-17T07:22:43.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:43 smithi103 ceph-mon[35495]: pgmap v690: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 737 op/s 2024-04-17T07:22:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:43 smithi086 ceph-mon[31733]: pgmap v690: 129 pgs: 129 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 737 op/s 2024-04-17T07:22:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 200.0 s, 221.4 tps, lat 450.721 ms stddev 1071.521 2024-04-17T07:22:44.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:44 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:22:44.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:44 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:22:44.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:44 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:22:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:45 smithi098 ceph-mon[35418]: pgmap v691: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.7 MiB/s wr, 675 op/s 2024-04-17T07:22:45.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:45 smithi103 ceph-mon[35495]: pgmap v691: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.7 MiB/s wr, 675 op/s 2024-04-17T07:22:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:45 smithi086 ceph-mon[31733]: pgmap v691: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.7 MiB/s wr, 675 op/s 2024-04-17T07:22:47.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:47 smithi098 ceph-mon[35418]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 549 op/s 2024-04-17T07:22:47.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:47 smithi103 ceph-mon[35495]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 549 op/s 2024-04-17T07:22:47.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:47 smithi086 ceph-mon[31733]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 549 op/s 2024-04-17T07:22:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 205.0 s, 678.6 tps, lat 146.864 ms stddev 133.245 2024-04-17T07:22:49.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:49 smithi103 ceph-mon[35495]: pgmap v693: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.2 MiB/s wr, 575 op/s 2024-04-17T07:22:49.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:49 smithi086 ceph-mon[31733]: pgmap v693: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.2 MiB/s wr, 575 op/s 2024-04-17T07:22:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:49 smithi098 ceph-mon[35418]: pgmap v693: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.2 MiB/s wr, 575 op/s 2024-04-17T07:22:51.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:51 smithi103 ceph-mon[35495]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.1 MiB/s wr, 519 op/s 2024-04-17T07:22:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:51 smithi086 ceph-mon[31733]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.1 MiB/s wr, 519 op/s 2024-04-17T07:22:51.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:51 smithi098 ceph-mon[35418]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.1 MiB/s wr, 519 op/s 2024-04-17T07:22:53.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:53 smithi103 ceph-mon[35495]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.8 MiB/s wr, 543 op/s 2024-04-17T07:22:53.554 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:53 smithi098 ceph-mon[35418]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.8 MiB/s wr, 543 op/s 2024-04-17T07:22:53.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:53 smithi086 ceph-mon[31733]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.8 MiB/s wr, 543 op/s 2024-04-17T07:22:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 210.0 s, 691.4 tps, lat 144.278 ms stddev 61.955 2024-04-17T07:22:55.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:55 smithi086 ceph-mon[31733]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-04-17T07:22:55.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:55 smithi098 ceph-mon[35418]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-04-17T07:22:55.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:55 smithi103 ceph-mon[35495]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.5 MiB/s wr, 543 op/s 2024-04-17T07:22:57.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:57 smithi086 ceph-mon[31733]: pgmap v697: 129 pgs: 129 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.0 MiB/s wr, 521 op/s 2024-04-17T07:22:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:57 smithi098 ceph-mon[35418]: pgmap v697: 129 pgs: 129 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.0 MiB/s wr, 521 op/s 2024-04-17T07:22:57.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:57 smithi103 ceph-mon[35495]: pgmap v697: 129 pgs: 129 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.0 MiB/s wr, 521 op/s 2024-04-17T07:22:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 215.0 s, 776.4 tps, lat 128.579 ms stddev 45.804 2024-04-17T07:22:59.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:22:59 smithi086 ceph-mon[31733]: pgmap v698: 129 pgs: 129 active+clean; 1.3 GiB data, 5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.5 MiB/s wr, 661 op/s 2024-04-17T07:22:59.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:22:59 smithi098 ceph-mon[35418]: pgmap v698: 129 pgs: 129 active+clean; 1.3 GiB data, 5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.5 MiB/s wr, 661 op/s 2024-04-17T07:22:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:22:59 smithi103 ceph-mon[35495]: pgmap v698: 129 pgs: 129 active+clean; 1.3 GiB data, 5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.5 MiB/s wr, 661 op/s 2024-04-17T07:23:01.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:01 smithi086 ceph-mon[31733]: pgmap v699: 129 pgs: 129 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.4 MiB/s wr, 677 op/s 2024-04-17T07:23:01.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:01 smithi098 ceph-mon[35418]: pgmap v699: 129 pgs: 129 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.4 MiB/s wr, 677 op/s 2024-04-17T07:23:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:01 smithi103 ceph-mon[35495]: pgmap v699: 129 pgs: 129 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.4 MiB/s wr, 677 op/s 2024-04-17T07:23:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:03 smithi086 ceph-mon[31733]: pgmap v700: 129 pgs: 129 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.8 MiB/s wr, 742 op/s 2024-04-17T07:23:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:03 smithi098 ceph-mon[35418]: pgmap v700: 129 pgs: 129 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.8 MiB/s wr, 742 op/s 2024-04-17T07:23:03.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:03 smithi103 ceph-mon[35495]: pgmap v700: 129 pgs: 129 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.8 MiB/s wr, 742 op/s 2024-04-17T07:23:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 220.0 s, 735.4 tps, lat 136.447 ms stddev 39.825 2024-04-17T07:23:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:05 smithi086 ceph-mon[31733]: pgmap v701: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.7 MiB/s wr, 764 op/s 2024-04-17T07:23:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:05 smithi098 ceph-mon[35418]: pgmap v701: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.7 MiB/s wr, 764 op/s 2024-04-17T07:23:05.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:05 smithi103 ceph-mon[35495]: pgmap v701: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.7 MiB/s wr, 764 op/s 2024-04-17T07:23:07.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:07 smithi086 ceph-mon[31733]: pgmap v702: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.3 MiB/s wr, 673 op/s 2024-04-17T07:23:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:07 smithi098 ceph-mon[35418]: pgmap v702: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.3 MiB/s wr, 673 op/s 2024-04-17T07:23:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:07 smithi103 ceph-mon[35495]: pgmap v702: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.3 MiB/s wr, 673 op/s 2024-04-17T07:23:08.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:08 smithi098 ceph-mon[35418]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.8 MiB/s wr, 752 op/s 2024-04-17T07:23:08.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:08 smithi103 ceph-mon[35495]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.8 MiB/s wr, 752 op/s 2024-04-17T07:23:08.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:08 smithi086 ceph-mon[31733]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.8 MiB/s wr, 752 op/s 2024-04-17T07:23:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 225.0 s, 676.2 tps, lat 147.065 ms stddev 38.652 2024-04-17T07:23:10.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:23:10.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:23:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:10 smithi098 ceph-mon[35418]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.1 MiB/s wr, 697 op/s 2024-04-17T07:23:10.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:10 smithi103 ceph-mon[35495]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.1 MiB/s wr, 697 op/s 2024-04-17T07:23:10.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.826+0000 7f1fc2400640 1 -- 172.21.15.86:0/256885376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fbc1006e0 msgr2=0x7f1fbc102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:10.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.826+0000 7f1fc2400640 1 --2- 172.21.15.86:0/256885376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fbc1006e0 0x7f1fbc102b10 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009900 tx=0x7f1fb002bc40 comp rx=0 tx=0).stop 2024-04-17T07:23:10.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.827+0000 7f1fc2400640 1 -- 172.21.15.86:0/256885376 shutdown_connections 2024-04-17T07:23:10.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.827+0000 7f1fc2400640 1 --2- 172.21.15.86:0/256885376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fbc105a00 0x7f1fbc105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.827+0000 7f1fc2400640 1 --2- 172.21.15.86:0/256885376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fbc103050 0x7f1fbc1054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.827+0000 7f1fc2400640 1 --2- 172.21.15.86:0/256885376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fbc1006e0 0x7f1fbc102b10 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.827+0000 7f1fc2400640 1 -- 172.21.15.86:0/256885376 >> 172.21.15.86:0/256885376 conn(0x7f1fbc0fc300 msgr2=0x7f1fbc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:10.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.828+0000 7f1fc2400640 1 -- 172.21.15.86:0/256885376 shutdown_connections 2024-04-17T07:23:10.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.828+0000 7f1fc2400640 1 -- 172.21.15.86:0/256885376 wait complete. 2024-04-17T07:23:10.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.829+0000 7f1fc2400640 1 Processor -- start 2024-04-17T07:23:10.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:10 smithi086 ceph-mon[31733]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.1 MiB/s wr, 697 op/s 2024-04-17T07:23:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.829+0000 7f1fc2400640 1 -- start start 2024-04-17T07:23:10.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.830+0000 7f1fc2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fbc1006e0 0x7f1fbc10bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.830+0000 7f1fc2400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fbc103050 0x7f1fbc10c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.830+0000 7f1fc2400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fbc105a00 0x7f1fbc1ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.831+0000 7f1fc1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fbc1006e0 0x7f1fbc10bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.831+0000 7f1fc2400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc113fe0 con 0x7f1fbc1006e0 2024-04-17T07:23:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.831+0000 7f1fc1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fbc1006e0 0x7f1fbc10bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35604/0 (socket says 172.21.15.86:35604) 2024-04-17T07:23:10.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.831+0000 7f1fc1000640 1 -- 172.21.15.86:0/3315858500 learned_addr learned my addr 172.21.15.86:0/3315858500 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.831+0000 7f1fbbe00640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fbc103050 0x7f1fbc10c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.831+0000 7f1fc2400640 1 -- 172.21.15.86:0/3315858500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc113e60 con 0x7f1fbc105a00 2024-04-17T07:23:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.831+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fbc105a00 0x7f1fbc1ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.831+0000 7f1fc2400640 1 -- 172.21.15.86:0/3315858500 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc114160 con 0x7f1fbc103050 2024-04-17T07:23:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.832+0000 7f1fc1000640 1 -- 172.21.15.86:0/3315858500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fbc103050 msgr2=0x7f1fbc10c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.832+0000 7f1fc1000640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fbc103050 0x7f1fbc10c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.832+0000 7f1fc1000640 1 -- 172.21.15.86:0/3315858500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fbc105a00 msgr2=0x7f1fbc1ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.832+0000 7f1fc1000640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fbc105a00 0x7f1fbc1ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.832+0000 7f1fc1000640 1 -- 172.21.15.86:0/3315858500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc1aec10 con 0x7f1fbc1006e0 2024-04-17T07:23:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.832+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fbc105a00 0x7f1fbc1ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.833+0000 7f1fc1000640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fbc1006e0 0x7f1fbc10bcb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0000c00 tx=0x7f1fb00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:10.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.833+0000 7f1fb9600640 1 -- 172.21.15.86:0/3315858500 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0043020 con 0x7f1fbc1006e0 2024-04-17T07:23:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.834+0000 7f1fb9600640 1 -- 172.21.15.86:0/3315858500 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fb0004aa0 con 0x7f1fbc1006e0 2024-04-17T07:23:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.834+0000 7f1fc2400640 1 -- 172.21.15.86:0/3315858500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc1aede0 con 0x7f1fbc1006e0 2024-04-17T07:23:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.834+0000 7f1fc2400640 1 -- 172.21.15.86:0/3315858500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc1af210 con 0x7f1fbc1006e0 2024-04-17T07:23:10.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.834+0000 7f1fb9600640 1 -- 172.21.15.86:0/3315858500 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0003d00 con 0x7f1fbc1006e0 2024-04-17T07:23:10.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.840+0000 7f1fb9600640 1 -- 172.21.15.86:0/3315858500 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1fb0046020 con 0x7f1fbc1006e0 2024-04-17T07:23:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.841+0000 7f1fb9600640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1fa40a3c30 0x7f1fa40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.842+0000 7f1fc681b740 1 Processor -- start 2024-04-17T07:23:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.842+0000 7f1fb9600640 1 -- 172.21.15.86:0/3315858500 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1fb00f29c0 con 0x7f1fbc1006e0 2024-04-17T07:23:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.842+0000 7f1fc681b740 1 -- start start 2024-04-17T07:23:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.842+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36dd370 0x558dc36dd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.842+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36ddd60 0x558dc36e2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.842+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36e2ed0 0x558dc36e5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.842+0000 7f1fbbe00640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1fa40a3c30 0x7f1fa40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.843+0000 7f1fc681b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558dc3644120 con 0x558dc36e2ed0 2024-04-17T07:23:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.843+0000 7f1fc681b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558dc35ea340 con 0x558dc36dd370 2024-04-17T07:23:10.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.843+0000 7f1fc681b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558dc36400e0 con 0x558dc36ddd60 2024-04-17T07:23:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.843+0000 7f1fbbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36ddd60 0x558dc36e2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.843+0000 7f1fc1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36e2ed0 0x558dc36e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.843+0000 7f1fc1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36e2ed0 0x558dc36e5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35608/0 (socket says 172.21.15.86:35608) 2024-04-17T07:23:10.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.843+0000 7f1fc1a00640 1 -- 172.21.15.86:0/3250885545 learned_addr learned my addr 172.21.15.86:0/3250885545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.843+0000 7f1fc1000640 1 --2- 172.21.15.86:0/3250885545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36dd370 0x558dc36dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.843+0000 7f1fbbe00640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1fa40a3c30 0x7f1fa40a60f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80058a0 tx=0x7f1fa800b7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.844+0000 7f1fc1a00640 1 -- 172.21.15.86:0/3250885545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36ddd60 msgr2=0x558dc36e2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.844+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/3250885545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36ddd60 0x558dc36e2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.844+0000 7f1fc1a00640 1 -- 172.21.15.86:0/3250885545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36dd370 msgr2=0x558dc36dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.844+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/3250885545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36dd370 0x558dc36dd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.844+0000 7f1fc1a00640 1 -- 172.21.15.86:0/3250885545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc34207d0 con 0x558dc36e2ed0 2024-04-17T07:23:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.844+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/3250885545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36e2ed0 0x558dc36e5360 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40117e0 tx=0x7f1fb4011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f720692506c929 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.845+0000 7f1fae000640 1 -- 172.21.15.86:0/3250885545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4011310 con 0x558dc36e2ed0 2024-04-17T07:23:10.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.845+0000 7f1fae000640 1 -- 172.21.15.86:0/3250885545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fb4003e20 con 0x558dc36e2ed0 2024-04-17T07:23:10.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.845+0000 7f1fae000640 1 -- 172.21.15.86:0/3250885545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40044d0 con 0x558dc36e2ed0 2024-04-17T07:23:10.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.846+0000 7f1fc681b740 1 -- 172.21.15.86:0/3250885545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36e2ed0 msgr2=0x558dc36e5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:10.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.846+0000 7f1fc681b740 1 --2- 172.21.15.86:0/3250885545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36e2ed0 0x558dc36e5360 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40117e0 tx=0x7f1fb4011ca0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.846+0000 7f1fc681b740 1 -- 172.21.15.86:0/3250885545 shutdown_connections 2024-04-17T07:23:10.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.846+0000 7f1fc681b740 1 --2- 172.21.15.86:0/3250885545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36e2ed0 0x558dc36e5360 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.846+0000 7f1fc681b740 1 --2- 172.21.15.86:0/3250885545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36ddd60 0x558dc36e2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.846+0000 7f1fc681b740 1 --2- 172.21.15.86:0/3250885545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36dd370 0x558dc36dd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.846+0000 7f1fc681b740 1 -- 172.21.15.86:0/3250885545 >> 172.21.15.86:0/3250885545 conn(0x558dc36d88e0 msgr2=0x558dc36d8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.847+0000 7f1fc681b740 1 -- 172.21.15.86:0/3250885545 shutdown_connections 2024-04-17T07:23:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.847+0000 7f1fc681b740 1 -- 172.21.15.86:0/3250885545 wait complete. 2024-04-17T07:23:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.847+0000 7f1fc681b740 1 -- start start 2024-04-17T07:23:10.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.847+0000 7f1fc681b740 1 Processor -- start 2024-04-17T07:23:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36dd370 0x558dc36e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 0x558dc36e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc681b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558dc36e5e80 con 0x558dc36ddd60 2024-04-17T07:23:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc681b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558dc35fd710 con 0x558dc36dd370 2024-04-17T07:23:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc681b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558dc36e6000 con 0x558dc36e2ed0 2024-04-17T07:23:10.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc681b740 10 client.0 _ll_register_callbacks cb 0x558dc35f8af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc1a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc1a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36eeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41004/0 (socket says 172.21.15.86:41004) 2024-04-17T07:23:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc1a00640 1 -- 172.21.15.86:0/2908771795 learned_addr learned my addr 172.21.15.86:0/2908771795 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:10.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.848+0000 7f1fc1000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36dd370 0x558dc36e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.849+0000 7f1fbbe00640 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 0x558dc36e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.849+0000 7f1fc1a00640 1 -- 172.21.15.86:0/2908771795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36dd370 msgr2=0x558dc36e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:10.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.849+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36dd370 0x558dc36e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.850+0000 7f1fc1a00640 1 -- 172.21.15.86:0/2908771795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 msgr2=0x558dc36e7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:10.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.850+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 0x558dc36e7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.850+0000 7f1fc1a00640 1 -- 172.21.15.86:0/2908771795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc33563f0 con 0x558dc36e2ed0 2024-04-17T07:23:10.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.850+0000 7f1fbbe00640 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 0x558dc36e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:10.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.851+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36eeef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4011340 tx=0x7f1fb4011370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad4d5636ab7b8e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.851+0000 7f1fa2a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.851+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2908771795 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb402a060 con 0x558dc36e2ed0 2024-04-17T07:23:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.851+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2908771795 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1fb4004c40 con 0x558dc36e2ed0 2024-04-17T07:23:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.852+0000 7f1fc681b740 1 -- 172.21.15.86:0/2908771795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fb4003e20 con 0x558dc36e2ed0 2024-04-17T07:23:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.852+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2908771795 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fb4004e40 con 0x558dc36e2ed0 2024-04-17T07:23:10.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.852+0000 7f1fc681b740 10 client.26611 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.852+0000 7f1fc681b740 1 -- 172.21.15.86:0/2908771795 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558dc33160f0 con 0x558dc36e2ed0 2024-04-17T07:23:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.853+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2908771795 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fb4003e20 con 0x558dc36e2ed0 2024-04-17T07:23:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.853+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2908771795 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1fb4018020 con 0x558dc36e2ed0 2024-04-17T07:23:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.853+0000 7f1fc681b740 10 client.26611 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.853+0000 7f1fc681b740 10 client.26611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.853+0000 7f1fc681b740 20 client.26611 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:10.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.853+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558dc36f3430 0x558dc36f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.853+0000 7f1fc681b740 4 client.26611 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:10.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.853+0000 7f1fc681b740 1 -- 172.21.15.86:0/2908771795 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dc3651b40 con 0x558dc36f3430 2024-04-17T07:23:10.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.854+0000 7f1fc1000640 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558dc36f3430 0x558dc36f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.855+0000 7f1fc1000640 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558dc36f3430 0x558dc36f58a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18cd91945ceb6d85 server_cookie=4108fc9565da8b60 in_seq=0 out_seq=0 2024-04-17T07:23:10.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.855+0000 7f1fa2a00640 10 client.26611 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:10.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.856+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2908771795 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558dc3651b40 con 0x558dc36f3430 2024-04-17T07:23:10.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.856+0000 7f1fa2a00640 10 client.26611 handle_command_reply: tid=0 2024-04-17T07:23:10.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.856+0000 7f1fc681b740 1 client.26611 shutdown 2024-04-17T07:23:10.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.857+0000 7f1fc681b740 1 -- 172.21.15.86:0/2908771795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 msgr2=0x558dc36eeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:10.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.857+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36eeef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4011340 tx=0x7f1fb4011370 comp rx=0 tx=0).stop 2024-04-17T07:23:10.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.857+0000 7f1fc681b740 1 -- 172.21.15.86:0/2908771795 shutdown_connections 2024-04-17T07:23:10.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.857+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558dc36f3430 0x558dc36f58a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.857+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36eeef0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.857+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 0x558dc36e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.857+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2908771795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36dd370 0x558dc36e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.857+0000 7f1fa2a00640 0 client.26611 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:10.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.857+0000 7f1fc681b740 1 -- 172.21.15.86:0/2908771795 >> 172.21.15.86:0/2908771795 conn(0x558dc36d88e0 msgr2=0x558dc36d7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:10.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.858+0000 7f1fc681b740 1 -- 172.21.15.86:0/2908771795 shutdown_connections 2024-04-17T07:23:10.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.858+0000 7f1fc681b740 1 -- 172.21.15.86:0/2908771795 wait complete. 2024-04-17T07:23:10.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.858+0000 7f1fc681b740 20 client.26611 trim_cache size 0 max 16384 2024-04-17T07:23:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.895+0000 7f1fc681b740 1 Processor -- start 2024-04-17T07:23:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.895+0000 7f1fc681b740 1 -- start start 2024-04-17T07:23:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.895+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36dd370 0x558dc36e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.895+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36ddd60 0x558dc36eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.895+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.895+0000 7f1fc681b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1b96b0 con 0x558dc36dd370 2024-04-17T07:23:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.895+0000 7f1fc681b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1bb1f0 con 0x558dc36ddd60 2024-04-17T07:23:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.895+0000 7f1fc681b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1b9d80 con 0x558dc36e2ed0 2024-04-17T07:23:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.896+0000 7f1fbbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36ddd60 0x558dc36eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.896+0000 7f1fc1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36dd370 0x558dc36e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.896+0000 7f1fbbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36ddd60 0x558dc36eeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43316/0 (socket says 172.21.15.86:43316) 2024-04-17T07:23:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.896+0000 7f1fbbe00640 1 -- 172.21.15.86:0/88350907 learned_addr learned my addr 172.21.15.86:0/88350907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.896+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/88350907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.897+0000 7f1fc1000640 1 -- 172.21.15.86:0/88350907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 msgr2=0x558dc36e68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.897+0000 7f1fc1000640 1 --2- 172.21.15.86:0/88350907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36e68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.897+0000 7f1fc1000640 1 -- 172.21.15.86:0/88350907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36ddd60 msgr2=0x558dc36eeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.897+0000 7f1fc1000640 1 --2- 172.21.15.86:0/88350907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36ddd60 0x558dc36eeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.897+0000 7f1fc1000640 1 -- 172.21.15.86:0/88350907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc36d9330 con 0x558dc36dd370 2024-04-17T07:23:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.897+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/88350907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.897+0000 7f1fc1000640 1 --2- 172.21.15.86:0/88350907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36dd370 0x558dc36e63a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00f0110 tx=0x7f1fb00f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297eea46f092bb63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.898+0000 7f1fad600640 1 -- 172.21.15.86:0/88350907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00056a0 con 0x558dc36dd370 2024-04-17T07:23:10.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.898+0000 7f1fad600640 1 -- 172.21.15.86:0/88350907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fb00bf020 con 0x558dc36dd370 2024-04-17T07:23:10.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.898+0000 7f1fad600640 1 -- 172.21.15.86:0/88350907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00bcc20 con 0x558dc36dd370 2024-04-17T07:23:10.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.900+0000 7f1fc681b740 1 -- 172.21.15.86:0/88350907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36dd370 msgr2=0x558dc36e63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:10.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.900+0000 7f1fc681b740 1 --2- 172.21.15.86:0/88350907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36dd370 0x558dc36e63a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00f0110 tx=0x7f1fb00f2040 comp rx=0 tx=0).stop 2024-04-17T07:23:10.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.901+0000 7f1fc681b740 1 -- 172.21.15.86:0/88350907 shutdown_connections 2024-04-17T07:23:10.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.901+0000 7f1fc681b740 1 --2- 172.21.15.86:0/88350907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36e2ed0 0x558dc36e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.901+0000 7f1fc681b740 1 --2- 172.21.15.86:0/88350907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36ddd60 0x558dc36eeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.901+0000 7f1fc681b740 1 --2- 172.21.15.86:0/88350907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36dd370 0x558dc36e63a0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.901+0000 7f1fc681b740 1 -- 172.21.15.86:0/88350907 >> 172.21.15.86:0/88350907 conn(0x558dc36d88e0 msgr2=0x558dc36e7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:10.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.901+0000 7f1fc681b740 1 -- 172.21.15.86:0/88350907 shutdown_connections 2024-04-17T07:23:10.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.901+0000 7f1fc681b740 1 -- 172.21.15.86:0/88350907 wait complete. 2024-04-17T07:23:10.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.902+0000 7f1fc681b740 1 -- start start 2024-04-17T07:23:10.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.902+0000 7f1fc681b740 1 Processor -- start 2024-04-17T07:23:10.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.903+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36dd370 0x558dc36ebb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.903+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 0x558dc36ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.903+0000 7f1fc681b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36e2ed0 0x558dc37579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.903+0000 7f1fc681b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1bab20 con 0x558dc36ddd60 2024-04-17T07:23:10.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.903+0000 7f1fc681b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1ba450 con 0x558dc36e2ed0 2024-04-17T07:23:10.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.903+0000 7f1fc681b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1b7680 con 0x558dc36dd370 2024-04-17T07:23:10.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.903+0000 7f1fc681b740 10 client.0 _ll_register_callbacks cb 0x7f1fb0004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:10.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.903+0000 7f1fc1a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36e2ed0 0x558dc37579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.904+0000 7f1fc1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36dd370 0x558dc36ebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.904+0000 7f1fc1a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36e2ed0 0x558dc37579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43318/0 (socket says 172.21.15.86:43318) 2024-04-17T07:23:10.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.904+0000 7f1fc1a00640 1 -- 172.21.15.86:0/2230693864 learned_addr learned my addr 172.21.15.86:0/2230693864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:10.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.904+0000 7f1fbbe00640 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 0x558dc36ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.904+0000 7f1fc1a00640 1 -- 172.21.15.86:0/2230693864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36dd370 msgr2=0x558dc36ebb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:10.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.904+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36dd370 0x558dc36ebb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.905+0000 7f1fc1a00640 1 -- 172.21.15.86:0/2230693864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 msgr2=0x558dc36ec0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:10.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.905+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 0x558dc36ec0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.905+0000 7f1fc1a00640 1 -- 172.21.15.86:0/2230693864 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc36ec690 con 0x558dc36e2ed0 2024-04-17T07:23:10.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.906+0000 7f1fc1000640 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36dd370 0x558dc36ebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:10.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.906+0000 7f1fbbe00640 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 0x558dc36ec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:10.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.907+0000 7f1fc1a00640 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36e2ed0 0x558dc37579d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4003cc0 tx=0x7f1fb4011760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2691d0ac43ef0094 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:10.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.907+0000 7f1fa2a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:23:10.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.908+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2230693864 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb402a040 con 0x558dc36e2ed0 2024-04-17T07:23:10.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.908+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2230693864 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1fb400fba0 con 0x558dc36e2ed0 2024-04-17T07:23:10.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.908+0000 7f1fc681b740 1 -- 172.21.15.86:0/2230693864 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fb00bf020 con 0x558dc36e2ed0 2024-04-17T07:23:10.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.910+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2230693864 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fb4034080 con 0x558dc36e2ed0 2024-04-17T07:23:10.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.910+0000 7f1fc681b740 10 client.36596 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:10.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.910+0000 7f1fc681b740 1 -- 172.21.15.86:0/2230693864 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558dc3758f90 con 0x558dc36e2ed0 2024-04-17T07:23:10.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.911+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2230693864 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fb00bf020 con 0x558dc36e2ed0 2024-04-17T07:23:10.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.912+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2230693864 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1fb4018070 con 0x558dc36e2ed0 2024-04-17T07:23:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.913+0000 7f1fc681b740 10 client.36596 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.913+0000 7f1fc681b740 10 client.36596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.913+0000 7f1fc681b740 20 client.36596 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.913+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558dc3759930 0x558dc375bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.913+0000 7f1fc681b740 4 client.36596 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:23:10.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.913+0000 7f1fc681b740 1 -- 172.21.15.86:0/2230693864 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558dc364b9b0 con 0x558dc3759930 2024-04-17T07:23:10.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.914+0000 7f1fc1000640 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558dc3759930 0x558dc375bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:10.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.915+0000 7f1fc1000640 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558dc3759930 0x558dc375bd50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56e545ac711688a server_cookie=8b68a017562ca7d6 in_seq=0 out_seq=0 2024-04-17T07:23:10.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.915+0000 7f1fa2a00640 10 client.36596 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:10.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.915+0000 7f1fa2a00640 1 -- 172.21.15.86:0/2230693864 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558dc364b9b0 con 0x558dc3759930 2024-04-17T07:23:10.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.915+0000 7f1fa2a00640 10 client.36596 handle_command_reply: tid=0 2024-04-17T07:23:10.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.916+0000 7f1fc681b740 1 client.36596 shutdown 2024-04-17T07:23:10.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.916+0000 7f1fc681b740 1 -- 172.21.15.86:0/2230693864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36e2ed0 msgr2=0x558dc37579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:10.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.916+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36e2ed0 0x558dc37579d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4003cc0 tx=0x7f1fb4011760 comp rx=0 tx=0).stop 2024-04-17T07:23:10.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.916+0000 7f1fc681b740 1 -- 172.21.15.86:0/2230693864 shutdown_connections 2024-04-17T07:23:10.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.916+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558dc3759930 0x558dc375bd50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.917+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dc36e2ed0 0x558dc37579d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.917+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558dc36ddd60 0x558dc36ec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.917+0000 7f1fc681b740 1 --2- 172.21.15.86:0/2230693864 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558dc36dd370 0x558dc36ebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.917+0000 7f1fc681b740 1 -- 172.21.15.86:0/2230693864 >> 172.21.15.86:0/2230693864 conn(0x558dc36d88e0 msgr2=0x558dc36e7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:10.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.917+0000 7f1fa2a00640 0 client.36596 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:10.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.917+0000 7f1fc681b740 1 -- 172.21.15.86:0/2230693864 shutdown_connections 2024-04-17T07:23:10.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.917+0000 7f1fc681b740 1 -- 172.21.15.86:0/2230693864 wait complete. 2024-04-17T07:23:10.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.918+0000 7f1fc681b740 20 client.36596 trim_cache size 0 max 16384 2024-04-17T07:23:10.918 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:23:10.919 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:23:10.919 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:23:10.919 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.919+0000 7f1fc2400640 1 -- 172.21.15.86:0/3315858500 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1fa40a3c30 msgr2=0x7f1fa40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.919+0000 7f1fc2400640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1fa40a3c30 0x7f1fa40a60f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80058a0 tx=0x7f1fa800b7e0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.919+0000 7f1fc2400640 1 -- 172.21.15.86:0/3315858500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fbc1006e0 msgr2=0x7f1fbc10bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.919+0000 7f1fc2400640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fbc1006e0 0x7f1fbc10bcb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0000c00 tx=0x7f1fb00042f0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.919+0000 7f1fc2400640 1 -- 172.21.15.86:0/3315858500 shutdown_connections 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.919+0000 7f1fc2400640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1fa40a3c30 0x7f1fa40a60f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.919+0000 7f1fc2400640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1fbc105a00 0x7f1fbc1ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.919+0000 7f1fc2400640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fbc103050 0x7f1fbc10c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.919+0000 7f1fc2400640 1 --2- 172.21.15.86:0/3315858500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fbc1006e0 0x7f1fbc10bcb0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.919+0000 7f1fc2400640 1 -- 172.21.15.86:0/3315858500 >> 172.21.15.86:0/3315858500 conn(0x7f1fbc0fc300 msgr2=0x7f1fbc103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.920+0000 7f1fc2400640 1 -- 172.21.15.86:0/3315858500 shutdown_connections 2024-04-17T07:23:10.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:10.920+0000 7f1fc2400640 1 -- 172.21.15.86:0/3315858500 wait complete. 2024-04-17T07:23:10.947 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:23:10.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:23:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.256+0000 7fb487200640 1 -- 172.21.15.86:0/2934094707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4880ffc70 msgr2=0x7fb488100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.256+0000 7fb487200640 1 --2- 172.21.15.86:0/2934094707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4880ffc70 0x7fb488100090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb478009860 tx=0x7fb47802bb70 comp rx=0 tx=0).stop 2024-04-17T07:23:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.257+0000 7fb487200640 1 -- 172.21.15.86:0/2934094707 shutdown_connections 2024-04-17T07:23:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.257+0000 7fb487200640 1 --2- 172.21.15.86:0/2934094707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb488101740 0x7fb488106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.257+0000 7fb487200640 1 --2- 172.21.15.86:0/2934094707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb488100da0 0x7fb488101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.257+0000 7fb487200640 1 --2- 172.21.15.86:0/2934094707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4880ffc70 0x7fb488100090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.257+0000 7fb487200640 1 -- 172.21.15.86:0/2934094707 >> 172.21.15.86:0/2934094707 conn(0x7fb4880fb590 msgr2=0x7fb4880fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.257+0000 7fb487200640 1 -- 172.21.15.86:0/2934094707 shutdown_connections 2024-04-17T07:23:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.259+0000 7fb487200640 1 -- 172.21.15.86:0/2934094707 wait complete. 2024-04-17T07:23:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.261+0000 7fb487200640 1 Processor -- start 2024-04-17T07:23:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.261+0000 7fb487200640 1 -- start start 2024-04-17T07:23:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.261+0000 7fb487200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb488100da0 0x7fb48810cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.261+0000 7fb487200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb488101740 0x7fb48810d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:11.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.261+0000 7fb487200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb48810e0a0 0x7fb48810e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:11.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.261+0000 7fb487200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb488113f20 con 0x7fb488100da0 2024-04-17T07:23:11.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.261+0000 7fb487200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb488113da0 con 0x7fb48810e0a0 2024-04-17T07:23:11.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.261+0000 7fb487200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881140a0 con 0x7fb488101740 2024-04-17T07:23:11.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.262+0000 7fb485400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb488101740 0x7fb48810d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.262+0000 7fb485400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb488101740 0x7fb48810d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41050/0 (socket says 172.21.15.86:41050) 2024-04-17T07:23:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.262+0000 7fb485400640 1 -- 172.21.15.86:0/2933919536 learned_addr learned my addr 172.21.15.86:0/2933919536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.263+0000 7fb485e00640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb488100da0 0x7fb48810cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.263+0000 7fb486800640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb48810e0a0 0x7fb48810e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.263+0000 7fb485400640 1 -- 172.21.15.86:0/2933919536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb48810e0a0 msgr2=0x7fb48810e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:11.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.263+0000 7fb485400640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb48810e0a0 0x7fb48810e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.263+0000 7fb485400640 1 -- 172.21.15.86:0/2933919536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb488100da0 msgr2=0x7fb48810cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.263+0000 7fb485400640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb488100da0 0x7fb48810cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.263+0000 7fb485400640 1 -- 172.21.15.86:0/2933919536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48810eb10 con 0x7fb488101740 2024-04-17T07:23:11.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.264+0000 7fb485400640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb488101740 0x7fb48810d060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb47400e9c0 tx=0x7fb47400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.265+0000 7fb47ea00640 1 -- 172.21.15.86:0/2933919536 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47400ccc0 con 0x7fb488101740 2024-04-17T07:23:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.265+0000 7fb47ea00640 1 -- 172.21.15.86:0/2933919536 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4740054d0 con 0x7fb488101740 2024-04-17T07:23:11.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.265+0000 7fb47ea00640 1 -- 172.21.15.86:0/2933919536 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474010430 con 0x7fb488101740 2024-04-17T07:23:11.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.265+0000 7fb487200640 1 -- 172.21.15.86:0/2933919536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48810ee00 con 0x7fb488101740 2024-04-17T07:23:11.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.266+0000 7fb487200640 1 -- 172.21.15.86:0/2933919536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48810f300 con 0x7fb488101740 2024-04-17T07:23:11.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.267+0000 7fb487200640 1 -- 172.21.15.86:0/2933919536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48810fd80 con 0x7fb488101740 2024-04-17T07:23:11.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.270+0000 7fb47ea00640 1 -- 172.21.15.86:0/2933919536 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb474005020 con 0x7fb488101740 2024-04-17T07:23:11.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.272+0000 7fb47ea00640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb46c0a3db0 0x7fb46c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:11.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.272+0000 7fb47ea00640 1 -- 172.21.15.86:0/2933919536 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb4740d0b80 con 0x7fb488101740 2024-04-17T07:23:11.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.272+0000 7fb485e00640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb46c0a3db0 0x7fb46c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.274+0000 7fb485e00640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb46c0a3db0 0x7fb46c0a6270 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb47802c100 tx=0x7fb478005780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:11.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.283+0000 7fb47ea00640 1 -- 172.21.15.86:0/2933919536 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb47409d180 con 0x7fb488101740 2024-04-17T07:23:11.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.802+0000 7fb487200640 1 -- 172.21.15.86:0/2933919536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb488104880 con 0x7fb488101740 2024-04-17T07:23:11.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.804+0000 7fb47ea00640 1 -- 172.21.15.86:0/2933919536 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fb47409ca80 con 0x7fb488101740 2024-04-17T07:23:11.805 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:23:11.805 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:23:11.805 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:23:11.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.807+0000 7fb487200640 1 -- 172.21.15.86:0/2933919536 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb46c0a3db0 msgr2=0x7fb46c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:11.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.807+0000 7fb487200640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb46c0a3db0 0x7fb46c0a6270 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb47802c100 tx=0x7fb478005780 comp rx=0 tx=0).stop 2024-04-17T07:23:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.807+0000 7fb487200640 1 -- 172.21.15.86:0/2933919536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb488101740 msgr2=0x7fb48810d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.807+0000 7fb487200640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb488101740 0x7fb48810d060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb47400e9c0 tx=0x7fb47400ee80 comp rx=0 tx=0).stop 2024-04-17T07:23:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.808+0000 7fb487200640 1 -- 172.21.15.86:0/2933919536 shutdown_connections 2024-04-17T07:23:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.808+0000 7fb487200640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb46c0a3db0 0x7fb46c0a6270 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.808+0000 7fb487200640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb48810e0a0 0x7fb48810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.808+0000 7fb487200640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb488101740 0x7fb48810d060 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.808+0000 7fb487200640 1 --2- 172.21.15.86:0/2933919536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb488100da0 0x7fb48810cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.808+0000 7fb487200640 1 -- 172.21.15.86:0/2933919536 >> 172.21.15.86:0/2933919536 conn(0x7fb4880fb590 msgr2=0x7fb488101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.809+0000 7fb487200640 1 -- 172.21.15.86:0/2933919536 shutdown_connections 2024-04-17T07:23:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:11.809+0000 7fb487200640 1 -- 172.21.15.86:0/2933919536 wait complete. 2024-04-17T07:23:11.839 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:23:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.152+0000 7fb4ca000640 1 -- 172.21.15.86:0/1560565822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4c4105d70 msgr2=0x7fb4c410c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.152+0000 7fb4ca000640 1 --2- 172.21.15.86:0/1560565822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4c4105d70 0x7fb4c410c600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009ca0 tx=0x7fb4bc02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.152+0000 7fb4ca000640 1 -- 172.21.15.86:0/1560565822 shutdown_connections 2024-04-17T07:23:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.152+0000 7fb4ca000640 1 --2- 172.21.15.86:0/1560565822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4c4105d70 0x7fb4c410c600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.152+0000 7fb4ca000640 1 --2- 172.21.15.86:0/1560565822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41053b0 0x7fb4c4105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.152+0000 7fb4ca000640 1 --2- 172.21.15.86:0/1560565822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4c41041c0 0x7fb4c41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.152+0000 7fb4ca000640 1 -- 172.21.15.86:0/1560565822 >> 172.21.15.86:0/1560565822 conn(0x7fb4c40ff750 msgr2=0x7fb4c4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.153+0000 7fb4ca000640 1 -- 172.21.15.86:0/1560565822 shutdown_connections 2024-04-17T07:23:12.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.153+0000 7fb4ca000640 1 -- 172.21.15.86:0/1560565822 wait complete. 2024-04-17T07:23:12.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.155+0000 7fb4ca000640 1 Processor -- start 2024-04-17T07:23:12.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.155+0000 7fb4ca000640 1 -- start start 2024-04-17T07:23:12.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.156+0000 7fb4ca000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41041c0 0x7fb4c419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.156+0000 7fb4ca000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4c41053b0 0x7fb4c419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.156+0000 7fb4ca000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4c4105d70 0x7fb4c41a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.156+0000 7fb4ca000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c410fd00 con 0x7fb4c41041c0 2024-04-17T07:23:12.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.156+0000 7fb4ca000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c410fb80 con 0x7fb4c4105d70 2024-04-17T07:23:12.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.156+0000 7fb4ca000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c410fe80 con 0x7fb4c41053b0 2024-04-17T07:23:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.156+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41041c0 0x7fb4c419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.156+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41041c0 0x7fb4c419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35658/0 (socket says 172.21.15.86:35658) 2024-04-17T07:23:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.156+0000 7fb4c3400640 1 -- 172.21.15.86:0/2270792922 learned_addr learned my addr 172.21.15.86:0/2270792922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:12.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.156+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4c41053b0 0x7fb4c419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.157+0000 7fb4c3400640 1 -- 172.21.15.86:0/2270792922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4c41053b0 msgr2=0x7fb4c419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.157+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4c41053b0 0x7fb4c419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.157+0000 7fb4c3400640 1 -- 172.21.15.86:0/2270792922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4c4105d70 msgr2=0x7fb4c41a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.157+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4c4105d70 0x7fb4c41a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.157+0000 7fb4c3400640 1 -- 172.21.15.86:0/2270792922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c406bab0 con 0x7fb4c41041c0 2024-04-17T07:23:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.158+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41041c0 0x7fb4c419e7b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e9c0 tx=0x7fb4b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.158+0000 7fb4bbe00640 1 -- 172.21.15.86:0/2270792922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400cd80 con 0x7fb4c41041c0 2024-04-17T07:23:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.159+0000 7fb4ca000640 1 -- 172.21.15.86:0/2270792922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c406bda0 con 0x7fb4c41041c0 2024-04-17T07:23:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.159+0000 7fb4bbe00640 1 -- 172.21.15.86:0/2270792922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4b40035f0 con 0x7fb4c41041c0 2024-04-17T07:23:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.159+0000 7fb4bbe00640 1 -- 172.21.15.86:0/2270792922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4010690 con 0x7fb4c41041c0 2024-04-17T07:23:12.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.159+0000 7fb4ca000640 1 -- 172.21.15.86:0/2270792922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c406c340 con 0x7fb4c41041c0 2024-04-17T07:23:12.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.164+0000 7fb4ccd6a740 1 Processor -- start 2024-04-17T07:23:12.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.165+0000 7fb4ccd6a740 1 -- start start 2024-04-17T07:23:12.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.165+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222bfa0 0x55b77222c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.165+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222c990 0x55b772231490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.165+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b772231b00 0x55b772233f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.165+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b772192020 con 0x55b772231b00 2024-04-17T07:23:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.165+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b772138460 con 0x55b77222bfa0 2024-04-17T07:23:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.165+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b77218dfe0 con 0x55b77222c990 2024-04-17T07:23:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.166+0000 7fb4c3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222bfa0 0x55b77222c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.166+0000 7fb4c3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222bfa0 0x55b77222c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43366/0 (socket says 172.21.15.86:43366) 2024-04-17T07:23:12.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.166+0000 7fb4c3e00640 1 -- 172.21.15.86:0/1538998827 learned_addr learned my addr 172.21.15.86:0/1538998827 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.167+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/1538998827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b772231b00 0x55b772233f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.167+0000 7fb4bbe00640 1 -- 172.21.15.86:0/2270792922 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb4b40037a0 con 0x7fb4c41041c0 2024-04-17T07:23:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.167+0000 7fb4c3e00640 1 -- 172.21.15.86:0/1538998827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222c990 msgr2=0x55b772231490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:23:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.167+0000 7fb4c3e00640 1 --2- 172.21.15.86:0/1538998827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222c990 0x55b772231490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.167+0000 7fb4c3e00640 1 -- 172.21.15.86:0/1538998827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b772231b00 msgr2=0x55b772233f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.167+0000 7fb4c3e00640 1 --2- 172.21.15.86:0/1538998827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b772231b00 0x55b772233f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.168+0000 7fb4c3e00640 1 -- 172.21.15.86:0/1538998827 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b772140040 con 0x55b77222bfa0 2024-04-17T07:23:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.168+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/1538998827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b772231b00 0x55b772233f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:12.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.168+0000 7fb4c3e00640 1 --2- 172.21.15.86:0/1538998827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222bfa0 0x55b77222c3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc02c630 tx=0x7fb4bc005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeb8f194a51792d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.169+0000 7fb4afe00640 1 -- 172.21.15.86:0/1538998827 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc041020 con 0x55b77222bfa0 2024-04-17T07:23:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.169+0000 7fb4afe00640 1 -- 172.21.15.86:0/1538998827 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4bc032bb0 con 0x55b77222bfa0 2024-04-17T07:23:12.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.169+0000 7fb4afe00640 1 -- 172.21.15.86:0/1538998827 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc039040 con 0x55b77222bfa0 2024-04-17T07:23:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.170+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/1538998827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222bfa0 msgr2=0x55b77222c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.171+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/1538998827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222bfa0 0x55b77222c3c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc02c630 tx=0x7fb4bc005f10 comp rx=0 tx=0).stop 2024-04-17T07:23:12.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.171+0000 7fb4bbe00640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4a80a3cb0 0x7fb4a80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.172+0000 7fb4c3e00640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4a80a3cb0 0x7fb4a80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.172+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/1538998827 shutdown_connections 2024-04-17T07:23:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.172+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/1538998827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b772231b00 0x55b772233f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.172+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/1538998827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222c990 0x55b772231490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.172+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/1538998827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222bfa0 0x55b77222c3c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.172+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/1538998827 >> 172.21.15.86:0/1538998827 conn(0x55b772227510 msgr2=0x55b772227910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.172+0000 7fb4bbe00640 1 -- 172.21.15.86:0/2270792922 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb4b40d0bb0 con 0x7fb4c41041c0 2024-04-17T07:23:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.174+0000 7fb4c3e00640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4a80a3cb0 0x7fb4a80a6170 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc004af0 tx=0x7fb4bc02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.174+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/1538998827 shutdown_connections 2024-04-17T07:23:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.175+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/1538998827 wait complete. 2024-04-17T07:23:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.176+0000 7fb4ccd6a740 1 -- start start 2024-04-17T07:23:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.176+0000 7fb4ccd6a740 1 Processor -- start 2024-04-17T07:23:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.177+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b7722364c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.177+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222c990 0x55b772236a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.177+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b772231b00 0x55b77223da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.177+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b772234a50 con 0x55b77222bfa0 2024-04-17T07:23:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.177+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b77214b830 con 0x55b772231b00 2024-04-17T07:23:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.177+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b772234bd0 con 0x55b77222c990 2024-04-17T07:23:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.177+0000 7fb4ccd6a740 10 client.0 _ll_register_callbacks cb 0x55b772119fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:12.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.178+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b7722364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.178+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b7722364c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35678/0 (socket says 172.21.15.86:35678) 2024-04-17T07:23:12.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.178+0000 7fb4c3400640 1 -- 172.21.15.86:0/2098220335 learned_addr learned my addr 172.21.15.86:0/2098220335 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:12.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.178+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222c990 0x55b772236a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.178+0000 7fb4c3e00640 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b772231b00 0x55b77223da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.179+0000 7fb4c3400640 1 -- 172.21.15.86:0/2098220335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222c990 msgr2=0x55b772236a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.179+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222c990 0x55b772236a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.179+0000 7fb4c3400640 1 -- 172.21.15.86:0/2098220335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b772231b00 msgr2=0x55b77223da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.179+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b772231b00 0x55b77223da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.179+0000 7fb4c3400640 1 -- 172.21.15.86:0/2098220335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77214f920 con 0x55b77222bfa0 2024-04-17T07:23:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.179+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b7722364c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4098f10 tx=0x7fb4b409c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b786ef4a1976d2d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.179+0000 7fb4acc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:23:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.179+0000 7fb4acc00640 1 -- 172.21.15.86:0/2098220335 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b409d1b0 con 0x55b77222bfa0 2024-04-17T07:23:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.179+0000 7fb4acc00640 1 -- 172.21.15.86:0/2098220335 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb4b40a5290 con 0x55b77222bfa0 2024-04-17T07:23:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.180+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2098220335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4bc032bb0 con 0x55b77222bfa0 2024-04-17T07:23:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.180+0000 7fb4acc00640 1 -- 172.21.15.86:0/2098220335 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4b409cab0 con 0x55b77222bfa0 2024-04-17T07:23:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.180+0000 7fb4ccd6a740 10 client.17667 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.180+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2098220335 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b77211d7f0 con 0x55b77222bfa0 2024-04-17T07:23:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.181+0000 7fb4acc00640 1 -- 172.21.15.86:0/2098220335 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4bc032bb0 con 0x55b77222bfa0 2024-04-17T07:23:12.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.181+0000 7fb4acc00640 1 -- 172.21.15.86:0/2098220335 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb4b409b970 con 0x55b77222bfa0 2024-04-17T07:23:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.181+0000 7fb4ccd6a740 10 client.17667 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.181+0000 7fb4ccd6a740 10 client.17667 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:23:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.181+0000 7fb4ccd6a740 20 client.17667 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.181+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55b772241f40 0x55b772244330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.181+0000 7fb4ccd6a740 4 client.17667 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.181+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2098220335 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b77219fa40 con 0x55b772241f40 2024-04-17T07:23:12.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.182+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55b772241f40 0x55b772244330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.183+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55b772241f40 0x55b772244330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f96f4b3c0dd789 server_cookie=dfc97f374c5ba295 in_seq=0 out_seq=0 2024-04-17T07:23:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.183+0000 7fb4acc00640 10 client.17667 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:23:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.184+0000 7fb4acc00640 1 -- 172.21.15.86:0/2098220335 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b77219fa40 con 0x55b772241f40 2024-04-17T07:23:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.184+0000 7fb4acc00640 10 client.17667 handle_command_reply: tid=0 2024-04-17T07:23:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.184+0000 7fb4ccd6a740 1 client.17667 shutdown 2024-04-17T07:23:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.185+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2098220335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 msgr2=0x55b7722364c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.185+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b7722364c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4098f10 tx=0x7fb4b409c6b0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.186+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2098220335 shutdown_connections 2024-04-17T07:23:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.186+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55b772241f40 0x55b772244330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.186+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b772231b00 0x55b77223da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.186+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222c990 0x55b772236a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.186+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2098220335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b7722364c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.186+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2098220335 >> 172.21.15.86:0/2098220335 conn(0x55b772227510 msgr2=0x55b772226710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.186+0000 7fb4acc00640 0 client.17667 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:23:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.186+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2098220335 shutdown_connections 2024-04-17T07:23:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.186+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2098220335 wait complete. 2024-04-17T07:23:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.187+0000 7fb4ccd6a740 20 client.17667 trim_cache size 0 max 16384 2024-04-17T07:23:12.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.211+0000 7fb4ccd6a740 1 Processor -- start 2024-04-17T07:23:12.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.211+0000 7fb4ccd6a740 1 -- start start 2024-04-17T07:23:12.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.212+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222bfa0 0x55b77222ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.212+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222c990 0x55b77223d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.212+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b772231b00 0x55b772248bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.212+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41bba00 con 0x55b77222c990 2024-04-17T07:23:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.212+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b97f0 con 0x55b772231b00 2024-04-17T07:23:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.212+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41bb330 con 0x55b77222bfa0 2024-04-17T07:23:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.212+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222bfa0 0x55b77222ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.213+0000 7fb4c3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222bfa0 0x55b77222ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41084/0 (socket says 172.21.15.86:41084) 2024-04-17T07:23:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.213+0000 7fb4c3400640 1 -- 172.21.15.86:0/2386365048 learned_addr learned my addr 172.21.15.86:0/2386365048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.213+0000 7fb4c3e00640 1 --2- 172.21.15.86:0/2386365048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b772231b00 0x55b772248bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.214+0000 7fb4c3400640 1 -- 172.21.15.86:0/2386365048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b772231b00 msgr2=0x55b772248bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.214+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2386365048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b772231b00 0x55b772248bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.214+0000 7fb4c3400640 1 -- 172.21.15.86:0/2386365048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222c990 msgr2=0x55b77223d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.214+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/2386365048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222c990 0x55b77223d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.214+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2386365048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222c990 0x55b77223d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.214+0000 7fb4c3400640 1 -- 172.21.15.86:0/2386365048 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b771ea43f0 con 0x55b77222bfa0 2024-04-17T07:23:12.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.214+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/2386365048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222c990 0x55b77223d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:23:12.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.215+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2386365048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222bfa0 0x55b77222ae80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400c6a0 tx=0x7fb4b409c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5536b94aa2a40d0a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.215+0000 7fb4af400640 1 -- 172.21.15.86:0/2386365048 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4080080 con 0x55b77222bfa0 2024-04-17T07:23:12.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.215+0000 7fb4af400640 1 -- 172.21.15.86:0/2386365048 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4b40b6020 con 0x55b77222bfa0 2024-04-17T07:23:12.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.215+0000 7fb4af400640 1 -- 172.21.15.86:0/2386365048 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4098800 con 0x55b77222bfa0 2024-04-17T07:23:12.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.216+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2386365048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222bfa0 msgr2=0x55b77222ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.216+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2386365048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222bfa0 0x55b77222ae80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400c6a0 tx=0x7fb4b409c720 comp rx=0 tx=0).stop 2024-04-17T07:23:12.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.217+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2386365048 shutdown_connections 2024-04-17T07:23:12.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.217+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2386365048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b772231b00 0x55b772248bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.217+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2386365048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222c990 0x55b77223d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.217+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2386365048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b77222bfa0 0x55b77222ae80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.217+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2386365048 >> 172.21.15.86:0/2386365048 conn(0x55b772227510 msgr2=0x55b77222a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.218+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2386365048 shutdown_connections 2024-04-17T07:23:12.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.218+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2386365048 wait complete. 2024-04-17T07:23:12.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.218+0000 7fb4ccd6a740 1 -- start start 2024-04-17T07:23:12.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.218+0000 7fb4ccd6a740 1 Processor -- start 2024-04-17T07:23:12.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.219+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b77223b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.219+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222c990 0x55b77223b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.219+0000 7fb4ccd6a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b772231b00 0x55b77223c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.219+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41ba590 con 0x55b77222bfa0 2024-04-17T07:23:12.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.219+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41b9ec0 con 0x55b77222c990 2024-04-17T07:23:12.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.219+0000 7fb4ccd6a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c41bac60 con 0x55b772231b00 2024-04-17T07:23:12.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.219+0000 7fb4ccd6a740 10 client.0 _ll_register_callbacks cb 0x7fb4b4098670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:12.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.219+0000 7fb4c2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222c990 0x55b77223b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.220+0000 7fb4c2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222c990 0x55b77223b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43396/0 (socket says 172.21.15.86:43396) 2024-04-17T07:23:12.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.220+0000 7fb4c2a00640 1 -- 172.21.15.86:0/2337342769 learned_addr learned my addr 172.21.15.86:0/2337342769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:12.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.220+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b77223b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.220+0000 7fb4c3e00640 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b772231b00 0x55b77223c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.220+0000 7fb4c3400640 1 -- 172.21.15.86:0/2337342769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b772231b00 msgr2=0x55b77223c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.220+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b772231b00 0x55b77223c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.220+0000 7fb4c3400640 1 -- 172.21.15.86:0/2337342769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222c990 msgr2=0x55b77223b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.220+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222c990 0x55b77223b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.220+0000 7fb4c3400640 1 -- 172.21.15.86:0/2337342769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b771f6e7d0 con 0x55b77222bfa0 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.221+0000 7fb4c3400640 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b77223b040 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4099430 tx=0x7fb4b4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ff717e88ed50ea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.221+0000 7fb4acc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.221+0000 7fb4acc00640 1 -- 172.21.15.86:0/2337342769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4080070 con 0x55b77222bfa0 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.221+0000 7fb4acc00640 1 -- 172.21.15.86:0/2337342769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb4b40a5490 con 0x55b77222bfa0 2024-04-17T07:23:12.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.221+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2337342769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b40b6020 con 0x55b77222bfa0 2024-04-17T07:23:12.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4acc00640 1 -- 172.21.15.86:0/2337342769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4b40a23c0 con 0x55b77222bfa0 2024-04-17T07:23:12.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4ccd6a740 10 client.17670 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:12.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2337342769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b77223dbd0 con 0x55b77222bfa0 2024-04-17T07:23:12.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4acc00640 1 -- 172.21.15.86:0/2337342769 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4b40b6020 con 0x55b77222bfa0 2024-04-17T07:23:12.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4acc00640 1 -- 172.21.15.86:0/2337342769 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb4b4098020 con 0x55b77222bfa0 2024-04-17T07:23:12.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4ccd6a740 10 client.17670 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:12.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4ccd6a740 10 client.17670 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:23:12.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4ccd6a740 20 client.17670 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:12.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55b7722a8850 0x55b7722aac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4ccd6a740 4 client.17670 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:23:12.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.222+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2337342769 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7721998b0 con 0x55b7722a8850 2024-04-17T07:23:12.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.224+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55b7722a8850 0x55b7722aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.226+0000 7fb4acc00640 10 client.17670 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:23:12.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.226+0000 7fb4c2a00640 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55b7722a8850 0x55b7722aac70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6050d0b5325c1c9 server_cookie=60a13ffa725b17a0 in_seq=0 out_seq=0 2024-04-17T07:23:12.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.226+0000 7fb4acc00640 1 -- 172.21.15.86:0/2337342769 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7721998b0 con 0x55b7722a8850 2024-04-17T07:23:12.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.227+0000 7fb4acc00640 10 client.17670 handle_command_reply: tid=0 2024-04-17T07:23:12.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.227+0000 7fb4ccd6a740 1 client.17670 shutdown 2024-04-17T07:23:12.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.227+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2337342769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 msgr2=0x55b77223b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.227+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b77223b040 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4099430 tx=0x7fb4b4005f10 comp rx=0 tx=0).stop 2024-04-17T07:23:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.227+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2337342769 shutdown_connections 2024-04-17T07:23:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.227+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55b7722a8850 0x55b7722aac70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.228+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b772231b00 0x55b77223c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.228+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b77222c990 0x55b77223b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.228+0000 7fb4ccd6a740 1 --2- 172.21.15.86:0/2337342769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b77222bfa0 0x55b77223b040 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.228+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2337342769 >> 172.21.15.86:0/2337342769 conn(0x55b772227510 msgr2=0x55b77222b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.228+0000 7fb4acc00640 0 client.17670 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:23:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.228+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2337342769 shutdown_connections 2024-04-17T07:23:12.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.228+0000 7fb4ccd6a740 1 -- 172.21.15.86:0/2337342769 wait complete. 2024-04-17T07:23:12.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.229+0000 7fb4ccd6a740 20 client.17670 trim_cache size 0 max 16384 2024-04-17T07:23:12.229 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:23:12.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.231+0000 7fb4ca000640 1 -- 172.21.15.86:0/2270792922 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4a80a3cb0 msgr2=0x7fb4a80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.231+0000 7fb4ca000640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4a80a3cb0 0x7fb4a80a6170 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc004af0 tx=0x7fb4bc02e040 comp rx=0 tx=0).stop 2024-04-17T07:23:12.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.231+0000 7fb4ca000640 1 -- 172.21.15.86:0/2270792922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41041c0 msgr2=0x7fb4c419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.231+0000 7fb4ca000640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41041c0 0x7fb4c419e7b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e9c0 tx=0x7fb4b400ee80 comp rx=0 tx=0).stop 2024-04-17T07:23:12.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.231+0000 7fb4ca000640 1 -- 172.21.15.86:0/2270792922 shutdown_connections 2024-04-17T07:23:12.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.231+0000 7fb4ca000640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4a80a3cb0 0x7fb4a80a6170 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.231+0000 7fb4ca000640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4c4105d70 0x7fb4c41a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.231+0000 7fb4ca000640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4c41053b0 0x7fb4c419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.231+0000 7fb4ca000640 1 --2- 172.21.15.86:0/2270792922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4c41041c0 0x7fb4c419e7b0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.231+0000 7fb4ca000640 1 -- 172.21.15.86:0/2270792922 >> 172.21.15.86:0/2270792922 conn(0x7fb4c40ff750 msgr2=0x7fb4c40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.232+0000 7fb4ca000640 1 -- 172.21.15.86:0/2270792922 shutdown_connections 2024-04-17T07:23:12.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.232+0000 7fb4ca000640 1 -- 172.21.15.86:0/2270792922 wait complete. 2024-04-17T07:23:12.262 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:23:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.571+0000 7f57f6600640 1 -- 172.21.15.86:0/439071710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f0100810 msgr2=0x7f57f0102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.571+0000 7f57f6600640 1 --2- 172.21.15.86:0/439071710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f0100810 0x7f57f0102c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009900 tx=0x7f57e002bc50 comp rx=0 tx=0).stop 2024-04-17T07:23:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.571+0000 7f57f6600640 1 -- 172.21.15.86:0/439071710 shutdown_connections 2024-04-17T07:23:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.571+0000 7f57f6600640 1 --2- 172.21.15.86:0/439071710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0105b30 0x7f57f0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.571+0000 7f57f6600640 1 --2- 172.21.15.86:0/439071710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f0103180 0x7f57f01055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.571+0000 7f57f6600640 1 --2- 172.21.15.86:0/439071710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f0100810 0x7f57f0102c40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.571+0000 7f57f6600640 1 -- 172.21.15.86:0/439071710 >> 172.21.15.86:0/439071710 conn(0x7f57f00fc410 msgr2=0x7f57f00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.571+0000 7f57f6600640 1 -- 172.21.15.86:0/439071710 shutdown_connections 2024-04-17T07:23:12.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.571+0000 7f57f6600640 1 -- 172.21.15.86:0/439071710 wait complete. 2024-04-17T07:23:12.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.573+0000 7f57f6600640 1 Processor -- start 2024-04-17T07:23:12.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.573+0000 7f57f6600640 1 -- start start 2024-04-17T07:23:12.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.573+0000 7f57f6600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f0100810 0x7f57f019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.573+0000 7f57f6600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f0103180 0x7f57f019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.573+0000 7f57f6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0105b30 0x7f57f01a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.574+0000 7f57f6600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f0114190 con 0x7f57f0105b30 2024-04-17T07:23:12.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.574+0000 7f57f6600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57f0114010 con 0x7f57f0100810 2024-04-17T07:23:12.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.574+0000 7f57f6600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57f0114310 con 0x7f57f0103180 2024-04-17T07:23:12.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.574+0000 7f57ef400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f0100810 0x7f57f019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.574+0000 7f57ef400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f0100810 0x7f57f019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43422/0 (socket says 172.21.15.86:43422) 2024-04-17T07:23:12.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.574+0000 7f57ef400640 1 -- 172.21.15.86:0/2936989391 learned_addr learned my addr 172.21.15.86:0/2936989391 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:12.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.574+0000 7f57efe00640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0105b30 0x7f57f01a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.574+0000 7f57eea00640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f0103180 0x7f57f019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.575+0000 7f57ef400640 1 -- 172.21.15.86:0/2936989391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f0103180 msgr2=0x7f57f019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.575+0000 7f57ef400640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f0103180 0x7f57f019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.575+0000 7f57ef400640 1 -- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0105b30 msgr2=0x7f57f01a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.575+0000 7f57ef400640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0105b30 0x7f57f01a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.575+0000 7f57ef400640 1 -- 172.21.15.86:0/2936989391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f01a64c0 con 0x7f57f0100810 2024-04-17T07:23:12.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.575+0000 7f57efe00640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0105b30 0x7f57f01a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:12.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.575+0000 7f57ef400640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f0100810 0x7f57f019e8a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57e0000c00 tx=0x7f57e00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.576+0000 7f57e7e00640 1 -- 172.21.15.86:0/2936989391 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0043020 con 0x7f57f0100810 2024-04-17T07:23:12.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.576+0000 7f57e7e00640 1 -- 172.21.15.86:0/2936989391 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57e0004aa0 con 0x7f57f0100810 2024-04-17T07:23:12.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.576+0000 7f57e7e00640 1 -- 172.21.15.86:0/2936989391 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0003d40 con 0x7f57f0100810 2024-04-17T07:23:12.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.576+0000 7f57f6600640 1 -- 172.21.15.86:0/2936989391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f01a6750 con 0x7f57f0100810 2024-04-17T07:23:12.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.576+0000 7f57f6600640 1 -- 172.21.15.86:0/2936989391 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f01a6cd0 con 0x7f57f0100810 2024-04-17T07:23:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:12 smithi086 ceph-mon[31733]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.6 MiB/s wr, 756 op/s 2024-04-17T07:23:12.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2933919536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:23:12.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.581+0000 7f57e7e00640 1 -- 172.21.15.86:0/2936989391 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f57e0046020 con 0x7f57f0100810 2024-04-17T07:23:12.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.582+0000 7f57f936e740 1 Processor -- start 2024-04-17T07:23:12.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.583+0000 7f57f936e740 1 -- start start 2024-04-17T07:23:12.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.583+0000 7f57f936e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984dfa0 0x56093984e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.583+0000 7f57f936e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 0x560939853490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.583+0000 7f57f936e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 0x560939855f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.583+0000 7f57f936e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5609397b4020 con 0x56093984e990 2024-04-17T07:23:12.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.583+0000 7f57f936e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56093975a460 con 0x56093984dfa0 2024-04-17T07:23:12.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.583+0000 7f57f936e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5609397affe0 con 0x560939853b00 2024-04-17T07:23:12.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.584+0000 7f57eea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 0x560939855f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.584+0000 7f57efe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984dfa0 0x56093984e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.584+0000 7f57eea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 0x560939855f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41100/0 (socket says 172.21.15.86:41100) 2024-04-17T07:23:12.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.584+0000 7f57eea00640 1 -- 172.21.15.86:0/1614692111 learned_addr learned my addr 172.21.15.86:0/1614692111 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:12.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.584+0000 7f57ef400640 1 --2- 172.21.15.86:0/1614692111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 0x560939853490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.584+0000 7f57efe00640 1 -- 172.21.15.86:0/1614692111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 msgr2=0x560939855f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.584+0000 7f57efe00640 1 --2- 172.21.15.86:0/1614692111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 0x560939855f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.585+0000 7f57efe00640 1 -- 172.21.15.86:0/1614692111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 msgr2=0x560939853490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.585+0000 7f57efe00640 1 --2- 172.21.15.86:0/1614692111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 0x560939853490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.585+0000 7f57efe00640 1 -- 172.21.15.86:0/1614692111 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560939762040 con 0x56093984dfa0 2024-04-17T07:23:12.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.585+0000 7f57eea00640 1 --2- 172.21.15.86:0/1614692111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 0x560939855f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:12.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.585+0000 7f57ef400640 1 --2- 172.21.15.86:0/1614692111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 0x560939853490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:23:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.585+0000 7f57efe00640 1 --2- 172.21.15.86:0/1614692111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984dfa0 0x56093984e3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009f90 tx=0x7f57e8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fa4b4ecce33f265 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.586+0000 7f57e4c00640 1 -- 172.21.15.86:0/1614692111 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8011700 con 0x56093984dfa0 2024-04-17T07:23:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.586+0000 7f57e4c00640 1 -- 172.21.15.86:0/1614692111 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57e8011d50 con 0x56093984dfa0 2024-04-17T07:23:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.586+0000 7f57e4c00640 1 -- 172.21.15.86:0/1614692111 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e801ad50 con 0x56093984dfa0 2024-04-17T07:23:12.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.586+0000 7f57e7e00640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57d40a3d80 0x7f57d40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.587+0000 7f57ef400640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57d40a3d80 0x7f57d40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.587+0000 7f57f936e740 1 -- 172.21.15.86:0/1614692111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984dfa0 msgr2=0x56093984e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.587+0000 7f57f936e740 1 --2- 172.21.15.86:0/1614692111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984dfa0 0x56093984e3c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009f90 tx=0x7f57e8010860 comp rx=0 tx=0).stop 2024-04-17T07:23:12.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.588+0000 7f57f936e740 1 -- 172.21.15.86:0/1614692111 shutdown_connections 2024-04-17T07:23:12.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.588+0000 7f57f936e740 1 --2- 172.21.15.86:0/1614692111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 0x560939855f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.588+0000 7f57f936e740 1 --2- 172.21.15.86:0/1614692111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 0x560939853490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.588+0000 7f57f936e740 1 --2- 172.21.15.86:0/1614692111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984dfa0 0x56093984e3c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.588+0000 7f57f936e740 1 -- 172.21.15.86:0/1614692111 >> 172.21.15.86:0/1614692111 conn(0x560939849510 msgr2=0x560939849910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.589+0000 7f57e7e00640 1 -- 172.21.15.86:0/2936989391 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f57e00f2af0 con 0x7f57f0100810 2024-04-17T07:23:12.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.589+0000 7f57f936e740 1 -- 172.21.15.86:0/1614692111 shutdown_connections 2024-04-17T07:23:12.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.589+0000 7f57f936e740 1 -- 172.21.15.86:0/1614692111 wait complete. 2024-04-17T07:23:12.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.590+0000 7f57ef400640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57d40a3d80 0x7f57d40a6240 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f57e00be980 tx=0x7f57e00df040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.590+0000 7f57f936e740 1 -- start start 2024-04-17T07:23:12.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.590+0000 7f57f936e740 1 Processor -- start 2024-04-17T07:23:12.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.592+0000 7f57f936e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 0x560939858520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.592+0000 7f57f936e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 0x560939858a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.592+0000 7f57f936e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560939853b00 0x56093985faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.593+0000 7f57efe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 0x560939858520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.593+0000 7f57efe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 0x560939858520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41112/0 (socket says 172.21.15.86:41112) 2024-04-17T07:23:12.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.593+0000 7f57efe00640 1 -- 172.21.15.86:0/2540815083 learned_addr learned my addr 172.21.15.86:0/2540815083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:12.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.593+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560939856ab0 con 0x56093984e990 2024-04-17T07:23:12.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.593+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56093976d830 con 0x560939853b00 2024-04-17T07:23:12.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.593+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560939856c30 con 0x56093984dfa0 2024-04-17T07:23:12.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.593+0000 7f57efe00640 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560939853b00 0x56093985faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.594+0000 7f57f936e740 10 client.0 _ll_register_callbacks cb 0x56093973bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:12.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.594+0000 7f57efe00640 1 -- 172.21.15.86:0/2540815083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560939853b00 msgr2=0x56093985faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.594+0000 7f57efe00640 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560939853b00 0x56093985faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.594+0000 7f57efe00640 1 -- 172.21.15.86:0/2540815083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 msgr2=0x560939858a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.594+0000 7f57eea00640 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 0x560939858a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.595+0000 7f57efe00640 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 0x560939858a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.595+0000 7f57efe00640 1 -- 172.21.15.86:0/2540815083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560939771920 con 0x56093984dfa0 2024-04-17T07:23:12.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.595+0000 7f57eea00640 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 0x560939858a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:23:12.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.595+0000 7f57efe00640 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 0x560939858520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f57e8011010 tx=0x7f57e801a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=176e2c92a6a8752e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.596+0000 7f57d9600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:12.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.597+0000 7f57d9600640 1 -- 172.21.15.86:0/2540815083 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8024040 con 0x56093984dfa0 2024-04-17T07:23:12.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.597+0000 7f57d9600640 1 -- 172.21.15.86:0/2540815083 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f57e8011500 con 0x56093984dfa0 2024-04-17T07:23:12.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.597+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e8011d50 con 0x56093984dfa0 2024-04-17T07:23:12.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.597+0000 7f57d9600640 1 -- 172.21.15.86:0/2540815083 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57e8011700 con 0x56093984dfa0 2024-04-17T07:23:12.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.597+0000 7f57f936e740 10 client.26647 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:12.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.597+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56093973f7f0 con 0x56093984dfa0 2024-04-17T07:23:12.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.598+0000 7f57d9600640 1 -- 172.21.15.86:0/2540815083 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57e8011d50 con 0x56093984dfa0 2024-04-17T07:23:12.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.598+0000 7f57d9600640 1 -- 172.21.15.86:0/2540815083 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f57e802b460 con 0x56093984dfa0 2024-04-17T07:23:12.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.598+0000 7f57f936e740 10 client.26647 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:12.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.598+0000 7f57f936e740 10 client.26647 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:23:12.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.599+0000 7f57f936e740 20 client.26647 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:12.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.599+0000 7f57f936e740 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560939863ba0 0x560939865ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.599+0000 7f57f936e740 4 client.26647 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:12.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.599+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609397c1a40 con 0x560939863ba0 2024-04-17T07:23:12.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.599+0000 7f57eea00640 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560939863ba0 0x560939865ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.600+0000 7f57eea00640 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560939863ba0 0x560939865ff0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a85d3d5a61827e8 server_cookie=689e4f9cc081cc4b in_seq=0 out_seq=0 2024-04-17T07:23:12.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.600+0000 7f57d9600640 10 client.26647 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:23:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.601+0000 7f57d9600640 1 -- 172.21.15.86:0/2540815083 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609397c1a40 con 0x560939863ba0 2024-04-17T07:23:12.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.601+0000 7f57d9600640 10 client.26647 handle_command_reply: tid=0 2024-04-17T07:23:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.602+0000 7f57f936e740 1 client.26647 shutdown 2024-04-17T07:23:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.602+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 msgr2=0x560939858520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.602+0000 7f57f936e740 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 0x560939858520 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f57e8011010 tx=0x7f57e801a750 comp rx=0 tx=0).stop 2024-04-17T07:23:12.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.603+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 shutdown_connections 2024-04-17T07:23:12.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.603+0000 7f57f936e740 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x560939863ba0 0x560939865ff0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.603+0000 7f57f936e740 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560939853b00 0x56093985faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.603+0000 7f57f936e740 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984e990 0x560939858a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.603+0000 7f57f936e740 1 --2- 172.21.15.86:0/2540815083 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 0x560939858520 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.603+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 >> 172.21.15.86:0/2540815083 conn(0x560939849510 msgr2=0x560939848710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.603+0000 7f57d9600640 0 client.26647 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:23:12.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.604+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 shutdown_connections 2024-04-17T07:23:12.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.605+0000 7f57f936e740 1 -- 172.21.15.86:0/2540815083 wait complete. 2024-04-17T07:23:12.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.606+0000 7f57f936e740 20 client.26647 trim_cache size 0 max 16384 2024-04-17T07:23:12.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.645+0000 7f57f936e740 1 Processor -- start 2024-04-17T07:23:12.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.645+0000 7f57f936e740 1 -- start start 2024-04-17T07:23:12.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.646+0000 7f57f936e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 0x5609398589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.646+0000 7f57f936e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.646+0000 7f57f936e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560939853b00 0x56093984ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.646+0000 7f57f936e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01b1120 con 0x560939853b00 2024-04-17T07:23:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.646+0000 7f57f936e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01b2c60 con 0x56093984e990 2024-04-17T07:23:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.646+0000 7f57f936e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01b17f0 con 0x56093984dfa0 2024-04-17T07:23:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.646+0000 7f57eea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.646+0000 7f57eea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43454/0 (socket says 172.21.15.86:43454) 2024-04-17T07:23:12.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.646+0000 7f57eea00640 1 -- 172.21.15.86:0/3018919213 learned_addr learned my addr 172.21.15.86:0/3018919213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:12.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.647+0000 7f57efe00640 1 --2- 172.21.15.86:0/3018919213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560939853b00 0x56093984ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.647+0000 7f57efe00640 1 --2- 172.21.15.86:0/3018919213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 0x5609398589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.647+0000 7f57eea00640 1 -- 172.21.15.86:0/3018919213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 msgr2=0x5609398589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.648+0000 7f57eea00640 1 --2- 172.21.15.86:0/3018919213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 0x5609398589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.648+0000 7f57eea00640 1 -- 172.21.15.86:0/3018919213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560939853b00 msgr2=0x56093984ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.648+0000 7f57eea00640 1 --2- 172.21.15.86:0/3018919213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560939853b00 0x56093984ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.648+0000 7f57eea00640 1 -- 172.21.15.86:0/3018919213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609394c63f0 con 0x56093984e990 2024-04-17T07:23:12.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.648+0000 7f57efe00640 1 --2- 172.21.15.86:0/3018919213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560939853b00 0x56093984ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:12.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.648+0000 7f57eea00640 1 --2- 172.21.15.86:0/3018919213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985fa30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f57dc000c00 tx=0x7f57dc008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e345a67856a9a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.648+0000 7f57dbe00640 1 -- 172.21.15.86:0/3018919213 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00f040 con 0x56093984e990 2024-04-17T07:23:12.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.649+0000 7f57dbe00640 1 -- 172.21.15.86:0/3018919213 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57dc01e030 con 0x56093984e990 2024-04-17T07:23:12.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.649+0000 7f57dbe00640 1 -- 172.21.15.86:0/3018919213 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0157d0 con 0x56093984e990 2024-04-17T07:23:12.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.649+0000 7f57f936e740 1 -- 172.21.15.86:0/3018919213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 msgr2=0x56093985fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.649+0000 7f57f936e740 1 --2- 172.21.15.86:0/3018919213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985fa30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f57dc000c00 tx=0x7f57dc008040 comp rx=0 tx=0).stop 2024-04-17T07:23:12.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.650+0000 7f57f936e740 1 -- 172.21.15.86:0/3018919213 shutdown_connections 2024-04-17T07:23:12.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.650+0000 7f57f936e740 1 --2- 172.21.15.86:0/3018919213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560939853b00 0x56093984ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.650+0000 7f57f936e740 1 --2- 172.21.15.86:0/3018919213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985fa30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.650+0000 7f57f936e740 1 --2- 172.21.15.86:0/3018919213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56093984dfa0 0x5609398589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.650+0000 7f57f936e740 1 -- 172.21.15.86:0/3018919213 >> 172.21.15.86:0/3018919213 conn(0x560939849510 msgr2=0x56093984c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.651+0000 7f57f936e740 1 -- 172.21.15.86:0/3018919213 shutdown_connections 2024-04-17T07:23:12.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.651+0000 7f57f936e740 1 -- 172.21.15.86:0/3018919213 wait complete. 2024-04-17T07:23:12.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.651+0000 7f57f936e740 1 -- start start 2024-04-17T07:23:12.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.652+0000 7f57f936e740 1 Processor -- start 2024-04-17T07:23:12.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.653+0000 7f57f936e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984dfa0 0x56093985b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.653+0000 7f57f936e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.653+0000 7f57f936e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 0x56093985e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.653+0000 7f57f936e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01b2590 con 0x56093984dfa0 2024-04-17T07:23:12.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.653+0000 7f57f936e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01b1ec0 con 0x56093984e990 2024-04-17T07:23:12.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.653+0000 7f57f936e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57f01af0f0 con 0x560939853b00 2024-04-17T07:23:12.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.653+0000 7f57f936e740 10 client.0 _ll_register_callbacks cb 0x7f57e801abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:12.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.654+0000 7f57efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984dfa0 0x56093985b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.654+0000 7f57efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984dfa0 0x56093985b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35732/0 (socket says 172.21.15.86:35732) 2024-04-17T07:23:12.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.654+0000 7f57efe00640 1 -- 172.21.15.86:0/2808871252 learned_addr learned my addr 172.21.15.86:0/2808871252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:12.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.654+0000 7f57eea00640 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.654+0000 7f57efe00640 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 0x56093985e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.654+0000 7f57eea00640 1 -- 172.21.15.86:0/2808871252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 msgr2=0x56093985e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.654+0000 7f57eea00640 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 0x56093985e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.654+0000 7f57eea00640 1 -- 172.21.15.86:0/2808871252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984dfa0 msgr2=0x56093985b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:12.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.654+0000 7f57eea00640 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984dfa0 0x56093985b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.654+0000 7f57eea00640 1 -- 172.21.15.86:0/2808871252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609395907d0 con 0x56093984e990 2024-04-17T07:23:12.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.655+0000 7f57eea00640 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985b760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f57dc01d040 tx=0x7f57dc017df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb34cedac32764d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:12.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.655+0000 7f57d9600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:23:12.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.656+0000 7f57d9600640 1 -- 172.21.15.86:0/2808871252 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc01a410 con 0x56093984e990 2024-04-17T07:23:12.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.656+0000 7f57d9600640 1 -- 172.21.15.86:0/2808871252 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f57dc01a5b0 con 0x56093984e990 2024-04-17T07:23:12.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.656+0000 7f57f936e740 1 -- 172.21.15.86:0/2808871252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57dc01e030 con 0x56093984e990 2024-04-17T07:23:12.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.657+0000 7f57d9600640 1 -- 172.21.15.86:0/2808871252 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57dc01a7b0 con 0x56093984e990 2024-04-17T07:23:12.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.657+0000 7f57f936e740 10 client.36635 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:12.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.657+0000 7f57f936e740 1 -- 172.21.15.86:0/2808871252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5609398db140 con 0x56093984e990 2024-04-17T07:23:12.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.657+0000 7f57d9600640 1 -- 172.21.15.86:0/2808871252 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57dc01e030 con 0x56093984e990 2024-04-17T07:23:12.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.657+0000 7f57d9600640 1 -- 172.21.15.86:0/2808871252 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f57dc02c6a0 con 0x56093984e990 2024-04-17T07:23:12.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.658+0000 7f57f936e740 10 client.36635 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:12.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.658+0000 7f57f936e740 10 client.36635 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:23:12.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.658+0000 7f57f936e740 20 client.36635 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:12.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.658+0000 7f57f936e740 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5609398db960 0x5609398ddd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:12.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.658+0000 7f57f936e740 4 client.36635 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:23:12.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.658+0000 7f57f936e740 1 -- 172.21.15.86:0/2808871252 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609397bb8b0 con 0x5609398db960 2024-04-17T07:23:12.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.658+0000 7f57efe00640 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5609398db960 0x5609398ddd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:12.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.659+0000 7f57efe00640 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5609398db960 0x5609398ddd80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d29ec92c22db1340 server_cookie=e5662d13989dbf94 in_seq=0 out_seq=0 2024-04-17T07:23:12.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.659+0000 7f57d9600640 10 client.36635 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:23:12.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.660+0000 7f57d9600640 1 -- 172.21.15.86:0/2808871252 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609397bb8b0 con 0x5609398db960 2024-04-17T07:23:12.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.660+0000 7f57d9600640 10 client.36635 handle_command_reply: tid=0 2024-04-17T07:23:12.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.660+0000 7f57f936e740 1 client.36635 shutdown 2024-04-17T07:23:12.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.661+0000 7f57f936e740 1 -- 172.21.15.86:0/2808871252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 msgr2=0x56093985b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.661+0000 7f57f936e740 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985b760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f57dc01d040 tx=0x7f57dc017df0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.661+0000 7f57f936e740 1 -- 172.21.15.86:0/2808871252 shutdown_connections 2024-04-17T07:23:12.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.661+0000 7f57f936e740 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5609398db960 0x5609398ddd80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.661+0000 7f57f936e740 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560939853b00 0x56093985e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.661+0000 7f57f936e740 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56093984e990 0x56093985b760 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.661+0000 7f57f936e740 1 --2- 172.21.15.86:0/2808871252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56093984dfa0 0x56093985b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.661+0000 7f57f936e740 1 -- 172.21.15.86:0/2808871252 >> 172.21.15.86:0/2808871252 conn(0x560939849510 msgr2=0x56093984d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.661+0000 7f57d9600640 0 client.36635 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:23:12.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.662+0000 7f57f936e740 1 -- 172.21.15.86:0/2808871252 shutdown_connections 2024-04-17T07:23:12.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.662+0000 7f57f936e740 1 -- 172.21.15.86:0/2808871252 wait complete. 2024-04-17T07:23:12.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.662+0000 7f57f936e740 20 client.36635 trim_cache size 0 max 16384 2024-04-17T07:23:12.663 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:23:12.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.665+0000 7f57f6600640 1 -- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57d40a3d80 msgr2=0x7f57d40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.665+0000 7f57f6600640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57d40a3d80 0x7f57d40a6240 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f57e00be980 tx=0x7f57e00df040 comp rx=0 tx=0).stop 2024-04-17T07:23:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.665+0000 7f57f6600640 1 -- 172.21.15.86:0/2936989391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f0100810 msgr2=0x7f57f019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.665+0000 7f57f6600640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f0100810 0x7f57f019e8a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57e0000c00 tx=0x7f57e00042f0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.665+0000 7f57f6600640 1 -- 172.21.15.86:0/2936989391 shutdown_connections 2024-04-17T07:23:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.665+0000 7f57f6600640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57d40a3d80 0x7f57d40a6240 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.665+0000 7f57f6600640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f0105b30 0x7f57f01a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.665+0000 7f57f6600640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57f0103180 0x7f57f019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.665+0000 7f57f6600640 1 --2- 172.21.15.86:0/2936989391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f0100810 0x7f57f019e8a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:12.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.665+0000 7f57f6600640 1 -- 172.21.15.86:0/2936989391 >> 172.21.15.86:0/2936989391 conn(0x7f57f00fc410 msgr2=0x7f57f0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.666+0000 7f57f6600640 1 -- 172.21.15.86:0/2936989391 shutdown_connections 2024-04-17T07:23:12.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:12.666+0000 7f57f6600640 1 -- 172.21.15.86:0/2936989391 wait complete. 2024-04-17T07:23:12.695 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:23:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:12 smithi098 ceph-mon[35418]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.6 MiB/s wr, 756 op/s 2024-04-17T07:23:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2933919536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:23:12.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:12 smithi103 ceph-mon[35495]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.6 MiB/s wr, 756 op/s 2024-04-17T07:23:12.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2933919536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:23:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.025+0000 7f9ee3000640 1 -- 172.21.15.86:0/3700797385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9edc102a20 msgr2=0x7f9edc104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.025+0000 7f9ee3000640 1 --2- 172.21.15.86:0/3700797385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9edc102a20 0x7f9edc104e10 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009900 tx=0x7f9ecc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:23:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.025+0000 7f9ee3000640 1 -- 172.21.15.86:0/3700797385 shutdown_connections 2024-04-17T07:23:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.025+0000 7f9ee3000640 1 --2- 172.21.15.86:0/3700797385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9edc107c80 0x7f9edc10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.025+0000 7f9ee3000640 1 --2- 172.21.15.86:0/3700797385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9edc105350 0x7f9edc107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.025+0000 7f9ee3000640 1 --2- 172.21.15.86:0/3700797385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9edc102a20 0x7f9edc104e10 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.025+0000 7f9ee3000640 1 -- 172.21.15.86:0/3700797385 >> 172.21.15.86:0/3700797385 conn(0x7f9edc0fc410 msgr2=0x7f9edc0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.025+0000 7f9ee3000640 1 -- 172.21.15.86:0/3700797385 shutdown_connections 2024-04-17T07:23:13.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.025+0000 7f9ee3000640 1 -- 172.21.15.86:0/3700797385 wait complete. 2024-04-17T07:23:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.027+0000 7f9ee3000640 1 Processor -- start 2024-04-17T07:23:13.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.027+0000 7f9ee3000640 1 -- start start 2024-04-17T07:23:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.028+0000 7f9ee3000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9edc102a20 0x7f9edc196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.028+0000 7f9ee3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9edc105350 0x7f9edc196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.028+0000 7f9ee3000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9edc107c80 0x7f9edc1a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.028+0000 7f9ee3000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10db30 con 0x7f9edc105350 2024-04-17T07:23:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.028+0000 7f9ee3000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10d9b0 con 0x7f9edc102a20 2024-04-17T07:23:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.028+0000 7f9ee3000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc10dcb0 con 0x7f9edc107c80 2024-04-17T07:23:13.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.028+0000 7f9edb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9edc105350 0x7f9edc196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.029+0000 7f9ee1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9edc107c80 0x7f9edc1a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.029+0000 7f9ee1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9edc107c80 0x7f9edc1a5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41158/0 (socket says 172.21.15.86:41158) 2024-04-17T07:23:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.029+0000 7f9ee1000640 1 -- 172.21.15.86:0/3230576093 learned_addr learned my addr 172.21.15.86:0/3230576093 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.029+0000 7f9edbe00640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9edc102a20 0x7f9edc196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.029+0000 7f9ee1000640 1 -- 172.21.15.86:0/3230576093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9edc102a20 msgr2=0x7f9edc196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.029+0000 7f9ee1000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9edc102a20 0x7f9edc196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.029+0000 7f9ee1000640 1 -- 172.21.15.86:0/3230576093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9edc105350 msgr2=0x7f9edc196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.029+0000 7f9ee1000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9edc105350 0x7f9edc196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.029+0000 7f9ee1000640 1 -- 172.21.15.86:0/3230576093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc1a64b0 con 0x7f9edc107c80 2024-04-17T07:23:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.031+0000 7f9ee1000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9edc107c80 0x7f9edc1a5e40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400eaa0 tx=0x7f9ed400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.032+0000 7f9ed8c00640 1 -- 172.21.15.86:0/3230576093 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400cdb0 con 0x7f9edc107c80 2024-04-17T07:23:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.032+0000 7f9ee3000640 1 -- 172.21.15.86:0/3230576093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc1a67a0 con 0x7f9edc107c80 2024-04-17T07:23:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.032+0000 7f9ed8c00640 1 -- 172.21.15.86:0/3230576093 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ed4005020 con 0x7f9edc107c80 2024-04-17T07:23:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.032+0000 7f9ed8c00640 1 -- 172.21.15.86:0/3230576093 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed401f060 con 0x7f9edc107c80 2024-04-17T07:23:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.032+0000 7f9ee3000640 1 -- 172.21.15.86:0/3230576093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc1a6cc0 con 0x7f9edc107c80 2024-04-17T07:23:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.038+0000 7f9ee5d90740 1 Processor -- start 2024-04-17T07:23:13.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.041+0000 7f9ed8c00640 1 -- 172.21.15.86:0/3230576093 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9ed40051d0 con 0x7f9edc107c80 2024-04-17T07:23:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.043+0000 7f9ee5d90740 1 -- start start 2024-04-17T07:23:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.043+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceaafa0 0x563f8ceab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.043+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceab990 0x563f8ceb0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.043+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8ceb0b00 0x563f8ceb2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.043+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f8ce11020 con 0x563f8ceab990 2024-04-17T07:23:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.043+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563f8cdb7460 con 0x563f8ceb0b00 2024-04-17T07:23:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.043+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f8ce0cfe0 con 0x563f8ceaafa0 2024-04-17T07:23:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.045+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceab990 0x563f8ceb0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.045+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceab990 0x563f8ceb0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35754/0 (socket says 172.21.15.86:35754) 2024-04-17T07:23:13.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.045+0000 7f9edbe00640 1 -- 172.21.15.86:0/3550304186 learned_addr learned my addr 172.21.15.86:0/3550304186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.047+0000 7f9edb400640 1 --2- 172.21.15.86:0/3550304186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceaafa0 0x563f8ceab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.047+0000 7f9edb400640 1 --2- 172.21.15.86:0/3550304186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8ceb0b00 0x563f8ceb2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.048+0000 7f9ed8c00640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec00a3c30 0x7f9ec00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.048+0000 7f9ed8c00640 1 -- 172.21.15.86:0/3230576093 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9ed40d08c0 con 0x7f9edc107c80 2024-04-17T07:23:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.049+0000 7f9ee1000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec00a3c30 0x7f9ec00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.049+0000 7f9edbe00640 1 -- 172.21.15.86:0/3550304186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceaafa0 msgr2=0x563f8ceab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.049+0000 7f9edbe00640 1 --2- 172.21.15.86:0/3550304186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceaafa0 0x563f8ceab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.049+0000 7f9edbe00640 1 -- 172.21.15.86:0/3550304186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8ceb0b00 msgr2=0x563f8ceb2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.049+0000 7f9edbe00640 1 --2- 172.21.15.86:0/3550304186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8ceb0b00 0x563f8ceb2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.049+0000 7f9edbe00640 1 -- 172.21.15.86:0/3550304186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8cdbf040 con 0x563f8ceab990 2024-04-17T07:23:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.050+0000 7f9edbe00640 1 --2- 172.21.15.86:0/3550304186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceab990 0x563f8ceb0490 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009230 tx=0x7f9ecc009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbd393c4cfeddec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.050+0000 7f9edb400640 1 --2- 172.21.15.86:0/3550304186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceaafa0 0x563f8ceab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:23:13.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.050+0000 7f9ee1000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec00a3c30 0x7f9ec00a60f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9ed409c7b0 tx=0x7f9ed400cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.051+0000 7f9ed1600640 1 -- 172.21.15.86:0/3550304186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc045020 con 0x563f8ceab990 2024-04-17T07:23:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.051+0000 7f9ed1600640 1 -- 172.21.15.86:0/3550304186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ecc02ca20 con 0x563f8ceab990 2024-04-17T07:23:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.052+0000 7f9ed1600640 1 -- 172.21.15.86:0/3550304186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc0046c0 con 0x563f8ceab990 2024-04-17T07:23:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.053+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3550304186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceab990 msgr2=0x563f8ceb0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.053+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/3550304186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceab990 0x563f8ceb0490 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009230 tx=0x7f9ecc009560 comp rx=0 tx=0).stop 2024-04-17T07:23:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.054+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3550304186 shutdown_connections 2024-04-17T07:23:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.054+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/3550304186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8ceb0b00 0x563f8ceb2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.054+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/3550304186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceab990 0x563f8ceb0490 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.054+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/3550304186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceaafa0 0x563f8ceab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.054+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3550304186 >> 172.21.15.86:0/3550304186 conn(0x563f8cea6510 msgr2=0x563f8cea6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.054+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3550304186 shutdown_connections 2024-04-17T07:23:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.054+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3550304186 wait complete. 2024-04-17T07:23:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.055+0000 7f9ee5d90740 1 -- start start 2024-04-17T07:23:13.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.056+0000 7f9ee5d90740 1 Processor -- start 2024-04-17T07:23:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.056+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cec64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.056+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceab990 0x563f8cec6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.056+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8ceb0b00 0x563f8cecda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.056+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f8cec4a60 con 0x563f8ceaafa0 2024-04-17T07:23:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.056+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563f8cdca830 con 0x563f8ceb0b00 2024-04-17T07:23:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.057+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f8cec4be0 con 0x563f8ceab990 2024-04-17T07:23:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.057+0000 7f9ee5d90740 10 client.0 _ll_register_callbacks cb 0x563f8cd98fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:13.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.057+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceab990 0x563f8cec6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.057+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceab990 0x563f8cec6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41182/0 (socket says 172.21.15.86:41182) 2024-04-17T07:23:13.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.057+0000 7f9edbe00640 1 -- 172.21.15.86:0/136725822 learned_addr learned my addr 172.21.15.86:0/136725822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.057+0000 7f9edb400640 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cec64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.058+0000 7f9edbe00640 1 -- 172.21.15.86:0/136725822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8ceb0b00 msgr2=0x563f8cecda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.058+0000 7f9edbe00640 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8ceb0b00 0x563f8cecda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.058+0000 7f9edbe00640 1 -- 172.21.15.86:0/136725822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 msgr2=0x563f8cec64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.058+0000 7f9edbe00640 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cec64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.058+0000 7f9edbe00640 1 -- 172.21.15.86:0/136725822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8cdce920 con 0x563f8ceab990 2024-04-17T07:23:13.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.058+0000 7f9edb400640 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cec64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:13.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.060+0000 7f9ec6000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:13.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.062+0000 7f9edbe00640 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceab990 0x563f8cec6a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc006010 tx=0x7f9ecc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df3acf899f37115c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.062+0000 7f9ec6000640 1 -- 172.21.15.86:0/136725822 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc045040 con 0x563f8ceab990 2024-04-17T07:23:13.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.062+0000 7f9ec6000640 1 -- 172.21.15.86:0/136725822 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9ecc0037f0 con 0x563f8ceab990 2024-04-17T07:23:13.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.064+0000 7f9ee5d90740 1 -- 172.21.15.86:0/136725822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ecc02ca20 con 0x563f8ceab990 2024-04-17T07:23:13.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.064+0000 7f9ec6000640 1 -- 172.21.15.86:0/136725822 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ecc0039f0 con 0x563f8ceab990 2024-04-17T07:23:13.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.065+0000 7f9ee5d90740 10 client.26662 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:13.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.065+0000 7f9ee5d90740 1 -- 172.21.15.86:0/136725822 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f8cd9c7f0 con 0x563f8ceab990 2024-04-17T07:23:13.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.066+0000 7f9ec6000640 1 -- 172.21.15.86:0/136725822 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ecc02ca20 con 0x563f8ceab990 2024-04-17T07:23:13.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.066+0000 7f9ec6000640 1 -- 172.21.15.86:0/136725822 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9ecc041c80 con 0x563f8ceab990 2024-04-17T07:23:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.067+0000 7f9ee5d90740 10 client.26662 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.067+0000 7f9ee5d90740 10 client.26662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:23:13.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.067+0000 7f9ee5d90740 20 client.26662 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:13.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.067+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f8ced1b20 0x563f8ced3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.067+0000 7f9ee5d90740 4 client.26662 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:13.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.067+0000 7f9ee5d90740 1 -- 172.21.15.86:0/136725822 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f8ce1ea40 con 0x563f8ced1b20 2024-04-17T07:23:13.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.068+0000 7f9edb400640 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f8ced1b20 0x563f8ced3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.069+0000 7f9edb400640 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f8ced1b20 0x563f8ced3f10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61173fc4765b667d server_cookie=7b40964a2d9557b5 in_seq=0 out_seq=0 2024-04-17T07:23:13.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.071+0000 7f9ec6000640 10 client.26662 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:23:13.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.071+0000 7f9ec6000640 1 -- 172.21.15.86:0/136725822 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f8ce1ea40 con 0x563f8ced1b20 2024-04-17T07:23:13.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.071+0000 7f9ec6000640 10 client.26662 handle_command_reply: tid=0 2024-04-17T07:23:13.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.071+0000 7f9ee5d90740 1 client.26662 shutdown 2024-04-17T07:23:13.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.074+0000 7f9ee5d90740 1 -- 172.21.15.86:0/136725822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceab990 msgr2=0x563f8cec6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.074+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceab990 0x563f8cec6a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc006010 tx=0x7f9ecc004270 comp rx=0 tx=0).stop 2024-04-17T07:23:13.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.074+0000 7f9ee5d90740 1 -- 172.21.15.86:0/136725822 shutdown_connections 2024-04-17T07:23:13.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.074+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f8ced1b20 0x563f8ced3f10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.074+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8ceb0b00 0x563f8cecda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.074+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceab990 0x563f8cec6a10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.074+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/136725822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cec64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.074+0000 7f9ee5d90740 1 -- 172.21.15.86:0/136725822 >> 172.21.15.86:0/136725822 conn(0x563f8cea6510 msgr2=0x563f8cea5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.074+0000 7f9ec6000640 0 client.26662 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:23:13.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.075+0000 7f9ee5d90740 1 -- 172.21.15.86:0/136725822 shutdown_connections 2024-04-17T07:23:13.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.075+0000 7f9ee5d90740 1 -- 172.21.15.86:0/136725822 wait complete. 2024-04-17T07:23:13.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.076+0000 7f9ee5d90740 20 client.26662 trim_cache size 0 max 16384 2024-04-17T07:23:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.104+0000 7f9ee5d90740 1 Processor -- start 2024-04-17T07:23:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.104+0000 7f9ee5d90740 1 -- start start 2024-04-17T07:23:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.105+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cec64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.105+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.105+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8cec5690 0x563f8cec6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.105+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1a8940 con 0x563f8ceaafa0 2024-04-17T07:23:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.105+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1aa480 con 0x563f8cec5690 2024-04-17T07:23:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.105+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1a9010 con 0x563f8ceb0b00 2024-04-17T07:23:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.106+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.106+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41190/0 (socket says 172.21.15.86:41190) 2024-04-17T07:23:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.106+0000 7f9edbe00640 1 -- 172.21.15.86:0/2686628380 learned_addr learned my addr 172.21.15.86:0/2686628380 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.106+0000 7f9edb400640 1 --2- 172.21.15.86:0/2686628380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8cec5690 0x563f8cec6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.106+0000 7f9edb400640 1 --2- 172.21.15.86:0/2686628380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cec64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.109+0000 7f9edbe00640 1 -- 172.21.15.86:0/2686628380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8cec5690 msgr2=0x563f8cec6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.109+0000 7f9edbe00640 1 --2- 172.21.15.86:0/2686628380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8cec5690 0x563f8cec6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.109+0000 7f9edbe00640 1 -- 172.21.15.86:0/2686628380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 msgr2=0x563f8cec64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.109+0000 7f9edbe00640 1 --2- 172.21.15.86:0/2686628380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cec64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.109+0000 7f9edbe00640 1 -- 172.21.15.86:0/2686628380 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8cb233f0 con 0x563f8ceb0b00 2024-04-17T07:23:13.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.111+0000 7f9edbe00640 1 --2- 172.21.15.86:0/2686628380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecda10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc004650 tx=0x7f9ecc02c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cecbf6b2264cde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.112+0000 7f9ed0c00640 1 -- 172.21.15.86:0/2686628380 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc0039f0 con 0x563f8ceb0b00 2024-04-17T07:23:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.112+0000 7f9ed0c00640 1 -- 172.21.15.86:0/2686628380 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ecc009520 con 0x563f8ceb0b00 2024-04-17T07:23:13.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.112+0000 7f9ed0c00640 1 -- 172.21.15.86:0/2686628380 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc004e70 con 0x563f8ceb0b00 2024-04-17T07:23:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.113+0000 7f9ee5d90740 1 -- 172.21.15.86:0/2686628380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 msgr2=0x563f8cecda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.113+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/2686628380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecda10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc004650 tx=0x7f9ecc02c7e0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.114+0000 7f9ee5d90740 1 -- 172.21.15.86:0/2686628380 shutdown_connections 2024-04-17T07:23:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.114+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/2686628380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8cec5690 0x563f8cec6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.114+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/2686628380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecda10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.114+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/2686628380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cec64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.114+0000 7f9ee5d90740 1 -- 172.21.15.86:0/2686628380 >> 172.21.15.86:0/2686628380 conn(0x563f8cea6510 msgr2=0x563f8cec4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.115+0000 7f9ee5d90740 1 -- 172.21.15.86:0/2686628380 shutdown_connections 2024-04-17T07:23:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.115+0000 7f9ee5d90740 1 -- 172.21.15.86:0/2686628380 wait complete. 2024-04-17T07:23:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.115+0000 7f9ee5d90740 1 -- start start 2024-04-17T07:23:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.116+0000 7f9ee5d90740 1 Processor -- start 2024-04-17T07:23:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.117+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cecb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.117+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.117+0000 7f9ee5d90740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8cec5690 0x563f8cecc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.117+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1a9db0 con 0x563f8ceaafa0 2024-04-17T07:23:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.117+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1a96e0 con 0x563f8cec5690 2024-04-17T07:23:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.117+0000 7f9ee5d90740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc073ad0 con 0x563f8ceb0b00 2024-04-17T07:23:13.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.117+0000 7f9ee5d90740 10 client.0 _ll_register_callbacks cb 0x7f9ecc02ccc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.118+0000 7f9edb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cecb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.118+0000 7f9edb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8cec5690 0x563f8cecc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.118+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.118+0000 7f9edbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41202/0 (socket says 172.21.15.86:41202) 2024-04-17T07:23:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.118+0000 7f9edbe00640 1 -- 172.21.15.86:0/3052442284 learned_addr learned my addr 172.21.15.86:0/3052442284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.119+0000 7f9edb400640 1 -- 172.21.15.86:0/3052442284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 msgr2=0x563f8cecb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.119+0000 7f9edb400640 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.119+0000 7f9edb400640 1 -- 172.21.15.86:0/3052442284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8cec5690 msgr2=0x563f8cecc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.119+0000 7f9edb400640 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8cec5690 0x563f8cecc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.119+0000 7f9edb400640 1 -- 172.21.15.86:0/3052442284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8cbed7d0 con 0x563f8ceaafa0 2024-04-17T07:23:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.119+0000 7f9edb400640 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cecb1f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x563f8ced46b0 tx=0x7f9ec8016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff847274a30378d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.119+0000 7f9ec6000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:23:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.120+0000 7f9ec6000640 1 -- 172.21.15.86:0/3052442284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8012bb0 con 0x563f8ceaafa0 2024-04-17T07:23:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.120+0000 7f9ec6000640 1 -- 172.21.15.86:0/3052442284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9ec8012d50 con 0x563f8ceaafa0 2024-04-17T07:23:13.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.120+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3052442284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ecc009520 con 0x563f8ceaafa0 2024-04-17T07:23:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.120+0000 7f9ec6000640 1 -- 172.21.15.86:0/3052442284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ec8004020 con 0x563f8ceaafa0 2024-04-17T07:23:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.121+0000 7f9ee5d90740 10 client.17697 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.121+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3052442284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f8cf37f80 con 0x563f8ceaafa0 2024-04-17T07:23:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.121+0000 7f9ec6000640 1 -- 172.21.15.86:0/3052442284 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ecc009520 con 0x563f8ceaafa0 2024-04-17T07:23:13.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.122+0000 7f9ec6000640 1 -- 172.21.15.86:0/3052442284 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9ec8003b70 con 0x563f8ceaafa0 2024-04-17T07:23:13.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.122+0000 7f9ee5d90740 10 client.17697 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:13.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.122+0000 7f9ee5d90740 10 client.17697 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:23:13.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.122+0000 7f9ee5d90740 20 client.17697 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:13.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.122+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f8cf38870 0x563f8cf3ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.122+0000 7f9ee5d90740 4 client.17697 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:23:13.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.122+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3052442284 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f8ce188b0 con 0x563f8cf38870 2024-04-17T07:23:13.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.123+0000 7f9edbe00640 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f8cf38870 0x563f8cf3ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.124+0000 7f9edbe00640 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f8cf38870 0x563f8cf3ac90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2636bb210d97b8e server_cookie=cce770bfa99bf886 in_seq=0 out_seq=0 2024-04-17T07:23:13.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.124+0000 7f9ec6000640 10 client.17697 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:23:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.125+0000 7f9ec6000640 1 -- 172.21.15.86:0/3052442284 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f8ce188b0 con 0x563f8cf38870 2024-04-17T07:23:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.125+0000 7f9ec6000640 10 client.17697 handle_command_reply: tid=0 2024-04-17T07:23:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.125+0000 7f9ee5d90740 1 client.17697 shutdown 2024-04-17T07:23:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.126+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3052442284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 msgr2=0x563f8cecb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.126+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cecb1f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x563f8ced46b0 tx=0x7f9ec8016040 comp rx=0 tx=0).stop 2024-04-17T07:23:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.126+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3052442284 shutdown_connections 2024-04-17T07:23:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.126+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563f8cf38870 0x563f8cf3ac90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.126+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f8cec5690 0x563f8cecc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.126+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f8ceb0b00 0x563f8cecb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.126+0000 7f9ee5d90740 1 --2- 172.21.15.86:0/3052442284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f8ceaafa0 0x563f8cecb1f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.126+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3052442284 >> 172.21.15.86:0/3052442284 conn(0x563f8cea6510 msgr2=0x563f8cec64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.126+0000 7f9ec6000640 0 client.17697 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:23:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.128+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3052442284 shutdown_connections 2024-04-17T07:23:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.128+0000 7f9ee5d90740 1 -- 172.21.15.86:0/3052442284 wait complete. 2024-04-17T07:23:13.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.128+0000 7f9ee5d90740 20 client.17697 trim_cache size 0 max 16384 2024-04-17T07:23:13.130 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:23:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.131+0000 7f9ee3000640 1 -- 172.21.15.86:0/3230576093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec00a3c30 msgr2=0x7f9ec00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.131+0000 7f9ee3000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec00a3c30 0x7f9ec00a60f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9ed409c7b0 tx=0x7f9ed400cf50 comp rx=0 tx=0).stop 2024-04-17T07:23:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.131+0000 7f9ee3000640 1 -- 172.21.15.86:0/3230576093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9edc107c80 msgr2=0x7f9edc1a5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.131+0000 7f9ee3000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9edc107c80 0x7f9edc1a5e40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400eaa0 tx=0x7f9ed400ef60 comp rx=0 tx=0).stop 2024-04-17T07:23:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.132+0000 7f9ee3000640 1 -- 172.21.15.86:0/3230576093 shutdown_connections 2024-04-17T07:23:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.132+0000 7f9ee3000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9ec00a3c30 0x7f9ec00a60f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.132+0000 7f9ee3000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9edc107c80 0x7f9edc1a5e40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.132+0000 7f9ee3000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9edc105350 0x7f9edc196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.132+0000 7f9ee3000640 1 --2- 172.21.15.86:0/3230576093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9edc102a20 0x7f9edc196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.132+0000 7f9ee3000640 1 -- 172.21.15.86:0/3230576093 >> 172.21.15.86:0/3230576093 conn(0x7f9edc0fc410 msgr2=0x7f9edc102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.133+0000 7f9ee3000640 1 -- 172.21.15.86:0/3230576093 shutdown_connections 2024-04-17T07:23:13.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.133+0000 7f9ee3000640 1 -- 172.21.15.86:0/3230576093 wait complete. 2024-04-17T07:23:13.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:23:13.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.499+0000 7f5d84c00640 1 -- 172.21.15.86:0/3215781985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d8006d4c0 msgr2=0x7f5d8006b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.499+0000 7f5d84c00640 1 --2- 172.21.15.86:0/3215781985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d8006d4c0 0x7f5d8006b910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009890 tx=0x7f5d7002bb90 comp rx=0 tx=0).stop 2024-04-17T07:23:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.500+0000 7f5d84c00640 1 -- 172.21.15.86:0/3215781985 shutdown_connections 2024-04-17T07:23:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.500+0000 7f5d84c00640 1 --2- 172.21.15.86:0/3215781985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d80116d70 0x7f5d80119160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.500+0000 7f5d84c00640 1 --2- 172.21.15.86:0/3215781985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8006bee0 0x7f5d8006c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.500+0000 7f5d84c00640 1 --2- 172.21.15.86:0/3215781985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d8006d4c0 0x7f5d8006b910 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.500+0000 7f5d84c00640 1 -- 172.21.15.86:0/3215781985 >> 172.21.15.86:0/3215781985 conn(0x7f5d80070f30 msgr2=0x7f5d80073370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.500+0000 7f5d84c00640 1 -- 172.21.15.86:0/3215781985 shutdown_connections 2024-04-17T07:23:13.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.500+0000 7f5d84c00640 1 -- 172.21.15.86:0/3215781985 wait complete. 2024-04-17T07:23:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.502+0000 7f5d84c00640 1 Processor -- start 2024-04-17T07:23:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.502+0000 7f5d84c00640 1 -- start start 2024-04-17T07:23:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.502+0000 7f5d84c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d8006bee0 0x7f5d801a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.502+0000 7f5d84c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8006d4c0 0x7f5d801a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.502+0000 7f5d84c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d80116d70 0x7f5d801ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.502+0000 7f5d84c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8011c8c0 con 0x7f5d8006bee0 2024-04-17T07:23:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.502+0000 7f5d84c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8011c740 con 0x7f5d80116d70 2024-04-17T07:23:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.502+0000 7f5d84c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8011ca40 con 0x7f5d8006d4c0 2024-04-17T07:23:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.503+0000 7f5d7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d8006bee0 0x7f5d801a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.503+0000 7f5d7de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d8006bee0 0x7f5d801a7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35812/0 (socket says 172.21.15.86:35812) 2024-04-17T07:23:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.503+0000 7f5d7de00640 1 -- 172.21.15.86:0/1279194290 learned_addr learned my addr 172.21.15.86:0/1279194290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.503+0000 7f5d7e800640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d80116d70 0x7f5d801ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.504+0000 7f5d7d400640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8006d4c0 0x7f5d801a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.504+0000 7f5d7de00640 1 -- 172.21.15.86:0/1279194290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8006d4c0 msgr2=0x7f5d801a7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.504+0000 7f5d7de00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8006d4c0 0x7f5d801a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.504+0000 7f5d7de00640 1 -- 172.21.15.86:0/1279194290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d80116d70 msgr2=0x7f5d801ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.504+0000 7f5d7de00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d80116d70 0x7f5d801ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.504+0000 7f5d7de00640 1 -- 172.21.15.86:0/1279194290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d801aec70 con 0x7f5d8006bee0 2024-04-17T07:23:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.504+0000 7f5d7e800640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d80116d70 0x7f5d801ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:23:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.504+0000 7f5d7de00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d8006bee0 0x7f5d801a7050 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5d7002c1a0 tx=0x7f5d700040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.505+0000 7f5d76a00640 1 -- 172.21.15.86:0/1279194290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d70041020 con 0x7f5d8006bee0 2024-04-17T07:23:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.505+0000 7f5d84c00640 1 -- 172.21.15.86:0/1279194290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d801aef00 con 0x7f5d8006bee0 2024-04-17T07:23:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.505+0000 7f5d84c00640 1 -- 172.21.15.86:0/1279194290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d801af360 con 0x7f5d8006bee0 2024-04-17T07:23:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.506+0000 7f5d76a00640 1 -- 172.21.15.86:0/1279194290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d7002c9c0 con 0x7f5d8006bee0 2024-04-17T07:23:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.506+0000 7f5d76a00640 1 -- 172.21.15.86:0/1279194290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d70039040 con 0x7f5d8006bee0 2024-04-17T07:23:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.510+0000 7f5d87a24740 1 Processor -- start 2024-04-17T07:23:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.511+0000 7f5d87a24740 1 -- start start 2024-04-17T07:23:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.511+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecaf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.511+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecaf990 0x557dbecb4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.511+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecb4b00 0x557dbecb6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.511+0000 7f5d87a24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557dbec15020 con 0x557dbecb4b00 2024-04-17T07:23:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.511+0000 7f5d87a24740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557dbebbb460 con 0x557dbecaf990 2024-04-17T07:23:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.511+0000 7f5d87a24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557dbec10fe0 con 0x557dbecaefa0 2024-04-17T07:23:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.512+0000 7f5d7e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecaf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.512+0000 7f5d7e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecaf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41232/0 (socket says 172.21.15.86:41232) 2024-04-17T07:23:13.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.512+0000 7f5d7e800640 1 -- 172.21.15.86:0/2161624062 learned_addr learned my addr 172.21.15.86:0/2161624062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.514+0000 7f5d7d400640 1 --2- 172.21.15.86:0/2161624062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecb4b00 0x557dbecb6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.514+0000 7f5d76a00640 1 -- 172.21.15.86:0/1279194290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5d70044020 con 0x7f5d8006bee0 2024-04-17T07:23:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.514+0000 7f5d7e800640 1 -- 172.21.15.86:0/2161624062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecaf990 msgr2=0x557dbecb4490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:23:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.515+0000 7f5d7e800640 1 --2- 172.21.15.86:0/2161624062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecaf990 0x557dbecb4490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.515+0000 7f5d7e800640 1 -- 172.21.15.86:0/2161624062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecb4b00 msgr2=0x557dbecb6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.515+0000 7f5d7e800640 1 --2- 172.21.15.86:0/2161624062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecb4b00 0x557dbecb6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.515+0000 7f5d7e800640 1 -- 172.21.15.86:0/2161624062 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbebc3040 con 0x557dbecaefa0 2024-04-17T07:23:13.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.515+0000 7f5d7d400640 1 --2- 172.21.15.86:0/2161624062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecb4b00 0x557dbecb6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:13.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.516+0000 7f5d7e800640 1 --2- 172.21.15.86:0/2161624062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecaf3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009f90 tx=0x7f5d7800f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f59d50cb7fd54b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.517+0000 7f5d76a00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d640a3c30 0x7f5d640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.517+0000 7f5d76a00640 1 -- 172.21.15.86:0/1279194290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5d700f2170 con 0x7f5d8006bee0 2024-04-17T07:23:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.518+0000 7f5d6b000640 1 -- 172.21.15.86:0/2161624062 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78003b70 con 0x557dbecaefa0 2024-04-17T07:23:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.518+0000 7f5d6b000640 1 -- 172.21.15.86:0/2161624062 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d78003d10 con 0x557dbecaefa0 2024-04-17T07:23:13.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.518+0000 7f5d6b000640 1 -- 172.21.15.86:0/2161624062 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78003b70 con 0x557dbecaefa0 2024-04-17T07:23:13.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.518+0000 7f5d7de00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d640a3c30 0x7f5d640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.519+0000 7f5d87a24740 1 -- 172.21.15.86:0/2161624062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 msgr2=0x557dbecaf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.519+0000 7f5d87a24740 1 --2- 172.21.15.86:0/2161624062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecaf3c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009f90 tx=0x7f5d7800f7f0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.519+0000 7f5d87a24740 1 -- 172.21.15.86:0/2161624062 shutdown_connections 2024-04-17T07:23:13.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.519+0000 7f5d87a24740 1 --2- 172.21.15.86:0/2161624062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecb4b00 0x557dbecb6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.519+0000 7f5d87a24740 1 --2- 172.21.15.86:0/2161624062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecaf990 0x557dbecb4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.519+0000 7f5d87a24740 1 --2- 172.21.15.86:0/2161624062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecaf3c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.519+0000 7f5d87a24740 1 -- 172.21.15.86:0/2161624062 >> 172.21.15.86:0/2161624062 conn(0x557dbecaa510 msgr2=0x557dbecaa910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.520+0000 7f5d7de00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d640a3c30 0x7f5d640a60f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009860 tx=0x7f5d70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.521+0000 7f5d87a24740 1 -- 172.21.15.86:0/2161624062 shutdown_connections 2024-04-17T07:23:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.521+0000 7f5d87a24740 1 -- 172.21.15.86:0/2161624062 wait complete. 2024-04-17T07:23:13.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.522+0000 7f5d87a24740 1 -- start start 2024-04-17T07:23:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.523+0000 7f5d87a24740 1 Processor -- start 2024-04-17T07:23:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.524+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.524+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaf990 0x557dbecb9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.524+0000 7f5d7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.524+0000 7f5d7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35822/0 (socket says 172.21.15.86:35822) 2024-04-17T07:23:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.524+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecb4b00 0x557dbecc08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.525+0000 7f5d7e800640 1 -- 172.21.15.86:0/227721184 learned_addr learned my addr 172.21.15.86:0/227721184 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.525+0000 7f5d7d400640 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaf990 0x557dbecb9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.525+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557dbecb7ab0 con 0x557dbecaefa0 2024-04-17T07:23:13.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.525+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557dbebce830 con 0x557dbecb4b00 2024-04-17T07:23:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.525+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557dbecb7c30 con 0x557dbecaf990 2024-04-17T07:23:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.525+0000 7f5d87a24740 10 client.0 _ll_register_callbacks cb 0x557dbeb9cfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.526+0000 7f5d7d400640 1 -- 172.21.15.86:0/227721184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecb4b00 msgr2=0x557dbecc08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.526+0000 7f5d7d400640 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecb4b00 0x557dbecc08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.526+0000 7f5d7d400640 1 -- 172.21.15.86:0/227721184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 msgr2=0x557dbecb9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.526+0000 7f5d7d400640 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.527+0000 7f5d7d400640 1 -- 172.21.15.86:0/227721184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbebd2920 con 0x557dbecaf990 2024-04-17T07:23:13.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.527+0000 7f5d7e800640 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:13.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.529+0000 7f5d7d400640 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaf990 0x557dbecb9880 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5d801a8560 tx=0x7f5d6c012950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5591bf78d11da259 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.529+0000 7f5d5f400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:13.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.531+0000 7f5d5f400640 1 -- 172.21.15.86:0/227721184 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c01f060 con 0x557dbecaf990 2024-04-17T07:23:13.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.531+0000 7f5d5f400640 1 -- 172.21.15.86:0/227721184 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5d6c004460 con 0x557dbecaf990 2024-04-17T07:23:13.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.531+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d78003d10 con 0x557dbecaf990 2024-04-17T07:23:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.532+0000 7f5d5f400640 1 -- 172.21.15.86:0/227721184 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d6c0035f0 con 0x557dbecaf990 2024-04-17T07:23:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.532+0000 7f5d87a24740 10 client.26677 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.532+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557dbeba07f0 con 0x557dbecaf990 2024-04-17T07:23:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.533+0000 7f5d5f400640 1 -- 172.21.15.86:0/227721184 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d78003d10 con 0x557dbecaf990 2024-04-17T07:23:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.533+0000 7f5d5f400640 1 -- 172.21.15.86:0/227721184 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5d6c019030 con 0x557dbecaf990 2024-04-17T07:23:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.533+0000 7f5d87a24740 10 client.26677 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.533+0000 7f5d87a24740 10 client.26677 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:23:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.533+0000 7f5d87a24740 20 client.26677 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.533+0000 7f5d87a24740 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x557dbecc52e0 0x557dbecc7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.533+0000 7f5d87a24740 4 client.26677 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.533+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dbec22a40 con 0x557dbecc52e0 2024-04-17T07:23:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.534+0000 7f5d7e800640 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x557dbecc52e0 0x557dbecc7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.536+0000 7f5d7e800640 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x557dbecc52e0 0x557dbecc7720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d066ae1ec8cb9838 server_cookie=b609cc1322457e68 in_seq=0 out_seq=0 2024-04-17T07:23:13.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.537+0000 7f5d5f400640 10 client.26677 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:23:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.539+0000 7f5d5f400640 1 -- 172.21.15.86:0/227721184 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557dbec22a40 con 0x557dbecc52e0 2024-04-17T07:23:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.539+0000 7f5d5f400640 10 client.26677 handle_command_reply: tid=0 2024-04-17T07:23:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.539+0000 7f5d87a24740 1 client.26677 shutdown 2024-04-17T07:23:13.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.540+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaf990 msgr2=0x557dbecb9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.540+0000 7f5d87a24740 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaf990 0x557dbecb9880 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5d801a8560 tx=0x7f5d6c012950 comp rx=0 tx=0).stop 2024-04-17T07:23:13.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.541+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 shutdown_connections 2024-04-17T07:23:13.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.541+0000 7f5d87a24740 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x557dbecc52e0 0x557dbecc7720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.541+0000 7f5d87a24740 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecb4b00 0x557dbecc08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.541+0000 7f5d87a24740 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaf990 0x557dbecb9880 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.541+0000 7f5d87a24740 1 --2- 172.21.15.86:0/227721184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.541+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 >> 172.21.15.86:0/227721184 conn(0x557dbecaa510 msgr2=0x557dbeca9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.541+0000 7f5d5f400640 0 client.26677 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:23:13.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.542+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 shutdown_connections 2024-04-17T07:23:13.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.543+0000 7f5d87a24740 1 -- 172.21.15.86:0/227721184 wait complete. 2024-04-17T07:23:13.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.543+0000 7f5d87a24740 20 client.26677 trim_cache size 0 max 16384 2024-04-17T07:23:13.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.565+0000 7f5d87a24740 1 Processor -- start 2024-04-17T07:23:13.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.566+0000 7f5d87a24740 1 -- start start 2024-04-17T07:23:13.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.566+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.566+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecaf990 0x557dbecc0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.566+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecb4b00 0x557dbeca8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.566+0000 7f5d87a24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6000fc20 con 0x557dbecaefa0 2024-04-17T07:23:13.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.566+0000 7f5d87a24740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5d60011760 con 0x557dbecaf990 2024-04-17T07:23:13.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.566+0000 7f5d87a24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d600102f0 con 0x557dbecb4b00 2024-04-17T07:23:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.567+0000 7f5d7d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecaf990 0x557dbecc0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.567+0000 7f5d7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.567+0000 7f5d7e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35826/0 (socket says 172.21.15.86:35826) 2024-04-17T07:23:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.567+0000 7f5d7e800640 1 -- 172.21.15.86:0/908215257 learned_addr learned my addr 172.21.15.86:0/908215257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.567+0000 7f5d7e800640 1 --2- 172.21.15.86:0/908215257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecb4b00 0x557dbeca8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.568+0000 7f5d7d400640 1 -- 172.21.15.86:0/908215257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecb4b00 msgr2=0x557dbeca8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.568+0000 7f5d7d400640 1 --2- 172.21.15.86:0/908215257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecb4b00 0x557dbeca8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.568+0000 7f5d7d400640 1 -- 172.21.15.86:0/908215257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 msgr2=0x557dbecb6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.568+0000 7f5d7d400640 1 --2- 172.21.15.86:0/908215257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.568+0000 7f5d7d400640 1 -- 172.21.15.86:0/908215257 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbe9273f0 con 0x557dbecaf990 2024-04-17T07:23:13.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.568+0000 7f5d7e800640 1 --2- 172.21.15.86:0/908215257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:13.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.568+0000 7f5d7d400640 1 --2- 172.21.15.86:0/908215257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecaf990 0x557dbecc0880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c006010 tx=0x7f5d6c010fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba8acd0c730f77a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.569+0000 7f5d6a600640 1 -- 172.21.15.86:0/908215257 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c032070 con 0x557dbecaf990 2024-04-17T07:23:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.570+0000 7f5d6a600640 1 -- 172.21.15.86:0/908215257 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d6c005790 con 0x557dbecaf990 2024-04-17T07:23:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.570+0000 7f5d6a600640 1 -- 172.21.15.86:0/908215257 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c029080 con 0x557dbecaf990 2024-04-17T07:23:13.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.571+0000 7f5d87a24740 1 -- 172.21.15.86:0/908215257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecaf990 msgr2=0x557dbecc0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.571+0000 7f5d87a24740 1 --2- 172.21.15.86:0/908215257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecaf990 0x557dbecc0880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c006010 tx=0x7f5d6c010fc0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.571+0000 7f5d87a24740 1 -- 172.21.15.86:0/908215257 shutdown_connections 2024-04-17T07:23:13.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.571+0000 7f5d87a24740 1 --2- 172.21.15.86:0/908215257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecb4b00 0x557dbeca8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.571+0000 7f5d87a24740 1 --2- 172.21.15.86:0/908215257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecaf990 0x557dbecc0880 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.571+0000 7f5d87a24740 1 --2- 172.21.15.86:0/908215257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaefa0 0x557dbecb6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.571+0000 7f5d87a24740 1 -- 172.21.15.86:0/908215257 >> 172.21.15.86:0/908215257 conn(0x557dbecaa510 msgr2=0x557dbecb0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.572+0000 7f5d87a24740 1 -- 172.21.15.86:0/908215257 shutdown_connections 2024-04-17T07:23:13.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.572+0000 7f5d87a24740 1 -- 172.21.15.86:0/908215257 wait complete. 2024-04-17T07:23:13.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.572+0000 7f5d87a24740 1 -- start start 2024-04-17T07:23:13.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.573+0000 7f5d87a24740 1 Processor -- start 2024-04-17T07:23:13.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.573+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecbdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.573+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaf990 0x557dbecbe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.573+0000 7f5d87a24740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecb4b00 0x557dbecbf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.573+0000 7f5d87a24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d60011090 con 0x557dbecaf990 2024-04-17T07:23:13.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.573+0000 7f5d87a24740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5d600109c0 con 0x557dbecb4b00 2024-04-17T07:23:13.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.573+0000 7f5d87a24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6000dbf0 con 0x557dbecaefa0 2024-04-17T07:23:13.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.573+0000 7f5d87a24740 10 client.0 _ll_register_callbacks cb 0x7f5d6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:13.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.574+0000 7f5d7d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaf990 0x557dbecbe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.574+0000 7f5d7d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaf990 0x557dbecbe1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35832/0 (socket says 172.21.15.86:35832) 2024-04-17T07:23:13.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.574+0000 7f5d7d400640 1 -- 172.21.15.86:0/257270495 learned_addr learned my addr 172.21.15.86:0/257270495 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.574+0000 7f5d7e800640 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecbdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.574+0000 7f5d7e800640 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecb4b00 0x557dbecbf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.575+0000 7f5d7d400640 1 -- 172.21.15.86:0/257270495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 msgr2=0x557dbecbdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.575+0000 7f5d7d400640 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecbdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.575+0000 7f5d7d400640 1 -- 172.21.15.86:0/257270495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecb4b00 msgr2=0x557dbecbf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.575+0000 7f5d7d400640 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecb4b00 0x557dbecbf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.575+0000 7f5d7d400640 1 -- 172.21.15.86:0/257270495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbe9f17d0 con 0x557dbecaf990 2024-04-17T07:23:13.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.575+0000 7f5d7e800640 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecbdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:23:13.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.575+0000 7f5d7d400640 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaf990 0x557dbecbe1f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c030040 tx=0x7f5d6c003790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70529786bb6b5bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.575+0000 7f5d5f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:23:13.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.576+0000 7f5d5f400640 1 -- 172.21.15.86:0/257270495 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c029070 con 0x557dbecaf990 2024-04-17T07:23:13.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.576+0000 7f5d5f400640 1 -- 172.21.15.86:0/257270495 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5d6c003a30 con 0x557dbecaf990 2024-04-17T07:23:13.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.576+0000 7f5d87a24740 1 -- 172.21.15.86:0/257270495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d6c005790 con 0x557dbecaf990 2024-04-17T07:23:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.579+0000 7f5d5f400640 1 -- 172.21.15.86:0/257270495 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d6c01f070 con 0x557dbecaf990 2024-04-17T07:23:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.579+0000 7f5d87a24740 10 client.17715 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.579+0000 7f5d87a24740 1 -- 172.21.15.86:0/257270495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557dbecc0920 con 0x557dbecaf990 2024-04-17T07:23:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.579+0000 7f5d5f400640 1 -- 172.21.15.86:0/257270495 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d6c005790 con 0x557dbecaf990 2024-04-17T07:23:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.580+0000 7f5d5f400640 1 -- 172.21.15.86:0/257270495 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5d6c019040 con 0x557dbecaf990 2024-04-17T07:23:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.580+0000 7f5d87a24740 10 client.17715 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.580+0000 7f5d87a24740 10 client.17715 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:23:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.580+0000 7f5d87a24740 20 client.17715 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.580+0000 7f5d87a24740 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x557dbed2bc10 0x557dbed2e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.580+0000 7f5d87a24740 4 client.17715 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:23:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.580+0000 7f5d87a24740 1 -- 172.21.15.86:0/257270495 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557dbec1c8b0 con 0x557dbed2bc10 2024-04-17T07:23:13.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.581+0000 7f5d7e800640 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x557dbed2bc10 0x557dbed2e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.581+0000 7f5d7e800640 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x557dbed2bc10 0x557dbed2e030 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56951c4bd4967161 server_cookie=ff40fdce0aade128 in_seq=0 out_seq=0 2024-04-17T07:23:13.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.581+0000 7f5d5f400640 10 client.17715 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:23:13.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.582+0000 7f5d5f400640 1 -- 172.21.15.86:0/257270495 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557dbec1c8b0 con 0x557dbed2bc10 2024-04-17T07:23:13.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.582+0000 7f5d5f400640 10 client.17715 handle_command_reply: tid=0 2024-04-17T07:23:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.582+0000 7f5d87a24740 1 client.17715 shutdown 2024-04-17T07:23:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.582+0000 7f5d87a24740 1 -- 172.21.15.86:0/257270495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaf990 msgr2=0x557dbecbe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.582+0000 7f5d87a24740 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaf990 0x557dbecbe1f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c030040 tx=0x7f5d6c003790 comp rx=0 tx=0).stop 2024-04-17T07:23:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.583+0000 7f5d87a24740 1 -- 172.21.15.86:0/257270495 shutdown_connections 2024-04-17T07:23:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.583+0000 7f5d87a24740 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x557dbed2bc10 0x557dbed2e030 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.583+0000 7f5d5f400640 0 client.17715 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:23:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.583+0000 7f5d87a24740 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557dbecb4b00 0x557dbecbf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.583+0000 7f5d87a24740 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dbecaf990 0x557dbecbe1f0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.583+0000 7f5d87a24740 1 --2- 172.21.15.86:0/257270495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dbecaefa0 0x557dbecbdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.583+0000 7f5d87a24740 1 -- 172.21.15.86:0/257270495 >> 172.21.15.86:0/257270495 conn(0x557dbecaa510 msgr2=0x557dbecb9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.584+0000 7f5d87a24740 1 -- 172.21.15.86:0/257270495 shutdown_connections 2024-04-17T07:23:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.584+0000 7f5d87a24740 1 -- 172.21.15.86:0/257270495 wait complete. 2024-04-17T07:23:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.584+0000 7f5d87a24740 20 client.17715 trim_cache size 0 max 16384 2024-04-17T07:23:13.585 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:23:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.586+0000 7f5d84c00640 1 -- 172.21.15.86:0/1279194290 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d640a3c30 msgr2=0x7f5d640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.586+0000 7f5d84c00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d640a3c30 0x7f5d640a60f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009860 tx=0x7f5d70005f90 comp rx=0 tx=0).stop 2024-04-17T07:23:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.586+0000 7f5d84c00640 1 -- 172.21.15.86:0/1279194290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d8006bee0 msgr2=0x7f5d801a7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.586+0000 7f5d84c00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d8006bee0 0x7f5d801a7050 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5d7002c1a0 tx=0x7f5d700040c0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.586+0000 7f5d84c00640 1 -- 172.21.15.86:0/1279194290 shutdown_connections 2024-04-17T07:23:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.586+0000 7f5d84c00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d640a3c30 0x7f5d640a60f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.586+0000 7f5d84c00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d80116d70 0x7f5d801ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.586+0000 7f5d84c00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8006d4c0 0x7f5d801a7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.586+0000 7f5d84c00640 1 --2- 172.21.15.86:0/1279194290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d8006bee0 0x7f5d801a7050 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.586+0000 7f5d84c00640 1 -- 172.21.15.86:0/1279194290 >> 172.21.15.86:0/1279194290 conn(0x7f5d80070f30 msgr2=0x7f5d8010a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.587+0000 7f5d84c00640 1 -- 172.21.15.86:0/1279194290 shutdown_connections 2024-04-17T07:23:13.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.587+0000 7f5d84c00640 1 -- 172.21.15.86:0/1279194290 wait complete. 2024-04-17T07:23:13.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:23:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.937+0000 7f0d97e00640 1 -- 172.21.15.86:0/4111590164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d901005f0 msgr2=0x7f0d90102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.937+0000 7f0d97e00640 1 --2- 172.21.15.86:0/4111590164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d901005f0 0x7f0d90102280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009900 tx=0x7f0d88034440 comp rx=0 tx=0).stop 2024-04-17T07:23:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.938+0000 7f0d97e00640 1 -- 172.21.15.86:0/4111590164 shutdown_connections 2024-04-17T07:23:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.938+0000 7f0d97e00640 1 --2- 172.21.15.86:0/4111590164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d901005f0 0x7f0d90102280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.938+0000 7f0d97e00640 1 --2- 172.21.15.86:0/4111590164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d900ffc30 0x7f0d901000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.938+0000 7f0d97e00640 1 --2- 172.21.15.86:0/4111590164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d901011e0 0x7f0d90101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.938+0000 7f0d97e00640 1 -- 172.21.15.86:0/4111590164 >> 172.21.15.86:0/4111590164 conn(0x7f0d900fb760 msgr2=0x7f0d900fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.938+0000 7f0d97e00640 1 -- 172.21.15.86:0/4111590164 shutdown_connections 2024-04-17T07:23:13.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.939+0000 7f0d97e00640 1 -- 172.21.15.86:0/4111590164 wait complete. 2024-04-17T07:23:13.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.941+0000 7f0d97e00640 1 Processor -- start 2024-04-17T07:23:13.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.941+0000 7f0d97e00640 1 -- start start 2024-04-17T07:23:13.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.941+0000 7f0d97e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d900ffc30 0x7f0d901a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.941+0000 7f0d97e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d901005f0 0x7f0d901a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.941+0000 7f0d97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d901011e0 0x7f0d901ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.942+0000 7f0d97e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901058e0 con 0x7f0d901011e0 2024-04-17T07:23:13.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.942+0000 7f0d97e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90105760 con 0x7f0d900ffc30 2024-04-17T07:23:13.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.942+0000 7f0d97e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90105a60 con 0x7f0d901005f0 2024-04-17T07:23:13.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.942+0000 7f0d95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d901011e0 0x7f0d901ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.942+0000 7f0d95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d901011e0 0x7f0d901ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35850/0 (socket says 172.21.15.86:35850) 2024-04-17T07:23:13.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.942+0000 7f0d95e00640 1 -- 172.21.15.86:0/52989163 learned_addr learned my addr 172.21.15.86:0/52989163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.942+0000 7f0d94a00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d901005f0 0x7f0d901a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.942+0000 7f0d95400640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d900ffc30 0x7f0d901a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.943+0000 7f0d95e00640 1 -- 172.21.15.86:0/52989163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d901005f0 msgr2=0x7f0d901a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.943+0000 7f0d95e00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d901005f0 0x7f0d901a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.943+0000 7f0d95e00640 1 -- 172.21.15.86:0/52989163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d900ffc30 msgr2=0x7f0d901a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.943+0000 7f0d95e00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d900ffc30 0x7f0d901a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.943+0000 7f0d95e00640 1 -- 172.21.15.86:0/52989163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9006bab0 con 0x7f0d901011e0 2024-04-17T07:23:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.943+0000 7f0d95e00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d901011e0 0x7f0d901ae540 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0d88005ab0 tx=0x7f0d88003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.944+0000 7f0d8e000640 1 -- 172.21.15.86:0/52989163 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8803f070 con 0x7f0d901011e0 2024-04-17T07:23:13.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.944+0000 7f0d8e000640 1 -- 172.21.15.86:0/52989163 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d880147b0 con 0x7f0d901011e0 2024-04-17T07:23:13.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.944+0000 7f0d8e000640 1 -- 172.21.15.86:0/52989163 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8803be20 con 0x7f0d901011e0 2024-04-17T07:23:13.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.944+0000 7f0d97e00640 1 -- 172.21.15.86:0/52989163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9006bd40 con 0x7f0d901011e0 2024-04-17T07:23:13.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.944+0000 7f0d97e00640 1 -- 172.21.15.86:0/52989163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9006c1e0 con 0x7f0d901011e0 2024-04-17T07:23:13.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.950+0000 7f0d8e000640 1 -- 172.21.15.86:0/52989163 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0d88010430 con 0x7f0d901011e0 2024-04-17T07:23:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.951+0000 7f0d9ab8d740 1 Processor -- start 2024-04-17T07:23:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.952+0000 7f0d9ab8d740 1 -- start start 2024-04-17T07:23:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.953+0000 7f0d8e000640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d740a3d00 0x7f0d740a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.953+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 0x55b73f6643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.953+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f664990 0x55b73f669490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.953+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f66bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.953+0000 7f0d9ab8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b73f5ca020 con 0x55b73f663fa0 2024-04-17T07:23:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.953+0000 7f0d9ab8d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b73f570460 con 0x55b73f664990 2024-04-17T07:23:13.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.953+0000 7f0d9ab8d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b73f5c5fe0 con 0x55b73f669b00 2024-04-17T07:23:13.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.953+0000 7f0d8e000640 1 -- 172.21.15.86:0/52989163 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0d880f2880 con 0x7f0d901011e0 2024-04-17T07:23:13.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.955+0000 7f0d95e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f66bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.955+0000 7f0d95e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f66bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41278/0 (socket says 172.21.15.86:41278) 2024-04-17T07:23:13.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.955+0000 7f0d95e00640 1 -- 172.21.15.86:0/1529601101 learned_addr learned my addr 172.21.15.86:0/1529601101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.955+0000 7f0d95400640 1 --2- 172.21.15.86:0/1529601101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 0x55b73f6643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.955+0000 7f0d94a00640 1 --2- 172.21.15.86:0/1529601101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f664990 0x55b73f669490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.955+0000 7f0d94a00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d740a3d00 0x7f0d740a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.956+0000 7f0d95e00640 1 -- 172.21.15.86:0/1529601101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f664990 msgr2=0x55b73f669490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.956+0000 7f0d95e00640 1 --2- 172.21.15.86:0/1529601101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f664990 0x55b73f669490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.956+0000 7f0d95e00640 1 -- 172.21.15.86:0/1529601101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 msgr2=0x55b73f6643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.956+0000 7f0d95e00640 1 --2- 172.21.15.86:0/1529601101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 0x55b73f6643c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.956+0000 7f0d95e00640 1 -- 172.21.15.86:0/1529601101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b73f578040 con 0x55b73f669b00 2024-04-17T07:23:13.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.956+0000 7f0d95e00640 1 --2- 172.21.15.86:0/1529601101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f66bf90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0d880bacf0 tx=0x7f0d880be3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51cca399dce07305 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.957+0000 7f0d86600640 1 -- 172.21.15.86:0/1529601101 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880be890 con 0x55b73f669b00 2024-04-17T07:23:13.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.957+0000 7f0d86600640 1 -- 172.21.15.86:0/1529601101 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d880c2b90 con 0x55b73f669b00 2024-04-17T07:23:13.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.957+0000 7f0d86600640 1 -- 172.21.15.86:0/1529601101 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880bdce0 con 0x55b73f669b00 2024-04-17T07:23:13.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.958+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1529601101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 msgr2=0x55b73f66bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.958+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1529601101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f66bf90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0d880bacf0 tx=0x7f0d880be3d0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.958+0000 7f0d94a00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d740a3d00 0x7f0d740a61c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x55b73f664d90 tx=0x7f0d80009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.958+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1529601101 shutdown_connections 2024-04-17T07:23:13.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.958+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1529601101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f66bf90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.958+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1529601101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f664990 0x55b73f669490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.958+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1529601101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 0x55b73f6643c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.958+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1529601101 >> 172.21.15.86:0/1529601101 conn(0x55b73f65f510 msgr2=0x55b73f65f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.959+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1529601101 shutdown_connections 2024-04-17T07:23:13.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.959+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1529601101 wait complete. 2024-04-17T07:23:13.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.960+0000 7f0d9ab8d740 1 -- start start 2024-04-17T07:23:13.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.960+0000 7f0d9ab8d740 1 Processor -- start 2024-04-17T07:23:13.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.961+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f663fa0 0x55b73f66e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.961+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f664990 0x55b73f66e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.962+0000 7f0d95400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f663fa0 0x55b73f66e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.962+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f669b00 0x55b73f6757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.962+0000 7f0d9ab8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b73f66c870 con 0x55b73f664990 2024-04-17T07:23:13.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.962+0000 7f0d9ab8d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b73f583830 con 0x55b73f669b00 2024-04-17T07:23:13.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.962+0000 7f0d9ab8d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b73f66c9f0 con 0x55b73f663fa0 2024-04-17T07:23:13.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.962+0000 7f0d9ab8d740 10 client.0 _ll_register_callbacks cb 0x55b73f551fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:13.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.962+0000 7f0d94a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f664990 0x55b73f66e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.962+0000 7f0d94a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f664990 0x55b73f66e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35878/0 (socket says 172.21.15.86:35878) 2024-04-17T07:23:13.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.962+0000 7f0d94a00640 1 -- 172.21.15.86:0/370458219 learned_addr learned my addr 172.21.15.86:0/370458219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:13.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.964+0000 7f0d94a00640 1 -- 172.21.15.86:0/370458219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f663fa0 msgr2=0x55b73f66e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:13.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.964+0000 7f0d94a00640 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f663fa0 0x55b73f66e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.964+0000 7f0d94a00640 1 -- 172.21.15.86:0/370458219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f669b00 msgr2=0x55b73f6757c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:23:13.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.964+0000 7f0d95e00640 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f669b00 0x55b73f6757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.964+0000 7f0d94a00640 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f669b00 0x55b73f6757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.964+0000 7f0d94a00640 1 -- 172.21.15.86:0/370458219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b73f587920 con 0x55b73f664990 2024-04-17T07:23:13.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.965+0000 7f0d7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:23:13.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.965+0000 7f0d94a00640 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f664990 0x55b73f66e780 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0d80004340 tx=0x7f0d80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfbb914da7b9c41 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:13.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.967+0000 7f0d7aa00640 1 -- 172.21.15.86:0/370458219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80019070 con 0x55b73f664990 2024-04-17T07:23:13.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.967+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/370458219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d880c2b90 con 0x55b73f664990 2024-04-17T07:23:13.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.967+0000 7f0d7aa00640 1 -- 172.21.15.86:0/370458219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0d80010b70 con 0x55b73f664990 2024-04-17T07:23:13.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.968+0000 7f0d7aa00640 1 -- 172.21.15.86:0/370458219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d80010d70 con 0x55b73f664990 2024-04-17T07:23:13.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.968+0000 7f0d9ab8d740 10 client.17727 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:13.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.968+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/370458219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b73f5557f0 con 0x55b73f664990 2024-04-17T07:23:13.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.969+0000 7f0d7aa00640 1 -- 172.21.15.86:0/370458219 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d880c2b90 con 0x55b73f664990 2024-04-17T07:23:13.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.969+0000 7f0d7aa00640 1 -- 172.21.15.86:0/370458219 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0d80028510 con 0x55b73f664990 2024-04-17T07:23:13.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.969+0000 7f0d9ab8d740 10 client.17727 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:13.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.969+0000 7f0d9ab8d740 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:13.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.969+0000 7f0d9ab8d740 20 client.17727 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:13.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.970+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b73f679d90 0x55b73f67c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:13.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.970+0000 7f0d9ab8d740 4 client.17727 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:13.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.970+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/370458219 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b73f5d7a40 con 0x55b73f679d90 2024-04-17T07:23:13.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.971+0000 7f0d95400640 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b73f679d90 0x55b73f67c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.973+0000 7f0d95400640 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b73f679d90 0x55b73f67c210 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2114b63b293d8349 server_cookie=33e55ecc5fd0c525 in_seq=0 out_seq=0 2024-04-17T07:23:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.973+0000 7f0d7aa00640 10 client.17727 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:13.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d7aa00640 1 -- 172.21.15.86:0/370458219 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b73f5d7a40 con 0x55b73f679d90 2024-04-17T07:23:13.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d7aa00640 10 client.17727 handle_command_reply: tid=0 2024-04-17T07:23:13.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d9ab8d740 1 client.17727 shutdown 2024-04-17T07:23:13.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/370458219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f664990 msgr2=0x55b73f66e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:13.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f664990 0x55b73f66e780 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0d80004340 tx=0x7f0d80004ab0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/370458219 shutdown_connections 2024-04-17T07:23:13.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b73f679d90 0x55b73f67c210 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f669b00 0x55b73f6757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f664990 0x55b73f66e780 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/370458219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f663fa0 0x55b73f66e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:13.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.976+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/370458219 >> 172.21.15.86:0/370458219 conn(0x55b73f65f510 msgr2=0x55b73f65e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:13.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.977+0000 7f0d7aa00640 0 client.17727 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:13.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.977+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/370458219 shutdown_connections 2024-04-17T07:23:13.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.977+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/370458219 wait complete. 2024-04-17T07:23:13.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:13.978+0000 7f0d9ab8d740 20 client.17727 trim_cache size 0 max 16384 2024-04-17T07:23:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.007+0000 7f0d9ab8d740 1 Processor -- start 2024-04-17T07:23:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.007+0000 7f0d9ab8d740 1 -- start start 2024-04-17T07:23:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.008+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f663fa0 0x55b73f67bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.008+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f664990 0x55b73f675780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.008+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f669b00 0x55b73f65de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.008+0000 7f0d9ab8d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901c2090 con 0x55b73f669b00 2024-04-17T07:23:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.008+0000 7f0d9ab8d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901c3bd0 con 0x55b73f663fa0 2024-04-17T07:23:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.008+0000 7f0d9ab8d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901c2760 con 0x55b73f664990 2024-04-17T07:23:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.009+0000 7f0d95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f669b00 0x55b73f65de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.009+0000 7f0d95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f669b00 0x55b73f65de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35886/0 (socket says 172.21.15.86:35886) 2024-04-17T07:23:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.009+0000 7f0d95e00640 1 -- 172.21.15.86:0/1379259653 learned_addr learned my addr 172.21.15.86:0/1379259653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.009+0000 7f0d95400640 1 --2- 172.21.15.86:0/1379259653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f663fa0 0x55b73f67bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.009+0000 7f0d94a00640 1 --2- 172.21.15.86:0/1379259653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f664990 0x55b73f675780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.010+0000 7f0d95e00640 1 -- 172.21.15.86:0/1379259653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f664990 msgr2=0x55b73f675780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.010+0000 7f0d95e00640 1 --2- 172.21.15.86:0/1379259653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f664990 0x55b73f675780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.010+0000 7f0d95e00640 1 -- 172.21.15.86:0/1379259653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f663fa0 msgr2=0x55b73f67bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.010+0000 7f0d95e00640 1 --2- 172.21.15.86:0/1379259653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f663fa0 0x55b73f67bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.010+0000 7f0d95e00640 1 -- 172.21.15.86:0/1379259653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b73f2dc3f0 con 0x55b73f669b00 2024-04-17T07:23:14.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.011+0000 7f0d95e00640 1 --2- 172.21.15.86:0/1379259653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f669b00 0x55b73f65de80 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0d880c6f20 tx=0x7f0d880149f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94dead2ffb00041f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:14.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.012+0000 7f0d85c00640 1 -- 172.21.15.86:0/1379259653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880af070 con 0x55b73f669b00 2024-04-17T07:23:14.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.012+0000 7f0d85c00640 1 -- 172.21.15.86:0/1379259653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d880c2b90 con 0x55b73f669b00 2024-04-17T07:23:14.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.012+0000 7f0d85c00640 1 -- 172.21.15.86:0/1379259653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880c6fc0 con 0x55b73f669b00 2024-04-17T07:23:14.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.013+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1379259653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f669b00 msgr2=0x55b73f65de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:14.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.013+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1379259653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f669b00 0x55b73f65de80 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0d880c6f20 tx=0x7f0d880149f0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.014+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1379259653 shutdown_connections 2024-04-17T07:23:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.014+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1379259653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f669b00 0x55b73f65de80 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.014+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1379259653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f664990 0x55b73f675780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.014+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1379259653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f663fa0 0x55b73f67bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.014+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1379259653 >> 172.21.15.86:0/1379259653 conn(0x55b73f65f510 msgr2=0x55b73f665ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.014+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1379259653 shutdown_connections 2024-04-17T07:23:14.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.014+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1379259653 wait complete. 2024-04-17T07:23:14.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.015+0000 7f0d9ab8d740 1 -- start start 2024-04-17T07:23:14.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.015+0000 7f0d9ab8d740 1 Processor -- start 2024-04-17T07:23:14.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.016+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 0x55b73f672e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:14.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.016+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f664990 0x55b73f673350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:14.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.016+0000 7f0d9ab8d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f674360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:14.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.016+0000 7f0d95400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 0x55b73f672e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:14.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.017+0000 7f0d95400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 0x55b73f672e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35888/0 (socket says 172.21.15.86:35888) 2024-04-17T07:23:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.017+0000 7f0d95400640 1 -- 172.21.15.86:0/1444838894 learned_addr learned my addr 172.21.15.86:0/1444838894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.017+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901c3500 con 0x55b73f663fa0 2024-04-17T07:23:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.017+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901c2e30 con 0x55b73f664990 2024-04-17T07:23:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.017+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90073e40 con 0x55b73f669b00 2024-04-17T07:23:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.017+0000 7f0d95e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f674360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.017+0000 7f0d9ab8d740 10 client.0 _ll_register_callbacks cb 0x7f0d80003dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.017+0000 7f0d94a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f664990 0x55b73f673350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:14.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.018+0000 7f0d95e00640 1 -- 172.21.15.86:0/1444838894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f664990 msgr2=0x55b73f673350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:14.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.018+0000 7f0d95e00640 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f664990 0x55b73f673350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.018+0000 7f0d95e00640 1 -- 172.21.15.86:0/1444838894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 msgr2=0x55b73f672e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:14.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.018+0000 7f0d95e00640 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 0x55b73f672e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.018+0000 7f0d95e00640 1 -- 172.21.15.86:0/1444838894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b73f3a67d0 con 0x55b73f669b00 2024-04-17T07:23:14.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.018+0000 7f0d95400640 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 0x55b73f672e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:14.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.019+0000 7f0d95e00640 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f674360 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55b73f675d80 tx=0x7f0d880be2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff42d9fbfee2ced0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:14.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.019+0000 7f0d7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.019+0000 7f0d7aa00640 1 -- 172.21.15.86:0/1444838894 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880af070 con 0x55b73f669b00 2024-04-17T07:23:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.020+0000 7f0d7aa00640 1 -- 172.21.15.86:0/1444838894 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0d88014300 con 0x55b73f669b00 2024-04-17T07:23:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.020+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d880c2b90 con 0x55b73f669b00 2024-04-17T07:23:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.020+0000 7f0d7aa00640 1 -- 172.21.15.86:0/1444838894 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d880dc050 con 0x55b73f669b00 2024-04-17T07:23:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.020+0000 7f0d9ab8d740 10 client.26698 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.020+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b73f675a80 con 0x55b73f669b00 2024-04-17T07:23:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.021+0000 7f0d7aa00640 1 -- 172.21.15.86:0/1444838894 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d880c2b90 con 0x55b73f669b00 2024-04-17T07:23:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.021+0000 7f0d7aa00640 1 -- 172.21.15.86:0/1444838894 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0d8804c030 con 0x55b73f669b00 2024-04-17T07:23:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.021+0000 7f0d9ab8d740 10 client.26698 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.021+0000 7f0d9ab8d740 10 client.26698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.021+0000 7f0d9ab8d740 20 client.26698 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.021+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b73f6e05f0 0x55b73f6e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.021+0000 7f0d9ab8d740 4 client.26698 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:23:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.021+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b73f5d18b0 con 0x55b73f6e05f0 2024-04-17T07:23:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.022+0000 7f0d95400640 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b73f6e05f0 0x55b73f6e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:14.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.023+0000 7f0d95400640 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b73f6e05f0 0x55b73f6e29e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1607972242d4f6 server_cookie=d53c6d470f6656bd in_seq=0 out_seq=0 2024-04-17T07:23:14.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.024+0000 7f0d7aa00640 10 client.26698 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.024+0000 7f0d7aa00640 1 -- 172.21.15.86:0/1444838894 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b73f5d18b0 con 0x55b73f6e05f0 2024-04-17T07:23:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.024+0000 7f0d7aa00640 10 client.26698 handle_command_reply: tid=0 2024-04-17T07:23:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.025+0000 7f0d9ab8d740 1 client.26698 shutdown 2024-04-17T07:23:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.025+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 msgr2=0x55b73f674360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.025+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f674360 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55b73f675d80 tx=0x7f0d880be2f0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.025+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 shutdown_connections 2024-04-17T07:23:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.025+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b73f6e05f0 0x55b73f6e29e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.025+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b73f669b00 0x55b73f674360 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.025+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b73f664990 0x55b73f673350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.025+0000 7f0d9ab8d740 1 --2- 172.21.15.86:0/1444838894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b73f663fa0 0x55b73f672e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.025+0000 7f0d7aa00640 0 client.26698 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.025+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 >> 172.21.15.86:0/1444838894 conn(0x55b73f65f510 msgr2=0x55b73f662770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.026+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 shutdown_connections 2024-04-17T07:23:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.026+0000 7f0d9ab8d740 1 -- 172.21.15.86:0/1444838894 wait complete. 2024-04-17T07:23:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.026+0000 7f0d9ab8d740 20 client.26698 trim_cache size 0 max 16384 2024-04-17T07:23:14.028 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:23:14.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.029+0000 7f0d97e00640 1 -- 172.21.15.86:0/52989163 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d740a3d00 msgr2=0x7f0d740a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:14.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.029+0000 7f0d97e00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d740a3d00 0x7f0d740a61c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x55b73f664d90 tx=0x7f0d80009500 comp rx=0 tx=0).stop 2024-04-17T07:23:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.029+0000 7f0d97e00640 1 -- 172.21.15.86:0/52989163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d901011e0 msgr2=0x7f0d901ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.029+0000 7f0d97e00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d901011e0 0x7f0d901ae540 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0d88005ab0 tx=0x7f0d88003b90 comp rx=0 tx=0).stop 2024-04-17T07:23:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.029+0000 7f0d97e00640 1 -- 172.21.15.86:0/52989163 shutdown_connections 2024-04-17T07:23:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.029+0000 7f0d97e00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d740a3d00 0x7f0d740a61c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.029+0000 7f0d97e00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d901011e0 0x7f0d901ae540 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.029+0000 7f0d97e00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d901005f0 0x7f0d901a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.029+0000 7f0d97e00640 1 --2- 172.21.15.86:0/52989163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d900ffc30 0x7f0d901a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.029+0000 7f0d97e00640 1 -- 172.21.15.86:0/52989163 >> 172.21.15.86:0/52989163 conn(0x7f0d900fb760 msgr2=0x7f0d900fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.030+0000 7f0d97e00640 1 -- 172.21.15.86:0/52989163 shutdown_connections 2024-04-17T07:23:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:14.030+0000 7f0d97e00640 1 -- 172.21.15.86:0/52989163 wait complete. 2024-04-17T07:23:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 230.0 s, 616.0 tps, lat 163.098 ms stddev 47.306 2024-04-17T07:23:14.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:14 smithi098 ceph-mon[35418]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.7 MiB/s wr, 804 op/s 2024-04-17T07:23:14.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:14 smithi103 ceph-mon[35495]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.7 MiB/s wr, 804 op/s 2024-04-17T07:23:14.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:14 smithi086 ceph-mon[31733]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.7 MiB/s wr, 804 op/s 2024-04-17T07:23:15.058 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:23:15.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:23:15.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:23:15.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.463+0000 7f5947e00640 1 -- 172.21.15.86:0/3622144911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59481102f0 msgr2=0x7f5948112790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:15.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.463+0000 7f5947e00640 1 --2- 172.21.15.86:0/3622144911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59481102f0 0x7f5948112790 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5940009ca0 tx=0x7f594002c1b0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.463+0000 7f5947e00640 1 -- 172.21.15.86:0/3622144911 shutdown_connections 2024-04-17T07:23:15.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.463+0000 7f5947e00640 1 --2- 172.21.15.86:0/3622144911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59481102f0 0x7f5948112790 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.463+0000 7f5947e00640 1 --2- 172.21.15.86:0/3622144911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59480735d0 0x7f5948073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.464+0000 7f5947e00640 1 --2- 172.21.15.86:0/3622144911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5948109c10 0x7f5948073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.464+0000 7f5947e00640 1 -- 172.21.15.86:0/3622144911 >> 172.21.15.86:0/3622144911 conn(0x7f594806ee70 msgr2=0x7f59480712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:15.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.464+0000 7f5947e00640 1 -- 172.21.15.86:0/3622144911 shutdown_connections 2024-04-17T07:23:15.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.464+0000 7f5947e00640 1 -- 172.21.15.86:0/3622144911 wait complete. 2024-04-17T07:23:15.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.467+0000 7f5947e00640 1 Processor -- start 2024-04-17T07:23:15.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.467+0000 7f5947e00640 1 -- start start 2024-04-17T07:23:15.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.467+0000 7f5947e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59480735d0 0x7f594810ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.467+0000 7f5947e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5948109c10 0x7f594810f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.468+0000 7f5947e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59481102f0 0x7f59481ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.468+0000 7f5947e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5948116230 con 0x7f59480735d0 2024-04-17T07:23:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.468+0000 7f5947e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f59481160b0 con 0x7f5948109c10 2024-04-17T07:23:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.468+0000 7f5947e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59481163b0 con 0x7f59481102f0 2024-04-17T07:23:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.468+0000 7f5946a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59480735d0 0x7f594810ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.468+0000 7f5946a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59480735d0 0x7f594810ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35910/0 (socket says 172.21.15.86:35910) 2024-04-17T07:23:15.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.468+0000 7f5946a00640 1 -- 172.21.15.86:0/432924994 learned_addr learned my addr 172.21.15.86:0/432924994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:15.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.468+0000 7f5947400640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59481102f0 0x7f59481ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.469+0000 7f5946a00640 1 -- 172.21.15.86:0/432924994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59481102f0 msgr2=0x7f59481ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:15.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.469+0000 7f5946a00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59481102f0 0x7f59481ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.469+0000 7f5946a00640 1 -- 172.21.15.86:0/432924994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5948109c10 msgr2=0x7f594810f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:15.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.469+0000 7f5946a00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5948109c10 0x7f594810f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.469+0000 7f5946a00640 1 -- 172.21.15.86:0/432924994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59481aeba0 con 0x7f59480735d0 2024-04-17T07:23:15.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.469+0000 7f5946a00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59480735d0 0x7f594810ecd0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f593800b450 tx=0x7f593800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:15.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.470+0000 7f594ca00640 1 -- 172.21.15.86:0/432924994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938015020 con 0x7f59480735d0 2024-04-17T07:23:15.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.470+0000 7f594ca00640 1 -- 172.21.15.86:0/432924994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5938011d40 con 0x7f59480735d0 2024-04-17T07:23:15.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.470+0000 7f594ca00640 1 -- 172.21.15.86:0/432924994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938010e20 con 0x7f59480735d0 2024-04-17T07:23:15.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.470+0000 7f5947e00640 1 -- 172.21.15.86:0/432924994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59481aee90 con 0x7f59480735d0 2024-04-17T07:23:15.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.470+0000 7f5947e00640 1 -- 172.21.15.86:0/432924994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59481af380 con 0x7f59480735d0 2024-04-17T07:23:15.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.476+0000 7f5950bde740 1 Processor -- start 2024-04-17T07:23:15.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.477+0000 7f594ca00640 1 -- 172.21.15.86:0/432924994 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5938018020 con 0x7f59480735d0 2024-04-17T07:23:15.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.477+0000 7f5950bde740 1 -- start start 2024-04-17T07:23:15.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.477+0000 7f5950bde740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.477+0000 7f5950bde740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.477+0000 7f5950bde740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.477+0000 7f5950bde740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5ea50bcb0 con 0x55c5ea5abc00 2024-04-17T07:23:15.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.477+0000 7f5950bde740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c5ea4b2370 con 0x55c5ea5a60a0 2024-04-17T07:23:15.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.477+0000 7f5950bde740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c5ea507c70 con 0x55c5ea5a6a90 2024-04-17T07:23:15.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.478+0000 7f5946000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.478+0000 7f5946a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.478+0000 7f5946000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35926/0 (socket says 172.21.15.86:35926) 2024-04-17T07:23:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.478+0000 7f5946a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5ab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41324/0 (socket says 172.21.15.86:41324) 2024-04-17T07:23:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.478+0000 7f5946000640 1 -- 172.21.15.86:0/2226384657 learned_addr learned my addr 172.21.15.86:0/2226384657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.478+0000 7f5947400640 1 --2- 172.21.15.86:0/2226384657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.479+0000 7f5947400640 1 -- 172.21.15.86:0/2226384657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 msgr2=0x55c5ea5ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:15.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.479+0000 7f5947400640 1 --2- 172.21.15.86:0/2226384657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5ab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.479+0000 7f5947400640 1 -- 172.21.15.86:0/2226384657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 msgr2=0x55c5ea5ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:15.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.479+0000 7f5947400640 1 --2- 172.21.15.86:0/2226384657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.479+0000 7f5947400640 1 -- 172.21.15.86:0/2226384657 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ea4d1910 con 0x55c5ea5a60a0 2024-04-17T07:23:15.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.479+0000 7f5946000640 1 --2- 172.21.15.86:0/2226384657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:15.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.479+0000 7f594ca00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f592c0a3db0 0x7f592c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.480+0000 7f594ca00640 1 -- 172.21.15.86:0/432924994 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f59380d0010 con 0x7f59480735d0 2024-04-17T07:23:15.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.480+0000 7f5946a00640 1 --2- 172.21.15.86:0/2226384657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-17T07:23:15.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.480+0000 7f5947400640 1 --2- 172.21.15.86:0/2226384657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5a64c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5940009bf0 tx=0x7f5940009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b795784e8de32d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:15.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.481+0000 7f5933e00640 1 -- 172.21.15.86:0/2226384657 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5940043070 con 0x55c5ea5a60a0 2024-04-17T07:23:15.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.481+0000 7f5933e00640 1 -- 172.21.15.86:0/2226384657 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5940003b70 con 0x55c5ea5a60a0 2024-04-17T07:23:15.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.481+0000 7f5933e00640 1 -- 172.21.15.86:0/2226384657 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5940039040 con 0x55c5ea5a60a0 2024-04-17T07:23:15.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.482+0000 7f5946a00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f592c0a3db0 0x7f592c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.482+0000 7f5950bde740 1 -- 172.21.15.86:0/2226384657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 msgr2=0x55c5ea5a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:15.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.482+0000 7f5950bde740 1 --2- 172.21.15.86:0/2226384657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5a64c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5940009bf0 tx=0x7f5940009870 comp rx=0 tx=0).stop 2024-04-17T07:23:15.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.482+0000 7f5950bde740 1 -- 172.21.15.86:0/2226384657 shutdown_connections 2024-04-17T07:23:15.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.482+0000 7f5950bde740 1 --2- 172.21.15.86:0/2226384657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.482+0000 7f5950bde740 1 --2- 172.21.15.86:0/2226384657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.482+0000 7f5950bde740 1 --2- 172.21.15.86:0/2226384657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5a64c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.482+0000 7f5950bde740 1 -- 172.21.15.86:0/2226384657 >> 172.21.15.86:0/2226384657 conn(0x55c5ea5a1610 msgr2=0x55c5ea5a1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:15.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.482+0000 7f5946a00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f592c0a3db0 0x7f592c0a6270 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f593809bea0 tx=0x7f5938009200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:15.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.483+0000 7f5950bde740 1 -- 172.21.15.86:0/2226384657 shutdown_connections 2024-04-17T07:23:15.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.483+0000 7f5950bde740 1 -- 172.21.15.86:0/2226384657 wait complete. 2024-04-17T07:23:15.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.484+0000 7f5950bde740 1 -- start start 2024-04-17T07:23:15.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.484+0000 7f5950bde740 1 Processor -- start 2024-04-17T07:23:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.484+0000 7f5950bde740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.484+0000 7f5950bde740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.484+0000 7f5950bde740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5950bde740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5ea5aeba0 con 0x55c5ea5abc00 2024-04-17T07:23:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5950bde740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c5ea4c5730 con 0x55c5ea5a60a0 2024-04-17T07:23:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5950bde740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c5ea5aed20 con 0x55c5ea5a6a90 2024-04-17T07:23:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5950bde740 10 client.0 _ll_register_callbacks cb 0x55c5ea4c7b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5946000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5946000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41338/0 (socket says 172.21.15.86:41338) 2024-04-17T07:23:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5946000640 1 -- 172.21.15.86:0/1607356525 learned_addr learned my addr 172.21.15.86:0/1607356525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:15.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5947400640 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5947400640 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5946000640 1 -- 172.21.15.86:0/1607356525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 msgr2=0x55c5ea5b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:15.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5946000640 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5946000640 1 -- 172.21.15.86:0/1607356525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 msgr2=0x55c5ea5b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:15.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5946000640 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5946000640 1 -- 172.21.15.86:0/1607356525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ea2e67e0 con 0x55c5ea5a6a90 2024-04-17T07:23:15.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.485+0000 7f5947400640 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:15.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.486+0000 7f5946000640 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b0b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5934004620 tx=0x7f5934011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f636c005302d1ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:15.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.486+0000 7f5930c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:15.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.486+0000 7f5930c00640 1 -- 172.21.15.86:0/1607356525 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593401e060 con 0x55c5ea5a6a90 2024-04-17T07:23:15.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.486+0000 7f5930c00640 1 -- 172.21.15.86:0/1607356525 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f593400fd90 con 0x55c5ea5a6a90 2024-04-17T07:23:15.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.487+0000 7f5950bde740 1 -- 172.21.15.86:0/1607356525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5940003b70 con 0x55c5ea5a6a90 2024-04-17T07:23:15.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.487+0000 7f5930c00640 1 -- 172.21.15.86:0/1607356525 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5934019430 con 0x55c5ea5a6a90 2024-04-17T07:23:15.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.487+0000 7f5950bde740 10 client.26707 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:15.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.487+0000 7f5950bde740 1 -- 172.21.15.86:0/1607356525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5ea346f70 con 0x55c5ea5a6a90 2024-04-17T07:23:15.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.488+0000 7f5930c00640 1 -- 172.21.15.86:0/1607356525 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5940003b70 con 0x55c5ea5a6a90 2024-04-17T07:23:15.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.488+0000 7f5930c00640 1 -- 172.21.15.86:0/1607356525 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f593400e5e0 con 0x55c5ea5a6a90 2024-04-17T07:23:15.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.488+0000 7f5950bde740 10 client.26707 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:15.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.488+0000 7f5950bde740 10 client.26707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:15.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.488+0000 7f5950bde740 20 client.26707 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:15.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.488+0000 7f5950bde740 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c5ea5bc4d0 0x55c5ea5be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.488+0000 7f5950bde740 4 client.26707 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:15.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.488+0000 7f5950bde740 1 -- 172.21.15.86:0/1607356525 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5ea5196d0 con 0x55c5ea5bc4d0 2024-04-17T07:23:15.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.489+0000 7f5947400640 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c5ea5bc4d0 0x55c5ea5be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.491+0000 7f5947400640 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c5ea5bc4d0 0x55c5ea5be8f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24be9e1429fe96c5 server_cookie=dc1b03b672b8e884 in_seq=0 out_seq=0 2024-04-17T07:23:15.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.491+0000 7f5930c00640 10 client.26707 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.493+0000 7f5930c00640 1 -- 172.21.15.86:0/1607356525 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5ea5196d0 con 0x55c5ea5bc4d0 2024-04-17T07:23:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.493+0000 7f5930c00640 10 client.26707 handle_command_reply: tid=0 2024-04-17T07:23:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.494+0000 7f5950bde740 1 client.26707 shutdown 2024-04-17T07:23:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.494+0000 7f5950bde740 1 -- 172.21.15.86:0/1607356525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 msgr2=0x55c5ea5b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.494+0000 7f5950bde740 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b0b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5934004620 tx=0x7f5934011b50 comp rx=0 tx=0).stop 2024-04-17T07:23:15.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.495+0000 7f5950bde740 1 -- 172.21.15.86:0/1607356525 shutdown_connections 2024-04-17T07:23:15.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.495+0000 7f5950bde740 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c5ea5bc4d0 0x55c5ea5be8f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.495+0000 7f5950bde740 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.495+0000 7f5950bde740 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b0b00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.495+0000 7f5950bde740 1 --2- 172.21.15.86:0/1607356525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.495+0000 7f5950bde740 1 -- 172.21.15.86:0/1607356525 >> 172.21.15.86:0/1607356525 conn(0x55c5ea5a1610 msgr2=0x55c5ea5a0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:15.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.495+0000 7f5930c00640 0 client.26707 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:15.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.495+0000 7f5950bde740 1 -- 172.21.15.86:0/1607356525 shutdown_connections 2024-04-17T07:23:15.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.495+0000 7f5950bde740 1 -- 172.21.15.86:0/1607356525 wait complete. 2024-04-17T07:23:15.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.496+0000 7f5950bde740 20 client.26707 trim_cache size 0 max 16384 2024-04-17T07:23:15.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.546+0000 7f5950bde740 1 Processor -- start 2024-04-17T07:23:15.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.547+0000 7f5950bde740 1 -- start start 2024-04-17T07:23:15.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.547+0000 7f5950bde740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.548+0000 7f5950bde740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.548+0000 7f5950bde740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5a0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.548+0000 7f5950bde740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f592800fb60 con 0x55c5ea5abc00 2024-04-17T07:23:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.548+0000 7f5950bde740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f59280116a0 con 0x55c5ea5a60a0 2024-04-17T07:23:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.548+0000 7f5950bde740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5928010230 con 0x55c5ea5a6a90 2024-04-17T07:23:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.549+0000 7f5946000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.549+0000 7f5946000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41340/0 (socket says 172.21.15.86:41340) 2024-04-17T07:23:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.549+0000 7f5946000640 1 -- 172.21.15.86:0/1429529895 learned_addr learned my addr 172.21.15.86:0/1429529895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.549+0000 7f5947400640 1 --2- 172.21.15.86:0/1429529895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.549+0000 7f5947400640 1 --2- 172.21.15.86:0/1429529895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.550+0000 7f5946000640 1 -- 172.21.15.86:0/1429529895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 msgr2=0x55c5ea5affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.550+0000 7f5946000640 1 --2- 172.21.15.86:0/1429529895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.550+0000 7f5946000640 1 -- 172.21.15.86:0/1429529895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 msgr2=0x55c5ea5a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.550+0000 7f5946000640 1 --2- 172.21.15.86:0/1429529895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.550+0000 7f5946000640 1 -- 172.21.15.86:0/1429529895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ea1de170 con 0x55c5ea5a6a90 2024-04-17T07:23:15.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.551+0000 7f5946000640 1 --2- 172.21.15.86:0/1429529895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b7a90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f59340095e0 tx=0x7f5934005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e929b9b7c03471d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:15.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.552+0000 7f5933400640 1 -- 172.21.15.86:0/1429529895 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593402e020 con 0x55c5ea5a6a90 2024-04-17T07:23:15.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.552+0000 7f5933400640 1 -- 172.21.15.86:0/1429529895 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f593400ec70 con 0x55c5ea5a6a90 2024-04-17T07:23:15.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.552+0000 7f5933400640 1 -- 172.21.15.86:0/1429529895 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934025030 con 0x55c5ea5a6a90 2024-04-17T07:23:15.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.553+0000 7f5950bde740 1 -- 172.21.15.86:0/1429529895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 msgr2=0x55c5ea5b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:15.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.553+0000 7f5950bde740 1 --2- 172.21.15.86:0/1429529895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b7a90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f59340095e0 tx=0x7f5934005e60 comp rx=0 tx=0).stop 2024-04-17T07:23:15.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.554+0000 7f5950bde740 1 -- 172.21.15.86:0/1429529895 shutdown_connections 2024-04-17T07:23:15.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.554+0000 7f5950bde740 1 --2- 172.21.15.86:0/1429529895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5abc00 0x55c5ea5a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.554+0000 7f5950bde740 1 --2- 172.21.15.86:0/1429529895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b7a90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.554+0000 7f5950bde740 1 --2- 172.21.15.86:0/1429529895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.554+0000 7f5950bde740 1 -- 172.21.15.86:0/1429529895 >> 172.21.15.86:0/1429529895 conn(0x55c5ea5a1610 msgr2=0x55c5ea5af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:15.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.554+0000 7f5950bde740 1 -- 172.21.15.86:0/1429529895 shutdown_connections 2024-04-17T07:23:15.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.554+0000 7f5950bde740 1 -- 172.21.15.86:0/1429529895 wait complete. 2024-04-17T07:23:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.555+0000 7f5950bde740 1 -- start start 2024-04-17T07:23:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.556+0000 7f5950bde740 1 Processor -- start 2024-04-17T07:23:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.557+0000 7f5950bde740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.558+0000 7f5950bde740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.559+0000 7f5950bde740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5abc00 0x55c5ea5b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.559+0000 7f5947400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.559+0000 7f5950bde740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5928010fd0 con 0x55c5ea5a60a0 2024-04-17T07:23:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.559+0000 7f5946000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.560+0000 7f5947400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35946/0 (socket says 172.21.15.86:35946) 2024-04-17T07:23:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.560+0000 7f5947400640 1 -- 172.21.15.86:0/2835607224 learned_addr learned my addr 172.21.15.86:0/2835607224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.560+0000 7f5950bde740 1 -- 172.21.15.86:0/2835607224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5928010900 con 0x55c5ea5abc00 2024-04-17T07:23:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.560+0000 7f5950bde740 1 -- 172.21.15.86:0/2835607224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f592800db30 con 0x55c5ea5a6a90 2024-04-17T07:23:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.560+0000 7f5950bde740 10 client.0 _ll_register_callbacks cb 0x7f5934009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.561+0000 7f5946000640 1 -- 172.21.15.86:0/2835607224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5abc00 msgr2=0x55c5ea5b67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:23:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.561+0000 7f5946000640 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5abc00 0x55c5ea5b67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.561+0000 7f5946000640 1 -- 172.21.15.86:0/2835607224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5a60a0 msgr2=0x55c5ea5b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.561+0000 7f5946000640 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.561+0000 7f5946000640 1 -- 172.21.15.86:0/2835607224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ea5b6de0 con 0x55c5ea5a6a90 2024-04-17T07:23:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.562+0000 7f5946000640 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b57e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5934009360 tx=0x7f5934009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0f81d523ebcc621 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.563+0000 7f5930c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:15.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.563+0000 7f5930c00640 1 -- 172.21.15.86:0/2835607224 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934025040 con 0x55c5ea5a6a90 2024-04-17T07:23:15.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.563+0000 7f5930c00640 1 -- 172.21.15.86:0/2835607224 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f59340195d0 con 0x55c5ea5a6a90 2024-04-17T07:23:15.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.563+0000 7f5950bde740 1 -- 172.21.15.86:0/2835607224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f593400ec70 con 0x55c5ea5a6a90 2024-04-17T07:23:15.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.564+0000 7f5930c00640 1 -- 172.21.15.86:0/2835607224 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f593401e020 con 0x55c5ea5a6a90 2024-04-17T07:23:15.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.564+0000 7f5950bde740 10 client.26713 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:15.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.564+0000 7f5950bde740 1 -- 172.21.15.86:0/2835607224 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5ea622bc0 con 0x55c5ea5a6a90 2024-04-17T07:23:15.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.565+0000 7f5930c00640 1 -- 172.21.15.86:0/2835607224 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f593400ec70 con 0x55c5ea5a6a90 2024-04-17T07:23:15.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.565+0000 7f5930c00640 1 -- 172.21.15.86:0/2835607224 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f593402a710 con 0x55c5ea5a6a90 2024-04-17T07:23:15.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.566+0000 7f5950bde740 10 client.26713 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:15.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.566+0000 7f5950bde740 10 client.26713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:15.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.566+0000 7f5950bde740 20 client.26713 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:15.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.566+0000 7f5950bde740 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c5ea623450 0x55c5ea625870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:15.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.566+0000 7f5950bde740 4 client.26713 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:23:15.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.566+0000 7f5950bde740 1 -- 172.21.15.86:0/2835607224 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c5ea513540 con 0x55c5ea623450 2024-04-17T07:23:15.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.567+0000 7f5947400640 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c5ea623450 0x55c5ea625870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:15.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.570+0000 7f5930c00640 10 client.26713 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:15.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.570+0000 7f5947400640 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c5ea623450 0x55c5ea625870 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3490cc7fca815638 server_cookie=1ac0be955d3df4cf in_seq=0 out_seq=0 2024-04-17T07:23:15.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.575+0000 7f5930c00640 1 -- 172.21.15.86:0/2835607224 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c5ea513540 con 0x55c5ea623450 2024-04-17T07:23:15.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.575+0000 7f5930c00640 10 client.26713 handle_command_reply: tid=0 2024-04-17T07:23:15.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.575+0000 7f5950bde740 1 client.26713 shutdown 2024-04-17T07:23:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.577+0000 7f5950bde740 1 -- 172.21.15.86:0/2835607224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 msgr2=0x55c5ea5b57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.577+0000 7f5950bde740 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b57e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5934009360 tx=0x7f5934009730 comp rx=0 tx=0).stop 2024-04-17T07:23:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.577+0000 7f5950bde740 1 -- 172.21.15.86:0/2835607224 shutdown_connections 2024-04-17T07:23:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.577+0000 7f5950bde740 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c5ea623450 0x55c5ea625870 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.577+0000 7f5950bde740 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c5ea5abc00 0x55c5ea5b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.577+0000 7f5930c00640 0 client.26713 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.577+0000 7f5950bde740 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c5ea5a6a90 0x55c5ea5b57e0 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5934009360 tx=0x7f5934009730 comp rx=0 tx=0).stop 2024-04-17T07:23:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.577+0000 7f5950bde740 1 --2- 172.21.15.86:0/2835607224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5ea5a60a0 0x55c5ea5b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.577+0000 7f5950bde740 1 -- 172.21.15.86:0/2835607224 >> 172.21.15.86:0/2835607224 conn(0x55c5ea5a1610 msgr2=0x55c5ea5a4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.578+0000 7f5950bde740 1 -- 172.21.15.86:0/2835607224 shutdown_connections 2024-04-17T07:23:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.578+0000 7f5950bde740 1 -- 172.21.15.86:0/2835607224 wait complete. 2024-04-17T07:23:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.579+0000 7f5950bde740 20 client.26713 trim_cache size 0 max 16384 2024-04-17T07:23:15.579 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:23:15.579 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:23:15.579 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "8d626bcd-663d-4578-b1c8-7650a4979612", 2024-04-17T07:23:15.579 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:23:15.579 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:23:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.580+0000 7f5947e00640 1 -- 172.21.15.86:0/432924994 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f592c0a3db0 msgr2=0x7f592c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.580+0000 7f5947e00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f592c0a3db0 0x7f592c0a6270 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f593809bea0 tx=0x7f5938009200 comp rx=0 tx=0).stop 2024-04-17T07:23:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.580+0000 7f5947e00640 1 -- 172.21.15.86:0/432924994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59480735d0 msgr2=0x7f594810ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.580+0000 7f5947e00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59480735d0 0x7f594810ecd0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f593800b450 tx=0x7f593800b910 comp rx=0 tx=0).stop 2024-04-17T07:23:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.581+0000 7f5947e00640 1 -- 172.21.15.86:0/432924994 shutdown_connections 2024-04-17T07:23:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.581+0000 7f5947e00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f592c0a3db0 0x7f592c0a6270 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.581+0000 7f5947e00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f59481102f0 0x7f59481ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.581+0000 7f5947e00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5948109c10 0x7f594810f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.581+0000 7f5947e00640 1 --2- 172.21.15.86:0/432924994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59480735d0 0x7f594810ecd0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.581+0000 7f5947e00640 1 -- 172.21.15.86:0/432924994 >> 172.21.15.86:0/432924994 conn(0x7f594806ee70 msgr2=0x7f594819b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.582+0000 7f5947e00640 1 -- 172.21.15.86:0/432924994 shutdown_connections 2024-04-17T07:23:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:15.583+0000 7f5947e00640 1 -- 172.21.15.86:0/432924994 wait complete. 2024-04-17T07:23:15.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:23:15.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:23:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.058+0000 7f4414c00640 1 -- 172.21.15.86:0/2236022395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4410101fb0 msgr2=0x7f44101023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:16.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.058+0000 7f4414c00640 1 --2- 172.21.15.86:0/2236022395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4410101fb0 0x7f44101023d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4400009950 tx=0x7f440002bcd0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.058+0000 7f4414c00640 1 -- 172.21.15.86:0/2236022395 shutdown_connections 2024-04-17T07:23:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.059+0000 7f4414c00640 1 --2- 172.21.15.86:0/2236022395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4410103b60 0x7f441010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.059+0000 7f4414c00640 1 --2- 172.21.15.86:0/2236022395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44101031a0 0x7f4410103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.059+0000 7f4414c00640 1 --2- 172.21.15.86:0/2236022395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4410101fb0 0x7f44101023d0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.059+0000 7f4414c00640 1 -- 172.21.15.86:0/2236022395 >> 172.21.15.86:0/2236022395 conn(0x7f44100fd540 msgr2=0x7f44100ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:16.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.059+0000 7f4414c00640 1 -- 172.21.15.86:0/2236022395 shutdown_connections 2024-04-17T07:23:16.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.060+0000 7f4414c00640 1 -- 172.21.15.86:0/2236022395 wait complete. 2024-04-17T07:23:16.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.061+0000 7f4414c00640 1 Processor -- start 2024-04-17T07:23:16.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.062+0000 7f4414c00640 1 -- start start 2024-04-17T07:23:16.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.062+0000 7f4414c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4410101fb0 0x7f4410196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.062+0000 7f4414c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44101031a0 0x7f4410196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.062+0000 7f4414c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4410103b60 0x7f44101aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.062+0000 7f4414c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f441010dc40 con 0x7f4410103b60 2024-04-17T07:23:16.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.062+0000 7f4414c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f441010dac0 con 0x7f44101031a0 2024-04-17T07:23:16.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.062+0000 7f4414c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f441010ddc0 con 0x7f4410101fb0 2024-04-17T07:23:16.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.063+0000 7f440de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4410101fb0 0x7f4410196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.063+0000 7f440de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4410101fb0 0x7f4410196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41368/0 (socket says 172.21.15.86:41368) 2024-04-17T07:23:16.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.063+0000 7f440de00640 1 -- 172.21.15.86:0/3502138546 learned_addr learned my addr 172.21.15.86:0/3502138546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:16.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.063+0000 7f440de00640 1 -- 172.21.15.86:0/3502138546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44101031a0 msgr2=0x7f4410196780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T07:23:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.063+0000 7f440de00640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44101031a0 0x7f4410196780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.064+0000 7f440de00640 1 -- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4410103b60 msgr2=0x7f44101aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:16.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.065+0000 7f440e800640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4410103b60 0x7f44101aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.066+0000 7f440de00640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4410103b60 0x7f44101aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.066+0000 7f440de00640 1 -- 172.21.15.86:0/3502138546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4410196e20 con 0x7f4410101fb0 2024-04-17T07:23:16.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.066+0000 7f440e800640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4410103b60 0x7f44101aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:23:16.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.067+0000 7f440de00640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4410101fb0 0x7f4410196240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4400006010 tx=0x7f4400004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:16.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.067+0000 7f4406a00640 1 -- 172.21.15.86:0/3502138546 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400043020 con 0x7f4410101fb0 2024-04-17T07:23:16.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.067+0000 7f4406a00640 1 -- 172.21.15.86:0/3502138546 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4400004490 con 0x7f4410101fb0 2024-04-17T07:23:16.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.067+0000 7f4406a00640 1 -- 172.21.15.86:0/3502138546 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440003f470 con 0x7f4410101fb0 2024-04-17T07:23:16.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.067+0000 7f4414c00640 1 -- 172.21.15.86:0/3502138546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44101af150 con 0x7f4410101fb0 2024-04-17T07:23:16.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.068+0000 7f4414c00640 1 -- 172.21.15.86:0/3502138546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44101af6a0 con 0x7f4410101fb0 2024-04-17T07:23:16.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.075+0000 7f4417824740 1 Processor -- start 2024-04-17T07:23:16.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.075+0000 7f4417824740 1 -- start start 2024-04-17T07:23:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.076+0000 7f4417824740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bca9190 0x56118bca95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.076+0000 7f4417824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.076+0000 7f4417824740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 0x56118bcb1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.076+0000 7f4417824740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56118bc10120 con 0x56118bcaecf0 2024-04-17T07:23:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.076+0000 7f4417824740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56118bbb6340 con 0x56118bca9190 2024-04-17T07:23:16.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.076+0000 7f4417824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56118bc0c0e0 con 0x56118bca9b80 2024-04-17T07:23:16.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.076+0000 7f4406a00640 1 -- 172.21.15.86:0/3502138546 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4400046020 con 0x7f4410101fb0 2024-04-17T07:23:16.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.077+0000 7f440de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.077+0000 7f440de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41382/0 (socket says 172.21.15.86:41382) 2024-04-17T07:23:16.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.077+0000 7f440de00640 1 -- 172.21.15.86:0/345840515 learned_addr learned my addr 172.21.15.86:0/345840515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:16.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.078+0000 7f440de00640 1 -- 172.21.15.86:0/345840515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bca9190 msgr2=0x56118bca95b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:23:16.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.078+0000 7f440de00640 1 --2- 172.21.15.86:0/345840515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bca9190 0x56118bca95b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.078+0000 7f440de00640 1 -- 172.21.15.86:0/345840515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 msgr2=0x56118bcb1180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:23:16.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.080+0000 7f4406a00640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43f40a3d80 0x7f43f40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.080+0000 7f4406a00640 1 -- 172.21.15.86:0/3502138546 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f44000f2fb0 con 0x7f4410101fb0 2024-04-17T07:23:16.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.081+0000 7f440d400640 1 --2- 172.21.15.86:0/345840515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 0x56118bcb1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.081+0000 7f440de00640 1 --2- 172.21.15.86:0/345840515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 0x56118bcb1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.081+0000 7f440de00640 1 -- 172.21.15.86:0/345840515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118b9ec7d0 con 0x56118bca9b80 2024-04-17T07:23:16.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.081+0000 7f440de00640 1 --2- 172.21.15.86:0/345840515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcae680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f44000bb280 tx=0x7f440002bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa42931348899ca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:16.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.081+0000 7f440e800640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43f40a3d80 0x7f43f40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.081+0000 7f43fb000640 1 -- 172.21.15.86:0/345840515 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000bb0c0 con 0x56118bca9b80 2024-04-17T07:23:16.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.082+0000 7f43fb000640 1 -- 172.21.15.86:0/345840515 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4400004c50 con 0x56118bca9b80 2024-04-17T07:23:16.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.082+0000 7f43fb000640 1 -- 172.21.15.86:0/345840515 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000bcff0 con 0x56118bca9b80 2024-04-17T07:23:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.083+0000 7f4417824740 1 -- 172.21.15.86:0/345840515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 msgr2=0x56118bcae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.083+0000 7f440e800640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43f40a3d80 0x7f43f40a6240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x56118bcaa3e0 tx=0x7f4408007290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.083+0000 7f4417824740 1 --2- 172.21.15.86:0/345840515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcae680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f44000bb280 tx=0x7f440002bcd0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.083+0000 7f4417824740 1 -- 172.21.15.86:0/345840515 shutdown_connections 2024-04-17T07:23:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.083+0000 7f4417824740 1 --2- 172.21.15.86:0/345840515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 0x56118bcb1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.083+0000 7f4417824740 1 --2- 172.21.15.86:0/345840515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcae680 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.083+0000 7f4417824740 1 --2- 172.21.15.86:0/345840515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bca9190 0x56118bca95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.083+0000 7f4417824740 1 -- 172.21.15.86:0/345840515 >> 172.21.15.86:0/345840515 conn(0x56118bca4700 msgr2=0x56118bca4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.083+0000 7f4417824740 1 -- 172.21.15.86:0/345840515 shutdown_connections 2024-04-17T07:23:16.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.083+0000 7f4417824740 1 -- 172.21.15.86:0/345840515 wait complete. 2024-04-17T07:23:16.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.084+0000 7f4417824740 1 -- start start 2024-04-17T07:23:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.084+0000 7f4417824740 1 Processor -- start 2024-04-17T07:23:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.085+0000 7f4417824740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bca9190 0x56118bcb3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.085+0000 7f4417824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcb39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.085+0000 7f4417824740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 0x56118bcba9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.085+0000 7f4417824740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56118bcb1a60 con 0x56118bcaecf0 2024-04-17T07:23:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.086+0000 7f4417824740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56118bbc9710 con 0x56118bca9190 2024-04-17T07:23:16.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.086+0000 7f4417824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56118bcb1be0 con 0x56118bca9b80 2024-04-17T07:23:16.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.086+0000 7f4417824740 10 client.0 _ll_register_callbacks cb 0x56118bbc4af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:16.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.085+0000 7f440d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcb39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.086+0000 7f440d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcb39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41386/0 (socket says 172.21.15.86:41386) 2024-04-17T07:23:16.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.086+0000 7f440d400640 1 -- 172.21.15.86:0/448003752 learned_addr learned my addr 172.21.15.86:0/448003752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.086+0000 7f440e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 0x56118bcba9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.086+0000 7f440de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bca9190 0x56118bcb3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.087+0000 7f440de00640 1 -- 172.21.15.86:0/448003752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 msgr2=0x56118bcb39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.087+0000 7f440de00640 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcb39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.087+0000 7f440de00640 1 -- 172.21.15.86:0/448003752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 msgr2=0x56118bcba9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.087+0000 7f440de00640 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 0x56118bcba9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.087+0000 7f440de00640 1 -- 172.21.15.86:0/448003752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118b9223f0 con 0x56118bca9190 2024-04-17T07:23:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.087+0000 7f440e800640 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 0x56118bcba9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.088+0000 7f440de00640 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bca9190 0x56118bcb3460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x56118bca9f80 tx=0x7f44000bef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5056194b47d64b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:16.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.088+0000 7f43ef400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:23:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.088+0000 7f43ef400640 1 -- 172.21.15.86:0/448003752 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000be810 con 0x56118bca9190 2024-04-17T07:23:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.088+0000 7f43ef400640 1 -- 172.21.15.86:0/448003752 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f44000bd1b0 con 0x56118bca9190 2024-04-17T07:23:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.088+0000 7f4417824740 1 -- 172.21.15.86:0/448003752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4400004c50 con 0x56118bca9190 2024-04-17T07:23:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.089+0000 7f43ef400640 1 -- 172.21.15.86:0/448003752 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44000be9f0 con 0x56118bca9190 2024-04-17T07:23:16.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.089+0000 7f4417824740 10 client.36674 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:16.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.089+0000 7f4417824740 1 -- 172.21.15.86:0/448003752 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56118b8e20f0 con 0x56118bca9190 2024-04-17T07:23:16.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.090+0000 7f43ef400640 1 -- 172.21.15.86:0/448003752 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4400004c50 con 0x56118bca9190 2024-04-17T07:23:16.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.091+0000 7f43ef400640 1 -- 172.21.15.86:0/448003752 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f44000d8220 con 0x56118bca9190 2024-04-17T07:23:16.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.091+0000 7f4417824740 10 client.36674 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:16.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.091+0000 7f4417824740 10 client.36674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:16.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.091+0000 7f4417824740 20 client.36674 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:16.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.091+0000 7f4417824740 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56118bccfee0 0x56118bcd22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.091+0000 7f4417824740 4 client.36674 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:16.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.091+0000 7f4417824740 1 -- 172.21.15.86:0/448003752 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56118bc1db40 con 0x56118bccfee0 2024-04-17T07:23:16.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.092+0000 7f440d400640 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56118bccfee0 0x56118bcd22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.093+0000 7f440d400640 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56118bccfee0 0x56118bcd22d0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ecee1bbc551dc31 server_cookie=7edf47c3ee45d50a in_seq=0 out_seq=0 2024-04-17T07:23:16.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.093+0000 7f43ef400640 10 client.36674 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f43ef400640 1 -- 172.21.15.86:0/448003752 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56118bc1db40 con 0x56118bccfee0 2024-04-17T07:23:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f43ef400640 10 client.36674 handle_command_reply: tid=0 2024-04-17T07:23:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f4417824740 1 client.36674 shutdown 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f4417824740 1 -- 172.21.15.86:0/448003752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bca9190 msgr2=0x56118bcb3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f4417824740 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bca9190 0x56118bcb3460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x56118bca9f80 tx=0x7f44000bef20 comp rx=0 tx=0).stop 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f4417824740 1 -- 172.21.15.86:0/448003752 shutdown_connections 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f4417824740 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56118bccfee0 0x56118bcd22d0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f4417824740 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bcaecf0 0x56118bcba9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f4417824740 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcb39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f4417824740 1 --2- 172.21.15.86:0/448003752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bca9190 0x56118bcb3460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f4417824740 1 -- 172.21.15.86:0/448003752 >> 172.21.15.86:0/448003752 conn(0x56118bca4700 msgr2=0x56118bca3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.095+0000 7f43ef400640 0 client.36674 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.096+0000 7f4417824740 1 -- 172.21.15.86:0/448003752 shutdown_connections 2024-04-17T07:23:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.096+0000 7f4417824740 1 -- 172.21.15.86:0/448003752 wait complete. 2024-04-17T07:23:16.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.096+0000 7f4417824740 20 client.36674 trim_cache size 0 max 16384 2024-04-17T07:23:16.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.145+0000 7f4417824740 1 Processor -- start 2024-04-17T07:23:16.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.145+0000 7f4417824740 1 -- start start 2024-04-17T07:23:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.146+0000 7f4417824740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9190 0x56118bcb3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.146+0000 7f4417824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.146+0000 7f4417824740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 0x56118bca3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.146+0000 7f4417824740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44101b1110 con 0x56118bca9190 2024-04-17T07:23:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.146+0000 7f4417824740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44101b2c50 con 0x56118bcaecf0 2024-04-17T07:23:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.146+0000 7f4417824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44101b17e0 con 0x56118bca9b80 2024-04-17T07:23:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.146+0000 7f440de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9190 0x56118bcb3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.146+0000 7f440de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9190 0x56118bcb3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35998/0 (socket says 172.21.15.86:35998) 2024-04-17T07:23:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.146+0000 7f440de00640 1 -- 172.21.15.86:0/3380345130 learned_addr learned my addr 172.21.15.86:0/3380345130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:16.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.146+0000 7f440e800640 1 --2- 172.21.15.86:0/3380345130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 0x56118bca3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.147+0000 7f440de00640 1 -- 172.21.15.86:0/3380345130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 msgr2=0x56118bcba9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.147+0000 7f440de00640 1 --2- 172.21.15.86:0/3380345130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.147+0000 7f440de00640 1 -- 172.21.15.86:0/3380345130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 msgr2=0x56118bca3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.147+0000 7f440de00640 1 --2- 172.21.15.86:0/3380345130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 0x56118bca3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.147+0000 7f440de00640 1 -- 172.21.15.86:0/3380345130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118bcb2540 con 0x56118bca9190 2024-04-17T07:23:16.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.149+0000 7f440de00640 1 --2- 172.21.15.86:0/3380345130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9190 0x56118bcb3780 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f44000bee40 tx=0x7f4400005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f837e666db131f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:16.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.149+0000 7f43fa600640 1 -- 172.21.15.86:0/3380345130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000bec80 con 0x56118bca9190 2024-04-17T07:23:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.150+0000 7f43fa600640 1 -- 172.21.15.86:0/3380345130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4400004c50 con 0x56118bca9190 2024-04-17T07:23:16.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.150+0000 7f43fa600640 1 -- 172.21.15.86:0/3380345130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000bcff0 con 0x56118bca9190 2024-04-17T07:23:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.151+0000 7f4417824740 1 -- 172.21.15.86:0/3380345130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9190 msgr2=0x56118bcb3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:16.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.151+0000 7f4417824740 1 --2- 172.21.15.86:0/3380345130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9190 0x56118bcb3780 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f44000bee40 tx=0x7f4400005d50 comp rx=0 tx=0).stop 2024-04-17T07:23:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.152+0000 7f4417824740 1 -- 172.21.15.86:0/3380345130 shutdown_connections 2024-04-17T07:23:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.152+0000 7f4417824740 1 --2- 172.21.15.86:0/3380345130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 0x56118bca3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.152+0000 7f4417824740 1 --2- 172.21.15.86:0/3380345130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9b80 0x56118bcba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.152+0000 7f4417824740 1 --2- 172.21.15.86:0/3380345130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9190 0x56118bcb3780 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.152+0000 7f4417824740 1 -- 172.21.15.86:0/3380345130 >> 172.21.15.86:0/3380345130 conn(0x56118bca4700 msgr2=0x56118bcb3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:16.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.153+0000 7f4417824740 1 -- 172.21.15.86:0/3380345130 shutdown_connections 2024-04-17T07:23:16.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.153+0000 7f4417824740 1 -- 172.21.15.86:0/3380345130 wait complete. 2024-04-17T07:23:16.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.154+0000 7f4417824740 1 -- start start 2024-04-17T07:23:16.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.154+0000 7f4417824740 1 Processor -- start 2024-04-17T07:23:16.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.155+0000 7f4417824740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9190 0x56118bcb7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.155+0000 7f4417824740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9b80 0x56118bcb8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.155+0000 7f4417824740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 0x56118bcb9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.155+0000 7f4417824740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44101b2580 con 0x56118bca9b80 2024-04-17T07:23:16.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.155+0000 7f4417824740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44101b1eb0 con 0x56118bcaecf0 2024-04-17T07:23:16.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.155+0000 7f4417824740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4410073a60 con 0x56118bca9190 2024-04-17T07:23:16.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.156+0000 7f4417824740 10 client.0 _ll_register_callbacks cb 0x7f44000f6d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:16.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.156+0000 7f440de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9190 0x56118bcb7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.156+0000 7f440de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9190 0x56118bcb7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41400/0 (socket says 172.21.15.86:41400) 2024-04-17T07:23:16.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.156+0000 7f440de00640 1 -- 172.21.15.86:0/3998451991 learned_addr learned my addr 172.21.15.86:0/3998451991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:16.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.156+0000 7f440d400640 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9b80 0x56118bcb8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.156+0000 7f440e800640 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 0x56118bcb9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.158+0000 7f440e800640 1 -- 172.21.15.86:0/3998451991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9190 msgr2=0x56118bcb7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.158+0000 7f440e800640 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9190 0x56118bcb7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.158+0000 7f440e800640 1 -- 172.21.15.86:0/3998451991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9b80 msgr2=0x56118bcb8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.158+0000 7f440e800640 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9b80 0x56118bcb8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.158+0000 7f440e800640 1 -- 172.21.15.86:0/3998451991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118bcb9620 con 0x56118bcaecf0 2024-04-17T07:23:16.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.158+0000 7f440d400640 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9b80 0x56118bcb8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:16.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.159+0000 7f440e800640 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 0x56118bcb9030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f440800ff10 tx=0x7f4408004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=912eba7a60f0f760 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:16.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.161+0000 7f43ef400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:23:16.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.161+0000 7f43ef400640 1 -- 172.21.15.86:0/3998451991 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408027470 con 0x56118bcaecf0 2024-04-17T07:23:16.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.161+0000 7f43ef400640 1 -- 172.21.15.86:0/3998451991 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4408027610 con 0x56118bcaecf0 2024-04-17T07:23:16.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.161+0000 7f4417824740 1 -- 172.21.15.86:0/3998451991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4400004c50 con 0x56118bcaecf0 2024-04-17T07:23:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.161+0000 7f43ef400640 1 -- 172.21.15.86:0/3998451991 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4408027810 con 0x56118bcaecf0 2024-04-17T07:23:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.161+0000 7f4417824740 10 client.36680 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.161+0000 7f4417824740 1 -- 172.21.15.86:0/3998451991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56118bcba9a0 con 0x56118bcaecf0 2024-04-17T07:23:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.162+0000 7f43ef400640 1 -- 172.21.15.86:0/3998451991 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4400004c50 con 0x56118bcaecf0 2024-04-17T07:23:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.162+0000 7f43ef400640 1 -- 172.21.15.86:0/3998451991 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4408031a80 con 0x56118bcaecf0 2024-04-17T07:23:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.164+0000 7f4417824740 10 client.36680 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.164+0000 7f4417824740 10 client.36680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:16.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.164+0000 7f4417824740 20 client.36680 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:16.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.164+0000 7f4417824740 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56118bd36780 0x56118bd38ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:16.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.164+0000 7f4417824740 4 client.36680 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:23:16.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.164+0000 7f4417824740 1 -- 172.21.15.86:0/3998451991 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56118bc179b0 con 0x56118bd36780 2024-04-17T07:23:16.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.165+0000 7f440de00640 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56118bd36780 0x56118bd38ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:16.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.168+0000 7f440de00640 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56118bd36780 0x56118bd38ba0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c8ef078c6944e8 server_cookie=3a0d2426814ce579 in_seq=0 out_seq=0 2024-04-17T07:23:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.169+0000 7f43ef400640 10 client.36680 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.169+0000 7f43ef400640 1 -- 172.21.15.86:0/3998451991 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56118bc179b0 con 0x56118bd36780 2024-04-17T07:23:16.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.169+0000 7f43ef400640 10 client.36680 handle_command_reply: tid=0 2024-04-17T07:23:16.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.169+0000 7f4417824740 1 client.36680 shutdown 2024-04-17T07:23:16.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.171+0000 7f4417824740 1 -- 172.21.15.86:0/3998451991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 msgr2=0x56118bcb9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:16.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.171+0000 7f4417824740 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 0x56118bcb9030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f440800ff10 tx=0x7f4408004ad0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.172+0000 7f4417824740 1 -- 172.21.15.86:0/3998451991 shutdown_connections 2024-04-17T07:23:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.172+0000 7f4417824740 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56118bd36780 0x56118bd38ba0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.172+0000 7f4417824740 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56118bcaecf0 0x56118bcb9030 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.172+0000 7f4417824740 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56118bca9b80 0x56118bcb8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.172+0000 7f43ef400640 0 client.36680 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.172+0000 7f4417824740 1 --2- 172.21.15.86:0/3998451991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56118bca9190 0x56118bcb7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.172+0000 7f4417824740 1 -- 172.21.15.86:0/3998451991 >> 172.21.15.86:0/3998451991 conn(0x56118bca4700 msgr2=0x56118bca3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.173+0000 7f4417824740 1 -- 172.21.15.86:0/3998451991 shutdown_connections 2024-04-17T07:23:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.173+0000 7f4417824740 1 -- 172.21.15.86:0/3998451991 wait complete. 2024-04-17T07:23:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.173+0000 7f4417824740 20 client.36680 trim_cache size 0 max 16384 2024-04-17T07:23:16.174 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:23:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:23:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:23:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "8d626bcd-663d-4578-b1c8-7650a4979612": { 2024-04-17T07:23:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:23:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "8d626bcd-663d-4578-b1c8-7650a4979612", 2024-04-17T07:23:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:23:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:23:16.174 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:23:16.174 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:23:16.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.175+0000 7f4414c00640 1 -- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43f40a3d80 msgr2=0x7f43f40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.177+0000 7f4414c00640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43f40a3d80 0x7f43f40a6240 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x56118bcaa3e0 tx=0x7f4408007290 comp rx=0 tx=0).stop 2024-04-17T07:23:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.177+0000 7f4414c00640 1 -- 172.21.15.86:0/3502138546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4410101fb0 msgr2=0x7f4410196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.177+0000 7f4414c00640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4410101fb0 0x7f4410196240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4400006010 tx=0x7f4400004f40 comp rx=0 tx=0).stop 2024-04-17T07:23:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.177+0000 7f4414c00640 1 -- 172.21.15.86:0/3502138546 shutdown_connections 2024-04-17T07:23:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.177+0000 7f4414c00640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f43f40a3d80 0x7f43f40a6240 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.177+0000 7f4414c00640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4410103b60 0x7f44101aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.177+0000 7f4414c00640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44101031a0 0x7f4410196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.177+0000 7f4414c00640 1 --2- 172.21.15.86:0/3502138546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4410101fb0 0x7f4410196240 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4400006010 tx=0x7f4400004f40 comp rx=0 tx=0).stop 2024-04-17T07:23:16.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.177+0000 7f4414c00640 1 -- 172.21.15.86:0/3502138546 >> 172.21.15.86:0/3502138546 conn(0x7f44100fd540 msgr2=0x7f4410108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.178+0000 7f4414c00640 1 -- 172.21.15.86:0/3502138546 shutdown_connections 2024-04-17T07:23:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:16.178+0000 7f4414c00640 1 -- 172.21.15.86:0/3502138546 wait complete. 2024-04-17T07:23:16.205 INFO:tasks.cephfs.filesystem:scrub status for tag:8d626bcd-663d-4578-b1c8-7650a4979612 - {'path': '/', 'tag': '8d626bcd-663d-4578-b1c8-7650a4979612', 'options': 'recursive,force'} 2024-04-17T07:23:16.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:16 smithi098 ceph-mon[35418]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 688 op/s 2024-04-17T07:23:16.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:16 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:23:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:16 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:23:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:16 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:16 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:23:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:16 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:16 smithi103 ceph-mon[35495]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 688 op/s 2024-04-17T07:23:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:16 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:23:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:16 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:23:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:16 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:16 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:23:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:16 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:16 smithi086 ceph-mon[31733]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 688 op/s 2024-04-17T07:23:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:16 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:23:16.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:16 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:23:16.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:16 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:16.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:16 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:23:16.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:16 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:17.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:17.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:17.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:17.834 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:17 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:18.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:18.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:18.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:18.063 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:17 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:18.064 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:18.064 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:18.064 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:18.064 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:17 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:19.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:19 smithi103 ceph-mon[35495]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.9 MiB/s wr, 768 op/s 2024-04-17T07:23:19.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:19 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:23:19.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:19 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:23:19.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:19 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:19.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:19 smithi086 ceph-mon[31733]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.9 MiB/s wr, 768 op/s 2024-04-17T07:23:19.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:19 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:23:19.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:19 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:23:19.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:19 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 235.0 s, 613.6 tps, lat 163.375 ms stddev 65.022 2024-04-17T07:23:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:19 smithi098 ceph-mon[35418]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.9 MiB/s wr, 768 op/s 2024-04-17T07:23:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:19 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:23:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:19 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:23:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:19 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:23:20.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:20 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:23:20.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:20 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:23:20.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:20 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:23:21.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:21 smithi086 ceph-mon[31733]: pgmap v709: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.1 MiB/s wr, 701 op/s 2024-04-17T07:23:21.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:21 smithi098 ceph-mon[35418]: pgmap v709: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.1 MiB/s wr, 701 op/s 2024-04-17T07:23:21.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:21 smithi103 ceph-mon[35495]: pgmap v709: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.1 MiB/s wr, 701 op/s 2024-04-17T07:23:23.305 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:23 smithi098 ceph-mon[35418]: pgmap v710: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.0 MiB/s wr, 765 op/s 2024-04-17T07:23:23.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:23 smithi086 ceph-mon[31733]: pgmap v710: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.0 MiB/s wr, 765 op/s 2024-04-17T07:23:23.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:23 smithi103 ceph-mon[35495]: pgmap v710: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.0 MiB/s wr, 765 op/s 2024-04-17T07:23:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 240.0 s, 713.4 tps, lat 139.279 ms stddev 38.832 2024-04-17T07:23:25.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:25 smithi086 ceph-mon[31733]: pgmap v711: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.7 MiB/s wr, 744 op/s 2024-04-17T07:23:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:25 smithi098 ceph-mon[35418]: pgmap v711: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.7 MiB/s wr, 744 op/s 2024-04-17T07:23:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:25 smithi103 ceph-mon[35495]: pgmap v711: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.7 MiB/s wr, 744 op/s 2024-04-17T07:23:27.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:27 smithi086 ceph-mon[31733]: pgmap v712: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 618 op/s 2024-04-17T07:23:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:27 smithi098 ceph-mon[35418]: pgmap v712: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 618 op/s 2024-04-17T07:23:27.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:27 smithi103 ceph-mon[35495]: pgmap v712: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 618 op/s 2024-04-17T07:23:29.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:29 smithi086 ceph-mon[31733]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.2 MiB/s wr, 721 op/s 2024-04-17T07:23:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 245.0 s, 734.8 tps, lat 136.716 ms stddev 38.960 2024-04-17T07:23:29.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:29 smithi098 ceph-mon[35418]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.2 MiB/s wr, 721 op/s 2024-04-17T07:23:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:29 smithi103 ceph-mon[35495]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.2 MiB/s wr, 721 op/s 2024-04-17T07:23:31.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:31 smithi086 ceph-mon[31733]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.0 MiB/s wr, 703 op/s 2024-04-17T07:23:31.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:31 smithi098 ceph-mon[35418]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.0 MiB/s wr, 703 op/s 2024-04-17T07:23:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:31 smithi103 ceph-mon[35495]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.0 MiB/s wr, 703 op/s 2024-04-17T07:23:33.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:33 smithi086 ceph-mon[31733]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.1 MiB/s wr, 792 op/s 2024-04-17T07:23:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:33 smithi098 ceph-mon[35418]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.1 MiB/s wr, 792 op/s 2024-04-17T07:23:33.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:33 smithi103 ceph-mon[35495]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.1 MiB/s wr, 792 op/s 2024-04-17T07:23:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 250.0 s, 683.0 tps, lat 146.090 ms stddev 39.171 2024-04-17T07:23:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:35 smithi098 ceph-mon[35418]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-04-17T07:23:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:35 smithi103 ceph-mon[35495]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-04-17T07:23:35.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:35 smithi086 ceph-mon[31733]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-04-17T07:23:37.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:37 smithi098 ceph-mon[35418]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-04-17T07:23:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:37 smithi103 ceph-mon[35495]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-04-17T07:23:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:37 smithi086 ceph-mon[31733]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-04-17T07:23:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 255.0 s, 716.0 tps, lat 139.528 ms stddev 38.071 2024-04-17T07:23:39.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:39 smithi098 ceph-mon[35418]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 774 op/s 2024-04-17T07:23:39.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:39 smithi103 ceph-mon[35495]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 774 op/s 2024-04-17T07:23:39.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:39 smithi086 ceph-mon[31733]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 774 op/s 2024-04-17T07:23:41.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:41 smithi098 ceph-mon[35418]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.1 MiB/s wr, 730 op/s 2024-04-17T07:23:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:41 smithi103 ceph-mon[35495]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.1 MiB/s wr, 730 op/s 2024-04-17T07:23:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:41 smithi086 ceph-mon[31733]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.1 MiB/s wr, 730 op/s 2024-04-17T07:23:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:43 smithi098 ceph-mon[35418]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.6 MiB/s wr, 783 op/s 2024-04-17T07:23:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:43 smithi103 ceph-mon[35495]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.6 MiB/s wr, 783 op/s 2024-04-17T07:23:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:43 smithi086 ceph-mon[31733]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.6 MiB/s wr, 783 op/s 2024-04-17T07:23:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 260.0 s, 711.6 tps, lat 140.629 ms stddev 38.786 2024-04-17T07:23:45.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:45 smithi098 ceph-mon[35418]: pgmap v721: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 768 op/s 2024-04-17T07:23:45.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:45 smithi103 ceph-mon[35495]: pgmap v721: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 768 op/s 2024-04-17T07:23:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:45 smithi086 ceph-mon[31733]: pgmap v721: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 768 op/s 2024-04-17T07:23:46.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:23:46.207 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:23:46.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.544+0000 7f3198c00640 1 -- 172.21.15.86:0/1819932477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3194076920 msgr2=0x7f3194076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.544+0000 7f3198c00640 1 --2- 172.21.15.86:0/1819932477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3194076920 0x7f3194076d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3184009860 tx=0x7f318402bb70 comp rx=0 tx=0).stop 2024-04-17T07:23:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.544+0000 7f3198c00640 1 -- 172.21.15.86:0/1819932477 shutdown_connections 2024-04-17T07:23:46.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.544+0000 7f3198c00640 1 --2- 172.21.15.86:0/1819932477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3194078500 0x7f3194078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.544+0000 7f3198c00640 1 --2- 172.21.15.86:0/1819932477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194077b60 0x7f3194077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.544+0000 7f3198c00640 1 --2- 172.21.15.86:0/1819932477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3194076920 0x7f3194076d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.544+0000 7f3198c00640 1 -- 172.21.15.86:0/1819932477 >> 172.21.15.86:0/1819932477 conn(0x7f31940ff750 msgr2=0x7f3194101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:46.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.546+0000 7f3198c00640 1 -- 172.21.15.86:0/1819932477 shutdown_connections 2024-04-17T07:23:46.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.546+0000 7f3198c00640 1 -- 172.21.15.86:0/1819932477 wait complete. 2024-04-17T07:23:46.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.548+0000 7f3198c00640 1 Processor -- start 2024-04-17T07:23:46.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.548+0000 7f3198c00640 1 -- start start 2024-04-17T07:23:46.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.548+0000 7f3198c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3194076920 0x7f3194070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.548+0000 7f3198c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3194077b60 0x7f3194070f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.548+0000 7f3198c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194078500 0x7f31941ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.548+0000 7f3198c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3194114120 con 0x7f3194078500 2024-04-17T07:23:46.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.548+0000 7f3198c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3194113fa0 con 0x7f3194076920 2024-04-17T07:23:46.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.548+0000 7f3198c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31941142a0 con 0x7f3194077b60 2024-04-17T07:23:46.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.549+0000 7f3191e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3194076920 0x7f3194070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.549+0000 7f3191400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3194077b60 0x7f3194070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.549+0000 7f3192800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194078500 0x7f31941ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.549+0000 7f3191400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3194077b60 0x7f3194070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47468/0 (socket says 172.21.15.86:47468) 2024-04-17T07:23:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.549+0000 7f3191400640 1 -- 172.21.15.86:0/2550215409 learned_addr learned my addr 172.21.15.86:0/2550215409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.550+0000 7f3191400640 1 -- 172.21.15.86:0/2550215409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3194076920 msgr2=0x7f3194070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.550+0000 7f3191400640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3194076920 0x7f3194070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.550+0000 7f3191400640 1 -- 172.21.15.86:0/2550215409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194078500 msgr2=0x7f31941ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.550+0000 7f3191400640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194078500 0x7f31941ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.550+0000 7f3191400640 1 -- 172.21.15.86:0/2550215409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31941aed30 con 0x7f3194077b60 2024-04-17T07:23:46.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.550+0000 7f3191400640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3194077b60 0x7f3194070f50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f318000e9c0 tx=0x7f318000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:46.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.551+0000 7f318aa00640 1 -- 172.21.15.86:0/2550215409 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318000ccc0 con 0x7f3194077b60 2024-04-17T07:23:46.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.551+0000 7f3198c00640 1 -- 172.21.15.86:0/2550215409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31941aefc0 con 0x7f3194077b60 2024-04-17T07:23:46.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.551+0000 7f3198c00640 1 -- 172.21.15.86:0/2550215409 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31941af480 con 0x7f3194077b60 2024-04-17T07:23:46.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.551+0000 7f318aa00640 1 -- 172.21.15.86:0/2550215409 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31800054d0 con 0x7f3194077b60 2024-04-17T07:23:46.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.551+0000 7f318aa00640 1 -- 172.21.15.86:0/2550215409 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180010430 con 0x7f3194077b60 2024-04-17T07:23:46.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.555+0000 7f319b880740 1 Processor -- start 2024-04-17T07:23:46.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.556+0000 7f319b880740 1 -- start start 2024-04-17T07:23:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.557+0000 7f319b880740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.557+0000 7f319b880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82a9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.557+0000 7f319b880740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a82ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.557+0000 7f319b880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640a820b120 con 0x5640a82a4b80 2024-04-17T07:23:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.557+0000 7f319b880740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5640a81b1340 con 0x5640a82a4190 2024-04-17T07:23:46.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.557+0000 7f319b880740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5640a82070e0 con 0x5640a82a9cf0 2024-04-17T07:23:46.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.558+0000 7f3192800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.558+0000 7f3191e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.558+0000 7f3192800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:40168/0 (socket says 172.21.15.86:40168) 2024-04-17T07:23:46.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.558+0000 7f3192800640 1 -- 172.21.15.86:0/861378664 learned_addr learned my addr 172.21.15.86:0/861378664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.558+0000 7f3191e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82a9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49054/0 (socket says 172.21.15.86:49054) 2024-04-17T07:23:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.558+0000 7f318aa00640 1 -- 172.21.15.86:0/2550215409 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f31800046c0 con 0x7f3194077b60 2024-04-17T07:23:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.559+0000 7f3191e00640 1 -- 172.21.15.86:0/861378664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 msgr2=0x5640a82ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:46.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.559+0000 7f3191400640 1 --2- 172.21.15.86:0/861378664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a82ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.559+0000 7f3191e00640 1 --2- 172.21.15.86:0/861378664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a82ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.559+0000 7f3191e00640 1 -- 172.21.15.86:0/861378664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 msgr2=0x5640a82a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.559+0000 7f3191e00640 1 --2- 172.21.15.86:0/861378664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.559+0000 7f3191e00640 1 -- 172.21.15.86:0/861378664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a7fe77d0 con 0x5640a82a4b80 2024-04-17T07:23:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.560+0000 7f3192800640 1 --2- 172.21.15.86:0/861378664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:46.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.560+0000 7f3191400640 1 --2- 172.21.15.86:0/861378664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a82ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:23:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.560+0000 7f318aa00640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31780a3cb0 0x7f31780a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.560+0000 7f3191e00640 1 --2- 172.21.15.86:0/861378664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82a9680 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f3184037040 tx=0x7f3184005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf5322ded916cff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.560+0000 7f318aa00640 1 -- 172.21.15.86:0/2550215409 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f31800d0af0 con 0x7f3194077b60 2024-04-17T07:23:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.561+0000 7f3192800640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31780a3cb0 0x7f31780a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.561+0000 7f317f000640 1 -- 172.21.15.86:0/861378664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184039040 con 0x5640a82a4b80 2024-04-17T07:23:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.561+0000 7f317f000640 1 -- 172.21.15.86:0/861378664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3184043070 con 0x5640a82a4b80 2024-04-17T07:23:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.561+0000 7f317f000640 1 -- 172.21.15.86:0/861378664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31840053c0 con 0x5640a82a4b80 2024-04-17T07:23:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.563+0000 7f3192800640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31780a3cb0 0x7f31780a6170 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x5640a82a53e0 tx=0x7f318c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.563+0000 7f319b880740 1 -- 172.21.15.86:0/861378664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 msgr2=0x5640a82a9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.563+0000 7f319b880740 1 --2- 172.21.15.86:0/861378664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82a9680 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f3184037040 tx=0x7f3184005dc0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.564+0000 7f319b880740 1 -- 172.21.15.86:0/861378664 shutdown_connections 2024-04-17T07:23:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.564+0000 7f319b880740 1 --2- 172.21.15.86:0/861378664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a82ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.564+0000 7f319b880740 1 --2- 172.21.15.86:0/861378664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82a9680 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.564+0000 7f319b880740 1 --2- 172.21.15.86:0/861378664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.564+0000 7f319b880740 1 -- 172.21.15.86:0/861378664 >> 172.21.15.86:0/861378664 conn(0x5640a829f700 msgr2=0x5640a829fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:46.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.564+0000 7f319b880740 1 -- 172.21.15.86:0/861378664 shutdown_connections 2024-04-17T07:23:46.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.565+0000 7f319b880740 1 -- 172.21.15.86:0/861378664 wait complete. 2024-04-17T07:23:46.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.565+0000 7f319b880740 1 -- start start 2024-04-17T07:23:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.566+0000 7f319b880740 1 Processor -- start 2024-04-17T07:23:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.566+0000 7f319b880740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.566+0000 7f319b880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.566+0000 7f319b880740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a82b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.566+0000 7f319b880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640a82acc40 con 0x5640a82a4b80 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.566+0000 7f319b880740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5640a81c4710 con 0x5640a82a4190 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.566+0000 7f319b880740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5640a82acdc0 con 0x5640a82a9cf0 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.566+0000 7f319b880740 10 client.0 _ll_register_callbacks cb 0x5640a81bfaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.567+0000 7f3191e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.567+0000 7f3191400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.567+0000 7f3191e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:40180/0 (socket says 172.21.15.86:40180) 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.567+0000 7f3191e00640 1 -- 172.21.15.86:0/403817016 learned_addr learned my addr 172.21.15.86:0/403817016 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.567+0000 7f3192800640 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a82b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.567+0000 7f3191e00640 1 -- 172.21.15.86:0/403817016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 msgr2=0x5640a82b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.567+0000 7f3191e00640 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a82b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.567+0000 7f3191e00640 1 -- 172.21.15.86:0/403817016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 msgr2=0x5640a82aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.567+0000 7f3191e00640 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.567+0000 7f3191e00640 1 -- 172.21.15.86:0/403817016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a7f1d3f0 con 0x5640a82a4190 2024-04-17T07:23:46.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.568+0000 7f3191e00640 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ae6a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3184004650 tx=0x7f31840044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22934e755ff297b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:46.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.568+0000 7f3173400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:23:46.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.570+0000 7f3173400640 1 -- 172.21.15.86:0/403817016 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184039040 con 0x5640a82a4190 2024-04-17T07:23:46.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.570+0000 7f319b880740 1 -- 172.21.15.86:0/403817016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3184043070 con 0x5640a82a4190 2024-04-17T07:23:46.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.570+0000 7f3173400640 1 -- 172.21.15.86:0/403817016 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3184034c90 con 0x5640a82a4190 2024-04-17T07:23:46.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.571+0000 7f3173400640 1 -- 172.21.15.86:0/403817016 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3184039040 con 0x5640a82a4190 2024-04-17T07:23:46.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.571+0000 7f319b880740 10 client.36692 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:46.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.571+0000 7f319b880740 1 -- 172.21.15.86:0/403817016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640a7edd0f0 con 0x5640a82a4190 2024-04-17T07:23:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.571+0000 7f3173400640 1 -- 172.21.15.86:0/403817016 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3184043070 con 0x5640a82a4190 2024-04-17T07:23:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.571+0000 7f3173400640 1 -- 172.21.15.86:0/403817016 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f318402c8f0 con 0x5640a82a4190 2024-04-17T07:23:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.572+0000 7f319b880740 10 client.36692 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.572+0000 7f319b880740 10 client.36692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.572+0000 7f319b880740 20 client.36692 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.572+0000 7f319b880740 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640a82ba080 0x5640a82bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.572+0000 7f319b880740 4 client.36692 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.572+0000 7f319b880740 1 -- 172.21.15.86:0/403817016 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640a8218b40 con 0x5640a82ba080 2024-04-17T07:23:46.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.572+0000 7f3191400640 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640a82ba080 0x5640a82bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.573+0000 7f3191400640 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640a82ba080 0x5640a82bc470 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb805e0f8343526 server_cookie=5d87d7c40caf52d7 in_seq=0 out_seq=0 2024-04-17T07:23:46.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.573+0000 7f3173400640 10 client.36692 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:46.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.574+0000 7f3173400640 1 -- 172.21.15.86:0/403817016 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640a8218b40 con 0x5640a82ba080 2024-04-17T07:23:46.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.574+0000 7f3173400640 10 client.36692 handle_command_reply: tid=0 2024-04-17T07:23:46.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.574+0000 7f319b880740 1 client.36692 shutdown 2024-04-17T07:23:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.575+0000 7f319b880740 1 -- 172.21.15.86:0/403817016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 msgr2=0x5640a82ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.575+0000 7f319b880740 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ae6a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3184004650 tx=0x7f31840044d0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.575+0000 7f319b880740 1 -- 172.21.15.86:0/403817016 shutdown_connections 2024-04-17T07:23:46.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.575+0000 7f319b880740 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640a82ba080 0x5640a82bc470 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.575+0000 7f319b880740 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a82b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.575+0000 7f319b880740 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.575+0000 7f319b880740 1 --2- 172.21.15.86:0/403817016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ae6a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.575+0000 7f319b880740 1 -- 172.21.15.86:0/403817016 >> 172.21.15.86:0/403817016 conn(0x5640a829f700 msgr2=0x5640a829e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:46.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.575+0000 7f3173400640 0 client.36692 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:46.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.576+0000 7f319b880740 1 -- 172.21.15.86:0/403817016 shutdown_connections 2024-04-17T07:23:46.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.576+0000 7f319b880740 1 -- 172.21.15.86:0/403817016 wait complete. 2024-04-17T07:23:46.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.576+0000 7f319b880740 20 client.36692 trim_cache size 0 max 16384 2024-04-17T07:23:46.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.614+0000 7f319b880740 1 Processor -- start 2024-04-17T07:23:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.615+0000 7f319b880740 1 -- start start 2024-04-17T07:23:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.615+0000 7f319b880740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.615+0000 7f319b880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.615+0000 7f319b880740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a829e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.615+0000 7f319b880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31941b9950 con 0x5640a82a4b80 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.615+0000 7f319b880740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f31941bb490 con 0x5640a82a4190 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.615+0000 7f319b880740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31941ba020 con 0x5640a82a9cf0 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.615+0000 7f3191400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.615+0000 7f3191400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49072/0 (socket says 172.21.15.86:49072) 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.615+0000 7f3191400640 1 -- 172.21.15.86:0/2514399157 learned_addr learned my addr 172.21.15.86:0/2514399157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.616+0000 7f3191e00640 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.616+0000 7f3192800640 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a829e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.616+0000 7f3191400640 1 -- 172.21.15.86:0/2514399157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 msgr2=0x5640a829e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.616+0000 7f3191400640 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a829e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.616+0000 7f3191400640 1 -- 172.21.15.86:0/2514399157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 msgr2=0x5640a82ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.616+0000 7f3191400640 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.616+0000 7f3191400640 1 -- 172.21.15.86:0/2514399157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a82a6760 con 0x5640a82a4b80 2024-04-17T07:23:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.616+0000 7f3192800640 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a829e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.616+0000 7f3191e00640 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.617+0000 7f3191400640 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82b5be0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f318009c980 tx=0x7f31800ce0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359f90b5a72d6f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.617+0000 7f317e600640 1 -- 172.21.15.86:0/2514399157 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318009c9f0 con 0x5640a82a4b80 2024-04-17T07:23:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.617+0000 7f317e600640 1 -- 172.21.15.86:0/2514399157 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31800a0da0 con 0x5640a82a4b80 2024-04-17T07:23:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.617+0000 7f317e600640 1 -- 172.21.15.86:0/2514399157 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31800a51d0 con 0x5640a82a4b80 2024-04-17T07:23:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.618+0000 7f319b880740 1 -- 172.21.15.86:0/2514399157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 msgr2=0x5640a82b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.618+0000 7f319b880740 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82b5be0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f318009c980 tx=0x7f31800ce0e0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.619+0000 7f319b880740 1 -- 172.21.15.86:0/2514399157 shutdown_connections 2024-04-17T07:23:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.619+0000 7f319b880740 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82a9cf0 0x5640a829e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.619+0000 7f319b880740 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4b80 0x5640a82b5be0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.619+0000 7f319b880740 1 --2- 172.21.15.86:0/2514399157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a4190 0x5640a82ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.619+0000 7f319b880740 1 -- 172.21.15.86:0/2514399157 >> 172.21.15.86:0/2514399157 conn(0x5640a829f700 msgr2=0x5640a829e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.619+0000 7f319b880740 1 -- 172.21.15.86:0/2514399157 shutdown_connections 2024-04-17T07:23:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.619+0000 7f319b880740 1 -- 172.21.15.86:0/2514399157 wait complete. 2024-04-17T07:23:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.620+0000 7f319b880740 1 -- start start 2024-04-17T07:23:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.620+0000 7f319b880740 1 Processor -- start 2024-04-17T07:23:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f319b880740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4190 0x5640a82b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f319b880740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a9cf0 0x5640a82b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f319b880740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82b4860 0x5640a82b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f319b880740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31941badc0 con 0x5640a82a4190 2024-04-17T07:23:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f319b880740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f31941ba6f0 con 0x5640a82a9cf0 2024-04-17T07:23:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f319b880740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31941b7920 con 0x5640a82b4860 2024-04-17T07:23:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f319b880740 10 client.0 _ll_register_callbacks cb 0x7f318c009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f3192800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82b4860 0x5640a82b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f3191400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a9cf0 0x5640a82b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f3192800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82b4860 0x5640a82b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47502/0 (socket says 172.21.15.86:47502) 2024-04-17T07:23:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.621+0000 7f3192800640 1 -- 172.21.15.86:0/1804415388 learned_addr learned my addr 172.21.15.86:0/1804415388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.622+0000 7f3191e00640 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4190 0x5640a82b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.623+0000 7f3191e00640 1 -- 172.21.15.86:0/1804415388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82b4860 msgr2=0x5640a82b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.623+0000 7f3191e00640 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82b4860 0x5640a82b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.623+0000 7f3191e00640 1 -- 172.21.15.86:0/1804415388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a9cf0 msgr2=0x5640a82b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.623+0000 7f3191e00640 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a9cf0 0x5640a82b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.623+0000 7f3191e00640 1 -- 172.21.15.86:0/1804415388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a82b5220 con 0x5640a82a4190 2024-04-17T07:23:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.623+0000 7f3192800640 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82b4860 0x5640a82b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.623+0000 7f3191e00640 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4190 0x5640a82b3310 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3184033840 tx=0x7f3184005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d2027a53fac069 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.623+0000 7f3173400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:23:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.624+0000 7f3173400640 1 -- 172.21.15.86:0/1804415388 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318402c710 con 0x5640a82a4190 2024-04-17T07:23:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.624+0000 7f3173400640 1 -- 172.21.15.86:0/1804415388 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3184033c30 con 0x5640a82a4190 2024-04-17T07:23:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.624+0000 7f319b880740 1 -- 172.21.15.86:0/1804415388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31800a0da0 con 0x5640a82a4190 2024-04-17T07:23:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.624+0000 7f3173400640 1 -- 172.21.15.86:0/1804415388 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3184046e70 con 0x5640a82a4190 2024-04-17T07:23:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.624+0000 7f319b880740 10 client.17781 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.624+0000 7f319b880740 1 -- 172.21.15.86:0/1804415388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640a8331810 con 0x5640a82a4190 2024-04-17T07:23:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.625+0000 7f3173400640 1 -- 172.21.15.86:0/1804415388 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31800a0da0 con 0x5640a82a4190 2024-04-17T07:23:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.625+0000 7f3173400640 1 -- 172.21.15.86:0/1804415388 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f318402c8f0 con 0x5640a82a4190 2024-04-17T07:23:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.625+0000 7f319b880740 10 client.17781 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.625+0000 7f319b880740 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.625+0000 7f319b880740 20 client.17781 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.626+0000 7f319b880740 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640a8332190 0x5640a83345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.626+0000 7f319b880740 4 client.17781 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:23:46.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.626+0000 7f319b880740 1 -- 172.21.15.86:0/1804415388 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640a82129b0 con 0x5640a8332190 2024-04-17T07:23:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.626+0000 7f3191400640 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640a8332190 0x5640a83345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.627+0000 7f3191400640 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640a8332190 0x5640a83345b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be93e5b0c564e3d server_cookie=c2b0dd954886a443 in_seq=0 out_seq=0 2024-04-17T07:23:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.627+0000 7f3173400640 10 client.17781 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.628+0000 7f3173400640 1 -- 172.21.15.86:0/1804415388 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5640a82129b0 con 0x5640a8332190 2024-04-17T07:23:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.628+0000 7f3173400640 10 client.17781 handle_command_reply: tid=0 2024-04-17T07:23:46.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.628+0000 7f319b880740 1 client.17781 shutdown 2024-04-17T07:23:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.629+0000 7f319b880740 1 -- 172.21.15.86:0/1804415388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4190 msgr2=0x5640a82b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.629+0000 7f319b880740 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4190 0x5640a82b3310 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3184033840 tx=0x7f3184005dc0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.629+0000 7f319b880740 1 -- 172.21.15.86:0/1804415388 shutdown_connections 2024-04-17T07:23:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.629+0000 7f319b880740 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640a8332190 0x5640a83345b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.630+0000 7f3173400640 0 client.17781 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.630+0000 7f319b880740 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a82b4860 0x5640a82b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.630+0000 7f319b880740 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a82a9cf0 0x5640a82b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.630+0000 7f319b880740 1 --2- 172.21.15.86:0/1804415388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a82a4190 0x5640a82b3310 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.630+0000 7f319b880740 1 -- 172.21.15.86:0/1804415388 >> 172.21.15.86:0/1804415388 conn(0x5640a829f700 msgr2=0x5640a829e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.630+0000 7f319b880740 1 -- 172.21.15.86:0/1804415388 shutdown_connections 2024-04-17T07:23:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.630+0000 7f319b880740 1 -- 172.21.15.86:0/1804415388 wait complete. 2024-04-17T07:23:46.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.631+0000 7f319b880740 20 client.17781 trim_cache size 0 max 16384 2024-04-17T07:23:46.631 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:23:46.631 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:23:46.632 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:23:46.632 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:23:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.632+0000 7f3198c00640 1 -- 172.21.15.86:0/2550215409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31780a3cb0 msgr2=0x7f31780a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.632+0000 7f3198c00640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31780a3cb0 0x7f31780a6170 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x5640a82a53e0 tx=0x7f318c009500 comp rx=0 tx=0).stop 2024-04-17T07:23:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.632+0000 7f3198c00640 1 -- 172.21.15.86:0/2550215409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3194077b60 msgr2=0x7f3194070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.632+0000 7f3198c00640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3194077b60 0x7f3194070f50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f318000e9c0 tx=0x7f318000ee80 comp rx=0 tx=0).stop 2024-04-17T07:23:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.633+0000 7f3198c00640 1 -- 172.21.15.86:0/2550215409 shutdown_connections 2024-04-17T07:23:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.633+0000 7f3198c00640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f31780a3cb0 0x7f31780a6170 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.633+0000 7f3198c00640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3194078500 0x7f31941ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.633+0000 7f3198c00640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3194077b60 0x7f3194070f50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.633+0000 7f3198c00640 1 --2- 172.21.15.86:0/2550215409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3194076920 0x7f3194070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.633+0000 7f3198c00640 1 -- 172.21.15.86:0/2550215409 >> 172.21.15.86:0/2550215409 conn(0x7f31940ff750 msgr2=0x7f3194102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.633+0000 7f3198c00640 1 -- 172.21.15.86:0/2550215409 shutdown_connections 2024-04-17T07:23:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.633+0000 7f3198c00640 1 -- 172.21.15.86:0/2550215409 wait complete. 2024-04-17T07:23:46.658 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:23:46.659 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:23:46.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.955+0000 7f94a3c00640 1 -- 172.21.15.86:0/1134050719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f949c100680 msgr2=0x7f949c107680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.955+0000 7f94a3c00640 1 --2- 172.21.15.86:0/1134050719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f949c100680 0x7f949c107680 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9488009900 tx=0x7f948802bc30 comp rx=0 tx=0).stop 2024-04-17T07:23:46.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.956+0000 7f94a3c00640 1 -- 172.21.15.86:0/1134050719 shutdown_connections 2024-04-17T07:23:46.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.956+0000 7f94a3c00640 1 --2- 172.21.15.86:0/1134050719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f949c107c10 0x7f949c10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.956+0000 7f94a3c00640 1 --2- 172.21.15.86:0/1134050719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f949c100680 0x7f949c107680 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.956+0000 7f94a3c00640 1 --2- 172.21.15.86:0/1134050719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f949c0ffd20 0x7f949c100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.956+0000 7f94a3c00640 1 -- 172.21.15.86:0/1134050719 >> 172.21.15.86:0/1134050719 conn(0x7f949c0fb750 msgr2=0x7f949c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:46.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.956+0000 7f94a3c00640 1 -- 172.21.15.86:0/1134050719 shutdown_connections 2024-04-17T07:23:46.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.957+0000 7f94a3c00640 1 -- 172.21.15.86:0/1134050719 wait complete. 2024-04-17T07:23:46.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.958+0000 7f94a3c00640 1 Processor -- start 2024-04-17T07:23:46.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.959+0000 7f94a3c00640 1 -- start start 2024-04-17T07:23:46.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.959+0000 7f94a3c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f949c0ffd20 0x7f949c102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.959+0000 7f94a3c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f949c100680 0x7f949c101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.959+0000 7f94a3c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f949c107c10 0x7f949c101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.959+0000 7f94a3c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f949c10db30 con 0x7f949c100680 2024-04-17T07:23:46.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.959+0000 7f94a3c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f949c10d9b0 con 0x7f949c107c10 2024-04-17T07:23:46.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.959+0000 7f94a3c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f949c10dcb0 con 0x7f949c0ffd20 2024-04-17T07:23:46.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.960+0000 7f949be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f949c100680 0x7f949c101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.960+0000 7f949be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f949c100680 0x7f949c101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33674/0 (socket says 172.21.15.86:33674) 2024-04-17T07:23:46.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.960+0000 7f949be00640 1 -- 172.21.15.86:0/1248599097 learned_addr learned my addr 172.21.15.86:0/1248599097 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:46.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.960+0000 7f94a1c00640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f949c107c10 0x7f949c101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.960+0000 7f94a1200640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f949c0ffd20 0x7f949c102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.961+0000 7f949be00640 1 -- 172.21.15.86:0/1248599097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f949c0ffd20 msgr2=0x7f949c102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.961+0000 7f949be00640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f949c0ffd20 0x7f949c102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.961+0000 7f949be00640 1 -- 172.21.15.86:0/1248599097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f949c107c10 msgr2=0x7f949c101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:46.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.961+0000 7f949be00640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f949c107c10 0x7f949c101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:46.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.961+0000 7f949be00640 1 -- 172.21.15.86:0/1248599097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949c1023e0 con 0x7f949c100680 2024-04-17T07:23:46.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.961+0000 7f949be00640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f949c100680 0x7f949c101610 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9488000c00 tx=0x7f9488004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:46.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.962+0000 7f9499600640 1 -- 172.21.15.86:0/1248599097 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488043020 con 0x7f949c100680 2024-04-17T07:23:46.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.962+0000 7f9499600640 1 -- 172.21.15.86:0/1248599097 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94880052b0 con 0x7f949c100680 2024-04-17T07:23:46.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.963+0000 7f9499600640 1 -- 172.21.15.86:0/1248599097 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488003c70 con 0x7f949c100680 2024-04-17T07:23:46.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.963+0000 7f94a3c00640 1 -- 172.21.15.86:0/1248599097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949c197e70 con 0x7f949c100680 2024-04-17T07:23:46.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.963+0000 7f94a3c00640 1 -- 172.21.15.86:0/1248599097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949c198370 con 0x7f949c100680 2024-04-17T07:23:46.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.965+0000 7f94a3c00640 1 -- 172.21.15.86:0/1248599097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f949c106490 con 0x7f949c100680 2024-04-17T07:23:46.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.971+0000 7f9499600640 1 -- 172.21.15.86:0/1248599097 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9488046020 con 0x7f949c100680 2024-04-17T07:23:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.978+0000 7f9499600640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94800a3c80 0x7f94800a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.978+0000 7f9499600640 1 -- 172.21.15.86:0/1248599097 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f94880f2e80 con 0x7f949c100680 2024-04-17T07:23:46.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.980+0000 7f94a1200640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94800a3c80 0x7f94800a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:46.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.980+0000 7f9499600640 1 -- 172.21.15.86:0/1248599097 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f948804d020 con 0x7f949c100680 2024-04-17T07:23:46.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:46.984+0000 7f94a1200640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94800a3c80 0x7f94800a6140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f948c009bd0 tx=0x7f948c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:47.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.357+0000 7f94a3c00640 1 -- 172.21.15.86:0/1248599097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f949c02d5d0 con 0x7f949c100680 2024-04-17T07:23:47.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.359+0000 7f9499600640 1 -- 172.21.15.86:0/1248599097 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f94880bf4e0 con 0x7f949c100680 2024-04-17T07:23:47.359 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:23:47.359 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:23:47.360 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:23:47.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 -- 172.21.15.86:0/1248599097 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94800a3c80 msgr2=0x7f94800a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94800a3c80 0x7f94800a6140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f948c009bd0 tx=0x7f948c005f90 comp rx=0 tx=0).stop 2024-04-17T07:23:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 -- 172.21.15.86:0/1248599097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f949c100680 msgr2=0x7f949c101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f949c100680 0x7f949c101610 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9488000c00 tx=0x7f9488004270 comp rx=0 tx=0).stop 2024-04-17T07:23:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 -- 172.21.15.86:0/1248599097 shutdown_connections 2024-04-17T07:23:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f94800a3c80 0x7f94800a6140 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f949c107c10 0x7f949c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f949c100680 0x7f949c101610 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 --2- 172.21.15.86:0/1248599097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f949c0ffd20 0x7f949c102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 -- 172.21.15.86:0/1248599097 >> 172.21.15.86:0/1248599097 conn(0x7f949c0fb750 msgr2=0x7f949c103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 -- 172.21.15.86:0/1248599097 shutdown_connections 2024-04-17T07:23:47.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.361+0000 7f94a3c00640 1 -- 172.21.15.86:0/1248599097 wait complete. 2024-04-17T07:23:47.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:47 smithi086 ceph-mon[31733]: pgmap v722: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 659 op/s 2024-04-17T07:23:47.378 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:23:47.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:47 smithi098 ceph-mon[35418]: pgmap v722: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 659 op/s 2024-04-17T07:23:47.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:47 smithi103 ceph-mon[35495]: pgmap v722: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 659 op/s 2024-04-17T07:23:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.674+0000 7f0663200640 1 -- 172.21.15.86:0/3830157917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640749c0 msgr2=0x7f0664074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.674+0000 7f0663200640 1 --2- 172.21.15.86:0/3830157917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640749c0 0x7f0664074e40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0650009930 tx=0x7f065002bc20 comp rx=0 tx=0).stop 2024-04-17T07:23:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.674+0000 7f0663200640 1 -- 172.21.15.86:0/3830157917 shutdown_connections 2024-04-17T07:23:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.675+0000 7f0663200640 1 --2- 172.21.15.86:0/3830157917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f066410e440 0x7f0664110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.675+0000 7f0663200640 1 --2- 172.21.15.86:0/3830157917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640749c0 0x7f0664074e40 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.675+0000 7f0663200640 1 --2- 172.21.15.86:0/3830157917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0664075fa0 0x7f06640743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.675+0000 7f0663200640 1 -- 172.21.15.86:0/3830157917 >> 172.21.15.86:0/3830157917 conn(0x7f06640fd400 msgr2=0x7f06640ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:47.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.675+0000 7f0663200640 1 -- 172.21.15.86:0/3830157917 shutdown_connections 2024-04-17T07:23:47.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.676+0000 7f0663200640 1 -- 172.21.15.86:0/3830157917 wait complete. 2024-04-17T07:23:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.677+0000 7f0663200640 1 Processor -- start 2024-04-17T07:23:47.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.677+0000 7f0663200640 1 -- start start 2024-04-17T07:23:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.678+0000 7f0663200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640749c0 0x7f066410ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.678+0000 7f0663200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0664075fa0 0x7f066410b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.678+0000 7f0663200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f066410e440 0x7f06641ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.678+0000 7f0663200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0664113fc0 con 0x7f06640749c0 2024-04-17T07:23:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.678+0000 7f0663200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0664113e40 con 0x7f066410e440 2024-04-17T07:23:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.678+0000 7f0661400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0664075fa0 0x7f066410b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.678+0000 7f0661400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0664075fa0 0x7f066410b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43668/0 (socket says 172.21.15.86:43668) 2024-04-17T07:23:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.678+0000 7f0661400640 1 -- 172.21.15.86:0/1906535472 learned_addr learned my addr 172.21.15.86:0/1906535472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.678+0000 7f0662800640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f066410e440 0x7f06641ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.679+0000 7f0663200640 1 -- 172.21.15.86:0/1906535472 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0664114140 con 0x7f0664075fa0 2024-04-17T07:23:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.679+0000 7f0661e00640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640749c0 0x7f066410ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.680+0000 7f0661400640 1 -- 172.21.15.86:0/1906535472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f066410e440 msgr2=0x7f06641ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.680+0000 7f0661400640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f066410e440 0x7f06641ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.680+0000 7f0661400640 1 -- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640749c0 msgr2=0x7f066410ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.680+0000 7f0661400640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640749c0 0x7f066410ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.680+0000 7f0661400640 1 -- 172.21.15.86:0/1906535472 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06641aec10 con 0x7f0664075fa0 2024-04-17T07:23:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.680+0000 7f0661e00640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640749c0 0x7f066410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:47.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.680+0000 7f0661400640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0664075fa0 0x7f066410b150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f065002c830 tx=0x7f0650004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.681+0000 7f065aa00640 1 -- 172.21.15.86:0/1906535472 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650043020 con 0x7f0664075fa0 2024-04-17T07:23:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.681+0000 7f065aa00640 1 -- 172.21.15.86:0/1906535472 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06500052b0 con 0x7f0664075fa0 2024-04-17T07:23:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.681+0000 7f065aa00640 1 -- 172.21.15.86:0/1906535472 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650003d90 con 0x7f0664075fa0 2024-04-17T07:23:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.681+0000 7f0663200640 1 -- 172.21.15.86:0/1906535472 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06641aede0 con 0x7f0664075fa0 2024-04-17T07:23:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.681+0000 7f0663200640 1 -- 172.21.15.86:0/1906535472 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06641af1f0 con 0x7f0664075fa0 2024-04-17T07:23:47.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.687+0000 7f066bb8b740 1 Processor -- start 2024-04-17T07:23:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.687+0000 7f066bb8b740 1 -- start start 2024-04-17T07:23:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.687+0000 7f065aa00640 1 -- 172.21.15.86:0/1906535472 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0650046020 con 0x7f0664075fa0 2024-04-17T07:23:47.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.687+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996160 0x562c2b996580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.687+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996b50 0x562c2b99b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.687+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b99bcc0 0x562c2b99e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.688+0000 7f066bb8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562c2b8fc020 con 0x562c2b996b50 2024-04-17T07:23:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.688+0000 7f066bb8b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562c2b8a2460 con 0x562c2b996160 2024-04-17T07:23:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.688+0000 7f066bb8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562c2b8f7fe0 con 0x562c2b99bcc0 2024-04-17T07:23:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.688+0000 7f0661400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b99bcc0 0x562c2b99e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.688+0000 7f0661400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b99bcc0 0x562c2b99e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43672/0 (socket says 172.21.15.86:43672) 2024-04-17T07:23:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.688+0000 7f0661400640 1 -- 172.21.15.86:0/3912978019 learned_addr learned my addr 172.21.15.86:0/3912978019 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.689+0000 7f0661400640 1 -- 172.21.15.86:0/3912978019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996160 msgr2=0x562c2b996580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:23:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.689+0000 7f0661400640 1 --2- 172.21.15.86:0/3912978019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996160 0x562c2b996580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.689+0000 7f0661400640 1 -- 172.21.15.86:0/3912978019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996b50 msgr2=0x562c2b99b650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:23:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.689+0000 7f0661400640 1 --2- 172.21.15.86:0/3912978019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996b50 0x562c2b99b650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.689+0000 7f0661400640 1 -- 172.21.15.86:0/3912978019 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2b8aa040 con 0x562c2b99bcc0 2024-04-17T07:23:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.689+0000 7f0661400640 1 --2- 172.21.15.86:0/3912978019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b99bcc0 0x562c2b99e150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f06500f8a90 tx=0x7f06500f8b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e59231622ba71da server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:47.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.690+0000 7f065aa00640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06480a3db0 0x7f06480a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.690+0000 7f064f400640 1 -- 172.21.15.86:0/3912978019 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650101070 con 0x562c2b99bcc0 2024-04-17T07:23:47.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.690+0000 7f064f400640 1 -- 172.21.15.86:0/3912978019 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06500bee60 con 0x562c2b99bcc0 2024-04-17T07:23:47.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.690+0000 7f064f400640 1 -- 172.21.15.86:0/3912978019 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500be760 con 0x562c2b99bcc0 2024-04-17T07:23:47.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.691+0000 7f0662800640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06480a3db0 0x7f06480a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.691+0000 7f065aa00640 1 -- 172.21.15.86:0/1906535472 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f06500f2860 con 0x7f0664075fa0 2024-04-17T07:23:47.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.691+0000 7f066bb8b740 1 -- 172.21.15.86:0/3912978019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b99bcc0 msgr2=0x562c2b99e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.691+0000 7f066bb8b740 1 --2- 172.21.15.86:0/3912978019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b99bcc0 0x562c2b99e150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f06500f8a90 tx=0x7f06500f8b80 comp rx=0 tx=0).stop 2024-04-17T07:23:47.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.693+0000 7f0662800640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06480a3db0 0x7f06480a6270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x562c2b9973b0 tx=0x7f065c007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:47.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.693+0000 7f066bb8b740 1 -- 172.21.15.86:0/3912978019 shutdown_connections 2024-04-17T07:23:47.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.693+0000 7f066bb8b740 1 --2- 172.21.15.86:0/3912978019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b99bcc0 0x562c2b99e150 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.693+0000 7f066bb8b740 1 --2- 172.21.15.86:0/3912978019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996b50 0x562c2b99b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.693+0000 7f066bb8b740 1 --2- 172.21.15.86:0/3912978019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996160 0x562c2b996580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.693+0000 7f066bb8b740 1 -- 172.21.15.86:0/3912978019 >> 172.21.15.86:0/3912978019 conn(0x562c2b9916d0 msgr2=0x562c2b991ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:47.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.693+0000 7f066bb8b740 1 -- 172.21.15.86:0/3912978019 shutdown_connections 2024-04-17T07:23:47.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.693+0000 7f066bb8b740 1 -- 172.21.15.86:0/3912978019 wait complete. 2024-04-17T07:23:47.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.693+0000 7f066bb8b740 1 -- start start 2024-04-17T07:23:47.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.694+0000 7f066bb8b740 1 Processor -- start 2024-04-17T07:23:47.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.694+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996160 0x562c2b9a0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 0x562c2b9a0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 0x562c2b9a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f066bb8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562c2b99ea30 con 0x562c2b99bcc0 2024-04-17T07:23:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f066bb8b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562c2b8b5830 con 0x562c2b996160 2024-04-17T07:23:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f066bb8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562c2b99ebb0 con 0x562c2b996b50 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f0661400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 0x562c2b9a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f0661400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 0x562c2b9a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43674/0 (socket says 172.21.15.86:43674) 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f0661400640 1 -- 172.21.15.86:0/1120475120 learned_addr learned my addr 172.21.15.86:0/1120475120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f0661e00640 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996160 0x562c2b9a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f0662800640 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 0x562c2b9a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.695+0000 7f066bb8b740 10 client.0 _ll_register_callbacks cb 0x562c2b883fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.696+0000 7f0661400640 1 -- 172.21.15.86:0/1120475120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996160 msgr2=0x562c2b9a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.696+0000 7f0661400640 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996160 0x562c2b9a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.696+0000 7f0661400640 1 -- 172.21.15.86:0/1120475120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 msgr2=0x562c2b9a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.696+0000 7f0661400640 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 0x562c2b9a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.696+0000 7f0661400640 1 -- 172.21.15.86:0/1120475120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2b8b9920 con 0x562c2b996b50 2024-04-17T07:23:47.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.696+0000 7f0662800640 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 0x562c2b9a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.697+0000 7f0661400640 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 0x562c2b9a0950 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0650000c00 tx=0x7f065002bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c6f958111c6511 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.698+0000 7f0643e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.698+0000 7f0643e00640 1 -- 172.21.15.86:0/1120475120 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650101050 con 0x562c2b996b50 2024-04-17T07:23:47.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.698+0000 7f0643e00640 1 -- 172.21.15.86:0/1120475120 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f06500be9c0 con 0x562c2b996b50 2024-04-17T07:23:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.699+0000 7f066bb8b740 1 -- 172.21.15.86:0/1120475120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06500bee60 con 0x562c2b996b50 2024-04-17T07:23:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.699+0000 7f0643e00640 1 -- 172.21.15.86:0/1120475120 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06500be530 con 0x562c2b996b50 2024-04-17T07:23:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.699+0000 7f066bb8b740 10 client.26755 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:47.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.699+0000 7f066bb8b740 1 -- 172.21.15.86:0/1120475120 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c2b8877f0 con 0x562c2b996b50 2024-04-17T07:23:47.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.700+0000 7f0643e00640 1 -- 172.21.15.86:0/1120475120 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06500bee60 con 0x562c2b996b50 2024-04-17T07:23:47.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.700+0000 7f0643e00640 1 -- 172.21.15.86:0/1120475120 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f06500c6f40 con 0x562c2b996b50 2024-04-17T07:23:47.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.700+0000 7f066bb8b740 10 client.26755 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:47.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.700+0000 7f066bb8b740 10 client.26755 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:23:47.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.701+0000 7f066bb8b740 20 client.26755 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:47.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.701+0000 7f066bb8b740 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x562c2b9abc10 0x562c2b9ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.701+0000 7f066bb8b740 4 client.26755 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:47.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.701+0000 7f066bb8b740 1 -- 172.21.15.86:0/1120475120 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c2b909a40 con 0x562c2b9abc10 2024-04-17T07:23:47.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.701+0000 7f0661e00640 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x562c2b9abc10 0x562c2b9ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.703+0000 7f0661e00640 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x562c2b9abc10 0x562c2b9ae030 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=85a2c68beca95d17 server_cookie=762ffc08b7d93714 in_seq=0 out_seq=0 2024-04-17T07:23:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.704+0000 7f0643e00640 10 client.26755 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:23:47.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.704+0000 7f0643e00640 1 -- 172.21.15.86:0/1120475120 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c2b909a40 con 0x562c2b9abc10 2024-04-17T07:23:47.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.705+0000 7f0643e00640 10 client.26755 handle_command_reply: tid=0 2024-04-17T07:23:47.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.705+0000 7f066bb8b740 1 client.26755 shutdown 2024-04-17T07:23:47.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.706+0000 7f066bb8b740 1 -- 172.21.15.86:0/1120475120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 msgr2=0x562c2b9a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.706+0000 7f066bb8b740 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 0x562c2b9a0950 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0650000c00 tx=0x7f065002bc20 comp rx=0 tx=0).stop 2024-04-17T07:23:47.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.706+0000 7f066bb8b740 1 -- 172.21.15.86:0/1120475120 shutdown_connections 2024-04-17T07:23:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.706+0000 7f066bb8b740 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x562c2b9abc10 0x562c2b9ae030 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.707+0000 7f0643e00640 0 client.26755 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:23:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.707+0000 7f066bb8b740 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 0x562c2b9a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.707+0000 7f066bb8b740 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 0x562c2b9a0950 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.707+0000 7f066bb8b740 1 --2- 172.21.15.86:0/1120475120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996160 0x562c2b9a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.708+0000 7f066bb8b740 1 -- 172.21.15.86:0/1120475120 >> 172.21.15.86:0/1120475120 conn(0x562c2b9916d0 msgr2=0x562c2b990930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:47.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.708+0000 7f066bb8b740 1 -- 172.21.15.86:0/1120475120 shutdown_connections 2024-04-17T07:23:47.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.708+0000 7f066bb8b740 1 -- 172.21.15.86:0/1120475120 wait complete. 2024-04-17T07:23:47.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.709+0000 7f066bb8b740 20 client.26755 trim_cache size 0 max 16384 2024-04-17T07:23:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.744+0000 7f066bb8b740 1 Processor -- start 2024-04-17T07:23:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.745+0000 7f066bb8b740 1 -- start start 2024-04-17T07:23:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.745+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996160 0x562c2b9900a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.745+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 0x562c2b9a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.745+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b99bcc0 0x562c2b9905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.745+0000 7f066bb8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06641b95f0 con 0x562c2b996160 2024-04-17T07:23:47.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.745+0000 7f066bb8b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f06641bb130 con 0x562c2b99bcc0 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.745+0000 7f066bb8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06641b9cc0 con 0x562c2b996b50 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.746+0000 7f0661e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996160 0x562c2b9900a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.746+0000 7f0661e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996160 0x562c2b9900a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33696/0 (socket says 172.21.15.86:33696) 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.746+0000 7f0661e00640 1 -- 172.21.15.86:0/1080324504 learned_addr learned my addr 172.21.15.86:0/1080324504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.746+0000 7f0661400640 1 --2- 172.21.15.86:0/1080324504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 0x562c2b9a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.746+0000 7f0662800640 1 --2- 172.21.15.86:0/1080324504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b99bcc0 0x562c2b9905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.746+0000 7f0661e00640 1 -- 172.21.15.86:0/1080324504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 msgr2=0x562c2b9a78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.746+0000 7f0661e00640 1 --2- 172.21.15.86:0/1080324504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 0x562c2b9a78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.747+0000 7f0661e00640 1 -- 172.21.15.86:0/1080324504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b99bcc0 msgr2=0x562c2b9905e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.747+0000 7f0661e00640 1 --2- 172.21.15.86:0/1080324504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b99bcc0 0x562c2b9905e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.747+0000 7f0661e00640 1 -- 172.21.15.86:0/1080324504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2b60e3f0 con 0x562c2b996160 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.747+0000 7f0661e00640 1 --2- 172.21.15.86:0/1080324504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996160 0x562c2b9900a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x562c2b9975c0 tx=0x7f065400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e920aec51cff7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:47.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.748+0000 7f064ea00640 1 -- 172.21.15.86:0/1080324504 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654004220 con 0x562c2b996160 2024-04-17T07:23:47.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.748+0000 7f064ea00640 1 -- 172.21.15.86:0/1080324504 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06540043c0 con 0x562c2b996160 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.748+0000 7f064ea00640 1 -- 172.21.15.86:0/1080324504 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06540047a0 con 0x562c2b996160 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.749+0000 7f066bb8b740 1 -- 172.21.15.86:0/1080324504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996160 msgr2=0x562c2b9900a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.749+0000 7f066bb8b740 1 --2- 172.21.15.86:0/1080324504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996160 0x562c2b9900a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x562c2b9975c0 tx=0x7f065400b390 comp rx=0 tx=0).stop 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.750+0000 7f066bb8b740 1 -- 172.21.15.86:0/1080324504 shutdown_connections 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.750+0000 7f066bb8b740 1 --2- 172.21.15.86:0/1080324504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b99bcc0 0x562c2b9905e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.750+0000 7f066bb8b740 1 --2- 172.21.15.86:0/1080324504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996b50 0x562c2b9a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.750+0000 7f066bb8b740 1 --2- 172.21.15.86:0/1080324504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b996160 0x562c2b9900a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.750+0000 7f066bb8b740 1 -- 172.21.15.86:0/1080324504 >> 172.21.15.86:0/1080324504 conn(0x562c2b9916d0 msgr2=0x562c2b998050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.750+0000 7f066bb8b740 1 -- 172.21.15.86:0/1080324504 shutdown_connections 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.750+0000 7f066bb8b740 1 -- 172.21.15.86:0/1080324504 wait complete. 2024-04-17T07:23:47.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.751+0000 7f066bb8b740 1 -- start start 2024-04-17T07:23:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.751+0000 7f066bb8b740 1 Processor -- start 2024-04-17T07:23:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.751+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996160 0x562c2b9a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.751+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996b50 0x562c2b9a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.752+0000 7f066bb8b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 0x562c2b9a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.752+0000 7f066bb8b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06641baa60 con 0x562c2b99bcc0 2024-04-17T07:23:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.752+0000 7f066bb8b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f06641ba390 con 0x562c2b996b50 2024-04-17T07:23:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.752+0000 7f066bb8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06641b75c0 con 0x562c2b996160 2024-04-17T07:23:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.752+0000 7f066bb8b740 10 client.0 _ll_register_callbacks cb 0x562c2b883fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.752+0000 7f0662800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 0x562c2b9a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.752+0000 7f0661400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996b50 0x562c2b9a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.752+0000 7f0662800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 0x562c2b9a64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33700/0 (socket says 172.21.15.86:33700) 2024-04-17T07:23:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.753+0000 7f0661400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996b50 0x562c2b9a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52202/0 (socket says 172.21.15.86:52202) 2024-04-17T07:23:47.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.753+0000 7f0661400640 1 -- 172.21.15.86:0/2977437835 learned_addr learned my addr 172.21.15.86:0/2977437835 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.753+0000 7f0661e00640 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996160 0x562c2b9a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.753+0000 7f0661400640 1 -- 172.21.15.86:0/2977437835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996160 msgr2=0x562c2b9a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.753+0000 7f0661400640 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996160 0x562c2b9a4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.753+0000 7f0661400640 1 -- 172.21.15.86:0/2977437835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 msgr2=0x562c2b9a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.753+0000 7f0661400640 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 0x562c2b9a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.753+0000 7f0661400640 1 -- 172.21.15.86:0/2977437835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2b6d87d0 con 0x562c2b996b50 2024-04-17T07:23:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.754+0000 7f0661400640 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996b50 0x562c2b9a54b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f06500c7470 tx=0x7f06500bcc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2477cd33d8ca0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:47.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.754+0000 7f0643e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:23:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.754+0000 7f0643e00640 1 -- 172.21.15.86:0/2977437835 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500db070 con 0x562c2b996b50 2024-04-17T07:23:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.754+0000 7f0643e00640 1 -- 172.21.15.86:0/2977437835 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f06500be9c0 con 0x562c2b996b50 2024-04-17T07:23:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.754+0000 7f066bb8b740 1 -- 172.21.15.86:0/2977437835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06540043c0 con 0x562c2b996b50 2024-04-17T07:23:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.755+0000 7f0643e00640 1 -- 172.21.15.86:0/2977437835 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0650101040 con 0x562c2b996b50 2024-04-17T07:23:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.755+0000 7f066bb8b740 10 client.36716 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:47.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.755+0000 7f066bb8b740 1 -- 172.21.15.86:0/2977437835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c2b6d87d0 con 0x562c2b996b50 2024-04-17T07:23:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.756+0000 7f0643e00640 1 -- 172.21.15.86:0/2977437835 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06540043c0 con 0x562c2b996b50 2024-04-17T07:23:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.757+0000 7f0643e00640 1 -- 172.21.15.86:0/2977437835 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f06500c6f40 con 0x562c2b996b50 2024-04-17T07:23:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.757+0000 7f066bb8b740 10 client.36716 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.757+0000 7f066bb8b740 10 client.36716 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:23:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.757+0000 7f066bb8b740 20 client.36716 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.757+0000 7f066bb8b740 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x562c2b9abc10 0x562c2ba145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.757+0000 7f066bb8b740 4 client.36716 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:23:47.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.757+0000 7f066bb8b740 1 -- 172.21.15.86:0/2977437835 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c2b9038b0 con 0x562c2b9abc10 2024-04-17T07:23:47.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.758+0000 7f0661e00640 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x562c2b9abc10 0x562c2ba145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:47.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.760+0000 7f0661e00640 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x562c2b9abc10 0x562c2ba145a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9a0007c08826370 server_cookie=818a74f7cbe2c676 in_seq=0 out_seq=0 2024-04-17T07:23:47.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.760+0000 7f0643e00640 10 client.36716 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:23:47.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.762+0000 7f0643e00640 1 -- 172.21.15.86:0/2977437835 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c2b9038b0 con 0x562c2b9abc10 2024-04-17T07:23:47.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.762+0000 7f0643e00640 10 client.36716 handle_command_reply: tid=0 2024-04-17T07:23:47.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.762+0000 7f066bb8b740 1 client.36716 shutdown 2024-04-17T07:23:47.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.762+0000 7f066bb8b740 1 -- 172.21.15.86:0/2977437835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996b50 msgr2=0x562c2b9a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.763+0000 7f066bb8b740 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996b50 0x562c2b9a54b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f06500c7470 tx=0x7f06500bcc70 comp rx=0 tx=0).stop 2024-04-17T07:23:47.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.763+0000 7f066bb8b740 1 -- 172.21.15.86:0/2977437835 shutdown_connections 2024-04-17T07:23:47.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.763+0000 7f066bb8b740 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x562c2b9abc10 0x562c2ba145a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.763+0000 7f066bb8b740 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562c2b99bcc0 0x562c2b9a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.763+0000 7f066bb8b740 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2b996b50 0x562c2b9a54b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.763+0000 7f066bb8b740 1 --2- 172.21.15.86:0/2977437835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562c2b996160 0x562c2b9a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.763+0000 7f066bb8b740 1 -- 172.21.15.86:0/2977437835 >> 172.21.15.86:0/2977437835 conn(0x562c2b9916d0 msgr2=0x562c2b994930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:47.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.763+0000 7f0643e00640 0 client.36716 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:23:47.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.764+0000 7f066bb8b740 1 -- 172.21.15.86:0/2977437835 shutdown_connections 2024-04-17T07:23:47.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.764+0000 7f066bb8b740 1 -- 172.21.15.86:0/2977437835 wait complete. 2024-04-17T07:23:47.766 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:23:47.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.765+0000 7f066bb8b740 20 client.36716 trim_cache size 0 max 16384 2024-04-17T07:23:47.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.768+0000 7f0663200640 1 -- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06480a3db0 msgr2=0x7f06480a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.768+0000 7f0663200640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06480a3db0 0x7f06480a6270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x562c2b9973b0 tx=0x7f065c007520 comp rx=0 tx=0).stop 2024-04-17T07:23:47.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.768+0000 7f0663200640 1 -- 172.21.15.86:0/1906535472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0664075fa0 msgr2=0x7f066410b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:47.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.768+0000 7f0663200640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0664075fa0 0x7f066410b150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f065002c830 tx=0x7f0650004270 comp rx=0 tx=0).stop 2024-04-17T07:23:47.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.768+0000 7f0663200640 1 -- 172.21.15.86:0/1906535472 shutdown_connections 2024-04-17T07:23:47.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.768+0000 7f0663200640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06480a3db0 0x7f06480a6270 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.768+0000 7f0663200640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f066410e440 0x7f06641ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.768+0000 7f0663200640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0664075fa0 0x7f066410b150 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.768+0000 7f0663200640 1 --2- 172.21.15.86:0/1906535472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640749c0 0x7f066410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.768+0000 7f0663200640 1 -- 172.21.15.86:0/1906535472 >> 172.21.15.86:0/1906535472 conn(0x7f06640fd400 msgr2=0x7f06641017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.769+0000 7f0663200640 1 -- 172.21.15.86:0/1906535472 shutdown_connections 2024-04-17T07:23:47.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:47.769+0000 7f0663200640 1 -- 172.21.15.86:0/1906535472 wait complete. 2024-04-17T07:23:47.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:23:48.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.101+0000 7f3738a00640 1 -- 172.21.15.86:0/1433573912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3734100e60 msgr2=0x7f3734101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.101+0000 7f3738a00640 1 --2- 172.21.15.86:0/1433573912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3734100e60 0x7f3734101280 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3724009860 tx=0x7f372402bb70 comp rx=0 tx=0).stop 2024-04-17T07:23:48.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.101+0000 7f3738a00640 1 -- 172.21.15.86:0/1433573912 shutdown_connections 2024-04-17T07:23:48.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.101+0000 7f3738a00640 1 --2- 172.21.15.86:0/1433573912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3734102a00 0x7f3734109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.101+0000 7f3738a00640 1 --2- 172.21.15.86:0/1433573912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3734102040 0x7f37341024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.101+0000 7f3738a00640 1 --2- 172.21.15.86:0/1433573912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3734100e60 0x7f3734101280 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.101+0000 7f3738a00640 1 -- 172.21.15.86:0/1433573912 >> 172.21.15.86:0/1433573912 conn(0x7f37340fc420 msgr2=0x7f37340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.101+0000 7f3738a00640 1 -- 172.21.15.86:0/1433573912 shutdown_connections 2024-04-17T07:23:48.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.102+0000 7f3738a00640 1 -- 172.21.15.86:0/1433573912 wait complete. 2024-04-17T07:23:48.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.103+0000 7f3738a00640 1 Processor -- start 2024-04-17T07:23:48.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.103+0000 7f3738a00640 1 -- start start 2024-04-17T07:23:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.104+0000 7f3738a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3734102040 0x7f373419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.104+0000 7f3738a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3734102a00 0x7f373419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.104+0000 7f3738a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37341a4090 0x7f37341a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.104+0000 7f3738a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f373410b950 con 0x7f3734102040 2024-04-17T07:23:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.104+0000 7f3738a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f373410b7d0 con 0x7f37341a4090 2024-04-17T07:23:48.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.104+0000 7f3738a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f373410bad0 con 0x7f3734102a00 2024-04-17T07:23:48.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.104+0000 7f3732800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37341a4090 0x7f37341a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.104+0000 7f3732800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37341a4090 0x7f37341a6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52230/0 (socket says 172.21.15.86:52230) 2024-04-17T07:23:48.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.104+0000 7f3732800640 1 -- 172.21.15.86:0/2386071218 learned_addr learned my addr 172.21.15.86:0/2386071218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:48.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.105+0000 7f3731e00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3734102040 0x7f373419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.105+0000 7f3731400640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3734102a00 0x7f373419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.105+0000 7f3731e00640 1 -- 172.21.15.86:0/2386071218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3734102a00 msgr2=0x7f373419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.105+0000 7f3731e00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3734102a00 0x7f373419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.105+0000 7f3731e00640 1 -- 172.21.15.86:0/2386071218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37341a4090 msgr2=0x7f37341a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.105+0000 7f3731e00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37341a4090 0x7f37341a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.105+0000 7f3731e00640 1 -- 172.21.15.86:0/2386071218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37341a6af0 con 0x7f3734102040 2024-04-17T07:23:48.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.105+0000 7f3731400640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3734102a00 0x7f373419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:23:48.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.106+0000 7f3731e00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3734102040 0x7f373419eaf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f37341009f0 tx=0x7f3724004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.107+0000 7f372aa00640 1 -- 172.21.15.86:0/2386071218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724039040 con 0x7f3734102040 2024-04-17T07:23:48.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.107+0000 7f372aa00640 1 -- 172.21.15.86:0/2386071218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f372402c9c0 con 0x7f3734102040 2024-04-17T07:23:48.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.107+0000 7f372aa00640 1 -- 172.21.15.86:0/2386071218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724004920 con 0x7f3734102040 2024-04-17T07:23:48.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.107+0000 7f3738a00640 1 -- 172.21.15.86:0/2386071218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37341a6d80 con 0x7f3734102040 2024-04-17T07:23:48.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.107+0000 7f3738a00640 1 -- 172.21.15.86:0/2386071218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37341a7190 con 0x7f3734102040 2024-04-17T07:23:48.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.112+0000 7f372aa00640 1 -- 172.21.15.86:0/2386071218 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3724044060 con 0x7f3734102040 2024-04-17T07:23:48.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.115+0000 7f372aa00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37180a3dc0 0x7f37180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.115+0000 7f372aa00640 1 -- 172.21.15.86:0/2386071218 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f37240f0fa0 con 0x7f3734102040 2024-04-17T07:23:48.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.115+0000 7f373b803740 1 Processor -- start 2024-04-17T07:23:48.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.115+0000 7f3731400640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37180a3dc0 0x7f37180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.116+0000 7f373b803740 1 -- start start 2024-04-17T07:23:48.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.116+0000 7f373b803740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f772753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.116+0000 7f373b803740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 0x559f7727a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.116+0000 7f373b803740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f7727ab00 0x559f7727cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.116+0000 7f373b803740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f771db020 con 0x559f7727ab00 2024-04-17T07:23:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.116+0000 7f373b803740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559f77181460 con 0x559f77275990 2024-04-17T07:23:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.116+0000 7f373b803740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559f771d6fe0 con 0x559f77274fa0 2024-04-17T07:23:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.116+0000 7f3731400640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37180a3dc0 0x7f37180a6280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f37341a0000 tx=0x7f3720009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.117+0000 7f3731e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f772753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.117+0000 7f3731e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f772753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43718/0 (socket says 172.21.15.86:43718) 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.117+0000 7f3731e00640 1 -- 172.21.15.86:0/2798292737 learned_addr learned my addr 172.21.15.86:0/2798292737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.117+0000 7f3731400640 1 --2- 172.21.15.86:0/2798292737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 0x559f7727a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.117+0000 7f3732800640 1 --2- 172.21.15.86:0/2798292737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f7727ab00 0x559f7727cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.117+0000 7f3731e00640 1 -- 172.21.15.86:0/2798292737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 msgr2=0x559f7727a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.117+0000 7f3731e00640 1 --2- 172.21.15.86:0/2798292737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 0x559f7727a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.117+0000 7f3731e00640 1 -- 172.21.15.86:0/2798292737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f7727ab00 msgr2=0x559f7727cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.117+0000 7f3731e00640 1 --2- 172.21.15.86:0/2798292737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f7727ab00 0x559f7727cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.117+0000 7f3731e00640 1 -- 172.21.15.86:0/2798292737 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f77189040 con 0x559f77274fa0 2024-04-17T07:23:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.118+0000 7f3731e00640 1 --2- 172.21.15.86:0/2798292737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f772753c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3724037040 tx=0x7f372402bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2de3ced6d6ef069 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.118+0000 7f371f000640 1 -- 172.21.15.86:0/2798292737 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37240d5070 con 0x559f77274fa0 2024-04-17T07:23:48.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.118+0000 7f371f000640 1 -- 172.21.15.86:0/2798292737 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37240bd5a0 con 0x559f77274fa0 2024-04-17T07:23:48.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.118+0000 7f371f000640 1 -- 172.21.15.86:0/2798292737 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37240ee7d0 con 0x559f77274fa0 2024-04-17T07:23:48.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.119+0000 7f373b803740 1 -- 172.21.15.86:0/2798292737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 msgr2=0x559f772753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.119+0000 7f373b803740 1 --2- 172.21.15.86:0/2798292737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f772753c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3724037040 tx=0x7f372402bb70 comp rx=0 tx=0).stop 2024-04-17T07:23:48.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.119+0000 7f373b803740 1 -- 172.21.15.86:0/2798292737 shutdown_connections 2024-04-17T07:23:48.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.119+0000 7f373b803740 1 --2- 172.21.15.86:0/2798292737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f7727ab00 0x559f7727cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.119+0000 7f373b803740 1 --2- 172.21.15.86:0/2798292737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 0x559f7727a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.119+0000 7f373b803740 1 --2- 172.21.15.86:0/2798292737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f772753c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.119+0000 7f373b803740 1 -- 172.21.15.86:0/2798292737 >> 172.21.15.86:0/2798292737 conn(0x559f77270510 msgr2=0x559f77270910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.120+0000 7f373b803740 1 -- 172.21.15.86:0/2798292737 shutdown_connections 2024-04-17T07:23:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.120+0000 7f373b803740 1 -- 172.21.15.86:0/2798292737 wait complete. 2024-04-17T07:23:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.120+0000 7f373b803740 1 -- start start 2024-04-17T07:23:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.121+0000 7f373b803740 1 Processor -- start 2024-04-17T07:23:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.121+0000 7f373b803740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f7727f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.121+0000 7f373b803740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 0x559f7727f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.121+0000 7f373b803740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f7727ab00 0x559f772867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.121+0000 7f373b803740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f7727d870 con 0x559f7727ab00 2024-04-17T07:23:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.121+0000 7f373b803740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559f77194830 con 0x559f77275990 2024-04-17T07:23:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.121+0000 7f373b803740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559f7727d9f0 con 0x559f77274fa0 2024-04-17T07:23:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.121+0000 7f373b803740 10 client.0 _ll_register_callbacks cb 0x559f77162fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.122+0000 7f3731e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f7727f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.122+0000 7f3731400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 0x559f7727f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.122+0000 7f3731400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 0x559f7727f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52240/0 (socket says 172.21.15.86:52240) 2024-04-17T07:23:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.122+0000 7f3731400640 1 -- 172.21.15.86:0/1820355592 learned_addr learned my addr 172.21.15.86:0/1820355592 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.123+0000 7f3731400640 1 -- 172.21.15.86:0/1820355592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 msgr2=0x559f7727f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.123+0000 7f3731400640 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f7727f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.123+0000 7f3731400640 1 -- 172.21.15.86:0/1820355592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f7727ab00 msgr2=0x559f772867e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.123+0000 7f3731400640 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f7727ab00 0x559f772867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.123+0000 7f3731400640 1 -- 172.21.15.86:0/1820355592 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f77198920 con 0x559f77275990 2024-04-17T07:23:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.124+0000 7f3731400640 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 0x559f7727f7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3720012700 tx=0x7f37200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc07d1aeda9bf7f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.124+0000 7f3713400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:23:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.125+0000 7f3713400640 1 -- 172.21.15.86:0/1820355592 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200209c0 con 0x559f77275990 2024-04-17T07:23:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.125+0000 7f373b803740 1 -- 172.21.15.86:0/1820355592 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37240bd5a0 con 0x559f77275990 2024-04-17T07:23:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.125+0000 7f3713400640 1 -- 172.21.15.86:0/1820355592 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3720020b60 con 0x559f77275990 2024-04-17T07:23:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.125+0000 7f3713400640 1 -- 172.21.15.86:0/1820355592 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3720032050 con 0x559f77275990 2024-04-17T07:23:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.125+0000 7f373b803740 10 client.36728 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.125+0000 7f373b803740 1 -- 172.21.15.86:0/1820355592 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f771667f0 con 0x559f77275990 2024-04-17T07:23:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.126+0000 7f3713400640 1 -- 172.21.15.86:0/1820355592 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37240bd5a0 con 0x559f77275990 2024-04-17T07:23:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.126+0000 7f3713400640 1 -- 172.21.15.86:0/1820355592 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f372000f040 con 0x559f77275990 2024-04-17T07:23:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.126+0000 7f373b803740 10 client.36728 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.127+0000 7f373b803740 10 client.36728 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:23:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.127+0000 7f373b803740 20 client.36728 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.127+0000 7f373b803740 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559f7728ada0 0x559f7728d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.127+0000 7f373b803740 4 client.36728 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.127+0000 7f373b803740 1 -- 172.21.15.86:0/1820355592 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f771e8a40 con 0x559f7728ada0 2024-04-17T07:23:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.127+0000 7f3731e00640 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559f7728ada0 0x559f7728d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.128+0000 7f3713400640 10 client.36728 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:23:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.129+0000 7f3731e00640 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559f7728ada0 0x559f7728d200 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4662aaeed63b06a server_cookie=882027a24d95b200 in_seq=0 out_seq=0 2024-04-17T07:23:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.130+0000 7f3713400640 1 -- 172.21.15.86:0/1820355592 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f771e8a40 con 0x559f7728ada0 2024-04-17T07:23:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.130+0000 7f3713400640 10 client.36728 handle_command_reply: tid=0 2024-04-17T07:23:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.130+0000 7f373b803740 1 client.36728 shutdown 2024-04-17T07:23:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.131+0000 7f373b803740 1 -- 172.21.15.86:0/1820355592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 msgr2=0x559f7727f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.131+0000 7f373b803740 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 0x559f7727f7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3720012700 tx=0x7f37200046c0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.131+0000 7f373b803740 1 -- 172.21.15.86:0/1820355592 shutdown_connections 2024-04-17T07:23:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.131+0000 7f373b803740 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559f7728ada0 0x559f7728d200 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.131+0000 7f373b803740 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f7727ab00 0x559f772867e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.131+0000 7f373b803740 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f77275990 0x559f7727f7a0 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3720012700 tx=0x7f37200046c0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.131+0000 7f373b803740 1 --2- 172.21.15.86:0/1820355592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f7727f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.131+0000 7f373b803740 1 -- 172.21.15.86:0/1820355592 >> 172.21.15.86:0/1820355592 conn(0x559f77270510 msgr2=0x559f7726f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.131+0000 7f3713400640 0 client.36728 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:23:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.132+0000 7f373b803740 1 -- 172.21.15.86:0/1820355592 shutdown_connections 2024-04-17T07:23:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.132+0000 7f373b803740 1 -- 172.21.15.86:0/1820355592 wait complete. 2024-04-17T07:23:48.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.133+0000 7f373b803740 20 client.36728 trim_cache size 0 max 16384 2024-04-17T07:23:48.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.160+0000 7f373b803740 1 Processor -- start 2024-04-17T07:23:48.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.160+0000 7f373b803740 1 -- start start 2024-04-17T07:23:48.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.161+0000 7f373b803740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f7727e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.161+0000 7f373b803740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 0x559f772867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.161+0000 7f373b803740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 0x559f7726ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.161+0000 7f373b803740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b1750 con 0x559f772775e0 2024-04-17T07:23:48.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.161+0000 7f373b803740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b3290 con 0x559f7727ab00 2024-04-17T07:23:48.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.161+0000 7f373b803740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b1e20 con 0x559f77274fa0 2024-04-17T07:23:48.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.161+0000 7f3731e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f7727e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.161+0000 7f3732800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 0x559f7726ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.161+0000 7f3731e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f7727e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43736/0 (socket says 172.21.15.86:43736) 2024-04-17T07:23:48.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.161+0000 7f3731e00640 1 -- 172.21.15.86:0/2724320560 learned_addr learned my addr 172.21.15.86:0/2724320560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:48.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.162+0000 7f3731400640 1 --2- 172.21.15.86:0/2724320560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 0x559f772867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.162+0000 7f3731e00640 1 -- 172.21.15.86:0/2724320560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 msgr2=0x559f772867a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.162+0000 7f3731e00640 1 --2- 172.21.15.86:0/2724320560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 0x559f772867a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.162+0000 7f3731e00640 1 -- 172.21.15.86:0/2724320560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 msgr2=0x559f7726ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.162+0000 7f3731e00640 1 --2- 172.21.15.86:0/2724320560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 0x559f7726ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.162+0000 7f3731e00640 1 -- 172.21.15.86:0/2724320560 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f76eed3f0 con 0x559f77274fa0 2024-04-17T07:23:48.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.163+0000 7f3731e00640 1 --2- 172.21.15.86:0/2724320560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f7727e880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f37240bcba0 tx=0x7f37240bcd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ed81e0ec1e148b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.163+0000 7f371e600640 1 -- 172.21.15.86:0/2724320560 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37240d5040 con 0x559f77274fa0 2024-04-17T07:23:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.163+0000 7f371e600640 1 -- 172.21.15.86:0/2724320560 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37240bcea0 con 0x559f77274fa0 2024-04-17T07:23:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.163+0000 7f371e600640 1 -- 172.21.15.86:0/2724320560 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37240b9510 con 0x559f77274fa0 2024-04-17T07:23:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.164+0000 7f373b803740 1 -- 172.21.15.86:0/2724320560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 msgr2=0x559f7727e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.164+0000 7f373b803740 1 --2- 172.21.15.86:0/2724320560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f7727e880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f37240bcba0 tx=0x7f37240bcd70 comp rx=0 tx=0).stop 2024-04-17T07:23:48.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.165+0000 7f373b803740 1 -- 172.21.15.86:0/2724320560 shutdown_connections 2024-04-17T07:23:48.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.165+0000 7f373b803740 1 --2- 172.21.15.86:0/2724320560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 0x559f7726ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.165+0000 7f373b803740 1 --2- 172.21.15.86:0/2724320560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 0x559f772867a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.165+0000 7f373b803740 1 --2- 172.21.15.86:0/2724320560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f7727e880 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.165+0000 7f373b803740 1 -- 172.21.15.86:0/2724320560 >> 172.21.15.86:0/2724320560 conn(0x559f77270510 msgr2=0x559f77276ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.165+0000 7f373b803740 1 -- 172.21.15.86:0/2724320560 shutdown_connections 2024-04-17T07:23:48.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.166+0000 7f373b803740 1 -- 172.21.15.86:0/2724320560 wait complete. 2024-04-17T07:23:48.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.166+0000 7f373b803740 1 -- start start 2024-04-17T07:23:48.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.166+0000 7f373b803740 1 Processor -- start 2024-04-17T07:23:48.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.167+0000 7f373b803740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f77284850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.167+0000 7f373b803740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 0x559f77284d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.167+0000 7f373b803740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 0x559f77285da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.167+0000 7f373b803740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b2bc0 con 0x559f772775e0 2024-04-17T07:23:48.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.167+0000 7f373b803740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37341b24f0 con 0x559f7727ab00 2024-04-17T07:23:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.167+0000 7f373b803740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37341af720 con 0x559f77274fa0 2024-04-17T07:23:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.167+0000 7f373b803740 10 client.0 _ll_register_callbacks cb 0x7f3720010d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.168+0000 7f3731400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 0x559f77284d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.168+0000 7f3731e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f77284850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.168+0000 7f3731e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f77284850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43746/0 (socket says 172.21.15.86:43746) 2024-04-17T07:23:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.168+0000 7f3731e00640 1 -- 172.21.15.86:0/2078165921 learned_addr learned my addr 172.21.15.86:0/2078165921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:48.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.168+0000 7f3732800640 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 0x559f77285da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.169+0000 7f3731e00640 1 -- 172.21.15.86:0/2078165921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 msgr2=0x559f77285da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.169+0000 7f3731e00640 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 0x559f77285da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.169+0000 7f3731e00640 1 -- 172.21.15.86:0/2078165921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 msgr2=0x559f77284d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.169+0000 7f3731e00640 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 0x559f77284d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.169+0000 7f3731e00640 1 -- 172.21.15.86:0/2078165921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f76fb77d0 con 0x559f77274fa0 2024-04-17T07:23:48.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.169+0000 7f3731400640 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 0x559f77284d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:48.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.169+0000 7f3732800640 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 0x559f77285da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:23:48.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.170+0000 7f3731e00640 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f77284850 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3724009560 tx=0x7f3724005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4e601b87d838fd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.170+0000 7f3713400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:48.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.171+0000 7f3713400640 1 -- 172.21.15.86:0/2078165921 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37240d5070 con 0x559f77274fa0 2024-04-17T07:23:48.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.171+0000 7f3713400640 1 -- 172.21.15.86:0/2078165921 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f37240b9510 con 0x559f77274fa0 2024-04-17T07:23:48.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.171+0000 7f373b803740 1 -- 172.21.15.86:0/2078165921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37240bcea0 con 0x559f77274fa0 2024-04-17T07:23:48.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.172+0000 7f3713400640 1 -- 172.21.15.86:0/2078165921 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f37240bd5a0 con 0x559f77274fa0 2024-04-17T07:23:48.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.172+0000 7f373b803740 10 client.26773 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:48.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.172+0000 7f373b803740 1 -- 172.21.15.86:0/2078165921 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f772f1a60 con 0x559f77274fa0 2024-04-17T07:23:48.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.174+0000 7f3713400640 1 -- 172.21.15.86:0/2078165921 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37240bcea0 con 0x559f77274fa0 2024-04-17T07:23:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.175+0000 7f3713400640 1 -- 172.21.15.86:0/2078165921 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f372403c070 con 0x559f77274fa0 2024-04-17T07:23:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.175+0000 7f373b803740 10 client.26773 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.175+0000 7f373b803740 10 client.26773 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:23:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.175+0000 7f373b803740 20 client.26773 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.175+0000 7f373b803740 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559f772f2070 0x559f772f4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.175+0000 7f373b803740 4 client.26773 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:23:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.175+0000 7f373b803740 1 -- 172.21.15.86:0/2078165921 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f7727f050 con 0x559f772f2070 2024-04-17T07:23:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.176+0000 7f3731400640 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559f772f2070 0x559f772f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.177+0000 7f3731400640 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559f772f2070 0x559f772f4490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bd8191bf150bef6f server_cookie=b1dee50d79a6e20b in_seq=0 out_seq=0 2024-04-17T07:23:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.178+0000 7f3713400640 10 client.26773 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:23:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.178+0000 7f3713400640 1 -- 172.21.15.86:0/2078165921 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f7727f050 con 0x559f772f2070 2024-04-17T07:23:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.178+0000 7f3713400640 10 client.26773 handle_command_reply: tid=0 2024-04-17T07:23:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.178+0000 7f373b803740 1 client.26773 shutdown 2024-04-17T07:23:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.178+0000 7f373b803740 1 -- 172.21.15.86:0/2078165921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 msgr2=0x559f77284850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.178+0000 7f373b803740 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f77284850 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3724009560 tx=0x7f3724005c90 comp rx=0 tx=0).stop 2024-04-17T07:23:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.178+0000 7f373b803740 1 -- 172.21.15.86:0/2078165921 shutdown_connections 2024-04-17T07:23:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.178+0000 7f373b803740 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559f772f2070 0x559f772f4490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.179+0000 7f3713400640 0 client.26773 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:23:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.179+0000 7f373b803740 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f7727ab00 0x559f77285da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.179+0000 7f373b803740 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f772775e0 0x559f77284d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.179+0000 7f373b803740 1 --2- 172.21.15.86:0/2078165921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f77274fa0 0x559f77284850 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.179+0000 7f373b803740 1 -- 172.21.15.86:0/2078165921 >> 172.21.15.86:0/2078165921 conn(0x559f77270510 msgr2=0x559f77273770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.180+0000 7f373b803740 1 -- 172.21.15.86:0/2078165921 shutdown_connections 2024-04-17T07:23:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.180+0000 7f373b803740 1 -- 172.21.15.86:0/2078165921 wait complete. 2024-04-17T07:23:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.180+0000 7f373b803740 20 client.26773 trim_cache size 0 max 16384 2024-04-17T07:23:48.181 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:23:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.181+0000 7f3738a00640 1 -- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37180a3dc0 msgr2=0x7f37180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.181+0000 7f3738a00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37180a3dc0 0x7f37180a6280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f37341a0000 tx=0x7f3720009260 comp rx=0 tx=0).stop 2024-04-17T07:23:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.181+0000 7f3738a00640 1 -- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3734102040 msgr2=0x7f373419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.181+0000 7f3738a00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3734102040 0x7f373419eaf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f37341009f0 tx=0x7f3724004270 comp rx=0 tx=0).stop 2024-04-17T07:23:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.182+0000 7f3738a00640 1 -- 172.21.15.86:0/2386071218 shutdown_connections 2024-04-17T07:23:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.182+0000 7f3738a00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37180a3dc0 0x7f37180a6280 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.182+0000 7f3738a00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37341a4090 0x7f37341a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.182+0000 7f3738a00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3734102a00 0x7f373419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.182+0000 7f3738a00640 1 --2- 172.21.15.86:0/2386071218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3734102040 0x7f373419eaf0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.182+0000 7f3738a00640 1 -- 172.21.15.86:0/2386071218 >> 172.21.15.86:0/2386071218 conn(0x7f37340fc420 msgr2=0x7f3734105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.182+0000 7f3738a00640 1 -- 172.21.15.86:0/2386071218 shutdown_connections 2024-04-17T07:23:48.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.183+0000 7f3738a00640 1 -- 172.21.15.86:0/2386071218 wait complete. 2024-04-17T07:23:48.223 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:23:48.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:48 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1248599097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:23:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:48 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1248599097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:23:48.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.549+0000 7f2e22000640 1 -- 172.21.15.86:0/1258591684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e1c1107e0 msgr2=0x7f2e1c112bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.549+0000 7f2e22000640 1 --2- 172.21.15.86:0/1258591684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e1c1107e0 0x7f2e1c112bd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009890 tx=0x7f2e1402bb70 comp rx=0 tx=0).stop 2024-04-17T07:23:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.551+0000 7f2e22000640 1 -- 172.21.15.86:0/1258591684 shutdown_connections 2024-04-17T07:23:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.551+0000 7f2e22000640 1 --2- 172.21.15.86:0/1258591684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e1c1107e0 0x7f2e1c112bd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.551+0000 7f2e22000640 1 --2- 172.21.15.86:0/1258591684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e1c1098c0 0x7f2e1c1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.551+0000 7f2e22000640 1 --2- 172.21.15.86:0/1258591684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c108f60 0x7f2e1c109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.551+0000 7f2e22000640 1 -- 172.21.15.86:0/1258591684 >> 172.21.15.86:0/1258591684 conn(0x7f2e1c06f690 msgr2=0x7f2e1c071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.551+0000 7f2e22000640 1 -- 172.21.15.86:0/1258591684 shutdown_connections 2024-04-17T07:23:48.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.551+0000 7f2e22000640 1 -- 172.21.15.86:0/1258591684 wait complete. 2024-04-17T07:23:48.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.552+0000 7f2e22000640 1 Processor -- start 2024-04-17T07:23:48.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.552+0000 7f2e22000640 1 -- start start 2024-04-17T07:23:48.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.553+0000 7f2e22000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c108f60 0x7f2e1c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.553+0000 7f2e22000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e1c1098c0 0x7f2e1c1a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.553+0000 7f2e22000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e1c1107e0 0x7f2e1c1a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.553+0000 7f2e22000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1162c0 con 0x7f2e1c108f60 2024-04-17T07:23:48.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.553+0000 7f2e22000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c116140 con 0x7f2e1c1107e0 2024-04-17T07:23:48.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.553+0000 7f2e22000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c116440 con 0x7f2e1c1098c0 2024-04-17T07:23:48.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.553+0000 7f2e1be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e1c1107e0 0x7f2e1c1a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.553+0000 7f2e1aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e1c1098c0 0x7f2e1c1a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.553+0000 7f2e1be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e1c1107e0 0x7f2e1c1a7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52282/0 (socket says 172.21.15.86:52282) 2024-04-17T07:23:48.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.554+0000 7f2e1be00640 1 -- 172.21.15.86:0/2887842014 learned_addr learned my addr 172.21.15.86:0/2887842014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:48.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.554+0000 7f2e1b400640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c108f60 0x7f2e1c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.554+0000 7f2e1be00640 1 -- 172.21.15.86:0/2887842014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e1c1098c0 msgr2=0x7f2e1c1a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.554+0000 7f2e1be00640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e1c1098c0 0x7f2e1c1a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.554+0000 7f2e1be00640 1 -- 172.21.15.86:0/2887842014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c108f60 msgr2=0x7f2e1c1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.554+0000 7f2e1be00640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c108f60 0x7f2e1c1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.554+0000 7f2e1be00640 1 -- 172.21.15.86:0/2887842014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c10a170 con 0x7f2e1c1107e0 2024-04-17T07:23:48.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.555+0000 7f2e1be00640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e1c1107e0 0x7f2e1c1a7fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e140057c0 tx=0x7f2e14004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.555+0000 7f2e13e00640 1 -- 172.21.15.86:0/2887842014 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14041030 con 0x7f2e1c1107e0 2024-04-17T07:23:48.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.556+0000 7f2e13e00640 1 -- 172.21.15.86:0/2887842014 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e1402c9c0 con 0x7f2e1c1107e0 2024-04-17T07:23:48.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.556+0000 7f2e13e00640 1 -- 172.21.15.86:0/2887842014 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14039040 con 0x7f2e1c1107e0 2024-04-17T07:23:48.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.556+0000 7f2e22000640 1 -- 172.21.15.86:0/2887842014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1c10a400 con 0x7f2e1c1107e0 2024-04-17T07:23:48.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.556+0000 7f2e22000640 1 -- 172.21.15.86:0/2887842014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1c10a8a0 con 0x7f2e1c1107e0 2024-04-17T07:23:48.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.561+0000 7f2e24d81740 1 Processor -- start 2024-04-17T07:23:48.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.563+0000 7f2e13e00640 1 -- 172.21.15.86:0/2887842014 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2e14044070 con 0x7f2e1c1107e0 2024-04-17T07:23:48.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.564+0000 7f2e24d81740 1 -- start start 2024-04-17T07:23:48.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.564+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b297b0 0x555c30b29bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.564+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b2a1a0 0x555c30b2eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.564+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2f310 0x555c30b317a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.564+0000 7f2e24d81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555c30a87020 con 0x555c30b297b0 2024-04-17T07:23:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.564+0000 7f2e24d81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c30a2d460 con 0x555c30b2a1a0 2024-04-17T07:23:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.564+0000 7f2e24d81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c30a82fe0 con 0x555c30b2f310 2024-04-17T07:23:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.565+0000 7f2e1aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2f310 0x555c30b317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.565+0000 7f2e13e00640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e000a3d80 0x7f2e000a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.565+0000 7f2e1aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2f310 0x555c30b317a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43778/0 (socket says 172.21.15.86:43778) 2024-04-17T07:23:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.565+0000 7f2e1aa00640 1 -- 172.21.15.86:0/1833997917 learned_addr learned my addr 172.21.15.86:0/1833997917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:48.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.565+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/1833997917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b297b0 0x555c30b29bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.566+0000 7f2e1b400640 1 --2- 172.21.15.86:0/1833997917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b2a1a0 0x555c30b2eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.566+0000 7f2e1aa00640 1 -- 172.21.15.86:0/1833997917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2f310 msgr2=0x555c30b317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.566+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/1833997917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2f310 0x555c30b317a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.566+0000 7f2e1aa00640 1 -- 172.21.15.86:0/1833997917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b2a1a0 msgr2=0x555c30b2eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.566+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/1833997917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b2a1a0 0x555c30b2eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.566+0000 7f2e1aa00640 1 -- 172.21.15.86:0/1833997917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c30a35040 con 0x555c30b297b0 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.566+0000 7f2e13e00640 1 -- 172.21.15.86:0/2887842014 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2e140f1b90 con 0x7f2e1c1107e0 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.566+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/1833997917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b297b0 0x555c30b29bd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0099f0 tx=0x7f2e0c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fed4fd74a750f41 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.567+0000 7f2e1be00640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e000a3d80 0x7f2e000a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.567+0000 7f2e0be00640 1 -- 172.21.15.86:0/1833997917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c022070 con 0x555c30b297b0 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.567+0000 7f2e0be00640 1 -- 172.21.15.86:0/1833997917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e0c00e810 con 0x555c30b297b0 2024-04-17T07:23:48.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.567+0000 7f2e0be00640 1 -- 172.21.15.86:0/1833997917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c01b340 con 0x555c30b297b0 2024-04-17T07:23:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.569+0000 7f2e24d81740 1 -- 172.21.15.86:0/1833997917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b297b0 msgr2=0x555c30b29bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.569+0000 7f2e24d81740 1 --2- 172.21.15.86:0/1833997917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b297b0 0x555c30b29bd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0099f0 tx=0x7f2e0c0095c0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.569+0000 7f2e1be00640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e000a3d80 0x7f2e000a6240 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2e140c61d0 tx=0x7f2e14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.569+0000 7f2e24d81740 1 -- 172.21.15.86:0/1833997917 shutdown_connections 2024-04-17T07:23:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.569+0000 7f2e24d81740 1 --2- 172.21.15.86:0/1833997917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2f310 0x555c30b317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.569+0000 7f2e24d81740 1 --2- 172.21.15.86:0/1833997917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b2a1a0 0x555c30b2eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.569+0000 7f2e24d81740 1 --2- 172.21.15.86:0/1833997917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b297b0 0x555c30b29bd0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.569+0000 7f2e24d81740 1 -- 172.21.15.86:0/1833997917 >> 172.21.15.86:0/1833997917 conn(0x555c30b1c510 msgr2=0x555c30b1c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.570+0000 7f2e24d81740 1 -- 172.21.15.86:0/1833997917 shutdown_connections 2024-04-17T07:23:48.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.570+0000 7f2e24d81740 1 -- 172.21.15.86:0/1833997917 wait complete. 2024-04-17T07:23:48.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.570+0000 7f2e24d81740 1 -- start start 2024-04-17T07:23:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.571+0000 7f2e24d81740 1 Processor -- start 2024-04-17T07:23:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.571+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b33db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.571+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.571+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b3b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.571+0000 7f2e24d81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555c30b322c0 con 0x555c30b2f310 2024-04-17T07:23:48.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.571+0000 7f2e24d81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c30a40830 con 0x555c30b297b0 2024-04-17T07:23:48.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.571+0000 7f2e24d81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c30b32440 con 0x555c30b2a1a0 2024-04-17T07:23:48.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.571+0000 7f2e24d81740 10 client.0 _ll_register_callbacks cb 0x555c30a0efb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:48.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.573+0000 7f2e1b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.573+0000 7f2e1b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43780/0 (socket says 172.21.15.86:43780) 2024-04-17T07:23:48.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.573+0000 7f2e1b400640 1 -- 172.21.15.86:0/1006298302 learned_addr learned my addr 172.21.15.86:0/1006298302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.573+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b3b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.573+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b33db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.574+0000 7f2e1b400640 1 -- 172.21.15.86:0/1006298302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 msgr2=0x555c30b33db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.574+0000 7f2e1b400640 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b33db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.574+0000 7f2e1b400640 1 -- 172.21.15.86:0/1006298302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 msgr2=0x555c30b3b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.574+0000 7f2e1b400640 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b3b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.574+0000 7f2e1b400640 1 -- 172.21.15.86:0/1006298302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c30a44920 con 0x555c30b2a1a0 2024-04-17T07:23:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.574+0000 7f2e1b400640 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b342f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e04012700 tx=0x7f2e04012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64c0039a615a881a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.574+0000 7f2e08c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.575+0000 7f2e08c00640 1 -- 172.21.15.86:0/1006298302 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04004030 con 0x555c30b2a1a0 2024-04-17T07:23:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.575+0000 7f2e08c00640 1 -- 172.21.15.86:0/1006298302 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2e040041d0 con 0x555c30b2a1a0 2024-04-17T07:23:48.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.575+0000 7f2e24d81740 1 -- 172.21.15.86:0/1006298302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0c00e810 con 0x555c30b2a1a0 2024-04-17T07:23:48.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.576+0000 7f2e08c00640 1 -- 172.21.15.86:0/1006298302 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e04025050 con 0x555c30b2a1a0 2024-04-17T07:23:48.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.576+0000 7f2e24d81740 10 client.26785 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:48.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.576+0000 7f2e24d81740 1 -- 172.21.15.86:0/1006298302 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c30a127f0 con 0x555c30b2a1a0 2024-04-17T07:23:48.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.577+0000 7f2e08c00640 1 -- 172.21.15.86:0/1006298302 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e0c00e810 con 0x555c30b2a1a0 2024-04-17T07:23:48.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.577+0000 7f2e08c00640 1 -- 172.21.15.86:0/1006298302 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2e04003940 con 0x555c30b2a1a0 2024-04-17T07:23:48.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.577+0000 7f2e24d81740 10 client.26785 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:48.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.577+0000 7f2e24d81740 10 client.26785 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:23:48.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.577+0000 7f2e24d81740 20 client.26785 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.577+0000 7f2e24d81740 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555c30b3f980 0x555c30b41de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.577+0000 7f2e24d81740 4 client.26785 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.577+0000 7f2e24d81740 1 -- 172.21.15.86:0/1006298302 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c30a94a40 con 0x555c30b3f980 2024-04-17T07:23:48.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.578+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555c30b3f980 0x555c30b41de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.579+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555c30b3f980 0x555c30b41de0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9c8665fc0548a1a server_cookie=73eac58b24f4fa53 in_seq=0 out_seq=0 2024-04-17T07:23:48.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.579+0000 7f2e08c00640 10 client.26785 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:23:48.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:48 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1248599097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:23:48.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.580+0000 7f2e08c00640 1 -- 172.21.15.86:0/1006298302 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c30a94a40 con 0x555c30b3f980 2024-04-17T07:23:48.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.580+0000 7f2e08c00640 10 client.26785 handle_command_reply: tid=0 2024-04-17T07:23:48.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.581+0000 7f2e24d81740 1 client.26785 shutdown 2024-04-17T07:23:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.584+0000 7f2e24d81740 1 -- 172.21.15.86:0/1006298302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 msgr2=0x555c30b342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.584+0000 7f2e24d81740 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b342f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e04012700 tx=0x7f2e04012bc0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.584+0000 7f2e24d81740 1 -- 172.21.15.86:0/1006298302 shutdown_connections 2024-04-17T07:23:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.584+0000 7f2e24d81740 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555c30b3f980 0x555c30b41de0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.584+0000 7f2e24d81740 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b3b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.584+0000 7f2e24d81740 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b342f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.584+0000 7f2e24d81740 1 --2- 172.21.15.86:0/1006298302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b33db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.584+0000 7f2e24d81740 1 -- 172.21.15.86:0/1006298302 >> 172.21.15.86:0/1006298302 conn(0x555c30b1c510 msgr2=0x555c30b1b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.584+0000 7f2e08c00640 0 client.26785 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:23:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.585+0000 7f2e24d81740 1 -- 172.21.15.86:0/1006298302 shutdown_connections 2024-04-17T07:23:48.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.585+0000 7f2e24d81740 1 -- 172.21.15.86:0/1006298302 wait complete. 2024-04-17T07:23:48.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.586+0000 7f2e24d81740 20 client.26785 trim_cache size 0 max 16384 2024-04-17T07:23:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.625+0000 7f2e24d81740 1 Processor -- start 2024-04-17T07:23:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.626+0000 7f2e24d81740 1 -- start start 2024-04-17T07:23:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.626+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.626+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b3b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.626+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b2b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.626+0000 7f2e24d81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b1170 con 0x555c30b2f310 2024-04-17T07:23:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.626+0000 7f2e24d81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b2cb0 con 0x555c30b297b0 2024-04-17T07:23:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.626+0000 7f2e24d81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b1840 con 0x555c30b2a1a0 2024-04-17T07:23:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.627+0000 7f2e1aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.627+0000 7f2e1aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b2aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52298/0 (socket says 172.21.15.86:52298) 2024-04-17T07:23:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.627+0000 7f2e1aa00640 1 -- 172.21.15.86:0/420507805 learned_addr learned my addr 172.21.15.86:0/420507805 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.627+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/420507805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b2b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.627+0000 7f2e1b400640 1 --2- 172.21.15.86:0/420507805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b3b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.628+0000 7f2e1aa00640 1 -- 172.21.15.86:0/420507805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 msgr2=0x555c30b3b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.628+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/420507805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b3b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.628+0000 7f2e1aa00640 1 -- 172.21.15.86:0/420507805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 msgr2=0x555c30b2aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.628+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/420507805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b2aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.628+0000 7f2e1aa00640 1 -- 172.21.15.86:0/420507805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c307993f0 con 0x555c30b2f310 2024-04-17T07:23:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.628+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/420507805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b2b770 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0099c0 tx=0x7f2e0c009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3a37139f06fa97 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.628+0000 7f2e0b400640 1 -- 172.21.15.86:0/420507805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c022030 con 0x555c30b2f310 2024-04-17T07:23:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.629+0000 7f2e0b400640 1 -- 172.21.15.86:0/420507805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e0c02ce50 con 0x555c30b2f310 2024-04-17T07:23:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.629+0000 7f2e0b400640 1 -- 172.21.15.86:0/420507805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c02c080 con 0x555c30b2f310 2024-04-17T07:23:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.629+0000 7f2e24d81740 1 -- 172.21.15.86:0/420507805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 msgr2=0x555c30b2b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.629+0000 7f2e24d81740 1 --2- 172.21.15.86:0/420507805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b2b770 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0099c0 tx=0x7f2e0c009f00 comp rx=0 tx=0).stop 2024-04-17T07:23:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.630+0000 7f2e24d81740 1 -- 172.21.15.86:0/420507805 shutdown_connections 2024-04-17T07:23:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.630+0000 7f2e24d81740 1 --2- 172.21.15.86:0/420507805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b2b770 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.630+0000 7f2e24d81740 1 --2- 172.21.15.86:0/420507805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b3b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.630+0000 7f2e24d81740 1 --2- 172.21.15.86:0/420507805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b2aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.630+0000 7f2e24d81740 1 -- 172.21.15.86:0/420507805 >> 172.21.15.86:0/420507805 conn(0x555c30b1c510 msgr2=0x555c30b33db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.630+0000 7f2e24d81740 1 -- 172.21.15.86:0/420507805 shutdown_connections 2024-04-17T07:23:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.630+0000 7f2e24d81740 1 -- 172.21.15.86:0/420507805 wait complete. 2024-04-17T07:23:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.630+0000 7f2e24d81740 1 -- start start 2024-04-17T07:23:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.630+0000 7f2e24d81740 1 Processor -- start 2024-04-17T07:23:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.631+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.631+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b38ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.631+0000 7f2e24d81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b39af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.631+0000 7f2e24d81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b25e0 con 0x555c30b2f310 2024-04-17T07:23:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.631+0000 7f2e24d81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1b1f10 con 0x555c30b297b0 2024-04-17T07:23:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.631+0000 7f2e24d81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c1af140 con 0x555c30b2a1a0 2024-04-17T07:23:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.631+0000 7f2e24d81740 10 client.0 _ll_register_callbacks cb 0x7f2e04009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.631+0000 7f2e1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b39af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.632+0000 7f2e1aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.632+0000 7f2e1aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52306/0 (socket says 172.21.15.86:52306) 2024-04-17T07:23:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.632+0000 7f2e1aa00640 1 -- 172.21.15.86:0/2085937975 learned_addr learned my addr 172.21.15.86:0/2085937975 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.632+0000 7f2e1b400640 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b38ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.632+0000 7f2e1aa00640 1 -- 172.21.15.86:0/2085937975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 msgr2=0x555c30b38ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.632+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b38ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.632+0000 7f2e1aa00640 1 -- 172.21.15.86:0/2085937975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 msgr2=0x555c30b385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.632+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.632+0000 7f2e1aa00640 1 -- 172.21.15.86:0/2085937975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c308637d0 con 0x555c30b2f310 2024-04-17T07:23:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.633+0000 7f2e1b400640 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b38ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.633+0000 7f2e1aa00640 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b39af0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00eb20 tx=0x7f2e0c00e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75891e18a40afd63 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.633+0000 7f2e08c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:23:48.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.634+0000 7f2e08c00640 1 -- 172.21.15.86:0/2085937975 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c022040 con 0x555c30b2f310 2024-04-17T07:23:48.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.634+0000 7f2e08c00640 1 -- 172.21.15.86:0/2085937975 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2e0c01d440 con 0x555c30b2f310 2024-04-17T07:23:48.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.634+0000 7f2e24d81740 1 -- 172.21.15.86:0/2085937975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0c02ce50 con 0x555c30b2f310 2024-04-17T07:23:48.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.635+0000 7f2e08c00640 1 -- 172.21.15.86:0/2085937975 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e0c00ee60 con 0x555c30b2f310 2024-04-17T07:23:48.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.635+0000 7f2e24d81740 10 client.17832 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:48.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.635+0000 7f2e24d81740 1 -- 172.21.15.86:0/2085937975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c30b3b210 con 0x555c30b2f310 2024-04-17T07:23:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.636+0000 7f2e08c00640 1 -- 172.21.15.86:0/2085937975 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e0c02ce50 con 0x555c30b2f310 2024-04-17T07:23:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.636+0000 7f2e08c00640 1 -- 172.21.15.86:0/2085937975 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2e0c037040 con 0x555c30b2f310 2024-04-17T07:23:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.636+0000 7f2e24d81740 10 client.17832 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.636+0000 7f2e24d81740 10 client.17832 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:23:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.636+0000 7f2e24d81740 20 client.17832 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.636+0000 7f2e24d81740 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555c30ba6050 0x555c30ba8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.636+0000 7f2e24d81740 4 client.17832 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:23:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.636+0000 7f2e24d81740 1 -- 172.21.15.86:0/2085937975 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c30b2b280 con 0x555c30ba6050 2024-04-17T07:23:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.637+0000 7f2e1b400640 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555c30ba6050 0x555c30ba8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.638+0000 7f2e1b400640 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555c30ba6050 0x555c30ba8440 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bbbd24d08e862388 server_cookie=9e797145f413fe55 in_seq=0 out_seq=0 2024-04-17T07:23:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.638+0000 7f2e08c00640 10 client.17832 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:23:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.638+0000 7f2e08c00640 1 -- 172.21.15.86:0/2085937975 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c30b2b280 con 0x555c30ba6050 2024-04-17T07:23:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.638+0000 7f2e08c00640 10 client.17832 handle_command_reply: tid=0 2024-04-17T07:23:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.638+0000 7f2e24d81740 1 client.17832 shutdown 2024-04-17T07:23:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 1 -- 172.21.15.86:0/2085937975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 msgr2=0x555c30b39af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b39af0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00eb20 tx=0x7f2e0c00e950 comp rx=0 tx=0).stop 2024-04-17T07:23:48.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 1 -- 172.21.15.86:0/2085937975 shutdown_connections 2024-04-17T07:23:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555c30ba6050 0x555c30ba8440 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c30b2f310 0x555c30b39af0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c30b2a1a0 0x555c30b38ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 1 --2- 172.21.15.86:0/2085937975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c30b297b0 0x555c30b385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 1 -- 172.21.15.86:0/2085937975 >> 172.21.15.86:0/2085937975 conn(0x555c30b1c510 msgr2=0x555c30b33c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e08c00640 0 client.17832 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:23:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 1 -- 172.21.15.86:0/2085937975 shutdown_connections 2024-04-17T07:23:48.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 1 -- 172.21.15.86:0/2085937975 wait complete. 2024-04-17T07:23:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.639+0000 7f2e24d81740 20 client.17832 trim_cache size 0 max 16384 2024-04-17T07:23:48.641 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:23:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.641+0000 7f2e22000640 1 -- 172.21.15.86:0/2887842014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e000a3d80 msgr2=0x7f2e000a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.641+0000 7f2e22000640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e000a3d80 0x7f2e000a6240 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2e140c61d0 tx=0x7f2e14005f90 comp rx=0 tx=0).stop 2024-04-17T07:23:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.642+0000 7f2e22000640 1 -- 172.21.15.86:0/2887842014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e1c1107e0 msgr2=0x7f2e1c1a7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.642+0000 7f2e22000640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e1c1107e0 0x7f2e1c1a7fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e140057c0 tx=0x7f2e14004270 comp rx=0 tx=0).stop 2024-04-17T07:23:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.642+0000 7f2e22000640 1 -- 172.21.15.86:0/2887842014 shutdown_connections 2024-04-17T07:23:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.642+0000 7f2e22000640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e000a3d80 0x7f2e000a6240 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.642+0000 7f2e22000640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e1c1107e0 0x7f2e1c1a7fc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.642+0000 7f2e22000640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e1c1098c0 0x7f2e1c1a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.642+0000 7f2e22000640 1 --2- 172.21.15.86:0/2887842014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e1c108f60 0x7f2e1c1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:48.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.642+0000 7f2e22000640 1 -- 172.21.15.86:0/2887842014 >> 172.21.15.86:0/2887842014 conn(0x7f2e1c06f690 msgr2=0x7f2e1c0713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:48.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.642+0000 7f2e22000640 1 -- 172.21.15.86:0/2887842014 shutdown_connections 2024-04-17T07:23:48.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:48.642+0000 7f2e22000640 1 -- 172.21.15.86:0/2887842014 wait complete. 2024-04-17T07:23:48.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:23:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.018+0000 7fec72c00640 1 -- 172.21.15.86:0/1073985378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec6c10e2a0 msgr2=0x7fec6c10e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.018+0000 7fec72c00640 1 --2- 172.21.15.86:0/1073985378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec6c10e2a0 0x7fec6c10e750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fec64009ca0 tx=0x7fec6402c120 comp rx=0 tx=0).stop 2024-04-17T07:23:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.019+0000 7fec72c00640 1 -- 172.21.15.86:0/1073985378 shutdown_connections 2024-04-17T07:23:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.019+0000 7fec72c00640 1 --2- 172.21.15.86:0/1073985378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec6c10e2a0 0x7fec6c10e750 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.019+0000 7fec72c00640 1 --2- 172.21.15.86:0/1073985378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c10b8d0 0x7fec6c10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.019+0000 7fec72c00640 1 --2- 172.21.15.86:0/1073985378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6c108f60 0x7fec6c10b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.019+0000 7fec72c00640 1 -- 172.21.15.86:0/1073985378 >> 172.21.15.86:0/1073985378 conn(0x7fec6c06edf0 msgr2=0x7fec6c071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.019+0000 7fec72c00640 1 -- 172.21.15.86:0/1073985378 shutdown_connections 2024-04-17T07:23:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.019+0000 7fec72c00640 1 -- 172.21.15.86:0/1073985378 wait complete. 2024-04-17T07:23:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.020+0000 7fec72c00640 1 Processor -- start 2024-04-17T07:23:49.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.020+0000 7fec72c00640 1 -- start start 2024-04-17T07:23:49.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.021+0000 7fec72c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c108f60 0x7fec6c1a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.021+0000 7fec72c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6c10b8d0 0x7fec6c1a7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.021+0000 7fec72c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec6c10e2a0 0x7fec6c1ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.021+0000 7fec72c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c11c950 con 0x7fec6c108f60 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.021+0000 7fec72c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c11c7d0 con 0x7fec6c10b8d0 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.021+0000 7fec72c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c11cad0 con 0x7fec6c10e2a0 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.021+0000 7fec6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c108f60 0x7fec6c1a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.022+0000 7fec6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c108f60 0x7fec6c1a7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33818/0 (socket says 172.21.15.86:33818) 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.022+0000 7fec6be00640 1 -- 172.21.15.86:0/2191273693 learned_addr learned my addr 172.21.15.86:0/2191273693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.022+0000 7fec70c00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec6c10e2a0 0x7fec6c1ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.022+0000 7fec6b400640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6c10b8d0 0x7fec6c1a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.022+0000 7fec6be00640 1 -- 172.21.15.86:0/2191273693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec6c10e2a0 msgr2=0x7fec6c1ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.022+0000 7fec6be00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec6c10e2a0 0x7fec6c1ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.022+0000 7fec6be00640 1 -- 172.21.15.86:0/2191273693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6c10b8d0 msgr2=0x7fec6c1a7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.022+0000 7fec6be00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6c10b8d0 0x7fec6c1a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.022+0000 7fec6be00640 1 -- 172.21.15.86:0/2191273693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c1aec70 con 0x7fec6c108f60 2024-04-17T07:23:49.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.023+0000 7fec6be00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c108f60 0x7fec6c1a7050 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00b450 tx=0x7fec5c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.025+0000 7fec68c00640 1 -- 172.21.15.86:0/2191273693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c015020 con 0x7fec6c108f60 2024-04-17T07:23:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.025+0000 7fec68c00640 1 -- 172.21.15.86:0/2191273693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec5c011d40 con 0x7fec6c108f60 2024-04-17T07:23:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.025+0000 7fec68c00640 1 -- 172.21.15.86:0/2191273693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00f340 con 0x7fec6c108f60 2024-04-17T07:23:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.025+0000 7fec72c00640 1 -- 172.21.15.86:0/2191273693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6c1aef60 con 0x7fec6c108f60 2024-04-17T07:23:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.025+0000 7fec72c00640 1 -- 172.21.15.86:0/2191273693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6c1af370 con 0x7fec6c108f60 2024-04-17T07:23:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.033+0000 7fec75802740 1 Processor -- start 2024-04-17T07:23:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.034+0000 7fec75802740 1 -- start start 2024-04-17T07:23:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.035+0000 7fec75802740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032faa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.035+0000 7fec75802740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032faf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.036+0000 7fec75802740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 0x561032fb1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.036+0000 7fec70c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032faa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.036+0000 7fec70c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032faa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43834/0 (socket says 172.21.15.86:43834) 2024-04-17T07:23:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.036+0000 7fec70c00640 1 -- 172.21.15.86:0/3948315539 learned_addr learned my addr 172.21.15.86:0/3948315539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.036+0000 7fec75802740 1 -- 172.21.15.86:0/3948315539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561032f10020 con 0x561032fafb00 2024-04-17T07:23:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.036+0000 7fec75802740 1 -- 172.21.15.86:0/3948315539 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561032eb6460 con 0x561032faa990 2024-04-17T07:23:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.036+0000 7fec75802740 1 -- 172.21.15.86:0/3948315539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561032f0bfe0 con 0x561032fa9fa0 2024-04-17T07:23:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.037+0000 7fec68c00640 1 -- 172.21.15.86:0/2191273693 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fec5c018020 con 0x7fec6c108f60 2024-04-17T07:23:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.038+0000 7fec70c00640 1 -- 172.21.15.86:0/3948315539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 msgr2=0x561032faf490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:23:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.038+0000 7fec70c00640 1 --2- 172.21.15.86:0/3948315539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032faf490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.038+0000 7fec70c00640 1 -- 172.21.15.86:0/3948315539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 msgr2=0x561032fb1f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:23:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.039+0000 7fec6b400640 1 --2- 172.21.15.86:0/3948315539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 0x561032fb1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.039+0000 7fec70c00640 1 --2- 172.21.15.86:0/3948315539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 0x561032fb1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.039+0000 7fec70c00640 1 -- 172.21.15.86:0/3948315539 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561032ebe040 con 0x561032fa9fa0 2024-04-17T07:23:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.039+0000 7fec68c00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec500a3cb0 0x7fec500a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.040+0000 7fec70c00640 1 --2- 172.21.15.86:0/3948315539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032faa3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec6c06bc80 tx=0x7fec6402cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c16a1481314d20d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.041+0000 7fec61200640 1 -- 172.21.15.86:0/3948315539 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64043070 con 0x561032fa9fa0 2024-04-17T07:23:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.042+0000 7fec61200640 1 -- 172.21.15.86:0/3948315539 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec640098a0 con 0x561032fa9fa0 2024-04-17T07:23:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.042+0000 7fec61200640 1 -- 172.21.15.86:0/3948315539 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64039040 con 0x561032fa9fa0 2024-04-17T07:23:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.043+0000 7fec6be00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec500a3cb0 0x7fec500a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.043+0000 7fec68c00640 1 -- 172.21.15.86:0/2191273693 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fec5c0d0100 con 0x7fec6c108f60 2024-04-17T07:23:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.043+0000 7fec75802740 1 -- 172.21.15.86:0/3948315539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 msgr2=0x561032faa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.043+0000 7fec75802740 1 --2- 172.21.15.86:0/3948315539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032faa3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fec6c06bc80 tx=0x7fec6402cd50 comp rx=0 tx=0).stop 2024-04-17T07:23:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.043+0000 7fec75802740 1 -- 172.21.15.86:0/3948315539 shutdown_connections 2024-04-17T07:23:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.043+0000 7fec75802740 1 --2- 172.21.15.86:0/3948315539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 0x561032fb1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.043+0000 7fec75802740 1 --2- 172.21.15.86:0/3948315539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032faf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.043+0000 7fec75802740 1 --2- 172.21.15.86:0/3948315539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032faa3c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.043+0000 7fec75802740 1 -- 172.21.15.86:0/3948315539 >> 172.21.15.86:0/3948315539 conn(0x561032fa5510 msgr2=0x561032fa5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.044+0000 7fec6be00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec500a3cb0 0x7fec500a6170 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009fd0 tx=0x7fec5c0092c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.044+0000 7fec75802740 1 -- 172.21.15.86:0/3948315539 shutdown_connections 2024-04-17T07:23:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.044+0000 7fec75802740 1 -- 172.21.15.86:0/3948315539 wait complete. 2024-04-17T07:23:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.045+0000 7fec75802740 1 -- start start 2024-04-17T07:23:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.045+0000 7fec75802740 1 Processor -- start 2024-04-17T07:23:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.046+0000 7fec75802740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032fb4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.046+0000 7fec75802740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032faa990 0x561032fb49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.046+0000 7fec75802740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032fafb00 0x561032fbba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.046+0000 7fec75802740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561032fb2a50 con 0x561032faa990 2024-04-17T07:23:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.046+0000 7fec75802740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561032ec9830 con 0x561032fafb00 2024-04-17T07:23:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.046+0000 7fec75802740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561032fb2bd0 con 0x561032fa9fa0 2024-04-17T07:23:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.046+0000 7fec75802740 10 client.0 _ll_register_callbacks cb 0x561032e97fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec70c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032fb4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec70c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032fb4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43838/0 (socket says 172.21.15.86:43838) 2024-04-17T07:23:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec70c00640 1 -- 172.21.15.86:0/1946666898 learned_addr learned my addr 172.21.15.86:0/1946666898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec70c00640 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032fafb00 0x561032fbba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec6b400640 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032faa990 0x561032fb49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec70c00640 1 -- 172.21.15.86:0/1946666898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032fafb00 msgr2=0x561032fbba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec70c00640 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032fafb00 0x561032fbba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec70c00640 1 -- 172.21.15.86:0/1946666898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032faa990 msgr2=0x561032fb49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec70c00640 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032faa990 0x561032fb49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec70c00640 1 -- 172.21.15.86:0/1946666898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561032ecd920 con 0x561032fa9fa0 2024-04-17T07:23:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.047+0000 7fec6b400640 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032faa990 0x561032fb49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.048+0000 7fec70c00640 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032fb4490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fec6402c660 tx=0x7fec6402e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=685cee4eb8d0c17c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.048+0000 7fec55600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.048+0000 7fec55600640 1 -- 172.21.15.86:0/1946666898 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64039070 con 0x561032fa9fa0 2024-04-17T07:23:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.048+0000 7fec75802740 1 -- 172.21.15.86:0/1946666898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec640098a0 con 0x561032fa9fa0 2024-04-17T07:23:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.048+0000 7fec55600640 1 -- 172.21.15.86:0/1946666898 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fec64004270 con 0x561032fa9fa0 2024-04-17T07:23:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.049+0000 7fec55600640 1 -- 172.21.15.86:0/1946666898 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec64043040 con 0x561032fa9fa0 2024-04-17T07:23:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.049+0000 7fec75802740 10 client.26803 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.049+0000 7fec75802740 1 -- 172.21.15.86:0/1946666898 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561032e9b7f0 con 0x561032fa9fa0 2024-04-17T07:23:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.050+0000 7fec55600640 1 -- 172.21.15.86:0/1946666898 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec640098a0 con 0x561032fa9fa0 2024-04-17T07:23:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.050+0000 7fec55600640 1 -- 172.21.15.86:0/1946666898 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fec64056550 con 0x561032fa9fa0 2024-04-17T07:23:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.051+0000 7fec75802740 10 client.26803 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.051+0000 7fec75802740 10 client.26803 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:23:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.051+0000 7fec75802740 20 client.26803 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.051+0000 7fec75802740 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561032fbfb50 0x561032fc1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.051+0000 7fec75802740 4 client.26803 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.051+0000 7fec75802740 1 -- 172.21.15.86:0/1946666898 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561032f1da40 con 0x561032fbfb50 2024-04-17T07:23:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.052+0000 7fec6b400640 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561032fbfb50 0x561032fc1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.053+0000 7fec6b400640 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561032fbfb50 0x561032fc1f80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69c1afa6aee0ae5a server_cookie=7c0c9eabd0d9a935 in_seq=0 out_seq=0 2024-04-17T07:23:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.053+0000 7fec55600640 10 client.26803 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:23:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.055+0000 7fec55600640 1 -- 172.21.15.86:0/1946666898 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561032f1da40 con 0x561032fbfb50 2024-04-17T07:23:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.055+0000 7fec55600640 10 client.26803 handle_command_reply: tid=0 2024-04-17T07:23:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.055+0000 7fec75802740 1 client.26803 shutdown 2024-04-17T07:23:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.056+0000 7fec75802740 1 -- 172.21.15.86:0/1946666898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 msgr2=0x561032fb4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.056+0000 7fec75802740 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032fb4490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fec6402c660 tx=0x7fec6402e040 comp rx=0 tx=0).stop 2024-04-17T07:23:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.056+0000 7fec75802740 1 -- 172.21.15.86:0/1946666898 shutdown_connections 2024-04-17T07:23:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.056+0000 7fec75802740 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561032fbfb50 0x561032fc1f80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.056+0000 7fec75802740 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032fafb00 0x561032fbba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.056+0000 7fec75802740 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032faa990 0x561032fb49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.056+0000 7fec75802740 1 --2- 172.21.15.86:0/1946666898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032fb4490 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.056+0000 7fec75802740 1 -- 172.21.15.86:0/1946666898 >> 172.21.15.86:0/1946666898 conn(0x561032fa5510 msgr2=0x561032fa4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.056+0000 7fec55600640 0 client.26803 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:23:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.056+0000 7fec75802740 1 -- 172.21.15.86:0/1946666898 shutdown_connections 2024-04-17T07:23:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.057+0000 7fec75802740 1 -- 172.21.15.86:0/1946666898 wait complete. 2024-04-17T07:23:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.057+0000 7fec75802740 20 client.26803 trim_cache size 0 max 16384 2024-04-17T07:23:49.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.083+0000 7fec75802740 1 Processor -- start 2024-04-17T07:23:49.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.083+0000 7fec75802740 1 -- start start 2024-04-17T07:23:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.083+0000 7fec75802740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032fa09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.083+0000 7fec75802740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032fbb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.083+0000 7fec75802740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 0x561032fb4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.083+0000 7fec75802740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c00fb60 con 0x561032fafb00 2024-04-17T07:23:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.083+0000 7fec75802740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c0116a0 con 0x561032faa990 2024-04-17T07:23:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.083+0000 7fec75802740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c010230 con 0x561032fa9fa0 2024-04-17T07:23:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.084+0000 7fec70c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 0x561032fb4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.084+0000 7fec70c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 0x561032fb4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33846/0 (socket says 172.21.15.86:33846) 2024-04-17T07:23:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.084+0000 7fec70c00640 1 -- 172.21.15.86:0/340344469 learned_addr learned my addr 172.21.15.86:0/340344469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.084+0000 7fec70c00640 1 --2- 172.21.15.86:0/340344469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032fa09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.084+0000 7fec6b400640 1 --2- 172.21.15.86:0/340344469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032fbb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.084+0000 7fec6b400640 1 -- 172.21.15.86:0/340344469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 msgr2=0x561032fa09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.084+0000 7fec6b400640 1 --2- 172.21.15.86:0/340344469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032fa09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.084+0000 7fec6b400640 1 -- 172.21.15.86:0/340344469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 msgr2=0x561032fb4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.084+0000 7fec6b400640 1 --2- 172.21.15.86:0/340344469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 0x561032fb4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.084+0000 7fec6b400640 1 -- 172.21.15.86:0/340344469 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561032c223f0 con 0x561032faa990 2024-04-17T07:23:49.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.085+0000 7fec6b400640 1 --2- 172.21.15.86:0/340344469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032fbb9d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x561032fc27f0 tx=0x7fec58005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80052e48283087ac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.086+0000 7fec57e00640 1 -- 172.21.15.86:0/340344469 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58005550 con 0x561032faa990 2024-04-17T07:23:49.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.086+0000 7fec57e00640 1 -- 172.21.15.86:0/340344469 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec5801b830 con 0x561032faa990 2024-04-17T07:23:49.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.086+0000 7fec57e00640 1 -- 172.21.15.86:0/340344469 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58012450 con 0x561032faa990 2024-04-17T07:23:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.087+0000 7fec75802740 1 -- 172.21.15.86:0/340344469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 msgr2=0x561032fbb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.087+0000 7fec75802740 1 --2- 172.21.15.86:0/340344469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032fbb9d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x561032fc27f0 tx=0x7fec58005f90 comp rx=0 tx=0).stop 2024-04-17T07:23:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.088+0000 7fec75802740 1 -- 172.21.15.86:0/340344469 shutdown_connections 2024-04-17T07:23:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.088+0000 7fec75802740 1 --2- 172.21.15.86:0/340344469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fafb00 0x561032fb4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.088+0000 7fec75802740 1 --2- 172.21.15.86:0/340344469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032fbb9d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.088+0000 7fec75802740 1 --2- 172.21.15.86:0/340344469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fa9fa0 0x561032fa09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.088+0000 7fec75802740 1 -- 172.21.15.86:0/340344469 >> 172.21.15.86:0/340344469 conn(0x561032fa5510 msgr2=0x561032fa8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.088+0000 7fec75802740 1 -- 172.21.15.86:0/340344469 shutdown_connections 2024-04-17T07:23:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.088+0000 7fec75802740 1 -- 172.21.15.86:0/340344469 wait complete. 2024-04-17T07:23:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.089+0000 7fec75802740 1 -- start start 2024-04-17T07:23:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.090+0000 7fec75802740 1 Processor -- start 2024-04-17T07:23:49.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.090+0000 7fec75802740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fa9fa0 0x561032fb8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.091+0000 7fec75802740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032fb8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.091+0000 7fec75802740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fafb00 0x5610330240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.091+0000 7fec75802740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c010fd0 con 0x561032fa9fa0 2024-04-17T07:23:49.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.091+0000 7fec75802740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c010900 con 0x561032faa990 2024-04-17T07:23:49.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.091+0000 7fec75802740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c00db30 con 0x561032fafb00 2024-04-17T07:23:49.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.091+0000 7fec75802740 10 client.0 _ll_register_callbacks cb 0x7fec640329d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:49.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.091+0000 7fec70c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fa9fa0 0x561032fb8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.092+0000 7fec6b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032fb8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.092+0000 7fec70c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fa9fa0 0x561032fb8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33862/0 (socket says 172.21.15.86:33862) 2024-04-17T07:23:49.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.092+0000 7fec70c00640 1 -- 172.21.15.86:0/1288467174 learned_addr learned my addr 172.21.15.86:0/1288467174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:49.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.093+0000 7fec70c00640 1 -- 172.21.15.86:0/1288467174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fafb00 msgr2=0x5610330240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.093+0000 7fec70c00640 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fafb00 0x5610330240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.093+0000 7fec70c00640 1 -- 172.21.15.86:0/1288467174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 msgr2=0x561032fb8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.093+0000 7fec70c00640 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032fb8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.093+0000 7fec70c00640 1 -- 172.21.15.86:0/1288467174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561032cec7d0 con 0x561032fa9fa0 2024-04-17T07:23:49.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.093+0000 7fec55600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:23:49.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.094+0000 7fec70c00640 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fa9fa0 0x561032fb8a00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fec64056a40 tx=0x7fec64000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a997b649a84725 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.094+0000 7fec55600640 1 -- 172.21.15.86:0/1288467174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64065060 con 0x561032fa9fa0 2024-04-17T07:23:49.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.094+0000 7fec55600640 1 -- 172.21.15.86:0/1288467174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fec64056450 con 0x561032fa9fa0 2024-04-17T07:23:49.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.095+0000 7fec75802740 1 -- 172.21.15.86:0/1288467174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5801b830 con 0x561032fa9fa0 2024-04-17T07:23:49.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.095+0000 7fec55600640 1 -- 172.21.15.86:0/1288467174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec64039040 con 0x561032fa9fa0 2024-04-17T07:23:49.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.095+0000 7fec75802740 10 client.17847 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:49.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.095+0000 7fec75802740 1 -- 172.21.15.86:0/1288467174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610330257a0 con 0x561032fa9fa0 2024-04-17T07:23:49.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.096+0000 7fec55600640 1 -- 172.21.15.86:0/1288467174 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec5801b830 con 0x561032fa9fa0 2024-04-17T07:23:49.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.096+0000 7fec55600640 1 -- 172.21.15.86:0/1288467174 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fec64060620 con 0x561032fa9fa0 2024-04-17T07:23:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.096+0000 7fec75802740 10 client.17847 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.097+0000 7fec75802740 10 client.17847 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:23:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.097+0000 7fec75802740 20 client.17847 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.097+0000 7fec75802740 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610330260a0 0x5610330284c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.097+0000 7fec75802740 4 client.17847 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:23:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.097+0000 7fec75802740 1 -- 172.21.15.86:0/1288467174 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561032f178b0 con 0x5610330260a0 2024-04-17T07:23:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.097+0000 7fec6b400640 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610330260a0 0x5610330284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.098+0000 7fec6b400640 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610330260a0 0x5610330284c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1457326a802af3e server_cookie=1dc8b87b526ce667 in_seq=0 out_seq=0 2024-04-17T07:23:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.098+0000 7fec55600640 10 client.17847 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:23:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.099+0000 7fec55600640 1 -- 172.21.15.86:0/1288467174 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561032f178b0 con 0x5610330260a0 2024-04-17T07:23:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.099+0000 7fec55600640 10 client.17847 handle_command_reply: tid=0 2024-04-17T07:23:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.099+0000 7fec75802740 1 client.17847 shutdown 2024-04-17T07:23:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.101+0000 7fec75802740 1 -- 172.21.15.86:0/1288467174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fa9fa0 msgr2=0x561032fb8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.101+0000 7fec75802740 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fa9fa0 0x561032fb8a00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fec64056a40 tx=0x7fec64000f40 comp rx=0 tx=0).stop 2024-04-17T07:23:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.102+0000 7fec75802740 1 -- 172.21.15.86:0/1288467174 shutdown_connections 2024-04-17T07:23:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.102+0000 7fec75802740 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561032fafb00 0x5610330240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.102+0000 7fec75802740 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610330260a0 0x5610330284c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.102+0000 7fec75802740 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561032faa990 0x561032fb8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.102+0000 7fec75802740 1 --2- 172.21.15.86:0/1288467174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561032fa9fa0 0x561032fb8a00 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.102+0000 7fec55600640 0 client.17847 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:23:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.102+0000 7fec75802740 1 -- 172.21.15.86:0/1288467174 >> 172.21.15.86:0/1288467174 conn(0x561032fa5510 msgr2=0x561032fa9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.102+0000 7fec75802740 1 -- 172.21.15.86:0/1288467174 shutdown_connections 2024-04-17T07:23:49.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.102+0000 7fec75802740 1 -- 172.21.15.86:0/1288467174 wait complete. 2024-04-17T07:23:49.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.104+0000 7fec75802740 20 client.17847 trim_cache size 0 max 16384 2024-04-17T07:23:49.104 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:23:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.105+0000 7fec72c00640 1 -- 172.21.15.86:0/2191273693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec500a3cb0 msgr2=0x7fec500a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.105+0000 7fec72c00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec500a3cb0 0x7fec500a6170 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009fd0 tx=0x7fec5c0092c0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.105+0000 7fec72c00640 1 -- 172.21.15.86:0/2191273693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c108f60 msgr2=0x7fec6c1a7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.105+0000 7fec72c00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c108f60 0x7fec6c1a7050 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00b450 tx=0x7fec5c00b910 comp rx=0 tx=0).stop 2024-04-17T07:23:49.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.107+0000 7fec72c00640 1 -- 172.21.15.86:0/2191273693 shutdown_connections 2024-04-17T07:23:49.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.107+0000 7fec72c00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec500a3cb0 0x7fec500a6170 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.107+0000 7fec72c00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec6c10e2a0 0x7fec6c1ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.107+0000 7fec72c00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6c10b8d0 0x7fec6c1a7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.107+0000 7fec72c00640 1 --2- 172.21.15.86:0/2191273693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6c108f60 0x7fec6c1a7050 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.107+0000 7fec72c00640 1 -- 172.21.15.86:0/2191273693 >> 172.21.15.86:0/2191273693 conn(0x7fec6c06edf0 msgr2=0x7fec6c109380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.107+0000 7fec72c00640 1 -- 172.21.15.86:0/2191273693 shutdown_connections 2024-04-17T07:23:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.108+0000 7fec72c00640 1 -- 172.21.15.86:0/2191273693 wait complete. 2024-04-17T07:23:49.143 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:23:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 265.0 s, 629.2 tps, lat 159.015 ms stddev 47.591 2024-04-17T07:23:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.447+0000 7f9702200640 1 -- 172.21.15.86:0/1171663436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96fc100820 msgr2=0x7f96fc100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.447+0000 7f9702200640 1 --2- 172.21.15.86:0/1171663436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96fc100820 0x7f96fc100c40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009860 tx=0x7f96ec02bb90 comp rx=0 tx=0).stop 2024-04-17T07:23:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.448+0000 7f9702200640 1 -- 172.21.15.86:0/1171663436 shutdown_connections 2024-04-17T07:23:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.448+0000 7f9702200640 1 --2- 172.21.15.86:0/1171663436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96fc101b20 0x7f96fc101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.448+0000 7f9702200640 1 --2- 172.21.15.86:0/1171663436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96fc101180 0x7f96fc1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.448+0000 7f9702200640 1 --2- 172.21.15.86:0/1171663436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96fc100820 0x7f96fc100c40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.448+0000 7f9702200640 1 -- 172.21.15.86:0/1171663436 >> 172.21.15.86:0/1171663436 conn(0x7f96fc0fc420 msgr2=0x7f96fc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.448+0000 7f9702200640 1 -- 172.21.15.86:0/1171663436 shutdown_connections 2024-04-17T07:23:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.448+0000 7f9702200640 1 -- 172.21.15.86:0/1171663436 wait complete. 2024-04-17T07:23:49.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.450+0000 7f9702200640 1 Processor -- start 2024-04-17T07:23:49.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.450+0000 7f9702200640 1 -- start start 2024-04-17T07:23:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.451+0000 7f9702200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96fc100820 0x7f96fc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.452+0000 7f96fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96fc100820 0x7f96fc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.452+0000 7f9702200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96fc101180 0x7f96fc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.452+0000 7f9702200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96fc101b20 0x7f96fc1a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.452+0000 7f9702200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc10fd00 con 0x7f96fc100820 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.452+0000 7f9702200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc10fb80 con 0x7f96fc101b20 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.452+0000 7f9702200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc10fe80 con 0x7f96fc101180 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.452+0000 7f96fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96fc100820 0x7f96fc19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33882/0 (socket says 172.21.15.86:33882) 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.452+0000 7f96fb400640 1 -- 172.21.15.86:0/2662988182 learned_addr learned my addr 172.21.15.86:0/2662988182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.452+0000 7f96fbe00640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96fc101b20 0x7f96fc1a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.452+0000 7f96faa00640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96fc101180 0x7f96fc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.453+0000 7f96fb400640 1 -- 172.21.15.86:0/2662988182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96fc101180 msgr2=0x7f96fc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.453+0000 7f96fb400640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96fc101180 0x7f96fc19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.453+0000 7f96fb400640 1 -- 172.21.15.86:0/2662988182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96fc101b20 msgr2=0x7f96fc1a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.453+0000 7f96fb400640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96fc101b20 0x7f96fc1a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.453+0000 7f96fb400640 1 -- 172.21.15.86:0/2662988182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc1a2060 con 0x7f96fc100820 2024-04-17T07:23:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.453+0000 7f96fb400640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96fc100820 0x7f96fc19a440 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f96ec037040 tx=0x7f96ec0348e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.454+0000 7f96f3e00640 1 -- 172.21.15.86:0/2662988182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec041020 con 0x7f96fc100820 2024-04-17T07:23:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.454+0000 7f96f3e00640 1 -- 172.21.15.86:0/2662988182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96ec02c9e0 con 0x7f96fc100820 2024-04-17T07:23:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.454+0000 7f96f3e00640 1 -- 172.21.15.86:0/2662988182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec039040 con 0x7f96fc100820 2024-04-17T07:23:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.454+0000 7f9702200640 1 -- 172.21.15.86:0/2662988182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc102a30 con 0x7f96fc100820 2024-04-17T07:23:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.454+0000 7f9702200640 1 -- 172.21.15.86:0/2662988182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc102ed0 con 0x7f96fc100820 2024-04-17T07:23:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.459+0000 7f9704f81740 1 Processor -- start 2024-04-17T07:23:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.460+0000 7f9704f81740 1 -- start start 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.460+0000 7f96f3e00640 1 -- 172.21.15.86:0/2662988182 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f96ec044020 con 0x7f96fc100820 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.460+0000 7f9704f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7bfa0 0x560dbef7c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.460+0000 7f9704f81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7c990 0x560dbef81490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.460+0000 7f9704f81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 0x560dbef83f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.460+0000 7f9704f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560dbeee2020 con 0x560dbef7bfa0 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.460+0000 7f9704f81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560dbee88460 con 0x560dbef81b00 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.460+0000 7f9704f81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560dbeeddfe0 con 0x560dbef7c990 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.461+0000 7f96fb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7c990 0x560dbef81490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.461+0000 7f96fb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7c990 0x560dbef81490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43894/0 (socket says 172.21.15.86:43894) 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.461+0000 7f96fb400640 1 -- 172.21.15.86:0/726588748 learned_addr learned my addr 172.21.15.86:0/726588748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.461+0000 7f96faa00640 1 --2- 172.21.15.86:0/726588748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 0x560dbef83f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.461+0000 7f96fb400640 1 -- 172.21.15.86:0/726588748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 msgr2=0x560dbef83f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.461+0000 7f96fb400640 1 --2- 172.21.15.86:0/726588748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 0x560dbef83f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.461+0000 7f96fb400640 1 -- 172.21.15.86:0/726588748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7bfa0 msgr2=0x560dbef7c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.461+0000 7f96fb400640 1 --2- 172.21.15.86:0/726588748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7bfa0 0x560dbef7c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.461+0000 7f96fb400640 1 -- 172.21.15.86:0/726588748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbee90040 con 0x560dbef7c990 2024-04-17T07:23:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.462+0000 7f96fb400640 1 --2- 172.21.15.86:0/726588748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7c990 0x560dbef81490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0f5f90 tx=0x7f96ec0fa1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1257149bdcf14d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.462+0000 7f96f3e00640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96e00a3d00 0x7f96e00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.462+0000 7f96e7e00640 1 -- 172.21.15.86:0/726588748 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0ba220 con 0x560dbef7c990 2024-04-17T07:23:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.462+0000 7f96e7e00640 1 -- 172.21.15.86:0/726588748 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96ec0ff030 con 0x560dbef7c990 2024-04-17T07:23:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.463+0000 7f96e7e00640 1 -- 172.21.15.86:0/726588748 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0bd3d0 con 0x560dbef7c990 2024-04-17T07:23:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.463+0000 7f9704f81740 1 -- 172.21.15.86:0/726588748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7c990 msgr2=0x560dbef81490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.463+0000 7f9704f81740 1 --2- 172.21.15.86:0/726588748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7c990 0x560dbef81490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0f5f90 tx=0x7f96ec0fa1a0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.464+0000 7f96fbe00640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96e00a3d00 0x7f96e00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.464+0000 7f96f3e00640 1 -- 172.21.15.86:0/2662988182 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f96ec0f2080 con 0x7f96fc100820 2024-04-17T07:23:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.464+0000 7f9704f81740 1 -- 172.21.15.86:0/726588748 shutdown_connections 2024-04-17T07:23:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.464+0000 7f9704f81740 1 --2- 172.21.15.86:0/726588748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 0x560dbef83f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.465+0000 7f9704f81740 1 --2- 172.21.15.86:0/726588748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7c990 0x560dbef81490 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.465+0000 7f9704f81740 1 --2- 172.21.15.86:0/726588748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7bfa0 0x560dbef7c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.465+0000 7f9704f81740 1 -- 172.21.15.86:0/726588748 >> 172.21.15.86:0/726588748 conn(0x560dbef77510 msgr2=0x560dbef77910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.466+0000 7f96fbe00640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96e00a3d00 0x7f96e00a61c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x560dbef7d1f0 tx=0x7f96f4018040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.466+0000 7f9704f81740 1 -- 172.21.15.86:0/726588748 shutdown_connections 2024-04-17T07:23:49.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.467+0000 7f9704f81740 1 -- 172.21.15.86:0/726588748 wait complete. 2024-04-17T07:23:49.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.468+0000 7f9704f81740 1 -- start start 2024-04-17T07:23:49.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.470+0000 7f9704f81740 1 Processor -- start 2024-04-17T07:23:49.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.471+0000 7f9704f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7bfa0 0x560dbef86180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.471+0000 7f9704f81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef7c990 0x560dbef866c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.471+0000 7f9704f81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 0x560dbef8d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.471+0000 7f9704f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560dbef84870 con 0x560dbef7bfa0 2024-04-17T07:23:49.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.471+0000 7f9704f81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560dbee9b830 con 0x560dbef7c990 2024-04-17T07:23:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.471+0000 7f9704f81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560dbef849f0 con 0x560dbef81b00 2024-04-17T07:23:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.471+0000 7f9704f81740 10 client.0 _ll_register_callbacks cb 0x560dbee69fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.471+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 0x560dbef8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.472+0000 7f96fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7bfa0 0x560dbef86180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.472+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 0x560dbef8d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43900/0 (socket says 172.21.15.86:43900) 2024-04-17T07:23:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.472+0000 7f96fbe00640 1 -- 172.21.15.86:0/2749252556 learned_addr learned my addr 172.21.15.86:0/2749252556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:49.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.472+0000 7f96faa00640 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef7c990 0x560dbef866c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.472+0000 7f96fbe00640 1 -- 172.21.15.86:0/2749252556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef7c990 msgr2=0x560dbef866c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.472+0000 7f96fbe00640 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef7c990 0x560dbef866c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.472+0000 7f96fbe00640 1 -- 172.21.15.86:0/2749252556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7bfa0 msgr2=0x560dbef86180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.472+0000 7f96fbe00640 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7bfa0 0x560dbef86180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.472+0000 7f96fbe00640 1 -- 172.21.15.86:0/2749252556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbee9f920 con 0x560dbef81b00 2024-04-17T07:23:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.473+0000 7f96fb400640 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7bfa0 0x560dbef86180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.473+0000 7f96fbe00640 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 0x560dbef8d700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f96f40226a0 tx=0x7f96f4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=511d834e0cba3859 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.475+0000 7f96e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.475+0000 7f96e4c00640 1 -- 172.21.15.86:0/2749252556 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4032060 con 0x560dbef81b00 2024-04-17T07:23:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.475+0000 7f96e4c00640 1 -- 172.21.15.86:0/2749252556 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f96f402d420 con 0x560dbef81b00 2024-04-17T07:23:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.475+0000 7f9704f81740 1 -- 172.21.15.86:0/2749252556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96ec0ff030 con 0x560dbef81b00 2024-04-17T07:23:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.475+0000 7f96e4c00640 1 -- 172.21.15.86:0/2749252556 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f96f4039050 con 0x560dbef81b00 2024-04-17T07:23:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.476+0000 7f9704f81740 10 client.26815 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.476+0000 7f9704f81740 1 -- 172.21.15.86:0/2749252556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560dbee6d7f0 con 0x560dbef81b00 2024-04-17T07:23:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.476+0000 7f96e4c00640 1 -- 172.21.15.86:0/2749252556 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96ec0ff030 con 0x560dbef81b00 2024-04-17T07:23:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.477+0000 7f96e4c00640 1 -- 172.21.15.86:0/2749252556 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f96f40204b0 con 0x560dbef81b00 2024-04-17T07:23:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.477+0000 7f9704f81740 10 client.26815 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.477+0000 7f9704f81740 10 client.26815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.477+0000 7f9704f81740 20 client.26815 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.477+0000 7f9704f81740 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560dbef91c70 0x560dbef940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.477+0000 7f9704f81740 4 client.26815 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.477+0000 7f9704f81740 1 -- 172.21.15.86:0/2749252556 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dbeeefa40 con 0x560dbef91c70 2024-04-17T07:23:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.478+0000 7f96fb400640 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560dbef91c70 0x560dbef940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.479+0000 7f96fb400640 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560dbef91c70 0x560dbef940f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a53a11307b39c1 server_cookie=590155806924da8a in_seq=0 out_seq=0 2024-04-17T07:23:49.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.479+0000 7f96e4c00640 10 client.26815 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:49.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.480+0000 7f96e4c00640 1 -- 172.21.15.86:0/2749252556 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560dbeeefa40 con 0x560dbef91c70 2024-04-17T07:23:49.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.480+0000 7f96e4c00640 10 client.26815 handle_command_reply: tid=0 2024-04-17T07:23:49.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.481+0000 7f9704f81740 1 client.26815 shutdown 2024-04-17T07:23:49.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.481+0000 7f9704f81740 1 -- 172.21.15.86:0/2749252556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 msgr2=0x560dbef8d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.481+0000 7f9704f81740 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 0x560dbef8d700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f96f40226a0 tx=0x7f96f4004ab0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.481+0000 7f9704f81740 1 -- 172.21.15.86:0/2749252556 shutdown_connections 2024-04-17T07:23:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.481+0000 7f9704f81740 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560dbef91c70 0x560dbef940f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.481+0000 7f9704f81740 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 0x560dbef8d700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.481+0000 7f9704f81740 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef7c990 0x560dbef866c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.481+0000 7f9704f81740 1 --2- 172.21.15.86:0/2749252556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7bfa0 0x560dbef86180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.481+0000 7f9704f81740 1 -- 172.21.15.86:0/2749252556 >> 172.21.15.86:0/2749252556 conn(0x560dbef77510 msgr2=0x560dbef76710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.482+0000 7f96e4c00640 0 client.26815 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.482+0000 7f9704f81740 1 -- 172.21.15.86:0/2749252556 shutdown_connections 2024-04-17T07:23:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.482+0000 7f9704f81740 1 -- 172.21.15.86:0/2749252556 wait complete. 2024-04-17T07:23:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.482+0000 7f9704f81740 20 client.26815 trim_cache size 0 max 16384 2024-04-17T07:23:49.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:49 smithi098 ceph-mon[35418]: pgmap v723: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 777 op/s 2024-04-17T07:23:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.507+0000 7f9704f81740 1 Processor -- start 2024-04-17T07:23:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.507+0000 7f9704f81740 1 -- start start 2024-04-17T07:23:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.507+0000 7f9704f81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7bfa0 0x560dbef778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.507+0000 7f9704f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.507+0000 7f9704f81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 0x560dbef75e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.508+0000 7f9704f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a89c0 con 0x560dbef7c990 2024-04-17T07:23:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.508+0000 7f9704f81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1aa500 con 0x560dbef81b00 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.508+0000 7f9704f81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a9090 con 0x560dbef7bfa0 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.508+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 0x560dbef75e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.508+0000 7f96faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.508+0000 7f96faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33912/0 (socket says 172.21.15.86:33912) 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.508+0000 7f96faa00640 1 -- 172.21.15.86:0/2172956319 learned_addr learned my addr 172.21.15.86:0/2172956319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.508+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 0x560dbef75e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52426/0 (socket says 172.21.15.86:52426) 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.508+0000 7f96fb400640 1 --2- 172.21.15.86:0/2172956319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7bfa0 0x560dbef778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.509+0000 7f96faa00640 1 -- 172.21.15.86:0/2172956319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7bfa0 msgr2=0x560dbef778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.509+0000 7f96faa00640 1 --2- 172.21.15.86:0/2172956319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7bfa0 0x560dbef778f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.509+0000 7f96faa00640 1 -- 172.21.15.86:0/2172956319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 msgr2=0x560dbef75e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.509+0000 7f96faa00640 1 --2- 172.21.15.86:0/2172956319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 0x560dbef75e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.509+0000 7f96faa00640 1 -- 172.21.15.86:0/2172956319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dbebf43f0 con 0x560dbef7c990 2024-04-17T07:23:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.509+0000 7f96fbe00640 1 --2- 172.21.15.86:0/2172956319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 0x560dbef75e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:23:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.509+0000 7f96faa00640 1 --2- 172.21.15.86:0/2172956319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8d6c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009ac0 tx=0x7f96e8009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13746f3f3495fc77 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.509+0000 7f96e7400640 1 -- 172.21.15.86:0/2172956319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8013ce0 con 0x560dbef7c990 2024-04-17T07:23:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.510+0000 7f96e7400640 1 -- 172.21.15.86:0/2172956319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96e800f900 con 0x560dbef7c990 2024-04-17T07:23:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.510+0000 7f96e7400640 1 -- 172.21.15.86:0/2172956319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8013ce0 con 0x560dbef7c990 2024-04-17T07:23:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.511+0000 7f9704f81740 1 -- 172.21.15.86:0/2172956319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 msgr2=0x560dbef8d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.511+0000 7f9704f81740 1 --2- 172.21.15.86:0/2172956319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8d6c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009ac0 tx=0x7f96e8009ec0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.511+0000 7f9704f81740 1 -- 172.21.15.86:0/2172956319 shutdown_connections 2024-04-17T07:23:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.511+0000 7f9704f81740 1 --2- 172.21.15.86:0/2172956319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef81b00 0x560dbef75e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.511+0000 7f9704f81740 1 --2- 172.21.15.86:0/2172956319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8d6c0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.511+0000 7f9704f81740 1 --2- 172.21.15.86:0/2172956319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef7bfa0 0x560dbef778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.511+0000 7f9704f81740 1 -- 172.21.15.86:0/2172956319 >> 172.21.15.86:0/2172956319 conn(0x560dbef77510 msgr2=0x560dbef727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.511+0000 7f9704f81740 1 -- 172.21.15.86:0/2172956319 shutdown_connections 2024-04-17T07:23:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.511+0000 7f9704f81740 1 -- 172.21.15.86:0/2172956319 wait complete. 2024-04-17T07:23:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.512+0000 7f9704f81740 1 -- start start 2024-04-17T07:23:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.512+0000 7f9704f81740 1 Processor -- start 2024-04-17T07:23:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.512+0000 7f9704f81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef7bfa0 0x560dbef8a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.513+0000 7f9704f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.513+0000 7f9704f81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 0x560dbef8bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.513+0000 7f9704f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a9e30 con 0x560dbef7c990 2024-04-17T07:23:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.513+0000 7f9704f81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a9760 con 0x560dbef7bfa0 2024-04-17T07:23:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.513+0000 7f9704f81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1a69b0 con 0x560dbef81b00 2024-04-17T07:23:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.513+0000 7f9704f81740 10 client.0 _ll_register_callbacks cb 0x7f96f40168b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.513+0000 7f96faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.513+0000 7f96faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33914/0 (socket says 172.21.15.86:33914) 2024-04-17T07:23:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.513+0000 7f96faa00640 1 -- 172.21.15.86:0/4198875772 learned_addr learned my addr 172.21.15.86:0/4198875772 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.514+0000 7f96fb400640 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef7bfa0 0x560dbef8a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.514+0000 7f96faa00640 1 -- 172.21.15.86:0/4198875772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 msgr2=0x560dbef8bf30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:23:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.514+0000 7f96faa00640 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 0x560dbef8bf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.514+0000 7f96faa00640 1 -- 172.21.15.86:0/4198875772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef7bfa0 msgr2=0x560dbef8a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.514+0000 7f96faa00640 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef7bfa0 0x560dbef8a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.514+0000 7f96faa00640 1 -- 172.21.15.86:0/4198875772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dbecbe7d0 con 0x560dbef7c990 2024-04-17T07:23:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.514+0000 7f96faa00640 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8af20 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f96e800fd10 tx=0x7f96e800fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e4ac0b62276d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.514+0000 7f96e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:23:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.514+0000 7f96e4c00640 1 -- 172.21.15.86:0/4198875772 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8021040 con 0x560dbef7c990 2024-04-17T07:23:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.515+0000 7f96e4c00640 1 -- 172.21.15.86:0/4198875772 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f96e8005240 con 0x560dbef7c990 2024-04-17T07:23:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.515+0000 7f9704f81740 1 -- 172.21.15.86:0/4198875772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e800f900 con 0x560dbef7c990 2024-04-17T07:23:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.515+0000 7f96e4c00640 1 -- 172.21.15.86:0/4198875772 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f96e8035050 con 0x560dbef7c990 2024-04-17T07:23:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.515+0000 7f9704f81740 10 client.17862 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.515+0000 7f9704f81740 1 -- 172.21.15.86:0/4198875772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560dbef8d680 con 0x560dbef7c990 2024-04-17T07:23:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.515+0000 7f96e4c00640 1 -- 172.21.15.86:0/4198875772 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96e800f900 con 0x560dbef7c990 2024-04-17T07:23:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.516+0000 7f96e4c00640 1 -- 172.21.15.86:0/4198875772 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f96e8024030 con 0x560dbef7c990 2024-04-17T07:23:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.516+0000 7f9704f81740 10 client.17862 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.516+0000 7f9704f81740 10 client.17862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.516+0000 7f9704f81740 20 client.17862 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:49.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.516+0000 7f9704f81740 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560dbeff85f0 0x560dbeffaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:49.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.516+0000 7f9704f81740 4 client.17862 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:23:49.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.516+0000 7f9704f81740 1 -- 172.21.15.86:0/4198875772 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560dbeee98b0 con 0x560dbeff85f0 2024-04-17T07:23:49.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.517+0000 7f96fb400640 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560dbeff85f0 0x560dbeffaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.519+0000 7f96e4c00640 10 client.17862 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.519+0000 7f96fb400640 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560dbeff85f0 0x560dbeffaa10 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c19429f0c670a9f server_cookie=2b1f61d88624c19d in_seq=0 out_seq=0 2024-04-17T07:23:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.519+0000 7f96e4c00640 1 -- 172.21.15.86:0/4198875772 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560dbeee98b0 con 0x560dbeff85f0 2024-04-17T07:23:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.519+0000 7f96e4c00640 10 client.17862 handle_command_reply: tid=0 2024-04-17T07:23:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.519+0000 7f9704f81740 1 client.17862 shutdown 2024-04-17T07:23:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.520+0000 7f9704f81740 1 -- 172.21.15.86:0/4198875772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 msgr2=0x560dbef8af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.520+0000 7f9704f81740 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8af20 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f96e800fd10 tx=0x7f96e800fd40 comp rx=0 tx=0).stop 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.520+0000 7f9704f81740 1 -- 172.21.15.86:0/4198875772 shutdown_connections 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.520+0000 7f9704f81740 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560dbeff85f0 0x560dbeffaa10 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.520+0000 7f9704f81740 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dbef81b00 0x560dbef8bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.520+0000 7f9704f81740 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560dbef7c990 0x560dbef8af20 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.520+0000 7f9704f81740 1 --2- 172.21.15.86:0/4198875772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560dbef7bfa0 0x560dbef8a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.520+0000 7f9704f81740 1 -- 172.21.15.86:0/4198875772 >> 172.21.15.86:0/4198875772 conn(0x560dbef77510 msgr2=0x560dbef778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.520+0000 7f96e4c00640 0 client.17862 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.521+0000 7f9704f81740 1 -- 172.21.15.86:0/4198875772 shutdown_connections 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.521+0000 7f9704f81740 1 -- 172.21.15.86:0/4198875772 wait complete. 2024-04-17T07:23:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.521+0000 7f9704f81740 20 client.17862 trim_cache size 0 max 16384 2024-04-17T07:23:49.522 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:23:49.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.522+0000 7f9702200640 1 -- 172.21.15.86:0/2662988182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96e00a3d00 msgr2=0x7f96e00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.522+0000 7f9702200640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96e00a3d00 0x7f96e00a61c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x560dbef7d1f0 tx=0x7f96f4018040 comp rx=0 tx=0).stop 2024-04-17T07:23:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.522+0000 7f9702200640 1 -- 172.21.15.86:0/2662988182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96fc100820 msgr2=0x7f96fc19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.522+0000 7f9702200640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96fc100820 0x7f96fc19a440 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f96ec037040 tx=0x7f96ec0348e0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.523+0000 7f9702200640 1 -- 172.21.15.86:0/2662988182 shutdown_connections 2024-04-17T07:23:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.523+0000 7f9702200640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96e00a3d00 0x7f96e00a61c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.523+0000 7f9702200640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96fc101b20 0x7f96fc1a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.523+0000 7f9702200640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96fc101180 0x7f96fc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.523+0000 7f9702200640 1 --2- 172.21.15.86:0/2662988182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96fc100820 0x7f96fc19a440 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.523+0000 7f9702200640 1 -- 172.21.15.86:0/2662988182 >> 172.21.15.86:0/2662988182 conn(0x7f96fc0fc420 msgr2=0x7f96fc0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:49.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:49 smithi103 ceph-mon[35495]: pgmap v723: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 777 op/s 2024-04-17T07:23:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.523+0000 7f9702200640 1 -- 172.21.15.86:0/2662988182 shutdown_connections 2024-04-17T07:23:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:49.523+0000 7f9702200640 1 -- 172.21.15.86:0/2662988182 wait complete. 2024-04-17T07:23:49.546 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:49 smithi086 ceph-mon[31733]: pgmap v723: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.6 MiB/s wr, 777 op/s 2024-04-17T07:23:50.551 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:23:50.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:23:50.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:23:50.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.866+0000 7f2e8f200640 1 -- 172.21.15.86:0/2386032313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e88109140 msgr2=0x7f2e8810b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:50.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.866+0000 7f2e8f200640 1 --2- 172.21.15.86:0/2386032313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e88109140 0x7f2e8810b530 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0074e0 tx=0x7f2e7c02c8b0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.866+0000 7f2e8f200640 1 -- 172.21.15.86:0/2386032313 shutdown_connections 2024-04-17T07:23:50.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.866+0000 7f2e8f200640 1 --2- 172.21.15.86:0/2386032313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e8810e510 0x7f2e881109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.866+0000 7f2e8f200640 1 --2- 172.21.15.86:0/2386032313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e8810ba70 0x7f2e8810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.866+0000 7f2e8f200640 1 --2- 172.21.15.86:0/2386032313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e88109140 0x7f2e8810b530 secure :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0074e0 tx=0x7f2e7c02c8b0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.866+0000 7f2e8f200640 1 -- 172.21.15.86:0/2386032313 >> 172.21.15.86:0/2386032313 conn(0x7f2e8806f060 msgr2=0x7f2e88071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:50.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.866+0000 7f2e8f200640 1 -- 172.21.15.86:0/2386032313 shutdown_connections 2024-04-17T07:23:50.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.866+0000 7f2e8f200640 1 -- 172.21.15.86:0/2386032313 wait complete. 2024-04-17T07:23:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.868+0000 7f2e8f200640 1 Processor -- start 2024-04-17T07:23:50.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.868+0000 7f2e8f200640 1 -- start start 2024-04-17T07:23:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.869+0000 7f2e8f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e8810ba70 0x7f2e8819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.869+0000 7f2e8f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8810e510 0x7f2e8819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.869+0000 7f2e8f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e881a3f10 0x7f2e881a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.870+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8810e510 0x7f2e8819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.870+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8810e510 0x7f2e8819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33930/0 (socket says 172.21.15.86:33930) 2024-04-17T07:23:50.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.870+0000 7f2e8d400640 1 -- 172.21.15.86:0/94076113 learned_addr learned my addr 172.21.15.86:0/94076113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:50.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.870+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88114060 con 0x7f2e8810e510 2024-04-17T07:23:50.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.870+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88113ee0 con 0x7f2e881a3f10 2024-04-17T07:23:50.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.870+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881141e0 con 0x7f2e8810ba70 2024-04-17T07:23:50.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.870+0000 7f2e8e800640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e881a3f10 0x7f2e881a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.870+0000 7f2e8de00640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e8810ba70 0x7f2e8819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.871+0000 7f2e8e800640 1 -- 172.21.15.86:0/94076113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e8810ba70 msgr2=0x7f2e8819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:50.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.871+0000 7f2e8e800640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e8810ba70 0x7f2e8819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.871+0000 7f2e8e800640 1 -- 172.21.15.86:0/94076113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8810e510 msgr2=0x7f2e8819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:50.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.871+0000 7f2e8e800640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8810e510 0x7f2e8819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.871+0000 7f2e8e800640 1 -- 172.21.15.86:0/94076113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e881a4980 con 0x7f2e881a3f10 2024-04-17T07:23:50.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.872+0000 7f2e8e800640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e881a3f10 0x7f2e881a43c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400be20 tx=0x7f2e8400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:50.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.873+0000 7f2e82a00640 1 -- 172.21.15.86:0/94076113 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84019070 con 0x7f2e881a3f10 2024-04-17T07:23:50.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.873+0000 7f2e82a00640 1 -- 172.21.15.86:0/94076113 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e840148f0 con 0x7f2e881a3f10 2024-04-17T07:23:50.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.873+0000 7f2e82a00640 1 -- 172.21.15.86:0/94076113 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e840127f0 con 0x7f2e881a3f10 2024-04-17T07:23:50.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.873+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e881a4c70 con 0x7f2e881a3f10 2024-04-17T07:23:50.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.873+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881a51f0 con 0x7f2e881a3f10 2024-04-17T07:23:50.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.879+0000 7f2e93618740 1 Processor -- start 2024-04-17T07:23:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.880+0000 7f2e82a00640 1 -- 172.21.15.86:0/94076113 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2e8401c070 con 0x7f2e881a3f10 2024-04-17T07:23:50.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.880+0000 7f2e93618740 1 -- start start 2024-04-17T07:23:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.880+0000 7f2e93618740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92930a0 0x5568e92934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.880+0000 7f2e93618740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 0x5568e9298590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.880+0000 7f2e93618740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9298c00 0x5568e929b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.880+0000 7f2e93618740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568e91f8cb0 con 0x5568e9298c00 2024-04-17T07:23:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.880+0000 7f2e93618740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5568e919f370 con 0x5568e9293a90 2024-04-17T07:23:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.880+0000 7f2e93618740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5568e91f4c70 con 0x5568e92930a0 2024-04-17T07:23:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.881+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92930a0 0x5568e92934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.881+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9298c00 0x5568e929b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.881+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9298c00 0x5568e929b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33938/0 (socket says 172.21.15.86:33938) 2024-04-17T07:23:50.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.881+0000 7f2e8d400640 1 -- 172.21.15.86:0/2561773780 learned_addr learned my addr 172.21.15.86:0/2561773780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.882+0000 7f2e82a00640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cb0 0x7f2e700a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.882+0000 7f2e8d400640 1 -- 172.21.15.86:0/2561773780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92930a0 msgr2=0x5568e92934c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.882+0000 7f2e8d400640 1 --2- 172.21.15.86:0/2561773780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92930a0 0x5568e92934c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.882+0000 7f2e8d400640 1 -- 172.21.15.86:0/2561773780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 msgr2=0x5568e9298590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:23:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.882+0000 7f2e8d400640 1 --2- 172.21.15.86:0/2561773780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 0x5568e9298590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.882+0000 7f2e8d400640 1 -- 172.21.15.86:0/2561773780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e91be910 con 0x5568e9298c00 2024-04-17T07:23:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.882+0000 7f2e82a00640 1 -- 172.21.15.86:0/94076113 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2e840cffb0 con 0x7f2e881a3f10 2024-04-17T07:23:50.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.883+0000 7f2e8d400640 1 --2- 172.21.15.86:0/2561773780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9298c00 0x5568e929b090 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2e78006ea0 tx=0x7f2e78009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38af961dbc937c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.883+0000 7f2e8e800640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cb0 0x7f2e700a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.883+0000 7f2e77000640 1 -- 172.21.15.86:0/2561773780 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78015870 con 0x5568e9298c00 2024-04-17T07:23:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.883+0000 7f2e77000640 1 -- 172.21.15.86:0/2561773780 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7801c070 con 0x5568e9298c00 2024-04-17T07:23:50.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.883+0000 7f2e77000640 1 -- 172.21.15.86:0/2561773780 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78017bb0 con 0x5568e9298c00 2024-04-17T07:23:50.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.885+0000 7f2e93618740 1 -- 172.21.15.86:0/2561773780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9298c00 msgr2=0x5568e929b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:50.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.885+0000 7f2e93618740 1 --2- 172.21.15.86:0/2561773780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9298c00 0x5568e929b090 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2e78006ea0 tx=0x7f2e78009190 comp rx=0 tx=0).stop 2024-04-17T07:23:50.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.885+0000 7f2e8e800640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cb0 0x7f2e700a6170 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400bdf0 tx=0x7f2e84004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:50.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.886+0000 7f2e93618740 1 -- 172.21.15.86:0/2561773780 shutdown_connections 2024-04-17T07:23:50.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.886+0000 7f2e93618740 1 --2- 172.21.15.86:0/2561773780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9298c00 0x5568e929b090 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.886+0000 7f2e93618740 1 --2- 172.21.15.86:0/2561773780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 0x5568e9298590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.886+0000 7f2e93618740 1 --2- 172.21.15.86:0/2561773780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92930a0 0x5568e92934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.886+0000 7f2e93618740 1 -- 172.21.15.86:0/2561773780 >> 172.21.15.86:0/2561773780 conn(0x5568e928e610 msgr2=0x5568e928ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.886+0000 7f2e93618740 1 -- 172.21.15.86:0/2561773780 shutdown_connections 2024-04-17T07:23:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.886+0000 7f2e93618740 1 -- 172.21.15.86:0/2561773780 wait complete. 2024-04-17T07:23:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.887+0000 7f2e93618740 1 -- start start 2024-04-17T07:23:50.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.887+0000 7f2e93618740 1 Processor -- start 2024-04-17T07:23:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.889+0000 7f2e93618740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e92930a0 0x5568e929d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.889+0000 7f2e93618740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e9293a90 0x5568e929dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.889+0000 7f2e93618740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92a2ca0 0x5568e92a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.889+0000 7f2e93618740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568e929bbb0 con 0x5568e92930a0 2024-04-17T07:23:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.889+0000 7f2e8de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e9293a90 0x5568e929dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.890+0000 7f2e8de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e9293a90 0x5568e929dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:43956/0 (socket says 172.21.15.86:43956) 2024-04-17T07:23:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.890+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e92930a0 0x5568e929d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.890+0000 7f2e8de00640 1 -- 172.21.15.86:0/1919971146 learned_addr learned my addr 172.21.15.86:0/1919971146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.890+0000 7f2e93618740 1 -- 172.21.15.86:0/1919971146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5568e91b2730 con 0x5568e92a2ca0 2024-04-17T07:23:50.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.890+0000 7f2e93618740 1 -- 172.21.15.86:0/1919971146 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5568e929bd30 con 0x5568e9293a90 2024-04-17T07:23:50.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.890+0000 7f2e93618740 10 client.0 _ll_register_callbacks cb 0x5568e91b4b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:50.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.891+0000 7f2e8d400640 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92a2ca0 0x5568e92a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.891+0000 7f2e8de00640 1 -- 172.21.15.86:0/1919971146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92a2ca0 msgr2=0x5568e92a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:50.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.891+0000 7f2e8de00640 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92a2ca0 0x5568e92a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.891+0000 7f2e8de00640 1 -- 172.21.15.86:0/1919971146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e92930a0 msgr2=0x5568e929d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:50.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.891+0000 7f2e8de00640 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e92930a0 0x5568e929d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.891+0000 7f2e8de00640 1 -- 172.21.15.86:0/1919971146 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e8fd37e0 con 0x5568e9293a90 2024-04-17T07:23:50.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.891+0000 7f2e8d400640 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e92930a0 0x5568e929d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:50.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.891+0000 7f2e8de00640 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e9293a90 0x5568e929dc70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aeb362524b541cb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:50.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.892+0000 7f2e6b400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:23:50.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.892+0000 7f2e6b400640 1 -- 172.21.15.86:0/1919971146 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c007c60 con 0x5568e9293a90 2024-04-17T07:23:50.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.892+0000 7f2e6b400640 1 -- 172.21.15.86:0/1919971146 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2e7c007e00 con 0x5568e9293a90 2024-04-17T07:23:50.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.892+0000 7f2e93618740 1 -- 172.21.15.86:0/1919971146 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7801c070 con 0x5568e9293a90 2024-04-17T07:23:50.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.893+0000 7f2e6b400640 1 -- 172.21.15.86:0/1919971146 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e7c04e050 con 0x5568e9293a90 2024-04-17T07:23:50.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.893+0000 7f2e93618740 10 client.26827 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:50.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.893+0000 7f2e93618740 1 -- 172.21.15.86:0/1919971146 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568e9033f70 con 0x5568e9293a90 2024-04-17T07:23:50.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.893+0000 7f2e6b400640 1 -- 172.21.15.86:0/1919971146 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7801c070 con 0x5568e9293a90 2024-04-17T07:23:50.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.894+0000 7f2e6b400640 1 -- 172.21.15.86:0/1919971146 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2e7c0035f0 con 0x5568e9293a90 2024-04-17T07:23:50.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.894+0000 7f2e93618740 10 client.26827 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.894+0000 7f2e93618740 10 client.26827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.894+0000 7f2e93618740 20 client.26827 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.894+0000 7f2e93618740 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568e92a9970 0x5568e92abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.894+0000 7f2e93618740 4 client.26827 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.894+0000 7f2e93618740 1 -- 172.21.15.86:0/1919971146 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568e92066d0 con 0x5568e92a9970 2024-04-17T07:23:50.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.895+0000 7f2e8d400640 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568e92a9970 0x5568e92abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.900+0000 7f2e8d400640 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568e92a9970 0x5568e92abdf0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8427202938c699f3 server_cookie=8667fdd590f5f5a0 in_seq=0 out_seq=0 2024-04-17T07:23:50.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.900+0000 7f2e6b400640 10 client.26827 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:50.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.901+0000 7f2e6b400640 1 -- 172.21.15.86:0/1919971146 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568e92066d0 con 0x5568e92a9970 2024-04-17T07:23:50.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.901+0000 7f2e6b400640 10 client.26827 handle_command_reply: tid=0 2024-04-17T07:23:50.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.902+0000 7f2e93618740 1 client.26827 shutdown 2024-04-17T07:23:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.902+0000 7f2e93618740 1 -- 172.21.15.86:0/1919971146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e9293a90 msgr2=0x5568e929dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.902+0000 7f2e93618740 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e9293a90 0x5568e929dc70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c005d50 comp rx=0 tx=0).stop 2024-04-17T07:23:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.902+0000 7f2e93618740 1 -- 172.21.15.86:0/1919971146 shutdown_connections 2024-04-17T07:23:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.902+0000 7f2e93618740 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568e92a9970 0x5568e92abdf0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.902+0000 7f2e93618740 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92a2ca0 0x5568e92a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.902+0000 7f2e93618740 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e9293a90 0x5568e929dc70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.902+0000 7f2e93618740 1 --2- 172.21.15.86:0/1919971146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e92930a0 0x5568e929d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.902+0000 7f2e93618740 1 -- 172.21.15.86:0/1919971146 >> 172.21.15.86:0/1919971146 conn(0x5568e928e610 msgr2=0x5568e928d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.902+0000 7f2e6b400640 0 client.26827 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:50.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.903+0000 7f2e93618740 1 -- 172.21.15.86:0/1919971146 shutdown_connections 2024-04-17T07:23:50.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.903+0000 7f2e93618740 1 -- 172.21.15.86:0/1919971146 wait complete. 2024-04-17T07:23:50.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.903+0000 7f2e93618740 20 client.26827 trim_cache size 0 max 16384 2024-04-17T07:23:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.939+0000 7f2e93618740 1 Processor -- start 2024-04-17T07:23:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.940+0000 7f2e93618740 1 -- start start 2024-04-17T07:23:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.940+0000 7f2e93618740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92930a0 0x5568e9295140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.940+0000 7f2e93618740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 0x5568e9295680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.940+0000 7f2e93618740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e92a2ca0 0x5568e928d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.940+0000 7f2e93618740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881c2520 con 0x5568e92a2ca0 2024-04-17T07:23:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.940+0000 7f2e93618740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881c4060 con 0x5568e9293a90 2024-04-17T07:23:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.940+0000 7f2e93618740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881c2bf0 con 0x5568e92930a0 2024-04-17T07:23:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.941+0000 7f2e8de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 0x5568e9295680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.941+0000 7f2e8de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 0x5568e9295680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52488/0 (socket says 172.21.15.86:52488) 2024-04-17T07:23:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.941+0000 7f2e8de00640 1 -- 172.21.15.86:0/239833583 learned_addr learned my addr 172.21.15.86:0/239833583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.941+0000 7f2e8d400640 1 --2- 172.21.15.86:0/239833583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92930a0 0x5568e9295140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.942+0000 7f2e8de00640 1 -- 172.21.15.86:0/239833583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92930a0 msgr2=0x5568e9295140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.942+0000 7f2e8de00640 1 --2- 172.21.15.86:0/239833583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92930a0 0x5568e9295140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.942+0000 7f2e8de00640 1 -- 172.21.15.86:0/239833583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e92a2ca0 msgr2=0x5568e928d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.942+0000 7f2e8de00640 1 --2- 172.21.15.86:0/239833583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e92a2ca0 0x5568e928d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.942+0000 7f2e8de00640 1 -- 172.21.15.86:0/239833583 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568e8ecb170 con 0x5568e9293a90 2024-04-17T07:23:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.942+0000 7f2e8de00640 1 --2- 172.21.15.86:0/239833583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 0x5568e9295680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c02c8f0 tx=0x7f2e7c003780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6c2f9ad1a227baa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.943+0000 7f2e76600640 1 -- 172.21.15.86:0/239833583 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c056020 con 0x5568e9293a90 2024-04-17T07:23:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.943+0000 7f2e76600640 1 -- 172.21.15.86:0/239833583 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7c04b070 con 0x5568e9293a90 2024-04-17T07:23:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.943+0000 7f2e76600640 1 -- 172.21.15.86:0/239833583 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c056020 con 0x5568e9293a90 2024-04-17T07:23:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.944+0000 7f2e93618740 1 -- 172.21.15.86:0/239833583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 msgr2=0x5568e9295680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.944+0000 7f2e93618740 1 --2- 172.21.15.86:0/239833583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 0x5568e9295680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c02c8f0 tx=0x7f2e7c003780 comp rx=0 tx=0).stop 2024-04-17T07:23:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.944+0000 7f2e93618740 1 -- 172.21.15.86:0/239833583 shutdown_connections 2024-04-17T07:23:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.944+0000 7f2e93618740 1 --2- 172.21.15.86:0/239833583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e92a2ca0 0x5568e928d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.944+0000 7f2e93618740 1 --2- 172.21.15.86:0/239833583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e9293a90 0x5568e9295680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.944+0000 7f2e93618740 1 --2- 172.21.15.86:0/239833583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92930a0 0x5568e9295140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.944+0000 7f2e93618740 1 -- 172.21.15.86:0/239833583 >> 172.21.15.86:0/239833583 conn(0x5568e928e610 msgr2=0x5568e9294590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:50.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.945+0000 7f2e93618740 1 -- 172.21.15.86:0/239833583 shutdown_connections 2024-04-17T07:23:50.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.945+0000 7f2e93618740 1 -- 172.21.15.86:0/239833583 wait complete. 2024-04-17T07:23:50.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.945+0000 7f2e93618740 1 -- start start 2024-04-17T07:23:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.946+0000 7f2e93618740 1 Processor -- start 2024-04-17T07:23:50.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.946+0000 7f2e93618740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92930a0 0x5568e92baf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.946+0000 7f2e93618740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9293a90 0x5568e92bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.946+0000 7f2e93618740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92bc420 0x5568e92bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.946+0000 7f2e93618740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881c3990 con 0x5568e9293a90 2024-04-17T07:23:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.946+0000 7f2e93618740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881c32c0 con 0x5568e92930a0 2024-04-17T07:23:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.946+0000 7f2e93618740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e881c04f0 con 0x5568e92bc420 2024-04-17T07:23:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.946+0000 7f2e93618740 10 client.0 _ll_register_callbacks cb 0x7f2e7c007200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.947+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92930a0 0x5568e92baf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.947+0000 7f2e8d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92930a0 0x5568e92baf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52492/0 (socket says 172.21.15.86:52492) 2024-04-17T07:23:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.947+0000 7f2e8d400640 1 -- 172.21.15.86:0/833304655 learned_addr learned my addr 172.21.15.86:0/833304655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.947+0000 7f2e8de00640 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9293a90 0x5568e92bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.947+0000 7f2e8d400640 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92bc420 0x5568e92bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.947+0000 7f2e8d400640 1 -- 172.21.15.86:0/833304655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92bc420 msgr2=0x5568e92bc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.947+0000 7f2e8d400640 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92bc420 0x5568e92bc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.948+0000 7f2e8d400640 1 -- 172.21.15.86:0/833304655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9293a90 msgr2=0x5568e92bb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.948+0000 7f2e8d400640 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9293a90 0x5568e92bb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.948+0000 7f2e8d400640 1 -- 172.21.15.86:0/833304655 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568e92bcde0 con 0x5568e92930a0 2024-04-17T07:23:50.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.948+0000 7f2e8d400640 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92930a0 0x5568e92baf70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e78006e70 tx=0x7f2e78009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2873d3d6f85c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:50.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.948+0000 7f2e6b400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:23:50.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.950+0000 7f2e6b400640 1 -- 172.21.15.86:0/833304655 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78017760 con 0x5568e92930a0 2024-04-17T07:23:50.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.950+0000 7f2e6b400640 1 -- 172.21.15.86:0/833304655 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2e7802cbc0 con 0x5568e92930a0 2024-04-17T07:23:50.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.950+0000 7f2e93618740 1 -- 172.21.15.86:0/833304655 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7c04b070 con 0x5568e92930a0 2024-04-17T07:23:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.951+0000 7f2e6b400640 1 -- 172.21.15.86:0/833304655 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e7802cde0 con 0x5568e92930a0 2024-04-17T07:23:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.951+0000 7f2e93618740 10 client.36776 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.951+0000 7f2e93618740 1 -- 172.21.15.86:0/833304655 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568e93103b0 con 0x5568e92930a0 2024-04-17T07:23:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.952+0000 7f2e6b400640 1 -- 172.21.15.86:0/833304655 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7c04b070 con 0x5568e92930a0 2024-04-17T07:23:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.952+0000 7f2e6b400640 1 -- 172.21.15.86:0/833304655 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2e78015930 con 0x5568e92930a0 2024-04-17T07:23:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.952+0000 7f2e93618740 10 client.36776 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.952+0000 7f2e93618740 10 client.36776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.952+0000 7f2e93618740 20 client.36776 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.952+0000 7f2e93618740 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568e9310990 0x5568e9312d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.953+0000 7f2e93618740 4 client.36776 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:23:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.953+0000 7f2e93618740 1 -- 172.21.15.86:0/833304655 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5568e9200540 con 0x5568e9310990 2024-04-17T07:23:50.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.954+0000 7f2e8de00640 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568e9310990 0x5568e9312d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.955+0000 7f2e6b400640 10 client.36776 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.955+0000 7f2e8de00640 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568e9310990 0x5568e9312d80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af1e3c6717a8eb97 server_cookie=c861d57a7c88c2f1 in_seq=0 out_seq=0 2024-04-17T07:23:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e6b400640 1 -- 172.21.15.86:0/833304655 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5568e9200540 con 0x5568e9310990 2024-04-17T07:23:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e6b400640 10 client.36776 handle_command_reply: tid=0 2024-04-17T07:23:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e93618740 1 client.36776 shutdown 2024-04-17T07:23:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e93618740 1 -- 172.21.15.86:0/833304655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92930a0 msgr2=0x5568e92baf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e93618740 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92930a0 0x5568e92baf70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e78006e70 tx=0x7f2e78009780 comp rx=0 tx=0).stop 2024-04-17T07:23:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e93618740 1 -- 172.21.15.86:0/833304655 shutdown_connections 2024-04-17T07:23:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e93618740 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568e9310990 0x5568e9312d80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e93618740 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568e92bc420 0x5568e92bc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e93618740 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568e9293a90 0x5568e92bb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e93618740 1 --2- 172.21.15.86:0/833304655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568e92930a0 0x5568e92baf70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.958+0000 7f2e93618740 1 -- 172.21.15.86:0/833304655 >> 172.21.15.86:0/833304655 conn(0x5568e928e610 msgr2=0x5568e929d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.959+0000 7f2e6b400640 0 client.36776 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.959+0000 7f2e93618740 1 -- 172.21.15.86:0/833304655 shutdown_connections 2024-04-17T07:23:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.959+0000 7f2e93618740 1 -- 172.21.15.86:0/833304655 wait complete. 2024-04-17T07:23:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.959+0000 7f2e93618740 20 client.36776 trim_cache size 0 max 16384 2024-04-17T07:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "6a417886-ce5a-4eb5-a906-6bab81792086", 2024-04-17T07:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:23:50.963 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cb0 msgr2=0x7f2e700a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cb0 0x7f2e700a6170 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400bdf0 tx=0x7f2e84004a40 comp rx=0 tx=0).stop 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e881a3f10 msgr2=0x7f2e881a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e881a3f10 0x7f2e881a43c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400be20 tx=0x7f2e8400bf10 comp rx=0 tx=0).stop 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 shutdown_connections 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2e700a3cb0 0x7f2e700a6170 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e881a3f10 0x7f2e881a43c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e8810e510 0x7f2e8819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 --2- 172.21.15.86:0/94076113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e8810ba70 0x7f2e8819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 >> 172.21.15.86:0/94076113 conn(0x7f2e8806f060 msgr2=0x7f2e8810c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.961+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 shutdown_connections 2024-04-17T07:23:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:50.962+0000 7f2e8f200640 1 -- 172.21.15.86:0/94076113 wait complete. 2024-04-17T07:23:50.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:23:50.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:23:51.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:51 smithi086 ceph-mon[31733]: pgmap v724: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 728 op/s 2024-04-17T07:23:51.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:51 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:23:51.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:51 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:51.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:51 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:23:51.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.346+0000 7fd5a9800640 1 -- 172.21.15.86:0/1364836641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5a41097b0 msgr2=0x7fd5a4109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:51.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.346+0000 7fd5a9800640 1 --2- 172.21.15.86:0/1364836641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5a41097b0 0x7fd5a4109c30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd590009900 tx=0x7fd59002bc40 comp rx=0 tx=0).stop 2024-04-17T07:23:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.347+0000 7fd5a9800640 1 -- 172.21.15.86:0/1364836641 shutdown_connections 2024-04-17T07:23:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.347+0000 7fd5a9800640 1 --2- 172.21.15.86:0/1364836641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a410a170 0x7fd5a4110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.347+0000 7fd5a9800640 1 --2- 172.21.15.86:0/1364836641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5a41097b0 0x7fd5a4109c30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.347+0000 7fd5a9800640 1 --2- 172.21.15.86:0/1364836641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a41085c0 0x7fd5a41089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.347+0000 7fd5a9800640 1 -- 172.21.15.86:0/1364836641 >> 172.21.15.86:0/1364836641 conn(0x7fd5a4077570 msgr2=0x7fd5a4079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:51.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.348+0000 7fd5a9800640 1 -- 172.21.15.86:0/1364836641 shutdown_connections 2024-04-17T07:23:51.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.349+0000 7fd5a9800640 1 -- 172.21.15.86:0/1364836641 wait complete. 2024-04-17T07:23:51.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.351+0000 7fd5a9800640 1 Processor -- start 2024-04-17T07:23:51.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.351+0000 7fd5a9800640 1 -- start start 2024-04-17T07:23:51.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.351+0000 7fd5a9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a41085c0 0x7fd5a40709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.351+0000 7fd5a9800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5a41097b0 0x7fd5a4070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.351+0000 7fd5a9800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a410a170 0x7fd5a41ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.351+0000 7fd5a9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a41141c0 con 0x7fd5a41085c0 2024-04-17T07:23:51.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.352+0000 7fd5a9800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4114040 con 0x7fd5a410a170 2024-04-17T07:23:51.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.352+0000 7fd5a9800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4114340 con 0x7fd5a41097b0 2024-04-17T07:23:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.352+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a410a170 0x7fd5a41ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.352+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a410a170 0x7fd5a41ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52510/0 (socket says 172.21.15.86:52510) 2024-04-17T07:23:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.352+0000 7fd5a3400640 1 -- 172.21.15.86:0/3338764466 learned_addr learned my addr 172.21.15.86:0/3338764466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.352+0000 7fd5a2a00640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a41085c0 0x7fd5a40709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.353+0000 7fd5a2000640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5a41097b0 0x7fd5a4070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.353+0000 7fd5a3400640 1 -- 172.21.15.86:0/3338764466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5a41097b0 msgr2=0x7fd5a4070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.353+0000 7fd5a3400640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5a41097b0 0x7fd5a4070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.353+0000 7fd5a3400640 1 -- 172.21.15.86:0/3338764466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a41085c0 msgr2=0x7fd5a40709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:51.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.353+0000 7fd5a3400640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a41085c0 0x7fd5a40709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.353+0000 7fd5a3400640 1 -- 172.21.15.86:0/3338764466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a407a7b0 con 0x7fd5a410a170 2024-04-17T07:23:51.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.354+0000 7fd5a3400640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a410a170 0x7fd5a41ae8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00eaa0 tx=0x7fd59c00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:51.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.354+0000 7fd59b400640 1 -- 172.21.15.86:0/3338764466 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c00cdb0 con 0x7fd5a410a170 2024-04-17T07:23:51.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.354+0000 7fd5a9800640 1 -- 172.21.15.86:0/3338764466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a407aaa0 con 0x7fd5a410a170 2024-04-17T07:23:51.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.355+0000 7fd5a9800640 1 -- 172.21.15.86:0/3338764466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a407b040 con 0x7fd5a410a170 2024-04-17T07:23:51.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.355+0000 7fd59b400640 1 -- 172.21.15.86:0/3338764466 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd59c005020 con 0x7fd5a410a170 2024-04-17T07:23:51.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.355+0000 7fd59b400640 1 -- 172.21.15.86:0/3338764466 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c01f060 con 0x7fd5a410a170 2024-04-17T07:23:51.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.360+0000 7fd59b400640 1 -- 172.21.15.86:0/3338764466 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fd59c0116d0 con 0x7fd5a410a170 2024-04-17T07:23:51.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.361+0000 7fd5ac47f740 1 Processor -- start 2024-04-17T07:23:51.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.361+0000 7fd5ac47f740 1 -- start start 2024-04-17T07:23:51.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.362+0000 7fd59b400640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5880a3c80 0x7fd5880a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.362+0000 7fd59b400640 1 -- 172.21.15.86:0/3338764466 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd59c0d0860 con 0x7fd5a410a170 2024-04-17T07:23:51.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.362+0000 7fd5a2000640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5880a3c80 0x7fd5880a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.363+0000 7fd5a2000640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5880a3c80 0x7fd5880a6140 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd590000c00 tx=0x7fd590005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.364+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35f7190 0x5565c35f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.364+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7b80 0x5565c35fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.364+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35fccf0 0x5565c35ff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.364+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5565c355e120 con 0x5565c35f7190 2024-04-17T07:23:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.364+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5565c3504340 con 0x5565c35fccf0 2024-04-17T07:23:51.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.364+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5565c355a0e0 con 0x5565c35f7b80 2024-04-17T07:23:51.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.365+0000 7fd5a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35f7190 0x5565c35f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.366+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35fccf0 0x5565c35ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.366+0000 7fd5a2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7b80 0x5565c35fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.366+0000 7fd5a2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7b80 0x5565c35fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44010/0 (socket says 172.21.15.86:44010) 2024-04-17T07:23:51.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.366+0000 7fd5a2000640 1 -- 172.21.15.86:0/3939260979 learned_addr learned my addr 172.21.15.86:0/3939260979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.367+0000 7fd5a2000640 1 -- 172.21.15.86:0/3939260979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35fccf0 msgr2=0x5565c35ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.367+0000 7fd5a2000640 1 --2- 172.21.15.86:0/3939260979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35fccf0 0x5565c35ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.367+0000 7fd5a2000640 1 -- 172.21.15.86:0/3939260979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35f7190 msgr2=0x5565c35f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.367+0000 7fd5a2000640 1 --2- 172.21.15.86:0/3939260979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35f7190 0x5565c35f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.367+0000 7fd5a2000640 1 -- 172.21.15.86:0/3939260979 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565c333a7d0 con 0x5565c35f7b80 2024-04-17T07:23:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.368+0000 7fd5a2000640 1 --2- 172.21.15.86:0/3939260979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7b80 0x5565c35fc680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd5900096c0 tx=0x7fd59003e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fd689ac5c79708 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.368+0000 7fd58fe00640 1 -- 172.21.15.86:0/3939260979 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900051c0 con 0x5565c35f7b80 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.368+0000 7fd58fe00640 1 -- 172.21.15.86:0/3939260979 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd590050070 con 0x5565c35f7b80 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.368+0000 7fd58fe00640 1 -- 172.21.15.86:0/3939260979 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590003850 con 0x5565c35f7b80 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.369+0000 7fd5ac47f740 1 -- 172.21.15.86:0/3939260979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7b80 msgr2=0x5565c35fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.370+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/3939260979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7b80 0x5565c35fc680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd5900096c0 tx=0x7fd59003e000 comp rx=0 tx=0).stop 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.370+0000 7fd5ac47f740 1 -- 172.21.15.86:0/3939260979 shutdown_connections 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.370+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/3939260979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35fccf0 0x5565c35ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.370+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/3939260979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7b80 0x5565c35fc680 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd5900096c0 tx=0x7fd59003e000 comp rx=0 tx=0).stop 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.371+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/3939260979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35f7190 0x5565c35f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.371+0000 7fd5ac47f740 1 -- 172.21.15.86:0/3939260979 >> 172.21.15.86:0/3939260979 conn(0x5565c35f2700 msgr2=0x5565c35f2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.371+0000 7fd5ac47f740 1 -- 172.21.15.86:0/3939260979 shutdown_connections 2024-04-17T07:23:51.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.371+0000 7fd5ac47f740 1 -- 172.21.15.86:0/3939260979 wait complete. 2024-04-17T07:23:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.372+0000 7fd5ac47f740 1 -- start start 2024-04-17T07:23:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.372+0000 7fd5ac47f740 1 Processor -- start 2024-04-17T07:23:51.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7190 0x5565c3601820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35fccf0 0x5565c3601d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c3606d90 0x5565c3609180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5565c35ffc40 con 0x5565c35fccf0 2024-04-17T07:23:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5565c3517710 con 0x5565c3606d90 2024-04-17T07:23:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5565c35ffdc0 con 0x5565c35f7190 2024-04-17T07:23:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5a2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7190 0x5565c3601820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5a2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35fccf0 0x5565c3601d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5a2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7190 0x5565c3601820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44016/0 (socket says 172.21.15.86:44016) 2024-04-17T07:23:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5a2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35fccf0 0x5565c3601d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34010/0 (socket says 172.21.15.86:34010) 2024-04-17T07:23:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5a2a00640 1 -- 172.21.15.86:0/483604154 learned_addr learned my addr 172.21.15.86:0/483604154 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5ac47f740 10 client.0 _ll_register_callbacks cb 0x5565c3512af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.376+0000 7fd5a3400640 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c3606d90 0x5565c3609180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.377+0000 7fd5a2000640 1 -- 172.21.15.86:0/483604154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7190 msgr2=0x5565c3601820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.377+0000 7fd5a2000640 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7190 0x5565c3601820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.377+0000 7fd5a2000640 1 -- 172.21.15.86:0/483604154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c3606d90 msgr2=0x5565c3609180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.377+0000 7fd5a2000640 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c3606d90 0x5565c3609180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.377+0000 7fd5a2000640 1 -- 172.21.15.86:0/483604154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565c32703f0 con 0x5565c35fccf0 2024-04-17T07:23:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.377+0000 7fd5a2000640 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35fccf0 0x5565c3601d60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x5565c35f7f80 tx=0x7fd590004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa7a130bbf47e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.378+0000 7fd58cc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:23:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.378+0000 7fd58cc00640 1 -- 172.21.15.86:0/483604154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590004d70 con 0x5565c35fccf0 2024-04-17T07:23:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.378+0000 7fd58cc00640 1 -- 172.21.15.86:0/483604154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd5900035f0 con 0x5565c35fccf0 2024-04-17T07:23:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.378+0000 7fd5ac47f740 1 -- 172.21.15.86:0/483604154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd590050070 con 0x5565c35fccf0 2024-04-17T07:23:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.379+0000 7fd58cc00640 1 -- 172.21.15.86:0/483604154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd5900037f0 con 0x5565c35fccf0 2024-04-17T07:23:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.379+0000 7fd5ac47f740 10 client.17886 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.379+0000 7fd5ac47f740 1 -- 172.21.15.86:0/483604154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565c32300f0 con 0x5565c35fccf0 2024-04-17T07:23:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.380+0000 7fd58cc00640 1 -- 172.21.15.86:0/483604154 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd590050070 con 0x5565c35fccf0 2024-04-17T07:23:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.380+0000 7fd58cc00640 1 -- 172.21.15.86:0/483604154 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fd5900419e0 con 0x5565c35fccf0 2024-04-17T07:23:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.380+0000 7fd5ac47f740 10 client.17886 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.380+0000 7fd5ac47f740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.380+0000 7fd5ac47f740 20 client.17886 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.380+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5565c360d630 0x5565c360fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.380+0000 7fd5ac47f740 4 client.17886 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:23:51.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.380+0000 7fd5ac47f740 1 -- 172.21.15.86:0/483604154 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565c356bb40 con 0x5565c360d630 2024-04-17T07:23:51.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.381+0000 7fd5a2a00640 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5565c360d630 0x5565c360fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.382+0000 7fd5a2a00640 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5565c360d630 0x5565c360fa20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd6160799288274f server_cookie=5bed0da7e0844f74 in_seq=0 out_seq=0 2024-04-17T07:23:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.382+0000 7fd58cc00640 10 client.17886 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.384+0000 7fd58cc00640 1 -- 172.21.15.86:0/483604154 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565c356bb40 con 0x5565c360d630 2024-04-17T07:23:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.384+0000 7fd58cc00640 10 client.17886 handle_command_reply: tid=0 2024-04-17T07:23:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.384+0000 7fd5ac47f740 1 client.17886 shutdown 2024-04-17T07:23:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.385+0000 7fd5ac47f740 1 -- 172.21.15.86:0/483604154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35fccf0 msgr2=0x5565c3601d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.385+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35fccf0 0x5565c3601d60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x5565c35f7f80 tx=0x7fd590004970 comp rx=0 tx=0).stop 2024-04-17T07:23:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.385+0000 7fd5ac47f740 1 -- 172.21.15.86:0/483604154 shutdown_connections 2024-04-17T07:23:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.385+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5565c360d630 0x5565c360fa20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.385+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c3606d90 0x5565c3609180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.385+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c35fccf0 0x5565c3601d60 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.385+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/483604154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35f7190 0x5565c3601820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.385+0000 7fd5ac47f740 1 -- 172.21.15.86:0/483604154 >> 172.21.15.86:0/483604154 conn(0x5565c35f2700 msgr2=0x5565c35f1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.385+0000 7fd58cc00640 0 client.17886 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.386+0000 7fd5ac47f740 1 -- 172.21.15.86:0/483604154 shutdown_connections 2024-04-17T07:23:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.386+0000 7fd5ac47f740 1 -- 172.21.15.86:0/483604154 wait complete. 2024-04-17T07:23:51.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.387+0000 7fd5ac47f740 20 client.17886 trim_cache size 0 max 16384 2024-04-17T07:23:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.426+0000 7fd5ac47f740 1 Processor -- start 2024-04-17T07:23:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.426+0000 7fd5ac47f740 1 -- start start 2024-04-17T07:23:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.426+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 0x5565c3600b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.426+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 0x5565c3601090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.426+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 0x5565c35f1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.426+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd58400fc20 con 0x5565c3606d90 2024-04-17T07:23:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.426+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd584011760 con 0x5565c35f7190 2024-04-17T07:23:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.426+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5840102f0 con 0x5565c35fccf0 2024-04-17T07:23:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.427+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 0x5565c35f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.427+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 0x5565c35f1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34022/0 (socket says 172.21.15.86:34022) 2024-04-17T07:23:51.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.427+0000 7fd5a3400640 1 -- 172.21.15.86:0/1760936288 learned_addr learned my addr 172.21.15.86:0/1760936288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.427+0000 7fd5a2000640 1 --2- 172.21.15.86:0/1760936288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 0x5565c3601090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.427+0000 7fd5a2a00640 1 --2- 172.21.15.86:0/1760936288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 0x5565c3600b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.427+0000 7fd5a2000640 1 -- 172.21.15.86:0/1760936288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 msgr2=0x5565c3600b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.427+0000 7fd5a2000640 1 --2- 172.21.15.86:0/1760936288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 0x5565c3600b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.427+0000 7fd5a2000640 1 -- 172.21.15.86:0/1760936288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 msgr2=0x5565c35f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.427+0000 7fd5a2000640 1 --2- 172.21.15.86:0/1760936288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 0x5565c35f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.427+0000 7fd5a2000640 1 -- 172.21.15.86:0/1760936288 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565c35ffb80 con 0x5565c35fccf0 2024-04-17T07:23:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.428+0000 7fd5a2000640 1 --2- 172.21.15.86:0/1760936288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 0x5565c3601090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd590041d80 tx=0x7fd590004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e6825629570a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.428+0000 7fd58f400640 1 -- 172.21.15.86:0/1760936288 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900044d0 con 0x5565c35fccf0 2024-04-17T07:23:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.429+0000 7fd58f400640 1 -- 172.21.15.86:0/1760936288 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd59006c020 con 0x5565c35fccf0 2024-04-17T07:23:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.429+0000 7fd58f400640 1 -- 172.21.15.86:0/1760936288 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590034440 con 0x5565c35fccf0 2024-04-17T07:23:51.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.430+0000 7fd5ac47f740 1 -- 172.21.15.86:0/1760936288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 msgr2=0x5565c3601090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:51.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.430+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/1760936288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 0x5565c3601090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd590041d80 tx=0x7fd590004a60 comp rx=0 tx=0).stop 2024-04-17T07:23:51.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.430+0000 7fd5ac47f740 1 -- 172.21.15.86:0/1760936288 shutdown_connections 2024-04-17T07:23:51.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.430+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/1760936288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 0x5565c35f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.430+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/1760936288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 0x5565c3601090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.430+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/1760936288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 0x5565c3600b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.430+0000 7fd5ac47f740 1 -- 172.21.15.86:0/1760936288 >> 172.21.15.86:0/1760936288 conn(0x5565c35f2700 msgr2=0x5565c35f9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:51.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.431+0000 7fd5ac47f740 1 -- 172.21.15.86:0/1760936288 shutdown_connections 2024-04-17T07:23:51.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.431+0000 7fd5ac47f740 1 -- 172.21.15.86:0/1760936288 wait complete. 2024-04-17T07:23:51.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.431+0000 7fd5ac47f740 1 -- start start 2024-04-17T07:23:51.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.431+0000 7fd5ac47f740 1 Processor -- start 2024-04-17T07:23:51.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.432+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 0x5565c3620510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.432+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 0x5565c3620a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.432+0000 7fd5ac47f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 0x5565c361eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.432+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd584011090 con 0x5565c3606d90 2024-04-17T07:23:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.432+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5840109c0 con 0x5565c35f7190 2024-04-17T07:23:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.432+0000 7fd5ac47f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd58400dbf0 con 0x5565c35fccf0 2024-04-17T07:23:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.432+0000 7fd5ac47f740 10 client.0 _ll_register_callbacks cb 0x7fd59400b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:23:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.432+0000 7fd5a2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 0x5565c3620510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.432+0000 7fd5a2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 0x5565c3620510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52550/0 (socket says 172.21.15.86:52550) 2024-04-17T07:23:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.432+0000 7fd5a2a00640 1 -- 172.21.15.86:0/2082905753 learned_addr learned my addr 172.21.15.86:0/2082905753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:23:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.433+0000 7fd5a2000640 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 0x5565c3620a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.433+0000 7fd5a3400640 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 0x5565c361eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.434+0000 7fd5a2a00640 1 -- 172.21.15.86:0/2082905753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 msgr2=0x5565c3620a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.434+0000 7fd5a2a00640 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 0x5565c3620a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.434+0000 7fd5a2a00640 1 -- 172.21.15.86:0/2082905753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 msgr2=0x5565c361eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:23:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.434+0000 7fd5a2a00640 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 0x5565c361eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.434+0000 7fd5a2a00640 1 -- 172.21.15.86:0/2082905753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565c361f160 con 0x5565c35f7190 2024-04-17T07:23:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.434+0000 7fd5a3400640 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 0x5565c361eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:23:51.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.435+0000 7fd5a2a00640 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 0x5565c3620510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5565c3610130 tx=0x7fd59401d600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b58fb1cd03a8637 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:23:51.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.435+0000 7fd58cc00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:23:51.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.436+0000 7fd58cc00640 1 -- 172.21.15.86:0/2082905753 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59401dbc0 con 0x5565c35f7190 2024-04-17T07:23:51.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.436+0000 7fd5ac47f740 1 -- 172.21.15.86:0/2082905753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd59006c020 con 0x5565c35f7190 2024-04-17T07:23:51.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.436+0000 7fd58cc00640 1 -- 172.21.15.86:0/2082905753 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd59401dd60 con 0x5565c35f7190 2024-04-17T07:23:51.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.436+0000 7fd58cc00640 1 -- 172.21.15.86:0/2082905753 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd59402f050 con 0x5565c35f7190 2024-04-17T07:23:51.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.436+0000 7fd5ac47f740 10 client.36794 fetch_fsmap learned FSMap version 28 2024-04-17T07:23:51.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.436+0000 7fd5ac47f740 1 -- 172.21.15.86:0/2082905753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565c3673f00 con 0x5565c35f7190 2024-04-17T07:23:51.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.438+0000 7fd58cc00640 1 -- 172.21.15.86:0/2082905753 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd59006c020 con 0x5565c35f7190 2024-04-17T07:23:51.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.438+0000 7fd58cc00640 1 -- 172.21.15.86:0/2082905753 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fd594026a60 con 0x5565c35f7190 2024-04-17T07:23:51.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.438+0000 7fd5ac47f740 10 client.36794 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:23:51.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.438+0000 7fd5ac47f740 10 client.36794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:23:51.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.438+0000 7fd5ac47f740 20 client.36794 populate_metadata read hostname 'smithi086' 2024-04-17T07:23:51.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.438+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5565c3674890 0x5565c3676cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:23:51.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.438+0000 7fd5ac47f740 4 client.36794 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:23:51.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.438+0000 7fd5ac47f740 1 -- 172.21.15.86:0/2082905753 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5565c35659b0 con 0x5565c3674890 2024-04-17T07:23:51.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.439+0000 7fd5a2000640 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5565c3674890 0x5565c3676cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:23:51.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.440+0000 7fd5a2000640 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5565c3674890 0x5565c3676cb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548e5aa9498c4ed4 server_cookie=7b1a8062255d4a7e in_seq=0 out_seq=0 2024-04-17T07:23:51.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.440+0000 7fd58cc00640 10 client.36794 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:51.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.441+0000 7fd58cc00640 1 -- 172.21.15.86:0/2082905753 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5565c35659b0 con 0x5565c3674890 2024-04-17T07:23:51.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.441+0000 7fd58cc00640 10 client.36794 handle_command_reply: tid=0 2024-04-17T07:23:51.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.441+0000 7fd5ac47f740 1 client.36794 shutdown 2024-04-17T07:23:51.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.442+0000 7fd5ac47f740 1 -- 172.21.15.86:0/2082905753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 msgr2=0x5565c3620510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:51.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.442+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 0x5565c3620510 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x5565c3610130 tx=0x7fd59401d600 comp rx=0 tx=0).stop 2024-04-17T07:23:51.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.442+0000 7fd5ac47f740 1 -- 172.21.15.86:0/2082905753 shutdown_connections 2024-04-17T07:23:51.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.442+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5565c3674890 0x5565c3676cb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.442+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565c3606d90 0x5565c361eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.442+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565c35fccf0 0x5565c3620a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.443+0000 7fd5ac47f740 1 --2- 172.21.15.86:0/2082905753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5565c35f7190 0x5565c3620510 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.443+0000 7fd5ac47f740 1 -- 172.21.15.86:0/2082905753 >> 172.21.15.86:0/2082905753 conn(0x5565c35f2700 msgr2=0x5565c3600b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:51.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.442+0000 7fd58cc00640 0 client.36794 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:23:51.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.443+0000 7fd5ac47f740 1 -- 172.21.15.86:0/2082905753 shutdown_connections 2024-04-17T07:23:51.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.443+0000 7fd5ac47f740 1 -- 172.21.15.86:0/2082905753 wait complete. 2024-04-17T07:23:51.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.444+0000 7fd5ac47f740 20 client.36794 trim_cache size 0 max 16384 2024-04-17T07:23:51.445 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:23:51.445 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:23:51.445 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:23:51.445 INFO:teuthology.orchestra.run.smithi086.stdout: "6a417886-ce5a-4eb5-a906-6bab81792086": { 2024-04-17T07:23:51.445 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:23:51.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "6a417886-ce5a-4eb5-a906-6bab81792086", 2024-04-17T07:23:51.445 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:23:51.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:23:51.446 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:23:51.446 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:23:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.446+0000 7fd5a9800640 1 -- 172.21.15.86:0/3338764466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5880a3c80 msgr2=0x7fd5880a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.446+0000 7fd5a9800640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5880a3c80 0x7fd5880a6140 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd590000c00 tx=0x7fd590005fd0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.446+0000 7fd5a9800640 1 -- 172.21.15.86:0/3338764466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a410a170 msgr2=0x7fd5a41ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:23:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.446+0000 7fd5a9800640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a410a170 0x7fd5a41ae8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00eaa0 tx=0x7fd59c00ef60 comp rx=0 tx=0).stop 2024-04-17T07:23:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.446+0000 7fd5a9800640 1 -- 172.21.15.86:0/3338764466 shutdown_connections 2024-04-17T07:23:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.446+0000 7fd5a9800640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd5880a3c80 0x7fd5880a6140 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.446+0000 7fd5a9800640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a410a170 0x7fd5a41ae8d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.446+0000 7fd5a9800640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5a41097b0 0x7fd5a4070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.446+0000 7fd5a9800640 1 --2- 172.21.15.86:0/3338764466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5a41085c0 0x7fd5a40709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:23:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.446+0000 7fd5a9800640 1 -- 172.21.15.86:0/3338764466 >> 172.21.15.86:0/3338764466 conn(0x7fd5a4077570 msgr2=0x7fd5a410cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:23:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.447+0000 7fd5a9800640 1 -- 172.21.15.86:0/3338764466 shutdown_connections 2024-04-17T07:23:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:23:51.447+0000 7fd5a9800640 1 -- 172.21.15.86:0/3338764466 wait complete. 2024-04-17T07:23:51.476 INFO:tasks.cephfs.filesystem:scrub status for tag:6a417886-ce5a-4eb5-a906-6bab81792086 - {'path': '/', 'tag': '6a417886-ce5a-4eb5-a906-6bab81792086', 'options': 'recursive,force'} 2024-04-17T07:23:51.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:51 smithi098 ceph-mon[35418]: pgmap v724: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 728 op/s 2024-04-17T07:23:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:51 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:23:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:51 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:51 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:23:51.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:51 smithi103 ceph-mon[35495]: pgmap v724: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 728 op/s 2024-04-17T07:23:51.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:51 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:23:51.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:51 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:51.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:51 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:23:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:52 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:52 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:52.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:52 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:23:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:53 smithi098 ceph-mon[35418]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.5 MiB/s wr, 768 op/s 2024-04-17T07:23:53.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:53 smithi103 ceph-mon[35495]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.5 MiB/s wr, 768 op/s 2024-04-17T07:23:53.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:53 smithi086 ceph-mon[31733]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.5 MiB/s wr, 768 op/s 2024-04-17T07:23:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 270.0 s, 673.6 tps, lat 148.417 ms stddev 43.884 2024-04-17T07:23:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:54 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:23:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:54 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:23:54.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:54 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:23:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:55 smithi098 ceph-mon[35418]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.3 MiB/s wr, 748 op/s 2024-04-17T07:23:55.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:55 smithi103 ceph-mon[35495]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.3 MiB/s wr, 748 op/s 2024-04-17T07:23:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:55 smithi086 ceph-mon[31733]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.3 MiB/s wr, 748 op/s 2024-04-17T07:23:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:57 smithi098 ceph-mon[35418]: pgmap v727: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-04-17T07:23:57.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:57 smithi103 ceph-mon[35495]: pgmap v727: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-04-17T07:23:57.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:57 smithi086 ceph-mon[31733]: pgmap v727: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 646 op/s 2024-04-17T07:23:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 275.0 s, 713.0 tps, lat 140.143 ms stddev 36.617 2024-04-17T07:23:59.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:23:59 smithi098 ceph-mon[35418]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.8 MiB/s wr, 725 op/s 2024-04-17T07:23:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:23:59 smithi103 ceph-mon[35495]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.8 MiB/s wr, 725 op/s 2024-04-17T07:23:59.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:23:59 smithi086 ceph-mon[31733]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.8 MiB/s wr, 725 op/s 2024-04-17T07:24:01.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:01 smithi098 ceph-mon[35418]: pgmap v729: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.6 MiB/s wr, 697 op/s 2024-04-17T07:24:01.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:01 smithi103 ceph-mon[35495]: pgmap v729: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.6 MiB/s wr, 697 op/s 2024-04-17T07:24:01.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:01 smithi086 ceph-mon[31733]: pgmap v729: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.6 MiB/s wr, 697 op/s 2024-04-17T07:24:03.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:03 smithi098 ceph-mon[35418]: pgmap v730: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.1 MiB/s wr, 753 op/s 2024-04-17T07:24:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:03 smithi103 ceph-mon[35495]: pgmap v730: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.1 MiB/s wr, 753 op/s 2024-04-17T07:24:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:03 smithi086 ceph-mon[31733]: pgmap v730: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.1 MiB/s wr, 753 op/s 2024-04-17T07:24:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 280.0 s, 699.4 tps, lat 142.987 ms stddev 84.049 2024-04-17T07:24:05.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:05 smithi098 ceph-mon[35418]: pgmap v731: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 772 op/s 2024-04-17T07:24:05.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:05 smithi103 ceph-mon[35495]: pgmap v731: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 772 op/s 2024-04-17T07:24:05.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:05 smithi086 ceph-mon[31733]: pgmap v731: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 772 op/s 2024-04-17T07:24:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:07 smithi098 ceph-mon[35418]: pgmap v732: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 673 op/s 2024-04-17T07:24:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:07 smithi103 ceph-mon[35495]: pgmap v732: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 673 op/s 2024-04-17T07:24:07.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:07 smithi086 ceph-mon[31733]: pgmap v732: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 673 op/s 2024-04-17T07:24:09.362 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 285.0 s, 725.2 tps, lat 137.792 ms stddev 35.366 2024-04-17T07:24:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:09 smithi098 ceph-mon[35418]: pgmap v733: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.2 MiB/s wr, 775 op/s 2024-04-17T07:24:09.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:09 smithi103 ceph-mon[35495]: pgmap v733: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.2 MiB/s wr, 775 op/s 2024-04-17T07:24:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:09 smithi086 ceph-mon[31733]: pgmap v733: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.2 MiB/s wr, 775 op/s 2024-04-17T07:24:11.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:11 smithi103 ceph-mon[35495]: pgmap v734: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-04-17T07:24:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:11 smithi086 ceph-mon[31733]: pgmap v734: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-04-17T07:24:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:11 smithi098 ceph-mon[35418]: pgmap v734: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 773 op/s 2024-04-17T07:24:13.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:13 smithi103 ceph-mon[35495]: pgmap v735: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.0 MiB/s wr, 768 op/s 2024-04-17T07:24:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:13 smithi086 ceph-mon[31733]: pgmap v735: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.0 MiB/s wr, 768 op/s 2024-04-17T07:24:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:13 smithi098 ceph-mon[35418]: pgmap v735: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.0 MiB/s wr, 768 op/s 2024-04-17T07:24:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 290.0 s, 699.0 tps, lat 142.662 ms stddev 39.389 2024-04-17T07:24:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:15 smithi103 ceph-mon[35495]: pgmap v736: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 805 op/s 2024-04-17T07:24:15.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:15 smithi086 ceph-mon[31733]: pgmap v736: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 805 op/s 2024-04-17T07:24:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:15 smithi098 ceph-mon[35418]: pgmap v736: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 805 op/s 2024-04-17T07:24:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:17 smithi103 ceph-mon[35495]: pgmap v737: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-04-17T07:24:17.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:17 smithi086 ceph-mon[31733]: pgmap v737: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-04-17T07:24:17.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:17 smithi098 ceph-mon[35418]: pgmap v737: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-04-17T07:24:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 295.0 s, 667.4 tps, lat 149.479 ms stddev 48.175 2024-04-17T07:24:19.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:19 smithi086 ceph-mon[31733]: pgmap v738: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.1 MiB/s wr, 757 op/s 2024-04-17T07:24:19.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:19 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:24:19.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:19 smithi098 ceph-mon[35418]: pgmap v738: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.1 MiB/s wr, 757 op/s 2024-04-17T07:24:19.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:19 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:24:19.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:19 smithi103 ceph-mon[35495]: pgmap v738: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.1 MiB/s wr, 757 op/s 2024-04-17T07:24:19.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:19 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:24:21.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:24:21.477 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:24:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:21 smithi086 ceph-mon[31733]: pgmap v739: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.8 MiB/s wr, 723 op/s 2024-04-17T07:24:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:24:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:24:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:24:21.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:21 smithi098 ceph-mon[35418]: pgmap v739: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.8 MiB/s wr, 723 op/s 2024-04-17T07:24:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:24:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:24:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:24:21.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.777+0000 7f55b3400640 1 -- 172.21.15.86:0/3429356026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac100810 msgr2=0x7f55ac100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.777+0000 7f55b3400640 1 --2- 172.21.15.86:0/3429356026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac100810 0x7f55ac100c30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5598009900 tx=0x7f559802bc30 comp rx=0 tx=0).stop 2024-04-17T07:24:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.777+0000 7f55b3400640 1 -- 172.21.15.86:0/3429356026 shutdown_connections 2024-04-17T07:24:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.777+0000 7f55b3400640 1 --2- 172.21.15.86:0/3429356026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55ac101b10 0x7f55ac101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.777+0000 7f55b3400640 1 --2- 172.21.15.86:0/3429356026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac101170 0x7f55ac1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.777+0000 7f55b3400640 1 --2- 172.21.15.86:0/3429356026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac100810 0x7f55ac100c30 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.777+0000 7f55b3400640 1 -- 172.21.15.86:0/3429356026 >> 172.21.15.86:0/3429356026 conn(0x7f55ac0fc450 msgr2=0x7f55ac0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.777+0000 7f55b3400640 1 -- 172.21.15.86:0/3429356026 shutdown_connections 2024-04-17T07:24:21.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.778+0000 7f55b3400640 1 -- 172.21.15.86:0/3429356026 wait complete. 2024-04-17T07:24:21.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:21 smithi103 ceph-mon[35495]: pgmap v739: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.8 MiB/s wr, 723 op/s 2024-04-17T07:24:21.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:24:21.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:24:21.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:24:21.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.781+0000 7f55b3400640 1 Processor -- start 2024-04-17T07:24:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.782+0000 7f55b3400640 1 -- start start 2024-04-17T07:24:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.782+0000 7f55b3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55ac100810 0x7f55ac19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.782+0000 7f55b3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac101170 0x7f55ac19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.782+0000 7f55b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac101b10 0x7f55ac1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.782+0000 7f55b3400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac10fcc0 con 0x7f55ac101b10 2024-04-17T07:24:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.782+0000 7f55b3400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac10fb40 con 0x7f55ac100810 2024-04-17T07:24:21.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.782+0000 7f55b3400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac10fe40 con 0x7f55ac101170 2024-04-17T07:24:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.783+0000 7f55b0a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55ac100810 0x7f55ac19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.783+0000 7f55b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac101b10 0x7f55ac1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.783+0000 7f55b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac101b10 0x7f55ac1a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37884/0 (socket says 172.21.15.86:37884) 2024-04-17T07:24:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.783+0000 7f55b1400640 1 -- 172.21.15.86:0/1604433977 learned_addr learned my addr 172.21.15.86:0/1604433977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:21.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.784+0000 7f55abe00640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac101170 0x7f55ac19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.785+0000 7f55b0a00640 1 -- 172.21.15.86:0/1604433977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac101170 msgr2=0x7f55ac19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.785+0000 7f55b0a00640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac101170 0x7f55ac19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.785+0000 7f55b0a00640 1 -- 172.21.15.86:0/1604433977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac101b10 msgr2=0x7f55ac1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.785+0000 7f55b0a00640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac101b10 0x7f55ac1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.785+0000 7f55b0a00640 1 -- 172.21.15.86:0/1604433977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ac06bab0 con 0x7f55ac100810 2024-04-17T07:24:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.785+0000 7f55b1400640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac101b10 0x7f55ac1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:21.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.785+0000 7f55b0a00640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55ac100810 0x7f55ac19a420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5598000c00 tx=0x7f5598004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.786+0000 7f55a9600640 1 -- 172.21.15.86:0/1604433977 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598043020 con 0x7f55ac100810 2024-04-17T07:24:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.786+0000 7f55a9600640 1 -- 172.21.15.86:0/1604433977 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5598004aa0 con 0x7f55ac100810 2024-04-17T07:24:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.786+0000 7f55a9600640 1 -- 172.21.15.86:0/1604433977 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598003ca0 con 0x7f55ac100810 2024-04-17T07:24:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.787+0000 7f55b3400640 1 -- 172.21.15.86:0/1604433977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ac06bd40 con 0x7f55ac100810 2024-04-17T07:24:21.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.787+0000 7f55b3400640 1 -- 172.21.15.86:0/1604433977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ac06c240 con 0x7f55ac100810 2024-04-17T07:24:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.792+0000 7f55b6042740 1 Processor -- start 2024-04-17T07:24:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.792+0000 7f55b6042740 1 -- start start 2024-04-17T07:24:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.792+0000 7f55b6042740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 0x5555ea6885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.792+0000 7f55b6042740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea68d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.792+0000 7f55b6042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea68dcf0 0x5555ea690180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.792+0000 7f55b6042740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5555ea5ef120 con 0x5555ea688b80 2024-04-17T07:24:21.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.792+0000 7f55b6042740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5555ea595340 con 0x5555ea688190 2024-04-17T07:24:21.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.792+0000 7f55b6042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5555ea5eb0e0 con 0x5555ea68dcf0 2024-04-17T07:24:21.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.793+0000 7f55b1400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 0x5555ea6885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.793+0000 7f55b1400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 0x5555ea6885b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42258/0 (socket says 172.21.15.86:42258) 2024-04-17T07:24:21.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.793+0000 7f55b1400640 1 -- 172.21.15.86:0/495130092 learned_addr learned my addr 172.21.15.86:0/495130092 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:21.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.793+0000 7f55a9600640 1 -- 172.21.15.86:0/1604433977 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5598046020 con 0x7f55ac100810 2024-04-17T07:24:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.793+0000 7f55b1400640 1 -- 172.21.15.86:0/495130092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea68dcf0 msgr2=0x5555ea690180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.793+0000 7f55b1400640 1 --2- 172.21.15.86:0/495130092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea68dcf0 0x5555ea690180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.793+0000 7f55b1400640 1 -- 172.21.15.86:0/495130092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 msgr2=0x5555ea68d680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:24:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.794+0000 7f55b1400640 1 --2- 172.21.15.86:0/495130092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea68d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.794+0000 7f55b1400640 1 -- 172.21.15.86:0/495130092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ea3cb7d0 con 0x5555ea688190 2024-04-17T07:24:21.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.794+0000 7f55b1400640 1 --2- 172.21.15.86:0/495130092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 0x5555ea6885b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f559c010550 tx=0x7f559c00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2081f57e66311510 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:21.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.795+0000 7f55a1c00640 1 -- 172.21.15.86:0/495130092 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c011760 con 0x5555ea688190 2024-04-17T07:24:21.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.795+0000 7f55a1c00640 1 -- 172.21.15.86:0/495130092 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f559c011db0 con 0x5555ea688190 2024-04-17T07:24:21.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.795+0000 7f55a1c00640 1 -- 172.21.15.86:0/495130092 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c00edc0 con 0x5555ea688190 2024-04-17T07:24:21.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.795+0000 7f55a9600640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f55900a3c30 0x7f55900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.796+0000 7f55a9600640 1 -- 172.21.15.86:0/1604433977 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f55980f2930 con 0x7f55ac100810 2024-04-17T07:24:21.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.797+0000 7f55b0a00640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f55900a3c30 0x7f55900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.799+0000 7f55b6042740 1 -- 172.21.15.86:0/495130092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 msgr2=0x5555ea6885b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:21.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.799+0000 7f55b6042740 1 --2- 172.21.15.86:0/495130092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 0x5555ea6885b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f559c010550 tx=0x7f559c00bf90 comp rx=0 tx=0).stop 2024-04-17T07:24:21.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.799+0000 7f55b0a00640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f55900a3c30 0x7f55900a60f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f55980c6fd0 tx=0x7f55980f2c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.800+0000 7f55b6042740 1 -- 172.21.15.86:0/495130092 shutdown_connections 2024-04-17T07:24:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.800+0000 7f55b6042740 1 --2- 172.21.15.86:0/495130092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea68dcf0 0x5555ea690180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.800+0000 7f55b6042740 1 --2- 172.21.15.86:0/495130092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea68d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.800+0000 7f55b6042740 1 --2- 172.21.15.86:0/495130092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 0x5555ea6885b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.800+0000 7f55b6042740 1 -- 172.21.15.86:0/495130092 >> 172.21.15.86:0/495130092 conn(0x5555ea683700 msgr2=0x5555ea683b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:21.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.802+0000 7f55b6042740 1 -- 172.21.15.86:0/495130092 shutdown_connections 2024-04-17T07:24:21.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.803+0000 7f55b6042740 1 -- 172.21.15.86:0/495130092 wait complete. 2024-04-17T07:24:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.803+0000 7f55b6042740 1 -- start start 2024-04-17T07:24:21.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.804+0000 7f55b6042740 1 Processor -- start 2024-04-17T07:24:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.805+0000 7f55b6042740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 0x5555ea69aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.805+0000 7f55b6042740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea69b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.805+0000 7f55b6042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea68dcf0 0x5555ea6a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.805+0000 7f55b6042740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5555ea690ca0 con 0x5555ea688b80 2024-04-17T07:24:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.805+0000 7f55b6042740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5555ea5a8710 con 0x5555ea688190 2024-04-17T07:24:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.805+0000 7f55b6042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5555ea690e20 con 0x5555ea68dcf0 2024-04-17T07:24:21.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.805+0000 7f55b6042740 10 client.0 _ll_register_callbacks cb 0x5555ea5a3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.806+0000 7f55b1400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 0x5555ea69aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.806+0000 7f55abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea69b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.806+0000 7f55abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea69b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37904/0 (socket says 172.21.15.86:37904) 2024-04-17T07:24:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.806+0000 7f55abe00640 1 -- 172.21.15.86:0/296488521 learned_addr learned my addr 172.21.15.86:0/296488521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.806+0000 7f55b1400640 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea68dcf0 0x5555ea6a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.807+0000 7f55abe00640 1 -- 172.21.15.86:0/296488521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea68dcf0 msgr2=0x5555ea6a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.807+0000 7f55abe00640 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea68dcf0 0x5555ea6a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.807+0000 7f55abe00640 1 -- 172.21.15.86:0/296488521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 msgr2=0x5555ea69aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:21.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.807+0000 7f55abe00640 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 0x5555ea69aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.807+0000 7f55abe00640 1 -- 172.21.15.86:0/296488521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ea3013f0 con 0x5555ea688b80 2024-04-17T07:24:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.807+0000 7f55abe00640 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea69b430 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f55a4010670 tx=0x7f55a4010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676d2233be0defb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.807+0000 7f5596000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:24:21.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.808+0000 7f5596000640 1 -- 172.21.15.86:0/296488521 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a400ea70 con 0x5555ea688b80 2024-04-17T07:24:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.808+0000 7f5596000640 1 -- 172.21.15.86:0/296488521 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f55a400ec10 con 0x5555ea688b80 2024-04-17T07:24:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.808+0000 7f55b6042740 1 -- 172.21.15.86:0/296488521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559c011db0 con 0x5555ea688b80 2024-04-17T07:24:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.809+0000 7f5596000640 1 -- 172.21.15.86:0/296488521 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55a4025050 con 0x5555ea688b80 2024-04-17T07:24:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.809+0000 7f55b6042740 10 client.17901 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:21.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.809+0000 7f55b6042740 1 -- 172.21.15.86:0/296488521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555ea2c10f0 con 0x5555ea688b80 2024-04-17T07:24:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.810+0000 7f5596000640 1 -- 172.21.15.86:0/296488521 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f559c011db0 con 0x5555ea688b80 2024-04-17T07:24:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.810+0000 7f5596000640 1 -- 172.21.15.86:0/296488521 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f55a400f040 con 0x5555ea688b80 2024-04-17T07:24:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.810+0000 7f55b6042740 10 client.17901 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:21.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.810+0000 7f55b6042740 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:24:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.810+0000 7f55b6042740 20 client.17901 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.810+0000 7f55b6042740 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5555ea6a6d90 0x5555ea6a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.810+0000 7f55b6042740 4 client.17901 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.810+0000 7f55b6042740 1 -- 172.21.15.86:0/296488521 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555ea5fcb40 con 0x5555ea6a6d90 2024-04-17T07:24:21.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.811+0000 7f55b1400640 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5555ea6a6d90 0x5555ea6a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.812+0000 7f55b1400640 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5555ea6a6d90 0x5555ea6a91f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d3f86fd16712dd8 server_cookie=acf23803f0b0fbbf in_seq=0 out_seq=0 2024-04-17T07:24:21.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.813+0000 7f5596000640 10 client.17901 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:21.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.815+0000 7f5596000640 1 -- 172.21.15.86:0/296488521 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555ea5fcb40 con 0x5555ea6a6d90 2024-04-17T07:24:21.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.815+0000 7f5596000640 10 client.17901 handle_command_reply: tid=0 2024-04-17T07:24:21.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.816+0000 7f55b6042740 1 client.17901 shutdown 2024-04-17T07:24:21.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.817+0000 7f55b6042740 1 -- 172.21.15.86:0/296488521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 msgr2=0x5555ea69b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.817+0000 7f55b6042740 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea69b430 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f55a4010670 tx=0x7f55a4010b30 comp rx=0 tx=0).stop 2024-04-17T07:24:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.817+0000 7f55b6042740 1 -- 172.21.15.86:0/296488521 shutdown_connections 2024-04-17T07:24:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.817+0000 7f55b6042740 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5555ea6a6d90 0x5555ea6a91f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.817+0000 7f55b6042740 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea68dcf0 0x5555ea6a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.817+0000 7f5596000640 0 client.17901 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:21.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.817+0000 7f55b6042740 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea69b430 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.817+0000 7f55b6042740 1 --2- 172.21.15.86:0/296488521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688190 0x5555ea69aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.817+0000 7f55b6042740 1 -- 172.21.15.86:0/296488521 >> 172.21.15.86:0/296488521 conn(0x5555ea683700 msgr2=0x5555ea682890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.817+0000 7f55b6042740 1 -- 172.21.15.86:0/296488521 shutdown_connections 2024-04-17T07:24:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.818+0000 7f55b6042740 1 -- 172.21.15.86:0/296488521 wait complete. 2024-04-17T07:24:21.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.818+0000 7f55b6042740 20 client.17901 trim_cache size 0 max 16384 2024-04-17T07:24:21.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.858+0000 7f55b6042740 1 Processor -- start 2024-04-17T07:24:21.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.858+0000 7f55b6042740 1 -- start start 2024-04-17T07:24:21.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.858+0000 7f55b6042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea682710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.858+0000 7f55b6042740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea6a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.858+0000 7f55b6042740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea68dcf0 0x5555ea6adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.858+0000 7f55b6042740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac072430 con 0x5555ea688b80 2024-04-17T07:24:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.858+0000 7f55b6042740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac073f70 con 0x5555ea68dcf0 2024-04-17T07:24:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.858+0000 7f55b6042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac072b00 con 0x5555ea688190 2024-04-17T07:24:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.859+0000 7f55b1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea682710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.859+0000 7f55b1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea682710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45972/0 (socket says 172.21.15.86:45972) 2024-04-17T07:24:21.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.859+0000 7f55b1400640 1 -- 172.21.15.86:0/4040019453 learned_addr learned my addr 172.21.15.86:0/4040019453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.859+0000 7f55b1400640 1 --2- 172.21.15.86:0/4040019453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea68dcf0 0x5555ea6adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.859+0000 7f55abe00640 1 --2- 172.21.15.86:0/4040019453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea6a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.859+0000 7f55b1400640 1 -- 172.21.15.86:0/4040019453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea68dcf0 msgr2=0x5555ea6adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.859+0000 7f55b1400640 1 --2- 172.21.15.86:0/4040019453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea68dcf0 0x5555ea6adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.859+0000 7f55b1400640 1 -- 172.21.15.86:0/4040019453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 msgr2=0x5555ea6a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.859+0000 7f55b1400640 1 --2- 172.21.15.86:0/4040019453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea6a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.859+0000 7f55b1400640 1 -- 172.21.15.86:0/4040019453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ea689410 con 0x5555ea688190 2024-04-17T07:24:21.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.860+0000 7f55b1400640 1 --2- 172.21.15.86:0/4040019453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea682710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b3d0 tx=0x7f559c010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921b3164a62515f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:21.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.860+0000 7f55a1200640 1 -- 172.21.15.86:0/4040019453 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c005020 con 0x5555ea688190 2024-04-17T07:24:21.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.861+0000 7f55a1200640 1 -- 172.21.15.86:0/4040019453 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f559c004660 con 0x5555ea688190 2024-04-17T07:24:21.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.861+0000 7f55a1200640 1 -- 172.21.15.86:0/4040019453 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c0117c0 con 0x5555ea688190 2024-04-17T07:24:21.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.862+0000 7f55b6042740 1 -- 172.21.15.86:0/4040019453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 msgr2=0x5555ea682710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:21.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.862+0000 7f55b6042740 1 --2- 172.21.15.86:0/4040019453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea682710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f559c00b3d0 tx=0x7f559c010fd0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.862+0000 7f55b6042740 1 -- 172.21.15.86:0/4040019453 shutdown_connections 2024-04-17T07:24:21.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.862+0000 7f55b6042740 1 --2- 172.21.15.86:0/4040019453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea68dcf0 0x5555ea6adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.862+0000 7f55b6042740 1 --2- 172.21.15.86:0/4040019453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea688b80 0x5555ea6a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.862+0000 7f55b6042740 1 --2- 172.21.15.86:0/4040019453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea682710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.862+0000 7f55b6042740 1 -- 172.21.15.86:0/4040019453 >> 172.21.15.86:0/4040019453 conn(0x5555ea683700 msgr2=0x5555ea682000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:21.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.862+0000 7f55b6042740 1 -- 172.21.15.86:0/4040019453 shutdown_connections 2024-04-17T07:24:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.863+0000 7f55b6042740 1 -- 172.21.15.86:0/4040019453 wait complete. 2024-04-17T07:24:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.864+0000 7f55b6042740 1 -- start start 2024-04-17T07:24:21.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.864+0000 7f55b6042740 1 Processor -- start 2024-04-17T07:24:21.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.865+0000 7f55b6042740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea6a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.865+0000 7f55b6042740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688b80 0x5555ea6a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.865+0000 7f55b6042740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea68dcf0 0x5555ea6a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.865+0000 7f55b6042740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac0738a0 con 0x5555ea68dcf0 2024-04-17T07:24:21.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.865+0000 7f55b6042740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac0731d0 con 0x5555ea688b80 2024-04-17T07:24:21.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.865+0000 7f55b6042740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac070400 con 0x5555ea688190 2024-04-17T07:24:21.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.865+0000 7f55b6042740 10 client.0 _ll_register_callbacks cb 0x7f55a4009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:21.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.866+0000 7f55b1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea6a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.866+0000 7f55b1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea6a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45974/0 (socket says 172.21.15.86:45974) 2024-04-17T07:24:21.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.866+0000 7f55b1400640 1 -- 172.21.15.86:0/2414173345 learned_addr learned my addr 172.21.15.86:0/2414173345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:21.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.867+0000 7f55b1400640 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea68dcf0 0x5555ea6a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.868+0000 7f55abe00640 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688b80 0x5555ea6a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.868+0000 7f55b1400640 1 -- 172.21.15.86:0/2414173345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 msgr2=0x5555ea6a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:21.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.868+0000 7f55b1400640 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea6a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.868+0000 7f55b1400640 1 -- 172.21.15.86:0/2414173345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688b80 msgr2=0x5555ea6a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.868+0000 7f55b1400640 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688b80 0x5555ea6a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.869+0000 7f55b1400640 1 -- 172.21.15.86:0/2414173345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ea6a1060 con 0x5555ea68dcf0 2024-04-17T07:24:21.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.869+0000 7f55abe00640 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688b80 0x5555ea6a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:24:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.870+0000 7f55b1400640 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea68dcf0 0x5555ea6a0b20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f559c00bed0 tx=0x7f559c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f427e0f57ba8740b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.871+0000 7f5596000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:24:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.871+0000 7f5596000640 1 -- 172.21.15.86:0/2414173345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c036040 con 0x5555ea68dcf0 2024-04-17T07:24:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.871+0000 7f5596000640 1 -- 172.21.15.86:0/2414173345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f559c004810 con 0x5555ea68dcf0 2024-04-17T07:24:21.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.871+0000 7f55b6042740 1 -- 172.21.15.86:0/2414173345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559c004660 con 0x5555ea68dcf0 2024-04-17T07:24:21.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.873+0000 7f5596000640 1 -- 172.21.15.86:0/2414173345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f559c0054c0 con 0x5555ea68dcf0 2024-04-17T07:24:21.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.873+0000 7f55b6042740 10 client.17904 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:21.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.873+0000 7f55b6042740 1 -- 172.21.15.86:0/2414173345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555ea70dfe0 con 0x5555ea68dcf0 2024-04-17T07:24:21.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.874+0000 7f5596000640 1 -- 172.21.15.86:0/2414173345 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f559c004660 con 0x5555ea68dcf0 2024-04-17T07:24:21.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.874+0000 7f5596000640 1 -- 172.21.15.86:0/2414173345 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f559c011bf0 con 0x5555ea68dcf0 2024-04-17T07:24:21.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.874+0000 7f55b6042740 10 client.17904 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:21.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.874+0000 7f55b6042740 10 client.17904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:24:21.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.874+0000 7f55b6042740 20 client.17904 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:21.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.874+0000 7f55b6042740 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5555ea6a6d90 0x5555ea710780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.874+0000 7f55b6042740 4 client.17904 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:24:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.874+0000 7f55b6042740 1 -- 172.21.15.86:0/2414173345 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5555ea5f69b0 con 0x5555ea6a6d90 2024-04-17T07:24:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.875+0000 7f55abe00640 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5555ea6a6d90 0x5555ea710780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.877+0000 7f5596000640 10 client.17904 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.877+0000 7f55abe00640 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5555ea6a6d90 0x5555ea710780 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813e8b1bdd459890 server_cookie=a75cb4a0d13d10c1 in_seq=0 out_seq=0 2024-04-17T07:24:21.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.877+0000 7f5596000640 1 -- 172.21.15.86:0/2414173345 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5555ea5f69b0 con 0x5555ea6a6d90 2024-04-17T07:24:21.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.877+0000 7f5596000640 10 client.17904 handle_command_reply: tid=0 2024-04-17T07:24:21.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.877+0000 7f55b6042740 1 client.17904 shutdown 2024-04-17T07:24:21.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.877+0000 7f55b6042740 1 -- 172.21.15.86:0/2414173345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea68dcf0 msgr2=0x5555ea6a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:21.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.877+0000 7f55b6042740 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea68dcf0 0x5555ea6a0b20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f559c00bed0 tx=0x7f559c00bf90 comp rx=0 tx=0).stop 2024-04-17T07:24:21.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.878+0000 7f55b6042740 1 -- 172.21.15.86:0/2414173345 shutdown_connections 2024-04-17T07:24:21.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.878+0000 7f55b6042740 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5555ea6a6d90 0x5555ea710780 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.878+0000 7f55b6042740 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5555ea68dcf0 0x5555ea6a0b20 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.878+0000 7f55b6042740 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5555ea688b80 0x5555ea6a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.878+0000 7f55b6042740 1 --2- 172.21.15.86:0/2414173345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555ea688190 0x5555ea6a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.878+0000 7f55b6042740 1 -- 172.21.15.86:0/2414173345 >> 172.21.15.86:0/2414173345 conn(0x5555ea683700 msgr2=0x5555ea682890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:21.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.878+0000 7f5596000640 0 client.17904 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:21.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.878+0000 7f55b6042740 1 -- 172.21.15.86:0/2414173345 shutdown_connections 2024-04-17T07:24:21.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.879+0000 7f55b6042740 1 -- 172.21.15.86:0/2414173345 wait complete. 2024-04-17T07:24:21.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.879+0000 7f55b6042740 20 client.17904 trim_cache size 0 max 16384 2024-04-17T07:24:21.879 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:24:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:24:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:24:21.879 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.880+0000 7f55b3400640 1 -- 172.21.15.86:0/1604433977 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f55900a3c30 msgr2=0x7f55900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.880+0000 7f55b3400640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f55900a3c30 0x7f55900a60f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f55980c6fd0 tx=0x7f55980f2c00 comp rx=0 tx=0).stop 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.880+0000 7f55b3400640 1 -- 172.21.15.86:0/1604433977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55ac100810 msgr2=0x7f55ac19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.880+0000 7f55b3400640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55ac100810 0x7f55ac19a420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5598000c00 tx=0x7f5598004270 comp rx=0 tx=0).stop 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.880+0000 7f55b3400640 1 -- 172.21.15.86:0/1604433977 shutdown_connections 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.880+0000 7f55b3400640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f55900a3c30 0x7f55900a60f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.880+0000 7f55b3400640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac101b10 0x7f55ac1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.880+0000 7f55b3400640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac101170 0x7f55ac19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.880+0000 7f55b3400640 1 --2- 172.21.15.86:0/1604433977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55ac100810 0x7f55ac19a420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.880+0000 7f55b3400640 1 -- 172.21.15.86:0/1604433977 >> 172.21.15.86:0/1604433977 conn(0x7f55ac0fc450 msgr2=0x7f55ac0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.881+0000 7f55b3400640 1 -- 172.21.15.86:0/1604433977 shutdown_connections 2024-04-17T07:24:21.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:21.881+0000 7f55b3400640 1 -- 172.21.15.86:0/1604433977 wait complete. 2024-04-17T07:24:21.904 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:24:21.904 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:24:22.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.209+0000 7f138b200640 1 -- 172.21.15.86:0/2119744890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13841097b0 msgr2=0x7f1384109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.209+0000 7f138b200640 1 --2- 172.21.15.86:0/2119744890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13841097b0 0x7f1384109c30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f137c009ca0 tx=0x7f137c02c120 comp rx=0 tx=0).stop 2024-04-17T07:24:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.209+0000 7f138b200640 1 -- 172.21.15.86:0/2119744890 shutdown_connections 2024-04-17T07:24:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.209+0000 7f138b200640 1 --2- 172.21.15.86:0/2119744890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f138410a170 0x7f1384110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.209+0000 7f138b200640 1 --2- 172.21.15.86:0/2119744890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13841097b0 0x7f1384109c30 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.209+0000 7f138b200640 1 --2- 172.21.15.86:0/2119744890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13841085c0 0x7f13841089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.209+0000 7f138b200640 1 -- 172.21.15.86:0/2119744890 >> 172.21.15.86:0/2119744890 conn(0x7f1384077580 msgr2=0x7f13840799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.209+0000 7f138b200640 1 -- 172.21.15.86:0/2119744890 shutdown_connections 2024-04-17T07:24:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.209+0000 7f138b200640 1 -- 172.21.15.86:0/2119744890 wait complete. 2024-04-17T07:24:22.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.211+0000 7f138b200640 1 Processor -- start 2024-04-17T07:24:22.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.211+0000 7f138b200640 1 -- start start 2024-04-17T07:24:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.212+0000 7f138b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13841085c0 0x7f138419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.212+0000 7f138b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13841097b0 0x7f138419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.212+0000 7f138b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f138410a170 0x7f13841a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.212+0000 7f138b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1384114230 con 0x7f13841085c0 2024-04-17T07:24:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.212+0000 7f138b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f13841140b0 con 0x7f138410a170 2024-04-17T07:24:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.212+0000 7f138b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13841143b0 con 0x7f13841097b0 2024-04-17T07:24:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.212+0000 7f1383e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13841085c0 0x7f138419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.212+0000 7f1383e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13841085c0 0x7f138419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37924/0 (socket says 172.21.15.86:37924) 2024-04-17T07:24:22.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.212+0000 7f1383e00640 1 -- 172.21.15.86:0/50835334 learned_addr learned my addr 172.21.15.86:0/50835334 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:22.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.213+0000 7f1383400640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13841097b0 0x7f138419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:22.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.213+0000 7f1389200640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f138410a170 0x7f13841a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:22.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.213+0000 7f1383400640 1 -- 172.21.15.86:0/50835334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f138410a170 msgr2=0x7f13841a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:22.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.214+0000 7f1383400640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f138410a170 0x7f13841a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:22.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.214+0000 7f1383400640 1 -- 172.21.15.86:0/50835334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13841085c0 msgr2=0x7f138419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:22.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.214+0000 7f1383400640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13841085c0 0x7f138419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:22.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.214+0000 7f1383400640 1 -- 172.21.15.86:0/50835334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13841a6470 con 0x7f13841097b0 2024-04-17T07:24:22.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.214+0000 7f1383e00640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13841085c0 0x7f138419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:22.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.214+0000 7f1383400640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13841097b0 0x7f138419ed90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f137c009bf0 tx=0x7f137c034830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.216+0000 7f1380c00640 1 -- 172.21.15.86:0/50835334 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c039040 con 0x7f13841097b0 2024-04-17T07:24:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.216+0000 7f1380c00640 1 -- 172.21.15.86:0/50835334 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f137c044070 con 0x7f13841097b0 2024-04-17T07:24:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.216+0000 7f1380c00640 1 -- 172.21.15.86:0/50835334 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c004840 con 0x7f13841097b0 2024-04-17T07:24:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.216+0000 7f138b200640 1 -- 172.21.15.86:0/50835334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13841a6700 con 0x7f13841097b0 2024-04-17T07:24:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.216+0000 7f138b200640 1 -- 172.21.15.86:0/50835334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13841a6b70 con 0x7f13841097b0 2024-04-17T07:24:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.218+0000 7f138b200640 1 -- 172.21.15.86:0/50835334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138410f880 con 0x7f13841097b0 2024-04-17T07:24:22.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.223+0000 7f1380c00640 1 -- 172.21.15.86:0/50835334 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f137c004040 con 0x7f13841097b0 2024-04-17T07:24:22.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.226+0000 7f1380c00640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13680a3d40 0x7f13680a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:22.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.227+0000 7f1383e00640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13680a3d40 0x7f13680a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:22.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.227+0000 7f1380c00640 1 -- 172.21.15.86:0/50835334 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f137c0f30d0 con 0x7f13841097b0 2024-04-17T07:24:22.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.229+0000 7f1383e00640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13680a3d40 0x7f13680a6200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1370006fd0 tx=0x7f1370009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:22.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.234+0000 7f1380c00640 1 -- 172.21.15.86:0/50835334 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f137c0bf7b0 con 0x7f13841097b0 2024-04-17T07:24:22.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.743+0000 7f138b200640 1 -- 172.21.15.86:0/50835334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f138402d820 con 0x7f13841097b0 2024-04-17T07:24:22.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.747+0000 7f1380c00640 1 -- 172.21.15.86:0/50835334 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f137c0bf0b0 con 0x7f13841097b0 2024-04-17T07:24:22.747 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:24:22.747 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:24:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:24:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.751+0000 7f138b200640 1 -- 172.21.15.86:0/50835334 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13680a3d40 msgr2=0x7f13680a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.751+0000 7f138b200640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13680a3d40 0x7f13680a6200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1370006fd0 tx=0x7f1370009190 comp rx=0 tx=0).stop 2024-04-17T07:24:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.751+0000 7f138b200640 1 -- 172.21.15.86:0/50835334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13841097b0 msgr2=0x7f138419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:22.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.751+0000 7f138b200640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13841097b0 0x7f138419ed90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f137c009bf0 tx=0x7f137c034830 comp rx=0 tx=0).stop 2024-04-17T07:24:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.751+0000 7f138b200640 1 -- 172.21.15.86:0/50835334 shutdown_connections 2024-04-17T07:24:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.751+0000 7f138b200640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f13680a3d40 0x7f13680a6200 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.751+0000 7f138b200640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f138410a170 0x7f13841a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.751+0000 7f138b200640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13841097b0 0x7f138419ed90 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f137c009bf0 tx=0x7f137c034830 comp rx=0 tx=0).stop 2024-04-17T07:24:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.751+0000 7f138b200640 1 --2- 172.21.15.86:0/50835334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13841085c0 0x7f138419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.751+0000 7f138b200640 1 -- 172.21.15.86:0/50835334 >> 172.21.15.86:0/50835334 conn(0x7f1384077580 msgr2=0x7f138410ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:22.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.752+0000 7f138b200640 1 -- 172.21.15.86:0/50835334 shutdown_connections 2024-04-17T07:24:22.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:22.753+0000 7f138b200640 1 -- 172.21.15.86:0/50835334 wait complete. 2024-04-17T07:24:22.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:24:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.115+0000 7f9a70c00640 1 -- 172.21.15.86:0/3599977706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6409c370 msgr2=0x7f9a6409c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.115+0000 7f9a70c00640 1 --2- 172.21.15.86:0/3599977706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6409c370 0x7f9a6409c790 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009900 tx=0x7f9a5802bc40 comp rx=0 tx=0).stop 2024-04-17T07:24:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.116+0000 7f9a70c00640 1 -- 172.21.15.86:0/3599977706 shutdown_connections 2024-04-17T07:24:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.116+0000 7f9a70c00640 1 --2- 172.21.15.86:0/3599977706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a6409df20 0x7f9a640a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.116+0000 7f9a70c00640 1 --2- 172.21.15.86:0/3599977706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6409d560 0x7f9a6409d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.116+0000 7f9a70c00640 1 --2- 172.21.15.86:0/3599977706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6409c370 0x7f9a6409c790 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.116+0000 7f9a70c00640 1 -- 172.21.15.86:0/3599977706 >> 172.21.15.86:0/3599977706 conn(0x7f9a6400b4f0 msgr2=0x7f9a6400d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.116+0000 7f9a70c00640 1 -- 172.21.15.86:0/3599977706 shutdown_connections 2024-04-17T07:24:23.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.116+0000 7f9a70c00640 1 -- 172.21.15.86:0/3599977706 wait complete. 2024-04-17T07:24:23.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.117+0000 7f9a70c00640 1 Processor -- start 2024-04-17T07:24:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.118+0000 7f9a70c00640 1 -- start start 2024-04-17T07:24:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.118+0000 7f9a70c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a6409c370 0x7f9a640130c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.118+0000 7f9a70c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6409d560 0x7f9a64013600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.118+0000 7f9a70c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6409df20 0x7f9a64142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.118+0000 7f9a70c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a640a7f60 con 0x7f9a6409df20 2024-04-17T07:24:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.118+0000 7f9a70c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a640a7de0 con 0x7f9a6409d560 2024-04-17T07:24:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.118+0000 7f9a70c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a640a80e0 con 0x7f9a6409c370 2024-04-17T07:24:23.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.118+0000 7f9a6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6409df20 0x7f9a64142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.119+0000 7f9a6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6409df20 0x7f9a64142670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37934/0 (socket says 172.21.15.86:37934) 2024-04-17T07:24:23.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.119+0000 7f9a6be00640 1 -- 172.21.15.86:0/2464944765 learned_addr learned my addr 172.21.15.86:0/2464944765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.119+0000 7f9a6b400640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a6409c370 0x7f9a640130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.119+0000 7f9a6aa00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6409d560 0x7f9a64013600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.119+0000 7f9a6be00640 1 -- 172.21.15.86:0/2464944765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a6409c370 msgr2=0x7f9a640130c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.119+0000 7f9a6be00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a6409c370 0x7f9a640130c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.119+0000 7f9a6be00640 1 -- 172.21.15.86:0/2464944765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6409d560 msgr2=0x7f9a64013600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.119+0000 7f9a6be00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6409d560 0x7f9a64013600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.119+0000 7f9a6be00640 1 -- 172.21.15.86:0/2464944765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a64142bb0 con 0x7f9a6409df20 2024-04-17T07:24:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.120+0000 7f9a6be00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6409df20 0x7f9a64142670 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c050aa0 tx=0x7f9a6c06adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.120+0000 7f9a5fe00640 1 -- 172.21.15.86:0/2464944765 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c081070 con 0x7f9a6409df20 2024-04-17T07:24:23.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.120+0000 7f9a5fe00640 1 -- 172.21.15.86:0/2464944765 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a6c07ac10 con 0x7f9a6409df20 2024-04-17T07:24:23.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.120+0000 7f9a5fe00640 1 -- 172.21.15.86:0/2464944765 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c07c610 con 0x7f9a6409df20 2024-04-17T07:24:23.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.120+0000 7f9a70c00640 1 -- 172.21.15.86:0/2464944765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a64142de0 con 0x7f9a6409df20 2024-04-17T07:24:23.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.120+0000 7f9a70c00640 1 -- 172.21.15.86:0/2464944765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a64143250 con 0x7f9a6409df20 2024-04-17T07:24:23.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.126+0000 7f9a74fdd740 1 Processor -- start 2024-04-17T07:24:23.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.131+0000 7f9a5fe00640 1 -- 172.21.15.86:0/2464944765 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9a6c084070 con 0x7f9a6409df20 2024-04-17T07:24:23.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.132+0000 7f9a74fdd740 1 -- start start 2024-04-17T07:24:23.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.133+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946cca5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.133+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 0x55946ccaa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.133+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 0x55946ccad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.133+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55946cc0b020 con 0x55946ccaacc0 2024-04-17T07:24:23.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.133+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55946cbb1460 con 0x55946cca5160 2024-04-17T07:24:23.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.133+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55946cc06fe0 con 0x55946cca5b50 2024-04-17T07:24:23.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.134+0000 7f9a6aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946cca5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.134+0000 7f9a6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 0x55946ccad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.134+0000 7f9a6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 0x55946ccad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37936/0 (socket says 172.21.15.86:37936) 2024-04-17T07:24:23.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.134+0000 7f9a6aa00640 1 -- 172.21.15.86:0/68312527 learned_addr learned my addr 172.21.15.86:0/68312527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:23.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.134+0000 7f9a6b400640 1 --2- 172.21.15.86:0/68312527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 0x55946ccaa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.135+0000 7f9a6b400640 1 -- 172.21.15.86:0/68312527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 msgr2=0x55946cca5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.135+0000 7f9a6b400640 1 --2- 172.21.15.86:0/68312527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946cca5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.135+0000 7f9a6b400640 1 -- 172.21.15.86:0/68312527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 msgr2=0x55946ccad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.135+0000 7f9a6b400640 1 --2- 172.21.15.86:0/68312527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 0x55946ccad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.135+0000 7f9a6b400640 1 -- 172.21.15.86:0/68312527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946cbb9040 con 0x55946cca5b50 2024-04-17T07:24:23.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.135+0000 7f9a5fe00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a540a3c30 0x7f9a540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.136+0000 7f9a6aa00640 1 --2- 172.21.15.86:0/68312527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 0x55946ccad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:23.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.136+0000 7f9a5fe00640 1 -- 172.21.15.86:0/2464944765 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9a6c13cd30 con 0x7f9a6409df20 2024-04-17T07:24:23.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.136+0000 7f9a6b400640 1 --2- 172.21.15.86:0/68312527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 0x55946ccaa650 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9a58036040 tx=0x7f9a5802c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d837c24f889047c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.137+0000 7f9a6aa00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a540a3c30 0x7f9a540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.137+0000 7f9a53e00640 1 -- 172.21.15.86:0/68312527 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58044020 con 0x55946cca5b50 2024-04-17T07:24:23.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.137+0000 7f9a53e00640 1 -- 172.21.15.86:0/68312527 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a5802ddd0 con 0x55946cca5b50 2024-04-17T07:24:23.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.137+0000 7f9a53e00640 1 -- 172.21.15.86:0/68312527 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58004920 con 0x55946cca5b50 2024-04-17T07:24:23.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.138+0000 7f9a74fdd740 1 -- 172.21.15.86:0/68312527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 msgr2=0x55946ccaa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.138+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/68312527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 0x55946ccaa650 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9a58036040 tx=0x7f9a5802c8e0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.138+0000 7f9a6aa00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a540a3c30 0x7f9a540a60f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55946cca63b0 tx=0x7f9a60004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.139+0000 7f9a74fdd740 1 -- 172.21.15.86:0/68312527 shutdown_connections 2024-04-17T07:24:23.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.139+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/68312527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 0x55946ccad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.139+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/68312527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 0x55946ccaa650 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.139+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/68312527 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946cca5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.139+0000 7f9a74fdd740 1 -- 172.21.15.86:0/68312527 >> 172.21.15.86:0/68312527 conn(0x55946cca06d0 msgr2=0x55946cca0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.143+0000 7f9a74fdd740 1 -- 172.21.15.86:0/68312527 shutdown_connections 2024-04-17T07:24:23.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.144+0000 7f9a74fdd740 1 -- 172.21.15.86:0/68312527 wait complete. 2024-04-17T07:24:23.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.146+0000 7f9a74fdd740 1 -- start start 2024-04-17T07:24:23.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.146+0000 7f9a74fdd740 1 Processor -- start 2024-04-17T07:24:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.149+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.150+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946cca5b50 0x55946ccafa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.150+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946ccaacc0 0x55946ccb69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.150+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55946ccadc10 con 0x55946cca5b50 2024-04-17T07:24:23.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.150+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55946cbc4830 con 0x55946cca5160 2024-04-17T07:24:23.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.150+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55946ccadd90 con 0x55946ccaacc0 2024-04-17T07:24:23.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.150+0000 7f9a74fdd740 10 client.0 _ll_register_callbacks cb 0x55946cb92fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.151+0000 7f9a6be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946ccaacc0 0x55946ccb69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.151+0000 7f9a6be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946ccaacc0 0x55946ccb69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46000/0 (socket says 172.21.15.86:46000) 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.151+0000 7f9a6be00640 1 -- 172.21.15.86:0/3044570652 learned_addr learned my addr 172.21.15.86:0/3044570652 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.151+0000 7f9a6aa00640 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946cca5b50 0x55946ccafa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.151+0000 7f9a6b400640 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.151+0000 7f9a6be00640 1 -- 172.21.15.86:0/3044570652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 msgr2=0x55946ccaf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.151+0000 7f9a6be00640 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.151+0000 7f9a6be00640 1 -- 172.21.15.86:0/3044570652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946cca5b50 msgr2=0x55946ccafa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.151+0000 7f9a6be00640 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946cca5b50 0x55946ccafa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.151+0000 7f9a6be00640 1 -- 172.21.15.86:0/3044570652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946cbc8920 con 0x55946ccaacc0 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.152+0000 7f9a6b400640 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:24:23.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.152+0000 7f9a6be00640 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946ccaacc0 0x55946ccb69f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c111370 tx=0x7f9a6c07ae80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbe17cdbf2366d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.152+0000 7f9a50c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:23.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.153+0000 7f9a50c00640 1 -- 172.21.15.86:0/3044570652 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c08b040 con 0x55946ccaacc0 2024-04-17T07:24:23.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.154+0000 7f9a50c00640 1 -- 172.21.15.86:0/3044570652 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9a6c1290f0 con 0x55946ccaacc0 2024-04-17T07:24:23.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.154+0000 7f9a74fdd740 1 -- 172.21.15.86:0/3044570652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a5802ddd0 con 0x55946ccaacc0 2024-04-17T07:24:23.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.157+0000 7f9a50c00640 1 -- 172.21.15.86:0/3044570652 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a6c10cfe0 con 0x55946ccaacc0 2024-04-17T07:24:23.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.157+0000 7f9a74fdd740 10 client.26878 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:23.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.157+0000 7f9a74fdd740 1 -- 172.21.15.86:0/3044570652 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55946cb967f0 con 0x55946ccaacc0 2024-04-17T07:24:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.159+0000 7f9a50c00640 1 -- 172.21.15.86:0/3044570652 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a5802ddd0 con 0x55946ccaacc0 2024-04-17T07:24:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.159+0000 7f9a50c00640 1 -- 172.21.15.86:0/3044570652 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9a6c111410 con 0x55946ccaacc0 2024-04-17T07:24:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.159+0000 7f9a74fdd740 10 client.26878 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:23.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.159+0000 7f9a74fdd740 10 client.26878 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:24:23.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.159+0000 7f9a74fdd740 20 client.26878 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:23.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.159+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55946ccbae90 0x55946ccbd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.159+0000 7f9a74fdd740 4 client.26878 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:23.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.159+0000 7f9a74fdd740 1 -- 172.21.15.86:0/3044570652 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55946cc18a40 con 0x55946ccbae90 2024-04-17T07:24:23.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.161+0000 7f9a6b400640 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55946ccbae90 0x55946ccbd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.163+0000 7f9a6b400640 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55946ccbae90 0x55946ccbd350 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b9c151099e68a5c server_cookie=b644875b4115bb1a in_seq=0 out_seq=0 2024-04-17T07:24:23.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.163+0000 7f9a50c00640 10 client.26878 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:24:23.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.164+0000 7f9a50c00640 1 -- 172.21.15.86:0/3044570652 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55946cc18a40 con 0x55946ccbae90 2024-04-17T07:24:23.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.164+0000 7f9a50c00640 10 client.26878 handle_command_reply: tid=0 2024-04-17T07:24:23.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.165+0000 7f9a74fdd740 1 client.26878 shutdown 2024-04-17T07:24:23.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.167+0000 7f9a74fdd740 1 -- 172.21.15.86:0/3044570652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946ccaacc0 msgr2=0x55946ccb69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.167+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946ccaacc0 0x55946ccb69f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c111370 tx=0x7f9a6c07ae80 comp rx=0 tx=0).stop 2024-04-17T07:24:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.167+0000 7f9a74fdd740 1 -- 172.21.15.86:0/3044570652 shutdown_connections 2024-04-17T07:24:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.167+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55946ccbae90 0x55946ccbd350 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.167+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946ccaacc0 0x55946ccb69f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.167+0000 7f9a50c00640 0 client.26878 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:24:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.167+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946cca5b50 0x55946ccafa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.167+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/3044570652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.167+0000 7f9a74fdd740 1 -- 172.21.15.86:0/3044570652 >> 172.21.15.86:0/3044570652 conn(0x55946cca06d0 msgr2=0x55946cc9f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.168+0000 7f9a74fdd740 1 -- 172.21.15.86:0/3044570652 shutdown_connections 2024-04-17T07:24:23.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.169+0000 7f9a74fdd740 1 -- 172.21.15.86:0/3044570652 wait complete. 2024-04-17T07:24:23.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.169+0000 7f9a74fdd740 20 client.26878 trim_cache size 0 max 16384 2024-04-17T07:24:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.201+0000 7f9a74fdd740 1 Processor -- start 2024-04-17T07:24:23.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.201+0000 7f9a74fdd740 1 -- start start 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.201+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.201+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 0x55946ccb69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.202+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 0x55946cc9f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.202+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6414d660 con 0x55946ccaacc0 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.202+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6414f1a0 con 0x55946cca5160 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.202+0000 7f9a6b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.202+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6414dd30 con 0x55946cca5b50 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.202+0000 7f9a6b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42366/0 (socket says 172.21.15.86:42366) 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.202+0000 7f9a6b400640 1 -- 172.21.15.86:0/297091144 learned_addr learned my addr 172.21.15.86:0/297091144 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.202+0000 7f9a6aa00640 1 --2- 172.21.15.86:0/297091144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 0x55946ccb69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.203+0000 7f9a6b400640 1 -- 172.21.15.86:0/297091144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 msgr2=0x55946ccb69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.203+0000 7f9a6b400640 1 --2- 172.21.15.86:0/297091144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 0x55946ccb69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.203+0000 7f9a6b400640 1 -- 172.21.15.86:0/297091144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 msgr2=0x55946cc9f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.203+0000 7f9a6b400640 1 --2- 172.21.15.86:0/297091144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 0x55946cc9f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.203+0000 7f9a6b400640 1 -- 172.21.15.86:0/297091144 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946c91d3f0 con 0x55946cca5160 2024-04-17T07:24:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.203+0000 7f9a6b400640 1 --2- 172.21.15.86:0/297091144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaa580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9a58000c00 tx=0x7f9a5802c5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7718d1de20c9227 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.204+0000 7f9a53400640 1 -- 172.21.15.86:0/297091144 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58044040 con 0x55946cca5160 2024-04-17T07:24:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.204+0000 7f9a53400640 1 -- 172.21.15.86:0/297091144 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a58005850 con 0x55946cca5160 2024-04-17T07:24:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.204+0000 7f9a53400640 1 -- 172.21.15.86:0/297091144 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58050820 con 0x55946cca5160 2024-04-17T07:24:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.206+0000 7f9a74fdd740 1 -- 172.21.15.86:0/297091144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 msgr2=0x55946ccaa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.206+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/297091144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaa580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9a58000c00 tx=0x7f9a5802c5e0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.206+0000 7f9a74fdd740 1 -- 172.21.15.86:0/297091144 shutdown_connections 2024-04-17T07:24:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.206+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/297091144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946ccaacc0 0x55946cc9f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.206+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/297091144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5b50 0x55946ccb69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.206+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/297091144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946cca5160 0x55946ccaa580 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.206+0000 7f9a74fdd740 1 -- 172.21.15.86:0/297091144 >> 172.21.15.86:0/297091144 conn(0x55946cca06d0 msgr2=0x55946cca6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.206+0000 7f9a74fdd740 1 -- 172.21.15.86:0/297091144 shutdown_connections 2024-04-17T07:24:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.207+0000 7f9a74fdd740 1 -- 172.21.15.86:0/297091144 wait complete. 2024-04-17T07:24:23.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.207+0000 7f9a74fdd740 1 -- start start 2024-04-17T07:24:23.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.207+0000 7f9a74fdd740 1 Processor -- start 2024-04-17T07:24:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.208+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5160 0x55946ccb1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.208+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946cca5b50 0x55946ccb21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.208+0000 7f9a74fdd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946ccaacc0 0x55946ccb5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.209+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6414ead0 con 0x55946cca5b50 2024-04-17T07:24:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.209+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6414e400 con 0x55946ccaacc0 2024-04-17T07:24:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.209+0000 7f9a74fdd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a6414b630 con 0x55946cca5160 2024-04-17T07:24:23.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.209+0000 7f9a74fdd740 10 client.0 _ll_register_callbacks cb 0x7f9a6c0502a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.209+0000 7f9a6be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946ccaacc0 0x55946ccb5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.209+0000 7f9a6b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5160 0x55946ccb1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.209+0000 7f9a6b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5160 0x55946ccb1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46020/0 (socket says 172.21.15.86:46020) 2024-04-17T07:24:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.209+0000 7f9a6b400640 1 -- 172.21.15.86:0/2731067635 learned_addr learned my addr 172.21.15.86:0/2731067635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.209+0000 7f9a6be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946ccaacc0 0x55946ccb5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42380/0 (socket says 172.21.15.86:42380) 2024-04-17T07:24:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.210+0000 7f9a6b400640 1 -- 172.21.15.86:0/2731067635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946ccaacc0 msgr2=0x55946ccb5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.210+0000 7f9a6b400640 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946ccaacc0 0x55946ccb5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.210+0000 7f9a6b400640 1 -- 172.21.15.86:0/2731067635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946cca5b50 msgr2=0x55946ccb21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.210+0000 7f9a6b400640 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946cca5b50 0x55946ccb21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.210+0000 7f9a6b400640 1 -- 172.21.15.86:0/2731067635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946c9e77d0 con 0x55946cca5160 2024-04-17T07:24:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.211+0000 7f9a6b400640 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5160 0x55946ccb1ca0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9a5802c9f0 tx=0x7f9a58004370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54e471c5f04292e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.211+0000 7f9a50c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.211+0000 7f9a50c00640 1 -- 172.21.15.86:0/2731067635 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58044040 con 0x55946cca5160 2024-04-17T07:24:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.211+0000 7f9a50c00640 1 -- 172.21.15.86:0/2731067635 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9a580035f0 con 0x55946cca5160 2024-04-17T07:24:23.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.211+0000 7f9a74fdd740 1 -- 172.21.15.86:0/2731067635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a58005850 con 0x55946cca5160 2024-04-17T07:24:23.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.213+0000 7f9a50c00640 1 -- 172.21.15.86:0/2731067635 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a580044e0 con 0x55946cca5160 2024-04-17T07:24:23.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.213+0000 7f9a74fdd740 10 client.26884 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:23.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.213+0000 7f9a74fdd740 1 -- 172.21.15.86:0/2731067635 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55946ccb6920 con 0x55946cca5160 2024-04-17T07:24:23.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.214+0000 7f9a50c00640 1 -- 172.21.15.86:0/2731067635 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a58005850 con 0x55946cca5160 2024-04-17T07:24:23.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.214+0000 7f9a50c00640 1 -- 172.21.15.86:0/2731067635 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9a580508c0 con 0x55946cca5160 2024-04-17T07:24:23.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.214+0000 7f9a74fdd740 10 client.26884 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:23.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.214+0000 7f9a74fdd740 10 client.26884 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:24:23.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.214+0000 7f9a74fdd740 20 client.26884 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:23.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.214+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55946ccbae90 0x55946cd344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.214+0000 7f9a74fdd740 4 client.26884 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:24:23.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.214+0000 7f9a74fdd740 1 -- 172.21.15.86:0/2731067635 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55946cc128b0 con 0x55946ccbae90 2024-04-17T07:24:23.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.215+0000 7f9a6aa00640 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55946ccbae90 0x55946cd344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.216+0000 7f9a6aa00640 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55946ccbae90 0x55946cd344b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2ade837041cff8d server_cookie=6a085b936d48a825 in_seq=0 out_seq=0 2024-04-17T07:24:23.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.216+0000 7f9a50c00640 10 client.26884 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:24:23.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.216+0000 7f9a50c00640 1 -- 172.21.15.86:0/2731067635 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55946cc128b0 con 0x55946ccbae90 2024-04-17T07:24:23.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.216+0000 7f9a50c00640 10 client.26884 handle_command_reply: tid=0 2024-04-17T07:24:23.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.216+0000 7f9a74fdd740 1 client.26884 shutdown 2024-04-17T07:24:23.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.217+0000 7f9a74fdd740 1 -- 172.21.15.86:0/2731067635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5160 msgr2=0x55946ccb1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.217+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5160 0x55946ccb1ca0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9a5802c9f0 tx=0x7f9a58004370 comp rx=0 tx=0).stop 2024-04-17T07:24:23.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.218+0000 7f9a74fdd740 1 -- 172.21.15.86:0/2731067635 shutdown_connections 2024-04-17T07:24:23.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.218+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55946ccbae90 0x55946cd344b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.218+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55946ccaacc0 0x55946ccb5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.218+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55946cca5b50 0x55946ccb21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.218+0000 7f9a74fdd740 1 --2- 172.21.15.86:0/2731067635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55946cca5160 0x55946ccb1ca0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.218+0000 7f9a74fdd740 1 -- 172.21.15.86:0/2731067635 >> 172.21.15.86:0/2731067635 conn(0x55946cca06d0 msgr2=0x55946ccaf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.218+0000 7f9a50c00640 0 client.26884 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:24:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.218+0000 7f9a74fdd740 1 -- 172.21.15.86:0/2731067635 shutdown_connections 2024-04-17T07:24:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.218+0000 7f9a74fdd740 1 -- 172.21.15.86:0/2731067635 wait complete. 2024-04-17T07:24:23.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.218+0000 7f9a74fdd740 20 client.26884 trim_cache size 0 max 16384 2024-04-17T07:24:23.218 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:24:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.220+0000 7f9a70c00640 1 -- 172.21.15.86:0/2464944765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a540a3c30 msgr2=0x7f9a540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.220+0000 7f9a70c00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a540a3c30 0x7f9a540a60f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55946cca63b0 tx=0x7f9a60004730 comp rx=0 tx=0).stop 2024-04-17T07:24:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.220+0000 7f9a70c00640 1 -- 172.21.15.86:0/2464944765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6409df20 msgr2=0x7f9a64142670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.220+0000 7f9a70c00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6409df20 0x7f9a64142670 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c050aa0 tx=0x7f9a6c06adb0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.220+0000 7f9a70c00640 1 -- 172.21.15.86:0/2464944765 shutdown_connections 2024-04-17T07:24:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.220+0000 7f9a70c00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a540a3c30 0x7f9a540a60f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.220+0000 7f9a70c00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a6409df20 0x7f9a64142670 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.220+0000 7f9a70c00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6409d560 0x7f9a64013600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.220+0000 7f9a70c00640 1 --2- 172.21.15.86:0/2464944765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a6409c370 0x7f9a640130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.220+0000 7f9a70c00640 1 -- 172.21.15.86:0/2464944765 >> 172.21.15.86:0/2464944765 conn(0x7f9a6400b4f0 msgr2=0x7f9a640a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.221+0000 7f9a70c00640 1 -- 172.21.15.86:0/2464944765 shutdown_connections 2024-04-17T07:24:23.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.221+0000 7f9a70c00640 1 -- 172.21.15.86:0/2464944765 wait complete. 2024-04-17T07:24:23.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:24:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.555+0000 7f1ce0a00640 1 -- 172.21.15.86:0/3569844980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cdc101fb0 msgr2=0x7f1cdc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.555+0000 7f1ce0a00640 1 --2- 172.21.15.86:0/3569844980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cdc101fb0 0x7f1cdc1023d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009860 tx=0x7f1cd402bb70 comp rx=0 tx=0).stop 2024-04-17T07:24:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.556+0000 7f1ce0a00640 1 -- 172.21.15.86:0/3569844980 shutdown_connections 2024-04-17T07:24:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.556+0000 7f1ce0a00640 1 --2- 172.21.15.86:0/3569844980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cdc103b60 0x7f1cdc10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.556+0000 7f1ce0a00640 1 --2- 172.21.15.86:0/3569844980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cdc1031a0 0x7f1cdc103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.556+0000 7f1ce0a00640 1 --2- 172.21.15.86:0/3569844980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cdc101fb0 0x7f1cdc1023d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.556+0000 7f1ce0a00640 1 -- 172.21.15.86:0/3569844980 >> 172.21.15.86:0/3569844980 conn(0x7f1cdc0fd540 msgr2=0x7f1cdc0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.557+0000 7f1ce0a00640 1 -- 172.21.15.86:0/3569844980 shutdown_connections 2024-04-17T07:24:23.556 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:23 smithi098 ceph-mon[35418]: pgmap v740: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.0 MiB/s wr, 757 op/s 2024-04-17T07:24:23.556 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:23 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/50835334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:24:23.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.557+0000 7f1ce0a00640 1 -- 172.21.15.86:0/3569844980 wait complete. 2024-04-17T07:24:23.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.559+0000 7f1ce0a00640 1 Processor -- start 2024-04-17T07:24:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.559+0000 7f1ce0a00640 1 -- start start 2024-04-17T07:24:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.560+0000 7f1ce0a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cdc101fb0 0x7f1cdc112ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.560+0000 7f1ce0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cdc1031a0 0x7f1cdc113010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.560+0000 7f1ce0a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cdc103b60 0x7f1cdc111110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.561+0000 7f1cd9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cdc1031a0 0x7f1cdc113010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.561+0000 7f1cd9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cdc1031a0 0x7f1cdc113010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37972/0 (socket says 172.21.15.86:37972) 2024-04-17T07:24:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.561+0000 7f1cd9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cdc101fb0 0x7f1cdc112ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.562+0000 7f1cda800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cdc103b60 0x7f1cdc111110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.562+0000 7f1cda800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cdc103b60 0x7f1cdc111110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42398/0 (socket says 172.21.15.86:42398) 2024-04-17T07:24:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.562+0000 7f1cd9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cdc101fb0 0x7f1cdc112ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46040/0 (socket says 172.21.15.86:46040) 2024-04-17T07:24:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.562+0000 7f1cd9400640 1 -- 172.21.15.86:0/828926058 learned_addr learned my addr 172.21.15.86:0/828926058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.562+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc10b900 con 0x7f1cdc1031a0 2024-04-17T07:24:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.562+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc10b780 con 0x7f1cdc103b60 2024-04-17T07:24:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.563+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc10ba80 con 0x7f1cdc101fb0 2024-04-17T07:24:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.564+0000 7f1cda800640 1 -- 172.21.15.86:0/828926058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cdc101fb0 msgr2=0x7f1cdc112ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.564+0000 7f1cda800640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cdc101fb0 0x7f1cdc112ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.564+0000 7f1cda800640 1 -- 172.21.15.86:0/828926058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cdc1031a0 msgr2=0x7f1cdc113010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.564+0000 7f1cda800640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cdc1031a0 0x7f1cdc113010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.564+0000 7f1cda800640 1 -- 172.21.15.86:0/828926058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc1117f0 con 0x7f1cdc103b60 2024-04-17T07:24:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.566+0000 7f1cda800640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cdc103b60 0x7f1cdc111110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400e970 tx=0x7f1cc400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.567+0000 7f1cd2a00640 1 -- 172.21.15.86:0/828926058 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400ccc0 con 0x7f1cdc103b60 2024-04-17T07:24:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.567+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cdc111ae0 con 0x7f1cdc103b60 2024-04-17T07:24:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.567+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cdc0719b0 con 0x7f1cdc103b60 2024-04-17T07:24:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.567+0000 7f1cd2a00640 1 -- 172.21.15.86:0/828926058 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cc4005020 con 0x7f1cdc103b60 2024-04-17T07:24:23.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.567+0000 7f1cd2a00640 1 -- 172.21.15.86:0/828926058 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4004a00 con 0x7f1cdc103b60 2024-04-17T07:24:23.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.572+0000 7f1ce37ef740 1 Processor -- start 2024-04-17T07:24:23.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.576+0000 7f1cd2a00640 1 -- 172.21.15.86:0/828926058 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1cc4004ba0 con 0x7f1cdc103b60 2024-04-17T07:24:23.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.576+0000 7f1ce37ef740 1 -- start start 2024-04-17T07:24:23.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.577+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf81463c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.577+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf814b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.577+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf814bb00 0x55ccf814df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.577+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ccf80ac020 con 0x55ccf8146990 2024-04-17T07:24:23.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.577+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ccf8052460 con 0x55ccf814bb00 2024-04-17T07:24:23.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.577+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ccf80a7fe0 con 0x55ccf8145fa0 2024-04-17T07:24:23.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.577+0000 7f1cd9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf814b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.577+0000 7f1cd9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf814b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37984/0 (socket says 172.21.15.86:37984) 2024-04-17T07:24:23.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.577+0000 7f1cd9e00640 1 -- 172.21.15.86:0/3746483268 learned_addr learned my addr 172.21.15.86:0/3746483268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:23.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.578+0000 7f1cd9e00640 1 -- 172.21.15.86:0/3746483268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 msgr2=0x55ccf81463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.578+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/3746483268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf81463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.578+0000 7f1cd9e00640 1 -- 172.21.15.86:0/3746483268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf814bb00 msgr2=0x55ccf814df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.578+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/3746483268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf814bb00 0x55ccf814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.578+0000 7f1cd9e00640 1 -- 172.21.15.86:0/3746483268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf805a040 con 0x55ccf8146990 2024-04-17T07:24:23.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.578+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/3746483268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf814b490 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000c00 tx=0x7f1cd40095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58e3fc1c5c17b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.579+0000 7f1ccb000640 1 -- 172.21.15.86:0/3746483268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4043020 con 0x55ccf8146990 2024-04-17T07:24:23.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.579+0000 7f1ccb000640 1 -- 172.21.15.86:0/3746483268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cd4034680 con 0x55ccf8146990 2024-04-17T07:24:23.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.579+0000 7f1ccb000640 1 -- 172.21.15.86:0/3746483268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4039040 con 0x55ccf8146990 2024-04-17T07:24:23.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.580+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3746483268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 msgr2=0x55ccf814b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.580+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/3746483268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf814b490 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000c00 tx=0x7f1cd40095a0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:23 smithi086 ceph-mon[31733]: pgmap v740: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.0 MiB/s wr, 757 op/s 2024-04-17T07:24:23.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:23 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/50835334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:24:23.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.581+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3746483268 shutdown_connections 2024-04-17T07:24:23.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.581+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/3746483268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf814bb00 0x55ccf814df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.581+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/3746483268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf814b490 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.581+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/3746483268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf81463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.581+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3746483268 >> 172.21.15.86:0/3746483268 conn(0x55ccf8141510 msgr2=0x55ccf8141910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.582+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3746483268 shutdown_connections 2024-04-17T07:24:23.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.582+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3746483268 wait complete. 2024-04-17T07:24:23.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.582+0000 7f1cd2a00640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1cc00a3b80 0x7f1cc00a6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.582+0000 7f1cd2a00640 1 -- 172.21.15.86:0/828926058 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1cc40d0580 con 0x7f1cdc103b60 2024-04-17T07:24:23.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.582+0000 7f1ce37ef740 1 -- start start 2024-04-17T07:24:23.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.583+0000 7f1ce37ef740 1 Processor -- start 2024-04-17T07:24:23.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.583+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1cc00a3b80 0x7f1cc00a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.583+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf81504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.584+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8150a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.584+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf814bb00 0x55ccf8157a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.584+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ccf814ea50 con 0x55ccf814bb00 2024-04-17T07:24:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.584+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ccf8065830 con 0x55ccf8146990 2024-04-17T07:24:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.584+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ccf814ebd0 con 0x55ccf8145fa0 2024-04-17T07:24:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.584+0000 7f1ce37ef740 10 client.0 _ll_register_callbacks cb 0x55ccf8033fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.584+0000 7f1cda800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf814bb00 0x55ccf8157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.584+0000 7f1cd9400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.584+0000 7f1cd9400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42408/0 (socket says 172.21.15.86:42408) 2024-04-17T07:24:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.584+0000 7f1cd9400640 1 -- 172.21.15.86:0/3229348274 learned_addr learned my addr 172.21.15.86:0/3229348274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:23.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.586+0000 7f1cd9400640 1 -- 172.21.15.86:0/3229348274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 msgr2=0x55ccf81504f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:24:23.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.586+0000 7f1cd9400640 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf81504f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.586+0000 7f1cd9400640 1 -- 172.21.15.86:0/3229348274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf814bb00 msgr2=0x55ccf8157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.586+0000 7f1cd9400640 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf814bb00 0x55ccf8157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.586+0000 7f1cd9400640 1 -- 172.21.15.86:0/3229348274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf8069920 con 0x55ccf8146990 2024-04-17T07:24:23.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.586+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1cc00a3b80 0x7f1cc00a6040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4037040 tx=0x7f1cd402bb70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.587+0000 7f1cd9400640 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8150a30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009c40 tx=0x7f1ccc00d7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=158dc250c569f62d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.587+0000 7f1cbf400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:24:23.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.588+0000 7f1cbf400640 1 -- 172.21.15.86:0/3229348274 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc003970 con 0x55ccf8146990 2024-04-17T07:24:23.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.589+0000 7f1cbf400640 1 -- 172.21.15.86:0/3229348274 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1ccc003b10 con 0x55ccf8146990 2024-04-17T07:24:23.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.589+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3229348274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd4034680 con 0x55ccf8146990 2024-04-17T07:24:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.590+0000 7f1cbf400640 1 -- 172.21.15.86:0/3229348274 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ccc003d10 con 0x55ccf8146990 2024-04-17T07:24:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.591+0000 7f1ce37ef740 10 client.36842 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.591+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3229348274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccf80377f0 con 0x55ccf8146990 2024-04-17T07:24:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.591+0000 7f1cbf400640 1 -- 172.21.15.86:0/3229348274 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cd4034680 con 0x55ccf8146990 2024-04-17T07:24:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.592+0000 7f1cbf400640 1 -- 172.21.15.86:0/3229348274 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1ccc00b040 con 0x55ccf8146990 2024-04-17T07:24:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.595+0000 7f1ce37ef740 10 client.36842 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.595+0000 7f1ce37ef740 10 client.36842 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:24:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.595+0000 7f1ce37ef740 20 client.36842 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.595+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ccf815c020 0x55ccf815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.595+0000 7f1ce37ef740 4 client.36842 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.595+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3229348274 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccf80b9a40 con 0x55ccf815c020 2024-04-17T07:24:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.596+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ccf815c020 0x55ccf815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.598+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ccf815c020 0x55ccf815e4e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8264d2f1494caca server_cookie=9dc0d67b47819f8f in_seq=0 out_seq=0 2024-04-17T07:24:23.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.598+0000 7f1cbf400640 10 client.36842 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:24:23.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.600+0000 7f1cbf400640 1 -- 172.21.15.86:0/3229348274 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccf80b9a40 con 0x55ccf815c020 2024-04-17T07:24:23.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.600+0000 7f1cbf400640 10 client.36842 handle_command_reply: tid=0 2024-04-17T07:24:23.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.601+0000 7f1ce37ef740 1 client.36842 shutdown 2024-04-17T07:24:23.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.602+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3229348274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 msgr2=0x55ccf8150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.602+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8150a30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009c40 tx=0x7f1ccc00d7b0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.602+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3229348274 shutdown_connections 2024-04-17T07:24:23.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.603+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ccf815c020 0x55ccf815e4e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.603+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf814bb00 0x55ccf8157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.603+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8150a30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.603+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/3229348274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf81504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.603+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3229348274 >> 172.21.15.86:0/3229348274 conn(0x55ccf8141510 msgr2=0x55ccf8140710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.603+0000 7f1cbf400640 0 client.36842 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:24:23.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.603+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3229348274 shutdown_connections 2024-04-17T07:24:23.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.604+0000 7f1ce37ef740 1 -- 172.21.15.86:0/3229348274 wait complete. 2024-04-17T07:24:23.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.604+0000 7f1ce37ef740 20 client.36842 trim_cache size 0 max 16384 2024-04-17T07:24:23.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.637+0000 7f1ce37ef740 1 Processor -- start 2024-04-17T07:24:23.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.637+0000 7f1ce37ef740 1 -- start start 2024-04-17T07:24:23.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.637+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf8140590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.637+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf8140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.638+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf814bb00 0x55ccf8157a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.638+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb800fc30 con 0x55ccf8146990 2024-04-17T07:24:23.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.638+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8011770 con 0x55ccf814bb00 2024-04-17T07:24:23.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.638+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8010300 con 0x55ccf8145fa0 2024-04-17T07:24:23.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.638+0000 7f1cd9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf8140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.638+0000 7f1cd9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf8140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46052/0 (socket says 172.21.15.86:46052) 2024-04-17T07:24:23.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.638+0000 7f1cd9e00640 1 -- 172.21.15.86:0/2317465467 learned_addr learned my addr 172.21.15.86:0/2317465467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:23.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.638+0000 7f1cda800640 1 --2- 172.21.15.86:0/2317465467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf814bb00 0x55ccf8157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.638+0000 7f1cd9400640 1 --2- 172.21.15.86:0/2317465467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf8140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.639+0000 7f1cd9e00640 1 -- 172.21.15.86:0/2317465467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf814bb00 msgr2=0x55ccf8157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.639+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/2317465467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf814bb00 0x55ccf8157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.639+0000 7f1cd9e00640 1 -- 172.21.15.86:0/2317465467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 msgr2=0x55ccf8140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.640+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/2317465467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf8140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.640+0000 7f1cd9e00640 1 -- 172.21.15.86:0/2317465467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf7dbe3f0 con 0x55ccf8145fa0 2024-04-17T07:24:23.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.640+0000 7f1cd9400640 1 --2- 172.21.15.86:0/2317465467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf8140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:23.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.641+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/2317465467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf8140590 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000c00 tx=0x7f1cd4048000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea368cbb1dcc216f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.641+0000 7f1cca600640 1 -- 172.21.15.86:0/2317465467 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4043040 con 0x55ccf8145fa0 2024-04-17T07:24:23.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.641+0000 7f1cca600640 1 -- 172.21.15.86:0/2317465467 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cd405d070 con 0x55ccf8145fa0 2024-04-17T07:24:23.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.642+0000 7f1cca600640 1 -- 172.21.15.86:0/2317465467 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4003850 con 0x55ccf8145fa0 2024-04-17T07:24:23.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.643+0000 7f1ce37ef740 1 -- 172.21.15.86:0/2317465467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 msgr2=0x55ccf8140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.643+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/2317465467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf8140590 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000c00 tx=0x7f1cd4048000 comp rx=0 tx=0).stop 2024-04-17T07:24:23.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.643+0000 7f1ce37ef740 1 -- 172.21.15.86:0/2317465467 shutdown_connections 2024-04-17T07:24:23.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.643+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/2317465467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf814bb00 0x55ccf8157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.643+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/2317465467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8146990 0x55ccf8140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.643+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/2317465467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf8145fa0 0x55ccf8140590 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.643+0000 7f1ce37ef740 1 -- 172.21.15.86:0/2317465467 >> 172.21.15.86:0/2317465467 conn(0x55ccf8141510 msgr2=0x55ccf813fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.643+0000 7f1ce37ef740 1 -- 172.21.15.86:0/2317465467 shutdown_connections 2024-04-17T07:24:23.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.643+0000 7f1ce37ef740 1 -- 172.21.15.86:0/2317465467 wait complete. 2024-04-17T07:24:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.644+0000 7f1ce37ef740 1 -- start start 2024-04-17T07:24:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.644+0000 7f1ce37ef740 1 Processor -- start 2024-04-17T07:24:23.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.646+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8145fa0 0x55ccf81550c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.646+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8155600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.646+0000 7f1ce37ef740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf814bb00 0x55ccf8156610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.646+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb80110a0 con 0x55ccf8145fa0 2024-04-17T07:24:23.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.646+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb80109d0 con 0x55ccf8146990 2024-04-17T07:24:23.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.646+0000 7f1ce37ef740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb800dc00 con 0x55ccf814bb00 2024-04-17T07:24:23.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.646+0000 7f1ce37ef740 10 client.0 _ll_register_callbacks cb 0x7f1ccc00e930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:23.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.647+0000 7f1cd9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8145fa0 0x55ccf81550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.647+0000 7f1cd9400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.647+0000 7f1cd9400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42428/0 (socket says 172.21.15.86:42428) 2024-04-17T07:24:23.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.647+0000 7f1cd9400640 1 -- 172.21.15.86:0/1258765033 learned_addr learned my addr 172.21.15.86:0/1258765033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:23.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.647+0000 7f1cda800640 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf814bb00 0x55ccf8156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.648+0000 7f1cd9400640 1 -- 172.21.15.86:0/1258765033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf814bb00 msgr2=0x55ccf8156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.648+0000 7f1cd9400640 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf814bb00 0x55ccf8156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.648+0000 7f1cd9400640 1 -- 172.21.15.86:0/1258765033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8145fa0 msgr2=0x55ccf81550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:23.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.648+0000 7f1cd9400640 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8145fa0 0x55ccf81550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.649+0000 7f1cd9400640 1 -- 172.21.15.86:0/1258765033 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf7e887d0 con 0x55ccf8146990 2024-04-17T07:24:23.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.650+0000 7f1cda800640 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf814bb00 0x55ccf8156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:24:23.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.650+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8145fa0 0x55ccf81550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:23.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.650+0000 7f1cd9400640 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8155600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009bf0 tx=0x7f1ccc017640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c50d6d1ed4dea2d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:23.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.651+0000 7f1cbf400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:24:23.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.652+0000 7f1cbf400640 1 -- 172.21.15.86:0/1258765033 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc02d020 con 0x55ccf8146990 2024-04-17T07:24:23.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.653+0000 7f1cbf400640 1 -- 172.21.15.86:0/1258765033 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1ccc003970 con 0x55ccf8146990 2024-04-17T07:24:23.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.653+0000 7f1ce37ef740 1 -- 172.21.15.86:0/1258765033 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd405d070 con 0x55ccf8146990 2024-04-17T07:24:23.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.655+0000 7f1cbf400640 1 -- 172.21.15.86:0/1258765033 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ccc017b00 con 0x55ccf8146990 2024-04-17T07:24:23.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.655+0000 7f1ce37ef740 10 client.36848 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:23.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.655+0000 7f1ce37ef740 1 -- 172.21.15.86:0/1258765033 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccf81c2620 con 0x55ccf8146990 2024-04-17T07:24:23.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.656+0000 7f1cbf400640 1 -- 172.21.15.86:0/1258765033 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cd405d070 con 0x55ccf8146990 2024-04-17T07:24:23.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.657+0000 7f1cbf400640 1 -- 172.21.15.86:0/1258765033 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1ccc00b040 con 0x55ccf8146990 2024-04-17T07:24:23.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.657+0000 7f1ce37ef740 10 client.36848 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:23.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.657+0000 7f1ce37ef740 10 client.36848 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:24:23.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.657+0000 7f1ce37ef740 20 client.36848 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:23.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.657+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ccf81c2e90 0x55ccf81c5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:23.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.657+0000 7f1ce37ef740 4 client.36848 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:24:23.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.657+0000 7f1ce37ef740 1 -- 172.21.15.86:0/1258765033 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccf80b38b0 con 0x55ccf81c2e90 2024-04-17T07:24:23.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.658+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ccf81c2e90 0x55ccf81c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:23.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.660+0000 7f1cd9e00640 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ccf81c2e90 0x55ccf81c5280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff44a9ea6ccff543 server_cookie=348f253ef170ba6c in_seq=0 out_seq=0 2024-04-17T07:24:23.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.660+0000 7f1cbf400640 10 client.36848 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:24:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.661+0000 7f1cbf400640 1 -- 172.21.15.86:0/1258765033 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccf80b38b0 con 0x55ccf81c2e90 2024-04-17T07:24:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.661+0000 7f1cbf400640 10 client.36848 handle_command_reply: tid=0 2024-04-17T07:24:23.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.661+0000 7f1ce37ef740 1 client.36848 shutdown 2024-04-17T07:24:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.662+0000 7f1ce37ef740 1 -- 172.21.15.86:0/1258765033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 msgr2=0x55ccf8155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.662+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8155600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009bf0 tx=0x7f1ccc017640 comp rx=0 tx=0).stop 2024-04-17T07:24:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.663+0000 7f1ce37ef740 1 -- 172.21.15.86:0/1258765033 shutdown_connections 2024-04-17T07:24:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.663+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ccf81c2e90 0x55ccf81c5280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.663+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ccf814bb00 0x55ccf8156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.663+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ccf8146990 0x55ccf8155600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.663+0000 7f1ce37ef740 1 --2- 172.21.15.86:0/1258765033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ccf8145fa0 0x55ccf81550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.663+0000 7f1cbf400640 0 client.36848 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:24:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.663+0000 7f1ce37ef740 1 -- 172.21.15.86:0/1258765033 >> 172.21.15.86:0/1258765033 conn(0x55ccf8141510 msgr2=0x55ccf8140710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.663+0000 7f1ce37ef740 1 -- 172.21.15.86:0/1258765033 shutdown_connections 2024-04-17T07:24:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.663+0000 7f1ce37ef740 1 -- 172.21.15.86:0/1258765033 wait complete. 2024-04-17T07:24:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.663+0000 7f1ce37ef740 20 client.36848 trim_cache size 0 max 16384 2024-04-17T07:24:23.663 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:24:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.664+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1cc00a3b80 msgr2=0x7f1cc00a6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.664+0000 7f1ce0a00640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1cc00a3b80 0x7f1cc00a6040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4037040 tx=0x7f1cd402bb70 comp rx=0 tx=0).stop 2024-04-17T07:24:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.664+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cdc103b60 msgr2=0x7f1cdc111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.664+0000 7f1ce0a00640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cdc103b60 0x7f1cdc111110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400e970 tx=0x7f1cc400ee30 comp rx=0 tx=0).stop 2024-04-17T07:24:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.664+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 shutdown_connections 2024-04-17T07:24:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.664+0000 7f1ce0a00640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1cc00a3b80 0x7f1cc00a6040 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.664+0000 7f1ce0a00640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1cdc103b60 0x7f1cdc111110 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.664+0000 7f1ce0a00640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cdc1031a0 0x7f1cdc113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.664+0000 7f1ce0a00640 1 --2- 172.21.15.86:0/828926058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cdc101fb0 0x7f1cdc112ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.665+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 >> 172.21.15.86:0/828926058 conn(0x7f1cdc0fd540 msgr2=0x7f1cdc106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.665+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 shutdown_connections 2024-04-17T07:24:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:23.665+0000 7f1ce0a00640 1 -- 172.21.15.86:0/828926058 wait complete. 2024-04-17T07:24:23.694 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:24:23.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:23 smithi103 ceph-mon[35495]: pgmap v740: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.0 MiB/s wr, 757 op/s 2024-04-17T07:24:23.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:23 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/50835334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:24:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.005+0000 7f5812a00640 1 -- 172.21.15.86:0/4013428651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c1013c0 msgr2=0x7f580c101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.005+0000 7f5812a00640 1 --2- 172.21.15.86:0/4013428651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c1013c0 0x7f580c101840 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f57f8008570 tx=0x7f57f800ac00 comp rx=0 tx=0).stop 2024-04-17T07:24:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.005+0000 7f5812a00640 1 -- 172.21.15.86:0/4013428651 shutdown_connections 2024-04-17T07:24:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.005+0000 7f5812a00640 1 --2- 172.21.15.86:0/4013428651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f580c101d80 0x7f580c108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.005+0000 7f5812a00640 1 --2- 172.21.15.86:0/4013428651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c1013c0 0x7f580c101840 secure :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f57f8008570 tx=0x7f57f800ac00 comp rx=0 tx=0).stop 2024-04-17T07:24:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.005+0000 7f5812a00640 1 --2- 172.21.15.86:0/4013428651 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f580c1001d0 0x7f580c1005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.005+0000 7f5812a00640 1 -- 172.21.15.86:0/4013428651 >> 172.21.15.86:0/4013428651 conn(0x7f580c0fb760 msgr2=0x7f580c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.006+0000 7f5812a00640 1 -- 172.21.15.86:0/4013428651 shutdown_connections 2024-04-17T07:24:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.006+0000 7f5812a00640 1 -- 172.21.15.86:0/4013428651 wait complete. 2024-04-17T07:24:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.008+0000 7f5812a00640 1 Processor -- start 2024-04-17T07:24:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.008+0000 7f5812a00640 1 -- start start 2024-04-17T07:24:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.008+0000 7f5812a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c1001d0 0x7f580c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.008+0000 7f5812a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f580c101d80 0x7f580c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.008+0000 7f5812a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f580c19fb70 0x7f580c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.008+0000 7f5812a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f580c10b8d0 con 0x7f580c1001d0 2024-04-17T07:24:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.008+0000 7f5812a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f580c10b750 con 0x7f580c19fb70 2024-04-17T07:24:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.008+0000 7f5812a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f580c10ba50 con 0x7f580c101d80 2024-04-17T07:24:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.009+0000 7f580be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c1001d0 0x7f580c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.009+0000 7f580b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f580c101d80 0x7f580c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.009+0000 7f5810a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f580c19fb70 0x7f580c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.009+0000 7f580b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f580c101d80 0x7f580c19ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46072/0 (socket says 172.21.15.86:46072) 2024-04-17T07:24:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.009+0000 7f580b400640 1 -- 172.21.15.86:0/1409598206 learned_addr learned my addr 172.21.15.86:0/1409598206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.009+0000 7f5810a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f580c19fb70 0x7f580c1a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42444/0 (socket says 172.21.15.86:42444) 2024-04-17T07:24:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.009+0000 7f580be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c1001d0 0x7f580c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38010/0 (socket says 172.21.15.86:38010) 2024-04-17T07:24:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.010+0000 7f5810a00640 1 -- 172.21.15.86:0/1409598206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f580c101d80 msgr2=0x7f580c19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.010+0000 7f5810a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f580c101d80 0x7f580c19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.010+0000 7f5810a00640 1 -- 172.21.15.86:0/1409598206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c1001d0 msgr2=0x7f580c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.010+0000 7f5810a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c1001d0 0x7f580c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.010+0000 7f5810a00640 1 -- 172.21.15.86:0/1409598206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c1a25d0 con 0x7f580c19fb70 2024-04-17T07:24:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.011+0000 7f5810a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f580c19fb70 0x7f580c1a1f60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58040091e0 tx=0x7f58040096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.011+0000 7f5808c00640 1 -- 172.21.15.86:0/1409598206 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804004d20 con 0x7f580c19fb70 2024-04-17T07:24:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.012+0000 7f5808c00640 1 -- 172.21.15.86:0/1409598206 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5804009e50 con 0x7f580c19fb70 2024-04-17T07:24:24.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.012+0000 7f5808c00640 1 -- 172.21.15.86:0/1409598206 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804005020 con 0x7f580c19fb70 2024-04-17T07:24:24.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.012+0000 7f5812a00640 1 -- 172.21.15.86:0/1409598206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c10c2a0 con 0x7f580c19fb70 2024-04-17T07:24:24.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.012+0000 7f5812a00640 1 -- 172.21.15.86:0/1409598206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c10c770 con 0x7f580c19fb70 2024-04-17T07:24:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.017+0000 7f581582d740 1 Processor -- start 2024-04-17T07:24:24.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.018+0000 7f5808c00640 1 -- 172.21.15.86:0/1409598206 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f58040051c0 con 0x7f580c19fb70 2024-04-17T07:24:24.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.019+0000 7f581582d740 1 -- start start 2024-04-17T07:24:24.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.019+0000 7f581582d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f23fa0 0x555d67f243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.019+0000 7f581582d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f24990 0x555d67f29490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.019+0000 7f581582d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f2bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.019+0000 7f581582d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555d67e8a020 con 0x555d67f24990 2024-04-17T07:24:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.019+0000 7f581582d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555d67e30460 con 0x555d67f23fa0 2024-04-17T07:24:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.019+0000 7f581582d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555d67e85fe0 con 0x555d67f29b00 2024-04-17T07:24:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.020+0000 7f580be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f24990 0x555d67f29490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.020+0000 7f580be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f24990 0x555d67f29490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38014/0 (socket says 172.21.15.86:38014) 2024-04-17T07:24:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.020+0000 7f580be00640 1 -- 172.21.15.86:0/3678305584 learned_addr learned my addr 172.21.15.86:0/3678305584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.020+0000 7f580b400640 1 --2- 172.21.15.86:0/3678305584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f23fa0 0x555d67f243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.021+0000 7f5808c00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57f00a3d30 0x7f57f00a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.021+0000 7f580be00640 1 -- 172.21.15.86:0/3678305584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 msgr2=0x555d67f2bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.021+0000 7f580be00640 1 --2- 172.21.15.86:0/3678305584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f2bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.021+0000 7f580be00640 1 -- 172.21.15.86:0/3678305584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f23fa0 msgr2=0x555d67f243c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.021+0000 7f5808c00640 1 -- 172.21.15.86:0/1409598206 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f58040d8700 con 0x7f580c19fb70 2024-04-17T07:24:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.022+0000 7f580be00640 1 --2- 172.21.15.86:0/3678305584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f23fa0 0x555d67f243c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.022+0000 7f580be00640 1 -- 172.21.15.86:0/3678305584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d67e38040 con 0x555d67f24990 2024-04-17T07:24:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.022+0000 7f580b400640 1 --2- 172.21.15.86:0/3678305584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f23fa0 0x555d67f243c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:24:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.022+0000 7f5810a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57f00a3d30 0x7f57f00a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.024+0000 7f5810a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57f00a3d30 0x7f57f00a61f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5804004070 tx=0x7f5804005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.024+0000 7f580be00640 1 --2- 172.21.15.86:0/3678305584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f24990 0x555d67f29490 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00bcb0 tx=0x7f57fc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adac4f51608a7cb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.025+0000 7f5801200640 1 -- 172.21.15.86:0/3678305584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc004020 con 0x555d67f24990 2024-04-17T07:24:24.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.025+0000 7f5801200640 1 -- 172.21.15.86:0/3678305584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57fc0054d0 con 0x555d67f24990 2024-04-17T07:24:24.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.025+0000 7f5801200640 1 -- 172.21.15.86:0/3678305584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc004020 con 0x555d67f24990 2024-04-17T07:24:24.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.027+0000 7f581582d740 1 -- 172.21.15.86:0/3678305584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f24990 msgr2=0x555d67f29490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.027+0000 7f581582d740 1 --2- 172.21.15.86:0/3678305584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f24990 0x555d67f29490 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00bcb0 tx=0x7f57fc0107a0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.028+0000 7f581582d740 1 -- 172.21.15.86:0/3678305584 shutdown_connections 2024-04-17T07:24:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.028+0000 7f581582d740 1 --2- 172.21.15.86:0/3678305584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f2bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.028+0000 7f581582d740 1 --2- 172.21.15.86:0/3678305584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f24990 0x555d67f29490 secure :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00bcb0 tx=0x7f57fc0107a0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.028+0000 7f581582d740 1 --2- 172.21.15.86:0/3678305584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f23fa0 0x555d67f243c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.028+0000 7f581582d740 1 -- 172.21.15.86:0/3678305584 >> 172.21.15.86:0/3678305584 conn(0x555d67f1f510 msgr2=0x555d67f1f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.032+0000 7f581582d740 1 -- 172.21.15.86:0/3678305584 shutdown_connections 2024-04-17T07:24:24.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.032+0000 7f581582d740 1 -- 172.21.15.86:0/3678305584 wait complete. 2024-04-17T07:24:24.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.033+0000 7f581582d740 1 -- start start 2024-04-17T07:24:24.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.033+0000 7f581582d740 1 Processor -- start 2024-04-17T07:24:24.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.034+0000 7f581582d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f2e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.034+0000 7f581582d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f2ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.034+0000 7f581582d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 0x555d67f36000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.034+0000 7f581582d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555d67f2cb50 con 0x555d67f23fa0 2024-04-17T07:24:24.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.034+0000 7f581582d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555d67e43830 con 0x555d67f33c10 2024-04-17T07:24:24.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.034+0000 7f581582d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555d67f2ccd0 con 0x555d67f29b00 2024-04-17T07:24:24.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.034+0000 7f581582d740 10 client.0 _ll_register_callbacks cb 0x555d67e11fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:24.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.036+0000 7f580b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 0x555d67f36000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.036+0000 7f580b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f2e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.036+0000 7f580b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f2e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38030/0 (socket says 172.21.15.86:38030) 2024-04-17T07:24:24.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.036+0000 7f580b400640 1 -- 172.21.15.86:0/3571896439 learned_addr learned my addr 172.21.15.86:0/3571896439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.036+0000 7f580be00640 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f2ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.037+0000 7f580be00640 1 -- 172.21.15.86:0/3571896439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 msgr2=0x555d67f36000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.037+0000 7f580be00640 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 0x555d67f36000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.037+0000 7f580be00640 1 -- 172.21.15.86:0/3571896439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 msgr2=0x555d67f2e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.037+0000 7f580be00640 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f2e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.037+0000 7f580be00640 1 -- 172.21.15.86:0/3571896439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d67e47920 con 0x555d67f29b00 2024-04-17T07:24:24.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.037+0000 7f580b400640 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f2e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:24.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.038+0000 7f580be00640 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f2ebe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x555d67f24e50 tx=0x7f57fc00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0dfc9cc24f67b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.038+0000 7f57f5600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:24.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.038+0000 7f57f5600640 1 -- 172.21.15.86:0/3571896439 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc02b060 con 0x555d67f29b00 2024-04-17T07:24:24.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.038+0000 7f581582d740 1 -- 172.21.15.86:0/3571896439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57fc0054d0 con 0x555d67f29b00 2024-04-17T07:24:24.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.039+0000 7f57f5600640 1 -- 172.21.15.86:0/3571896439 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f57fc004bf0 con 0x555d67f29b00 2024-04-17T07:24:24.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.039+0000 7f57f5600640 1 -- 172.21.15.86:0/3571896439 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57fc032050 con 0x555d67f29b00 2024-04-17T07:24:24.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.039+0000 7f581582d740 10 client.26902 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:24.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.039+0000 7f581582d740 1 -- 172.21.15.86:0/3571896439 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d67e157f0 con 0x555d67f29b00 2024-04-17T07:24:24.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.040+0000 7f57f5600640 1 -- 172.21.15.86:0/3571896439 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57fc0054d0 con 0x555d67f29b00 2024-04-17T07:24:24.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.040+0000 7f57f5600640 1 -- 172.21.15.86:0/3571896439 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f57fc011740 con 0x555d67f29b00 2024-04-17T07:24:24.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.040+0000 7f581582d740 10 client.26902 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:24.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.040+0000 7f581582d740 10 client.26902 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:24:24.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.040+0000 7f581582d740 20 client.26902 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:24.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.040+0000 7f581582d740 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555d67f3a210 0x555d67f3c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.040+0000 7f581582d740 4 client.26902 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:24.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.040+0000 7f581582d740 1 -- 172.21.15.86:0/3571896439 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d67e97a40 con 0x555d67f3a210 2024-04-17T07:24:24.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.041+0000 7f580b400640 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555d67f3a210 0x555d67f3c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.041+0000 7f580b400640 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555d67f3a210 0x555d67f3c600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6819ee495caaa8a server_cookie=705765e23267d4ce in_seq=0 out_seq=0 2024-04-17T07:24:24.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.041+0000 7f57f5600640 10 client.26902 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:24:24.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.042+0000 7f57f5600640 1 -- 172.21.15.86:0/3571896439 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d67e97a40 con 0x555d67f3a210 2024-04-17T07:24:24.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.042+0000 7f57f5600640 10 client.26902 handle_command_reply: tid=0 2024-04-17T07:24:24.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.042+0000 7f581582d740 1 client.26902 shutdown 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f581582d740 1 -- 172.21.15.86:0/3571896439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 msgr2=0x555d67f2ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f581582d740 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f2ebe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x555d67f24e50 tx=0x7f57fc00bfd0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f581582d740 1 -- 172.21.15.86:0/3571896439 shutdown_connections 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f581582d740 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555d67f3a210 0x555d67f3c600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f581582d740 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 0x555d67f36000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f581582d740 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f2ebe0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f581582d740 1 --2- 172.21.15.86:0/3571896439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f2e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f581582d740 1 -- 172.21.15.86:0/3571896439 >> 172.21.15.86:0/3571896439 conn(0x555d67f1f510 msgr2=0x555d67f1e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f57f5600640 0 client.26902 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f581582d740 1 -- 172.21.15.86:0/3571896439 shutdown_connections 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.043+0000 7f581582d740 1 -- 172.21.15.86:0/3571896439 wait complete. 2024-04-17T07:24:24.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.044+0000 7f581582d740 20 client.26902 trim_cache size 0 max 16384 2024-04-17T07:24:24.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.067+0000 7f581582d740 1 Processor -- start 2024-04-17T07:24:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.067+0000 7f581582d740 1 -- start start 2024-04-17T07:24:24.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.067+0000 7f581582d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f25fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.068+0000 7f581582d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f29b00 0x555d67f26510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.068+0000 7f581582d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f33c10 0x555d67f1de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.068+0000 7f581582d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1a8ea0 con 0x555d67f23fa0 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.068+0000 7f581582d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1aa9e0 con 0x555d67f29b00 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.068+0000 7f581582d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1a9570 con 0x555d67f33c10 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.068+0000 7f580b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f25fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.068+0000 7f580b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f25fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38042/0 (socket says 172.21.15.86:38042) 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.068+0000 7f580b400640 1 -- 172.21.15.86:0/3867021997 learned_addr learned my addr 172.21.15.86:0/3867021997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.068+0000 7f580be00640 1 --2- 172.21.15.86:0/3867021997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f29b00 0x555d67f26510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.069+0000 7f580b400640 1 -- 172.21.15.86:0/3867021997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f33c10 msgr2=0x555d67f1de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.069+0000 7f580b400640 1 --2- 172.21.15.86:0/3867021997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f33c10 0x555d67f1de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.069+0000 7f580b400640 1 -- 172.21.15.86:0/3867021997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f29b00 msgr2=0x555d67f26510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.069+0000 7f580b400640 1 --2- 172.21.15.86:0/3867021997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f29b00 0x555d67f26510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.069+0000 7f580b400640 1 -- 172.21.15.86:0/3867021997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d67b9c3f0 con 0x555d67f23fa0 2024-04-17T07:24:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.069+0000 7f580b400640 1 --2- 172.21.15.86:0/3867021997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f25fd0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005ab0 tx=0x7f57f80082b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0cf8f513296e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.070+0000 7f57f7e00640 1 -- 172.21.15.86:0/3867021997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8054070 con 0x555d67f23fa0 2024-04-17T07:24:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.070+0000 7f57f7e00640 1 -- 172.21.15.86:0/3867021997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57f80044c0 con 0x555d67f23fa0 2024-04-17T07:24:24.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.070+0000 7f57f7e00640 1 -- 172.21.15.86:0/3867021997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8003ca0 con 0x555d67f23fa0 2024-04-17T07:24:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.071+0000 7f581582d740 1 -- 172.21.15.86:0/3867021997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 msgr2=0x555d67f25fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.071+0000 7f581582d740 1 --2- 172.21.15.86:0/3867021997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f25fd0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005ab0 tx=0x7f57f80082b0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.072+0000 7f581582d740 1 -- 172.21.15.86:0/3867021997 shutdown_connections 2024-04-17T07:24:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.072+0000 7f581582d740 1 --2- 172.21.15.86:0/3867021997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f33c10 0x555d67f1de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.072+0000 7f581582d740 1 --2- 172.21.15.86:0/3867021997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f29b00 0x555d67f26510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.072+0000 7f581582d740 1 --2- 172.21.15.86:0/3867021997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f25fd0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.072+0000 7f581582d740 1 -- 172.21.15.86:0/3867021997 >> 172.21.15.86:0/3867021997 conn(0x555d67f1f510 msgr2=0x555d67f2e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.072+0000 7f581582d740 1 -- 172.21.15.86:0/3867021997 shutdown_connections 2024-04-17T07:24:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.072+0000 7f581582d740 1 -- 172.21.15.86:0/3867021997 wait complete. 2024-04-17T07:24:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.073+0000 7f581582d740 1 -- start start 2024-04-17T07:24:24.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.073+0000 7f581582d740 1 Processor -- start 2024-04-17T07:24:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.074+0000 7f581582d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f4a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.074+0000 7f581582d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f4ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.074+0000 7f581582d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 0x555d67f9e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.074+0000 7f581582d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1aa310 con 0x555d67f23fa0 2024-04-17T07:24:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.074+0000 7f581582d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1a9c40 con 0x555d67f33c10 2024-04-17T07:24:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.074+0000 7f581582d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1a6e70 con 0x555d67f29b00 2024-04-17T07:24:24.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.074+0000 7f581582d740 10 client.0 _ll_register_callbacks cb 0x7f57fc0044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.075+0000 7f580b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f4a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.075+0000 7f580be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f4ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.075+0000 7f580b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f4a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38050/0 (socket says 172.21.15.86:38050) 2024-04-17T07:24:24.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.075+0000 7f580b400640 1 -- 172.21.15.86:0/4200573634 learned_addr learned my addr 172.21.15.86:0/4200573634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.075+0000 7f580b400640 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 0x555d67f9e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.076+0000 7f580be00640 1 -- 172.21.15.86:0/4200573634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 msgr2=0x555d67f9e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.076+0000 7f580be00640 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 0x555d67f9e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.076+0000 7f580be00640 1 -- 172.21.15.86:0/4200573634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 msgr2=0x555d67f4a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.076+0000 7f580be00640 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f4a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.076+0000 7f580be00640 1 -- 172.21.15.86:0/4200573634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d67c667d0 con 0x555d67f29b00 2024-04-17T07:24:24.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.076+0000 7f580b400640 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 0x555d67f9e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:24:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.077+0000 7f580be00640 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f4ab10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f57fc011b40 tx=0x7f57fc01b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=224b3773e71ec85c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.077+0000 7f57f5600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.077+0000 7f57f5600640 1 -- 172.21.15.86:0/4200573634 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc03b060 con 0x555d67f29b00 2024-04-17T07:24:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.077+0000 7f57f5600640 1 -- 172.21.15.86:0/4200573634 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f57fc011440 con 0x555d67f29b00 2024-04-17T07:24:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.077+0000 7f581582d740 1 -- 172.21.15.86:0/4200573634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d67e918b0 con 0x555d67f29b00 2024-04-17T07:24:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.078+0000 7f57f5600640 1 -- 172.21.15.86:0/4200573634 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57fc02b070 con 0x555d67f29b00 2024-04-17T07:24:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.078+0000 7f581582d740 10 client.26905 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.078+0000 7f581582d740 1 -- 172.21.15.86:0/4200573634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d67fb04e0 con 0x555d67f29b00 2024-04-17T07:24:24.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.079+0000 7f57f5600640 1 -- 172.21.15.86:0/4200573634 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555d67e918b0 con 0x555d67f29b00 2024-04-17T07:24:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.079+0000 7f57f5600640 1 -- 172.21.15.86:0/4200573634 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f57fc01b990 con 0x555d67f29b00 2024-04-17T07:24:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.079+0000 7f581582d740 10 client.26905 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.079+0000 7f581582d740 10 client.26905 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:24:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.079+0000 7f581582d740 20 client.26905 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:24.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.079+0000 7f581582d740 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555d67fb0e60 0x555d67fb3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.079+0000 7f581582d740 4 client.26905 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:24:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.079+0000 7f581582d740 1 -- 172.21.15.86:0/4200573634 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d67e0d570 con 0x555d67fb0e60 2024-04-17T07:24:24.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.080+0000 7f580b400640 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555d67fb0e60 0x555d67fb3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.081+0000 7f580b400640 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555d67fb0e60 0x555d67fb3280 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87aab36191251563 server_cookie=4dc525c15d5044ce in_seq=0 out_seq=0 2024-04-17T07:24:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.081+0000 7f57f5600640 10 client.26905 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:24:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.081+0000 7f57f5600640 1 -- 172.21.15.86:0/4200573634 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d67e0d570 con 0x555d67fb0e60 2024-04-17T07:24:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.081+0000 7f57f5600640 10 client.26905 handle_command_reply: tid=0 2024-04-17T07:24:24.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.081+0000 7f581582d740 1 client.26905 shutdown 2024-04-17T07:24:24.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.082+0000 7f581582d740 1 -- 172.21.15.86:0/4200573634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 msgr2=0x555d67f4ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.082+0000 7f581582d740 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f4ab10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f57fc011b40 tx=0x7f57fc01b430 comp rx=0 tx=0).stop 2024-04-17T07:24:24.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.082+0000 7f581582d740 1 -- 172.21.15.86:0/4200573634 shutdown_connections 2024-04-17T07:24:24.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.082+0000 7f581582d740 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x555d67fb0e60 0x555d67fb3280 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.082+0000 7f581582d740 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d67f33c10 0x555d67f9e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.082+0000 7f581582d740 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d67f29b00 0x555d67f4ab10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.082+0000 7f581582d740 1 --2- 172.21.15.86:0/4200573634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d67f23fa0 0x555d67f4a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.082+0000 7f581582d740 1 -- 172.21.15.86:0/4200573634 >> 172.21.15.86:0/4200573634 conn(0x555d67f1f510 msgr2=0x555d67f2e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.082+0000 7f57f5600640 0 client.26905 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:24:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.082+0000 7f581582d740 1 -- 172.21.15.86:0/4200573634 shutdown_connections 2024-04-17T07:24:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.083+0000 7f581582d740 1 -- 172.21.15.86:0/4200573634 wait complete. 2024-04-17T07:24:24.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.083+0000 7f581582d740 20 client.26905 trim_cache size 0 max 16384 2024-04-17T07:24:24.083 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:24:24.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.085+0000 7f5812a00640 1 -- 172.21.15.86:0/1409598206 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57f00a3d30 msgr2=0x7f57f00a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.085+0000 7f5812a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57f00a3d30 0x7f57f00a61f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5804004070 tx=0x7f5804005f50 comp rx=0 tx=0).stop 2024-04-17T07:24:24.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.085+0000 7f5812a00640 1 -- 172.21.15.86:0/1409598206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f580c19fb70 msgr2=0x7f580c1a1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.085+0000 7f5812a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f580c19fb70 0x7f580c1a1f60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58040091e0 tx=0x7f58040096a0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.086+0000 7f5812a00640 1 -- 172.21.15.86:0/1409598206 shutdown_connections 2024-04-17T07:24:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.086+0000 7f5812a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57f00a3d30 0x7f57f00a61f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.086+0000 7f5812a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f580c19fb70 0x7f580c1a1f60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.086+0000 7f5812a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f580c101d80 0x7f580c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.086+0000 7f5812a00640 1 --2- 172.21.15.86:0/1409598206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f580c1001d0 0x7f580c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.086+0000 7f5812a00640 1 -- 172.21.15.86:0/1409598206 >> 172.21.15.86:0/1409598206 conn(0x7f580c0fb760 msgr2=0x7f580c104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.087+0000 7f5812a00640 1 -- 172.21.15.86:0/1409598206 shutdown_connections 2024-04-17T07:24:24.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.087+0000 7f5812a00640 1 -- 172.21.15.86:0/1409598206 wait complete. 2024-04-17T07:24:24.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:24:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 300.0 s, 613.8 tps, lat 162.797 ms stddev 43.769 2024-04-17T07:24:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.439+0000 7f84b1200640 1 -- 172.21.15.86:0/992640450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a40a4fe0 msgr2=0x7f84a40a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.440+0000 7f84b1200640 1 --2- 172.21.15.86:0/992640450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a40a4fe0 0x7f84a40a5400 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84ac067c60 tx=0x7f84ac091990 comp rx=0 tx=0).stop 2024-04-17T07:24:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.440+0000 7f84b1200640 1 -- 172.21.15.86:0/992640450 shutdown_connections 2024-04-17T07:24:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.440+0000 7f84b1200640 1 --2- 172.21.15.86:0/992640450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40a6b90 0x7f84a40ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.440+0000 7f84b1200640 1 --2- 172.21.15.86:0/992640450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84a40a61d0 0x7f84a40a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.440+0000 7f84b1200640 1 --2- 172.21.15.86:0/992640450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a40a4fe0 0x7f84a40a5400 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.440+0000 7f84b1200640 1 -- 172.21.15.86:0/992640450 >> 172.21.15.86:0/992640450 conn(0x7f84a40a0530 msgr2=0x7f84a40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.440+0000 7f84b1200640 1 -- 172.21.15.86:0/992640450 shutdown_connections 2024-04-17T07:24:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.441+0000 7f84b1200640 1 -- 172.21.15.86:0/992640450 wait complete. 2024-04-17T07:24:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.442+0000 7f84b1200640 1 Processor -- start 2024-04-17T07:24:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.443+0000 7f84b1200640 1 -- start start 2024-04-17T07:24:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.443+0000 7f84b1200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84a40a4fe0 0x7f84a4143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.443+0000 7f84b1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40a61d0 0x7f84a4143b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.443+0000 7f84b1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a40a6b90 0x7f84a414ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.443+0000 7f84b1200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84a40b0600 con 0x7f84a40a61d0 2024-04-17T07:24:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.443+0000 7f84b1200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f84a40b0480 con 0x7f84a40a6b90 2024-04-17T07:24:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.443+0000 7f84b1200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84a40b0780 con 0x7f84a40a4fe0 2024-04-17T07:24:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.444+0000 7f84aaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40a61d0 0x7f84a4143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.444+0000 7f84aaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40a61d0 0x7f84a4143b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38060/0 (socket says 172.21.15.86:38060) 2024-04-17T07:24:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.444+0000 7f84aaa00640 1 -- 172.21.15.86:0/3199792616 learned_addr learned my addr 172.21.15.86:0/3199792616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.444+0000 7f84abe00640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a40a6b90 0x7f84a414ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.444+0000 7f84ab400640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84a40a4fe0 0x7f84a4143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.445+0000 7f84aaa00640 1 -- 172.21.15.86:0/3199792616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84a40a4fe0 msgr2=0x7f84a4143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.445+0000 7f84aaa00640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84a40a4fe0 0x7f84a4143620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.445+0000 7f84aaa00640 1 -- 172.21.15.86:0/3199792616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a40a6b90 msgr2=0x7f84a414ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.445+0000 7f84aaa00640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a40a6b90 0x7f84a414ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.445+0000 7f84aaa00640 1 -- 172.21.15.86:0/3199792616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a414b1f0 con 0x7f84a40a61d0 2024-04-17T07:24:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.446+0000 7f84aaa00640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40a61d0 0x7f84a4143b60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f84a000e9f0 tx=0x7f84a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.447+0000 7f849fe00640 1 -- 172.21.15.86:0/3199792616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a000cd80 con 0x7f84a40a61d0 2024-04-17T07:24:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.448+0000 7f849fe00640 1 -- 172.21.15.86:0/3199792616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a00108e0 con 0x7f84a40a61d0 2024-04-17T07:24:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.448+0000 7f849fe00640 1 -- 172.21.15.86:0/3199792616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0018920 con 0x7f84a40a61d0 2024-04-17T07:24:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.448+0000 7f84b1200640 1 -- 172.21.15.86:0/3199792616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a414b4e0 con 0x7f84a40a61d0 2024-04-17T07:24:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.449+0000 7f84b1200640 1 -- 172.21.15.86:0/3199792616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a414b9e0 con 0x7f84a40a61d0 2024-04-17T07:24:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.455+0000 7f84b5419740 1 Processor -- start 2024-04-17T07:24:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.456+0000 7f84b5419740 1 -- start start 2024-04-17T07:24:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.457+0000 7f84b5419740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedaa160 0x5602bedaa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.458+0000 7f849fe00640 1 -- 172.21.15.86:0/3199792616 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f84a0018ac0 con 0x7f84a40a61d0 2024-04-17T07:24:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.458+0000 7f84b5419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 0x5602bedaf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.458+0000 7f84b5419740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedafcc0 0x5602bedb2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.458+0000 7f84b5419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5602bed10020 con 0x5602bedaab50 2024-04-17T07:24:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.458+0000 7f84b5419740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5602becb6460 con 0x5602bedaa160 2024-04-17T07:24:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.458+0000 7f84b5419740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602bed0bfe0 con 0x5602bedafcc0 2024-04-17T07:24:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.459+0000 7f84abe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedaa160 0x5602bedaa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.459+0000 7f84abe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedaa160 0x5602bedaa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42508/0 (socket says 172.21.15.86:42508) 2024-04-17T07:24:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.459+0000 7f84abe00640 1 -- 172.21.15.86:0/1055539163 learned_addr learned my addr 172.21.15.86:0/1055539163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.459+0000 7f84aaa00640 1 --2- 172.21.15.86:0/1055539163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedafcc0 0x5602bedb2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.459+0000 7f84ab400640 1 --2- 172.21.15.86:0/1055539163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 0x5602bedaf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.462+0000 7f84abe00640 1 -- 172.21.15.86:0/1055539163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedafcc0 msgr2=0x5602bedb2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.462+0000 7f84abe00640 1 --2- 172.21.15.86:0/1055539163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedafcc0 0x5602bedb2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.463+0000 7f84abe00640 1 -- 172.21.15.86:0/1055539163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 msgr2=0x5602bedaf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.463+0000 7f849fe00640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f84940a3d80 0x7f84940a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.463+0000 7f84abe00640 1 --2- 172.21.15.86:0/1055539163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 0x5602bedaf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.463+0000 7f84abe00640 1 -- 172.21.15.86:0/1055539163 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602becbe040 con 0x5602bedaa160 2024-04-17T07:24:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.464+0000 7f84abe00640 1 --2- 172.21.15.86:0/1055539163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedaa160 0x5602bedaa580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8498009c00 tx=0x7f84980117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a22e0e0a72bf6d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.464+0000 7f84aaa00640 1 --2- 172.21.15.86:0/1055539163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedafcc0 0x5602bedb2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.464+0000 7f84abe00640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f84940a3d80 0x7f84940a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.465+0000 7f849fe00640 1 -- 172.21.15.86:0/3199792616 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f84a00d0a80 con 0x7f84a40a61d0 2024-04-17T07:24:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.466+0000 7f84abe00640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f84940a3d80 0x7f84940a6240 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f849801c040 tx=0x7f8498017000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.466+0000 7f849cc00640 1 -- 172.21.15.86:0/1055539163 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498012970 con 0x5602bedaa160 2024-04-17T07:24:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.466+0000 7f849cc00640 1 -- 172.21.15.86:0/1055539163 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8498024430 con 0x5602bedaa160 2024-04-17T07:24:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.466+0000 7f849cc00640 1 -- 172.21.15.86:0/1055539163 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849800f6a0 con 0x5602bedaa160 2024-04-17T07:24:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.468+0000 7f84b5419740 1 -- 172.21.15.86:0/1055539163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedaa160 msgr2=0x5602bedaa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.468+0000 7f84b5419740 1 --2- 172.21.15.86:0/1055539163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedaa160 0x5602bedaa580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8498009c00 tx=0x7f84980117b0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.468+0000 7f84b5419740 1 -- 172.21.15.86:0/1055539163 shutdown_connections 2024-04-17T07:24:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.469+0000 7f84b5419740 1 --2- 172.21.15.86:0/1055539163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedafcc0 0x5602bedb2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.469+0000 7f84b5419740 1 --2- 172.21.15.86:0/1055539163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 0x5602bedaf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.469+0000 7f84b5419740 1 --2- 172.21.15.86:0/1055539163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedaa160 0x5602bedaa580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.469+0000 7f84b5419740 1 -- 172.21.15.86:0/1055539163 >> 172.21.15.86:0/1055539163 conn(0x5602beda56d0 msgr2=0x5602beda5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.469+0000 7f84b5419740 1 -- 172.21.15.86:0/1055539163 shutdown_connections 2024-04-17T07:24:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.469+0000 7f84b5419740 1 -- 172.21.15.86:0/1055539163 wait complete. 2024-04-17T07:24:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.470+0000 7f84b5419740 1 -- start start 2024-04-17T07:24:24.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.470+0000 7f84b5419740 1 Processor -- start 2024-04-17T07:24:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.471+0000 7f84b5419740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaa160 0x5602bedb4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.471+0000 7f84b5419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 0x5602bedb4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.471+0000 7f84b5419740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602bedbba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.472+0000 7f84aaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 0x5602bedb4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.472+0000 7f84b5419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5602bedb2c70 con 0x5602bedaab50 2024-04-17T07:24:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.472+0000 7f84aaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 0x5602bedb4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38066/0 (socket says 172.21.15.86:38066) 2024-04-17T07:24:24.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.472+0000 7f84aaa00640 1 -- 172.21.15.86:0/1749036887 learned_addr learned my addr 172.21.15.86:0/1749036887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.472+0000 7f84b5419740 1 -- 172.21.15.86:0/1749036887 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5602becc9830 con 0x5602bedafcc0 2024-04-17T07:24:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.472+0000 7f84b5419740 1 -- 172.21.15.86:0/1749036887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5602bedb2df0 con 0x5602bedaa160 2024-04-17T07:24:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.472+0000 7f84b5419740 10 client.0 _ll_register_callbacks cb 0x5602bec97fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.472+0000 7f84ab400640 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaa160 0x5602bedb4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.473+0000 7f84abe00640 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602bedbba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.473+0000 7f84ab400640 1 -- 172.21.15.86:0/1749036887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 msgr2=0x5602bedbba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.473+0000 7f84ab400640 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602bedbba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.473+0000 7f84ab400640 1 -- 172.21.15.86:0/1749036887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 msgr2=0x5602bedb4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.475+0000 7f84ab400640 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 0x5602bedb4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.475+0000 7f84ab400640 1 -- 172.21.15.86:0/1749036887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602beccd920 con 0x5602bedaa160 2024-04-17T07:24:24.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.476+0000 7f84ab400640 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaa160 0x5602bedb4560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f84ac067ad0 tx=0x7f84ac0a2670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71e0b682e883cac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.476+0000 7f8491600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:24.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.476+0000 7f8491600640 1 -- 172.21.15.86:0/1749036887 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac068180 con 0x5602bedaa160 2024-04-17T07:24:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.476+0000 7f8491600640 1 -- 172.21.15.86:0/1749036887 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f84ac0a2a70 con 0x5602bedaa160 2024-04-17T07:24:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.477+0000 7f84b5419740 1 -- 172.21.15.86:0/1749036887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8498024430 con 0x5602bedaa160 2024-04-17T07:24:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.477+0000 7f8491600640 1 -- 172.21.15.86:0/1749036887 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84ac06a280 con 0x5602bedaa160 2024-04-17T07:24:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.477+0000 7f84b5419740 10 client.26917 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:24.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.477+0000 7f84b5419740 1 -- 172.21.15.86:0/1749036887 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602bec9b7f0 con 0x5602bedaa160 2024-04-17T07:24:24.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.478+0000 7f8491600640 1 -- 172.21.15.86:0/1749036887 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8498024430 con 0x5602bedaa160 2024-04-17T07:24:24.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.479+0000 7f8491600640 1 -- 172.21.15.86:0/1749036887 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f84ac0a9020 con 0x5602bedaa160 2024-04-17T07:24:24.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.479+0000 7f84b5419740 10 client.26917 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:24.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.479+0000 7f84b5419740 10 client.26917 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:24:24.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.479+0000 7f84b5419740 20 client.26917 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:24.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.479+0000 7f84b5419740 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5602bedbff80 0x5602bedc23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.480+0000 7f84b5419740 4 client.26917 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:24.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.480+0000 7f84b5419740 1 -- 172.21.15.86:0/1749036887 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602bed1da40 con 0x5602bedbff80 2024-04-17T07:24:24.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.480+0000 7f84aaa00640 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5602bedbff80 0x5602bedc23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.481+0000 7f84aaa00640 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5602bedbff80 0x5602bedc23e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5cf7673bf958f14d server_cookie=ab68867e890c5914 in_seq=0 out_seq=0 2024-04-17T07:24:24.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.481+0000 7f8491600640 10 client.26917 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:24:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.483+0000 7f8491600640 1 -- 172.21.15.86:0/1749036887 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602bed1da40 con 0x5602bedbff80 2024-04-17T07:24:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.483+0000 7f8491600640 10 client.26917 handle_command_reply: tid=0 2024-04-17T07:24:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.483+0000 7f84b5419740 1 client.26917 shutdown 2024-04-17T07:24:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 1 -- 172.21.15.86:0/1749036887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaa160 msgr2=0x5602bedb4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaa160 0x5602bedb4560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f84ac067ad0 tx=0x7f84ac0a2670 comp rx=0 tx=0).stop 2024-04-17T07:24:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 1 -- 172.21.15.86:0/1749036887 shutdown_connections 2024-04-17T07:24:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5602bedbff80 0x5602bedc23e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602bedbba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaab50 0x5602bedb4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 1 --2- 172.21.15.86:0/1749036887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaa160 0x5602bedb4560 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 1 -- 172.21.15.86:0/1749036887 >> 172.21.15.86:0/1749036887 conn(0x5602beda56d0 msgr2=0x5602beda4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f8491600640 0 client.26917 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:24:24.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 1 -- 172.21.15.86:0/1749036887 shutdown_connections 2024-04-17T07:24:24.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 1 -- 172.21.15.86:0/1749036887 wait complete. 2024-04-17T07:24:24.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.484+0000 7f84b5419740 20 client.26917 trim_cache size 0 max 16384 2024-04-17T07:24:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.511+0000 7f84b5419740 1 Processor -- start 2024-04-17T07:24:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.511+0000 7f84b5419740 1 -- start start 2024-04-17T07:24:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.512+0000 7f84b5419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 0x5602beda40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.512+0000 7f84b5419740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 0x5602bedbba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.512+0000 7f84b5419740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602beda45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.512+0000 7f84b5419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4155e60 con 0x5602bedaa160 2024-04-17T07:24:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.512+0000 7f84b5419740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41579a0 con 0x5602bedafcc0 2024-04-17T07:24:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.512+0000 7f84b5419740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4156530 con 0x5602bedaab50 2024-04-17T07:24:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.513+0000 7f84ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 0x5602beda40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.513+0000 7f84ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 0x5602beda40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38068/0 (socket says 172.21.15.86:38068) 2024-04-17T07:24:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.513+0000 7f84ab400640 1 -- 172.21.15.86:0/2049039601 learned_addr learned my addr 172.21.15.86:0/2049039601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.513+0000 7f84aaa00640 1 --2- 172.21.15.86:0/2049039601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 0x5602bedbba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.513+0000 7f84abe00640 1 --2- 172.21.15.86:0/2049039601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602beda45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.514+0000 7f84ab400640 1 -- 172.21.15.86:0/2049039601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 msgr2=0x5602bedbba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.514+0000 7f84ab400640 1 --2- 172.21.15.86:0/2049039601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 0x5602bedbba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.514+0000 7f84ab400640 1 -- 172.21.15.86:0/2049039601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 msgr2=0x5602beda45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.514+0000 7f84ab400640 1 --2- 172.21.15.86:0/2049039601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602beda45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.514+0000 7f84ab400640 1 -- 172.21.15.86:0/2049039601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602bea223f0 con 0x5602bedaa160 2024-04-17T07:24:24.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.516+0000 7f84ab400640 1 --2- 172.21.15.86:0/2049039601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 0x5602beda40a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f84ac050f60 tx=0x7f84ac0633f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5269e9f006c2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.517+0000 7f8493e00640 1 -- 172.21.15.86:0/2049039601 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac06a280 con 0x5602bedaa160 2024-04-17T07:24:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.518+0000 7f8493e00640 1 -- 172.21.15.86:0/2049039601 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84ac06a280 con 0x5602bedaa160 2024-04-17T07:24:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.518+0000 7f8493e00640 1 -- 172.21.15.86:0/2049039601 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac0a2a70 con 0x5602bedaa160 2024-04-17T07:24:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.519+0000 7f84b5419740 1 -- 172.21.15.86:0/2049039601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 msgr2=0x5602beda40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.519+0000 7f84b5419740 1 --2- 172.21.15.86:0/2049039601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 0x5602beda40a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f84ac050f60 tx=0x7f84ac0633f0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.520+0000 7f84b5419740 1 -- 172.21.15.86:0/2049039601 shutdown_connections 2024-04-17T07:24:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.520+0000 7f84b5419740 1 --2- 172.21.15.86:0/2049039601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602beda45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.520+0000 7f84b5419740 1 --2- 172.21.15.86:0/2049039601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 0x5602bedbba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.520+0000 7f84b5419740 1 --2- 172.21.15.86:0/2049039601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 0x5602beda40a0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.520+0000 7f84b5419740 1 -- 172.21.15.86:0/2049039601 >> 172.21.15.86:0/2049039601 conn(0x5602beda56d0 msgr2=0x5602bedabf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.520+0000 7f84b5419740 1 -- 172.21.15.86:0/2049039601 shutdown_connections 2024-04-17T07:24:24.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.521+0000 7f84b5419740 1 -- 172.21.15.86:0/2049039601 wait complete. 2024-04-17T07:24:24.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.521+0000 7f84b5419740 1 -- start start 2024-04-17T07:24:24.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.522+0000 7f84b5419740 1 Processor -- start 2024-04-17T07:24:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.522+0000 7f84b5419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 0x5602bedb6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.522+0000 7f84b5419740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 0x5602bedb6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.522+0000 7f84b5419740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602bee35520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.522+0000 7f84b5419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41572d0 con 0x5602bedaa160 2024-04-17T07:24:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.522+0000 7f84b5419740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4156c00 con 0x5602bedafcc0 2024-04-17T07:24:24.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.522+0000 7f84b5419740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4153e30 con 0x5602bedaab50 2024-04-17T07:24:24.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.522+0000 7f84b5419740 10 client.0 _ll_register_callbacks cb 0x7f84ac0723e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:24.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.523+0000 7f84ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 0x5602bedb6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.523+0000 7f84aaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 0x5602bedb6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.523+0000 7f84aaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 0x5602bedb6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46154/0 (socket says 172.21.15.86:46154) 2024-04-17T07:24:24.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.523+0000 7f84aaa00640 1 -- 172.21.15.86:0/3396019765 learned_addr learned my addr 172.21.15.86:0/3396019765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.523+0000 7f84abe00640 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602bee35520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.524+0000 7f84aaa00640 1 -- 172.21.15.86:0/3396019765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 msgr2=0x5602bee35520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.524+0000 7f84aaa00640 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602bee35520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.524+0000 7f84aaa00640 1 -- 172.21.15.86:0/3396019765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 msgr2=0x5602bedb6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.524+0000 7f84aaa00640 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 0x5602bedb6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.524+0000 7f84aaa00640 1 -- 172.21.15.86:0/3396019765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602beaec7d0 con 0x5602bedaab50 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.524+0000 7f84abe00640 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602bee35520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.524+0000 7f84aaa00640 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 0x5602bedb6bd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84a00d0db0 tx=0x7f84a00bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b92d3baa2c9e49ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.524+0000 7f8491600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.525+0000 7f8491600640 1 -- 172.21.15.86:0/3396019765 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a009d080 con 0x5602bedaab50 2024-04-17T07:24:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.525+0000 7f84b5419740 1 -- 172.21.15.86:0/3396019765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84ac06a280 con 0x5602bedaab50 2024-04-17T07:24:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.525+0000 7f8491600640 1 -- 172.21.15.86:0/3396019765 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f84a0010a90 con 0x5602bedaab50 2024-04-17T07:24:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.525+0000 7f8491600640 1 -- 172.21.15.86:0/3396019765 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84a009c980 con 0x5602bedaab50 2024-04-17T07:24:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.526+0000 7f84b5419740 10 client.26923 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.526+0000 7f84b5419740 1 -- 172.21.15.86:0/3396019765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602bee36990 con 0x5602bedaab50 2024-04-17T07:24:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.526+0000 7f8491600640 1 -- 172.21.15.86:0/3396019765 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84ac06a280 con 0x5602bedaab50 2024-04-17T07:24:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.527+0000 7f8491600640 1 -- 172.21.15.86:0/3396019765 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f84a0010430 con 0x5602bedaab50 2024-04-17T07:24:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.527+0000 7f84b5419740 10 client.26923 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.527+0000 7f84b5419740 10 client.26923 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:24:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.527+0000 7f84b5419740 20 client.26923 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.527+0000 7f84b5419740 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5602bee372f0 0x5602bee39710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.527+0000 7f84b5419740 4 client.26923 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:24:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.527+0000 7f84b5419740 1 -- 172.21.15.86:0/3396019765 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602bed178b0 con 0x5602bee372f0 2024-04-17T07:24:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.528+0000 7f84ab400640 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5602bee372f0 0x5602bee39710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.529+0000 7f84ab400640 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5602bee372f0 0x5602bee39710 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3a28a1b39ab15f7 server_cookie=865c8bbd324470ff in_seq=0 out_seq=0 2024-04-17T07:24:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.529+0000 7f8491600640 10 client.26923 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:24:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.530+0000 7f8491600640 1 -- 172.21.15.86:0/3396019765 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602bed178b0 con 0x5602bee372f0 2024-04-17T07:24:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.530+0000 7f8491600640 10 client.26923 handle_command_reply: tid=0 2024-04-17T07:24:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.530+0000 7f84b5419740 1 client.26923 shutdown 2024-04-17T07:24:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.530+0000 7f84b5419740 1 -- 172.21.15.86:0/3396019765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 msgr2=0x5602bedb6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.530+0000 7f84b5419740 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 0x5602bedb6bd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84a00d0db0 tx=0x7f84a00bd040 comp rx=0 tx=0).stop 2024-04-17T07:24:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.531+0000 7f84b5419740 1 -- 172.21.15.86:0/3396019765 shutdown_connections 2024-04-17T07:24:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.531+0000 7f84b5419740 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5602bee372f0 0x5602bee39710 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.531+0000 7f84b5419740 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602bedafcc0 0x5602bee35520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.531+0000 7f84b5419740 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5602bedaab50 0x5602bedb6bd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.531+0000 7f84b5419740 1 --2- 172.21.15.86:0/3396019765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602bedaa160 0x5602bedb6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.531+0000 7f84b5419740 1 -- 172.21.15.86:0/3396019765 >> 172.21.15.86:0/3396019765 conn(0x5602beda56d0 msgr2=0x5602beda8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.531+0000 7f8491600640 0 client.26923 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:24:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.531+0000 7f84b5419740 1 -- 172.21.15.86:0/3396019765 shutdown_connections 2024-04-17T07:24:24.532 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:24:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.531+0000 7f84b5419740 1 -- 172.21.15.86:0/3396019765 wait complete. 2024-04-17T07:24:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.531+0000 7f84b5419740 20 client.26923 trim_cache size 0 max 16384 2024-04-17T07:24:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.532+0000 7f84b1200640 1 -- 172.21.15.86:0/3199792616 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f84940a3d80 msgr2=0x7f84940a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.532+0000 7f84b1200640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f84940a3d80 0x7f84940a6240 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f849801c040 tx=0x7f8498017000 comp rx=0 tx=0).stop 2024-04-17T07:24:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.532+0000 7f84b1200640 1 -- 172.21.15.86:0/3199792616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40a61d0 msgr2=0x7f84a4143b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.532+0000 7f84b1200640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40a61d0 0x7f84a4143b60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f84a000e9f0 tx=0x7f84a000eeb0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.532+0000 7f84b1200640 1 -- 172.21.15.86:0/3199792616 shutdown_connections 2024-04-17T07:24:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.532+0000 7f84b1200640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f84940a3d80 0x7f84940a6240 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.532+0000 7f84b1200640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a40a6b90 0x7f84a414ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.532+0000 7f84b1200640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40a61d0 0x7f84a4143b60 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.532+0000 7f84b1200640 1 --2- 172.21.15.86:0/3199792616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84a40a4fe0 0x7f84a4143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.532+0000 7f84b1200640 1 -- 172.21.15.86:0/3199792616 >> 172.21.15.86:0/3199792616 conn(0x7f84a40a0530 msgr2=0x7f84a40a73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.533+0000 7f84b1200640 1 -- 172.21.15.86:0/3199792616 shutdown_connections 2024-04-17T07:24:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.533+0000 7f84b1200640 1 -- 172.21.15.86:0/3199792616 wait complete. 2024-04-17T07:24:24.568 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:24:24.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.876+0000 7fecb1c00640 1 -- 172.21.15.86:0/935560155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecac1041c0 msgr2=0x7fecac1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.876+0000 7fecb1c00640 1 --2- 172.21.15.86:0/935560155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecac1041c0 0x7fecac1045e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009900 tx=0x7fec9c02bc50 comp rx=0 tx=0).stop 2024-04-17T07:24:24.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.876+0000 7fecb1c00640 1 -- 172.21.15.86:0/935560155 shutdown_connections 2024-04-17T07:24:24.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.876+0000 7fecb1c00640 1 --2- 172.21.15.86:0/935560155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecac105d70 0x7fecac10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.876+0000 7fecb1c00640 1 --2- 172.21.15.86:0/935560155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac1053b0 0x7fecac105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.876+0000 7fecb1c00640 1 --2- 172.21.15.86:0/935560155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecac1041c0 0x7fecac1045e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.876+0000 7fecb1c00640 1 -- 172.21.15.86:0/935560155 >> 172.21.15.86:0/935560155 conn(0x7fecac0ff750 msgr2=0x7fecac101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.876+0000 7fecb1c00640 1 -- 172.21.15.86:0/935560155 shutdown_connections 2024-04-17T07:24:24.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.876+0000 7fecb1c00640 1 -- 172.21.15.86:0/935560155 wait complete. 2024-04-17T07:24:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.878+0000 7fecb1c00640 1 Processor -- start 2024-04-17T07:24:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.878+0000 7fecb1c00640 1 -- start start 2024-04-17T07:24:24.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.878+0000 7fecb1c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecac1041c0 0x7fecac19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.878+0000 7fecb1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecac1053b0 0x7fecac19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.878+0000 7fecb1c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac105d70 0x7fecac1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.878+0000 7fecb1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecac10fd50 con 0x7fecac1053b0 2024-04-17T07:24:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.878+0000 7fecb1c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fecac10fbd0 con 0x7fecac105d70 2024-04-17T07:24:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.878+0000 7fecb1c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fecac10fed0 con 0x7fecac1041c0 2024-04-17T07:24:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.879+0000 7fecaa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecac1053b0 0x7fecac19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.879+0000 7fecaa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecac1053b0 0x7fecac19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38086/0 (socket says 172.21.15.86:38086) 2024-04-17T07:24:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.879+0000 7fecaa000640 1 -- 172.21.15.86:0/1804744361 learned_addr learned my addr 172.21.15.86:0/1804744361 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.879+0000 7fecab400640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac105d70 0x7fecac1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.880+0000 7fecaa000640 1 -- 172.21.15.86:0/1804744361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecac1041c0 msgr2=0x7fecac19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.880+0000 7fecaa000640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecac1041c0 0x7fecac19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.880+0000 7fecaa000640 1 -- 172.21.15.86:0/1804744361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac105d70 msgr2=0x7fecac1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.880+0000 7fecaa000640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac105d70 0x7fecac1a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.880+0000 7fecaa000640 1 -- 172.21.15.86:0/1804744361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecac1a2090 con 0x7fecac1053b0 2024-04-17T07:24:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.880+0000 7fecaa000640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecac1053b0 0x7fecac19aa00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fec9800e9f0 tx=0x7fec9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.881+0000 7feca3400640 1 -- 172.21.15.86:0/1804744361 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9800cd80 con 0x7fecac1053b0 2024-04-17T07:24:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.881+0000 7fecb1c00640 1 -- 172.21.15.86:0/1804744361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecac1a2380 con 0x7fecac1053b0 2024-04-17T07:24:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.881+0000 7fecb1c00640 1 -- 172.21.15.86:0/1804744361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecac1a28d0 con 0x7fecac1053b0 2024-04-17T07:24:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.881+0000 7feca3400640 1 -- 172.21.15.86:0/1804744361 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec980054d0 con 0x7fecac1053b0 2024-04-17T07:24:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.881+0000 7feca3400640 1 -- 172.21.15.86:0/1804744361 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98010470 con 0x7fecac1053b0 2024-04-17T07:24:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.887+0000 7fecb4813740 1 Processor -- start 2024-04-17T07:24:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.888+0000 7feca3400640 1 -- 172.21.15.86:0/1804744361 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fec98010750 con 0x7fecac1053b0 2024-04-17T07:24:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.888+0000 7fecb4813740 1 -- start start 2024-04-17T07:24:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.889+0000 7fecb4813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b8fa0 0x55b2a20b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.889+0000 7fecb4813740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b9990 0x55b2a20be490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.889+0000 7fecb4813740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20beb00 0x55b2a20c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.889+0000 7fecb4813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2a201f020 con 0x55b2a20b8fa0 2024-04-17T07:24:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.889+0000 7fecb4813740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b2a1fc5460 con 0x55b2a20beb00 2024-04-17T07:24:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.889+0000 7fecb4813740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b2a201afe0 con 0x55b2a20b9990 2024-04-17T07:24:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.890+0000 7fecab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b8fa0 0x55b2a20b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.890+0000 7fecab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b8fa0 0x55b2a20b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38088/0 (socket says 172.21.15.86:38088) 2024-04-17T07:24:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.890+0000 7fecab400640 1 -- 172.21.15.86:0/1257033833 learned_addr learned my addr 172.21.15.86:0/1257033833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.890+0000 7fecaa000640 1 --2- 172.21.15.86:0/1257033833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20beb00 0x55b2a20c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.890+0000 7fecab400640 1 -- 172.21.15.86:0/1257033833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b9990 msgr2=0x55b2a20be490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.890+0000 7fecab400640 1 --2- 172.21.15.86:0/1257033833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b9990 0x55b2a20be490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.890+0000 7fecab400640 1 -- 172.21.15.86:0/1257033833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20beb00 msgr2=0x55b2a20c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.890+0000 7fecab400640 1 --2- 172.21.15.86:0/1257033833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20beb00 0x55b2a20c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.890+0000 7fecab400640 1 -- 172.21.15.86:0/1257033833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2a1fcd040 con 0x55b2a20b8fa0 2024-04-17T07:24:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.891+0000 7feca3400640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec900a3d00 0x7fec900a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.891+0000 7fecab400640 1 --2- 172.21.15.86:0/1257033833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b8fa0 0x55b2a20b93c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7feca4009fa0 tx=0x7feca400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4aca3f551d1701b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.891+0000 7feca3400640 1 -- 172.21.15.86:0/1804744361 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fec980d0b60 con 0x7fecac1053b0 2024-04-17T07:24:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.892+0000 7fec97e00640 1 -- 172.21.15.86:0/1257033833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4015070 con 0x55b2a20b8fa0 2024-04-17T07:24:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.893+0000 7fec97e00640 1 -- 172.21.15.86:0/1257033833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feca401ab10 con 0x55b2a20b8fa0 2024-04-17T07:24:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.893+0000 7fec97e00640 1 -- 172.21.15.86:0/1257033833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4011b90 con 0x55b2a20b8fa0 2024-04-17T07:24:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.893+0000 7fecaaa00640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec900a3d00 0x7fec900a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.894+0000 7fecb4813740 1 -- 172.21.15.86:0/1257033833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b8fa0 msgr2=0x55b2a20b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.894+0000 7fecb4813740 1 --2- 172.21.15.86:0/1257033833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b8fa0 0x55b2a20b93c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7feca4009fa0 tx=0x7feca400f7f0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.895+0000 7fecb4813740 1 -- 172.21.15.86:0/1257033833 shutdown_connections 2024-04-17T07:24:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.895+0000 7fecb4813740 1 --2- 172.21.15.86:0/1257033833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20beb00 0x55b2a20c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.895+0000 7fecb4813740 1 --2- 172.21.15.86:0/1257033833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b9990 0x55b2a20be490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.895+0000 7fecb4813740 1 --2- 172.21.15.86:0/1257033833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b8fa0 0x55b2a20b93c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.895+0000 7fecb4813740 1 -- 172.21.15.86:0/1257033833 >> 172.21.15.86:0/1257033833 conn(0x55b2a20b4510 msgr2=0x55b2a20b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.896+0000 7fecb4813740 1 -- 172.21.15.86:0/1257033833 shutdown_connections 2024-04-17T07:24:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.896+0000 7fecb4813740 1 -- 172.21.15.86:0/1257033833 wait complete. 2024-04-17T07:24:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.897+0000 7fecb4813740 1 -- start start 2024-04-17T07:24:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.897+0000 7fecaaa00640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec900a3d00 0x7fec900a61c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009230 tx=0x7fec9c02c680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.897+0000 7fecb4813740 1 Processor -- start 2024-04-17T07:24:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.898+0000 7fecb4813740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.898+0000 7fecb4813740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b9990 0x55b2a20c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.898+0000 7fecb4813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 0x55b2a20caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.898+0000 7fecb4813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2a20c1b00 con 0x55b2a20beb00 2024-04-17T07:24:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.898+0000 7fecb4813740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b2a1fd8830 con 0x55b2a20b9990 2024-04-17T07:24:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.898+0000 7fecb4813740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b2a20c1c80 con 0x55b2a20b8fa0 2024-04-17T07:24:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.898+0000 7fecb4813740 10 client.0 _ll_register_callbacks cb 0x55b2a1fa6fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.899+0000 7fecaaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.899+0000 7fecaaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46200/0 (socket says 172.21.15.86:46200) 2024-04-17T07:24:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.899+0000 7fecaaa00640 1 -- 172.21.15.86:0/1847674870 learned_addr learned my addr 172.21.15.86:0/1847674870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.899+0000 7fecab400640 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 0x55b2a20caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.899+0000 7fecaa000640 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b9990 0x55b2a20c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.899+0000 7fecaaa00640 1 -- 172.21.15.86:0/1847674870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b9990 msgr2=0x55b2a20c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.899+0000 7fecaaa00640 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b9990 0x55b2a20c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.899+0000 7fecaaa00640 1 -- 172.21.15.86:0/1847674870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 msgr2=0x55b2a20caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.900+0000 7fecaaa00640 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 0x55b2a20caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.900+0000 7fecaaa00640 1 -- 172.21.15.86:0/1847674870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2a1fdc920 con 0x55b2a20b8fa0 2024-04-17T07:24:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.900+0000 7fecab400640 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 0x55b2a20caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:24.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.901+0000 7fecaaa00640 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c34b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fecac105210 tx=0x7fec9c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff787c9280c2ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.902+0000 7fec94c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.903+0000 7fec94c00640 1 -- 172.21.15.86:0/1847674870 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c004240 con 0x55b2a20b8fa0 2024-04-17T07:24:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.903+0000 7fec94c00640 1 -- 172.21.15.86:0/1847674870 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fec9c0043e0 con 0x55b2a20b8fa0 2024-04-17T07:24:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.903+0000 7fecb4813740 1 -- 172.21.15.86:0/1847674870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca401ab10 con 0x55b2a20b8fa0 2024-04-17T07:24:24.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.904+0000 7fec94c00640 1 -- 172.21.15.86:0/1847674870 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec9c0035f0 con 0x55b2a20b8fa0 2024-04-17T07:24:24.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.904+0000 7fecb4813740 10 client.26929 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:24.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.904+0000 7fecb4813740 1 -- 172.21.15.86:0/1847674870 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2a1faa7f0 con 0x55b2a20b8fa0 2024-04-17T07:24:24.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.904+0000 7fec94c00640 1 -- 172.21.15.86:0/1847674870 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feca401ab10 con 0x55b2a20b8fa0 2024-04-17T07:24:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.905+0000 7fec94c00640 1 -- 172.21.15.86:0/1847674870 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fec9c034950 con 0x55b2a20b8fa0 2024-04-17T07:24:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.905+0000 7fecb4813740 10 client.26929 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:24.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.905+0000 7fecb4813740 10 client.26929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:24:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.905+0000 7fecb4813740 20 client.26929 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.906+0000 7fecb4813740 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b2a20cef40 0x55b2a20d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.906+0000 7fecb4813740 4 client.26929 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.906+0000 7fecb4813740 1 -- 172.21.15.86:0/1847674870 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2a202ca40 con 0x55b2a20cef40 2024-04-17T07:24:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.906+0000 7fecaa000640 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b2a20cef40 0x55b2a20d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.908+0000 7fec94c00640 10 client.26929 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:24.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.909+0000 7fecaa000640 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b2a20cef40 0x55b2a20d13c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b69e1ca632604424 server_cookie=f700d56fd379d1e0 in_seq=0 out_seq=0 2024-04-17T07:24:24.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.912+0000 7fec94c00640 1 -- 172.21.15.86:0/1847674870 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2a202ca40 con 0x55b2a20cef40 2024-04-17T07:24:24.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.912+0000 7fec94c00640 10 client.26929 handle_command_reply: tid=0 2024-04-17T07:24:24.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.912+0000 7fecb4813740 1 client.26929 shutdown 2024-04-17T07:24:24.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.913+0000 7fecb4813740 1 -- 172.21.15.86:0/1847674870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b8fa0 msgr2=0x55b2a20c34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.913+0000 7fecb4813740 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c34b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fecac105210 tx=0x7fec9c005fd0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.913+0000 7fecb4813740 1 -- 172.21.15.86:0/1847674870 shutdown_connections 2024-04-17T07:24:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.914+0000 7fecb4813740 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b2a20cef40 0x55b2a20d13c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.914+0000 7fecb4813740 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 0x55b2a20caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.914+0000 7fecb4813740 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b9990 0x55b2a20c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.914+0000 7fecb4813740 1 --2- 172.21.15.86:0/1847674870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c34b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.914+0000 7fec94c00640 0 client.26929 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:24.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.914+0000 7fecb4813740 1 -- 172.21.15.86:0/1847674870 >> 172.21.15.86:0/1847674870 conn(0x55b2a20b4510 msgr2=0x55b2a20b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.915+0000 7fecb4813740 1 -- 172.21.15.86:0/1847674870 shutdown_connections 2024-04-17T07:24:24.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.915+0000 7fecb4813740 1 -- 172.21.15.86:0/1847674870 wait complete. 2024-04-17T07:24:24.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.915+0000 7fecb4813740 20 client.26929 trim_cache size 0 max 16384 2024-04-17T07:24:24.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.943+0000 7fecb4813740 1 Processor -- start 2024-04-17T07:24:24.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.944+0000 7fecb4813740 1 -- start start 2024-04-17T07:24:24.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.945+0000 7fecb4813740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.945+0000 7fecb4813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b9990 0x55b2a20ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.945+0000 7fecb4813740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20beb00 0x55b2a20b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.945+0000 7fecb4813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c00fc70 con 0x55b2a20b9990 2024-04-17T07:24:24.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.945+0000 7fecb4813740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c0117b0 con 0x55b2a20b8fa0 2024-04-17T07:24:24.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.945+0000 7fecb4813740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c010340 con 0x55b2a20beb00 2024-04-17T07:24:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.945+0000 7fecaaa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.945+0000 7fecaaa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42564/0 (socket says 172.21.15.86:42564) 2024-04-17T07:24:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.945+0000 7fecaaa00640 1 -- 172.21.15.86:0/2033249306 learned_addr learned my addr 172.21.15.86:0/2033249306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.946+0000 7fecaa000640 1 --2- 172.21.15.86:0/2033249306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b9990 0x55b2a20ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.946+0000 7fecab400640 1 --2- 172.21.15.86:0/2033249306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20beb00 0x55b2a20b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.947+0000 7fecaa000640 1 -- 172.21.15.86:0/2033249306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20beb00 msgr2=0x55b2a20b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.947+0000 7fecaa000640 1 --2- 172.21.15.86:0/2033249306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20beb00 0x55b2a20b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.947+0000 7fecaa000640 1 -- 172.21.15.86:0/2033249306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 msgr2=0x55b2a20c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.947+0000 7fecaa000640 1 --2- 172.21.15.86:0/2033249306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.947+0000 7fecaa000640 1 -- 172.21.15.86:0/2033249306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2a1d313f0 con 0x55b2a20b9990 2024-04-17T07:24:24.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.947+0000 7fecab400640 1 --2- 172.21.15.86:0/2033249306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20beb00 0x55b2a20b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:24:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.949+0000 7fecaa000640 1 --2- 172.21.15.86:0/2033249306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b9990 0x55b2a20ca9c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fec9809c760 tx=0x7fec980ce150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75ff009f99339c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.950+0000 7fec97400640 1 -- 172.21.15.86:0/2033249306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98005680 con 0x55b2a20b9990 2024-04-17T07:24:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.950+0000 7fec97400640 1 -- 172.21.15.86:0/2033249306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec980bd0b0 con 0x55b2a20b9990 2024-04-17T07:24:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.950+0000 7fec97400640 1 -- 172.21.15.86:0/2033249306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9809ad60 con 0x55b2a20b9990 2024-04-17T07:24:24.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.951+0000 7fecb4813740 1 -- 172.21.15.86:0/2033249306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b9990 msgr2=0x55b2a20ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.951+0000 7fecb4813740 1 --2- 172.21.15.86:0/2033249306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b9990 0x55b2a20ca9c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fec9809c760 tx=0x7fec980ce150 comp rx=0 tx=0).stop 2024-04-17T07:24:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.951+0000 7fecb4813740 1 -- 172.21.15.86:0/2033249306 shutdown_connections 2024-04-17T07:24:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.951+0000 7fecb4813740 1 --2- 172.21.15.86:0/2033249306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20beb00 0x55b2a20b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.951+0000 7fecb4813740 1 --2- 172.21.15.86:0/2033249306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20b9990 0x55b2a20ca9c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.951+0000 7fecb4813740 1 --2- 172.21.15.86:0/2033249306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.951+0000 7fecb4813740 1 -- 172.21.15.86:0/2033249306 >> 172.21.15.86:0/2033249306 conn(0x55b2a20b4510 msgr2=0x55b2a20c30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.952+0000 7fecb4813740 1 -- 172.21.15.86:0/2033249306 shutdown_connections 2024-04-17T07:24:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.952+0000 7fecb4813740 1 -- 172.21.15.86:0/2033249306 wait complete. 2024-04-17T07:24:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.952+0000 7fecb4813740 1 -- start start 2024-04-17T07:24:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.953+0000 7fecb4813740 1 Processor -- start 2024-04-17T07:24:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.953+0000 7fecb4813740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.953+0000 7fecb4813740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 0x55b2a20c8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.953+0000 7fecb4813740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20c98a0 0x55b2a20c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.953+0000 7fecb4813740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c0110e0 con 0x55b2a20beb00 2024-04-17T07:24:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.953+0000 7fecb4813740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c010a10 con 0x55b2a20b8fa0 2024-04-17T07:24:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.954+0000 7fecb4813740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c00dc40 con 0x55b2a20c98a0 2024-04-17T07:24:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.954+0000 7fecb4813740 10 client.0 _ll_register_callbacks cb 0x7fec9c02cd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:24.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.954+0000 7fecaaa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.955+0000 7fecaaa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42576/0 (socket says 172.21.15.86:42576) 2024-04-17T07:24:24.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.955+0000 7fecaaa00640 1 -- 172.21.15.86:0/3642436655 learned_addr learned my addr 172.21.15.86:0/3642436655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:24.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.955+0000 7fecaa000640 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 0x55b2a20c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.955+0000 7fecab400640 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20c98a0 0x55b2a20c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.956+0000 7fecab400640 1 -- 172.21.15.86:0/3642436655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 msgr2=0x55b2a20c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.956+0000 7fecab400640 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.956+0000 7fecab400640 1 -- 172.21.15.86:0/3642436655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 msgr2=0x55b2a20c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.956+0000 7fecab400640 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 0x55b2a20c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.956+0000 7fecab400640 1 -- 172.21.15.86:0/3642436655 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2a1dfb7d0 con 0x55b2a20c98a0 2024-04-17T07:24:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.956+0000 7fecaaa00640 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.956+0000 7fecaa000640 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 0x55b2a20c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.956+0000 7fecab400640 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20c98a0 0x55b2a20c9d20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feca4006010 tx=0x7feca4004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e374bab0fb4a459d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.956+0000 7fec94c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.957+0000 7fec94c00640 1 -- 172.21.15.86:0/3642436655 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca402b060 con 0x55b2a20c98a0 2024-04-17T07:24:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.957+0000 7fec94c00640 1 -- 172.21.15.86:0/3642436655 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feca4011a50 con 0x55b2a20c98a0 2024-04-17T07:24:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.957+0000 7fecb4813740 1 -- 172.21.15.86:0/3642436655 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec980bd0b0 con 0x55b2a20c98a0 2024-04-17T07:24:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.958+0000 7fec94c00640 1 -- 172.21.15.86:0/3642436655 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feca4032050 con 0x55b2a20c98a0 2024-04-17T07:24:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.958+0000 7fecb4813740 10 client.26935 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.958+0000 7fecb4813740 1 -- 172.21.15.86:0/3642436655 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2a21469b0 con 0x55b2a20c98a0 2024-04-17T07:24:24.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.959+0000 7fec94c00640 1 -- 172.21.15.86:0/3642436655 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec980bd0b0 con 0x55b2a20c98a0 2024-04-17T07:24:24.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.960+0000 7fec94c00640 1 -- 172.21.15.86:0/3642436655 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7feca4005040 con 0x55b2a20c98a0 2024-04-17T07:24:24.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.960+0000 7fecb4813740 10 client.26935 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:24.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.960+0000 7fecb4813740 10 client.26935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:24:24.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.960+0000 7fecb4813740 20 client.26935 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:24.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.960+0000 7fecb4813740 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b2a2146fe0 0x55b2a2149400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:24.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.960+0000 7fecb4813740 4 client.26935 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:24:24.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.960+0000 7fecb4813740 1 -- 172.21.15.86:0/3642436655 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2a20268b0 con 0x55b2a2146fe0 2024-04-17T07:24:24.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.960+0000 7fecaaa00640 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b2a2146fe0 0x55b2a2149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.961+0000 7fecaaa00640 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b2a2146fe0 0x55b2a2149400 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e32ecb4d9d932812 server_cookie=90c1736b55fdbf9a in_seq=0 out_seq=0 2024-04-17T07:24:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.962+0000 7fec94c00640 10 client.26935 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.962+0000 7fec94c00640 1 -- 172.21.15.86:0/3642436655 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2a20268b0 con 0x55b2a2146fe0 2024-04-17T07:24:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.962+0000 7fec94c00640 10 client.26935 handle_command_reply: tid=0 2024-04-17T07:24:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.962+0000 7fecb4813740 1 client.26935 shutdown 2024-04-17T07:24:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.962+0000 7fecb4813740 1 -- 172.21.15.86:0/3642436655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20c98a0 msgr2=0x55b2a20c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.962+0000 7fecb4813740 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20c98a0 0x55b2a20c9d20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feca4006010 tx=0x7feca4004c80 comp rx=0 tx=0).stop 2024-04-17T07:24:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.963+0000 7fecb4813740 1 -- 172.21.15.86:0/3642436655 shutdown_connections 2024-04-17T07:24:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.963+0000 7fecb4813740 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55b2a2146fe0 0x55b2a2149400 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.963+0000 7fecb4813740 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b2a20c98a0 0x55b2a20c9d20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.963+0000 7fecb4813740 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2a20beb00 0x55b2a20c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.963+0000 7fecb4813740 1 --2- 172.21.15.86:0/3642436655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b2a20b8fa0 0x55b2a20c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.963+0000 7fecb4813740 1 -- 172.21.15.86:0/3642436655 >> 172.21.15.86:0/3642436655 conn(0x55b2a20b4510 msgr2=0x55b2a20b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.963+0000 7fec94c00640 0 client.26935 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.963+0000 7fecb4813740 1 -- 172.21.15.86:0/3642436655 shutdown_connections 2024-04-17T07:24:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.963+0000 7fecb4813740 1 -- 172.21.15.86:0/3642436655 wait complete. 2024-04-17T07:24:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.964+0000 7fecb4813740 20 client.26935 trim_cache size 0 max 16384 2024-04-17T07:24:24.964 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:24:24.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.968+0000 7fecb1c00640 1 -- 172.21.15.86:0/1804744361 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec900a3d00 msgr2=0x7fec900a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.968+0000 7fecb1c00640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec900a3d00 0x7fec900a61c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009230 tx=0x7fec9c02c680 comp rx=0 tx=0).stop 2024-04-17T07:24:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.968+0000 7fecb1c00640 1 -- 172.21.15.86:0/1804744361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecac1053b0 msgr2=0x7fecac19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.968+0000 7fecb1c00640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecac1053b0 0x7fecac19aa00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fec9800e9f0 tx=0x7fec9800edf0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.969+0000 7fecb1c00640 1 -- 172.21.15.86:0/1804744361 shutdown_connections 2024-04-17T07:24:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.969+0000 7fecb1c00640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fec900a3d00 0x7fec900a61c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.969+0000 7fecb1c00640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecac105d70 0x7fecac1a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.969+0000 7fecb1c00640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecac1053b0 0x7fecac19aa00 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.969+0000 7fecb1c00640 1 --2- 172.21.15.86:0/1804744361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fecac1041c0 0x7fecac19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.969+0000 7fecb1c00640 1 -- 172.21.15.86:0/1804744361 >> 172.21.15.86:0/1804744361 conn(0x7fecac0ff750 msgr2=0x7fecac1087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.969+0000 7fecb1c00640 1 -- 172.21.15.86:0/1804744361 shutdown_connections 2024-04-17T07:24:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:24.969+0000 7fecb1c00640 1 -- 172.21.15.86:0/1804744361 wait complete. 2024-04-17T07:24:25.300 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:25 smithi086 ceph-mon[31733]: pgmap v741: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 739 op/s 2024-04-17T07:24:25.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:25 smithi098 ceph-mon[35418]: pgmap v741: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 739 op/s 2024-04-17T07:24:25.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:25 smithi103 ceph-mon[35495]: pgmap v741: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.8 MiB/s wr, 739 op/s 2024-04-17T07:24:25.996 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:24:25.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:24:25.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:24:26.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.331+0000 7fae77800640 1 -- 172.21.15.86:0/2273787969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae701085c0 msgr2=0x7fae701089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.331+0000 7fae77800640 1 --2- 172.21.15.86:0/2273787969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae701085c0 0x7fae701089e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae64009900 tx=0x7fae6402bc30 comp rx=0 tx=0).stop 2024-04-17T07:24:26.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.332+0000 7fae77800640 1 -- 172.21.15.86:0/2273787969 shutdown_connections 2024-04-17T07:24:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.332+0000 7fae77800640 1 --2- 172.21.15.86:0/2273787969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae7010a170 0x7fae70110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.332+0000 7fae77800640 1 --2- 172.21.15.86:0/2273787969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae701097b0 0x7fae70109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.332+0000 7fae77800640 1 --2- 172.21.15.86:0/2273787969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae701085c0 0x7fae701089e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.332+0000 7fae77800640 1 -- 172.21.15.86:0/2273787969 >> 172.21.15.86:0/2273787969 conn(0x7fae70077560 msgr2=0x7fae700799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.332+0000 7fae77800640 1 -- 172.21.15.86:0/2273787969 shutdown_connections 2024-04-17T07:24:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.332+0000 7fae77800640 1 -- 172.21.15.86:0/2273787969 wait complete. 2024-04-17T07:24:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.333+0000 7fae77800640 1 Processor -- start 2024-04-17T07:24:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.334+0000 7fae77800640 1 -- start start 2024-04-17T07:24:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.334+0000 7fae77800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae701085c0 0x7fae70080820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.334+0000 7fae77800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae701097b0 0x7fae70080d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.334+0000 7fae77800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae7010a170 0x7fae70081da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.334+0000 7fae77800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae7007adc0 con 0x7fae701097b0 2024-04-17T07:24:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.334+0000 7fae77800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae7007ac40 con 0x7fae701085c0 2024-04-17T07:24:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.334+0000 7fae77800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae7007af40 con 0x7fae7010a170 2024-04-17T07:24:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.335+0000 7fae6fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae701097b0 0x7fae70080d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.335+0000 7fae74e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae701085c0 0x7fae70080820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.335+0000 7fae74e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae701085c0 0x7fae70080820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42582/0 (socket says 172.21.15.86:42582) 2024-04-17T07:24:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.335+0000 7fae74e00640 1 -- 172.21.15.86:0/1401489185 learned_addr learned my addr 172.21.15.86:0/1401489185 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.335+0000 7fae75800640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae7010a170 0x7fae70081da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.336+0000 7fae74e00640 1 -- 172.21.15.86:0/1401489185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae7010a170 msgr2=0x7fae70081da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.336+0000 7fae74e00640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae7010a170 0x7fae70081da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.336+0000 7fae74e00640 1 -- 172.21.15.86:0/1401489185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae701097b0 msgr2=0x7fae70080d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.336+0000 7fae74e00640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae701097b0 0x7fae70080d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.336+0000 7fae74e00640 1 -- 172.21.15.86:0/1401489185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae70082410 con 0x7fae701085c0 2024-04-17T07:24:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.336+0000 7fae6fe00640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae701097b0 0x7fae70080d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.336+0000 7fae74e00640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae701085c0 0x7fae70080820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fae64000c00 tx=0x7fae64004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.337+0000 7fae6d600640 1 -- 172.21.15.86:0/1401489185 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64043020 con 0x7fae701085c0 2024-04-17T07:24:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.337+0000 7fae6d600640 1 -- 172.21.15.86:0/1401489185 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae640052b0 con 0x7fae701085c0 2024-04-17T07:24:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.337+0000 7fae77800640 1 -- 172.21.15.86:0/1401489185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae700826a0 con 0x7fae701085c0 2024-04-17T07:24:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.337+0000 7fae6d600640 1 -- 172.21.15.86:0/1401489185 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64003c70 con 0x7fae701085c0 2024-04-17T07:24:26.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.337+0000 7fae77800640 1 -- 172.21.15.86:0/1401489185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae70082bf0 con 0x7fae701085c0 2024-04-17T07:24:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.344+0000 7fae7a57b740 1 Processor -- start 2024-04-17T07:24:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.346+0000 7fae7a57b740 1 -- start start 2024-04-17T07:24:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.346+0000 7fae6d600640 1 -- 172.21.15.86:0/1401489185 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fae64046020 con 0x7fae701085c0 2024-04-17T07:24:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.346+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.346+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba1d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.346+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 0x55f2fba1feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.346+0000 7fae7a57b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb97dcb0 con 0x55f2fba1da20 2024-04-17T07:24:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.346+0000 7fae7a57b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb924370 con 0x55f2fba188b0 2024-04-17T07:24:26.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.346+0000 7fae7a57b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb979c70 con 0x55f2fba17ec0 2024-04-17T07:24:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.347+0000 7fae75800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.347+0000 7fae75800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba182e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46244/0 (socket says 172.21.15.86:46244) 2024-04-17T07:24:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.347+0000 7fae75800640 1 -- 172.21.15.86:0/3908838357 learned_addr learned my addr 172.21.15.86:0/3908838357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:26.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.347+0000 7fae74e00640 1 --2- 172.21.15.86:0/3908838357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba1d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.348+0000 7fae75800640 1 -- 172.21.15.86:0/3908838357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 msgr2=0x55f2fba1d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.348+0000 7fae75800640 1 --2- 172.21.15.86:0/3908838357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba1d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.348+0000 7fae75800640 1 -- 172.21.15.86:0/3908838357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 msgr2=0x55f2fba1feb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:24:26.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.349+0000 7fae6d600640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae580a3c00 0x7fae580a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.349+0000 7fae6fe00640 1 --2- 172.21.15.86:0/3908838357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 0x55f2fba1feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.349+0000 7fae74e00640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae580a3c00 0x7fae580a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.349+0000 7fae75800640 1 --2- 172.21.15.86:0/3908838357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 0x55f2fba1feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.349+0000 7fae75800640 1 -- 172.21.15.86:0/3908838357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fb943910 con 0x55f2fba17ec0 2024-04-17T07:24:26.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.349+0000 7fae6fe00640 1 --2- 172.21.15.86:0/3908838357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 0x55f2fba1feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:24:26.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.350+0000 7fae6d600640 1 -- 172.21.15.86:0/1401489185 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fae640f2690 con 0x7fae701085c0 2024-04-17T07:24:26.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.350+0000 7fae74e00640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae580a3c00 0x7fae580a60c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fae640f0180 tx=0x7fae640c1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.350+0000 7fae75800640 1 --2- 172.21.15.86:0/3908838357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba182e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fae680039f0 tx=0x7fae68003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22db37f3c1006df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.351+0000 7fae62000640 1 -- 172.21.15.86:0/3908838357 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6801a470 con 0x55f2fba17ec0 2024-04-17T07:24:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.351+0000 7fae62000640 1 -- 172.21.15.86:0/3908838357 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae68015070 con 0x55f2fba17ec0 2024-04-17T07:24:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.351+0000 7fae62000640 1 -- 172.21.15.86:0/3908838357 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6801a470 con 0x55f2fba17ec0 2024-04-17T07:24:26.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.353+0000 7fae7a57b740 1 -- 172.21.15.86:0/3908838357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 msgr2=0x55f2fba182e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.353+0000 7fae7a57b740 1 --2- 172.21.15.86:0/3908838357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba182e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fae680039f0 tx=0x7fae68003eb0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.353+0000 7fae7a57b740 1 -- 172.21.15.86:0/3908838357 shutdown_connections 2024-04-17T07:24:26.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.353+0000 7fae7a57b740 1 --2- 172.21.15.86:0/3908838357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 0x55f2fba1feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.353+0000 7fae7a57b740 1 --2- 172.21.15.86:0/3908838357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba1d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.353+0000 7fae7a57b740 1 --2- 172.21.15.86:0/3908838357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba182e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.353+0000 7fae7a57b740 1 -- 172.21.15.86:0/3908838357 >> 172.21.15.86:0/3908838357 conn(0x55f2fba13430 msgr2=0x55f2fba13830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.354+0000 7fae7a57b740 1 -- 172.21.15.86:0/3908838357 shutdown_connections 2024-04-17T07:24:26.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.354+0000 7fae7a57b740 1 -- 172.21.15.86:0/3908838357 wait complete. 2024-04-17T07:24:26.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.354+0000 7fae7a57b740 1 -- start start 2024-04-17T07:24:26.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.355+0000 7fae7a57b740 1 Processor -- start 2024-04-17T07:24:26.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.356+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.356+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba22980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.356+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 0x55f2fba299c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.356+0000 7fae7a57b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2fba209d0 con 0x55f2fba1da20 2024-04-17T07:24:26.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.356+0000 7fae7a57b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f2fb937730 con 0x55f2fba188b0 2024-04-17T07:24:26.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.356+0000 7fae7a57b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2fba20b50 con 0x55f2fba17ec0 2024-04-17T07:24:26.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.356+0000 7fae7a57b740 10 client.0 _ll_register_callbacks cb 0x55f2fb939b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:26.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.356+0000 7fae75800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.356+0000 7fae75800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46250/0 (socket says 172.21.15.86:46250) 2024-04-17T07:24:26.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.356+0000 7fae75800640 1 -- 172.21.15.86:0/1617074614 learned_addr learned my addr 172.21.15.86:0/1617074614 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:26.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.357+0000 7fae75800640 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 0x55f2fba299c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.357+0000 7fae6fe00640 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba22980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.357+0000 7fae75800640 1 -- 172.21.15.86:0/1617074614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 msgr2=0x55f2fba22980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.357+0000 7fae75800640 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba22980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.357+0000 7fae75800640 1 -- 172.21.15.86:0/1617074614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 msgr2=0x55f2fba299c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.357+0000 7fae75800640 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 0x55f2fba299c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.357+0000 7fae75800640 1 -- 172.21.15.86:0/1617074614 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fb7587e0 con 0x55f2fba17ec0 2024-04-17T07:24:26.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.358+0000 7fae6fe00640 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba22980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:24:26.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.358+0000 7fae75800640 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae680049d0 tx=0x7fae68005a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be46be1633bb465f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.358+0000 7fae56a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:26.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.359+0000 7fae56a00640 1 -- 172.21.15.86:0/1617074614 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6802d060 con 0x55f2fba17ec0 2024-04-17T07:24:26.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.359+0000 7fae56a00640 1 -- 172.21.15.86:0/1617074614 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fae68010740 con 0x55f2fba17ec0 2024-04-17T07:24:26.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.359+0000 7fae7a57b740 1 -- 172.21.15.86:0/1617074614 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae68015070 con 0x55f2fba17ec0 2024-04-17T07:24:26.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.360+0000 7fae56a00640 1 -- 172.21.15.86:0/1617074614 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae68010940 con 0x55f2fba17ec0 2024-04-17T07:24:26.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.360+0000 7fae7a57b740 10 client.26947 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:26.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.360+0000 7fae7a57b740 1 -- 172.21.15.86:0/1617074614 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2fb7b8f70 con 0x55f2fba17ec0 2024-04-17T07:24:26.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.362+0000 7fae56a00640 1 -- 172.21.15.86:0/1617074614 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae68015070 con 0x55f2fba17ec0 2024-04-17T07:24:26.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.363+0000 7fae56a00640 1 -- 172.21.15.86:0/1617074614 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fae6801aae0 con 0x55f2fba17ec0 2024-04-17T07:24:26.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.363+0000 7fae7a57b740 10 client.26947 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:26.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.363+0000 7fae7a57b740 10 client.26947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:24:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.364+0000 7fae7a57b740 20 client.26947 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.364+0000 7fae7a57b740 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f2fba2de20 0x55f2fba30210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.364+0000 7fae7a57b740 4 client.26947 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.364+0000 7fae7a57b740 1 -- 172.21.15.86:0/1617074614 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2fb98b6d0 con 0x55f2fba2de20 2024-04-17T07:24:26.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.364+0000 7fae6fe00640 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f2fba2de20 0x55f2fba30210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.366+0000 7fae6fe00640 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f2fba2de20 0x55f2fba30210 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a90d6f8d3dfa205a server_cookie=bf433bb1b0fe28d5 in_seq=0 out_seq=0 2024-04-17T07:24:26.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.366+0000 7fae56a00640 10 client.26947 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:26.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.370+0000 7fae56a00640 1 -- 172.21.15.86:0/1617074614 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2fb98b6d0 con 0x55f2fba2de20 2024-04-17T07:24:26.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.370+0000 7fae56a00640 10 client.26947 handle_command_reply: tid=0 2024-04-17T07:24:26.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.370+0000 7fae7a57b740 1 client.26947 shutdown 2024-04-17T07:24:26.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.370+0000 7fae7a57b740 1 -- 172.21.15.86:0/1617074614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 msgr2=0x55f2fba22440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.370+0000 7fae7a57b740 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22440 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae680049d0 tx=0x7fae68005a50 comp rx=0 tx=0).stop 2024-04-17T07:24:26.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.371+0000 7fae7a57b740 1 -- 172.21.15.86:0/1617074614 shutdown_connections 2024-04-17T07:24:26.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.371+0000 7fae7a57b740 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f2fba2de20 0x55f2fba30210 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.371+0000 7fae7a57b740 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba1da20 0x55f2fba299c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.371+0000 7fae7a57b740 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba22980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.371+0000 7fae7a57b740 1 --2- 172.21.15.86:0/1617074614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22440 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.371+0000 7fae7a57b740 1 -- 172.21.15.86:0/1617074614 >> 172.21.15.86:0/1617074614 conn(0x55f2fba13430 msgr2=0x55f2fba125e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.371+0000 7fae56a00640 0 client.26947 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:26.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.372+0000 7fae7a57b740 1 -- 172.21.15.86:0/1617074614 shutdown_connections 2024-04-17T07:24:26.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.372+0000 7fae7a57b740 1 -- 172.21.15.86:0/1617074614 wait complete. 2024-04-17T07:24:26.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.372+0000 7fae7a57b740 20 client.26947 trim_cache size 0 max 16384 2024-04-17T07:24:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.412+0000 7fae7a57b740 1 Processor -- start 2024-04-17T07:24:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.413+0000 7fae7a57b740 1 -- start start 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.413+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.413+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba188b0 0x55f2fba29950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.413+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba1da20 0x55f2fba22590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.413+0000 7fae7a57b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae701b9910 con 0x55f2fba188b0 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.413+0000 7fae7a57b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae701bb450 con 0x55f2fba1da20 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.413+0000 7fae7a57b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae701b9fe0 con 0x55f2fba17ec0 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.414+0000 7fae75800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.414+0000 7fae75800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:46258/0 (socket says 172.21.15.86:46258) 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.414+0000 7fae75800640 1 -- 172.21.15.86:0/130479483 learned_addr learned my addr 172.21.15.86:0/130479483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.414+0000 7fae6fe00640 1 --2- 172.21.15.86:0/130479483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba188b0 0x55f2fba29950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.414+0000 7fae75800640 1 --2- 172.21.15.86:0/130479483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba1da20 0x55f2fba22590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.415+0000 7fae75800640 1 -- 172.21.15.86:0/130479483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba1da20 msgr2=0x55f2fba22590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.415+0000 7fae75800640 1 --2- 172.21.15.86:0/130479483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba1da20 0x55f2fba22590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.415+0000 7fae75800640 1 -- 172.21.15.86:0/130479483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba188b0 msgr2=0x55f2fba29950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.415+0000 7fae75800640 1 --2- 172.21.15.86:0/130479483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba188b0 0x55f2fba29950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.415+0000 7fae75800640 1 -- 172.21.15.86:0/130479483 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fb650170 con 0x55f2fba17ec0 2024-04-17T07:24:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.416+0000 7fae75800640 1 --2- 172.21.15.86:0/130479483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae68009720 tx=0x7fae68013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=296fbfd3a9775b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.417+0000 7fae61600640 1 -- 172.21.15.86:0/130479483 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6802d040 con 0x55f2fba17ec0 2024-04-17T07:24:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.417+0000 7fae61600640 1 -- 172.21.15.86:0/130479483 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae68015070 con 0x55f2fba17ec0 2024-04-17T07:24:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.417+0000 7fae61600640 1 -- 172.21.15.86:0/130479483 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6802d040 con 0x55f2fba17ec0 2024-04-17T07:24:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.418+0000 7fae7a57b740 1 -- 172.21.15.86:0/130479483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 msgr2=0x55f2fba22050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.418+0000 7fae7a57b740 1 --2- 172.21.15.86:0/130479483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae68009720 tx=0x7fae68013040 comp rx=0 tx=0).stop 2024-04-17T07:24:26.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.420+0000 7fae7a57b740 1 -- 172.21.15.86:0/130479483 shutdown_connections 2024-04-17T07:24:26.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.420+0000 7fae7a57b740 1 --2- 172.21.15.86:0/130479483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba1da20 0x55f2fba22590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.420+0000 7fae7a57b740 1 --2- 172.21.15.86:0/130479483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba188b0 0x55f2fba29950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.420+0000 7fae7a57b740 1 --2- 172.21.15.86:0/130479483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba17ec0 0x55f2fba22050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.420+0000 7fae7a57b740 1 -- 172.21.15.86:0/130479483 >> 172.21.15.86:0/130479483 conn(0x55f2fba13430 msgr2=0x55f2fba19df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.421+0000 7fae7a57b740 1 -- 172.21.15.86:0/130479483 shutdown_connections 2024-04-17T07:24:26.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.422+0000 7fae7a57b740 1 -- 172.21.15.86:0/130479483 wait complete. 2024-04-17T07:24:26.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.422+0000 7fae7a57b740 1 -- start start 2024-04-17T07:24:26.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.423+0000 7fae7a57b740 1 Processor -- start 2024-04-17T07:24:26.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.423+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba17ec0 0x55f2fba26f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.424+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba27450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.424+0000 7fae7a57b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba1da20 0x55f2fba28460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.424+0000 7fae7a57b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae701bad80 con 0x55f2fba17ec0 2024-04-17T07:24:26.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.424+0000 7fae7a57b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae701ba6b0 con 0x55f2fba188b0 2024-04-17T07:24:26.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.424+0000 7fae7a57b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae701b78e0 con 0x55f2fba1da20 2024-04-17T07:24:26.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.424+0000 7fae7a57b740 10 client.0 _ll_register_callbacks cb 0x7fae68009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.424+0000 7fae75800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba17ec0 0x55f2fba26f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.424+0000 7fae6fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba27450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.424+0000 7fae6fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba27450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:42620/0 (socket says 172.21.15.86:42620) 2024-04-17T07:24:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.424+0000 7fae6fe00640 1 -- 172.21.15.86:0/2654296530 learned_addr learned my addr 172.21.15.86:0/2654296530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.425+0000 7fae6fe00640 1 -- 172.21.15.86:0/2654296530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba1da20 msgr2=0x55f2fba28460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.425+0000 7fae6fe00640 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba1da20 0x55f2fba28460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.425+0000 7fae6fe00640 1 -- 172.21.15.86:0/2654296530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba17ec0 msgr2=0x55f2fba26f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.425+0000 7fae6fe00640 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba17ec0 0x55f2fba26f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.425+0000 7fae6fe00640 1 -- 172.21.15.86:0/2654296530 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fba28a50 con 0x55f2fba188b0 2024-04-17T07:24:26.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.425+0000 7fae75800640 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba17ec0 0x55f2fba26f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:24:26.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.426+0000 7fae6fe00640 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba27450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae70081d30 tx=0x7fae5c013910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c57906d0037bc32c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.426+0000 7fae56a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:24:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.426+0000 7fae56a00640 1 -- 172.21.15.86:0/2654296530 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c004030 con 0x55f2fba188b0 2024-04-17T07:24:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.427+0000 7fae56a00640 1 -- 172.21.15.86:0/2654296530 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fae5c0041d0 con 0x55f2fba188b0 2024-04-17T07:24:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.427+0000 7fae7a57b740 1 -- 172.21.15.86:0/2654296530 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae68015070 con 0x55f2fba188b0 2024-04-17T07:24:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.427+0000 7fae56a00640 1 -- 172.21.15.86:0/2654296530 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae5c02f050 con 0x55f2fba188b0 2024-04-17T07:24:26.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.427+0000 7fae7a57b740 10 client.36911 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:26.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.427+0000 7fae7a57b740 1 -- 172.21.15.86:0/2654296530 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2fba94260 con 0x55f2fba188b0 2024-04-17T07:24:26.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.428+0000 7fae56a00640 1 -- 172.21.15.86:0/2654296530 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae68015070 con 0x55f2fba188b0 2024-04-17T07:24:26.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.428+0000 7fae56a00640 1 -- 172.21.15.86:0/2654296530 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fae5c010510 con 0x55f2fba188b0 2024-04-17T07:24:26.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.429+0000 7fae7a57b740 10 client.36911 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:26.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.429+0000 7fae7a57b740 10 client.36911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:24:26.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.429+0000 7fae7a57b740 20 client.36911 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:26.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.429+0000 7fae7a57b740 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f2fba94ad0 0x55f2fba96ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.429+0000 7fae7a57b740 4 client.36911 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:24:26.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.429+0000 7fae7a57b740 1 -- 172.21.15.86:0/2654296530 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f2fb985540 con 0x55f2fba94ad0 2024-04-17T07:24:26.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.430+0000 7fae75800640 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f2fba94ad0 0x55f2fba96ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.431+0000 7fae75800640 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f2fba94ad0 0x55f2fba96ef0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca0477beaebdc1b server_cookie=cbbac66ef78b8a8d in_seq=0 out_seq=0 2024-04-17T07:24:26.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.431+0000 7fae56a00640 10 client.36911 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.435+0000 7fae56a00640 1 -- 172.21.15.86:0/2654296530 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f2fb985540 con 0x55f2fba94ad0 2024-04-17T07:24:26.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.435+0000 7fae56a00640 10 client.36911 handle_command_reply: tid=0 2024-04-17T07:24:26.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.436+0000 7fae7a57b740 1 client.36911 shutdown 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.436+0000 7fae7a57b740 1 -- 172.21.15.86:0/2654296530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 msgr2=0x55f2fba27450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.436+0000 7fae7a57b740 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba27450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae70081d30 tx=0x7fae5c013910 comp rx=0 tx=0).stop 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.436+0000 7fae7a57b740 1 -- 172.21.15.86:0/2654296530 shutdown_connections 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.436+0000 7fae7a57b740 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f2fba94ad0 0x55f2fba96ef0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.437+0000 7fae7a57b740 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2fba1da20 0x55f2fba28460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.437+0000 7fae7a57b740 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2fba188b0 0x55f2fba27450 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.437+0000 7fae7a57b740 1 --2- 172.21.15.86:0/2654296530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2fba17ec0 0x55f2fba26f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.437+0000 7fae7a57b740 1 -- 172.21.15.86:0/2654296530 >> 172.21.15.86:0/2654296530 conn(0x55f2fba13430 msgr2=0x55f2fba21fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.437+0000 7fae56a00640 0 client.36911 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.437+0000 7fae7a57b740 1 -- 172.21.15.86:0/2654296530 shutdown_connections 2024-04-17T07:24:26.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.437+0000 7fae7a57b740 1 -- 172.21.15.86:0/2654296530 wait complete. 2024-04-17T07:24:26.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.438+0000 7fae7a57b740 20 client.36911 trim_cache size 0 max 16384 2024-04-17T07:24:26.439 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:24:26.439 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:24:26.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "d3203a97-ee8b-467e-8260-2cd27c4ca4f1", 2024-04-17T07:24:26.439 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:24:26.439 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:24:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.440+0000 7fae77800640 1 -- 172.21.15.86:0/1401489185 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae580a3c00 msgr2=0x7fae580a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.440+0000 7fae77800640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae580a3c00 0x7fae580a60c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fae640f0180 tx=0x7fae640c1040 comp rx=0 tx=0).stop 2024-04-17T07:24:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.440+0000 7fae77800640 1 -- 172.21.15.86:0/1401489185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae701085c0 msgr2=0x7fae70080820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.440+0000 7fae77800640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae701085c0 0x7fae70080820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fae64000c00 tx=0x7fae64004270 comp rx=0 tx=0).stop 2024-04-17T07:24:26.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.440+0000 7fae77800640 1 -- 172.21.15.86:0/1401489185 shutdown_connections 2024-04-17T07:24:26.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.440+0000 7fae77800640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fae580a3c00 0x7fae580a60c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.440+0000 7fae77800640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae7010a170 0x7fae70081da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.440+0000 7fae77800640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae701097b0 0x7fae70080d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.440+0000 7fae77800640 1 --2- 172.21.15.86:0/1401489185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae701085c0 0x7fae70080820 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.440+0000 7fae77800640 1 -- 172.21.15.86:0/1401489185 >> 172.21.15.86:0/1401489185 conn(0x7fae70077560 msgr2=0x7fae7010cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.441+0000 7fae77800640 1 -- 172.21.15.86:0/1401489185 shutdown_connections 2024-04-17T07:24:26.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.441+0000 7fae77800640 1 -- 172.21.15.86:0/1401489185 wait complete. 2024-04-17T07:24:26.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:24:26.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:24:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.691+0000 7f7ecb800640 1 -- 172.21.15.86:0/2506723659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec41041e0 msgr2=0x7f7ec4104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.691+0000 7f7ecb800640 1 --2- 172.21.15.86:0/2506723659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec41041e0 0x7f7ec4104600 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009ca0 tx=0x7f7ebc02c100 comp rx=0 tx=0).stop 2024-04-17T07:24:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.692+0000 7f7ecb800640 1 -- 172.21.15.86:0/2506723659 shutdown_connections 2024-04-17T07:24:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.692+0000 7f7ecb800640 1 --2- 172.21.15.86:0/2506723659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec4105d90 0x7f7ec410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.692+0000 7f7ecb800640 1 --2- 172.21.15.86:0/2506723659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec41053d0 0x7f7ec4105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.692+0000 7f7ecb800640 1 --2- 172.21.15.86:0/2506723659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec41041e0 0x7f7ec4104600 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.692+0000 7f7ecb800640 1 -- 172.21.15.86:0/2506723659 >> 172.21.15.86:0/2506723659 conn(0x7f7ec40ff750 msgr2=0x7f7ec4101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.692+0000 7f7ecb800640 1 -- 172.21.15.86:0/2506723659 shutdown_connections 2024-04-17T07:24:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.692+0000 7f7ecb800640 1 -- 172.21.15.86:0/2506723659 wait complete. 2024-04-17T07:24:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ecb800640 1 Processor -- start 2024-04-17T07:24:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ecb800640 1 -- start start 2024-04-17T07:24:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ecb800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec41041e0 0x7f7ec419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ecb800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec41053d0 0x7f7ec419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ecb800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec4105d90 0x7f7ec41a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ecb800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec410fd20 con 0x7f7ec41053d0 2024-04-17T07:24:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ecb800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec410fba0 con 0x7f7ec4105d90 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ecb800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec410fea0 con 0x7f7ec41041e0 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec41053d0 0x7f7ec419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec41053d0 0x7f7ec419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38206/0 (socket says 172.21.15.86:38206) 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.693+0000 7f7ec3e00640 1 -- 172.21.15.86:0/1028592421 learned_addr learned my addr 172.21.15.86:0/1028592421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec41041e0 0x7f7ec419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec9800640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec4105d90 0x7f7ec41a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec3e00640 1 -- 172.21.15.86:0/1028592421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec41041e0 msgr2=0x7f7ec419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec41041e0 0x7f7ec419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec3e00640 1 -- 172.21.15.86:0/1028592421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec4105d90 msgr2=0x7f7ec41a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec4105d90 0x7f7ec41a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec3e00640 1 -- 172.21.15.86:0/1028592421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec41a1ff0 con 0x7f7ec41053d0 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec41053d0 0x7f7ec419a910 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00c970 tx=0x7f7eac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec1600640 1 -- 172.21.15.86:0/1028592421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac007bb0 con 0x7f7ec41053d0 2024-04-17T07:24:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec1600640 1 -- 172.21.15.86:0/1028592421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eac007d50 con 0x7f7ec41053d0 2024-04-17T07:24:26.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ecb800640 1 -- 172.21.15.86:0/1028592421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec41a22e0 con 0x7f7ec41053d0 2024-04-17T07:24:26.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.694+0000 7f7ec1600640 1 -- 172.21.15.86:0/1028592421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00f450 con 0x7f7ec41053d0 2024-04-17T07:24:26.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.695+0000 7f7ecb800640 1 -- 172.21.15.86:0/1028592421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec41a27e0 con 0x7f7ec41053d0 2024-04-17T07:24:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.697+0000 7f7ece5e6740 1 Processor -- start 2024-04-17T07:24:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ec1600640 1 -- 172.21.15.86:0/1028592421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7eac004f20 con 0x7f7ec41053d0 2024-04-17T07:24:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ece5e6740 1 -- start start 2024-04-17T07:24:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400a190 0x55e26400a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400ab80 0x55e26400f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400fcf0 0x55e264012180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e263f71120 con 0x55e26400fcf0 2024-04-17T07:24:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e263f17340 con 0x55e26400a190 2024-04-17T07:24:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e263f6d0e0 con 0x55e26400ab80 2024-04-17T07:24:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400fcf0 0x55e264012180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400fcf0 0x55e264012180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38222/0 (socket says 172.21.15.86:38222) 2024-04-17T07:24:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ec3e00640 1 -- 172.21.15.86:0/2975616088 learned_addr learned my addr 172.21.15.86:0/2975616088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/2975616088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400ab80 0x55e26400f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.698+0000 7f7ec9800640 1 --2- 172.21.15.86:0/2975616088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400a190 0x55e26400a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.699+0000 7f7ec3e00640 1 -- 172.21.15.86:0/2975616088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400ab80 msgr2=0x55e26400f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.699+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/2975616088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400ab80 0x55e26400f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.699+0000 7f7ec3e00640 1 -- 172.21.15.86:0/2975616088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400a190 msgr2=0x55e26400a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.699+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/2975616088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400a190 0x55e26400a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.699+0000 7f7ec3e00640 1 -- 172.21.15.86:0/2975616088 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e263d4d7d0 con 0x55e26400fcf0 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.699+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/2975616088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400fcf0 0x55e264012180 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0d6660 tx=0x7f7eac0d6b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca7d9604cc1a6da server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.699+0000 7f7eba000640 1 -- 172.21.15.86:0/2975616088 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0e3070 con 0x55e26400fcf0 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.699+0000 7f7ec1600640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7ea80a3bb0 0x7f7ea80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.699+0000 7f7eba000640 1 -- 172.21.15.86:0/2975616088 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eac0a0810 con 0x55e26400fcf0 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.699+0000 7f7eba000640 1 -- 172.21.15.86:0/2975616088 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac09d9c0 con 0x55e26400fcf0 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ec1600640 1 -- 172.21.15.86:0/1028592421 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7eac0d0560 con 0x7f7ec41053d0 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ec9800640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7ea80a3bb0 0x7f7ea80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2975616088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400fcf0 msgr2=0x55e264012180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ec9800640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7ea80a3bb0 0x7f7ea80a6070 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55e26400b3e0 tx=0x7f7eb400c430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/2975616088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400fcf0 0x55e264012180 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0d6660 tx=0x7f7eac0d6b20 comp rx=0 tx=0).stop 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2975616088 shutdown_connections 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/2975616088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400fcf0 0x55e264012180 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/2975616088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400ab80 0x55e26400f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/2975616088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400a190 0x55e26400a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2975616088 >> 172.21.15.86:0/2975616088 conn(0x55e264005700 msgr2=0x55e264005b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2975616088 shutdown_connections 2024-04-17T07:24:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.700+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2975616088 wait complete. 2024-04-17T07:24:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.701+0000 7f7ece5e6740 1 -- start start 2024-04-17T07:24:26.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.701+0000 7f7ece5e6740 1 Processor -- start 2024-04-17T07:24:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.701+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400a190 0x55e264014620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.701+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400ab80 0x55e264014b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.701+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400fcf0 0x55e26401bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.701+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e264012ca0 con 0x55e26400a190 2024-04-17T07:24:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.701+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e263f2a710 con 0x55e26400ab80 2024-04-17T07:24:26.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.701+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e264012e20 con 0x55e26400fcf0 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ece5e6740 10 client.0 _ll_register_callbacks cb 0x55e263f25af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ec8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400a190 0x55e264014620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ec8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400a190 0x55e264014620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38232/0 (socket says 172.21.15.86:38232) 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ec8e00640 1 -- 172.21.15.86:0/158045270 learned_addr learned my addr 172.21.15.86:0/158045270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400ab80 0x55e264014b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ec8e00640 1 -- 172.21.15.86:0/158045270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400fcf0 msgr2=0x55e26401bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400fcf0 0x55e26401bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ec8e00640 1 -- 172.21.15.86:0/158045270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400ab80 msgr2=0x55e264014b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400ab80 0x55e264014b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ec8e00640 1 -- 172.21.15.86:0/158045270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e263c833f0 con 0x55e26400a190 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400a190 0x55e264014620 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4105230 tx=0x7f7ebc0325c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5889e97baa595343 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.702+0000 7f7eb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7eb2a00640 1 -- 172.21.15.86:0/158045270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc044070 con 0x55e26400a190 2024-04-17T07:24:26.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7eb2a00640 1 -- 172.21.15.86:0/158045270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7ebc003b70 con 0x55e26400a190 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7ece5e6740 1 -- 172.21.15.86:0/158045270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eac0a0810 con 0x55e26400a190 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7eb2a00640 1 -- 172.21.15.86:0/158045270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ebc051050 con 0x55e26400a190 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7ece5e6740 10 client.18021 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7ece5e6740 1 -- 172.21.15.86:0/158045270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e263c430f0 con 0x55e26400a190 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7eb2a00640 1 -- 172.21.15.86:0/158045270 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eac0a0810 con 0x55e26400a190 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7eb2a00640 1 -- 172.21.15.86:0/158045270 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7ebc034560 con 0x55e26400a190 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7ece5e6740 10 client.18021 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7ece5e6740 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7ece5e6740 20 client.18021 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e2640204f0 0x55e2640229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7ece5e6740 4 client.18021 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:26.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.703+0000 7f7ece5e6740 1 -- 172.21.15.86:0/158045270 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e263f7eb40 con 0x55e2640204f0 2024-04-17T07:24:26.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.834+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e2640204f0 0x55e2640229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.839+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e2640204f0 0x55e2640229b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86696683c9ef41ed server_cookie=a584954b30bb3fa6 in_seq=0 out_seq=0 2024-04-17T07:24:26.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.839+0000 7f7eb2a00640 10 client.18021 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:26.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.851+0000 7f7eb2a00640 1 -- 172.21.15.86:0/158045270 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e263f7eb40 con 0x55e2640204f0 2024-04-17T07:24:26.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.851+0000 7f7eb2a00640 10 client.18021 handle_command_reply: tid=0 2024-04-17T07:24:26.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.851+0000 7f7ece5e6740 1 client.18021 shutdown 2024-04-17T07:24:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7ece5e6740 1 -- 172.21.15.86:0/158045270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400a190 msgr2=0x55e264014620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400a190 0x55e264014620 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4105230 tx=0x7f7ebc0325c0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7ece5e6740 1 -- 172.21.15.86:0/158045270 shutdown_connections 2024-04-17T07:24:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e2640204f0 0x55e2640229b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400fcf0 0x55e26401bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400ab80 0x55e264014b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/158045270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400a190 0x55e264014620 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7ece5e6740 1 -- 172.21.15.86:0/158045270 >> 172.21.15.86:0/158045270 conn(0x55e264005700 msgr2=0x55e264004890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7eb2a00640 0 client.18021 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7ece5e6740 1 -- 172.21.15.86:0/158045270 shutdown_connections 2024-04-17T07:24:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.853+0000 7f7ece5e6740 1 -- 172.21.15.86:0/158045270 wait complete. 2024-04-17T07:24:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.854+0000 7f7ece5e6740 20 client.18021 trim_cache size 0 max 16384 2024-04-17T07:24:26.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.899+0000 7f7ece5e6740 1 Processor -- start 2024-04-17T07:24:26.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.899+0000 7f7ece5e6740 1 -- start start 2024-04-17T07:24:26.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.900+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264004710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.900+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 0x55e26401bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.900+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400fcf0 0x55e264013c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.900+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41acc30 con 0x55e26400ab80 2024-04-17T07:24:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.900+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 0x55e26401bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.900+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 0x55e26401bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52442/0 (socket says 172.21.15.86:52442) 2024-04-17T07:24:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.900+0000 7f7ec3e00640 1 -- 172.21.15.86:0/2870931880 learned_addr learned my addr 172.21.15.86:0/2870931880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.900+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41ae770 con 0x55e26400fcf0 2024-04-17T07:24:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.901+0000 7f7ec9800640 1 --2- 172.21.15.86:0/2870931880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400fcf0 0x55e264013c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.901+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2870931880 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41ad300 con 0x55e26400a190 2024-04-17T07:24:26.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.901+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/2870931880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264004710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.903+0000 7f7ec8e00640 1 -- 172.21.15.86:0/2870931880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400fcf0 msgr2=0x55e264013c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.903+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/2870931880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400fcf0 0x55e264013c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.903+0000 7f7ec8e00640 1 -- 172.21.15.86:0/2870931880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 msgr2=0x55e26401bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.903+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/2870931880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 0x55e26401bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.903+0000 7f7ec8e00640 1 -- 172.21.15.86:0/2870931880 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e264006170 con 0x55e26400a190 2024-04-17T07:24:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.904+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/2870931880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264004710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc02c640 tx=0x7f7ebc009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29adda00030485b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.904+0000 7f7eb9600640 1 -- 172.21.15.86:0/2870931880 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc044070 con 0x55e26400a190 2024-04-17T07:24:26.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.905+0000 7f7eb9600640 1 -- 172.21.15.86:0/2870931880 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ebc040430 con 0x55e26400a190 2024-04-17T07:24:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.906+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2870931880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 msgr2=0x55e264004710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.906+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/2870931880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264004710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc02c640 tx=0x7f7ebc009800 comp rx=0 tx=0).stop 2024-04-17T07:24:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.906+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2870931880 shutdown_connections 2024-04-17T07:24:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.906+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/2870931880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400fcf0 0x55e264013c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.906+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/2870931880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 0x55e26401bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.906+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/2870931880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264004710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.906+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2870931880 >> 172.21.15.86:0/2870931880 conn(0x55e264005700 msgr2=0x55e264004000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.906+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2870931880 shutdown_connections 2024-04-17T07:24:26.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.907+0000 7f7ece5e6740 1 -- 172.21.15.86:0/2870931880 wait complete. 2024-04-17T07:24:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.908+0000 7f7ece5e6740 1 -- start start 2024-04-17T07:24:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.908+0000 7f7ece5e6740 1 Processor -- start 2024-04-17T07:24:26.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.910+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264018ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.910+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 0x55e264019020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.910+0000 7f7ece5e6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400fcf0 0x55e26408d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.911+0000 7f7ec8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264018ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.911+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 0x55e264019020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.911+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41ae0a0 con 0x55e26400ab80 2024-04-17T07:24:26.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.911+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41ad9d0 con 0x55e26400fcf0 2024-04-17T07:24:26.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.911+0000 7f7ece5e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41aac00 con 0x55e26400a190 2024-04-17T07:24:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.911+0000 7f7ec8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264018ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:56844/0 (socket says 172.21.15.86:56844) 2024-04-17T07:24:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.911+0000 7f7ece5e6740 10 client.0 _ll_register_callbacks cb 0x7f7ebc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.911+0000 7f7ec8e00640 1 -- 172.21.15.86:0/1386599492 learned_addr learned my addr 172.21.15.86:0/1386599492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.912+0000 7f7ec9800640 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400fcf0 0x55e26408d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.912+0000 7f7ec8e00640 1 -- 172.21.15.86:0/1386599492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400fcf0 msgr2=0x55e26408d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.912+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400fcf0 0x55e26408d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.912+0000 7f7ec8e00640 1 -- 172.21.15.86:0/1386599492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 msgr2=0x55e264019020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.912+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 0x55e264019020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.912+0000 7f7ec8e00640 1 -- 172.21.15.86:0/1386599492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26408d8e0 con 0x55e26400a190 2024-04-17T07:24:26.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.913+0000 7f7ec8e00640 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264018ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc002230 tx=0x7f7ebc034e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e27fbdfed743866 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:26.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.913+0000 7f7eb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:26.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.913+0000 7f7eb2a00640 1 -- 172.21.15.86:0/1386599492 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc044040 con 0x55e26400a190 2024-04-17T07:24:26.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.913+0000 7f7eb2a00640 1 -- 172.21.15.86:0/1386599492 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7ebc0405e0 con 0x55e26400a190 2024-04-17T07:24:26.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.914+0000 7f7ece5e6740 1 -- 172.21.15.86:0/1386599492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ebc040430 con 0x55e26400a190 2024-04-17T07:24:26.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.915+0000 7f7eb2a00640 1 -- 172.21.15.86:0/1386599492 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ebc004b40 con 0x55e26400a190 2024-04-17T07:24:26.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.915+0000 7f7ece5e6740 10 client.26962 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:26.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.915+0000 7f7ece5e6740 1 -- 172.21.15.86:0/1386599492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e26408ebf0 con 0x55e26400a190 2024-04-17T07:24:26.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.915+0000 7f7eb2a00640 1 -- 172.21.15.86:0/1386599492 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ebc040430 con 0x55e26400a190 2024-04-17T07:24:26.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.915+0000 7f7eb2a00640 1 -- 172.21.15.86:0/1386599492 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7ebc048b70 con 0x55e26400a190 2024-04-17T07:24:26.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.916+0000 7f7ece5e6740 10 client.26962 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:26.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.916+0000 7f7ece5e6740 10 client.26962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:24:26.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.916+0000 7f7ece5e6740 20 client.26962 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:26.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.916+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e26408f540 0x55e264091960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:26.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.916+0000 7f7ece5e6740 4 client.26962 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:24:26.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.916+0000 7f7ece5e6740 1 -- 172.21.15.86:0/1386599492 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e263f789b0 con 0x55e26408f540 2024-04-17T07:24:26.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.917+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e26408f540 0x55e264091960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:26.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.919+0000 7f7ec3e00640 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e26408f540 0x55e264091960 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c3f508f70891362 server_cookie=fa3d96fbda146baa in_seq=0 out_seq=0 2024-04-17T07:24:26.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.919+0000 7f7eb2a00640 10 client.26962 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:26.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.920+0000 7f7eb2a00640 1 -- 172.21.15.86:0/1386599492 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55e263f789b0 con 0x55e26408f540 2024-04-17T07:24:26.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.920+0000 7f7eb2a00640 10 client.26962 handle_command_reply: tid=0 2024-04-17T07:24:26.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.920+0000 7f7ece5e6740 1 client.26962 shutdown 2024-04-17T07:24:26.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.921+0000 7f7ece5e6740 1 -- 172.21.15.86:0/1386599492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 msgr2=0x55e264018ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.921+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264018ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc002230 tx=0x7f7ebc034e20 comp rx=0 tx=0).stop 2024-04-17T07:24:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.921+0000 7f7ece5e6740 1 -- 172.21.15.86:0/1386599492 shutdown_connections 2024-04-17T07:24:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.921+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e26408f540 0x55e264091960 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.921+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e26400fcf0 0x55e26408d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.921+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e26400ab80 0x55e264019020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.921+0000 7f7ece5e6740 1 --2- 172.21.15.86:0/1386599492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e26400a190 0x55e264018ae0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.921+0000 7f7eb2a00640 0 client.26962 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:26.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.921+0000 7f7ece5e6740 1 -- 172.21.15.86:0/1386599492 >> 172.21.15.86:0/1386599492 conn(0x55e264005700 msgr2=0x55e264004890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.922+0000 7f7ece5e6740 1 -- 172.21.15.86:0/1386599492 shutdown_connections 2024-04-17T07:24:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.922+0000 7f7ece5e6740 1 -- 172.21.15.86:0/1386599492 wait complete. 2024-04-17T07:24:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.922+0000 7f7ece5e6740 20 client.26962 trim_cache size 0 max 16384 2024-04-17T07:24:26.923 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:24:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (922 inodes in the stack)", 2024-04-17T07:24:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:24:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "d3203a97-ee8b-467e-8260-2cd27c4ca4f1": { 2024-04-17T07:24:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:24:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "d3203a97-ee8b-467e-8260-2cd27c4ca4f1", 2024-04-17T07:24:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:24:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:24:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:24:26.924 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:24:26.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.924+0000 7f7ecb800640 1 -- 172.21.15.86:0/1028592421 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7ea80a3bb0 msgr2=0x7f7ea80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.924+0000 7f7ecb800640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7ea80a3bb0 0x7f7ea80a6070 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55e26400b3e0 tx=0x7f7eb400c430 comp rx=0 tx=0).stop 2024-04-17T07:24:26.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.924+0000 7f7ecb800640 1 -- 172.21.15.86:0/1028592421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec41053d0 msgr2=0x7f7ec419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:26.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.924+0000 7f7ecb800640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec41053d0 0x7f7ec419a910 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00c970 tx=0x7f7eac00ce30 comp rx=0 tx=0).stop 2024-04-17T07:24:26.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.924+0000 7f7ecb800640 1 -- 172.21.15.86:0/1028592421 shutdown_connections 2024-04-17T07:24:26.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.924+0000 7f7ecb800640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7ea80a3bb0 0x7f7ea80a6070 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.924+0000 7f7ecb800640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec4105d90 0x7f7ec41a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.924+0000 7f7ecb800640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ec41053d0 0x7f7ec419a910 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.924+0000 7f7ecb800640 1 --2- 172.21.15.86:0/1028592421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ec41041e0 0x7f7ec419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.925+0000 7f7ecb800640 1 -- 172.21.15.86:0/1028592421 >> 172.21.15.86:0/1028592421 conn(0x7f7ec40ff750 msgr2=0x7f7ec4108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.925+0000 7f7ecb800640 1 -- 172.21.15.86:0/1028592421 shutdown_connections 2024-04-17T07:24:26.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:26.925+0000 7f7ecb800640 1 -- 172.21.15.86:0/1028592421 wait complete. 2024-04-17T07:24:26.954 INFO:tasks.cephfs.filesystem:scrub status for tag:d3203a97-ee8b-467e-8260-2cd27c4ca4f1 - {'path': '/', 'tag': 'd3203a97-ee8b-467e-8260-2cd27c4ca4f1', 'options': 'recursive,force'} 2024-04-17T07:24:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:27 smithi086 ceph-mon[31733]: pgmap v742: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 624 op/s 2024-04-17T07:24:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:27 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:24:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:27 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:24:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:27 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:24:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:27 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:24:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:27 smithi098 ceph-mon[35418]: pgmap v742: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 624 op/s 2024-04-17T07:24:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:27 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:24:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:27 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:24:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:27 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:24:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:27 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:24:27.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:27 smithi103 ceph-mon[35495]: pgmap v742: 129 pgs: 129 active+clean; 1.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 624 op/s 2024-04-17T07:24:27.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:27 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:24:27.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:27 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:24:27.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:27 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:24:27.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:27 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:24:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 305.0 s, 601.4 tps, lat 160.815 ms stddev 89.076 2024-04-17T07:24:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:29 smithi086 ceph-mon[31733]: pgmap v743: 129 pgs: 129 active+clean; 1.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.4 MiB/s wr, 697 op/s 2024-04-17T07:24:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:29 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:24:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:29 smithi098 ceph-mon[35418]: pgmap v743: 129 pgs: 129 active+clean; 1.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.4 MiB/s wr, 697 op/s 2024-04-17T07:24:29.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:29 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:24:29.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:29 smithi103 ceph-mon[35495]: pgmap v743: 129 pgs: 129 active+clean; 1.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.4 MiB/s wr, 697 op/s 2024-04-17T07:24:29.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:29 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:24:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:31 smithi086 ceph-mon[31733]: pgmap v744: 129 pgs: 129 active+clean; 1.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.9 MiB/s wr, 644 op/s 2024-04-17T07:24:31.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:31 smithi098 ceph-mon[35418]: pgmap v744: 129 pgs: 129 active+clean; 1.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.9 MiB/s wr, 644 op/s 2024-04-17T07:24:31.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:31 smithi103 ceph-mon[35495]: pgmap v744: 129 pgs: 129 active+clean; 1.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.9 MiB/s wr, 644 op/s 2024-04-17T07:24:33.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:33 smithi098 ceph-mon[35418]: pgmap v745: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 702 op/s 2024-04-17T07:24:33.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:33 smithi103 ceph-mon[35495]: pgmap v745: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 702 op/s 2024-04-17T07:24:33.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:33 smithi086 ceph-mon[31733]: pgmap v745: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 702 op/s 2024-04-17T07:24:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 310.0 s, 631.8 tps, lat 163.709 ms stddev 146.643 2024-04-17T07:24:35.633 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:35 smithi086 ceph-mon[31733]: pgmap v746: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.2 MiB/s wr, 669 op/s 2024-04-17T07:24:35.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:35 smithi098 ceph-mon[35418]: pgmap v746: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.2 MiB/s wr, 669 op/s 2024-04-17T07:24:35.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:35 smithi103 ceph-mon[35495]: pgmap v746: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.2 MiB/s wr, 669 op/s 2024-04-17T07:24:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:37 smithi098 ceph-mon[35418]: pgmap v747: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 597 op/s 2024-04-17T07:24:37.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:37 smithi103 ceph-mon[35495]: pgmap v747: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 597 op/s 2024-04-17T07:24:37.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:37 smithi086 ceph-mon[31733]: pgmap v747: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 597 op/s 2024-04-17T07:24:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:38 smithi098 ceph-mon[35418]: pgmap v748: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-04-17T07:24:38.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:38 smithi103 ceph-mon[35495]: pgmap v748: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-04-17T07:24:38.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:38 smithi086 ceph-mon[31733]: pgmap v748: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 681 op/s 2024-04-17T07:24:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 315.0 s, 718.4 tps, lat 139.928 ms stddev 40.762 2024-04-17T07:24:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:40 smithi098 ceph-mon[35418]: pgmap v749: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-04-17T07:24:40.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:40 smithi103 ceph-mon[35495]: pgmap v749: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-04-17T07:24:40.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:40 smithi086 ceph-mon[31733]: pgmap v749: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.5 MiB/s wr, 681 op/s 2024-04-17T07:24:42.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:42 smithi098 ceph-mon[35418]: pgmap v750: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 MiB/s wr, 726 op/s 2024-04-17T07:24:42.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:42 smithi103 ceph-mon[35495]: pgmap v750: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 MiB/s wr, 726 op/s 2024-04-17T07:24:42.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:42 smithi086 ceph-mon[31733]: pgmap v750: 129 pgs: 129 active+clean; 1.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 MiB/s wr, 726 op/s 2024-04-17T07:24:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 320.0 s, 638.2 tps, lat 156.873 ms stddev 60.319 2024-04-17T07:24:44.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:44 smithi103 ceph-mon[35495]: pgmap v751: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.6 MiB/s wr, 695 op/s 2024-04-17T07:24:44.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:44 smithi086 ceph-mon[31733]: pgmap v751: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.6 MiB/s wr, 695 op/s 2024-04-17T07:24:45.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:44 smithi098 ceph-mon[35418]: pgmap v751: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.6 MiB/s wr, 695 op/s 2024-04-17T07:24:46.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:46 smithi103 ceph-mon[35495]: pgmap v752: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-04-17T07:24:46.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:46 smithi086 ceph-mon[31733]: pgmap v752: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-04-17T07:24:47.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:46 smithi098 ceph-mon[35418]: pgmap v752: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-04-17T07:24:48.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:48 smithi103 ceph-mon[35495]: pgmap v753: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-04-17T07:24:48.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:48 smithi086 ceph-mon[31733]: pgmap v753: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-04-17T07:24:49.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:48 smithi098 ceph-mon[35418]: pgmap v753: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 670 op/s 2024-04-17T07:24:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 325.0 s, 644.2 tps, lat 153.809 ms stddev 46.934 2024-04-17T07:24:50.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:50 smithi086 ceph-mon[31733]: pgmap v754: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 637 op/s 2024-04-17T07:24:51.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:50 smithi098 ceph-mon[35418]: pgmap v754: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 637 op/s 2024-04-17T07:24:51.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:50 smithi103 ceph-mon[35495]: pgmap v754: 129 pgs: 129 active+clean; 1.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 637 op/s 2024-04-17T07:24:52.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:52 smithi086 ceph-mon[31733]: pgmap v755: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.6 MiB/s wr, 640 op/s 2024-04-17T07:24:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:52 smithi098 ceph-mon[35418]: pgmap v755: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.6 MiB/s wr, 640 op/s 2024-04-17T07:24:53.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:52 smithi103 ceph-mon[35495]: pgmap v755: 129 pgs: 129 active+clean; 1.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.6 MiB/s wr, 640 op/s 2024-04-17T07:24:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 330.0 s, 625.6 tps, lat 160.082 ms stddev 72.085 2024-04-17T07:24:54.805 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:54 smithi098 ceph-mon[35418]: pgmap v756: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 658 op/s 2024-04-17T07:24:54.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:54 smithi086 ceph-mon[31733]: pgmap v756: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 658 op/s 2024-04-17T07:24:55.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:54 smithi103 ceph-mon[35495]: pgmap v756: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 658 op/s 2024-04-17T07:24:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:56 smithi086 ceph-mon[31733]: pgmap v757: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-04-17T07:24:56.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:24:56.956 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:24:57.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:56 smithi098 ceph-mon[35418]: pgmap v757: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-04-17T07:24:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:56 smithi103 ceph-mon[35495]: pgmap v757: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-04-17T07:24:57.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.275+0000 7f3a40c00640 1 -- 172.21.15.86:0/3918181905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a3c100e70 msgr2=0x7f3a3c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.275+0000 7f3a40c00640 1 --2- 172.21.15.86:0/3918181905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a3c100e70 0x7f3a3c101290 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009900 tx=0x7f3a2c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:24:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.275+0000 7f3a40c00640 1 -- 172.21.15.86:0/3918181905 shutdown_connections 2024-04-17T07:24:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.275+0000 7f3a40c00640 1 --2- 172.21.15.86:0/3918181905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a3c102a10 0x7f3a3c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.275+0000 7f3a40c00640 1 --2- 172.21.15.86:0/3918181905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a3c102050 0x7f3a3c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.275+0000 7f3a40c00640 1 --2- 172.21.15.86:0/3918181905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a3c100e70 0x7f3a3c101290 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.275+0000 7f3a40c00640 1 -- 172.21.15.86:0/3918181905 >> 172.21.15.86:0/3918181905 conn(0x7f3a3c0fc430 msgr2=0x7f3a3c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:57.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.276+0000 7f3a40c00640 1 -- 172.21.15.86:0/3918181905 shutdown_connections 2024-04-17T07:24:57.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.277+0000 7f3a40c00640 1 -- 172.21.15.86:0/3918181905 wait complete. 2024-04-17T07:24:57.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.278+0000 7f3a40c00640 1 Processor -- start 2024-04-17T07:24:57.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.279+0000 7f3a40c00640 1 -- start start 2024-04-17T07:24:57.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.281+0000 7f3a40c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a3c102050 0x7f3a3c070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.281+0000 7f3a40c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a3c102a10 0x7f3a3c0710d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.281+0000 7f3a40c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a3c1a62c0 0x7f3a3c1a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.281+0000 7f3a40c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c10dbc0 con 0x7f3a3c1a62c0 2024-04-17T07:24:57.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.281+0000 7f3a40c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c10da40 con 0x7f3a3c102a10 2024-04-17T07:24:57.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.281+0000 7f3a40c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c10dd40 con 0x7f3a3c102050 2024-04-17T07:24:57.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.281+0000 7f3a39400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a3c102a10 0x7f3a3c0710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.282+0000 7f3a39e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a3c102050 0x7f3a3c070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.282+0000 7f3a39e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a3c102050 0x7f3a3c070b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47712/0 (socket says 172.21.15.86:47712) 2024-04-17T07:24:57.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.282+0000 7f3a39e00640 1 -- 172.21.15.86:0/3080813387 learned_addr learned my addr 172.21.15.86:0/3080813387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:57.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.281+0000 7f3a39400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a3c102a10 0x7f3a3c0710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39200/0 (socket says 172.21.15.86:39200) 2024-04-17T07:24:57.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.282+0000 7f3a3a800640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a3c1a62c0 0x7f3a3c1a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.283+0000 7f3a39e00640 1 -- 172.21.15.86:0/3080813387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a3c102a10 msgr2=0x7f3a3c0710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.283+0000 7f3a39e00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a3c102a10 0x7f3a3c0710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.283+0000 7f3a39e00640 1 -- 172.21.15.86:0/3080813387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a3c1a62c0 msgr2=0x7f3a3c1a86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.283+0000 7f3a39e00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a3c1a62c0 0x7f3a3c1a86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.283+0000 7f3a39e00640 1 -- 172.21.15.86:0/3080813387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3c1a8d20 con 0x7f3a3c102050 2024-04-17T07:24:57.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.283+0000 7f3a39e00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a3c102050 0x7f3a3c070b90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c02c250 tx=0x7f3a2c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:57.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.286+0000 7f3a32a00640 1 -- 172.21.15.86:0/3080813387 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c0044c0 con 0x7f3a3c102050 2024-04-17T07:24:57.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.286+0000 7f3a32a00640 1 -- 172.21.15.86:0/3080813387 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a2c03eb00 con 0x7f3a3c102050 2024-04-17T07:24:57.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.286+0000 7f3a32a00640 1 -- 172.21.15.86:0/3080813387 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c0055c0 con 0x7f3a3c102050 2024-04-17T07:24:57.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.287+0000 7f3a40c00640 1 -- 172.21.15.86:0/3080813387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3c109cf0 con 0x7f3a3c102050 2024-04-17T07:24:57.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.287+0000 7f3a40c00640 1 -- 172.21.15.86:0/3080813387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a3c10a240 con 0x7f3a3c102050 2024-04-17T07:24:57.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.292+0000 7f3a43819740 1 Processor -- start 2024-04-17T07:24:57.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.292+0000 7f3a43819740 1 -- start start 2024-04-17T07:24:57.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.292+0000 7f3a32a00640 1 -- 172.21.15.86:0/3080813387 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3a2c03ecb0 con 0x7f3a3c102050 2024-04-17T07:24:57.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.293+0000 7f3a43819740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.294+0000 7f3a3a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.294+0000 7f3a32a00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a200a3d80 0x7f3a200a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.294+0000 7f3a3a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47718/0 (socket says 172.21.15.86:47718) 2024-04-17T07:24:57.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.294+0000 7f3a3a800640 1 -- 172.21.15.86:0/596236431 learned_addr learned my addr 172.21.15.86:0/596236431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:57.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.295+0000 7f3a32a00640 1 -- 172.21.15.86:0/3080813387 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3a2c0f2030 con 0x7f3a3c102050 2024-04-17T07:24:57.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.295+0000 7f3a43819740 1 --2- 172.21.15.86:0/596236431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a2390 0x55ecb38a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.295+0000 7f3a43819740 1 --2- 172.21.15.86:0/596236431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a7500 0x55ecb38a9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.295+0000 7f3a43819740 1 -- 172.21.15.86:0/596236431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ecb3800120 con 0x55ecb38a2390 2024-04-17T07:24:57.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.295+0000 7f3a43819740 1 -- 172.21.15.86:0/596236431 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ecb37a6340 con 0x55ecb38a7500 2024-04-17T07:24:57.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.295+0000 7f3a43819740 1 -- 172.21.15.86:0/596236431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ecb37fc0e0 con 0x55ecb38a19a0 2024-04-17T07:24:57.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.296+0000 7f3a39400640 1 --2- 172.21.15.86:0/596236431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a2390 0x55ecb38a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.296+0000 7f3a3a800640 1 -- 172.21.15.86:0/596236431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a7500 msgr2=0x55ecb38a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:57.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.296+0000 7f3a3a800640 1 --2- 172.21.15.86:0/596236431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a7500 0x55ecb38a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.296+0000 7f3a39e00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a200a3d80 0x7f3a200a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.296+0000 7f3a3a800640 1 -- 172.21.15.86:0/596236431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a2390 msgr2=0x55ecb38a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:57.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.296+0000 7f3a3a800640 1 --2- 172.21.15.86:0/596236431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a2390 0x55ecb38a6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.296+0000 7f3a3a800640 1 -- 172.21.15.86:0/596236431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb35dc7d0 con 0x55ecb38a19a0 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.296+0000 7f3a3a800640 1 --2- 172.21.15.86:0/596236431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38a1dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400a550 tx=0x7f3a3400aa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c5b8a0e2a1bc2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.297+0000 7f3a27000640 1 -- 172.21.15.86:0/596236431 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34018020 con 0x55ecb38a19a0 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.297+0000 7f3a27000640 1 -- 172.21.15.86:0/596236431 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a34012b80 con 0x55ecb38a19a0 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.297+0000 7f3a27000640 1 -- 172.21.15.86:0/596236431 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34014be0 con 0x55ecb38a19a0 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.297+0000 7f3a39e00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a200a3d80 0x7f3a200a6240 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c005820 tx=0x7f3a2c02bc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.298+0000 7f3a43819740 1 -- 172.21.15.86:0/596236431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 msgr2=0x55ecb38a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.298+0000 7f3a43819740 1 --2- 172.21.15.86:0/596236431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38a1dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400a550 tx=0x7f3a3400aa10 comp rx=0 tx=0).stop 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.298+0000 7f3a43819740 1 -- 172.21.15.86:0/596236431 shutdown_connections 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.298+0000 7f3a43819740 1 --2- 172.21.15.86:0/596236431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a7500 0x55ecb38a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.298+0000 7f3a43819740 1 --2- 172.21.15.86:0/596236431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a2390 0x55ecb38a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.298+0000 7f3a43819740 1 --2- 172.21.15.86:0/596236431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38a1dc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.298+0000 7f3a43819740 1 -- 172.21.15.86:0/596236431 >> 172.21.15.86:0/596236431 conn(0x55ecb389cf10 msgr2=0x55ecb389d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.298+0000 7f3a43819740 1 -- 172.21.15.86:0/596236431 shutdown_connections 2024-04-17T07:24:57.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.298+0000 7f3a43819740 1 -- 172.21.15.86:0/596236431 wait complete. 2024-04-17T07:24:57.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.299+0000 7f3a43819740 1 -- start start 2024-04-17T07:24:57.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.299+0000 7f3a43819740 1 Processor -- start 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a43819740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a43819740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 0x55ecb38ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a43819740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a7500 0x55ecb38b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a43819740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ecb38aa4b0 con 0x55ecb38a7500 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a43819740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ecb37b9710 con 0x55ecb38a2390 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a43819740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ecb38aa630 con 0x55ecb38a19a0 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a43819740 10 client.0 _ll_register_callbacks cb 0x55ecb37b4af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a3a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a3a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38abfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47720/0 (socket says 172.21.15.86:47720) 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a3a800640 1 -- 172.21.15.86:0/2086703832 learned_addr learned my addr 172.21.15.86:0/2086703832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a3a800640 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a7500 0x55ecb38b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.300+0000 7f3a39400640 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 0x55ecb38ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.302+0000 7f3a3a800640 1 -- 172.21.15.86:0/2086703832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 msgr2=0x55ecb38ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:57.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.302+0000 7f3a3a800640 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 0x55ecb38ac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.302+0000 7f3a3a800640 1 -- 172.21.15.86:0/2086703832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a7500 msgr2=0x55ecb38b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:57.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.302+0000 7f3a3a800640 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a7500 0x55ecb38b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.302+0000 7f3a3a800640 1 -- 172.21.15.86:0/2086703832 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb35123f0 con 0x55ecb38a19a0 2024-04-17T07:24:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.302+0000 7f3a3a800640 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38abfa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a34014d40 tx=0x7f3a34004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9fc0590f9e3612 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.302+0000 7f3a1b400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.303+0000 7f3a1b400640 1 -- 172.21.15.86:0/2086703832 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34018040 con 0x55ecb38a19a0 2024-04-17T07:24:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.303+0000 7f3a1b400640 1 -- 172.21.15.86:0/2086703832 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3a34025030 con 0x55ecb38a19a0 2024-04-17T07:24:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.303+0000 7f3a43819740 1 -- 172.21.15.86:0/2086703832 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a34012b80 con 0x55ecb38a19a0 2024-04-17T07:24:57.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.304+0000 7f3a1b400640 1 -- 172.21.15.86:0/2086703832 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a34012910 con 0x55ecb38a19a0 2024-04-17T07:24:57.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.306+0000 7f3a43819740 10 client.26974 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:57.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.306+0000 7f3a43819740 1 -- 172.21.15.86:0/2086703832 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ecb34d20f0 con 0x55ecb38a19a0 2024-04-17T07:24:57.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.306+0000 7f3a1b400640 1 -- 172.21.15.86:0/2086703832 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a34012b80 con 0x55ecb38a19a0 2024-04-17T07:24:57.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.306+0000 7f3a1b400640 1 -- 172.21.15.86:0/2086703832 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3a3401f440 con 0x55ecb38a19a0 2024-04-17T07:24:57.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.306+0000 7f3a43819740 10 client.26974 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:57.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.306+0000 7f3a43819740 10 client.26974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:24:57.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.306+0000 7f3a43819740 20 client.26974 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:57.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.306+0000 7f3a43819740 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ecb38b7bc0 0x55ecb38b9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.306+0000 7f3a43819740 4 client.26974 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:57.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.306+0000 7f3a43819740 1 -- 172.21.15.86:0/2086703832 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecb380db40 con 0x55ecb38b7bc0 2024-04-17T07:24:57.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.307+0000 7f3a39400640 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ecb38b7bc0 0x55ecb38b9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.308+0000 7f3a39400640 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ecb38b7bc0 0x55ecb38b9fd0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2746f930ac32d52 server_cookie=95877efe0f291458 in_seq=0 out_seq=0 2024-04-17T07:24:57.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.308+0000 7f3a1b400640 10 client.26974 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:57.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.309+0000 7f3a1b400640 1 -- 172.21.15.86:0/2086703832 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecb380db40 con 0x55ecb38b7bc0 2024-04-17T07:24:57.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.309+0000 7f3a1b400640 10 client.26974 handle_command_reply: tid=0 2024-04-17T07:24:57.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.310+0000 7f3a43819740 1 client.26974 shutdown 2024-04-17T07:24:57.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.310+0000 7f3a43819740 1 -- 172.21.15.86:0/2086703832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 msgr2=0x55ecb38abfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.310+0000 7f3a43819740 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38abfa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a34014d40 tx=0x7f3a34004b50 comp rx=0 tx=0).stop 2024-04-17T07:24:57.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.310+0000 7f3a43819740 1 -- 172.21.15.86:0/2086703832 shutdown_connections 2024-04-17T07:24:57.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.310+0000 7f3a43819740 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ecb38b7bc0 0x55ecb38b9fd0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.311+0000 7f3a43819740 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a7500 0x55ecb38b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.311+0000 7f3a43819740 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 0x55ecb38ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.311+0000 7f3a43819740 1 --2- 172.21.15.86:0/2086703832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a19a0 0x55ecb38abfa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.311+0000 7f3a43819740 1 -- 172.21.15.86:0/2086703832 >> 172.21.15.86:0/2086703832 conn(0x55ecb389cf10 msgr2=0x55ecb3893890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:57.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.311+0000 7f3a1b400640 0 client.26974 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:57.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.311+0000 7f3a43819740 1 -- 172.21.15.86:0/2086703832 shutdown_connections 2024-04-17T07:24:57.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.311+0000 7f3a43819740 1 -- 172.21.15.86:0/2086703832 wait complete. 2024-04-17T07:24:57.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.311+0000 7f3a43819740 20 client.26974 trim_cache size 0 max 16384 2024-04-17T07:24:57.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.345+0000 7f3a43819740 1 Processor -- start 2024-04-17T07:24:57.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.346+0000 7f3a43819740 1 -- start start 2024-04-17T07:24:57.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.346+0000 7f3a43819740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a19a0 0x55ecb38abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.346+0000 7f3a43819740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 0x55ecb38b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.346+0000 7f3a43819740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb38ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.346+0000 7f3a43819740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b17b0 con 0x55ecb38a19a0 2024-04-17T07:24:57.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.346+0000 7f3a43819740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b32f0 con 0x55ecb38a2390 2024-04-17T07:24:57.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.346+0000 7f3a43819740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b1e80 con 0x55ecb38a7500 2024-04-17T07:24:57.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.347+0000 7f3a39400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 0x55ecb38b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.347+0000 7f3a39400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 0x55ecb38b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39232/0 (socket says 172.21.15.86:39232) 2024-04-17T07:24:57.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.347+0000 7f3a39400640 1 -- 172.21.15.86:0/1557990755 learned_addr learned my addr 172.21.15.86:0/1557990755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:57.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.347+0000 7f3a3a800640 1 --2- 172.21.15.86:0/1557990755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a19a0 0x55ecb38abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.347+0000 7f3a3a800640 1 --2- 172.21.15.86:0/1557990755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb38ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.347+0000 7f3a3a800640 1 -- 172.21.15.86:0/1557990755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 msgr2=0x55ecb38b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:57.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.347+0000 7f3a3a800640 1 --2- 172.21.15.86:0/1557990755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 0x55ecb38b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.347+0000 7f3a3a800640 1 -- 172.21.15.86:0/1557990755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a19a0 msgr2=0x55ecb38abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:57.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.347+0000 7f3a3a800640 1 --2- 172.21.15.86:0/1557990755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a19a0 0x55ecb38abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.347+0000 7f3a3a800640 1 -- 172.21.15.86:0/1557990755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb388ece0 con 0x55ecb38a7500 2024-04-17T07:24:57.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.348+0000 7f3a3a800640 1 --2- 172.21.15.86:0/1557990755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb38ac0f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a340097c0 tx=0x7f3a3400aa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fa24a1cf0d80e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:57.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.348+0000 7f3a26600640 1 -- 172.21.15.86:0/1557990755 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34018070 con 0x55ecb38a7500 2024-04-17T07:24:57.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.349+0000 7f3a26600640 1 -- 172.21.15.86:0/1557990755 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a34024020 con 0x55ecb38a7500 2024-04-17T07:24:57.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.349+0000 7f3a26600640 1 -- 172.21.15.86:0/1557990755 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3401fc50 con 0x55ecb38a7500 2024-04-17T07:24:57.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.349+0000 7f3a43819740 1 -- 172.21.15.86:0/1557990755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 msgr2=0x55ecb38ac0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.349+0000 7f3a43819740 1 --2- 172.21.15.86:0/1557990755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb38ac0f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a340097c0 tx=0x7f3a3400aa10 comp rx=0 tx=0).stop 2024-04-17T07:24:57.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.349+0000 7f3a43819740 1 -- 172.21.15.86:0/1557990755 shutdown_connections 2024-04-17T07:24:57.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.349+0000 7f3a43819740 1 --2- 172.21.15.86:0/1557990755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb38ac0f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.349+0000 7f3a43819740 1 --2- 172.21.15.86:0/1557990755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a2390 0x55ecb38b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.349+0000 7f3a43819740 1 --2- 172.21.15.86:0/1557990755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a19a0 0x55ecb38abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.349+0000 7f3a43819740 1 -- 172.21.15.86:0/1557990755 >> 172.21.15.86:0/1557990755 conn(0x55ecb389cf10 msgr2=0x55ecb38aaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.350+0000 7f3a43819740 1 -- 172.21.15.86:0/1557990755 shutdown_connections 2024-04-17T07:24:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.350+0000 7f3a43819740 1 -- 172.21.15.86:0/1557990755 wait complete. 2024-04-17T07:24:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.350+0000 7f3a43819740 1 -- start start 2024-04-17T07:24:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.351+0000 7f3a43819740 1 Processor -- start 2024-04-17T07:24:57.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.351+0000 7f3a43819740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a19a0 0x55ecb38b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a43819740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a2390 0x55ecb38b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a43819740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb391c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a43819740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b2c20 con 0x55ecb38a2390 2024-04-17T07:24:57.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a43819740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1b2550 con 0x55ecb38a19a0 2024-04-17T07:24:57.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a43819740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c1af780 con 0x55ecb38a7500 2024-04-17T07:24:57.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a43819740 10 client.0 _ll_register_callbacks cb 0x7f3a3400a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a3a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb391c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a3a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb391c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47726/0 (socket says 172.21.15.86:47726) 2024-04-17T07:24:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a3a800640 1 -- 172.21.15.86:0/208304911 learned_addr learned my addr 172.21.15.86:0/208304911 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a3a800640 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a19a0 0x55ecb38b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.352+0000 7f3a39400640 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a2390 0x55ecb38b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.353+0000 7f3a3a800640 1 -- 172.21.15.86:0/208304911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a19a0 msgr2=0x55ecb38b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.353+0000 7f3a3a800640 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a19a0 0x55ecb38b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.353+0000 7f3a3a800640 1 -- 172.21.15.86:0/208304911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a2390 msgr2=0x55ecb38b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.353+0000 7f3a3a800640 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a2390 0x55ecb38b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.353+0000 7f3a3a800640 1 -- 172.21.15.86:0/208304911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb38b0e20 con 0x55ecb38a7500 2024-04-17T07:24:57.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.353+0000 7f3a3a800640 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb391c100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3a34014d10 tx=0x7f3a34015e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aaf35c19f29962b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:57.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.353+0000 7f3a1b400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:57.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.353+0000 7f3a1b400640 1 -- 172.21.15.86:0/208304911 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34018040 con 0x55ecb38a7500 2024-04-17T07:24:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.354+0000 7f3a1b400640 1 -- 172.21.15.86:0/208304911 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3a340145f0 con 0x55ecb38a7500 2024-04-17T07:24:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.354+0000 7f3a43819740 1 -- 172.21.15.86:0/208304911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a34024020 con 0x55ecb38a7500 2024-04-17T07:24:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.354+0000 7f3a1b400640 1 -- 172.21.15.86:0/208304911 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a34047050 con 0x55ecb38a7500 2024-04-17T07:24:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.354+0000 7f3a43819740 10 client.26980 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.354+0000 7f3a43819740 1 -- 172.21.15.86:0/208304911 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ecb391d720 con 0x55ecb38a7500 2024-04-17T07:24:57.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.354+0000 7f3a1b400640 1 -- 172.21.15.86:0/208304911 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a34024020 con 0x55ecb38a7500 2024-04-17T07:24:57.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.355+0000 7f3a1b400640 1 -- 172.21.15.86:0/208304911 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3a34035030 con 0x55ecb38a7500 2024-04-17T07:24:57.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.355+0000 7f3a43819740 10 client.26980 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:57.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.355+0000 7f3a43819740 10 client.26980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:24:57.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.355+0000 7f3a43819740 20 client.26980 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:57.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.355+0000 7f3a43819740 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ecb391e050 0x55ecb3920470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.355+0000 7f3a43819740 4 client.26980 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:24:57.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.355+0000 7f3a43819740 1 -- 172.21.15.86:0/208304911 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ecb38079b0 con 0x55ecb391e050 2024-04-17T07:24:57.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.356+0000 7f3a39400640 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ecb391e050 0x55ecb3920470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.358+0000 7f3a39400640 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ecb391e050 0x55ecb3920470 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1719ef81f30f1aed server_cookie=543e5b27c2ad2e93 in_seq=0 out_seq=0 2024-04-17T07:24:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.358+0000 7f3a1b400640 10 client.26980 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.358+0000 7f3a1b400640 1 -- 172.21.15.86:0/208304911 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ecb38079b0 con 0x55ecb391e050 2024-04-17T07:24:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.358+0000 7f3a1b400640 10 client.26980 handle_command_reply: tid=0 2024-04-17T07:24:57.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.358+0000 7f3a43819740 1 client.26980 shutdown 2024-04-17T07:24:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.360+0000 7f3a43819740 1 -- 172.21.15.86:0/208304911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 msgr2=0x55ecb391c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.360+0000 7f3a43819740 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb391c100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3a34014d10 tx=0x7f3a34015e90 comp rx=0 tx=0).stop 2024-04-17T07:24:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.360+0000 7f3a43819740 1 -- 172.21.15.86:0/208304911 shutdown_connections 2024-04-17T07:24:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.360+0000 7f3a43819740 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ecb391e050 0x55ecb3920470 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.360+0000 7f3a43819740 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ecb38a7500 0x55ecb391c100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.360+0000 7f3a43819740 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecb38a2390 0x55ecb38b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.360+0000 7f3a43819740 1 --2- 172.21.15.86:0/208304911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ecb38a19a0 0x55ecb38b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.360+0000 7f3a43819740 1 -- 172.21.15.86:0/208304911 >> 172.21.15.86:0/208304911 conn(0x55ecb389cf10 msgr2=0x55ecb38abb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.360+0000 7f3a1b400640 0 client.26980 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:24:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.361+0000 7f3a43819740 1 -- 172.21.15.86:0/208304911 shutdown_connections 2024-04-17T07:24:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.361+0000 7f3a43819740 1 -- 172.21.15.86:0/208304911 wait complete. 2024-04-17T07:24:57.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.361+0000 7f3a43819740 20 client.26980 trim_cache size 0 max 16384 2024-04-17T07:24:57.362 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:24:57.362 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:24:57.362 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:24:57.362 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:24:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.362+0000 7f3a40c00640 1 -- 172.21.15.86:0/3080813387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a200a3d80 msgr2=0x7f3a200a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.362+0000 7f3a40c00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a200a3d80 0x7f3a200a6240 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c005820 tx=0x7f3a2c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:24:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.362+0000 7f3a40c00640 1 -- 172.21.15.86:0/3080813387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a3c102050 msgr2=0x7f3a3c070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.362+0000 7f3a40c00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a3c102050 0x7f3a3c070b90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c02c250 tx=0x7f3a2c0040c0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.362+0000 7f3a40c00640 1 -- 172.21.15.86:0/3080813387 shutdown_connections 2024-04-17T07:24:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.362+0000 7f3a40c00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3a200a3d80 0x7f3a200a6240 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.362+0000 7f3a40c00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a3c1a62c0 0x7f3a3c1a86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.362+0000 7f3a40c00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a3c102a10 0x7f3a3c0710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.362+0000 7f3a40c00640 1 --2- 172.21.15.86:0/3080813387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a3c102050 0x7f3a3c070b90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.362+0000 7f3a40c00640 1 -- 172.21.15.86:0/3080813387 >> 172.21.15.86:0/3080813387 conn(0x7f3a3c0fc430 msgr2=0x7f3a3c1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.363+0000 7f3a40c00640 1 -- 172.21.15.86:0/3080813387 shutdown_connections 2024-04-17T07:24:57.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.363+0000 7f3a40c00640 1 -- 172.21.15.86:0/3080813387 wait complete. 2024-04-17T07:24:57.395 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:24:57.395 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:24:57.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.711+0000 7f0429400640 1 -- 172.21.15.86:0/3760664132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04241107e0 msgr2=0x7f0424112bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.711+0000 7f0429400640 1 --2- 172.21.15.86:0/3760664132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04241107e0 0x7f0424112bd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0414009900 tx=0x7f041402bc40 comp rx=0 tx=0).stop 2024-04-17T07:24:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.712+0000 7f0429400640 1 -- 172.21.15.86:0/3760664132 shutdown_connections 2024-04-17T07:24:57.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.712+0000 7f0429400640 1 --2- 172.21.15.86:0/3760664132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04241107e0 0x7f0424112bd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.712+0000 7f0429400640 1 --2- 172.21.15.86:0/3760664132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04241098c0 0x7f04241102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.712+0000 7f0429400640 1 --2- 172.21.15.86:0/3760664132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424108f60 0x7f0424109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.712+0000 7f0429400640 1 -- 172.21.15.86:0/3760664132 >> 172.21.15.86:0/3760664132 conn(0x7f042406ee00 msgr2=0x7f0424071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.712+0000 7f0429400640 1 -- 172.21.15.86:0/3760664132 shutdown_connections 2024-04-17T07:24:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.712+0000 7f0429400640 1 -- 172.21.15.86:0/3760664132 wait complete. 2024-04-17T07:24:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.713+0000 7f0429400640 1 Processor -- start 2024-04-17T07:24:57.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.713+0000 7f0429400640 1 -- start start 2024-04-17T07:24:57.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.714+0000 7f0429400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424108f60 0x7f042419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.714+0000 7f0429400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04241098c0 0x7f042419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.714+0000 7f0429400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04241107e0 0x7f04241a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.714+0000 7f0429400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0424114130 con 0x7f0424108f60 2024-04-17T07:24:57.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.714+0000 7f0429400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0424113fb0 con 0x7f04241107e0 2024-04-17T07:24:57.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.714+0000 7f0429400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f04241142b0 con 0x7f04241098c0 2024-04-17T07:24:57.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.714+0000 7f0423200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04241107e0 0x7f04241a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.714+0000 7f0423200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04241107e0 0x7f04241a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39252/0 (socket says 172.21.15.86:39252) 2024-04-17T07:24:57.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.714+0000 7f0423200640 1 -- 172.21.15.86:0/3112057556 learned_addr learned my addr 172.21.15.86:0/3112057556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.715+0000 7f0422800640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424108f60 0x7f042419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.715+0000 7f0421e00640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04241098c0 0x7f042419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.715+0000 7f0423200640 1 -- 172.21.15.86:0/3112057556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04241098c0 msgr2=0x7f042419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.715+0000 7f0423200640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04241098c0 0x7f042419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.715+0000 7f0423200640 1 -- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424108f60 msgr2=0x7f042419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.715+0000 7f0423200640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424108f60 0x7f042419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.715+0000 7f0423200640 1 -- 172.21.15.86:0/3112057556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04241aec00 con 0x7f04241107e0 2024-04-17T07:24:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.715+0000 7f0422800640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424108f60 0x7f042419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:24:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.715+0000 7f0421e00640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04241098c0 0x7f042419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:24:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.715+0000 7f0423200640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04241107e0 0x7f04241a5d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0414036040 tx=0x7f0414004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.716+0000 7f041b400640 1 -- 172.21.15.86:0/3112057556 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414043020 con 0x7f04241107e0 2024-04-17T07:24:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.716+0000 7f041b400640 1 -- 172.21.15.86:0/3112057556 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04140043e0 con 0x7f04241107e0 2024-04-17T07:24:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.716+0000 7f041b400640 1 -- 172.21.15.86:0/3112057556 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414003bb0 con 0x7f04241107e0 2024-04-17T07:24:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.717+0000 7f0429400640 1 -- 172.21.15.86:0/3112057556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04241aee90 con 0x7f04241107e0 2024-04-17T07:24:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.717+0000 7f0429400640 1 -- 172.21.15.86:0/3112057556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04241af300 con 0x7f04241107e0 2024-04-17T07:24:57.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.719+0000 7f0429400640 1 -- 172.21.15.86:0/3112057556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042410f9a0 con 0x7f04241107e0 2024-04-17T07:24:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.722+0000 7f041b400640 1 -- 172.21.15.86:0/3112057556 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0414046020 con 0x7f04241107e0 2024-04-17T07:24:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.724+0000 7f041b400640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04080a3c70 0x7f04080a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.724+0000 7f041b400640 1 -- 172.21.15.86:0/3112057556 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f04140f25e0 con 0x7f04241107e0 2024-04-17T07:24:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.725+0000 7f0422800640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04080a3c70 0x7f04080a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.726+0000 7f0422800640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04080a3c70 0x7f04080a6130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f041c005fd0 tx=0x7f041c0093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:57.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:57.741+0000 7f041b400640 1 -- 172.21.15.86:0/3112057556 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f041404d070 con 0x7f04241107e0 2024-04-17T07:24:58.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.218+0000 7f0429400640 1 -- 172.21.15.86:0/3112057556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f042402d5d0 con 0x7f04241107e0 2024-04-17T07:24:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.221+0000 7f041b400640 1 -- 172.21.15.86:0/3112057556 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f04140bec40 con 0x7f04241107e0 2024-04-17T07:24:58.221 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:24:58.221 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:24:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:24:58.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 -- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04080a3c70 msgr2=0x7f04080a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04080a3c70 0x7f04080a6130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f041c005fd0 tx=0x7f041c0093e0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 -- 172.21.15.86:0/3112057556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04241107e0 msgr2=0x7f04241a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04241107e0 0x7f04241a5d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0414036040 tx=0x7f0414004060 comp rx=0 tx=0).stop 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 -- 172.21.15.86:0/3112057556 shutdown_connections 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04080a3c70 0x7f04080a6130 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04241107e0 0x7f04241a5d80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04241098c0 0x7f042419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 --2- 172.21.15.86:0/3112057556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0424108f60 0x7f042419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 -- 172.21.15.86:0/3112057556 >> 172.21.15.86:0/3112057556 conn(0x7f042406ee00 msgr2=0x7f0424070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.225+0000 7f0429400640 1 -- 172.21.15.86:0/3112057556 shutdown_connections 2024-04-17T07:24:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.226+0000 7f0429400640 1 -- 172.21.15.86:0/3112057556 wait complete. 2024-04-17T07:24:58.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:24:58.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.567+0000 7f863f800640 1 -- 172.21.15.86:0/2461810156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8638109560 msgr2=0x7f86381099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.567+0000 7f863f800640 1 --2- 172.21.15.86:0/2461810156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8638109560 0x7f86381099c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8628009900 tx=0x7f862802bc60 comp rx=0 tx=0).stop 2024-04-17T07:24:58.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.567+0000 7f863f800640 1 -- 172.21.15.86:0/2461810156 shutdown_connections 2024-04-17T07:24:58.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.567+0000 7f863f800640 1 --2- 172.21.15.86:0/2461810156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638109f00 0x7f863810a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.567+0000 7f863f800640 1 --2- 172.21.15.86:0/2461810156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8638109560 0x7f86381099c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.567+0000 7f863f800640 1 --2- 172.21.15.86:0/2461810156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8638109140 0x7f863810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.567+0000 7f863f800640 1 -- 172.21.15.86:0/2461810156 >> 172.21.15.86:0/2461810156 conn(0x7f863806f190 msgr2=0x7f86380715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:58.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.567+0000 7f863f800640 1 -- 172.21.15.86:0/2461810156 shutdown_connections 2024-04-17T07:24:58.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.567+0000 7f863f800640 1 -- 172.21.15.86:0/2461810156 wait complete. 2024-04-17T07:24:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.569+0000 7f863f800640 1 Processor -- start 2024-04-17T07:24:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.569+0000 7f863f800640 1 -- start start 2024-04-17T07:24:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.569+0000 7f863f800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8638109140 0x7f86381a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.569+0000 7f863f800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8638109560 0x7f86381a7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.569+0000 7f863f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638109f00 0x7f86381ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.569+0000 7f863f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8638114090 con 0x7f8638109f00 2024-04-17T07:24:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.569+0000 7f863f800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8638113f10 con 0x7f8638109560 2024-04-17T07:24:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.569+0000 7f863f800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8638114210 con 0x7f8638109140 2024-04-17T07:24:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.570+0000 7f863d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638109f00 0x7f86381ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.570+0000 7f863ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8638109140 0x7f86381a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.570+0000 7f863ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8638109140 0x7f86381a6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47764/0 (socket says 172.21.15.86:47764) 2024-04-17T07:24:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.570+0000 7f863d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638109f00 0x7f86381ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44616/0 (socket says 172.21.15.86:44616) 2024-04-17T07:24:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.570+0000 7f863ce00640 1 -- 172.21.15.86:0/2281792504 learned_addr learned my addr 172.21.15.86:0/2281792504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.570+0000 7f8637e00640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8638109560 0x7f86381a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.570+0000 7f863d800640 1 -- 172.21.15.86:0/2281792504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8638109140 msgr2=0x7f86381a6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.570+0000 7f863d800640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8638109140 0x7f86381a6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.571+0000 7f863d800640 1 -- 172.21.15.86:0/2281792504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8638109560 msgr2=0x7f86381a7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.571+0000 7f863d800640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8638109560 0x7f86381a7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.571+0000 7f863d800640 1 -- 172.21.15.86:0/2281792504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86381aec00 con 0x7f8638109f00 2024-04-17T07:24:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.571+0000 7f8637e00640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8638109560 0x7f86381a7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.571+0000 7f863d800640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638109f00 0x7f86381ae590 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f862400e970 tx=0x7f862400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:58.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.572+0000 7f8635600640 1 -- 172.21.15.86:0/2281792504 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862400ccc0 con 0x7f8638109f00 2024-04-17T07:24:58.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.572+0000 7f8635600640 1 -- 172.21.15.86:0/2281792504 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8624005020 con 0x7f8638109f00 2024-04-17T07:24:58.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.572+0000 7f8635600640 1 -- 172.21.15.86:0/2281792504 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624004a00 con 0x7f8638109f00 2024-04-17T07:24:58.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.572+0000 7f863f800640 1 -- 172.21.15.86:0/2281792504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86381aeef0 con 0x7f8638109f00 2024-04-17T07:24:58.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.572+0000 7f863f800640 1 -- 172.21.15.86:0/2281792504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86381af360 con 0x7f8638109f00 2024-04-17T07:24:58.575 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:58 smithi086 ceph-mon[31733]: pgmap v758: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 649 op/s 2024-04-17T07:24:58.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.580+0000 7f8635600640 1 -- 172.21.15.86:0/2281792504 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8624004ba0 con 0x7f8638109f00 2024-04-17T07:24:58.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.582+0000 7f8635600640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f861c0a3c30 0x7f861c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.582+0000 7f8635600640 1 -- 172.21.15.86:0/2281792504 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f86240d0670 con 0x7f8638109f00 2024-04-17T07:24:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.583+0000 7f86425d2740 1 Processor -- start 2024-04-17T07:24:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.584+0000 7f863ce00640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f861c0a3c30 0x7f861c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.584+0000 7f86425d2740 1 -- start start 2024-04-17T07:24:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.584+0000 7f86425d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c6823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.584+0000 7f86425d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 0x56207c687490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.584+0000 7f86425d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c687b00 0x56207c689f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.584+0000 7f86425d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56207c5e8020 con 0x56207c682990 2024-04-17T07:24:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.584+0000 7f86425d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56207c58e460 con 0x56207c687b00 2024-04-17T07:24:58.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.584+0000 7f86425d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56207c5e3fe0 con 0x56207c681fa0 2024-04-17T07:24:58.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.585+0000 7f863ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c6823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.585+0000 7f863ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c6823c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47772/0 (socket says 172.21.15.86:47772) 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.585+0000 7f863ce00640 1 -- 172.21.15.86:0/172523436 learned_addr learned my addr 172.21.15.86:0/172523436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.585+0000 7f863d800640 1 --2- 172.21.15.86:0/172523436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c687b00 0x56207c689f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.586+0000 7f863ce00640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f861c0a3c30 0x7f861c0a60f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8630011570 tx=0x7f8630011970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.586+0000 7f863ce00640 1 -- 172.21.15.86:0/172523436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c687b00 msgr2=0x56207c689f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.586+0000 7f863ce00640 1 --2- 172.21.15.86:0/172523436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c687b00 0x56207c689f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.586+0000 7f863ce00640 1 -- 172.21.15.86:0/172523436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 msgr2=0x56207c687490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.586+0000 7f863ce00640 1 --2- 172.21.15.86:0/172523436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 0x56207c687490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.586+0000 7f863ce00640 1 -- 172.21.15.86:0/172523436 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207c596040 con 0x56207c681fa0 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.586+0000 7f863ce00640 1 --2- 172.21.15.86:0/172523436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c6823c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f863000f940 tx=0x7f863001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3015e3261990cf9b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:58.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.587+0000 7f862e000640 1 -- 172.21.15.86:0/172523436 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630023070 con 0x56207c681fa0 2024-04-17T07:24:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.587+0000 7f862e000640 1 -- 172.21.15.86:0/172523436 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8630019070 con 0x56207c681fa0 2024-04-17T07:24:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.587+0000 7f862e000640 1 -- 172.21.15.86:0/172523436 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630017040 con 0x56207c681fa0 2024-04-17T07:24:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.588+0000 7f86425d2740 1 -- 172.21.15.86:0/172523436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 msgr2=0x56207c6823c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.588+0000 7f86425d2740 1 --2- 172.21.15.86:0/172523436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c6823c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f863000f940 tx=0x7f863001e000 comp rx=0 tx=0).stop 2024-04-17T07:24:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.588+0000 7f86425d2740 1 -- 172.21.15.86:0/172523436 shutdown_connections 2024-04-17T07:24:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.588+0000 7f86425d2740 1 --2- 172.21.15.86:0/172523436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c687b00 0x56207c689f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.588+0000 7f86425d2740 1 --2- 172.21.15.86:0/172523436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 0x56207c687490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.588+0000 7f86425d2740 1 --2- 172.21.15.86:0/172523436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c6823c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.588+0000 7f86425d2740 1 -- 172.21.15.86:0/172523436 >> 172.21.15.86:0/172523436 conn(0x56207c67d510 msgr2=0x56207c67d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.589+0000 7f86425d2740 1 -- 172.21.15.86:0/172523436 shutdown_connections 2024-04-17T07:24:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.589+0000 7f86425d2740 1 -- 172.21.15.86:0/172523436 wait complete. 2024-04-17T07:24:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.589+0000 7f86425d2740 1 -- start start 2024-04-17T07:24:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.589+0000 7f86425d2740 1 Processor -- start 2024-04-17T07:24:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.590+0000 7f86425d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c681fa0 0x56207c68c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.590+0000 7f86425d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 0x56207c68c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.590+0000 7f86425d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c687b00 0x56207c6939b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.590+0000 7f86425d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56207c68aa50 con 0x56207c682990 2024-04-17T07:24:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.590+0000 7f86425d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56207c5a1830 con 0x56207c681fa0 2024-04-17T07:24:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.590+0000 7f86425d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56207c68abd0 con 0x56207c687b00 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f86425d2740 10 client.0 _ll_register_callbacks cb 0x56207c56ffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f8637e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 0x56207c68c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f863d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c687b00 0x56207c6939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f8637e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 0x56207c68c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44642/0 (socket says 172.21.15.86:44642) 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f8637e00640 1 -- 172.21.15.86:0/1908943254 learned_addr learned my addr 172.21.15.86:0/1908943254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f863ce00640 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c681fa0 0x56207c68c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f8637e00640 1 -- 172.21.15.86:0/1908943254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c687b00 msgr2=0x56207c6939b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f8637e00640 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c687b00 0x56207c6939b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f8637e00640 1 -- 172.21.15.86:0/1908943254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c681fa0 msgr2=0x56207c68c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f8637e00640 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c681fa0 0x56207c68c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f8637e00640 1 -- 172.21.15.86:0/1908943254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207c5a5920 con 0x56207c682990 2024-04-17T07:24:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.591+0000 7f863ce00640 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c681fa0 0x56207c68c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:24:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.592+0000 7f8637e00640 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 0x56207c68c970 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8628009900 tx=0x7f8628005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5af5f34426c2a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.592+0000 7f8622a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:24:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.592+0000 7f8622a00640 1 -- 172.21.15.86:0/1908943254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628043020 con 0x56207c682990 2024-04-17T07:24:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.592+0000 7f8622a00640 1 -- 172.21.15.86:0/1908943254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f862802c600 con 0x56207c682990 2024-04-17T07:24:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.593+0000 7f86425d2740 1 -- 172.21.15.86:0/1908943254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8630019070 con 0x56207c682990 2024-04-17T07:24:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.593+0000 7f8622a00640 1 -- 172.21.15.86:0/1908943254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8628005d50 con 0x56207c682990 2024-04-17T07:24:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.593+0000 7f86425d2740 10 client.18057 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.593+0000 7f86425d2740 1 -- 172.21.15.86:0/1908943254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56207c5737f0 con 0x56207c682990 2024-04-17T07:24:58.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.594+0000 7f8622a00640 1 -- 172.21.15.86:0/1908943254 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8630019070 con 0x56207c682990 2024-04-17T07:24:58.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.594+0000 7f8622a00640 1 -- 172.21.15.86:0/1908943254 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f862803f730 con 0x56207c682990 2024-04-17T07:24:58.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.595+0000 7f86425d2740 10 client.18057 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:58.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.595+0000 7f86425d2740 10 client.18057 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:24:58.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.595+0000 7f86425d2740 20 client.18057 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:58.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.595+0000 7f86425d2740 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56207c698260 0x56207c69a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.595+0000 7f86425d2740 4 client.18057 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:58.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.595+0000 7f86425d2740 1 -- 172.21.15.86:0/1908943254 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56207c5f5a40 con 0x56207c698260 2024-04-17T07:24:58.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.595+0000 7f863ce00640 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56207c698260 0x56207c69a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.596+0000 7f8622a00640 10 client.18057 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:24:58.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.596+0000 7f863ce00640 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56207c698260 0x56207c69a700 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5c622aa37033631f server_cookie=6cc5797f7e43152c in_seq=0 out_seq=0 2024-04-17T07:24:58.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f8622a00640 1 -- 172.21.15.86:0/1908943254 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56207c5f5a40 con 0x56207c698260 2024-04-17T07:24:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f8622a00640 10 client.18057 handle_command_reply: tid=0 2024-04-17T07:24:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f86425d2740 1 client.18057 shutdown 2024-04-17T07:24:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f86425d2740 1 -- 172.21.15.86:0/1908943254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 msgr2=0x56207c68c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f86425d2740 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 0x56207c68c970 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8628009900 tx=0x7f8628005590 comp rx=0 tx=0).stop 2024-04-17T07:24:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f86425d2740 1 -- 172.21.15.86:0/1908943254 shutdown_connections 2024-04-17T07:24:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f86425d2740 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56207c698260 0x56207c69a700 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f86425d2740 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c687b00 0x56207c6939b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f86425d2740 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c682990 0x56207c68c970 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f86425d2740 1 --2- 172.21.15.86:0/1908943254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c681fa0 0x56207c68c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.598+0000 7f86425d2740 1 -- 172.21.15.86:0/1908943254 >> 172.21.15.86:0/1908943254 conn(0x56207c67d510 msgr2=0x56207c67c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.599+0000 7f8622a00640 0 client.18057 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:24:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.599+0000 7f86425d2740 1 -- 172.21.15.86:0/1908943254 shutdown_connections 2024-04-17T07:24:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.599+0000 7f86425d2740 1 -- 172.21.15.86:0/1908943254 wait complete. 2024-04-17T07:24:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.599+0000 7f86425d2740 20 client.18057 trim_cache size 0 max 16384 2024-04-17T07:24:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.634+0000 7f86425d2740 1 Processor -- start 2024-04-17T07:24:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.634+0000 7f86425d2740 1 -- start start 2024-04-17T07:24:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.635+0000 7f86425d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c681fa0 0x56207c683ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.635+0000 7f86425d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c693970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.635+0000 7f86425d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c687b00 0x56207c6843e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.635+0000 7f86425d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86381b99f0 con 0x56207c681fa0 2024-04-17T07:24:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.635+0000 7f86425d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f86381bb530 con 0x56207c682990 2024-04-17T07:24:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.635+0000 7f86425d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86381ba0c0 con 0x56207c687b00 2024-04-17T07:24:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.635+0000 7f863ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c681fa0 0x56207c683ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.636+0000 7f8637e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c693970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.636+0000 7f8637e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c693970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39298/0 (socket says 172.21.15.86:39298) 2024-04-17T07:24:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.636+0000 7f8637e00640 1 -- 172.21.15.86:0/2103448447 learned_addr learned my addr 172.21.15.86:0/2103448447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.636+0000 7f863d800640 1 --2- 172.21.15.86:0/2103448447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c687b00 0x56207c6843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.636+0000 7f8637e00640 1 -- 172.21.15.86:0/2103448447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c687b00 msgr2=0x56207c6843e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.636+0000 7f8637e00640 1 --2- 172.21.15.86:0/2103448447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c687b00 0x56207c6843e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.636+0000 7f8637e00640 1 -- 172.21.15.86:0/2103448447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c681fa0 msgr2=0x56207c683ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.636+0000 7f8637e00640 1 --2- 172.21.15.86:0/2103448447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c681fa0 0x56207c683ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.636+0000 7f8637e00640 1 -- 172.21.15.86:0/2103448447 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207c2fa3f0 con 0x56207c682990 2024-04-17T07:24:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.637+0000 7f8637e00640 1 --2- 172.21.15.86:0/2103448447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c693970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f862803fb80 tx=0x7f862802bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50c05f5048631833 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:58.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.637+0000 7f862d600640 1 -- 172.21.15.86:0/2103448447 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628043040 con 0x56207c682990 2024-04-17T07:24:58.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.638+0000 7f862d600640 1 -- 172.21.15.86:0/2103448447 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8628055070 con 0x56207c682990 2024-04-17T07:24:58.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.638+0000 7f862d600640 1 -- 172.21.15.86:0/2103448447 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628034a70 con 0x56207c682990 2024-04-17T07:24:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.639+0000 7f86425d2740 1 -- 172.21.15.86:0/2103448447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 msgr2=0x56207c693970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.639+0000 7f86425d2740 1 --2- 172.21.15.86:0/2103448447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c693970 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f862803fb80 tx=0x7f862802bc60 comp rx=0 tx=0).stop 2024-04-17T07:24:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.640+0000 7f86425d2740 1 -- 172.21.15.86:0/2103448447 shutdown_connections 2024-04-17T07:24:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.640+0000 7f86425d2740 1 --2- 172.21.15.86:0/2103448447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c687b00 0x56207c6843e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.640+0000 7f86425d2740 1 --2- 172.21.15.86:0/2103448447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c693970 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.640+0000 7f86425d2740 1 --2- 172.21.15.86:0/2103448447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c681fa0 0x56207c683ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.640+0000 7f86425d2740 1 -- 172.21.15.86:0/2103448447 >> 172.21.15.86:0/2103448447 conn(0x56207c67d510 msgr2=0x56207c6832f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.640+0000 7f86425d2740 1 -- 172.21.15.86:0/2103448447 shutdown_connections 2024-04-17T07:24:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.640+0000 7f86425d2740 1 -- 172.21.15.86:0/2103448447 wait complete. 2024-04-17T07:24:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.640+0000 7f86425d2740 1 -- start start 2024-04-17T07:24:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.641+0000 7f86425d2740 1 Processor -- start 2024-04-17T07:24:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.642+0000 7f86425d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c690dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.642+0000 7f86425d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c691300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.642+0000 7f86425d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c687b00 0x56207c692310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.643+0000 7f863ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c690dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.643+0000 7f8637e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c691300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.643+0000 7f8637e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c691300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39304/0 (socket says 172.21.15.86:39304) 2024-04-17T07:24:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.643+0000 7f863ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c690dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47796/0 (socket says 172.21.15.86:47796) 2024-04-17T07:24:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.644+0000 7f863d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c687b00 0x56207c692310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.644+0000 7f863d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c687b00 0x56207c692310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44648/0 (socket says 172.21.15.86:44648) 2024-04-17T07:24:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.644+0000 7f863d800640 1 -- 172.21.15.86:0/2586469528 learned_addr learned my addr 172.21.15.86:0/2586469528 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.644+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86381bae60 con 0x56207c687b00 2024-04-17T07:24:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.644+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f86381ba790 con 0x56207c682990 2024-04-17T07:24:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.644+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86381b79c0 con 0x56207c681fa0 2024-04-17T07:24:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.644+0000 7f86425d2740 10 client.0 _ll_register_callbacks cb 0x7f8628005430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.645+0000 7f8637e00640 1 -- 172.21.15.86:0/2586469528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 msgr2=0x56207c690dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.645+0000 7f8637e00640 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c690dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.645+0000 7f8637e00640 1 -- 172.21.15.86:0/2586469528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c687b00 msgr2=0x56207c692310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.645+0000 7f8637e00640 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c687b00 0x56207c692310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.645+0000 7f8637e00640 1 -- 172.21.15.86:0/2586469528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207c3c47d0 con 0x56207c682990 2024-04-17T07:24:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.645+0000 7f8637e00640 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c691300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86280057c0 tx=0x7f862803f8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e8df7757ab132 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.645+0000 7f8622a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:24:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.646+0000 7f8622a00640 1 -- 172.21.15.86:0/2586469528 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628043040 con 0x56207c682990 2024-04-17T07:24:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.646+0000 7f8622a00640 1 -- 172.21.15.86:0/2586469528 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8628034a00 con 0x56207c682990 2024-04-17T07:24:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.646+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8628055070 con 0x56207c682990 2024-04-17T07:24:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.647+0000 7f8622a00640 1 -- 172.21.15.86:0/2586469528 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8628009520 con 0x56207c682990 2024-04-17T07:24:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.647+0000 7f86425d2740 10 client.36953 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.647+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56207c693a30 con 0x56207c682990 2024-04-17T07:24:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.648+0000 7f8622a00640 1 -- 172.21.15.86:0/2586469528 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8628055070 con 0x56207c682990 2024-04-17T07:24:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.648+0000 7f8622a00640 1 -- 172.21.15.86:0/2586469528 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8628059450 con 0x56207c682990 2024-04-17T07:24:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.648+0000 7f86425d2740 10 client.36953 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.648+0000 7f86425d2740 10 client.36953 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:24:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.648+0000 7f86425d2740 20 client.36953 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.648+0000 7f86425d2740 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56207c6feb00 0x56207c700ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.648+0000 7f86425d2740 4 client.36953 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:24:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.648+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56207c5ef8b0 con 0x56207c6feb00 2024-04-17T07:24:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.650+0000 7f863ce00640 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56207c6feb00 0x56207c700ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.651+0000 7f863ce00640 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56207c6feb00 0x56207c700ef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=865f84ed38a408e2 server_cookie=8f258785153499d2 in_seq=0 out_seq=0 2024-04-17T07:24:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.651+0000 7f8622a00640 10 client.36953 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:24:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.651+0000 7f8622a00640 1 -- 172.21.15.86:0/2586469528 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56207c5ef8b0 con 0x56207c6feb00 2024-04-17T07:24:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.651+0000 7f8622a00640 10 client.36953 handle_command_reply: tid=0 2024-04-17T07:24:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.651+0000 7f86425d2740 1 client.36953 shutdown 2024-04-17T07:24:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.652+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 msgr2=0x56207c691300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.652+0000 7f86425d2740 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c691300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86280057c0 tx=0x7f862803f8e0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.652+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 shutdown_connections 2024-04-17T07:24:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.652+0000 7f86425d2740 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56207c6feb00 0x56207c700ef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.652+0000 7f86425d2740 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56207c687b00 0x56207c692310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.652+0000 7f86425d2740 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56207c682990 0x56207c691300 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86280057c0 tx=0x7f862803f8e0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.652+0000 7f86425d2740 1 --2- 172.21.15.86:0/2586469528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56207c681fa0 0x56207c690dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.652+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 >> 172.21.15.86:0/2586469528 conn(0x56207c67d510 msgr2=0x56207c68c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:58.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.652+0000 7f8622a00640 0 client.36953 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:24:58.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.653+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 shutdown_connections 2024-04-17T07:24:58.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.654+0000 7f86425d2740 1 -- 172.21.15.86:0/2586469528 wait complete. 2024-04-17T07:24:58.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.654+0000 7f86425d2740 20 client.36953 trim_cache size 0 max 16384 2024-04-17T07:24:58.655 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:24:58.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.656+0000 7f863f800640 1 -- 172.21.15.86:0/2281792504 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f861c0a3c30 msgr2=0x7f861c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.656+0000 7f863f800640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f861c0a3c30 0x7f861c0a60f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8630011570 tx=0x7f8630011970 comp rx=0 tx=0).stop 2024-04-17T07:24:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.656+0000 7f863f800640 1 -- 172.21.15.86:0/2281792504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638109f00 msgr2=0x7f86381ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:58.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.656+0000 7f863f800640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638109f00 0x7f86381ae590 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f862400e970 tx=0x7f862400ee30 comp rx=0 tx=0).stop 2024-04-17T07:24:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.657+0000 7f863f800640 1 -- 172.21.15.86:0/2281792504 shutdown_connections 2024-04-17T07:24:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.657+0000 7f863f800640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f861c0a3c30 0x7f861c0a60f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.657+0000 7f863f800640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8638109f00 0x7f86381ae590 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.657+0000 7f863f800640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8638109560 0x7f86381a7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.657+0000 7f863f800640 1 --2- 172.21.15.86:0/2281792504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8638109140 0x7f86381a6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.657+0000 7f863f800640 1 -- 172.21.15.86:0/2281792504 >> 172.21.15.86:0/2281792504 conn(0x7f863806f190 msgr2=0x7f8638114a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.657+0000 7f863f800640 1 -- 172.21.15.86:0/2281792504 shutdown_connections 2024-04-17T07:24:58.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:58.657+0000 7f863f800640 1 -- 172.21.15.86:0/2281792504 wait complete. 2024-04-17T07:24:58.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:24:58.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:24:58 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3112057556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:24:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:58 smithi098 ceph-mon[35418]: pgmap v758: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 649 op/s 2024-04-17T07:24:59.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:24:58 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3112057556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:24:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.011+0000 7f3995e00640 1 -- 172.21.15.86:0/1378544184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39901001d0 msgr2=0x7f39901005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.011+0000 7f3995e00640 1 --2- 172.21.15.86:0/1378544184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39901001d0 0x7f39901005f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f397c009900 tx=0x7f397c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:24:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.011+0000 7f3995e00640 1 -- 172.21.15.86:0/1378544184 shutdown_connections 2024-04-17T07:24:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.011+0000 7f3995e00640 1 --2- 172.21.15.86:0/1378544184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3990101d80 0x7f3990108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.011+0000 7f3995e00640 1 --2- 172.21.15.86:0/1378544184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39901013c0 0x7f3990101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.011+0000 7f3995e00640 1 --2- 172.21.15.86:0/1378544184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39901001d0 0x7f39901005f0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.011+0000 7f3995e00640 1 -- 172.21.15.86:0/1378544184 >> 172.21.15.86:0/1378544184 conn(0x7f39900fb760 msgr2=0x7f39900fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.013+0000 7f3995e00640 1 -- 172.21.15.86:0/1378544184 shutdown_connections 2024-04-17T07:24:59.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.013+0000 7f3995e00640 1 -- 172.21.15.86:0/1378544184 wait complete. 2024-04-17T07:24:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.015+0000 7f3995e00640 1 Processor -- start 2024-04-17T07:24:59.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.015+0000 7f3995e00640 1 -- start start 2024-04-17T07:24:59.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.016+0000 7f3995e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39901001d0 0x7f3990196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.016+0000 7f3995e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39901013c0 0x7f3990196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.016+0000 7f3995e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3990101d80 0x7f399019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.016+0000 7f3995e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f399010b910 con 0x7f39901001d0 2024-04-17T07:24:59.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.016+0000 7f3995e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f399010b790 con 0x7f3990101d80 2024-04-17T07:24:59.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.016+0000 7f3995e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f399010ba90 con 0x7f39901013c0 2024-04-17T07:24:59.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.017+0000 7f398ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39901001d0 0x7f3990196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.017+0000 7f398f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3990101d80 0x7f399019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.017+0000 7f398f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3990101d80 0x7f399019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39328/0 (socket says 172.21.15.86:39328) 2024-04-17T07:24:59.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.017+0000 7f398f400640 1 -- 172.21.15.86:0/499549184 learned_addr learned my addr 172.21.15.86:0/499549184 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.017+0000 7f398e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39901013c0 0x7f3990196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.019+0000 7f398e000640 1 -- 172.21.15.86:0/499549184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3990101d80 msgr2=0x7f399019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.019+0000 7f398e000640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3990101d80 0x7f399019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.019+0000 7f398e000640 1 -- 172.21.15.86:0/499549184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39901001d0 msgr2=0x7f3990196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.019+0000 7f398e000640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39901001d0 0x7f3990196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.019+0000 7f398e000640 1 -- 172.21.15.86:0/499549184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399019dc30 con 0x7f39901013c0 2024-04-17T07:24:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.019+0000 7f398e000640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39901013c0 0x7f3990196550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f398000c970 tx=0x7f398000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.020+0000 7f3994a00640 1 -- 172.21.15.86:0/499549184 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980007bb0 con 0x7f39901013c0 2024-04-17T07:24:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.020+0000 7f3994a00640 1 -- 172.21.15.86:0/499549184 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3980007d50 con 0x7f39901013c0 2024-04-17T07:24:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.020+0000 7f3994a00640 1 -- 172.21.15.86:0/499549184 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980004e40 con 0x7f39901013c0 2024-04-17T07:24:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.020+0000 7f3995e00640 1 -- 172.21.15.86:0/499549184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399019df20 con 0x7f39901013c0 2024-04-17T07:24:59.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.020+0000 7f3995e00640 1 -- 172.21.15.86:0/499549184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f399019e390 con 0x7f39901013c0 2024-04-17T07:24:59.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:58 smithi103 ceph-mon[35495]: pgmap v758: 129 pgs: 129 active+clean; 1.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 649 op/s 2024-04-17T07:24:59.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:24:58 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3112057556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:24:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.027+0000 7f3998bca740 1 Processor -- start 2024-04-17T07:24:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.028+0000 7f3998bca740 1 -- start start 2024-04-17T07:24:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.028+0000 7f3998bca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.028+0000 7f3998bca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 0x563e3d6f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.028+0000 7f3998bca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6f2310 0x563e3d6f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.028+0000 7f3998bca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e3d64a020 con 0x563e3d6ec7b0 2024-04-17T07:24:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.028+0000 7f3998bca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563e3d5f0460 con 0x563e3d6f2310 2024-04-17T07:24:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.028+0000 7f3998bca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e3d645fe0 con 0x563e3d6ed1a0 2024-04-17T07:24:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.029+0000 7f398ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 0x563e3d6f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.029+0000 7f398ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 0x563e3d6f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47820/0 (socket says 172.21.15.86:47820) 2024-04-17T07:24:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.029+0000 7f398f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.029+0000 7f398f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6ecbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44680/0 (socket says 172.21.15.86:44680) 2024-04-17T07:24:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.029+0000 7f398f400640 1 -- 172.21.15.86:0/754522328 learned_addr learned my addr 172.21.15.86:0/754522328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.030+0000 7f398f400640 1 -- 172.21.15.86:0/754522328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 msgr2=0x563e3d6f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.030+0000 7f398f400640 1 --2- 172.21.15.86:0/754522328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 0x563e3d6f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.030+0000 7f398f400640 1 -- 172.21.15.86:0/754522328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6f2310 msgr2=0x563e3d6f47a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:24:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.030+0000 7f398f400640 1 --2- 172.21.15.86:0/754522328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6f2310 0x563e3d6f47a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.030+0000 7f398f400640 1 -- 172.21.15.86:0/754522328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3d5f8040 con 0x563e3d6ec7b0 2024-04-17T07:24:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.031+0000 7f398f400640 1 --2- 172.21.15.86:0/754522328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6ecbd0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3988011570 tx=0x7f3988011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2226351dec3a195a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.031+0000 7f397be00640 1 -- 172.21.15.86:0/754522328 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988004b20 con 0x563e3d6ec7b0 2024-04-17T07:24:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.032+0000 7f397be00640 1 -- 172.21.15.86:0/754522328 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3988004cc0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.032+0000 7f397be00640 1 -- 172.21.15.86:0/754522328 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988005480 con 0x563e3d6ec7b0 2024-04-17T07:24:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.033+0000 7f3998bca740 1 -- 172.21.15.86:0/754522328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 msgr2=0x563e3d6ecbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.033+0000 7f3994a00640 1 -- 172.21.15.86:0/499549184 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3980005000 con 0x7f39901013c0 2024-04-17T07:24:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.033+0000 7f3998bca740 1 --2- 172.21.15.86:0/754522328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6ecbd0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3988011570 tx=0x7f3988011a30 comp rx=0 tx=0).stop 2024-04-17T07:24:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.033+0000 7f3998bca740 1 -- 172.21.15.86:0/754522328 shutdown_connections 2024-04-17T07:24:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.033+0000 7f3998bca740 1 --2- 172.21.15.86:0/754522328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6f2310 0x563e3d6f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.033+0000 7f3998bca740 1 --2- 172.21.15.86:0/754522328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 0x563e3d6f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.033+0000 7f3998bca740 1 --2- 172.21.15.86:0/754522328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6ecbd0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.033+0000 7f3998bca740 1 -- 172.21.15.86:0/754522328 >> 172.21.15.86:0/754522328 conn(0x563e3d6df510 msgr2=0x563e3d6df910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.035+0000 7f3998bca740 1 -- 172.21.15.86:0/754522328 shutdown_connections 2024-04-17T07:24:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.035+0000 7f3998bca740 1 -- 172.21.15.86:0/754522328 wait complete. 2024-04-17T07:24:59.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.036+0000 7f3998bca740 1 -- start start 2024-04-17T07:24:59.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.036+0000 7f3998bca740 1 Processor -- start 2024-04-17T07:24:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.037+0000 7f3994a00640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39740a3bb0 0x7f39740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.037+0000 7f3994a00640 1 -- 172.21.15.86:0/499549184 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f39800d02f0 con 0x7f39901013c0 2024-04-17T07:24:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.037+0000 7f3998bca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.037+0000 7f3998bca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 0x563e3d6f7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.037+0000 7f398f400640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39740a3bb0 0x7f39740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.037+0000 7f3998bca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6f2310 0x563e3d70f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.037+0000 7f3998bca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e3d6f52c0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.037+0000 7f3998bca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563e3d603830 con 0x563e3d6f2310 2024-04-17T07:24:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.037+0000 7f3998bca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563e3d6f5440 con 0x563e3d6ed1a0 2024-04-17T07:24:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.038+0000 7f3998bca740 10 client.0 _ll_register_callbacks cb 0x563e3d5d1fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.038+0000 7f398ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.038+0000 7f398ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6f6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44688/0 (socket says 172.21.15.86:44688) 2024-04-17T07:24:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.038+0000 7f398ea00640 1 -- 172.21.15.86:0/2567817762 learned_addr learned my addr 172.21.15.86:0/2567817762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.039+0000 7f398e000640 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 0x563e3d6f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.039+0000 7f398f400640 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6f2310 0x563e3d70f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.041+0000 7f398ea00640 1 -- 172.21.15.86:0/2567817762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 msgr2=0x563e3d6f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.041+0000 7f398ea00640 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 0x563e3d6f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.041+0000 7f398ea00640 1 -- 172.21.15.86:0/2567817762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6f2310 msgr2=0x563e3d70f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.041+0000 7f398ea00640 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6f2310 0x563e3d70f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.041+0000 7f398ea00640 1 -- 172.21.15.86:0/2567817762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3d607920 con 0x563e3d6ec7b0 2024-04-17T07:24:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.042+0000 7f398f400640 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6f2310 0x563e3d70f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:24:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.042+0000 7f398ea00640 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6f6d00 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f397c005ab0 tx=0x7f397c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca8096185636ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.043+0000 7f398f400640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39740a3bb0 0x7f39740a6070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f398801b040 tx=0x7f3988004ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.043+0000 7f3978c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:24:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.045+0000 7f3978c00640 1 -- 172.21.15.86:0/2567817762 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c005250 con 0x563e3d6ec7b0 2024-04-17T07:24:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.045+0000 7f3978c00640 1 -- 172.21.15.86:0/2567817762 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f397c0053f0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.045+0000 7f3998bca740 1 -- 172.21.15.86:0/2567817762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3988004cc0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.046+0000 7f3978c00640 1 -- 172.21.15.86:0/2567817762 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f397c0055f0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.046+0000 7f3998bca740 10 client.18075 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.046+0000 7f3998bca740 1 -- 172.21.15.86:0/2567817762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e3d5d57f0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.046+0000 7f3978c00640 1 -- 172.21.15.86:0/2567817762 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3988004cc0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.049+0000 7f3978c00640 1 -- 172.21.15.86:0/2567817762 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f397c004c90 con 0x563e3d6ec7b0 2024-04-17T07:24:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.049+0000 7f3998bca740 10 client.18075 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.049+0000 7f3998bca740 10 client.18075 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:24:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.049+0000 7f3998bca740 20 client.18075 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.049+0000 7f3998bca740 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563e3d7137e0 0x563e3d715c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.049+0000 7f3998bca740 4 client.18075 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.049+0000 7f3998bca740 1 -- 172.21.15.86:0/2567817762 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e3d657a40 con 0x563e3d7137e0 2024-04-17T07:24:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.050+0000 7f398e000640 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563e3d7137e0 0x563e3d715c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.051+0000 7f398e000640 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563e3d7137e0 0x563e3d715c10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1bbb1588c04e44b server_cookie=42d45ace7bdb3657 in_seq=0 out_seq=0 2024-04-17T07:24:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.051+0000 7f3978c00640 10 client.18075 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:24:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.053+0000 7f3978c00640 1 -- 172.21.15.86:0/2567817762 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e3d657a40 con 0x563e3d7137e0 2024-04-17T07:24:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.053+0000 7f3978c00640 10 client.18075 handle_command_reply: tid=0 2024-04-17T07:24:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.054+0000 7f3998bca740 1 client.18075 shutdown 2024-04-17T07:24:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.055+0000 7f3998bca740 1 -- 172.21.15.86:0/2567817762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 msgr2=0x563e3d6f6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.055+0000 7f3998bca740 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6f6d00 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f397c005ab0 tx=0x7f397c004270 comp rx=0 tx=0).stop 2024-04-17T07:24:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.057+0000 7f3998bca740 1 -- 172.21.15.86:0/2567817762 shutdown_connections 2024-04-17T07:24:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.057+0000 7f3998bca740 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563e3d7137e0 0x563e3d715c10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.057+0000 7f3998bca740 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6f2310 0x563e3d70f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.057+0000 7f3998bca740 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6ed1a0 0x563e3d6f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.057+0000 7f3998bca740 1 --2- 172.21.15.86:0/2567817762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6f6d00 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.057+0000 7f3978c00640 0 client.18075 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:24:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.057+0000 7f3998bca740 1 -- 172.21.15.86:0/2567817762 >> 172.21.15.86:0/2567817762 conn(0x563e3d6df510 msgr2=0x563e3d6de710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.057+0000 7f3998bca740 1 -- 172.21.15.86:0/2567817762 shutdown_connections 2024-04-17T07:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.057+0000 7f3998bca740 1 -- 172.21.15.86:0/2567817762 wait complete. 2024-04-17T07:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.057+0000 7f3998bca740 20 client.18075 trim_cache size 0 max 16384 2024-04-17T07:24:59.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.096+0000 7f3998bca740 1 Processor -- start 2024-04-17T07:24:59.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.097+0000 7f3998bca740 1 -- start start 2024-04-17T07:24:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.097+0000 7f3998bca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.097+0000 7f3998bca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ed1a0 0x563e3d71a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.097+0000 7f3998bca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 0x563e3d71a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.097+0000 7f3998bca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39901a00d0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.097+0000 7f3998bca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39901a1c10 con 0x563e3d6ed1a0 2024-04-17T07:24:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.098+0000 7f3998bca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f39901a07a0 con 0x563e3d6f2310 2024-04-17T07:24:59.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.098+0000 7f398ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.098+0000 7f398f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 0x563e3d71a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.099+0000 7f398ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6e2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44694/0 (socket says 172.21.15.86:44694) 2024-04-17T07:24:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.099+0000 7f398ea00640 1 -- 172.21.15.86:0/1751626486 learned_addr learned my addr 172.21.15.86:0/1751626486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.098+0000 7f398e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ed1a0 0x563e3d71a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.099+0000 7f398ea00640 1 -- 172.21.15.86:0/1751626486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 msgr2=0x563e3d71a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.099+0000 7f398ea00640 1 --2- 172.21.15.86:0/1751626486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 0x563e3d71a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.099+0000 7f398ea00640 1 -- 172.21.15.86:0/1751626486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ed1a0 msgr2=0x563e3d71a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.099+0000 7f398ea00640 1 --2- 172.21.15.86:0/1751626486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ed1a0 0x563e3d71a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.099+0000 7f398ea00640 1 -- 172.21.15.86:0/1751626486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3d35c3f0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.101+0000 7f398ea00640 1 --2- 172.21.15.86:0/1751626486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6e2e80 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f397c004520 tx=0x7f397c009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af05583ad3b472c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.102+0000 7f397b400640 1 -- 172.21.15.86:0/1751626486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c005250 con 0x563e3d6ec7b0 2024-04-17T07:24:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.102+0000 7f397b400640 1 -- 172.21.15.86:0/1751626486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f397c0053f0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.103+0000 7f397b400640 1 -- 172.21.15.86:0/1751626486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c034440 con 0x563e3d6ec7b0 2024-04-17T07:24:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.104+0000 7f3998bca740 1 -- 172.21.15.86:0/1751626486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 msgr2=0x563e3d6e2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.104+0000 7f3998bca740 1 --2- 172.21.15.86:0/1751626486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6e2e80 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f397c004520 tx=0x7f397c009230 comp rx=0 tx=0).stop 2024-04-17T07:24:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.104+0000 7f3998bca740 1 -- 172.21.15.86:0/1751626486 shutdown_connections 2024-04-17T07:24:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.105+0000 7f3998bca740 1 --2- 172.21.15.86:0/1751626486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 0x563e3d71a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.105+0000 7f3998bca740 1 --2- 172.21.15.86:0/1751626486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ed1a0 0x563e3d71a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.105+0000 7f3998bca740 1 --2- 172.21.15.86:0/1751626486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ec7b0 0x563e3d6e2e80 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.105+0000 7f3998bca740 1 -- 172.21.15.86:0/1751626486 >> 172.21.15.86:0/1751626486 conn(0x563e3d6df510 msgr2=0x563e3d6e2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.105+0000 7f3998bca740 1 -- 172.21.15.86:0/1751626486 shutdown_connections 2024-04-17T07:24:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.106+0000 7f3998bca740 1 -- 172.21.15.86:0/1751626486 wait complete. 2024-04-17T07:24:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.106+0000 7f3998bca740 1 -- start start 2024-04-17T07:24:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.106+0000 7f3998bca740 1 Processor -- start 2024-04-17T07:24:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.108+0000 7f3998bca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ec7b0 0x563e3d722350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.108+0000 7f3998bca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ed1a0 0x563e3d722890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.108+0000 7f3998bca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 0x563e3d7258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.109+0000 7f398ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ec7b0 0x563e3d722350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.109+0000 7f398ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ec7b0 0x563e3d722350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39346/0 (socket says 172.21.15.86:39346) 2024-04-17T07:24:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.109+0000 7f398f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 0x563e3d7258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.109+0000 7f398f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 0x563e3d7258b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47854/0 (socket says 172.21.15.86:47854) 2024-04-17T07:24:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.109+0000 7f398f400640 1 -- 172.21.15.86:0/3738574357 learned_addr learned my addr 172.21.15.86:0/3738574357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.109+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39901a1540 con 0x563e3d6ed1a0 2024-04-17T07:24:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.110+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39901a0e70 con 0x563e3d6ec7b0 2024-04-17T07:24:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.110+0000 7f398e000640 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ed1a0 0x563e3d722890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.110+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3990073c10 con 0x563e3d6f2310 2024-04-17T07:24:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.112+0000 7f3998bca740 10 client.0 _ll_register_callbacks cb 0x7f397c02cc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.112+0000 7f398ea00640 1 -- 172.21.15.86:0/3738574357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 msgr2=0x563e3d7258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.112+0000 7f398ea00640 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 0x563e3d7258b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.112+0000 7f398ea00640 1 -- 172.21.15.86:0/3738574357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ed1a0 msgr2=0x563e3d722890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.112+0000 7f398ea00640 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ed1a0 0x563e3d722890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.112+0000 7f398ea00640 1 -- 172.21.15.86:0/3738574357 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3d4267d0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.113+0000 7f398ea00640 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ec7b0 0x563e3d722350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x563e3d6df4e0 tx=0x7f397c004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51f21e5a95bf9ff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.113+0000 7f3978c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:24:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.114+0000 7f3978c00640 1 -- 172.21.15.86:0/3738574357 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c005250 con 0x563e3d6ec7b0 2024-04-17T07:24:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.114+0000 7f3978c00640 1 -- 172.21.15.86:0/3738574357 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f397c0037f0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.114+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f397c0053f0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.115+0000 7f3978c00640 1 -- 172.21.15.86:0/3738574357 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f397c004e60 con 0x563e3d6ec7b0 2024-04-17T07:24:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.115+0000 7f3998bca740 10 client.36962 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.115+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e3d726fb0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.116+0000 7f3978c00640 1 -- 172.21.15.86:0/3738574357 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f397c0053f0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.117+0000 7f3978c00640 1 -- 172.21.15.86:0/3738574357 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f397c0344d0 con 0x563e3d6ec7b0 2024-04-17T07:24:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.117+0000 7f3998bca740 10 client.36962 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.117+0000 7f3998bca740 10 client.36962 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:24:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.117+0000 7f3998bca740 20 client.36962 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.117+0000 7f3998bca740 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563e3d6e7fb0 0x563e3d6ea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.117+0000 7f3998bca740 4 client.36962 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:24:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.117+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e3d6518b0 con 0x563e3d6e7fb0 2024-04-17T07:24:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.119+0000 7f398e000640 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563e3d6e7fb0 0x563e3d6ea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.120+0000 7f398e000640 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563e3d6e7fb0 0x563e3d6ea3d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=269e885ac99a6492 server_cookie=208fbab0f614ed37 in_seq=0 out_seq=0 2024-04-17T07:24:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.120+0000 7f3978c00640 10 client.36962 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:24:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.124+0000 7f3978c00640 1 -- 172.21.15.86:0/3738574357 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e3d6518b0 con 0x563e3d6e7fb0 2024-04-17T07:24:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.124+0000 7f3978c00640 10 client.36962 handle_command_reply: tid=0 2024-04-17T07:24:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.124+0000 7f3998bca740 1 client.36962 shutdown 2024-04-17T07:24:59.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.125+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ec7b0 msgr2=0x563e3d722350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.125+0000 7f3998bca740 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ec7b0 0x563e3d722350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x563e3d6df4e0 tx=0x7f397c004a30 comp rx=0 tx=0).stop 2024-04-17T07:24:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.125+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 shutdown_connections 2024-04-17T07:24:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.126+0000 7f3998bca740 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563e3d6e7fb0 0x563e3d6ea3d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.126+0000 7f3998bca740 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563e3d6f2310 0x563e3d7258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.126+0000 7f3978c00640 0 client.36962 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:24:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.126+0000 7f3998bca740 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e3d6ed1a0 0x563e3d722890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.126+0000 7f3998bca740 1 --2- 172.21.15.86:0/3738574357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563e3d6ec7b0 0x563e3d722350 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x563e3d6df4e0 tx=0x7f397c004a30 comp rx=0 tx=0).stop 2024-04-17T07:24:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.126+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 >> 172.21.15.86:0/3738574357 conn(0x563e3d6df510 msgr2=0x563e3d6e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.127+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 shutdown_connections 2024-04-17T07:24:59.128 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:24:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.128+0000 7f3998bca740 1 -- 172.21.15.86:0/3738574357 wait complete. 2024-04-17T07:24:59.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.128+0000 7f3998bca740 20 client.36962 trim_cache size 0 max 16384 2024-04-17T07:24:59.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.129+0000 7f3995e00640 1 -- 172.21.15.86:0/499549184 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39740a3bb0 msgr2=0x7f39740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.129+0000 7f3995e00640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39740a3bb0 0x7f39740a6070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f398801b040 tx=0x7f3988004ef0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.130+0000 7f3995e00640 1 -- 172.21.15.86:0/499549184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39901013c0 msgr2=0x7f3990196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.130+0000 7f3995e00640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39901013c0 0x7f3990196550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f398000c970 tx=0x7f398000ce30 comp rx=0 tx=0).stop 2024-04-17T07:24:59.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.130+0000 7f3995e00640 1 -- 172.21.15.86:0/499549184 shutdown_connections 2024-04-17T07:24:59.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.130+0000 7f3995e00640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f39740a3bb0 0x7f39740a6070 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.130+0000 7f3995e00640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3990101d80 0x7f399019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.130+0000 7f3995e00640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39901013c0 0x7f3990196550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.130+0000 7f3995e00640 1 --2- 172.21.15.86:0/499549184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39901001d0 0x7f3990196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.130+0000 7f3995e00640 1 -- 172.21.15.86:0/499549184 >> 172.21.15.86:0/499549184 conn(0x7f39900fb760 msgr2=0x7f39901047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.132+0000 7f3995e00640 1 -- 172.21.15.86:0/499549184 shutdown_connections 2024-04-17T07:24:59.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.133+0000 7f3995e00640 1 -- 172.21.15.86:0/499549184 wait complete. 2024-04-17T07:24:59.160 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:24:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 335.0 s, 626.4 tps, lat 159.497 ms stddev 50.778 2024-04-17T07:24:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.502+0000 7f9655800640 1 -- 172.21.15.86:0/4042584976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9650075fa0 msgr2=0x7f96500743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.502+0000 7f9655800640 1 --2- 172.21.15.86:0/4042584976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9650075fa0 0x7f96500743f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f9640009930 tx=0x7f964002bc30 comp rx=0 tx=0).stop 2024-04-17T07:24:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.503+0000 7f9655800640 1 -- 172.21.15.86:0/4042584976 shutdown_connections 2024-04-17T07:24:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.503+0000 7f9655800640 1 --2- 172.21.15.86:0/4042584976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f965010e440 0x7f9650110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.503+0000 7f9655800640 1 --2- 172.21.15.86:0/4042584976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96500749c0 0x7f9650074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.503+0000 7f9655800640 1 --2- 172.21.15.86:0/4042584976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9650075fa0 0x7f96500743f0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.503+0000 7f9655800640 1 -- 172.21.15.86:0/4042584976 >> 172.21.15.86:0/4042584976 conn(0x7f96500fd400 msgr2=0x7f96500ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.504+0000 7f9655800640 1 -- 172.21.15.86:0/4042584976 shutdown_connections 2024-04-17T07:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.504+0000 7f9655800640 1 -- 172.21.15.86:0/4042584976 wait complete. 2024-04-17T07:24:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.507+0000 7f9655800640 1 Processor -- start 2024-04-17T07:24:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.508+0000 7f9655800640 1 -- start start 2024-04-17T07:24:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.509+0000 7f9655800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96500749c0 0x7f965019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.509+0000 7f9655800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9650075fa0 0x7f965019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.509+0000 7f9655800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f965010e440 0x7f96501a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.509+0000 7f9655800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9650113fc0 con 0x7f9650075fa0 2024-04-17T07:24:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.510+0000 7f9654e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f965010e440 0x7f96501a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.510+0000 7f964f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9650075fa0 0x7f965019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.510+0000 7f9654e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f965010e440 0x7f96501a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39366/0 (socket says 172.21.15.86:39366) 2024-04-17T07:24:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.510+0000 7f9654e00640 1 -- 172.21.15.86:0/481072419 learned_addr learned my addr 172.21.15.86:0/481072419 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.510+0000 7f964fe00640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96500749c0 0x7f965019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.510+0000 7f9655800640 1 -- 172.21.15.86:0/481072419 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9650113e40 con 0x7f965010e440 2024-04-17T07:24:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.510+0000 7f9655800640 1 -- 172.21.15.86:0/481072419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9650114140 con 0x7f96500749c0 2024-04-17T07:24:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.513+0000 7f9654e00640 1 -- 172.21.15.86:0/481072419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96500749c0 msgr2=0x7f965019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.513+0000 7f9654e00640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96500749c0 0x7f965019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.513+0000 7f9654e00640 1 -- 172.21.15.86:0/481072419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9650075fa0 msgr2=0x7f965019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.513+0000 7f9654e00640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9650075fa0 0x7f965019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.513+0000 7f9654e00640 1 -- 172.21.15.86:0/481072419 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9650105d60 con 0x7f965010e440 2024-04-17T07:24:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.513+0000 7f964fe00640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96500749c0 0x7f965019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:24:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.514+0000 7f9654e00640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f965010e440 0x7f96501a5c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f964800eaa0 tx=0x7f964800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.515+0000 7f964cc00640 1 -- 172.21.15.86:0/481072419 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964800cdb0 con 0x7f965010e440 2024-04-17T07:24:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.515+0000 7f964cc00640 1 -- 172.21.15.86:0/481072419 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9648005020 con 0x7f965010e440 2024-04-17T07:24:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.515+0000 7f964cc00640 1 -- 172.21.15.86:0/481072419 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96480116d0 con 0x7f965010e440 2024-04-17T07:24:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.516+0000 7f9655800640 1 -- 172.21.15.86:0/481072419 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9650106050 con 0x7f965010e440 2024-04-17T07:24:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.516+0000 7f9655800640 1 -- 172.21.15.86:0/481072419 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9650106550 con 0x7f965010e440 2024-04-17T07:24:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.521+0000 7f96599e2740 1 Processor -- start 2024-04-17T07:24:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.521+0000 7f96599e2740 1 -- start start 2024-04-17T07:24:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.521+0000 7f96599e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f273b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.522+0000 7f96599e2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273bb50 0x5575f2740650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.522+0000 7f96599e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f2743150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.522+0000 7f96599e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5575f26a1020 con 0x5575f2740cc0 2024-04-17T07:24:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.522+0000 7f96599e2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5575f2647460 con 0x5575f273bb50 2024-04-17T07:24:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.522+0000 7f96599e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5575f269cfe0 con 0x5575f273b160 2024-04-17T07:24:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.523+0000 7f964fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273bb50 0x5575f2740650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.523+0000 7f964f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f273b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.523+0000 7f964f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f273b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47874/0 (socket says 172.21.15.86:47874) 2024-04-17T07:24:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.523+0000 7f964f400640 1 -- 172.21.15.86:0/413680718 learned_addr learned my addr 172.21.15.86:0/413680718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.523+0000 7f964f400640 1 --2- 172.21.15.86:0/413680718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f2743150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.524+0000 7f964f400640 1 -- 172.21.15.86:0/413680718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273bb50 msgr2=0x5575f2740650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.524+0000 7f964f400640 1 --2- 172.21.15.86:0/413680718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273bb50 0x5575f2740650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.524+0000 7f964f400640 1 -- 172.21.15.86:0/413680718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 msgr2=0x5575f2743150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.524+0000 7f964f400640 1 --2- 172.21.15.86:0/413680718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f2743150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.524+0000 7f964f400640 1 -- 172.21.15.86:0/413680718 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f264f040 con 0x5575f273b160 2024-04-17T07:24:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.524+0000 7f964cc00640 1 -- 172.21.15.86:0/481072419 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f96480051d0 con 0x7f965010e440 2024-04-17T07:24:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.525+0000 7f964f400640 1 --2- 172.21.15.86:0/413680718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f273b580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f963c011510 tx=0x7f963c00a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=882822e665786156 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.525+0000 7f9645600640 1 -- 172.21.15.86:0/413680718 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c005220 con 0x5575f273b160 2024-04-17T07:24:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.525+0000 7f9645600640 1 -- 172.21.15.86:0/413680718 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f963c004c50 con 0x5575f273b160 2024-04-17T07:24:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.525+0000 7f9645600640 1 -- 172.21.15.86:0/413680718 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c00fb20 con 0x5575f273b160 2024-04-17T07:24:59.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.526+0000 7f964cc00640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96340a3d00 0x7f96340a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.527+0000 7f964cc00640 1 -- 172.21.15.86:0/481072419 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f96480d0930 con 0x7f965010e440 2024-04-17T07:24:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.527+0000 7f96599e2740 1 -- 172.21.15.86:0/413680718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 msgr2=0x5575f273b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.527+0000 7f96599e2740 1 --2- 172.21.15.86:0/413680718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f273b580 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f963c011510 tx=0x7f963c00a750 comp rx=0 tx=0).stop 2024-04-17T07:24:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.527+0000 7f96599e2740 1 -- 172.21.15.86:0/413680718 shutdown_connections 2024-04-17T07:24:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.527+0000 7f96599e2740 1 --2- 172.21.15.86:0/413680718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f2743150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.527+0000 7f96599e2740 1 --2- 172.21.15.86:0/413680718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273bb50 0x5575f2740650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.527+0000 7f96599e2740 1 --2- 172.21.15.86:0/413680718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f273b580 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.527+0000 7f96599e2740 1 -- 172.21.15.86:0/413680718 >> 172.21.15.86:0/413680718 conn(0x5575f27366d0 msgr2=0x5575f2736ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.528+0000 7f96599e2740 1 -- 172.21.15.86:0/413680718 shutdown_connections 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.528+0000 7f96599e2740 1 -- 172.21.15.86:0/413680718 wait complete. 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.528+0000 7f96599e2740 1 -- start start 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.529+0000 7f96599e2740 1 Processor -- start 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.529+0000 7f96599e2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273b160 0x5575f27567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.529+0000 7f96599e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f273bb50 0x5575f2756ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.530+0000 7f964fe00640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96340a3d00 0x7f96340a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.530+0000 7f96599e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f2740cc0 0x5575f275dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.530+0000 7f964fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273b160 0x5575f27567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.530+0000 7f96599e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5575f2754c80 con 0x5575f273bb50 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.530+0000 7f96599e2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5575f265a830 con 0x5575f273b160 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.530+0000 7f96599e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5575f2754e00 con 0x5575f2740cc0 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.530+0000 7f96599e2740 10 client.0 _ll_register_callbacks cb 0x5575f2628fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.530+0000 7f964fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273b160 0x5575f27567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39376/0 (socket says 172.21.15.86:39376) 2024-04-17T07:24:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.530+0000 7f964fe00640 1 -- 172.21.15.86:0/1501202833 learned_addr learned my addr 172.21.15.86:0/1501202833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.530+0000 7f9654e00640 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f2740cc0 0x5575f275dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.531+0000 7f964f400640 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f273bb50 0x5575f2756ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.531+0000 7f9654e00640 1 -- 172.21.15.86:0/1501202833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273b160 msgr2=0x5575f27567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.531+0000 7f9654e00640 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273b160 0x5575f27567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.531+0000 7f9654e00640 1 -- 172.21.15.86:0/1501202833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f273bb50 msgr2=0x5575f2756ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.532+0000 7f9654e00640 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f273bb50 0x5575f2756ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.532+0000 7f9654e00640 1 -- 172.21.15.86:0/1501202833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f265e920 con 0x5575f2740cc0 2024-04-17T07:24:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.532+0000 7f9654e00640 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f2740cc0 0x5575f275dcb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96480ce050 tx=0x7f964809d070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cf0e9e5055520b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.532+0000 7f963a000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.533+0000 7f963a000640 1 -- 172.21.15.86:0/1501202833 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964801f040 con 0x5575f2740cc0 2024-04-17T07:24:59.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.533+0000 7f96599e2740 1 -- 172.21.15.86:0/1501202833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f963c004c50 con 0x5575f2740cc0 2024-04-17T07:24:59.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.533+0000 7f963a000640 1 -- 172.21.15.86:0/1501202833 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f964809ca90 con 0x5575f2740cc0 2024-04-17T07:24:59.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.533+0000 7f963a000640 1 -- 172.21.15.86:0/1501202833 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f964809cc90 con 0x5575f2740cc0 2024-04-17T07:24:59.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.533+0000 7f964fe00640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96340a3d00 0x7f96340a61c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f964002ce10 tx=0x7f9640000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.534+0000 7f96599e2740 10 client.27025 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:59.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.534+0000 7f96599e2740 1 -- 172.21.15.86:0/1501202833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5575f262c7f0 con 0x5575f2740cc0 2024-04-17T07:24:59.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.534+0000 7f963a000640 1 -- 172.21.15.86:0/1501202833 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f963c004c50 con 0x5575f2740cc0 2024-04-17T07:24:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.534+0000 7f963a000640 1 -- 172.21.15.86:0/1501202833 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f964800f040 con 0x5575f2740cc0 2024-04-17T07:24:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.534+0000 7f96599e2740 10 client.27025 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.534+0000 7f96599e2740 10 client.27025 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:24:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.535+0000 7f96599e2740 20 client.27025 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.535+0000 7f96599e2740 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5575f2762250 0x5575f2764640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.535+0000 7f96599e2740 4 client.27025 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:24:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.535+0000 7f96599e2740 1 -- 172.21.15.86:0/1501202833 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575f26aea40 con 0x5575f2762250 2024-04-17T07:24:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.537+0000 7f964fe00640 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5575f2762250 0x5575f2764640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.538+0000 7f964fe00640 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5575f2762250 0x5575f2764640 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2754b135b561681 server_cookie=bdf65ec8fbbb1125 in_seq=0 out_seq=0 2024-04-17T07:24:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.538+0000 7f963a000640 10 client.27025 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:24:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.540+0000 7f963a000640 1 -- 172.21.15.86:0/1501202833 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575f26aea40 con 0x5575f2762250 2024-04-17T07:24:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.540+0000 7f963a000640 10 client.27025 handle_command_reply: tid=0 2024-04-17T07:24:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.540+0000 7f96599e2740 1 client.27025 shutdown 2024-04-17T07:24:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.541+0000 7f96599e2740 1 -- 172.21.15.86:0/1501202833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f2740cc0 msgr2=0x5575f275dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.541+0000 7f96599e2740 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f2740cc0 0x5575f275dcb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96480ce050 tx=0x7f964809d070 comp rx=0 tx=0).stop 2024-04-17T07:24:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.541+0000 7f96599e2740 1 -- 172.21.15.86:0/1501202833 shutdown_connections 2024-04-17T07:24:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.541+0000 7f96599e2740 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5575f2762250 0x5575f2764640 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.541+0000 7f963a000640 0 client.27025 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:24:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.541+0000 7f96599e2740 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f2740cc0 0x5575f275dcb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.541+0000 7f96599e2740 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f273bb50 0x5575f2756ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.541+0000 7f96599e2740 1 --2- 172.21.15.86:0/1501202833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273b160 0x5575f27567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.541+0000 7f96599e2740 1 -- 172.21.15.86:0/1501202833 >> 172.21.15.86:0/1501202833 conn(0x5575f27366d0 msgr2=0x5575f2735930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.542+0000 7f96599e2740 1 -- 172.21.15.86:0/1501202833 shutdown_connections 2024-04-17T07:24:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.542+0000 7f96599e2740 1 -- 172.21.15.86:0/1501202833 wait complete. 2024-04-17T07:24:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.543+0000 7f96599e2740 20 client.27025 trim_cache size 0 max 16384 2024-04-17T07:24:59.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.584+0000 7f96599e2740 1 Processor -- start 2024-04-17T07:24:59.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.585+0000 7f96599e2740 1 -- start start 2024-04-17T07:24:59.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.585+0000 7f96599e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f2736d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.585+0000 7f96599e2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273bb50 0x5575f27350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.585+0000 7f96599e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f27355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.585+0000 7f96599e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f965006df60 con 0x5575f2740cc0 2024-04-17T07:24:59.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.585+0000 7f96599e2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f965006faa0 con 0x5575f273bb50 2024-04-17T07:24:59.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.585+0000 7f96599e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f965006e630 con 0x5575f273b160 2024-04-17T07:24:59.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.586+0000 7f964fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f2736d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.586+0000 7f964fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f2736d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47900/0 (socket says 172.21.15.86:47900) 2024-04-17T07:24:59.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.586+0000 7f964fe00640 1 -- 172.21.15.86:0/4241241190 learned_addr learned my addr 172.21.15.86:0/4241241190 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.586+0000 7f964fe00640 1 -- 172.21.15.86:0/4241241190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273bb50 msgr2=0x5575f27350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.586+0000 7f964fe00640 1 --2- 172.21.15.86:0/4241241190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273bb50 0x5575f27350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.586+0000 7f964fe00640 1 -- 172.21.15.86:0/4241241190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 msgr2=0x5575f27355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.586+0000 7f964fe00640 1 --2- 172.21.15.86:0/4241241190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f27355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.586+0000 7f964fe00640 1 -- 172.21.15.86:0/4241241190 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f23b33f0 con 0x5575f273b160 2024-04-17T07:24:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.587+0000 7f964fe00640 1 --2- 172.21.15.86:0/4241241190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f2736d00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9640009230 tx=0x7f9640005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c3d75c8bd0a5622 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.588+0000 7f9644c00640 1 -- 172.21.15.86:0/4241241190 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96400044e0 con 0x5575f273b160 2024-04-17T07:24:59.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.588+0000 7f9644c00640 1 -- 172.21.15.86:0/4241241190 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9640058030 con 0x5575f273b160 2024-04-17T07:24:59.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.588+0000 7f9644c00640 1 -- 172.21.15.86:0/4241241190 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640004e60 con 0x5575f273b160 2024-04-17T07:24:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.589+0000 7f96599e2740 1 -- 172.21.15.86:0/4241241190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 msgr2=0x5575f2736d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.589+0000 7f96599e2740 1 --2- 172.21.15.86:0/4241241190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f2736d00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9640009230 tx=0x7f9640005fd0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.589+0000 7f96599e2740 1 -- 172.21.15.86:0/4241241190 shutdown_connections 2024-04-17T07:24:59.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.589+0000 7f96599e2740 1 --2- 172.21.15.86:0/4241241190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f27355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.589+0000 7f96599e2740 1 --2- 172.21.15.86:0/4241241190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f273bb50 0x5575f27350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.589+0000 7f96599e2740 1 --2- 172.21.15.86:0/4241241190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273b160 0x5575f2736d00 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9640009230 tx=0x7f9640005fd0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.589+0000 7f96599e2740 1 -- 172.21.15.86:0/4241241190 >> 172.21.15.86:0/4241241190 conn(0x5575f27366d0 msgr2=0x5575f27551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.590+0000 7f96599e2740 1 -- 172.21.15.86:0/4241241190 shutdown_connections 2024-04-17T07:24:59.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.590+0000 7f96599e2740 1 -- 172.21.15.86:0/4241241190 wait complete. 2024-04-17T07:24:59.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.591+0000 7f96599e2740 1 -- start start 2024-04-17T07:24:59.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.593+0000 7f96599e2740 1 Processor -- start 2024-04-17T07:24:59.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.593+0000 7f96599e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273bb50 0x5575f275a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.593+0000 7f96599e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f275a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.593+0000 7f96599e2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f275b8f0 0x5575f275bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.593+0000 7f96599e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f965006f3d0 con 0x5575f2740cc0 2024-04-17T07:24:59.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.593+0000 7f96599e2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f965006ed00 con 0x5575f275b8f0 2024-04-17T07:24:59.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.593+0000 7f96599e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f965010e0e0 con 0x5575f273bb50 2024-04-17T07:24:59.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.594+0000 7f96599e2740 10 client.0 _ll_register_callbacks cb 0x7f964809c370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:24:59.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.597+0000 7f964fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273bb50 0x5575f275a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.597+0000 7f964fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273bb50 0x5575f275a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47902/0 (socket says 172.21.15.86:47902) 2024-04-17T07:24:59.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.597+0000 7f964fe00640 1 -- 172.21.15.86:0/1165474393 learned_addr learned my addr 172.21.15.86:0/1165474393 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.597+0000 7f9654e00640 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f275b8f0 0x5575f275bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.597+0000 7f964f400640 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f275a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.597+0000 7f964fe00640 1 -- 172.21.15.86:0/1165474393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f275b8f0 msgr2=0x5575f275bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.597+0000 7f964fe00640 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f275b8f0 0x5575f275bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.597+0000 7f964fe00640 1 -- 172.21.15.86:0/1165474393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 msgr2=0x5575f275a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.599+0000 7f964fe00640 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f275a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.599+0000 7f964fe00640 1 -- 172.21.15.86:0/1165474393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f247d7d0 con 0x5575f273bb50 2024-04-17T07:24:59.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.599+0000 7f964f400640 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f275a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:24:59.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.600+0000 7f964fe00640 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273bb50 0x5575f275a3f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f964002bc70 tx=0x7f9640005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cba6cabf38135e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.600+0000 7f963a000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:24:59.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.601+0000 7f963a000640 1 -- 172.21.15.86:0/1165474393 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640063070 con 0x5575f273bb50 2024-04-17T07:24:59.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.601+0000 7f963a000640 1 -- 172.21.15.86:0/1165474393 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9640054e00 con 0x5575f273bb50 2024-04-17T07:24:59.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.601+0000 7f96599e2740 1 -- 172.21.15.86:0/1165474393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9640058030 con 0x5575f273bb50 2024-04-17T07:24:59.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.602+0000 7f963a000640 1 -- 172.21.15.86:0/1165474393 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f964006d050 con 0x5575f273bb50 2024-04-17T07:24:59.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.602+0000 7f96599e2740 10 client.27031 fetch_fsmap learned FSMap version 28 2024-04-17T07:24:59.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.602+0000 7f96599e2740 1 -- 172.21.15.86:0/1165474393 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5575f275d3e0 con 0x5575f273bb50 2024-04-17T07:24:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.603+0000 7f963a000640 1 -- 172.21.15.86:0/1165474393 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9640058030 con 0x5575f273bb50 2024-04-17T07:24:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.603+0000 7f963a000640 1 -- 172.21.15.86:0/1165474393 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f964004f050 con 0x5575f273bb50 2024-04-17T07:24:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.603+0000 7f96599e2740 10 client.27031 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:24:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.603+0000 7f96599e2740 10 client.27031 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:24:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.603+0000 7f96599e2740 20 client.27031 populate_metadata read hostname 'smithi086' 2024-04-17T07:24:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.603+0000 7f96599e2740 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5575f27c8d50 0x5575f27cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.603+0000 7f96599e2740 4 client.27031 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:24:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.603+0000 7f96599e2740 1 -- 172.21.15.86:0/1165474393 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575f26a88b0 con 0x5575f27c8d50 2024-04-17T07:24:59.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.604+0000 7f964f400640 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5575f27c8d50 0x5575f27cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.605+0000 7f963a000640 10 client.27031 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:24:59.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.605+0000 7f964f400640 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5575f27c8d50 0x5575f27cb170 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74cd780de7d59e1a server_cookie=419c759e076b2392 in_seq=0 out_seq=0 2024-04-17T07:24:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.606+0000 7f963a000640 1 -- 172.21.15.86:0/1165474393 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575f26a88b0 con 0x5575f27c8d50 2024-04-17T07:24:59.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.606+0000 7f963a000640 10 client.27031 handle_command_reply: tid=0 2024-04-17T07:24:59.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.607+0000 7f96599e2740 1 client.27031 shutdown 2024-04-17T07:24:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.608+0000 7f96599e2740 1 -- 172.21.15.86:0/1165474393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273bb50 msgr2=0x5575f275a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.608+0000 7f96599e2740 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273bb50 0x5575f275a3f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f964002bc70 tx=0x7f9640005d50 comp rx=0 tx=0).stop 2024-04-17T07:24:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.608+0000 7f96599e2740 1 -- 172.21.15.86:0/1165474393 shutdown_connections 2024-04-17T07:24:59.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.608+0000 7f96599e2740 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5575f27c8d50 0x5575f27cb170 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.608+0000 7f96599e2740 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5575f275b8f0 0x5575f275bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.608+0000 7f96599e2740 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5575f2740cc0 0x5575f275a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.608+0000 7f96599e2740 1 --2- 172.21.15.86:0/1165474393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5575f273bb50 0x5575f275a3f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.608+0000 7f96599e2740 1 -- 172.21.15.86:0/1165474393 >> 172.21.15.86:0/1165474393 conn(0x5575f27366d0 msgr2=0x5575f27350a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.608+0000 7f963a000640 0 client.27031 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:24:59.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.609+0000 7f96599e2740 1 -- 172.21.15.86:0/1165474393 shutdown_connections 2024-04-17T07:24:59.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.609+0000 7f96599e2740 1 -- 172.21.15.86:0/1165474393 wait complete. 2024-04-17T07:24:59.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.609+0000 7f96599e2740 20 client.27031 trim_cache size 0 max 16384 2024-04-17T07:24:59.610 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:24:59.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.611+0000 7f9655800640 1 -- 172.21.15.86:0/481072419 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96340a3d00 msgr2=0x7f96340a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.611+0000 7f9655800640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96340a3d00 0x7f96340a61c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f964002ce10 tx=0x7f9640000bc0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.611+0000 7f9655800640 1 -- 172.21.15.86:0/481072419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f965010e440 msgr2=0x7f96501a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.611+0000 7f9655800640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f965010e440 0x7f96501a5c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f964800eaa0 tx=0x7f964800ef60 comp rx=0 tx=0).stop 2024-04-17T07:24:59.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.611+0000 7f9655800640 1 -- 172.21.15.86:0/481072419 shutdown_connections 2024-04-17T07:24:59.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.611+0000 7f9655800640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f96340a3d00 0x7f96340a61c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.611+0000 7f9655800640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f965010e440 0x7f96501a5c90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.611+0000 7f9655800640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9650075fa0 0x7f965019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.611+0000 7f9655800640 1 --2- 172.21.15.86:0/481072419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96500749c0 0x7f965019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.611+0000 7f9655800640 1 -- 172.21.15.86:0/481072419 >> 172.21.15.86:0/481072419 conn(0x7f96500fd400 msgr2=0x7f96501017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.612+0000 7f9655800640 1 -- 172.21.15.86:0/481072419 shutdown_connections 2024-04-17T07:24:59.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.612+0000 7f9655800640 1 -- 172.21.15.86:0/481072419 wait complete. 2024-04-17T07:24:59.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:24:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.980+0000 7ff925a00640 1 -- 172.21.15.86:0/3564197735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff920073240 msgr2=0x7ff920073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.980+0000 7ff925a00640 1 --2- 172.21.15.86:0/3564197735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff920073240 0x7ff920073660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff910009860 tx=0x7ff91002bb70 comp rx=0 tx=0).stop 2024-04-17T07:24:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.981+0000 7ff925a00640 1 -- 172.21.15.86:0/3564197735 shutdown_connections 2024-04-17T07:24:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.981+0000 7ff925a00640 1 --2- 172.21.15.86:0/3564197735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff920109f00 0x7ff92010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.981+0000 7ff925a00640 1 --2- 172.21.15.86:0/3564197735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920109540 0x7ff9201099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.981+0000 7ff925a00640 1 --2- 172.21.15.86:0/3564197735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff920073240 0x7ff920073660 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.981+0000 7ff925a00640 1 -- 172.21.15.86:0/3564197735 >> 172.21.15.86:0/3564197735 conn(0x7ff92006edb0 msgr2=0x7ff9200711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:24:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.981+0000 7ff925a00640 1 -- 172.21.15.86:0/3564197735 shutdown_connections 2024-04-17T07:24:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.981+0000 7ff925a00640 1 -- 172.21.15.86:0/3564197735 wait complete. 2024-04-17T07:24:59.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.983+0000 7ff925a00640 1 Processor -- start 2024-04-17T07:24:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.983+0000 7ff925a00640 1 -- start start 2024-04-17T07:24:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.983+0000 7ff925a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff920073240 0x7ff9201a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.983+0000 7ff925a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff920109540 0x7ff9201a7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.983+0000 7ff925a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920109f00 0x7ff9201ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.983+0000 7ff925a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff92011c6d0 con 0x7ff920109f00 2024-04-17T07:24:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.983+0000 7ff925a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff92011c550 con 0x7ff920109540 2024-04-17T07:24:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.983+0000 7ff925a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff92011c850 con 0x7ff920073240 2024-04-17T07:24:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.984+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff920073240 0x7ff9201a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.984+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff920073240 0x7ff9201a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47912/0 (socket says 172.21.15.86:47912) 2024-04-17T07:24:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.984+0000 7ff91fe00640 1 -- 172.21.15.86:0/3933755182 learned_addr learned my addr 172.21.15.86:0/3933755182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.984+0000 7ff925000640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920109f00 0x7ff9201ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.984+0000 7ff91f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff920109540 0x7ff9201a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.985+0000 7ff925000640 1 -- 172.21.15.86:0/3933755182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff920073240 msgr2=0x7ff9201a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.985+0000 7ff925000640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff920073240 0x7ff9201a6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.985+0000 7ff925000640 1 -- 172.21.15.86:0/3933755182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff920109540 msgr2=0x7ff9201a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:24:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.985+0000 7ff925000640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff920109540 0x7ff9201a7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.985+0000 7ff925000640 1 -- 172.21.15.86:0/3933755182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9201aea10 con 0x7ff920109f00 2024-04-17T07:24:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.986+0000 7ff925000640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920109f00 0x7ff9201ae3a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e940 tx=0x7ff91800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.986+0000 7ff91cc00640 1 -- 172.21.15.86:0/3933755182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918004d20 con 0x7ff920109f00 2024-04-17T07:24:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.986+0000 7ff925a00640 1 -- 172.21.15.86:0/3933755182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9201aed00 con 0x7ff920109f00 2024-04-17T07:24:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.987+0000 7ff925a00640 1 -- 172.21.15.86:0/3933755182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9201af1f0 con 0x7ff920109f00 2024-04-17T07:24:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.987+0000 7ff91cc00640 1 -- 172.21.15.86:0/3933755182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff918005020 con 0x7ff920109f00 2024-04-17T07:24:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.987+0000 7ff91cc00640 1 -- 172.21.15.86:0/3933755182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91801f060 con 0x7ff920109f00 2024-04-17T07:24:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.992+0000 7ff91cc00640 1 -- 172.21.15.86:0/3933755182 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff9180116d0 con 0x7ff920109f00 2024-04-17T07:24:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.993+0000 7ff929d98740 1 Processor -- start 2024-04-17T07:24:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.993+0000 7ff929d98740 1 -- start start 2024-04-17T07:24:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.994+0000 7ff929d98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d51a0 0x5594325d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.994+0000 7ff929d98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d5b90 0x5594325da690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.994+0000 7ff929d98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325dd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.994+0000 7ff929d98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55943253b020 con 0x5594325dad00 2024-04-17T07:24:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.994+0000 7ff929d98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5594324e1460 con 0x5594325d51a0 2024-04-17T07:24:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.994+0000 7ff929d98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559432536fe0 con 0x5594325d5b90 2024-04-17T07:24:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.994+0000 7ff91f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.995+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d5b90 0x5594325da690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.995+0000 7ff91f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d51a0 0x5594325d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.995+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d5b90 0x5594325da690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47920/0 (socket says 172.21.15.86:47920) 2024-04-17T07:24:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.995+0000 7ff91fe00640 1 -- 172.21.15.86:0/2940699780 learned_addr learned my addr 172.21.15.86:0/2940699780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:24:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.995+0000 7ff91fe00640 1 -- 172.21.15.86:0/2940699780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d51a0 msgr2=0x5594325d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.995+0000 7ff91fe00640 1 --2- 172.21.15.86:0/2940699780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d51a0 0x5594325d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.995+0000 7ff91fe00640 1 -- 172.21.15.86:0/2940699780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 msgr2=0x5594325dd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:24:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.995+0000 7ff91fe00640 1 --2- 172.21.15.86:0/2940699780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325dd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:24:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.995+0000 7ff91fe00640 1 -- 172.21.15.86:0/2940699780 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594324e9040 con 0x5594325d5b90 2024-04-17T07:24:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.996+0000 7ff91cc00640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9040a3e30 0x7ff9040a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:24:59.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.996+0000 7ff91f400640 1 --2- 172.21.15.86:0/2940699780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325dd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:24:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.996+0000 7ff91cc00640 1 -- 172.21.15.86:0/3933755182 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff918011470 con 0x7ff920109f00 2024-04-17T07:24:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.996+0000 7ff91fe00640 1 --2- 172.21.15.86:0/2940699780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d5b90 0x5594325da690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff910009510 tx=0x7ff910005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a2b525b1bc38b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:24:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.997+0000 7ff91f400640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9040a3e30 0x7ff9040a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:24:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.997+0000 7ff915600640 1 -- 172.21.15.86:0/2940699780 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910043020 con 0x5594325d5b90 2024-04-17T07:24:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.997+0000 7ff915600640 1 -- 172.21.15.86:0/2940699780 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff910004040 con 0x5594325d5b90 2024-04-17T07:24:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.997+0000 7ff915600640 1 -- 172.21.15.86:0/2940699780 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910039040 con 0x5594325d5b90 2024-04-17T07:25:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.998+0000 7ff929d98740 1 -- 172.21.15.86:0/2940699780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d5b90 msgr2=0x5594325da690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.998+0000 7ff929d98740 1 --2- 172.21.15.86:0/2940699780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d5b90 0x5594325da690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff910009510 tx=0x7ff910005b20 comp rx=0 tx=0).stop 2024-04-17T07:25:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.998+0000 7ff929d98740 1 -- 172.21.15.86:0/2940699780 shutdown_connections 2024-04-17T07:25:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.998+0000 7ff929d98740 1 --2- 172.21.15.86:0/2940699780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325dd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.998+0000 7ff929d98740 1 --2- 172.21.15.86:0/2940699780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d5b90 0x5594325da690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.998+0000 7ff929d98740 1 --2- 172.21.15.86:0/2940699780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d51a0 0x5594325d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:24:59.998+0000 7ff929d98740 1 -- 172.21.15.86:0/2940699780 >> 172.21.15.86:0/2940699780 conn(0x5594325d0710 msgr2=0x5594325d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.000+0000 7ff929d98740 1 -- 172.21.15.86:0/2940699780 shutdown_connections 2024-04-17T07:25:00.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.000+0000 7ff91f400640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9040a3e30 0x7ff9040a62f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5594325d63f0 tx=0x7ff90c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.000+0000 7ff929d98740 1 -- 172.21.15.86:0/2940699780 wait complete. 2024-04-17T07:25:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.000+0000 7ff929d98740 1 -- start start 2024-04-17T07:25:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.001+0000 7ff929d98740 1 Processor -- start 2024-04-17T07:25:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.001+0000 7ff929d98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325d51a0 0x5594325df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.001+0000 7ff929d98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 0x5594325dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.001+0000 7ff929d98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325dad00 0x5594325e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.002+0000 7ff929d98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594325ddc50 con 0x5594325d51a0 2024-04-17T07:25:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.002+0000 7ff929d98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5594324f4830 con 0x5594325d5b90 2024-04-17T07:25:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.002+0000 7ff929d98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5594325dddd0 con 0x5594325dad00 2024-04-17T07:25:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.002+0000 7ff929d98740 10 client.0 _ll_register_callbacks cb 0x5594324c2fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.002+0000 7ff91f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 0x5594325dfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.002+0000 7ff925000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325dad00 0x5594325e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.002+0000 7ff925000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325dad00 0x5594325e6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47934/0 (socket says 172.21.15.86:47934) 2024-04-17T07:25:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.002+0000 7ff925000640 1 -- 172.21.15.86:0/3079716587 learned_addr learned my addr 172.21.15.86:0/3079716587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.003+0000 7ff91fe00640 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325d51a0 0x5594325df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.003+0000 7ff925000640 1 -- 172.21.15.86:0/3079716587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 msgr2=0x5594325dfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.003+0000 7ff925000640 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 0x5594325dfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.003+0000 7ff925000640 1 -- 172.21.15.86:0/3079716587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325d51a0 msgr2=0x5594325df660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.003+0000 7ff925000640 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325d51a0 0x5594325df660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.003+0000 7ff925000640 1 -- 172.21.15.86:0/3079716587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594324f8920 con 0x5594325dad00 2024-04-17T07:25:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.003+0000 7ff925000640 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325dad00 0x5594325e6be0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff9180cdc70 tx=0x7ff9180057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d990942adfa04516 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.003+0000 7ff90a000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.004+0000 7ff90a000640 1 -- 172.21.15.86:0/3079716587 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91801f040 con 0x5594325dad00 2024-04-17T07:25:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.004+0000 7ff929d98740 1 -- 172.21.15.86:0/3079716587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff910004040 con 0x5594325dad00 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.004+0000 7ff90a000640 1 -- 172.21.15.86:0/3079716587 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff91809b950 con 0x5594325dad00 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.004+0000 7ff90a000640 1 -- 172.21.15.86:0/3079716587 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff918098040 con 0x5594325dad00 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.004+0000 7ff929d98740 10 client.27043 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.004+0000 7ff929d98740 1 -- 172.21.15.86:0/3079716587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594324c67f0 con 0x5594325dad00 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.004+0000 7ff90a000640 1 -- 172.21.15.86:0/3079716587 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff910004040 con 0x5594325dad00 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.005+0000 7ff90a000640 1 -- 172.21.15.86:0/3079716587 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff91800f040 con 0x5594325dad00 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.005+0000 7ff929d98740 10 client.27043 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.005+0000 7ff929d98740 10 client.27043 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.005+0000 7ff929d98740 20 client.27043 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.005+0000 7ff929d98740 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594325eb200 0x5594325ed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.005+0000 7ff929d98740 4 client.27043 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.005+0000 7ff929d98740 1 -- 172.21.15.86:0/3079716587 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559432548a40 con 0x5594325eb200 2024-04-17T07:25:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.006+0000 7ff91fe00640 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594325eb200 0x5594325ed6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.007+0000 7ff91fe00640 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594325eb200 0x5594325ed6c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=443f21e188977c49 server_cookie=8e7982b8dd06d22a in_seq=0 out_seq=0 2024-04-17T07:25:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.007+0000 7ff90a000640 10 client.27043 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:25:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.009+0000 7ff90a000640 1 -- 172.21.15.86:0/3079716587 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559432548a40 con 0x5594325eb200 2024-04-17T07:25:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.009+0000 7ff90a000640 10 client.27043 handle_command_reply: tid=0 2024-04-17T07:25:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.009+0000 7ff929d98740 1 client.27043 shutdown 2024-04-17T07:25:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.009+0000 7ff929d98740 1 -- 172.21.15.86:0/3079716587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325dad00 msgr2=0x5594325e6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.009+0000 7ff929d98740 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325dad00 0x5594325e6be0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff9180cdc70 tx=0x7ff9180057d0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.009+0000 7ff929d98740 1 -- 172.21.15.86:0/3079716587 shutdown_connections 2024-04-17T07:25:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.009+0000 7ff929d98740 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325dad00 0x5594325e6be0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.009+0000 7ff929d98740 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594325eb200 0x5594325ed6c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.010+0000 7ff929d98740 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 0x5594325dfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.010+0000 7ff929d98740 1 --2- 172.21.15.86:0/3079716587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325d51a0 0x5594325df660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.010+0000 7ff929d98740 1 -- 172.21.15.86:0/3079716587 >> 172.21.15.86:0/3079716587 conn(0x5594325d0710 msgr2=0x5594325cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.010+0000 7ff90a000640 0 client.27043 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:25:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.011+0000 7ff929d98740 1 -- 172.21.15.86:0/3079716587 shutdown_connections 2024-04-17T07:25:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.011+0000 7ff929d98740 1 -- 172.21.15.86:0/3079716587 wait complete. 2024-04-17T07:25:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.011+0000 7ff929d98740 20 client.27043 trim_cache size 0 max 16384 2024-04-17T07:25:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.049+0000 7ff929d98740 1 Processor -- start 2024-04-17T07:25:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.049+0000 7ff929d98740 1 -- start start 2024-04-17T07:25:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.050+0000 7ff929d98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d51a0 0x5594325d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.050+0000 7ff929d98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 0x5594325e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.050+0000 7ff929d98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325d7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.050+0000 7ff929d98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff90000fb10 con 0x5594325dad00 2024-04-17T07:25:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.050+0000 7ff929d98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff900011650 con 0x5594325d5b90 2024-04-17T07:25:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.050+0000 7ff929d98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9000101e0 con 0x5594325d51a0 2024-04-17T07:25:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.050+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d51a0 0x5594325d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.050+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d51a0 0x5594325d6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47946/0 (socket says 172.21.15.86:47946) 2024-04-17T07:25:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.050+0000 7ff91fe00640 1 -- 172.21.15.86:0/1407553661 learned_addr learned my addr 172.21.15.86:0/1407553661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.051+0000 7ff925000640 1 --2- 172.21.15.86:0/1407553661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.051+0000 7ff91fe00640 1 -- 172.21.15.86:0/1407553661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 msgr2=0x5594325e6ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:25:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.051+0000 7ff91fe00640 1 --2- 172.21.15.86:0/1407553661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 0x5594325e6ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.051+0000 7ff91fe00640 1 -- 172.21.15.86:0/1407553661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 msgr2=0x5594325d7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.051+0000 7ff91fe00640 1 --2- 172.21.15.86:0/1407553661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325d7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.051+0000 7ff91fe00640 1 -- 172.21.15.86:0/1407553661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55943224d3f0 con 0x5594325d51a0 2024-04-17T07:25:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.052+0000 7ff925000640 1 --2- 172.21.15.86:0/1407553661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325d7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.052+0000 7ff91fe00640 1 --2- 172.21.15.86:0/1407553661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d51a0 0x5594325d6ff0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff910009540 tx=0x7ff910034980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5074abbef59dbb56 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.054+0000 7ff914c00640 1 -- 172.21.15.86:0/1407553661 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910043040 con 0x5594325d51a0 2024-04-17T07:25:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.054+0000 7ff914c00640 1 -- 172.21.15.86:0/1407553661 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff910004310 con 0x5594325d51a0 2024-04-17T07:25:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.054+0000 7ff914c00640 1 -- 172.21.15.86:0/1407553661 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910039040 con 0x5594325d51a0 2024-04-17T07:25:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.055+0000 7ff929d98740 1 -- 172.21.15.86:0/1407553661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d51a0 msgr2=0x5594325d6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.055+0000 7ff929d98740 1 --2- 172.21.15.86:0/1407553661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d51a0 0x5594325d6ff0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff910009540 tx=0x7ff910034980 comp rx=0 tx=0).stop 2024-04-17T07:25:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.055+0000 7ff929d98740 1 -- 172.21.15.86:0/1407553661 shutdown_connections 2024-04-17T07:25:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.055+0000 7ff929d98740 1 --2- 172.21.15.86:0/1407553661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325d7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.055+0000 7ff929d98740 1 --2- 172.21.15.86:0/1407553661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 0x5594325e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.055+0000 7ff929d98740 1 --2- 172.21.15.86:0/1407553661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594325d51a0 0x5594325d6ff0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.055+0000 7ff929d98740 1 -- 172.21.15.86:0/1407553661 >> 172.21.15.86:0/1407553661 conn(0x5594325d0710 msgr2=0x5594325df660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.056+0000 7ff929d98740 1 -- 172.21.15.86:0/1407553661 shutdown_connections 2024-04-17T07:25:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.056+0000 7ff929d98740 1 -- 172.21.15.86:0/1407553661 wait complete. 2024-04-17T07:25:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.056+0000 7ff929d98740 1 -- start start 2024-04-17T07:25:00.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.057+0000 7ff929d98740 1 Processor -- start 2024-04-17T07:25:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.057+0000 7ff929d98740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 0x5594325e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.057+0000 7ff929d98740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325e4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.058+0000 7ff929d98740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55943264d7e0 0x55943264fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.058+0000 7ff929d98740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff900010f80 con 0x5594325dad00 2024-04-17T07:25:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.058+0000 7ff929d98740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9000108b0 con 0x5594325d5b90 2024-04-17T07:25:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.058+0000 7ff929d98740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff90000dae0 con 0x55943264d7e0 2024-04-17T07:25:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.058+0000 7ff929d98740 10 client.0 _ll_register_callbacks cb 0x7ff91809c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.058+0000 7ff91f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.058+0000 7ff91f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325e4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44802/0 (socket says 172.21.15.86:44802) 2024-04-17T07:25:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.058+0000 7ff91f400640 1 -- 172.21.15.86:0/3707105835 learned_addr learned my addr 172.21.15.86:0/3707105835 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.058+0000 7ff925000640 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55943264d7e0 0x55943264fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.059+0000 7ff91f400640 1 -- 172.21.15.86:0/3707105835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55943264d7e0 msgr2=0x55943264fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.059+0000 7ff91f400640 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55943264d7e0 0x55943264fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.059+0000 7ff91f400640 1 -- 172.21.15.86:0/3707105835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 msgr2=0x5594325e3ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:25:00.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.059+0000 7ff91f400640 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 0x5594325e3ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.059+0000 7ff91f400640 1 -- 172.21.15.86:0/3707105835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594323177d0 con 0x5594325dad00 2024-04-17T07:25:00.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.060+0000 7ff91f400640 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325e4020 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005fd0 tx=0x7ff90c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349a58aa2ffd9a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:00.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.060+0000 7ff90a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:25:00.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.060+0000 7ff90a000640 1 -- 172.21.15.86:0/3707105835 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c02d020 con 0x5594325dad00 2024-04-17T07:25:00.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.060+0000 7ff90a000640 1 -- 172.21.15.86:0/3707105835 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff90c011b40 con 0x5594325dad00 2024-04-17T07:25:00.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.060+0000 7ff929d98740 1 -- 172.21.15.86:0/3707105835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff910004310 con 0x5594325dad00 2024-04-17T07:25:00.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.061+0000 7ff90a000640 1 -- 172.21.15.86:0/3707105835 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff90c038050 con 0x5594325dad00 2024-04-17T07:25:00.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.061+0000 7ff929d98740 10 client.18114 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:00.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.061+0000 7ff929d98740 1 -- 172.21.15.86:0/3707105835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559432651200 con 0x5594325dad00 2024-04-17T07:25:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.062+0000 7ff90a000640 1 -- 172.21.15.86:0/3707105835 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff910004310 con 0x5594325dad00 2024-04-17T07:25:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.062+0000 7ff90a000640 1 -- 172.21.15.86:0/3707105835 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff90c0283b0 con 0x5594325dad00 2024-04-17T07:25:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.062+0000 7ff929d98740 10 client.18114 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.062+0000 7ff929d98740 10 client.18114 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:25:00.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.062+0000 7ff929d98740 20 client.18114 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:00.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.062+0000 7ff929d98740 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x559432651c60 0x559432654080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.062+0000 7ff929d98740 4 client.18114 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:25:00.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.062+0000 7ff929d98740 1 -- 172.21.15.86:0/3707105835 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594325428b0 con 0x559432651c60 2024-04-17T07:25:00.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.063+0000 7ff91fe00640 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x559432651c60 0x559432654080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.064+0000 7ff91fe00640 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x559432651c60 0x559432654080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba0d36d78ae4ecd3 server_cookie=393d8bf30a688b41 in_seq=0 out_seq=0 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.064+0000 7ff90a000640 10 client.18114 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.065+0000 7ff90a000640 1 -- 172.21.15.86:0/3707105835 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594325428b0 con 0x559432651c60 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.065+0000 7ff90a000640 10 client.18114 handle_command_reply: tid=0 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.065+0000 7ff929d98740 1 client.18114 shutdown 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.065+0000 7ff929d98740 1 -- 172.21.15.86:0/3707105835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 msgr2=0x5594325e4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.065+0000 7ff929d98740 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325e4020 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005fd0 tx=0x7ff90c004730 comp rx=0 tx=0).stop 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.066+0000 7ff929d98740 1 -- 172.21.15.86:0/3707105835 shutdown_connections 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.066+0000 7ff929d98740 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55943264d7e0 0x55943264fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.066+0000 7ff929d98740 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594325dad00 0x5594325e4020 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.066+0000 7ff929d98740 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x559432651c60 0x559432654080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.066+0000 7ff929d98740 1 --2- 172.21.15.86:0/3707105835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594325d5b90 0x5594325e3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.066+0000 7ff929d98740 1 -- 172.21.15.86:0/3707105835 >> 172.21.15.86:0/3707105835 conn(0x5594325d0710 msgr2=0x5594325df5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.066+0000 7ff90a000640 0 client.18114 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:25:00.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.067+0000 7ff929d98740 1 -- 172.21.15.86:0/3707105835 shutdown_connections 2024-04-17T07:25:00.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.067+0000 7ff929d98740 1 -- 172.21.15.86:0/3707105835 wait complete. 2024-04-17T07:25:00.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.067+0000 7ff929d98740 20 client.18114 trim_cache size 0 max 16384 2024-04-17T07:25:00.070 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:25:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.070+0000 7ff925a00640 1 -- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9040a3e30 msgr2=0x7ff9040a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.070+0000 7ff925a00640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9040a3e30 0x7ff9040a62f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5594325d63f0 tx=0x7ff90c009260 comp rx=0 tx=0).stop 2024-04-17T07:25:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.070+0000 7ff925a00640 1 -- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920109f00 msgr2=0x7ff9201ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.070+0000 7ff925a00640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920109f00 0x7ff9201ae3a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e940 tx=0x7ff91800ee00 comp rx=0 tx=0).stop 2024-04-17T07:25:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.071+0000 7ff925a00640 1 -- 172.21.15.86:0/3933755182 shutdown_connections 2024-04-17T07:25:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.071+0000 7ff925a00640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff9040a3e30 0x7ff9040a62f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.071+0000 7ff925a00640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff920109f00 0x7ff9201ae3a0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.071+0000 7ff925a00640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff920109540 0x7ff9201a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.071+0000 7ff925a00640 1 --2- 172.21.15.86:0/3933755182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff920073240 0x7ff9201a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.071+0000 7ff925a00640 1 -- 172.21.15.86:0/3933755182 >> 172.21.15.86:0/3933755182 conn(0x7ff92006edb0 msgr2=0x7ff920070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.071+0000 7ff925a00640 1 -- 172.21.15.86:0/3933755182 shutdown_connections 2024-04-17T07:25:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.071+0000 7ff925a00640 1 -- 172.21.15.86:0/3933755182 wait complete. 2024-04-17T07:25:00.105 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:25:00.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.478+0000 7fb1ebe00640 1 -- 172.21.15.86:0/4068821097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e4099910 msgr2=0x7fb1e4099d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.478+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/4068821097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e4099910 0x7fb1e4099d90 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec067c60 tx=0x7fb1ec0983a0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.478+0000 7fb1ebe00640 1 -- 172.21.15.86:0/4068821097 shutdown_connections 2024-04-17T07:25:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.478+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/4068821097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e4099910 0x7fb1e4099d90 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.478+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/4068821097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e4096f60 0x7fb1e40993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.478+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/4068821097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e40945f0 0x7fb1e4096a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.478+0000 7fb1ebe00640 1 -- 172.21.15.86:0/4068821097 >> 172.21.15.86:0/4068821097 conn(0x7fb1e40901d0 msgr2=0x7fb1e4092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.479+0000 7fb1ebe00640 1 -- 172.21.15.86:0/4068821097 shutdown_connections 2024-04-17T07:25:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.479+0000 7fb1ebe00640 1 -- 172.21.15.86:0/4068821097 wait complete. 2024-04-17T07:25:00.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.480+0000 7fb1ebe00640 1 Processor -- start 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.480+0000 7fb1ebe00640 1 -- start start 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.481+0000 7fb1ebe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40945f0 0x7fb1e4132310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.481+0000 7fb1ebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e4096f60 0x7fb1e4132850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.481+0000 7fb1ebe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e4099910 0x7fb1e4139870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.481+0000 7fb1ebe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e40a7e80 con 0x7fb1e4096f60 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.481+0000 7fb1ebe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e40a7d00 con 0x7fb1e40945f0 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.481+0000 7fb1ebe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e40a8000 con 0x7fb1e4099910 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.482+0000 7fb1eaa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40945f0 0x7fb1e4132310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.482+0000 7fb1eaa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40945f0 0x7fb1e4132310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39482/0 (socket says 172.21.15.86:39482) 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.482+0000 7fb1eaa00640 1 -- 172.21.15.86:0/519632773 learned_addr learned my addr 172.21.15.86:0/519632773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.482+0000 7fb1ea000640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e4096f60 0x7fb1e4132850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.482+0000 7fb1eb400640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e4099910 0x7fb1e4139870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.483+0000 7fb1eb400640 1 -- 172.21.15.86:0/519632773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40945f0 msgr2=0x7fb1e4132310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.483+0000 7fb1eb400640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40945f0 0x7fb1e4132310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.483+0000 7fb1eb400640 1 -- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e4096f60 msgr2=0x7fb1e4132850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.483+0000 7fb1eb400640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e4096f60 0x7fb1e4132850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.483+0000 7fb1eb400640 1 -- 172.21.15.86:0/519632773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e4139ee0 con 0x7fb1e4099910 2024-04-17T07:25:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.483+0000 7fb1ea000640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e4096f60 0x7fb1e4132850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.483+0000 7fb1eb400640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e4099910 0x7fb1e4139870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec067850 tx=0x7fb1ec068ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.484+0000 7fb1f0a00640 1 -- 172.21.15.86:0/519632773 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec068e10 con 0x7fb1e4099910 2024-04-17T07:25:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.484+0000 7fb1f0a00640 1 -- 172.21.15.86:0/519632773 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1ec0a6040 con 0x7fb1e4099910 2024-04-17T07:25:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.484+0000 7fb1f0a00640 1 -- 172.21.15.86:0/519632773 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec0a0be0 con 0x7fb1e4099910 2024-04-17T07:25:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.484+0000 7fb1ebe00640 1 -- 172.21.15.86:0/519632773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e413a170 con 0x7fb1e4099910 2024-04-17T07:25:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.484+0000 7fb1ebe00640 1 -- 172.21.15.86:0/519632773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e413a5e0 con 0x7fb1e4099910 2024-04-17T07:25:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.491+0000 7fb1f4b81740 1 Processor -- start 2024-04-17T07:25:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.493+0000 7fb1f0a00640 1 -- 172.21.15.86:0/519632773 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb1ec0b2030 con 0x7fb1e4099910 2024-04-17T07:25:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.493+0000 7fb1f4b81740 1 -- start start 2024-04-17T07:25:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.494+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808a160 0x5586f808a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.494+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808ab50 0x5586f808f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.494+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 0x5586f8092150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.494+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586f7ff0020 con 0x5586f808a160 2024-04-17T07:25:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.494+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5586f7f96460 con 0x5586f808fcc0 2024-04-17T07:25:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.494+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5586f7febfe0 con 0x5586f808ab50 2024-04-17T07:25:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.494+0000 7fb1f0a00640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3c30 0x7fb1d00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.494+0000 7fb1f0a00640 1 -- 172.21.15.86:0/519632773 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb1ec15e760 con 0x7fb1e4099910 2024-04-17T07:25:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.494+0000 7fb1ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808a160 0x5586f808a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.495+0000 7fb1ea000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808a160 0x5586f808a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44832/0 (socket says 172.21.15.86:44832) 2024-04-17T07:25:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.495+0000 7fb1ea000640 1 -- 172.21.15.86:0/3858513174 learned_addr learned my addr 172.21.15.86:0/3858513174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.495+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/3858513174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808ab50 0x5586f808f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.495+0000 7fb1ea000640 1 --2- 172.21.15.86:0/3858513174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 0x5586f8092150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.495+0000 7fb1eaa00640 1 -- 172.21.15.86:0/3858513174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 msgr2=0x5586f8092150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.495+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/3858513174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 0x5586f8092150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.495+0000 7fb1eaa00640 1 -- 172.21.15.86:0/3858513174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808a160 msgr2=0x5586f808a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.495+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/3858513174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808a160 0x5586f808a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.495+0000 7fb1eaa00640 1 -- 172.21.15.86:0/3858513174 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f7f9e040 con 0x5586f808ab50 2024-04-17T07:25:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.495+0000 7fb1ea000640 1 --2- 172.21.15.86:0/3858513174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 0x5586f8092150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:25:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.496+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/3858513174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808ab50 0x5586f808f650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009c80 tx=0x7fb1dc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15fd3d05716dcc4c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.497+0000 7fb1eb400640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3c30 0x7fb1d00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.497+0000 7fb1d7e00640 1 -- 172.21.15.86:0/3858513174 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc004230 con 0x5586f808ab50 2024-04-17T07:25:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.497+0000 7fb1d7e00640 1 -- 172.21.15.86:0/3858513174 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1dc0043d0 con 0x5586f808ab50 2024-04-17T07:25:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.497+0000 7fb1d7e00640 1 -- 172.21.15.86:0/3858513174 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc004d70 con 0x5586f808ab50 2024-04-17T07:25:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.499+0000 7fb1f4b81740 1 -- 172.21.15.86:0/3858513174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808ab50 msgr2=0x5586f808f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.499+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/3858513174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808ab50 0x5586f808f650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009c80 tx=0x7fb1dc011810 comp rx=0 tx=0).stop 2024-04-17T07:25:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.499+0000 7fb1f4b81740 1 -- 172.21.15.86:0/3858513174 shutdown_connections 2024-04-17T07:25:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.499+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/3858513174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 0x5586f8092150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.499+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/3858513174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808ab50 0x5586f808f650 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.499+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/3858513174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808a160 0x5586f808a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.499+0000 7fb1f4b81740 1 -- 172.21.15.86:0/3858513174 >> 172.21.15.86:0/3858513174 conn(0x5586f80856d0 msgr2=0x5586f8085ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.499+0000 7fb1f4b81740 1 -- 172.21.15.86:0/3858513174 shutdown_connections 2024-04-17T07:25:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.500+0000 7fb1f4b81740 1 -- 172.21.15.86:0/3858513174 wait complete. 2024-04-17T07:25:00.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.500+0000 7fb1f4b81740 1 -- start start 2024-04-17T07:25:00.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.501+0000 7fb1f4b81740 1 Processor -- start 2024-04-17T07:25:00.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.501+0000 7fb1eb400640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3c30 0x7fb1d00a60f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0677b0 tx=0x7fb1ec068270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:00.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.504+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808a160 0x5586f80946e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.504+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f8094c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.504+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 0x5586f809bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.504+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586f8092c70 con 0x5586f808ab50 2024-04-17T07:25:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.505+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5586f7fa9830 con 0x5586f808fcc0 2024-04-17T07:25:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.505+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5586f8092df0 con 0x5586f808a160 2024-04-17T07:25:00.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.505+0000 7fb1f4b81740 10 client.0 _ll_register_callbacks cb 0x5586f7f77fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.506+0000 7fb1eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f8094c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.506+0000 7fb1ea000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808a160 0x5586f80946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.506+0000 7fb1eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f8094c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44846/0 (socket says 172.21.15.86:44846) 2024-04-17T07:25:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.506+0000 7fb1eaa00640 1 -- 172.21.15.86:0/4223641717 learned_addr learned my addr 172.21.15.86:0/4223641717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.506+0000 7fb1ea000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808a160 0x5586f80946e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:48012/0 (socket says 172.21.15.86:48012) 2024-04-17T07:25:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.506+0000 7fb1ea000640 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 0x5586f809bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.507+0000 7fb1ea000640 1 -- 172.21.15.86:0/4223641717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 msgr2=0x5586f809bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.507+0000 7fb1ea000640 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 0x5586f809bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.507+0000 7fb1ea000640 1 -- 172.21.15.86:0/4223641717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 msgr2=0x5586f8094c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.507+0000 7fb1ea000640 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f8094c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.507+0000 7fb1ea000640 1 -- 172.21.15.86:0/4223641717 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f7fad920 con 0x5586f808a160 2024-04-17T07:25:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.509+0000 7fb1ea000640 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808a160 0x5586f80946e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800b620 tx=0x7fb1d8014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15dc562d91ca89ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:00.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.509+0000 7fb1d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:00.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.509+0000 7fb1d4c00640 1 -- 172.21.15.86:0/4223641717 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8012070 con 0x5586f808a160 2024-04-17T07:25:00.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.509+0000 7fb1d4c00640 1 -- 172.21.15.86:0/4223641717 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb1d801b040 con 0x5586f808a160 2024-04-17T07:25:00.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.509+0000 7fb1f4b81740 1 -- 172.21.15.86:0/4223641717 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1dc0043d0 con 0x5586f808a160 2024-04-17T07:25:00.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.510+0000 7fb1d4c00640 1 -- 172.21.15.86:0/4223641717 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb1d8012210 con 0x5586f808a160 2024-04-17T07:25:00.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.510+0000 7fb1f4b81740 10 client.27061 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:00.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.510+0000 7fb1f4b81740 1 -- 172.21.15.86:0/4223641717 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586f7f7b7f0 con 0x5586f808a160 2024-04-17T07:25:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.510+0000 7fb1d4c00640 1 -- 172.21.15.86:0/4223641717 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1dc0043d0 con 0x5586f808a160 2024-04-17T07:25:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.512+0000 7fb1d4c00640 1 -- 172.21.15.86:0/4223641717 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb1d80054a0 con 0x5586f808a160 2024-04-17T07:25:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.512+0000 7fb1f4b81740 10 client.27061 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.512+0000 7fb1f4b81740 10 client.27061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.512+0000 7fb1f4b81740 20 client.27061 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.512+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5586f80a00c0 0x5586f80a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.512+0000 7fb1f4b81740 4 client.27061 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.512+0000 7fb1f4b81740 1 -- 172.21.15.86:0/4223641717 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586f7ffda40 con 0x5586f80a00c0 2024-04-17T07:25:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.513+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5586f80a00c0 0x5586f80a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.515+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5586f80a00c0 0x5586f80a2540 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1b33fb5f76cfe04 server_cookie=78c925df5888153f in_seq=0 out_seq=0 2024-04-17T07:25:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.515+0000 7fb1d4c00640 10 client.27061 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.517+0000 7fb1d4c00640 1 -- 172.21.15.86:0/4223641717 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586f7ffda40 con 0x5586f80a00c0 2024-04-17T07:25:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.517+0000 7fb1d4c00640 10 client.27061 handle_command_reply: tid=0 2024-04-17T07:25:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.517+0000 7fb1f4b81740 1 client.27061 shutdown 2024-04-17T07:25:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.518+0000 7fb1f4b81740 1 -- 172.21.15.86:0/4223641717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808a160 msgr2=0x5586f80946e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.518+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808a160 0x5586f80946e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800b620 tx=0x7fb1d8014040 comp rx=0 tx=0).stop 2024-04-17T07:25:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.519+0000 7fb1f4b81740 1 -- 172.21.15.86:0/4223641717 shutdown_connections 2024-04-17T07:25:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.519+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5586f80a00c0 0x5586f80a2540 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.519+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808fcc0 0x5586f809bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.519+0000 7fb1d4c00640 0 client.27061 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.519+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f8094c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.519+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/4223641717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808a160 0x5586f80946e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.519+0000 7fb1f4b81740 1 -- 172.21.15.86:0/4223641717 >> 172.21.15.86:0/4223641717 conn(0x5586f80856d0 msgr2=0x5586f8084930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.520+0000 7fb1f4b81740 1 -- 172.21.15.86:0/4223641717 shutdown_connections 2024-04-17T07:25:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.520+0000 7fb1f4b81740 1 -- 172.21.15.86:0/4223641717 wait complete. 2024-04-17T07:25:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.520+0000 7fb1f4b81740 20 client.27061 trim_cache size 0 max 16384 2024-04-17T07:25:00.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.548+0000 7fb1f4b81740 1 Processor -- start 2024-04-17T07:25:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.549+0000 7fb1f4b81740 1 -- start start 2024-04-17T07:25:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.549+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808a160 0x5586f8094b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.549+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f809bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.549+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 0x5586f80847b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.549+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4144c90 con 0x5586f808ab50 2024-04-17T07:25:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.549+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e41467d0 con 0x5586f808a160 2024-04-17T07:25:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.549+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4145360 con 0x5586f808fcc0 2024-04-17T07:25:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.550+0000 7fb1eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f809bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.550+0000 7fb1eaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f809bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44848/0 (socket says 172.21.15.86:44848) 2024-04-17T07:25:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.550+0000 7fb1eaa00640 1 -- 172.21.15.86:0/2980164119 learned_addr learned my addr 172.21.15.86:0/2980164119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.550+0000 7fb1ea000640 1 --2- 172.21.15.86:0/2980164119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808a160 0x5586f8094b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.550+0000 7fb1eaa00640 1 -- 172.21.15.86:0/2980164119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 msgr2=0x5586f80847b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.550+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/2980164119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 0x5586f80847b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.550+0000 7fb1eaa00640 1 -- 172.21.15.86:0/2980164119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808a160 msgr2=0x5586f8094b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.550+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/2980164119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808a160 0x5586f8094b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.550+0000 7fb1eaa00640 1 -- 172.21.15.86:0/2980164119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f7d023f0 con 0x5586f808ab50 2024-04-17T07:25:00.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.551+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/2980164119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f809bc20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc011d10 tx=0x7fb1dc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640e9ea2e768b159 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:00.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.551+0000 7fb1d7400640 1 -- 172.21.15.86:0/2980164119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc01e040 con 0x5586f808ab50 2024-04-17T07:25:00.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.551+0000 7fb1d7400640 1 -- 172.21.15.86:0/2980164119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1dc01f040 con 0x5586f808ab50 2024-04-17T07:25:00.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.551+0000 7fb1d7400640 1 -- 172.21.15.86:0/2980164119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc026960 con 0x5586f808ab50 2024-04-17T07:25:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.552+0000 7fb1f4b81740 1 -- 172.21.15.86:0/2980164119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 msgr2=0x5586f809bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.552+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/2980164119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f809bc20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc011d10 tx=0x7fb1dc011810 comp rx=0 tx=0).stop 2024-04-17T07:25:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.552+0000 7fb1f4b81740 1 -- 172.21.15.86:0/2980164119 shutdown_connections 2024-04-17T07:25:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.552+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/2980164119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 0x5586f80847b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.552+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/2980164119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f808ab50 0x5586f809bc20 secure :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc011d10 tx=0x7fb1dc011810 comp rx=0 tx=0).stop 2024-04-17T07:25:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.552+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/2980164119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808a160 0x5586f8094b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.552+0000 7fb1f4b81740 1 -- 172.21.15.86:0/2980164119 >> 172.21.15.86:0/2980164119 conn(0x5586f80856d0 msgr2=0x5586f80840a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.553+0000 7fb1f4b81740 1 -- 172.21.15.86:0/2980164119 shutdown_connections 2024-04-17T07:25:00.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.554+0000 7fb1f4b81740 1 -- 172.21.15.86:0/2980164119 wait complete. 2024-04-17T07:25:00.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.555+0000 7fb1f4b81740 1 -- start start 2024-04-17T07:25:00.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.556+0000 7fb1f4b81740 1 Processor -- start 2024-04-17T07:25:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.557+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808a160 0x5586f8099400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.557+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 0x5586f8099940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.557+0000 7fb1f4b81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f80a00c0 0x5586f809a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.557+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4146100 con 0x5586f80a00c0 2024-04-17T07:25:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.557+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4145a30 con 0x5586f808a160 2024-04-17T07:25:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.557+0000 7fb1f4b81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4142c60 con 0x5586f808fcc0 2024-04-17T07:25:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.557+0000 7fb1f4b81740 10 client.0 _ll_register_callbacks cb 0x7fb1d8012c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.558+0000 7fb1eaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 0x5586f8099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.558+0000 7fb1eaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 0x5586f8099940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:48028/0 (socket says 172.21.15.86:48028) 2024-04-17T07:25:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.558+0000 7fb1eaa00640 1 -- 172.21.15.86:0/1760639599 learned_addr learned my addr 172.21.15.86:0/1760639599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:00.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.558+0000 7fb1ea000640 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f80a00c0 0x5586f809a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.558+0000 7fb1ea000640 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808a160 0x5586f8099400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.559+0000 7fb1eaa00640 1 -- 172.21.15.86:0/1760639599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808a160 msgr2=0x5586f8099400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.559+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808a160 0x5586f8099400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.559+0000 7fb1eaa00640 1 -- 172.21.15.86:0/1760639599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f80a00c0 msgr2=0x5586f809a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.559+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f80a00c0 0x5586f809a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.559+0000 7fb1eaa00640 1 -- 172.21.15.86:0/1760639599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f7dcc7d0 con 0x5586f808fcc0 2024-04-17T07:25:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.560+0000 7fb1ea000640 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f80a00c0 0x5586f809a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:00.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.560+0000 7fb1eaa00640 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 0x5586f8099940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc005fd0 tx=0x7fb1dc004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=259b1143b8a931e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.561+0000 7fb1d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.561+0000 7fb1d4c00640 1 -- 172.21.15.86:0/1760639599 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc02dce0 con 0x5586f808fcc0 2024-04-17T07:25:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.561+0000 7fb1d4c00640 1 -- 172.21.15.86:0/1760639599 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb1dc02d2f0 con 0x5586f808fcc0 2024-04-17T07:25:00.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.561+0000 7fb1f4b81740 1 -- 172.21.15.86:0/1760639599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1dc01f040 con 0x5586f808fcc0 2024-04-17T07:25:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.562+0000 7fb1d4c00640 1 -- 172.21.15.86:0/1760639599 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb1dc02d510 con 0x5586f808fcc0 2024-04-17T07:25:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.562+0000 7fb1f4b81740 10 client.27064 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.562+0000 7fb1f4b81740 1 -- 172.21.15.86:0/1760639599 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586f8106620 con 0x5586f808fcc0 2024-04-17T07:25:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.562+0000 7fb1d4c00640 1 -- 172.21.15.86:0/1760639599 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1dc01f040 con 0x5586f808fcc0 2024-04-17T07:25:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.562+0000 7fb1d4c00640 1 -- 172.21.15.86:0/1760639599 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb1dc024040 con 0x5586f808fcc0 2024-04-17T07:25:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.563+0000 7fb1f4b81740 10 client.27064 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.563+0000 7fb1f4b81740 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.563+0000 7fb1f4b81740 20 client.27064 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.563+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5586f8106be0 0x5586f8109000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.563+0000 7fb1f4b81740 4 client.27064 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:25:00.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.563+0000 7fb1f4b81740 1 -- 172.21.15.86:0/1760639599 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586f7ff78b0 con 0x5586f8106be0 2024-04-17T07:25:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.564+0000 7fb1ea000640 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5586f8106be0 0x5586f8109000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:00.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.567+0000 7fb1ea000640 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5586f8106be0 0x5586f8109000 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d00924d6d08b4c6 server_cookie=b7c723a606d109af in_seq=0 out_seq=0 2024-04-17T07:25:00.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.568+0000 7fb1d4c00640 10 client.27064 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:00.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.568+0000 7fb1d4c00640 1 -- 172.21.15.86:0/1760639599 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586f7ff78b0 con 0x5586f8106be0 2024-04-17T07:25:00.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.568+0000 7fb1d4c00640 10 client.27064 handle_command_reply: tid=0 2024-04-17T07:25:00.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.568+0000 7fb1f4b81740 1 client.27064 shutdown 2024-04-17T07:25:00.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.570+0000 7fb1f4b81740 1 -- 172.21.15.86:0/1760639599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 msgr2=0x5586f8099940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.570+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 0x5586f8099940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc005fd0 tx=0x7fb1dc004820 comp rx=0 tx=0).stop 2024-04-17T07:25:00.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.570+0000 7fb1f4b81740 1 -- 172.21.15.86:0/1760639599 shutdown_connections 2024-04-17T07:25:00.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.570+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5586f8106be0 0x5586f8109000 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.570+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586f80a00c0 0x5586f809a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.570+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586f808fcc0 0x5586f8099940 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.570+0000 7fb1f4b81740 1 --2- 172.21.15.86:0/1760639599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586f808a160 0x5586f8099400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.570+0000 7fb1f4b81740 1 -- 172.21.15.86:0/1760639599 >> 172.21.15.86:0/1760639599 conn(0x5586f80856d0 msgr2=0x5586f8094a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.570+0000 7fb1d4c00640 0 client.27064 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.571+0000 7fb1f4b81740 1 -- 172.21.15.86:0/1760639599 shutdown_connections 2024-04-17T07:25:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.571+0000 7fb1f4b81740 1 -- 172.21.15.86:0/1760639599 wait complete. 2024-04-17T07:25:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.571+0000 7fb1f4b81740 20 client.27064 trim_cache size 0 max 16384 2024-04-17T07:25:00.573 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:25:00.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.573+0000 7fb1ebe00640 1 -- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3c30 msgr2=0x7fb1d00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.573+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3c30 0x7fb1d00a60f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0677b0 tx=0x7fb1ec068270 comp rx=0 tx=0).stop 2024-04-17T07:25:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.573+0000 7fb1ebe00640 1 -- 172.21.15.86:0/519632773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e4099910 msgr2=0x7fb1e4139870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.573+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e4099910 0x7fb1e4139870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec067850 tx=0x7fb1ec068ca0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.574+0000 7fb1ebe00640 1 -- 172.21.15.86:0/519632773 shutdown_connections 2024-04-17T07:25:00.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.574+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1d00a3c30 0x7fb1d00a60f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.574+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1e4099910 0x7fb1e4139870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.574+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1e4096f60 0x7fb1e4132850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.574+0000 7fb1ebe00640 1 --2- 172.21.15.86:0/519632773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40945f0 0x7fb1e4132310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.574+0000 7fb1ebe00640 1 -- 172.21.15.86:0/519632773 >> 172.21.15.86:0/519632773 conn(0x7fb1e40901d0 msgr2=0x7fb1e40952a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.575+0000 7fb1ebe00640 1 -- 172.21.15.86:0/519632773 shutdown_connections 2024-04-17T07:25:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:00.575+0000 7fb1ebe00640 1 -- 172.21.15.86:0/519632773 wait complete. 2024-04-17T07:25:01.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:00 smithi098 ceph-mon[35418]: pgmap v759: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.6 MiB/s wr, 623 op/s 2024-04-17T07:25:01.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:00 smithi103 ceph-mon[35495]: pgmap v759: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.6 MiB/s wr, 623 op/s 2024-04-17T07:25:01.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:00 smithi086 ceph-mon[31733]: pgmap v759: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.6 MiB/s wr, 623 op/s 2024-04-17T07:25:01.606 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:25:01.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:25:01.606 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:25:01.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.914+0000 7f2226200640 1 -- 172.21.15.86:0/4020838411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22201091d0 msgr2=0x7f222010b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:01.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.914+0000 7f2226200640 1 --2- 172.21.15.86:0/4020838411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22201091d0 0x7f222010b5c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2210009860 tx=0x7f221002bb70 comp rx=0 tx=0).stop 2024-04-17T07:25:01.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.914+0000 7f2226200640 1 -- 172.21.15.86:0/4020838411 shutdown_connections 2024-04-17T07:25:01.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.914+0000 7f2226200640 1 --2- 172.21.15.86:0/4020838411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f222010e650 0x7f2220110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.914+0000 7f2226200640 1 --2- 172.21.15.86:0/4020838411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222010bb90 0x7f222010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.914+0000 7f2226200640 1 --2- 172.21.15.86:0/4020838411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22201091d0 0x7f222010b5c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.914+0000 7f2226200640 1 -- 172.21.15.86:0/4020838411 >> 172.21.15.86:0/4020838411 conn(0x7f222006f190 msgr2=0x7f22200715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:01.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.915+0000 7f2226200640 1 -- 172.21.15.86:0/4020838411 shutdown_connections 2024-04-17T07:25:01.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.915+0000 7f2226200640 1 -- 172.21.15.86:0/4020838411 wait complete. 2024-04-17T07:25:01.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.917+0000 7f2226200640 1 Processor -- start 2024-04-17T07:25:01.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.917+0000 7f2226200640 1 -- start start 2024-04-17T07:25:01.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.917+0000 7f2226200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22201091d0 0x7f22201a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.917+0000 7f2226200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f222010bb90 0x7f22201a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.917+0000 7f2226200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222010e650 0x7f22201ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.918+0000 7f2226200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22201140e0 con 0x7f222010e650 2024-04-17T07:25:01.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.918+0000 7f2226200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2220113f60 con 0x7f22201091d0 2024-04-17T07:25:01.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.918+0000 7f2226200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2220114260 con 0x7f222010bb90 2024-04-17T07:25:01.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.918+0000 7f221ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f222010bb90 0x7f22201a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.918+0000 7f221ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f222010bb90 0x7f22201a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:48046/0 (socket says 172.21.15.86:48046) 2024-04-17T07:25:01.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.918+0000 7f221fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222010e650 0x7f22201ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.918+0000 7f221ea00640 1 -- 172.21.15.86:0/1559420785 learned_addr learned my addr 172.21.15.86:0/1559420785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:01.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.918+0000 7f221f400640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22201091d0 0x7f22201a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.919+0000 7f221ea00640 1 -- 172.21.15.86:0/1559420785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22201091d0 msgr2=0x7f22201a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:01.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.919+0000 7f221ea00640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22201091d0 0x7f22201a7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.919+0000 7f221ea00640 1 -- 172.21.15.86:0/1559420785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222010e650 msgr2=0x7f22201ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:01.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.919+0000 7f221ea00640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222010e650 0x7f22201ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.919+0000 7f221ea00640 1 -- 172.21.15.86:0/1559420785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22201aeca0 con 0x7f222010bb90 2024-04-17T07:25:01.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.919+0000 7f221fe00640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222010e650 0x7f22201ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:01.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.920+0000 7f221ea00640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f222010bb90 0x7f22201a75c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f220c0097d0 tx=0x7f220c009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:01.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.920+0000 7f2217e00640 1 -- 172.21.15.86:0/1559420785 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c018030 con 0x7f222010bb90 2024-04-17T07:25:01.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.920+0000 7f2217e00640 1 -- 172.21.15.86:0/1559420785 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f220c0054d0 con 0x7f222010bb90 2024-04-17T07:25:01.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.920+0000 7f2217e00640 1 -- 172.21.15.86:0/1559420785 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c00e910 con 0x7f222010bb90 2024-04-17T07:25:01.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.920+0000 7f2226200640 1 -- 172.21.15.86:0/1559420785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22201aef90 con 0x7f222010bb90 2024-04-17T07:25:01.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.920+0000 7f2226200640 1 -- 172.21.15.86:0/1559420785 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22201af400 con 0x7f222010bb90 2024-04-17T07:25:01.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.925+0000 7f2228fb8740 1 Processor -- start 2024-04-17T07:25:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.925+0000 7f2228fb8740 1 -- start start 2024-04-17T07:25:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.926+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debbec0 0x55a32debc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.926+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debc8b0 0x55a32dec13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.926+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32dec1a20 0x55a32dec3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.926+0000 7f2228fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a32de21cb0 con 0x55a32debc8b0 2024-04-17T07:25:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.926+0000 7f2228fb8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a32ddc8370 con 0x55a32dec1a20 2024-04-17T07:25:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.926+0000 7f2228fb8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a32de1dc70 con 0x55a32debbec0 2024-04-17T07:25:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.926+0000 7f221f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debc8b0 0x55a32dec13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.926+0000 7f221f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debc8b0 0x55a32dec13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44890/0 (socket says 172.21.15.86:44890) 2024-04-17T07:25:01.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.927+0000 7f221f400640 1 -- 172.21.15.86:0/2591348711 learned_addr learned my addr 172.21.15.86:0/2591348711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:01.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.927+0000 7f2217e00640 1 -- 172.21.15.86:0/1559420785 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f220c01b070 con 0x7f222010bb90 2024-04-17T07:25:01.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.927+0000 7f221fe00640 1 --2- 172.21.15.86:0/2591348711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debbec0 0x55a32debc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.927+0000 7f221ea00640 1 --2- 172.21.15.86:0/2591348711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32dec1a20 0x55a32dec3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.929+0000 7f221fe00640 1 -- 172.21.15.86:0/2591348711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32dec1a20 msgr2=0x55a32dec3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:01.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.929+0000 7f2217e00640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22040a3c30 0x7f22040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.929+0000 7f221fe00640 1 --2- 172.21.15.86:0/2591348711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32dec1a20 0x55a32dec3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.929+0000 7f221fe00640 1 -- 172.21.15.86:0/2591348711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debc8b0 msgr2=0x55a32dec13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:01.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.929+0000 7f221fe00640 1 --2- 172.21.15.86:0/2591348711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debc8b0 0x55a32dec13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.929+0000 7f221fe00640 1 -- 172.21.15.86:0/2591348711 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32dde7910 con 0x55a32debbec0 2024-04-17T07:25:01.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.929+0000 7f2217e00640 1 -- 172.21.15.86:0/1559420785 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f220c0cfff0 con 0x7f222010bb90 2024-04-17T07:25:01.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.930+0000 7f221ea00640 1 --2- 172.21.15.86:0/2591348711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32dec1a20 0x55a32dec3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:01.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.932+0000 7f221fe00640 1 --2- 172.21.15.86:0/2591348711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debbec0 0x55a32debc2e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2218005fd0 tx=0x7f2218011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de0a9b611c9742ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:01.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.932+0000 7f221fe00640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22040a3c30 0x7f22040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.932+0000 7f220be00640 1 -- 172.21.15.86:0/2591348711 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221800f9a0 con 0x55a32debbec0 2024-04-17T07:25:01.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.932+0000 7f220be00640 1 -- 172.21.15.86:0/2591348711 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f221800fbc0 con 0x55a32debbec0 2024-04-17T07:25:01.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.932+0000 7f220be00640 1 -- 172.21.15.86:0/2591348711 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221801c810 con 0x55a32debbec0 2024-04-17T07:25:01.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.934+0000 7f221fe00640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22040a3c30 0x7f22040a60f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2218020000 tx=0x7f221801ca70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:01.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.934+0000 7f2228fb8740 1 -- 172.21.15.86:0/2591348711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debbec0 msgr2=0x55a32debc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:01.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.934+0000 7f2228fb8740 1 --2- 172.21.15.86:0/2591348711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debbec0 0x55a32debc2e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2218005fd0 tx=0x7f2218011bf0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.935+0000 7f2228fb8740 1 -- 172.21.15.86:0/2591348711 shutdown_connections 2024-04-17T07:25:01.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.935+0000 7f2228fb8740 1 --2- 172.21.15.86:0/2591348711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32dec1a20 0x55a32dec3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.936+0000 7f2228fb8740 1 --2- 172.21.15.86:0/2591348711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debc8b0 0x55a32dec13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.936+0000 7f2228fb8740 1 --2- 172.21.15.86:0/2591348711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debbec0 0x55a32debc2e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.936+0000 7f2228fb8740 1 -- 172.21.15.86:0/2591348711 >> 172.21.15.86:0/2591348711 conn(0x55a32deb7430 msgr2=0x55a32deb7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:01.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.936+0000 7f2228fb8740 1 -- 172.21.15.86:0/2591348711 shutdown_connections 2024-04-17T07:25:01.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.936+0000 7f2228fb8740 1 -- 172.21.15.86:0/2591348711 wait complete. 2024-04-17T07:25:01.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.937+0000 7f2228fb8740 1 -- start start 2024-04-17T07:25:01.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.937+0000 7f2228fb8740 1 Processor -- start 2024-04-17T07:25:01.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.938+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 0x55a32dec61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.938+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 0x55a32dec6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.938+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32decd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.938+0000 7f2228fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a32dec49d0 con 0x55a32debbec0 2024-04-17T07:25:01.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.938+0000 7f2228fb8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a32dddb730 con 0x55a32debc8b0 2024-04-17T07:25:01.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.938+0000 7f2228fb8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a32dec4b50 con 0x55a32dec1a20 2024-04-17T07:25:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.938+0000 7f2228fb8740 10 client.0 _ll_register_callbacks cb 0x55a32ddddb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.939+0000 7f221fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32decd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.939+0000 7f221fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32decd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:48064/0 (socket says 172.21.15.86:48064) 2024-04-17T07:25:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.939+0000 7f221fe00640 1 -- 172.21.15.86:0/394339000 learned_addr learned my addr 172.21.15.86:0/394339000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.939+0000 7f221f400640 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 0x55a32dec61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.939+0000 7f221ea00640 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 0x55a32dec6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.940+0000 7f221fe00640 1 -- 172.21.15.86:0/394339000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 msgr2=0x55a32dec6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:01.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.940+0000 7f221fe00640 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 0x55a32dec6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.940+0000 7f221fe00640 1 -- 172.21.15.86:0/394339000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 msgr2=0x55a32dec61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:01.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.941+0000 7f221fe00640 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 0x55a32dec61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.942+0000 7f221fe00640 1 -- 172.21.15.86:0/394339000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32dbfc7e0 con 0x55a32dec1a20 2024-04-17T07:25:01.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.942+0000 7f221fe00640 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32decd750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2218003d10 tx=0x7f2218016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba3e716bca19d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:01.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.942+0000 7f2208c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:01.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.942+0000 7f2208c00640 1 -- 172.21.15.86:0/394339000 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218036060 con 0x55a32dec1a20 2024-04-17T07:25:01.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.943+0000 7f2208c00640 1 -- 172.21.15.86:0/394339000 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f221802e070 con 0x55a32dec1a20 2024-04-17T07:25:01.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.943+0000 7f2228fb8740 1 -- 172.21.15.86:0/394339000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221800fbc0 con 0x55a32dec1a20 2024-04-17T07:25:01.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.945+0000 7f2208c00640 1 -- 172.21.15.86:0/394339000 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f221803d050 con 0x55a32dec1a20 2024-04-17T07:25:01.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.947+0000 7f2228fb8740 10 client.27076 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:01.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.947+0000 7f2228fb8740 1 -- 172.21.15.86:0/394339000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a32dc5cf70 con 0x55a32dec1a20 2024-04-17T07:25:01.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.948+0000 7f2208c00640 1 -- 172.21.15.86:0/394339000 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f221800fbc0 con 0x55a32dec1a20 2024-04-17T07:25:01.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.951+0000 7f2208c00640 1 -- 172.21.15.86:0/394339000 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2218004a00 con 0x55a32dec1a20 2024-04-17T07:25:01.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.951+0000 7f2228fb8740 10 client.27076 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:01.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.951+0000 7f2228fb8740 10 client.27076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:01.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.951+0000 7f2228fb8740 20 client.27076 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:01.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.951+0000 7f2228fb8740 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a32ded23b0 0x55a32ded4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:01.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.951+0000 7f2228fb8740 4 client.27076 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:01.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.951+0000 7f2228fb8740 1 -- 172.21.15.86:0/394339000 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a32de2f6d0 con 0x55a32ded23b0 2024-04-17T07:25:01.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.953+0000 7f221f400640 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a32ded23b0 0x55a32ded4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:01.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.955+0000 7f221f400640 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a32ded23b0 0x55a32ded4820 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c665639f3bee97 server_cookie=dd2684b3a30deb5e in_seq=0 out_seq=0 2024-04-17T07:25:01.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.955+0000 7f2208c00640 10 client.27076 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:01.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.958+0000 7f2208c00640 1 -- 172.21.15.86:0/394339000 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a32de2f6d0 con 0x55a32ded23b0 2024-04-17T07:25:01.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.958+0000 7f2208c00640 10 client.27076 handle_command_reply: tid=0 2024-04-17T07:25:01.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.958+0000 7f2228fb8740 1 client.27076 shutdown 2024-04-17T07:25:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.958+0000 7f2228fb8740 1 -- 172.21.15.86:0/394339000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 msgr2=0x55a32decd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.958+0000 7f2228fb8740 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32decd750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2218003d10 tx=0x7f2218016000 comp rx=0 tx=0).stop 2024-04-17T07:25:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.959+0000 7f2228fb8740 1 -- 172.21.15.86:0/394339000 shutdown_connections 2024-04-17T07:25:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.960+0000 7f2228fb8740 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a32ded23b0 0x55a32ded4820 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.960+0000 7f2228fb8740 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32decd750 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.960+0000 7f2228fb8740 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 0x55a32dec6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.960+0000 7f2228fb8740 1 --2- 172.21.15.86:0/394339000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 0x55a32dec61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.960+0000 7f2228fb8740 1 -- 172.21.15.86:0/394339000 >> 172.21.15.86:0/394339000 conn(0x55a32deb7430 msgr2=0x55a32deb65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.960+0000 7f2208c00640 0 client.27076 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.961+0000 7f2228fb8740 1 -- 172.21.15.86:0/394339000 shutdown_connections 2024-04-17T07:25:01.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.962+0000 7f2228fb8740 1 -- 172.21.15.86:0/394339000 wait complete. 2024-04-17T07:25:01.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:01.962+0000 7f2228fb8740 20 client.27076 trim_cache size 0 max 16384 2024-04-17T07:25:02.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.010+0000 7f2228fb8740 1 Processor -- start 2024-04-17T07:25:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.010+0000 7f2228fb8740 1 -- start start 2024-04-17T07:25:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.011+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 0x55a32dec61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.011+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 0x55a32decd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.011+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32dec6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.011+0000 7f2228fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b99f0 con 0x55a32debbec0 2024-04-17T07:25:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.012+0000 7f221fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32dec6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.012+0000 7f221fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32dec6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:48076/0 (socket says 172.21.15.86:48076) 2024-04-17T07:25:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.012+0000 7f221fe00640 1 -- 172.21.15.86:0/3897655714 learned_addr learned my addr 172.21.15.86:0/3897655714 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.012+0000 7f221ea00640 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 0x55a32decd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.012+0000 7f2228fb8740 1 -- 172.21.15.86:0/3897655714 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f22201bb530 con 0x55a32debc8b0 2024-04-17T07:25:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.012+0000 7f2228fb8740 1 -- 172.21.15.86:0/3897655714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f22201ba0c0 con 0x55a32dec1a20 2024-04-17T07:25:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.012+0000 7f221f400640 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 0x55a32dec61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.013+0000 7f221fe00640 1 -- 172.21.15.86:0/3897655714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 msgr2=0x55a32decd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.013+0000 7f221fe00640 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 0x55a32decd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.013+0000 7f221fe00640 1 -- 172.21.15.86:0/3897655714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 msgr2=0x55a32dec61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.013+0000 7f221fe00640 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 0x55a32dec61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.013+0000 7f221fe00640 1 -- 172.21.15.86:0/3897655714 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32daf4170 con 0x55a32dec1a20 2024-04-17T07:25:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.013+0000 7f221ea00640 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 0x55a32decd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.013+0000 7f221f400640 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 0x55a32dec61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.014+0000 7f221fe00640 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32dec6710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2218003b60 tx=0x7f2218011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16862925a9d6afbe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.014+0000 7f220b400640 1 -- 172.21.15.86:0/3897655714 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218046070 con 0x55a32dec1a20 2024-04-17T07:25:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.015+0000 7f220b400640 1 -- 172.21.15.86:0/3897655714 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f221800fbc0 con 0x55a32dec1a20 2024-04-17T07:25:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.015+0000 7f220b400640 1 -- 172.21.15.86:0/3897655714 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218036040 con 0x55a32dec1a20 2024-04-17T07:25:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.016+0000 7f2228fb8740 1 -- 172.21.15.86:0/3897655714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 msgr2=0x55a32dec6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.016+0000 7f2228fb8740 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32dec6710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2218003b60 tx=0x7f2218011bf0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.016+0000 7f2228fb8740 1 -- 172.21.15.86:0/3897655714 shutdown_connections 2024-04-17T07:25:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.016+0000 7f2228fb8740 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32dec1a20 0x55a32dec6710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.016+0000 7f2228fb8740 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debc8b0 0x55a32decd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.016+0000 7f2228fb8740 1 --2- 172.21.15.86:0/3897655714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32debbec0 0x55a32dec61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.016+0000 7f2228fb8740 1 -- 172.21.15.86:0/3897655714 >> 172.21.15.86:0/3897655714 conn(0x55a32deb7430 msgr2=0x55a32debdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.016+0000 7f2228fb8740 1 -- 172.21.15.86:0/3897655714 shutdown_connections 2024-04-17T07:25:02.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.016+0000 7f2228fb8740 1 -- 172.21.15.86:0/3897655714 wait complete. 2024-04-17T07:25:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.018+0000 7f2228fb8740 1 -- start start 2024-04-17T07:25:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.018+0000 7f2228fb8740 1 Processor -- start 2024-04-17T07:25:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debbec0 0x55a32deca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debc8b0 0x55a32deca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f2228fb8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32dec1a20 0x55a32df36960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f2228fb8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f22201bae60 con 0x55a32dec1a20 2024-04-17T07:25:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f2228fb8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f22201ba790 con 0x55a32debbec0 2024-04-17T07:25:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f2228fb8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f22201b79c0 con 0x55a32debc8b0 2024-04-17T07:25:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f2228fb8740 10 client.0 _ll_register_callbacks cb 0x7f221801f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f221ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debc8b0 0x55a32deca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f221fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32dec1a20 0x55a32df36960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f221ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debc8b0 0x55a32deca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:48088/0 (socket says 172.21.15.86:48088) 2024-04-17T07:25:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.019+0000 7f221ea00640 1 -- 172.21.15.86:0/2776793062 learned_addr learned my addr 172.21.15.86:0/2776793062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.020+0000 7f221f400640 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debbec0 0x55a32deca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.020+0000 7f221fe00640 1 -- 172.21.15.86:0/2776793062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debc8b0 msgr2=0x55a32deca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.020+0000 7f221fe00640 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debc8b0 0x55a32deca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.020+0000 7f221fe00640 1 -- 172.21.15.86:0/2776793062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debbec0 msgr2=0x55a32deca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.020+0000 7f221fe00640 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debbec0 0x55a32deca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.020+0000 7f221fe00640 1 -- 172.21.15.86:0/2776793062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32decd920 con 0x55a32dec1a20 2024-04-17T07:25:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.021+0000 7f221fe00640 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32dec1a20 0x55a32df36960 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f221801e010 tx=0x7f2218003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4a201bafde0a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.021+0000 7f2208c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:25:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.021+0000 7f2208c00640 1 -- 172.21.15.86:0/2776793062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218046070 con 0x55a32dec1a20 2024-04-17T07:25:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.021+0000 7f2208c00640 1 -- 172.21.15.86:0/2776793062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f221802e040 con 0x55a32dec1a20 2024-04-17T07:25:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.022+0000 7f2228fb8740 1 -- 172.21.15.86:0/2776793062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221800fbc0 con 0x55a32dec1a20 2024-04-17T07:25:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.022+0000 7f2208c00640 1 -- 172.21.15.86:0/2776793062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2218036040 con 0x55a32dec1a20 2024-04-17T07:25:02.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.022+0000 7f2228fb8740 10 client.18147 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.025+0000 7f2228fb8740 1 -- 172.21.15.86:0/2776793062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a32df37e60 con 0x55a32dec1a20 2024-04-17T07:25:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.026+0000 7f2208c00640 1 -- 172.21.15.86:0/2776793062 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f221800fbc0 con 0x55a32dec1a20 2024-04-17T07:25:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.026+0000 7f2208c00640 1 -- 172.21.15.86:0/2776793062 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2218004290 con 0x55a32dec1a20 2024-04-17T07:25:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.027+0000 7f2228fb8740 10 client.18147 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.027+0000 7f2228fb8740 10 client.18147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.027+0000 7f2228fb8740 20 client.18147 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.027+0000 7f2228fb8740 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a32df38790 0x55a32df3abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.027+0000 7f2228fb8740 4 client.18147 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:25:02.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.027+0000 7f2228fb8740 1 -- 172.21.15.86:0/2776793062 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a32de29540 con 0x55a32df38790 2024-04-17T07:25:02.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.027+0000 7f221f400640 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a32df38790 0x55a32df3abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.029+0000 7f221f400640 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a32df38790 0x55a32df3abb0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365fce5b3e4b3406 server_cookie=3e2c081000db47b2 in_seq=0 out_seq=0 2024-04-17T07:25:02.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.029+0000 7f2208c00640 10 client.18147 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:02.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.032+0000 7f2208c00640 1 -- 172.21.15.86:0/2776793062 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a32de29540 con 0x55a32df38790 2024-04-17T07:25:02.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.032+0000 7f2208c00640 10 client.18147 handle_command_reply: tid=0 2024-04-17T07:25:02.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.032+0000 7f2228fb8740 1 client.18147 shutdown 2024-04-17T07:25:02.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.033+0000 7f2228fb8740 1 -- 172.21.15.86:0/2776793062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32dec1a20 msgr2=0x55a32df36960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.033+0000 7f2228fb8740 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32dec1a20 0x55a32df36960 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f221801e010 tx=0x7f2218003f10 comp rx=0 tx=0).stop 2024-04-17T07:25:02.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.034+0000 7f2228fb8740 1 -- 172.21.15.86:0/2776793062 shutdown_connections 2024-04-17T07:25:02.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.034+0000 7f2228fb8740 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a32df38790 0x55a32df3abb0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.034+0000 7f2228fb8740 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a32dec1a20 0x55a32df36960 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.034+0000 7f2228fb8740 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a32debc8b0 0x55a32deca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.034+0000 7f2228fb8740 1 --2- 172.21.15.86:0/2776793062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a32debbec0 0x55a32deca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.034+0000 7f2228fb8740 1 -- 172.21.15.86:0/2776793062 >> 172.21.15.86:0/2776793062 conn(0x55a32deb7430 msgr2=0x55a32dec61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:02.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.034+0000 7f2208c00640 0 client.18147 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:02.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.034+0000 7f2228fb8740 1 -- 172.21.15.86:0/2776793062 shutdown_connections 2024-04-17T07:25:02.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.034+0000 7f2228fb8740 1 -- 172.21.15.86:0/2776793062 wait complete. 2024-04-17T07:25:02.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.034+0000 7f2228fb8740 20 client.18147 trim_cache size 0 max 16384 2024-04-17T07:25:02.035 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:25:02.036 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:25:02.036 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "983d7830-f96e-4e6a-b84d-8d87f9e6cfd6", 2024-04-17T07:25:02.036 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:25:02.036 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:25:02.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.035+0000 7f2226200640 1 -- 172.21.15.86:0/1559420785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22040a3c30 msgr2=0x7f22040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.035+0000 7f2226200640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22040a3c30 0x7f22040a60f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2218020000 tx=0x7f221801ca70 comp rx=0 tx=0).stop 2024-04-17T07:25:02.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.035+0000 7f2226200640 1 -- 172.21.15.86:0/1559420785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f222010bb90 msgr2=0x7f22201a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.035+0000 7f2226200640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f222010bb90 0x7f22201a75c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f220c0097d0 tx=0x7f220c009c90 comp rx=0 tx=0).stop 2024-04-17T07:25:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.036+0000 7f2226200640 1 -- 172.21.15.86:0/1559420785 shutdown_connections 2024-04-17T07:25:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.036+0000 7f2226200640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22040a3c30 0x7f22040a60f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.036+0000 7f2226200640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f222010e650 0x7f22201ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.036+0000 7f2226200640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f222010bb90 0x7f22201a75c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.036+0000 7f2226200640 1 --2- 172.21.15.86:0/1559420785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22201091d0 0x7f22201a7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.036+0000 7f2226200640 1 -- 172.21.15.86:0/1559420785 >> 172.21.15.86:0/1559420785 conn(0x7f222006f190 msgr2=0x7f222010b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.036+0000 7f2226200640 1 -- 172.21.15.86:0/1559420785 shutdown_connections 2024-04-17T07:25:02.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.036+0000 7f2226200640 1 -- 172.21.15.86:0/1559420785 wait complete. 2024-04-17T07:25:02.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:25:02.074 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:25:02.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.411+0000 7f6e56a00640 1 -- 172.21.15.86:0/2848404544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e480aa190 msgr2=0x7f6e480aa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.411+0000 7f6e56a00640 1 --2- 172.21.15.86:0/2848404544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e480aa190 0x7f6e480aa610 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009900 tx=0x7f6e4002bc30 comp rx=0 tx=0).stop 2024-04-17T07:25:02.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.412+0000 7f6e56a00640 1 -- 172.21.15.86:0/2848404544 shutdown_connections 2024-04-17T07:25:02.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.412+0000 7f6e56a00640 1 --2- 172.21.15.86:0/2848404544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e480aab50 0x7f6e480b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.412+0000 7f6e56a00640 1 --2- 172.21.15.86:0/2848404544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e480aa190 0x7f6e480aa610 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.412+0000 7f6e56a00640 1 --2- 172.21.15.86:0/2848404544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e480a8fa0 0x7f6e480a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.412+0000 7f6e56a00640 1 -- 172.21.15.86:0/2848404544 >> 172.21.15.86:0/2848404544 conn(0x7f6e480a4530 msgr2=0x7f6e480a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.412+0000 7f6e56a00640 1 -- 172.21.15.86:0/2848404544 shutdown_connections 2024-04-17T07:25:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.412+0000 7f6e56a00640 1 -- 172.21.15.86:0/2848404544 wait complete. 2024-04-17T07:25:02.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.413+0000 7f6e56a00640 1 Processor -- start 2024-04-17T07:25:02.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.414+0000 7f6e56a00640 1 -- start start 2024-04-17T07:25:02.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.416+0000 7f6e56a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e480a8fa0 0x7f6e4813adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.416+0000 7f6e56a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e480aa190 0x7f6e4813b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.416+0000 7f6e56a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e480aab50 0x7f6e48142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.417+0000 7f6e56a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480b4b30 con 0x7f6e480aab50 2024-04-17T07:25:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.417+0000 7f6e55600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e480a8fa0 0x7f6e4813adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.417+0000 7f6e56000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e480aab50 0x7f6e48142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.417+0000 7f6e56000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e480aab50 0x7f6e48142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44920/0 (socket says 172.21.15.86:44920) 2024-04-17T07:25:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.417+0000 7f6e56000640 1 -- 172.21.15.86:0/2552274464 learned_addr learned my addr 172.21.15.86:0/2552274464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:02.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.417+0000 7f6e56a00640 1 -- 172.21.15.86:0/2552274464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480b49b0 con 0x7f6e480a8fa0 2024-04-17T07:25:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.417+0000 7f6e56a00640 1 -- 172.21.15.86:0/2552274464 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480b4cb0 con 0x7f6e480aa190 2024-04-17T07:25:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.417+0000 7f6e54c00640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e480aa190 0x7f6e4813b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.418+0000 7f6e56000640 1 -- 172.21.15.86:0/2552274464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e480aa190 msgr2=0x7f6e4813b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.418+0000 7f6e56000640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e480aa190 0x7f6e4813b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.418+0000 7f6e56000640 1 -- 172.21.15.86:0/2552274464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e480a8fa0 msgr2=0x7f6e4813adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.418+0000 7f6e56000640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e480a8fa0 0x7f6e4813adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.418+0000 7f6e56000640 1 -- 172.21.15.86:0/2552274464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e481539f0 con 0x7f6e480aab50 2024-04-17T07:25:02.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.418+0000 7f6e55600640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e480a8fa0 0x7f6e4813adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:02.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.419+0000 7f6e56000640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e480aab50 0x7f6e48142350 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6e50067ad0 tx=0x7f6e50073a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:02.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.420+0000 7f6e4e000640 1 -- 172.21.15.86:0/2552274464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50069220 con 0x7f6e480aab50 2024-04-17T07:25:02.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.420+0000 7f6e4e000640 1 -- 172.21.15.86:0/2552274464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e50075bb0 con 0x7f6e480aab50 2024-04-17T07:25:02.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.421+0000 7f6e56a00640 1 -- 172.21.15.86:0/2552274464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e48153ce0 con 0x7f6e480aab50 2024-04-17T07:25:02.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.421+0000 7f6e56a00640 1 -- 172.21.15.86:0/2552274464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e481541b0 con 0x7f6e480aab50 2024-04-17T07:25:02.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.423+0000 7f6e4e000640 1 -- 172.21.15.86:0/2552274464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50075900 con 0x7f6e480aab50 2024-04-17T07:25:02.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.429+0000 7f6e5ad77740 1 Processor -- start 2024-04-17T07:25:02.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.430+0000 7f6e4e000640 1 -- 172.21.15.86:0/2552274464 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f6e50068410 con 0x7f6e480aab50 2024-04-17T07:25:02.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.431+0000 7f6e5ad77740 1 -- start start 2024-04-17T07:25:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.431+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3066b370 0x563f3066b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.431+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066bd60 0x563f30670860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.431+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 0x563f30673360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.431+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f305d2120 con 0x563f30670ed0 2024-04-17T07:25:02.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.431+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563f30578340 con 0x563f3066b370 2024-04-17T07:25:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.431+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f305ce0e0 con 0x563f3066bd60 2024-04-17T07:25:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.432+0000 7f6e54c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 0x563f30673360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.432+0000 7f6e54c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3066b370 0x563f3066b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.432+0000 7f6e54c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 0x563f30673360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44924/0 (socket says 172.21.15.86:44924) 2024-04-17T07:25:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.432+0000 7f6e54c00640 1 -- 172.21.15.86:0/2854964150 learned_addr learned my addr 172.21.15.86:0/2854964150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.432+0000 7f6e55600640 1 --2- 172.21.15.86:0/2854964150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066bd60 0x563f30670860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.433+0000 7f6e4e000640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c00 0x7f6e380a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.433+0000 7f6e55600640 1 -- 172.21.15.86:0/2854964150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3066b370 msgr2=0x563f3066b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.433+0000 7f6e55600640 1 --2- 172.21.15.86:0/2854964150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3066b370 0x563f3066b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.433+0000 7f6e55600640 1 -- 172.21.15.86:0/2854964150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 msgr2=0x563f30673360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.433+0000 7f6e55600640 1 --2- 172.21.15.86:0/2854964150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 0x563f30673360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.433+0000 7f6e55600640 1 -- 172.21.15.86:0/2854964150 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f303ae7d0 con 0x563f3066bd60 2024-04-17T07:25:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.433+0000 7f6e4e000640 1 -- 172.21.15.86:0/2552274464 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6e501347f0 con 0x7f6e480aab50 2024-04-17T07:25:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.434+0000 7f6e56000640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c00 0x7f6e380a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.434+0000 7f6e55600640 1 --2- 172.21.15.86:0/2854964150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066bd60 0x563f30670860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6e480a9ff0 tx=0x7f6e4400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9ae8cfb1e9c90eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.435+0000 7f6e3e600640 1 -- 172.21.15.86:0/2854964150 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44015070 con 0x563f3066bd60 2024-04-17T07:25:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.435+0000 7f6e3e600640 1 -- 172.21.15.86:0/2854964150 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e4401a910 con 0x563f3066bd60 2024-04-17T07:25:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.436+0000 7f6e3e600640 1 -- 172.21.15.86:0/2854964150 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44011990 con 0x563f3066bd60 2024-04-17T07:25:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.436+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2854964150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066bd60 msgr2=0x563f30670860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.436+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2854964150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066bd60 0x563f30670860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6e480a9ff0 tx=0x7f6e4400f730 comp rx=0 tx=0).stop 2024-04-17T07:25:02.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.437+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2854964150 shutdown_connections 2024-04-17T07:25:02.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.437+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2854964150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 0x563f30673360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.437+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2854964150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066bd60 0x563f30670860 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.437+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2854964150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f3066b370 0x563f3066b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.437+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2854964150 >> 172.21.15.86:0/2854964150 conn(0x563f306668e0 msgr2=0x563f30666ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:02.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.437+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2854964150 shutdown_connections 2024-04-17T07:25:02.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.438+0000 7f6e56000640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c00 0x7f6e380a60c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6e500fd150 tx=0x7f6e50073f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:02.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.438+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2854964150 wait complete. 2024-04-17T07:25:02.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.438+0000 7f6e5ad77740 1 -- start start 2024-04-17T07:25:02.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.439+0000 7f6e5ad77740 1 Processor -- start 2024-04-17T07:25:02.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.439+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30675930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.439+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066bd60 0x563f30675e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.439+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 0x563f3067ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.439+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563f30673ed0 con 0x563f3066bd60 2024-04-17T07:25:02.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.440+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563f3058b710 con 0x563f30670ed0 2024-04-17T07:25:02.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.440+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f30674050 con 0x563f3066b370 2024-04-17T07:25:02.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.440+0000 7f6e5ad77740 10 client.0 _ll_register_callbacks cb 0x563f30586af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:02.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.441+0000 7f6e54c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 0x563f3067ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.441+0000 7f6e54c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30675930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.441+0000 7f6e54c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30675930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:48116/0 (socket says 172.21.15.86:48116) 2024-04-17T07:25:02.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.441+0000 7f6e54c00640 1 -- 172.21.15.86:0/1798613473 learned_addr learned my addr 172.21.15.86:0/1798613473 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:02.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.441+0000 7f6e55600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066bd60 0x563f30675e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.442+0000 7f6e55600640 1 -- 172.21.15.86:0/1798613473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 msgr2=0x563f30675930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.442+0000 7f6e55600640 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30675930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.442+0000 7f6e55600640 1 -- 172.21.15.86:0/1798613473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 msgr2=0x563f3067ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.442+0000 7f6e55600640 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 0x563f3067ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.442+0000 7f6e55600640 1 -- 172.21.15.86:0/1798613473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f302e43f0 con 0x563f3066bd60 2024-04-17T07:25:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.442+0000 7f6e54c00640 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30675930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:02.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.442+0000 7f6e55600640 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066bd60 0x563f30675e70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x563f30667850 tx=0x7f6e4401a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9923b9d58f420d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:02.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.442+0000 7f6e32a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:25:02.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.443+0000 7f6e32a00640 1 -- 172.21.15.86:0/1798613473 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44028060 con 0x563f3066bd60 2024-04-17T07:25:02.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.443+0000 7f6e32a00640 1 -- 172.21.15.86:0/1798613473 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6e44005040 con 0x563f3066bd60 2024-04-17T07:25:02.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.443+0000 7f6e5ad77740 1 -- 172.21.15.86:0/1798613473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e4401a910 con 0x563f3066bd60 2024-04-17T07:25:02.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.443+0000 7f6e32a00640 1 -- 172.21.15.86:0/1798613473 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e44015040 con 0x563f3066bd60 2024-04-17T07:25:02.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.444+0000 7f6e5ad77740 10 client.18159 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:02.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.444+0000 7f6e5ad77740 1 -- 172.21.15.86:0/1798613473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f302a40f0 con 0x563f3066bd60 2024-04-17T07:25:02.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.444+0000 7f6e32a00640 1 -- 172.21.15.86:0/1798613473 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e4401a910 con 0x563f3066bd60 2024-04-17T07:25:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.444+0000 7f6e32a00640 1 -- 172.21.15.86:0/1798613473 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6e44022030 con 0x563f3066bd60 2024-04-17T07:25:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.445+0000 7f6e5ad77740 10 client.18159 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.445+0000 7f6e5ad77740 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.445+0000 7f6e5ad77740 20 client.18159 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.445+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563f306814a0 0x563f30683890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.445+0000 7f6e5ad77740 4 client.18159 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.445+0000 7f6e5ad77740 1 -- 172.21.15.86:0/1798613473 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f305dfb40 con 0x563f306814a0 2024-04-17T07:25:02.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.446+0000 7f6e54c00640 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563f306814a0 0x563f30683890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.447+0000 7f6e54c00640 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563f306814a0 0x563f30683890 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb53ce7a89124e6 server_cookie=5bdf7f5b6cc807c9 in_seq=0 out_seq=0 2024-04-17T07:25:02.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.447+0000 7f6e32a00640 10 client.18159 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:02.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.450+0000 7f6e32a00640 1 -- 172.21.15.86:0/1798613473 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f305dfb40 con 0x563f306814a0 2024-04-17T07:25:02.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.450+0000 7f6e32a00640 10 client.18159 handle_command_reply: tid=0 2024-04-17T07:25:02.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.450+0000 7f6e5ad77740 1 client.18159 shutdown 2024-04-17T07:25:02.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.452+0000 7f6e5ad77740 1 -- 172.21.15.86:0/1798613473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066bd60 msgr2=0x563f30675e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.452+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066bd60 0x563f30675e70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x563f30667850 tx=0x7f6e4401a690 comp rx=0 tx=0).stop 2024-04-17T07:25:02.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.452+0000 7f6e5ad77740 1 -- 172.21.15.86:0/1798613473 shutdown_connections 2024-04-17T07:25:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.452+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563f306814a0 0x563f30683890 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.452+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 0x563f3067ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.452+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066bd60 0x563f30675e70 secure :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x563f30667850 tx=0x7f6e4401a690 comp rx=0 tx=0).stop 2024-04-17T07:25:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.452+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/1798613473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30675930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.452+0000 7f6e5ad77740 1 -- 172.21.15.86:0/1798613473 >> 172.21.15.86:0/1798613473 conn(0x563f306668e0 msgr2=0x563f30665ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.452+0000 7f6e32a00640 0 client.18159 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.456+0000 7f6e5ad77740 1 -- 172.21.15.86:0/1798613473 shutdown_connections 2024-04-17T07:25:02.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.456+0000 7f6e5ad77740 1 -- 172.21.15.86:0/1798613473 wait complete. 2024-04-17T07:25:02.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.457+0000 7f6e5ad77740 20 client.18159 trim_cache size 0 max 16384 2024-04-17T07:25:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.502+0000 7f6e5ad77740 1 Processor -- start 2024-04-17T07:25:02.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.502+0000 7f6e5ad77740 1 -- start start 2024-04-17T07:25:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.503+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066b370 0x563f3066d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.503+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 0x563f3066d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.503+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f30674c20 0x563f306750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.503+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4815e6b0 con 0x563f3066b370 2024-04-17T07:25:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.503+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e481601f0 con 0x563f30670ed0 2024-04-17T07:25:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.503+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4815ed80 con 0x563f30674c20 2024-04-17T07:25:02.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.503+0000 7f6e54c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066b370 0x563f3066d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.503+0000 7f6e54c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066b370 0x563f3066d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44930/0 (socket says 172.21.15.86:44930) 2024-04-17T07:25:02.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.503+0000 7f6e54c00640 1 -- 172.21.15.86:0/2607988917 learned_addr learned my addr 172.21.15.86:0/2607988917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:02.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.503+0000 7f6e55600640 1 --2- 172.21.15.86:0/2607988917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 0x563f3066d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.505+0000 7f6e54c00640 1 --2- 172.21.15.86:0/2607988917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f30674c20 0x563f306750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.506+0000 7f6e54c00640 1 -- 172.21.15.86:0/2607988917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f30674c20 msgr2=0x563f306750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.506+0000 7f6e54c00640 1 --2- 172.21.15.86:0/2607988917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f30674c20 0x563f306750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.506+0000 7f6e54c00640 1 -- 172.21.15.86:0/2607988917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 msgr2=0x563f3066d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.506+0000 7f6e54c00640 1 --2- 172.21.15.86:0/2607988917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 0x563f3066d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.506+0000 7f6e54c00640 1 -- 172.21.15.86:0/2607988917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f30683b90 con 0x563f3066b370 2024-04-17T07:25:02.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.506+0000 7f6e55600640 1 --2- 172.21.15.86:0/2607988917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 0x563f3066d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:02.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.507+0000 7f6e54c00640 1 --2- 172.21.15.86:0/2607988917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066b370 0x563f3066d1c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005ab0 tx=0x7f6e400095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fa79d888c45a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:02.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.508+0000 7f6e3dc00640 1 -- 172.21.15.86:0/2607988917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4004d2b0 con 0x563f3066b370 2024-04-17T07:25:02.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.508+0000 7f6e3dc00640 1 -- 172.21.15.86:0/2607988917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e40053030 con 0x563f3066b370 2024-04-17T07:25:02.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.508+0000 7f6e3dc00640 1 -- 172.21.15.86:0/2607988917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40003c40 con 0x563f3066b370 2024-04-17T07:25:02.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.509+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2607988917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066b370 msgr2=0x563f3066d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.509+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2607988917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066b370 0x563f3066d1c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005ab0 tx=0x7f6e400095e0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.509+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2607988917 shutdown_connections 2024-04-17T07:25:02.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.509+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2607988917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f30674c20 0x563f306750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.509+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2607988917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30670ed0 0x563f3066d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.509+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2607988917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f3066b370 0x563f3066d1c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.509+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2607988917 >> 172.21.15.86:0/2607988917 conn(0x563f306668e0 msgr2=0x563f30666cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:02.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.510+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2607988917 shutdown_connections 2024-04-17T07:25:02.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.510+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2607988917 wait complete. 2024-04-17T07:25:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.510+0000 7f6e5ad77740 1 -- start start 2024-04-17T07:25:02.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.511+0000 7f6e5ad77740 1 Processor -- start 2024-04-17T07:25:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.512+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30679da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.512+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 0x563f3067a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.512+0000 7f6e5ad77740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30674c20 0x563f3067b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.512+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4815fb20 con 0x563f30670ed0 2024-04-17T07:25:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.512+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4815f450 con 0x563f30674c20 2024-04-17T07:25:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.512+0000 7f6e5ad77740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4815c680 con 0x563f3066b370 2024-04-17T07:25:02.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.512+0000 7f6e5ad77740 10 client.0 _ll_register_callbacks cb 0x7f6e4400bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.512+0000 7f6e55600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 0x563f3067a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.512+0000 7f6e55600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 0x563f3067a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44940/0 (socket says 172.21.15.86:44940) 2024-04-17T07:25:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.513+0000 7f6e55600640 1 -- 172.21.15.86:0/2824030470 learned_addr learned my addr 172.21.15.86:0/2824030470 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:02.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.513+0000 7f6e54c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30679da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.513+0000 7f6e54c00640 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30674c20 0x563f3067b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.515+0000 7f6e54c00640 1 -- 172.21.15.86:0/2824030470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30674c20 msgr2=0x563f3067b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.515+0000 7f6e54c00640 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30674c20 0x563f3067b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.515+0000 7f6e54c00640 1 -- 172.21.15.86:0/2824030470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 msgr2=0x563f3067a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.515+0000 7f6e54c00640 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 0x563f3067a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.515+0000 7f6e54c00640 1 -- 172.21.15.86:0/2824030470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f3067b870 con 0x563f3066b370 2024-04-17T07:25:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.515+0000 7f6e54c00640 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30679da0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x563f30673af0 tx=0x7f6e40009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f580fb09f7e8b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:02.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.515+0000 7f6e32a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:02.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.516+0000 7f6e32a00640 1 -- 172.21.15.86:0/2824030470 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4005f070 con 0x563f3066b370 2024-04-17T07:25:02.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.516+0000 7f6e32a00640 1 -- 172.21.15.86:0/2824030470 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6e40003bc0 con 0x563f3066b370 2024-04-17T07:25:02.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.516+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2824030470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e40053030 con 0x563f3066b370 2024-04-17T07:25:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.517+0000 7f6e32a00640 1 -- 172.21.15.86:0/2824030470 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e40003dc0 con 0x563f3066b370 2024-04-17T07:25:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.517+0000 7f6e5ad77740 10 client.27100 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:02.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.517+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2824030470 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f306e7b40 con 0x563f3066b370 2024-04-17T07:25:02.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.518+0000 7f6e32a00640 1 -- 172.21.15.86:0/2824030470 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e40053030 con 0x563f3066b370 2024-04-17T07:25:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.519+0000 7f6e32a00640 1 -- 172.21.15.86:0/2824030470 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6e40040040 con 0x563f3066b370 2024-04-17T07:25:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.519+0000 7f6e5ad77740 10 client.27100 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.519+0000 7f6e5ad77740 10 client.27100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.519+0000 7f6e5ad77740 20 client.27100 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.519+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563f306e8330 0x563f306ea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:02.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.519+0000 7f6e5ad77740 4 client.27100 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:25:02.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.519+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2824030470 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f305d99b0 con 0x563f306e8330 2024-04-17T07:25:02.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.520+0000 7f6e55600640 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563f306e8330 0x563f306ea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.522+0000 7f6e55600640 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563f306e8330 0x563f306ea720 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c291da90e25633 server_cookie=d3626ee3f9362339 in_seq=0 out_seq=0 2024-04-17T07:25:02.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.522+0000 7f6e32a00640 10 client.27100 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.523+0000 7f6e32a00640 1 -- 172.21.15.86:0/2824030470 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563f305d99b0 con 0x563f306e8330 2024-04-17T07:25:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.523+0000 7f6e32a00640 10 client.27100 handle_command_reply: tid=0 2024-04-17T07:25:02.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.524+0000 7f6e5ad77740 1 client.27100 shutdown 2024-04-17T07:25:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.524+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2824030470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 msgr2=0x563f30679da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.524+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30679da0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x563f30673af0 tx=0x7f6e40009650 comp rx=0 tx=0).stop 2024-04-17T07:25:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.525+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2824030470 shutdown_connections 2024-04-17T07:25:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.525+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563f306e8330 0x563f306ea720 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.525+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563f30674c20 0x563f3067b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.525+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563f30670ed0 0x563f3067a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.525+0000 7f6e5ad77740 1 --2- 172.21.15.86:0/2824030470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f3066b370 0x563f30679da0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.525+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2824030470 >> 172.21.15.86:0/2824030470 conn(0x563f306668e0 msgr2=0x563f30669b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:02.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.525+0000 7f6e32a00640 0 client.27100 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:02.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.526+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2824030470 shutdown_connections 2024-04-17T07:25:02.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.526+0000 7f6e5ad77740 1 -- 172.21.15.86:0/2824030470 wait complete. 2024-04-17T07:25:02.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.526+0000 7f6e5ad77740 20 client.27100 trim_cache size 0 max 16384 2024-04-17T07:25:02.527 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:25:02.527 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:25:02.527 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:25:02.527 INFO:teuthology.orchestra.run.smithi086.stdout: "983d7830-f96e-4e6a-b84d-8d87f9e6cfd6": { 2024-04-17T07:25:02.527 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:25:02.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "983d7830-f96e-4e6a-b84d-8d87f9e6cfd6", 2024-04-17T07:25:02.527 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:25:02.528 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:25:02.528 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:25:02.528 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:25:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.528+0000 7f6e56a00640 1 -- 172.21.15.86:0/2552274464 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c00 msgr2=0x7f6e380a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.528+0000 7f6e56a00640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c00 0x7f6e380a60c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6e500fd150 tx=0x7f6e50073f90 comp rx=0 tx=0).stop 2024-04-17T07:25:02.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.528+0000 7f6e56a00640 1 -- 172.21.15.86:0/2552274464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e480aab50 msgr2=0x7f6e48142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:02.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.528+0000 7f6e56a00640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e480aab50 0x7f6e48142350 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6e50067ad0 tx=0x7f6e50073a10 comp rx=0 tx=0).stop 2024-04-17T07:25:02.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.529+0000 7f6e56a00640 1 -- 172.21.15.86:0/2552274464 shutdown_connections 2024-04-17T07:25:02.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.529+0000 7f6e56a00640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e380a3c00 0x7f6e380a60c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.529+0000 7f6e56a00640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e480aab50 0x7f6e48142350 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.529+0000 7f6e56a00640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e480aa190 0x7f6e4813b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.529+0000 7f6e56a00640 1 --2- 172.21.15.86:0/2552274464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e480a8fa0 0x7f6e4813adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:02.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.529+0000 7f6e56a00640 1 -- 172.21.15.86:0/2552274464 >> 172.21.15.86:0/2552274464 conn(0x7f6e480a4530 msgr2=0x7f6e4801ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:02.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.529+0000 7f6e56a00640 1 -- 172.21.15.86:0/2552274464 shutdown_connections 2024-04-17T07:25:02.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:02.529+0000 7f6e56a00640 1 -- 172.21.15.86:0/2552274464 wait complete. 2024-04-17T07:25:02.566 INFO:tasks.cephfs.filesystem:scrub status for tag:983d7830-f96e-4e6a-b84d-8d87f9e6cfd6 - {'path': '/', 'tag': '983d7830-f96e-4e6a-b84d-8d87f9e6cfd6', 'options': 'recursive,force'} 2024-04-17T07:25:02.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:02 smithi086 ceph-mon[31733]: pgmap v760: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 662 op/s 2024-04-17T07:25:02.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:02 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:25:02.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:02 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:02.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:02 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:25:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:02 smithi098 ceph-mon[35418]: pgmap v760: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 662 op/s 2024-04-17T07:25:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:02 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:25:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:02 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:03.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:02 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:25:03.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:02 smithi103 ceph-mon[35495]: pgmap v760: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 662 op/s 2024-04-17T07:25:03.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:02 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:25:03.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:02 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:03.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:02 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:25:04.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:03 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:04.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:03 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:04.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:03 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 340.0 s, 612.8 tps, lat 163.515 ms stddev 62.990 2024-04-17T07:25:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:04 smithi098 ceph-mon[35418]: pgmap v761: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 645 op/s 2024-04-17T07:25:05.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:04 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:25:05.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:04 smithi103 ceph-mon[35495]: pgmap v761: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 645 op/s 2024-04-17T07:25:05.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:04 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:25:05.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:04 smithi086 ceph-mon[31733]: pgmap v761: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 645 op/s 2024-04-17T07:25:05.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:04 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:25:06.883 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:06 smithi086 ceph-mon[31733]: pgmap v762: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 552 op/s 2024-04-17T07:25:07.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:06 smithi098 ceph-mon[35418]: pgmap v762: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 552 op/s 2024-04-17T07:25:07.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:06 smithi103 ceph-mon[35495]: pgmap v762: 129 pgs: 129 active+clean; 1.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 552 op/s 2024-04-17T07:25:09.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:08 smithi098 ceph-mon[35418]: pgmap v763: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.5 MiB/s wr, 617 op/s 2024-04-17T07:25:09.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:08 smithi103 ceph-mon[35495]: pgmap v763: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.5 MiB/s wr, 617 op/s 2024-04-17T07:25:09.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:08 smithi086 ceph-mon[31733]: pgmap v763: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.5 MiB/s wr, 617 op/s 2024-04-17T07:25:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 345.0 s, 644.8 tps, lat 154.752 ms stddev 56.908 2024-04-17T07:25:11.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:10 smithi098 ceph-mon[35418]: pgmap v764: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 585 op/s 2024-04-17T07:25:11.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:10 smithi103 ceph-mon[35495]: pgmap v764: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 585 op/s 2024-04-17T07:25:11.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:10 smithi086 ceph-mon[31733]: pgmap v764: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 585 op/s 2024-04-17T07:25:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:12 smithi098 ceph-mon[35418]: pgmap v765: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 655 op/s 2024-04-17T07:25:13.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:12 smithi103 ceph-mon[35495]: pgmap v765: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 655 op/s 2024-04-17T07:25:13.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:12 smithi086 ceph-mon[31733]: pgmap v765: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 655 op/s 2024-04-17T07:25:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 350.0 s, 545.6 tps, lat 183.437 ms stddev 104.726 2024-04-17T07:25:15.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:14 smithi098 ceph-mon[35418]: pgmap v766: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.4 MiB/s wr, 622 op/s 2024-04-17T07:25:15.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:14 smithi103 ceph-mon[35495]: pgmap v766: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.4 MiB/s wr, 622 op/s 2024-04-17T07:25:15.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:14 smithi086 ceph-mon[31733]: pgmap v766: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.4 MiB/s wr, 622 op/s 2024-04-17T07:25:17.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:16 smithi098 ceph-mon[35418]: pgmap v767: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.9 MiB/s wr, 559 op/s 2024-04-17T07:25:17.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:16 smithi103 ceph-mon[35495]: pgmap v767: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.9 MiB/s wr, 559 op/s 2024-04-17T07:25:17.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:16 smithi086 ceph-mon[31733]: pgmap v767: 129 pgs: 129 active+clean; 1.3 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.9 MiB/s wr, 559 op/s 2024-04-17T07:25:19.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:18 smithi098 ceph-mon[35418]: pgmap v768: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 658 op/s 2024-04-17T07:25:19.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:18 smithi103 ceph-mon[35495]: pgmap v768: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 658 op/s 2024-04-17T07:25:19.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:18 smithi086 ceph-mon[31733]: pgmap v768: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 658 op/s 2024-04-17T07:25:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 355.0 s, 709.4 tps, lat 140.927 ms stddev 40.214 2024-04-17T07:25:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:20 smithi098 ceph-mon[35418]: pgmap v769: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 603 op/s 2024-04-17T07:25:21.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:20 smithi103 ceph-mon[35495]: pgmap v769: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 603 op/s 2024-04-17T07:25:21.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:20 smithi086 ceph-mon[31733]: pgmap v769: 129 pgs: 129 active+clean; 1.3 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 603 op/s 2024-04-17T07:25:22.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:21 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:25:22.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:21 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:25:22.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:21 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:25:23.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:22 smithi098 ceph-mon[35418]: pgmap v770: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 685 op/s 2024-04-17T07:25:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:22 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:22 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:23.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:22 smithi103 ceph-mon[35495]: pgmap v770: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 685 op/s 2024-04-17T07:25:23.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:22 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:23.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:22 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:23.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:22 smithi086 ceph-mon[31733]: pgmap v770: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 685 op/s 2024-04-17T07:25:23.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:23.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:22 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:24.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 360.0 s, 628.4 tps, lat 158.757 ms stddev 43.236 2024-04-17T07:25:25.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:25 smithi098 ceph-mon[35418]: pgmap v771: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 648 op/s 2024-04-17T07:25:25.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:25 smithi103 ceph-mon[35495]: pgmap v771: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 648 op/s 2024-04-17T07:25:25.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:24 smithi086 ceph-mon[31733]: pgmap v771: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 648 op/s 2024-04-17T07:25:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:26 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:26 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:26 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:26 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:26 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:26 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:26 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:26 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:26 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:26 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:26 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.530 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:26 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:26.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:26 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:27.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:27 smithi098 ceph-mon[35418]: pgmap v772: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 575 op/s 2024-04-17T07:25:27.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:27 smithi103 ceph-mon[35495]: pgmap v772: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 575 op/s 2024-04-17T07:25:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:27 smithi086 ceph-mon[31733]: pgmap v772: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 575 op/s 2024-04-17T07:25:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:25:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:25:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:25:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:25:28.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:25:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:25:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:25:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 365.0 s, 560.6 tps, lat 170.840 ms stddev 54.710 2024-04-17T07:25:29.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:29 smithi098 ceph-mon[35418]: pgmap v773: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 667 op/s 2024-04-17T07:25:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:29 smithi103 ceph-mon[35495]: pgmap v773: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 667 op/s 2024-04-17T07:25:29.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:29 smithi086 ceph-mon[31733]: pgmap v773: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 667 op/s 2024-04-17T07:25:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:31 smithi098 ceph-mon[35418]: pgmap v774: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.1 MiB/s wr, 594 op/s 2024-04-17T07:25:31.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:31 smithi103 ceph-mon[35495]: pgmap v774: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.1 MiB/s wr, 594 op/s 2024-04-17T07:25:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:31 smithi086 ceph-mon[31733]: pgmap v774: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.1 MiB/s wr, 594 op/s 2024-04-17T07:25:32.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:25:32.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:25:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.884+0000 7fc954a00640 1 -- 172.21.15.86:0/3969894675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948094c80 msgr2=0x7fc9480950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.884+0000 7fc954a00640 1 --2- 172.21.15.86:0/3969894675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948094c80 0x7fc9480950a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc944009860 tx=0x7fc94402bb70 comp rx=0 tx=0).stop 2024-04-17T07:25:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.884+0000 7fc954a00640 1 -- 172.21.15.86:0/3969894675 shutdown_connections 2024-04-17T07:25:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.884+0000 7fc954a00640 1 --2- 172.21.15.86:0/3969894675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc948096830 0x7fc948004cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.884+0000 7fc954a00640 1 --2- 172.21.15.86:0/3969894675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc948095e70 0x7fc9480962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.884+0000 7fc954a00640 1 --2- 172.21.15.86:0/3969894675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948094c80 0x7fc9480950a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.884+0000 7fc954a00640 1 -- 172.21.15.86:0/3969894675 >> 172.21.15.86:0/3969894675 conn(0x7fc9480901d0 msgr2=0x7fc948092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:32.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.885+0000 7fc954a00640 1 -- 172.21.15.86:0/3969894675 shutdown_connections 2024-04-17T07:25:32.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.885+0000 7fc954a00640 1 -- 172.21.15.86:0/3969894675 wait complete. 2024-04-17T07:25:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.887+0000 7fc954a00640 1 Processor -- start 2024-04-17T07:25:32.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.887+0000 7fc954a00640 1 -- start start 2024-04-17T07:25:32.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.888+0000 7fc954a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc948094c80 0x7fc94812c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.888+0000 7fc954a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc948095e70 0x7fc94812c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.888+0000 7fc954a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948096830 0x7fc948133660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.888+0000 7fc954a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9480a1890 con 0x7fc948094c80 2024-04-17T07:25:32.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.888+0000 7fc954a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc9480a1710 con 0x7fc948095e70 2024-04-17T07:25:32.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.888+0000 7fc954a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9480a1a10 con 0x7fc948096830 2024-04-17T07:25:32.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.889+0000 7fc94f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc948094c80 0x7fc94812c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.889+0000 7fc94ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc948095e70 0x7fc94812c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.889+0000 7fc94ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc948095e70 0x7fc94812c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50470/0 (socket says 172.21.15.86:50470) 2024-04-17T07:25:32.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.889+0000 7fc94ea00640 1 -- 172.21.15.86:0/4067412566 learned_addr learned my addr 172.21.15.86:0/4067412566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:32.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.889+0000 7fc94fe00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948096830 0x7fc948133660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.890+0000 7fc94ea00640 1 -- 172.21.15.86:0/4067412566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948096830 msgr2=0x7fc948133660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:32.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.890+0000 7fc94ea00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948096830 0x7fc948133660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.890+0000 7fc94ea00640 1 -- 172.21.15.86:0/4067412566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc948094c80 msgr2=0x7fc94812c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:32.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.890+0000 7fc94ea00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc948094c80 0x7fc94812c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.890+0000 7fc94ea00640 1 -- 172.21.15.86:0/4067412566 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc948133cd0 con 0x7fc948095e70 2024-04-17T07:25:32.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.890+0000 7fc94fe00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948096830 0x7fc948133660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:32.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.890+0000 7fc94ea00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc948095e70 0x7fc94812c5f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00d950 tx=0x7fc93c00dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:32.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.891+0000 7fc943e00640 1 -- 172.21.15.86:0/4067412566 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c017070 con 0x7fc948095e70 2024-04-17T07:25:32.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.891+0000 7fc954a00640 1 -- 172.21.15.86:0/4067412566 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc948133fc0 con 0x7fc948095e70 2024-04-17T07:25:32.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.891+0000 7fc943e00640 1 -- 172.21.15.86:0/4067412566 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc93c00de80 con 0x7fc948095e70 2024-04-17T07:25:32.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.891+0000 7fc943e00640 1 -- 172.21.15.86:0/4067412566 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00f620 con 0x7fc948095e70 2024-04-17T07:25:32.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.891+0000 7fc954a00640 1 -- 172.21.15.86:0/4067412566 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9481344d0 con 0x7fc948095e70 2024-04-17T07:25:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.897+0000 7fc958dda740 1 Processor -- start 2024-04-17T07:25:32.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.897+0000 7fc958dda740 1 -- start start 2024-04-17T07:25:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.897+0000 7fc943e00640 1 -- 172.21.15.86:0/4067412566 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc93c00f7c0 con 0x7fc948095e70 2024-04-17T07:25:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.898+0000 7fc958dda740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0170 0x5618836c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.898+0000 7fc958dda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0b60 0x5618836c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.898+0000 7fc958dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 0x5618836c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.898+0000 7fc958dda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56188361e120 con 0x5618836c0b60 2024-04-17T07:25:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.898+0000 7fc958dda740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5618835c4340 con 0x5618836c0170 2024-04-17T07:25:32.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.898+0000 7fc958dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56188361a0e0 con 0x5618836c5760 2024-04-17T07:25:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.899+0000 7fc94fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0170 0x5618836c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.899+0000 7fc94ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 0x5618836c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.899+0000 7fc94ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 0x5618836c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36014/0 (socket says 172.21.15.86:36014) 2024-04-17T07:25:32.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.899+0000 7fc94ea00640 1 -- 172.21.15.86:0/2452216335 learned_addr learned my addr 172.21.15.86:0/2452216335 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:32.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.900+0000 7fc94f400640 1 --2- 172.21.15.86:0/2452216335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0b60 0x5618836c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.900+0000 7fc94fe00640 1 -- 172.21.15.86:0/2452216335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 msgr2=0x5618836c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:32.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.900+0000 7fc94fe00640 1 --2- 172.21.15.86:0/2452216335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 0x5618836c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.900+0000 7fc94fe00640 1 -- 172.21.15.86:0/2452216335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0b60 msgr2=0x5618836c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:32.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.900+0000 7fc94fe00640 1 --2- 172.21.15.86:0/2452216335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0b60 0x5618836c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.900+0000 7fc94fe00640 1 -- 172.21.15.86:0/2452216335 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618833fa7d0 con 0x5618836c0170 2024-04-17T07:25:32.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.900+0000 7fc94ea00640 1 --2- 172.21.15.86:0/2452216335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 0x5618836c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:32.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.901+0000 7fc943e00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9380a3c30 0x7fc9380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.902+0000 7fc94fe00640 1 --2- 172.21.15.86:0/2452216335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0170 0x5618836c0590 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc950000b60 tx=0x7fc950050d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee4dc4f2ae46cc2c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:32.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.901+0000 7fc943e00640 1 -- 172.21.15.86:0/4067412566 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc93c0d0310 con 0x7fc948095e70 2024-04-17T07:25:32.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.902+0000 7fc94f400640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9380a3c30 0x7fc9380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.903+0000 7fc937e00640 1 -- 172.21.15.86:0/2452216335 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950079830 con 0x5618836c0170 2024-04-17T07:25:32.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.903+0000 7fc937e00640 1 -- 172.21.15.86:0/2452216335 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc95007a420 con 0x5618836c0170 2024-04-17T07:25:32.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.903+0000 7fc937e00640 1 -- 172.21.15.86:0/2452216335 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950077670 con 0x5618836c0170 2024-04-17T07:25:32.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.904+0000 7fc958dda740 1 -- 172.21.15.86:0/2452216335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0170 msgr2=0x5618836c0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:32.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.904+0000 7fc958dda740 1 --2- 172.21.15.86:0/2452216335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0170 0x5618836c0590 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc950000b60 tx=0x7fc950050d10 comp rx=0 tx=0).stop 2024-04-17T07:25:32.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.905+0000 7fc94f400640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9380a3c30 0x7fc9380a60f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc944009830 tx=0x7fc944034980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:32.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.905+0000 7fc958dda740 1 -- 172.21.15.86:0/2452216335 shutdown_connections 2024-04-17T07:25:32.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.905+0000 7fc958dda740 1 --2- 172.21.15.86:0/2452216335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 0x5618836c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.905+0000 7fc958dda740 1 --2- 172.21.15.86:0/2452216335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0b60 0x5618836c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.905+0000 7fc958dda740 1 --2- 172.21.15.86:0/2452216335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0170 0x5618836c0590 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.905+0000 7fc958dda740 1 -- 172.21.15.86:0/2452216335 >> 172.21.15.86:0/2452216335 conn(0x5618836b2860 msgr2=0x5618836b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.906+0000 7fc958dda740 1 -- 172.21.15.86:0/2452216335 shutdown_connections 2024-04-17T07:25:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.906+0000 7fc958dda740 1 -- 172.21.15.86:0/2452216335 wait complete. 2024-04-17T07:25:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.907+0000 7fc958dda740 1 -- start start 2024-04-17T07:25:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.907+0000 7fc958dda740 1 Processor -- start 2024-04-17T07:25:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.909+0000 7fc958dda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0170 0x5618836ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.909+0000 7fc958dda740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 0x5618836ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.909+0000 7fc958dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 0x5618836d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.910+0000 7fc958dda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5618836c86f0 con 0x5618836c0170 2024-04-17T07:25:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.910+0000 7fc958dda740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5618835d7710 con 0x5618836c0b60 2024-04-17T07:25:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.910+0000 7fc958dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618836c8870 con 0x5618836c5760 2024-04-17T07:25:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.910+0000 7fc94ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 0x5618836ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.910+0000 7fc94f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0170 0x5618836ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.910+0000 7fc958dda740 10 client.0 _ll_register_callbacks cb 0x5618835d2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.910+0000 7fc94ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 0x5618836ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50488/0 (socket says 172.21.15.86:50488) 2024-04-17T07:25:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.910+0000 7fc94ea00640 1 -- 172.21.15.86:0/3506373877 learned_addr learned my addr 172.21.15.86:0/3506373877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:32.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.910+0000 7fc94fe00640 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 0x5618836d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.911+0000 7fc94ea00640 1 -- 172.21.15.86:0/3506373877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 msgr2=0x5618836d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:32.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.911+0000 7fc94ea00640 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 0x5618836d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.911+0000 7fc94ea00640 1 -- 172.21.15.86:0/3506373877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0170 msgr2=0x5618836ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:32.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.911+0000 7fc94ea00640 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0170 0x5618836ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.911+0000 7fc94ea00640 1 -- 172.21.15.86:0/3506373877 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618833303f0 con 0x5618836c0b60 2024-04-17T07:25:32.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.911+0000 7fc94f400640 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0170 0x5618836ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:32.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.912+0000 7fc934c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:25:32.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.912+0000 7fc94ea00640 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 0x5618836ca5f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0a4950 tx=0x7fc93c0cd970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13a94a615029c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:32.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.912+0000 7fc934c00640 1 -- 172.21.15.86:0/3506373877 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c020070 con 0x5618836c0b60 2024-04-17T07:25:32.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.913+0000 7fc934c00640 1 -- 172.21.15.86:0/3506373877 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc93c009c30 con 0x5618836c0b60 2024-04-17T07:25:32.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.913+0000 7fc958dda740 1 -- 172.21.15.86:0/3506373877 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc95007a420 con 0x5618836c0b60 2024-04-17T07:25:32.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.913+0000 7fc934c00640 1 -- 172.21.15.86:0/3506373877 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc93c09bd50 con 0x5618836c0b60 2024-04-17T07:25:32.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.914+0000 7fc958dda740 10 client.37046 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:32.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.914+0000 7fc958dda740 1 -- 172.21.15.86:0/3506373877 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5618832f00f0 con 0x5618836c0b60 2024-04-17T07:25:32.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.914+0000 7fc934c00640 1 -- 172.21.15.86:0/3506373877 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc95007a420 con 0x5618836c0b60 2024-04-17T07:25:32.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.914+0000 7fc934c00640 1 -- 172.21.15.86:0/3506373877 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc93c0a0160 con 0x5618836c0b60 2024-04-17T07:25:32.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.915+0000 7fc958dda740 10 client.37046 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:32.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.915+0000 7fc958dda740 10 client.37046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:32.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.915+0000 7fc958dda740 20 client.37046 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:32.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.915+0000 7fc958dda740 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5618836d5e60 0x5618836d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.915+0000 7fc958dda740 4 client.37046 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:32.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.915+0000 7fc958dda740 1 -- 172.21.15.86:0/3506373877 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56188362bb40 con 0x5618836d5e60 2024-04-17T07:25:32.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.916+0000 7fc94f400640 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5618836d5e60 0x5618836d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.917+0000 7fc94f400640 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5618836d5e60 0x5618836d82a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f43a55e2f2d3c945 server_cookie=d93cb6d75d20901d in_seq=0 out_seq=0 2024-04-17T07:25:32.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.917+0000 7fc934c00640 10 client.37046 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:32.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.922+0000 7fc934c00640 1 -- 172.21.15.86:0/3506373877 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56188362bb40 con 0x5618836d5e60 2024-04-17T07:25:32.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.922+0000 7fc934c00640 10 client.37046 handle_command_reply: tid=0 2024-04-17T07:25:32.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.922+0000 7fc958dda740 1 client.37046 shutdown 2024-04-17T07:25:32.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.923+0000 7fc958dda740 1 -- 172.21.15.86:0/3506373877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 msgr2=0x5618836ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:32.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.923+0000 7fc958dda740 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 0x5618836ca5f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0a4950 tx=0x7fc93c0cd970 comp rx=0 tx=0).stop 2024-04-17T07:25:32.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.924+0000 7fc958dda740 1 -- 172.21.15.86:0/3506373877 shutdown_connections 2024-04-17T07:25:32.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.924+0000 7fc958dda740 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5618836d5e60 0x5618836d82a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.924+0000 7fc958dda740 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c5760 0x5618836d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.924+0000 7fc958dda740 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 0x5618836ca5f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.924+0000 7fc958dda740 1 --2- 172.21.15.86:0/3506373877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0170 0x5618836ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.924+0000 7fc958dda740 1 -- 172.21.15.86:0/3506373877 >> 172.21.15.86:0/3506373877 conn(0x5618836b2860 msgr2=0x5618836bf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:32.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.924+0000 7fc934c00640 0 client.37046 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:32.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.924+0000 7fc958dda740 1 -- 172.21.15.86:0/3506373877 shutdown_connections 2024-04-17T07:25:32.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.924+0000 7fc958dda740 1 -- 172.21.15.86:0/3506373877 wait complete. 2024-04-17T07:25:32.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.926+0000 7fc958dda740 20 client.37046 trim_cache size 0 max 16384 2024-04-17T07:25:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.974+0000 7fc958dda740 1 Processor -- start 2024-04-17T07:25:32.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.974+0000 7fc958dda740 1 -- start start 2024-04-17T07:25:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.975+0000 7fc958dda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0170 0x5618836c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.975+0000 7fc958dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0b60 0x5618836d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.975+0000 7fc958dda740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c5760 0x5618836bef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.975+0000 7fc958dda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc94813c8b0 con 0x5618836c0170 2024-04-17T07:25:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.975+0000 7fc958dda740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc94813e3f0 con 0x5618836c5760 2024-04-17T07:25:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.975+0000 7fc958dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc94813cf80 con 0x5618836c0b60 2024-04-17T07:25:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.975+0000 7fc94ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0b60 0x5618836d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.976+0000 7fc94ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0b60 0x5618836d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36038/0 (socket says 172.21.15.86:36038) 2024-04-17T07:25:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.976+0000 7fc94ea00640 1 -- 172.21.15.86:0/3769208034 learned_addr learned my addr 172.21.15.86:0/3769208034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.976+0000 7fc94ea00640 1 -- 172.21.15.86:0/3769208034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c5760 msgr2=0x5618836bef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.976+0000 7fc94ea00640 1 --2- 172.21.15.86:0/3769208034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c5760 0x5618836bef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.976+0000 7fc94ea00640 1 -- 172.21.15.86:0/3769208034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0170 msgr2=0x5618836c0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.976+0000 7fc94ea00640 1 --2- 172.21.15.86:0/3769208034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0170 0x5618836c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.976+0000 7fc94ea00640 1 -- 172.21.15.86:0/3769208034 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618836d2ab0 con 0x5618836c0b60 2024-04-17T07:25:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.978+0000 7fc94ea00640 1 --2- 172.21.15.86:0/3769208034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0b60 0x5618836d15f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc93c09a4c0 tx=0x7fc93c0d0590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa65ba1a6fd86ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.978+0000 7fc937400640 1 -- 172.21.15.86:0/3769208034 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c020040 con 0x5618836c0b60 2024-04-17T07:25:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.979+0000 7fc937400640 1 -- 172.21.15.86:0/3769208034 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc93c09c910 con 0x5618836c0b60 2024-04-17T07:25:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.979+0000 7fc937400640 1 -- 172.21.15.86:0/3769208034 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c09bb70 con 0x5618836c0b60 2024-04-17T07:25:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.980+0000 7fc958dda740 1 -- 172.21.15.86:0/3769208034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0b60 msgr2=0x5618836d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.980+0000 7fc958dda740 1 --2- 172.21.15.86:0/3769208034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0b60 0x5618836d15f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc93c09a4c0 tx=0x7fc93c0d0590 comp rx=0 tx=0).stop 2024-04-17T07:25:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.980+0000 7fc958dda740 1 -- 172.21.15.86:0/3769208034 shutdown_connections 2024-04-17T07:25:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.980+0000 7fc958dda740 1 --2- 172.21.15.86:0/3769208034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c5760 0x5618836bef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.980+0000 7fc958dda740 1 --2- 172.21.15.86:0/3769208034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0b60 0x5618836d15f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.980+0000 7fc958dda740 1 --2- 172.21.15.86:0/3769208034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c0170 0x5618836c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.980+0000 7fc958dda740 1 -- 172.21.15.86:0/3769208034 >> 172.21.15.86:0/3769208034 conn(0x5618836b2860 msgr2=0x5618836c1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.980+0000 7fc958dda740 1 -- 172.21.15.86:0/3769208034 shutdown_connections 2024-04-17T07:25:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.980+0000 7fc958dda740 1 -- 172.21.15.86:0/3769208034 wait complete. 2024-04-17T07:25:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.981+0000 7fc958dda740 1 -- start start 2024-04-17T07:25:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.981+0000 7fc958dda740 1 Processor -- start 2024-04-17T07:25:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc958dda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0170 0x5618836ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc958dda740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 0x5618836cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc958dda740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c5760 0x5618836d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc958dda740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc94813dd20 con 0x5618836c5760 2024-04-17T07:25:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc958dda740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc94813d650 con 0x5618836c0b60 2024-04-17T07:25:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc958dda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc94813a880 con 0x5618836c0170 2024-04-17T07:25:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc958dda740 10 client.0 _ll_register_callbacks cb 0x7fc93c010be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc94f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0170 0x5618836ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc94f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0170 0x5618836ced70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36050/0 (socket says 172.21.15.86:36050) 2024-04-17T07:25:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc94f400640 1 -- 172.21.15.86:0/3567916315 learned_addr learned my addr 172.21.15.86:0/3567916315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.982+0000 7fc94ea00640 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 0x5618836cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.983+0000 7fc94f400640 1 -- 172.21.15.86:0/3567916315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 msgr2=0x5618836cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.983+0000 7fc94f400640 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 0x5618836cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.983+0000 7fc94f400640 1 -- 172.21.15.86:0/3567916315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c5760 msgr2=0x5618836d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.983+0000 7fc94f400640 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c5760 0x5618836d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.983+0000 7fc94f400640 1 -- 172.21.15.86:0/3567916315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618836d07f0 con 0x5618836c0170 2024-04-17T07:25:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.983+0000 7fc94f400640 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0170 0x5618836ced70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc944005eb0 tx=0x7fc94402bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1a7f1952b2ecdac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.983+0000 7fc934c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.983+0000 7fc934c00640 1 -- 172.21.15.86:0/3567916315 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944058060 con 0x5618836c0170 2024-04-17T07:25:32.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.984+0000 7fc934c00640 1 -- 172.21.15.86:0/3567916315 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc9440043c0 con 0x5618836c0170 2024-04-17T07:25:32.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.984+0000 7fc958dda740 1 -- 172.21.15.86:0/3567916315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc93c09c910 con 0x5618836c0170 2024-04-17T07:25:32.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.984+0000 7fc934c00640 1 -- 172.21.15.86:0/3567916315 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc94405f050 con 0x5618836c0170 2024-04-17T07:25:32.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.984+0000 7fc958dda740 10 client.27118 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:32.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.984+0000 7fc958dda740 1 -- 172.21.15.86:0/3567916315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56188374d430 con 0x5618836c0170 2024-04-17T07:25:32.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.984+0000 7fc934c00640 1 -- 172.21.15.86:0/3567916315 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc93c09c910 con 0x5618836c0170 2024-04-17T07:25:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.985+0000 7fc934c00640 1 -- 172.21.15.86:0/3567916315 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc944049040 con 0x5618836c0170 2024-04-17T07:25:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.985+0000 7fc958dda740 10 client.27118 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.985+0000 7fc958dda740 10 client.27118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.985+0000 7fc958dda740 20 client.27118 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.985+0000 7fc958dda740 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56188374de00 0x561883750220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.985+0000 7fc958dda740 4 client.27118 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:25:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.985+0000 7fc958dda740 1 -- 172.21.15.86:0/3567916315 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5618836259b0 con 0x56188374de00 2024-04-17T07:25:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.985+0000 7fc94ea00640 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56188374de00 0x561883750220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.986+0000 7fc94ea00640 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56188374de00 0x561883750220 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a83cb1c1b09af2f server_cookie=3b17ea34d1c3923c in_seq=0 out_seq=0 2024-04-17T07:25:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.986+0000 7fc934c00640 10 client.27118 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc934c00640 1 -- 172.21.15.86:0/3567916315 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5618836259b0 con 0x56188374de00 2024-04-17T07:25:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc934c00640 10 client.27118 handle_command_reply: tid=0 2024-04-17T07:25:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc958dda740 1 client.27118 shutdown 2024-04-17T07:25:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc958dda740 1 -- 172.21.15.86:0/3567916315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0170 msgr2=0x5618836ced70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc958dda740 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0170 0x5618836ced70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc944005eb0 tx=0x7fc94402bb70 comp rx=0 tx=0).stop 2024-04-17T07:25:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc958dda740 1 -- 172.21.15.86:0/3567916315 shutdown_connections 2024-04-17T07:25:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc958dda740 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56188374de00 0x561883750220 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc958dda740 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618836c5760 0x5618836d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc958dda740 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618836c0b60 0x5618836cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc934c00640 0 client.27118 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc958dda740 1 --2- 172.21.15.86:0/3567916315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618836c0170 0x5618836ced70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.987+0000 7fc958dda740 1 -- 172.21.15.86:0/3567916315 >> 172.21.15.86:0/3567916315 conn(0x5618836b2860 msgr2=0x5618836b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.988+0000 7fc958dda740 1 -- 172.21.15.86:0/3567916315 shutdown_connections 2024-04-17T07:25:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.988+0000 7fc958dda740 1 -- 172.21.15.86:0/3567916315 wait complete. 2024-04-17T07:25:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.988+0000 7fc958dda740 20 client.27118 trim_cache size 0 max 16384 2024-04-17T07:25:32.989 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:25:32.989 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:25:32.989 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:25:32.989 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:25:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 -- 172.21.15.86:0/4067412566 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9380a3c30 msgr2=0x7fc9380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9380a3c30 0x7fc9380a60f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc944009830 tx=0x7fc944034980 comp rx=0 tx=0).stop 2024-04-17T07:25:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 -- 172.21.15.86:0/4067412566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc948095e70 msgr2=0x7fc94812c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc948095e70 0x7fc94812c5f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00d950 tx=0x7fc93c00dd50 comp rx=0 tx=0).stop 2024-04-17T07:25:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 -- 172.21.15.86:0/4067412566 shutdown_connections 2024-04-17T07:25:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9380a3c30 0x7fc9380a60f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc948096830 0x7fc948133660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc948095e70 0x7fc94812c5f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 --2- 172.21.15.86:0/4067412566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc948094c80 0x7fc94812c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 -- 172.21.15.86:0/4067412566 >> 172.21.15.86:0/4067412566 conn(0x7fc9480901d0 msgr2=0x7fc948091cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 -- 172.21.15.86:0/4067412566 shutdown_connections 2024-04-17T07:25:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:32.989+0000 7fc954a00640 1 -- 172.21.15.86:0/4067412566 wait complete. 2024-04-17T07:25:33.022 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:25:33.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:25:33.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.320+0000 7f17d4200640 1 -- 172.21.15.86:0/986549438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17cc105d90 msgr2=0x7f17cc10c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:33.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.320+0000 7f17d4200640 1 --2- 172.21.15.86:0/986549438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17cc105d90 0x7f17cc10c620 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009900 tx=0x7f17c402bc60 comp rx=0 tx=0).stop 2024-04-17T07:25:33.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.320+0000 7f17d4200640 1 -- 172.21.15.86:0/986549438 shutdown_connections 2024-04-17T07:25:33.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.320+0000 7f17d4200640 1 --2- 172.21.15.86:0/986549438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17cc105d90 0x7f17cc10c620 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.321+0000 7f17d4200640 1 --2- 172.21.15.86:0/986549438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17cc1053d0 0x7f17cc105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.321+0000 7f17d4200640 1 --2- 172.21.15.86:0/986549438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc1041e0 0x7f17cc104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.321+0000 7f17d4200640 1 -- 172.21.15.86:0/986549438 >> 172.21.15.86:0/986549438 conn(0x7f17cc0ff750 msgr2=0x7f17cc101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:33.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.321+0000 7f17d4200640 1 -- 172.21.15.86:0/986549438 shutdown_connections 2024-04-17T07:25:33.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.321+0000 7f17d4200640 1 -- 172.21.15.86:0/986549438 wait complete. 2024-04-17T07:25:33.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.324+0000 7f17d4200640 1 Processor -- start 2024-04-17T07:25:33.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.324+0000 7f17d4200640 1 -- start start 2024-04-17T07:25:33.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.324+0000 7f17d4200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17cc1041e0 0x7f17cc079d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:33.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.324+0000 7f17d4200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc1053d0 0x7f17cc07a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:33.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.325+0000 7f17d4200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17cc105d90 0x7f17cc076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:33.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.325+0000 7f17d4200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc10fcf0 con 0x7f17cc1053d0 2024-04-17T07:25:33.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.325+0000 7f17d4200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc10fb70 con 0x7f17cc105d90 2024-04-17T07:25:33.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.325+0000 7f17d4200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc10fe70 con 0x7f17cc1041e0 2024-04-17T07:25:33.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.325+0000 7f17d1800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17cc1041e0 0x7f17cc079d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:33.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.325+0000 7f17d0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc1053d0 0x7f17cc07a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:33.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.325+0000 7f17d1800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17cc1041e0 0x7f17cc079d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36062/0 (socket says 172.21.15.86:36062) 2024-04-17T07:25:33.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.325+0000 7f17d1800640 1 -- 172.21.15.86:0/1443284470 learned_addr learned my addr 172.21.15.86:0/1443284470 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:33.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.326+0000 7f17d2200640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17cc105d90 0x7f17cc076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:33.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.326+0000 7f17d0e00640 1 -- 172.21.15.86:0/1443284470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17cc1041e0 msgr2=0x7f17cc079d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:33.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.326+0000 7f17d0e00640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17cc1041e0 0x7f17cc079d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.326+0000 7f17d0e00640 1 -- 172.21.15.86:0/1443284470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17cc105d90 msgr2=0x7f17cc076880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:33.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.326+0000 7f17d0e00640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17cc105d90 0x7f17cc076880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.326+0000 7f17d0e00640 1 -- 172.21.15.86:0/1443284470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc077140 con 0x7f17cc1053d0 2024-04-17T07:25:33.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.326+0000 7f17d1800640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17cc1041e0 0x7f17cc079d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:33.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.327+0000 7f17d0e00640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc1053d0 0x7f17cc07a250 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e9f0 tx=0x7f17b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:33.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.327+0000 7f17ca000640 1 -- 172.21.15.86:0/1443284470 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b800ccc0 con 0x7f17cc1053d0 2024-04-17T07:25:33.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.327+0000 7f17d4200640 1 -- 172.21.15.86:0/1443284470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc1a6840 con 0x7f17cc1053d0 2024-04-17T07:25:33.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.328+0000 7f17ca000640 1 -- 172.21.15.86:0/1443284470 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17b80108e0 con 0x7f17cc1053d0 2024-04-17T07:25:33.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.328+0000 7f17ca000640 1 -- 172.21.15.86:0/1443284470 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8018880 con 0x7f17cc1053d0 2024-04-17T07:25:33.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.328+0000 7f17d4200640 1 -- 172.21.15.86:0/1443284470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc1a6ca0 con 0x7f17cc1053d0 2024-04-17T07:25:33.332 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:33 smithi086 ceph-mon[31733]: pgmap v775: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-04-17T07:25:33.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.330+0000 7f17d4200640 1 -- 172.21.15.86:0/1443284470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc068d80 con 0x7f17cc1053d0 2024-04-17T07:25:33.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.337+0000 7f17ca000640 1 -- 172.21.15.86:0/1443284470 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f17b8010a90 con 0x7f17cc1053d0 2024-04-17T07:25:33.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.340+0000 7f17ca000640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17b40a3d00 0x7f17b40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:33.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.340+0000 7f17ca000640 1 -- 172.21.15.86:0/1443284470 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f17b80d0b80 con 0x7f17cc1053d0 2024-04-17T07:25:33.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.342+0000 7f17d1800640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17b40a3d00 0x7f17b40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:33.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.343+0000 7f17d1800640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17b40a3d00 0x7f17b40a61c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f17c00117a0 tx=0x7f17c00096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:33.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.348+0000 7f17ca000640 1 -- 172.21.15.86:0/1443284470 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f17b809d180 con 0x7f17cc1053d0 2024-04-17T07:25:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:33 smithi098 ceph-mon[35418]: pgmap v775: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-04-17T07:25:33.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:33 smithi103 ceph-mon[35495]: pgmap v775: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-04-17T07:25:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.844+0000 7f17d4200640 1 -- 172.21.15.86:0/1443284470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17cc10b520 con 0x7f17cc1053d0 2024-04-17T07:25:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.846+0000 7f17ca000640 1 -- 172.21.15.86:0/1443284470 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f17b80bd0c0 con 0x7f17cc1053d0 2024-04-17T07:25:33.846 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:25:33.846 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:25:33.846 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:25:33.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.849+0000 7f17d4200640 1 -- 172.21.15.86:0/1443284470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17b40a3d00 msgr2=0x7f17b40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:33.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.849+0000 7f17d4200640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17b40a3d00 0x7f17b40a61c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f17c00117a0 tx=0x7f17c00096d0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.849+0000 7f17d4200640 1 -- 172.21.15.86:0/1443284470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc1053d0 msgr2=0x7f17cc07a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:33.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.849+0000 7f17d4200640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc1053d0 0x7f17cc07a250 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e9f0 tx=0x7f17b800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.849+0000 7f17d4200640 1 -- 172.21.15.86:0/1443284470 shutdown_connections 2024-04-17T07:25:33.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.849+0000 7f17d4200640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17b40a3d00 0x7f17b40a61c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.849+0000 7f17d4200640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17cc105d90 0x7f17cc076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.849+0000 7f17d4200640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc1053d0 0x7f17cc07a250 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.849+0000 7f17d4200640 1 --2- 172.21.15.86:0/1443284470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17cc1041e0 0x7f17cc079d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:33.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.849+0000 7f17d4200640 1 -- 172.21.15.86:0/1443284470 >> 172.21.15.86:0/1443284470 conn(0x7f17cc0ff750 msgr2=0x7f17cc1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:33.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.850+0000 7f17d4200640 1 -- 172.21.15.86:0/1443284470 shutdown_connections 2024-04-17T07:25:33.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:33.850+0000 7f17d4200640 1 -- 172.21.15.86:0/1443284470 wait complete. 2024-04-17T07:25:33.883 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:25:34.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.193+0000 7f7d50000640 1 -- 172.21.15.86:0/3162928558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d48076920 msgr2=0x7f7d48076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.193+0000 7f7d50000640 1 --2- 172.21.15.86:0/3162928558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d48076920 0x7f7d48076d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009be0 tx=0x7f7d4002bf10 comp rx=0 tx=0).stop 2024-04-17T07:25:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.196+0000 7f7d50000640 1 -- 172.21.15.86:0/3162928558 shutdown_connections 2024-04-17T07:25:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.196+0000 7f7d50000640 1 --2- 172.21.15.86:0/3162928558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d48078500 0x7f7d48078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.196+0000 7f7d50000640 1 --2- 172.21.15.86:0/3162928558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d48077b60 0x7f7d48077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.196+0000 7f7d50000640 1 --2- 172.21.15.86:0/3162928558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d48076920 0x7f7d48076d40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.196+0000 7f7d50000640 1 -- 172.21.15.86:0/3162928558 >> 172.21.15.86:0/3162928558 conn(0x7f7d480ff710 msgr2=0x7f7d48101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.196+0000 7f7d50000640 1 -- 172.21.15.86:0/3162928558 shutdown_connections 2024-04-17T07:25:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.196+0000 7f7d50000640 1 -- 172.21.15.86:0/3162928558 wait complete. 2024-04-17T07:25:34.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.197+0000 7f7d50000640 1 Processor -- start 2024-04-17T07:25:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.198+0000 7f7d50000640 1 -- start start 2024-04-17T07:25:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.198+0000 7f7d50000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d48076920 0x7f7d4810fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.199+0000 7f7d50000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d48077b60 0x7f7d481103d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.199+0000 7f7d50000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d48078500 0x7f7d48111410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.199+0000 7f7d50000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d481044b0 con 0x7f7d48078500 2024-04-17T07:25:34.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.199+0000 7f7d50000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7d48104330 con 0x7f7d48076920 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.199+0000 7f7d50000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d48104630 con 0x7f7d48077b60 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.200+0000 7f7d4d600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d48076920 0x7f7d4810fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.200+0000 7f7d4e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d48078500 0x7f7d48111410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.200+0000 7f7d4d600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d48076920 0x7f7d4810fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50556/0 (socket says 172.21.15.86:50556) 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.200+0000 7f7d4d600640 1 -- 172.21.15.86:0/251849236 learned_addr learned my addr 172.21.15.86:0/251849236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.200+0000 7f7d4cc00640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d48077b60 0x7f7d481103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.200+0000 7f7d4d600640 1 -- 172.21.15.86:0/251849236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d48077b60 msgr2=0x7f7d481103d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.200+0000 7f7d4d600640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d48077b60 0x7f7d481103d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.200+0000 7f7d4d600640 1 -- 172.21.15.86:0/251849236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d48078500 msgr2=0x7f7d48111410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.200+0000 7f7d4d600640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d48078500 0x7f7d48111410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.200+0000 7f7d4d600640 1 -- 172.21.15.86:0/251849236 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d48111a80 con 0x7f7d48076920 2024-04-17T07:25:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.201+0000 7f7d4d600640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d48076920 0x7f7d4810fe90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d4002c520 tx=0x7f7d4002cae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.202+0000 7f7d46000640 1 -- 172.21.15.86:0/251849236 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40043020 con 0x7f7d48076920 2024-04-17T07:25:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.202+0000 7f7d50000640 1 -- 172.21.15.86:0/251849236 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d48111d10 con 0x7f7d48076920 2024-04-17T07:25:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.202+0000 7f7d46000640 1 -- 172.21.15.86:0/251849236 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d40033c50 con 0x7f7d48076920 2024-04-17T07:25:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.203+0000 7f7d50000640 1 -- 172.21.15.86:0/251849236 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d481af5d0 con 0x7f7d48076920 2024-04-17T07:25:34.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.203+0000 7f7d46000640 1 -- 172.21.15.86:0/251849236 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40043020 con 0x7f7d48076920 2024-04-17T07:25:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.209+0000 7f7d46000640 1 -- 172.21.15.86:0/251849236 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7d40046020 con 0x7f7d48076920 2024-04-17T07:25:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.209+0000 7f7d52dc6740 1 Processor -- start 2024-04-17T07:25:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.210+0000 7f7d52dc6740 1 -- start start 2024-04-17T07:25:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.210+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.210+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.210+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101bbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.210+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad1011a020 con 0x55ad101b4990 2024-04-17T07:25:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.210+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ad100c0460 con 0x55ad101b9b00 2024-04-17T07:25:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.210+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ad10115fe0 con 0x55ad101b3fa0 2024-04-17T07:25:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.211+0000 7f7d4cc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101bbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.211+0000 7f7d4cc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101bbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50558/0 (socket says 172.21.15.86:50558) 2024-04-17T07:25:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.211+0000 7f7d4cc00640 1 -- 172.21.15.86:0/3225575305 learned_addr learned my addr 172.21.15.86:0/3225575305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.211+0000 7f7d46000640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d300a3e70 0x7f7d300a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.212+0000 7f7d46000640 1 -- 172.21.15.86:0/251849236 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7d4002ce00 con 0x7f7d48076920 2024-04-17T07:25:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.212+0000 7f7d4d600640 1 --2- 172.21.15.86:0/3225575305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.212+0000 7f7d4e000640 1 --2- 172.21.15.86:0/3225575305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.213+0000 7f7d4e000640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d300a3e70 0x7f7d300a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.213+0000 7f7d4cc00640 1 -- 172.21.15.86:0/3225575305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 msgr2=0x55ad101b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.213+0000 7f7d4cc00640 1 --2- 172.21.15.86:0/3225575305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.213+0000 7f7d4cc00640 1 -- 172.21.15.86:0/3225575305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 msgr2=0x55ad101b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.213+0000 7f7d4cc00640 1 --2- 172.21.15.86:0/3225575305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.213+0000 7f7d4cc00640 1 -- 172.21.15.86:0/3225575305 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad100c8040 con 0x55ad101b9b00 2024-04-17T07:25:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.213+0000 7f7d4d600640 1 --2- 172.21.15.86:0/3225575305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.214+0000 7f7d4cc00640 1 --2- 172.21.15.86:0/3225575305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101bbf90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800abf0 tx=0x7f7d3800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1a2ea5f170a160b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.215+0000 7f7d36600640 1 -- 172.21.15.86:0/3225575305 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3801d070 con 0x55ad101b9b00 2024-04-17T07:25:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.215+0000 7f7d36600640 1 -- 172.21.15.86:0/3225575305 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d38018af0 con 0x55ad101b9b00 2024-04-17T07:25:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.215+0000 7f7d36600640 1 -- 172.21.15.86:0/3225575305 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38010aa0 con 0x55ad101b9b00 2024-04-17T07:25:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.216+0000 7f7d4e000640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d300a3e70 0x7f7d300a6330 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55ad101b51f0 tx=0x7f7d3c004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.216+0000 7f7d52dc6740 1 -- 172.21.15.86:0/3225575305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 msgr2=0x55ad101bbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.216+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/3225575305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101bbf90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800abf0 tx=0x7f7d3800f790 comp rx=0 tx=0).stop 2024-04-17T07:25:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.217+0000 7f7d52dc6740 1 -- 172.21.15.86:0/3225575305 shutdown_connections 2024-04-17T07:25:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.217+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/3225575305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101bbf90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.217+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/3225575305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.217+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/3225575305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.217+0000 7f7d52dc6740 1 -- 172.21.15.86:0/3225575305 >> 172.21.15.86:0/3225575305 conn(0x55ad101af510 msgr2=0x55ad101af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.217+0000 7f7d52dc6740 1 -- 172.21.15.86:0/3225575305 shutdown_connections 2024-04-17T07:25:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.217+0000 7f7d52dc6740 1 -- 172.21.15.86:0/3225575305 wait complete. 2024-04-17T07:25:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.217+0000 7f7d52dc6740 1 -- start start 2024-04-17T07:25:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.218+0000 7f7d52dc6740 1 Processor -- start 2024-04-17T07:25:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.218+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101be4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.218+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101bea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.218+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.218+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad101bcab0 con 0x55ad101b4990 2024-04-17T07:25:34.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.218+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ad100d3830 con 0x55ad101b9b00 2024-04-17T07:25:34.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.218+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ad101bcc30 con 0x55ad101b3fa0 2024-04-17T07:25:34.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.218+0000 7f7d52dc6740 10 client.0 _ll_register_callbacks cb 0x55ad100a1fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.219+0000 7f7d4cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101bea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.219+0000 7f7d4d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101be4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.219+0000 7f7d4d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101be4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36114/0 (socket says 172.21.15.86:36114) 2024-04-17T07:25:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.219+0000 7f7d4d600640 1 -- 172.21.15.86:0/2287576120 learned_addr learned my addr 172.21.15.86:0/2287576120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.219+0000 7f7d4e000640 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.219+0000 7f7d4cc00640 1 -- 172.21.15.86:0/2287576120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 msgr2=0x55ad101be4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.219+0000 7f7d4cc00640 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101be4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.220+0000 7f7d4cc00640 1 -- 172.21.15.86:0/2287576120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 msgr2=0x55ad101c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.220+0000 7f7d4cc00640 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.220+0000 7f7d4cc00640 1 -- 172.21.15.86:0/2287576120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad100d7920 con 0x55ad101b4990 2024-04-17T07:25:34.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.220+0000 7f7d4cc00640 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101bea20 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7d380058c0 tx=0x7f7d38005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a36c142f8d3590 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.221+0000 7f7d2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:25:34.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.221+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2287576120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3801d040 con 0x55ad101b4990 2024-04-17T07:25:34.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.221+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2287576120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7d38005030 con 0x55ad101b4990 2024-04-17T07:25:34.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.221+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2287576120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d38018af0 con 0x55ad101b4990 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.222+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2287576120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7d38031050 con 0x55ad101b4990 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.222+0000 7f7d52dc6740 10 client.18192 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.222+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2287576120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad100a57f0 con 0x55ad101b4990 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.222+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2287576120 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d38018af0 con 0x55ad101b4990 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.222+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2287576120 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7d380046c0 con 0x55ad101b4990 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.222+0000 7f7d52dc6740 10 client.18192 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.222+0000 7f7d52dc6740 10 client.18192 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.223+0000 7f7d52dc6740 20 client.18192 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.223+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ad101c9bc0 0x55ad101cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.223+0000 7f7d52dc6740 4 client.18192 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:34.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.223+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2287576120 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad10127a40 con 0x55ad101c9bc0 2024-04-17T07:25:34.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.223+0000 7f7d4d600640 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ad101c9bc0 0x55ad101cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.224+0000 7f7d4d600640 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ad101c9bc0 0x55ad101cbfb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=767d2217489fa314 server_cookie=cc9ca749cee69bc3 in_seq=0 out_seq=0 2024-04-17T07:25:34.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.224+0000 7f7d2aa00640 10 client.18192 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:25:34.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.225+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2287576120 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad10127a40 con 0x55ad101c9bc0 2024-04-17T07:25:34.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.225+0000 7f7d2aa00640 10 client.18192 handle_command_reply: tid=0 2024-04-17T07:25:34.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.226+0000 7f7d52dc6740 1 client.18192 shutdown 2024-04-17T07:25:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.226+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2287576120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 msgr2=0x55ad101bea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.226+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101bea20 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7d380058c0 tx=0x7f7d38005dc0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.226+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2287576120 shutdown_connections 2024-04-17T07:25:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.226+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ad101c9bc0 0x55ad101cbfb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.226+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.226+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101bea20 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.226+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2287576120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101be4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.226+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2287576120 >> 172.21.15.86:0/2287576120 conn(0x55ad101af510 msgr2=0x55ad101ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.226+0000 7f7d2aa00640 0 client.18192 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:25:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.227+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2287576120 shutdown_connections 2024-04-17T07:25:34.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.227+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2287576120 wait complete. 2024-04-17T07:25:34.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.227+0000 7f7d52dc6740 20 client.18192 trim_cache size 0 max 16384 2024-04-17T07:25:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.260+0000 7f7d52dc6740 1 Processor -- start 2024-04-17T07:25:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.260+0000 7f7d52dc6740 1 -- start start 2024-04-17T07:25:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.260+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.260+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.260+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.260+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d481b9950 con 0x55ad101b4990 2024-04-17T07:25:34.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.260+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7d481bb490 con 0x55ad101b9b00 2024-04-17T07:25:34.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.260+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d481ba020 con 0x55ad101b3fa0 2024-04-17T07:25:34.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.261+0000 7f7d4d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.261+0000 7f7d4cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.261+0000 7f7d4e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.261+0000 7f7d4e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101b5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50582/0 (socket says 172.21.15.86:50582) 2024-04-17T07:25:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.261+0000 7f7d4e000640 1 -- 172.21.15.86:0/1952120061 learned_addr learned my addr 172.21.15.86:0/1952120061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:34.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.262+0000 7f7d4cc00640 1 -- 172.21.15.86:0/1952120061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 msgr2=0x55ad101bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.262+0000 7f7d4cc00640 1 --2- 172.21.15.86:0/1952120061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101bdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.262+0000 7f7d4cc00640 1 -- 172.21.15.86:0/1952120061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 msgr2=0x55ad101b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.262+0000 7f7d4cc00640 1 --2- 172.21.15.86:0/1952120061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.262+0000 7f7d4cc00640 1 -- 172.21.15.86:0/1952120061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad0fe2c3f0 con 0x55ad101b4990 2024-04-17T07:25:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.262+0000 7f7d4e000640 1 --2- 172.21.15.86:0/1952120061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.263+0000 7f7d4cc00640 1 --2- 172.21.15.86:0/1952120061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101c5a20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7d38004c20 tx=0x7f7d38010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e862622d87ecb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.263+0000 7f7d35c00640 1 -- 172.21.15.86:0/1952120061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3801d070 con 0x55ad101b4990 2024-04-17T07:25:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.264+0000 7f7d35c00640 1 -- 172.21.15.86:0/1952120061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d38004660 con 0x55ad101b4990 2024-04-17T07:25:34.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.264+0000 7f7d35c00640 1 -- 172.21.15.86:0/1952120061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38010ce0 con 0x55ad101b4990 2024-04-17T07:25:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.265+0000 7f7d52dc6740 1 -- 172.21.15.86:0/1952120061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 msgr2=0x55ad101c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.265+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/1952120061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101c5a20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7d38004c20 tx=0x7f7d38010f90 comp rx=0 tx=0).stop 2024-04-17T07:25:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.265+0000 7f7d52dc6740 1 -- 172.21.15.86:0/1952120061 shutdown_connections 2024-04-17T07:25:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.265+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/1952120061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.265+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/1952120061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101b4990 0x55ad101c5a20 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.265+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/1952120061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101bdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.265+0000 7f7d52dc6740 1 -- 172.21.15.86:0/1952120061 >> 172.21.15.86:0/1952120061 conn(0x55ad101af510 msgr2=0x55ad101bcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.266+0000 7f7d52dc6740 1 -- 172.21.15.86:0/1952120061 shutdown_connections 2024-04-17T07:25:34.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.266+0000 7f7d52dc6740 1 -- 172.21.15.86:0/1952120061 wait complete. 2024-04-17T07:25:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.267+0000 7f7d52dc6740 1 -- start start 2024-04-17T07:25:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.267+0000 7f7d52dc6740 1 Processor -- start 2024-04-17T07:25:34.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.268+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.268+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101c3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.268+0000 7f7d52dc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101c42d0 0x55ad101c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.269+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d481badc0 con 0x55ad101c42d0 2024-04-17T07:25:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.269+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7d481ba6f0 con 0x55ad101b9b00 2024-04-17T07:25:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.269+0000 7f7d52dc6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d481b7920 con 0x55ad101b3fa0 2024-04-17T07:25:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.269+0000 7f7d52dc6740 10 client.0 _ll_register_callbacks cb 0x7f7d3c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.269+0000 7f7d4cc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.268+0000 7f7d4d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.269+0000 7f7d4e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101c42d0 0x55ad101c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.269+0000 7f7d4d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101c2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36130/0 (socket says 172.21.15.86:36130) 2024-04-17T07:25:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.269+0000 7f7d4d600640 1 -- 172.21.15.86:0/2933582207 learned_addr learned my addr 172.21.15.86:0/2933582207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.270+0000 7f7d4e000640 1 -- 172.21.15.86:0/2933582207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 msgr2=0x55ad101c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.270+0000 7f7d4e000640 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101c2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.270+0000 7f7d4e000640 1 -- 172.21.15.86:0/2933582207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 msgr2=0x55ad101c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.270+0000 7f7d4e000640 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.270+0000 7f7d4e000640 1 -- 172.21.15.86:0/2933582207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad0fef67d0 con 0x55ad101c42d0 2024-04-17T07:25:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.270+0000 7f7d4e000640 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101c42d0 0x55ad101c4750 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c004650 tx=0x7f7d3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51da4c885e7026d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.270+0000 7f7d2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:25:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.270+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2933582207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c005220 con 0x55ad101c42d0 2024-04-17T07:25:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.270+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2933582207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7d3c0053c0 con 0x55ad101c42d0 2024-04-17T07:25:34.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.271+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2933582207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d38004660 con 0x55ad101c42d0 2024-04-17T07:25:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.271+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2933582207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7d3c0055c0 con 0x55ad101c42d0 2024-04-17T07:25:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.271+0000 7f7d52dc6740 10 client.18198 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.271+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2933582207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad102300a0 con 0x55ad101c42d0 2024-04-17T07:25:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.271+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2933582207 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d38004660 con 0x55ad101c42d0 2024-04-17T07:25:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.272+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2933582207 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7d3c02b030 con 0x55ad101c42d0 2024-04-17T07:25:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.272+0000 7f7d52dc6740 10 client.18198 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.272+0000 7f7d52dc6740 10 client.18198 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:25:34.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.272+0000 7f7d52dc6740 20 client.18198 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:34.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.272+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ad10230990 0x55ad10232db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.272+0000 7f7d52dc6740 4 client.18198 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:25:34.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.272+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2933582207 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad101be2d0 con 0x55ad10230990 2024-04-17T07:25:34.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.273+0000 7f7d4d600640 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ad10230990 0x55ad10232db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.273+0000 7f7d4d600640 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ad10230990 0x55ad10232db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=272de305ce701b93 server_cookie=546ffb7db61189b in_seq=0 out_seq=0 2024-04-17T07:25:34.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.273+0000 7f7d2aa00640 10 client.18198 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:25:34.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d2aa00640 1 -- 172.21.15.86:0/2933582207 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad101be2d0 con 0x55ad10230990 2024-04-17T07:25:34.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d2aa00640 10 client.18198 handle_command_reply: tid=0 2024-04-17T07:25:34.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d52dc6740 1 client.18198 shutdown 2024-04-17T07:25:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2933582207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101c42d0 msgr2=0x55ad101c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101c42d0 0x55ad101c4750 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c004650 tx=0x7f7d3c0046c0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2933582207 shutdown_connections 2024-04-17T07:25:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ad10230990 0x55ad10232db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad101c42d0 0x55ad101c4750 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad101b9b00 0x55ad101c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d52dc6740 1 --2- 172.21.15.86:0/2933582207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad101b3fa0 0x55ad101c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2933582207 >> 172.21.15.86:0/2933582207 conn(0x55ad101af510 msgr2=0x55ad101be480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.274+0000 7f7d2aa00640 0 client.18198 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:25:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.275+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2933582207 shutdown_connections 2024-04-17T07:25:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.275+0000 7f7d52dc6740 1 -- 172.21.15.86:0/2933582207 wait complete. 2024-04-17T07:25:34.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.275+0000 7f7d52dc6740 20 client.18198 trim_cache size 0 max 16384 2024-04-17T07:25:34.276 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:25:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.276+0000 7f7d50000640 1 -- 172.21.15.86:0/251849236 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d300a3e70 msgr2=0x7f7d300a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.276+0000 7f7d50000640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d300a3e70 0x7f7d300a6330 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55ad101b51f0 tx=0x7f7d3c004730 comp rx=0 tx=0).stop 2024-04-17T07:25:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.276+0000 7f7d50000640 1 -- 172.21.15.86:0/251849236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d48076920 msgr2=0x7f7d4810fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.276+0000 7f7d50000640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d48076920 0x7f7d4810fe90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d4002c520 tx=0x7f7d4002cae0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.276+0000 7f7d50000640 1 -- 172.21.15.86:0/251849236 shutdown_connections 2024-04-17T07:25:34.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.276+0000 7f7d50000640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7d300a3e70 0x7f7d300a6330 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.276+0000 7f7d50000640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d48078500 0x7f7d48111410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.276+0000 7f7d50000640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d48077b60 0x7f7d481103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.276+0000 7f7d50000640 1 --2- 172.21.15.86:0/251849236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d48076920 0x7f7d4810fe90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.276+0000 7f7d50000640 1 -- 172.21.15.86:0/251849236 >> 172.21.15.86:0/251849236 conn(0x7f7d480ff710 msgr2=0x7f7d48100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.277+0000 7f7d50000640 1 -- 172.21.15.86:0/251849236 shutdown_connections 2024-04-17T07:25:34.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.277+0000 7f7d50000640 1 -- 172.21.15.86:0/251849236 wait complete. 2024-04-17T07:25:34.307 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:25:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 370.0 s, 713.8 tps, lat 145.853 ms stddev 62.095 2024-04-17T07:25:34.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:34 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1443284470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:25:34.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:34 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1443284470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:25:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:34 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1443284470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:25:34.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.601+0000 7fa386e00640 1 -- 172.21.15.86:0/2523141283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380105d70 msgr2=0x7fa38010c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.601+0000 7fa386e00640 1 --2- 172.21.15.86:0/2523141283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380105d70 0x7fa38010c600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa378009be0 tx=0x7fa37802bf30 comp rx=0 tx=0).stop 2024-04-17T07:25:34.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.602+0000 7fa386e00640 1 -- 172.21.15.86:0/2523141283 shutdown_connections 2024-04-17T07:25:34.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.602+0000 7fa386e00640 1 --2- 172.21.15.86:0/2523141283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa380105d70 0x7fa38010c600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.602+0000 7fa386e00640 1 --2- 172.21.15.86:0/2523141283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3801053b0 0x7fa380105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.602+0000 7fa386e00640 1 --2- 172.21.15.86:0/2523141283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801041c0 0x7fa3801045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.602+0000 7fa386e00640 1 -- 172.21.15.86:0/2523141283 >> 172.21.15.86:0/2523141283 conn(0x7fa3800ff710 msgr2=0x7fa380101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.602+0000 7fa386e00640 1 -- 172.21.15.86:0/2523141283 shutdown_connections 2024-04-17T07:25:34.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.602+0000 7fa386e00640 1 -- 172.21.15.86:0/2523141283 wait complete. 2024-04-17T07:25:34.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.604+0000 7fa386e00640 1 Processor -- start 2024-04-17T07:25:34.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.604+0000 7fa386e00640 1 -- start start 2024-04-17T07:25:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.605+0000 7fa386e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3801041c0 0x7fa38019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.605+0000 7fa386e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801053b0 0x7fa38019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.605+0000 7fa386e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa380105d70 0x7fa3801a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.605+0000 7fa386e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa380077550 con 0x7fa380105d70 2024-04-17T07:25:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.605+0000 7fa386e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3800773d0 con 0x7fa3801053b0 2024-04-17T07:25:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.605+0000 7fa386e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3800776d0 con 0x7fa3801041c0 2024-04-17T07:25:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.605+0000 7fa37fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3801041c0 0x7fa38019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.606+0000 7fa37fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3801041c0 0x7fa38019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36152/0 (socket says 172.21.15.86:36152) 2024-04-17T07:25:34.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.606+0000 7fa37fe00640 1 -- 172.21.15.86:0/1135484765 learned_addr learned my addr 172.21.15.86:0/1135484765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.606+0000 7fa384e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa380105d70 0x7fa3801a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.606+0000 7fa377e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801053b0 0x7fa38019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.606+0000 7fa37fe00640 1 -- 172.21.15.86:0/1135484765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801053b0 msgr2=0x7fa38019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.606+0000 7fa37fe00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801053b0 0x7fa38019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.606+0000 7fa37fe00640 1 -- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa380105d70 msgr2=0x7fa3801a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.606+0000 7fa37fe00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa380105d70 0x7fa3801a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.606+0000 7fa37fe00640 1 -- 172.21.15.86:0/1135484765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3801a6380 con 0x7fa3801041c0 2024-04-17T07:25:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.606+0000 7fa384e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa380105d70 0x7fa3801a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.607+0000 7fa37fe00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3801041c0 0x7fa38019e760 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa36800e9a0 tx=0x7fa36800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.607+0000 7fa37d600640 1 -- 172.21.15.86:0/1135484765 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36800cd80 con 0x7fa3801041c0 2024-04-17T07:25:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.607+0000 7fa37d600640 1 -- 172.21.15.86:0/1135484765 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3680108e0 con 0x7fa3801041c0 2024-04-17T07:25:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.607+0000 7fa386e00640 1 -- 172.21.15.86:0/1135484765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3801a6670 con 0x7fa3801041c0 2024-04-17T07:25:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.607+0000 7fa37d600640 1 -- 172.21.15.86:0/1135484765 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36800cd80 con 0x7fa3801041c0 2024-04-17T07:25:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.607+0000 7fa386e00640 1 -- 172.21.15.86:0/1135484765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3801a6c10 con 0x7fa3801041c0 2024-04-17T07:25:34.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.611+0000 7fa389b9a740 1 Processor -- start 2024-04-17T07:25:34.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.613+0000 7fa389b9a740 1 -- start start 2024-04-17T07:25:34.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.613+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x55959220d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.613+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55959220d990 0x559592212490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.613+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x559592214f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.613+0000 7fa389b9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559592173020 con 0x55959220d990 2024-04-17T07:25:34.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.613+0000 7fa389b9a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559592119460 con 0x559592212b00 2024-04-17T07:25:34.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.613+0000 7fa389b9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55959216efe0 con 0x55959220cfa0 2024-04-17T07:25:34.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.614+0000 7fa377e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x559592214f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.614+0000 7fa377e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x559592214f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50626/0 (socket says 172.21.15.86:50626) 2024-04-17T07:25:34.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.614+0000 7fa377e00640 1 -- 172.21.15.86:0/329565773 learned_addr learned my addr 172.21.15.86:0/329565773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:34.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.615+0000 7fa384e00640 1 --2- 172.21.15.86:0/329565773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x55959220d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.616+0000 7fa377e00640 1 -- 172.21.15.86:0/329565773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 msgr2=0x55959220d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.616+0000 7fa377e00640 1 --2- 172.21.15.86:0/329565773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x55959220d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.616+0000 7fa377e00640 1 -- 172.21.15.86:0/329565773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55959220d990 msgr2=0x559592212490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:25:34.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.616+0000 7fa377e00640 1 --2- 172.21.15.86:0/329565773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55959220d990 0x559592212490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.616+0000 7fa377e00640 1 -- 172.21.15.86:0/329565773 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559592121040 con 0x559592212b00 2024-04-17T07:25:34.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.616+0000 7fa377e00640 1 --2- 172.21.15.86:0/329565773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x559592214f90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa370009c90 tx=0x7fa370011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ba2bc0ea3482277 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.616+0000 7fa37d600640 1 -- 172.21.15.86:0/1135484765 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa368010a90 con 0x7fa3801041c0 2024-04-17T07:25:34.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.617+0000 7fa375600640 1 -- 172.21.15.86:0/329565773 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37001e070 con 0x559592212b00 2024-04-17T07:25:34.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.617+0000 7fa375600640 1 -- 172.21.15.86:0/329565773 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa370019ae0 con 0x559592212b00 2024-04-17T07:25:34.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.617+0000 7fa375600640 1 -- 172.21.15.86:0/329565773 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37000e920 con 0x559592212b00 2024-04-17T07:25:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.618+0000 7fa37d600640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa3640a3e70 0x7fa3640a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.618+0000 7fa389b9a740 1 -- 172.21.15.86:0/329565773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 msgr2=0x559592214f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.618+0000 7fa389b9a740 1 --2- 172.21.15.86:0/329565773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x559592214f90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa370009c90 tx=0x7fa370011810 comp rx=0 tx=0).stop 2024-04-17T07:25:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.619+0000 7fa37d600640 1 -- 172.21.15.86:0/1135484765 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa3680d06e0 con 0x7fa3801041c0 2024-04-17T07:25:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.619+0000 7fa384e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa3640a3e70 0x7fa3640a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.619+0000 7fa389b9a740 1 -- 172.21.15.86:0/329565773 shutdown_connections 2024-04-17T07:25:34.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.619+0000 7fa389b9a740 1 --2- 172.21.15.86:0/329565773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x559592214f90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.619+0000 7fa389b9a740 1 --2- 172.21.15.86:0/329565773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55959220d990 0x559592212490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.619+0000 7fa389b9a740 1 --2- 172.21.15.86:0/329565773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x55959220d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.619+0000 7fa389b9a740 1 -- 172.21.15.86:0/329565773 >> 172.21.15.86:0/329565773 conn(0x559592208510 msgr2=0x559592208910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.620+0000 7fa389b9a740 1 -- 172.21.15.86:0/329565773 shutdown_connections 2024-04-17T07:25:34.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.620+0000 7fa389b9a740 1 -- 172.21.15.86:0/329565773 wait complete. 2024-04-17T07:25:34.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.621+0000 7fa389b9a740 1 -- start start 2024-04-17T07:25:34.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.622+0000 7fa384e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa3640a3e70 0x7fa3640a6330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa378006010 tx=0x7fa37802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.622+0000 7fa389b9a740 1 Processor -- start 2024-04-17T07:25:34.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.623+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x559592217570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.623+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 0x559592217ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.623+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559592212b00 0x55959221eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.623+0000 7fa389b9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559592215ab0 con 0x559592212b00 2024-04-17T07:25:34.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.623+0000 7fa389b9a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55959212c830 con 0x55959220d990 2024-04-17T07:25:34.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.623+0000 7fa389b9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559592215c30 con 0x55959220cfa0 2024-04-17T07:25:34.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.623+0000 7fa389b9a740 10 client.0 _ll_register_callbacks cb 0x5595920fafb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:34.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.623+0000 7fa37fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x559592217570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.623+0000 7fa37fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x559592217570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36166/0 (socket says 172.21.15.86:36166) 2024-04-17T07:25:34.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.623+0000 7fa37fe00640 1 -- 172.21.15.86:0/1638486830 learned_addr learned my addr 172.21.15.86:0/1638486830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:34.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.624+0000 7fa377e00640 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 0x559592217ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.624+0000 7fa37fe00640 1 -- 172.21.15.86:0/1638486830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 msgr2=0x559592217ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.624+0000 7fa37fe00640 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 0x559592217ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.625+0000 7fa37fe00640 1 -- 172.21.15.86:0/1638486830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559592212b00 msgr2=0x55959221eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.625+0000 7fa37fe00640 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559592212b00 0x55959221eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.625+0000 7fa37fe00640 1 -- 172.21.15.86:0/1638486830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559592130920 con 0x55959220cfa0 2024-04-17T07:25:34.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.625+0000 7fa37fe00640 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x559592217570 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c6a0 tx=0x7fa36809c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed0aaac0fa9b7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.625+0000 7fa36e000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:34.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.625+0000 7fa377e00640 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 0x559592217ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:25:34.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.625+0000 7fa36e000640 1 -- 172.21.15.86:0/1638486830 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3680b4060 con 0x55959220cfa0 2024-04-17T07:25:34.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.625+0000 7fa36e000640 1 -- 172.21.15.86:0/1638486830 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa368085b70 con 0x55959220cfa0 2024-04-17T07:25:34.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.626+0000 7fa389b9a740 1 -- 172.21.15.86:0/1638486830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa370019ae0 con 0x55959220cfa0 2024-04-17T07:25:34.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.626+0000 7fa36e000640 1 -- 172.21.15.86:0/1638486830 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa36809cce0 con 0x55959220cfa0 2024-04-17T07:25:34.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.626+0000 7fa389b9a740 10 client.27154 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:34.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.626+0000 7fa389b9a740 1 -- 172.21.15.86:0/1638486830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595920fe7f0 con 0x55959220cfa0 2024-04-17T07:25:34.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.627+0000 7fa36e000640 1 -- 172.21.15.86:0/1638486830 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa370019ae0 con 0x55959220cfa0 2024-04-17T07:25:34.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.627+0000 7fa36e000640 1 -- 172.21.15.86:0/1638486830 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa368010430 con 0x55959220cfa0 2024-04-17T07:25:34.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.627+0000 7fa389b9a740 10 client.27154 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:34.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.627+0000 7fa389b9a740 10 client.27154 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:25:34.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.628+0000 7fa389b9a740 20 client.27154 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:34.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.628+0000 7fa389b9a740 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559592223050 0x5595922254c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.628+0000 7fa389b9a740 4 client.27154 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:34.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.628+0000 7fa389b9a740 1 -- 172.21.15.86:0/1638486830 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559592180a40 con 0x559592223050 2024-04-17T07:25:34.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.629+0000 7fa377e00640 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559592223050 0x5595922254c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.630+0000 7fa377e00640 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559592223050 0x5595922254c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4eeb0f8818e66cdc server_cookie=88c187eb90812465 in_seq=0 out_seq=0 2024-04-17T07:25:34.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.630+0000 7fa36e000640 10 client.27154 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:25:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.631+0000 7fa36e000640 1 -- 172.21.15.86:0/1638486830 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559592180a40 con 0x559592223050 2024-04-17T07:25:34.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.631+0000 7fa36e000640 10 client.27154 handle_command_reply: tid=0 2024-04-17T07:25:34.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.632+0000 7fa389b9a740 1 client.27154 shutdown 2024-04-17T07:25:34.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.633+0000 7fa389b9a740 1 -- 172.21.15.86:0/1638486830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 msgr2=0x559592217570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.633+0000 7fa389b9a740 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x559592217570 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c6a0 tx=0x7fa36809c4a0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.634+0000 7fa389b9a740 1 -- 172.21.15.86:0/1638486830 shutdown_connections 2024-04-17T07:25:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.634+0000 7fa389b9a740 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559592223050 0x5595922254c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.634+0000 7fa389b9a740 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559592212b00 0x55959221eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.634+0000 7fa389b9a740 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 0x559592217ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.634+0000 7fa389b9a740 1 --2- 172.21.15.86:0/1638486830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x559592217570 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.634+0000 7fa389b9a740 1 -- 172.21.15.86:0/1638486830 >> 172.21.15.86:0/1638486830 conn(0x559592208510 msgr2=0x559592207710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.634+0000 7fa36e000640 0 client.27154 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:25:34.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.634+0000 7fa389b9a740 1 -- 172.21.15.86:0/1638486830 shutdown_connections 2024-04-17T07:25:34.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.634+0000 7fa389b9a740 1 -- 172.21.15.86:0/1638486830 wait complete. 2024-04-17T07:25:34.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.635+0000 7fa389b9a740 20 client.27154 trim_cache size 0 max 16384 2024-04-17T07:25:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.664+0000 7fa389b9a740 1 Processor -- start 2024-04-17T07:25:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.664+0000 7fa389b9a740 1 -- start start 2024-04-17T07:25:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.664+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55959220cfa0 0x559592217a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.664+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220d990 0x55959221eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.664+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x55959220be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.664+0000 7fa389b9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801b11a0 con 0x55959220cfa0 2024-04-17T07:25:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.664+0000 7fa389b9a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801b2ce0 con 0x559592212b00 2024-04-17T07:25:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.664+0000 7fa389b9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801b1870 con 0x55959220d990 2024-04-17T07:25:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.665+0000 7fa384e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x55959220be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.665+0000 7fa384e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x55959220be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50644/0 (socket says 172.21.15.86:50644) 2024-04-17T07:25:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.665+0000 7fa384e00640 1 -- 172.21.15.86:0/2351243211 learned_addr learned my addr 172.21.15.86:0/2351243211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.665+0000 7fa377e00640 1 --2- 172.21.15.86:0/2351243211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220d990 0x55959221eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.665+0000 7fa37fe00640 1 --2- 172.21.15.86:0/2351243211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55959220cfa0 0x559592217a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.665+0000 7fa384e00640 1 -- 172.21.15.86:0/2351243211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220d990 msgr2=0x55959221eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.665+0000 7fa384e00640 1 --2- 172.21.15.86:0/2351243211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220d990 0x55959221eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.665+0000 7fa384e00640 1 -- 172.21.15.86:0/2351243211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55959220cfa0 msgr2=0x559592217a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.665+0000 7fa384e00640 1 --2- 172.21.15.86:0/2351243211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55959220cfa0 0x559592217a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.666+0000 7fa384e00640 1 -- 172.21.15.86:0/2351243211 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559591e853f0 con 0x559592212b00 2024-04-17T07:25:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.666+0000 7fa384e00640 1 --2- 172.21.15.86:0/2351243211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x55959220be80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa378035000 tx=0x7fa378041070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f130da27b7c50f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.667+0000 7fa374c00640 1 -- 172.21.15.86:0/2351243211 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378045070 con 0x559592212b00 2024-04-17T07:25:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.667+0000 7fa374c00640 1 -- 172.21.15.86:0/2351243211 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa378033900 con 0x559592212b00 2024-04-17T07:25:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.667+0000 7fa374c00640 1 -- 172.21.15.86:0/2351243211 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37804b9c0 con 0x559592212b00 2024-04-17T07:25:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.668+0000 7fa389b9a740 1 -- 172.21.15.86:0/2351243211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 msgr2=0x55959220be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.668+0000 7fa389b9a740 1 --2- 172.21.15.86:0/2351243211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x55959220be80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa378035000 tx=0x7fa378041070 comp rx=0 tx=0).stop 2024-04-17T07:25:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.668+0000 7fa389b9a740 1 -- 172.21.15.86:0/2351243211 shutdown_connections 2024-04-17T07:25:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.668+0000 7fa389b9a740 1 --2- 172.21.15.86:0/2351243211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559592212b00 0x55959220be80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.668+0000 7fa389b9a740 1 --2- 172.21.15.86:0/2351243211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220d990 0x55959221eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.668+0000 7fa389b9a740 1 --2- 172.21.15.86:0/2351243211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55959220cfa0 0x559592217a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.668+0000 7fa389b9a740 1 -- 172.21.15.86:0/2351243211 >> 172.21.15.86:0/2351243211 conn(0x559592208510 msgr2=0x55959220b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.669+0000 7fa389b9a740 1 -- 172.21.15.86:0/2351243211 shutdown_connections 2024-04-17T07:25:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.669+0000 7fa389b9a740 1 -- 172.21.15.86:0/2351243211 wait complete. 2024-04-17T07:25:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.669+0000 7fa389b9a740 1 -- start start 2024-04-17T07:25:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.670+0000 7fa389b9a740 1 Processor -- start 2024-04-17T07:25:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.672+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x55959221bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.672+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 0x55959221c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.672+0000 7fa389b9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559592212b00 0x55959221d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.672+0000 7fa389b9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801b2610 con 0x559592212b00 2024-04-17T07:25:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.672+0000 7fa389b9a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801b1f40 con 0x55959220d990 2024-04-17T07:25:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.672+0000 7fa389b9a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801af170 con 0x55959220cfa0 2024-04-17T07:25:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.672+0000 7fa389b9a740 10 client.0 _ll_register_callbacks cb 0x7fa368098910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.672+0000 7fa37fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x55959221bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.672+0000 7fa377e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 0x55959221c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.672+0000 7fa37fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x55959221bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36194/0 (socket says 172.21.15.86:36194) 2024-04-17T07:25:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.673+0000 7fa37fe00640 1 -- 172.21.15.86:0/3163044915 learned_addr learned my addr 172.21.15.86:0/3163044915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.673+0000 7fa384e00640 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559592212b00 0x55959221d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.673+0000 7fa37fe00640 1 -- 172.21.15.86:0/3163044915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 msgr2=0x55959221c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.673+0000 7fa37fe00640 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 0x55959221c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.673+0000 7fa37fe00640 1 -- 172.21.15.86:0/3163044915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559592212b00 msgr2=0x55959221d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.673+0000 7fa37fe00640 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559592212b00 0x55959221d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.673+0000 7fa37fe00640 1 -- 172.21.15.86:0/3163044915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559591f4f7d0 con 0x55959220cfa0 2024-04-17T07:25:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.674+0000 7fa384e00640 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559592212b00 0x55959221d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:25:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.674+0000 7fa37fe00640 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x55959221bf60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa368098a40 tx=0x7fa36809c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a8cebfa297e7755 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.674+0000 7fa36e000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:34.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.674+0000 7fa36e000640 1 -- 172.21.15.86:0/3163044915 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3680b4070 con 0x55959220cfa0 2024-04-17T07:25:34.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.674+0000 7fa36e000640 1 -- 172.21.15.86:0/3163044915 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa3680bb470 con 0x55959220cfa0 2024-04-17T07:25:34.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.674+0000 7fa389b9a740 1 -- 172.21.15.86:0/3163044915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa378033900 con 0x55959220cfa0 2024-04-17T07:25:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.674+0000 7fa36e000640 1 -- 172.21.15.86:0/3163044915 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa3680a0990 con 0x55959220cfa0 2024-04-17T07:25:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.675+0000 7fa389b9a740 10 client.27160 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.675+0000 7fa389b9a740 1 -- 172.21.15.86:0/3163044915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55959221ec40 con 0x55959220cfa0 2024-04-17T07:25:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.675+0000 7fa36e000640 1 -- 172.21.15.86:0/3163044915 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa378033900 con 0x55959220cfa0 2024-04-17T07:25:34.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.675+0000 7fa36e000640 1 -- 172.21.15.86:0/3163044915 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa36809b4a0 con 0x55959220cfa0 2024-04-17T07:25:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.678+0000 7fa389b9a740 10 client.27160 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.678+0000 7fa389b9a740 10 client.27160 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:25:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.678+0000 7fa389b9a740 20 client.27160 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.678+0000 7fa389b9a740 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5595922899e0 0x55959228be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.678+0000 7fa389b9a740 4 client.27160 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:25:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.678+0000 7fa389b9a740 1 -- 172.21.15.86:0/3163044915 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55959217a8b0 con 0x5595922899e0 2024-04-17T07:25:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.679+0000 7fa377e00640 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5595922899e0 0x55959228be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.680+0000 7fa377e00640 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5595922899e0 0x55959228be00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4aeb145bf3604a0a server_cookie=140dceba5cc787a2 in_seq=0 out_seq=0 2024-04-17T07:25:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.681+0000 7fa36e000640 10 client.27160 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:25:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.681+0000 7fa36e000640 1 -- 172.21.15.86:0/3163044915 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55959217a8b0 con 0x5595922899e0 2024-04-17T07:25:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.681+0000 7fa36e000640 10 client.27160 handle_command_reply: tid=0 2024-04-17T07:25:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.681+0000 7fa389b9a740 1 client.27160 shutdown 2024-04-17T07:25:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.681+0000 7fa389b9a740 1 -- 172.21.15.86:0/3163044915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 msgr2=0x55959221bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.681+0000 7fa389b9a740 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x55959221bf60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa368098a40 tx=0x7fa36809c3b0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.682+0000 7fa389b9a740 1 -- 172.21.15.86:0/3163044915 shutdown_connections 2024-04-17T07:25:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.682+0000 7fa389b9a740 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5595922899e0 0x55959228be00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.682+0000 7fa36e000640 0 client.27160 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:25:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.682+0000 7fa389b9a740 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559592212b00 0x55959221d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.682+0000 7fa389b9a740 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55959220d990 0x55959221c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.682+0000 7fa389b9a740 1 --2- 172.21.15.86:0/3163044915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55959220cfa0 0x55959221bf60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.682+0000 7fa389b9a740 1 -- 172.21.15.86:0/3163044915 >> 172.21.15.86:0/3163044915 conn(0x559592208510 msgr2=0x559592217890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.683+0000 7fa389b9a740 1 -- 172.21.15.86:0/3163044915 shutdown_connections 2024-04-17T07:25:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.683+0000 7fa389b9a740 1 -- 172.21.15.86:0/3163044915 wait complete. 2024-04-17T07:25:34.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.683+0000 7fa389b9a740 20 client.27160 trim_cache size 0 max 16384 2024-04-17T07:25:34.684 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:25:34.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.685+0000 7fa386e00640 1 -- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa3640a3e70 msgr2=0x7fa3640a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.685+0000 7fa386e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa3640a3e70 0x7fa3640a6330 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa378006010 tx=0x7fa37802e040 comp rx=0 tx=0).stop 2024-04-17T07:25:34.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.685+0000 7fa386e00640 1 -- 172.21.15.86:0/1135484765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3801041c0 msgr2=0x7fa38019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:34.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.685+0000 7fa386e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3801041c0 0x7fa38019e760 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa36800e9a0 tx=0x7fa36800ee60 comp rx=0 tx=0).stop 2024-04-17T07:25:34.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.686+0000 7fa386e00640 1 -- 172.21.15.86:0/1135484765 shutdown_connections 2024-04-17T07:25:34.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.686+0000 7fa386e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa3640a3e70 0x7fa3640a6330 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.686+0000 7fa386e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa380105d70 0x7fa3801a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.686+0000 7fa386e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3801053b0 0x7fa38019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.687+0000 7fa386e00640 1 --2- 172.21.15.86:0/1135484765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3801041c0 0x7fa38019e760 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:34.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.687+0000 7fa386e00640 1 -- 172.21.15.86:0/1135484765 >> 172.21.15.86:0/1135484765 conn(0x7fa3800ff710 msgr2=0x7fa380079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:34.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.687+0000 7fa386e00640 1 -- 172.21.15.86:0/1135484765 shutdown_connections 2024-04-17T07:25:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:34.688+0000 7fa386e00640 1 -- 172.21.15.86:0/1135484765 wait complete. 2024-04-17T07:25:34.717 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:25:35.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.016+0000 7f4a9f000640 1 -- 172.21.15.86:0/3256209659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a98103b60 msgr2=0x7f4a9810a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.016+0000 7f4a9f000640 1 --2- 172.21.15.86:0/3256209659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a98103b60 0x7f4a9810a3f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009be0 tx=0x7f4a9002bef0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.017+0000 7f4a9f000640 1 -- 172.21.15.86:0/3256209659 shutdown_connections 2024-04-17T07:25:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.017+0000 7f4a9f000640 1 --2- 172.21.15.86:0/3256209659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a98103b60 0x7f4a9810a3f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.017+0000 7f4a9f000640 1 --2- 172.21.15.86:0/3256209659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a981031a0 0x7f4a98103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.017+0000 7f4a9f000640 1 --2- 172.21.15.86:0/3256209659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a98101fb0 0x7f4a981023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.017+0000 7f4a9f000640 1 -- 172.21.15.86:0/3256209659 >> 172.21.15.86:0/3256209659 conn(0x7f4a980fd540 msgr2=0x7f4a980ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.017+0000 7f4a9f000640 1 -- 172.21.15.86:0/3256209659 shutdown_connections 2024-04-17T07:25:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.017+0000 7f4a9f000640 1 -- 172.21.15.86:0/3256209659 wait complete. 2024-04-17T07:25:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.019+0000 7f4a9f000640 1 Processor -- start 2024-04-17T07:25:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.019+0000 7f4a9f000640 1 -- start start 2024-04-17T07:25:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.020+0000 7f4a9f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98101fb0 0x7f4a98070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.020+0000 7f4a9f000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a981031a0 0x7f4a98070e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.020+0000 7f4a9f000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a98103b60 0x7f4a981a8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.020+0000 7f4a9f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9810db00 con 0x7f4a98101fb0 2024-04-17T07:25:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.020+0000 7f4a9f000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9810d980 con 0x7f4a981031a0 2024-04-17T07:25:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.020+0000 7f4a9f000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9810dc80 con 0x7f4a98103b60 2024-04-17T07:25:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.021+0000 7f4a8fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a981031a0 0x7f4a98070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.021+0000 7f4a9d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a98103b60 0x7f4a981a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.021+0000 7f4a9d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a98103b60 0x7f4a981a8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36202/0 (socket says 172.21.15.86:36202) 2024-04-17T07:25:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.021+0000 7f4a9d000640 1 -- 172.21.15.86:0/502590689 learned_addr learned my addr 172.21.15.86:0/502590689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.021+0000 7f4a9d000640 1 -- 172.21.15.86:0/502590689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a981031a0 msgr2=0x7f4a98070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.021+0000 7f4a9d000640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a981031a0 0x7f4a98070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.022+0000 7f4a9d000640 1 -- 172.21.15.86:0/502590689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98101fb0 msgr2=0x7f4a98070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.022+0000 7f4a97e00640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98101fb0 0x7f4a98070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.022+0000 7f4a9d000640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98101fb0 0x7f4a98070920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.022+0000 7f4a9d000640 1 -- 172.21.15.86:0/502590689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a981a8750 con 0x7f4a98103b60 2024-04-17T07:25:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.023+0000 7f4a9d000640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a98103b60 0x7f4a981a8210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4a90034fd0 tx=0x7f4a9002cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.023+0000 7f4a95600640 1 -- 172.21.15.86:0/502590689 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90043020 con 0x7f4a98103b60 2024-04-17T07:25:35.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.023+0000 7f4a9f000640 1 -- 172.21.15.86:0/502590689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a98074190 con 0x7f4a98103b60 2024-04-17T07:25:35.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.024+0000 7f4a9f000640 1 -- 172.21.15.86:0/502590689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a98074730 con 0x7f4a98103b60 2024-04-17T07:25:35.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.024+0000 7f4a95600640 1 -- 172.21.15.86:0/502590689 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a90003e20 con 0x7f4a98103b60 2024-04-17T07:25:35.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.024+0000 7f4a95600640 1 -- 172.21.15.86:0/502590689 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90043020 con 0x7f4a98103b60 2024-04-17T07:25:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.029+0000 7f4aa1bf8740 1 Processor -- start 2024-04-17T07:25:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.029+0000 7f4a95600640 1 -- 172.21.15.86:0/502590689 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4a90046020 con 0x7f4a98103b60 2024-04-17T07:25:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.030+0000 7f4aa1bf8740 1 -- start start 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.032+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.032+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 0x55567b99b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.032+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 0x55567b99df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.032+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55567b8fc020 con 0x55567b996990 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.032+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55567b8a2460 con 0x55567b99bb00 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.032+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55567b8f7fe0 con 0x55567b995fa0 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.033+0000 7f4a8fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.034+0000 7f4a8fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36214/0 (socket says 172.21.15.86:36214) 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.034+0000 7f4a8fe00640 1 -- 172.21.15.86:0/1591340029 learned_addr learned my addr 172.21.15.86:0/1591340029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.034+0000 7f4a97e00640 1 --2- 172.21.15.86:0/1591340029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 0x55567b99b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.034+0000 7f4a8fe00640 1 -- 172.21.15.86:0/1591340029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 msgr2=0x55567b99df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.034+0000 7f4a8fe00640 1 --2- 172.21.15.86:0/1591340029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 0x55567b99df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.034+0000 7f4a8fe00640 1 -- 172.21.15.86:0/1591340029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 msgr2=0x55567b99b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.035+0000 7f4a8fe00640 1 --2- 172.21.15.86:0/1591340029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 0x55567b99b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.035+0000 7f4a8fe00640 1 -- 172.21.15.86:0/1591340029 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567b8aa040 con 0x55567b995fa0 2024-04-17T07:25:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.035+0000 7f4a97e00640 1 --2- 172.21.15.86:0/1591340029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 0x55567b99b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:25:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.035+0000 7f4a95600640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4a7c0a3eb0 0x7f4a7c0a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.035+0000 7f4a95600640 1 -- 172.21.15.86:0/502590689 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4a900f2c50 con 0x7f4a98103b60 2024-04-17T07:25:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.035+0000 7f4a97e00640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4a7c0a3eb0 0x7f4a7c0a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.035+0000 7f4a8fe00640 1 --2- 172.21.15.86:0/1591340029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9963c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4a80010dd0 tx=0x7f4a800099d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ba9c8fdc0ffd13 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.036+0000 7f4a8d600640 1 -- 172.21.15.86:0/1591340029 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80004310 con 0x55567b995fa0 2024-04-17T07:25:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.036+0000 7f4a8d600640 1 -- 172.21.15.86:0/1591340029 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a8001f070 con 0x55567b995fa0 2024-04-17T07:25:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.036+0000 7f4a8d600640 1 -- 172.21.15.86:0/1591340029 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8001a780 con 0x55567b995fa0 2024-04-17T07:25:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.037+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/1591340029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 msgr2=0x55567b9963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.037+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/1591340029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9963c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4a80010dd0 tx=0x7f4a800099d0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.037+0000 7f4a97e00640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4a7c0a3eb0 0x7f4a7c0a6370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55567b996d90 tx=0x7f4a88005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.038+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/1591340029 shutdown_connections 2024-04-17T07:25:35.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.038+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/1591340029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 0x55567b99df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.038+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/1591340029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 0x55567b99b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.038+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/1591340029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9963c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.038+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/1591340029 >> 172.21.15.86:0/1591340029 conn(0x55567b991510 msgr2=0x55567b991910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.038+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/1591340029 shutdown_connections 2024-04-17T07:25:35.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.038+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/1591340029 wait complete. 2024-04-17T07:25:35.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.039+0000 7f4aa1bf8740 1 -- start start 2024-04-17T07:25:35.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.039+0000 7f4aa1bf8740 1 Processor -- start 2024-04-17T07:25:35.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.040+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b995fa0 0x55567b9a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.040+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 0x55567b9a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.040+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b99bb00 0x55567b9b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.040+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55567b99eab0 con 0x55567b995fa0 2024-04-17T07:25:35.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.040+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55567b8b5830 con 0x55567b996990 2024-04-17T07:25:35.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.040+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55567b99ec30 con 0x55567b99bb00 2024-04-17T07:25:35.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.040+0000 7f4aa1bf8740 10 client.0 _ll_register_callbacks cb 0x55567b883fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:35.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.041+0000 7f4a9d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b99bb00 0x55567b9b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.041+0000 7f4a9d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b99bb00 0x55567b9b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36216/0 (socket says 172.21.15.86:36216) 2024-04-17T07:25:35.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.041+0000 7f4a9d000640 1 -- 172.21.15.86:0/3770818098 learned_addr learned my addr 172.21.15.86:0/3770818098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.041+0000 7f4a97e00640 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b995fa0 0x55567b9a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.041+0000 7f4a8fe00640 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 0x55567b9a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.041+0000 7f4a9d000640 1 -- 172.21.15.86:0/3770818098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 msgr2=0x55567b9a92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.041+0000 7f4a9d000640 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 0x55567b9a92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.041+0000 7f4a9d000640 1 -- 172.21.15.86:0/3770818098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b995fa0 msgr2=0x55567b9a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.041+0000 7f4a9d000640 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b995fa0 0x55567b9a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.041+0000 7f4a9d000640 1 -- 172.21.15.86:0/3770818098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55567b8b9920 con 0x55567b99bb00 2024-04-17T07:25:35.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.042+0000 7f4a9d000640 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b99bb00 0x55567b9b0300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a900c7290 tx=0x7f4a900be980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a7af9fa15799a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.042+0000 7f4a86000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:35.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.042+0000 7f4a97e00640 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b995fa0 0x55567b9a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:35.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.042+0000 7f4a86000640 1 -- 172.21.15.86:0/3770818098 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a900bd0d0 con 0x55567b99bb00 2024-04-17T07:25:35.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.042+0000 7f4a86000640 1 -- 172.21.15.86:0/3770818098 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4a900041b0 con 0x55567b99bb00 2024-04-17T07:25:35.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.042+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3770818098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a8001f070 con 0x55567b99bb00 2024-04-17T07:25:35.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.042+0000 7f4a86000640 1 -- 172.21.15.86:0/3770818098 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a9003c350 con 0x55567b99bb00 2024-04-17T07:25:35.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.042+0000 7f4aa1bf8740 10 client.27172 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:35.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.043+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3770818098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55567b8877f0 con 0x55567b99bb00 2024-04-17T07:25:35.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.043+0000 7f4a86000640 1 -- 172.21.15.86:0/3770818098 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a8001f070 con 0x55567b99bb00 2024-04-17T07:25:35.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.043+0000 7f4a86000640 1 -- 172.21.15.86:0/3770818098 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4a90050060 con 0x55567b99bb00 2024-04-17T07:25:35.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.044+0000 7f4aa1bf8740 10 client.27172 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:35.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.044+0000 7f4aa1bf8740 10 client.27172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:25:35.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.044+0000 7f4aa1bf8740 20 client.27172 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:35.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.044+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55567b9b4760 0x55567b9b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.044+0000 7f4aa1bf8740 4 client.27172 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:35.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.044+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3770818098 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55567b909a40 con 0x55567b9b4760 2024-04-17T07:25:35.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.044+0000 7f4a97e00640 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55567b9b4760 0x55567b9b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.048+0000 7f4a97e00640 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55567b9b4760 0x55567b9b6bd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fedfaf182345fe28 server_cookie=439ab31fb462318c in_seq=0 out_seq=0 2024-04-17T07:25:35.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.048+0000 7f4a86000640 10 client.27172 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:25:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.050+0000 7f4a86000640 1 -- 172.21.15.86:0/3770818098 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55567b909a40 con 0x55567b9b4760 2024-04-17T07:25:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.050+0000 7f4a86000640 10 client.27172 handle_command_reply: tid=0 2024-04-17T07:25:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.050+0000 7f4aa1bf8740 1 client.27172 shutdown 2024-04-17T07:25:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3770818098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b99bb00 msgr2=0x55567b9b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b99bb00 0x55567b9b0300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a900c7290 tx=0x7f4a900be980 comp rx=0 tx=0).stop 2024-04-17T07:25:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3770818098 shutdown_connections 2024-04-17T07:25:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55567b9b4760 0x55567b9b6bd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b99bb00 0x55567b9b0300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 0x55567b9a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3770818098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b995fa0 0x55567b9a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3770818098 >> 172.21.15.86:0/3770818098 conn(0x55567b991510 msgr2=0x55567b990710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4a86000640 0 client.27172 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:25:35.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3770818098 shutdown_connections 2024-04-17T07:25:35.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.051+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3770818098 wait complete. 2024-04-17T07:25:35.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.052+0000 7f4aa1bf8740 20 client.27172 trim_cache size 0 max 16384 2024-04-17T07:25:35.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.088+0000 7f4aa1bf8740 1 Processor -- start 2024-04-17T07:25:35.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.088+0000 7f4aa1bf8740 1 -- start start 2024-04-17T07:25:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.088+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b98fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.088+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 0x55567b9b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.088+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 0x55567b9903c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.088+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b10d0 con 0x55567b996990 2024-04-17T07:25:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.088+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b2c10 con 0x55567b99bb00 2024-04-17T07:25:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.088+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b17a0 con 0x55567b995fa0 2024-04-17T07:25:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.091+0000 7f4a9d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 0x55567b9903c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.091+0000 7f4a9d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 0x55567b9903c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50706/0 (socket says 172.21.15.86:50706) 2024-04-17T07:25:35.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.091+0000 7f4a9d000640 1 -- 172.21.15.86:0/3985152181 learned_addr learned my addr 172.21.15.86:0/3985152181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.091+0000 7f4a8fe00640 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 0x55567b9b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.091+0000 7f4a97e00640 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b98fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.092+0000 7f4a9d000640 1 -- 172.21.15.86:0/3985152181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 msgr2=0x55567b98fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.092+0000 7f4a9d000640 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b98fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.092+0000 7f4a9d000640 1 -- 172.21.15.86:0/3985152181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 msgr2=0x55567b9b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.093+0000 7f4a9d000640 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 0x55567b9b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.093+0000 7f4a9d000640 1 -- 172.21.15.86:0/3985152181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567b60e3f0 con 0x55567b99bb00 2024-04-17T07:25:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.093+0000 7f4a97e00640 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b98fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.095+0000 7f4a9d000640 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 0x55567b9903c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009330 tx=0x7f4a900bb060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b959854e65a17f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.095+0000 7f4a8fe00640 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 0x55567b9b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:35.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.096+0000 7f4a8cc00640 1 -- 172.21.15.86:0/3985152181 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a900bd0d0 con 0x55567b99bb00 2024-04-17T07:25:35.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.096+0000 7f4a8cc00640 1 -- 172.21.15.86:0/3985152181 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a900be400 con 0x55567b99bb00 2024-04-17T07:25:35.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.096+0000 7f4a8cc00640 1 -- 172.21.15.86:0/3985152181 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a900041b0 con 0x55567b99bb00 2024-04-17T07:25:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.098+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3985152181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 msgr2=0x55567b9903c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.098+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 0x55567b9903c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009330 tx=0x7f4a900bb060 comp rx=0 tx=0).stop 2024-04-17T07:25:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.098+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3985152181 shutdown_connections 2024-04-17T07:25:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.098+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b99bb00 0x55567b9903c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.098+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b996990 0x55567b9b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.098+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3985152181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b98fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.098+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3985152181 >> 172.21.15.86:0/3985152181 conn(0x55567b991510 msgr2=0x55567b997f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.099+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3985152181 shutdown_connections 2024-04-17T07:25:35.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.099+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3985152181 wait complete. 2024-04-17T07:25:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.100+0000 7f4aa1bf8740 1 -- start start 2024-04-17T07:25:35.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.100+0000 7f4aa1bf8740 1 Processor -- start 2024-04-17T07:25:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.101+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.101+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 0x55567b9ad550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.101+0000 7f4aa1bf8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b99bb00 0x55567ba18d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.101+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b2540 con 0x55567b99bb00 2024-04-17T07:25:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.101+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981b1e70 con 0x55567b996990 2024-04-17T07:25:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.101+0000 7f4aa1bf8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a981af0a0 con 0x55567b995fa0 2024-04-17T07:25:35.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.101+0000 7f4aa1bf8740 10 client.0 _ll_register_callbacks cb 0x7f4a900beb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:35.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a8fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 0x55567b9ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a8fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 0x55567b9ad550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50716/0 (socket says 172.21.15.86:50716) 2024-04-17T07:25:35.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a8fe00640 1 -- 172.21.15.86:0/3550822355 learned_addr learned my addr 172.21.15.86:0/3550822355 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a97e00640 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a9d000640 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b99bb00 0x55567ba18d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a97e00640 1 -- 172.21.15.86:0/3550822355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 msgr2=0x55567b9ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a97e00640 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 0x55567b9ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a97e00640 1 -- 172.21.15.86:0/3550822355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b99bb00 msgr2=0x55567ba18d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a97e00640 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b99bb00 0x55567ba18d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a97e00640 1 -- 172.21.15.86:0/3550822355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55567b6d87d0 con 0x55567b995fa0 2024-04-17T07:25:35.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.102+0000 7f4a9d000640 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b99bb00 0x55567ba18d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.103+0000 7f4a97e00640 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9ad010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55567b9b7420 tx=0x7f4a880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7462efdac55825ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.104+0000 7f4a86000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.104+0000 7f4a86000640 1 -- 172.21.15.86:0/3550822355 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88009910 con 0x55567b995fa0 2024-04-17T07:25:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.104+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3550822355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a900be400 con 0x55567b995fa0 2024-04-17T07:25:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.104+0000 7f4a86000640 1 -- 172.21.15.86:0/3550822355 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4a88011da0 con 0x55567b995fa0 2024-04-17T07:25:35.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.105+0000 7f4a86000640 1 -- 172.21.15.86:0/3550822355 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4a8803a050 con 0x55567b995fa0 2024-04-17T07:25:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.105+0000 7f4aa1bf8740 10 client.27178 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:35.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.106+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3550822355 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55567ba1a1c0 con 0x55567b995fa0 2024-04-17T07:25:35.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.106+0000 7f4a86000640 1 -- 172.21.15.86:0/3550822355 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a900be400 con 0x55567b995fa0 2024-04-17T07:25:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.107+0000 7f4a86000640 1 -- 172.21.15.86:0/3550822355 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4a8801e4b0 con 0x55567b995fa0 2024-04-17T07:25:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.107+0000 7f4aa1bf8740 10 client.27178 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.107+0000 7f4aa1bf8740 10 client.27178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:25:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.107+0000 7f4aa1bf8740 20 client.27178 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.107+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55567ba1ab10 0x55567ba1cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.107+0000 7f4aa1bf8740 4 client.27178 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:25:35.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.107+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3550822355 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55567b9038b0 con 0x55567ba1ab10 2024-04-17T07:25:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.107+0000 7f4a8fe00640 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55567ba1ab10 0x55567ba1cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.108+0000 7f4a86000640 10 client.27178 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:25:35.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.108+0000 7f4a8fe00640 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55567ba1ab10 0x55567ba1cf00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e07ddc594c27ba2 server_cookie=4e83e2332ee0d2e0 in_seq=0 out_seq=0 2024-04-17T07:25:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.109+0000 7f4a86000640 1 -- 172.21.15.86:0/3550822355 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55567b9038b0 con 0x55567ba1ab10 2024-04-17T07:25:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.109+0000 7f4a86000640 10 client.27178 handle_command_reply: tid=0 2024-04-17T07:25:35.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.109+0000 7f4aa1bf8740 1 client.27178 shutdown 2024-04-17T07:25:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.109+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3550822355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 msgr2=0x55567b9ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.109+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9ad010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55567b9b7420 tx=0x7f4a880046c0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.110+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3550822355 shutdown_connections 2024-04-17T07:25:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.110+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55567ba1ab10 0x55567ba1cf00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.110+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55567b99bb00 0x55567ba18d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.110+0000 7f4a86000640 0 client.27178 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:25:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.110+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55567b996990 0x55567b9ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.110+0000 7f4aa1bf8740 1 --2- 172.21.15.86:0/3550822355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55567b995fa0 0x55567b9ad010 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.110+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3550822355 >> 172.21.15.86:0/3550822355 conn(0x55567b991510 msgr2=0x55567b994770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.110+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3550822355 shutdown_connections 2024-04-17T07:25:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.111+0000 7f4aa1bf8740 1 -- 172.21.15.86:0/3550822355 wait complete. 2024-04-17T07:25:35.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.111+0000 7f4aa1bf8740 20 client.27178 trim_cache size 0 max 16384 2024-04-17T07:25:35.112 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:25:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.112+0000 7f4a9f000640 1 -- 172.21.15.86:0/502590689 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4a7c0a3eb0 msgr2=0x7f4a7c0a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.112+0000 7f4a9f000640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4a7c0a3eb0 0x7f4a7c0a6370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55567b996d90 tx=0x7f4a88005eb0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.113+0000 7f4a9f000640 1 -- 172.21.15.86:0/502590689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a98103b60 msgr2=0x7f4a981a8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.113+0000 7f4a9f000640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a98103b60 0x7f4a981a8210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4a90034fd0 tx=0x7f4a9002cbc0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.113+0000 7f4a9f000640 1 -- 172.21.15.86:0/502590689 shutdown_connections 2024-04-17T07:25:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.113+0000 7f4a9f000640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4a7c0a3eb0 0x7f4a7c0a6370 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.113+0000 7f4a9f000640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a98103b60 0x7f4a981a8210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.113+0000 7f4a9f000640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a981031a0 0x7f4a98070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.113+0000 7f4a9f000640 1 --2- 172.21.15.86:0/502590689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a98101fb0 0x7f4a98070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.113+0000 7f4a9f000640 1 -- 172.21.15.86:0/502590689 >> 172.21.15.86:0/502590689 conn(0x7f4a980fd540 msgr2=0x7f4a98106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.113+0000 7f4a9f000640 1 -- 172.21.15.86:0/502590689 shutdown_connections 2024-04-17T07:25:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.113+0000 7f4a9f000640 1 -- 172.21.15.86:0/502590689 wait complete. 2024-04-17T07:25:35.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:25:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.465+0000 7f3d98000640 1 -- 172.21.15.86:0/892028331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d9010b8d0 msgr2=0x7f3d9010dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.465+0000 7f3d98000640 1 --2- 172.21.15.86:0/892028331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d9010b8d0 0x7f3d9010dd60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009860 tx=0x7f3d7c02bb90 comp rx=0 tx=0).stop 2024-04-17T07:25:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.466+0000 7f3d98000640 1 -- 172.21.15.86:0/892028331 shutdown_connections 2024-04-17T07:25:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.466+0000 7f3d98000640 1 --2- 172.21.15.86:0/892028331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d9010e2a0 0x7f3d9010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.466+0000 7f3d98000640 1 --2- 172.21.15.86:0/892028331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d9010b8d0 0x7f3d9010dd60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.466+0000 7f3d98000640 1 --2- 172.21.15.86:0/892028331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d90108f60 0x7f3d9010b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.466+0000 7f3d98000640 1 -- 172.21.15.86:0/892028331 >> 172.21.15.86:0/892028331 conn(0x7f3d9006edf0 msgr2=0x7f3d90071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.467+0000 7f3d98000640 1 -- 172.21.15.86:0/892028331 shutdown_connections 2024-04-17T07:25:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.467+0000 7f3d98000640 1 -- 172.21.15.86:0/892028331 wait complete. 2024-04-17T07:25:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.469+0000 7f3d98000640 1 Processor -- start 2024-04-17T07:25:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.470+0000 7f3d98000640 1 -- start start 2024-04-17T07:25:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.470+0000 7f3d98000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108f60 0x7f3d901a7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.470+0000 7f3d98000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9010b8d0 0x7f3d901a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.470+0000 7f3d98000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d9010e2a0 0x7f3d901ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.470+0000 7f3d98000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9011c910 con 0x7f3d9010b8d0 2024-04-17T07:25:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.470+0000 7f3d98000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9011c790 con 0x7f3d9010e2a0 2024-04-17T07:25:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.470+0000 7f3d98000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9011ca90 con 0x7f3d90108f60 2024-04-17T07:25:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.471+0000 7f3d94c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9010b8d0 0x7f3d901a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.471+0000 7f3d95600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108f60 0x7f3d901a7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.471+0000 7f3d95600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108f60 0x7f3d901a7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36240/0 (socket says 172.21.15.86:36240) 2024-04-17T07:25:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.471+0000 7f3d95600640 1 -- 172.21.15.86:0/1496253705 learned_addr learned my addr 172.21.15.86:0/1496253705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.472+0000 7f3d95600640 1 -- 172.21.15.86:0/1496253705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d9010e2a0 msgr2=0x7f3d901ae680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T07:25:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.472+0000 7f3d95600640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d9010e2a0 0x7f3d901ae680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.472+0000 7f3d95600640 1 -- 172.21.15.86:0/1496253705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9010b8d0 msgr2=0x7f3d901a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.472+0000 7f3d95600640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9010b8d0 0x7f3d901a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.472+0000 7f3d95600640 1 -- 172.21.15.86:0/1496253705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d901aecf0 con 0x7f3d90108f60 2024-04-17T07:25:35.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.473+0000 7f3d95600640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108f60 0x7f3d901a7120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400c890 tx=0x7f3d8400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.474+0000 7f3d8e000640 1 -- 172.21.15.86:0/1496253705 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84004220 con 0x7f3d90108f60 2024-04-17T07:25:35.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.474+0000 7f3d8e000640 1 -- 172.21.15.86:0/1496253705 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d840043c0 con 0x7f3d90108f60 2024-04-17T07:25:35.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.474+0000 7f3d8e000640 1 -- 172.21.15.86:0/1496253705 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84004a00 con 0x7f3d90108f60 2024-04-17T07:25:35.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.474+0000 7f3d98000640 1 -- 172.21.15.86:0/1496253705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d901aefe0 con 0x7f3d90108f60 2024-04-17T07:25:35.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.475+0000 7f3d98000640 1 -- 172.21.15.86:0/1496253705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d901af3f0 con 0x7f3d90108f60 2024-04-17T07:25:35.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.481+0000 7f3d8e000640 1 -- 172.21.15.86:0/1496253705 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3d84004ba0 con 0x7f3d90108f60 2024-04-17T07:25:35.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.483+0000 7f3d8e000640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3d780a3d30 0x7f3d780a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.484+0000 7f3d8e000640 1 -- 172.21.15.86:0/1496253705 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3d840d0330 con 0x7f3d90108f60 2024-04-17T07:25:35.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.484+0000 7f3d94c00640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3d780a3d30 0x7f3d780a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.485+0000 7f3d94c00640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3d780a3d30 0x7f3d780a61f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009830 tx=0x7f3d7c034980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.489+0000 7f3d9ad84740 1 Processor -- start 2024-04-17T07:25:35.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.489+0000 7f3d9ad84740 1 -- start start 2024-04-17T07:25:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.490+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a53fa0 0x558c10a543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.490+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a54990 0x558c10a59490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.490+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a59b00 0x558c10a5bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.491+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c109ba020 con 0x558c10a59b00 2024-04-17T07:25:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.491+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558c10960460 con 0x558c10a54990 2024-04-17T07:25:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.491+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558c109b5fe0 con 0x558c10a53fa0 2024-04-17T07:25:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.491+0000 7f3d95600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a53fa0 0x558c10a543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.492+0000 7f3d95600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a53fa0 0x558c10a543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36242/0 (socket says 172.21.15.86:36242) 2024-04-17T07:25:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.492+0000 7f3d95600640 1 -- 172.21.15.86:0/1368330434 learned_addr learned my addr 172.21.15.86:0/1368330434 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.492+0000 7f3d94c00640 1 --2- 172.21.15.86:0/1368330434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a54990 0x558c10a59490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.492+0000 7f3d96000640 1 --2- 172.21.15.86:0/1368330434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a59b00 0x558c10a5bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.493+0000 7f3d95600640 1 -- 172.21.15.86:0/1368330434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a54990 msgr2=0x558c10a59490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.493+0000 7f3d95600640 1 --2- 172.21.15.86:0/1368330434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a54990 0x558c10a59490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.493+0000 7f3d95600640 1 -- 172.21.15.86:0/1368330434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a59b00 msgr2=0x558c10a5bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.493+0000 7f3d95600640 1 --2- 172.21.15.86:0/1368330434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a59b00 0x558c10a5bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.493+0000 7f3d95600640 1 -- 172.21.15.86:0/1368330434 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c10968040 con 0x558c10a53fa0 2024-04-17T07:25:35.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.493+0000 7f3d96000640 1 --2- 172.21.15.86:0/1368330434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a59b00 0x558c10a5bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:35.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.494+0000 7f3d95600640 1 --2- 172.21.15.86:0/1368330434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a53fa0 0x558c10a543c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3d8409c220 tx=0x7f3d8409bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3f204fcbcc1e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.495+0000 7f3d82600640 1 -- 172.21.15.86:0/1368330434 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d840b5070 con 0x558c10a53fa0 2024-04-17T07:25:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.495+0000 7f3d82600640 1 -- 172.21.15.86:0/1368330434 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d840a0640 con 0x558c10a53fa0 2024-04-17T07:25:35.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.495+0000 7f3d82600640 1 -- 172.21.15.86:0/1368330434 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8401d070 con 0x558c10a53fa0 2024-04-17T07:25:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.496+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1368330434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a53fa0 msgr2=0x558c10a543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.496+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/1368330434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a53fa0 0x558c10a543c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3d8409c220 tx=0x7f3d8409bd00 comp rx=0 tx=0).stop 2024-04-17T07:25:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.496+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1368330434 shutdown_connections 2024-04-17T07:25:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.496+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/1368330434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a59b00 0x558c10a5bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.496+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/1368330434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a54990 0x558c10a59490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.496+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/1368330434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a53fa0 0x558c10a543c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.496+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1368330434 >> 172.21.15.86:0/1368330434 conn(0x558c10a4f510 msgr2=0x558c10a4f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.497+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1368330434 shutdown_connections 2024-04-17T07:25:35.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.497+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1368330434 wait complete. 2024-04-17T07:25:35.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.498+0000 7f3d9ad84740 1 -- start start 2024-04-17T07:25:35.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.499+0000 7f3d9ad84740 1 Processor -- start 2024-04-17T07:25:35.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.499+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a53fa0 0x558c10a5e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.499+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a54990 0x558c10a5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.499+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a59b00 0x558c10a65b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.499+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558c10a5cab0 con 0x558c10a54990 2024-04-17T07:25:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.499+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558c10973830 con 0x558c10a53fa0 2024-04-17T07:25:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.499+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558c10a5cc30 con 0x558c10a59b00 2024-04-17T07:25:35.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.500+0000 7f3d9ad84740 10 client.0 _ll_register_callbacks cb 0x558c10941fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.500+0000 7f3d95600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a53fa0 0x558c10a5e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.500+0000 7f3d96000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a59b00 0x558c10a65b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.501+0000 7f3d96000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a59b00 0x558c10a65b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36250/0 (socket says 172.21.15.86:36250) 2024-04-17T07:25:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.501+0000 7f3d96000640 1 -- 172.21.15.86:0/4119319715 learned_addr learned my addr 172.21.15.86:0/4119319715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.501+0000 7f3d94c00640 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a54990 0x558c10a5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:35 smithi098 ceph-mon[35418]: pgmap v776: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 648 op/s 2024-04-17T07:25:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.502+0000 7f3d96000640 1 -- 172.21.15.86:0/4119319715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a53fa0 msgr2=0x558c10a5e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.502+0000 7f3d96000640 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a53fa0 0x558c10a5e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.502+0000 7f3d96000640 1 -- 172.21.15.86:0/4119319715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a54990 msgr2=0x558c10a5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.502+0000 7f3d96000640 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a54990 0x558c10a5eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.502+0000 7f3d96000640 1 -- 172.21.15.86:0/4119319715 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c10977920 con 0x558c10a59b00 2024-04-17T07:25:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.503+0000 7f3d95600640 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a53fa0 0x558c10a5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.503+0000 7f3d96000640 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a59b00 0x558c10a65b40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800fbd0 tx=0x7f3d8800c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3efe3ae64dd33c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.503+0000 7f3d76a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.504+0000 7f3d76a00640 1 -- 172.21.15.86:0/4119319715 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8801d060 con 0x558c10a59b00 2024-04-17T07:25:35.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.504+0000 7f3d76a00640 1 -- 172.21.15.86:0/4119319715 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3d8800ce10 con 0x558c10a59b00 2024-04-17T07:25:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.504+0000 7f3d9ad84740 1 -- 172.21.15.86:0/4119319715 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d840a0640 con 0x558c10a59b00 2024-04-17T07:25:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.505+0000 7f3d76a00640 1 -- 172.21.15.86:0/4119319715 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d8800d5f0 con 0x558c10a59b00 2024-04-17T07:25:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.505+0000 7f3d9ad84740 10 client.27190 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.505+0000 7f3d9ad84740 1 -- 172.21.15.86:0/4119319715 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c109457f0 con 0x558c10a59b00 2024-04-17T07:25:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.505+0000 7f3d76a00640 1 -- 172.21.15.86:0/4119319715 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d840a0640 con 0x558c10a59b00 2024-04-17T07:25:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.506+0000 7f3d76a00640 1 -- 172.21.15.86:0/4119319715 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3d8800b040 con 0x558c10a59b00 2024-04-17T07:25:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.506+0000 7f3d9ad84740 10 client.27190 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.506+0000 7f3d9ad84740 10 client.27190 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:25:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.506+0000 7f3d9ad84740 20 client.27190 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.506+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558c10a6a0c0 0x558c10a6c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.506+0000 7f3d9ad84740 4 client.27190 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:35.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.506+0000 7f3d9ad84740 1 -- 172.21.15.86:0/4119319715 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c109c7a40 con 0x558c10a6a0c0 2024-04-17T07:25:35.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.507+0000 7f3d95600640 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558c10a6a0c0 0x558c10a6c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.508+0000 7f3d95600640 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558c10a6a0c0 0x558c10a6c520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6867e5e158e702e server_cookie=95dfde10615b078c in_seq=0 out_seq=0 2024-04-17T07:25:35.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.508+0000 7f3d76a00640 10 client.27190 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:25:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d76a00640 1 -- 172.21.15.86:0/4119319715 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c109c7a40 con 0x558c10a6a0c0 2024-04-17T07:25:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d76a00640 10 client.27190 handle_command_reply: tid=0 2024-04-17T07:25:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d9ad84740 1 client.27190 shutdown 2024-04-17T07:25:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d9ad84740 1 -- 172.21.15.86:0/4119319715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a59b00 msgr2=0x558c10a65b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a59b00 0x558c10a65b40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800fbd0 tx=0x7f3d8800c740 comp rx=0 tx=0).stop 2024-04-17T07:25:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d9ad84740 1 -- 172.21.15.86:0/4119319715 shutdown_connections 2024-04-17T07:25:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a59b00 0x558c10a65b40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a54990 0x558c10a5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558c10a6a0c0 0x558c10a6c520 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/4119319715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a53fa0 0x558c10a5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.510+0000 7f3d9ad84740 1 -- 172.21.15.86:0/4119319715 >> 172.21.15.86:0/4119319715 conn(0x558c10a4f510 msgr2=0x558c10a4e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.511+0000 7f3d76a00640 0 client.27190 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:25:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.511+0000 7f3d9ad84740 1 -- 172.21.15.86:0/4119319715 shutdown_connections 2024-04-17T07:25:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.511+0000 7f3d9ad84740 1 -- 172.21.15.86:0/4119319715 wait complete. 2024-04-17T07:25:35.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.511+0000 7f3d9ad84740 20 client.27190 trim_cache size 0 max 16384 2024-04-17T07:25:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:35 smithi103 ceph-mon[35495]: pgmap v776: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 648 op/s 2024-04-17T07:25:35.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.536+0000 7f3d9ad84740 1 Processor -- start 2024-04-17T07:25:35.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.536+0000 7f3d9ad84740 1 -- start start 2024-04-17T07:25:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.537+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 0x558c10a5e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.537+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a65b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.537+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a59b00 0x558c10a5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.537+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901b9a50 con 0x558c10a53fa0 2024-04-17T07:25:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.537+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901bb590 con 0x558c10a59b00 2024-04-17T07:25:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.537+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901ba120 con 0x558c10a54990 2024-04-17T07:25:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.537+0000 7f3d94c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a65b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.538+0000 7f3d95600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 0x558c10a5e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.538+0000 7f3d94c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a65b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36256/0 (socket says 172.21.15.86:36256) 2024-04-17T07:25:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.538+0000 7f3d94c00640 1 -- 172.21.15.86:0/306762923 learned_addr learned my addr 172.21.15.86:0/306762923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.538+0000 7f3d96000640 1 --2- 172.21.15.86:0/306762923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a59b00 0x558c10a5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.538+0000 7f3d94c00640 1 -- 172.21.15.86:0/306762923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a59b00 msgr2=0x558c10a5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.538+0000 7f3d94c00640 1 --2- 172.21.15.86:0/306762923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a59b00 0x558c10a5eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.538+0000 7f3d94c00640 1 -- 172.21.15.86:0/306762923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 msgr2=0x558c10a5e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.539+0000 7f3d94c00640 1 --2- 172.21.15.86:0/306762923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 0x558c10a5e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.539+0000 7f3d94c00640 1 -- 172.21.15.86:0/306762923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c106cc3f0 con 0x558c10a54990 2024-04-17T07:25:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.539+0000 7f3d95600640 1 --2- 172.21.15.86:0/306762923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 0x558c10a5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.539+0000 7f3d94c00640 1 --2- 172.21.15.86:0/306762923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a65b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009860 tx=0x7f3d7c03e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57c80b69bdabeb48 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.539+0000 7f3d81c00640 1 -- 172.21.15.86:0/306762923 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c039040 con 0x558c10a54990 2024-04-17T07:25:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.539+0000 7f3d81c00640 1 -- 172.21.15.86:0/306762923 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d7c02c530 con 0x558c10a54990 2024-04-17T07:25:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.539+0000 7f3d81c00640 1 -- 172.21.15.86:0/306762923 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c004920 con 0x558c10a54990 2024-04-17T07:25:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.540+0000 7f3d9ad84740 1 -- 172.21.15.86:0/306762923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 msgr2=0x558c10a65b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.540+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/306762923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a65b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009860 tx=0x7f3d7c03e000 comp rx=0 tx=0).stop 2024-04-17T07:25:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.541+0000 7f3d9ad84740 1 -- 172.21.15.86:0/306762923 shutdown_connections 2024-04-17T07:25:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.541+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/306762923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a59b00 0x558c10a5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.541+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/306762923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a65b00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.541+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/306762923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 0x558c10a5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.541+0000 7f3d9ad84740 1 -- 172.21.15.86:0/306762923 >> 172.21.15.86:0/306762923 conn(0x558c10a4f510 msgr2=0x558c10a56040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.541+0000 7f3d9ad84740 1 -- 172.21.15.86:0/306762923 shutdown_connections 2024-04-17T07:25:35.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.542+0000 7f3d9ad84740 1 -- 172.21.15.86:0/306762923 wait complete. 2024-04-17T07:25:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.543+0000 7f3d9ad84740 1 -- start start 2024-04-17T07:25:35.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.543+0000 7f3d9ad84740 1 Processor -- start 2024-04-17T07:25:35.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.547+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 0x558c10a60830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.547+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a60d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.547+0000 7f3d9ad84740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a59b00 0x558c10adf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.547+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901baec0 con 0x558c10a53fa0 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.547+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901ba7f0 con 0x558c10a59b00 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.547+0000 7f3d9ad84740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901b7a20 con 0x558c10a54990 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.547+0000 7f3d9ad84740 10 client.0 _ll_register_callbacks cb 0x7f3d8800f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.547+0000 7f3d94c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a60d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.547+0000 7f3d94c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a60d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36272/0 (socket says 172.21.15.86:36272) 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.547+0000 7f3d94c00640 1 -- 172.21.15.86:0/1583282228 learned_addr learned my addr 172.21.15.86:0/1583282228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.548+0000 7f3d96000640 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a59b00 0x558c10adf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.548+0000 7f3d95600640 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 0x558c10a60830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.548+0000 7f3d94c00640 1 -- 172.21.15.86:0/1583282228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a59b00 msgr2=0x558c10adf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.548+0000 7f3d94c00640 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a59b00 0x558c10adf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.548+0000 7f3d94c00640 1 -- 172.21.15.86:0/1583282228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 msgr2=0x558c10a60830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.548+0000 7f3d94c00640 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 0x558c10a60830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.548+0000 7f3d94c00640 1 -- 172.21.15.86:0/1583282228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c107967d0 con 0x558c10a54990 2024-04-17T07:25:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.549+0000 7f3d94c00640 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a60d70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c000c00 tx=0x7f3d7c034730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c35a4541ac0dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.549+0000 7f3d76a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.550+0000 7f3d76a00640 1 -- 172.21.15.86:0/1583282228 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c039050 con 0x558c10a54990 2024-04-17T07:25:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.550+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1583282228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d7c02c530 con 0x558c10a54990 2024-04-17T07:25:35.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.550+0000 7f3d76a00640 1 -- 172.21.15.86:0/1583282228 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3d7c02c8e0 con 0x558c10a54990 2024-04-17T07:25:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.551+0000 7f3d76a00640 1 -- 172.21.15.86:0/1583282228 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3d7c067050 con 0x558c10a54990 2024-04-17T07:25:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.551+0000 7f3d9ad84740 10 client.27196 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.551+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1583282228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c10ae0ac0 con 0x558c10a54990 2024-04-17T07:25:35.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.552+0000 7f3d76a00640 1 -- 172.21.15.86:0/1583282228 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d7c02c530 con 0x558c10a54990 2024-04-17T07:25:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.552+0000 7f3d76a00640 1 -- 172.21.15.86:0/1583282228 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3d7c055640 con 0x558c10a54990 2024-04-17T07:25:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.552+0000 7f3d9ad84740 10 client.27196 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.552+0000 7f3d9ad84740 10 client.27196 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:25:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.552+0000 7f3d9ad84740 20 client.27196 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.552+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558c10ae1480 0x558c10ae38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.552+0000 7f3d9ad84740 4 client.27196 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:25:35.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.552+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1583282228 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c109c18b0 con 0x558c10ae1480 2024-04-17T07:25:35.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.553+0000 7f3d95600640 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558c10ae1480 0x558c10ae38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.557+0000 7f3d95600640 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558c10ae1480 0x558c10ae38a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f188009069a40cc server_cookie=2583a5f535fb0199 in_seq=0 out_seq=0 2024-04-17T07:25:35.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.557+0000 7f3d76a00640 10 client.27196 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:25:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.558+0000 7f3d76a00640 1 -- 172.21.15.86:0/1583282228 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c109c18b0 con 0x558c10ae1480 2024-04-17T07:25:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.558+0000 7f3d76a00640 10 client.27196 handle_command_reply: tid=0 2024-04-17T07:25:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.558+0000 7f3d9ad84740 1 client.27196 shutdown 2024-04-17T07:25:35.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.558+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1583282228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 msgr2=0x558c10a60d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.558+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a60d70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c000c00 tx=0x7f3d7c034730 comp rx=0 tx=0).stop 2024-04-17T07:25:35.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.559+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1583282228 shutdown_connections 2024-04-17T07:25:35.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.559+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x558c10ae1480 0x558c10ae38a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.559+0000 7f3d76a00640 0 client.27196 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:25:35.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.559+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558c10a59b00 0x558c10adf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.559+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558c10a54990 0x558c10a60d70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.559+0000 7f3d9ad84740 1 --2- 172.21.15.86:0/1583282228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558c10a53fa0 0x558c10a60830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.559+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1583282228 >> 172.21.15.86:0/1583282228 conn(0x558c10a4f510 msgr2=0x558c10a5e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.559+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1583282228 shutdown_connections 2024-04-17T07:25:35.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.559+0000 7f3d9ad84740 1 -- 172.21.15.86:0/1583282228 wait complete. 2024-04-17T07:25:35.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.560+0000 7f3d9ad84740 20 client.27196 trim_cache size 0 max 16384 2024-04-17T07:25:35.560 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:25:35.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.561+0000 7f3d98000640 1 -- 172.21.15.86:0/1496253705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3d780a3d30 msgr2=0x7f3d780a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.561+0000 7f3d98000640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3d780a3d30 0x7f3d780a61f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009830 tx=0x7f3d7c034980 comp rx=0 tx=0).stop 2024-04-17T07:25:35.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.561+0000 7f3d98000640 1 -- 172.21.15.86:0/1496253705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108f60 msgr2=0x7f3d901a7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.561+0000 7f3d98000640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108f60 0x7f3d901a7120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400c890 tx=0x7f3d8400cd50 comp rx=0 tx=0).stop 2024-04-17T07:25:35.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.562+0000 7f3d98000640 1 -- 172.21.15.86:0/1496253705 shutdown_connections 2024-04-17T07:25:35.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.562+0000 7f3d98000640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3d780a3d30 0x7f3d780a61f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.562+0000 7f3d98000640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d9010e2a0 0x7f3d901ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.562+0000 7f3d98000640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9010b8d0 0x7f3d901a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.562+0000 7f3d98000640 1 --2- 172.21.15.86:0/1496253705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d90108f60 0x7f3d901a7120 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.562+0000 7f3d98000640 1 -- 172.21.15.86:0/1496253705 >> 172.21.15.86:0/1496253705 conn(0x7f3d9006edf0 msgr2=0x7f3d90109380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.562+0000 7f3d98000640 1 -- 172.21.15.86:0/1496253705 shutdown_connections 2024-04-17T07:25:35.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.563+0000 7f3d98000640 1 -- 172.21.15.86:0/1496253705 wait complete. 2024-04-17T07:25:35.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:35 smithi086 ceph-mon[31733]: pgmap v776: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 648 op/s 2024-04-17T07:25:35.596 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:25:35.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.917+0000 7ff749600640 1 -- 172.21.15.86:0/1228807456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7441107e0 msgr2=0x7ff744112bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.917+0000 7ff749600640 1 --2- 172.21.15.86:0/1228807456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7441107e0 0x7ff744112bd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009ca0 tx=0x7ff73c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.917+0000 7ff749600640 1 -- 172.21.15.86:0/1228807456 shutdown_connections 2024-04-17T07:25:35.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.917+0000 7ff749600640 1 --2- 172.21.15.86:0/1228807456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7441107e0 0x7ff744112bd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.917+0000 7ff749600640 1 --2- 172.21.15.86:0/1228807456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7441098c0 0x7ff7441102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.917+0000 7ff749600640 1 --2- 172.21.15.86:0/1228807456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff744108f60 0x7ff744109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.917+0000 7ff749600640 1 -- 172.21.15.86:0/1228807456 >> 172.21.15.86:0/1228807456 conn(0x7ff74406edf0 msgr2=0x7ff744071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.918+0000 7ff749600640 1 -- 172.21.15.86:0/1228807456 shutdown_connections 2024-04-17T07:25:35.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.918+0000 7ff749600640 1 -- 172.21.15.86:0/1228807456 wait complete. 2024-04-17T07:25:35.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.919+0000 7ff749600640 1 Processor -- start 2024-04-17T07:25:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.919+0000 7ff749600640 1 -- start start 2024-04-17T07:25:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.920+0000 7ff749600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff744108f60 0x7ff74410bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.920+0000 7ff749600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7441098c0 0x7ff74410a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.920+0000 7ff749600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7441107e0 0x7ff74410a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.921+0000 7ff749600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff744116310 con 0x7ff7441098c0 2024-04-17T07:25:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.921+0000 7ff742a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff744108f60 0x7ff74410bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.921+0000 7ff742a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff744108f60 0x7ff74410bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50802/0 (socket says 172.21.15.86:50802) 2024-04-17T07:25:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.921+0000 7ff742a00640 1 -- 172.21.15.86:0/799850232 learned_addr learned my addr 172.21.15.86:0/799850232 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.921+0000 7ff742000640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7441098c0 0x7ff74410a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.921+0000 7ff749600640 1 -- 172.21.15.86:0/799850232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff744116190 con 0x7ff744108f60 2024-04-17T07:25:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.921+0000 7ff749600640 1 -- 172.21.15.86:0/799850232 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff744116490 con 0x7ff7441107e0 2024-04-17T07:25:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.922+0000 7ff743400640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7441107e0 0x7ff74410a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.922+0000 7ff742000640 1 -- 172.21.15.86:0/799850232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7441107e0 msgr2=0x7ff74410a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.922+0000 7ff742000640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7441107e0 0x7ff74410a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.922+0000 7ff742000640 1 -- 172.21.15.86:0/799850232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff744108f60 msgr2=0x7ff74410bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.922+0000 7ff742000640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff744108f60 0x7ff74410bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.922+0000 7ff742000640 1 -- 172.21.15.86:0/799850232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74410acb0 con 0x7ff7441098c0 2024-04-17T07:25:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.922+0000 7ff743400640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7441107e0 0x7ff74410a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.922+0000 7ff742000640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7441098c0 0x7ff74410a230 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff73000c940 tx=0x7ff73000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.923+0000 7ff73b400640 1 -- 172.21.15.86:0/799850232 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730004220 con 0x7ff7441098c0 2024-04-17T07:25:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.923+0000 7ff73b400640 1 -- 172.21.15.86:0/799850232 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7300043c0 con 0x7ff7441098c0 2024-04-17T07:25:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.923+0000 7ff73b400640 1 -- 172.21.15.86:0/799850232 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730003e00 con 0x7ff7441098c0 2024-04-17T07:25:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.923+0000 7ff749600640 1 -- 172.21.15.86:0/799850232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74410afa0 con 0x7ff7441098c0 2024-04-17T07:25:35.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.924+0000 7ff749600640 1 -- 172.21.15.86:0/799850232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7441af4d0 con 0x7ff7441098c0 2024-04-17T07:25:35.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.927+0000 7ff74c373740 1 Processor -- start 2024-04-17T07:25:35.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.927+0000 7ff74c373740 1 -- start start 2024-04-17T07:25:35.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.928+0000 7ff74c373740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c7574c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.928+0000 7ff74c373740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c75751490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.928+0000 7ff74c373740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c75753f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.928+0000 7ff74c373740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555c756b2020 con 0x555c7574c990 2024-04-17T07:25:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.928+0000 7ff74c373740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c75658460 con 0x555c75751b00 2024-04-17T07:25:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.928+0000 7ff74c373740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c756adfe0 con 0x555c7574bfa0 2024-04-17T07:25:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.929+0000 7ff742a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c75751490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.929+0000 7ff742a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c75751490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49586/0 (socket says 172.21.15.86:49586) 2024-04-17T07:25:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.929+0000 7ff742a00640 1 -- 172.21.15.86:0/1354873756 learned_addr learned my addr 172.21.15.86:0/1354873756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.929+0000 7ff73b400640 1 -- 172.21.15.86:0/799850232 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff7300079b0 con 0x7ff7441098c0 2024-04-17T07:25:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.930+0000 7ff742a00640 1 -- 172.21.15.86:0/1354873756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 msgr2=0x555c7574c3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:25:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.930+0000 7ff742a00640 1 --2- 172.21.15.86:0/1354873756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c7574c3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.930+0000 7ff742a00640 1 -- 172.21.15.86:0/1354873756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 msgr2=0x555c75753f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.930+0000 7ff742000640 1 --2- 172.21.15.86:0/1354873756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c75753f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.930+0000 7ff742a00640 1 --2- 172.21.15.86:0/1354873756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c75753f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.930+0000 7ff742a00640 1 -- 172.21.15.86:0/1354873756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c75660040 con 0x555c7574c990 2024-04-17T07:25:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.930+0000 7ff742000640 1 --2- 172.21.15.86:0/1354873756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c75753f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:25:35.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.931+0000 7ff742a00640 1 --2- 172.21.15.86:0/1354873756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c75751490 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ff734009d30 tx=0x7ff7340117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ce3e368d774a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.931+0000 7ff73b400640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7280a3d30 0x7ff7280a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.932+0000 7ff743400640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7280a3d30 0x7ff7280a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.932+0000 7ff73b400640 1 -- 172.21.15.86:0/799850232 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff7300d0460 con 0x7ff7441098c0 2024-04-17T07:25:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.932+0000 7ff72f400640 1 -- 172.21.15.86:0/1354873756 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734012650 con 0x555c7574c990 2024-04-17T07:25:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.932+0000 7ff72f400640 1 -- 172.21.15.86:0/1354873756 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff734009e20 con 0x555c7574c990 2024-04-17T07:25:35.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.932+0000 7ff72f400640 1 -- 172.21.15.86:0/1354873756 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73400fc30 con 0x555c7574c990 2024-04-17T07:25:35.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.933+0000 7ff74c373740 1 -- 172.21.15.86:0/1354873756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 msgr2=0x555c75751490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.933+0000 7ff743400640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7280a3d30 0x7ff7280a61f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009bf0 tx=0x7ff73c032510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.933+0000 7ff74c373740 1 --2- 172.21.15.86:0/1354873756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c75751490 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ff734009d30 tx=0x7ff7340117b0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.934+0000 7ff74c373740 1 -- 172.21.15.86:0/1354873756 shutdown_connections 2024-04-17T07:25:35.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.934+0000 7ff74c373740 1 --2- 172.21.15.86:0/1354873756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c75753f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.934+0000 7ff74c373740 1 --2- 172.21.15.86:0/1354873756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c75751490 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.934+0000 7ff74c373740 1 --2- 172.21.15.86:0/1354873756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c7574c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.934+0000 7ff74c373740 1 -- 172.21.15.86:0/1354873756 >> 172.21.15.86:0/1354873756 conn(0x555c75747510 msgr2=0x555c75747910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.934+0000 7ff74c373740 1 -- 172.21.15.86:0/1354873756 shutdown_connections 2024-04-17T07:25:35.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.934+0000 7ff74c373740 1 -- 172.21.15.86:0/1354873756 wait complete. 2024-04-17T07:25:35.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.935+0000 7ff74c373740 1 -- start start 2024-04-17T07:25:35.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.935+0000 7ff74c373740 1 Processor -- start 2024-04-17T07:25:35.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff74c373740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c75756510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff74c373740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c75756a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff74c373740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c7575da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff74c373740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555c75754ab0 con 0x555c7574c990 2024-04-17T07:25:35.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff74c373740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c7566b830 con 0x555c75751b00 2024-04-17T07:25:35.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff74c373740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c75754c30 con 0x555c7574bfa0 2024-04-17T07:25:35.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff742a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c75756510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff742000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c75756a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff742a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c75756510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36312/0 (socket says 172.21.15.86:36312) 2024-04-17T07:25:35.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff742a00640 1 -- 172.21.15.86:0/815875460 learned_addr learned my addr 172.21.15.86:0/815875460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.937+0000 7ff743400640 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c7575da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.938+0000 7ff74c373740 10 client.0 _ll_register_callbacks cb 0x555c75639fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:35.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.938+0000 7ff743400640 1 -- 172.21.15.86:0/815875460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 msgr2=0x555c75756510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.938+0000 7ff743400640 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c75756510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.938+0000 7ff743400640 1 -- 172.21.15.86:0/815875460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 msgr2=0x555c75756a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.938+0000 7ff743400640 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c75756a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.938+0000 7ff743400640 1 -- 172.21.15.86:0/815875460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c7566f920 con 0x555c75751b00 2024-04-17T07:25:35.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.938+0000 7ff742a00640 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c75756510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:25:35.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.939+0000 7ff743400640 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c7575da90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff73c032420 tx=0x7ff73c02c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c087522ee642a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.940+0000 7ff723e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:25:35.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.940+0000 7ff723e00640 1 -- 172.21.15.86:0/815875460 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c050060 con 0x555c75751b00 2024-04-17T07:25:35.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.940+0000 7ff723e00640 1 -- 172.21.15.86:0/815875460 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff73c004180 con 0x555c75751b00 2024-04-17T07:25:35.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.941+0000 7ff74c373740 1 -- 172.21.15.86:0/815875460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff734009e20 con 0x555c75751b00 2024-04-17T07:25:35.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.941+0000 7ff723e00640 1 -- 172.21.15.86:0/815875460 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff73c039040 con 0x555c75751b00 2024-04-17T07:25:35.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.941+0000 7ff74c373740 10 client.37121 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:35.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.941+0000 7ff74c373740 1 -- 172.21.15.86:0/815875460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c7563d7f0 con 0x555c75751b00 2024-04-17T07:25:35.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.942+0000 7ff723e00640 1 -- 172.21.15.86:0/815875460 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff734009e20 con 0x555c75751b00 2024-04-17T07:25:35.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.942+0000 7ff723e00640 1 -- 172.21.15.86:0/815875460 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff73c03fa30 con 0x555c75751b00 2024-04-17T07:25:35.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.943+0000 7ff74c373740 10 client.37121 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:35.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.943+0000 7ff74c373740 10 client.37121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:35.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.943+0000 7ff74c373740 20 client.37121 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:35.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.943+0000 7ff74c373740 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555c75761e70 0x555c75764310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.943+0000 7ff74c373740 4 client.37121 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:35.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.943+0000 7ff74c373740 1 -- 172.21.15.86:0/815875460 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c756bfa40 con 0x555c75761e70 2024-04-17T07:25:35.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.944+0000 7ff742a00640 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555c75761e70 0x555c75764310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.944+0000 7ff742a00640 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555c75761e70 0x555c75764310 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c81ccc7994f8a560 server_cookie=266331b7db1df037 in_seq=0 out_seq=0 2024-04-17T07:25:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.945+0000 7ff723e00640 10 client.37121 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.946+0000 7ff723e00640 1 -- 172.21.15.86:0/815875460 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c756bfa40 con 0x555c75761e70 2024-04-17T07:25:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.946+0000 7ff723e00640 10 client.37121 handle_command_reply: tid=0 2024-04-17T07:25:35.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.946+0000 7ff74c373740 1 client.37121 shutdown 2024-04-17T07:25:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.947+0000 7ff74c373740 1 -- 172.21.15.86:0/815875460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 msgr2=0x555c7575da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.947+0000 7ff74c373740 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c7575da90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff73c032420 tx=0x7ff73c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.947+0000 7ff74c373740 1 -- 172.21.15.86:0/815875460 shutdown_connections 2024-04-17T07:25:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.947+0000 7ff74c373740 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555c75761e70 0x555c75764310 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.947+0000 7ff74c373740 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c7575da90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.947+0000 7ff74c373740 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c75756a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.947+0000 7ff74c373740 1 --2- 172.21.15.86:0/815875460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c75756510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.947+0000 7ff74c373740 1 -- 172.21.15.86:0/815875460 >> 172.21.15.86:0/815875460 conn(0x555c75747510 msgr2=0x555c75746710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.947+0000 7ff723e00640 0 client.37121 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:35.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.947+0000 7ff74c373740 1 -- 172.21.15.86:0/815875460 shutdown_connections 2024-04-17T07:25:35.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.948+0000 7ff74c373740 1 -- 172.21.15.86:0/815875460 wait complete. 2024-04-17T07:25:35.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.948+0000 7ff74c373740 20 client.37121 trim_cache size 0 max 16384 2024-04-17T07:25:35.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.975+0000 7ff74c373740 1 Processor -- start 2024-04-17T07:25:35.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.975+0000 7ff74c373740 1 -- start start 2024-04-17T07:25:35.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.976+0000 7ff74c373740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c75756510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.976+0000 7ff74c373740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7574c990 0x555c75756a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.976+0000 7ff74c373740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c75751b00 0x555c7575da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.976+0000 7ff74c373740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441b9a10 con 0x555c75751b00 2024-04-17T07:25:35.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.976+0000 7ff74c373740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441bb550 con 0x555c7574c990 2024-04-17T07:25:35.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.976+0000 7ff74c373740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441ba0e0 con 0x555c7574bfa0 2024-04-17T07:25:35.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.976+0000 7ff743400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c75751b00 0x555c7575da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.977+0000 7ff743400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c75751b00 0x555c7575da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49602/0 (socket says 172.21.15.86:49602) 2024-04-17T07:25:35.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.977+0000 7ff742a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c75756510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.977+0000 7ff743400640 1 -- 172.21.15.86:0/3117626 learned_addr learned my addr 172.21.15.86:0/3117626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.977+0000 7ff742000640 1 --2- 172.21.15.86:0/3117626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7574c990 0x555c75756a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.977+0000 7ff743400640 1 -- 172.21.15.86:0/3117626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 msgr2=0x555c75756510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.977+0000 7ff743400640 1 --2- 172.21.15.86:0/3117626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c75756510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.977+0000 7ff743400640 1 -- 172.21.15.86:0/3117626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7574c990 msgr2=0x555c75756a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.977+0000 7ff743400640 1 --2- 172.21.15.86:0/3117626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7574c990 0x555c75756a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.977+0000 7ff743400640 1 -- 172.21.15.86:0/3117626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c753c43f0 con 0x555c75751b00 2024-04-17T07:25:35.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.977+0000 7ff742000640 1 --2- 172.21.15.86:0/3117626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7574c990 0x555c75756a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:35.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.978+0000 7ff743400640 1 --2- 172.21.15.86:0/3117626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c75751b00 0x555c7575da50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff73c03fe50 tx=0x7ff73c02c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1315e4bb386065ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.978+0000 7ff72ea00640 1 -- 172.21.15.86:0/3117626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c050070 con 0x555c75751b00 2024-04-17T07:25:35.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.978+0000 7ff72ea00640 1 -- 172.21.15.86:0/3117626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff73c047040 con 0x555c75751b00 2024-04-17T07:25:35.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.978+0000 7ff72ea00640 1 -- 172.21.15.86:0/3117626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c05a080 con 0x555c75751b00 2024-04-17T07:25:35.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.979+0000 7ff74c373740 1 -- 172.21.15.86:0/3117626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c75751b00 msgr2=0x555c7575da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.979+0000 7ff74c373740 1 --2- 172.21.15.86:0/3117626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c75751b00 0x555c7575da50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff73c03fe50 tx=0x7ff73c02c7f0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.980+0000 7ff74c373740 1 -- 172.21.15.86:0/3117626 shutdown_connections 2024-04-17T07:25:35.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.980+0000 7ff74c373740 1 --2- 172.21.15.86:0/3117626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c75751b00 0x555c7575da50 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.980+0000 7ff74c373740 1 --2- 172.21.15.86:0/3117626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7574c990 0x555c75756a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.980+0000 7ff74c373740 1 --2- 172.21.15.86:0/3117626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c75756510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.980+0000 7ff74c373740 1 -- 172.21.15.86:0/3117626 >> 172.21.15.86:0/3117626 conn(0x555c75747510 msgr2=0x555c75763e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.980+0000 7ff74c373740 1 -- 172.21.15.86:0/3117626 shutdown_connections 2024-04-17T07:25:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.981+0000 7ff74c373740 1 -- 172.21.15.86:0/3117626 wait complete. 2024-04-17T07:25:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.981+0000 7ff74c373740 1 -- start start 2024-04-17T07:25:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.981+0000 7ff74c373740 1 Processor -- start 2024-04-17T07:25:35.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.982+0000 7ff74c373740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c7575d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.983+0000 7ff74c373740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c7575a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.983+0000 7ff742a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c7575d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.983+0000 7ff742000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c7575a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.983+0000 7ff742a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c7575d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:36330/0 (socket says 172.21.15.86:36330) 2024-04-17T07:25:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.983+0000 7ff742a00640 1 -- 172.21.15.86:0/608698281 learned_addr learned my addr 172.21.15.86:0/608698281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.983+0000 7ff74c373740 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c7575a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.983+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441bae80 con 0x555c7574c990 2024-04-17T07:25:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.983+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441ba7b0 con 0x555c75751b00 2024-04-17T07:25:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.984+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7441b79e0 con 0x555c7574bfa0 2024-04-17T07:25:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.984+0000 7ff74c373740 10 client.0 _ll_register_callbacks cb 0x7ff73c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.984+0000 7ff743400640 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c7575a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.984+0000 7ff742000640 1 -- 172.21.15.86:0/608698281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 msgr2=0x555c7575d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.984+0000 7ff742000640 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c7575d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.984+0000 7ff742000640 1 -- 172.21.15.86:0/608698281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 msgr2=0x555c7575a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.984+0000 7ff742a00640 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c7575d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.984+0000 7ff742000640 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c7575a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.984+0000 7ff742000640 1 -- 172.21.15.86:0/608698281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c7548e7d0 con 0x555c7574c990 2024-04-17T07:25:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.985+0000 7ff742000640 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c7575a2c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff7300cdc30 tx=0x7ff73009bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac82153b6e7a53d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.985+0000 7ff723e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:25:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.985+0000 7ff723e00640 1 -- 172.21.15.86:0/608698281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300b0e60 con 0x555c7574c990 2024-04-17T07:25:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.986+0000 7ff723e00640 1 -- 172.21.15.86:0/608698281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff7300d0190 con 0x555c7574c990 2024-04-17T07:25:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.986+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff73c047040 con 0x555c7574c990 2024-04-17T07:25:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.986+0000 7ff723e00640 1 -- 172.21.15.86:0/608698281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff73001d030 con 0x555c7574c990 2024-04-17T07:25:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.987+0000 7ff74c373740 10 client.18249 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.987+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c757c7460 con 0x555c7574c990 2024-04-17T07:25:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.987+0000 7ff723e00640 1 -- 172.21.15.86:0/608698281 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff73c047040 con 0x555c7574c990 2024-04-17T07:25:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.987+0000 7ff723e00640 1 -- 172.21.15.86:0/608698281 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff73009b810 con 0x555c7574c990 2024-04-17T07:25:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.988+0000 7ff74c373740 10 client.18249 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.988+0000 7ff74c373740 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.988+0000 7ff74c373740 20 client.18249 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.988+0000 7ff74c373740 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555c757c7e10 0x555c757ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.988+0000 7ff74c373740 4 client.18249 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:25:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.988+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c756b98b0 con 0x555c757c7e10 2024-04-17T07:25:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.989+0000 7ff742a00640 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555c757c7e10 0x555c757ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:35.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.990+0000 7ff742a00640 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555c757c7e10 0x555c757ca230 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17c890aa823953 server_cookie=76af1bb3e9cce004 in_seq=0 out_seq=0 2024-04-17T07:25:35.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.990+0000 7ff723e00640 10 client.18249 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:35.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.990+0000 7ff723e00640 1 -- 172.21.15.86:0/608698281 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c756b98b0 con 0x555c757c7e10 2024-04-17T07:25:35.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.990+0000 7ff723e00640 10 client.18249 handle_command_reply: tid=0 2024-04-17T07:25:35.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.990+0000 7ff74c373740 1 client.18249 shutdown 2024-04-17T07:25:35.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 msgr2=0x555c7575a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff74c373740 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c7575a2c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff7300cdc30 tx=0x7ff73009bd20 comp rx=0 tx=0).stop 2024-04-17T07:25:35.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 shutdown_connections 2024-04-17T07:25:35.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff74c373740 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555c757c7e10 0x555c757ca230 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff723e00640 0 client.18249 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:35.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff74c373740 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c75751b00 0x555c7575a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff74c373740 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555c7574c990 0x555c7575a2c0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff74c373740 1 --2- 172.21.15.86:0/608698281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c7574bfa0 0x555c7575d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 >> 172.21.15.86:0/608698281 conn(0x555c75747510 msgr2=0x555c757564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 shutdown_connections 2024-04-17T07:25:35.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.991+0000 7ff74c373740 1 -- 172.21.15.86:0/608698281 wait complete. 2024-04-17T07:25:35.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.992+0000 7ff74c373740 20 client.18249 trim_cache size 0 max 16384 2024-04-17T07:25:35.992 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:25:35.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.992+0000 7ff749600640 1 -- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7280a3d30 msgr2=0x7ff7280a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.992+0000 7ff749600640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7280a3d30 0x7ff7280a61f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009bf0 tx=0x7ff73c032510 comp rx=0 tx=0).stop 2024-04-17T07:25:35.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.992+0000 7ff749600640 1 -- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7441098c0 msgr2=0x7ff74410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:35.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.992+0000 7ff749600640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7441098c0 0x7ff74410a230 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff73000c940 tx=0x7ff73000ce00 comp rx=0 tx=0).stop 2024-04-17T07:25:35.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.993+0000 7ff749600640 1 -- 172.21.15.86:0/799850232 shutdown_connections 2024-04-17T07:25:35.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.993+0000 7ff749600640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff7280a3d30 0x7ff7280a61f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.993+0000 7ff749600640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7441107e0 0x7ff74410a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.993+0000 7ff749600640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7441098c0 0x7ff74410a230 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.993+0000 7ff749600640 1 --2- 172.21.15.86:0/799850232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff744108f60 0x7ff74410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:35.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.993+0000 7ff749600640 1 -- 172.21.15.86:0/799850232 >> 172.21.15.86:0/799850232 conn(0x7ff74406edf0 msgr2=0x7ff744070b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:35.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.993+0000 7ff749600640 1 -- 172.21.15.86:0/799850232 shutdown_connections 2024-04-17T07:25:35.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:35.993+0000 7ff749600640 1 -- 172.21.15.86:0/799850232 wait complete. 2024-04-17T07:25:37.018 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:25:37.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:25:37.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:25:37.333 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:37 smithi086 ceph-mon[31733]: pgmap v777: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.2 MiB/s wr, 582 op/s 2024-04-17T07:25:37.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.352+0000 7f45e7800640 1 -- 172.21.15.86:0/1692344121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e0100620 msgr2=0x7f45e0104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.352+0000 7f45e7800640 1 --2- 172.21.15.86:0/1692344121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e0100620 0x7f45e0104240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009ca0 tx=0x7f45d802c100 comp rx=0 tx=0).stop 2024-04-17T07:25:37.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.354+0000 7f45e7800640 1 -- 172.21.15.86:0/1692344121 shutdown_connections 2024-04-17T07:25:37.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.354+0000 7f45e7800640 1 --2- 172.21.15.86:0/1692344121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e0100620 0x7f45e0104240 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.354+0000 7f45e7800640 1 --2- 172.21.15.86:0/1692344121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45e00ffc60 0x7f45e01000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.355+0000 7f45e7800640 1 --2- 172.21.15.86:0/1692344121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e0101190 0x7f45e01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.355+0000 7f45e7800640 1 -- 172.21.15.86:0/1692344121 >> 172.21.15.86:0/1692344121 conn(0x7f45e00fb760 msgr2=0x7f45e00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.356+0000 7f45e7800640 1 -- 172.21.15.86:0/1692344121 shutdown_connections 2024-04-17T07:25:37.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.356+0000 7f45e7800640 1 -- 172.21.15.86:0/1692344121 wait complete. 2024-04-17T07:25:37.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.357+0000 7f45e7800640 1 Processor -- start 2024-04-17T07:25:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.358+0000 7f45e7800640 1 -- start start 2024-04-17T07:25:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.358+0000 7f45e7800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e00ffc60 0x7f45e019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.359+0000 7f45e7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e0100620 0x7f45e019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.359+0000 7f45e7800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45e0101190 0x7f45e01a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.359+0000 7f45e4e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e00ffc60 0x7f45e019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.360+0000 7f45e7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45e01078b0 con 0x7f45e0100620 2024-04-17T07:25:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.360+0000 7f45e7800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0107730 con 0x7f45e00ffc60 2024-04-17T07:25:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.360+0000 7f45e7800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0114370 con 0x7f45e0101190 2024-04-17T07:25:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.360+0000 7f45e4e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e00ffc60 0x7f45e019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:35718/0 (socket says 172.21.15.86:35718) 2024-04-17T07:25:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.360+0000 7f45e4e00640 1 -- 172.21.15.86:0/2212684176 learned_addr learned my addr 172.21.15.86:0/2212684176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.360+0000 7f45dfe00640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e0100620 0x7f45e019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.360+0000 7f45e5800640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45e0101190 0x7f45e01a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.361+0000 7f45e4e00640 1 -- 172.21.15.86:0/2212684176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45e0101190 msgr2=0x7f45e01a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.361+0000 7f45e4e00640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45e0101190 0x7f45e01a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.361+0000 7f45e4e00640 1 -- 172.21.15.86:0/2212684176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e0100620 msgr2=0x7f45e019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.361+0000 7f45e4e00640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e0100620 0x7f45e019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.361+0000 7f45e4e00640 1 -- 172.21.15.86:0/2212684176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e01a6360 con 0x7f45e00ffc60 2024-04-17T07:25:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.362+0000 7f45e4e00640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e00ffc60 0x7f45e019e740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f45d000ea60 tx=0x7f45d000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.362+0000 7f45dd600640 1 -- 172.21.15.86:0/2212684176 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d000cde0 con 0x7f45e00ffc60 2024-04-17T07:25:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.362+0000 7f45e7800640 1 -- 172.21.15.86:0/2212684176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e01a6650 con 0x7f45e00ffc60 2024-04-17T07:25:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.362+0000 7f45e7800640 1 -- 172.21.15.86:0/2212684176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e01a6bf0 con 0x7f45e00ffc60 2024-04-17T07:25:37.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.363+0000 7f45dd600640 1 -- 172.21.15.86:0/2212684176 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45d0009470 con 0x7f45e00ffc60 2024-04-17T07:25:37.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.363+0000 7f45dd600640 1 -- 172.21.15.86:0/2212684176 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d000cde0 con 0x7f45e00ffc60 2024-04-17T07:25:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.367+0000 7f45ea47e740 1 Processor -- start 2024-04-17T07:25:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.367+0000 7f45ea47e740 1 -- start start 2024-04-17T07:25:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.368+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a90ec0 0x55f0a3a912e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.368+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a918b0 0x55f0a3a963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.368+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a96a20 0x55f0a3a98eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.368+0000 7f45ea47e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f0a39f6cb0 con 0x55f0a3a90ec0 2024-04-17T07:25:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.368+0000 7f45ea47e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f0a399d370 con 0x55f0a3a96a20 2024-04-17T07:25:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.368+0000 7f45ea47e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f0a39f2c70 con 0x55f0a3a918b0 2024-04-17T07:25:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.369+0000 7f45e5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a90ec0 0x55f0a3a912e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.369+0000 7f45e5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a90ec0 0x55f0a3a912e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52448/0 (socket says 172.21.15.86:52448) 2024-04-17T07:25:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.369+0000 7f45e5800640 1 -- 172.21.15.86:0/3928779377 learned_addr learned my addr 172.21.15.86:0/3928779377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.370+0000 7f45dd600640 1 -- 172.21.15.86:0/2212684176 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f45d0004020 con 0x7f45e00ffc60 2024-04-17T07:25:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.370+0000 7f45e5800640 1 -- 172.21.15.86:0/3928779377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a918b0 msgr2=0x55f0a3a963b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:25:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.370+0000 7f45e5800640 1 --2- 172.21.15.86:0/3928779377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a918b0 0x55f0a3a963b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.370+0000 7f45e5800640 1 -- 172.21.15.86:0/3928779377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a96a20 msgr2=0x55f0a3a98eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:25:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.370+0000 7f45e5800640 1 --2- 172.21.15.86:0/3928779377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a96a20 0x55f0a3a98eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.370+0000 7f45e5800640 1 -- 172.21.15.86:0/3928779377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a39bc910 con 0x55f0a3a90ec0 2024-04-17T07:25:37.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.371+0000 7f45e5800640 1 --2- 172.21.15.86:0/3928779377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a90ec0 0x55f0a3a912e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f45d802c640 tx=0x7f45d8005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b009447c20bc019a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.372+0000 7f45ce000640 1 -- 172.21.15.86:0/3928779377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8043070 con 0x55f0a3a90ec0 2024-04-17T07:25:37.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.372+0000 7f45ce000640 1 -- 172.21.15.86:0/3928779377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45d8003e20 con 0x55f0a3a90ec0 2024-04-17T07:25:37.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.372+0000 7f45ce000640 1 -- 172.21.15.86:0/3928779377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8039040 con 0x55f0a3a90ec0 2024-04-17T07:25:37.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.373+0000 7f45dd600640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45c80a3eb0 0x7f45c80a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.373+0000 7f45dd600640 1 -- 172.21.15.86:0/2212684176 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f45d00d04c0 con 0x7f45e00ffc60 2024-04-17T07:25:37.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.373+0000 7f45e4e00640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45c80a3eb0 0x7f45c80a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.373+0000 7f45ea47e740 1 -- 172.21.15.86:0/3928779377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a90ec0 msgr2=0x55f0a3a912e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.374+0000 7f45ea47e740 1 --2- 172.21.15.86:0/3928779377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a90ec0 0x55f0a3a912e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f45d802c640 tx=0x7f45d8005f80 comp rx=0 tx=0).stop 2024-04-17T07:25:37.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.374+0000 7f45ea47e740 1 -- 172.21.15.86:0/3928779377 shutdown_connections 2024-04-17T07:25:37.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.374+0000 7f45ea47e740 1 --2- 172.21.15.86:0/3928779377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a96a20 0x55f0a3a98eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.374+0000 7f45ea47e740 1 --2- 172.21.15.86:0/3928779377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a918b0 0x55f0a3a963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.374+0000 7f45ea47e740 1 --2- 172.21.15.86:0/3928779377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a90ec0 0x55f0a3a912e0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.374+0000 7f45ea47e740 1 -- 172.21.15.86:0/3928779377 >> 172.21.15.86:0/3928779377 conn(0x55f0a3a8c430 msgr2=0x55f0a3a8c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.374+0000 7f45ea47e740 1 -- 172.21.15.86:0/3928779377 shutdown_connections 2024-04-17T07:25:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.374+0000 7f45e4e00640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45c80a3eb0 0x7f45c80a6370 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f45d009c350 tx=0x7f45d0098010 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.374+0000 7f45ea47e740 1 -- 172.21.15.86:0/3928779377 wait complete. 2024-04-17T07:25:37.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.375+0000 7f45ea47e740 1 -- start start 2024-04-17T07:25:37.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.375+0000 7f45ea47e740 1 Processor -- start 2024-04-17T07:25:37.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.376+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 0x55f0a3aac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.376+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a918b0 0x55f0a3aac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.376+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.376+0000 7f45ea47e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f0a3aaa980 con 0x55f0a3a96a20 2024-04-17T07:25:37.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.376+0000 7f45ea47e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f0a39b0730 con 0x55f0a3a918b0 2024-04-17T07:25:37.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.376+0000 7f45ea47e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f0a3aaab00 con 0x55f0a3a90ec0 2024-04-17T07:25:37.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.376+0000 7f45ea47e740 10 client.0 _ll_register_callbacks cb 0x55f0a39b2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:37.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.376+0000 7f45dfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a918b0 0x55f0a3aac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.376+0000 7f45dfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a918b0 0x55f0a3aac8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:35742/0 (socket says 172.21.15.86:35742) 2024-04-17T07:25:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.376+0000 7f45dfe00640 1 -- 172.21.15.86:0/3248033128 learned_addr learned my addr 172.21.15.86:0/3248033128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.377+0000 7f45e5800640 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 0x55f0a3aac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.377+0000 7f45dfe00640 1 -- 172.21.15.86:0/3248033128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 msgr2=0x55f0a3aac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.377+0000 7f45dfe00640 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 0x55f0a3aac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.377+0000 7f45dfe00640 1 -- 172.21.15.86:0/3248033128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a96a20 msgr2=0x55f0a3ab3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.377+0000 7f45e5800640 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.377+0000 7f45dfe00640 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.377+0000 7f45dfe00640 1 -- 172.21.15.86:0/3248033128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a37d17e0 con 0x55f0a3a918b0 2024-04-17T07:25:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.377+0000 7f45e5800640 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:25:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.377+0000 7f45dfe00640 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a918b0 0x55f0a3aac8f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f45d400bd80 tx=0x7f45d40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b38f0c481e3b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.378+0000 7f45c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:25:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.378+0000 7f45c2a00640 1 -- 172.21.15.86:0/3248033128 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4004220 con 0x55f0a3a918b0 2024-04-17T07:25:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.378+0000 7f45c2a00640 1 -- 172.21.15.86:0/3248033128 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f45d40043c0 con 0x55f0a3a918b0 2024-04-17T07:25:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.378+0000 7f45ea47e740 1 -- 172.21.15.86:0/3248033128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d8003e20 con 0x55f0a3a918b0 2024-04-17T07:25:37.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.378+0000 7f45c2a00640 1 -- 172.21.15.86:0/3248033128 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45d4011650 con 0x55f0a3a918b0 2024-04-17T07:25:37.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.379+0000 7f45ea47e740 10 client.37136 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:37.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.379+0000 7f45ea47e740 1 -- 172.21.15.86:0/3248033128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f0a3831f70 con 0x55f0a3a918b0 2024-04-17T07:25:37.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.379+0000 7f45c2a00640 1 -- 172.21.15.86:0/3248033128 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45d8003e20 con 0x55f0a3a918b0 2024-04-17T07:25:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.379+0000 7f45c2a00640 1 -- 172.21.15.86:0/3248033128 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f45d40038f0 con 0x55f0a3a918b0 2024-04-17T07:25:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.379+0000 7f45ea47e740 10 client.37136 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.379+0000 7f45ea47e740 10 client.37136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.380+0000 7f45ea47e740 20 client.37136 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.380+0000 7f45ea47e740 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f0a3ab8680 0x55f0a3abaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.380+0000 7f45ea47e740 4 client.37136 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.380+0000 7f45ea47e740 1 -- 172.21.15.86:0/3248033128 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0a3a046d0 con 0x55f0a3ab8680 2024-04-17T07:25:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.381+0000 7f45e5800640 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f0a3ab8680 0x55f0a3abaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.381+0000 7f45e5800640 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f0a3ab8680 0x55f0a3abaae0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee649ed0665e6070 server_cookie=1a303f19423ca663 in_seq=0 out_seq=0 2024-04-17T07:25:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.381+0000 7f45c2a00640 10 client.37136 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:37.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.383+0000 7f45c2a00640 1 -- 172.21.15.86:0/3248033128 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0a3a046d0 con 0x55f0a3ab8680 2024-04-17T07:25:37.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.383+0000 7f45c2a00640 10 client.37136 handle_command_reply: tid=0 2024-04-17T07:25:37.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.384+0000 7f45ea47e740 1 client.37136 shutdown 2024-04-17T07:25:37.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.384+0000 7f45ea47e740 1 -- 172.21.15.86:0/3248033128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a918b0 msgr2=0x55f0a3aac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.384+0000 7f45ea47e740 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a918b0 0x55f0a3aac8f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f45d400bd80 tx=0x7f45d40107a0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.384+0000 7f45ea47e740 1 -- 172.21.15.86:0/3248033128 shutdown_connections 2024-04-17T07:25:37.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.384+0000 7f45ea47e740 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f0a3ab8680 0x55f0a3abaae0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.384+0000 7f45ea47e740 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.384+0000 7f45ea47e740 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a918b0 0x55f0a3aac8f0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f45d400bd80 tx=0x7f45d40107a0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.384+0000 7f45c2a00640 0 client.37136 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:37.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.384+0000 7f45ea47e740 1 --2- 172.21.15.86:0/3248033128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 0x55f0a3aac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.384+0000 7f45ea47e740 1 -- 172.21.15.86:0/3248033128 >> 172.21.15.86:0/3248033128 conn(0x55f0a3a8c430 msgr2=0x55f0a3a8b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.385+0000 7f45ea47e740 1 -- 172.21.15.86:0/3248033128 shutdown_connections 2024-04-17T07:25:37.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.385+0000 7f45ea47e740 1 -- 172.21.15.86:0/3248033128 wait complete. 2024-04-17T07:25:37.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.385+0000 7f45ea47e740 20 client.37136 trim_cache size 0 max 16384 2024-04-17T07:25:37.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.420+0000 7f45ea47e740 1 Processor -- start 2024-04-17T07:25:37.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.420+0000 7f45ea47e740 1 -- start start 2024-04-17T07:25:37.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.421+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a90ec0 0x55f0a3aabd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.421+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.422+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a8ad50 0x55f0a3a8b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.422+0000 7f45ea47e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45c400fc20 con 0x55f0a3a8ad50 2024-04-17T07:25:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.422+0000 7f45ea47e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4011760 con 0x55f0a3a90ec0 2024-04-17T07:25:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.422+0000 7f45ea47e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45c40102f0 con 0x55f0a3a96a20 2024-04-17T07:25:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.422+0000 7f45e5800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a90ec0 0x55f0a3aabd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.422+0000 7f45e5800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a90ec0 0x55f0a3aabd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:35744/0 (socket says 172.21.15.86:35744) 2024-04-17T07:25:37.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.422+0000 7f45e5800640 1 -- 172.21.15.86:0/2346977251 learned_addr learned my addr 172.21.15.86:0/2346977251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:37.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.422+0000 7f45e5800640 1 --2- 172.21.15.86:0/2346977251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a8ad50 0x55f0a3a8b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.423+0000 7f45dfe00640 1 --2- 172.21.15.86:0/2346977251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.423+0000 7f45e5800640 1 -- 172.21.15.86:0/2346977251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a96a20 msgr2=0x55f0a3ab38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.423+0000 7f45e5800640 1 --2- 172.21.15.86:0/2346977251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.423+0000 7f45e5800640 1 -- 172.21.15.86:0/2346977251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a8ad50 msgr2=0x55f0a3a8b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.423+0000 7f45e5800640 1 --2- 172.21.15.86:0/2346977251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a8ad50 0x55f0a3a8b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.423+0000 7f45e5800640 1 -- 172.21.15.86:0/2346977251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a36c9170 con 0x55f0a3a90ec0 2024-04-17T07:25:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.424+0000 7f45e5800640 1 --2- 172.21.15.86:0/2346977251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a90ec0 0x55f0a3aabd90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000f80 tx=0x7f45d802e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acaf4ba81e6594e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.424+0000 7f45cd600640 1 -- 172.21.15.86:0/2346977251 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8057070 con 0x55f0a3a90ec0 2024-04-17T07:25:37.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.425+0000 7f45cd600640 1 -- 172.21.15.86:0/2346977251 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45d8051300 con 0x55f0a3a90ec0 2024-04-17T07:25:37.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.425+0000 7f45cd600640 1 -- 172.21.15.86:0/2346977251 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8057070 con 0x55f0a3a90ec0 2024-04-17T07:25:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.426+0000 7f45ea47e740 1 -- 172.21.15.86:0/2346977251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a90ec0 msgr2=0x55f0a3aabd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.426+0000 7f45ea47e740 1 --2- 172.21.15.86:0/2346977251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a90ec0 0x55f0a3aabd90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000f80 tx=0x7f45d802e040 comp rx=0 tx=0).stop 2024-04-17T07:25:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.426+0000 7f45ea47e740 1 -- 172.21.15.86:0/2346977251 shutdown_connections 2024-04-17T07:25:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.426+0000 7f45ea47e740 1 --2- 172.21.15.86:0/2346977251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a8ad50 0x55f0a3a8b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.426+0000 7f45ea47e740 1 --2- 172.21.15.86:0/2346977251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.426+0000 7f45ea47e740 1 --2- 172.21.15.86:0/2346977251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a90ec0 0x55f0a3aabd90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.426+0000 7f45ea47e740 1 -- 172.21.15.86:0/2346977251 >> 172.21.15.86:0/2346977251 conn(0x55f0a3a8c430 msgr2=0x55f0a3aab680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.426+0000 7f45ea47e740 1 -- 172.21.15.86:0/2346977251 shutdown_connections 2024-04-17T07:25:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.426+0000 7f45ea47e740 1 -- 172.21.15.86:0/2346977251 wait complete. 2024-04-17T07:25:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.427+0000 7f45ea47e740 1 -- start start 2024-04-17T07:25:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.427+0000 7f45ea47e740 1 Processor -- start 2024-04-17T07:25:37.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.428+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a8ad50 0x55f0a3ab30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.428+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 0x55f0a3ab3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.429+0000 7f45ea47e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.429+0000 7f45e5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a8ad50 0x55f0a3ab30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.429+0000 7f45ea47e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45c4011090 con 0x55f0a3a8ad50 2024-04-17T07:25:37.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.429+0000 7f45ea47e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45c40109c0 con 0x55f0a3a96a20 2024-04-17T07:25:37.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.429+0000 7f45ea47e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45c400dbf0 con 0x55f0a3a90ec0 2024-04-17T07:25:37.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.429+0000 7f45ea47e740 10 client.0 _ll_register_callbacks cb 0x7f45d400b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:37.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.429+0000 7f45dfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 0x55f0a3ab3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.429+0000 7f45dfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 0x55f0a3ab3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:35206/0 (socket says 172.21.15.86:35206) 2024-04-17T07:25:37.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.429+0000 7f45dfe00640 1 -- 172.21.15.86:0/2942514634 learned_addr learned my addr 172.21.15.86:0/2942514634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:37.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.429+0000 7f45e5800640 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.430+0000 7f45dfe00640 1 -- 172.21.15.86:0/2942514634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a96a20 msgr2=0x55f0a3ab1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.430+0000 7f45dfe00640 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.430+0000 7f45dfe00640 1 -- 172.21.15.86:0/2942514634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a8ad50 msgr2=0x55f0a3ab30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.430+0000 7f45dfe00640 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a8ad50 0x55f0a3ab30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.430+0000 7f45dfe00640 1 -- 172.21.15.86:0/2942514634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a3ab1d10 con 0x55f0a3a90ec0 2024-04-17T07:25:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.431+0000 7f45dfe00640 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 0x55f0a3ab3630 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55f0a3aad200 tx=0x7f45d400b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efc6ba2300b57f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.431+0000 7f45c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:25:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.431+0000 7f45c2a00640 1 -- 172.21.15.86:0/2942514634 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d40052b0 con 0x55f0a3a90ec0 2024-04-17T07:25:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.431+0000 7f45ea47e740 1 -- 172.21.15.86:0/2942514634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d8051300 con 0x55f0a3a90ec0 2024-04-17T07:25:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.431+0000 7f45c2a00640 1 -- 172.21.15.86:0/2942514634 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f45d4005450 con 0x55f0a3a90ec0 2024-04-17T07:25:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.432+0000 7f45c2a00640 1 -- 172.21.15.86:0/2942514634 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f45d4003a80 con 0x55f0a3a90ec0 2024-04-17T07:25:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.432+0000 7f45ea47e740 10 client.27223 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.432+0000 7f45ea47e740 1 -- 172.21.15.86:0/2942514634 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f0a3b1f5d0 con 0x55f0a3a90ec0 2024-04-17T07:25:37.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.433+0000 7f45c2a00640 1 -- 172.21.15.86:0/2942514634 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45d8051300 con 0x55f0a3a90ec0 2024-04-17T07:25:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.433+0000 7f45c2a00640 1 -- 172.21.15.86:0/2942514634 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f45d401b660 con 0x55f0a3a90ec0 2024-04-17T07:25:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.434+0000 7f45ea47e740 10 client.27223 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.434+0000 7f45ea47e740 10 client.27223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.434+0000 7f45ea47e740 20 client.27223 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.434+0000 7f45ea47e740 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f0a3b1ffd0 0x55f0a3b223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.434+0000 7f45ea47e740 4 client.27223 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:25:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.434+0000 7f45ea47e740 1 -- 172.21.15.86:0/2942514634 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f0a39fe540 con 0x55f0a3b1ffd0 2024-04-17T07:25:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.434+0000 7f45e5800640 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f0a3b1ffd0 0x55f0a3b223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.435+0000 7f45e5800640 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f0a3b1ffd0 0x55f0a3b223f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ddf95820c46c31 server_cookie=93896eb8f9650f5b in_seq=0 out_seq=0 2024-04-17T07:25:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.435+0000 7f45c2a00640 10 client.27223 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:37.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.437+0000 7f45c2a00640 1 -- 172.21.15.86:0/2942514634 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f0a39fe540 con 0x55f0a3b1ffd0 2024-04-17T07:25:37.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.437+0000 7f45c2a00640 10 client.27223 handle_command_reply: tid=0 2024-04-17T07:25:37.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.437+0000 7f45ea47e740 1 client.27223 shutdown 2024-04-17T07:25:37.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45ea47e740 1 -- 172.21.15.86:0/2942514634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 msgr2=0x55f0a3ab3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45ea47e740 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 0x55f0a3ab3630 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55f0a3aad200 tx=0x7f45d400b570 comp rx=0 tx=0).stop 2024-04-17T07:25:37.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45ea47e740 1 -- 172.21.15.86:0/2942514634 shutdown_connections 2024-04-17T07:25:37.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45ea47e740 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f0a3b1ffd0 0x55f0a3b223f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45ea47e740 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f0a3a96a20 0x55f0a3ab1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45ea47e740 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0a3a90ec0 0x55f0a3ab3630 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55f0a3aad200 tx=0x7f45d400b570 comp rx=0 tx=0).stop 2024-04-17T07:25:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45ea47e740 1 --2- 172.21.15.86:0/2942514634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f0a3a8ad50 0x55f0a3ab30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45ea47e740 1 -- 172.21.15.86:0/2942514634 >> 172.21.15.86:0/2942514634 conn(0x55f0a3a8c430 msgr2=0x55f0a3aab680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45c2a00640 0 client.27223 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45ea47e740 1 -- 172.21.15.86:0/2942514634 shutdown_connections 2024-04-17T07:25:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.438+0000 7f45ea47e740 1 -- 172.21.15.86:0/2942514634 wait complete. 2024-04-17T07:25:37.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.439+0000 7f45ea47e740 20 client.27223 trim_cache size 0 max 16384 2024-04-17T07:25:37.440 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:25:37.440 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:25:37.440 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "ede7540a-3019-4bd1-a492-9806c2b7911b", 2024-04-17T07:25:37.440 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:25:37.440 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:25:37.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.440+0000 7f45e7800640 1 -- 172.21.15.86:0/2212684176 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45c80a3eb0 msgr2=0x7f45c80a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.440+0000 7f45e7800640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45c80a3eb0 0x7f45c80a6370 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f45d009c350 tx=0x7f45d0098010 comp rx=0 tx=0).stop 2024-04-17T07:25:37.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.440+0000 7f45e7800640 1 -- 172.21.15.86:0/2212684176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e00ffc60 msgr2=0x7f45e019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.440+0000 7f45e7800640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e00ffc60 0x7f45e019e740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f45d000ea60 tx=0x7f45d000ef20 comp rx=0 tx=0).stop 2024-04-17T07:25:37.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.440+0000 7f45e7800640 1 -- 172.21.15.86:0/2212684176 shutdown_connections 2024-04-17T07:25:37.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.440+0000 7f45e7800640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f45c80a3eb0 0x7f45c80a6370 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.440+0000 7f45e7800640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45e0101190 0x7f45e01a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.440+0000 7f45e7800640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45e0100620 0x7f45e019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.440+0000 7f45e7800640 1 --2- 172.21.15.86:0/2212684176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e00ffc60 0x7f45e019e740 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.440+0000 7f45e7800640 1 -- 172.21.15.86:0/2212684176 >> 172.21.15.86:0/2212684176 conn(0x7f45e00fb760 msgr2=0x7f45e0107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.441+0000 7f45e7800640 1 -- 172.21.15.86:0/2212684176 shutdown_connections 2024-04-17T07:25:37.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.441+0000 7f45e7800640 1 -- 172.21.15.86:0/2212684176 wait complete. 2024-04-17T07:25:37.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:25:37.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:25:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:37 smithi098 ceph-mon[35418]: pgmap v777: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.2 MiB/s wr, 582 op/s 2024-04-17T07:25:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:37 smithi103 ceph-mon[35495]: pgmap v777: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.2 MiB/s wr, 582 op/s 2024-04-17T07:25:37.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.776+0000 7f3e47a00640 1 -- 172.21.15.86:0/3015564241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e40077b60 msgr2=0x7f3e40077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.776+0000 7f3e47a00640 1 --2- 172.21.15.86:0/3015564241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e40077b60 0x7f3e40077fc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009900 tx=0x7f3e2c02bc50 comp rx=0 tx=0).stop 2024-04-17T07:25:37.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.776+0000 7f3e47a00640 1 -- 172.21.15.86:0/3015564241 shutdown_connections 2024-04-17T07:25:37.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.776+0000 7f3e47a00640 1 --2- 172.21.15.86:0/3015564241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e40078500 0x7f3e40078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.776+0000 7f3e47a00640 1 --2- 172.21.15.86:0/3015564241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e40077b60 0x7f3e40077fc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.776+0000 7f3e47a00640 1 --2- 172.21.15.86:0/3015564241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40076920 0x7f3e40076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.776+0000 7f3e47a00640 1 -- 172.21.15.86:0/3015564241 >> 172.21.15.86:0/3015564241 conn(0x7f3e400ff750 msgr2=0x7f3e40101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.777+0000 7f3e47a00640 1 -- 172.21.15.86:0/3015564241 shutdown_connections 2024-04-17T07:25:37.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.777+0000 7f3e47a00640 1 -- 172.21.15.86:0/3015564241 wait complete. 2024-04-17T07:25:37.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.778+0000 7f3e47a00640 1 Processor -- start 2024-04-17T07:25:37.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.778+0000 7f3e47a00640 1 -- start start 2024-04-17T07:25:37.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.778+0000 7f3e47a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e40076920 0x7f3e4010fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.778+0000 7f3e47a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40077b60 0x7f3e401103a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.779+0000 7f3e47a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e40078500 0x7f3e401113e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.779+0000 7f3e47a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401044d0 con 0x7f3e40077b60 2024-04-17T07:25:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.779+0000 7f3e47a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40104350 con 0x7f3e40076920 2024-04-17T07:25:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.779+0000 7f3e47a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40104650 con 0x7f3e40078500 2024-04-17T07:25:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.779+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40077b60 0x7f3e401103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.779+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40077b60 0x7f3e401103a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52502/0 (socket says 172.21.15.86:52502) 2024-04-17T07:25:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.779+0000 7f3e3fe00640 1 -- 172.21.15.86:0/1355896641 learned_addr learned my addr 172.21.15.86:0/1355896641 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.779+0000 7f3e45a00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e40078500 0x7f3e401113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.780+0000 7f3e45000640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e40076920 0x7f3e4010fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.780+0000 7f3e45a00640 1 -- 172.21.15.86:0/1355896641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e40076920 msgr2=0x7f3e4010fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.780+0000 7f3e45a00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e40076920 0x7f3e4010fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.780+0000 7f3e45a00640 1 -- 172.21.15.86:0/1355896641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40077b60 msgr2=0x7f3e401103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.780+0000 7f3e45a00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40077b60 0x7f3e401103a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.780+0000 7f3e45a00640 1 -- 172.21.15.86:0/1355896641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e40111a50 con 0x7f3e40078500 2024-04-17T07:25:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.781+0000 7f3e45a00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e40078500 0x7f3e401113e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009fd0 tx=0x7f3e3800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.781+0000 7f3e3d600640 1 -- 172.21.15.86:0/1355896641 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3800ce20 con 0x7f3e40078500 2024-04-17T07:25:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.781+0000 7f3e47a00640 1 -- 172.21.15.86:0/1355896641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e40111d40 con 0x7f3e40078500 2024-04-17T07:25:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.781+0000 7f3e47a00640 1 -- 172.21.15.86:0/1355896641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e401af5f0 con 0x7f3e40078500 2024-04-17T07:25:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.781+0000 7f3e3d600640 1 -- 172.21.15.86:0/1355896641 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e38005020 con 0x7f3e40078500 2024-04-17T07:25:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.781+0000 7f3e3d600640 1 -- 172.21.15.86:0/1355896641 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38010690 con 0x7f3e40078500 2024-04-17T07:25:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.787+0000 7f3e4a7e4740 1 Processor -- start 2024-04-17T07:25:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.787+0000 7f3e3d600640 1 -- 172.21.15.86:0/1355896641 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3e38004a40 con 0x7f3e40078500 2024-04-17T07:25:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.787+0000 7f3e4a7e4740 1 -- start start 2024-04-17T07:25:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.788+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984190 0x55bdad9845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.788+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984b80 0x55bdad989680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.788+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad989cf0 0x55bdad98c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.788+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bdad8eb120 con 0x55bdad984b80 2024-04-17T07:25:37.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.788+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bdad891340 con 0x55bdad984190 2024-04-17T07:25:37.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.788+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bdad8e70e0 con 0x55bdad989cf0 2024-04-17T07:25:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.789+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984190 0x55bdad9845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.789+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984190 0x55bdad9845b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:35772/0 (socket says 172.21.15.86:35772) 2024-04-17T07:25:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.789+0000 7f3e3fe00640 1 -- 172.21.15.86:0/3760362875 learned_addr learned my addr 172.21.15.86:0/3760362875 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.789+0000 7f3e45000640 1 --2- 172.21.15.86:0/3760362875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984b80 0x55bdad989680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.790+0000 7f3e45000640 1 -- 172.21.15.86:0/3760362875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad989cf0 msgr2=0x55bdad98c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.790+0000 7f3e45000640 1 --2- 172.21.15.86:0/3760362875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad989cf0 0x55bdad98c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.790+0000 7f3e45000640 1 -- 172.21.15.86:0/3760362875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984190 msgr2=0x55bdad9845b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.790+0000 7f3e45000640 1 --2- 172.21.15.86:0/3760362875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984190 0x55bdad9845b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.790+0000 7f3e45000640 1 -- 172.21.15.86:0/3760362875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdad6c77d0 con 0x55bdad984b80 2024-04-17T07:25:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.790+0000 7f3e45000640 1 --2- 172.21.15.86:0/3760362875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984b80 0x55bdad989680 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400bbe0 tx=0x7f3e3400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb4517d11d53310 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.790+0000 7f3e3d600640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e280a3c30 0x7f3e280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.791+0000 7f3e32000640 1 -- 172.21.15.86:0/3760362875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34015070 con 0x55bdad984b80 2024-04-17T07:25:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.791+0000 7f3e3d600640 1 -- 172.21.15.86:0/1355896641 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3e380d0ed0 con 0x7f3e40078500 2024-04-17T07:25:37.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.791+0000 7f3e32000640 1 -- 172.21.15.86:0/3760362875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e340035f0 con 0x55bdad984b80 2024-04-17T07:25:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.792+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3760362875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984b80 msgr2=0x55bdad989680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.792+0000 7f3e32000640 1 -- 172.21.15.86:0/3760362875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34015070 con 0x55bdad984b80 2024-04-17T07:25:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.792+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/3760362875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984b80 0x55bdad989680 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400bbe0 tx=0x7f3e3400f730 comp rx=0 tx=0).stop 2024-04-17T07:25:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.792+0000 7f3e3fe00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e280a3c30 0x7f3e280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.792+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3760362875 shutdown_connections 2024-04-17T07:25:37.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.793+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/3760362875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad989cf0 0x55bdad98c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.793+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/3760362875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984b80 0x55bdad989680 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.793+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/3760362875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984190 0x55bdad9845b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.793+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3760362875 >> 172.21.15.86:0/3760362875 conn(0x55bdad97f700 msgr2=0x55bdad97fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.794+0000 7f3e3fe00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e280a3c30 0x7f3e280a60f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005c40 tx=0x7f3e2c005df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.794+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3760362875 shutdown_connections 2024-04-17T07:25:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.794+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3760362875 wait complete. 2024-04-17T07:25:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.795+0000 7f3e4a7e4740 1 -- start start 2024-04-17T07:25:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.795+0000 7f3e4a7e4740 1 Processor -- start 2024-04-17T07:25:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.796+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984190 0x55bdad98e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.796+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 0x55bdad98e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.796+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad989cf0 0x55bdad9959b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.796+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bdad98ccf0 con 0x55bdad989cf0 2024-04-17T07:25:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.796+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bdad8a4710 con 0x55bdad984b80 2024-04-17T07:25:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.796+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bdad98ce70 con 0x55bdad984190 2024-04-17T07:25:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.796+0000 7f3e4a7e4740 10 client.0 _ll_register_callbacks cb 0x55bdad89faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.797+0000 7f3e45a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad989cf0 0x55bdad9959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.797+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 0x55bdad98e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.797+0000 7f3e45a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad989cf0 0x55bdad9959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52508/0 (socket says 172.21.15.86:52508) 2024-04-17T07:25:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.797+0000 7f3e45a00640 1 -- 172.21.15.86:0/3055778850 learned_addr learned my addr 172.21.15.86:0/3055778850 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.797+0000 7f3e45000640 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984190 0x55bdad98e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.797+0000 7f3e45a00640 1 -- 172.21.15.86:0/3055778850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984190 msgr2=0x55bdad98e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.797+0000 7f3e45a00640 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984190 0x55bdad98e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.798+0000 7f3e45a00640 1 -- 172.21.15.86:0/3055778850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 msgr2=0x55bdad98e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.798+0000 7f3e45a00640 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 0x55bdad98e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.798+0000 7f3e45a00640 1 -- 172.21.15.86:0/3055778850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdad5fd3f0 con 0x55bdad989cf0 2024-04-17T07:25:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.798+0000 7f3e45000640 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984190 0x55bdad98e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:25:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.799+0000 7f3e45a00640 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad989cf0 0x55bdad9959b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f3e38099230 tx=0x7f3e3809ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e718d5af828a526a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.799+0000 7f3e26a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:25:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.799+0000 7f3e26a00640 1 -- 172.21.15.86:0/3055778850 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380a55b0 con 0x55bdad989cf0 2024-04-17T07:25:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.799+0000 7f3e26a00640 1 -- 172.21.15.86:0/3055778850 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3e380a5750 con 0x55bdad989cf0 2024-04-17T07:25:37.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.800+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3055778850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e340035f0 con 0x55bdad989cf0 2024-04-17T07:25:37.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.802+0000 7f3e26a00640 1 -- 172.21.15.86:0/3055778850 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e380b6200 con 0x55bdad989cf0 2024-04-17T07:25:37.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.802+0000 7f3e4a7e4740 10 client.18276 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:37.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.802+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3055778850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bdad5bd0f0 con 0x55bdad989cf0 2024-04-17T07:25:37.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.802+0000 7f3e26a00640 1 -- 172.21.15.86:0/3055778850 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e340035f0 con 0x55bdad989cf0 2024-04-17T07:25:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.803+0000 7f3e26a00640 1 -- 172.21.15.86:0/3055778850 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3e38098b20 con 0x55bdad989cf0 2024-04-17T07:25:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.804+0000 7f3e4a7e4740 10 client.18276 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.804+0000 7f3e4a7e4740 10 client.18276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.804+0000 7f3e4a7e4740 20 client.18276 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.804+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bdad99a300 0x55bdad99c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.804+0000 7f3e4a7e4740 4 client.18276 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:25:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.804+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3055778850 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdad8f8b40 con 0x55bdad99a300 2024-04-17T07:25:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.804+0000 7f3e45000640 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bdad99a300 0x55bdad99c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.805+0000 7f3e45000640 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bdad99a300 0x55bdad99c7c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d53adcdc420c1873 server_cookie=fca1cc26d240cd8c in_seq=0 out_seq=0 2024-04-17T07:25:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.805+0000 7f3e26a00640 10 client.18276 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.806+0000 7f3e26a00640 1 -- 172.21.15.86:0/3055778850 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdad8f8b40 con 0x55bdad99a300 2024-04-17T07:25:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.806+0000 7f3e26a00640 10 client.18276 handle_command_reply: tid=0 2024-04-17T07:25:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.806+0000 7f3e4a7e4740 1 client.18276 shutdown 2024-04-17T07:25:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.807+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3055778850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad989cf0 msgr2=0x55bdad9959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.807+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad989cf0 0x55bdad9959b0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f3e38099230 tx=0x7f3e3809ce90 comp rx=0 tx=0).stop 2024-04-17T07:25:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.807+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3055778850 shutdown_connections 2024-04-17T07:25:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.807+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bdad99a300 0x55bdad99c7c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.807+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad989cf0 0x55bdad9959b0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.807+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 0x55bdad98e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.807+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/3055778850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984190 0x55bdad98e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.807+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3055778850 >> 172.21.15.86:0/3055778850 conn(0x55bdad97f700 msgr2=0x55bdad97e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.807+0000 7f3e26a00640 0 client.18276 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.807+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3055778850 shutdown_connections 2024-04-17T07:25:37.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.808+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/3055778850 wait complete. 2024-04-17T07:25:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.808+0000 7f3e4a7e4740 20 client.18276 trim_cache size 0 max 16384 2024-04-17T07:25:37.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.850+0000 7f3e4a7e4740 1 Processor -- start 2024-04-17T07:25:37.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.851+0000 7f3e4a7e4740 1 -- start start 2024-04-17T07:25:37.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.851+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad99c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.851+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 0x55bdad995970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.852+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad989cf0 0x55bdad9a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.852+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20020bc0 con 0x55bdad984190 2024-04-17T07:25:37.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.852+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20022700 con 0x55bdad984b80 2024-04-17T07:25:37.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.852+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20021290 con 0x55bdad989cf0 2024-04-17T07:25:37.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.852+0000 7f3e45000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad99c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.852+0000 7f3e45000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad99c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52514/0 (socket says 172.21.15.86:52514) 2024-04-17T07:25:37.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.852+0000 7f3e45000640 1 -- 172.21.15.86:0/2147070198 learned_addr learned my addr 172.21.15.86:0/2147070198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:37.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.852+0000 7f3e3fe00640 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 0x55bdad995970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.853+0000 7f3e45a00640 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad989cf0 0x55bdad9a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.853+0000 7f3e45000640 1 -- 172.21.15.86:0/2147070198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad989cf0 msgr2=0x55bdad9a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.853+0000 7f3e45000640 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad989cf0 0x55bdad9a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.853+0000 7f3e45000640 1 -- 172.21.15.86:0/2147070198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 msgr2=0x55bdad995970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.853+0000 7f3e45000640 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 0x55bdad995970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.853+0000 7f3e45000640 1 -- 172.21.15.86:0/2147070198 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdad985410 con 0x55bdad984190 2024-04-17T07:25:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.853+0000 7f3e45a00640 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad989cf0 0x55bdad9a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:25:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.853+0000 7f3e3fe00640 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 0x55bdad995970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:25:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.853+0000 7f3e45000640 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad99c310 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400bb00 tx=0x7f3e3400ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686be9c067d0c680 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.854+0000 7f3e31600640 1 -- 172.21.15.86:0/2147070198 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3402b020 con 0x55bdad984190 2024-04-17T07:25:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.855+0000 7f3e31600640 1 -- 172.21.15.86:0/2147070198 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e3400b3a0 con 0x55bdad984190 2024-04-17T07:25:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.855+0000 7f3e31600640 1 -- 172.21.15.86:0/2147070198 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34015040 con 0x55bdad984190 2024-04-17T07:25:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.856+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2147070198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 msgr2=0x55bdad99c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.856+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad99c310 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400bb00 tx=0x7f3e3400ffb0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.856+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2147070198 shutdown_connections 2024-04-17T07:25:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.856+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad989cf0 0x55bdad9a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.856+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad984b80 0x55bdad995970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.856+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/2147070198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad99c310 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.856+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2147070198 >> 172.21.15.86:0/2147070198 conn(0x55bdad97f700 msgr2=0x55bdad98e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.856+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2147070198 shutdown_connections 2024-04-17T07:25:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.856+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2147070198 wait complete. 2024-04-17T07:25:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.856+0000 7f3e4a7e4740 1 -- start start 2024-04-17T07:25:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.857+0000 7f3e4a7e4740 1 Processor -- start 2024-04-17T07:25:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.857+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad992e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.857+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984b80 0x55bdad9933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.857+0000 7f3e4a7e4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad989cf0 0x55bdad9943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.857+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20022030 con 0x55bdad984190 2024-04-17T07:25:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.857+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20021960 con 0x55bdad989cf0 2024-04-17T07:25:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.857+0000 7f3e4a7e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2001eb90 con 0x55bdad984b80 2024-04-17T07:25:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.857+0000 7f3e4a7e4740 10 client.0 _ll_register_callbacks cb 0x7f3e38005bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:25:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.858+0000 7f3e45000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad992e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.858+0000 7f3e45000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad992e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52520/0 (socket says 172.21.15.86:52520) 2024-04-17T07:25:37.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.858+0000 7f3e45000640 1 -- 172.21.15.86:0/2483928360 learned_addr learned my addr 172.21.15.86:0/2483928360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:25:37.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.858+0000 7f3e3fe00640 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984b80 0x55bdad9933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.858+0000 7f3e45a00640 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad989cf0 0x55bdad9943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.859+0000 7f3e45000640 1 -- 172.21.15.86:0/2483928360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984b80 msgr2=0x55bdad9933b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.859+0000 7f3e45000640 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984b80 0x55bdad9933b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.859+0000 7f3e45000640 1 -- 172.21.15.86:0/2483928360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad989cf0 msgr2=0x55bdad9943c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:25:37.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.859+0000 7f3e45000640 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad989cf0 0x55bdad9943c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.859+0000 7f3e45000640 1 -- 172.21.15.86:0/2483928360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdad9949b0 con 0x55bdad984190 2024-04-17T07:25:37.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.860+0000 7f3e45000640 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad992e70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400f770 tx=0x7f3e34003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decde0fdc1a87874 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:25:37.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.860+0000 7f3e26a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:25:37.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.861+0000 7f3e26a00640 1 -- 172.21.15.86:0/2483928360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34033030 con 0x55bdad984190 2024-04-17T07:25:37.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.861+0000 7f3e26a00640 1 -- 172.21.15.86:0/2483928360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3e340052f0 con 0x55bdad984190 2024-04-17T07:25:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.861+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2483928360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bdad8f29b0 con 0x55bdad984190 2024-04-17T07:25:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.862+0000 7f3e26a00640 1 -- 172.21.15.86:0/2483928360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e34015070 con 0x55bdad984190 2024-04-17T07:25:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.862+0000 7f3e4a7e4740 10 client.18282 fetch_fsmap learned FSMap version 28 2024-04-17T07:25:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.862+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2483928360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bdad995c30 con 0x55bdad984190 2024-04-17T07:25:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.863+0000 7f3e26a00640 1 -- 172.21.15.86:0/2483928360 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bdad8f29b0 con 0x55bdad984190 2024-04-17T07:25:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.863+0000 7f3e26a00640 1 -- 172.21.15.86:0/2483928360 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3e340048e0 con 0x55bdad984190 2024-04-17T07:25:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.863+0000 7f3e4a7e4740 10 client.18282 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:25:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.863+0000 7f3e4a7e4740 10 client.18282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:25:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.863+0000 7f3e4a7e4740 20 client.18282 populate_metadata read hostname 'smithi086' 2024-04-17T07:25:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.863+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bdada00fc0 0x55bdada033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:25:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.863+0000 7f3e4a7e4740 4 client.18282 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:25:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.863+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2483928360 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bdad86e420 con 0x55bdada00fc0 2024-04-17T07:25:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.864+0000 7f3e3fe00640 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bdada00fc0 0x55bdada033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:25:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.866+0000 7f3e26a00640 10 client.18282 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.866+0000 7f3e3fe00640 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bdada00fc0 0x55bdada033e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a2ac4f5532e4b6 server_cookie=7d07554f64c81ff9 in_seq=0 out_seq=0 2024-04-17T07:25:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e26a00640 1 -- 172.21.15.86:0/2483928360 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bdad86e420 con 0x55bdada00fc0 2024-04-17T07:25:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e26a00640 10 client.18282 handle_command_reply: tid=0 2024-04-17T07:25:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e4a7e4740 1 client.18282 shutdown 2024-04-17T07:25:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2483928360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 msgr2=0x55bdad992e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad992e70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400f770 tx=0x7f3e34003800 comp rx=0 tx=0).stop 2024-04-17T07:25:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2483928360 shutdown_connections 2024-04-17T07:25:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bdada00fc0 0x55bdada033e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bdad989cf0 0x55bdad9943c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bdad984b80 0x55bdad9933b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e4a7e4740 1 --2- 172.21.15.86:0/2483928360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdad984190 0x55bdad992e70 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e26a00640 0 client.18282 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:25:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.867+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2483928360 >> 172.21.15.86:0/2483928360 conn(0x55bdad97f700 msgr2=0x55bdad98e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.868+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2483928360 shutdown_connections 2024-04-17T07:25:37.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.869+0000 7f3e4a7e4740 1 -- 172.21.15.86:0/2483928360 wait complete. 2024-04-17T07:25:37.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.869+0000 7f3e4a7e4740 20 client.18282 trim_cache size 0 max 16384 2024-04-17T07:25:37.871 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:25:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:25:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:25:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "ede7540a-3019-4bd1-a492-9806c2b7911b": { 2024-04-17T07:25:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:25:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "ede7540a-3019-4bd1-a492-9806c2b7911b", 2024-04-17T07:25:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:25:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:25:37.871 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:25:37.871 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:25:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.871+0000 7f3e47a00640 1 -- 172.21.15.86:0/1355896641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e280a3c30 msgr2=0x7f3e280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.871+0000 7f3e47a00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e280a3c30 0x7f3e280a60f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005c40 tx=0x7f3e2c005df0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.871+0000 7f3e47a00640 1 -- 172.21.15.86:0/1355896641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e40078500 msgr2=0x7f3e401113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:25:37.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.871+0000 7f3e47a00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e40078500 0x7f3e401113e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009fd0 tx=0x7f3e3800ef10 comp rx=0 tx=0).stop 2024-04-17T07:25:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.873+0000 7f3e47a00640 1 -- 172.21.15.86:0/1355896641 shutdown_connections 2024-04-17T07:25:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.873+0000 7f3e47a00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e280a3c30 0x7f3e280a60f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.873+0000 7f3e47a00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e40078500 0x7f3e401113e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.873+0000 7f3e47a00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e40077b60 0x7f3e401103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.873+0000 7f3e47a00640 1 --2- 172.21.15.86:0/1355896641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e40076920 0x7f3e4010fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:25:37.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.873+0000 7f3e47a00640 1 -- 172.21.15.86:0/1355896641 >> 172.21.15.86:0/1355896641 conn(0x7f3e400ff750 msgr2=0x7f3e40101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:25:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.873+0000 7f3e47a00640 1 -- 172.21.15.86:0/1355896641 shutdown_connections 2024-04-17T07:25:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:25:37.874+0000 7f3e47a00640 1 -- 172.21.15.86:0/1355896641 wait complete. 2024-04-17T07:25:37.904 INFO:tasks.cephfs.filesystem:scrub status for tag:ede7540a-3019-4bd1-a492-9806c2b7911b - {'path': '/', 'tag': 'ede7540a-3019-4bd1-a492-9806c2b7911b', 'options': 'recursive,force'} 2024-04-17T07:25:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:38 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:25:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:38 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:38 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:25:38.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:38 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:38 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:25:38.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:38 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:38.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:38 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:25:38.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:38 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:38.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:38 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:25:38.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:38 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:38.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:38 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:25:38.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:38 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:25:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 375.0 s, 605.0 tps, lat 166.018 ms stddev 54.221 2024-04-17T07:25:39.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:39 smithi098 ceph-mon[35418]: pgmap v778: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 652 op/s 2024-04-17T07:25:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:39 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:25:39.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:39 smithi103 ceph-mon[35495]: pgmap v778: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 652 op/s 2024-04-17T07:25:39.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:39 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:25:39.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:39 smithi086 ceph-mon[31733]: pgmap v778: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 652 op/s 2024-04-17T07:25:39.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:39 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:25:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:41 smithi098 ceph-mon[35418]: pgmap v779: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 629 op/s 2024-04-17T07:25:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:41 smithi103 ceph-mon[35495]: pgmap v779: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 629 op/s 2024-04-17T07:25:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:41 smithi086 ceph-mon[31733]: pgmap v779: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 629 op/s 2024-04-17T07:25:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:43 smithi103 ceph-mon[35495]: pgmap v780: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 MiB/s wr, 684 op/s 2024-04-17T07:25:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:43 smithi086 ceph-mon[31733]: pgmap v780: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 MiB/s wr, 684 op/s 2024-04-17T07:25:43.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:43 smithi098 ceph-mon[35418]: pgmap v780: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 MiB/s wr, 684 op/s 2024-04-17T07:25:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 380.0 s, 687.6 tps, lat 145.876 ms stddev 45.241 2024-04-17T07:25:45.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:45 smithi103 ceph-mon[35495]: pgmap v781: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 687 op/s 2024-04-17T07:25:45.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:45 smithi086 ceph-mon[31733]: pgmap v781: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 687 op/s 2024-04-17T07:25:45.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:45 smithi098 ceph-mon[35418]: pgmap v781: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 687 op/s 2024-04-17T07:25:47.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:47 smithi103 ceph-mon[35495]: pgmap v782: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.0 MiB/s wr, 577 op/s 2024-04-17T07:25:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:47 smithi086 ceph-mon[31733]: pgmap v782: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.0 MiB/s wr, 577 op/s 2024-04-17T07:25:47.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:47 smithi098 ceph-mon[35418]: pgmap v782: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.0 MiB/s wr, 577 op/s 2024-04-17T07:25:49.363 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 385.0 s, 638.8 tps, lat 155.492 ms stddev 50.302 2024-04-17T07:25:49.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:49 smithi103 ceph-mon[35495]: pgmap v783: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 669 op/s 2024-04-17T07:25:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:49 smithi086 ceph-mon[31733]: pgmap v783: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 669 op/s 2024-04-17T07:25:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:49 smithi098 ceph-mon[35418]: pgmap v783: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 669 op/s 2024-04-17T07:25:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:51 smithi086 ceph-mon[31733]: pgmap v784: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 636 op/s 2024-04-17T07:25:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:51 smithi098 ceph-mon[35418]: pgmap v784: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 636 op/s 2024-04-17T07:25:51.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:51 smithi103 ceph-mon[35495]: pgmap v784: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 636 op/s 2024-04-17T07:25:53.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:53 smithi086 ceph-mon[31733]: pgmap v785: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.9 MiB/s wr, 667 op/s 2024-04-17T07:25:53.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:53 smithi098 ceph-mon[35418]: pgmap v785: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.9 MiB/s wr, 667 op/s 2024-04-17T07:25:53.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:53 smithi103 ceph-mon[35495]: pgmap v785: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.9 MiB/s wr, 667 op/s 2024-04-17T07:25:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 390.0 s, 613.6 tps, lat 162.855 ms stddev 76.870 2024-04-17T07:25:55.556 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:55 smithi098 ceph-mon[35418]: pgmap v786: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-04-17T07:25:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:55 smithi086 ceph-mon[31733]: pgmap v786: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-04-17T07:25:55.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:55 smithi103 ceph-mon[35495]: pgmap v786: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-04-17T07:25:57.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:57 smithi086 ceph-mon[31733]: pgmap v787: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.8 MiB/s wr, 557 op/s 2024-04-17T07:25:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:57 smithi098 ceph-mon[35418]: pgmap v787: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.8 MiB/s wr, 557 op/s 2024-04-17T07:25:57.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:57 smithi103 ceph-mon[35495]: pgmap v787: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.8 MiB/s wr, 557 op/s 2024-04-17T07:25:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 395.0 s, 676.8 tps, lat 148.129 ms stddev 48.336 2024-04-17T07:25:59.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:25:59 smithi086 ceph-mon[31733]: pgmap v788: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-04-17T07:25:59.620 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:25:59 smithi103 ceph-mon[35495]: pgmap v788: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-04-17T07:25:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:25:59 smithi098 ceph-mon[35418]: pgmap v788: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 MiB/s wr, 651 op/s 2024-04-17T07:26:01.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:01 smithi086 ceph-mon[31733]: pgmap v789: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 583 op/s 2024-04-17T07:26:01.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:01 smithi098 ceph-mon[35418]: pgmap v789: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 583 op/s 2024-04-17T07:26:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:01 smithi103 ceph-mon[35495]: pgmap v789: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.0 MiB/s wr, 583 op/s 2024-04-17T07:26:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:03 smithi098 ceph-mon[35418]: pgmap v790: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.8 MiB/s wr, 659 op/s 2024-04-17T07:26:03.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:03 smithi103 ceph-mon[35495]: pgmap v790: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.8 MiB/s wr, 659 op/s 2024-04-17T07:26:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:03 smithi086 ceph-mon[31733]: pgmap v790: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.8 MiB/s wr, 659 op/s 2024-04-17T07:26:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 400.0 s, 649.8 tps, lat 154.139 ms stddev 44.226 2024-04-17T07:26:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:05 smithi098 ceph-mon[35418]: pgmap v791: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 631 op/s 2024-04-17T07:26:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:05 smithi103 ceph-mon[35495]: pgmap v791: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 631 op/s 2024-04-17T07:26:05.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:05 smithi086 ceph-mon[31733]: pgmap v791: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.4 MiB/s wr, 631 op/s 2024-04-17T07:26:07.635 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:07 smithi086 ceph-mon[31733]: pgmap v792: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-04-17T07:26:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:07 smithi098 ceph-mon[35418]: pgmap v792: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-04-17T07:26:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:07 smithi103 ceph-mon[35495]: pgmap v792: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-04-17T07:26:07.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:26:07.906 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:26:08.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.899+0000 7f4909400640 1 -- 172.21.15.86:0/835916486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49041097b0 msgr2=0x7f4904109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:08.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.899+0000 7f4909400640 1 --2- 172.21.15.86:0/835916486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49041097b0 0x7f4904109c30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009900 tx=0x7f48f402bc30 comp rx=0 tx=0).stop 2024-04-17T07:26:08.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.899+0000 7f4909400640 1 -- 172.21.15.86:0/835916486 shutdown_connections 2024-04-17T07:26:08.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.899+0000 7f4909400640 1 --2- 172.21.15.86:0/835916486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490410a170 0x7f4904110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.899+0000 7f4909400640 1 --2- 172.21.15.86:0/835916486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49041097b0 0x7f4904109c30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.899+0000 7f4909400640 1 --2- 172.21.15.86:0/835916486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49041085c0 0x7f49041089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.899+0000 7f4909400640 1 -- 172.21.15.86:0/835916486 >> 172.21.15.86:0/835916486 conn(0x7f4904077560 msgr2=0x7f49040799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:08.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.900+0000 7f4909400640 1 -- 172.21.15.86:0/835916486 shutdown_connections 2024-04-17T07:26:08.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.900+0000 7f4909400640 1 -- 172.21.15.86:0/835916486 wait complete. 2024-04-17T07:26:08.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.902+0000 7f4909400640 1 Processor -- start 2024-04-17T07:26:08.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.903+0000 7f4909400640 1 -- start start 2024-04-17T07:26:08.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.905+0000 7f4909400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49041085c0 0x7f4904070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.905+0000 7f4909400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49041097b0 0x7f4904070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.905+0000 7f4909400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490410a170 0x7f49041ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.905+0000 7f4909400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4904114130 con 0x7f490410a170 2024-04-17T07:26:08.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.905+0000 7f4909400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4904113fb0 con 0x7f49041097b0 2024-04-17T07:26:08.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.905+0000 7f4909400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f49041142b0 con 0x7f49041085c0 2024-04-17T07:26:08.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.906+0000 7f4902800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49041085c0 0x7f4904070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.906+0000 7f48f9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49041097b0 0x7f4904070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.906+0000 7f4902800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49041085c0 0x7f4904070970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44400/0 (socket says 172.21.15.86:44400) 2024-04-17T07:26:08.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.906+0000 7f4902800640 1 -- 172.21.15.86:0/2477542160 learned_addr learned my addr 172.21.15.86:0/2477542160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:08.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.906+0000 7f4903200640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490410a170 0x7f49041ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.906+0000 7f4902800640 1 -- 172.21.15.86:0/2477542160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49041097b0 msgr2=0x7f4904070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:08.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.906+0000 7f4902800640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49041097b0 0x7f4904070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.906+0000 7f4902800640 1 -- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490410a170 msgr2=0x7f49041ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:08.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.907+0000 7f4902800640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490410a170 0x7f49041ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.907+0000 7f4902800640 1 -- 172.21.15.86:0/2477542160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49041aed80 con 0x7f49041085c0 2024-04-17T07:26:08.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.907+0000 7f4903200640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490410a170 0x7f49041ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.907+0000 7f4902800640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49041085c0 0x7f4904070970 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00d770 tx=0x7f48ec00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.908+0000 7f48fbe00640 1 -- 172.21.15.86:0/2477542160 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec017070 con 0x7f49041085c0 2024-04-17T07:26:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.908+0000 7f48fbe00640 1 -- 172.21.15.86:0/2477542160 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48ec009c30 con 0x7f49041085c0 2024-04-17T07:26:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.908+0000 7f48fbe00640 1 -- 172.21.15.86:0/2477542160 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec003d00 con 0x7f49041085c0 2024-04-17T07:26:08.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.908+0000 7f4909400640 1 -- 172.21.15.86:0/2477542160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49041aef50 con 0x7f49041085c0 2024-04-17T07:26:08.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.908+0000 7f4909400640 1 -- 172.21.15.86:0/2477542160 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49041af3c0 con 0x7f49041085c0 2024-04-17T07:26:08.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.913+0000 7f48fbe00640 1 -- 172.21.15.86:0/2477542160 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f48ec010460 con 0x7f49041085c0 2024-04-17T07:26:08.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.915+0000 7f48fbe00640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f48e80a3c30 0x7f48e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.916+0000 7f48fbe00640 1 -- 172.21.15.86:0/2477542160 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f48ec0d13c0 con 0x7f49041085c0 2024-04-17T07:26:08.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.916+0000 7f48f9e00640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f48e80a3c30 0x7f48e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.928+0000 7f48f9e00640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f48e80a3c30 0x7f48e80a60f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f48f4000c00 tx=0x7f48f402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:08.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.929+0000 7f490c17f740 1 Processor -- start 2024-04-17T07:26:08.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.930+0000 7f490c17f740 1 -- start start 2024-04-17T07:26:08.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.930+0000 7f490c17f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfe190 0x557736cfe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.931+0000 7f490c17f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfeb80 0x557736d03680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.931+0000 7f490c17f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 0x557736d06180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.931+0000 7f490c17f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557736c65120 con 0x557736d03cf0 2024-04-17T07:26:08.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.931+0000 7f490c17f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557736c0b340 con 0x557736cfeb80 2024-04-17T07:26:08.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.931+0000 7f490c17f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557736c610e0 con 0x557736cfe190 2024-04-17T07:26:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.931+0000 7f4902800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfe190 0x557736cfe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.931+0000 7f48f9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfeb80 0x557736d03680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.931+0000 7f48f9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfeb80 0x557736d03680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37286/0 (socket says 172.21.15.86:37286) 2024-04-17T07:26:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.931+0000 7f48f9e00640 1 -- 172.21.15.86:0/566179741 learned_addr learned my addr 172.21.15.86:0/566179741 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.931+0000 7f4903200640 1 --2- 172.21.15.86:0/566179741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 0x557736d06180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.932+0000 7f4902800640 1 -- 172.21.15.86:0/566179741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfeb80 msgr2=0x557736d03680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.932+0000 7f4902800640 1 --2- 172.21.15.86:0/566179741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfeb80 0x557736d03680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.932+0000 7f4902800640 1 -- 172.21.15.86:0/566179741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 msgr2=0x557736d06180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.932+0000 7f4902800640 1 --2- 172.21.15.86:0/566179741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 0x557736d06180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.932+0000 7f4902800640 1 -- 172.21.15.86:0/566179741 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557736a417d0 con 0x557736cfe190 2024-04-17T07:26:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.932+0000 7f4903200640 1 --2- 172.21.15.86:0/566179741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 0x557736d06180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:08.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.932+0000 7f4902800640 1 --2- 172.21.15.86:0/566179741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfe190 0x557736cfe5b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f48ec004610 tx=0x7f48ec009de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221ccf221a52ec1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.933+0000 7f48f3400640 1 -- 172.21.15.86:0/566179741 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec020040 con 0x557736cfe190 2024-04-17T07:26:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.933+0000 7f48f3400640 1 -- 172.21.15.86:0/566179741 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48ec0d11b0 con 0x557736cfe190 2024-04-17T07:26:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.933+0000 7f48f3400640 1 -- 172.21.15.86:0/566179741 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0a5aa0 con 0x557736cfe190 2024-04-17T07:26:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.934+0000 7f490c17f740 1 -- 172.21.15.86:0/566179741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfe190 msgr2=0x557736cfe5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.934+0000 7f490c17f740 1 --2- 172.21.15.86:0/566179741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfe190 0x557736cfe5b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f48ec004610 tx=0x7f48ec009de0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.934+0000 7f490c17f740 1 -- 172.21.15.86:0/566179741 shutdown_connections 2024-04-17T07:26:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.934+0000 7f490c17f740 1 --2- 172.21.15.86:0/566179741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 0x557736d06180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.935+0000 7f490c17f740 1 --2- 172.21.15.86:0/566179741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfeb80 0x557736d03680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.935+0000 7f490c17f740 1 --2- 172.21.15.86:0/566179741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfe190 0x557736cfe5b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.935+0000 7f490c17f740 1 -- 172.21.15.86:0/566179741 >> 172.21.15.86:0/566179741 conn(0x557736cf9700 msgr2=0x557736cf9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.935+0000 7f490c17f740 1 -- 172.21.15.86:0/566179741 shutdown_connections 2024-04-17T07:26:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.936+0000 7f490c17f740 1 -- 172.21.15.86:0/566179741 wait complete. 2024-04-17T07:26:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.936+0000 7f490c17f740 1 -- start start 2024-04-17T07:26:08.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.936+0000 7f490c17f740 1 Processor -- start 2024-04-17T07:26:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.937+0000 7f490c17f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d08670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.937+0000 7f490c17f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d08bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.937+0000 7f490c17f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 0x557736d0fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.937+0000 7f490c17f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557736d06be0 con 0x557736cfe190 2024-04-17T07:26:08.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.937+0000 7f490c17f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557736c1e710 con 0x557736d03cf0 2024-04-17T07:26:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.937+0000 7f490c17f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557736d06d60 con 0x557736cfeb80 2024-04-17T07:26:08.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.937+0000 7f490c17f740 10 client.0 _ll_register_callbacks cb 0x557736c19af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.938+0000 7f4902800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d08670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.938+0000 7f4902800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d08670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52114/0 (socket says 172.21.15.86:52114) 2024-04-17T07:26:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.938+0000 7f4903200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 0x557736d0fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.938+0000 7f4902800640 1 -- 172.21.15.86:0/755462195 learned_addr learned my addr 172.21.15.86:0/755462195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.938+0000 7f48f9e00640 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d08bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.938+0000 7f4903200640 1 -- 172.21.15.86:0/755462195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 msgr2=0x557736d08bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.938+0000 7f4903200640 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d08bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.939+0000 7f4903200640 1 -- 172.21.15.86:0/755462195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 msgr2=0x557736d08670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.939+0000 7f4903200640 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d08670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.939+0000 7f4903200640 1 -- 172.21.15.86:0/755462195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577369773f0 con 0x557736d03cf0 2024-04-17T07:26:08.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.939+0000 7f4902800640 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d08670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.939+0000 7f4903200640 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 0x557736d0fbf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005fd0 tx=0x7f48fc012b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f15bcd73beaf893e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.939+0000 7f48e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:26:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.940+0000 7f48e7e00640 1 -- 172.21.15.86:0/755462195 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc01f060 con 0x557736d03cf0 2024-04-17T07:26:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.940+0000 7f48e7e00640 1 -- 172.21.15.86:0/755462195 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f48fc010d60 con 0x557736d03cf0 2024-04-17T07:26:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.940+0000 7f490c17f740 1 -- 172.21.15.86:0/755462195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48ec0d11b0 con 0x557736d03cf0 2024-04-17T07:26:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.940+0000 7f48e7e00640 1 -- 172.21.15.86:0/755462195 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f48fc026050 con 0x557736d03cf0 2024-04-17T07:26:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.940+0000 7f490c17f740 10 client.37172 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:08.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.941+0000 7f490c17f740 1 -- 172.21.15.86:0/755462195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577369370f0 con 0x557736d03cf0 2024-04-17T07:26:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.941+0000 7f48e7e00640 1 -- 172.21.15.86:0/755462195 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48ec0d11b0 con 0x557736d03cf0 2024-04-17T07:26:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.941+0000 7f48e7e00640 1 -- 172.21.15.86:0/755462195 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f48fc019030 con 0x557736d03cf0 2024-04-17T07:26:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.941+0000 7f490c17f740 10 client.37172 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.941+0000 7f490c17f740 10 client.37172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.942+0000 7f490c17f740 20 client.37172 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.942+0000 7f490c17f740 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557736d144b0 0x557736d16930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.942+0000 7f490c17f740 4 client.37172 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.942+0000 7f490c17f740 1 -- 172.21.15.86:0/755462195 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557736c72b40 con 0x557736d144b0 2024-04-17T07:26:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.942+0000 7f4902800640 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557736d144b0 0x557736d16930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.944+0000 7f4902800640 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557736d144b0 0x557736d16930 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf552514609906e0 server_cookie=84a153cde486b57d in_seq=0 out_seq=0 2024-04-17T07:26:08.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.944+0000 7f48e7e00640 10 client.37172 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:08.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.945+0000 7f48e7e00640 1 -- 172.21.15.86:0/755462195 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557736c72b40 con 0x557736d144b0 2024-04-17T07:26:08.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.945+0000 7f48e7e00640 10 client.37172 handle_command_reply: tid=0 2024-04-17T07:26:08.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.945+0000 7f490c17f740 1 client.37172 shutdown 2024-04-17T07:26:08.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 1 -- 172.21.15.86:0/755462195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 msgr2=0x557736d0fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 0x557736d0fbf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005fd0 tx=0x7f48fc012b80 comp rx=0 tx=0).stop 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 1 -- 172.21.15.86:0/755462195 shutdown_connections 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557736d144b0 0x557736d16930 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 0x557736d0fbf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d08bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 1 --2- 172.21.15.86:0/755462195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d08670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 1 -- 172.21.15.86:0/755462195 >> 172.21.15.86:0/755462195 conn(0x557736cf9700 msgr2=0x557736cf8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f48e7e00640 0 client.37172 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 1 -- 172.21.15.86:0/755462195 shutdown_connections 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 1 -- 172.21.15.86:0/755462195 wait complete. 2024-04-17T07:26:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.946+0000 7f490c17f740 20 client.37172 trim_cache size 0 max 16384 2024-04-17T07:26:08.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.984+0000 7f490c17f740 1 Processor -- start 2024-04-17T07:26:08.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.985+0000 7f490c17f740 1 -- start start 2024-04-17T07:26:08.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.985+0000 7f490c17f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfe190 0x557736cf8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.985+0000 7f490c17f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d0fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.985+0000 7f490c17f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 0x557736d07100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.985+0000 7f490c17f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49041b99f0 con 0x557736d03cf0 2024-04-17T07:26:08.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.985+0000 7f490c17f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f49041bb530 con 0x557736cfe190 2024-04-17T07:26:08.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.985+0000 7f490c17f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f49041ba0c0 con 0x557736cfeb80 2024-04-17T07:26:08.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.986+0000 7f4903200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 0x557736d07100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.986+0000 7f48f9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d0fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.986+0000 7f48f9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d0fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44416/0 (socket says 172.21.15.86:44416) 2024-04-17T07:26:08.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.986+0000 7f48f9e00640 1 -- 172.21.15.86:0/3753654048 learned_addr learned my addr 172.21.15.86:0/3753654048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:08.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.987+0000 7f4902800640 1 --2- 172.21.15.86:0/3753654048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfe190 0x557736cf8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.987+0000 7f4902800640 1 -- 172.21.15.86:0/3753654048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 msgr2=0x557736d0fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.987+0000 7f4902800640 1 --2- 172.21.15.86:0/3753654048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d0fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.987+0000 7f4902800640 1 -- 172.21.15.86:0/3753654048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 msgr2=0x557736d07100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.987+0000 7f4902800640 1 --2- 172.21.15.86:0/3753654048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 0x557736d07100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.987+0000 7f4902800640 1 -- 172.21.15.86:0/3753654048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557736cf3ce0 con 0x557736cfe190 2024-04-17T07:26:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.988+0000 7f4903200640 1 --2- 172.21.15.86:0/3753654048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 0x557736d07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:08.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.988+0000 7f4902800640 1 --2- 172.21.15.86:0/3753654048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfe190 0x557736cf8710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009610 tx=0x7f48ec09d1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3293b746357c8369 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:08.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.989+0000 7f48f2a00640 1 -- 172.21.15.86:0/3753654048 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec086020 con 0x557736cfe190 2024-04-17T07:26:08.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.990+0000 7f48f2a00640 1 -- 172.21.15.86:0/3753654048 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48ec0b6070 con 0x557736cfe190 2024-04-17T07:26:08.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.990+0000 7f48f2a00640 1 -- 172.21.15.86:0/3753654048 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec020040 con 0x557736cfe190 2024-04-17T07:26:08.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.991+0000 7f490c17f740 1 -- 172.21.15.86:0/3753654048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfe190 msgr2=0x557736cf8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:08.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.991+0000 7f490c17f740 1 --2- 172.21.15.86:0/3753654048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfe190 0x557736cf8710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009610 tx=0x7f48ec09d1c0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.991+0000 7f490c17f740 1 -- 172.21.15.86:0/3753654048 shutdown_connections 2024-04-17T07:26:08.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.991+0000 7f490c17f740 1 --2- 172.21.15.86:0/3753654048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736d03cf0 0x557736d07100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.991+0000 7f490c17f740 1 --2- 172.21.15.86:0/3753654048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d0fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.991+0000 7f490c17f740 1 --2- 172.21.15.86:0/3753654048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736cfe190 0x557736cf8710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.991+0000 7f490c17f740 1 -- 172.21.15.86:0/3753654048 >> 172.21.15.86:0/3753654048 conn(0x557736cf9700 msgr2=0x557736cf8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.992+0000 7f490c17f740 1 -- 172.21.15.86:0/3753654048 shutdown_connections 2024-04-17T07:26:08.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.992+0000 7f490c17f740 1 -- 172.21.15.86:0/3753654048 wait complete. 2024-04-17T07:26:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.993+0000 7f490c17f740 1 -- start start 2024-04-17T07:26:08.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.993+0000 7f490c17f740 1 Processor -- start 2024-04-17T07:26:08.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.994+0000 7f490c17f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d0d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.995+0000 7f490c17f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d0d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.995+0000 7f4902800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d0d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.995+0000 7f490c17f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 0x557736d0e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.995+0000 7f4902800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d0d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52124/0 (socket says 172.21.15.86:52124) 2024-04-17T07:26:08.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.995+0000 7f4902800640 1 -- 172.21.15.86:0/1622776799 learned_addr learned my addr 172.21.15.86:0/1622776799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.995+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49041bae60 con 0x557736cfe190 2024-04-17T07:26:08.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.995+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f49041ba790 con 0x557736d03cf0 2024-04-17T07:26:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.995+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f49041b79c0 con 0x557736cfeb80 2024-04-17T07:26:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.995+0000 7f490c17f740 10 client.0 _ll_register_callbacks cb 0x7f48fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.996+0000 7f4903200640 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 0x557736d0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.996+0000 7f4903200640 1 -- 172.21.15.86:0/1622776799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 msgr2=0x557736d0d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.996+0000 7f48f9e00640 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d0d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.996+0000 7f4903200640 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d0d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.996+0000 7f4903200640 1 -- 172.21.15.86:0/1622776799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 msgr2=0x557736d0d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:08.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.997+0000 7f4903200640 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d0d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.997+0000 7f4903200640 1 -- 172.21.15.86:0/1622776799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557736d0eb70 con 0x557736d03cf0 2024-04-17T07:26:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.997+0000 7f4902800640 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d0d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.998+0000 7f48e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:26:08.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.998+0000 7f4903200640 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 0x557736d0e580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f48fc010740 tx=0x7f48fc0096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1524f17007f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.999+0000 7f48e7e00640 1 -- 172.21.15.86:0/1622776799 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc010d60 con 0x557736d03cf0 2024-04-17T07:26:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.999+0000 7f48e7e00640 1 -- 172.21.15.86:0/1622776799 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f48fc004a00 con 0x557736d03cf0 2024-04-17T07:26:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.999+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48ec0b6070 con 0x557736d03cf0 2024-04-17T07:26:08.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:08.999+0000 7f48e7e00640 1 -- 172.21.15.86:0/1622776799 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f48fc004c20 con 0x557736d03cf0 2024-04-17T07:26:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.000+0000 7f490c17f740 10 client.37178 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.000+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557736d0fe60 con 0x557736d03cf0 2024-04-17T07:26:09.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.000+0000 7f48e7e00640 1 -- 172.21.15.86:0/1622776799 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48ec0b6070 con 0x557736d03cf0 2024-04-17T07:26:09.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.001+0000 7f48e7e00640 1 -- 172.21.15.86:0/1622776799 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f48fc019050 con 0x557736d03cf0 2024-04-17T07:26:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.002+0000 7f490c17f740 10 client.37178 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.002+0000 7f490c17f740 10 client.37178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.002+0000 7f490c17f740 20 client.37178 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.002+0000 7f490c17f740 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557736d7b090 0x557736d7d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.002+0000 7f490c17f740 4 client.37178 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:26:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.002+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557736c6c9b0 con 0x557736d7b090 2024-04-17T07:26:09.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.002+0000 7f4902800640 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557736d7b090 0x557736d7d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:09.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.004+0000 7f4902800640 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557736d7b090 0x557736d7d4b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28526c1fd5280c43 server_cookie=bfaec8c059435af4 in_seq=0 out_seq=0 2024-04-17T07:26:09.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.004+0000 7f48e7e00640 10 client.37178 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:09.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.004+0000 7f48e7e00640 1 -- 172.21.15.86:0/1622776799 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557736c6c9b0 con 0x557736d7b090 2024-04-17T07:26:09.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.004+0000 7f48e7e00640 10 client.37178 handle_command_reply: tid=0 2024-04-17T07:26:09.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.004+0000 7f490c17f740 1 client.37178 shutdown 2024-04-17T07:26:09.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.005+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 msgr2=0x557736d0e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:09.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.005+0000 7f490c17f740 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 0x557736d0e580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f48fc010740 tx=0x7f48fc0096a0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.005+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 shutdown_connections 2024-04-17T07:26:09.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.005+0000 7f490c17f740 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557736d7b090 0x557736d7d4b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.005+0000 7f490c17f740 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557736d03cf0 0x557736d0e580 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.005+0000 7f490c17f740 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557736cfeb80 0x557736d0d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.005+0000 7f490c17f740 1 --2- 172.21.15.86:0/1622776799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557736cfe190 0x557736d0d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.005+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 >> 172.21.15.86:0/1622776799 conn(0x557736cf9700 msgr2=0x557736cf8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:09.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.005+0000 7f48e7e00640 0 client.37178 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:09.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.006+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 shutdown_connections 2024-04-17T07:26:09.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.006+0000 7f490c17f740 1 -- 172.21.15.86:0/1622776799 wait complete. 2024-04-17T07:26:09.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.006+0000 7f490c17f740 20 client.37178 trim_cache size 0 max 16384 2024-04-17T07:26:09.007 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:26:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:26:09.007 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:26:09.007 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.007+0000 7f4909400640 1 -- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f48e80a3c30 msgr2=0x7f48e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.007+0000 7f4909400640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f48e80a3c30 0x7f48e80a60f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f48f4000c00 tx=0x7f48f402c7e0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.007+0000 7f4909400640 1 -- 172.21.15.86:0/2477542160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49041085c0 msgr2=0x7f4904070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.007+0000 7f4909400640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49041085c0 0x7f4904070970 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00d770 tx=0x7f48ec00dc30 comp rx=0 tx=0).stop 2024-04-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.008+0000 7f4909400640 1 -- 172.21.15.86:0/2477542160 shutdown_connections 2024-04-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.008+0000 7f4909400640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f48e80a3c30 0x7f48e80a60f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.008+0000 7f4909400640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f490410a170 0x7f49041ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.008+0000 7f4909400640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49041097b0 0x7f4904070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.008+0000 7f4909400640 1 --2- 172.21.15.86:0/2477542160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49041085c0 0x7f4904070970 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.008+0000 7f4909400640 1 -- 172.21.15.86:0/2477542160 >> 172.21.15.86:0/2477542160 conn(0x7f4904077560 msgr2=0x7f490410cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.008+0000 7f4909400640 1 -- 172.21.15.86:0/2477542160 shutdown_connections 2024-04-17T07:26:09.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.008+0000 7f4909400640 1 -- 172.21.15.86:0/2477542160 wait complete. 2024-04-17T07:26:09.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:26:09.052 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:26:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 405.0 s, 702.2 tps, lat 142.242 ms stddev 41.015 2024-04-17T07:26:09.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.513+0000 7f1c80c00640 1 -- 172.21.15.86:0/3185405208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c7c101b10 msgr2=0x7f1c7c101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:09.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.513+0000 7f1c80c00640 1 --2- 172.21.15.86:0/3185405208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c7c101b10 0x7f1c7c101f90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009ca0 tx=0x7f1c7402c120 comp rx=0 tx=0).stop 2024-04-17T07:26:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.514+0000 7f1c80c00640 1 -- 172.21.15.86:0/3185405208 shutdown_connections 2024-04-17T07:26:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.514+0000 7f1c80c00640 1 --2- 172.21.15.86:0/3185405208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c7c101b10 0x7f1c7c101f90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.514+0000 7f1c80c00640 1 --2- 172.21.15.86:0/3185405208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101170 0x7f1c7c1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.514+0000 7f1c80c00640 1 --2- 172.21.15.86:0/3185405208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c7c100810 0x7f1c7c100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.514+0000 7f1c80c00640 1 -- 172.21.15.86:0/3185405208 >> 172.21.15.86:0/3185405208 conn(0x7f1c7c0fc410 msgr2=0x7f1c7c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.514+0000 7f1c80c00640 1 -- 172.21.15.86:0/3185405208 shutdown_connections 2024-04-17T07:26:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.514+0000 7f1c80c00640 1 -- 172.21.15.86:0/3185405208 wait complete. 2024-04-17T07:26:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.516+0000 7f1c80c00640 1 Processor -- start 2024-04-17T07:26:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.516+0000 7f1c80c00640 1 -- start start 2024-04-17T07:26:09.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.517+0000 7f1c80c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c7c100810 0x7f1c7c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.517+0000 7f1c80c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c7c101170 0x7f1c7c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.517+0000 7f1c80c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101b10 0x7f1c7c1a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.517+0000 7f1c80c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10fcf0 con 0x7f1c7c101b10 2024-04-17T07:26:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.517+0000 7f1c80c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10fb70 con 0x7f1c7c100810 2024-04-17T07:26:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.517+0000 7f1c80c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c10fe70 con 0x7f1c7c101170 2024-04-17T07:26:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.518+0000 7f1c79400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c7c101170 0x7f1c7c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.518+0000 7f1c7a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101b10 0x7f1c7c1a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.518+0000 7f1c79400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c7c101170 0x7f1c7c19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44442/0 (socket says 172.21.15.86:44442) 2024-04-17T07:26:09.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.518+0000 7f1c79400640 1 -- 172.21.15.86:0/479618106 learned_addr learned my addr 172.21.15.86:0/479618106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:09.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.519+0000 7f1c79e00640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c7c100810 0x7f1c7c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:09.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.519+0000 7f1c79400640 1 -- 172.21.15.86:0/479618106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c7c100810 msgr2=0x7f1c7c19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:09.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.519+0000 7f1c79400640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c7c100810 0x7f1c7c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.519+0000 7f1c79400640 1 -- 172.21.15.86:0/479618106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101b10 msgr2=0x7f1c7c1a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.519+0000 7f1c79400640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101b10 0x7f1c7c1a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.519+0000 7f1c79400640 1 -- 172.21.15.86:0/479618106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c1a63a0 con 0x7f1c7c101170 2024-04-17T07:26:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.519+0000 7f1c7a800640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101b10 0x7f1c7c1a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.519+0000 7f1c79400640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c7c101170 0x7f1c7c19ecc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00e9f0 tx=0x7f1c6c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.520+0000 7f1c72a00640 1 -- 172.21.15.86:0/479618106 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00cd80 con 0x7f1c7c101170 2024-04-17T07:26:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.520+0000 7f1c72a00640 1 -- 172.21.15.86:0/479618106 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c6c0108e0 con 0x7f1c7c101170 2024-04-17T07:26:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.520+0000 7f1c80c00640 1 -- 172.21.15.86:0/479618106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7c1a6690 con 0x7f1c7c101170 2024-04-17T07:26:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.520+0000 7f1c72a00640 1 -- 172.21.15.86:0/479618106 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c018920 con 0x7f1c7c101170 2024-04-17T07:26:09.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.520+0000 7f1c80c00640 1 -- 172.21.15.86:0/479618106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7c1a6be0 con 0x7f1c7c101170 2024-04-17T07:26:09.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.523+0000 7f1c80c00640 1 -- 172.21.15.86:0/479618106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7c107380 con 0x7f1c7c101170 2024-04-17T07:26:09.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.527+0000 7f1c72a00640 1 -- 172.21.15.86:0/479618106 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1c6c010430 con 0x7f1c7c101170 2024-04-17T07:26:09.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.529+0000 7f1c72a00640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1c600a3d80 0x7f1c600a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:09.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.529+0000 7f1c72a00640 1 -- 172.21.15.86:0/479618106 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1c6c0d0a80 con 0x7f1c7c101170 2024-04-17T07:26:09.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.530+0000 7f1c79e00640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1c600a3d80 0x7f1c600a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:09.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.531+0000 7f1c79e00640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1c600a3d80 0x7f1c600a6240 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009aa0 tx=0x7f1c68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:09.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:09.540+0000 7f1c72a00640 1 -- 172.21.15.86:0/479618106 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1c6c09d080 con 0x7f1c7c101170 2024-04-17T07:26:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:09 smithi086 ceph-mon[31733]: pgmap v793: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.8 MiB/s wr, 667 op/s 2024-04-17T07:26:09.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:09 smithi098 ceph-mon[35418]: pgmap v793: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.8 MiB/s wr, 667 op/s 2024-04-17T07:26:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:09 smithi103 ceph-mon[35495]: pgmap v793: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.8 MiB/s wr, 667 op/s 2024-04-17T07:26:10.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.068+0000 7f1c80c00640 1 -- 172.21.15.86:0/479618106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c7c062b60 con 0x7f1c7c101170 2024-04-17T07:26:10.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.069+0000 7f1c72a00640 1 -- 172.21.15.86:0/479618106 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f1c6c09c980 con 0x7f1c7c101170 2024-04-17T07:26:10.071 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:26:10.071 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:26:10.071 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:26:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.075+0000 7f1c80c00640 1 -- 172.21.15.86:0/479618106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1c600a3d80 msgr2=0x7f1c600a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.075+0000 7f1c80c00640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1c600a3d80 0x7f1c600a6240 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009aa0 tx=0x7f1c68005f90 comp rx=0 tx=0).stop 2024-04-17T07:26:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.075+0000 7f1c80c00640 1 -- 172.21.15.86:0/479618106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c7c101170 msgr2=0x7f1c7c19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.075+0000 7f1c80c00640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c7c101170 0x7f1c7c19ecc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00e9f0 tx=0x7f1c6c00eeb0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.075+0000 7f1c80c00640 1 -- 172.21.15.86:0/479618106 shutdown_connections 2024-04-17T07:26:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.075+0000 7f1c80c00640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1c600a3d80 0x7f1c600a6240 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.075+0000 7f1c80c00640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c7c101b10 0x7f1c7c1a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.075+0000 7f1c80c00640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c7c101170 0x7f1c7c19ecc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.075+0000 7f1c80c00640 1 --2- 172.21.15.86:0/479618106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c7c100810 0x7f1c7c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.075+0000 7f1c80c00640 1 -- 172.21.15.86:0/479618106 >> 172.21.15.86:0/479618106 conn(0x7f1c7c0fc410 msgr2=0x7f1c7c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.076+0000 7f1c80c00640 1 -- 172.21.15.86:0/479618106 shutdown_connections 2024-04-17T07:26:10.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.076+0000 7f1c80c00640 1 -- 172.21.15.86:0/479618106 wait complete. 2024-04-17T07:26:10.111 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:26:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.649+0000 7f0c70a00640 1 -- 172.21.15.86:0/629988536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c1011e0 msgr2=0x7f0c6c101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.649+0000 7f0c70a00640 1 --2- 172.21.15.86:0/629988536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c1011e0 0x7f0c6c101600 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f0c54009900 tx=0x7f0c5402bc30 comp rx=0 tx=0).stop 2024-04-17T07:26:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.649+0000 7f0c70a00640 1 -- 172.21.15.86:0/629988536 shutdown_connections 2024-04-17T07:26:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.649+0000 7f0c70a00640 1 --2- 172.21.15.86:0/629988536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6c1005f0 0x7f0c6c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.649+0000 7f0c70a00640 1 --2- 172.21.15.86:0/629988536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6c0ffc30 0x7f0c6c1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.649+0000 7f0c70a00640 1 --2- 172.21.15.86:0/629988536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c1011e0 0x7f0c6c101600 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.649+0000 7f0c70a00640 1 -- 172.21.15.86:0/629988536 >> 172.21.15.86:0/629988536 conn(0x7f0c6c0fb760 msgr2=0x7f0c6c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.650+0000 7f0c70a00640 1 -- 172.21.15.86:0/629988536 shutdown_connections 2024-04-17T07:26:10.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.650+0000 7f0c70a00640 1 -- 172.21.15.86:0/629988536 wait complete. 2024-04-17T07:26:10.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.652+0000 7f0c70a00640 1 Processor -- start 2024-04-17T07:26:10.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.652+0000 7f0c70a00640 1 -- start start 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.653+0000 7f0c70a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c0ffc30 0x7f0c6c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.653+0000 7f0c70a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6c1005f0 0x7f0c6c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.653+0000 7f0c70a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6c1011e0 0x7f0c6c1a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.653+0000 7f0c70a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c10fd00 con 0x7f0c6c0ffc30 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.653+0000 7f0c70a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c10fb80 con 0x7f0c6c1005f0 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.653+0000 7f0c6a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6c1011e0 0x7f0c6c1a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.653+0000 7f0c6a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6c1011e0 0x7f0c6c1a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44472/0 (socket says 172.21.15.86:44472) 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.653+0000 7f0c69e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c0ffc30 0x7f0c6c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.653+0000 7f0c6a800640 1 -- 172.21.15.86:0/311348147 learned_addr learned my addr 172.21.15.86:0/311348147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.654+0000 7f0c70a00640 1 -- 172.21.15.86:0/311348147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c10fe80 con 0x7f0c6c1011e0 2024-04-17T07:26:10.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.654+0000 7f0c63e00640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6c1005f0 0x7f0c6c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.655+0000 7f0c6a800640 1 -- 172.21.15.86:0/311348147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6c1005f0 msgr2=0x7f0c6c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.655+0000 7f0c6a800640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6c1005f0 0x7f0c6c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.655+0000 7f0c6a800640 1 -- 172.21.15.86:0/311348147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c0ffc30 msgr2=0x7f0c6c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.655+0000 7f0c6a800640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c0ffc30 0x7f0c6c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.655+0000 7f0c6a800640 1 -- 172.21.15.86:0/311348147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c195510 con 0x7f0c6c1011e0 2024-04-17T07:26:10.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.655+0000 7f0c69e00640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c0ffc30 0x7f0c6c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:10.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.656+0000 7f0c6a800640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6c1011e0 0x7f0c6c1a5dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400eb10 tx=0x7f0c6400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:10.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.657+0000 7f0c62a00640 1 -- 172.21.15.86:0/311348147 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64004b20 con 0x7f0c6c1011e0 2024-04-17T07:26:10.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.657+0000 7f0c62a00640 1 -- 172.21.15.86:0/311348147 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c64004cc0 con 0x7f0c6c1011e0 2024-04-17T07:26:10.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.657+0000 7f0c62a00640 1 -- 172.21.15.86:0/311348147 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64010690 con 0x7f0c6c1011e0 2024-04-17T07:26:10.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.658+0000 7f0c70a00640 1 -- 172.21.15.86:0/311348147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c195800 con 0x7f0c6c1011e0 2024-04-17T07:26:10.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.658+0000 7f0c70a00640 1 -- 172.21.15.86:0/311348147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c195cd0 con 0x7f0c6c1011e0 2024-04-17T07:26:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.666+0000 7f0c7377c740 1 Processor -- start 2024-04-17T07:26:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.666+0000 7f0c62a00640 1 -- 172.21.15.86:0/311348147 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0c64005000 con 0x7f0c6c1011e0 2024-04-17T07:26:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.667+0000 7f0c7377c740 1 -- start start 2024-04-17T07:26:10.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.668+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 0x55df0d0383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.668+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 0x55df0d03d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.668+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d03ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.668+0000 7f0c7377c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df0cf9e020 con 0x55df0d03db00 2024-04-17T07:26:10.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.668+0000 7f0c7377c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55df0cf44460 con 0x55df0d038990 2024-04-17T07:26:10.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.668+0000 7f0c7377c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55df0cf99fe0 con 0x55df0d037fa0 2024-04-17T07:26:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.669+0000 7f0c69e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 0x55df0d03d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.669+0000 7f0c69e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 0x55df0d03d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37342/0 (socket says 172.21.15.86:37342) 2024-04-17T07:26:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.669+0000 7f0c69e00640 1 -- 172.21.15.86:0/1703738026 learned_addr learned my addr 172.21.15.86:0/1703738026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.669+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1703738026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 0x55df0d0383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.670+0000 7f0c62a00640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c500a3cb0 0x7f0c500a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.670+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1703738026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d03ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.670+0000 7f0c62a00640 1 -- 172.21.15.86:0/311348147 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0c640d0d40 con 0x7f0c6c1011e0 2024-04-17T07:26:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.671+0000 7f0c63e00640 1 -- 172.21.15.86:0/1703738026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 msgr2=0x55df0d03d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.671+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1703738026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 0x55df0d03d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.671+0000 7f0c63e00640 1 -- 172.21.15.86:0/1703738026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 msgr2=0x55df0d03ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.671+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1703738026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d03ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.671+0000 7f0c63e00640 1 -- 172.21.15.86:0/1703738026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0cf4c040 con 0x55df0d037fa0 2024-04-17T07:26:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.671+0000 7f0c6a800640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c500a3cb0 0x7f0c500a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.671+0000 7f0c69e00640 1 --2- 172.21.15.86:0/1703738026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 0x55df0d03d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:10.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.672+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1703738026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 0x55df0d0383c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c19fd20 tx=0x7f0c5c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b44af896fae3e6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:10.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.672+0000 7f0c6a800640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c500a3cb0 0x7f0c500a6170 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0c6409cbd0 tx=0x7f0c64005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:10.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.673+0000 7f0c5b000640 1 -- 172.21.15.86:0/1703738026 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c004030 con 0x55df0d037fa0 2024-04-17T07:26:10.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.673+0000 7f0c5b000640 1 -- 172.21.15.86:0/1703738026 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c5c005020 con 0x55df0d037fa0 2024-04-17T07:26:10.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.673+0000 7f0c5b000640 1 -- 172.21.15.86:0/1703738026 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c004a70 con 0x55df0d037fa0 2024-04-17T07:26:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.674+0000 7f0c7377c740 1 -- 172.21.15.86:0/1703738026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 msgr2=0x55df0d0383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.674+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1703738026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 0x55df0d0383c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c19fd20 tx=0x7f0c5c005850 comp rx=0 tx=0).stop 2024-04-17T07:26:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.674+0000 7f0c7377c740 1 -- 172.21.15.86:0/1703738026 shutdown_connections 2024-04-17T07:26:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.674+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1703738026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d03ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.674+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1703738026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 0x55df0d03d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.674+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1703738026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 0x55df0d0383c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.674+0000 7f0c7377c740 1 -- 172.21.15.86:0/1703738026 >> 172.21.15.86:0/1703738026 conn(0x55df0d033510 msgr2=0x55df0d033910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.674+0000 7f0c7377c740 1 -- 172.21.15.86:0/1703738026 shutdown_connections 2024-04-17T07:26:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.675+0000 7f0c7377c740 1 -- 172.21.15.86:0/1703738026 wait complete. 2024-04-17T07:26:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.675+0000 7f0c7377c740 1 -- start start 2024-04-17T07:26:10.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.676+0000 7f0c7377c740 1 Processor -- start 2024-04-17T07:26:10.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.676+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d037fa0 0x55df0d042540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.676+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d038990 0x55df0d042a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.677+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d049ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.677+0000 7f0c7377c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df0d040ab0 con 0x55df0d03db00 2024-04-17T07:26:10.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.677+0000 7f0c7377c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55df0cf57830 con 0x55df0d037fa0 2024-04-17T07:26:10.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.677+0000 7f0c7377c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55df0d040c30 con 0x55df0d038990 2024-04-17T07:26:10.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.677+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d037fa0 0x55df0d042540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.677+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d037fa0 0x55df0d042540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37350/0 (socket says 172.21.15.86:37350) 2024-04-17T07:26:10.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.677+0000 7f0c63e00640 1 -- 172.21.15.86:0/1107256906 learned_addr learned my addr 172.21.15.86:0/1107256906 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:10.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.677+0000 7f0c7377c740 10 client.0 _ll_register_callbacks cb 0x55df0cf25fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:10.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.678+0000 7f0c69e00640 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d038990 0x55df0d042a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.678+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d049ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.679+0000 7f0c63e00640 1 -- 172.21.15.86:0/1107256906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d038990 msgr2=0x55df0d042a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:10.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.679+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d038990 0x55df0d042a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.679+0000 7f0c63e00640 1 -- 172.21.15.86:0/1107256906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d037fa0 msgr2=0x55df0d042540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:10.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.679+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d037fa0 0x55df0d042540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.679+0000 7f0c63e00640 1 -- 172.21.15.86:0/1107256906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0cf5b920 con 0x55df0d03db00 2024-04-17T07:26:10.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.679+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d049ac0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009c30 tx=0x7f0c5c009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d962b00e56c5e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:10.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.679+0000 7f0c4f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:10.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.680+0000 7f0c4f400640 1 -- 172.21.15.86:0/1107256906 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c005420 con 0x55df0d03db00 2024-04-17T07:26:10.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.680+0000 7f0c7377c740 1 -- 172.21.15.86:0/1107256906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c5c005020 con 0x55df0d03db00 2024-04-17T07:26:10.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.680+0000 7f0c4f400640 1 -- 172.21.15.86:0/1107256906 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0c5c004810 con 0x55df0d03db00 2024-04-17T07:26:10.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.681+0000 7f0c4f400640 1 -- 172.21.15.86:0/1107256906 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c5c035050 con 0x55df0d03db00 2024-04-17T07:26:10.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.681+0000 7f0c7377c740 10 client.18315 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:10.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.681+0000 7f0c7377c740 1 -- 172.21.15.86:0/1107256906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df0cf297f0 con 0x55df0d03db00 2024-04-17T07:26:10.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.682+0000 7f0c4f400640 1 -- 172.21.15.86:0/1107256906 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c5c005020 con 0x55df0d03db00 2024-04-17T07:26:10.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.682+0000 7f0c4f400640 1 -- 172.21.15.86:0/1107256906 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0c5c0035f0 con 0x55df0d03db00 2024-04-17T07:26:10.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.682+0000 7f0c7377c740 10 client.18315 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:10.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.682+0000 7f0c7377c740 10 client.18315 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:26:10.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.682+0000 7f0c7377c740 20 client.18315 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:10.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.682+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55df0d04dd90 0x55df0d050180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.682+0000 7f0c7377c740 4 client.18315 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:10.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.682+0000 7f0c7377c740 1 -- 172.21.15.86:0/1107256906 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df0cfaba40 con 0x55df0d04dd90 2024-04-17T07:26:10.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.683+0000 7f0c69e00640 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55df0d04dd90 0x55df0d050180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.684+0000 7f0c69e00640 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55df0d04dd90 0x55df0d050180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8bf6f9bdd76e862 server_cookie=7d8335606c3f9862 in_seq=0 out_seq=0 2024-04-17T07:26:10.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.684+0000 7f0c4f400640 10 client.18315 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:26:10.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.686+0000 7f0c4f400640 1 -- 172.21.15.86:0/1107256906 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df0cfaba40 con 0x55df0d04dd90 2024-04-17T07:26:10.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.686+0000 7f0c4f400640 10 client.18315 handle_command_reply: tid=0 2024-04-17T07:26:10.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.687+0000 7f0c7377c740 1 client.18315 shutdown 2024-04-17T07:26:10.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.688+0000 7f0c7377c740 1 -- 172.21.15.86:0/1107256906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 msgr2=0x55df0d049ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.690+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d049ac0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009c30 tx=0x7f0c5c009e40 comp rx=0 tx=0).stop 2024-04-17T07:26:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.690+0000 7f0c7377c740 1 -- 172.21.15.86:0/1107256906 shutdown_connections 2024-04-17T07:26:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.690+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55df0d04dd90 0x55df0d050180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.690+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d049ac0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.690+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d038990 0x55df0d042a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.690+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1107256906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d037fa0 0x55df0d042540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.690+0000 7f0c4f400640 0 client.18315 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:26:10.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.690+0000 7f0c7377c740 1 -- 172.21.15.86:0/1107256906 >> 172.21.15.86:0/1107256906 conn(0x55df0d033510 msgr2=0x55df0d032710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:10.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.692+0000 7f0c7377c740 1 -- 172.21.15.86:0/1107256906 shutdown_connections 2024-04-17T07:26:10.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.692+0000 7f0c7377c740 1 -- 172.21.15.86:0/1107256906 wait complete. 2024-04-17T07:26:10.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.692+0000 7f0c7377c740 20 client.18315 trim_cache size 0 max 16384 2024-04-17T07:26:10.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.734+0000 7f0c7377c740 1 Processor -- start 2024-04-17T07:26:10.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.736+0000 7f0c7377c740 1 -- start start 2024-04-17T07:26:10.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.736+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d037fa0 0x55df0d036e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.736+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d038990 0x55df0d049a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.736+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d03db00 0x55df0d054930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.736+0000 7f0c7377c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1b9a50 con 0x55df0d037fa0 2024-04-17T07:26:10.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.736+0000 7f0c7377c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1bb590 con 0x55df0d03db00 2024-04-17T07:26:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.736+0000 7f0c7377c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1ba120 con 0x55df0d038990 2024-04-17T07:26:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.737+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d037fa0 0x55df0d036e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.737+0000 7f0c69e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d038990 0x55df0d049a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.737+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d037fa0 0x55df0d036e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52168/0 (socket says 172.21.15.86:52168) 2024-04-17T07:26:10.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.737+0000 7f0c63e00640 1 -- 172.21.15.86:0/1502368771 learned_addr learned my addr 172.21.15.86:0/1502368771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:10.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.737+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1502368771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d03db00 0x55df0d054930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.738+0000 7f0c63e00640 1 -- 172.21.15.86:0/1502368771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d038990 msgr2=0x55df0d049a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:10.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.738+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1502368771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d038990 0x55df0d049a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.738+0000 7f0c63e00640 1 -- 172.21.15.86:0/1502368771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d03db00 msgr2=0x55df0d054930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:10.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.738+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1502368771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d03db00 0x55df0d054930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.738+0000 7f0c63e00640 1 -- 172.21.15.86:0/1502368771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0ccb03f0 con 0x55df0d037fa0 2024-04-17T07:26:10.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.738+0000 7f0c63e00640 1 --2- 172.21.15.86:0/1502368771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d037fa0 0x55df0d036e80 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c003ef0 tx=0x7f0c5c0099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08fba4c75e36fda server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:10.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.739+0000 7f0c5a600640 1 -- 172.21.15.86:0/1502368771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c004520 con 0x55df0d037fa0 2024-04-17T07:26:10.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.739+0000 7f0c5a600640 1 -- 172.21.15.86:0/1502368771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c5c003be0 con 0x55df0d037fa0 2024-04-17T07:26:10.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.739+0000 7f0c5a600640 1 -- 172.21.15.86:0/1502368771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00fb90 con 0x55df0d037fa0 2024-04-17T07:26:10.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.741+0000 7f0c7377c740 1 -- 172.21.15.86:0/1502368771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d037fa0 msgr2=0x55df0d036e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.741+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1502368771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d037fa0 0x55df0d036e80 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c003ef0 tx=0x7f0c5c0099a0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.742+0000 7f0c7377c740 1 -- 172.21.15.86:0/1502368771 shutdown_connections 2024-04-17T07:26:10.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.742+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1502368771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d03db00 0x55df0d054930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.742+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1502368771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d038990 0x55df0d049a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.742+0000 7f0c7377c740 1 --2- 172.21.15.86:0/1502368771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d037fa0 0x55df0d036e80 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.742+0000 7f0c7377c740 1 -- 172.21.15.86:0/1502368771 >> 172.21.15.86:0/1502368771 conn(0x55df0d033510 msgr2=0x55df0d036770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:10.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.742+0000 7f0c7377c740 1 -- 172.21.15.86:0/1502368771 shutdown_connections 2024-04-17T07:26:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.742+0000 7f0c7377c740 1 -- 172.21.15.86:0/1502368771 wait complete. 2024-04-17T07:26:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.743+0000 7f0c7377c740 1 -- start start 2024-04-17T07:26:10.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.743+0000 7f0c7377c740 1 Processor -- start 2024-04-17T07:26:10.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.745+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 0x55df0d046b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.745+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 0x55df0d0470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.745+0000 7f0c7377c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d0480c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.745+0000 7f0c7377c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1baec0 con 0x55df0d03db00 2024-04-17T07:26:10.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.745+0000 7f0c7377c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1ba7f0 con 0x55df0d038990 2024-04-17T07:26:10.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.745+0000 7f0c7377c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c19d9e0 con 0x55df0d037fa0 2024-04-17T07:26:10.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.745+0000 7f0c7377c740 10 client.0 _ll_register_callbacks cb 0x7f0c5c0043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:10.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.746+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d0480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.746+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d0480c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52172/0 (socket says 172.21.15.86:52172) 2024-04-17T07:26:10.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.746+0000 7f0c63e00640 1 -- 172.21.15.86:0/322627136 learned_addr learned my addr 172.21.15.86:0/322627136 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:10.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.746+0000 7f0c63e00640 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 0x55df0d046b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.746+0000 7f0c69e00640 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 0x55df0d0470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.747+0000 7f0c63e00640 1 -- 172.21.15.86:0/322627136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 msgr2=0x55df0d046b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:10.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.747+0000 7f0c63e00640 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 0x55df0d046b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.747+0000 7f0c63e00640 1 -- 172.21.15.86:0/322627136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 msgr2=0x55df0d0470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:10.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.747+0000 7f0c63e00640 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 0x55df0d0470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.747+0000 7f0c63e00640 1 -- 172.21.15.86:0/322627136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0cd7a7d0 con 0x55df0d03db00 2024-04-17T07:26:10.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.748+0000 7f0c63e00640 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d0480c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009380 tx=0x7f0c5c021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f9b86da38fb058 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:10.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.748+0000 7f0c4f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:10.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.748+0000 7f0c4f400640 1 -- 172.21.15.86:0/322627136 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c011230 con 0x55df0d03db00 2024-04-17T07:26:10.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.749+0000 7f0c4f400640 1 -- 172.21.15.86:0/322627136 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0c5c00fb20 con 0x55df0d03db00 2024-04-17T07:26:10.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.749+0000 7f0c7377c740 1 -- 172.21.15.86:0/322627136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c5c003be0 con 0x55df0d03db00 2024-04-17T07:26:10.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.749+0000 7f0c4f400640 1 -- 172.21.15.86:0/322627136 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c5c0035f0 con 0x55df0d03db00 2024-04-17T07:26:10.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.749+0000 7f0c7377c740 10 client.18321 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:10.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.749+0000 7f0c7377c740 1 -- 172.21.15.86:0/322627136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df0d0497e0 con 0x55df0d03db00 2024-04-17T07:26:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.750+0000 7f0c4f400640 1 -- 172.21.15.86:0/322627136 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c5c003be0 con 0x55df0d03db00 2024-04-17T07:26:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.751+0000 7f0c4f400640 1 -- 172.21.15.86:0/322627136 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0c5c0484d0 con 0x55df0d03db00 2024-04-17T07:26:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.751+0000 7f0c7377c740 10 client.18321 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.751+0000 7f0c7377c740 10 client.18321 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:26:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.751+0000 7f0c7377c740 20 client.18321 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.751+0000 7f0c7377c740 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55df0d0b41b0 0x55df0d0b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.751+0000 7f0c7377c740 4 client.18321 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:26:10.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.751+0000 7f0c7377c740 1 -- 172.21.15.86:0/322627136 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df0cfa58b0 con 0x55df0d0b41b0 2024-04-17T07:26:10.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.752+0000 7f0c69e00640 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55df0d0b41b0 0x55df0d0b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:10.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:10 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/479618106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:26:10.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.754+0000 7f0c69e00640 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55df0d0b41b0 0x55df0d0b65a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d8deefb4c51e5844 server_cookie=34b24807f665a3aa in_seq=0 out_seq=0 2024-04-17T07:26:10.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.754+0000 7f0c4f400640 10 client.18321 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:26:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.754+0000 7f0c4f400640 1 -- 172.21.15.86:0/322627136 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df0cfa58b0 con 0x55df0d0b41b0 2024-04-17T07:26:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.755+0000 7f0c4f400640 10 client.18321 handle_command_reply: tid=0 2024-04-17T07:26:10.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.755+0000 7f0c7377c740 1 client.18321 shutdown 2024-04-17T07:26:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.755+0000 7f0c7377c740 1 -- 172.21.15.86:0/322627136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 msgr2=0x55df0d0480c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.755+0000 7f0c7377c740 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d0480c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009380 tx=0x7f0c5c021040 comp rx=0 tx=0).stop 2024-04-17T07:26:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.756+0000 7f0c7377c740 1 -- 172.21.15.86:0/322627136 shutdown_connections 2024-04-17T07:26:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.756+0000 7f0c7377c740 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55df0d0b41b0 0x55df0d0b65a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.756+0000 7f0c7377c740 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df0d03db00 0x55df0d0480c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.756+0000 7f0c4f400640 0 client.18321 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:26:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.756+0000 7f0c7377c740 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55df0d038990 0x55df0d0470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.756+0000 7f0c7377c740 1 --2- 172.21.15.86:0/322627136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df0d037fa0 0x55df0d046b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.756+0000 7f0c7377c740 1 -- 172.21.15.86:0/322627136 >> 172.21.15.86:0/322627136 conn(0x55df0d033510 msgr2=0x55df0d037000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:10.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.756+0000 7f0c7377c740 1 -- 172.21.15.86:0/322627136 shutdown_connections 2024-04-17T07:26:10.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.757+0000 7f0c7377c740 1 -- 172.21.15.86:0/322627136 wait complete. 2024-04-17T07:26:10.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.757+0000 7f0c7377c740 20 client.18321 trim_cache size 0 max 16384 2024-04-17T07:26:10.758 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:26:10.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 -- 172.21.15.86:0/311348147 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c500a3cb0 msgr2=0x7f0c500a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c500a3cb0 0x7f0c500a6170 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0c6409cbd0 tx=0x7f0c64005b40 comp rx=0 tx=0).stop 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 -- 172.21.15.86:0/311348147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6c1011e0 msgr2=0x7f0c6c1a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6c1011e0 0x7f0c6c1a5dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400eb10 tx=0x7f0c6400c2d0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 -- 172.21.15.86:0/311348147 shutdown_connections 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0c500a3cb0 0x7f0c500a6170 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c6c1011e0 0x7f0c6c1a5dc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c6c1005f0 0x7f0c6c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 --2- 172.21.15.86:0/311348147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c0ffc30 0x7f0c6c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 -- 172.21.15.86:0/311348147 >> 172.21.15.86:0/311348147 conn(0x7f0c6c0fb760 msgr2=0x7f0c6c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.758+0000 7f0c70a00640 1 -- 172.21.15.86:0/311348147 shutdown_connections 2024-04-17T07:26:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:10.759+0000 7f0c70a00640 1 -- 172.21.15.86:0/311348147 wait complete. 2024-04-17T07:26:10.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:10 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/479618106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:26:10.782 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:10 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/479618106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:26:10.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:26:11.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.298+0000 7fe3b6a00640 1 -- 172.21.15.86:0/1860642134 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a802cbf0 con 0x7fe3b8101e90 2024-04-17T07:26:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.299+0000 7fe3bdc00640 1 -- 172.21.15.86:0/1860642134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b8101e90 msgr2=0x7fe3b81022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.299+0000 7fe3bdc00640 1 --2- 172.21.15.86:0/1860642134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b8101e90 0x7fe3b81022b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009900 tx=0x7fe3a802bc40 comp rx=0 tx=0).stop 2024-04-17T07:26:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.300+0000 7fe3bdc00640 1 -- 172.21.15.86:0/1860642134 shutdown_connections 2024-04-17T07:26:11.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.300+0000 7fe3bdc00640 1 --2- 172.21.15.86:0/1860642134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b81039a0 0x7fe3b810a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.300+0000 7fe3bdc00640 1 --2- 172.21.15.86:0/1860642134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3b8102fe0 0x7fe3b8103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.300+0000 7fe3bdc00640 1 --2- 172.21.15.86:0/1860642134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b8101e90 0x7fe3b81022b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.300+0000 7fe3bdc00640 1 -- 172.21.15.86:0/1860642134 >> 172.21.15.86:0/1860642134 conn(0x7fe3b80fd400 msgr2=0x7fe3b80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.300+0000 7fe3bdc00640 1 -- 172.21.15.86:0/1860642134 shutdown_connections 2024-04-17T07:26:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.300+0000 7fe3bdc00640 1 -- 172.21.15.86:0/1860642134 wait complete. 2024-04-17T07:26:11.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.301+0000 7fe3bdc00640 1 Processor -- start 2024-04-17T07:26:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.301+0000 7fe3bdc00640 1 -- start start 2024-04-17T07:26:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.302+0000 7fe3bdc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b8101e90 0x7fe3b819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.302+0000 7fe3bdc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3b8102fe0 0x7fe3b819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.302+0000 7fe3bdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b81039a0 0x7fe3b81a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.302+0000 7fe3bdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b8075c80 con 0x7fe3b81039a0 2024-04-17T07:26:11.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.302+0000 7fe3bdc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b8075b00 con 0x7fe3b8101e90 2024-04-17T07:26:11.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.302+0000 7fe3bdc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b8075e00 con 0x7fe3b8102fe0 2024-04-17T07:26:11.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.303+0000 7fe3bd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b81039a0 0x7fe3b81a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.303+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b8101e90 0x7fe3b819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.303+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3b8102fe0 0x7fe3b819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.303+0000 7fe3bd200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b81039a0 0x7fe3b81a5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52188/0 (socket says 172.21.15.86:52188) 2024-04-17T07:26:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.303+0000 7fe3bd200640 1 -- 172.21.15.86:0/3424529838 learned_addr learned my addr 172.21.15.86:0/3424529838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.303+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3b8102fe0 0x7fe3b819ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44512/0 (socket says 172.21.15.86:44512) 2024-04-17T07:26:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.303+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b8101e90 0x7fe3b819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37374/0 (socket says 172.21.15.86:37374) 2024-04-17T07:26:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.304+0000 7fe3bd200640 1 -- 172.21.15.86:0/3424529838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3b8102fe0 msgr2=0x7fe3b819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.304+0000 7fe3bd200640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3b8102fe0 0x7fe3b819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.304+0000 7fe3bd200640 1 -- 172.21.15.86:0/3424529838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b8101e90 msgr2=0x7fe3b819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.304+0000 7fe3bd200640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b8101e90 0x7fe3b819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.304+0000 7fe3bd200640 1 -- 172.21.15.86:0/3424529838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b806ba80 con 0x7fe3b81039a0 2024-04-17T07:26:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.305+0000 7fe3bd200640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b81039a0 0x7fe3b81a5ca0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000eb10 tx=0x7fe3b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.305+0000 7fe3b4c00640 1 -- 172.21.15.86:0/3424529838 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0004b20 con 0x7fe3b81039a0 2024-04-17T07:26:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.306+0000 7fe3b4c00640 1 -- 172.21.15.86:0/3424529838 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3b0004cc0 con 0x7fe3b81039a0 2024-04-17T07:26:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.306+0000 7fe3b4c00640 1 -- 172.21.15.86:0/3424529838 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0010720 con 0x7fe3b81039a0 2024-04-17T07:26:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.306+0000 7fe3bdc00640 1 -- 172.21.15.86:0/3424529838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b806bd70 con 0x7fe3b81039a0 2024-04-17T07:26:11.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.306+0000 7fe3bdc00640 1 -- 172.21.15.86:0/3424529838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b806c270 con 0x7fe3b81039a0 2024-04-17T07:26:11.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.311+0000 7fe3c1e4e740 1 Processor -- start 2024-04-17T07:26:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.312+0000 7fe3b4c00640 1 -- 172.21.15.86:0/3424529838 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fe3b0005000 con 0x7fe3b81039a0 2024-04-17T07:26:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.312+0000 7fe3c1e4e740 1 -- start start 2024-04-17T07:26:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.313+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb33050580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.314+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb33055650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.314+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb33058150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.314+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb32fb6020 con 0x55cb33050b50 2024-04-17T07:26:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.314+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cb32f5c460 con 0x55cb33055cc0 2024-04-17T07:26:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.314+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cb32fb1fe0 con 0x55cb33050160 2024-04-17T07:26:11.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.314+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb33055650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.314+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb33050580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.314+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb33055650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52204/0 (socket says 172.21.15.86:52204) 2024-04-17T07:26:11.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.314+0000 7fe3b7e00640 1 -- 172.21.15.86:0/1163043546 learned_addr learned my addr 172.21.15.86:0/1163043546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:11.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.314+0000 7fe3b7400640 1 --2- 172.21.15.86:0/1163043546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb33058150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.315+0000 7fe3b4c00640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe39c0a3c30 0x7fe39c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.315+0000 7fe3b4c00640 1 -- 172.21.15.86:0/3424529838 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe3b00d0e20 con 0x7fe3b81039a0 2024-04-17T07:26:11.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.315+0000 7fe3b7e00640 1 -- 172.21.15.86:0/1163043546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 msgr2=0x55cb33050580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:11.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.315+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/1163043546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb33050580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.316+0000 7fe3b7e00640 1 -- 172.21.15.86:0/1163043546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 msgr2=0x55cb33058150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:11.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.316+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/1163043546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb33058150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.316+0000 7fe3b7e00640 1 -- 172.21.15.86:0/1163043546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb32f64040 con 0x55cb33050b50 2024-04-17T07:26:11.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.316+0000 7fe3b7400640 1 --2- 172.21.15.86:0/1163043546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb33058150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:11.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.316+0000 7fe3bd200640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe39c0a3c30 0x7fe39c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.316+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/1163043546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb33055650 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009230 tx=0x7fe3a80095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b72b0aa9fa74ce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:11.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.317+0000 7fe3ad600640 1 -- 172.21.15.86:0/1163043546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8045020 con 0x55cb33050b50 2024-04-17T07:26:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.317+0000 7fe3ad600640 1 -- 172.21.15.86:0/1163043546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3a802cab0 con 0x55cb33050b50 2024-04-17T07:26:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.317+0000 7fe3ad600640 1 -- 172.21.15.86:0/1163043546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a80046c0 con 0x55cb33050b50 2024-04-17T07:26:11.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.318+0000 7fe3bd200640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe39c0a3c30 0x7fe39c0a60f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe3b009ca80 tx=0x7fe3b000c950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.319+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/1163043546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 msgr2=0x55cb33055650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.319+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/1163043546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb33055650 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009230 tx=0x7fe3a80095e0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.320+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/1163043546 shutdown_connections 2024-04-17T07:26:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.320+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/1163043546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb33058150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.320+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/1163043546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb33055650 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.320+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/1163043546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb33050580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.320+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/1163043546 >> 172.21.15.86:0/1163043546 conn(0x55cb3304b6d0 msgr2=0x55cb3304bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.320+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/1163043546 shutdown_connections 2024-04-17T07:26:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.320+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/1163043546 wait complete. 2024-04-17T07:26:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.321+0000 7fe3c1e4e740 1 -- start start 2024-04-17T07:26:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.321+0000 7fe3c1e4e740 1 Processor -- start 2024-04-17T07:26:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb3305ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb33061c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cb33058c10 con 0x55cb33050b50 2024-04-17T07:26:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cb32f6f830 con 0x55cb33055cc0 2024-04-17T07:26:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cb33058d90 con 0x55cb33050160 2024-04-17T07:26:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3c1e4e740 10 client.0 _ll_register_callbacks cb 0x55cb32f3dfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb3305ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44516/0 (socket says 172.21.15.86:44516) 2024-04-17T07:26:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.322+0000 7fe3b7400640 1 -- 172.21.15.86:0/4226471410 learned_addr learned my addr 172.21.15.86:0/4226471410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.323+0000 7fe3b7400640 1 -- 172.21.15.86:0/4226471410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 msgr2=0x55cb33061c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.323+0000 7fe3b7400640 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb33061c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.323+0000 7fe3b7400640 1 -- 172.21.15.86:0/4226471410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 msgr2=0x55cb3305ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.323+0000 7fe3b7400640 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb3305ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.323+0000 7fe3b7400640 1 -- 172.21.15.86:0/4226471410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb32f73920 con 0x55cb33050160 2024-04-17T07:26:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.323+0000 7fe3b7400640 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305a6d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400eab0 tx=0x7fe3a4014f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48dc640cd6e8a272 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.323+0000 7fe3a2000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.324+0000 7fe3a2000640 1 -- 172.21.15.86:0/4226471410 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4011e50 con 0x55cb33050160 2024-04-17T07:26:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.324+0000 7fe3a2000640 1 -- 172.21.15.86:0/4226471410 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe3a4004030 con 0x55cb33050160 2024-04-17T07:26:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.325+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/4226471410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3a802cab0 con 0x55cb33050160 2024-04-17T07:26:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.325+0000 7fe3a2000640 1 -- 172.21.15.86:0/4226471410 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3a4004230 con 0x55cb33050160 2024-04-17T07:26:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.325+0000 7fe3c1e4e740 10 client.27280 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.325+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/4226471410 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cb32f417f0 con 0x55cb33050160 2024-04-17T07:26:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.326+0000 7fe3a2000640 1 -- 172.21.15.86:0/4226471410 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3a802cab0 con 0x55cb33050160 2024-04-17T07:26:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.327+0000 7fe3a2000640 1 -- 172.21.15.86:0/4226471410 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe3a401e630 con 0x55cb33050160 2024-04-17T07:26:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.327+0000 7fe3c1e4e740 10 client.27280 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.327+0000 7fe3c1e4e740 10 client.27280 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:26:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.327+0000 7fe3c1e4e740 20 client.27280 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.327+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cb33066110 0x55cb33068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.327+0000 7fe3c1e4e740 4 client.27280 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.327+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/4226471410 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb32fc3a40 con 0x55cb33066110 2024-04-17T07:26:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.329+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cb33066110 0x55cb33068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.331+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cb33066110 0x55cb33068590 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93b5937c9cf7ba8b server_cookie=3d6e025f8d049210 in_seq=0 out_seq=0 2024-04-17T07:26:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.331+0000 7fe3a2000640 10 client.27280 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:26:11.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.333+0000 7fe3a2000640 1 -- 172.21.15.86:0/4226471410 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb32fc3a40 con 0x55cb33066110 2024-04-17T07:26:11.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.333+0000 7fe3a2000640 10 client.27280 handle_command_reply: tid=0 2024-04-17T07:26:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.333+0000 7fe3c1e4e740 1 client.27280 shutdown 2024-04-17T07:26:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.334+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/4226471410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 msgr2=0x55cb3305a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.334+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305a6d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400eab0 tx=0x7fe3a4014f90 comp rx=0 tx=0).stop 2024-04-17T07:26:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.335+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/4226471410 shutdown_connections 2024-04-17T07:26:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.335+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cb33066110 0x55cb33068590 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.335+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb33061c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.335+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb3305ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.335+0000 7fe3a2000640 0 client.27280 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:26:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.335+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/4226471410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305a6d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.335+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/4226471410 >> 172.21.15.86:0/4226471410 conn(0x55cb3304b6d0 msgr2=0x55cb3304a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.336+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/4226471410 shutdown_connections 2024-04-17T07:26:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.336+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/4226471410 wait complete. 2024-04-17T07:26:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.337+0000 7fe3c1e4e740 20 client.27280 trim_cache size 0 max 16384 2024-04-17T07:26:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.364+0000 7fe3c1e4e740 1 Processor -- start 2024-04-17T07:26:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.364+0000 7fe3c1e4e740 1 -- start start 2024-04-17T07:26:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.364+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.364+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb33061c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.364+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb3304f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.365+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b81b97d0 con 0x55cb33050b50 2024-04-17T07:26:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.365+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b81bb310 con 0x55cb33055cc0 2024-04-17T07:26:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.365+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b81b9ea0 con 0x55cb33050160 2024-04-17T07:26:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.365+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.365+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44520/0 (socket says 172.21.15.86:44520) 2024-04-17T07:26:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.365+0000 7fe3b7400640 1 -- 172.21.15.86:0/455071378 learned_addr learned my addr 172.21.15.86:0/455071378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.365+0000 7fe3b7400640 1 --2- 172.21.15.86:0/455071378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb3304f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.366+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/455071378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb33061c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.367+0000 7fe3b7400640 1 -- 172.21.15.86:0/455071378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 msgr2=0x55cb3304f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.367+0000 7fe3b7400640 1 --2- 172.21.15.86:0/455071378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb3304f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.367+0000 7fe3b7400640 1 -- 172.21.15.86:0/455071378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 msgr2=0x55cb33061c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.367+0000 7fe3b7400640 1 --2- 172.21.15.86:0/455071378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb33061c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.367+0000 7fe3b7400640 1 -- 172.21.15.86:0/455071378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb32cc83f0 con 0x55cb33050160 2024-04-17T07:26:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.367+0000 7fe3b7400640 1 --2- 172.21.15.86:0/455071378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305ab70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006010 tx=0x7fe3a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=924ed9ce2441c12a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.368+0000 7fe3acc00640 1 -- 172.21.15.86:0/455071378 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a401e670 con 0x55cb33050160 2024-04-17T07:26:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.368+0000 7fe3acc00640 1 -- 172.21.15.86:0/455071378 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3a402e070 con 0x55cb33050160 2024-04-17T07:26:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.368+0000 7fe3acc00640 1 -- 172.21.15.86:0/455071378 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4028d60 con 0x55cb33050160 2024-04-17T07:26:11.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.370+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/455071378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 msgr2=0x55cb3305ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:11.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.370+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/455071378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305ab70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006010 tx=0x7fe3a40046c0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.371+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/455071378 shutdown_connections 2024-04-17T07:26:11.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.371+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/455071378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33055cc0 0x55cb3304f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.371+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/455071378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050b50 0x55cb33061c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.371+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/455071378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33050160 0x55cb3305ab70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.371+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/455071378 >> 172.21.15.86:0/455071378 conn(0x55cb3304b6d0 msgr2=0x55cb3304e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:11.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.372+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/455071378 shutdown_connections 2024-04-17T07:26:11.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.372+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/455071378 wait complete. 2024-04-17T07:26:11.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.374+0000 7fe3c1e4e740 1 -- start start 2024-04-17T07:26:11.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.374+0000 7fe3c1e4e740 1 Processor -- start 2024-04-17T07:26:11.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.376+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050160 0x55cb3305f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.376+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33050b50 0x55cb3305f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.376+0000 7fe3c1e4e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33055cc0 0x55cb33060740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.376+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b81bac40 con 0x55cb33050160 2024-04-17T07:26:11.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.376+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b81ba570 con 0x55cb33050b50 2024-04-17T07:26:11.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.376+0000 7fe3c1e4e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b8073e10 con 0x55cb33055cc0 2024-04-17T07:26:11.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.377+0000 7fe3c1e4e740 10 client.0 _ll_register_callbacks cb 0x7fe3a4013360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:11.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.378+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33050b50 0x55cb3305f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.378+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33050b50 0x55cb3305f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37422/0 (socket says 172.21.15.86:37422) 2024-04-17T07:26:11.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.378+0000 7fe3b7e00640 1 -- 172.21.15.86:0/944511493 learned_addr learned my addr 172.21.15.86:0/944511493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:11.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.378+0000 7fe3b7400640 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050160 0x55cb3305f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.379+0000 7fe3b7e00640 1 -- 172.21.15.86:0/944511493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33055cc0 msgr2=0x55cb33060740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:26:11.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.379+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33055cc0 0x55cb33060740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.379+0000 7fe3b7e00640 1 -- 172.21.15.86:0/944511493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050160 msgr2=0x55cb3305f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.379+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050160 0x55cb3305f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.379+0000 7fe3b7e00640 1 -- 172.21.15.86:0/944511493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb32d927d0 con 0x55cb33050b50 2024-04-17T07:26:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.380+0000 7fe3b7e00640 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33050b50 0x55cb3305f730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8005ab0 tx=0x7fe3a80092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5acddd3c17b94cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.380+0000 7fe3a2000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:26:11.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.380+0000 7fe3a2000640 1 -- 172.21.15.86:0/944511493 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8045040 con 0x55cb33050b50 2024-04-17T07:26:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.380+0000 7fe3a2000640 1 -- 172.21.15.86:0/944511493 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fe3a8004240 con 0x55cb33050b50 2024-04-17T07:26:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.380+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/944511493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3a402e070 con 0x55cb33050b50 2024-04-17T07:26:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.383+0000 7fe3a2000640 1 -- 172.21.15.86:0/944511493 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3a8004440 con 0x55cb33050b50 2024-04-17T07:26:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.384+0000 7fe3c1e4e740 10 client.37211 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.384+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/944511493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cb33061e60 con 0x55cb33050b50 2024-04-17T07:26:11.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.385+0000 7fe3a2000640 1 -- 172.21.15.86:0/944511493 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3a402e070 con 0x55cb33050b50 2024-04-17T07:26:11.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.385+0000 7fe3a2000640 1 -- 172.21.15.86:0/944511493 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fe3a8041620 con 0x55cb33050b50 2024-04-17T07:26:11.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.385+0000 7fe3c1e4e740 10 client.37211 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:11.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.385+0000 7fe3c1e4e740 10 client.37211 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:26:11.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.385+0000 7fe3c1e4e740 20 client.37211 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:11.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.385+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cb330dd920 0x55cb330dfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:11.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.385+0000 7fe3c1e4e740 4 client.37211 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:26:11.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.385+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/944511493 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb32fbd8b0 con 0x55cb330dd920 2024-04-17T07:26:11.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.388+0000 7fe3b7400640 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cb330dd920 0x55cb330dfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.389+0000 7fe3b7400640 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cb330dd920 0x55cb330dfd10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b27efa3b92c38bfb server_cookie=7d9ad8398b69f7da in_seq=0 out_seq=0 2024-04-17T07:26:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.389+0000 7fe3a2000640 10 client.37211 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:26:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.389+0000 7fe3a2000640 1 -- 172.21.15.86:0/944511493 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb32fbd8b0 con 0x55cb330dd920 2024-04-17T07:26:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.389+0000 7fe3a2000640 10 client.37211 handle_command_reply: tid=0 2024-04-17T07:26:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.389+0000 7fe3c1e4e740 1 client.37211 shutdown 2024-04-17T07:26:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/944511493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33050b50 msgr2=0x55cb3305f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:11.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33050b50 0x55cb3305f730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8005ab0 tx=0x7fe3a80092d0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/944511493 shutdown_connections 2024-04-17T07:26:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55cb330dd920 0x55cb330dfd10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb33055cc0 0x55cb33060740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb33050b50 0x55cb3305f730 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3c1e4e740 1 --2- 172.21.15.86:0/944511493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cb33050160 0x55cb3305f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/944511493 >> 172.21.15.86:0/944511493 conn(0x55cb3304b6d0 msgr2=0x55cb3304f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3a2000640 0 client.37211 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:26:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/944511493 shutdown_connections 2024-04-17T07:26:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.390+0000 7fe3c1e4e740 1 -- 172.21.15.86:0/944511493 wait complete. 2024-04-17T07:26:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.391+0000 7fe3c1e4e740 20 client.37211 trim_cache size 0 max 16384 2024-04-17T07:26:11.392 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:26:11.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.392+0000 7fe3bdc00640 1 -- 172.21.15.86:0/3424529838 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe39c0a3c30 msgr2=0x7fe39c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:11.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.392+0000 7fe3bdc00640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe39c0a3c30 0x7fe39c0a60f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe3b009ca80 tx=0x7fe3b000c950 comp rx=0 tx=0).stop 2024-04-17T07:26:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.392+0000 7fe3bdc00640 1 -- 172.21.15.86:0/3424529838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b81039a0 msgr2=0x7fe3b81a5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.392+0000 7fe3bdc00640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b81039a0 0x7fe3b81a5ca0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000eb10 tx=0x7fe3b000c2d0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.393+0000 7fe3bdc00640 1 -- 172.21.15.86:0/3424529838 shutdown_connections 2024-04-17T07:26:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.393+0000 7fe3bdc00640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fe39c0a3c30 0x7fe39c0a60f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.393+0000 7fe3bdc00640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b81039a0 0x7fe3b81a5ca0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.393+0000 7fe3bdc00640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe3b8102fe0 0x7fe3b819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.393+0000 7fe3bdc00640 1 --2- 172.21.15.86:0/3424529838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b8101e90 0x7fe3b819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:11.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.393+0000 7fe3bdc00640 1 -- 172.21.15.86:0/3424529838 >> 172.21.15.86:0/3424529838 conn(0x7fe3b80fd400 msgr2=0x7fe3b810e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:11.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.393+0000 7fe3bdc00640 1 -- 172.21.15.86:0/3424529838 shutdown_connections 2024-04-17T07:26:11.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:11.393+0000 7fe3bdc00640 1 -- 172.21.15.86:0/3424529838 wait complete. 2024-04-17T07:26:11.433 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:26:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:11 smithi098 ceph-mon[35418]: pgmap v794: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.3 MiB/s wr, 612 op/s 2024-04-17T07:26:11.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:11 smithi103 ceph-mon[35495]: pgmap v794: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.3 MiB/s wr, 612 op/s 2024-04-17T07:26:12.033 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:11 smithi086 ceph-mon[31733]: pgmap v794: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.3 MiB/s wr, 612 op/s 2024-04-17T07:26:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.073+0000 7f3fe2a00640 1 -- 172.21.15.86:0/2061748212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4034d80 con 0x7f3fe4077b60 2024-04-17T07:26:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.074+0000 7f3feae00640 1 -- 172.21.15.86:0/2061748212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe4077b60 msgr2=0x7f3fe4077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.074+0000 7f3feae00640 1 --2- 172.21.15.86:0/2061748212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe4077b60 0x7f3fe4077fc0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009860 tx=0x7f3fd402bb70 comp rx=0 tx=0).stop 2024-04-17T07:26:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.074+0000 7f3feae00640 1 -- 172.21.15.86:0/2061748212 shutdown_connections 2024-04-17T07:26:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.074+0000 7f3feae00640 1 --2- 172.21.15.86:0/2061748212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fe4078500 0x7f3fe4078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.074+0000 7f3feae00640 1 --2- 172.21.15.86:0/2061748212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe4077b60 0x7f3fe4077fc0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.074+0000 7f3feae00640 1 --2- 172.21.15.86:0/2061748212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fe4076920 0x7f3fe4076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.074+0000 7f3feae00640 1 -- 172.21.15.86:0/2061748212 >> 172.21.15.86:0/2061748212 conn(0x7f3fe40ff750 msgr2=0x7f3fe4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.075+0000 7f3feae00640 1 -- 172.21.15.86:0/2061748212 shutdown_connections 2024-04-17T07:26:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.075+0000 7f3feae00640 1 -- 172.21.15.86:0/2061748212 wait complete. 2024-04-17T07:26:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.107+0000 7f3feae00640 1 Processor -- start 2024-04-17T07:26:12.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.108+0000 7f3feae00640 1 -- start start 2024-04-17T07:26:12.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.108+0000 7f3feae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe4076920 0x7f3fe419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.109+0000 7f3feae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fe4077b60 0x7f3fe419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.109+0000 7f3feae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fe4078500 0x7f3fe41a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.109+0000 7f3feae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe4114180 con 0x7f3fe4076920 2024-04-17T07:26:12.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.109+0000 7f3feae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe4114000 con 0x7f3fe4078500 2024-04-17T07:26:12.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.109+0000 7f3feae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe4114300 con 0x7f3fe4077b60 2024-04-17T07:26:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.109+0000 7f3fe3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fe4077b60 0x7f3fe419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.110+0000 7f3fe3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fe4077b60 0x7f3fe419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44536/0 (socket says 172.21.15.86:44536) 2024-04-17T07:26:12.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.110+0000 7f3fe3400640 1 -- 172.21.15.86:0/1554899319 learned_addr learned my addr 172.21.15.86:0/1554899319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.110+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe4076920 0x7f3fe419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.110+0000 7f3fe3400640 1 -- 172.21.15.86:0/1554899319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fe4078500 msgr2=0x7f3fe41a5e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T07:26:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.110+0000 7f3fe3400640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fe4078500 0x7f3fe41a5e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.110+0000 7f3fe3400640 1 -- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe4076920 msgr2=0x7f3fe419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.110+0000 7f3fe3400640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe4076920 0x7f3fe419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.110+0000 7f3fe3400640 1 -- 172.21.15.86:0/1554899319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe41a6470 con 0x7f3fe4077b60 2024-04-17T07:26:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.110+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe4076920 0x7f3fe419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:12.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.111+0000 7f3fe3400640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fe4077b60 0x7f3fe419ed90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4005940 tx=0x7f3fd4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:12.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.112+0000 7f3fe0c00640 1 -- 172.21.15.86:0/1554899319 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4041020 con 0x7f3fe4077b60 2024-04-17T07:26:12.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.112+0000 7f3fe0c00640 1 -- 172.21.15.86:0/1554899319 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fd402c9c0 con 0x7f3fe4077b60 2024-04-17T07:26:12.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.112+0000 7f3fe0c00640 1 -- 172.21.15.86:0/1554899319 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4039040 con 0x7f3fe4077b60 2024-04-17T07:26:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.112+0000 7f3feae00640 1 -- 172.21.15.86:0/1554899319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe41a6700 con 0x7f3fe4077b60 2024-04-17T07:26:12.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.113+0000 7f3feae00640 1 -- 172.21.15.86:0/1554899319 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe41a6c20 con 0x7f3fe4077b60 2024-04-17T07:26:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.116+0000 7f3feda75740 1 Processor -- start 2024-04-17T07:26:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.117+0000 7f3feda75740 1 -- start start 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.118+0000 7f3feda75740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcf8fa0 0x55995fcf93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.118+0000 7f3feda75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 0x55995fcfe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.118+0000 7f3feda75740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcfeb00 0x55995fd00f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.118+0000 7f3feda75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55995fc5f020 con 0x55995fcf9990 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.118+0000 7f3feda75740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55995fc05460 con 0x55995fcfeb00 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.118+0000 7f3feda75740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55995fc5afe0 con 0x55995fcf8fa0 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.119+0000 7f3fe8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcf8fa0 0x55995fcf93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.119+0000 7f3fe8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcf8fa0 0x55995fcf93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44542/0 (socket says 172.21.15.86:44542) 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.119+0000 7f3fe8e00640 1 -- 172.21.15.86:0/496548125 learned_addr learned my addr 172.21.15.86:0/496548125 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.120+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/496548125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 0x55995fcfe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.120+0000 7f3fe8e00640 1 -- 172.21.15.86:0/496548125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcfeb00 msgr2=0x55995fd00f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:26:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.120+0000 7f3fe8e00640 1 --2- 172.21.15.86:0/496548125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcfeb00 0x55995fd00f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.120+0000 7f3fe8e00640 1 -- 172.21.15.86:0/496548125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 msgr2=0x55995fcfe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.120+0000 7f3fe8e00640 1 --2- 172.21.15.86:0/496548125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 0x55995fcfe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.120+0000 7f3fe8e00640 1 -- 172.21.15.86:0/496548125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55995fc0d040 con 0x55995fcf8fa0 2024-04-17T07:26:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.120+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/496548125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 0x55995fcfe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.121+0000 7f3fe0c00640 1 -- 172.21.15.86:0/1554899319 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3fd4044020 con 0x7f3fe4077b60 2024-04-17T07:26:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.121+0000 7f3fe8e00640 1 --2- 172.21.15.86:0/496548125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcf8fa0 0x55995fcf93c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009d00 tx=0x7f3fdc00b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a020d4c89662cce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:12.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.121+0000 7f3fd9600640 1 -- 172.21.15.86:0/496548125 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00bd40 con 0x55995fcf8fa0 2024-04-17T07:26:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.124+0000 7f3fd9600640 1 -- 172.21.15.86:0/496548125 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fdc017070 con 0x55995fcf8fa0 2024-04-17T07:26:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.124+0000 7f3fd9600640 1 -- 172.21.15.86:0/496548125 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0115d0 con 0x55995fcf8fa0 2024-04-17T07:26:12.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.124+0000 7f3fe0c00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fc80a3cb0 0x7f3fc80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.124+0000 7f3fe0c00640 1 -- 172.21.15.86:0/1554899319 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3fd40f2220 con 0x7f3fe4077b60 2024-04-17T07:26:12.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.124+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fc80a3cb0 0x7f3fc80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.128+0000 7f3feda75740 1 -- 172.21.15.86:0/496548125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcf8fa0 msgr2=0x55995fcf93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.128+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fc80a3cb0 0x7f3fc80a6170 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x55995fcf9d90 tx=0x7f3fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.128+0000 7f3feda75740 1 --2- 172.21.15.86:0/496548125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcf8fa0 0x55995fcf93c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009d00 tx=0x7f3fdc00b420 comp rx=0 tx=0).stop 2024-04-17T07:26:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.128+0000 7f3feda75740 1 -- 172.21.15.86:0/496548125 shutdown_connections 2024-04-17T07:26:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.128+0000 7f3feda75740 1 --2- 172.21.15.86:0/496548125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcfeb00 0x55995fd00f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.128+0000 7f3feda75740 1 --2- 172.21.15.86:0/496548125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 0x55995fcfe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.128+0000 7f3feda75740 1 --2- 172.21.15.86:0/496548125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcf8fa0 0x55995fcf93c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.128+0000 7f3feda75740 1 -- 172.21.15.86:0/496548125 >> 172.21.15.86:0/496548125 conn(0x55995fcf4510 msgr2=0x55995fcf4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:12.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.128+0000 7f3feda75740 1 -- 172.21.15.86:0/496548125 shutdown_connections 2024-04-17T07:26:12.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.128+0000 7f3feda75740 1 -- 172.21.15.86:0/496548125 wait complete. 2024-04-17T07:26:12.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.129+0000 7f3feda75740 1 -- start start 2024-04-17T07:26:12.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.129+0000 7f3feda75740 1 Processor -- start 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.130+0000 7f3feda75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd03580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.130+0000 7f3feda75740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 0x55995fd03ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.130+0000 7f3feda75740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd0ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.130+0000 7f3feda75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55995fd01ab0 con 0x55995fcf8fa0 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.130+0000 7f3feda75740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55995fc18830 con 0x55995fcf9990 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.130+0000 7f3feda75740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55995fd01c30 con 0x55995fcfeb00 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.130+0000 7f3feda75740 10 client.0 _ll_register_callbacks cb 0x55995fbe6fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.131+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd03580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.131+0000 7f3fe3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 0x55995fd03ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.131+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd03580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52256/0 (socket says 172.21.15.86:52256) 2024-04-17T07:26:12.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.131+0000 7f3fe3e00640 1 -- 172.21.15.86:0/949902245 learned_addr learned my addr 172.21.15.86:0/949902245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:12.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.131+0000 7f3fe8e00640 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd0ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.131+0000 7f3fe3e00640 1 -- 172.21.15.86:0/949902245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 msgr2=0x55995fd0ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:12.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.131+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd0ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.131+0000 7f3fe3e00640 1 -- 172.21.15.86:0/949902245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 msgr2=0x55995fd03ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:12.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.131+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 0x55995fd03ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.131+0000 7f3fe3e00640 1 -- 172.21.15.86:0/949902245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55995fc1c920 con 0x55995fcf8fa0 2024-04-17T07:26:12.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.132+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd03580 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0012930 tx=0x7f3fd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafaad0bf3068a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:12.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.132+0000 7f3fce000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:12.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.132+0000 7f3fce000640 1 -- 172.21.15.86:0/949902245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0022070 con 0x55995fcf8fa0 2024-04-17T07:26:12.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.132+0000 7f3fce000640 1 -- 172.21.15.86:0/949902245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3fd000fce0 con 0x55995fcf8fa0 2024-04-17T07:26:12.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.133+0000 7f3feda75740 1 -- 172.21.15.86:0/949902245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fdc017070 con 0x55995fcf8fa0 2024-04-17T07:26:12.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.133+0000 7f3fce000640 1 -- 172.21.15.86:0/949902245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fd0027460 con 0x55995fcf8fa0 2024-04-17T07:26:12.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.134+0000 7f3feda75740 10 client.18342 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:12.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.134+0000 7f3feda75740 1 -- 172.21.15.86:0/949902245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55995fbea7f0 con 0x55995fcf8fa0 2024-04-17T07:26:12.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.134+0000 7f3fce000640 1 -- 172.21.15.86:0/949902245 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fdc017070 con 0x55995fcf8fa0 2024-04-17T07:26:12.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.134+0000 7f3fce000640 1 -- 172.21.15.86:0/949902245 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3fd0031540 con 0x55995fcf8fa0 2024-04-17T07:26:12.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.135+0000 7f3feda75740 10 client.18342 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:12.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.135+0000 7f3feda75740 10 client.18342 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:26:12.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.135+0000 7f3feda75740 20 client.18342 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:12.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.135+0000 7f3feda75740 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55995fd0f3e0 0x55995fd11860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.135+0000 7f3feda75740 4 client.18342 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:12.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.135+0000 7f3feda75740 1 -- 172.21.15.86:0/949902245 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55995fc6ca40 con 0x55995fd0f3e0 2024-04-17T07:26:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.136+0000 7f3fe3400640 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55995fd0f3e0 0x55995fd11860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.137+0000 7f3fe3400640 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55995fd0f3e0 0x55995fd11860 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e407e0d0f255ab74 server_cookie=afa6511de1988e09 in_seq=0 out_seq=0 2024-04-17T07:26:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.137+0000 7f3fce000640 10 client.18342 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:26:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.139+0000 7f3fce000640 1 -- 172.21.15.86:0/949902245 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55995fc6ca40 con 0x55995fd0f3e0 2024-04-17T07:26:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.139+0000 7f3fce000640 10 client.18342 handle_command_reply: tid=0 2024-04-17T07:26:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.140+0000 7f3feda75740 1 client.18342 shutdown 2024-04-17T07:26:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.141+0000 7f3feda75740 1 -- 172.21.15.86:0/949902245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 msgr2=0x55995fd03580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.141+0000 7f3feda75740 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd03580 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0012930 tx=0x7f3fd00046c0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.142+0000 7f3feda75740 1 -- 172.21.15.86:0/949902245 shutdown_connections 2024-04-17T07:26:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.142+0000 7f3feda75740 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55995fd0f3e0 0x55995fd11860 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.142+0000 7f3feda75740 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd0ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.142+0000 7f3fce000640 0 client.18342 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:26:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.142+0000 7f3feda75740 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 0x55995fd03ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.142+0000 7f3feda75740 1 --2- 172.21.15.86:0/949902245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd03580 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.142+0000 7f3feda75740 1 -- 172.21.15.86:0/949902245 >> 172.21.15.86:0/949902245 conn(0x55995fcf4510 msgr2=0x55995fcf3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.143+0000 7f3feda75740 1 -- 172.21.15.86:0/949902245 shutdown_connections 2024-04-17T07:26:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.143+0000 7f3feda75740 1 -- 172.21.15.86:0/949902245 wait complete. 2024-04-17T07:26:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.144+0000 7f3feda75740 20 client.18342 trim_cache size 0 max 16384 2024-04-17T07:26:12.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.185+0000 7f3feda75740 1 Processor -- start 2024-04-17T07:26:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.185+0000 7f3feda75740 1 -- start start 2024-04-17T07:26:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.186+0000 7f3feda75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd02000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.187+0000 7f3feda75740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 0x55995fd0aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.187+0000 7f3feda75740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd03190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.187+0000 7f3feda75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe41b1300 con 0x55995fcf8fa0 2024-04-17T07:26:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.187+0000 7f3feda75740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe41b2e40 con 0x55995fcf9990 2024-04-17T07:26:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.187+0000 7f3feda75740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe41b19d0 con 0x55995fcfeb00 2024-04-17T07:26:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.188+0000 7f3fe3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 0x55995fd0aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.188+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd02000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.188+0000 7f3fe8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd03190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.188+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd02000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52260/0 (socket says 172.21.15.86:52260) 2024-04-17T07:26:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.188+0000 7f3fe8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd03190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44566/0 (socket says 172.21.15.86:44566) 2024-04-17T07:26:12.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.188+0000 7f3fe3e00640 1 -- 172.21.15.86:0/2381524537 learned_addr learned my addr 172.21.15.86:0/2381524537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.189+0000 7f3fe3e00640 1 -- 172.21.15.86:0/2381524537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 msgr2=0x55995fd03190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.189+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/2381524537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd03190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.189+0000 7f3fe3e00640 1 -- 172.21.15.86:0/2381524537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 msgr2=0x55995fd0aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.189+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/2381524537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 0x55995fd0aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.189+0000 7f3fe3e00640 1 -- 172.21.15.86:0/2381524537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55995f9713f0 con 0x55995fcf8fa0 2024-04-17T07:26:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.189+0000 7f3fe3400640 1 --2- 172.21.15.86:0/2381524537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 0x55995fd0aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.190+0000 7f3fe8e00640 1 --2- 172.21.15.86:0/2381524537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd03190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.191+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/2381524537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd02000 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000a010 tx=0x7f3fd0031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cdf0744595314c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.192+0000 7f3fd8c00640 1 -- 172.21.15.86:0/2381524537 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0036070 con 0x55995fcf8fa0 2024-04-17T07:26:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.192+0000 7f3fd8c00640 1 -- 172.21.15.86:0/2381524537 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fd0031b20 con 0x55995fcf8fa0 2024-04-17T07:26:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.192+0000 7f3fd8c00640 1 -- 172.21.15.86:0/2381524537 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0022040 con 0x55995fcf8fa0 2024-04-17T07:26:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.195+0000 7f3feda75740 1 -- 172.21.15.86:0/2381524537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 msgr2=0x55995fd02000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.195+0000 7f3feda75740 1 --2- 172.21.15.86:0/2381524537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd02000 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000a010 tx=0x7f3fd0031ef0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.195+0000 7f3feda75740 1 -- 172.21.15.86:0/2381524537 shutdown_connections 2024-04-17T07:26:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.195+0000 7f3feda75740 1 --2- 172.21.15.86:0/2381524537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd03190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.195+0000 7f3feda75740 1 --2- 172.21.15.86:0/2381524537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf9990 0x55995fd0aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.195+0000 7f3feda75740 1 --2- 172.21.15.86:0/2381524537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf8fa0 0x55995fd02000 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.195+0000 7f3feda75740 1 -- 172.21.15.86:0/2381524537 >> 172.21.15.86:0/2381524537 conn(0x55995fcf4510 msgr2=0x55995fcfaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.196+0000 7f3feda75740 1 -- 172.21.15.86:0/2381524537 shutdown_connections 2024-04-17T07:26:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.196+0000 7f3feda75740 1 -- 172.21.15.86:0/2381524537 wait complete. 2024-04-17T07:26:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.196+0000 7f3feda75740 1 -- start start 2024-04-17T07:26:12.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.197+0000 7f3feda75740 1 Processor -- start 2024-04-17T07:26:12.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.197+0000 7f3feda75740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf8fa0 0x55995fd07cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.197+0000 7f3feda75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 0x55995fd08210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.198+0000 7f3feda75740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd09160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.198+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf8fa0 0x55995fd07cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.198+0000 7f3feda75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe41b2770 con 0x55995fcf9990 2024-04-17T07:26:12.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.198+0000 7f3feda75740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe41b20a0 con 0x55995fcf8fa0 2024-04-17T07:26:12.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.198+0000 7f3feda75740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe41af2d0 con 0x55995fcfeb00 2024-04-17T07:26:12.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.198+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf8fa0 0x55995fd07cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37484/0 (socket says 172.21.15.86:37484) 2024-04-17T07:26:12.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.198+0000 7f3fe3e00640 1 -- 172.21.15.86:0/3353575755 learned_addr learned my addr 172.21.15.86:0/3353575755 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:12.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.198+0000 7f3feda75740 10 client.0 _ll_register_callbacks cb 0x7f3fd00125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:12.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.198+0000 7f3fe3400640 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 0x55995fd08210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.199+0000 7f3fe8e00640 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd09160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.199+0000 7f3fe3400640 1 -- 172.21.15.86:0/3353575755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 msgr2=0x55995fd09160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:12.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.199+0000 7f3fe3400640 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd09160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.199+0000 7f3fe3400640 1 -- 172.21.15.86:0/3353575755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf8fa0 msgr2=0x55995fd07cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:12.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.199+0000 7f3fe3400640 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf8fa0 0x55995fd07cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.199+0000 7f3fe3400640 1 -- 172.21.15.86:0/3353575755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55995fa3b7d0 con 0x55995fcf9990 2024-04-17T07:26:12.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.200+0000 7f3fe3400640 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 0x55995fd08210 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40ef940 tx=0x7f3fd402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d93fd83c79ff36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:12.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.200+0000 7f3fce000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:12.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.201+0000 7f3fce000640 1 -- 172.21.15.86:0/3353575755 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40c24d0 con 0x55995fcf9990 2024-04-17T07:26:12.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.201+0000 7f3fce000640 1 -- 172.21.15.86:0/3353575755 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3fd40bc420 con 0x55995fcf9990 2024-04-17T07:26:12.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.201+0000 7f3feda75740 1 -- 172.21.15.86:0/3353575755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fd0031b20 con 0x55995fcf9990 2024-04-17T07:26:12.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.201+0000 7f3fce000640 1 -- 172.21.15.86:0/3353575755 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fd40dd080 con 0x55995fcf9990 2024-04-17T07:26:12.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.201+0000 7f3feda75740 10 client.18348 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:12.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.201+0000 7f3feda75740 1 -- 172.21.15.86:0/3353575755 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55995fd0a950 con 0x55995fcf9990 2024-04-17T07:26:12.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.202+0000 7f3fce000640 1 -- 172.21.15.86:0/3353575755 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fd0031b20 con 0x55995fcf9990 2024-04-17T07:26:12.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.202+0000 7f3fce000640 1 -- 172.21.15.86:0/3353575755 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3fd403c030 con 0x55995fcf9990 2024-04-17T07:26:12.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.202+0000 7f3feda75740 10 client.18348 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:12.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.202+0000 7f3feda75740 10 client.18348 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:26:12.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.202+0000 7f3feda75740 20 client.18348 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:12.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.202+0000 7f3feda75740 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55995fd86ce0 0x55995fd890d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:12.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.202+0000 7f3feda75740 4 client.18348 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:26:12.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.202+0000 7f3feda75740 1 -- 172.21.15.86:0/3353575755 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55995fc668b0 con 0x55995fd86ce0 2024-04-17T07:26:12.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.203+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55995fd86ce0 0x55995fd890d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:12.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.205+0000 7f3fe3e00640 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55995fd86ce0 0x55995fd890d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5bd64a6d3408f0f4 server_cookie=668467a2e34fcddf in_seq=0 out_seq=0 2024-04-17T07:26:12.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.205+0000 7f3fce000640 10 client.18348 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:26:12.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.207+0000 7f3fce000640 1 -- 172.21.15.86:0/3353575755 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55995fc668b0 con 0x55995fd86ce0 2024-04-17T07:26:12.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.207+0000 7f3fce000640 10 client.18348 handle_command_reply: tid=0 2024-04-17T07:26:12.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.207+0000 7f3feda75740 1 client.18348 shutdown 2024-04-17T07:26:12.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.207+0000 7f3feda75740 1 -- 172.21.15.86:0/3353575755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 msgr2=0x55995fd08210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:12.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.207+0000 7f3feda75740 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 0x55995fd08210 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40ef940 tx=0x7f3fd402e040 comp rx=0 tx=0).stop 2024-04-17T07:26:12.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.208+0000 7f3feda75740 1 -- 172.21.15.86:0/3353575755 shutdown_connections 2024-04-17T07:26:12.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.208+0000 7f3feda75740 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55995fd86ce0 0x55995fd890d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.208+0000 7f3feda75740 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55995fcfeb00 0x55995fd09160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.208+0000 7f3feda75740 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55995fcf9990 0x55995fd08210 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.208+0000 7f3feda75740 1 --2- 172.21.15.86:0/3353575755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55995fcf8fa0 0x55995fd07cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.208+0000 7f3feda75740 1 -- 172.21.15.86:0/3353575755 >> 172.21.15.86:0/3353575755 conn(0x55995fcf4510 msgr2=0x55995fd03190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:12.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.208+0000 7f3fce000640 0 client.18348 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:26:12.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.210+0000 7f3feda75740 1 -- 172.21.15.86:0/3353575755 shutdown_connections 2024-04-17T07:26:12.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.210+0000 7f3feda75740 1 -- 172.21.15.86:0/3353575755 wait complete. 2024-04-17T07:26:12.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.210+0000 7f3feda75740 20 client.18348 trim_cache size 0 max 16384 2024-04-17T07:26:12.211 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:26:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.212+0000 7f3feae00640 1 -- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fc80a3cb0 msgr2=0x7f3fc80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.212+0000 7f3feae00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fc80a3cb0 0x7f3fc80a6170 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x55995fcf9d90 tx=0x7f3fd0009260 comp rx=0 tx=0).stop 2024-04-17T07:26:12.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.212+0000 7f3feae00640 1 -- 172.21.15.86:0/1554899319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fe4077b60 msgr2=0x7f3fe419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.212+0000 7f3feae00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fe4077b60 0x7f3fe419ed90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4005940 tx=0x7f3fd4004270 comp rx=0 tx=0).stop 2024-04-17T07:26:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.212+0000 7f3feae00640 1 -- 172.21.15.86:0/1554899319 shutdown_connections 2024-04-17T07:26:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.212+0000 7f3feae00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3fc80a3cb0 0x7f3fc80a6170 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.212+0000 7f3feae00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fe4078500 0x7f3fe41a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.212+0000 7f3feae00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fe4077b60 0x7f3fe419ed90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.212+0000 7f3feae00640 1 --2- 172.21.15.86:0/1554899319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fe4076920 0x7f3fe419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.212+0000 7f3feae00640 1 -- 172.21.15.86:0/1554899319 >> 172.21.15.86:0/1554899319 conn(0x7f3fe40ff750 msgr2=0x7f3fe4101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.213+0000 7f3feae00640 1 -- 172.21.15.86:0/1554899319 shutdown_connections 2024-04-17T07:26:12.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:12.214+0000 7f3feae00640 1 -- 172.21.15.86:0/1554899319 wait complete. 2024-04-17T07:26:12.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:26:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.129+0000 7f3763e00640 1 -- 172.21.15.86:0/3830795806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c106a60 msgr2=0x7f375c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.129+0000 7f3763e00640 1 --2- 172.21.15.86:0/3830795806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c106a60 0x7f375c108f20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f374c009860 tx=0x7f374c02bb70 comp rx=0 tx=0).stop 2024-04-17T07:26:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.131+0000 7f3763e00640 1 -- 172.21.15.86:0/3830795806 shutdown_connections 2024-04-17T07:26:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.131+0000 7f3763e00640 1 --2- 172.21.15.86:0/3830795806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c106a60 0x7f375c108f20 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.131+0000 7f3763e00640 1 --2- 172.21.15.86:0/3830795806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375c104050 0x7f375c106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.131+0000 7f3763e00640 1 --2- 172.21.15.86:0/3830795806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f375c101720 0x7f375c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.131+0000 7f3763e00640 1 -- 172.21.15.86:0/3830795806 >> 172.21.15.86:0/3830795806 conn(0x7f375c0fb760 msgr2=0x7f375c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.132+0000 7f3763e00640 1 -- 172.21.15.86:0/3830795806 shutdown_connections 2024-04-17T07:26:13.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.132+0000 7f3763e00640 1 -- 172.21.15.86:0/3830795806 wait complete. 2024-04-17T07:26:13.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.134+0000 7f3763e00640 1 Processor -- start 2024-04-17T07:26:13.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.135+0000 7f3763e00640 1 -- start start 2024-04-17T07:26:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.138+0000 7f3763e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c101720 0x7f375c196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.138+0000 7f3763e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375c104050 0x7f375c196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.138+0000 7f3763e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f375c106a60 0x7f375c19d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.138+0000 7f3763e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f375c10c9e0 con 0x7f375c101720 2024-04-17T07:26:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.138+0000 7f3763e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f375c10c860 con 0x7f375c106a60 2024-04-17T07:26:13.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.138+0000 7f3763e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f375c10cb60 con 0x7f375c104050 2024-04-17T07:26:13.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.139+0000 7f3760a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375c104050 0x7f375c196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.139+0000 7f3760a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375c104050 0x7f375c196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44588/0 (socket says 172.21.15.86:44588) 2024-04-17T07:26:13.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.139+0000 7f3760a00640 1 -- 172.21.15.86:0/3567476250 learned_addr learned my addr 172.21.15.86:0/3567476250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:13.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.139+0000 7f3761e00640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f375c106a60 0x7f375c19d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.139+0000 7f3761400640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c101720 0x7f375c196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.139+0000 7f3760a00640 1 -- 172.21.15.86:0/3567476250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f375c106a60 msgr2=0x7f375c19d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.139+0000 7f3760a00640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f375c106a60 0x7f375c19d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.140+0000 7f3760a00640 1 -- 172.21.15.86:0/3567476250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c101720 msgr2=0x7f375c196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.140+0000 7f3760a00640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c101720 0x7f375c196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.140+0000 7f3760a00640 1 -- 172.21.15.86:0/3567476250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c19dc60 con 0x7f375c104050 2024-04-17T07:26:13.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.141+0000 7f3760a00640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375c104050 0x7f375c196580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f375000e9f0 tx=0x7f375000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.142+0000 7f375a000640 1 -- 172.21.15.86:0/3567476250 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375000ccc0 con 0x7f375c104050 2024-04-17T07:26:13.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.142+0000 7f375a000640 1 -- 172.21.15.86:0/3567476250 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37500054d0 con 0x7f375c104050 2024-04-17T07:26:13.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.142+0000 7f375a000640 1 -- 172.21.15.86:0/3567476250 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750010430 con 0x7f375c104050 2024-04-17T07:26:13.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.142+0000 7f3763e00640 1 -- 172.21.15.86:0/3567476250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c19df50 con 0x7f375c104050 2024-04-17T07:26:13.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.142+0000 7f3763e00640 1 -- 172.21.15.86:0/3567476250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c19e4a0 con 0x7f375c104050 2024-04-17T07:26:13.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.148+0000 7f3766ba9740 1 Processor -- start 2024-04-17T07:26:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.150+0000 7f3766ba9740 1 -- start start 2024-04-17T07:26:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.151+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 0x56083cba73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.151+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.151+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbaef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.151+0000 7f3766ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56083cb0d020 con 0x56083cba6fa0 2024-04-17T07:26:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.151+0000 7f3766ba9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56083cab3460 con 0x56083cbacb00 2024-04-17T07:26:13.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.151+0000 7f3766ba9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56083cb08fe0 con 0x56083cba7990 2024-04-17T07:26:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.151+0000 7f375a000640 1 -- 172.21.15.86:0/3567476250 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f37500046c0 con 0x7f375c104050 2024-04-17T07:26:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.152+0000 7f3761e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 0x56083cba73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.152+0000 7f3761400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.152+0000 7f3761400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44594/0 (socket says 172.21.15.86:44594) 2024-04-17T07:26:13.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.152+0000 7f3761400640 1 -- 172.21.15.86:0/2441750701 learned_addr learned my addr 172.21.15.86:0/2441750701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.152+0000 7f3761e00640 1 -- 172.21.15.86:0/2441750701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 msgr2=0x56083cbac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.152+0000 7f3761e00640 1 --2- 172.21.15.86:0/2441750701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.152+0000 7f3761e00640 1 -- 172.21.15.86:0/2441750701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 msgr2=0x56083cbaef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.152+0000 7f3760a00640 1 --2- 172.21.15.86:0/2441750701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbaef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.153+0000 7f3761e00640 1 --2- 172.21.15.86:0/2441750701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbaef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.153+0000 7f3761e00640 1 -- 172.21.15.86:0/2441750701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083cabb040 con 0x56083cba6fa0 2024-04-17T07:26:13.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.153+0000 7f3761400640 1 --2- 172.21.15.86:0/2441750701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.153+0000 7f3760a00640 1 --2- 172.21.15.86:0/2441750701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbaef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.154+0000 7f375a000640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37440a3db0 0x7f37440a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.154+0000 7f375a000640 1 -- 172.21.15.86:0/3567476250 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f37500d0970 con 0x7f375c104050 2024-04-17T07:26:13.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.154+0000 7f3761400640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37440a3db0 0x7f37440a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.154+0000 7f3761e00640 1 --2- 172.21.15.86:0/2441750701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 0x56083cba73c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f374c02c180 tx=0x7f374c009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d38e731f0c3c6db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.156+0000 7f374a600640 1 -- 172.21.15.86:0/2441750701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c043020 con 0x56083cba6fa0 2024-04-17T07:26:13.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.156+0000 7f374a600640 1 -- 172.21.15.86:0/2441750701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c004040 con 0x56083cba6fa0 2024-04-17T07:26:13.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.156+0000 7f374a600640 1 -- 172.21.15.86:0/2441750701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c043020 con 0x56083cba6fa0 2024-04-17T07:26:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.157+0000 7f3766ba9740 1 -- 172.21.15.86:0/2441750701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 msgr2=0x56083cba73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.157+0000 7f3766ba9740 1 --2- 172.21.15.86:0/2441750701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 0x56083cba73c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f374c02c180 tx=0x7f374c009490 comp rx=0 tx=0).stop 2024-04-17T07:26:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.157+0000 7f3761400640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37440a3db0 0x7f37440a6270 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x56083cba7d90 tx=0x7f3754009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.158+0000 7f3766ba9740 1 -- 172.21.15.86:0/2441750701 shutdown_connections 2024-04-17T07:26:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.158+0000 7f3766ba9740 1 --2- 172.21.15.86:0/2441750701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbaef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.158+0000 7f3766ba9740 1 --2- 172.21.15.86:0/2441750701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.158+0000 7f3766ba9740 1 --2- 172.21.15.86:0/2441750701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 0x56083cba73c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.158+0000 7f3766ba9740 1 -- 172.21.15.86:0/2441750701 >> 172.21.15.86:0/2441750701 conn(0x56083cba2510 msgr2=0x56083cba2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.158+0000 7f3766ba9740 1 -- 172.21.15.86:0/2441750701 shutdown_connections 2024-04-17T07:26:13.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.158+0000 7f3766ba9740 1 -- 172.21.15.86:0/2441750701 wait complete. 2024-04-17T07:26:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.159+0000 7f3766ba9740 1 -- start start 2024-04-17T07:26:13.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.159+0000 7f3766ba9740 1 Processor -- start 2024-04-17T07:26:13.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.160+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 0x56083cbb1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.160+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbb1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.160+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbb8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.160+0000 7f3766ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56083cbafa50 con 0x56083cba6fa0 2024-04-17T07:26:13.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.160+0000 7f3766ba9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56083cac6830 con 0x56083cbacb00 2024-04-17T07:26:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.160+0000 7f3766ba9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56083cbafbd0 con 0x56083cba7990 2024-04-17T07:26:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.161+0000 7f3760a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbb1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.161+0000 7f3766ba9740 10 client.0 _ll_register_callbacks cb 0x56083ca94fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.161+0000 7f3761400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 0x56083cbb1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.161+0000 7f3760a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbb1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44598/0 (socket says 172.21.15.86:44598) 2024-04-17T07:26:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.161+0000 7f3760a00640 1 -- 172.21.15.86:0/3517240196 learned_addr learned my addr 172.21.15.86:0/3517240196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.161+0000 7f3761e00640 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbb8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.162+0000 7f3760a00640 1 -- 172.21.15.86:0/3517240196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 msgr2=0x56083cbb8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.162+0000 7f3760a00640 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbb8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.162+0000 7f3760a00640 1 -- 172.21.15.86:0/3517240196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 msgr2=0x56083cbb1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.162+0000 7f3760a00640 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 0x56083cbb1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.162+0000 7f3760a00640 1 -- 172.21.15.86:0/3517240196 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083caca920 con 0x56083cba7990 2024-04-17T07:26:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.163+0000 7f3760a00640 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbb1850 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f375000e9c0 tx=0x7f375009c1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c63928ea8c8efb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.163+0000 7f373ea00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.163+0000 7f373ea00640 1 -- 172.21.15.86:0/3517240196 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37500d3e60 con 0x56083cba7990 2024-04-17T07:26:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.163+0000 7f373ea00640 1 -- 172.21.15.86:0/3517240196 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f37500a5250 con 0x56083cba7990 2024-04-17T07:26:13.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.163+0000 7f3766ba9740 1 -- 172.21.15.86:0/3517240196 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374c004040 con 0x56083cba7990 2024-04-17T07:26:13.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.164+0000 7f373ea00640 1 -- 172.21.15.86:0/3517240196 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f375009c870 con 0x56083cba7990 2024-04-17T07:26:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.164+0000 7f3766ba9740 10 client.27313 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.164+0000 7f3766ba9740 1 -- 172.21.15.86:0/3517240196 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56083ca987f0 con 0x56083cba7990 2024-04-17T07:26:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.164+0000 7f373ea00640 1 -- 172.21.15.86:0/3517240196 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c004040 con 0x56083cba7990 2024-04-17T07:26:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.164+0000 7f373ea00640 1 -- 172.21.15.86:0/3517240196 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3750005090 con 0x56083cba7990 2024-04-17T07:26:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.165+0000 7f3766ba9740 10 client.27313 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.165+0000 7f3766ba9740 10 client.27313 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:26:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.165+0000 7f3766ba9740 20 client.27313 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.165+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56083cbbcdf0 0x56083cbbf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.165+0000 7f3766ba9740 4 client.27313 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:13.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.165+0000 7f3766ba9740 1 -- 172.21.15.86:0/3517240196 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56083cb1aa40 con 0x56083cbbcdf0 2024-04-17T07:26:13.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.165+0000 7f3761400640 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56083cbbcdf0 0x56083cbbf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.167+0000 7f3761400640 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56083cbbcdf0 0x56083cbbf210 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a9312db512ad765 server_cookie=3f4b86dcf1d8d71b in_seq=0 out_seq=0 2024-04-17T07:26:13.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.167+0000 7f373ea00640 10 client.27313 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:26:13.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.168+0000 7f373ea00640 1 -- 172.21.15.86:0/3517240196 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56083cb1aa40 con 0x56083cbbcdf0 2024-04-17T07:26:13.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.169+0000 7f373ea00640 10 client.27313 handle_command_reply: tid=0 2024-04-17T07:26:13.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.169+0000 7f3766ba9740 1 client.27313 shutdown 2024-04-17T07:26:13.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.169+0000 7f3766ba9740 1 -- 172.21.15.86:0/3517240196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 msgr2=0x56083cbb1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.169+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbb1850 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f375000e9c0 tx=0x7f375009c1d0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.169+0000 7f3766ba9740 1 -- 172.21.15.86:0/3517240196 shutdown_connections 2024-04-17T07:26:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.169+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56083cbbcdf0 0x56083cbbf210 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.170+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbb8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.170+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbb1850 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.170+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3517240196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba6fa0 0x56083cbb1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.170+0000 7f3766ba9740 1 -- 172.21.15.86:0/3517240196 >> 172.21.15.86:0/3517240196 conn(0x56083cba2510 msgr2=0x56083cba1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.170+0000 7f373ea00640 0 client.27313 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:26:13.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.170+0000 7f3766ba9740 1 -- 172.21.15.86:0/3517240196 shutdown_connections 2024-04-17T07:26:13.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.171+0000 7f3766ba9740 1 -- 172.21.15.86:0/3517240196 wait complete. 2024-04-17T07:26:13.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.171+0000 7f3766ba9740 20 client.27313 trim_cache size 0 max 16384 2024-04-17T07:26:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.210+0000 7f3766ba9740 1 Processor -- start 2024-04-17T07:26:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.210+0000 7f3766ba9740 1 -- start start 2024-04-17T07:26:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.210+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cba6fa0 0x56083cba0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.210+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbb8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.211+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cbacb00 0x56083cba13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.211+0000 7f3766ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1b98d0 con 0x56083cbacb00 2024-04-17T07:26:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.211+0000 7f3766ba9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1bb410 con 0x56083cba6fa0 2024-04-17T07:26:13.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.211+0000 7f3766ba9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1b9fa0 con 0x56083cba7990 2024-04-17T07:26:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.211+0000 7f3761e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cbacb00 0x56083cba13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.211+0000 7f3761400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cba6fa0 0x56083cba0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.211+0000 7f3761400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cba6fa0 0x56083cba0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37534/0 (socket says 172.21.15.86:37534) 2024-04-17T07:26:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.211+0000 7f3761400640 1 -- 172.21.15.86:0/2725341121 learned_addr learned my addr 172.21.15.86:0/2725341121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.212+0000 7f3760a00640 1 --2- 172.21.15.86:0/2725341121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbb8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.212+0000 7f3761400640 1 -- 172.21.15.86:0/2725341121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 msgr2=0x56083cbb8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.212+0000 7f3761400640 1 --2- 172.21.15.86:0/2725341121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbb8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.212+0000 7f3761400640 1 -- 172.21.15.86:0/2725341121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cbacb00 msgr2=0x56083cba13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.212+0000 7f3761400640 1 --2- 172.21.15.86:0/2725341121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cbacb00 0x56083cba13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.212+0000 7f3761400640 1 -- 172.21.15.86:0/2725341121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083c81f3f0 con 0x56083cba6fa0 2024-04-17T07:26:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.212+0000 7f3761e00640 1 --2- 172.21.15.86:0/2725341121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cbacb00 0x56083cba13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.214+0000 7f3761400640 1 --2- 172.21.15.86:0/2725341121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cba6fa0 0x56083cba0e80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x56083cba3480 tx=0x7f375401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79e7bde153290cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.214+0000 7f3749c00640 1 -- 172.21.15.86:0/2725341121 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754032030 con 0x56083cba6fa0 2024-04-17T07:26:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.216+0000 7f3749c00640 1 -- 172.21.15.86:0/2725341121 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f375402d5a0 con 0x56083cba6fa0 2024-04-17T07:26:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.216+0000 7f3749c00640 1 -- 172.21.15.86:0/2725341121 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375401d950 con 0x56083cba6fa0 2024-04-17T07:26:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.218+0000 7f3766ba9740 1 -- 172.21.15.86:0/2725341121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cba6fa0 msgr2=0x56083cba0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.218+0000 7f3766ba9740 1 --2- 172.21.15.86:0/2725341121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cba6fa0 0x56083cba0e80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x56083cba3480 tx=0x7f375401b000 comp rx=0 tx=0).stop 2024-04-17T07:26:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.218+0000 7f3766ba9740 1 -- 172.21.15.86:0/2725341121 shutdown_connections 2024-04-17T07:26:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.218+0000 7f3766ba9740 1 --2- 172.21.15.86:0/2725341121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cbacb00 0x56083cba13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.218+0000 7f3766ba9740 1 --2- 172.21.15.86:0/2725341121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba7990 0x56083cbb8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.218+0000 7f3766ba9740 1 --2- 172.21.15.86:0/2725341121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cba6fa0 0x56083cba0e80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.218+0000 7f3766ba9740 1 -- 172.21.15.86:0/2725341121 >> 172.21.15.86:0/2725341121 conn(0x56083cba2510 msgr2=0x56083cba8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.219+0000 7f3766ba9740 1 -- 172.21.15.86:0/2725341121 shutdown_connections 2024-04-17T07:26:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.219+0000 7f3766ba9740 1 -- 172.21.15.86:0/2725341121 wait complete. 2024-04-17T07:26:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.219+0000 7f3766ba9740 1 -- start start 2024-04-17T07:26:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.220+0000 7f3766ba9740 1 Processor -- start 2024-04-17T07:26:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.222+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba6fa0 0x56083cbb5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.222+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba7990 0x56083cbb64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.222+0000 7f3766ba9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbb7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.222+0000 7f3766ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1bad40 con 0x56083cba7990 2024-04-17T07:26:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.222+0000 7f3766ba9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1ba670 con 0x56083cbacb00 2024-04-17T07:26:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.222+0000 7f3766ba9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1b7880 con 0x56083cba6fa0 2024-04-17T07:26:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.222+0000 7f3766ba9740 10 client.0 _ll_register_callbacks cb 0x7f3750098ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.223+0000 7f3761400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba6fa0 0x56083cbb5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.223+0000 7f3761400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba6fa0 0x56083cbb5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44630/0 (socket says 172.21.15.86:44630) 2024-04-17T07:26:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.223+0000 7f3761400640 1 -- 172.21.15.86:0/3821952803 learned_addr learned my addr 172.21.15.86:0/3821952803 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.223+0000 7f3761e00640 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbb7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.223+0000 7f3760a00640 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba7990 0x56083cbb64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.223+0000 7f3761400640 1 -- 172.21.15.86:0/3821952803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 msgr2=0x56083cbb7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.223+0000 7f3761400640 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbb7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.223+0000 7f3761400640 1 -- 172.21.15.86:0/3821952803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba7990 msgr2=0x56083cbb64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.224+0000 7f3761400640 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba7990 0x56083cbb64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.224+0000 7f3761400640 1 -- 172.21.15.86:0/3821952803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083c8e97d0 con 0x56083cba6fa0 2024-04-17T07:26:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.224+0000 7f3760a00640 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba7990 0x56083cbb64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.225+0000 7f3761400640 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba6fa0 0x56083cbb5fa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f37540093c0 tx=0x7f375401d490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dad551dc6ab61f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.225+0000 7f373ea00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.225+0000 7f373ea00640 1 -- 172.21.15.86:0/3821952803 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375403b070 con 0x56083cba6fa0 2024-04-17T07:26:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.225+0000 7f373ea00640 1 -- 172.21.15.86:0/3821952803 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f375402d2f0 con 0x56083cba6fa0 2024-04-17T07:26:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.225+0000 7f3766ba9740 1 -- 172.21.15.86:0/3821952803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375402d5a0 con 0x56083cba6fa0 2024-04-17T07:26:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.226+0000 7f373ea00640 1 -- 172.21.15.86:0/3821952803 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3754045050 con 0x56083cba6fa0 2024-04-17T07:26:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.226+0000 7f3766ba9740 10 client.27316 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.226+0000 7f3766ba9740 1 -- 172.21.15.86:0/3821952803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56083cbb8c10 con 0x56083cba6fa0 2024-04-17T07:26:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.226+0000 7f373ea00640 1 -- 172.21.15.86:0/3821952803 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f375402d5a0 con 0x56083cba6fa0 2024-04-17T07:26:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.227+0000 7f373ea00640 1 -- 172.21.15.86:0/3821952803 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3754005040 con 0x56083cba6fa0 2024-04-17T07:26:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.227+0000 7f3766ba9740 10 client.27316 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.227+0000 7f3766ba9740 10 client.27316 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:26:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.227+0000 7f3766ba9740 20 client.27316 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.227+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56083cc23b00 0x56083cc25f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.227+0000 7f3766ba9740 4 client.27316 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:26:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.227+0000 7f3766ba9740 1 -- 172.21.15.86:0/3821952803 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56083cb148b0 con 0x56083cc23b00 2024-04-17T07:26:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.228+0000 7f3760a00640 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56083cc23b00 0x56083cc25f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.230+0000 7f3760a00640 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56083cc23b00 0x56083cc25f20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e256e9ed66d79fcf server_cookie=7e27f2c4e8c8c598 in_seq=0 out_seq=0 2024-04-17T07:26:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.230+0000 7f373ea00640 10 client.27316 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:26:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f373ea00640 1 -- 172.21.15.86:0/3821952803 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56083cb148b0 con 0x56083cc23b00 2024-04-17T07:26:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f373ea00640 10 client.27316 handle_command_reply: tid=0 2024-04-17T07:26:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f3766ba9740 1 client.27316 shutdown 2024-04-17T07:26:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f3766ba9740 1 -- 172.21.15.86:0/3821952803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba6fa0 msgr2=0x56083cbb5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba6fa0 0x56083cbb5fa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f37540093c0 tx=0x7f375401d490 comp rx=0 tx=0).stop 2024-04-17T07:26:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f3766ba9740 1 -- 172.21.15.86:0/3821952803 shutdown_connections 2024-04-17T07:26:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56083cc23b00 0x56083cc25f20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56083cbacb00 0x56083cbb7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56083cba7990 0x56083cbb64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f3766ba9740 1 --2- 172.21.15.86:0/3821952803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56083cba6fa0 0x56083cbb5fa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f3766ba9740 1 -- 172.21.15.86:0/3821952803 >> 172.21.15.86:0/3821952803 conn(0x56083cba2510 msgr2=0x56083cba5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.231+0000 7f373ea00640 0 client.27316 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:26:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.232+0000 7f3766ba9740 1 -- 172.21.15.86:0/3821952803 shutdown_connections 2024-04-17T07:26:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.232+0000 7f3766ba9740 1 -- 172.21.15.86:0/3821952803 wait complete. 2024-04-17T07:26:13.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.233+0000 7f3766ba9740 20 client.27316 trim_cache size 0 max 16384 2024-04-17T07:26:13.235 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:26:13.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.236+0000 7f3763e00640 1 -- 172.21.15.86:0/3567476250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37440a3db0 msgr2=0x7f37440a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.236+0000 7f3763e00640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37440a3db0 0x7f37440a6270 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x56083cba7d90 tx=0x7f3754009500 comp rx=0 tx=0).stop 2024-04-17T07:26:13.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.236+0000 7f3763e00640 1 -- 172.21.15.86:0/3567476250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375c104050 msgr2=0x7f375c196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.236+0000 7f3763e00640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375c104050 0x7f375c196580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f375000e9f0 tx=0x7f375000eeb0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.236+0000 7f3763e00640 1 -- 172.21.15.86:0/3567476250 shutdown_connections 2024-04-17T07:26:13.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.236+0000 7f3763e00640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f37440a3db0 0x7f37440a6270 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.236+0000 7f3763e00640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f375c106a60 0x7f375c19d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.236+0000 7f3763e00640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f375c104050 0x7f375c196580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.236+0000 7f3763e00640 1 --2- 172.21.15.86:0/3567476250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f375c101720 0x7f375c196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.236+0000 7f3763e00640 1 -- 172.21.15.86:0/3567476250 >> 172.21.15.86:0/3567476250 conn(0x7f375c0fb760 msgr2=0x7f375c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.237+0000 7f3763e00640 1 -- 172.21.15.86:0/3567476250 shutdown_connections 2024-04-17T07:26:13.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.237+0000 7f3763e00640 1 -- 172.21.15.86:0/3567476250 wait complete. 2024-04-17T07:26:13.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:26:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:13 smithi086 ceph-mon[31733]: pgmap v795: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.3 MiB/s wr, 708 op/s 2024-04-17T07:26:13.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.619+0000 7f5f36200640 1 -- 172.21.15.86:0/3421315299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2809c380 msgr2=0x7f5f2809c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.619+0000 7f5f36200640 1 --2- 172.21.15.86:0/3421315299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2809c380 0x7f5f2809c7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009900 tx=0x7f5f2402bc40 comp rx=0 tx=0).stop 2024-04-17T07:26:13.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.620+0000 7f5f36200640 1 -- 172.21.15.86:0/3421315299 shutdown_connections 2024-04-17T07:26:13.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.620+0000 7f5f36200640 1 --2- 172.21.15.86:0/3421315299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f2809df30 0x7f5f280a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.620+0000 7f5f36200640 1 --2- 172.21.15.86:0/3421315299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f2809d570 0x7f5f2809d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.620+0000 7f5f36200640 1 --2- 172.21.15.86:0/3421315299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2809c380 0x7f5f2809c7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.620+0000 7f5f36200640 1 -- 172.21.15.86:0/3421315299 >> 172.21.15.86:0/3421315299 conn(0x7f5f2800b470 msgr2=0x7f5f2800d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.620+0000 7f5f36200640 1 -- 172.21.15.86:0/3421315299 shutdown_connections 2024-04-17T07:26:13.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.620+0000 7f5f36200640 1 -- 172.21.15.86:0/3421315299 wait complete. 2024-04-17T07:26:13.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.622+0000 7f5f36200640 1 Processor -- start 2024-04-17T07:26:13.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.623+0000 7f5f36200640 1 -- start start 2024-04-17T07:26:13.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.623+0000 7f5f36200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f2809c380 0x7f5f281325d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.623+0000 7f5f36200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2809d570 0x7f5f28132b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.623+0000 7f5f36200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f2809df30 0x7f5f28139b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.623+0000 7f5f36200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280a7f10 con 0x7f5f2809c380 2024-04-17T07:26:13.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.623+0000 7f5f36200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280a7d90 con 0x7f5f2809df30 2024-04-17T07:26:13.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.623+0000 7f5f36200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280a8090 con 0x7f5f2809d570 2024-04-17T07:26:13.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.624+0000 7f5f2fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2809d570 0x7f5f28132b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.624+0000 7f5f2fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2809d570 0x7f5f28132b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44648/0 (socket says 172.21.15.86:44648) 2024-04-17T07:26:13.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.624+0000 7f5f2fe00640 1 -- 172.21.15.86:0/688227282 learned_addr learned my addr 172.21.15.86:0/688227282 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:13.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.624+0000 7f5f35800640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f2809df30 0x7f5f28139b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.624+0000 7f5f34e00640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f2809c380 0x7f5f281325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.624+0000 7f5f2fe00640 1 -- 172.21.15.86:0/688227282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f2809df30 msgr2=0x7f5f28139b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.624+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f2809df30 0x7f5f28139b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.625+0000 7f5f2fe00640 1 -- 172.21.15.86:0/688227282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f2809c380 msgr2=0x7f5f281325d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.625+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f2809c380 0x7f5f281325d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.625+0000 7f5f2fe00640 1 -- 172.21.15.86:0/688227282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2813a1f0 con 0x7f5f2809d570 2024-04-17T07:26:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.625+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2809d570 0x7f5f28132b10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00e9f0 tx=0x7f5f1c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.626+0000 7f5f2d600640 1 -- 172.21.15.86:0/688227282 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00cd80 con 0x7f5f2809d570 2024-04-17T07:26:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.626+0000 7f5f2d600640 1 -- 172.21.15.86:0/688227282 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1c0054d0 con 0x7f5f2809d570 2024-04-17T07:26:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.626+0000 7f5f36200640 1 -- 172.21.15.86:0/688227282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2813a4e0 con 0x7f5f2809d570 2024-04-17T07:26:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.626+0000 7f5f2d600640 1 -- 172.21.15.86:0/688227282 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c010430 con 0x7f5f2809d570 2024-04-17T07:26:13.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.626+0000 7f5f36200640 1 -- 172.21.15.86:0/688227282 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2813aa30 con 0x7f5f2809d570 2024-04-17T07:26:13.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.631+0000 7f5f3a1fd740 1 Processor -- start 2024-04-17T07:26:13.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.634+0000 7f5f3a1fd740 1 -- start start 2024-04-17T07:26:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.635+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 0x558fa48b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.635+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 0x558fa48b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.635+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b6cc0 0x558fa48b9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.635+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fa4817020 con 0x558fa48b1b50 2024-04-17T07:26:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.635+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558fa47bd460 con 0x558fa48b6cc0 2024-04-17T07:26:13.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.635+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fa4812fe0 con 0x558fa48b1160 2024-04-17T07:26:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.635+0000 7f5f34e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 0x558fa48b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.636+0000 7f5f35800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 0x558fa48b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.636+0000 7f5f35800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 0x558fa48b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44662/0 (socket says 172.21.15.86:44662) 2024-04-17T07:26:13.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.636+0000 7f5f35800640 1 -- 172.21.15.86:0/2853957621 learned_addr learned my addr 172.21.15.86:0/2853957621 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.637+0000 7f5f35800640 1 -- 172.21.15.86:0/2853957621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b6cc0 msgr2=0x558fa48b9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.637+0000 7f5f2d600640 1 -- 172.21.15.86:0/688227282 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5f1c005020 con 0x7f5f2809d570 2024-04-17T07:26:13.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.638+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/2853957621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b6cc0 0x558fa48b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.638+0000 7f5f35800640 1 --2- 172.21.15.86:0/2853957621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b6cc0 0x558fa48b9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.638+0000 7f5f35800640 1 -- 172.21.15.86:0/2853957621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 msgr2=0x558fa48b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.638+0000 7f5f35800640 1 --2- 172.21.15.86:0/2853957621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 0x558fa48b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.638+0000 7f5f35800640 1 -- 172.21.15.86:0/2853957621 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa47c5040 con 0x558fa48b1160 2024-04-17T07:26:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.638+0000 7f5f34e00640 1 --2- 172.21.15.86:0/2853957621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 0x558fa48b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:13.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.638+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/2853957621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b6cc0 0x558fa48b9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:13.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.640+0000 7f5f35800640 1 --2- 172.21.15.86:0/2853957621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 0x558fa48b1580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5f30000b60 tx=0x7f5f3006b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b3af95a4cb9e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.641+0000 7f5f22000640 1 -- 172.21.15.86:0/2853957621 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30085640 con 0x558fa48b1160 2024-04-17T07:26:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.641+0000 7f5f22000640 1 -- 172.21.15.86:0/2853957621 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f300798d0 con 0x558fa48b1160 2024-04-17T07:26:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.641+0000 7f5f22000640 1 -- 172.21.15.86:0/2853957621 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300779a0 con 0x558fa48b1160 2024-04-17T07:26:13.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.641+0000 7f5f2d600640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3db0 0x7f5f180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.642+0000 7f5f2d600640 1 -- 172.21.15.86:0/688227282 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5f1c0d0c20 con 0x7f5f2809d570 2024-04-17T07:26:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.643+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2853957621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 msgr2=0x558fa48b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.643+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2853957621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 0x558fa48b1580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5f30000b60 tx=0x7f5f3006b880 comp rx=0 tx=0).stop 2024-04-17T07:26:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.644+0000 7f5f34e00640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3db0 0x7f5f180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.644+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2853957621 shutdown_connections 2024-04-17T07:26:13.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.644+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2853957621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b6cc0 0x558fa48b9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.644+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2853957621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 0x558fa48b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.644+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2853957621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 0x558fa48b1580 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.644+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2853957621 >> 172.21.15.86:0/2853957621 conn(0x558fa48ac6d0 msgr2=0x558fa48acad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.645+0000 7f5f34e00640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3db0 0x7f5f180a6270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402c250 tx=0x7f5f2402c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.646+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2853957621 shutdown_connections 2024-04-17T07:26:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.646+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2853957621 wait complete. 2024-04-17T07:26:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.646+0000 7f5f3a1fd740 1 -- start start 2024-04-17T07:26:13.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.647+0000 7f5f3a1fd740 1 Processor -- start 2024-04-17T07:26:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.647+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1160 0x558fa48bb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.648+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 0x558fa48bba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.648+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b6cc0 0x558fa48c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.648+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fa48b9c70 con 0x558fa48b1b50 2024-04-17T07:26:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.648+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558fa47d0830 con 0x558fa48b1160 2024-04-17T07:26:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.648+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fa48b9df0 con 0x558fa48b6cc0 2024-04-17T07:26:13.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.648+0000 7f5f3a1fd740 10 client.0 _ll_register_callbacks cb 0x558fa479efb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.649+0000 7f5f2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 0x558fa48bba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.649+0000 7f5f34e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1160 0x558fa48bb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.649+0000 7f5f2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 0x558fa48bba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52346/0 (socket says 172.21.15.86:52346) 2024-04-17T07:26:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.649+0000 7f5f2fe00640 1 -- 172.21.15.86:0/2604991288 learned_addr learned my addr 172.21.15.86:0/2604991288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:13.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.649+0000 7f5f35800640 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b6cc0 0x558fa48c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.650+0000 7f5f34e00640 1 -- 172.21.15.86:0/2604991288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b6cc0 msgr2=0x558fa48c2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.650+0000 7f5f34e00640 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b6cc0 0x558fa48c2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.650+0000 7f5f34e00640 1 -- 172.21.15.86:0/2604991288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 msgr2=0x558fa48bba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.650+0000 7f5f34e00640 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 0x558fa48bba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.650+0000 7f5f34e00640 1 -- 172.21.15.86:0/2604991288 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa47d4920 con 0x558fa48b1160 2024-04-17T07:26:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.651+0000 7f5f34e00640 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1160 0x558fa48bb500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5f24036040 tx=0x7f5f2402bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e14005d0df86472 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.651+0000 7f5f16a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:26:13.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.652+0000 7f5f16a00640 1 -- 172.21.15.86:0/2604991288 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2404e030 con 0x558fa48b1160 2024-04-17T07:26:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.652+0000 7f5f16a00640 1 -- 172.21.15.86:0/2604991288 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5f240046c0 con 0x558fa48b1160 2024-04-17T07:26:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.652+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2604991288 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f300798d0 con 0x558fa48b1160 2024-04-17T07:26:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.653+0000 7f5f16a00640 1 -- 172.21.15.86:0/2604991288 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f2405b050 con 0x558fa48b1160 2024-04-17T07:26:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.653+0000 7f5f3a1fd740 10 client.37244 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:13.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.653+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2604991288 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558fa47a27f0 con 0x558fa48b1160 2024-04-17T07:26:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.654+0000 7f5f16a00640 1 -- 172.21.15.86:0/2604991288 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f300798d0 con 0x558fa48b1160 2024-04-17T07:26:13.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.654+0000 7f5f16a00640 1 -- 172.21.15.86:0/2604991288 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5f24003a30 con 0x558fa48b1160 2024-04-17T07:26:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.654+0000 7f5f3a1fd740 10 client.37244 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.654+0000 7f5f3a1fd740 10 client.37244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.654+0000 7f5f3a1fd740 20 client.37244 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.654+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558fa48c6f30 0x558fa48c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.654+0000 7f5f3a1fd740 4 client.37244 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.654+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2604991288 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fa4824a40 con 0x558fa48c6f30 2024-04-17T07:26:13.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.655+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558fa48c6f30 0x558fa48c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.656+0000 7f5f16a00640 10 client.37244 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:13.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.656+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558fa48c6f30 0x558fa48c93a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b16dd21f7c479ecb server_cookie=2f59d9deaa7689af in_seq=0 out_seq=0 2024-04-17T07:26:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.657+0000 7f5f16a00640 1 -- 172.21.15.86:0/2604991288 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558fa4824a40 con 0x558fa48c6f30 2024-04-17T07:26:13.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.657+0000 7f5f16a00640 10 client.37244 handle_command_reply: tid=0 2024-04-17T07:26:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.658+0000 7f5f3a1fd740 1 client.37244 shutdown 2024-04-17T07:26:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.658+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2604991288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1160 msgr2=0x558fa48bb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.658+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1160 0x558fa48bb500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5f24036040 tx=0x7f5f2402bc40 comp rx=0 tx=0).stop 2024-04-17T07:26:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.658+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2604991288 shutdown_connections 2024-04-17T07:26:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.658+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558fa48c6f30 0x558fa48c93a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.658+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b6cc0 0x558fa48c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.658+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1b50 0x558fa48bba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.658+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2604991288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1160 0x558fa48bb500 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.658+0000 7f5f16a00640 0 client.37244 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.658+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2604991288 >> 172.21.15.86:0/2604991288 conn(0x558fa48ac6d0 msgr2=0x558fa48ab930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.659+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2604991288 shutdown_connections 2024-04-17T07:26:13.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.659+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2604991288 wait complete. 2024-04-17T07:26:13.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.660+0000 7f5f3a1fd740 20 client.37244 trim_cache size 0 max 16384 2024-04-17T07:26:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.687+0000 7f5f3a1fd740 1 Processor -- start 2024-04-17T07:26:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.687+0000 7f5f3a1fd740 1 -- start start 2024-04-17T07:26:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.687+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1160 0x558fa48bb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.687+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1b50 0x558fa48c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.687+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b6cc0 0x558fa48b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.687+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28144f40 con 0x558fa48b1160 2024-04-17T07:26:13.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.687+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28146a80 con 0x558fa48b1b50 2024-04-17T07:26:13.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.687+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28145610 con 0x558fa48b6cc0 2024-04-17T07:26:13.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.688+0000 7f5f2fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1b50 0x558fa48c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.688+0000 7f5f34e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1160 0x558fa48bb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.688+0000 7f5f2fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1b50 0x558fa48c2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37596/0 (socket says 172.21.15.86:37596) 2024-04-17T07:26:13.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.688+0000 7f5f2fe00640 1 -- 172.21.15.86:0/419311950 learned_addr learned my addr 172.21.15.86:0/419311950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:13.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.688+0000 7f5f35800640 1 --2- 172.21.15.86:0/419311950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b6cc0 0x558fa48b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.688+0000 7f5f2fe00640 1 -- 172.21.15.86:0/419311950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b6cc0 msgr2=0x558fa48b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.688+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/419311950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b6cc0 0x558fa48b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.688+0000 7f5f2fe00640 1 -- 172.21.15.86:0/419311950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1160 msgr2=0x558fa48bb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.689+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/419311950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1160 0x558fa48bb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.689+0000 7f5f2fe00640 1 -- 172.21.15.86:0/419311950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa45293f0 con 0x558fa48b1b50 2024-04-17T07:26:13.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.689+0000 7f5f34e00640 1 --2- 172.21.15.86:0/419311950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1160 0x558fa48bb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:13.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.689+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/419311950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1b50 0x558fa48c2a40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c09c820 tx=0x7f5f1c0ce210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=986ced194cd2fdc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.690+0000 7f5f21600640 1 -- 172.21.15.86:0/419311950 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0d5040 con 0x558fa48b1b50 2024-04-17T07:26:13.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.690+0000 7f5f21600640 1 -- 172.21.15.86:0/419311950 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1c0b6070 con 0x558fa48b1b50 2024-04-17T07:26:13.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.690+0000 7f5f21600640 1 -- 172.21.15.86:0/419311950 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0a5790 con 0x558fa48b1b50 2024-04-17T07:26:13.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.691+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/419311950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1b50 msgr2=0x558fa48c2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.691+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/419311950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1b50 0x558fa48c2a40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c09c820 tx=0x7f5f1c0ce210 comp rx=0 tx=0).stop 2024-04-17T07:26:13.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.691+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/419311950 shutdown_connections 2024-04-17T07:26:13.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.691+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/419311950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b6cc0 0x558fa48b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.691+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/419311950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48b1b50 0x558fa48c2a40 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c09c820 tx=0x7f5f1c0ce210 comp rx=0 tx=0).stop 2024-04-17T07:26:13.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.691+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/419311950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b1160 0x558fa48bb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.691+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/419311950 >> 172.21.15.86:0/419311950 conn(0x558fa48ac6d0 msgr2=0x558fa48af930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.692+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/419311950 shutdown_connections 2024-04-17T07:26:13.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.692+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/419311950 wait complete. 2024-04-17T07:26:13.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.692+0000 7f5f3a1fd740 1 -- start start 2024-04-17T07:26:13.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.693+0000 7f5f3a1fd740 1 Processor -- start 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.694+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 0x558fa48c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.694+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b6cc0 0x558fa48c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.694+0000 7f5f3a1fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48c1d20 0x558fa48c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.694+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281463b0 con 0x558fa48b6cc0 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.694+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28145ce0 con 0x558fa48c1d20 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.694+0000 7f5f3a1fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28142f10 con 0x558fa48b1160 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.694+0000 7f5f3a1fd740 10 client.0 _ll_register_callbacks cb 0x7f5f1c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.694+0000 7f5f35800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48c1d20 0x558fa48c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.695+0000 7f5f2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b6cc0 0x558fa48c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.695+0000 7f5f2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b6cc0 0x558fa48c0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52364/0 (socket says 172.21.15.86:52364) 2024-04-17T07:26:13.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.695+0000 7f5f2fe00640 1 -- 172.21.15.86:0/2747769257 learned_addr learned my addr 172.21.15.86:0/2747769257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.695+0000 7f5f34e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 0x558fa48c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.695+0000 7f5f2fe00640 1 -- 172.21.15.86:0/2747769257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 msgr2=0x558fa48c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.695+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 0x558fa48c08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.696+0000 7f5f2fe00640 1 -- 172.21.15.86:0/2747769257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48c1d20 msgr2=0x558fa48c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.696+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48c1d20 0x558fa48c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.696+0000 7f5f2fe00640 1 -- 172.21.15.86:0/2747769257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa45f37d0 con 0x558fa48b6cc0 2024-04-17T07:26:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.696+0000 7f5f2fe00640 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b6cc0 0x558fa48c0e20 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0994f0 tx=0x7f5f1c0056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f311ee21f00e074b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.696+0000 7f5f16a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.696+0000 7f5f16a00640 1 -- 172.21.15.86:0/2747769257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c09d220 con 0x558fa48b6cc0 2024-04-17T07:26:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.697+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2747769257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1c0b6070 con 0x558fa48b6cc0 2024-04-17T07:26:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.697+0000 7f5f16a00640 1 -- 172.21.15.86:0/2747769257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5f1c0ce450 con 0x558fa48b6cc0 2024-04-17T07:26:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.697+0000 7f5f16a00640 1 -- 172.21.15.86:0/2747769257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c0a5790 con 0x558fa48b6cc0 2024-04-17T07:26:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.698+0000 7f5f3a1fd740 10 client.18381 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.698+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2747769257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558fa493ee00 con 0x558fa48b6cc0 2024-04-17T07:26:13.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.699+0000 7f5f16a00640 1 -- 172.21.15.86:0/2747769257 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1c0b6070 con 0x558fa48b6cc0 2024-04-17T07:26:13.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.699+0000 7f5f16a00640 1 -- 172.21.15.86:0/2747769257 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5f1c0b8070 con 0x558fa48b6cc0 2024-04-17T07:26:13.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.700+0000 7f5f3a1fd740 10 client.18381 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:13.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.700+0000 7f5f3a1fd740 10 client.18381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:13.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.700+0000 7f5f3a1fd740 20 client.18381 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:13.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.700+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558fa493f660 0x558fa4941a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:13.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.700+0000 7f5f3a1fd740 4 client.18381 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:26:13.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.700+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2747769257 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558fa481e8b0 con 0x558fa493f660 2024-04-17T07:26:13.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.701+0000 7f5f34e00640 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558fa493f660 0x558fa4941a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:13.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.702+0000 7f5f34e00640 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558fa493f660 0x558fa4941a80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53696c130ef0d72 server_cookie=7417ae3ead0418e4 in_seq=0 out_seq=0 2024-04-17T07:26:13.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.702+0000 7f5f16a00640 10 client.18381 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:13.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.702+0000 7f5f16a00640 1 -- 172.21.15.86:0/2747769257 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558fa481e8b0 con 0x558fa493f660 2024-04-17T07:26:13.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.702+0000 7f5f16a00640 10 client.18381 handle_command_reply: tid=0 2024-04-17T07:26:13.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.702+0000 7f5f3a1fd740 1 client.18381 shutdown 2024-04-17T07:26:13.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2747769257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b6cc0 msgr2=0x558fa48c0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b6cc0 0x558fa48c0e20 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0994f0 tx=0x7f5f1c0056f0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2747769257 shutdown_connections 2024-04-17T07:26:13.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558fa493f660 0x558fa4941a80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558fa48c1d20 0x558fa48c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fa48b6cc0 0x558fa48c0e20 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f3a1fd740 1 --2- 172.21.15.86:0/2747769257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa48b1160 0x558fa48c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2747769257 >> 172.21.15.86:0/2747769257 conn(0x558fa48ac6d0 msgr2=0x558fa48bb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f16a00640 0 client.18381 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2747769257 shutdown_connections 2024-04-17T07:26:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.703+0000 7f5f3a1fd740 1 -- 172.21.15.86:0/2747769257 wait complete. 2024-04-17T07:26:13.705 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:26:13.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.704+0000 7f5f3a1fd740 20 client.18381 trim_cache size 0 max 16384 2024-04-17T07:26:13.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.704+0000 7f5f36200640 1 -- 172.21.15.86:0/688227282 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3db0 msgr2=0x7f5f180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.704+0000 7f5f36200640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3db0 0x7f5f180a6270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402c250 tx=0x7f5f2402c7f0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.704+0000 7f5f36200640 1 -- 172.21.15.86:0/688227282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2809d570 msgr2=0x7f5f28132b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.704+0000 7f5f36200640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2809d570 0x7f5f28132b10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00e9f0 tx=0x7f5f1c00edf0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.705+0000 7f5f36200640 1 -- 172.21.15.86:0/688227282 shutdown_connections 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.705+0000 7f5f36200640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5f180a3db0 0x7f5f180a6270 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.705+0000 7f5f36200640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f2809df30 0x7f5f28139b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.705+0000 7f5f36200640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f2809d570 0x7f5f28132b10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.705+0000 7f5f36200640 1 --2- 172.21.15.86:0/688227282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f2809c380 0x7f5f281325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.705+0000 7f5f36200640 1 -- 172.21.15.86:0/688227282 >> 172.21.15.86:0/688227282 conn(0x7f5f2800b470 msgr2=0x7f5f280a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.705+0000 7f5f36200640 1 -- 172.21.15.86:0/688227282 shutdown_connections 2024-04-17T07:26:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:13.705+0000 7f5f36200640 1 -- 172.21.15.86:0/688227282 wait complete. 2024-04-17T07:26:13.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:13 smithi098 ceph-mon[35418]: pgmap v795: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.3 MiB/s wr, 708 op/s 2024-04-17T07:26:13.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:13 smithi103 ceph-mon[35495]: pgmap v795: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.3 MiB/s wr, 708 op/s 2024-04-17T07:26:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 410.0 s, 598.0 tps, lat 165.589 ms stddev 46.929 2024-04-17T07:26:14.738 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:26:14.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:26:14.739 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:26:15.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.100+0000 7fa91be00640 1 -- 172.21.15.86:0/3012225219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa91c100d70 msgr2=0x7fa91c101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.100+0000 7fa91be00640 1 --2- 172.21.15.86:0/3012225219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa91c100d70 0x7fa91c101190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009ca0 tx=0x7fa90c02c100 comp rx=0 tx=0).stop 2024-04-17T07:26:15.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.102+0000 7fa91be00640 1 -- 172.21.15.86:0/3012225219 shutdown_connections 2024-04-17T07:26:15.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.102+0000 7fa91be00640 1 --2- 172.21.15.86:0/3012225219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c102920 0x7fa91c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.102+0000 7fa91be00640 1 --2- 172.21.15.86:0/3012225219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c101f60 0x7fa91c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.102+0000 7fa91be00640 1 --2- 172.21.15.86:0/3012225219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa91c100d70 0x7fa91c101190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.102+0000 7fa91be00640 1 -- 172.21.15.86:0/3012225219 >> 172.21.15.86:0/3012225219 conn(0x7fa91c0fc300 msgr2=0x7fa91c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.102+0000 7fa91be00640 1 -- 172.21.15.86:0/3012225219 shutdown_connections 2024-04-17T07:26:15.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.102+0000 7fa91be00640 1 -- 172.21.15.86:0/3012225219 wait complete. 2024-04-17T07:26:15.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.103+0000 7fa91be00640 1 Processor -- start 2024-04-17T07:26:15.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.103+0000 7fa91be00640 1 -- start start 2024-04-17T07:26:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.103+0000 7fa91be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa91c100d70 0x7fa91c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.104+0000 7fa91be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c101f60 0x7fa91c19f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.104+0000 7fa91be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c102920 0x7fa91c1a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.104+0000 7fa91be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c10c8f0 con 0x7fa91c101f60 2024-04-17T07:26:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.104+0000 7fa91be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c10c770 con 0x7fa91c100d70 2024-04-17T07:26:15.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.104+0000 7fa91be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c10ca70 con 0x7fa91c102920 2024-04-17T07:26:15.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.104+0000 7fa91aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa91c100d70 0x7fa91c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.104+0000 7fa91aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa91c100d70 0x7fa91c19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37640/0 (socket says 172.21.15.86:37640) 2024-04-17T07:26:15.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.104+0000 7fa91aa00640 1 -- 172.21.15.86:0/1701708338 learned_addr learned my addr 172.21.15.86:0/1701708338 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:15.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.104+0000 7fa91a000640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c101f60 0x7fa91c19f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.105+0000 7fa91b400640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c102920 0x7fa91c1a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.105+0000 7fa91aa00640 1 -- 172.21.15.86:0/1701708338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c102920 msgr2=0x7fa91c1a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.105+0000 7fa91aa00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c102920 0x7fa91c1a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.105+0000 7fa91aa00640 1 -- 172.21.15.86:0/1701708338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c101f60 msgr2=0x7fa91c19f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.105+0000 7fa91aa00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c101f60 0x7fa91c19f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.105+0000 7fa91aa00640 1 -- 172.21.15.86:0/1701708338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c1a63d0 con 0x7fa91c100d70 2024-04-17T07:26:15.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.105+0000 7fa91aa00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa91c100d70 0x7fa91c19ee10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa90c000f80 tx=0x7fa90c035d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.106+0000 7fa920a00640 1 -- 172.21.15.86:0/1701708338 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c039040 con 0x7fa91c100d70 2024-04-17T07:26:15.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.106+0000 7fa91be00640 1 -- 172.21.15.86:0/1701708338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c1a6660 con 0x7fa91c100d70 2024-04-17T07:26:15.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.106+0000 7fa920a00640 1 -- 172.21.15.86:0/1701708338 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa90c044070 con 0x7fa91c100d70 2024-04-17T07:26:15.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.106+0000 7fa920a00640 1 -- 172.21.15.86:0/1701708338 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0043a0 con 0x7fa91c100d70 2024-04-17T07:26:15.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.106+0000 7fa91be00640 1 -- 172.21.15.86:0/1701708338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c1a6c20 con 0x7fa91c100d70 2024-04-17T07:26:15.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.112+0000 7fa924bb1740 1 Processor -- start 2024-04-17T07:26:15.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.112+0000 7fa924bb1740 1 -- start start 2024-04-17T07:26:15.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.112+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.112+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b0a90 0x5568480b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.112+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b5c00 0x5568480b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.112+0000 7fa924bb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556848015cb0 con 0x5568480b00a0 2024-04-17T07:26:15.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.112+0000 7fa924bb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556847fbc370 con 0x5568480b0a90 2024-04-17T07:26:15.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.112+0000 7fa924bb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556848011c70 con 0x5568480b5c00 2024-04-17T07:26:15.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.113+0000 7fa920a00640 1 -- 172.21.15.86:0/1701708338 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa90c004540 con 0x7fa91c100d70 2024-04-17T07:26:15.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.113+0000 7fa91aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b0a90 0x5568480b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.113+0000 7fa91b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.113+0000 7fa91aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b0a90 0x5568480b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37656/0 (socket says 172.21.15.86:37656) 2024-04-17T07:26:15.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.113+0000 7fa91b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52402/0 (socket says 172.21.15.86:52402) 2024-04-17T07:26:15.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.113+0000 7fa91aa00640 1 -- 172.21.15.86:0/2295558946 learned_addr learned my addr 172.21.15.86:0/2295558946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:15.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.114+0000 7fa91b400640 1 -- 172.21.15.86:0/2295558946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b5c00 msgr2=0x5568480b8090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:26:15.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.114+0000 7fa91b400640 1 --2- 172.21.15.86:0/2295558946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b5c00 0x5568480b8090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.114+0000 7fa91b400640 1 -- 172.21.15.86:0/2295558946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b0a90 msgr2=0x5568480b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.114+0000 7fa91b400640 1 --2- 172.21.15.86:0/2295558946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b0a90 0x5568480b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.114+0000 7fa91b400640 1 -- 172.21.15.86:0/2295558946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556847fdb910 con 0x5568480b00a0 2024-04-17T07:26:15.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.115+0000 7fa920a00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9000a3bb0 0x7fa9000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.115+0000 7fa920a00640 1 -- 172.21.15.86:0/1701708338 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa90c0f2de0 con 0x7fa91c100d70 2024-04-17T07:26:15.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.115+0000 7fa91b400640 1 --2- 172.21.15.86:0/2295558946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480b04c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa914009af0 tx=0x7fa9140113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff88d765e3cb4dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.116+0000 7fa91aa00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9000a3bb0 0x7fa9000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.116+0000 7fa907e00640 1 -- 172.21.15.86:0/2295558946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914011e10 con 0x5568480b00a0 2024-04-17T07:26:15.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.116+0000 7fa907e00640 1 -- 172.21.15.86:0/2295558946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9140054d0 con 0x5568480b00a0 2024-04-17T07:26:15.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.116+0000 7fa907e00640 1 -- 172.21.15.86:0/2295558946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914004e60 con 0x5568480b00a0 2024-04-17T07:26:15.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.118+0000 7fa924bb1740 1 -- 172.21.15.86:0/2295558946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 msgr2=0x5568480b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.118+0000 7fa924bb1740 1 --2- 172.21.15.86:0/2295558946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480b04c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa914009af0 tx=0x7fa9140113e0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.118+0000 7fa91aa00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9000a3bb0 0x7fa9000a6070 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0bb6b0 tx=0x7fa90c0f6820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.119+0000 7fa924bb1740 1 -- 172.21.15.86:0/2295558946 shutdown_connections 2024-04-17T07:26:15.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.119+0000 7fa924bb1740 1 --2- 172.21.15.86:0/2295558946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b5c00 0x5568480b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.119+0000 7fa924bb1740 1 --2- 172.21.15.86:0/2295558946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b0a90 0x5568480b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.119+0000 7fa924bb1740 1 --2- 172.21.15.86:0/2295558946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480b04c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.119+0000 7fa924bb1740 1 -- 172.21.15.86:0/2295558946 >> 172.21.15.86:0/2295558946 conn(0x5568480ab610 msgr2=0x5568480aba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.119+0000 7fa924bb1740 1 -- 172.21.15.86:0/2295558946 shutdown_connections 2024-04-17T07:26:15.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.119+0000 7fa924bb1740 1 -- 172.21.15.86:0/2295558946 wait complete. 2024-04-17T07:26:15.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.120+0000 7fa924bb1740 1 -- start start 2024-04-17T07:26:15.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.121+0000 7fa924bb1740 1 Processor -- start 2024-04-17T07:26:15.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.122+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.122+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b0a90 0x5568480cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.122+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.122+0000 7fa924bb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568480c9bc0 con 0x5568480b00a0 2024-04-17T07:26:15.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.122+0000 7fa924bb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556847fcf730 con 0x5568480b5c00 2024-04-17T07:26:15.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.122+0000 7fa924bb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5568480c9d40 con 0x5568480b0a90 2024-04-17T07:26:15.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.122+0000 7fa924bb1740 10 client.0 _ll_register_callbacks cb 0x556847fd1b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:15.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.123+0000 7fa91b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.123+0000 7fa91b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52408/0 (socket says 172.21.15.86:52408) 2024-04-17T07:26:15.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.123+0000 7fa91b400640 1 -- 172.21.15.86:0/1394873467 learned_addr learned my addr 172.21.15.86:0/1394873467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:15.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.123+0000 7fa91b400640 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.124+0000 7fa91a000640 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b0a90 0x5568480cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.124+0000 7fa91b400640 1 -- 172.21.15.86:0/1394873467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b0a90 msgr2=0x5568480cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.124+0000 7fa91b400640 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b0a90 0x5568480cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.124+0000 7fa91b400640 1 -- 172.21.15.86:0/1394873467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 msgr2=0x5568480d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.124+0000 7fa91b400640 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480d2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.124+0000 7fa91b400640 1 -- 172.21.15.86:0/1394873467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556847df07e0 con 0x5568480b00a0 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.125+0000 7fa91b400640 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480cb5b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa9140057c0 tx=0x7fa914005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bd218644d3eecb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.125+0000 7fa904c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.125+0000 7fa904c00640 1 -- 172.21.15.86:0/1394873467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91402e060 con 0x5568480b00a0 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.125+0000 7fa904c00640 1 -- 172.21.15.86:0/1394873467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa914012450 con 0x5568480b00a0 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.126+0000 7fa924bb1740 1 -- 172.21.15.86:0/1394873467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9140054d0 con 0x5568480b00a0 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.126+0000 7fa904c00640 1 -- 172.21.15.86:0/1394873467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa91401f040 con 0x5568480b00a0 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.126+0000 7fa924bb1740 10 client.18393 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.126+0000 7fa924bb1740 1 -- 172.21.15.86:0/1394873467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556847e50f70 con 0x5568480b00a0 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.126+0000 7fa904c00640 1 -- 172.21.15.86:0/1394873467 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9140054d0 con 0x5568480b00a0 2024-04-17T07:26:15.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.127+0000 7fa904c00640 1 -- 172.21.15.86:0/1394873467 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa91401a490 con 0x5568480b00a0 2024-04-17T07:26:15.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.127+0000 7fa924bb1740 10 client.18393 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:15.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.127+0000 7fa924bb1740 10 client.18393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:15.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.127+0000 7fa924bb1740 20 client.18393 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:15.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.127+0000 7fa924bb1740 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568480d7020 0x5568480d9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.127+0000 7fa924bb1740 4 client.18393 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:15.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.127+0000 7fa924bb1740 1 -- 172.21.15.86:0/1394873467 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568480236d0 con 0x5568480d7020 2024-04-17T07:26:15.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.128+0000 7fa91a000640 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568480d7020 0x5568480d9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.129+0000 7fa91a000640 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568480d7020 0x5568480d9410 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed0d81906f54e33 server_cookie=d955ba63312c3aaf in_seq=0 out_seq=0 2024-04-17T07:26:15.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.129+0000 7fa904c00640 10 client.18393 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:15.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.131+0000 7fa904c00640 1 -- 172.21.15.86:0/1394873467 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568480236d0 con 0x5568480d7020 2024-04-17T07:26:15.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.131+0000 7fa904c00640 10 client.18393 handle_command_reply: tid=0 2024-04-17T07:26:15.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.132+0000 7fa924bb1740 1 client.18393 shutdown 2024-04-17T07:26:15.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.132+0000 7fa924bb1740 1 -- 172.21.15.86:0/1394873467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 msgr2=0x5568480cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.132+0000 7fa924bb1740 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480cb5b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa9140057c0 tx=0x7fa914005d30 comp rx=0 tx=0).stop 2024-04-17T07:26:15.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.132+0000 7fa924bb1740 1 -- 172.21.15.86:0/1394873467 shutdown_connections 2024-04-17T07:26:15.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.132+0000 7fa924bb1740 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5568480d7020 0x5568480d9410 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.132+0000 7fa924bb1740 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.132+0000 7fa924bb1740 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b0a90 0x5568480cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.132+0000 7fa924bb1740 1 --2- 172.21.15.86:0/1394873467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480cb5b0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.132+0000 7fa924bb1740 1 -- 172.21.15.86:0/1394873467 >> 172.21.15.86:0/1394873467 conn(0x5568480ab610 msgr2=0x5568480aa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.132+0000 7fa904c00640 0 client.18393 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:15.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.133+0000 7fa924bb1740 1 -- 172.21.15.86:0/1394873467 shutdown_connections 2024-04-17T07:26:15.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.133+0000 7fa924bb1740 1 -- 172.21.15.86:0/1394873467 wait complete. 2024-04-17T07:26:15.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.134+0000 7fa924bb1740 20 client.18393 trim_cache size 0 max 16384 2024-04-17T07:26:15.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.166+0000 7fa924bb1740 1 Processor -- start 2024-04-17T07:26:15.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.166+0000 7fa924bb1740 1 -- start start 2024-04-17T07:26:15.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.167+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b00a0 0x5568480cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.167+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b0a90 0x5568480d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.167+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.167+0000 7fa924bb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1a8860 con 0x5568480b0a90 2024-04-17T07:26:15.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.167+0000 7fa924bb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1aa3a0 con 0x5568480b5c00 2024-04-17T07:26:15.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.167+0000 7fa924bb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1a8f30 con 0x5568480b00a0 2024-04-17T07:26:15.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.167+0000 7fa91b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b00a0 0x5568480cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.167+0000 7fa91b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b00a0 0x5568480cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44750/0 (socket says 172.21.15.86:44750) 2024-04-17T07:26:15.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.167+0000 7fa91b400640 1 -- 172.21.15.86:0/3635013312 learned_addr learned my addr 172.21.15.86:0/3635013312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:15.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.168+0000 7fa91a000640 1 --2- 172.21.15.86:0/3635013312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b0a90 0x5568480d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.168+0000 7fa91b400640 1 --2- 172.21.15.86:0/3635013312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.168+0000 7fa91b400640 1 -- 172.21.15.86:0/3635013312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 msgr2=0x5568480ddc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.168+0000 7fa91b400640 1 --2- 172.21.15.86:0/3635013312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480ddc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.168+0000 7fa91b400640 1 -- 172.21.15.86:0/3635013312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b0a90 msgr2=0x5568480d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.169+0000 7fa91b400640 1 --2- 172.21.15.86:0/3635013312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b0a90 0x5568480d2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.169+0000 7fa91b400640 1 -- 172.21.15.86:0/3635013312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556847ce8170 con 0x5568480b00a0 2024-04-17T07:26:15.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.169+0000 7fa91a000640 1 --2- 172.21.15.86:0/3635013312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b0a90 0x5568480d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.169+0000 7fa91b400640 1 --2- 172.21.15.86:0/3635013312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b00a0 0x5568480cb830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa914009900 tx=0x7fa914001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c909f26933ef2c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.169+0000 7fa907400640 1 -- 172.21.15.86:0/3635013312 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914038080 con 0x5568480b00a0 2024-04-17T07:26:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.170+0000 7fa907400640 1 -- 172.21.15.86:0/3635013312 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa914004bc0 con 0x5568480b00a0 2024-04-17T07:26:15.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.170+0000 7fa907400640 1 -- 172.21.15.86:0/3635013312 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91401f040 con 0x5568480b00a0 2024-04-17T07:26:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.171+0000 7fa924bb1740 1 -- 172.21.15.86:0/3635013312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b00a0 msgr2=0x5568480cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.171+0000 7fa924bb1740 1 --2- 172.21.15.86:0/3635013312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b00a0 0x5568480cb830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa914009900 tx=0x7fa914001e50 comp rx=0 tx=0).stop 2024-04-17T07:26:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.171+0000 7fa924bb1740 1 -- 172.21.15.86:0/3635013312 shutdown_connections 2024-04-17T07:26:15.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.171+0000 7fa924bb1740 1 --2- 172.21.15.86:0/3635013312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480ddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.171+0000 7fa924bb1740 1 --2- 172.21.15.86:0/3635013312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b0a90 0x5568480d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.171+0000 7fa924bb1740 1 --2- 172.21.15.86:0/3635013312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b00a0 0x5568480cb830 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.171+0000 7fa924bb1740 1 -- 172.21.15.86:0/3635013312 >> 172.21.15.86:0/3635013312 conn(0x5568480ab610 msgr2=0x5568480af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.172+0000 7fa924bb1740 1 -- 172.21.15.86:0/3635013312 shutdown_connections 2024-04-17T07:26:15.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.172+0000 7fa924bb1740 1 -- 172.21.15.86:0/3635013312 wait complete. 2024-04-17T07:26:15.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.172+0000 7fa924bb1740 1 -- start start 2024-04-17T07:26:15.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.173+0000 7fa924bb1740 1 Processor -- start 2024-04-17T07:26:15.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.174+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.174+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b0a90 0x5568480d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.174+0000 7fa924bb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.174+0000 7fa924bb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1a9cd0 con 0x5568480b00a0 2024-04-17T07:26:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.174+0000 7fa924bb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1a9600 con 0x5568480b5c00 2024-04-17T07:26:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.174+0000 7fa924bb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c19e4f0 con 0x5568480b0a90 2024-04-17T07:26:15.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.174+0000 7fa924bb1740 10 client.0 _ll_register_callbacks cb 0x7fa90c0f66f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.174+0000 7fa91b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.174+0000 7fa91b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480d25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52432/0 (socket says 172.21.15.86:52432) 2024-04-17T07:26:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.174+0000 7fa91b400640 1 -- 172.21.15.86:0/3810867692 learned_addr learned my addr 172.21.15.86:0/3810867692 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.175+0000 7fa91a000640 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b0a90 0x5568480d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.175+0000 7fa91b400640 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.175+0000 7fa91b400640 1 -- 172.21.15.86:0/3810867692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b0a90 msgr2=0x5568480d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.175+0000 7fa91b400640 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b0a90 0x5568480d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.175+0000 7fa91b400640 1 -- 172.21.15.86:0/3810867692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 msgr2=0x5568480d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.175+0000 7fa91b400640 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.175+0000 7fa91b400640 1 -- 172.21.15.86:0/3810867692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568480d1650 con 0x5568480b00a0 2024-04-17T07:26:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.176+0000 7fa91b400640 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480d25c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa91401a980 tx=0x7fa9140096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c287df93c2b2319 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.176+0000 7fa904c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.177+0000 7fa904c00640 1 -- 172.21.15.86:0/3810867692 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914046030 con 0x5568480b00a0 2024-04-17T07:26:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.177+0000 7fa904c00640 1 -- 172.21.15.86:0/3810867692 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa914004020 con 0x5568480b00a0 2024-04-17T07:26:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.177+0000 7fa924bb1740 1 -- 172.21.15.86:0/3810867692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa914004bc0 con 0x5568480b00a0 2024-04-17T07:26:15.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.178+0000 7fa904c00640 1 -- 172.21.15.86:0/3810867692 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa914038040 con 0x5568480b00a0 2024-04-17T07:26:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.178+0000 7fa924bb1740 10 client.18399 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.178+0000 7fa924bb1740 1 -- 172.21.15.86:0/3810867692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55684813e130 con 0x5568480b00a0 2024-04-17T07:26:15.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.178+0000 7fa904c00640 1 -- 172.21.15.86:0/3810867692 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa914004bc0 con 0x5568480b00a0 2024-04-17T07:26:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.179+0000 7fa904c00640 1 -- 172.21.15.86:0/3810867692 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa9140035f0 con 0x5568480b00a0 2024-04-17T07:26:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.179+0000 7fa924bb1740 10 client.18399 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.179+0000 7fa924bb1740 10 client.18399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.179+0000 7fa924bb1740 20 client.18399 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.179+0000 7fa924bb1740 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55684813ea80 0x556848140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.179+0000 7fa924bb1740 4 client.18399 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:26:15.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.179+0000 7fa924bb1740 1 -- 172.21.15.86:0/3810867692 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55684801d540 con 0x55684813ea80 2024-04-17T07:26:15.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.180+0000 7fa91a000640 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55684813ea80 0x556848140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.181+0000 7fa91a000640 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55684813ea80 0x556848140ea0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5b466bbd6d13f0 server_cookie=993d1ff6930bdd5 in_seq=0 out_seq=0 2024-04-17T07:26:15.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.181+0000 7fa904c00640 10 client.18399 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:15.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.183+0000 7fa904c00640 1 -- 172.21.15.86:0/3810867692 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55684801d540 con 0x55684813ea80 2024-04-17T07:26:15.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.183+0000 7fa904c00640 10 client.18399 handle_command_reply: tid=0 2024-04-17T07:26:15.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.184+0000 7fa924bb1740 1 client.18399 shutdown 2024-04-17T07:26:15.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.184+0000 7fa924bb1740 1 -- 172.21.15.86:0/3810867692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 msgr2=0x5568480d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.184+0000 7fa924bb1740 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480d25c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa91401a980 tx=0x7fa9140096c0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.184+0000 7fa924bb1740 1 -- 172.21.15.86:0/3810867692 shutdown_connections 2024-04-17T07:26:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.184+0000 7fa924bb1740 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55684813ea80 0x556848140ea0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.184+0000 7fa924bb1740 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5568480b5c00 0x5568480d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.184+0000 7fa924bb1740 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5568480b0a90 0x5568480d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.184+0000 7fa924bb1740 1 --2- 172.21.15.86:0/3810867692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568480b00a0 0x5568480d25c0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.184+0000 7fa924bb1740 1 -- 172.21.15.86:0/3810867692 >> 172.21.15.86:0/3810867692 conn(0x5568480ab610 msgr2=0x5568480aa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.184+0000 7fa904c00640 0 client.18399 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:15.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.185+0000 7fa924bb1740 1 -- 172.21.15.86:0/3810867692 shutdown_connections 2024-04-17T07:26:15.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.185+0000 7fa924bb1740 1 -- 172.21.15.86:0/3810867692 wait complete. 2024-04-17T07:26:15.186 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:26:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:26:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "26b32fb6-6916-4e04-849b-3b545d732fb8", 2024-04-17T07:26:15.186 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:26:15.187 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:26:15.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.185+0000 7fa924bb1740 20 client.18399 trim_cache size 0 max 16384 2024-04-17T07:26:15.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.186+0000 7fa91be00640 1 -- 172.21.15.86:0/1701708338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9000a3bb0 msgr2=0x7fa9000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.186+0000 7fa91be00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9000a3bb0 0x7fa9000a6070 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0bb6b0 tx=0x7fa90c0f6820 comp rx=0 tx=0).stop 2024-04-17T07:26:15.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.186+0000 7fa91be00640 1 -- 172.21.15.86:0/1701708338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa91c100d70 msgr2=0x7fa91c19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.186+0000 7fa91be00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa91c100d70 0x7fa91c19ee10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa90c000f80 tx=0x7fa90c035d60 comp rx=0 tx=0).stop 2024-04-17T07:26:15.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.186+0000 7fa91be00640 1 -- 172.21.15.86:0/1701708338 shutdown_connections 2024-04-17T07:26:15.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.186+0000 7fa91be00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa9000a3bb0 0x7fa9000a6070 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.186+0000 7fa91be00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa91c102920 0x7fa91c1a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.186+0000 7fa91be00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa91c101f60 0x7fa91c19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.186+0000 7fa91be00640 1 --2- 172.21.15.86:0/1701708338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa91c100d70 0x7fa91c19ee10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.186+0000 7fa91be00640 1 -- 172.21.15.86:0/1701708338 >> 172.21.15.86:0/1701708338 conn(0x7fa91c0fc300 msgr2=0x7fa91c0691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.187+0000 7fa91be00640 1 -- 172.21.15.86:0/1701708338 shutdown_connections 2024-04-17T07:26:15.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.187+0000 7fa91be00640 1 -- 172.21.15.86:0/1701708338 wait complete. 2024-04-17T07:26:15.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:26:15.224 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:26:15.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.569+0000 7ff182a00640 1 -- 172.21.15.86:0/3320371288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff17c1041c0 msgr2=0x7ff17c1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.569+0000 7ff182a00640 1 --2- 172.21.15.86:0/3320371288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff17c1041c0 0x7ff17c1045e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009900 tx=0x7ff16c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:26:15.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.569+0000 7ff182a00640 1 -- 172.21.15.86:0/3320371288 shutdown_connections 2024-04-17T07:26:15.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.569+0000 7ff182a00640 1 --2- 172.21.15.86:0/3320371288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c105d70 0x7ff17c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.569+0000 7ff182a00640 1 --2- 172.21.15.86:0/3320371288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff17c1053b0 0x7ff17c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.569+0000 7ff182a00640 1 --2- 172.21.15.86:0/3320371288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff17c1041c0 0x7ff17c1045e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.569+0000 7ff182a00640 1 -- 172.21.15.86:0/3320371288 >> 172.21.15.86:0/3320371288 conn(0x7ff17c0ff710 msgr2=0x7ff17c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.570+0000 7ff182a00640 1 -- 172.21.15.86:0/3320371288 shutdown_connections 2024-04-17T07:26:15.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.570+0000 7ff182a00640 1 -- 172.21.15.86:0/3320371288 wait complete. 2024-04-17T07:26:15.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.571+0000 7ff182a00640 1 Processor -- start 2024-04-17T07:26:15.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.572+0000 7ff182a00640 1 -- start start 2024-04-17T07:26:15.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.573+0000 7ff182a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c1041c0 0x7ff17c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.573+0000 7ff182a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff17c1053b0 0x7ff17c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.573+0000 7ff182a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff17c105d70 0x7ff17c1ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.574+0000 7ff17b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff17c1053b0 0x7ff17c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.574+0000 7ff17b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff17c1053b0 0x7ff17c06ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44790/0 (socket says 172.21.15.86:44790) 2024-04-17T07:26:15.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.574+0000 7ff17be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c1041c0 0x7ff17c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.574+0000 7ff180a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff17c105d70 0x7ff17c1ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.575+0000 7ff17b400640 1 -- 172.21.15.86:0/3933191228 learned_addr learned my addr 172.21.15.86:0/3933191228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:15.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.575+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c077cd0 con 0x7ff17c1041c0 2024-04-17T07:26:15.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.575+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c077b50 con 0x7ff17c105d70 2024-04-17T07:26:15.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.575+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c077e50 con 0x7ff17c1053b0 2024-04-17T07:26:15.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.576+0000 7ff17b400640 1 -- 172.21.15.86:0/3933191228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff17c105d70 msgr2=0x7ff17c1ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.576+0000 7ff17b400640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff17c105d70 0x7ff17c1ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.576+0000 7ff17b400640 1 -- 172.21.15.86:0/3933191228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c1041c0 msgr2=0x7ff17c06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.576+0000 7ff17b400640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c1041c0 0x7ff17c06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.576+0000 7ff17b400640 1 -- 172.21.15.86:0/3933191228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c06f480 con 0x7ff17c1053b0 2024-04-17T07:26:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.576+0000 7ff17be00640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c1041c0 0x7ff17c06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.578+0000 7ff17b400640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff17c1053b0 0x7ff17c06ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff16800e9f0 tx=0x7ff16800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.579+0000 7ff178c00640 1 -- 172.21.15.86:0/3933191228 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16800cd80 con 0x7ff17c1053b0 2024-04-17T07:26:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.579+0000 7ff178c00640 1 -- 172.21.15.86:0/3933191228 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1680108e0 con 0x7ff17c1053b0 2024-04-17T07:26:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.579+0000 7ff178c00640 1 -- 172.21.15.86:0/3933191228 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168018920 con 0x7ff17c1053b0 2024-04-17T07:26:15.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:15 smithi086 ceph-mon[31733]: pgmap v796: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.1 MiB/s wr, 696 op/s 2024-04-17T07:26:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.580+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c1aef90 con 0x7ff17c1053b0 2024-04-17T07:26:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.580+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c1af400 con 0x7ff17c1053b0 2024-04-17T07:26:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.585+0000 7ff185765740 1 Processor -- start 2024-04-17T07:26:15.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.587+0000 7ff185765740 1 -- start start 2024-04-17T07:26:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.588+0000 7ff185765740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5190 0x55ca819d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.588+0000 7ff185765740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5b80 0x55ca819da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.588+0000 7ff185765740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819dacf0 0x55ca819dd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.588+0000 7ff185765740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca8193c120 con 0x55ca819dacf0 2024-04-17T07:26:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.588+0000 7ff185765740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ca818e2340 con 0x55ca819d5b80 2024-04-17T07:26:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.588+0000 7ff185765740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca819380e0 con 0x55ca819d5190 2024-04-17T07:26:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.589+0000 7ff180a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5190 0x55ca819d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.589+0000 7ff180a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5190 0x55ca819d55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44796/0 (socket says 172.21.15.86:44796) 2024-04-17T07:26:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.589+0000 7ff180a00640 1 -- 172.21.15.86:0/155000208 learned_addr learned my addr 172.21.15.86:0/155000208 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:15.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.589+0000 7ff17be00640 1 --2- 172.21.15.86:0/155000208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5b80 0x55ca819da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.590+0000 7ff180a00640 1 -- 172.21.15.86:0/155000208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5b80 msgr2=0x55ca819da680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.590+0000 7ff180a00640 1 --2- 172.21.15.86:0/155000208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5b80 0x55ca819da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.590+0000 7ff180a00640 1 -- 172.21.15.86:0/155000208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819dacf0 msgr2=0x55ca819dd180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:26:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.590+0000 7ff180a00640 1 --2- 172.21.15.86:0/155000208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819dacf0 0x55ca819dd180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.590+0000 7ff180a00640 1 -- 172.21.15.86:0/155000208 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca817187d0 con 0x55ca819d5190 2024-04-17T07:26:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.591+0000 7ff180a00640 1 --2- 172.21.15.86:0/155000208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5190 0x55ca819d55b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff1740039f0 tx=0x7ff174003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a88ee71fd9a024b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.592+0000 7ff178c00640 1 -- 172.21.15.86:0/3933191228 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff168010430 con 0x7ff17c1053b0 2024-04-17T07:26:15.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.593+0000 7ff171200640 1 -- 172.21.15.86:0/155000208 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17401a470 con 0x55ca819d5190 2024-04-17T07:26:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.593+0000 7ff171200640 1 -- 172.21.15.86:0/155000208 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff174015070 con 0x55ca819d5190 2024-04-17T07:26:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.593+0000 7ff171200640 1 -- 172.21.15.86:0/155000208 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17401a470 con 0x55ca819d5190 2024-04-17T07:26:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.594+0000 7ff185765740 1 -- 172.21.15.86:0/155000208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5190 msgr2=0x55ca819d55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.594+0000 7ff185765740 1 --2- 172.21.15.86:0/155000208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5190 0x55ca819d55b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff1740039f0 tx=0x7ff174003eb0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.595+0000 7ff185765740 1 -- 172.21.15.86:0/155000208 shutdown_connections 2024-04-17T07:26:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.595+0000 7ff185765740 1 --2- 172.21.15.86:0/155000208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819dacf0 0x55ca819dd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.595+0000 7ff185765740 1 --2- 172.21.15.86:0/155000208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5b80 0x55ca819da680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.595+0000 7ff185765740 1 --2- 172.21.15.86:0/155000208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5190 0x55ca819d55b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.595+0000 7ff185765740 1 -- 172.21.15.86:0/155000208 >> 172.21.15.86:0/155000208 conn(0x55ca819d0700 msgr2=0x55ca819d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.596+0000 7ff185765740 1 -- 172.21.15.86:0/155000208 shutdown_connections 2024-04-17T07:26:15.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.596+0000 7ff185765740 1 -- 172.21.15.86:0/155000208 wait complete. 2024-04-17T07:26:15.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.596+0000 7ff185765740 1 -- start start 2024-04-17T07:26:15.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.596+0000 7ff178c00640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1600a3c40 0x7ff1600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.597+0000 7ff185765740 1 Processor -- start 2024-04-17T07:26:15.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.597+0000 7ff178c00640 1 -- 172.21.15.86:0/3933191228 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff1680d0a60 con 0x7ff17c1053b0 2024-04-17T07:26:15.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.598+0000 7ff180a00640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1600a3c40 0x7ff1600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.598+0000 7ff185765740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 0x55ca819df710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.598+0000 7ff185765740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5b80 0x55ca819dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.598+0000 7ff185765740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819dacf0 0x55ca819e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.598+0000 7ff185765740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca819ddca0 con 0x55ca819d5b80 2024-04-17T07:26:15.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.598+0000 7ff185765740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ca818f5710 con 0x55ca819d5190 2024-04-17T07:26:15.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.598+0000 7ff185765740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca819dde20 con 0x55ca819dacf0 2024-04-17T07:26:15.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.598+0000 7ff185765740 10 client.0 _ll_register_callbacks cb 0x55ca818f0af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:15.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.599+0000 7ff180a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819dacf0 0x55ca819e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.599+0000 7ff180a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819dacf0 0x55ca819e6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44810/0 (socket says 172.21.15.86:44810) 2024-04-17T07:26:15.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.599+0000 7ff180a00640 1 -- 172.21.15.86:0/2481911895 learned_addr learned my addr 172.21.15.86:0/2481911895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:15.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.599+0000 7ff180a00640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1600a3c40 0x7ff1600a6100 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff174011fd0 tx=0x7ff174005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.599+0000 7ff17be00640 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 0x55ca819df710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.600+0000 7ff17b400640 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5b80 0x55ca819dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.600+0000 7ff180a00640 1 -- 172.21.15.86:0/2481911895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 msgr2=0x55ca819df710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.600+0000 7ff180a00640 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 0x55ca819df710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.600+0000 7ff180a00640 1 -- 172.21.15.86:0/2481911895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5b80 msgr2=0x55ca819dfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.600+0000 7ff180a00640 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5b80 0x55ca819dfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.600+0000 7ff180a00640 1 -- 172.21.15.86:0/2481911895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8164e3f0 con 0x55ca819dacf0 2024-04-17T07:26:15.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.600+0000 7ff180a00640 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819dacf0 0x55ca819e6cb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff174009df0 tx=0x7ff1740096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd8b4e72b36ca7ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.600+0000 7ff165600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:15.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.601+0000 7ff165600640 1 -- 172.21.15.86:0/2481911895 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174032020 con 0x55ca819dacf0 2024-04-17T07:26:15.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.601+0000 7ff165600640 1 -- 172.21.15.86:0/2481911895 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff174010d50 con 0x55ca819dacf0 2024-04-17T07:26:15.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.602+0000 7ff185765740 1 -- 172.21.15.86:0/2481911895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff174015070 con 0x55ca819dacf0 2024-04-17T07:26:15.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.603+0000 7ff165600640 1 -- 172.21.15.86:0/2481911895 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff17402a040 con 0x55ca819dacf0 2024-04-17T07:26:15.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.603+0000 7ff185765740 10 client.27358 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:15.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.603+0000 7ff185765740 1 -- 172.21.15.86:0/2481911895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca8160e0f0 con 0x55ca819dacf0 2024-04-17T07:26:15.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.604+0000 7ff165600640 1 -- 172.21.15.86:0/2481911895 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff174015070 con 0x55ca819dacf0 2024-04-17T07:26:15.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.605+0000 7ff165600640 1 -- 172.21.15.86:0/2481911895 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff17403f020 con 0x55ca819dacf0 2024-04-17T07:26:15.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.606+0000 7ff185765740 10 client.27358 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:15.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.606+0000 7ff185765740 10 client.27358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:15.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.607+0000 7ff185765740 20 client.27358 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:15.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.608+0000 7ff185765740 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ca819fc250 0x55ca819fe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.608+0000 7ff185765740 4 client.27358 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:15.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.608+0000 7ff185765740 1 -- 172.21.15.86:0/2481911895 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca81949b40 con 0x55ca819fc250 2024-04-17T07:26:15.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.609+0000 7ff17be00640 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ca819fc250 0x55ca819fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.611+0000 7ff165600640 10 client.27358 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:15.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.611+0000 7ff17be00640 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ca819fc250 0x55ca819fe640 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34566d79db01cc40 server_cookie=54a19dc2aa737f21 in_seq=0 out_seq=0 2024-04-17T07:26:15.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.615+0000 7ff165600640 1 -- 172.21.15.86:0/2481911895 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca81949b40 con 0x55ca819fc250 2024-04-17T07:26:15.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.615+0000 7ff165600640 10 client.27358 handle_command_reply: tid=0 2024-04-17T07:26:15.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.615+0000 7ff185765740 1 client.27358 shutdown 2024-04-17T07:26:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.616+0000 7ff185765740 1 -- 172.21.15.86:0/2481911895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819dacf0 msgr2=0x55ca819e6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.616+0000 7ff185765740 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819dacf0 0x55ca819e6cb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff174009df0 tx=0x7ff1740096a0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.616+0000 7ff185765740 1 -- 172.21.15.86:0/2481911895 shutdown_connections 2024-04-17T07:26:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.616+0000 7ff185765740 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ca819fc250 0x55ca819fe640 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.617+0000 7ff185765740 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819dacf0 0x55ca819e6cb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.617+0000 7ff185765740 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5b80 0x55ca819dfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.617+0000 7ff185765740 1 --2- 172.21.15.86:0/2481911895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 0x55ca819df710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.617+0000 7ff185765740 1 -- 172.21.15.86:0/2481911895 >> 172.21.15.86:0/2481911895 conn(0x55ca819d0700 msgr2=0x55ca819cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.617+0000 7ff165600640 0 client.27358 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.619+0000 7ff185765740 1 -- 172.21.15.86:0/2481911895 shutdown_connections 2024-04-17T07:26:15.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.619+0000 7ff185765740 1 -- 172.21.15.86:0/2481911895 wait complete. 2024-04-17T07:26:15.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.620+0000 7ff185765740 20 client.27358 trim_cache size 0 max 16384 2024-04-17T07:26:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.652+0000 7ff185765740 1 Processor -- start 2024-04-17T07:26:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.652+0000 7ff185765740 1 -- start start 2024-04-17T07:26:15.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.654+0000 7ff185765740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5190 0x55ca819cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.655+0000 7ff185765740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 0x55ca819e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.655+0000 7ff185765740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819dacf0 0x55ca819cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.655+0000 7ff185765740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1b11a0 con 0x55ca819d5190 2024-04-17T07:26:15.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.655+0000 7ff185765740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1b2ce0 con 0x55ca819dacf0 2024-04-17T07:26:15.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.655+0000 7ff185765740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1b1870 con 0x55ca819d5b80 2024-04-17T07:26:15.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.656+0000 7ff17be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5190 0x55ca819cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.656+0000 7ff17b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 0x55ca819e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.656+0000 7ff17b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 0x55ca819e6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44816/0 (socket says 172.21.15.86:44816) 2024-04-17T07:26:15.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.656+0000 7ff17b400640 1 -- 172.21.15.86:0/1748791519 learned_addr learned my addr 172.21.15.86:0/1748791519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:15.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.656+0000 7ff180a00640 1 --2- 172.21.15.86:0/1748791519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819dacf0 0x55ca819cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.656+0000 7ff17be00640 1 -- 172.21.15.86:0/1748791519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 msgr2=0x55ca819e6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.656+0000 7ff17be00640 1 --2- 172.21.15.86:0/1748791519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 0x55ca819e6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.656+0000 7ff17be00640 1 -- 172.21.15.86:0/1748791519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819dacf0 msgr2=0x55ca819cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.656+0000 7ff17be00640 1 --2- 172.21.15.86:0/1748791519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819dacf0 0x55ca819cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.656+0000 7ff17be00640 1 -- 172.21.15.86:0/1748791519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca819cace0 con 0x55ca819d5190 2024-04-17T07:26:15.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.657+0000 7ff17be00640 1 --2- 172.21.15.86:0/1748791519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5190 0x55ca819cf000 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff16c036040 tx=0x7ff16c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d17fb8a8ea7eb36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.657+0000 7ff167e00640 1 -- 172.21.15.86:0/1748791519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0040c0 con 0x55ca819d5190 2024-04-17T07:26:15.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.657+0000 7ff167e00640 1 -- 172.21.15.86:0/1748791519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff16c0054f0 con 0x55ca819d5190 2024-04-17T07:26:15.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.657+0000 7ff167e00640 1 -- 172.21.15.86:0/1748791519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c004ae0 con 0x55ca819d5190 2024-04-17T07:26:15.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.658+0000 7ff185765740 1 -- 172.21.15.86:0/1748791519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5190 msgr2=0x55ca819cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.658+0000 7ff185765740 1 --2- 172.21.15.86:0/1748791519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5190 0x55ca819cf000 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff16c036040 tx=0x7ff16c009260 comp rx=0 tx=0).stop 2024-04-17T07:26:15.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.659+0000 7ff185765740 1 -- 172.21.15.86:0/1748791519 shutdown_connections 2024-04-17T07:26:15.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.659+0000 7ff185765740 1 --2- 172.21.15.86:0/1748791519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819dacf0 0x55ca819cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.659+0000 7ff185765740 1 --2- 172.21.15.86:0/1748791519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 0x55ca819e6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.659+0000 7ff185765740 1 --2- 172.21.15.86:0/1748791519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819d5190 0x55ca819cf000 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.659+0000 7ff185765740 1 -- 172.21.15.86:0/1748791519 >> 172.21.15.86:0/1748791519 conn(0x55ca819d0700 msgr2=0x55ca819df710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.659+0000 7ff185765740 1 -- 172.21.15.86:0/1748791519 shutdown_connections 2024-04-17T07:26:15.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.660+0000 7ff185765740 1 -- 172.21.15.86:0/1748791519 wait complete. 2024-04-17T07:26:15.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.660+0000 7ff185765740 1 -- start start 2024-04-17T07:26:15.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.660+0000 7ff185765740 1 Processor -- start 2024-04-17T07:26:15.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.661+0000 7ff185765740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 0x55ca819e4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.661+0000 7ff185765740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 0x55ca819e5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.662+0000 7ff185765740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819dacf0 0x55ca819e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.662+0000 7ff17be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 0x55ca819e4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.662+0000 7ff17be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 0x55ca819e4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37758/0 (socket says 172.21.15.86:37758) 2024-04-17T07:26:15.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.662+0000 7ff17be00640 1 -- 172.21.15.86:0/881179657 learned_addr learned my addr 172.21.15.86:0/881179657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:15.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.663+0000 7ff17b400640 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 0x55ca819e5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.663+0000 7ff180a00640 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819dacf0 0x55ca819e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.663+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1b2610 con 0x55ca819dacf0 2024-04-17T07:26:15.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.663+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1b1f40 con 0x55ca819d5190 2024-04-17T07:26:15.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.663+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c116670 con 0x55ca819d5b80 2024-04-17T07:26:15.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.663+0000 7ff185765740 10 client.0 _ll_register_callbacks cb 0x7ff174009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:15.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.664+0000 7ff180a00640 1 -- 172.21.15.86:0/881179657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 msgr2=0x55ca819e5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.664+0000 7ff180a00640 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 0x55ca819e5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.664+0000 7ff180a00640 1 -- 172.21.15.86:0/881179657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 msgr2=0x55ca819e4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:15.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.664+0000 7ff180a00640 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 0x55ca819e4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.664+0000 7ff180a00640 1 -- 172.21.15.86:0/881179657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca819e65b0 con 0x55ca819dacf0 2024-04-17T07:26:15.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.664+0000 7ff180a00640 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819dacf0 0x55ca819e5fc0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff17402e570 tx=0x7ff174009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7768f20cb834617d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:15.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.665+0000 7ff165600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:15.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.665+0000 7ff165600640 1 -- 172.21.15.86:0/881179657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174032040 con 0x55ca819dacf0 2024-04-17T07:26:15.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.665+0000 7ff165600640 1 -- 172.21.15.86:0/881179657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff17402edb0 con 0x55ca819dacf0 2024-04-17T07:26:15.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.665+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16c0054f0 con 0x55ca819dacf0 2024-04-17T07:26:15.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.665+0000 7ff165600640 1 -- 172.21.15.86:0/881179657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff17402e9f0 con 0x55ca819dacf0 2024-04-17T07:26:15.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.665+0000 7ff185765740 10 client.18414 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:15.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.665+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca81a62e90 con 0x55ca819dacf0 2024-04-17T07:26:15.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.666+0000 7ff165600640 1 -- 172.21.15.86:0/881179657 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff16c0054f0 con 0x55ca819dacf0 2024-04-17T07:26:15.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.667+0000 7ff165600640 1 -- 172.21.15.86:0/881179657 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff17403f070 con 0x55ca819dacf0 2024-04-17T07:26:15.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.667+0000 7ff185765740 10 client.18414 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:15.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.667+0000 7ff185765740 10 client.18414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:15.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.667+0000 7ff185765740 20 client.18414 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:15.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.667+0000 7ff185765740 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ca81a63680 0x55ca81a65aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:15.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.667+0000 7ff185765740 4 client.18414 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:26:15.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.667+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca819439b0 con 0x55ca81a63680 2024-04-17T07:26:15.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.668+0000 7ff17be00640 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ca81a63680 0x55ca81a65aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:15.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.669+0000 7ff17be00640 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ca81a63680 0x55ca81a65aa0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1e8aa837cc21ee server_cookie=d38cd046354bb15e in_seq=0 out_seq=0 2024-04-17T07:26:15.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.669+0000 7ff165600640 10 client.18414 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:15.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.670+0000 7ff165600640 1 -- 172.21.15.86:0/881179657 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ca819439b0 con 0x55ca81a63680 2024-04-17T07:26:15.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.670+0000 7ff165600640 10 client.18414 handle_command_reply: tid=0 2024-04-17T07:26:15.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.670+0000 7ff185765740 1 client.18414 shutdown 2024-04-17T07:26:15.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.671+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819dacf0 msgr2=0x55ca819e5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.671+0000 7ff185765740 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819dacf0 0x55ca819e5fc0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff17402e570 tx=0x7ff174009750 comp rx=0 tx=0).stop 2024-04-17T07:26:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.674+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 shutdown_connections 2024-04-17T07:26:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.674+0000 7ff185765740 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ca81a63680 0x55ca81a65aa0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.674+0000 7ff185765740 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca819dacf0 0x55ca819e5fc0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.674+0000 7ff185765740 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca819d5b80 0x55ca819e5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.674+0000 7ff185765740 1 --2- 172.21.15.86:0/881179657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ca819d5190 0x55ca819e4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.674+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 >> 172.21.15.86:0/881179657 conn(0x55ca819d0700 msgr2=0x55ca819cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.674+0000 7ff165600640 0 client.18414 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.674+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 shutdown_connections 2024-04-17T07:26:15.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.675+0000 7ff185765740 1 -- 172.21.15.86:0/881179657 wait complete. 2024-04-17T07:26:15.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.675+0000 7ff185765740 20 client.18414 trim_cache size 0 max 16384 2024-04-17T07:26:15.676 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:26:15.676 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:26:15.676 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:26:15.676 INFO:teuthology.orchestra.run.smithi086.stdout: "26b32fb6-6916-4e04-849b-3b545d732fb8": { 2024-04-17T07:26:15.676 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:26:15.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "26b32fb6-6916-4e04-849b-3b545d732fb8", 2024-04-17T07:26:15.676 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:26:15.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:26:15.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:26:15.677 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:26:15.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.676+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1600a3c40 msgr2=0x7ff1600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.676+0000 7ff182a00640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1600a3c40 0x7ff1600a6100 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff174011fd0 tx=0x7ff174005fd0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.676+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff17c1053b0 msgr2=0x7ff17c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.676+0000 7ff182a00640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff17c1053b0 0x7ff17c06ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff16800e9f0 tx=0x7ff16800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.676+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 shutdown_connections 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.676+0000 7ff182a00640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1600a3c40 0x7ff1600a6100 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.676+0000 7ff182a00640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff17c105d70 0x7ff17c1ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.676+0000 7ff182a00640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff17c1053b0 0x7ff17c06ede0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.676+0000 7ff182a00640 1 --2- 172.21.15.86:0/3933191228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff17c1041c0 0x7ff17c06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.676+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 >> 172.21.15.86:0/3933191228 conn(0x7ff17c0ff710 msgr2=0x7ff17c1065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.677+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 shutdown_connections 2024-04-17T07:26:15.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:15.677+0000 7ff182a00640 1 -- 172.21.15.86:0/3933191228 wait complete. 2024-04-17T07:26:15.711 INFO:tasks.cephfs.filesystem:scrub status for tag:26b32fb6-6916-4e04-849b-3b545d732fb8 - {'path': '/', 'tag': '26b32fb6-6916-4e04-849b-3b545d732fb8', 'options': 'recursive,force'} 2024-04-17T07:26:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:15 smithi098 ceph-mon[35418]: pgmap v796: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.1 MiB/s wr, 696 op/s 2024-04-17T07:26:15.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:15 smithi103 ceph-mon[35495]: pgmap v796: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.1 MiB/s wr, 696 op/s 2024-04-17T07:26:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:16 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:26:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:16 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:16 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:26:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:16 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:16.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:16 smithi098 ceph-mon[35418]: pgmap v797: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 637 op/s 2024-04-17T07:26:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:16 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:26:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:16 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:16 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:26:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:16 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:16 smithi103 ceph-mon[35495]: pgmap v797: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 637 op/s 2024-04-17T07:26:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:16 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:26:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:16 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:16.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:16 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:26:16.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:16 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:16.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:16 smithi086 ceph-mon[31733]: pgmap v797: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 637 op/s 2024-04-17T07:26:18.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:18 smithi086 ceph-mon[31733]: pgmap v798: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 728 op/s 2024-04-17T07:26:19.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:18 smithi098 ceph-mon[35418]: pgmap v798: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 728 op/s 2024-04-17T07:26:19.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:18 smithi103 ceph-mon[35495]: pgmap v798: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 728 op/s 2024-04-17T07:26:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 415.0 s, 667.4 tps, lat 150.836 ms stddev 67.550 2024-04-17T07:26:19.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:19 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:26:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:19 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:26:20.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:19 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:26:20.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:20 smithi086 ceph-mon[31733]: pgmap v799: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.8 MiB/s wr, 675 op/s 2024-04-17T07:26:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:20 smithi098 ceph-mon[35418]: pgmap v799: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.8 MiB/s wr, 675 op/s 2024-04-17T07:26:21.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:20 smithi103 ceph-mon[35495]: pgmap v799: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.8 MiB/s wr, 675 op/s 2024-04-17T07:26:22.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:22 smithi086 ceph-mon[31733]: pgmap v800: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 739 op/s 2024-04-17T07:26:23.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:22 smithi098 ceph-mon[35418]: pgmap v800: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 739 op/s 2024-04-17T07:26:23.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:22 smithi103 ceph-mon[35495]: pgmap v800: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 739 op/s 2024-04-17T07:26:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 420.0 s, 786.8 tps, lat 127.807 ms stddev 41.396 2024-04-17T07:26:24.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:24 smithi086 ceph-mon[31733]: pgmap v801: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-04-17T07:26:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:24 smithi098 ceph-mon[35418]: pgmap v801: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-04-17T07:26:25.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:24 smithi103 ceph-mon[35495]: pgmap v801: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-04-17T07:26:27.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:26 smithi098 ceph-mon[35418]: pgmap v802: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 655 op/s 2024-04-17T07:26:27.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:26 smithi103 ceph-mon[35495]: pgmap v802: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 655 op/s 2024-04-17T07:26:27.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:26 smithi086 ceph-mon[31733]: pgmap v802: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 655 op/s 2024-04-17T07:26:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:28 smithi098 ceph-mon[35418]: pgmap v803: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 756 op/s 2024-04-17T07:26:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:28 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:26:29.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:28 smithi103 ceph-mon[35495]: pgmap v803: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 756 op/s 2024-04-17T07:26:29.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:28 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:26:29.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:28 smithi086 ceph-mon[31733]: pgmap v803: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 756 op/s 2024-04-17T07:26:29.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:28 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:26:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 425.0 s, 649.0 tps, lat 153.510 ms stddev 126.170 2024-04-17T07:26:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:30 smithi098 ceph-mon[35418]: pgmap v804: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 730 op/s 2024-04-17T07:26:31.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:30 smithi103 ceph-mon[35495]: pgmap v804: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 730 op/s 2024-04-17T07:26:31.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:30 smithi086 ceph-mon[31733]: pgmap v804: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 730 op/s 2024-04-17T07:26:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:32 smithi098 ceph-mon[35418]: pgmap v805: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 762 op/s 2024-04-17T07:26:33.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:32 smithi103 ceph-mon[35495]: pgmap v805: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 762 op/s 2024-04-17T07:26:33.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:32 smithi086 ceph-mon[31733]: pgmap v805: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 762 op/s 2024-04-17T07:26:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:26:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:26:34.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:33 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:26:34.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:26:34.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:26:34.030 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:33 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:26:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:26:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:26:34.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:33 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:26:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 430.0 s, 658.8 tps, lat 151.995 ms stddev 65.337 2024-04-17T07:26:35.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:35 smithi098 ceph-mon[35418]: pgmap v806: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 12 MiB/s wr, 750 op/s 2024-04-17T07:26:35.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:35 smithi103 ceph-mon[35495]: pgmap v806: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 12 MiB/s wr, 750 op/s 2024-04-17T07:26:35.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:35 smithi086 ceph-mon[31733]: pgmap v806: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 12 MiB/s wr, 750 op/s 2024-04-17T07:26:37.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:37 smithi103 ceph-mon[35495]: pgmap v807: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 10 MiB/s wr, 652 op/s 2024-04-17T07:26:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:37 smithi086 ceph-mon[31733]: pgmap v807: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 10 MiB/s wr, 652 op/s 2024-04-17T07:26:37.507 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:37 smithi098 ceph-mon[35418]: pgmap v807: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 10 MiB/s wr, 652 op/s 2024-04-17T07:26:39.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:39 smithi086 ceph-mon[31733]: pgmap v808: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.9 MiB/s wr, 701 op/s 2024-04-17T07:26:39.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:39 smithi103 ceph-mon[35495]: pgmap v808: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.9 MiB/s wr, 701 op/s 2024-04-17T07:26:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 435.0 s, 728.6 tps, lat 136.605 ms stddev 52.151 2024-04-17T07:26:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:39 smithi098 ceph-mon[35418]: pgmap v808: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.9 MiB/s wr, 701 op/s 2024-04-17T07:26:41.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:41 smithi103 ceph-mon[35495]: pgmap v809: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.5 MiB/s wr, 643 op/s 2024-04-17T07:26:41.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:41 smithi086 ceph-mon[31733]: pgmap v809: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.5 MiB/s wr, 643 op/s 2024-04-17T07:26:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:41 smithi098 ceph-mon[35418]: pgmap v809: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.5 MiB/s wr, 643 op/s 2024-04-17T07:26:43.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:43 smithi086 ceph-mon[31733]: pgmap v810: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.6 MiB/s wr, 661 op/s 2024-04-17T07:26:43.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:43 smithi098 ceph-mon[35418]: pgmap v810: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.6 MiB/s wr, 661 op/s 2024-04-17T07:26:43.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:43 smithi103 ceph-mon[35495]: pgmap v810: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.6 MiB/s wr, 661 op/s 2024-04-17T07:26:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 440.0 s, 759.6 tps, lat 131.850 ms stddev 44.035 2024-04-17T07:26:45.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:45 smithi086 ceph-mon[31733]: pgmap v811: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 10 MiB/s wr, 676 op/s 2024-04-17T07:26:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:45 smithi098 ceph-mon[35418]: pgmap v811: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 10 MiB/s wr, 676 op/s 2024-04-17T07:26:45.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:45 smithi103 ceph-mon[35495]: pgmap v811: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 10 MiB/s wr, 676 op/s 2024-04-17T07:26:45.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:26:45.712 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:26:47.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:47 smithi086 ceph-mon[31733]: pgmap v812: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.3 MiB/s wr, 603 op/s 2024-04-17T07:26:47.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.402+0000 7fcc8e400640 1 -- 172.21.15.86:0/1049123179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800945f0 msgr2=0x7fcc80096a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.402+0000 7fcc8e400640 1 --2- 172.21.15.86:0/1049123179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800945f0 0x7fcc80096a20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009900 tx=0x7fcc7c02bc50 comp rx=0 tx=0).stop 2024-04-17T07:26:47.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.402+0000 7fcc8e400640 1 -- 172.21.15.86:0/1049123179 shutdown_connections 2024-04-17T07:26:47.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.402+0000 7fcc8e400640 1 --2- 172.21.15.86:0/1049123179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc80099910 0x7fcc80099d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.402+0000 7fcc8e400640 1 --2- 172.21.15.86:0/1049123179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc80096f60 0x7fcc800993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.402+0000 7fcc8e400640 1 --2- 172.21.15.86:0/1049123179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800945f0 0x7fcc80096a20 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.402+0000 7fcc8e400640 1 -- 172.21.15.86:0/1049123179 >> 172.21.15.86:0/1049123179 conn(0x7fcc800901d0 msgr2=0x7fcc80092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:47.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.403+0000 7fcc8e400640 1 -- 172.21.15.86:0/1049123179 shutdown_connections 2024-04-17T07:26:47.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.403+0000 7fcc8e400640 1 -- 172.21.15.86:0/1049123179 wait complete. 2024-04-17T07:26:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.404+0000 7fcc8e400640 1 Processor -- start 2024-04-17T07:26:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.405+0000 7fcc8e400640 1 -- start start 2024-04-17T07:26:47.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.405+0000 7fcc8e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800945f0 0x7fcc8009b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.406+0000 7fcc8e400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc80096f60 0x7fcc8009bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.406+0000 7fcc8e400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc80099910 0x7fcc800a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.406+0000 7fcc8e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc800a7f20 con 0x7fcc800945f0 2024-04-17T07:26:47.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.406+0000 7fcc8e400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcc800a7da0 con 0x7fcc80096f60 2024-04-17T07:26:47.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.406+0000 7fcc8e400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc800a80a0 con 0x7fcc80099910 2024-04-17T07:26:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.406+0000 7fcc8d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800945f0 0x7fcc8009b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.406+0000 7fcc87e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc80096f60 0x7fcc8009bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.407+0000 7fcc8d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800945f0 0x7fcc8009b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53682/0 (socket says 172.21.15.86:53682) 2024-04-17T07:26:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.407+0000 7fcc8d000640 1 -- 172.21.15.86:0/337027343 learned_addr learned my addr 172.21.15.86:0/337027343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.407+0000 7fcc8da00640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc80099910 0x7fcc800a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.407+0000 7fcc8d000640 1 -- 172.21.15.86:0/337027343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc80099910 msgr2=0x7fcc800a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.408+0000 7fcc8d000640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc80099910 0x7fcc800a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.408+0000 7fcc8d000640 1 -- 172.21.15.86:0/337027343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc80096f60 msgr2=0x7fcc8009bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.408+0000 7fcc8d000640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc80096f60 0x7fcc8009bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.408+0000 7fcc8d000640 1 -- 172.21.15.86:0/337027343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc800a3190 con 0x7fcc800945f0 2024-04-17T07:26:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.409+0000 7fcc8d000640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800945f0 0x7fcc8009b5c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02c630 tx=0x7fcc7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.410+0000 7fcc85600640 1 -- 172.21.15.86:0/337027343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c043020 con 0x7fcc800945f0 2024-04-17T07:26:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.410+0000 7fcc85600640 1 -- 172.21.15.86:0/337027343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc7c004aa0 con 0x7fcc800945f0 2024-04-17T07:26:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.410+0000 7fcc85600640 1 -- 172.21.15.86:0/337027343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c003c80 con 0x7fcc800945f0 2024-04-17T07:26:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.410+0000 7fcc8e400640 1 -- 172.21.15.86:0/337027343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc80142c00 con 0x7fcc800945f0 2024-04-17T07:26:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.412+0000 7fcc8e400640 1 -- 172.21.15.86:0/337027343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc80143100 con 0x7fcc800945f0 2024-04-17T07:26:47.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.418+0000 7fcc927be740 1 Processor -- start 2024-04-17T07:26:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.418+0000 7fcc927be740 1 -- start start 2024-04-17T07:26:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.419+0000 7fcc927be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.419+0000 7fcc927be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x5642678f4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.419+0000 7fcc927be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x5642678f7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.420+0000 7fcc927be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564267856120 con 0x5642678f4ed0 2024-04-17T07:26:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.420+0000 7fcc927be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5642677fc340 con 0x5642678ef370 2024-04-17T07:26:47.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.420+0000 7fcc927be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5642678520e0 con 0x5642678efd60 2024-04-17T07:26:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.420+0000 7fcc8da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.420+0000 7fcc8da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678ef790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43726/0 (socket says 172.21.15.86:43726) 2024-04-17T07:26:47.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.420+0000 7fcc8da00640 1 -- 172.21.15.86:0/2940498440 learned_addr learned my addr 172.21.15.86:0/2940498440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.421+0000 7fcc8da00640 1 -- 172.21.15.86:0/2940498440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 msgr2=0x5642678f4860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:26:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.421+0000 7fcc8da00640 1 --2- 172.21.15.86:0/2940498440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x5642678f4860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.421+0000 7fcc8da00640 1 -- 172.21.15.86:0/2940498440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 msgr2=0x5642678f7360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:26:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.421+0000 7fcc87e00640 1 --2- 172.21.15.86:0/2940498440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x5642678f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.422+0000 7fcc8da00640 1 --2- 172.21.15.86:0/2940498440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x5642678f7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.422+0000 7fcc8da00640 1 -- 172.21.15.86:0/2940498440 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642676327d0 con 0x5642678ef370 2024-04-17T07:26:47.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.422+0000 7fcc87e00640 1 --2- 172.21.15.86:0/2940498440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x5642678f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:47.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.423+0000 7fcc8da00640 1 --2- 172.21.15.86:0/2940498440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678ef790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcc88050aa0 tx=0x7fcc88050d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13d8dd97b64b47d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.425+0000 7fcc85600640 1 -- 172.21.15.86:0/337027343 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fcc7c046020 con 0x7fcc800945f0 2024-04-17T07:26:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.427+0000 7fcc7a000640 1 -- 172.21.15.86:0/2940498440 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8806a0d0 con 0x5642678ef370 2024-04-17T07:26:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.428+0000 7fcc7a000640 1 -- 172.21.15.86:0/2940498440 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc8806a270 con 0x5642678ef370 2024-04-17T07:26:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.428+0000 7fcc7a000640 1 -- 172.21.15.86:0/2940498440 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8807a780 con 0x5642678ef370 2024-04-17T07:26:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.429+0000 7fcc927be740 1 -- 172.21.15.86:0/2940498440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 msgr2=0x5642678ef790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.429+0000 7fcc927be740 1 --2- 172.21.15.86:0/2940498440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678ef790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcc88050aa0 tx=0x7fcc88050d70 comp rx=0 tx=0).stop 2024-04-17T07:26:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.430+0000 7fcc927be740 1 -- 172.21.15.86:0/2940498440 shutdown_connections 2024-04-17T07:26:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.430+0000 7fcc927be740 1 --2- 172.21.15.86:0/2940498440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x5642678f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.430+0000 7fcc927be740 1 --2- 172.21.15.86:0/2940498440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x5642678f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.430+0000 7fcc927be740 1 --2- 172.21.15.86:0/2940498440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678ef790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.430+0000 7fcc927be740 1 -- 172.21.15.86:0/2940498440 >> 172.21.15.86:0/2940498440 conn(0x5642678ea8e0 msgr2=0x5642678eace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.430+0000 7fcc927be740 1 -- 172.21.15.86:0/2940498440 shutdown_connections 2024-04-17T07:26:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.430+0000 7fcc927be740 1 -- 172.21.15.86:0/2940498440 wait complete. 2024-04-17T07:26:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.431+0000 7fcc85600640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc700a3c20 0x7fcc700a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.431+0000 7fcc927be740 1 -- start start 2024-04-17T07:26:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.431+0000 7fcc85600640 1 -- 172.21.15.86:0/337027343 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcc7c0f22e0 con 0x7fcc800945f0 2024-04-17T07:26:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.431+0000 7fcc927be740 1 Processor -- start 2024-04-17T07:26:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.432+0000 7fcc8da00640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc700a3c20 0x7fcc700a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.432+0000 7fcc927be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678ef370 0x5642678f9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.432+0000 7fcc927be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678efd60 0x5642678f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.432+0000 7fcc927be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x564267900c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.432+0000 7fcc927be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642678f7e80 con 0x5642678f4ed0 2024-04-17T07:26:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.432+0000 7fcc927be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56426780f710 con 0x5642678efd60 2024-04-17T07:26:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.432+0000 7fcc927be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5642678f8000 con 0x5642678ef370 2024-04-17T07:26:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.432+0000 7fcc927be740 10 client.0 _ll_register_callbacks cb 0x56426780aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.433+0000 7fcc87e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678efd60 0x5642678f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.433+0000 7fcc87e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678efd60 0x5642678f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43736/0 (socket says 172.21.15.86:43736) 2024-04-17T07:26:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.433+0000 7fcc87e00640 1 -- 172.21.15.86:0/1415152936 learned_addr learned my addr 172.21.15.86:0/1415152936 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.433+0000 7fcc8d000640 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678ef370 0x5642678f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.436+0000 7fcc87e00640 1 -- 172.21.15.86:0/1415152936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678ef370 msgr2=0x5642678f9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.436+0000 7fcc87e00640 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678ef370 0x5642678f9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.436+0000 7fcc87e00640 1 -- 172.21.15.86:0/1415152936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 msgr2=0x564267900c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:26:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.437+0000 7fcc8da00640 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x564267900c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.437+0000 7fcc87e00640 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x564267900c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.437+0000 7fcc87e00640 1 -- 172.21.15.86:0/1415152936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642675683f0 con 0x5642678efd60 2024-04-17T07:26:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.437+0000 7fcc6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:26:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.438+0000 7fcc87e00640 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678efd60 0x5642678f9c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc740136e0 tx=0x7fcc74013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b97c32e3abf42d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.438+0000 7fcc6ea00640 1 -- 172.21.15.86:0/1415152936 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74004460 con 0x5642678efd60 2024-04-17T07:26:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.438+0000 7fcc6ea00640 1 -- 172.21.15.86:0/1415152936 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcc740119c0 con 0x5642678efd60 2024-04-17T07:26:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.439+0000 7fcc8da00640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc700a3c20 0x7fcc700a60e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc8804ffc0 tx=0x7fcc88067e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.439+0000 7fcc927be740 1 -- 172.21.15.86:0/1415152936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564267863b40 con 0x5642678efd60 2024-04-17T07:26:47.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.439+0000 7fcc6ea00640 1 -- 172.21.15.86:0/1415152936 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcc74011bc0 con 0x5642678efd60 2024-04-17T07:26:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.439+0000 7fcc927be740 10 client.37295 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.439+0000 7fcc927be740 1 -- 172.21.15.86:0/1415152936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642675280f0 con 0x5642678efd60 2024-04-17T07:26:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.440+0000 7fcc6ea00640 1 -- 172.21.15.86:0/1415152936 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564267863b40 con 0x5642678efd60 2024-04-17T07:26:47.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.440+0000 7fcc6ea00640 1 -- 172.21.15.86:0/1415152936 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcc7401a070 con 0x5642678efd60 2024-04-17T07:26:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.440+0000 7fcc927be740 10 client.37295 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.440+0000 7fcc927be740 10 client.37295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.440+0000 7fcc927be740 20 client.37295 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.441+0000 7fcc927be740 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564267905580 0x5642679079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.441+0000 7fcc927be740 4 client.37295 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.441+0000 7fcc927be740 1 -- 172.21.15.86:0/1415152936 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56426785d9b0 con 0x564267905580 2024-04-17T07:26:47.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.441+0000 7fcc8d000640 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564267905580 0x5642679079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.443+0000 7fcc8d000640 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564267905580 0x5642679079a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e823874f489985c server_cookie=420f73690089f24 in_seq=0 out_seq=0 2024-04-17T07:26:47.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.443+0000 7fcc6ea00640 10 client.37295 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.445+0000 7fcc6ea00640 1 -- 172.21.15.86:0/1415152936 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56426785d9b0 con 0x564267905580 2024-04-17T07:26:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.445+0000 7fcc6ea00640 10 client.37295 handle_command_reply: tid=0 2024-04-17T07:26:47.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.445+0000 7fcc927be740 1 client.37295 shutdown 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.446+0000 7fcc927be740 1 -- 172.21.15.86:0/1415152936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678efd60 msgr2=0x5642678f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.446+0000 7fcc927be740 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678efd60 0x5642678f9c40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc740136e0 tx=0x7fcc74013ba0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.446+0000 7fcc927be740 1 -- 172.21.15.86:0/1415152936 shutdown_connections 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.446+0000 7fcc927be740 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564267905580 0x5642679079a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.446+0000 7fcc927be740 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x564267900c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.446+0000 7fcc6ea00640 0 client.37295 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.446+0000 7fcc927be740 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678efd60 0x5642678f9c40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.446+0000 7fcc927be740 1 --2- 172.21.15.86:0/1415152936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678ef370 0x5642678f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.446+0000 7fcc927be740 1 -- 172.21.15.86:0/1415152936 >> 172.21.15.86:0/1415152936 conn(0x5642678ea8e0 msgr2=0x5642678e9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.447+0000 7fcc927be740 1 -- 172.21.15.86:0/1415152936 shutdown_connections 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.447+0000 7fcc927be740 1 -- 172.21.15.86:0/1415152936 wait complete. 2024-04-17T07:26:47.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.447+0000 7fcc927be740 20 client.37295 trim_cache size 0 max 16384 2024-04-17T07:26:47.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.494+0000 7fcc927be740 1 Processor -- start 2024-04-17T07:26:47.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.494+0000 7fcc927be740 1 -- start start 2024-04-17T07:26:47.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.495+0000 7fcc927be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.495+0000 7fcc927be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x564267900bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.495+0000 7fcc927be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x5642678f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.495+0000 7fcc927be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8014d530 con 0x5642678f4ed0 2024-04-17T07:26:47.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.495+0000 7fcc927be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8014f070 con 0x5642678ef370 2024-04-17T07:26:47.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.495+0000 7fcc927be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8014dc00 con 0x5642678efd60 2024-04-17T07:26:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.495+0000 7fcc8d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.495+0000 7fcc87e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x564267900bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.495+0000 7fcc8d000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678f89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43742/0 (socket says 172.21.15.86:43742) 2024-04-17T07:26:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.495+0000 7fcc8d000640 1 -- 172.21.15.86:0/823561328 learned_addr learned my addr 172.21.15.86:0/823561328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.496+0000 7fcc8d000640 1 -- 172.21.15.86:0/823561328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 msgr2=0x564267900bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.496+0000 7fcc8d000640 1 --2- 172.21.15.86:0/823561328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x564267900bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.496+0000 7fcc8d000640 1 -- 172.21.15.86:0/823561328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 msgr2=0x5642678f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:47.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.496+0000 7fcc8da00640 1 --2- 172.21.15.86:0/823561328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x5642678f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.498+0000 7fcc8d000640 1 --2- 172.21.15.86:0/823561328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x5642678f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.498+0000 7fcc8d000640 1 -- 172.21.15.86:0/823561328 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564267907d10 con 0x5642678ef370 2024-04-17T07:26:47.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.498+0000 7fcc8da00640 1 --2- 172.21.15.86:0/823561328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x5642678f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.499+0000 7fcc8d000640 1 --2- 172.21.15.86:0/823561328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678f89d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0efab0 tx=0x7fcc7c0f1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e54cfeef2eb9bb6d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.499+0000 7fcc79600640 1 -- 172.21.15.86:0/823561328 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c003e20 con 0x5642678ef370 2024-04-17T07:26:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.499+0000 7fcc79600640 1 -- 172.21.15.86:0/823561328 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc7c0be240 con 0x5642678ef370 2024-04-17T07:26:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.499+0000 7fcc79600640 1 -- 172.21.15.86:0/823561328 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c0bc540 con 0x5642678ef370 2024-04-17T07:26:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.500+0000 7fcc927be740 1 -- 172.21.15.86:0/823561328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 msgr2=0x5642678f89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.500+0000 7fcc927be740 1 --2- 172.21.15.86:0/823561328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678f89d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0efab0 tx=0x7fcc7c0f1040 comp rx=0 tx=0).stop 2024-04-17T07:26:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.501+0000 7fcc927be740 1 -- 172.21.15.86:0/823561328 shutdown_connections 2024-04-17T07:26:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.501+0000 7fcc927be740 1 --2- 172.21.15.86:0/823561328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678f4ed0 0x5642678f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.501+0000 7fcc927be740 1 --2- 172.21.15.86:0/823561328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x564267900bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.501+0000 7fcc927be740 1 --2- 172.21.15.86:0/823561328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678ef370 0x5642678f89d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.501+0000 7fcc927be740 1 -- 172.21.15.86:0/823561328 >> 172.21.15.86:0/823561328 conn(0x5642678ea8e0 msgr2=0x5642678f1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:47.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.501+0000 7fcc927be740 1 -- 172.21.15.86:0/823561328 shutdown_connections 2024-04-17T07:26:47.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.502+0000 7fcc927be740 1 -- 172.21.15.86:0/823561328 wait complete. 2024-04-17T07:26:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:47 smithi098 ceph-mon[35418]: pgmap v812: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.3 MiB/s wr, 603 op/s 2024-04-17T07:26:47.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.503+0000 7fcc927be740 1 -- start start 2024-04-17T07:26:47.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.503+0000 7fcc927be740 1 Processor -- start 2024-04-17T07:26:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.503+0000 7fcc927be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678ef370 0x5642678fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.503+0000 7fcc927be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x5642678fe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.504+0000 7fcc927be740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678f4ed0 0x5642678ff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.504+0000 7fcc927be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8014e9a0 con 0x5642678ef370 2024-04-17T07:26:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.504+0000 7fcc927be740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8014e2d0 con 0x5642678f4ed0 2024-04-17T07:26:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.504+0000 7fcc927be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8014b500 con 0x5642678efd60 2024-04-17T07:26:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.504+0000 7fcc927be740 10 client.0 _ll_register_callbacks cb 0x7fcc74009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.504+0000 7fcc8d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678ef370 0x5642678fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.504+0000 7fcc8d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678ef370 0x5642678fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53720/0 (socket says 172.21.15.86:53720) 2024-04-17T07:26:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.504+0000 7fcc8d000640 1 -- 172.21.15.86:0/3927074808 learned_addr learned my addr 172.21.15.86:0/3927074808 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.504+0000 7fcc8da00640 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678f4ed0 0x5642678ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.505+0000 7fcc87e00640 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x5642678fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.505+0000 7fcc87e00640 1 -- 172.21.15.86:0/3927074808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678f4ed0 msgr2=0x5642678ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.505+0000 7fcc87e00640 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678f4ed0 0x5642678ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.505+0000 7fcc87e00640 1 -- 172.21.15.86:0/3927074808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678ef370 msgr2=0x5642678fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.505+0000 7fcc87e00640 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678ef370 0x5642678fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.505+0000 7fcc87e00640 1 -- 172.21.15.86:0/3927074808 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642678ffc80 con 0x5642678efd60 2024-04-17T07:26:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.505+0000 7fcc8da00640 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678f4ed0 0x5642678ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.506+0000 7fcc8d000640 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678ef370 0x5642678fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.506+0000 7fcc87e00640 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x5642678fe680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcc7401b700 tx=0x7fcc740117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af0a43b0599da342 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.506+0000 7fcc6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.507+0000 7fcc6ea00640 1 -- 172.21.15.86:0/3927074808 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7402e070 con 0x5642678efd60 2024-04-17T07:26:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.507+0000 7fcc6ea00640 1 -- 172.21.15.86:0/3927074808 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcc7401b8f0 con 0x5642678efd60 2024-04-17T07:26:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.507+0000 7fcc927be740 1 -- 172.21.15.86:0/3927074808 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc7c0be240 con 0x5642678efd60 2024-04-17T07:26:47.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.507+0000 7fcc6ea00640 1 -- 172.21.15.86:0/3927074808 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcc7403a050 con 0x5642678efd60 2024-04-17T07:26:47.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.507+0000 7fcc927be740 10 client.27379 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:47.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.507+0000 7fcc927be740 1 -- 172.21.15.86:0/3927074808 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564267900f70 con 0x5642678efd60 2024-04-17T07:26:47.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.508+0000 7fcc6ea00640 1 -- 172.21.15.86:0/3927074808 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc7c0be240 con 0x5642678efd60 2024-04-17T07:26:47.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.508+0000 7fcc6ea00640 1 -- 172.21.15.86:0/3927074808 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcc7401a050 con 0x5642678efd60 2024-04-17T07:26:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.508+0000 7fcc927be740 10 client.27379 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.508+0000 7fcc927be740 10 client.27379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.508+0000 7fcc927be740 20 client.27379 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.508+0000 7fcc927be740 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564267905580 0x56426796e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.508+0000 7fcc927be740 4 client.27379 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:26:47.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.508+0000 7fcc927be740 1 -- 172.21.15.86:0/3927074808 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5642677d9420 con 0x564267905580 2024-04-17T07:26:47.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.510+0000 7fcc8d000640 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564267905580 0x56426796e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.511+0000 7fcc8d000640 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564267905580 0x56426796e390 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6bc1804b2ac9b24 server_cookie=a335d1deb7f63a6b in_seq=0 out_seq=0 2024-04-17T07:26:47.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.511+0000 7fcc6ea00640 10 client.27379 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:47.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.511+0000 7fcc6ea00640 1 -- 172.21.15.86:0/3927074808 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5642677d9420 con 0x564267905580 2024-04-17T07:26:47.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.511+0000 7fcc6ea00640 10 client.27379 handle_command_reply: tid=0 2024-04-17T07:26:47.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.511+0000 7fcc927be740 1 client.27379 shutdown 2024-04-17T07:26:47.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.512+0000 7fcc927be740 1 -- 172.21.15.86:0/3927074808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 msgr2=0x5642678fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.512+0000 7fcc927be740 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x5642678fe680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcc7401b700 tx=0x7fcc740117b0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.512+0000 7fcc927be740 1 -- 172.21.15.86:0/3927074808 shutdown_connections 2024-04-17T07:26:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.512+0000 7fcc927be740 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564267905580 0x56426796e390 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.512+0000 7fcc927be740 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642678f4ed0 0x5642678ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.512+0000 7fcc927be740 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642678efd60 0x5642678fe680 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.512+0000 7fcc927be740 1 --2- 172.21.15.86:0/3927074808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642678ef370 0x5642678fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.512+0000 7fcc927be740 1 -- 172.21.15.86:0/3927074808 >> 172.21.15.86:0/3927074808 conn(0x5642678ea8e0 msgr2=0x5642678f89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:47.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.512+0000 7fcc6ea00640 0 client.27379 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:47.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.513+0000 7fcc927be740 1 -- 172.21.15.86:0/3927074808 shutdown_connections 2024-04-17T07:26:47.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.513+0000 7fcc927be740 1 -- 172.21.15.86:0/3927074808 wait complete. 2024-04-17T07:26:47.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.513+0000 7fcc927be740 20 client.27379 trim_cache size 0 max 16384 2024-04-17T07:26:47.514 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:26:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:26:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:26:47.514 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:26:47.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.514+0000 7fcc8e400640 1 -- 172.21.15.86:0/337027343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc700a3c20 msgr2=0x7fcc700a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.514+0000 7fcc8e400640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc700a3c20 0x7fcc700a60e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcc8804ffc0 tx=0x7fcc88067e30 comp rx=0 tx=0).stop 2024-04-17T07:26:47.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.514+0000 7fcc8e400640 1 -- 172.21.15.86:0/337027343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800945f0 msgr2=0x7fcc8009b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.514+0000 7fcc8e400640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800945f0 0x7fcc8009b5c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02c630 tx=0x7fcc7c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.515+0000 7fcc8e400640 1 -- 172.21.15.86:0/337027343 shutdown_connections 2024-04-17T07:26:47.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.515+0000 7fcc8e400640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc700a3c20 0x7fcc700a60e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.515+0000 7fcc8e400640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc80099910 0x7fcc800a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.515+0000 7fcc8e400640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc80096f60 0x7fcc8009bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.515+0000 7fcc8e400640 1 --2- 172.21.15.86:0/337027343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc800945f0 0x7fcc8009b5c0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.515+0000 7fcc8e400640 1 -- 172.21.15.86:0/337027343 >> 172.21.15.86:0/337027343 conn(0x7fcc800901d0 msgr2=0x7fcc80097d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.515+0000 7fcc8e400640 1 -- 172.21.15.86:0/337027343 shutdown_connections 2024-04-17T07:26:47.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.515+0000 7fcc8e400640 1 -- 172.21.15.86:0/337027343 wait complete. 2024-04-17T07:26:47.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:47 smithi103 ceph-mon[35495]: pgmap v812: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.3 MiB/s wr, 603 op/s 2024-04-17T07:26:47.560 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:26:47.560 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:26:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.878+0000 7f32c3000640 1 -- 172.21.15.86:0/4143385252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc105d70 msgr2=0x7f32bc10c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.878+0000 7f32c3000640 1 --2- 172.21.15.86:0/4143385252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc105d70 0x7f32bc10c600 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009ca0 tx=0x7f32b402c100 comp rx=0 tx=0).stop 2024-04-17T07:26:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.879+0000 7f32c3000640 1 -- 172.21.15.86:0/4143385252 shutdown_connections 2024-04-17T07:26:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.879+0000 7f32c3000640 1 --2- 172.21.15.86:0/4143385252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc105d70 0x7f32bc10c600 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.879+0000 7f32c3000640 1 --2- 172.21.15.86:0/4143385252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc1053b0 0x7f32bc105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.879+0000 7f32c3000640 1 --2- 172.21.15.86:0/4143385252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32bc1041c0 0x7f32bc1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.879+0000 7f32c3000640 1 -- 172.21.15.86:0/4143385252 >> 172.21.15.86:0/4143385252 conn(0x7f32bc0ff750 msgr2=0x7f32bc101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:47.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.879+0000 7f32c3000640 1 -- 172.21.15.86:0/4143385252 shutdown_connections 2024-04-17T07:26:47.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.879+0000 7f32c3000640 1 -- 172.21.15.86:0/4143385252 wait complete. 2024-04-17T07:26:47.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.880+0000 7f32c3000640 1 Processor -- start 2024-04-17T07:26:47.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.881+0000 7f32c3000640 1 -- start start 2024-04-17T07:26:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.881+0000 7f32c3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc1041c0 0x7f32bc19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.881+0000 7f32c3000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32bc1053b0 0x7f32bc19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.881+0000 7f32c3000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc105d70 0x7f32bc1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.881+0000 7f32c3000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc10fcd0 con 0x7f32bc1041c0 2024-04-17T07:26:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.881+0000 7f32c3000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc10fb50 con 0x7f32bc1053b0 2024-04-17T07:26:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.881+0000 7f32c3000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc10fe50 con 0x7f32bc105d70 2024-04-17T07:26:47.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.882+0000 7f32bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc1041c0 0x7f32bc19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.882+0000 7f32bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc1041c0 0x7f32bc19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53740/0 (socket says 172.21.15.86:53740) 2024-04-17T07:26:47.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.882+0000 7f32bbe00640 1 -- 172.21.15.86:0/1823552381 learned_addr learned my addr 172.21.15.86:0/1823552381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:47.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.882+0000 7f32c1000640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc105d70 0x7f32bc1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.882+0000 7f32bb400640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32bc1053b0 0x7f32bc19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.883+0000 7f32bbe00640 1 -- 172.21.15.86:0/1823552381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc105d70 msgr2=0x7f32bc1a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.883+0000 7f32bbe00640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc105d70 0x7f32bc1a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.883+0000 7f32bbe00640 1 -- 172.21.15.86:0/1823552381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32bc1053b0 msgr2=0x7f32bc19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:47.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.883+0000 7f32bbe00640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32bc1053b0 0x7f32bc19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:47.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.883+0000 7f32bbe00640 1 -- 172.21.15.86:0/1823552381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32bc0763a0 con 0x7f32bc1041c0 2024-04-17T07:26:47.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.883+0000 7f32bbe00640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc1041c0 0x7f32bc19a400 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d770 tx=0x7f32ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:47.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.884+0000 7f32b8c00640 1 -- 172.21.15.86:0/1823552381 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac017070 con 0x7f32bc1041c0 2024-04-17T07:26:47.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.884+0000 7f32b8c00640 1 -- 172.21.15.86:0/1823552381 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32ac004220 con 0x7f32bc1041c0 2024-04-17T07:26:47.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.884+0000 7f32b8c00640 1 -- 172.21.15.86:0/1823552381 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac003c90 con 0x7f32bc1041c0 2024-04-17T07:26:47.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.884+0000 7f32c3000640 1 -- 172.21.15.86:0/1823552381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32bc076690 con 0x7f32bc1041c0 2024-04-17T07:26:47.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.884+0000 7f32c3000640 1 -- 172.21.15.86:0/1823552381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32bc076b90 con 0x7f32bc1041c0 2024-04-17T07:26:47.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.886+0000 7f32c3000640 1 -- 172.21.15.86:0/1823552381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32bc10b560 con 0x7f32bc1041c0 2024-04-17T07:26:47.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.889+0000 7f32b8c00640 1 -- 172.21.15.86:0/1823552381 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f32ac010460 con 0x7f32bc1041c0 2024-04-17T07:26:47.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.891+0000 7f32b8c00640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a00a3c80 0x7f32a00a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:47.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.892+0000 7f32b8c00640 1 -- 172.21.15.86:0/1823552381 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f32ac0d0330 con 0x7f32bc1041c0 2024-04-17T07:26:47.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.893+0000 7f32bb400640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a00a3c80 0x7f32a00a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:47.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.896+0000 7f32bb400640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a00a3c80 0x7f32a00a6140 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f32bc19b910 tx=0x7f32a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:47.900+0000 7f32b8c00640 1 -- 172.21.15.86:0/1823552381 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f32ac003e30 con 0x7f32bc1041c0 2024-04-17T07:26:48.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.410+0000 7f32c3000640 1 -- 172.21.15.86:0/1823552381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32bc02d5d0 con 0x7f32bc1041c0 2024-04-17T07:26:48.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.412+0000 7f32b8c00640 1 -- 172.21.15.86:0/1823552381 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f32bc02d5d0 con 0x7f32bc1041c0 2024-04-17T07:26:48.416 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:26:48.416 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:26:48.417 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:26:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.419+0000 7f32c3000640 1 -- 172.21.15.86:0/1823552381 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a00a3c80 msgr2=0x7f32a00a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.419+0000 7f32c3000640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a00a3c80 0x7f32a00a6140 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f32bc19b910 tx=0x7f32a400b390 comp rx=0 tx=0).stop 2024-04-17T07:26:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.419+0000 7f32c3000640 1 -- 172.21.15.86:0/1823552381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc1041c0 msgr2=0x7f32bc19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.419+0000 7f32c3000640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc1041c0 0x7f32bc19a400 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d770 tx=0x7f32ac00dc30 comp rx=0 tx=0).stop 2024-04-17T07:26:48.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.419+0000 7f32c3000640 1 -- 172.21.15.86:0/1823552381 shutdown_connections 2024-04-17T07:26:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.419+0000 7f32c3000640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a00a3c80 0x7f32a00a6140 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.419+0000 7f32c3000640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32bc105d70 0x7f32bc1a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.419+0000 7f32c3000640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32bc1053b0 0x7f32bc19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.419+0000 7f32c3000640 1 --2- 172.21.15.86:0/1823552381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32bc1041c0 0x7f32bc19a400 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.419+0000 7f32c3000640 1 -- 172.21.15.86:0/1823552381 >> 172.21.15.86:0/1823552381 conn(0x7f32bc0ff750 msgr2=0x7f32bc108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.420+0000 7f32c3000640 1 -- 172.21.15.86:0/1823552381 shutdown_connections 2024-04-17T07:26:48.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.420+0000 7f32c3000640 1 -- 172.21.15.86:0/1823552381 wait complete. 2024-04-17T07:26:48.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:26:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.745+0000 7f93fea00640 1 -- 172.21.15.86:0/3648387102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8077b60 msgr2=0x7f93f8077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.745+0000 7f93fea00640 1 --2- 172.21.15.86:0/3648387102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8077b60 0x7f93f8077fc0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009860 tx=0x7f93e802bb70 comp rx=0 tx=0).stop 2024-04-17T07:26:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.745+0000 7f93fea00640 1 -- 172.21.15.86:0/3648387102 shutdown_connections 2024-04-17T07:26:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.745+0000 7f93fea00640 1 --2- 172.21.15.86:0/3648387102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8078500 0x7f93f8078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.745+0000 7f93fea00640 1 --2- 172.21.15.86:0/3648387102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8077b60 0x7f93f8077fc0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.745+0000 7f93fea00640 1 --2- 172.21.15.86:0/3648387102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93f8076920 0x7f93f8076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.745+0000 7f93fea00640 1 -- 172.21.15.86:0/3648387102 >> 172.21.15.86:0/3648387102 conn(0x7f93f80ff710 msgr2=0x7f93f8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:48.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.747+0000 7f93fea00640 1 -- 172.21.15.86:0/3648387102 shutdown_connections 2024-04-17T07:26:48.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.749+0000 7f93fea00640 1 -- 172.21.15.86:0/3648387102 wait complete. 2024-04-17T07:26:48.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.750+0000 7f93fea00640 1 Processor -- start 2024-04-17T07:26:48.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.751+0000 7f93fea00640 1 -- start start 2024-04-17T07:26:48.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.751+0000 7f93fea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93f8076920 0x7f93f819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.751+0000 7f93fea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8077b60 0x7f93f819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.751+0000 7f93fea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8078500 0x7f93f81a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.751+0000 7f93fea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8114100 con 0x7f93f8077b60 2024-04-17T07:26:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.751+0000 7f93fea00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8113f80 con 0x7f93f8076920 2024-04-17T07:26:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.751+0000 7f93fea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8114280 con 0x7f93f8078500 2024-04-17T07:26:48.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.752+0000 7f93fca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8078500 0x7f93f81a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.752+0000 7f93fca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8078500 0x7f93f81a5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49308/0 (socket says 172.21.15.86:49308) 2024-04-17T07:26:48.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.752+0000 7f93fca00640 1 -- 172.21.15.86:0/258640883 learned_addr learned my addr 172.21.15.86:0/258640883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:48.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.752+0000 7f93f7400640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8077b60 0x7f93f819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.752+0000 7f93f7e00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93f8076920 0x7f93f819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.753+0000 7f93fca00640 1 -- 172.21.15.86:0/258640883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93f8076920 msgr2=0x7f93f819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.753+0000 7f93fca00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93f8076920 0x7f93f819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.753+0000 7f93fca00640 1 -- 172.21.15.86:0/258640883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8077b60 msgr2=0x7f93f819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.753+0000 7f93fca00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8077b60 0x7f93f819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.753+0000 7f93fca00640 1 -- 172.21.15.86:0/258640883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f81a6410 con 0x7f93f8078500 2024-04-17T07:26:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.753+0000 7f93f7e00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93f8076920 0x7f93f819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.753+0000 7f93fca00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8078500 0x7f93f81a5da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009fd0 tx=0x7f93f000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.754+0000 7f93f4c00640 1 -- 172.21.15.86:0/258640883 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0004b20 con 0x7f93f8078500 2024-04-17T07:26:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.754+0000 7f93fea00640 1 -- 172.21.15.86:0/258640883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f81a6700 con 0x7f93f8078500 2024-04-17T07:26:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.754+0000 7f93f4c00640 1 -- 172.21.15.86:0/258640883 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93f0003e20 con 0x7f93f8078500 2024-04-17T07:26:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.754+0000 7f93f4c00640 1 -- 172.21.15.86:0/258640883 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0004820 con 0x7f93f8078500 2024-04-17T07:26:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.754+0000 7f93fea00640 1 -- 172.21.15.86:0/258640883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a6c20 con 0x7f93f8078500 2024-04-17T07:26:48.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.760+0000 7f9401643740 1 Processor -- start 2024-04-17T07:26:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.760+0000 7f93f4c00640 1 -- 172.21.15.86:0/258640883 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f93f001d020 con 0x7f93f8078500 2024-04-17T07:26:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.761+0000 7f9401643740 1 -- start start 2024-04-17T07:26:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.762+0000 7f9401643740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 0x5576844553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.762+0000 7f9401643740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x55768445a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.764+0000 7f9401643740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55768445ab00 0x55768445cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.764+0000 7f9401643740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576843bb020 con 0x557684455990 2024-04-17T07:26:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.764+0000 7f9401643740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557684361460 con 0x55768445ab00 2024-04-17T07:26:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.764+0000 7f9401643740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5576843b6fe0 con 0x557684454fa0 2024-04-17T07:26:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.762+0000 7f93f7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 0x5576844553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.764+0000 7f93f7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 0x5576844553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49320/0 (socket says 172.21.15.86:49320) 2024-04-17T07:26:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.764+0000 7f93f7400640 1 -- 172.21.15.86:0/1952791608 learned_addr learned my addr 172.21.15.86:0/1952791608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.764+0000 7f93f4c00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93dc0a3cb0 0x7f93dc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.764+0000 7f93fca00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93dc0a3cb0 0x7f93dc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.764+0000 7f93f7e00640 1 --2- 172.21.15.86:0/1952791608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x55768445a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.765+0000 7f93f4c00640 1 -- 172.21.15.86:0/258640883 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93f00d0790 con 0x7f93f8078500 2024-04-17T07:26:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.765+0000 7f93f7400640 1 --2- 172.21.15.86:0/1952791608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55768445ab00 0x55768445cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.765+0000 7f93f7400640 1 -- 172.21.15.86:0/1952791608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55768445ab00 msgr2=0x55768445cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.765+0000 7f93f7400640 1 --2- 172.21.15.86:0/1952791608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55768445ab00 0x55768445cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.765+0000 7f93f7400640 1 -- 172.21.15.86:0/1952791608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 msgr2=0x55768445a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.765+0000 7f93f7400640 1 --2- 172.21.15.86:0/1952791608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x55768445a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.765+0000 7f93f7400640 1 -- 172.21.15.86:0/1952791608 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557684369040 con 0x557684454fa0 2024-04-17T07:26:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.766+0000 7f93fca00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93dc0a3cb0 0x7f93dc0a6170 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f93f00a4dd0 tx=0x7f93f006b0b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.765+0000 7f93f7e00640 1 --2- 172.21.15.86:0/1952791608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x55768445a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.766+0000 7f93f7400640 1 --2- 172.21.15.86:0/1952791608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 0x5576844553c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009830 tx=0x7f93e8005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c30baafc59e1b966 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.766+0000 7f93ed200640 1 -- 172.21.15.86:0/1952791608 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8048070 con 0x557684454fa0 2024-04-17T07:26:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.766+0000 7f93ed200640 1 -- 172.21.15.86:0/1952791608 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e802c9c0 con 0x557684454fa0 2024-04-17T07:26:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.767+0000 7f93ed200640 1 -- 172.21.15.86:0/1952791608 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8039040 con 0x557684454fa0 2024-04-17T07:26:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.767+0000 7f9401643740 1 -- 172.21.15.86:0/1952791608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 msgr2=0x5576844553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.767+0000 7f9401643740 1 --2- 172.21.15.86:0/1952791608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 0x5576844553c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009830 tx=0x7f93e8005760 comp rx=0 tx=0).stop 2024-04-17T07:26:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.768+0000 7f9401643740 1 -- 172.21.15.86:0/1952791608 shutdown_connections 2024-04-17T07:26:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.768+0000 7f9401643740 1 --2- 172.21.15.86:0/1952791608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55768445ab00 0x55768445cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.768+0000 7f9401643740 1 --2- 172.21.15.86:0/1952791608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x55768445a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.768+0000 7f9401643740 1 --2- 172.21.15.86:0/1952791608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 0x5576844553c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.768+0000 7f9401643740 1 -- 172.21.15.86:0/1952791608 >> 172.21.15.86:0/1952791608 conn(0x557684450510 msgr2=0x557684450910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.768+0000 7f9401643740 1 -- 172.21.15.86:0/1952791608 shutdown_connections 2024-04-17T07:26:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.768+0000 7f9401643740 1 -- 172.21.15.86:0/1952791608 wait complete. 2024-04-17T07:26:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.768+0000 7f9401643740 1 -- start start 2024-04-17T07:26:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.769+0000 7f9401643740 1 Processor -- start 2024-04-17T07:26:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.769+0000 7f9401643740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x55768445f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.769+0000 7f9401643740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x55768445fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.769+0000 7f9401643740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55768445ab00 0x557684466aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f9401643740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55768445da50 con 0x557684455990 2024-04-17T07:26:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f9401643740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557684374830 con 0x557684454fa0 2024-04-17T07:26:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f9401643740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55768445dbd0 con 0x55768445ab00 2024-04-17T07:26:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f9401643740 10 client.0 _ll_register_callbacks cb 0x557684342fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f93f7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x55768445f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f93f7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x55768445f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43800/0 (socket says 172.21.15.86:43800) 2024-04-17T07:26:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f93f7400640 1 -- 172.21.15.86:0/1355891943 learned_addr learned my addr 172.21.15.86:0/1355891943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f93f7400640 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55768445ab00 0x557684466aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f93f7e00640 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x55768445fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f93f7400640 1 -- 172.21.15.86:0/1355891943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55768445ab00 msgr2=0x557684466aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f93f7400640 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55768445ab00 0x557684466aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f93f7400640 1 -- 172.21.15.86:0/1355891943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 msgr2=0x55768445fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f93f7400640 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x55768445fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.770+0000 7f93f7400640 1 -- 172.21.15.86:0/1355891943 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557684378920 con 0x557684454fa0 2024-04-17T07:26:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.771+0000 7f93f7400640 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x55768445f520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f93e802c0b0 tx=0x7f93e802bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ce45752b2b11648 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:48.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.771+0000 7f93e1600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:26:48.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.772+0000 7f93e1600640 1 -- 172.21.15.86:0/1355891943 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8048020 con 0x557684454fa0 2024-04-17T07:26:48.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.772+0000 7f93e1600640 1 -- 172.21.15.86:0/1355891943 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93e8005520 con 0x557684454fa0 2024-04-17T07:26:48.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.772+0000 7f9401643740 1 -- 172.21.15.86:0/1355891943 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e802c9c0 con 0x557684454fa0 2024-04-17T07:26:48.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.773+0000 7f93e1600640 1 -- 172.21.15.86:0/1355891943 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93e805b050 con 0x557684454fa0 2024-04-17T07:26:48.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.774+0000 7f9401643740 10 client.37316 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:48.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.774+0000 7f9401643740 1 -- 172.21.15.86:0/1355891943 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5576843467f0 con 0x557684454fa0 2024-04-17T07:26:48.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.774+0000 7f93e1600640 1 -- 172.21.15.86:0/1355891943 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e802c9c0 con 0x557684454fa0 2024-04-17T07:26:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.774+0000 7f93e1600640 1 -- 172.21.15.86:0/1355891943 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f93e803c070 con 0x557684454fa0 2024-04-17T07:26:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.775+0000 7f9401643740 10 client.37316 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.775+0000 7f9401643740 10 client.37316 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:26:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.775+0000 7f9401643740 20 client.37316 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.775+0000 7f9401643740 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55768446ac00 0x55768446cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.775+0000 7f9401643740 4 client.37316 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.775+0000 7f9401643740 1 -- 172.21.15.86:0/1355891943 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576843c8a40 con 0x55768446ac00 2024-04-17T07:26:48.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.775+0000 7f93f7e00640 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55768446ac00 0x55768446cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.776+0000 7f93e1600640 10 client.37316 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:26:48.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.776+0000 7f93f7e00640 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55768446ac00 0x55768446cff0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=13a41beaa7b96715 server_cookie=79c56fb03b3b5fac in_seq=0 out_seq=0 2024-04-17T07:26:48.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.777+0000 7f93e1600640 1 -- 172.21.15.86:0/1355891943 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576843c8a40 con 0x55768446ac00 2024-04-17T07:26:48.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.777+0000 7f93e1600640 10 client.37316 handle_command_reply: tid=0 2024-04-17T07:26:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.777+0000 7f9401643740 1 client.37316 shutdown 2024-04-17T07:26:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f9401643740 1 -- 172.21.15.86:0/1355891943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 msgr2=0x55768445f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f9401643740 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x55768445f520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f93e802c0b0 tx=0x7f93e802bb70 comp rx=0 tx=0).stop 2024-04-17T07:26:48.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f9401643740 1 -- 172.21.15.86:0/1355891943 shutdown_connections 2024-04-17T07:26:48.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f9401643740 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55768446ac00 0x55768446cff0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f9401643740 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55768445ab00 0x557684466aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f93e1600640 0 client.37316 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:26:48.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f9401643740 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x55768445fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f9401643740 1 --2- 172.21.15.86:0/1355891943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x55768445f520 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f9401643740 1 -- 172.21.15.86:0/1355891943 >> 172.21.15.86:0/1355891943 conn(0x557684450510 msgr2=0x55768444f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:48.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f9401643740 1 -- 172.21.15.86:0/1355891943 shutdown_connections 2024-04-17T07:26:48.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.778+0000 7f9401643740 1 -- 172.21.15.86:0/1355891943 wait complete. 2024-04-17T07:26:48.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.779+0000 7f9401643740 20 client.37316 trim_cache size 0 max 16384 2024-04-17T07:26:48.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.806+0000 7f9401643740 1 Processor -- start 2024-04-17T07:26:48.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.806+0000 7f9401643740 1 -- start start 2024-04-17T07:26:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.806+0000 7f9401643740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 0x55768445f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.806+0000 7f9401643740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684455990 0x557684466a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.806+0000 7f9401643740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55768445ab00 0x55768444ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.806+0000 7f9401643740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b1000 con 0x55768445ab00 2024-04-17T07:26:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.806+0000 7f9401643740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b2b40 con 0x557684455990 2024-04-17T07:26:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.806+0000 7f9401643740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b16d0 con 0x557684454fa0 2024-04-17T07:26:48.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684455990 0x557684466a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684455990 0x557684466a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43814/0 (socket says 172.21.15.86:43814) 2024-04-17T07:26:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7e00640 1 -- 172.21.15.86:0/2401068822 learned_addr learned my addr 172.21.15.86:0/2401068822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7400640 1 --2- 172.21.15.86:0/2401068822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55768445ab00 0x55768444ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7e00640 1 -- 172.21.15.86:0/2401068822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 msgr2=0x55768445f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7e00640 1 --2- 172.21.15.86:0/2401068822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 0x55768445f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7e00640 1 -- 172.21.15.86:0/2401068822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55768445ab00 msgr2=0x55768444ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7e00640 1 --2- 172.21.15.86:0/2401068822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55768445ab00 0x55768444ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7e00640 1 -- 172.21.15.86:0/2401068822 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576840cd3f0 con 0x557684455990 2024-04-17T07:26:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7400640 1 --2- 172.21.15.86:0/2401068822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55768445ab00 0x55768444ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.807+0000 7f93f7e00640 1 --2- 172.21.15.86:0/2401068822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684455990 0x557684466a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f93e400ccc0 tx=0x7f93e4010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa5b5e069f8a95f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.808+0000 7f93e3e00640 1 -- 172.21.15.86:0/2401068822 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4004220 con 0x557684455990 2024-04-17T07:26:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.808+0000 7f93e3e00640 1 -- 172.21.15.86:0/2401068822 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e401c8e0 con 0x557684455990 2024-04-17T07:26:48.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.808+0000 7f93e3e00640 1 -- 172.21.15.86:0/2401068822 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4003d90 con 0x557684455990 2024-04-17T07:26:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.809+0000 7f9401643740 1 -- 172.21.15.86:0/2401068822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684455990 msgr2=0x557684466a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.809+0000 7f9401643740 1 --2- 172.21.15.86:0/2401068822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684455990 0x557684466a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f93e400ccc0 tx=0x7f93e4010800 comp rx=0 tx=0).stop 2024-04-17T07:26:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.809+0000 7f9401643740 1 -- 172.21.15.86:0/2401068822 shutdown_connections 2024-04-17T07:26:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.809+0000 7f9401643740 1 --2- 172.21.15.86:0/2401068822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55768445ab00 0x55768444ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.809+0000 7f9401643740 1 --2- 172.21.15.86:0/2401068822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684455990 0x557684466a30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.809+0000 7f9401643740 1 --2- 172.21.15.86:0/2401068822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557684454fa0 0x55768445f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.809+0000 7f9401643740 1 -- 172.21.15.86:0/2401068822 >> 172.21.15.86:0/2401068822 conn(0x557684450510 msgr2=0x55768445f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.809+0000 7f9401643740 1 -- 172.21.15.86:0/2401068822 shutdown_connections 2024-04-17T07:26:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.809+0000 7f9401643740 1 -- 172.21.15.86:0/2401068822 wait complete. 2024-04-17T07:26:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.809+0000 7f9401643740 1 -- start start 2024-04-17T07:26:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.810+0000 7f9401643740 1 Processor -- start 2024-04-17T07:26:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.810+0000 7f9401643740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x557684463d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.810+0000 7f9401643740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x557684464240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.810+0000 7f9401643740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55768445ab00 0x5576844651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.810+0000 7f9401643740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b2470 con 0x557684455990 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.810+0000 7f9401643740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81b1da0 con 0x557684454fa0 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.810+0000 7f9401643740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93f81aefd0 con 0x55768445ab00 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.810+0000 7f9401643740 10 client.0 _ll_register_callbacks cb 0x7f93e802c890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.811+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x557684464240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.811+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x557684464240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53790/0 (socket says 172.21.15.86:53790) 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.811+0000 7f93f7e00640 1 -- 172.21.15.86:0/2799686606 learned_addr learned my addr 172.21.15.86:0/2799686606 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.811+0000 7f93f7e00640 1 -- 172.21.15.86:0/2799686606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55768445ab00 msgr2=0x5576844651b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.812+0000 7f93f7e00640 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55768445ab00 0x5576844651b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.812+0000 7f93f7e00640 1 -- 172.21.15.86:0/2799686606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 msgr2=0x557684463d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.812+0000 7f93f7400640 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x557684463d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.812+0000 7f93f7e00640 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x557684463d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.812+0000 7f93f7e00640 1 -- 172.21.15.86:0/2799686606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576841977d0 con 0x557684455990 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.812+0000 7f93f7400640 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x557684463d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.812+0000 7f93f7e00640 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x557684464240 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f93e4003670 tx=0x7f93e40036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e948388f63dd3ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.812+0000 7f93e1600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.813+0000 7f93e1600640 1 -- 172.21.15.86:0/2799686606 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4003b10 con 0x557684455990 2024-04-17T07:26:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.813+0000 7f93e1600640 1 -- 172.21.15.86:0/2799686606 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f93e4003cb0 con 0x557684455990 2024-04-17T07:26:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.814+0000 7f9401643740 1 -- 172.21.15.86:0/2799686606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e401c8e0 con 0x557684455990 2024-04-17T07:26:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.814+0000 7f93e1600640 1 -- 172.21.15.86:0/2799686606 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93e4003eb0 con 0x557684455990 2024-04-17T07:26:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.814+0000 7f9401643740 10 client.18444 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.814+0000 7f9401643740 1 -- 172.21.15.86:0/2799686606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557684466870 con 0x557684455990 2024-04-17T07:26:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.815+0000 7f93e1600640 1 -- 172.21.15.86:0/2799686606 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e401c8e0 con 0x557684455990 2024-04-17T07:26:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.816+0000 7f93e1600640 1 -- 172.21.15.86:0/2799686606 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f93e402c020 con 0x557684455990 2024-04-17T07:26:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.816+0000 7f9401643740 10 client.18444 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.816+0000 7f9401643740 10 client.18444 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:26:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.816+0000 7f9401643740 20 client.18444 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.816+0000 7f9401643740 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5576844d1160 0x5576844d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.816+0000 7f9401643740 4 client.18444 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:26:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.816+0000 7f9401643740 1 -- 172.21.15.86:0/2799686606 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576843c28b0 con 0x5576844d1160 2024-04-17T07:26:48.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.816+0000 7f93f7400640 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5576844d1160 0x5576844d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:48.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.817+0000 7f93f7400640 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5576844d1160 0x5576844d3550 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fabf56b764b2cae2 server_cookie=68c513dc2223054e in_seq=0 out_seq=0 2024-04-17T07:26:48.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.817+0000 7f93e1600640 10 client.18444 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:26:48.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.817+0000 7f93e1600640 1 -- 172.21.15.86:0/2799686606 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576843c28b0 con 0x5576844d1160 2024-04-17T07:26:48.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.817+0000 7f93e1600640 10 client.18444 handle_command_reply: tid=0 2024-04-17T07:26:48.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.817+0000 7f9401643740 1 client.18444 shutdown 2024-04-17T07:26:48.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.818+0000 7f9401643740 1 -- 172.21.15.86:0/2799686606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 msgr2=0x557684464240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.818+0000 7f9401643740 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x557684464240 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f93e4003670 tx=0x7f93e40036a0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.818+0000 7f9401643740 1 -- 172.21.15.86:0/2799686606 shutdown_connections 2024-04-17T07:26:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.818+0000 7f9401643740 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5576844d1160 0x5576844d3550 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.818+0000 7f93e1600640 0 client.18444 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:26:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.818+0000 7f9401643740 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55768445ab00 0x5576844651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.818+0000 7f9401643740 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557684455990 0x557684464240 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.818+0000 7f9401643740 1 --2- 172.21.15.86:0/2799686606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557684454fa0 0x557684463d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.818+0000 7f9401643740 1 -- 172.21.15.86:0/2799686606 >> 172.21.15.86:0/2799686606 conn(0x557684450510 msgr2=0x557684453770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:48.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.819+0000 7f9401643740 1 -- 172.21.15.86:0/2799686606 shutdown_connections 2024-04-17T07:26:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.819+0000 7f9401643740 1 -- 172.21.15.86:0/2799686606 wait complete. 2024-04-17T07:26:48.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.819+0000 7f9401643740 20 client.18444 trim_cache size 0 max 16384 2024-04-17T07:26:48.820 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:26:48.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.821+0000 7f93fea00640 1 -- 172.21.15.86:0/258640883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93dc0a3cb0 msgr2=0x7f93dc0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.821+0000 7f93fea00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93dc0a3cb0 0x7f93dc0a6170 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f93f00a4dd0 tx=0x7f93f006b0b0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.821+0000 7f93fea00640 1 -- 172.21.15.86:0/258640883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8078500 msgr2=0x7f93f81a5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:48.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.821+0000 7f93fea00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8078500 0x7f93f81a5da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009fd0 tx=0x7f93f000c800 comp rx=0 tx=0).stop 2024-04-17T07:26:48.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.823+0000 7f93fea00640 1 -- 172.21.15.86:0/258640883 shutdown_connections 2024-04-17T07:26:48.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.823+0000 7f93fea00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f93dc0a3cb0 0x7f93dc0a6170 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.823+0000 7f93fea00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8078500 0x7f93f81a5da0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.823+0000 7f93fea00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93f8077b60 0x7f93f819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.823+0000 7f93fea00640 1 --2- 172.21.15.86:0/258640883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93f8076920 0x7f93f819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:48.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.823+0000 7f93fea00640 1 -- 172.21.15.86:0/258640883 >> 172.21.15.86:0/258640883 conn(0x7f93f80ff710 msgr2=0x7f93f8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:48.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.824+0000 7f93fea00640 1 -- 172.21.15.86:0/258640883 shutdown_connections 2024-04-17T07:26:48.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:48.824+0000 7f93fea00640 1 -- 172.21.15.86:0/258640883 wait complete. 2024-04-17T07:26:48.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:26:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.158+0000 7f197e800640 1 -- 172.21.15.86:0/1846117887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f197810c870 msgr2=0x7f197810cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.158+0000 7f197e800640 1 --2- 172.21.15.86:0/1846117887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f197810c870 0x7f197810cc90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f196c009860 tx=0x7f196c02bb70 comp rx=0 tx=0).stop 2024-04-17T07:26:49.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.158+0000 7f197e800640 1 -- 172.21.15.86:0/1846117887 shutdown_connections 2024-04-17T07:26:49.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.158+0000 7f197e800640 1 --2- 172.21.15.86:0/1846117887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197810e420 0x7f1978114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.158+0000 7f197e800640 1 --2- 172.21.15.86:0/1846117887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197810da60 0x7f197810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.158+0000 7f197e800640 1 --2- 172.21.15.86:0/1846117887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f197810c870 0x7f197810cc90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.158+0000 7f197e800640 1 -- 172.21.15.86:0/1846117887 >> 172.21.15.86:0/1846117887 conn(0x7f197806c740 msgr2=0x7f197806cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.160+0000 7f197e800640 1 -- 172.21.15.86:0/1846117887 shutdown_connections 2024-04-17T07:26:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.160+0000 7f197e800640 1 -- 172.21.15.86:0/1846117887 wait complete. 2024-04-17T07:26:49.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.162+0000 7f197e800640 1 Processor -- start 2024-04-17T07:26:49.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.162+0000 7f197e800640 1 -- start start 2024-04-17T07:26:49.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.162+0000 7f197e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f197810c870 0x7f19781a6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.162+0000 7f197e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197810da60 0x7f19781a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.162+0000 7f197e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197810e420 0x7f19781ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.162+0000 7f197e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f197806ea30 con 0x7f197810e420 2024-04-17T07:26:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.162+0000 7f197e800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f197806e8b0 con 0x7f197810da60 2024-04-17T07:26:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.162+0000 7f197e800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f197806ebb0 con 0x7f197810c870 2024-04-17T07:26:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.163+0000 7f197d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f197810c870 0x7f19781a6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.163+0000 7f197d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f197810c870 0x7f19781a6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49372/0 (socket says 172.21.15.86:49372) 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.163+0000 7f197d400640 1 -- 172.21.15.86:0/559625057 learned_addr learned my addr 172.21.15.86:0/559625057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.163+0000 7f197ca00640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197810da60 0x7f19781a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.163+0000 7f197de00640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197810e420 0x7f19781ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.163+0000 7f197d400640 1 -- 172.21.15.86:0/559625057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197810da60 msgr2=0x7f19781a73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.163+0000 7f197d400640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197810da60 0x7f19781a73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.163+0000 7f197d400640 1 -- 172.21.15.86:0/559625057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197810e420 msgr2=0x7f19781ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.163+0000 7f197d400640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197810e420 0x7f19781ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.163+0000 7f197d400640 1 -- 172.21.15.86:0/559625057 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19781aead0 con 0x7f197810c870 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.164+0000 7f197d400640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f197810c870 0x7f19781a6eb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f196c0057c0 tx=0x7f196c035d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.164+0000 7f1976000640 1 -- 172.21.15.86:0/559625057 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c039040 con 0x7f197810c870 2024-04-17T07:26:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.164+0000 7f197e800640 1 -- 172.21.15.86:0/559625057 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19781aed60 con 0x7f197810c870 2024-04-17T07:26:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.164+0000 7f197e800640 1 -- 172.21.15.86:0/559625057 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19781af1e0 con 0x7f197810c870 2024-04-17T07:26:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.164+0000 7f1976000640 1 -- 172.21.15.86:0/559625057 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f196c02cb80 con 0x7f197810c870 2024-04-17T07:26:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.164+0000 7f1976000640 1 -- 172.21.15.86:0/559625057 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c0046c0 con 0x7f197810c870 2024-04-17T07:26:49.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.169+0000 7f1982b66740 1 Processor -- start 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.169+0000 7f1982b66740 1 -- start start 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.169+0000 7f1976000640 1 -- 172.21.15.86:0/559625057 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f196c044020 con 0x7f197810c870 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.169+0000 7f1982b66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fc4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.169+0000 7f1982b66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fc9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.169+0000 7f1982b66740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc9cc0 0x559897fcc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.169+0000 7f1982b66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559897f2a020 con 0x559897fc4b50 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.169+0000 7f1982b66740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559897ed0460 con 0x559897fc9cc0 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.169+0000 7f1982b66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559897f25fe0 con 0x559897fc4160 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.170+0000 7f197d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fc9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.170+0000 7f197d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fc9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53822/0 (socket says 172.21.15.86:53822) 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.170+0000 7f197d400640 1 -- 172.21.15.86:0/1894971706 learned_addr learned my addr 172.21.15.86:0/1894971706 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.170+0000 7f197de00640 1 --2- 172.21.15.86:0/1894971706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fc4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.170+0000 7f197ca00640 1 --2- 172.21.15.86:0/1894971706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc9cc0 0x559897fcc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.171+0000 7f197d400640 1 -- 172.21.15.86:0/1894971706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 msgr2=0x559897fc4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.171+0000 7f197d400640 1 --2- 172.21.15.86:0/1894971706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fc4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.171+0000 7f197d400640 1 -- 172.21.15.86:0/1894971706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc9cc0 msgr2=0x559897fcc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.171+0000 7f197d400640 1 --2- 172.21.15.86:0/1894971706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc9cc0 0x559897fcc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.171+0000 7f197d400640 1 -- 172.21.15.86:0/1894971706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559897ed8040 con 0x559897fc4b50 2024-04-17T07:26:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.171+0000 7f1976000640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19600a3cb0 0x7f19600a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.172+0000 7f197d400640 1 --2- 172.21.15.86:0/1894971706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fc9650 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f196c0f7a90 tx=0x7f196c0f7b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db57be3b4db8e12e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.172+0000 7f197de00640 1 --2- 172.21.15.86:0/1894971706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fc4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.172+0000 7f196a600640 1 -- 172.21.15.86:0/1894971706 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c0b9310 con 0x559897fc4b50 2024-04-17T07:26:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.172+0000 7f196a600640 1 -- 172.21.15.86:0/1894971706 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f196c0bdea0 con 0x559897fc4b50 2024-04-17T07:26:49.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.172+0000 7f1976000640 1 -- 172.21.15.86:0/559625057 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f196c0f18a0 con 0x7f197810c870 2024-04-17T07:26:49.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.173+0000 7f197de00640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19600a3cb0 0x7f19600a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.173+0000 7f196a600640 1 -- 172.21.15.86:0/1894971706 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c0b9310 con 0x559897fc4b50 2024-04-17T07:26:49.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.173+0000 7f1982b66740 1 -- 172.21.15.86:0/1894971706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 msgr2=0x559897fc9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.173+0000 7f1982b66740 1 --2- 172.21.15.86:0/1894971706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fc9650 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f196c0f7a90 tx=0x7f196c0f7b80 comp rx=0 tx=0).stop 2024-04-17T07:26:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.174+0000 7f1982b66740 1 -- 172.21.15.86:0/1894971706 shutdown_connections 2024-04-17T07:26:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.174+0000 7f1982b66740 1 --2- 172.21.15.86:0/1894971706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc9cc0 0x559897fcc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.174+0000 7f1982b66740 1 --2- 172.21.15.86:0/1894971706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fc9650 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.174+0000 7f1982b66740 1 --2- 172.21.15.86:0/1894971706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fc4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.174+0000 7f1982b66740 1 -- 172.21.15.86:0/1894971706 >> 172.21.15.86:0/1894971706 conn(0x559897fbf6d0 msgr2=0x559897fbfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.175+0000 7f1982b66740 1 -- 172.21.15.86:0/1894971706 shutdown_connections 2024-04-17T07:26:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.176+0000 7f1982b66740 1 -- 172.21.15.86:0/1894971706 wait complete. 2024-04-17T07:26:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.176+0000 7f197de00640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19600a3cb0 0x7f19600a6170 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1970009bd0 tx=0x7f1970009730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.176+0000 7f1982b66740 1 -- start start 2024-04-17T07:26:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.177+0000 7f1982b66740 1 Processor -- start 2024-04-17T07:26:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.177+0000 7f1982b66740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 0x559897fce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.177+0000 7f1982b66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.177+0000 7f1982b66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 0x559897fd5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.177+0000 7f1982b66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559897fcca30 con 0x559897fc4b50 2024-04-17T07:26:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.177+0000 7f1982b66740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559897ee3830 con 0x559897fc4160 2024-04-17T07:26:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.177+0000 7f1982b66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559897fccbb0 con 0x559897fc9cc0 2024-04-17T07:26:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.177+0000 7f1982b66740 10 client.0 _ll_register_callbacks cb 0x559897eb1fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.178+0000 7f197de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 0x559897fd5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.178+0000 7f197de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 0x559897fd5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49398/0 (socket says 172.21.15.86:49398) 2024-04-17T07:26:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.178+0000 7f197de00640 1 -- 172.21.15.86:0/372297481 learned_addr learned my addr 172.21.15.86:0/372297481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.179+0000 7f197d400640 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 0x559897fce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.179+0000 7f197ca00640 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.180+0000 7f197de00640 1 -- 172.21.15.86:0/372297481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 msgr2=0x559897fce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.180+0000 7f197de00640 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 0x559897fce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.180+0000 7f197de00640 1 -- 172.21.15.86:0/372297481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 msgr2=0x559897fce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.180+0000 7f197de00640 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.180+0000 7f197de00640 1 -- 172.21.15.86:0/372297481 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559897ee7920 con 0x559897fc9cc0 2024-04-17T07:26:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.180+0000 7f197ca00640 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.181+0000 7f197de00640 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 0x559897fd5920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1970010950 tx=0x7f1970004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1802d28ae28e5776 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.181+0000 7f195ea00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.182+0000 7f195ea00640 1 -- 172.21.15.86:0/372297481 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000ecf0 con 0x559897fc9cc0 2024-04-17T07:26:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.182+0000 7f195ea00640 1 -- 172.21.15.86:0/372297481 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1970027400 con 0x559897fc9cc0 2024-04-17T07:26:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.182+0000 7f1982b66740 1 -- 172.21.15.86:0/372297481 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f196c0bdea0 con 0x559897fc9cc0 2024-04-17T07:26:49.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.182+0000 7f195ea00640 1 -- 172.21.15.86:0/372297481 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1970027600 con 0x559897fc9cc0 2024-04-17T07:26:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.182+0000 7f1982b66740 10 client.27409 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.182+0000 7f1982b66740 1 -- 172.21.15.86:0/372297481 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559897eb57f0 con 0x559897fc9cc0 2024-04-17T07:26:49.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.183+0000 7f195ea00640 1 -- 172.21.15.86:0/372297481 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1970030070 con 0x559897fc9cc0 2024-04-17T07:26:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.183+0000 7f195ea00640 1 -- 172.21.15.86:0/372297481 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f197000f040 con 0x559897fc9cc0 2024-04-17T07:26:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.184+0000 7f1982b66740 10 client.27409 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.184+0000 7f1982b66740 10 client.27409 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:26:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.184+0000 7f1982b66740 20 client.27409 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.184+0000 7f1982b66740 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559897fd9fa0 0x559897fdc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.184+0000 7f1982b66740 4 client.27409 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.184+0000 7f1982b66740 1 -- 172.21.15.86:0/372297481 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559897f37a40 con 0x559897fd9fa0 2024-04-17T07:26:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.184+0000 7f197d400640 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559897fd9fa0 0x559897fdc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.185+0000 7f197d400640 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559897fd9fa0 0x559897fdc420 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88198002e3e2e104 server_cookie=e3b1d06dcf1203d8 in_seq=0 out_seq=0 2024-04-17T07:26:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.185+0000 7f195ea00640 10 client.27409 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:26:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.186+0000 7f195ea00640 1 -- 172.21.15.86:0/372297481 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559897f37a40 con 0x559897fd9fa0 2024-04-17T07:26:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.186+0000 7f195ea00640 10 client.27409 handle_command_reply: tid=0 2024-04-17T07:26:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.186+0000 7f1982b66740 1 client.27409 shutdown 2024-04-17T07:26:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.187+0000 7f1982b66740 1 -- 172.21.15.86:0/372297481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 msgr2=0x559897fd5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.187+0000 7f1982b66740 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 0x559897fd5920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1970010950 tx=0x7f1970004a40 comp rx=0 tx=0).stop 2024-04-17T07:26:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.187+0000 7f1982b66740 1 -- 172.21.15.86:0/372297481 shutdown_connections 2024-04-17T07:26:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.187+0000 7f1982b66740 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559897fd9fa0 0x559897fdc420 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.187+0000 7f1982b66740 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 0x559897fd5920 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.187+0000 7f1982b66740 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.187+0000 7f1982b66740 1 --2- 172.21.15.86:0/372297481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 0x559897fce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.187+0000 7f1982b66740 1 -- 172.21.15.86:0/372297481 >> 172.21.15.86:0/372297481 conn(0x559897fbf6d0 msgr2=0x559897fbe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.188+0000 7f195ea00640 0 client.27409 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:26:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.188+0000 7f1982b66740 1 -- 172.21.15.86:0/372297481 shutdown_connections 2024-04-17T07:26:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.188+0000 7f1982b66740 1 -- 172.21.15.86:0/372297481 wait complete. 2024-04-17T07:26:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.188+0000 7f1982b66740 20 client.27409 trim_cache size 0 max 16384 2024-04-17T07:26:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.213+0000 7f1982b66740 1 Processor -- start 2024-04-17T07:26:49.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.213+0000 7f1982b66740 1 -- start start 2024-04-17T07:26:49.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.213+0000 7f1982b66740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 0x559897fcdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.213+0000 7f1982b66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fd58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.213+0000 7f1982b66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 0x559897fdbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.214+0000 7f1982b66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f195800fc20 con 0x559897fc4b50 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.214+0000 7f1982b66740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1958011760 con 0x559897fc4160 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.214+0000 7f1982b66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19580102f0 con 0x559897fc9cc0 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.214+0000 7f197d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 0x559897fcdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.214+0000 7f197d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 0x559897fcdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43864/0 (socket says 172.21.15.86:43864) 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.214+0000 7f197d400640 1 -- 172.21.15.86:0/2851426518 learned_addr learned my addr 172.21.15.86:0/2851426518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.214+0000 7f197ca00640 1 --2- 172.21.15.86:0/2851426518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fd58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.214+0000 7f197de00640 1 --2- 172.21.15.86:0/2851426518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 0x559897fdbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.215+0000 7f197d400640 1 -- 172.21.15.86:0/2851426518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 msgr2=0x559897fdbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.215+0000 7f197d400640 1 --2- 172.21.15.86:0/2851426518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 0x559897fdbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.215+0000 7f197d400640 1 -- 172.21.15.86:0/2851426518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 msgr2=0x559897fd58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.215+0000 7f197d400640 1 --2- 172.21.15.86:0/2851426518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fd58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.215+0000 7f197d400640 1 -- 172.21.15.86:0/2851426518 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559897c3c3f0 con 0x559897fc4160 2024-04-17T07:26:49.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.215+0000 7f197ca00640 1 --2- 172.21.15.86:0/2851426518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fd58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:49.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.215+0000 7f197d400640 1 --2- 172.21.15.86:0/2851426518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 0x559897fcdd80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f196c0bd4a0 tx=0x7f196c0c0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ee56be2d488c726 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.215+0000 7f1969c00640 1 -- 172.21.15.86:0/2851426518 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c0bd100 con 0x559897fc4160 2024-04-17T07:26:49.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.216+0000 7f1969c00640 1 -- 172.21.15.86:0/2851426518 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559897f37a40 con 0x559897fc4160 2024-04-17T07:26:49.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.216+0000 7f1969c00640 1 -- 172.21.15.86:0/2851426518 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c0bd2e0 con 0x559897fc4160 2024-04-17T07:26:49.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.216+0000 7f1982b66740 1 -- 172.21.15.86:0/2851426518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 msgr2=0x559897fcdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.216+0000 7f1982b66740 1 --2- 172.21.15.86:0/2851426518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 0x559897fcdd80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f196c0bd4a0 tx=0x7f196c0c0040 comp rx=0 tx=0).stop 2024-04-17T07:26:49.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.217+0000 7f1982b66740 1 -- 172.21.15.86:0/2851426518 shutdown_connections 2024-04-17T07:26:49.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.217+0000 7f1982b66740 1 --2- 172.21.15.86:0/2851426518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc9cc0 0x559897fdbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.217+0000 7f1982b66740 1 --2- 172.21.15.86:0/2851426518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc4b50 0x559897fd58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.217+0000 7f1982b66740 1 --2- 172.21.15.86:0/2851426518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4160 0x559897fcdd80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.217+0000 7f1982b66740 1 -- 172.21.15.86:0/2851426518 >> 172.21.15.86:0/2851426518 conn(0x559897fbf6d0 msgr2=0x559897fcd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.217+0000 7f1982b66740 1 -- 172.21.15.86:0/2851426518 shutdown_connections 2024-04-17T07:26:49.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.217+0000 7f1982b66740 1 -- 172.21.15.86:0/2851426518 wait complete. 2024-04-17T07:26:49.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.217+0000 7f1982b66740 1 -- start start 2024-04-17T07:26:49.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.217+0000 7f1982b66740 1 Processor -- start 2024-04-17T07:26:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.218+0000 7f1982b66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fd3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.218+0000 7f1982b66740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4b50 0x559897fd38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.218+0000 7f1982b66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc9cc0 0x559897fd4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.219+0000 7f1982b66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1958011090 con 0x559897fc9cc0 2024-04-17T07:26:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.219+0000 7f1982b66740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19580109c0 con 0x559897fc4b50 2024-04-17T07:26:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.219+0000 7f1982b66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f195800dbf0 con 0x559897fc4160 2024-04-17T07:26:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.219+0000 7f1982b66740 10 client.0 _ll_register_callbacks cb 0x7f19700105d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.219+0000 7f197d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fd3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.219+0000 7f197ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4b50 0x559897fd38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.219+0000 7f197d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fd3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49418/0 (socket says 172.21.15.86:49418) 2024-04-17T07:26:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.219+0000 7f197ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4b50 0x559897fd38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43874/0 (socket says 172.21.15.86:43874) 2024-04-17T07:26:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.219+0000 7f197d400640 1 -- 172.21.15.86:0/841044529 learned_addr learned my addr 172.21.15.86:0/841044529 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.219+0000 7f197de00640 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc9cc0 0x559897fd4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f197d400640 1 -- 172.21.15.86:0/841044529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4b50 msgr2=0x559897fd38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f197d400640 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4b50 0x559897fd38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f197d400640 1 -- 172.21.15.86:0/841044529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc9cc0 msgr2=0x559897fd4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f197d400640 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc9cc0 0x559897fd4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f197d400640 1 -- 172.21.15.86:0/841044529 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559897d067d0 con 0x559897fc4160 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f197ca00640 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4b50 0x559897fd38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f197d400640 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fd3370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f196c0bba50 tx=0x7f196c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e15046ffb58cfd9b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f195ea00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f195ea00640 1 -- 172.21.15.86:0/841044529 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c0bd570 con 0x559897fc4160 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f1982b66740 1 -- 172.21.15.86:0/841044529 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559897f37a40 con 0x559897fc4160 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.220+0000 7f195ea00640 1 -- 172.21.15.86:0/841044529 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f196c0bbaa0 con 0x559897fc4160 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.221+0000 7f195ea00640 1 -- 172.21.15.86:0/841044529 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f196c0bd2e0 con 0x559897fc4160 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.221+0000 7f1982b66740 10 client.27415 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.221+0000 7f1982b66740 1 -- 172.21.15.86:0/841044529 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559898040690 con 0x559897fc4160 2024-04-17T07:26:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.221+0000 7f195ea00640 1 -- 172.21.15.86:0/841044529 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559897f37a40 con 0x559897fc4160 2024-04-17T07:26:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.222+0000 7f195ea00640 1 -- 172.21.15.86:0/841044529 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f196c0d5070 con 0x559897fc4160 2024-04-17T07:26:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.222+0000 7f1982b66740 10 client.27415 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.222+0000 7f1982b66740 10 client.27415 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:26:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.222+0000 7f1982b66740 20 client.27415 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.222+0000 7f1982b66740 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559898040ca0 0x559898043090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.222+0000 7f1982b66740 4 client.27415 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:26:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.222+0000 7f1982b66740 1 -- 172.21.15.86:0/841044529 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559897f318b0 con 0x559898040ca0 2024-04-17T07:26:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.223+0000 7f197ca00640 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559898040ca0 0x559898043090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.224+0000 7f197ca00640 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559898040ca0 0x559898043090 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9e07719efd4da29a server_cookie=c8080d7cf13e3ace in_seq=0 out_seq=0 2024-04-17T07:26:49.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.224+0000 7f195ea00640 10 client.27415 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:26:49.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.224+0000 7f195ea00640 1 -- 172.21.15.86:0/841044529 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559897f318b0 con 0x559898040ca0 2024-04-17T07:26:49.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.224+0000 7f195ea00640 10 client.27415 handle_command_reply: tid=0 2024-04-17T07:26:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.224+0000 7f1982b66740 1 client.27415 shutdown 2024-04-17T07:26:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.224+0000 7f1982b66740 1 -- 172.21.15.86:0/841044529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 msgr2=0x559897fd3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.224+0000 7f1982b66740 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fd3370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f196c0bba50 tx=0x7f196c005f90 comp rx=0 tx=0).stop 2024-04-17T07:26:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.225+0000 7f1982b66740 1 -- 172.21.15.86:0/841044529 shutdown_connections 2024-04-17T07:26:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.225+0000 7f1982b66740 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x559898040ca0 0x559898043090 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.225+0000 7f1982b66740 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559897fc9cc0 0x559897fd4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.225+0000 7f1982b66740 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559897fc4b50 0x559897fd38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.225+0000 7f195ea00640 0 client.27415 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:26:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.225+0000 7f1982b66740 1 --2- 172.21.15.86:0/841044529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559897fc4160 0x559897fd3370 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.225+0000 7f1982b66740 1 -- 172.21.15.86:0/841044529 >> 172.21.15.86:0/841044529 conn(0x559897fbf6d0 msgr2=0x559897fc2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.225+0000 7f1982b66740 1 -- 172.21.15.86:0/841044529 shutdown_connections 2024-04-17T07:26:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.225+0000 7f1982b66740 1 -- 172.21.15.86:0/841044529 wait complete. 2024-04-17T07:26:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.226+0000 7f1982b66740 20 client.27415 trim_cache size 0 max 16384 2024-04-17T07:26:49.226 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:26:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.226+0000 7f197e800640 1 -- 172.21.15.86:0/559625057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19600a3cb0 msgr2=0x7f19600a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.226+0000 7f197e800640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19600a3cb0 0x7f19600a6170 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1970009bd0 tx=0x7f1970009730 comp rx=0 tx=0).stop 2024-04-17T07:26:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.226+0000 7f197e800640 1 -- 172.21.15.86:0/559625057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f197810c870 msgr2=0x7f19781a6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.226+0000 7f197e800640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f197810c870 0x7f19781a6eb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f196c0057c0 tx=0x7f196c035d40 comp rx=0 tx=0).stop 2024-04-17T07:26:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.227+0000 7f197e800640 1 -- 172.21.15.86:0/559625057 shutdown_connections 2024-04-17T07:26:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.227+0000 7f197e800640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19600a3cb0 0x7f19600a6170 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.227+0000 7f197e800640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f197810e420 0x7f19781ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.227+0000 7f197e800640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197810da60 0x7f19781a73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.227+0000 7f197e800640 1 --2- 172.21.15.86:0/559625057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f197810c870 0x7f19781a6eb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.227+0000 7f197e800640 1 -- 172.21.15.86:0/559625057 >> 172.21.15.86:0/559625057 conn(0x7f197806c740 msgr2=0x7f197810b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.227+0000 7f197e800640 1 -- 172.21.15.86:0/559625057 shutdown_connections 2024-04-17T07:26:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.227+0000 7f197e800640 1 -- 172.21.15.86:0/559625057 wait complete. 2024-04-17T07:26:49.248 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:49 smithi086 ceph-mon[31733]: pgmap v813: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 10 MiB/s wr, 703 op/s 2024-04-17T07:26:49.248 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:49 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1823552381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:26:49.251 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:26:49.330 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:49 smithi103 ceph-mon[35495]: pgmap v813: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 10 MiB/s wr, 703 op/s 2024-04-17T07:26:49.330 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:49 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1823552381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:26:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 445.0 s, 632.8 tps, lat 151.226 ms stddev 50.628 2024-04-17T07:26:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.469+0000 7f5750200640 1 -- 172.21.15.86:0/4014490232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57481053b0 msgr2=0x7f5748105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.469+0000 7f5750200640 1 --2- 172.21.15.86:0/4014490232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57481053b0 0x7f5748105830 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f5734009900 tx=0x7f573402bc30 comp rx=0 tx=0).stop 2024-04-17T07:26:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.469+0000 7f5750200640 1 -- 172.21.15.86:0/4014490232 shutdown_connections 2024-04-17T07:26:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.469+0000 7f5750200640 1 --2- 172.21.15.86:0/4014490232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5748105d70 0x7f574810c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.469+0000 7f5750200640 1 --2- 172.21.15.86:0/4014490232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57481053b0 0x7f5748105830 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.469+0000 7f5750200640 1 --2- 172.21.15.86:0/4014490232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57481041c0 0x7f57481045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.469+0000 7f5750200640 1 -- 172.21.15.86:0/4014490232 >> 172.21.15.86:0/4014490232 conn(0x7f57480ff750 msgr2=0x7f5748101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.469+0000 7f5750200640 1 -- 172.21.15.86:0/4014490232 shutdown_connections 2024-04-17T07:26:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.469+0000 7f5750200640 1 -- 172.21.15.86:0/4014490232 wait complete. 2024-04-17T07:26:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.470+0000 7f5750200640 1 Processor -- start 2024-04-17T07:26:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.470+0000 7f5750200640 1 -- start start 2024-04-17T07:26:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.470+0000 7f5750200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57481041c0 0x7f574819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.470+0000 7f5750200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57481053b0 0x7f574819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.470+0000 7f5750200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5748105d70 0x7f57481a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.470+0000 7f5750200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f574810fd90 con 0x7f57481053b0 2024-04-17T07:26:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.470+0000 7f5750200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f574810fc10 con 0x7f57481041c0 2024-04-17T07:26:49.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.471+0000 7f5750200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f574810ff10 con 0x7f5748105d70 2024-04-17T07:26:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.471+0000 7f574ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57481053b0 0x7f574819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.471+0000 7f574ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57481053b0 0x7f574819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53874/0 (socket says 172.21.15.86:53874) 2024-04-17T07:26:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.471+0000 7f574ce00640 1 -- 172.21.15.86:0/235124851 learned_addr learned my addr 172.21.15.86:0/235124851 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.471+0000 7f574d800640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57481041c0 0x7f574819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.471+0000 7f574ce00640 1 -- 172.21.15.86:0/235124851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5748105d70 msgr2=0x7f57481a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.471+0000 7f574ce00640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5748105d70 0x7f57481a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.471+0000 7f574ce00640 1 -- 172.21.15.86:0/235124851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57481041c0 msgr2=0x7f574819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.471+0000 7f574ce00640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57481041c0 0x7f574819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.471+0000 7f574ce00640 1 -- 172.21.15.86:0/235124851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57481a2090 con 0x7f57481053b0 2024-04-17T07:26:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.472+0000 7f574ce00640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57481053b0 0x7f574819a9b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5734000c00 tx=0x7f57340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.472+0000 7f5746000640 1 -- 172.21.15.86:0/235124851 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734043020 con 0x7f57481053b0 2024-04-17T07:26:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.472+0000 7f5746000640 1 -- 172.21.15.86:0/235124851 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57340054c0 con 0x7f57481053b0 2024-04-17T07:26:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.472+0000 7f5750200640 1 -- 172.21.15.86:0/235124851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57481a2320 con 0x7f57481053b0 2024-04-17T07:26:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.472+0000 7f5746000640 1 -- 172.21.15.86:0/235124851 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734003d70 con 0x7f57481053b0 2024-04-17T07:26:49.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.472+0000 7f5750200640 1 -- 172.21.15.86:0/235124851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57481a2940 con 0x7f57481053b0 2024-04-17T07:26:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.476+0000 7f5752fd6740 1 Processor -- start 2024-04-17T07:26:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.476+0000 7f5752fd6740 1 -- start start 2024-04-17T07:26:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.476+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e5b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.476+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e60490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.476+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 0x563504e62f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f5752fd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563504dc1020 con 0x563504e5b990 2024-04-17T07:26:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f5752fd6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563504d67460 con 0x563504e5afa0 2024-04-17T07:26:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f5752fd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563504dbcfe0 con 0x563504e60b00 2024-04-17T07:26:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f5746000640 1 -- 172.21.15.86:0/235124851 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5734046020 con 0x7f57481053b0 2024-04-17T07:26:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e60490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e60490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53876/0 (socket says 172.21.15.86:53876) 2024-04-17T07:26:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574d800640 1 -- 172.21.15.86:0/1038728084 learned_addr learned my addr 172.21.15.86:0/1038728084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574e200640 1 --2- 172.21.15.86:0/1038728084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e5b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574ce00640 1 --2- 172.21.15.86:0/1038728084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 0x563504e62f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574d800640 1 -- 172.21.15.86:0/1038728084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 msgr2=0x563504e62f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574d800640 1 --2- 172.21.15.86:0/1038728084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 0x563504e62f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574d800640 1 -- 172.21.15.86:0/1038728084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 msgr2=0x563504e5b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574d800640 1 --2- 172.21.15.86:0/1038728084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e5b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574d800640 1 -- 172.21.15.86:0/1038728084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563504d6f040 con 0x563504e5b990 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.477+0000 7f574ce00640 1 --2- 172.21.15.86:0/1038728084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 0x563504e62f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.478+0000 7f574d800640 1 --2- 172.21.15.86:0/1038728084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e60490 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f573c00bc90 tx=0x7f573c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680cad94907f60c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.478+0000 7f573aa00640 1 -- 172.21.15.86:0/1038728084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c015070 con 0x563504e5b990 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.478+0000 7f573aa00640 1 -- 172.21.15.86:0/1038728084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f573c0035f0 con 0x563504e5b990 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.478+0000 7f573aa00640 1 -- 172.21.15.86:0/1038728084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c0116a0 con 0x563504e5b990 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.478+0000 7f5746000640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57300a3c30 0x7f57300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.478+0000 7f5746000640 1 -- 172.21.15.86:0/235124851 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f57340f2c10 con 0x7f57481053b0 2024-04-17T07:26:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.478+0000 7f574e200640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57300a3c30 0x7f57300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.479+0000 7f5752fd6740 1 -- 172.21.15.86:0/1038728084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 msgr2=0x563504e60490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.479+0000 7f5752fd6740 1 --2- 172.21.15.86:0/1038728084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e60490 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f573c00bc90 tx=0x7f573c00f730 comp rx=0 tx=0).stop 2024-04-17T07:26:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.479+0000 7f574e200640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57300a3c30 0x7f57300a60f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x563504e5c1f0 tx=0x7f5740009430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.479+0000 7f5752fd6740 1 -- 172.21.15.86:0/1038728084 shutdown_connections 2024-04-17T07:26:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.479+0000 7f5752fd6740 1 --2- 172.21.15.86:0/1038728084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 0x563504e62f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.479+0000 7f5752fd6740 1 --2- 172.21.15.86:0/1038728084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e60490 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.479+0000 7f5752fd6740 1 --2- 172.21.15.86:0/1038728084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e5b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.479+0000 7f5752fd6740 1 -- 172.21.15.86:0/1038728084 >> 172.21.15.86:0/1038728084 conn(0x563504e56510 msgr2=0x563504e56910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.479+0000 7f5752fd6740 1 -- 172.21.15.86:0/1038728084 shutdown_connections 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.479+0000 7f5752fd6740 1 -- 172.21.15.86:0/1038728084 wait complete. 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.480+0000 7f5752fd6740 1 -- start start 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.480+0000 7f5752fd6740 1 Processor -- start 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.480+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e654e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.480+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e65a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.480+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 0x563504e6ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.480+0000 7f5752fd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563504e63ab0 con 0x563504e5b990 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.480+0000 7f5752fd6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563504d7a830 con 0x563504e5afa0 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.480+0000 7f5752fd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563504e63c30 con 0x563504e60b00 2024-04-17T07:26:49.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.480+0000 7f5752fd6740 10 client.0 _ll_register_callbacks cb 0x563504d48fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:49.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.481+0000 7f574ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e65a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.481+0000 7f574ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e65a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53880/0 (socket says 172.21.15.86:53880) 2024-04-17T07:26:49.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.481+0000 7f574ce00640 1 -- 172.21.15.86:0/2828308068 learned_addr learned my addr 172.21.15.86:0/2828308068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.481+0000 7f574e200640 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 0x563504e6ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.481+0000 7f574d800640 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.482+0000 7f574ce00640 1 -- 172.21.15.86:0/2828308068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 msgr2=0x563504e6ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.482+0000 7f574ce00640 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 0x563504e6ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.482+0000 7f574ce00640 1 -- 172.21.15.86:0/2828308068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 msgr2=0x563504e654e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.482+0000 7f574ce00640 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e654e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.482+0000 7f574ce00640 1 -- 172.21.15.86:0/2828308068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563504d7e920 con 0x563504e5b990 2024-04-17T07:26:49.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.482+0000 7f574ce00640 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e65a20 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f57340bb660 tx=0x7f57340beb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550baa760d9b05f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.482+0000 7f572f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:49.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.484+0000 7f572f400640 1 -- 172.21.15.86:0/2828308068 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57340c7670 con 0x563504e5b990 2024-04-17T07:26:49.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.484+0000 7f572f400640 1 -- 172.21.15.86:0/2828308068 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5734005010 con 0x563504e5b990 2024-04-17T07:26:49.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.484+0000 7f5752fd6740 1 -- 172.21.15.86:0/2828308068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f573c0035f0 con 0x563504e5b990 2024-04-17T07:26:49.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.486+0000 7f572f400640 1 -- 172.21.15.86:0/2828308068 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f57340bf390 con 0x563504e5b990 2024-04-17T07:26:49.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.487+0000 7f5752fd6740 10 client.18474 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:49.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.487+0000 7f5752fd6740 1 -- 172.21.15.86:0/2828308068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563504d4c7f0 con 0x563504e5b990 2024-04-17T07:26:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.489+0000 7f572f400640 1 -- 172.21.15.86:0/2828308068 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f573c0035f0 con 0x563504e5b990 2024-04-17T07:26:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.489+0000 7f572f400640 1 -- 172.21.15.86:0/2828308068 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f57340ba9e0 con 0x563504e5b990 2024-04-17T07:26:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.490+0000 7f5752fd6740 10 client.18474 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.490+0000 7f5752fd6740 10 client.18474 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:26:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.490+0000 7f5752fd6740 20 client.18474 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.490+0000 7f5752fd6740 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563504e70df0 0x563504e732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.490+0000 7f5752fd6740 4 client.18474 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.490+0000 7f5752fd6740 1 -- 172.21.15.86:0/2828308068 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563504dcea40 con 0x563504e70df0 2024-04-17T07:26:49.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.494+0000 7f574d800640 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563504e70df0 0x563504e732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.499+0000 7f572f400640 10 client.18474 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:26:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.499+0000 7f574d800640 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563504e70df0 0x563504e732b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0090c9d84160661 server_cookie=204820543d64f8b1 in_seq=0 out_seq=0 2024-04-17T07:26:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:49 smithi098 ceph-mon[35418]: pgmap v813: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 10 MiB/s wr, 703 op/s 2024-04-17T07:26:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:49 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1823552381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:26:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.504+0000 7f572f400640 1 -- 172.21.15.86:0/2828308068 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563504dcea40 con 0x563504e70df0 2024-04-17T07:26:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.504+0000 7f572f400640 10 client.18474 handle_command_reply: tid=0 2024-04-17T07:26:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.504+0000 7f5752fd6740 1 client.18474 shutdown 2024-04-17T07:26:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.504+0000 7f5752fd6740 1 -- 172.21.15.86:0/2828308068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 msgr2=0x563504e65a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.504+0000 7f5752fd6740 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e65a20 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f57340bb660 tx=0x7f57340beb20 comp rx=0 tx=0).stop 2024-04-17T07:26:49.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.506+0000 7f5752fd6740 1 -- 172.21.15.86:0/2828308068 shutdown_connections 2024-04-17T07:26:49.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.506+0000 7f5752fd6740 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563504e70df0 0x563504e732b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.506+0000 7f5752fd6740 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e60b00 0x563504e6ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.506+0000 7f5752fd6740 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e5b990 0x563504e65a20 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.506+0000 7f5752fd6740 1 --2- 172.21.15.86:0/2828308068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e654e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.506+0000 7f5752fd6740 1 -- 172.21.15.86:0/2828308068 >> 172.21.15.86:0/2828308068 conn(0x563504e56510 msgr2=0x563504e55710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.506+0000 7f572f400640 0 client.18474 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:26:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.507+0000 7f5752fd6740 1 -- 172.21.15.86:0/2828308068 shutdown_connections 2024-04-17T07:26:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.507+0000 7f5752fd6740 1 -- 172.21.15.86:0/2828308068 wait complete. 2024-04-17T07:26:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.510+0000 7f5752fd6740 20 client.18474 trim_cache size 0 max 16384 2024-04-17T07:26:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.540+0000 7f5752fd6740 1 Processor -- start 2024-04-17T07:26:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.541+0000 7f5752fd6740 1 -- start start 2024-04-17T07:26:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.541+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e59e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.542+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5b990 0x563504e6ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.542+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e60b00 0x563504e77bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.542+0000 7f5752fd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57481acf20 con 0x563504e60b00 2024-04-17T07:26:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.542+0000 7f5752fd6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57481aea60 con 0x563504e5afa0 2024-04-17T07:26:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.542+0000 7f5752fd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57481ad5f0 con 0x563504e5b990 2024-04-17T07:26:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.542+0000 7f574d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e59e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5b990 0x563504e6ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e60b00 0x563504e77bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5b990 0x563504e6ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49468/0 (socket says 172.21.15.86:49468) 2024-04-17T07:26:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574ce00640 1 -- 172.21.15.86:0/3074979102 learned_addr learned my addr 172.21.15.86:0/3074979102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e59e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43922/0 (socket says 172.21.15.86:43922) 2024-04-17T07:26:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574ce00640 1 -- 172.21.15.86:0/3074979102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 msgr2=0x563504e59e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574ce00640 1 --2- 172.21.15.86:0/3074979102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e59e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574ce00640 1 -- 172.21.15.86:0/3074979102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e60b00 msgr2=0x563504e77bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574ce00640 1 --2- 172.21.15.86:0/3074979102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e60b00 0x563504e77bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574ce00640 1 -- 172.21.15.86:0/3074979102 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563504e60850 con 0x563504e5b990 2024-04-17T07:26:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.543+0000 7f574d800640 1 --2- 172.21.15.86:0/3074979102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e59e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.544+0000 7f574ce00640 1 --2- 172.21.15.86:0/3074979102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5b990 0x563504e6ca20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f573402c6a0 tx=0x7f57340bd1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a55f16b978a297 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.545+0000 7f573a000640 1 -- 172.21.15.86:0/3074979102 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57340c7900 con 0x563504e5b990 2024-04-17T07:26:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.545+0000 7f573a000640 1 -- 172.21.15.86:0/3074979102 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57340d8070 con 0x563504e5b990 2024-04-17T07:26:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.545+0000 7f573a000640 1 -- 172.21.15.86:0/3074979102 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57340c7470 con 0x563504e5b990 2024-04-17T07:26:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.546+0000 7f5752fd6740 1 -- 172.21.15.86:0/3074979102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5b990 msgr2=0x563504e6ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.546+0000 7f5752fd6740 1 --2- 172.21.15.86:0/3074979102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5b990 0x563504e6ca20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f573402c6a0 tx=0x7f57340bd1a0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.546+0000 7f5752fd6740 1 -- 172.21.15.86:0/3074979102 shutdown_connections 2024-04-17T07:26:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.546+0000 7f5752fd6740 1 --2- 172.21.15.86:0/3074979102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e60b00 0x563504e77bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.546+0000 7f5752fd6740 1 --2- 172.21.15.86:0/3074979102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5b990 0x563504e6ca20 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f573402c6a0 tx=0x7f57340bd1a0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.546+0000 7f5752fd6740 1 --2- 172.21.15.86:0/3074979102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e5afa0 0x563504e59e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.546+0000 7f5752fd6740 1 -- 172.21.15.86:0/3074979102 >> 172.21.15.86:0/3074979102 conn(0x563504e56510 msgr2=0x563504e59770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.547+0000 7f5752fd6740 1 -- 172.21.15.86:0/3074979102 shutdown_connections 2024-04-17T07:26:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.547+0000 7f5752fd6740 1 -- 172.21.15.86:0/3074979102 wait complete. 2024-04-17T07:26:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.548+0000 7f5752fd6740 1 -- start start 2024-04-17T07:26:49.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.548+0000 7f5752fd6740 1 Processor -- start 2024-04-17T07:26:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.549+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5afa0 0x563504e6a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.549+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e60b00 0x563504e6aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.549+0000 7f5752fd6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e6ba80 0x563504e6bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.549+0000 7f5752fd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57481ae390 con 0x563504e60b00 2024-04-17T07:26:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.549+0000 7f5752fd6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57481adcc0 con 0x563504e6ba80 2024-04-17T07:26:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.549+0000 7f5752fd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57481aaef0 con 0x563504e5afa0 2024-04-17T07:26:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.549+0000 7f5752fd6740 10 client.0 _ll_register_callbacks cb 0x7f5734009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.550+0000 7f574d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5afa0 0x563504e6a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.550+0000 7f574d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5afa0 0x563504e6a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49478/0 (socket says 172.21.15.86:49478) 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.550+0000 7f574d800640 1 -- 172.21.15.86:0/139288721 learned_addr learned my addr 172.21.15.86:0/139288721 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.550+0000 7f574ce00640 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e60b00 0x563504e6aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.550+0000 7f574e200640 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e6ba80 0x563504e6bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.550+0000 7f574d800640 1 -- 172.21.15.86:0/139288721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e6ba80 msgr2=0x563504e6bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.550+0000 7f574d800640 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e6ba80 0x563504e6bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.550+0000 7f574d800640 1 -- 172.21.15.86:0/139288721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e60b00 msgr2=0x563504e6aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.550+0000 7f574d800640 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e60b00 0x563504e6aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.550+0000 7f574d800640 1 -- 172.21.15.86:0/139288721 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563504ad33f0 con 0x563504e5afa0 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.551+0000 7f574d800640 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5afa0 0x563504e6a530 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f573c00bc90 tx=0x7f573c005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503582cc9167cd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.551+0000 7f572f400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.551+0000 7f572f400640 1 -- 172.21.15.86:0/139288721 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c015040 con 0x563504e5afa0 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.551+0000 7f572f400640 1 -- 172.21.15.86:0/139288721 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f573c004070 con 0x563504e5afa0 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.551+0000 7f5752fd6740 1 -- 172.21.15.86:0/139288721 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57340d8070 con 0x563504e5afa0 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.551+0000 7f572f400640 1 -- 172.21.15.86:0/139288721 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f573c039050 con 0x563504e5afa0 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.551+0000 7f5752fd6740 10 client.27424 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.552+0000 7f5752fd6740 1 -- 172.21.15.86:0/139288721 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563504b9d7d0 con 0x563504e5afa0 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.552+0000 7f572f400640 1 -- 172.21.15.86:0/139288721 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57340d8070 con 0x563504e5afa0 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.552+0000 7f572f400640 1 -- 172.21.15.86:0/139288721 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f573c029150 con 0x563504e5afa0 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.552+0000 7f5752fd6740 10 client.27424 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.552+0000 7f5752fd6740 10 client.27424 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.552+0000 7f5752fd6740 20 client.27424 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.552+0000 7f5752fd6740 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563504ee0760 0x563504ee2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.552+0000 7f5752fd6740 4 client.27424 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:26:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.552+0000 7f5752fd6740 1 -- 172.21.15.86:0/139288721 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563504dc88b0 con 0x563504ee0760 2024-04-17T07:26:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.553+0000 7f574ce00640 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563504ee0760 0x563504ee2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.554+0000 7f574ce00640 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563504ee0760 0x563504ee2b50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af0083e6d9a81679 server_cookie=6803eeff9471874b in_seq=0 out_seq=0 2024-04-17T07:26:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.554+0000 7f572f400640 10 client.27424 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:26:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f572f400640 1 -- 172.21.15.86:0/139288721 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563504dc88b0 con 0x563504ee0760 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f572f400640 10 client.27424 handle_command_reply: tid=0 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 client.27424 shutdown 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 -- 172.21.15.86:0/139288721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5afa0 msgr2=0x563504e6a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5afa0 0x563504e6a530 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f573c00bc90 tx=0x7f573c005d20 comp rx=0 tx=0).stop 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 -- 172.21.15.86:0/139288721 shutdown_connections 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x563504ee0760 0x563504ee2b50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563504e6ba80 0x563504e6bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563504e60b00 0x563504e6aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 --2- 172.21.15.86:0/139288721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563504e5afa0 0x563504e6a530 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 -- 172.21.15.86:0/139288721 >> 172.21.15.86:0/139288721 conn(0x563504e56510 msgr2=0x563504e5a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f572f400640 0 client.27424 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 -- 172.21.15.86:0/139288721 shutdown_connections 2024-04-17T07:26:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.555+0000 7f5752fd6740 1 -- 172.21.15.86:0/139288721 wait complete. 2024-04-17T07:26:49.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.556+0000 7f5752fd6740 20 client.27424 trim_cache size 0 max 16384 2024-04-17T07:26:49.557 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.556+0000 7f5750200640 1 -- 172.21.15.86:0/235124851 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57300a3c30 msgr2=0x7f57300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.556+0000 7f5750200640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57300a3c30 0x7f57300a60f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x563504e5c1f0 tx=0x7f5740009430 comp rx=0 tx=0).stop 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.556+0000 7f5750200640 1 -- 172.21.15.86:0/235124851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57481053b0 msgr2=0x7f574819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.556+0000 7f5750200640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57481053b0 0x7f574819a9b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5734000c00 tx=0x7f57340042f0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.557+0000 7f5750200640 1 -- 172.21.15.86:0/235124851 shutdown_connections 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.557+0000 7f5750200640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57300a3c30 0x7f57300a60f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.557+0000 7f5750200640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5748105d70 0x7f57481a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.557+0000 7f5750200640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57481053b0 0x7f574819a9b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.557+0000 7f5750200640 1 --2- 172.21.15.86:0/235124851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57481041c0 0x7f574819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.557+0000 7f5750200640 1 -- 172.21.15.86:0/235124851 >> 172.21.15.86:0/235124851 conn(0x7f57480ff750 msgr2=0x7f5748108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.557+0000 7f5750200640 1 -- 172.21.15.86:0/235124851 shutdown_connections 2024-04-17T07:26:49.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.557+0000 7f5750200640 1 -- 172.21.15.86:0/235124851 wait complete. 2024-04-17T07:26:49.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:26:49.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.867+0000 7faa23800640 1 -- 172.21.15.86:0/765565532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa1c1085c0 msgr2=0x7faa1c1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.867+0000 7faa1b400640 1 -- 172.21.15.86:0/765565532 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1002cbe0 con 0x7faa1c1085c0 2024-04-17T07:26:49.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.867+0000 7faa23800640 1 --2- 172.21.15.86:0/765565532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa1c1085c0 0x7faa1c1089e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faa10009900 tx=0x7faa1002bc30 comp rx=0 tx=0).stop 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.868+0000 7faa23800640 1 -- 172.21.15.86:0/765565532 shutdown_connections 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.868+0000 7faa23800640 1 --2- 172.21.15.86:0/765565532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa1c10a170 0x7faa1c110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.868+0000 7faa23800640 1 --2- 172.21.15.86:0/765565532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c1097b0 0x7faa1c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.868+0000 7faa23800640 1 --2- 172.21.15.86:0/765565532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa1c1085c0 0x7faa1c1089e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.868+0000 7faa23800640 1 -- 172.21.15.86:0/765565532 >> 172.21.15.86:0/765565532 conn(0x7faa1c077560 msgr2=0x7faa1c0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.868+0000 7faa23800640 1 -- 172.21.15.86:0/765565532 shutdown_connections 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.868+0000 7faa23800640 1 -- 172.21.15.86:0/765565532 wait complete. 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.869+0000 7faa23800640 1 Processor -- start 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.869+0000 7faa23800640 1 -- start start 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.869+0000 7faa23800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c1085c0 0x7faa1c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.869+0000 7faa23800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa1c1097b0 0x7faa1c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.870+0000 7faa23800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa1c10a170 0x7faa1c1a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.870+0000 7faa23800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c114140 con 0x7faa1c1085c0 2024-04-17T07:26:49.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.870+0000 7faa23800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c113fc0 con 0x7faa1c1097b0 2024-04-17T07:26:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.870+0000 7faa23800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1142c0 con 0x7faa1c10a170 2024-04-17T07:26:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.870+0000 7faa1be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa1c1097b0 0x7faa1c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.870+0000 7faa21800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa1c10a170 0x7faa1c1a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.870+0000 7faa1be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa1c1097b0 0x7faa1c19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43936/0 (socket says 172.21.15.86:43936) 2024-04-17T07:26:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.870+0000 7faa21800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa1c10a170 0x7faa1c1a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49488/0 (socket says 172.21.15.86:49488) 2024-04-17T07:26:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.870+0000 7faa1be00640 1 -- 172.21.15.86:0/1603552876 learned_addr learned my addr 172.21.15.86:0/1603552876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.870+0000 7faa20e00640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c1085c0 0x7faa1c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.871+0000 7faa21800640 1 -- 172.21.15.86:0/1603552876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa1c1097b0 msgr2=0x7faa1c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.871+0000 7faa21800640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa1c1097b0 0x7faa1c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.871+0000 7faa21800640 1 -- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c1085c0 msgr2=0x7faa1c19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.871+0000 7faa21800640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c1085c0 0x7faa1c19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.871+0000 7faa21800640 1 -- 172.21.15.86:0/1603552876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c1a6250 con 0x7faa1c10a170 2024-04-17T07:26:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.871+0000 7faa20e00640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c1085c0 0x7faa1c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.872+0000 7faa21800640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa1c10a170 0x7faa1c1a5be0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faa1400e940 tx=0x7faa1400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.873+0000 7faa19600640 1 -- 172.21.15.86:0/1603552876 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14004d20 con 0x7faa1c10a170 2024-04-17T07:26:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.873+0000 7faa19600640 1 -- 172.21.15.86:0/1603552876 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa14005020 con 0x7faa1c10a170 2024-04-17T07:26:49.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.873+0000 7faa19600640 1 -- 172.21.15.86:0/1603552876 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa140116d0 con 0x7faa1c10a170 2024-04-17T07:26:49.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.873+0000 7faa23800640 1 -- 172.21.15.86:0/1603552876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1c1a6540 con 0x7faa1c10a170 2024-04-17T07:26:49.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.873+0000 7faa23800640 1 -- 172.21.15.86:0/1603552876 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1c1a6a40 con 0x7faa1c10a170 2024-04-17T07:26:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.881+0000 7faa19600640 1 -- 172.21.15.86:0/1603552876 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7faa1400cb20 con 0x7faa1c10a170 2024-04-17T07:26:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.883+0000 7faa265e3740 1 Processor -- start 2024-04-17T07:26:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.883+0000 7faa265e3740 1 -- start start 2024-04-17T07:26:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.884+0000 7faa19600640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa040a3c70 0x7faa040a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.884+0000 7faa265e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef3fa0 0x556dc1ef43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.884+0000 7faa265e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1ef9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.884+0000 7faa265e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1efbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.884+0000 7faa265e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556dc1e5a020 con 0x556dc1ef3fa0 2024-04-17T07:26:49.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.884+0000 7faa265e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556dc1e00460 con 0x556dc1ef9b00 2024-04-17T07:26:49.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.884+0000 7faa265e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556dc1e55fe0 con 0x556dc1ef4990 2024-04-17T07:26:49.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.884+0000 7faa19600640 1 -- 172.21.15.86:0/1603552876 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faa140d0a90 con 0x7faa1c10a170 2024-04-17T07:26:49.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.884+0000 7faa1be00640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa040a3c70 0x7faa040a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.885+0000 7faa1be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1ef9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.885+0000 7faa21800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1efbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.885+0000 7faa21800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1efbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43948/0 (socket says 172.21.15.86:43948) 2024-04-17T07:26:49.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.885+0000 7faa21800640 1 -- 172.21.15.86:0/4263244647 learned_addr learned my addr 172.21.15.86:0/4263244647 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.885+0000 7faa20e00640 1 --2- 172.21.15.86:0/4263244647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef3fa0 0x556dc1ef43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.886+0000 7faa1be00640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa040a3c70 0x7faa040a6130 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7faa1c19fb40 tx=0x7faa0800b910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.886+0000 7faa1be00640 1 -- 172.21.15.86:0/4263244647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 msgr2=0x556dc1efbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.886+0000 7faa1be00640 1 --2- 172.21.15.86:0/4263244647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1efbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.886+0000 7faa1be00640 1 -- 172.21.15.86:0/4263244647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef3fa0 msgr2=0x556dc1ef43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.886+0000 7faa1be00640 1 --2- 172.21.15.86:0/4263244647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef3fa0 0x556dc1ef43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.886+0000 7faa1be00640 1 -- 172.21.15.86:0/4263244647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc1e08040 con 0x556dc1ef4990 2024-04-17T07:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.886+0000 7faa1be00640 1 --2- 172.21.15.86:0/4263244647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1ef9490 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faa080058c0 tx=0x7faa0801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be642d94cc4b3c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.887+0000 7faa0e000640 1 -- 172.21.15.86:0/4263244647 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08012230 con 0x556dc1ef4990 2024-04-17T07:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.887+0000 7faa0e000640 1 -- 172.21.15.86:0/4263244647 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa080054d0 con 0x556dc1ef4990 2024-04-17T07:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.887+0000 7faa0e000640 1 -- 172.21.15.86:0/4263244647 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0801f770 con 0x556dc1ef4990 2024-04-17T07:26:49.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.889+0000 7faa265e3740 1 -- 172.21.15.86:0/4263244647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 msgr2=0x556dc1ef9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.889+0000 7faa265e3740 1 --2- 172.21.15.86:0/4263244647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1ef9490 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faa080058c0 tx=0x7faa0801e000 comp rx=0 tx=0).stop 2024-04-17T07:26:49.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.889+0000 7faa265e3740 1 -- 172.21.15.86:0/4263244647 shutdown_connections 2024-04-17T07:26:49.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.889+0000 7faa265e3740 1 --2- 172.21.15.86:0/4263244647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1efbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.889+0000 7faa265e3740 1 --2- 172.21.15.86:0/4263244647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1ef9490 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.889+0000 7faa265e3740 1 --2- 172.21.15.86:0/4263244647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef3fa0 0x556dc1ef43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.889+0000 7faa265e3740 1 -- 172.21.15.86:0/4263244647 >> 172.21.15.86:0/4263244647 conn(0x556dc1eef510 msgr2=0x556dc1eef910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.889+0000 7faa265e3740 1 -- 172.21.15.86:0/4263244647 shutdown_connections 2024-04-17T07:26:49.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.889+0000 7faa265e3740 1 -- 172.21.15.86:0/4263244647 wait complete. 2024-04-17T07:26:49.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.890+0000 7faa265e3740 1 -- start start 2024-04-17T07:26:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.890+0000 7faa265e3740 1 Processor -- start 2024-04-17T07:26:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.891+0000 7faa265e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef3fa0 0x556dc1f0f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.891+0000 7faa265e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef4990 0x556dc1f0f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.891+0000 7faa265e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1f169b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.891+0000 7faa265e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556dc1f0da60 con 0x556dc1ef4990 2024-04-17T07:26:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.891+0000 7faa265e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556dc1e13830 con 0x556dc1ef9b00 2024-04-17T07:26:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.891+0000 7faa265e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556dc1f0dbe0 con 0x556dc1ef3fa0 2024-04-17T07:26:49.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.891+0000 7faa265e3740 10 client.0 _ll_register_callbacks cb 0x556dc1de1fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:49.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.892+0000 7faa21800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1f169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.892+0000 7faa21800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1f169b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43954/0 (socket says 172.21.15.86:43954) 2024-04-17T07:26:49.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.892+0000 7faa21800640 1 -- 172.21.15.86:0/3921025392 learned_addr learned my addr 172.21.15.86:0/3921025392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.892+0000 7faa20e00640 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef3fa0 0x556dc1f0f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.893+0000 7faa20e00640 1 -- 172.21.15.86:0/3921025392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 msgr2=0x556dc1f169b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.893+0000 7faa20e00640 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1f169b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.893+0000 7faa20e00640 1 -- 172.21.15.86:0/3921025392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef4990 msgr2=0x556dc1f0f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.894+0000 7faa20e00640 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef4990 0x556dc1f0f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.895+0000 7faa20e00640 1 -- 172.21.15.86:0/3921025392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc1e17920 con 0x556dc1ef3fa0 2024-04-17T07:26:49.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.895+0000 7faa21800640 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1f169b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:49.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.896+0000 7faa20e00640 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef3fa0 0x556dc1f0f430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faa10009850 tx=0x7faa100045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62ef1748f24276c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.896+0000 7faa02a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:49.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.897+0000 7faa02a00640 1 -- 172.21.15.86:0/3921025392 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10005d50 con 0x556dc1ef3fa0 2024-04-17T07:26:49.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.897+0000 7faa02a00640 1 -- 172.21.15.86:0/3921025392 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faa10003800 con 0x556dc1ef3fa0 2024-04-17T07:26:49.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.898+0000 7faa265e3740 1 -- 172.21.15.86:0/3921025392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa080054d0 con 0x556dc1ef3fa0 2024-04-17T07:26:49.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.898+0000 7faa02a00640 1 -- 172.21.15.86:0/3921025392 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa10003a00 con 0x556dc1ef3fa0 2024-04-17T07:26:49.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.899+0000 7faa265e3740 10 client.27436 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:49.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.899+0000 7faa265e3740 1 -- 172.21.15.86:0/3921025392 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556dc1de57f0 con 0x556dc1ef3fa0 2024-04-17T07:26:49.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.899+0000 7faa02a00640 1 -- 172.21.15.86:0/3921025392 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa080054d0 con 0x556dc1ef3fa0 2024-04-17T07:26:49.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.899+0000 7faa02a00640 1 -- 172.21.15.86:0/3921025392 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7faa1004acb0 con 0x556dc1ef3fa0 2024-04-17T07:26:49.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.899+0000 7faa265e3740 10 client.27436 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:49.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.899+0000 7faa265e3740 10 client.27436 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:26:49.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.900+0000 7faa265e3740 20 client.27436 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:49.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.900+0000 7faa265e3740 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x556dc1f1b330 0x556dc1f1d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.900+0000 7faa265e3740 4 client.27436 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:49.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.900+0000 7faa265e3740 1 -- 172.21.15.86:0/3921025392 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dc1e67a40 con 0x556dc1f1b330 2024-04-17T07:26:49.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.900+0000 7faa1be00640 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x556dc1f1b330 0x556dc1f1d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.902+0000 7faa02a00640 10 client.27436 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:26:49.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.902+0000 7faa1be00640 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x556dc1f1b330 0x556dc1f1d790 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf2fb2334bfd4266 server_cookie=2ea52aa400927c8 in_seq=0 out_seq=0 2024-04-17T07:26:49.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.903+0000 7faa02a00640 1 -- 172.21.15.86:0/3921025392 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556dc1e67a40 con 0x556dc1f1b330 2024-04-17T07:26:49.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.903+0000 7faa02a00640 10 client.27436 handle_command_reply: tid=0 2024-04-17T07:26:49.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.903+0000 7faa265e3740 1 client.27436 shutdown 2024-04-17T07:26:49.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.904+0000 7faa265e3740 1 -- 172.21.15.86:0/3921025392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef3fa0 msgr2=0x556dc1f0f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.904+0000 7faa265e3740 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef3fa0 0x556dc1f0f430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faa10009850 tx=0x7faa100045e0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.904+0000 7faa265e3740 1 -- 172.21.15.86:0/3921025392 shutdown_connections 2024-04-17T07:26:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.904+0000 7faa265e3740 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x556dc1f1b330 0x556dc1f1d790 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.904+0000 7faa265e3740 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef9b00 0x556dc1f169b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.904+0000 7faa265e3740 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef4990 0x556dc1f0f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.904+0000 7faa265e3740 1 --2- 172.21.15.86:0/3921025392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef3fa0 0x556dc1f0f430 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.904+0000 7faa02a00640 0 client.27436 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:26:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.904+0000 7faa265e3740 1 -- 172.21.15.86:0/3921025392 >> 172.21.15.86:0/3921025392 conn(0x556dc1eef510 msgr2=0x556dc1eee710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.905+0000 7faa265e3740 1 -- 172.21.15.86:0/3921025392 shutdown_connections 2024-04-17T07:26:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.905+0000 7faa265e3740 1 -- 172.21.15.86:0/3921025392 wait complete. 2024-04-17T07:26:49.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.905+0000 7faa265e3740 20 client.27436 trim_cache size 0 max 16384 2024-04-17T07:26:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.926+0000 7faa265e3740 1 Processor -- start 2024-04-17T07:26:49.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.926+0000 7faa265e3740 1 -- start start 2024-04-17T07:26:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.927+0000 7faa265e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 0x556dc1eede80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.927+0000 7faa265e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1f16970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.927+0000 7faa265e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1eee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.927+0000 7faa265e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1a8880 con 0x556dc1ef9b00 2024-04-17T07:26:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.927+0000 7faa265e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1aa3c0 con 0x556dc1ef3fa0 2024-04-17T07:26:49.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.927+0000 7faa265e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1a8f50 con 0x556dc1ef4990 2024-04-17T07:26:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.927+0000 7faa1be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1f16970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.927+0000 7faa21800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1eee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.928+0000 7faa21800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1eee3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53944/0 (socket says 172.21.15.86:53944) 2024-04-17T07:26:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.928+0000 7faa21800640 1 -- 172.21.15.86:0/789907216 learned_addr learned my addr 172.21.15.86:0/789907216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.928+0000 7faa20e00640 1 --2- 172.21.15.86:0/789907216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 0x556dc1eede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.928+0000 7faa21800640 1 -- 172.21.15.86:0/789907216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 msgr2=0x556dc1f16970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.928+0000 7faa21800640 1 --2- 172.21.15.86:0/789907216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1f16970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.928+0000 7faa21800640 1 -- 172.21.15.86:0/789907216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 msgr2=0x556dc1eede80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.928+0000 7faa21800640 1 --2- 172.21.15.86:0/789907216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 0x556dc1eede80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.928+0000 7faa21800640 1 -- 172.21.15.86:0/789907216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc1b6c3f0 con 0x556dc1ef9b00 2024-04-17T07:26:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.929+0000 7faa21800640 1 --2- 172.21.15.86:0/789907216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1eee3c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c640 tx=0x7faa14005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583314cd226a8cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.929+0000 7faa0d600640 1 -- 172.21.15.86:0/789907216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa140b7030 con 0x556dc1ef9b00 2024-04-17T07:26:49.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.929+0000 7faa0d600640 1 -- 172.21.15.86:0/789907216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa140a0da0 con 0x556dc1ef9b00 2024-04-17T07:26:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.929+0000 7faa0d600640 1 -- 172.21.15.86:0/789907216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa140b5020 con 0x556dc1ef9b00 2024-04-17T07:26:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.930+0000 7faa265e3740 1 -- 172.21.15.86:0/789907216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 msgr2=0x556dc1eee3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.930+0000 7faa265e3740 1 --2- 172.21.15.86:0/789907216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1eee3c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c640 tx=0x7faa14005bc0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.931+0000 7faa265e3740 1 -- 172.21.15.86:0/789907216 shutdown_connections 2024-04-17T07:26:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.931+0000 7faa265e3740 1 --2- 172.21.15.86:0/789907216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1eee3c0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.931+0000 7faa265e3740 1 --2- 172.21.15.86:0/789907216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1f16970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.931+0000 7faa265e3740 1 --2- 172.21.15.86:0/789907216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 0x556dc1eede80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.931+0000 7faa265e3740 1 -- 172.21.15.86:0/789907216 >> 172.21.15.86:0/789907216 conn(0x556dc1eef510 msgr2=0x556dc1eef8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.931+0000 7faa265e3740 1 -- 172.21.15.86:0/789907216 shutdown_connections 2024-04-17T07:26:49.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.931+0000 7faa265e3740 1 -- 172.21.15.86:0/789907216 wait complete. 2024-04-17T07:26:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.931+0000 7faa265e3740 1 -- start start 2024-04-17T07:26:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.932+0000 7faa265e3740 1 Processor -- start 2024-04-17T07:26:49.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.932+0000 7faa265e3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 0x556dc1f138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.932+0000 7faa265e3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1f13e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.932+0000 7faa265e3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1f7f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.933+0000 7faa265e3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1a9cf0 con 0x556dc1ef9b00 2024-04-17T07:26:49.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.933+0000 7faa265e3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1a9620 con 0x556dc1ef3fa0 2024-04-17T07:26:49.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.933+0000 7faa265e3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c073ca0 con 0x556dc1ef4990 2024-04-17T07:26:49.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.933+0000 7faa265e3740 10 client.0 _ll_register_callbacks cb 0x7faa1002cbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:49.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.933+0000 7faa21800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1f7f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.933+0000 7faa20e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 0x556dc1f138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.933+0000 7faa20e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 0x556dc1f138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43978/0 (socket says 172.21.15.86:43978) 2024-04-17T07:26:49.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.933+0000 7faa20e00640 1 -- 172.21.15.86:0/1871289766 learned_addr learned my addr 172.21.15.86:0/1871289766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.933+0000 7faa1be00640 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1f13e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.934+0000 7faa21800640 1 -- 172.21.15.86:0/1871289766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 msgr2=0x556dc1f13e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.934+0000 7faa21800640 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1f13e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.934+0000 7faa21800640 1 -- 172.21.15.86:0/1871289766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 msgr2=0x556dc1f138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.934+0000 7faa21800640 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 0x556dc1f138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.934+0000 7faa21800640 1 -- 172.21.15.86:0/1871289766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc1c367d0 con 0x556dc1ef9b00 2024-04-17T07:26:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.934+0000 7faa21800640 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1f7f900 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7faa14099420 tx=0x7faa140bd120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cddacbec7957be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.934+0000 7faa02a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.935+0000 7faa02a00640 1 -- 172.21.15.86:0/1871289766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14080060 con 0x556dc1ef9b00 2024-04-17T07:26:49.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.935+0000 7faa02a00640 1 -- 172.21.15.86:0/1871289766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faa140bbdc0 con 0x556dc1ef9b00 2024-04-17T07:26:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.935+0000 7faa265e3740 1 -- 172.21.15.86:0/1871289766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa140a0da0 con 0x556dc1ef9b00 2024-04-17T07:26:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.936+0000 7faa02a00640 1 -- 172.21.15.86:0/1871289766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa14080060 con 0x556dc1ef9b00 2024-04-17T07:26:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.936+0000 7faa265e3740 10 client.18495 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.936+0000 7faa265e3740 1 -- 172.21.15.86:0/1871289766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556dc1f80fc0 con 0x556dc1ef9b00 2024-04-17T07:26:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.936+0000 7faa02a00640 1 -- 172.21.15.86:0/1871289766 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa140a0da0 con 0x556dc1ef9b00 2024-04-17T07:26:49.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.936+0000 7faa02a00640 1 -- 172.21.15.86:0/1871289766 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7faa14083050 con 0x556dc1ef9b00 2024-04-17T07:26:49.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.936+0000 7faa265e3740 10 client.18495 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:49.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.936+0000 7faa265e3740 10 client.18495 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:26:49.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.936+0000 7faa265e3740 20 client.18495 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:49.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.937+0000 7faa265e3740 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x556dc1f81800 0x556dc1f83bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:49.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.937+0000 7faa265e3740 4 client.18495 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:26:49.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.937+0000 7faa265e3740 1 -- 172.21.15.86:0/1871289766 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556dc1e618b0 con 0x556dc1f81800 2024-04-17T07:26:49.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.937+0000 7faa20e00640 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x556dc1f81800 0x556dc1f83bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:49.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.938+0000 7faa20e00640 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x556dc1f81800 0x556dc1f83bf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=419123c3b942aadd server_cookie=72400d74cd688042 in_seq=0 out_seq=0 2024-04-17T07:26:49.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.938+0000 7faa02a00640 10 client.18495 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:26:49.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.939+0000 7faa02a00640 1 -- 172.21.15.86:0/1871289766 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556dc1e618b0 con 0x556dc1f81800 2024-04-17T07:26:49.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.939+0000 7faa02a00640 10 client.18495 handle_command_reply: tid=0 2024-04-17T07:26:49.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.939+0000 7faa265e3740 1 client.18495 shutdown 2024-04-17T07:26:49.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.939+0000 7faa265e3740 1 -- 172.21.15.86:0/1871289766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 msgr2=0x556dc1f7f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.939+0000 7faa265e3740 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1f7f900 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7faa14099420 tx=0x7faa140bd120 comp rx=0 tx=0).stop 2024-04-17T07:26:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.940+0000 7faa265e3740 1 -- 172.21.15.86:0/1871289766 shutdown_connections 2024-04-17T07:26:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.940+0000 7faa265e3740 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556dc1ef9b00 0x556dc1f7f900 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.940+0000 7faa265e3740 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556dc1ef4990 0x556dc1f13e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.940+0000 7faa265e3740 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x556dc1f81800 0x556dc1f83bf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.940+0000 7faa265e3740 1 --2- 172.21.15.86:0/1871289766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556dc1ef3fa0 0x556dc1f138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.940+0000 7faa265e3740 1 -- 172.21.15.86:0/1871289766 >> 172.21.15.86:0/1871289766 conn(0x556dc1eef510 msgr2=0x556dc1ef2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.940+0000 7faa02a00640 0 client.18495 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:26:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.942+0000 7faa265e3740 1 -- 172.21.15.86:0/1871289766 shutdown_connections 2024-04-17T07:26:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.943+0000 7faa265e3740 1 -- 172.21.15.86:0/1871289766 wait complete. 2024-04-17T07:26:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.943+0000 7faa265e3740 20 client.18495 trim_cache size 0 max 16384 2024-04-17T07:26:49.943 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:26:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.944+0000 7faa23800640 1 -- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa040a3c70 msgr2=0x7faa040a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.944+0000 7faa23800640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa040a3c70 0x7faa040a6130 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7faa1c19fb40 tx=0x7faa0800b910 comp rx=0 tx=0).stop 2024-04-17T07:26:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.944+0000 7faa23800640 1 -- 172.21.15.86:0/1603552876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa1c10a170 msgr2=0x7faa1c1a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.944+0000 7faa23800640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa1c10a170 0x7faa1c1a5be0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faa1400e940 tx=0x7faa1400ee00 comp rx=0 tx=0).stop 2024-04-17T07:26:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.945+0000 7faa23800640 1 -- 172.21.15.86:0/1603552876 shutdown_connections 2024-04-17T07:26:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.945+0000 7faa23800640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faa040a3c70 0x7faa040a6130 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.945+0000 7faa23800640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa1c10a170 0x7faa1c1a5be0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.945+0000 7faa23800640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa1c1097b0 0x7faa1c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.945+0000 7faa23800640 1 --2- 172.21.15.86:0/1603552876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c1085c0 0x7faa1c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.945+0000 7faa23800640 1 -- 172.21.15.86:0/1603552876 >> 172.21.15.86:0/1603552876 conn(0x7faa1c077560 msgr2=0x7faa1c10c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:49.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.945+0000 7faa23800640 1 -- 172.21.15.86:0/1603552876 shutdown_connections 2024-04-17T07:26:49.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:49.945+0000 7faa23800640 1 -- 172.21.15.86:0/1603552876 wait complete. 2024-04-17T07:26:49.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:26:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.289+0000 7feda2000640 1 -- 172.21.15.86:0/2147424071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c103180 msgr2=0x7fed9c1055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.289+0000 7feda2000640 1 --2- 172.21.15.86:0/2147424071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c103180 0x7fed9c1055f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fed88009860 tx=0x7fed8802bb70 comp rx=0 tx=0).stop 2024-04-17T07:26:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.289+0000 7feda2000640 1 -- 172.21.15.86:0/2147424071 shutdown_connections 2024-04-17T07:26:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.289+0000 7feda2000640 1 --2- 172.21.15.86:0/2147424071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed9c105b30 0x7fed9c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.289+0000 7feda2000640 1 --2- 172.21.15.86:0/2147424071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c103180 0x7fed9c1055f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.289+0000 7feda2000640 1 --2- 172.21.15.86:0/2147424071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c100810 0x7fed9c102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.289+0000 7feda2000640 1 -- 172.21.15.86:0/2147424071 >> 172.21.15.86:0/2147424071 conn(0x7fed9c0fc410 msgr2=0x7fed9c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.290+0000 7feda2000640 1 -- 172.21.15.86:0/2147424071 shutdown_connections 2024-04-17T07:26:50.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.290+0000 7feda2000640 1 -- 172.21.15.86:0/2147424071 wait complete. 2024-04-17T07:26:50.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.291+0000 7feda2000640 1 Processor -- start 2024-04-17T07:26:50.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.292+0000 7feda2000640 1 -- start start 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.292+0000 7feda2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c100810 0x7fed9c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.292+0000 7feda2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c103180 0x7fed9c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.292+0000 7feda2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed9c105b30 0x7fed9c1a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.292+0000 7feda2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1140b0 con 0x7fed9c103180 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.292+0000 7feda2000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c113f30 con 0x7fed9c105b30 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.292+0000 7feda2000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c114230 con 0x7fed9c100810 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.293+0000 7fed9b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c100810 0x7fed9c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.293+0000 7fed9b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c100810 0x7fed9c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49548/0 (socket says 172.21.15.86:49548) 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.293+0000 7fed9b400640 1 -- 172.21.15.86:0/1076472752 learned_addr learned my addr 172.21.15.86:0/1076472752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.293+0000 7fed9aa00640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c103180 0x7fed9c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.293+0000 7fed9be00640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed9c105b30 0x7fed9c1a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.294+0000 7fed9aa00640 1 -- 172.21.15.86:0/1076472752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c100810 msgr2=0x7fed9c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:50.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.294+0000 7fed9aa00640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c100810 0x7fed9c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.294+0000 7fed9aa00640 1 -- 172.21.15.86:0/1076472752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed9c105b30 msgr2=0x7fed9c1a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:50.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.294+0000 7fed9aa00640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed9c105b30 0x7fed9c1a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.294+0000 7fed9aa00640 1 -- 172.21.15.86:0/1076472752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c1a6420 con 0x7fed9c103180 2024-04-17T07:26:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.294+0000 7fed9b400640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c100810 0x7fed9c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.294+0000 7fed9aa00640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c103180 0x7fed9c19ed90 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fed8802c0b0 tx=0x7fed880040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.295+0000 7fed93e00640 1 -- 172.21.15.86:0/1076472752 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88041020 con 0x7fed9c103180 2024-04-17T07:26:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.295+0000 7fed93e00640 1 -- 172.21.15.86:0/1076472752 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed880044d0 con 0x7fed9c103180 2024-04-17T07:26:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.295+0000 7fed93e00640 1 -- 172.21.15.86:0/1076472752 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88039040 con 0x7fed9c103180 2024-04-17T07:26:50.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.295+0000 7feda2000640 1 -- 172.21.15.86:0/1076472752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9c1a66b0 con 0x7fed9c103180 2024-04-17T07:26:50.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.295+0000 7feda2000640 1 -- 172.21.15.86:0/1076472752 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9c1a6b50 con 0x7fed9c103180 2024-04-17T07:26:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.300+0000 7feda4c76740 1 Processor -- start 2024-04-17T07:26:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.301+0000 7feda4c76740 1 -- start start 2024-04-17T07:26:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.301+0000 7feda4c76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f1fa0 0x5587125f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.301+0000 7feda4c76740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.301+0000 7feda4c76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f7b00 0x5587125f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.301+0000 7feda4c76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558712558020 con 0x5587125f1fa0 2024-04-17T07:26:50.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.301+0000 7feda4c76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5587124fe460 con 0x5587125f7b00 2024-04-17T07:26:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.301+0000 7feda4c76740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558712553fe0 con 0x5587125f2990 2024-04-17T07:26:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.301+0000 7fed93e00640 1 -- 172.21.15.86:0/1076472752 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fed88044020 con 0x7fed9c103180 2024-04-17T07:26:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f1fa0 0x5587125f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f7b00 0x5587125f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49560/0 (socket says 172.21.15.86:49560) 2024-04-17T07:26:50.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9b400640 1 -- 172.21.15.86:0/760821405 learned_addr learned my addr 172.21.15.86:0/760821405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9be00640 1 -- 172.21.15.86:0/760821405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 msgr2=0x5587125f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9be00640 1 --2- 172.21.15.86:0/760821405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9be00640 1 -- 172.21.15.86:0/760821405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f7b00 msgr2=0x5587125f9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9be00640 1 --2- 172.21.15.86:0/760821405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f7b00 0x5587125f9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9be00640 1 -- 172.21.15.86:0/760821405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558712506040 con 0x5587125f1fa0 2024-04-17T07:26:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.302+0000 7fed9aa00640 1 --2- 172.21.15.86:0/760821405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f7b00 0x5587125f9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:50.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.303+0000 7fed9be00640 1 --2- 172.21.15.86:0/760821405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f1fa0 0x5587125f23c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fed94009f00 tx=0x7fed94011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4427717fe2b7136 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.303+0000 7fed93e00640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fed800a3cb0 0x7fed800a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.304+0000 7fed93e00640 1 -- 172.21.15.86:0/1076472752 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fed880f1ff0 con 0x7fed9c103180 2024-04-17T07:26:50.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.304+0000 7fed87e00640 1 -- 172.21.15.86:0/760821405 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94004d20 con 0x5587125f1fa0 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.304+0000 7fed9b400640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fed800a3cb0 0x7fed800a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.304+0000 7fed87e00640 1 -- 172.21.15.86:0/760821405 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed94004030 con 0x5587125f1fa0 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.304+0000 7fed87e00640 1 -- 172.21.15.86:0/760821405 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94005280 con 0x5587125f1fa0 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.305+0000 7feda4c76740 1 -- 172.21.15.86:0/760821405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f1fa0 msgr2=0x5587125f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.305+0000 7feda4c76740 1 --2- 172.21.15.86:0/760821405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f1fa0 0x5587125f23c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fed94009f00 tx=0x7fed94011810 comp rx=0 tx=0).stop 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.305+0000 7fed9b400640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fed800a3cb0 0x7fed800a6170 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x5587125f2d90 tx=0x7fed8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.305+0000 7feda4c76740 1 -- 172.21.15.86:0/760821405 shutdown_connections 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.305+0000 7feda4c76740 1 --2- 172.21.15.86:0/760821405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f7b00 0x5587125f9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.305+0000 7feda4c76740 1 --2- 172.21.15.86:0/760821405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.305+0000 7feda4c76740 1 --2- 172.21.15.86:0/760821405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f1fa0 0x5587125f23c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.305+0000 7feda4c76740 1 -- 172.21.15.86:0/760821405 >> 172.21.15.86:0/760821405 conn(0x5587125ed510 msgr2=0x5587125ed910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.305+0000 7feda4c76740 1 -- 172.21.15.86:0/760821405 shutdown_connections 2024-04-17T07:26:50.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.306+0000 7feda4c76740 1 -- 172.21.15.86:0/760821405 wait complete. 2024-04-17T07:26:50.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.306+0000 7feda4c76740 1 -- start start 2024-04-17T07:26:50.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.307+0000 7feda4c76740 1 Processor -- start 2024-04-17T07:26:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.307+0000 7feda4c76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587125fc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.307+0000 7feda4c76740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125fca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.307+0000 7feda4c76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 0x558712603ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.307+0000 7feda4c76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5587125faab0 con 0x5587125f7b00 2024-04-17T07:26:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.307+0000 7feda4c76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558712511830 con 0x5587125f1fa0 2024-04-17T07:26:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.307+0000 7feda4c76740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5587125fac30 con 0x5587125f2990 2024-04-17T07:26:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.307+0000 7feda4c76740 10 client.0 _ll_register_callbacks cb 0x5587124dffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:50.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125fca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587125fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125fca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49576/0 (socket says 172.21.15.86:49576) 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9aa00640 1 -- 172.21.15.86:0/2059321936 learned_addr learned my addr 172.21.15.86:0/2059321936 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9be00640 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 0x558712603ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9aa00640 1 -- 172.21.15.86:0/2059321936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 msgr2=0x5587125fc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9aa00640 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587125fc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9aa00640 1 -- 172.21.15.86:0/2059321936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 msgr2=0x558712603ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9aa00640 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 0x558712603ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9aa00640 1 -- 172.21.15.86:0/2059321936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558712515920 con 0x5587125f2990 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9be00640 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 0x558712603ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.308+0000 7fed9b400640 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587125fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.309+0000 7fed9aa00640 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125fca80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fed880c6630 tx=0x7fed880095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab38f3a223250fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:50.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.309+0000 7fed84c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:50.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.309+0000 7fed84c00640 1 -- 172.21.15.86:0/2059321936 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed880bda30 con 0x5587125f2990 2024-04-17T07:26:50.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.309+0000 7fed84c00640 1 -- 172.21.15.86:0/2059321936 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fed880c66d0 con 0x5587125f2990 2024-04-17T07:26:50.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.309+0000 7feda4c76740 1 -- 172.21.15.86:0/2059321936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed94004030 con 0x5587125f2990 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.309+0000 7fed84c00640 1 -- 172.21.15.86:0/2059321936 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed880bd850 con 0x5587125f2990 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.310+0000 7feda4c76740 10 client.27454 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.310+0000 7feda4c76740 1 -- 172.21.15.86:0/2059321936 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587124e37f0 con 0x5587125f2990 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.310+0000 7fed84c00640 1 -- 172.21.15.86:0/2059321936 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed94004030 con 0x5587125f2990 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.311+0000 7fed84c00640 1 -- 172.21.15.86:0/2059321936 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fed8803c030 con 0x5587125f2990 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.311+0000 7feda4c76740 10 client.27454 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.311+0000 7feda4c76740 10 client.27454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.311+0000 7feda4c76740 20 client.27454 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.311+0000 7feda4c76740 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558712607f60 0x55871260a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.311+0000 7feda4c76740 4 client.27454 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.311+0000 7feda4c76740 1 -- 172.21.15.86:0/2059321936 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558712565a40 con 0x558712607f60 2024-04-17T07:26:50.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.312+0000 7fed9b400640 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558712607f60 0x55871260a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.312+0000 7fed9b400640 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558712607f60 0x55871260a3b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e43bcd72953520 server_cookie=35df27bec3d8f4c5 in_seq=0 out_seq=0 2024-04-17T07:26:50.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.312+0000 7fed84c00640 10 client.27454 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.314+0000 7fed84c00640 1 -- 172.21.15.86:0/2059321936 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558712565a40 con 0x558712607f60 2024-04-17T07:26:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.314+0000 7fed84c00640 10 client.27454 handle_command_reply: tid=0 2024-04-17T07:26:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.314+0000 7feda4c76740 1 client.27454 shutdown 2024-04-17T07:26:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.314+0000 7feda4c76740 1 -- 172.21.15.86:0/2059321936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 msgr2=0x5587125fca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:50.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.314+0000 7feda4c76740 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125fca80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fed880c6630 tx=0x7fed880095b0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.315+0000 7feda4c76740 1 -- 172.21.15.86:0/2059321936 shutdown_connections 2024-04-17T07:26:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.315+0000 7feda4c76740 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558712607f60 0x55871260a3b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.315+0000 7feda4c76740 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 0x558712603ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.315+0000 7feda4c76740 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x5587125fca80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.315+0000 7feda4c76740 1 --2- 172.21.15.86:0/2059321936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587125fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.315+0000 7feda4c76740 1 -- 172.21.15.86:0/2059321936 >> 172.21.15.86:0/2059321936 conn(0x5587125ed510 msgr2=0x5587125ec710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:50.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.315+0000 7fed84c00640 0 client.27454 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.315+0000 7feda4c76740 1 -- 172.21.15.86:0/2059321936 shutdown_connections 2024-04-17T07:26:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.316+0000 7feda4c76740 1 -- 172.21.15.86:0/2059321936 wait complete. 2024-04-17T07:26:50.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.316+0000 7feda4c76740 20 client.27454 trim_cache size 0 max 16384 2024-04-17T07:26:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.344+0000 7feda4c76740 1 Processor -- start 2024-04-17T07:26:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.344+0000 7feda4c76740 1 -- start start 2024-04-17T07:26:50.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.344+0000 7feda4c76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587125f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.344+0000 7feda4c76740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x558712603a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.344+0000 7feda4c76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 0x5587125f4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.345+0000 7feda4c76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1b1160 con 0x5587125f7b00 2024-04-17T07:26:50.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.345+0000 7feda4c76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1b2ca0 con 0x5587125f1fa0 2024-04-17T07:26:50.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.345+0000 7feda4c76740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1b1830 con 0x5587125f2990 2024-04-17T07:26:50.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.345+0000 7fed9aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x558712603a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.345+0000 7fed9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 0x5587125f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.346+0000 7fed9aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x558712603a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49592/0 (socket says 172.21.15.86:49592) 2024-04-17T07:26:50.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.346+0000 7fed9aa00640 1 -- 172.21.15.86:0/855828883 learned_addr learned my addr 172.21.15.86:0/855828883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:50.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.346+0000 7fed9b400640 1 --2- 172.21.15.86:0/855828883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587125f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.346+0000 7fed9aa00640 1 -- 172.21.15.86:0/855828883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 msgr2=0x5587125f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:50.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.346+0000 7fed9aa00640 1 --2- 172.21.15.86:0/855828883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587125f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.346+0000 7fed9aa00640 1 -- 172.21.15.86:0/855828883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 msgr2=0x5587125f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:50.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.346+0000 7fed9aa00640 1 --2- 172.21.15.86:0/855828883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 0x5587125f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.346+0000 7fed9aa00640 1 -- 172.21.15.86:0/855828883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871226a3f0 con 0x5587125f2990 2024-04-17T07:26:50.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.347+0000 7fed9be00640 1 --2- 172.21.15.86:0/855828883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 0x5587125f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:50.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.348+0000 7fed9aa00640 1 --2- 172.21.15.86:0/855828883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x558712603a80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fed88009830 tx=0x7fed880ef790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4523a2c53899acd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:50.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.349+0000 7fed87400640 1 -- 172.21.15.86:0/855828883 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed880bda30 con 0x5587125f2990 2024-04-17T07:26:50.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.349+0000 7fed87400640 1 -- 172.21.15.86:0/855828883 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed880be5f0 con 0x5587125f2990 2024-04-17T07:26:50.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.349+0000 7fed87400640 1 -- 172.21.15.86:0/855828883 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed880f50d0 con 0x5587125f2990 2024-04-17T07:26:50.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.350+0000 7feda4c76740 1 -- 172.21.15.86:0/855828883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 msgr2=0x558712603a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:50.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.350+0000 7feda4c76740 1 --2- 172.21.15.86:0/855828883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x558712603a80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fed88009830 tx=0x7fed880ef790 comp rx=0 tx=0).stop 2024-04-17T07:26:50.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.350+0000 7feda4c76740 1 -- 172.21.15.86:0/855828883 shutdown_connections 2024-04-17T07:26:50.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.350+0000 7feda4c76740 1 --2- 172.21.15.86:0/855828883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587125f7b00 0x5587125f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.350+0000 7feda4c76740 1 --2- 172.21.15.86:0/855828883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f2990 0x558712603a80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.350+0000 7feda4c76740 1 --2- 172.21.15.86:0/855828883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587125f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.350+0000 7feda4c76740 1 -- 172.21.15.86:0/855828883 >> 172.21.15.86:0/855828883 conn(0x5587125ed510 msgr2=0x5587125ed8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:50.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.350+0000 7feda4c76740 1 -- 172.21.15.86:0/855828883 shutdown_connections 2024-04-17T07:26:50.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.351+0000 7feda4c76740 1 -- 172.21.15.86:0/855828883 wait complete. 2024-04-17T07:26:50.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.351+0000 7feda4c76740 1 -- start start 2024-04-17T07:26:50.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.351+0000 7feda4c76740 1 Processor -- start 2024-04-17T07:26:50.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.352+0000 7feda4c76740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587126011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.352+0000 7feda4c76740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f7b00 0x5587126016f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.353+0000 7feda4c76740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587126026b0 0x558712602b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.353+0000 7feda4c76740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1b25d0 con 0x5587126026b0 2024-04-17T07:26:50.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.353+0000 7feda4c76740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1b1f00 con 0x5587125f1fa0 2024-04-17T07:26:50.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.353+0000 7feda4c76740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1af130 con 0x5587125f7b00 2024-04-17T07:26:50.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.353+0000 7feda4c76740 10 client.0 _ll_register_callbacks cb 0x7fed880f56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:50.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.353+0000 7fed9aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f7b00 0x5587126016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.353+0000 7fed9aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f7b00 0x5587126016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49604/0 (socket says 172.21.15.86:49604) 2024-04-17T07:26:50.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.353+0000 7fed9aa00640 1 -- 172.21.15.86:0/2801512438 learned_addr learned my addr 172.21.15.86:0/2801512438 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:50.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.353+0000 7fed9b400640 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587126011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.353+0000 7fed9be00640 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587126026b0 0x558712602b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.354+0000 7fed9aa00640 1 -- 172.21.15.86:0/2801512438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 msgr2=0x5587126011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:50.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.354+0000 7fed9aa00640 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587126011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.354+0000 7fed9aa00640 1 -- 172.21.15.86:0/2801512438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587126026b0 msgr2=0x558712602b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:50.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.354+0000 7fed9aa00640 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587126026b0 0x558712602b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.354+0000 7fed9aa00640 1 -- 172.21.15.86:0/2801512438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587123347d0 con 0x5587125f7b00 2024-04-17T07:26:50.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.355+0000 7fed9aa00640 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f7b00 0x5587126016f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed880ba870 tx=0x7fed880f2300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1c48fb4585e5fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:50.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.355+0000 7fed84c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:50.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.355+0000 7fed84c00640 1 -- 172.21.15.86:0/2801512438 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed880bda30 con 0x5587125f7b00 2024-04-17T07:26:50.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.357+0000 7feda4c76740 1 -- 172.21.15.86:0/2801512438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed880be5f0 con 0x5587125f7b00 2024-04-17T07:26:50.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.357+0000 7fed84c00640 1 -- 172.21.15.86:0/2801512438 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fed880efdd0 con 0x5587125f7b00 2024-04-17T07:26:50.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.357+0000 7fed84c00640 1 -- 172.21.15.86:0/2801512438 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed880f52e0 con 0x5587125f7b00 2024-04-17T07:26:50.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.358+0000 7feda4c76740 10 client.27460 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:50.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.358+0000 7feda4c76740 1 -- 172.21.15.86:0/2801512438 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587123347d0 con 0x5587125f7b00 2024-04-17T07:26:50.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.359+0000 7fed84c00640 1 -- 172.21.15.86:0/2801512438 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed880be5f0 con 0x5587125f7b00 2024-04-17T07:26:50.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.359+0000 7fed84c00640 1 -- 172.21.15.86:0/2801512438 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fed8803c040 con 0x5587125f7b00 2024-04-17T07:26:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.359+0000 7feda4c76740 10 client.27460 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.360+0000 7feda4c76740 10 client.27460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.360+0000 7feda4c76740 20 client.27460 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.360+0000 7feda4c76740 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55871266ebe0 0x558712671000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.360+0000 7feda4c76740 4 client.27460 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:26:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.360+0000 7feda4c76740 1 -- 172.21.15.86:0/2801512438 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55871255f8b0 con 0x55871266ebe0 2024-04-17T07:26:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.360+0000 7fed9b400640 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55871266ebe0 0x558712671000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.362+0000 7fed9b400640 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55871266ebe0 0x558712671000 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d01883c4b29cd66 server_cookie=2bd25d59509436f9 in_seq=0 out_seq=0 2024-04-17T07:26:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.362+0000 7fed84c00640 10 client.27460 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.362+0000 7fed84c00640 1 -- 172.21.15.86:0/2801512438 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55871255f8b0 con 0x55871266ebe0 2024-04-17T07:26:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.362+0000 7fed84c00640 10 client.27460 handle_command_reply: tid=0 2024-04-17T07:26:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.363+0000 7feda4c76740 1 client.27460 shutdown 2024-04-17T07:26:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.363+0000 7feda4c76740 1 -- 172.21.15.86:0/2801512438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f7b00 msgr2=0x5587126016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.363+0000 7feda4c76740 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f7b00 0x5587126016f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed880ba870 tx=0x7fed880f2300 comp rx=0 tx=0).stop 2024-04-17T07:26:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.363+0000 7feda4c76740 1 -- 172.21.15.86:0/2801512438 shutdown_connections 2024-04-17T07:26:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.363+0000 7feda4c76740 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55871266ebe0 0x558712671000 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.363+0000 7feda4c76740 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5587126026b0 0x558712602b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.363+0000 7feda4c76740 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5587125f7b00 0x5587126016f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.363+0000 7feda4c76740 1 --2- 172.21.15.86:0/2801512438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5587125f1fa0 0x5587126011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.363+0000 7fed84c00640 0 client.27460 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.363+0000 7feda4c76740 1 -- 172.21.15.86:0/2801512438 >> 172.21.15.86:0/2801512438 conn(0x5587125ed510 msgr2=0x5587125fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.364+0000 7feda4c76740 1 -- 172.21.15.86:0/2801512438 shutdown_connections 2024-04-17T07:26:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.364+0000 7feda4c76740 1 -- 172.21.15.86:0/2801512438 wait complete. 2024-04-17T07:26:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.364+0000 7feda4c76740 20 client.27460 trim_cache size 0 max 16384 2024-04-17T07:26:50.365 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:26:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.366+0000 7feda2000640 1 -- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fed800a3cb0 msgr2=0x7fed800a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.366+0000 7feda2000640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fed800a3cb0 0x7fed800a6170 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x5587125f2d90 tx=0x7fed8c009260 comp rx=0 tx=0).stop 2024-04-17T07:26:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.366+0000 7feda2000640 1 -- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c103180 msgr2=0x7fed9c19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.366+0000 7feda2000640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c103180 0x7fed9c19ed90 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fed8802c0b0 tx=0x7fed880040e0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.366+0000 7feda2000640 1 -- 172.21.15.86:0/1076472752 shutdown_connections 2024-04-17T07:26:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.366+0000 7feda2000640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fed800a3cb0 0x7fed800a6170 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.366+0000 7feda2000640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed9c105b30 0x7fed9c1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.366+0000 7feda2000640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed9c103180 0x7fed9c19ed90 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.366+0000 7feda2000640 1 --2- 172.21.15.86:0/1076472752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c100810 0x7fed9c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.366+0000 7feda2000640 1 -- 172.21.15.86:0/1076472752 >> 172.21.15.86:0/1076472752 conn(0x7fed9c0fc410 msgr2=0x7fed9c1012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.368+0000 7feda2000640 1 -- 172.21.15.86:0/1076472752 shutdown_connections 2024-04-17T07:26:50.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:50.368+0000 7feda2000640 1 -- 172.21.15.86:0/1076472752 wait complete. 2024-04-17T07:26:51.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:51 smithi086 ceph-mon[31733]: pgmap v814: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.9 MiB/s wr, 677 op/s 2024-04-17T07:26:51.400 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:26:51.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:26:51.400 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:26:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:51 smithi098 ceph-mon[35418]: pgmap v814: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.9 MiB/s wr, 677 op/s 2024-04-17T07:26:51.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:51 smithi103 ceph-mon[35495]: pgmap v814: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.9 MiB/s wr, 677 op/s 2024-04-17T07:26:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.724+0000 7f3084400640 1 -- 172.21.15.86:0/3332255905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c1085c0 msgr2=0x7f307c1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.724+0000 7f3084400640 1 --2- 172.21.15.86:0/3332255905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c1085c0 0x7f307c1089e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3074009be0 tx=0x7f307402bef0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.725+0000 7f3084400640 1 -- 172.21.15.86:0/3332255905 shutdown_connections 2024-04-17T07:26:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.725+0000 7f3084400640 1 --2- 172.21.15.86:0/3332255905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c10a170 0x7f307c110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.725+0000 7f3084400640 1 --2- 172.21.15.86:0/3332255905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f307c1097b0 0x7f307c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.725+0000 7f3084400640 1 --2- 172.21.15.86:0/3332255905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c1085c0 0x7f307c1089e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.725+0000 7f3084400640 1 -- 172.21.15.86:0/3332255905 >> 172.21.15.86:0/3332255905 conn(0x7f307c077560 msgr2=0x7f307c0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.725+0000 7f3084400640 1 -- 172.21.15.86:0/3332255905 shutdown_connections 2024-04-17T07:26:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.725+0000 7f3084400640 1 -- 172.21.15.86:0/3332255905 wait complete. 2024-04-17T07:26:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.727+0000 7f3084400640 1 Processor -- start 2024-04-17T07:26:51.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.727+0000 7f3084400640 1 -- start start 2024-04-17T07:26:51.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.727+0000 7f3084400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c1085c0 0x7f307c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.727+0000 7f3084400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f307c1097b0 0x7f307c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.727+0000 7f3084400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c10a170 0x7f307c1a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.727+0000 7f3084400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f307c114130 con 0x7f307c1085c0 2024-04-17T07:26:51.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.727+0000 7f3084400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f307c113fb0 con 0x7f307c1097b0 2024-04-17T07:26:51.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.727+0000 7f3084400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1142b0 con 0x7f307c10a170 2024-04-17T07:26:51.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.728+0000 7f3081a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c1085c0 0x7f307c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.728+0000 7f3081a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c1085c0 0x7f307c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54012/0 (socket says 172.21.15.86:54012) 2024-04-17T07:26:51.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.728+0000 7f3081a00640 1 -- 172.21.15.86:0/1431433514 learned_addr learned my addr 172.21.15.86:0/1431433514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:51.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.728+0000 7f3081000640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f307c1097b0 0x7f307c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.728+0000 7f3082400640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c10a170 0x7f307c1a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.729+0000 7f3081a00640 1 -- 172.21.15.86:0/1431433514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c10a170 msgr2=0x7f307c1a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:51.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.729+0000 7f3081a00640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c10a170 0x7f307c1a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.729+0000 7f3081a00640 1 -- 172.21.15.86:0/1431433514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f307c1097b0 msgr2=0x7f307c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:51.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.729+0000 7f3081a00640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f307c1097b0 0x7f307c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.729+0000 7f3081a00640 1 -- 172.21.15.86:0/1431433514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307c1a6470 con 0x7f307c1085c0 2024-04-17T07:26:51.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.729+0000 7f3081a00640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c1085c0 0x7f307c19e850 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3074000f80 tx=0x7f3074033c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:51.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.730+0000 7f307a000640 1 -- 172.21.15.86:0/1431433514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074043020 con 0x7f307c1085c0 2024-04-17T07:26:51.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.730+0000 7f307a000640 1 -- 172.21.15.86:0/1431433514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3074033e30 con 0x7f307c1085c0 2024-04-17T07:26:51.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.730+0000 7f307a000640 1 -- 172.21.15.86:0/1431433514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074034690 con 0x7f307c1085c0 2024-04-17T07:26:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.731+0000 7f3084400640 1 -- 172.21.15.86:0/1431433514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307c1a6700 con 0x7f307c1085c0 2024-04-17T07:26:51.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.731+0000 7f3084400640 1 -- 172.21.15.86:0/1431433514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307c1a6c00 con 0x7f307c1085c0 2024-04-17T07:26:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.736+0000 7f30871d2740 1 Processor -- start 2024-04-17T07:26:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.736+0000 7f30871d2740 1 -- start start 2024-04-17T07:26:51.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.736+0000 7f30871d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3ecc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.737+0000 7f30871d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecc8b0 0x5581d3ed13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.737+0000 7f30871d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ed1a20 0x5581d3ed3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.737+0000 7f30871d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581d3e31cb0 con 0x5581d3ecbec0 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.737+0000 7f3082400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3ecc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.737+0000 7f3082400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3ecc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54016/0 (socket says 172.21.15.86:54016) 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.737+0000 7f3082400640 1 -- 172.21.15.86:0/4273863486 learned_addr learned my addr 172.21.15.86:0/4273863486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.737+0000 7f30871d2740 1 -- 172.21.15.86:0/4273863486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5581d3dd8370 con 0x5581d3ecc8b0 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.737+0000 7f30871d2740 1 -- 172.21.15.86:0/4273863486 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5581d3e2dc70 con 0x5581d3ed1a20 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.738+0000 7f3081000640 1 --2- 172.21.15.86:0/4273863486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ed1a20 0x5581d3ed3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.738+0000 7f3082400640 1 -- 172.21.15.86:0/4273863486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ed1a20 msgr2=0x5581d3ed3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.738+0000 7f3082400640 1 --2- 172.21.15.86:0/4273863486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ed1a20 0x5581d3ed3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.738+0000 7f3082400640 1 -- 172.21.15.86:0/4273863486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecc8b0 msgr2=0x5581d3ed13b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:26:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.738+0000 7f3082400640 1 --2- 172.21.15.86:0/4273863486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecc8b0 0x5581d3ed13b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.738+0000 7f3082400640 1 -- 172.21.15.86:0/4273863486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d3df7910 con 0x5581d3ecbec0 2024-04-17T07:26:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.739+0000 7f3082400640 1 --2- 172.21.15.86:0/4273863486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3ecc2e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3070009ca0 tx=0x7f3070011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145fb986df9c0c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.739+0000 7f307a000640 1 -- 172.21.15.86:0/1431433514 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3074046020 con 0x7f307c1085c0 2024-04-17T07:26:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.739+0000 7f306ea00640 1 -- 172.21.15.86:0/4273863486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307001e070 con 0x5581d3ecbec0 2024-04-17T07:26:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.739+0000 7f306ea00640 1 -- 172.21.15.86:0/4273863486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3070009e50 con 0x5581d3ecbec0 2024-04-17T07:26:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.739+0000 7f306ea00640 1 -- 172.21.15.86:0/4273863486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307000e9a0 con 0x5581d3ecbec0 2024-04-17T07:26:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.740+0000 7f30871d2740 1 -- 172.21.15.86:0/4273863486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 msgr2=0x5581d3ecc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.740+0000 7f30871d2740 1 --2- 172.21.15.86:0/4273863486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3ecc2e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3070009ca0 tx=0x7f3070011810 comp rx=0 tx=0).stop 2024-04-17T07:26:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.741+0000 7f30871d2740 1 -- 172.21.15.86:0/4273863486 shutdown_connections 2024-04-17T07:26:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.741+0000 7f30871d2740 1 --2- 172.21.15.86:0/4273863486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ed1a20 0x5581d3ed3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.741+0000 7f30871d2740 1 --2- 172.21.15.86:0/4273863486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecc8b0 0x5581d3ed13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.741+0000 7f30871d2740 1 --2- 172.21.15.86:0/4273863486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3ecc2e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.741+0000 7f30871d2740 1 -- 172.21.15.86:0/4273863486 >> 172.21.15.86:0/4273863486 conn(0x5581d3ec7430 msgr2=0x5581d3ec7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.742+0000 7f30871d2740 1 -- 172.21.15.86:0/4273863486 shutdown_connections 2024-04-17T07:26:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.742+0000 7f30871d2740 1 -- 172.21.15.86:0/4273863486 wait complete. 2024-04-17T07:26:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.742+0000 7f30871d2740 1 -- start start 2024-04-17T07:26:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.743+0000 7f30871d2740 1 Processor -- start 2024-04-17T07:26:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.743+0000 7f30871d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ecbec0 0x5581d3ee7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.743+0000 7f30871d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ee7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.744+0000 7f30871d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3eecba0 0x5581d3eeef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.744+0000 7f30871d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581d3ee59e0 con 0x5581d3ed1a20 2024-04-17T07:26:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.744+0000 7f30871d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5581d3deb730 con 0x5581d3eecba0 2024-04-17T07:26:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.744+0000 7f30871d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5581d3ee5b60 con 0x5581d3ecbec0 2024-04-17T07:26:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.744+0000 7f30871d2740 10 client.0 _ll_register_callbacks cb 0x5581d3dedb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.744+0000 7f307a000640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f30640a3c40 0x7f30640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.744+0000 7f3082400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ecbec0 0x5581d3ee7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.744+0000 7f3081000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ee7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.745+0000 7f3082400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3eecba0 0x5581d3eeef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.745+0000 7f3081000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ee7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54018/0 (socket says 172.21.15.86:54018) 2024-04-17T07:26:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.745+0000 7f3081000640 1 -- 172.21.15.86:0/96549440 learned_addr learned my addr 172.21.15.86:0/96549440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.745+0000 7f3081000640 1 -- 172.21.15.86:0/96549440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ecbec0 msgr2=0x5581d3ee7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.745+0000 7f3081000640 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ecbec0 0x5581d3ee7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.745+0000 7f3081000640 1 -- 172.21.15.86:0/96549440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3eecba0 msgr2=0x5581d3eeef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.745+0000 7f307a000640 1 -- 172.21.15.86:0/1431433514 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f30740f2e90 con 0x7f307c1085c0 2024-04-17T07:26:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.745+0000 7f3081a00640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f30640a3c40 0x7f30640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.746+0000 7f3081000640 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3eecba0 0x5581d3eeef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.746+0000 7f3081000640 1 -- 172.21.15.86:0/96549440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d3c0c7e0 con 0x5581d3ed1a20 2024-04-17T07:26:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.746+0000 7f3081000640 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ee7b70 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3068012700 tx=0x7f3068012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df3f26d9d92ce96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.747+0000 7f3063400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.747+0000 7f3063400640 1 -- 172.21.15.86:0/96549440 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068010d60 con 0x5581d3ed1a20 2024-04-17T07:26:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.747+0000 7f30871d2740 1 -- 172.21.15.86:0/96549440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3070009e50 con 0x5581d3ed1a20 2024-04-17T07:26:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.747+0000 7f3063400640 1 -- 172.21.15.86:0/96549440 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3068004020 con 0x5581d3ed1a20 2024-04-17T07:26:51.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.747+0000 7f3081a00640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f30640a3c40 0x7f30640a6100 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x5581d3eccd70 tx=0x7f3074009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.748+0000 7f3063400640 1 -- 172.21.15.86:0/96549440 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30680042a0 con 0x5581d3ed1a20 2024-04-17T07:26:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.748+0000 7f30871d2740 10 client.18525 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.748+0000 7f30871d2740 1 -- 172.21.15.86:0/96549440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581d3c6cf70 con 0x5581d3ed1a20 2024-04-17T07:26:51.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.748+0000 7f3063400640 1 -- 172.21.15.86:0/96549440 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3070009e50 con 0x5581d3ed1a20 2024-04-17T07:26:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.749+0000 7f3063400640 1 -- 172.21.15.86:0/96549440 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3068019030 con 0x5581d3ed1a20 2024-04-17T07:26:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.749+0000 7f30871d2740 10 client.18525 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.749+0000 7f30871d2740 10 client.18525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.749+0000 7f30871d2740 20 client.18525 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.749+0000 7f30871d2740 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581d3ef3a10 0x5581d3ef5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.749+0000 7f30871d2740 4 client.18525 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:51.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.749+0000 7f30871d2740 1 -- 172.21.15.86:0/96549440 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581d3e3f6d0 con 0x5581d3ef3a10 2024-04-17T07:26:51.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.750+0000 7f3082400640 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581d3ef3a10 0x5581d3ef5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.753+0000 7f3082400640 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581d3ef3a10 0x5581d3ef5e20 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50cd55926e5a1c0 server_cookie=587c4139294ca87c in_seq=0 out_seq=0 2024-04-17T07:26:51.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.753+0000 7f3063400640 10 client.18525 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:51.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.756+0000 7f3063400640 1 -- 172.21.15.86:0/96549440 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581d3e3f6d0 con 0x5581d3ef3a10 2024-04-17T07:26:51.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.756+0000 7f3063400640 10 client.18525 handle_command_reply: tid=0 2024-04-17T07:26:51.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.756+0000 7f30871d2740 1 client.18525 shutdown 2024-04-17T07:26:51.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.757+0000 7f30871d2740 1 -- 172.21.15.86:0/96549440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 msgr2=0x5581d3ee7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:51.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.757+0000 7f30871d2740 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ee7b70 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3068012700 tx=0x7f3068012bc0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.757+0000 7f30871d2740 1 -- 172.21.15.86:0/96549440 shutdown_connections 2024-04-17T07:26:51.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.757+0000 7f30871d2740 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581d3ef3a10 0x5581d3ef5e20 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.757+0000 7f30871d2740 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3eecba0 0x5581d3eeef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.757+0000 7f30871d2740 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ee7b70 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.757+0000 7f30871d2740 1 --2- 172.21.15.86:0/96549440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ecbec0 0x5581d3ee7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.757+0000 7f30871d2740 1 -- 172.21.15.86:0/96549440 >> 172.21.15.86:0/96549440 conn(0x5581d3ec7430 msgr2=0x5581d3ec65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.761+0000 7f3063400640 0 client.18525 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.761+0000 7f30871d2740 1 -- 172.21.15.86:0/96549440 shutdown_connections 2024-04-17T07:26:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.761+0000 7f30871d2740 1 -- 172.21.15.86:0/96549440 wait complete. 2024-04-17T07:26:51.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.761+0000 7f30871d2740 20 client.18525 trim_cache size 0 max 16384 2024-04-17T07:26:51.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.804+0000 7f30871d2740 1 Processor -- start 2024-04-17T07:26:51.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.804+0000 7f30871d2740 1 -- start start 2024-04-17T07:26:51.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.804+0000 7f30871d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecbec0 0x5581d3ece000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.804+0000 7f30871d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ece540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.804+0000 7f30871d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3eecba0 0x5581d3ee5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.804+0000 7f30871d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1a87c0 con 0x5581d3ed1a20 2024-04-17T07:26:51.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.804+0000 7f30871d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1aa300 con 0x5581d3ecbec0 2024-04-17T07:26:51.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.804+0000 7f30871d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1a8e90 con 0x5581d3eecba0 2024-04-17T07:26:51.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.805+0000 7f3081000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ece540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.805+0000 7f3082400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecbec0 0x5581d3ece000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.805+0000 7f3082400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecbec0 0x5581d3ece000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:44072/0 (socket says 172.21.15.86:44072) 2024-04-17T07:26:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.805+0000 7f3082400640 1 -- 172.21.15.86:0/4216250658 learned_addr learned my addr 172.21.15.86:0/4216250658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.805+0000 7f3082400640 1 --2- 172.21.15.86:0/4216250658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3eecba0 0x5581d3ee5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.806+0000 7f3081000640 1 -- 172.21.15.86:0/4216250658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3eecba0 msgr2=0x5581d3ee5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.806+0000 7f3081000640 1 --2- 172.21.15.86:0/4216250658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3eecba0 0x5581d3ee5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.806+0000 7f3081000640 1 -- 172.21.15.86:0/4216250658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecbec0 msgr2=0x5581d3ece000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.806+0000 7f3081000640 1 --2- 172.21.15.86:0/4216250658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecbec0 0x5581d3ece000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.806+0000 7f3081000640 1 -- 172.21.15.86:0/4216250658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d3b04170 con 0x5581d3ed1a20 2024-04-17T07:26:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.806+0000 7f3082400640 1 --2- 172.21.15.86:0/4216250658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecbec0 0x5581d3ece000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:51.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.807+0000 7f3081000640 1 --2- 172.21.15.86:0/4216250658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ece540 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3068001eb0 tx=0x7f3068009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ae52539086437d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:51.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.807+0000 7f306e000640 1 -- 172.21.15.86:0/4216250658 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068033070 con 0x5581d3ed1a20 2024-04-17T07:26:51.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.808+0000 7f306e000640 1 -- 172.21.15.86:0/4216250658 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f306800f810 con 0x5581d3ed1a20 2024-04-17T07:26:51.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.808+0000 7f306e000640 1 -- 172.21.15.86:0/4216250658 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30680052a0 con 0x5581d3ed1a20 2024-04-17T07:26:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.809+0000 7f30871d2740 1 -- 172.21.15.86:0/4216250658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 msgr2=0x5581d3ece540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.809+0000 7f30871d2740 1 --2- 172.21.15.86:0/4216250658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ece540 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3068001eb0 tx=0x7f3068009400 comp rx=0 tx=0).stop 2024-04-17T07:26:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.809+0000 7f30871d2740 1 -- 172.21.15.86:0/4216250658 shutdown_connections 2024-04-17T07:26:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.809+0000 7f30871d2740 1 --2- 172.21.15.86:0/4216250658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3eecba0 0x5581d3ee5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.809+0000 7f30871d2740 1 --2- 172.21.15.86:0/4216250658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ed1a20 0x5581d3ece540 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.809+0000 7f30871d2740 1 --2- 172.21.15.86:0/4216250658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3ecbec0 0x5581d3ece000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.809+0000 7f30871d2740 1 -- 172.21.15.86:0/4216250658 >> 172.21.15.86:0/4216250658 conn(0x5581d3ec7430 msgr2=0x5581d3ee7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:51.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.810+0000 7f30871d2740 1 -- 172.21.15.86:0/4216250658 shutdown_connections 2024-04-17T07:26:51.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.811+0000 7f30871d2740 1 -- 172.21.15.86:0/4216250658 wait complete. 2024-04-17T07:26:51.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.811+0000 7f30871d2740 1 -- start start 2024-04-17T07:26:51.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.811+0000 7f30871d2740 1 Processor -- start 2024-04-17T07:26:51.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.812+0000 7f30871d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3f04bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.813+0000 7f3082400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3f04bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.813+0000 7f3082400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3f04bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54036/0 (socket says 172.21.15.86:54036) 2024-04-17T07:26:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.813+0000 7f30871d2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ed1a20 0x5581d3f05100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.813+0000 7f30871d2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3eecba0 0x5581d3f060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.813+0000 7f30871d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1a9c30 con 0x5581d3ecbec0 2024-04-17T07:26:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.813+0000 7f30871d2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1a9560 con 0x5581d3eecba0 2024-04-17T07:26:51.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.813+0000 7f30871d2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f307c0828e0 con 0x5581d3ed1a20 2024-04-17T07:26:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.813+0000 7f30871d2740 10 client.0 _ll_register_callbacks cb 0x7f30680095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.813+0000 7f3082400640 1 -- 172.21.15.86:0/3444292325 learned_addr learned my addr 172.21.15.86:0/3444292325 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:51.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.814+0000 7f3081000640 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ed1a20 0x5581d3f05100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.814+0000 7f3082400640 1 -- 172.21.15.86:0/3444292325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ed1a20 msgr2=0x5581d3f05100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.814+0000 7f3082400640 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ed1a20 0x5581d3f05100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.814+0000 7f3082400640 1 -- 172.21.15.86:0/3444292325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3eecba0 msgr2=0x5581d3f060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.814+0000 7f3082400640 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3eecba0 0x5581d3f060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.814+0000 7f3082400640 1 -- 172.21.15.86:0/3444292325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d3f066b0 con 0x5581d3ecbec0 2024-04-17T07:26:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.814+0000 7f3082400640 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3f04bc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3070009c70 tx=0x7f3070005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376471506909046 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:51.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.815+0000 7f3063400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:26:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.815+0000 7f3063400640 1 -- 172.21.15.86:0/3444292325 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307001e040 con 0x5581d3ecbec0 2024-04-17T07:26:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.815+0000 7f3063400640 1 -- 172.21.15.86:0/3444292325 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f30700053e0 con 0x5581d3ecbec0 2024-04-17T07:26:51.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.815+0000 7f30871d2740 1 -- 172.21.15.86:0/3444292325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f306800f810 con 0x5581d3ecbec0 2024-04-17T07:26:51.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.817+0000 7f3063400640 1 -- 172.21.15.86:0/3444292325 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f307003f050 con 0x5581d3ecbec0 2024-04-17T07:26:51.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.818+0000 7f30871d2740 10 client.18531 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:51.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.818+0000 7f30871d2740 1 -- 172.21.15.86:0/3444292325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581d3f59c80 con 0x5581d3ecbec0 2024-04-17T07:26:51.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.819+0000 7f3063400640 1 -- 172.21.15.86:0/3444292325 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f306800f810 con 0x5581d3ecbec0 2024-04-17T07:26:51.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.819+0000 7f3063400640 1 -- 172.21.15.86:0/3444292325 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3070019660 con 0x5581d3ecbec0 2024-04-17T07:26:51.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.819+0000 7f30871d2740 10 client.18531 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:51.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.819+0000 7f30871d2740 10 client.18531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:51.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.819+0000 7f30871d2740 20 client.18531 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:51.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.819+0000 7f30871d2740 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581d3f5a510 0x5581d3f5c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:51.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.819+0000 7f30871d2740 4 client.18531 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:26:51.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.819+0000 7f30871d2740 1 -- 172.21.15.86:0/3444292325 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5581d3ee7420 con 0x5581d3f5a510 2024-04-17T07:26:51.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.820+0000 7f3081000640 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581d3f5a510 0x5581d3f5c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:51.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.821+0000 7f3081000640 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581d3f5a510 0x5581d3f5c930 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f135c7a5beb9718b server_cookie=bb4289c5adbef282 in_seq=0 out_seq=0 2024-04-17T07:26:51.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.821+0000 7f3063400640 10 client.18531 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:51.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.824+0000 7f3063400640 1 -- 172.21.15.86:0/3444292325 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5581d3ee7420 con 0x5581d3f5a510 2024-04-17T07:26:51.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.824+0000 7f3063400640 10 client.18531 handle_command_reply: tid=0 2024-04-17T07:26:51.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.824+0000 7f30871d2740 1 client.18531 shutdown 2024-04-17T07:26:51.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.826+0000 7f30871d2740 1 -- 172.21.15.86:0/3444292325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 msgr2=0x5581d3f04bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:51.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.826+0000 7f30871d2740 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3f04bc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3070009c70 tx=0x7f3070005270 comp rx=0 tx=0).stop 2024-04-17T07:26:51.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.826+0000 7f30871d2740 1 -- 172.21.15.86:0/3444292325 shutdown_connections 2024-04-17T07:26:51.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.826+0000 7f30871d2740 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581d3f5a510 0x5581d3f5c930 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.826+0000 7f30871d2740 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581d3eecba0 0x5581d3f060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.826+0000 7f30871d2740 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581d3ed1a20 0x5581d3f05100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.826+0000 7f30871d2740 1 --2- 172.21.15.86:0/3444292325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581d3ecbec0 0x5581d3f04bc0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.826+0000 7f30871d2740 1 -- 172.21.15.86:0/3444292325 >> 172.21.15.86:0/3444292325 conn(0x5581d3ec7430 msgr2=0x5581d3ee75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:51.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.826+0000 7f3063400640 0 client.18531 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.829+0000 7f30871d2740 1 -- 172.21.15.86:0/3444292325 shutdown_connections 2024-04-17T07:26:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.829+0000 7f30871d2740 1 -- 172.21.15.86:0/3444292325 wait complete. 2024-04-17T07:26:51.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.829+0000 7f30871d2740 20 client.18531 trim_cache size 0 max 16384 2024-04-17T07:26:51.830 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:26:51.830 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:26:51.831 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "0e9db878-7d03-4be0-850e-b27feb28f077", 2024-04-17T07:26:51.831 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:26:51.831 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:26:51.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.831+0000 7f3084400640 1 -- 172.21.15.86:0/1431433514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f30640a3c40 msgr2=0x7f30640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:51.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.831+0000 7f3084400640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f30640a3c40 0x7f30640a6100 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x5581d3eccd70 tx=0x7f3074009500 comp rx=0 tx=0).stop 2024-04-17T07:26:51.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.831+0000 7f3084400640 1 -- 172.21.15.86:0/1431433514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c1085c0 msgr2=0x7f307c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:51.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.831+0000 7f3084400640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c1085c0 0x7f307c19e850 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3074000f80 tx=0x7f3074033c40 comp rx=0 tx=0).stop 2024-04-17T07:26:51.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.834+0000 7f3084400640 1 -- 172.21.15.86:0/1431433514 shutdown_connections 2024-04-17T07:26:51.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.834+0000 7f3084400640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f30640a3c40 0x7f30640a6100 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.834+0000 7f3084400640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f307c10a170 0x7f307c1a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.834+0000 7f3084400640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f307c1097b0 0x7f307c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.834+0000 7f3084400640 1 --2- 172.21.15.86:0/1431433514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f307c1085c0 0x7f307c19e850 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.834+0000 7f3084400640 1 -- 172.21.15.86:0/1431433514 >> 172.21.15.86:0/1431433514 conn(0x7f307c077560 msgr2=0x7f307c10cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.835+0000 7f3084400640 1 -- 172.21.15.86:0/1431433514 shutdown_connections 2024-04-17T07:26:51.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:51.835+0000 7f3084400640 1 -- 172.21.15.86:0/1431433514 wait complete. 2024-04-17T07:26:51.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:26:51.866 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:26:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.200+0000 7f6891600640 1 -- 172.21.15.86:0/742110404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c10b8d0 msgr2=0x7f688c10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.200+0000 7f6891600640 1 --2- 172.21.15.86:0/742110404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c10b8d0 0x7f688c10dd60 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f6878009900 tx=0x7f687802bc30 comp rx=0 tx=0).stop 2024-04-17T07:26:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.200+0000 7f6891600640 1 -- 172.21.15.86:0/742110404 shutdown_connections 2024-04-17T07:26:52.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.200+0000 7f6891600640 1 --2- 172.21.15.86:0/742110404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c10e2a0 0x7f688c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.200+0000 7f6891600640 1 --2- 172.21.15.86:0/742110404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c10b8d0 0x7f688c10dd60 secure :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f6878009900 tx=0x7f687802bc30 comp rx=0 tx=0).stop 2024-04-17T07:26:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.200+0000 7f6891600640 1 --2- 172.21.15.86:0/742110404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f688c108f60 0x7f688c10b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.200+0000 7f6891600640 1 -- 172.21.15.86:0/742110404 >> 172.21.15.86:0/742110404 conn(0x7f688c06edf0 msgr2=0x7f688c071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.200+0000 7f6891600640 1 -- 172.21.15.86:0/742110404 shutdown_connections 2024-04-17T07:26:52.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.200+0000 7f6891600640 1 -- 172.21.15.86:0/742110404 wait complete. 2024-04-17T07:26:52.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.202+0000 7f6891600640 1 Processor -- start 2024-04-17T07:26:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.202+0000 7f6891600640 1 -- start start 2024-04-17T07:26:52.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.203+0000 7f6891600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c108f60 0x7f688c1a7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.203+0000 7f6891600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f688c10e2a0 0x7f688c1a7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.203+0000 7f6891600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c1ac7f0 0x7f688c1aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.203+0000 7f6891600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f688c11c910 con 0x7f688c108f60 2024-04-17T07:26:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.203+0000 7f6891600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f688c11c790 con 0x7f688c1ac7f0 2024-04-17T07:26:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.203+0000 7f6891600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f688c11ca90 con 0x7f688c10e2a0 2024-04-17T07:26:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.204+0000 7f688aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c108f60 0x7f688c1a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.204+0000 7f688aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c108f60 0x7f688c1a7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54056/0 (socket says 172.21.15.86:54056) 2024-04-17T07:26:52.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.204+0000 7f688aa00640 1 -- 172.21.15.86:0/3022573857 learned_addr learned my addr 172.21.15.86:0/3022573857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.204+0000 7f688a000640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f688c10e2a0 0x7f688c1a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.204+0000 7f688b400640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c1ac7f0 0x7f688c1aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.205+0000 7f688aa00640 1 -- 172.21.15.86:0/3022573857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f688c10e2a0 msgr2=0x7f688c1a7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:52.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.205+0000 7f688aa00640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f688c10e2a0 0x7f688c1a7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.205+0000 7f688aa00640 1 -- 172.21.15.86:0/3022573857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c1ac7f0 msgr2=0x7f688c1aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:52.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.205+0000 7f688aa00640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c1ac7f0 0x7f688c1aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.205+0000 7f688aa00640 1 -- 172.21.15.86:0/3022573857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c1af250 con 0x7f688c108f60 2024-04-17T07:26:52.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.206+0000 7f688aa00640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c108f60 0x7f688c1a7250 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f687c00d950 tx=0x7f687c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.207+0000 7f6883400640 1 -- 172.21.15.86:0/3022573857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c017070 con 0x7f688c108f60 2024-04-17T07:26:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.207+0000 7f6883400640 1 -- 172.21.15.86:0/3022573857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f687c009c30 con 0x7f688c108f60 2024-04-17T07:26:52.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.207+0000 7f6883400640 1 -- 172.21.15.86:0/3022573857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c00f620 con 0x7f688c108f60 2024-04-17T07:26:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.207+0000 7f6891600640 1 -- 172.21.15.86:0/3022573857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688c1af540 con 0x7f688c108f60 2024-04-17T07:26:52.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.207+0000 7f6891600640 1 -- 172.21.15.86:0/3022573857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688c1afa10 con 0x7f688c108f60 2024-04-17T07:26:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.214+0000 7f6883400640 1 -- 172.21.15.86:0/3022573857 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f687c00f7c0 con 0x7f688c108f60 2024-04-17T07:26:52.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.216+0000 7f68943a2740 1 Processor -- start 2024-04-17T07:26:52.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.217+0000 7f68943a2740 1 -- start start 2024-04-17T07:26:52.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.217+0000 7f68943a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.217+0000 7f68943a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b64b80 0x5603f9b69680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.217+0000 7f68943a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b69cf0 0x5603f9b6c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.217+0000 7f68943a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5603f9acb120 con 0x5603f9b64b80 2024-04-17T07:26:52.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.217+0000 7f68943a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5603f9a71340 con 0x5603f9b69cf0 2024-04-17T07:26:52.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.217+0000 7f68943a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5603f9ac70e0 con 0x5603f9b64190 2024-04-17T07:26:52.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.218+0000 7f6883400640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68700a3c80 0x7f68700a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.218+0000 7f688b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.218+0000 7f688b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b645b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49672/0 (socket says 172.21.15.86:49672) 2024-04-17T07:26:52.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.218+0000 7f688b400640 1 -- 172.21.15.86:0/2441919294 learned_addr learned my addr 172.21.15.86:0/2441919294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:52.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.218+0000 7f688aa00640 1 --2- 172.21.15.86:0/2441919294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b64b80 0x5603f9b69680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.219+0000 7f688aa00640 1 -- 172.21.15.86:0/2441919294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 msgr2=0x5603f9b645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.219+0000 7f688aa00640 1 --2- 172.21.15.86:0/2441919294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.219+0000 7f688aa00640 1 -- 172.21.15.86:0/2441919294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b69cf0 msgr2=0x5603f9b6c180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:26:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.219+0000 7f688aa00640 1 --2- 172.21.15.86:0/2441919294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b69cf0 0x5603f9b6c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.219+0000 7f688aa00640 1 -- 172.21.15.86:0/2441919294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f98a77d0 con 0x5603f9b64b80 2024-04-17T07:26:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.219+0000 7f688b400640 1 --2- 172.21.15.86:0/2441919294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.219+0000 7f688b400640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68700a3c80 0x7f68700a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.220+0000 7f688aa00640 1 --2- 172.21.15.86:0/2441919294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b64b80 0x5603f9b69680 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f687c005fd0 tx=0x7f687c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3ecca6aae30c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:52.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.221+0000 7f6883400640 1 -- 172.21.15.86:0/3022573857 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f687c0104c0 con 0x7f688c108f60 2024-04-17T07:26:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.222+0000 7f6877400640 1 -- 172.21.15.86:0/2441919294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c09b5e0 con 0x5603f9b64b80 2024-04-17T07:26:52.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.222+0000 7f6877400640 1 -- 172.21.15.86:0/2441919294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f687c0df020 con 0x5603f9b64b80 2024-04-17T07:26:52.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.222+0000 7f6877400640 1 -- 172.21.15.86:0/2441919294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c09d6a0 con 0x5603f9b64b80 2024-04-17T07:26:52.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.222+0000 7f688b400640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68700a3c80 0x7f68700a6140 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x5603f9b653e0 tx=0x7f6884004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:52.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.223+0000 7f68943a2740 1 -- 172.21.15.86:0/2441919294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b64b80 msgr2=0x5603f9b69680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.223+0000 7f68943a2740 1 --2- 172.21.15.86:0/2441919294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b64b80 0x5603f9b69680 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f687c005fd0 tx=0x7f687c010000 comp rx=0 tx=0).stop 2024-04-17T07:26:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.224+0000 7f68943a2740 1 -- 172.21.15.86:0/2441919294 shutdown_connections 2024-04-17T07:26:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.224+0000 7f68943a2740 1 --2- 172.21.15.86:0/2441919294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b69cf0 0x5603f9b6c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.224+0000 7f68943a2740 1 --2- 172.21.15.86:0/2441919294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b64b80 0x5603f9b69680 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.224+0000 7f68943a2740 1 --2- 172.21.15.86:0/2441919294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.225+0000 7f68943a2740 1 -- 172.21.15.86:0/2441919294 >> 172.21.15.86:0/2441919294 conn(0x5603f9b5f700 msgr2=0x5603f9b5fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.225+0000 7f68943a2740 1 -- 172.21.15.86:0/2441919294 shutdown_connections 2024-04-17T07:26:52.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.226+0000 7f68943a2740 1 -- 172.21.15.86:0/2441919294 wait complete. 2024-04-17T07:26:52.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.226+0000 7f68943a2740 1 -- start start 2024-04-17T07:26:52.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.227+0000 7f68943a2740 1 Processor -- start 2024-04-17T07:26:52.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.228+0000 7f68943a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b6e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.228+0000 7f68943a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64b80 0x5603f9b6ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.228+0000 7f68943a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b69cf0 0x5603f9b75cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.228+0000 7f68943a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5603f9b6cca0 con 0x5603f9b69cf0 2024-04-17T07:26:52.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.228+0000 7f68943a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5603f9a84710 con 0x5603f9b64b80 2024-04-17T07:26:52.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.228+0000 7f68943a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5603f9b6ce20 con 0x5603f9b64190 2024-04-17T07:26:52.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.228+0000 7f68943a2740 10 client.0 _ll_register_callbacks cb 0x5603f9a7faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:52.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.228+0000 7f688aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b6e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.229+0000 7f688aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b6e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49682/0 (socket says 172.21.15.86:49682) 2024-04-17T07:26:52.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.229+0000 7f688aa00640 1 -- 172.21.15.86:0/1776941611 learned_addr learned my addr 172.21.15.86:0/1776941611 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:52.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.229+0000 7f688b400640 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b69cf0 0x5603f9b75cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.229+0000 7f688aa00640 1 -- 172.21.15.86:0/1776941611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64b80 msgr2=0x5603f9b6ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:52.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.229+0000 7f688a000640 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64b80 0x5603f9b6ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.230+0000 7f688aa00640 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64b80 0x5603f9b6ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.230+0000 7f688aa00640 1 -- 172.21.15.86:0/1776941611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b69cf0 msgr2=0x5603f9b75cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:52.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.230+0000 7f688aa00640 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b69cf0 0x5603f9b75cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.230+0000 7f688aa00640 1 -- 172.21.15.86:0/1776941611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f97dd3f0 con 0x5603f9b64190 2024-04-17T07:26:52.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.230+0000 7f688aa00640 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b6e740 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f687c0cea10 tx=0x7f687c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4a8a959476d2aff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:52.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.230+0000 7f686be00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:52.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.231+0000 7f686be00640 1 -- 172.21.15.86:0/1776941611 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c0cee60 con 0x5603f9b64190 2024-04-17T07:26:52.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.231+0000 7f686be00640 1 -- 172.21.15.86:0/1776941611 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f687c09d690 con 0x5603f9b64190 2024-04-17T07:26:52.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.231+0000 7f68943a2740 1 -- 172.21.15.86:0/1776941611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687c0df020 con 0x5603f9b64190 2024-04-17T07:26:52.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.232+0000 7f686be00640 1 -- 172.21.15.86:0/1776941611 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f687c020070 con 0x5603f9b64190 2024-04-17T07:26:52.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.232+0000 7f68943a2740 10 client.27487 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:52.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.232+0000 7f68943a2740 1 -- 172.21.15.86:0/1776941611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603f979d0f0 con 0x5603f9b64190 2024-04-17T07:26:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.232+0000 7f686be00640 1 -- 172.21.15.86:0/1776941611 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f687c0df020 con 0x5603f9b64190 2024-04-17T07:26:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.233+0000 7f686be00640 1 -- 172.21.15.86:0/1776941611 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f687c010920 con 0x5603f9b64190 2024-04-17T07:26:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.233+0000 7f68943a2740 10 client.27487 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.233+0000 7f68943a2740 10 client.27487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.233+0000 7f68943a2740 20 client.27487 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:52.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.233+0000 7f68943a2740 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5603f9b7a190 0x5603f9b7c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.233+0000 7f68943a2740 4 client.27487 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:26:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.233+0000 7f68943a2740 1 -- 172.21.15.86:0/1776941611 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603f9ad8b40 con 0x5603f9b7a190 2024-04-17T07:26:52.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.234+0000 7f688a000640 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5603f9b7a190 0x5603f9b7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.236+0000 7f688a000640 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5603f9b7a190 0x5603f9b7c580 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a63197416acb85e server_cookie=3cdf2190c6ada8a7 in_seq=0 out_seq=0 2024-04-17T07:26:52.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.236+0000 7f686be00640 10 client.27487 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.238+0000 7f686be00640 1 -- 172.21.15.86:0/1776941611 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603f9ad8b40 con 0x5603f9b7a190 2024-04-17T07:26:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.239+0000 7f686be00640 10 client.27487 handle_command_reply: tid=0 2024-04-17T07:26:52.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.239+0000 7f68943a2740 1 client.27487 shutdown 2024-04-17T07:26:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.239+0000 7f68943a2740 1 -- 172.21.15.86:0/1776941611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 msgr2=0x5603f9b6e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.239+0000 7f68943a2740 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b6e740 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f687c0cea10 tx=0x7f687c005bd0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.239+0000 7f68943a2740 1 -- 172.21.15.86:0/1776941611 shutdown_connections 2024-04-17T07:26:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.239+0000 7f68943a2740 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5603f9b7a190 0x5603f9b7c580 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.239+0000 7f68943a2740 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b69cf0 0x5603f9b75cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.239+0000 7f68943a2740 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64b80 0x5603f9b6ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.240+0000 7f68943a2740 1 --2- 172.21.15.86:0/1776941611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64190 0x5603f9b6e740 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.240+0000 7f68943a2740 1 -- 172.21.15.86:0/1776941611 >> 172.21.15.86:0/1776941611 conn(0x5603f9b5f700 msgr2=0x5603f9b5e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:52.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.240+0000 7f686be00640 0 client.27487 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:52.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.240+0000 7f68943a2740 1 -- 172.21.15.86:0/1776941611 shutdown_connections 2024-04-17T07:26:52.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.240+0000 7f68943a2740 1 -- 172.21.15.86:0/1776941611 wait complete. 2024-04-17T07:26:52.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.242+0000 7f68943a2740 20 client.27487 trim_cache size 0 max 16384 2024-04-17T07:26:52.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.290+0000 7f68943a2740 1 Processor -- start 2024-04-17T07:26:52.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.291+0000 7f68943a2740 1 -- start start 2024-04-17T07:26:52.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.291+0000 7f68943a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 0x5603f9b6e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.291+0000 7f68943a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64b80 0x5603f9b75c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.291+0000 7f68943a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b69cf0 0x5603f9b6e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.293+0000 7f68943a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f686c00fbe0 con 0x5603f9b69cf0 2024-04-17T07:26:52.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.293+0000 7f688a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64b80 0x5603f9b75c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.295+0000 7f68943a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f686c011720 con 0x5603f9b64190 2024-04-17T07:26:52.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.295+0000 7f68943a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f686c0102b0 con 0x5603f9b64b80 2024-04-17T07:26:52.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.295+0000 7f688a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64b80 0x5603f9b75c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49684/0 (socket says 172.21.15.86:49684) 2024-04-17T07:26:52.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.295+0000 7f688a000640 1 -- 172.21.15.86:0/2554985540 learned_addr learned my addr 172.21.15.86:0/2554985540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:52.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.293+0000 7f688b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b69cf0 0x5603f9b6e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.293+0000 7f688aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 0x5603f9b6e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.296+0000 7f688a000640 1 -- 172.21.15.86:0/2554985540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 msgr2=0x5603f9b6e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:52.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.296+0000 7f688a000640 1 --2- 172.21.15.86:0/2554985540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 0x5603f9b6e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.296+0000 7f688a000640 1 -- 172.21.15.86:0/2554985540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b69cf0 msgr2=0x5603f9b6e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:52.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.296+0000 7f688a000640 1 --2- 172.21.15.86:0/2554985540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b69cf0 0x5603f9b6e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.296+0000 7f688aa00640 1 --2- 172.21.15.86:0/2554985540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 0x5603f9b6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:26:52.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.296+0000 7f688a000640 1 -- 172.21.15.86:0/2554985540 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f9b60170 con 0x5603f9b64b80 2024-04-17T07:26:52.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.297+0000 7f688a000640 1 --2- 172.21.15.86:0/2554985540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64b80 0x5603f9b75c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6878009230 tx=0x7f6878005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a3c0c8faffa98b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:52.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.298+0000 7f6876a00640 1 -- 172.21.15.86:0/2554985540 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878000e60 con 0x5603f9b64b80 2024-04-17T07:26:52.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.298+0000 7f6876a00640 1 -- 172.21.15.86:0/2554985540 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6878004040 con 0x5603f9b64b80 2024-04-17T07:26:52.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.298+0000 7f6876a00640 1 -- 172.21.15.86:0/2554985540 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780055b0 con 0x5603f9b64b80 2024-04-17T07:26:52.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.299+0000 7f68943a2740 1 -- 172.21.15.86:0/2554985540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64b80 msgr2=0x5603f9b75c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:52.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.299+0000 7f68943a2740 1 --2- 172.21.15.86:0/2554985540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64b80 0x5603f9b75c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6878009230 tx=0x7f6878005d70 comp rx=0 tx=0).stop 2024-04-17T07:26:52.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.299+0000 7f68943a2740 1 -- 172.21.15.86:0/2554985540 shutdown_connections 2024-04-17T07:26:52.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.300+0000 7f68943a2740 1 --2- 172.21.15.86:0/2554985540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b69cf0 0x5603f9b6e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.300+0000 7f68943a2740 1 --2- 172.21.15.86:0/2554985540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b64b80 0x5603f9b75c80 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6878009230 tx=0x7f6878005d70 comp rx=0 tx=0).stop 2024-04-17T07:26:52.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.300+0000 7f68943a2740 1 --2- 172.21.15.86:0/2554985540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 0x5603f9b6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.300+0000 7f68943a2740 1 -- 172.21.15.86:0/2554985540 >> 172.21.15.86:0/2554985540 conn(0x5603f9b5f700 msgr2=0x5603f9b5a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.300+0000 7f68943a2740 1 -- 172.21.15.86:0/2554985540 shutdown_connections 2024-04-17T07:26:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.300+0000 7f68943a2740 1 -- 172.21.15.86:0/2554985540 wait complete. 2024-04-17T07:26:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.301+0000 7f68943a2740 1 -- start start 2024-04-17T07:26:52.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.301+0000 7f68943a2740 1 Processor -- start 2024-04-17T07:26:52.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.303+0000 7f68943a2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 0x5603f9b73150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.304+0000 7f68943a2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b69cf0 0x5603f9b73690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.304+0000 7f68943a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b746a0 0x5603f9b74b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.304+0000 7f68943a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f686c011050 con 0x5603f9b746a0 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.304+0000 7f68943a2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f686c010980 con 0x5603f9b64190 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.304+0000 7f68943a2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f686c00dbb0 con 0x5603f9b69cf0 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.304+0000 7f68943a2740 10 client.0 _ll_register_callbacks cb 0x7f687c009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.304+0000 7f688b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b746a0 0x5603f9b74b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.304+0000 7f688aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 0x5603f9b73150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.304+0000 7f688a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b69cf0 0x5603f9b73690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.305+0000 7f688a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b69cf0 0x5603f9b73690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49686/0 (socket says 172.21.15.86:49686) 2024-04-17T07:26:52.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.305+0000 7f688a000640 1 -- 172.21.15.86:0/1908114368 learned_addr learned my addr 172.21.15.86:0/1908114368 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:26:52.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.306+0000 7f688a000640 1 -- 172.21.15.86:0/1908114368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 msgr2=0x5603f9b73150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.306+0000 7f688a000640 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 0x5603f9b73150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.306+0000 7f688a000640 1 -- 172.21.15.86:0/1908114368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b746a0 msgr2=0x5603f9b74b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:26:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.306+0000 7f688a000640 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b746a0 0x5603f9b74b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.306+0000 7f688a000640 1 -- 172.21.15.86:0/1908114368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f9b75060 con 0x5603f9b69cf0 2024-04-17T07:26:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.306+0000 7f688b400640 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b746a0 0x5603f9b74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:26:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.306+0000 7f688aa00640 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 0x5603f9b73150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:26:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.306+0000 7f688a000640 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b69cf0 0x5603f9b73690 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6878009850 tx=0x7f6878004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b1cc2a98ded5ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:26:52.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.306+0000 7f686be00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:26:52.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.307+0000 7f686be00640 1 -- 172.21.15.86:0/1908114368 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6878000e60 con 0x5603f9b69cf0 2024-04-17T07:26:52.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.307+0000 7f686be00640 1 -- 172.21.15.86:0/1908114368 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f687802c8e0 con 0x5603f9b69cf0 2024-04-17T07:26:52.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.308+0000 7f68943a2740 1 -- 172.21.15.86:0/1908114368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6878004040 con 0x5603f9b69cf0 2024-04-17T07:26:52.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.308+0000 7f686be00640 1 -- 172.21.15.86:0/1908114368 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f687802cb20 con 0x5603f9b69cf0 2024-04-17T07:26:52.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.309+0000 7f68943a2740 10 client.27493 fetch_fsmap learned FSMap version 28 2024-04-17T07:26:52.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.309+0000 7f68943a2740 1 -- 172.21.15.86:0/1908114368 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603f9be0650 con 0x5603f9b69cf0 2024-04-17T07:26:52.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.309+0000 7f686be00640 1 -- 172.21.15.86:0/1908114368 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6878004040 con 0x5603f9b69cf0 2024-04-17T07:26:52.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.309+0000 7f686be00640 1 -- 172.21.15.86:0/1908114368 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6878048070 con 0x5603f9b69cf0 2024-04-17T07:26:52.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.310+0000 7f68943a2740 10 client.27493 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:26:52.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.310+0000 7f68943a2740 10 client.27493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:26:52.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.310+0000 7f68943a2740 20 client.27493 populate_metadata read hostname 'smithi086' 2024-04-17T07:26:52.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.310+0000 7f68943a2740 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5603f9be0f10 0x5603f9be3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:26:52.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.310+0000 7f68943a2740 4 client.27493 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:26:52.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.310+0000 7f68943a2740 1 -- 172.21.15.86:0/1908114368 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5603f9ad29b0 con 0x5603f9be0f10 2024-04-17T07:26:52.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.311+0000 7f688aa00640 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5603f9be0f10 0x5603f9be3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:26:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.313+0000 7f688aa00640 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5603f9be0f10 0x5603f9be3330 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c61344151da01a server_cookie=710e3f21786e2065 in_seq=0 out_seq=0 2024-04-17T07:26:52.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.313+0000 7f686be00640 10 client.27493 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.314+0000 7f686be00640 1 -- 172.21.15.86:0/1908114368 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5603f9ad29b0 con 0x5603f9be0f10 2024-04-17T07:26:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.314+0000 7f686be00640 10 client.27493 handle_command_reply: tid=0 2024-04-17T07:26:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.314+0000 7f68943a2740 1 client.27493 shutdown 2024-04-17T07:26:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.314+0000 7f68943a2740 1 -- 172.21.15.86:0/1908114368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b69cf0 msgr2=0x5603f9b73690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.314+0000 7f68943a2740 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b69cf0 0x5603f9b73690 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6878009850 tx=0x7f6878004420 comp rx=0 tx=0).stop 2024-04-17T07:26:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.315+0000 7f68943a2740 1 -- 172.21.15.86:0/1908114368 shutdown_connections 2024-04-17T07:26:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.315+0000 7f68943a2740 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5603f9be0f10 0x5603f9be3330 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.315+0000 7f68943a2740 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603f9b746a0 0x5603f9b74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.315+0000 7f68943a2740 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5603f9b69cf0 0x5603f9b73690 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.315+0000 7f68943a2740 1 --2- 172.21.15.86:0/1908114368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5603f9b64190 0x5603f9b73150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.315+0000 7f68943a2740 1 -- 172.21.15.86:0/1908114368 >> 172.21.15.86:0/1908114368 conn(0x5603f9b5f700 msgr2=0x5603f9b6e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:52.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.315+0000 7f686be00640 0 client.27493 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:26:52.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.315+0000 7f68943a2740 1 -- 172.21.15.86:0/1908114368 shutdown_connections 2024-04-17T07:26:52.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.315+0000 7f68943a2740 1 -- 172.21.15.86:0/1908114368 wait complete. 2024-04-17T07:26:52.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.315+0000 7f68943a2740 20 client.27493 trim_cache size 0 max 16384 2024-04-17T07:26:52.316 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:26:52.317 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:26:52.317 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:26:52.317 INFO:teuthology.orchestra.run.smithi086.stdout: "0e9db878-7d03-4be0-850e-b27feb28f077": { 2024-04-17T07:26:52.317 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:26:52.317 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "0e9db878-7d03-4be0-850e-b27feb28f077", 2024-04-17T07:26:52.317 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:26:52.317 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:26:52.317 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:26:52.317 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:26:52.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.316+0000 7f6891600640 1 -- 172.21.15.86:0/3022573857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68700a3c80 msgr2=0x7f68700a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:52.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.316+0000 7f6891600640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68700a3c80 0x7f68700a6140 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x5603f9b653e0 tx=0x7f6884004a40 comp rx=0 tx=0).stop 2024-04-17T07:26:52.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.316+0000 7f6891600640 1 -- 172.21.15.86:0/3022573857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c108f60 msgr2=0x7f688c1a7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:26:52.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.316+0000 7f6891600640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c108f60 0x7f688c1a7250 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f687c00d950 tx=0x7f687c00de10 comp rx=0 tx=0).stop 2024-04-17T07:26:52.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.316+0000 7f6891600640 1 -- 172.21.15.86:0/3022573857 shutdown_connections 2024-04-17T07:26:52.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.316+0000 7f6891600640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f68700a3c80 0x7f68700a6140 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.316+0000 7f6891600640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c1ac7f0 0x7f688c1aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.316+0000 7f6891600640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f688c10e2a0 0x7f688c1a7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.316+0000 7f6891600640 1 --2- 172.21.15.86:0/3022573857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c108f60 0x7f688c1a7250 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:26:52.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.316+0000 7f6891600640 1 -- 172.21.15.86:0/3022573857 >> 172.21.15.86:0/3022573857 conn(0x7f688c06edf0 msgr2=0x7f688c109380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:26:52.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.317+0000 7f6891600640 1 -- 172.21.15.86:0/3022573857 shutdown_connections 2024-04-17T07:26:52.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:26:52.317+0000 7f6891600640 1 -- 172.21.15.86:0/3022573857 wait complete. 2024-04-17T07:26:52.344 INFO:tasks.cephfs.filesystem:scrub status for tag:0e9db878-7d03-4be0-850e-b27feb28f077 - {'path': '/', 'tag': '0e9db878-7d03-4be0-850e-b27feb28f077', 'options': 'recursive,force'} 2024-04-17T07:26:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:53 smithi086 ceph-mon[31733]: pgmap v815: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 10 MiB/s wr, 724 op/s 2024-04-17T07:26:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:53 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:26:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:53 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:53 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:26:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:53 smithi098 ceph-mon[35418]: pgmap v815: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 10 MiB/s wr, 724 op/s 2024-04-17T07:26:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:53 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:26:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:53 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:53 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:26:53.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:53 smithi103 ceph-mon[35495]: pgmap v815: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 10 MiB/s wr, 724 op/s 2024-04-17T07:26:53.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:53 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:26:53.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:53 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:53.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:53 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:26:54.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:54 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 450.0 s, 665.4 tps, lat 156.881 ms stddev 79.551 2024-04-17T07:26:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:54 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:54 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:26:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:55 smithi098 ceph-mon[35418]: pgmap v816: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 727 op/s 2024-04-17T07:26:55.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:55 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:26:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:55 smithi103 ceph-mon[35495]: pgmap v816: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 727 op/s 2024-04-17T07:26:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:55 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:26:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:55 smithi086 ceph-mon[31733]: pgmap v816: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 727 op/s 2024-04-17T07:26:55.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:55 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:26:57.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:57 smithi098 ceph-mon[35418]: pgmap v817: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 629 op/s 2024-04-17T07:26:57.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:57 smithi103 ceph-mon[35495]: pgmap v817: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 629 op/s 2024-04-17T07:26:57.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:57 smithi086 ceph-mon[31733]: pgmap v817: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 629 op/s 2024-04-17T07:26:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 455.0 s, 669.0 tps, lat 148.972 ms stddev 215.324 2024-04-17T07:26:59.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:26:59 smithi098 ceph-mon[35418]: pgmap v818: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.4 MiB/s wr, 713 op/s 2024-04-17T07:26:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:26:59 smithi103 ceph-mon[35495]: pgmap v818: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.4 MiB/s wr, 713 op/s 2024-04-17T07:26:59.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:26:59 smithi086 ceph-mon[31733]: pgmap v818: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.4 MiB/s wr, 713 op/s 2024-04-17T07:27:01.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:01 smithi098 ceph-mon[35418]: pgmap v819: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.0 MiB/s wr, 661 op/s 2024-04-17T07:27:01.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:01 smithi103 ceph-mon[35495]: pgmap v819: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.0 MiB/s wr, 661 op/s 2024-04-17T07:27:01.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:01 smithi086 ceph-mon[31733]: pgmap v819: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 9.0 MiB/s wr, 661 op/s 2024-04-17T07:27:03.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:03 smithi098 ceph-mon[35418]: pgmap v820: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 697 op/s 2024-04-17T07:27:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:03 smithi103 ceph-mon[35495]: pgmap v820: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 697 op/s 2024-04-17T07:27:03.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:03 smithi086 ceph-mon[31733]: pgmap v820: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 10 MiB/s wr, 697 op/s 2024-04-17T07:27:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 460.0 s, 624.4 tps, lat 160.311 ms stddev 237.799 2024-04-17T07:27:05.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:05 smithi098 ceph-mon[35418]: pgmap v821: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 9.8 MiB/s wr, 685 op/s 2024-04-17T07:27:05.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:05 smithi103 ceph-mon[35495]: pgmap v821: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 9.8 MiB/s wr, 685 op/s 2024-04-17T07:27:05.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:05 smithi086 ceph-mon[31733]: pgmap v821: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 9.8 MiB/s wr, 685 op/s 2024-04-17T07:27:07.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:07 smithi086 ceph-mon[31733]: pgmap v822: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.4 MiB/s wr, 594 op/s 2024-04-17T07:27:07.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:07 smithi098 ceph-mon[35418]: pgmap v822: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.4 MiB/s wr, 594 op/s 2024-04-17T07:27:07.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:07 smithi103 ceph-mon[35495]: pgmap v822: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.4 MiB/s wr, 594 op/s 2024-04-17T07:27:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 465.0 s, 658.8 tps, lat 152.622 ms stddev 220.681 2024-04-17T07:27:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:09 smithi098 ceph-mon[35418]: pgmap v823: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 691 op/s 2024-04-17T07:27:09.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:09 smithi103 ceph-mon[35495]: pgmap v823: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 691 op/s 2024-04-17T07:27:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:09 smithi086 ceph-mon[31733]: pgmap v823: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 691 op/s 2024-04-17T07:27:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:11 smithi098 ceph-mon[35418]: pgmap v824: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 630 op/s 2024-04-17T07:27:11.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:11 smithi103 ceph-mon[35495]: pgmap v824: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 630 op/s 2024-04-17T07:27:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:11 smithi086 ceph-mon[31733]: pgmap v824: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 630 op/s 2024-04-17T07:27:13.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:13 smithi098 ceph-mon[35418]: pgmap v825: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 671 op/s 2024-04-17T07:27:13.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:13 smithi103 ceph-mon[35495]: pgmap v825: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 671 op/s 2024-04-17T07:27:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:13 smithi086 ceph-mon[31733]: pgmap v825: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 671 op/s 2024-04-17T07:27:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 470.0 s, 517.4 tps, lat 176.847 ms stddev 201.887 2024-04-17T07:27:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:15 smithi098 ceph-mon[35418]: pgmap v826: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 10 MiB/s wr, 652 op/s 2024-04-17T07:27:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:15 smithi103 ceph-mon[35495]: pgmap v826: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 10 MiB/s wr, 652 op/s 2024-04-17T07:27:15.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:15 smithi086 ceph-mon[31733]: pgmap v826: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 10 MiB/s wr, 652 op/s 2024-04-17T07:27:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:17 smithi098 ceph-mon[35418]: pgmap v827: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.8 MiB/s wr, 566 op/s 2024-04-17T07:27:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:17 smithi103 ceph-mon[35495]: pgmap v827: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.8 MiB/s wr, 566 op/s 2024-04-17T07:27:17.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:17 smithi086 ceph-mon[31733]: pgmap v827: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.8 MiB/s wr, 566 op/s 2024-04-17T07:27:19.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 475.0 s, 753.5 tps, lat 143.623 ms stddev 167.039 2024-04-17T07:27:19.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:19 smithi098 ceph-mon[35418]: pgmap v828: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 646 op/s 2024-04-17T07:27:19.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:19 smithi103 ceph-mon[35495]: pgmap v828: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 646 op/s 2024-04-17T07:27:19.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:19 smithi086 ceph-mon[31733]: pgmap v828: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 646 op/s 2024-04-17T07:27:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:21 smithi098 ceph-mon[35418]: pgmap v829: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.1 MiB/s wr, 580 op/s 2024-04-17T07:27:21.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:21 smithi103 ceph-mon[35495]: pgmap v829: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.1 MiB/s wr, 580 op/s 2024-04-17T07:27:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:21 smithi086 ceph-mon[31733]: pgmap v829: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 9.1 MiB/s wr, 580 op/s 2024-04-17T07:27:22.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:27:22.346 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:27:23.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.097+0000 7f28e3200640 1 -- 172.21.15.86:0/2730893142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28dc1053b0 msgr2=0x7f28dc105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.097+0000 7f28e3200640 1 --2- 172.21.15.86:0/2730893142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28dc1053b0 0x7f28dc105830 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009900 tx=0x7f28cc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:27:23.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.097+0000 7f28e3200640 1 -- 172.21.15.86:0/2730893142 shutdown_connections 2024-04-17T07:27:23.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.097+0000 7f28e3200640 1 --2- 172.21.15.86:0/2730893142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28dc105d70 0x7f28dc10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.097+0000 7f28e3200640 1 --2- 172.21.15.86:0/2730893142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28dc1053b0 0x7f28dc105830 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.097+0000 7f28e3200640 1 --2- 172.21.15.86:0/2730893142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28dc1041c0 0x7f28dc1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.097+0000 7f28e3200640 1 -- 172.21.15.86:0/2730893142 >> 172.21.15.86:0/2730893142 conn(0x7f28dc0ff710 msgr2=0x7f28dc101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:23.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.097+0000 7f28e3200640 1 -- 172.21.15.86:0/2730893142 shutdown_connections 2024-04-17T07:27:23.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.098+0000 7f28e3200640 1 -- 172.21.15.86:0/2730893142 wait complete. 2024-04-17T07:27:23.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.099+0000 7f28e3200640 1 Processor -- start 2024-04-17T07:27:23.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.099+0000 7f28e3200640 1 -- start start 2024-04-17T07:27:23.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.099+0000 7f28e3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28dc1041c0 0x7f28dc19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.099+0000 7f28e3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28dc1053b0 0x7f28dc19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.099+0000 7f28e3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28dc105d70 0x7f28dc1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.099+0000 7f28e3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc10fd40 con 0x7f28dc1041c0 2024-04-17T07:27:23.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.099+0000 7f28e3200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc10fbc0 con 0x7f28dc105d70 2024-04-17T07:27:23.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.099+0000 7f28e3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc10fec0 con 0x7f28dc1053b0 2024-04-17T07:27:23.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.099+0000 7f28dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28dc1041c0 0x7f28dc19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.099+0000 7f28dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28dc1041c0 0x7f28dc19a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56954/0 (socket says 172.21.15.86:56954) 2024-04-17T07:27:23.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.100+0000 7f28dbe00640 1 -- 172.21.15.86:0/2862331660 learned_addr learned my addr 172.21.15.86:0/2862331660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:23.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.100+0000 7f28db400640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28dc1053b0 0x7f28dc19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.100+0000 7f28e1200640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28dc105d70 0x7f28dc1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.100+0000 7f28dbe00640 1 -- 172.21.15.86:0/2862331660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28dc1053b0 msgr2=0x7f28dc19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.100+0000 7f28dbe00640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28dc1053b0 0x7f28dc19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.100+0000 7f28dbe00640 1 -- 172.21.15.86:0/2862331660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28dc105d70 msgr2=0x7f28dc1a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.100+0000 7f28dbe00640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28dc105d70 0x7f28dc1a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.100+0000 7f28dbe00640 1 -- 172.21.15.86:0/2862331660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc1a1fe0 con 0x7f28dc1041c0 2024-04-17T07:27:23.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.101+0000 7f28dbe00640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28dc1041c0 0x7f28dc19a3c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c970 tx=0x7f28c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:23.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.103+0000 7f28d8c00640 1 -- 172.21.15.86:0/2862331660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8007c10 con 0x7f28dc1041c0 2024-04-17T07:27:23.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.103+0000 7f28e3200640 1 -- 172.21.15.86:0/2862331660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28dc1a22d0 con 0x7f28dc1041c0 2024-04-17T07:27:23.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.103+0000 7f28e3200640 1 -- 172.21.15.86:0/2862331660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28dc1a28f0 con 0x7f28dc1041c0 2024-04-17T07:27:23.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.103+0000 7f28d8c00640 1 -- 172.21.15.86:0/2862331660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28c8007db0 con 0x7f28dc1041c0 2024-04-17T07:27:23.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.103+0000 7f28d8c00640 1 -- 172.21.15.86:0/2862331660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8007c10 con 0x7f28dc1041c0 2024-04-17T07:27:23.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.108+0000 7f28e5fc0740 1 Processor -- start 2024-04-17T07:27:23.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.109+0000 7f28e5fc0740 1 -- start start 2024-04-17T07:27:23.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.109+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 0x55bc6e84adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.109+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e84fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.109+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e852990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.109+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e7a9120 con 0x55bc6e850500 2024-04-17T07:27:23.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.109+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e74f340 con 0x55bc6e84a9a0 2024-04-17T07:27:23.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.109+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e7a50e0 con 0x55bc6e84b390 2024-04-17T07:27:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.109+0000 7f28d8c00640 1 -- 172.21.15.86:0/2862331660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f28c800e5d0 con 0x7f28dc1041c0 2024-04-17T07:27:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.110+0000 7f28db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e852990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.110+0000 7f28e1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 0x55bc6e84adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.110+0000 7f28e1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 0x55bc6e84adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:48742/0 (socket says 172.21.15.86:48742) 2024-04-17T07:27:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.110+0000 7f28e1200640 1 -- 172.21.15.86:0/2121378331 learned_addr learned my addr 172.21.15.86:0/2121378331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.110+0000 7f28dbe00640 1 --2- 172.21.15.86:0/2121378331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e84fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.110+0000 7f28e1200640 1 -- 172.21.15.86:0/2121378331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 msgr2=0x55bc6e84fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.112+0000 7f28e1200640 1 --2- 172.21.15.86:0/2121378331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e84fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.112+0000 7f28e1200640 1 -- 172.21.15.86:0/2121378331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 msgr2=0x55bc6e852990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.112+0000 7f28e1200640 1 --2- 172.21.15.86:0/2121378331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e852990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.112+0000 7f28e1200640 1 -- 172.21.15.86:0/2121378331 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6e5857d0 con 0x55bc6e84a9a0 2024-04-17T07:27:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.112+0000 7f28db400640 1 --2- 172.21.15.86:0/2121378331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e852990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:27:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.112+0000 7f28dbe00640 1 --2- 172.21.15.86:0/2121378331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e84fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:27:23.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.113+0000 7f28e1200640 1 --2- 172.21.15.86:0/2121378331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 0x55bc6e84adc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009f20 tx=0x7f28d400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2066c6217518cd12 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:23.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.114+0000 7f28d1600640 1 -- 172.21.15.86:0/2121378331 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d401a070 con 0x55bc6e84a9a0 2024-04-17T07:27:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.114+0000 7f28d8c00640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28c00a3e30 0x7f28c00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.114+0000 7f28d1600640 1 -- 172.21.15.86:0/2121378331 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28d40158c0 con 0x55bc6e84a9a0 2024-04-17T07:27:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.114+0000 7f28d1600640 1 -- 172.21.15.86:0/2121378331 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d400a820 con 0x55bc6e84a9a0 2024-04-17T07:27:23.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.115+0000 7f28d8c00640 1 -- 172.21.15.86:0/2862331660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f28c80d03c0 con 0x7f28dc1041c0 2024-04-17T07:27:23.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.116+0000 7f28e5fc0740 1 -- 172.21.15.86:0/2121378331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 msgr2=0x55bc6e84adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.116+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/2121378331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 0x55bc6e84adc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009f20 tx=0x7f28d400c8b0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.117+0000 7f28dbe00640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28c00a3e30 0x7f28c00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.117+0000 7f28e5fc0740 1 -- 172.21.15.86:0/2121378331 shutdown_connections 2024-04-17T07:27:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.117+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/2121378331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e852990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.117+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/2121378331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e84fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.117+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/2121378331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 0x55bc6e84adc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.117+0000 7f28e5fc0740 1 -- 172.21.15.86:0/2121378331 >> 172.21.15.86:0/2121378331 conn(0x55bc6e83d700 msgr2=0x55bc6e83db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:23.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.117+0000 7f28e5fc0740 1 -- 172.21.15.86:0/2121378331 shutdown_connections 2024-04-17T07:27:23.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.117+0000 7f28e5fc0740 1 -- 172.21.15.86:0/2121378331 wait complete. 2024-04-17T07:27:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.118+0000 7f28e5fc0740 1 -- start start 2024-04-17T07:27:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.119+0000 7f28e5fc0740 1 Processor -- start 2024-04-17T07:27:23.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.119+0000 7f28dbe00640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28c00a3e30 0x7f28c00a62f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f28c8098d20 tx=0x7f28c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84a9a0 0x55bc6e854f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84b390 0x55bc6e855450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e8534b0 con 0x55bc6e850500 2024-04-17T07:27:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e762710 con 0x55bc6e84b390 2024-04-17T07:27:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bc6e853630 con 0x55bc6e84a9a0 2024-04-17T07:27:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e5fc0740 10 client.0 _ll_register_callbacks cb 0x55bc6e75daf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56982/0 (socket says 172.21.15.86:56982) 2024-04-17T07:27:23.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e1200640 1 -- 172.21.15.86:0/1630337187 learned_addr learned my addr 172.21.15.86:0/1630337187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:23.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.120+0000 7f28e1200640 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84a9a0 0x55bc6e854f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.121+0000 7f28e1200640 1 -- 172.21.15.86:0/1630337187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84a9a0 msgr2=0x55bc6e854f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:23.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.121+0000 7f28e1200640 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84a9a0 0x55bc6e854f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.121+0000 7f28e1200640 1 -- 172.21.15.86:0/1630337187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84b390 msgr2=0x55bc6e855450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:23.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.121+0000 7f28e1200640 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84b390 0x55bc6e855450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.121+0000 7f28e1200640 1 -- 172.21.15.86:0/1630337187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6e4bb3f0 con 0x55bc6e850500 2024-04-17T07:27:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.122+0000 7f28e1200640 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85c490 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009fd0 tx=0x7f28d4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8eb516b1eb8e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.122+0000 7f28c6000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:27:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.122+0000 7f28c6000640 1 -- 172.21.15.86:0/1630337187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d401a070 con 0x55bc6e850500 2024-04-17T07:27:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.122+0000 7f28c6000640 1 -- 172.21.15.86:0/1630337187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f28d400b9f0 con 0x55bc6e850500 2024-04-17T07:27:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.122+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1630337187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d40158c0 con 0x55bc6e850500 2024-04-17T07:27:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.123+0000 7f28c6000640 1 -- 172.21.15.86:0/1630337187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28d400bbf0 con 0x55bc6e850500 2024-04-17T07:27:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.123+0000 7f28e5fc0740 10 client.18558 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:23.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.123+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1630337187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc6e47b0f0 con 0x55bc6e850500 2024-04-17T07:27:23.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.124+0000 7f28c6000640 1 -- 172.21.15.86:0/1630337187 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28d40158c0 con 0x55bc6e850500 2024-04-17T07:27:23.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.124+0000 7f28c6000640 1 -- 172.21.15.86:0/1630337187 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f28d4025cc0 con 0x55bc6e850500 2024-04-17T07:27:23.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.124+0000 7f28e5fc0740 10 client.18558 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:23.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.124+0000 7f28e5fc0740 10 client.18558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:27:23.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.124+0000 7f28e5fc0740 20 client.18558 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:23.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.124+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bc6e860950 0x55bc6e862d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.124+0000 7f28e5fc0740 4 client.18558 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:27:23.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.124+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1630337187 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc6e7b6b40 con 0x55bc6e860950 2024-04-17T07:27:23.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.125+0000 7f28db400640 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bc6e860950 0x55bc6e862d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.126+0000 7f28db400640 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bc6e860950 0x55bc6e862d40 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb92602abfb0295 server_cookie=84a7fa284d956a1 in_seq=0 out_seq=0 2024-04-17T07:27:23.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.126+0000 7f28c6000640 10 client.18558 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:23.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.128+0000 7f28c6000640 1 -- 172.21.15.86:0/1630337187 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc6e7b6b40 con 0x55bc6e860950 2024-04-17T07:27:23.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.128+0000 7f28c6000640 10 client.18558 handle_command_reply: tid=0 2024-04-17T07:27:23.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.128+0000 7f28e5fc0740 1 client.18558 shutdown 2024-04-17T07:27:23.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1630337187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 msgr2=0x55bc6e85c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85c490 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009fd0 tx=0x7f28d4009a50 comp rx=0 tx=0).stop 2024-04-17T07:27:23.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1630337187 shutdown_connections 2024-04-17T07:27:23.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bc6e860950 0x55bc6e862d40 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85c490 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84b390 0x55bc6e855450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1630337187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84a9a0 0x55bc6e854f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28c6000640 0 client.18558 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:23.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1630337187 >> 172.21.15.86:0/1630337187 conn(0x55bc6e83d700 msgr2=0x55bc6e83c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:23.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1630337187 shutdown_connections 2024-04-17T07:27:23.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.129+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1630337187 wait complete. 2024-04-17T07:27:23.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.130+0000 7f28e5fc0740 20 client.18558 trim_cache size 0 max 16384 2024-04-17T07:27:23.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.156+0000 7f28e5fc0740 1 Processor -- start 2024-04-17T07:27:23.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.156+0000 7f28e5fc0740 1 -- start start 2024-04-17T07:27:23.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.156+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e84a9a0 0x55bc6e854f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.156+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e855450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.156+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e850500 0x55bc6e85c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.156+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc00fb50 con 0x55bc6e84a9a0 2024-04-17T07:27:23.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.156+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc011690 con 0x55bc6e850500 2024-04-17T07:27:23.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.157+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc010220 con 0x55bc6e84b390 2024-04-17T07:27:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.157+0000 7f28e1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e84a9a0 0x55bc6e854f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.157+0000 7f28e1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e84a9a0 0x55bc6e854f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56990/0 (socket says 172.21.15.86:56990) 2024-04-17T07:27:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.157+0000 7f28e1200640 1 -- 172.21.15.86:0/2086739292 learned_addr learned my addr 172.21.15.86:0/2086739292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.157+0000 7f28db400640 1 --2- 172.21.15.86:0/2086739292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e855450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.157+0000 7f28e1200640 1 --2- 172.21.15.86:0/2086739292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e850500 0x55bc6e85c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.158+0000 7f28db400640 1 -- 172.21.15.86:0/2086739292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e850500 msgr2=0x55bc6e85c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.158+0000 7f28db400640 1 --2- 172.21.15.86:0/2086739292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e850500 0x55bc6e85c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.158+0000 7f28db400640 1 -- 172.21.15.86:0/2086739292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e84a9a0 msgr2=0x55bc6e854f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.158+0000 7f28db400640 1 --2- 172.21.15.86:0/2086739292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e84a9a0 0x55bc6e854f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.158+0000 7f28db400640 1 -- 172.21.15.86:0/2086739292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6e84fb20 con 0x55bc6e84b390 2024-04-17T07:27:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.158+0000 7f28e1200640 1 --2- 172.21.15.86:0/2086739292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e84a9a0 0x55bc6e854f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:27:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.159+0000 7f28db400640 1 --2- 172.21.15.86:0/2086739292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e855450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009230 tx=0x7f28cc02cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dafffe45cd20c35d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:23.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.159+0000 7f28d0c00640 1 -- 172.21.15.86:0/2086739292 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc004020 con 0x55bc6e84b390 2024-04-17T07:27:23.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.160+0000 7f28d0c00640 1 -- 172.21.15.86:0/2086739292 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28cc04a020 con 0x55bc6e84b390 2024-04-17T07:27:23.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.160+0000 7f28e5fc0740 1 -- 172.21.15.86:0/2086739292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 msgr2=0x55bc6e855450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.160+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/2086739292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e855450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009230 tx=0x7f28cc02cb90 comp rx=0 tx=0).stop 2024-04-17T07:27:23.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.162+0000 7f28e5fc0740 1 -- 172.21.15.86:0/2086739292 shutdown_connections 2024-04-17T07:27:23.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.162+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/2086739292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e850500 0x55bc6e85c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.162+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/2086739292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e855450 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.162+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/2086739292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e84a9a0 0x55bc6e854f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.162+0000 7f28e5fc0740 1 -- 172.21.15.86:0/2086739292 >> 172.21.15.86:0/2086739292 conn(0x55bc6e83d700 msgr2=0x55bc6e83dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:23.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.164+0000 7f28e5fc0740 1 -- 172.21.15.86:0/2086739292 shutdown_connections 2024-04-17T07:27:23.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.164+0000 7f28e5fc0740 1 -- 172.21.15.86:0/2086739292 wait complete. 2024-04-17T07:27:23.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.164+0000 7f28e5fc0740 1 -- start start 2024-04-17T07:27:23.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.164+0000 7f28e5fc0740 1 Processor -- start 2024-04-17T07:27:23.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.165+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 0x55bc6e857720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.165+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e857c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.165+0000 7f28e5fc0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.165+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc010fc0 con 0x55bc6e850500 2024-04-17T07:27:23.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.165+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc0108f0 con 0x55bc6e84a9a0 2024-04-17T07:27:23.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.165+0000 7f28e5fc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc00db20 con 0x55bc6e84b390 2024-04-17T07:27:23.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.165+0000 7f28e5fc0740 10 client.0 _ll_register_callbacks cb 0x7f28d4018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:23.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.166+0000 7f28db400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e857c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.166+0000 7f28e1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.166+0000 7f28db400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e857c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:32982/0 (socket says 172.21.15.86:32982) 2024-04-17T07:27:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.166+0000 7f28db400640 1 -- 172.21.15.86:0/1673873896 learned_addr learned my addr 172.21.15.86:0/1673873896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.166+0000 7f28e1200640 1 -- 172.21.15.86:0/1673873896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 msgr2=0x55bc6e857c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.166+0000 7f28e1200640 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e857c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.167+0000 7f28e1200640 1 -- 172.21.15.86:0/1673873896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 msgr2=0x55bc6e857720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:27:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.167+0000 7f28e1200640 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 0x55bc6e857720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.167+0000 7f28e1200640 1 -- 172.21.15.86:0/1673873896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6e85b270 con 0x55bc6e850500 2024-04-17T07:27:23.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.167+0000 7f28e1200640 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85ac80 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f28d40098f0 tx=0x7f28d4004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293f261cbcc24758 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:23.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.167+0000 7f28c6000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:27:23.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.169+0000 7f28c6000640 1 -- 172.21.15.86:0/1673873896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d401a030 con 0x55bc6e850500 2024-04-17T07:27:23.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.169+0000 7f28c6000640 1 -- 172.21.15.86:0/1673873896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f28d4024720 con 0x55bc6e850500 2024-04-17T07:27:23.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.169+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1673873896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28cc04a020 con 0x55bc6e850500 2024-04-17T07:27:23.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.170+0000 7f28c6000640 1 -- 172.21.15.86:0/1673873896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28d4024920 con 0x55bc6e850500 2024-04-17T07:27:23.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.170+0000 7f28e5fc0740 10 client.18564 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:23.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.170+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1673873896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc6e85c4f0 con 0x55bc6e850500 2024-04-17T07:27:23.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.171+0000 7f28c6000640 1 -- 172.21.15.86:0/1673873896 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28cc04a020 con 0x55bc6e850500 2024-04-17T07:27:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.172+0000 7f28c6000640 1 -- 172.21.15.86:0/1673873896 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f28d4043480 con 0x55bc6e850500 2024-04-17T07:27:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.173+0000 7f28e5fc0740 10 client.18564 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.173+0000 7f28e5fc0740 10 client.18564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:27:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.173+0000 7f28e5fc0740 20 client.18564 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.173+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bc6e8d82f0 0x55bc6e8da710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.173+0000 7f28e5fc0740 4 client.18564 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:27:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.173+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1673873896 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc6e7b09b0 con 0x55bc6e8d82f0 2024-04-17T07:27:23.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.174+0000 7f28db400640 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bc6e8d82f0 0x55bc6e8da710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.175+0000 7f28db400640 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bc6e8d82f0 0x55bc6e8da710 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3040428954a04d0a server_cookie=78b07a9631c0370c in_seq=0 out_seq=0 2024-04-17T07:27:23.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.175+0000 7f28c6000640 10 client.18564 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:23.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.176+0000 7f28c6000640 1 -- 172.21.15.86:0/1673873896 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bc6e7b09b0 con 0x55bc6e8d82f0 2024-04-17T07:27:23.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.176+0000 7f28c6000640 10 client.18564 handle_command_reply: tid=0 2024-04-17T07:27:23.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.176+0000 7f28e5fc0740 1 client.18564 shutdown 2024-04-17T07:27:23.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.178+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1673873896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 msgr2=0x55bc6e85ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.178+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85ac80 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f28d40098f0 tx=0x7f28d4004a20 comp rx=0 tx=0).stop 2024-04-17T07:27:23.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.178+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1673873896 shutdown_connections 2024-04-17T07:27:23.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.178+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55bc6e8d82f0 0x55bc6e8da710 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.179+0000 7f28c6000640 0 client.18564 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:23.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.179+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bc6e850500 0x55bc6e85ac80 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.179+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bc6e84b390 0x55bc6e857c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.179+0000 7f28e5fc0740 1 --2- 172.21.15.86:0/1673873896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bc6e84a9a0 0x55bc6e857720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.179+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1673873896 >> 172.21.15.86:0/1673873896 conn(0x55bc6e83d700 msgr2=0x55bc6e854ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:23.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.179+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1673873896 shutdown_connections 2024-04-17T07:27:23.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.179+0000 7f28e5fc0740 1 -- 172.21.15.86:0/1673873896 wait complete. 2024-04-17T07:27:23.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.179+0000 7f28e5fc0740 20 client.18564 trim_cache size 0 max 16384 2024-04-17T07:27:23.180 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:27:23.180 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:27:23.180 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:27:23.180 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:27:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.180+0000 7f28e3200640 1 -- 172.21.15.86:0/2862331660 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28c00a3e30 msgr2=0x7f28c00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.180+0000 7f28e3200640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28c00a3e30 0x7f28c00a62f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f28c8098d20 tx=0x7f28c8000bc0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.180+0000 7f28e3200640 1 -- 172.21.15.86:0/2862331660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28dc1041c0 msgr2=0x7f28dc19a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.180+0000 7f28e3200640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28dc1041c0 0x7f28dc19a3c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c970 tx=0x7f28c800ce30 comp rx=0 tx=0).stop 2024-04-17T07:27:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.180+0000 7f28e3200640 1 -- 172.21.15.86:0/2862331660 shutdown_connections 2024-04-17T07:27:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.180+0000 7f28e3200640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28c00a3e30 0x7f28c00a62f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.180+0000 7f28e3200640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28dc105d70 0x7f28dc1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.180+0000 7f28e3200640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28dc1053b0 0x7f28dc19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.180+0000 7f28e3200640 1 --2- 172.21.15.86:0/2862331660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28dc1041c0 0x7f28dc19a3c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.180+0000 7f28e3200640 1 -- 172.21.15.86:0/2862331660 >> 172.21.15.86:0/2862331660 conn(0x7f28dc0ff710 msgr2=0x7f28dc1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:23.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.182+0000 7f28e3200640 1 -- 172.21.15.86:0/2862331660 shutdown_connections 2024-04-17T07:27:23.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.182+0000 7f28e3200640 1 -- 172.21.15.86:0/2862331660 wait complete. 2024-04-17T07:27:23.199 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:27:23.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:27:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.446+0000 7f2241400640 1 -- 172.21.15.86:0/1021757847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f223c103980 msgr2=0x7f223c10a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.446+0000 7f2241400640 1 --2- 172.21.15.86:0/1021757847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f223c103980 0x7f223c10a210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2234009be0 tx=0x7f223402bef0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.446+0000 7f2241400640 1 -- 172.21.15.86:0/1021757847 shutdown_connections 2024-04-17T07:27:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.446+0000 7f2241400640 1 --2- 172.21.15.86:0/1021757847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f223c103980 0x7f223c10a210 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.446+0000 7f2241400640 1 --2- 172.21.15.86:0/1021757847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c102fc0 0x7f223c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.446+0000 7f2241400640 1 --2- 172.21.15.86:0/1021757847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f223c101dd0 0x7f223c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.446+0000 7f2241400640 1 -- 172.21.15.86:0/1021757847 >> 172.21.15.86:0/1021757847 conn(0x7f223c0fd340 msgr2=0x7f223c0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.447+0000 7f2241400640 1 -- 172.21.15.86:0/1021757847 shutdown_connections 2024-04-17T07:27:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.447+0000 7f2241400640 1 -- 172.21.15.86:0/1021757847 wait complete. 2024-04-17T07:27:23.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.450+0000 7f2241400640 1 Processor -- start 2024-04-17T07:27:23.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.451+0000 7f2241400640 1 -- start start 2024-04-17T07:27:23.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.451+0000 7f2241400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c101dd0 0x7f223c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.451+0000 7f2241400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f223c102fc0 0x7f223c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.451+0000 7f2241400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f223c103980 0x7f223c1a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.451+0000 7f2241400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f223c10d9b0 con 0x7f223c101dd0 2024-04-17T07:27:23.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.451+0000 7f2241400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f223c10d830 con 0x7f223c103980 2024-04-17T07:27:23.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.451+0000 7f2241400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f223c10db30 con 0x7f223c102fc0 2024-04-17T07:27:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.452+0000 7f223be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c101dd0 0x7f223c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.452+0000 7f223b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f223c102fc0 0x7f223c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.452+0000 7f223be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c101dd0 0x7f223c19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57028/0 (socket says 172.21.15.86:57028) 2024-04-17T07:27:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.452+0000 7f223be00640 1 -- 172.21.15.86:0/3600316092 learned_addr learned my addr 172.21.15.86:0/3600316092 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.452+0000 7f223b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f223c102fc0 0x7f223c19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33004/0 (socket says 172.21.15.86:33004) 2024-04-17T07:27:23.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.452+0000 7f2240a00640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f223c103980 0x7f223c1a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.454+0000 7f223b400640 1 -- 172.21.15.86:0/3600316092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f223c103980 msgr2=0x7f223c1a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.454+0000 7f223b400640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f223c103980 0x7f223c1a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.454+0000 7f223b400640 1 -- 172.21.15.86:0/3600316092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c101dd0 msgr2=0x7f223c19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.454+0000 7f223b400640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c101dd0 0x7f223c19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.454+0000 7f223b400640 1 -- 172.21.15.86:0/3600316092 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c1931c0 con 0x7f223c102fc0 2024-04-17T07:27:23.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.454+0000 7f223b400640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f223c102fc0 0x7f223c19ebe0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b3f0 tx=0x7f222c00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:23.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.455+0000 7f2238c00640 1 -- 172.21.15.86:0/3600316092 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c015020 con 0x7f223c102fc0 2024-04-17T07:27:23.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.455+0000 7f2238c00640 1 -- 172.21.15.86:0/3600316092 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f222c011ce0 con 0x7f223c102fc0 2024-04-17T07:27:23.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.455+0000 7f2238c00640 1 -- 172.21.15.86:0/3600316092 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c010d80 con 0x7f223c102fc0 2024-04-17T07:27:23.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.456+0000 7f2241400640 1 -- 172.21.15.86:0/3600316092 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c1934b0 con 0x7f223c102fc0 2024-04-17T07:27:23.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.457+0000 7f2241400640 1 -- 172.21.15.86:0/3600316092 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c193a00 con 0x7f223c102fc0 2024-04-17T07:27:23.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.459+0000 7f2241400640 1 -- 172.21.15.86:0/3600316092 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f223c193c60 con 0x7f223c102fc0 2024-04-17T07:27:23.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.462+0000 7f2238c00640 1 -- 172.21.15.86:0/3600316092 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f222c018020 con 0x7f223c102fc0 2024-04-17T07:27:23.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.465+0000 7f2238c00640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22200a3db0 0x7f22200a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:23.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.465+0000 7f2238c00640 1 -- 172.21.15.86:0/3600316092 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f222c010b20 con 0x7f223c102fc0 2024-04-17T07:27:23.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.465+0000 7f223be00640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22200a3db0 0x7f22200a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:23.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.467+0000 7f223be00640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22200a3db0 0x7f22200a6270 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2228009bf0 tx=0x7f2228009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:23.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.476+0000 7f2238c00640 1 -- 172.21.15.86:0/3600316092 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f222c018310 con 0x7f223c102fc0 2024-04-17T07:27:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:23 smithi098 ceph-mon[35418]: pgmap v830: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-04-17T07:27:23.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:23 smithi103 ceph-mon[35495]: pgmap v830: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-04-17T07:27:23.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:23 smithi086 ceph-mon[31733]: pgmap v830: 129 pgs: 129 active+clean; 1.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 660 op/s 2024-04-17T07:27:23.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.993+0000 7f2241400640 1 -- 172.21.15.86:0/3600316092 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f223c193ee0 con 0x7f223c102fc0 2024-04-17T07:27:23.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.995+0000 7f2238c00640 1 -- 172.21.15.86:0/3600316092 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f222c09cba0 con 0x7f223c102fc0 2024-04-17T07:27:23.996 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:27:23.996 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:27:23.996 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:27:23.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.998+0000 7f2241400640 1 -- 172.21.15.86:0/3600316092 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22200a3db0 msgr2=0x7f22200a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.998+0000 7f2241400640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22200a3db0 0x7f22200a6270 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2228009bf0 tx=0x7f2228009260 comp rx=0 tx=0).stop 2024-04-17T07:27:23.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.998+0000 7f2241400640 1 -- 172.21.15.86:0/3600316092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f223c102fc0 msgr2=0x7f223c19ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:23.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.998+0000 7f2241400640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f223c102fc0 0x7f223c19ebe0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b3f0 tx=0x7f222c00b8b0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.999+0000 7f2241400640 1 -- 172.21.15.86:0/3600316092 shutdown_connections 2024-04-17T07:27:23.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.999+0000 7f2241400640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f22200a3db0 0x7f22200a6270 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.999+0000 7f2241400640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f223c103980 0x7f223c1a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.999+0000 7f2241400640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f223c102fc0 0x7f223c19ebe0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.999+0000 7f2241400640 1 --2- 172.21.15.86:0/3600316092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f223c101dd0 0x7f223c19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:23.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:23.999+0000 7f2241400640 1 -- 172.21.15.86:0/3600316092 >> 172.21.15.86:0/3600316092 conn(0x7f223c0fd340 msgr2=0x7f223c074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.000+0000 7f2241400640 1 -- 172.21.15.86:0/3600316092 shutdown_connections 2024-04-17T07:27:24.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.000+0000 7f2241400640 1 -- 172.21.15.86:0/3600316092 wait complete. 2024-04-17T07:27:24.034 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:27:24.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:24 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3600316092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:27:24.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 480.0 s, 552.8 tps, lat 179.446 ms stddev 243.153 2024-04-17T07:27:24.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.373+0000 7f44cbe00640 1 -- 172.21.15.86:0/2632997453 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c002cbe0 con 0x7f44cc101fb0 2024-04-17T07:27:24.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.373+0000 7f44d4400640 1 -- 172.21.15.86:0/2632997453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44cc101fb0 msgr2=0x7f44cc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.373+0000 7f44d4400640 1 --2- 172.21.15.86:0/2632997453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44cc101fb0 0x7f44cc1023d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009900 tx=0x7f44c002bc30 comp rx=0 tx=0).stop 2024-04-17T07:27:24.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.374+0000 7f44d4400640 1 -- 172.21.15.86:0/2632997453 shutdown_connections 2024-04-17T07:27:24.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.374+0000 7f44d4400640 1 --2- 172.21.15.86:0/2632997453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc103b60 0x7f44cc10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.374+0000 7f44d4400640 1 --2- 172.21.15.86:0/2632997453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc1031a0 0x7f44cc103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.374+0000 7f44d4400640 1 --2- 172.21.15.86:0/2632997453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44cc101fb0 0x7f44cc1023d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.374+0000 7f44d4400640 1 -- 172.21.15.86:0/2632997453 >> 172.21.15.86:0/2632997453 conn(0x7f44cc0fd540 msgr2=0x7f44cc0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.374+0000 7f44d4400640 1 -- 172.21.15.86:0/2632997453 shutdown_connections 2024-04-17T07:27:24.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.374+0000 7f44d4400640 1 -- 172.21.15.86:0/2632997453 wait complete. 2024-04-17T07:27:24.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.376+0000 7f44d4400640 1 Processor -- start 2024-04-17T07:27:24.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.376+0000 7f44d4400640 1 -- start start 2024-04-17T07:27:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.377+0000 7f44d4400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44cc101fb0 0x7f44cc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.377+0000 7f44d4400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc1031a0 0x7f44cc19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.377+0000 7f44d4400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc103b60 0x7f44cc1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.377+0000 7f44d4400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10fd40 con 0x7f44cc1031a0 2024-04-17T07:27:24.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.377+0000 7f44d4400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10fbc0 con 0x7f44cc103b60 2024-04-17T07:27:24.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.377+0000 7f44d4400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc10fec0 con 0x7f44cc101fb0 2024-04-17T07:27:24.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.377+0000 7f44d1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc1031a0 0x7f44cc19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.378+0000 7f44d1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc1031a0 0x7f44cc19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57048/0 (socket says 172.21.15.86:57048) 2024-04-17T07:27:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.378+0000 7f44d1000640 1 -- 172.21.15.86:0/503138987 learned_addr learned my addr 172.21.15.86:0/503138987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.378+0000 7f44d1a00640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44cc101fb0 0x7f44cc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.378+0000 7f44d2400640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc103b60 0x7f44cc1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.378+0000 7f44d1000640 1 -- 172.21.15.86:0/503138987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44cc101fb0 msgr2=0x7f44cc19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.378+0000 7f44d1000640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44cc101fb0 0x7f44cc19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.378+0000 7f44d1000640 1 -- 172.21.15.86:0/503138987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc103b60 msgr2=0x7f44cc1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.378+0000 7f44d1000640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc103b60 0x7f44cc1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.378+0000 7f44d1000640 1 -- 172.21.15.86:0/503138987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44cc1a2040 con 0x7f44cc1031a0 2024-04-17T07:27:24.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.379+0000 7f44d1000640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc1031a0 0x7f44cc19a960 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f44b800d950 tx=0x7f44b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.380+0000 7f44ca000640 1 -- 172.21.15.86:0/503138987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8017070 con 0x7f44cc1031a0 2024-04-17T07:27:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.380+0000 7f44ca000640 1 -- 172.21.15.86:0/503138987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b8009c30 con 0x7f44cc1031a0 2024-04-17T07:27:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.380+0000 7f44ca000640 1 -- 172.21.15.86:0/503138987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8017070 con 0x7f44cc1031a0 2024-04-17T07:27:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.381+0000 7f44d4400640 1 -- 172.21.15.86:0/503138987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44cc1a2330 con 0x7f44cc1031a0 2024-04-17T07:27:24.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.381+0000 7f44d4400640 1 -- 172.21.15.86:0/503138987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44cc1a2830 con 0x7f44cc1031a0 2024-04-17T07:27:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.386+0000 7f44d7193740 1 Processor -- start 2024-04-17T07:27:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.386+0000 7f44ca000640 1 -- 172.21.15.86:0/503138987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f44b80109a0 con 0x7f44cc1031a0 2024-04-17T07:27:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.386+0000 7f44d7193740 1 -- start start 2024-04-17T07:27:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d7193740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9cfa0 0x5629efa9d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d7193740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 0x5629efaa2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d7193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efaa4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d7193740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629efa03020 con 0x5629efa9d990 2024-04-17T07:27:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d7193740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5629ef9a9460 con 0x5629efa9cfa0 2024-04-17T07:27:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d7193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5629ef9fefe0 con 0x5629efaa2b00 2024-04-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efaa4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efaa4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33050/0 (socket says 172.21.15.86:33050) 2024-04-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d1000640 1 -- 172.21.15.86:0/2021740186 learned_addr learned my addr 172.21.15.86:0/2021740186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d2400640 1 --2- 172.21.15.86:0/2021740186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9cfa0 0x5629efa9d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d1000640 1 -- 172.21.15.86:0/2021740186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9cfa0 msgr2=0x5629efa9d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d1000640 1 --2- 172.21.15.86:0/2021740186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9cfa0 0x5629efa9d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d1000640 1 -- 172.21.15.86:0/2021740186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 msgr2=0x5629efaa2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d1a00640 1 --2- 172.21.15.86:0/2021740186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 0x5629efaa2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d1000640 1 --2- 172.21.15.86:0/2021740186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 0x5629efaa2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44d1000640 1 -- 172.21.15.86:0/2021740186 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ef9b1040 con 0x5629efaa2b00 2024-04-17T07:27:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.388+0000 7f44ca000640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44b40a4020 0x7f44b40a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.389+0000 7f44d1a00640 1 --2- 172.21.15.86:0/2021740186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 0x5629efaa2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:27:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.389+0000 7f44d2400640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44b40a4020 0x7f44b40a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.389+0000 7f44ca000640 1 -- 172.21.15.86:0/503138987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f44b8009a30 con 0x7f44cc1031a0 2024-04-17T07:27:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.389+0000 7f44d1000640 1 --2- 172.21.15.86:0/2021740186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efaa4f90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f44b80d56a0 tx=0x7f44b80d5b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbe24276176dae43 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.390+0000 7f44bea00640 1 -- 172.21.15.86:0/2021740186 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8020070 con 0x5629efaa2b00 2024-04-17T07:27:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.390+0000 7f44bea00640 1 -- 172.21.15.86:0/2021740186 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b809be20 con 0x5629efaa2b00 2024-04-17T07:27:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.390+0000 7f44bea00640 1 -- 172.21.15.86:0/2021740186 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8020070 con 0x5629efaa2b00 2024-04-17T07:27:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.391+0000 7f44d7193740 1 -- 172.21.15.86:0/2021740186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 msgr2=0x5629efaa4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.391+0000 7f44d7193740 1 --2- 172.21.15.86:0/2021740186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efaa4f90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f44b80d56a0 tx=0x7f44b80d5b60 comp rx=0 tx=0).stop 2024-04-17T07:27:24.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.391+0000 7f44d2400640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44b40a4020 0x7f44b40a64e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f44c40099a0 tx=0x7f44c4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.392+0000 7f44d7193740 1 -- 172.21.15.86:0/2021740186 shutdown_connections 2024-04-17T07:27:24.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.392+0000 7f44d7193740 1 --2- 172.21.15.86:0/2021740186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efaa4f90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.392+0000 7f44d7193740 1 --2- 172.21.15.86:0/2021740186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 0x5629efaa2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.392+0000 7f44d7193740 1 --2- 172.21.15.86:0/2021740186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9cfa0 0x5629efa9d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.392+0000 7f44d7193740 1 -- 172.21.15.86:0/2021740186 >> 172.21.15.86:0/2021740186 conn(0x5629efa98510 msgr2=0x5629efa98910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.392+0000 7f44d7193740 1 -- 172.21.15.86:0/2021740186 shutdown_connections 2024-04-17T07:27:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.392+0000 7f44d7193740 1 -- 172.21.15.86:0/2021740186 wait complete. 2024-04-17T07:27:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.393+0000 7f44d7193740 1 -- start start 2024-04-17T07:27:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.393+0000 7f44d7193740 1 Processor -- start 2024-04-17T07:27:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.393+0000 7f44d7193740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efaafab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.393+0000 7f44d7193740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 0x5629efaafff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.394+0000 7f44d7193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efab7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.394+0000 7f44d7193740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629efaa5ab0 con 0x5629efa9cfa0 2024-04-17T07:27:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.394+0000 7f44d7193740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5629ef9bc830 con 0x5629efa9d990 2024-04-17T07:27:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.394+0000 7f44d7193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5629efaa5c30 con 0x5629efaa2b00 2024-04-17T07:27:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.394+0000 7f44d7193740 10 client.0 _ll_register_callbacks cb 0x5629ef98afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.394+0000 7f44d1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efaafab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.394+0000 7f44d1000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 0x5629efaafff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.394+0000 7f44d1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efaafab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57064/0 (socket says 172.21.15.86:57064) 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.394+0000 7f44d1a00640 1 -- 172.21.15.86:0/270509421 learned_addr learned my addr 172.21.15.86:0/270509421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.394+0000 7f44d2400640 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efab7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.395+0000 7f44d1a00640 1 -- 172.21.15.86:0/270509421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 msgr2=0x5629efab7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.395+0000 7f44d1a00640 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efab7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.395+0000 7f44d1a00640 1 -- 172.21.15.86:0/270509421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 msgr2=0x5629efaafff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.395+0000 7f44d1a00640 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 0x5629efaafff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.395+0000 7f44d1a00640 1 -- 172.21.15.86:0/270509421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ef9c0920 con 0x5629efa9cfa0 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.395+0000 7f44d1000640 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 0x5629efaafff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.395+0000 7f44d1a00640 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efaafab0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009900 tx=0x7f44c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bbedb650f77547 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.395+0000 7f44b3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:27:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.395+0000 7f44b3400640 1 -- 172.21.15.86:0/270509421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0005d50 con 0x5629efa9cfa0 2024-04-17T07:27:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.396+0000 7f44b3400640 1 -- 172.21.15.86:0/270509421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f44c0005230 con 0x5629efa9cfa0 2024-04-17T07:27:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.396+0000 7f44d7193740 1 -- 172.21.15.86:0/270509421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b809be20 con 0x5629efa9cfa0 2024-04-17T07:27:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.396+0000 7f44b3400640 1 -- 172.21.15.86:0/270509421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44c0005430 con 0x5629efa9cfa0 2024-04-17T07:27:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.397+0000 7f44d7193740 10 client.18576 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:24.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.397+0000 7f44d7193740 1 -- 172.21.15.86:0/270509421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629ef98e7f0 con 0x5629efa9cfa0 2024-04-17T07:27:24.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.397+0000 7f44b3400640 1 -- 172.21.15.86:0/270509421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b809be20 con 0x5629efa9cfa0 2024-04-17T07:27:24.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.398+0000 7f44b3400640 1 -- 172.21.15.86:0/270509421 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f44c00039d0 con 0x5629efa9cfa0 2024-04-17T07:27:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.398+0000 7f44d7193740 10 client.18576 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.398+0000 7f44d7193740 10 client.18576 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:27:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.398+0000 7f44d7193740 20 client.18576 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.398+0000 7f44d7193740 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5629efabb540 0x5629efabd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.398+0000 7f44d7193740 4 client.18576 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:27:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.398+0000 7f44d7193740 1 -- 172.21.15.86:0/270509421 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629efa10a40 con 0x5629efabb540 2024-04-17T07:27:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.400+0000 7f44d1000640 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5629efabb540 0x5629efabd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.401+0000 7f44b3400640 10 client.18576 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:27:24.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.401+0000 7f44d1000640 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5629efabb540 0x5629efabd9d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c6cce96452bce59 server_cookie=ab9efbb976c76d5a in_seq=0 out_seq=0 2024-04-17T07:27:24.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.401+0000 7f44b3400640 1 -- 172.21.15.86:0/270509421 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629efa10a40 con 0x5629efabb540 2024-04-17T07:27:24.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.401+0000 7f44b3400640 10 client.18576 handle_command_reply: tid=0 2024-04-17T07:27:24.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.402+0000 7f44d7193740 1 client.18576 shutdown 2024-04-17T07:27:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.403+0000 7f44d7193740 1 -- 172.21.15.86:0/270509421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 msgr2=0x5629efaafab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.403+0000 7f44d7193740 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efaafab0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009900 tx=0x7f44c0004270 comp rx=0 tx=0).stop 2024-04-17T07:27:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.403+0000 7f44d7193740 1 -- 172.21.15.86:0/270509421 shutdown_connections 2024-04-17T07:27:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.403+0000 7f44d7193740 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5629efabb540 0x5629efabd9d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.403+0000 7f44d7193740 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efab7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.403+0000 7f44d7193740 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 0x5629efaafff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.403+0000 7f44d7193740 1 --2- 172.21.15.86:0/270509421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efaafab0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.403+0000 7f44b3400640 0 client.18576 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:27:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.403+0000 7f44d7193740 1 -- 172.21.15.86:0/270509421 >> 172.21.15.86:0/270509421 conn(0x5629efa98510 msgr2=0x5629efa97710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.404+0000 7f44d7193740 1 -- 172.21.15.86:0/270509421 shutdown_connections 2024-04-17T07:27:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.404+0000 7f44d7193740 1 -- 172.21.15.86:0/270509421 wait complete. 2024-04-17T07:27:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.404+0000 7f44d7193740 20 client.18576 trim_cache size 0 max 16384 2024-04-17T07:27:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.442+0000 7f44d7193740 1 Processor -- start 2024-04-17T07:27:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.443+0000 7f44d7193740 1 -- start start 2024-04-17T07:27:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.443+0000 7f44d7193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efa98510 0x5629efaafd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.443+0000 7f44d7193740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efab6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.443+0000 7f44d7193740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 0x5629efa97590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.443+0000 7f44d7193740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc1a89b0 con 0x5629efa9cfa0 2024-04-17T07:27:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.443+0000 7f44d7193740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc1aa4f0 con 0x5629efa9d990 2024-04-17T07:27:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.443+0000 7f44d7193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc1a9080 con 0x5629efa98510 2024-04-17T07:27:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.444+0000 7f44d1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efab6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.444+0000 7f44d1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efab6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57078/0 (socket says 172.21.15.86:57078) 2024-04-17T07:27:24.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.444+0000 7f44d1000640 1 -- 172.21.15.86:0/908174469 learned_addr learned my addr 172.21.15.86:0/908174469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.444+0000 7f44d2400640 1 --2- 172.21.15.86:0/908174469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 0x5629efa97590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.444+0000 7f44d1000640 1 -- 172.21.15.86:0/908174469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efa98510 msgr2=0x5629efaafd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.444+0000 7f44d1000640 1 --2- 172.21.15.86:0/908174469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efa98510 0x5629efaafd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.444+0000 7f44d1000640 1 -- 172.21.15.86:0/908174469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 msgr2=0x5629efa97590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.444+0000 7f44d1000640 1 --2- 172.21.15.86:0/908174469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 0x5629efa97590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.444+0000 7f44d1000640 1 -- 172.21.15.86:0/908174469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ef7153f0 con 0x5629efa9cfa0 2024-04-17T07:27:24.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.445+0000 7f44d1000640 1 --2- 172.21.15.86:0/908174469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efab6ff0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f44b8097b80 tx=0x7f44b800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53abba4d1032851 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.445+0000 7f44be000640 1 -- 172.21.15.86:0/908174469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8020040 con 0x5629efa9cfa0 2024-04-17T07:27:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.445+0000 7f44be000640 1 -- 172.21.15.86:0/908174469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b809be20 con 0x5629efa9cfa0 2024-04-17T07:27:24.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.445+0000 7f44be000640 1 -- 172.21.15.86:0/908174469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8017030 con 0x5629efa9cfa0 2024-04-17T07:27:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.446+0000 7f44d7193740 1 -- 172.21.15.86:0/908174469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 msgr2=0x5629efab6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.446+0000 7f44d7193740 1 --2- 172.21.15.86:0/908174469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efab6ff0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f44b8097b80 tx=0x7f44b800bf60 comp rx=0 tx=0).stop 2024-04-17T07:27:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.447+0000 7f44d7193740 1 -- 172.21.15.86:0/908174469 shutdown_connections 2024-04-17T07:27:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.447+0000 7f44d7193740 1 --2- 172.21.15.86:0/908174469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa9d990 0x5629efa97590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.447+0000 7f44d7193740 1 --2- 172.21.15.86:0/908174469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9cfa0 0x5629efab6ff0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.447+0000 7f44d7193740 1 --2- 172.21.15.86:0/908174469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efa98510 0x5629efaafd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.447+0000 7f44d7193740 1 -- 172.21.15.86:0/908174469 >> 172.21.15.86:0/908174469 conn(0x5629efaa2640 msgr2=0x5629efa96e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.447+0000 7f44d7193740 1 -- 172.21.15.86:0/908174469 shutdown_connections 2024-04-17T07:27:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.447+0000 7f44d7193740 1 -- 172.21.15.86:0/908174469 wait complete. 2024-04-17T07:27:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.447+0000 7f44d7193740 1 -- start start 2024-04-17T07:27:24.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.448+0000 7f44d7193740 1 Processor -- start 2024-04-17T07:27:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d7193740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa98510 0x5629efab4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d7193740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 0x5629efab4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d7193740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efab5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d7193740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc1a9e20 con 0x5629efa9d990 2024-04-17T07:27:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d7193740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc1a9750 con 0x5629efa98510 2024-04-17T07:27:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d7193740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc1a6980 con 0x5629efaa2b00 2024-04-17T07:27:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d1a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa98510 0x5629efab4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 0x5629efab4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d7193740 10 client.0 _ll_register_callbacks cb 0x7f44c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d1a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa98510 0x5629efab4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:48856/0 (socket says 172.21.15.86:48856) 2024-04-17T07:27:24.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d1a00640 1 -- 172.21.15.86:0/1603821474 learned_addr learned my addr 172.21.15.86:0/1603821474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.449+0000 7f44d2400640 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efab5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.450+0000 7f44d1000640 1 -- 172.21.15.86:0/1603821474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 msgr2=0x5629efab5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.450+0000 7f44d1000640 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efab5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.450+0000 7f44d1000640 1 -- 172.21.15.86:0/1603821474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa98510 msgr2=0x5629efab4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.450+0000 7f44d1000640 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa98510 0x5629efab4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.450+0000 7f44d1000640 1 -- 172.21.15.86:0/1603821474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ef7df7d0 con 0x5629efa9d990 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.450+0000 7f44d1000640 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 0x5629efab4ed0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x5629efaa25f0 tx=0x7f44b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c36f8aae94e217 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.451+0000 7f44b3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.451+0000 7f44b3400640 1 -- 172.21.15.86:0/1603821474 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8017070 con 0x5629efa9d990 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.451+0000 7f44b3400640 1 -- 172.21.15.86:0/1603821474 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f44b8099a20 con 0x5629efa9d990 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.451+0000 7f44d7193740 1 -- 172.21.15.86:0/1603821474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b809be20 con 0x5629efa9d990 2024-04-17T07:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.451+0000 7f44b3400640 1 -- 172.21.15.86:0/1603821474 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f44b8020050 con 0x5629efa9d990 2024-04-17T07:27:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.452+0000 7f44d7193740 10 client.18582 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.452+0000 7f44d7193740 1 -- 172.21.15.86:0/1603821474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629efb21c00 con 0x5629efa9d990 2024-04-17T07:27:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.454+0000 7f44b3400640 1 -- 172.21.15.86:0/1603821474 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44b809be20 con 0x5629efa9d990 2024-04-17T07:27:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.455+0000 7f44b3400640 1 -- 172.21.15.86:0/1603821474 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f44b80b3080 con 0x5629efa9d990 2024-04-17T07:27:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.455+0000 7f44d7193740 10 client.18582 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:24.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.455+0000 7f44d7193740 10 client.18582 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:27:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.455+0000 7f44d7193740 20 client.18582 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.456+0000 7f44d7193740 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5629efb21ed0 0x5629efb242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.456+0000 7f44d7193740 4 client.18582 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:27:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.456+0000 7f44d7193740 1 -- 172.21.15.86:0/1603821474 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629efa0a8b0 con 0x5629efb21ed0 2024-04-17T07:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.457+0000 7f44d1a00640 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5629efb21ed0 0x5629efb242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.459+0000 7f44d1a00640 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5629efb21ed0 0x5629efb242c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76f3fc0a3c81fccc server_cookie=6d64eb5d439e5856 in_seq=0 out_seq=0 2024-04-17T07:27:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.459+0000 7f44b3400640 10 client.18582 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:27:24.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.460+0000 7f44b3400640 1 -- 172.21.15.86:0/1603821474 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629efa0a8b0 con 0x5629efb21ed0 2024-04-17T07:27:24.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.460+0000 7f44b3400640 10 client.18582 handle_command_reply: tid=0 2024-04-17T07:27:24.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.460+0000 7f44d7193740 1 client.18582 shutdown 2024-04-17T07:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.461+0000 7f44d7193740 1 -- 172.21.15.86:0/1603821474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 msgr2=0x5629efab4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.461+0000 7f44d7193740 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 0x5629efab4ed0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x5629efaa25f0 tx=0x7f44b8005880 comp rx=0 tx=0).stop 2024-04-17T07:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.461+0000 7f44d7193740 1 -- 172.21.15.86:0/1603821474 shutdown_connections 2024-04-17T07:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.461+0000 7f44d7193740 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5629efb21ed0 0x5629efb242c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.461+0000 7f44d7193740 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629efaa2b00 0x5629efab5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.461+0000 7f44d7193740 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629efa9d990 0x5629efab4ed0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.461+0000 7f44d7193740 1 --2- 172.21.15.86:0/1603821474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5629efa98510 0x5629efab4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.461+0000 7f44d7193740 1 -- 172.21.15.86:0/1603821474 >> 172.21.15.86:0/1603821474 conn(0x5629efaa2640 msgr2=0x5629efa97710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.461+0000 7f44b3400640 0 client.18582 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:27:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.461+0000 7f44d7193740 1 -- 172.21.15.86:0/1603821474 shutdown_connections 2024-04-17T07:27:24.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.462+0000 7f44d7193740 1 -- 172.21.15.86:0/1603821474 wait complete. 2024-04-17T07:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.463+0000 7f44d7193740 20 client.18582 trim_cache size 0 max 16384 2024-04-17T07:27:24.464 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:27:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.464+0000 7f44d4400640 1 -- 172.21.15.86:0/503138987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44b40a4020 msgr2=0x7f44b40a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.464+0000 7f44d4400640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44b40a4020 0x7f44b40a64e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f44c40099a0 tx=0x7f44c4009500 comp rx=0 tx=0).stop 2024-04-17T07:27:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.464+0000 7f44d4400640 1 -- 172.21.15.86:0/503138987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc1031a0 msgr2=0x7f44cc19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.464+0000 7f44d4400640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc1031a0 0x7f44cc19a960 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f44b800d950 tx=0x7f44b800de10 comp rx=0 tx=0).stop 2024-04-17T07:27:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.465+0000 7f44d4400640 1 -- 172.21.15.86:0/503138987 shutdown_connections 2024-04-17T07:27:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.465+0000 7f44d4400640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f44b40a4020 0x7f44b40a64e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.465+0000 7f44d4400640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44cc103b60 0x7f44cc1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.465+0000 7f44d4400640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc1031a0 0x7f44cc19a960 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.465+0000 7f44d4400640 1 --2- 172.21.15.86:0/503138987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44cc101fb0 0x7f44cc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.465+0000 7f44d4400640 1 -- 172.21.15.86:0/503138987 >> 172.21.15.86:0/503138987 conn(0x7f44cc0fd540 msgr2=0x7f44cc106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.465+0000 7f44d4400640 1 -- 172.21.15.86:0/503138987 shutdown_connections 2024-04-17T07:27:24.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.465+0000 7f44d4400640 1 -- 172.21.15.86:0/503138987 wait complete. 2024-04-17T07:27:24.497 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:27:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:24 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3600316092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:27:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:24 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3600316092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:27:24.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.814+0000 7fde25a00640 1 -- 172.21.15.86:0/632408783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2010c9b0 msgr2=0x7fde2010cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.814+0000 7fde25a00640 1 --2- 172.21.15.86:0/632408783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2010c9b0 0x7fde2010cdd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fde10009900 tx=0x7fde1002bc30 comp rx=0 tx=0).stop 2024-04-17T07:27:24.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.814+0000 7fde25a00640 1 -- 172.21.15.86:0/632408783 shutdown_connections 2024-04-17T07:27:24.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.814+0000 7fde25a00640 1 --2- 172.21.15.86:0/632408783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde2010e560 0x7fde20114df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.814+0000 7fde25a00640 1 --2- 172.21.15.86:0/632408783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde2010dba0 0x7fde2010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.814+0000 7fde25a00640 1 --2- 172.21.15.86:0/632408783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2010c9b0 0x7fde2010cdd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.815+0000 7fde25a00640 1 -- 172.21.15.86:0/632408783 >> 172.21.15.86:0/632408783 conn(0x7fde2006c990 msgr2=0x7fde2006cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.815+0000 7fde25a00640 1 -- 172.21.15.86:0/632408783 shutdown_connections 2024-04-17T07:27:24.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.815+0000 7fde25a00640 1 -- 172.21.15.86:0/632408783 wait complete. 2024-04-17T07:27:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.817+0000 7fde25a00640 1 Processor -- start 2024-04-17T07:27:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.817+0000 7fde25a00640 1 -- start start 2024-04-17T07:27:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.818+0000 7fde25a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2010c9b0 0x7fde201a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.818+0000 7fde25a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde2010dba0 0x7fde201a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.818+0000 7fde25a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde2010e560 0x7fde2007a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.818+0000 7fde25a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde20118500 con 0x7fde2010dba0 2024-04-17T07:27:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.818+0000 7fde25a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde20118380 con 0x7fde2010c9b0 2024-04-17T07:27:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.818+0000 7fde25a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde20118680 con 0x7fde2010e560 2024-04-17T07:27:24.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.819+0000 7fde1f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde2010e560 0x7fde2007a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.819+0000 7fde1ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2010c9b0 0x7fde201a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.819+0000 7fde1f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde2010e560 0x7fde2007a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33078/0 (socket says 172.21.15.86:33078) 2024-04-17T07:27:24.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.819+0000 7fde1f400640 1 -- 172.21.15.86:0/4184257215 learned_addr learned my addr 172.21.15.86:0/4184257215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.819+0000 7fde1f400640 1 -- 172.21.15.86:0/4184257215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2010c9b0 msgr2=0x7fde201a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.819+0000 7fde1f400640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2010c9b0 0x7fde201a2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.819+0000 7fde1f400640 1 -- 172.21.15.86:0/4184257215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde2010dba0 msgr2=0x7fde201a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.819+0000 7fde1f400640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde2010dba0 0x7fde201a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.819+0000 7fde1f400640 1 -- 172.21.15.86:0/4184257215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2007a860 con 0x7fde2010e560 2024-04-17T07:27:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.820+0000 7fde1f400640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde2010e560 0x7fde2007a1c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fde1800efd0 tx=0x7fde1800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.821+0000 7fde17400640 1 -- 172.21.15.86:0/4184257215 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18003b70 con 0x7fde2010e560 2024-04-17T07:27:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.821+0000 7fde17400640 1 -- 172.21.15.86:0/4184257215 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde18003d10 con 0x7fde2010e560 2024-04-17T07:27:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.821+0000 7fde17400640 1 -- 172.21.15.86:0/4184257215 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180108f0 con 0x7fde2010e560 2024-04-17T07:27:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.821+0000 7fde25a00640 1 -- 172.21.15.86:0/4184257215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2007ab50 con 0x7fde2010e560 2024-04-17T07:27:24.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.821+0000 7fde25a00640 1 -- 172.21.15.86:0/4184257215 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2007b020 con 0x7fde2010e560 2024-04-17T07:27:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.826+0000 7fde28822740 1 Processor -- start 2024-04-17T07:27:24.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.827+0000 7fde17400640 1 -- 172.21.15.86:0/4184257215 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fde18004020 con 0x7fde2010e560 2024-04-17T07:27:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.829+0000 7fde28822740 1 -- start start 2024-04-17T07:27:24.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.830+0000 7fde28822740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 0x55ce828acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.830+0000 7fde28822740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.830+0000 7fde28822740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.830+0000 7fde28822740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ce8280a020 con 0x55ce828ad1a0 2024-04-17T07:27:24.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.830+0000 7fde28822740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ce827b0460 con 0x55ce828b2310 2024-04-17T07:27:24.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.830+0000 7fde28822740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ce82805fe0 con 0x55ce828ac7b0 2024-04-17T07:27:24.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.831+0000 7fde1e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 0x55ce828acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.831+0000 7fde1e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 0x55ce828acbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33080/0 (socket says 172.21.15.86:33080) 2024-04-17T07:27:24.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.831+0000 7fde1e000640 1 -- 172.21.15.86:0/2191627378 learned_addr learned my addr 172.21.15.86:0/2191627378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:24.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.832+0000 7fde1e000640 1 --2- 172.21.15.86:0/2191627378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.832+0000 7fde1e000640 1 -- 172.21.15.86:0/2191627378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 msgr2=0x55ce828b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.832+0000 7fde1e000640 1 --2- 172.21.15.86:0/2191627378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.832+0000 7fde1e000640 1 -- 172.21.15.86:0/2191627378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 msgr2=0x55ce828b1ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:27:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.833+0000 7fde17400640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde040a3d80 0x7fde040a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.833+0000 7fde17400640 1 -- 172.21.15.86:0/4184257215 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fde180d0d90 con 0x7fde2010e560 2024-04-17T07:27:24.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.833+0000 7fde1e000640 1 --2- 172.21.15.86:0/2191627378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.833+0000 7fde1e000640 1 -- 172.21.15.86:0/2191627378 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce827b8040 con 0x55ce828ac7b0 2024-04-17T07:27:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.834+0000 7fde1e000640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde040a3d80 0x7fde040a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.835+0000 7fde1e000640 1 --2- 172.21.15.86:0/2191627378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 0x55ce828acbd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005fd0 tx=0x7fde0c00af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57b1b30d4826473 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.835+0000 7fde0be00640 1 -- 172.21.15.86:0/2191627378 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c0205f0 con 0x55ce828ac7b0 2024-04-17T07:27:24.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.836+0000 7fde1e000640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde040a3d80 0x7fde040a6240 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde0c012460 tx=0x7fde0c01c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.836+0000 7fde0be00640 1 -- 172.21.15.86:0/2191627378 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde0c020c40 con 0x55ce828ac7b0 2024-04-17T07:27:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.836+0000 7fde0be00640 1 -- 172.21.15.86:0/2191627378 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c010c80 con 0x55ce828ac7b0 2024-04-17T07:27:24.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.839+0000 7fde28822740 1 -- 172.21.15.86:0/2191627378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 msgr2=0x55ce828acbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.839+0000 7fde28822740 1 --2- 172.21.15.86:0/2191627378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 0x55ce828acbd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005fd0 tx=0x7fde0c00af90 comp rx=0 tx=0).stop 2024-04-17T07:27:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.839+0000 7fde28822740 1 -- 172.21.15.86:0/2191627378 shutdown_connections 2024-04-17T07:27:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.839+0000 7fde28822740 1 --2- 172.21.15.86:0/2191627378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.839+0000 7fde28822740 1 --2- 172.21.15.86:0/2191627378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.839+0000 7fde28822740 1 --2- 172.21.15.86:0/2191627378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 0x55ce828acbd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.839+0000 7fde28822740 1 -- 172.21.15.86:0/2191627378 >> 172.21.15.86:0/2191627378 conn(0x55ce8289f510 msgr2=0x55ce8289f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.839+0000 7fde28822740 1 -- 172.21.15.86:0/2191627378 shutdown_connections 2024-04-17T07:27:24.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.839+0000 7fde28822740 1 -- 172.21.15.86:0/2191627378 wait complete. 2024-04-17T07:27:24.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.840+0000 7fde28822740 1 -- start start 2024-04-17T07:27:24.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.840+0000 7fde28822740 1 Processor -- start 2024-04-17T07:27:24.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.842+0000 7fde28822740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828ac7b0 0x55ce828b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.842+0000 7fde28822740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.842+0000 7fde28822740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828b2310 0x55ce828be220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.842+0000 7fde28822740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ce828b52c0 con 0x55ce828ad1a0 2024-04-17T07:27:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.842+0000 7fde28822740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ce827c3830 con 0x55ce828ac7b0 2024-04-17T07:27:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.842+0000 7fde28822740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ce828b5440 con 0x55ce828b2310 2024-04-17T07:27:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.842+0000 7fde28822740 10 client.0 _ll_register_callbacks cb 0x55ce82791fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.842+0000 7fde1ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828ac7b0 0x55ce828b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.842+0000 7fde1ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828ac7b0 0x55ce828b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:48906/0 (socket says 172.21.15.86:48906) 2024-04-17T07:27:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.842+0000 7fde1ea00640 1 -- 172.21.15.86:0/1661051404 learned_addr learned my addr 172.21.15.86:0/1661051404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.843+0000 7fde1e000640 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.843+0000 7fde1f400640 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828b2310 0x55ce828be220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.843+0000 7fde1ea00640 1 -- 172.21.15.86:0/1661051404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828b2310 msgr2=0x55ce828be220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.843+0000 7fde1ea00640 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828b2310 0x55ce828be220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.843+0000 7fde1ea00640 1 -- 172.21.15.86:0/1661051404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 msgr2=0x55ce828b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.843+0000 7fde1ea00640 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828b71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.843+0000 7fde1ea00640 1 -- 172.21.15.86:0/1661051404 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce827c7920 con 0x55ce828ac7b0 2024-04-17T07:27:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.844+0000 7fde1ea00640 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828ac7b0 0x55ce828b6ca0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fde10000c00 tx=0x7fde10004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faea2f6e5dfb14f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.844+0000 7fde08c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:27:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.844+0000 7fde08c00640 1 -- 172.21.15.86:0/1661051404 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10045020 con 0x55ce828ac7b0 2024-04-17T07:27:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.845+0000 7fde08c00640 1 -- 172.21.15.86:0/1661051404 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fde10005040 con 0x55ce828ac7b0 2024-04-17T07:27:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.845+0000 7fde28822740 1 -- 172.21.15.86:0/1661051404 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde0c020c40 con 0x55ce828ac7b0 2024-04-17T07:27:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.845+0000 7fde08c00640 1 -- 172.21.15.86:0/1661051404 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde10050050 con 0x55ce828ac7b0 2024-04-17T07:27:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.846+0000 7fde28822740 10 client.37454 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.846+0000 7fde28822740 1 -- 172.21.15.86:0/1661051404 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ce827957f0 con 0x55ce828ac7b0 2024-04-17T07:27:24.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.847+0000 7fde08c00640 1 -- 172.21.15.86:0/1661051404 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde0c020c40 con 0x55ce828ac7b0 2024-04-17T07:27:24.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.847+0000 7fde08c00640 1 -- 172.21.15.86:0/1661051404 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fde10004ca0 con 0x55ce828ac7b0 2024-04-17T07:27:24.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.847+0000 7fde28822740 10 client.37454 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:24.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.847+0000 7fde28822740 10 client.37454 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:27:24.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.847+0000 7fde28822740 20 client.37454 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:24.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.847+0000 7fde28822740 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ce828c2760 0x55ce828c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.847+0000 7fde28822740 4 client.37454 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:27:24.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.847+0000 7fde28822740 1 -- 172.21.15.86:0/1661051404 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce82817a40 con 0x55ce828c2760 2024-04-17T07:27:24.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.848+0000 7fde1e000640 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ce828c2760 0x55ce828c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.850+0000 7fde1e000640 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ce828c2760 0x55ce828c4c00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22cec0945c500dc8 server_cookie=f6f9deee41524bea in_seq=0 out_seq=0 2024-04-17T07:27:24.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.850+0000 7fde08c00640 10 client.37454 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:27:24.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.852+0000 7fde08c00640 1 -- 172.21.15.86:0/1661051404 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce82817a40 con 0x55ce828c2760 2024-04-17T07:27:24.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.852+0000 7fde08c00640 10 client.37454 handle_command_reply: tid=0 2024-04-17T07:27:24.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.852+0000 7fde28822740 1 client.37454 shutdown 2024-04-17T07:27:24.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.853+0000 7fde28822740 1 -- 172.21.15.86:0/1661051404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828ac7b0 msgr2=0x55ce828b6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.853+0000 7fde28822740 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828ac7b0 0x55ce828b6ca0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fde10000c00 tx=0x7fde10004250 comp rx=0 tx=0).stop 2024-04-17T07:27:24.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.853+0000 7fde28822740 1 -- 172.21.15.86:0/1661051404 shutdown_connections 2024-04-17T07:27:24.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.853+0000 7fde28822740 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ce828c2760 0x55ce828c4c00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.853+0000 7fde28822740 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828b2310 0x55ce828be220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.853+0000 7fde28822740 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.853+0000 7fde28822740 1 --2- 172.21.15.86:0/1661051404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828ac7b0 0x55ce828b6ca0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.853+0000 7fde28822740 1 -- 172.21.15.86:0/1661051404 >> 172.21.15.86:0/1661051404 conn(0x55ce8289f510 msgr2=0x55ce8289e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.853+0000 7fde08c00640 0 client.37454 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:27:24.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.856+0000 7fde28822740 1 -- 172.21.15.86:0/1661051404 shutdown_connections 2024-04-17T07:27:24.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.856+0000 7fde28822740 1 -- 172.21.15.86:0/1661051404 wait complete. 2024-04-17T07:27:24.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.856+0000 7fde28822740 20 client.37454 trim_cache size 0 max 16384 2024-04-17T07:27:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.885+0000 7fde28822740 1 Processor -- start 2024-04-17T07:27:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.886+0000 7fde28822740 1 -- start start 2024-04-17T07:27:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.887+0000 7fde28822740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ac7b0 0x55ce828b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.887+0000 7fde28822740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ad1a0 0x55ce828be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.887+0000 7fde28822740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.888+0000 7fde1e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ad1a0 0x55ce828be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.888+0000 7fde28822740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde200810f0 con 0x55ce828ac7b0 2024-04-17T07:27:24.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.888+0000 7fde1e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ad1a0 0x55ce828be1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33090/0 (socket says 172.21.15.86:33090) 2024-04-17T07:27:24.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.888+0000 7fde1e000640 1 -- 172.21.15.86:0/2174746059 learned_addr learned my addr 172.21.15.86:0/2174746059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:24.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.888+0000 7fde28822740 1 -- 172.21.15.86:0/2174746059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde20082c30 con 0x55ce828b2310 2024-04-17T07:27:24.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.888+0000 7fde28822740 1 -- 172.21.15.86:0/2174746059 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde200817c0 con 0x55ce828ad1a0 2024-04-17T07:27:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.888+0000 7fde1f400640 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.888+0000 7fde1ea00640 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ac7b0 0x55ce828b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.890+0000 7fde1f400640 1 -- 172.21.15.86:0/2174746059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ad1a0 msgr2=0x55ce828be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.890+0000 7fde1f400640 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ad1a0 0x55ce828be1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.890+0000 7fde1f400640 1 -- 172.21.15.86:0/2174746059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ac7b0 msgr2=0x55ce828b62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.890+0000 7fde1f400640 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ac7b0 0x55ce828b62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.890+0000 7fde1f400640 1 -- 172.21.15.86:0/2174746059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce8251c3f0 con 0x55ce828b2310 2024-04-17T07:27:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.890+0000 7fde1ea00640 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ac7b0 0x55ce828b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:27:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.890+0000 7fde1e000640 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ad1a0 0x55ce828be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:27:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.891+0000 7fde1f400640 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828b6800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fde180ce480 tx=0x7fde18004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d117bdaf970966d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.891+0000 7fde0b400640 1 -- 172.21.15.86:0/2174746059 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180b5020 con 0x55ce828b2310 2024-04-17T07:27:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.891+0000 7fde0b400640 1 -- 172.21.15.86:0/2174746059 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde180bbe50 con 0x55ce828b2310 2024-04-17T07:27:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.891+0000 7fde0b400640 1 -- 172.21.15.86:0/2174746059 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180a5900 con 0x55ce828b2310 2024-04-17T07:27:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.892+0000 7fde28822740 1 -- 172.21.15.86:0/2174746059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 msgr2=0x55ce828b6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.892+0000 7fde28822740 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828b6800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fde180ce480 tx=0x7fde18004a40 comp rx=0 tx=0).stop 2024-04-17T07:27:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.893+0000 7fde28822740 1 -- 172.21.15.86:0/2174746059 shutdown_connections 2024-04-17T07:27:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.893+0000 7fde28822740 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828b6800 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.893+0000 7fde28822740 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ad1a0 0x55ce828be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.893+0000 7fde28822740 1 --2- 172.21.15.86:0/2174746059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ac7b0 0x55ce828b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.893+0000 7fde28822740 1 -- 172.21.15.86:0/2174746059 >> 172.21.15.86:0/2174746059 conn(0x55ce8289f510 msgr2=0x55ce828a3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.893+0000 7fde28822740 1 -- 172.21.15.86:0/2174746059 shutdown_connections 2024-04-17T07:27:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.893+0000 7fde28822740 1 -- 172.21.15.86:0/2174746059 wait complete. 2024-04-17T07:27:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.894+0000 7fde28822740 1 -- start start 2024-04-17T07:27:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.894+0000 7fde28822740 1 Processor -- start 2024-04-17T07:27:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde28822740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 0x55ce828bb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde28822740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828bbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde28822740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde28822740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde20082560 con 0x55ce828ad1a0 2024-04-17T07:27:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde28822740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde20081e90 con 0x55ce828b2310 2024-04-17T07:27:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde28822740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde2007f0e0 con 0x55ce828ac7b0 2024-04-17T07:27:24.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde28822740 10 client.0 _ll_register_callbacks cb 0x7fde1002cd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde1e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828bbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde1e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828bbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57140/0 (socket says 172.21.15.86:57140) 2024-04-17T07:27:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde1e000640 1 -- 172.21.15.86:0/722108076 learned_addr learned my addr 172.21.15.86:0/722108076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.895+0000 7fde1f400640 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.896+0000 7fde1f400640 1 -- 172.21.15.86:0/722108076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 msgr2=0x55ce828bb7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:27:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.896+0000 7fde1f400640 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 0x55ce828bb7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.896+0000 7fde1f400640 1 -- 172.21.15.86:0/722108076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 msgr2=0x55ce828bbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.896+0000 7fde1f400640 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828bbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.896+0000 7fde1f400640 1 -- 172.21.15.86:0/722108076 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce825e67d0 con 0x55ce828b2310 2024-04-17T07:27:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.896+0000 7fde1f400640 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828bcce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fde1809c990 tx=0x7fde180988a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2360e5becfca2b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.896+0000 7fde08c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:27:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.897+0000 7fde08c00640 1 -- 172.21.15.86:0/722108076 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180b5070 con 0x55ce828b2310 2024-04-17T07:27:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.897+0000 7fde08c00640 1 -- 172.21.15.86:0/722108076 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fde180ce5c0 con 0x55ce828b2310 2024-04-17T07:27:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.897+0000 7fde28822740 1 -- 172.21.15.86:0/722108076 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde180bbe50 con 0x55ce828b2310 2024-04-17T07:27:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.897+0000 7fde08c00640 1 -- 172.21.15.86:0/722108076 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde18004560 con 0x55ce828b2310 2024-04-17T07:27:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.897+0000 7fde28822740 10 client.37460 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.897+0000 7fde28822740 1 -- 172.21.15.86:0/722108076 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ce828be400 con 0x55ce828b2310 2024-04-17T07:27:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.898+0000 7fde08c00640 1 -- 172.21.15.86:0/722108076 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde180bbe50 con 0x55ce828b2310 2024-04-17T07:27:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.898+0000 7fde08c00640 1 -- 172.21.15.86:0/722108076 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fde1807bb80 con 0x55ce828b2310 2024-04-17T07:27:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.898+0000 7fde28822740 10 client.37460 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:24.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.898+0000 7fde28822740 10 client.37460 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:27:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.898+0000 7fde28822740 20 client.37460 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.898+0000 7fde28822740 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ce829292e0 0x55ce8292b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.898+0000 7fde28822740 4 client.37460 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:27:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.898+0000 7fde28822740 1 -- 172.21.15.86:0/722108076 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce828118b0 con 0x55ce829292e0 2024-04-17T07:27:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.899+0000 7fde1ea00640 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ce829292e0 0x55ce8292b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.900+0000 7fde1ea00640 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ce829292e0 0x55ce8292b700 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45207b881a9b9ad1 server_cookie=96a3bac6d3ea3896 in_seq=0 out_seq=0 2024-04-17T07:27:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.901+0000 7fde08c00640 10 client.37460 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:27:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.901+0000 7fde08c00640 1 -- 172.21.15.86:0/722108076 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce828118b0 con 0x55ce829292e0 2024-04-17T07:27:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.901+0000 7fde08c00640 10 client.37460 handle_command_reply: tid=0 2024-04-17T07:27:24.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.901+0000 7fde28822740 1 client.37460 shutdown 2024-04-17T07:27:24.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.902+0000 7fde28822740 1 -- 172.21.15.86:0/722108076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 msgr2=0x55ce828bcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.902+0000 7fde28822740 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828bcce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fde1809c990 tx=0x7fde180988a0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.902+0000 7fde28822740 1 -- 172.21.15.86:0/722108076 shutdown_connections 2024-04-17T07:27:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.902+0000 7fde28822740 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ce829292e0 0x55ce8292b700 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.903+0000 7fde28822740 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ce828b2310 0x55ce828bcce0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.903+0000 7fde28822740 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ce828ad1a0 0x55ce828bbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.903+0000 7fde08c00640 0 client.37460 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:27:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.903+0000 7fde28822740 1 --2- 172.21.15.86:0/722108076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ce828ac7b0 0x55ce828bb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.903+0000 7fde28822740 1 -- 172.21.15.86:0/722108076 >> 172.21.15.86:0/722108076 conn(0x55ce8289f510 msgr2=0x55ce8289de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.903+0000 7fde28822740 1 -- 172.21.15.86:0/722108076 shutdown_connections 2024-04-17T07:27:24.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.903+0000 7fde28822740 1 -- 172.21.15.86:0/722108076 wait complete. 2024-04-17T07:27:24.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.903+0000 7fde28822740 20 client.37460 trim_cache size 0 max 16384 2024-04-17T07:27:24.904 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:27:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.906+0000 7fde25a00640 1 -- 172.21.15.86:0/4184257215 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde040a3d80 msgr2=0x7fde040a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.906+0000 7fde25a00640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde040a3d80 0x7fde040a6240 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde0c012460 tx=0x7fde0c01c000 comp rx=0 tx=0).stop 2024-04-17T07:27:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.906+0000 7fde25a00640 1 -- 172.21.15.86:0/4184257215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde2010e560 msgr2=0x7fde2007a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.906+0000 7fde25a00640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde2010e560 0x7fde2007a1c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fde1800efd0 tx=0x7fde1800c6a0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.907+0000 7fde25a00640 1 -- 172.21.15.86:0/4184257215 shutdown_connections 2024-04-17T07:27:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.907+0000 7fde25a00640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fde040a3d80 0x7fde040a6240 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.907+0000 7fde25a00640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde2010e560 0x7fde2007a1c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.907+0000 7fde25a00640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde2010dba0 0x7fde201a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.907+0000 7fde25a00640 1 --2- 172.21.15.86:0/4184257215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2010c9b0 0x7fde201a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:24.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.907+0000 7fde25a00640 1 -- 172.21.15.86:0/4184257215 >> 172.21.15.86:0/4184257215 conn(0x7fde2006c990 msgr2=0x7fde20110fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:24.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.907+0000 7fde25a00640 1 -- 172.21.15.86:0/4184257215 shutdown_connections 2024-04-17T07:27:24.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:24.907+0000 7fde25a00640 1 -- 172.21.15.86:0/4184257215 wait complete. 2024-04-17T07:27:24.938 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:27:25.208 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:25 smithi086 ceph-mon[31733]: pgmap v831: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 664 op/s 2024-04-17T07:27:25.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.234+0000 7fc395800640 1 -- 172.21.15.86:0/2832925790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390105b90 msgr2=0x7fc39010c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.234+0000 7fc395800640 1 --2- 172.21.15.86:0/2832925790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390105b90 0x7fc39010c420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc378009860 tx=0x7fc37802bb70 comp rx=0 tx=0).stop 2024-04-17T07:27:25.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.234+0000 7fc395800640 1 -- 172.21.15.86:0/2832925790 shutdown_connections 2024-04-17T07:27:25.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.234+0000 7fc395800640 1 --2- 172.21.15.86:0/2832925790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390105b90 0x7fc39010c420 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.234+0000 7fc395800640 1 --2- 172.21.15.86:0/2832925790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3901051d0 0x7fc390105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.234+0000 7fc395800640 1 --2- 172.21.15.86:0/2832925790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc390103fe0 0x7fc390104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.234+0000 7fc395800640 1 -- 172.21.15.86:0/2832925790 >> 172.21.15.86:0/2832925790 conn(0x7fc3900ff570 msgr2=0x7fc390101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.234+0000 7fc395800640 1 -- 172.21.15.86:0/2832925790 shutdown_connections 2024-04-17T07:27:25.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.234+0000 7fc395800640 1 -- 172.21.15.86:0/2832925790 wait complete. 2024-04-17T07:27:25.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.236+0000 7fc395800640 1 Processor -- start 2024-04-17T07:27:25.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.236+0000 7fc395800640 1 -- start start 2024-04-17T07:27:25.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.236+0000 7fc395800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc390103fe0 0x7fc39019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.236+0000 7fc395800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3901051d0 0x7fc39019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc390103fe0 0x7fc39019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc395800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390105b90 0x7fc3901a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc395800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc39010fb80 con 0x7fc3901051d0 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc395800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc39010fa00 con 0x7fc390103fe0 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc395800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc39010fd00 con 0x7fc390105b90 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc390103fe0 0x7fc39019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:48944/0 (socket says 172.21.15.86:48944) 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc38fe00640 1 -- 172.21.15.86:0/2235000211 learned_addr learned my addr 172.21.15.86:0/2235000211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc38fe00640 1 -- 172.21.15.86:0/2235000211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390105b90 msgr2=0x7fc3901a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc394e00640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390105b90 0x7fc3901a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc38fe00640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390105b90 0x7fc3901a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc38fe00640 1 -- 172.21.15.86:0/2235000211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3901051d0 msgr2=0x7fc39019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc38fe00640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3901051d0 0x7fc39019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.237+0000 7fc38fe00640 1 -- 172.21.15.86:0/2235000211 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3901a6210 con 0x7fc390103fe0 2024-04-17T07:27:25.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.238+0000 7fc394e00640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390105b90 0x7fc3901a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:27:25.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.238+0000 7fc38fe00640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc390103fe0 0x7fc39019e5f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c3a0 tx=0x7fc38800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.239+0000 7fc38cc00640 1 -- 172.21.15.86:0/2235000211 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388003990 con 0x7fc390103fe0 2024-04-17T07:27:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.241+0000 7fc38cc00640 1 -- 172.21.15.86:0/2235000211 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc388004020 con 0x7fc390103fe0 2024-04-17T07:27:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.241+0000 7fc38cc00640 1 -- 172.21.15.86:0/2235000211 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38800b8d0 con 0x7fc390103fe0 2024-04-17T07:27:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.241+0000 7fc395800640 1 -- 172.21.15.86:0/2235000211 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3901a6500 con 0x7fc390103fe0 2024-04-17T07:27:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.242+0000 7fc395800640 1 -- 172.21.15.86:0/2235000211 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3901a6a80 con 0x7fc390103fe0 2024-04-17T07:27:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.246+0000 7fc399c10740 1 Processor -- start 2024-04-17T07:27:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.247+0000 7fc399c10740 1 -- start start 2024-04-17T07:27:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.247+0000 7fc399c10740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95d91a0 0x557de95d95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.247+0000 7fc399c10740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d9b90 0x557de95de690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.247+0000 7fc399c10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95e1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.247+0000 7fc399c10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557de953f020 con 0x557de95ded00 2024-04-17T07:27:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.247+0000 7fc399c10740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557de94e5460 con 0x557de95d9b90 2024-04-17T07:27:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.247+0000 7fc399c10740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557de953afe0 con 0x557de95d91a0 2024-04-17T07:27:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.248+0000 7fc394e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95d91a0 0x557de95d95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.248+0000 7fc394e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95d91a0 0x557de95d95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33130/0 (socket says 172.21.15.86:33130) 2024-04-17T07:27:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.248+0000 7fc394e00640 1 -- 172.21.15.86:0/2559591073 learned_addr learned my addr 172.21.15.86:0/2559591073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.248+0000 7fc38f400640 1 --2- 172.21.15.86:0/2559591073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.248+0000 7fc38cc00640 1 -- 172.21.15.86:0/2235000211 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc388003b30 con 0x7fc390103fe0 2024-04-17T07:27:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.248+0000 7fc394e00640 1 -- 172.21.15.86:0/2559591073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d9b90 msgr2=0x557de95de690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:27:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.248+0000 7fc394e00640 1 --2- 172.21.15.86:0/2559591073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d9b90 0x557de95de690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.248+0000 7fc394e00640 1 -- 172.21.15.86:0/2559591073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 msgr2=0x557de95e1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.248+0000 7fc394e00640 1 --2- 172.21.15.86:0/2559591073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95e1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.249+0000 7fc394e00640 1 -- 172.21.15.86:0/2559591073 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de94ed040 con 0x557de95d91a0 2024-04-17T07:27:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.249+0000 7fc38f400640 1 --2- 172.21.15.86:0/2559591073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95e1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:27:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.251+0000 7fc38cc00640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc3740a3c30 0x7fc3740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.251+0000 7fc38cc00640 1 -- 172.21.15.86:0/2235000211 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc3880d0a80 con 0x7fc390103fe0 2024-04-17T07:27:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.251+0000 7fc394e00640 1 --2- 172.21.15.86:0/2559591073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95d91a0 0x557de95d95c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc378009780 tx=0x7fc378005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc4017ab76a70c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.252+0000 7fc385600640 1 -- 172.21.15.86:0/2559591073 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378043070 con 0x557de95d91a0 2024-04-17T07:27:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.252+0000 7fc385600640 1 -- 172.21.15.86:0/2559591073 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc37803a040 con 0x557de95d91a0 2024-04-17T07:27:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.252+0000 7fc385600640 1 -- 172.21.15.86:0/2559591073 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378039030 con 0x557de95d91a0 2024-04-17T07:27:25.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.253+0000 7fc38fe00640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc3740a3c30 0x7fc3740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.255+0000 7fc399c10740 1 -- 172.21.15.86:0/2559591073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95d91a0 msgr2=0x557de95d95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.255+0000 7fc399c10740 1 --2- 172.21.15.86:0/2559591073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95d91a0 0x557de95d95c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc378009780 tx=0x7fc378005d80 comp rx=0 tx=0).stop 2024-04-17T07:27:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.255+0000 7fc399c10740 1 -- 172.21.15.86:0/2559591073 shutdown_connections 2024-04-17T07:27:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.255+0000 7fc399c10740 1 --2- 172.21.15.86:0/2559591073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95e1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.255+0000 7fc399c10740 1 --2- 172.21.15.86:0/2559591073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d9b90 0x557de95de690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.255+0000 7fc399c10740 1 --2- 172.21.15.86:0/2559591073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95d91a0 0x557de95d95c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.255+0000 7fc399c10740 1 -- 172.21.15.86:0/2559591073 >> 172.21.15.86:0/2559591073 conn(0x557de95d4710 msgr2=0x557de95d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.255+0000 7fc38fe00640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc3740a3c30 0x7fc3740a60f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc38809c680 tx=0x7fc388004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.256+0000 7fc399c10740 1 -- 172.21.15.86:0/2559591073 shutdown_connections 2024-04-17T07:27:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.256+0000 7fc399c10740 1 -- 172.21.15.86:0/2559591073 wait complete. 2024-04-17T07:27:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.257+0000 7fc399c10740 1 -- start start 2024-04-17T07:27:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.257+0000 7fc399c10740 1 Processor -- start 2024-04-17T07:27:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.258+0000 7fc399c10740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.258+0000 7fc399c10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d9b90 0x557de95e3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.258+0000 7fc399c10740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 0x557de95eac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.259+0000 7fc394e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.259+0000 7fc399c10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557de95e1c50 con 0x557de95d9b90 2024-04-17T07:27:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.259+0000 7fc399c10740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557de94f8830 con 0x557de95d91a0 2024-04-17T07:27:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.259+0000 7fc399c10740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557de95e1dd0 con 0x557de95ded00 2024-04-17T07:27:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.259+0000 7fc399c10740 10 client.0 _ll_register_callbacks cb 0x557de94c6fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.259+0000 7fc38f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d9b90 0x557de95e3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.259+0000 7fc394e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:48960/0 (socket says 172.21.15.86:48960) 2024-04-17T07:27:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.259+0000 7fc394e00640 1 -- 172.21.15.86:0/1864153002 learned_addr learned my addr 172.21.15.86:0/1864153002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.260+0000 7fc394e00640 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 0x557de95eac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.260+0000 7fc38f400640 1 -- 172.21.15.86:0/1864153002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 msgr2=0x557de95eac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.260+0000 7fc38f400640 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 0x557de95eac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.260+0000 7fc38f400640 1 -- 172.21.15.86:0/1864153002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 msgr2=0x557de95e36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.260+0000 7fc38f400640 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.260+0000 7fc38f400640 1 -- 172.21.15.86:0/1864153002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de94fc920 con 0x557de95d9b90 2024-04-17T07:27:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.261+0000 7fc38f400640 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d9b90 0x557de95e3bf0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fc39019fb00 tx=0x7fc38000d530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5231a5132cdac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.261+0000 7fc37e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:27:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.262+0000 7fc37e000640 1 -- 172.21.15.86:0/1864153002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380013480 con 0x557de95d9b90 2024-04-17T07:27:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.262+0000 7fc37e000640 1 -- 172.21.15.86:0/1864153002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc380013620 con 0x557de95d9b90 2024-04-17T07:27:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.262+0000 7fc399c10740 1 -- 172.21.15.86:0/1864153002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc37803a040 con 0x557de95d9b90 2024-04-17T07:27:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.263+0000 7fc37e000640 1 -- 172.21.15.86:0/1864153002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc380021050 con 0x557de95d9b90 2024-04-17T07:27:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.263+0000 7fc399c10740 10 client.18600 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.263+0000 7fc399c10740 1 -- 172.21.15.86:0/1864153002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557de94ca7f0 con 0x557de95d9b90 2024-04-17T07:27:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.264+0000 7fc37e000640 1 -- 172.21.15.86:0/1864153002 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc37803a040 con 0x557de95d9b90 2024-04-17T07:27:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.264+0000 7fc37e000640 1 -- 172.21.15.86:0/1864153002 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc380012760 con 0x557de95d9b90 2024-04-17T07:27:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.265+0000 7fc399c10740 10 client.18600 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.265+0000 7fc399c10740 10 client.18600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:27:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.265+0000 7fc399c10740 20 client.18600 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.265+0000 7fc399c10740 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x557de95ef1c0 0x557de95f1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.265+0000 7fc399c10740 4 client.18600 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:27:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.265+0000 7fc399c10740 1 -- 172.21.15.86:0/1864153002 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557de954ca40 con 0x557de95ef1c0 2024-04-17T07:27:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.266+0000 7fc394e00640 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x557de95ef1c0 0x557de95f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.267+0000 7fc394e00640 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x557de95ef1c0 0x557de95f1620 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=329b9f87c39f6c86 server_cookie=38f12607dc2bd405 in_seq=0 out_seq=0 2024-04-17T07:27:25.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.267+0000 7fc37e000640 10 client.18600 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.268+0000 7fc37e000640 1 -- 172.21.15.86:0/1864153002 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557de954ca40 con 0x557de95ef1c0 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.268+0000 7fc37e000640 10 client.18600 handle_command_reply: tid=0 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.268+0000 7fc399c10740 1 client.18600 shutdown 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.269+0000 7fc399c10740 1 -- 172.21.15.86:0/1864153002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d9b90 msgr2=0x557de95e3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.269+0000 7fc399c10740 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d9b90 0x557de95e3bf0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fc39019fb00 tx=0x7fc38000d530 comp rx=0 tx=0).stop 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.269+0000 7fc399c10740 1 -- 172.21.15.86:0/1864153002 shutdown_connections 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.269+0000 7fc399c10740 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x557de95ef1c0 0x557de95f1620 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.269+0000 7fc37e000640 0 client.18600 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.269+0000 7fc399c10740 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 0x557de95eac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.269+0000 7fc399c10740 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d9b90 0x557de95e3bf0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.269+0000 7fc399c10740 1 --2- 172.21.15.86:0/1864153002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.269+0000 7fc399c10740 1 -- 172.21.15.86:0/1864153002 >> 172.21.15.86:0/1864153002 conn(0x557de95d4710 msgr2=0x557de95d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.269+0000 7fc399c10740 1 -- 172.21.15.86:0/1864153002 shutdown_connections 2024-04-17T07:27:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.270+0000 7fc399c10740 1 -- 172.21.15.86:0/1864153002 wait complete. 2024-04-17T07:27:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.271+0000 7fc399c10740 20 client.18600 trim_cache size 0 max 16384 2024-04-17T07:27:25.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.302+0000 7fc399c10740 1 Processor -- start 2024-04-17T07:27:25.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.302+0000 7fc399c10740 1 -- start start 2024-04-17T07:27:25.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc399c10740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc399c10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc399c10740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ef1c0 0x557de95e26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc399c10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b10b0 con 0x557de95ded00 2024-04-17T07:27:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc399c10740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b2bf0 con 0x557de95d91a0 2024-04-17T07:27:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc399c10740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b1780 con 0x557de95ef1c0 2024-04-17T07:27:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc38f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc394e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc394e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:48972/0 (socket says 172.21.15.86:48972) 2024-04-17T07:27:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc38f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95eabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57194/0 (socket says 172.21.15.86:57194) 2024-04-17T07:27:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.304+0000 7fc394e00640 1 -- 172.21.15.86:0/2926952758 learned_addr learned my addr 172.21.15.86:0/2926952758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:25.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.305+0000 7fc394e00640 1 -- 172.21.15.86:0/2926952758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ef1c0 msgr2=0x557de95e26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.305+0000 7fc394e00640 1 --2- 172.21.15.86:0/2926952758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ef1c0 0x557de95e26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.305+0000 7fc394e00640 1 -- 172.21.15.86:0/2926952758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 msgr2=0x557de95eabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.305+0000 7fc394e00640 1 --2- 172.21.15.86:0/2926952758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95eabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.305+0000 7fc394e00640 1 -- 172.21.15.86:0/2926952758 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de92513f0 con 0x557de95d91a0 2024-04-17T07:27:25.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.305+0000 7fc38f400640 1 --2- 172.21.15.86:0/2926952758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:27:25.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.306+0000 7fc394e00640 1 --2- 172.21.15.86:0/2926952758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e2170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc378004580 tx=0x7fc378005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cec38789d30586d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.307+0000 7fc384c00640 1 -- 172.21.15.86:0/2926952758 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378039040 con 0x557de95d91a0 2024-04-17T07:27:25.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.307+0000 7fc384c00640 1 -- 172.21.15.86:0/2926952758 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc378050070 con 0x557de95d91a0 2024-04-17T07:27:25.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.307+0000 7fc384c00640 1 -- 172.21.15.86:0/2926952758 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378039040 con 0x557de95d91a0 2024-04-17T07:27:25.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.308+0000 7fc399c10740 1 -- 172.21.15.86:0/2926952758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 msgr2=0x557de95e2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.308+0000 7fc399c10740 1 --2- 172.21.15.86:0/2926952758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e2170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc378004580 tx=0x7fc378005620 comp rx=0 tx=0).stop 2024-04-17T07:27:25.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.308+0000 7fc399c10740 1 -- 172.21.15.86:0/2926952758 shutdown_connections 2024-04-17T07:27:25.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.308+0000 7fc399c10740 1 --2- 172.21.15.86:0/2926952758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ef1c0 0x557de95e26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.308+0000 7fc399c10740 1 --2- 172.21.15.86:0/2926952758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95ded00 0x557de95eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.308+0000 7fc399c10740 1 --2- 172.21.15.86:0/2926952758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95d91a0 0x557de95e2170 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.308+0000 7fc399c10740 1 -- 172.21.15.86:0/2926952758 >> 172.21.15.86:0/2926952758 conn(0x557de95d4710 msgr2=0x557de95db120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.309+0000 7fc399c10740 1 -- 172.21.15.86:0/2926952758 shutdown_connections 2024-04-17T07:27:25.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.309+0000 7fc399c10740 1 -- 172.21.15.86:0/2926952758 wait complete. 2024-04-17T07:27:25.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.309+0000 7fc399c10740 1 -- start start 2024-04-17T07:27:25.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.310+0000 7fc399c10740 1 Processor -- start 2024-04-17T07:27:25.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.310+0000 7fc399c10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d91a0 0x557de95e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.310+0000 7fc399c10740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 0x557de95e8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.310+0000 7fc399c10740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95ef1c0 0x557de95e9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.310+0000 7fc399c10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b2520 con 0x557de95d91a0 2024-04-17T07:27:25.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.310+0000 7fc399c10740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901b1e50 con 0x557de95ef1c0 2024-04-17T07:27:25.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.310+0000 7fc399c10740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901af080 con 0x557de95ded00 2024-04-17T07:27:25.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.310+0000 7fc399c10740 10 client.0 _ll_register_callbacks cb 0x7fc380007b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:25.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.311+0000 7fc394e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d91a0 0x557de95e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.311+0000 7fc394e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d91a0 0x557de95e7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57206/0 (socket says 172.21.15.86:57206) 2024-04-17T07:27:25.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.311+0000 7fc394e00640 1 -- 172.21.15.86:0/482323088 learned_addr learned my addr 172.21.15.86:0/482323088 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:25.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.312+0000 7fc38f400640 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 0x557de95e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.312+0000 7fc394e00640 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95ef1c0 0x557de95e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.313+0000 7fc38f400640 1 -- 172.21.15.86:0/482323088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95ef1c0 msgr2=0x557de95e9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.313+0000 7fc38f400640 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95ef1c0 0x557de95e9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.313+0000 7fc38f400640 1 -- 172.21.15.86:0/482323088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d91a0 msgr2=0x557de95e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.313+0000 7fc38f400640 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d91a0 0x557de95e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.313+0000 7fc38f400640 1 -- 172.21.15.86:0/482323088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de931b7d0 con 0x557de95ded00 2024-04-17T07:27:25.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.313+0000 7fc394e00640 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d91a0 0x557de95e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:27:25.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.314+0000 7fc38f400640 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 0x557de95e8480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc380011000 tx=0x7fc38000dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81f82ed991ecf54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.314+0000 7fc37e000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:27:25.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.316+0000 7fc37e000640 1 -- 172.21.15.86:0/482323088 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3800123e0 con 0x557de95ded00 2024-04-17T07:27:25.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.316+0000 7fc37e000640 1 -- 172.21.15.86:0/482323088 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc380012580 con 0x557de95ded00 2024-04-17T07:27:25.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.317+0000 7fc399c10740 1 -- 172.21.15.86:0/482323088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc378050070 con 0x557de95ded00 2024-04-17T07:27:25.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.318+0000 7fc37e000640 1 -- 172.21.15.86:0/482323088 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc3800127c0 con 0x557de95ded00 2024-04-17T07:27:25.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.318+0000 7fc399c10740 10 client.27553 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:25.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.318+0000 7fc399c10740 1 -- 172.21.15.86:0/482323088 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557de95eabb0 con 0x557de95ded00 2024-04-17T07:27:25.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.319+0000 7fc37e000640 1 -- 172.21.15.86:0/482323088 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc378050070 con 0x557de95ded00 2024-04-17T07:27:25.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.319+0000 7fc37e000640 1 -- 172.21.15.86:0/482323088 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc380013480 con 0x557de95ded00 2024-04-17T07:27:25.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.319+0000 7fc399c10740 10 client.27553 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.319+0000 7fc399c10740 10 client.27553 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:27:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.319+0000 7fc399c10740 20 client.27553 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.319+0000 7fc399c10740 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x557de9666bb0 0x557de9668fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.319+0000 7fc399c10740 4 client.27553 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:27:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.319+0000 7fc399c10740 1 -- 172.21.15.86:0/482323088 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557de95468b0 con 0x557de9666bb0 2024-04-17T07:27:25.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.320+0000 7fc394e00640 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x557de9666bb0 0x557de9668fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.322+0000 7fc394e00640 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x557de9666bb0 0x557de9668fd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab171d2fc8a07aea server_cookie=856cdbccc5f38304 in_seq=0 out_seq=0 2024-04-17T07:27:25.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.322+0000 7fc37e000640 10 client.27553 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:27:25.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.323+0000 7fc37e000640 1 -- 172.21.15.86:0/482323088 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557de95468b0 con 0x557de9666bb0 2024-04-17T07:27:25.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.323+0000 7fc37e000640 10 client.27553 handle_command_reply: tid=0 2024-04-17T07:27:25.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.323+0000 7fc399c10740 1 client.27553 shutdown 2024-04-17T07:27:25.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.324+0000 7fc399c10740 1 -- 172.21.15.86:0/482323088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 msgr2=0x557de95e8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.324+0000 7fc399c10740 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 0x557de95e8480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc380011000 tx=0x7fc38000dae0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.325+0000 7fc399c10740 1 -- 172.21.15.86:0/482323088 shutdown_connections 2024-04-17T07:27:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.325+0000 7fc399c10740 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x557de9666bb0 0x557de9668fd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.325+0000 7fc399c10740 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557de95ef1c0 0x557de95e9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.325+0000 7fc37e000640 0 client.27553 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:27:25.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.325+0000 7fc399c10740 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557de95ded00 0x557de95e8480 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.325+0000 7fc399c10740 1 --2- 172.21.15.86:0/482323088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557de95d91a0 0x557de95e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.325+0000 7fc399c10740 1 -- 172.21.15.86:0/482323088 >> 172.21.15.86:0/482323088 conn(0x557de95d4710 msgr2=0x557de95e3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.326+0000 7fc399c10740 1 -- 172.21.15.86:0/482323088 shutdown_connections 2024-04-17T07:27:25.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.326+0000 7fc399c10740 1 -- 172.21.15.86:0/482323088 wait complete. 2024-04-17T07:27:25.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.328+0000 7fc399c10740 20 client.27553 trim_cache size 0 max 16384 2024-04-17T07:27:25.327 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:27:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.329+0000 7fc395800640 1 -- 172.21.15.86:0/2235000211 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc3740a3c30 msgr2=0x7fc3740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.329+0000 7fc395800640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc3740a3c30 0x7fc3740a60f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc38809c680 tx=0x7fc388004420 comp rx=0 tx=0).stop 2024-04-17T07:27:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.329+0000 7fc395800640 1 -- 172.21.15.86:0/2235000211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc390103fe0 msgr2=0x7fc39019e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.329+0000 7fc395800640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc390103fe0 0x7fc39019e5f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c3a0 tx=0x7fc38800c860 comp rx=0 tx=0).stop 2024-04-17T07:27:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.329+0000 7fc395800640 1 -- 172.21.15.86:0/2235000211 shutdown_connections 2024-04-17T07:27:25.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.329+0000 7fc395800640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc3740a3c30 0x7fc3740a60f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.329+0000 7fc395800640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390105b90 0x7fc3901a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.329+0000 7fc395800640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3901051d0 0x7fc39019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.329+0000 7fc395800640 1 --2- 172.21.15.86:0/2235000211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc390103fe0 0x7fc39019e5f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.329+0000 7fc395800640 1 -- 172.21.15.86:0/2235000211 >> 172.21.15.86:0/2235000211 conn(0x7fc3900ff570 msgr2=0x7fc3900783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.330+0000 7fc395800640 1 -- 172.21.15.86:0/2235000211 shutdown_connections 2024-04-17T07:27:25.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.330+0000 7fc395800640 1 -- 172.21.15.86:0/2235000211 wait complete. 2024-04-17T07:27:25.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:27:25.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:25 smithi098 ceph-mon[35418]: pgmap v831: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 664 op/s 2024-04-17T07:27:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:25 smithi103 ceph-mon[35495]: pgmap v831: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 10 MiB/s wr, 664 op/s 2024-04-17T07:27:25.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.692+0000 7f2d50000640 1 -- 172.21.15.86:0/1030707345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d4810e2a0 msgr2=0x7f2d4810e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.692+0000 7f2d50000640 1 --2- 172.21.15.86:0/1030707345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d4810e2a0 0x7f2d4810e750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009ca0 tx=0x7f2d4002c100 comp rx=0 tx=0).stop 2024-04-17T07:27:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.693+0000 7f2d50000640 1 -- 172.21.15.86:0/1030707345 shutdown_connections 2024-04-17T07:27:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.694+0000 7f2d50000640 1 --2- 172.21.15.86:0/1030707345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d4810e2a0 0x7f2d4810e750 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.695+0000 7f2d50000640 1 --2- 172.21.15.86:0/1030707345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4810b8d0 0x7f2d4810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.695+0000 7f2d50000640 1 --2- 172.21.15.86:0/1030707345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d48108f60 0x7f2d4810b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.695+0000 7f2d50000640 1 -- 172.21.15.86:0/1030707345 >> 172.21.15.86:0/1030707345 conn(0x7f2d4806edf0 msgr2=0x7f2d48071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.696+0000 7f2d50000640 1 -- 172.21.15.86:0/1030707345 shutdown_connections 2024-04-17T07:27:25.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.696+0000 7f2d50000640 1 -- 172.21.15.86:0/1030707345 wait complete. 2024-04-17T07:27:25.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.698+0000 7f2d50000640 1 Processor -- start 2024-04-17T07:27:25.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.699+0000 7f2d50000640 1 -- start start 2024-04-17T07:27:25.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.699+0000 7f2d50000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d48108f60 0x7f2d481a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.699+0000 7f2d50000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d4810b8d0 0x7f2d481a7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.699+0000 7f2d50000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4810e2a0 0x7f2d481ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.699+0000 7f2d50000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4811c930 con 0x7f2d48108f60 2024-04-17T07:27:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.699+0000 7f2d50000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4811c7b0 con 0x7f2d4810e2a0 2024-04-17T07:27:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.699+0000 7f2d50000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4811cab0 con 0x7f2d4810b8d0 2024-04-17T07:27:25.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.700+0000 7f2d4e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4810e2a0 0x7f2d481ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.700+0000 7f2d4e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4810e2a0 0x7f2d481ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:48998/0 (socket says 172.21.15.86:48998) 2024-04-17T07:27:25.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.700+0000 7f2d4e000640 1 -- 172.21.15.86:0/2091392112 learned_addr learned my addr 172.21.15.86:0/2091392112 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:25.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.700+0000 7f2d4d600640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d48108f60 0x7f2d481a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.700+0000 7f2d4e000640 1 -- 172.21.15.86:0/2091392112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d4810b8d0 msgr2=0x7f2d481a7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.700+0000 7f2d4e000640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d4810b8d0 0x7f2d481a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.700+0000 7f2d4e000640 1 -- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d48108f60 msgr2=0x7f2d481a70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.700+0000 7f2d4e000640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d48108f60 0x7f2d481a70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.700+0000 7f2d4e000640 1 -- 172.21.15.86:0/2091392112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d481aeca0 con 0x7f2d4810e2a0 2024-04-17T07:27:25.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.700+0000 7f2d4d600640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d48108f60 0x7f2d481a70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:27:25.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.701+0000 7f2d4e000640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4810e2a0 0x7f2d481ae630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d40037040 tx=0x7f2d40035ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.701+0000 7f2d46000640 1 -- 172.21.15.86:0/2091392112 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40039040 con 0x7f2d4810e2a0 2024-04-17T07:27:25.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.702+0000 7f2d50000640 1 -- 172.21.15.86:0/2091392112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d481aef30 con 0x7f2d4810e2a0 2024-04-17T07:27:25.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.702+0000 7f2d46000640 1 -- 172.21.15.86:0/2091392112 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d40044070 con 0x7f2d4810e2a0 2024-04-17T07:27:25.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.702+0000 7f2d46000640 1 -- 172.21.15.86:0/2091392112 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40004430 con 0x7f2d4810e2a0 2024-04-17T07:27:25.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.702+0000 7f2d50000640 1 -- 172.21.15.86:0/2091392112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d481af4c0 con 0x7f2d4810e2a0 2024-04-17T07:27:25.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.707+0000 7f2d52dc0740 1 Processor -- start 2024-04-17T07:27:25.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.709+0000 7f2d46000640 1 -- 172.21.15.86:0/2091392112 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2d400045d0 con 0x7f2d4810e2a0 2024-04-17T07:27:25.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.709+0000 7f2d52dc0740 1 -- start start 2024-04-17T07:27:25.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.710+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.710+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cc990 0x55db6b8d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.710+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8d1b00 0x55db6b8d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.710+0000 7f2d52dc0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db6b832020 con 0x55db6b8cc990 2024-04-17T07:27:25.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.710+0000 7f2d52dc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55db6b7d8460 con 0x55db6b8cbfa0 2024-04-17T07:27:25.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.710+0000 7f2d52dc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55db6b82dfe0 con 0x55db6b8d1b00 2024-04-17T07:27:25.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.710+0000 7f2d4d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cc990 0x55db6b8d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.710+0000 7f2d4d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cc990 0x55db6b8d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57238/0 (socket says 172.21.15.86:57238) 2024-04-17T07:27:25.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.710+0000 7f2d4d600640 1 -- 172.21.15.86:0/4085628119 learned_addr learned my addr 172.21.15.86:0/4085628119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:25.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.711+0000 7f2d4cc00640 1 --2- 172.21.15.86:0/4085628119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.711+0000 7f2d4d600640 1 -- 172.21.15.86:0/4085628119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8d1b00 msgr2=0x55db6b8d3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.711+0000 7f2d4d600640 1 --2- 172.21.15.86:0/4085628119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8d1b00 0x55db6b8d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.711+0000 7f2d4d600640 1 -- 172.21.15.86:0/4085628119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 msgr2=0x55db6b8cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.711+0000 7f2d46000640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d300a3d00 0x7f2d300a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.711+0000 7f2d46000640 1 -- 172.21.15.86:0/2091392112 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2d400f27f0 con 0x7f2d4810e2a0 2024-04-17T07:27:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.711+0000 7f2d4d600640 1 --2- 172.21.15.86:0/4085628119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.711+0000 7f2d4d600640 1 -- 172.21.15.86:0/4085628119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db6b7e0040 con 0x55db6b8cc990 2024-04-17T07:27:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.711+0000 7f2d4cc00640 1 --2- 172.21.15.86:0/4085628119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:27:25.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.712+0000 7f2d4cc00640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d300a3d00 0x7f2d300a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.713+0000 7f2d4d600640 1 --2- 172.21.15.86:0/4085628119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cc990 0x55db6b8d1490 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c007850 tx=0x7f2d3c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ca80b224659389 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.714+0000 7f2d4cc00640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d300a3d00 0x7f2d300a61c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55db6b8cd1f0 tx=0x7f2d380046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.716+0000 7f2d36600640 1 -- 172.21.15.86:0/4085628119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c00d950 con 0x55db6b8cc990 2024-04-17T07:27:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.716+0000 7f2d36600640 1 -- 172.21.15.86:0/4085628119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d3c018070 con 0x55db6b8cc990 2024-04-17T07:27:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.717+0000 7f2d52dc0740 1 -- 172.21.15.86:0/4085628119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cc990 msgr2=0x55db6b8d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.717+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/4085628119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cc990 0x55db6b8d1490 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c007850 tx=0x7f2d3c007d10 comp rx=0 tx=0).stop 2024-04-17T07:27:25.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.718+0000 7f2d52dc0740 1 -- 172.21.15.86:0/4085628119 shutdown_connections 2024-04-17T07:27:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.718+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/4085628119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8d1b00 0x55db6b8d3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.718+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/4085628119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cc990 0x55db6b8d1490 secure :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c007850 tx=0x7f2d3c007d10 comp rx=0 tx=0).stop 2024-04-17T07:27:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.718+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/4085628119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.718+0000 7f2d52dc0740 1 -- 172.21.15.86:0/4085628119 >> 172.21.15.86:0/4085628119 conn(0x55db6b8c7510 msgr2=0x55db6b8c7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.721+0000 7f2d52dc0740 1 -- 172.21.15.86:0/4085628119 shutdown_connections 2024-04-17T07:27:25.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.721+0000 7f2d52dc0740 1 -- 172.21.15.86:0/4085628119 wait complete. 2024-04-17T07:27:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.721+0000 7f2d52dc0740 1 -- start start 2024-04-17T07:27:25.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.722+0000 7f2d52dc0740 1 Processor -- start 2024-04-17T07:27:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.723+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8d6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.723+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8d1b00 0x55db6b8d6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.723+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8dbd70 0x55db6b8de160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.723+0000 7f2d4d600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8d6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.723+0000 7f2d4d600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8d6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:49028/0 (socket says 172.21.15.86:49028) 2024-04-17T07:27:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.724+0000 7f2d52dc0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55db6b8d4b00 con 0x55db6b8dbd70 2024-04-17T07:27:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.724+0000 7f2d4cc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8d1b00 0x55db6b8d6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.724+0000 7f2d4cc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8d1b00 0x55db6b8d6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33200/0 (socket says 172.21.15.86:33200) 2024-04-17T07:27:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.724+0000 7f2d4cc00640 1 -- 172.21.15.86:0/341060551 learned_addr learned my addr 172.21.15.86:0/341060551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.724+0000 7f2d52dc0740 1 -- 172.21.15.86:0/341060551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55db6b7eb830 con 0x55db6b8cbfa0 2024-04-17T07:27:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.724+0000 7f2d52dc0740 1 -- 172.21.15.86:0/341060551 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55db6b8d4c80 con 0x55db6b8d1b00 2024-04-17T07:27:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.724+0000 7f2d52dc0740 10 client.0 _ll_register_callbacks cb 0x55db6b7b9fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:25.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.724+0000 7f2d4e000640 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8dbd70 0x55db6b8de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.726+0000 7f2d4d600640 1 -- 172.21.15.86:0/341060551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8d1b00 msgr2=0x55db6b8d6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.726+0000 7f2d4d600640 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8d1b00 0x55db6b8d6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.726+0000 7f2d4d600640 1 -- 172.21.15.86:0/341060551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8dbd70 msgr2=0x55db6b8de160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.726+0000 7f2d4d600640 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8dbd70 0x55db6b8de160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.726+0000 7f2d4d600640 1 -- 172.21.15.86:0/341060551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db6b7ef920 con 0x55db6b8cbfa0 2024-04-17T07:27:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.727+0000 7f2d2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:27:25.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.727+0000 7f2d4d600640 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8d6800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55db6b8cce50 tx=0x7f2d3c00db80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a6da6b84002f798 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.728+0000 7f2d2aa00640 1 -- 172.21.15.86:0/341060551 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c022020 con 0x55db6b8cbfa0 2024-04-17T07:27:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.729+0000 7f2d2aa00640 1 -- 172.21.15.86:0/341060551 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2d3c013680 con 0x55db6b8cbfa0 2024-04-17T07:27:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.729+0000 7f2d52dc0740 1 -- 172.21.15.86:0/341060551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db6b83fa40 con 0x55db6b8cbfa0 2024-04-17T07:27:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.730+0000 7f2d2aa00640 1 -- 172.21.15.86:0/341060551 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d3c013880 con 0x55db6b8cbfa0 2024-04-17T07:27:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.730+0000 7f2d52dc0740 10 client.37487 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.731+0000 7f2d52dc0740 1 -- 172.21.15.86:0/341060551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db6b7bd7f0 con 0x55db6b8cbfa0 2024-04-17T07:27:25.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.731+0000 7f2d2aa00640 1 -- 172.21.15.86:0/341060551 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55db6b83fa40 con 0x55db6b8cbfa0 2024-04-17T07:27:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.734+0000 7f2d2aa00640 1 -- 172.21.15.86:0/341060551 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2d3c01c410 con 0x55db6b8cbfa0 2024-04-17T07:27:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.735+0000 7f2d52dc0740 10 client.37487 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.735+0000 7f2d52dc0740 10 client.37487 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:27:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.735+0000 7f2d52dc0740 20 client.37487 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.735+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55db6b8e2400 0x55db6b8e4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.735+0000 7f2d52dc0740 4 client.37487 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:27:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.736+0000 7f2d52dc0740 1 -- 172.21.15.86:0/341060551 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db6b8398b0 con 0x55db6b8e2400 2024-04-17T07:27:25.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.736+0000 7f2d4cc00640 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55db6b8e2400 0x55db6b8e4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.738+0000 7f2d4cc00640 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55db6b8e2400 0x55db6b8e4830 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b64d378635bea891 server_cookie=a818303b77af1d50 in_seq=0 out_seq=0 2024-04-17T07:27:25.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.739+0000 7f2d2aa00640 10 client.37487 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:27:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.741+0000 7f2d2aa00640 1 -- 172.21.15.86:0/341060551 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db6b8398b0 con 0x55db6b8e2400 2024-04-17T07:27:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.741+0000 7f2d2aa00640 10 client.37487 handle_command_reply: tid=0 2024-04-17T07:27:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.741+0000 7f2d52dc0740 1 client.37487 shutdown 2024-04-17T07:27:25.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.742+0000 7f2d52dc0740 1 -- 172.21.15.86:0/341060551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 msgr2=0x55db6b8d6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.742+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8d6800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55db6b8cce50 tx=0x7f2d3c00db80 comp rx=0 tx=0).stop 2024-04-17T07:27:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.742+0000 7f2d52dc0740 1 -- 172.21.15.86:0/341060551 shutdown_connections 2024-04-17T07:27:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.742+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8dbd70 0x55db6b8de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.742+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8d1b00 0x55db6b8d6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.742+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55db6b8e2400 0x55db6b8e4830 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.742+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/341060551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8cbfa0 0x55db6b8d6800 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.742+0000 7f2d52dc0740 1 -- 172.21.15.86:0/341060551 >> 172.21.15.86:0/341060551 conn(0x55db6b8c7510 msgr2=0x55db6b8c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.742+0000 7f2d2aa00640 0 client.37487 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:27:25.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.743+0000 7f2d52dc0740 1 -- 172.21.15.86:0/341060551 shutdown_connections 2024-04-17T07:27:25.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.743+0000 7f2d52dc0740 1 -- 172.21.15.86:0/341060551 wait complete. 2024-04-17T07:27:25.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.746+0000 7f2d52dc0740 20 client.37487 trim_cache size 0 max 16384 2024-04-17T07:27:25.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.785+0000 7f2d52dc0740 1 Processor -- start 2024-04-17T07:27:25.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.785+0000 7f2d52dc0740 1 -- start start 2024-04-17T07:27:25.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.786+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8cbfa0 0x55db6b8c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.786+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8d1b00 0x55db6b8c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.787+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8dbd70 0x55db6b8c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.787+0000 7f2d52dc0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c00fb50 con 0x55db6b8d1b00 2024-04-17T07:27:25.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.787+0000 7f2d52dc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c011690 con 0x55db6b8dbd70 2024-04-17T07:27:25.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.787+0000 7f2d4cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8d1b00 0x55db6b8c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.787+0000 7f2d4cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8d1b00 0x55db6b8c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57266/0 (socket says 172.21.15.86:57266) 2024-04-17T07:27:25.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.787+0000 7f2d4cc00640 1 -- 172.21.15.86:0/2035591275 learned_addr learned my addr 172.21.15.86:0/2035591275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:25.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.787+0000 7f2d4d600640 1 --2- 172.21.15.86:0/2035591275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8cbfa0 0x55db6b8c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.787+0000 7f2d52dc0740 1 -- 172.21.15.86:0/2035591275 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c010220 con 0x55db6b8cbfa0 2024-04-17T07:27:25.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.787+0000 7f2d4e000640 1 --2- 172.21.15.86:0/2035591275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8dbd70 0x55db6b8c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.788+0000 7f2d4e000640 1 -- 172.21.15.86:0/2035591275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8cbfa0 msgr2=0x55db6b8c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.788+0000 7f2d4e000640 1 --2- 172.21.15.86:0/2035591275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8cbfa0 0x55db6b8c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.788+0000 7f2d4e000640 1 -- 172.21.15.86:0/2035591275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8d1b00 msgr2=0x55db6b8c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.788+0000 7f2d4e000640 1 --2- 172.21.15.86:0/2035591275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8d1b00 0x55db6b8c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.789+0000 7f2d4e000640 1 -- 172.21.15.86:0/2035591275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db6b5443f0 con 0x55db6b8dbd70 2024-04-17T07:27:25.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.790+0000 7f2d4e000640 1 --2- 172.21.15.86:0/2035591275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8dbd70 0x55db6b8c6900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d400be3f0 tx=0x7f2d400a9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c33b7b7bdf7b8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.790+0000 7f2d35c00640 1 -- 172.21.15.86:0/2035591275 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d400bedf0 con 0x55db6b8dbd70 2024-04-17T07:27:25.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.792+0000 7f2d35c00640 1 -- 172.21.15.86:0/2035591275 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d400be6f0 con 0x55db6b8dbd70 2024-04-17T07:27:25.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.792+0000 7f2d35c00640 1 -- 172.21.15.86:0/2035591275 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40046310 con 0x55db6b8dbd70 2024-04-17T07:27:25.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.793+0000 7f2d52dc0740 1 -- 172.21.15.86:0/2035591275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8dbd70 msgr2=0x55db6b8c6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.793+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/2035591275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8dbd70 0x55db6b8c6900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d400be3f0 tx=0x7f2d400a9040 comp rx=0 tx=0).stop 2024-04-17T07:27:25.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.794+0000 7f2d52dc0740 1 -- 172.21.15.86:0/2035591275 shutdown_connections 2024-04-17T07:27:25.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.794+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/2035591275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8dbd70 0x55db6b8c6900 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.794+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/2035591275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8d1b00 0x55db6b8c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.794+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/2035591275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8cbfa0 0x55db6b8c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.794+0000 7f2d52dc0740 1 -- 172.21.15.86:0/2035591275 >> 172.21.15.86:0/2035591275 conn(0x55db6b8c7510 msgr2=0x55db6b8ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.795+0000 7f2d52dc0740 1 -- 172.21.15.86:0/2035591275 shutdown_connections 2024-04-17T07:27:25.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.795+0000 7f2d52dc0740 1 -- 172.21.15.86:0/2035591275 wait complete. 2024-04-17T07:27:25.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.796+0000 7f2d52dc0740 1 -- start start 2024-04-17T07:27:25.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.797+0000 7f2d52dc0740 1 Processor -- start 2024-04-17T07:27:25.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.798+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cbfa0 0x55db6b8f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.800+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8d1b00 0x55db6b8f3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.800+0000 7f2d52dc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8dbd70 0x55db6b946480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.800+0000 7f2d52dc0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c010fc0 con 0x55db6b8cbfa0 2024-04-17T07:27:25.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.800+0000 7f2d52dc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c0108f0 con 0x55db6b8d1b00 2024-04-17T07:27:25.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.800+0000 7f2d52dc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c00db20 con 0x55db6b8dbd70 2024-04-17T07:27:25.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.800+0000 7f2d52dc0740 10 client.0 _ll_register_callbacks cb 0x7f2d3c0074a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:25.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.800+0000 7f2d4d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cbfa0 0x55db6b8f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.800+0000 7f2d4d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cbfa0 0x55db6b8f2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57278/0 (socket says 172.21.15.86:57278) 2024-04-17T07:27:25.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.800+0000 7f2d4d600640 1 -- 172.21.15.86:0/1901514793 learned_addr learned my addr 172.21.15.86:0/1901514793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:25.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.801+0000 7f2d4e000640 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8dbd70 0x55db6b946480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.801+0000 7f2d4cc00640 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8d1b00 0x55db6b8f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.801+0000 7f2d4d600640 1 -- 172.21.15.86:0/1901514793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8dbd70 msgr2=0x55db6b946480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.801+0000 7f2d4d600640 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8dbd70 0x55db6b946480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.801+0000 7f2d4d600640 1 -- 172.21.15.86:0/1901514793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8d1b00 msgr2=0x55db6b8f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:25.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.801+0000 7f2d4d600640 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8d1b00 0x55db6b8f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.801+0000 7f2d4d600640 1 -- 172.21.15.86:0/1901514793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db6b60e7d0 con 0x55db6b8cbfa0 2024-04-17T07:27:25.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.802+0000 7f2d4d600640 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cbfa0 0x55db6b8f2c30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c000c00 tx=0x7f2d3c0136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5dd7b558825c2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:25.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.802+0000 7f2d2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:27:25.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.803+0000 7f2d2aa00640 1 -- 172.21.15.86:0/1901514793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c022040 con 0x55db6b8cbfa0 2024-04-17T07:27:25.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.803+0000 7f2d2aa00640 1 -- 172.21.15.86:0/1901514793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2d3c01e460 con 0x55db6b8cbfa0 2024-04-17T07:27:25.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.803+0000 7f2d52dc0740 1 -- 172.21.15.86:0/1901514793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d400be6f0 con 0x55db6b8cbfa0 2024-04-17T07:27:25.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.804+0000 7f2d2aa00640 1 -- 172.21.15.86:0/1901514793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d3c01ce50 con 0x55db6b8cbfa0 2024-04-17T07:27:25.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.804+0000 7f2d52dc0740 10 client.18621 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:25.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.804+0000 7f2d52dc0740 1 -- 172.21.15.86:0/1901514793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db6b947ac0 con 0x55db6b8cbfa0 2024-04-17T07:27:25.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.805+0000 7f2d2aa00640 1 -- 172.21.15.86:0/1901514793 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d400be6f0 con 0x55db6b8cbfa0 2024-04-17T07:27:25.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.805+0000 7f2d2aa00640 1 -- 172.21.15.86:0/1901514793 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2d3c039450 con 0x55db6b8cbfa0 2024-04-17T07:27:25.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.805+0000 7f2d52dc0740 10 client.18621 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:25.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.805+0000 7f2d52dc0740 10 client.18621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:27:25.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.805+0000 7f2d52dc0740 20 client.18621 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:25.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.805+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55db6b948460 0x55db6b94a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:25.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.805+0000 7f2d52dc0740 4 client.18621 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:27:25.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.805+0000 7f2d52dc0740 1 -- 172.21.15.86:0/1901514793 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db6b7b5570 con 0x55db6b948460 2024-04-17T07:27:25.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.806+0000 7f2d4cc00640 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55db6b948460 0x55db6b94a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:25.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.807+0000 7f2d4cc00640 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55db6b948460 0x55db6b94a880 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=168624a0a785524c server_cookie=89536991e124929c in_seq=0 out_seq=0 2024-04-17T07:27:25.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.807+0000 7f2d2aa00640 10 client.18621 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:27:25.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.809+0000 7f2d2aa00640 1 -- 172.21.15.86:0/1901514793 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db6b7b5570 con 0x55db6b948460 2024-04-17T07:27:25.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.809+0000 7f2d2aa00640 10 client.18621 handle_command_reply: tid=0 2024-04-17T07:27:25.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.809+0000 7f2d52dc0740 1 client.18621 shutdown 2024-04-17T07:27:25.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.809+0000 7f2d52dc0740 1 -- 172.21.15.86:0/1901514793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cbfa0 msgr2=0x55db6b8f2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.809+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cbfa0 0x55db6b8f2c30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c000c00 tx=0x7f2d3c0136b0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.810+0000 7f2d52dc0740 1 -- 172.21.15.86:0/1901514793 shutdown_connections 2024-04-17T07:27:25.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.810+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55db6b8dbd70 0x55db6b946480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.810+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55db6b948460 0x55db6b94a880 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.810+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55db6b8d1b00 0x55db6b8f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.810+0000 7f2d52dc0740 1 --2- 172.21.15.86:0/1901514793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55db6b8cbfa0 0x55db6b8f2c30 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.811+0000 7f2d52dc0740 1 -- 172.21.15.86:0/1901514793 >> 172.21.15.86:0/1901514793 conn(0x55db6b8c7510 msgr2=0x55db6b8ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.811+0000 7f2d2aa00640 0 client.18621 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:27:25.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.811+0000 7f2d52dc0740 1 -- 172.21.15.86:0/1901514793 shutdown_connections 2024-04-17T07:27:25.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.811+0000 7f2d52dc0740 1 -- 172.21.15.86:0/1901514793 wait complete. 2024-04-17T07:27:25.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.812+0000 7f2d52dc0740 20 client.18621 trim_cache size 0 max 16384 2024-04-17T07:27:25.812 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:27:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.813+0000 7f2d50000640 1 -- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d300a3d00 msgr2=0x7f2d300a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.813+0000 7f2d50000640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d300a3d00 0x7f2d300a61c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55db6b8cd1f0 tx=0x7f2d380046c0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.814+0000 7f2d50000640 1 -- 172.21.15.86:0/2091392112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4810e2a0 msgr2=0x7f2d481ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.814+0000 7f2d50000640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4810e2a0 0x7f2d481ae630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d40037040 tx=0x7f2d40035ce0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.814+0000 7f2d50000640 1 -- 172.21.15.86:0/2091392112 shutdown_connections 2024-04-17T07:27:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.814+0000 7f2d50000640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2d300a3d00 0x7f2d300a61c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.814+0000 7f2d50000640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d4810e2a0 0x7f2d481ae630 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.814+0000 7f2d50000640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d4810b8d0 0x7f2d481a7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.814+0000 7f2d50000640 1 --2- 172.21.15.86:0/2091392112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d48108f60 0x7f2d481a70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.814+0000 7f2d50000640 1 -- 172.21.15.86:0/2091392112 >> 172.21.15.86:0/2091392112 conn(0x7f2d4806edf0 msgr2=0x7f2d4810bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.815+0000 7f2d50000640 1 -- 172.21.15.86:0/2091392112 shutdown_connections 2024-04-17T07:27:25.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:25.815+0000 7f2d50000640 1 -- 172.21.15.86:0/2091392112 wait complete. 2024-04-17T07:27:25.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:27:26.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.161+0000 7fcdd9400640 1 -- 172.21.15.86:0/3197757280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdd410a380 msgr2=0x7fcdd410a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:26.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.161+0000 7fcdd1e00640 1 -- 172.21.15.86:0/3197757280 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc033710 con 0x7fcdd410a380 2024-04-17T07:27:26.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.161+0000 7fcdd9400640 1 --2- 172.21.15.86:0/3197757280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdd410a380 0x7fcdd410a800 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009be0 tx=0x7fcdcc02bef0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.162+0000 7fcdd9400640 1 -- 172.21.15.86:0/3197757280 shutdown_connections 2024-04-17T07:27:26.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.162+0000 7fcdd9400640 1 --2- 172.21.15.86:0/3197757280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdd410a380 0x7fcdd410a800 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.162+0000 7fcdd9400640 1 --2- 172.21.15.86:0/3197757280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd41099e0 0x7fcdd4109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.162+0000 7fcdd9400640 1 --2- 172.21.15.86:0/3197757280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdd4109080 0x7fcdd41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.162+0000 7fcdd9400640 1 -- 172.21.15.86:0/3197757280 >> 172.21.15.86:0/3197757280 conn(0x7fcdd406fe40 msgr2=0x7fcdd4072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:26.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.165+0000 7fcdd9400640 1 -- 172.21.15.86:0/3197757280 shutdown_connections 2024-04-17T07:27:26.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.165+0000 7fcdd9400640 1 -- 172.21.15.86:0/3197757280 wait complete. 2024-04-17T07:27:26.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.167+0000 7fcdd9400640 1 Processor -- start 2024-04-17T07:27:26.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.167+0000 7fcdd9400640 1 -- start start 2024-04-17T07:27:26.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.167+0000 7fcdd9400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdd4109080 0x7fcdd4074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.168+0000 7fcdd9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd41099e0 0x7fcdd4075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.168+0000 7fcdd9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdd410a380 0x7fcdd407c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.168+0000 7fcdd2800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdd4109080 0x7fcdd4074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.168+0000 7fcdd2800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdd4109080 0x7fcdd4074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:49068/0 (socket says 172.21.15.86:49068) 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.168+0000 7fcdd2800640 1 -- 172.21.15.86:0/1823607768 learned_addr learned my addr 172.21.15.86:0/1823607768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.168+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4118590 con 0x7fcdd41099e0 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.168+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4118410 con 0x7fcdd4109080 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.168+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4118710 con 0x7fcdd410a380 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.169+0000 7fcdd3200640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdd410a380 0x7fcdd407c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.169+0000 7fcdd3200640 1 -- 172.21.15.86:0/1823607768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdd4109080 msgr2=0x7fcdd4074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.169+0000 7fcdd3200640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdd4109080 0x7fcdd4074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.169+0000 7fcdd3200640 1 -- 172.21.15.86:0/1823607768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd41099e0 msgr2=0x7fcdd4075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.169+0000 7fcdd3200640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd41099e0 0x7fcdd4075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.169+0000 7fcdd3200640 1 -- 172.21.15.86:0/1823607768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd407c7a0 con 0x7fcdd410a380 2024-04-17T07:27:26.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.170+0000 7fcdd3200640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdd410a380 0x7fcdd407c100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc036040 tx=0x7fcdcc033690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:26.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.170+0000 7fcdcb400640 1 -- 172.21.15.86:0/1823607768 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc043020 con 0x7fcdd410a380 2024-04-17T07:27:26.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.170+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd407ca30 con 0x7fcdd410a380 2024-04-17T07:27:26.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.171+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd407d050 con 0x7fcdd410a380 2024-04-17T07:27:26.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.171+0000 7fcdcb400640 1 -- 172.21.15.86:0/1823607768 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdcc003b80 con 0x7fcdd410a380 2024-04-17T07:27:26.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.171+0000 7fcdcb400640 1 -- 172.21.15.86:0/1823607768 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc034430 con 0x7fcdd410a380 2024-04-17T07:27:26.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.179+0000 7fcddbff1740 1 Processor -- start 2024-04-17T07:27:26.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.180+0000 7fcdcb400640 1 -- 172.21.15.86:0/1823607768 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fcdcc046020 con 0x7fcdd410a380 2024-04-17T07:27:26.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.182+0000 7fcdcb400640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcdb80a3c70 0x7fcdb80a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.183+0000 7fcdcbe00640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcdb80a3c70 0x7fcdb80a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.183+0000 7fcddbff1740 1 -- start start 2024-04-17T07:27:26.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.184+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3bfa0 0x561af9c3c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.184+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3c990 0x561af9c41490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.184+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c43f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.184+0000 7fcddbff1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561af9ba2020 con 0x561af9c3bfa0 2024-04-17T07:27:26.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.184+0000 7fcddbff1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561af9b48460 con 0x561af9c41b00 2024-04-17T07:27:26.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.184+0000 7fcddbff1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561af9b9dfe0 con 0x561af9c3c990 2024-04-17T07:27:26.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.185+0000 7fcdd3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c43f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.185+0000 7fcdd3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c43f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:49072/0 (socket says 172.21.15.86:49072) 2024-04-17T07:27:26.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.185+0000 7fcdd3200640 1 -- 172.21.15.86:0/679360375 learned_addr learned my addr 172.21.15.86:0/679360375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:26.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.185+0000 7fcdd2800640 1 --2- 172.21.15.86:0/679360375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3bfa0 0x561af9c3c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.185+0000 7fcdcb400640 1 -- 172.21.15.86:0/1823607768 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcdcc0bd040 con 0x7fcdd410a380 2024-04-17T07:27:26.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.186+0000 7fcdd3200640 1 -- 172.21.15.86:0/679360375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3c990 msgr2=0x561af9c41490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:27:26.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.186+0000 7fcdd3200640 1 --2- 172.21.15.86:0/679360375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3c990 0x561af9c41490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.186+0000 7fcdd3200640 1 -- 172.21.15.86:0/679360375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3bfa0 msgr2=0x561af9c3c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:26.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.186+0000 7fcdd3200640 1 --2- 172.21.15.86:0/679360375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3bfa0 0x561af9c3c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.186+0000 7fcdd3200640 1 -- 172.21.15.86:0/679360375 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af9b50040 con 0x561af9c41b00 2024-04-17T07:27:26.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.187+0000 7fcdd3200640 1 --2- 172.21.15.86:0/679360375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c43f90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009b30 tx=0x7fcdcc0c6660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21114cdf698d583 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:26.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.187+0000 7fcdc3400640 1 -- 172.21.15.86:0/679360375 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0bb9d0 con 0x561af9c41b00 2024-04-17T07:27:26.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.188+0000 7fcdc3400640 1 -- 172.21.15.86:0/679360375 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdcc0bf6a0 con 0x561af9c41b00 2024-04-17T07:27:26.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.188+0000 7fcdc3400640 1 -- 172.21.15.86:0/679360375 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0bbb70 con 0x561af9c41b00 2024-04-17T07:27:26.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.190+0000 7fcdcbe00640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcdb80a3c70 0x7fcdb80a6130 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4075c30 tx=0x7fcdbc006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:26.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.190+0000 7fcddbff1740 1 -- 172.21.15.86:0/679360375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 msgr2=0x561af9c43f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:26.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.190+0000 7fcddbff1740 1 --2- 172.21.15.86:0/679360375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c43f90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009b30 tx=0x7fcdcc0c6660 comp rx=0 tx=0).stop 2024-04-17T07:27:26.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.191+0000 7fcddbff1740 1 -- 172.21.15.86:0/679360375 shutdown_connections 2024-04-17T07:27:26.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.191+0000 7fcddbff1740 1 --2- 172.21.15.86:0/679360375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c43f90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.191+0000 7fcddbff1740 1 --2- 172.21.15.86:0/679360375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3c990 0x561af9c41490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.191+0000 7fcddbff1740 1 --2- 172.21.15.86:0/679360375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3bfa0 0x561af9c3c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.191+0000 7fcddbff1740 1 -- 172.21.15.86:0/679360375 >> 172.21.15.86:0/679360375 conn(0x561af9c37510 msgr2=0x561af9c37910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:26.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.192+0000 7fcddbff1740 1 -- 172.21.15.86:0/679360375 shutdown_connections 2024-04-17T07:27:26.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.192+0000 7fcddbff1740 1 -- 172.21.15.86:0/679360375 wait complete. 2024-04-17T07:27:26.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.193+0000 7fcddbff1740 1 -- start start 2024-04-17T07:27:26.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.193+0000 7fcddbff1740 1 Processor -- start 2024-04-17T07:27:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.196+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c3bfa0 0x561af9c46060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.196+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3c990 0x561af9c465a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.196+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c4d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.196+0000 7fcddbff1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561af9ba2020 con 0x561af9c41b00 2024-04-17T07:27:26.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.196+0000 7fcddbff1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561af9b5b830 con 0x561af9c3bfa0 2024-04-17T07:27:26.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.196+0000 7fcddbff1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561af9b48460 con 0x561af9c3c990 2024-04-17T07:27:26.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.196+0000 7fcddbff1740 10 client.0 _ll_register_callbacks cb 0x561af9b29fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:26.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.198+0000 7fcdd3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c4d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.198+0000 7fcdd3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c4d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57322/0 (socket says 172.21.15.86:57322) 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.198+0000 7fcdd3200640 1 -- 172.21.15.86:0/987711674 learned_addr learned my addr 172.21.15.86:0/987711674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.198+0000 7fcdcbe00640 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3c990 0x561af9c465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.198+0000 7fcdd3200640 1 -- 172.21.15.86:0/987711674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3c990 msgr2=0x561af9c465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.198+0000 7fcdd3200640 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3c990 0x561af9c465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.198+0000 7fcdd3200640 1 -- 172.21.15.86:0/987711674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c3bfa0 msgr2=0x561af9c46060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.198+0000 7fcdd3200640 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c3bfa0 0x561af9c46060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.198+0000 7fcdd3200640 1 -- 172.21.15.86:0/987711674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af9b5f920 con 0x561af9c41b00 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.199+0000 7fcdd3200640 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c4d570 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0bb400 tx=0x7fcdcc033760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a804ef3b97db533c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.199+0000 7fcdb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.199+0000 7fcdb7e00640 1 -- 172.21.15.86:0/987711674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0c7e60 con 0x561af9c41b00 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.199+0000 7fcdb7e00640 1 -- 172.21.15.86:0/987711674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcdcc0df360 con 0x561af9c41b00 2024-04-17T07:27:26.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.199+0000 7fcddbff1740 1 -- 172.21.15.86:0/987711674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdcc0bf6a0 con 0x561af9c41b00 2024-04-17T07:27:26.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.201+0000 7fcdb7e00640 1 -- 172.21.15.86:0/987711674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc0cf050 con 0x561af9c41b00 2024-04-17T07:27:26.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.203+0000 7fcddbff1740 10 client.18627 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:26.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.203+0000 7fcddbff1740 1 -- 172.21.15.86:0/987711674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561af9b2d7f0 con 0x561af9c41b00 2024-04-17T07:27:26.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.204+0000 7fcdb7e00640 1 -- 172.21.15.86:0/987711674 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdcc0bf6a0 con 0x561af9c41b00 2024-04-17T07:27:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.205+0000 7fcdb7e00640 1 -- 172.21.15.86:0/987711674 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcdcc050060 con 0x561af9c41b00 2024-04-17T07:27:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.205+0000 7fcddbff1740 10 client.18627 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:26.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.205+0000 7fcddbff1740 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:27:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.205+0000 7fcddbff1740 20 client.18627 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.205+0000 7fcddbff1740 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561af9c51940 0x561af9c53dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.205+0000 7fcddbff1740 4 client.18627 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:27:26.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.205+0000 7fcddbff1740 1 -- 172.21.15.86:0/987711674 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561af9bafa40 con 0x561af9c51940 2024-04-17T07:27:26.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.206+0000 7fcdd2800640 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561af9c51940 0x561af9c53dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.210+0000 7fcdd2800640 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561af9c51940 0x561af9c53dc0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1059ac2e1d07a1b8 server_cookie=8cd74264dd353c35 in_seq=0 out_seq=0 2024-04-17T07:27:26.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.210+0000 7fcdb7e00640 10 client.18627 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:26.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.211+0000 7fcdb7e00640 1 -- 172.21.15.86:0/987711674 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561af9bafa40 con 0x561af9c51940 2024-04-17T07:27:26.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.211+0000 7fcdb7e00640 10 client.18627 handle_command_reply: tid=0 2024-04-17T07:27:26.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.212+0000 7fcddbff1740 1 client.18627 shutdown 2024-04-17T07:27:26.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.213+0000 7fcddbff1740 1 -- 172.21.15.86:0/987711674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 msgr2=0x561af9c4d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:26.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.213+0000 7fcddbff1740 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c4d570 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0bb400 tx=0x7fcdcc033760 comp rx=0 tx=0).stop 2024-04-17T07:27:26.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.213+0000 7fcddbff1740 1 -- 172.21.15.86:0/987711674 shutdown_connections 2024-04-17T07:27:26.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.213+0000 7fcddbff1740 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561af9c51940 0x561af9c53dc0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.213+0000 7fcddbff1740 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c4d570 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.213+0000 7fcddbff1740 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3c990 0x561af9c465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.213+0000 7fcddbff1740 1 --2- 172.21.15.86:0/987711674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c3bfa0 0x561af9c46060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.214+0000 7fcddbff1740 1 -- 172.21.15.86:0/987711674 >> 172.21.15.86:0/987711674 conn(0x561af9c37510 msgr2=0x561af9c36710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:26.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.214+0000 7fcdb7e00640 0 client.18627 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:26.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.214+0000 7fcddbff1740 1 -- 172.21.15.86:0/987711674 shutdown_connections 2024-04-17T07:27:26.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.215+0000 7fcddbff1740 1 -- 172.21.15.86:0/987711674 wait complete. 2024-04-17T07:27:26.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.216+0000 7fcddbff1740 20 client.18627 trim_cache size 0 max 16384 2024-04-17T07:27:26.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.245+0000 7fcddbff1740 1 Processor -- start 2024-04-17T07:27:26.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.245+0000 7fcddbff1740 1 -- start start 2024-04-17T07:27:26.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.245+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c3ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.245+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c3c990 0x561af9c4d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.245+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c44ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.245+0000 7fcddbff1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb000fb10 con 0x561af9c41b00 2024-04-17T07:27:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.245+0000 7fcddbff1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb0011650 con 0x561af9c3c990 2024-04-17T07:27:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.245+0000 7fcddbff1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb00101e0 con 0x561af9c3bfa0 2024-04-17T07:27:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.246+0000 7fcdd2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c3ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.246+0000 7fcdd2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c3ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33258/0 (socket says 172.21.15.86:33258) 2024-04-17T07:27:26.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.246+0000 7fcdd2800640 1 -- 172.21.15.86:0/4011752265 learned_addr learned my addr 172.21.15.86:0/4011752265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:26.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.246+0000 7fcdcbe00640 1 --2- 172.21.15.86:0/4011752265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c3c990 0x561af9c4d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.247+0000 7fcdd3200640 1 --2- 172.21.15.86:0/4011752265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c44ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.247+0000 7fcdcbe00640 1 -- 172.21.15.86:0/4011752265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 msgr2=0x561af9c3ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:26.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.247+0000 7fcdcbe00640 1 --2- 172.21.15.86:0/4011752265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c3ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.247+0000 7fcdcbe00640 1 -- 172.21.15.86:0/4011752265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 msgr2=0x561af9c44ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:26.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.248+0000 7fcdcbe00640 1 --2- 172.21.15.86:0/4011752265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c44ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.248+0000 7fcdcbe00640 1 -- 172.21.15.86:0/4011752265 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af98b43f0 con 0x561af9c3c990 2024-04-17T07:27:26.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.248+0000 7fcdd3200640 1 --2- 172.21.15.86:0/4011752265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c44ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:27:26.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.248+0000 7fcdcbe00640 1 --2- 172.21.15.86:0/4011752265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c3c990 0x561af9c4d530 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x561af9c3cd90 tx=0x7fcdbc00e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d198ee32d81c5b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:26.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.249+0000 7fcdc2a00640 1 -- 172.21.15.86:0/4011752265 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc023070 con 0x561af9c3c990 2024-04-17T07:27:26.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.250+0000 7fcdc2a00640 1 -- 172.21.15.86:0/4011752265 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdbc01e400 con 0x561af9c3c990 2024-04-17T07:27:26.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.250+0000 7fcdc2a00640 1 -- 172.21.15.86:0/4011752265 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc012610 con 0x561af9c3c990 2024-04-17T07:27:26.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.251+0000 7fcddbff1740 1 -- 172.21.15.86:0/4011752265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c3c990 msgr2=0x561af9c4d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:26.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.251+0000 7fcddbff1740 1 --2- 172.21.15.86:0/4011752265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c3c990 0x561af9c4d530 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x561af9c3cd90 tx=0x7fcdbc00e000 comp rx=0 tx=0).stop 2024-04-17T07:27:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.251+0000 7fcddbff1740 1 -- 172.21.15.86:0/4011752265 shutdown_connections 2024-04-17T07:27:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.251+0000 7fcddbff1740 1 --2- 172.21.15.86:0/4011752265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c41b00 0x561af9c44ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.251+0000 7fcddbff1740 1 --2- 172.21.15.86:0/4011752265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c3c990 0x561af9c4d530 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.251+0000 7fcddbff1740 1 --2- 172.21.15.86:0/4011752265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c3ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.251+0000 7fcddbff1740 1 -- 172.21.15.86:0/4011752265 >> 172.21.15.86:0/4011752265 conn(0x561af9c37510 msgr2=0x561af9c3a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.252+0000 7fcddbff1740 1 -- 172.21.15.86:0/4011752265 shutdown_connections 2024-04-17T07:27:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.252+0000 7fcddbff1740 1 -- 172.21.15.86:0/4011752265 wait complete. 2024-04-17T07:27:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.252+0000 7fcddbff1740 1 -- start start 2024-04-17T07:27:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.253+0000 7fcddbff1740 1 Processor -- start 2024-04-17T07:27:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.253+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c4a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.253+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3c990 0x561af9c4aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.253+0000 7fcddbff1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c4baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.253+0000 7fcddbff1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb0010f80 con 0x561af9c3c990 2024-04-17T07:27:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.253+0000 7fcddbff1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb00108b0 con 0x561af9c41b00 2024-04-17T07:27:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.253+0000 7fcddbff1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb000dae0 con 0x561af9c3bfa0 2024-04-17T07:27:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.253+0000 7fcddbff1740 10 client.0 _ll_register_callbacks cb 0x7fcdcc009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.254+0000 7fcdd2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c4a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.254+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3c990 0x561af9c4aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.254+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3c990 0x561af9c4aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57338/0 (socket says 172.21.15.86:57338) 2024-04-17T07:27:26.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.254+0000 7fcdd2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c4a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33270/0 (socket says 172.21.15.86:33270) 2024-04-17T07:27:26.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.255+0000 7fcdd2800640 1 -- 172.21.15.86:0/1063355220 learned_addr learned my addr 172.21.15.86:0/1063355220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:26.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.256+0000 7fcdd3200640 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c4baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.256+0000 7fcdd2800640 1 -- 172.21.15.86:0/1063355220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 msgr2=0x561af9c4baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:26.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.256+0000 7fcdd2800640 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c4baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.256+0000 7fcdd2800640 1 -- 172.21.15.86:0/1063355220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3c990 msgr2=0x561af9c4aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:26.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.256+0000 7fcdd2800640 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3c990 0x561af9c4aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.256+0000 7fcdd2800640 1 -- 172.21.15.86:0/1063355220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af997e7d0 con 0x561af9c3bfa0 2024-04-17T07:27:26.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.257+0000 7fcdcbe00640 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3c990 0x561af9c4aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:27:26.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.257+0000 7fcdd3200640 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c4baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:27:26.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.258+0000 7fcdd2800640 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c4a5a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x561af9c3d1f0 tx=0x7fcdc4010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aa014851cc8a93e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:26.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.258+0000 7fcdb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:27:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.258+0000 7fcdb7e00640 1 -- 172.21.15.86:0/1063355220 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc401d790 con 0x561af9c3bfa0 2024-04-17T07:27:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.259+0000 7fcdb7e00640 1 -- 172.21.15.86:0/1063355220 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcdc4026470 con 0x561af9c3bfa0 2024-04-17T07:27:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.259+0000 7fcddbff1740 1 -- 172.21.15.86:0/1063355220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdbc01e400 con 0x561af9c3bfa0 2024-04-17T07:27:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.261+0000 7fcdb7e00640 1 -- 172.21.15.86:0/1063355220 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcdc402f050 con 0x561af9c3bfa0 2024-04-17T07:27:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.261+0000 7fcddbff1740 10 client.27577 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.261+0000 7fcddbff1740 1 -- 172.21.15.86:0/1063355220 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561af9c4d310 con 0x561af9c3bfa0 2024-04-17T07:27:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.261+0000 7fcdb7e00640 1 -- 172.21.15.86:0/1063355220 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdbc01e400 con 0x561af9c3bfa0 2024-04-17T07:27:26.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.264+0000 7fcdb7e00640 1 -- 172.21.15.86:0/1063355220 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcdc400f470 con 0x561af9c3bfa0 2024-04-17T07:27:26.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.265+0000 7fcddbff1740 10 client.27577 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:26.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.265+0000 7fcddbff1740 10 client.27577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:27:26.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.265+0000 7fcddbff1740 20 client.27577 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:26.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.265+0000 7fcddbff1740 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561af9cb81a0 0x561af9cba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:26.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.265+0000 7fcddbff1740 4 client.27577 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:27:26.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.265+0000 7fcddbff1740 1 -- 172.21.15.86:0/1063355220 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561af9ba98b0 con 0x561af9cb81a0 2024-04-17T07:27:26.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.266+0000 7fcdcbe00640 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561af9cb81a0 0x561af9cba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:26.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.268+0000 7fcdb7e00640 10 client.27577 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:26.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.268+0000 7fcdcbe00640 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561af9cb81a0 0x561af9cba5c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a55a80611be5275 server_cookie=68d1fc2840bd0f53 in_seq=0 out_seq=0 2024-04-17T07:27:26.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.268+0000 7fcdb7e00640 1 -- 172.21.15.86:0/1063355220 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561af9ba98b0 con 0x561af9cb81a0 2024-04-17T07:27:26.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.268+0000 7fcdb7e00640 10 client.27577 handle_command_reply: tid=0 2024-04-17T07:27:26.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.268+0000 7fcddbff1740 1 client.27577 shutdown 2024-04-17T07:27:26.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.271+0000 7fcddbff1740 1 -- 172.21.15.86:0/1063355220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 msgr2=0x561af9c4a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:26.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.271+0000 7fcddbff1740 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c4a5a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x561af9c3d1f0 tx=0x7fcdc4010800 comp rx=0 tx=0).stop 2024-04-17T07:27:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.271+0000 7fcddbff1740 1 -- 172.21.15.86:0/1063355220 shutdown_connections 2024-04-17T07:27:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.271+0000 7fcddbff1740 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561af9cb81a0 0x561af9cba5c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.272+0000 7fcddbff1740 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561af9c41b00 0x561af9c4baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.272+0000 7fcddbff1740 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561af9c3c990 0x561af9c4aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.272+0000 7fcddbff1740 1 --2- 172.21.15.86:0/1063355220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561af9c3bfa0 0x561af9c4a5a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.272+0000 7fcddbff1740 1 -- 172.21.15.86:0/1063355220 >> 172.21.15.86:0/1063355220 conn(0x561af9c37510 msgr2=0x561af9c3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.272+0000 7fcdb7e00640 0 client.27577 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:26.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.273+0000 7fcddbff1740 1 -- 172.21.15.86:0/1063355220 shutdown_connections 2024-04-17T07:27:26.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.273+0000 7fcddbff1740 1 -- 172.21.15.86:0/1063355220 wait complete. 2024-04-17T07:27:26.273 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:27:26.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.274+0000 7fcddbff1740 20 client.27577 trim_cache size 0 max 16384 2024-04-17T07:27:26.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.275+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcdb80a3c70 msgr2=0x7fcdb80a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:26.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.275+0000 7fcdd9400640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcdb80a3c70 0x7fcdb80a6130 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4075c30 tx=0x7fcdbc006c90 comp rx=0 tx=0).stop 2024-04-17T07:27:26.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.275+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdd410a380 msgr2=0x7fcdd407c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:26.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.275+0000 7fcdd9400640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdd410a380 0x7fcdd407c100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc036040 tx=0x7fcdcc033690 comp rx=0 tx=0).stop 2024-04-17T07:27:26.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.275+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 shutdown_connections 2024-04-17T07:27:26.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.275+0000 7fcdd9400640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcdb80a3c70 0x7fcdb80a6130 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.275+0000 7fcdd9400640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdd410a380 0x7fcdd407c100 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.275+0000 7fcdd9400640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd41099e0 0x7fcdd4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.275+0000 7fcdd9400640 1 --2- 172.21.15.86:0/1823607768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdd4109080 0x7fcdd4074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:26.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.275+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 >> 172.21.15.86:0/1823607768 conn(0x7fcdd406fe40 msgr2=0x7fcdd4071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:26.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.276+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 shutdown_connections 2024-04-17T07:27:26.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:26.276+0000 7fcdd9400640 1 -- 172.21.15.86:0/1823607768 wait complete. 2024-04-17T07:27:27.307 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:27:27.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:27:27.308 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:27:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:27 smithi098 ceph-mon[35418]: pgmap v832: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.7 MiB/s wr, 588 op/s 2024-04-17T07:27:27.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:27 smithi103 ceph-mon[35495]: pgmap v832: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.7 MiB/s wr, 588 op/s 2024-04-17T07:27:27.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:27 smithi086 ceph-mon[31733]: pgmap v832: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.7 MiB/s wr, 588 op/s 2024-04-17T07:27:27.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.637+0000 7f76e2a00640 1 -- 172.21.15.86:0/3360651121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d40094f0 msgr2=0x7f76d4009910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:27.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.637+0000 7f76e2a00640 1 --2- 172.21.15.86:0/3360651121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d40094f0 0x7f76d4009910 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009900 tx=0x7f76d002bc30 comp rx=0 tx=0).stop 2024-04-17T07:27:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.639+0000 7f76e2a00640 1 -- 172.21.15.86:0/3360651121 shutdown_connections 2024-04-17T07:27:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.639+0000 7f76e2a00640 1 --2- 172.21.15.86:0/3360651121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76d400b0d0 0x7f76d400b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.639+0000 7f76e2a00640 1 --2- 172.21.15.86:0/3360651121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76d400a730 0x7f76d400ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.639+0000 7f76e2a00640 1 --2- 172.21.15.86:0/3360651121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d40094f0 0x7f76d4009910 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.639+0000 7f76e2a00640 1 -- 172.21.15.86:0/3360651121 >> 172.21.15.86:0/3360651121 conn(0x7f76d4093520 msgr2=0x7f76d4095940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.640+0000 7f76e2a00640 1 -- 172.21.15.86:0/3360651121 shutdown_connections 2024-04-17T07:27:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.640+0000 7f76e2a00640 1 -- 172.21.15.86:0/3360651121 wait complete. 2024-04-17T07:27:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.641+0000 7f76e2a00640 1 Processor -- start 2024-04-17T07:27:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.642+0000 7f76e2a00640 1 -- start start 2024-04-17T07:27:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.643+0000 7f76e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d40094f0 0x7f76d41435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.643+0000 7f76e2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76d400a730 0x7f76d4143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.643+0000 7f76e2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76d400b0d0 0x7f76d414ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.643+0000 7f76e2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76d40a7eb0 con 0x7f76d40094f0 2024-04-17T07:27:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.643+0000 7f76e2a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76d40a7d30 con 0x7f76d400a730 2024-04-17T07:27:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.643+0000 7f76e2a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76d40a8030 con 0x7f76d400b0d0 2024-04-17T07:27:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.644+0000 7f76e2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76d400b0d0 0x7f76d414ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.644+0000 7f76e1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d40094f0 0x7f76d41435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.644+0000 7f76e2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76d400b0d0 0x7f76d414ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:57778/0 (socket says 172.21.15.86:57778) 2024-04-17T07:27:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.644+0000 7f76e2000640 1 -- 172.21.15.86:0/3295684449 learned_addr learned my addr 172.21.15.86:0/3295684449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.644+0000 7f76e0c00640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76d400a730 0x7f76d4143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.644+0000 7f76e2000640 1 -- 172.21.15.86:0/3295684449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76d400a730 msgr2=0x7f76d4143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.644+0000 7f76e2000640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76d400a730 0x7f76d4143b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.644+0000 7f76e2000640 1 -- 172.21.15.86:0/3295684449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d40094f0 msgr2=0x7f76d41435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.644+0000 7f76e2000640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d40094f0 0x7f76d41435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.644+0000 7f76e2000640 1 -- 172.21.15.86:0/3295684449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d414b1e0 con 0x7f76d400b0d0 2024-04-17T07:27:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.645+0000 7f76e2000640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76d400b0d0 0x7f76d414ab70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f76dc067ad0 tx=0x7f76dc079b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.645+0000 7f76da000640 1 -- 172.21.15.86:0/3295684449 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc069020 con 0x7f76d400b0d0 2024-04-17T07:27:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.645+0000 7f76e2a00640 1 -- 172.21.15.86:0/3295684449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d414b4d0 con 0x7f76d400b0d0 2024-04-17T07:27:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.645+0000 7f76da000640 1 -- 172.21.15.86:0/3295684449 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76dc0691c0 con 0x7f76d400b0d0 2024-04-17T07:27:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.645+0000 7f76da000640 1 -- 172.21.15.86:0/3295684449 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc07abc0 con 0x7f76d400b0d0 2024-04-17T07:27:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.645+0000 7f76e2a00640 1 -- 172.21.15.86:0/3295684449 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d414b9a0 con 0x7f76d400b0d0 2024-04-17T07:27:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.650+0000 7f76e6d83740 1 Processor -- start 2024-04-17T07:27:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.650+0000 7f76da000640 1 -- 172.21.15.86:0/3295684449 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f76dc081020 con 0x7f76d400b0d0 2024-04-17T07:27:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.651+0000 7f76e6d83740 1 -- start start 2024-04-17T07:27:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.651+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a10a0 0x55de7f6a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.651+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.651+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a6c00 0x55de7f6a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.651+0000 7f76e6d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de7f606cb0 con 0x55de7f6a1a90 2024-04-17T07:27:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.651+0000 7f76e6d83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55de7f5ad370 con 0x55de7f6a6c00 2024-04-17T07:27:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.651+0000 7f76e6d83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55de7f602c70 con 0x55de7f6a10a0 2024-04-17T07:27:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.652+0000 7f76e1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.652+0000 7f76e1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33880/0 (socket says 172.21.15.86:33880) 2024-04-17T07:27:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.652+0000 7f76e1600640 1 -- 172.21.15.86:0/1418195588 learned_addr learned my addr 172.21.15.86:0/1418195588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.652+0000 7f76e0c00640 1 --2- 172.21.15.86:0/1418195588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a10a0 0x55de7f6a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.652+0000 7f76e0c00640 1 --2- 172.21.15.86:0/1418195588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a6c00 0x55de7f6a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.652+0000 7f76e1600640 1 -- 172.21.15.86:0/1418195588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a10a0 msgr2=0x55de7f6a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.652+0000 7f76e1600640 1 --2- 172.21.15.86:0/1418195588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a10a0 0x55de7f6a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.652+0000 7f76e1600640 1 -- 172.21.15.86:0/1418195588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a6c00 msgr2=0x55de7f6a9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.652+0000 7f76e1600640 1 --2- 172.21.15.86:0/1418195588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a6c00 0x55de7f6a9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.652+0000 7f76e1600640 1 -- 172.21.15.86:0/1418195588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de7f5cc910 con 0x55de7f6a1a90 2024-04-17T07:27:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.653+0000 7f76e1600640 1 --2- 172.21.15.86:0/1418195588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6a6590 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f76d002c840 tx=0x7f76d0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4569140ed80ac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.653+0000 7f76da000640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76c40a3c80 0x7f76c40a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.653+0000 7f76da000640 1 -- 172.21.15.86:0/3295684449 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f76dc1340c0 con 0x7f76d400b0d0 2024-04-17T07:27:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.653+0000 7f76ce600640 1 -- 172.21.15.86:0/1418195588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0045020 con 0x55de7f6a1a90 2024-04-17T07:27:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.654+0000 7f76ce600640 1 -- 172.21.15.86:0/1418195588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76d002c9a0 con 0x55de7f6a1a90 2024-04-17T07:27:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.654+0000 7f76ce600640 1 -- 172.21.15.86:0/1418195588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0045020 con 0x55de7f6a1a90 2024-04-17T07:27:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.654+0000 7f76e0c00640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76c40a3c80 0x7f76c40a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.655+0000 7f76e6d83740 1 -- 172.21.15.86:0/1418195588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 msgr2=0x55de7f6a6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.655+0000 7f76e6d83740 1 --2- 172.21.15.86:0/1418195588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6a6590 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f76d002c840 tx=0x7f76d0009560 comp rx=0 tx=0).stop 2024-04-17T07:27:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.656+0000 7f76e0c00640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76c40a3c80 0x7f76c40a6140 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55de7f6a22f0 tx=0x7f76c8013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.656+0000 7f76e6d83740 1 -- 172.21.15.86:0/1418195588 shutdown_connections 2024-04-17T07:27:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.656+0000 7f76e6d83740 1 --2- 172.21.15.86:0/1418195588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a6c00 0x55de7f6a9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.656+0000 7f76e6d83740 1 --2- 172.21.15.86:0/1418195588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6a6590 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.656+0000 7f76e6d83740 1 --2- 172.21.15.86:0/1418195588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a10a0 0x55de7f6a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.656+0000 7f76e6d83740 1 -- 172.21.15.86:0/1418195588 >> 172.21.15.86:0/1418195588 conn(0x55de7f69c610 msgr2=0x55de7f69ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.656+0000 7f76e6d83740 1 -- 172.21.15.86:0/1418195588 shutdown_connections 2024-04-17T07:27:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.656+0000 7f76e6d83740 1 -- 172.21.15.86:0/1418195588 wait complete. 2024-04-17T07:27:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.657+0000 7f76e6d83740 1 -- start start 2024-04-17T07:27:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.657+0000 7f76e6d83740 1 Processor -- start 2024-04-17T07:27:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.658+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a10a0 0x55de7f6ab390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.658+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a1a90 0x55de7f6ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.658+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 0x55de7f6b2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.658+0000 7f76e6d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de7f6a9b50 con 0x55de7f6a10a0 2024-04-17T07:27:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.658+0000 7f76e6d83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55de7f5c0730 con 0x55de7f6a1a90 2024-04-17T07:27:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.658+0000 7f76e6d83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55de7f6a9cd0 con 0x55de7f6a6c00 2024-04-17T07:27:27.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.658+0000 7f76e6d83740 10 client.0 _ll_register_callbacks cb 0x55de7f5c2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.658+0000 7f76e1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a10a0 0x55de7f6ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.658+0000 7f76e0c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a1a90 0x55de7f6ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.659+0000 7f76e0c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a1a90 0x55de7f6ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:59302/0 (socket says 172.21.15.86:59302) 2024-04-17T07:27:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.659+0000 7f76e0c00640 1 -- 172.21.15.86:0/4020593554 learned_addr learned my addr 172.21.15.86:0/4020593554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:27.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.659+0000 7f76e2000640 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 0x55de7f6b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.659+0000 7f76e1600640 1 -- 172.21.15.86:0/4020593554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 msgr2=0x55de7f6b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.659+0000 7f76e1600640 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 0x55de7f6b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.659+0000 7f76e1600640 1 -- 172.21.15.86:0/4020593554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a1a90 msgr2=0x55de7f6ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.659+0000 7f76e1600640 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a1a90 0x55de7f6ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.659+0000 7f76e1600640 1 -- 172.21.15.86:0/4020593554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de7f3e17e0 con 0x55de7f6a10a0 2024-04-17T07:27:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.660+0000 7f76e1600640 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a10a0 0x55de7f6ab390 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x55de7f69d580 tx=0x7f76d0004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb524d362350b720 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.660+0000 7f76c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:27:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.660+0000 7f76c2a00640 1 -- 172.21.15.86:0/4020593554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0050070 con 0x55de7f6a10a0 2024-04-17T07:27:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.660+0000 7f76c2a00640 1 -- 172.21.15.86:0/4020593554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f76d0003a00 con 0x55de7f6a10a0 2024-04-17T07:27:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.660+0000 7f76e6d83740 1 -- 172.21.15.86:0/4020593554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76d002c9a0 con 0x55de7f6a10a0 2024-04-17T07:27:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.661+0000 7f76c2a00640 1 -- 172.21.15.86:0/4020593554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76d0059050 con 0x55de7f6a10a0 2024-04-17T07:27:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.661+0000 7f76e6d83740 10 client.18645 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.661+0000 7f76e6d83740 1 -- 172.21.15.86:0/4020593554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de7f441f70 con 0x55de7f6a10a0 2024-04-17T07:27:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.661+0000 7f76c2a00640 1 -- 172.21.15.86:0/4020593554 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76d002c9a0 con 0x55de7f6a10a0 2024-04-17T07:27:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.661+0000 7f76c2a00640 1 -- 172.21.15.86:0/4020593554 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f76d0053020 con 0x55de7f6a10a0 2024-04-17T07:27:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.662+0000 7f76e6d83740 10 client.18645 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.662+0000 7f76e6d83740 10 client.18645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:27:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.662+0000 7f76e6d83740 20 client.18645 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.662+0000 7f76e6d83740 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de7f6b72a0 0x55de7f6b9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.662+0000 7f76e6d83740 4 client.18645 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:27:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.662+0000 7f76e6d83740 1 -- 172.21.15.86:0/4020593554 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de7f6146d0 con 0x55de7f6b72a0 2024-04-17T07:27:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.663+0000 7f76e0c00640 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de7f6b72a0 0x55de7f6b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.665+0000 7f76e0c00640 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de7f6b72a0 0x55de7f6b9740 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921a75efc60248c9 server_cookie=3100b4e806c7b8a0 in_seq=0 out_seq=0 2024-04-17T07:27:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.665+0000 7f76c2a00640 10 client.18645 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.666+0000 7f76c2a00640 1 -- 172.21.15.86:0/4020593554 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de7f6146d0 con 0x55de7f6b72a0 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.666+0000 7f76c2a00640 10 client.18645 handle_command_reply: tid=0 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.666+0000 7f76e6d83740 1 client.18645 shutdown 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.666+0000 7f76e6d83740 1 -- 172.21.15.86:0/4020593554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a10a0 msgr2=0x55de7f6ab390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.666+0000 7f76e6d83740 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a10a0 0x55de7f6ab390 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x55de7f69d580 tx=0x7f76d0004250 comp rx=0 tx=0).stop 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.666+0000 7f76e6d83740 1 -- 172.21.15.86:0/4020593554 shutdown_connections 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.666+0000 7f76e6d83740 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de7f6b72a0 0x55de7f6b9740 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.666+0000 7f76e6d83740 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 0x55de7f6b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.667+0000 7f76e6d83740 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a1a90 0x55de7f6ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.667+0000 7f76e6d83740 1 --2- 172.21.15.86:0/4020593554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a10a0 0x55de7f6ab390 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.667+0000 7f76e6d83740 1 -- 172.21.15.86:0/4020593554 >> 172.21.15.86:0/4020593554 conn(0x55de7f69c610 msgr2=0x55de7f69b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.667+0000 7f76c2a00640 0 client.18645 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.667+0000 7f76e6d83740 1 -- 172.21.15.86:0/4020593554 shutdown_connections 2024-04-17T07:27:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.667+0000 7f76e6d83740 1 -- 172.21.15.86:0/4020593554 wait complete. 2024-04-17T07:27:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.667+0000 7f76e6d83740 20 client.18645 trim_cache size 0 max 16384 2024-04-17T07:27:27.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.713+0000 7f76e6d83740 1 Processor -- start 2024-04-17T07:27:27.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.714+0000 7f76e6d83740 1 -- start start 2024-04-17T07:27:27.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.714+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a1a90 0x55de7f6aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.714+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a6c00 0x55de7f6b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.714+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a6810 0x55de7f6aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.714+0000 7f76e6d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc00fc70 con 0x55de7f6a6c00 2024-04-17T07:27:27.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.714+0000 7f76e6d83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc0117b0 con 0x55de7f6a6810 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.714+0000 7f76e6d83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc010340 con 0x55de7f6a1a90 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.715+0000 7f76e1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a1a90 0x55de7f6aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.715+0000 7f76e1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a1a90 0x55de7f6aa9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:57808/0 (socket says 172.21.15.86:57808) 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.715+0000 7f76e1600640 1 -- 172.21.15.86:0/3689763150 learned_addr learned my addr 172.21.15.86:0/3689763150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.716+0000 7f76e2000640 1 --2- 172.21.15.86:0/3689763150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a6810 0x55de7f6aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.716+0000 7f76e0c00640 1 --2- 172.21.15.86:0/3689763150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a6c00 0x55de7f6b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.716+0000 7f76e1600640 1 -- 172.21.15.86:0/3689763150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a6810 msgr2=0x55de7f6aaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.716+0000 7f76e1600640 1 --2- 172.21.15.86:0/3689763150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a6810 0x55de7f6aaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.716+0000 7f76e1600640 1 -- 172.21.15.86:0/3689763150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a6c00 msgr2=0x55de7f6b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.716+0000 7f76e1600640 1 --2- 172.21.15.86:0/3689763150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a6c00 0x55de7f6b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.716+0000 7f76e1600640 1 -- 172.21.15.86:0/3689763150 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de7f2d9170 con 0x55de7f6a1a90 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.717+0000 7f76e1600640 1 --2- 172.21.15.86:0/3689763150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a1a90 0x55de7f6aa9e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f76d002c840 tx=0x7f76d002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2fc9c6d81150b88 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:27.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.718+0000 7f76cdc00640 1 -- 172.21.15.86:0/3689763150 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0050070 con 0x55de7f6a1a90 2024-04-17T07:27:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.718+0000 7f76cdc00640 1 -- 172.21.15.86:0/3689763150 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76d0003cb0 con 0x55de7f6a1a90 2024-04-17T07:27:27.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.718+0000 7f76cdc00640 1 -- 172.21.15.86:0/3689763150 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0045040 con 0x55de7f6a1a90 2024-04-17T07:27:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.720+0000 7f76e6d83740 1 -- 172.21.15.86:0/3689763150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a1a90 msgr2=0x55de7f6aa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.720+0000 7f76e6d83740 1 --2- 172.21.15.86:0/3689763150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a1a90 0x55de7f6aa9e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f76d002c840 tx=0x7f76d002bc30 comp rx=0 tx=0).stop 2024-04-17T07:27:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.720+0000 7f76e6d83740 1 -- 172.21.15.86:0/3689763150 shutdown_connections 2024-04-17T07:27:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.721+0000 7f76e6d83740 1 --2- 172.21.15.86:0/3689763150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6a6810 0x55de7f6aaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.721+0000 7f76e6d83740 1 --2- 172.21.15.86:0/3689763150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a6c00 0x55de7f6b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.721+0000 7f76e6d83740 1 --2- 172.21.15.86:0/3689763150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a1a90 0x55de7f6aa9e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.721+0000 7f76e6d83740 1 -- 172.21.15.86:0/3689763150 >> 172.21.15.86:0/3689763150 conn(0x55de7f69c610 msgr2=0x55de7f6a0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:27.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.722+0000 7f76e6d83740 1 -- 172.21.15.86:0/3689763150 shutdown_connections 2024-04-17T07:27:27.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.724+0000 7f76e6d83740 1 -- 172.21.15.86:0/3689763150 wait complete. 2024-04-17T07:27:27.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.724+0000 7f76e6d83740 1 -- start start 2024-04-17T07:27:27.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.725+0000 7f76e6d83740 1 Processor -- start 2024-04-17T07:27:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.726+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.726+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 0x55de7f6b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.726+0000 7f76e6d83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6b17b0 0x55de7f6b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.727+0000 7f76e6d83740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc0110e0 con 0x55de7f6a1a90 2024-04-17T07:27:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.727+0000 7f76e6d83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc010a10 con 0x55de7f6b17b0 2024-04-17T07:27:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.727+0000 7f76e6d83740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc00dc40 con 0x55de7f6a6c00 2024-04-17T07:27:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.727+0000 7f76e6d83740 10 client.0 _ll_register_callbacks cb 0x7f76c8009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.727+0000 7f76e2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6b17b0 0x55de7f6b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.727+0000 7f76e2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6b17b0 0x55de7f6b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:59308/0 (socket says 172.21.15.86:59308) 2024-04-17T07:27:27.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.727+0000 7f76e2000640 1 -- 172.21.15.86:0/1100144600 learned_addr learned my addr 172.21.15.86:0/1100144600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.727+0000 7f76e1600640 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.728+0000 7f76e2000640 1 -- 172.21.15.86:0/1100144600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 msgr2=0x55de7f6b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:27.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.728+0000 7f76e0c00640 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 0x55de7f6b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.728+0000 7f76e2000640 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 0x55de7f6b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.728+0000 7f76e2000640 1 -- 172.21.15.86:0/1100144600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 msgr2=0x55de7f6b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.728+0000 7f76e2000640 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.728+0000 7f76e2000640 1 -- 172.21.15.86:0/1100144600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de7f6b2170 con 0x55de7f6b17b0 2024-04-17T07:27:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.728+0000 7f76e1600640 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:27:27.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.729+0000 7f76e0c00640 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 0x55de7f6b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:27:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.731+0000 7f76e2000640 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6b17b0 0x55de7f6b1c30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0ffcc0 tx=0x7f76dc100780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02123197e8d45ca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:27.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.731+0000 7f76c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:27:27.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.732+0000 7f76c2a00640 1 -- 172.21.15.86:0/1100144600 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc118070 con 0x55de7f6b17b0 2024-04-17T07:27:27.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.732+0000 7f76c2a00640 1 -- 172.21.15.86:0/1100144600 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f76dc0fc630 con 0x55de7f6b17b0 2024-04-17T07:27:27.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.732+0000 7f76e6d83740 1 -- 172.21.15.86:0/1100144600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76d0003cb0 con 0x55de7f6b17b0 2024-04-17T07:27:27.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.733+0000 7f76c2a00640 1 -- 172.21.15.86:0/1100144600 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76dc0ffd90 con 0x55de7f6b17b0 2024-04-17T07:27:27.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.733+0000 7f76e6d83740 10 client.37526 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.733+0000 7f76e6d83740 1 -- 172.21.15.86:0/1100144600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de7f71e030 con 0x55de7f6b17b0 2024-04-17T07:27:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.734+0000 7f76c2a00640 1 -- 172.21.15.86:0/1100144600 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76dc120040 con 0x55de7f6b17b0 2024-04-17T07:27:27.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.734+0000 7f76c2a00640 1 -- 172.21.15.86:0/1100144600 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f76dc102040 con 0x55de7f6b17b0 2024-04-17T07:27:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.735+0000 7f76e6d83740 10 client.37526 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.735+0000 7f76e6d83740 10 client.37526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:27:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.735+0000 7f76e6d83740 20 client.37526 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.735+0000 7f76e6d83740 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de7f71ea30 0x55de7f720e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.735+0000 7f76e6d83740 4 client.37526 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:27:27.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.735+0000 7f76e6d83740 1 -- 172.21.15.86:0/1100144600 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f76d0003cb0 con 0x55de7f71ea30 2024-04-17T07:27:27.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.736+0000 7f76e1600640 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de7f71ea30 0x55de7f720e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.738+0000 7f76e1600640 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de7f71ea30 0x55de7f720e50 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0d0416a2dc5bbf server_cookie=9abbbe59ccaf7e47 in_seq=0 out_seq=0 2024-04-17T07:27:27.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.738+0000 7f76c2a00640 10 client.37526 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.743+0000 7f76c2a00640 1 -- 172.21.15.86:0/1100144600 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f76d0003cb0 con 0x55de7f71ea30 2024-04-17T07:27:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.743+0000 7f76c2a00640 10 client.37526 handle_command_reply: tid=0 2024-04-17T07:27:27.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.743+0000 7f76e6d83740 1 client.37526 shutdown 2024-04-17T07:27:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.744+0000 7f76e6d83740 1 -- 172.21.15.86:0/1100144600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6b17b0 msgr2=0x55de7f6b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:27.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.744+0000 7f76e6d83740 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6b17b0 0x55de7f6b1c30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76dc0ffcc0 tx=0x7f76dc100780 comp rx=0 tx=0).stop 2024-04-17T07:27:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.744+0000 7f76e6d83740 1 -- 172.21.15.86:0/1100144600 shutdown_connections 2024-04-17T07:27:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.744+0000 7f76e6d83740 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de7f71ea30 0x55de7f720e50 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.744+0000 7f76c2a00640 0 client.37526 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.744+0000 7f76e6d83740 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de7f6b17b0 0x55de7f6b1c30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.744+0000 7f76e6d83740 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de7f6a6c00 0x55de7f6b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.744+0000 7f76e6d83740 1 --2- 172.21.15.86:0/1100144600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de7f6a1a90 0x55de7f6b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.744+0000 7f76e6d83740 1 -- 172.21.15.86:0/1100144600 >> 172.21.15.86:0/1100144600 conn(0x55de7f69c610 msgr2=0x55de7f69b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.746+0000 7f76e6d83740 1 -- 172.21.15.86:0/1100144600 shutdown_connections 2024-04-17T07:27:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.746+0000 7f76e6d83740 1 -- 172.21.15.86:0/1100144600 wait complete. 2024-04-17T07:27:27.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.746+0000 7f76e6d83740 20 client.37526 trim_cache size 0 max 16384 2024-04-17T07:27:27.746 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:27:27.746 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:27:27.747 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "6540cc55-eb64-47fe-a53e-00c5db39951e", 2024-04-17T07:27:27.747 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:27:27.747 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:27:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.749+0000 7f76e2a00640 1 -- 172.21.15.86:0/3295684449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76c40a3c80 msgr2=0x7f76c40a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.749+0000 7f76e2a00640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76c40a3c80 0x7f76c40a6140 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55de7f6a22f0 tx=0x7f76c8013040 comp rx=0 tx=0).stop 2024-04-17T07:27:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.749+0000 7f76e2a00640 1 -- 172.21.15.86:0/3295684449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76d400b0d0 msgr2=0x7f76d414ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:27.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.749+0000 7f76e2a00640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76d400b0d0 0x7f76d414ab70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f76dc067ad0 tx=0x7f76dc079b00 comp rx=0 tx=0).stop 2024-04-17T07:27:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.749+0000 7f76e2a00640 1 -- 172.21.15.86:0/3295684449 shutdown_connections 2024-04-17T07:27:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.749+0000 7f76e2a00640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f76c40a3c80 0x7f76c40a6140 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.749+0000 7f76e2a00640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76d400b0d0 0x7f76d414ab70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.749+0000 7f76e2a00640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76d400a730 0x7f76d4143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.749+0000 7f76e2a00640 1 --2- 172.21.15.86:0/3295684449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76d40094f0 0x7f76d41435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.749+0000 7f76e2a00640 1 -- 172.21.15.86:0/3295684449 >> 172.21.15.86:0/3295684449 conn(0x7f76d4093520 msgr2=0x7f76d4094c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.750+0000 7f76e2a00640 1 -- 172.21.15.86:0/3295684449 shutdown_connections 2024-04-17T07:27:27.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:27.750+0000 7f76e2a00640 1 -- 172.21.15.86:0/3295684449 wait complete. 2024-04-17T07:27:27.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:27:27.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:27:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.111+0000 7f3e2d200640 1 -- 172.21.15.86:0/1732464906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e2810e580 msgr2=0x7f3e28110970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.111+0000 7f3e2d200640 1 --2- 172.21.15.86:0/1732464906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e2810e580 0x7f3e28110970 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009cd0 tx=0x7f3e2002c0e0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.113+0000 7f3e2d200640 1 -- 172.21.15.86:0/1732464906 shutdown_connections 2024-04-17T07:27:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.113+0000 7f3e2d200640 1 --2- 172.21.15.86:0/1732464906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e2810e580 0x7f3e28110970 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.113+0000 7f3e2d200640 1 --2- 172.21.15.86:0/1732464906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e280749c0 0x7f3e28074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.113+0000 7f3e2d200640 1 --2- 172.21.15.86:0/1732464906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e28075fa0 0x7f3e280743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.113+0000 7f3e2d200640 1 -- 172.21.15.86:0/1732464906 >> 172.21.15.86:0/1732464906 conn(0x7f3e280fd540 msgr2=0x7f3e280ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.114+0000 7f3e2d200640 1 -- 172.21.15.86:0/1732464906 shutdown_connections 2024-04-17T07:27:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.115+0000 7f3e2d200640 1 -- 172.21.15.86:0/1732464906 wait complete. 2024-04-17T07:27:28.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.116+0000 7f3e2d200640 1 Processor -- start 2024-04-17T07:27:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.117+0000 7f3e2d200640 1 -- start start 2024-04-17T07:27:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.117+0000 7f3e2d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e280749c0 0x7f3e2819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.117+0000 7f3e2d200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e28075fa0 0x7f3e2819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.117+0000 7f3e2d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e2810e580 0x7f3e281a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.118+0000 7f3e2d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e281140f0 con 0x7f3e280749c0 2024-04-17T07:27:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.118+0000 7f3e2d200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e28113f70 con 0x7f3e28075fa0 2024-04-17T07:27:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.118+0000 7f3e2d200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e28114270 con 0x7f3e2810e580 2024-04-17T07:27:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.118+0000 7f3e25400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e28075fa0 0x7f3e2819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.118+0000 7f3e25400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e28075fa0 0x7f3e2819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:59316/0 (socket says 172.21.15.86:59316) 2024-04-17T07:27:28.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.118+0000 7f3e25400640 1 -- 172.21.15.86:0/3903307001 learned_addr learned my addr 172.21.15.86:0/3903307001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:28.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.118+0000 7f3e26800640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e2810e580 0x7f3e281a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.119+0000 7f3e26800640 1 -- 172.21.15.86:0/3903307001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e28075fa0 msgr2=0x7f3e2819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:28.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.119+0000 7f3e26800640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e28075fa0 0x7f3e2819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.119+0000 7f3e26800640 1 -- 172.21.15.86:0/3903307001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e280749c0 msgr2=0x7f3e2819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.119+0000 7f3e26800640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e280749c0 0x7f3e2819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.119+0000 7f3e26800640 1 -- 172.21.15.86:0/3903307001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e281a6390 con 0x7f3e2810e580 2024-04-17T07:27:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.119+0000 7f3e26800640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e2810e580 0x7f3e281a5d20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009bf0 tx=0x7f3e20003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.120+0000 7f3e1ea00640 1 -- 172.21.15.86:0/3903307001 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20039040 con 0x7f3e2810e580 2024-04-17T07:27:28.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.120+0000 7f3e2d200640 1 -- 172.21.15.86:0/3903307001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e281a6620 con 0x7f3e2810e580 2024-04-17T07:27:28.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.120+0000 7f3e2d200640 1 -- 172.21.15.86:0/3903307001 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e281a6b30 con 0x7f3e2810e580 2024-04-17T07:27:28.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.120+0000 7f3e1ea00640 1 -- 172.21.15.86:0/3903307001 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e20003d10 con 0x7f3e2810e580 2024-04-17T07:27:28.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.120+0000 7f3e1ea00640 1 -- 172.21.15.86:0/3903307001 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20032ac0 con 0x7f3e2810e580 2024-04-17T07:27:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.125+0000 7f3e30022740 1 Processor -- start 2024-04-17T07:27:28.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.127+0000 7f3e1ea00640 1 -- 172.21.15.86:0/3903307001 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3e20046030 con 0x7f3e2810e580 2024-04-17T07:27:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.129+0000 7f3e1ea00640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e0c0a3c30 0x7f3e0c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.129+0000 7f3e25400640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e0c0a3c30 0x7f3e0c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.130+0000 7f3e1ea00640 1 -- 172.21.15.86:0/3903307001 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3e200f2360 con 0x7f3e2810e580 2024-04-17T07:27:28.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.130+0000 7f3e30022740 1 -- start start 2024-04-17T07:27:28.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.131+0000 7f3e30022740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e458190 0x564a6e4585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.131+0000 7f3e30022740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e45d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.131+0000 7f3e30022740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e45dcf0 0x564a6e460180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.131+0000 7f3e30022740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a6e3bf120 con 0x564a6e458b80 2024-04-17T07:27:28.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.131+0000 7f3e30022740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564a6e365340 con 0x564a6e458190 2024-04-17T07:27:28.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.131+0000 7f3e30022740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a6e3bb0e0 con 0x564a6e45dcf0 2024-04-17T07:27:28.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.131+0000 7f3e25400640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e0c0a3c30 0x7f3e0c0a60f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3e2819fc80 tx=0x7f3e1400b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.132+0000 7f3e25400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e45d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.132+0000 7f3e25400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e45d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33916/0 (socket says 172.21.15.86:33916) 2024-04-17T07:27:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.132+0000 7f3e25400640 1 -- 172.21.15.86:0/1518675394 learned_addr learned my addr 172.21.15.86:0/1518675394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.132+0000 7f3e26800640 1 --2- 172.21.15.86:0/1518675394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e45dcf0 0x564a6e460180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.133+0000 7f3e25400640 1 -- 172.21.15.86:0/1518675394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e45dcf0 msgr2=0x564a6e460180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.133+0000 7f3e25400640 1 --2- 172.21.15.86:0/1518675394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e45dcf0 0x564a6e460180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.133+0000 7f3e25400640 1 -- 172.21.15.86:0/1518675394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e458190 msgr2=0x564a6e4585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.133+0000 7f3e25400640 1 --2- 172.21.15.86:0/1518675394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e458190 0x564a6e4585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.133+0000 7f3e25400640 1 -- 172.21.15.86:0/1518675394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a6e19b7d0 con 0x564a6e458b80 2024-04-17T07:27:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.133+0000 7f3e25400640 1 --2- 172.21.15.86:0/1518675394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e45d680 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3e14011490 tx=0x7f3e14013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bf3672e025f4a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.134+0000 7f3e13400640 1 -- 172.21.15.86:0/1518675394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14005220 con 0x564a6e458b80 2024-04-17T07:27:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.134+0000 7f3e13400640 1 -- 172.21.15.86:0/1518675394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e14004c50 con 0x564a6e458b80 2024-04-17T07:27:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.134+0000 7f3e13400640 1 -- 172.21.15.86:0/1518675394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1401e970 con 0x564a6e458b80 2024-04-17T07:27:28.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.135+0000 7f3e30022740 1 -- 172.21.15.86:0/1518675394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 msgr2=0x564a6e45d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:28.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.135+0000 7f3e30022740 1 --2- 172.21.15.86:0/1518675394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e45d680 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3e14011490 tx=0x7f3e14013000 comp rx=0 tx=0).stop 2024-04-17T07:27:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.136+0000 7f3e30022740 1 -- 172.21.15.86:0/1518675394 shutdown_connections 2024-04-17T07:27:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.136+0000 7f3e30022740 1 --2- 172.21.15.86:0/1518675394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e45dcf0 0x564a6e460180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.136+0000 7f3e30022740 1 --2- 172.21.15.86:0/1518675394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e45d680 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.136+0000 7f3e30022740 1 --2- 172.21.15.86:0/1518675394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e458190 0x564a6e4585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.136+0000 7f3e30022740 1 -- 172.21.15.86:0/1518675394 >> 172.21.15.86:0/1518675394 conn(0x564a6e453700 msgr2=0x564a6e453b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:28.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.137+0000 7f3e30022740 1 -- 172.21.15.86:0/1518675394 shutdown_connections 2024-04-17T07:27:28.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.137+0000 7f3e30022740 1 -- 172.21.15.86:0/1518675394 wait complete. 2024-04-17T07:27:28.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.138+0000 7f3e30022740 1 -- start start 2024-04-17T07:27:28.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.138+0000 7f3e30022740 1 Processor -- start 2024-04-17T07:27:28.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.139+0000 7f3e30022740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458190 0x564a6e4736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.139+0000 7f3e30022740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458b80 0x564a6e473c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.139+0000 7f3e30022740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 0x564a6e47ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.140+0000 7f3e30022740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a6e471cb0 con 0x564a6e458190 2024-04-17T07:27:28.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.140+0000 7f3e30022740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564a6e378710 con 0x564a6e45dcf0 2024-04-17T07:27:28.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.140+0000 7f3e25400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458b80 0x564a6e473c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.140+0000 7f3e26800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 0x564a6e47ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.140+0000 7f3e26800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 0x564a6e47ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:59330/0 (socket says 172.21.15.86:59330) 2024-04-17T07:27:28.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.140+0000 7f3e26800640 1 -- 172.21.15.86:0/222912618 learned_addr learned my addr 172.21.15.86:0/222912618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:28.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.141+0000 7f3e30022740 1 -- 172.21.15.86:0/222912618 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a6e471e30 con 0x564a6e458b80 2024-04-17T07:27:28.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.141+0000 7f3e30022740 10 client.0 _ll_register_callbacks cb 0x564a6e373af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:28.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.141+0000 7f3e26800640 1 -- 172.21.15.86:0/222912618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458b80 msgr2=0x564a6e473c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:28.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.141+0000 7f3e26800640 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458b80 0x564a6e473c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.141+0000 7f3e26800640 1 -- 172.21.15.86:0/222912618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458190 msgr2=0x564a6e4736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:28.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.141+0000 7f3e26800640 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458190 0x564a6e4736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.141+0000 7f3e26800640 1 -- 172.21.15.86:0/222912618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a6e0d13f0 con 0x564a6e45dcf0 2024-04-17T07:27:28.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.142+0000 7f3e25400640 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458b80 0x564a6e473c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:27:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.142+0000 7f3e26800640 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 0x564a6e47ac40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e20037040 tx=0x7f3e200be280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c2b88c88b95daa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.142+0000 7f3e07e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:27:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.142+0000 7f3e07e00640 1 -- 172.21.15.86:0/222912618 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e200d7060 con 0x564a6e45dcf0 2024-04-17T07:27:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.142+0000 7f3e07e00640 1 -- 172.21.15.86:0/222912618 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3e200bc6f0 con 0x564a6e45dcf0 2024-04-17T07:27:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.142+0000 7f3e30022740 1 -- 172.21.15.86:0/222912618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e14004c50 con 0x564a6e45dcf0 2024-04-17T07:27:28.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.143+0000 7f3e07e00640 1 -- 172.21.15.86:0/222912618 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e200be3f0 con 0x564a6e45dcf0 2024-04-17T07:27:28.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.143+0000 7f3e30022740 10 client.37535 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:28.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.143+0000 7f3e30022740 1 -- 172.21.15.86:0/222912618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564a6e0910f0 con 0x564a6e45dcf0 2024-04-17T07:27:28.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.144+0000 7f3e07e00640 1 -- 172.21.15.86:0/222912618 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e14004c50 con 0x564a6e45dcf0 2024-04-17T07:27:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.146+0000 7f3e07e00640 1 -- 172.21.15.86:0/222912618 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3e2003c030 con 0x564a6e45dcf0 2024-04-17T07:27:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.147+0000 7f3e30022740 10 client.37535 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.147+0000 7f3e30022740 10 client.37535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:27:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.147+0000 7f3e30022740 20 client.37535 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.147+0000 7f3e30022740 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564a6e47f450 0x564a6e4818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.147+0000 7f3e30022740 4 client.37535 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:27:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.147+0000 7f3e30022740 1 -- 172.21.15.86:0/222912618 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a6e3ccb40 con 0x564a6e47f450 2024-04-17T07:27:28.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.147+0000 7f3e25e00640 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564a6e47f450 0x564a6e4818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.149+0000 7f3e25e00640 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564a6e47f450 0x564a6e4818c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a400cbc3d2f2a server_cookie=26cc87646067fa00 in_seq=0 out_seq=0 2024-04-17T07:27:28.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.149+0000 7f3e07e00640 10 client.37535 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.150+0000 7f3e07e00640 1 -- 172.21.15.86:0/222912618 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a6e3ccb40 con 0x564a6e47f450 2024-04-17T07:27:28.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.150+0000 7f3e07e00640 10 client.37535 handle_command_reply: tid=0 2024-04-17T07:27:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.152+0000 7f3e30022740 1 client.37535 shutdown 2024-04-17T07:27:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.154+0000 7f3e30022740 1 -- 172.21.15.86:0/222912618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 msgr2=0x564a6e47ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:28.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.154+0000 7f3e30022740 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 0x564a6e47ac40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e20037040 tx=0x7f3e200be280 comp rx=0 tx=0).stop 2024-04-17T07:27:28.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.154+0000 7f3e30022740 1 -- 172.21.15.86:0/222912618 shutdown_connections 2024-04-17T07:27:28.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.154+0000 7f3e30022740 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564a6e47f450 0x564a6e4818c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.154+0000 7f3e30022740 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 0x564a6e47ac40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.154+0000 7f3e30022740 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458b80 0x564a6e473c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.154+0000 7f3e30022740 1 --2- 172.21.15.86:0/222912618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458190 0x564a6e4736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.154+0000 7f3e30022740 1 -- 172.21.15.86:0/222912618 >> 172.21.15.86:0/222912618 conn(0x564a6e453700 msgr2=0x564a6e452890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:28.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.155+0000 7f3e07e00640 0 client.37535 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.155+0000 7f3e30022740 1 -- 172.21.15.86:0/222912618 shutdown_connections 2024-04-17T07:27:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.156+0000 7f3e30022740 1 -- 172.21.15.86:0/222912618 wait complete. 2024-04-17T07:27:28.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.156+0000 7f3e30022740 20 client.37535 trim_cache size 0 max 16384 2024-04-17T07:27:28.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.192+0000 7f3e30022740 1 Processor -- start 2024-04-17T07:27:28.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.192+0000 7f3e30022740 1 -- start start 2024-04-17T07:27:28.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.192+0000 7f3e30022740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458190 0x564a6e452000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.192+0000 7f3e30022740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e458b80 0x564a6e47ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.192+0000 7f3e30022740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e45dcf0 0x564a6e452540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.192+0000 7f3e30022740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e281a8780 con 0x564a6e45dcf0 2024-04-17T07:27:28.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.192+0000 7f3e30022740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e281aa2c0 con 0x564a6e458b80 2024-04-17T07:27:28.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.192+0000 7f3e30022740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e281a8e50 con 0x564a6e458190 2024-04-17T07:27:28.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.193+0000 7f3e26800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e45dcf0 0x564a6e452540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.193+0000 7f3e26800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e45dcf0 0x564a6e452540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33944/0 (socket says 172.21.15.86:33944) 2024-04-17T07:27:28.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.193+0000 7f3e26800640 1 -- 172.21.15.86:0/2220536157 learned_addr learned my addr 172.21.15.86:0/2220536157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:28.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.193+0000 7f3e25e00640 1 --2- 172.21.15.86:0/2220536157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458190 0x564a6e452000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.193+0000 7f3e25400640 1 --2- 172.21.15.86:0/2220536157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e458b80 0x564a6e47ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.194+0000 7f3e25400640 1 -- 172.21.15.86:0/2220536157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458190 msgr2=0x564a6e452000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:28.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.194+0000 7f3e25400640 1 --2- 172.21.15.86:0/2220536157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458190 0x564a6e452000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.194+0000 7f3e25400640 1 -- 172.21.15.86:0/2220536157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e45dcf0 msgr2=0x564a6e452540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:28.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.194+0000 7f3e25400640 1 --2- 172.21.15.86:0/2220536157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e45dcf0 0x564a6e452540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.194+0000 7f3e25400640 1 -- 172.21.15.86:0/2220536157 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a6e454170 con 0x564a6e458b80 2024-04-17T07:27:28.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.194+0000 7f3e26800640 1 --2- 172.21.15.86:0/2220536157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e45dcf0 0x564a6e452540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:27:28.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.195+0000 7f3e25400640 1 --2- 172.21.15.86:0/2220536157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e458b80 0x564a6e47ac00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x564a6e458f80 tx=0x7f3e14005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c8951bcc4871f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:28.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.195+0000 7f3e12a00640 1 -- 172.21.15.86:0/2220536157 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14005310 con 0x564a6e458b80 2024-04-17T07:27:28.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.195+0000 7f3e12a00640 1 -- 172.21.15.86:0/2220536157 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e140054b0 con 0x564a6e458b80 2024-04-17T07:27:28.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.195+0000 7f3e12a00640 1 -- 172.21.15.86:0/2220536157 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14005310 con 0x564a6e458b80 2024-04-17T07:27:28.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.197+0000 7f3e30022740 1 -- 172.21.15.86:0/2220536157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e458b80 msgr2=0x564a6e47ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:28.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.197+0000 7f3e30022740 1 --2- 172.21.15.86:0/2220536157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e458b80 0x564a6e47ac00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x564a6e458f80 tx=0x7f3e14005fa0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.197+0000 7f3e30022740 1 -- 172.21.15.86:0/2220536157 shutdown_connections 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.197+0000 7f3e30022740 1 --2- 172.21.15.86:0/2220536157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e45dcf0 0x564a6e452540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.197+0000 7f3e30022740 1 --2- 172.21.15.86:0/2220536157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e458b80 0x564a6e47ac00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.197+0000 7f3e30022740 1 --2- 172.21.15.86:0/2220536157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458190 0x564a6e452000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.197+0000 7f3e30022740 1 -- 172.21.15.86:0/2220536157 >> 172.21.15.86:0/2220536157 conn(0x564a6e453700 msgr2=0x564a6e4736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.197+0000 7f3e30022740 1 -- 172.21.15.86:0/2220536157 shutdown_connections 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.197+0000 7f3e30022740 1 -- 172.21.15.86:0/2220536157 wait complete. 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.198+0000 7f3e30022740 1 -- start start 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.198+0000 7f3e30022740 1 Processor -- start 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.198+0000 7f3e30022740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458190 0x564a6e4786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.198+0000 7f3e30022740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e478be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.198+0000 7f3e30022740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 0x564a6e479bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.199+0000 7f3e30022740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e281a9bf0 con 0x564a6e458b80 2024-04-17T07:27:28.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.199+0000 7f3e30022740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e281a9520 con 0x564a6e45dcf0 2024-04-17T07:27:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.199+0000 7f3e30022740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e28073b00 con 0x564a6e458190 2024-04-17T07:27:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.199+0000 7f3e30022740 10 client.0 _ll_register_callbacks cb 0x7f3e200baa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.199+0000 7f3e25400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e478be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.199+0000 7f3e25400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e478be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33954/0 (socket says 172.21.15.86:33954) 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.199+0000 7f3e25400640 1 -- 172.21.15.86:0/2193925259 learned_addr learned my addr 172.21.15.86:0/2193925259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.200+0000 7f3e26800640 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 0x564a6e479bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.200+0000 7f3e25400640 1 -- 172.21.15.86:0/2193925259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458190 msgr2=0x564a6e4786a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.200+0000 7f3e25400640 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458190 0x564a6e4786a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.200+0000 7f3e25400640 1 -- 172.21.15.86:0/2193925259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 msgr2=0x564a6e479bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.200+0000 7f3e25400640 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 0x564a6e479bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.200+0000 7f3e25400640 1 -- 172.21.15.86:0/2193925259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a6e47a1e0 con 0x564a6e458b80 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.201+0000 7f3e25400640 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e478be0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3e14011460 tx=0x7f3e14004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf771cde8c60f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.201+0000 7f3e07e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.202+0000 7f3e07e00640 1 -- 172.21.15.86:0/2193925259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1401e970 con 0x564a6e458b80 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.202+0000 7f3e07e00640 1 -- 172.21.15.86:0/2193925259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3e1401eb10 con 0x564a6e458b80 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.202+0000 7f3e30022740 1 -- 172.21.15.86:0/2193925259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e140054b0 con 0x564a6e458b80 2024-04-17T07:27:28.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.202+0000 7f3e07e00640 1 -- 172.21.15.86:0/2193925259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e1400bd10 con 0x564a6e458b80 2024-04-17T07:27:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.202+0000 7f3e30022740 10 client.18660 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.202+0000 7f3e30022740 1 -- 172.21.15.86:0/2193925259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564a6e4e5db0 con 0x564a6e458b80 2024-04-17T07:27:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.203+0000 7f3e07e00640 1 -- 172.21.15.86:0/2193925259 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e140054b0 con 0x564a6e458b80 2024-04-17T07:27:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.203+0000 7f3e07e00640 1 -- 172.21.15.86:0/2193925259 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3e14039b10 con 0x564a6e458b80 2024-04-17T07:27:28.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.203+0000 7f3e30022740 10 client.18660 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.203+0000 7f3e30022740 10 client.18660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:27:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.203+0000 7f3e30022740 20 client.18660 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.203+0000 7f3e30022740 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564a6e4e64f0 0x564a6e4e8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.203+0000 7f3e30022740 4 client.18660 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:27:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.203+0000 7f3e30022740 1 -- 172.21.15.86:0/2193925259 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a6e3c69b0 con 0x564a6e4e64f0 2024-04-17T07:27:28.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.205+0000 7f3e25e00640 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564a6e4e64f0 0x564a6e4e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.206+0000 7f3e25e00640 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564a6e4e64f0 0x564a6e4e8910 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa43ee2c1a877136 server_cookie=94dab2f270cc35d4 in_seq=0 out_seq=0 2024-04-17T07:27:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.206+0000 7f3e07e00640 10 client.18660 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.207+0000 7f3e07e00640 1 -- 172.21.15.86:0/2193925259 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564a6e3c69b0 con 0x564a6e4e64f0 2024-04-17T07:27:28.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.207+0000 7f3e07e00640 10 client.18660 handle_command_reply: tid=0 2024-04-17T07:27:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.207+0000 7f3e30022740 1 client.18660 shutdown 2024-04-17T07:27:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.207+0000 7f3e30022740 1 -- 172.21.15.86:0/2193925259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 msgr2=0x564a6e478be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:28.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.207+0000 7f3e30022740 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e478be0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3e14011460 tx=0x7f3e14004930 comp rx=0 tx=0).stop 2024-04-17T07:27:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.208+0000 7f3e30022740 1 -- 172.21.15.86:0/2193925259 shutdown_connections 2024-04-17T07:27:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.208+0000 7f3e30022740 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564a6e4e64f0 0x564a6e4e8910 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.208+0000 7f3e30022740 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564a6e45dcf0 0x564a6e479bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.208+0000 7f3e30022740 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e458b80 0x564a6e478be0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.208+0000 7f3e30022740 1 --2- 172.21.15.86:0/2193925259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a6e458190 0x564a6e4786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.208+0000 7f3e30022740 1 -- 172.21.15.86:0/2193925259 >> 172.21.15.86:0/2193925259 conn(0x564a6e453700 msgr2=0x564a6e452000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.208+0000 7f3e07e00640 0 client.18660 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.208+0000 7f3e30022740 1 -- 172.21.15.86:0/2193925259 shutdown_connections 2024-04-17T07:27:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.208+0000 7f3e30022740 1 -- 172.21.15.86:0/2193925259 wait complete. 2024-04-17T07:27:28.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.208+0000 7f3e30022740 20 client.18660 trim_cache size 0 max 16384 2024-04-17T07:27:28.209 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:27:28.209 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:27:28.209 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:27:28.209 INFO:teuthology.orchestra.run.smithi086.stdout: "6540cc55-eb64-47fe-a53e-00c5db39951e": { 2024-04-17T07:27:28.209 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:27:28.209 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "6540cc55-eb64-47fe-a53e-00c5db39951e", 2024-04-17T07:27:28.209 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:27:28.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:27:28.209 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:27:28.209 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:27:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.209+0000 7f3e2d200640 1 -- 172.21.15.86:0/3903307001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e0c0a3c30 msgr2=0x7f3e0c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.209+0000 7f3e2d200640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e0c0a3c30 0x7f3e0c0a60f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3e2819fc80 tx=0x7f3e1400b480 comp rx=0 tx=0).stop 2024-04-17T07:27:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.209+0000 7f3e2d200640 1 -- 172.21.15.86:0/3903307001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e2810e580 msgr2=0x7f3e281a5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.209+0000 7f3e2d200640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e2810e580 0x7f3e281a5d20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009bf0 tx=0x7f3e20003ba0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.209+0000 7f3e2d200640 1 -- 172.21.15.86:0/3903307001 shutdown_connections 2024-04-17T07:27:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.209+0000 7f3e2d200640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3e0c0a3c30 0x7f3e0c0a60f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.209+0000 7f3e2d200640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e2810e580 0x7f3e281a5d20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.209+0000 7f3e2d200640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e28075fa0 0x7f3e2819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.209+0000 7f3e2d200640 1 --2- 172.21.15.86:0/3903307001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e280749c0 0x7f3e2819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.209+0000 7f3e2d200640 1 -- 172.21.15.86:0/3903307001 >> 172.21.15.86:0/3903307001 conn(0x7f3e280fd540 msgr2=0x7f3e28101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.210+0000 7f3e2d200640 1 -- 172.21.15.86:0/3903307001 shutdown_connections 2024-04-17T07:27:28.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:28.210+0000 7f3e2d200640 1 -- 172.21.15.86:0/3903307001 wait complete. 2024-04-17T07:27:28.247 INFO:tasks.cephfs.filesystem:scrub status for tag:6540cc55-eb64-47fe-a53e-00c5db39951e - {'path': '/', 'tag': '6540cc55-eb64-47fe-a53e-00c5db39951e', 'options': 'recursive,force'} 2024-04-17T07:27:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:28 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:27:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:28 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:27:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:28 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:27:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:28 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:27:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:28 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:27:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:28 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:27:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:28 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:27:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:28 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:27:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:28 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:27:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:28 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:27:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:28 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:27:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:28 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:27:29.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 485.0 s, 474.9 tps, lat 167.925 ms stddev 58.030 2024-04-17T07:27:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:29 smithi098 ceph-mon[35418]: pgmap v833: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 625 op/s 2024-04-17T07:27:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:29 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:27:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:29 smithi103 ceph-mon[35495]: pgmap v833: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 625 op/s 2024-04-17T07:27:29.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:29 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:27:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:29 smithi086 ceph-mon[31733]: pgmap v833: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 625 op/s 2024-04-17T07:27:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:29 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:27:31.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:31 smithi098 ceph-mon[35418]: pgmap v834: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.8 MiB/s wr, 582 op/s 2024-04-17T07:27:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:31 smithi103 ceph-mon[35495]: pgmap v834: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.8 MiB/s wr, 582 op/s 2024-04-17T07:27:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:31 smithi086 ceph-mon[31733]: pgmap v834: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 8.8 MiB/s wr, 582 op/s 2024-04-17T07:27:33.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:33 smithi098 ceph-mon[35418]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.4 MiB/s wr, 630 op/s 2024-04-17T07:27:33.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:33 smithi103 ceph-mon[35495]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.4 MiB/s wr, 630 op/s 2024-04-17T07:27:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:33 smithi086 ceph-mon[31733]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.4 MiB/s wr, 630 op/s 2024-04-17T07:27:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 490.0 s, 749.9 tps, lat 160.146 ms stddev 200.361 2024-04-17T07:27:34.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:34 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:27:34.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:34 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:27:34.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:34 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:27:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:35 smithi103 ceph-mon[35495]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 9.6 MiB/s wr, 594 op/s 2024-04-17T07:27:35.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:35 smithi086 ceph-mon[31733]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 9.6 MiB/s wr, 594 op/s 2024-04-17T07:27:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:35 smithi098 ceph-mon[35418]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 9.6 MiB/s wr, 594 op/s 2024-04-17T07:27:36.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:27:36.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:27:36.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:27:36.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:27:36.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:27:36.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:27:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:27:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:27:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:27:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:37 smithi103 ceph-mon[35495]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.9 MiB/s wr, 480 op/s 2024-04-17T07:27:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:37 smithi086 ceph-mon[31733]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.9 MiB/s wr, 480 op/s 2024-04-17T07:27:37.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:37 smithi098 ceph-mon[35418]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.9 MiB/s wr, 480 op/s 2024-04-17T07:27:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 495.0 s, 581.4 tps, lat 172.749 ms stddev 292.221 2024-04-17T07:27:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:39 smithi103 ceph-mon[35495]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.7 MiB/s wr, 576 op/s 2024-04-17T07:27:39.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:39 smithi086 ceph-mon[31733]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.7 MiB/s wr, 576 op/s 2024-04-17T07:27:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:39 smithi098 ceph-mon[35418]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.7 MiB/s wr, 576 op/s 2024-04-17T07:27:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:41 smithi086 ceph-mon[31733]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.2 MiB/s wr, 589 op/s 2024-04-17T07:27:41.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:41 smithi098 ceph-mon[35418]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.2 MiB/s wr, 589 op/s 2024-04-17T07:27:41.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:41 smithi103 ceph-mon[35495]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.2 MiB/s wr, 589 op/s 2024-04-17T07:27:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:43 smithi086 ceph-mon[31733]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.0 MiB/s wr, 626 op/s 2024-04-17T07:27:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:43 smithi098 ceph-mon[35418]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.0 MiB/s wr, 626 op/s 2024-04-17T07:27:43.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:43 smithi103 ceph-mon[35495]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.0 MiB/s wr, 626 op/s 2024-04-17T07:27:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 500.0 s, 676.4 tps, lat 147.616 ms stddev 42.998 2024-04-17T07:27:45.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:45 smithi086 ceph-mon[31733]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.9 MiB/s wr, 623 op/s 2024-04-17T07:27:45.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:45 smithi098 ceph-mon[35418]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.9 MiB/s wr, 623 op/s 2024-04-17T07:27:45.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:45 smithi103 ceph-mon[35495]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.9 MiB/s wr, 623 op/s 2024-04-17T07:27:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:47 smithi086 ceph-mon[31733]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.7 MiB/s wr, 564 op/s 2024-04-17T07:27:47.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:47 smithi098 ceph-mon[35418]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.7 MiB/s wr, 564 op/s 2024-04-17T07:27:47.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:47 smithi103 ceph-mon[35495]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.7 MiB/s wr, 564 op/s 2024-04-17T07:27:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 505.0 s, 659.0 tps, lat 150.645 ms stddev 219.359 2024-04-17T07:27:49.579 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:49 smithi103 ceph-mon[35495]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.2 MiB/s wr, 668 op/s 2024-04-17T07:27:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:49 smithi086 ceph-mon[31733]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.2 MiB/s wr, 668 op/s 2024-04-17T07:27:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:49 smithi098 ceph-mon[35418]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.2 MiB/s wr, 668 op/s 2024-04-17T07:27:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:51 smithi086 ceph-mon[31733]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.6 MiB/s wr, 615 op/s 2024-04-17T07:27:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:51 smithi098 ceph-mon[35418]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.6 MiB/s wr, 615 op/s 2024-04-17T07:27:51.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:51 smithi103 ceph-mon[35495]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.6 MiB/s wr, 615 op/s 2024-04-17T07:27:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:53 smithi098 ceph-mon[35418]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.6 MiB/s wr, 652 op/s 2024-04-17T07:27:53.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:53 smithi103 ceph-mon[35495]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.6 MiB/s wr, 652 op/s 2024-04-17T07:27:53.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:53 smithi086 ceph-mon[31733]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.6 MiB/s wr, 652 op/s 2024-04-17T07:27:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 510.0 s, 606.1 tps, lat 148.445 ms stddev 56.559 2024-04-17T07:27:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:55 smithi098 ceph-mon[35418]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.6 MiB/s wr, 680 op/s 2024-04-17T07:27:55.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:55 smithi103 ceph-mon[35495]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.6 MiB/s wr, 680 op/s 2024-04-17T07:27:55.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:55 smithi086 ceph-mon[31733]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.6 MiB/s wr, 680 op/s 2024-04-17T07:27:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:57 smithi098 ceph-mon[35418]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.1 MiB/s wr, 614 op/s 2024-04-17T07:27:57.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:57 smithi103 ceph-mon[35495]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.1 MiB/s wr, 614 op/s 2024-04-17T07:27:57.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:57 smithi086 ceph-mon[31733]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.1 MiB/s wr, 614 op/s 2024-04-17T07:27:58.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:27:58.248 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:27:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.172+0000 7fb059800640 1 -- 172.21.15.86:0/2741327127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb054100810 msgr2=0x7fb054102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.172+0000 7fb059800640 1 --2- 172.21.15.86:0/2741327127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb054100810 0x7fb054102c40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb044009860 tx=0x7fb04402bbb0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.172+0000 7fb059800640 1 -- 172.21.15.86:0/2741327127 shutdown_connections 2024-04-17T07:27:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.172+0000 7fb059800640 1 --2- 172.21.15.86:0/2741327127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb054105b30 0x7fb054105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.172+0000 7fb059800640 1 --2- 172.21.15.86:0/2741327127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054103180 0x7fb0541055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.172+0000 7fb059800640 1 --2- 172.21.15.86:0/2741327127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb054100810 0x7fb054102c40 secure :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb044009860 tx=0x7fb04402bbb0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.172+0000 7fb059800640 1 -- 172.21.15.86:0/2741327127 >> 172.21.15.86:0/2741327127 conn(0x7fb0540fc450 msgr2=0x7fb0540fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:59.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.173+0000 7fb059800640 1 -- 172.21.15.86:0/2741327127 shutdown_connections 2024-04-17T07:27:59.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.173+0000 7fb059800640 1 -- 172.21.15.86:0/2741327127 wait complete. 2024-04-17T07:27:59.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.174+0000 7fb059800640 1 Processor -- start 2024-04-17T07:27:59.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.174+0000 7fb059800640 1 -- start start 2024-04-17T07:27:59.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.175+0000 7fb059800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054103180 0x7fb05419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.176+0000 7fb059800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb054105b30 0x7fb05419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.176+0000 7fb059800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0541a3ff0 0x7fb0541a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.176+0000 7fb059800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb054114070 con 0x7fb054103180 2024-04-17T07:27:59.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.176+0000 7fb059800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb054113ef0 con 0x7fb0541a3ff0 2024-04-17T07:27:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.176+0000 7fb059800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541141f0 con 0x7fb054105b30 2024-04-17T07:27:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.176+0000 7fb053400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0541a3ff0 0x7fb0541a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.176+0000 7fb053400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0541a3ff0 0x7fb0541a63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38920/0 (socket says 172.21.15.86:38920) 2024-04-17T07:27:59.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.176+0000 7fb053400640 1 -- 172.21.15.86:0/35215472 learned_addr learned my addr 172.21.15.86:0/35215472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.177+0000 7fb052a00640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054103180 0x7fb05419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.177+0000 7fb052000640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb054105b30 0x7fb05419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.177+0000 7fb053400640 1 -- 172.21.15.86:0/35215472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb054105b30 msgr2=0x7fb05419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.177+0000 7fb053400640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb054105b30 0x7fb05419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.177+0000 7fb053400640 1 -- 172.21.15.86:0/35215472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054103180 msgr2=0x7fb05419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.177+0000 7fb053400640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054103180 0x7fb05419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.177+0000 7fb053400640 1 -- 172.21.15.86:0/35215472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0541a6a50 con 0x7fb0541a3ff0 2024-04-17T07:27:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.178+0000 7fb053400640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0541a3ff0 0x7fb0541a63e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00dac0 tx=0x7fb04c00df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.178+0000 7fb04b400640 1 -- 172.21.15.86:0/35215472 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c017070 con 0x7fb0541a3ff0 2024-04-17T07:27:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.178+0000 7fb059800640 1 -- 172.21.15.86:0/35215472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0541a6d40 con 0x7fb0541a3ff0 2024-04-17T07:27:59.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.178+0000 7fb04b400640 1 -- 172.21.15.86:0/35215472 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb04c00be30 con 0x7fb0541a3ff0 2024-04-17T07:27:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.179+0000 7fb04b400640 1 -- 172.21.15.86:0/35215472 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c010460 con 0x7fb0541a3ff0 2024-04-17T07:27:59.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.179+0000 7fb059800640 1 -- 172.21.15.86:0/35215472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541a7280 con 0x7fb0541a3ff0 2024-04-17T07:27:59.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.184+0000 7fb05c5a4740 1 Processor -- start 2024-04-17T07:27:59.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.185+0000 7fb04b400640 1 -- 172.21.15.86:0/35215472 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb04c010620 con 0x7fb0541a3ff0 2024-04-17T07:27:59.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.187+0000 7fb05c5a4740 1 -- start start 2024-04-17T07:27:59.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.188+0000 7fb04b400640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0380a3d00 0x7fb0380a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.188+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.188+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbc5b80 0x55797fbca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.188+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbcacf0 0x55797fbcd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.188+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55797fb2c120 con 0x55797fbc5190 2024-04-17T07:27:59.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.188+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55797fad2340 con 0x55797fbcacf0 2024-04-17T07:27:59.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.188+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55797fb280e0 con 0x55797fbc5b80 2024-04-17T07:27:59.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.189+0000 7fb053400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbcacf0 0x55797fbcd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.189+0000 7fb053400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbcacf0 0x55797fbcd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38928/0 (socket says 172.21.15.86:38928) 2024-04-17T07:27:59.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.189+0000 7fb053400640 1 -- 172.21.15.86:0/1622295210 learned_addr learned my addr 172.21.15.86:0/1622295210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:59.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.189+0000 7fb04b400640 1 -- 172.21.15.86:0/35215472 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb04c0d0ac0 con 0x7fb0541a3ff0 2024-04-17T07:27:59.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.189+0000 7fb052a00640 1 --2- 172.21.15.86:0/1622295210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.190+0000 7fb053400640 1 -- 172.21.15.86:0/1622295210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbc5b80 msgr2=0x55797fbca680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:27:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.190+0000 7fb053400640 1 --2- 172.21.15.86:0/1622295210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbc5b80 0x55797fbca680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.190+0000 7fb053400640 1 -- 172.21.15.86:0/1622295210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 msgr2=0x55797fbc55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.190+0000 7fb052000640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0380a3d00 0x7fb0380a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.190+0000 7fb053400640 1 --2- 172.21.15.86:0/1622295210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.190+0000 7fb053400640 1 -- 172.21.15.86:0/1622295210 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55797f9087d0 con 0x55797fbcacf0 2024-04-17T07:27:59.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.190+0000 7fb052a00640 1 --2- 172.21.15.86:0/1622295210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:27:59.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.192+0000 7fb052000640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0380a3d00 0x7fb0380a61c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb05419ff60 tx=0x7fb040009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:59.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.192+0000 7fb053400640 1 --2- 172.21.15.86:0/1622295210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbcacf0 0x55797fbcd180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb04c098e20 tx=0x7fb04c09c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ba3fe5cc8c952e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:59.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.192+0000 7fb03fe00640 1 -- 172.21.15.86:0/1622295210 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c09d0c0 con 0x55797fbcacf0 2024-04-17T07:27:59.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.193+0000 7fb03fe00640 1 -- 172.21.15.86:0/1622295210 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb04c09c9c0 con 0x55797fbcacf0 2024-04-17T07:27:59.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.193+0000 7fb03fe00640 1 -- 172.21.15.86:0/1622295210 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c099180 con 0x55797fbcacf0 2024-04-17T07:27:59.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.194+0000 7fb05c5a4740 1 -- 172.21.15.86:0/1622295210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbcacf0 msgr2=0x55797fbcd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.194+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/1622295210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbcacf0 0x55797fbcd180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb04c098e20 tx=0x7fb04c09c320 comp rx=0 tx=0).stop 2024-04-17T07:27:59.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.194+0000 7fb05c5a4740 1 -- 172.21.15.86:0/1622295210 shutdown_connections 2024-04-17T07:27:59.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.194+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/1622295210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbcacf0 0x55797fbcd180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.194+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/1622295210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbc5b80 0x55797fbca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.194+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/1622295210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.194+0000 7fb05c5a4740 1 -- 172.21.15.86:0/1622295210 >> 172.21.15.86:0/1622295210 conn(0x55797fbc0700 msgr2=0x55797fbc0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:59.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.195+0000 7fb05c5a4740 1 -- 172.21.15.86:0/1622295210 shutdown_connections 2024-04-17T07:27:59.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.195+0000 7fb05c5a4740 1 -- 172.21.15.86:0/1622295210 wait complete. 2024-04-17T07:27:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.196+0000 7fb05c5a4740 1 -- start start 2024-04-17T07:27:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.196+0000 7fb05c5a4740 1 Processor -- start 2024-04-17T07:27:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.198+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbc5190 0x55797fbcf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.198+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbcfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.198+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbcacf0 0x55797fbd6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.198+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55797fbcdc40 con 0x55797fbcacf0 2024-04-17T07:27:59.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.198+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55797fae5710 con 0x55797fbc5b80 2024-04-17T07:27:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.198+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55797fbcddc0 con 0x55797fbc5190 2024-04-17T07:27:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.198+0000 7fb05c5a4740 10 client.0 _ll_register_callbacks cb 0x55797fae0af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.199+0000 7fb052000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbcfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.199+0000 7fb052000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbcfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38940/0 (socket says 172.21.15.86:38940) 2024-04-17T07:27:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.199+0000 7fb052000640 1 -- 172.21.15.86:0/2725231022 learned_addr learned my addr 172.21.15.86:0/2725231022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:59.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.200+0000 7fb052000640 1 -- 172.21.15.86:0/2725231022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbc5190 msgr2=0x55797fbcf680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:27:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.200+0000 7fb052000640 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbc5190 0x55797fbcf680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.200+0000 7fb052000640 1 -- 172.21.15.86:0/2725231022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbcacf0 msgr2=0x55797fbd6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.200+0000 7fb052000640 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbcacf0 0x55797fbd6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.200+0000 7fb052000640 1 -- 172.21.15.86:0/2725231022 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55797f83e3f0 con 0x55797fbc5b80 2024-04-17T07:27:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.201+0000 7fb052000640 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbcfbc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb040010ee0 tx=0x7fb0400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73cb6a5f103d1b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:59.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.201+0000 7fb03cc00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:27:59.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.203+0000 7fb03cc00640 1 -- 172.21.15.86:0/2725231022 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040023070 con 0x55797fbc5b80 2024-04-17T07:27:59.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.203+0000 7fb03cc00640 1 -- 172.21.15.86:0/2725231022 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb04001f650 con 0x55797fbc5b80 2024-04-17T07:27:59.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.203+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2725231022 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb04c09c9c0 con 0x55797fbc5b80 2024-04-17T07:27:59.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.203+0000 7fb03cc00640 1 -- 172.21.15.86:0/2725231022 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb04002f050 con 0x55797fbc5b80 2024-04-17T07:27:59.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.203+0000 7fb05c5a4740 10 client.37553 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:59.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.203+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2725231022 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55797f7fe0f0 con 0x55797fbc5b80 2024-04-17T07:27:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.205+0000 7fb03cc00640 1 -- 172.21.15.86:0/2725231022 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb04c09c9c0 con 0x55797fbc5b80 2024-04-17T07:27:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.205+0000 7fb03cc00640 1 -- 172.21.15.86:0/2725231022 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb04000f040 con 0x55797fbc5b80 2024-04-17T07:27:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.206+0000 7fb05c5a4740 10 client.37553 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.206+0000 7fb05c5a4740 10 client.37553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:27:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.206+0000 7fb05c5a4740 20 client.37553 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.206+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55797fbdb440 0x55797fbdd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.206+0000 7fb05c5a4740 4 client.37553 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:27:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.206+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2725231022 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55797fb39b40 con 0x55797fbdb440 2024-04-17T07:27:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.207+0000 7fb052a00640 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55797fbdb440 0x55797fbdd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.208+0000 7fb03cc00640 10 client.37553 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.208+0000 7fb052a00640 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55797fbdb440 0x55797fbdd8a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d9f01116dadfa3 server_cookie=319a2a9aba5e57fc in_seq=0 out_seq=0 2024-04-17T07:27:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.211+0000 7fb03cc00640 1 -- 172.21.15.86:0/2725231022 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55797fb39b40 con 0x55797fbdb440 2024-04-17T07:27:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.212+0000 7fb03cc00640 10 client.37553 handle_command_reply: tid=0 2024-04-17T07:27:59.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.212+0000 7fb05c5a4740 1 client.37553 shutdown 2024-04-17T07:27:59.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.212+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2725231022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 msgr2=0x55797fbcfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.212+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbcfbc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb040010ee0 tx=0x7fb0400046c0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.213+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2725231022 shutdown_connections 2024-04-17T07:27:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.213+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55797fbdb440 0x55797fbdd8a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.213+0000 7fb03cc00640 0 client.37553 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.213+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbcacf0 0x55797fbd6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.213+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbcfbc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.213+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2725231022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbc5190 0x55797fbcf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.213+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2725231022 >> 172.21.15.86:0/2725231022 conn(0x55797fbc0700 msgr2=0x55797fbbf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.213+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2725231022 shutdown_connections 2024-04-17T07:27:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.214+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2725231022 wait complete. 2024-04-17T07:27:59.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.214+0000 7fb05c5a4740 20 client.37553 trim_cache size 0 max 16384 2024-04-17T07:27:59.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.250+0000 7fb05c5a4740 1 Processor -- start 2024-04-17T07:27:59.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.250+0000 7fb05c5a4740 1 -- start start 2024-04-17T07:27:59.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.251+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.251+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbd6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.251+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 0x55797fbc70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.251+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b18a0 con 0x55797fbc5190 2024-04-17T07:27:59.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.251+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b33e0 con 0x55797fbc5b80 2024-04-17T07:27:59.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.251+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b1f70 con 0x55797fbcacf0 2024-04-17T07:27:59.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.251+0000 7fb052a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.251+0000 7fb052a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41322/0 (socket says 172.21.15.86:41322) 2024-04-17T07:27:59.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.251+0000 7fb052a00640 1 -- 172.21.15.86:0/215027233 learned_addr learned my addr 172.21.15.86:0/215027233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:59.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.252+0000 7fb053400640 1 --2- 172.21.15.86:0/215027233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 0x55797fbc70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.252+0000 7fb052000640 1 --2- 172.21.15.86:0/215027233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbd6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.252+0000 7fb052a00640 1 -- 172.21.15.86:0/215027233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 msgr2=0x55797fbc70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.252+0000 7fb052a00640 1 --2- 172.21.15.86:0/215027233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 0x55797fbc70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.252+0000 7fb052a00640 1 -- 172.21.15.86:0/215027233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 msgr2=0x55797fbd6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.252+0000 7fb052a00640 1 --2- 172.21.15.86:0/215027233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbd6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.252+0000 7fb052a00640 1 -- 172.21.15.86:0/215027233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55797fbc5f60 con 0x55797fbc5190 2024-04-17T07:27:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.253+0000 7fb052a00640 1 --2- 172.21.15.86:0/215027233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc6580 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb044034900 tx=0x7fb044005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d04dbf59ff73900 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.253+0000 7fb03f400640 1 -- 172.21.15.86:0/215027233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044039040 con 0x55797fbc5190 2024-04-17T07:27:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.254+0000 7fb03f400640 1 -- 172.21.15.86:0/215027233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb04403a040 con 0x55797fbc5190 2024-04-17T07:27:59.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.254+0000 7fb03f400640 1 -- 172.21.15.86:0/215027233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044005280 con 0x55797fbc5190 2024-04-17T07:27:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.255+0000 7fb05c5a4740 1 -- 172.21.15.86:0/215027233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 msgr2=0x55797fbc6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.255+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/215027233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc6580 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb044034900 tx=0x7fb044005b20 comp rx=0 tx=0).stop 2024-04-17T07:27:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.255+0000 7fb05c5a4740 1 -- 172.21.15.86:0/215027233 shutdown_connections 2024-04-17T07:27:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.255+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/215027233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 0x55797fbc70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.256+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/215027233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbd6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.256+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/215027233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbc6580 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.256+0000 7fb05c5a4740 1 -- 172.21.15.86:0/215027233 >> 172.21.15.86:0/215027233 conn(0x55797fbc0700 msgr2=0x55797fbcf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.256+0000 7fb05c5a4740 1 -- 172.21.15.86:0/215027233 shutdown_connections 2024-04-17T07:27:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.256+0000 7fb05c5a4740 1 -- 172.21.15.86:0/215027233 wait complete. 2024-04-17T07:27:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.257+0000 7fb05c5a4740 1 -- start start 2024-04-17T07:27:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.257+0000 7fb05c5a4740 1 Processor -- start 2024-04-17T07:27:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.260+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbd24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.260+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbd2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.260+0000 7fb05c5a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 0x55797fbd5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.260+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b2d10 con 0x55797fbc5190 2024-04-17T07:27:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.260+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b2640 con 0x55797fbc5b80 2024-04-17T07:27:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.260+0000 7fb05c5a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541af870 con 0x55797fbcacf0 2024-04-17T07:27:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.261+0000 7fb05c5a4740 10 client.0 _ll_register_callbacks cb 0x7fb040016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:27:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.261+0000 7fb053400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 0x55797fbd5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.261+0000 7fb053400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 0x55797fbd5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58486/0 (socket says 172.21.15.86:58486) 2024-04-17T07:27:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.261+0000 7fb053400640 1 -- 172.21.15.86:0/2234602503 learned_addr learned my addr 172.21.15.86:0/2234602503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.261+0000 7fb052000640 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbd2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.262+0000 7fb052a00640 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbd24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.262+0000 7fb052000640 1 -- 172.21.15.86:0/2234602503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 msgr2=0x55797fbd5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.262+0000 7fb052000640 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 0x55797fbd5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.262+0000 7fb052000640 1 -- 172.21.15.86:0/2234602503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 msgr2=0x55797fbd24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:27:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.262+0000 7fb052000640 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbd24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.262+0000 7fb052000640 1 -- 172.21.15.86:0/2234602503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55797fbd6010 con 0x55797fbc5b80 2024-04-17T07:27:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.263+0000 7fb052000640 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbd2a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb040010eb0 tx=0x7fb04001e650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c36458a754626e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.266+0000 7fb03cc00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:27:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.266+0000 7fb03cc00640 1 -- 172.21.15.86:0/2234602503 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040023040 con 0x55797fbc5b80 2024-04-17T07:27:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.266+0000 7fb03cc00640 1 -- 172.21.15.86:0/2234602503 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb04001fc60 con 0x55797fbc5b80 2024-04-17T07:27:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.267+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2234602503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb04403a040 con 0x55797fbc5b80 2024-04-17T07:27:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.268+0000 7fb03cc00640 1 -- 172.21.15.86:0/2234602503 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb040040050 con 0x55797fbc5b80 2024-04-17T07:27:59.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.268+0000 7fb05c5a4740 10 client.37559 fetch_fsmap learned FSMap version 28 2024-04-17T07:27:59.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.268+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2234602503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55797fc52b80 con 0x55797fbc5b80 2024-04-17T07:27:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.270+0000 7fb03cc00640 1 -- 172.21.15.86:0/2234602503 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb04403a040 con 0x55797fbc5b80 2024-04-17T07:27:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.270+0000 7fb03cc00640 1 -- 172.21.15.86:0/2234602503 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb040011500 con 0x55797fbc5b80 2024-04-17T07:27:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.270+0000 7fb05c5a4740 10 client.37559 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:27:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.270+0000 7fb05c5a4740 10 client.37559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:27:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.270+0000 7fb05c5a4740 20 client.37559 populate_metadata read hostname 'smithi086' 2024-04-17T07:27:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.270+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55797fbdb440 0x55797fc55510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.270+0000 7fb05c5a4740 4 client.37559 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:27:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.270+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2234602503 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55797fb339b0 con 0x55797fbdb440 2024-04-17T07:27:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.271+0000 7fb052a00640 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55797fbdb440 0x55797fc55510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.273+0000 7fb052a00640 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55797fbdb440 0x55797fc55510 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=842839b6c92e793 server_cookie=3b4dbb93eb75b4e9 in_seq=0 out_seq=0 2024-04-17T07:27:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.273+0000 7fb03cc00640 10 client.37559 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:59.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.274+0000 7fb03cc00640 1 -- 172.21.15.86:0/2234602503 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55797fb339b0 con 0x55797fbdb440 2024-04-17T07:27:59.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.274+0000 7fb03cc00640 10 client.37559 handle_command_reply: tid=0 2024-04-17T07:27:59.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.274+0000 7fb05c5a4740 1 client.37559 shutdown 2024-04-17T07:27:59.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.275+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2234602503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 msgr2=0x55797fbd2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.275+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbd2a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb040010eb0 tx=0x7fb04001e650 comp rx=0 tx=0).stop 2024-04-17T07:27:59.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.275+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2234602503 shutdown_connections 2024-04-17T07:27:59.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.275+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55797fbdb440 0x55797fc55510 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.275+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55797fbcacf0 0x55797fbd5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.275+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55797fbc5b80 0x55797fbd2a00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.275+0000 7fb05c5a4740 1 --2- 172.21.15.86:0/2234602503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55797fbc5190 0x55797fbd24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.275+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2234602503 >> 172.21.15.86:0/2234602503 conn(0x55797fbc0700 msgr2=0x55797fbc70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:59.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.275+0000 7fb03cc00640 0 client.37559 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:27:59.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.279+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2234602503 shutdown_connections 2024-04-17T07:27:59.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.279+0000 7fb05c5a4740 1 -- 172.21.15.86:0/2234602503 wait complete. 2024-04-17T07:27:59.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.279+0000 7fb05c5a4740 20 client.37559 trim_cache size 0 max 16384 2024-04-17T07:27:59.280 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:27:59.280 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:27:59.280 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:27:59.280 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:27:59.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.280+0000 7fb059800640 1 -- 172.21.15.86:0/35215472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0380a3d00 msgr2=0x7fb0380a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0380a3d00 0x7fb0380a61c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb05419ff60 tx=0x7fb040009260 comp rx=0 tx=0).stop 2024-04-17T07:27:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 -- 172.21.15.86:0/35215472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0541a3ff0 msgr2=0x7fb0541a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0541a3ff0 0x7fb0541a63e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00dac0 tx=0x7fb04c00df80 comp rx=0 tx=0).stop 2024-04-17T07:27:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 -- 172.21.15.86:0/35215472 shutdown_connections 2024-04-17T07:27:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb0380a3d00 0x7fb0380a61c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0541a3ff0 0x7fb0541a63e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb054105b30 0x7fb05419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 --2- 172.21.15.86:0/35215472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054103180 0x7fb05419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 -- 172.21.15.86:0/35215472 >> 172.21.15.86:0/35215472 conn(0x7fb0540fc450 msgr2=0x7fb0541050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 -- 172.21.15.86:0/35215472 shutdown_connections 2024-04-17T07:27:59.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.281+0000 7fb059800640 1 -- 172.21.15.86:0/35215472 wait complete. 2024-04-17T07:27:59.311 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:27:59.311 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:27:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 515.0 s, 629.1 tps, lat 176.197 ms stddev 232.448 2024-04-17T07:27:59.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.575+0000 7f57c7000640 1 -- 172.21.15.86:0/314798426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0073d90 msgr2=0x7f57c010af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.575+0000 7f57c7000640 1 --2- 172.21.15.86:0/314798426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0073d90 0x7f57c010af70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009900 tx=0x7f57b402bc30 comp rx=0 tx=0).stop 2024-04-17T07:27:59.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.576+0000 7f57c7000640 1 -- 172.21.15.86:0/314798426 shutdown_connections 2024-04-17T07:27:59.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.576+0000 7f57c7000640 1 --2- 172.21.15.86:0/314798426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c010e040 0x7f57c0118ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.576+0000 7f57c7000640 1 --2- 172.21.15.86:0/314798426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57c010b540 0x7f57c010d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.576+0000 7f57c7000640 1 --2- 172.21.15.86:0/314798426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0073d90 0x7f57c010af70 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.576+0000 7f57c7000640 1 -- 172.21.15.86:0/314798426 >> 172.21.15.86:0/314798426 conn(0x7f57c006ecc0 msgr2=0x7f57c0071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:59.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.576+0000 7f57c7000640 1 -- 172.21.15.86:0/314798426 shutdown_connections 2024-04-17T07:27:59.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.576+0000 7f57c7000640 1 -- 172.21.15.86:0/314798426 wait complete. 2024-04-17T07:27:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.578+0000 7f57c7000640 1 Processor -- start 2024-04-17T07:27:59.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.578+0000 7f57c7000640 1 -- start start 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.579+0000 7f57c7000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0073d90 0x7f57c01a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.579+0000 7f57c7000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c010b540 0x7f57c01a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.579+0000 7f57c7000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57c010e040 0x7f57c01ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.579+0000 7f57c7000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57c011c730 con 0x7f57c0073d90 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.579+0000 7f57c7000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57c011c5b0 con 0x7f57c010e040 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.579+0000 7f57c7000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57c011c8b0 con 0x7f57c010b540 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.579+0000 7f57c5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0073d90 0x7f57c01a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.579+0000 7f57c5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0073d90 0x7f57c01a6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41352/0 (socket says 172.21.15.86:41352) 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.579+0000 7f57c5c00640 1 -- 172.21.15.86:0/1225327096 learned_addr learned my addr 172.21.15.86:0/1225327096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.579+0000 7f57c6600640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57c010e040 0x7f57c01ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.580+0000 7f57c5200640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c010b540 0x7f57c01a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.580+0000 7f57c5c00640 1 -- 172.21.15.86:0/1225327096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c010b540 msgr2=0x7f57c01a73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.580+0000 7f57c5c00640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c010b540 0x7f57c01a73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.580+0000 7f57c5c00640 1 -- 172.21.15.86:0/1225327096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57c010e040 msgr2=0x7f57c01ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.580+0000 7f57c5c00640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57c010e040 0x7f57c01ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.580+0000 7f57c5c00640 1 -- 172.21.15.86:0/1225327096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c01aea30 con 0x7f57c0073d90 2024-04-17T07:27:59.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:27:59 smithi086 ceph-mon[31733]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.0 MiB/s wr, 648 op/s 2024-04-17T07:27:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.580+0000 7f57c5c00640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0073d90 0x7f57c01a6e60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f57b4000c00 tx=0x7f57b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.581+0000 7f57be000640 1 -- 172.21.15.86:0/1225327096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4043020 con 0x7f57c0073d90 2024-04-17T07:27:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.581+0000 7f57be000640 1 -- 172.21.15.86:0/1225327096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b40052b0 con 0x7f57c0073d90 2024-04-17T07:27:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.581+0000 7f57be000640 1 -- 172.21.15.86:0/1225327096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4003cc0 con 0x7f57c0073d90 2024-04-17T07:27:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.581+0000 7f57c7000640 1 -- 172.21.15.86:0/1225327096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c01aecc0 con 0x7f57c0073d90 2024-04-17T07:27:59.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.581+0000 7f57c7000640 1 -- 172.21.15.86:0/1225327096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c01af1b0 con 0x7f57c0073d90 2024-04-17T07:27:59.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.583+0000 7f57c7000640 1 -- 172.21.15.86:0/1225327096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c00717c0 con 0x7f57c0073d90 2024-04-17T07:27:59.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.587+0000 7f57be000640 1 -- 172.21.15.86:0/1225327096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f57b4046020 con 0x7f57c0073d90 2024-04-17T07:27:59.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.590+0000 7f57be000640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a80a3c80 0x7f57a80a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:27:59.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.590+0000 7f57be000640 1 -- 172.21.15.86:0/1225327096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f57b40f2c80 con 0x7f57c0073d90 2024-04-17T07:27:59.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.591+0000 7f57c5200640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a80a3c80 0x7f57a80a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:27:59.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.593+0000 7f57c5200640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a80a3c80 0x7f57a80a6140 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f57ac005fd0 tx=0x7f57ac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:27:59.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.596+0000 7f57be000640 1 -- 172.21.15.86:0/1225327096 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f57b404d020 con 0x7f57c0073d90 2024-04-17T07:27:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:27:59 smithi098 ceph-mon[35418]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.0 MiB/s wr, 648 op/s 2024-04-17T07:27:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:27:59 smithi103 ceph-mon[35495]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.0 MiB/s wr, 648 op/s 2024-04-17T07:27:59.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.958+0000 7f57c7000640 1 -- 172.21.15.86:0/1225327096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57c0062b60 con 0x7f57c0073d90 2024-04-17T07:27:59.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.959+0000 7f57be000640 1 -- 172.21.15.86:0/1225327096 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f57b40bf280 con 0x7f57c0073d90 2024-04-17T07:27:59.960 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:27:59.960 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:27:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:27:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 -- 172.21.15.86:0/1225327096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a80a3c80 msgr2=0x7f57a80a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a80a3c80 0x7f57a80a6140 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f57ac005fd0 tx=0x7f57ac005850 comp rx=0 tx=0).stop 2024-04-17T07:27:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 -- 172.21.15.86:0/1225327096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0073d90 msgr2=0x7f57c01a6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:27:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0073d90 0x7f57c01a6e60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f57b4000c00 tx=0x7f57b4004270 comp rx=0 tx=0).stop 2024-04-17T07:27:59.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 -- 172.21.15.86:0/1225327096 shutdown_connections 2024-04-17T07:27:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f57a80a3c80 0x7f57a80a6140 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57c010e040 0x7f57c01ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57c010b540 0x7f57c01a73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 --2- 172.21.15.86:0/1225327096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57c0073d90 0x7f57c01a6e60 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:27:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 -- 172.21.15.86:0/1225327096 >> 172.21.15.86:0/1225327096 conn(0x7f57c006ecc0 msgr2=0x7f57c0108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:27:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.963+0000 7f57c7000640 1 -- 172.21.15.86:0/1225327096 shutdown_connections 2024-04-17T07:27:59.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:27:59.964+0000 7f57c7000640 1 -- 172.21.15.86:0/1225327096 wait complete. 2024-04-17T07:27:59.989 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:28:00.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.308+0000 7f0d59800640 1 -- 172.21.15.86:0/4266636155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d54077b60 msgr2=0x7f0d54077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.308+0000 7f0d59800640 1 --2- 172.21.15.86:0/4266636155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d54077b60 0x7f0d54077fc0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009900 tx=0x7f0d4002bc30 comp rx=0 tx=0).stop 2024-04-17T07:28:00.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.308+0000 7f0d59800640 1 -- 172.21.15.86:0/4266636155 shutdown_connections 2024-04-17T07:28:00.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.308+0000 7f0d59800640 1 --2- 172.21.15.86:0/4266636155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d54078500 0x7f0d54078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.308+0000 7f0d59800640 1 --2- 172.21.15.86:0/4266636155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d54077b60 0x7f0d54077fc0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.308+0000 7f0d59800640 1 --2- 172.21.15.86:0/4266636155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d54076920 0x7f0d54076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.308+0000 7f0d59800640 1 -- 172.21.15.86:0/4266636155 >> 172.21.15.86:0/4266636155 conn(0x7f0d540ff710 msgr2=0x7f0d54101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.308+0000 7f0d59800640 1 -- 172.21.15.86:0/4266636155 shutdown_connections 2024-04-17T07:28:00.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.308+0000 7f0d59800640 1 -- 172.21.15.86:0/4266636155 wait complete. 2024-04-17T07:28:00.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.312+0000 7f0d59800640 1 Processor -- start 2024-04-17T07:28:00.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.312+0000 7f0d59800640 1 -- start start 2024-04-17T07:28:00.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.312+0000 7f0d59800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d54076920 0x7f0d5419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.312+0000 7f0d59800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d54077b60 0x7f0d5419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.312+0000 7f0d59800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d54078500 0x7f0d541a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.312+0000 7f0d59800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d54114120 con 0x7f0d54077b60 2024-04-17T07:28:00.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.312+0000 7f0d59800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0d54113fa0 con 0x7f0d54078500 2024-04-17T07:28:00.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.312+0000 7f0d59800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541142a0 con 0x7f0d54076920 2024-04-17T07:28:00.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.313+0000 7f0d52a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d54076920 0x7f0d5419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.313+0000 7f0d52000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d54077b60 0x7f0d5419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.313+0000 7f0d52a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d54076920 0x7f0d5419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58520/0 (socket says 172.21.15.86:58520) 2024-04-17T07:28:00.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.313+0000 7f0d52a00640 1 -- 172.21.15.86:0/3361523672 learned_addr learned my addr 172.21.15.86:0/3361523672 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:00.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.313+0000 7f0d53400640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d54078500 0x7f0d541a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.313+0000 7f0d52a00640 1 -- 172.21.15.86:0/3361523672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d54078500 msgr2=0x7f0d541a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.313+0000 7f0d52a00640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d54078500 0x7f0d541a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.313+0000 7f0d52a00640 1 -- 172.21.15.86:0/3361523672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d54077b60 msgr2=0x7f0d5419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.313+0000 7f0d52a00640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d54077b60 0x7f0d5419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.314+0000 7f0d52a00640 1 -- 172.21.15.86:0/3361523672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d541a6420 con 0x7f0d54076920 2024-04-17T07:28:00.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.314+0000 7f0d52000640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d54077b60 0x7f0d5419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:00.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.314+0000 7f0d52a00640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d54076920 0x7f0d5419e800 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400e9f0 tx=0x7f0d4400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.314+0000 7f0d4b400640 1 -- 172.21.15.86:0/3361523672 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400cd80 con 0x7f0d54076920 2024-04-17T07:28:00.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.315+0000 7f0d59800640 1 -- 172.21.15.86:0/3361523672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d541a6710 con 0x7f0d54076920 2024-04-17T07:28:00.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.315+0000 7f0d4b400640 1 -- 172.21.15.86:0/3361523672 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d440054d0 con 0x7f0d54076920 2024-04-17T07:28:00.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.315+0000 7f0d4b400640 1 -- 172.21.15.86:0/3361523672 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44010490 con 0x7f0d54076920 2024-04-17T07:28:00.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.315+0000 7f0d59800640 1 -- 172.21.15.86:0/3361523672 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d541a6c30 con 0x7f0d54076920 2024-04-17T07:28:00.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.321+0000 7f0d4b400640 1 -- 172.21.15.86:0/3361523672 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0d44005020 con 0x7f0d54076920 2024-04-17T07:28:00.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.321+0000 7f0d5c587740 1 Processor -- start 2024-04-17T07:28:00.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.321+0000 7f0d5c587740 1 -- start start 2024-04-17T07:28:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.322+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b7fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.322+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b800490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.322+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b802f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.322+0000 7f0d5c587740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c31b761020 con 0x55c31b800b00 2024-04-17T07:28:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.322+0000 7f0d5c587740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c31b707460 con 0x55c31b7fafa0 2024-04-17T07:28:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.322+0000 7f0d5c587740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c31b75cfe0 con 0x55c31b7fb990 2024-04-17T07:28:00.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.323+0000 7f0d52a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b800490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.323+0000 7f0d53400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b7fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.323+0000 7f0d52a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b800490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58530/0 (socket says 172.21.15.86:58530) 2024-04-17T07:28:00.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.323+0000 7f0d52a00640 1 -- 172.21.15.86:0/751699105 learned_addr learned my addr 172.21.15.86:0/751699105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:00.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.323+0000 7f0d4b400640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d380a3bb0 0x7f0d380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.323+0000 7f0d53400640 1 -- 172.21.15.86:0/751699105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 msgr2=0x55c31b800490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.323+0000 7f0d53400640 1 --2- 172.21.15.86:0/751699105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b800490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.324+0000 7f0d53400640 1 -- 172.21.15.86:0/751699105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 msgr2=0x55c31b802f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.324+0000 7f0d53400640 1 --2- 172.21.15.86:0/751699105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b802f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.324+0000 7f0d53400640 1 -- 172.21.15.86:0/751699105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31b70f040 con 0x55c31b7fafa0 2024-04-17T07:28:00.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.324+0000 7f0d4b400640 1 -- 172.21.15.86:0/3361523672 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0d440d0c00 con 0x7f0d54076920 2024-04-17T07:28:00.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.324+0000 7f0d53400640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d380a3bb0 0x7f0d380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.324+0000 7f0d53400640 1 --2- 172.21.15.86:0/751699105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b7fb3c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009fd0 tx=0x7f0d4c011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8eb6f674e1c61e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.326+0000 7f0d3fe00640 1 -- 172.21.15.86:0/751699105 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c017070 con 0x55c31b7fafa0 2024-04-17T07:28:00.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.326+0000 7f0d3fe00640 1 -- 172.21.15.86:0/751699105 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d4c00fbf0 con 0x55c31b7fafa0 2024-04-17T07:28:00.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.326+0000 7f0d3fe00640 1 -- 172.21.15.86:0/751699105 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00e940 con 0x55c31b7fafa0 2024-04-17T07:28:00.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.326+0000 7f0d53400640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d380a3bb0 0x7f0d380a6070 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c022000 tx=0x7f0d4c023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.327+0000 7f0d5c587740 1 -- 172.21.15.86:0/751699105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 msgr2=0x55c31b7fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.327+0000 7f0d5c587740 1 --2- 172.21.15.86:0/751699105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b7fb3c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009fd0 tx=0x7f0d4c011920 comp rx=0 tx=0).stop 2024-04-17T07:28:00.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.328+0000 7f0d5c587740 1 -- 172.21.15.86:0/751699105 shutdown_connections 2024-04-17T07:28:00.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.328+0000 7f0d5c587740 1 --2- 172.21.15.86:0/751699105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b802f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.328+0000 7f0d5c587740 1 --2- 172.21.15.86:0/751699105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b800490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.328+0000 7f0d5c587740 1 --2- 172.21.15.86:0/751699105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b7fb3c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.328+0000 7f0d5c587740 1 -- 172.21.15.86:0/751699105 >> 172.21.15.86:0/751699105 conn(0x55c31b7f6510 msgr2=0x55c31b7f6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.328+0000 7f0d5c587740 1 -- 172.21.15.86:0/751699105 shutdown_connections 2024-04-17T07:28:00.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.328+0000 7f0d5c587740 1 -- 172.21.15.86:0/751699105 wait complete. 2024-04-17T07:28:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.329+0000 7f0d5c587740 1 -- start start 2024-04-17T07:28:00.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.330+0000 7f0d5c587740 1 Processor -- start 2024-04-17T07:28:00.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.331+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b8054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.331+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b805a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.331+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b80ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.331+0000 7f0d52000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b805a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.331+0000 7f0d52000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b805a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58540/0 (socket says 172.21.15.86:58540) 2024-04-17T07:28:00.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.332+0000 7f0d52000640 1 -- 172.21.15.86:0/1873065163 learned_addr learned my addr 172.21.15.86:0/1873065163 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:00.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.332+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c31b803ab0 con 0x55c31b800b00 2024-04-17T07:28:00.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.332+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c31b71a830 con 0x55c31b7fafa0 2024-04-17T07:28:00.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.332+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c31b803c30 con 0x55c31b7fb990 2024-04-17T07:28:00.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.332+0000 7f0d53400640 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b80ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.332+0000 7f0d5c587740 10 client.0 _ll_register_callbacks cb 0x55c31b6e8fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:00.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.332+0000 7f0d52a00640 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b8054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.333+0000 7f0d52000640 1 -- 172.21.15.86:0/1873065163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 msgr2=0x55c31b8054e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.333+0000 7f0d52000640 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b8054e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.333+0000 7f0d52000640 1 -- 172.21.15.86:0/1873065163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 msgr2=0x55c31b80ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.333+0000 7f0d52000640 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b80ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.333+0000 7f0d52000640 1 -- 172.21.15.86:0/1873065163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31b71e920 con 0x55c31b7fb990 2024-04-17T07:28:00.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.333+0000 7f0d52a00640 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b8054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:00.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.333+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d4c00fbf0 con 0x55c31b7fb990 2024-04-17T07:28:00.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.333+0000 7f0d53400640 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b80ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:00.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.334+0000 7f0d52000640 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b805a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d40005ab0 tx=0x7f0d40004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6b50e7c16c8a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.334+0000 7f0d3cc00640 10 client.27628 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:28:00.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.334+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1873065163 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40044020 con 0x55c31b7fb990 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.334+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1873065163 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0d40005240 con 0x55c31b7fb990 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.334+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1873065163 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d40005d50 con 0x55c31b7fb990 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.335+0000 7f0d5c587740 10 client.27628 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.335+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55c31b6ec7f0 con 0x55c31b7fb990 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.335+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1873065163 <== mon.1 v2:172.21.15.103:3300/0 4 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0d40005440 con 0x55c31b7fb990 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.335+0000 7f0d5c587740 10 client.27628 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.335+0000 7f0d5c587740 10 client.27628 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.335+0000 7f0d5c587740 20 client.27628 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.335+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55c31b810da0 0x55c31b813230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.335+0000 7f0d5c587740 4 client.27628 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:00.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.335+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c31b76ea40 con 0x55c31b810da0 2024-04-17T07:28:00.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.336+0000 7f0d52a00640 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55c31b810da0 0x55c31b813230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.337+0000 7f0d52a00640 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55c31b810da0 0x55c31b813230 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e8d891c0214c026 server_cookie=ac9cd878281eb8ef in_seq=0 out_seq=0 2024-04-17T07:28:00.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.337+0000 7f0d3cc00640 10 client.27628 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:28:00.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.338+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1873065163 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c31b76ea40 con 0x55c31b810da0 2024-04-17T07:28:00.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.338+0000 7f0d3cc00640 10 client.27628 handle_command_reply: tid=0 2024-04-17T07:28:00.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.338+0000 7f0d5c587740 1 client.27628 shutdown 2024-04-17T07:28:00.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.339+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 msgr2=0x55c31b805a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.339+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b805a20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d40005ab0 tx=0x7f0d40004270 comp rx=0 tx=0).stop 2024-04-17T07:28:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.340+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 shutdown_connections 2024-04-17T07:28:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.340+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55c31b810da0 0x55c31b813230 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.340+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b80ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.340+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b805a20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.340+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1873065163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b8054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.340+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 >> 172.21.15.86:0/1873065163 conn(0x55c31b7f6510 msgr2=0x55c31b7f5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.340+0000 7f0d3cc00640 0 client.27628 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:28:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.340+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 shutdown_connections 2024-04-17T07:28:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.340+0000 7f0d5c587740 1 -- 172.21.15.86:0/1873065163 wait complete. 2024-04-17T07:28:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.341+0000 7f0d5c587740 20 client.27628 trim_cache size 0 max 16384 2024-04-17T07:28:00.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.359+0000 7f0d5c587740 1 Processor -- start 2024-04-17T07:28:00.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.359+0000 7f0d5c587740 1 -- start start 2024-04-17T07:28:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.360+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b7fce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.360+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b80ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.360+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b7fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.360+0000 7f0d5c587740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541b1120 con 0x55c31b800b00 2024-04-17T07:28:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.360+0000 7f0d5c587740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541b2c60 con 0x55c31b7fafa0 2024-04-17T07:28:00.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.360+0000 7f0d5c587740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541b17f0 con 0x55c31b7fb990 2024-04-17T07:28:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d52000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b80ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d52000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b80ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58554/0 (socket says 172.21.15.86:58554) 2024-04-17T07:28:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d52000640 1 -- 172.21.15.86:0/3554329778 learned_addr learned my addr 172.21.15.86:0/3554329778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d52a00640 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b7fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d53400640 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b7fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d52000640 1 -- 172.21.15.86:0/3554329778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 msgr2=0x55c31b7fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d52000640 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b7fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d52000640 1 -- 172.21.15.86:0/3554329778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 msgr2=0x55c31b7fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d52000640 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b7fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d52000640 1 -- 172.21.15.86:0/3554329778 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31b4733f0 con 0x55c31b7fb990 2024-04-17T07:28:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d52a00640 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b7fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:00.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.361+0000 7f0d53400640 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b7fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:00.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.362+0000 7f0d52000640 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b80ca20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d4002c6a0 tx=0x7f0d40040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3ea0d4546752c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.363+0000 7f0d3f400640 1 -- 172.21.15.86:0/3554329778 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40044040 con 0x55c31b7fb990 2024-04-17T07:28:00.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.363+0000 7f0d3f400640 1 -- 172.21.15.86:0/3554329778 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d4c00fbf0 con 0x55c31b7fb990 2024-04-17T07:28:00.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.363+0000 7f0d3f400640 1 -- 172.21.15.86:0/3554329778 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40040a00 con 0x55c31b7fb990 2024-04-17T07:28:00.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.364+0000 7f0d5c587740 1 -- 172.21.15.86:0/3554329778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 msgr2=0x55c31b80ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.364+0000 7f0d5c587740 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b80ca20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d4002c6a0 tx=0x7f0d40040000 comp rx=0 tx=0).stop 2024-04-17T07:28:00.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.365+0000 7f0d5c587740 1 -- 172.21.15.86:0/3554329778 shutdown_connections 2024-04-17T07:28:00.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.365+0000 7f0d5c587740 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b800b00 0x55c31b7fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.365+0000 7f0d5c587740 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b7fb990 0x55c31b80ca20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.365+0000 7f0d5c587740 1 --2- 172.21.15.86:0/3554329778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fafa0 0x55c31b7fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.365+0000 7f0d5c587740 1 -- 172.21.15.86:0/3554329778 >> 172.21.15.86:0/3554329778 conn(0x55c31b7f6510 msgr2=0x55c31b803fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.366+0000 7f0d5c587740 1 -- 172.21.15.86:0/3554329778 shutdown_connections 2024-04-17T07:28:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.366+0000 7f0d5c587740 1 -- 172.21.15.86:0/3554329778 wait complete. 2024-04-17T07:28:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.367+0000 7f0d5c587740 1 -- start start 2024-04-17T07:28:00.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.367+0000 7f0d5c587740 1 Processor -- start 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.368+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b7fafa0 0x55c31b807d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.368+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fb990 0x55c31b8082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.368+0000 7f0d5c587740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b800b00 0x55c31b80b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.368+0000 7f0d5c587740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541b2590 con 0x55c31b7fafa0 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.368+0000 7f0d5c587740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541b1ec0 con 0x55c31b7fb990 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.368+0000 7f0d5c587740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d541af0f0 con 0x55c31b800b00 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.368+0000 7f0d5c587740 10 client.0 _ll_register_callbacks cb 0x7f0d40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.369+0000 7f0d52000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fb990 0x55c31b8082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.369+0000 7f0d52000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fb990 0x55c31b8082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39024/0 (socket says 172.21.15.86:39024) 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.369+0000 7f0d52000640 1 -- 172.21.15.86:0/1726262616 learned_addr learned my addr 172.21.15.86:0/1726262616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.369+0000 7f0d52a00640 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b7fafa0 0x55c31b807d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.370+0000 7f0d52000640 1 -- 172.21.15.86:0/1726262616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b800b00 msgr2=0x55c31b80b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.370+0000 7f0d53400640 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b800b00 0x55c31b80b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.370+0000 7f0d52000640 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b800b00 0x55c31b80b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.370+0000 7f0d52000640 1 -- 172.21.15.86:0/1726262616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b7fafa0 msgr2=0x55c31b807d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.370+0000 7f0d52000640 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b7fafa0 0x55c31b807d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.370+0000 7f0d52000640 1 -- 172.21.15.86:0/1726262616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31b53d7d0 con 0x55c31b7fb990 2024-04-17T07:28:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.370+0000 7f0d52a00640 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b7fafa0 0x55c31b807d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.371+0000 7f0d52000640 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fb990 0x55c31b8082a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0d4002c240 tx=0x7f0d40041db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f48cdcbcc5d4fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.371+0000 7f0d3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.372+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1726262616 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4005e030 con 0x55c31b7fb990 2024-04-17T07:28:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.372+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1726262616 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0d40003800 con 0x55c31b7fb990 2024-04-17T07:28:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.372+0000 7f0d5c587740 1 -- 172.21.15.86:0/1726262616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d4c00fbf0 con 0x55c31b7fb990 2024-04-17T07:28:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.379+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1726262616 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d40044040 con 0x55c31b7fb990 2024-04-17T07:28:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.380+0000 7f0d5c587740 10 client.37583 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.380+0000 7f0d5c587740 1 -- 172.21.15.86:0/1726262616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c31b80c950 con 0x55c31b7fb990 2024-04-17T07:28:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.380+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1726262616 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d4c00fbf0 con 0x55c31b7fb990 2024-04-17T07:28:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.380+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1726262616 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0d40052860 con 0x55c31b7fb990 2024-04-17T07:28:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.380+0000 7f0d5c587740 10 client.37583 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.380+0000 7f0d5c587740 10 client.37583 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:28:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.380+0000 7f0d5c587740 20 client.37583 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.381+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55c31b8995f0 0x55c31b89b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.381+0000 7f0d5c587740 4 client.37583 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:28:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.381+0000 7f0d5c587740 1 -- 172.21.15.86:0/1726262616 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c31b7688b0 con 0x55c31b8995f0 2024-04-17T07:28:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.381+0000 7f0d52a00640 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55c31b8995f0 0x55c31b89b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.382+0000 7f0d3cc00640 10 client.37583 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:28:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.382+0000 7f0d52a00640 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55c31b8995f0 0x55c31b89b9e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=18228408238ade5e server_cookie=757824cb6f908610 in_seq=0 out_seq=0 2024-04-17T07:28:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.382+0000 7f0d3cc00640 1 -- 172.21.15.86:0/1726262616 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c31b7688b0 con 0x55c31b8995f0 2024-04-17T07:28:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.382+0000 7f0d3cc00640 10 client.37583 handle_command_reply: tid=0 2024-04-17T07:28:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.382+0000 7f0d5c587740 1 client.37583 shutdown 2024-04-17T07:28:00.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.382+0000 7f0d5c587740 1 -- 172.21.15.86:0/1726262616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fb990 msgr2=0x55c31b8082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.382+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fb990 0x55c31b8082a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0d4002c240 tx=0x7f0d40041db0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.383+0000 7f0d5c587740 1 -- 172.21.15.86:0/1726262616 shutdown_connections 2024-04-17T07:28:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.383+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55c31b8995f0 0x55c31b89b9e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.383+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c31b800b00 0x55c31b80b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.383+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c31b7fb990 0x55c31b8082a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.383+0000 7f0d5c587740 1 --2- 172.21.15.86:0/1726262616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c31b7fafa0 0x55c31b807d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.383+0000 7f0d3cc00640 0 client.37583 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:28:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.383+0000 7f0d5c587740 1 -- 172.21.15.86:0/1726262616 >> 172.21.15.86:0/1726262616 conn(0x55c31b7f6510 msgr2=0x55c31b8054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.383+0000 7f0d5c587740 1 -- 172.21.15.86:0/1726262616 shutdown_connections 2024-04-17T07:28:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.383+0000 7f0d5c587740 1 -- 172.21.15.86:0/1726262616 wait complete. 2024-04-17T07:28:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.383+0000 7f0d5c587740 20 client.37583 trim_cache size 0 max 16384 2024-04-17T07:28:00.384 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:28:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 -- 172.21.15.86:0/3361523672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d380a3bb0 msgr2=0x7f0d380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d380a3bb0 0x7f0d380a6070 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c022000 tx=0x7f0d4c023000 comp rx=0 tx=0).stop 2024-04-17T07:28:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 -- 172.21.15.86:0/3361523672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d54076920 msgr2=0x7f0d5419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d54076920 0x7f0d5419e800 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400e9f0 tx=0x7f0d4400edf0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 -- 172.21.15.86:0/3361523672 shutdown_connections 2024-04-17T07:28:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0d380a3bb0 0x7f0d380a6070 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d54078500 0x7f0d541a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d54077b60 0x7f0d5419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 --2- 172.21.15.86:0/3361523672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d54076920 0x7f0d5419e800 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 -- 172.21.15.86:0/3361523672 >> 172.21.15.86:0/3361523672 conn(0x7f0d540ff710 msgr2=0x7f0d54101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 -- 172.21.15.86:0/3361523672 shutdown_connections 2024-04-17T07:28:00.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.384+0000 7f0d59800640 1 -- 172.21.15.86:0/3361523672 wait complete. 2024-04-17T07:28:00.410 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:28:00.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.712+0000 7f7f2ea00640 1 -- 172.21.15.86:0/3671197149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281097b0 msgr2=0x7f7f28109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.712+0000 7f7f2ea00640 1 --2- 172.21.15.86:0/3671197149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281097b0 0x7f7f28109c30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009900 tx=0x7f7f1402bc50 comp rx=0 tx=0).stop 2024-04-17T07:28:00.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.713+0000 7f7f2ea00640 1 -- 172.21.15.86:0/3671197149 shutdown_connections 2024-04-17T07:28:00.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.713+0000 7f7f2ea00640 1 --2- 172.21.15.86:0/3671197149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f2810a170 0x7f7f28110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.713+0000 7f7f2ea00640 1 --2- 172.21.15.86:0/3671197149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281097b0 0x7f7f28109c30 secure :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009900 tx=0x7f7f1402bc50 comp rx=0 tx=0).stop 2024-04-17T07:28:00.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.713+0000 7f7f2ea00640 1 --2- 172.21.15.86:0/3671197149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f281085c0 0x7f7f281089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.713+0000 7f7f2ea00640 1 -- 172.21.15.86:0/3671197149 >> 172.21.15.86:0/3671197149 conn(0x7f7f28077560 msgr2=0x7f7f280799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.713+0000 7f7f2ea00640 1 -- 172.21.15.86:0/3671197149 shutdown_connections 2024-04-17T07:28:00.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.713+0000 7f7f2ea00640 1 -- 172.21.15.86:0/3671197149 wait complete. 2024-04-17T07:28:00.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.714+0000 7f7f2ea00640 1 Processor -- start 2024-04-17T07:28:00.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.715+0000 7f7f2ea00640 1 -- start start 2024-04-17T07:28:00.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.716+0000 7f7f2ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281085c0 0x7f7f2819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.716+0000 7f7f2ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f2810a170 0x7f7f2819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.716+0000 7f7f2ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f281a4020 0x7f7f281a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.716+0000 7f7f2ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28114130 con 0x7f7f281085c0 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.716+0000 7f7f2ea00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28113fb0 con 0x7f7f281a4020 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.716+0000 7f7f2ea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281142b0 con 0x7f7f2810a170 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.716+0000 7f7f2ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f281a4020 0x7f7f281a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.716+0000 7f7f2ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f281a4020 0x7f7f281a6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39040/0 (socket says 172.21.15.86:39040) 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.716+0000 7f7f2ca00640 1 -- 172.21.15.86:0/692338319 learned_addr learned my addr 172.21.15.86:0/692338319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.717+0000 7f7f27400640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f2810a170 0x7f7f2819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.717+0000 7f7f2ca00640 1 -- 172.21.15.86:0/692338319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f2810a170 msgr2=0x7f7f2819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.718+0000 7f7f2ca00640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f2810a170 0x7f7f2819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.718+0000 7f7f2ca00640 1 -- 172.21.15.86:0/692338319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281085c0 msgr2=0x7f7f2819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.718+0000 7f7f2ca00640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281085c0 0x7f7f2819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.718+0000 7f7f2ca00640 1 -- 172.21.15.86:0/692338319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f281a6a80 con 0x7f7f281a4020 2024-04-17T07:28:00.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.718+0000 7f7f27400640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f2810a170 0x7f7f2819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:00.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.718+0000 7f7f2ca00640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f281a4020 0x7f7f281a6410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000cad0 tx=0x7f7f2000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.719+0000 7f7f24c00640 1 -- 172.21.15.86:0/692338319 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000ae20 con 0x7f7f281a4020 2024-04-17T07:28:00.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.720+0000 7f7f24c00640 1 -- 172.21.15.86:0/692338319 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f20005020 con 0x7f7f281a4020 2024-04-17T07:28:00.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.720+0000 7f7f2ea00640 1 -- 172.21.15.86:0/692338319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f281a6d70 con 0x7f7f281a4020 2024-04-17T07:28:00.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.720+0000 7f7f2ea00640 1 -- 172.21.15.86:0/692338319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f281a7260 con 0x7f7f281a4020 2024-04-17T07:28:00.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.720+0000 7f7f24c00640 1 -- 172.21.15.86:0/692338319 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000f6b0 con 0x7f7f281a4020 2024-04-17T07:28:00.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.727+0000 7f7f31620740 1 Processor -- start 2024-04-17T07:28:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.728+0000 7f7f24c00640 1 -- 172.21.15.86:0/692338319 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7f200051d0 con 0x7f7f281a4020 2024-04-17T07:28:00.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.728+0000 7f7f31620740 1 -- start start 2024-04-17T07:28:00.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.729+0000 7f7f31620740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 0x55977c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.729+0000 7f7f31620740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d0990 0x55977c0d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.729+0000 7f7f31620740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 0x55977c0d7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.729+0000 7f7f31620740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55977c036020 con 0x55977c0cffa0 2024-04-17T07:28:00.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.729+0000 7f7f31620740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55977bfdc460 con 0x55977c0d0990 2024-04-17T07:28:00.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.729+0000 7f7f31620740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55977c031fe0 con 0x55977c0d5b00 2024-04-17T07:28:00.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.729+0000 7f7f27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 0x55977c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.729+0000 7f7f27e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d0990 0x55977c0d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.729+0000 7f7f27e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d0990 0x55977c0d5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39044/0 (socket says 172.21.15.86:39044) 2024-04-17T07:28:00.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.729+0000 7f7f27e00640 1 -- 172.21.15.86:0/1788624565 learned_addr learned my addr 172.21.15.86:0/1788624565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:00.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.730+0000 7f7f27400640 1 --2- 172.21.15.86:0/1788624565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 0x55977c0d7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.730+0000 7f7f27e00640 1 -- 172.21.15.86:0/1788624565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 msgr2=0x55977c0d7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.730+0000 7f7f27e00640 1 --2- 172.21.15.86:0/1788624565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 0x55977c0d7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.730+0000 7f7f27e00640 1 -- 172.21.15.86:0/1788624565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 msgr2=0x55977c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.730+0000 7f7f27e00640 1 --2- 172.21.15.86:0/1788624565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 0x55977c0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.730+0000 7f7f27e00640 1 -- 172.21.15.86:0/1788624565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977bfe4040 con 0x55977c0d0990 2024-04-17T07:28:00.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.730+0000 7f7f24c00640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f0c0a3c30 0x7f7f0c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.730+0000 7f7f24c00640 1 -- 172.21.15.86:0/692338319 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7f200d07e0 con 0x7f7f281a4020 2024-04-17T07:28:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.730+0000 7f7f27400640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f0c0a3c30 0x7f7f0c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.730+0000 7f7f27e00640 1 --2- 172.21.15.86:0/1788624565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d0990 0x55977c0d5490 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800bb20 tx=0x7f7f1800d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7ac90e7060d825 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.731+0000 7f7f1d200640 1 -- 172.21.15.86:0/1788624565 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18016020 con 0x55977c0d0990 2024-04-17T07:28:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.731+0000 7f7f1d200640 1 -- 172.21.15.86:0/1788624565 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f1800dd10 con 0x55977c0d0990 2024-04-17T07:28:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.731+0000 7f7f1d200640 1 -- 172.21.15.86:0/1788624565 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1800f7c0 con 0x55977c0d0990 2024-04-17T07:28:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.732+0000 7f7f31620740 1 -- 172.21.15.86:0/1788624565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d0990 msgr2=0x55977c0d5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.732+0000 7f7f27400640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f0c0a3c30 0x7f7f0c0a60f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7f1402c6c0 tx=0x7f7f14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.732+0000 7f7f31620740 1 --2- 172.21.15.86:0/1788624565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d0990 0x55977c0d5490 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800bb20 tx=0x7f7f1800d2e0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.732+0000 7f7f31620740 1 -- 172.21.15.86:0/1788624565 shutdown_connections 2024-04-17T07:28:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.732+0000 7f7f31620740 1 --2- 172.21.15.86:0/1788624565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 0x55977c0d7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.732+0000 7f7f31620740 1 --2- 172.21.15.86:0/1788624565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d0990 0x55977c0d5490 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.732+0000 7f7f31620740 1 --2- 172.21.15.86:0/1788624565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 0x55977c0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.732+0000 7f7f31620740 1 -- 172.21.15.86:0/1788624565 >> 172.21.15.86:0/1788624565 conn(0x55977c0cb510 msgr2=0x55977c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.733+0000 7f7f31620740 1 -- 172.21.15.86:0/1788624565 shutdown_connections 2024-04-17T07:28:00.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.733+0000 7f7f31620740 1 -- 172.21.15.86:0/1788624565 wait complete. 2024-04-17T07:28:00.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.734+0000 7f7f31620740 1 -- start start 2024-04-17T07:28:00.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.734+0000 7f7f31620740 1 Processor -- start 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.735+0000 7f7f31620740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 0x55977c0da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.735+0000 7f7f31620740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d0990 0x55977c0daa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.735+0000 7f7f31620740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 0x55977c0e19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.735+0000 7f7f31620740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55977c0d8ab0 con 0x55977c0cffa0 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.735+0000 7f7f31620740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55977bfef830 con 0x55977c0d5b00 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.735+0000 7f7f31620740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55977c0d8c30 con 0x55977c0d0990 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.735+0000 7f7f31620740 10 client.0 _ll_register_callbacks cb 0x55977bfbdfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.736+0000 7f7f27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 0x55977c0da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.736+0000 7f7f27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 0x55977c0da4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41436/0 (socket says 172.21.15.86:41436) 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.736+0000 7f7f27e00640 1 -- 172.21.15.86:0/1932535492 learned_addr learned my addr 172.21.15.86:0/1932535492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:00.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.736+0000 7f7f27400640 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d0990 0x55977c0daa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.736+0000 7f7f2ca00640 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 0x55977c0e19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.736+0000 7f7f2ca00640 1 -- 172.21.15.86:0/1932535492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d0990 msgr2=0x55977c0daa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.736+0000 7f7f2ca00640 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d0990 0x55977c0daa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.736+0000 7f7f2ca00640 1 -- 172.21.15.86:0/1932535492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 msgr2=0x55977c0da4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.736+0000 7f7f2ca00640 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 0x55977c0da4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.736+0000 7f7f2ca00640 1 -- 172.21.15.86:0/1932535492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55977bff3920 con 0x55977c0d5b00 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.737+0000 7f7f2ca00640 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 0x55977c0e19f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7f200cdf60 tx=0x7f7f2009cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b74daad5baeae87 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.737+0000 7f7f11600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.737+0000 7f7f11600640 1 -- 172.21.15.86:0/1932535492 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2001d040 con 0x55977c0d5b00 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.737+0000 7f7f11600640 1 -- 172.21.15.86:0/1932535492 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7f20005560 con 0x55977c0d5b00 2024-04-17T07:28:00.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.737+0000 7f7f31620740 1 -- 172.21.15.86:0/1932535492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f1800dd10 con 0x55977c0d5b00 2024-04-17T07:28:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.738+0000 7f7f11600640 1 -- 172.21.15.86:0/1932535492 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f200a53b0 con 0x55977c0d5b00 2024-04-17T07:28:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.738+0000 7f7f31620740 10 client.37595 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.738+0000 7f7f31620740 1 -- 172.21.15.86:0/1932535492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55977bfc17f0 con 0x55977c0d5b00 2024-04-17T07:28:00.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.738+0000 7f7f11600640 1 -- 172.21.15.86:0/1932535492 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f1800dd10 con 0x55977c0d5b00 2024-04-17T07:28:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.739+0000 7f7f11600640 1 -- 172.21.15.86:0/1932535492 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7f2009b800 con 0x55977c0d5b00 2024-04-17T07:28:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.739+0000 7f7f31620740 10 client.37595 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.739+0000 7f7f31620740 10 client.37595 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:28:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.740+0000 7f7f31620740 20 client.37595 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.740+0000 7f7f31620740 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55977c0f6ed0 0x55977c0f9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.740+0000 7f7f31620740 4 client.37595 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:00.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.740+0000 7f7f31620740 1 -- 172.21.15.86:0/1932535492 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55977c043a40 con 0x55977c0f6ed0 2024-04-17T07:28:00.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.741+0000 7f7f27e00640 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55977c0f6ed0 0x55977c0f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.742+0000 7f7f27e00640 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55977c0f6ed0 0x55977c0f9330 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac3960c4251d0281 server_cookie=1addd93eca4c666c in_seq=0 out_seq=0 2024-04-17T07:28:00.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.742+0000 7f7f11600640 10 client.37595 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:28:00.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.743+0000 7f7f11600640 1 -- 172.21.15.86:0/1932535492 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55977c043a40 con 0x55977c0f6ed0 2024-04-17T07:28:00.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.743+0000 7f7f11600640 10 client.37595 handle_command_reply: tid=0 2024-04-17T07:28:00.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.744+0000 7f7f31620740 1 client.37595 shutdown 2024-04-17T07:28:00.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.744+0000 7f7f31620740 1 -- 172.21.15.86:0/1932535492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 msgr2=0x55977c0e19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.744+0000 7f7f31620740 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 0x55977c0e19f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7f200cdf60 tx=0x7f7f2009cf80 comp rx=0 tx=0).stop 2024-04-17T07:28:00.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.744+0000 7f7f31620740 1 -- 172.21.15.86:0/1932535492 shutdown_connections 2024-04-17T07:28:00.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.744+0000 7f7f31620740 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55977c0f6ed0 0x55977c0f9330 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.744+0000 7f7f31620740 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 0x55977c0e19f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.744+0000 7f7f31620740 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d0990 0x55977c0daa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.744+0000 7f7f31620740 1 --2- 172.21.15.86:0/1932535492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0cffa0 0x55977c0da4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.744+0000 7f7f31620740 1 -- 172.21.15.86:0/1932535492 >> 172.21.15.86:0/1932535492 conn(0x55977c0cb510 msgr2=0x55977c0ca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.744+0000 7f7f11600640 0 client.37595 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:28:00.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.745+0000 7f7f31620740 1 -- 172.21.15.86:0/1932535492 shutdown_connections 2024-04-17T07:28:00.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.745+0000 7f7f31620740 1 -- 172.21.15.86:0/1932535492 wait complete. 2024-04-17T07:28:00.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.745+0000 7f7f31620740 20 client.37595 trim_cache size 0 max 16384 2024-04-17T07:28:00.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:00 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1225327096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:28:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.775+0000 7f7f31620740 1 Processor -- start 2024-04-17T07:28:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f31620740 1 -- start start 2024-04-17T07:28:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f31620740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0cffa0 0x55977c0c6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f31620740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0e19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f31620740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 0x55977c0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f31620740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0800fb60 con 0x55977c0d0990 2024-04-17T07:28:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f31620740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7f080116a0 con 0x55977c0cffa0 2024-04-17T07:28:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f31620740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08010230 con 0x55977c0d5b00 2024-04-17T07:28:00.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:00 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1225327096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:28:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0e19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f2ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 0x55977c0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f27e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0cffa0 0x55977c0c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.776+0000 7f7f2ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 0x55977c0c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58616/0 (socket says 172.21.15.86:58616) 2024-04-17T07:28:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.777+0000 7f7f2ca00640 1 -- 172.21.15.86:0/165793966 learned_addr learned my addr 172.21.15.86:0/165793966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.777+0000 7f7f27e00640 1 -- 172.21.15.86:0/165793966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 msgr2=0x55977c0c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.777+0000 7f7f27e00640 1 --2- 172.21.15.86:0/165793966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 0x55977c0c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.777+0000 7f7f27e00640 1 -- 172.21.15.86:0/165793966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 msgr2=0x55977c0e19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.777+0000 7f7f27e00640 1 --2- 172.21.15.86:0/165793966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0e19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.777+0000 7f7f27e00640 1 -- 172.21.15.86:0/165793966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977bd483f0 con 0x55977c0cffa0 2024-04-17T07:28:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.777+0000 7f7f27400640 1 --2- 172.21.15.86:0/165793966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0e19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.778+0000 7f7f27e00640 1 --2- 172.21.15.86:0/165793966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0cffa0 0x55977c0c6570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7f18005940 tx=0x7f7f180058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d27c78f489e7d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.778+0000 7f7f13e00640 1 -- 172.21.15.86:0/165793966 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18016040 con 0x55977c0cffa0 2024-04-17T07:28:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.778+0000 7f7f13e00640 1 -- 172.21.15.86:0/165793966 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f18014070 con 0x55977c0cffa0 2024-04-17T07:28:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.778+0000 7f7f13e00640 1 -- 172.21.15.86:0/165793966 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f180047a0 con 0x55977c0cffa0 2024-04-17T07:28:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.779+0000 7f7f31620740 1 -- 172.21.15.86:0/165793966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0cffa0 msgr2=0x55977c0c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.779+0000 7f7f31620740 1 --2- 172.21.15.86:0/165793966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0cffa0 0x55977c0c6570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7f18005940 tx=0x7f7f180058d0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.780+0000 7f7f31620740 1 -- 172.21.15.86:0/165793966 shutdown_connections 2024-04-17T07:28:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.780+0000 7f7f31620740 1 --2- 172.21.15.86:0/165793966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0d5b00 0x55977c0c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.780+0000 7f7f31620740 1 --2- 172.21.15.86:0/165793966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0e19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.780+0000 7f7f31620740 1 --2- 172.21.15.86:0/165793966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0cffa0 0x55977c0c6570 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.780+0000 7f7f31620740 1 -- 172.21.15.86:0/165793966 >> 172.21.15.86:0/165793966 conn(0x55977c0cb510 msgr2=0x55977c0f8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.780+0000 7f7f31620740 1 -- 172.21.15.86:0/165793966 shutdown_connections 2024-04-17T07:28:00.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.780+0000 7f7f31620740 1 -- 172.21.15.86:0/165793966 wait complete. 2024-04-17T07:28:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.781+0000 7f7f31620740 1 -- start start 2024-04-17T07:28:00.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.781+0000 7f7f31620740 1 Processor -- start 2024-04-17T07:28:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.781+0000 7f7f31620740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0cffa0 0x55977c0df0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.781+0000 7f7f31620740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.781+0000 7f7f31620740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 0x55977c0e05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.782+0000 7f7f31620740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08010fd0 con 0x55977c0d0990 2024-04-17T07:28:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.782+0000 7f7f31620740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7f08010900 con 0x55977c0d5b00 2024-04-17T07:28:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.782+0000 7f7f31620740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0800db30 con 0x55977c0cffa0 2024-04-17T07:28:00.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.782+0000 7f7f31620740 10 client.0 _ll_register_callbacks cb 0x7f7f2009ce40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.782+0000 7f7f27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.782+0000 7f7f27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0df5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41448/0 (socket says 172.21.15.86:41448) 2024-04-17T07:28:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.782+0000 7f7f27400640 1 -- 172.21.15.86:0/4118346950 learned_addr learned my addr 172.21.15.86:0/4118346950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.782+0000 7f7f2ca00640 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 0x55977c0e05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.782+0000 7f7f27e00640 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0cffa0 0x55977c0df0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.783+0000 7f7f27400640 1 -- 172.21.15.86:0/4118346950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0cffa0 msgr2=0x55977c0df0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.783+0000 7f7f27400640 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0cffa0 0x55977c0df0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.783+0000 7f7f27400640 1 -- 172.21.15.86:0/4118346950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 msgr2=0x55977c0e05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.783+0000 7f7f27400640 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 0x55977c0e05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.783+0000 7f7f27400640 1 -- 172.21.15.86:0/4118346950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55977be127d0 con 0x55977c0d0990 2024-04-17T07:28:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.783+0000 7f7f2ca00640 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 0x55977c0e05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.783+0000 7f7f27400640 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0df5f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f7f28109240 tx=0x7f7f14005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd03a6e60179c626 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.783+0000 7f7f11600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:28:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.784+0000 7f7f11600640 1 -- 172.21.15.86:0/4118346950 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14051030 con 0x55977c0d0990 2024-04-17T07:28:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.784+0000 7f7f11600640 1 -- 172.21.15.86:0/4118346950 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7f1403a040 con 0x55977c0d0990 2024-04-17T07:28:00.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.784+0000 7f7f31620740 1 -- 172.21.15.86:0/4118346950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f18014070 con 0x55977c0d0990 2024-04-17T07:28:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.785+0000 7f7f11600640 1 -- 172.21.15.86:0/4118346950 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f1405c050 con 0x55977c0d0990 2024-04-17T07:28:00.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.785+0000 7f7f31620740 10 client.18702 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:00.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.786+0000 7f7f31620740 1 -- 172.21.15.86:0/4118346950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55977c16e4c0 con 0x55977c0d0990 2024-04-17T07:28:00.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.786+0000 7f7f11600640 1 -- 172.21.15.86:0/4118346950 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f18014070 con 0x55977c0d0990 2024-04-17T07:28:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.787+0000 7f7f11600640 1 -- 172.21.15.86:0/4118346950 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7f1404b040 con 0x55977c0d0990 2024-04-17T07:28:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.787+0000 7f7f31620740 10 client.18702 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.787+0000 7f7f31620740 10 client.18702 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:28:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.787+0000 7f7f31620740 20 client.18702 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.787+0000 7f7f31620740 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55977c16edd0 0x55977c1711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.787+0000 7f7f31620740 4 client.18702 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:28:00.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.787+0000 7f7f31620740 1 -- 172.21.15.86:0/4118346950 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55977c03d8b0 con 0x55977c16edd0 2024-04-17T07:28:00.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.788+0000 7f7f27e00640 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55977c16edd0 0x55977c1711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.789+0000 7f7f27e00640 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55977c16edd0 0x55977c1711f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb16f65ff4b9b88f server_cookie=84b541331cc44569 in_seq=0 out_seq=0 2024-04-17T07:28:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.789+0000 7f7f11600640 10 client.18702 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:28:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.789+0000 7f7f11600640 1 -- 172.21.15.86:0/4118346950 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55977c03d8b0 con 0x55977c16edd0 2024-04-17T07:28:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.789+0000 7f7f11600640 10 client.18702 handle_command_reply: tid=0 2024-04-17T07:28:00.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.789+0000 7f7f31620740 1 client.18702 shutdown 2024-04-17T07:28:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.790+0000 7f7f31620740 1 -- 172.21.15.86:0/4118346950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 msgr2=0x55977c0df5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.790+0000 7f7f31620740 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0df5f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f7f28109240 tx=0x7f7f14005fd0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.790+0000 7f7f31620740 1 -- 172.21.15.86:0/4118346950 shutdown_connections 2024-04-17T07:28:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.790+0000 7f7f31620740 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55977c16edd0 0x55977c1711f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.790+0000 7f7f31620740 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55977c0d5b00 0x55977c0e05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.790+0000 7f7f31620740 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55977c0d0990 0x55977c0df5f0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.790+0000 7f7f31620740 1 --2- 172.21.15.86:0/4118346950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55977c0cffa0 0x55977c0df0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.790+0000 7f7f31620740 1 -- 172.21.15.86:0/4118346950 >> 172.21.15.86:0/4118346950 conn(0x55977c0cb510 msgr2=0x55977c0ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.790+0000 7f7f11600640 0 client.18702 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:28:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.790+0000 7f7f31620740 1 -- 172.21.15.86:0/4118346950 shutdown_connections 2024-04-17T07:28:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.791+0000 7f7f31620740 1 -- 172.21.15.86:0/4118346950 wait complete. 2024-04-17T07:28:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.791+0000 7f7f31620740 20 client.18702 trim_cache size 0 max 16384 2024-04-17T07:28:00.791 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:28:00.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.792+0000 7f7f2ea00640 1 -- 172.21.15.86:0/692338319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f0c0a3c30 msgr2=0x7f7f0c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.792+0000 7f7f2ea00640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f0c0a3c30 0x7f7f0c0a60f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7f1402c6c0 tx=0x7f7f14000bc0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.792+0000 7f7f2ea00640 1 -- 172.21.15.86:0/692338319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f281a4020 msgr2=0x7f7f281a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.792+0000 7f7f2ea00640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f281a4020 0x7f7f281a6410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000cad0 tx=0x7f7f2000cf90 comp rx=0 tx=0).stop 2024-04-17T07:28:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.792+0000 7f7f2ea00640 1 -- 172.21.15.86:0/692338319 shutdown_connections 2024-04-17T07:28:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.792+0000 7f7f2ea00640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7f0c0a3c30 0x7f7f0c0a60f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.792+0000 7f7f2ea00640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f281a4020 0x7f7f281a6410 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.792+0000 7f7f2ea00640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f2810a170 0x7f7f2819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.792+0000 7f7f2ea00640 1 --2- 172.21.15.86:0/692338319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f281085c0 0x7f7f2819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.792+0000 7f7f2ea00640 1 -- 172.21.15.86:0/692338319 >> 172.21.15.86:0/692338319 conn(0x7f7f28077560 msgr2=0x7f7f2810cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.793+0000 7f7f2ea00640 1 -- 172.21.15.86:0/692338319 shutdown_connections 2024-04-17T07:28:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:00.793+0000 7f7f2ea00640 1 -- 172.21.15.86:0/692338319 wait complete. 2024-04-17T07:28:00.818 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:00 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1225327096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:28:00.826 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:28:01.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.135+0000 7fef81a00640 1 -- 172.21.15.86:0/887020585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef7c101fb0 msgr2=0x7fef7c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.135+0000 7fef81a00640 1 --2- 172.21.15.86:0/887020585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef7c101fb0 0x7fef7c1023d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fef68009860 tx=0x7fef6802bb70 comp rx=0 tx=0).stop 2024-04-17T07:28:01.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.136+0000 7fef81a00640 1 -- 172.21.15.86:0/887020585 shutdown_connections 2024-04-17T07:28:01.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.136+0000 7fef81a00640 1 --2- 172.21.15.86:0/887020585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7c103b60 0x7fef7c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.136+0000 7fef81a00640 1 --2- 172.21.15.86:0/887020585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef7c1031a0 0x7fef7c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.136+0000 7fef81a00640 1 --2- 172.21.15.86:0/887020585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef7c101fb0 0x7fef7c1023d0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.136+0000 7fef81a00640 1 -- 172.21.15.86:0/887020585 >> 172.21.15.86:0/887020585 conn(0x7fef7c0fd540 msgr2=0x7fef7c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.136+0000 7fef81a00640 1 -- 172.21.15.86:0/887020585 shutdown_connections 2024-04-17T07:28:01.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.136+0000 7fef81a00640 1 -- 172.21.15.86:0/887020585 wait complete. 2024-04-17T07:28:01.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.138+0000 7fef81a00640 1 Processor -- start 2024-04-17T07:28:01.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.138+0000 7fef81a00640 1 -- start start 2024-04-17T07:28:01.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.138+0000 7fef81a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef7c101fb0 0x7fef7c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.138+0000 7fef81a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7c1031a0 0x7fef7c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.138+0000 7fef81a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef7c103b60 0x7fef7c1a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.138+0000 7fef81a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c10dab0 con 0x7fef7c103b60 2024-04-17T07:28:01.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.139+0000 7fef81a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c10d930 con 0x7fef7c1031a0 2024-04-17T07:28:01.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.139+0000 7fef81a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c10dc30 con 0x7fef7c101fb0 2024-04-17T07:28:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.139+0000 7fef7a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7c1031a0 0x7fef7c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.139+0000 7fef7a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7c1031a0 0x7fef7c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39096/0 (socket says 172.21.15.86:39096) 2024-04-17T07:28:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.139+0000 7fef7a000640 1 -- 172.21.15.86:0/214066907 learned_addr learned my addr 172.21.15.86:0/214066907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.139+0000 7fef7aa00640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef7c101fb0 0x7fef7c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.140+0000 7fef7b400640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef7c103b60 0x7fef7c1a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.140+0000 7fef7a000640 1 -- 172.21.15.86:0/214066907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef7c101fb0 msgr2=0x7fef7c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.140+0000 7fef7a000640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef7c101fb0 0x7fef7c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.140+0000 7fef7a000640 1 -- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef7c103b60 msgr2=0x7fef7c1a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.140+0000 7fef7a000640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef7c103b60 0x7fef7c1a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.140+0000 7fef7a000640 1 -- 172.21.15.86:0/214066907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c1932c0 con 0x7fef7c1031a0 2024-04-17T07:28:01.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.140+0000 7fef7b400640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef7c103b60 0x7fef7c1a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:01.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.141+0000 7fef7a000640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7c1031a0 0x7fef7c19eda0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fef7400eab0 tx=0x7fef7400ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.141+0000 7fef73400640 1 -- 172.21.15.86:0/214066907 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7400ce40 con 0x7fef7c1031a0 2024-04-17T07:28:01.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.141+0000 7fef81a00640 1 -- 172.21.15.86:0/214066907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c1935b0 con 0x7fef7c1031a0 2024-04-17T07:28:01.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.141+0000 7fef73400640 1 -- 172.21.15.86:0/214066907 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef74011920 con 0x7fef7c1031a0 2024-04-17T07:28:01.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.141+0000 7fef73400640 1 -- 172.21.15.86:0/214066907 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef740109e0 con 0x7fef7c1031a0 2024-04-17T07:28:01.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.142+0000 7fef81a00640 1 -- 172.21.15.86:0/214066907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c193b50 con 0x7fef7c1031a0 2024-04-17T07:28:01.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.147+0000 7fef847b0740 1 Processor -- start 2024-04-17T07:28:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.149+0000 7fef847b0740 1 -- start start 2024-04-17T07:28:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.149+0000 7fef847b0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 0x556009a943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.149+0000 7fef847b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 0x556009a99490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.149+0000 7fef847b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a99b00 0x556009a9bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.149+0000 7fef847b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560099fa020 con 0x556009a94990 2024-04-17T07:28:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.149+0000 7fef847b0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5560099a0460 con 0x556009a93fa0 2024-04-17T07:28:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.149+0000 7fef847b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560099f5fe0 con 0x556009a99b00 2024-04-17T07:28:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.149+0000 7fef7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 0x556009a99490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.149+0000 7fef7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 0x556009a99490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41478/0 (socket says 172.21.15.86:41478) 2024-04-17T07:28:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.149+0000 7fef7aa00640 1 -- 172.21.15.86:0/1263995764 learned_addr learned my addr 172.21.15.86:0/1263995764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.150+0000 7fef73400640 1 -- 172.21.15.86:0/214066907 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fef74011ad0 con 0x7fef7c1031a0 2024-04-17T07:28:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.150+0000 7fef7aa00640 1 -- 172.21.15.86:0/1263995764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a99b00 msgr2=0x556009a9bf90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:28:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.150+0000 7fef7b400640 1 --2- 172.21.15.86:0/1263995764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 0x556009a943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.150+0000 7fef7aa00640 1 --2- 172.21.15.86:0/1263995764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a99b00 0x556009a9bf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.150+0000 7fef7aa00640 1 -- 172.21.15.86:0/1263995764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 msgr2=0x556009a943c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.150+0000 7fef7aa00640 1 --2- 172.21.15.86:0/1263995764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 0x556009a943c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.150+0000 7fef7aa00640 1 -- 172.21.15.86:0/1263995764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560099a8040 con 0x556009a94990 2024-04-17T07:28:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.150+0000 7fef7b400640 1 --2- 172.21.15.86:0/1263995764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 0x556009a943c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.151+0000 7fef7aa00640 1 --2- 172.21.15.86:0/1263995764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 0x556009a99490 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fef6802c180 tx=0x7fef68005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63f83c0aecba784 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.152+0000 7fef73400640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef600a3cb0 0x7fef600a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.152+0000 7fef73400640 1 -- 172.21.15.86:0/214066907 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fef740d0d70 con 0x7fef7c1031a0 2024-04-17T07:28:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.152+0000 7fef67e00640 1 -- 172.21.15.86:0/1263995764 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68043020 con 0x556009a94990 2024-04-17T07:28:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.153+0000 7fef67e00640 1 -- 172.21.15.86:0/1263995764 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef680346c0 con 0x556009a94990 2024-04-17T07:28:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.153+0000 7fef67e00640 1 -- 172.21.15.86:0/1263995764 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68039040 con 0x556009a94990 2024-04-17T07:28:01.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.153+0000 7fef7b400640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef600a3cb0 0x7fef600a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.154+0000 7fef7b400640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef600a3cb0 0x7fef600a6170 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fef6c005ab0 tx=0x7fef6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.154+0000 7fef847b0740 1 -- 172.21.15.86:0/1263995764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 msgr2=0x556009a99490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.155+0000 7fef847b0740 1 --2- 172.21.15.86:0/1263995764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 0x556009a99490 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fef6802c180 tx=0x7fef68005b20 comp rx=0 tx=0).stop 2024-04-17T07:28:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.155+0000 7fef847b0740 1 -- 172.21.15.86:0/1263995764 shutdown_connections 2024-04-17T07:28:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.155+0000 7fef847b0740 1 --2- 172.21.15.86:0/1263995764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a99b00 0x556009a9bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.155+0000 7fef847b0740 1 --2- 172.21.15.86:0/1263995764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 0x556009a99490 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.155+0000 7fef847b0740 1 --2- 172.21.15.86:0/1263995764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 0x556009a943c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.155+0000 7fef847b0740 1 -- 172.21.15.86:0/1263995764 >> 172.21.15.86:0/1263995764 conn(0x556009a8f510 msgr2=0x556009a8f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.156+0000 7fef847b0740 1 -- 172.21.15.86:0/1263995764 shutdown_connections 2024-04-17T07:28:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.156+0000 7fef847b0740 1 -- 172.21.15.86:0/1263995764 wait complete. 2024-04-17T07:28:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.156+0000 7fef847b0740 1 -- start start 2024-04-17T07:28:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.157+0000 7fef847b0740 1 Processor -- start 2024-04-17T07:28:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.157+0000 7fef847b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a93fa0 0x556009aaf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.157+0000 7fef847b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009aafa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.157+0000 7fef847b0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 0x556009ab6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.157+0000 7fef847b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556009aada60 con 0x556009a93fa0 2024-04-17T07:28:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.157+0000 7fef847b0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5560099b3830 con 0x556009a99b00 2024-04-17T07:28:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.157+0000 7fef847b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556009aadbe0 con 0x556009a94990 2024-04-17T07:28:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.157+0000 7fef847b0740 10 client.0 _ll_register_callbacks cb 0x556009981fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.158+0000 7fef7b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 0x556009ab6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.158+0000 7fef7a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009aafa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.158+0000 7fef7a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009aafa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58648/0 (socket says 172.21.15.86:58648) 2024-04-17T07:28:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.158+0000 7fef7b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 0x556009ab6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39104/0 (socket says 172.21.15.86:39104) 2024-04-17T07:28:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.158+0000 7fef7a000640 1 -- 172.21.15.86:0/2049395102 learned_addr learned my addr 172.21.15.86:0/2049395102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.158+0000 7fef7a000640 1 -- 172.21.15.86:0/2049395102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 msgr2=0x556009ab6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.158+0000 7fef7a000640 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 0x556009ab6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.158+0000 7fef7a000640 1 -- 172.21.15.86:0/2049395102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a93fa0 msgr2=0x556009aaf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.158+0000 7fef7a000640 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a93fa0 0x556009aaf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.158+0000 7fef7a000640 1 -- 172.21.15.86:0/2049395102 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560099b7920 con 0x556009a94990 2024-04-17T07:28:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.159+0000 7fef7a000640 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009aafa30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fef7409cc00 tx=0x7fef7409c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51fcd08091c76157 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.159+0000 7fef64c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:28:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.160+0000 7fef64c00640 1 -- 172.21.15.86:0/2049395102 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef740b6060 con 0x556009a94990 2024-04-17T07:28:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.160+0000 7fef64c00640 1 -- 172.21.15.86:0/2049395102 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fef7409ced0 con 0x556009a94990 2024-04-17T07:28:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.160+0000 7fef847b0740 1 -- 172.21.15.86:0/2049395102 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef680346c0 con 0x556009a94990 2024-04-17T07:28:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.160+0000 7fef64c00640 1 -- 172.21.15.86:0/2049395102 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef740bd2c0 con 0x556009a94990 2024-04-17T07:28:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.160+0000 7fef847b0740 10 client.27652 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.160+0000 7fef847b0740 1 -- 172.21.15.86:0/2049395102 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560099857f0 con 0x556009a94990 2024-04-17T07:28:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.161+0000 7fef64c00640 1 -- 172.21.15.86:0/2049395102 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef680346c0 con 0x556009a94990 2024-04-17T07:28:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.162+0000 7fef64c00640 1 -- 172.21.15.86:0/2049395102 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fef7400f040 con 0x556009a94990 2024-04-17T07:28:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.162+0000 7fef847b0740 10 client.27652 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.162+0000 7fef847b0740 10 client.27652 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:28:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.162+0000 7fef847b0740 20 client.27652 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.162+0000 7fef847b0740 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556009abaf10 0x556009abd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.162+0000 7fef847b0740 4 client.27652 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.162+0000 7fef847b0740 1 -- 172.21.15.86:0/2049395102 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556009a07a40 con 0x556009abaf10 2024-04-17T07:28:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.163+0000 7fef7aa00640 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556009abaf10 0x556009abd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.164+0000 7fef7aa00640 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556009abaf10 0x556009abd3d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85568f69d66ac54f server_cookie=43772df8fcfb00c7 in_seq=0 out_seq=0 2024-04-17T07:28:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.164+0000 7fef64c00640 10 client.27652 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:28:01.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.165+0000 7fef64c00640 1 -- 172.21.15.86:0/2049395102 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556009a07a40 con 0x556009abaf10 2024-04-17T07:28:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.166+0000 7fef64c00640 10 client.27652 handle_command_reply: tid=0 2024-04-17T07:28:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.166+0000 7fef847b0740 1 client.27652 shutdown 2024-04-17T07:28:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.168+0000 7fef847b0740 1 -- 172.21.15.86:0/2049395102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 msgr2=0x556009aafa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.168+0000 7fef847b0740 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009aafa30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fef7409cc00 tx=0x7fef7409c690 comp rx=0 tx=0).stop 2024-04-17T07:28:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.168+0000 7fef847b0740 1 -- 172.21.15.86:0/2049395102 shutdown_connections 2024-04-17T07:28:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.168+0000 7fef847b0740 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556009abaf10 0x556009abd3d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.168+0000 7fef847b0740 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 0x556009ab6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.168+0000 7fef847b0740 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009aafa30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.168+0000 7fef847b0740 1 --2- 172.21.15.86:0/2049395102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a93fa0 0x556009aaf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.168+0000 7fef847b0740 1 -- 172.21.15.86:0/2049395102 >> 172.21.15.86:0/2049395102 conn(0x556009a8f510 msgr2=0x556009a8e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.168+0000 7fef64c00640 0 client.27652 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:28:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.169+0000 7fef847b0740 1 -- 172.21.15.86:0/2049395102 shutdown_connections 2024-04-17T07:28:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.169+0000 7fef847b0740 1 -- 172.21.15.86:0/2049395102 wait complete. 2024-04-17T07:28:01.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.169+0000 7fef847b0740 20 client.27652 trim_cache size 0 max 16384 2024-04-17T07:28:01.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.197+0000 7fef847b0740 1 Processor -- start 2024-04-17T07:28:01.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.197+0000 7fef847b0740 1 -- start start 2024-04-17T07:28:01.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.198+0000 7fef847b0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 0x556009a95ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.198+0000 7fef847b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009ab6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.198+0000 7fef847b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a99b00 0x556009a96430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.198+0000 7fef847b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c06dee0 con 0x556009a99b00 2024-04-17T07:28:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.198+0000 7fef847b0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c06fa20 con 0x556009a93fa0 2024-04-17T07:28:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.198+0000 7fef847b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c06e5b0 con 0x556009a94990 2024-04-17T07:28:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.198+0000 7fef7a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009ab6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.198+0000 7fef7a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009ab6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58660/0 (socket says 172.21.15.86:58660) 2024-04-17T07:28:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.199+0000 7fef7aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 0x556009a95ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.199+0000 7fef7a000640 1 -- 172.21.15.86:0/1756643884 learned_addr learned my addr 172.21.15.86:0/1756643884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.199+0000 7fef7b400640 1 --2- 172.21.15.86:0/1756643884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a99b00 0x556009a96430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.199+0000 7fef7a000640 1 -- 172.21.15.86:0/1756643884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 msgr2=0x556009a95ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.199+0000 7fef7a000640 1 --2- 172.21.15.86:0/1756643884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 0x556009a95ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.199+0000 7fef7a000640 1 -- 172.21.15.86:0/1756643884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a99b00 msgr2=0x556009a96430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.199+0000 7fef7a000640 1 --2- 172.21.15.86:0/1756643884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a99b00 0x556009a96430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.199+0000 7fef7a000640 1 -- 172.21.15.86:0/1756643884 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55600970c3f0 con 0x556009a94990 2024-04-17T07:28:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.200+0000 7fef7a000640 1 --2- 172.21.15.86:0/1756643884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009ab6a30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fef7409d1f0 tx=0x7fef7409c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f9088c3f5c52cde server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.200+0000 7fef67400640 1 -- 172.21.15.86:0/1756643884 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef740b6050 con 0x556009a94990 2024-04-17T07:28:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.200+0000 7fef67400640 1 -- 172.21.15.86:0/1756643884 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef7409cc70 con 0x556009a94990 2024-04-17T07:28:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.200+0000 7fef67400640 1 -- 172.21.15.86:0/1756643884 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7409c320 con 0x556009a94990 2024-04-17T07:28:01.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.201+0000 7fef847b0740 1 -- 172.21.15.86:0/1756643884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 msgr2=0x556009ab6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.201+0000 7fef847b0740 1 --2- 172.21.15.86:0/1756643884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009ab6a30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fef7409d1f0 tx=0x7fef7409c700 comp rx=0 tx=0).stop 2024-04-17T07:28:01.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.201+0000 7fef847b0740 1 -- 172.21.15.86:0/1756643884 shutdown_connections 2024-04-17T07:28:01.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.201+0000 7fef847b0740 1 --2- 172.21.15.86:0/1756643884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a99b00 0x556009a96430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.201+0000 7fef847b0740 1 --2- 172.21.15.86:0/1756643884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a94990 0x556009ab6a30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.201+0000 7fef847b0740 1 --2- 172.21.15.86:0/1756643884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a93fa0 0x556009a95ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.201+0000 7fef847b0740 1 -- 172.21.15.86:0/1756643884 >> 172.21.15.86:0/1756643884 conn(0x556009a8f510 msgr2=0x556009aaf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.201+0000 7fef847b0740 1 -- 172.21.15.86:0/1756643884 shutdown_connections 2024-04-17T07:28:01.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.201+0000 7fef847b0740 1 -- 172.21.15.86:0/1756643884 wait complete. 2024-04-17T07:28:01.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.202+0000 7fef847b0740 1 -- start start 2024-04-17T07:28:01.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.202+0000 7fef847b0740 1 Processor -- start 2024-04-17T07:28:01.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.203+0000 7fef847b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a93fa0 0x556009ab3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.203+0000 7fef847b0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 0x556009ab3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.203+0000 7fef847b0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 0x556009b1f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.203+0000 7fef847b0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c06f350 con 0x556009a94990 2024-04-17T07:28:01.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.203+0000 7fef847b0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c06ec80 con 0x556009a99b00 2024-04-17T07:28:01.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.203+0000 7fef847b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c19b5c0 con 0x556009a93fa0 2024-04-17T07:28:01.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.203+0000 7fef7b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 0x556009b1f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.204+0000 7fef7b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 0x556009b1f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39122/0 (socket says 172.21.15.86:39122) 2024-04-17T07:28:01.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.204+0000 7fef7b400640 1 -- 172.21.15.86:0/3618378121 learned_addr learned my addr 172.21.15.86:0/3618378121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.204+0000 7fef847b0740 10 client.0 _ll_register_callbacks cb 0x7fef74098fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:01.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.203+0000 7fef7a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 0x556009ab3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.203+0000 7fef7aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a93fa0 0x556009ab3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.205+0000 7fef7aa00640 1 -- 172.21.15.86:0/3618378121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 msgr2=0x556009b1f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.205+0000 7fef7aa00640 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 0x556009b1f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.205+0000 7fef7aa00640 1 -- 172.21.15.86:0/3618378121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 msgr2=0x556009ab3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.205+0000 7fef7aa00640 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 0x556009ab3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.205+0000 7fef7aa00640 1 -- 172.21.15.86:0/3618378121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560097d67d0 con 0x556009a93fa0 2024-04-17T07:28:01.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.207+0000 7fef7aa00640 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a93fa0 0x556009ab3500 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x556009a90480 tx=0x7fef6802bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a707086fe15c373f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.207+0000 7fef64c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:28:01.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.208+0000 7fef64c00640 1 -- 172.21.15.86:0/3618378121 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68043040 con 0x556009a93fa0 2024-04-17T07:28:01.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.208+0000 7fef64c00640 1 -- 172.21.15.86:0/3618378121 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fef68004040 con 0x556009a93fa0 2024-04-17T07:28:01.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.208+0000 7fef847b0740 1 -- 172.21.15.86:0/3618378121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef7409cc70 con 0x556009a93fa0 2024-04-17T07:28:01.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.209+0000 7fef64c00640 1 -- 172.21.15.86:0/3618378121 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef68039040 con 0x556009a93fa0 2024-04-17T07:28:01.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.209+0000 7fef847b0740 10 client.27658 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:01.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.209+0000 7fef847b0740 1 -- 172.21.15.86:0/3618378121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556009b20760 con 0x556009a93fa0 2024-04-17T07:28:01.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.210+0000 7fef64c00640 1 -- 172.21.15.86:0/3618378121 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef7409cc70 con 0x556009a93fa0 2024-04-17T07:28:01.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.210+0000 7fef64c00640 1 -- 172.21.15.86:0/3618378121 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fef6802c510 con 0x556009a93fa0 2024-04-17T07:28:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.210+0000 7fef847b0740 10 client.27658 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.210+0000 7fef847b0740 10 client.27658 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:28:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.210+0000 7fef847b0740 20 client.27658 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.210+0000 7fef847b0740 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556009b20cf0 0x556009b23110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.210+0000 7fef847b0740 4 client.27658 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:28:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.210+0000 7fef847b0740 1 -- 172.21.15.86:0/3618378121 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556009a018b0 con 0x556009b20cf0 2024-04-17T07:28:01.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.211+0000 7fef7a000640 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556009b20cf0 0x556009b23110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.212+0000 7fef7a000640 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556009b20cf0 0x556009b23110 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb735759150d8e60 server_cookie=f451cfb67cf43b00 in_seq=0 out_seq=0 2024-04-17T07:28:01.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.212+0000 7fef64c00640 10 client.27658 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:28:01.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef64c00640 1 -- 172.21.15.86:0/3618378121 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556009a018b0 con 0x556009b20cf0 2024-04-17T07:28:01.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef64c00640 10 client.27658 handle_command_reply: tid=0 2024-04-17T07:28:01.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef847b0740 1 client.27658 shutdown 2024-04-17T07:28:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef847b0740 1 -- 172.21.15.86:0/3618378121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a93fa0 msgr2=0x556009ab3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef847b0740 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a93fa0 0x556009ab3500 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x556009a90480 tx=0x7fef6802bb70 comp rx=0 tx=0).stop 2024-04-17T07:28:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef847b0740 1 -- 172.21.15.86:0/3618378121 shutdown_connections 2024-04-17T07:28:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef847b0740 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556009b20cf0 0x556009b23110 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef847b0740 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556009a99b00 0x556009b1f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef847b0740 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556009a94990 0x556009ab3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef847b0740 1 --2- 172.21.15.86:0/3618378121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556009a93fa0 0x556009ab3500 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef847b0740 1 -- 172.21.15.86:0/3618378121 >> 172.21.15.86:0/3618378121 conn(0x556009a8f510 msgr2=0x556009aaf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.213+0000 7fef64c00640 0 client.27658 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:28:01.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.214+0000 7fef847b0740 1 -- 172.21.15.86:0/3618378121 shutdown_connections 2024-04-17T07:28:01.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.215+0000 7fef847b0740 1 -- 172.21.15.86:0/3618378121 wait complete. 2024-04-17T07:28:01.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.215+0000 7fef847b0740 20 client.27658 trim_cache size 0 max 16384 2024-04-17T07:28:01.215 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:28:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 -- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef600a3cb0 msgr2=0x7fef600a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef600a3cb0 0x7fef600a6170 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fef6c005ab0 tx=0x7fef6c009260 comp rx=0 tx=0).stop 2024-04-17T07:28:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 -- 172.21.15.86:0/214066907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7c1031a0 msgr2=0x7fef7c19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7c1031a0 0x7fef7c19eda0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fef7400eab0 tx=0x7fef7400ef70 comp rx=0 tx=0).stop 2024-04-17T07:28:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 -- 172.21.15.86:0/214066907 shutdown_connections 2024-04-17T07:28:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fef600a3cb0 0x7fef600a6170 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef7c103b60 0x7fef7c1a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7c1031a0 0x7fef7c19eda0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 --2- 172.21.15.86:0/214066907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef7c101fb0 0x7fef7c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 -- 172.21.15.86:0/214066907 >> 172.21.15.86:0/214066907 conn(0x7fef7c0fd540 msgr2=0x7fef7c104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.217+0000 7fef81a00640 1 -- 172.21.15.86:0/214066907 shutdown_connections 2024-04-17T07:28:01.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.218+0000 7fef81a00640 1 -- 172.21.15.86:0/214066907 wait complete. 2024-04-17T07:28:01.254 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:28:01.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.576+0000 7ffa58400640 1 -- 172.21.15.86:0/951218668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa501041c0 msgr2=0x7ffa501045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.576+0000 7ffa58400640 1 --2- 172.21.15.86:0/951218668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa501041c0 0x7ffa501045e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009900 tx=0x7ffa3c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:28:01.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.577+0000 7ffa58400640 1 -- 172.21.15.86:0/951218668 shutdown_connections 2024-04-17T07:28:01.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.577+0000 7ffa58400640 1 --2- 172.21.15.86:0/951218668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa50105d70 0x7ffa5010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.577+0000 7ffa58400640 1 --2- 172.21.15.86:0/951218668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa501053b0 0x7ffa50105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.577+0000 7ffa58400640 1 --2- 172.21.15.86:0/951218668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa501041c0 0x7ffa501045e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.577+0000 7ffa58400640 1 -- 172.21.15.86:0/951218668 >> 172.21.15.86:0/951218668 conn(0x7ffa500ff710 msgr2=0x7ffa50101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.577+0000 7ffa58400640 1 -- 172.21.15.86:0/951218668 shutdown_connections 2024-04-17T07:28:01.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.577+0000 7ffa58400640 1 -- 172.21.15.86:0/951218668 wait complete. 2024-04-17T07:28:01.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.578+0000 7ffa58400640 1 Processor -- start 2024-04-17T07:28:01.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.578+0000 7ffa58400640 1 -- start start 2024-04-17T07:28:01.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.579+0000 7ffa58400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa501041c0 0x7ffa5019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:01 smithi086 ceph-mon[31733]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.9 MiB/s wr, 613 op/s 2024-04-17T07:28:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.580+0000 7ffa58400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa501053b0 0x7ffa5019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.580+0000 7ffa58400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa50105d70 0x7ffa501a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.580+0000 7ffa58400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5010fcd0 con 0x7ffa501041c0 2024-04-17T07:28:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.580+0000 7ffa58400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5010fb50 con 0x7ffa501053b0 2024-04-17T07:28:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.580+0000 7ffa58400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5010fe50 con 0x7ffa50105d70 2024-04-17T07:28:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.580+0000 7ffa55a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa501041c0 0x7ffa5019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.580+0000 7ffa55a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa501041c0 0x7ffa5019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41542/0 (socket says 172.21.15.86:41542) 2024-04-17T07:28:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.580+0000 7ffa55a00640 1 -- 172.21.15.86:0/3914504261 learned_addr learned my addr 172.21.15.86:0/3914504261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.581+0000 7ffa55a00640 1 -- 172.21.15.86:0/3914504261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa50105d70 msgr2=0x7ffa501a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.581+0000 7ffa55a00640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa50105d70 0x7ffa501a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.581+0000 7ffa55a00640 1 -- 172.21.15.86:0/3914504261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa501053b0 msgr2=0x7ffa5019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.581+0000 7ffa55a00640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa501053b0 0x7ffa5019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.581+0000 7ffa55a00640 1 -- 172.21.15.86:0/3914504261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa501a6430 con 0x7ffa501041c0 2024-04-17T07:28:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.582+0000 7ffa55a00640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa501041c0 0x7ffa5019e810 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c000c00 tx=0x7ffa3c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.582+0000 7ffa4e000640 1 -- 172.21.15.86:0/3914504261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c043020 con 0x7ffa501041c0 2024-04-17T07:28:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.583+0000 7ffa4e000640 1 -- 172.21.15.86:0/3914504261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c004440 con 0x7ffa501041c0 2024-04-17T07:28:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.583+0000 7ffa58400640 1 -- 172.21.15.86:0/3914504261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa501a66c0 con 0x7ffa501041c0 2024-04-17T07:28:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.583+0000 7ffa4e000640 1 -- 172.21.15.86:0/3914504261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c003c90 con 0x7ffa501041c0 2024-04-17T07:28:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.583+0000 7ffa58400640 1 -- 172.21.15.86:0/3914504261 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa501a6c60 con 0x7ffa501041c0 2024-04-17T07:28:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.587+0000 7ffa5b1ee740 1 Processor -- start 2024-04-17T07:28:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.587+0000 7ffa5b1ee740 1 -- start start 2024-04-17T07:28:01.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.588+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6904fa0 0x5598b69053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.588+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b6905990 0x5598b690a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.588+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b690cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.588+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5598b686b020 con 0x5598b690ab00 2024-04-17T07:28:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.588+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5598b6811460 con 0x5598b6904fa0 2024-04-17T07:28:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.588+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5598b6866fe0 con 0x5598b6905990 2024-04-17T07:28:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.588+0000 7ffa56400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6904fa0 0x5598b69053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.588+0000 7ffa55000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b690cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.589+0000 7ffa4e000640 1 -- 172.21.15.86:0/3914504261 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ffa3c046020 con 0x7ffa501041c0 2024-04-17T07:28:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.589+0000 7ffa55000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b690cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41546/0 (socket says 172.21.15.86:41546) 2024-04-17T07:28:01.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.589+0000 7ffa55000640 1 -- 172.21.15.86:0/2542017999 learned_addr learned my addr 172.21.15.86:0/2542017999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.590+0000 7ffa55a00640 1 --2- 172.21.15.86:0/2542017999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b6905990 0x5598b690a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.590+0000 7ffa55000640 1 -- 172.21.15.86:0/2542017999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b6905990 msgr2=0x5598b690a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.590+0000 7ffa55000640 1 --2- 172.21.15.86:0/2542017999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b6905990 0x5598b690a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.590+0000 7ffa55000640 1 -- 172.21.15.86:0/2542017999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6904fa0 msgr2=0x5598b69053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.590+0000 7ffa55000640 1 --2- 172.21.15.86:0/2542017999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6904fa0 0x5598b69053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.590+0000 7ffa55000640 1 -- 172.21.15.86:0/2542017999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b6819040 con 0x5598b690ab00 2024-04-17T07:28:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.591+0000 7ffa55000640 1 --2- 172.21.15.86:0/2542017999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b690cf90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009be0 tx=0x7ffa4400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546bedf521f88503 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.591+0000 7ffa4e000640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffa380a3c30 0x7ffa380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.591+0000 7ffa4e000640 1 -- 172.21.15.86:0/3914504261 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ffa3c0f29f0 con 0x7ffa501041c0 2024-04-17T07:28:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.592+0000 7ffa42a00640 1 -- 172.21.15.86:0/2542017999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44005020 con 0x5598b690ab00 2024-04-17T07:28:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.592+0000 7ffa42a00640 1 -- 172.21.15.86:0/2542017999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa440047a0 con 0x5598b690ab00 2024-04-17T07:28:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.592+0000 7ffa42a00640 1 -- 172.21.15.86:0/2542017999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44010690 con 0x5598b690ab00 2024-04-17T07:28:01.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.592+0000 7ffa56400640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffa380a3c30 0x7ffa380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.593+0000 7ffa56400640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffa380a3c30 0x7ffa380a60f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009910 tx=0x7ffa480098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.593+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2542017999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 msgr2=0x5598b690cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.593+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/2542017999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b690cf90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009be0 tx=0x7ffa4400e720 comp rx=0 tx=0).stop 2024-04-17T07:28:01.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.594+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2542017999 shutdown_connections 2024-04-17T07:28:01.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.594+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/2542017999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b690cf90 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.594+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/2542017999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b6905990 0x5598b690a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.594+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/2542017999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6904fa0 0x5598b69053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.594+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2542017999 >> 172.21.15.86:0/2542017999 conn(0x5598b6900510 msgr2=0x5598b6900910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.594+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2542017999 shutdown_connections 2024-04-17T07:28:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.594+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2542017999 wait complete. 2024-04-17T07:28:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.595+0000 7ffa5b1ee740 1 -- start start 2024-04-17T07:28:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.595+0000 7ffa5b1ee740 1 Processor -- start 2024-04-17T07:28:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.596+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b6904fa0 0x5598b690f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.596+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b690fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.596+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b6916a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.596+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5598b690dab0 con 0x5598b690ab00 2024-04-17T07:28:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.596+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5598b6824830 con 0x5598b6905990 2024-04-17T07:28:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.596+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5598b690dc30 con 0x5598b6904fa0 2024-04-17T07:28:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.596+0000 7ffa5b1ee740 10 client.0 _ll_register_callbacks cb 0x5598b67f2fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.597+0000 7ffa56400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b6916a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.597+0000 7ffa55000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b690fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.597+0000 7ffa56400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b6916a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41548/0 (socket says 172.21.15.86:41548) 2024-04-17T07:28:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.597+0000 7ffa56400640 1 -- 172.21.15.86:0/3969215743 learned_addr learned my addr 172.21.15.86:0/3969215743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.597+0000 7ffa55a00640 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b6904fa0 0x5598b690f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.597+0000 7ffa55000640 1 -- 172.21.15.86:0/3969215743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b6904fa0 msgr2=0x5598b690f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.597+0000 7ffa55000640 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b6904fa0 0x5598b690f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.597+0000 7ffa55000640 1 -- 172.21.15.86:0/3969215743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 msgr2=0x5598b6916a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.597+0000 7ffa55000640 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b6916a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.597+0000 7ffa55000640 1 -- 172.21.15.86:0/3969215743 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b6828920 con 0x5598b6905990 2024-04-17T07:28:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.598+0000 7ffa55000640 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b690fa20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffa44000c00 tx=0x7ffa44004e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=873c605c7cc58680 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.598+0000 7ffa37400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.598+0000 7ffa37400640 1 -- 172.21.15.86:0/3969215743 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4401d040 con 0x5598b6905990 2024-04-17T07:28:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.598+0000 7ffa37400640 1 -- 172.21.15.86:0/3969215743 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ffa44004050 con 0x5598b6905990 2024-04-17T07:28:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.598+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/3969215743 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa440047a0 con 0x5598b6905990 2024-04-17T07:28:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.599+0000 7ffa37400640 1 -- 172.21.15.86:0/3969215743 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa44030050 con 0x5598b6905990 2024-04-17T07:28:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.599+0000 7ffa5b1ee740 10 client.37622 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.599+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/3969215743 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598b67f67f0 con 0x5598b6905990 2024-04-17T07:28:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.599+0000 7ffa37400640 1 -- 172.21.15.86:0/3969215743 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa440047a0 con 0x5598b6905990 2024-04-17T07:28:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.600+0000 7ffa37400640 1 -- 172.21.15.86:0/3969215743 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ffa440037b0 con 0x5598b6905990 2024-04-17T07:28:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.600+0000 7ffa5b1ee740 10 client.37622 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.600+0000 7ffa5b1ee740 10 client.37622 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:28:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.600+0000 7ffa5b1ee740 20 client.37622 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.600+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5598b691abc0 0x5598b691cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.600+0000 7ffa5b1ee740 4 client.37622 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.600+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/3969215743 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598b6878a40 con 0x5598b691abc0 2024-04-17T07:28:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.601+0000 7ffa55a00640 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5598b691abc0 0x5598b691cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.602+0000 7ffa37400640 10 client.37622 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:28:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.602+0000 7ffa55a00640 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5598b691abc0 0x5598b691cfb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7340f8a79041ce server_cookie=a4470f9759877c58 in_seq=0 out_seq=0 2024-04-17T07:28:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.603+0000 7ffa37400640 1 -- 172.21.15.86:0/3969215743 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598b6878a40 con 0x5598b691abc0 2024-04-17T07:28:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.603+0000 7ffa37400640 10 client.37622 handle_command_reply: tid=0 2024-04-17T07:28:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.603+0000 7ffa5b1ee740 1 client.37622 shutdown 2024-04-17T07:28:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.604+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/3969215743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 msgr2=0x5598b690fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.604+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b690fa20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffa44000c00 tx=0x7ffa44004e50 comp rx=0 tx=0).stop 2024-04-17T07:28:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.604+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/3969215743 shutdown_connections 2024-04-17T07:28:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.604+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b690ab00 0x5598b6916a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.604+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5598b691abc0 0x5598b691cfb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.604+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b690fa20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.604+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/3969215743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b6904fa0 0x5598b690f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.604+0000 7ffa37400640 0 client.37622 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:28:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.604+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/3969215743 >> 172.21.15.86:0/3969215743 conn(0x5598b6900510 msgr2=0x5598b68ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.605+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/3969215743 shutdown_connections 2024-04-17T07:28:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.605+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/3969215743 wait complete. 2024-04-17T07:28:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.605+0000 7ffa5b1ee740 20 client.37622 trim_cache size 0 max 16384 2024-04-17T07:28:01.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.636+0000 7ffa5b1ee740 1 Processor -- start 2024-04-17T07:28:01.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.636+0000 7ffa5b1ee740 1 -- start start 2024-04-17T07:28:01.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.636+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 0x5598b690eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.636+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b6916a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.636+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b690ab00 0x5598b68fee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.636+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b11e0 con 0x5598b6904fa0 2024-04-17T07:28:01.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.637+0000 7ffa55000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b6916a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.637+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b2d20 con 0x5598b6905990 2024-04-17T07:28:01.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.637+0000 7ffa56400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b690ab00 0x5598b68fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.637+0000 7ffa56400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b690ab00 0x5598b68fee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58710/0 (socket says 172.21.15.86:58710) 2024-04-17T07:28:01.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.637+0000 7ffa56400640 1 -- 172.21.15.86:0/2757045467 learned_addr learned my addr 172.21.15.86:0/2757045467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.637+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2757045467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b18b0 con 0x5598b690ab00 2024-04-17T07:28:01.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.637+0000 7ffa55a00640 1 --2- 172.21.15.86:0/2757045467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 0x5598b690eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.639+0000 7ffa55000640 1 -- 172.21.15.86:0/2757045467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b690ab00 msgr2=0x5598b68fee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.639+0000 7ffa55000640 1 --2- 172.21.15.86:0/2757045467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b690ab00 0x5598b68fee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.639+0000 7ffa55000640 1 -- 172.21.15.86:0/2757045467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 msgr2=0x5598b690eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.639+0000 7ffa55000640 1 --2- 172.21.15.86:0/2757045467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 0x5598b690eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.639+0000 7ffa55000640 1 -- 172.21.15.86:0/2757045467 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b657d3f0 con 0x5598b6905990 2024-04-17T07:28:01.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.641+0000 7ffa55000640 1 --2- 172.21.15.86:0/2757045467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b6916a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ffa44003d70 tx=0x7ffa44009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71197e0c3ce0915f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.642+0000 7ffa42000640 1 -- 172.21.15.86:0/2757045467 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44038070 con 0x5598b6905990 2024-04-17T07:28:01.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.642+0000 7ffa42000640 1 -- 172.21.15.86:0/2757045467 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa44027070 con 0x5598b6905990 2024-04-17T07:28:01.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.642+0000 7ffa42000640 1 -- 172.21.15.86:0/2757045467 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44018ac0 con 0x5598b6905990 2024-04-17T07:28:01.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.645+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2757045467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 msgr2=0x5598b6916a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.645+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/2757045467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b6916a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ffa44003d70 tx=0x7ffa44009b90 comp rx=0 tx=0).stop 2024-04-17T07:28:01.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.646+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2757045467 shutdown_connections 2024-04-17T07:28:01.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.646+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/2757045467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b690ab00 0x5598b68fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.646+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/2757045467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b6916a20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.646+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/2757045467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 0x5598b690eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.646+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2757045467 >> 172.21.15.86:0/2757045467 conn(0x5598b6900510 msgr2=0x5598b6906e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.646+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2757045467 shutdown_connections 2024-04-17T07:28:01.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.646+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/2757045467 wait complete. 2024-04-17T07:28:01.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.647+0000 7ffa5b1ee740 1 -- start start 2024-04-17T07:28:01.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.647+0000 7ffa5b1ee740 1 Processor -- start 2024-04-17T07:28:01.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 0x5598b6913dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b6914310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa5b1ee740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b690ab00 0x5598b69152d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b2650 con 0x5598b6904fa0 2024-04-17T07:28:01.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b1f80 con 0x5598b6905990 2024-04-17T07:28:01.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa5b1ee740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501af1b0 con 0x5598b690ab00 2024-04-17T07:28:01.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa5b1ee740 10 client.0 _ll_register_callbacks cb 0x7ffa44004cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:01.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa55000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b6914310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa55a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 0x5598b6913dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa55a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 0x5598b6913dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41568/0 (socket says 172.21.15.86:41568) 2024-04-17T07:28:01.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.648+0000 7ffa55a00640 1 -- 172.21.15.86:0/1761830673 learned_addr learned my addr 172.21.15.86:0/1761830673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.649+0000 7ffa55000640 1 -- 172.21.15.86:0/1761830673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b690ab00 msgr2=0x5598b69152d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.649+0000 7ffa55000640 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b690ab00 0x5598b69152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.649+0000 7ffa55000640 1 -- 172.21.15.86:0/1761830673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 msgr2=0x5598b6913dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:01.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.649+0000 7ffa55000640 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 0x5598b6913dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.649+0000 7ffa55000640 1 -- 172.21.15.86:0/1761830673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b66477d0 con 0x5598b6905990 2024-04-17T07:28:01.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.649+0000 7ffa55a00640 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 0x5598b6913dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:01.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.650+0000 7ffa55000640 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b6914310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400f010 tx=0x7ffa44009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260e3cd44940459 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.650+0000 7ffa37400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:01.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.650+0000 7ffa37400640 1 -- 172.21.15.86:0/1761830673 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44038040 con 0x5598b6905990 2024-04-17T07:28:01.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.650+0000 7ffa37400640 1 -- 172.21.15.86:0/1761830673 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ffa440037f0 con 0x5598b6905990 2024-04-17T07:28:01.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.651+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/1761830673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa44027070 con 0x5598b6905990 2024-04-17T07:28:01.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.651+0000 7ffa37400640 1 -- 172.21.15.86:0/1761830673 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa44018800 con 0x5598b6905990 2024-04-17T07:28:01.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.651+0000 7ffa5b1ee740 10 client.37628 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:01.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.651+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/1761830673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598b69169f0 con 0x5598b6905990 2024-04-17T07:28:01.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.652+0000 7ffa37400640 1 -- 172.21.15.86:0/1761830673 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa44027070 con 0x5598b6905990 2024-04-17T07:28:01.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.652+0000 7ffa37400640 1 -- 172.21.15.86:0/1761830673 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ffa44004230 con 0x5598b6905990 2024-04-17T07:28:01.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.652+0000 7ffa5b1ee740 10 client.37628 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:01.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.652+0000 7ffa5b1ee740 10 client.37628 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:28:01.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.652+0000 7ffa5b1ee740 20 client.37628 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:01.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.652+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5598b6981460 0x5598b6983880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.652+0000 7ffa5b1ee740 4 client.37628 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:28:01.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.652+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/1761830673 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598b690f2d0 con 0x5598b6981460 2024-04-17T07:28:01.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.653+0000 7ffa55a00640 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5598b6981460 0x5598b6983880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.654+0000 7ffa55a00640 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5598b6981460 0x5598b6983880 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=745ec8d5242ccc2e server_cookie=42a665846a0d2f1e in_seq=0 out_seq=0 2024-04-17T07:28:01.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.654+0000 7ffa37400640 10 client.37628 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:28:01.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.655+0000 7ffa37400640 1 -- 172.21.15.86:0/1761830673 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598b690f2d0 con 0x5598b6981460 2024-04-17T07:28:01.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.655+0000 7ffa37400640 10 client.37628 handle_command_reply: tid=0 2024-04-17T07:28:01.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.655+0000 7ffa5b1ee740 1 client.37628 shutdown 2024-04-17T07:28:01.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.655+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/1761830673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 msgr2=0x5598b6914310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.655+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b6914310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400f010 tx=0x7ffa44009f00 comp rx=0 tx=0).stop 2024-04-17T07:28:01.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.655+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/1761830673 shutdown_connections 2024-04-17T07:28:01.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.655+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5598b690ab00 0x5598b69152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.655+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5598b6981460 0x5598b6983880 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.656+0000 7ffa37400640 0 client.37628 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:28:01.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.656+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5598b6905990 0x5598b6914310 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.656+0000 7ffa5b1ee740 1 --2- 172.21.15.86:0/1761830673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5598b6904fa0 0x5598b6913dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.656+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/1761830673 >> 172.21.15.86:0/1761830673 conn(0x5598b6900510 msgr2=0x5598b6904000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.656+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/1761830673 shutdown_connections 2024-04-17T07:28:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.656+0000 7ffa5b1ee740 1 -- 172.21.15.86:0/1761830673 wait complete. 2024-04-17T07:28:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.656+0000 7ffa5b1ee740 20 client.37628 trim_cache size 0 max 16384 2024-04-17T07:28:01.657 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:28:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.657+0000 7ffa58400640 1 -- 172.21.15.86:0/3914504261 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffa380a3c30 msgr2=0x7ffa380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.657+0000 7ffa58400640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffa380a3c30 0x7ffa380a60f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009910 tx=0x7ffa480098a0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.657+0000 7ffa58400640 1 -- 172.21.15.86:0/3914504261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa501041c0 msgr2=0x7ffa5019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.657+0000 7ffa58400640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa501041c0 0x7ffa5019e810 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c000c00 tx=0x7ffa3c004040 comp rx=0 tx=0).stop 2024-04-17T07:28:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.658+0000 7ffa58400640 1 -- 172.21.15.86:0/3914504261 shutdown_connections 2024-04-17T07:28:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.658+0000 7ffa58400640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffa380a3c30 0x7ffa380a60f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.658+0000 7ffa58400640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa50105d70 0x7ffa501a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.658+0000 7ffa58400640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa501053b0 0x7ffa5019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.658+0000 7ffa58400640 1 --2- 172.21.15.86:0/3914504261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa501041c0 0x7ffa5019e810 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.658+0000 7ffa58400640 1 -- 172.21.15.86:0/3914504261 >> 172.21.15.86:0/3914504261 conn(0x7ffa500ff710 msgr2=0x7ffa500783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.658+0000 7ffa58400640 1 -- 172.21.15.86:0/3914504261 shutdown_connections 2024-04-17T07:28:01.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.658+0000 7ffa58400640 1 -- 172.21.15.86:0/3914504261 wait complete. 2024-04-17T07:28:01.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:28:01.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:01 smithi098 ceph-mon[35418]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.9 MiB/s wr, 613 op/s 2024-04-17T07:28:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:01 smithi103 ceph-mon[35495]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.9 MiB/s wr, 613 op/s 2024-04-17T07:28:01.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.985+0000 7fea5f400640 1 -- 172.21.15.86:0/1140625965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea58109720 msgr2=0x7fea58109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.985+0000 7fea5f400640 1 --2- 172.21.15.86:0/1140625965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea58109720 0x7fea58109ba0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fea44009860 tx=0x7fea4402bb70 comp rx=0 tx=0).stop 2024-04-17T07:28:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.987+0000 7fea5f400640 1 -- 172.21.15.86:0/1140625965 shutdown_connections 2024-04-17T07:28:01.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.987+0000 7fea5f400640 1 --2- 172.21.15.86:0/1140625965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea5810a0e0 0x7fea58110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.987+0000 7fea5f400640 1 --2- 172.21.15.86:0/1140625965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea58109720 0x7fea58109ba0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.988+0000 7fea5f400640 1 --2- 172.21.15.86:0/1140625965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea58108530 0x7fea58108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.988+0000 7fea5f400640 1 -- 172.21.15.86:0/1140625965 >> 172.21.15.86:0/1140625965 conn(0x7fea580774d0 msgr2=0x7fea58079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.988+0000 7fea5f400640 1 -- 172.21.15.86:0/1140625965 shutdown_connections 2024-04-17T07:28:01.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.988+0000 7fea5f400640 1 -- 172.21.15.86:0/1140625965 wait complete. 2024-04-17T07:28:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.989+0000 7fea5f400640 1 Processor -- start 2024-04-17T07:28:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.990+0000 7fea5f400640 1 -- start start 2024-04-17T07:28:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.990+0000 7fea5f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea58108530 0x7fea5807f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.990+0000 7fea5f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea58109720 0x7fea5807fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.990+0000 7fea5f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea5810a0e0 0x7fea581ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.991+0000 7fea57e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea58109720 0x7fea5807fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.991+0000 7fea5ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea58108530 0x7fea5807f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.991+0000 7fea57e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea58109720 0x7fea5807fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41592/0 (socket says 172.21.15.86:41592) 2024-04-17T07:28:01.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.991+0000 7fea57e00640 1 -- 172.21.15.86:0/449621200 learned_addr learned my addr 172.21.15.86:0/449621200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.991+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea58114000 con 0x7fea58109720 2024-04-17T07:28:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.991+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea58113e80 con 0x7fea58108530 2024-04-17T07:28:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.991+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea58114180 con 0x7fea5810a0e0 2024-04-17T07:28:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.991+0000 7fea5d400640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea5810a0e0 0x7fea581ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.992+0000 7fea5d400640 1 -- 172.21.15.86:0/449621200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea58108530 msgr2=0x7fea5807f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.992+0000 7fea5d400640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea58108530 0x7fea5807f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.992+0000 7fea5d400640 1 -- 172.21.15.86:0/449621200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea58109720 msgr2=0x7fea5807fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.992+0000 7fea5d400640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea58109720 0x7fea5807fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.992+0000 7fea5d400640 1 -- 172.21.15.86:0/449621200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea581aec10 con 0x7fea5810a0e0 2024-04-17T07:28:01.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.992+0000 7fea5d400640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea5810a0e0 0x7fea581ae6d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fea4800e970 tx=0x7fea4800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.993+0000 7fea55600640 1 -- 172.21.15.86:0/449621200 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4800ccc0 con 0x7fea5810a0e0 2024-04-17T07:28:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.993+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea581aee40 con 0x7fea5810a0e0 2024-04-17T07:28:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.993+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581af2b0 con 0x7fea5810a0e0 2024-04-17T07:28:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.994+0000 7fea55600640 1 -- 172.21.15.86:0/449621200 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea48005020 con 0x7fea5810a0e0 2024-04-17T07:28:01.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.994+0000 7fea55600640 1 -- 172.21.15.86:0/449621200 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4800ccc0 con 0x7fea5810a0e0 2024-04-17T07:28:01.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:01.998+0000 7fea621a5740 1 Processor -- start 2024-04-17T07:28:02.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.001+0000 7fea55600640 1 -- 172.21.15.86:0/449621200 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fea48004020 con 0x7fea5810a0e0 2024-04-17T07:28:02.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.001+0000 7fea621a5740 1 -- start start 2024-04-17T07:28:02.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.002+0000 7fea621a5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc64580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.002+0000 7fea621a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc69650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.003+0000 7fea621a5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc6c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.003+0000 7fea621a5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56079fbca020 con 0x56079fc69cc0 2024-04-17T07:28:02.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.003+0000 7fea621a5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56079fb70460 con 0x56079fc64160 2024-04-17T07:28:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.003+0000 7fea621a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56079fbc5fe0 con 0x56079fc64b50 2024-04-17T07:28:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.003+0000 7fea57e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc64580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.003+0000 7fea57e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc64580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39190/0 (socket says 172.21.15.86:39190) 2024-04-17T07:28:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.003+0000 7fea57e00640 1 -- 172.21.15.86:0/966419791 learned_addr learned my addr 172.21.15.86:0/966419791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.003+0000 7fea5ca00640 1 --2- 172.21.15.86:0/966419791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc69650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.004+0000 7fea5ca00640 1 -- 172.21.15.86:0/966419791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 msgr2=0x56079fc64580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.004+0000 7fea5ca00640 1 --2- 172.21.15.86:0/966419791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc64580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.004+0000 7fea5ca00640 1 -- 172.21.15.86:0/966419791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 msgr2=0x56079fc6c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.004+0000 7fea5ca00640 1 --2- 172.21.15.86:0/966419791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc6c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.004+0000 7fea5ca00640 1 -- 172.21.15.86:0/966419791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56079fb78040 con 0x56079fc64b50 2024-04-17T07:28:02.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.005+0000 7fea5ca00640 1 --2- 172.21.15.86:0/966419791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc69650 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fea50005fd0 tx=0x7fea500095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c7f67558eb656c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:02.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.005+0000 7fea4dc00640 1 -- 172.21.15.86:0/966419791 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50003b70 con 0x56079fc64b50 2024-04-17T07:28:02.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.005+0000 7fea4dc00640 1 -- 172.21.15.86:0/966419791 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea50003d10 con 0x56079fc64b50 2024-04-17T07:28:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.007+0000 7fea621a5740 1 -- 172.21.15.86:0/966419791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 msgr2=0x56079fc69650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.007+0000 7fea621a5740 1 --2- 172.21.15.86:0/966419791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc69650 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fea50005fd0 tx=0x7fea500095c0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.007+0000 7fea55600640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea3c0a3e30 0x7fea3c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.007+0000 7fea55600640 1 -- 172.21.15.86:0/449621200 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fea480d0200 con 0x7fea5810a0e0 2024-04-17T07:28:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.008+0000 7fea621a5740 1 -- 172.21.15.86:0/966419791 shutdown_connections 2024-04-17T07:28:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.008+0000 7fea621a5740 1 --2- 172.21.15.86:0/966419791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc6c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.008+0000 7fea621a5740 1 --2- 172.21.15.86:0/966419791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc69650 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.008+0000 7fea621a5740 1 --2- 172.21.15.86:0/966419791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc64580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.008+0000 7fea621a5740 1 -- 172.21.15.86:0/966419791 >> 172.21.15.86:0/966419791 conn(0x56079fc5f6d0 msgr2=0x56079fc5fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.008+0000 7fea621a5740 1 -- 172.21.15.86:0/966419791 shutdown_connections 2024-04-17T07:28:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.008+0000 7fea57e00640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea3c0a3e30 0x7fea3c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.008+0000 7fea621a5740 1 -- 172.21.15.86:0/966419791 wait complete. 2024-04-17T07:28:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.009+0000 7fea621a5740 1 -- start start 2024-04-17T07:28:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.009+0000 7fea621a5740 1 Processor -- start 2024-04-17T07:28:02.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.009+0000 7fea57e00640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea3c0a3e30 0x7fea3c0a62f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fea44009830 tx=0x7fea4402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.010+0000 7fea621a5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc6e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.010+0000 7fea621a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc6ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.010+0000 7fea621a5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc75c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.010+0000 7fea621a5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56079fc6cd10 con 0x56079fc69cc0 2024-04-17T07:28:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.010+0000 7fea621a5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56079fb83830 con 0x56079fc64160 2024-04-17T07:28:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.010+0000 7fea621a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56079fc6ce90 con 0x56079fc64b50 2024-04-17T07:28:02.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.010+0000 7fea621a5740 10 client.0 _ll_register_callbacks cb 0x56079fb51fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.011+0000 7fea5ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc6e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.011+0000 7fea5ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc6e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39198/0 (socket says 172.21.15.86:39198) 2024-04-17T07:28:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.011+0000 7fea5ca00640 1 -- 172.21.15.86:0/1659520907 learned_addr learned my addr 172.21.15.86:0/1659520907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.011+0000 7fea57e00640 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc6ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.012+0000 7fea5d400640 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc75c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.012+0000 7fea5ca00640 1 -- 172.21.15.86:0/1659520907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 msgr2=0x56079fc6ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.012+0000 7fea5ca00640 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc6ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.013+0000 7fea5ca00640 1 -- 172.21.15.86:0/1659520907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 msgr2=0x56079fc75c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.013+0000 7fea5ca00640 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc75c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.013+0000 7fea5ca00640 1 -- 172.21.15.86:0/1659520907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56079fb87920 con 0x56079fc64160 2024-04-17T07:28:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.013+0000 7fea57e00640 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc6ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.013+0000 7fea5d400640 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc75c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.014+0000 7fea5ca00640 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc6e6b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea50000f80 tx=0x7fea50003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655b5ac38bdbf739 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:02.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.014+0000 7fea42000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.015+0000 7fea42000640 1 -- 172.21.15.86:0/1659520907 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5000db80 con 0x56079fc64160 2024-04-17T07:28:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.015+0000 7fea42000640 1 -- 172.21.15.86:0/1659520907 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fea50023430 con 0x56079fc64160 2024-04-17T07:28:02.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.015+0000 7fea621a5740 1 -- 172.21.15.86:0/1659520907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea50003d10 con 0x56079fc64160 2024-04-17T07:28:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.016+0000 7fea42000640 1 -- 172.21.15.86:0/1659520907 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea50032050 con 0x56079fc64160 2024-04-17T07:28:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.016+0000 7fea621a5740 10 client.37640 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:02.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.016+0000 7fea621a5740 1 -- 172.21.15.86:0/1659520907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56079fb557f0 con 0x56079fc64160 2024-04-17T07:28:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.017+0000 7fea42000640 1 -- 172.21.15.86:0/1659520907 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea50003d10 con 0x56079fc64160 2024-04-17T07:28:02.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.018+0000 7fea42000640 1 -- 172.21.15.86:0/1659520907 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fea5000b040 con 0x56079fc64160 2024-04-17T07:28:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.018+0000 7fea621a5740 10 client.37640 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.018+0000 7fea621a5740 10 client.37640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.018+0000 7fea621a5740 20 client.37640 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.018+0000 7fea621a5740 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56079fc79e70 0x56079fc7c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.018+0000 7fea621a5740 4 client.37640 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:02.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.018+0000 7fea621a5740 1 -- 172.21.15.86:0/1659520907 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56079fbd7a40 con 0x56079fc79e70 2024-04-17T07:28:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.020+0000 7fea57e00640 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56079fc79e70 0x56079fc7c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.020+0000 7fea57e00640 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56079fc79e70 0x56079fc7c260 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b3f50b2869bba1 server_cookie=633e83312d35ad35 in_seq=0 out_seq=0 2024-04-17T07:28:02.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.020+0000 7fea42000640 10 client.37640 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.023+0000 7fea42000640 1 -- 172.21.15.86:0/1659520907 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56079fbd7a40 con 0x56079fc79e70 2024-04-17T07:28:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.023+0000 7fea42000640 10 client.37640 handle_command_reply: tid=0 2024-04-17T07:28:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.023+0000 7fea621a5740 1 client.37640 shutdown 2024-04-17T07:28:02.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.023+0000 7fea621a5740 1 -- 172.21.15.86:0/1659520907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 msgr2=0x56079fc6e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:02.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.023+0000 7fea621a5740 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc6e6b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea50000f80 tx=0x7fea50003b80 comp rx=0 tx=0).stop 2024-04-17T07:28:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.024+0000 7fea621a5740 1 -- 172.21.15.86:0/1659520907 shutdown_connections 2024-04-17T07:28:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.024+0000 7fea621a5740 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56079fc79e70 0x56079fc7c260 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.025+0000 7fea621a5740 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc75c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.025+0000 7fea621a5740 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc6ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.025+0000 7fea621a5740 1 --2- 172.21.15.86:0/1659520907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc6e6b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.025+0000 7fea621a5740 1 -- 172.21.15.86:0/1659520907 >> 172.21.15.86:0/1659520907 conn(0x56079fc5f6d0 msgr2=0x56079fc5e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:02.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.025+0000 7fea42000640 0 client.37640 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.025+0000 7fea621a5740 1 -- 172.21.15.86:0/1659520907 shutdown_connections 2024-04-17T07:28:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.026+0000 7fea621a5740 1 -- 172.21.15.86:0/1659520907 wait complete. 2024-04-17T07:28:02.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.026+0000 7fea621a5740 20 client.37640 trim_cache size 0 max 16384 2024-04-17T07:28:02.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.058+0000 7fea621a5740 1 Processor -- start 2024-04-17T07:28:02.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.058+0000 7fea621a5740 1 -- start start 2024-04-17T07:28:02.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.059+0000 7fea621a5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc5e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.059+0000 7fea621a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc75bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.059+0000 7fea621a5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc5e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.059+0000 7fea621a5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea3800fc70 con 0x56079fc69cc0 2024-04-17T07:28:02.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.059+0000 7fea621a5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea380117b0 con 0x56079fc64160 2024-04-17T07:28:02.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.059+0000 7fea621a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea38010340 con 0x56079fc64b50 2024-04-17T07:28:02.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.060+0000 7fea5ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc5e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.060+0000 7fea5ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc5e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39210/0 (socket says 172.21.15.86:39210) 2024-04-17T07:28:02.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.060+0000 7fea5ca00640 1 -- 172.21.15.86:0/2768211523 learned_addr learned my addr 172.21.15.86:0/2768211523 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:02.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.060+0000 7fea57e00640 1 --2- 172.21.15.86:0/2768211523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc75bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.061+0000 7fea5ca00640 1 -- 172.21.15.86:0/2768211523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 msgr2=0x56079fc75bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.061+0000 7fea5ca00640 1 --2- 172.21.15.86:0/2768211523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc75bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.061+0000 7fea5ca00640 1 -- 172.21.15.86:0/2768211523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 msgr2=0x56079fc5e5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:28:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.063+0000 7fea5d400640 1 --2- 172.21.15.86:0/2768211523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc5e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.063+0000 7fea5ca00640 1 --2- 172.21.15.86:0/2768211523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc5e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.063+0000 7fea5ca00640 1 -- 172.21.15.86:0/2768211523 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56079f8dc3f0 con 0x56079fc64160 2024-04-17T07:28:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.063+0000 7fea5d400640 1 --2- 172.21.15.86:0/2768211523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc5e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.064+0000 7fea5ca00640 1 --2- 172.21.15.86:0/2768211523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc5e0a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea50006010 tx=0x7fea50009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c1c4c773b70830 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:02.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.064+0000 7fea4d200640 1 -- 172.21.15.86:0/2768211523 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea500042b0 con 0x56079fc64160 2024-04-17T07:28:02.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.065+0000 7fea4d200640 1 -- 172.21.15.86:0/2768211523 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea50027030 con 0x56079fc64160 2024-04-17T07:28:02.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.065+0000 7fea4d200640 1 -- 172.21.15.86:0/2768211523 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5000dbe0 con 0x56079fc64160 2024-04-17T07:28:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.066+0000 7fea621a5740 1 -- 172.21.15.86:0/2768211523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 msgr2=0x56079fc5e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.066+0000 7fea621a5740 1 --2- 172.21.15.86:0/2768211523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc5e0a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea50006010 tx=0x7fea50009d30 comp rx=0 tx=0).stop 2024-04-17T07:28:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.066+0000 7fea621a5740 1 -- 172.21.15.86:0/2768211523 shutdown_connections 2024-04-17T07:28:02.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.066+0000 7fea621a5740 1 --2- 172.21.15.86:0/2768211523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc69cc0 0x56079fc5e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.066+0000 7fea621a5740 1 --2- 172.21.15.86:0/2768211523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc75bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.066+0000 7fea621a5740 1 --2- 172.21.15.86:0/2768211523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc64160 0x56079fc5e0a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.068+0000 7fea621a5740 1 -- 172.21.15.86:0/2768211523 >> 172.21.15.86:0/2768211523 conn(0x56079fc5f6d0 msgr2=0x56079fc65f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:02.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.069+0000 7fea621a5740 1 -- 172.21.15.86:0/2768211523 shutdown_connections 2024-04-17T07:28:02.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.069+0000 7fea621a5740 1 -- 172.21.15.86:0/2768211523 wait complete. 2024-04-17T07:28:02.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.070+0000 7fea621a5740 1 -- start start 2024-04-17T07:28:02.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.071+0000 7fea621a5740 1 Processor -- start 2024-04-17T07:28:02.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.071+0000 7fea621a5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc64160 0x56079fc726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.071+0000 7fea621a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc72c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.071+0000 7fea621a5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc69cc0 0x56079fcde380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.071+0000 7fea621a5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea380110e0 con 0x56079fc64160 2024-04-17T07:28:02.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.071+0000 7fea621a5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea38010a10 con 0x56079fc69cc0 2024-04-17T07:28:02.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.071+0000 7fea621a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea3800dc40 con 0x56079fc64b50 2024-04-17T07:28:02.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.071+0000 7fea621a5740 10 client.0 _ll_register_callbacks cb 0x7fea50005a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:02.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.072+0000 7fea5ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc64160 0x56079fc726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.072+0000 7fea5ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc64160 0x56079fc726f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41632/0 (socket says 172.21.15.86:41632) 2024-04-17T07:28:02.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.072+0000 7fea5ca00640 1 -- 172.21.15.86:0/444071057 learned_addr learned my addr 172.21.15.86:0/444071057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.072+0000 7fea57e00640 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc72c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.073+0000 7fea5ca00640 1 -- 172.21.15.86:0/444071057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 msgr2=0x56079fc72c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.073+0000 7fea5ca00640 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc72c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.073+0000 7fea5ca00640 1 -- 172.21.15.86:0/444071057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc69cc0 msgr2=0x56079fcde380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:02.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.073+0000 7fea5ca00640 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc69cc0 0x56079fcde380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.073+0000 7fea5ca00640 1 -- 172.21.15.86:0/444071057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56079f9a67d0 con 0x56079fc64160 2024-04-17T07:28:02.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.074+0000 7fea5ca00640 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc64160 0x56079fc726f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fea50017510 tx=0x7fea500042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f515087b9b6f73ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:02.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.074+0000 7fea42000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:28:02.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.074+0000 7fea42000640 1 -- 172.21.15.86:0/444071057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50043060 con 0x56079fc64160 2024-04-17T07:28:02.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.074+0000 7fea42000640 1 -- 172.21.15.86:0/444071057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fea50003b80 con 0x56079fc64160 2024-04-17T07:28:02.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.074+0000 7fea621a5740 1 -- 172.21.15.86:0/444071057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea50027030 con 0x56079fc64160 2024-04-17T07:28:02.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.075+0000 7fea42000640 1 -- 172.21.15.86:0/444071057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea5004a050 con 0x56079fc64160 2024-04-17T07:28:02.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.075+0000 7fea621a5740 10 client.18741 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:02.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.075+0000 7fea621a5740 1 -- 172.21.15.86:0/444071057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56079fcdf570 con 0x56079fc64160 2024-04-17T07:28:02.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.075+0000 7fea42000640 1 -- 172.21.15.86:0/444071057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea50027030 con 0x56079fc64160 2024-04-17T07:28:02.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.075+0000 7fea42000640 1 -- 172.21.15.86:0/444071057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fea5000b040 con 0x56079fc64160 2024-04-17T07:28:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.076+0000 7fea621a5740 10 client.18741 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.076+0000 7fea621a5740 10 client.18741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.076+0000 7fea621a5740 20 client.18741 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.076+0000 7fea621a5740 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56079fcdfe70 0x56079fce2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.076+0000 7fea621a5740 4 client.18741 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:28:02.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.076+0000 7fea621a5740 1 -- 172.21.15.86:0/444071057 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56079fbd18b0 con 0x56079fcdfe70 2024-04-17T07:28:02.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.077+0000 7fea57e00640 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56079fcdfe70 0x56079fce2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:02.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.080+0000 7fea57e00640 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56079fcdfe70 0x56079fce2290 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f791caab2b1924 server_cookie=4f19c7210e5d5ad in_seq=0 out_seq=0 2024-04-17T07:28:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.080+0000 7fea42000640 10 client.18741 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.081+0000 7fea42000640 1 -- 172.21.15.86:0/444071057 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56079fbd18b0 con 0x56079fcdfe70 2024-04-17T07:28:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.081+0000 7fea42000640 10 client.18741 handle_command_reply: tid=0 2024-04-17T07:28:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.081+0000 7fea621a5740 1 client.18741 shutdown 2024-04-17T07:28:02.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.082+0000 7fea621a5740 1 -- 172.21.15.86:0/444071057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc64160 msgr2=0x56079fc726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:02.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.082+0000 7fea621a5740 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc64160 0x56079fc726f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fea50017510 tx=0x7fea500042b0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.083+0000 7fea621a5740 1 -- 172.21.15.86:0/444071057 shutdown_connections 2024-04-17T07:28:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.083+0000 7fea621a5740 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56079fcdfe70 0x56079fce2290 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.083+0000 7fea621a5740 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56079fc69cc0 0x56079fcde380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.083+0000 7fea621a5740 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56079fc64b50 0x56079fc72c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.083+0000 7fea42000640 0 client.18741 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.083+0000 7fea621a5740 1 --2- 172.21.15.86:0/444071057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56079fc64160 0x56079fc726f0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.083+0000 7fea621a5740 1 -- 172.21.15.86:0/444071057 >> 172.21.15.86:0/444071057 conn(0x56079fc5f6d0 msgr2=0x56079fc62930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.083+0000 7fea621a5740 1 -- 172.21.15.86:0/444071057 shutdown_connections 2024-04-17T07:28:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.083+0000 7fea621a5740 1 -- 172.21.15.86:0/444071057 wait complete. 2024-04-17T07:28:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.083+0000 7fea621a5740 20 client.18741 trim_cache size 0 max 16384 2024-04-17T07:28:02.084 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:28:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.084+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea3c0a3e30 msgr2=0x7fea3c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.084+0000 7fea5f400640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea3c0a3e30 0x7fea3c0a62f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fea44009830 tx=0x7fea4402e040 comp rx=0 tx=0).stop 2024-04-17T07:28:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.084+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea5810a0e0 msgr2=0x7fea581ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.084+0000 7fea5f400640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea5810a0e0 0x7fea581ae6d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fea4800e970 tx=0x7fea4800ee30 comp rx=0 tx=0).stop 2024-04-17T07:28:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.085+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 shutdown_connections 2024-04-17T07:28:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.085+0000 7fea5f400640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea3c0a3e30 0x7fea3c0a62f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.085+0000 7fea5f400640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea5810a0e0 0x7fea581ae6d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.085+0000 7fea5f400640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea58109720 0x7fea5807fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.085+0000 7fea5f400640 1 --2- 172.21.15.86:0/449621200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea58108530 0x7fea5807f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:02.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.085+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 >> 172.21.15.86:0/449621200 conn(0x7fea580774d0 msgr2=0x7fea5810ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:02.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.085+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 shutdown_connections 2024-04-17T07:28:02.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:02.085+0000 7fea5f400640 1 -- 172.21.15.86:0/449621200 wait complete. 2024-04-17T07:28:03.118 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:28:03.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:28:03.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:28:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.472+0000 7fed1ca00640 1 -- 172.21.15.86:0/3809949127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18101f60 msgr2=0x7fed181023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.472+0000 7fed1ca00640 1 --2- 172.21.15.86:0/3809949127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18101f60 0x7fed181023e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fed04009900 tx=0x7fed0402bc30 comp rx=0 tx=0).stop 2024-04-17T07:28:03.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.473+0000 7fed1ca00640 1 -- 172.21.15.86:0/3809949127 shutdown_connections 2024-04-17T07:28:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.473+0000 7fed1ca00640 1 --2- 172.21.15.86:0/3809949127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed18102920 0x7fed181091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.473+0000 7fed1ca00640 1 --2- 172.21.15.86:0/3809949127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18101f60 0x7fed181023e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.473+0000 7fed1ca00640 1 --2- 172.21.15.86:0/3809949127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed18100d70 0x7fed18101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.473+0000 7fed1ca00640 1 -- 172.21.15.86:0/3809949127 >> 172.21.15.86:0/3809949127 conn(0x7fed180fc300 msgr2=0x7fed180fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.474+0000 7fed1ca00640 1 -- 172.21.15.86:0/3809949127 shutdown_connections 2024-04-17T07:28:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.474+0000 7fed1ca00640 1 -- 172.21.15.86:0/3809949127 wait complete. 2024-04-17T07:28:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.476+0000 7fed1ca00640 1 Processor -- start 2024-04-17T07:28:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.477+0000 7fed1ca00640 1 -- start start 2024-04-17T07:28:03.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.478+0000 7fed1ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed18100d70 0x7fed18106af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.478+0000 7fed1ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18101f60 0x7fed18107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.478+0000 7fed1ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed18102920 0x7fed18103660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.478+0000 7fed1ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed1810da60 con 0x7fed18102920 2024-04-17T07:28:03.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.478+0000 7fed1ca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed1810d8e0 con 0x7fed18101f60 2024-04-17T07:28:03.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.478+0000 7fed1ca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed1810dbe0 con 0x7fed18100d70 2024-04-17T07:28:03.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.478+0000 7fed16800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed18100d70 0x7fed18106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.479+0000 7fed17200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed18102920 0x7fed18103660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.479+0000 7fed15e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18101f60 0x7fed18107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.479+0000 7fed15e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18101f60 0x7fed18107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39240/0 (socket says 172.21.15.86:39240) 2024-04-17T07:28:03.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.479+0000 7fed15e00640 1 -- 172.21.15.86:0/2833477404 learned_addr learned my addr 172.21.15.86:0/2833477404 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:03.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.480+0000 7fed15e00640 1 -- 172.21.15.86:0/2833477404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed18100d70 msgr2=0x7fed18106af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.480+0000 7fed15e00640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed18100d70 0x7fed18106af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.480+0000 7fed15e00640 1 -- 172.21.15.86:0/2833477404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed18102920 msgr2=0x7fed18103660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.480+0000 7fed15e00640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed18102920 0x7fed18103660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.480+0000 7fed15e00640 1 -- 172.21.15.86:0/2833477404 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed18103f20 con 0x7fed18101f60 2024-04-17T07:28:03.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.481+0000 7fed15e00640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18101f60 0x7fed18107030 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fed04000c00 tx=0x7fed040042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:03.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.482+0000 7fed0f400640 1 -- 172.21.15.86:0/2833477404 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04043020 con 0x7fed18101f60 2024-04-17T07:28:03.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.482+0000 7fed0f400640 1 -- 172.21.15.86:0/2833477404 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed04004aa0 con 0x7fed18101f60 2024-04-17T07:28:03.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.482+0000 7fed0f400640 1 -- 172.21.15.86:0/2833477404 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04003c90 con 0x7fed18101f60 2024-04-17T07:28:03.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.483+0000 7fed1ca00640 1 -- 172.21.15.86:0/2833477404 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed181a67e0 con 0x7fed18101f60 2024-04-17T07:28:03.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.483+0000 7fed1ca00640 1 -- 172.21.15.86:0/2833477404 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed181a6c10 con 0x7fed18101f60 2024-04-17T07:28:03.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.488+0000 7fed201b1740 1 Processor -- start 2024-04-17T07:28:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.488+0000 7fed201b1740 1 -- start start 2024-04-17T07:28:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.489+0000 7fed201b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4fdf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.489+0000 7fed201b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4fe4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.489+0000 7fed201b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fe4c00 0x55f1a4fe7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.489+0000 7fed201b1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f1a4f44cb0 con 0x55f1a4fdfa90 2024-04-17T07:28:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.489+0000 7fed201b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f1a4eeb370 con 0x55f1a4fdf0a0 2024-04-17T07:28:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.489+0000 7fed201b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f1a4f40c70 con 0x55f1a4fe4c00 2024-04-17T07:28:03.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.490+0000 7fed17200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4fdf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.490+0000 7fed16800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4fe4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.490+0000 7fed17200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4fdf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39244/0 (socket says 172.21.15.86:39244) 2024-04-17T07:28:03.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.491+0000 7fed17200640 1 -- 172.21.15.86:0/2615303264 learned_addr learned my addr 172.21.15.86:0/2615303264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.491+0000 7fed16800640 1 -- 172.21.15.86:0/2615303264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fe4c00 msgr2=0x55f1a4fe7090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:28:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.491+0000 7fed16800640 1 --2- 172.21.15.86:0/2615303264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fe4c00 0x55f1a4fe7090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.491+0000 7fed16800640 1 -- 172.21.15.86:0/2615303264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fdf0a0 msgr2=0x55f1a4fdf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.491+0000 7fed16800640 1 --2- 172.21.15.86:0/2615303264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4fdf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.491+0000 7fed16800640 1 -- 172.21.15.86:0/2615303264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1a4f0a910 con 0x55f1a4fdfa90 2024-04-17T07:28:03.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.492+0000 7fed0f400640 1 -- 172.21.15.86:0/2833477404 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fed04046020 con 0x7fed18101f60 2024-04-17T07:28:03.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.494+0000 7fed16800640 1 --2- 172.21.15.86:0/2615303264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4fe4590 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fed08009970 tx=0x7fed08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68053bd1bf213200 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:03.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.495+0000 7fed03400640 1 -- 172.21.15.86:0/2615303264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08017070 con 0x55f1a4fdfa90 2024-04-17T07:28:03.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.495+0000 7fed03400640 1 -- 172.21.15.86:0/2615303264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed08013470 con 0x55f1a4fdfa90 2024-04-17T07:28:03.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.495+0000 7fed03400640 1 -- 172.21.15.86:0/2615303264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed08012630 con 0x55f1a4fdfa90 2024-04-17T07:28:03.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.497+0000 7fed0f400640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fecfc0a3d80 0x7fecfc0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.497+0000 7fed0f400640 1 -- 172.21.15.86:0/2833477404 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fed040f2ac0 con 0x7fed18101f60 2024-04-17T07:28:03.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.498+0000 7fed201b1740 1 -- 172.21.15.86:0/2615303264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 msgr2=0x55f1a4fe4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.498+0000 7fed201b1740 1 --2- 172.21.15.86:0/2615303264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4fe4590 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fed08009970 tx=0x7fed08009e30 comp rx=0 tx=0).stop 2024-04-17T07:28:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.499+0000 7fed17200640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fecfc0a3d80 0x7fecfc0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.499+0000 7fed201b1740 1 -- 172.21.15.86:0/2615303264 shutdown_connections 2024-04-17T07:28:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.499+0000 7fed201b1740 1 --2- 172.21.15.86:0/2615303264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fe4c00 0x55f1a4fe7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.499+0000 7fed201b1740 1 --2- 172.21.15.86:0/2615303264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4fe4590 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.499+0000 7fed201b1740 1 --2- 172.21.15.86:0/2615303264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4fdf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.499+0000 7fed201b1740 1 -- 172.21.15.86:0/2615303264 >> 172.21.15.86:0/2615303264 conn(0x55f1a4fda610 msgr2=0x55f1a4fdaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.499+0000 7fed201b1740 1 -- 172.21.15.86:0/2615303264 shutdown_connections 2024-04-17T07:28:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.499+0000 7fed201b1740 1 -- 172.21.15.86:0/2615303264 wait complete. 2024-04-17T07:28:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.500+0000 7fed201b1740 1 -- start start 2024-04-17T07:28:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.500+0000 7fed201b1740 1 Processor -- start 2024-04-17T07:28:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.502+0000 7fed17200640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fecfc0a3d80 0x7fecfc0a6240 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fed10005f10 tx=0x7fed10009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.502+0000 7fed201b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4ffa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.502+0000 7fed201b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4ffab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.502+0000 7fed201b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a5001bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.502+0000 7fed201b1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f1a4ff8bc0 con 0x55f1a4fdfa90 2024-04-17T07:28:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.502+0000 7fed201b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f1a4efe730 con 0x55f1a4fe4c00 2024-04-17T07:28:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.502+0000 7fed201b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f1a4ff8d40 con 0x55f1a4fdf0a0 2024-04-17T07:28:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.502+0000 7fed201b1740 10 client.0 _ll_register_callbacks cb 0x55f1a4f00b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.503+0000 7fed15e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4ffab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.503+0000 7fed15e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4ffab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41680/0 (socket says 172.21.15.86:41680) 2024-04-17T07:28:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.503+0000 7fed16800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4ffa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.503+0000 7fed15e00640 1 -- 172.21.15.86:0/355679299 learned_addr learned my addr 172.21.15.86:0/355679299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.503+0000 7fed17200640 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a5001bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.504+0000 7fed15e00640 1 -- 172.21.15.86:0/355679299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdf0a0 msgr2=0x55f1a4ffa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.504+0000 7fed15e00640 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4ffa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.504+0000 7fed15e00640 1 -- 172.21.15.86:0/355679299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 msgr2=0x55f1a5001bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.504+0000 7fed15e00640 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a5001bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.504+0000 7fed15e00640 1 -- 172.21.15.86:0/355679299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1a4d1f7e0 con 0x55f1a4fdfa90 2024-04-17T07:28:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.505+0000 7fed15e00640 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4ffab80 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fed040bae20 tx=0x7fed040be850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2088f25b7d14bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.505+0000 7fecf7e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:28:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.507+0000 7fecf7e00640 1 -- 172.21.15.86:0/355679299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed040035f0 con 0x55f1a4fdfa90 2024-04-17T07:28:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.507+0000 7fecf7e00640 1 -- 172.21.15.86:0/355679299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fed040bec20 con 0x55f1a4fdfa90 2024-04-17T07:28:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.507+0000 7fed201b1740 1 -- 172.21.15.86:0/355679299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed08013470 con 0x55f1a4fdfa90 2024-04-17T07:28:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.508+0000 7fecf7e00640 1 -- 172.21.15.86:0/355679299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed04003e30 con 0x55f1a4fdfa90 2024-04-17T07:28:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.508+0000 7fed201b1740 10 client.18747 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.508+0000 7fed201b1740 1 -- 172.21.15.86:0/355679299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1a4d7ff70 con 0x55f1a4fdfa90 2024-04-17T07:28:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.509+0000 7fecf7e00640 1 -- 172.21.15.86:0/355679299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed08013470 con 0x55f1a4fdfa90 2024-04-17T07:28:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.509+0000 7fecf7e00640 1 -- 172.21.15.86:0/355679299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fed040c71a0 con 0x55f1a4fdfa90 2024-04-17T07:28:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.510+0000 7fed201b1740 10 client.18747 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.510+0000 7fed201b1740 10 client.18747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.510+0000 7fed201b1740 20 client.18747 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.510+0000 7fed201b1740 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1a50063a0 0x55f1a5008800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.510+0000 7fed201b1740 4 client.18747 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.510+0000 7fed201b1740 1 -- 172.21.15.86:0/355679299 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1a4f526d0 con 0x55f1a50063a0 2024-04-17T07:28:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.511+0000 7fed16800640 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1a50063a0 0x55f1a5008800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.512+0000 7fed16800640 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1a50063a0 0x55f1a5008800 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c7fcbf8c25b8ef server_cookie=4f80587ca862a1a2 in_seq=0 out_seq=0 2024-04-17T07:28:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.512+0000 7fecf7e00640 10 client.18747 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.514+0000 7fecf7e00640 1 -- 172.21.15.86:0/355679299 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1a4f526d0 con 0x55f1a50063a0 2024-04-17T07:28:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.514+0000 7fecf7e00640 10 client.18747 handle_command_reply: tid=0 2024-04-17T07:28:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.514+0000 7fed201b1740 1 client.18747 shutdown 2024-04-17T07:28:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.515+0000 7fed201b1740 1 -- 172.21.15.86:0/355679299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 msgr2=0x55f1a4ffab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.515+0000 7fed201b1740 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4ffab80 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fed040bae20 tx=0x7fed040be850 comp rx=0 tx=0).stop 2024-04-17T07:28:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.515+0000 7fed201b1740 1 -- 172.21.15.86:0/355679299 shutdown_connections 2024-04-17T07:28:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.515+0000 7fed201b1740 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1a50063a0 0x55f1a5008800 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.515+0000 7fed201b1740 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a5001bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.515+0000 7fed201b1740 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a4ffab80 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.515+0000 7fed201b1740 1 --2- 172.21.15.86:0/355679299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4ffa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.515+0000 7fed201b1740 1 -- 172.21.15.86:0/355679299 >> 172.21.15.86:0/355679299 conn(0x55f1a4fda610 msgr2=0x55f1a4fd9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.515+0000 7fecf7e00640 0 client.18747 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.515+0000 7fed201b1740 1 -- 172.21.15.86:0/355679299 shutdown_connections 2024-04-17T07:28:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.516+0000 7fed201b1740 1 -- 172.21.15.86:0/355679299 wait complete. 2024-04-17T07:28:03.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.516+0000 7fed201b1740 20 client.18747 trim_cache size 0 max 16384 2024-04-17T07:28:03.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.563+0000 7fed201b1740 1 Processor -- start 2024-04-17T07:28:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.563+0000 7fed201b1740 1 -- start start 2024-04-17T07:28:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.564+0000 7fed201b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4fd9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.564+0000 7fed201b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a5001b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.564+0000 7fed201b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a4fd9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.564+0000 7fed201b1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed181a8840 con 0x55f1a4fdfa90 2024-04-17T07:28:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.564+0000 7fed201b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed181aa380 con 0x55f1a4fe4c00 2024-04-17T07:28:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.564+0000 7fed201b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed181a8f10 con 0x55f1a4fdf0a0 2024-04-17T07:28:03.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.565+0000 7fed16800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4fd9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.565+0000 7fed17200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a4fd9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.565+0000 7fed17200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a4fd9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39254/0 (socket says 172.21.15.86:39254) 2024-04-17T07:28:03.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.565+0000 7fed17200640 1 -- 172.21.15.86:0/63911928 learned_addr learned my addr 172.21.15.86:0/63911928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:03.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.566+0000 7fed15e00640 1 --2- 172.21.15.86:0/63911928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a5001b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.566+0000 7fed17200640 1 -- 172.21.15.86:0/63911928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdf0a0 msgr2=0x55f1a4fd9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.566+0000 7fed17200640 1 --2- 172.21.15.86:0/63911928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4fd9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.566+0000 7fed17200640 1 -- 172.21.15.86:0/63911928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 msgr2=0x55f1a5001b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.566+0000 7fed17200640 1 --2- 172.21.15.86:0/63911928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a5001b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.566+0000 7fed17200640 1 -- 172.21.15.86:0/63911928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1a4c17170 con 0x55f1a4fe4c00 2024-04-17T07:28:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.566+0000 7fed15e00640 1 --2- 172.21.15.86:0/63911928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a5001b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:03.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.567+0000 7fed17200640 1 --2- 172.21.15.86:0/63911928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a4fd9640 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fed10010860 tx=0x7fed10015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61164b70f90e0bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:03.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.568+0000 7fed02a00640 1 -- 172.21.15.86:0/63911928 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10004b20 con 0x55f1a4fe4c00 2024-04-17T07:28:03.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.568+0000 7fed02a00640 1 -- 172.21.15.86:0/63911928 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed10004cc0 con 0x55f1a4fe4c00 2024-04-17T07:28:03.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.568+0000 7fed02a00640 1 -- 172.21.15.86:0/63911928 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10027670 con 0x55f1a4fe4c00 2024-04-17T07:28:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.570+0000 7fed201b1740 1 -- 172.21.15.86:0/63911928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 msgr2=0x55f1a4fd9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.570+0000 7fed201b1740 1 --2- 172.21.15.86:0/63911928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a4fd9640 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fed10010860 tx=0x7fed10015000 comp rx=0 tx=0).stop 2024-04-17T07:28:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.570+0000 7fed201b1740 1 -- 172.21.15.86:0/63911928 shutdown_connections 2024-04-17T07:28:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.570+0000 7fed201b1740 1 --2- 172.21.15.86:0/63911928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a4fd9640 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.570+0000 7fed201b1740 1 --2- 172.21.15.86:0/63911928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdfa90 0x55f1a5001b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.570+0000 7fed201b1740 1 --2- 172.21.15.86:0/63911928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4fd9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.570+0000 7fed201b1740 1 -- 172.21.15.86:0/63911928 >> 172.21.15.86:0/63911928 conn(0x55f1a4fda610 msgr2=0x55f1a4fe1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.570+0000 7fed201b1740 1 -- 172.21.15.86:0/63911928 shutdown_connections 2024-04-17T07:28:03.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.571+0000 7fed201b1740 1 -- 172.21.15.86:0/63911928 wait complete. 2024-04-17T07:28:03.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.572+0000 7fed201b1740 1 -- start start 2024-04-17T07:28:03.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.572+0000 7fed201b1740 1 Processor -- start 2024-04-17T07:28:03.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.574+0000 7fed201b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4ffee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.574+0000 7fed201b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdfa90 0x55f1a4fff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.574+0000 7fed201b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a50003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.574+0000 7fed201b1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed181a9cb0 con 0x55f1a4fdf0a0 2024-04-17T07:28:03.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.574+0000 7fed201b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed181a95e0 con 0x55f1a4fe4c00 2024-04-17T07:28:03.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.574+0000 7fed201b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed181a20f0 con 0x55f1a4fdfa90 2024-04-17T07:28:03.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.574+0000 7fed201b1740 10 client.0 _ll_register_callbacks cb 0x7fed04009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:03.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.575+0000 7fed15e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdfa90 0x55f1a4fff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.575+0000 7fed15e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdfa90 0x55f1a4fff3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58852/0 (socket says 172.21.15.86:58852) 2024-04-17T07:28:03.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.575+0000 7fed15e00640 1 -- 172.21.15.86:0/794721981 learned_addr learned my addr 172.21.15.86:0/794721981 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:03.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.575+0000 7fed17200640 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a50003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.575+0000 7fed16800640 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4ffee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.575+0000 7fed17200640 1 -- 172.21.15.86:0/794721981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdfa90 msgr2=0x55f1a4fff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.575+0000 7fed17200640 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdfa90 0x55f1a4fff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.575+0000 7fed17200640 1 -- 172.21.15.86:0/794721981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdf0a0 msgr2=0x55f1a4ffee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.575+0000 7fed17200640 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4ffee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.575+0000 7fed17200640 1 -- 172.21.15.86:0/794721981 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1a50009a0 con 0x55f1a4fe4c00 2024-04-17T07:28:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.576+0000 7fed17200640 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a50003b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fed10009c20 tx=0x7fed10005120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39ef35062779e45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.576+0000 7fecf7e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:03.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.577+0000 7fecf7e00640 1 -- 172.21.15.86:0/794721981 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10035020 con 0x55f1a4fe4c00 2024-04-17T07:28:03.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.577+0000 7fed201b1740 1 -- 172.21.15.86:0/794721981 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed10004cc0 con 0x55f1a4fe4c00 2024-04-17T07:28:03.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.577+0000 7fecf7e00640 1 -- 172.21.15.86:0/794721981 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fed10005520 con 0x55f1a4fe4c00 2024-04-17T07:28:03.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.577+0000 7fecf7e00640 1 -- 172.21.15.86:0/794721981 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed10040050 con 0x55f1a4fe4c00 2024-04-17T07:28:03.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.577+0000 7fed201b1740 10 client.37661 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:03.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.577+0000 7fed201b1740 1 -- 172.21.15.86:0/794721981 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1a5001ca0 con 0x55f1a4fe4c00 2024-04-17T07:28:03.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.578+0000 7fecf7e00640 1 -- 172.21.15.86:0/794721981 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed10004cc0 con 0x55f1a4fe4c00 2024-04-17T07:28:03.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.578+0000 7fecf7e00640 1 -- 172.21.15.86:0/794721981 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fed1001e410 con 0x55f1a4fe4c00 2024-04-17T07:28:03.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.578+0000 7fed201b1740 10 client.37661 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:03.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.578+0000 7fed201b1740 10 client.37661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:03.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.578+0000 7fed201b1740 20 client.37661 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:03.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.579+0000 7fed201b1740 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1a506ceb0 0x55f1a506f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.579+0000 7fed201b1740 4 client.37661 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:28:03.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.579+0000 7fed201b1740 1 -- 172.21.15.86:0/794721981 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f1a4f4c540 con 0x55f1a506ceb0 2024-04-17T07:28:03.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.579+0000 7fed16800640 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1a506ceb0 0x55f1a506f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:03 smithi086 ceph-mon[31733]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.1 MiB/s wr, 643 op/s 2024-04-17T07:28:03.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.580+0000 7fed16800640 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1a506ceb0 0x55f1a506f2d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83fb8ff5e1a0922 server_cookie=4dbfd8712537ba03 in_seq=0 out_seq=0 2024-04-17T07:28:03.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.580+0000 7fecf7e00640 10 client.37661 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:03.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.585+0000 7fecf7e00640 1 -- 172.21.15.86:0/794721981 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f1a4f4c540 con 0x55f1a506ceb0 2024-04-17T07:28:03.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.586+0000 7fecf7e00640 10 client.37661 handle_command_reply: tid=0 2024-04-17T07:28:03.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.586+0000 7fed201b1740 1 client.37661 shutdown 2024-04-17T07:28:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.586+0000 7fed201b1740 1 -- 172.21.15.86:0/794721981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 msgr2=0x55f1a50003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.586+0000 7fed201b1740 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a50003b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fed10009c20 tx=0x7fed10005120 comp rx=0 tx=0).stop 2024-04-17T07:28:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.587+0000 7fed201b1740 1 -- 172.21.15.86:0/794721981 shutdown_connections 2024-04-17T07:28:03.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.587+0000 7fed201b1740 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1a506ceb0 0x55f1a506f2d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.587+0000 7fed201b1740 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1a4fe4c00 0x55f1a50003b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.587+0000 7fed201b1740 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1a4fdfa90 0x55f1a4fff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.587+0000 7fed201b1740 1 --2- 172.21.15.86:0/794721981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1a4fdf0a0 0x55f1a4ffee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.587+0000 7fed201b1740 1 -- 172.21.15.86:0/794721981 >> 172.21.15.86:0/794721981 conn(0x55f1a4fda610 msgr2=0x55f1a4fdd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.587+0000 7fecf7e00640 0 client.37661 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.588+0000 7fed201b1740 1 -- 172.21.15.86:0/794721981 shutdown_connections 2024-04-17T07:28:03.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.588+0000 7fed201b1740 1 -- 172.21.15.86:0/794721981 wait complete. 2024-04-17T07:28:03.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.590+0000 7fed201b1740 20 client.37661 trim_cache size 0 max 16384 2024-04-17T07:28:03.590 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:28:03.591 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:28:03.591 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "06c5aa49-e95c-4461-82ad-c788884fed4b", 2024-04-17T07:28:03.591 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:28:03.591 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:28:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.591+0000 7fed1ca00640 1 -- 172.21.15.86:0/2833477404 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fecfc0a3d80 msgr2=0x7fecfc0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.591+0000 7fed1ca00640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fecfc0a3d80 0x7fecfc0a6240 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fed10005f10 tx=0x7fed10009500 comp rx=0 tx=0).stop 2024-04-17T07:28:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.592+0000 7fed1ca00640 1 -- 172.21.15.86:0/2833477404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18101f60 msgr2=0x7fed18107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.592+0000 7fed1ca00640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18101f60 0x7fed18107030 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fed04000c00 tx=0x7fed040042f0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.592+0000 7fed1ca00640 1 -- 172.21.15.86:0/2833477404 shutdown_connections 2024-04-17T07:28:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.592+0000 7fed1ca00640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fecfc0a3d80 0x7fecfc0a6240 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.592+0000 7fed1ca00640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed18102920 0x7fed18103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.592+0000 7fed1ca00640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18101f60 0x7fed18107030 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.592+0000 7fed1ca00640 1 --2- 172.21.15.86:0/2833477404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed18100d70 0x7fed18106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.592+0000 7fed1ca00640 1 -- 172.21.15.86:0/2833477404 >> 172.21.15.86:0/2833477404 conn(0x7fed180fc300 msgr2=0x7fed18109b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.593+0000 7fed1ca00640 1 -- 172.21.15.86:0/2833477404 shutdown_connections 2024-04-17T07:28:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.593+0000 7fed1ca00640 1 -- 172.21.15.86:0/2833477404 wait complete. 2024-04-17T07:28:03.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:28:03.628 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:28:03.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:03 smithi098 ceph-mon[35418]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.1 MiB/s wr, 643 op/s 2024-04-17T07:28:03.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:03 smithi103 ceph-mon[35495]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.1 MiB/s wr, 643 op/s 2024-04-17T07:28:03.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.966+0000 7f32c6400640 1 -- 172.21.15.86:0/400778129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c010e2a0 msgr2=0x7f32c010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.966+0000 7f32c6400640 1 --2- 172.21.15.86:0/400778129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c010e2a0 0x7f32c010e750 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009ca0 tx=0x7f32b802c100 comp rx=0 tx=0).stop 2024-04-17T07:28:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.967+0000 7f32c6400640 1 -- 172.21.15.86:0/400778129 shutdown_connections 2024-04-17T07:28:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.967+0000 7f32c6400640 1 --2- 172.21.15.86:0/400778129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c010e2a0 0x7f32c010e750 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.967+0000 7f32c6400640 1 --2- 172.21.15.86:0/400778129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c010b8d0 0x7f32c010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.967+0000 7f32c6400640 1 --2- 172.21.15.86:0/400778129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32c0108f60 0x7f32c010b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.967+0000 7f32c6400640 1 -- 172.21.15.86:0/400778129 >> 172.21.15.86:0/400778129 conn(0x7f32c006edf0 msgr2=0x7f32c0071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:03.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.967+0000 7f32c6400640 1 -- 172.21.15.86:0/400778129 shutdown_connections 2024-04-17T07:28:03.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.968+0000 7f32c6400640 1 -- 172.21.15.86:0/400778129 wait complete. 2024-04-17T07:28:03.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.969+0000 7f32c6400640 1 Processor -- start 2024-04-17T07:28:03.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.969+0000 7f32c6400640 1 -- start start 2024-04-17T07:28:03.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.970+0000 7f32c6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0108f60 0x7f32c01a7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.970+0000 7f32c6400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32c010b8d0 0x7f32c01a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.970+0000 7f32c6400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c010e2a0 0x7f32c01ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.970+0000 7f32c6400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32c011c8e0 con 0x7f32c0108f60 2024-04-17T07:28:03.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.970+0000 7f32c6400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32c011c760 con 0x7f32c010e2a0 2024-04-17T07:28:03.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.970+0000 7f32c6400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32c011ca60 con 0x7f32c010b8d0 2024-04-17T07:28:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.971+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c010e2a0 0x7f32c01ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.971+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c010e2a0 0x7f32c01ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39288/0 (socket says 172.21.15.86:39288) 2024-04-17T07:28:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.971+0000 7f32bfe00640 1 -- 172.21.15.86:0/1103250357 learned_addr learned my addr 172.21.15.86:0/1103250357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.971+0000 7f32bea00640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32c010b8d0 0x7f32c01a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.971+0000 7f32bf400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0108f60 0x7f32c01a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.972+0000 7f32bf400640 1 -- 172.21.15.86:0/1103250357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32c010b8d0 msgr2=0x7f32c01a75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.972+0000 7f32bf400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32c010b8d0 0x7f32c01a75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.972+0000 7f32bf400640 1 -- 172.21.15.86:0/1103250357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c010e2a0 msgr2=0x7f32c01ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.972+0000 7f32bf400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c010e2a0 0x7f32c01ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.972+0000 7f32bf400640 1 -- 172.21.15.86:0/1103250357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c01aec30 con 0x7f32c0108f60 2024-04-17T07:28:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.972+0000 7f32bfe00640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c010e2a0 0x7f32c01ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.973+0000 7f32bf400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0108f60 0x7f32c01a7060 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f32b000d6d0 tx=0x7f32b000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.973+0000 7f32b7e00640 1 -- 172.21.15.86:0/1103250357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0017070 con 0x7f32c0108f60 2024-04-17T07:28:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.973+0000 7f32b7e00640 1 -- 172.21.15.86:0/1103250357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32b00047a0 con 0x7f32c0108f60 2024-04-17T07:28:03.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.973+0000 7f32b7e00640 1 -- 172.21.15.86:0/1103250357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0009ba0 con 0x7f32c0108f60 2024-04-17T07:28:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.973+0000 7f32c6400640 1 -- 172.21.15.86:0/1103250357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c01aef20 con 0x7f32c0108f60 2024-04-17T07:28:03.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.974+0000 7f32c6400640 1 -- 172.21.15.86:0/1103250357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01af330 con 0x7f32c0108f60 2024-04-17T07:28:03.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.979+0000 7f32c91c8740 1 Processor -- start 2024-04-17T07:28:03.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.979+0000 7f32c91c8740 1 -- start start 2024-04-17T07:28:03.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.980+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a7725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.980+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 0x561b7a777680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.980+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 0x561b7a77a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.981+0000 7f32c91c8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b7a6d9120 con 0x561b7a777cf0 2024-04-17T07:28:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.981+0000 7f32c91c8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561b7a67f340 con 0x561b7a772b80 2024-04-17T07:28:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.981+0000 7f32c91c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561b7a6d50e0 con 0x561b7a772190 2024-04-17T07:28:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.981+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a7725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.981+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a7725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58888/0 (socket says 172.21.15.86:58888) 2024-04-17T07:28:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.981+0000 7f32bfe00640 1 -- 172.21.15.86:0/34996828 learned_addr learned my addr 172.21.15.86:0/34996828 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:03.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.981+0000 7f32b7e00640 1 -- 172.21.15.86:0/1103250357 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f32b0004950 con 0x7f32c0108f60 2024-04-17T07:28:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.982+0000 7f32bf400640 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 0x561b7a777680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.982+0000 7f32bfe00640 1 -- 172.21.15.86:0/34996828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 msgr2=0x561b7a777680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.982+0000 7f32bfe00640 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 0x561b7a777680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.982+0000 7f32bea00640 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 0x561b7a77a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.982+0000 7f32bfe00640 1 -- 172.21.15.86:0/34996828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 msgr2=0x561b7a77a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.982+0000 7f32bfe00640 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 0x561b7a77a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.982+0000 7f32bfe00640 1 -- 172.21.15.86:0/34996828 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b7a4b57d0 con 0x561b7a772190 2024-04-17T07:28:03.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.983+0000 7f32bf400640 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 0x561b7a777680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:03.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.983+0000 7f32bea00640 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 0x561b7a77a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:03.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.983+0000 7f32b7e00640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a40a3c80 0x7f32a40a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.983+0000 7f32bfe00640 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a7725b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32b8005f90 tx=0x7f32b8009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2efcc5d96e5030de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:03.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.984+0000 7f32b7e00640 1 -- 172.21.15.86:0/1103250357 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f32b00cff30 con 0x7f32c0108f60 2024-04-17T07:28:03.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.984+0000 7f32bf400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a40a3c80 0x7f32a40a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.984+0000 7f32bf400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a40a3c80 0x7f32a40a6140 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009610 tx=0x7f32b0004eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:03.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.985+0000 7f32b4c00640 1 -- 172.21.15.86:0/34996828 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8043070 con 0x561b7a772190 2024-04-17T07:28:03.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.985+0000 7f32b4c00640 1 -- 172.21.15.86:0/34996828 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32b8003e20 con 0x561b7a772190 2024-04-17T07:28:03.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.985+0000 7f32b4c00640 1 -- 172.21.15.86:0/34996828 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8039040 con 0x561b7a772190 2024-04-17T07:28:03.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.986+0000 7f32c91c8740 1 -- 172.21.15.86:0/34996828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 msgr2=0x561b7a7725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:03.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.986+0000 7f32c91c8740 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a7725b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32b8005f90 tx=0x7f32b8009800 comp rx=0 tx=0).stop 2024-04-17T07:28:03.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.987+0000 7f32c91c8740 1 -- 172.21.15.86:0/34996828 shutdown_connections 2024-04-17T07:28:03.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.987+0000 7f32c91c8740 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 0x561b7a77a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.987+0000 7f32c91c8740 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 0x561b7a777680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.987+0000 7f32c91c8740 1 --2- 172.21.15.86:0/34996828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a7725b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.987+0000 7f32c91c8740 1 -- 172.21.15.86:0/34996828 >> 172.21.15.86:0/34996828 conn(0x561b7a76d700 msgr2=0x561b7a76db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.987+0000 7f32c91c8740 1 -- 172.21.15.86:0/34996828 shutdown_connections 2024-04-17T07:28:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.987+0000 7f32c91c8740 1 -- 172.21.15.86:0/34996828 wait complete. 2024-04-17T07:28:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.988+0000 7f32c91c8740 1 -- start start 2024-04-17T07:28:03.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.988+0000 7f32c91c8740 1 Processor -- start 2024-04-17T07:28:03.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.989+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 0x561b7a77c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.989+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 0x561b7a77cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.989+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a777cf0 0x561b7a783c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.989+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 0x561b7a77c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.989+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 0x561b7a77c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:41718/0 (socket says 172.21.15.86:41718) 2024-04-17T07:28:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.989+0000 7f32bfe00640 1 -- 172.21.15.86:0/1960668742 learned_addr learned my addr 172.21.15.86:0/1960668742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.990+0000 7f32bea00640 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 0x561b7a77cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.990+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b7a77ac40 con 0x561b7a772190 2024-04-17T07:28:03.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.990+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561b7a692710 con 0x561b7a777cf0 2024-04-17T07:28:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.990+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561b7a77adc0 con 0x561b7a772b80 2024-04-17T07:28:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.990+0000 7f32bfe00640 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a777cf0 0x561b7a783c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.990+0000 7f32c91c8740 10 client.0 _ll_register_callbacks cb 0x561b7a68daf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:03.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.991+0000 7f32bfe00640 1 -- 172.21.15.86:0/1960668742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 msgr2=0x561b7a77cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.991+0000 7f32bfe00640 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 0x561b7a77cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.991+0000 7f32bfe00640 1 -- 172.21.15.86:0/1960668742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 msgr2=0x561b7a77c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.991+0000 7f32bfe00640 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 0x561b7a77c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.991+0000 7f32bfe00640 1 -- 172.21.15.86:0/1960668742 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b7a3eb3f0 con 0x561b7a777cf0 2024-04-17T07:28:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.992+0000 7f32bfe00640 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a777cf0 0x561b7a783c00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009bf0 tx=0x7f32b8005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd7fd6fdd244db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.992+0000 7f32a9600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.993+0000 7f32a9600640 1 -- 172.21.15.86:0/1960668742 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8039050 con 0x561b7a777cf0 2024-04-17T07:28:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.993+0000 7f32a9600640 1 -- 172.21.15.86:0/1960668742 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f32b80327d0 con 0x561b7a777cf0 2024-04-17T07:28:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.993+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b8003e20 con 0x561b7a777cf0 2024-04-17T07:28:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.994+0000 7f32a9600640 1 -- 172.21.15.86:0/1960668742 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32b805c050 con 0x561b7a777cf0 2024-04-17T07:28:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.994+0000 7f32c91c8740 10 client.37670 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.994+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b7a3ab0f0 con 0x561b7a777cf0 2024-04-17T07:28:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.995+0000 7f32a9600640 1 -- 172.21.15.86:0/1960668742 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32b8003e20 con 0x561b7a777cf0 2024-04-17T07:28:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.995+0000 7f32a9600640 1 -- 172.21.15.86:0/1960668742 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f32b803f440 con 0x561b7a777cf0 2024-04-17T07:28:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.995+0000 7f32c91c8740 10 client.37670 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.995+0000 7f32c91c8740 10 client.37670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.996+0000 7f32c91c8740 20 client.37670 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.996+0000 7f32c91c8740 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561b7a788040 0x561b7a78a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.996+0000 7f32c91c8740 4 client.37670 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.996+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b7a6e6b40 con 0x561b7a788040 2024-04-17T07:28:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.997+0000 7f32bea00640 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561b7a788040 0x561b7a78a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:03.998+0000 7f32bea00640 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561b7a788040 0x561b7a78a430 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d40855f98fba1c server_cookie=62003113ed2cd6e7 in_seq=0 out_seq=0 2024-04-17T07:28:04.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.000+0000 7f32a9600640 10 client.37670 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.001+0000 7f32a9600640 1 -- 172.21.15.86:0/1960668742 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b7a6e6b40 con 0x561b7a788040 2024-04-17T07:28:04.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.001+0000 7f32a9600640 10 client.37670 handle_command_reply: tid=0 2024-04-17T07:28:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.002+0000 7f32c91c8740 1 client.37670 shutdown 2024-04-17T07:28:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.003+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a777cf0 msgr2=0x561b7a783c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.003+0000 7f32c91c8740 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a777cf0 0x561b7a783c00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009bf0 tx=0x7f32b8005ef0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.004+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 shutdown_connections 2024-04-17T07:28:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.004+0000 7f32c91c8740 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561b7a788040 0x561b7a78a430 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.004+0000 7f32c91c8740 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a777cf0 0x561b7a783c00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.004+0000 7f32c91c8740 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 0x561b7a77cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.004+0000 7f32c91c8740 1 --2- 172.21.15.86:0/1960668742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 0x561b7a77c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.004+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 >> 172.21.15.86:0/1960668742 conn(0x561b7a76d700 msgr2=0x561b7a76c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.004+0000 7f32a9600640 0 client.37670 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.005+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 shutdown_connections 2024-04-17T07:28:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.006+0000 7f32c91c8740 1 -- 172.21.15.86:0/1960668742 wait complete. 2024-04-17T07:28:04.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.006+0000 7f32c91c8740 20 client.37670 trim_cache size 0 max 16384 2024-04-17T07:28:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.043+0000 7f32c91c8740 1 Processor -- start 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.043+0000 7f32c91c8740 1 -- start start 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.044+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a76c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.044+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 0x561b7a783b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.044+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 0x561b7a76c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.044+0000 7f32c91c8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01b99a0 con 0x561b7a777cf0 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.044+0000 7f32c91c8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01bb4e0 con 0x561b7a772b80 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.044+0000 7f32c91c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01ba070 con 0x561b7a772190 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.044+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 0x561b7a76c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.044+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a76c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.045+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a76c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58906/0 (socket says 172.21.15.86:58906) 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.045+0000 7f32bfe00640 1 -- 172.21.15.86:0/488344708 learned_addr learned my addr 172.21.15.86:0/488344708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.045+0000 7f32bea00640 1 --2- 172.21.15.86:0/488344708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 0x561b7a783b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:04.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.045+0000 7f32bfe00640 1 -- 172.21.15.86:0/488344708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 msgr2=0x561b7a76c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:04.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.045+0000 7f32bfe00640 1 --2- 172.21.15.86:0/488344708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a76c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.045+0000 7f32bfe00640 1 -- 172.21.15.86:0/488344708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 msgr2=0x561b7a783b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:04.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.045+0000 7f32bfe00640 1 --2- 172.21.15.86:0/488344708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 0x561b7a783b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.045+0000 7f32bfe00640 1 -- 172.21.15.86:0/488344708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b7a76e170 con 0x561b7a777cf0 2024-04-17T07:28:04.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.046+0000 7f32bfe00640 1 --2- 172.21.15.86:0/488344708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 0x561b7a76c540 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f32b802c6e0 tx=0x7f32b8032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b341e58089d4c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:04.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.046+0000 7f32abe00640 1 -- 172.21.15.86:0/488344708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8039070 con 0x561b7a777cf0 2024-04-17T07:28:04.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.047+0000 7f32abe00640 1 -- 172.21.15.86:0/488344708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32b8003a80 con 0x561b7a777cf0 2024-04-17T07:28:04.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.047+0000 7f32abe00640 1 -- 172.21.15.86:0/488344708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8043040 con 0x561b7a777cf0 2024-04-17T07:28:04.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.048+0000 7f32c91c8740 1 -- 172.21.15.86:0/488344708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 msgr2=0x561b7a76c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:04.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.048+0000 7f32c91c8740 1 --2- 172.21.15.86:0/488344708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 0x561b7a76c540 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f32b802c6e0 tx=0x7f32b8032340 comp rx=0 tx=0).stop 2024-04-17T07:28:04.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.048+0000 7f32c91c8740 1 -- 172.21.15.86:0/488344708 shutdown_connections 2024-04-17T07:28:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.048+0000 7f32c91c8740 1 --2- 172.21.15.86:0/488344708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a777cf0 0x561b7a76c540 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.048+0000 7f32c91c8740 1 --2- 172.21.15.86:0/488344708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a772b80 0x561b7a783b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.048+0000 7f32c91c8740 1 --2- 172.21.15.86:0/488344708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772190 0x561b7a76c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.048+0000 7f32c91c8740 1 -- 172.21.15.86:0/488344708 >> 172.21.15.86:0/488344708 conn(0x561b7a76d700 msgr2=0x561b7a77b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.048+0000 7f32c91c8740 1 -- 172.21.15.86:0/488344708 shutdown_connections 2024-04-17T07:28:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.049+0000 7f32c91c8740 1 -- 172.21.15.86:0/488344708 wait complete. 2024-04-17T07:28:04.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.049+0000 7f32c91c8740 1 -- start start 2024-04-17T07:28:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.049+0000 7f32c91c8740 1 Processor -- start 2024-04-17T07:28:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.050+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 0x561b7a7816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.050+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 0x561b7a781c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.050+0000 7f32c91c8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a777cf0 0x561b7a782c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.050+0000 7f32c91c8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01bae10 con 0x561b7a772190 2024-04-17T07:28:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.050+0000 7f32c91c8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01ba740 con 0x561b7a777cf0 2024-04-17T07:28:04.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.050+0000 7f32c91c8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01b7970 con 0x561b7a772b80 2024-04-17T07:28:04.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.050+0000 7f32c91c8740 10 client.0 _ll_register_callbacks cb 0x7f32b8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:04.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.051+0000 7f32bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 0x561b7a7816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:04.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.051+0000 7f32bea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 0x561b7a781c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:04.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.051+0000 7f32bea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 0x561b7a781c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:58922/0 (socket says 172.21.15.86:58922) 2024-04-17T07:28:04.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.051+0000 7f32bea00640 1 -- 172.21.15.86:0/4025514373 learned_addr learned my addr 172.21.15.86:0/4025514373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:04.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.051+0000 7f32bea00640 1 -- 172.21.15.86:0/4025514373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a777cf0 msgr2=0x561b7a782c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:28:04.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.051+0000 7f32bea00640 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a777cf0 0x561b7a782c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.051+0000 7f32bea00640 1 -- 172.21.15.86:0/4025514373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 msgr2=0x561b7a7816d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:04.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.051+0000 7f32bea00640 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 0x561b7a7816d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.051+0000 7f32bea00640 1 -- 172.21.15.86:0/4025514373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b7a783210 con 0x561b7a772b80 2024-04-17T07:28:04.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.052+0000 7f32bfe00640 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 0x561b7a7816d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:04.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.052+0000 7f32bea00640 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 0x561b7a781c10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32c01a8520 tx=0x7f32ac004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d92e04e353f6f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:04.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.053+0000 7f32a9600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:28:04.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.053+0000 7f32a9600640 1 -- 172.21.15.86:0/4025514373 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac003a00 con 0x561b7a772b80 2024-04-17T07:28:04.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.053+0000 7f32a9600640 1 -- 172.21.15.86:0/4025514373 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f32ac003ba0 con 0x561b7a772b80 2024-04-17T07:28:04.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.054+0000 7f32c91c8740 1 -- 172.21.15.86:0/4025514373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b8003a80 con 0x561b7a772b80 2024-04-17T07:28:04.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.054+0000 7f32a9600640 1 -- 172.21.15.86:0/4025514373 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32ac02d050 con 0x561b7a772b80 2024-04-17T07:28:04.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.055+0000 7f32c91c8740 10 client.27718 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:04.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.055+0000 7f32c91c8740 1 -- 172.21.15.86:0/4025514373 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b7a7ee910 con 0x561b7a772b80 2024-04-17T07:28:04.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.056+0000 7f32a9600640 1 -- 172.21.15.86:0/4025514373 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32b8003a80 con 0x561b7a772b80 2024-04-17T07:28:04.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.056+0000 7f32a9600640 1 -- 172.21.15.86:0/4025514373 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f32ac030020 con 0x561b7a772b80 2024-04-17T07:28:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.057+0000 7f32c91c8740 10 client.27718 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.057+0000 7f32c91c8740 10 client.27718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.057+0000 7f32c91c8740 20 client.27718 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.057+0000 7f32c91c8740 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561b7a7ef390 0x561b7a7f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.057+0000 7f32c91c8740 4 client.27718 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:28:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.057+0000 7f32c91c8740 1 -- 172.21.15.86:0/4025514373 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b7a6e09b0 con 0x561b7a7ef390 2024-04-17T07:28:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.058+0000 7f32bfe00640 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561b7a7ef390 0x561b7a7f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.059+0000 7f32bfe00640 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561b7a7ef390 0x561b7a7f17b0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=108f8f863ff2266 server_cookie=13eb291317feee93 in_seq=0 out_seq=0 2024-04-17T07:28:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.060+0000 7f32a9600640 10 client.27718 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.060+0000 7f32a9600640 1 -- 172.21.15.86:0/4025514373 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561b7a6e09b0 con 0x561b7a7ef390 2024-04-17T07:28:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.060+0000 7f32a9600640 10 client.27718 handle_command_reply: tid=0 2024-04-17T07:28:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.060+0000 7f32c91c8740 1 client.27718 shutdown 2024-04-17T07:28:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.061+0000 7f32c91c8740 1 -- 172.21.15.86:0/4025514373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 msgr2=0x561b7a781c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.061+0000 7f32c91c8740 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 0x561b7a781c10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32c01a8520 tx=0x7f32ac004570 comp rx=0 tx=0).stop 2024-04-17T07:28:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.061+0000 7f32c91c8740 1 -- 172.21.15.86:0/4025514373 shutdown_connections 2024-04-17T07:28:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.061+0000 7f32c91c8740 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561b7a7ef390 0x561b7a7f17b0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.061+0000 7f32c91c8740 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561b7a777cf0 0x561b7a782c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.061+0000 7f32c91c8740 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b7a772b80 0x561b7a781c10 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32c01a8520 tx=0x7f32ac004570 comp rx=0 tx=0).stop 2024-04-17T07:28:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.061+0000 7f32c91c8740 1 --2- 172.21.15.86:0/4025514373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b7a772190 0x561b7a7816d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.061+0000 7f32c91c8740 1 -- 172.21.15.86:0/4025514373 >> 172.21.15.86:0/4025514373 conn(0x561b7a76d700 msgr2=0x561b7a76c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.061+0000 7f32a9600640 0 client.27718 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.061+0000 7f32c91c8740 1 -- 172.21.15.86:0/4025514373 shutdown_connections 2024-04-17T07:28:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.062+0000 7f32c91c8740 1 -- 172.21.15.86:0/4025514373 wait complete. 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.062+0000 7f32c91c8740 20 client.27718 trim_cache size 0 max 16384 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "06c5aa49-e95c-4461-82ad-c788884fed4b": { 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "06c5aa49-e95c-4461-82ad-c788884fed4b", 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:28:04.063 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:28:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.063+0000 7f32c6400640 1 -- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a40a3c80 msgr2=0x7f32a40a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.063+0000 7f32c6400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a40a3c80 0x7f32a40a6140 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009610 tx=0x7f32b0004eb0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.064+0000 7f32c6400640 1 -- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0108f60 msgr2=0x7f32c01a7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.064+0000 7f32c6400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0108f60 0x7f32c01a7060 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f32b000d6d0 tx=0x7f32b000db90 comp rx=0 tx=0).stop 2024-04-17T07:28:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.064+0000 7f32c6400640 1 -- 172.21.15.86:0/1103250357 shutdown_connections 2024-04-17T07:28:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.064+0000 7f32c6400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f32a40a3c80 0x7f32a40a6140 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.064+0000 7f32c6400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32c010e2a0 0x7f32c01ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.064+0000 7f32c6400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32c010b8d0 0x7f32c01a75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.064+0000 7f32c6400640 1 --2- 172.21.15.86:0/1103250357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c0108f60 0x7f32c01a7060 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.064+0000 7f32c6400640 1 -- 172.21.15.86:0/1103250357 >> 172.21.15.86:0/1103250357 conn(0x7f32c006edf0 msgr2=0x7f32c010bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.065+0000 7f32c6400640 1 -- 172.21.15.86:0/1103250357 shutdown_connections 2024-04-17T07:28:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:04.065+0000 7f32c6400640 1 -- 172.21.15.86:0/1103250357 wait complete. 2024-04-17T07:28:04.104 INFO:tasks.cephfs.filesystem:scrub status for tag:06c5aa49-e95c-4461-82ad-c788884fed4b - {'path': '/', 'tag': '06c5aa49-e95c-4461-82ad-c788884fed4b', 'options': 'recursive,force'} 2024-04-17T07:28:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 520.0 s, 601.6 tps, lat 165.251 ms stddev 84.083 2024-04-17T07:28:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:04 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:28:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:04 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:04 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:28:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:04 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:04.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:04 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:28:04.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:04 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:28:04.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:04 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:04.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:04 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:28:04.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:04 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:04.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:04 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:28:04.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:04 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:28:04.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:04 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:04.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:04 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:28:04.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:04 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:04.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:04 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:28:05.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:05 smithi098 ceph-mon[35418]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.1 MiB/s wr, 647 op/s 2024-04-17T07:28:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:05 smithi103 ceph-mon[35495]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.1 MiB/s wr, 647 op/s 2024-04-17T07:28:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:05 smithi086 ceph-mon[31733]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.1 MiB/s wr, 647 op/s 2024-04-17T07:28:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:07 smithi098 ceph-mon[35418]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.8 MiB/s wr, 549 op/s 2024-04-17T07:28:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:07 smithi103 ceph-mon[35495]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.8 MiB/s wr, 549 op/s 2024-04-17T07:28:07.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:07 smithi086 ceph-mon[31733]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.8 MiB/s wr, 549 op/s 2024-04-17T07:28:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 525.0 s, 685.4 tps, lat 147.320 ms stddev 60.503 2024-04-17T07:28:09.755 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:09 smithi098 ceph-mon[35418]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.9 MiB/s wr, 631 op/s 2024-04-17T07:28:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:09 smithi103 ceph-mon[35495]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.9 MiB/s wr, 631 op/s 2024-04-17T07:28:09.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:09 smithi086 ceph-mon[31733]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.9 MiB/s wr, 631 op/s 2024-04-17T07:28:10.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:10 smithi098 ceph-mon[35418]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 612 op/s 2024-04-17T07:28:10.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:10 smithi103 ceph-mon[35495]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 612 op/s 2024-04-17T07:28:10.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:10 smithi086 ceph-mon[31733]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 612 op/s 2024-04-17T07:28:13.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:12 smithi098 ceph-mon[35418]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 650 op/s 2024-04-17T07:28:13.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:12 smithi103 ceph-mon[35495]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 650 op/s 2024-04-17T07:28:13.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:12 smithi086 ceph-mon[31733]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.8 MiB/s wr, 650 op/s 2024-04-17T07:28:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 530.0 s, 555.6 tps, lat 153.452 ms stddev 105.607 2024-04-17T07:28:15.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:14 smithi098 ceph-mon[35418]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 6.6 MiB/s wr, 658 op/s 2024-04-17T07:28:15.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:14 smithi103 ceph-mon[35495]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 6.6 MiB/s wr, 658 op/s 2024-04-17T07:28:15.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:14 smithi086 ceph-mon[31733]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 6.6 MiB/s wr, 658 op/s 2024-04-17T07:28:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:16 smithi098 ceph-mon[35418]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.2 MiB/s wr, 598 op/s 2024-04-17T07:28:17.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:16 smithi103 ceph-mon[35495]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.2 MiB/s wr, 598 op/s 2024-04-17T07:28:17.083 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:16 smithi086 ceph-mon[31733]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.2 MiB/s wr, 598 op/s 2024-04-17T07:28:19.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:18 smithi098 ceph-mon[35418]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.1 MiB/s wr, 739 op/s 2024-04-17T07:28:19.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:18 smithi103 ceph-mon[35495]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.1 MiB/s wr, 739 op/s 2024-04-17T07:28:19.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:18 smithi086 ceph-mon[31733]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 8.1 MiB/s wr, 739 op/s 2024-04-17T07:28:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 535.0 s, 637.8 tps, lat 170.531 ms stddev 193.817 2024-04-17T07:28:21.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:20 smithi098 ceph-mon[35418]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 684 op/s 2024-04-17T07:28:21.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:20 smithi103 ceph-mon[35495]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 684 op/s 2024-04-17T07:28:21.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:20 smithi086 ceph-mon[31733]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.4 MiB/s wr, 684 op/s 2024-04-17T07:28:23.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:22 smithi098 ceph-mon[35418]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 740 op/s 2024-04-17T07:28:23.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:22 smithi103 ceph-mon[35495]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 740 op/s 2024-04-17T07:28:23.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:22 smithi086 ceph-mon[31733]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 740 op/s 2024-04-17T07:28:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 540.0 s, 528.8 tps, lat 199.219 ms stddev 186.068 2024-04-17T07:28:25.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:24 smithi098 ceph-mon[35418]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.6 MiB/s wr, 774 op/s 2024-04-17T07:28:25.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:24 smithi103 ceph-mon[35495]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.6 MiB/s wr, 774 op/s 2024-04-17T07:28:25.079 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:24 smithi086 ceph-mon[31733]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.6 MiB/s wr, 774 op/s 2024-04-17T07:28:27.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:26 smithi098 ceph-mon[35418]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 718 op/s 2024-04-17T07:28:27.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:26 smithi103 ceph-mon[35495]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 718 op/s 2024-04-17T07:28:27.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:26 smithi086 ceph-mon[31733]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 718 op/s 2024-04-17T07:28:29.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:28 smithi098 ceph-mon[35418]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 767 op/s 2024-04-17T07:28:29.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:28 smithi103 ceph-mon[35495]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 767 op/s 2024-04-17T07:28:29.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:28 smithi086 ceph-mon[31733]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 767 op/s 2024-04-17T07:28:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 545.0 s, 665.0 tps, lat 150.478 ms stddev 107.124 2024-04-17T07:28:31.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:30 smithi098 ceph-mon[35418]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 669 op/s 2024-04-17T07:28:31.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:30 smithi103 ceph-mon[35495]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 669 op/s 2024-04-17T07:28:31.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:30 smithi086 ceph-mon[31733]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 669 op/s 2024-04-17T07:28:33.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:32 smithi098 ceph-mon[35418]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.2 MiB/s wr, 721 op/s 2024-04-17T07:28:33.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:32 smithi103 ceph-mon[35495]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.2 MiB/s wr, 721 op/s 2024-04-17T07:28:33.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:32 smithi086 ceph-mon[31733]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.2 MiB/s wr, 721 op/s 2024-04-17T07:28:34.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:28:34.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:28:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 550.0 s, 705.6 tps, lat 141.132 ms stddev 70.277 2024-04-17T07:28:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.458+0000 7ff236600640 1 -- 172.21.15.86:0/2695031023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff230105290 msgr2=0x7ff230105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.458+0000 7ff236600640 1 --2- 172.21.15.86:0/2695031023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff230105290 0x7ff230105710 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009900 tx=0x7ff21c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:28:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.458+0000 7ff236600640 1 -- 172.21.15.86:0/2695031023 shutdown_connections 2024-04-17T07:28:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.458+0000 7ff236600640 1 --2- 172.21.15.86:0/2695031023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff230105c50 0x7ff23010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.458+0000 7ff236600640 1 --2- 172.21.15.86:0/2695031023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff230105290 0x7ff230105710 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.458+0000 7ff236600640 1 --2- 172.21.15.86:0/2695031023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2301040a0 0x7ff2301044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.458+0000 7ff236600640 1 -- 172.21.15.86:0/2695031023 >> 172.21.15.86:0/2695031023 conn(0x7ff2300ff610 msgr2=0x7ff230101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.458+0000 7ff236600640 1 -- 172.21.15.86:0/2695031023 shutdown_connections 2024-04-17T07:28:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.458+0000 7ff236600640 1 -- 172.21.15.86:0/2695031023 wait complete. 2024-04-17T07:28:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.459+0000 7ff236600640 1 Processor -- start 2024-04-17T07:28:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.460+0000 7ff236600640 1 -- start start 2024-04-17T07:28:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.460+0000 7ff236600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2301040a0 0x7ff23019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.460+0000 7ff236600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff230105290 0x7ff23019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.460+0000 7ff236600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff230105c50 0x7ff2301a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.460+0000 7ff236600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff23010fc30 con 0x7ff2301040a0 2024-04-17T07:28:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.460+0000 7ff236600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff23010fab0 con 0x7ff230105290 2024-04-17T07:28:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.460+0000 7ff236600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff23010fdb0 con 0x7ff230105c50 2024-04-17T07:28:34.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.460+0000 7ff235200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2301040a0 0x7ff23019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.461+0000 7ff235200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2301040a0 0x7ff23019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34180/0 (socket says 172.21.15.86:34180) 2024-04-17T07:28:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.461+0000 7ff235200640 1 -- 172.21.15.86:0/4135823890 learned_addr learned my addr 172.21.15.86:0/4135823890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.461+0000 7ff22fe00640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff230105290 0x7ff23019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.461+0000 7ff235c00640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff230105c50 0x7ff2301a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.461+0000 7ff235200640 1 -- 172.21.15.86:0/4135823890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff230105c50 msgr2=0x7ff2301a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.461+0000 7ff235200640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff230105c50 0x7ff2301a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.461+0000 7ff235200640 1 -- 172.21.15.86:0/4135823890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff230105290 msgr2=0x7ff23019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.461+0000 7ff235200640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff230105290 0x7ff23019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.461+0000 7ff235200640 1 -- 172.21.15.86:0/4135823890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2301a1ee0 con 0x7ff2301040a0 2024-04-17T07:28:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.461+0000 7ff235c00640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff230105c50 0x7ff2301a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.462+0000 7ff235200640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2301040a0 0x7ff23019a2c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff22400d770 tx=0x7ff22400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.462+0000 7ff22d600640 1 -- 172.21.15.86:0/4135823890 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224017070 con 0x7ff2301040a0 2024-04-17T07:28:34.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.462+0000 7ff236600640 1 -- 172.21.15.86:0/4135823890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2301a21d0 con 0x7ff2301040a0 2024-04-17T07:28:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.462+0000 7ff236600640 1 -- 172.21.15.86:0/4135823890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2301a2740 con 0x7ff2301040a0 2024-04-17T07:28:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.464+0000 7ff22d600640 1 -- 172.21.15.86:0/4135823890 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff224009c30 con 0x7ff2301040a0 2024-04-17T07:28:34.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.464+0000 7ff22d600640 1 -- 172.21.15.86:0/4135823890 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224020060 con 0x7ff2301040a0 2024-04-17T07:28:34.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.467+0000 7ff22d600640 1 -- 172.21.15.86:0/4135823890 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff224010460 con 0x7ff2301040a0 2024-04-17T07:28:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.468+0000 7ff23a863740 1 Processor -- start 2024-04-17T07:28:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.468+0000 7ff23a863740 1 -- start start 2024-04-17T07:28:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff23a863740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098681fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff23a863740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x564098687070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff23a863740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 0x564098689b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff23a863740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640985e0120 con 0x564098681b80 2024-04-17T07:28:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff23a863740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564098586340 con 0x564098682570 2024-04-17T07:28:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff23a863740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5640985dc0e0 con 0x5640986876e0 2024-04-17T07:28:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff235c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098681fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff235c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098681fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34182/0 (socket says 172.21.15.86:34182) 2024-04-17T07:28:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff235c00640 1 -- 172.21.15.86:0/461773712 learned_addr learned my addr 172.21.15.86:0/461773712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:34.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff22d600640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2180a3c30 0x7ff2180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.469+0000 7ff22fe00640 1 --2- 172.21.15.86:0/461773712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 0x564098689b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.470+0000 7ff235200640 1 --2- 172.21.15.86:0/461773712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x564098687070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.470+0000 7ff22d600640 1 -- 172.21.15.86:0/4135823890 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff2240035f0 con 0x7ff2301040a0 2024-04-17T07:28:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.470+0000 7ff235c00640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2180a3c30 0x7ff2180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.470+0000 7ff22fe00640 1 -- 172.21.15.86:0/461773712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 msgr2=0x564098687070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.470+0000 7ff22fe00640 1 --2- 172.21.15.86:0/461773712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x564098687070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.470+0000 7ff22fe00640 1 -- 172.21.15.86:0/461773712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 msgr2=0x564098681fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.470+0000 7ff22fe00640 1 --2- 172.21.15.86:0/461773712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098681fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.470+0000 7ff22fe00640 1 -- 172.21.15.86:0/461773712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640983bc7d0 con 0x5640986876e0 2024-04-17T07:28:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.471+0000 7ff22fe00640 1 --2- 172.21.15.86:0/461773712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 0x564098689b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff21c02c180 tx=0x7ff21c009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c30e7e85ccf23ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.471+0000 7ff235c00640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2180a3c30 0x7ff2180a60f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x564098682dd0 tx=0x7ff228004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.471+0000 7ff222000640 1 -- 172.21.15.86:0/461773712 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c044020 con 0x5640986876e0 2024-04-17T07:28:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.471+0000 7ff222000640 1 -- 172.21.15.86:0/461773712 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff21c02cbb0 con 0x5640986876e0 2024-04-17T07:28:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.471+0000 7ff222000640 1 -- 172.21.15.86:0/461773712 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c0046c0 con 0x5640986876e0 2024-04-17T07:28:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.472+0000 7ff23a863740 1 -- 172.21.15.86:0/461773712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 msgr2=0x564098689b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.472+0000 7ff23a863740 1 --2- 172.21.15.86:0/461773712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 0x564098689b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff21c02c180 tx=0x7ff21c009610 comp rx=0 tx=0).stop 2024-04-17T07:28:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.472+0000 7ff23a863740 1 -- 172.21.15.86:0/461773712 shutdown_connections 2024-04-17T07:28:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.472+0000 7ff23a863740 1 --2- 172.21.15.86:0/461773712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 0x564098689b70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.472+0000 7ff23a863740 1 --2- 172.21.15.86:0/461773712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x564098687070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.472+0000 7ff23a863740 1 --2- 172.21.15.86:0/461773712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098681fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.472+0000 7ff23a863740 1 -- 172.21.15.86:0/461773712 >> 172.21.15.86:0/461773712 conn(0x5640986748e0 msgr2=0x564098674ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:34.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.472+0000 7ff23a863740 1 -- 172.21.15.86:0/461773712 shutdown_connections 2024-04-17T07:28:34.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.472+0000 7ff23a863740 1 -- 172.21.15.86:0/461773712 wait complete. 2024-04-17T07:28:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.473+0000 7ff23a863740 1 -- start start 2024-04-17T07:28:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.473+0000 7ff23a863740 1 Processor -- start 2024-04-17T07:28:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff23a863740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098681b80 0x56409868c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff23a863740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x56409868c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff23a863740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 0x5640986935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff23a863740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56409868a630 con 0x5640986876e0 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff23a863740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564098599710 con 0x564098682570 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff23a863740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56409868a7b0 con 0x564098681b80 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff23a863740 10 client.0 _ll_register_callbacks cb 0x564098594af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff235c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 0x5640986935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff235200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098681b80 0x56409868c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff235c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 0x5640986935c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34198/0 (socket says 172.21.15.86:34198) 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff235c00640 1 -- 172.21.15.86:0/1939874697 learned_addr learned my addr 172.21.15.86:0/1939874697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff235200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098681b80 0x56409868c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:59260/0 (socket says 172.21.15.86:59260) 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.474+0000 7ff22fe00640 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x56409868c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.475+0000 7ff235200640 1 -- 172.21.15.86:0/1939874697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 msgr2=0x56409868c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.475+0000 7ff235200640 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x56409868c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.475+0000 7ff235200640 1 -- 172.21.15.86:0/1939874697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 msgr2=0x5640986935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.475+0000 7ff235200640 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 0x5640986935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.475+0000 7ff235200640 1 -- 172.21.15.86:0/1939874697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640982f23f0 con 0x564098681b80 2024-04-17T07:28:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.475+0000 7ff235200640 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098681b80 0x56409868c040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff224000c00 tx=0x7ff22409cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8ac7fef2757545a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.475+0000 7ff216a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:28:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.476+0000 7ff216a00640 1 -- 172.21.15.86:0/1939874697 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2240b5020 con 0x564098681b80 2024-04-17T07:28:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.476+0000 7ff216a00640 1 -- 172.21.15.86:0/1939874697 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff224098b80 con 0x564098681b80 2024-04-17T07:28:34.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.476+0000 7ff23a863740 1 -- 172.21.15.86:0/1939874697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff21c02cbb0 con 0x564098681b80 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.477+0000 7ff216a00640 1 -- 172.21.15.86:0/1939874697 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff2240d1020 con 0x564098681b80 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.477+0000 7ff23a863740 10 client.27727 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.477+0000 7ff23a863740 1 -- 172.21.15.86:0/1939874697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640982b20f0 con 0x564098681b80 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.477+0000 7ff216a00640 1 -- 172.21.15.86:0/1939874697 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff21c02cbb0 con 0x564098681b80 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.478+0000 7ff216a00640 1 -- 172.21.15.86:0/1939874697 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff2240bb120 con 0x564098681b80 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.478+0000 7ff23a863740 10 client.27727 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.478+0000 7ff23a863740 10 client.27727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.478+0000 7ff23a863740 20 client.27727 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.478+0000 7ff23a863740 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564098697e20 0x56409869a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.478+0000 7ff23a863740 4 client.27727 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.478+0000 7ff23a863740 1 -- 172.21.15.86:0/1939874697 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640985edb40 con 0x564098697e20 2024-04-17T07:28:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.479+0000 7ff22fe00640 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564098697e20 0x56409869a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.480+0000 7ff22fe00640 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564098697e20 0x56409869a2e0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43096af3b274d03f server_cookie=5dec5f6d9b2a46bd in_seq=0 out_seq=0 2024-04-17T07:28:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.480+0000 7ff216a00640 10 client.27727 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.482+0000 7ff216a00640 1 -- 172.21.15.86:0/1939874697 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640985edb40 con 0x564098697e20 2024-04-17T07:28:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.482+0000 7ff216a00640 10 client.27727 handle_command_reply: tid=0 2024-04-17T07:28:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.482+0000 7ff23a863740 1 client.27727 shutdown 2024-04-17T07:28:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff23a863740 1 -- 172.21.15.86:0/1939874697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098681b80 msgr2=0x56409868c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff23a863740 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098681b80 0x56409868c040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff224000c00 tx=0x7ff22409cc10 comp rx=0 tx=0).stop 2024-04-17T07:28:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff23a863740 1 -- 172.21.15.86:0/1939874697 shutdown_connections 2024-04-17T07:28:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff23a863740 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564098697e20 0x56409869a2e0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff216a00640 0 client.27727 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff23a863740 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 0x5640986935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff23a863740 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x56409868c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff23a863740 1 --2- 172.21.15.86:0/1939874697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098681b80 0x56409868c040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff23a863740 1 -- 172.21.15.86:0/1939874697 >> 172.21.15.86:0/1939874697 conn(0x5640986748e0 msgr2=0x564098673ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff23a863740 1 -- 172.21.15.86:0/1939874697 shutdown_connections 2024-04-17T07:28:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.483+0000 7ff23a863740 1 -- 172.21.15.86:0/1939874697 wait complete. 2024-04-17T07:28:34.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.484+0000 7ff23a863740 20 client.27727 trim_cache size 0 max 16384 2024-04-17T07:28:34.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.521+0000 7ff23a863740 1 Processor -- start 2024-04-17T07:28:34.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.521+0000 7ff23a863740 1 -- start start 2024-04-17T07:28:34.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.522+0000 7ff23a863740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098673250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.522+0000 7ff23a863740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x564098693580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.522+0000 7ff23a863740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 0x564098673790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.522+0000 7ff23a863740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff21000fbe0 con 0x564098681b80 2024-04-17T07:28:34.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.522+0000 7ff23a863740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff210011720 con 0x564098682570 2024-04-17T07:28:34.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.522+0000 7ff23a863740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2100102b0 con 0x5640986876e0 2024-04-17T07:28:34.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.522+0000 7ff235200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098673250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.522+0000 7ff235200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098673250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34210/0 (socket says 172.21.15.86:34210) 2024-04-17T07:28:34.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.522+0000 7ff235200640 1 -- 172.21.15.86:0/2004604720 learned_addr learned my addr 172.21.15.86:0/2004604720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:34.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.523+0000 7ff22fe00640 1 --2- 172.21.15.86:0/2004604720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x564098693580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.523+0000 7ff235c00640 1 --2- 172.21.15.86:0/2004604720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 0x564098673790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.523+0000 7ff22fe00640 1 -- 172.21.15.86:0/2004604720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 msgr2=0x564098673790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:34.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.523+0000 7ff22fe00640 1 --2- 172.21.15.86:0/2004604720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 0x564098673790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.523+0000 7ff22fe00640 1 -- 172.21.15.86:0/2004604720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 msgr2=0x564098673250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:34.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.523+0000 7ff22fe00640 1 --2- 172.21.15.86:0/2004604720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098673250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.523+0000 7ff22fe00640 1 -- 172.21.15.86:0/2004604720 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564098675350 con 0x564098682570 2024-04-17T07:28:34.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.523+0000 7ff235200640 1 --2- 172.21.15.86:0/2004604720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098673250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:34.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.524+0000 7ff22fe00640 1 --2- 172.21.15.86:0/2004604720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x564098693580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff21c02c6b0 tx=0x7ff21c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4edf45350cf577d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:34.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.524+0000 7ff221600640 1 -- 172.21.15.86:0/2004604720 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c051070 con 0x564098682570 2024-04-17T07:28:34.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.524+0000 7ff221600640 1 -- 172.21.15.86:0/2004604720 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff21c053350 con 0x564098682570 2024-04-17T07:28:34.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.525+0000 7ff23a863740 1 -- 172.21.15.86:0/2004604720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 msgr2=0x564098693580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.525+0000 7ff221600640 1 -- 172.21.15.86:0/2004604720 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c044040 con 0x564098682570 2024-04-17T07:28:34.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.525+0000 7ff23a863740 1 --2- 172.21.15.86:0/2004604720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x564098693580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff21c02c6b0 tx=0x7ff21c009680 comp rx=0 tx=0).stop 2024-04-17T07:28:34.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.526+0000 7ff23a863740 1 -- 172.21.15.86:0/2004604720 shutdown_connections 2024-04-17T07:28:34.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.526+0000 7ff23a863740 1 --2- 172.21.15.86:0/2004604720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640986876e0 0x564098673790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.526+0000 7ff23a863740 1 --2- 172.21.15.86:0/2004604720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098682570 0x564098693580 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.526+0000 7ff23a863740 1 --2- 172.21.15.86:0/2004604720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564098681b80 0x564098673250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.526+0000 7ff23a863740 1 -- 172.21.15.86:0/2004604720 >> 172.21.15.86:0/2004604720 conn(0x5640986748e0 msgr2=0x56409868bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:34.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.526+0000 7ff23a863740 1 -- 172.21.15.86:0/2004604720 shutdown_connections 2024-04-17T07:28:34.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.526+0000 7ff23a863740 1 -- 172.21.15.86:0/2004604720 wait complete. 2024-04-17T07:28:34.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.527+0000 7ff23a863740 1 -- start start 2024-04-17T07:28:34.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.527+0000 7ff23a863740 1 Processor -- start 2024-04-17T07:28:34.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.527+0000 7ff23a863740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098681b80 0x564098691340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.527+0000 7ff23a863740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098682570 0x564098691880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.527+0000 7ff23a863740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 0x564098692820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.527+0000 7ff23a863740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff210011050 con 0x5640986876e0 2024-04-17T07:28:34.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.527+0000 7ff23a863740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff210010980 con 0x564098681b80 2024-04-17T07:28:34.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.527+0000 7ff23a863740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff21000dbb0 con 0x564098682570 2024-04-17T07:28:34.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.528+0000 7ff23a863740 10 client.0 _ll_register_callbacks cb 0x7ff22409be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:34.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.528+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098682570 0x564098691880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.528+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098682570 0x564098691880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:59284/0 (socket says 172.21.15.86:59284) 2024-04-17T07:28:34.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.528+0000 7ff22fe00640 1 -- 172.21.15.86:0/3856134247 learned_addr learned my addr 172.21.15.86:0/3856134247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:34.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.528+0000 7ff235200640 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098681b80 0x564098691340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.528+0000 7ff235c00640 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 0x564098692820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.529+0000 7ff235200640 1 -- 172.21.15.86:0/3856134247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098682570 msgr2=0x564098691880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:34.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.529+0000 7ff235200640 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098682570 0x564098691880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.529+0000 7ff235200640 1 -- 172.21.15.86:0/3856134247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 msgr2=0x564098692820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:34.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.529+0000 7ff235200640 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 0x564098692820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.529+0000 7ff235200640 1 -- 172.21.15.86:0/3856134247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564098692e10 con 0x564098681b80 2024-04-17T07:28:34.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.529+0000 7ff235200640 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098681b80 0x564098691340 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff22409c210 tx=0x7ff224004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1a4e5a61b54edd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:34.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.529+0000 7ff216a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:34.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.530+0000 7ff216a00640 1 -- 172.21.15.86:0/3856134247 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2240b5040 con 0x564098681b80 2024-04-17T07:28:34.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.530+0000 7ff23a863740 1 -- 172.21.15.86:0/3856134247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff21c053350 con 0x564098681b80 2024-04-17T07:28:34.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.530+0000 7ff216a00640 1 -- 172.21.15.86:0/3856134247 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff224098b80 con 0x564098681b80 2024-04-17T07:28:34.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.530+0000 7ff216a00640 1 -- 172.21.15.86:0/3856134247 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff2240a5180 con 0x564098681b80 2024-04-17T07:28:34.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.530+0000 7ff23a863740 10 client.37691 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:34.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.530+0000 7ff23a863740 1 -- 172.21.15.86:0/3856134247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640986fea80 con 0x564098681b80 2024-04-17T07:28:34.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.531+0000 7ff216a00640 1 -- 172.21.15.86:0/3856134247 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff21c053350 con 0x564098681b80 2024-04-17T07:28:34.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.533+0000 7ff216a00640 1 -- 172.21.15.86:0/3856134247 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff224085aa0 con 0x564098681b80 2024-04-17T07:28:34.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.533+0000 7ff23a863740 10 client.37691 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:34.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.533+0000 7ff23a863740 10 client.37691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:34.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.533+0000 7ff23a863740 20 client.37691 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:34.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.533+0000 7ff23a863740 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640986ff3d0 0x5640987017f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.533+0000 7ff23a863740 4 client.37691 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:28:34.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.533+0000 7ff23a863740 1 -- 172.21.15.86:0/3856134247 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640985e79b0 con 0x5640986ff3d0 2024-04-17T07:28:34.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.534+0000 7ff22fe00640 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640986ff3d0 0x5640987017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.537+0000 7ff22fe00640 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640986ff3d0 0x5640987017f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3edf95b4967b34c server_cookie=6d8434464ef8e876 in_seq=0 out_seq=0 2024-04-17T07:28:34.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.537+0000 7ff216a00640 10 client.37691 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:34.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.538+0000 7ff216a00640 1 -- 172.21.15.86:0/3856134247 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5640985e79b0 con 0x5640986ff3d0 2024-04-17T07:28:34.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.538+0000 7ff216a00640 10 client.37691 handle_command_reply: tid=0 2024-04-17T07:28:34.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.539+0000 7ff23a863740 1 client.37691 shutdown 2024-04-17T07:28:34.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.539+0000 7ff23a863740 1 -- 172.21.15.86:0/3856134247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098681b80 msgr2=0x564098691340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.539+0000 7ff23a863740 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098681b80 0x564098691340 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff22409c210 tx=0x7ff224004750 comp rx=0 tx=0).stop 2024-04-17T07:28:34.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.539+0000 7ff23a863740 1 -- 172.21.15.86:0/3856134247 shutdown_connections 2024-04-17T07:28:34.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.539+0000 7ff23a863740 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5640986ff3d0 0x5640987017f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.539+0000 7ff23a863740 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640986876e0 0x564098692820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.539+0000 7ff23a863740 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564098682570 0x564098691880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.539+0000 7ff23a863740 1 --2- 172.21.15.86:0/3856134247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564098681b80 0x564098691340 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.539+0000 7ff23a863740 1 -- 172.21.15.86:0/3856134247 >> 172.21.15.86:0/3856134247 conn(0x5640986748e0 msgr2=0x56409868ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:34.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.539+0000 7ff216a00640 0 client.37691 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:34.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.540+0000 7ff23a863740 1 -- 172.21.15.86:0/3856134247 shutdown_connections 2024-04-17T07:28:34.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.540+0000 7ff23a863740 1 -- 172.21.15.86:0/3856134247 wait complete. 2024-04-17T07:28:34.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.540+0000 7ff23a863740 20 client.37691 trim_cache size 0 max 16384 2024-04-17T07:28:34.541 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:28:34.541 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:28:34.541 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:28:34.542 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:28:34.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.541+0000 7ff236600640 1 -- 172.21.15.86:0/4135823890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2180a3c30 msgr2=0x7ff2180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.541+0000 7ff236600640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2180a3c30 0x7ff2180a60f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x564098682dd0 tx=0x7ff228004ab0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.541+0000 7ff236600640 1 -- 172.21.15.86:0/4135823890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2301040a0 msgr2=0x7ff23019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.541+0000 7ff236600640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2301040a0 0x7ff23019a2c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff22400d770 tx=0x7ff22400dc30 comp rx=0 tx=0).stop 2024-04-17T07:28:34.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.541+0000 7ff236600640 1 -- 172.21.15.86:0/4135823890 shutdown_connections 2024-04-17T07:28:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.541+0000 7ff236600640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff2180a3c30 0x7ff2180a60f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.541+0000 7ff236600640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff230105c50 0x7ff2301a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.541+0000 7ff236600640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff230105290 0x7ff23019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.541+0000 7ff236600640 1 --2- 172.21.15.86:0/4135823890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2301040a0 0x7ff23019a2c0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.541+0000 7ff236600640 1 -- 172.21.15.86:0/4135823890 >> 172.21.15.86:0/4135823890 conn(0x7ff2300ff610 msgr2=0x7ff230108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.542+0000 7ff236600640 1 -- 172.21.15.86:0/4135823890 shutdown_connections 2024-04-17T07:28:34.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.542+0000 7ff236600640 1 -- 172.21.15.86:0/4135823890 wait complete. 2024-04-17T07:28:34.571 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:28:34.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:28:34.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:34 smithi086 ceph-mon[31733]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.9 MiB/s wr, 738 op/s 2024-04-17T07:28:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.885+0000 7f79e7000640 1 -- 172.21.15.86:0/3763731280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e0100650 msgr2=0x7f79e0102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.885+0000 7f79e7000640 1 --2- 172.21.15.86:0/3763731280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e0100650 0x7f79e0102a80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009ca0 tx=0x7f79d802c0f0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.886+0000 7f79e7000640 1 -- 172.21.15.86:0/3763731280 shutdown_connections 2024-04-17T07:28:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.886+0000 7f79e7000640 1 --2- 172.21.15.86:0/3763731280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0105970 0x7f79e0105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.886+0000 7f79e7000640 1 --2- 172.21.15.86:0/3763731280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e0102fc0 0x7f79e0105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.886+0000 7f79e7000640 1 --2- 172.21.15.86:0/3763731280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e0100650 0x7f79e0102a80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.886+0000 7f79e7000640 1 -- 172.21.15.86:0/3763731280 >> 172.21.15.86:0/3763731280 conn(0x7f79e00fc230 msgr2=0x7f79e00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.887+0000 7f79e7000640 1 -- 172.21.15.86:0/3763731280 shutdown_connections 2024-04-17T07:28:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.887+0000 7f79e7000640 1 -- 172.21.15.86:0/3763731280 wait complete. 2024-04-17T07:28:34.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.888+0000 7f79e7000640 1 Processor -- start 2024-04-17T07:28:34.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.888+0000 7f79e7000640 1 -- start start 2024-04-17T07:28:34.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.888+0000 7f79e7000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0100650 0x7f79e019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.888+0000 7f79e7000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e0102fc0 0x7f79e019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.889+0000 7f79e7000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e0105970 0x7f79e01a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.889+0000 7f79e7000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79e0113f30 con 0x7f79e0100650 2024-04-17T07:28:34.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.889+0000 7f79e7000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f79e0113db0 con 0x7f79e0102fc0 2024-04-17T07:28:34.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.889+0000 7f79e7000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01140b0 con 0x7f79e0105970 2024-04-17T07:28:34.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.889+0000 7f79e5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0100650 0x7f79e019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.889+0000 7f79e5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0100650 0x7f79e019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34226/0 (socket says 172.21.15.86:34226) 2024-04-17T07:28:34.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.889+0000 7f79e5c00640 1 -- 172.21.15.86:0/2556982075 learned_addr learned my addr 172.21.15.86:0/2556982075 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:34.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.889+0000 7f79e6600640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e0105970 0x7f79e01a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.890+0000 7f79e5200640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e0102fc0 0x7f79e019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.890+0000 7f79e5c00640 1 -- 172.21.15.86:0/2556982075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e0105970 msgr2=0x7f79e01a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.890+0000 7f79e5c00640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e0105970 0x7f79e01a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.890+0000 7f79e5c00640 1 -- 172.21.15.86:0/2556982075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e0102fc0 msgr2=0x7f79e019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:34.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.890+0000 7f79e5c00640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e0102fc0 0x7f79e019eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:34.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.890+0000 7f79e5c00640 1 -- 172.21.15.86:0/2556982075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e01a6210 con 0x7f79e0100650 2024-04-17T07:28:34.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.891+0000 7f79e5c00640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0100650 0x7f79e019e5f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f79d8000f80 tx=0x7f79d80328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:34.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.892+0000 7f79de000640 1 -- 172.21.15.86:0/2556982075 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8041020 con 0x7f79e0100650 2024-04-17T07:28:34.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.892+0000 7f79de000640 1 -- 172.21.15.86:0/2556982075 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79d8034720 con 0x7f79e0100650 2024-04-17T07:28:34.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.892+0000 7f79de000640 1 -- 172.21.15.86:0/2556982075 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8039040 con 0x7f79e0100650 2024-04-17T07:28:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.893+0000 7f79e7000640 1 -- 172.21.15.86:0/2556982075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e01a64a0 con 0x7f79e0100650 2024-04-17T07:28:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.893+0000 7f79e7000640 1 -- 172.21.15.86:0/2556982075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e01a6940 con 0x7f79e0100650 2024-04-17T07:28:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.895+0000 7f79e7000640 1 -- 172.21.15.86:0/2556982075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e00fdd20 con 0x7f79e0100650 2024-04-17T07:28:34.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.898+0000 7f79de000640 1 -- 172.21.15.86:0/2556982075 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f79d8044020 con 0x7f79e0100650 2024-04-17T07:28:34.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.900+0000 7f79de000640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f79c40a3c30 0x7f79c40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:34.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.901+0000 7f79de000640 1 -- 172.21.15.86:0/2556982075 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f79d80f2a10 con 0x7f79e0100650 2024-04-17T07:28:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.902+0000 7f79e5200640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f79c40a3c30 0x7f79c40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:34.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.905+0000 7f79e5200640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f79c40a3c30 0x7f79c40a60f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f79e019fb00 tx=0x7f79cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:34.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:34.909+0000 7f79de000640 1 -- 172.21.15.86:0/2556982075 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f79d804b020 con 0x7f79e0100650 2024-04-17T07:28:35.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:34 smithi098 ceph-mon[35418]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.9 MiB/s wr, 738 op/s 2024-04-17T07:28:35.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:34 smithi103 ceph-mon[35495]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.9 MiB/s wr, 738 op/s 2024-04-17T07:28:35.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.411+0000 7f79e7000640 1 -- 172.21.15.86:0/2556982075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79e0062b60 con 0x7f79e0100650 2024-04-17T07:28:35.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.415+0000 7f79de000640 1 -- 172.21.15.86:0/2556982075 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f79d80bf010 con 0x7f79e0100650 2024-04-17T07:28:35.415 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:28:35.416 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:28:35.416 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:28:35.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.420+0000 7f79e7000640 1 -- 172.21.15.86:0/2556982075 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f79c40a3c30 msgr2=0x7f79c40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.420+0000 7f79e7000640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f79c40a3c30 0x7f79c40a60f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f79e019fb00 tx=0x7f79cc005f90 comp rx=0 tx=0).stop 2024-04-17T07:28:35.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.420+0000 7f79e7000640 1 -- 172.21.15.86:0/2556982075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0100650 msgr2=0x7f79e019e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.420+0000 7f79e7000640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0100650 0x7f79e019e5f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f79d8000f80 tx=0x7f79d80328a0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.421+0000 7f79e7000640 1 -- 172.21.15.86:0/2556982075 shutdown_connections 2024-04-17T07:28:35.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.421+0000 7f79e7000640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f79c40a3c30 0x7f79c40a60f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.421+0000 7f79e7000640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e0105970 0x7f79e01a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.421+0000 7f79e7000640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e0102fc0 0x7f79e019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.421+0000 7f79e7000640 1 --2- 172.21.15.86:0/2556982075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0100650 0x7f79e019e5f0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.421+0000 7f79e7000640 1 -- 172.21.15.86:0/2556982075 >> 172.21.15.86:0/2556982075 conn(0x7f79e00fc230 msgr2=0x7f79e0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:35.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.422+0000 7f79e7000640 1 -- 172.21.15.86:0/2556982075 shutdown_connections 2024-04-17T07:28:35.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.422+0000 7f79e7000640 1 -- 172.21.15.86:0/2556982075 wait complete. 2024-04-17T07:28:35.462 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:28:35.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.799+0000 7fbbb9a00640 1 -- 172.21.15.86:0/373523100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb4076920 msgr2=0x7fbbb4076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.799+0000 7fbbb9a00640 1 --2- 172.21.15.86:0/373523100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb4076920 0x7fbbb4076d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009be0 tx=0x7fbbac02bef0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.800+0000 7fbbb9a00640 1 -- 172.21.15.86:0/373523100 shutdown_connections 2024-04-17T07:28:35.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.801+0000 7fbbb9a00640 1 --2- 172.21.15.86:0/373523100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb4078500 0x7fbbb4078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.801+0000 7fbbb9a00640 1 --2- 172.21.15.86:0/373523100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4077b60 0x7fbbb4077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.801+0000 7fbbb9a00640 1 --2- 172.21.15.86:0/373523100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb4076920 0x7fbbb4076d40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.801+0000 7fbbb9a00640 1 -- 172.21.15.86:0/373523100 >> 172.21.15.86:0/373523100 conn(0x7fbbb40ff750 msgr2=0x7fbbb4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:35.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.801+0000 7fbbb9a00640 1 -- 172.21.15.86:0/373523100 shutdown_connections 2024-04-17T07:28:35.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.802+0000 7fbbb9a00640 1 -- 172.21.15.86:0/373523100 wait complete. 2024-04-17T07:28:35.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.803+0000 7fbbb9a00640 1 Processor -- start 2024-04-17T07:28:35.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.804+0000 7fbbb9a00640 1 -- start start 2024-04-17T07:28:35.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.804+0000 7fbbb9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4076920 0x7fbbb410ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.804+0000 7fbbb9a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb4077b60 0x7fbbb410f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.804+0000 7fbbb9a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb4078500 0x7fbbb41ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.804+0000 7fbbb9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4114140 con 0x7fbbb4076920 2024-04-17T07:28:35.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.804+0000 7fbbb9a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4113fc0 con 0x7fbbb4078500 2024-04-17T07:28:35.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.804+0000 7fbbb9a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41142c0 con 0x7fbbb4077b60 2024-04-17T07:28:35.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.805+0000 7fbbb2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb4077b60 0x7fbbb410f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.805+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4076920 0x7fbbb410ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.805+0000 7fbbb2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb4077b60 0x7fbbb410f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:59322/0 (socket says 172.21.15.86:59322) 2024-04-17T07:28:35.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.805+0000 7fbbb2000640 1 -- 172.21.15.86:0/3842182582 learned_addr learned my addr 172.21.15.86:0/3842182582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:35.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.805+0000 7fbbb3400640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb4078500 0x7fbbb41ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.806+0000 7fbbb2000640 1 -- 172.21.15.86:0/3842182582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb4078500 msgr2=0x7fbbb41ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.806+0000 7fbbb2000640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb4078500 0x7fbbb41ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.806+0000 7fbbb2000640 1 -- 172.21.15.86:0/3842182582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4076920 msgr2=0x7fbbb410ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.806+0000 7fbbb2000640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4076920 0x7fbbb410ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.806+0000 7fbbb2000640 1 -- 172.21.15.86:0/3842182582 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb41aed50 con 0x7fbbb4077b60 2024-04-17T07:28:35.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.806+0000 7fbbb3400640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb4078500 0x7fbbb41ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:35.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.806+0000 7fbbb2000640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb4077b60 0x7fbbb410f1c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbba400e9c0 tx=0x7fbba400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:35.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.807+0000 7fbbab400640 1 -- 172.21.15.86:0/3842182582 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba400ccc0 con 0x7fbbb4077b60 2024-04-17T07:28:35.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.807+0000 7fbbab400640 1 -- 172.21.15.86:0/3842182582 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba40054d0 con 0x7fbbb4077b60 2024-04-17T07:28:35.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.807+0000 7fbbab400640 1 -- 172.21.15.86:0/3842182582 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4010430 con 0x7fbbb4077b60 2024-04-17T07:28:35.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.807+0000 7fbbb9a00640 1 -- 172.21.15.86:0/3842182582 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb41aef80 con 0x7fbbb4077b60 2024-04-17T07:28:35.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.807+0000 7fbbb9a00640 1 -- 172.21.15.86:0/3842182582 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb41af3f0 con 0x7fbbb4077b60 2024-04-17T07:28:35.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.812+0000 7fbbbc656740 1 Processor -- start 2024-04-17T07:28:35.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.813+0000 7fbbab400640 1 -- 172.21.15.86:0/3842182582 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fbba4004f50 con 0x7fbbb4077b60 2024-04-17T07:28:35.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.813+0000 7fbbbc656740 1 -- start start 2024-04-17T07:28:35.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.813+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 0x55614895c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.813+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 0x556148961490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.813+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 0x556148963f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.813+0000 7fbbbc656740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561488c2020 con 0x55614895bfa0 2024-04-17T07:28:35.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.813+0000 7fbbbc656740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556148868460 con 0x55614895c990 2024-04-17T07:28:35.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.814+0000 7fbbbc656740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5561488bdfe0 con 0x556148961b00 2024-04-17T07:28:35.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.814+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 0x55614895c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.814+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 0x55614895c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34262/0 (socket says 172.21.15.86:34262) 2024-04-17T07:28:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.814+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 0x556148961490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.814+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 0x556148961490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:44938/0 (socket says 172.21.15.86:44938) 2024-04-17T07:28:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.814+0000 7fbbb2a00640 1 -- 172.21.15.86:0/1439422479 learned_addr learned my addr 172.21.15.86:0/1439422479 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.815+0000 7fbbab400640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb980a3c30 0x7fbb980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.815+0000 7fbbab400640 1 -- 172.21.15.86:0/3842182582 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbba40d0ae0 con 0x7fbbb4077b60 2024-04-17T07:28:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.816+0000 7fbbb3400640 1 -- 172.21.15.86:0/1439422479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 msgr2=0x556148963f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:35.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.816+0000 7fbbb3400640 1 --2- 172.21.15.86:0/1439422479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 0x556148963f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.816+0000 7fbbb3400640 1 -- 172.21.15.86:0/1439422479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 msgr2=0x556148961490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.816+0000 7fbbb3400640 1 --2- 172.21.15.86:0/1439422479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 0x556148961490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.816+0000 7fbbb3400640 1 -- 172.21.15.86:0/1439422479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556148870040 con 0x55614895bfa0 2024-04-17T07:28:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.816+0000 7fbbb2a00640 1 --2- 172.21.15.86:0/1439422479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 0x556148961490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.816+0000 7fbbb3400640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb980a3c30 0x7fbb980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.817+0000 7fbbb3400640 1 --2- 172.21.15.86:0/1439422479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 0x55614895c3c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009cf0 tx=0x7fbb9c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32ae3a93ce85078 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.817+0000 7fbba3e00640 1 -- 172.21.15.86:0/1439422479 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0146d0 con 0x55614895bfa0 2024-04-17T07:28:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.818+0000 7fbba3e00640 1 -- 172.21.15.86:0/1439422479 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb9c014da0 con 0x55614895bfa0 2024-04-17T07:28:35.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.818+0000 7fbba3e00640 1 -- 172.21.15.86:0/1439422479 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0146d0 con 0x55614895bfa0 2024-04-17T07:28:35.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.819+0000 7fbbb3400640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb980a3c30 0x7fbb980a60f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c020000 tx=0x7fbb9c00fee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:35.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.820+0000 7fbbbc656740 1 -- 172.21.15.86:0/1439422479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 msgr2=0x55614895c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.820+0000 7fbbbc656740 1 --2- 172.21.15.86:0/1439422479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 0x55614895c3c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009cf0 tx=0x7fbb9c0117b0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.821+0000 7fbbbc656740 1 -- 172.21.15.86:0/1439422479 shutdown_connections 2024-04-17T07:28:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.821+0000 7fbbbc656740 1 --2- 172.21.15.86:0/1439422479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 0x556148963f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.821+0000 7fbbbc656740 1 --2- 172.21.15.86:0/1439422479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 0x556148961490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.821+0000 7fbbbc656740 1 --2- 172.21.15.86:0/1439422479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 0x55614895c3c0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.821+0000 7fbbbc656740 1 -- 172.21.15.86:0/1439422479 >> 172.21.15.86:0/1439422479 conn(0x556148957510 msgr2=0x556148957910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:35.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.821+0000 7fbbbc656740 1 -- 172.21.15.86:0/1439422479 shutdown_connections 2024-04-17T07:28:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.821+0000 7fbbbc656740 1 -- 172.21.15.86:0/1439422479 wait complete. 2024-04-17T07:28:35.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.822+0000 7fbbbc656740 1 -- start start 2024-04-17T07:28:35.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.822+0000 7fbbbc656740 1 Processor -- start 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.823+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895bfa0 0x5561489772e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.823+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895c990 0x556148977820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.823+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556148961b00 0x55614897e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.823+0000 7fbbbc656740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556148975ac0 con 0x55614895c990 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.823+0000 7fbbbc656740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55614887b830 con 0x556148961b00 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.823+0000 7fbbbc656740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556148975c40 con 0x55614895bfa0 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.823+0000 7fbbbc656740 10 client.0 _ll_register_callbacks cb 0x556148849fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.823+0000 7fbbb2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895c990 0x556148977820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.824+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895bfa0 0x5561489772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.824+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556148961b00 0x55614897e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.824+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895bfa0 0x5561489772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:59346/0 (socket says 172.21.15.86:59346) 2024-04-17T07:28:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.824+0000 7fbbb2a00640 1 -- 172.21.15.86:0/995391382 learned_addr learned my addr 172.21.15.86:0/995391382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.825+0000 7fbbb3400640 1 -- 172.21.15.86:0/995391382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895bfa0 msgr2=0x5561489772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.825+0000 7fbbb3400640 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895bfa0 0x5561489772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.825+0000 7fbbb3400640 1 -- 172.21.15.86:0/995391382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895c990 msgr2=0x556148977820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.825+0000 7fbbb3400640 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895c990 0x556148977820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.825+0000 7fbbb3400640 1 -- 172.21.15.86:0/995391382 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614887f920 con 0x556148961b00 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.825+0000 7fbbb3400640 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556148961b00 0x55614897e860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009cf0 tx=0x7fbb9c018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6af989229d8e106 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.826+0000 7fbba0c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.826+0000 7fbba0c00640 1 -- 172.21.15.86:0/995391382 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c00f630 con 0x556148961b00 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.826+0000 7fbba0c00640 1 -- 172.21.15.86:0/995391382 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbb9c0146d0 con 0x556148961b00 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.826+0000 7fbbbc656740 1 -- 172.21.15.86:0/995391382 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9c014da0 con 0x556148961b00 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.827+0000 7fbba0c00640 1 -- 172.21.15.86:0/995391382 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb9c0148d0 con 0x556148961b00 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.827+0000 7fbbbc656740 10 client.37706 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:35.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.827+0000 7fbbbc656740 1 -- 172.21.15.86:0/995391382 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55614884d7f0 con 0x556148961b00 2024-04-17T07:28:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.828+0000 7fbba0c00640 1 -- 172.21.15.86:0/995391382 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb9c014da0 con 0x556148961b00 2024-04-17T07:28:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.828+0000 7fbba0c00640 1 -- 172.21.15.86:0/995391382 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fbb9c014a70 con 0x556148961b00 2024-04-17T07:28:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.828+0000 7fbbbc656740 10 client.37706 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.828+0000 7fbbbc656740 10 client.37706 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:28:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.829+0000 7fbbbc656740 20 client.37706 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.829+0000 7fbbbc656740 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x556148982a80 0x556148984e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.829+0000 7fbbbc656740 4 client.37706 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.829+0000 7fbbbc656740 1 -- 172.21.15.86:0/995391382 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561488cfa40 con 0x556148982a80 2024-04-17T07:28:35.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.829+0000 7fbbb2a00640 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x556148982a80 0x556148984e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:35 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2556982075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:28:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.830+0000 7fbbb2a00640 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x556148982a80 0x556148984e70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9bd8b82772e3dd52 server_cookie=3396469ccf937899 in_seq=0 out_seq=0 2024-04-17T07:28:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.830+0000 7fbba0c00640 10 client.37706 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:28:35.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.831+0000 7fbba0c00640 1 -- 172.21.15.86:0/995391382 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561488cfa40 con 0x556148982a80 2024-04-17T07:28:35.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.831+0000 7fbba0c00640 10 client.37706 handle_command_reply: tid=0 2024-04-17T07:28:35.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.831+0000 7fbbbc656740 1 client.37706 shutdown 2024-04-17T07:28:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.832+0000 7fbbbc656740 1 -- 172.21.15.86:0/995391382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556148961b00 msgr2=0x55614897e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.832+0000 7fbbbc656740 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556148961b00 0x55614897e860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009cf0 tx=0x7fbb9c018000 comp rx=0 tx=0).stop 2024-04-17T07:28:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.832+0000 7fbbbc656740 1 -- 172.21.15.86:0/995391382 shutdown_connections 2024-04-17T07:28:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.832+0000 7fbbbc656740 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x556148982a80 0x556148984e70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.832+0000 7fbbbc656740 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556148961b00 0x55614897e860 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.832+0000 7fbbbc656740 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895c990 0x556148977820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.832+0000 7fbbbc656740 1 --2- 172.21.15.86:0/995391382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895bfa0 0x5561489772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.832+0000 7fbbbc656740 1 -- 172.21.15.86:0/995391382 >> 172.21.15.86:0/995391382 conn(0x556148957510 msgr2=0x556148956710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:35.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.832+0000 7fbba0c00640 0 client.37706 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:28:35.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.832+0000 7fbbbc656740 1 -- 172.21.15.86:0/995391382 shutdown_connections 2024-04-17T07:28:35.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.833+0000 7fbbbc656740 1 -- 172.21.15.86:0/995391382 wait complete. 2024-04-17T07:28:35.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.833+0000 7fbbbc656740 20 client.37706 trim_cache size 0 max 16384 2024-04-17T07:28:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.869+0000 7fbbbc656740 1 Processor -- start 2024-04-17T07:28:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.869+0000 7fbbbc656740 1 -- start start 2024-04-17T07:28:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.870+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895bfa0 0x55614895d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.870+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895c990 0x55614897e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.870+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556148961b00 0x55614895dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.870+0000 7fbbbc656740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b12e0 con 0x556148961b00 2024-04-17T07:28:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.870+0000 7fbbbc656740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b2e20 con 0x55614895bfa0 2024-04-17T07:28:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.870+0000 7fbbbc656740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b19b0 con 0x55614895c990 2024-04-17T07:28:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.870+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895bfa0 0x55614895d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.871+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895bfa0 0x55614895d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:44968/0 (socket says 172.21.15.86:44968) 2024-04-17T07:28:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.871+0000 7fbbb2a00640 1 -- 172.21.15.86:0/3424400746 learned_addr learned my addr 172.21.15.86:0/3424400746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:35.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.871+0000 7fbbb2000640 1 --2- 172.21.15.86:0/3424400746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895c990 0x55614897e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.871+0000 7fbbb3400640 1 --2- 172.21.15.86:0/3424400746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556148961b00 0x55614895dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.871+0000 7fbbb2a00640 1 -- 172.21.15.86:0/3424400746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895c990 msgr2=0x55614897e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.871+0000 7fbbb2a00640 1 --2- 172.21.15.86:0/3424400746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895c990 0x55614897e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.871+0000 7fbbb2a00640 1 -- 172.21.15.86:0/3424400746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556148961b00 msgr2=0x55614895dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.871+0000 7fbbb2a00640 1 --2- 172.21.15.86:0/3424400746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556148961b00 0x55614895dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.871+0000 7fbbb2a00640 1 -- 172.21.15.86:0/3424400746 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561485d43f0 con 0x55614895bfa0 2024-04-17T07:28:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.872+0000 7fbbb2a00640 1 --2- 172.21.15.86:0/3424400746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895bfa0 0x55614895d080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbbac004af0 tx=0x7fbbac009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75464067594554f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.873+0000 7fbba3400640 1 -- 172.21.15.86:0/3424400746 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac04d070 con 0x55614895bfa0 2024-04-17T07:28:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.873+0000 7fbba3400640 1 -- 172.21.15.86:0/3424400746 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbac003970 con 0x55614895bfa0 2024-04-17T07:28:35.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.873+0000 7fbba3400640 1 -- 172.21.15.86:0/3424400746 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac02cb80 con 0x55614895bfa0 2024-04-17T07:28:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.875+0000 7fbbbc656740 1 -- 172.21.15.86:0/3424400746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895bfa0 msgr2=0x55614895d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.875+0000 7fbbbc656740 1 --2- 172.21.15.86:0/3424400746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895bfa0 0x55614895d080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbbac004af0 tx=0x7fbbac009840 comp rx=0 tx=0).stop 2024-04-17T07:28:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.875+0000 7fbbbc656740 1 -- 172.21.15.86:0/3424400746 shutdown_connections 2024-04-17T07:28:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.875+0000 7fbbbc656740 1 --2- 172.21.15.86:0/3424400746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556148961b00 0x55614895dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.875+0000 7fbbbc656740 1 --2- 172.21.15.86:0/3424400746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55614895c990 0x55614897e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.875+0000 7fbbbc656740 1 --2- 172.21.15.86:0/3424400746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895bfa0 0x55614895d080 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.875+0000 7fbbbc656740 1 -- 172.21.15.86:0/3424400746 >> 172.21.15.86:0/3424400746 conn(0x556148957510 msgr2=0x5561489772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.875+0000 7fbbbc656740 1 -- 172.21.15.86:0/3424400746 shutdown_connections 2024-04-17T07:28:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.875+0000 7fbbbc656740 1 -- 172.21.15.86:0/3424400746 wait complete. 2024-04-17T07:28:35.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.876+0000 7fbbbc656740 1 -- start start 2024-04-17T07:28:35.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.876+0000 7fbbbc656740 1 Processor -- start 2024-04-17T07:28:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.878+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 0x55614897b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.878+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 0x55614897bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.878+0000 7fbbbc656740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 0x5561489e6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.878+0000 7fbbbc656740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b2750 con 0x55614895bfa0 2024-04-17T07:28:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.878+0000 7fbbbc656740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b2080 con 0x55614895c990 2024-04-17T07:28:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.878+0000 7fbbbc656740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4073d50 con 0x556148961b00 2024-04-17T07:28:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.878+0000 7fbbbc656740 10 client.0 _ll_register_callbacks cb 0x7fbb9c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.878+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 0x5561489e6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.878+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 0x5561489e6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:59374/0 (socket says 172.21.15.86:59374) 2024-04-17T07:28:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.878+0000 7fbbb3400640 1 -- 172.21.15.86:0/3590941214 learned_addr learned my addr 172.21.15.86:0/3590941214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.879+0000 7fbbb2000640 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 0x55614897bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.879+0000 7fbbb2a00640 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 0x55614897b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.879+0000 7fbbb3400640 1 -- 172.21.15.86:0/3590941214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 msgr2=0x55614897bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.879+0000 7fbbb3400640 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 0x55614897bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.879+0000 7fbbb3400640 1 -- 172.21.15.86:0/3590941214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 msgr2=0x55614897b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.879+0000 7fbbb3400640 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 0x55614897b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.879+0000 7fbbb3400640 1 -- 172.21.15.86:0/3590941214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614869e7d0 con 0x556148961b00 2024-04-17T07:28:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.880+0000 7fbbb3400640 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 0x5561489e6f50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0117f0 tx=0x7fbb9c00fe70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81269ed81874c275 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:35.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.880+0000 7fbba0c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:28:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.880+0000 7fbba0c00640 1 -- 172.21.15.86:0/3590941214 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c047060 con 0x556148961b00 2024-04-17T07:28:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.880+0000 7fbbbc656740 1 -- 172.21.15.86:0/3590941214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbac003970 con 0x556148961b00 2024-04-17T07:28:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.881+0000 7fbba0c00640 1 -- 172.21.15.86:0/3590941214 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbb9c0129f0 con 0x556148961b00 2024-04-17T07:28:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.881+0000 7fbba0c00640 1 -- 172.21.15.86:0/3590941214 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb9c04e050 con 0x556148961b00 2024-04-17T07:28:35.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.881+0000 7fbbbc656740 10 client.27754 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.881+0000 7fbbbc656740 1 -- 172.21.15.86:0/3590941214 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5561489e8430 con 0x556148961b00 2024-04-17T07:28:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.882+0000 7fbba0c00640 1 -- 172.21.15.86:0/3590941214 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbac003970 con 0x556148961b00 2024-04-17T07:28:35.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.882+0000 7fbba0c00640 1 -- 172.21.15.86:0/3590941214 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fbb9c00f630 con 0x556148961b00 2024-04-17T07:28:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.883+0000 7fbbbc656740 10 client.27754 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.883+0000 7fbbbc656740 10 client.27754 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:28:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.883+0000 7fbbbc656740 20 client.27754 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.883+0000 7fbbbc656740 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5561489e8d60 0x5561489eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.883+0000 7fbbbc656740 4 client.27754 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:28:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.883+0000 7fbbbc656740 1 -- 172.21.15.86:0/3590941214 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561488c98b0 con 0x5561489e8d60 2024-04-17T07:28:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.883+0000 7fbbb2a00640 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5561489e8d60 0x5561489eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.884+0000 7fbbb2a00640 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5561489e8d60 0x5561489eb180 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25e63ca445961d94 server_cookie=13c0bf6657a91d0b in_seq=0 out_seq=0 2024-04-17T07:28:35.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.884+0000 7fbba0c00640 10 client.27754 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:28:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.885+0000 7fbba0c00640 1 -- 172.21.15.86:0/3590941214 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561488c98b0 con 0x5561489e8d60 2024-04-17T07:28:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.885+0000 7fbba0c00640 10 client.27754 handle_command_reply: tid=0 2024-04-17T07:28:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.885+0000 7fbbbc656740 1 client.27754 shutdown 2024-04-17T07:28:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.885+0000 7fbbbc656740 1 -- 172.21.15.86:0/3590941214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 msgr2=0x5561489e6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.885+0000 7fbbbc656740 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 0x5561489e6f50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0117f0 tx=0x7fbb9c00fe70 comp rx=0 tx=0).stop 2024-04-17T07:28:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.886+0000 7fbbbc656740 1 -- 172.21.15.86:0/3590941214 shutdown_connections 2024-04-17T07:28:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.886+0000 7fbbbc656740 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5561489e8d60 0x5561489eb180 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.886+0000 7fbbbc656740 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556148961b00 0x5561489e6f50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.886+0000 7fbbbc656740 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55614895c990 0x55614897bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.886+0000 7fbbbc656740 1 --2- 172.21.15.86:0/3590941214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55614895bfa0 0x55614897b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.886+0000 7fbbbc656740 1 -- 172.21.15.86:0/3590941214 >> 172.21.15.86:0/3590941214 conn(0x556148957510 msgr2=0x55614895dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.886+0000 7fbba0c00640 0 client.27754 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:28:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.886+0000 7fbbbc656740 1 -- 172.21.15.86:0/3590941214 shutdown_connections 2024-04-17T07:28:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.886+0000 7fbbbc656740 1 -- 172.21.15.86:0/3590941214 wait complete. 2024-04-17T07:28:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.886+0000 7fbbbc656740 20 client.27754 trim_cache size 0 max 16384 2024-04-17T07:28:35.887 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:28:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 -- 172.21.15.86:0/3842182582 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb980a3c30 msgr2=0x7fbb980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb980a3c30 0x7fbb980a60f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c020000 tx=0x7fbb9c00fee0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 -- 172.21.15.86:0/3842182582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb4077b60 msgr2=0x7fbbb410f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb4077b60 0x7fbbb410f1c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbba400e9c0 tx=0x7fbba400ee80 comp rx=0 tx=0).stop 2024-04-17T07:28:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 -- 172.21.15.86:0/3842182582 shutdown_connections 2024-04-17T07:28:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbb980a3c30 0x7fbb980a60f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbbb4078500 0x7fbbb41ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb4077b60 0x7fbbb410f1c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 --2- 172.21.15.86:0/3842182582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4076920 0x7fbbb410ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 -- 172.21.15.86:0/3842182582 >> 172.21.15.86:0/3842182582 conn(0x7fbbb40ff750 msgr2=0x7fbbb4101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.888+0000 7fbbb9a00640 1 -- 172.21.15.86:0/3842182582 shutdown_connections 2024-04-17T07:28:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:35.889+0000 7fbbb9a00640 1 -- 172.21.15.86:0/3842182582 wait complete. 2024-04-17T07:28:35.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:28:36.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:35 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2556982075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:28:36.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:35 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2556982075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:28:36.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.294+0000 7fb982200640 1 -- 172.21.15.86:0/4230344721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb97c102fa0 msgr2=0x7fb97c105410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.294+0000 7fb982200640 1 --2- 172.21.15.86:0/4230344721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb97c102fa0 0x7fb97c105410 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb968009900 tx=0x7fb96802bc40 comp rx=0 tx=0).stop 2024-04-17T07:28:36.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.295+0000 7fb982200640 1 -- 172.21.15.86:0/4230344721 shutdown_connections 2024-04-17T07:28:36.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.295+0000 7fb982200640 1 --2- 172.21.15.86:0/4230344721 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c105950 0x7fb97c105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.295+0000 7fb982200640 1 --2- 172.21.15.86:0/4230344721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb97c102fa0 0x7fb97c105410 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.295+0000 7fb982200640 1 --2- 172.21.15.86:0/4230344721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97c100630 0x7fb97c102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.295+0000 7fb982200640 1 -- 172.21.15.86:0/4230344721 >> 172.21.15.86:0/4230344721 conn(0x7fb97c0fc270 msgr2=0x7fb97c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.296+0000 7fb982200640 1 -- 172.21.15.86:0/4230344721 shutdown_connections 2024-04-17T07:28:36.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.296+0000 7fb982200640 1 -- 172.21.15.86:0/4230344721 wait complete. 2024-04-17T07:28:36.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.298+0000 7fb982200640 1 Processor -- start 2024-04-17T07:28:36.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.299+0000 7fb982200640 1 -- start start 2024-04-17T07:28:36.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.299+0000 7fb982200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c100630 0x7fb97c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.299+0000 7fb982200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97c102fa0 0x7fb97c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.299+0000 7fb982200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb97c105950 0x7fb97c1a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.299+0000 7fb982200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c113f30 con 0x7fb97c102fa0 2024-04-17T07:28:36.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.299+0000 7fb982200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c113db0 con 0x7fb97c100630 2024-04-17T07:28:36.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.300+0000 7fb97be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97c102fa0 0x7fb97c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.300+0000 7fb97be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97c102fa0 0x7fb97c19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34310/0 (socket says 172.21.15.86:34310) 2024-04-17T07:28:36.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.300+0000 7fb97be00640 1 -- 172.21.15.86:0/1852033642 learned_addr learned my addr 172.21.15.86:0/1852033642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:36.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.300+0000 7fb980e00640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c100630 0x7fb97c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.301+0000 7fb982200640 1 -- 172.21.15.86:0/1852033642 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1140b0 con 0x7fb97c105950 2024-04-17T07:28:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.301+0000 7fb981800640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb97c105950 0x7fb97c1a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.302+0000 7fb97be00640 1 -- 172.21.15.86:0/1852033642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb97c105950 msgr2=0x7fb97c1a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.302+0000 7fb97be00640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb97c105950 0x7fb97c1a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.302+0000 7fb97be00640 1 -- 172.21.15.86:0/1852033642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c100630 msgr2=0x7fb97c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.302+0000 7fb97be00640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c100630 0x7fb97c19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.302+0000 7fb97be00640 1 -- 172.21.15.86:0/1852033642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c06bab0 con 0x7fb97c102fa0 2024-04-17T07:28:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.302+0000 7fb980e00640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c100630 0x7fb97c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:36.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.303+0000 7fb97be00640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97c102fa0 0x7fb97c19eaf0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb968000c00 tx=0x7fb968004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.306+0000 7fb979600640 1 -- 172.21.15.86:0/1852033642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968043020 con 0x7fb97c102fa0 2024-04-17T07:28:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.306+0000 7fb979600640 1 -- 172.21.15.86:0/1852033642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9680052b0 con 0x7fb97c102fa0 2024-04-17T07:28:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.306+0000 7fb979600640 1 -- 172.21.15.86:0/1852033642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb968003cc0 con 0x7fb97c102fa0 2024-04-17T07:28:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.306+0000 7fb982200640 1 -- 172.21.15.86:0/1852033642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c06bd40 con 0x7fb97c102fa0 2024-04-17T07:28:36.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.307+0000 7fb982200640 1 -- 172.21.15.86:0/1852033642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c06c180 con 0x7fb97c102fa0 2024-04-17T07:28:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.314+0000 7fb9865e1740 1 Processor -- start 2024-04-17T07:28:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.314+0000 7fb9865e1740 1 -- start start 2024-04-17T07:28:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.315+0000 7fb979600640 1 -- 172.21.15.86:0/1852033642 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb968046020 con 0x7fb97c102fa0 2024-04-17T07:28:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.318+0000 7fb979600640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb9640a3d30 0x7fb9640a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.318+0000 7fb979600640 1 -- 172.21.15.86:0/1852033642 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb9680f2b00 con 0x7fb97c102fa0 2024-04-17T07:28:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.319+0000 7fb981800640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb9640a3d30 0x7fb9640a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.319+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d8895c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.319+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d88e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.319+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 0x55952d891190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.319+0000 7fb9865e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55952d7ef020 con 0x55952d889b90 2024-04-17T07:28:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.319+0000 7fb9865e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55952d795460 con 0x55952d88ed00 2024-04-17T07:28:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.319+0000 7fb9865e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55952d7eafe0 con 0x55952d8891a0 2024-04-17T07:28:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.319+0000 7fb980e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d8895c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.319+0000 7fb980e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d8895c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:59398/0 (socket says 172.21.15.86:59398) 2024-04-17T07:28:36.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.319+0000 7fb97be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d88e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.320+0000 7fb980e00640 1 -- 172.21.15.86:0/4128376919 learned_addr learned my addr 172.21.15.86:0/4128376919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:36.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.320+0000 7fb981800640 1 --2- 172.21.15.86:0/4128376919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 0x55952d891190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.321+0000 7fb97be00640 1 -- 172.21.15.86:0/4128376919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 msgr2=0x55952d8895c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.321+0000 7fb97be00640 1 --2- 172.21.15.86:0/4128376919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d8895c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.321+0000 7fb97be00640 1 -- 172.21.15.86:0/4128376919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 msgr2=0x55952d891190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.321+0000 7fb97be00640 1 --2- 172.21.15.86:0/4128376919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 0x55952d891190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.321+0000 7fb97be00640 1 -- 172.21.15.86:0/4128376919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55952d79d040 con 0x55952d889b90 2024-04-17T07:28:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.321+0000 7fb97be00640 1 --2- 172.21.15.86:0/4128376919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d88e690 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb9680be9f0 tx=0x7fb96802bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef7377614da2ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.321+0000 7fb981800640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb9640a3d30 0x7fb9640a61f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb974005fd0 tx=0x7fb9740098b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.322+0000 7fb96e000640 1 -- 172.21.15.86:0/4128376919 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680bf160 con 0x55952d889b90 2024-04-17T07:28:36.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.322+0000 7fb96e000640 1 -- 172.21.15.86:0/4128376919 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9680bea60 con 0x55952d889b90 2024-04-17T07:28:36.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.322+0000 7fb96e000640 1 -- 172.21.15.86:0/4128376919 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680c7240 con 0x55952d889b90 2024-04-17T07:28:36.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.323+0000 7fb9865e1740 1 -- 172.21.15.86:0/4128376919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 msgr2=0x55952d88e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.323+0000 7fb9865e1740 1 --2- 172.21.15.86:0/4128376919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d88e690 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb9680be9f0 tx=0x7fb96802bc40 comp rx=0 tx=0).stop 2024-04-17T07:28:36.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.324+0000 7fb9865e1740 1 -- 172.21.15.86:0/4128376919 shutdown_connections 2024-04-17T07:28:36.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.324+0000 7fb9865e1740 1 --2- 172.21.15.86:0/4128376919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 0x55952d891190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.324+0000 7fb9865e1740 1 --2- 172.21.15.86:0/4128376919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d88e690 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.324+0000 7fb9865e1740 1 --2- 172.21.15.86:0/4128376919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d8895c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.324+0000 7fb9865e1740 1 -- 172.21.15.86:0/4128376919 >> 172.21.15.86:0/4128376919 conn(0x55952d884710 msgr2=0x55952d884b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.324+0000 7fb9865e1740 1 -- 172.21.15.86:0/4128376919 shutdown_connections 2024-04-17T07:28:36.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.324+0000 7fb9865e1740 1 -- 172.21.15.86:0/4128376919 wait complete. 2024-04-17T07:28:36.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.324+0000 7fb9865e1740 1 -- start start 2024-04-17T07:28:36.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.325+0000 7fb9865e1740 1 Processor -- start 2024-04-17T07:28:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.325+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d8891a0 0x55952d8934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.325+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d8939f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.325+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d88ed00 0x55952d89a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.325+0000 7fb9865e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55952d891a70 con 0x55952d889b90 2024-04-17T07:28:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.325+0000 7fb9865e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55952d7a8830 con 0x55952d8891a0 2024-04-17T07:28:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.325+0000 7fb9865e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55952d891bf0 con 0x55952d88ed00 2024-04-17T07:28:36.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.326+0000 7fb9865e1740 10 client.0 _ll_register_callbacks cb 0x55952d776fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:36.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.326+0000 7fb980e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d8891a0 0x55952d8934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.327+0000 7fb980e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d8891a0 0x55952d8934b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45022/0 (socket says 172.21.15.86:45022) 2024-04-17T07:28:36.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.327+0000 7fb980e00640 1 -- 172.21.15.86:0/2149872306 learned_addr learned my addr 172.21.15.86:0/2149872306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:36.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.327+0000 7fb981800640 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d88ed00 0x55952d89a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.327+0000 7fb97be00640 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d8939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.328+0000 7fb980e00640 1 -- 172.21.15.86:0/2149872306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d88ed00 msgr2=0x55952d89a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.328+0000 7fb980e00640 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d88ed00 0x55952d89a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.328+0000 7fb980e00640 1 -- 172.21.15.86:0/2149872306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 msgr2=0x55952d8939f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.328+0000 7fb980e00640 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d8939f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.328+0000 7fb980e00640 1 -- 172.21.15.86:0/2149872306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55952d7ac920 con 0x55952d8891a0 2024-04-17T07:28:36.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.328+0000 7fb981800640 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d88ed00 0x55952d89a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:36.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.328+0000 7fb962a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:36.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.329+0000 7fb980e00640 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d8891a0 0x55952d8934b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb970011800 tx=0x7fb970011cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13a5da060c826d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.329+0000 7fb962a00640 1 -- 172.21.15.86:0/2149872306 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970010ba0 con 0x55952d8891a0 2024-04-17T07:28:36.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.329+0000 7fb962a00640 1 -- 172.21.15.86:0/2149872306 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb970010d40 con 0x55952d8891a0 2024-04-17T07:28:36.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.329+0000 7fb9865e1740 1 -- 172.21.15.86:0/2149872306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9680bea60 con 0x55952d8891a0 2024-04-17T07:28:36.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.330+0000 7fb962a00640 1 -- 172.21.15.86:0/2149872306 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb970025050 con 0x55952d8891a0 2024-04-17T07:28:36.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.330+0000 7fb9865e1740 10 client.37724 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:36.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.330+0000 7fb9865e1740 1 -- 172.21.15.86:0/2149872306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55952d77a7f0 con 0x55952d8891a0 2024-04-17T07:28:36.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.331+0000 7fb962a00640 1 -- 172.21.15.86:0/2149872306 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9680bea60 con 0x55952d8891a0 2024-04-17T07:28:36.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.331+0000 7fb962a00640 1 -- 172.21.15.86:0/2149872306 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb97000f4f0 con 0x55952d8891a0 2024-04-17T07:28:36.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.331+0000 7fb9865e1740 10 client.37724 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.331+0000 7fb9865e1740 10 client.37724 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:28:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.331+0000 7fb9865e1740 20 client.37724 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.331+0000 7fb9865e1740 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55952d8aff80 0x55952d8b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.331+0000 7fb9865e1740 4 client.37724 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:36.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.331+0000 7fb9865e1740 1 -- 172.21.15.86:0/2149872306 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55952d7fca40 con 0x55952d8aff80 2024-04-17T07:28:36.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.333+0000 7fb97be00640 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55952d8aff80 0x55952d8b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.334+0000 7fb97be00640 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55952d8aff80 0x55952d8b2400 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=20eb9fe596ef4de8 server_cookie=966a03333a4aad38 in_seq=0 out_seq=0 2024-04-17T07:28:36.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.335+0000 7fb962a00640 10 client.37724 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:28:36.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.337+0000 7fb962a00640 1 -- 172.21.15.86:0/2149872306 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55952d7fca40 con 0x55952d8aff80 2024-04-17T07:28:36.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.337+0000 7fb962a00640 10 client.37724 handle_command_reply: tid=0 2024-04-17T07:28:36.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.340+0000 7fb9865e1740 1 client.37724 shutdown 2024-04-17T07:28:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.341+0000 7fb9865e1740 1 -- 172.21.15.86:0/2149872306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d8891a0 msgr2=0x55952d8934b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.341+0000 7fb9865e1740 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d8891a0 0x55952d8934b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb970011800 tx=0x7fb970011cc0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.342+0000 7fb9865e1740 1 -- 172.21.15.86:0/2149872306 shutdown_connections 2024-04-17T07:28:36.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.342+0000 7fb9865e1740 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55952d8aff80 0x55952d8b2400 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.342+0000 7fb9865e1740 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d88ed00 0x55952d89a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.343+0000 7fb962a00640 0 client.37724 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:28:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.343+0000 7fb9865e1740 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d8939f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.343+0000 7fb9865e1740 1 --2- 172.21.15.86:0/2149872306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d8891a0 0x55952d8934b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.343+0000 7fb9865e1740 1 -- 172.21.15.86:0/2149872306 >> 172.21.15.86:0/2149872306 conn(0x55952d884710 msgr2=0x55952d883930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.344+0000 7fb9865e1740 1 -- 172.21.15.86:0/2149872306 shutdown_connections 2024-04-17T07:28:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.344+0000 7fb9865e1740 1 -- 172.21.15.86:0/2149872306 wait complete. 2024-04-17T07:28:36.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.344+0000 7fb9865e1740 20 client.37724 trim_cache size 0 max 16384 2024-04-17T07:28:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.368+0000 7fb9865e1740 1 Processor -- start 2024-04-17T07:28:36.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.370+0000 7fb9865e1740 1 -- start start 2024-04-17T07:28:36.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.370+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d8830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.370+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d89a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.370+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 0x55952d8835e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.371+0000 7fb9865e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c109270 con 0x55952d889b90 2024-04-17T07:28:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.371+0000 7fb981800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 0x55952d8835e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.371+0000 7fb981800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 0x55952d8835e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45036/0 (socket says 172.21.15.86:45036) 2024-04-17T07:28:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.371+0000 7fb981800640 1 -- 172.21.15.86:0/1132108401 learned_addr learned my addr 172.21.15.86:0/1132108401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.371+0000 7fb980e00640 1 --2- 172.21.15.86:0/1132108401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d8830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.374+0000 7fb97be00640 1 --2- 172.21.15.86:0/1132108401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d89a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.374+0000 7fb9865e1740 1 -- 172.21.15.86:0/1132108401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c10adb0 con 0x55952d88ed00 2024-04-17T07:28:36.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.374+0000 7fb9865e1740 1 -- 172.21.15.86:0/1132108401 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c109940 con 0x55952d8891a0 2024-04-17T07:28:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.374+0000 7fb981800640 1 -- 172.21.15.86:0/1132108401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 msgr2=0x55952d8830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.374+0000 7fb981800640 1 --2- 172.21.15.86:0/1132108401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d8830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.374+0000 7fb981800640 1 -- 172.21.15.86:0/1132108401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 msgr2=0x55952d89a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.375+0000 7fb981800640 1 --2- 172.21.15.86:0/1132108401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d89a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.375+0000 7fb981800640 1 -- 172.21.15.86:0/1132108401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55952d5013f0 con 0x55952d88ed00 2024-04-17T07:28:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.375+0000 7fb981800640 1 --2- 172.21.15.86:0/1132108401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 0x55952d8835e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb974013990 tx=0x7fb974015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c698d47b1cebb8c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.376+0000 7fb96d600640 1 -- 172.21.15.86:0/1132108401 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97402d070 con 0x55952d88ed00 2024-04-17T07:28:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.377+0000 7fb96d600640 1 -- 172.21.15.86:0/1132108401 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb97401fde0 con 0x55952d88ed00 2024-04-17T07:28:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.377+0000 7fb96d600640 1 -- 172.21.15.86:0/1132108401 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974028de0 con 0x55952d88ed00 2024-04-17T07:28:36.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.378+0000 7fb9865e1740 1 -- 172.21.15.86:0/1132108401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 msgr2=0x55952d8835e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.378+0000 7fb9865e1740 1 --2- 172.21.15.86:0/1132108401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 0x55952d8835e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb974013990 tx=0x7fb974015000 comp rx=0 tx=0).stop 2024-04-17T07:28:36.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.378+0000 7fb9865e1740 1 -- 172.21.15.86:0/1132108401 shutdown_connections 2024-04-17T07:28:36.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.378+0000 7fb9865e1740 1 --2- 172.21.15.86:0/1132108401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d88ed00 0x55952d8835e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.378+0000 7fb9865e1740 1 --2- 172.21.15.86:0/1132108401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d889b90 0x55952d89a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.378+0000 7fb9865e1740 1 --2- 172.21.15.86:0/1132108401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d8830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.378+0000 7fb9865e1740 1 -- 172.21.15.86:0/1132108401 >> 172.21.15.86:0/1132108401 conn(0x55952d884710 msgr2=0x55952d88b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.380+0000 7fb9865e1740 1 -- 172.21.15.86:0/1132108401 shutdown_connections 2024-04-17T07:28:36.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.380+0000 7fb9865e1740 1 -- 172.21.15.86:0/1132108401 wait complete. 2024-04-17T07:28:36.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.380+0000 7fb9865e1740 1 -- start start 2024-04-17T07:28:36.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.381+0000 7fb9865e1740 1 Processor -- start 2024-04-17T07:28:36.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.383+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d897cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.384+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d889b90 0x55952d898230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.384+0000 7fb9865e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d88ed00 0x55952d899240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.384+0000 7fb980e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d897cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.384+0000 7fb980e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d897cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:59426/0 (socket says 172.21.15.86:59426) 2024-04-17T07:28:36.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.384+0000 7fb97be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d889b90 0x55952d898230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.385+0000 7fb97be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d889b90 0x55952d898230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45046/0 (socket says 172.21.15.86:45046) 2024-04-17T07:28:36.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.385+0000 7fb981800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d88ed00 0x55952d899240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.385+0000 7fb981800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d88ed00 0x55952d899240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34356/0 (socket says 172.21.15.86:34356) 2024-04-17T07:28:36.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.385+0000 7fb9865e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c10a6e0 con 0x55952d88ed00 2024-04-17T07:28:36.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.385+0000 7fb9865e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c10a010 con 0x55952d889b90 2024-04-17T07:28:36.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.385+0000 7fb9865e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c073e70 con 0x55952d8891a0 2024-04-17T07:28:36.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.385+0000 7fb9865e1740 10 client.0 _ll_register_callbacks cb 0x7fb970011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:36.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.385+0000 7fb980e00640 1 -- 172.21.15.86:0/1210487033 learned_addr learned my addr 172.21.15.86:0/1210487033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:36.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.386+0000 7fb981800640 1 -- 172.21.15.86:0/1210487033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 msgr2=0x55952d897cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.386+0000 7fb981800640 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d897cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.386+0000 7fb981800640 1 -- 172.21.15.86:0/1210487033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d889b90 msgr2=0x55952d898230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.386+0000 7fb981800640 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d889b90 0x55952d898230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.386+0000 7fb981800640 1 -- 172.21.15.86:0/1210487033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55952d5cb7d0 con 0x55952d88ed00 2024-04-17T07:28:36.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.386+0000 7fb980e00640 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d897cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:36.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.387+0000 7fb981800640 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d88ed00 0x55952d899240 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb97402b040 tx=0x7fb97401fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d7151141f7d6ca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.387+0000 7fb962a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:28:36.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.387+0000 7fb962a00640 1 -- 172.21.15.86:0/1210487033 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97402d070 con 0x55952d88ed00 2024-04-17T07:28:36.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.387+0000 7fb962a00640 1 -- 172.21.15.86:0/1210487033 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb974028ae0 con 0x55952d88ed00 2024-04-17T07:28:36.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.388+0000 7fb9865e1740 1 -- 172.21.15.86:0/1210487033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb97401fde0 con 0x55952d88ed00 2024-04-17T07:28:36.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.388+0000 7fb962a00640 1 -- 172.21.15.86:0/1210487033 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb97402d070 con 0x55952d88ed00 2024-04-17T07:28:36.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.388+0000 7fb9865e1740 10 client.18825 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:36.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.388+0000 7fb9865e1740 1 -- 172.21.15.86:0/1210487033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55952d89a960 con 0x55952d88ed00 2024-04-17T07:28:36.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.389+0000 7fb962a00640 1 -- 172.21.15.86:0/1210487033 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb974028ce0 con 0x55952d88ed00 2024-04-17T07:28:36.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.389+0000 7fb962a00640 1 -- 172.21.15.86:0/1210487033 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb974027070 con 0x55952d88ed00 2024-04-17T07:28:36.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.389+0000 7fb9865e1740 10 client.18825 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:36.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.389+0000 7fb9865e1740 10 client.18825 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:28:36.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.390+0000 7fb9865e1740 20 client.18825 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:36.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.390+0000 7fb9865e1740 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55952d916830 0x55952d918c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.390+0000 7fb9865e1740 4 client.18825 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:28:36.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.390+0000 7fb9865e1740 1 -- 172.21.15.86:0/1210487033 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb97401fde0 con 0x55952d916830 2024-04-17T07:28:36.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.390+0000 7fb980e00640 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55952d916830 0x55952d918c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.391+0000 7fb980e00640 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55952d916830 0x55952d918c20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4520bcebbc25b46 server_cookie=57fe06b46cddae88 in_seq=0 out_seq=0 2024-04-17T07:28:36.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.392+0000 7fb962a00640 10 client.18825 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:28:36.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.392+0000 7fb962a00640 1 -- 172.21.15.86:0/1210487033 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb97401fde0 con 0x55952d916830 2024-04-17T07:28:36.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.392+0000 7fb962a00640 10 client.18825 handle_command_reply: tid=0 2024-04-17T07:28:36.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.392+0000 7fb9865e1740 1 client.18825 shutdown 2024-04-17T07:28:36.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.393+0000 7fb9865e1740 1 -- 172.21.15.86:0/1210487033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d88ed00 msgr2=0x55952d899240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.393+0000 7fb9865e1740 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d88ed00 0x55952d899240 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb97402b040 tx=0x7fb97401fd60 comp rx=0 tx=0).stop 2024-04-17T07:28:36.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.393+0000 7fb9865e1740 1 -- 172.21.15.86:0/1210487033 shutdown_connections 2024-04-17T07:28:36.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.393+0000 7fb9865e1740 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55952d916830 0x55952d918c20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.393+0000 7fb9865e1740 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55952d88ed00 0x55952d899240 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.393+0000 7fb9865e1740 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55952d889b90 0x55952d898230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.393+0000 7fb9865e1740 1 --2- 172.21.15.86:0/1210487033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55952d8891a0 0x55952d897cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.394+0000 7fb9865e1740 1 -- 172.21.15.86:0/1210487033 >> 172.21.15.86:0/1210487033 conn(0x55952d884710 msgr2=0x55952d887970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.394+0000 7fb962a00640 0 client.18825 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:28:36.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.394+0000 7fb9865e1740 1 -- 172.21.15.86:0/1210487033 shutdown_connections 2024-04-17T07:28:36.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.395+0000 7fb9865e1740 1 -- 172.21.15.86:0/1210487033 wait complete. 2024-04-17T07:28:36.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.395+0000 7fb9865e1740 20 client.18825 trim_cache size 0 max 16384 2024-04-17T07:28:36.397 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:28:36.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.398+0000 7fb982200640 1 -- 172.21.15.86:0/1852033642 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb9640a3d30 msgr2=0x7fb9640a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.398+0000 7fb982200640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb9640a3d30 0x7fb9640a61f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb974005fd0 tx=0x7fb9740098b0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.398+0000 7fb982200640 1 -- 172.21.15.86:0/1852033642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97c102fa0 msgr2=0x7fb97c19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.398+0000 7fb982200640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97c102fa0 0x7fb97c19eaf0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb968000c00 tx=0x7fb968004270 comp rx=0 tx=0).stop 2024-04-17T07:28:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.401+0000 7fb982200640 1 -- 172.21.15.86:0/1852033642 shutdown_connections 2024-04-17T07:28:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.402+0000 7fb982200640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb9640a3d30 0x7fb9640a61f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.402+0000 7fb982200640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb97c105950 0x7fb97c1a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.402+0000 7fb982200640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb97c102fa0 0x7fb97c19eaf0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.402+0000 7fb982200640 1 --2- 172.21.15.86:0/1852033642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c100630 0x7fb97c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.402+0000 7fb982200640 1 -- 172.21.15.86:0/1852033642 >> 172.21.15.86:0/1852033642 conn(0x7fb97c0fc270 msgr2=0x7fb97c100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.404+0000 7fb982200640 1 -- 172.21.15.86:0/1852033642 shutdown_connections 2024-04-17T07:28:36.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.404+0000 7fb982200640 1 -- 172.21.15.86:0/1852033642 wait complete. 2024-04-17T07:28:36.435 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:28:36.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.813+0000 7f8f81600640 1 -- 172.21.15.86:0/1945083361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f7c078500 msgr2=0x7f8f7c078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.813+0000 7f8f81600640 1 --2- 172.21.15.86:0/1945083361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f7c078500 0x7f8f7c078da0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009ca0 tx=0x7f8f7402c100 comp rx=0 tx=0).stop 2024-04-17T07:28:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.813+0000 7f8f81600640 1 -- 172.21.15.86:0/1945083361 shutdown_connections 2024-04-17T07:28:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.813+0000 7f8f81600640 1 --2- 172.21.15.86:0/1945083361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f7c078500 0x7f8f7c078da0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.813+0000 7f8f81600640 1 --2- 172.21.15.86:0/1945083361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f7c077b60 0x7f8f7c077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.813+0000 7f8f81600640 1 --2- 172.21.15.86:0/1945083361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f7c076920 0x7f8f7c076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.813+0000 7f8f81600640 1 -- 172.21.15.86:0/1945083361 >> 172.21.15.86:0/1945083361 conn(0x7f8f7c0ff750 msgr2=0x7f8f7c101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.813+0000 7f8f81600640 1 -- 172.21.15.86:0/1945083361 shutdown_connections 2024-04-17T07:28:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.813+0000 7f8f81600640 1 -- 172.21.15.86:0/1945083361 wait complete. 2024-04-17T07:28:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.815+0000 7f8f81600640 1 Processor -- start 2024-04-17T07:28:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.815+0000 7f8f81600640 1 -- start start 2024-04-17T07:28:36.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.816+0000 7f8f81600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f7c076920 0x7f8f7c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.816+0000 7f8f81600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f7c077b60 0x7f8f7c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.816+0000 7f8f81600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f7c078500 0x7f8f7c1a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.816+0000 7f8f81600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c114160 con 0x7f8f7c076920 2024-04-17T07:28:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.817+0000 7f8f81600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c113fe0 con 0x7f8f7c078500 2024-04-17T07:28:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.817+0000 7f8f81600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1142e0 con 0x7f8f7c077b60 2024-04-17T07:28:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.817+0000 7f8f7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f7c076920 0x7f8f7c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.817+0000 7f8f7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f7c076920 0x7f8f7c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49768/0 (socket says 172.21.15.86:49768) 2024-04-17T07:28:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.817+0000 7f8f7aa00640 1 -- 172.21.15.86:0/227909470 learned_addr learned my addr 172.21.15.86:0/227909470 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:36.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.817+0000 7f8f7b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f7c078500 0x7f8f7c1a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.817+0000 7f8f7a000640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f7c077b60 0x7f8f7c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.818+0000 7f8f7aa00640 1 -- 172.21.15.86:0/227909470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f7c077b60 msgr2=0x7f8f7c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.818+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f7c077b60 0x7f8f7c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.818+0000 7f8f7aa00640 1 -- 172.21.15.86:0/227909470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f7c078500 msgr2=0x7f8f7c1a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.818+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f7c078500 0x7f8f7c1a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.818+0000 7f8f7aa00640 1 -- 172.21.15.86:0/227909470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c1a6350 con 0x7f8f7c076920 2024-04-17T07:28:36.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.820+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f7c076920 0x7f8f7c19e780 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00d950 tx=0x7f8f6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.822+0000 7f8f73400640 1 -- 172.21.15.86:0/227909470 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c017030 con 0x7f8f7c076920 2024-04-17T07:28:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.822+0000 7f8f81600640 1 -- 172.21.15.86:0/227909470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c1a6640 con 0x7f8f7c076920 2024-04-17T07:28:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.823+0000 7f8f81600640 1 -- 172.21.15.86:0/227909470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c1a6be0 con 0x7f8f7c076920 2024-04-17T07:28:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.823+0000 7f8f73400640 1 -- 172.21.15.86:0/227909470 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f6c004020 con 0x7f8f7c076920 2024-04-17T07:28:36.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.823+0000 7f8f73400640 1 -- 172.21.15.86:0/227909470 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c017030 con 0x7f8f7c076920 2024-04-17T07:28:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.828+0000 7f8f843e1740 1 Processor -- start 2024-04-17T07:28:36.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.828+0000 7f8f73400640 1 -- 172.21.15.86:0/227909470 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8f6c005440 con 0x7f8f7c076920 2024-04-17T07:28:36.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.829+0000 7f8f843e1740 1 -- start start 2024-04-17T07:28:36.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.830+0000 7f8f73400640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f600a3ff0 0x7f8f600a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.830+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.831+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a3a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.831+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 0x564062a3cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.831+0000 7f8f843e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56406299b020 con 0x564062a3ab00 2024-04-17T07:28:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.831+0000 7f8f843e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564062941460 con 0x564062a35990 2024-04-17T07:28:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.831+0000 7f8f843e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564062996fe0 con 0x564062a34fa0 2024-04-17T07:28:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.831+0000 7f8f73400640 1 -- 172.21.15.86:0/227909470 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8f6c018070 con 0x7f8f7c076920 2024-04-17T07:28:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.831+0000 7f8f7a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a3a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.831+0000 7f8f7a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a3a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:32876/0 (socket says 172.21.15.86:32876) 2024-04-17T07:28:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.831+0000 7f8f7a000640 1 -- 172.21.15.86:0/3200155799 learned_addr learned my addr 172.21.15.86:0/3200155799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:36.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.832+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.833+0000 7f8f7b400640 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 0x564062a3cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.833+0000 7f8f7a000640 1 -- 172.21.15.86:0/3200155799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 msgr2=0x564062a353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.833+0000 7f8f7a000640 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a353c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.833+0000 7f8f7a000640 1 -- 172.21.15.86:0/3200155799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 msgr2=0x564062a3cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.833+0000 7f8f7a000640 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 0x564062a3cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.833+0000 7f8f7a000640 1 -- 172.21.15.86:0/3200155799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564062949040 con 0x564062a35990 2024-04-17T07:28:36.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.833+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.833+0000 7f8f7b400640 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 0x564062a3cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.834+0000 7f8f7b400640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f600a3ff0 0x7f8f600a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.835+0000 7f8f7b400640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f600a3ff0 0x7f8f600a64b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8f74006010 tx=0x7f8f7402c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.835+0000 7f8f7a000640 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a3a490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800bc00 tx=0x7f8f6800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2746b26487ec2ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.836+0000 7f8f67400640 1 -- 172.21.15.86:0/3200155799 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68015070 con 0x564062a35990 2024-04-17T07:28:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.836+0000 7f8f67400640 1 -- 172.21.15.86:0/3200155799 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f680054d0 con 0x564062a35990 2024-04-17T07:28:36.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.836+0000 7f8f67400640 1 -- 172.21.15.86:0/3200155799 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68011440 con 0x564062a35990 2024-04-17T07:28:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.837+0000 7f8f843e1740 1 -- 172.21.15.86:0/3200155799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 msgr2=0x564062a3a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.837+0000 7f8f843e1740 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a3a490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800bc00 tx=0x7f8f6800f730 comp rx=0 tx=0).stop 2024-04-17T07:28:36.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.839+0000 7f8f843e1740 1 -- 172.21.15.86:0/3200155799 shutdown_connections 2024-04-17T07:28:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.839+0000 7f8f843e1740 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 0x564062a3cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.839+0000 7f8f843e1740 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a3a490 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.839+0000 7f8f843e1740 1 --2- 172.21.15.86:0/3200155799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.839+0000 7f8f843e1740 1 -- 172.21.15.86:0/3200155799 >> 172.21.15.86:0/3200155799 conn(0x564062a30510 msgr2=0x564062a30910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.839+0000 7f8f843e1740 1 -- 172.21.15.86:0/3200155799 shutdown_connections 2024-04-17T07:28:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.839+0000 7f8f843e1740 1 -- 172.21.15.86:0/3200155799 wait complete. 2024-04-17T07:28:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.841+0000 7f8f843e1740 1 -- start start 2024-04-17T07:28:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.841+0000 7f8f843e1740 1 Processor -- start 2024-04-17T07:28:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.842+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a3f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.842+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 0x564062a3fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.842+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a3ab00 0x564062a46a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.842+0000 7f8f843e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564062a3dab0 con 0x564062a35990 2024-04-17T07:28:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.842+0000 7f8f843e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564062954830 con 0x564062a3ab00 2024-04-17T07:28:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.842+0000 7f8f843e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564062a3dc30 con 0x564062a34fa0 2024-04-17T07:28:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.842+0000 7f8f843e1740 10 client.0 _ll_register_callbacks cb 0x564062922fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.843+0000 7f8f7a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 0x564062a3fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.843+0000 7f8f7b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a3ab00 0x564062a46a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.843+0000 7f8f7b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a3ab00 0x564062a46a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:32888/0 (socket says 172.21.15.86:32888) 2024-04-17T07:28:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.843+0000 7f8f7b400640 1 -- 172.21.15.86:0/571317511 learned_addr learned my addr 172.21.15.86:0/571317511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.843+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a3f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.843+0000 7f8f7b400640 1 -- 172.21.15.86:0/571317511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 msgr2=0x564062a3f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.843+0000 7f8f7b400640 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a3f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.843+0000 7f8f7b400640 1 -- 172.21.15.86:0/571317511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 msgr2=0x564062a3fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.843+0000 7f8f7b400640 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 0x564062a3fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.843+0000 7f8f7b400640 1 -- 172.21.15.86:0/571317511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564062958920 con 0x564062a3ab00 2024-04-17T07:28:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.844+0000 7f8f7b400640 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a3ab00 0x564062a46a90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f74036000 tx=0x7f8f740095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=910a5e43bb3e2a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.844+0000 7f8f5be00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.844+0000 7f8f5be00640 1 -- 172.21.15.86:0/571317511 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74051020 con 0x564062a3ab00 2024-04-17T07:28:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.844+0000 7f8f843e1740 1 -- 172.21.15.86:0/571317511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f680054d0 con 0x564062a3ab00 2024-04-17T07:28:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.844+0000 7f8f5be00640 1 -- 172.21.15.86:0/571317511 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8f74004240 con 0x564062a3ab00 2024-04-17T07:28:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.845+0000 7f8f5be00640 1 -- 172.21.15.86:0/571317511 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f74049040 con 0x564062a3ab00 2024-04-17T07:28:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.846+0000 7f8f843e1740 10 client.37742 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.846+0000 7f8f843e1740 1 -- 172.21.15.86:0/571317511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640629267f0 con 0x564062a3ab00 2024-04-17T07:28:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.849+0000 7f8f5be00640 1 -- 172.21.15.86:0/571317511 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f680054d0 con 0x564062a3ab00 2024-04-17T07:28:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.849+0000 7f8f5be00640 1 -- 172.21.15.86:0/571317511 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8f7404c030 con 0x564062a3ab00 2024-04-17T07:28:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.850+0000 7f8f843e1740 10 client.37742 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.850+0000 7f8f843e1740 10 client.37742 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:28:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.851+0000 7f8f843e1740 20 client.37742 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.851+0000 7f8f843e1740 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564062a4afa0 0x564062a4d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.851+0000 7f8f843e1740 4 client.37742 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.851+0000 7f8f843e1740 1 -- 172.21.15.86:0/571317511 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640629a8a40 con 0x564062a4afa0 2024-04-17T07:28:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.851+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564062a4afa0 0x564062a4d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.852+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564062a4afa0 0x564062a4d3e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d58e96cd32662b73 server_cookie=48eb590b0299af93 in_seq=0 out_seq=0 2024-04-17T07:28:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.852+0000 7f8f5be00640 10 client.37742 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:28:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.853+0000 7f8f5be00640 1 -- 172.21.15.86:0/571317511 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640629a8a40 con 0x564062a4afa0 2024-04-17T07:28:36.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.853+0000 7f8f5be00640 10 client.37742 handle_command_reply: tid=0 2024-04-17T07:28:36.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.854+0000 7f8f843e1740 1 client.37742 shutdown 2024-04-17T07:28:36.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.854+0000 7f8f843e1740 1 -- 172.21.15.86:0/571317511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a3ab00 msgr2=0x564062a46a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.854+0000 7f8f843e1740 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a3ab00 0x564062a46a90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8f74036000 tx=0x7f8f740095c0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.854+0000 7f8f843e1740 1 -- 172.21.15.86:0/571317511 shutdown_connections 2024-04-17T07:28:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.854+0000 7f8f843e1740 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564062a4afa0 0x564062a4d3e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.854+0000 7f8f843e1740 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a3ab00 0x564062a46a90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.854+0000 7f8f843e1740 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 0x564062a3fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.854+0000 7f8f843e1740 1 --2- 172.21.15.86:0/571317511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a3f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.854+0000 7f8f843e1740 1 -- 172.21.15.86:0/571317511 >> 172.21.15.86:0/571317511 conn(0x564062a30510 msgr2=0x564062a2f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.854+0000 7f8f5be00640 0 client.37742 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:28:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.855+0000 7f8f843e1740 1 -- 172.21.15.86:0/571317511 shutdown_connections 2024-04-17T07:28:36.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.855+0000 7f8f843e1740 1 -- 172.21.15.86:0/571317511 wait complete. 2024-04-17T07:28:36.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.855+0000 7f8f843e1740 20 client.37742 trim_cache size 0 max 16384 2024-04-17T07:28:36.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.886+0000 7f8f843e1740 1 Processor -- start 2024-04-17T07:28:36.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.886+0000 7f8f843e1740 1 -- start start 2024-04-17T07:28:36.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.887+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a3eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.887+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a46a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.887+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 0x564062a2ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f7a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a46a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f7a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a46a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:32894/0 (socket says 172.21.15.86:32894) 2024-04-17T07:28:36.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f7aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a3eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f843e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c00fb50 con 0x564062a3ab00 2024-04-17T07:28:36.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f843e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c011690 con 0x564062a35990 2024-04-17T07:28:36.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f843e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c010220 con 0x564062a34fa0 2024-04-17T07:28:36.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f7aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a3eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38690/0 (socket says 172.21.15.86:38690) 2024-04-17T07:28:36.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f7aa00640 1 -- 172.21.15.86:0/3143075969 learned_addr learned my addr 172.21.15.86:0/3143075969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:36.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f7aa00640 1 -- 172.21.15.86:0/3143075969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 msgr2=0x564062a46a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/3143075969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a46a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.888+0000 7f8f7aa00640 1 -- 172.21.15.86:0/3143075969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 msgr2=0x564062a2ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.889+0000 7f8f7b400640 1 --2- 172.21.15.86:0/3143075969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 0x564062a2ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.889+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/3143075969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 0x564062a2ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.889+0000 7f8f7aa00640 1 -- 172.21.15.86:0/3143075969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640626ad3f0 con 0x564062a34fa0 2024-04-17T07:28:36.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.889+0000 7f8f7b400640 1 --2- 172.21.15.86:0/3143075969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 0x564062a2ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:36.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.889+0000 7f8f7a000640 1 --2- 172.21.15.86:0/3143075969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a46a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:36.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.889+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/3143075969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a3eef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c097fb0 tx=0x7f8f6c0cf040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8186b621e0261a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.890+0000 7f8f66a00640 1 -- 172.21.15.86:0/3143075969 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c017040 con 0x564062a34fa0 2024-04-17T07:28:36.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.890+0000 7f8f66a00640 1 -- 172.21.15.86:0/3143075969 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f6c014040 con 0x564062a34fa0 2024-04-17T07:28:36.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.891+0000 7f8f843e1740 1 -- 172.21.15.86:0/3143075969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 msgr2=0x564062a3eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.891+0000 7f8f66a00640 1 -- 172.21.15.86:0/3143075969 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c017040 con 0x564062a34fa0 2024-04-17T07:28:36.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.892+0000 7f8f843e1740 1 --2- 172.21.15.86:0/3143075969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a3eef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c097fb0 tx=0x7f8f6c0cf040 comp rx=0 tx=0).stop 2024-04-17T07:28:36.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.892+0000 7f8f843e1740 1 -- 172.21.15.86:0/3143075969 shutdown_connections 2024-04-17T07:28:36.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.892+0000 7f8f843e1740 1 --2- 172.21.15.86:0/3143075969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a3ab00 0x564062a2ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.892+0000 7f8f843e1740 1 --2- 172.21.15.86:0/3143075969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a35990 0x564062a46a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.892+0000 7f8f843e1740 1 --2- 172.21.15.86:0/3143075969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a34fa0 0x564062a3eef0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.892+0000 7f8f843e1740 1 -- 172.21.15.86:0/3143075969 >> 172.21.15.86:0/3143075969 conn(0x564062a30510 msgr2=0x564062a3e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.893+0000 7f8f843e1740 1 -- 172.21.15.86:0/3143075969 shutdown_connections 2024-04-17T07:28:36.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.893+0000 7f8f843e1740 1 -- 172.21.15.86:0/3143075969 wait complete. 2024-04-17T07:28:36.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.894+0000 7f8f843e1740 1 -- start start 2024-04-17T07:28:36.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.894+0000 7f8f843e1740 1 Processor -- start 2024-04-17T07:28:36.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.895+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a34fa0 0x564062a44570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.895+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 0x564062a44ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f7aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a34fa0 0x564062a44570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f843e1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a3ab00 0x564062a45a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f843e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c010fc0 con 0x564062a35990 2024-04-17T07:28:36.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f843e1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c0108f0 con 0x564062a34fa0 2024-04-17T07:28:36.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f843e1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c00db20 con 0x564062a3ab00 2024-04-17T07:28:36.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f843e1740 10 client.0 _ll_register_callbacks cb 0x7f8f74032970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:36.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f7aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a34fa0 0x564062a44570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:32904/0 (socket says 172.21.15.86:32904) 2024-04-17T07:28:36.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f7aa00640 1 -- 172.21.15.86:0/1944828710 learned_addr learned my addr 172.21.15.86:0/1944828710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:36.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f7a000640 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 0x564062a44ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f7aa00640 1 -- 172.21.15.86:0/1944828710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a3ab00 msgr2=0x564062a45a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a3ab00 0x564062a45a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.896+0000 7f8f7aa00640 1 -- 172.21.15.86:0/1944828710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 msgr2=0x564062a44ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:36.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.897+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 0x564062a44ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.897+0000 7f8f7aa00640 1 -- 172.21.15.86:0/1944828710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640627777d0 con 0x564062a34fa0 2024-04-17T07:28:36.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.897+0000 7f8f7a000640 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 0x564062a44ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:36.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.897+0000 7f8f7aa00640 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a34fa0 0x564062a44570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0a4290 tx=0x7f8f6c004400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1594367414b57151 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:36.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.897+0000 7f8f5be00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:36.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.898+0000 7f8f5be00640 1 -- 172.21.15.86:0/1944828710 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c017070 con 0x564062a34fa0 2024-04-17T07:28:36.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.898+0000 7f8f5be00640 1 -- 172.21.15.86:0/1944828710 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8f6c09afa0 con 0x564062a34fa0 2024-04-17T07:28:36.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.898+0000 7f8f843e1740 1 -- 172.21.15.86:0/1944828710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f6c014040 con 0x564062a34fa0 2024-04-17T07:28:36.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.899+0000 7f8f5be00640 1 -- 172.21.15.86:0/1944828710 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f6c066050 con 0x564062a34fa0 2024-04-17T07:28:36.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.900+0000 7f8f843e1740 10 client.37748 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:36.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.900+0000 7f8f843e1740 1 -- 172.21.15.86:0/1944828710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564062ab1750 con 0x564062a34fa0 2024-04-17T07:28:36.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.901+0000 7f8f5be00640 1 -- 172.21.15.86:0/1944828710 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f6c014040 con 0x564062a34fa0 2024-04-17T07:28:36.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.901+0000 7f8f5be00640 1 -- 172.21.15.86:0/1944828710 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8f6c097a10 con 0x564062a34fa0 2024-04-17T07:28:36.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.901+0000 7f8f843e1740 10 client.37748 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:36.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.901+0000 7f8f843e1740 10 client.37748 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:28:36.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.901+0000 7f8f843e1740 20 client.37748 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:36.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.901+0000 7f8f843e1740 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564062ab20c0 0x564062ab44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:36.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.901+0000 7f8f843e1740 4 client.37748 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:28:36.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.901+0000 7f8f843e1740 1 -- 172.21.15.86:0/1944828710 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640629a28b0 con 0x564062ab20c0 2024-04-17T07:28:36.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.901+0000 7f8f7a000640 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564062ab20c0 0x564062ab44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:36.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.905+0000 7f8f7a000640 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564062ab20c0 0x564062ab44e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fcc38d95221764b server_cookie=2eb75dbed5fa896e in_seq=0 out_seq=0 2024-04-17T07:28:36.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.905+0000 7f8f5be00640 10 client.37748 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:28:36.909 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:28:36.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.906+0000 7f8f5be00640 1 -- 172.21.15.86:0/1944828710 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640629a28b0 con 0x564062ab20c0 2024-04-17T07:28:36.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.906+0000 7f8f5be00640 10 client.37748 handle_command_reply: tid=0 2024-04-17T07:28:36.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.906+0000 7f8f843e1740 1 client.37748 shutdown 2024-04-17T07:28:36.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.906+0000 7f8f843e1740 1 -- 172.21.15.86:0/1944828710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a34fa0 msgr2=0x564062a44570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.906+0000 7f8f843e1740 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a34fa0 0x564062a44570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0a4290 tx=0x7f8f6c004400 comp rx=0 tx=0).stop 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.906+0000 7f8f843e1740 1 -- 172.21.15.86:0/1944828710 shutdown_connections 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.907+0000 7f8f843e1740 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564062ab20c0 0x564062ab44e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.907+0000 7f8f843e1740 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564062a3ab00 0x564062a45a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.907+0000 7f8f843e1740 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564062a35990 0x564062a44ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.907+0000 7f8f5be00640 0 client.37748 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.907+0000 7f8f843e1740 1 --2- 172.21.15.86:0/1944828710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564062a34fa0 0x564062a44570 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.907+0000 7f8f843e1740 1 -- 172.21.15.86:0/1944828710 >> 172.21.15.86:0/1944828710 conn(0x564062a30510 msgr2=0x564062a2ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.907+0000 7f8f843e1740 1 -- 172.21.15.86:0/1944828710 shutdown_connections 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.907+0000 7f8f843e1740 1 -- 172.21.15.86:0/1944828710 wait complete. 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.907+0000 7f8f843e1740 20 client.37748 trim_cache size 0 max 16384 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.909+0000 7f8f81600640 1 -- 172.21.15.86:0/227909470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f600a3ff0 msgr2=0x7f8f600a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.909+0000 7f8f81600640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f600a3ff0 0x7f8f600a64b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8f74006010 tx=0x7f8f7402c100 comp rx=0 tx=0).stop 2024-04-17T07:28:36.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.909+0000 7f8f81600640 1 -- 172.21.15.86:0/227909470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f7c076920 msgr2=0x7f8f7c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:36.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.909+0000 7f8f81600640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f7c076920 0x7f8f7c19e780 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00d950 tx=0x7f8f6c00de10 comp rx=0 tx=0).stop 2024-04-17T07:28:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.909+0000 7f8f81600640 1 -- 172.21.15.86:0/227909470 shutdown_connections 2024-04-17T07:28:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.909+0000 7f8f81600640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f600a3ff0 0x7f8f600a64b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.909+0000 7f8f81600640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f7c078500 0x7f8f7c1a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.909+0000 7f8f81600640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f7c077b60 0x7f8f7c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.909+0000 7f8f81600640 1 --2- 172.21.15.86:0/227909470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f7c076920 0x7f8f7c19e780 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.909+0000 7f8f81600640 1 -- 172.21.15.86:0/227909470 >> 172.21.15.86:0/227909470 conn(0x7f8f7c0ff750 msgr2=0x7f8f7c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.910+0000 7f8f81600640 1 -- 172.21.15.86:0/227909470 shutdown_connections 2024-04-17T07:28:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:36.911+0000 7f8f81600640 1 -- 172.21.15.86:0/227909470 wait complete. 2024-04-17T07:28:36.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:28:37.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:36 smithi098 ceph-mon[35418]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.8 MiB/s wr, 596 op/s 2024-04-17T07:28:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:36 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:28:37.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:36 smithi103 ceph-mon[35495]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.8 MiB/s wr, 596 op/s 2024-04-17T07:28:37.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:36 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:28:37.069 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:36 smithi086 ceph-mon[31733]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.8 MiB/s wr, 596 op/s 2024-04-17T07:28:37.069 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:36 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:28:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.351+0000 7f8562c00640 1 -- 172.21.15.86:0/3223381788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f855c10bb90 msgr2=0x7f855c10dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.352+0000 7f8562c00640 1 --2- 172.21.15.86:0/3223381788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f855c10bb90 0x7f855c10dfc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8548009900 tx=0x7f854802bc80 comp rx=0 tx=0).stop 2024-04-17T07:28:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.355+0000 7f8562c00640 1 -- 172.21.15.86:0/3223381788 shutdown_connections 2024-04-17T07:28:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.355+0000 7f8562c00640 1 --2- 172.21.15.86:0/3223381788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f855c10e650 0x7f855c110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.355+0000 7f8562c00640 1 --2- 172.21.15.86:0/3223381788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f855c10bb90 0x7f855c10dfc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.355+0000 7f8562c00640 1 --2- 172.21.15.86:0/3223381788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f855c1091d0 0x7f855c10b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.355+0000 7f8562c00640 1 -- 172.21.15.86:0/3223381788 >> 172.21.15.86:0/3223381788 conn(0x7f855c06f190 msgr2=0x7f855c0715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.355+0000 7f8562c00640 1 -- 172.21.15.86:0/3223381788 shutdown_connections 2024-04-17T07:28:37.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.355+0000 7f8562c00640 1 -- 172.21.15.86:0/3223381788 wait complete. 2024-04-17T07:28:37.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.357+0000 7f8562c00640 1 Processor -- start 2024-04-17T07:28:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.357+0000 7f8562c00640 1 -- start start 2024-04-17T07:28:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.358+0000 7f8562c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f855c1091d0 0x7f855c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.358+0000 7f8562c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f855c10bb90 0x7f855c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.358+0000 7f8562c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f855c10e650 0x7f855c1a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.358+0000 7f8562c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f855c1140f0 con 0x7f855c1091d0 2024-04-17T07:28:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.358+0000 7f8562c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f855c113f70 con 0x7f855c10bb90 2024-04-17T07:28:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.358+0000 7f8562c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f855c114270 con 0x7f855c10e650 2024-04-17T07:28:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.359+0000 7f855b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f855c10bb90 0x7f855c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.359+0000 7f855b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f855c10bb90 0x7f855c19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:32922/0 (socket says 172.21.15.86:32922) 2024-04-17T07:28:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.359+0000 7f855b400640 1 -- 172.21.15.86:0/884931684 learned_addr learned my addr 172.21.15.86:0/884931684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.359+0000 7f855be00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f855c1091d0 0x7f855c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.359+0000 7f8560c00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f855c10e650 0x7f855c1a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.359+0000 7f855b400640 1 -- 172.21.15.86:0/884931684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f855c10e650 msgr2=0x7f855c1a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.360+0000 7f855b400640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f855c10e650 0x7f855c1a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.360+0000 7f855b400640 1 -- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f855c1091d0 msgr2=0x7f855c19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.360+0000 7f855b400640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f855c1091d0 0x7f855c19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.360+0000 7f855b400640 1 -- 172.21.15.86:0/884931684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c1a63f0 con 0x7f855c10bb90 2024-04-17T07:28:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.360+0000 7f855be00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f855c1091d0 0x7f855c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.360+0000 7f855b400640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f855c10bb90 0x7f855c19ed10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8548000c00 tx=0x7f854802eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.361+0000 7f8558c00640 1 -- 172.21.15.86:0/884931684 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548041020 con 0x7f855c10bb90 2024-04-17T07:28:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.361+0000 7f8558c00640 1 -- 172.21.15.86:0/884931684 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f854802c620 con 0x7f855c10bb90 2024-04-17T07:28:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.361+0000 7f8558c00640 1 -- 172.21.15.86:0/884931684 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548004f40 con 0x7f855c10bb90 2024-04-17T07:28:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.361+0000 7f8562c00640 1 -- 172.21.15.86:0/884931684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855c1a6680 con 0x7f855c10bb90 2024-04-17T07:28:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.361+0000 7f8562c00640 1 -- 172.21.15.86:0/884931684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855c1a6a60 con 0x7f855c10bb90 2024-04-17T07:28:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.368+0000 7f8558c00640 1 -- 172.21.15.86:0/884931684 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8548044020 con 0x7f855c10bb90 2024-04-17T07:28:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.369+0000 7f85659a3740 1 Processor -- start 2024-04-17T07:28:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.369+0000 7f85659a3740 1 -- start start 2024-04-17T07:28:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.369+0000 7f85659a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40f4fa0 0x5610f40f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.369+0000 7f85659a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f40f5990 0x5610f40fa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.369+0000 7f85659a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40fab00 0x5610f40fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.369+0000 7f85659a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5610f405b020 con 0x5610f40f4fa0 2024-04-17T07:28:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.369+0000 7f85659a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5610f4001460 con 0x5610f40f5990 2024-04-17T07:28:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.369+0000 7f85659a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5610f4056fe0 con 0x5610f40fab00 2024-04-17T07:28:37.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.370+0000 7f855be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f40f5990 0x5610f40fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.370+0000 7f855b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40fab00 0x5610f40fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.370+0000 7f855be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f40f5990 0x5610f40fa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:32926/0 (socket says 172.21.15.86:32926) 2024-04-17T07:28:37.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.370+0000 7f855be00640 1 -- 172.21.15.86:0/2717594984 learned_addr learned my addr 172.21.15.86:0/2717594984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.370+0000 7f8560c00640 1 --2- 172.21.15.86:0/2717594984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40f4fa0 0x5610f40f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.371+0000 7f855be00640 1 -- 172.21.15.86:0/2717594984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40fab00 msgr2=0x5610f40fcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.371+0000 7f855be00640 1 --2- 172.21.15.86:0/2717594984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40fab00 0x5610f40fcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.371+0000 7f855be00640 1 -- 172.21.15.86:0/2717594984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40f4fa0 msgr2=0x5610f40f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.371+0000 7f855be00640 1 --2- 172.21.15.86:0/2717594984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40f4fa0 0x5610f40f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.371+0000 7f8558c00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85400a3c10 0x7f85400a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.372+0000 7f855be00640 1 -- 172.21.15.86:0/2717594984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f4009040 con 0x5610f40f5990 2024-04-17T07:28:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.372+0000 7f8560c00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85400a3c10 0x7f85400a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.374+0000 7f8558c00640 1 -- 172.21.15.86:0/884931684 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f85480f1ef0 con 0x7f855c10bb90 2024-04-17T07:28:37.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.374+0000 7f855be00640 1 --2- 172.21.15.86:0/2717594984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f40f5990 0x5610f40fa490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f854c00c550 tx=0x7f854c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a48063a34027b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.374+0000 7f8560c00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85400a3c10 0x7f85400a60d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x5610f40f61f0 tx=0x7f8554007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.376+0000 7f8551200640 1 -- 172.21.15.86:0/2717594984 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c025070 con 0x5610f40f5990 2024-04-17T07:28:37.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.376+0000 7f8551200640 1 -- 172.21.15.86:0/2717594984 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f854c004230 con 0x5610f40f5990 2024-04-17T07:28:37.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.376+0000 7f8551200640 1 -- 172.21.15.86:0/2717594984 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c00b920 con 0x5610f40f5990 2024-04-17T07:28:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.377+0000 7f85659a3740 1 -- 172.21.15.86:0/2717594984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f40f5990 msgr2=0x5610f40fa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.377+0000 7f85659a3740 1 --2- 172.21.15.86:0/2717594984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f40f5990 0x5610f40fa490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f854c00c550 tx=0x7f854c005f90 comp rx=0 tx=0).stop 2024-04-17T07:28:37.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.379+0000 7f85659a3740 1 -- 172.21.15.86:0/2717594984 shutdown_connections 2024-04-17T07:28:37.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.379+0000 7f85659a3740 1 --2- 172.21.15.86:0/2717594984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40fab00 0x5610f40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.379+0000 7f85659a3740 1 --2- 172.21.15.86:0/2717594984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f40f5990 0x5610f40fa490 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f854c00c550 tx=0x7f854c005f90 comp rx=0 tx=0).stop 2024-04-17T07:28:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.379+0000 7f85659a3740 1 --2- 172.21.15.86:0/2717594984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40f4fa0 0x5610f40f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.379+0000 7f85659a3740 1 -- 172.21.15.86:0/2717594984 >> 172.21.15.86:0/2717594984 conn(0x5610f40f0510 msgr2=0x5610f40f0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.381+0000 7f85659a3740 1 -- 172.21.15.86:0/2717594984 shutdown_connections 2024-04-17T07:28:37.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.381+0000 7f85659a3740 1 -- 172.21.15.86:0/2717594984 wait complete. 2024-04-17T07:28:37.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.382+0000 7f85659a3740 1 -- start start 2024-04-17T07:28:37.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.382+0000 7f85659a3740 1 Processor -- start 2024-04-17T07:28:37.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.385+0000 7f85659a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f40ff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.386+0000 7f85659a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.386+0000 7f85659a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f4107030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.387+0000 7f8560c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f4107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.387+0000 7f8560c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f4107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:32934/0 (socket says 172.21.15.86:32934) 2024-04-17T07:28:37.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.387+0000 7f8560c00640 1 -- 172.21.15.86:0/4181782536 learned_addr learned my addr 172.21.15.86:0/4181782536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:37.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.387+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5610f40fdab0 con 0x5610f40fab00 2024-04-17T07:28:37.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.387+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5610f4014830 con 0x5610f4104c40 2024-04-17T07:28:37.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.387+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5610f40fdc30 con 0x5610f40f4fa0 2024-04-17T07:28:37.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.387+0000 7f85659a3740 10 client.0 _ll_register_callbacks cb 0x5610f3fe2fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:37.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.388+0000 7f855be00640 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f40ff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.389+0000 7f855b400640 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.389+0000 7f8560c00640 1 -- 172.21.15.86:0/4181782536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 msgr2=0x5610f40ff6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.389+0000 7f8560c00640 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f40ff6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.389+0000 7f8560c00640 1 -- 172.21.15.86:0/4181782536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 msgr2=0x5610f40ffc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.389+0000 7f8560c00640 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40ffc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.389+0000 7f8560c00640 1 -- 172.21.15.86:0/4181782536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610f4018920 con 0x5610f4104c40 2024-04-17T07:28:37.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.389+0000 7f855b400640 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:37.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.390+0000 7f8560c00640 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f4107030 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f85540045c0 tx=0x7f8554004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0bec84eb7a8d4ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.390+0000 7f8545600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:37.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.390+0000 7f8545600640 1 -- 172.21.15.86:0/4181782536 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855400fdc0 con 0x5610f4104c40 2024-04-17T07:28:37.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.391+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f854c004230 con 0x5610f4104c40 2024-04-17T07:28:37.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.391+0000 7f8545600640 1 -- 172.21.15.86:0/4181782536 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f855401e420 con 0x5610f4104c40 2024-04-17T07:28:37.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.393+0000 7f8545600640 1 -- 172.21.15.86:0/4181782536 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f855401e620 con 0x5610f4104c40 2024-04-17T07:28:37.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.393+0000 7f85659a3740 10 client.37757 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:37.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.393+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610f3fe67f0 con 0x5610f4104c40 2024-04-17T07:28:37.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.394+0000 7f8545600640 1 -- 172.21.15.86:0/4181782536 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f854c004230 con 0x5610f4104c40 2024-04-17T07:28:37.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.394+0000 7f8545600640 1 -- 172.21.15.86:0/4181782536 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8554030970 con 0x5610f4104c40 2024-04-17T07:28:37.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.395+0000 7f85659a3740 10 client.37757 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:37.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.395+0000 7f85659a3740 10 client.37757 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:28:37.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.395+0000 7f85659a3740 20 client.37757 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:37.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.395+0000 7f85659a3740 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610f410b5e0 0x5610f410da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.395+0000 7f85659a3740 4 client.37757 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:37.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.395+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610f4068a40 con 0x5610f410b5e0 2024-04-17T07:28:37.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.396+0000 7f855be00640 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610f410b5e0 0x5610f410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.397+0000 7f855be00640 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610f410b5e0 0x5610f410da40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbcf68c36c9320d9 server_cookie=ab76aeae4bb4455d in_seq=0 out_seq=0 2024-04-17T07:28:37.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.398+0000 7f8545600640 10 client.37757 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:28:37.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.400+0000 7f8545600640 1 -- 172.21.15.86:0/4181782536 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610f4068a40 con 0x5610f410b5e0 2024-04-17T07:28:37.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.400+0000 7f8545600640 10 client.37757 handle_command_reply: tid=0 2024-04-17T07:28:37.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.400+0000 7f85659a3740 1 client.37757 shutdown 2024-04-17T07:28:37.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 msgr2=0x5610f4107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f85659a3740 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f4107030 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f85540045c0 tx=0x7f8554004a40 comp rx=0 tx=0).stop 2024-04-17T07:28:37.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 shutdown_connections 2024-04-17T07:28:37.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f85659a3740 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610f410b5e0 0x5610f410da40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f85659a3740 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f4107030 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f85659a3740 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f85659a3740 1 --2- 172.21.15.86:0/4181782536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f40ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 >> 172.21.15.86:0/4181782536 conn(0x5610f40f0510 msgr2=0x5610f40ef710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f8545600640 0 client.37757 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:28:37.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 shutdown_connections 2024-04-17T07:28:37.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.401+0000 7f85659a3740 1 -- 172.21.15.86:0/4181782536 wait complete. 2024-04-17T07:28:37.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.402+0000 7f85659a3740 20 client.37757 trim_cache size 0 max 16384 2024-04-17T07:28:37.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.436+0000 7f85659a3740 1 Processor -- start 2024-04-17T07:28:37.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.436+0000 7f85659a3740 1 -- start start 2024-04-17T07:28:37.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.437+0000 7f85659a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f40fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.437+0000 7f85659a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40fef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.437+0000 7f85659a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f40eee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.437+0000 7f85659a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f853c00fb50 con 0x5610f40fab00 2024-04-17T07:28:37.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.437+0000 7f85659a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f853c011690 con 0x5610f4104c40 2024-04-17T07:28:37.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.437+0000 7f85659a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f853c010220 con 0x5610f40f4fa0 2024-04-17T07:28:37.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.437+0000 7f855b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.438+0000 7f855b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40fef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49868/0 (socket says 172.21.15.86:49868) 2024-04-17T07:28:37.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.438+0000 7f855be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f40fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.438+0000 7f855be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f40fe9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38734/0 (socket says 172.21.15.86:38734) 2024-04-17T07:28:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.438+0000 7f855be00640 1 -- 172.21.15.86:0/326813932 learned_addr learned my addr 172.21.15.86:0/326813932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.438+0000 7f8560c00640 1 --2- 172.21.15.86:0/326813932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f40eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.439+0000 7f855b400640 1 -- 172.21.15.86:0/326813932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 msgr2=0x5610f40fe9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.439+0000 7f855b400640 1 --2- 172.21.15.86:0/326813932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f40fe9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.439+0000 7f855b400640 1 -- 172.21.15.86:0/326813932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 msgr2=0x5610f40eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.439+0000 7f855b400640 1 --2- 172.21.15.86:0/326813932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f40eee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.439+0000 7f855b400640 1 -- 172.21.15.86:0/326813932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f3d6d3f0 con 0x5610f40fab00 2024-04-17T07:28:37.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.439+0000 7f855b400640 1 --2- 172.21.15.86:0/326813932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40fef00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f85480ef6c0 tx=0x7f85480f55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c3b3d9ef9df6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.440+0000 7f8547e00640 1 -- 172.21.15.86:0/326813932 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480ba0f0 con 0x5610f40fab00 2024-04-17T07:28:37.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.440+0000 7f8547e00640 1 -- 172.21.15.86:0/326813932 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85480c0020 con 0x5610f40fab00 2024-04-17T07:28:37.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.440+0000 7f8547e00640 1 -- 172.21.15.86:0/326813932 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480bd7b0 con 0x5610f40fab00 2024-04-17T07:28:37.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.442+0000 7f85659a3740 1 -- 172.21.15.86:0/326813932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 msgr2=0x5610f40fef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.442+0000 7f85659a3740 1 --2- 172.21.15.86:0/326813932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40fef00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f85480ef6c0 tx=0x7f85480f55c0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.443+0000 7f85659a3740 1 -- 172.21.15.86:0/326813932 shutdown_connections 2024-04-17T07:28:37.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.443+0000 7f85659a3740 1 --2- 172.21.15.86:0/326813932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f40eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.443+0000 7f85659a3740 1 --2- 172.21.15.86:0/326813932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f40fef00 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.443+0000 7f85659a3740 1 --2- 172.21.15.86:0/326813932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f40fe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.443+0000 7f85659a3740 1 -- 172.21.15.86:0/326813932 >> 172.21.15.86:0/326813932 conn(0x5610f40f0510 msgr2=0x5610f40f7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.444+0000 7f85659a3740 1 -- 172.21.15.86:0/326813932 shutdown_connections 2024-04-17T07:28:37.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.444+0000 7f85659a3740 1 -- 172.21.15.86:0/326813932 wait complete. 2024-04-17T07:28:37.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.445+0000 7f85659a3740 1 -- start start 2024-04-17T07:28:37.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.446+0000 7f85659a3740 1 Processor -- start 2024-04-17T07:28:37.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.446+0000 7f85659a3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f411c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.446+0000 7f85659a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f411cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.446+0000 7f85659a3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f411deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.446+0000 7f85659a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f853c010fc0 con 0x5610f40fab00 2024-04-17T07:28:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.446+0000 7f85659a3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f853c0108f0 con 0x5610f4104c40 2024-04-17T07:28:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.446+0000 7f85659a3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f853c00db20 con 0x5610f40f4fa0 2024-04-17T07:28:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.447+0000 7f85659a3740 10 client.0 _ll_register_callbacks cb 0x7f8554004270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.447+0000 7f855b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f411cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.447+0000 7f8560c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f411deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.447+0000 7f8560c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f411deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:32952/0 (socket says 172.21.15.86:32952) 2024-04-17T07:28:37.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.447+0000 7f8560c00640 1 -- 172.21.15.86:0/161119548 learned_addr learned my addr 172.21.15.86:0/161119548 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:37.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.447+0000 7f855be00640 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f411c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.448+0000 7f8560c00640 1 -- 172.21.15.86:0/161119548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 msgr2=0x5610f411c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.449+0000 7f8560c00640 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f411c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.449+0000 7f8560c00640 1 -- 172.21.15.86:0/161119548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 msgr2=0x5610f411cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.449+0000 7f8560c00640 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f411cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.449+0000 7f8560c00640 1 -- 172.21.15.86:0/161119548 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610f3e377d0 con 0x5610f4104c40 2024-04-17T07:28:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.449+0000 7f855b400640 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f411cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.450+0000 7f8560c00640 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f411deb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8554000f80 tx=0x7f8554004d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a44f44f80f3fc23 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.450+0000 7f8545600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.451+0000 7f8545600640 1 -- 172.21.15.86:0/161119548 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855401e9d0 con 0x5610f4104c40 2024-04-17T07:28:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.451+0000 7f8545600640 1 -- 172.21.15.86:0/161119548 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f85540071c0 con 0x5610f4104c40 2024-04-17T07:28:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.451+0000 7f85659a3740 1 -- 172.21.15.86:0/161119548 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85480c0020 con 0x5610f4104c40 2024-04-17T07:28:37.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.453+0000 7f8545600640 1 -- 172.21.15.86:0/161119548 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f855401eb70 con 0x5610f4104c40 2024-04-17T07:28:37.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.453+0000 7f85659a3740 10 client.37763 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:37.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.453+0000 7f85659a3740 1 -- 172.21.15.86:0/161119548 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610f4171860 con 0x5610f4104c40 2024-04-17T07:28:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.454+0000 7f8545600640 1 -- 172.21.15.86:0/161119548 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85480c0020 con 0x5610f4104c40 2024-04-17T07:28:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.454+0000 7f8545600640 1 -- 172.21.15.86:0/161119548 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8554005040 con 0x5610f4104c40 2024-04-17T07:28:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.454+0000 7f85659a3740 10 client.37763 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.454+0000 7f85659a3740 10 client.37763 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:28:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.454+0000 7f85659a3740 20 client.37763 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.454+0000 7f85659a3740 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610f4172190 0x5610f41745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.454+0000 7f85659a3740 4 client.37763 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:28:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.454+0000 7f85659a3740 1 -- 172.21.15.86:0/161119548 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610f40628b0 con 0x5610f4172190 2024-04-17T07:28:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.455+0000 7f855be00640 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610f4172190 0x5610f41745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.456+0000 7f855be00640 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610f4172190 0x5610f41745b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2208381b0b16af03 server_cookie=77c6bb1cd133b07d in_seq=0 out_seq=0 2024-04-17T07:28:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f8545600640 10 client.37763 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:28:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f8545600640 1 -- 172.21.15.86:0/161119548 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610f40628b0 con 0x5610f4172190 2024-04-17T07:28:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f8545600640 10 client.37763 handle_command_reply: tid=0 2024-04-17T07:28:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f85659a3740 1 client.37763 shutdown 2024-04-17T07:28:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f85659a3740 1 -- 172.21.15.86:0/161119548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 msgr2=0x5610f411deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f85659a3740 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f411deb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8554000f80 tx=0x7f8554004d60 comp rx=0 tx=0).stop 2024-04-17T07:28:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f85659a3740 1 -- 172.21.15.86:0/161119548 shutdown_connections 2024-04-17T07:28:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f85659a3740 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5610f4172190 0x5610f41745b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f85659a3740 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5610f4104c40 0x5610f411deb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f85659a3740 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610f40fab00 0x5610f411cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f85659a3740 1 --2- 172.21.15.86:0/161119548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5610f40f4fa0 0x5610f411c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f85659a3740 1 -- 172.21.15.86:0/161119548 >> 172.21.15.86:0/161119548 conn(0x5610f40f0510 msgr2=0x5610f40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.458+0000 7f8545600640 0 client.37763 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:28:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.459+0000 7f85659a3740 1 -- 172.21.15.86:0/161119548 shutdown_connections 2024-04-17T07:28:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.459+0000 7f85659a3740 1 -- 172.21.15.86:0/161119548 wait complete. 2024-04-17T07:28:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.460+0000 7f85659a3740 20 client.37763 trim_cache size 0 max 16384 2024-04-17T07:28:37.461 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:28:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.462+0000 7f8562c00640 1 -- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85400a3c10 msgr2=0x7f85400a60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.462+0000 7f8562c00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85400a3c10 0x7f85400a60d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x5610f40f61f0 tx=0x7f8554007520 comp rx=0 tx=0).stop 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.462+0000 7f8562c00640 1 -- 172.21.15.86:0/884931684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f855c10bb90 msgr2=0x7f855c19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.462+0000 7f8562c00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f855c10bb90 0x7f855c19ed10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8548000c00 tx=0x7f854802eb90 comp rx=0 tx=0).stop 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.462+0000 7f8562c00640 1 -- 172.21.15.86:0/884931684 shutdown_connections 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.462+0000 7f8562c00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85400a3c10 0x7f85400a60d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.462+0000 7f8562c00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f855c10e650 0x7f855c1a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.462+0000 7f8562c00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f855c10bb90 0x7f855c19ed10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.462+0000 7f8562c00640 1 --2- 172.21.15.86:0/884931684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f855c1091d0 0x7f855c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.462+0000 7f8562c00640 1 -- 172.21.15.86:0/884931684 >> 172.21.15.86:0/884931684 conn(0x7f855c06f190 msgr2=0x7f855c109b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.463+0000 7f8562c00640 1 -- 172.21.15.86:0/884931684 shutdown_connections 2024-04-17T07:28:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.463+0000 7f8562c00640 1 -- 172.21.15.86:0/884931684 wait complete. 2024-04-17T07:28:37.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:28:37.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.861+0000 7faef4e00640 1 -- 172.21.15.86:0/1898453573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef0103190 msgr2=0x7faef0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.861+0000 7faef4e00640 1 --2- 172.21.15.86:0/1898453573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef0103190 0x7faef0103610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faee0009900 tx=0x7faee002bc40 comp rx=0 tx=0).stop 2024-04-17T07:28:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.862+0000 7faef4e00640 1 -- 172.21.15.86:0/1898453573 shutdown_connections 2024-04-17T07:28:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.862+0000 7faef4e00640 1 --2- 172.21.15.86:0/1898453573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef0103b50 0x7faef010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.862+0000 7faef4e00640 1 --2- 172.21.15.86:0/1898453573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef0103190 0x7faef0103610 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.862+0000 7faef4e00640 1 --2- 172.21.15.86:0/1898453573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faef0101fa0 0x7faef01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.862+0000 7faef4e00640 1 -- 172.21.15.86:0/1898453573 >> 172.21.15.86:0/1898453573 conn(0x7faef00fd510 msgr2=0x7faef00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.862+0000 7faef4e00640 1 -- 172.21.15.86:0/1898453573 shutdown_connections 2024-04-17T07:28:37.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.863+0000 7faef4e00640 1 -- 172.21.15.86:0/1898453573 wait complete. 2024-04-17T07:28:37.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.864+0000 7faef4e00640 1 Processor -- start 2024-04-17T07:28:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.865+0000 7faef4e00640 1 -- start start 2024-04-17T07:28:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.865+0000 7faef4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef0101fa0 0x7faef0107d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.865+0000 7faef4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef0103190 0x7faef0108260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.865+0000 7faef4e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faef0103b50 0x7faef0104890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.865+0000 7faef4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faef010db30 con 0x7faef0103190 2024-04-17T07:28:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.865+0000 7faef4e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faef010d9b0 con 0x7faef0103b50 2024-04-17T07:28:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.865+0000 7faef4e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faef010dcb0 con 0x7faef0101fa0 2024-04-17T07:28:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.866+0000 7faeede00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef0101fa0 0x7faef0107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.866+0000 7faeede00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef0101fa0 0x7faef0107d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38770/0 (socket says 172.21.15.86:38770) 2024-04-17T07:28:37.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.866+0000 7faeede00640 1 -- 172.21.15.86:0/3027393129 learned_addr learned my addr 172.21.15.86:0/3027393129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.866+0000 7faeee800640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faef0103b50 0x7faef0104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.866+0000 7faeed400640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef0103190 0x7faef0108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.866+0000 7faeede00640 1 -- 172.21.15.86:0/3027393129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faef0103b50 msgr2=0x7faef0104890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.866+0000 7faeede00640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faef0103b50 0x7faef0104890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.866+0000 7faeede00640 1 -- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef0103190 msgr2=0x7faef0108260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.866+0000 7faeede00640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef0103190 0x7faef0108260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.867+0000 7faeede00640 1 -- 172.21.15.86:0/3027393129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef0105150 con 0x7faef0101fa0 2024-04-17T07:28:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.867+0000 7faeed400640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef0103190 0x7faef0108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.868+0000 7faeede00640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef0101fa0 0x7faef0107d20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faedc00e9c0 tx=0x7faedc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.868+0000 7faee6a00640 1 -- 172.21.15.86:0/3027393129 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc00cd80 con 0x7faef0101fa0 2024-04-17T07:28:37.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.868+0000 7faee6a00640 1 -- 172.21.15.86:0/3027393129 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faedc0035f0 con 0x7faef0101fa0 2024-04-17T07:28:37.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.868+0000 7faee6a00640 1 -- 172.21.15.86:0/3027393129 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc010690 con 0x7faef0101fa0 2024-04-17T07:28:37.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.868+0000 7faef4e00640 1 -- 172.21.15.86:0/3027393129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef01a6850 con 0x7faef0101fa0 2024-04-17T07:28:37.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.868+0000 7faef4e00640 1 -- 172.21.15.86:0/3027393129 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef01a6c60 con 0x7faef0101fa0 2024-04-17T07:28:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.876+0000 7faef7b9c740 1 Processor -- start 2024-04-17T07:28:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.877+0000 7faef7b9c740 1 -- start start 2024-04-17T07:28:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.877+0000 7faee6a00640 1 -- 172.21.15.86:0/3027393129 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7faedc0037a0 con 0x7faef0101fa0 2024-04-17T07:28:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.877+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff876ffa0 0x561ff87703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.877+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8770990 0x561ff8775490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.877+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff8775b00 0x561ff8777f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.877+0000 7faef7b9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ff86d6020 con 0x561ff8775b00 2024-04-17T07:28:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.877+0000 7faef7b9c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561ff867c460 con 0x561ff876ffa0 2024-04-17T07:28:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.877+0000 7faef7b9c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561ff86d1fe0 con 0x561ff8770990 2024-04-17T07:28:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.878+0000 7faeed400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff8775b00 0x561ff8777f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.878+0000 7faeede00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8770990 0x561ff8775490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.878+0000 7faeede00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8770990 0x561ff8775490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38780/0 (socket says 172.21.15.86:38780) 2024-04-17T07:28:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.878+0000 7faeede00640 1 -- 172.21.15.86:0/852035944 learned_addr learned my addr 172.21.15.86:0/852035944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.879+0000 7faee6a00640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faed40a3c30 0x7faed40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.879+0000 7faeede00640 1 -- 172.21.15.86:0/852035944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff876ffa0 msgr2=0x561ff87703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.879+0000 7faeede00640 1 --2- 172.21.15.86:0/852035944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff876ffa0 0x561ff87703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.879+0000 7faeede00640 1 -- 172.21.15.86:0/852035944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff8775b00 msgr2=0x561ff8777f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.879+0000 7faeede00640 1 --2- 172.21.15.86:0/852035944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff8775b00 0x561ff8777f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.879+0000 7faee6a00640 1 -- 172.21.15.86:0/3027393129 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faedc0d0b50 con 0x7faef0101fa0 2024-04-17T07:28:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.879+0000 7faeede00640 1 -- 172.21.15.86:0/852035944 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff8684040 con 0x561ff8770990 2024-04-17T07:28:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.879+0000 7faeed400640 1 --2- 172.21.15.86:0/852035944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff8775b00 0x561ff8777f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.880+0000 7faeede00640 1 --2- 172.21.15.86:0/852035944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8770990 0x561ff8775490 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faedc0ce2d0 tx=0x7faedc003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=831f5b2c72ba30c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.880+0000 7faeee800640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faed40a3c30 0x7faed40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.882+0000 7faedb400640 1 -- 172.21.15.86:0/852035944 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc09c880 con 0x561ff8770990 2024-04-17T07:28:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.882+0000 7faedb400640 1 -- 172.21.15.86:0/852035944 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faedc003b30 con 0x561ff8770990 2024-04-17T07:28:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.882+0000 7faedb400640 1 -- 172.21.15.86:0/852035944 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc09c410 con 0x561ff8770990 2024-04-17T07:28:37.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.883+0000 7faef7b9c740 1 -- 172.21.15.86:0/852035944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8770990 msgr2=0x561ff8775490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.883+0000 7faef7b9c740 1 --2- 172.21.15.86:0/852035944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8770990 0x561ff8775490 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faedc0ce2d0 tx=0x7faedc003e30 comp rx=0 tx=0).stop 2024-04-17T07:28:37.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.883+0000 7faeee800640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faed40a3c30 0x7faed40a60f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x561ff87711f0 tx=0x7faee8010740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.884+0000 7faef7b9c740 1 -- 172.21.15.86:0/852035944 shutdown_connections 2024-04-17T07:28:37.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.884+0000 7faef7b9c740 1 --2- 172.21.15.86:0/852035944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff8775b00 0x561ff8777f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.884+0000 7faef7b9c740 1 --2- 172.21.15.86:0/852035944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8770990 0x561ff8775490 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.884+0000 7faef7b9c740 1 --2- 172.21.15.86:0/852035944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff876ffa0 0x561ff87703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.884+0000 7faef7b9c740 1 -- 172.21.15.86:0/852035944 >> 172.21.15.86:0/852035944 conn(0x561ff876b510 msgr2=0x561ff876b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.884+0000 7faef7b9c740 1 -- 172.21.15.86:0/852035944 shutdown_connections 2024-04-17T07:28:37.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.885+0000 7faef7b9c740 1 -- 172.21.15.86:0/852035944 wait complete. 2024-04-17T07:28:37.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.885+0000 7faef7b9c740 1 -- start start 2024-04-17T07:28:37.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.885+0000 7faef7b9c740 1 Processor -- start 2024-04-17T07:28:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.886+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 0x561ff877a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.886+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff877a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.886+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8775b00 0x561ff8781a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.886+0000 7faef7b9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561ff8778a50 con 0x561ff876ffa0 2024-04-17T07:28:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.886+0000 7faef7b9c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561ff868f830 con 0x561ff8770990 2024-04-17T07:28:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.886+0000 7faef7b9c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561ff8778bd0 con 0x561ff8775b00 2024-04-17T07:28:37.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.886+0000 7faef7b9c740 10 client.0 _ll_register_callbacks cb 0x561ff865dfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.887+0000 7faeede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 0x561ff877a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.887+0000 7faeede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 0x561ff877a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49928/0 (socket says 172.21.15.86:49928) 2024-04-17T07:28:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.887+0000 7faeede00640 1 -- 172.21.15.86:0/3721755260 learned_addr learned my addr 172.21.15.86:0/3721755260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.887+0000 7faeed400640 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff877a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.888+0000 7faeee800640 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8775b00 0x561ff8781a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.889+0000 7faeee800640 1 -- 172.21.15.86:0/3721755260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 msgr2=0x561ff877a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.889+0000 7faeee800640 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff877a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.889+0000 7faeee800640 1 -- 172.21.15.86:0/3721755260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 msgr2=0x561ff877a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.889+0000 7faeee800640 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 0x561ff877a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.889+0000 7faeee800640 1 -- 172.21.15.86:0/3721755260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff8693920 con 0x561ff8775b00 2024-04-17T07:28:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.889+0000 7faeee800640 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8775b00 0x561ff8781a00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faee8004a80 tx=0x7faee8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f7a061dca68e9ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.890+0000 7faecfe00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:28:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.890+0000 7faecfe00640 1 -- 172.21.15.86:0/3721755260 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8023070 con 0x561ff8775b00 2024-04-17T07:28:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.890+0000 7faecfe00640 1 -- 172.21.15.86:0/3721755260 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faee80119b0 con 0x561ff8775b00 2024-04-17T07:28:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.890+0000 7faef7b9c740 1 -- 172.21.15.86:0/3721755260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ff86e3a40 con 0x561ff8775b00 2024-04-17T07:28:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.891+0000 7faecfe00640 1 -- 172.21.15.86:0/3721755260 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faee802d050 con 0x561ff8775b00 2024-04-17T07:28:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.891+0000 7faef7b9c740 10 client.27817 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.891+0000 7faef7b9c740 1 -- 172.21.15.86:0/3721755260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561ff86617f0 con 0x561ff8775b00 2024-04-17T07:28:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.892+0000 7faecfe00640 1 -- 172.21.15.86:0/3721755260 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561ff86e3a40 con 0x561ff8775b00 2024-04-17T07:28:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.892+0000 7faecfe00640 1 -- 172.21.15.86:0/3721755260 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7faee800f040 con 0x561ff8775b00 2024-04-17T07:28:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.893+0000 7faef7b9c740 10 client.27817 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.893+0000 7faef7b9c740 10 client.27817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.893+0000 7faef7b9c740 20 client.27817 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.893+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561ff8786000 0x561ff8788440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.893+0000 7faef7b9c740 4 client.27817 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:37.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.893+0000 7faef7b9c740 1 -- 172.21.15.86:0/3721755260 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ff86dd8b0 con 0x561ff8786000 2024-04-17T07:28:37.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.893+0000 7faeede00640 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561ff8786000 0x561ff8788440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.895+0000 7faeede00640 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561ff8786000 0x561ff8788440 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f891a28c619f41 server_cookie=bb20abf8cf74ee1d in_seq=0 out_seq=0 2024-04-17T07:28:37.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.895+0000 7faecfe00640 10 client.27817 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:37.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.898+0000 7faecfe00640 1 -- 172.21.15.86:0/3721755260 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ff86dd8b0 con 0x561ff8786000 2024-04-17T07:28:37.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.898+0000 7faecfe00640 10 client.27817 handle_command_reply: tid=0 2024-04-17T07:28:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.898+0000 7faef7b9c740 1 client.27817 shutdown 2024-04-17T07:28:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.900+0000 7faef7b9c740 1 -- 172.21.15.86:0/3721755260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8775b00 msgr2=0x561ff8781a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.900+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8775b00 0x561ff8781a00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faee8004a80 tx=0x7faee8004ab0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.901+0000 7faef7b9c740 1 -- 172.21.15.86:0/3721755260 shutdown_connections 2024-04-17T07:28:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.901+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561ff8786000 0x561ff8788440 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.901+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff8775b00 0x561ff8781a00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.901+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff877a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.901+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3721755260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 0x561ff877a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.901+0000 7faef7b9c740 1 -- 172.21.15.86:0/3721755260 >> 172.21.15.86:0/3721755260 conn(0x561ff876b510 msgr2=0x561ff876a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.901+0000 7faecfe00640 0 client.27817 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.901+0000 7faef7b9c740 1 -- 172.21.15.86:0/3721755260 shutdown_connections 2024-04-17T07:28:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.902+0000 7faef7b9c740 1 -- 172.21.15.86:0/3721755260 wait complete. 2024-04-17T07:28:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.902+0000 7faef7b9c740 20 client.27817 trim_cache size 0 max 16384 2024-04-17T07:28:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.934+0000 7faef7b9c740 1 Processor -- start 2024-04-17T07:28:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.935+0000 7faef7b9c740 1 -- start start 2024-04-17T07:28:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.935+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876b510 0x561ff876ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.935+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876ffa0 0x561ff87819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.935+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff8769e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.935+0000 7faef7b9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faef01b1200 con 0x561ff876b510 2024-04-17T07:28:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.935+0000 7faef7b9c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faef01b2d40 con 0x561ff8770990 2024-04-17T07:28:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.935+0000 7faef7b9c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faef01b18d0 con 0x561ff876ffa0 2024-04-17T07:28:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.936+0000 7faeed400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876ffa0 0x561ff87819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.936+0000 7faeed400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876ffa0 0x561ff87819c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38806/0 (socket says 172.21.15.86:38806) 2024-04-17T07:28:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.936+0000 7faeed400640 1 -- 172.21.15.86:0/207326011 learned_addr learned my addr 172.21.15.86:0/207326011 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:37.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.936+0000 7faeee800640 1 --2- 172.21.15.86:0/207326011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff8769e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.937+0000 7faeede00640 1 --2- 172.21.15.86:0/207326011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876b510 0x561ff876ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.937+0000 7faeed400640 1 -- 172.21.15.86:0/207326011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 msgr2=0x561ff8769e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.937+0000 7faeed400640 1 --2- 172.21.15.86:0/207326011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff8769e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.937+0000 7faeed400640 1 -- 172.21.15.86:0/207326011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876b510 msgr2=0x561ff876ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.937+0000 7faeed400640 1 --2- 172.21.15.86:0/207326011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876b510 0x561ff876ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.937+0000 7faeed400640 1 -- 172.21.15.86:0/207326011 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff83e83f0 con 0x561ff876ffa0 2024-04-17T07:28:37.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.938+0000 7faeede00640 1 --2- 172.21.15.86:0/207326011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876b510 0x561ff876ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:37.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.938+0000 7faeed400640 1 --2- 172.21.15.86:0/207326011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876ffa0 0x561ff87819c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faee0005ab0 tx=0x7faee0005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84209cd8e1b27327 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.938+0000 7faedaa00640 1 -- 172.21.15.86:0/207326011 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0044070 con 0x561ff876ffa0 2024-04-17T07:28:37.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.938+0000 7faedaa00640 1 -- 172.21.15.86:0/207326011 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faee0004040 con 0x561ff876ffa0 2024-04-17T07:28:37.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.938+0000 7faedaa00640 1 -- 172.21.15.86:0/207326011 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0005620 con 0x561ff876ffa0 2024-04-17T07:28:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.939+0000 7faef7b9c740 1 -- 172.21.15.86:0/207326011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876ffa0 msgr2=0x561ff87819c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.939+0000 7faef7b9c740 1 --2- 172.21.15.86:0/207326011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876ffa0 0x561ff87819c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faee0005ab0 tx=0x7faee0005a20 comp rx=0 tx=0).stop 2024-04-17T07:28:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.940+0000 7faef7b9c740 1 -- 172.21.15.86:0/207326011 shutdown_connections 2024-04-17T07:28:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.940+0000 7faef7b9c740 1 --2- 172.21.15.86:0/207326011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff8769e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.940+0000 7faef7b9c740 1 --2- 172.21.15.86:0/207326011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876ffa0 0x561ff87819c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.940+0000 7faef7b9c740 1 --2- 172.21.15.86:0/207326011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876b510 0x561ff876ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.940+0000 7faef7b9c740 1 -- 172.21.15.86:0/207326011 >> 172.21.15.86:0/207326011 conn(0x561ff8771770 msgr2=0x561ff876e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.940+0000 7faef7b9c740 1 -- 172.21.15.86:0/207326011 shutdown_connections 2024-04-17T07:28:37.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.940+0000 7faef7b9c740 1 -- 172.21.15.86:0/207326011 wait complete. 2024-04-17T07:28:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.940+0000 7faef7b9c740 1 -- start start 2024-04-17T07:28:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.941+0000 7faef7b9c740 1 Processor -- start 2024-04-17T07:28:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.941+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876b510 0x561ff877d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.941+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 0x561ff877d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.941+0000 7faef7b9c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff8780680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.942+0000 7faef7b9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faef01b2670 con 0x561ff876ffa0 2024-04-17T07:28:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.942+0000 7faef7b9c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faef01b1fa0 con 0x561ff8770990 2024-04-17T07:28:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.942+0000 7faef7b9c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faef01af1d0 con 0x561ff876b510 2024-04-17T07:28:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.942+0000 7faeee800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff8780680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.942+0000 7faeee800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff8780680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:33002/0 (socket says 172.21.15.86:33002) 2024-04-17T07:28:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.942+0000 7faeee800640 1 -- 172.21.15.86:0/3833285761 learned_addr learned my addr 172.21.15.86:0/3833285761 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.942+0000 7faeed400640 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 0x561ff877d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.942+0000 7faef7b9c740 10 client.0 _ll_register_callbacks cb 0x7faee8010f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:37.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.942+0000 7faeede00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876b510 0x561ff877d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.944+0000 7faeede00640 1 -- 172.21.15.86:0/3833285761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 msgr2=0x561ff8780680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.944+0000 7faeede00640 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff8780680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.944+0000 7faeede00640 1 -- 172.21.15.86:0/3833285761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 msgr2=0x561ff877d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.944+0000 7faeede00640 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 0x561ff877d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.944+0000 7faeede00640 1 -- 172.21.15.86:0/3833285761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff84b27d0 con 0x561ff876b510 2024-04-17T07:28:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.944+0000 7faeee800640 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff8780680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:37.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.945+0000 7faeede00640 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876b510 0x561ff877d1c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faedc0ce320 tx=0x7faedc0d4150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e5d7f5fff4a8b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.945+0000 7faecfe00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:28:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.945+0000 7faecfe00640 1 -- 172.21.15.86:0/3833285761 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc09c5f0 con 0x561ff876b510 2024-04-17T07:28:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.945+0000 7faecfe00640 1 -- 172.21.15.86:0/3833285761 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faedc09ab50 con 0x561ff876b510 2024-04-17T07:28:37.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.945+0000 7faef7b9c740 1 -- 172.21.15.86:0/3833285761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faee0004040 con 0x561ff876b510 2024-04-17T07:28:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.946+0000 7faecfe00640 1 -- 172.21.15.86:0/3833285761 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faedc09cab0 con 0x561ff876b510 2024-04-17T07:28:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.946+0000 7faef7b9c740 10 client.27823 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.946+0000 7faef7b9c740 1 -- 172.21.15.86:0/3833285761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561ff8781d80 con 0x561ff876b510 2024-04-17T07:28:37.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.947+0000 7faecfe00640 1 -- 172.21.15.86:0/3833285761 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faee0004040 con 0x561ff876b510 2024-04-17T07:28:37.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.948+0000 7faecfe00640 1 -- 172.21.15.86:0/3833285761 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7faedc0d8070 con 0x561ff876b510 2024-04-17T07:28:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.950+0000 7faef7b9c740 10 client.27823 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.950+0000 7faef7b9c740 10 client.27823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.950+0000 7faef7b9c740 20 client.27823 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.950+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561ff8775b00 0x561ff87ff990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:37.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.950+0000 7faef7b9c740 4 client.27823 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:28:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.950+0000 7faef7b9c740 1 -- 172.21.15.86:0/3833285761 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ff8659570 con 0x561ff8775b00 2024-04-17T07:28:37.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.951+0000 7faeed400640 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561ff8775b00 0x561ff87ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.952+0000 7faeed400640 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561ff8775b00 0x561ff87ff990 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=505db4532e5df206 server_cookie=95b61b9df6c83ac8 in_seq=0 out_seq=0 2024-04-17T07:28:37.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.952+0000 7faecfe00640 10 client.27823 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.953+0000 7faecfe00640 1 -- 172.21.15.86:0/3833285761 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ff8659570 con 0x561ff8775b00 2024-04-17T07:28:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.953+0000 7faecfe00640 10 client.27823 handle_command_reply: tid=0 2024-04-17T07:28:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.953+0000 7faef7b9c740 1 client.27823 shutdown 2024-04-17T07:28:37.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.954+0000 7faef7b9c740 1 -- 172.21.15.86:0/3833285761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876b510 msgr2=0x561ff877d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.954+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876b510 0x561ff877d1c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faedc0ce320 tx=0x7faedc0d4150 comp rx=0 tx=0).stop 2024-04-17T07:28:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.954+0000 7faef7b9c740 1 -- 172.21.15.86:0/3833285761 shutdown_connections 2024-04-17T07:28:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.954+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561ff8775b00 0x561ff87ff990 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.954+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561ff8770990 0x561ff8780680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.954+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561ff876ffa0 0x561ff877d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.954+0000 7faef7b9c740 1 --2- 172.21.15.86:0/3833285761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561ff876b510 0x561ff877d1c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.954+0000 7faecfe00640 0 client.27823 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.954+0000 7faef7b9c740 1 -- 172.21.15.86:0/3833285761 >> 172.21.15.86:0/3833285761 conn(0x561ff8771770 msgr2=0x561ff876f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.955+0000 7faef7b9c740 1 -- 172.21.15.86:0/3833285761 shutdown_connections 2024-04-17T07:28:37.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.955+0000 7faef7b9c740 1 -- 172.21.15.86:0/3833285761 wait complete. 2024-04-17T07:28:37.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.955+0000 7faef7b9c740 20 client.27823 trim_cache size 0 max 16384 2024-04-17T07:28:37.956 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:28:37.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.957+0000 7faef4e00640 1 -- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faed40a3c30 msgr2=0x7faed40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.957+0000 7faef4e00640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faed40a3c30 0x7faed40a60f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x561ff87711f0 tx=0x7faee8010740 comp rx=0 tx=0).stop 2024-04-17T07:28:37.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.957+0000 7faef4e00640 1 -- 172.21.15.86:0/3027393129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef0101fa0 msgr2=0x7faef0107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:37.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.957+0000 7faef4e00640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef0101fa0 0x7faef0107d20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faedc00e9c0 tx=0x7faedc00ee80 comp rx=0 tx=0).stop 2024-04-17T07:28:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.957+0000 7faef4e00640 1 -- 172.21.15.86:0/3027393129 shutdown_connections 2024-04-17T07:28:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.957+0000 7faef4e00640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faed40a3c30 0x7faed40a60f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.957+0000 7faef4e00640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faef0103b50 0x7faef0104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.957+0000 7faef4e00640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef0103190 0x7faef0108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.957+0000 7faef4e00640 1 --2- 172.21.15.86:0/3027393129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faef0101fa0 0x7faef0107d20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.957+0000 7faef4e00640 1 -- 172.21.15.86:0/3027393129 >> 172.21.15.86:0/3027393129 conn(0x7faef00fd510 msgr2=0x7faef0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:37.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.958+0000 7faef4e00640 1 -- 172.21.15.86:0/3027393129 shutdown_connections 2024-04-17T07:28:37.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:37.959+0000 7faef4e00640 1 -- 172.21.15.86:0/3027393129 wait complete. 2024-04-17T07:28:38.999 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:28:38.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:28:38.999 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:28:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:38 smithi098 ceph-mon[35418]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.2 MiB/s wr, 647 op/s 2024-04-17T07:28:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:38 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:28:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:38 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:28:39.006 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:38 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:28:39.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:38 smithi103 ceph-mon[35495]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.2 MiB/s wr, 647 op/s 2024-04-17T07:28:39.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:38 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:28:39.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:38 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:28:39.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:38 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:28:39.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:38 smithi086 ceph-mon[31733]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.2 MiB/s wr, 647 op/s 2024-04-17T07:28:39.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:38 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:28:39.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:38 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:28:39.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:38 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:28:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.341+0000 7fea8f400640 1 -- 172.21.15.86:0/3672914090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881001d0 msgr2=0x7fea881005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.341+0000 7fea8f400640 1 --2- 172.21.15.86:0/3672914090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881001d0 0x7fea881005f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fea7400aa10 tx=0x7fea7402cf90 comp rx=0 tx=0).stop 2024-04-17T07:28:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.342+0000 7fea8f400640 1 -- 172.21.15.86:0/3672914090 shutdown_connections 2024-04-17T07:28:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.342+0000 7fea8f400640 1 --2- 172.21.15.86:0/3672914090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101d80 0x7fea88108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.342+0000 7fea8f400640 1 --2- 172.21.15.86:0/3672914090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea881013c0 0x7fea88101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.342+0000 7fea8f400640 1 --2- 172.21.15.86:0/3672914090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881001d0 0x7fea881005f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.342+0000 7fea8f400640 1 -- 172.21.15.86:0/3672914090 >> 172.21.15.86:0/3672914090 conn(0x7fea880fb760 msgr2=0x7fea880fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.342+0000 7fea8f400640 1 -- 172.21.15.86:0/3672914090 shutdown_connections 2024-04-17T07:28:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.342+0000 7fea8f400640 1 -- 172.21.15.86:0/3672914090 wait complete. 2024-04-17T07:28:39.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.344+0000 7fea8f400640 1 Processor -- start 2024-04-17T07:28:39.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.344+0000 7fea8f400640 1 -- start start 2024-04-17T07:28:39.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.345+0000 7fea8f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea881001d0 0x7fea88198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.345+0000 7fea8f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881013c0 0x7fea881987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.345+0000 7fea8f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea88101d80 0x7fea8819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.345+0000 7fea8f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea8810b910 con 0x7fea881001d0 2024-04-17T07:28:39.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.345+0000 7fea8f400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea8810b790 con 0x7fea881013c0 2024-04-17T07:28:39.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.345+0000 7fea8f400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea8810ba90 con 0x7fea88101d80 2024-04-17T07:28:39.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.345+0000 7fea87e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881013c0 0x7fea881987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.346+0000 7fea87e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881013c0 0x7fea881987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:33006/0 (socket says 172.21.15.86:33006) 2024-04-17T07:28:39.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.346+0000 7fea87e00640 1 -- 172.21.15.86:0/2367195496 learned_addr learned my addr 172.21.15.86:0/2367195496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:39.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.346+0000 7fea8d400640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea88101d80 0x7fea8819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.346+0000 7fea8d400640 1 -- 172.21.15.86:0/2367195496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881013c0 msgr2=0x7fea881987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:39.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.346+0000 7fea8d400640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881013c0 0x7fea881987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.346+0000 7fea8d400640 1 -- 172.21.15.86:0/2367195496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea881001d0 msgr2=0x7fea88198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:39.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.346+0000 7fea8d400640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea881001d0 0x7fea88198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.346+0000 7fea8d400640 1 -- 172.21.15.86:0/2367195496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8819fe90 con 0x7fea88101d80 2024-04-17T07:28:39.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.347+0000 7fea8d400640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea88101d80 0x7fea8819f820 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fea8000e940 tx=0x7fea8000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:39.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.347+0000 7fea85600640 1 -- 172.21.15.86:0/2367195496 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80004d20 con 0x7fea88101d80 2024-04-17T07:28:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.347+0000 7fea8f400640 1 -- 172.21.15.86:0/2367195496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea88102640 con 0x7fea88101d80 2024-04-17T07:28:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.348+0000 7fea85600640 1 -- 172.21.15.86:0/2367195496 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea80005020 con 0x7fea88101d80 2024-04-17T07:28:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.348+0000 7fea85600640 1 -- 172.21.15.86:0/2367195496 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800116d0 con 0x7fea88101d80 2024-04-17T07:28:39.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.348+0000 7fea8f400640 1 -- 172.21.15.86:0/2367195496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea88102b90 con 0x7fea88101d80 2024-04-17T07:28:39.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.356+0000 7fea92216740 1 Processor -- start 2024-04-17T07:28:39.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.357+0000 7fea85600640 1 -- 172.21.15.86:0/2367195496 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fea8000cb20 con 0x7fea88101d80 2024-04-17T07:28:39.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.359+0000 7fea92216740 1 -- start start 2024-04-17T07:28:39.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.359+0000 7fea92216740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f782e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.359+0000 7fea92216740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f788b0 0x55e6f3f7d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.359+0000 7fea85600640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea700a3d00 0x7fea700a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.360+0000 7fea92216740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f7da20 0x55e6f3f7feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.360+0000 7fea92216740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6f3eddcb0 con 0x55e6f3f788b0 2024-04-17T07:28:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.360+0000 7fea92216740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e6f3e84370 con 0x55e6f3f77ec0 2024-04-17T07:28:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.360+0000 7fea92216740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e6f3ed9c70 con 0x55e6f3f7da20 2024-04-17T07:28:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.360+0000 7fea85600640 1 -- 172.21.15.86:0/2367195496 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fea800d07e0 con 0x7fea88101d80 2024-04-17T07:28:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 555.0 s, 531.2 tps, lat 186.864 ms stddev 76.194 2024-04-17T07:28:39.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.360+0000 7fea8ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f788b0 0x55e6f3f7d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.360+0000 7fea87e00640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea700a3d00 0x7fea700a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.361+0000 7fea8d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f7da20 0x55e6f3f7feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.361+0000 7fea87e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f782e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.361+0000 7fea8d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f7da20 0x55e6f3f7feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38828/0 (socket says 172.21.15.86:38828) 2024-04-17T07:28:39.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.361+0000 7fea8d400640 1 -- 172.21.15.86:0/3515021017 learned_addr learned my addr 172.21.15.86:0/3515021017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:39.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.362+0000 7fea8ca00640 1 -- 172.21.15.86:0/3515021017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f7da20 msgr2=0x55e6f3f7feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:39.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.362+0000 7fea8ca00640 1 --2- 172.21.15.86:0/3515021017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f7da20 0x55e6f3f7feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.362+0000 7fea8ca00640 1 -- 172.21.15.86:0/3515021017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f77ec0 msgr2=0x55e6f3f782e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.362+0000 7fea8ca00640 1 --2- 172.21.15.86:0/3515021017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f782e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.362+0000 7fea8ca00640 1 -- 172.21.15.86:0/3515021017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f3ea3910 con 0x55e6f3f788b0 2024-04-17T07:28:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.363+0000 7fea8ca00640 1 --2- 172.21.15.86:0/3515021017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f788b0 0x55e6f3f7d3b0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fea74037040 tx=0x7fea74005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8684b44acc1e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.363+0000 7fea87e00640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea700a3d00 0x7fea700a61c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00b770 tx=0x7fea7c00b9e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.364+0000 7fea79c00640 1 -- 172.21.15.86:0/3515021017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74042020 con 0x55e6f3f788b0 2024-04-17T07:28:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.364+0000 7fea79c00640 1 -- 172.21.15.86:0/3515021017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea7402de00 con 0x55e6f3f788b0 2024-04-17T07:28:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.364+0000 7fea79c00640 1 -- 172.21.15.86:0/3515021017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74005260 con 0x55e6f3f788b0 2024-04-17T07:28:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.365+0000 7fea92216740 1 -- 172.21.15.86:0/3515021017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f788b0 msgr2=0x55e6f3f7d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.365+0000 7fea92216740 1 --2- 172.21.15.86:0/3515021017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f788b0 0x55e6f3f7d3b0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fea74037040 tx=0x7fea74005be0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.365+0000 7fea92216740 1 -- 172.21.15.86:0/3515021017 shutdown_connections 2024-04-17T07:28:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.365+0000 7fea92216740 1 --2- 172.21.15.86:0/3515021017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f7da20 0x55e6f3f7feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.365+0000 7fea92216740 1 --2- 172.21.15.86:0/3515021017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f788b0 0x55e6f3f7d3b0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.365+0000 7fea92216740 1 --2- 172.21.15.86:0/3515021017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.365+0000 7fea92216740 1 -- 172.21.15.86:0/3515021017 >> 172.21.15.86:0/3515021017 conn(0x55e6f3f73430 msgr2=0x55e6f3f73830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:39.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.365+0000 7fea92216740 1 -- 172.21.15.86:0/3515021017 shutdown_connections 2024-04-17T07:28:39.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.365+0000 7fea92216740 1 -- 172.21.15.86:0/3515021017 wait complete. 2024-04-17T07:28:39.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.366+0000 7fea92216740 1 -- start start 2024-04-17T07:28:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.366+0000 7fea92216740 1 Processor -- start 2024-04-17T07:28:39.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.368+0000 7fea92216740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f8abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.368+0000 7fea92216740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f8b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.368+0000 7fea92216740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 0x55e6f3f92140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.369+0000 7fea92216740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e6f3f80970 con 0x55e6f3f77ec0 2024-04-17T07:28:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.369+0000 7fea92216740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e6f3e97730 con 0x55e6f3f7da20 2024-04-17T07:28:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.369+0000 7fea92216740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e6f3f80af0 con 0x55e6f3f788b0 2024-04-17T07:28:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.369+0000 7fea87e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f8b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.369+0000 7fea92216740 10 client.0 _ll_register_callbacks cb 0x55e6f3e99b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.369+0000 7fea87e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f8b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38834/0 (socket says 172.21.15.86:38834) 2024-04-17T07:28:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.369+0000 7fea87e00640 1 -- 172.21.15.86:0/1476368051 learned_addr learned my addr 172.21.15.86:0/1476368051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.369+0000 7fea8d400640 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 0x55e6f3f92140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.369+0000 7fea8ca00640 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f8abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.370+0000 7fea8d400640 1 -- 172.21.15.86:0/1476368051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 msgr2=0x55e6f3f8b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.370+0000 7fea8d400640 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f8b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.370+0000 7fea8d400640 1 -- 172.21.15.86:0/1476368051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 msgr2=0x55e6f3f8abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.370+0000 7fea8d400640 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f8abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.370+0000 7fea8d400640 1 -- 172.21.15.86:0/1476368051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f3cb87e0 con 0x55e6f3f7da20 2024-04-17T07:28:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.370+0000 7fea87e00640 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f8b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:28:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.370+0000 7fea8ca00640 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f8abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.370+0000 7fea8d400640 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 0x55e6f3f92140 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea800990b0 tx=0x7fea80098a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb6c7f08b261210b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.370+0000 7fea6e000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:39.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.371+0000 7fea6e000640 1 -- 172.21.15.86:0/1476368051 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8001f040 con 0x55e6f3f7da20 2024-04-17T07:28:39.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.371+0000 7fea6e000640 1 -- 172.21.15.86:0/1476368051 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fea800ce010 con 0x55e6f3f7da20 2024-04-17T07:28:39.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.371+0000 7fea92216740 1 -- 172.21.15.86:0/1476368051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea7402de00 con 0x55e6f3f7da20 2024-04-17T07:28:39.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.372+0000 7fea6e000640 1 -- 172.21.15.86:0/1476368051 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea800a0a90 con 0x55e6f3f7da20 2024-04-17T07:28:39.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.372+0000 7fea92216740 10 client.37790 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:39.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.372+0000 7fea92216740 1 -- 172.21.15.86:0/1476368051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e6f3d18f70 con 0x55e6f3f7da20 2024-04-17T07:28:39.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.373+0000 7fea6e000640 1 -- 172.21.15.86:0/1476368051 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea7402de00 con 0x55e6f3f7da20 2024-04-17T07:28:39.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.373+0000 7fea6e000640 1 -- 172.21.15.86:0/1476368051 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fea800b5020 con 0x55e6f3f7da20 2024-04-17T07:28:39.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.374+0000 7fea92216740 10 client.37790 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:39.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.374+0000 7fea92216740 10 client.37790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:39.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.374+0000 7fea92216740 20 client.37790 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:39.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.374+0000 7fea92216740 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e6f3f96980 0x55e6f3f98e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.374+0000 7fea92216740 4 client.37790 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:39.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.374+0000 7fea92216740 1 -- 172.21.15.86:0/1476368051 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6f3eeb6d0 con 0x55e6f3f96980 2024-04-17T07:28:39.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.375+0000 7fea8ca00640 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e6f3f96980 0x55e6f3f98e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.377+0000 7fea8ca00640 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e6f3f96980 0x55e6f3f98e20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ced6252d5f2168 server_cookie=7ebb2b51e94745f1 in_seq=0 out_seq=0 2024-04-17T07:28:39.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.377+0000 7fea6e000640 10 client.37790 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:39.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.378+0000 7fea6e000640 1 -- 172.21.15.86:0/1476368051 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6f3eeb6d0 con 0x55e6f3f96980 2024-04-17T07:28:39.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.378+0000 7fea6e000640 10 client.37790 handle_command_reply: tid=0 2024-04-17T07:28:39.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.379+0000 7fea92216740 1 client.37790 shutdown 2024-04-17T07:28:39.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.379+0000 7fea92216740 1 -- 172.21.15.86:0/1476368051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 msgr2=0x55e6f3f92140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:39.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.379+0000 7fea92216740 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 0x55e6f3f92140 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea800990b0 tx=0x7fea80098a70 comp rx=0 tx=0).stop 2024-04-17T07:28:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.380+0000 7fea92216740 1 -- 172.21.15.86:0/1476368051 shutdown_connections 2024-04-17T07:28:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.380+0000 7fea92216740 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e6f3f96980 0x55e6f3f98e20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.380+0000 7fea92216740 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 0x55e6f3f92140 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.380+0000 7fea92216740 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f8b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.380+0000 7fea92216740 1 --2- 172.21.15.86:0/1476368051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f8abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.380+0000 7fea92216740 1 -- 172.21.15.86:0/1476368051 >> 172.21.15.86:0/1476368051 conn(0x55e6f3f73430 msgr2=0x55e6f3f725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:39.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.380+0000 7fea6e000640 0 client.37790 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.380+0000 7fea92216740 1 -- 172.21.15.86:0/1476368051 shutdown_connections 2024-04-17T07:28:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.380+0000 7fea92216740 1 -- 172.21.15.86:0/1476368051 wait complete. 2024-04-17T07:28:39.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.380+0000 7fea92216740 20 client.37790 trim_cache size 0 max 16384 2024-04-17T07:28:39.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.416+0000 7fea92216740 1 Processor -- start 2024-04-17T07:28:39.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.417+0000 7fea92216740 1 -- start start 2024-04-17T07:28:39.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.417+0000 7fea92216740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f71d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.417+0000 7fea92216740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f92100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.417+0000 7fea92216740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f7da20 0x55e6f3f72290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.417+0000 7fea92216740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea6800fc00 con 0x55e6f3f7da20 2024-04-17T07:28:39.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.417+0000 7fea92216740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea68011740 con 0x55e6f3f77ec0 2024-04-17T07:28:39.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.417+0000 7fea92216740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea680102d0 con 0x55e6f3f788b0 2024-04-17T07:28:39.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.418+0000 7fea87e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f92100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.418+0000 7fea87e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f92100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38836/0 (socket says 172.21.15.86:38836) 2024-04-17T07:28:39.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.418+0000 7fea87e00640 1 -- 172.21.15.86:0/3409113729 learned_addr learned my addr 172.21.15.86:0/3409113729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:39.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.419+0000 7fea8d400640 1 --2- 172.21.15.86:0/3409113729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f7da20 0x55e6f3f72290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.418+0000 7fea8ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f71d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.420+0000 7fea87e00640 1 -- 172.21.15.86:0/3409113729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f77ec0 msgr2=0x55e6f3f71d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:39.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.420+0000 7fea87e00640 1 --2- 172.21.15.86:0/3409113729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f71d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.420+0000 7fea87e00640 1 -- 172.21.15.86:0/3409113729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f7da20 msgr2=0x55e6f3f72290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:39.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.420+0000 7fea87e00640 1 --2- 172.21.15.86:0/3409113729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f7da20 0x55e6f3f72290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.420+0000 7fea87e00640 1 -- 172.21.15.86:0/3409113729 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f3bb0170 con 0x55e6f3f788b0 2024-04-17T07:28:39.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.420+0000 7fea87e00640 1 --2- 172.21.15.86:0/3409113729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f92100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00bbd0 tx=0x7fea7c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e59269ae488e7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:39.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.421+0000 7fea79200640 1 -- 172.21.15.86:0/3409113729 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c020940 con 0x55e6f3f788b0 2024-04-17T07:28:39.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.422+0000 7fea79200640 1 -- 172.21.15.86:0/3409113729 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea7c005020 con 0x55e6f3f788b0 2024-04-17T07:28:39.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.422+0000 7fea79200640 1 -- 172.21.15.86:0/3409113729 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0048e0 con 0x55e6f3f788b0 2024-04-17T07:28:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.426+0000 7fea92216740 1 -- 172.21.15.86:0/3409113729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 msgr2=0x55e6f3f92100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.426+0000 7fea92216740 1 --2- 172.21.15.86:0/3409113729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f92100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00bbd0 tx=0x7fea7c017000 comp rx=0 tx=0).stop 2024-04-17T07:28:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.426+0000 7fea92216740 1 -- 172.21.15.86:0/3409113729 shutdown_connections 2024-04-17T07:28:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.426+0000 7fea92216740 1 --2- 172.21.15.86:0/3409113729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f7da20 0x55e6f3f72290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.426+0000 7fea92216740 1 --2- 172.21.15.86:0/3409113729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f92100 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.426+0000 7fea92216740 1 --2- 172.21.15.86:0/3409113729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f71d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.426+0000 7fea92216740 1 -- 172.21.15.86:0/3409113729 >> 172.21.15.86:0/3409113729 conn(0x55e6f3f73430 msgr2=0x55e6f3f79db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.427+0000 7fea92216740 1 -- 172.21.15.86:0/3409113729 shutdown_connections 2024-04-17T07:28:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.427+0000 7fea92216740 1 -- 172.21.15.86:0/3409113729 wait complete. 2024-04-17T07:28:39.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.428+0000 7fea92216740 1 -- start start 2024-04-17T07:28:39.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.429+0000 7fea92216740 1 Processor -- start 2024-04-17T07:28:39.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.429+0000 7fea92216740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f90090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.429+0000 7fea92216740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f905d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.429+0000 7fea92216740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 0x55e6f3f91590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.429+0000 7fea92216740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea68011070 con 0x55e6f3f77ec0 2024-04-17T07:28:39.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.429+0000 7fea92216740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea680109a0 con 0x55e6f3f7da20 2024-04-17T07:28:39.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.429+0000 7fea92216740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea6800dbd0 con 0x55e6f3f788b0 2024-04-17T07:28:39.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.430+0000 7fea92216740 10 client.0 _ll_register_callbacks cb 0x7fea8009c0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.430+0000 7fea8ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f90090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.430+0000 7fea8ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f90090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49986/0 (socket says 172.21.15.86:49986) 2024-04-17T07:28:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.430+0000 7fea8ca00640 1 -- 172.21.15.86:0/3244570852 learned_addr learned my addr 172.21.15.86:0/3244570852 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.430+0000 7fea8d400640 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 0x55e6f3f91590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.431+0000 7fea87e00640 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f905d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.431+0000 7fea8ca00640 1 -- 172.21.15.86:0/3244570852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 msgr2=0x55e6f3f905d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:39.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.431+0000 7fea8ca00640 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f905d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.431+0000 7fea8ca00640 1 -- 172.21.15.86:0/3244570852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 msgr2=0x55e6f3f91590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:39.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.431+0000 7fea8ca00640 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 0x55e6f3f91590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.431+0000 7fea8ca00640 1 -- 172.21.15.86:0/3244570852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f3f91b80 con 0x55e6f3f77ec0 2024-04-17T07:28:39.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.431+0000 7fea87e00640 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:39.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.431+0000 7fea8d400640 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 0x55e6f3f91590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:28:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.432+0000 7fea8ca00640 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f90090 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fea74004c20 tx=0x7fea74005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f1aa880421352c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.432+0000 7fea6e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:28:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.432+0000 7fea6e000640 1 -- 172.21.15.86:0/3244570852 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74052070 con 0x55e6f3f77ec0 2024-04-17T07:28:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.432+0000 7fea6e000640 1 -- 172.21.15.86:0/3244570852 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fea7404eb50 con 0x55e6f3f77ec0 2024-04-17T07:28:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.432+0000 7fea92216740 1 -- 172.21.15.86:0/3244570852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea7c005020 con 0x55e6f3f77ec0 2024-04-17T07:28:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.433+0000 7fea6e000640 1 -- 172.21.15.86:0/3244570852 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea7405c050 con 0x55e6f3f77ec0 2024-04-17T07:28:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.433+0000 7fea92216740 10 client.18882 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.433+0000 7fea92216740 1 -- 172.21.15.86:0/3244570852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e6f3ffd8c0 con 0x55e6f3f77ec0 2024-04-17T07:28:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.434+0000 7fea6e000640 1 -- 172.21.15.86:0/3244570852 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea7c005020 con 0x55e6f3f77ec0 2024-04-17T07:28:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.434+0000 7fea6e000640 1 -- 172.21.15.86:0/3244570852 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fea7400b040 con 0x55e6f3f77ec0 2024-04-17T07:28:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.434+0000 7fea92216740 10 client.18882 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.434+0000 7fea92216740 10 client.18882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.434+0000 7fea92216740 20 client.18882 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.434+0000 7fea92216740 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e6f3ffdf40 0x55e6f4000330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.434+0000 7fea92216740 4 client.18882 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:28:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.434+0000 7fea92216740 1 -- 172.21.15.86:0/3244570852 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e6f3ee5540 con 0x55e6f3ffdf40 2024-04-17T07:28:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.435+0000 7fea87e00640 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e6f3ffdf40 0x55e6f4000330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.437+0000 7fea87e00640 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e6f3ffdf40 0x55e6f4000330 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2277d09ac66202d0 server_cookie=36ee3bd9497e5504 in_seq=0 out_seq=0 2024-04-17T07:28:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.437+0000 7fea6e000640 10 client.18882 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.440+0000 7fea6e000640 1 -- 172.21.15.86:0/3244570852 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e6f3ee5540 con 0x55e6f3ffdf40 2024-04-17T07:28:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.440+0000 7fea6e000640 10 client.18882 handle_command_reply: tid=0 2024-04-17T07:28:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.440+0000 7fea92216740 1 client.18882 shutdown 2024-04-17T07:28:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.441+0000 7fea92216740 1 -- 172.21.15.86:0/3244570852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 msgr2=0x55e6f3f90090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.441+0000 7fea92216740 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f90090 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fea74004c20 tx=0x7fea74005d50 comp rx=0 tx=0).stop 2024-04-17T07:28:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.441+0000 7fea92216740 1 -- 172.21.15.86:0/3244570852 shutdown_connections 2024-04-17T07:28:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.441+0000 7fea92216740 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e6f3ffdf40 0x55e6f4000330 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.441+0000 7fea92216740 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e6f3f7da20 0x55e6f3f91590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.441+0000 7fea92216740 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6f3f788b0 0x55e6f3f905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.441+0000 7fea92216740 1 --2- 172.21.15.86:0/3244570852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e6f3f77ec0 0x55e6f3f90090 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.441+0000 7fea92216740 1 -- 172.21.15.86:0/3244570852 >> 172.21.15.86:0/3244570852 conn(0x55e6f3f73430 msgr2=0x55e6f3f71d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.441+0000 7fea6e000640 0 client.18882 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.441+0000 7fea92216740 1 -- 172.21.15.86:0/3244570852 shutdown_connections 2024-04-17T07:28:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.442+0000 7fea92216740 1 -- 172.21.15.86:0/3244570852 wait complete. 2024-04-17T07:28:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.442+0000 7fea92216740 20 client.18882 trim_cache size 0 max 16384 2024-04-17T07:28:39.443 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:28:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:28:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "53784ac7-8424-4361-98d4-25c39d73728f", 2024-04-17T07:28:39.443 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:28:39.443 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:28:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.444+0000 7fea8f400640 1 -- 172.21.15.86:0/2367195496 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea700a3d00 msgr2=0x7fea700a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.444+0000 7fea8f400640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea700a3d00 0x7fea700a61c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00b770 tx=0x7fea7c00b9e0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.444+0000 7fea8f400640 1 -- 172.21.15.86:0/2367195496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea88101d80 msgr2=0x7fea8819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.444+0000 7fea8f400640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea88101d80 0x7fea8819f820 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fea8000e940 tx=0x7fea8000ee00 comp rx=0 tx=0).stop 2024-04-17T07:28:39.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.445+0000 7fea8f400640 1 -- 172.21.15.86:0/2367195496 shutdown_connections 2024-04-17T07:28:39.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.445+0000 7fea8f400640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fea700a3d00 0x7fea700a61c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.445+0000 7fea8f400640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea88101d80 0x7fea8819f820 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.445+0000 7fea8f400640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea881013c0 0x7fea881987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.445+0000 7fea8f400640 1 --2- 172.21.15.86:0/2367195496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea881001d0 0x7fea88198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:39.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.445+0000 7fea8f400640 1 -- 172.21.15.86:0/2367195496 >> 172.21.15.86:0/2367195496 conn(0x7fea880fb760 msgr2=0x7fea88106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:39.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.445+0000 7fea8f400640 1 -- 172.21.15.86:0/2367195496 shutdown_connections 2024-04-17T07:28:39.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:39.445+0000 7fea8f400640 1 -- 172.21.15.86:0/2367195496 wait complete. 2024-04-17T07:28:39.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:28:39.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:28:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.040+0000 7f9a5ca00640 1 -- 172.21.15.86:0/2567989549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a500967f0 msgr2=0x7f9a50098be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.040+0000 7f9a5ca00640 1 --2- 172.21.15.86:0/2567989549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a500967f0 0x7f9a50098be0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009900 tx=0x7f9a4c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:28:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.041+0000 7f9a5ca00640 1 -- 172.21.15.86:0/2567989549 shutdown_connections 2024-04-17T07:28:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.041+0000 7f9a5ca00640 1 --2- 172.21.15.86:0/2567989549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5009ba50 0x7f9a5009dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.041+0000 7f9a5ca00640 1 --2- 172.21.15.86:0/2567989549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a50099120 0x7f9a5009b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.041+0000 7f9a5ca00640 1 --2- 172.21.15.86:0/2567989549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a500967f0 0x7f9a50098be0 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009900 tx=0x7f9a4c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:28:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.041+0000 7f9a5ca00640 1 -- 172.21.15.86:0/2567989549 >> 172.21.15.86:0/2567989549 conn(0x7f9a500901c0 msgr2=0x7f9a50092620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:40.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.041+0000 7f9a5ca00640 1 -- 172.21.15.86:0/2567989549 shutdown_connections 2024-04-17T07:28:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.041+0000 7f9a5ca00640 1 -- 172.21.15.86:0/2567989549 wait complete. 2024-04-17T07:28:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.043+0000 7f9a5ca00640 1 Processor -- start 2024-04-17T07:28:40.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.043+0000 7f9a5ca00640 1 -- start start 2024-04-17T07:28:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.044+0000 7f9a5ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a50099120 0x7f9a5012a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.044+0000 7f9a5ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a5009ba50 0x7f9a5012a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.044+0000 7f9a5ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5012f5c0 0x7f9a50131a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.045+0000 7f9a57400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a50099120 0x7f9a5012a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.045+0000 7f9a57400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a50099120 0x7f9a5012a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:33062/0 (socket says 172.21.15.86:33062) 2024-04-17T07:28:40.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.045+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5012f5c0 0x7f9a50131a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.045+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5012f5c0 0x7f9a50131a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50010/0 (socket says 172.21.15.86:50010) 2024-04-17T07:28:40.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.045+0000 7f9a5ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a500a1920 con 0x7f9a5012f5c0 2024-04-17T07:28:40.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.045+0000 7f9a5ca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a500a17a0 con 0x7f9a50099120 2024-04-17T07:28:40.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.045+0000 7f9a5ca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a500a1aa0 con 0x7f9a5009ba50 2024-04-17T07:28:40.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.045+0000 7f9a57400640 1 -- 172.21.15.86:0/497235310 learned_addr learned my addr 172.21.15.86:0/497235310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:40.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.045+0000 7f9a56a00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a5009ba50 0x7f9a5012a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.046+0000 7f9a57e00640 1 -- 172.21.15.86:0/497235310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a5009ba50 msgr2=0x7f9a5012a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:40.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.047+0000 7f9a57e00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a5009ba50 0x7f9a5012a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.047+0000 7f9a57e00640 1 -- 172.21.15.86:0/497235310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a50099120 msgr2=0x7f9a5012a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:40.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.047+0000 7f9a57e00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a50099120 0x7f9a5012a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.047+0000 7f9a57e00640 1 -- 172.21.15.86:0/497235310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a50132040 con 0x7f9a5012f5c0 2024-04-17T07:28:40.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.049+0000 7f9a57e00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5012f5c0 0x7f9a50131a80 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9a58067c60 tx=0x7f9a5807be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:40.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.050+0000 7f9a4be00640 1 -- 172.21.15.86:0/497235310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5806ac40 con 0x7f9a5012f5c0 2024-04-17T07:28:40.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.050+0000 7f9a5ca00640 1 -- 172.21.15.86:0/497235310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a50132330 con 0x7f9a5012f5c0 2024-04-17T07:28:40.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.050+0000 7f9a5ca00640 1 -- 172.21.15.86:0/497235310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a50132800 con 0x7f9a5012f5c0 2024-04-17T07:28:40.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.051+0000 7f9a4be00640 1 -- 172.21.15.86:0/497235310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a5806ade0 con 0x7f9a5012f5c0 2024-04-17T07:28:40.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.051+0000 7f9a4be00640 1 -- 172.21.15.86:0/497235310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58079660 con 0x7f9a5012f5c0 2024-04-17T07:28:40.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.054+0000 7f9a60dd5740 1 Processor -- start 2024-04-17T07:28:40.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.055+0000 7f9a60dd5740 1 -- start start 2024-04-17T07:28:40.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.055+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 0x55f1064c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.055+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c0d90 0x55f1064c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.055+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c5f00 0x55f1064c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.055+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f106427120 con 0x55f1064c03a0 2024-04-17T07:28:40.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.055+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f1063cd340 con 0x55f1064c5f00 2024-04-17T07:28:40.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.055+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f1064230e0 con 0x55f1064c0d90 2024-04-17T07:28:40.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.056+0000 7f9a57400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c0d90 0x55f1064c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.056+0000 7f9a56a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 0x55f1064c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.056+0000 7f9a56a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 0x55f1064c07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50020/0 (socket says 172.21.15.86:50020) 2024-04-17T07:28:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.056+0000 7f9a56a00640 1 -- 172.21.15.86:0/882828983 learned_addr learned my addr 172.21.15.86:0/882828983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.057+0000 7f9a56a00640 1 -- 172.21.15.86:0/882828983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c0d90 msgr2=0x55f1064c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:40.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.057+0000 7f9a56a00640 1 --2- 172.21.15.86:0/882828983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c0d90 0x55f1064c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.057+0000 7f9a56a00640 1 -- 172.21.15.86:0/882828983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c5f00 msgr2=0x55f1064c8390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:28:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.057+0000 7f9a56a00640 1 --2- 172.21.15.86:0/882828983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c5f00 0x55f1064c8390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.057+0000 7f9a56a00640 1 -- 172.21.15.86:0/882828983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1062037d0 con 0x55f1064c03a0 2024-04-17T07:28:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.057+0000 7f9a56a00640 1 --2- 172.21.15.86:0/882828983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 0x55f1064c07c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009c30 tx=0x7f9a44011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f3b151b431a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:40.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.058+0000 7f9a3fe00640 1 -- 172.21.15.86:0/882828983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4401e070 con 0x55f1064c03a0 2024-04-17T07:28:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.060+0000 7f9a3fe00640 1 -- 172.21.15.86:0/882828983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a44019ae0 con 0x55f1064c03a0 2024-04-17T07:28:40.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.060+0000 7f9a3fe00640 1 -- 172.21.15.86:0/882828983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400e9b0 con 0x55f1064c03a0 2024-04-17T07:28:40.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.062+0000 7f9a60dd5740 1 -- 172.21.15.86:0/882828983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 msgr2=0x55f1064c07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.062+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/882828983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 0x55f1064c07c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009c30 tx=0x7f9a44011810 comp rx=0 tx=0).stop 2024-04-17T07:28:40.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.064+0000 7f9a4be00640 1 -- 172.21.15.86:0/497235310 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9a58079800 con 0x7f9a5012f5c0 2024-04-17T07:28:40.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.065+0000 7f9a60dd5740 1 -- 172.21.15.86:0/882828983 shutdown_connections 2024-04-17T07:28:40.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.065+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/882828983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c5f00 0x55f1064c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.065+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/882828983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c0d90 0x55f1064c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.065+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/882828983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 0x55f1064c07c0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.065+0000 7f9a60dd5740 1 -- 172.21.15.86:0/882828983 >> 172.21.15.86:0/882828983 conn(0x55f1064bb910 msgr2=0x55f1064bbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.066+0000 7f9a60dd5740 1 -- 172.21.15.86:0/882828983 shutdown_connections 2024-04-17T07:28:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.066+0000 7f9a60dd5740 1 -- 172.21.15.86:0/882828983 wait complete. 2024-04-17T07:28:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.067+0000 7f9a60dd5740 1 -- start start 2024-04-17T07:28:40.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.067+0000 7f9a4be00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a400a3bc0 0x7f9a400a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.067+0000 7f9a60dd5740 1 Processor -- start 2024-04-17T07:28:40.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.067+0000 7f9a4be00640 1 -- 172.21.15.86:0/497235310 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9a5813cec0 con 0x7f9a5012f5c0 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.068+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 0x55f1064ec920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.068+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064ece60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.068+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c5f00 0x55f1064f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.068+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f1064eaec0 con 0x55f1064c03a0 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.068+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f1063e0710 con 0x55f1064c0d90 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.068+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f1064eb040 con 0x55f1064c5f00 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.068+0000 7f9a60dd5740 10 client.0 _ll_register_callbacks cb 0x55f1063dbaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.068+0000 7f9a56a00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a400a3bc0 0x7f9a400a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.069+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c5f00 0x55f1064f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.069+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c5f00 0x55f1064f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38868/0 (socket says 172.21.15.86:38868) 2024-04-17T07:28:40.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.069+0000 7f9a57e00640 1 -- 172.21.15.86:0/687107420 learned_addr learned my addr 172.21.15.86:0/687107420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:40.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.069+0000 7f9a57400640 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 0x55f1064ec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.069+0000 7f9a57e00640 1 -- 172.21.15.86:0/687107420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 msgr2=0x55f1064ece60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:28:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.069+0000 7f9a57e00640 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064ece60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.069+0000 7f9a57e00640 1 -- 172.21.15.86:0/687107420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 msgr2=0x55f1064ec920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.069+0000 7f9a57e00640 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 0x55f1064ec920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.070+0000 7f9a57e00640 1 -- 172.21.15.86:0/687107420 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1061393f0 con 0x55f1064c5f00 2024-04-17T07:28:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.070+0000 7f9a57e00640 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c5f00 0x55f1064f3ea0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9a580515f0 tx=0x7f9a58108900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a23ed18e8b67324e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.070+0000 7f9a3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:28:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.071+0000 7f9a3cc00640 1 -- 172.21.15.86:0/687107420 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58121020 con 0x55f1064c5f00 2024-04-17T07:28:40.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.071+0000 7f9a3cc00640 1 -- 172.21.15.86:0/687107420 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9a58109020 con 0x55f1064c5f00 2024-04-17T07:28:40.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.071+0000 7f9a60dd5740 1 -- 172.21.15.86:0/687107420 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a44019ae0 con 0x55f1064c5f00 2024-04-17T07:28:40.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.071+0000 7f9a56a00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a400a3bc0 0x7f9a400a6080 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9a440099f0 tx=0x7f9a44009ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:40.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.072+0000 7f9a3cc00640 1 -- 172.21.15.86:0/687107420 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a58108dc0 con 0x55f1064c5f00 2024-04-17T07:28:40.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.072+0000 7f9a60dd5740 10 client.27847 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:40.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.072+0000 7f9a60dd5740 1 -- 172.21.15.86:0/687107420 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1060f90f0 con 0x55f1064c5f00 2024-04-17T07:28:40.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.073+0000 7f9a3cc00640 1 -- 172.21.15.86:0/687107420 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a44019ae0 con 0x55f1064c5f00 2024-04-17T07:28:40.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.073+0000 7f9a3cc00640 1 -- 172.21.15.86:0/687107420 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9a58082030 con 0x55f1064c5f00 2024-04-17T07:28:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.073+0000 7f9a60dd5740 10 client.27847 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.073+0000 7f9a60dd5740 10 client.27847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.073+0000 7f9a60dd5740 20 client.27847 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.073+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1064f8820 0x55f1064fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.073+0000 7f9a60dd5740 4 client.27847 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:28:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.073+0000 7f9a60dd5740 1 -- 172.21.15.86:0/687107420 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f106434b40 con 0x55f1064f8820 2024-04-17T07:28:40.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.074+0000 7f9a57400640 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1064f8820 0x55f1064fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.076+0000 7f9a3cc00640 10 client.27847 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:40.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.076+0000 7f9a57400640 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1064f8820 0x55f1064fac80 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af49586a33748da1 server_cookie=c2da89db27ba42aa in_seq=0 out_seq=0 2024-04-17T07:28:40.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.077+0000 7f9a3cc00640 1 -- 172.21.15.86:0/687107420 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f106434b40 con 0x55f1064f8820 2024-04-17T07:28:40.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.077+0000 7f9a3cc00640 10 client.27847 handle_command_reply: tid=0 2024-04-17T07:28:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.078+0000 7f9a60dd5740 1 client.27847 shutdown 2024-04-17T07:28:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.079+0000 7f9a60dd5740 1 -- 172.21.15.86:0/687107420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c5f00 msgr2=0x55f1064f3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.079+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c5f00 0x55f1064f3ea0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9a580515f0 tx=0x7f9a58108900 comp rx=0 tx=0).stop 2024-04-17T07:28:40.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.079+0000 7f9a60dd5740 1 -- 172.21.15.86:0/687107420 shutdown_connections 2024-04-17T07:28:40.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.079+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f1064f8820 0x55f1064fac80 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.079+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c5f00 0x55f1064f3ea0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.079+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064ece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.079+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/687107420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c03a0 0x55f1064ec920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.079+0000 7f9a60dd5740 1 -- 172.21.15.86:0/687107420 >> 172.21.15.86:0/687107420 conn(0x55f1064bb910 msgr2=0x55f1064bab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:40.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.079+0000 7f9a3cc00640 0 client.27847 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.080+0000 7f9a60dd5740 1 -- 172.21.15.86:0/687107420 shutdown_connections 2024-04-17T07:28:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.080+0000 7f9a60dd5740 1 -- 172.21.15.86:0/687107420 wait complete. 2024-04-17T07:28:40.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.081+0000 7f9a60dd5740 20 client.27847 trim_cache size 0 max 16384 2024-04-17T07:28:40.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.129+0000 7f9a60dd5740 1 Processor -- start 2024-04-17T07:28:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.129+0000 7f9a60dd5740 1 -- start start 2024-04-17T07:28:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.130+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 0x55f1064bf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.130+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.130+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 0x55f1064ff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.130+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3800fa90 con 0x55f1064c5f00 2024-04-17T07:28:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.130+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a380115d0 con 0x55f1064c0d90 2024-04-17T07:28:40.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.130+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a38010160 con 0x55f1064c03a0 2024-04-17T07:28:40.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.133+0000 7f9a56a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.133+0000 7f9a56a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:33076/0 (socket says 172.21.15.86:33076) 2024-04-17T07:28:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.133+0000 7f9a56a00640 1 -- 172.21.15.86:0/2014481586 learned_addr learned my addr 172.21.15.86:0/2014481586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.133+0000 7f9a57400640 1 --2- 172.21.15.86:0/2014481586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 0x55f1064bf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.134+0000 7f9a57e00640 1 --2- 172.21.15.86:0/2014481586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 0x55f1064ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.134+0000 7f9a56a00640 1 -- 172.21.15.86:0/2014481586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 msgr2=0x55f1064bf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:40.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.134+0000 7f9a56a00640 1 --2- 172.21.15.86:0/2014481586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 0x55f1064bf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.134+0000 7f9a56a00640 1 -- 172.21.15.86:0/2014481586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 msgr2=0x55f1064ff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:40.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.134+0000 7f9a56a00640 1 --2- 172.21.15.86:0/2014481586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 0x55f1064ff540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.134+0000 7f9a56a00640 1 -- 172.21.15.86:0/2014481586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1064c1170 con 0x55f1064c0d90 2024-04-17T07:28:40.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.135+0000 7f9a56a00640 1 --2- 172.21.15.86:0/2014481586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f3e60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009bd0 tx=0x7f9a44022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b585898fce03484 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.136+0000 7f9a3f400640 1 -- 172.21.15.86:0/2014481586 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4401e070 con 0x55f1064c0d90 2024-04-17T07:28:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.136+0000 7f9a3f400640 1 -- 172.21.15.86:0/2014481586 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a4401f040 con 0x55f1064c0d90 2024-04-17T07:28:40.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.136+0000 7f9a3f400640 1 -- 172.21.15.86:0/2014481586 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4401e070 con 0x55f1064c0d90 2024-04-17T07:28:40.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.141+0000 7f9a60dd5740 1 -- 172.21.15.86:0/2014481586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 msgr2=0x55f1064f3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:40.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.141+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/2014481586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f3e60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009bd0 tx=0x7f9a44022000 comp rx=0 tx=0).stop 2024-04-17T07:28:40.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.141+0000 7f9a60dd5740 1 -- 172.21.15.86:0/2014481586 shutdown_connections 2024-04-17T07:28:40.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.141+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/2014481586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 0x55f1064ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.141+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/2014481586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f3e60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.141+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/2014481586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 0x55f1064bf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.141+0000 7f9a60dd5740 1 -- 172.21.15.86:0/2014481586 >> 172.21.15.86:0/2014481586 conn(0x55f1064bb910 msgr2=0x55f1064beb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:40.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.144+0000 7f9a60dd5740 1 -- 172.21.15.86:0/2014481586 shutdown_connections 2024-04-17T07:28:40.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.145+0000 7f9a60dd5740 1 -- 172.21.15.86:0/2014481586 wait complete. 2024-04-17T07:28:40.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.146+0000 7f9a60dd5740 1 -- start start 2024-04-17T07:28:40.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.147+0000 7f9a60dd5740 1 Processor -- start 2024-04-17T07:28:40.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.150+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 0x55f1064f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.150+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.150+0000 7f9a60dd5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 0x55f1064f3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.150+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a38010f00 con 0x55f1064c5f00 2024-04-17T07:28:40.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.150+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a38010830 con 0x55f1064c0d90 2024-04-17T07:28:40.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.150+0000 7f9a60dd5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3800da60 con 0x55f1064c03a0 2024-04-17T07:28:40.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.150+0000 7f9a60dd5740 10 client.0 _ll_register_callbacks cb 0x7f9a5807bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:28:40.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.151+0000 7f9a57400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 0x55f1064f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.152+0000 7f9a56a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.152+0000 7f9a57400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 0x55f1064f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:38880/0 (socket says 172.21.15.86:38880) 2024-04-17T07:28:40.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.152+0000 7f9a57400640 1 -- 172.21.15.86:0/621564059 learned_addr learned my addr 172.21.15.86:0/621564059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:28:40.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.153+0000 7f9a56a00640 1 -- 172.21.15.86:0/621564059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 msgr2=0x55f1064f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:40.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.153+0000 7f9a56a00640 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 0x55f1064f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.153+0000 7f9a56a00640 1 -- 172.21.15.86:0/621564059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 msgr2=0x55f1064f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:28:40.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.154+0000 7f9a57e00640 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 0x55f1064f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.154+0000 7f9a56a00640 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 0x55f1064f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.154+0000 7f9a56a00640 1 -- 172.21.15.86:0/621564059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1064f3780 con 0x55f1064c0d90 2024-04-17T07:28:40.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.155+0000 7f9a57e00640 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 0x55f1064f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:28:40.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.156+0000 7f9a56a00640 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f2180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9a4401c040 tx=0x7f9a44019b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f88cddce6e3c9a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:28:40.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.156+0000 7f9a3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:28:40.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.158+0000 7f9a3cc00640 1 -- 172.21.15.86:0/621564059 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4403c070 con 0x55f1064c0d90 2024-04-17T07:28:40.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.158+0000 7f9a3cc00640 1 -- 172.21.15.86:0/621564059 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9a44034040 con 0x55f1064c0d90 2024-04-17T07:28:40.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.158+0000 7f9a60dd5740 1 -- 172.21.15.86:0/621564059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a4401f040 con 0x55f1064c0d90 2024-04-17T07:28:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.159+0000 7f9a3cc00640 1 -- 172.21.15.86:0/621564059 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a44046050 con 0x55f1064c0d90 2024-04-17T07:28:40.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.159+0000 7f9a60dd5740 10 client.37808 fetch_fsmap learned FSMap version 28 2024-04-17T07:28:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.159+0000 7f9a60dd5740 1 -- 172.21.15.86:0/621564059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f10655f3c0 con 0x55f1064c0d90 2024-04-17T07:28:40.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.160+0000 7f9a3cc00640 1 -- 172.21.15.86:0/621564059 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a4401f040 con 0x55f1064c0d90 2024-04-17T07:28:40.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.161+0000 7f9a3cc00640 1 -- 172.21.15.86:0/621564059 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9a4402e070 con 0x55f1064c0d90 2024-04-17T07:28:40.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.161+0000 7f9a60dd5740 10 client.37808 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:28:40.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.161+0000 7f9a60dd5740 10 client.37808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:28:40.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.161+0000 7f9a60dd5740 20 client.37808 populate_metadata read hostname 'smithi086' 2024-04-17T07:28:40.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.161+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f10655fb00 0x55f106561ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:28:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.161+0000 7f9a60dd5740 4 client.37808 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:28:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.161+0000 7f9a60dd5740 1 -- 172.21.15.86:0/621564059 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f10642e9b0 con 0x55f10655fb00 2024-04-17T07:28:40.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.162+0000 7f9a57400640 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f10655fb00 0x55f106561ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:28:40.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.165+0000 7f9a57400640 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f10655fb00 0x55f106561ef0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c446896fd4bb228c server_cookie=623ce1d70d2bfa29 in_seq=0 out_seq=0 2024-04-17T07:28:40.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.166+0000 7f9a3cc00640 10 client.37808 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:40.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.167+0000 7f9a3cc00640 1 -- 172.21.15.86:0/621564059 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f10642e9b0 con 0x55f10655fb00 2024-04-17T07:28:40.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.168+0000 7f9a3cc00640 10 client.37808 handle_command_reply: tid=0 2024-04-17T07:28:40.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.168+0000 7f9a60dd5740 1 client.37808 shutdown 2024-04-17T07:28:40.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.168+0000 7f9a60dd5740 1 -- 172.21.15.86:0/621564059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 msgr2=0x55f1064f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:40.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.168+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f2180 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9a4401c040 tx=0x7f9a44019b30 comp rx=0 tx=0).stop 2024-04-17T07:28:40.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.169+0000 7f9a60dd5740 1 -- 172.21.15.86:0/621564059 shutdown_connections 2024-04-17T07:28:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.169+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f10655fb00 0x55f106561ef0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.169+0000 7f9a3cc00640 0 client.37808 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:28:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.169+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f1064c5f00 0x55f1064f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.169+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f1064c0d90 0x55f1064f2180 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.169+0000 7f9a60dd5740 1 --2- 172.21.15.86:0/621564059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f1064c03a0 0x55f1064f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.169+0000 7f9a60dd5740 1 -- 172.21.15.86:0/621564059 >> 172.21.15.86:0/621564059 conn(0x55f1064bb910 msgr2=0x55f1064bf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:40.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.170+0000 7f9a60dd5740 1 -- 172.21.15.86:0/621564059 shutdown_connections 2024-04-17T07:28:40.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.170+0000 7f9a60dd5740 1 -- 172.21.15.86:0/621564059 wait complete. 2024-04-17T07:28:40.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.170+0000 7f9a60dd5740 20 client.37808 trim_cache size 0 max 16384 2024-04-17T07:28:40.172 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:28:40.172 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:28:40.172 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:28:40.172 INFO:teuthology.orchestra.run.smithi086.stdout: "53784ac7-8424-4361-98d4-25c39d73728f": { 2024-04-17T07:28:40.172 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:28:40.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "53784ac7-8424-4361-98d4-25c39d73728f", 2024-04-17T07:28:40.173 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:28:40.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:28:40.173 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:28:40.173 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:28:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 -- 172.21.15.86:0/497235310 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a400a3bc0 msgr2=0x7f9a400a6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a400a3bc0 0x7f9a400a6080 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9a440099f0 tx=0x7f9a44009ea0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 -- 172.21.15.86:0/497235310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5012f5c0 msgr2=0x7f9a50131a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:28:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5012f5c0 0x7f9a50131a80 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9a58067c60 tx=0x7f9a5807be20 comp rx=0 tx=0).stop 2024-04-17T07:28:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 -- 172.21.15.86:0/497235310 shutdown_connections 2024-04-17T07:28:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9a400a3bc0 0x7f9a400a6080 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5012f5c0 0x7f9a50131a80 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a5009ba50 0x7f9a5012a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 --2- 172.21.15.86:0/497235310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a50099120 0x7f9a5012a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:28:40.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 -- 172.21.15.86:0/497235310 >> 172.21.15.86:0/497235310 conn(0x7f9a500901c0 msgr2=0x7f9a50095f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:28:40.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.173+0000 7f9a5ca00640 1 -- 172.21.15.86:0/497235310 shutdown_connections 2024-04-17T07:28:40.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:28:40.174+0000 7f9a5ca00640 1 -- 172.21.15.86:0/497235310 wait complete. 2024-04-17T07:28:40.670 INFO:tasks.cephfs.filesystem:scrub status for tag:53784ac7-8424-4361-98d4-25c39d73728f - {'path': '/', 'tag': '53784ac7-8424-4361-98d4-25c39d73728f', 'options': 'recursive,force'} 2024-04-17T07:28:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:40 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:28:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:40 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:41.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:40 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:28:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:40 smithi098 ceph-mon[35418]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 603 op/s 2024-04-17T07:28:41.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:40 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:41.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:40 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:28:41.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:40 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:41.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:40 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:28:41.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:40 smithi103 ceph-mon[35495]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 603 op/s 2024-04-17T07:28:41.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:40 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:41.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:40 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:28:41.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:40 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:41.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:40 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:28:41.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:40 smithi086 ceph-mon[31733]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 603 op/s 2024-04-17T07:28:41.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:40 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:28:43.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:42 smithi098 ceph-mon[35418]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.2 MiB/s wr, 640 op/s 2024-04-17T07:28:43.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:42 smithi103 ceph-mon[35495]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.2 MiB/s wr, 640 op/s 2024-04-17T07:28:43.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:42 smithi086 ceph-mon[31733]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.2 MiB/s wr, 640 op/s 2024-04-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 560.0 s, 606.4 tps, lat 166.199 ms stddev 97.558 2024-04-17T07:28:45.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:44 smithi098 ceph-mon[35418]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 632 op/s 2024-04-17T07:28:45.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:44 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:28:45.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:44 smithi103 ceph-mon[35495]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 632 op/s 2024-04-17T07:28:45.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:44 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:28:45.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:44 smithi086 ceph-mon[31733]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 632 op/s 2024-04-17T07:28:45.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:44 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:28:47.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:46 smithi098 ceph-mon[35418]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 515 op/s 2024-04-17T07:28:47.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:46 smithi103 ceph-mon[35495]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 515 op/s 2024-04-17T07:28:47.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:46 smithi086 ceph-mon[31733]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 515 op/s 2024-04-17T07:28:49.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:48 smithi103 ceph-mon[35495]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 643 op/s 2024-04-17T07:28:49.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:48 smithi086 ceph-mon[31733]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 643 op/s 2024-04-17T07:28:49.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:48 smithi098 ceph-mon[35418]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.4 MiB/s wr, 643 op/s 2024-04-17T07:28:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 565.0 s, 697.2 tps, lat 143.794 ms stddev 44.341 2024-04-17T07:28:51.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:50 smithi103 ceph-mon[35495]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.8 MiB/s wr, 614 op/s 2024-04-17T07:28:51.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:50 smithi086 ceph-mon[31733]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.8 MiB/s wr, 614 op/s 2024-04-17T07:28:51.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:50 smithi098 ceph-mon[35418]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.8 MiB/s wr, 614 op/s 2024-04-17T07:28:53.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:52 smithi103 ceph-mon[35495]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.3 MiB/s wr, 675 op/s 2024-04-17T07:28:53.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:52 smithi086 ceph-mon[31733]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.3 MiB/s wr, 675 op/s 2024-04-17T07:28:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:52 smithi098 ceph-mon[35418]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.3 MiB/s wr, 675 op/s 2024-04-17T07:28:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 570.0 s, 630.6 tps, lat 158.097 ms stddev 64.700 2024-04-17T07:28:55.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:54 smithi103 ceph-mon[35495]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.4 MiB/s wr, 698 op/s 2024-04-17T07:28:55.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:54 smithi086 ceph-mon[31733]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.4 MiB/s wr, 698 op/s 2024-04-17T07:28:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:54 smithi098 ceph-mon[35418]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.4 MiB/s wr, 698 op/s 2024-04-17T07:28:57.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:56 smithi086 ceph-mon[31733]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 MiB/s wr, 631 op/s 2024-04-17T07:28:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:56 smithi098 ceph-mon[35418]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 MiB/s wr, 631 op/s 2024-04-17T07:28:57.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:56 smithi103 ceph-mon[35495]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.9 MiB/s wr, 631 op/s 2024-04-17T07:28:59.055 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:28:58 smithi098 ceph-mon[35418]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 709 op/s 2024-04-17T07:28:59.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:28:58 smithi086 ceph-mon[31733]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 709 op/s 2024-04-17T07:28:59.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:28:58 smithi103 ceph-mon[35495]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 709 op/s 2024-04-17T07:28:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 575.0 s, 621.8 tps, lat 160.526 ms stddev 51.314 2024-04-17T07:29:01.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:00 smithi086 ceph-mon[31733]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.9 MiB/s wr, 648 op/s 2024-04-17T07:29:01.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:00 smithi098 ceph-mon[35418]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.9 MiB/s wr, 648 op/s 2024-04-17T07:29:01.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:00 smithi103 ceph-mon[35495]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.9 MiB/s wr, 648 op/s 2024-04-17T07:29:03.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:02 smithi086 ceph-mon[31733]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 702 op/s 2024-04-17T07:29:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:02 smithi098 ceph-mon[35418]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 702 op/s 2024-04-17T07:29:03.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:02 smithi103 ceph-mon[35495]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 702 op/s 2024-04-17T07:29:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 580.0 s, 569.2 tps, lat 172.808 ms stddev 76.782 2024-04-17T07:29:05.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:04 smithi086 ceph-mon[31733]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.5 MiB/s wr, 680 op/s 2024-04-17T07:29:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:04 smithi098 ceph-mon[35418]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.5 MiB/s wr, 680 op/s 2024-04-17T07:29:05.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:04 smithi103 ceph-mon[35495]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.5 MiB/s wr, 680 op/s 2024-04-17T07:29:07.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:06 smithi086 ceph-mon[31733]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 585 op/s 2024-04-17T07:29:07.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:06 smithi098 ceph-mon[35418]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 585 op/s 2024-04-17T07:29:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:06 smithi103 ceph-mon[35495]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.4 MiB/s wr, 585 op/s 2024-04-17T07:29:09.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:08 smithi086 ceph-mon[31733]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.1 MiB/s wr, 675 op/s 2024-04-17T07:29:09.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:08 smithi098 ceph-mon[35418]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.1 MiB/s wr, 675 op/s 2024-04-17T07:29:09.278 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:08 smithi103 ceph-mon[35495]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.1 MiB/s wr, 675 op/s 2024-04-17T07:29:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 585.0 s, 512.6 tps, lat 198.488 ms stddev 471.920 2024-04-17T07:29:10.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:29:10.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:29:11.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:10 smithi086 ceph-mon[31733]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 659 op/s 2024-04-17T07:29:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:10 smithi098 ceph-mon[35418]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 659 op/s 2024-04-17T07:29:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:10 smithi103 ceph-mon[35495]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.2 MiB/s wr, 659 op/s 2024-04-17T07:29:11.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.303+0000 7f7475000640 1 -- 172.21.15.86:0/3848542377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7470101fa0 msgr2=0x7f74701023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.303+0000 7f7475000640 1 --2- 172.21.15.86:0/3848542377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7470101fa0 0x7f74701023c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7460009be0 tx=0x7f746002bef0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.304+0000 7f7475000640 1 -- 172.21.15.86:0/3848542377 shutdown_connections 2024-04-17T07:29:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.304+0000 7f7475000640 1 --2- 172.21.15.86:0/3848542377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7470103b50 0x7f747010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.304+0000 7f7475000640 1 --2- 172.21.15.86:0/3848542377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7470103190 0x7f7470103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.304+0000 7f7475000640 1 --2- 172.21.15.86:0/3848542377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7470101fa0 0x7f74701023c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.304+0000 7f7475000640 1 -- 172.21.15.86:0/3848542377 >> 172.21.15.86:0/3848542377 conn(0x7f74700fd510 msgr2=0x7f74700ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.305+0000 7f7475000640 1 -- 172.21.15.86:0/3848542377 shutdown_connections 2024-04-17T07:29:11.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.305+0000 7f7475000640 1 -- 172.21.15.86:0/3848542377 wait complete. 2024-04-17T07:29:11.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.307+0000 7f7475000640 1 Processor -- start 2024-04-17T07:29:11.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.308+0000 7f7475000640 1 -- start start 2024-04-17T07:29:11.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.308+0000 7f7475000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7470101fa0 0x7f7470198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.308+0000 7f7475000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7470103190 0x7f74701987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.308+0000 7f7475000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7470103b50 0x7f747019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.308+0000 7f7475000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f747010daf0 con 0x7f7470101fa0 2024-04-17T07:29:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.308+0000 7f7475000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f747010d970 con 0x7f7470103b50 2024-04-17T07:29:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.308+0000 7f7475000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f747010dc70 con 0x7f7470103190 2024-04-17T07:29:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.309+0000 7f746e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7470103b50 0x7f747019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.309+0000 7f746d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7470103190 0x7f74701987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.309+0000 7f746d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7470103190 0x7f74701987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44472/0 (socket says 172.21.15.86:44472) 2024-04-17T07:29:11.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.309+0000 7f746d400640 1 -- 172.21.15.86:0/1523599831 learned_addr learned my addr 172.21.15.86:0/1523599831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.310+0000 7f746de00640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7470101fa0 0x7f7470198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.310+0000 7f746e800640 1 -- 172.21.15.86:0/1523599831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7470103190 msgr2=0x7f74701987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.310+0000 7f746e800640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7470103190 0x7f74701987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.310+0000 7f746e800640 1 -- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7470101fa0 msgr2=0x7f7470198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.310+0000 7f746e800640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7470101fa0 0x7f7470198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.310+0000 7f746e800640 1 -- 172.21.15.86:0/1523599831 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747006bab0 con 0x7f7470103b50 2024-04-17T07:29:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.310+0000 7f746d400640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7470103190 0x7f74701987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.311+0000 7f746de00640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7470101fa0 0x7f7470198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.311+0000 7f746e800640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7470103b50 0x7f747019f7f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7468009a00 tx=0x7f7468009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:11.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.312+0000 7f7466a00640 1 -- 172.21.15.86:0/1523599831 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468004020 con 0x7f7470103b50 2024-04-17T07:29:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.312+0000 7f7466a00640 1 -- 172.21.15.86:0/1523599831 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7468007d10 con 0x7f7470103b50 2024-04-17T07:29:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.312+0000 7f7466a00640 1 -- 172.21.15.86:0/1523599831 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468003cc0 con 0x7f7470103b50 2024-04-17T07:29:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.313+0000 7f7475000640 1 -- 172.21.15.86:0/1523599831 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747006bda0 con 0x7f7470103b50 2024-04-17T07:29:11.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.313+0000 7f7475000640 1 -- 172.21.15.86:0/1523599831 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f747006c2a0 con 0x7f7470103b50 2024-04-17T07:29:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.319+0000 7f7466a00640 1 -- 172.21.15.86:0/1523599831 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7468007860 con 0x7f7470103b50 2024-04-17T07:29:11.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.319+0000 7f7477bf5740 1 Processor -- start 2024-04-17T07:29:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.319+0000 7f7477bf5740 1 -- start start 2024-04-17T07:29:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.320+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f65edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.320+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f663e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.320+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 0x56174f666990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.320+0000 7f7477bf5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56174f5bd120 con 0x56174f65f390 2024-04-17T07:29:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.320+0000 7f7477bf5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56174f563340 con 0x56174f664500 2024-04-17T07:29:11.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.320+0000 7f7477bf5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56174f5b90e0 con 0x56174f65e9a0 2024-04-17T07:29:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.320+0000 7f746d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f65edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.320+0000 7f746de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f663e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.320+0000 7f746d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f65edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44486/0 (socket says 172.21.15.86:44486) 2024-04-17T07:29:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.321+0000 7f746d400640 1 -- 172.21.15.86:0/1552510556 learned_addr learned my addr 172.21.15.86:0/1552510556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:11.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.321+0000 7f746d400640 1 --2- 172.21.15.86:0/1552510556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 0x56174f666990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.321+0000 7f746d400640 1 -- 172.21.15.86:0/1552510556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 msgr2=0x56174f666990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.321+0000 7f746d400640 1 --2- 172.21.15.86:0/1552510556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 0x56174f666990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.321+0000 7f746d400640 1 -- 172.21.15.86:0/1552510556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 msgr2=0x56174f663e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.321+0000 7f746d400640 1 --2- 172.21.15.86:0/1552510556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f663e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.321+0000 7f746d400640 1 -- 172.21.15.86:0/1552510556 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56174f3997d0 con 0x56174f65e9a0 2024-04-17T07:29:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.322+0000 7f746de00640 1 --2- 172.21.15.86:0/1552510556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f663e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:11.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.322+0000 7f746d400640 1 --2- 172.21.15.86:0/1552510556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f65edc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7458011ca0 tx=0x7f7458009c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d69565867ade77 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.322+0000 7f745f400640 1 -- 172.21.15.86:0/1552510556 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745800fc50 con 0x56174f65e9a0 2024-04-17T07:29:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.322+0000 7f745f400640 1 -- 172.21.15.86:0/1552510556 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f745800fdf0 con 0x56174f65e9a0 2024-04-17T07:29:11.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.322+0000 7f745f400640 1 -- 172.21.15.86:0/1552510556 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745800fc50 con 0x56174f65e9a0 2024-04-17T07:29:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.323+0000 7f7466a00640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f74540a3d00 0x7f74540a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.324+0000 7f7477bf5740 1 -- 172.21.15.86:0/1552510556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 msgr2=0x56174f65edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.324+0000 7f7477bf5740 1 --2- 172.21.15.86:0/1552510556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f65edc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7458011ca0 tx=0x7f7458009c10 comp rx=0 tx=0).stop 2024-04-17T07:29:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.324+0000 7f7477bf5740 1 -- 172.21.15.86:0/1552510556 shutdown_connections 2024-04-17T07:29:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.324+0000 7f7477bf5740 1 --2- 172.21.15.86:0/1552510556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 0x56174f666990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.324+0000 7f746de00640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f74540a3d00 0x7f74540a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.324+0000 7f7477bf5740 1 --2- 172.21.15.86:0/1552510556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f663e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.324+0000 7f7477bf5740 1 --2- 172.21.15.86:0/1552510556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f65edc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.324+0000 7f7477bf5740 1 -- 172.21.15.86:0/1552510556 >> 172.21.15.86:0/1552510556 conn(0x56174f659f10 msgr2=0x56174f65a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.324+0000 7f7477bf5740 1 -- 172.21.15.86:0/1552510556 shutdown_connections 2024-04-17T07:29:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.325+0000 7f7477bf5740 1 -- 172.21.15.86:0/1552510556 wait complete. 2024-04-17T07:29:11.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.325+0000 7f7477bf5740 1 -- start start 2024-04-17T07:29:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.326+0000 7f7477bf5740 1 Processor -- start 2024-04-17T07:29:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.327+0000 7f7466a00640 1 -- 172.21.15.86:0/1523599831 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f74680cf840 con 0x7f7470103b50 2024-04-17T07:29:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.327+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f668e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.327+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65f390 0x56174f669390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.327+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f664500 0x56174f6703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.327+0000 7f7477bf5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56174f6674b0 con 0x56174f664500 2024-04-17T07:29:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f7477bf5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56174f576710 con 0x56174f65f390 2024-04-17T07:29:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f7477bf5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56174f667630 con 0x56174f65e9a0 2024-04-17T07:29:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f746de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f668e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f746de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f668e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44502/0 (socket says 172.21.15.86:44502) 2024-04-17T07:29:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f746de00640 1 -- 172.21.15.86:0/3410918977 learned_addr learned my addr 172.21.15.86:0/3410918977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f746d400640 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65f390 0x56174f669390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f746de00640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f74540a3d00 0x7f74540a61c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x56174f65fe00 tx=0x7f746002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f7477bf5740 10 client.0 _ll_register_callbacks cb 0x56174f571af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f746e800640 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f664500 0x56174f6703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f746de00640 1 -- 172.21.15.86:0/3410918977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65f390 msgr2=0x56174f669390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f746de00640 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65f390 0x56174f669390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f746de00640 1 -- 172.21.15.86:0/3410918977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f664500 msgr2=0x56174f6703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.328+0000 7f746de00640 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f664500 0x56174f6703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.330+0000 7f746de00640 1 -- 172.21.15.86:0/3410918977 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174f2cf3f0 con 0x56174f65e9a0 2024-04-17T07:29:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.330+0000 7f746de00640 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f668e50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7460009be0 tx=0x7f746002bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ebe67d6d252db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.330+0000 7f7453e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.331+0000 7f7453e00640 1 -- 172.21.15.86:0/3410918977 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460045070 con 0x56174f65e9a0 2024-04-17T07:29:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.331+0000 7f7477bf5740 1 -- 172.21.15.86:0/3410918977 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745800fdf0 con 0x56174f65e9a0 2024-04-17T07:29:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.331+0000 7f7453e00640 1 -- 172.21.15.86:0/3410918977 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7460004240 con 0x56174f65e9a0 2024-04-17T07:29:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.332+0000 7f7453e00640 1 -- 172.21.15.86:0/3410918977 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7460059050 con 0x56174f65e9a0 2024-04-17T07:29:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.332+0000 7f7477bf5740 10 client.27862 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.332+0000 7f7477bf5740 1 -- 172.21.15.86:0/3410918977 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56174f28f0f0 con 0x56174f65e9a0 2024-04-17T07:29:11.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.333+0000 7f7453e00640 1 -- 172.21.15.86:0/3410918977 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f745800fdf0 con 0x56174f65e9a0 2024-04-17T07:29:11.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.333+0000 7f7453e00640 1 -- 172.21.15.86:0/3410918977 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f746003fb00 con 0x56174f65e9a0 2024-04-17T07:29:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.333+0000 7f7477bf5740 10 client.27862 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.333+0000 7f7477bf5740 10 client.27862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.334+0000 7f7477bf5740 20 client.27862 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.334+0000 7f7477bf5740 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56174f674ca0 0x56174f677160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.334+0000 7f7477bf5740 4 client.27862 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.334+0000 7f7477bf5740 1 -- 172.21.15.86:0/3410918977 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56174f5cab40 con 0x56174f674ca0 2024-04-17T07:29:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.334+0000 7f746d400640 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56174f674ca0 0x56174f677160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.338+0000 7f746d400640 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56174f674ca0 0x56174f677160 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91d00f839e4f0f0 server_cookie=9d552b0f0fa9cde9 in_seq=0 out_seq=0 2024-04-17T07:29:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.339+0000 7f7453e00640 10 client.27862 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.341+0000 7f7453e00640 1 -- 172.21.15.86:0/3410918977 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56174f5cab40 con 0x56174f674ca0 2024-04-17T07:29:11.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.341+0000 7f7453e00640 10 client.27862 handle_command_reply: tid=0 2024-04-17T07:29:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.342+0000 7f7477bf5740 1 client.27862 shutdown 2024-04-17T07:29:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.344+0000 7f7477bf5740 1 -- 172.21.15.86:0/3410918977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 msgr2=0x56174f668e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.344+0000 7f7477bf5740 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f668e50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7460009be0 tx=0x7f746002bef0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.345+0000 7f7477bf5740 1 -- 172.21.15.86:0/3410918977 shutdown_connections 2024-04-17T07:29:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.345+0000 7f7477bf5740 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56174f674ca0 0x56174f677160 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.345+0000 7f7477bf5740 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f664500 0x56174f6703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.345+0000 7f7477bf5740 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65f390 0x56174f669390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.345+0000 7f7477bf5740 1 --2- 172.21.15.86:0/3410918977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f668e50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.345+0000 7f7453e00640 0 client.27862 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.345+0000 7f7477bf5740 1 -- 172.21.15.86:0/3410918977 >> 172.21.15.86:0/3410918977 conn(0x56174f659f10 msgr2=0x56174f650890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.345+0000 7f7477bf5740 1 -- 172.21.15.86:0/3410918977 shutdown_connections 2024-04-17T07:29:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.345+0000 7f7477bf5740 1 -- 172.21.15.86:0/3410918977 wait complete. 2024-04-17T07:29:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.346+0000 7f7477bf5740 20 client.27862 trim_cache size 0 max 16384 2024-04-17T07:29:11.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.379+0000 7f7477bf5740 1 Processor -- start 2024-04-17T07:29:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.380+0000 7f7477bf5740 1 -- start start 2024-04-17T07:29:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.380+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f669170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.380+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f670390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.380+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 0x56174f650000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.380+0000 7f7477bf5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74701b1120 con 0x56174f65f390 2024-04-17T07:29:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.380+0000 7f7477bf5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f74701b2c60 con 0x56174f664500 2024-04-17T07:29:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.380+0000 7f7477bf5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74701b17f0 con 0x56174f65e9a0 2024-04-17T07:29:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.381+0000 7f746e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 0x56174f650000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.381+0000 7f746e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 0x56174f650000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50548/0 (socket says 172.21.15.86:50548) 2024-04-17T07:29:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.381+0000 7f746e800640 1 -- 172.21.15.86:0/1234409768 learned_addr learned my addr 172.21.15.86:0/1234409768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.381+0000 7f746de00640 1 --2- 172.21.15.86:0/1234409768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f669170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.381+0000 7f746d400640 1 --2- 172.21.15.86:0/1234409768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f670390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.382+0000 7f746e800640 1 -- 172.21.15.86:0/1234409768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 msgr2=0x56174f669170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.382+0000 7f746e800640 1 --2- 172.21.15.86:0/1234409768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f669170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.382+0000 7f746e800640 1 -- 172.21.15.86:0/1234409768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 msgr2=0x56174f670390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.382+0000 7f746e800640 1 --2- 172.21.15.86:0/1234409768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f670390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.382+0000 7f746e800640 1 -- 172.21.15.86:0/1234409768 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56174f65fc20 con 0x56174f664500 2024-04-17T07:29:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.382+0000 7f746e800640 1 --2- 172.21.15.86:0/1234409768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 0x56174f650000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f746809c580 tx=0x7f746809fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9792073389e7d667 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.383+0000 7f745ea00640 1 -- 172.21.15.86:0/1234409768 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468003e60 con 0x56174f664500 2024-04-17T07:29:11.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.383+0000 7f745ea00640 1 -- 172.21.15.86:0/1234409768 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74680a02b0 con 0x56174f664500 2024-04-17T07:29:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.383+0000 7f745ea00640 1 -- 172.21.15.86:0/1234409768 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74680041c0 con 0x56174f664500 2024-04-17T07:29:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.384+0000 7f7477bf5740 1 -- 172.21.15.86:0/1234409768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 msgr2=0x56174f650000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.384+0000 7f7477bf5740 1 --2- 172.21.15.86:0/1234409768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 0x56174f650000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f746809c580 tx=0x7f746809fa70 comp rx=0 tx=0).stop 2024-04-17T07:29:11.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.384+0000 7f7477bf5740 1 -- 172.21.15.86:0/1234409768 shutdown_connections 2024-04-17T07:29:11.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.384+0000 7f7477bf5740 1 --2- 172.21.15.86:0/1234409768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f664500 0x56174f650000 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.384+0000 7f7477bf5740 1 --2- 172.21.15.86:0/1234409768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f670390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.384+0000 7f7477bf5740 1 --2- 172.21.15.86:0/1234409768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f65e9a0 0x56174f669170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.384+0000 7f7477bf5740 1 -- 172.21.15.86:0/1234409768 >> 172.21.15.86:0/1234409768 conn(0x56174f659f10 msgr2=0x56174f668a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:11.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.385+0000 7f7477bf5740 1 -- 172.21.15.86:0/1234409768 shutdown_connections 2024-04-17T07:29:11.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.385+0000 7f7477bf5740 1 -- 172.21.15.86:0/1234409768 wait complete. 2024-04-17T07:29:11.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.385+0000 7f7477bf5740 1 -- start start 2024-04-17T07:29:11.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.386+0000 7f7477bf5740 1 Processor -- start 2024-04-17T07:29:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.388+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65e9a0 0x56174f66dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.388+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f66e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.388+0000 7f7477bf5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f664500 0x56174f66f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.388+0000 7f7477bf5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74701b2590 con 0x56174f65f390 2024-04-17T07:29:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.388+0000 7f7477bf5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f74701b1ec0 con 0x56174f65e9a0 2024-04-17T07:29:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.388+0000 7f7477bf5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7470074750 con 0x56174f664500 2024-04-17T07:29:11.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.388+0000 7f7477bf5740 10 client.0 _ll_register_callbacks cb 0x7f746002c980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.390+0000 7f746de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65e9a0 0x56174f66dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.390+0000 7f746d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f66e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.390+0000 7f746de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65e9a0 0x56174f66dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50562/0 (socket says 172.21.15.86:50562) 2024-04-17T07:29:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.390+0000 7f746de00640 1 -- 172.21.15.86:0/2059706484 learned_addr learned my addr 172.21.15.86:0/2059706484 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.391+0000 7f746e800640 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f664500 0x56174f66f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.391+0000 7f746de00640 1 -- 172.21.15.86:0/2059706484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f664500 msgr2=0x56174f66f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:11.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.391+0000 7f746de00640 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f664500 0x56174f66f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.391+0000 7f746de00640 1 -- 172.21.15.86:0/2059706484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 msgr2=0x56174f66e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:11.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.391+0000 7f746de00640 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f66e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.391+0000 7f746de00640 1 -- 172.21.15.86:0/2059706484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56174f66f850 con 0x56174f65e9a0 2024-04-17T07:29:11.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.392+0000 7f746d400640 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f66e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.392+0000 7f746de00640 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65e9a0 0x56174f66dd80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7460004af0 tx=0x7f7460033b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62eeb5273fb4ffff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.392+0000 7f7453e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:29:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.392+0000 7f746e800640 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f664500 0x56174f66f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:29:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.392+0000 7f7453e00640 1 -- 172.21.15.86:0/2059706484 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460045040 con 0x56174f65e9a0 2024-04-17T07:29:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.393+0000 7f7477bf5740 1 -- 172.21.15.86:0/2059706484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74680a02b0 con 0x56174f65e9a0 2024-04-17T07:29:11.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.393+0000 7f7453e00640 1 -- 172.21.15.86:0/2059706484 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f746003f440 con 0x56174f65e9a0 2024-04-17T07:29:11.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.393+0000 7f7453e00640 1 -- 172.21.15.86:0/2059706484 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f74600334d0 con 0x56174f65e9a0 2024-04-17T07:29:11.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.393+0000 7f7477bf5740 10 client.37826 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:11.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.393+0000 7f7477bf5740 1 -- 172.21.15.86:0/2059706484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56174f651400 con 0x56174f65e9a0 2024-04-17T07:29:11.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.394+0000 7f7453e00640 1 -- 172.21.15.86:0/2059706484 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74680a02b0 con 0x56174f65e9a0 2024-04-17T07:29:11.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.394+0000 7f7453e00640 1 -- 172.21.15.86:0/2059706484 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f74600040e0 con 0x56174f65e9a0 2024-04-17T07:29:11.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.394+0000 7f7477bf5740 10 client.37826 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:11.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.394+0000 7f7477bf5740 10 client.37826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:11.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.394+0000 7f7477bf5740 20 client.37826 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:11.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.394+0000 7f7477bf5740 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56174f651d70 0x56174f654190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.394+0000 7f7477bf5740 4 client.37826 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:29:11.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.394+0000 7f7477bf5740 1 -- 172.21.15.86:0/2059706484 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56174f5c49b0 con 0x56174f651d70 2024-04-17T07:29:11.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.395+0000 7f746d400640 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56174f651d70 0x56174f654190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.395+0000 7f746d400640 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56174f651d70 0x56174f654190 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=847c790cfdeef833 server_cookie=f798eb9b52c87d8f in_seq=0 out_seq=0 2024-04-17T07:29:11.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.395+0000 7f7453e00640 10 client.37826 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:11.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.396+0000 7f7453e00640 1 -- 172.21.15.86:0/2059706484 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56174f5c49b0 con 0x56174f651d70 2024-04-17T07:29:11.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.396+0000 7f7453e00640 10 client.37826 handle_command_reply: tid=0 2024-04-17T07:29:11.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.397+0000 7f7477bf5740 1 client.37826 shutdown 2024-04-17T07:29:11.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.397+0000 7f7477bf5740 1 -- 172.21.15.86:0/2059706484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65e9a0 msgr2=0x56174f66dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.397+0000 7f7477bf5740 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65e9a0 0x56174f66dd80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7460004af0 tx=0x7f7460033b80 comp rx=0 tx=0).stop 2024-04-17T07:29:11.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.398+0000 7f7477bf5740 1 -- 172.21.15.86:0/2059706484 shutdown_connections 2024-04-17T07:29:11.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.398+0000 7f7477bf5740 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56174f651d70 0x56174f654190 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.398+0000 7f7477bf5740 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56174f664500 0x56174f66f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.398+0000 7f7477bf5740 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56174f65f390 0x56174f66e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.398+0000 7f7477bf5740 1 --2- 172.21.15.86:0/2059706484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56174f65e9a0 0x56174f66dd80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.398+0000 7f7477bf5740 1 -- 172.21.15.86:0/2059706484 >> 172.21.15.86:0/2059706484 conn(0x56174f659f10 msgr2=0x56174f65d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:11.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.398+0000 7f7453e00640 0 client.37826 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:11.400 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:29:11.401 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:29:11.401 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:29:11.401 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.399+0000 7f7477bf5740 1 -- 172.21.15.86:0/2059706484 shutdown_connections 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.399+0000 7f7477bf5740 1 -- 172.21.15.86:0/2059706484 wait complete. 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.399+0000 7f7477bf5740 20 client.37826 trim_cache size 0 max 16384 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 -- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f74540a3d00 msgr2=0x7f74540a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f74540a3d00 0x7f74540a61c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x56174f65fe00 tx=0x7f746002e040 comp rx=0 tx=0).stop 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 -- 172.21.15.86:0/1523599831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7470103b50 msgr2=0x7f747019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7470103b50 0x7f747019f7f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7468009a00 tx=0x7f7468009ec0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 -- 172.21.15.86:0/1523599831 shutdown_connections 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f74540a3d00 0x7f74540a61c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7470103b50 0x7f747019f7f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7470103190 0x7f74701987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 --2- 172.21.15.86:0/1523599831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7470101fa0 0x7f7470198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 -- 172.21.15.86:0/1523599831 >> 172.21.15.86:0/1523599831 conn(0x7f74700fd510 msgr2=0x7f7470106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:11.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 -- 172.21.15.86:0/1523599831 shutdown_connections 2024-04-17T07:29:11.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.401+0000 7f7475000640 1 -- 172.21.15.86:0/1523599831 wait complete. 2024-04-17T07:29:11.441 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:29:11.441 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:29:11.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.785+0000 7fdfc3c00640 1 -- 172.21.15.86:0/2878055762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfbc103b60 msgr2=0x7fdfbc10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.785+0000 7fdfc3c00640 1 --2- 172.21.15.86:0/2878055762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfbc103b60 0x7fdfbc10a3f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009be0 tx=0x7fdfb402bef0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.785+0000 7fdfc3c00640 1 -- 172.21.15.86:0/2878055762 shutdown_connections 2024-04-17T07:29:11.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.785+0000 7fdfc3c00640 1 --2- 172.21.15.86:0/2878055762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfbc103b60 0x7fdfbc10a3f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.785+0000 7fdfc3c00640 1 --2- 172.21.15.86:0/2878055762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfbc1031a0 0x7fdfbc103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.785+0000 7fdfc3c00640 1 --2- 172.21.15.86:0/2878055762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfbc101fb0 0x7fdfbc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.785+0000 7fdfc3c00640 1 -- 172.21.15.86:0/2878055762 >> 172.21.15.86:0/2878055762 conn(0x7fdfbc0fd540 msgr2=0x7fdfbc0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:11.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.786+0000 7fdfc3c00640 1 -- 172.21.15.86:0/2878055762 shutdown_connections 2024-04-17T07:29:11.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.786+0000 7fdfc3c00640 1 -- 172.21.15.86:0/2878055762 wait complete. 2024-04-17T07:29:11.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.787+0000 7fdfc3c00640 1 Processor -- start 2024-04-17T07:29:11.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.787+0000 7fdfc3c00640 1 -- start start 2024-04-17T07:29:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.787+0000 7fdfc3c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfbc101fb0 0x7fdfbc19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.787+0000 7fdfc3c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfbc1031a0 0x7fdfbc19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.788+0000 7fdfc3c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfbc103b60 0x7fdfbc1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.788+0000 7fdfc3c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc10db00 con 0x7fdfbc103b60 2024-04-17T07:29:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.788+0000 7fdfc3c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc10d980 con 0x7fdfbc101fb0 2024-04-17T07:29:11.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.788+0000 7fdfc3c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc10dc80 con 0x7fdfbc1031a0 2024-04-17T07:29:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.788+0000 7fdfc1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfbc103b60 0x7fdfbc1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.788+0000 7fdfbbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfbc1031a0 0x7fdfbc19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.788+0000 7fdfbbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfbc1031a0 0x7fdfbc19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44544/0 (socket says 172.21.15.86:44544) 2024-04-17T07:29:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.788+0000 7fdfc1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfbc103b60 0x7fdfbc1a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34298/0 (socket says 172.21.15.86:34298) 2024-04-17T07:29:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.788+0000 7fdfbbe00640 1 -- 172.21.15.86:0/3491908443 learned_addr learned my addr 172.21.15.86:0/3491908443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.788+0000 7fdfc1200640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfbc101fb0 0x7fdfbc19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.789+0000 7fdfc1c00640 1 -- 172.21.15.86:0/3491908443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfbc1031a0 msgr2=0x7fdfbc19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.789+0000 7fdfc1c00640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfbc1031a0 0x7fdfbc19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.789+0000 7fdfc1c00640 1 -- 172.21.15.86:0/3491908443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfbc101fb0 msgr2=0x7fdfbc19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.789+0000 7fdfc1c00640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfbc101fb0 0x7fdfbc19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.789+0000 7fdfc1c00640 1 -- 172.21.15.86:0/3491908443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc1a1fb0 con 0x7fdfbc103b60 2024-04-17T07:29:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.789+0000 7fdfc1200640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfbc101fb0 0x7fdfbc19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:29:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.789+0000 7fdfc1c00640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfbc103b60 0x7fdfbc1a1940 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4036040 tx=0x7fdfb4033c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.789+0000 7fdfb9600640 1 -- 172.21.15.86:0/3491908443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4043020 con 0x7fdfbc103b60 2024-04-17T07:29:11.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.790+0000 7fdfb9600640 1 -- 172.21.15.86:0/3491908443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfb4003e20 con 0x7fdfbc103b60 2024-04-17T07:29:11.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.790+0000 7fdfc3c00640 1 -- 172.21.15.86:0/3491908443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc1a2240 con 0x7fdfbc103b60 2024-04-17T07:29:11.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.790+0000 7fdfb9600640 1 -- 172.21.15.86:0/3491908443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4034430 con 0x7fdfbc103b60 2024-04-17T07:29:11.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.790+0000 7fdfc3c00640 1 -- 172.21.15.86:0/3491908443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1a2760 con 0x7fdfbc103b60 2024-04-17T07:29:11.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.792+0000 7fdfc3c00640 1 -- 172.21.15.86:0/3491908443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfbc1a2960 con 0x7fdfbc103b60 2024-04-17T07:29:11.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.801+0000 7fdfb9600640 1 -- 172.21.15.86:0/3491908443 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fdfb4046020 con 0x7fdfbc103b60 2024-04-17T07:29:11.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.807+0000 7fdfb9600640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdfa00a3bb0 0x7fdfa00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.808+0000 7fdfb9600640 1 -- 172.21.15.86:0/3491908443 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdfb40f3630 con 0x7fdfbc103b60 2024-04-17T07:29:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.808+0000 7fdfc1200640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdfa00a3bb0 0x7fdfa00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:11.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.809+0000 7fdfb9600640 1 -- 172.21.15.86:0/3491908443 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdfb404d020 con 0x7fdfbc103b60 2024-04-17T07:29:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:11.810+0000 7fdfc1200640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdfa00a3bb0 0x7fdfa00a6070 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc103000 tx=0x7fdfac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.286+0000 7fdfc3c00640 1 -- 172.21.15.86:0/3491908443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfbc1a2c50 con 0x7fdfbc103b60 2024-04-17T07:29:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.288+0000 7fdfb9600640 1 -- 172.21.15.86:0/3491908443 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fdfb40bfc90 con 0x7fdfbc103b60 2024-04-17T07:29:12.289 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:29:12.289 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:29:12.289 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:29:12.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.292+0000 7fdfc3c00640 1 -- 172.21.15.86:0/3491908443 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdfa00a3bb0 msgr2=0x7fdfa00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:12.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.292+0000 7fdfc3c00640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdfa00a3bb0 0x7fdfa00a6070 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc103000 tx=0x7fdfac009260 comp rx=0 tx=0).stop 2024-04-17T07:29:12.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.292+0000 7fdfc3c00640 1 -- 172.21.15.86:0/3491908443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfbc103b60 msgr2=0x7fdfbc1a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.292+0000 7fdfc3c00640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfbc103b60 0x7fdfbc1a1940 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4036040 tx=0x7fdfb4033c40 comp rx=0 tx=0).stop 2024-04-17T07:29:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.293+0000 7fdfc3c00640 1 -- 172.21.15.86:0/3491908443 shutdown_connections 2024-04-17T07:29:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.293+0000 7fdfc3c00640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdfa00a3bb0 0x7fdfa00a6070 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.293+0000 7fdfc3c00640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfbc103b60 0x7fdfbc1a1940 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.293+0000 7fdfc3c00640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfbc1031a0 0x7fdfbc19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.293+0000 7fdfc3c00640 1 --2- 172.21.15.86:0/3491908443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfbc101fb0 0x7fdfbc19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.293+0000 7fdfc3c00640 1 -- 172.21.15.86:0/3491908443 >> 172.21.15.86:0/3491908443 conn(0x7fdfbc0fd540 msgr2=0x7fdfbc074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.293+0000 7fdfc3c00640 1 -- 172.21.15.86:0/3491908443 shutdown_connections 2024-04-17T07:29:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.293+0000 7fdfc3c00640 1 -- 172.21.15.86:0/3491908443 wait complete. 2024-04-17T07:29:12.335 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:29:12.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.670+0000 7f9e0f400640 1 -- 172.21.15.86:0/3256229671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e081085c0 msgr2=0x7f9e081089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:12.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.670+0000 7f9e0f400640 1 --2- 172.21.15.86:0/3256229671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e081085c0 0x7f9e081089e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009ca0 tx=0x7f9e0002c100 comp rx=0 tx=0).stop 2024-04-17T07:29:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.672+0000 7f9e0f400640 1 -- 172.21.15.86:0/3256229671 shutdown_connections 2024-04-17T07:29:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.672+0000 7f9e0f400640 1 --2- 172.21.15.86:0/3256229671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0810a170 0x7f9e08110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.672+0000 7f9e0f400640 1 --2- 172.21.15.86:0/3256229671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e081097b0 0x7f9e08109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.672+0000 7f9e0f400640 1 --2- 172.21.15.86:0/3256229671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e081085c0 0x7f9e081089e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.672+0000 7f9e0f400640 1 -- 172.21.15.86:0/3256229671 >> 172.21.15.86:0/3256229671 conn(0x7f9e08077560 msgr2=0x7f9e080799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:12.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.672+0000 7f9e0f400640 1 -- 172.21.15.86:0/3256229671 shutdown_connections 2024-04-17T07:29:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.672+0000 7f9e0f400640 1 -- 172.21.15.86:0/3256229671 wait complete. 2024-04-17T07:29:12.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.673+0000 7f9e0f400640 1 Processor -- start 2024-04-17T07:29:12.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.674+0000 7f9e0f400640 1 -- start start 2024-04-17T07:29:12.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.674+0000 7f9e0f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e081085c0 0x7f9e0807f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.674+0000 7f9e0f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e081097b0 0x7f9e0807fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.674+0000 7f9e0f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0810a170 0x7f9e081ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.674+0000 7f9e0f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08114130 con 0x7f9e081097b0 2024-04-17T07:29:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.674+0000 7f9e0f400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08113fb0 con 0x7f9e0810a170 2024-04-17T07:29:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.674+0000 7f9e0f400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081142b0 con 0x7f9e081085c0 2024-04-17T07:29:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.675+0000 7f9e0ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e081085c0 0x7f9e0807f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.675+0000 7f9e0ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e081085c0 0x7f9e0807f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44566/0 (socket says 172.21.15.86:44566) 2024-04-17T07:29:12.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.675+0000 7f9e0ca00640 1 -- 172.21.15.86:0/1288286103 learned_addr learned my addr 172.21.15.86:0/1288286103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:12.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.675+0000 7f9e0d400640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0810a170 0x7f9e081ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.675+0000 7f9e0ca00640 1 -- 172.21.15.86:0/1288286103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0810a170 msgr2=0x7f9e081ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:12.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.675+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0810a170 0x7f9e081ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.675+0000 7f9e0ca00640 1 -- 172.21.15.86:0/1288286103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e081097b0 msgr2=0x7f9e0807fcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T07:29:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.676+0000 7f9e07e00640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e081097b0 0x7f9e0807fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.676+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e081097b0 0x7f9e0807fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.676+0000 7f9e0ca00640 1 -- 172.21.15.86:0/1288286103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e081aed80 con 0x7f9e081085c0 2024-04-17T07:29:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.676+0000 7f9e0d400640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0810a170 0x7f9e081ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:29:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.676+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e081085c0 0x7f9e0807f770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9e00032380 tx=0x7f9e00035d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.676+0000 7f9e05600640 1 -- 172.21.15.86:0/1288286103 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00039040 con 0x7f9e081085c0 2024-04-17T07:29:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.677+0000 7f9e05600640 1 -- 172.21.15.86:0/1288286103 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e00044070 con 0x7f9e081085c0 2024-04-17T07:29:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.677+0000 7f9e0f400640 1 -- 172.21.15.86:0/1288286103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e081aef50 con 0x7f9e081085c0 2024-04-17T07:29:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.677+0000 7f9e05600640 1 -- 172.21.15.86:0/1288286103 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000045a0 con 0x7f9e081085c0 2024-04-17T07:29:12.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.677+0000 7f9e0f400640 1 -- 172.21.15.86:0/1288286103 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e081af300 con 0x7f9e081085c0 2024-04-17T07:29:12.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.682+0000 7f9e1221a740 1 Processor -- start 2024-04-17T07:29:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.683+0000 7f9e1221a740 1 -- start start 2024-04-17T07:29:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.684+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb530fa0 0x55f8eb5313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.684+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb531990 0x55f8eb536490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.684+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb538f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.684+0000 7f9e1221a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb497020 con 0x55f8eb531990 2024-04-17T07:29:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.684+0000 7f9e1221a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb43d460 con 0x55f8eb530fa0 2024-04-17T07:29:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.684+0000 7f9e1221a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb492fe0 con 0x55f8eb536b00 2024-04-17T07:29:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.684+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb530fa0 0x55f8eb5313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.684+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb530fa0 0x55f8eb5313c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50586/0 (socket says 172.21.15.86:50586) 2024-04-17T07:29:12.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.684+0000 7f9e0d400640 1 -- 172.21.15.86:0/1071436929 learned_addr learned my addr 172.21.15.86:0/1071436929 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.685+0000 7f9e07e00640 1 --2- 172.21.15.86:0/1071436929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb538f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.686+0000 7f9e0d400640 1 -- 172.21.15.86:0/1071436929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 msgr2=0x55f8eb538f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.686+0000 7f9e0d400640 1 --2- 172.21.15.86:0/1071436929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb538f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.686+0000 7f9e0d400640 1 -- 172.21.15.86:0/1071436929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb531990 msgr2=0x55f8eb536490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:29:12.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.686+0000 7f9e0d400640 1 --2- 172.21.15.86:0/1071436929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb531990 0x55f8eb536490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.686+0000 7f9e0d400640 1 -- 172.21.15.86:0/1071436929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8eb445040 con 0x55f8eb530fa0 2024-04-17T07:29:12.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.687+0000 7f9e0d400640 1 --2- 172.21.15.86:0/1071436929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb530fa0 0x55f8eb5313c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9df000ac90 tx=0x7f9df000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a91c27d44aadadf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:12.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.688+0000 7f9dfdc00640 1 -- 172.21.15.86:0/1071436929 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0005220 con 0x55f8eb530fa0 2024-04-17T07:29:12.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.688+0000 7f9dfdc00640 1 -- 172.21.15.86:0/1071436929 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9df0004c50 con 0x55f8eb530fa0 2024-04-17T07:29:12.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.688+0000 7f9dfdc00640 1 -- 172.21.15.86:0/1071436929 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df00108b0 con 0x55f8eb530fa0 2024-04-17T07:29:12.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.690+0000 7f9e05600640 1 -- 172.21.15.86:0/1288286103 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9e00003970 con 0x7f9e081085c0 2024-04-17T07:29:12.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.691+0000 7f9e1221a740 1 -- 172.21.15.86:0/1071436929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb530fa0 msgr2=0x55f8eb5313c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:12.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.691+0000 7f9e1221a740 1 --2- 172.21.15.86:0/1071436929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb530fa0 0x55f8eb5313c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9df000ac90 tx=0x7f9df000e720 comp rx=0 tx=0).stop 2024-04-17T07:29:12.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.692+0000 7f9e1221a740 1 -- 172.21.15.86:0/1071436929 shutdown_connections 2024-04-17T07:29:12.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.692+0000 7f9e1221a740 1 --2- 172.21.15.86:0/1071436929 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb538f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.692+0000 7f9e1221a740 1 --2- 172.21.15.86:0/1071436929 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb531990 0x55f8eb536490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.692+0000 7f9e1221a740 1 --2- 172.21.15.86:0/1071436929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb530fa0 0x55f8eb5313c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.692+0000 7f9e1221a740 1 -- 172.21.15.86:0/1071436929 >> 172.21.15.86:0/1071436929 conn(0x55f8eb52c510 msgr2=0x55f8eb52c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:12.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.693+0000 7f9e1221a740 1 -- 172.21.15.86:0/1071436929 shutdown_connections 2024-04-17T07:29:12.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.693+0000 7f9e05600640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9dec0a3ba0 0x7f9dec0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.694+0000 7f9e1221a740 1 -- 172.21.15.86:0/1071436929 wait complete. 2024-04-17T07:29:12.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.694+0000 7f9e05600640 1 -- 172.21.15.86:0/1288286103 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9e000f2ff0 con 0x7f9e081085c0 2024-04-17T07:29:12.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.694+0000 7f9e07e00640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9dec0a3ba0 0x7f9dec0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.695+0000 7f9e07e00640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9dec0a3ba0 0x7f9dec0a6060 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55f8eb52d480 tx=0x7f9df8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:12.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.695+0000 7f9e1221a740 1 -- start start 2024-04-17T07:29:12.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.696+0000 7f9e1221a740 1 Processor -- start 2024-04-17T07:29:12.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.697+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb54c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.697+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb54cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.697+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb553b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.697+0000 7f9e1221a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb54aac0 con 0x55f8eb530fa0 2024-04-17T07:29:12.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.697+0000 7f9e1221a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb450830 con 0x55f8eb531990 2024-04-17T07:29:12.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.697+0000 7f9e1221a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f8eb54ac40 con 0x55f8eb536b00 2024-04-17T07:29:12.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.697+0000 7f9e1221a740 10 client.0 _ll_register_callbacks cb 0x55f8eb41efb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:12.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.698+0000 7f9e07e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb54cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.698+0000 7f9e07e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb54cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50594/0 (socket says 172.21.15.86:50594) 2024-04-17T07:29:12.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.698+0000 7f9e07e00640 1 -- 172.21.15.86:0/4144221162 learned_addr learned my addr 172.21.15.86:0/4144221162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:12.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.699+0000 7f9e0d400640 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb553b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.699+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb54c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.699+0000 7f9e07e00640 1 -- 172.21.15.86:0/4144221162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 msgr2=0x55f8eb553b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:12.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.699+0000 7f9e07e00640 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb553b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.699+0000 7f9e07e00640 1 -- 172.21.15.86:0/4144221162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 msgr2=0x55f8eb54c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:12.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.699+0000 7f9e07e00640 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb54c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.699+0000 7f9e07e00640 1 -- 172.21.15.86:0/4144221162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8eb454920 con 0x55f8eb531990 2024-04-17T07:29:12.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.700+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb54c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:12.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.700+0000 7f9e07e00640 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb54cb00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9df801b9e0 tx=0x7f9df80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44275231526e21ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:12.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.700+0000 7f9df6000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:29:12.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.700+0000 7f9df6000640 1 -- 172.21.15.86:0/4144221162 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df8005220 con 0x55f8eb531990 2024-04-17T07:29:12.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.701+0000 7f9df6000640 1 -- 172.21.15.86:0/4144221162 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9df8005020 con 0x55f8eb531990 2024-04-17T07:29:12.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.701+0000 7f9e1221a740 1 -- 172.21.15.86:0/4144221162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df0004c50 con 0x55f8eb531990 2024-04-17T07:29:12.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.703+0000 7f9df6000640 1 -- 172.21.15.86:0/4144221162 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9df8005220 con 0x55f8eb531990 2024-04-17T07:29:12.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.703+0000 7f9e1221a740 10 client.37841 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:12.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.705+0000 7f9e1221a740 1 -- 172.21.15.86:0/4144221162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f8eb4227f0 con 0x55f8eb531990 2024-04-17T07:29:12.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.706+0000 7f9df6000640 1 -- 172.21.15.86:0/4144221162 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9df0004c50 con 0x55f8eb531990 2024-04-17T07:29:12.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.706+0000 7f9df6000640 1 -- 172.21.15.86:0/4144221162 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9df801c920 con 0x55f8eb531990 2024-04-17T07:29:12.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.708+0000 7f9e1221a740 10 client.37841 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:12.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.708+0000 7f9e1221a740 10 client.37841 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:29:12.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.708+0000 7f9e1221a740 20 client.37841 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:12.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.708+0000 7f9e1221a740 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f8eb557fe0 0x55f8eb55a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.708+0000 7f9e1221a740 4 client.37841 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:12.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.708+0000 7f9e1221a740 1 -- 172.21.15.86:0/4144221162 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8eb4a4a40 con 0x55f8eb557fe0 2024-04-17T07:29:12.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.709+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f8eb557fe0 0x55f8eb55a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.710+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f8eb557fe0 0x55f8eb55a480 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=79c1bf4ad6e671d1 server_cookie=7cbd85700a12aa92 in_seq=0 out_seq=0 2024-04-17T07:29:12.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.710+0000 7f9df6000640 10 client.37841 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:29:12.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.714+0000 7f9df6000640 1 -- 172.21.15.86:0/4144221162 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f8eb4a4a40 con 0x55f8eb557fe0 2024-04-17T07:29:12.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.714+0000 7f9df6000640 10 client.37841 handle_command_reply: tid=0 2024-04-17T07:29:12.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.714+0000 7f9e1221a740 1 client.37841 shutdown 2024-04-17T07:29:12.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.715+0000 7f9e1221a740 1 -- 172.21.15.86:0/4144221162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 msgr2=0x55f8eb54cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:12.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.715+0000 7f9e1221a740 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb54cb00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9df801b9e0 tx=0x7f9df80046c0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.716+0000 7f9e1221a740 1 -- 172.21.15.86:0/4144221162 shutdown_connections 2024-04-17T07:29:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.716+0000 7f9e1221a740 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f8eb557fe0 0x55f8eb55a480 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.716+0000 7f9e1221a740 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb553b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.716+0000 7f9e1221a740 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb54cb00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.716+0000 7f9e1221a740 1 --2- 172.21.15.86:0/4144221162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb54c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.716+0000 7f9df6000640 0 client.37841 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:29:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.716+0000 7f9e1221a740 1 -- 172.21.15.86:0/4144221162 >> 172.21.15.86:0/4144221162 conn(0x55f8eb52c510 msgr2=0x55f8eb52b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.716+0000 7f9e1221a740 1 -- 172.21.15.86:0/4144221162 shutdown_connections 2024-04-17T07:29:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.716+0000 7f9e1221a740 1 -- 172.21.15.86:0/4144221162 wait complete. 2024-04-17T07:29:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.717+0000 7f9e1221a740 20 client.37841 trim_cache size 0 max 16384 2024-04-17T07:29:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.747+0000 7f9e1221a740 1 Processor -- start 2024-04-17T07:29:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.747+0000 7f9e1221a740 1 -- start start 2024-04-17T07:29:12.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.747+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb532fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.747+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb531990 0x55f8eb553b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.747+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb536b00 0x55f8eb5334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.747+0000 7f9e1221a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081b1170 con 0x55f8eb530fa0 2024-04-17T07:29:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.747+0000 7f9e1221a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081b2cb0 con 0x55f8eb536b00 2024-04-17T07:29:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.747+0000 7f9e1221a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081b1840 con 0x55f8eb531990 2024-04-17T07:29:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.748+0000 7f9e0ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb532fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.749+0000 7f9e0ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb532fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34336/0 (socket says 172.21.15.86:34336) 2024-04-17T07:29:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.749+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb536b00 0x55f8eb5334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.749+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb536b00 0x55f8eb5334e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50598/0 (socket says 172.21.15.86:50598) 2024-04-17T07:29:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.749+0000 7f9e0d400640 1 -- 172.21.15.86:0/725986275 learned_addr learned my addr 172.21.15.86:0/725986275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.749+0000 7f9e07e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb531990 0x55f8eb553b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.750+0000 7f9e0ca00640 1 -- 172.21.15.86:0/725986275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb531990 msgr2=0x55f8eb553b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.750+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/725986275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb531990 0x55f8eb553b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.750+0000 7f9e0ca00640 1 -- 172.21.15.86:0/725986275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb536b00 msgr2=0x55f8eb5334e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.750+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/725986275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb536b00 0x55f8eb5334e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.750+0000 7f9e0ca00640 1 -- 172.21.15.86:0/725986275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8eb1a93f0 con 0x55f8eb530fa0 2024-04-17T07:29:12.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.751+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/725986275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb532fa0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x55f8eb532400 tx=0x7f9e00000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbf5a94daab1f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.751+0000 7f9dfd200640 1 -- 172.21.15.86:0/725986275 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000c3300 con 0x55f8eb530fa0 2024-04-17T07:29:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.752+0000 7f9dfd200640 1 -- 172.21.15.86:0/725986275 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e000c7bc0 con 0x55f8eb530fa0 2024-04-17T07:29:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.752+0000 7f9dfd200640 1 -- 172.21.15.86:0/725986275 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e00003c60 con 0x55f8eb530fa0 2024-04-17T07:29:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.753+0000 7f9e1221a740 1 -- 172.21.15.86:0/725986275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 msgr2=0x55f8eb532fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:12.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.753+0000 7f9e1221a740 1 --2- 172.21.15.86:0/725986275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb532fa0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x55f8eb532400 tx=0x7f9e00000f40 comp rx=0 tx=0).stop 2024-04-17T07:29:12.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.754+0000 7f9e1221a740 1 -- 172.21.15.86:0/725986275 shutdown_connections 2024-04-17T07:29:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.754+0000 7f9e1221a740 1 --2- 172.21.15.86:0/725986275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb536b00 0x55f8eb5334e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.754+0000 7f9e1221a740 1 --2- 172.21.15.86:0/725986275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb531990 0x55f8eb553b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.754+0000 7f9e1221a740 1 --2- 172.21.15.86:0/725986275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb532fa0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.754+0000 7f9e1221a740 1 -- 172.21.15.86:0/725986275 >> 172.21.15.86:0/725986275 conn(0x55f8eb52c510 msgr2=0x55f8eb54b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.755+0000 7f9e1221a740 1 -- 172.21.15.86:0/725986275 shutdown_connections 2024-04-17T07:29:12.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.755+0000 7f9e1221a740 1 -- 172.21.15.86:0/725986275 wait complete. 2024-04-17T07:29:12.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.756+0000 7f9e1221a740 1 -- start start 2024-04-17T07:29:12.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.756+0000 7f9e1221a740 1 Processor -- start 2024-04-17T07:29:12.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.757+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb550be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.757+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb551120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.757+0000 7f9e1221a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb552130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.758+0000 7f9e1221a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081b25e0 con 0x55f8eb530fa0 2024-04-17T07:29:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.758+0000 7f9e1221a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081b1f10 con 0x55f8eb531990 2024-04-17T07:29:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.758+0000 7f9e1221a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08073d30 con 0x55f8eb536b00 2024-04-17T07:29:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.758+0000 7f9e1221a740 10 client.0 _ll_register_callbacks cb 0x7f9df801b660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.758+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb552130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.758+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb552130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44592/0 (socket says 172.21.15.86:44592) 2024-04-17T07:29:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.758+0000 7f9e0d400640 1 -- 172.21.15.86:0/3692246715 learned_addr learned my addr 172.21.15.86:0/3692246715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.759+0000 7f9e07e00640 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb551120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.759+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb550be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.759+0000 7f9e0d400640 1 -- 172.21.15.86:0/3692246715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 msgr2=0x55f8eb551120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:12.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.759+0000 7f9e0d400640 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb551120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.759+0000 7f9e0d400640 1 -- 172.21.15.86:0/3692246715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 msgr2=0x55f8eb550be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.759+0000 7f9e0d400640 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb550be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.759+0000 7f9e0d400640 1 -- 172.21.15.86:0/3692246715 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8eb2737d0 con 0x55f8eb536b00 2024-04-17T07:29:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.760+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb550be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.760+0000 7f9df6000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.760+0000 7f9e0d400640 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb552130 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9df0006010 tx=0x7f9df0005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5652a5833ed54ef server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:12.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.761+0000 7f9df6000640 1 -- 172.21.15.86:0/3692246715 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0004020 con 0x55f8eb536b00 2024-04-17T07:29:12.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.762+0000 7f9e07e00640 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb551120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:12.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.762+0000 7f9df6000640 1 -- 172.21.15.86:0/3692246715 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9df00041c0 con 0x55f8eb536b00 2024-04-17T07:29:12.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.762+0000 7f9e1221a740 1 -- 172.21.15.86:0/3692246715 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e000c7bc0 con 0x55f8eb536b00 2024-04-17T07:29:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.763+0000 7f9df6000640 1 -- 172.21.15.86:0/3692246715 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9df00043c0 con 0x55f8eb536b00 2024-04-17T07:29:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.763+0000 7f9e1221a740 10 client.27892 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:12.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.763+0000 7f9e1221a740 1 -- 172.21.15.86:0/3692246715 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f8eb5538c0 con 0x55f8eb536b00 2024-04-17T07:29:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.764+0000 7f9df6000640 1 -- 172.21.15.86:0/3692246715 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e000c7bc0 con 0x55f8eb536b00 2024-04-17T07:29:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.764+0000 7f9df6000640 1 -- 172.21.15.86:0/3692246715 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9df00037f0 con 0x55f8eb536b00 2024-04-17T07:29:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.764+0000 7f9e1221a740 10 client.27892 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.764+0000 7f9e1221a740 10 client.27892 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:29:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.764+0000 7f9e1221a740 20 client.27892 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.764+0000 7f9e1221a740 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f8eb5be630 0x55f8eb5c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:12.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.764+0000 7f9e1221a740 4 client.27892 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:29:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.764+0000 7f9e1221a740 1 -- 172.21.15.86:0/3692246715 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8eb49e8b0 con 0x55f8eb5be630 2024-04-17T07:29:12.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.764+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f8eb5be630 0x55f8eb5c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.766+0000 7f9e0ca00640 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f8eb5be630 0x55f8eb5c0a20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a21a4b1250069cab server_cookie=8569af87a561680f in_seq=0 out_seq=0 2024-04-17T07:29:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.766+0000 7f9df6000640 10 client.27892 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:29:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.766+0000 7f9df6000640 1 -- 172.21.15.86:0/3692246715 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8eb49e8b0 con 0x55f8eb5be630 2024-04-17T07:29:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.766+0000 7f9df6000640 10 client.27892 handle_command_reply: tid=0 2024-04-17T07:29:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.766+0000 7f9e1221a740 1 client.27892 shutdown 2024-04-17T07:29:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.767+0000 7f9e1221a740 1 -- 172.21.15.86:0/3692246715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 msgr2=0x55f8eb552130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.767+0000 7f9e1221a740 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb552130 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9df0006010 tx=0x7f9df0005770 comp rx=0 tx=0).stop 2024-04-17T07:29:12.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.767+0000 7f9e1221a740 1 -- 172.21.15.86:0/3692246715 shutdown_connections 2024-04-17T07:29:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.767+0000 7f9e1221a740 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f8eb5be630 0x55f8eb5c0a20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.767+0000 7f9e1221a740 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8eb536b00 0x55f8eb552130 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.767+0000 7f9e1221a740 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f8eb531990 0x55f8eb551120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.767+0000 7f9df6000640 0 client.27892 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:29:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.767+0000 7f9e1221a740 1 --2- 172.21.15.86:0/3692246715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8eb530fa0 0x55f8eb550be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.767+0000 7f9e1221a740 1 -- 172.21.15.86:0/3692246715 >> 172.21.15.86:0/3692246715 conn(0x55f8eb52c510 msgr2=0x55f8eb532fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.768+0000 7f9e1221a740 1 -- 172.21.15.86:0/3692246715 shutdown_connections 2024-04-17T07:29:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.768+0000 7f9e1221a740 1 -- 172.21.15.86:0/3692246715 wait complete. 2024-04-17T07:29:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.768+0000 7f9e1221a740 20 client.27892 trim_cache size 0 max 16384 2024-04-17T07:29:12.770 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:29:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.770+0000 7f9e0f400640 1 -- 172.21.15.86:0/1288286103 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9dec0a3ba0 msgr2=0x7f9dec0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.770+0000 7f9e0f400640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9dec0a3ba0 0x7f9dec0a6060 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55f8eb52d480 tx=0x7f9df8005f90 comp rx=0 tx=0).stop 2024-04-17T07:29:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.770+0000 7f9e0f400640 1 -- 172.21.15.86:0/1288286103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e081085c0 msgr2=0x7f9e0807f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.770+0000 7f9e0f400640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e081085c0 0x7f9e0807f770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9e00032380 tx=0x7f9e00035d50 comp rx=0 tx=0).stop 2024-04-17T07:29:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.771+0000 7f9e0f400640 1 -- 172.21.15.86:0/1288286103 shutdown_connections 2024-04-17T07:29:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.771+0000 7f9e0f400640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9dec0a3ba0 0x7f9dec0a6060 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.771+0000 7f9e0f400640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0810a170 0x7f9e081ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.771+0000 7f9e0f400640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e081097b0 0x7f9e0807fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.771+0000 7f9e0f400640 1 --2- 172.21.15.86:0/1288286103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e081085c0 0x7f9e0807f770 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.771+0000 7f9e0f400640 1 -- 172.21.15.86:0/1288286103 >> 172.21.15.86:0/1288286103 conn(0x7f9e08077560 msgr2=0x7f9e0810cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.771+0000 7f9e0f400640 1 -- 172.21.15.86:0/1288286103 shutdown_connections 2024-04-17T07:29:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:12.771+0000 7f9e0f400640 1 -- 172.21.15.86:0/1288286103 wait complete. 2024-04-17T07:29:12.818 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:29:13.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:12 smithi086 ceph-mon[31733]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-04-17T07:29:13.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3491908443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:29:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.186+0000 7faedfc00640 1 -- 172.21.15.86:0/4139340030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed8105d70 msgr2=0x7faed810c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.186+0000 7faedfc00640 1 --2- 172.21.15.86:0/4139340030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed8105d70 0x7faed810c600 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7faed0009ca0 tx=0x7faed002c100 comp rx=0 tx=0).stop 2024-04-17T07:29:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.187+0000 7faedfc00640 1 -- 172.21.15.86:0/4139340030 shutdown_connections 2024-04-17T07:29:13.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.187+0000 7faedfc00640 1 --2- 172.21.15.86:0/4139340030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed8105d70 0x7faed810c600 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.187+0000 7faedfc00640 1 --2- 172.21.15.86:0/4139340030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed81053b0 0x7faed8105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.187+0000 7faedfc00640 1 --2- 172.21.15.86:0/4139340030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed81041c0 0x7faed81045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.187+0000 7faedfc00640 1 -- 172.21.15.86:0/4139340030 >> 172.21.15.86:0/4139340030 conn(0x7faed80ff710 msgr2=0x7faed8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.187+0000 7faedfc00640 1 -- 172.21.15.86:0/4139340030 shutdown_connections 2024-04-17T07:29:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.187+0000 7faedfc00640 1 -- 172.21.15.86:0/4139340030 wait complete. 2024-04-17T07:29:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.188+0000 7faedfc00640 1 Processor -- start 2024-04-17T07:29:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.189+0000 7faedfc00640 1 -- start start 2024-04-17T07:29:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.189+0000 7faedfc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed81041c0 0x7faed80708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.189+0000 7faedfc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed81053b0 0x7faed8070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.189+0000 7faedfc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed8105d70 0x7faed81aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.189+0000 7faedfc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faed810fd00 con 0x7faed81053b0 2024-04-17T07:29:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.189+0000 7faedfc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faed810fb80 con 0x7faed81041c0 2024-04-17T07:29:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.189+0000 7faedfc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faed810fe80 con 0x7faed8105d70 2024-04-17T07:29:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.190+0000 7faedd200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed81041c0 0x7faed80708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.190+0000 7faedd200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed81041c0 0x7faed80708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50628/0 (socket says 172.21.15.86:50628) 2024-04-17T07:29:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.190+0000 7faedd200640 1 -- 172.21.15.86:0/4194601116 learned_addr learned my addr 172.21.15.86:0/4194601116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.190+0000 7faed7e00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed81053b0 0x7faed8070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.190+0000 7faeddc00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed8105d70 0x7faed81aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.190+0000 7faed7e00640 1 -- 172.21.15.86:0/4194601116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed8105d70 msgr2=0x7faed81aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.190+0000 7faed7e00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed8105d70 0x7faed81aa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.190+0000 7faed7e00640 1 -- 172.21.15.86:0/4194601116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed81041c0 msgr2=0x7faed80708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.190+0000 7faed7e00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed81041c0 0x7faed80708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.190+0000 7faed7e00640 1 -- 172.21.15.86:0/4194601116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed8071490 con 0x7faed81053b0 2024-04-17T07:29:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.191+0000 7faedd200640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed81041c0 0x7faed80708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.191+0000 7faed7e00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed81053b0 0x7faed8070df0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7faec400c970 tx=0x7faec400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.192+0000 7faed5600640 1 -- 172.21.15.86:0/4194601116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4007bb0 con 0x7faed81053b0 2024-04-17T07:29:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.192+0000 7faed5600640 1 -- 172.21.15.86:0/4194601116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec4007d50 con 0x7faed81053b0 2024-04-17T07:29:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.192+0000 7faed5600640 1 -- 172.21.15.86:0/4194601116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4018780 con 0x7faed81053b0 2024-04-17T07:29:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.192+0000 7faedfc00640 1 -- 172.21.15.86:0/4194601116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed81aaa50 con 0x7faed81053b0 2024-04-17T07:29:13.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.192+0000 7faedfc00640 1 -- 172.21.15.86:0/4194601116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed81aaf70 con 0x7faed81053b0 2024-04-17T07:29:13.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.199+0000 7faed5600640 1 -- 172.21.15.86:0/4194601116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7faec400f450 con 0x7faed81053b0 2024-04-17T07:29:13.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.200+0000 7faee29d8740 1 Processor -- start 2024-04-17T07:29:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.202+0000 7faed5600640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faec00a3c80 0x7faec00a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.203+0000 7faed5600640 1 -- 172.21.15.86:0/4194601116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faec40d0750 con 0x7faed81053b0 2024-04-17T07:29:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.203+0000 7faee29d8740 1 -- start start 2024-04-17T07:29:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.203+0000 7faee29d8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 0x56407106cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.204+0000 7faee29d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x564071071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.204+0000 7faee29d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 0x5640710747a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.204+0000 7faee29d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564070fca020 con 0x564071072310 2024-04-17T07:29:13.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.204+0000 7faee29d8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564070f70460 con 0x56407106c7b0 2024-04-17T07:29:13.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.204+0000 7faee29d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564070fc5fe0 con 0x56407106d1a0 2024-04-17T07:29:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.204+0000 7faeddc00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faec00a3c80 0x7faec00a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.204+0000 7faedd200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 0x56407106cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.205+0000 7faedd200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 0x56407106cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50632/0 (socket says 172.21.15.86:50632) 2024-04-17T07:29:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.205+0000 7faedd200640 1 -- 172.21.15.86:0/2885248648 learned_addr learned my addr 172.21.15.86:0/2885248648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.205+0000 7faeddc00640 1 --2- 172.21.15.86:0/2885248648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 0x5640710747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.205+0000 7faedd200640 1 -- 172.21.15.86:0/2885248648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 msgr2=0x564071071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.205+0000 7faedd200640 1 --2- 172.21.15.86:0/2885248648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x564071071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.205+0000 7faedd200640 1 -- 172.21.15.86:0/2885248648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 msgr2=0x5640710747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.205+0000 7faedd200640 1 --2- 172.21.15.86:0/2885248648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 0x5640710747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.206+0000 7faedd200640 1 -- 172.21.15.86:0/2885248648 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564070f78040 con 0x56407106c7b0 2024-04-17T07:29:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.206+0000 7faeddc00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faec00a3c80 0x7faec00a6140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faed0032380 tx=0x7faed00326f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.206+0000 7faedd200640 1 --2- 172.21.15.86:0/2885248648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 0x56407106cbd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faecc00f470 tx=0x7faecc00f930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad798691550682f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.207+0000 7faeca000640 1 -- 172.21.15.86:0/2885248648 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc010960 con 0x56407106c7b0 2024-04-17T07:29:13.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.208+0000 7faeca000640 1 -- 172.21.15.86:0/2885248648 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faecc0183f0 con 0x56407106c7b0 2024-04-17T07:29:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.208+0000 7faeca000640 1 -- 172.21.15.86:0/2885248648 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc00e610 con 0x56407106c7b0 2024-04-17T07:29:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.209+0000 7faee29d8740 1 -- 172.21.15.86:0/2885248648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 msgr2=0x56407106cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.209+0000 7faee29d8740 1 --2- 172.21.15.86:0/2885248648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 0x56407106cbd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faecc00f470 tx=0x7faecc00f930 comp rx=0 tx=0).stop 2024-04-17T07:29:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.209+0000 7faee29d8740 1 -- 172.21.15.86:0/2885248648 shutdown_connections 2024-04-17T07:29:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.209+0000 7faee29d8740 1 --2- 172.21.15.86:0/2885248648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 0x5640710747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.209+0000 7faee29d8740 1 --2- 172.21.15.86:0/2885248648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x564071071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.209+0000 7faee29d8740 1 --2- 172.21.15.86:0/2885248648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 0x56407106cbd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.209+0000 7faee29d8740 1 -- 172.21.15.86:0/2885248648 >> 172.21.15.86:0/2885248648 conn(0x56407105f510 msgr2=0x56407105f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.210+0000 7faee29d8740 1 -- 172.21.15.86:0/2885248648 shutdown_connections 2024-04-17T07:29:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.210+0000 7faee29d8740 1 -- 172.21.15.86:0/2885248648 wait complete. 2024-04-17T07:29:13.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.211+0000 7faee29d8740 1 -- start start 2024-04-17T07:29:13.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.211+0000 7faee29d8740 1 Processor -- start 2024-04-17T07:29:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.212+0000 7faee29d8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 0x564071065ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.213+0000 7faee29d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x564071066220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.213+0000 7faee29d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 0x56407106b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.213+0000 7faee29d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564071064180 con 0x564071072310 2024-04-17T07:29:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.213+0000 7faee29d8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564070f83830 con 0x56407106c7b0 2024-04-17T07:29:13.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.213+0000 7faee29d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564071064300 con 0x56407106d1a0 2024-04-17T07:29:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.213+0000 7faee29d8740 10 client.0 _ll_register_callbacks cb 0x564070f51fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.214+0000 7faed7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x564071066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.214+0000 7faed7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x564071066220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44626/0 (socket says 172.21.15.86:44626) 2024-04-17T07:29:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.214+0000 7faed7e00640 1 -- 172.21.15.86:0/3808805717 learned_addr learned my addr 172.21.15.86:0/3808805717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.214+0000 7faedd200640 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 0x564071065ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.215+0000 7faeddc00640 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 0x56407106b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.215+0000 7faed7e00640 1 -- 172.21.15.86:0/3808805717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 msgr2=0x564071065ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.215+0000 7faed7e00640 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 0x564071065ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.215+0000 7faed7e00640 1 -- 172.21.15.86:0/3808805717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 msgr2=0x56407106b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.215+0000 7faed7e00640 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 0x56407106b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.215+0000 7faed7e00640 1 -- 172.21.15.86:0/3808805717 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564070f87920 con 0x56407106d1a0 2024-04-17T07:29:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.215+0000 7faeddc00640 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 0x56407106b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.216+0000 7faed7e00640 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x564071066220 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faec4098a80 tx=0x7faec409c0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d1b1f4d4aed849 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.216+0000 7faebea00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:13.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.216+0000 7faebea00640 1 -- 172.21.15.86:0/3808805717 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec40a5090 con 0x56407106d1a0 2024-04-17T07:29:13.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.216+0000 7faebea00640 1 -- 172.21.15.86:0/3808805717 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faec4085bb0 con 0x56407106d1a0 2024-04-17T07:29:13.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.216+0000 7faee29d8740 1 -- 172.21.15.86:0/3808805717 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faecc0183f0 con 0x56407106d1a0 2024-04-17T07:29:13.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.217+0000 7faebea00640 1 -- 172.21.15.86:0/3808805717 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faec409cdb0 con 0x56407106d1a0 2024-04-17T07:29:13.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.217+0000 7faee29d8740 10 client.27898 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:13.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.217+0000 7faee29d8740 1 -- 172.21.15.86:0/3808805717 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564070f557f0 con 0x56407106d1a0 2024-04-17T07:29:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.217+0000 7faebea00640 1 -- 172.21.15.86:0/3808805717 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faecc0183f0 con 0x56407106d1a0 2024-04-17T07:29:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.217+0000 7faebea00640 1 -- 172.21.15.86:0/3808805717 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7faec4098370 con 0x56407106d1a0 2024-04-17T07:29:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.218+0000 7faee29d8740 10 client.27898 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.218+0000 7faee29d8740 10 client.27898 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:29:13.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.218+0000 7faee29d8740 20 client.27898 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.218+0000 7faee29d8740 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56407108afe0 0x56407108d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.218+0000 7faee29d8740 4 client.27898 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:13.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.218+0000 7faee29d8740 1 -- 172.21.15.86:0/3808805717 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564070fd7a40 con 0x56407108afe0 2024-04-17T07:29:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.220+0000 7faedd200640 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56407108afe0 0x56407108d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.221+0000 7faebea00640 10 client.27898 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:29:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.221+0000 7faedd200640 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56407108afe0 0x56407108d400 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8095f01f65399566 server_cookie=779b7f1fa501e22f in_seq=0 out_seq=0 2024-04-17T07:29:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.223+0000 7faebea00640 1 -- 172.21.15.86:0/3808805717 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564070fd7a40 con 0x56407108afe0 2024-04-17T07:29:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.223+0000 7faebea00640 10 client.27898 handle_command_reply: tid=0 2024-04-17T07:29:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.223+0000 7faee29d8740 1 client.27898 shutdown 2024-04-17T07:29:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.224+0000 7faee29d8740 1 -- 172.21.15.86:0/3808805717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 msgr2=0x564071066220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.224+0000 7faee29d8740 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x564071066220 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faec4098a80 tx=0x7faec409c0d0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.224+0000 7faee29d8740 1 -- 172.21.15.86:0/3808805717 shutdown_connections 2024-04-17T07:29:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.224+0000 7faee29d8740 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56407108afe0 0x56407108d400 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.224+0000 7faee29d8740 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564071072310 0x56407106b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.224+0000 7faee29d8740 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x564071066220 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.224+0000 7faee29d8740 1 --2- 172.21.15.86:0/3808805717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56407106c7b0 0x564071065ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.224+0000 7faee29d8740 1 -- 172.21.15.86:0/3808805717 >> 172.21.15.86:0/3808805717 conn(0x56407105f510 msgr2=0x56407105e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.225+0000 7faebea00640 0 client.27898 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:29:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.225+0000 7faee29d8740 1 -- 172.21.15.86:0/3808805717 shutdown_connections 2024-04-17T07:29:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.225+0000 7faee29d8740 1 -- 172.21.15.86:0/3808805717 wait complete. 2024-04-17T07:29:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.225+0000 7faee29d8740 20 client.27898 trim_cache size 0 max 16384 2024-04-17T07:29:13.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:12 smithi098 ceph-mon[35418]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-04-17T07:29:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3491908443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:29:13.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.267+0000 7faee29d8740 1 Processor -- start 2024-04-17T07:29:13.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.267+0000 7faee29d8740 1 -- start start 2024-04-17T07:29:13.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.268+0000 7faee29d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407106c7b0 0x56407106e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.268+0000 7faee29d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407106ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.268+0000 7faee29d8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564071072310 0x56407105de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.268+0000 7faee29d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faed81b0fc0 con 0x56407106c7b0 2024-04-17T07:29:13.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.268+0000 7faee29d8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faed81b2b00 con 0x564071072310 2024-04-17T07:29:13.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.268+0000 7faee29d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faed81b1690 con 0x56407106d1a0 2024-04-17T07:29:13.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.269+0000 7faed7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407106ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.269+0000 7faed7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407106ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44630/0 (socket says 172.21.15.86:44630) 2024-04-17T07:29:13.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.269+0000 7faed7e00640 1 -- 172.21.15.86:0/4042389705 learned_addr learned my addr 172.21.15.86:0/4042389705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:13.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.269+0000 7faeddc00640 1 --2- 172.21.15.86:0/4042389705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564071072310 0x56407105de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.269+0000 7faed7e00640 1 -- 172.21.15.86:0/4042389705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564071072310 msgr2=0x56407105de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.269+0000 7faed7e00640 1 --2- 172.21.15.86:0/4042389705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564071072310 0x56407105de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.269+0000 7faed7e00640 1 -- 172.21.15.86:0/4042389705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407106c7b0 msgr2=0x56407106e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.269+0000 7faed7e00640 1 --2- 172.21.15.86:0/4042389705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407106c7b0 0x56407106e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.269+0000 7faed7e00640 1 -- 172.21.15.86:0/4042389705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564070cdc3f0 con 0x56407106d1a0 2024-04-17T07:29:13.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.270+0000 7faed7e00640 1 --2- 172.21.15.86:0/4042389705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407106ed90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faec400fe40 tx=0x7faec409a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a7f66ebda80ee3d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.271+0000 7faec9600640 1 -- 172.21.15.86:0/4042389705 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec40a0a60 con 0x56407106d1a0 2024-04-17T07:29:13.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.271+0000 7faec9600640 1 -- 172.21.15.86:0/4042389705 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec40a5320 con 0x56407106d1a0 2024-04-17T07:29:13.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.271+0000 7faec9600640 1 -- 172.21.15.86:0/4042389705 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4085bb0 con 0x56407106d1a0 2024-04-17T07:29:13.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.272+0000 7faee29d8740 1 -- 172.21.15.86:0/4042389705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 msgr2=0x56407106ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.272+0000 7faee29d8740 1 --2- 172.21.15.86:0/4042389705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407106ed90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faec400fe40 tx=0x7faec409a6c0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.272+0000 7faee29d8740 1 -- 172.21.15.86:0/4042389705 shutdown_connections 2024-04-17T07:29:13.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.272+0000 7faee29d8740 1 --2- 172.21.15.86:0/4042389705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564071072310 0x56407105de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.272+0000 7faee29d8740 1 --2- 172.21.15.86:0/4042389705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407106ed90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.272+0000 7faee29d8740 1 --2- 172.21.15.86:0/4042389705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407106c7b0 0x56407106e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.273+0000 7faee29d8740 1 -- 172.21.15.86:0/4042389705 >> 172.21.15.86:0/4042389705 conn(0x56407105f510 msgr2=0x564071065ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.273+0000 7faee29d8740 1 -- 172.21.15.86:0/4042389705 shutdown_connections 2024-04-17T07:29:13.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.273+0000 7faee29d8740 1 -- 172.21.15.86:0/4042389705 wait complete. 2024-04-17T07:29:13.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.274+0000 7faee29d8740 1 -- start start 2024-04-17T07:29:13.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.275+0000 7faee29d8740 1 Processor -- start 2024-04-17T07:29:13.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.275+0000 7faee29d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407106c7b0 0x56407109bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.275+0000 7faee29d8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407109c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.275+0000 7faee29d8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564071072310 0x56407109d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.275+0000 7faee29d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faed81b2430 con 0x56407106c7b0 2024-04-17T07:29:13.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.275+0000 7faee29d8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faed81b1d60 con 0x564071072310 2024-04-17T07:29:13.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.275+0000 7faee29d8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faed81aef90 con 0x56407106d1a0 2024-04-17T07:29:13.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.275+0000 7faee29d8740 10 client.0 _ll_register_callbacks cb 0x7faec409c1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:13.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:12 smithi103 ceph-mon[35495]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-04-17T07:29:13.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3491908443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:29:13.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.276+0000 7faed7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407109c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.276+0000 7faed7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407109c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44638/0 (socket says 172.21.15.86:44638) 2024-04-17T07:29:13.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.276+0000 7faed7e00640 1 -- 172.21.15.86:0/3899409591 learned_addr learned my addr 172.21.15.86:0/3899409591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:13.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.276+0000 7faedd200640 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407106c7b0 0x56407109bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.276+0000 7faed7e00640 1 -- 172.21.15.86:0/3899409591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564071072310 msgr2=0x56407109d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:29:13.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.276+0000 7faed7e00640 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564071072310 0x56407109d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.277+0000 7faed7e00640 1 -- 172.21.15.86:0/3899409591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407106c7b0 msgr2=0x56407109bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.277+0000 7faed7e00640 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407106c7b0 0x56407109bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.277+0000 7faed7e00640 1 -- 172.21.15.86:0/3899409591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564070da67d0 con 0x56407106d1a0 2024-04-17T07:29:13.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.277+0000 7faed7e00640 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407109c3e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faec4000c00 tx=0x7faec4086dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea182a4a08db7f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.277+0000 7faebea00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:13.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.278+0000 7faebea00640 1 -- 172.21.15.86:0/3899409591 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec409c6b0 con 0x56407106d1a0 2024-04-17T07:29:13.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.278+0000 7faebea00640 1 -- 172.21.15.86:0/3899409591 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7faec409b570 con 0x56407106d1a0 2024-04-17T07:29:13.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.278+0000 7faee29d8740 1 -- 172.21.15.86:0/3899409591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec40a5320 con 0x56407106d1a0 2024-04-17T07:29:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.279+0000 7faebea00640 1 -- 172.21.15.86:0/3899409591 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faec4098370 con 0x56407106d1a0 2024-04-17T07:29:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.279+0000 7faee29d8740 10 client.27904 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.279+0000 7faee29d8740 1 -- 172.21.15.86:0/3899409591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564070da67d0 con 0x56407106d1a0 2024-04-17T07:29:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.279+0000 7faebea00640 1 -- 172.21.15.86:0/3899409591 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec40a5320 con 0x56407106d1a0 2024-04-17T07:29:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.279+0000 7faebea00640 1 -- 172.21.15.86:0/3899409591 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7faec409dad0 con 0x56407106d1a0 2024-04-17T07:29:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.280+0000 7faee29d8740 10 client.27904 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.280+0000 7faee29d8740 10 client.27904 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:29:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.280+0000 7faee29d8740 20 client.27904 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.280+0000 7faee29d8740 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5640710f13e0 0x5640710f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.280+0000 7faee29d8740 4 client.27904 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:29:13.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.280+0000 7faee29d8740 1 -- 172.21.15.86:0/3899409591 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564070fd18b0 con 0x5640710f13e0 2024-04-17T07:29:13.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.280+0000 7faedd200640 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5640710f13e0 0x5640710f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.281+0000 7faebea00640 10 client.27904 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:29:13.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.281+0000 7faedd200640 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5640710f13e0 0x5640710f37d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef6edcb0f0fe25e6 server_cookie=4843b36a08626b5d in_seq=0 out_seq=0 2024-04-17T07:29:13.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faebea00640 1 -- 172.21.15.86:0/3899409591 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564070fd18b0 con 0x5640710f13e0 2024-04-17T07:29:13.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faebea00640 10 client.27904 handle_command_reply: tid=0 2024-04-17T07:29:13.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faee29d8740 1 client.27904 shutdown 2024-04-17T07:29:13.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faee29d8740 1 -- 172.21.15.86:0/3899409591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 msgr2=0x56407109c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faee29d8740 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407109c3e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faec4000c00 tx=0x7faec4086dd0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faee29d8740 1 -- 172.21.15.86:0/3899409591 shutdown_connections 2024-04-17T07:29:13.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faee29d8740 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5640710f13e0 0x5640710f37d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faee29d8740 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564071072310 0x56407109d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faebea00640 0 client.27904 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:29:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faee29d8740 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56407106d1a0 0x56407109c3e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.282+0000 7faee29d8740 1 --2- 172.21.15.86:0/3899409591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56407106c7b0 0x56407109bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.283+0000 7faee29d8740 1 -- 172.21.15.86:0/3899409591 >> 172.21.15.86:0/3899409591 conn(0x56407105f510 msgr2=0x564071063000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.283+0000 7faee29d8740 1 -- 172.21.15.86:0/3899409591 shutdown_connections 2024-04-17T07:29:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.283+0000 7faee29d8740 1 -- 172.21.15.86:0/3899409591 wait complete. 2024-04-17T07:29:13.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.283+0000 7faee29d8740 20 client.27904 trim_cache size 0 max 16384 2024-04-17T07:29:13.285 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:29:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.284+0000 7faedfc00640 1 -- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faec00a3c80 msgr2=0x7faec00a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.284+0000 7faedfc00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faec00a3c80 0x7faec00a6140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faed0032380 tx=0x7faed00326f0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.284+0000 7faedfc00640 1 -- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed81053b0 msgr2=0x7faed8070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.284+0000 7faedfc00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed81053b0 0x7faed8070df0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7faec400c970 tx=0x7faec400ce30 comp rx=0 tx=0).stop 2024-04-17T07:29:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.285+0000 7faedfc00640 1 -- 172.21.15.86:0/4194601116 shutdown_connections 2024-04-17T07:29:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.285+0000 7faedfc00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7faec00a3c80 0x7faec00a6140 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.285+0000 7faedfc00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faed8105d70 0x7faed81aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.285+0000 7faedfc00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faed81053b0 0x7faed8070df0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.285+0000 7faedfc00640 1 --2- 172.21.15.86:0/4194601116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed81041c0 0x7faed80708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.285+0000 7faedfc00640 1 -- 172.21.15.86:0/4194601116 >> 172.21.15.86:0/4194601116 conn(0x7faed80ff710 msgr2=0x7faed8108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.285+0000 7faedfc00640 1 -- 172.21.15.86:0/4194601116 shutdown_connections 2024-04-17T07:29:13.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.285+0000 7faedfc00640 1 -- 172.21.15.86:0/4194601116 wait complete. 2024-04-17T07:29:13.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:29:13.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.677+0000 7fb212e00640 1 -- 172.21.15.86:0/865440326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb20c0ffe40 msgr2=0x7fb20c100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.677+0000 7fb212e00640 1 --2- 172.21.15.86:0/865440326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb20c0ffe40 0x7fb20c100260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009900 tx=0x7fb1fc02bc30 comp rx=0 tx=0).stop 2024-04-17T07:29:13.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.677+0000 7fb212e00640 1 -- 172.21.15.86:0/865440326 shutdown_connections 2024-04-17T07:29:13.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.678+0000 7fb212e00640 1 --2- 172.21.15.86:0/865440326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c101910 0x7fb20c1061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.678+0000 7fb212e00640 1 --2- 172.21.15.86:0/865440326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c100f70 0x7fb20c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.678+0000 7fb212e00640 1 --2- 172.21.15.86:0/865440326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb20c0ffe40 0x7fb20c100260 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.678+0000 7fb212e00640 1 -- 172.21.15.86:0/865440326 >> 172.21.15.86:0/865440326 conn(0x7fb20c0fb760 msgr2=0x7fb20c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.678+0000 7fb212e00640 1 -- 172.21.15.86:0/865440326 shutdown_connections 2024-04-17T07:29:13.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.678+0000 7fb212e00640 1 -- 172.21.15.86:0/865440326 wait complete. 2024-04-17T07:29:13.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.681+0000 7fb212e00640 1 Processor -- start 2024-04-17T07:29:13.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.682+0000 7fb212e00640 1 -- start start 2024-04-17T07:29:13.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.682+0000 7fb212e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c0ffe40 0x7fb20c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.682+0000 7fb212e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c100f70 0x7fb20c070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.682+0000 7fb212e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb20c101910 0x7fb20c1ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.682+0000 7fb212e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c114150 con 0x7fb20c0ffe40 2024-04-17T07:29:13.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.682+0000 7fb212e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c113fd0 con 0x7fb20c100f70 2024-04-17T07:29:13.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.682+0000 7fb212e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1142d0 con 0x7fb20c101910 2024-04-17T07:29:13.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.683+0000 7fb210e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb20c101910 0x7fb20c1ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.683+0000 7fb210e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb20c101910 0x7fb20c1ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44656/0 (socket says 172.21.15.86:44656) 2024-04-17T07:29:13.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.683+0000 7fb210e00640 1 -- 172.21.15.86:0/3938855806 learned_addr learned my addr 172.21.15.86:0/3938855806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:13.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.683+0000 7fb20be00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c0ffe40 0x7fb20c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.683+0000 7fb20b400640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c100f70 0x7fb20c070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.684+0000 7fb20be00640 1 -- 172.21.15.86:0/3938855806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb20c101910 msgr2=0x7fb20c1ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.684+0000 7fb20be00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb20c101910 0x7fb20c1ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.684+0000 7fb20be00640 1 -- 172.21.15.86:0/3938855806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c100f70 msgr2=0x7fb20c070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.684+0000 7fb20be00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c100f70 0x7fb20c070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.684+0000 7fb20be00640 1 -- 172.21.15.86:0/3938855806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c1aeda0 con 0x7fb20c0ffe40 2024-04-17T07:29:13.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.685+0000 7fb20be00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c0ffe40 0x7fb20c0709d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009850 tx=0x7fb1fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.687+0000 7fb208c00640 1 -- 172.21.15.86:0/3938855806 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc043020 con 0x7fb20c0ffe40 2024-04-17T07:29:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.687+0000 7fb208c00640 1 -- 172.21.15.86:0/3938855806 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1fc0052b0 con 0x7fb20c0ffe40 2024-04-17T07:29:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.687+0000 7fb208c00640 1 -- 172.21.15.86:0/3938855806 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc003bb0 con 0x7fb20c0ffe40 2024-04-17T07:29:13.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.687+0000 7fb212e00640 1 -- 172.21.15.86:0/3938855806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c1aef70 con 0x7fb20c0ffe40 2024-04-17T07:29:13.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.688+0000 7fb212e00640 1 -- 172.21.15.86:0/3938855806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c1af400 con 0x7fb20c0ffe40 2024-04-17T07:29:13.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.693+0000 7fb215a3e740 1 Processor -- start 2024-04-17T07:29:13.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.694+0000 7fb215a3e740 1 -- start start 2024-04-17T07:29:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.695+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192db53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.696+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db5990 0x564192dba490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.696+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dbcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.696+0000 7fb215a3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564192d1b020 con 0x564192db4fa0 2024-04-17T07:29:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.696+0000 7fb215a3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564192cc1460 con 0x564192dbab00 2024-04-17T07:29:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.696+0000 7fb215a3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564192d16fe0 con 0x564192db5990 2024-04-17T07:29:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.697+0000 7fb210e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192db53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.697+0000 7fb210e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192db53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34422/0 (socket says 172.21.15.86:34422) 2024-04-17T07:29:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.697+0000 7fb210e00640 1 -- 172.21.15.86:0/1878144648 learned_addr learned my addr 172.21.15.86:0/1878144648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.697+0000 7fb20b400640 1 --2- 172.21.15.86:0/1878144648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dbcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.698+0000 7fb210e00640 1 -- 172.21.15.86:0/1878144648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db5990 msgr2=0x564192dba490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:29:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.698+0000 7fb210e00640 1 --2- 172.21.15.86:0/1878144648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db5990 0x564192dba490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.698+0000 7fb210e00640 1 -- 172.21.15.86:0/1878144648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 msgr2=0x564192dbcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.698+0000 7fb210e00640 1 --2- 172.21.15.86:0/1878144648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dbcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.698+0000 7fb210e00640 1 -- 172.21.15.86:0/1878144648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564192cc9040 con 0x564192db4fa0 2024-04-17T07:29:13.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.698+0000 7fb20b400640 1 --2- 172.21.15.86:0/1878144648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dbcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:13.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.699+0000 7fb208c00640 1 -- 172.21.15.86:0/3938855806 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb1fc046020 con 0x7fb20c0ffe40 2024-04-17T07:29:13.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.699+0000 7fb210e00640 1 --2- 172.21.15.86:0/1878144648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192db53c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb204011570 tx=0x7fb204011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5267d424f420a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.700+0000 7fb201600640 1 -- 172.21.15.86:0/1878144648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204004b20 con 0x564192db4fa0 2024-04-17T07:29:13.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.700+0000 7fb201600640 1 -- 172.21.15.86:0/1878144648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb204004cc0 con 0x564192db4fa0 2024-04-17T07:29:13.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.700+0000 7fb201600640 1 -- 172.21.15.86:0/1878144648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204004b20 con 0x564192db4fa0 2024-04-17T07:29:13.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.701+0000 7fb208c00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1f00a3d00 0x7fb1f00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.701+0000 7fb215a3e740 1 -- 172.21.15.86:0/1878144648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 msgr2=0x564192db53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.701+0000 7fb215a3e740 1 --2- 172.21.15.86:0/1878144648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192db53c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb204011570 tx=0x7fb204011a30 comp rx=0 tx=0).stop 2024-04-17T07:29:13.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.702+0000 7fb208c00640 1 -- 172.21.15.86:0/3938855806 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb1fc0f2a60 con 0x7fb20c0ffe40 2024-04-17T07:29:13.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.702+0000 7fb20be00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1f00a3d00 0x7fb1f00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.704+0000 7fb215a3e740 1 -- 172.21.15.86:0/1878144648 shutdown_connections 2024-04-17T07:29:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.704+0000 7fb215a3e740 1 --2- 172.21.15.86:0/1878144648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dbcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.704+0000 7fb215a3e740 1 --2- 172.21.15.86:0/1878144648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db5990 0x564192dba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.704+0000 7fb215a3e740 1 --2- 172.21.15.86:0/1878144648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192db53c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.704+0000 7fb215a3e740 1 -- 172.21.15.86:0/1878144648 >> 172.21.15.86:0/1878144648 conn(0x564192db0510 msgr2=0x564192db0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.704+0000 7fb20be00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1f00a3d00 0x7fb1f00a61c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0c70a0 tx=0x7fb1fc02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.704+0000 7fb215a3e740 1 -- 172.21.15.86:0/1878144648 shutdown_connections 2024-04-17T07:29:13.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.704+0000 7fb215a3e740 1 -- 172.21.15.86:0/1878144648 wait complete. 2024-04-17T07:29:13.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.705+0000 7fb215a3e740 1 -- start start 2024-04-17T07:29:13.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.705+0000 7fb215a3e740 1 Processor -- start 2024-04-17T07:29:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192dbf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db5990 0x564192dbf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dc69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb215a3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564192dbdb00 con 0x564192db4fa0 2024-04-17T07:29:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb215a3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564192cd4830 con 0x564192dbab00 2024-04-17T07:29:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb215a3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564192dbdc80 con 0x564192db5990 2024-04-17T07:29:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb215a3e740 10 client.0 _ll_register_callbacks cb 0x564192ca2fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb210e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192dbf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb210e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192dbf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34428/0 (socket says 172.21.15.86:34428) 2024-04-17T07:29:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb210e00640 1 -- 172.21.15.86:0/2845123889 learned_addr learned my addr 172.21.15.86:0/2845123889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.706+0000 7fb20b400640 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db5990 0x564192dbf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.707+0000 7fb210e00640 1 -- 172.21.15.86:0/2845123889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db5990 msgr2=0x564192dbf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.707+0000 7fb210e00640 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db5990 0x564192dbf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.707+0000 7fb210e00640 1 -- 172.21.15.86:0/2845123889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 msgr2=0x564192dc69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.707+0000 7fb210e00640 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dc69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.707+0000 7fb210e00640 1 -- 172.21.15.86:0/2845123889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564192cd8920 con 0x564192db4fa0 2024-04-17T07:29:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.708+0000 7fb210e00640 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192dbf460 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb204009a30 tx=0x7fb204009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563619171a7b64ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.708+0000 7fb1f6000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.711+0000 7fb1f6000640 1 -- 172.21.15.86:0/2845123889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204028070 con 0x564192db4fa0 2024-04-17T07:29:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.711+0000 7fb1f6000640 1 -- 172.21.15.86:0/2845123889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb204016040 con 0x564192db4fa0 2024-04-17T07:29:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.711+0000 7fb215a3e740 1 -- 172.21.15.86:0/2845123889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb204004cc0 con 0x564192db4fa0 2024-04-17T07:29:13.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.713+0000 7fb1f6000640 1 -- 172.21.15.86:0/2845123889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb204032050 con 0x564192db4fa0 2024-04-17T07:29:13.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.713+0000 7fb215a3e740 10 client.18936 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:13.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.713+0000 7fb215a3e740 1 -- 172.21.15.86:0/2845123889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564192ca67f0 con 0x564192db4fa0 2024-04-17T07:29:13.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.714+0000 7fb1f6000640 1 -- 172.21.15.86:0/2845123889 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb204004cc0 con 0x564192db4fa0 2024-04-17T07:29:13.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.715+0000 7fb1f6000640 1 -- 172.21.15.86:0/2845123889 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb20402d610 con 0x564192db4fa0 2024-04-17T07:29:13.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.715+0000 7fb215a3e740 10 client.18936 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:13.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.715+0000 7fb215a3e740 10 client.18936 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:29:13.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.715+0000 7fb215a3e740 20 client.18936 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:13.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.715+0000 7fb215a3e740 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564192dcadb0 0x564192dcd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.715+0000 7fb215a3e740 4 client.18936 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:13.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.715+0000 7fb215a3e740 1 -- 172.21.15.86:0/2845123889 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564192d28a40 con 0x564192dcadb0 2024-04-17T07:29:13.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.716+0000 7fb20b400640 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564192dcadb0 0x564192dcd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.719+0000 7fb20b400640 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564192dcadb0 0x564192dcd1e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ab3768d8d8b0a71 server_cookie=f3f80746716efa44 in_seq=0 out_seq=0 2024-04-17T07:29:13.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.719+0000 7fb1f6000640 10 client.18936 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:29:13.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.721+0000 7fb1f6000640 1 -- 172.21.15.86:0/2845123889 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564192d28a40 con 0x564192dcadb0 2024-04-17T07:29:13.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.721+0000 7fb1f6000640 10 client.18936 handle_command_reply: tid=0 2024-04-17T07:29:13.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.721+0000 7fb215a3e740 1 client.18936 shutdown 2024-04-17T07:29:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.722+0000 7fb215a3e740 1 -- 172.21.15.86:0/2845123889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 msgr2=0x564192dbf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.722+0000 7fb215a3e740 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192dbf460 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb204009a30 tx=0x7fb204009d70 comp rx=0 tx=0).stop 2024-04-17T07:29:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.722+0000 7fb215a3e740 1 -- 172.21.15.86:0/2845123889 shutdown_connections 2024-04-17T07:29:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.722+0000 7fb215a3e740 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564192dcadb0 0x564192dcd1e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.722+0000 7fb215a3e740 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dc69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.722+0000 7fb215a3e740 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db5990 0x564192dbf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.722+0000 7fb215a3e740 1 --2- 172.21.15.86:0/2845123889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db4fa0 0x564192dbf460 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.722+0000 7fb215a3e740 1 -- 172.21.15.86:0/2845123889 >> 172.21.15.86:0/2845123889 conn(0x564192db0510 msgr2=0x564192daf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.722+0000 7fb1f6000640 0 client.18936 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:29:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.723+0000 7fb215a3e740 1 -- 172.21.15.86:0/2845123889 shutdown_connections 2024-04-17T07:29:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.723+0000 7fb215a3e740 1 -- 172.21.15.86:0/2845123889 wait complete. 2024-04-17T07:29:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.723+0000 7fb215a3e740 20 client.18936 trim_cache size 0 max 16384 2024-04-17T07:29:13.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.749+0000 7fb215a3e740 1 Processor -- start 2024-04-17T07:29:13.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.749+0000 7fb215a3e740 1 -- start start 2024-04-17T07:29:13.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.750+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 0x564192dbf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.750+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db5990 0x564192dc6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.750+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192db3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.750+0000 7fb215a3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b99c0 con 0x564192db5990 2024-04-17T07:29:13.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.750+0000 7fb215a3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1bb500 con 0x564192dbab00 2024-04-17T07:29:13.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.750+0000 7fb215a3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1ba090 con 0x564192db4fa0 2024-04-17T07:29:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.750+0000 7fb20b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db5990 0x564192dc6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.750+0000 7fb20b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db5990 0x564192dc6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34430/0 (socket says 172.21.15.86:34430) 2024-04-17T07:29:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.750+0000 7fb20b400640 1 -- 172.21.15.86:0/3742947604 learned_addr learned my addr 172.21.15.86:0/3742947604 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.750+0000 7fb210e00640 1 --2- 172.21.15.86:0/3742947604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192db3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.751+0000 7fb210e00640 1 --2- 172.21.15.86:0/3742947604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 0x564192dbf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.751+0000 7fb20b400640 1 -- 172.21.15.86:0/3742947604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 msgr2=0x564192dbf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.751+0000 7fb20b400640 1 --2- 172.21.15.86:0/3742947604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 0x564192dbf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.751+0000 7fb20b400640 1 -- 172.21.15.86:0/3742947604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 msgr2=0x564192db3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.751+0000 7fb20b400640 1 --2- 172.21.15.86:0/3742947604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192db3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.751+0000 7fb20b400640 1 -- 172.21.15.86:0/3742947604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564192a2d3f0 con 0x564192db5990 2024-04-17T07:29:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.752+0000 7fb210e00640 1 --2- 172.21.15.86:0/3742947604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192db3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:29:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.752+0000 7fb20b400640 1 --2- 172.21.15.86:0/3742947604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db5990 0x564192dc6970 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800bb00 tx=0x7fb1f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44563ae9e113d0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.753+0000 7fb200c00640 1 -- 172.21.15.86:0/3742947604 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f801c0e0 con 0x564192db5990 2024-04-17T07:29:13.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.753+0000 7fb200c00640 1 -- 172.21.15.86:0/3742947604 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1f801c730 con 0x564192db5990 2024-04-17T07:29:13.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.753+0000 7fb200c00640 1 -- 172.21.15.86:0/3742947604 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f801abf0 con 0x564192db5990 2024-04-17T07:29:13.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.754+0000 7fb215a3e740 1 -- 172.21.15.86:0/3742947604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db5990 msgr2=0x564192dc6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.754+0000 7fb215a3e740 1 --2- 172.21.15.86:0/3742947604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db5990 0x564192dc6970 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800bb00 tx=0x7fb1f8008040 comp rx=0 tx=0).stop 2024-04-17T07:29:13.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.754+0000 7fb215a3e740 1 -- 172.21.15.86:0/3742947604 shutdown_connections 2024-04-17T07:29:13.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.754+0000 7fb215a3e740 1 --2- 172.21.15.86:0/3742947604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192db3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.754+0000 7fb215a3e740 1 --2- 172.21.15.86:0/3742947604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192db5990 0x564192dc6970 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.754+0000 7fb215a3e740 1 --2- 172.21.15.86:0/3742947604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 0x564192dbf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.754+0000 7fb215a3e740 1 -- 172.21.15.86:0/3742947604 >> 172.21.15.86:0/3742947604 conn(0x564192db0510 msgr2=0x564192db3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.755+0000 7fb215a3e740 1 -- 172.21.15.86:0/3742947604 shutdown_connections 2024-04-17T07:29:13.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.755+0000 7fb215a3e740 1 -- 172.21.15.86:0/3742947604 wait complete. 2024-04-17T07:29:13.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.755+0000 7fb215a3e740 1 -- start start 2024-04-17T07:29:13.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.756+0000 7fb215a3e740 1 Processor -- start 2024-04-17T07:29:13.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.756+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 0x564192dc3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.756+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dc43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.756+0000 7fb215a3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192dc53b0 0x564192dc5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.756+0000 7fb215a3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1bae30 con 0x564192dc53b0 2024-04-17T07:29:13.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.756+0000 7fb215a3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1ba760 con 0x564192dbab00 2024-04-17T07:29:13.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.756+0000 7fb215a3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1b7990 con 0x564192db4fa0 2024-04-17T07:29:13.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.756+0000 7fb215a3e740 10 client.0 _ll_register_callbacks cb 0x7fb204011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.757+0000 7fb210e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192dc53b0 0x564192dc5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.757+0000 7fb210e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 0x564192dc3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.757+0000 7fb210e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 0x564192dc3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44690/0 (socket says 172.21.15.86:44690) 2024-04-17T07:29:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.757+0000 7fb210e00640 1 -- 172.21.15.86:0/504025286 learned_addr learned my addr 172.21.15.86:0/504025286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.757+0000 7fb20b400640 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dc43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.757+0000 7fb210e00640 1 -- 172.21.15.86:0/504025286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 msgr2=0x564192dc43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.757+0000 7fb210e00640 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dc43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.757+0000 7fb210e00640 1 -- 172.21.15.86:0/504025286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192dc53b0 msgr2=0x564192dc5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.757+0000 7fb210e00640 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192dc53b0 0x564192dc5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.758+0000 7fb210e00640 1 -- 172.21.15.86:0/504025286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564192af77d0 con 0x564192db4fa0 2024-04-17T07:29:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.758+0000 7fb210e00640 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 0x564192dc3e60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb20400fc40 tx=0x7fb204009d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=833727c4274d1ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.758+0000 7fb1f6000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.758+0000 7fb1f6000640 1 -- 172.21.15.86:0/504025286 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204028040 con 0x564192db4fa0 2024-04-17T07:29:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.758+0000 7fb215a3e740 1 -- 172.21.15.86:0/504025286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f801c730 con 0x564192db4fa0 2024-04-17T07:29:13.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.758+0000 7fb1f6000640 1 -- 172.21.15.86:0/504025286 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb2040052a0 con 0x564192db4fa0 2024-04-17T07:29:13.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.760+0000 7fb1f6000640 1 -- 172.21.15.86:0/504025286 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2040054a0 con 0x564192db4fa0 2024-04-17T07:29:13.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.760+0000 7fb215a3e740 10 client.27919 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:13.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.760+0000 7fb215a3e740 1 -- 172.21.15.86:0/504025286 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564192e31320 con 0x564192db4fa0 2024-04-17T07:29:13.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.761+0000 7fb1f6000640 1 -- 172.21.15.86:0/504025286 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb204037440 con 0x564192db4fa0 2024-04-17T07:29:13.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.761+0000 7fb1f6000640 1 -- 172.21.15.86:0/504025286 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb204024430 con 0x564192db4fa0 2024-04-17T07:29:13.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.762+0000 7fb215a3e740 10 client.27919 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:13.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.762+0000 7fb215a3e740 10 client.27919 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:29:13.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.762+0000 7fb215a3e740 20 client.27919 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:13.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.762+0000 7fb215a3e740 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564192e31b50 0x564192e33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:13.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.762+0000 7fb215a3e740 4 client.27919 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:29:13.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.762+0000 7fb215a3e740 1 -- 172.21.15.86:0/504025286 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb1f801c730 con 0x564192e31b50 2024-04-17T07:29:13.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.763+0000 7fb20b400640 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564192e31b50 0x564192e33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:13.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.765+0000 7fb20b400640 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564192e31b50 0x564192e33f40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bea9675accde4211 server_cookie=61d15af0e80f0799 in_seq=0 out_seq=0 2024-04-17T07:29:13.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.765+0000 7fb1f6000640 10 client.27919 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:29:13.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.766+0000 7fb1f6000640 1 -- 172.21.15.86:0/504025286 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb1f801c730 con 0x564192e31b50 2024-04-17T07:29:13.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.766+0000 7fb1f6000640 10 client.27919 handle_command_reply: tid=0 2024-04-17T07:29:13.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.766+0000 7fb215a3e740 1 client.27919 shutdown 2024-04-17T07:29:13.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.767+0000 7fb215a3e740 1 -- 172.21.15.86:0/504025286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 msgr2=0x564192dc3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.767+0000 7fb215a3e740 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 0x564192dc3e60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb20400fc40 tx=0x7fb204009d70 comp rx=0 tx=0).stop 2024-04-17T07:29:13.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.768+0000 7fb215a3e740 1 -- 172.21.15.86:0/504025286 shutdown_connections 2024-04-17T07:29:13.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.768+0000 7fb215a3e740 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x564192e31b50 0x564192e33f40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.768+0000 7fb215a3e740 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564192dc53b0 0x564192dc5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.768+0000 7fb1f6000640 0 client.27919 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:29:13.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.768+0000 7fb215a3e740 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564192dbab00 0x564192dc43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.768+0000 7fb215a3e740 1 --2- 172.21.15.86:0/504025286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564192db4fa0 0x564192dc3e60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.768+0000 7fb215a3e740 1 -- 172.21.15.86:0/504025286 >> 172.21.15.86:0/504025286 conn(0x564192db0510 msgr2=0x564192db4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.768+0000 7fb215a3e740 1 -- 172.21.15.86:0/504025286 shutdown_connections 2024-04-17T07:29:13.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.768+0000 7fb215a3e740 1 -- 172.21.15.86:0/504025286 wait complete. 2024-04-17T07:29:13.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.769+0000 7fb215a3e740 20 client.27919 trim_cache size 0 max 16384 2024-04-17T07:29:13.770 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:29:13.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.771+0000 7fb212e00640 1 -- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1f00a3d00 msgr2=0x7fb1f00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.771+0000 7fb212e00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1f00a3d00 0x7fb1f00a61c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0c70a0 tx=0x7fb1fc02c7e0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.771+0000 7fb212e00640 1 -- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c0ffe40 msgr2=0x7fb20c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:13.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.771+0000 7fb212e00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c0ffe40 0x7fb20c0709d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009850 tx=0x7fb1fc004270 comp rx=0 tx=0).stop 2024-04-17T07:29:13.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.772+0000 7fb212e00640 1 -- 172.21.15.86:0/3938855806 shutdown_connections 2024-04-17T07:29:13.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.772+0000 7fb212e00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb1f00a3d00 0x7fb1f00a61c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.772+0000 7fb212e00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb20c101910 0x7fb20c1ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.772+0000 7fb212e00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c100f70 0x7fb20c070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.772+0000 7fb212e00640 1 --2- 172.21.15.86:0/3938855806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb20c0ffe40 0x7fb20c0709d0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.772+0000 7fb212e00640 1 -- 172.21.15.86:0/3938855806 >> 172.21.15.86:0/3938855806 conn(0x7fb20c0fb760 msgr2=0x7fb20c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.773+0000 7fb212e00640 1 -- 172.21.15.86:0/3938855806 shutdown_connections 2024-04-17T07:29:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:13.774+0000 7fb212e00640 1 -- 172.21.15.86:0/3938855806 wait complete. 2024-04-17T07:29:13.812 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:29:14.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.161+0000 7f10b7200640 1 -- 172.21.15.86:0/2257697667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10b8103980 msgr2=0x7f10b810a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.161+0000 7f10b7200640 1 --2- 172.21.15.86:0/2257697667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10b8103980 0x7f10b810a2a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009ca0 tx=0x7f10b002c100 comp rx=0 tx=0).stop 2024-04-17T07:29:14.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.162+0000 7f10b7200640 1 -- 172.21.15.86:0/2257697667 shutdown_connections 2024-04-17T07:29:14.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.162+0000 7f10b7200640 1 --2- 172.21.15.86:0/2257697667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10b8103980 0x7f10b810a2a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.162+0000 7f10b7200640 1 --2- 172.21.15.86:0/2257697667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b8102fc0 0x7f10b8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.162+0000 7f10b7200640 1 --2- 172.21.15.86:0/2257697667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b8101e70 0x7f10b8102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.162+0000 7f10b7200640 1 -- 172.21.15.86:0/2257697667 >> 172.21.15.86:0/2257697667 conn(0x7f10b80fd400 msgr2=0x7f10b80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.163+0000 7f10b7200640 1 -- 172.21.15.86:0/2257697667 shutdown_connections 2024-04-17T07:29:14.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.163+0000 7f10b7200640 1 -- 172.21.15.86:0/2257697667 wait complete. 2024-04-17T07:29:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.165+0000 7f10b7200640 1 Processor -- start 2024-04-17T07:29:14.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.165+0000 7f10b7200640 1 -- start start 2024-04-17T07:29:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.166+0000 7f10b7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b8101e70 0x7f10b8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.166+0000 7f10b7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b8102fc0 0x7f10b8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.166+0000 7f10b7200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10b8103980 0x7f10b819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.166+0000 7f10b7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10b810d950 con 0x7f10b8101e70 2024-04-17T07:29:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.166+0000 7f10b7200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10b810d7d0 con 0x7f10b8103980 2024-04-17T07:29:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.166+0000 7f10b7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10b810dad0 con 0x7f10b8102fc0 2024-04-17T07:29:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.167+0000 7f10b5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b8102fc0 0x7f10b8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.167+0000 7f10b5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b8102fc0 0x7f10b8198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44706/0 (socket says 172.21.15.86:44706) 2024-04-17T07:29:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.167+0000 7f10b5400640 1 -- 172.21.15.86:0/1773164093 learned_addr learned my addr 172.21.15.86:0/1773164093 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.167+0000 7f10b5e00640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b8101e70 0x7f10b8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.167+0000 7f10b6800640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10b8103980 0x7f10b819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.167+0000 7f10b5400640 1 -- 172.21.15.86:0/1773164093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10b8103980 msgr2=0x7f10b819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.167+0000 7f10b5400640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10b8103980 0x7f10b819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.168+0000 7f10b5400640 1 -- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b8101e70 msgr2=0x7f10b8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.168+0000 7f10b5400640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b8101e70 0x7f10b8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.168+0000 7f10b5400640 1 -- 172.21.15.86:0/1773164093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b80742c0 con 0x7f10b8102fc0 2024-04-17T07:29:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.168+0000 7f10b5e00640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b8101e70 0x7f10b8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.168+0000 7f10b5400640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b8102fc0 0x7f10b8198530 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f10a400e9c0 tx=0x7f10a400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.169+0000 7f10aea00640 1 -- 172.21.15.86:0/1773164093 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a400cd80 con 0x7f10b8102fc0 2024-04-17T07:29:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.169+0000 7f10aea00640 1 -- 172.21.15.86:0/1773164093 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10a40035f0 con 0x7f10b8102fc0 2024-04-17T07:29:14.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.169+0000 7f10aea00640 1 -- 172.21.15.86:0/1773164093 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4010690 con 0x7f10b8102fc0 2024-04-17T07:29:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.169+0000 7f10b7200640 1 -- 172.21.15.86:0/1773164093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b80745b0 con 0x7f10b8102fc0 2024-04-17T07:29:14.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.169+0000 7f10b7200640 1 -- 172.21.15.86:0/1773164093 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b8074ab0 con 0x7f10b8102fc0 2024-04-17T07:29:14.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.177+0000 7f10bfa05740 1 Processor -- start 2024-04-17T07:29:14.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.177+0000 7f10aea00640 1 -- 172.21.15.86:0/1773164093 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f10a40037a0 con 0x7f10b8102fc0 2024-04-17T07:29:14.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.178+0000 7f10bfa05740 1 -- start start 2024-04-17T07:29:14.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.178+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.178+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.178+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.178+0000 7f10bfa05740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558305724020 con 0x5583057beb50 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.178+0000 7f10bfa05740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5583056ca460 con 0x5583057be160 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.178+0000 7f10bfa05740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55830571ffe0 con 0x5583057c3cc0 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.179+0000 7f10b6800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.179+0000 7f10b6800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50740/0 (socket says 172.21.15.86:50740) 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.179+0000 7f10b6800640 1 -- 172.21.15.86:0/2378735271 learned_addr learned my addr 172.21.15.86:0/2378735271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.179+0000 7f10b6800640 1 -- 172.21.15.86:0/2378735271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 msgr2=0x5583057c6150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.179+0000 7f10b6800640 1 --2- 172.21.15.86:0/2378735271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057c6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.179+0000 7f10b6800640 1 -- 172.21.15.86:0/2378735271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 msgr2=0x5583057c3650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.179+0000 7f10b6800640 1 --2- 172.21.15.86:0/2378735271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057c3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.179+0000 7f10b6800640 1 -- 172.21.15.86:0/2378735271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583056d2040 con 0x5583057be160 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.180+0000 7f10aea00640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f109c0a3d00 0x7f109c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.180+0000 7f10aea00640 1 -- 172.21.15.86:0/1773164093 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f10a40d0b00 con 0x7f10b8102fc0 2024-04-17T07:29:14.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.180+0000 7f10b6800640 1 --2- 172.21.15.86:0/2378735271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057be580 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f10b002c710 tx=0x7f10b0005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=296334fc3acaf298 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.180+0000 7f10a3400640 1 -- 172.21.15.86:0/2378735271 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0053030 con 0x5583057be160 2024-04-17T07:29:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.181+0000 7f10a3400640 1 -- 172.21.15.86:0/2378735271 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10b0003e20 con 0x5583057be160 2024-04-17T07:29:14.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.181+0000 7f10a3400640 1 -- 172.21.15.86:0/2378735271 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0039040 con 0x5583057be160 2024-04-17T07:29:14.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.181+0000 7f10b5e00640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f109c0a3d00 0x7f109c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.182+0000 7f10bfa05740 1 -- 172.21.15.86:0/2378735271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 msgr2=0x5583057be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.182+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2378735271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057be580 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f10b002c710 tx=0x7f10b0005f80 comp rx=0 tx=0).stop 2024-04-17T07:29:14.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.182+0000 7f10b5e00640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f109c0a3d00 0x7f109c0a61c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x5583057bef50 tx=0x7f10a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.182+0000 7f10bfa05740 1 -- 172.21.15.86:0/2378735271 shutdown_connections 2024-04-17T07:29:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.182+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2378735271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.182+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2378735271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.182+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2378735271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057be580 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.182+0000 7f10bfa05740 1 -- 172.21.15.86:0/2378735271 >> 172.21.15.86:0/2378735271 conn(0x5583057b96d0 msgr2=0x5583057b9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.183+0000 7f10bfa05740 1 -- 172.21.15.86:0/2378735271 shutdown_connections 2024-04-17T07:29:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.183+0000 7f10bfa05740 1 -- 172.21.15.86:0/2378735271 wait complete. 2024-04-17T07:29:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.183+0000 7f10bfa05740 1 -- start start 2024-04-17T07:29:14.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.184+0000 7f10bfa05740 1 Processor -- start 2024-04-17T07:29:14.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.185+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.185+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.185+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.185+0000 7f10bfa05740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5583057c6c10 con 0x5583057beb50 2024-04-17T07:29:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.185+0000 7f10bfa05740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5583056dd830 con 0x5583057be160 2024-04-17T07:29:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.185+0000 7f10bfa05740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5583057c6d90 con 0x5583057c3cc0 2024-04-17T07:29:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.185+0000 7f10bfa05740 10 client.0 _ll_register_callbacks cb 0x5583056abfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.185+0000 7f10b5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.185+0000 7f10b5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50754/0 (socket says 172.21.15.86:50754) 2024-04-17T07:29:14.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.185+0000 7f10b5e00640 1 -- 172.21.15.86:0/2801588831 learned_addr learned my addr 172.21.15.86:0/2801588831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:14.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.186+0000 7f10b5400640 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.186+0000 7f10b6800640 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.187+0000 7f10b5e00640 1 -- 172.21.15.86:0/2801588831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 msgr2=0x5583057cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.187+0000 7f10b5e00640 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.187+0000 7f10b5e00640 1 -- 172.21.15.86:0/2801588831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 msgr2=0x5583057c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.187+0000 7f10b5e00640 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.187+0000 7f10b5e00640 1 -- 172.21.15.86:0/2801588831 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583056e1920 con 0x5583057be160 2024-04-17T07:29:14.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.187+0000 7f10b6800640 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:14.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.187+0000 7f10b5e00640 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c86e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f10a8003870 tx=0x7f10a8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9deef720f07c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.187+0000 7f1097e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:29:14.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.188+0000 7f1097e00640 1 -- 172.21.15.86:0/2801588831 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a800ecc0 con 0x5583057be160 2024-04-17T07:29:14.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.188+0000 7f1097e00640 1 -- 172.21.15.86:0/2801588831 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f10a801e420 con 0x5583057be160 2024-04-17T07:29:14.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.189+0000 7f10bfa05740 1 -- 172.21.15.86:0/2801588831 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b0003e20 con 0x5583057be160 2024-04-17T07:29:14.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.189+0000 7f1097e00640 1 -- 172.21.15.86:0/2801588831 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10a800ee60 con 0x5583057be160 2024-04-17T07:29:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.189+0000 7f10bfa05740 10 client.37883 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:14.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.189+0000 7f10bfa05740 1 -- 172.21.15.86:0/2801588831 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583056af7f0 con 0x5583057be160 2024-04-17T07:29:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.191+0000 7f1097e00640 1 -- 172.21.15.86:0/2801588831 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10b0003e20 con 0x5583057be160 2024-04-17T07:29:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.193+0000 7f1097e00640 1 -- 172.21.15.86:0/2801588831 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f10a8030620 con 0x5583057be160 2024-04-17T07:29:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.193+0000 7f10bfa05740 10 client.37883 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.193+0000 7f10bfa05740 10 client.37883 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:29:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.193+0000 7f10bfa05740 20 client.37883 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.193+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5583057d4190 0x5583057d6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.193+0000 7f10bfa05740 4 client.37883 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.193+0000 7f10bfa05740 1 -- 172.21.15.86:0/2801588831 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558305731a40 con 0x5583057d4190 2024-04-17T07:29:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.193+0000 7f10b5400640 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5583057d4190 0x5583057d6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.196+0000 7f1097e00640 10 client.37883 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:29:14.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.196+0000 7f10b5400640 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5583057d4190 0x5583057d6610 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d8ab337e3cecc53 server_cookie=918623af350b3e66 in_seq=0 out_seq=0 2024-04-17T07:29:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.198+0000 7f1097e00640 1 -- 172.21.15.86:0/2801588831 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558305731a40 con 0x5583057d4190 2024-04-17T07:29:14.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.198+0000 7f1097e00640 10 client.37883 handle_command_reply: tid=0 2024-04-17T07:29:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.198+0000 7f10bfa05740 1 client.37883 shutdown 2024-04-17T07:29:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.199+0000 7f10bfa05740 1 -- 172.21.15.86:0/2801588831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 msgr2=0x5583057c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.199+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c86e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f10a8003870 tx=0x7f10a8004730 comp rx=0 tx=0).stop 2024-04-17T07:29:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.200+0000 7f10bfa05740 1 -- 172.21.15.86:0/2801588831 shutdown_connections 2024-04-17T07:29:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.200+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.200+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.200+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5583057d4190 0x5583057d6610 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.200+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2801588831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c86e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.200+0000 7f10bfa05740 1 -- 172.21.15.86:0/2801588831 >> 172.21.15.86:0/2801588831 conn(0x5583057b96d0 msgr2=0x5583057b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.200+0000 7f1097e00640 0 client.37883 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:29:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.201+0000 7f10bfa05740 1 -- 172.21.15.86:0/2801588831 shutdown_connections 2024-04-17T07:29:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.201+0000 7f10bfa05740 1 -- 172.21.15.86:0/2801588831 wait complete. 2024-04-17T07:29:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.201+0000 7f10bfa05740 20 client.37883 trim_cache size 0 max 16384 2024-04-17T07:29:14.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.235+0000 7f10bfa05740 1 Processor -- start 2024-04-17T07:29:14.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.235+0000 7f10bfa05740 1 -- start start 2024-04-17T07:29:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.236+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.236+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.236+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057bd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.236+0000 7f10bfa05740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81a8600 con 0x5583057beb50 2024-04-17T07:29:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.236+0000 7f10bfa05740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81aa140 con 0x5583057be160 2024-04-17T07:29:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.236+0000 7f10bfa05740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81a8cd0 con 0x5583057c3cc0 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.237+0000 7f10b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.237+0000 7f10b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34456/0 (socket says 172.21.15.86:34456) 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.237+0000 7f10b5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.237+0000 7f10b5400640 1 -- 172.21.15.86:0/512169763 learned_addr learned my addr 172.21.15.86:0/512169763 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.237+0000 7f10b5e00640 1 -- 172.21.15.86:0/512169763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 msgr2=0x5583057bd040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.237+0000 7f10b5e00640 1 --2- 172.21.15.86:0/512169763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057bd040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.237+0000 7f10b5e00640 1 -- 172.21.15.86:0/512169763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 msgr2=0x5583057cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.237+0000 7f10b5e00640 1 --2- 172.21.15.86:0/512169763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.237+0000 7f10b5e00640 1 -- 172.21.15.86:0/512169763 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583054363f0 con 0x5583057be160 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.238+0000 7f10b5e00640 1 --2- 172.21.15.86:0/512169763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c8b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f10a8030a90 tx=0x7f10a80044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7633281f58537918 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.238+0000 7f10a2a00640 1 -- 172.21.15.86:0/512169763 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a803e070 con 0x5583057be160 2024-04-17T07:29:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.239+0000 7f10a2a00640 1 -- 172.21.15.86:0/512169763 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10a8005bf0 con 0x5583057be160 2024-04-17T07:29:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.239+0000 7f10a2a00640 1 -- 172.21.15.86:0/512169763 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a803e070 con 0x5583057be160 2024-04-17T07:29:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.241+0000 7f10bfa05740 1 -- 172.21.15.86:0/512169763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 msgr2=0x5583057c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.241+0000 7f10bfa05740 1 --2- 172.21.15.86:0/512169763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c8b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f10a8030a90 tx=0x7f10a80044c0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.242+0000 7f10bfa05740 1 -- 172.21.15.86:0/512169763 shutdown_connections 2024-04-17T07:29:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.242+0000 7f10bfa05740 1 --2- 172.21.15.86:0/512169763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057c3cc0 0x5583057bd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.242+0000 7f10bfa05740 1 --2- 172.21.15.86:0/512169763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057beb50 0x5583057cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.242+0000 7f10bfa05740 1 --2- 172.21.15.86:0/512169763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057c8b80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.242+0000 7f10bfa05740 1 -- 172.21.15.86:0/512169763 >> 172.21.15.86:0/512169763 conn(0x5583057b96d0 msgr2=0x5583057bc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.242+0000 7f10bfa05740 1 -- 172.21.15.86:0/512169763 shutdown_connections 2024-04-17T07:29:14.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.243+0000 7f10bfa05740 1 -- 172.21.15.86:0/512169763 wait complete. 2024-04-17T07:29:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.243+0000 7f10bfa05740 1 -- start start 2024-04-17T07:29:14.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.243+0000 7f10bfa05740 1 Processor -- start 2024-04-17T07:29:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.245+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.245+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057beb50 0x5583057cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.245+0000 7f10bfa05740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057c3cc0 0x5583057ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.245+0000 7f10bfa05740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81a9a70 con 0x5583057c3cc0 2024-04-17T07:29:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.245+0000 7f10bfa05740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81a93a0 con 0x5583057be160 2024-04-17T07:29:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.245+0000 7f10b5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.246+0000 7f10bfa05740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81a65d0 con 0x5583057beb50 2024-04-17T07:29:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.246+0000 7f10b5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057beb50 0x5583057cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.246+0000 7f10bfa05740 10 client.0 _ll_register_callbacks cb 0x7f10a8004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.246+0000 7f10b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057c3cc0 0x5583057ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.246+0000 7f10b5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057beb50 0x5583057cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44736/0 (socket says 172.21.15.86:44736) 2024-04-17T07:29:14.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.246+0000 7f10b5400640 1 -- 172.21.15.86:0/2650243055 learned_addr learned my addr 172.21.15.86:0/2650243055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:14.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.247+0000 7f10b5400640 1 -- 172.21.15.86:0/2650243055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 msgr2=0x5583057cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.247+0000 7f10b5400640 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.247+0000 7f10b5400640 1 -- 172.21.15.86:0/2650243055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057c3cc0 msgr2=0x5583057ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.247+0000 7f10b5400640 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057c3cc0 0x5583057ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.247+0000 7f10b5400640 1 -- 172.21.15.86:0/2650243055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583055007d0 con 0x5583057beb50 2024-04-17T07:29:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.248+0000 7f10b5400640 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057beb50 0x5583057cd9b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f10a409c990 tx=0x7f10a4085f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d51e78a432ae864 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.248+0000 7f1097e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.248+0000 7f1097e00640 1 -- 172.21.15.86:0/2650243055 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a40b5070 con 0x5583057beb50 2024-04-17T07:29:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.248+0000 7f1097e00640 1 -- 172.21.15.86:0/2650243055 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f10a4099070 con 0x5583057beb50 2024-04-17T07:29:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.249+0000 7f10bfa05740 1 -- 172.21.15.86:0/2650243055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55830572b8b0 con 0x5583057beb50 2024-04-17T07:29:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.249+0000 7f1097e00640 1 -- 172.21.15.86:0/2650243055 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10a409ca00 con 0x5583057beb50 2024-04-17T07:29:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.250+0000 7f10bfa05740 10 client.27928 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.250+0000 7f10bfa05740 1 -- 172.21.15.86:0/2650243055 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55830583a7b0 con 0x5583057beb50 2024-04-17T07:29:14.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.250+0000 7f1097e00640 1 -- 172.21.15.86:0/2650243055 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55830572b8b0 con 0x5583057beb50 2024-04-17T07:29:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.253+0000 7f1097e00640 1 -- 172.21.15.86:0/2650243055 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f10a40bbbc0 con 0x5583057beb50 2024-04-17T07:29:14.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.254+0000 7f10bfa05740 10 client.27928 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:14.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.254+0000 7f10bfa05740 10 client.27928 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:29:14.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.254+0000 7f10bfa05740 20 client.27928 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:14.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.254+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55830583b160 0x55830583d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.254+0000 7f10bfa05740 4 client.27928 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:29:14.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.254+0000 7f10bfa05740 1 -- 172.21.15.86:0/2650243055 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583056a7570 con 0x55830583b160 2024-04-17T07:29:14.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.255+0000 7f10b5e00640 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55830583b160 0x55830583d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.257+0000 7f10b5e00640 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55830583b160 0x55830583d580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ad1d3c2400531e4 server_cookie=96550312f528023d in_seq=0 out_seq=0 2024-04-17T07:29:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.257+0000 7f1097e00640 10 client.27928 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.258+0000 7f1097e00640 1 -- 172.21.15.86:0/2650243055 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583056a7570 con 0x55830583b160 2024-04-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.258+0000 7f1097e00640 10 client.27928 handle_command_reply: tid=0 2024-04-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.258+0000 7f10bfa05740 1 client.27928 shutdown 2024-04-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 1 -- 172.21.15.86:0/2650243055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057beb50 msgr2=0x5583057cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057beb50 0x5583057cd9b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f10a409c990 tx=0x7f10a4085f90 comp rx=0 tx=0).stop 2024-04-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 1 -- 172.21.15.86:0/2650243055 shutdown_connections 2024-04-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5583057c3cc0 0x5583057ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5583057beb50 0x5583057cd9b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55830583b160 0x55830583d580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 1 --2- 172.21.15.86:0/2650243055 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583057be160 0x5583057cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f1097e00640 0 client.27928 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 1 -- 172.21.15.86:0/2650243055 >> 172.21.15.86:0/2650243055 conn(0x5583057b96d0 msgr2=0x5583057c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 1 -- 172.21.15.86:0/2650243055 shutdown_connections 2024-04-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 1 -- 172.21.15.86:0/2650243055 wait complete. 2024-04-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.259+0000 7f10bfa05740 20 client.27928 trim_cache size 0 max 16384 2024-04-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.261+0000 7f10b7200640 1 -- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f109c0a3d00 msgr2=0x7f109c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.261+0000 7f10b7200640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f109c0a3d00 0x7f109c0a61c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x5583057bef50 tx=0x7f10a8005f90 comp rx=0 tx=0).stop 2024-04-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.261+0000 7f10b7200640 1 -- 172.21.15.86:0/1773164093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b8102fc0 msgr2=0x7f10b8198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.261+0000 7f10b7200640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b8102fc0 0x7f10b8198530 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f10a400e9c0 tx=0x7f10a400ee80 comp rx=0 tx=0).stop 2024-04-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.262+0000 7f10b7200640 1 -- 172.21.15.86:0/1773164093 shutdown_connections 2024-04-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.262+0000 7f10b7200640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f109c0a3d00 0x7f109c0a61c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.262+0000 7f10b7200640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10b8103980 0x7f10b819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.262+0000 7f10b7200640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10b8102fc0 0x7f10b8198530 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.262+0000 7f10b7200640 1 --2- 172.21.15.86:0/1773164093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b8101e70 0x7f10b8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.262+0000 7f10b7200640 1 -- 172.21.15.86:0/1773164093 >> 172.21.15.86:0/1773164093 conn(0x7f10b80fd400 msgr2=0x7f10b81063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.263+0000 7f10b7200640 1 -- 172.21.15.86:0/1773164093 shutdown_connections 2024-04-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.263+0000 7f10b7200640 1 -- 172.21.15.86:0/1773164093 wait complete. 2024-04-17T07:29:14.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 590.0 s, 529.4 tps, lat 187.514 ms stddev 62.221 2024-04-17T07:29:14.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.654+0000 7f1df5400640 1 -- 172.21.15.86:0/2241017456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df0100ea0 msgr2=0x7f1df01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.654+0000 7f1df5400640 1 --2- 172.21.15.86:0/2241017456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df0100ea0 0x7f1df01012c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009900 tx=0x7f1de002bc40 comp rx=0 tx=0).stop 2024-04-17T07:29:14.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.654+0000 7f1df5400640 1 -- 172.21.15.86:0/2241017456 shutdown_connections 2024-04-17T07:29:14.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.654+0000 7f1df5400640 1 --2- 172.21.15.86:0/2241017456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1df01029b0 0x7f1df01092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.654+0000 7f1df5400640 1 --2- 172.21.15.86:0/2241017456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df0101ff0 0x7f1df0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.654+0000 7f1df5400640 1 --2- 172.21.15.86:0/2241017456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df0100ea0 0x7f1df01012c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.654+0000 7f1df5400640 1 -- 172.21.15.86:0/2241017456 >> 172.21.15.86:0/2241017456 conn(0x7f1df00fc410 msgr2=0x7f1df00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.655+0000 7f1df5400640 1 -- 172.21.15.86:0/2241017456 shutdown_connections 2024-04-17T07:29:14.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.655+0000 7f1df5400640 1 -- 172.21.15.86:0/2241017456 wait complete. 2024-04-17T07:29:14.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.657+0000 7f1df5400640 1 Processor -- start 2024-04-17T07:29:14.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.657+0000 7f1df5400640 1 -- start start 2024-04-17T07:29:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.658+0000 7f1df5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df0100ea0 0x7f1df019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.658+0000 7f1df5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1df0101ff0 0x7f1df019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.658+0000 7f1df5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df01029b0 0x7f1df01a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.658+0000 7f1df5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1df010b8e0 con 0x7f1df01029b0 2024-04-17T07:29:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.658+0000 7f1df5400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1df010b760 con 0x7f1df0101ff0 2024-04-17T07:29:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.658+0000 7f1df5400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1df010ba60 con 0x7f1df0100ea0 2024-04-17T07:29:14.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.658+0000 7f1dee800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df0100ea0 0x7f1df019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.658+0000 7f1dee800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df0100ea0 0x7f1df019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44752/0 (socket says 172.21.15.86:44752) 2024-04-17T07:29:14.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.658+0000 7f1dee800640 1 -- 172.21.15.86:0/3476255770 learned_addr learned my addr 172.21.15.86:0/3476255770 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:14.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.658+0000 7f1dede00640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1df0101ff0 0x7f1df019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.659+0000 7f1def200640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df01029b0 0x7f1df01a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.659+0000 7f1dee800640 1 -- 172.21.15.86:0/3476255770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1df0101ff0 msgr2=0x7f1df019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.659+0000 7f1dee800640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1df0101ff0 0x7f1df019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.659+0000 7f1dee800640 1 -- 172.21.15.86:0/3476255770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df01029b0 msgr2=0x7f1df01a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.659+0000 7f1dee800640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df01029b0 0x7f1df01a5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.659+0000 7f1dee800640 1 -- 172.21.15.86:0/3476255770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df01a61c0 con 0x7f1df0100ea0 2024-04-17T07:29:14.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.660+0000 7f1dee800640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df0100ea0 0x7f1df019e5c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1de0036040 tx=0x7f1de00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.662+0000 7f1de7400640 1 -- 172.21.15.86:0/3476255770 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0043020 con 0x7f1df0100ea0 2024-04-17T07:29:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.662+0000 7f1de7400640 1 -- 172.21.15.86:0/3476255770 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1de00054c0 con 0x7f1df0100ea0 2024-04-17T07:29:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.662+0000 7f1de7400640 1 -- 172.21.15.86:0/3476255770 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de003e450 con 0x7f1df0100ea0 2024-04-17T07:29:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.662+0000 7f1df5400640 1 -- 172.21.15.86:0/3476255770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df01a6450 con 0x7f1df0100ea0 2024-04-17T07:29:14.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.662+0000 7f1df5400640 1 -- 172.21.15.86:0/3476255770 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df01a6950 con 0x7f1df0100ea0 2024-04-17T07:29:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.667+0000 7f1df8186740 1 Processor -- start 2024-04-17T07:29:14.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.667+0000 7f1df8186740 1 -- start start 2024-04-17T07:29:14.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.668+0000 7f1df8186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.668+0000 7f1df8186740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 0x5557402a7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.668+0000 7f1df8186740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a7b00 0x5557402a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.668+0000 7f1df8186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555740208020 con 0x5557402a1fa0 2024-04-17T07:29:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.668+0000 7f1df8186740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5557401ae460 con 0x5557402a7b00 2024-04-17T07:29:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.668+0000 7f1df8186740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555740203fe0 con 0x5557402a2990 2024-04-17T07:29:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.669+0000 7f1de7400640 1 -- 172.21.15.86:0/3476255770 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1de0046020 con 0x7f1df0100ea0 2024-04-17T07:29:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.669+0000 7f1def200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.669+0000 7f1dee800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 0x5557402a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.669+0000 7f1dee800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 0x5557402a7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44766/0 (socket says 172.21.15.86:44766) 2024-04-17T07:29:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.669+0000 7f1dee800640 1 -- 172.21.15.86:0/1316369463 learned_addr learned my addr 172.21.15.86:0/1316369463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.670+0000 7f1dede00640 1 --2- 172.21.15.86:0/1316369463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a7b00 0x5557402a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.670+0000 7f1dee800640 1 -- 172.21.15.86:0/1316369463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a7b00 msgr2=0x5557402a9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.670+0000 7f1dee800640 1 --2- 172.21.15.86:0/1316369463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a7b00 0x5557402a9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.670+0000 7f1dee800640 1 -- 172.21.15.86:0/1316369463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 msgr2=0x5557402a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.671+0000 7f1dee800640 1 --2- 172.21.15.86:0/1316369463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.671+0000 7f1dee800640 1 -- 172.21.15.86:0/1316369463 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557401b6040 con 0x5557402a2990 2024-04-17T07:29:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.671+0000 7f1de7400640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dd40a3d80 0x7f1dd40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.671+0000 7f1de7400640 1 -- 172.21.15.86:0/3476255770 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1de00f2dc0 con 0x7f1df0100ea0 2024-04-17T07:29:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.672+0000 7f1def200640 1 --2- 172.21.15.86:0/1316369463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:29:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.672+0000 7f1dee800640 1 --2- 172.21.15.86:0/1316369463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 0x5557402a7490 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1de00f6cf0 tx=0x7f1de00f6de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78fe8c693b995f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.673+0000 7f1dede00640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dd40a3d80 0x7f1dd40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.673+0000 7f1ddb400640 1 -- 172.21.15.86:0/1316369463 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de00bea90 con 0x5557402a2990 2024-04-17T07:29:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.673+0000 7f1ddb400640 1 -- 172.21.15.86:0/1316369463 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1de00035f0 con 0x5557402a2990 2024-04-17T07:29:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.673+0000 7f1ddb400640 1 -- 172.21.15.86:0/1316369463 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de00bce00 con 0x5557402a2990 2024-04-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.674+0000 7f1df8186740 1 -- 172.21.15.86:0/1316369463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 msgr2=0x5557402a7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.674+0000 7f1df8186740 1 --2- 172.21.15.86:0/1316369463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 0x5557402a7490 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1de00f6cf0 tx=0x7f1de00f6de0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.674+0000 7f1df8186740 1 -- 172.21.15.86:0/1316369463 shutdown_connections 2024-04-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.674+0000 7f1df8186740 1 --2- 172.21.15.86:0/1316369463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a7b00 0x5557402a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.674+0000 7f1df8186740 1 --2- 172.21.15.86:0/1316369463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 0x5557402a7490 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.674+0000 7f1df8186740 1 --2- 172.21.15.86:0/1316369463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.674+0000 7f1df8186740 1 -- 172.21.15.86:0/1316369463 >> 172.21.15.86:0/1316369463 conn(0x55574029d510 msgr2=0x55574029d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.675+0000 7f1dede00640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dd40a3d80 0x7f1dd40a6240 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1df0100bc0 tx=0x7f1ddc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.675+0000 7f1df8186740 1 -- 172.21.15.86:0/1316369463 shutdown_connections 2024-04-17T07:29:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.675+0000 7f1df8186740 1 -- 172.21.15.86:0/1316369463 wait complete. 2024-04-17T07:29:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.675+0000 7f1df8186740 1 -- start start 2024-04-17T07:29:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.676+0000 7f1df8186740 1 Processor -- start 2024-04-17T07:29:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.678+0000 7f1df8186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.678+0000 7f1df8186740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a2990 0x5557402ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.678+0000 7f1df8186740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 0x5557402b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.678+0000 7f1df8186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557402aa870 con 0x5557402a1fa0 2024-04-17T07:29:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.678+0000 7f1df8186740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5557401c1830 con 0x5557402a2990 2024-04-17T07:29:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.678+0000 7f1df8186740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5557402aa9f0 con 0x5557402a7b00 2024-04-17T07:29:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.678+0000 7f1df8186740 10 client.0 _ll_register_callbacks cb 0x55574018ffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.679+0000 7f1dede00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a2990 0x5557402ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.679+0000 7f1dede00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a2990 0x5557402ac750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50814/0 (socket says 172.21.15.86:50814) 2024-04-17T07:29:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.679+0000 7f1dede00640 1 -- 172.21.15.86:0/3376894870 learned_addr learned my addr 172.21.15.86:0/3376894870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.679+0000 7f1dee800640 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.680+0000 7f1def200640 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 0x5557402b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.680+0000 7f1dee800640 1 -- 172.21.15.86:0/3376894870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 msgr2=0x5557402b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.680+0000 7f1dee800640 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 0x5557402b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.680+0000 7f1dee800640 1 -- 172.21.15.86:0/3376894870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a2990 msgr2=0x5557402ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.680+0000 7f1dee800640 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a2990 0x5557402ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.680+0000 7f1dee800640 1 -- 172.21.15.86:0/3376894870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557401c5920 con 0x5557402a1fa0 2024-04-17T07:29:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.680+0000 7f1dede00640 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a2990 0x5557402ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.680+0000 7f1dee800640 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402ac210 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1de00be9c0 tx=0x7f1de00df300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f419c33ebe5f79bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.681+0000 7f1dcfe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:14.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.681+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3376894870 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de00d7070 con 0x5557402a1fa0 2024-04-17T07:29:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.681+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3376894870 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1de00bf130 con 0x5557402a1fa0 2024-04-17T07:29:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.681+0000 7f1df8186740 1 -- 172.21.15.86:0/3376894870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555740215a40 con 0x5557402a1fa0 2024-04-17T07:29:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.682+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3376894870 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1de00be620 con 0x5557402a1fa0 2024-04-17T07:29:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.682+0000 7f1df8186740 10 client.18963 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.682+0000 7f1df8186740 1 -- 172.21.15.86:0/3376894870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557401937f0 con 0x5557402a1fa0 2024-04-17T07:29:14.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.683+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3376894870 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555740215a40 con 0x5557402a1fa0 2024-04-17T07:29:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.683+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3376894870 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1de00c74a0 con 0x5557402a1fa0 2024-04-17T07:29:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.683+0000 7f1df8186740 10 client.18963 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.683+0000 7f1df8186740 10 client.18963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.683+0000 7f1df8186740 20 client.18963 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.683+0000 7f1df8186740 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557402b7990 0x5557402b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.683+0000 7f1df8186740 4 client.18963 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.683+0000 7f1df8186740 1 -- 172.21.15.86:0/3376894870 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55574020f8b0 con 0x5557402b7990 2024-04-17T07:29:14.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.684+0000 7f1dede00640 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557402b7990 0x5557402b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.687+0000 7f1dede00640 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557402b7990 0x5557402b9dc0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb439ff63cb824 server_cookie=aa1ce327f499dd3f in_seq=0 out_seq=0 2024-04-17T07:29:14.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.687+0000 7f1dcfe00640 10 client.18963 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:14.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.689+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3376894870 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55574020f8b0 con 0x5557402b7990 2024-04-17T07:29:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.689+0000 7f1dcfe00640 10 client.18963 handle_command_reply: tid=0 2024-04-17T07:29:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.690+0000 7f1df8186740 1 client.18963 shutdown 2024-04-17T07:29:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.690+0000 7f1df8186740 1 -- 172.21.15.86:0/3376894870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 msgr2=0x5557402ac210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.690+0000 7f1df8186740 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402ac210 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1de00be9c0 tx=0x7f1de00df300 comp rx=0 tx=0).stop 2024-04-17T07:29:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.691+0000 7f1df8186740 1 -- 172.21.15.86:0/3376894870 shutdown_connections 2024-04-17T07:29:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.691+0000 7f1df8186740 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557402b7990 0x5557402b9dc0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.691+0000 7f1df8186740 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 0x5557402b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.691+0000 7f1df8186740 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a2990 0x5557402ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.691+0000 7f1df8186740 1 --2- 172.21.15.86:0/3376894870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402ac210 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.691+0000 7f1df8186740 1 -- 172.21.15.86:0/3376894870 >> 172.21.15.86:0/3376894870 conn(0x55574029d510 msgr2=0x55574029c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.691+0000 7f1dcfe00640 0 client.18963 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:14.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.691+0000 7f1df8186740 1 -- 172.21.15.86:0/3376894870 shutdown_connections 2024-04-17T07:29:14.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.691+0000 7f1df8186740 1 -- 172.21.15.86:0/3376894870 wait complete. 2024-04-17T07:29:14.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.692+0000 7f1df8186740 20 client.18963 trim_cache size 0 max 16384 2024-04-17T07:29:14.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.718+0000 7f1df8186740 1 Processor -- start 2024-04-17T07:29:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.718+0000 7f1df8186740 1 -- start start 2024-04-17T07:29:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.719+0000 7f1df8186740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a1fa0 0x5557402a3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.719+0000 7f1df8186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a2990 0x5557402b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.719+0000 7f1df8186740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 0x5557402ac530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.719+0000 7f1df8186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01b0f70 con 0x5557402a2990 2024-04-17T07:29:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.719+0000 7f1df8186740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01b2ab0 con 0x5557402a1fa0 2024-04-17T07:29:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.719+0000 7f1df8186740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01b1640 con 0x5557402a7b00 2024-04-17T07:29:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.719+0000 7f1dede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a2990 0x5557402b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.719+0000 7f1dede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a2990 0x5557402b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34514/0 (socket says 172.21.15.86:34514) 2024-04-17T07:29:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.719+0000 7f1dede00640 1 -- 172.21.15.86:0/376710924 learned_addr learned my addr 172.21.15.86:0/376710924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.719+0000 7f1def200640 1 --2- 172.21.15.86:0/376710924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 0x5557402ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.720+0000 7f1def200640 1 -- 172.21.15.86:0/376710924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a1fa0 msgr2=0x5557402a3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.720+0000 7f1def200640 1 --2- 172.21.15.86:0/376710924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a1fa0 0x5557402a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.720+0000 7f1def200640 1 -- 172.21.15.86:0/376710924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a2990 msgr2=0x5557402b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.720+0000 7f1def200640 1 --2- 172.21.15.86:0/376710924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a2990 0x5557402b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.720+0000 7f1def200640 1 -- 172.21.15.86:0/376710924 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55573ff1a3f0 con 0x5557402a7b00 2024-04-17T07:29:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.720+0000 7f1def200640 1 --2- 172.21.15.86:0/376710924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 0x5557402ac530 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005fd0 tx=0x7f1de8012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bdb76d0bb3f64c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.721+0000 7f1ddaa00640 1 -- 172.21.15.86:0/376710924 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8003b70 con 0x5557402a7b00 2024-04-17T07:29:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.721+0000 7f1ddaa00640 1 -- 172.21.15.86:0/376710924 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1de8003d10 con 0x5557402a7b00 2024-04-17T07:29:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.721+0000 7f1ddaa00640 1 -- 172.21.15.86:0/376710924 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8003b70 con 0x5557402a7b00 2024-04-17T07:29:14.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.722+0000 7f1df8186740 1 -- 172.21.15.86:0/376710924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 msgr2=0x5557402ac530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.722+0000 7f1df8186740 1 --2- 172.21.15.86:0/376710924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 0x5557402ac530 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005fd0 tx=0x7f1de8012820 comp rx=0 tx=0).stop 2024-04-17T07:29:14.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.723+0000 7f1df8186740 1 -- 172.21.15.86:0/376710924 shutdown_connections 2024-04-17T07:29:14.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.723+0000 7f1df8186740 1 --2- 172.21.15.86:0/376710924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a7b00 0x5557402ac530 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.723+0000 7f1df8186740 1 --2- 172.21.15.86:0/376710924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a2990 0x5557402b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.723+0000 7f1df8186740 1 --2- 172.21.15.86:0/376710924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a1fa0 0x5557402a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.723+0000 7f1df8186740 1 -- 172.21.15.86:0/376710924 >> 172.21.15.86:0/376710924 conn(0x55574029d510 msgr2=0x5557402abe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.723+0000 7f1df8186740 1 -- 172.21.15.86:0/376710924 shutdown_connections 2024-04-17T07:29:14.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.723+0000 7f1df8186740 1 -- 172.21.15.86:0/376710924 wait complete. 2024-04-17T07:29:14.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.724+0000 7f1df8186740 1 -- start start 2024-04-17T07:29:14.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.724+0000 7f1df8186740 1 Processor -- start 2024-04-17T07:29:14.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.725+0000 7f1df8186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.726+0000 7f1df8186740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 0x5557402b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.726+0000 7f1df8186740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a7b00 0x5557402b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.726+0000 7f1df8186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01b23e0 con 0x5557402a1fa0 2024-04-17T07:29:14.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.726+0000 7f1df8186740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01b1d10 con 0x5557402a7b00 2024-04-17T07:29:14.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.726+0000 7f1df8186740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01aef40 con 0x5557402a2990 2024-04-17T07:29:14.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.726+0000 7f1df8186740 10 client.0 _ll_register_callbacks cb 0x7f1de00f6f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:14.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.726+0000 7f1dee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.726+0000 7f1dee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34522/0 (socket says 172.21.15.86:34522) 2024-04-17T07:29:14.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.726+0000 7f1dee800640 1 -- 172.21.15.86:0/3322920543 learned_addr learned my addr 172.21.15.86:0/3322920543 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:14.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.727+0000 7f1def200640 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a7b00 0x5557402b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.727+0000 7f1dee800640 1 -- 172.21.15.86:0/3322920543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 msgr2=0x5557402b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.727+0000 7f1dee800640 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 0x5557402b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.727+0000 7f1dee800640 1 -- 172.21.15.86:0/3322920543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a7b00 msgr2=0x5557402b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:14.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.727+0000 7f1dee800640 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a7b00 0x5557402b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.727+0000 7f1dee800640 1 -- 172.21.15.86:0/3322920543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55573ffe47d0 con 0x5557402a1fa0 2024-04-17T07:29:14.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.728+0000 7f1dee800640 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402b0a40 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1de0004650 tx=0x7f1de00df2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6578a56e11af851 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:14.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.728+0000 7f1dcfe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:14.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.729+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3322920543 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de00d7070 con 0x5557402a1fa0 2024-04-17T07:29:14.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.729+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3322920543 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1de00c7b80 con 0x5557402a1fa0 2024-04-17T07:29:14.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.730+0000 7f1df8186740 1 -- 172.21.15.86:0/3322920543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1de8003d10 con 0x5557402a1fa0 2024-04-17T07:29:14.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.730+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3322920543 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1de00be800 con 0x5557402a1fa0 2024-04-17T07:29:14.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.730+0000 7f1df8186740 10 client.18969 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:14.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.730+0000 7f1df8186740 1 -- 172.21.15.86:0/3322920543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55573ffe47d0 con 0x5557402a1fa0 2024-04-17T07:29:14.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.731+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3322920543 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1de8003d10 con 0x5557402a1fa0 2024-04-17T07:29:14.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.731+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3322920543 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1de00be110 con 0x5557402a1fa0 2024-04-17T07:29:14.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.731+0000 7f1df8186740 10 client.18969 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:14.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.732+0000 7f1df8186740 10 client.18969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:14.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.732+0000 7f1df8186740 20 client.18969 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:14.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.732+0000 7f1df8186740 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55574032ef50 0x555740331340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:14.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.732+0000 7f1df8186740 4 client.18969 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:29:14.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.732+0000 7f1df8186740 1 -- 172.21.15.86:0/3322920543 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55574018b570 con 0x55574032ef50 2024-04-17T07:29:14.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.733+0000 7f1dede00640 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55574032ef50 0x555740331340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:14.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.735+0000 7f1dede00640 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55574032ef50 0x555740331340 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d602049ae59103 server_cookie=c01be5abce9a7b9e in_seq=0 out_seq=0 2024-04-17T07:29:14.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.735+0000 7f1dcfe00640 10 client.18969 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:14.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.737+0000 7f1dcfe00640 1 -- 172.21.15.86:0/3322920543 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55574018b570 con 0x55574032ef50 2024-04-17T07:29:14.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.737+0000 7f1dcfe00640 10 client.18969 handle_command_reply: tid=0 2024-04-17T07:29:14.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.737+0000 7f1df8186740 1 client.18969 shutdown 2024-04-17T07:29:14.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.737+0000 7f1df8186740 1 -- 172.21.15.86:0/3322920543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 msgr2=0x5557402b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.737+0000 7f1df8186740 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402b0a40 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1de0004650 tx=0x7f1de00df2e0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.738+0000 7f1df8186740 1 -- 172.21.15.86:0/3322920543 shutdown_connections 2024-04-17T07:29:14.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.738+0000 7f1df8186740 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55574032ef50 0x555740331340 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.738+0000 7f1df8186740 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557402a7b00 0x5557402b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.738+0000 7f1df8186740 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557402a2990 0x5557402b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.738+0000 7f1df8186740 1 --2- 172.21.15.86:0/3322920543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557402a1fa0 0x5557402b0a40 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.738+0000 7f1df8186740 1 -- 172.21.15.86:0/3322920543 >> 172.21.15.86:0/3322920543 conn(0x55574029d510 msgr2=0x5557402a0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.738+0000 7f1dcfe00640 0 client.18969 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.738+0000 7f1df8186740 1 -- 172.21.15.86:0/3322920543 shutdown_connections 2024-04-17T07:29:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.738+0000 7f1df8186740 1 -- 172.21.15.86:0/3322920543 wait complete. 2024-04-17T07:29:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.738+0000 7f1df8186740 20 client.18969 trim_cache size 0 max 16384 2024-04-17T07:29:14.739 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:29:14.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.742+0000 7f1df5400640 1 -- 172.21.15.86:0/3476255770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dd40a3d80 msgr2=0x7f1dd40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.742+0000 7f1df5400640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dd40a3d80 0x7f1dd40a6240 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1df0100bc0 tx=0x7f1ddc005f90 comp rx=0 tx=0).stop 2024-04-17T07:29:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.742+0000 7f1df5400640 1 -- 172.21.15.86:0/3476255770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df0100ea0 msgr2=0x7f1df019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.742+0000 7f1df5400640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df0100ea0 0x7f1df019e5c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1de0036040 tx=0x7f1de00042f0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.743+0000 7f1df5400640 1 -- 172.21.15.86:0/3476255770 shutdown_connections 2024-04-17T07:29:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.743+0000 7f1df5400640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1dd40a3d80 0x7f1dd40a6240 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.743+0000 7f1df5400640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df01029b0 0x7f1df01a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.743+0000 7f1df5400640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1df0101ff0 0x7f1df019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.743+0000 7f1df5400640 1 --2- 172.21.15.86:0/3476255770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1df0100ea0 0x7f1df019e5c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.743+0000 7f1df5400640 1 -- 172.21.15.86:0/3476255770 >> 172.21.15.86:0/3476255770 conn(0x7f1df00fc410 msgr2=0x7f1df010e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:14.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.744+0000 7f1df5400640 1 -- 172.21.15.86:0/3476255770 shutdown_connections 2024-04-17T07:29:14.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:14.744+0000 7f1df5400640 1 -- 172.21.15.86:0/3476255770 wait complete. 2024-04-17T07:29:15.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:14 smithi086 ceph-mon[31733]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.7 MiB/s wr, 707 op/s 2024-04-17T07:29:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:14 smithi098 ceph-mon[35418]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.7 MiB/s wr, 707 op/s 2024-04-17T07:29:15.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:14 smithi103 ceph-mon[35495]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.7 MiB/s wr, 707 op/s 2024-04-17T07:29:15.785 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:29:15.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:29:15.786 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:29:16.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.157+0000 7f29a2400640 1 -- 172.21.15.86:0/3913638166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29940094f0 msgr2=0x7f2994009910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.157+0000 7f29a2400640 1 --2- 172.21.15.86:0/3913638166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29940094f0 0x7f2994009910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2990009860 tx=0x7f299002bb70 comp rx=0 tx=0).stop 2024-04-17T07:29:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.157+0000 7f29a2400640 1 -- 172.21.15.86:0/3913638166 shutdown_connections 2024-04-17T07:29:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.157+0000 7f29a2400640 1 --2- 172.21.15.86:0/3913638166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f299400b0d0 0x7f299400b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.157+0000 7f29a2400640 1 --2- 172.21.15.86:0/3913638166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f299400a730 0x7f299400ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.157+0000 7f29a2400640 1 --2- 172.21.15.86:0/3913638166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29940094f0 0x7f2994009910 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.157+0000 7f29a2400640 1 -- 172.21.15.86:0/3913638166 >> 172.21.15.86:0/3913638166 conn(0x7f2994093520 msgr2=0x7f2994095940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.157+0000 7f29a2400640 1 -- 172.21.15.86:0/3913638166 shutdown_connections 2024-04-17T07:29:16.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.158+0000 7f29a2400640 1 -- 172.21.15.86:0/3913638166 wait complete. 2024-04-17T07:29:16.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.160+0000 7f29a2400640 1 Processor -- start 2024-04-17T07:29:16.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.160+0000 7f29a2400640 1 -- start start 2024-04-17T07:29:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.162+0000 7f29a2400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29940094f0 0x7f2994129e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.162+0000 7f29a2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f299400a730 0x7f299412a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.162+0000 7f29a2400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f299400b0d0 0x7f29941313b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.162+0000 7f29a2400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29940996b0 con 0x7f299400a730 2024-04-17T07:29:16.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.162+0000 7f29a2400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2994099530 con 0x7f299400b0d0 2024-04-17T07:29:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.162+0000 7f29a2400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2994099830 con 0x7f29940094f0 2024-04-17T07:29:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f29a1a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f299400b0d0 0x7f29941313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f29a1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29940094f0 0x7f2994129e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f29a1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29940094f0 0x7f2994129e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44802/0 (socket says 172.21.15.86:44802) 2024-04-17T07:29:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f29a1a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f299400b0d0 0x7f29941313b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50840/0 (socket says 172.21.15.86:50840) 2024-04-17T07:29:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f29a1000640 1 -- 172.21.15.86:0/184622782 learned_addr learned my addr 172.21.15.86:0/184622782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f299be00640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f299400a730 0x7f299412a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f299be00640 1 -- 172.21.15.86:0/184622782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29940094f0 msgr2=0x7f2994129e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f299be00640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29940094f0 0x7f2994129e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f299be00640 1 -- 172.21.15.86:0/184622782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f299400b0d0 msgr2=0x7f29941313b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f299be00640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f299400b0d0 0x7f29941313b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.163+0000 7f299be00640 1 -- 172.21.15.86:0/184622782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2994131a20 con 0x7f299400a730 2024-04-17T07:29:16.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.164+0000 7f299be00640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f299400a730 0x7f299412a340 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f298800b3f0 tx=0x7f298800b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.165+0000 7f2999600640 1 -- 172.21.15.86:0/184622782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988015020 con 0x7f299400a730 2024-04-17T07:29:16.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.165+0000 7f2999600640 1 -- 172.21.15.86:0/184622782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2988011ce0 con 0x7f299400a730 2024-04-17T07:29:16.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.165+0000 7f2999600640 1 -- 172.21.15.86:0/184622782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988010d50 con 0x7f299400a730 2024-04-17T07:29:16.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.165+0000 7f29a2400640 1 -- 172.21.15.86:0/184622782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2994131d10 con 0x7f299400a730 2024-04-17T07:29:16.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.166+0000 7f29a2400640 1 -- 172.21.15.86:0/184622782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29941322d0 con 0x7f299400a730 2024-04-17T07:29:16.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.173+0000 7f29a662f740 1 Processor -- start 2024-04-17T07:29:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.175+0000 7f29a662f740 1 -- start start 2024-04-17T07:29:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.176+0000 7f29a662f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.176+0000 7f29a662f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb389a90 0x5600fb38e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.176+0000 7f29a662f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 0x5600fb391090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.176+0000 7f29a662f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5600fb2eecb0 con 0x5600fb38ec00 2024-04-17T07:29:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.176+0000 7f29a662f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5600fb295370 con 0x5600fb389a90 2024-04-17T07:29:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.176+0000 7f29a662f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600fb2eac70 con 0x5600fb3890a0 2024-04-17T07:29:16.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.176+0000 7f2999600640 1 -- 172.21.15.86:0/184622782 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2988018020 con 0x7f299400a730 2024-04-17T07:29:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.177+0000 7f29a1000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb389a90 0x5600fb38e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.177+0000 7f29a1000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb389a90 0x5600fb38e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50856/0 (socket says 172.21.15.86:50856) 2024-04-17T07:29:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.177+0000 7f29a1000640 1 -- 172.21.15.86:0/2614565140 learned_addr learned my addr 172.21.15.86:0/2614565140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.178+0000 7f29a1a00640 1 --2- 172.21.15.86:0/2614565140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.178+0000 7f29a1000640 1 -- 172.21.15.86:0/2614565140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 msgr2=0x5600fb3894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.178+0000 7f29a1000640 1 --2- 172.21.15.86:0/2614565140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3894c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.179+0000 7f299be00640 1 --2- 172.21.15.86:0/2614565140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 0x5600fb391090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.179+0000 7f29a1000640 1 -- 172.21.15.86:0/2614565140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 msgr2=0x5600fb391090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.179+0000 7f29a1000640 1 --2- 172.21.15.86:0/2614565140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 0x5600fb391090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.179+0000 7f29a1000640 1 -- 172.21.15.86:0/2614565140 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fb2b4910 con 0x5600fb389a90 2024-04-17T07:29:16.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.179+0000 7f299be00640 1 --2- 172.21.15.86:0/2614565140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 0x5600fb391090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.181+0000 7f2999600640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29840a3cb0 0x7f29840a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.181+0000 7f2999600640 1 -- 172.21.15.86:0/184622782 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f29880cfe20 con 0x7f299400a730 2024-04-17T07:29:16.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.183+0000 7f29a1a00640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29840a3cb0 0x7f29840a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.183+0000 7f29a1000640 1 --2- 172.21.15.86:0/2614565140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb389a90 0x5600fb38e590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f29900344c0 tx=0x7f29900344f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b081711f459d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.184+0000 7f298e000640 1 -- 172.21.15.86:0/2614565140 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990039040 con 0x5600fb389a90 2024-04-17T07:29:16.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.184+0000 7f298e000640 1 -- 172.21.15.86:0/2614565140 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2990043070 con 0x5600fb389a90 2024-04-17T07:29:16.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.184+0000 7f298e000640 1 -- 172.21.15.86:0/2614565140 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990005600 con 0x5600fb389a90 2024-04-17T07:29:16.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.186+0000 7f29a662f740 1 -- 172.21.15.86:0/2614565140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb389a90 msgr2=0x5600fb38e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.186+0000 7f29a662f740 1 --2- 172.21.15.86:0/2614565140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb389a90 0x5600fb38e590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f29900344c0 tx=0x7f29900344f0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.186+0000 7f29a1a00640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29840a3cb0 0x7f29840a6170 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f299c067730 tx=0x7f299c06c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.188+0000 7f29a662f740 1 -- 172.21.15.86:0/2614565140 shutdown_connections 2024-04-17T07:29:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.188+0000 7f29a662f740 1 --2- 172.21.15.86:0/2614565140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 0x5600fb391090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.188+0000 7f29a662f740 1 --2- 172.21.15.86:0/2614565140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb389a90 0x5600fb38e590 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.188+0000 7f29a662f740 1 --2- 172.21.15.86:0/2614565140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3894c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.188+0000 7f29a662f740 1 -- 172.21.15.86:0/2614565140 >> 172.21.15.86:0/2614565140 conn(0x5600fb384610 msgr2=0x5600fb384a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.189+0000 7f29a662f740 1 -- 172.21.15.86:0/2614565140 shutdown_connections 2024-04-17T07:29:16.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.189+0000 7f29a662f740 1 -- 172.21.15.86:0/2614565140 wait complete. 2024-04-17T07:29:16.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.190+0000 7f29a662f740 1 -- start start 2024-04-17T07:29:16.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.191+0000 7f29a662f740 1 Processor -- start 2024-04-17T07:29:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.191+0000 7f29a662f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb3890a0 0x5600fb3a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.191+0000 7f29a662f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb389a90 0x5600fb3a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.191+0000 7f29a662f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 0x5600fb3abb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.191+0000 7f29a662f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5600fb3a2b60 con 0x5600fb38ec00 2024-04-17T07:29:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.191+0000 7f29a662f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5600fb2a8730 con 0x5600fb3890a0 2024-04-17T07:29:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.191+0000 7f29a662f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5600fb3a2ce0 con 0x5600fb389a90 2024-04-17T07:29:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.191+0000 7f29a662f740 10 client.0 _ll_register_callbacks cb 0x5600fb2aab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.192+0000 7f299be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb389a90 0x5600fb3a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.192+0000 7f299be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb389a90 0x5600fb3a4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44818/0 (socket says 172.21.15.86:44818) 2024-04-17T07:29:16.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.192+0000 7f299be00640 1 -- 172.21.15.86:0/1450673159 learned_addr learned my addr 172.21.15.86:0/1450673159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:16.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.193+0000 7f299be00640 1 -- 172.21.15.86:0/1450673159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb3890a0 msgr2=0x5600fb3a45d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:29:16.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.193+0000 7f299be00640 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb3890a0 0x5600fb3a45d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.193+0000 7f299be00640 1 -- 172.21.15.86:0/1450673159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 msgr2=0x5600fb3abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.193+0000 7f29a1a00640 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 0x5600fb3abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.193+0000 7f299be00640 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 0x5600fb3abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.193+0000 7f299be00640 1 -- 172.21.15.86:0/1450673159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fb0c97e0 con 0x5600fb389a90 2024-04-17T07:29:16.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.194+0000 7f29a1a00640 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 0x5600fb3abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.195+0000 7f299be00640 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb389a90 0x5600fb3a4b10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2988098180 tx=0x7f298809b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5aa017821e82fee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.195+0000 7f2982a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.195+0000 7f2982a00640 1 -- 172.21.15.86:0/1450673159 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29880b5060 con 0x5600fb389a90 2024-04-17T07:29:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.195+0000 7f2982a00640 1 -- 172.21.15.86:0/1450673159 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f29880b5200 con 0x5600fb389a90 2024-04-17T07:29:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.196+0000 7f29a662f740 1 -- 172.21.15.86:0/1450673159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2990043070 con 0x5600fb389a90 2024-04-17T07:29:16.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.196+0000 7f2982a00640 1 -- 172.21.15.86:0/1450673159 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29880ba050 con 0x5600fb389a90 2024-04-17T07:29:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.196+0000 7f29a662f740 10 client.27955 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.196+0000 7f29a662f740 1 -- 172.21.15.86:0/1450673159 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5600fb129f70 con 0x5600fb389a90 2024-04-17T07:29:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.197+0000 7f2982a00640 1 -- 172.21.15.86:0/1450673159 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2990043070 con 0x5600fb389a90 2024-04-17T07:29:16.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.197+0000 7f2982a00640 1 -- 172.21.15.86:0/1450673159 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2988098390 con 0x5600fb389a90 2024-04-17T07:29:16.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.197+0000 7f29a662f740 10 client.27955 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:16.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.197+0000 7f29a662f740 10 client.27955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:16.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.197+0000 7f29a662f740 20 client.27955 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:16.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.197+0000 7f29a662f740 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600fb3b0470 0x5600fb3b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.198+0000 7f29a1000640 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600fb3b0470 0x5600fb3b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.199+0000 7f29a662f740 4 client.27955 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:16.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.199+0000 7f29a662f740 1 -- 172.21.15.86:0/1450673159 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600fb2fc6d0 con 0x5600fb3b0470 2024-04-17T07:29:16.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.201+0000 7f29a1000640 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600fb3b0470 0x5600fb3b2930 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7158d0d0b688ca47 server_cookie=484c602cfb4c24c4 in_seq=0 out_seq=0 2024-04-17T07:29:16.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.201+0000 7f2982a00640 10 client.27955 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:16.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.202+0000 7f2982a00640 1 -- 172.21.15.86:0/1450673159 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600fb2fc6d0 con 0x5600fb3b0470 2024-04-17T07:29:16.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.202+0000 7f2982a00640 10 client.27955 handle_command_reply: tid=0 2024-04-17T07:29:16.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.202+0000 7f29a662f740 1 client.27955 shutdown 2024-04-17T07:29:16.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.203+0000 7f29a662f740 1 -- 172.21.15.86:0/1450673159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb389a90 msgr2=0x5600fb3a4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.203+0000 7f29a662f740 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb389a90 0x5600fb3a4b10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2988098180 tx=0x7f298809b860 comp rx=0 tx=0).stop 2024-04-17T07:29:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.205+0000 7f29a662f740 1 -- 172.21.15.86:0/1450673159 shutdown_connections 2024-04-17T07:29:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.205+0000 7f29a662f740 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600fb3b0470 0x5600fb3b2930 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.205+0000 7f29a662f740 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb38ec00 0x5600fb3abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.205+0000 7f29a662f740 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb389a90 0x5600fb3a4b10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.205+0000 7f29a662f740 1 --2- 172.21.15.86:0/1450673159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb3890a0 0x5600fb3a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.205+0000 7f29a662f740 1 -- 172.21.15.86:0/1450673159 >> 172.21.15.86:0/1450673159 conn(0x5600fb384610 msgr2=0x5600fb383990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.205+0000 7f2982a00640 0 client.27955 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:16.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.205+0000 7f29a662f740 1 -- 172.21.15.86:0/1450673159 shutdown_connections 2024-04-17T07:29:16.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.206+0000 7f29a662f740 1 -- 172.21.15.86:0/1450673159 wait complete. 2024-04-17T07:29:16.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.206+0000 7f29a662f740 20 client.27955 trim_cache size 0 max 16384 2024-04-17T07:29:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.250+0000 7f29a662f740 1 Processor -- start 2024-04-17T07:29:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.250+0000 7f29a662f740 1 -- start start 2024-04-17T07:29:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.250+0000 7f29a662f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.250+0000 7f29a662f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 0x5600fb3abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.251+0000 7f29a662f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 0x5600fb383100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.251+0000 7f29a662f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f299413c710 con 0x5600fb389a90 2024-04-17T07:29:16.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.251+0000 7f29a662f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f299413e250 con 0x5600fb38ec00 2024-04-17T07:29:16.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.251+0000 7f29a662f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f299413cde0 con 0x5600fb3890a0 2024-04-17T07:29:16.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.251+0000 7f29a1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.251+0000 7f29a1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44820/0 (socket says 172.21.15.86:44820) 2024-04-17T07:29:16.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.251+0000 7f29a1000640 1 -- 172.21.15.86:0/2945922007 learned_addr learned my addr 172.21.15.86:0/2945922007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:16.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.252+0000 7f299be00640 1 --2- 172.21.15.86:0/2945922007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 0x5600fb3abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.252+0000 7f29a1a00640 1 --2- 172.21.15.86:0/2945922007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 0x5600fb383100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.252+0000 7f299be00640 1 -- 172.21.15.86:0/2945922007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 msgr2=0x5600fb3a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.252+0000 7f299be00640 1 --2- 172.21.15.86:0/2945922007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.252+0000 7f299be00640 1 -- 172.21.15.86:0/2945922007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 msgr2=0x5600fb383100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.252+0000 7f299be00640 1 --2- 172.21.15.86:0/2945922007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 0x5600fb383100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.252+0000 7f299be00640 1 -- 172.21.15.86:0/2945922007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fafc1170 con 0x5600fb389a90 2024-04-17T07:29:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.253+0000 7f299be00640 1 --2- 172.21.15.86:0/2945922007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 0x5600fb3abb10 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f298809bcb0 tx=0x7f2988010380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524b61cb5cd6c256 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.253+0000 7f298d600640 1 -- 172.21.15.86:0/2945922007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298807f080 con 0x5600fb389a90 2024-04-17T07:29:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.253+0000 7f298d600640 1 -- 172.21.15.86:0/2945922007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f298809c420 con 0x5600fb389a90 2024-04-17T07:29:16.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.254+0000 7f298d600640 1 -- 172.21.15.86:0/2945922007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298807f080 con 0x5600fb389a90 2024-04-17T07:29:16.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.255+0000 7f29a662f740 1 -- 172.21.15.86:0/2945922007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 msgr2=0x5600fb3abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.255+0000 7f29a662f740 1 --2- 172.21.15.86:0/2945922007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 0x5600fb3abb10 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f298809bcb0 tx=0x7f2988010380 comp rx=0 tx=0).stop 2024-04-17T07:29:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.259+0000 7f29a662f740 1 -- 172.21.15.86:0/2945922007 shutdown_connections 2024-04-17T07:29:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.259+0000 7f29a662f740 1 --2- 172.21.15.86:0/2945922007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 0x5600fb383100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.259+0000 7f29a662f740 1 --2- 172.21.15.86:0/2945922007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 0x5600fb3abb10 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.259+0000 7f29a662f740 1 --2- 172.21.15.86:0/2945922007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.259+0000 7f29a662f740 1 -- 172.21.15.86:0/2945922007 >> 172.21.15.86:0/2945922007 conn(0x5600fb384610 msgr2=0x5600fb3a38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.259+0000 7f29a662f740 1 -- 172.21.15.86:0/2945922007 shutdown_connections 2024-04-17T07:29:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.259+0000 7f29a662f740 1 -- 172.21.15.86:0/2945922007 wait complete. 2024-04-17T07:29:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.260+0000 7f29a662f740 1 -- start start 2024-04-17T07:29:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.260+0000 7f29a662f740 1 Processor -- start 2024-04-17T07:29:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.262+0000 7f29a662f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.262+0000 7f29a662f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 0x5600fb3a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.262+0000 7f29a662f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 0x5600fb3aa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.262+0000 7f29a1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.263+0000 7f29a1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44830/0 (socket says 172.21.15.86:44830) 2024-04-17T07:29:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.263+0000 7f29a1000640 1 -- 172.21.15.86:0/511643661 learned_addr learned my addr 172.21.15.86:0/511643661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.263+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f299413db80 con 0x5600fb389a90 2024-04-17T07:29:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.263+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f299413d4b0 con 0x5600fb38ec00 2024-04-17T07:29:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.263+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f299413a6e0 con 0x5600fb3890a0 2024-04-17T07:29:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.263+0000 7f299be00640 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 0x5600fb3a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.263+0000 7f29a1a00640 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 0x5600fb3aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.263+0000 7f29a662f740 10 client.0 _ll_register_callbacks cb 0x7f2988010600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.263+0000 7f29a1000640 1 -- 172.21.15.86:0/511643661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 msgr2=0x5600fb3aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.265+0000 7f29a1000640 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 0x5600fb3aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.265+0000 7f29a1000640 1 -- 172.21.15.86:0/511643661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 msgr2=0x5600fb3a95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.265+0000 7f29a1000640 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 0x5600fb3a95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.265+0000 7f29a1000640 1 -- 172.21.15.86:0/511643661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fb3aab80 con 0x5600fb3890a0 2024-04-17T07:29:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.265+0000 7f29a1a00640 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 0x5600fb3aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.265+0000 7f299be00640 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 0x5600fb3a95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.266+0000 7f29a1000640 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a9090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2990005e00 tx=0x7f299002bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40fa526a608959e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.266+0000 7f2982a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:16.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.267+0000 7f2982a00640 1 -- 172.21.15.86:0/511643661 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990039040 con 0x5600fb3890a0 2024-04-17T07:29:16.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.267+0000 7f2982a00640 1 -- 172.21.15.86:0/511643661 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f299004eb50 con 0x5600fb3890a0 2024-04-17T07:29:16.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.268+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298809c420 con 0x5600fb3890a0 2024-04-17T07:29:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.272+0000 7f2982a00640 1 -- 172.21.15.86:0/511643661 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f299004ed50 con 0x5600fb3890a0 2024-04-17T07:29:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.272+0000 7f29a662f740 10 client.27961 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.272+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5600fb416ab0 con 0x5600fb3890a0 2024-04-17T07:29:16.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.273+0000 7f2982a00640 1 -- 172.21.15.86:0/511643661 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f298809c420 con 0x5600fb3890a0 2024-04-17T07:29:16.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.276+0000 7f2982a00640 1 -- 172.21.15.86:0/511643661 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f299004e4b0 con 0x5600fb3890a0 2024-04-17T07:29:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.276+0000 7f29a662f740 10 client.27961 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.276+0000 7f29a662f740 10 client.27961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.276+0000 7f29a662f740 20 client.27961 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.276+0000 7f29a662f740 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600fb416f20 0x5600fb419340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.276+0000 7f29a662f740 4 client.27961 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:29:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.276+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5600fb2f6540 con 0x5600fb416f20 2024-04-17T07:29:16.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.277+0000 7f299be00640 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600fb416f20 0x5600fb419340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.278+0000 7f2982a00640 10 client.27961 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:16.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.278+0000 7f299be00640 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600fb416f20 0x5600fb419340 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3034d1c2f6d0707 server_cookie=f95f7995f0d4b94a in_seq=0 out_seq=0 2024-04-17T07:29:16.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.280+0000 7f2982a00640 1 -- 172.21.15.86:0/511643661 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5600fb2f6540 con 0x5600fb416f20 2024-04-17T07:29:16.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.280+0000 7f2982a00640 10 client.27961 handle_command_reply: tid=0 2024-04-17T07:29:16.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.280+0000 7f29a662f740 1 client.27961 shutdown 2024-04-17T07:29:16.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.281+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 msgr2=0x5600fb3a9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.281+0000 7f29a662f740 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a9090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2990005e00 tx=0x7f299002bb70 comp rx=0 tx=0).stop 2024-04-17T07:29:16.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.281+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 shutdown_connections 2024-04-17T07:29:16.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.281+0000 7f29a662f740 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5600fb416f20 0x5600fb419340 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.281+0000 7f2982a00640 0 client.27961 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:16.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.281+0000 7f29a662f740 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5600fb38ec00 0x5600fb3aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.281+0000 7f29a662f740 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5600fb389a90 0x5600fb3a95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.281+0000 7f29a662f740 1 --2- 172.21.15.86:0/511643661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5600fb3890a0 0x5600fb3a9090 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.281+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 >> 172.21.15.86:0/511643661 conn(0x5600fb384610 msgr2=0x5600fb387870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.282+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 shutdown_connections 2024-04-17T07:29:16.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.282+0000 7f29a662f740 1 -- 172.21.15.86:0/511643661 wait complete. 2024-04-17T07:29:16.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.282+0000 7f29a662f740 20 client.27961 trim_cache size 0 max 16384 2024-04-17T07:29:16.283 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:29:16.283 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:29:16.283 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "18fd36b2-2f2d-4423-af53-cf5abbdbaa41", 2024-04-17T07:29:16.283 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:29:16.283 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:29:16.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.283+0000 7f29a2400640 1 -- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29840a3cb0 msgr2=0x7f29840a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.283+0000 7f29a2400640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29840a3cb0 0x7f29840a6170 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f299c067730 tx=0x7f299c06c040 comp rx=0 tx=0).stop 2024-04-17T07:29:16.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.283+0000 7f29a2400640 1 -- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f299400a730 msgr2=0x7f299412a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.283+0000 7f29a2400640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f299400a730 0x7f299412a340 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f298800b3f0 tx=0x7f298800b7f0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.284+0000 7f29a2400640 1 -- 172.21.15.86:0/184622782 shutdown_connections 2024-04-17T07:29:16.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.284+0000 7f29a2400640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f29840a3cb0 0x7f29840a6170 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.284+0000 7f29a2400640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f299400b0d0 0x7f29941313b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.284+0000 7f29a2400640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f299400a730 0x7f299412a340 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.284+0000 7f29a2400640 1 --2- 172.21.15.86:0/184622782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29940094f0 0x7f2994129e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.284+0000 7f29a2400640 1 -- 172.21.15.86:0/184622782 >> 172.21.15.86:0/184622782 conn(0x7f2994093520 msgr2=0x7f2994094c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.284+0000 7f29a2400640 1 -- 172.21.15.86:0/184622782 shutdown_connections 2024-04-17T07:29:16.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.284+0000 7f29a2400640 1 -- 172.21.15.86:0/184622782 wait complete. 2024-04-17T07:29:16.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:29:16.315 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:29:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.691+0000 7f7a91200640 1 -- 172.21.15.86:0/822756624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c076920 msgr2=0x7f7a8c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.691+0000 7f7a91200640 1 --2- 172.21.15.86:0/822756624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c076920 0x7f7a8c076d40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009900 tx=0x7f7a7c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:29:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.691+0000 7f7a91200640 1 -- 172.21.15.86:0/822756624 shutdown_connections 2024-04-17T07:29:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.691+0000 7f7a91200640 1 --2- 172.21.15.86:0/822756624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c078500 0x7f7a8c078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.691+0000 7f7a91200640 1 --2- 172.21.15.86:0/822756624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8c077b60 0x7f7a8c077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.691+0000 7f7a91200640 1 --2- 172.21.15.86:0/822756624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c076920 0x7f7a8c076d40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.691+0000 7f7a91200640 1 -- 172.21.15.86:0/822756624 >> 172.21.15.86:0/822756624 conn(0x7f7a8c0ff750 msgr2=0x7f7a8c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.692+0000 7f7a91200640 1 -- 172.21.15.86:0/822756624 shutdown_connections 2024-04-17T07:29:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.692+0000 7f7a91200640 1 -- 172.21.15.86:0/822756624 wait complete. 2024-04-17T07:29:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.693+0000 7f7a91200640 1 Processor -- start 2024-04-17T07:29:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.693+0000 7f7a91200640 1 -- start start 2024-04-17T07:29:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.694+0000 7f7a91200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8c076920 0x7f7a8c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.694+0000 7f7a91200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c077b60 0x7f7a8c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.694+0000 7f7a91200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c078500 0x7f7a8c1a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.694+0000 7f7a91200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c114140 con 0x7f7a8c077b60 2024-04-17T07:29:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.694+0000 7f7a91200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c113fc0 con 0x7f7a8c078500 2024-04-17T07:29:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.694+0000 7f7a91200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1142c0 con 0x7f7a8c076920 2024-04-17T07:29:16.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.694+0000 7f7a89e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8c076920 0x7f7a8c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.695+0000 7f7a89400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c077b60 0x7f7a8c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.695+0000 7f7a89e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8c076920 0x7f7a8c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:44858/0 (socket says 172.21.15.86:44858) 2024-04-17T07:29:16.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.695+0000 7f7a89e00640 1 -- 172.21.15.86:0/1452516190 learned_addr learned my addr 172.21.15.86:0/1452516190 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:16.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.695+0000 7f7a8a800640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c078500 0x7f7a8c1a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.695+0000 7f7a89e00640 1 -- 172.21.15.86:0/1452516190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c078500 msgr2=0x7f7a8c1a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.695+0000 7f7a89e00640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c078500 0x7f7a8c1a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.695+0000 7f7a89e00640 1 -- 172.21.15.86:0/1452516190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c077b60 msgr2=0x7f7a8c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.697+0000 7f7a89e00640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c077b60 0x7f7a8c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.697+0000 7f7a89e00640 1 -- 172.21.15.86:0/1452516190 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1a63c0 con 0x7f7a8c076920 2024-04-17T07:29:16.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.697+0000 7f7a89400640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c077b60 0x7f7a8c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:29:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.698+0000 7f7a89e00640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8c076920 0x7f7a8c19e7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000c00 tx=0x7f7a7c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.698+0000 7f7a82a00640 1 -- 172.21.15.86:0/1452516190 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c043020 con 0x7f7a8c076920 2024-04-17T07:29:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.699+0000 7f7a82a00640 1 -- 172.21.15.86:0/1452516190 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a7c0052b0 con 0x7f7a8c076920 2024-04-17T07:29:16.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.699+0000 7f7a91200640 1 -- 172.21.15.86:0/1452516190 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1a6650 con 0x7f7a8c076920 2024-04-17T07:29:16.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.699+0000 7f7a91200640 1 -- 172.21.15.86:0/1452516190 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1a6ba0 con 0x7f7a8c076920 2024-04-17T07:29:16.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.699+0000 7f7a82a00640 1 -- 172.21.15.86:0/1452516190 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c003c70 con 0x7f7a8c076920 2024-04-17T07:29:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.705+0000 7f7a82a00640 1 -- 172.21.15.86:0/1452516190 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7a7c046020 con 0x7f7a8c076920 2024-04-17T07:29:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.705+0000 7f7a93e2c740 1 Processor -- start 2024-04-17T07:29:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.706+0000 7f7a93e2c740 1 -- start start 2024-04-17T07:29:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.706+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c5685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.707+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c568b80 0x55ff1c56d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.707+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 0x55ff1c570180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.707+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff1c4cf120 con 0x55ff1c56dcf0 2024-04-17T07:29:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.707+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ff1c475340 con 0x55ff1c568190 2024-04-17T07:29:16.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.707+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ff1c4cb0e0 con 0x55ff1c568b80 2024-04-17T07:29:16.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.707+0000 7f7a8a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c5685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.707+0000 7f7a8a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c5685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50908/0 (socket says 172.21.15.86:50908) 2024-04-17T07:29:16.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.707+0000 7f7a8a800640 1 -- 172.21.15.86:0/2124279195 learned_addr learned my addr 172.21.15.86:0/2124279195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.708+0000 7f7a8a800640 1 -- 172.21.15.86:0/2124279195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c568b80 msgr2=0x55ff1c56d680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:29:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.708+0000 7f7a8a800640 1 --2- 172.21.15.86:0/2124279195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c568b80 0x55ff1c56d680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.708+0000 7f7a8a800640 1 -- 172.21.15.86:0/2124279195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 msgr2=0x55ff1c570180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.708+0000 7f7a8a800640 1 --2- 172.21.15.86:0/2124279195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 0x55ff1c570180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.708+0000 7f7a8a800640 1 -- 172.21.15.86:0/2124279195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1c2ab7d0 con 0x55ff1c568190 2024-04-17T07:29:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.708+0000 7f7a8a800640 1 --2- 172.21.15.86:0/2124279195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c5685b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007e50 tx=0x7f7a8400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea62da9ad67402b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.709+0000 7f7a77400640 1 -- 172.21.15.86:0/2124279195 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84014070 con 0x55ff1c568190 2024-04-17T07:29:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.709+0000 7f7a77400640 1 -- 172.21.15.86:0/2124279195 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a8400ee90 con 0x55ff1c568190 2024-04-17T07:29:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.709+0000 7f7a77400640 1 -- 172.21.15.86:0/2124279195 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a840053a0 con 0x55ff1c568190 2024-04-17T07:29:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.710+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2124279195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 msgr2=0x55ff1c5685b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.710+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/2124279195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c5685b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007e50 tx=0x7f7a8400e720 comp rx=0 tx=0).stop 2024-04-17T07:29:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.710+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2124279195 shutdown_connections 2024-04-17T07:29:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.710+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/2124279195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 0x55ff1c570180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.710+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/2124279195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c568b80 0x55ff1c56d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.710+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/2124279195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c5685b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.710+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2124279195 >> 172.21.15.86:0/2124279195 conn(0x55ff1c563700 msgr2=0x55ff1c563b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.710+0000 7f7a82a00640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a700a3db0 0x7f7a700a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.711+0000 7f7a82a00640 1 -- 172.21.15.86:0/1452516190 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7a7c0f2ab0 con 0x7f7a8c076920 2024-04-17T07:29:16.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.711+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2124279195 shutdown_connections 2024-04-17T07:29:16.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.711+0000 7f7a8a800640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a700a3db0 0x7f7a700a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.712+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2124279195 wait complete. 2024-04-17T07:29:16.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.713+0000 7f7a93e2c740 1 -- start start 2024-04-17T07:29:16.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.714+0000 7f7a8a800640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a700a3db0 0x7f7a700a6270 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007e50 tx=0x7f7a84000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.715+0000 7f7a93e2c740 1 Processor -- start 2024-04-17T07:29:16.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.717+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c568190 0x55ff1c572740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.718+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c572c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.718+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c56dcf0 0x55ff1c579cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.718+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff1c570ca0 con 0x55ff1c568190 2024-04-17T07:29:16.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.718+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ff1c488710 con 0x55ff1c568b80 2024-04-17T07:29:16.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.718+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ff1c570e20 con 0x55ff1c56dcf0 2024-04-17T07:29:16.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.718+0000 7f7a93e2c740 10 client.0 _ll_register_callbacks cb 0x55ff1c483af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:16.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.719+0000 7f7a89400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c572c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.719+0000 7f7a89400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c572c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50912/0 (socket says 172.21.15.86:50912) 2024-04-17T07:29:16.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.719+0000 7f7a89400640 1 -- 172.21.15.86:0/78488131 learned_addr learned my addr 172.21.15.86:0/78488131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:16.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.720+0000 7f7a89e00640 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c568190 0x55ff1c572740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.720+0000 7f7a8a800640 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c56dcf0 0x55ff1c579cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.721+0000 7f7a89400640 1 -- 172.21.15.86:0/78488131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c56dcf0 msgr2=0x55ff1c579cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.721+0000 7f7a89400640 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c56dcf0 0x55ff1c579cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.721+0000 7f7a89400640 1 -- 172.21.15.86:0/78488131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c568190 msgr2=0x55ff1c572740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.721+0000 7f7a89400640 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c568190 0x55ff1c572740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.721+0000 7f7a89400640 1 -- 172.21.15.86:0/78488131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1c1e13f0 con 0x55ff1c568b80 2024-04-17T07:29:16.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.722+0000 7f7a89e00640 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c568190 0x55ff1c572740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:29:16.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.722+0000 7f7a89400640 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c572c80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7a78011450 tx=0x7f7a78011910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a788289f9a2815a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.722+0000 7f7a6be00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:29:16.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.722+0000 7f7a6be00640 1 -- 172.21.15.86:0/78488131 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78004460 con 0x55ff1c568b80 2024-04-17T07:29:16.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.723+0000 7f7a6be00640 1 -- 172.21.15.86:0/78488131 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7a78004260 con 0x55ff1c568b80 2024-04-17T07:29:16.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.723+0000 7f7a93e2c740 1 -- 172.21.15.86:0/78488131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a8400ee90 con 0x55ff1c568b80 2024-04-17T07:29:16.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.724+0000 7f7a6be00640 1 -- 172.21.15.86:0/78488131 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a78004460 con 0x55ff1c568b80 2024-04-17T07:29:16.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.724+0000 7f7a93e2c740 10 client.37931 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:16.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.724+0000 7f7a93e2c740 1 -- 172.21.15.86:0/78488131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff1c1a10f0 con 0x55ff1c568b80 2024-04-17T07:29:16.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.725+0000 7f7a6be00640 1 -- 172.21.15.86:0/78488131 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a8400ee90 con 0x55ff1c568b80 2024-04-17T07:29:16.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.725+0000 7f7a6be00640 1 -- 172.21.15.86:0/78488131 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7a78012980 con 0x55ff1c568b80 2024-04-17T07:29:16.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.725+0000 7f7a93e2c740 10 client.37931 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:16.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.725+0000 7f7a93e2c740 10 client.37931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:16.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.726+0000 7f7a93e2c740 20 client.37931 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:16.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.726+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ff1c57e640 0x55ff1c580aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.726+0000 7f7a93e2c740 4 client.37931 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:16.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.726+0000 7f7a93e2c740 1 -- 172.21.15.86:0/78488131 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff1c4dcb40 con 0x55ff1c57e640 2024-04-17T07:29:16.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.727+0000 7f7a89e00640 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ff1c57e640 0x55ff1c580aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.730+0000 7f7a89e00640 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ff1c57e640 0x55ff1c580aa0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=463796ac3e2ce057 server_cookie=823881400e7b6d9e in_seq=0 out_seq=0 2024-04-17T07:29:16.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.731+0000 7f7a6be00640 10 client.37931 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:16.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.732+0000 7f7a6be00640 1 -- 172.21.15.86:0/78488131 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff1c4dcb40 con 0x55ff1c57e640 2024-04-17T07:29:16.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.732+0000 7f7a6be00640 10 client.37931 handle_command_reply: tid=0 2024-04-17T07:29:16.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.732+0000 7f7a93e2c740 1 client.37931 shutdown 2024-04-17T07:29:16.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.733+0000 7f7a93e2c740 1 -- 172.21.15.86:0/78488131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 msgr2=0x55ff1c572c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.733+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c572c80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7a78011450 tx=0x7f7a78011910 comp rx=0 tx=0).stop 2024-04-17T07:29:16.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.733+0000 7f7a93e2c740 1 -- 172.21.15.86:0/78488131 shutdown_connections 2024-04-17T07:29:16.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.733+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ff1c57e640 0x55ff1c580aa0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.733+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c56dcf0 0x55ff1c579cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.733+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c572c80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.733+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/78488131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c568190 0x55ff1c572740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.733+0000 7f7a93e2c740 1 -- 172.21.15.86:0/78488131 >> 172.21.15.86:0/78488131 conn(0x55ff1c563700 msgr2=0x55ff1c562890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.733+0000 7f7a6be00640 0 client.37931 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:16.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.734+0000 7f7a93e2c740 1 -- 172.21.15.86:0/78488131 shutdown_connections 2024-04-17T07:29:16.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.734+0000 7f7a93e2c740 1 -- 172.21.15.86:0/78488131 wait complete. 2024-04-17T07:29:16.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.734+0000 7f7a93e2c740 20 client.37931 trim_cache size 0 max 16384 2024-04-17T07:29:16.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.771+0000 7f7a93e2c740 1 Processor -- start 2024-04-17T07:29:16.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.772+0000 7f7a93e2c740 1 -- start start 2024-04-17T07:29:16.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.772+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c56a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.772+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c568b80 0x55ff1c56a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.772+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c56dcf0 0x55ff1c579c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.772+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c00fc10 con 0x55ff1c568b80 2024-04-17T07:29:16.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.772+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c011750 con 0x55ff1c568190 2024-04-17T07:29:16.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.772+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c0102e0 con 0x55ff1c56dcf0 2024-04-17T07:29:16.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.773+0000 7f7a89e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c56a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.773+0000 7f7a89e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c56a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50914/0 (socket says 172.21.15.86:50914) 2024-04-17T07:29:16.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.773+0000 7f7a89e00640 1 -- 172.21.15.86:0/1625936445 learned_addr learned my addr 172.21.15.86:0/1625936445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:16.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.773+0000 7f7a8a800640 1 --2- 172.21.15.86:0/1625936445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c56dcf0 0x55ff1c579c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.774+0000 7f7a89e00640 1 -- 172.21.15.86:0/1625936445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c56dcf0 msgr2=0x55ff1c579c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.774+0000 7f7a89e00640 1 --2- 172.21.15.86:0/1625936445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c56dcf0 0x55ff1c579c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.774+0000 7f7a89e00640 1 -- 172.21.15.86:0/1625936445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c568b80 msgr2=0x55ff1c56a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.774+0000 7f7a89e00640 1 --2- 172.21.15.86:0/1625936445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c568b80 0x55ff1c56a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.774+0000 7f7a89e00640 1 -- 172.21.15.86:0/1625936445 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1c564470 con 0x55ff1c568190 2024-04-17T07:29:16.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.774+0000 7f7a89e00640 1 --2- 172.21.15.86:0/1625936445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c56a0c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c004be0 tx=0x7f7a7c0f2040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9131cf80f5975ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.775+0000 7f7a76a00640 1 -- 172.21.15.86:0/1625936445 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0d7070 con 0x55ff1c568190 2024-04-17T07:29:16.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.775+0000 7f7a76a00640 1 -- 172.21.15.86:0/1625936445 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a7c0035f0 con 0x55ff1c568190 2024-04-17T07:29:16.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.775+0000 7f7a76a00640 1 -- 172.21.15.86:0/1625936445 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0c7620 con 0x55ff1c568190 2024-04-17T07:29:16.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.776+0000 7f7a93e2c740 1 -- 172.21.15.86:0/1625936445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 msgr2=0x55ff1c56a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.776+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/1625936445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c56a0c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c004be0 tx=0x7f7a7c0f2040 comp rx=0 tx=0).stop 2024-04-17T07:29:16.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.776+0000 7f7a93e2c740 1 -- 172.21.15.86:0/1625936445 shutdown_connections 2024-04-17T07:29:16.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.776+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/1625936445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c56dcf0 0x55ff1c579c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.776+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/1625936445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c568b80 0x55ff1c56a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.776+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/1625936445 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568190 0x55ff1c56a0c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.776+0000 7f7a93e2c740 1 -- 172.21.15.86:0/1625936445 >> 172.21.15.86:0/1625936445 conn(0x55ff1c563700 msgr2=0x55ff1c571250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.776+0000 7f7a93e2c740 1 -- 172.21.15.86:0/1625936445 shutdown_connections 2024-04-17T07:29:16.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.776+0000 7f7a93e2c740 1 -- 172.21.15.86:0/1625936445 wait complete. 2024-04-17T07:29:16.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.777+0000 7f7a93e2c740 1 -- start start 2024-04-17T07:29:16.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.777+0000 7f7a93e2c740 1 Processor -- start 2024-04-17T07:29:16.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.778+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c575850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.778+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 0x55ff1c575d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.778+0000 7f7a93e2c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c578db0 0x55ff1c579230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.778+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c011080 con 0x55ff1c56dcf0 2024-04-17T07:29:16.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.778+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c0109b0 con 0x55ff1c568b80 2024-04-17T07:29:16.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.778+0000 7f7a93e2c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c00dbe0 con 0x55ff1c578db0 2024-04-17T07:29:16.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.778+0000 7f7a93e2c740 10 client.0 _ll_register_callbacks cb 0x7f7a78009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:16.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.779+0000 7f7a89e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c575850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.779+0000 7f7a89400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 0x55ff1c575d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.779+0000 7f7a89e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c575850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:50922/0 (socket says 172.21.15.86:50922) 2024-04-17T07:29:16.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.779+0000 7f7a89400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 0x55ff1c575d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34638/0 (socket says 172.21.15.86:34638) 2024-04-17T07:29:16.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.779+0000 7f7a89e00640 1 -- 172.21.15.86:0/2321888050 learned_addr learned my addr 172.21.15.86:0/2321888050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:16.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.779+0000 7f7a8a800640 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c578db0 0x55ff1c579230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.780+0000 7f7a89400640 1 -- 172.21.15.86:0/2321888050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c578db0 msgr2=0x55ff1c579230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.780+0000 7f7a89400640 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c578db0 0x55ff1c579230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.780+0000 7f7a89400640 1 -- 172.21.15.86:0/2321888050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 msgr2=0x55ff1c575850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:16.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.780+0000 7f7a89400640 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c575850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.780+0000 7f7a89400640 1 -- 172.21.15.86:0/2321888050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1c579770 con 0x55ff1c56dcf0 2024-04-17T07:29:16.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.781+0000 7f7a89400640 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 0x55ff1c575d90 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x55ff1c55f0e0 tx=0x7f7a78009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b82d3b2a6268b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:16.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.781+0000 7f7a6be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:16.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.781+0000 7f7a6be00640 1 -- 172.21.15.86:0/2321888050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78032060 con 0x55ff1c56dcf0 2024-04-17T07:29:16.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.781+0000 7f7a6be00640 1 -- 172.21.15.86:0/2321888050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7a78005350 con 0x55ff1c56dcf0 2024-04-17T07:29:16.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.782+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2321888050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff1c570600 con 0x55ff1c56dcf0 2024-04-17T07:29:16.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.783+0000 7f7a6be00640 1 -- 172.21.15.86:0/2321888050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a78005550 con 0x55ff1c56dcf0 2024-04-17T07:29:16.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.783+0000 7f7a93e2c740 10 client.18987 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:16.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.783+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2321888050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff1c5f65d0 con 0x55ff1c56dcf0 2024-04-17T07:29:16.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.784+0000 7f7a6be00640 1 -- 172.21.15.86:0/2321888050 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ff1c570600 con 0x55ff1c56dcf0 2024-04-17T07:29:16.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.784+0000 7f7a6be00640 1 -- 172.21.15.86:0/2321888050 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7a78035050 con 0x55ff1c56dcf0 2024-04-17T07:29:16.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.784+0000 7f7a93e2c740 10 client.18987 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:16.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.784+0000 7f7a93e2c740 10 client.18987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:16.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.784+0000 7f7a93e2c740 20 client.18987 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:16.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.785+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ff1c5f6f30 0x55ff1c5f9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:16.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.785+0000 7f7a93e2c740 4 client.18987 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:29:16.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.785+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2321888050 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff1c4d69b0 con 0x55ff1c5f6f30 2024-04-17T07:29:16.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.785+0000 7f7a89e00640 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ff1c5f6f30 0x55ff1c5f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:16.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.786+0000 7f7a89e00640 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ff1c5f6f30 0x55ff1c5f9350 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5486a067d300f35d server_cookie=f664fdbe670da371 in_seq=0 out_seq=0 2024-04-17T07:29:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.786+0000 7f7a6be00640 10 client.18987 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.787+0000 7f7a6be00640 1 -- 172.21.15.86:0/2321888050 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ff1c4d69b0 con 0x55ff1c5f6f30 2024-04-17T07:29:16.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.787+0000 7f7a6be00640 10 client.18987 handle_command_reply: tid=0 2024-04-17T07:29:16.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.787+0000 7f7a93e2c740 1 client.18987 shutdown 2024-04-17T07:29:16.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.789+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2321888050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 msgr2=0x55ff1c575d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.789+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 0x55ff1c575d90 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x55ff1c55f0e0 tx=0x7f7a78009d80 comp rx=0 tx=0).stop 2024-04-17T07:29:16.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.789+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2321888050 shutdown_connections 2024-04-17T07:29:16.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.789+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ff1c5f6f30 0x55ff1c5f9350 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.789+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ff1c578db0 0x55ff1c579230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.789+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff1c56dcf0 0x55ff1c575d90 secure :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x55ff1c55f0e0 tx=0x7f7a78009d80 comp rx=0 tx=0).stop 2024-04-17T07:29:16.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.789+0000 7f7a93e2c740 1 --2- 172.21.15.86:0/2321888050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ff1c568b80 0x55ff1c575850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.789+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2321888050 >> 172.21.15.86:0/2321888050 conn(0x55ff1c563700 msgr2=0x55ff1c572700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.789+0000 7f7a6be00640 0 client.18987 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:16.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.790+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2321888050 shutdown_connections 2024-04-17T07:29:16.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.790+0000 7f7a93e2c740 1 -- 172.21.15.86:0/2321888050 wait complete. 2024-04-17T07:29:16.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.790+0000 7f7a93e2c740 20 client.18987 trim_cache size 0 max 16384 2024-04-17T07:29:16.791 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:29:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:29:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:29:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "18fd36b2-2f2d-4423-af53-cf5abbdbaa41": { 2024-04-17T07:29:16.791 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:29:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "18fd36b2-2f2d-4423-af53-cf5abbdbaa41", 2024-04-17T07:29:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:29:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:29:16.792 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:29:16.792 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:29:16.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.791+0000 7f7a91200640 1 -- 172.21.15.86:0/1452516190 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a700a3db0 msgr2=0x7f7a700a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.791+0000 7f7a91200640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a700a3db0 0x7f7a700a6270 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007e50 tx=0x7f7a84000f40 comp rx=0 tx=0).stop 2024-04-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.791+0000 7f7a91200640 1 -- 172.21.15.86:0/1452516190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8c076920 msgr2=0x7f7a8c19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.791+0000 7f7a91200640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8c076920 0x7f7a8c19e7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c000c00 tx=0x7f7a7c004270 comp rx=0 tx=0).stop 2024-04-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.792+0000 7f7a91200640 1 -- 172.21.15.86:0/1452516190 shutdown_connections 2024-04-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.792+0000 7f7a91200640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7a700a3db0 0x7f7a700a6270 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.792+0000 7f7a91200640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c078500 0x7f7a8c1a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.792+0000 7f7a91200640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a8c077b60 0x7f7a8c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.792+0000 7f7a91200640 1 --2- 172.21.15.86:0/1452516190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8c076920 0x7f7a8c19e7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.792+0000 7f7a91200640 1 -- 172.21.15.86:0/1452516190 >> 172.21.15.86:0/1452516190 conn(0x7f7a8c0ff750 msgr2=0x7f7a8c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.792+0000 7f7a91200640 1 -- 172.21.15.86:0/1452516190 shutdown_connections 2024-04-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:16.792+0000 7f7a91200640 1 -- 172.21.15.86:0/1452516190 wait complete. 2024-04-17T07:29:16.824 INFO:tasks.cephfs.filesystem:scrub status for tag:18fd36b2-2f2d-4423-af53-cf5abbdbaa41 - {'path': '/', 'tag': '18fd36b2-2f2d-4423-af53-cf5abbdbaa41', 'options': 'recursive,force'} 2024-04-17T07:29:17.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:16 smithi086 ceph-mon[31733]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.7 MiB/s wr, 621 op/s 2024-04-17T07:29:17.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:16 smithi098 ceph-mon[35418]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.7 MiB/s wr, 621 op/s 2024-04-17T07:29:17.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:16 smithi103 ceph-mon[35495]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.7 MiB/s wr, 621 op/s 2024-04-17T07:29:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:17 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:29:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:17 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:17 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:29:18.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:17 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:18.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:17 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:29:18.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:17 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:18.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:17 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:29:18.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:17 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:18.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:17 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:29:18.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:17 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:18.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:17 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:29:18.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:17 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:18 smithi098 ceph-mon[35418]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.3 MiB/s wr, 688 op/s 2024-04-17T07:29:19.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:18 smithi103 ceph-mon[35495]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.3 MiB/s wr, 688 op/s 2024-04-17T07:29:19.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:18 smithi086 ceph-mon[31733]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.3 MiB/s wr, 688 op/s 2024-04-17T07:29:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 595.0 s, 507.0 tps, lat 197.713 ms stddev 135.211 2024-04-17T07:29:20.178 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:19 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:29:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:19 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:29:20.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:19 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:29:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:20 smithi098 ceph-mon[35418]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.1 MiB/s wr, 622 op/s 2024-04-17T07:29:21.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:20 smithi103 ceph-mon[35495]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.1 MiB/s wr, 622 op/s 2024-04-17T07:29:21.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:20 smithi086 ceph-mon[31733]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.1 MiB/s wr, 622 op/s 2024-04-17T07:29:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:23 smithi098 ceph-mon[35418]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 662 op/s 2024-04-17T07:29:23.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:23 smithi103 ceph-mon[35495]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 662 op/s 2024-04-17T07:29:23.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:23 smithi086 ceph-mon[31733]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 662 op/s 2024-04-17T07:29:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 600.0 s, 597.6 tps, lat 167.523 ms stddev 47.705 2024-04-17T07:29:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:25 smithi098 ceph-mon[35418]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.1 MiB/s wr, 635 op/s 2024-04-17T07:29:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:25 smithi103 ceph-mon[35495]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.1 MiB/s wr, 635 op/s 2024-04-17T07:29:25.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:25 smithi086 ceph-mon[31733]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.1 MiB/s wr, 635 op/s 2024-04-17T07:29:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:27 smithi098 ceph-mon[35418]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 556 op/s 2024-04-17T07:29:27.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:27 smithi103 ceph-mon[35495]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 556 op/s 2024-04-17T07:29:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:27 smithi086 ceph-mon[31733]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 556 op/s 2024-04-17T07:29:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 605.0 s, 579.2 tps, lat 173.242 ms stddev 48.357 2024-04-17T07:29:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:29 smithi098 ceph-mon[35418]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.7 MiB/s wr, 640 op/s 2024-04-17T07:29:29.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:29 smithi103 ceph-mon[35495]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.7 MiB/s wr, 640 op/s 2024-04-17T07:29:29.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:29 smithi086 ceph-mon[31733]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.7 MiB/s wr, 640 op/s 2024-04-17T07:29:31.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:31 smithi098 ceph-mon[35418]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 624 op/s 2024-04-17T07:29:31.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:31 smithi103 ceph-mon[35495]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 624 op/s 2024-04-17T07:29:31.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:31 smithi086 ceph-mon[31733]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 624 op/s 2024-04-17T07:29:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:33 smithi098 ceph-mon[35418]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.8 MiB/s wr, 678 op/s 2024-04-17T07:29:33.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:33 smithi103 ceph-mon[35495]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.8 MiB/s wr, 678 op/s 2024-04-17T07:29:33.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:33 smithi086 ceph-mon[31733]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.8 MiB/s wr, 678 op/s 2024-04-17T07:29:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 610.0 s, 553.6 tps, lat 178.318 ms stddev 77.979 2024-04-17T07:29:35.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:35 smithi098 ceph-mon[35418]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 677 op/s 2024-04-17T07:29:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:35 smithi103 ceph-mon[35495]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 677 op/s 2024-04-17T07:29:35.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:35 smithi086 ceph-mon[31733]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 677 op/s 2024-04-17T07:29:37.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:37 smithi098 ceph-mon[35418]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 608 op/s 2024-04-17T07:29:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:37 smithi103 ceph-mon[35495]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 608 op/s 2024-04-17T07:29:37.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:37 smithi086 ceph-mon[31733]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 608 op/s 2024-04-17T07:29:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 615.0 s, 514.6 tps, lat 193.509 ms stddev 116.890 2024-04-17T07:29:39.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:39 smithi098 ceph-mon[35418]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.5 MiB/s wr, 676 op/s 2024-04-17T07:29:39.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:39 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:29:39.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:39 smithi103 ceph-mon[35495]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.5 MiB/s wr, 676 op/s 2024-04-17T07:29:39.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:39 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:29:39.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:39 smithi086 ceph-mon[31733]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.5 MiB/s wr, 676 op/s 2024-04-17T07:29:39.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:39 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:29:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:41 smithi098 ceph-mon[35418]: pgmap v899: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 648 op/s 2024-04-17T07:29:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:29:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:29:41.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:29:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:41 smithi103 ceph-mon[35495]: pgmap v899: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 648 op/s 2024-04-17T07:29:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:29:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:29:41.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:29:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:41 smithi086 ceph-mon[31733]: pgmap v899: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 648 op/s 2024-04-17T07:29:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:29:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:29:41.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:29:43.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:43 smithi098 ceph-mon[35418]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.4 MiB/s wr, 677 op/s 2024-04-17T07:29:43.550 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:43 smithi103 ceph-mon[35495]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.4 MiB/s wr, 677 op/s 2024-04-17T07:29:43.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:43 smithi086 ceph-mon[31733]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.4 MiB/s wr, 677 op/s 2024-04-17T07:29:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 620.0 s, 547.4 tps, lat 185.471 ms stddev 126.407 2024-04-17T07:29:45.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:45 smithi098 ceph-mon[35418]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 636 op/s 2024-04-17T07:29:45.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:45 smithi103 ceph-mon[35495]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 636 op/s 2024-04-17T07:29:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:45 smithi086 ceph-mon[31733]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 636 op/s 2024-04-17T07:29:46.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:29:46.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:29:47.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.168+0000 7f16b5400640 1 -- 172.21.15.86:0/3428451447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16b0105d90 msgr2=0x7f16b010c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.168+0000 7f16b5400640 1 --2- 172.21.15.86:0/3428451447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16b0105d90 0x7f16b010c620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009ca0 tx=0x7f16a802c100 comp rx=0 tx=0).stop 2024-04-17T07:29:47.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.168+0000 7f16b5400640 1 -- 172.21.15.86:0/3428451447 shutdown_connections 2024-04-17T07:29:47.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.168+0000 7f16b5400640 1 --2- 172.21.15.86:0/3428451447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16b0105d90 0x7f16b010c620 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.168+0000 7f16b5400640 1 --2- 172.21.15.86:0/3428451447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16b01053d0 0x7f16b0105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.168+0000 7f16b5400640 1 --2- 172.21.15.86:0/3428451447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b01041e0 0x7f16b0104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.168+0000 7f16b5400640 1 -- 172.21.15.86:0/3428451447 >> 172.21.15.86:0/3428451447 conn(0x7f16b00ff750 msgr2=0x7f16b0101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:47.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.168+0000 7f16b5400640 1 -- 172.21.15.86:0/3428451447 shutdown_connections 2024-04-17T07:29:47.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.168+0000 7f16b5400640 1 -- 172.21.15.86:0/3428451447 wait complete. 2024-04-17T07:29:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.171+0000 7f16b5400640 1 Processor -- start 2024-04-17T07:29:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.171+0000 7f16b5400640 1 -- start start 2024-04-17T07:29:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.171+0000 7f16b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16b01041e0 0x7f16b006db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.171+0000 7f16b5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b01053d0 0x7f16b006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.171+0000 7f16b5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16b0105d90 0x7f16b0073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.171+0000 7f16b5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0076ff0 con 0x7f16b01041e0 2024-04-17T07:29:47.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.171+0000 7f16b5400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0076e70 con 0x7f16b01053d0 2024-04-17T07:29:47.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.171+0000 7f16b5400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0077170 con 0x7f16b0105d90 2024-04-17T07:29:47.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.171+0000 7f16ae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16b01041e0 0x7f16b006db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.172+0000 7f16af200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16b0105d90 0x7f16b0073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.172+0000 7f16ae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16b01041e0 0x7f16b006db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47184/0 (socket says 172.21.15.86:47184) 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.172+0000 7f16ae800640 1 -- 172.21.15.86:0/1517907923 learned_addr learned my addr 172.21.15.86:0/1517907923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.172+0000 7f16ade00640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b01053d0 0x7f16b006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.172+0000 7f16af200640 1 -- 172.21.15.86:0/1517907923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b01053d0 msgr2=0x7f16b006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.172+0000 7f16af200640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b01053d0 0x7f16b006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.172+0000 7f16af200640 1 -- 172.21.15.86:0/1517907923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16b01041e0 msgr2=0x7f16b006db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.173+0000 7f16af200640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16b01041e0 0x7f16b006db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.173+0000 7f16af200640 1 -- 172.21.15.86:0/1517907923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b00737a0 con 0x7f16b0105d90 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.173+0000 7f16ae800640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16b01041e0 0x7f16b006db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.174+0000 7f16af200640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16b0105d90 0x7f16b0073130 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f16a8037040 tx=0x7f16a8035ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.174+0000 7f16a7400640 1 -- 172.21.15.86:0/1517907923 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8039040 con 0x7f16b0105d90 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.174+0000 7f16b5400640 1 -- 172.21.15.86:0/1517907923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b0073a30 con 0x7f16b0105d90 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.174+0000 7f16a7400640 1 -- 172.21.15.86:0/1517907923 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a8044070 con 0x7f16b0105d90 2024-04-17T07:29:47.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.174+0000 7f16b5400640 1 -- 172.21.15.86:0/1517907923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b0073f70 con 0x7f16b0105d90 2024-04-17T07:29:47.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.175+0000 7f16a7400640 1 -- 172.21.15.86:0/1517907923 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8004330 con 0x7f16b0105d90 2024-04-17T07:29:47.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.179+0000 7f16a7400640 1 -- 172.21.15.86:0/1517907923 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f16a8003970 con 0x7f16b0105d90 2024-04-17T07:29:47.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.183+0000 7f16b8006740 1 Processor -- start 2024-04-17T07:29:47.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.183+0000 7f16a7400640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16940a3d00 0x7f16940a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.183+0000 7f16ade00640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16940a3d00 0x7f16940a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.183+0000 7f16b8006740 1 -- start start 2024-04-17T07:29:47.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.185+0000 7f16b8006740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.185+0000 7f16b8006740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7ab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.185+0000 7f16b8006740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7ae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.185+0000 7f16a7400640 1 -- 172.21.15.86:0/1517907923 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f16a80f2a60 con 0x7f16b0105d90 2024-04-17T07:29:47.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.185+0000 7f16ade00640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16940a3d00 0x7f16940a61c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f16b006f0a0 tx=0x7f169c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:47.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.185+0000 7f16b8006740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5eb70d120 con 0x55a5eb7a6190 2024-04-17T07:29:47.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.185+0000 7f16ade00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.186+0000 7f16af200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.186+0000 7f16ade00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7ab680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45318/0 (socket says 172.21.15.86:45318) 2024-04-17T07:29:47.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.186+0000 7f16ade00640 1 -- 172.21.15.86:0/4212004094 learned_addr learned my addr 172.21.15.86:0/4212004094 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:47.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.185+0000 7f16b8006740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a5eb6b3340 con 0x55a5eb7a6b80 2024-04-17T07:29:47.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.186+0000 7f16b8006740 1 -- 172.21.15.86:0/4212004094 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5eb7090e0 con 0x55a5eb7abcf0 2024-04-17T07:29:47.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.186+0000 7f16ae800640 1 --2- 172.21.15.86:0/4212004094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.187+0000 7f16ae800640 1 -- 172.21.15.86:0/4212004094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7abcf0 msgr2=0x55a5eb7ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:47.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.187+0000 7f16ae800640 1 --2- 172.21.15.86:0/4212004094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.187+0000 7f16ae800640 1 -- 172.21.15.86:0/4212004094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 msgr2=0x55a5eb7ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:47.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.187+0000 7f16ae800640 1 --2- 172.21.15.86:0/4212004094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7ab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.187+0000 7f16ae800640 1 -- 172.21.15.86:0/4212004094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5eb4e97d0 con 0x55a5eb7a6190 2024-04-17T07:29:47.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.188+0000 7f16ae800640 1 --2- 172.21.15.86:0/4212004094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7a65b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f16a00114c0 tx=0x7f16a00118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2b70441504f29b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:47.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.192+0000 7f169b400640 1 -- 172.21.15.86:0/4212004094 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0005020 con 0x55a5eb7a6190 2024-04-17T07:29:47.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.193+0000 7f169b400640 1 -- 172.21.15.86:0/4212004094 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a00047a0 con 0x55a5eb7a6190 2024-04-17T07:29:47.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.193+0000 7f169b400640 1 -- 172.21.15.86:0/4212004094 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a000f780 con 0x55a5eb7a6190 2024-04-17T07:29:47.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.194+0000 7f16b8006740 1 -- 172.21.15.86:0/4212004094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 msgr2=0x55a5eb7a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.194+0000 7f16b8006740 1 --2- 172.21.15.86:0/4212004094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7a65b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f16a00114c0 tx=0x7f16a00118c0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.195+0000 7f16b8006740 1 -- 172.21.15.86:0/4212004094 shutdown_connections 2024-04-17T07:29:47.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.195+0000 7f16b8006740 1 --2- 172.21.15.86:0/4212004094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.195+0000 7f16b8006740 1 --2- 172.21.15.86:0/4212004094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.195+0000 7f16b8006740 1 --2- 172.21.15.86:0/4212004094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7a65b0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.195+0000 7f16b8006740 1 -- 172.21.15.86:0/4212004094 >> 172.21.15.86:0/4212004094 conn(0x55a5eb7a1700 msgr2=0x55a5eb7a1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.195+0000 7f16b8006740 1 -- 172.21.15.86:0/4212004094 shutdown_connections 2024-04-17T07:29:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.195+0000 7f16b8006740 1 -- 172.21.15.86:0/4212004094 wait complete. 2024-04-17T07:29:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.196+0000 7f16b8006740 1 -- start start 2024-04-17T07:29:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.196+0000 7f16b8006740 1 Processor -- start 2024-04-17T07:29:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.197+0000 7f16b8006740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6190 0x55a5eb7b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.197+0000 7f16b8006740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.197+0000 7f16b8006740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.197+0000 7f16b8006740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5eb7aeca0 con 0x55a5eb7abcf0 2024-04-17T07:29:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.197+0000 7f16b8006740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a5eb6c6710 con 0x55a5eb7a6b80 2024-04-17T07:29:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.197+0000 7f16b8006740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5eb7aee20 con 0x55a5eb7a6190 2024-04-17T07:29:47.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.197+0000 7f16b8006740 10 client.0 _ll_register_callbacks cb 0x55a5eb6c1af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:47.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.198+0000 7f16ae800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6190 0x55a5eb7b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.198+0000 7f16ae800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6190 0x55a5eb7b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49688/0 (socket says 172.21.15.86:49688) 2024-04-17T07:29:47.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.198+0000 7f16af200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.198+0000 7f16ae800640 1 -- 172.21.15.86:0/3100931737 learned_addr learned my addr 172.21.15.86:0/3100931737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:47.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.198+0000 7f16ade00640 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.199+0000 7f16af200640 1 -- 172.21.15.86:0/3100931737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6190 msgr2=0x55a5eb7b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.199+0000 7f16af200640 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6190 0x55a5eb7b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.199+0000 7f16af200640 1 -- 172.21.15.86:0/3100931737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 msgr2=0x55a5eb7b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.199+0000 7f16af200640 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.199+0000 7f16af200640 1 -- 172.21.15.86:0/3100931737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5eb41f3f0 con 0x55a5eb7abcf0 2024-04-17T07:29:47.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.200+0000 7f16af200640 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b7cd0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009ca0 tx=0x7f16a80f0050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8a78fc07f35d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:47.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.200+0000 7f168fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:47.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.200+0000 7f168fe00640 1 -- 172.21.15.86:0/3100931737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80bf060 con 0x55a5eb7abcf0 2024-04-17T07:29:47.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.201+0000 7f168fe00640 1 -- 172.21.15.86:0/3100931737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f16a80ced50 con 0x55a5eb7abcf0 2024-04-17T07:29:47.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.201+0000 7f16b8006740 1 -- 172.21.15.86:0/3100931737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a00047a0 con 0x55a5eb7abcf0 2024-04-17T07:29:47.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.202+0000 7f168fe00640 1 -- 172.21.15.86:0/3100931737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16a80cef50 con 0x55a5eb7abcf0 2024-04-17T07:29:47.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.202+0000 7f16b8006740 10 client.18996 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:47.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.202+0000 7f16b8006740 1 -- 172.21.15.86:0/3100931737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5eb3df0f0 con 0x55a5eb7abcf0 2024-04-17T07:29:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.202+0000 7f168fe00640 1 -- 172.21.15.86:0/3100931737 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a00047a0 con 0x55a5eb7abcf0 2024-04-17T07:29:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.203+0000 7f168fe00640 1 -- 172.21.15.86:0/3100931737 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f16a803c030 con 0x55a5eb7abcf0 2024-04-17T07:29:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.203+0000 7f16b8006740 10 client.18996 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:47.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.203+0000 7f16b8006740 10 client.18996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:47.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.203+0000 7f16b8006740 20 client.18996 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:47.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.203+0000 7f16b8006740 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a5eb7bc630 0x55a5eb7bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.203+0000 7f16b8006740 4 client.18996 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:47.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.203+0000 7f16b8006740 1 -- 172.21.15.86:0/3100931737 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5eb71ab40 con 0x55a5eb7bc630 2024-04-17T07:29:47.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.204+0000 7f16ae800640 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a5eb7bc630 0x55a5eb7bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.205+0000 7f16ae800640 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a5eb7bc630 0x55a5eb7bea90 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04325806f9cacb3 server_cookie=57641a91fc89c471 in_seq=0 out_seq=0 2024-04-17T07:29:47.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.205+0000 7f168fe00640 10 client.18996 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.208+0000 7f168fe00640 1 -- 172.21.15.86:0/3100931737 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5eb71ab40 con 0x55a5eb7bc630 2024-04-17T07:29:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.208+0000 7f168fe00640 10 client.18996 handle_command_reply: tid=0 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.208+0000 7f16b8006740 1 client.18996 shutdown 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.208+0000 7f16b8006740 1 -- 172.21.15.86:0/3100931737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7abcf0 msgr2=0x55a5eb7b7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.208+0000 7f16b8006740 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b7cd0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009ca0 tx=0x7f16a80f0050 comp rx=0 tx=0).stop 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.208+0000 7f16b8006740 1 -- 172.21.15.86:0/3100931737 shutdown_connections 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.208+0000 7f16b8006740 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a5eb7bc630 0x55a5eb7bea90 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.208+0000 7f16b8006740 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b7cd0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.208+0000 7f16b8006740 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.208+0000 7f16b8006740 1 --2- 172.21.15.86:0/3100931737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6190 0x55a5eb7b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.209+0000 7f16b8006740 1 -- 172.21.15.86:0/3100931737 >> 172.21.15.86:0/3100931737 conn(0x55a5eb7a1700 msgr2=0x55a5eb7a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.209+0000 7f168fe00640 0 client.18996 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.209+0000 7f16b8006740 1 -- 172.21.15.86:0/3100931737 shutdown_connections 2024-04-17T07:29:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.209+0000 7f16b8006740 1 -- 172.21.15.86:0/3100931737 wait complete. 2024-04-17T07:29:47.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.210+0000 7f16b8006740 20 client.18996 trim_cache size 0 max 16384 2024-04-17T07:29:47.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.253+0000 7f16b8006740 1 Processor -- start 2024-04-17T07:29:47.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.253+0000 7f16b8006740 1 -- start start 2024-04-17T07:29:47.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.253+0000 7f16b8006740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.253+0000 7f16b8006740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.253+0000 7f16b8006740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.253+0000 7f16b8006740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f169000fb50 con 0x55a5eb7a6190 2024-04-17T07:29:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.253+0000 7f16b8006740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1690011690 con 0x55a5eb7a6b80 2024-04-17T07:29:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.253+0000 7f16b8006740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1690010220 con 0x55a5eb7abcf0 2024-04-17T07:29:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.254+0000 7f16ade00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.254+0000 7f16ade00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45336/0 (socket says 172.21.15.86:45336) 2024-04-17T07:29:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.254+0000 7f16ade00640 1 -- 172.21.15.86:0/3714026575 learned_addr learned my addr 172.21.15.86:0/3714026575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.254+0000 7f16ae800640 1 --2- 172.21.15.86:0/3714026575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.254+0000 7f16af200640 1 --2- 172.21.15.86:0/3714026575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.256+0000 7f16ae800640 1 -- 172.21.15.86:0/3714026575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7abcf0 msgr2=0x55a5eb7af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:47.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.256+0000 7f16ae800640 1 --2- 172.21.15.86:0/3714026575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.256+0000 7f16ae800640 1 -- 172.21.15.86:0/3714026575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 msgr2=0x55a5eb7b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:47.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.256+0000 7f16ae800640 1 --2- 172.21.15.86:0/3714026575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.256+0000 7f16ae800640 1 -- 172.21.15.86:0/3714026575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5eb7a2170 con 0x55a5eb7a6190 2024-04-17T07:29:47.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.257+0000 7f16ae800640 1 --2- 172.21.15.86:0/3714026575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7af250 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x55a5eb7bf300 tx=0x7f16a0009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6f0b4ae8e313c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:47.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.257+0000 7f169aa00640 1 -- 172.21.15.86:0/3714026575 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0004bb0 con 0x55a5eb7a6190 2024-04-17T07:29:47.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.257+0000 7f169aa00640 1 -- 172.21.15.86:0/3714026575 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a00044f0 con 0x55a5eb7a6190 2024-04-17T07:29:47.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.258+0000 7f169aa00640 1 -- 172.21.15.86:0/3714026575 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a00055a0 con 0x55a5eb7a6190 2024-04-17T07:29:47.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.258+0000 7f16b8006740 1 -- 172.21.15.86:0/3714026575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 msgr2=0x55a5eb7af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.258+0000 7f16b8006740 1 --2- 172.21.15.86:0/3714026575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7af250 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x55a5eb7bf300 tx=0x7f16a0009550 comp rx=0 tx=0).stop 2024-04-17T07:29:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.260+0000 7f16b8006740 1 -- 172.21.15.86:0/3714026575 shutdown_connections 2024-04-17T07:29:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.260+0000 7f16b8006740 1 --2- 172.21.15.86:0/3714026575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.260+0000 7f16b8006740 1 --2- 172.21.15.86:0/3714026575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.260+0000 7f16b8006740 1 --2- 172.21.15.86:0/3714026575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7af250 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.260+0000 7f16b8006740 1 -- 172.21.15.86:0/3714026575 >> 172.21.15.86:0/3714026575 conn(0x55a5eb7a1700 msgr2=0x55a5eb7a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.260+0000 7f16b8006740 1 -- 172.21.15.86:0/3714026575 shutdown_connections 2024-04-17T07:29:47.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.260+0000 7f16b8006740 1 -- 172.21.15.86:0/3714026575 wait complete. 2024-04-17T07:29:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.261+0000 7f16b8006740 1 -- start start 2024-04-17T07:29:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.261+0000 7f16b8006740 1 Processor -- start 2024-04-17T07:29:47.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.262+0000 7f16b8006740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7b5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.262+0000 7f16b8006740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16b8006740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16b8006740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1690010fc0 con 0x55a5eb7a6190 2024-04-17T07:29:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16b8006740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f16900108f0 con 0x55a5eb7abcf0 2024-04-17T07:29:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16b8006740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f169000db20 con 0x55a5eb7a6b80 2024-04-17T07:29:47.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16b8006740 10 client.0 _ll_register_callbacks cb 0x7f16a80ba540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16ade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16af200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16af200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45348/0 (socket says 172.21.15.86:45348) 2024-04-17T07:29:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16af200640 1 -- 172.21.15.86:0/3225445407 learned_addr learned my addr 172.21.15.86:0/3225445407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16ae800640 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16ae800640 1 -- 172.21.15.86:0/3225445407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6b80 msgr2=0x55a5eb7b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16ae800640 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16ae800640 1 -- 172.21.15.86:0/3225445407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7abcf0 msgr2=0x55a5eb7b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16ae800640 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.263+0000 7f16ae800640 1 -- 172.21.15.86:0/3225445407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5eb7b6c50 con 0x55a5eb7a6190 2024-04-17T07:29:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.264+0000 7f16ae800640 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7b5110 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x55a5eb7bc280 tx=0x7f16a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db21827d7d79620a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.264+0000 7f16af200640 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:47.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.264+0000 7f16ade00640 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:47.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.264+0000 7f168fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:47.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.265+0000 7f168fe00640 1 -- 172.21.15.86:0/3225445407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a003a060 con 0x55a5eb7a6190 2024-04-17T07:29:47.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.265+0000 7f16b8006740 1 -- 172.21.15.86:0/3225445407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a00044f0 con 0x55a5eb7a6190 2024-04-17T07:29:47.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.265+0000 7f168fe00640 1 -- 172.21.15.86:0/3225445407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f16a0031040 con 0x55a5eb7a6190 2024-04-17T07:29:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.267+0000 7f168fe00640 1 -- 172.21.15.86:0/3225445407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16a00047a0 con 0x55a5eb7a6190 2024-04-17T07:29:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.267+0000 7f16b8006740 10 client.19002 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.267+0000 7f16b8006740 1 -- 172.21.15.86:0/3225445407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5eb7b7f60 con 0x55a5eb7a6190 2024-04-17T07:29:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.268+0000 7f168fe00640 1 -- 172.21.15.86:0/3225445407 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a00044f0 con 0x55a5eb7a6190 2024-04-17T07:29:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.268+0000 7f168fe00640 1 -- 172.21.15.86:0/3225445407 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f16a002a070 con 0x55a5eb7a6190 2024-04-17T07:29:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.268+0000 7f16b8006740 10 client.19002 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.268+0000 7f16b8006740 10 client.19002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.268+0000 7f16b8006740 20 client.19002 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.268+0000 7f16b8006740 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a5eb8230a0 0x55a5eb825490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.268+0000 7f16b8006740 4 client.19002 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:29:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.268+0000 7f16b8006740 1 -- 172.21.15.86:0/3225445407 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5eb7149b0 con 0x55a5eb8230a0 2024-04-17T07:29:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.269+0000 7f16ade00640 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a5eb8230a0 0x55a5eb825490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.270+0000 7f16ade00640 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a5eb8230a0 0x55a5eb825490 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d768bbecdc7d425 server_cookie=4b86ae2bc011377 in_seq=0 out_seq=0 2024-04-17T07:29:47.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.270+0000 7f168fe00640 10 client.19002 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.271+0000 7f168fe00640 1 -- 172.21.15.86:0/3225445407 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a5eb7149b0 con 0x55a5eb8230a0 2024-04-17T07:29:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.271+0000 7f168fe00640 10 client.19002 handle_command_reply: tid=0 2024-04-17T07:29:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.271+0000 7f16b8006740 1 client.19002 shutdown 2024-04-17T07:29:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.271+0000 7f16b8006740 1 -- 172.21.15.86:0/3225445407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 msgr2=0x55a5eb7b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.271+0000 7f16b8006740 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7b5110 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x55a5eb7bc280 tx=0x7f16a0009da0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.272+0000 7f16b8006740 1 -- 172.21.15.86:0/3225445407 shutdown_connections 2024-04-17T07:29:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.272+0000 7f16b8006740 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a5eb8230a0 0x55a5eb825490 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.272+0000 7f16b8006740 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5eb7abcf0 0x55a5eb7b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.272+0000 7f16b8006740 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5eb7a6b80 0x55a5eb7b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.272+0000 7f16b8006740 1 --2- 172.21.15.86:0/3225445407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5eb7a6190 0x55a5eb7b5110 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.272+0000 7f16b8006740 1 -- 172.21.15.86:0/3225445407 >> 172.21.15.86:0/3225445407 conn(0x55a5eb7a1700 msgr2=0x55a5eb7a4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.272+0000 7f168fe00640 0 client.19002 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.272+0000 7f16b8006740 1 -- 172.21.15.86:0/3225445407 shutdown_connections 2024-04-17T07:29:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.272+0000 7f16b8006740 1 -- 172.21.15.86:0/3225445407 wait complete. 2024-04-17T07:29:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.273+0000 7f16b8006740 20 client.19002 trim_cache size 0 max 16384 2024-04-17T07:29:47.273 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:29:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:29:47.273 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:29:47.273 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.274+0000 7f16b5400640 1 -- 172.21.15.86:0/1517907923 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16940a3d00 msgr2=0x7f16940a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.274+0000 7f16b5400640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16940a3d00 0x7f16940a61c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f16b006f0a0 tx=0x7f169c00a4c0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.274+0000 7f16b5400640 1 -- 172.21.15.86:0/1517907923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16b0105d90 msgr2=0x7f16b0073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.274+0000 7f16b5400640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16b0105d90 0x7f16b0073130 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f16a8037040 tx=0x7f16a8035ce0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.274+0000 7f16b5400640 1 -- 172.21.15.86:0/1517907923 shutdown_connections 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.274+0000 7f16b5400640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f16940a3d00 0x7f16940a61c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.274+0000 7f16b5400640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16b0105d90 0x7f16b0073130 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.274+0000 7f16b5400640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f16b01053d0 0x7f16b006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.274+0000 7f16b5400640 1 --2- 172.21.15.86:0/1517907923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16b01041e0 0x7f16b006db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.274+0000 7f16b5400640 1 -- 172.21.15.86:0/1517907923 >> 172.21.15.86:0/1517907923 conn(0x7f16b00ff750 msgr2=0x7f16b0079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.275+0000 7f16b5400640 1 -- 172.21.15.86:0/1517907923 shutdown_connections 2024-04-17T07:29:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.275+0000 7f16b5400640 1 -- 172.21.15.86:0/1517907923 wait complete. 2024-04-17T07:29:47.306 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:47 smithi086 ceph-mon[31733]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 582 op/s 2024-04-17T07:29:47.313 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:29:47.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:29:47.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:47 smithi098 ceph-mon[35418]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 582 op/s 2024-04-17T07:29:47.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:47 smithi103 ceph-mon[35495]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 582 op/s 2024-04-17T07:29:47.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.641+0000 7f5da0a00640 1 -- 172.21.15.86:0/1453176036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d9c076920 msgr2=0x7f5d9c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.641+0000 7f5da0a00640 1 --2- 172.21.15.86:0/1453176036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d9c076920 0x7f5d9c076d40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009900 tx=0x7f5d8c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:29:47.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.642+0000 7f5da0a00640 1 -- 172.21.15.86:0/1453176036 shutdown_connections 2024-04-17T07:29:47.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.642+0000 7f5da0a00640 1 --2- 172.21.15.86:0/1453176036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c078500 0x7f5d9c078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.642+0000 7f5da0a00640 1 --2- 172.21.15.86:0/1453176036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d9c077b60 0x7f5d9c077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.642+0000 7f5da0a00640 1 --2- 172.21.15.86:0/1453176036 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d9c076920 0x7f5d9c076d40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.642+0000 7f5da0a00640 1 -- 172.21.15.86:0/1453176036 >> 172.21.15.86:0/1453176036 conn(0x7f5d9c0ff570 msgr2=0x7f5d9c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:47.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.642+0000 7f5da0a00640 1 -- 172.21.15.86:0/1453176036 shutdown_connections 2024-04-17T07:29:47.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.642+0000 7f5da0a00640 1 -- 172.21.15.86:0/1453176036 wait complete. 2024-04-17T07:29:47.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.644+0000 7f5da0a00640 1 Processor -- start 2024-04-17T07:29:47.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.644+0000 7f5da0a00640 1 -- start start 2024-04-17T07:29:47.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.645+0000 7f5da0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d9c076920 0x7f5d9c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.645+0000 7f5da0a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d9c077b60 0x7f5d9c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.645+0000 7f5da0a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c078500 0x7f5d9c1a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.646+0000 7f5d9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d9c076920 0x7f5d9c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.646+0000 7f5d9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d9c076920 0x7f5d9c19e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47246/0 (socket says 172.21.15.86:47246) 2024-04-17T07:29:47.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.646+0000 7f5d9a800640 1 -- 172.21.15.86:0/664445824 learned_addr learned my addr 172.21.15.86:0/664445824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:47.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.646+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c113f60 con 0x7f5d9c076920 2024-04-17T07:29:47.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.646+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c113de0 con 0x7f5d9c077b60 2024-04-17T07:29:47.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.646+0000 7f5d99e00640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d9c077b60 0x7f5d9c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.646+0000 7f5d9b200640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c078500 0x7f5d9c1a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.646+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1140e0 con 0x7f5d9c078500 2024-04-17T07:29:47.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.647+0000 7f5d9a800640 1 -- 172.21.15.86:0/664445824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c078500 msgr2=0x7f5d9c1a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.647+0000 7f5d9a800640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c078500 0x7f5d9c1a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.647+0000 7f5d9a800640 1 -- 172.21.15.86:0/664445824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d9c077b60 msgr2=0x7f5d9c19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:47.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.647+0000 7f5d9a800640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d9c077b60 0x7f5d9c19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:47.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.647+0000 7f5d9a800640 1 -- 172.21.15.86:0/664445824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c1a6180 con 0x7f5d9c076920 2024-04-17T07:29:47.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.647+0000 7f5d9b200640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c078500 0x7f5d9c1a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:47.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.648+0000 7f5d9a800640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d9c076920 0x7f5d9c19e560 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c000c00 tx=0x7f5d8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.648+0000 7f5d93400640 1 -- 172.21.15.86:0/664445824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c043020 con 0x7f5d9c076920 2024-04-17T07:29:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.649+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c1a6410 con 0x7f5d9c076920 2024-04-17T07:29:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.649+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c1a69b0 con 0x7f5d9c076920 2024-04-17T07:29:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.649+0000 7f5d93400640 1 -- 172.21.15.86:0/664445824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d8c004aa0 con 0x7f5d9c076920 2024-04-17T07:29:47.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.649+0000 7f5d93400640 1 -- 172.21.15.86:0/664445824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c003c60 con 0x7f5d9c076920 2024-04-17T07:29:47.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.653+0000 7f5d93400640 1 -- 172.21.15.86:0/664445824 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5d8c046020 con 0x7f5d9c076920 2024-04-17T07:29:47.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.653+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d7c0052c0 con 0x7f5d9c076920 2024-04-17T07:29:47.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.655+0000 7f5d93400640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d800a3cb0 0x7f5d800a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:47.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.655+0000 7f5d93400640 1 -- 172.21.15.86:0/664445824 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5d8c0f2970 con 0x7f5d9c076920 2024-04-17T07:29:47.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.657+0000 7f5d99e00640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d800a3cb0 0x7f5d800a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:47.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.658+0000 7f5d99e00640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d800a3cb0 0x7f5d800a6170 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c19fa70 tx=0x7f5d8800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:47.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:47.662+0000 7f5d93400640 1 -- 172.21.15.86:0/664445824 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5d8c04d020 con 0x7f5d9c076920 2024-04-17T07:29:48.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.172+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d7c0055b0 con 0x7f5d9c076920 2024-04-17T07:29:48.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.173+0000 7f5d93400640 1 -- 172.21.15.86:0/664445824 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f5d8c0035f0 con 0x7f5d9c076920 2024-04-17T07:29:48.172 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:29:48.173 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:29:48.173 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:29:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.176+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d800a3cb0 msgr2=0x7f5d800a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.177+0000 7f5da0a00640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d800a3cb0 0x7f5d800a6170 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c19fa70 tx=0x7f5d8800a2d0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.177+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d9c076920 msgr2=0x7f5d9c19e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.177+0000 7f5da0a00640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d9c076920 0x7f5d9c19e560 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c000c00 tx=0x7f5d8c0042f0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.177+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 shutdown_connections 2024-04-17T07:29:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.177+0000 7f5da0a00640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5d800a3cb0 0x7f5d800a6170 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.177+0000 7f5da0a00640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c078500 0x7f5d9c1a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.177+0000 7f5da0a00640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d9c077b60 0x7f5d9c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.177+0000 7f5da0a00640 1 --2- 172.21.15.86:0/664445824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d9c076920 0x7f5d9c19e560 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.177+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 >> 172.21.15.86:0/664445824 conn(0x7f5d9c0ff570 msgr2=0x7f5d9c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.177+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 shutdown_connections 2024-04-17T07:29:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.178+0000 7f5da0a00640 1 -- 172.21.15.86:0/664445824 wait complete. 2024-04-17T07:29:48.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:29:48.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:48 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/664445824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:29:48.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:48 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/664445824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:29:48.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:48 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/664445824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:29:48.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.584+0000 7fb850000640 1 -- 172.21.15.86:0/2517281634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb848077b60 msgr2=0x7fb848077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.584+0000 7fb850000640 1 --2- 172.21.15.86:0/2517281634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb848077b60 0x7fb848077fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb834009900 tx=0x7fb83402bc40 comp rx=0 tx=0).stop 2024-04-17T07:29:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.584+0000 7fb850000640 1 -- 172.21.15.86:0/2517281634 shutdown_connections 2024-04-17T07:29:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.584+0000 7fb850000640 1 --2- 172.21.15.86:0/2517281634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb848078500 0x7fb848078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.584+0000 7fb850000640 1 --2- 172.21.15.86:0/2517281634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb848077b60 0x7fb848077fc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.585+0000 7fb850000640 1 --2- 172.21.15.86:0/2517281634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb848076920 0x7fb848076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.585+0000 7fb850000640 1 -- 172.21.15.86:0/2517281634 >> 172.21.15.86:0/2517281634 conn(0x7fb8480ff750 msgr2=0x7fb848101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.586+0000 7fb850000640 1 -- 172.21.15.86:0/2517281634 shutdown_connections 2024-04-17T07:29:48.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.586+0000 7fb850000640 1 -- 172.21.15.86:0/2517281634 wait complete. 2024-04-17T07:29:48.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.587+0000 7fb850000640 1 Processor -- start 2024-04-17T07:29:48.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.587+0000 7fb850000640 1 -- start start 2024-04-17T07:29:48.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.588+0000 7fb850000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb848076920 0x7fb84819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.588+0000 7fb850000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb848077b60 0x7fb84819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.588+0000 7fb850000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb848078500 0x7fb8481a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.588+0000 7fb850000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb848114120 con 0x7fb848077b60 2024-04-17T07:29:48.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.588+0000 7fb850000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb848113fa0 con 0x7fb848076920 2024-04-17T07:29:48.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.588+0000 7fb850000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481142a0 con 0x7fb848078500 2024-04-17T07:29:48.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.588+0000 7fb84e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb848078500 0x7fb8481a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.588+0000 7fb84e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb848078500 0x7fb8481a5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49730/0 (socket says 172.21.15.86:49730) 2024-04-17T07:29:48.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.588+0000 7fb84e000640 1 -- 172.21.15.86:0/2869617025 learned_addr learned my addr 172.21.15.86:0/2869617025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:48.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.589+0000 7fb84d600640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb848076920 0x7fb84819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.589+0000 7fb84cc00640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb848077b60 0x7fb84819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.590+0000 7fb84e000640 1 -- 172.21.15.86:0/2869617025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb848076920 msgr2=0x7fb84819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.590+0000 7fb84e000640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb848076920 0x7fb84819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.590+0000 7fb84e000640 1 -- 172.21.15.86:0/2869617025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb848077b60 msgr2=0x7fb84819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.590+0000 7fb84e000640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb848077b60 0x7fb84819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.590+0000 7fb84e000640 1 -- 172.21.15.86:0/2869617025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84806bab0 con 0x7fb848078500 2024-04-17T07:29:48.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.591+0000 7fb84e000640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb848078500 0x7fb8481a5dd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb84000eb10 tx=0x7fb84000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.592+0000 7fb846000640 1 -- 172.21.15.86:0/2869617025 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840004b20 con 0x7fb848078500 2024-04-17T07:29:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.592+0000 7fb846000640 1 -- 172.21.15.86:0/2869617025 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb840004cc0 con 0x7fb848078500 2024-04-17T07:29:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.592+0000 7fb846000640 1 -- 172.21.15.86:0/2869617025 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840010690 con 0x7fb848078500 2024-04-17T07:29:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.592+0000 7fb850000640 1 -- 172.21.15.86:0/2869617025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84806bda0 con 0x7fb848078500 2024-04-17T07:29:48.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.593+0000 7fb850000640 1 -- 172.21.15.86:0/2869617025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84806c270 con 0x7fb848078500 2024-04-17T07:29:48.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.598+0000 7fb852dc1740 1 Processor -- start 2024-04-17T07:29:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.600+0000 7fb846000640 1 -- 172.21.15.86:0/2869617025 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb840005020 con 0x7fb848078500 2024-04-17T07:29:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.601+0000 7fb852dc1740 1 -- start start 2024-04-17T07:29:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.601+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e17fa0 0x55ef73e183c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.601+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 0x55ef73e1d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.601+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e1ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.601+0000 7fb852dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef73d7e020 con 0x55ef73e1db00 2024-04-17T07:29:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.601+0000 7fb852dc1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ef73d24460 con 0x55ef73e17fa0 2024-04-17T07:29:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.601+0000 7fb852dc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef73d79fe0 con 0x55ef73e18990 2024-04-17T07:29:48.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.602+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e17fa0 0x55ef73e183c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.603+0000 7fb84d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 0x55ef73e1d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.603+0000 7fb84d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 0x55ef73e1d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49738/0 (socket says 172.21.15.86:49738) 2024-04-17T07:29:48.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.603+0000 7fb84d600640 1 -- 172.21.15.86:0/2542420892 learned_addr learned my addr 172.21.15.86:0/2542420892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:48.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.603+0000 7fb84d600640 1 -- 172.21.15.86:0/2542420892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e17fa0 msgr2=0x55ef73e183c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:48.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.603+0000 7fb84d600640 1 --2- 172.21.15.86:0/2542420892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e17fa0 0x55ef73e183c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.603+0000 7fb84d600640 1 -- 172.21.15.86:0/2542420892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 msgr2=0x55ef73e1ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:29:48.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.604+0000 7fb84d600640 1 --2- 172.21.15.86:0/2542420892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e1ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.604+0000 7fb84d600640 1 -- 172.21.15.86:0/2542420892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef73d2c040 con 0x55ef73e18990 2024-04-17T07:29:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.604+0000 7fb846000640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8300a3d00 0x7fb8300a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.605+0000 7fb846000640 1 -- 172.21.15.86:0/2869617025 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb8400d0d80 con 0x7fb848078500 2024-04-17T07:29:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.605+0000 7fb84d600640 1 --2- 172.21.15.86:0/2542420892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 0x55ef73e1d490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009cf0 tx=0x7fb83c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52b7b5696af0c871 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.605+0000 7fb84cc00640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8300a3d00 0x7fb8300a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.606+0000 7fb83a600640 1 -- 172.21.15.86:0/2542420892 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c004230 con 0x55ef73e18990 2024-04-17T07:29:48.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.606+0000 7fb83a600640 1 -- 172.21.15.86:0/2542420892 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb83c0043d0 con 0x55ef73e18990 2024-04-17T07:29:48.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.606+0000 7fb83a600640 1 -- 172.21.15.86:0/2542420892 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0197e0 con 0x55ef73e18990 2024-04-17T07:29:48.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.607+0000 7fb852dc1740 1 -- 172.21.15.86:0/2542420892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 msgr2=0x55ef73e1d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.607+0000 7fb852dc1740 1 --2- 172.21.15.86:0/2542420892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 0x55ef73e1d490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009cf0 tx=0x7fb83c011810 comp rx=0 tx=0).stop 2024-04-17T07:29:48.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.607+0000 7fb84cc00640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8300a3d00 0x7fb8300a61c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb834006010 tx=0x7fb83402c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.611+0000 7fb852dc1740 1 -- 172.21.15.86:0/2542420892 shutdown_connections 2024-04-17T07:29:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.611+0000 7fb852dc1740 1 --2- 172.21.15.86:0/2542420892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e1ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.611+0000 7fb852dc1740 1 --2- 172.21.15.86:0/2542420892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 0x55ef73e1d490 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.611+0000 7fb852dc1740 1 --2- 172.21.15.86:0/2542420892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e17fa0 0x55ef73e183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.611+0000 7fb852dc1740 1 -- 172.21.15.86:0/2542420892 >> 172.21.15.86:0/2542420892 conn(0x55ef73e13510 msgr2=0x55ef73e13910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.611+0000 7fb852dc1740 1 -- 172.21.15.86:0/2542420892 shutdown_connections 2024-04-17T07:29:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.612+0000 7fb852dc1740 1 -- 172.21.15.86:0/2542420892 wait complete. 2024-04-17T07:29:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.613+0000 7fb852dc1740 1 -- start start 2024-04-17T07:29:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.614+0000 7fb852dc1740 1 Processor -- start 2024-04-17T07:29:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.615+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 0x55ef73e224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.615+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e18990 0x55ef73e22a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.615+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e1db00 0x55ef73e29a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.615+0000 7fb852dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef73e20ab0 con 0x55ef73e18990 2024-04-17T07:29:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.615+0000 7fb852dc1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ef73d37830 con 0x55ef73e1db00 2024-04-17T07:29:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.615+0000 7fb852dc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef73e20c30 con 0x55ef73e17fa0 2024-04-17T07:29:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.615+0000 7fb852dc1740 10 client.0 _ll_register_callbacks cb 0x55ef73d05fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.616+0000 7fb84e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e1db00 0x55ef73e29a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.616+0000 7fb84e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e1db00 0x55ef73e29a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45396/0 (socket says 172.21.15.86:45396) 2024-04-17T07:29:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.616+0000 7fb84e000640 1 -- 172.21.15.86:0/4125238665 learned_addr learned my addr 172.21.15.86:0/4125238665 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.616+0000 7fb84d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 0x55ef73e224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.616+0000 7fb84cc00640 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e18990 0x55ef73e22a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.617+0000 7fb84e000640 1 -- 172.21.15.86:0/4125238665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 msgr2=0x55ef73e224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.617+0000 7fb84e000640 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 0x55ef73e224e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.617+0000 7fb84e000640 1 -- 172.21.15.86:0/4125238665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e18990 msgr2=0x55ef73e22a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.617+0000 7fb84e000640 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e18990 0x55ef73e22a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.617+0000 7fb84e000640 1 -- 172.21.15.86:0/4125238665 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef73d3b920 con 0x55ef73e1db00 2024-04-17T07:29:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.617+0000 7fb84e000640 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e1db00 0x55ef73e29a60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb840005850 tx=0x7fb84009c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc8b6e3a79f0d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.617+0000 7fb82ea00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:29:48.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.618+0000 7fb82ea00640 1 -- 172.21.15.86:0/4125238665 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8400bd270 con 0x55ef73e1db00 2024-04-17T07:29:48.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.618+0000 7fb852dc1740 1 -- 172.21.15.86:0/4125238665 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83c0043d0 con 0x55ef73e1db00 2024-04-17T07:29:48.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.618+0000 7fb82ea00640 1 -- 172.21.15.86:0/4125238665 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb8400a5460 con 0x55ef73e1db00 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.618+0000 7fb82ea00640 1 -- 172.21.15.86:0/4125238665 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8400b6200 con 0x55ef73e1db00 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.618+0000 7fb852dc1740 10 client.37970 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.618+0000 7fb852dc1740 1 -- 172.21.15.86:0/4125238665 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef73d097f0 con 0x55ef73e1db00 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.619+0000 7fb82ea00640 1 -- 172.21.15.86:0/4125238665 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb83c0043d0 con 0x55ef73e1db00 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.619+0000 7fb82ea00640 1 -- 172.21.15.86:0/4125238665 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb8400989d0 con 0x55ef73e1db00 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.619+0000 7fb852dc1740 10 client.37970 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.619+0000 7fb852dc1740 10 client.37970 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.619+0000 7fb852dc1740 20 client.37970 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.619+0000 7fb852dc1740 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ef73e2de90 0x55ef73e302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.619+0000 7fb852dc1740 4 client.37970 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:48.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.619+0000 7fb852dc1740 1 -- 172.21.15.86:0/4125238665 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef73d8ba40 con 0x55ef73e2de90 2024-04-17T07:29:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.620+0000 7fb84d600640 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ef73e2de90 0x55ef73e302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.621+0000 7fb84d600640 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ef73e2de90 0x55ef73e302f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c9f5fc82c15524df server_cookie=19b2e5bc79463a00 in_seq=0 out_seq=0 2024-04-17T07:29:48.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.621+0000 7fb82ea00640 10 client.37970 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:29:48.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.622+0000 7fb82ea00640 1 -- 172.21.15.86:0/4125238665 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef73d8ba40 con 0x55ef73e2de90 2024-04-17T07:29:48.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.622+0000 7fb82ea00640 10 client.37970 handle_command_reply: tid=0 2024-04-17T07:29:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.623+0000 7fb852dc1740 1 client.37970 shutdown 2024-04-17T07:29:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.624+0000 7fb852dc1740 1 -- 172.21.15.86:0/4125238665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e1db00 msgr2=0x55ef73e29a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.624+0000 7fb852dc1740 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e1db00 0x55ef73e29a60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb840005850 tx=0x7fb84009c6a0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.625+0000 7fb852dc1740 1 -- 172.21.15.86:0/4125238665 shutdown_connections 2024-04-17T07:29:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.625+0000 7fb852dc1740 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ef73e2de90 0x55ef73e302f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.625+0000 7fb852dc1740 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e1db00 0x55ef73e29a60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.625+0000 7fb852dc1740 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e18990 0x55ef73e22a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.625+0000 7fb852dc1740 1 --2- 172.21.15.86:0/4125238665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 0x55ef73e224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.625+0000 7fb852dc1740 1 -- 172.21.15.86:0/4125238665 >> 172.21.15.86:0/4125238665 conn(0x55ef73e13510 msgr2=0x55ef73e12710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.625+0000 7fb82ea00640 0 client.37970 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:29:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.625+0000 7fb852dc1740 1 -- 172.21.15.86:0/4125238665 shutdown_connections 2024-04-17T07:29:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.625+0000 7fb852dc1740 1 -- 172.21.15.86:0/4125238665 wait complete. 2024-04-17T07:29:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.625+0000 7fb852dc1740 20 client.37970 trim_cache size 0 max 16384 2024-04-17T07:29:48.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.651+0000 7fb852dc1740 1 Processor -- start 2024-04-17T07:29:48.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.651+0000 7fb852dc1740 1 -- start start 2024-04-17T07:29:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.651+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e17fa0 0x55ef73e19e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.651+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 0x55ef73e29a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.651+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e1a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.651+0000 7fb852dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481b9a20 con 0x55ef73e1db00 2024-04-17T07:29:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.651+0000 7fb852dc1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481bb560 con 0x55ef73e17fa0 2024-04-17T07:29:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.651+0000 7fb852dc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481ba0f0 con 0x55ef73e18990 2024-04-17T07:29:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.652+0000 7fb84e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e1a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.652+0000 7fb84e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e1a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47288/0 (socket says 172.21.15.86:47288) 2024-04-17T07:29:48.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.652+0000 7fb84d600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e17fa0 0x55ef73e19e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.652+0000 7fb84e000640 1 -- 172.21.15.86:0/1771047305 learned_addr learned my addr 172.21.15.86:0/1771047305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.653+0000 7fb84cc00640 1 --2- 172.21.15.86:0/1771047305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 0x55ef73e29a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.653+0000 7fb84e000640 1 -- 172.21.15.86:0/1771047305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 msgr2=0x55ef73e29a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.653+0000 7fb84e000640 1 --2- 172.21.15.86:0/1771047305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 0x55ef73e29a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.653+0000 7fb84e000640 1 -- 172.21.15.86:0/1771047305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e17fa0 msgr2=0x55ef73e19e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.653+0000 7fb84e000640 1 --2- 172.21.15.86:0/1771047305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e17fa0 0x55ef73e19e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.653+0000 7fb84e000640 1 -- 172.21.15.86:0/1771047305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef73a903f0 con 0x55ef73e1db00 2024-04-17T07:29:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.653+0000 7fb84e000640 1 --2- 172.21.15.86:0/1771047305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e1a3d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb8400049d0 tx=0x7fb84009ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bf38a0a543d825 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.654+0000 7fb839c00640 1 -- 172.21.15.86:0/1771047305 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840080070 con 0x55ef73e1db00 2024-04-17T07:29:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.654+0000 7fb839c00640 1 -- 172.21.15.86:0/1771047305 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8400a1030 con 0x55ef73e1db00 2024-04-17T07:29:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.655+0000 7fb839c00640 1 -- 172.21.15.86:0/1771047305 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8400a5460 con 0x55ef73e1db00 2024-04-17T07:29:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.656+0000 7fb852dc1740 1 -- 172.21.15.86:0/1771047305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 msgr2=0x55ef73e1a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.656+0000 7fb852dc1740 1 --2- 172.21.15.86:0/1771047305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e1a3d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb8400049d0 tx=0x7fb84009ce00 comp rx=0 tx=0).stop 2024-04-17T07:29:48.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.657+0000 7fb852dc1740 1 -- 172.21.15.86:0/1771047305 shutdown_connections 2024-04-17T07:29:48.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.657+0000 7fb852dc1740 1 --2- 172.21.15.86:0/1771047305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e1a3d0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.657+0000 7fb852dc1740 1 --2- 172.21.15.86:0/1771047305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e18990 0x55ef73e29a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.657+0000 7fb852dc1740 1 --2- 172.21.15.86:0/1771047305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e17fa0 0x55ef73e19e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.657+0000 7fb852dc1740 1 -- 172.21.15.86:0/1771047305 >> 172.21.15.86:0/1771047305 conn(0x55ef73e13510 msgr2=0x55ef73e20fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:48.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.657+0000 7fb852dc1740 1 -- 172.21.15.86:0/1771047305 shutdown_connections 2024-04-17T07:29:48.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.657+0000 7fb852dc1740 1 -- 172.21.15.86:0/1771047305 wait complete. 2024-04-17T07:29:48.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.658+0000 7fb852dc1740 1 -- start start 2024-04-17T07:29:48.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.658+0000 7fb852dc1740 1 Processor -- start 2024-04-17T07:29:48.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.659+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 0x55ef73e26af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.659+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e18990 0x55ef73e27030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.660+0000 7fb852dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e28040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.660+0000 7fb84d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 0x55ef73e26af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.660+0000 7fb852dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481bae90 con 0x55ef73e1db00 2024-04-17T07:29:48.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.660+0000 7fb852dc1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8481ba7c0 con 0x55ef73e18990 2024-04-17T07:29:48.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.660+0000 7fb852dc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb848073f50 con 0x55ef73e17fa0 2024-04-17T07:29:48.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.660+0000 7fb84e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e28040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.660+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e18990 0x55ef73e27030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.661+0000 7fb84d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 0x55ef73e26af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49754/0 (socket says 172.21.15.86:49754) 2024-04-17T07:29:48.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.661+0000 7fb84e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e28040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47304/0 (socket says 172.21.15.86:47304) 2024-04-17T07:29:48.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.661+0000 7fb84d600640 1 -- 172.21.15.86:0/393183745 learned_addr learned my addr 172.21.15.86:0/393183745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:48.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.661+0000 7fb852dc1740 10 client.0 _ll_register_callbacks cb 0x7fb84009cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:48.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.662+0000 7fb84d600640 1 -- 172.21.15.86:0/393183745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e18990 msgr2=0x55ef73e27030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:48.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.662+0000 7fb84d600640 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e18990 0x55ef73e27030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.662+0000 7fb84d600640 1 -- 172.21.15.86:0/393183745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 msgr2=0x55ef73e28040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:48.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.662+0000 7fb84d600640 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e28040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.662+0000 7fb84d600640 1 -- 172.21.15.86:0/393183745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef73b5a7d0 con 0x55ef73e17fa0 2024-04-17T07:29:48.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.663+0000 7fb84e000640 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e28040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:48.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.664+0000 7fb84d600640 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 0x55ef73e26af0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb83c000c00 tx=0x7fb83c02cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=689736289c6e14bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:48.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.664+0000 7fb82ea00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:48.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.665+0000 7fb82ea00640 1 -- 172.21.15.86:0/393183745 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c003e80 con 0x55ef73e17fa0 2024-04-17T07:29:48.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.665+0000 7fb82ea00640 1 -- 172.21.15.86:0/393183745 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb83c02b070 con 0x55ef73e17fa0 2024-04-17T07:29:48.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.665+0000 7fb852dc1740 1 -- 172.21.15.86:0/393183745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8400a1030 con 0x55ef73e17fa0 2024-04-17T07:29:48.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.666+0000 7fb82ea00640 1 -- 172.21.15.86:0/393183745 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb83c004020 con 0x55ef73e17fa0 2024-04-17T07:29:48.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.666+0000 7fb852dc1740 10 client.28018 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:48.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.666+0000 7fb852dc1740 1 -- 172.21.15.86:0/393183745 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef73e29760 con 0x55ef73e17fa0 2024-04-17T07:29:48.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.667+0000 7fb82ea00640 1 -- 172.21.15.86:0/393183745 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8400a1030 con 0x55ef73e17fa0 2024-04-17T07:29:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.667+0000 7fb82ea00640 1 -- 172.21.15.86:0/393183745 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb83c003760 con 0x55ef73e17fa0 2024-04-17T07:29:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.667+0000 7fb852dc1740 10 client.28018 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:48.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.667+0000 7fb852dc1740 10 client.28018 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:29:48.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.667+0000 7fb852dc1740 20 client.28018 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:48.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.667+0000 7fb852dc1740 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ef73e94430 0x55ef73e96820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:48.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.667+0000 7fb852dc1740 4 client.28018 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:29:48.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.667+0000 7fb852dc1740 1 -- 172.21.15.86:0/393183745 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef73d858b0 con 0x55ef73e94430 2024-04-17T07:29:48.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.668+0000 7fb84cc00640 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ef73e94430 0x55ef73e96820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.669+0000 7fb84cc00640 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ef73e94430 0x55ef73e96820 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f51e73576fd97be server_cookie=6885c5f82afad525 in_seq=0 out_seq=0 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.669+0000 7fb82ea00640 10 client.28018 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.669+0000 7fb82ea00640 1 -- 172.21.15.86:0/393183745 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef73d858b0 con 0x55ef73e94430 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.669+0000 7fb82ea00640 10 client.28018 handle_command_reply: tid=0 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.669+0000 7fb852dc1740 1 client.28018 shutdown 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.669+0000 7fb852dc1740 1 -- 172.21.15.86:0/393183745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 msgr2=0x55ef73e26af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.669+0000 7fb852dc1740 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 0x55ef73e26af0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb83c000c00 tx=0x7fb83c02cb50 comp rx=0 tx=0).stop 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.670+0000 7fb852dc1740 1 -- 172.21.15.86:0/393183745 shutdown_connections 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.670+0000 7fb852dc1740 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ef73e94430 0x55ef73e96820 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.670+0000 7fb852dc1740 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef73e1db00 0x55ef73e28040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.670+0000 7fb852dc1740 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef73e18990 0x55ef73e27030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.670+0000 7fb852dc1740 1 --2- 172.21.15.86:0/393183745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73e17fa0 0x55ef73e26af0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.670+0000 7fb852dc1740 1 -- 172.21.15.86:0/393183745 >> 172.21.15.86:0/393183745 conn(0x55ef73e13510 msgr2=0x55ef73e19e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:48.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.670+0000 7fb82ea00640 0 client.28018 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:29:48.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.670+0000 7fb852dc1740 1 -- 172.21.15.86:0/393183745 shutdown_connections 2024-04-17T07:29:48.670 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:29:48.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.670+0000 7fb852dc1740 1 -- 172.21.15.86:0/393183745 wait complete. 2024-04-17T07:29:48.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.671+0000 7fb852dc1740 20 client.28018 trim_cache size 0 max 16384 2024-04-17T07:29:48.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.672+0000 7fb850000640 1 -- 172.21.15.86:0/2869617025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8300a3d00 msgr2=0x7fb8300a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.672+0000 7fb850000640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8300a3d00 0x7fb8300a61c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb834006010 tx=0x7fb83402c7f0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.672+0000 7fb850000640 1 -- 172.21.15.86:0/2869617025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb848078500 msgr2=0x7fb8481a5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:48.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.672+0000 7fb850000640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb848078500 0x7fb8481a5dd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb84000eb10 tx=0x7fb84000c2d0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.672+0000 7fb850000640 1 -- 172.21.15.86:0/2869617025 shutdown_connections 2024-04-17T07:29:48.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.672+0000 7fb850000640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8300a3d00 0x7fb8300a61c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.672+0000 7fb850000640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb848078500 0x7fb8481a5dd0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.672+0000 7fb850000640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb848077b60 0x7fb84819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.672+0000 7fb850000640 1 --2- 172.21.15.86:0/2869617025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb848076920 0x7fb84819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:48.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.672+0000 7fb850000640 1 -- 172.21.15.86:0/2869617025 >> 172.21.15.86:0/2869617025 conn(0x7fb8480ff750 msgr2=0x7fb848101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:48.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.673+0000 7fb850000640 1 -- 172.21.15.86:0/2869617025 shutdown_connections 2024-04-17T07:29:48.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:48.673+0000 7fb850000640 1 -- 172.21.15.86:0/2869617025 wait complete. 2024-04-17T07:29:48.698 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:29:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.027+0000 7fdd96800640 1 -- 172.21.15.86:0/1743630540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd90105a10 msgr2=0x7fdd90105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.027+0000 7fdd96800640 1 --2- 172.21.15.86:0/1743630540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd90105a10 0x7fdd90105e90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009be0 tx=0x7fdd8802bef0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.027+0000 7fdd96800640 1 -- 172.21.15.86:0/1743630540 shutdown_connections 2024-04-17T07:29:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.027+0000 7fdd96800640 1 --2- 172.21.15.86:0/1743630540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd90105a10 0x7fdd90105e90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.027+0000 7fdd96800640 1 --2- 172.21.15.86:0/1743630540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd90103060 0x7fdd901054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.027+0000 7fdd96800640 1 --2- 172.21.15.86:0/1743630540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd901006f0 0x7fdd90102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.027+0000 7fdd96800640 1 -- 172.21.15.86:0/1743630540 >> 172.21.15.86:0/1743630540 conn(0x7fdd900fc310 msgr2=0x7fdd900fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.028+0000 7fdd96800640 1 -- 172.21.15.86:0/1743630540 shutdown_connections 2024-04-17T07:29:49.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.028+0000 7fdd96800640 1 -- 172.21.15.86:0/1743630540 wait complete. 2024-04-17T07:29:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.030+0000 7fdd96800640 1 Processor -- start 2024-04-17T07:29:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.030+0000 7fdd96800640 1 -- start start 2024-04-17T07:29:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.030+0000 7fdd96800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd901006f0 0x7fdd9019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.030+0000 7fdd96800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd90103060 0x7fdd9019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.030+0000 7fdd96800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd90105a10 0x7fdd901a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.030+0000 7fdd96800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90113fd0 con 0x7fdd90103060 2024-04-17T07:29:49.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.030+0000 7fdd96800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90113e50 con 0x7fdd90105a10 2024-04-17T07:29:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.030+0000 7fdd96800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90114150 con 0x7fdd901006f0 2024-04-17T07:29:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.031+0000 7fdd94a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd90103060 0x7fdd9019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.031+0000 7fdd94a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd90103060 0x7fdd9019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47326/0 (socket says 172.21.15.86:47326) 2024-04-17T07:29:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.031+0000 7fdd94a00640 1 -- 172.21.15.86:0/831774664 learned_addr learned my addr 172.21.15.86:0/831774664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.031+0000 7fdd95400640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd901006f0 0x7fdd9019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.031+0000 7fdd95e00640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd90105a10 0x7fdd901a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.032+0000 7fdd94a00640 1 -- 172.21.15.86:0/831774664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd901006f0 msgr2=0x7fdd9019e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.032+0000 7fdd94a00640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd901006f0 0x7fdd9019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.032+0000 7fdd94a00640 1 -- 172.21.15.86:0/831774664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd90105a10 msgr2=0x7fdd901a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.032+0000 7fdd94a00640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd90105a10 0x7fdd901a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.032+0000 7fdd94a00640 1 -- 172.21.15.86:0/831774664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd901a6230 con 0x7fdd90103060 2024-04-17T07:29:49.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.032+0000 7fdd94a00640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd90103060 0x7fdd9019eb50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00ea60 tx=0x7fdd7c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.033+0000 7fdd8e000640 1 -- 172.21.15.86:0/831774664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00cde0 con 0x7fdd90103060 2024-04-17T07:29:49.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.033+0000 7fdd8e000640 1 -- 172.21.15.86:0/831774664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd7c0035f0 con 0x7fdd90103060 2024-04-17T07:29:49.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.033+0000 7fdd8e000640 1 -- 172.21.15.86:0/831774664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c010690 con 0x7fdd90103060 2024-04-17T07:29:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.034+0000 7fdd96800640 1 -- 172.21.15.86:0/831774664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd901a6520 con 0x7fdd90103060 2024-04-17T07:29:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.034+0000 7fdd96800640 1 -- 172.21.15.86:0/831774664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd901a69f0 con 0x7fdd90103060 2024-04-17T07:29:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.039+0000 7fdd9abb0740 1 Processor -- start 2024-04-17T07:29:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.040+0000 7fdd8e000640 1 -- 172.21.15.86:0/831774664 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fdd7c0037a0 con 0x7fdd90103060 2024-04-17T07:29:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.040+0000 7fdd9abb0740 1 -- start start 2024-04-17T07:29:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.040+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e78580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.041+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e78b50 0x555d42e7d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.041+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e7dcc0 0x555d42e80150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.041+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555d42dde020 con 0x555d42e7dcc0 2024-04-17T07:29:49.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.041+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555d42d84460 con 0x555d42e78160 2024-04-17T07:29:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.041+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555d42dd9fe0 con 0x555d42e78b50 2024-04-17T07:29:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.041+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.041+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e78580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45442/0 (socket says 172.21.15.86:45442) 2024-04-17T07:29:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.041+0000 7fdd95e00640 1 -- 172.21.15.86:0/3665227447 learned_addr learned my addr 172.21.15.86:0/3665227447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.041+0000 7fdd95400640 1 --2- 172.21.15.86:0/3665227447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e78b50 0x555d42e7d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.042+0000 7fdd94a00640 1 --2- 172.21.15.86:0/3665227447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e7dcc0 0x555d42e80150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.042+0000 7fdd95e00640 1 -- 172.21.15.86:0/3665227447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e78b50 msgr2=0x555d42e7d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.042+0000 7fdd95e00640 1 --2- 172.21.15.86:0/3665227447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e78b50 0x555d42e7d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.042+0000 7fdd95e00640 1 -- 172.21.15.86:0/3665227447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e7dcc0 msgr2=0x555d42e80150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.042+0000 7fdd95e00640 1 --2- 172.21.15.86:0/3665227447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e7dcc0 0x555d42e80150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.042+0000 7fdd95e00640 1 -- 172.21.15.86:0/3665227447 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d42d8c040 con 0x555d42e78160 2024-04-17T07:29:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.042+0000 7fdd94a00640 1 --2- 172.21.15.86:0/3665227447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e7dcc0 0x555d42e80150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:29:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.043+0000 7fdd8e000640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd780a3c80 0x7fdd780a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.043+0000 7fdd95e00640 1 --2- 172.21.15.86:0/3665227447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e78580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009be0 tx=0x7fdd88009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c901e0001030af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.043+0000 7fdd8e000640 1 -- 172.21.15.86:0/831774664 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdd7c0d0bc0 con 0x7fdd90103060 2024-04-17T07:29:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.043+0000 7fdd95400640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd780a3c80 0x7fdd780a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.044+0000 7fdd82600640 1 -- 172.21.15.86:0/3665227447 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88043070 con 0x555d42e78160 2024-04-17T07:29:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.044+0000 7fdd82600640 1 -- 172.21.15.86:0/3665227447 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd880334d0 con 0x555d42e78160 2024-04-17T07:29:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.044+0000 7fdd82600640 1 -- 172.21.15.86:0/3665227447 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8802c890 con 0x555d42e78160 2024-04-17T07:29:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.045+0000 7fdd9abb0740 1 -- 172.21.15.86:0/3665227447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 msgr2=0x555d42e78580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.045+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/3665227447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e78580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009be0 tx=0x7fdd88009940 comp rx=0 tx=0).stop 2024-04-17T07:29:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.045+0000 7fdd95400640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd780a3c80 0x7fdd780a6140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x555d42e78f50 tx=0x7fdd840093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.046+0000 7fdd9abb0740 1 -- 172.21.15.86:0/3665227447 shutdown_connections 2024-04-17T07:29:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.046+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/3665227447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e7dcc0 0x555d42e80150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.046+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/3665227447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e78b50 0x555d42e7d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.046+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/3665227447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e78580 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.046+0000 7fdd9abb0740 1 -- 172.21.15.86:0/3665227447 >> 172.21.15.86:0/3665227447 conn(0x555d42e736d0 msgr2=0x555d42e73ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.046+0000 7fdd9abb0740 1 -- 172.21.15.86:0/3665227447 shutdown_connections 2024-04-17T07:29:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.046+0000 7fdd9abb0740 1 -- 172.21.15.86:0/3665227447 wait complete. 2024-04-17T07:29:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.047+0000 7fdd9abb0740 1 -- start start 2024-04-17T07:29:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.047+0000 7fdd9abb0740 1 Processor -- start 2024-04-17T07:29:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78160 0x555d42e826a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78b50 0x555d42e82be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 0x555d42e89c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555d42e80c10 con 0x555d42e78160 2024-04-17T07:29:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555d42d97830 con 0x555d42e78b50 2024-04-17T07:29:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555d42e80d90 con 0x555d42e7dcc0 2024-04-17T07:29:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd9abb0740 10 client.0 _ll_register_callbacks cb 0x555d42d65fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 0x555d42e89c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd94a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78b50 0x555d42e82be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 0x555d42e89c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49782/0 (socket says 172.21.15.86:49782) 2024-04-17T07:29:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.048+0000 7fdd95e00640 1 -- 172.21.15.86:0/1494814455 learned_addr learned my addr 172.21.15.86:0/1494814455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.049+0000 7fdd95e00640 1 -- 172.21.15.86:0/1494814455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78b50 msgr2=0x555d42e82be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.049+0000 7fdd95e00640 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78b50 0x555d42e82be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.049+0000 7fdd95e00640 1 -- 172.21.15.86:0/1494814455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78160 msgr2=0x555d42e826a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.049+0000 7fdd95e00640 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78160 0x555d42e826a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.049+0000 7fdd95e00640 1 -- 172.21.15.86:0/1494814455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d42d9b920 con 0x555d42e7dcc0 2024-04-17T07:29:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.049+0000 7fdd95e00640 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 0x555d42e89c20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd88036040 tx=0x7fdd88033e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bce239325cac2f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.050+0000 7fdd76a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.050+0000 7fdd76a00640 1 -- 172.21.15.86:0/1494814455 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88043070 con 0x555d42e7dcc0 2024-04-17T07:29:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.050+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1494814455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd880334d0 con 0x555d42e7dcc0 2024-04-17T07:29:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.050+0000 7fdd76a00640 1 -- 172.21.15.86:0/1494814455 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdd8802c890 con 0x555d42e7dcc0 2024-04-17T07:29:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.050+0000 7fdd76a00640 1 -- 172.21.15.86:0/1494814455 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd88003e30 con 0x555d42e7dcc0 2024-04-17T07:29:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.051+0000 7fdd9abb0740 10 client.28027 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.051+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1494814455 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d42d697f0 con 0x555d42e7dcc0 2024-04-17T07:29:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.052+0000 7fdd76a00640 1 -- 172.21.15.86:0/1494814455 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd880334d0 con 0x555d42e7dcc0 2024-04-17T07:29:49.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.053+0000 7fdd76a00640 1 -- 172.21.15.86:0/1494814455 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fdd880046c0 con 0x555d42e7dcc0 2024-04-17T07:29:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.054+0000 7fdd9abb0740 10 client.28027 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.054+0000 7fdd9abb0740 10 client.28027 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:29:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.054+0000 7fdd9abb0740 20 client.28027 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.054+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x555d42e8dd80 0x555d42e90170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.054+0000 7fdd9abb0740 4 client.28027 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.054+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1494814455 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d42deba40 con 0x555d42e8dd80 2024-04-17T07:29:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.055+0000 7fdd95400640 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x555d42e8dd80 0x555d42e90170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.058+0000 7fdd95400640 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x555d42e8dd80 0x555d42e90170 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9034d3d937afa730 server_cookie=b6be1f781d124953 in_seq=0 out_seq=0 2024-04-17T07:29:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.058+0000 7fdd76a00640 10 client.28027 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:29:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.060+0000 7fdd76a00640 1 -- 172.21.15.86:0/1494814455 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d42deba40 con 0x555d42e8dd80 2024-04-17T07:29:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.060+0000 7fdd76a00640 10 client.28027 handle_command_reply: tid=0 2024-04-17T07:29:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.061+0000 7fdd9abb0740 1 client.28027 shutdown 2024-04-17T07:29:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.061+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1494814455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 msgr2=0x555d42e89c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.061+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 0x555d42e89c20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd88036040 tx=0x7fdd88033e90 comp rx=0 tx=0).stop 2024-04-17T07:29:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.062+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1494814455 shutdown_connections 2024-04-17T07:29:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.062+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x555d42e8dd80 0x555d42e90170 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.062+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 0x555d42e89c20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.062+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78b50 0x555d42e82be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.062+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1494814455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78160 0x555d42e826a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.062+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1494814455 >> 172.21.15.86:0/1494814455 conn(0x555d42e736d0 msgr2=0x555d42e72930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.062+0000 7fdd76a00640 0 client.28027 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:29:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.063+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1494814455 shutdown_connections 2024-04-17T07:29:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.063+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1494814455 wait complete. 2024-04-17T07:29:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.063+0000 7fdd9abb0740 20 client.28027 trim_cache size 0 max 16384 2024-04-17T07:29:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.098+0000 7fdd9abb0740 1 Processor -- start 2024-04-17T07:29:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.098+0000 7fdd9abb0740 1 -- start start 2024-04-17T07:29:49.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.098+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.098+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78b50 0x555d42e89be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.098+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 0x555d42e949e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.098+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901b0f80 con 0x555d42e78b50 2024-04-17T07:29:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.098+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901b2ac0 con 0x555d42e78160 2024-04-17T07:29:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.098+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901b1650 con 0x555d42e7dcc0 2024-04-17T07:29:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.099+0000 7fdd95400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.099+0000 7fdd95400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45454/0 (socket says 172.21.15.86:45454) 2024-04-17T07:29:49.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.099+0000 7fdd95400640 1 -- 172.21.15.86:0/2820920542 learned_addr learned my addr 172.21.15.86:0/2820920542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.099+0000 7fdd94a00640 1 --2- 172.21.15.86:0/2820920542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78b50 0x555d42e89be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.099+0000 7fdd95400640 1 -- 172.21.15.86:0/2820920542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 msgr2=0x555d42e949e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.099+0000 7fdd95400640 1 --2- 172.21.15.86:0/2820920542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 0x555d42e949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.099+0000 7fdd95400640 1 -- 172.21.15.86:0/2820920542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78b50 msgr2=0x555d42e89be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.099+0000 7fdd95400640 1 --2- 172.21.15.86:0/2820920542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78b50 0x555d42e89be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.099+0000 7fdd95400640 1 -- 172.21.15.86:0/2820920542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d42af03f0 con 0x555d42e78160 2024-04-17T07:29:49.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.100+0000 7fdd95400640 1 --2- 172.21.15.86:0/2820920542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e727b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x555d42e90990 tx=0x7fdd84014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da6d908ab4a659b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.101+0000 7fdd81c00640 1 -- 172.21.15.86:0/2820920542 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84027bc0 con 0x555d42e78160 2024-04-17T07:29:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.101+0000 7fdd81c00640 1 -- 172.21.15.86:0/2820920542 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd84027d60 con 0x555d42e78160 2024-04-17T07:29:49.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.101+0000 7fdd81c00640 1 -- 172.21.15.86:0/2820920542 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840047a0 con 0x555d42e78160 2024-04-17T07:29:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.102+0000 7fdd9abb0740 1 -- 172.21.15.86:0/2820920542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 msgr2=0x555d42e727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.102+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/2820920542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e727b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x555d42e90990 tx=0x7fdd84014000 comp rx=0 tx=0).stop 2024-04-17T07:29:49.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.102+0000 7fdd9abb0740 1 -- 172.21.15.86:0/2820920542 shutdown_connections 2024-04-17T07:29:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.103+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/2820920542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e7dcc0 0x555d42e949e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.103+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/2820920542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78b50 0x555d42e89be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.103+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/2820920542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e78160 0x555d42e727b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.103+0000 7fdd9abb0740 1 -- 172.21.15.86:0/2820920542 >> 172.21.15.86:0/2820920542 conn(0x555d42e736d0 msgr2=0x555d42e720a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.103+0000 7fdd9abb0740 1 -- 172.21.15.86:0/2820920542 shutdown_connections 2024-04-17T07:29:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.103+0000 7fdd9abb0740 1 -- 172.21.15.86:0/2820920542 wait complete. 2024-04-17T07:29:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.104+0000 7fdd9abb0740 1 -- start start 2024-04-17T07:29:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.104+0000 7fdd9abb0740 1 Processor -- start 2024-04-17T07:29:49.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.104+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e78160 0x555d42e875a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.105+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78b50 0x555d42e87ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.105+0000 7fdd9abb0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e7dcc0 0x555d42e88af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.105+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901b23f0 con 0x555d42e78b50 2024-04-17T07:29:49.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.105+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901b1d20 con 0x555d42e7dcc0 2024-04-17T07:29:49.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.105+0000 7fdd9abb0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901aef40 con 0x555d42e78160 2024-04-17T07:29:49.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.105+0000 7fdd9abb0740 10 client.0 _ll_register_callbacks cb 0x7fdd88033d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:49.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.105+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e7dcc0 0x555d42e88af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.105+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e7dcc0 0x555d42e88af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45466/0 (socket says 172.21.15.86:45466) 2024-04-17T07:29:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.105+0000 7fdd95e00640 1 -- 172.21.15.86:0/1530810576 learned_addr learned my addr 172.21.15.86:0/1530810576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.106+0000 7fdd94a00640 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78b50 0x555d42e87ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.106+0000 7fdd95400640 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e78160 0x555d42e875a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.106+0000 7fdd95e00640 1 -- 172.21.15.86:0/1530810576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e78160 msgr2=0x555d42e875a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.106+0000 7fdd95e00640 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e78160 0x555d42e875a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.106+0000 7fdd95e00640 1 -- 172.21.15.86:0/1530810576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78b50 msgr2=0x555d42e87ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.106+0000 7fdd95e00640 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78b50 0x555d42e87ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.106+0000 7fdd95e00640 1 -- 172.21.15.86:0/1530810576 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d42bba7d0 con 0x555d42e7dcc0 2024-04-17T07:29:49.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.107+0000 7fdd95e00640 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e7dcc0 0x555d42e88af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdd88004af0 tx=0x7fdd880021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1972287cc5b806e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.107+0000 7fdd76a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:29:49.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.108+0000 7fdd76a00640 1 -- 172.21.15.86:0/1530810576 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88043070 con 0x555d42e7dcc0 2024-04-17T07:29:49.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.108+0000 7fdd76a00640 1 -- 172.21.15.86:0/1530810576 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdd880039f0 con 0x555d42e7dcc0 2024-04-17T07:29:49.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.108+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1530810576 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd84027d60 con 0x555d42e7dcc0 2024-04-17T07:29:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.109+0000 7fdd76a00640 1 -- 172.21.15.86:0/1530810576 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd88034720 con 0x555d42e7dcc0 2024-04-17T07:29:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.109+0000 7fdd9abb0740 10 client.37994 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:49.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.109+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1530810576 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d42ef4350 con 0x555d42e7dcc0 2024-04-17T07:29:49.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.110+0000 7fdd76a00640 1 -- 172.21.15.86:0/1530810576 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd84027d60 con 0x555d42e7dcc0 2024-04-17T07:29:49.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.110+0000 7fdd76a00640 1 -- 172.21.15.86:0/1530810576 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fdd88003fd0 con 0x555d42e7dcc0 2024-04-17T07:29:49.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.111+0000 7fdd9abb0740 10 client.37994 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:49.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.111+0000 7fdd9abb0740 10 client.37994 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:29:49.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.111+0000 7fdd9abb0740 20 client.37994 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:49.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.111+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x555d42ef4c80 0x555d42ef70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.111+0000 7fdd9abb0740 4 client.37994 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:29:49.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.111+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1530810576 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d42e82490 con 0x555d42ef4c80 2024-04-17T07:29:49.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.111+0000 7fdd95400640 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x555d42ef4c80 0x555d42ef70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.115+0000 7fdd95400640 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x555d42ef4c80 0x555d42ef70a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fffbe60fe58e8956 server_cookie=3008d24ee62bec68 in_seq=0 out_seq=0 2024-04-17T07:29:49.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.115+0000 7fdd76a00640 10 client.37994 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:29:49.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.116+0000 7fdd76a00640 1 -- 172.21.15.86:0/1530810576 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d42e82490 con 0x555d42ef4c80 2024-04-17T07:29:49.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.116+0000 7fdd76a00640 10 client.37994 handle_command_reply: tid=0 2024-04-17T07:29:49.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.116+0000 7fdd9abb0740 1 client.37994 shutdown 2024-04-17T07:29:49.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.117+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1530810576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e7dcc0 msgr2=0x555d42e88af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.117+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e7dcc0 0x555d42e88af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdd88004af0 tx=0x7fdd880021d0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.119+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1530810576 shutdown_connections 2024-04-17T07:29:49.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.119+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x555d42ef4c80 0x555d42ef70a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.119+0000 7fdd76a00640 0 client.37994 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:29:49.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.119+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555d42e7dcc0 0x555d42e88af0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.119+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555d42e78b50 0x555d42e87ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.119+0000 7fdd9abb0740 1 --2- 172.21.15.86:0/1530810576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555d42e78160 0x555d42e875a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.119+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1530810576 >> 172.21.15.86:0/1530810576 conn(0x555d42e736d0 msgr2=0x555d42e771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.119+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1530810576 shutdown_connections 2024-04-17T07:29:49.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.119+0000 7fdd9abb0740 1 -- 172.21.15.86:0/1530810576 wait complete. 2024-04-17T07:29:49.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.119+0000 7fdd9abb0740 20 client.37994 trim_cache size 0 max 16384 2024-04-17T07:29:49.120 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:29:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.122+0000 7fdd96800640 1 -- 172.21.15.86:0/831774664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd780a3c80 msgr2=0x7fdd780a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.122+0000 7fdd96800640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd780a3c80 0x7fdd780a6140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x555d42e78f50 tx=0x7fdd840093d0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.122+0000 7fdd96800640 1 -- 172.21.15.86:0/831774664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd90103060 msgr2=0x7fdd9019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.122+0000 7fdd96800640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd90103060 0x7fdd9019eb50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00ea60 tx=0x7fdd7c00ef20 comp rx=0 tx=0).stop 2024-04-17T07:29:49.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.123+0000 7fdd96800640 1 -- 172.21.15.86:0/831774664 shutdown_connections 2024-04-17T07:29:49.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.123+0000 7fdd96800640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdd780a3c80 0x7fdd780a6140 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.123+0000 7fdd96800640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdd90105a10 0x7fdd901a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.123+0000 7fdd96800640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd90103060 0x7fdd9019eb50 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.123+0000 7fdd96800640 1 --2- 172.21.15.86:0/831774664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd901006f0 0x7fdd9019e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.123+0000 7fdd96800640 1 -- 172.21.15.86:0/831774664 >> 172.21.15.86:0/831774664 conn(0x7fdd900fc310 msgr2=0x7fdd90101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.123+0000 7fdd96800640 1 -- 172.21.15.86:0/831774664 shutdown_connections 2024-04-17T07:29:49.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.123+0000 7fdd96800640 1 -- 172.21.15.86:0/831774664 wait complete. 2024-04-17T07:29:49.159 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:29:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 625.0 s, 525.8 tps, lat 189.291 ms stddev 67.584 2024-04-17T07:29:49.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:49 smithi098 ceph-mon[35418]: pgmap v903: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 646 op/s 2024-04-17T07:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.514+0000 7fd92be00640 1 -- 172.21.15.86:0/3726268195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c1040a0 msgr2=0x7fd92c1044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.514+0000 7fd92be00640 1 --2- 172.21.15.86:0/3726268195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c1040a0 0x7fd92c1044c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd924009ca0 tx=0x7fd92402c120 comp rx=0 tx=0).stop 2024-04-17T07:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.515+0000 7fd92be00640 1 -- 172.21.15.86:0/3726268195 shutdown_connections 2024-04-17T07:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.515+0000 7fd92be00640 1 --2- 172.21.15.86:0/3726268195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd92c105c50 0x7fd92c10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.515+0000 7fd92be00640 1 --2- 172.21.15.86:0/3726268195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd92c105290 0x7fd92c105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.515+0000 7fd92be00640 1 --2- 172.21.15.86:0/3726268195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c1040a0 0x7fd92c1044c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.515+0000 7fd92be00640 1 -- 172.21.15.86:0/3726268195 >> 172.21.15.86:0/3726268195 conn(0x7fd92c0ff610 msgr2=0x7fd92c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.515+0000 7fd92be00640 1 -- 172.21.15.86:0/3726268195 shutdown_connections 2024-04-17T07:29:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.516+0000 7fd92be00640 1 -- 172.21.15.86:0/3726268195 wait complete. 2024-04-17T07:29:49.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.518+0000 7fd92be00640 1 Processor -- start 2024-04-17T07:29:49.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.518+0000 7fd92be00640 1 -- start start 2024-04-17T07:29:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.519+0000 7fd92be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd92c1040a0 0x7fd92c195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.519+0000 7fd92be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c105290 0x7fd92c1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.519+0000 7fd92be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd92c105c50 0x7fd92c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.519+0000 7fd92be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c10fba0 con 0x7fd92c105c50 2024-04-17T07:29:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.519+0000 7fd92be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c10fa20 con 0x7fd92c105290 2024-04-17T07:29:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.519+0000 7fd92be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c10fd20 con 0x7fd92c1040a0 2024-04-17T07:29:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.519+0000 7fd92aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd92c1040a0 0x7fd92c195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.519+0000 7fd92aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd92c1040a0 0x7fd92c195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49802/0 (socket says 172.21.15.86:49802) 2024-04-17T07:29:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.519+0000 7fd92aa00640 1 -- 172.21.15.86:0/1672551849 learned_addr learned my addr 172.21.15.86:0/1672551849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.519+0000 7fd92a000640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c105290 0x7fd92c1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.520+0000 7fd92aa00640 1 -- 172.21.15.86:0/1672551849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c105290 msgr2=0x7fd92c1964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.520+0000 7fd92aa00640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c105290 0x7fd92c1964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.521+0000 7fd92aa00640 1 -- 172.21.15.86:0/1672551849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd92c105c50 msgr2=0x7fd92c19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.521+0000 7fd92aa00640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd92c105c50 0x7fd92c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.521+0000 7fd92aa00640 1 -- 172.21.15.86:0/1672551849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92c1aeb50 con 0x7fd92c1040a0 2024-04-17T07:29:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.521+0000 7fd92aa00640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd92c1040a0 0x7fd92c195fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd924032540 tx=0x7fd924034cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.522+0000 7fd923400640 1 -- 172.21.15.86:0/1672551849 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924041020 con 0x7fd92c1040a0 2024-04-17T07:29:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.522+0000 7fd923400640 1 -- 172.21.15.86:0/1672551849 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9240327e0 con 0x7fd92c1040a0 2024-04-17T07:29:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.522+0000 7fd923400640 1 -- 172.21.15.86:0/1672551849 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924039040 con 0x7fd92c1040a0 2024-04-17T07:29:49.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.522+0000 7fd92be00640 1 -- 172.21.15.86:0/1672551849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92c1aede0 con 0x7fd92c1040a0 2024-04-17T07:29:49.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.522+0000 7fd92be00640 1 -- 172.21.15.86:0/1672551849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92c1af1c0 con 0x7fd92c1040a0 2024-04-17T07:29:49.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:49 smithi103 ceph-mon[35495]: pgmap v903: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 646 op/s 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.526+0000 7fd934874740 1 Processor -- start 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.527+0000 7fd934874740 1 -- start start 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.527+0000 7fd934874740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 0x5597e2a9b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.527+0000 7fd934874740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2a9bb50 0x5597e2aa0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.527+0000 7fd934874740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2aa0cc0 0x5597e2aa3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.527+0000 7fd934874740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597e2a01020 con 0x5597e2a9b160 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.527+0000 7fd934874740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5597e29a7460 con 0x5597e2a9bb50 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.527+0000 7fd934874740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5597e29fcfe0 con 0x5597e2aa0cc0 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.528+0000 7fd92a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2aa0cc0 0x5597e2aa3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.528+0000 7fd92a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2aa0cc0 0x5597e2aa3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49804/0 (socket says 172.21.15.86:49804) 2024-04-17T07:29:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.528+0000 7fd92a000640 1 -- 172.21.15.86:0/1338578874 learned_addr learned my addr 172.21.15.86:0/1338578874 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.528+0000 7fd92b400640 1 --2- 172.21.15.86:0/1338578874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 0x5597e2a9b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.529+0000 7fd92a000640 1 -- 172.21.15.86:0/1338578874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2a9bb50 msgr2=0x5597e2aa0650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:29:49.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.529+0000 7fd92a000640 1 --2- 172.21.15.86:0/1338578874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2a9bb50 0x5597e2aa0650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.529+0000 7fd92a000640 1 -- 172.21.15.86:0/1338578874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 msgr2=0x5597e2a9b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.529+0000 7fd92a000640 1 --2- 172.21.15.86:0/1338578874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 0x5597e2a9b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.529+0000 7fd92a000640 1 -- 172.21.15.86:0/1338578874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e29af040 con 0x5597e2aa0cc0 2024-04-17T07:29:49.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.530+0000 7fd92a000640 1 --2- 172.21.15.86:0/1338578874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2aa0cc0 0x5597e2aa3150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd91800bc80 tx=0x7fd91800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=899505a686fa2e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.531+0000 7fd917e00640 1 -- 172.21.15.86:0/1338578874 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918015070 con 0x5597e2aa0cc0 2024-04-17T07:29:49.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.531+0000 7fd917e00640 1 -- 172.21.15.86:0/1338578874 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91801a910 con 0x5597e2aa0cc0 2024-04-17T07:29:49.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.531+0000 7fd917e00640 1 -- 172.21.15.86:0/1338578874 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9180118b0 con 0x5597e2aa0cc0 2024-04-17T07:29:49.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.531+0000 7fd923400640 1 -- 172.21.15.86:0/1672551849 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fd924044020 con 0x7fd92c1040a0 2024-04-17T07:29:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.533+0000 7fd923400640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd9100a3cb0 0x7fd9100a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.533+0000 7fd934874740 1 -- 172.21.15.86:0/1338578874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2aa0cc0 msgr2=0x5597e2aa3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.533+0000 7fd934874740 1 --2- 172.21.15.86:0/1338578874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2aa0cc0 0x5597e2aa3150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd91800bc80 tx=0x7fd91800f730 comp rx=0 tx=0).stop 2024-04-17T07:29:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.533+0000 7fd923400640 1 -- 172.21.15.86:0/1672551849 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd9240f2af0 con 0x7fd92c1040a0 2024-04-17T07:29:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.534+0000 7fd934874740 1 -- 172.21.15.86:0/1338578874 shutdown_connections 2024-04-17T07:29:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.534+0000 7fd934874740 1 --2- 172.21.15.86:0/1338578874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2aa0cc0 0x5597e2aa3150 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.534+0000 7fd934874740 1 --2- 172.21.15.86:0/1338578874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2a9bb50 0x5597e2aa0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.534+0000 7fd934874740 1 --2- 172.21.15.86:0/1338578874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 0x5597e2a9b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.534+0000 7fd934874740 1 -- 172.21.15.86:0/1338578874 >> 172.21.15.86:0/1338578874 conn(0x5597e2a966d0 msgr2=0x5597e2a96ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.535+0000 7fd934874740 1 -- 172.21.15.86:0/1338578874 shutdown_connections 2024-04-17T07:29:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.535+0000 7fd92b400640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd9100a3cb0 0x7fd9100a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.535+0000 7fd934874740 1 -- 172.21.15.86:0/1338578874 wait complete. 2024-04-17T07:29:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.536+0000 7fd934874740 1 -- start start 2024-04-17T07:29:49.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.536+0000 7fd934874740 1 Processor -- start 2024-04-17T07:29:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.536+0000 7fd92b400640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd9100a3cb0 0x7fd9100a6170 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x5597e2a9c3b0 tx=0x7fd91c009390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.537+0000 7fd934874740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.537+0000 7fd934874740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2a9bb50 0x5597e2aa5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.537+0000 7fd934874740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2aa0cc0 0x5597e2aaccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.537+0000 7fd934874740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5597e2aa3c70 con 0x5597e2aa0cc0 2024-04-17T07:29:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.537+0000 7fd934874740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5597e29ba830 con 0x5597e2a9bb50 2024-04-17T07:29:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.537+0000 7fd934874740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5597e2aa3df0 con 0x5597e2a9b160 2024-04-17T07:29:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.537+0000 7fd934874740 10 client.0 _ll_register_callbacks cb 0x5597e2988fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.537+0000 7fd92aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.538+0000 7fd92aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49810/0 (socket says 172.21.15.86:49810) 2024-04-17T07:29:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.538+0000 7fd92aa00640 1 -- 172.21.15.86:0/2325719494 learned_addr learned my addr 172.21.15.86:0/2325719494 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.538+0000 7fd92b400640 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2aa0cc0 0x5597e2aaccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.538+0000 7fd92a000640 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2a9bb50 0x5597e2aa5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.538+0000 7fd92aa00640 1 -- 172.21.15.86:0/2325719494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2a9bb50 msgr2=0x5597e2aa5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.538+0000 7fd92aa00640 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2a9bb50 0x5597e2aa5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.539+0000 7fd92aa00640 1 -- 172.21.15.86:0/2325719494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2aa0cc0 msgr2=0x5597e2aaccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.539+0000 7fd92aa00640 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2aa0cc0 0x5597e2aaccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.539+0000 7fd92aa00640 1 -- 172.21.15.86:0/2325719494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e29be920 con 0x5597e2a9b160 2024-04-17T07:29:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.539+0000 7fd92aa00640 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa5730 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd924006010 tx=0x7fd9240be350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6cac22f3805bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.539+0000 7fd914c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.540+0000 7fd914c00640 1 -- 172.21.15.86:0/2325719494 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9240098b0 con 0x5597e2a9b160 2024-04-17T07:29:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.540+0000 7fd914c00640 1 -- 172.21.15.86:0/2325719494 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd9240bec50 con 0x5597e2a9b160 2024-04-17T07:29:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.540+0000 7fd934874740 1 -- 172.21.15.86:0/2325719494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd91801a910 con 0x5597e2a9b160 2024-04-17T07:29:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.541+0000 7fd914c00640 1 -- 172.21.15.86:0/2325719494 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9240be9f0 con 0x5597e2a9b160 2024-04-17T07:29:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.541+0000 7fd934874740 10 client.28042 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:49.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.541+0000 7fd934874740 1 -- 172.21.15.86:0/2325719494 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597e298c7f0 con 0x5597e2a9b160 2024-04-17T07:29:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.541+0000 7fd914c00640 1 -- 172.21.15.86:0/2325719494 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91801a910 con 0x5597e2a9b160 2024-04-17T07:29:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.541+0000 7fd914c00640 1 -- 172.21.15.86:0/2325719494 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fd92403c030 con 0x5597e2a9b160 2024-04-17T07:29:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.542+0000 7fd934874740 10 client.28042 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.542+0000 7fd934874740 10 client.28042 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:29:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.542+0000 7fd934874740 20 client.28042 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.542+0000 7fd934874740 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5597e2ab13c0 0x5597e2ab3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.542+0000 7fd934874740 4 client.28042 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.542+0000 7fd934874740 1 -- 172.21.15.86:0/2325719494 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597e2a0ea40 con 0x5597e2ab13c0 2024-04-17T07:29:49.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.543+0000 7fd92a000640 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5597e2ab13c0 0x5597e2ab3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.544+0000 7fd92a000640 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5597e2ab13c0 0x5597e2ab3810 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8986e7d049a0cabc server_cookie=12fbd98c5d21a73 in_seq=0 out_seq=0 2024-04-17T07:29:49.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.544+0000 7fd914c00640 10 client.28042 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:29:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.545+0000 7fd914c00640 1 -- 172.21.15.86:0/2325719494 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597e2a0ea40 con 0x5597e2ab13c0 2024-04-17T07:29:49.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.545+0000 7fd914c00640 10 client.28042 handle_command_reply: tid=0 2024-04-17T07:29:49.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.546+0000 7fd934874740 1 client.28042 shutdown 2024-04-17T07:29:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.546+0000 7fd934874740 1 -- 172.21.15.86:0/2325719494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 msgr2=0x5597e2aa5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.546+0000 7fd934874740 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa5730 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd924006010 tx=0x7fd9240be350 comp rx=0 tx=0).stop 2024-04-17T07:29:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.547+0000 7fd934874740 1 -- 172.21.15.86:0/2325719494 shutdown_connections 2024-04-17T07:29:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.547+0000 7fd934874740 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5597e2ab13c0 0x5597e2ab3810 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.547+0000 7fd934874740 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2aa0cc0 0x5597e2aaccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.547+0000 7fd914c00640 0 client.28042 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:29:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.547+0000 7fd934874740 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2a9bb50 0x5597e2aa5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.547+0000 7fd934874740 1 --2- 172.21.15.86:0/2325719494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa5730 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.547+0000 7fd934874740 1 -- 172.21.15.86:0/2325719494 >> 172.21.15.86:0/2325719494 conn(0x5597e2a966d0 msgr2=0x5597e2a95930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.547+0000 7fd934874740 1 -- 172.21.15.86:0/2325719494 shutdown_connections 2024-04-17T07:29:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.547+0000 7fd934874740 1 -- 172.21.15.86:0/2325719494 wait complete. 2024-04-17T07:29:49.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.547+0000 7fd934874740 20 client.28042 trim_cache size 0 max 16384 2024-04-17T07:29:49.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.570+0000 7fd934874740 1 Processor -- start 2024-04-17T07:29:49.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.570+0000 7fd934874740 1 -- start start 2024-04-17T07:29:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.570+0000 7fd934874740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 0x5597e2aa5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.570+0000 7fd934874740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9bb50 0x5597e2aacc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.570+0000 7fd934874740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 0x5597e2a950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.570+0000 7fd934874740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c1b9650 con 0x5597e2a9b160 2024-04-17T07:29:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.570+0000 7fd934874740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c1bb190 con 0x5597e2aa0cc0 2024-04-17T07:29:49.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.570+0000 7fd934874740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c1b9d20 con 0x5597e2a9bb50 2024-04-17T07:29:49.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.571+0000 7fd92aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 0x5597e2aa5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.571+0000 7fd92b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 0x5597e2a950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.572+0000 7fd92b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 0x5597e2a950a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45484/0 (socket says 172.21.15.86:45484) 2024-04-17T07:29:49.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.572+0000 7fd92b400640 1 -- 172.21.15.86:0/1190343892 learned_addr learned my addr 172.21.15.86:0/1190343892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.572+0000 7fd92a000640 1 --2- 172.21.15.86:0/1190343892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9bb50 0x5597e2aacc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.572+0000 7fd92aa00640 1 -- 172.21.15.86:0/1190343892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9bb50 msgr2=0x5597e2aacc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.572+0000 7fd92aa00640 1 --2- 172.21.15.86:0/1190343892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9bb50 0x5597e2aacc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.572+0000 7fd92aa00640 1 -- 172.21.15.86:0/1190343892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 msgr2=0x5597e2a950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.572+0000 7fd92aa00640 1 --2- 172.21.15.86:0/1190343892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 0x5597e2a950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.573+0000 7fd92aa00640 1 -- 172.21.15.86:0/1190343892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e27133f0 con 0x5597e2a9b160 2024-04-17T07:29:49.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.573+0000 7fd92a000640 1 --2- 172.21.15.86:0/1190343892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9bb50 0x5597e2aacc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:49.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.573+0000 7fd92aa00640 1 --2- 172.21.15.86:0/1190343892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 0x5597e2aa5a50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd9240bb370 tx=0x7fd9240be480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420f4acc6bb34d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.574+0000 7fd917400640 1 -- 172.21.15.86:0/1190343892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9240d9070 con 0x5597e2a9b160 2024-04-17T07:29:49.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.574+0000 7fd917400640 1 -- 172.21.15.86:0/1190343892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9240ddc50 con 0x5597e2a9b160 2024-04-17T07:29:49.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.574+0000 7fd917400640 1 -- 172.21.15.86:0/1190343892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9240d7070 con 0x5597e2a9b160 2024-04-17T07:29:49.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.575+0000 7fd934874740 1 -- 172.21.15.86:0/1190343892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 msgr2=0x5597e2aa5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.575+0000 7fd934874740 1 --2- 172.21.15.86:0/1190343892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 0x5597e2aa5a50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd9240bb370 tx=0x7fd9240be480 comp rx=0 tx=0).stop 2024-04-17T07:29:49.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.575+0000 7fd934874740 1 -- 172.21.15.86:0/1190343892 shutdown_connections 2024-04-17T07:29:49.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.575+0000 7fd934874740 1 --2- 172.21.15.86:0/1190343892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 0x5597e2a950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.575+0000 7fd934874740 1 --2- 172.21.15.86:0/1190343892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9bb50 0x5597e2aacc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.575+0000 7fd934874740 1 --2- 172.21.15.86:0/1190343892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9b160 0x5597e2aa5a50 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.575+0000 7fd934874740 1 -- 172.21.15.86:0/1190343892 >> 172.21.15.86:0/1190343892 conn(0x5597e2a966d0 msgr2=0x5597e2aa5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.575+0000 7fd934874740 1 -- 172.21.15.86:0/1190343892 shutdown_connections 2024-04-17T07:29:49.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.576+0000 7fd934874740 1 -- 172.21.15.86:0/1190343892 wait complete. 2024-04-17T07:29:49.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.576+0000 7fd934874740 1 -- start start 2024-04-17T07:29:49.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.577+0000 7fd934874740 1 Processor -- start 2024-04-17T07:29:49.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.577+0000 7fd934874740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.577+0000 7fd934874740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9bb50 0x5597e2aaa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.577+0000 7fd934874740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 0x5597e2b15980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.577+0000 7fd934874740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c1baac0 con 0x5597e2a9bb50 2024-04-17T07:29:49.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.577+0000 7fd934874740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c1ba3f0 con 0x5597e2aa0cc0 2024-04-17T07:29:49.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.577+0000 7fd934874740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c1b7620 con 0x5597e2a9b160 2024-04-17T07:29:49.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.577+0000 7fd934874740 10 client.0 _ll_register_callbacks cb 0x7fd9240bac90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:49.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.578+0000 7fd92aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.578+0000 7fd92a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9bb50 0x5597e2aaa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.578+0000 7fd92aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49822/0 (socket says 172.21.15.86:49822) 2024-04-17T07:29:49.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.578+0000 7fd92aa00640 1 -- 172.21.15.86:0/2752954883 learned_addr learned my addr 172.21.15.86:0/2752954883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.578+0000 7fd92b400640 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 0x5597e2b15980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.578+0000 7fd92a000640 1 -- 172.21.15.86:0/2752954883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 msgr2=0x5597e2aa9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.578+0000 7fd92a000640 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.578+0000 7fd92a000640 1 -- 172.21.15.86:0/2752954883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 msgr2=0x5597e2b15980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:49.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.578+0000 7fd92a000640 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 0x5597e2b15980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.578+0000 7fd92a000640 1 -- 172.21.15.86:0/2752954883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e27dd7d0 con 0x5597e2a9bb50 2024-04-17T07:29:49.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.579+0000 7fd92a000640 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9bb50 0x5597e2aaa190 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd918005d00 tx=0x7fd91800b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2a2573c95a96b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.579+0000 7fd914c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:49.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.579+0000 7fd914c00640 1 -- 172.21.15.86:0/2752954883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918015040 con 0x5597e2a9bb50 2024-04-17T07:29:49.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.579+0000 7fd914c00640 1 -- 172.21.15.86:0/2752954883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd918005310 con 0x5597e2a9bb50 2024-04-17T07:29:49.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.579+0000 7fd934874740 1 -- 172.21.15.86:0/2752954883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9240ddc50 con 0x5597e2a9bb50 2024-04-17T07:29:49.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.580+0000 7fd914c00640 1 -- 172.21.15.86:0/2752954883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd918039050 con 0x5597e2a9bb50 2024-04-17T07:29:49.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.580+0000 7fd934874740 10 client.19047 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:49.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.580+0000 7fd934874740 1 -- 172.21.15.86:0/2752954883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597e2b16fc0 con 0x5597e2a9bb50 2024-04-17T07:29:49.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:49 smithi086 ceph-mon[31733]: pgmap v903: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 646 op/s 2024-04-17T07:29:49.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.581+0000 7fd914c00640 1 -- 172.21.15.86:0/2752954883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9240ddc50 con 0x5597e2a9bb50 2024-04-17T07:29:49.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.581+0000 7fd914c00640 1 -- 172.21.15.86:0/2752954883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fd91802a5c0 con 0x5597e2a9bb50 2024-04-17T07:29:49.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.581+0000 7fd934874740 10 client.19047 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:49.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.581+0000 7fd934874740 10 client.19047 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:29:49.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.581+0000 7fd934874740 20 client.19047 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:49.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.581+0000 7fd934874740 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5597e2b17610 0x5597e2b19a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.581+0000 7fd934874740 4 client.19047 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:29:49.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.581+0000 7fd934874740 1 -- 172.21.15.86:0/2752954883 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597e2a088b0 con 0x5597e2b17610 2024-04-17T07:29:49.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.582+0000 7fd92aa00640 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5597e2b17610 0x5597e2b19a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.583+0000 7fd92aa00640 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5597e2b17610 0x5597e2b19a30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8f0bee6274af39e server_cookie=e9b1795dde6bf105 in_seq=0 out_seq=0 2024-04-17T07:29:49.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.583+0000 7fd914c00640 10 client.19047 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:29:49.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd914c00640 1 -- 172.21.15.86:0/2752954883 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597e2a088b0 con 0x5597e2b17610 2024-04-17T07:29:49.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd914c00640 10 client.19047 handle_command_reply: tid=0 2024-04-17T07:29:49.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd934874740 1 client.19047 shutdown 2024-04-17T07:29:49.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd934874740 1 -- 172.21.15.86:0/2752954883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9bb50 msgr2=0x5597e2aaa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd934874740 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9bb50 0x5597e2aaa190 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd918005d00 tx=0x7fd91800b430 comp rx=0 tx=0).stop 2024-04-17T07:29:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd934874740 1 -- 172.21.15.86:0/2752954883 shutdown_connections 2024-04-17T07:29:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd934874740 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5597e2b17610 0x5597e2b19a30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd934874740 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5597e2aa0cc0 0x5597e2b15980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd934874740 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5597e2a9bb50 0x5597e2aaa190 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd934874740 1 --2- 172.21.15.86:0/2752954883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5597e2a9b160 0x5597e2aa9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd914c00640 0 client.19047 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:29:49.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.584+0000 7fd934874740 1 -- 172.21.15.86:0/2752954883 >> 172.21.15.86:0/2752954883 conn(0x5597e2a966d0 msgr2=0x5597e2a99930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.585+0000 7fd934874740 1 -- 172.21.15.86:0/2752954883 shutdown_connections 2024-04-17T07:29:49.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.585+0000 7fd934874740 1 -- 172.21.15.86:0/2752954883 wait complete. 2024-04-17T07:29:49.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.585+0000 7fd934874740 20 client.19047 trim_cache size 0 max 16384 2024-04-17T07:29:49.586 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:29:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.586+0000 7fd92be00640 1 -- 172.21.15.86:0/1672551849 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd9100a3cb0 msgr2=0x7fd9100a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.586+0000 7fd92be00640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd9100a3cb0 0x7fd9100a6170 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x5597e2a9c3b0 tx=0x7fd91c009390 comp rx=0 tx=0).stop 2024-04-17T07:29:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.587+0000 7fd92be00640 1 -- 172.21.15.86:0/1672551849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd92c1040a0 msgr2=0x7fd92c195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.587+0000 7fd92be00640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd92c1040a0 0x7fd92c195fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd924032540 tx=0x7fd924034cd0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.587+0000 7fd92be00640 1 -- 172.21.15.86:0/1672551849 shutdown_connections 2024-04-17T07:29:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.587+0000 7fd92be00640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd9100a3cb0 0x7fd9100a6170 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.587+0000 7fd92be00640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd92c105c50 0x7fd92c19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.587+0000 7fd92be00640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c105290 0x7fd92c1964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.587+0000 7fd92be00640 1 --2- 172.21.15.86:0/1672551849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd92c1040a0 0x7fd92c195fb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.587+0000 7fd92be00640 1 -- 172.21.15.86:0/1672551849 >> 172.21.15.86:0/1672551849 conn(0x7fd92c0ff610 msgr2=0x7fd92c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.587+0000 7fd92be00640 1 -- 172.21.15.86:0/1672551849 shutdown_connections 2024-04-17T07:29:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.587+0000 7fd92be00640 1 -- 172.21.15.86:0/1672551849 wait complete. 2024-04-17T07:29:49.619 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:29:49.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.979+0000 7fd1a7e00640 1 -- 172.21.15.86:0/2791404954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a0099910 msgr2=0x7fd1a0099d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.979+0000 7fd1a7e00640 1 --2- 172.21.15.86:0/2791404954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a0099910 0x7fd1a0099d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8067c60 tx=0x7fd1a80983d0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.982+0000 7fd1a7e00640 1 -- 172.21.15.86:0/2791404954 shutdown_connections 2024-04-17T07:29:49.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.982+0000 7fd1a7e00640 1 --2- 172.21.15.86:0/2791404954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a0099910 0x7fd1a0099d90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.982+0000 7fd1a7e00640 1 --2- 172.21.15.86:0/2791404954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1a0096f60 0x7fd1a00993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.983+0000 7fd1a7e00640 1 --2- 172.21.15.86:0/2791404954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1a00945f0 0x7fd1a0096a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.983+0000 7fd1a7e00640 1 -- 172.21.15.86:0/2791404954 >> 172.21.15.86:0/2791404954 conn(0x7fd1a00901d0 msgr2=0x7fd1a0092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:49.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.983+0000 7fd1a7e00640 1 -- 172.21.15.86:0/2791404954 shutdown_connections 2024-04-17T07:29:49.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.984+0000 7fd1a7e00640 1 -- 172.21.15.86:0/2791404954 wait complete. 2024-04-17T07:29:49.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.986+0000 7fd1a7e00640 1 Processor -- start 2024-04-17T07:29:49.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.987+0000 7fd1a7e00640 1 -- start start 2024-04-17T07:29:49.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.987+0000 7fd1a7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a00945f0 0x7fd1a009fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.987+0000 7fd1a7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1a0096f60 0x7fd1a00a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.987+0000 7fd1a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1a0099910 0x7fd1a0142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.988+0000 7fd1a7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a00a7e50 con 0x7fd1a0099910 2024-04-17T07:29:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.988+0000 7fd1a7e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a00a7cd0 con 0x7fd1a00945f0 2024-04-17T07:29:49.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.988+0000 7fd1a7e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a00a7fd0 con 0x7fd1a0096f60 2024-04-17T07:29:49.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.988+0000 7fd1a6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a00945f0 0x7fd1a009fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.988+0000 7fd1a6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a00945f0 0x7fd1a009fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45504/0 (socket says 172.21.15.86:45504) 2024-04-17T07:29:49.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.988+0000 7fd1a6a00640 1 -- 172.21.15.86:0/3577828167 learned_addr learned my addr 172.21.15.86:0/3577828167 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:49.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.988+0000 7fd1a6000640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1a0096f60 0x7fd1a00a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.989+0000 7fd1a6a00640 1 -- 172.21.15.86:0/3577828167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1a0096f60 msgr2=0x7fd1a00a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.989+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1a0096f60 0x7fd1a00a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.989+0000 7fd1a7400640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1a0099910 0x7fd1a0142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.989+0000 7fd1a6a00640 1 -- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1a0099910 msgr2=0x7fd1a0142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.989+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1a0099910 0x7fd1a0142560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.989+0000 7fd1a6a00640 1 -- 172.21.15.86:0/3577828167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a0142aa0 con 0x7fd1a00945f0 2024-04-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.989+0000 7fd1a7400640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1a0099910 0x7fd1a0142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.989+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a00945f0 0x7fd1a009fc10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd19400b4a0 tx=0x7fd19400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.990+0000 7fd19f400640 1 -- 172.21.15.86:0/3577828167 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194014070 con 0x7fd1a00945f0 2024-04-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.990+0000 7fd1a7e00640 1 -- 172.21.15.86:0/3577828167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a0142c70 con 0x7fd1a00945f0 2024-04-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.990+0000 7fd1a7e00640 1 -- 172.21.15.86:0/3577828167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a01431b0 con 0x7fd1a00945f0 2024-04-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.990+0000 7fd19f400640 1 -- 172.21.15.86:0/3577828167 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd194010ac0 con 0x7fd1a00945f0 2024-04-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.990+0000 7fd19f400640 1 -- 172.21.15.86:0/3577828167 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194017c30 con 0x7fd1a00945f0 2024-04-17T07:29:49.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.998+0000 7fd1b0627740 1 Processor -- start 2024-04-17T07:29:49.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.999+0000 7fd19f400640 1 -- 172.21.15.86:0/3577828167 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fd194017070 con 0x7fd1a00945f0 2024-04-17T07:29:49.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.999+0000 7fd1b0627740 1 -- start start 2024-04-17T07:29:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.999+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425836580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.999+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836b50 0x55942583b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.999+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55942583bcc0 0x55942583e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.999+0000 7fd1b0627740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55942579c020 con 0x559425836160 2024-04-17T07:29:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.999+0000 7fd1b0627740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559425742460 con 0x559425836b50 2024-04-17T07:29:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:49.999+0000 7fd1b0627740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559425797fe0 con 0x55942583bcc0 2024-04-17T07:29:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.000+0000 7fd1a6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55942583bcc0 0x55942583e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.000+0000 7fd1a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425836580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.000+0000 7fd1a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425836580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47450/0 (socket says 172.21.15.86:47450) 2024-04-17T07:29:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.000+0000 7fd1a7400640 1 -- 172.21.15.86:0/2971164461 learned_addr learned my addr 172.21.15.86:0/2971164461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:50.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.001+0000 7fd1a6000640 1 -- 172.21.15.86:0/2971164461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836b50 msgr2=0x55942583b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.001+0000 7fd1a6000640 1 --2- 172.21.15.86:0/2971164461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836b50 0x55942583b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.001+0000 7fd1a6000640 1 -- 172.21.15.86:0/2971164461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 msgr2=0x559425836580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.001+0000 7fd1a6000640 1 --2- 172.21.15.86:0/2971164461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425836580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.001+0000 7fd1a6000640 1 -- 172.21.15.86:0/2971164461 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55942574a040 con 0x55942583bcc0 2024-04-17T07:29:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.001+0000 7fd1a6000640 1 --2- 172.21.15.86:0/2971164461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55942583bcc0 0x55942583e150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd198009bf0 tx=0x7fd198011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f467a0c7adb2ca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.001+0000 7fd193e00640 1 -- 172.21.15.86:0/2971164461 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19801e070 con 0x55942583bcc0 2024-04-17T07:29:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.001+0000 7fd193e00640 1 -- 172.21.15.86:0/2971164461 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd198009da0 con 0x55942583bcc0 2024-04-17T07:29:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.002+0000 7fd193e00640 1 -- 172.21.15.86:0/2971164461 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1980187d0 con 0x55942583bcc0 2024-04-17T07:29:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.002+0000 7fd19f400640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd18c0a3c30 0x7fd18c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.002+0000 7fd1b0627740 1 -- 172.21.15.86:0/2971164461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55942583bcc0 msgr2=0x55942583e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.002+0000 7fd1b0627740 1 --2- 172.21.15.86:0/2971164461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55942583bcc0 0x55942583e150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd198009bf0 tx=0x7fd198011810 comp rx=0 tx=0).stop 2024-04-17T07:29:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.003+0000 7fd19f400640 1 -- 172.21.15.86:0/3577828167 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd1940cfdd0 con 0x7fd1a00945f0 2024-04-17T07:29:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.003+0000 7fd1a7400640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd18c0a3c30 0x7fd18c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.003+0000 7fd1b0627740 1 -- 172.21.15.86:0/2971164461 shutdown_connections 2024-04-17T07:29:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.003+0000 7fd1b0627740 1 --2- 172.21.15.86:0/2971164461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55942583bcc0 0x55942583e150 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.003+0000 7fd1b0627740 1 --2- 172.21.15.86:0/2971164461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836b50 0x55942583b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.003+0000 7fd1b0627740 1 --2- 172.21.15.86:0/2971164461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425836580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.003+0000 7fd1b0627740 1 -- 172.21.15.86:0/2971164461 >> 172.21.15.86:0/2971164461 conn(0x5594258316d0 msgr2=0x559425831ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.003+0000 7fd1b0627740 1 -- 172.21.15.86:0/2971164461 shutdown_connections 2024-04-17T07:29:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.003+0000 7fd1b0627740 1 -- 172.21.15.86:0/2971164461 wait complete. 2024-04-17T07:29:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.004+0000 7fd1a7400640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd18c0a3c30 0x7fd18c0a60f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8067e70 tx=0x7fd1a8063410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.004+0000 7fd1b0627740 1 -- start start 2024-04-17T07:29:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.004+0000 7fd1b0627740 1 Processor -- start 2024-04-17T07:29:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.005+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425840720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.005+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836b50 0x559425840c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.005+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55942583bcc0 0x559425847ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.005+0000 7fd1b0627740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55942583ecc0 con 0x559425836160 2024-04-17T07:29:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.005+0000 7fd1b0627740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559425755830 con 0x559425836b50 2024-04-17T07:29:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.005+0000 7fd1b0627740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55942583ee40 con 0x55942583bcc0 2024-04-17T07:29:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.005+0000 7fd1b0627740 10 client.0 _ll_register_callbacks cb 0x559425723fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.006+0000 7fd1a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425840720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.006+0000 7fd1a6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425840720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47462/0 (socket says 172.21.15.86:47462) 2024-04-17T07:29:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.006+0000 7fd1a6a00640 1 -- 172.21.15.86:0/2871457496 learned_addr learned my addr 172.21.15.86:0/2871457496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.006+0000 7fd1a6000640 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836b50 0x559425840c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.006+0000 7fd1a6a00640 1 -- 172.21.15.86:0/2871457496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55942583bcc0 msgr2=0x559425847ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:29:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.007+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55942583bcc0 0x559425847ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.007+0000 7fd1a6a00640 1 -- 172.21.15.86:0/2871457496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836b50 msgr2=0x559425840c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.007+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836b50 0x559425840c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.007+0000 7fd1a6a00640 1 -- 172.21.15.86:0/2871457496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559425759920 con 0x559425836160 2024-04-17T07:29:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.008+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425840720 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd19400be60 tx=0x7fd19409c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c2156d5d13aca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.009+0000 7fd190c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.009+0000 7fd190c00640 1 -- 172.21.15.86:0/2871457496 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19401e050 con 0x559425836160 2024-04-17T07:29:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.009+0000 7fd190c00640 1 -- 172.21.15.86:0/2871457496 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd19409a030 con 0x559425836160 2024-04-17T07:29:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.009+0000 7fd1b0627740 1 -- 172.21.15.86:0/2871457496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd198009da0 con 0x559425836160 2024-04-17T07:29:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.010+0000 7fd190c00640 1 -- 172.21.15.86:0/2871457496 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1940a49a0 con 0x559425836160 2024-04-17T07:29:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.010+0000 7fd1b0627740 10 client.19053 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.010+0000 7fd1b0627740 1 -- 172.21.15.86:0/2871457496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594257277f0 con 0x559425836160 2024-04-17T07:29:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.011+0000 7fd190c00640 1 -- 172.21.15.86:0/2871457496 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd198009da0 con 0x559425836160 2024-04-17T07:29:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.011+0000 7fd190c00640 1 -- 172.21.15.86:0/2871457496 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fd1940a4510 con 0x559425836160 2024-04-17T07:29:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.011+0000 7fd1b0627740 10 client.19053 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.011+0000 7fd1b0627740 10 client.19053 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:29:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.011+0000 7fd1b0627740 20 client.19053 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.011+0000 7fd1b0627740 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55942584c1e0 0x55942584e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.011+0000 7fd1b0627740 4 client.19053 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.011+0000 7fd1b0627740 1 -- 172.21.15.86:0/2871457496 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594257a9a40 con 0x55942584c1e0 2024-04-17T07:29:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.012+0000 7fd1a6000640 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55942584c1e0 0x55942584e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.014+0000 7fd190c00640 10 client.19053 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:29:50.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.014+0000 7fd1a6000640 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55942584c1e0 0x55942584e660 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9419f6b918c45fef server_cookie=62cfb6ecdaa79376 in_seq=0 out_seq=0 2024-04-17T07:29:50.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.018+0000 7fd190c00640 1 -- 172.21.15.86:0/2871457496 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594257a9a40 con 0x55942584c1e0 2024-04-17T07:29:50.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.018+0000 7fd190c00640 10 client.19053 handle_command_reply: tid=0 2024-04-17T07:29:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.018+0000 7fd1b0627740 1 client.19053 shutdown 2024-04-17T07:29:50.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.021+0000 7fd1b0627740 1 -- 172.21.15.86:0/2871457496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 msgr2=0x559425840720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.021+0000 7fd1b0627740 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425840720 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd19400be60 tx=0x7fd19409c4f0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.021+0000 7fd1b0627740 1 -- 172.21.15.86:0/2871457496 shutdown_connections 2024-04-17T07:29:50.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.021+0000 7fd1b0627740 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55942583bcc0 0x559425847ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.021+0000 7fd1b0627740 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55942584c1e0 0x55942584e660 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.021+0000 7fd1b0627740 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836b50 0x559425840c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.021+0000 7fd1b0627740 1 --2- 172.21.15.86:0/2871457496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x559425840720 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.021+0000 7fd1b0627740 1 -- 172.21.15.86:0/2871457496 >> 172.21.15.86:0/2871457496 conn(0x5594258316d0 msgr2=0x559425830930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.021+0000 7fd190c00640 0 client.19053 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:29:50.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.021+0000 7fd1b0627740 1 -- 172.21.15.86:0/2871457496 shutdown_connections 2024-04-17T07:29:50.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.022+0000 7fd1b0627740 1 -- 172.21.15.86:0/2871457496 wait complete. 2024-04-17T07:29:50.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.022+0000 7fd1b0627740 20 client.19053 trim_cache size 0 max 16384 2024-04-17T07:29:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.057+0000 7fd1b0627740 1 Processor -- start 2024-04-17T07:29:50.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.058+0000 7fd1b0627740 1 -- start start 2024-04-17T07:29:50.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.059+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x5594258372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.059+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 0x559425847c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.060+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55942583bcc0 0x559425840a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.060+0000 7fd1b0627740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd18800fb30 con 0x559425836160 2024-04-17T07:29:50.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.060+0000 7fd1b0627740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd188011670 con 0x55942583bcc0 2024-04-17T07:29:50.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.060+0000 7fd1b0627740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd188010200 con 0x559425836b50 2024-04-17T07:29:50.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.061+0000 7fd1a6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 0x559425847c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.061+0000 7fd1a7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55942583bcc0 0x559425840a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.061+0000 7fd1a7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55942583bcc0 0x559425840a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45526/0 (socket says 172.21.15.86:45526) 2024-04-17T07:29:50.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.061+0000 7fd1a7400640 1 -- 172.21.15.86:0/836618309 learned_addr learned my addr 172.21.15.86:0/836618309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:50.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.062+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/836618309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x5594258372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.063+0000 7fd1a6000640 1 -- 172.21.15.86:0/836618309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55942583bcc0 msgr2=0x559425840a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.063+0000 7fd1a6000640 1 --2- 172.21.15.86:0/836618309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55942583bcc0 0x559425840a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.063+0000 7fd1a6000640 1 -- 172.21.15.86:0/836618309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 msgr2=0x5594258372c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.063+0000 7fd1a6000640 1 --2- 172.21.15.86:0/836618309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x5594258372c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.063+0000 7fd1a6000640 1 -- 172.21.15.86:0/836618309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594254ae3f0 con 0x559425836b50 2024-04-17T07:29:50.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.063+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/836618309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x5594258372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:50.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.064+0000 7fd1a6000640 1 --2- 172.21.15.86:0/836618309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 0x559425847c60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x559425841c00 tx=0x7fd198005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=973de493e3de3576 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.066+0000 7fd193400640 1 -- 172.21.15.86:0/836618309 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19801e040 con 0x559425836b50 2024-04-17T07:29:50.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.066+0000 7fd193400640 1 -- 172.21.15.86:0/836618309 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd198029070 con 0x559425836b50 2024-04-17T07:29:50.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.066+0000 7fd193400640 1 -- 172.21.15.86:0/836618309 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198019a20 con 0x559425836b50 2024-04-17T07:29:50.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.067+0000 7fd1b0627740 1 -- 172.21.15.86:0/836618309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 msgr2=0x559425847c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.067+0000 7fd1b0627740 1 --2- 172.21.15.86:0/836618309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 0x559425847c60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x559425841c00 tx=0x7fd198005f90 comp rx=0 tx=0).stop 2024-04-17T07:29:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.067+0000 7fd1b0627740 1 -- 172.21.15.86:0/836618309 shutdown_connections 2024-04-17T07:29:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.067+0000 7fd1b0627740 1 --2- 172.21.15.86:0/836618309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55942583bcc0 0x559425840a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.067+0000 7fd1b0627740 1 --2- 172.21.15.86:0/836618309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 0x559425847c60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.067+0000 7fd1b0627740 1 --2- 172.21.15.86:0/836618309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559425836160 0x5594258372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.068+0000 7fd1b0627740 1 -- 172.21.15.86:0/836618309 >> 172.21.15.86:0/836618309 conn(0x5594258316d0 msgr2=0x559425840330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.068+0000 7fd1b0627740 1 -- 172.21.15.86:0/836618309 shutdown_connections 2024-04-17T07:29:50.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.068+0000 7fd1b0627740 1 -- 172.21.15.86:0/836618309 wait complete. 2024-04-17T07:29:50.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.069+0000 7fd1b0627740 1 -- start start 2024-04-17T07:29:50.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.069+0000 7fd1b0627740 1 Processor -- start 2024-04-17T07:29:50.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.070+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836160 0x559425844fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.070+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 0x5594258454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.070+0000 7fd1b0627740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55942583bcc0 0x5594258464a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.070+0000 7fd1b0627740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd188010fa0 con 0x55942583bcc0 2024-04-17T07:29:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.070+0000 7fd1b0627740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1880108d0 con 0x559425836160 2024-04-17T07:29:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.070+0000 7fd1b0627740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd18800db00 con 0x559425836b50 2024-04-17T07:29:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.070+0000 7fd1b0627740 10 client.0 _ll_register_callbacks cb 0x7fd19409b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.071+0000 7fd1a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55942583bcc0 0x5594258464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.071+0000 7fd1a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55942583bcc0 0x5594258464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47480/0 (socket says 172.21.15.86:47480) 2024-04-17T07:29:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.071+0000 7fd1a7400640 1 -- 172.21.15.86:0/318218985 learned_addr learned my addr 172.21.15.86:0/318218985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.071+0000 7fd1a6000640 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 0x5594258454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.072+0000 7fd1a7400640 1 -- 172.21.15.86:0/318218985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 msgr2=0x5594258454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.072+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836160 0x559425844fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.072+0000 7fd1a7400640 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 0x5594258454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.073+0000 7fd1a7400640 1 -- 172.21.15.86:0/318218985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836160 msgr2=0x559425844fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.073+0000 7fd1a7400640 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836160 0x559425844fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.073+0000 7fd1a7400640 1 -- 172.21.15.86:0/318218985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594255787d0 con 0x55942583bcc0 2024-04-17T07:29:50.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.073+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836160 0x559425844fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:50.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.073+0000 7fd1a6000640 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 0x5594258454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:29:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.073+0000 7fd1a7400640 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55942583bcc0 0x5594258464a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8050f60 tx=0x7fd1a80983d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1759376be726a2d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.073+0000 7fd190c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.074+0000 7fd190c00640 1 -- 172.21.15.86:0/318218985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80be070 con 0x55942583bcc0 2024-04-17T07:29:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.074+0000 7fd190c00640 1 -- 172.21.15.86:0/318218985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fd1a80a5070 con 0x55942583bcc0 2024-04-17T07:29:50.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.075+0000 7fd1b0627740 1 -- 172.21.15.86:0/318218985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd198029070 con 0x55942583bcc0 2024-04-17T07:29:50.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.076+0000 7fd190c00640 1 -- 172.21.15.86:0/318218985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1a80c8050 con 0x55942583bcc0 2024-04-17T07:29:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.076+0000 7fd1b0627740 10 client.19056 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.076+0000 7fd1b0627740 1 -- 172.21.15.86:0/318218985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559425847ba0 con 0x55942583bcc0 2024-04-17T07:29:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.077+0000 7fd190c00640 1 -- 172.21.15.86:0/318218985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd198029070 con 0x55942583bcc0 2024-04-17T07:29:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.077+0000 7fd190c00640 1 -- 172.21.15.86:0/318218985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fd1a80b7050 con 0x55942583bcc0 2024-04-17T07:29:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.078+0000 7fd1b0627740 10 client.19056 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.078+0000 7fd1b0627740 10 client.19056 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:29:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.078+0000 7fd1b0627740 20 client.19056 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:50.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.078+0000 7fd1b0627740 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594258b2be0 0x5594258b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.078+0000 7fd1b0627740 4 client.19056 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:29:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.078+0000 7fd1b0627740 1 -- 172.21.15.86:0/318218985 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594257a38b0 con 0x5594258b2be0 2024-04-17T07:29:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.079+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594258b2be0 0x5594258b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.080+0000 7fd1a6a00640 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594258b2be0 0x5594258b5000 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bae3621860f17d1 server_cookie=9da378b5b80873da in_seq=0 out_seq=0 2024-04-17T07:29:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.080+0000 7fd190c00640 10 client.19056 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:29:50.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.081+0000 7fd190c00640 1 -- 172.21.15.86:0/318218985 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594257a38b0 con 0x5594258b2be0 2024-04-17T07:29:50.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.081+0000 7fd190c00640 10 client.19056 handle_command_reply: tid=0 2024-04-17T07:29:50.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.081+0000 7fd1b0627740 1 client.19056 shutdown 2024-04-17T07:29:50.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.082+0000 7fd1b0627740 1 -- 172.21.15.86:0/318218985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55942583bcc0 msgr2=0x5594258464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.082+0000 7fd1b0627740 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55942583bcc0 0x5594258464a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8050f60 tx=0x7fd1a80983d0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.082+0000 7fd1b0627740 1 -- 172.21.15.86:0/318218985 shutdown_connections 2024-04-17T07:29:50.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.082+0000 7fd1b0627740 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55942583bcc0 0x5594258464a0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.082+0000 7fd1b0627740 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559425836b50 0x5594258454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.082+0000 7fd1b0627740 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594258b2be0 0x5594258b5000 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.082+0000 7fd1b0627740 1 --2- 172.21.15.86:0/318218985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559425836160 0x559425844fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.082+0000 7fd190c00640 0 client.19056 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:29:50.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.082+0000 7fd1b0627740 1 -- 172.21.15.86:0/318218985 >> 172.21.15.86:0/318218985 conn(0x5594258316d0 msgr2=0x559425834930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.083+0000 7fd1b0627740 1 -- 172.21.15.86:0/318218985 shutdown_connections 2024-04-17T07:29:50.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.083+0000 7fd1b0627740 1 -- 172.21.15.86:0/318218985 wait complete. 2024-04-17T07:29:50.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.083+0000 7fd1b0627740 20 client.19056 trim_cache size 0 max 16384 2024-04-17T07:29:50.083 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:29:50.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.085+0000 7fd1a7e00640 1 -- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd18c0a3c30 msgr2=0x7fd18c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.085+0000 7fd1a7e00640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd18c0a3c30 0x7fd18c0a60f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8067e70 tx=0x7fd1a8063410 comp rx=0 tx=0).stop 2024-04-17T07:29:50.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.085+0000 7fd1a7e00640 1 -- 172.21.15.86:0/3577828167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a00945f0 msgr2=0x7fd1a009fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.085+0000 7fd1a7e00640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a00945f0 0x7fd1a009fc10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd19400b4a0 tx=0x7fd19400b960 comp rx=0 tx=0).stop 2024-04-17T07:29:50.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.085+0000 7fd1a7e00640 1 -- 172.21.15.86:0/3577828167 shutdown_connections 2024-04-17T07:29:50.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.085+0000 7fd1a7e00640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fd18c0a3c30 0x7fd18c0a60f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.085+0000 7fd1a7e00640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1a0099910 0x7fd1a0142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.085+0000 7fd1a7e00640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1a0096f60 0x7fd1a00a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.085+0000 7fd1a7e00640 1 --2- 172.21.15.86:0/3577828167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a00945f0 0x7fd1a009fc10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.085+0000 7fd1a7e00640 1 -- 172.21.15.86:0/3577828167 >> 172.21.15.86:0/3577828167 conn(0x7fd1a00901d0 msgr2=0x7fd1a0097d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.086+0000 7fd1a7e00640 1 -- 172.21.15.86:0/3577828167 shutdown_connections 2024-04-17T07:29:50.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.086+0000 7fd1a7e00640 1 -- 172.21.15.86:0/3577828167 wait complete. 2024-04-17T07:29:50.116 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:29:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.465+0000 7f4c73000640 1 -- 172.21.15.86:0/2584361013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c64009550 msgr2=0x7f4c64009970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.465+0000 7f4c73000640 1 --2- 172.21.15.86:0/2584361013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c64009550 0x7f4c64009970 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009900 tx=0x7f4c6002bc30 comp rx=0 tx=0).stop 2024-04-17T07:29:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.466+0000 7f4c73000640 1 -- 172.21.15.86:0/2584361013 shutdown_connections 2024-04-17T07:29:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.466+0000 7f4c73000640 1 --2- 172.21.15.86:0/2584361013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c6400b130 0x7f4c6400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.466+0000 7f4c73000640 1 --2- 172.21.15.86:0/2584361013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c6400a790 0x7f4c6400abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.466+0000 7f4c73000640 1 --2- 172.21.15.86:0/2584361013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c64009550 0x7f4c64009970 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.466+0000 7f4c73000640 1 -- 172.21.15.86:0/2584361013 >> 172.21.15.86:0/2584361013 conn(0x7f4c640934e0 msgr2=0x7f4c64095920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.468+0000 7f4c73000640 1 -- 172.21.15.86:0/2584361013 shutdown_connections 2024-04-17T07:29:50.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.469+0000 7f4c73000640 1 -- 172.21.15.86:0/2584361013 wait complete. 2024-04-17T07:29:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.470+0000 7f4c73000640 1 Processor -- start 2024-04-17T07:29:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.470+0000 7f4c73000640 1 -- start start 2024-04-17T07:29:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.471+0000 7f4c73000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c64009550 0x7f4c640a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.471+0000 7f4c73000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c6400a790 0x7f4c640a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.471+0000 7f4c73000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c6400b130 0x7f4c641425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.472+0000 7f4c72600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c6400b130 0x7f4c641425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.472+0000 7f4c72600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c6400b130 0x7f4c641425e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47492/0 (socket says 172.21.15.86:47492) 2024-04-17T07:29:50.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.472+0000 7f4c71200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c6400a790 0x7f4c640a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.472+0000 7f4c71200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c6400a790 0x7f4c640a2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45552/0 (socket says 172.21.15.86:45552) 2024-04-17T07:29:50.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.472+0000 7f4c72600640 1 -- 172.21.15.86:0/682197454 learned_addr learned my addr 172.21.15.86:0/682197454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:50.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.472+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640a7ef0 con 0x7f4c6400b130 2024-04-17T07:29:50.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.473+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640a7d70 con 0x7f4c6400a790 2024-04-17T07:29:50.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.473+0000 7f4c71c00640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c64009550 0x7f4c640a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.473+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640a8070 con 0x7f4c64009550 2024-04-17T07:29:50.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.474+0000 7f4c71c00640 1 -- 172.21.15.86:0/682197454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c6400a790 msgr2=0x7f4c640a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.474+0000 7f4c71c00640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c6400a790 0x7f4c640a2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.474+0000 7f4c71c00640 1 -- 172.21.15.86:0/682197454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c6400b130 msgr2=0x7f4c641425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.474+0000 7f4c71c00640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c6400b130 0x7f4c641425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.474+0000 7f4c71c00640 1 -- 172.21.15.86:0/682197454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c64142b20 con 0x7f4c64009550 2024-04-17T07:29:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.475+0000 7f4c71c00640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c64009550 0x7f4c640a29a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4c6002c170 tx=0x7f4c600040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.475+0000 7f4c6a000640 1 -- 172.21.15.86:0/682197454 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60043020 con 0x7f4c64009550 2024-04-17T07:29:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.476+0000 7f4c6a000640 1 -- 172.21.15.86:0/682197454 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c600044c0 con 0x7f4c64009550 2024-04-17T07:29:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.476+0000 7f4c6a000640 1 -- 172.21.15.86:0/682197454 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60003aa0 con 0x7f4c64009550 2024-04-17T07:29:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.476+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c64142cf0 con 0x7f4c64009550 2024-04-17T07:29:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.476+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c64143100 con 0x7f4c64009550 2024-04-17T07:29:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.482+0000 7f4c7738e740 1 Processor -- start 2024-04-17T07:29:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.483+0000 7f4c6a000640 1 -- 172.21.15.86:0/682197454 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4c60046020 con 0x7f4c64009550 2024-04-17T07:29:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.485+0000 7f4c6a000640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4c540a3c30 0x7f4c540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.486+0000 7f4c7738e740 1 -- start start 2024-04-17T07:29:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.487+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 0x558e09095580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.487+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 0x558e0909a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.487+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e0909acc0 0x558e0909d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.487+0000 7f4c7738e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e08ffb020 con 0x558e0909acc0 2024-04-17T07:29:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.487+0000 7f4c7738e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558e08fa1460 con 0x558e09095160 2024-04-17T07:29:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.487+0000 7f4c7738e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e08ff6fe0 con 0x558e09095b50 2024-04-17T07:29:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.487+0000 7f4c72600640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4c540a3c30 0x7f4c540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.488+0000 7f4c71200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 0x558e0909a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.488+0000 7f4c71200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 0x558e0909a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49916/0 (socket says 172.21.15.86:49916) 2024-04-17T07:29:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.488+0000 7f4c71200640 1 -- 172.21.15.86:0/2905519382 learned_addr learned my addr 172.21.15.86:0/2905519382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.489+0000 7f4c6a000640 1 -- 172.21.15.86:0/682197454 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4c600f28b0 con 0x7f4c64009550 2024-04-17T07:29:50.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.489+0000 7f4c71200640 1 -- 172.21.15.86:0/2905519382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 msgr2=0x558e09095580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:29:50.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.489+0000 7f4c71200640 1 --2- 172.21.15.86:0/2905519382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 0x558e09095580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.489+0000 7f4c71200640 1 -- 172.21.15.86:0/2905519382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e0909acc0 msgr2=0x558e0909d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.489+0000 7f4c71200640 1 --2- 172.21.15.86:0/2905519382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e0909acc0 0x558e0909d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.489+0000 7f4c71200640 1 -- 172.21.15.86:0/2905519382 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e08fa9040 con 0x558e09095b50 2024-04-17T07:29:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.489+0000 7f4c72600640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4c540a3c30 0x7f4c540a60f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c000b60 tx=0x7f4c6c050d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.490+0000 7f4c71200640 1 --2- 172.21.15.86:0/2905519382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 0x558e0909a650 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800bcb0 tx=0x7f4c580107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d747488b34bdc823 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.492+0000 7f4c5ea00640 1 -- 172.21.15.86:0/2905519382 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58004030 con 0x558e09095b50 2024-04-17T07:29:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.492+0000 7f4c5ea00640 1 -- 172.21.15.86:0/2905519382 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c580041d0 con 0x558e09095b50 2024-04-17T07:29:50.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.492+0000 7f4c5ea00640 1 -- 172.21.15.86:0/2905519382 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c580116c0 con 0x558e09095b50 2024-04-17T07:29:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.493+0000 7f4c7738e740 1 -- 172.21.15.86:0/2905519382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 msgr2=0x558e0909a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.493+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2905519382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 0x558e0909a650 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800bcb0 tx=0x7f4c580107a0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.494+0000 7f4c7738e740 1 -- 172.21.15.86:0/2905519382 shutdown_connections 2024-04-17T07:29:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.494+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2905519382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e0909acc0 0x558e0909d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.494+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2905519382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 0x558e0909a650 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.494+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2905519382 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 0x558e09095580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.494+0000 7f4c7738e740 1 -- 172.21.15.86:0/2905519382 >> 172.21.15.86:0/2905519382 conn(0x558e090906d0 msgr2=0x558e09090ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.495+0000 7f4c7738e740 1 -- 172.21.15.86:0/2905519382 shutdown_connections 2024-04-17T07:29:50.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.495+0000 7f4c7738e740 1 -- 172.21.15.86:0/2905519382 wait complete. 2024-04-17T07:29:50.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.495+0000 7f4c7738e740 1 -- start start 2024-04-17T07:29:50.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.496+0000 7f4c7738e740 1 Processor -- start 2024-04-17T07:29:50.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.497+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e0909f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.497+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 0x558e0909fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.497+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e0909acc0 0x558e090a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.498+0000 7f4c7738e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e0909dc70 con 0x558e09095b50 2024-04-17T07:29:50.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.498+0000 7f4c7738e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558e08fb4830 con 0x558e0909acc0 2024-04-17T07:29:50.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.498+0000 7f4c7738e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e0909ddf0 con 0x558e09095160 2024-04-17T07:29:50.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.498+0000 7f4c7738e740 10 client.0 _ll_register_callbacks cb 0x558e08f82fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:50.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.498+0000 7f4c71c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e0909f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.498+0000 7f4c71200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 0x558e0909fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.498+0000 7f4c71c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e0909f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49920/0 (socket says 172.21.15.86:49920) 2024-04-17T07:29:50.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.498+0000 7f4c71c00640 1 -- 172.21.15.86:0/2724755569 learned_addr learned my addr 172.21.15.86:0/2724755569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:50.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.498+0000 7f4c72600640 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e0909acc0 0x558e090a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.502+0000 7f4c71c00640 1 -- 172.21.15.86:0/2724755569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e0909acc0 msgr2=0x558e090a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.502+0000 7f4c71c00640 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e0909acc0 0x558e090a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.502+0000 7f4c71c00640 1 -- 172.21.15.86:0/2724755569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 msgr2=0x558e0909fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.502+0000 7f4c71c00640 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 0x558e0909fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.502+0000 7f4c71c00640 1 -- 172.21.15.86:0/2724755569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e08fb8920 con 0x558e09095160 2024-04-17T07:29:50.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.503+0000 7f4c71c00640 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e0909f770 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4c6002c680 tx=0x7f4c600be2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2004186259a9d677 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.503+0000 7f4c53400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.504+0000 7f4c53400640 1 -- 172.21.15.86:0/2724755569 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60003e40 con 0x558e09095160 2024-04-17T07:29:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.504+0000 7f4c53400640 1 -- 172.21.15.86:0/2724755569 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4c600c70b0 con 0x558e09095160 2024-04-17T07:29:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.505+0000 7f4c7738e740 1 -- 172.21.15.86:0/2724755569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c580041d0 con 0x558e09095160 2024-04-17T07:29:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.505+0000 7f4c53400640 1 -- 172.21.15.86:0/2724755569 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c600c2c80 con 0x558e09095160 2024-04-17T07:29:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.505+0000 7f4c7738e740 10 client.28075 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.505+0000 7f4c7738e740 1 -- 172.21.15.86:0/2724755569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e08f867f0 con 0x558e09095160 2024-04-17T07:29:50.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.506+0000 7f4c53400640 1 -- 172.21.15.86:0/2724755569 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c600c7540 con 0x558e09095160 2024-04-17T07:29:50.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.506+0000 7f4c53400640 1 -- 172.21.15.86:0/2724755569 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4c600bd790 con 0x558e09095160 2024-04-17T07:29:50.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.506+0000 7f4c7738e740 10 client.28075 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.506+0000 7f4c7738e740 10 client.28075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.507+0000 7f4c7738e740 20 client.28075 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.507+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558e090ab230 0x558e090ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.507+0000 7f4c7738e740 4 client.28075 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.507+0000 7f4c7738e740 1 -- 172.21.15.86:0/2724755569 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4c580041d0 con 0x558e090ab230 2024-04-17T07:29:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.508+0000 7f4c71200640 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558e090ab230 0x558e090ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.509+0000 7f4c71200640 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558e090ab230 0x558e090ad620 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92237983e5720d9a server_cookie=33f9349c8bf60359 in_seq=0 out_seq=0 2024-04-17T07:29:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.509+0000 7f4c53400640 10 client.28075 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.512+0000 7f4c53400640 1 -- 172.21.15.86:0/2724755569 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f4c580041d0 con 0x558e090ab230 2024-04-17T07:29:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.512+0000 7f4c53400640 10 client.28075 handle_command_reply: tid=0 2024-04-17T07:29:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.512+0000 7f4c7738e740 1 client.28075 shutdown 2024-04-17T07:29:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.513+0000 7f4c7738e740 1 -- 172.21.15.86:0/2724755569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 msgr2=0x558e0909f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.513+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e0909f770 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4c6002c680 tx=0x7f4c600be2b0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.514+0000 7f4c7738e740 1 -- 172.21.15.86:0/2724755569 shutdown_connections 2024-04-17T07:29:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.514+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558e090ab230 0x558e090ad620 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.514+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e0909acc0 0x558e090a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.514+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 0x558e0909fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.514+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2724755569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e0909f770 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.514+0000 7f4c7738e740 1 -- 172.21.15.86:0/2724755569 >> 172.21.15.86:0/2724755569 conn(0x558e090906d0 msgr2=0x558e0908f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.514+0000 7f4c53400640 0 client.28075 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.515+0000 7f4c7738e740 1 -- 172.21.15.86:0/2724755569 shutdown_connections 2024-04-17T07:29:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.516+0000 7f4c7738e740 1 -- 172.21.15.86:0/2724755569 wait complete. 2024-04-17T07:29:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.516+0000 7f4c7738e740 20 client.28075 trim_cache size 0 max 16384 2024-04-17T07:29:50.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.562+0000 7f4c7738e740 1 Processor -- start 2024-04-17T07:29:50.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.562+0000 7f4c7738e740 1 -- start start 2024-04-17T07:29:50.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.562+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 0x558e0909fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.562+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 0x558e090a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.562+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e0909acc0 0x558e0908f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.562+0000 7f4c7738e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6414d540 con 0x558e0909acc0 2024-04-17T07:29:50.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.562+0000 7f4c7738e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6414f080 con 0x558e09095160 2024-04-17T07:29:50.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.562+0000 7f4c7738e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6414dc10 con 0x558e09095b50 2024-04-17T07:29:50.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.563+0000 7f4c72600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e0909acc0 0x558e0908f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.563+0000 7f4c71c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 0x558e0909fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.563+0000 7f4c71c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 0x558e0909fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45588/0 (socket says 172.21.15.86:45588) 2024-04-17T07:29:50.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.563+0000 7f4c71c00640 1 -- 172.21.15.86:0/2703925922 learned_addr learned my addr 172.21.15.86:0/2703925922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:50.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.563+0000 7f4c71200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 0x558e090a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.564+0000 7f4c71c00640 1 -- 172.21.15.86:0/2703925922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 msgr2=0x558e090a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.564+0000 7f4c71c00640 1 --2- 172.21.15.86:0/2703925922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 0x558e090a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.564+0000 7f4c71c00640 1 -- 172.21.15.86:0/2703925922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e0909acc0 msgr2=0x558e0908f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.564+0000 7f4c71c00640 1 --2- 172.21.15.86:0/2703925922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e0909acc0 0x558e0908f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.564+0000 7f4c71c00640 1 -- 172.21.15.86:0/2703925922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e08d0d3f0 con 0x558e09095160 2024-04-17T07:29:50.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.564+0000 7f4c71c00640 1 --2- 172.21.15.86:0/2703925922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 0x558e0909fa90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4c600be3a0 tx=0x7f4c600bea50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5136b76d79af680e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.565+0000 7f4c5e000640 1 -- 172.21.15.86:0/2703925922 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600c7540 con 0x558e09095160 2024-04-17T07:29:50.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.565+0000 7f4c5e000640 1 -- 172.21.15.86:0/2703925922 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c600bd790 con 0x558e09095160 2024-04-17T07:29:50.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.565+0000 7f4c5e000640 1 -- 172.21.15.86:0/2703925922 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600c4190 con 0x558e09095160 2024-04-17T07:29:50.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.567+0000 7f4c7738e740 1 -- 172.21.15.86:0/2703925922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 msgr2=0x558e0909fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.567+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2703925922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 0x558e0909fa90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4c600be3a0 tx=0x7f4c600bea50 comp rx=0 tx=0).stop 2024-04-17T07:29:50.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.567+0000 7f4c7738e740 1 -- 172.21.15.86:0/2703925922 shutdown_connections 2024-04-17T07:29:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.567+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2703925922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e0909acc0 0x558e0908f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.567+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2703925922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095b50 0x558e090a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.567+0000 7f4c7738e740 1 --2- 172.21.15.86:0/2703925922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e09095160 0x558e0909fa90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.567+0000 7f4c7738e740 1 -- 172.21.15.86:0/2703925922 >> 172.21.15.86:0/2703925922 conn(0x558e090906d0 msgr2=0x558e0909f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.567+0000 7f4c7738e740 1 -- 172.21.15.86:0/2703925922 shutdown_connections 2024-04-17T07:29:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.567+0000 7f4c7738e740 1 -- 172.21.15.86:0/2703925922 wait complete. 2024-04-17T07:29:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.568+0000 7f4c7738e740 1 -- start start 2024-04-17T07:29:50.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.568+0000 7f4c7738e740 1 Processor -- start 2024-04-17T07:29:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.574+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e090a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.574+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 0x558e090a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.575+0000 7f4c7738e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e0909acc0 0x558e0910f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.575+0000 7f4c71c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e090a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.575+0000 7f4c7738e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6414e9b0 con 0x558e09095b50 2024-04-17T07:29:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.575+0000 7f4c71200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 0x558e090a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.575+0000 7f4c7738e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6414e2e0 con 0x558e0909acc0 2024-04-17T07:29:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.575+0000 7f4c7738e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6414b510 con 0x558e09095160 2024-04-17T07:29:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.575+0000 7f4c71c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e090a3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49926/0 (socket says 172.21.15.86:49926) 2024-04-17T07:29:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.575+0000 7f4c7738e740 10 client.0 _ll_register_callbacks cb 0x7f4c600be410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.575+0000 7f4c71c00640 1 -- 172.21.15.86:0/3043088307 learned_addr learned my addr 172.21.15.86:0/3043088307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.576+0000 7f4c72600640 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e0909acc0 0x558e0910f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.576+0000 7f4c71200640 1 -- 172.21.15.86:0/3043088307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 msgr2=0x558e090a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.576+0000 7f4c71200640 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e090a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.576+0000 7f4c71200640 1 -- 172.21.15.86:0/3043088307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e0909acc0 msgr2=0x558e0910f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.576+0000 7f4c71200640 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e0909acc0 0x558e0910f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.576+0000 7f4c71200640 1 -- 172.21.15.86:0/3043088307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e08dd77d0 con 0x558e09095b50 2024-04-17T07:29:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.577+0000 7f4c71200640 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 0x558e090a40e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x558e09091640 tx=0x7f4c5800b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae68fd45a3f686ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:50.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.577+0000 7f4c53400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:50.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.578+0000 7f4c53400640 1 -- 172.21.15.86:0/3043088307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58005360 con 0x558e09095b50 2024-04-17T07:29:50.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.578+0000 7f4c53400640 1 -- 172.21.15.86:0/3043088307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4c58005500 con 0x558e09095b50 2024-04-17T07:29:50.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.579+0000 7f4c7738e740 1 -- 172.21.15.86:0/3043088307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c600bd790 con 0x558e09095b50 2024-04-17T07:29:50.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.579+0000 7f4c53400640 1 -- 172.21.15.86:0/3043088307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c58038050 con 0x558e09095b50 2024-04-17T07:29:50.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.580+0000 7f4c7738e740 10 client.19071 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:50.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.580+0000 7f4c7738e740 1 -- 172.21.15.86:0/3043088307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e09110ce0 con 0x558e09095b50 2024-04-17T07:29:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.582+0000 7f4c53400640 1 -- 172.21.15.86:0/3043088307 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c600bd790 con 0x558e09095b50 2024-04-17T07:29:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.585+0000 7f4c53400640 1 -- 172.21.15.86:0/3043088307 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4c580037f0 con 0x558e09095b50 2024-04-17T07:29:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.585+0000 7f4c7738e740 10 client.19071 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.585+0000 7f4c7738e740 10 client.19071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.585+0000 7f4c7738e740 20 client.19071 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:50.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.585+0000 7f4c7738e740 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558e091113e0 0x558e09113800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.585+0000 7f4c7738e740 4 client.19071 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:29:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.585+0000 7f4c7738e740 1 -- 172.21.15.86:0/3043088307 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e09008a40 con 0x558e091113e0 2024-04-17T07:29:50.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.586+0000 7f4c71c00640 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558e091113e0 0x558e09113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.587+0000 7f4c71c00640 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558e091113e0 0x558e09113800 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c29d9e62e9b65ad server_cookie=ccff6251712582d3 in_seq=0 out_seq=0 2024-04-17T07:29:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.587+0000 7f4c53400640 10 client.19071 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.587+0000 7f4c53400640 1 -- 172.21.15.86:0/3043088307 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e09008a40 con 0x558e091113e0 2024-04-17T07:29:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.587+0000 7f4c53400640 10 client.19071 handle_command_reply: tid=0 2024-04-17T07:29:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.587+0000 7f4c7738e740 1 client.19071 shutdown 2024-04-17T07:29:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.588+0000 7f4c7738e740 1 -- 172.21.15.86:0/3043088307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 msgr2=0x558e090a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.588+0000 7f4c7738e740 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 0x558e090a40e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x558e09091640 tx=0x7f4c5800b820 comp rx=0 tx=0).stop 2024-04-17T07:29:50.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.588+0000 7f4c7738e740 1 -- 172.21.15.86:0/3043088307 shutdown_connections 2024-04-17T07:29:50.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.588+0000 7f4c7738e740 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x558e091113e0 0x558e09113800 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.588+0000 7f4c7738e740 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558e0909acc0 0x558e0910f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.588+0000 7f4c7738e740 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e09095b50 0x558e090a40e0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.588+0000 7f4c7738e740 1 --2- 172.21.15.86:0/3043088307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e09095160 0x558e090a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.588+0000 7f4c53400640 0 client.19071 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:50.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.588+0000 7f4c7738e740 1 -- 172.21.15.86:0/3043088307 >> 172.21.15.86:0/3043088307 conn(0x558e090906d0 msgr2=0x558e09093930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.591+0000 7f4c7738e740 1 -- 172.21.15.86:0/3043088307 shutdown_connections 2024-04-17T07:29:50.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.591+0000 7f4c7738e740 1 -- 172.21.15.86:0/3043088307 wait complete. 2024-04-17T07:29:50.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.592+0000 7f4c7738e740 20 client.19071 trim_cache size 0 max 16384 2024-04-17T07:29:50.592 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:29:50.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.594+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4c540a3c30 msgr2=0x7f4c540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.594+0000 7f4c73000640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4c540a3c30 0x7f4c540a60f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c000b60 tx=0x7f4c6c050d10 comp rx=0 tx=0).stop 2024-04-17T07:29:50.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.594+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c64009550 msgr2=0x7f4c640a29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:50.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.594+0000 7f4c73000640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c64009550 0x7f4c640a29a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4c6002c170 tx=0x7f4c600040c0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.595+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 shutdown_connections 2024-04-17T07:29:50.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.595+0000 7f4c73000640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4c540a3c30 0x7f4c540a60f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.595+0000 7f4c73000640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c6400b130 0x7f4c641425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.595+0000 7f4c73000640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c6400a790 0x7f4c640a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.595+0000 7f4c73000640 1 --2- 172.21.15.86:0/682197454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c64009550 0x7f4c640a29a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:50.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.595+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 >> 172.21.15.86:0/682197454 conn(0x7f4c640934e0 msgr2=0x7f4c64094fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:50.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.595+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 shutdown_connections 2024-04-17T07:29:50.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:50.595+0000 7f4c73000640 1 -- 172.21.15.86:0/682197454 wait complete. 2024-04-17T07:29:51.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:51 smithi098 ceph-mon[35418]: pgmap v904: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 618 op/s 2024-04-17T07:29:51.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:51 smithi103 ceph-mon[35495]: pgmap v904: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 618 op/s 2024-04-17T07:29:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:51 smithi086 ceph-mon[31733]: pgmap v904: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 618 op/s 2024-04-17T07:29:51.630 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:29:51.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:29:51.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:29:51.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.985+0000 7fbc57200640 1 -- 172.21.15.86:0/776674045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc58109690 msgr2=0x7fbc58109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:51.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.985+0000 7fbc57200640 1 --2- 172.21.15.86:0/776674045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc58109690 0x7fbc58109b10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009900 tx=0x7fbc4402bc40 comp rx=0 tx=0).stop 2024-04-17T07:29:51.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.986+0000 7fbc57200640 1 -- 172.21.15.86:0/776674045 shutdown_connections 2024-04-17T07:29:51.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.986+0000 7fbc57200640 1 --2- 172.21.15.86:0/776674045 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5810a050 0x7fbc581108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:51.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.986+0000 7fbc57200640 1 --2- 172.21.15.86:0/776674045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc58109690 0x7fbc58109b10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:51.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.986+0000 7fbc57200640 1 --2- 172.21.15.86:0/776674045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc581084a0 0x7fbc581088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:51.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.986+0000 7fbc57200640 1 -- 172.21.15.86:0/776674045 >> 172.21.15.86:0/776674045 conn(0x7fbc58077430 msgr2=0x7fbc58079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:51.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.987+0000 7fbc57200640 1 -- 172.21.15.86:0/776674045 shutdown_connections 2024-04-17T07:29:51.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.987+0000 7fbc57200640 1 -- 172.21.15.86:0/776674045 wait complete. 2024-04-17T07:29:51.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.989+0000 7fbc57200640 1 Processor -- start 2024-04-17T07:29:51.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.989+0000 7fbc57200640 1 -- start start 2024-04-17T07:29:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.989+0000 7fbc57200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc581084a0 0x7fbc5807f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.989+0000 7fbc57200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc58109690 0x7fbc5807fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.989+0000 7fbc57200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5810a050 0x7fbc581ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.989+0000 7fbc57200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58114090 con 0x7fbc581084a0 2024-04-17T07:29:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.989+0000 7fbc57200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58113f10 con 0x7fbc58109690 2024-04-17T07:29:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.989+0000 7fbc57200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58114210 con 0x7fbc5810a050 2024-04-17T07:29:51.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.990+0000 7fbc55e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc581084a0 0x7fbc5807f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:51.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.990+0000 7fbc55e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc581084a0 0x7fbc5807f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47536/0 (socket says 172.21.15.86:47536) 2024-04-17T07:29:51.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.990+0000 7fbc55400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc58109690 0x7fbc5807fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:51.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.990+0000 7fbc55e00640 1 -- 172.21.15.86:0/2651536988 learned_addr learned my addr 172.21.15.86:0/2651536988 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:51.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.990+0000 7fbc56800640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5810a050 0x7fbc581ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:51.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.992+0000 7fbc56800640 1 -- 172.21.15.86:0/2651536988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc58109690 msgr2=0x7fbc5807fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:51.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.992+0000 7fbc56800640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc58109690 0x7fbc5807fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:51.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.992+0000 7fbc56800640 1 -- 172.21.15.86:0/2651536988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc581084a0 msgr2=0x7fbc5807f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:51.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.992+0000 7fbc56800640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc581084a0 0x7fbc5807f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:51.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.992+0000 7fbc56800640 1 -- 172.21.15.86:0/2651536988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc58082c80 con 0x7fbc5810a050 2024-04-17T07:29:51.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.992+0000 7fbc55e00640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc581084a0 0x7fbc5807f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:29:51.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.994+0000 7fbc56800640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5810a050 0x7fbc581ae7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800d680 tx=0x7fbc4800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:51.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.994+0000 7fbc4ea00640 1 -- 172.21.15.86:0/2651536988 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48017070 con 0x7fbc5810a050 2024-04-17T07:29:51.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.995+0000 7fbc4ea00640 1 -- 172.21.15.86:0/2651536988 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc48009820 con 0x7fbc5810a050 2024-04-17T07:29:51.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.995+0000 7fbc4ea00640 1 -- 172.21.15.86:0/2651536988 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48004cf0 con 0x7fbc5810a050 2024-04-17T07:29:51.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.995+0000 7fbc57200640 1 -- 172.21.15.86:0/2651536988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc581aece0 con 0x7fbc5810a050 2024-04-17T07:29:51.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:51.995+0000 7fbc57200640 1 -- 172.21.15.86:0/2651536988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc581af1b0 con 0x7fbc5810a050 2024-04-17T07:29:52.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.005+0000 7fbc5fc78740 1 Processor -- start 2024-04-17T07:29:52.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.006+0000 7fbc4ea00640 1 -- 172.21.15.86:0/2651536988 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fbc480099d0 con 0x7fbc5810a050 2024-04-17T07:29:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.006+0000 7fbc5fc78740 1 -- start start 2024-04-17T07:29:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.008+0000 7fbc4ea00640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbc3c0a3d00 0x7fbc3c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.008+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.008+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030f8a90 0x5606030fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.008+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030fdc00 0x560603100090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.008+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56060305dcb0 con 0x5606030f8a90 2024-04-17T07:29:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.008+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560603004370 con 0x5606030fdc00 2024-04-17T07:29:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.008+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560603059c70 con 0x5606030f80a0 2024-04-17T07:29:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.008+0000 7fbc4ea00640 1 -- 172.21.15.86:0/2651536988 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbc480d0010 con 0x7fbc5810a050 2024-04-17T07:29:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.009+0000 7fbc55e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.009+0000 7fbc55e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49960/0 (socket says 172.21.15.86:49960) 2024-04-17T07:29:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.009+0000 7fbc55e00640 1 -- 172.21.15.86:0/1015462510 learned_addr learned my addr 172.21.15.86:0/1015462510 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.009+0000 7fbc55400640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbc3c0a3d00 0x7fbc3c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.009+0000 7fbc56800640 1 --2- 172.21.15.86:0/1015462510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030fdc00 0x560603100090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.009+0000 7fbc55e00640 1 -- 172.21.15.86:0/1015462510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030fdc00 msgr2=0x560603100090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.009+0000 7fbc55e00640 1 --2- 172.21.15.86:0/1015462510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030fdc00 0x560603100090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.009+0000 7fbc55e00640 1 -- 172.21.15.86:0/1015462510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030f8a90 msgr2=0x5606030fd590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:29:52.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.010+0000 7fbc55400640 1 --2- 172.21.15.86:0/1015462510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030f8a90 0x5606030fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.010+0000 7fbc55e00640 1 --2- 172.21.15.86:0/1015462510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030f8a90 0x5606030fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.010+0000 7fbc55e00640 1 -- 172.21.15.86:0/1015462510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560603023910 con 0x5606030f80a0 2024-04-17T07:29:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.010+0000 7fbc55400640 1 --2- 172.21.15.86:0/1015462510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030f8a90 0x5606030fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:29:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.011+0000 7fbc55e00640 1 --2- 172.21.15.86:0/1015462510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f84c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000c570 tx=0x7fbc50009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db26ef9345dd2c6c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.011+0000 7fbc55400640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbc3c0a3d00 0x7fbc3c0a61c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005ab0 tx=0x7fbc44000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.011+0000 7fbc43400640 1 -- 172.21.15.86:0/1015462510 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50018020 con 0x5606030f80a0 2024-04-17T07:29:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.011+0000 7fbc43400640 1 -- 172.21.15.86:0/1015462510 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc50014d40 con 0x5606030f80a0 2024-04-17T07:29:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.011+0000 7fbc43400640 1 -- 172.21.15.86:0/1015462510 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50013e10 con 0x5606030f80a0 2024-04-17T07:29:52.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.016+0000 7fbc5fc78740 1 -- 172.21.15.86:0/1015462510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 msgr2=0x5606030f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.016+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/1015462510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f84c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000c570 tx=0x7fbc50009760 comp rx=0 tx=0).stop 2024-04-17T07:29:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.017+0000 7fbc5fc78740 1 -- 172.21.15.86:0/1015462510 shutdown_connections 2024-04-17T07:29:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.017+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/1015462510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030fdc00 0x560603100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.017+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/1015462510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030f8a90 0x5606030fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.017+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/1015462510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f84c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.017+0000 7fbc5fc78740 1 -- 172.21.15.86:0/1015462510 >> 172.21.15.86:0/1015462510 conn(0x5606030f3610 msgr2=0x5606030f3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.018+0000 7fbc5fc78740 1 -- 172.21.15.86:0/1015462510 shutdown_connections 2024-04-17T07:29:52.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.018+0000 7fbc5fc78740 1 -- 172.21.15.86:0/1015462510 wait complete. 2024-04-17T07:29:52.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.020+0000 7fbc5fc78740 1 -- start start 2024-04-17T07:29:52.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.020+0000 7fbc5fc78740 1 Processor -- start 2024-04-17T07:29:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.021+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606031025e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.021+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 0x560603102b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.021+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 0x560603109b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.021+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560603100bb0 con 0x5606030fdc00 2024-04-17T07:29:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.021+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560603017730 con 0x5606030f8a90 2024-04-17T07:29:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.021+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560603100d30 con 0x5606030f80a0 2024-04-17T07:29:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.021+0000 7fbc5fc78740 10 client.0 _ll_register_callbacks cb 0x560603019b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.022+0000 7fbc55400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 0x560603102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.022+0000 7fbc55400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 0x560603102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45640/0 (socket says 172.21.15.86:45640) 2024-04-17T07:29:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.022+0000 7fbc55400640 1 -- 172.21.15.86:0/526303685 learned_addr learned my addr 172.21.15.86:0/526303685 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.022+0000 7fbc55e00640 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606031025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.022+0000 7fbc56800640 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 0x560603109b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.023+0000 7fbc55400640 1 -- 172.21.15.86:0/526303685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 msgr2=0x5606031025e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.023+0000 7fbc55400640 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606031025e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.023+0000 7fbc55400640 1 -- 172.21.15.86:0/526303685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 msgr2=0x560603109b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.023+0000 7fbc55400640 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 0x560603109b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.023+0000 7fbc55400640 1 -- 172.21.15.86:0/526303685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560602e387e0 con 0x5606030f8a90 2024-04-17T07:29:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.023+0000 7fbc56800640 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 0x560603109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:29:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.024+0000 7fbc55400640 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 0x560603102b20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc44035000 tx=0x7fbc44005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=142014e33bf6ca8f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.024+0000 7fbc37e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:29:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.024+0000 7fbc37e00640 1 -- 172.21.15.86:0/526303685 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44050020 con 0x5606030f8a90 2024-04-17T07:29:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.024+0000 7fbc37e00640 1 -- 172.21.15.86:0/526303685 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbc44004940 con 0x5606030f8a90 2024-04-17T07:29:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.024+0000 7fbc5fc78740 1 -- 172.21.15.86:0/526303685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc50014d40 con 0x5606030f8a90 2024-04-17T07:29:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.025+0000 7fbc37e00640 1 -- 172.21.15.86:0/526303685 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc44045040 con 0x5606030f8a90 2024-04-17T07:29:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.025+0000 7fbc5fc78740 10 client.38048 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.025+0000 7fbc5fc78740 1 -- 172.21.15.86:0/526303685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560602e98f70 con 0x5606030f8a90 2024-04-17T07:29:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.026+0000 7fbc37e00640 1 -- 172.21.15.86:0/526303685 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc50014d40 con 0x5606030f8a90 2024-04-17T07:29:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.026+0000 7fbc37e00640 1 -- 172.21.15.86:0/526303685 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fbc4405d020 con 0x5606030f8a90 2024-04-17T07:29:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.026+0000 7fbc5fc78740 10 client.38048 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.026+0000 7fbc5fc78740 10 client.38048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.027+0000 7fbc5fc78740 20 client.38048 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.027+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56060310e3d0 0x560603110890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.027+0000 7fbc5fc78740 4 client.38048 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.027+0000 7fbc5fc78740 1 -- 172.21.15.86:0/526303685 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56060306b6d0 con 0x56060310e3d0 2024-04-17T07:29:52.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.027+0000 7fbc55e00640 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56060310e3d0 0x560603110890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.028+0000 7fbc55e00640 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56060310e3d0 0x560603110890 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da024ceeb20bfa33 server_cookie=7147ad026c4206aa in_seq=0 out_seq=0 2024-04-17T07:29:52.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.028+0000 7fbc37e00640 10 client.38048 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.029+0000 7fbc37e00640 1 -- 172.21.15.86:0/526303685 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56060306b6d0 con 0x56060310e3d0 2024-04-17T07:29:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.029+0000 7fbc37e00640 10 client.38048 handle_command_reply: tid=0 2024-04-17T07:29:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.030+0000 7fbc5fc78740 1 client.38048 shutdown 2024-04-17T07:29:52.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.030+0000 7fbc5fc78740 1 -- 172.21.15.86:0/526303685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 msgr2=0x560603102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.030+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 0x560603102b20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbc44035000 tx=0x7fbc44005fd0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.031+0000 7fbc5fc78740 1 -- 172.21.15.86:0/526303685 shutdown_connections 2024-04-17T07:29:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.031+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56060310e3d0 0x560603110890 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.031+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 0x560603109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.031+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 0x560603102b20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.031+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/526303685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606031025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.031+0000 7fbc5fc78740 1 -- 172.21.15.86:0/526303685 >> 172.21.15.86:0/526303685 conn(0x5606030f3610 msgr2=0x5606030f2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.031+0000 7fbc37e00640 0 client.38048 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.031+0000 7fbc5fc78740 1 -- 172.21.15.86:0/526303685 shutdown_connections 2024-04-17T07:29:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.032+0000 7fbc5fc78740 1 -- 172.21.15.86:0/526303685 wait complete. 2024-04-17T07:29:52.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.032+0000 7fbc5fc78740 20 client.38048 trim_cache size 0 max 16384 2024-04-17T07:29:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.087+0000 7fbc5fc78740 1 Processor -- start 2024-04-17T07:29:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.087+0000 7fbc5fc78740 1 -- start start 2024-04-17T07:29:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.088+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.088+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 0x560603109b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.088+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 0x5606030f2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.088+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581b9650 con 0x5606030fdc00 2024-04-17T07:29:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.088+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581bb190 con 0x5606030f8a90 2024-04-17T07:29:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.088+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581b9d20 con 0x5606030f80a0 2024-04-17T07:29:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.088+0000 7fbc55e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.089+0000 7fbc55e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49982/0 (socket says 172.21.15.86:49982) 2024-04-17T07:29:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.089+0000 7fbc55e00640 1 -- 172.21.15.86:0/497700927 learned_addr learned my addr 172.21.15.86:0/497700927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.089+0000 7fbc55400640 1 --2- 172.21.15.86:0/497700927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 0x560603109b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.088+0000 7fbc56800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 0x5606030f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.089+0000 7fbc55e00640 1 -- 172.21.15.86:0/497700927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 msgr2=0x560603109b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.089+0000 7fbc55e00640 1 --2- 172.21.15.86:0/497700927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 0x560603109b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.089+0000 7fbc55e00640 1 -- 172.21.15.86:0/497700927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 msgr2=0x5606030f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.089+0000 7fbc55e00640 1 --2- 172.21.15.86:0/497700927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 0x5606030f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.089+0000 7fbc55e00640 1 -- 172.21.15.86:0/497700927 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560602d30170 con 0x5606030f80a0 2024-04-17T07:29:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.090+0000 7fbc55e00640 1 --2- 172.21.15.86:0/497700927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f2100 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbc50004a80 tx=0x7fbc50007000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74c8142ea42b60f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.091+0000 7fbc42a00640 1 -- 172.21.15.86:0/497700927 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50018040 con 0x5606030f80a0 2024-04-17T07:29:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.091+0000 7fbc42a00640 1 -- 172.21.15.86:0/497700927 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc5002c030 con 0x5606030f80a0 2024-04-17T07:29:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.091+0000 7fbc42a00640 1 -- 172.21.15.86:0/497700927 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50018040 con 0x5606030f80a0 2024-04-17T07:29:52.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.092+0000 7fbc5fc78740 1 -- 172.21.15.86:0/497700927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 msgr2=0x5606030f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.092+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/497700927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f2100 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbc50004a80 tx=0x7fbc50007000 comp rx=0 tx=0).stop 2024-04-17T07:29:52.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.092+0000 7fbc5fc78740 1 -- 172.21.15.86:0/497700927 shutdown_connections 2024-04-17T07:29:52.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.092+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/497700927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030fdc00 0x5606030f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.092+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/497700927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f8a90 0x560603109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.092+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/497700927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030f80a0 0x5606030f2100 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.092+0000 7fbc5fc78740 1 -- 172.21.15.86:0/497700927 >> 172.21.15.86:0/497700927 conn(0x5606030f3610 msgr2=0x5606030f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.092+0000 7fbc5fc78740 1 -- 172.21.15.86:0/497700927 shutdown_connections 2024-04-17T07:29:52.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.093+0000 7fbc5fc78740 1 -- 172.21.15.86:0/497700927 wait complete. 2024-04-17T07:29:52.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.093+0000 7fbc5fc78740 1 -- start start 2024-04-17T07:29:52.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.093+0000 7fbc5fc78740 1 Processor -- start 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.094+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f80a0 0x560603107420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.094+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030f8a90 0x560603107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.094+0000 7fbc5fc78740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030fdc00 0x560603108920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.094+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581baac0 con 0x5606030f8a90 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.094+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581ba3f0 con 0x5606030f80a0 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.094+0000 7fbc5fc78740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc581b7620 con 0x5606030fdc00 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.094+0000 7fbc5fc78740 10 client.0 _ll_register_callbacks cb 0x7fbc44009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.095+0000 7fbc55e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f80a0 0x560603107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.095+0000 7fbc56800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030fdc00 0x560603108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.095+0000 7fbc56800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030fdc00 0x560603108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49996/0 (socket says 172.21.15.86:49996) 2024-04-17T07:29:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.095+0000 7fbc56800640 1 -- 172.21.15.86:0/3750684325 learned_addr learned my addr 172.21.15.86:0/3750684325 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.095+0000 7fbc56800640 1 -- 172.21.15.86:0/3750684325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f80a0 msgr2=0x560603107420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.095+0000 7fbc56800640 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f80a0 0x560603107420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.095+0000 7fbc56800640 1 -- 172.21.15.86:0/3750684325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030f8a90 msgr2=0x560603107960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.095+0000 7fbc56800640 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030f8a90 0x560603107960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.095+0000 7fbc56800640 1 -- 172.21.15.86:0/3750684325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560603108f10 con 0x5606030fdc00 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.096+0000 7fbc56800640 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030fdc00 0x560603108920 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbc4809bf00 tx=0x7fbc480d02a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad985ad12485f2ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.096+0000 7fbc37e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.096+0000 7fbc37e00640 1 -- 172.21.15.86:0/3750684325 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48020040 con 0x5606030fdc00 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.096+0000 7fbc5fc78740 1 -- 172.21.15.86:0/3750684325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc5002c030 con 0x5606030fdc00 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.096+0000 7fbc37e00640 1 -- 172.21.15.86:0/3750684325 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbc480b5170 con 0x5606030fdc00 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.097+0000 7fbc37e00640 1 -- 172.21.15.86:0/3750684325 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc480a4b80 con 0x5606030fdc00 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.097+0000 7fbc5fc78740 10 client.28099 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.097+0000 7fbc5fc78740 1 -- 172.21.15.86:0/3750684325 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560603174bb0 con 0x5606030fdc00 2024-04-17T07:29:52.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.097+0000 7fbc37e00640 1 -- 172.21.15.86:0/3750684325 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc5002c030 con 0x5606030fdc00 2024-04-17T07:29:52.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.097+0000 7fbc37e00640 1 -- 172.21.15.86:0/3750684325 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fbc48005220 con 0x5606030fdc00 2024-04-17T07:29:52.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.097+0000 7fbc5fc78740 10 client.28099 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:52.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.097+0000 7fbc5fc78740 10 client.28099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:52.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.098+0000 7fbc5fc78740 20 client.28099 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:52.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.098+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560603175580 0x5606031779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.098+0000 7fbc5fc78740 4 client.28099 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:29:52.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.098+0000 7fbc5fc78740 1 -- 172.21.15.86:0/3750684325 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560603065540 con 0x560603175580 2024-04-17T07:29:52.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.098+0000 7fbc55e00640 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560603175580 0x5606031779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.100+0000 7fbc55e00640 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560603175580 0x5606031779a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c56263e067a8fb4 server_cookie=922c546e700754ec in_seq=0 out_seq=0 2024-04-17T07:29:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.100+0000 7fbc37e00640 10 client.28099 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.102+0000 7fbc37e00640 1 -- 172.21.15.86:0/3750684325 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560603065540 con 0x560603175580 2024-04-17T07:29:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.102+0000 7fbc37e00640 10 client.28099 handle_command_reply: tid=0 2024-04-17T07:29:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.102+0000 7fbc5fc78740 1 client.28099 shutdown 2024-04-17T07:29:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.104+0000 7fbc5fc78740 1 -- 172.21.15.86:0/3750684325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030fdc00 msgr2=0x560603108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.104+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030fdc00 0x560603108920 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbc4809bf00 tx=0x7fbc480d02a0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.104+0000 7fbc5fc78740 1 -- 172.21.15.86:0/3750684325 shutdown_connections 2024-04-17T07:29:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.104+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560603175580 0x5606031779a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.104+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606030fdc00 0x560603108920 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.104+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606030f8a90 0x560603107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.104+0000 7fbc5fc78740 1 --2- 172.21.15.86:0/3750684325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606030f80a0 0x560603107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.104+0000 7fbc37e00640 0 client.28099 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:52.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.104+0000 7fbc5fc78740 1 -- 172.21.15.86:0/3750684325 >> 172.21.15.86:0/3750684325 conn(0x5606030f3610 msgr2=0x5606030f6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.105+0000 7fbc5fc78740 1 -- 172.21.15.86:0/3750684325 shutdown_connections 2024-04-17T07:29:52.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.106+0000 7fbc5fc78740 1 -- 172.21.15.86:0/3750684325 wait complete. 2024-04-17T07:29:52.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.107+0000 7fbc5fc78740 20 client.28099 trim_cache size 0 max 16384 2024-04-17T07:29:52.106 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:29:52.106 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:29:52.106 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "52e3e4e5-0288-448c-becf-de597b99724b", 2024-04-17T07:29:52.107 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:29:52.107 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:29:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.108+0000 7fbc57200640 1 -- 172.21.15.86:0/2651536988 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbc3c0a3d00 msgr2=0x7fbc3c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.108+0000 7fbc57200640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbc3c0a3d00 0x7fbc3c0a61c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005ab0 tx=0x7fbc44000bc0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.108+0000 7fbc57200640 1 -- 172.21.15.86:0/2651536988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5810a050 msgr2=0x7fbc581ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.108+0000 7fbc57200640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5810a050 0x7fbc581ae7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc4800d680 tx=0x7fbc4800db40 comp rx=0 tx=0).stop 2024-04-17T07:29:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.109+0000 7fbc57200640 1 -- 172.21.15.86:0/2651536988 shutdown_connections 2024-04-17T07:29:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.109+0000 7fbc57200640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbc3c0a3d00 0x7fbc3c0a61c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.109+0000 7fbc57200640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5810a050 0x7fbc581ae7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.109+0000 7fbc57200640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc58109690 0x7fbc5807fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.109+0000 7fbc57200640 1 --2- 172.21.15.86:0/2651536988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc581084a0 0x7fbc5807f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.109+0000 7fbc57200640 1 -- 172.21.15.86:0/2651536988 >> 172.21.15.86:0/2651536988 conn(0x7fbc58077430 msgr2=0x7fbc5810c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.109+0000 7fbc57200640 1 -- 172.21.15.86:0/2651536988 shutdown_connections 2024-04-17T07:29:52.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.109+0000 7fbc57200640 1 -- 172.21.15.86:0/2651536988 wait complete. 2024-04-17T07:29:52.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:29:52.148 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:29:52.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:52 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:29:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:52 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:52.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:52 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:29:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.525+0000 7f343f200640 1 -- 172.21.15.86:0/1969678407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3440102fe0 msgr2=0x7f3440103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.525+0000 7f343f200640 1 --2- 172.21.15.86:0/1969678407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3440102fe0 0x7f3440103460 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f342c009900 tx=0x7f342c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:29:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:52 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:29:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:52 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:52.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:52 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:29:52.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.529+0000 7f343f200640 1 -- 172.21.15.86:0/1969678407 shutdown_connections 2024-04-17T07:29:52.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.529+0000 7f343f200640 1 --2- 172.21.15.86:0/1969678407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34401039a0 0x7f344010a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.529+0000 7f343f200640 1 --2- 172.21.15.86:0/1969678407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3440102fe0 0x7f3440103460 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.529+0000 7f343f200640 1 --2- 172.21.15.86:0/1969678407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3440101e90 0x7f34401022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.529+0000 7f343f200640 1 -- 172.21.15.86:0/1969678407 >> 172.21.15.86:0/1969678407 conn(0x7f34400fd400 msgr2=0x7f34400ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.529+0000 7f343f200640 1 -- 172.21.15.86:0/1969678407 shutdown_connections 2024-04-17T07:29:52.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.530+0000 7f343f200640 1 -- 172.21.15.86:0/1969678407 wait complete. 2024-04-17T07:29:52.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.531+0000 7f343f200640 1 Processor -- start 2024-04-17T07:29:52.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.532+0000 7f343f200640 1 -- start start 2024-04-17T07:29:52.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.532+0000 7f343f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440101e90 0x7f344019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.532+0000 7f343f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3440102fe0 0x7f344019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.532+0000 7f343f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34401039a0 0x7f34401a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.532+0000 7f343f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f344010da20 con 0x7f3440101e90 2024-04-17T07:29:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.532+0000 7f343f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f344010d8a0 con 0x7f3440102fe0 2024-04-17T07:29:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.532+0000 7f343f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f344010dba0 con 0x7f34401039a0 2024-04-17T07:29:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.533+0000 7f343d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3440102fe0 0x7f344019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.533+0000 7f343de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440101e90 0x7f344019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.533+0000 7f343d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3440102fe0 0x7f344019ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45678/0 (socket says 172.21.15.86:45678) 2024-04-17T07:29:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.533+0000 7f343d400640 1 -- 172.21.15.86:0/2308585358 learned_addr learned my addr 172.21.15.86:0/2308585358 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.533+0000 7f343e800640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34401039a0 0x7f34401a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.534+0000 7f343d400640 1 -- 172.21.15.86:0/2308585358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34401039a0 msgr2=0x7f34401a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.534+0000 7f343d400640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34401039a0 0x7f34401a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.534+0000 7f343d400640 1 -- 172.21.15.86:0/2308585358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440101e90 msgr2=0x7f344019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.534+0000 7f343d400640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440101e90 0x7f344019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.534+0000 7f343d400640 1 -- 172.21.15.86:0/2308585358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34401a6330 con 0x7f3440102fe0 2024-04-17T07:29:52.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.534+0000 7f343d400640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3440102fe0 0x7f344019ec50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f342c000c00 tx=0x7f342c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.537+0000 7f3436a00640 1 -- 172.21.15.86:0/2308585358 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c043020 con 0x7f3440102fe0 2024-04-17T07:29:52.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.537+0000 7f3436a00640 1 -- 172.21.15.86:0/2308585358 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f342c0044c0 con 0x7f3440102fe0 2024-04-17T07:29:52.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.537+0000 7f3436a00640 1 -- 172.21.15.86:0/2308585358 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c003c20 con 0x7f3440102fe0 2024-04-17T07:29:52.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.537+0000 7f343f200640 1 -- 172.21.15.86:0/2308585358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34401a65c0 con 0x7f3440102fe0 2024-04-17T07:29:52.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.537+0000 7f343f200640 1 -- 172.21.15.86:0/2308585358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34401a6ac0 con 0x7f3440102fe0 2024-04-17T07:29:52.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.544+0000 7f3436a00640 1 -- 172.21.15.86:0/2308585358 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f342c046020 con 0x7f3440102fe0 2024-04-17T07:29:52.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.546+0000 7f3447966740 1 Processor -- start 2024-04-17T07:29:52.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.546+0000 7f3447966740 1 -- start start 2024-04-17T07:29:52.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.547+0000 7f3436a00640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34240a3d00 0x7f34240a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.547+0000 7f3447966740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4d370 0x561769f4d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.547+0000 7f3447966740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f52860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.547+0000 7f3447966740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f55360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.547+0000 7f3447966740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561769eb4120 con 0x561769f4d370 2024-04-17T07:29:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.547+0000 7f3447966740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561769e5a340 con 0x561769f4dd60 2024-04-17T07:29:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.547+0000 7f3447966740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561769eb00e0 con 0x561769f52ed0 2024-04-17T07:29:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.548+0000 7f3436a00640 1 -- 172.21.15.86:0/2308585358 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f342c0f2920 con 0x7f3440102fe0 2024-04-17T07:29:52.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.548+0000 7f343de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4d370 0x561769f4d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.548+0000 7f343d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f52860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.549+0000 7f343d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f52860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45682/0 (socket says 172.21.15.86:45682) 2024-04-17T07:29:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.549+0000 7f343d400640 1 -- 172.21.15.86:0/3192437207 learned_addr learned my addr 172.21.15.86:0/3192437207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.549+0000 7f343e800640 1 --2- 172.21.15.86:0/3192437207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f55360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.549+0000 7f343e800640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34240a3d00 0x7f34240a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.550+0000 7f343d400640 1 -- 172.21.15.86:0/3192437207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 msgr2=0x561769f55360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.550+0000 7f343d400640 1 --2- 172.21.15.86:0/3192437207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f55360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.550+0000 7f343d400640 1 -- 172.21.15.86:0/3192437207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4d370 msgr2=0x561769f4d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.550+0000 7f343d400640 1 --2- 172.21.15.86:0/3192437207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4d370 0x561769f4d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.550+0000 7f343d400640 1 -- 172.21.15.86:0/3192437207 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561769c907d0 con 0x561769f4dd60 2024-04-17T07:29:52.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.551+0000 7f343d400640 1 --2- 172.21.15.86:0/3192437207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f52860 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f342c0be580 tx=0x7f342c0be480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd1468cab9a5409 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.553+0000 7f343e800640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34240a3d00 0x7f34240a61c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3438009fd0 tx=0x7f34380098c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.554+0000 7f342b000640 1 -- 172.21.15.86:0/3192437207 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0d8000 con 0x561769f4dd60 2024-04-17T07:29:52.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.554+0000 7f342b000640 1 -- 172.21.15.86:0/3192437207 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f342c004e00 con 0x561769f4dd60 2024-04-17T07:29:52.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.554+0000 7f342b000640 1 -- 172.21.15.86:0/3192437207 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0bcb80 con 0x561769f4dd60 2024-04-17T07:29:52.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.555+0000 7f3447966740 1 -- 172.21.15.86:0/3192437207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 msgr2=0x561769f52860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.555+0000 7f3447966740 1 --2- 172.21.15.86:0/3192437207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f52860 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f342c0be580 tx=0x7f342c0be480 comp rx=0 tx=0).stop 2024-04-17T07:29:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.555+0000 7f3447966740 1 -- 172.21.15.86:0/3192437207 shutdown_connections 2024-04-17T07:29:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.556+0000 7f3447966740 1 --2- 172.21.15.86:0/3192437207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f55360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.556+0000 7f3447966740 1 --2- 172.21.15.86:0/3192437207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f52860 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.556+0000 7f3447966740 1 --2- 172.21.15.86:0/3192437207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4d370 0x561769f4d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.556+0000 7f3447966740 1 -- 172.21.15.86:0/3192437207 >> 172.21.15.86:0/3192437207 conn(0x561769f488e0 msgr2=0x561769f48ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.557+0000 7f3447966740 1 -- 172.21.15.86:0/3192437207 shutdown_connections 2024-04-17T07:29:52.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.557+0000 7f3447966740 1 -- 172.21.15.86:0/3192437207 wait complete. 2024-04-17T07:29:52.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.558+0000 7f3447966740 1 -- start start 2024-04-17T07:29:52.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.559+0000 7f3447966740 1 Processor -- start 2024-04-17T07:29:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.559+0000 7f3447966740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 0x561769f57660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.559+0000 7f3447966740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4dd60 0x561769f57ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.559+0000 7f3447966740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f5ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.560+0000 7f343de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 0x561769f57660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.560+0000 7f3447966740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561769f55c40 con 0x561769f4dd60 2024-04-17T07:29:52.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.560+0000 7f343d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4dd60 0x561769f57ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.560+0000 7f343d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4dd60 0x561769f57ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47604/0 (socket says 172.21.15.86:47604) 2024-04-17T07:29:52.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.560+0000 7f343d400640 1 -- 172.21.15.86:0/3274422522 learned_addr learned my addr 172.21.15.86:0/3274422522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:52.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.560+0000 7f3447966740 1 -- 172.21.15.86:0/3274422522 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561769e6d710 con 0x561769f4d370 2024-04-17T07:29:52.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.560+0000 7f343e800640 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f5ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.561+0000 7f3447966740 1 -- 172.21.15.86:0/3274422522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561769f55dc0 con 0x561769f52ed0 2024-04-17T07:29:52.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.561+0000 7f3447966740 10 client.0 _ll_register_callbacks cb 0x561769e68af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.562+0000 7f343d400640 1 -- 172.21.15.86:0/3274422522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 msgr2=0x561769f5ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.562+0000 7f343d400640 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f5ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.562+0000 7f343d400640 1 -- 172.21.15.86:0/3274422522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 msgr2=0x561769f57660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.562+0000 7f343d400640 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 0x561769f57660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.562+0000 7f343d400640 1 -- 172.21.15.86:0/3274422522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561769bc63f0 con 0x561769f4dd60 2024-04-17T07:29:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.563+0000 7f343d400640 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4dd60 0x561769f57ba0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f342c0f0410 tx=0x7f342c0d7200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae46c9ad19ac92f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.563+0000 7f341f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:29:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.564+0000 7f341f400640 1 -- 172.21.15.86:0/3274422522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0d8000 con 0x561769f4dd60 2024-04-17T07:29:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.564+0000 7f341f400640 1 -- 172.21.15.86:0/3274422522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f342c0d3320 con 0x561769f4dd60 2024-04-17T07:29:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.564+0000 7f3447966740 1 -- 172.21.15.86:0/3274422522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f342c004e00 con 0x561769f4dd60 2024-04-17T07:29:52.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.565+0000 7f341f400640 1 -- 172.21.15.86:0/3274422522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f342c0bef80 con 0x561769f4dd60 2024-04-17T07:29:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.566+0000 7f3447966740 10 client.19092 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:52.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.566+0000 7f3447966740 1 -- 172.21.15.86:0/3274422522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561769b860f0 con 0x561769f4dd60 2024-04-17T07:29:52.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.567+0000 7f341f400640 1 -- 172.21.15.86:0/3274422522 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f342c004e00 con 0x561769f4dd60 2024-04-17T07:29:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.567+0000 7f341f400640 1 -- 172.21.15.86:0/3274422522 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f342c0bfbb0 con 0x561769f4dd60 2024-04-17T07:29:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.567+0000 7f3447966740 10 client.19092 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.567+0000 7f3447966740 10 client.19092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.567+0000 7f3447966740 20 client.19092 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.567+0000 7f3447966740 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561769f632a0 0x561769f65690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.567+0000 7f3447966740 4 client.19092 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:29:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.567+0000 7f3447966740 1 -- 172.21.15.86:0/3274422522 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561769ec1b40 con 0x561769f632a0 2024-04-17T07:29:52.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.568+0000 7f343de00640 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561769f632a0 0x561769f65690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.573+0000 7f343de00640 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561769f632a0 0x561769f65690 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b04a1a20e0cd328f server_cookie=fc412cb4c4bb6035 in_seq=0 out_seq=0 2024-04-17T07:29:52.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.573+0000 7f341f400640 10 client.19092 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:52.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.575+0000 7f341f400640 1 -- 172.21.15.86:0/3274422522 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561769ec1b40 con 0x561769f632a0 2024-04-17T07:29:52.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.575+0000 7f341f400640 10 client.19092 handle_command_reply: tid=0 2024-04-17T07:29:52.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.575+0000 7f3447966740 1 client.19092 shutdown 2024-04-17T07:29:52.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.575+0000 7f3447966740 1 -- 172.21.15.86:0/3274422522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4dd60 msgr2=0x561769f57ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.575+0000 7f3447966740 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4dd60 0x561769f57ba0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f342c0f0410 tx=0x7f342c0d7200 comp rx=0 tx=0).stop 2024-04-17T07:29:52.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.577+0000 7f3447966740 1 -- 172.21.15.86:0/3274422522 shutdown_connections 2024-04-17T07:29:52.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.577+0000 7f3447966740 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561769f632a0 0x561769f65690 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.577+0000 7f3447966740 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f5ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.577+0000 7f3447966740 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4dd60 0x561769f57ba0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.577+0000 7f3447966740 1 --2- 172.21.15.86:0/3274422522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 0x561769f57660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.577+0000 7f3447966740 1 -- 172.21.15.86:0/3274422522 >> 172.21.15.86:0/3274422522 conn(0x561769f488e0 msgr2=0x561769f47ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.577+0000 7f341f400640 0 client.19092 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:52.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.577+0000 7f3447966740 1 -- 172.21.15.86:0/3274422522 shutdown_connections 2024-04-17T07:29:52.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.577+0000 7f3447966740 1 -- 172.21.15.86:0/3274422522 wait complete. 2024-04-17T07:29:52.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.578+0000 7f3447966740 20 client.19092 trim_cache size 0 max 16384 2024-04-17T07:29:52.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:52 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:29:52.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:52 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:52.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:52 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:29:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.617+0000 7f3447966740 1 Processor -- start 2024-04-17T07:29:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.617+0000 7f3447966740 1 -- start start 2024-04-17T07:29:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.618+0000 7f3447966740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4d370 0x561769f47250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.618+0000 7f3447966740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f5eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.618+0000 7f3447966740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f47790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f3447966740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34401b0ea0 con 0x561769f4d370 2024-04-17T07:29:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f3447966740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34401b29e0 con 0x561769f4dd60 2024-04-17T07:29:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f3447966740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34401b1570 con 0x561769f52ed0 2024-04-17T07:29:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f343d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f5eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f343d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f5eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45692/0 (socket says 172.21.15.86:45692) 2024-04-17T07:29:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f343d400640 1 -- 172.21.15.86:0/2539249211 learned_addr learned my addr 172.21.15.86:0/2539249211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f343d400640 1 -- 172.21.15.86:0/2539249211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 msgr2=0x561769f47790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f343d400640 1 --2- 172.21.15.86:0/2539249211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f47790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f343d400640 1 -- 172.21.15.86:0/2539249211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4d370 msgr2=0x561769f47250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f343d400640 1 --2- 172.21.15.86:0/2539249211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4d370 0x561769f47250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.619+0000 7f343d400640 1 -- 172.21.15.86:0/2539249211 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561769f4e140 con 0x561769f4dd60 2024-04-17T07:29:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.620+0000 7f343d400640 1 --2- 172.21.15.86:0/2539249211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f5eba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f342c0be810 tx=0x7f342c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b5d1d962f1005bb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.621+0000 7f342a600640 1 -- 172.21.15.86:0/2539249211 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0cb030 con 0x561769f4dd60 2024-04-17T07:29:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.621+0000 7f342a600640 1 -- 172.21.15.86:0/2539249211 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f342c004e00 con 0x561769f4dd60 2024-04-17T07:29:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.621+0000 7f342a600640 1 -- 172.21.15.86:0/2539249211 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0c7060 con 0x561769f4dd60 2024-04-17T07:29:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.622+0000 7f3447966740 1 -- 172.21.15.86:0/2539249211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 msgr2=0x561769f5eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.622+0000 7f3447966740 1 --2- 172.21.15.86:0/2539249211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f5eba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f342c0be810 tx=0x7f342c009260 comp rx=0 tx=0).stop 2024-04-17T07:29:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.624+0000 7f3447966740 1 -- 172.21.15.86:0/2539249211 shutdown_connections 2024-04-17T07:29:52.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.624+0000 7f3447966740 1 --2- 172.21.15.86:0/2539249211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f52ed0 0x561769f47790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.624+0000 7f3447966740 1 --2- 172.21.15.86:0/2539249211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4dd60 0x561769f5eba0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.624+0000 7f3447966740 1 --2- 172.21.15.86:0/2539249211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f4d370 0x561769f47250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.624+0000 7f3447966740 1 -- 172.21.15.86:0/2539249211 >> 172.21.15.86:0/2539249211 conn(0x561769f488e0 msgr2=0x561769f560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.624+0000 7f3447966740 1 -- 172.21.15.86:0/2539249211 shutdown_connections 2024-04-17T07:29:52.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.624+0000 7f3447966740 1 -- 172.21.15.86:0/2539249211 wait complete. 2024-04-17T07:29:52.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.625+0000 7f3447966740 1 -- start start 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.625+0000 7f3447966740 1 Processor -- start 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.626+0000 7f3447966740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 0x561769f5bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.626+0000 7f3447966740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f4dd60 0x561769f5c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.626+0000 7f3447966740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f52ed0 0x561769f5d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.627+0000 7f343de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 0x561769f5bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.627+0000 7f3447966740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34401b2310 con 0x561769f52ed0 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.627+0000 7f3447966740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34401b1c40 con 0x561769f4d370 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.627+0000 7f3447966740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34401aee70 con 0x561769f4dd60 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.627+0000 7f3447966740 10 client.0 _ll_register_callbacks cb 0x7f343000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.627+0000 7f343de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 0x561769f5bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:45696/0 (socket says 172.21.15.86:45696) 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.627+0000 7f343d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f4dd60 0x561769f5c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.627+0000 7f343e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f52ed0 0x561769f5d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.627+0000 7f343de00640 1 -- 172.21.15.86:0/1979768734 learned_addr learned my addr 172.21.15.86:0/1979768734 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:29:52.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.628+0000 7f343d400640 1 -- 172.21.15.86:0/1979768734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 msgr2=0x561769f5bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.628+0000 7f343d400640 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 0x561769f5bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.628+0000 7f343d400640 1 -- 172.21.15.86:0/1979768734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f52ed0 msgr2=0x561769f5d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:29:52.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.628+0000 7f343d400640 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f52ed0 0x561769f5d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.628+0000 7f343d400640 1 -- 172.21.15.86:0/1979768734 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561769f5d7d0 con 0x561769f4dd60 2024-04-17T07:29:52.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.628+0000 7f343de00640 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 0x561769f5bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-17T07:29:52.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.630+0000 7f341f400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:29:52.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.630+0000 7f343d400640 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f4dd60 0x561769f5c1d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x561769f4ea70 tx=0x7f342c003650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9bae6a3f40ff6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:29:52.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.631+0000 7f341f400640 1 -- 172.21.15.86:0/1979768734 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0cb070 con 0x561769f4dd60 2024-04-17T07:29:52.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.631+0000 7f341f400640 1 -- 172.21.15.86:0/1979768734 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f342c0d3320 con 0x561769f4dd60 2024-04-17T07:29:52.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.631+0000 7f3447966740 1 -- 172.21.15.86:0/1979768734 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f342c004e00 con 0x561769f4dd60 2024-04-17T07:29:52.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.632+0000 7f341f400640 1 -- 172.21.15.86:0/1979768734 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f342c0c74f0 con 0x561769f4dd60 2024-04-17T07:29:52.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.634+0000 7f3447966740 10 client.28111 fetch_fsmap learned FSMap version 28 2024-04-17T07:29:52.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.634+0000 7f3447966740 1 -- 172.21.15.86:0/1979768734 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561769f5d7d0 con 0x561769f4dd60 2024-04-17T07:29:52.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.635+0000 7f341f400640 1 -- 172.21.15.86:0/1979768734 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f342c004e00 con 0x561769f4dd60 2024-04-17T07:29:52.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.636+0000 7f341f400640 1 -- 172.21.15.86:0/1979768734 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f342c0bd740 con 0x561769f4dd60 2024-04-17T07:29:52.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.636+0000 7f3447966740 10 client.28111 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:29:52.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.636+0000 7f3447966740 10 client.28111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:29:52.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.636+0000 7f3447966740 20 client.28111 populate_metadata read hostname 'smithi086' 2024-04-17T07:29:52.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.636+0000 7f3447966740 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561769fda770 0x561769fdcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:29:52.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.636+0000 7f3447966740 4 client.28111 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:29:52.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.636+0000 7f3447966740 1 -- 172.21.15.86:0/1979768734 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561769f44140 con 0x561769fda770 2024-04-17T07:29:52.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.637+0000 7f343de00640 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561769fda770 0x561769fdcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:29:52.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.638+0000 7f343de00640 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561769fda770 0x561769fdcb60 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0890f8f0a9a306 server_cookie=c538936e75cf843b in_seq=0 out_seq=0 2024-04-17T07:29:52.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.638+0000 7f341f400640 10 client.28111 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:52.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.640+0000 7f341f400640 1 -- 172.21.15.86:0/1979768734 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561769f44140 con 0x561769fda770 2024-04-17T07:29:52.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.640+0000 7f341f400640 10 client.28111 handle_command_reply: tid=0 2024-04-17T07:29:52.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.640+0000 7f3447966740 1 client.28111 shutdown 2024-04-17T07:29:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.641+0000 7f3447966740 1 -- 172.21.15.86:0/1979768734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f4dd60 msgr2=0x561769f5c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.641+0000 7f3447966740 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f4dd60 0x561769f5c1d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x561769f4ea70 tx=0x7f342c003650 comp rx=0 tx=0).stop 2024-04-17T07:29:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.641+0000 7f3447966740 1 -- 172.21.15.86:0/1979768734 shutdown_connections 2024-04-17T07:29:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.641+0000 7f3447966740 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561769fda770 0x561769fdcb60 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.641+0000 7f3447966740 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561769f52ed0 0x561769f5d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.641+0000 7f3447966740 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561769f4dd60 0x561769f5c1d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.641+0000 7f3447966740 1 --2- 172.21.15.86:0/1979768734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561769f4d370 0x561769f5bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.641+0000 7f3447966740 1 -- 172.21.15.86:0/1979768734 >> 172.21.15.86:0/1979768734 conn(0x561769f488e0 msgr2=0x561769f4bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.642+0000 7f341f400640 0 client.28111 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:29:52.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.642+0000 7f3447966740 1 -- 172.21.15.86:0/1979768734 shutdown_connections 2024-04-17T07:29:52.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.642+0000 7f3447966740 1 -- 172.21.15.86:0/1979768734 wait complete. 2024-04-17T07:29:52.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.643+0000 7f3447966740 20 client.28111 trim_cache size 0 max 16384 2024-04-17T07:29:52.644 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:29:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:29:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:29:52.644 INFO:teuthology.orchestra.run.smithi086.stdout: "52e3e4e5-0288-448c-becf-de597b99724b": { 2024-04-17T07:29:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:29:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "52e3e4e5-0288-448c-becf-de597b99724b", 2024-04-17T07:29:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:29:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:29:52.645 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:29:52.645 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.644+0000 7f343f200640 1 -- 172.21.15.86:0/2308585358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34240a3d00 msgr2=0x7f34240a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.644+0000 7f343f200640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34240a3d00 0x7f34240a61c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3438009fd0 tx=0x7f34380098c0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.644+0000 7f343f200640 1 -- 172.21.15.86:0/2308585358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3440102fe0 msgr2=0x7f344019ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.644+0000 7f343f200640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3440102fe0 0x7f344019ec50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f342c000c00 tx=0x7f342c0040c0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.645+0000 7f343f200640 1 -- 172.21.15.86:0/2308585358 shutdown_connections 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.645+0000 7f343f200640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f34240a3d00 0x7f34240a61c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.645+0000 7f343f200640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34401039a0 0x7f34401a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.645+0000 7f343f200640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3440102fe0 0x7f344019ec50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.645+0000 7f343f200640 1 --2- 172.21.15.86:0/2308585358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440101e90 0x7f344019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.645+0000 7f343f200640 1 -- 172.21.15.86:0/2308585358 >> 172.21.15.86:0/2308585358 conn(0x7f34400fd400 msgr2=0x7f3440104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:29:52.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.646+0000 7f343f200640 1 -- 172.21.15.86:0/2308585358 shutdown_connections 2024-04-17T07:29:52.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:29:52.646+0000 7f343f200640 1 -- 172.21.15.86:0/2308585358 wait complete. 2024-04-17T07:29:52.672 INFO:tasks.cephfs.filesystem:scrub status for tag:52e3e4e5-0288-448c-becf-de597b99724b - {'path': '/', 'tag': '52e3e4e5-0288-448c-becf-de597b99724b', 'options': 'recursive,force'} 2024-04-17T07:29:53.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:53 smithi098 ceph-mon[35418]: pgmap v905: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-04-17T07:29:53.506 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:53 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:53.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:53 smithi103 ceph-mon[35495]: pgmap v905: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-04-17T07:29:53.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:53 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:53.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:53 smithi086 ceph-mon[31733]: pgmap v905: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-04-17T07:29:53.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:53 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:29:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 630.0 s, 487.8 tps, lat 204.512 ms stddev 91.862 2024-04-17T07:29:54.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:54 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:29:54.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:54 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:29:54.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:54 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:29:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:55 smithi098 ceph-mon[35418]: pgmap v906: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.0 MiB/s wr, 606 op/s 2024-04-17T07:29:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:55 smithi103 ceph-mon[35495]: pgmap v906: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.0 MiB/s wr, 606 op/s 2024-04-17T07:29:55.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:55 smithi086 ceph-mon[31733]: pgmap v906: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.0 MiB/s wr, 606 op/s 2024-04-17T07:29:57.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:57 smithi103 ceph-mon[35495]: pgmap v907: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 609 op/s 2024-04-17T07:29:57.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:57 smithi086 ceph-mon[31733]: pgmap v907: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 609 op/s 2024-04-17T07:29:57.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:57 smithi098 ceph-mon[35418]: pgmap v907: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 609 op/s 2024-04-17T07:29:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 635.0 s, 501.2 tps, lat 199.772 ms stddev 223.813 2024-04-17T07:29:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:29:59 smithi103 ceph-mon[35495]: pgmap v908: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 674 op/s 2024-04-17T07:29:59.555 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:29:59 smithi098 ceph-mon[35418]: pgmap v908: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 674 op/s 2024-04-17T07:29:59.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:29:59 smithi086 ceph-mon[31733]: pgmap v908: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 674 op/s 2024-04-17T07:30:00.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:00 smithi086 ceph-mon[31733]: overall HEALTH_OK 2024-04-17T07:30:00.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:00 smithi098 ceph-mon[35418]: overall HEALTH_OK 2024-04-17T07:30:00.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:00 smithi103 ceph-mon[35495]: overall HEALTH_OK 2024-04-17T07:30:01.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:01 smithi086 ceph-mon[31733]: pgmap v909: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.7 MiB/s wr, 669 op/s 2024-04-17T07:30:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:01 smithi098 ceph-mon[35418]: pgmap v909: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.7 MiB/s wr, 669 op/s 2024-04-17T07:30:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:01 smithi103 ceph-mon[35495]: pgmap v909: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.7 MiB/s wr, 669 op/s 2024-04-17T07:30:03.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:03 smithi086 ceph-mon[31733]: pgmap v910: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 707 op/s 2024-04-17T07:30:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:03 smithi098 ceph-mon[35418]: pgmap v910: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 707 op/s 2024-04-17T07:30:03.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:03 smithi103 ceph-mon[35495]: pgmap v910: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 707 op/s 2024-04-17T07:30:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 640.0 s, 567.2 tps, lat 177.726 ms stddev 86.845 2024-04-17T07:30:05.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:05 smithi086 ceph-mon[31733]: pgmap v911: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.9 MiB/s wr, 686 op/s 2024-04-17T07:30:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:05 smithi098 ceph-mon[35418]: pgmap v911: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.9 MiB/s wr, 686 op/s 2024-04-17T07:30:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:05 smithi103 ceph-mon[35495]: pgmap v911: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.9 MiB/s wr, 686 op/s 2024-04-17T07:30:07.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:07 smithi086 ceph-mon[31733]: pgmap v912: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 669 op/s 2024-04-17T07:30:07.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:07 smithi098 ceph-mon[35418]: pgmap v912: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 669 op/s 2024-04-17T07:30:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:07 smithi103 ceph-mon[35495]: pgmap v912: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 MiB/s wr, 669 op/s 2024-04-17T07:30:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 645.0 s, 498.8 tps, lat 192.068 ms stddev 125.615 2024-04-17T07:30:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:09 smithi086 ceph-mon[31733]: pgmap v913: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 726 op/s 2024-04-17T07:30:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:09 smithi098 ceph-mon[35418]: pgmap v913: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 726 op/s 2024-04-17T07:30:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:09 smithi103 ceph-mon[35495]: pgmap v913: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 726 op/s 2024-04-17T07:30:11.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:11 smithi098 ceph-mon[35418]: pgmap v914: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 657 op/s 2024-04-17T07:30:11.779 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:11 smithi103 ceph-mon[35495]: pgmap v914: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 657 op/s 2024-04-17T07:30:11.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:11 smithi086 ceph-mon[31733]: pgmap v914: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 657 op/s 2024-04-17T07:30:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:13 smithi098 ceph-mon[35418]: pgmap v915: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-04-17T07:30:13.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:13 smithi103 ceph-mon[35495]: pgmap v915: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-04-17T07:30:13.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:13 smithi086 ceph-mon[31733]: pgmap v915: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-04-17T07:30:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 650.0 s, 396.4 tps, lat 262.577 ms stddev 264.715 2024-04-17T07:30:15.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:15 smithi098 ceph-mon[35418]: pgmap v916: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.4 MiB/s wr, 655 op/s 2024-04-17T07:30:15.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:15 smithi103 ceph-mon[35495]: pgmap v916: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.4 MiB/s wr, 655 op/s 2024-04-17T07:30:15.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:15 smithi086 ceph-mon[31733]: pgmap v916: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.4 MiB/s wr, 655 op/s 2024-04-17T07:30:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:17 smithi098 ceph-mon[35418]: pgmap v917: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 647 op/s 2024-04-17T07:30:17.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:17 smithi103 ceph-mon[35495]: pgmap v917: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 647 op/s 2024-04-17T07:30:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:17 smithi086 ceph-mon[31733]: pgmap v917: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 647 op/s 2024-04-17T07:30:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 655.0 s, 598.8 tps, lat 165.780 ms stddev 51.001 2024-04-17T07:30:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:19 smithi098 ceph-mon[35418]: pgmap v918: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.9 MiB/s wr, 682 op/s 2024-04-17T07:30:19.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:19 smithi103 ceph-mon[35495]: pgmap v918: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.9 MiB/s wr, 682 op/s 2024-04-17T07:30:19.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:19 smithi086 ceph-mon[31733]: pgmap v918: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.9 MiB/s wr, 682 op/s 2024-04-17T07:30:21.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:21 smithi098 ceph-mon[35418]: pgmap v919: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.3 MiB/s wr, 630 op/s 2024-04-17T07:30:21.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:21 smithi103 ceph-mon[35495]: pgmap v919: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.3 MiB/s wr, 630 op/s 2024-04-17T07:30:21.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:21 smithi086 ceph-mon[31733]: pgmap v919: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.3 MiB/s wr, 630 op/s 2024-04-17T07:30:22.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:30:22.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.929+0000 7f126ec00640 1 -- 172.21.15.86:0/3890383066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126810e510 msgr2=0x7f12681109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.929+0000 7f126ec00640 1 --2- 172.21.15.86:0/3890383066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126810e510 0x7f12681109d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1260005fd0 tx=0x7f126002cc30 comp rx=0 tx=0).stop 2024-04-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.930+0000 7f126ec00640 1 -- 172.21.15.86:0/3890383066 shutdown_connections 2024-04-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.930+0000 7f126ec00640 1 --2- 172.21.15.86:0/3890383066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126810e510 0x7f12681109d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.930+0000 7f126ec00640 1 --2- 172.21.15.86:0/3890383066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f126810ba70 0x7f126810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.930+0000 7f126ec00640 1 --2- 172.21.15.86:0/3890383066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1268109140 0x7f126810b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.930+0000 7f126ec00640 1 -- 172.21.15.86:0/3890383066 >> 172.21.15.86:0/3890383066 conn(0x7f126806f060 msgr2=0x7f1268071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.931+0000 7f126ec00640 1 -- 172.21.15.86:0/3890383066 shutdown_connections 2024-04-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.931+0000 7f126ec00640 1 -- 172.21.15.86:0/3890383066 wait complete. 2024-04-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.934+0000 7f126ec00640 1 Processor -- start 2024-04-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.934+0000 7f126ec00640 1 -- start start 2024-04-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.934+0000 7f126ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268109140 0x7f12681a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.935+0000 7f126ec00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126810ba70 0x7f12681a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.935+0000 7f126ec00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126810e510 0x7f12681ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.935+0000 7f126ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1268113a70 con 0x7f1268109140 2024-04-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.935+0000 7f126ec00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f12681138f0 con 0x7f126810e510 2024-04-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.935+0000 7f126ec00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1268113bf0 con 0x7f126810ba70 2024-04-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.935+0000 7f126ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126810ba70 0x7f12681a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.935+0000 7f126ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126810ba70 0x7f12681a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33576/0 (socket says 172.21.15.86:33576) 2024-04-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.935+0000 7f126ce00640 1 -- 172.21.15.86:0/4203368674 learned_addr learned my addr 172.21.15.86:0/4203368674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.935+0000 7f126d800640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268109140 0x7f12681a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.935+0000 7f126e200640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126810e510 0x7f12681ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.936+0000 7f126e200640 1 -- 172.21.15.86:0/4203368674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126810ba70 msgr2=0x7f12681a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.936+0000 7f126e200640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126810ba70 0x7f12681a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.936+0000 7f126e200640 1 -- 172.21.15.86:0/4203368674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268109140 msgr2=0x7f12681a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.936+0000 7f126e200640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268109140 0x7f12681a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.936+0000 7f126e200640 1 -- 172.21.15.86:0/4203368674 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1268119d60 con 0x7f126810e510 2024-04-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.937+0000 7f126e200640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126810e510 0x7f12681ae760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f126002d170 tx=0x7f1260007a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.937+0000 7f1266000640 1 -- 172.21.15.86:0/4203368674 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260044070 con 0x7f126810e510 2024-04-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.937+0000 7f1266000640 1 -- 172.21.15.86:0/4203368674 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1260007e50 con 0x7f126810e510 2024-04-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.937+0000 7f1266000640 1 -- 172.21.15.86:0/4203368674 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260004630 con 0x7f126810e510 2024-04-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.940+0000 7f126ec00640 1 -- 172.21.15.86:0/4203368674 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12681a3f40 con 0x7f126810e510 2024-04-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.940+0000 7f126ec00640 1 -- 172.21.15.86:0/4203368674 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12681a4410 con 0x7f126810e510 2024-04-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.945+0000 7f1266000640 1 -- 172.21.15.86:0/4203368674 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1260047070 con 0x7f126810e510 2024-04-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.946+0000 7f127301c740 1 Processor -- start 2024-04-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.946+0000 7f127301c740 1 -- start start 2024-04-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.947+0000 7f127301c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516ae5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.947+0000 7f127301c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aea860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.947+0000 7f127301c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516aed360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.947+0000 7f127301c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a516a4c120 con 0x55a516ae5d60 2024-04-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.947+0000 7f127301c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a5169f2340 con 0x55a516aeaed0 2024-04-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.947+0000 7f127301c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a516a480e0 con 0x55a516ae5370 2024-04-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f126d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f126d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aea860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48674/0 (socket says 172.21.15.86:48674) 2024-04-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f126d800640 1 -- 172.21.15.86:0/3436122863 learned_addr learned my addr 172.21.15.86:0/3436122863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f1266000640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f12500a3cb0 0x7f12500a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f1266000640 1 -- 172.21.15.86:0/4203368674 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f12600f1ba0 con 0x7f126810e510 2024-04-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f126ce00640 1 --2- 172.21.15.86:0/3436122863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516aed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f126d800640 1 -- 172.21.15.86:0/3436122863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 msgr2=0x55a516ae5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f126ce00640 1 --2- 172.21.15.86:0/3436122863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516ae5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f126d800640 1 --2- 172.21.15.86:0/3436122863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516ae5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f126d800640 1 -- 172.21.15.86:0/3436122863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 msgr2=0x55a516aed360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.948+0000 7f126d800640 1 --2- 172.21.15.86:0/3436122863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516aed360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.949+0000 7f126d800640 1 -- 172.21.15.86:0/3436122863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5168287d0 con 0x55a516ae5d60 2024-04-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.949+0000 7f126ce00640 1 --2- 172.21.15.86:0/3436122863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516ae5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.951+0000 7f126d800640 1 --2- 172.21.15.86:0/3436122863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aea860 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b740 tx=0x7f125c00bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324eb293d02211af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.951+0000 7f126e200640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f12500a3cb0 0x7f12500a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.951+0000 7f125aa00640 1 -- 172.21.15.86:0/3436122863 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c003dd0 con 0x55a516ae5d60 2024-04-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.951+0000 7f125aa00640 1 -- 172.21.15.86:0/3436122863 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f125c015070 con 0x55a516ae5d60 2024-04-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.951+0000 7f125aa00640 1 -- 172.21.15.86:0/3436122863 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c004e40 con 0x55a516ae5d60 2024-04-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.952+0000 7f127301c740 1 -- 172.21.15.86:0/3436122863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 msgr2=0x55a516aea860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.952+0000 7f126e200640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f12500a3cb0 0x7f12500a6170 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f12600ba520 tx=0x7f126000aec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.952+0000 7f127301c740 1 --2- 172.21.15.86:0/3436122863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aea860 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b740 tx=0x7f125c00bc00 comp rx=0 tx=0).stop 2024-04-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.952+0000 7f127301c740 1 -- 172.21.15.86:0/3436122863 shutdown_connections 2024-04-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.953+0000 7f127301c740 1 --2- 172.21.15.86:0/3436122863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516aed360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.953+0000 7f127301c740 1 --2- 172.21.15.86:0/3436122863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aea860 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.953+0000 7f127301c740 1 --2- 172.21.15.86:0/3436122863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516ae5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.953+0000 7f127301c740 1 -- 172.21.15.86:0/3436122863 >> 172.21.15.86:0/3436122863 conn(0x55a516ae08e0 msgr2=0x55a516ae0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.953+0000 7f127301c740 1 -- 172.21.15.86:0/3436122863 shutdown_connections 2024-04-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.953+0000 7f127301c740 1 -- 172.21.15.86:0/3436122863 wait complete. 2024-04-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.953+0000 7f127301c740 1 -- start start 2024-04-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.954+0000 7f127301c740 1 Processor -- start 2024-04-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.954+0000 7f127301c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516aef930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.954+0000 7f127301c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aefe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.954+0000 7f127301c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516af6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.954+0000 7f127301c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a516aeded0 con 0x55a516ae5d60 2024-04-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.955+0000 7f127301c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a516a05710 con 0x55a516aeaed0 2024-04-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.955+0000 7f127301c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a516aee050 con 0x55a516ae5370 2024-04-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.955+0000 7f127301c740 10 client.0 _ll_register_callbacks cb 0x55a516a00af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.955+0000 7f126d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aefe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.955+0000 7f126d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aefe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48682/0 (socket says 172.21.15.86:48682) 2024-04-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.955+0000 7f126d800640 1 -- 172.21.15.86:0/1475700248 learned_addr learned my addr 172.21.15.86:0/1475700248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.955+0000 7f126ce00640 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516aef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.956+0000 7f126ce00640 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516af6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.956+0000 7f126d800640 1 -- 172.21.15.86:0/1475700248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 msgr2=0x55a516aef930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.956+0000 7f126d800640 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516aef930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.956+0000 7f126d800640 1 -- 172.21.15.86:0/1475700248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 msgr2=0x55a516af6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.956+0000 7f126d800640 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516af6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.956+0000 7f126d800640 1 -- 172.21.15.86:0/1475700248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51675e3f0 con 0x55a516ae5d60 2024-04-17T07:30:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.956+0000 7f126d800640 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aefe70 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b710 tx=0x7f125c003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8104f1ecd372a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.956+0000 7f124f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:30:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.957+0000 7f124f400640 1 -- 172.21.15.86:0/1475700248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c028060 con 0x55a516ae5d60 2024-04-17T07:30:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.957+0000 7f127301c740 1 -- 172.21.15.86:0/1475700248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125c015070 con 0x55a516ae5d60 2024-04-17T07:30:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.958+0000 7f124f400640 1 -- 172.21.15.86:0/1475700248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f125c011850 con 0x55a516ae5d60 2024-04-17T07:30:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.958+0000 7f124f400640 1 -- 172.21.15.86:0/1475700248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f125c011a50 con 0x55a516ae5d60 2024-04-17T07:30:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.958+0000 7f127301c740 10 client.19104 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.958+0000 7f127301c740 1 -- 172.21.15.86:0/1475700248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a51671e0f0 con 0x55a516ae5d60 2024-04-17T07:30:22.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.959+0000 7f124f400640 1 -- 172.21.15.86:0/1475700248 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f125c015070 con 0x55a516ae5d60 2024-04-17T07:30:22.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.959+0000 7f124f400640 1 -- 172.21.15.86:0/1475700248 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f125c022070 con 0x55a516ae5d60 2024-04-17T07:30:22.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.960+0000 7f127301c740 10 client.19104 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:22.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.960+0000 7f127301c740 10 client.19104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.960+0000 7f127301c740 20 client.19104 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.960+0000 7f127301c740 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a516afb360 0x55a516afd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.961+0000 7f127301c740 4 client.19104 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.961+0000 7f127301c740 1 -- 172.21.15.86:0/1475700248 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a516a59b40 con 0x55a516afb360 2024-04-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.961+0000 7f126ce00640 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a516afb360 0x55a516afd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.962+0000 7f126ce00640 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a516afb360 0x55a516afd750 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80dfe8b3d4cf6e1f server_cookie=7c9c86b09e2d4a3b in_seq=0 out_seq=0 2024-04-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.962+0000 7f124f400640 10 client.19104 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.964+0000 7f124f400640 1 -- 172.21.15.86:0/1475700248 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a516a59b40 con 0x55a516afb360 2024-04-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.964+0000 7f124f400640 10 client.19104 handle_command_reply: tid=0 2024-04-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.964+0000 7f127301c740 1 client.19104 shutdown 2024-04-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.965+0000 7f127301c740 1 -- 172.21.15.86:0/1475700248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 msgr2=0x55a516aefe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.965+0000 7f127301c740 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aefe70 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b710 tx=0x7f125c003e00 comp rx=0 tx=0).stop 2024-04-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.965+0000 7f127301c740 1 -- 172.21.15.86:0/1475700248 shutdown_connections 2024-04-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.965+0000 7f127301c740 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a516afb360 0x55a516afd750 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.965+0000 7f127301c740 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516af6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.965+0000 7f127301c740 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516ae5d60 0x55a516aefe70 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.965+0000 7f127301c740 1 --2- 172.21.15.86:0/1475700248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516aef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.965+0000 7f127301c740 1 -- 172.21.15.86:0/1475700248 >> 172.21.15.86:0/1475700248 conn(0x55a516ae08e0 msgr2=0x55a516adfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.965+0000 7f124f400640 0 client.19104 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.966+0000 7f127301c740 1 -- 172.21.15.86:0/1475700248 shutdown_connections 2024-04-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.966+0000 7f127301c740 1 -- 172.21.15.86:0/1475700248 wait complete. 2024-04-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:22.966+0000 7f127301c740 20 client.19104 trim_cache size 0 max 16384 2024-04-17T07:30:23.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.000+0000 7f127301c740 1 Processor -- start 2024-04-17T07:30:23.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.000+0000 7f127301c740 1 -- start start 2024-04-17T07:30:23.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.001+0000 7f127301c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516aeef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.001+0000 7f127301c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516aeaed0 0x55a516af6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.001+0000 7f127301c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeeb30 0x55a516ae7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.001+0000 7f127301c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12681c1f00 con 0x55a516aeaed0 2024-04-17T07:30:23.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.001+0000 7f127301c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f12681c3a40 con 0x55a516aeeb30 2024-04-17T07:30:23.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.001+0000 7f127301c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12681c25d0 con 0x55a516ae5370 2024-04-17T07:30:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.001+0000 7f126ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516aeef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.001+0000 7f126ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516aeef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33610/0 (socket says 172.21.15.86:33610) 2024-04-17T07:30:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.002+0000 7f126d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516aeaed0 0x55a516af6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.002+0000 7f126ce00640 1 -- 172.21.15.86:0/102522208 learned_addr learned my addr 172.21.15.86:0/102522208 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.002+0000 7f126ce00640 1 --2- 172.21.15.86:0/102522208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeeb30 0x55a516ae7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.002+0000 7f126d800640 1 -- 172.21.15.86:0/102522208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 msgr2=0x55a516aeef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.002+0000 7f126d800640 1 --2- 172.21.15.86:0/102522208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516aeef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.002+0000 7f126d800640 1 -- 172.21.15.86:0/102522208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeeb30 msgr2=0x55a516ae7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.002+0000 7f126d800640 1 --2- 172.21.15.86:0/102522208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeeb30 0x55a516ae7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.002+0000 7f126d800640 1 -- 172.21.15.86:0/102522208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a516ae6140 con 0x55a516aeaed0 2024-04-17T07:30:23.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.003+0000 7f126ce00640 1 --2- 172.21.15.86:0/102522208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeeb30 0x55a516ae7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:30:23.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.003+0000 7f126d800640 1 --2- 172.21.15.86:0/102522208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516aeaed0 0x55a516af6e70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f125c013000 tx=0x7f125c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a499f7fe5e749ad server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:23.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.005+0000 7f125a000640 1 -- 172.21.15.86:0/102522208 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c000e60 con 0x55a516aeaed0 2024-04-17T07:30:23.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.005+0000 7f125a000640 1 -- 172.21.15.86:0/102522208 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f125c015040 con 0x55a516aeaed0 2024-04-17T07:30:23.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.005+0000 7f125a000640 1 -- 172.21.15.86:0/102522208 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c011a50 con 0x55a516aeaed0 2024-04-17T07:30:23.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.006+0000 7f127301c740 1 -- 172.21.15.86:0/102522208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516aeaed0 msgr2=0x55a516af6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:23.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.006+0000 7f127301c740 1 --2- 172.21.15.86:0/102522208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516aeaed0 0x55a516af6e70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f125c013000 tx=0x7f125c005f30 comp rx=0 tx=0).stop 2024-04-17T07:30:23.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.006+0000 7f127301c740 1 -- 172.21.15.86:0/102522208 shutdown_connections 2024-04-17T07:30:23.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.006+0000 7f127301c740 1 --2- 172.21.15.86:0/102522208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeeb30 0x55a516ae7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.006+0000 7f127301c740 1 --2- 172.21.15.86:0/102522208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516aeaed0 0x55a516af6e70 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.006+0000 7f127301c740 1 --2- 172.21.15.86:0/102522208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516aeef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.006+0000 7f127301c740 1 -- 172.21.15.86:0/102522208 >> 172.21.15.86:0/102522208 conn(0x55a516ae08e0 msgr2=0x55a516aee420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:23.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.007+0000 7f127301c740 1 -- 172.21.15.86:0/102522208 shutdown_connections 2024-04-17T07:30:23.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.007+0000 7f127301c740 1 -- 172.21.15.86:0/102522208 wait complete. 2024-04-17T07:30:23.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.007+0000 7f127301c740 1 -- start start 2024-04-17T07:30:23.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.008+0000 7f127301c740 1 Processor -- start 2024-04-17T07:30:23.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.008+0000 7f127301c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516af44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.008+0000 7f127301c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516af4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.008+0000 7f127301c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516af5950 0x55a516af5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.008+0000 7f127301c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12681c3370 con 0x55a516af5950 2024-04-17T07:30:23.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.008+0000 7f127301c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f12681c2ca0 con 0x55a516aeaed0 2024-04-17T07:30:23.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.008+0000 7f127301c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12681abff0 con 0x55a516ae5370 2024-04-17T07:30:23.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.009+0000 7f127301c740 10 client.0 _ll_register_callbacks cb 0x7f125c004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.010+0000 7f126d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516af4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.010+0000 7f126d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516af4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37492/0 (socket says 172.21.15.86:37492) 2024-04-17T07:30:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.010+0000 7f126d800640 1 -- 172.21.15.86:0/541983843 learned_addr learned my addr 172.21.15.86:0/541983843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.010+0000 7f126ce00640 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516af44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.010+0000 7f126ce00640 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516af5950 0x55a516af5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.011+0000 7f126d800640 1 -- 172.21.15.86:0/541983843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 msgr2=0x55a516af44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.011+0000 7f126d800640 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516af44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.011+0000 7f126d800640 1 -- 172.21.15.86:0/541983843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516af5950 msgr2=0x55a516af5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.011+0000 7f126d800640 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516af5950 0x55a516af5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.011+0000 7f126d800640 1 -- 172.21.15.86:0/541983843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a516af6310 con 0x55a516aeaed0 2024-04-17T07:30:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.012+0000 7f126d800640 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516af4a00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b740 tx=0x7f125c004210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc1ed262c6cc3f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.012+0000 7f124f400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:30:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.013+0000 7f124f400640 1 -- 172.21.15.86:0/541983843 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c011a50 con 0x55a516aeaed0 2024-04-17T07:30:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.013+0000 7f124f400640 1 -- 172.21.15.86:0/541983843 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f125c019610 con 0x55a516aeaed0 2024-04-17T07:30:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.013+0000 7f127301c740 1 -- 172.21.15.86:0/541983843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f125c015040 con 0x55a516aeaed0 2024-04-17T07:30:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.014+0000 7f124f400640 1 -- 172.21.15.86:0/541983843 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f125c019850 con 0x55a516aeaed0 2024-04-17T07:30:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.014+0000 7f127301c740 10 client.38084 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.014+0000 7f127301c740 1 -- 172.21.15.86:0/541983843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a516b619e0 con 0x55a516aeaed0 2024-04-17T07:30:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.014+0000 7f124f400640 1 -- 172.21.15.86:0/541983843 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f125c015040 con 0x55a516aeaed0 2024-04-17T07:30:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.015+0000 7f124f400640 1 -- 172.21.15.86:0/541983843 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f125c022040 con 0x55a516aeaed0 2024-04-17T07:30:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.015+0000 7f127301c740 10 client.38084 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.015+0000 7f127301c740 10 client.38084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:30:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.015+0000 7f127301c740 20 client.38084 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.016+0000 7f127301c740 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a516b62310 0x55a516b64730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.016+0000 7f127301c740 4 client.38084 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:30:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.016+0000 7f127301c740 1 -- 172.21.15.86:0/541983843 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a516aef720 con 0x55a516b62310 2024-04-17T07:30:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.016+0000 7f126ce00640 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a516b62310 0x55a516b64730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.018+0000 7f126ce00640 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a516b62310 0x55a516b64730 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ce7add2e8eb974b server_cookie=c3a77d36c3806e6f in_seq=0 out_seq=0 2024-04-17T07:30:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.018+0000 7f124f400640 10 client.38084 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f124f400640 1 -- 172.21.15.86:0/541983843 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a516aef720 con 0x55a516b62310 2024-04-17T07:30:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f124f400640 10 client.38084 handle_command_reply: tid=0 2024-04-17T07:30:23.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f127301c740 1 client.38084 shutdown 2024-04-17T07:30:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f127301c740 1 -- 172.21.15.86:0/541983843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 msgr2=0x55a516af4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f127301c740 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516af4a00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f125c00b740 tx=0x7f125c004210 comp rx=0 tx=0).stop 2024-04-17T07:30:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f127301c740 1 -- 172.21.15.86:0/541983843 shutdown_connections 2024-04-17T07:30:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f127301c740 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a516b62310 0x55a516b64730 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f127301c740 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a516af5950 0x55a516af5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f127301c740 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a516aeaed0 0x55a516af4a00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f127301c740 1 --2- 172.21.15.86:0/541983843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a516ae5370 0x55a516af44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f127301c740 1 -- 172.21.15.86:0/541983843 >> 172.21.15.86:0/541983843 conn(0x55a516ae08e0 msgr2=0x55a516afd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:23.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.019+0000 7f124f400640 0 client.38084 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.020+0000 7f127301c740 1 -- 172.21.15.86:0/541983843 shutdown_connections 2024-04-17T07:30:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.020+0000 7f127301c740 1 -- 172.21.15.86:0/541983843 wait complete. 2024-04-17T07:30:23.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.020+0000 7f127301c740 20 client.38084 trim_cache size 0 max 16384 2024-04-17T07:30:23.021 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:30:23.021 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:30:23.021 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:30:23.022 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:30:23.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.021+0000 7f126ec00640 1 -- 172.21.15.86:0/4203368674 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f12500a3cb0 msgr2=0x7f12500a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:23.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.021+0000 7f126ec00640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f12500a3cb0 0x7f12500a6170 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f12600ba520 tx=0x7f126000aec0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.021+0000 7f126ec00640 1 -- 172.21.15.86:0/4203368674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126810e510 msgr2=0x7f12681ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:23.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.021+0000 7f126ec00640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126810e510 0x7f12681ae760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f126002d170 tx=0x7f1260007a50 comp rx=0 tx=0).stop 2024-04-17T07:30:23.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.021+0000 7f126ec00640 1 -- 172.21.15.86:0/4203368674 shutdown_connections 2024-04-17T07:30:23.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.022+0000 7f126ec00640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f12500a3cb0 0x7f12500a6170 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.022+0000 7f126ec00640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126810e510 0x7f12681ae760 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.022+0000 7f126ec00640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f126810ba70 0x7f12681a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.022+0000 7f126ec00640 1 --2- 172.21.15.86:0/4203368674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1268109140 0x7f12681a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.022+0000 7f126ec00640 1 -- 172.21.15.86:0/4203368674 >> 172.21.15.86:0/4203368674 conn(0x7f126806f060 msgr2=0x7f126810a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:23.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.022+0000 7f126ec00640 1 -- 172.21.15.86:0/4203368674 shutdown_connections 2024-04-17T07:30:23.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.023+0000 7f126ec00640 1 -- 172.21.15.86:0/4203368674 wait complete. 2024-04-17T07:30:23.058 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:30:23.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:30:23.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.390+0000 7fb51f800640 1 -- 172.21.15.86:0/3358422465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181013d0 msgr2=0x7fb518101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:23.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.390+0000 7fb51f800640 1 --2- 172.21.15.86:0/3358422465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181013d0 0x7fb518101850 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb5040074e0 tx=0x7fb50402c8b0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.391+0000 7fb51f800640 1 -- 172.21.15.86:0/3358422465 shutdown_connections 2024-04-17T07:30:23.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.391+0000 7fb51f800640 1 --2- 172.21.15.86:0/3358422465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb518101d90 0x7fb518108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.391+0000 7fb51f800640 1 --2- 172.21.15.86:0/3358422465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181013d0 0x7fb518101850 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.391+0000 7fb51f800640 1 --2- 172.21.15.86:0/3358422465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5181001e0 0x7fb518100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.391+0000 7fb51f800640 1 -- 172.21.15.86:0/3358422465 >> 172.21.15.86:0/3358422465 conn(0x7fb5180fb750 msgr2=0x7fb5180fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:23.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.391+0000 7fb51f800640 1 -- 172.21.15.86:0/3358422465 shutdown_connections 2024-04-17T07:30:23.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.392+0000 7fb51f800640 1 -- 172.21.15.86:0/3358422465 wait complete. 2024-04-17T07:30:23.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.393+0000 7fb51f800640 1 Processor -- start 2024-04-17T07:30:23.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.393+0000 7fb51f800640 1 -- start start 2024-04-17T07:30:23.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.393+0000 7fb51f800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5181001e0 0x7fb518198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.393+0000 7fb51f800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181013d0 0x7fb5181986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.394+0000 7fb51f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb518101d90 0x7fb5181a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.394+0000 7fb51ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5181001e0 0x7fb518198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.394+0000 7fb51ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5181001e0 0x7fb518198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37504/0 (socket says 172.21.15.86:37504) 2024-04-17T07:30:23.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.394+0000 7fb50fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181013d0 0x7fb5181986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.394+0000 7fb50fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181013d0 0x7fb5181986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33646/0 (socket says 172.21.15.86:33646) 2024-04-17T07:30:23.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.394+0000 7fb51f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810b8b0 con 0x7fb518101d90 2024-04-17T07:30:23.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.394+0000 7fb51f800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810b730 con 0x7fb5181001e0 2024-04-17T07:30:23.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.394+0000 7fb51f800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810ba30 con 0x7fb5181013d0 2024-04-17T07:30:23.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.394+0000 7fb51ce00640 1 -- 172.21.15.86:0/1430436358 learned_addr learned my addr 172.21.15.86:0/1430436358 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:23.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.395+0000 7fb51d800640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb518101d90 0x7fb5181a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.395+0000 7fb50fe00640 1 -- 172.21.15.86:0/1430436358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5181001e0 msgr2=0x7fb518198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:23.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.395+0000 7fb50fe00640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5181001e0 0x7fb518198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.395+0000 7fb50fe00640 1 -- 172.21.15.86:0/1430436358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb518101d90 msgr2=0x7fb5181a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:23.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.395+0000 7fb50fe00640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb518101d90 0x7fb5181a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.396+0000 7fb50fe00640 1 -- 172.21.15.86:0/1430436358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5181a2070 con 0x7fb5181013d0 2024-04-17T07:30:23.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.396+0000 7fb51ce00640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5181001e0 0x7fb518198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.396+0000 7fb50fe00640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181013d0 0x7fb5181986c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb5040074e0 tx=0x7fb50400aa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.397+0000 7fb516000640 1 -- 172.21.15.86:0/1430436358 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504041df0 con 0x7fb5181013d0 2024-04-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.397+0000 7fb516000640 1 -- 172.21.15.86:0/1430436358 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb504004040 con 0x7fb5181013d0 2024-04-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.397+0000 7fb516000640 1 -- 172.21.15.86:0/1430436358 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5040035f0 con 0x7fb5181013d0 2024-04-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.398+0000 7fb51f800640 1 -- 172.21.15.86:0/1430436358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51819d780 con 0x7fb5181013d0 2024-04-17T07:30:23.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.398+0000 7fb51f800640 1 -- 172.21.15.86:0/1430436358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51819dc80 con 0x7fb5181013d0 2024-04-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.400+0000 7fb51f800640 1 -- 172.21.15.86:0/1430436358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5180691d0 con 0x7fb5181013d0 2024-04-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.406+0000 7fb516000640 1 -- 172.21.15.86:0/1430436358 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb504031070 con 0x7fb5181013d0 2024-04-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.408+0000 7fb516000640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4f40a3c30 0x7fb4f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.409+0000 7fb516000640 1 -- 172.21.15.86:0/1430436358 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb5040f9ea0 con 0x7fb5181013d0 2024-04-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.409+0000 7fb51ce00640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4f40a3c30 0x7fb4f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.410+0000 7fb51ce00640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4f40a3c30 0x7fb4f40a60f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00aca0 tx=0x7fb4fc00a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.416+0000 7fb516000640 1 -- 172.21.15.86:0/1430436358 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb504050060 con 0x7fb5181013d0 2024-04-17T07:30:23.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:23 smithi098 ceph-mon[35418]: pgmap v920: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.8 MiB/s wr, 687 op/s 2024-04-17T07:30:23.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:23 smithi103 ceph-mon[35495]: pgmap v920: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.8 MiB/s wr, 687 op/s 2024-04-17T07:30:23.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:23 smithi086 ceph-mon[31733]: pgmap v920: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.8 MiB/s wr, 687 op/s 2024-04-17T07:30:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.927+0000 7fb51f800640 1 -- 172.21.15.86:0/1430436358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb51819dfa0 con 0x7fb5181013d0 2024-04-17T07:30:23.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.928+0000 7fb516000640 1 -- 172.21.15.86:0/1430436358 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fb5040c6580 con 0x7fb5181013d0 2024-04-17T07:30:23.930 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:30:23.930 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:30:23.930 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:30:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.934+0000 7fb51f800640 1 -- 172.21.15.86:0/1430436358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4f40a3c30 msgr2=0x7fb4f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.934+0000 7fb51f800640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4f40a3c30 0x7fb4f40a60f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00aca0 tx=0x7fb4fc00a680 comp rx=0 tx=0).stop 2024-04-17T07:30:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.934+0000 7fb51f800640 1 -- 172.21.15.86:0/1430436358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181013d0 msgr2=0x7fb5181986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.934+0000 7fb51f800640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181013d0 0x7fb5181986c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb5040074e0 tx=0x7fb50400aa20 comp rx=0 tx=0).stop 2024-04-17T07:30:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.934+0000 7fb51f800640 1 -- 172.21.15.86:0/1430436358 shutdown_connections 2024-04-17T07:30:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.934+0000 7fb51f800640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb4f40a3c30 0x7fb4f40a60f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.934+0000 7fb51f800640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb518101d90 0x7fb5181a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.935+0000 7fb51f800640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb5181013d0 0x7fb5181986c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.935+0000 7fb51f800640 1 --2- 172.21.15.86:0/1430436358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5181001e0 0x7fb518198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.935+0000 7fb51f800640 1 -- 172.21.15.86:0/1430436358 >> 172.21.15.86:0/1430436358 conn(0x7fb5180fb750 msgr2=0x7fb518106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.935+0000 7fb51f800640 1 -- 172.21.15.86:0/1430436358 shutdown_connections 2024-04-17T07:30:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:23.935+0000 7fb51f800640 1 -- 172.21.15.86:0/1430436358 wait complete. 2024-04-17T07:30:23.965 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:30:24.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.234+0000 7fb2d2200640 1 -- 172.21.15.86:0/1422912950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2cc105d90 msgr2=0x7fb2cc10c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.234+0000 7fb2d2200640 1 --2- 172.21.15.86:0/1422912950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2cc105d90 0x7fb2cc10c620 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009ca0 tx=0x7fb2c402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.234+0000 7fb2d2200640 1 -- 172.21.15.86:0/1422912950 shutdown_connections 2024-04-17T07:30:24.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.234+0000 7fb2d2200640 1 --2- 172.21.15.86:0/1422912950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2cc105d90 0x7fb2cc10c620 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.234+0000 7fb2d2200640 1 --2- 172.21.15.86:0/1422912950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc1053d0 0x7fb2cc105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.234+0000 7fb2d2200640 1 --2- 172.21.15.86:0/1422912950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2cc1041e0 0x7fb2cc104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.234+0000 7fb2d2200640 1 -- 172.21.15.86:0/1422912950 >> 172.21.15.86:0/1422912950 conn(0x7fb2cc0ff750 msgr2=0x7fb2cc101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.234+0000 7fb2d2200640 1 -- 172.21.15.86:0/1422912950 shutdown_connections 2024-04-17T07:30:24.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.234+0000 7fb2d2200640 1 -- 172.21.15.86:0/1422912950 wait complete. 2024-04-17T07:30:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.235+0000 7fb2d2200640 1 Processor -- start 2024-04-17T07:30:24.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.235+0000 7fb2d2200640 1 -- start start 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.235+0000 7fb2d2200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2cc1041e0 0x7fb2cc19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.235+0000 7fb2d2200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc1053d0 0x7fb2cc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.235+0000 7fb2d2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2cc105d90 0x7fb2cc1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.235+0000 7fb2d2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc10fd20 con 0x7fb2cc105d90 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.235+0000 7fb2d2200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc10fba0 con 0x7fb2cc1053d0 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.235+0000 7fb2d2200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc10fea0 con 0x7fb2cc1041e0 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2cc105d90 0x7fb2cc1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2cc105d90 0x7fb2cc1a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48712/0 (socket says 172.21.15.86:48712) 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2cbe00640 1 -- 172.21.15.86:0/3663940770 learned_addr learned my addr 172.21.15.86:0/3663940770 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2caa00640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc1053d0 0x7fb2cc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2cb400640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2cc1041e0 0x7fb2cc19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2cbe00640 1 -- 172.21.15.86:0/3663940770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2cc1041e0 msgr2=0x7fb2cc19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2cbe00640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2cc1041e0 0x7fb2cc19a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2cbe00640 1 -- 172.21.15.86:0/3663940770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc1053d0 msgr2=0x7fb2cc19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2cbe00640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc1053d0 0x7fb2cc19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2cbe00640 1 -- 172.21.15.86:0/3663940770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc1a1ff0 con 0x7fb2cc105d90 2024-04-17T07:30:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.236+0000 7fb2cbe00640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2cc105d90 0x7fb2cc1a1980 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb2c402c700 tx=0x7fb2c40328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.237+0000 7fb2c3e00640 1 -- 172.21.15.86:0/3663940770 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4041020 con 0x7fb2cc105d90 2024-04-17T07:30:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.237+0000 7fb2c3e00640 1 -- 172.21.15.86:0/3663940770 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2c4003e20 con 0x7fb2cc105d90 2024-04-17T07:30:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.237+0000 7fb2d2200640 1 -- 172.21.15.86:0/3663940770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc1a2280 con 0x7fb2cc105d90 2024-04-17T07:30:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.237+0000 7fb2c3e00640 1 -- 172.21.15.86:0/3663940770 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4039040 con 0x7fb2cc105d90 2024-04-17T07:30:24.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.237+0000 7fb2d2200640 1 -- 172.21.15.86:0/3663940770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2cc1a2770 con 0x7fb2cc105d90 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.240+0000 7fb2d5032740 1 Processor -- start 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.240+0000 7fb2c3e00640 1 -- 172.21.15.86:0/3663940770 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb2c4044020 con 0x7fb2cc105d90 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.240+0000 7fb2d5032740 1 -- start start 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.240+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d4e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.240+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d53490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.240+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d53b00 0x55ddc2d55f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.240+0000 7fb2d5032740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ddc2cb4020 con 0x55ddc2d4e990 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2d5032740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ddc2c5a460 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2d5032740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ddc2caffe0 con 0x55ddc2d53b00 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d53490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d53490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48714/0 (socket says 172.21.15.86:48714) 2024-04-17T07:30:24.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2cb400640 1 -- 172.21.15.86:0/812235161 learned_addr learned my addr 172.21.15.86:0/812235161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2caa00640 1 --2- 172.21.15.86:0/812235161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d4e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2caa00640 1 --2- 172.21.15.86:0/812235161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d53b00 0x55ddc2d55f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2cb400640 1 -- 172.21.15.86:0/812235161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d53b00 msgr2=0x55ddc2d55f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2cb400640 1 --2- 172.21.15.86:0/812235161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d53b00 0x55ddc2d55f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2cb400640 1 -- 172.21.15.86:0/812235161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d4dfa0 msgr2=0x55ddc2d4e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2cb400640 1 --2- 172.21.15.86:0/812235161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d4e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2cb400640 1 -- 172.21.15.86:0/812235161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddc2c62040 con 0x55ddc2d4e990 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2caa00640 1 --2- 172.21.15.86:0/812235161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d4e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.241+0000 7fb2cb400640 1 --2- 172.21.15.86:0/812235161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d53490 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009b70 tx=0x7fb2bc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc13b41d131ac11 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.242+0000 7fb2c3e00640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2b00a3bb0 0x7fb2b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.242+0000 7fb2c3e00640 1 -- 172.21.15.86:0/3663940770 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb2c40f2cd0 con 0x7fb2cc105d90 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.242+0000 7fb2b7e00640 1 -- 172.21.15.86:0/812235161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc016070 con 0x55ddc2d4e990 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.242+0000 7fb2b7e00640 1 -- 172.21.15.86:0/812235161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2bc0054d0 con 0x55ddc2d4e990 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.242+0000 7fb2b7e00640 1 -- 172.21.15.86:0/812235161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc011470 con 0x55ddc2d4e990 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.242+0000 7fb2caa00640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2b00a3bb0 0x7fb2b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 -- 172.21.15.86:0/812235161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 msgr2=0x55ddc2d53490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2caa00640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2b00a3bb0 0x7fb2b00a6070 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55ddc2d4f1f0 tx=0x7fb2b8004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 --2- 172.21.15.86:0/812235161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d53490 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009b70 tx=0x7fb2bc00e6c0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 -- 172.21.15.86:0/812235161 shutdown_connections 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 --2- 172.21.15.86:0/812235161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d53b00 0x55ddc2d55f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 --2- 172.21.15.86:0/812235161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d53490 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 --2- 172.21.15.86:0/812235161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d4e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 -- 172.21.15.86:0/812235161 >> 172.21.15.86:0/812235161 conn(0x55ddc2d49510 msgr2=0x55ddc2d49910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 -- 172.21.15.86:0/812235161 shutdown_connections 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 -- 172.21.15.86:0/812235161 wait complete. 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 -- start start 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.243+0000 7fb2d5032740 1 Processor -- start 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d58520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d58a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d5faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2d5032740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ddc2d56ab0 con 0x55ddc2d4e990 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2d5032740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ddc2c6d830 con 0x55ddc2d53b00 2024-04-17T07:30:24.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2d5032740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ddc2d56c30 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2d5032740 10 client.0 _ll_register_callbacks cb 0x55ddc2c3bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2caa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d58a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2caa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d58a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48724/0 (socket says 172.21.15.86:48724) 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2caa00640 1 -- 172.21.15.86:0/2130105694 learned_addr learned my addr 172.21.15.86:0/2130105694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.244+0000 7fb2cb400640 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d58520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2caa00640 1 -- 172.21.15.86:0/2130105694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 msgr2=0x55ddc2d58520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2caa00640 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d58520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2cbe00640 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d5faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2caa00640 1 -- 172.21.15.86:0/2130105694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 msgr2=0x55ddc2d5faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2caa00640 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d5faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2caa00640 1 -- 172.21.15.86:0/2130105694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddc2c71920 con 0x55ddc2d4e990 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2cb400640 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d58520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2cbe00640 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d5faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2caa00640 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d58a60 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8005c80 tx=0x7fb2b8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2727cf42b0bbed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2b4c00640 1 -- 172.21.15.86:0/2130105694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8029060 con 0x55ddc2d4e990 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2b4c00640 1 -- 172.21.15.86:0/2130105694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb2b801ab10 con 0x55ddc2d4e990 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2d5032740 1 -- 172.21.15.86:0/2130105694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2bc0054d0 con 0x55ddc2d4e990 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2b4c00640 1 -- 172.21.15.86:0/2130105694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2b801ad10 con 0x55ddc2d4e990 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2d5032740 10 client.19125 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.245+0000 7fb2d5032740 1 -- 172.21.15.86:0/2130105694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ddc2c3f7f0 con 0x55ddc2d4e990 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.246+0000 7fb2b4c00640 1 -- 172.21.15.86:0/2130105694 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2bc0054d0 con 0x55ddc2d4e990 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.246+0000 7fb2b4c00640 1 -- 172.21.15.86:0/2130105694 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb2b8018040 con 0x55ddc2d4e990 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.246+0000 7fb2d5032740 10 client.19125 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:24.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.246+0000 7fb2d5032740 10 client.19125 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:30:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.246+0000 7fb2d5032740 20 client.19125 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.246+0000 7fb2d5032740 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ddc2d64060 0x55ddc2d66500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.246+0000 7fb2d5032740 4 client.19125 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:30:24.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.246+0000 7fb2d5032740 1 -- 172.21.15.86:0/2130105694 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddc2cc1a40 con 0x55ddc2d64060 2024-04-17T07:30:24.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.247+0000 7fb2cb400640 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ddc2d64060 0x55ddc2d66500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.248+0000 7fb2cb400640 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ddc2d64060 0x55ddc2d66500 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66a0720744bc3c89 server_cookie=ff74f0312eba9afb in_seq=0 out_seq=0 2024-04-17T07:30:24.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.248+0000 7fb2b4c00640 10 client.19125 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:30:24.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2b4c00640 1 -- 172.21.15.86:0/2130105694 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddc2cc1a40 con 0x55ddc2d64060 2024-04-17T07:30:24.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2b4c00640 10 client.19125 handle_command_reply: tid=0 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2d5032740 1 client.19125 shutdown 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2d5032740 1 -- 172.21.15.86:0/2130105694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 msgr2=0x55ddc2d58a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2d5032740 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d58a60 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8005c80 tx=0x7fb2b8005850 comp rx=0 tx=0).stop 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2d5032740 1 -- 172.21.15.86:0/2130105694 shutdown_connections 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2d5032740 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ddc2d64060 0x55ddc2d66500 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2d5032740 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d5faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2d5032740 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d58a60 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2b4c00640 0 client.19125 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2d5032740 1 --2- 172.21.15.86:0/2130105694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d58520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.249+0000 7fb2d5032740 1 -- 172.21.15.86:0/2130105694 >> 172.21.15.86:0/2130105694 conn(0x55ddc2d49510 msgr2=0x55ddc2d48710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.250+0000 7fb2d5032740 1 -- 172.21.15.86:0/2130105694 shutdown_connections 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.250+0000 7fb2d5032740 1 -- 172.21.15.86:0/2130105694 wait complete. 2024-04-17T07:30:24.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.250+0000 7fb2d5032740 20 client.19125 trim_cache size 0 max 16384 2024-04-17T07:30:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2d5032740 1 Processor -- start 2024-04-17T07:30:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2d5032740 1 -- start start 2024-04-17T07:30:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d57fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d47e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2d5032740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1accf0 con 0x55ddc2d4e990 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2d5032740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1ae830 con 0x55ddc2d53b00 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2d5032740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1ad3c0 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2caa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2caa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48728/0 (socket says 172.21.15.86:48728) 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.264+0000 7fb2caa00640 1 -- 172.21.15.86:0/3534479663 learned_addr learned my addr 172.21.15.86:0/3534479663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2cb400640 1 --2- 172.21.15.86:0/3534479663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d57fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2caa00640 1 -- 172.21.15.86:0/3534479663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 msgr2=0x55ddc2d57fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2caa00640 1 --2- 172.21.15.86:0/3534479663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d57fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2caa00640 1 -- 172.21.15.86:0/3534479663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 msgr2=0x55ddc2d47e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2cbe00640 1 --2- 172.21.15.86:0/3534479663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d47e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2caa00640 1 --2- 172.21.15.86:0/3534479663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d47e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2caa00640 1 -- 172.21.15.86:0/3534479663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddc29c63f0 con 0x55ddc2d4e990 2024-04-17T07:30:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2cbe00640 1 --2- 172.21.15.86:0/3534479663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d47e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:30:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2caa00640 1 --2- 172.21.15.86:0/3534479663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5fa60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8005c50 tx=0x7fb2b801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cbf6afc4352909 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2b7400640 1 -- 172.21.15.86:0/3534479663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8039070 con 0x55ddc2d4e990 2024-04-17T07:30:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2b7400640 1 -- 172.21.15.86:0/3534479663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2b8023b80 con 0x55ddc2d4e990 2024-04-17T07:30:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.265+0000 7fb2b7400640 1 -- 172.21.15.86:0/3534479663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8034aa0 con 0x55ddc2d4e990 2024-04-17T07:30:24.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.266+0000 7fb2d5032740 1 -- 172.21.15.86:0/3534479663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 msgr2=0x55ddc2d5fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.266+0000 7fb2d5032740 1 --2- 172.21.15.86:0/3534479663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5fa60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8005c50 tx=0x7fb2b801aca0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.266+0000 7fb2d5032740 1 -- 172.21.15.86:0/3534479663 shutdown_connections 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.266+0000 7fb2d5032740 1 --2- 172.21.15.86:0/3534479663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d47e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.266+0000 7fb2d5032740 1 --2- 172.21.15.86:0/3534479663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5fa60 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.266+0000 7fb2d5032740 1 --2- 172.21.15.86:0/3534479663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d57fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.266+0000 7fb2d5032740 1 -- 172.21.15.86:0/3534479663 >> 172.21.15.86:0/3534479663 conn(0x55ddc2d49510 msgr2=0x55ddc2d57000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.266+0000 7fb2d5032740 1 -- 172.21.15.86:0/3534479663 shutdown_connections 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.266+0000 7fb2d5032740 1 -- 172.21.15.86:0/3534479663 wait complete. 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.266+0000 7fb2d5032740 1 -- start start 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2d5032740 1 Processor -- start 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d5cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2d5032740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d5e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2d5032740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1ae160 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2d5032740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1ada90 con 0x55ddc2d53b00 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2d5032740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1aacc0 con 0x55ddc2d4e990 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2d5032740 10 client.0 _ll_register_callbacks cb 0x7fb2b801c890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d5cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2cb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d5cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48742/0 (socket says 172.21.15.86:48742) 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2cb400640 1 -- 172.21.15.86:0/3190395332 learned_addr learned my addr 172.21.15.86:0/3190395332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.267+0000 7fb2caa00640 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2cb400640 1 -- 172.21.15.86:0/3190395332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4e990 msgr2=0x55ddc2d5d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2cb400640 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2cbe00640 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d5e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2cb400640 1 -- 172.21.15.86:0/3190395332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 msgr2=0x55ddc2d5e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2cb400640 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d5e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2cb400640 1 -- 172.21.15.86:0/3190395332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddc2a907d0 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2caa00640 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:30:24.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2cb400640 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d5cad0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x55ddc2d4f400 tx=0x7fb2bc009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b26c64830aed526 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2b4c00640 1 -- 172.21.15.86:0/3190395332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc02b060 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2b4c00640 1 -- 172.21.15.86:0/3190395332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb2bc025650 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2d5032740 1 -- 172.21.15.86:0/3190395332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b8023b80 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2b4c00640 1 -- 172.21.15.86:0/3190395332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2bc032050 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2d5032740 10 client.19131 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.268+0000 7fb2d5032740 1 -- 172.21.15.86:0/3190395332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ddc2d5f750 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.269+0000 7fb2b4c00640 1 -- 172.21.15.86:0/3190395332 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2b8023b80 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.269+0000 7fb2b4c00640 1 -- 172.21.15.86:0/3190395332 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb2bc00f040 con 0x55ddc2d4dfa0 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.269+0000 7fb2d5032740 10 client.19131 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.269+0000 7fb2d5032740 10 client.19131 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.269+0000 7fb2d5032740 20 client.19131 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:24.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.269+0000 7fb2d5032740 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ddc2dca640 0x55ddc2dcca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.269+0000 7fb2d5032740 4 client.19131 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:30:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.269+0000 7fb2d5032740 1 -- 172.21.15.86:0/3190395332 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddc2cbb8b0 con 0x55ddc2dca640 2024-04-17T07:30:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.270+0000 7fb2caa00640 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ddc2dca640 0x55ddc2dcca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.270+0000 7fb2caa00640 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ddc2dca640 0x55ddc2dcca30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=22188300e61fc5ea server_cookie=b3dbd8e9fd8f3752 in_seq=0 out_seq=0 2024-04-17T07:30:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.270+0000 7fb2b4c00640 10 client.19131 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:30:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2b4c00640 1 -- 172.21.15.86:0/3190395332 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddc2cbb8b0 con 0x55ddc2dca640 2024-04-17T07:30:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2b4c00640 10 client.19131 handle_command_reply: tid=0 2024-04-17T07:30:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 client.19131 shutdown 2024-04-17T07:30:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 -- 172.21.15.86:0/3190395332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4dfa0 msgr2=0x55ddc2d5cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d5cad0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x55ddc2d4f400 tx=0x7fb2bc009480 comp rx=0 tx=0).stop 2024-04-17T07:30:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 -- 172.21.15.86:0/3190395332 shutdown_connections 2024-04-17T07:30:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ddc2dca640 0x55ddc2dcca30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ddc2d53b00 0x55ddc2d5e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddc2d4e990 0x55ddc2d5d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2b4c00640 0 client.19131 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:30:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 --2- 172.21.15.86:0/3190395332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddc2d4dfa0 0x55ddc2d5cad0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 -- 172.21.15.86:0/3190395332 >> 172.21.15.86:0/3190395332 conn(0x55ddc2d49510 msgr2=0x55ddc2d4c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 -- 172.21.15.86:0/3190395332 shutdown_connections 2024-04-17T07:30:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 1 -- 172.21.15.86:0/3190395332 wait complete. 2024-04-17T07:30:24.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.271+0000 7fb2d5032740 20 client.19131 trim_cache size 0 max 16384 2024-04-17T07:30:24.276 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:30:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 -- 172.21.15.86:0/3663940770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2b00a3bb0 msgr2=0x7fb2b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2b00a3bb0 0x7fb2b00a6070 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55ddc2d4f1f0 tx=0x7fb2b8004730 comp rx=0 tx=0).stop 2024-04-17T07:30:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 -- 172.21.15.86:0/3663940770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2cc105d90 msgr2=0x7fb2cc1a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2cc105d90 0x7fb2cc1a1980 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb2c402c700 tx=0x7fb2c40328a0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 -- 172.21.15.86:0/3663940770 shutdown_connections 2024-04-17T07:30:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb2b00a3bb0 0x7fb2b00a6070 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2cc105d90 0x7fb2cc1a1980 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc1053d0 0x7fb2cc19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 --2- 172.21.15.86:0/3663940770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2cc1041e0 0x7fb2cc19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 -- 172.21.15.86:0/3663940770 >> 172.21.15.86:0/3663940770 conn(0x7fb2cc0ff750 msgr2=0x7fb2cc1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 -- 172.21.15.86:0/3663940770 shutdown_connections 2024-04-17T07:30:24.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.272+0000 7fb2d2200640 1 -- 172.21.15.86:0/3663940770 wait complete. 2024-04-17T07:30:24.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:30:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 660.0 s, 479.6 tps, lat 206.421 ms stddev 167.036 2024-04-17T07:30:24.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.552+0000 7f4d42200640 1 -- 172.21.15.86:0/2861976435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d3c06de20 msgr2=0x7f4d3c06e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.552+0000 7f4d42200640 1 --2- 172.21.15.86:0/2861976435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d3c06de20 0x7f4d3c06e240 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009900 tx=0x7f4d3402bc40 comp rx=0 tx=0).stop 2024-04-17T07:30:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.552+0000 7f4d42200640 1 -- 172.21.15.86:0/2861976435 shutdown_connections 2024-04-17T07:30:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.552+0000 7f4d42200640 1 --2- 172.21.15.86:0/2861976435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d3c06fa00 0x7f4d3c0702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.552+0000 7f4d42200640 1 --2- 172.21.15.86:0/2861976435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d3c06f060 0x7f4d3c06f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.552+0000 7f4d42200640 1 --2- 172.21.15.86:0/2861976435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d3c06de20 0x7f4d3c06e240 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.552+0000 7f4d42200640 1 -- 172.21.15.86:0/2861976435 >> 172.21.15.86:0/2861976435 conn(0x7f4d3c06c9f0 msgr2=0x7f4d3c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.552+0000 7f4d42200640 1 -- 172.21.15.86:0/2861976435 shutdown_connections 2024-04-17T07:30:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.552+0000 7f4d42200640 1 -- 172.21.15.86:0/2861976435 wait complete. 2024-04-17T07:30:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.553+0000 7f4d42200640 1 Processor -- start 2024-04-17T07:30:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.554+0000 7f4d42200640 1 -- start start 2024-04-17T07:30:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.554+0000 7f4d42200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d3c06de20 0x7f4d3c1a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.554+0000 7f4d42200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d3c06f060 0x7f4d3c1a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.554+0000 7f4d42200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d3c06fa00 0x7f4d3c1ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.554+0000 7f4d42200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c11c910 con 0x7f4d3c06de20 2024-04-17T07:30:24.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.554+0000 7f4d42200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c11c790 con 0x7f4d3c06fa00 2024-04-17T07:30:24.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.554+0000 7f4d42200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c11ca90 con 0x7f4d3c06f060 2024-04-17T07:30:24.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d3c06f060 0x7f4d3c1a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d3c06f060 0x7f4d3c1a74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33714/0 (socket says 172.21.15.86:33714) 2024-04-17T07:30:24.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3aa00640 1 -- 172.21.15.86:0/4281073767 learned_addr learned my addr 172.21.15.86:0/4281073767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3b400640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d3c06de20 0x7f4d3c1a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3be00640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d3c06fa00 0x7f4d3c1ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3be00640 1 -- 172.21.15.86:0/4281073767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d3c06f060 msgr2=0x7f4d3c1a74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3be00640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d3c06f060 0x7f4d3c1a74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3be00640 1 -- 172.21.15.86:0/4281073767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d3c06de20 msgr2=0x7f4d3c1a6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3be00640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d3c06de20 0x7f4d3c1a6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3be00640 1 -- 172.21.15.86:0/4281073767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3c1aeb80 con 0x7f4d3c06fa00 2024-04-17T07:30:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.555+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d3c06f060 0x7f4d3c1a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:30:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.556+0000 7f4d3be00640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d3c06fa00 0x7f4d3c1ae510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800d680 tx=0x7f4d2800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.556+0000 7f4d33e00640 1 -- 172.21.15.86:0/4281073767 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28017070 con 0x7f4d3c06fa00 2024-04-17T07:30:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.556+0000 7f4d33e00640 1 -- 172.21.15.86:0/4281073767 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d28009820 con 0x7f4d3c06fa00 2024-04-17T07:30:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.556+0000 7f4d33e00640 1 -- 172.21.15.86:0/4281073767 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28004cf0 con 0x7f4d3c06fa00 2024-04-17T07:30:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.556+0000 7f4d42200640 1 -- 172.21.15.86:0/4281073767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3c1aee70 con 0x7f4d3c06fa00 2024-04-17T07:30:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.557+0000 7f4d42200640 1 -- 172.21.15.86:0/4281073767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d3c1af2e0 con 0x7f4d3c06fa00 2024-04-17T07:30:24.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.562+0000 7f4d44e3e740 1 Processor -- start 2024-04-17T07:30:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.562+0000 7f4d33e00640 1 -- 172.21.15.86:0/4281073767 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4d280099d0 con 0x7f4d3c06fa00 2024-04-17T07:30:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.563+0000 7f4d44e3e740 1 -- start start 2024-04-17T07:30:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.563+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c96fa0 0x55ae81c973c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.563+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 0x55ae81c9c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.563+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c9cb00 0x55ae81c9ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.563+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae81bfd020 con 0x55ae81c96fa0 2024-04-17T07:30:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.563+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ae81ba3460 con 0x55ae81c97990 2024-04-17T07:30:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.563+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae81bf8fe0 con 0x55ae81c9cb00 2024-04-17T07:30:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.563+0000 7f4d3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c96fa0 0x55ae81c973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.564+0000 7f4d3aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c96fa0 0x55ae81c973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48788/0 (socket says 172.21.15.86:48788) 2024-04-17T07:30:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.564+0000 7f4d3aa00640 1 -- 172.21.15.86:0/1466509594 learned_addr learned my addr 172.21.15.86:0/1466509594 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.564+0000 7f4d3b400640 1 --2- 172.21.15.86:0/1466509594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 0x55ae81c9c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.564+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/1466509594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c9cb00 0x55ae81c9ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.564+0000 7f4d3aa00640 1 -- 172.21.15.86:0/1466509594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c9cb00 msgr2=0x55ae81c9ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.564+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/1466509594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c9cb00 0x55ae81c9ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.564+0000 7f4d3aa00640 1 -- 172.21.15.86:0/1466509594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 msgr2=0x55ae81c9c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.564+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/1466509594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 0x55ae81c9c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.564+0000 7f4d3aa00640 1 -- 172.21.15.86:0/1466509594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae81bab040 con 0x55ae81c96fa0 2024-04-17T07:30:24.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.565+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/1466509594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c96fa0 0x55ae81c973c0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c011c40 tx=0x7f4d2c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370e4ccbbb6f52be server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.565+0000 7f4d33e00640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d200a3c30 0x7f4d200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.566+0000 7f4d27e00640 1 -- 172.21.15.86:0/1466509594 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c019030 con 0x55ae81c96fa0 2024-04-17T07:30:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.566+0000 7f4d27e00640 1 -- 172.21.15.86:0/1466509594 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d2c00fe70 con 0x55ae81c96fa0 2024-04-17T07:30:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.566+0000 7f4d27e00640 1 -- 172.21.15.86:0/1466509594 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0058d0 con 0x55ae81c96fa0 2024-04-17T07:30:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.566+0000 7f4d33e00640 1 -- 172.21.15.86:0/4281073767 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4d280cffa0 con 0x7f4d3c06fa00 2024-04-17T07:30:24.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.566+0000 7f4d3b400640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d200a3c30 0x7f4d200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.567+0000 7f4d44e3e740 1 -- 172.21.15.86:0/1466509594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c96fa0 msgr2=0x55ae81c973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.567+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/1466509594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c96fa0 0x55ae81c973c0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c011c40 tx=0x7f4d2c007f90 comp rx=0 tx=0).stop 2024-04-17T07:30:24.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.567+0000 7f4d44e3e740 1 -- 172.21.15.86:0/1466509594 shutdown_connections 2024-04-17T07:30:24.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.567+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/1466509594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c9cb00 0x55ae81c9ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.567+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/1466509594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 0x55ae81c9c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.567+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/1466509594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c96fa0 0x55ae81c973c0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.567+0000 7f4d44e3e740 1 -- 172.21.15.86:0/1466509594 >> 172.21.15.86:0/1466509594 conn(0x55ae81c92510 msgr2=0x55ae81c92910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.567+0000 7f4d44e3e740 1 -- 172.21.15.86:0/1466509594 shutdown_connections 2024-04-17T07:30:24.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.568+0000 7f4d44e3e740 1 -- 172.21.15.86:0/1466509594 wait complete. 2024-04-17T07:30:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.568+0000 7f4d3b400640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d200a3c30 0x7f4d200a60f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4d34000c00 tx=0x7f4d3402c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.568+0000 7f4d44e3e740 1 -- start start 2024-04-17T07:30:24.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.568+0000 7f4d44e3e740 1 Processor -- start 2024-04-17T07:30:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.569+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 0x55ae81ca1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.569+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c9cb00 0x55ae81ca1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.569+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81ca6b80 0x55ae81ca8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.569+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae81c9fab0 con 0x55ae81c9cb00 2024-04-17T07:30:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.569+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ae81bb6830 con 0x55ae81ca6b80 2024-04-17T07:30:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.569+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae81c9fc30 con 0x55ae81c97990 2024-04-17T07:30:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.569+0000 7f4d44e3e740 10 client.0 _ll_register_callbacks cb 0x55ae81b84fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:24.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.570+0000 7f4d3be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81ca6b80 0x55ae81ca8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.570+0000 7f4d3be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81ca6b80 0x55ae81ca8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37600/0 (socket says 172.21.15.86:37600) 2024-04-17T07:30:24.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.570+0000 7f4d3be00640 1 -- 172.21.15.86:0/4171266956 learned_addr learned my addr 172.21.15.86:0/4171266956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.570+0000 7f4d3b400640 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 0x55ae81ca1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.570+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c9cb00 0x55ae81ca1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.571+0000 7f4d3be00640 1 -- 172.21.15.86:0/4171266956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 msgr2=0x55ae81ca1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.571+0000 7f4d3be00640 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 0x55ae81ca1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.571+0000 7f4d3be00640 1 -- 172.21.15.86:0/4171266956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c9cb00 msgr2=0x55ae81ca1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.571+0000 7f4d3be00640 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c9cb00 0x55ae81ca1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.571+0000 7f4d3be00640 1 -- 172.21.15.86:0/4171266956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae81bba920 con 0x55ae81ca6b80 2024-04-17T07:30:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.571+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c9cb00 0x55ae81ca1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:30:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.571+0000 7f4d3be00640 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81ca6b80 0x55ae81ca8f70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d2809bcb0 tx=0x7f4d28009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d87948ca23521893 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.571+0000 7f4d24c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:30:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.572+0000 7f4d24c00640 1 -- 172.21.15.86:0/4171266956 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28014030 con 0x55ae81ca6b80 2024-04-17T07:30:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.572+0000 7f4d24c00640 1 -- 172.21.15.86:0/4171266956 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4d28005220 con 0x55ae81ca6b80 2024-04-17T07:30:24.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.572+0000 7f4d44e3e740 1 -- 172.21.15.86:0/4171266956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d2c00fe70 con 0x55ae81ca6b80 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.572+0000 7f4d24c00640 1 -- 172.21.15.86:0/4171266956 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d280cdbe0 con 0x55ae81ca6b80 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.572+0000 7f4d44e3e740 10 client.38105 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.572+0000 7f4d44e3e740 1 -- 172.21.15.86:0/4171266956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae81b887f0 con 0x55ae81ca6b80 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.573+0000 7f4d24c00640 1 -- 172.21.15.86:0/4171266956 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d2c00fe70 con 0x55ae81ca6b80 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.573+0000 7f4d24c00640 1 -- 172.21.15.86:0/4171266956 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4d28096b20 con 0x55ae81ca6b80 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.573+0000 7f4d44e3e740 10 client.38105 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.573+0000 7f4d44e3e740 10 client.38105 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.573+0000 7f4d44e3e740 20 client.38105 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.573+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ae81cad400 0x55ae81caf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.573+0000 7f4d44e3e740 4 client.38105 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:30:24.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.573+0000 7f4d44e3e740 1 -- 172.21.15.86:0/4171266956 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae81c0aa40 con 0x55ae81cad400 2024-04-17T07:30:24.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.575+0000 7f4d3b400640 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ae81cad400 0x55ae81caf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.576+0000 7f4d3b400640 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ae81cad400 0x55ae81caf840 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49dba84e409fe2e7 server_cookie=6043910366be1eda in_seq=0 out_seq=0 2024-04-17T07:30:24.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.576+0000 7f4d24c00640 10 client.38105 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:30:24.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.578+0000 7f4d24c00640 1 -- 172.21.15.86:0/4171266956 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae81c0aa40 con 0x55ae81cad400 2024-04-17T07:30:24.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.578+0000 7f4d24c00640 10 client.38105 handle_command_reply: tid=0 2024-04-17T07:30:24.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.578+0000 7f4d44e3e740 1 client.38105 shutdown 2024-04-17T07:30:24.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.579+0000 7f4d44e3e740 1 -- 172.21.15.86:0/4171266956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81ca6b80 msgr2=0x55ae81ca8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.579+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81ca6b80 0x55ae81ca8f70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d2809bcb0 tx=0x7f4d28009ea0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.579+0000 7f4d44e3e740 1 -- 172.21.15.86:0/4171266956 shutdown_connections 2024-04-17T07:30:24.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.579+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ae81cad400 0x55ae81caf840 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.579+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81ca6b80 0x55ae81ca8f70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.579+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81c9cb00 0x55ae81ca1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.579+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/4171266956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 0x55ae81ca1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.579+0000 7f4d44e3e740 1 -- 172.21.15.86:0/4171266956 >> 172.21.15.86:0/4171266956 conn(0x55ae81c92510 msgr2=0x55ae81c91710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.579+0000 7f4d24c00640 0 client.38105 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:30:24.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.579+0000 7f4d44e3e740 1 -- 172.21.15.86:0/4171266956 shutdown_connections 2024-04-17T07:30:24.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:24 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1430436358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:30:24.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.580+0000 7f4d44e3e740 1 -- 172.21.15.86:0/4171266956 wait complete. 2024-04-17T07:30:24.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.580+0000 7f4d44e3e740 20 client.38105 trim_cache size 0 max 16384 2024-04-17T07:30:24.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.604+0000 7f4d44e3e740 1 Processor -- start 2024-04-17T07:30:24.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.605+0000 7f4d44e3e740 1 -- start start 2024-04-17T07:30:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.605+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 0x55ae81caf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.605+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c9cb00 0x55ae81c99040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.605+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81ca6b80 0x55ae81c99580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.605+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1b9990 con 0x55ae81ca6b80 2024-04-17T07:30:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.605+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1bb4d0 con 0x55ae81c97990 2024-04-17T07:30:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.605+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1ba060 con 0x55ae81c9cb00 2024-04-17T07:30:24.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.606+0000 7f4d3b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 0x55ae81caf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.606+0000 7f4d3b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 0x55ae81caf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37612/0 (socket says 172.21.15.86:37612) 2024-04-17T07:30:24.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.606+0000 7f4d3b400640 1 -- 172.21.15.86:0/284568701 learned_addr learned my addr 172.21.15.86:0/284568701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.606+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/284568701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c9cb00 0x55ae81c99040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.607+0000 7f4d3be00640 1 --2- 172.21.15.86:0/284568701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81ca6b80 0x55ae81c99580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.607+0000 7f4d3b400640 1 -- 172.21.15.86:0/284568701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c9cb00 msgr2=0x55ae81c99040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.607+0000 7f4d3b400640 1 --2- 172.21.15.86:0/284568701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c9cb00 0x55ae81c99040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.607+0000 7f4d3b400640 1 -- 172.21.15.86:0/284568701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81ca6b80 msgr2=0x55ae81c99580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.607+0000 7f4d3b400640 1 --2- 172.21.15.86:0/284568701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81ca6b80 0x55ae81c99580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.607+0000 7f4d3b400640 1 -- 172.21.15.86:0/284568701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae8190f3f0 con 0x55ae81c97990 2024-04-17T07:30:24.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.608+0000 7f4d3b400640 1 --2- 172.21.15.86:0/284568701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 0x55ae81caf410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4d34005ab0 tx=0x7f4d34045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82fb9fd69f894967 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.609+0000 7f4d27400640 1 -- 172.21.15.86:0/284568701 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34004480 con 0x55ae81c97990 2024-04-17T07:30:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.609+0000 7f4d27400640 1 -- 172.21.15.86:0/284568701 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d3405c070 con 0x55ae81c97990 2024-04-17T07:30:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.609+0000 7f4d27400640 1 -- 172.21.15.86:0/284568701 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d340046c0 con 0x55ae81c97990 2024-04-17T07:30:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.611+0000 7f4d44e3e740 1 -- 172.21.15.86:0/284568701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 msgr2=0x55ae81caf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.611+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/284568701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 0x55ae81caf410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4d34005ab0 tx=0x7f4d34045000 comp rx=0 tx=0).stop 2024-04-17T07:30:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.611+0000 7f4d44e3e740 1 -- 172.21.15.86:0/284568701 shutdown_connections 2024-04-17T07:30:24.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.611+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/284568701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81ca6b80 0x55ae81c99580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.611+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/284568701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c9cb00 0x55ae81c99040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.611+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/284568701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c97990 0x55ae81caf410 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.611+0000 7f4d44e3e740 1 -- 172.21.15.86:0/284568701 >> 172.21.15.86:0/284568701 conn(0x55ae81c92510 msgr2=0x55ae81ca1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.612+0000 7f4d44e3e740 1 -- 172.21.15.86:0/284568701 shutdown_connections 2024-04-17T07:30:24.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.612+0000 7f4d44e3e740 1 -- 172.21.15.86:0/284568701 wait complete. 2024-04-17T07:30:24.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.612+0000 7f4d44e3e740 1 -- start start 2024-04-17T07:30:24.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.612+0000 7f4d44e3e740 1 Processor -- start 2024-04-17T07:30:24.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.613+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 0x55ae81d10510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.613+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c9cb00 0x55ae81d10a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.614+0000 7f4d44e3e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81ca6b80 0x55ae81cbfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.614+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1bae00 con 0x55ae81ca6b80 2024-04-17T07:30:24.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.614+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1ba730 con 0x55ae81c9cb00 2024-04-17T07:30:24.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.614+0000 7f4d44e3e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1b7960 con 0x55ae81c97990 2024-04-17T07:30:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.614+0000 7f4d44e3e740 10 client.0 _ll_register_callbacks cb 0x7f4d2809b910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.614+0000 7f4d3aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c9cb00 0x55ae81d10a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.614+0000 7f4d3b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 0x55ae81d10510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.614+0000 7f4d3b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 0x55ae81d10510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33758/0 (socket says 172.21.15.86:33758) 2024-04-17T07:30:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.614+0000 7f4d3b400640 1 -- 172.21.15.86:0/2588317319 learned_addr learned my addr 172.21.15.86:0/2588317319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.614+0000 7f4d3be00640 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81ca6b80 0x55ae81cbfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.615+0000 7f4d3aa00640 1 -- 172.21.15.86:0/2588317319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 msgr2=0x55ae81d10510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.615+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 0x55ae81d10510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.615+0000 7f4d3aa00640 1 -- 172.21.15.86:0/2588317319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81ca6b80 msgr2=0x55ae81cbfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.615+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81ca6b80 0x55ae81cbfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.615+0000 7f4d3aa00640 1 -- 172.21.15.86:0/2588317319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae819d97d0 con 0x55ae81c9cb00 2024-04-17T07:30:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.615+0000 7f4d3aa00640 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c9cb00 0x55ae81d10a50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55ae81c934b0 tx=0x7f4d2c0113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d4f1b7a9408393 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.615+0000 7f4d24c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:30:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.616+0000 7f4d24c00640 1 -- 172.21.15.86:0/2588317319 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c019040 con 0x55ae81c9cb00 2024-04-17T07:30:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.616+0000 7f4d44e3e740 1 -- 172.21.15.86:0/2588317319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3405c070 con 0x55ae81c9cb00 2024-04-17T07:30:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.616+0000 7f4d24c00640 1 -- 172.21.15.86:0/2588317319 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4d2c025030 con 0x55ae81c9cb00 2024-04-17T07:30:24.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.618+0000 7f4d24c00640 1 -- 172.21.15.86:0/2588317319 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d2c019040 con 0x55ae81c9cb00 2024-04-17T07:30:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.619+0000 7f4d44e3e740 10 client.38111 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.619+0000 7f4d44e3e740 1 -- 172.21.15.86:0/2588317319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae81d246f0 con 0x55ae81c9cb00 2024-04-17T07:30:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.619+0000 7f4d24c00640 1 -- 172.21.15.86:0/2588317319 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d3405c070 con 0x55ae81c9cb00 2024-04-17T07:30:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.620+0000 7f4d24c00640 1 -- 172.21.15.86:0/2588317319 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4d2c01e460 con 0x55ae81c9cb00 2024-04-17T07:30:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.620+0000 7f4d44e3e740 10 client.38111 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.620+0000 7f4d44e3e740 10 client.38111 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:30:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.620+0000 7f4d44e3e740 20 client.38111 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.620+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ae81d24d10 0x55ae81d27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.620+0000 7f4d44e3e740 4 client.38111 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:30:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.620+0000 7f4d44e3e740 1 -- 172.21.15.86:0/2588317319 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae81c048b0 con 0x55ae81d24d10 2024-04-17T07:30:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.620+0000 7f4d3b400640 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ae81d24d10 0x55ae81d27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.621+0000 7f4d3b400640 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ae81d24d10 0x55ae81d27130 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce7e94edf6d03b19 server_cookie=4260ef034a54ea8d in_seq=0 out_seq=0 2024-04-17T07:30:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.621+0000 7f4d24c00640 10 client.38111 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:30:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.622+0000 7f4d24c00640 1 -- 172.21.15.86:0/2588317319 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae81c048b0 con 0x55ae81d24d10 2024-04-17T07:30:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.622+0000 7f4d24c00640 10 client.38111 handle_command_reply: tid=0 2024-04-17T07:30:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.622+0000 7f4d44e3e740 1 client.38111 shutdown 2024-04-17T07:30:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d44e3e740 1 -- 172.21.15.86:0/2588317319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c9cb00 msgr2=0x55ae81d10a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c9cb00 0x55ae81d10a50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55ae81c934b0 tx=0x7f4d2c0113e0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d44e3e740 1 -- 172.21.15.86:0/2588317319 shutdown_connections 2024-04-17T07:30:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ae81d24d10 0x55ae81d27130 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae81ca6b80 0x55ae81cbfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae81c9cb00 0x55ae81d10a50 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55ae81c934b0 tx=0x7f4d2c0113e0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d44e3e740 1 --2- 172.21.15.86:0/2588317319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae81c97990 0x55ae81d10510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d44e3e740 1 -- 172.21.15.86:0/2588317319 >> 172.21.15.86:0/2588317319 conn(0x55ae81c92510 msgr2=0x55ae81ca1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d24c00640 0 client.38111 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:30:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d44e3e740 1 -- 172.21.15.86:0/2588317319 shutdown_connections 2024-04-17T07:30:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.623+0000 7f4d44e3e740 1 -- 172.21.15.86:0/2588317319 wait complete. 2024-04-17T07:30:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.624+0000 7f4d44e3e740 20 client.38111 trim_cache size 0 max 16384 2024-04-17T07:30:24.625 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:30:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 -- 172.21.15.86:0/4281073767 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d200a3c30 msgr2=0x7f4d200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d200a3c30 0x7f4d200a60f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4d34000c00 tx=0x7f4d3402c7f0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 -- 172.21.15.86:0/4281073767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d3c06fa00 msgr2=0x7f4d3c1ae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d3c06fa00 0x7f4d3c1ae510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800d680 tx=0x7f4d2800db40 comp rx=0 tx=0).stop 2024-04-17T07:30:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 -- 172.21.15.86:0/4281073767 shutdown_connections 2024-04-17T07:30:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d200a3c30 0x7f4d200a60f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d3c06fa00 0x7f4d3c1ae510 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d3c06f060 0x7f4d3c1a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 --2- 172.21.15.86:0/4281073767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d3c06de20 0x7f4d3c1a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 -- 172.21.15.86:0/4281073767 >> 172.21.15.86:0/4281073767 conn(0x7f4d3c06c9f0 msgr2=0x7f4d3c10aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 -- 172.21.15.86:0/4281073767 shutdown_connections 2024-04-17T07:30:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.625+0000 7f4d42200640 1 -- 172.21.15.86:0/4281073767 wait complete. 2024-04-17T07:30:24.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:30:24.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:24 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1430436358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:30:24.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:24 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1430436358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:30:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.947+0000 7f2334400640 1 -- 172.21.15.86:0/968986201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232c107ca0 msgr2=0x7f232c10a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.947+0000 7f2334400640 1 --2- 172.21.15.86:0/968986201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232c107ca0 0x7f232c10a0f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f231c009ca0 tx=0x7f231c02c100 comp rx=0 tx=0).stop 2024-04-17T07:30:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.948+0000 7f2334400640 1 -- 172.21.15.86:0/968986201 shutdown_connections 2024-04-17T07:30:24.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.948+0000 7f2334400640 1 --2- 172.21.15.86:0/968986201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232c107ca0 0x7f232c10a0f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.948+0000 7f2334400640 1 --2- 172.21.15.86:0/968986201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f232c105370 0x7f232c107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.948+0000 7f2334400640 1 --2- 172.21.15.86:0/968986201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f232c102a40 0x7f232c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.948+0000 7f2334400640 1 -- 172.21.15.86:0/968986201 >> 172.21.15.86:0/968986201 conn(0x7f232c0fc430 msgr2=0x7f232c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.948+0000 7f2334400640 1 -- 172.21.15.86:0/968986201 shutdown_connections 2024-04-17T07:30:24.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.949+0000 7f2334400640 1 -- 172.21.15.86:0/968986201 wait complete. 2024-04-17T07:30:24.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.950+0000 7f2334400640 1 Processor -- start 2024-04-17T07:30:24.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.950+0000 7f2334400640 1 -- start start 2024-04-17T07:30:24.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.951+0000 7f2334400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232c102a40 0x7f232c195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.951+0000 7f2334400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f232c105370 0x7f232c196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.951+0000 7f2334400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f232c107ca0 0x7f232c1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.951+0000 7f2334400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f232c10db00 con 0x7f232c107ca0 2024-04-17T07:30:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.951+0000 7f2334400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f232c10d980 con 0x7f232c102a40 2024-04-17T07:30:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.951+0000 7f2334400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f232c10dc80 con 0x7f232c105370 2024-04-17T07:30:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.951+0000 7f2331a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232c102a40 0x7f232c195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.951+0000 7f2331a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232c102a40 0x7f232c195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37642/0 (socket says 172.21.15.86:37642) 2024-04-17T07:30:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.951+0000 7f2331a00640 1 -- 172.21.15.86:0/1736666450 learned_addr learned my addr 172.21.15.86:0/1736666450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.951+0000 7f2332400640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f232c107ca0 0x7f232c1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.952+0000 7f2331000640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f232c105370 0x7f232c196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.952+0000 7f2331a00640 1 -- 172.21.15.86:0/1736666450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f232c105370 msgr2=0x7f232c196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.952+0000 7f2331a00640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f232c105370 0x7f232c196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.952+0000 7f2331a00640 1 -- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f232c107ca0 msgr2=0x7f232c1a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.953+0000 7f2331a00640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f232c107ca0 0x7f232c1a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.953+0000 7f2331a00640 1 -- 172.21.15.86:0/1736666450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c1a6420 con 0x7f232c102a40 2024-04-17T07:30:24.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.953+0000 7f2332400640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f232c107ca0 0x7f232c1a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:30:24.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.955+0000 7f2331a00640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232c102a40 0x7f232c195fe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f232400ea60 tx=0x7f232400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.956+0000 7f232a000640 1 -- 172.21.15.86:0/1736666450 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232400cde0 con 0x7f232c102a40 2024-04-17T07:30:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.956+0000 7f232a000640 1 -- 172.21.15.86:0/1736666450 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23240035f0 con 0x7f232c102a40 2024-04-17T07:30:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.956+0000 7f232a000640 1 -- 172.21.15.86:0/1736666450 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324010690 con 0x7f232c102a40 2024-04-17T07:30:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.956+0000 7f2334400640 1 -- 172.21.15.86:0/1736666450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c1a6710 con 0x7f232c102a40 2024-04-17T07:30:24.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.956+0000 7f2334400640 1 -- 172.21.15.86:0/1736666450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1a6cb0 con 0x7f232c102a40 2024-04-17T07:30:24.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.960+0000 7f23371a8740 1 Processor -- start 2024-04-17T07:30:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.961+0000 7f23371a8740 1 -- start start 2024-04-17T07:30:24.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.961+0000 7f232a000640 1 -- 172.21.15.86:0/1736666450 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f23240037a0 con 0x7f232c102a40 2024-04-17T07:30:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.962+0000 7f23371a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893d4fa0 0x5627893d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.962+0000 7f23371a8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893da490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.962+0000 7f23371a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893dab00 0x5627893dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.962+0000 7f23371a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56278933b020 con 0x5627893dab00 2024-04-17T07:30:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.962+0000 7f23371a8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5627892e1460 con 0x5627893d5990 2024-04-17T07:30:24.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.962+0000 7f23371a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562789336fe0 con 0x5627893d4fa0 2024-04-17T07:30:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.963+0000 7f2331000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893dab00 0x5627893dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.963+0000 7f2332400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893d4fa0 0x5627893d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.963+0000 7f2332400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893d4fa0 0x5627893d53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33790/0 (socket says 172.21.15.86:33790) 2024-04-17T07:30:24.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.963+0000 7f2332400640 1 -- 172.21.15.86:0/1567939678 learned_addr learned my addr 172.21.15.86:0/1567939678 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.963+0000 7f2332400640 1 -- 172.21.15.86:0/1567939678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 msgr2=0x5627893da490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:30:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.963+0000 7f2332400640 1 --2- 172.21.15.86:0/1567939678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893da490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.963+0000 7f2332400640 1 -- 172.21.15.86:0/1567939678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893dab00 msgr2=0x5627893dcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.963+0000 7f2332400640 1 --2- 172.21.15.86:0/1567939678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893dab00 0x5627893dcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.963+0000 7f2332400640 1 -- 172.21.15.86:0/1567939678 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627892e9040 con 0x5627893d4fa0 2024-04-17T07:30:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.963+0000 7f232a000640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23100a3c30 0x7f23100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.964+0000 7f232a000640 1 -- 172.21.15.86:0/1736666450 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f23240d0aa0 con 0x7f232c102a40 2024-04-17T07:30:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.964+0000 7f2332400640 1 --2- 172.21.15.86:0/1567939678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893d4fa0 0x5627893d53c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f231c005e30 tx=0x7f231c005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc5ce5753c48e6fe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.965+0000 7f2322a00640 1 -- 172.21.15.86:0/1567939678 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c043070 con 0x5627893d4fa0 2024-04-17T07:30:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.965+0000 7f2331a00640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23100a3c30 0x7f23100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.965+0000 7f2322a00640 1 -- 172.21.15.86:0/1567939678 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f231c003e20 con 0x5627893d4fa0 2024-04-17T07:30:24.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.965+0000 7f2322a00640 1 -- 172.21.15.86:0/1567939678 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c039040 con 0x5627893d4fa0 2024-04-17T07:30:24.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.966+0000 7f2331a00640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23100a3c30 0x7f23100a60f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2324099370 tx=0x7f2324009490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.966+0000 7f23371a8740 1 -- 172.21.15.86:0/1567939678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893d4fa0 msgr2=0x5627893d53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.966+0000 7f23371a8740 1 --2- 172.21.15.86:0/1567939678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893d4fa0 0x5627893d53c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f231c005e30 tx=0x7f231c005f80 comp rx=0 tx=0).stop 2024-04-17T07:30:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.966+0000 7f23371a8740 1 -- 172.21.15.86:0/1567939678 shutdown_connections 2024-04-17T07:30:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.966+0000 7f23371a8740 1 --2- 172.21.15.86:0/1567939678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893dab00 0x5627893dcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.966+0000 7f23371a8740 1 --2- 172.21.15.86:0/1567939678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893da490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.966+0000 7f23371a8740 1 --2- 172.21.15.86:0/1567939678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893d4fa0 0x5627893d53c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.966+0000 7f23371a8740 1 -- 172.21.15.86:0/1567939678 >> 172.21.15.86:0/1567939678 conn(0x5627893d0510 msgr2=0x5627893d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.967+0000 7f23371a8740 1 -- 172.21.15.86:0/1567939678 shutdown_connections 2024-04-17T07:30:24.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.967+0000 7f23371a8740 1 -- 172.21.15.86:0/1567939678 wait complete. 2024-04-17T07:30:24.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.967+0000 7f23371a8740 1 -- start start 2024-04-17T07:30:24.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.968+0000 7f23371a8740 1 Processor -- start 2024-04-17T07:30:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.968+0000 7f23371a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893df520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.968+0000 7f23371a8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.968+0000 7f23371a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x5627893e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.968+0000 7f23371a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5627893dda50 con 0x5627893d4fa0 2024-04-17T07:30:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.968+0000 7f23371a8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5627892f4830 con 0x5627893d5990 2024-04-17T07:30:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.968+0000 7f23371a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5627893ddbd0 con 0x5627893dab00 2024-04-17T07:30:24.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.968+0000 7f23371a8740 10 client.0 _ll_register_callbacks cb 0x5627892c2fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.969+0000 7f2332400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x5627893e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.969+0000 7f2332400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x5627893e6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33796/0 (socket says 172.21.15.86:33796) 2024-04-17T07:30:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.969+0000 7f2332400640 1 -- 172.21.15.86:0/667332027 learned_addr learned my addr 172.21.15.86:0/667332027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.969+0000 7f2332400640 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893df520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.969+0000 7f2331000640 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.969+0000 7f2332400640 1 -- 172.21.15.86:0/667332027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 msgr2=0x5627893dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.969+0000 7f2332400640 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893dfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.969+0000 7f2332400640 1 -- 172.21.15.86:0/667332027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 msgr2=0x5627893df520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.969+0000 7f2332400640 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893df520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.970+0000 7f2332400640 1 -- 172.21.15.86:0/667332027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627892f8920 con 0x5627893dab00 2024-04-17T07:30:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.970+0000 7f2332400640 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x5627893e6aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f231c005c40 tx=0x7f231c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6a379b672b6278 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:24.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.970+0000 7f2317400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:30:24.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.970+0000 7f2317400640 1 -- 172.21.15.86:0/667332027 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c043070 con 0x5627893dab00 2024-04-17T07:30:24.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.971+0000 7f23371a8740 1 -- 172.21.15.86:0/667332027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f231c003e20 con 0x5627893dab00 2024-04-17T07:30:24.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.971+0000 7f2317400640 1 -- 172.21.15.86:0/667332027 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f231c004410 con 0x5627893dab00 2024-04-17T07:30:24.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.971+0000 7f2317400640 1 -- 172.21.15.86:0/667332027 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f231c05a050 con 0x5627893dab00 2024-04-17T07:30:24.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.972+0000 7f23371a8740 10 client.28168 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:24.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.972+0000 7f23371a8740 1 -- 172.21.15.86:0/667332027 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5627892c67f0 con 0x5627893dab00 2024-04-17T07:30:24.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.973+0000 7f2317400640 1 -- 172.21.15.86:0/667332027 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f231c003e20 con 0x5627893dab00 2024-04-17T07:30:24.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.973+0000 7f2317400640 1 -- 172.21.15.86:0/667332027 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f231c048b90 con 0x5627893dab00 2024-04-17T07:30:24.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.973+0000 7f23371a8740 10 client.28168 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:24.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.973+0000 7f23371a8740 10 client.28168 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:30:24.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.973+0000 7f23371a8740 20 client.28168 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:24.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.973+0000 7f23371a8740 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5627893eac00 0x5627893ecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.973+0000 7f23371a8740 4 client.28168 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:30:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.973+0000 7f23371a8740 1 -- 172.21.15.86:0/667332027 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562789348a40 con 0x5627893eac00 2024-04-17T07:30:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.973+0000 7f2331000640 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5627893eac00 0x5627893ecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:24.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.975+0000 7f2331000640 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5627893eac00 0x5627893ecff0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f993c668c348cd7 server_cookie=1bb3b1b0f5df4e3b in_seq=0 out_seq=0 2024-04-17T07:30:24.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.975+0000 7f2317400640 10 client.28168 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:30:24.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.976+0000 7f2317400640 1 -- 172.21.15.86:0/667332027 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562789348a40 con 0x5627893eac00 2024-04-17T07:30:24.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.976+0000 7f2317400640 10 client.28168 handle_command_reply: tid=0 2024-04-17T07:30:24.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.977+0000 7f23371a8740 1 client.28168 shutdown 2024-04-17T07:30:24.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.977+0000 7f23371a8740 1 -- 172.21.15.86:0/667332027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 msgr2=0x5627893e6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:24.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.977+0000 7f23371a8740 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x5627893e6aa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f231c005c40 tx=0x7f231c005bd0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.977+0000 7f23371a8740 1 -- 172.21.15.86:0/667332027 shutdown_connections 2024-04-17T07:30:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.977+0000 7f23371a8740 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5627893eac00 0x5627893ecff0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.978+0000 7f23371a8740 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x5627893e6aa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.978+0000 7f23371a8740 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.978+0000 7f23371a8740 1 --2- 172.21.15.86:0/667332027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893df520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:24.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.978+0000 7f23371a8740 1 -- 172.21.15.86:0/667332027 >> 172.21.15.86:0/667332027 conn(0x5627893d0510 msgr2=0x5627893cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:24.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.978+0000 7f2317400640 0 client.28168 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:30:24.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.978+0000 7f23371a8740 1 -- 172.21.15.86:0/667332027 shutdown_connections 2024-04-17T07:30:24.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.978+0000 7f23371a8740 1 -- 172.21.15.86:0/667332027 wait complete. 2024-04-17T07:30:24.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:24.978+0000 7f23371a8740 20 client.28168 trim_cache size 0 max 16384 2024-04-17T07:30:25.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.005+0000 7f23371a8740 1 Processor -- start 2024-04-17T07:30:25.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.005+0000 7f23371a8740 1 -- start start 2024-04-17T07:30:25.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.005+0000 7f23371a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893ddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.005+0000 7f23371a8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.005+0000 7f23371a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x5627893de4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.005+0000 7f23371a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1b1160 con 0x5627893d4fa0 2024-04-17T07:30:25.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.005+0000 7f23371a8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1b2ca0 con 0x5627893d5990 2024-04-17T07:30:25.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.005+0000 7f23371a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1b1830 con 0x5627893dab00 2024-04-17T07:30:25.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2332400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893ddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2332400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893ddfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48852/0 (socket says 172.21.15.86:48852) 2024-04-17T07:30:25.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2332400640 1 -- 172.21.15.86:0/1309766947 learned_addr learned my addr 172.21.15.86:0/1309766947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:25.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2332400640 1 --2- 172.21.15.86:0/1309766947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x5627893de4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2331000640 1 --2- 172.21.15.86:0/1309766947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2332400640 1 -- 172.21.15.86:0/1309766947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 msgr2=0x5627893de4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2332400640 1 --2- 172.21.15.86:0/1309766947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x5627893de4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2332400640 1 -- 172.21.15.86:0/1309766947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 msgr2=0x5627893e6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2332400640 1 --2- 172.21.15.86:0/1309766947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893e6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2332400640 1 -- 172.21.15.86:0/1309766947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278904d3f0 con 0x5627893d4fa0 2024-04-17T07:30:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.006+0000 7f2331000640 1 --2- 172.21.15.86:0/1309766947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:30:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.007+0000 7f2332400640 1 --2- 172.21.15.86:0/1309766947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893ddfa0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f231c002230 tx=0x7f231c048f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a909a695cd0b624b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.007+0000 7f2322000640 1 -- 172.21.15.86:0/1309766947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c05a050 con 0x5627893d4fa0 2024-04-17T07:30:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.007+0000 7f2322000640 1 -- 172.21.15.86:0/1309766947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f231c003e20 con 0x5627893d4fa0 2024-04-17T07:30:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.007+0000 7f2322000640 1 -- 172.21.15.86:0/1309766947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c039040 con 0x5627893d4fa0 2024-04-17T07:30:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.008+0000 7f23371a8740 1 -- 172.21.15.86:0/1309766947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 msgr2=0x5627893ddfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.008+0000 7f23371a8740 1 --2- 172.21.15.86:0/1309766947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893ddfa0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f231c002230 tx=0x7f231c048f90 comp rx=0 tx=0).stop 2024-04-17T07:30:25.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.009+0000 7f23371a8740 1 -- 172.21.15.86:0/1309766947 shutdown_connections 2024-04-17T07:30:25.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.009+0000 7f23371a8740 1 --2- 172.21.15.86:0/1309766947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x5627893de4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.009+0000 7f23371a8740 1 --2- 172.21.15.86:0/1309766947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.009+0000 7f23371a8740 1 --2- 172.21.15.86:0/1309766947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893ddfa0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.009+0000 7f23371a8740 1 -- 172.21.15.86:0/1309766947 >> 172.21.15.86:0/1309766947 conn(0x5627893d0510 msgr2=0x5627893d6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.009+0000 7f23371a8740 1 -- 172.21.15.86:0/1309766947 shutdown_connections 2024-04-17T07:30:25.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.009+0000 7f23371a8740 1 -- 172.21.15.86:0/1309766947 wait complete. 2024-04-17T07:30:25.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.009+0000 7f23371a8740 1 -- start start 2024-04-17T07:30:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.010+0000 7f23371a8740 1 Processor -- start 2024-04-17T07:30:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.010+0000 7f23371a8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.010+0000 7f23371a8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.010+0000 7f23371a8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x56278944ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.010+0000 7f23371a8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1b25d0 con 0x5627893d4fa0 2024-04-17T07:30:25.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.010+0000 7f23371a8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1b1f00 con 0x5627893d5990 2024-04-17T07:30:25.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.010+0000 7f23371a8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1af0f0 con 0x5627893dab00 2024-04-17T07:30:25.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.010+0000 7f23371a8740 10 client.0 _ll_register_callbacks cb 0x7f231c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:25.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.010+0000 7f2332400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.011+0000 7f2332400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893e35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48862/0 (socket says 172.21.15.86:48862) 2024-04-17T07:30:25.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.011+0000 7f2332400640 1 -- 172.21.15.86:0/1453162392 learned_addr learned my addr 172.21.15.86:0/1453162392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:25.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.011+0000 7f2331000640 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.011+0000 7f2332400640 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x56278944ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.011+0000 7f2332400640 1 -- 172.21.15.86:0/1453162392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 msgr2=0x56278944ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.011+0000 7f2332400640 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x56278944ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.011+0000 7f2332400640 1 -- 172.21.15.86:0/1453162392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 msgr2=0x5627893e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.011+0000 7f2332400640 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893e3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.011+0000 7f2332400640 1 -- 172.21.15.86:0/1453162392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627891177d0 con 0x5627893d4fa0 2024-04-17T07:30:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.012+0000 7f2332400640 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893e35d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f231c0347c0 tx=0x7f231c02c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0508bd40eddfb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.012+0000 7f2317400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:30:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.012+0000 7f2317400640 1 -- 172.21.15.86:0/1453162392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c039030 con 0x5627893d4fa0 2024-04-17T07:30:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.012+0000 7f23371a8740 1 -- 172.21.15.86:0/1453162392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f231c003e20 con 0x5627893d4fa0 2024-04-17T07:30:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.012+0000 7f2317400640 1 -- 172.21.15.86:0/1453162392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f231c003fd0 con 0x5627893d4fa0 2024-04-17T07:30:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.013+0000 7f2317400640 1 -- 172.21.15.86:0/1453162392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f231c05a040 con 0x5627893d4fa0 2024-04-17T07:30:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.013+0000 7f23371a8740 10 client.19164 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:25.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.013+0000 7f23371a8740 1 -- 172.21.15.86:0/1453162392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562789450120 con 0x5627893d4fa0 2024-04-17T07:30:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.014+0000 7f2317400640 1 -- 172.21.15.86:0/1453162392 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f231c003e20 con 0x5627893d4fa0 2024-04-17T07:30:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.014+0000 7f2317400640 1 -- 172.21.15.86:0/1453162392 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f231c048910 con 0x5627893d4fa0 2024-04-17T07:30:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.014+0000 7f23371a8740 10 client.19164 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.014+0000 7f23371a8740 10 client.19164 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:30:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.014+0000 7f23371a8740 20 client.19164 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.014+0000 7f23371a8740 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562789450980 0x562789452da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.014+0000 7f23371a8740 4 client.19164 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:30:25.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.014+0000 7f23371a8740 1 -- 172.21.15.86:0/1453162392 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627893428b0 con 0x562789450980 2024-04-17T07:30:25.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.015+0000 7f2331000640 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562789450980 0x562789452da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.016+0000 7f2331000640 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562789450980 0x562789452da0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=203291632f0453f1 server_cookie=e185e70435ef8c9f in_seq=0 out_seq=0 2024-04-17T07:30:25.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.016+0000 7f2317400640 10 client.19164 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:30:25.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.016+0000 7f2317400640 1 -- 172.21.15.86:0/1453162392 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627893428b0 con 0x562789450980 2024-04-17T07:30:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.016+0000 7f2317400640 10 client.19164 handle_command_reply: tid=0 2024-04-17T07:30:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.016+0000 7f23371a8740 1 client.19164 shutdown 2024-04-17T07:30:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f23371a8740 1 -- 172.21.15.86:0/1453162392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 msgr2=0x5627893e35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f23371a8740 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893e35d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f231c0347c0 tx=0x7f231c02c100 comp rx=0 tx=0).stop 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f23371a8740 1 -- 172.21.15.86:0/1453162392 shutdown_connections 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f23371a8740 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x562789450980 0x562789452da0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f23371a8740 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5627893dab00 0x56278944ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f23371a8740 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5627893d5990 0x5627893e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f23371a8740 1 --2- 172.21.15.86:0/1453162392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5627893d4fa0 0x5627893e35d0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f23371a8740 1 -- 172.21.15.86:0/1453162392 >> 172.21.15.86:0/1453162392 conn(0x5627893d0510 msgr2=0x5627893d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f2317400640 0 client.19164 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f23371a8740 1 -- 172.21.15.86:0/1453162392 shutdown_connections 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.017+0000 7f23371a8740 1 -- 172.21.15.86:0/1453162392 wait complete. 2024-04-17T07:30:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.018+0000 7f23371a8740 20 client.19164 trim_cache size 0 max 16384 2024-04-17T07:30:25.019 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 -- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23100a3c30 msgr2=0x7f23100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23100a3c30 0x7f23100a60f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2324099370 tx=0x7f2324009490 comp rx=0 tx=0).stop 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 -- 172.21.15.86:0/1736666450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232c102a40 msgr2=0x7f232c195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232c102a40 0x7f232c195fe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f232400ea60 tx=0x7f232400ef20 comp rx=0 tx=0).stop 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 -- 172.21.15.86:0/1736666450 shutdown_connections 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f23100a3c30 0x7f23100a60f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f232c107ca0 0x7f232c1a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f232c105370 0x7f232c196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 --2- 172.21.15.86:0/1736666450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232c102a40 0x7f232c195fe0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 -- 172.21.15.86:0/1736666450 >> 172.21.15.86:0/1736666450 conn(0x7f232c0fc430 msgr2=0x7f232c102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 -- 172.21.15.86:0/1736666450 shutdown_connections 2024-04-17T07:30:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.019+0000 7f2334400640 1 -- 172.21.15.86:0/1736666450 wait complete. 2024-04-17T07:30:25.052 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:30:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.419+0000 7f70cde00640 1 -- 172.21.15.86:0/2711484867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c810e510 msgr2=0x7f70c81109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.419+0000 7f70cde00640 1 --2- 172.21.15.86:0/2711484867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c810e510 0x7f70c81109d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009be0 tx=0x7f70c002bef0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.420+0000 7f70cde00640 1 -- 172.21.15.86:0/2711484867 shutdown_connections 2024-04-17T07:30:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.420+0000 7f70cde00640 1 --2- 172.21.15.86:0/2711484867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c810e510 0x7f70c81109d0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.420+0000 7f70cde00640 1 --2- 172.21.15.86:0/2711484867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c810ba70 0x7f70c810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.420+0000 7f70cde00640 1 --2- 172.21.15.86:0/2711484867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70c8109140 0x7f70c810b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.420+0000 7f70cde00640 1 -- 172.21.15.86:0/2711484867 >> 172.21.15.86:0/2711484867 conn(0x7f70c806ee70 msgr2=0x7f70c80712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.422+0000 7f70cde00640 1 -- 172.21.15.86:0/2711484867 shutdown_connections 2024-04-17T07:30:25.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.423+0000 7f70cde00640 1 -- 172.21.15.86:0/2711484867 wait complete. 2024-04-17T07:30:25.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.426+0000 7f70cde00640 1 Processor -- start 2024-04-17T07:30:25.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.426+0000 7f70cde00640 1 -- start start 2024-04-17T07:30:25.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.426+0000 7f70cde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c8109140 0x7f70c81a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.426+0000 7f70cde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70c810ba70 0x7f70c81a7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.426+0000 7f70cde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c810e510 0x7f70c81ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.426+0000 7f70cde00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8113ff0 con 0x7f70c810e510 2024-04-17T07:30:25.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.427+0000 7f70cde00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8113e70 con 0x7f70c810ba70 2024-04-17T07:30:25.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.427+0000 7f70cde00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8114170 con 0x7f70c8109140 2024-04-17T07:30:25.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.427+0000 7f70c7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70c810ba70 0x7f70c81a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.427+0000 7f70c7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70c810ba70 0x7f70c81a7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37674/0 (socket says 172.21.15.86:37674) 2024-04-17T07:30:25.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.427+0000 7f70c7e00640 1 -- 172.21.15.86:0/1460051931 learned_addr learned my addr 172.21.15.86:0/1460051931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:25.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.428+0000 7f70c7e00640 1 -- 172.21.15.86:0/1460051931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c8109140 msgr2=0x7f70c81a6f40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T07:30:25.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.428+0000 7f70c7e00640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c8109140 0x7f70c81a6f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.428+0000 7f70c7e00640 1 -- 172.21.15.86:0/1460051931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c810e510 msgr2=0x7f70c81ae450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:30:25.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.430+0000 7f70c7e00640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c810e510 0x7f70c81ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.430+0000 7f70c7e00640 1 -- 172.21.15.86:0/1460051931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c81aeac0 con 0x7f70c810ba70 2024-04-17T07:30:25.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.431+0000 7f70c7e00640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70c810ba70 0x7f70c81a7480 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f70b400e9e0 tx=0x7f70b400eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.432+0000 7f70c5600640 1 -- 172.21.15.86:0/1460051931 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b400ccb0 con 0x7f70c810ba70 2024-04-17T07:30:25.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.432+0000 7f70cde00640 1 -- 172.21.15.86:0/1460051931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c81aedb0 con 0x7f70c810ba70 2024-04-17T07:30:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.433+0000 7f70cde00640 1 -- 172.21.15.86:0/1460051931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c81af270 con 0x7f70c810ba70 2024-04-17T07:30:25.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.433+0000 7f70c5600640 1 -- 172.21.15.86:0/1460051931 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70b400ce50 con 0x7f70c810ba70 2024-04-17T07:30:25.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.434+0000 7f70c5600640 1 -- 172.21.15.86:0/1460051931 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b400ccb0 con 0x7f70c810ba70 2024-04-17T07:30:25.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.438+0000 7f70c5600640 1 -- 172.21.15.86:0/1460051931 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f70b4016410 con 0x7f70c810ba70 2024-04-17T07:30:25.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.439+0000 7f70d2024740 1 Processor -- start 2024-04-17T07:30:25.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.440+0000 7f70c5600640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70b00a3e30 0x7f70b00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.441+0000 7f70c5600640 1 -- 172.21.15.86:0/1460051931 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f70b4005020 con 0x7f70c810ba70 2024-04-17T07:30:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.441+0000 7f70d2024740 1 -- start start 2024-04-17T07:30:25.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.441+0000 7f70d2024740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a61f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.441+0000 7f70d2024740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61fb50 0x561e4a624650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.441+0000 7f70d2024740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a624cc0 0x561e4a627150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.441+0000 7f70d2024740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e4a585020 con 0x561e4a61fb50 2024-04-17T07:30:25.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.441+0000 7f70d2024740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561e4a52b460 con 0x561e4a61f160 2024-04-17T07:30:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.441+0000 7f70d2024740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561e4a580fe0 con 0x561e4a624cc0 2024-04-17T07:30:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.442+0000 7f70cd400640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70b00a3e30 0x7f70b00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.442+0000 7f70cca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a61f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.442+0000 7f70cca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a61f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37676/0 (socket says 172.21.15.86:37676) 2024-04-17T07:30:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.442+0000 7f70cca00640 1 -- 172.21.15.86:0/3355263161 learned_addr learned my addr 172.21.15.86:0/3355263161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.442+0000 7f70cd400640 1 --2- 172.21.15.86:0/3355263161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a624cc0 0x561e4a627150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.443+0000 7f70cca00640 1 -- 172.21.15.86:0/3355263161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a624cc0 msgr2=0x561e4a627150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.443+0000 7f70cca00640 1 --2- 172.21.15.86:0/3355263161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a624cc0 0x561e4a627150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.443+0000 7f70cca00640 1 -- 172.21.15.86:0/3355263161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61fb50 msgr2=0x561e4a624650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.443+0000 7f70cca00640 1 --2- 172.21.15.86:0/3355263161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61fb50 0x561e4a624650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.443+0000 7f70cca00640 1 -- 172.21.15.86:0/3355263161 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e4a533040 con 0x561e4a61f160 2024-04-17T07:30:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.443+0000 7f70cd400640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70b00a3e30 0x7f70b00a62f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f70c00099a0 tx=0x7f70c0009930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.444+0000 7f70cca00640 1 --2- 172.21.15.86:0/3355263161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a61f580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009a60 tx=0x7f70bc009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=908202fc4d1e725 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.444+0000 7f70b9c00640 1 -- 172.21.15.86:0/3355263161 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc007b30 con 0x561e4a61f160 2024-04-17T07:30:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.444+0000 7f70b9c00640 1 -- 172.21.15.86:0/3355263161 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70bc007cd0 con 0x561e4a61f160 2024-04-17T07:30:25.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.444+0000 7f70b9c00640 1 -- 172.21.15.86:0/3355263161 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0125b0 con 0x561e4a61f160 2024-04-17T07:30:25.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.445+0000 7f70d2024740 1 -- 172.21.15.86:0/3355263161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 msgr2=0x561e4a61f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.445+0000 7f70d2024740 1 --2- 172.21.15.86:0/3355263161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a61f580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009a60 tx=0x7f70bc009f20 comp rx=0 tx=0).stop 2024-04-17T07:30:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.446+0000 7f70d2024740 1 -- 172.21.15.86:0/3355263161 shutdown_connections 2024-04-17T07:30:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.447+0000 7f70d2024740 1 --2- 172.21.15.86:0/3355263161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a624cc0 0x561e4a627150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.447+0000 7f70d2024740 1 --2- 172.21.15.86:0/3355263161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61fb50 0x561e4a624650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.447+0000 7f70d2024740 1 --2- 172.21.15.86:0/3355263161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a61f580 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.447+0000 7f70d2024740 1 -- 172.21.15.86:0/3355263161 >> 172.21.15.86:0/3355263161 conn(0x561e4a61a6d0 msgr2=0x561e4a61aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.447+0000 7f70d2024740 1 -- 172.21.15.86:0/3355263161 shutdown_connections 2024-04-17T07:30:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.447+0000 7f70d2024740 1 -- 172.21.15.86:0/3355263161 wait complete. 2024-04-17T07:30:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.448+0000 7f70d2024740 1 -- start start 2024-04-17T07:30:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.449+0000 7f70d2024740 1 Processor -- start 2024-04-17T07:30:25.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.450+0000 7f70d2024740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.450+0000 7f70d2024740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a629c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.450+0000 7f70d2024740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a624cc0 0x561e4a630c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.450+0000 7f70d2024740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561e4a627c70 con 0x561e4a61f160 2024-04-17T07:30:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.451+0000 7f70c7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a629c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.451+0000 7f70d2024740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561e4a53e830 con 0x561e4a624cc0 2024-04-17T07:30:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.451+0000 7f70d2024740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561e4a627df0 con 0x561e4a61fb50 2024-04-17T07:30:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.452+0000 7f70c7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a629c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33846/0 (socket says 172.21.15.86:33846) 2024-04-17T07:30:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.452+0000 7f70c7e00640 1 -- 172.21.15.86:0/3708372451 learned_addr learned my addr 172.21.15.86:0/3708372451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.452+0000 7f70cca00640 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.452+0000 7f70cd400640 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a624cc0 0x561e4a630c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.452+0000 7f70d2024740 10 client.0 _ll_register_callbacks cb 0x561e4a50cfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.452+0000 7f70c7e00640 1 -- 172.21.15.86:0/3708372451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a624cc0 msgr2=0x561e4a630c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.452+0000 7f70c7e00640 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a624cc0 0x561e4a630c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.452+0000 7f70c7e00640 1 -- 172.21.15.86:0/3708372451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61f160 msgr2=0x561e4a6296c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.452+0000 7f70c7e00640 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.452+0000 7f70c7e00640 1 -- 172.21.15.86:0/3708372451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e4a542920 con 0x561e4a61fb50 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.453+0000 7f70cca00640 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.453+0000 7f70c7e00640 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a629c00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004b00 tx=0x7f70b4004f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38959f7aec06d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.453+0000 7f70ae000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.454+0000 7f70ae000640 1 -- 172.21.15.86:0/3708372451 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b40b5020 con 0x561e4a61fb50 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.454+0000 7f70ae000640 1 -- 172.21.15.86:0/3708372451 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f70b40a57b0 con 0x561e4a61fb50 2024-04-17T07:30:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.454+0000 7f70d2024740 1 -- 172.21.15.86:0/3708372451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70bc007cd0 con 0x561e4a61fb50 2024-04-17T07:30:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.454+0000 7f70ae000640 1 -- 172.21.15.86:0/3708372451 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70b40a1380 con 0x561e4a61fb50 2024-04-17T07:30:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.455+0000 7f70d2024740 10 client.28183 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:25.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.455+0000 7f70d2024740 1 -- 172.21.15.86:0/3708372451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561e4a5107f0 con 0x561e4a61fb50 2024-04-17T07:30:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.456+0000 7f70ae000640 1 -- 172.21.15.86:0/3708372451 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70bc007cd0 con 0x561e4a61fb50 2024-04-17T07:30:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.456+0000 7f70ae000640 1 -- 172.21.15.86:0/3708372451 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f70b400b040 con 0x561e4a61fb50 2024-04-17T07:30:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.458+0000 7f70d2024740 10 client.28183 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.458+0000 7f70d2024740 10 client.28183 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:30:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.458+0000 7f70d2024740 20 client.28183 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.458+0000 7f70d2024740 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561e4a6350a0 0x561e4a637520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.458+0000 7f70d2024740 4 client.28183 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:30:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.458+0000 7f70d2024740 1 -- 172.21.15.86:0/3708372451 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e4a592a40 con 0x561e4a6350a0 2024-04-17T07:30:25.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.460+0000 7f70cca00640 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561e4a6350a0 0x561e4a637520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.462+0000 7f70cca00640 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561e4a6350a0 0x561e4a637520 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26a696a2eb3d9f10 server_cookie=865465ec048fd14 in_seq=0 out_seq=0 2024-04-17T07:30:25.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.462+0000 7f70ae000640 10 client.28183 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:30:25.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.463+0000 7f70ae000640 1 -- 172.21.15.86:0/3708372451 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e4a592a40 con 0x561e4a6350a0 2024-04-17T07:30:25.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.463+0000 7f70ae000640 10 client.28183 handle_command_reply: tid=0 2024-04-17T07:30:25.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.464+0000 7f70d2024740 1 client.28183 shutdown 2024-04-17T07:30:25.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.465+0000 7f70d2024740 1 -- 172.21.15.86:0/3708372451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 msgr2=0x561e4a629c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.465+0000 7f70d2024740 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a629c00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f70b4004b00 tx=0x7f70b4004f10 comp rx=0 tx=0).stop 2024-04-17T07:30:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.467+0000 7f70d2024740 1 -- 172.21.15.86:0/3708372451 shutdown_connections 2024-04-17T07:30:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.467+0000 7f70d2024740 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561e4a6350a0 0x561e4a637520 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.467+0000 7f70d2024740 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a624cc0 0x561e4a630c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.467+0000 7f70ae000640 0 client.28183 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:30:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.467+0000 7f70d2024740 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a629c00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.467+0000 7f70d2024740 1 --2- 172.21.15.86:0/3708372451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.467+0000 7f70d2024740 1 -- 172.21.15.86:0/3708372451 >> 172.21.15.86:0/3708372451 conn(0x561e4a61a6d0 msgr2=0x561e4a619930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.467+0000 7f70d2024740 1 -- 172.21.15.86:0/3708372451 shutdown_connections 2024-04-17T07:30:25.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.468+0000 7f70d2024740 1 -- 172.21.15.86:0/3708372451 wait complete. 2024-04-17T07:30:25.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.470+0000 7f70d2024740 20 client.28183 trim_cache size 0 max 16384 2024-04-17T07:30:25.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.509+0000 7f70d2024740 1 Processor -- start 2024-04-17T07:30:25.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.510+0000 7f70d2024740 1 -- start start 2024-04-17T07:30:25.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.510+0000 7f70d2024740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.510+0000 7f70d2024740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61fb50 0x561e4a630c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.510+0000 7f70d2024740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a624cc0 0x561e4a629c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.511+0000 7f70c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61fb50 0x561e4a630c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.511+0000 7f70d2024740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70a800fa90 con 0x561e4a61fb50 2024-04-17T07:30:25.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.511+0000 7f70d2024740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70a80115d0 con 0x561e4a624cc0 2024-04-17T07:30:25.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.511+0000 7f70d2024740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70a8010160 con 0x561e4a61f160 2024-04-17T07:30:25.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.511+0000 7f70cca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.511+0000 7f70cd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a624cc0 0x561e4a629c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.511+0000 7f70cca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33862/0 (socket says 172.21.15.86:33862) 2024-04-17T07:30:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.511+0000 7f70cca00640 1 -- 172.21.15.86:0/2038242013 learned_addr learned my addr 172.21.15.86:0/2038242013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.512+0000 7f70cca00640 1 -- 172.21.15.86:0/2038242013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a624cc0 msgr2=0x561e4a629c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.513+0000 7f70cca00640 1 --2- 172.21.15.86:0/2038242013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a624cc0 0x561e4a629c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.513+0000 7f70cca00640 1 -- 172.21.15.86:0/2038242013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61fb50 msgr2=0x561e4a630c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.513+0000 7f70cca00640 1 --2- 172.21.15.86:0/2038242013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61fb50 0x561e4a630c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.513+0000 7f70cca00640 1 -- 172.21.15.86:0/2038242013 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e4a6287c0 con 0x561e4a61f160 2024-04-17T07:30:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.513+0000 7f70c7e00640 1 --2- 172.21.15.86:0/2038242013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61fb50 0x561e4a630c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:30:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.513+0000 7f70cca00640 1 --2- 172.21.15.86:0/2038242013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f70bc007fd0 tx=0x7f70bc009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cebbdaec6357b269 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.514+0000 7f70b9200640 1 -- 172.21.15.86:0/2038242013 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc007b30 con 0x561e4a61f160 2024-04-17T07:30:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.514+0000 7f70b9200640 1 -- 172.21.15.86:0/2038242013 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70bc007cd0 con 0x561e4a61f160 2024-04-17T07:30:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.514+0000 7f70b9200640 1 -- 172.21.15.86:0/2038242013 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc013570 con 0x561e4a61f160 2024-04-17T07:30:25.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.515+0000 7f70d2024740 1 -- 172.21.15.86:0/2038242013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61f160 msgr2=0x561e4a6296c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.515+0000 7f70d2024740 1 --2- 172.21.15.86:0/2038242013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f70bc007fd0 tx=0x7f70bc009360 comp rx=0 tx=0).stop 2024-04-17T07:30:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.515+0000 7f70d2024740 1 -- 172.21.15.86:0/2038242013 shutdown_connections 2024-04-17T07:30:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.516+0000 7f70d2024740 1 --2- 172.21.15.86:0/2038242013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a624cc0 0x561e4a629c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.516+0000 7f70d2024740 1 --2- 172.21.15.86:0/2038242013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a61fb50 0x561e4a630c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.516+0000 7f70d2024740 1 --2- 172.21.15.86:0/2038242013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61f160 0x561e4a6296c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.516+0000 7f70d2024740 1 -- 172.21.15.86:0/2038242013 >> 172.21.15.86:0/2038242013 conn(0x561e4a61a6d0 msgr2=0x561e4a6210e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.516+0000 7f70d2024740 1 -- 172.21.15.86:0/2038242013 shutdown_connections 2024-04-17T07:30:25.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.516+0000 7f70d2024740 1 -- 172.21.15.86:0/2038242013 wait complete. 2024-04-17T07:30:25.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.517+0000 7f70d2024740 1 -- start start 2024-04-17T07:30:25.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.517+0000 7f70d2024740 1 Processor -- start 2024-04-17T07:30:25.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.518+0000 7f70d2024740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a62e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70d2024740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a62ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70d2024740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a624cc0 0x561e4a62fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70d2024740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70a8010f00 con 0x561e4a624cc0 2024-04-17T07:30:25.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70d2024740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70a8010830 con 0x561e4a61f160 2024-04-17T07:30:25.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70d2024740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70a800da60 con 0x561e4a61fb50 2024-04-17T07:30:25.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70d2024740 10 client.0 _ll_register_callbacks cb 0x7f70b409c930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70c7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a62ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70c7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a62ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33878/0 (socket says 172.21.15.86:33878) 2024-04-17T07:30:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70c7e00640 1 -- 172.21.15.86:0/1251959547 learned_addr learned my addr 172.21.15.86:0/1251959547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70cd400640 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a624cc0 0x561e4a62fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.519+0000 7f70cca00640 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a62e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.520+0000 7f70cca00640 1 -- 172.21.15.86:0/1251959547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 msgr2=0x561e4a62ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.520+0000 7f70cca00640 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a62ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.520+0000 7f70cca00640 1 -- 172.21.15.86:0/1251959547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a624cc0 msgr2=0x561e4a62fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.520+0000 7f70cca00640 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a624cc0 0x561e4a62fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.520+0000 7f70cca00640 1 -- 172.21.15.86:0/1251959547 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e4a2973f0 con 0x561e4a61f160 2024-04-17T07:30:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.520+0000 7f70cd400640 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a624cc0 0x561e4a62fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:30:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.521+0000 7f70cca00640 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a62e6d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f70bc000c00 tx=0x7f70bc007ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3421ddb8e6407b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.521+0000 7f70ae000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:30:25.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.521+0000 7f70ae000640 1 -- 172.21.15.86:0/1251959547 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc02d070 con 0x561e4a61f160 2024-04-17T07:30:25.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.521+0000 7f70d2024740 1 -- 172.21.15.86:0/1251959547 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70bc007cd0 con 0x561e4a61f160 2024-04-17T07:30:25.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.521+0000 7f70ae000640 1 -- 172.21.15.86:0/1251959547 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f70bc0263d0 con 0x561e4a61f160 2024-04-17T07:30:25.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.522+0000 7f70ae000640 1 -- 172.21.15.86:0/1251959547 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70bc0265d0 con 0x561e4a61f160 2024-04-17T07:30:25.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.522+0000 7f70d2024740 10 client.38138 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:25.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.522+0000 7f70d2024740 1 -- 172.21.15.86:0/1251959547 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561e4a3617d0 con 0x561e4a61f160 2024-04-17T07:30:25.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.523+0000 7f70ae000640 1 -- 172.21.15.86:0/1251959547 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70bc007cd0 con 0x561e4a61f160 2024-04-17T07:30:25.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.523+0000 7f70ae000640 1 -- 172.21.15.86:0/1251959547 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f70bc013810 con 0x561e4a61f160 2024-04-17T07:30:25.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.523+0000 7f70d2024740 10 client.38138 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:25.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.523+0000 7f70d2024740 10 client.38138 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:30:25.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.523+0000 7f70d2024740 20 client.38138 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.523+0000 7f70d2024740 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561e4a69be20 0x561e4a69e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.523+0000 7f70d2024740 4 client.38138 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:30:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.523+0000 7f70d2024740 1 -- 172.21.15.86:0/1251959547 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e4a58c8b0 con 0x561e4a69be20 2024-04-17T07:30:25.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.525+0000 7f70c7e00640 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561e4a69be20 0x561e4a69e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.527+0000 7f70c7e00640 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561e4a69be20 0x561e4a69e240 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5279acce1ad1164 server_cookie=9d617b6a60579943 in_seq=0 out_seq=0 2024-04-17T07:30:25.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.527+0000 7f70ae000640 10 client.38138 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:30:25.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.527+0000 7f70ae000640 1 -- 172.21.15.86:0/1251959547 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e4a58c8b0 con 0x561e4a69be20 2024-04-17T07:30:25.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.527+0000 7f70ae000640 10 client.38138 handle_command_reply: tid=0 2024-04-17T07:30:25.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.527+0000 7f70d2024740 1 client.38138 shutdown 2024-04-17T07:30:25.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.527+0000 7f70d2024740 1 -- 172.21.15.86:0/1251959547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 msgr2=0x561e4a62e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.527+0000 7f70d2024740 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a62e6d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f70bc000c00 tx=0x7f70bc007ba0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.528+0000 7f70d2024740 1 -- 172.21.15.86:0/1251959547 shutdown_connections 2024-04-17T07:30:25.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.528+0000 7f70d2024740 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561e4a624cc0 0x561e4a62fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.528+0000 7f70d2024740 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561e4a61fb50 0x561e4a62ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.528+0000 7f70d2024740 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x561e4a69be20 0x561e4a69e240 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.528+0000 7f70d2024740 1 --2- 172.21.15.86:0/1251959547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e4a61f160 0x561e4a62e6d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.528+0000 7f70d2024740 1 -- 172.21.15.86:0/1251959547 >> 172.21.15.86:0/1251959547 conn(0x561e4a61a6d0 msgr2=0x561e4a6295e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.528+0000 7f70ae000640 0 client.38138 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:30:25.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.528+0000 7f70d2024740 1 -- 172.21.15.86:0/1251959547 shutdown_connections 2024-04-17T07:30:25.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.529+0000 7f70d2024740 1 -- 172.21.15.86:0/1251959547 wait complete. 2024-04-17T07:30:25.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.529+0000 7f70d2024740 20 client.38138 trim_cache size 0 max 16384 2024-04-17T07:30:25.530 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:30:25.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 -- 172.21.15.86:0/1460051931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70b00a3e30 msgr2=0x7f70b00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70b00a3e30 0x7f70b00a62f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f70c00099a0 tx=0x7f70c0009930 comp rx=0 tx=0).stop 2024-04-17T07:30:25.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 -- 172.21.15.86:0/1460051931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70c810ba70 msgr2=0x7f70c81a7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70c810ba70 0x7f70c81a7480 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f70b400e9e0 tx=0x7f70b400eea0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 -- 172.21.15.86:0/1460051931 shutdown_connections 2024-04-17T07:30:25.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f70b00a3e30 0x7f70b00a62f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c810e510 0x7f70c81ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70c810ba70 0x7f70c81a7480 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 --2- 172.21.15.86:0/1460051931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70c8109140 0x7f70c81a6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 -- 172.21.15.86:0/1460051931 >> 172.21.15.86:0/1460051931 conn(0x7f70c806ee70 msgr2=0x7f70c810b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 -- 172.21.15.86:0/1460051931 shutdown_connections 2024-04-17T07:30:25.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.530+0000 7f70cde00640 1 -- 172.21.15.86:0/1460051931 wait complete. 2024-04-17T07:30:25.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:30:25.672 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:25 smithi086 ceph-mon[31733]: pgmap v921: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.3 MiB/s wr, 635 op/s 2024-04-17T07:30:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:25 smithi098 ceph-mon[35418]: pgmap v921: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.3 MiB/s wr, 635 op/s 2024-04-17T07:30:25.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:25 smithi103 ceph-mon[35495]: pgmap v921: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.3 MiB/s wr, 635 op/s 2024-04-17T07:30:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.904+0000 7f8e3a600640 1 -- 172.21.15.86:0/1097016798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2c00f500 msgr2=0x7f8e2c013dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.904+0000 7f8e3a600640 1 --2- 172.21.15.86:0/1097016798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2c00f500 0x7f8e2c013dc0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009900 tx=0x7f8e2402bc50 comp rx=0 tx=0).stop 2024-04-17T07:30:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.904+0000 7f8e3a600640 1 -- 172.21.15.86:0/1097016798 shutdown_connections 2024-04-17T07:30:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.904+0000 7f8e3a600640 1 --2- 172.21.15.86:0/1097016798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2c00f500 0x7f8e2c013dc0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.904+0000 7f8e3a600640 1 --2- 172.21.15.86:0/1097016798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e2c00eb60 0x7f8e2c00efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.904+0000 7f8e3a600640 1 --2- 172.21.15.86:0/1097016798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e2c00d920 0x7f8e2c00dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.904+0000 7f8e3a600640 1 -- 172.21.15.86:0/1097016798 >> 172.21.15.86:0/1097016798 conn(0x7f8e2c00b390 msgr2=0x7f8e2c00b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.907+0000 7f8e3a600640 1 -- 172.21.15.86:0/1097016798 shutdown_connections 2024-04-17T07:30:25.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.907+0000 7f8e3a600640 1 -- 172.21.15.86:0/1097016798 wait complete. 2024-04-17T07:30:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.911+0000 7f8e3a600640 1 Processor -- start 2024-04-17T07:30:25.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.913+0000 7f8e3a600640 1 -- start start 2024-04-17T07:30:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.914+0000 7f8e3a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2c00d920 0x7f8e2c13ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.914+0000 7f8e3a600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e2c00eb60 0x7f8e2c13b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.914+0000 7f8e3a600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e2c00f500 0x7f8e2c1422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.914+0000 7f8e3a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c0b0710 con 0x7f8e2c00d920 2024-04-17T07:30:25.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.914+0000 7f8e3a600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c0b0590 con 0x7f8e2c00f500 2024-04-17T07:30:25.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.914+0000 7f8e3a600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c0b0890 con 0x7f8e2c00eb60 2024-04-17T07:30:25.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.914+0000 7f8e39c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e2c00f500 0x7f8e2c1422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.914+0000 7f8e39c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e2c00f500 0x7f8e2c1422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37704/0 (socket says 172.21.15.86:37704) 2024-04-17T07:30:25.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.914+0000 7f8e39c00640 1 -- 172.21.15.86:0/3376964749 learned_addr learned my addr 172.21.15.86:0/3376964749 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:25.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.915+0000 7f8e39200640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2c00d920 0x7f8e2c13ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.915+0000 7f8e39200640 1 -- 172.21.15.86:0/3376964749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e2c00eb60 msgr2=0x7f8e2c13b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.915+0000 7f8e39200640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e2c00eb60 0x7f8e2c13b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.915+0000 7f8e39200640 1 -- 172.21.15.86:0/3376964749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e2c00f500 msgr2=0x7f8e2c1422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.915+0000 7f8e39200640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e2c00f500 0x7f8e2c1422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.916+0000 7f8e39200640 1 -- 172.21.15.86:0/3376964749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c142960 con 0x7f8e2c00d920 2024-04-17T07:30:25.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.917+0000 7f8e39200640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2c00d920 0x7f8e2c13ada0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8e340515f0 tx=0x7f8e340775f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.917+0000 7f8e31600640 1 -- 172.21.15.86:0/3376964749 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34081070 con 0x7f8e2c00d920 2024-04-17T07:30:25.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.918+0000 7f8e31600640 1 -- 172.21.15.86:0/3376964749 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e34077e50 con 0x7f8e2c00d920 2024-04-17T07:30:25.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.918+0000 7f8e31600640 1 -- 172.21.15.86:0/3376964749 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3407b710 con 0x7f8e2c00d920 2024-04-17T07:30:25.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.919+0000 7f8e3a600640 1 -- 172.21.15.86:0/3376964749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2c142c50 con 0x7f8e2c00d920 2024-04-17T07:30:25.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.919+0000 7f8e3a600640 1 -- 172.21.15.86:0/3376964749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2c143120 con 0x7f8e2c00d920 2024-04-17T07:30:25.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.926+0000 7f8e3e965740 1 Processor -- start 2024-04-17T07:30:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.926+0000 7f8e3e965740 1 -- start start 2024-04-17T07:30:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.927+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 0x561400f9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.927+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9eb50 0x561400fa3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.927+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400fa3cc0 0x561400fa6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.927+0000 7f8e3e965740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561400f04020 con 0x561400f9eb50 2024-04-17T07:30:25.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.927+0000 7f8e3e965740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561400eaa460 con 0x561400fa3cc0 2024-04-17T07:30:25.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.927+0000 7f8e3e965740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561400efffe0 con 0x561400f9e160 2024-04-17T07:30:25.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.928+0000 7f8e33e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400fa3cc0 0x561400fa6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.928+0000 7f8e33e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400fa3cc0 0x561400fa6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37714/0 (socket says 172.21.15.86:37714) 2024-04-17T07:30:25.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.928+0000 7f8e33e00640 1 -- 172.21.15.86:0/467920446 learned_addr learned my addr 172.21.15.86:0/467920446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:25.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.928+0000 7f8e39c00640 1 --2- 172.21.15.86:0/467920446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 0x561400f9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.929+0000 7f8e39c00640 1 -- 172.21.15.86:0/467920446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400fa3cc0 msgr2=0x561400fa6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.929+0000 7f8e39c00640 1 --2- 172.21.15.86:0/467920446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400fa3cc0 0x561400fa6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.929+0000 7f8e39c00640 1 -- 172.21.15.86:0/467920446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9eb50 msgr2=0x561400fa3650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:30:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.929+0000 7f8e39c00640 1 --2- 172.21.15.86:0/467920446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9eb50 0x561400fa3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.929+0000 7f8e39c00640 1 -- 172.21.15.86:0/467920446 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561400eb2040 con 0x561400f9e160 2024-04-17T07:30:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.930+0000 7f8e39c00640 1 --2- 172.21.15.86:0/467920446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 0x561400f9e580 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8e2402c860 tx=0x7f8e2402cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=201b82db5bea8889 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.932+0000 7f8e22000640 1 -- 172.21.15.86:0/467920446 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24045020 con 0x561400f9e160 2024-04-17T07:30:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.932+0000 7f8e22000640 1 -- 172.21.15.86:0/467920446 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e24005040 con 0x561400f9e160 2024-04-17T07:30:25.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.932+0000 7f8e22000640 1 -- 172.21.15.86:0/467920446 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24004920 con 0x561400f9e160 2024-04-17T07:30:25.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.935+0000 7f8e3e965740 1 -- 172.21.15.86:0/467920446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 msgr2=0x561400f9e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.935+0000 7f8e3e965740 1 --2- 172.21.15.86:0/467920446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 0x561400f9e580 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8e2402c860 tx=0x7f8e2402cb30 comp rx=0 tx=0).stop 2024-04-17T07:30:25.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.935+0000 7f8e31600640 1 -- 172.21.15.86:0/3376964749 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8e34084070 con 0x7f8e2c00d920 2024-04-17T07:30:25.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.937+0000 7f8e3e965740 1 -- 172.21.15.86:0/467920446 shutdown_connections 2024-04-17T07:30:25.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.937+0000 7f8e3e965740 1 --2- 172.21.15.86:0/467920446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400fa3cc0 0x561400fa6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.937+0000 7f8e3e965740 1 --2- 172.21.15.86:0/467920446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9eb50 0x561400fa3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.937+0000 7f8e3e965740 1 --2- 172.21.15.86:0/467920446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 0x561400f9e580 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.937+0000 7f8e3e965740 1 -- 172.21.15.86:0/467920446 >> 172.21.15.86:0/467920446 conn(0x561400f996d0 msgr2=0x561400f99ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.937+0000 7f8e3e965740 1 -- 172.21.15.86:0/467920446 shutdown_connections 2024-04-17T07:30:25.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.937+0000 7f8e3e965740 1 -- 172.21.15.86:0/467920446 wait complete. 2024-04-17T07:30:25.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.938+0000 7f8e3e965740 1 -- start start 2024-04-17T07:30:25.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.939+0000 7f8e3e965740 1 Processor -- start 2024-04-17T07:30:25.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.940+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 0x561400fa8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.940+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fa8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.940+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 0x561400fafcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.940+0000 7f8e3e965740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561400fa6c10 con 0x561400f9e160 2024-04-17T07:30:25.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.940+0000 7f8e3e965740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561400ebd830 con 0x561400f9eb50 2024-04-17T07:30:25.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.940+0000 7f8e3e965740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561400fa6d90 con 0x561400fa3cc0 2024-04-17T07:30:25.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.940+0000 7f8e3e965740 10 client.0 _ll_register_callbacks cb 0x561400e8bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.941+0000 7f8e39c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 0x561400fa8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.941+0000 7f8e39200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fa8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.941+0000 7f8e39c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 0x561400fa8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48924/0 (socket says 172.21.15.86:48924) 2024-04-17T07:30:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.941+0000 7f8e39c00640 1 -- 172.21.15.86:0/1681477841 learned_addr learned my addr 172.21.15.86:0/1681477841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.941+0000 7f8e39200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fa8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37726/0 (socket says 172.21.15.86:37726) 2024-04-17T07:30:25.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.941+0000 7f8e31600640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8e1c0a3b90 0x7f8e1c0a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.941+0000 7f8e31600640 1 -- 172.21.15.86:0/3376964749 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8e3413c840 con 0x7f8e2c00d920 2024-04-17T07:30:25.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.942+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 0x561400fafcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.942+0000 7f8e39200640 1 -- 172.21.15.86:0/1681477841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 msgr2=0x561400fafcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.942+0000 7f8e39200640 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 0x561400fafcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.942+0000 7f8e39200640 1 -- 172.21.15.86:0/1681477841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 msgr2=0x561400fa8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:25.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.942+0000 7f8e39200640 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 0x561400fa8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.942+0000 7f8e39200640 1 -- 172.21.15.86:0/1681477841 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561400ec1920 con 0x561400f9eb50 2024-04-17T07:30:25.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.944+0000 7f8e39c00640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8e1c0a3b90 0x7f8e1c0a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.944+0000 7f8e16a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:30:25.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.944+0000 7f8e39200640 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fa8c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x561400f9f5c0 tx=0x7f8e34068160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=680a79e5ef69a173 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.944+0000 7f8e39c00640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8e1c0a3b90 0x7f8e1c0a6050 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8e2402c230 tx=0x7f8e2402ca10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:25.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.945+0000 7f8e16a00640 1 -- 172.21.15.86:0/1681477841 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e341087a0 con 0x561400f9eb50 2024-04-17T07:30:25.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.946+0000 7f8e3e965740 1 -- 172.21.15.86:0/1681477841 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e24005040 con 0x561400f9eb50 2024-04-17T07:30:25.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.946+0000 7f8e16a00640 1 -- 172.21.15.86:0/1681477841 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8e34107bf0 con 0x561400f9eb50 2024-04-17T07:30:25.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.947+0000 7f8e16a00640 1 -- 172.21.15.86:0/1681477841 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e3410cb50 con 0x561400f9eb50 2024-04-17T07:30:25.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.950+0000 7f8e3e965740 10 client.38147 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:25.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.950+0000 7f8e3e965740 1 -- 172.21.15.86:0/1681477841 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561400e8f7f0 con 0x561400f9eb50 2024-04-17T07:30:25.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.950+0000 7f8e16a00640 1 -- 172.21.15.86:0/1681477841 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e24005040 con 0x561400f9eb50 2024-04-17T07:30:25.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.951+0000 7f8e16a00640 1 -- 172.21.15.86:0/1681477841 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8e3410dce0 con 0x561400f9eb50 2024-04-17T07:30:25.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.951+0000 7f8e3e965740 10 client.38147 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:25.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.951+0000 7f8e3e965740 10 client.38147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:30:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.951+0000 7f8e3e965740 20 client.38147 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.951+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561400fb40b0 0x561400fb64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.951+0000 7f8e3e965740 4 client.38147 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:30:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.951+0000 7f8e3e965740 1 -- 172.21.15.86:0/1681477841 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561400f11a40 con 0x561400fb40b0 2024-04-17T07:30:25.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.952+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561400fb40b0 0x561400fb64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.953+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561400fb40b0 0x561400fb64f0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb93e170c51f7fcc server_cookie=8a7c01027c5eebd1 in_seq=0 out_seq=0 2024-04-17T07:30:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.953+0000 7f8e16a00640 10 client.38147 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.954+0000 7f8e16a00640 1 -- 172.21.15.86:0/1681477841 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561400f11a40 con 0x561400fb40b0 2024-04-17T07:30:25.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.954+0000 7f8e16a00640 10 client.38147 handle_command_reply: tid=0 2024-04-17T07:30:25.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.955+0000 7f8e3e965740 1 client.38147 shutdown 2024-04-17T07:30:25.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.955+0000 7f8e3e965740 1 -- 172.21.15.86:0/1681477841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 msgr2=0x561400fa8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:25.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.956+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fa8c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x561400f9f5c0 tx=0x7f8e34068160 comp rx=0 tx=0).stop 2024-04-17T07:30:25.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.956+0000 7f8e3e965740 1 -- 172.21.15.86:0/1681477841 shutdown_connections 2024-04-17T07:30:25.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.956+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561400fb40b0 0x561400fb64f0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.956+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 0x561400fafcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.956+0000 7f8e16a00640 0 client.38147 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:25.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.956+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fa8c70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.956+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1681477841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 0x561400fa8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:25.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.956+0000 7f8e3e965740 1 -- 172.21.15.86:0/1681477841 >> 172.21.15.86:0/1681477841 conn(0x561400f996d0 msgr2=0x561400f98930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:25.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.956+0000 7f8e3e965740 1 -- 172.21.15.86:0/1681477841 shutdown_connections 2024-04-17T07:30:25.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.957+0000 7f8e3e965740 1 -- 172.21.15.86:0/1681477841 wait complete. 2024-04-17T07:30:25.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:25.957+0000 7f8e3e965740 20 client.38147 trim_cache size 0 max 16384 2024-04-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.012+0000 7f8e3e965740 1 Processor -- start 2024-04-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.012+0000 7f8e3e965740 1 -- start start 2024-04-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.012+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 0x561400fa00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.013+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fafc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.013+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400fa3cc0 0x561400fa05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.014+0000 7f8e39c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400fa3cc0 0x561400fa05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.013+0000 7f8e39200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 0x561400fa00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.014+0000 7f8e3e965740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c14d4d0 con 0x561400fa3cc0 2024-04-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.014+0000 7f8e3e965740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c14f010 con 0x561400f9eb50 2024-04-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.014+0000 7f8e3e965740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c14dba0 con 0x561400f9e160 2024-04-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.014+0000 7f8e39200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 0x561400fa00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33906/0 (socket says 172.21.15.86:33906) 2024-04-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.014+0000 7f8e39200640 1 -- 172.21.15.86:0/122424894 learned_addr learned my addr 172.21.15.86:0/122424894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.014+0000 7f8e33e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fafc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.015+0000 7f8e39c00640 1 -- 172.21.15.86:0/122424894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 msgr2=0x561400fa00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.015+0000 7f8e39c00640 1 --2- 172.21.15.86:0/122424894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 0x561400fa00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.015+0000 7f8e39c00640 1 -- 172.21.15.86:0/122424894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 msgr2=0x561400fafc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.015+0000 7f8e39c00640 1 --2- 172.21.15.86:0/122424894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fafc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.015+0000 7f8e39c00640 1 -- 172.21.15.86:0/122424894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561400c163f0 con 0x561400fa3cc0 2024-04-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.016+0000 7f8e39c00640 1 --2- 172.21.15.86:0/122424894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400fa3cc0 0x561400fa05f0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009850 tx=0x7f8e2402cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1d50a9f6c0feaa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.018+0000 7f8e21600640 1 -- 172.21.15.86:0/122424894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24045040 con 0x561400fa3cc0 2024-04-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.018+0000 7f8e21600640 1 -- 172.21.15.86:0/122424894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e24004490 con 0x561400fa3cc0 2024-04-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.018+0000 7f8e21600640 1 -- 172.21.15.86:0/122424894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24004dc0 con 0x561400fa3cc0 2024-04-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.020+0000 7f8e3e965740 1 -- 172.21.15.86:0/122424894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400fa3cc0 msgr2=0x561400fa05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.020+0000 7f8e3e965740 1 --2- 172.21.15.86:0/122424894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400fa3cc0 0x561400fa05f0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009850 tx=0x7f8e2402cce0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.020+0000 7f8e3e965740 1 -- 172.21.15.86:0/122424894 shutdown_connections 2024-04-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.020+0000 7f8e3e965740 1 --2- 172.21.15.86:0/122424894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400fa3cc0 0x561400fa05f0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.020+0000 7f8e3e965740 1 --2- 172.21.15.86:0/122424894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fafc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.021+0000 7f8e3e965740 1 --2- 172.21.15.86:0/122424894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400f9e160 0x561400fa00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.021+0000 7f8e3e965740 1 -- 172.21.15.86:0/122424894 >> 172.21.15.86:0/122424894 conn(0x561400f996d0 msgr2=0x561400fa8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.021+0000 7f8e3e965740 1 -- 172.21.15.86:0/122424894 shutdown_connections 2024-04-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.021+0000 7f8e3e965740 1 -- 172.21.15.86:0/122424894 wait complete. 2024-04-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.022+0000 7f8e3e965740 1 -- start start 2024-04-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.022+0000 7f8e3e965740 1 Processor -- start 2024-04-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.023+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 0x561400facd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.023+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.023+0000 7f8e3e965740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 0x561400fae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.023+0000 7f8e3e965740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c14e940 con 0x561400f9e160 2024-04-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.023+0000 7f8e3e965740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c14e270 con 0x561400f9eb50 2024-04-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.023+0000 7f8e3e965740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c14b4a0 con 0x561400fa3cc0 2024-04-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.023+0000 7f8e3e965740 10 client.0 _ll_register_callbacks cb 0x7f8e34108570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.024+0000 7f8e39c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 0x561400fae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.024+0000 7f8e39c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 0x561400fae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:33912/0 (socket says 172.21.15.86:33912) 2024-04-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.024+0000 7f8e39c00640 1 -- 172.21.15.86:0/1449798197 learned_addr learned my addr 172.21.15.86:0/1449798197 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.024+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.024+0000 7f8e39200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 0x561400facd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.025+0000 7f8e33e00640 1 -- 172.21.15.86:0/1449798197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 msgr2=0x561400fae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.025+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 0x561400fae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.025+0000 7f8e33e00640 1 -- 172.21.15.86:0/1449798197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 msgr2=0x561400facd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.025+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 0x561400facd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.025+0000 7f8e33e00640 1 -- 172.21.15.86:0/1449798197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561400ce07d0 con 0x561400f9eb50 2024-04-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.025+0000 7f8e39c00640 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 0x561400fae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.026+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fad270 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8e28000c00 tx=0x7f8e2801af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e28209e804c52f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.026+0000 7f8e16a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.028+0000 7f8e16a00640 1 -- 172.21.15.86:0/1449798197 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28005460 con 0x561400f9eb50 2024-04-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.028+0000 7f8e16a00640 1 -- 172.21.15.86:0/1449798197 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8e28005600 con 0x561400f9eb50 2024-04-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.028+0000 7f8e3e965740 1 -- 172.21.15.86:0/1449798197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e24004490 con 0x561400f9eb50 2024-04-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.030+0000 7f8e16a00640 1 -- 172.21.15.86:0/1449798197 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e2802c050 con 0x561400f9eb50 2024-04-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.030+0000 7f8e3e965740 10 client.38153 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.030+0000 7f8e3e965740 1 -- 172.21.15.86:0/1449798197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561400faf970 con 0x561400f9eb50 2024-04-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.032+0000 7f8e16a00640 1 -- 172.21.15.86:0/1449798197 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e24004490 con 0x561400f9eb50 2024-04-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.032+0000 7f8e16a00640 1 -- 172.21.15.86:0/1449798197 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8e28021020 con 0x561400f9eb50 2024-04-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.032+0000 7f8e3e965740 10 client.38153 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.032+0000 7f8e3e965740 10 client.38153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.032+0000 7f8e3e965740 20 client.38153 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.032+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56140101a750 0x56140101cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.033+0000 7f8e3e965740 4 client.38153 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.033+0000 7f8e3e965740 1 -- 172.21.15.86:0/1449798197 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561400f0b8b0 con 0x56140101a750 2024-04-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.034+0000 7f8e39200640 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56140101a750 0x56140101cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.038+0000 7f8e39200640 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56140101a750 0x56140101cb70 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943a179c92e24c53 server_cookie=17ba9f255b628b55 in_seq=0 out_seq=0 2024-04-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.038+0000 7f8e16a00640 10 client.38153 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.038+0000 7f8e16a00640 1 -- 172.21.15.86:0/1449798197 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561400f0b8b0 con 0x56140101a750 2024-04-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.038+0000 7f8e16a00640 10 client.38153 handle_command_reply: tid=0 2024-04-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.039+0000 7f8e3e965740 1 client.38153 shutdown 2024-04-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.040+0000 7f8e3e965740 1 -- 172.21.15.86:0/1449798197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 msgr2=0x561400fad270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.040+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fad270 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8e28000c00 tx=0x7f8e2801af90 comp rx=0 tx=0).stop 2024-04-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.042+0000 7f8e3e965740 1 -- 172.21.15.86:0/1449798197 shutdown_connections 2024-04-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.042+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56140101a750 0x56140101cb70 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.042+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561400fa3cc0 0x561400fae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.042+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561400f9eb50 0x561400fad270 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.042+0000 7f8e3e965740 1 --2- 172.21.15.86:0/1449798197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561400f9e160 0x561400facd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.042+0000 7f8e3e965740 1 -- 172.21.15.86:0/1449798197 >> 172.21.15.86:0/1449798197 conn(0x561400f996d0 msgr2=0x561400fa71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.043+0000 7f8e16a00640 0 client.38153 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.043+0000 7f8e3e965740 1 -- 172.21.15.86:0/1449798197 shutdown_connections 2024-04-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.044+0000 7f8e3e965740 1 -- 172.21.15.86:0/1449798197 wait complete. 2024-04-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.044+0000 7f8e3e965740 20 client.38153 trim_cache size 0 max 16384 2024-04-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.046+0000 7f8e3a600640 1 -- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8e1c0a3b90 msgr2=0x7f8e1c0a6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.046+0000 7f8e3a600640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8e1c0a3b90 0x7f8e1c0a6050 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8e2402c230 tx=0x7f8e2402ca10 comp rx=0 tx=0).stop 2024-04-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.046+0000 7f8e3a600640 1 -- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2c00d920 msgr2=0x7f8e2c13ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.046+0000 7f8e3a600640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2c00d920 0x7f8e2c13ada0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8e340515f0 tx=0x7f8e340775f0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.048+0000 7f8e3a600640 1 -- 172.21.15.86:0/3376964749 shutdown_connections 2024-04-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.048+0000 7f8e3a600640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8e1c0a3b90 0x7f8e1c0a6050 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.048+0000 7f8e3a600640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e2c00f500 0x7f8e2c1422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.048+0000 7f8e3a600640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e2c00eb60 0x7f8e2c13b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.048+0000 7f8e3a600640 1 --2- 172.21.15.86:0/3376964749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2c00d920 0x7f8e2c13ada0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.048+0000 7f8e3a600640 1 -- 172.21.15.86:0/3376964749 >> 172.21.15.86:0/3376964749 conn(0x7f8e2c00b390 msgr2=0x7f8e2c0ace40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.048+0000 7f8e3a600640 1 -- 172.21.15.86:0/3376964749 shutdown_connections 2024-04-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:26.049+0000 7f8e3a600640 1 -- 172.21.15.86:0/3376964749 wait complete. 2024-04-17T07:30:27.096 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:30:27.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:30:27.097 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:30:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.467+0000 7fcf6f200640 1 -- 172.21.15.86:0/458281901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf70109640 msgr2=0x7fcf70109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.467+0000 7fcf6f200640 1 --2- 172.21.15.86:0/458281901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf70109640 0x7fcf70109aa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009900 tx=0x7fcf6002bc30 comp rx=0 tx=0).stop 2024-04-17T07:30:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.467+0000 7fcf6f200640 1 -- 172.21.15.86:0/458281901 shutdown_connections 2024-04-17T07:30:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.468+0000 7fcf6f200640 1 --2- 172.21.15.86:0/458281901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf70109fe0 0x7fcf7010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.468+0000 7fcf6f200640 1 --2- 172.21.15.86:0/458281901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf70109640 0x7fcf70109aa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.468+0000 7fcf6f200640 1 --2- 172.21.15.86:0/458281901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf700733a0 0x7fcf700737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.468+0000 7fcf6f200640 1 -- 172.21.15.86:0/458281901 >> 172.21.15.86:0/458281901 conn(0x7fcf7006ee70 msgr2=0x7fcf700712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.468+0000 7fcf6f200640 1 -- 172.21.15.86:0/458281901 shutdown_connections 2024-04-17T07:30:27.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.468+0000 7fcf6f200640 1 -- 172.21.15.86:0/458281901 wait complete. 2024-04-17T07:30:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.469+0000 7fcf6f200640 1 Processor -- start 2024-04-17T07:30:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.469+0000 7fcf6f200640 1 -- start start 2024-04-17T07:30:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.470+0000 7fcf6f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf700733a0 0x7fcf701a6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.470+0000 7fcf6f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf70109640 0x7fcf701a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.470+0000 7fcf6f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf70109fe0 0x7fcf701ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.470+0000 7fcf6f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7011c7f0 con 0x7fcf700733a0 2024-04-17T07:30:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.470+0000 7fcf6f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7011c670 con 0x7fcf70109640 2024-04-17T07:30:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.470+0000 7fcf6f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7011c970 con 0x7fcf70109fe0 2024-04-17T07:30:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.470+0000 7fcf6d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf70109640 0x7fcf701a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.470+0000 7fcf6d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf70109640 0x7fcf701a73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43338/0 (socket says 172.21.15.86:43338) 2024-04-17T07:30:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.470+0000 7fcf6d400640 1 -- 172.21.15.86:0/3906272748 learned_addr learned my addr 172.21.15.86:0/3906272748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.471+0000 7fcf6e800640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf70109fe0 0x7fcf701ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.471+0000 7fcf6d400640 1 -- 172.21.15.86:0/3906272748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf70109fe0 msgr2=0x7fcf701ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.471+0000 7fcf6d400640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf70109fe0 0x7fcf701ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.471+0000 7fcf6d400640 1 -- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf700733a0 msgr2=0x7fcf701a6eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:30:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.471+0000 7fcf6de00640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf700733a0 0x7fcf701a6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.472+0000 7fcf6d400640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf700733a0 0x7fcf701a6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.472+0000 7fcf6d400640 1 -- 172.21.15.86:0/3906272748 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf701aead0 con 0x7fcf70109640 2024-04-17T07:30:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.472+0000 7fcf6de00640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf700733a0 0x7fcf701a6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:30:27.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.472+0000 7fcf6d400640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf70109640 0x7fcf701a73f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcf60000c00 tx=0x7fcf60004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.473+0000 7fcf66a00640 1 -- 172.21.15.86:0/3906272748 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60043020 con 0x7fcf70109640 2024-04-17T07:30:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.473+0000 7fcf66a00640 1 -- 172.21.15.86:0/3906272748 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf600052b0 con 0x7fcf70109640 2024-04-17T07:30:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.473+0000 7fcf66a00640 1 -- 172.21.15.86:0/3906272748 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60003c70 con 0x7fcf70109640 2024-04-17T07:30:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.473+0000 7fcf6f200640 1 -- 172.21.15.86:0/3906272748 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf701aed60 con 0x7fcf70109640 2024-04-17T07:30:27.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.473+0000 7fcf6f200640 1 -- 172.21.15.86:0/3906272748 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf701af200 con 0x7fcf70109640 2024-04-17T07:30:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.479+0000 7fcf66a00640 1 -- 172.21.15.86:0/3906272748 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fcf60046020 con 0x7fcf70109640 2024-04-17T07:30:27.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.479+0000 7fcf7796f740 1 Processor -- start 2024-04-17T07:30:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.480+0000 7fcf7796f740 1 -- start start 2024-04-17T07:30:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.480+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41def90a0 0x55f41def94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.480+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41defe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.480+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41defec00 0x55f41df01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.480+0000 7fcf7796f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f41de5ecb0 con 0x55f41def90a0 2024-04-17T07:30:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.480+0000 7fcf7796f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f41de05370 con 0x55f41defec00 2024-04-17T07:30:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.480+0000 7fcf7796f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f41de5ac70 con 0x55f41def9a90 2024-04-17T07:30:27.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.481+0000 7fcf6de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41defe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.481+0000 7fcf6de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41defe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:35652/0 (socket says 172.21.15.86:35652) 2024-04-17T07:30:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.481+0000 7fcf6de00640 1 -- 172.21.15.86:0/1345362105 learned_addr learned my addr 172.21.15.86:0/1345362105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.482+0000 7fcf6d400640 1 --2- 172.21.15.86:0/1345362105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41defec00 0x55f41df01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.482+0000 7fcf6e800640 1 --2- 172.21.15.86:0/1345362105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41def90a0 0x55f41def94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.482+0000 7fcf66a00640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf540a3c80 0x7fcf540a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.483+0000 7fcf66a00640 1 -- 172.21.15.86:0/3906272748 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcf600f2a80 con 0x7fcf70109640 2024-04-17T07:30:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.483+0000 7fcf6de00640 1 -- 172.21.15.86:0/1345362105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41defec00 msgr2=0x55f41df01090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.483+0000 7fcf6de00640 1 --2- 172.21.15.86:0/1345362105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41defec00 0x55f41df01090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.483+0000 7fcf6de00640 1 -- 172.21.15.86:0/1345362105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41def90a0 msgr2=0x55f41def94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.483+0000 7fcf6de00640 1 --2- 172.21.15.86:0/1345362105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41def90a0 0x55f41def94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.483+0000 7fcf6de00640 1 -- 172.21.15.86:0/1345362105 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f41de24910 con 0x55f41def9a90 2024-04-17T07:30:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.483+0000 7fcf6e800640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf540a3c80 0x7fcf540a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.485+0000 7fcf6e800640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf540a3c80 0x7fcf540a6140 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55f41defa2f0 tx=0x7fcf68004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.486+0000 7fcf6de00640 1 --2- 172.21.15.86:0/1345362105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41defe590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009c50 tx=0x7fcf5c00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccfb6cf7e975546 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.487+0000 7fcf5b000640 1 -- 172.21.15.86:0/1345362105 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c016070 con 0x55f41def9a90 2024-04-17T07:30:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.487+0000 7fcf5b000640 1 -- 172.21.15.86:0/1345362105 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf5c0054d0 con 0x55f41def9a90 2024-04-17T07:30:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.487+0000 7fcf5b000640 1 -- 172.21.15.86:0/1345362105 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c011470 con 0x55f41def9a90 2024-04-17T07:30:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.489+0000 7fcf7796f740 1 -- 172.21.15.86:0/1345362105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 msgr2=0x55f41defe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.489+0000 7fcf7796f740 1 --2- 172.21.15.86:0/1345362105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41defe590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009c50 tx=0x7fcf5c00e6c0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.490+0000 7fcf7796f740 1 -- 172.21.15.86:0/1345362105 shutdown_connections 2024-04-17T07:30:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.490+0000 7fcf7796f740 1 --2- 172.21.15.86:0/1345362105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41defec00 0x55f41df01090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.490+0000 7fcf7796f740 1 --2- 172.21.15.86:0/1345362105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41defe590 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.490+0000 7fcf7796f740 1 --2- 172.21.15.86:0/1345362105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41def90a0 0x55f41def94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.490+0000 7fcf7796f740 1 -- 172.21.15.86:0/1345362105 >> 172.21.15.86:0/1345362105 conn(0x55f41def4610 msgr2=0x55f41def4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:27.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.490+0000 7fcf7796f740 1 -- 172.21.15.86:0/1345362105 shutdown_connections 2024-04-17T07:30:27.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.491+0000 7fcf7796f740 1 -- 172.21.15.86:0/1345362105 wait complete. 2024-04-17T07:30:27.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.492+0000 7fcf7796f740 1 -- start start 2024-04-17T07:30:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.492+0000 7fcf7796f740 1 Processor -- start 2024-04-17T07:30:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.493+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41def90a0 0x55f41df14660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.494+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df14ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.494+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41defec00 0x55f41df1bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.494+0000 7fcf7796f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f41df12bc0 con 0x55f41def90a0 2024-04-17T07:30:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.494+0000 7fcf7796f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f41de18730 con 0x55f41defec00 2024-04-17T07:30:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.494+0000 7fcf7796f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f41df12d40 con 0x55f41def9a90 2024-04-17T07:30:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.494+0000 7fcf7796f740 10 client.0 _ll_register_callbacks cb 0x55f41de1ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.495+0000 7fcf6d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df14ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.495+0000 7fcf6e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41defec00 0x55f41df1bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.495+0000 7fcf6de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41def90a0 0x55f41df14660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.495+0000 7fcf6d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df14ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:35656/0 (socket says 172.21.15.86:35656) 2024-04-17T07:30:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.495+0000 7fcf6d400640 1 -- 172.21.15.86:0/17957641 learned_addr learned my addr 172.21.15.86:0/17957641 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.496+0000 7fcf6d400640 1 -- 172.21.15.86:0/17957641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41defec00 msgr2=0x55f41df1bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.496+0000 7fcf6d400640 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41defec00 0x55f41df1bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.496+0000 7fcf6d400640 1 -- 172.21.15.86:0/17957641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41def90a0 msgr2=0x55f41df14660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.496+0000 7fcf6d400640 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41def90a0 0x55f41df14660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.496+0000 7fcf6d400640 1 -- 172.21.15.86:0/17957641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f41dc397e0 con 0x55f41def9a90 2024-04-17T07:30:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.496+0000 7fcf6d400640 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df14ba0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcf60003820 tx=0x7fcf60005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a9939fc98f79712 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.496+0000 7fcf4f400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:30:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.497+0000 7fcf4f400640 1 -- 172.21.15.86:0/17957641 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600035f0 con 0x55f41def9a90 2024-04-17T07:30:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.497+0000 7fcf4f400640 1 -- 172.21.15.86:0/17957641 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcf600c70d0 con 0x55f41def9a90 2024-04-17T07:30:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.497+0000 7fcf7796f740 1 -- 172.21.15.86:0/17957641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5c0054d0 con 0x55f41def9a90 2024-04-17T07:30:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.498+0000 7fcf4f400640 1 -- 172.21.15.86:0/17957641 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcf600df0b0 con 0x55f41def9a90 2024-04-17T07:30:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.498+0000 7fcf7796f740 10 client.28213 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.498+0000 7fcf7796f740 1 -- 172.21.15.86:0/17957641 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f41dc99f70 con 0x55f41def9a90 2024-04-17T07:30:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.498+0000 7fcf4f400640 1 -- 172.21.15.86:0/17957641 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf5c0054d0 con 0x55f41def9a90 2024-04-17T07:30:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.498+0000 7fcf4f400640 1 -- 172.21.15.86:0/17957641 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcf600c72d0 con 0x55f41def9a90 2024-04-17T07:30:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.499+0000 7fcf7796f740 10 client.28213 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.499+0000 7fcf7796f740 10 client.28213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:30:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.499+0000 7fcf7796f740 20 client.28213 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:27.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.499+0000 7fcf7796f740 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f41df20460 0x55f41df22900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.499+0000 7fcf7796f740 4 client.28213 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:30:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.499+0000 7fcf7796f740 1 -- 172.21.15.86:0/17957641 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f41de6c6d0 con 0x55f41df20460 2024-04-17T07:30:27.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.500+0000 7fcf6de00640 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f41df20460 0x55f41df22900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.503+0000 7fcf6de00640 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f41df20460 0x55f41df22900 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e343e1a96e1c711a server_cookie=1636736c48ce3b4d in_seq=0 out_seq=0 2024-04-17T07:30:27.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.503+0000 7fcf4f400640 10 client.28213 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.504+0000 7fcf4f400640 1 -- 172.21.15.86:0/17957641 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f41de6c6d0 con 0x55f41df20460 2024-04-17T07:30:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.504+0000 7fcf4f400640 10 client.28213 handle_command_reply: tid=0 2024-04-17T07:30:27.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.504+0000 7fcf7796f740 1 client.28213 shutdown 2024-04-17T07:30:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.504+0000 7fcf7796f740 1 -- 172.21.15.86:0/17957641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 msgr2=0x55f41df14ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.504+0000 7fcf7796f740 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df14ba0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcf60003820 tx=0x7fcf60005bc0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.504+0000 7fcf7796f740 1 -- 172.21.15.86:0/17957641 shutdown_connections 2024-04-17T07:30:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.504+0000 7fcf7796f740 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f41df20460 0x55f41df22900 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.504+0000 7fcf7796f740 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41defec00 0x55f41df1bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.504+0000 7fcf7796f740 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df14ba0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.504+0000 7fcf7796f740 1 --2- 172.21.15.86:0/17957641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41def90a0 0x55f41df14660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.505+0000 7fcf7796f740 1 -- 172.21.15.86:0/17957641 >> 172.21.15.86:0/17957641 conn(0x55f41def4610 msgr2=0x55f41def3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:27.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.505+0000 7fcf4f400640 0 client.28213 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:27.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.505+0000 7fcf7796f740 1 -- 172.21.15.86:0/17957641 shutdown_connections 2024-04-17T07:30:27.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.505+0000 7fcf7796f740 1 -- 172.21.15.86:0/17957641 wait complete. 2024-04-17T07:30:27.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.506+0000 7fcf7796f740 20 client.28213 trim_cache size 0 max 16384 2024-04-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.557+0000 7fcf7796f740 1 Processor -- start 2024-04-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.557+0000 7fcf7796f740 1 -- start start 2024-04-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.558+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 0x55f41def3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.558+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df1bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.558+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 0x55f41def3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.558+0000 7fcf7796f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5000fc70 con 0x55f41defec00 2024-04-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.558+0000 7fcf7796f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcf500117b0 con 0x55f41def90a0 2024-04-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.558+0000 7fcf7796f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50010340 con 0x55f41def9a90 2024-04-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.558+0000 7fcf6d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df1bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.559+0000 7fcf6d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df1bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:35672/0 (socket says 172.21.15.86:35672) 2024-04-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.559+0000 7fcf6d400640 1 -- 172.21.15.86:0/672634496 learned_addr learned my addr 172.21.15.86:0/672634496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.559+0000 7fcf6de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 0x55f41def3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.559+0000 7fcf6e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 0x55f41def3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.560+0000 7fcf6d400640 1 -- 172.21.15.86:0/672634496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 msgr2=0x55f41def3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.560+0000 7fcf6d400640 1 --2- 172.21.15.86:0/672634496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 0x55f41def3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.560+0000 7fcf6d400640 1 -- 172.21.15.86:0/672634496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 msgr2=0x55f41def3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.560+0000 7fcf6d400640 1 --2- 172.21.15.86:0/672634496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 0x55f41def3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.560+0000 7fcf6d400640 1 -- 172.21.15.86:0/672634496 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f41db31170 con 0x55f41def9a90 2024-04-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.562+0000 7fcf6d400640 1 --2- 172.21.15.86:0/672634496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df1bba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcf600096c0 tx=0x7fcf600045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1edb1e5f8be2490e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.563+0000 7fcf5a600640 1 -- 172.21.15.86:0/672634496 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600df0b0 con 0x55f41def9a90 2024-04-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.563+0000 7fcf5a600640 1 -- 172.21.15.86:0/672634496 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf600f9020 con 0x55f41def9a90 2024-04-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.563+0000 7fcf5a600640 1 -- 172.21.15.86:0/672634496 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600be910 con 0x55f41def9a90 2024-04-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.565+0000 7fcf7796f740 1 -- 172.21.15.86:0/672634496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 msgr2=0x55f41df1bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.565+0000 7fcf7796f740 1 --2- 172.21.15.86:0/672634496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df1bba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcf600096c0 tx=0x7fcf600045f0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.565+0000 7fcf7796f740 1 -- 172.21.15.86:0/672634496 shutdown_connections 2024-04-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.565+0000 7fcf7796f740 1 --2- 172.21.15.86:0/672634496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 0x55f41def3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.565+0000 7fcf7796f740 1 --2- 172.21.15.86:0/672634496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df1bba0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.565+0000 7fcf7796f740 1 --2- 172.21.15.86:0/672634496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 0x55f41def3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.565+0000 7fcf7796f740 1 -- 172.21.15.86:0/672634496 >> 172.21.15.86:0/672634496 conn(0x55f41def4610 msgr2=0x55f41defb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.567+0000 7fcf7796f740 1 -- 172.21.15.86:0/672634496 shutdown_connections 2024-04-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.567+0000 7fcf7796f740 1 -- 172.21.15.86:0/672634496 wait complete. 2024-04-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.568+0000 7fcf7796f740 1 -- start start 2024-04-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.568+0000 7fcf7796f740 1 Processor -- start 2024-04-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.569+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 0x55f41df19480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.569+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.569+0000 7fcf7796f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 0x55f41df1a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.569+0000 7fcf7796f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf500110e0 con 0x55f41defec00 2024-04-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.569+0000 7fcf7796f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50010a10 con 0x55f41def90a0 2024-04-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.569+0000 7fcf7796f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5000dc00 con 0x55f41def9a90 2024-04-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.569+0000 7fcf7796f740 10 client.0 _ll_register_callbacks cb 0x7fcf600f6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.570+0000 7fcf6d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.570+0000 7fcf6de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 0x55f41df19480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.570+0000 7fcf6de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 0x55f41df19480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43368/0 (socket says 172.21.15.86:43368) 2024-04-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.570+0000 7fcf6de00640 1 -- 172.21.15.86:0/1639186008 learned_addr learned my addr 172.21.15.86:0/1639186008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.571+0000 7fcf6e800640 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 0x55f41df1a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.571+0000 7fcf6d400640 1 -- 172.21.15.86:0/1639186008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 msgr2=0x55f41df19480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.571+0000 7fcf6d400640 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 0x55f41df19480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.571+0000 7fcf6d400640 1 -- 172.21.15.86:0/1639186008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 msgr2=0x55f41df1a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.571+0000 7fcf6d400640 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 0x55f41df1a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.571+0000 7fcf6d400640 1 -- 172.21.15.86:0/1639186008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f41df1afc0 con 0x55f41def9a90 2024-04-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.571+0000 7fcf6e800640 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 0x55f41df1a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.572+0000 7fcf6d400640 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df199c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcf60003820 tx=0x7fcf600f0280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e127e0867d89185 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.572+0000 7fcf4f400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.572+0000 7fcf4f400640 1 -- 172.21.15.86:0/1639186008 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600df0b0 con 0x55f41def9a90 2024-04-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.572+0000 7fcf7796f740 1 -- 172.21.15.86:0/1639186008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf600f9020 con 0x55f41def9a90 2024-04-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.573+0000 7fcf4f400640 1 -- 172.21.15.86:0/1639186008 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcf600c7070 con 0x55f41def9a90 2024-04-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.574+0000 7fcf4f400640 1 -- 172.21.15.86:0/1639186008 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcf600be910 con 0x55f41def9a90 2024-04-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.574+0000 7fcf7796f740 10 client.28219 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.574+0000 7fcf7796f740 1 -- 172.21.15.86:0/1639186008 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f41df1afc0 con 0x55f41def9a90 2024-04-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.574+0000 7fcf4f400640 1 -- 172.21.15.86:0/1639186008 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf600f9020 con 0x55f41def9a90 2024-04-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.575+0000 7fcf4f400640 1 -- 172.21.15.86:0/1639186008 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcf600c7270 con 0x55f41def9a90 2024-04-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.575+0000 7fcf7796f740 10 client.28219 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.575+0000 7fcf7796f740 10 client.28219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.575+0000 7fcf7796f740 20 client.28219 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.575+0000 7fcf7796f740 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f41df872a0 0x55f41df896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.575+0000 7fcf7796f740 4 client.28219 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.575+0000 7fcf7796f740 1 -- 172.21.15.86:0/1639186008 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f41de66540 con 0x55f41df872a0 2024-04-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.576+0000 7fcf6de00640 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f41df872a0 0x55f41df896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.577+0000 7fcf4f400640 10 client.28219 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.577+0000 7fcf6de00640 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f41df872a0 0x55f41df896c0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa9d26d5bc8af68 server_cookie=5d68c9cb9f8f1f62 in_seq=0 out_seq=0 2024-04-17T07:30:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:27 smithi086 ceph-mon[31733]: pgmap v922: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 625 op/s 2024-04-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.581+0000 7fcf4f400640 1 -- 172.21.15.86:0/1639186008 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f41de66540 con 0x55f41df872a0 2024-04-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.581+0000 7fcf4f400640 10 client.28219 handle_command_reply: tid=0 2024-04-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.581+0000 7fcf7796f740 1 client.28219 shutdown 2024-04-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.583+0000 7fcf7796f740 1 -- 172.21.15.86:0/1639186008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 msgr2=0x55f41df199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.583+0000 7fcf7796f740 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df199c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcf60003820 tx=0x7fcf600f0280 comp rx=0 tx=0).stop 2024-04-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.583+0000 7fcf7796f740 1 -- 172.21.15.86:0/1639186008 shutdown_connections 2024-04-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.583+0000 7fcf7796f740 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55f41df872a0 0x55f41df896c0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.583+0000 7fcf4f400640 0 client.28219 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.583+0000 7fcf7796f740 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f41defec00 0x55f41df1a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.583+0000 7fcf7796f740 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f41def9a90 0x55f41df199c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.583+0000 7fcf7796f740 1 --2- 172.21.15.86:0/1639186008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f41def90a0 0x55f41df19480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.583+0000 7fcf7796f740 1 -- 172.21.15.86:0/1639186008 >> 172.21.15.86:0/1639186008 conn(0x55f41def4610 msgr2=0x55f41def7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.584+0000 7fcf7796f740 1 -- 172.21.15.86:0/1639186008 shutdown_connections 2024-04-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.584+0000 7fcf7796f740 1 -- 172.21.15.86:0/1639186008 wait complete. 2024-04-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.584+0000 7fcf7796f740 20 client.28219 trim_cache size 0 max 16384 2024-04-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "e35f2c5c-699d-4418-86ad-ba45c51ed36c", 2024-04-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.585+0000 7fcf6f200640 1 -- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf540a3c80 msgr2=0x7fcf540a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.586+0000 7fcf6f200640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf540a3c80 0x7fcf540a6140 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55f41defa2f0 tx=0x7fcf68004a40 comp rx=0 tx=0).stop 2024-04-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.586+0000 7fcf6f200640 1 -- 172.21.15.86:0/3906272748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf70109640 msgr2=0x7fcf701a73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.586+0000 7fcf6f200640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf70109640 0x7fcf701a73f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcf60000c00 tx=0x7fcf60004270 comp rx=0 tx=0).stop 2024-04-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.586+0000 7fcf6f200640 1 -- 172.21.15.86:0/3906272748 shutdown_connections 2024-04-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.586+0000 7fcf6f200640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcf540a3c80 0x7fcf540a6140 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.586+0000 7fcf6f200640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf70109fe0 0x7fcf701ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.586+0000 7fcf6f200640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf70109640 0x7fcf701a73f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.586+0000 7fcf6f200640 1 --2- 172.21.15.86:0/3906272748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf700733a0 0x7fcf701a6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.586+0000 7fcf6f200640 1 -- 172.21.15.86:0/3906272748 >> 172.21.15.86:0/3906272748 conn(0x7fcf7006ee70 msgr2=0x7fcf70070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.587+0000 7fcf6f200640 1 -- 172.21.15.86:0/3906272748 shutdown_connections 2024-04-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:27.587+0000 7fcf6f200640 1 -- 172.21.15.86:0/3906272748 wait complete. 2024-04-17T07:30:27.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:30:27.623 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:30:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:27 smithi098 ceph-mon[35418]: pgmap v922: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 625 op/s 2024-04-17T07:30:27.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:27 smithi103 ceph-mon[35495]: pgmap v922: 129 pgs: 129 active+clean; 1.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 625 op/s 2024-04-17T07:30:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.015+0000 7f65f7c00640 1 -- 172.21.15.86:0/575646731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65f0101fb0 msgr2=0x7f65f01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.015+0000 7f65f7c00640 1 --2- 172.21.15.86:0/575646731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65f0101fb0 0x7f65f01023d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009ca0 tx=0x7f65e802c100 comp rx=0 tx=0).stop 2024-04-17T07:30:28.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.016+0000 7f65f7c00640 1 -- 172.21.15.86:0/575646731 shutdown_connections 2024-04-17T07:30:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.016+0000 7f65f7c00640 1 --2- 172.21.15.86:0/575646731 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f0103b60 0x7f65f010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.016+0000 7f65f7c00640 1 --2- 172.21.15.86:0/575646731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f01031a0 0x7f65f0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.016+0000 7f65f7c00640 1 --2- 172.21.15.86:0/575646731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65f0101fb0 0x7f65f01023d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.016+0000 7f65f7c00640 1 -- 172.21.15.86:0/575646731 >> 172.21.15.86:0/575646731 conn(0x7f65f00fd540 msgr2=0x7f65f00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.016+0000 7f65f7c00640 1 -- 172.21.15.86:0/575646731 shutdown_connections 2024-04-17T07:30:28.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.016+0000 7f65f7c00640 1 -- 172.21.15.86:0/575646731 wait complete. 2024-04-17T07:30:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.019+0000 7f65f7c00640 1 Processor -- start 2024-04-17T07:30:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.019+0000 7f65f7c00640 1 -- start start 2024-04-17T07:30:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.019+0000 7f65f7c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f0101fb0 0x7f65f0075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.019+0000 7f65f7c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65f01031a0 0x7f65f0074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.019+0000 7f65f7c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0103b60 0x7f65f0074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.019+0000 7f65f7c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65f010db40 con 0x7f65f0103b60 2024-04-17T07:30:28.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.019+0000 7f65f7c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65f010d9c0 con 0x7f65f01031a0 2024-04-17T07:30:28.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.019+0000 7f65f7c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65f010dcc0 con 0x7f65f0101fb0 2024-04-17T07:30:28.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.020+0000 7f65efe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65f01031a0 0x7f65f0074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.020+0000 7f65f5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f0101fb0 0x7f65f0075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.021+0000 7f65f5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f0101fb0 0x7f65f0075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:35694/0 (socket says 172.21.15.86:35694) 2024-04-17T07:30:28.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.021+0000 7f65f5200640 1 -- 172.21.15.86:0/4015248985 learned_addr learned my addr 172.21.15.86:0/4015248985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.021+0000 7f65f5c00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0103b60 0x7f65f0074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.022+0000 7f65efe00640 1 -- 172.21.15.86:0/4015248985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f0101fb0 msgr2=0x7f65f0075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.022+0000 7f65efe00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f0101fb0 0x7f65f0075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.022+0000 7f65efe00640 1 -- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0103b60 msgr2=0x7f65f0074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:28.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.022+0000 7f65efe00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0103b60 0x7f65f0074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.022+0000 7f65efe00640 1 -- 172.21.15.86:0/4015248985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f0074dd0 con 0x7f65f01031a0 2024-04-17T07:30:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.022+0000 7f65f5c00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0103b60 0x7f65f0074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:30:28.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.023+0000 7f65efe00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65f01031a0 0x7f65f0074350 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00e9f0 tx=0x7f65dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.024+0000 7f65ed600640 1 -- 172.21.15.86:0/4015248985 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00ccc0 con 0x7f65f01031a0 2024-04-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.024+0000 7f65ed600640 1 -- 172.21.15.86:0/4015248985 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65dc0108e0 con 0x7f65f01031a0 2024-04-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.024+0000 7f65ed600640 1 -- 172.21.15.86:0/4015248985 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc018880 con 0x7f65f01031a0 2024-04-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.025+0000 7f65f7c00640 1 -- 172.21.15.86:0/4015248985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f00750c0 con 0x7f65f01031a0 2024-04-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.025+0000 7f65f7c00640 1 -- 172.21.15.86:0/4015248985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f01a6ab0 con 0x7f65f01031a0 2024-04-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.032+0000 7f65fa987740 1 Processor -- start 2024-04-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.033+0000 7f65fa987740 1 -- start start 2024-04-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.034+0000 7f65ed600640 1 -- 172.21.15.86:0/4015248985 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f65dc01f020 con 0x7f65f01031a0 2024-04-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.034+0000 7f65fa987740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 0x557ce1eb55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.034+0000 7f65fa987740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1eba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.034+0000 7f65fa987740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1ebacf0 0x557ce1ebd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.034+0000 7f65fa987740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ce1e1c120 con 0x557ce1ebacf0 2024-04-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.034+0000 7f65fa987740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557ce1dc2340 con 0x557ce1eb5190 2024-04-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.034+0000 7f65fa987740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ce1e180e0 con 0x557ce1eb5b80 2024-04-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.034+0000 7f65f5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1eba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.035+0000 7f65f5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1eba680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:35708/0 (socket says 172.21.15.86:35708) 2024-04-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.035+0000 7f65f5200640 1 -- 172.21.15.86:0/3361734639 learned_addr learned my addr 172.21.15.86:0/3361734639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.035+0000 7f65f5c00640 1 --2- 172.21.15.86:0/3361734639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 0x557ce1eb55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.037+0000 7f65ed600640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f65d40a3d30 0x7f65d40a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.037+0000 7f65ed600640 1 -- 172.21.15.86:0/4015248985 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f65dc0d0820 con 0x7f65f01031a0 2024-04-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.038+0000 7f65f5c00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f65d40a3d30 0x7f65d40a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.038+0000 7f65f5c00640 1 -- 172.21.15.86:0/3361734639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 msgr2=0x557ce1eba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.038+0000 7f65f5c00640 1 --2- 172.21.15.86:0/3361734639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1eba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.038+0000 7f65f5c00640 1 -- 172.21.15.86:0/3361734639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1ebacf0 msgr2=0x557ce1ebd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.038+0000 7f65f5c00640 1 --2- 172.21.15.86:0/3361734639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1ebacf0 0x557ce1ebd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.038+0000 7f65f5c00640 1 -- 172.21.15.86:0/3361734639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce1bf87d0 con 0x557ce1eb5190 2024-04-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.039+0000 7f65f5c00640 1 --2- 172.21.15.86:0/3361734639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 0x557ce1eb55b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65e0011af0 tx=0x7f65e000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a0e4269742f192 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.039+0000 7f65e6000640 1 -- 172.21.15.86:0/3361734639 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e001a040 con 0x557ce1eb5190 2024-04-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.040+0000 7f65f5c00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f65d40a3d30 0x7f65d40a61f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f65e0023000 tx=0x7f65e0020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.040+0000 7f65e6000640 1 -- 172.21.15.86:0/3361734639 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65e0004c50 con 0x557ce1eb5190 2024-04-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.040+0000 7f65e6000640 1 -- 172.21.15.86:0/3361734639 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e00035f0 con 0x557ce1eb5190 2024-04-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.041+0000 7f65fa987740 1 -- 172.21.15.86:0/3361734639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 msgr2=0x557ce1eb55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.041+0000 7f65fa987740 1 --2- 172.21.15.86:0/3361734639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 0x557ce1eb55b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65e0011af0 tx=0x7f65e000f3c0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.042+0000 7f65fa987740 1 -- 172.21.15.86:0/3361734639 shutdown_connections 2024-04-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.042+0000 7f65fa987740 1 --2- 172.21.15.86:0/3361734639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1ebacf0 0x557ce1ebd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.042+0000 7f65fa987740 1 --2- 172.21.15.86:0/3361734639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1eba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.042+0000 7f65fa987740 1 --2- 172.21.15.86:0/3361734639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 0x557ce1eb55b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.042+0000 7f65fa987740 1 -- 172.21.15.86:0/3361734639 >> 172.21.15.86:0/3361734639 conn(0x557ce1eb0700 msgr2=0x557ce1eb0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.042+0000 7f65fa987740 1 -- 172.21.15.86:0/3361734639 shutdown_connections 2024-04-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.042+0000 7f65fa987740 1 -- 172.21.15.86:0/3361734639 wait complete. 2024-04-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.044+0000 7f65fa987740 1 -- start start 2024-04-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.044+0000 7f65fa987740 1 Processor -- start 2024-04-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.045+0000 7f65fa987740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5190 0x557ce1ebf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.045+0000 7f65fa987740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5b80 0x557ce1ebfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.045+0000 7f65fa987740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 0x557ce1ec6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.045+0000 7f65fa987740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ce1ebdca0 con 0x557ce1eb5b80 2024-04-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.045+0000 7f65fa987740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557ce1dd5710 con 0x557ce1ebacf0 2024-04-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.045+0000 7f65fa987740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ce1ebde20 con 0x557ce1eb5190 2024-04-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.045+0000 7f65fa987740 10 client.0 _ll_register_callbacks cb 0x557ce1dd0af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.046+0000 7f65f5c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 0x557ce1ec6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.046+0000 7f65f5c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 0x557ce1ec6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:43402/0 (socket says 172.21.15.86:43402) 2024-04-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.046+0000 7f65f5c00640 1 -- 172.21.15.86:0/982096250 learned_addr learned my addr 172.21.15.86:0/982096250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.046+0000 7f65f5c00640 1 -- 172.21.15.86:0/982096250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5190 msgr2=0x557ce1ebf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.047+0000 7f65efe00640 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5b80 0x557ce1ebfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.047+0000 7f65f5200640 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5190 0x557ce1ebf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.047+0000 7f65f5c00640 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5190 0x557ce1ebf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.047+0000 7f65f5c00640 1 -- 172.21.15.86:0/982096250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5b80 msgr2=0x557ce1ebfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.047+0000 7f65f5c00640 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5b80 0x557ce1ebfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.047+0000 7f65f5c00640 1 -- 172.21.15.86:0/982096250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce1b2e3f0 con 0x557ce1ebacf0 2024-04-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.048+0000 7f65efe00640 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5b80 0x557ce1ebfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.048+0000 7f65f5c00640 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 0x557ce1ec6b30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009250 tx=0x7f65e0011580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a41d9f0118f02ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.048+0000 7f65daa00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.049+0000 7f65daa00640 1 -- 172.21.15.86:0/982096250 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e001a070 con 0x557ce1ebacf0 2024-04-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.049+0000 7f65daa00640 1 -- 172.21.15.86:0/982096250 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f65e00048d0 con 0x557ce1ebacf0 2024-04-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.049+0000 7f65fa987740 1 -- 172.21.15.86:0/982096250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e0004c50 con 0x557ce1ebacf0 2024-04-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.050+0000 7f65daa00640 1 -- 172.21.15.86:0/982096250 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65e003a050 con 0x557ce1ebacf0 2024-04-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.050+0000 7f65fa987740 10 client.38183 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.050+0000 7f65fa987740 1 -- 172.21.15.86:0/982096250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ce1aee0f0 con 0x557ce1ebacf0 2024-04-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.051+0000 7f65daa00640 1 -- 172.21.15.86:0/982096250 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65e0004c50 con 0x557ce1ebacf0 2024-04-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.051+0000 7f65daa00640 1 -- 172.21.15.86:0/982096250 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f65e00285b0 con 0x557ce1ebacf0 2024-04-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.052+0000 7f65fa987740 10 client.38183 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.052+0000 7f65fa987740 10 client.38183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.052+0000 7f65fa987740 20 client.38183 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.052+0000 7f65fa987740 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557ce1ecb240 0x557ce1ecd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.052+0000 7f65fa987740 4 client.38183 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.053+0000 7f65fa987740 1 -- 172.21.15.86:0/982096250 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ce1e29b40 con 0x557ce1ecb240 2024-04-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.053+0000 7f65f5200640 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557ce1ecb240 0x557ce1ecd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.056+0000 7f65f5200640 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557ce1ecb240 0x557ce1ecd680 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c96ecf75ee3749c server_cookie=1e4018eb8b4858e7 in_seq=0 out_seq=0 2024-04-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.056+0000 7f65daa00640 10 client.38183 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.060+0000 7f65daa00640 1 -- 172.21.15.86:0/982096250 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ce1e29b40 con 0x557ce1ecb240 2024-04-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.060+0000 7f65daa00640 10 client.38183 handle_command_reply: tid=0 2024-04-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.060+0000 7f65fa987740 1 client.38183 shutdown 2024-04-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.060+0000 7f65fa987740 1 -- 172.21.15.86:0/982096250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 msgr2=0x557ce1ec6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.060+0000 7f65fa987740 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 0x557ce1ec6b30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009250 tx=0x7f65e0011580 comp rx=0 tx=0).stop 2024-04-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.061+0000 7f65fa987740 1 -- 172.21.15.86:0/982096250 shutdown_connections 2024-04-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.061+0000 7f65fa987740 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557ce1ecb240 0x557ce1ecd680 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.061+0000 7f65fa987740 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 0x557ce1ec6b30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.061+0000 7f65fa987740 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5b80 0x557ce1ebfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.061+0000 7f65fa987740 1 --2- 172.21.15.86:0/982096250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5190 0x557ce1ebf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.061+0000 7f65daa00640 0 client.38183 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.061+0000 7f65fa987740 1 -- 172.21.15.86:0/982096250 >> 172.21.15.86:0/982096250 conn(0x557ce1eb0700 msgr2=0x557ce1eaf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.061+0000 7f65fa987740 1 -- 172.21.15.86:0/982096250 shutdown_connections 2024-04-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.061+0000 7f65fa987740 1 -- 172.21.15.86:0/982096250 wait complete. 2024-04-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.061+0000 7f65fa987740 20 client.38183 trim_cache size 0 max 16384 2024-04-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.098+0000 7f65fa987740 1 Processor -- start 2024-04-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.098+0000 7f65fa987740 1 -- start start 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.098+0000 7f65fa987740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 0x557ce1ebf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.098+0000 7f65fa987740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1ec6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.098+0000 7f65fa987740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1ebacf0 0x557ce1ebfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.098+0000 7f65fa987740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b0fc0 con 0x557ce1ebacf0 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.098+0000 7f65fa987740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b2b00 con 0x557ce1eb5190 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.098+0000 7f65fa987740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b1690 con 0x557ce1eb5b80 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.099+0000 7f65efe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1ec6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.099+0000 7f65efe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1ec6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:35718/0 (socket says 172.21.15.86:35718) 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.099+0000 7f65efe00640 1 -- 172.21.15.86:0/2202893395 learned_addr learned my addr 172.21.15.86:0/2202893395 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.099+0000 7f65f5200640 1 --2- 172.21.15.86:0/2202893395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 0x557ce1ebf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.099+0000 7f65efe00640 1 -- 172.21.15.86:0/2202893395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 msgr2=0x557ce1ebf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.099+0000 7f65efe00640 1 --2- 172.21.15.86:0/2202893395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 0x557ce1ebf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.099+0000 7f65efe00640 1 -- 172.21.15.86:0/2202893395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1ebacf0 msgr2=0x557ce1ebfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.099+0000 7f65efe00640 1 --2- 172.21.15.86:0/2202893395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1ebacf0 0x557ce1ebfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.099+0000 7f65efe00640 1 -- 172.21.15.86:0/2202893395 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce1eaace0 con 0x557ce1eb5b80 2024-04-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.101+0000 7f65efe00640 1 --2- 172.21.15.86:0/2202893395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1ec6af0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00cf10 tx=0x7f65dc010e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=619ab37c60920234 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.101+0000 7f65e5600640 1 -- 172.21.15.86:0/2202893395 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0a0ad0 con 0x557ce1eb5b80 2024-04-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.101+0000 7f65e5600640 1 -- 172.21.15.86:0/2202893395 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65dc0a5390 con 0x557ce1eb5b80 2024-04-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.101+0000 7f65e5600640 1 -- 172.21.15.86:0/2202893395 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc09bb70 con 0x557ce1eb5b80 2024-04-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.102+0000 7f65fa987740 1 -- 172.21.15.86:0/2202893395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 msgr2=0x557ce1ec6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.102+0000 7f65fa987740 1 --2- 172.21.15.86:0/2202893395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1ec6af0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00cf10 tx=0x7f65dc010e00 comp rx=0 tx=0).stop 2024-04-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.103+0000 7f65fa987740 1 -- 172.21.15.86:0/2202893395 shutdown_connections 2024-04-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.103+0000 7f65fa987740 1 --2- 172.21.15.86:0/2202893395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1ebacf0 0x557ce1ebfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.103+0000 7f65fa987740 1 --2- 172.21.15.86:0/2202893395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1eb5b80 0x557ce1ec6af0 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00cf10 tx=0x7f65dc010e00 comp rx=0 tx=0).stop 2024-04-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.103+0000 7f65fa987740 1 --2- 172.21.15.86:0/2202893395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1eb5190 0x557ce1ebf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.103+0000 7f65fa987740 1 -- 172.21.15.86:0/2202893395 >> 172.21.15.86:0/2202893395 conn(0x557ce1eb0700 msgr2=0x557ce1eab720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.104+0000 7f65fa987740 1 -- 172.21.15.86:0/2202893395 shutdown_connections 2024-04-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.104+0000 7f65fa987740 1 -- 172.21.15.86:0/2202893395 wait complete. 2024-04-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.105+0000 7f65fa987740 1 -- start start 2024-04-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.105+0000 7f65fa987740 1 Processor -- start 2024-04-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.106+0000 7f65fa987740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5190 0x557ce1ec4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.106+0000 7f65fa987740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 0x557ce1ec4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.106+0000 7f65fa987740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1ec5eb0 0x557ce1ec6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.106+0000 7f65fa987740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b2430 con 0x557ce1eb5190 2024-04-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.106+0000 7f65fa987740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01b1d60 con 0x557ce1ebacf0 2024-04-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.106+0000 7f65fa987740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01aef90 con 0x557ce1ec5eb0 2024-04-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.106+0000 7f65fa987740 10 client.0 _ll_register_callbacks cb 0x7f65dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.107+0000 7f65f5c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1ec5eb0 0x557ce1ec6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.107+0000 7f65f5c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1ec5eb0 0x557ce1ec6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:35732/0 (socket says 172.21.15.86:35732) 2024-04-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.107+0000 7f65f5c00640 1 -- 172.21.15.86:0/2579231681 learned_addr learned my addr 172.21.15.86:0/2579231681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.107+0000 7f65f5200640 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5190 0x557ce1ec4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.108+0000 7f65f5c00640 1 -- 172.21.15.86:0/2579231681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 msgr2=0x557ce1ec4ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.108+0000 7f65f5c00640 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 0x557ce1ec4ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.108+0000 7f65f5c00640 1 -- 172.21.15.86:0/2579231681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5190 msgr2=0x557ce1ec4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.108+0000 7f65f5c00640 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5190 0x557ce1ec4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.108+0000 7f65f5c00640 1 -- 172.21.15.86:0/2579231681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce1ec6870 con 0x557ce1ec5eb0 2024-04-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.108+0000 7f65f5c00640 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1ec5eb0 0x557ce1ec6330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f65e0031d60 tx=0x7f65e0031b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a9bc292e599e1a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.108+0000 7f65daa00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.109+0000 7f65daa00640 1 -- 172.21.15.86:0/2579231681 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e001a040 con 0x557ce1ec5eb0 2024-04-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.109+0000 7f65daa00640 1 -- 172.21.15.86:0/2579231681 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f65e00323a0 con 0x557ce1ec5eb0 2024-04-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.109+0000 7f65fa987740 1 -- 172.21.15.86:0/2579231681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65dc0a5390 con 0x557ce1ec5eb0 2024-04-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.109+0000 7f65daa00640 1 -- 172.21.15.86:0/2579231681 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65e000e3b0 con 0x557ce1ec5eb0 2024-04-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.110+0000 7f65fa987740 10 client.28234 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.110+0000 7f65fa987740 1 -- 172.21.15.86:0/2579231681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ce1f32310 con 0x557ce1ec5eb0 2024-04-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.110+0000 7f65daa00640 1 -- 172.21.15.86:0/2579231681 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65dc0a5390 con 0x557ce1ec5eb0 2024-04-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.111+0000 7f65daa00640 1 -- 172.21.15.86:0/2579231681 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f65e00285b0 con 0x557ce1ec5eb0 2024-04-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.111+0000 7f65fa987740 10 client.28234 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.111+0000 7f65fa987740 10 client.28234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.111+0000 7f65fa987740 20 client.28234 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.111+0000 7f65fa987740 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557ce1f32930 0x557ce1f34d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.111+0000 7f65fa987740 4 client.28234 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.111+0000 7f65fa987740 1 -- 172.21.15.86:0/2579231681 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557ce1e239b0 con 0x557ce1f32930 2024-04-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.112+0000 7f65f5200640 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557ce1f32930 0x557ce1f34d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.113+0000 7f65f5200640 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557ce1f32930 0x557ce1f34d50 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f6df45eb9fb445 server_cookie=8cff01a2bf8efca4 in_seq=0 out_seq=0 2024-04-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.113+0000 7f65daa00640 10 client.28234 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.113+0000 7f65daa00640 1 -- 172.21.15.86:0/2579231681 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557ce1e239b0 con 0x557ce1f32930 2024-04-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.113+0000 7f65daa00640 10 client.28234 handle_command_reply: tid=0 2024-04-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.113+0000 7f65fa987740 1 client.28234 shutdown 2024-04-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.114+0000 7f65fa987740 1 -- 172.21.15.86:0/2579231681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1ec5eb0 msgr2=0x557ce1ec6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.114+0000 7f65fa987740 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1ec5eb0 0x557ce1ec6330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f65e0031d60 tx=0x7f65e0031b70 comp rx=0 tx=0).stop 2024-04-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.115+0000 7f65fa987740 1 -- 172.21.15.86:0/2579231681 shutdown_connections 2024-04-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.115+0000 7f65fa987740 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x557ce1f32930 0x557ce1f34d50 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.115+0000 7f65fa987740 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ce1ec5eb0 0x557ce1ec6330 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.115+0000 7f65fa987740 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ce1ebacf0 0x557ce1ec4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.115+0000 7f65fa987740 1 --2- 172.21.15.86:0/2579231681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ce1eb5190 0x557ce1ec4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.115+0000 7f65fa987740 1 -- 172.21.15.86:0/2579231681 >> 172.21.15.86:0/2579231681 conn(0x557ce1eb0700 msgr2=0x557ce1ebf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.115+0000 7f65daa00640 0 client.28234 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.116+0000 7f65fa987740 1 -- 172.21.15.86:0/2579231681 shutdown_connections 2024-04-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.116+0000 7f65fa987740 1 -- 172.21.15.86:0/2579231681 wait complete. 2024-04-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.116+0000 7f65fa987740 20 client.28234 trim_cache size 0 max 16384 2024-04-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "e35f2c5c-699d-4418-86ad-ba45c51ed36c": { 2024-04-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e35f2c5c-699d-4418-86ad-ba45c51ed36c", 2024-04-17T07:30:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:30:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:30:28.118 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:30:28.118 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:30:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.117+0000 7f65f7c00640 1 -- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f65d40a3d30 msgr2=0x7f65d40a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.117+0000 7f65f7c00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f65d40a3d30 0x7f65d40a61f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f65e0023000 tx=0x7f65e0020000 comp rx=0 tx=0).stop 2024-04-17T07:30:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.117+0000 7f65f7c00640 1 -- 172.21.15.86:0/4015248985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65f01031a0 msgr2=0x7f65f0074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.117+0000 7f65f7c00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65f01031a0 0x7f65f0074350 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00e9f0 tx=0x7f65dc00eeb0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.117+0000 7f65f7c00640 1 -- 172.21.15.86:0/4015248985 shutdown_connections 2024-04-17T07:30:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.117+0000 7f65f7c00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f65d40a3d30 0x7f65d40a61f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.117+0000 7f65f7c00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65f0103b60 0x7f65f0074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.117+0000 7f65f7c00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65f01031a0 0x7f65f0074350 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.117+0000 7f65f7c00640 1 --2- 172.21.15.86:0/4015248985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65f0101fb0 0x7f65f0075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:28.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.117+0000 7f65f7c00640 1 -- 172.21.15.86:0/4015248985 >> 172.21.15.86:0/4015248985 conn(0x7f65f00fd540 msgr2=0x7f65f01063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:28.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.118+0000 7f65f7c00640 1 -- 172.21.15.86:0/4015248985 shutdown_connections 2024-04-17T07:30:28.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:28.118+0000 7f65f7c00640 1 -- 172.21.15.86:0/4015248985 wait complete. 2024-04-17T07:30:28.149 INFO:tasks.cephfs.filesystem:scrub status for tag:e35f2c5c-699d-4418-86ad-ba45c51ed36c - {'path': '/', 'tag': 'e35f2c5c-699d-4418-86ad-ba45c51ed36c', 'options': 'recursive,force'} 2024-04-17T07:30:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:28 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:30:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:28 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:30:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:28 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:30:28.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:28 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:30:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:28 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:30:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:28 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:30:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:28 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:30:28.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:28 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:30:28.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:28 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:30:28.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:28 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:30:28.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:28 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:30:28.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:28 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:30:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 665.0 s, 554.0 tps, lat 183.411 ms stddev 101.121 2024-04-17T07:30:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:29 smithi098 ceph-mon[35418]: pgmap v923: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 656 op/s 2024-04-17T07:30:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:29 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:30:29.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:29 smithi103 ceph-mon[35495]: pgmap v923: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 656 op/s 2024-04-17T07:30:29.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:29 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:30:29.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:29 smithi086 ceph-mon[31733]: pgmap v923: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 656 op/s 2024-04-17T07:30:29.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:29 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:30:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:31 smithi098 ceph-mon[35418]: pgmap v924: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-04-17T07:30:31.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:31 smithi103 ceph-mon[35495]: pgmap v924: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-04-17T07:30:31.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:31 smithi086 ceph-mon[31733]: pgmap v924: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-04-17T07:30:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:33 smithi098 ceph-mon[35418]: pgmap v925: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 699 op/s 2024-04-17T07:30:33.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:33 smithi103 ceph-mon[35495]: pgmap v925: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 699 op/s 2024-04-17T07:30:33.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:33 smithi086 ceph-mon[31733]: pgmap v925: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 699 op/s 2024-04-17T07:30:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 670.0 s, 584.8 tps, lat 170.087 ms stddev 56.818 2024-04-17T07:30:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:35 smithi098 ceph-mon[35418]: pgmap v926: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 MiB/s wr, 674 op/s 2024-04-17T07:30:35.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:35 smithi103 ceph-mon[35495]: pgmap v926: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 MiB/s wr, 674 op/s 2024-04-17T07:30:35.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:35 smithi086 ceph-mon[31733]: pgmap v926: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.4 MiB/s wr, 674 op/s 2024-04-17T07:30:36.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:36 smithi103 ceph-mon[35495]: pgmap v927: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.3 MiB/s wr, 659 op/s 2024-04-17T07:30:36.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:36 smithi086 ceph-mon[31733]: pgmap v927: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.3 MiB/s wr, 659 op/s 2024-04-17T07:30:37.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:36 smithi098 ceph-mon[35418]: pgmap v927: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.3 MiB/s wr, 659 op/s 2024-04-17T07:30:39.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:38 smithi098 ceph-mon[35418]: pgmap v928: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 688 op/s 2024-04-17T07:30:39.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:38 smithi103 ceph-mon[35495]: pgmap v928: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 688 op/s 2024-04-17T07:30:39.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:38 smithi086 ceph-mon[31733]: pgmap v928: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 688 op/s 2024-04-17T07:30:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 675.0 s, 484.8 tps, lat 205.414 ms stddev 142.296 2024-04-17T07:30:41.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:40 smithi098 ceph-mon[35418]: pgmap v929: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 689 op/s 2024-04-17T07:30:41.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:40 smithi103 ceph-mon[35495]: pgmap v929: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 689 op/s 2024-04-17T07:30:41.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:40 smithi086 ceph-mon[31733]: pgmap v929: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 689 op/s 2024-04-17T07:30:42.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:41 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:30:42.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:41 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:30:42.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:41 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:30:43.002 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:42 smithi098 ceph-mon[35418]: pgmap v930: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 729 op/s 2024-04-17T07:30:43.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:42 smithi103 ceph-mon[35495]: pgmap v930: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 729 op/s 2024-04-17T07:30:43.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:42 smithi086 ceph-mon[31733]: pgmap v930: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 729 op/s 2024-04-17T07:30:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:43 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-17T07:30:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:43 smithi098 ceph-mon[35418]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:30:44.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:30:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:30:44.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:43 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:30:44.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:43 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-17T07:30:44.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:43 smithi086 ceph-mon[31733]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:30:44.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:30:44.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:30:44.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:43 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:30:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 680.0 s, 456.4 tps, lat 153.055 ms stddev 87.726 2024-04-17T07:30:44.585 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.585 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.585 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.585 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:30:44.585 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:43 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi086 to 176.4M 2024-04-17T07:30:44.585 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:43 smithi103 ceph-mon[35495]: Unable to set osd_memory_target on smithi086 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:30:44.585 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:30:44.585 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:30:44.585 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:43 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:30:45.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:44 smithi098 ceph-mon[35418]: pgmap v931: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.5 MiB/s wr, 694 op/s 2024-04-17T07:30:45.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:44 smithi103 ceph-mon[35495]: pgmap v931: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.5 MiB/s wr, 694 op/s 2024-04-17T07:30:45.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:44 smithi086 ceph-mon[31733]: pgmap v931: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.5 MiB/s wr, 694 op/s 2024-04-17T07:30:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:47 smithi098 ceph-mon[35418]: pgmap v932: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 MiB/s wr, 661 op/s 2024-04-17T07:30:47.281 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:47 smithi103 ceph-mon[35495]: pgmap v932: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 MiB/s wr, 661 op/s 2024-04-17T07:30:47.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:47 smithi086 ceph-mon[31733]: pgmap v932: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.2 MiB/s wr, 661 op/s 2024-04-17T07:30:49.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:49 smithi086 ceph-mon[31733]: pgmap v933: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 706 op/s 2024-04-17T07:30:49.337 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:49 smithi103 ceph-mon[35495]: pgmap v933: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 706 op/s 2024-04-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 685.0 s, 488.8 tps, lat 260.699 ms stddev 656.585 2024-04-17T07:30:49.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:49 smithi098 ceph-mon[35418]: pgmap v933: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 706 op/s 2024-04-17T07:30:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:51 smithi103 ceph-mon[35495]: pgmap v934: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 717 op/s 2024-04-17T07:30:51.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:51 smithi086 ceph-mon[31733]: pgmap v934: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 717 op/s 2024-04-17T07:30:51.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:51 smithi098 ceph-mon[35418]: pgmap v934: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 717 op/s 2024-04-17T07:30:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:53 smithi103 ceph-mon[35495]: pgmap v935: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-04-17T07:30:53.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:53 smithi086 ceph-mon[31733]: pgmap v935: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-04-17T07:30:53.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:53 smithi098 ceph-mon[35418]: pgmap v935: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-04-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 690.0 s, 570.6 tps, lat 179.994 ms stddev 119.621 2024-04-17T07:30:55.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:55 smithi086 ceph-mon[31733]: pgmap v936: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 695 op/s 2024-04-17T07:30:55.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:55 smithi098 ceph-mon[35418]: pgmap v936: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 695 op/s 2024-04-17T07:30:55.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:55 smithi103 ceph-mon[35495]: pgmap v936: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 695 op/s 2024-04-17T07:30:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:57 smithi086 ceph-mon[31733]: pgmap v937: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-04-17T07:30:57.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:57 smithi098 ceph-mon[35418]: pgmap v937: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-04-17T07:30:57.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:57 smithi103 ceph-mon[35495]: pgmap v937: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-04-17T07:30:58.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:30:58.151 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:30:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.520+0000 7f834f800640 1 -- 172.21.15.86:0/1052602801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83481031a0 msgr2=0x7f8348103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.520+0000 7f834f800640 1 --2- 172.21.15.86:0/1052602801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83481031a0 0x7f8348103620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f833c009900 tx=0x7f833c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:30:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.521+0000 7f834f800640 1 -- 172.21.15.86:0/1052602801 shutdown_connections 2024-04-17T07:30:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.521+0000 7f834f800640 1 --2- 172.21.15.86:0/1052602801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8348103b60 0x7f834810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.521+0000 7f834f800640 1 --2- 172.21.15.86:0/1052602801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83481031a0 0x7f8348103620 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.521+0000 7f834f800640 1 --2- 172.21.15.86:0/1052602801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8348101fb0 0x7f83481023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.521+0000 7f834f800640 1 -- 172.21.15.86:0/1052602801 >> 172.21.15.86:0/1052602801 conn(0x7f83480fd540 msgr2=0x7f83480ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:58.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.522+0000 7f834f800640 1 -- 172.21.15.86:0/1052602801 shutdown_connections 2024-04-17T07:30:58.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.522+0000 7f834f800640 1 -- 172.21.15.86:0/1052602801 wait complete. 2024-04-17T07:30:58.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.523+0000 7f834f800640 1 Processor -- start 2024-04-17T07:30:58.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.525+0000 7f834f800640 1 -- start start 2024-04-17T07:30:58.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.526+0000 7f834f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8348101fb0 0x7f834819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.527+0000 7f834f800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83481031a0 0x7f834819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.527+0000 7f834ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8348101fb0 0x7f834819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.527+0000 7f834ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8348101fb0 0x7f834819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46896/0 (socket says 172.21.15.86:46896) 2024-04-17T07:30:58.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.528+0000 7f834ce00640 1 -- 172.21.15.86:0/2178675709 learned_addr learned my addr 172.21.15.86:0/2178675709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:58.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.528+0000 7f834f800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8348103b60 0x7f83481a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.528+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f834810fd50 con 0x7f8348101fb0 2024-04-17T07:30:58.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.528+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f834810fbd0 con 0x7f83481031a0 2024-04-17T07:30:58.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.528+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f834810fed0 con 0x7f8348103b60 2024-04-17T07:30:58.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.528+0000 7f834d800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8348103b60 0x7f83481a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.530+0000 7f8347e00640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83481031a0 0x7f834819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.531+0000 7f834d800640 1 -- 172.21.15.86:0/2178675709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83481031a0 msgr2=0x7f834819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.531+0000 7f834d800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83481031a0 0x7f834819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.531+0000 7f834d800640 1 -- 172.21.15.86:0/2178675709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8348101fb0 msgr2=0x7f834819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.531+0000 7f834d800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8348101fb0 0x7f834819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.531+0000 7f834d800640 1 -- 172.21.15.86:0/2178675709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83481a1ff0 con 0x7f8348103b60 2024-04-17T07:30:58.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.531+0000 7f834d800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8348103b60 0x7f83481a1980 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8340009fd0 tx=0x7f834000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:58.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.532+0000 7f8345600640 1 -- 172.21.15.86:0/2178675709 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834000ce20 con 0x7f8348103b60 2024-04-17T07:30:58.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.532+0000 7f8345600640 1 -- 172.21.15.86:0/2178675709 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8340005020 con 0x7f8348103b60 2024-04-17T07:30:58.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.532+0000 7f8345600640 1 -- 172.21.15.86:0/2178675709 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340010690 con 0x7f8348103b60 2024-04-17T07:30:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.532+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83481a22e0 con 0x7f8348103b60 2024-04-17T07:30:58.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.532+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83481a27b0 con 0x7f8348103b60 2024-04-17T07:30:58.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.538+0000 7f8345600640 1 -- 172.21.15.86:0/2178675709 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8340004a40 con 0x7f8348103b60 2024-04-17T07:30:58.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.538+0000 7f83525c2740 1 Processor -- start 2024-04-17T07:30:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.539+0000 7f83525c2740 1 -- start start 2024-04-17T07:30:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.539+0000 7f83525c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a1190 0x5581216a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.539+0000 7f83525c2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216a6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.539+0000 7f83525c2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a6cf0 0x5581216a9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.539+0000 7f83525c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558121608120 con 0x5581216a1190 2024-04-17T07:30:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.539+0000 7f83525c2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5581215ae340 con 0x5581216a1b80 2024-04-17T07:30:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.539+0000 7f83525c2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5581216040e0 con 0x5581216a6cf0 2024-04-17T07:30:58.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.540+0000 7f834ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.540+0000 7f834ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216a6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58094/0 (socket says 172.21.15.86:58094) 2024-04-17T07:30:58.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.540+0000 7f834ce00640 1 -- 172.21.15.86:0/247827409 learned_addr learned my addr 172.21.15.86:0/247827409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:58.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.540+0000 7f8347e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a1190 0x5581216a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.541+0000 7f834ce00640 1 -- 172.21.15.86:0/247827409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a6cf0 msgr2=0x5581216a9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:58.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.541+0000 7f834ce00640 1 --2- 172.21.15.86:0/247827409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a6cf0 0x5581216a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.541+0000 7f834ce00640 1 -- 172.21.15.86:0/247827409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a1190 msgr2=0x5581216a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:58.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.541+0000 7f834ce00640 1 --2- 172.21.15.86:0/247827409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a1190 0x5581216a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.541+0000 7f834ce00640 1 -- 172.21.15.86:0/247827409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581213e47d0 con 0x5581216a1b80 2024-04-17T07:30:58.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.542+0000 7f834ce00640 1 --2- 172.21.15.86:0/247827409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216a6680 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f833400bd90 tx=0x7f83340107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c10c0961764aaf8b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:58.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.542+0000 7f8345600640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83300a3d00 0x7f83300a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.543+0000 7f8345600640 1 -- 172.21.15.86:0/2178675709 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f83400d0c70 con 0x7f8348103b60 2024-04-17T07:30:58.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.544+0000 7f8347e00640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83300a3d00 0x7f83300a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.545+0000 7f833a000640 1 -- 172.21.15.86:0/247827409 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334004220 con 0x5581216a1b80 2024-04-17T07:30:58.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.545+0000 7f833a000640 1 -- 172.21.15.86:0/247827409 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83340043c0 con 0x5581216a1b80 2024-04-17T07:30:58.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.546+0000 7f833a000640 1 -- 172.21.15.86:0/247827409 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334011800 con 0x5581216a1b80 2024-04-17T07:30:58.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.546+0000 7f8347e00640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83300a3d00 0x7f83300a61c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f833c000c00 tx=0x7f833c02c620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:58.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.547+0000 7f83525c2740 1 -- 172.21.15.86:0/247827409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 msgr2=0x5581216a6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:58.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.547+0000 7f83525c2740 1 --2- 172.21.15.86:0/247827409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216a6680 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f833400bd90 tx=0x7f83340107a0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.548+0000 7f83525c2740 1 -- 172.21.15.86:0/247827409 shutdown_connections 2024-04-17T07:30:58.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.548+0000 7f83525c2740 1 --2- 172.21.15.86:0/247827409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a6cf0 0x5581216a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.548+0000 7f83525c2740 1 --2- 172.21.15.86:0/247827409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216a6680 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.548+0000 7f83525c2740 1 --2- 172.21.15.86:0/247827409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a1190 0x5581216a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.548+0000 7f83525c2740 1 -- 172.21.15.86:0/247827409 >> 172.21.15.86:0/247827409 conn(0x55812169c700 msgr2=0x55812169cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:58.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.549+0000 7f83525c2740 1 -- 172.21.15.86:0/247827409 shutdown_connections 2024-04-17T07:30:58.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.549+0000 7f83525c2740 1 -- 172.21.15.86:0/247827409 wait complete. 2024-04-17T07:30:58.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.549+0000 7f83525c2740 1 -- start start 2024-04-17T07:30:58.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.550+0000 7f83525c2740 1 Processor -- start 2024-04-17T07:30:58.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.550+0000 7f83525c2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1190 0x5581216bc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.550+0000 7f83525c2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1b80 0x5581216bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.550+0000 7f83525c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a6cf0 0x5581216c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.550+0000 7f83525c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5581216bacb0 con 0x5581216a6cf0 2024-04-17T07:30:58.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.550+0000 7f83525c2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5581215c1710 con 0x5581216a1190 2024-04-17T07:30:58.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.550+0000 7f83525c2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5581216bae30 con 0x5581216a1b80 2024-04-17T07:30:58.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.550+0000 7f83525c2740 10 client.0 _ll_register_callbacks cb 0x5581215bcaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:58.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.551+0000 7f834ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1190 0x5581216bc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.551+0000 7f834ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1190 0x5581216bc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58098/0 (socket says 172.21.15.86:58098) 2024-04-17T07:30:58.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.551+0000 7f834ce00640 1 -- 172.21.15.86:0/4023072950 learned_addr learned my addr 172.21.15.86:0/4023072950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:58.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.551+0000 7f834d800640 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a6cf0 0x5581216c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.551+0000 7f8347e00640 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1b80 0x5581216bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.551+0000 7f834ce00640 1 -- 172.21.15.86:0/4023072950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1b80 msgr2=0x5581216bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:58.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.551+0000 7f834ce00640 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1b80 0x5581216bcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.551+0000 7f834ce00640 1 -- 172.21.15.86:0/4023072950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a6cf0 msgr2=0x5581216c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:58.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.551+0000 7f834ce00640 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a6cf0 0x5581216c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.552+0000 7f834ce00640 1 -- 172.21.15.86:0/4023072950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55812131a3f0 con 0x5581216a1190 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.552+0000 7f834ce00640 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1190 0x5581216bc6f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8334000c00 tx=0x7f83340041f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc112c3e669e37ca server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.552+0000 7f832ea00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.552+0000 7f832ea00640 1 -- 172.21.15.86:0/4023072950 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334029060 con 0x5581216a1190 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.552+0000 7f832ea00640 1 -- 172.21.15.86:0/4023072950 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8334003a00 con 0x5581216a1190 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.552+0000 7f83525c2740 1 -- 172.21.15.86:0/4023072950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83340043c0 con 0x5581216a1190 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.553+0000 7f832ea00640 1 -- 172.21.15.86:0/4023072950 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8334030050 con 0x5581216a1190 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.553+0000 7f83525c2740 10 client.38198 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.553+0000 7f83525c2740 1 -- 172.21.15.86:0/4023072950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581212da0f0 con 0x5581216a1190 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.553+0000 7f832ea00640 1 -- 172.21.15.86:0/4023072950 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83340043c0 con 0x5581216a1190 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.553+0000 7f832ea00640 1 -- 172.21.15.86:0/4023072950 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8334023030 con 0x5581216a1190 2024-04-17T07:30:58.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.554+0000 7f83525c2740 10 client.38198 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.554+0000 7f83525c2740 10 client.38198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:30:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.554+0000 7f83525c2740 20 client.38198 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.554+0000 7f83525c2740 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581216c8160 0x5581216ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.554+0000 7f83525c2740 4 client.38198 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:30:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.554+0000 7f83525c2740 1 -- 172.21.15.86:0/4023072950 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558121615b40 con 0x5581216c8160 2024-04-17T07:30:58.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.555+0000 7f8347e00640 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581216c8160 0x5581216ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.557+0000 7f8347e00640 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581216c8160 0x5581216ca550 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50978eafd9d4a824 server_cookie=717781e9f4080846 in_seq=0 out_seq=0 2024-04-17T07:30:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.557+0000 7f832ea00640 10 client.38198 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f832ea00640 1 -- 172.21.15.86:0/4023072950 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558121615b40 con 0x5581216c8160 2024-04-17T07:30:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f832ea00640 10 client.38198 handle_command_reply: tid=0 2024-04-17T07:30:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f83525c2740 1 client.38198 shutdown 2024-04-17T07:30:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f83525c2740 1 -- 172.21.15.86:0/4023072950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1190 msgr2=0x5581216bc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:58.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f83525c2740 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1190 0x5581216bc6f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8334000c00 tx=0x7f83340041f0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f83525c2740 1 -- 172.21.15.86:0/4023072950 shutdown_connections 2024-04-17T07:30:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f83525c2740 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581216c8160 0x5581216ca550 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f83525c2740 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a6cf0 0x5581216c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f83525c2740 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1b80 0x5581216bcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f83525c2740 1 --2- 172.21.15.86:0/4023072950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1190 0x5581216bc6f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.558+0000 7f83525c2740 1 -- 172.21.15.86:0/4023072950 >> 172.21.15.86:0/4023072950 conn(0x55812169c700 msgr2=0x55812169b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.559+0000 7f832ea00640 0 client.38198 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.559+0000 7f83525c2740 1 -- 172.21.15.86:0/4023072950 shutdown_connections 2024-04-17T07:30:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.559+0000 7f83525c2740 1 -- 172.21.15.86:0/4023072950 wait complete. 2024-04-17T07:30:58.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.559+0000 7f83525c2740 20 client.38198 trim_cache size 0 max 16384 2024-04-17T07:30:58.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.596+0000 7f83525c2740 1 Processor -- start 2024-04-17T07:30:58.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.596+0000 7f83525c2740 1 -- start start 2024-04-17T07:30:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.597+0000 7f83525c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55812169c700 0x558121697720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.597+0000 7f83525c2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 0x5581216c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.597+0000 7f83525c2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216bb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.598+0000 7f83525c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f832800fb50 con 0x55812169c700 2024-04-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.598+0000 7f83525c2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8328011690 con 0x5581216a1b80 2024-04-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.598+0000 7f834d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216bb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.598+0000 7f834d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216bb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58114/0 (socket says 172.21.15.86:58114) 2024-04-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.598+0000 7f834d800640 1 -- 172.21.15.86:0/2716122223 learned_addr learned my addr 172.21.15.86:0/2716122223 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.598+0000 7f83525c2740 1 -- 172.21.15.86:0/2716122223 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8328010220 con 0x5581216a1190 2024-04-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.599+0000 7f834ce00640 1 --2- 172.21.15.86:0/2716122223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55812169c700 0x558121697720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.599+0000 7f8347e00640 1 --2- 172.21.15.86:0/2716122223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 0x5581216c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.599+0000 7f834d800640 1 -- 172.21.15.86:0/2716122223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 msgr2=0x5581216c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.599+0000 7f834d800640 1 --2- 172.21.15.86:0/2716122223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 0x5581216c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.599+0000 7f834d800640 1 -- 172.21.15.86:0/2716122223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55812169c700 msgr2=0x558121697720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.599+0000 7f834d800640 1 --2- 172.21.15.86:0/2716122223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55812169c700 0x558121697720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.599+0000 7f834d800640 1 -- 172.21.15.86:0/2716122223 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581216a6330 con 0x5581216a1b80 2024-04-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.601+0000 7f834d800640 1 --2- 172.21.15.86:0/2716122223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216bb9e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8340098fd0 tx=0x7f83400a0fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=478fc6fbc2638733 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.602+0000 7f8339600640 1 -- 172.21.15.86:0/2716122223 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83400b5070 con 0x5581216a1b80 2024-04-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.602+0000 7f8339600640 1 -- 172.21.15.86:0/2716122223 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83400bd0c0 con 0x5581216a1b80 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.602+0000 7f8339600640 1 -- 172.21.15.86:0/2716122223 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83400a55b0 con 0x5581216a1b80 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.603+0000 7f83525c2740 1 -- 172.21.15.86:0/2716122223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 msgr2=0x5581216bb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.603+0000 7f83525c2740 1 --2- 172.21.15.86:0/2716122223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216bb9e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8340098fd0 tx=0x7f83400a0fe0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.604+0000 7f83525c2740 1 -- 172.21.15.86:0/2716122223 shutdown_connections 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.604+0000 7f83525c2740 1 --2- 172.21.15.86:0/2716122223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5581216a1b80 0x5581216bb9e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.604+0000 7f83525c2740 1 --2- 172.21.15.86:0/2716122223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 0x5581216c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.604+0000 7f83525c2740 1 --2- 172.21.15.86:0/2716122223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55812169c700 0x558121697720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.604+0000 7f83525c2740 1 -- 172.21.15.86:0/2716122223 >> 172.21.15.86:0/2716122223 conn(0x5581216a2960 msgr2=0x55812169b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.605+0000 7f83525c2740 1 -- 172.21.15.86:0/2716122223 shutdown_connections 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.605+0000 7f83525c2740 1 -- 172.21.15.86:0/2716122223 wait complete. 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.606+0000 7f83525c2740 1 -- start start 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.606+0000 7f83525c2740 1 Processor -- start 2024-04-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.607+0000 7f83525c2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55812169c700 0x5581216c1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.607+0000 7f83525c2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 0x5581216c18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.607+0000 7f83525c2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a1b80 0x5581216c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.608+0000 7f83525c2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8328010fc0 con 0x5581216a1b80 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.608+0000 7f83525c2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f83280108f0 con 0x55812169c700 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.608+0000 7f83525c2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f832800db20 con 0x5581216a1190 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.608+0000 7f8347e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 0x5581216c18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.608+0000 7f83525c2740 10 client.0 _ll_register_callbacks cb 0x7f833c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.608+0000 7f8347e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 0x5581216c18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54426/0 (socket says 172.21.15.86:54426) 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.608+0000 7f8347e00640 1 -- 172.21.15.86:0/2060235549 learned_addr learned my addr 172.21.15.86:0/2060235549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.608+0000 7f834ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55812169c700 0x5581216c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.608+0000 7f834d800640 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a1b80 0x5581216c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.609+0000 7f8347e00640 1 -- 172.21.15.86:0/2060235549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55812169c700 msgr2=0x5581216c1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.609+0000 7f8347e00640 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55812169c700 0x5581216c1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.609+0000 7f8347e00640 1 -- 172.21.15.86:0/2060235549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a1b80 msgr2=0x5581216c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.609+0000 7f8347e00640 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a1b80 0x5581216c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.609+0000 7f8347e00640 1 -- 172.21.15.86:0/2060235549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581216c2ed0 con 0x5581216a1190 2024-04-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.609+0000 7f834ce00640 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55812169c700 0x5581216c1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.610+0000 7f8347e00640 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 0x5581216c18d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f833c009230 tx=0x7f833c041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e091f7e99c11de7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.610+0000 7f832ea00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.610+0000 7f832ea00640 1 -- 172.21.15.86:0/2060235549 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c005040 con 0x5581216a1190 2024-04-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.610+0000 7f83525c2740 1 -- 172.21.15.86:0/2060235549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83400bd0c0 con 0x5581216a1190 2024-04-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.611+0000 7f832ea00640 1 -- 172.21.15.86:0/2060235549 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f833c0051e0 con 0x5581216a1190 2024-04-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.611+0000 7f832ea00640 1 -- 172.21.15.86:0/2060235549 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f833c064050 con 0x5581216a1190 2024-04-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.611+0000 7f83525c2740 10 client.28249 fetch_fsmap learned FSMap version 28 2024-04-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.611+0000 7f83525c2740 1 -- 172.21.15.86:0/2060235549 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558121736fd0 con 0x5581216a1190 2024-04-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.613+0000 7f832ea00640 1 -- 172.21.15.86:0/2060235549 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83400bd0c0 con 0x5581216a1190 2024-04-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.613+0000 7f832ea00640 1 -- 172.21.15.86:0/2060235549 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f833c0046c0 con 0x5581216a1190 2024-04-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.613+0000 7f83525c2740 10 client.28249 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.613+0000 7f83525c2740 10 client.28249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.613+0000 7f83525c2740 20 client.28249 populate_metadata read hostname 'smithi086' 2024-04-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.613+0000 7f83525c2740 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581216a6cf0 0x5581217399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.613+0000 7f83525c2740 4 client.28249 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.613+0000 7f83525c2740 1 -- 172.21.15.86:0/2060235549 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55812160f9b0 con 0x5581216a6cf0 2024-04-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.614+0000 7f834ce00640 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581216a6cf0 0x5581217399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.616+0000 7f834ce00640 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581216a6cf0 0x5581217399c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e09a593feae324 server_cookie=5f63fafc3c572aef in_seq=0 out_seq=0 2024-04-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.616+0000 7f832ea00640 10 client.28249 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.617+0000 7f832ea00640 1 -- 172.21.15.86:0/2060235549 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55812160f9b0 con 0x5581216a6cf0 2024-04-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.617+0000 7f832ea00640 10 client.28249 handle_command_reply: tid=0 2024-04-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.617+0000 7f83525c2740 1 client.28249 shutdown 2024-04-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.618+0000 7f83525c2740 1 -- 172.21.15.86:0/2060235549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 msgr2=0x5581216c18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.618+0000 7f83525c2740 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 0x5581216c18d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f833c009230 tx=0x7f833c041000 comp rx=0 tx=0).stop 2024-04-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.618+0000 7f83525c2740 1 -- 172.21.15.86:0/2060235549 shutdown_connections 2024-04-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.618+0000 7f83525c2740 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5581216a6cf0 0x5581217399c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.619+0000 7f832ea00640 0 client.28249 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.619+0000 7f83525c2740 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5581216a1b80 0x5581216c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.619+0000 7f83525c2740 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581216a1190 0x5581216c18d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.619+0000 7f83525c2740 1 --2- 172.21.15.86:0/2060235549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55812169c700 0x5581216c1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.619+0000 7f83525c2740 1 -- 172.21.15.86:0/2060235549 >> 172.21.15.86:0/2060235549 conn(0x5581216a2960 msgr2=0x55812169f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.619+0000 7f83525c2740 1 -- 172.21.15.86:0/2060235549 shutdown_connections 2024-04-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.619+0000 7f83525c2740 1 -- 172.21.15.86:0/2060235549 wait complete. 2024-04-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.620+0000 7f83525c2740 20 client.28249 trim_cache size 0 max 16384 2024-04-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.621+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83300a3d00 msgr2=0x7f83300a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.621+0000 7f834f800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83300a3d00 0x7f83300a61c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f833c000c00 tx=0x7f833c02c620 comp rx=0 tx=0).stop 2024-04-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.621+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8348103b60 msgr2=0x7f83481a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.621+0000 7f834f800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8348103b60 0x7f83481a1980 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8340009fd0 tx=0x7f834000ef10 comp rx=0 tx=0).stop 2024-04-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.622+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 shutdown_connections 2024-04-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.622+0000 7f834f800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83300a3d00 0x7f83300a61c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.622+0000 7f834f800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8348103b60 0x7f83481a1980 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.622+0000 7f834f800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83481031a0 0x7f834819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.622+0000 7f834f800640 1 --2- 172.21.15.86:0/2178675709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8348101fb0 0x7f834819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.622+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 >> 172.21.15.86:0/2178675709 conn(0x7f83480fd540 msgr2=0x7f8348106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.623+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 shutdown_connections 2024-04-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:58.623+0000 7f834f800640 1 -- 172.21.15.86:0/2178675709 wait complete. 2024-04-17T07:30:58.662 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:30:58.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:30:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.021+0000 7fb732e00640 1 -- 172.21.15.86:0/3217843133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c106610 msgr2=0x7fb72c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.021+0000 7fb732e00640 1 --2- 172.21.15.86:0/3217843133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c106610 0x7fb72c108a00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb71400da30 tx=0x7fb71402dcd0 comp rx=0 tx=0).stop 2024-04-17T07:30:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.021+0000 7fb732e00640 1 -- 172.21.15.86:0/3217843133 shutdown_connections 2024-04-17T07:30:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.021+0000 7fb732e00640 1 --2- 172.21.15.86:0/3217843133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c106610 0x7fb72c108a00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.021+0000 7fb732e00640 1 --2- 172.21.15.86:0/3217843133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c101910 0x7fb72c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.021+0000 7fb732e00640 1 --2- 172.21.15.86:0/3217843133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0684f0 0x7fb72c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.021+0000 7fb732e00640 1 -- 172.21.15.86:0/3217843133 >> 172.21.15.86:0/3217843133 conn(0x7fb72c0fb750 msgr2=0x7fb72c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.022+0000 7fb732e00640 1 -- 172.21.15.86:0/3217843133 shutdown_connections 2024-04-17T07:30:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.022+0000 7fb732e00640 1 -- 172.21.15.86:0/3217843133 wait complete. 2024-04-17T07:30:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.024+0000 7fb732e00640 1 Processor -- start 2024-04-17T07:30:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.024+0000 7fb732e00640 1 -- start start 2024-04-17T07:30:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.025+0000 7fb732e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0684f0 0x7fb72c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.025+0000 7fb732e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c101910 0x7fb72c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.025+0000 7fb732e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c106610 0x7fb72c1a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.025+0000 7fb732e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c10afa0 con 0x7fb72c0684f0 2024-04-17T07:30:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.025+0000 7fb732e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c10ae20 con 0x7fb72c106610 2024-04-17T07:30:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.025+0000 7fb732e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c10b120 con 0x7fb72c101910 2024-04-17T07:30:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.026+0000 7fb72b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c101910 0x7fb72c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.026+0000 7fb730e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c106610 0x7fb72c1a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.026+0000 7fb72b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c101910 0x7fb72c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54452/0 (socket says 172.21.15.86:54452) 2024-04-17T07:30:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.026+0000 7fb72b400640 1 -- 172.21.15.86:0/4234583521 learned_addr learned my addr 172.21.15.86:0/4234583521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:30:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.026+0000 7fb72be00640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0684f0 0x7fb72c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.027+0000 7fb72b400640 1 -- 172.21.15.86:0/4234583521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c106610 msgr2=0x7fb72c1a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.027+0000 7fb72b400640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c106610 0x7fb72c1a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.027+0000 7fb72b400640 1 -- 172.21.15.86:0/4234583521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0684f0 msgr2=0x7fb72c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.027+0000 7fb72b400640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0684f0 0x7fb72c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.027+0000 7fb72b400640 1 -- 172.21.15.86:0/4234583521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c1a6460 con 0x7fb72c101910 2024-04-17T07:30:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.028+0000 7fb72b400640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c101910 0x7fb72c19ed80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00d770 tx=0x7fb71c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.028+0000 7fb728c00640 1 -- 172.21.15.86:0/4234583521 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c017070 con 0x7fb72c101910 2024-04-17T07:30:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.028+0000 7fb732e00640 1 -- 172.21.15.86:0/4234583521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c1a6750 con 0x7fb72c101910 2024-04-17T07:30:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.028+0000 7fb728c00640 1 -- 172.21.15.86:0/4234583521 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb71c009c30 con 0x7fb72c101910 2024-04-17T07:30:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.029+0000 7fb728c00640 1 -- 172.21.15.86:0/4234583521 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c003d00 con 0x7fb72c101910 2024-04-17T07:30:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.029+0000 7fb732e00640 1 -- 172.21.15.86:0/4234583521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c1a6d70 con 0x7fb72c101910 2024-04-17T07:30:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.031+0000 7fb732e00640 1 -- 172.21.15.86:0/4234583521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c1a7030 con 0x7fb72c101910 2024-04-17T07:30:59.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.036+0000 7fb728c00640 1 -- 172.21.15.86:0/4234583521 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb71c003ea0 con 0x7fb72c101910 2024-04-17T07:30:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.038+0000 7fb728c00640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3c30 0x7fb7100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:30:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.038+0000 7fb728c00640 1 -- 172.21.15.86:0/4234583521 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb71c0d03e0 con 0x7fb72c101910 2024-04-17T07:30:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.039+0000 7fb72be00640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3c30 0x7fb7100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:30:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.040+0000 7fb72be00640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3c30 0x7fb7100a60f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb72c101230 tx=0x7fb72400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:30:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.048+0000 7fb728c00640 1 -- 172.21.15.86:0/4234583521 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb71c009de0 con 0x7fb72c101910 2024-04-17T07:30:59.305 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:30:59 smithi098 ceph-mon[35418]: pgmap v938: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 695 op/s 2024-04-17T07:30:59.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:30:59 smithi086 ceph-mon[31733]: pgmap v938: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 695 op/s 2024-04-17T07:30:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 695.0 s, 554.0 tps, lat 180.542 ms stddev 70.681 2024-04-17T07:30:59.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:30:59 smithi103 ceph-mon[35495]: pgmap v938: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 695 op/s 2024-04-17T07:30:59.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.625+0000 7fb732e00640 1 -- 172.21.15.86:0/4234583521 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb72c1a7200 con 0x7fb72c101910 2024-04-17T07:30:59.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.626+0000 7fb728c00640 1 -- 172.21.15.86:0/4234583521 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fb71c09c410 con 0x7fb72c101910 2024-04-17T07:30:59.629 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:30:59.629 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:30:59.630 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:30:59.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.632+0000 7fb732e00640 1 -- 172.21.15.86:0/4234583521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3c30 msgr2=0x7fb7100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:59.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.632+0000 7fb732e00640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3c30 0x7fb7100a60f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb72c101230 tx=0x7fb72400d040 comp rx=0 tx=0).stop 2024-04-17T07:30:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.632+0000 7fb732e00640 1 -- 172.21.15.86:0/4234583521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c101910 msgr2=0x7fb72c19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:30:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.632+0000 7fb732e00640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c101910 0x7fb72c19ed80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00d770 tx=0x7fb71c00dc30 comp rx=0 tx=0).stop 2024-04-17T07:30:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.634+0000 7fb732e00640 1 -- 172.21.15.86:0/4234583521 shutdown_connections 2024-04-17T07:30:59.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.634+0000 7fb732e00640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7100a3c30 0x7fb7100a60f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.634+0000 7fb732e00640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb72c106610 0x7fb72c1a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.634+0000 7fb732e00640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72c101910 0x7fb72c19ed80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.634+0000 7fb732e00640 1 --2- 172.21.15.86:0/4234583521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb72c0684f0 0x7fb72c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:30:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.634+0000 7fb732e00640 1 -- 172.21.15.86:0/4234583521 >> 172.21.15.86:0/4234583521 conn(0x7fb72c0fb750 msgr2=0x7fb72c10b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:30:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.635+0000 7fb732e00640 1 -- 172.21.15.86:0/4234583521 shutdown_connections 2024-04-17T07:30:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:30:59.635+0000 7fb732e00640 1 -- 172.21.15.86:0/4234583521 wait complete. 2024-04-17T07:30:59.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:31:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.070+0000 7f4636e00640 1 -- 172.21.15.86:0/3232042637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463010e2a0 msgr2=0x7f463010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.070+0000 7f4636e00640 1 --2- 172.21.15.86:0/3232042637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463010e2a0 0x7f463010e750 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f4628009be0 tx=0x7f462802bef0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.072+0000 7f4636e00640 1 -- 172.21.15.86:0/3232042637 shutdown_connections 2024-04-17T07:31:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.072+0000 7f4636e00640 1 --2- 172.21.15.86:0/3232042637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f463010e2a0 0x7f463010e750 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.072+0000 7f4636e00640 1 --2- 172.21.15.86:0/3232042637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f463010b8d0 0x7f463010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.072+0000 7f4636e00640 1 --2- 172.21.15.86:0/3232042637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4630108f60 0x7f463010b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.072+0000 7f4636e00640 1 -- 172.21.15.86:0/3232042637 >> 172.21.15.86:0/3232042637 conn(0x7f463006edf0 msgr2=0x7f4630071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.072+0000 7f4636e00640 1 -- 172.21.15.86:0/3232042637 shutdown_connections 2024-04-17T07:31:00.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.073+0000 7f4636e00640 1 -- 172.21.15.86:0/3232042637 wait complete. 2024-04-17T07:31:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.074+0000 7f4636e00640 1 Processor -- start 2024-04-17T07:31:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.074+0000 7f4636e00640 1 -- start start 2024-04-17T07:31:00.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.074+0000 7f4636e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4630108f60 0x7f46301a6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.074+0000 7f4636e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463010b8d0 0x7f46301a7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.075+0000 7f4636e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f463010e2a0 0x7f46301ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.075+0000 7f4636e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f463011c8e0 con 0x7f4630108f60 2024-04-17T07:31:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.075+0000 7f4636e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f463011c760 con 0x7f463010e2a0 2024-04-17T07:31:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.075+0000 7f4636e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f463011ca60 con 0x7f463010b8d0 2024-04-17T07:31:00.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.075+0000 7f462fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4630108f60 0x7f46301a6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.075+0000 7f462fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4630108f60 0x7f46301a6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46978/0 (socket says 172.21.15.86:46978) 2024-04-17T07:31:00.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.075+0000 7f4634e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f463010e2a0 0x7f46301ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.076+0000 7f4634e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f463010e2a0 0x7f46301ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58160/0 (socket says 172.21.15.86:58160) 2024-04-17T07:31:00.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.076+0000 7f4634e00640 1 -- 172.21.15.86:0/3221215411 learned_addr learned my addr 172.21.15.86:0/3221215411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:00.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.075+0000 7f462f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463010b8d0 0x7f46301a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.076+0000 7f4634e00640 1 -- 172.21.15.86:0/3221215411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463010b8d0 msgr2=0x7f46301a7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.076+0000 7f4634e00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463010b8d0 0x7f46301a7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.076+0000 7f4634e00640 1 -- 172.21.15.86:0/3221215411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4630108f60 msgr2=0x7f46301a6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.076+0000 7f4634e00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4630108f60 0x7f46301a6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.076+0000 7f4634e00640 1 -- 172.21.15.86:0/3221215411 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46301aebe0 con 0x7f463010e2a0 2024-04-17T07:31:00.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.077+0000 7f4634e00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f463010e2a0 0x7f46301ae570 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4628034fd0 tx=0x7f4628033c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.078+0000 7f462cc00640 1 -- 172.21.15.86:0/3221215411 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628043020 con 0x7f463010e2a0 2024-04-17T07:31:00.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.078+0000 7f4636e00640 1 -- 172.21.15.86:0/3221215411 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46301aee70 con 0x7f463010e2a0 2024-04-17T07:31:00.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.078+0000 7f462cc00640 1 -- 172.21.15.86:0/3221215411 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4628003e20 con 0x7f463010e2a0 2024-04-17T07:31:00.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.078+0000 7f462cc00640 1 -- 172.21.15.86:0/3221215411 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628034430 con 0x7f463010e2a0 2024-04-17T07:31:00.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.078+0000 7f4636e00640 1 -- 172.21.15.86:0/3221215411 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46301af380 con 0x7f463010e2a0 2024-04-17T07:31:00.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.083+0000 7f46397fd740 1 Processor -- start 2024-04-17T07:31:00.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.084+0000 7f462cc00640 1 -- 172.21.15.86:0/3221215411 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4628046020 con 0x7f463010e2a0 2024-04-17T07:31:00.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.084+0000 7f46397fd740 1 -- start start 2024-04-17T07:31:00.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.084+0000 7f46397fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3617fa0 0x5618d36183c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.084+0000 7f46397fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d3618990 0x5618d361d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.084+0000 7f46397fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d361db00 0x5618d361ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.084+0000 7f46397fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5618d357e020 con 0x5618d3618990 2024-04-17T07:31:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.084+0000 7f46397fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5618d3524460 con 0x5618d361db00 2024-04-17T07:31:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.084+0000 7f46397fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618d3579fe0 con 0x5618d3617fa0 2024-04-17T07:31:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.085+0000 7f462f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3617fa0 0x5618d36183c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.085+0000 7f462fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d3618990 0x5618d361d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.085+0000 7f462f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3617fa0 0x5618d36183c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54468/0 (socket says 172.21.15.86:54468) 2024-04-17T07:31:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.085+0000 7f462f400640 1 -- 172.21.15.86:0/493838868 learned_addr learned my addr 172.21.15.86:0/493838868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:00.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.085+0000 7f462f400640 1 --2- 172.21.15.86:0/493838868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d361db00 0x5618d361ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.085+0000 7f462f400640 1 -- 172.21.15.86:0/493838868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d361db00 msgr2=0x5618d361ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.086+0000 7f462f400640 1 --2- 172.21.15.86:0/493838868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d361db00 0x5618d361ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.086+0000 7f462f400640 1 -- 172.21.15.86:0/493838868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d3618990 msgr2=0x5618d361d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.086+0000 7f462f400640 1 --2- 172.21.15.86:0/493838868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d3618990 0x5618d361d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.086+0000 7f462f400640 1 -- 172.21.15.86:0/493838868 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d352c040 con 0x5618d3617fa0 2024-04-17T07:31:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.086+0000 7f462f400640 1 --2- 172.21.15.86:0/493838868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3617fa0 0x5618d36183c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f461c0137c0 tx=0x7f461c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28699a1eb864d511 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.086+0000 7f462cc00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46140a3d00 0x7f46140a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.086+0000 7f462cc00640 1 -- 172.21.15.86:0/3221215411 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f46280f2e30 con 0x7f463010e2a0 2024-04-17T07:31:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.087+0000 7f462fe00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46140a3d00 0x7f46140a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.087+0000 7f4625600640 1 -- 172.21.15.86:0/493838868 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c013e00 con 0x5618d3617fa0 2024-04-17T07:31:00.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.087+0000 7f4625600640 1 -- 172.21.15.86:0/493838868 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c005020 con 0x5618d3617fa0 2024-04-17T07:31:00.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.087+0000 7f4625600640 1 -- 172.21.15.86:0/493838868 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c004a00 con 0x5618d3617fa0 2024-04-17T07:31:00.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.088+0000 7f46397fd740 1 -- 172.21.15.86:0/493838868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3617fa0 msgr2=0x5618d36183c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.088+0000 7f46397fd740 1 --2- 172.21.15.86:0/493838868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3617fa0 0x5618d36183c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f461c0137c0 tx=0x7f461c005850 comp rx=0 tx=0).stop 2024-04-17T07:31:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.088+0000 7f46397fd740 1 -- 172.21.15.86:0/493838868 shutdown_connections 2024-04-17T07:31:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.088+0000 7f46397fd740 1 --2- 172.21.15.86:0/493838868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d361db00 0x5618d361ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.088+0000 7f46397fd740 1 --2- 172.21.15.86:0/493838868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d3618990 0x5618d361d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.088+0000 7f46397fd740 1 --2- 172.21.15.86:0/493838868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3617fa0 0x5618d36183c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.088+0000 7f46397fd740 1 -- 172.21.15.86:0/493838868 >> 172.21.15.86:0/493838868 conn(0x5618d3613510 msgr2=0x5618d3613910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.088+0000 7f46397fd740 1 -- 172.21.15.86:0/493838868 shutdown_connections 2024-04-17T07:31:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.088+0000 7f46397fd740 1 -- 172.21.15.86:0/493838868 wait complete. 2024-04-17T07:31:00.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.089+0000 7f46397fd740 1 -- start start 2024-04-17T07:31:00.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.089+0000 7f46397fd740 1 Processor -- start 2024-04-17T07:31:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.090+0000 7f46397fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d3617fa0 0x5618d3622580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.090+0000 7f46397fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3618990 0x5618d3622ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.090+0000 7f46397fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d361db00 0x5618d3629b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.090+0000 7f46397fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5618d3620ab0 con 0x5618d3617fa0 2024-04-17T07:31:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.090+0000 7f46397fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5618d3537830 con 0x5618d3618990 2024-04-17T07:31:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.090+0000 7f46397fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5618d3620c30 con 0x5618d361db00 2024-04-17T07:31:00.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.090+0000 7f46397fd740 10 client.0 _ll_register_callbacks cb 0x5618d3505fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f462f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3618990 0x5618d3622ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f462f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3618990 0x5618d3622ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58176/0 (socket says 172.21.15.86:58176) 2024-04-17T07:31:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f462f400640 1 -- 172.21.15.86:0/1647714191 learned_addr learned my addr 172.21.15.86:0/1647714191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f4634e00640 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d361db00 0x5618d3629b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f462fe00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46140a3d00 0x7f46140a61c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5618d3614480 tx=0x7f4620005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f462fe00640 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d3617fa0 0x5618d3622580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f462f400640 1 -- 172.21.15.86:0/1647714191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d361db00 msgr2=0x5618d3629b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f462f400640 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d361db00 0x5618d3629b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f462f400640 1 -- 172.21.15.86:0/1647714191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d3617fa0 msgr2=0x5618d3622580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f462f400640 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d3617fa0 0x5618d3622580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.091+0000 7f462f400640 1 -- 172.21.15.86:0/1647714191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d353b920 con 0x5618d3618990 2024-04-17T07:31:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.092+0000 7f462f400640 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3618990 0x5618d3622ac0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f461c005790 tx=0x7f461c005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddceca5cef5789e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.092+0000 7f461a000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:31:00.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.092+0000 7f461a000640 1 -- 172.21.15.86:0/1647714191 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c011a80 con 0x5618d3618990 2024-04-17T07:31:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.092+0000 7f461a000640 1 -- 172.21.15.86:0/1647714191 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f461c011c20 con 0x5618d3618990 2024-04-17T07:31:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.092+0000 7f46397fd740 1 -- 172.21.15.86:0/1647714191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c005020 con 0x5618d3618990 2024-04-17T07:31:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.093+0000 7f461a000640 1 -- 172.21.15.86:0/1647714191 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f461c011e20 con 0x5618d3618990 2024-04-17T07:31:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.093+0000 7f46397fd740 10 client.38222 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.093+0000 7f46397fd740 1 -- 172.21.15.86:0/1647714191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5618d35097f0 con 0x5618d3618990 2024-04-17T07:31:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.093+0000 7f461a000640 1 -- 172.21.15.86:0/1647714191 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c005020 con 0x5618d3618990 2024-04-17T07:31:00.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.094+0000 7f461a000640 1 -- 172.21.15.86:0/1647714191 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f461c01f660 con 0x5618d3618990 2024-04-17T07:31:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.094+0000 7f46397fd740 10 client.38222 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.094+0000 7f46397fd740 10 client.38222 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:31:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.094+0000 7f46397fd740 20 client.38222 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.094+0000 7f46397fd740 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5618d362dc60 0x5618d3630070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.094+0000 7f46397fd740 4 client.38222 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:00.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.094+0000 7f46397fd740 1 -- 172.21.15.86:0/1647714191 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618d358ba40 con 0x5618d362dc60 2024-04-17T07:31:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.095+0000 7f462fe00640 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5618d362dc60 0x5618d3630070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.096+0000 7f462fe00640 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5618d362dc60 0x5618d3630070 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c540fdc546c229a server_cookie=c523c39bc9727827 in_seq=0 out_seq=0 2024-04-17T07:31:00.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.096+0000 7f461a000640 10 client.38222 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:31:00.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.098+0000 7f461a000640 1 -- 172.21.15.86:0/1647714191 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618d358ba40 con 0x5618d362dc60 2024-04-17T07:31:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.098+0000 7f461a000640 10 client.38222 handle_command_reply: tid=0 2024-04-17T07:31:00.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.098+0000 7f46397fd740 1 client.38222 shutdown 2024-04-17T07:31:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.101+0000 7f46397fd740 1 -- 172.21.15.86:0/1647714191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3618990 msgr2=0x5618d3622ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.101+0000 7f46397fd740 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3618990 0x5618d3622ac0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f461c005790 tx=0x7f461c005640 comp rx=0 tx=0).stop 2024-04-17T07:31:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.101+0000 7f46397fd740 1 -- 172.21.15.86:0/1647714191 shutdown_connections 2024-04-17T07:31:00.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.101+0000 7f46397fd740 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5618d362dc60 0x5618d3630070 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.101+0000 7f46397fd740 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d361db00 0x5618d3629b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.101+0000 7f46397fd740 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3618990 0x5618d3622ac0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.101+0000 7f46397fd740 1 --2- 172.21.15.86:0/1647714191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d3617fa0 0x5618d3622580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.101+0000 7f46397fd740 1 -- 172.21.15.86:0/1647714191 >> 172.21.15.86:0/1647714191 conn(0x5618d3613510 msgr2=0x5618d3612710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.101+0000 7f461a000640 0 client.38222 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:31:00.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.102+0000 7f46397fd740 1 -- 172.21.15.86:0/1647714191 shutdown_connections 2024-04-17T07:31:00.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.103+0000 7f46397fd740 1 -- 172.21.15.86:0/1647714191 wait complete. 2024-04-17T07:31:00.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.104+0000 7f46397fd740 20 client.38222 trim_cache size 0 max 16384 2024-04-17T07:31:00.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.139+0000 7f46397fd740 1 Processor -- start 2024-04-17T07:31:00.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.139+0000 7f46397fd740 1 -- start start 2024-04-17T07:31:00.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.140+0000 7f46397fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 0x5618d360e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.140+0000 7f46397fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 0x5618d3629ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.140+0000 7f46397fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d3611e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.140+0000 7f46397fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46301b9870 con 0x5618d361db00 2024-04-17T07:31:00.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.140+0000 7f46397fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f46301bb3b0 con 0x5618d3617fa0 2024-04-17T07:31:00.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.140+0000 7f46397fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46301b9f40 con 0x5618d3618990 2024-04-17T07:31:00.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.141+0000 7f462f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 0x5618d3629ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.141+0000 7f462f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 0x5618d3629ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54486/0 (socket says 172.21.15.86:54486) 2024-04-17T07:31:00.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.141+0000 7f462f400640 1 -- 172.21.15.86:0/1952207707 learned_addr learned my addr 172.21.15.86:0/1952207707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:00.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.142+0000 7f4634e00640 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d3611e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.143+0000 7f462fe00640 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 0x5618d360e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.143+0000 7f462f400640 1 -- 172.21.15.86:0/1952207707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 msgr2=0x5618d360e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.143+0000 7f462f400640 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 0x5618d360e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.143+0000 7f462f400640 1 -- 172.21.15.86:0/1952207707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 msgr2=0x5618d3611e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.143+0000 7f462f400640 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d3611e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.143+0000 7f462f400640 1 -- 172.21.15.86:0/1952207707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d32903f0 con 0x5618d3618990 2024-04-17T07:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.143+0000 7f4634e00640 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d3611e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.143+0000 7f462fe00640 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 0x5618d360e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.144+0000 7f462f400640 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 0x5618d3629ac0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f461c01fb00 tx=0x7f461c005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8c2ad9fa7222d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.145+0000 7f4624c00640 1 -- 172.21.15.86:0/1952207707 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c011c80 con 0x5618d3618990 2024-04-17T07:31:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.145+0000 7f4624c00640 1 -- 172.21.15.86:0/1952207707 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c0099b0 con 0x5618d3618990 2024-04-17T07:31:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.145+0000 7f4624c00640 1 -- 172.21.15.86:0/1952207707 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c003850 con 0x5618d3618990 2024-04-17T07:31:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.146+0000 7f46397fd740 1 -- 172.21.15.86:0/1952207707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 msgr2=0x5618d3629ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.146+0000 7f46397fd740 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 0x5618d3629ac0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f461c01fb00 tx=0x7f461c005970 comp rx=0 tx=0).stop 2024-04-17T07:31:00.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.147+0000 7f46397fd740 1 -- 172.21.15.86:0/1952207707 shutdown_connections 2024-04-17T07:31:00.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.147+0000 7f46397fd740 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d3611e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.147+0000 7f46397fd740 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 0x5618d3629ac0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.147+0000 7f46397fd740 1 --2- 172.21.15.86:0/1952207707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 0x5618d360e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.147+0000 7f46397fd740 1 -- 172.21.15.86:0/1952207707 >> 172.21.15.86:0/1952207707 conn(0x5618d3613510 msgr2=0x5618d3620fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.147+0000 7f46397fd740 1 -- 172.21.15.86:0/1952207707 shutdown_connections 2024-04-17T07:31:00.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.147+0000 7f46397fd740 1 -- 172.21.15.86:0/1952207707 wait complete. 2024-04-17T07:31:00.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.148+0000 7f46397fd740 1 -- start start 2024-04-17T07:31:00.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.148+0000 7f46397fd740 1 Processor -- start 2024-04-17T07:31:00.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.149+0000 7f46397fd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 0x5618d36265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.149+0000 7f46397fd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 0x5618d3626b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.149+0000 7f46397fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d36921a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.149+0000 7f46397fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46301bace0 con 0x5618d361db00 2024-04-17T07:31:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.149+0000 7f46397fd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f46301ba610 con 0x5618d3617fa0 2024-04-17T07:31:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.149+0000 7f46397fd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46301b7840 con 0x5618d3618990 2024-04-17T07:31:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.149+0000 7f46397fd740 10 client.0 _ll_register_callbacks cb 0x7f461c0136c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.151+0000 7f4634e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d36921a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.151+0000 7f462fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 0x5618d36265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.151+0000 7f4634e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d36921a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47006/0 (socket says 172.21.15.86:47006) 2024-04-17T07:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.151+0000 7f4634e00640 1 -- 172.21.15.86:0/4092524212 learned_addr learned my addr 172.21.15.86:0/4092524212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.151+0000 7f462f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 0x5618d3626b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.152+0000 7f4634e00640 1 -- 172.21.15.86:0/4092524212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 msgr2=0x5618d3626b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.152+0000 7f4634e00640 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 0x5618d3626b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.152+0000 7f4634e00640 1 -- 172.21.15.86:0/4092524212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 msgr2=0x5618d36265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.152+0000 7f4634e00640 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 0x5618d36265c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.152+0000 7f4634e00640 1 -- 172.21.15.86:0/4092524212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d335a7d0 con 0x5618d361db00 2024-04-17T07:31:00.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.154+0000 7f4634e00640 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d36921a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f462804e000 tx=0x7f4628009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7ec68f88433dff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.154+0000 7f461a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:00.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.156+0000 7f461a000640 1 -- 172.21.15.86:0/4092524212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46280d7020 con 0x5618d361db00 2024-04-17T07:31:00.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.156+0000 7f46397fd740 1 -- 172.21.15.86:0/4092524212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c0099b0 con 0x5618d361db00 2024-04-17T07:31:00.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.156+0000 7f461a000640 1 -- 172.21.15.86:0/4092524212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f46280040d0 con 0x5618d361db00 2024-04-17T07:31:00.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.157+0000 7f461a000640 1 -- 172.21.15.86:0/4092524212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46280c30e0 con 0x5618d361db00 2024-04-17T07:31:00.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.157+0000 7f46397fd740 10 client.19251 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:00.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.157+0000 7f46397fd740 1 -- 172.21.15.86:0/4092524212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5618d3693410 con 0x5618d361db00 2024-04-17T07:31:00.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.158+0000 7f461a000640 1 -- 172.21.15.86:0/4092524212 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c0099b0 con 0x5618d361db00 2024-04-17T07:31:00.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.158+0000 7f461a000640 1 -- 172.21.15.86:0/4092524212 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4628004b40 con 0x5618d361db00 2024-04-17T07:31:00.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.159+0000 7f46397fd740 10 client.19251 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:00.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.159+0000 7f46397fd740 10 client.19251 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:31:00.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.159+0000 7f46397fd740 20 client.19251 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:00.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.159+0000 7f46397fd740 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5618d3693d90 0x5618d36961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.159+0000 7f46397fd740 4 client.19251 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:31:00.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.159+0000 7f46397fd740 1 -- 172.21.15.86:0/4092524212 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618d35858b0 con 0x5618d3693d90 2024-04-17T07:31:00.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.160+0000 7f462fe00640 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5618d3693d90 0x5618d36961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.161+0000 7f462fe00640 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5618d3693d90 0x5618d36961b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=182ec6656d0ba44c server_cookie=bf71df181c16d826 in_seq=0 out_seq=0 2024-04-17T07:31:00.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.161+0000 7f461a000640 10 client.19251 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:31:00.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.162+0000 7f461a000640 1 -- 172.21.15.86:0/4092524212 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618d35858b0 con 0x5618d3693d90 2024-04-17T07:31:00.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.162+0000 7f461a000640 10 client.19251 handle_command_reply: tid=0 2024-04-17T07:31:00.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.162+0000 7f46397fd740 1 client.19251 shutdown 2024-04-17T07:31:00.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.163+0000 7f46397fd740 1 -- 172.21.15.86:0/4092524212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 msgr2=0x5618d36921a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.163+0000 7f46397fd740 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d36921a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f462804e000 tx=0x7f4628009500 comp rx=0 tx=0).stop 2024-04-17T07:31:00.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.164+0000 7f46397fd740 1 -- 172.21.15.86:0/4092524212 shutdown_connections 2024-04-17T07:31:00.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.164+0000 7f46397fd740 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5618d3693d90 0x5618d36961b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.164+0000 7f46397fd740 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618d361db00 0x5618d36921a0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.164+0000 7f46397fd740 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5618d3618990 0x5618d3626b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.164+0000 7f46397fd740 1 --2- 172.21.15.86:0/4092524212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618d3617fa0 0x5618d36265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.164+0000 7f46397fd740 1 -- 172.21.15.86:0/4092524212 >> 172.21.15.86:0/4092524212 conn(0x5618d3613510 msgr2=0x5618d3616770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.164+0000 7f461a000640 0 client.19251 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:31:00.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.164+0000 7f46397fd740 1 -- 172.21.15.86:0/4092524212 shutdown_connections 2024-04-17T07:31:00.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.164+0000 7f46397fd740 1 -- 172.21.15.86:0/4092524212 wait complete. 2024-04-17T07:31:00.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.164+0000 7f46397fd740 20 client.19251 trim_cache size 0 max 16384 2024-04-17T07:31:00.165 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:31:00.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.166+0000 7f4636e00640 1 -- 172.21.15.86:0/3221215411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46140a3d00 msgr2=0x7f46140a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.167+0000 7f4636e00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46140a3d00 0x7f46140a61c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5618d3614480 tx=0x7f4620005f90 comp rx=0 tx=0).stop 2024-04-17T07:31:00.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.167+0000 7f4636e00640 1 -- 172.21.15.86:0/3221215411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f463010e2a0 msgr2=0x7f46301ae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.167+0000 7f4636e00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f463010e2a0 0x7f46301ae570 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4628034fd0 tx=0x7f4628033c40 comp rx=0 tx=0).stop 2024-04-17T07:31:00.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.167+0000 7f4636e00640 1 -- 172.21.15.86:0/3221215411 shutdown_connections 2024-04-17T07:31:00.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.167+0000 7f4636e00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f46140a3d00 0x7f46140a61c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.167+0000 7f4636e00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f463010e2a0 0x7f46301ae570 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.167+0000 7f4636e00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f463010b8d0 0x7f46301a7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.167+0000 7f4636e00640 1 --2- 172.21.15.86:0/3221215411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4630108f60 0x7f46301a6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.167+0000 7f4636e00640 1 -- 172.21.15.86:0/3221215411 >> 172.21.15.86:0/3221215411 conn(0x7f463006edf0 msgr2=0x7f463010bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.168+0000 7f4636e00640 1 -- 172.21.15.86:0/3221215411 shutdown_connections 2024-04-17T07:31:00.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.168+0000 7f4636e00640 1 -- 172.21.15.86:0/3221215411 wait complete. 2024-04-17T07:31:00.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:31:00.326 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:00 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4234583521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:31:00.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:00 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4234583521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:31:00.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:00 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4234583521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:31:00.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.569+0000 7f98cf200640 1 -- 172.21.15.86:0/3639134148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98c409d560 msgr2=0x7f98c409d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.569+0000 7f98cf200640 1 --2- 172.21.15.86:0/3639134148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98c409d560 0x7f98c409d9e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f98c8067c60 tx=0x7f98c8098310 comp rx=0 tx=0).stop 2024-04-17T07:31:00.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.570+0000 7f98cf200640 1 -- 172.21.15.86:0/3639134148 shutdown_connections 2024-04-17T07:31:00.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.570+0000 7f98cf200640 1 --2- 172.21.15.86:0/3639134148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c409df20 0x7f98c40a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.570+0000 7f98cf200640 1 --2- 172.21.15.86:0/3639134148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98c409d560 0x7f98c409d9e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.570+0000 7f98cf200640 1 --2- 172.21.15.86:0/3639134148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c409c370 0x7f98c409c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.570+0000 7f98cf200640 1 -- 172.21.15.86:0/3639134148 >> 172.21.15.86:0/3639134148 conn(0x7f98c400b310 msgr2=0x7f98c409a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.571+0000 7f98cf200640 1 -- 172.21.15.86:0/3639134148 shutdown_connections 2024-04-17T07:31:00.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.571+0000 7f98cf200640 1 -- 172.21.15.86:0/3639134148 wait complete. 2024-04-17T07:31:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.575+0000 7f98cf200640 1 Processor -- start 2024-04-17T07:31:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.576+0000 7f98cf200640 1 -- start start 2024-04-17T07:31:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.576+0000 7f98cf200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98c409c370 0x7f98c41325a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.576+0000 7f98cf200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c409d560 0x7f98c4132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.576+0000 7f98cf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c409df20 0x7f98c4139b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.576+0000 7f98cf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98c40a7e80 con 0x7f98c409df20 2024-04-17T07:31:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.576+0000 7f98cf200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98c40a7d00 con 0x7f98c409c370 2024-04-17T07:31:00.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.576+0000 7f98cf200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98c40a8000 con 0x7f98c409d560 2024-04-17T07:31:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.577+0000 7f98cd400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c409d560 0x7f98c4132ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.577+0000 7f98ce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c409df20 0x7f98c4139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.577+0000 7f98cd400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c409d560 0x7f98c4132ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54500/0 (socket says 172.21.15.86:54500) 2024-04-17T07:31:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.577+0000 7f98cd400640 1 -- 172.21.15.86:0/2311568373 learned_addr learned my addr 172.21.15.86:0/2311568373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.578+0000 7f98ce800640 1 -- 172.21.15.86:0/2311568373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c409d560 msgr2=0x7f98c4132ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.578+0000 7f98ce800640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c409d560 0x7f98c4132ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.578+0000 7f98ce800640 1 -- 172.21.15.86:0/2311568373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98c409c370 msgr2=0x7f98c41325a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.578+0000 7f98ce800640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98c409c370 0x7f98c41325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.578+0000 7f98ce800640 1 -- 172.21.15.86:0/2311568373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c413a1c0 con 0x7f98c409df20 2024-04-17T07:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.578+0000 7f98ce800640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c409df20 0x7f98c4139b50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00b410 tx=0x7f98bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.579+0000 7f98c2a00640 1 -- 172.21.15.86:0/2311568373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc015020 con 0x7f98c409df20 2024-04-17T07:31:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.579+0000 7f98c2a00640 1 -- 172.21.15.86:0/2311568373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98bc011ce0 con 0x7f98c409df20 2024-04-17T07:31:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.579+0000 7f98c2a00640 1 -- 172.21.15.86:0/2311568373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc010ce0 con 0x7f98c409df20 2024-04-17T07:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.580+0000 7f98cf200640 1 -- 172.21.15.86:0/2311568373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c413a4b0 con 0x7f98c409df20 2024-04-17T07:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.580+0000 7f98cf200640 1 -- 172.21.15.86:0/2311568373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c413aa00 con 0x7f98c409df20 2024-04-17T07:31:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.587+0000 7f98c2a00640 1 -- 172.21.15.86:0/2311568373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f98bc018020 con 0x7f98c409df20 2024-04-17T07:31:00.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.591+0000 7f98d342f740 1 Processor -- start 2024-04-17T07:31:00.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.591+0000 7f98d342f740 1 -- start start 2024-04-17T07:31:00.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.590+0000 7f98c2a00640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b00a3d00 0x7f98b00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.592+0000 7f98d342f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.592+0000 7f98d342f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 0x55f2611d9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.592+0000 7f98d342f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d9cc0 0x55f2611dc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.592+0000 7f98d342f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f26113a020 con 0x55f2611d4b50 2024-04-17T07:31:00.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.592+0000 7f98d342f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f2610e0460 con 0x55f2611d4160 2024-04-17T07:31:00.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.592+0000 7f98d342f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f261135fe0 con 0x55f2611d9cc0 2024-04-17T07:31:00.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.592+0000 7f98c2a00640 1 -- 172.21.15.86:0/2311568373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f98bc0cfe20 con 0x7f98c409df20 2024-04-17T07:31:00.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.592+0000 7f98cd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 0x55f2611d9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.592+0000 7f98cd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 0x55f2611d9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47036/0 (socket says 172.21.15.86:47036) 2024-04-17T07:31:00.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.593+0000 7f98cd400640 1 -- 172.21.15.86:0/1991095764 learned_addr learned my addr 172.21.15.86:0/1991095764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:00.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.593+0000 7f98ce800640 1 --2- 172.21.15.86:0/1991095764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d9cc0 0x55f2611dc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.593+0000 7f98cde00640 1 --2- 172.21.15.86:0/1991095764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.593+0000 7f98cd400640 1 -- 172.21.15.86:0/1991095764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d9cc0 msgr2=0x55f2611dc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.593+0000 7f98cd400640 1 --2- 172.21.15.86:0/1991095764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d9cc0 0x55f2611dc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.593+0000 7f98cd400640 1 -- 172.21.15.86:0/1991095764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 msgr2=0x55f2611d4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.593+0000 7f98cd400640 1 --2- 172.21.15.86:0/1991095764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611d4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.593+0000 7f98cd400640 1 -- 172.21.15.86:0/1991095764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2610e8040 con 0x55f2611d4b50 2024-04-17T07:31:00.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.594+0000 7f98cd400640 1 --2- 172.21.15.86:0/1991095764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 0x55f2611d9650 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f98c8067750 tx=0x7f98c806a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa785292d948dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.594+0000 7f98cde00640 1 --2- 172.21.15.86:0/1991095764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.594+0000 7f98cde00640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b00a3d00 0x7f98b00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.594+0000 7f98b7000640 1 -- 172.21.15.86:0/1991095764 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c80a06d0 con 0x55f2611d4b50 2024-04-17T07:31:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.594+0000 7f98b7000640 1 -- 172.21.15.86:0/1991095764 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98c809c040 con 0x55f2611d4b50 2024-04-17T07:31:00.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.594+0000 7f98b7000640 1 -- 172.21.15.86:0/1991095764 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8068670 con 0x55f2611d4b50 2024-04-17T07:31:00.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.595+0000 7f98d342f740 1 -- 172.21.15.86:0/1991095764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 msgr2=0x55f2611d9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.595+0000 7f98d342f740 1 --2- 172.21.15.86:0/1991095764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 0x55f2611d9650 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f98c8067750 tx=0x7f98c806a330 comp rx=0 tx=0).stop 2024-04-17T07:31:00.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.596+0000 7f98d342f740 1 -- 172.21.15.86:0/1991095764 shutdown_connections 2024-04-17T07:31:00.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.596+0000 7f98d342f740 1 --2- 172.21.15.86:0/1991095764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d9cc0 0x55f2611dc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.596+0000 7f98d342f740 1 --2- 172.21.15.86:0/1991095764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 0x55f2611d9650 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.596+0000 7f98d342f740 1 --2- 172.21.15.86:0/1991095764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.596+0000 7f98d342f740 1 -- 172.21.15.86:0/1991095764 >> 172.21.15.86:0/1991095764 conn(0x55f2611cf6d0 msgr2=0x55f2611cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.596+0000 7f98d342f740 1 -- 172.21.15.86:0/1991095764 shutdown_connections 2024-04-17T07:31:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.597+0000 7f98cde00640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b00a3d00 0x7f98b00a61c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55f2611d53b0 tx=0x7f98b8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.597+0000 7f98d342f740 1 -- 172.21.15.86:0/1991095764 wait complete. 2024-04-17T07:31:00.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.598+0000 7f98d342f740 1 -- start start 2024-04-17T07:31:00.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.598+0000 7f98d342f740 1 Processor -- start 2024-04-17T07:31:00.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98d342f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4160 0x55f2611ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98d342f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4b50 0x55f2611efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98d342f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d9cc0 0x55f2611f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98d342f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f2611edc20 con 0x55f2611d4160 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98d342f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f2610f3830 con 0x55f2611d4b50 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98d342f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2611edda0 con 0x55f2611d9cc0 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98d342f740 10 client.0 _ll_register_callbacks cb 0x55f2610c1fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98cde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4160 0x55f2611ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98cd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4b50 0x55f2611efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98cd400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4b50 0x55f2611efbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58234/0 (socket says 172.21.15.86:58234) 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.603+0000 7f98cd400640 1 -- 172.21.15.86:0/4223514714 learned_addr learned my addr 172.21.15.86:0/4223514714 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.604+0000 7f98cd400640 1 -- 172.21.15.86:0/4223514714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d9cc0 msgr2=0x55f2611f6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.604+0000 7f98cd400640 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d9cc0 0x55f2611f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.604+0000 7f98cd400640 1 -- 172.21.15.86:0/4223514714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4160 msgr2=0x55f2611ef6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.604+0000 7f98cd400640 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4160 0x55f2611ef6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.604+0000 7f98cd400640 1 -- 172.21.15.86:0/4223514714 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2610f7920 con 0x55f2611d4b50 2024-04-17T07:31:00.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.605+0000 7f98cd400640 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4b50 0x55f2611efbe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f98c8067850 tx=0x7f98c80a0d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3446242080c3cbd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.605+0000 7f98ab400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:31:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.607+0000 7f98ab400640 1 -- 172.21.15.86:0/4223514714 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c80b0030 con 0x55f2611d4b50 2024-04-17T07:31:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.607+0000 7f98ab400640 1 -- 172.21.15.86:0/4223514714 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f98c80a06e0 con 0x55f2611d4b50 2024-04-17T07:31:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.607+0000 7f98d342f740 1 -- 172.21.15.86:0/4223514714 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c809c040 con 0x55f2611d4b50 2024-04-17T07:31:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.608+0000 7f98ab400640 1 -- 172.21.15.86:0/4223514714 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98c80ba050 con 0x55f2611d4b50 2024-04-17T07:31:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.608+0000 7f98d342f740 10 client.38231 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.608+0000 7f98d342f740 1 -- 172.21.15.86:0/4223514714 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2610c57f0 con 0x55f2611d4b50 2024-04-17T07:31:00.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.608+0000 7f98ab400640 1 -- 172.21.15.86:0/4223514714 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98c809c040 con 0x55f2611d4b50 2024-04-17T07:31:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.609+0000 7f98ab400640 1 -- 172.21.15.86:0/4223514714 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f98c8072020 con 0x55f2611d4b50 2024-04-17T07:31:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.609+0000 7f98d342f740 10 client.38231 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.609+0000 7f98d342f740 10 client.38231 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:31:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.609+0000 7f98d342f740 20 client.38231 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.609+0000 7f98d342f740 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55f2611fad40 0x55f2611fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.610+0000 7f98d342f740 4 client.38231 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.610+0000 7f98d342f740 1 -- 172.21.15.86:0/4223514714 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f261147a40 con 0x55f2611fad40 2024-04-17T07:31:00.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.610+0000 7f98cde00640 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55f2611fad40 0x55f2611fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.613+0000 7f98cde00640 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55f2611fad40 0x55f2611fd130 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ae1d7dcd6beda1c server_cookie=937ceab50e38da26 in_seq=0 out_seq=0 2024-04-17T07:31:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.613+0000 7f98ab400640 10 client.38231 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:31:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.616+0000 7f98ab400640 1 -- 172.21.15.86:0/4223514714 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f261147a40 con 0x55f2611fad40 2024-04-17T07:31:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.616+0000 7f98ab400640 10 client.38231 handle_command_reply: tid=0 2024-04-17T07:31:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.617+0000 7f98d342f740 1 client.38231 shutdown 2024-04-17T07:31:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.617+0000 7f98d342f740 1 -- 172.21.15.86:0/4223514714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4b50 msgr2=0x55f2611efbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.617+0000 7f98d342f740 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4b50 0x55f2611efbe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f98c8067850 tx=0x7f98c80a0d00 comp rx=0 tx=0).stop 2024-04-17T07:31:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.617+0000 7f98d342f740 1 -- 172.21.15.86:0/4223514714 shutdown_connections 2024-04-17T07:31:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.617+0000 7f98d342f740 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55f2611fad40 0x55f2611fd130 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.617+0000 7f98d342f740 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d9cc0 0x55f2611f6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.617+0000 7f98d342f740 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4b50 0x55f2611efbe0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.617+0000 7f98d342f740 1 --2- 172.21.15.86:0/4223514714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4160 0x55f2611ef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.617+0000 7f98d342f740 1 -- 172.21.15.86:0/4223514714 >> 172.21.15.86:0/4223514714 conn(0x55f2611cf6d0 msgr2=0x55f2611ce930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.618+0000 7f98ab400640 0 client.38231 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:31:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.618+0000 7f98d342f740 1 -- 172.21.15.86:0/4223514714 shutdown_connections 2024-04-17T07:31:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.618+0000 7f98d342f740 1 -- 172.21.15.86:0/4223514714 wait complete. 2024-04-17T07:31:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.618+0000 7f98d342f740 20 client.38231 trim_cache size 0 max 16384 2024-04-17T07:31:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.652+0000 7f98d342f740 1 Processor -- start 2024-04-17T07:31:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.653+0000 7f98d342f740 1 -- start start 2024-04-17T07:31:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.653+0000 7f98d342f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d4160 0x55f2611d53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.653+0000 7f98d342f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 0x55f2611f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.653+0000 7f98d342f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d9cc0 0x55f2611ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.653+0000 7f98d342f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98c4144e10 con 0x55f2611d4b50 2024-04-17T07:31:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.653+0000 7f98d342f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98c4146950 con 0x55f2611d9cc0 2024-04-17T07:31:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.653+0000 7f98d342f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98c41454e0 con 0x55f2611d4160 2024-04-17T07:31:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.654+0000 7f98ce800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d9cc0 0x55f2611ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.654+0000 7f98ce800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d9cc0 0x55f2611ef6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58248/0 (socket says 172.21.15.86:58248) 2024-04-17T07:31:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.654+0000 7f98ce800640 1 -- 172.21.15.86:0/2100660684 learned_addr learned my addr 172.21.15.86:0/2100660684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.654+0000 7f98cde00640 1 --2- 172.21.15.86:0/2100660684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d4160 0x55f2611d53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.654+0000 7f98ce800640 1 -- 172.21.15.86:0/2100660684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d4160 msgr2=0x55f2611d53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.654+0000 7f98ce800640 1 --2- 172.21.15.86:0/2100660684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d4160 0x55f2611d53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.654+0000 7f98ce800640 1 -- 172.21.15.86:0/2100660684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 msgr2=0x55f2611f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.654+0000 7f98ce800640 1 --2- 172.21.15.86:0/2100660684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 0x55f2611f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.654+0000 7f98ce800640 1 -- 172.21.15.86:0/2100660684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f260e4c3f0 con 0x55f2611d9cc0 2024-04-17T07:31:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.655+0000 7f98ce800640 1 --2- 172.21.15.86:0/2100660684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d9cc0 0x55f2611ef6a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f98bc09bcb0 tx=0x7f98bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628b0b9ba3239ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.656+0000 7f98b6600640 1 -- 172.21.15.86:0/2100660684 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0b5270 con 0x55f2611d9cc0 2024-04-17T07:31:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.656+0000 7f98b6600640 1 -- 172.21.15.86:0/2100660684 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98bc0a00d0 con 0x55f2611d9cc0 2024-04-17T07:31:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.656+0000 7f98b6600640 1 -- 172.21.15.86:0/2100660684 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0bc0b0 con 0x55f2611d9cc0 2024-04-17T07:31:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.657+0000 7f98d342f740 1 -- 172.21.15.86:0/2100660684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d9cc0 msgr2=0x55f2611ef6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.657+0000 7f98d342f740 1 --2- 172.21.15.86:0/2100660684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d9cc0 0x55f2611ef6a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f98bc09bcb0 tx=0x7f98bc0046c0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.657+0000 7f98d342f740 1 -- 172.21.15.86:0/2100660684 shutdown_connections 2024-04-17T07:31:00.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.657+0000 7f98d342f740 1 --2- 172.21.15.86:0/2100660684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d9cc0 0x55f2611ef6a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.657+0000 7f98d342f740 1 --2- 172.21.15.86:0/2100660684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d4b50 0x55f2611f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.657+0000 7f98d342f740 1 --2- 172.21.15.86:0/2100660684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d4160 0x55f2611d53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.657+0000 7f98d342f740 1 -- 172.21.15.86:0/2100660684 >> 172.21.15.86:0/2100660684 conn(0x55f2611cf6d0 msgr2=0x55f2611eecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.658+0000 7f98d342f740 1 -- 172.21.15.86:0/2100660684 shutdown_connections 2024-04-17T07:31:00.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.658+0000 7f98d342f740 1 -- 172.21.15.86:0/2100660684 wait complete. 2024-04-17T07:31:00.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.659+0000 7f98d342f740 1 -- start start 2024-04-17T07:31:00.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.660+0000 7f98d342f740 1 Processor -- start 2024-04-17T07:31:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.661+0000 7f98d342f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.661+0000 7f98d342f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d4b50 0x55f2611f4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.661+0000 7f98d342f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d9cc0 0x55f26125f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.661+0000 7f98d342f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98c4146280 con 0x55f2611d9cc0 2024-04-17T07:31:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.661+0000 7f98d342f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98c4145bb0 con 0x55f2611d4160 2024-04-17T07:31:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.661+0000 7f98d342f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98c4142de0 con 0x55f2611d4b50 2024-04-17T07:31:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.661+0000 7f98d342f740 10 client.0 _ll_register_callbacks cb 0x7f98c806a050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.661+0000 7f98cde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.661+0000 7f98cde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58250/0 (socket says 172.21.15.86:58250) 2024-04-17T07:31:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.661+0000 7f98cde00640 1 -- 172.21.15.86:0/1480567394 learned_addr learned my addr 172.21.15.86:0/1480567394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.662+0000 7f98ce800640 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d9cc0 0x55f26125f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.662+0000 7f98cde00640 1 -- 172.21.15.86:0/1480567394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d4b50 msgr2=0x55f2611f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.662+0000 7f98cde00640 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d4b50 0x55f2611f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.662+0000 7f98cde00640 1 -- 172.21.15.86:0/1480567394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d9cc0 msgr2=0x55f26125f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.663+0000 7f98cde00640 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d9cc0 0x55f26125f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.663+0000 7f98cde00640 1 -- 172.21.15.86:0/1480567394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f260f167d0 con 0x55f2611d4160 2024-04-17T07:31:00.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.664+0000 7f98ce800640 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d9cc0 0x55f26125f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:00.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.665+0000 7f98cde00640 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611f3c00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f98c409d3c0 tx=0x7f98b8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61571bca3668127b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:00.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.665+0000 7f98ab400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:31:00.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.666+0000 7f98ab400640 1 -- 172.21.15.86:0/1480567394 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8050070 con 0x55f2611d4160 2024-04-17T07:31:00.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.666+0000 7f98ab400640 1 -- 172.21.15.86:0/1480567394 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f98b8005220 con 0x55f2611d4160 2024-04-17T07:31:00.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.666+0000 7f98d342f740 1 -- 172.21.15.86:0/1480567394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98bc0a00d0 con 0x55f2611d4160 2024-04-17T07:31:00.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.667+0000 7f98ab400640 1 -- 172.21.15.86:0/1480567394 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98b805b050 con 0x55f2611d4160 2024-04-17T07:31:00.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.668+0000 7f98d342f740 10 client.38237 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:00.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.668+0000 7f98d342f740 1 -- 172.21.15.86:0/1480567394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f26125f8e0 con 0x55f2611d4160 2024-04-17T07:31:00.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.669+0000 7f98ab400640 1 -- 172.21.15.86:0/1480567394 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98bc0a00d0 con 0x55f2611d4160 2024-04-17T07:31:00.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.669+0000 7f98ab400640 1 -- 172.21.15.86:0/1480567394 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f98b8028310 con 0x55f2611d4160 2024-04-17T07:31:00.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.669+0000 7f98d342f740 10 client.38237 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:00.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.669+0000 7f98d342f740 10 client.38237 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:31:00.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.669+0000 7f98d342f740 20 client.38237 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:00.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.670+0000 7f98d342f740 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55f261260270 0x55f261262690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:00.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.670+0000 7f98d342f740 4 client.38237 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:31:00.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.670+0000 7f98d342f740 1 -- 172.21.15.86:0/1480567394 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2611418b0 con 0x55f261260270 2024-04-17T07:31:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.673+0000 7f98cd400640 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55f261260270 0x55f261262690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:00.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.676+0000 7f98cd400640 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55f261260270 0x55f261262690 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb2d4055c3046ea4 server_cookie=6f1d3b2493b88045 in_seq=0 out_seq=0 2024-04-17T07:31:00.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.676+0000 7f98ab400640 10 client.38237 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:31:00.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.676+0000 7f98ab400640 1 -- 172.21.15.86:0/1480567394 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2611418b0 con 0x55f261260270 2024-04-17T07:31:00.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.676+0000 7f98ab400640 10 client.38237 handle_command_reply: tid=0 2024-04-17T07:31:00.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.676+0000 7f98d342f740 1 client.38237 shutdown 2024-04-17T07:31:00.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.677+0000 7f98d342f740 1 -- 172.21.15.86:0/1480567394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 msgr2=0x55f2611f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.677+0000 7f98d342f740 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611f3c00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f98c409d3c0 tx=0x7f98b8009260 comp rx=0 tx=0).stop 2024-04-17T07:31:00.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.677+0000 7f98d342f740 1 -- 172.21.15.86:0/1480567394 shutdown_connections 2024-04-17T07:31:00.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.677+0000 7f98d342f740 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55f261260270 0x55f261262690 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.677+0000 7f98ab400640 0 client.38237 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:31:00.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.677+0000 7f98d342f740 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f2611d9cc0 0x55f26125f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.677+0000 7f98d342f740 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2611d4b50 0x55f2611f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.677+0000 7f98d342f740 1 --2- 172.21.15.86:0/1480567394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f2611d4160 0x55f2611f3c00 secure :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f98c409d3c0 tx=0x7f98b8009260 comp rx=0 tx=0).stop 2024-04-17T07:31:00.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.677+0000 7f98d342f740 1 -- 172.21.15.86:0/1480567394 >> 172.21.15.86:0/1480567394 conn(0x55f2611cf6d0 msgr2=0x55f2611eebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.679+0000 7f98d342f740 1 -- 172.21.15.86:0/1480567394 shutdown_connections 2024-04-17T07:31:00.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.679+0000 7f98d342f740 1 -- 172.21.15.86:0/1480567394 wait complete. 2024-04-17T07:31:00.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.680+0000 7f98d342f740 20 client.38237 trim_cache size 0 max 16384 2024-04-17T07:31:00.680 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:31:00.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.682+0000 7f98cf200640 1 -- 172.21.15.86:0/2311568373 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b00a3d00 msgr2=0x7f98b00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.682+0000 7f98cf200640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b00a3d00 0x7f98b00a61c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55f2611d53b0 tx=0x7f98b8005850 comp rx=0 tx=0).stop 2024-04-17T07:31:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.682+0000 7f98cf200640 1 -- 172.21.15.86:0/2311568373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c409df20 msgr2=0x7f98c4139b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.682+0000 7f98cf200640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c409df20 0x7f98c4139b50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00b410 tx=0x7f98bc00b8d0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.682+0000 7f98cf200640 1 -- 172.21.15.86:0/2311568373 shutdown_connections 2024-04-17T07:31:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.682+0000 7f98cf200640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b00a3d00 0x7f98b00a61c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.682+0000 7f98cf200640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c409df20 0x7f98c4139b50 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.682+0000 7f98cf200640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c409d560 0x7f98c4132ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.682+0000 7f98cf200640 1 --2- 172.21.15.86:0/2311568373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98c409c370 0x7f98c41325a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:00.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.682+0000 7f98cf200640 1 -- 172.21.15.86:0/2311568373 >> 172.21.15.86:0/2311568373 conn(0x7f98c400b310 msgr2=0x7f98c40a0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:00.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.683+0000 7f98cf200640 1 -- 172.21.15.86:0/2311568373 shutdown_connections 2024-04-17T07:31:00.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:00.683+0000 7f98cf200640 1 -- 172.21.15.86:0/2311568373 wait complete. 2024-04-17T07:31:00.719 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:31:01.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.082+0000 7f5cb9400640 1 -- 172.21.15.86:0/2722186513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cb410e280 msgr2=0x7f5cb410e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.082+0000 7f5cb9400640 1 --2- 172.21.15.86:0/2722186513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cb410e280 0x7f5cb410e700 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009be0 tx=0x7f5cac02bf30 comp rx=0 tx=0).stop 2024-04-17T07:31:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.083+0000 7f5cb9400640 1 -- 172.21.15.86:0/2722186513 shutdown_connections 2024-04-17T07:31:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.083+0000 7f5cb9400640 1 --2- 172.21.15.86:0/2722186513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cb410e280 0x7f5cb410e700 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.083+0000 7f5cb9400640 1 --2- 172.21.15.86:0/2722186513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb410b8d0 0x7f5cb410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.083+0000 7f5cb9400640 1 --2- 172.21.15.86:0/2722186513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb4108f60 0x7f5cb410b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.083+0000 7f5cb9400640 1 -- 172.21.15.86:0/2722186513 >> 172.21.15.86:0/2722186513 conn(0x7f5cb406f690 msgr2=0x7f5cb4071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.083+0000 7f5cb9400640 1 -- 172.21.15.86:0/2722186513 shutdown_connections 2024-04-17T07:31:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.083+0000 7f5cb9400640 1 -- 172.21.15.86:0/2722186513 wait complete. 2024-04-17T07:31:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.084+0000 7f5cb9400640 1 Processor -- start 2024-04-17T07:31:01.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.085+0000 7f5cb9400640 1 -- start start 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.085+0000 7f5cb9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cb4108f60 0x7f5cb41a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.085+0000 7f5cb9400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb410b8d0 0x7f5cb41a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.086+0000 7f5cb9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb410e280 0x7f5cb41ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.086+0000 7f5cb9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb411c950 con 0x7f5cb410e280 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.086+0000 7f5cb9400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb411c7d0 con 0x7f5cb410b8d0 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.086+0000 7f5cb9400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb411cad0 con 0x7f5cb4108f60 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.086+0000 7f5cb1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb410b8d0 0x7f5cb41a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.086+0000 7f5cb1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb410b8d0 0x7f5cb41a74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58264/0 (socket says 172.21.15.86:58264) 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.086+0000 7f5cb1e00640 1 -- 172.21.15.86:0/4072185570 learned_addr learned my addr 172.21.15.86:0/4072185570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.086+0000 7f5cb2800640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cb4108f60 0x7f5cb41a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.087+0000 7f5cb3200640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb410e280 0x7f5cb41ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.087+0000 7f5cb1e00640 1 -- 172.21.15.86:0/4072185570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cb4108f60 msgr2=0x7f5cb41a6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.087+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cb4108f60 0x7f5cb41a6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.087+0000 7f5cb1e00640 1 -- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb410e280 msgr2=0x7f5cb41ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.087+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb410e280 0x7f5cb41ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.087+0000 7f5cb1e00640 1 -- 172.21.15.86:0/4072185570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb41aebc0 con 0x7f5cb410b8d0 2024-04-17T07:31:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.087+0000 7f5cb3200640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb410e280 0x7f5cb41ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.087+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb410b8d0 0x7f5cb41a74e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000d950 tx=0x7f5ca000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.088+0000 7f5cab400640 1 -- 172.21.15.86:0/4072185570 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0017070 con 0x7f5cb410b8d0 2024-04-17T07:31:01.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.088+0000 7f5cab400640 1 -- 172.21.15.86:0/4072185570 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ca0009c30 con 0x7f5cb410b8d0 2024-04-17T07:31:01.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.088+0000 7f5cab400640 1 -- 172.21.15.86:0/4072185570 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000f620 con 0x7f5cb410b8d0 2024-04-17T07:31:01.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.088+0000 7f5cb9400640 1 -- 172.21.15.86:0/4072185570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb41aeeb0 con 0x7f5cb410b8d0 2024-04-17T07:31:01.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.089+0000 7f5cb9400640 1 -- 172.21.15.86:0/4072185570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb41af320 con 0x7f5cb410b8d0 2024-04-17T07:31:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.094+0000 7f5cbc1b8740 1 Processor -- start 2024-04-17T07:31:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.094+0000 7f5cbc1b8740 1 -- start start 2024-04-17T07:31:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.095+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb801fa0 0x55f5bb8023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.095+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb802990 0x55f5bb807490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.095+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb807b00 0x55f5bb809f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.095+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5bb768020 con 0x55f5bb807b00 2024-04-17T07:31:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.095+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f5bb70e460 con 0x55f5bb801fa0 2024-04-17T07:31:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.095+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f5bb763fe0 con 0x55f5bb802990 2024-04-17T07:31:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.096+0000 7f5cb2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb802990 0x55f5bb807490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.096+0000 7f5cb2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb802990 0x55f5bb807490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54580/0 (socket says 172.21.15.86:54580) 2024-04-17T07:31:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.096+0000 7f5cb2800640 1 -- 172.21.15.86:0/3390406260 learned_addr learned my addr 172.21.15.86:0/3390406260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.096+0000 7f5cb2800640 1 -- 172.21.15.86:0/3390406260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb801fa0 msgr2=0x55f5bb8023c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:31:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.096+0000 7f5cb2800640 1 --2- 172.21.15.86:0/3390406260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb801fa0 0x55f5bb8023c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.096+0000 7f5cb2800640 1 -- 172.21.15.86:0/3390406260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb807b00 msgr2=0x55f5bb809f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:31:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.096+0000 7f5cab400640 1 -- 172.21.15.86:0/4072185570 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5ca00109a0 con 0x7f5cb410b8d0 2024-04-17T07:31:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.097+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/3390406260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb807b00 0x55f5bb809f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.097+0000 7f5cb2800640 1 --2- 172.21.15.86:0/3390406260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb807b00 0x55f5bb809f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.097+0000 7f5cb2800640 1 -- 172.21.15.86:0/3390406260 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5bb716040 con 0x55f5bb802990 2024-04-17T07:31:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.097+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/3390406260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb807b00 0x55f5bb809f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.098+0000 7f5cb2800640 1 --2- 172.21.15.86:0/3390406260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb802990 0x55f5bb807490 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b740 tx=0x7f5ca400bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a74c36e1dc1ffa7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.098+0000 7f5c9f400640 1 -- 172.21.15.86:0/3390406260 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4003dd0 con 0x55f5bb802990 2024-04-17T07:31:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.098+0000 7f5c9f400640 1 -- 172.21.15.86:0/3390406260 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ca4015070 con 0x55f5bb802990 2024-04-17T07:31:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.099+0000 7f5c9f400640 1 -- 172.21.15.86:0/3390406260 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40116a0 con 0x55f5bb802990 2024-04-17T07:31:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.100+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/3390406260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb802990 msgr2=0x55f5bb807490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.100+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/3390406260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb802990 0x55f5bb807490 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b740 tx=0x7f5ca400bc00 comp rx=0 tx=0).stop 2024-04-17T07:31:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.100+0000 7f5cab400640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5c980a3cb0 0x7f5c980a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.101+0000 7f5cab400640 1 -- 172.21.15.86:0/4072185570 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5ca00cfcc0 con 0x7f5cb410b8d0 2024-04-17T07:31:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.101+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/3390406260 shutdown_connections 2024-04-17T07:31:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.101+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/3390406260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb807b00 0x55f5bb809f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.101+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/3390406260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb802990 0x55f5bb807490 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.101+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/3390406260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb801fa0 0x55f5bb8023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.101+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/3390406260 >> 172.21.15.86:0/3390406260 conn(0x55f5bb7fd510 msgr2=0x55f5bb7fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.101+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/3390406260 shutdown_connections 2024-04-17T07:31:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.102+0000 7f5cb3200640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5c980a3cb0 0x7f5c980a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.104+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/3390406260 wait complete. 2024-04-17T07:31:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.104+0000 7f5cbc1b8740 1 -- start start 2024-04-17T07:31:01.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.105+0000 7f5cbc1b8740 1 Processor -- start 2024-04-17T07:31:01.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.106+0000 7f5cb3200640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5c980a3cb0 0x7f5c980a6170 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0099a0 tx=0x7f5cac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.106+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb801fa0 0x55f5bb80c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.106+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb802990 0x55f5bb80ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.106+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.107+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5bb80aab0 con 0x55f5bb802990 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.107+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f5bb721830 con 0x55f5bb807b00 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.107+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f5bb80ac30 con 0x55f5bb801fa0 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.107+0000 7f5cb3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.107+0000 7f5cb3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58270/0 (socket says 172.21.15.86:58270) 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.107+0000 7f5cb3200640 1 -- 172.21.15.86:0/890164230 learned_addr learned my addr 172.21.15.86:0/890164230 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.107+0000 7f5cbc1b8740 10 client.0 _ll_register_callbacks cb 0x55f5bb6effb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.108+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb802990 0x55f5bb80ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.109+0000 7f5cb2800640 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb801fa0 0x55f5bb80c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.109+0000 7f5cb1e00640 1 -- 172.21.15.86:0/890164230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb801fa0 msgr2=0x55f5bb80c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.109+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb801fa0 0x55f5bb80c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.109+0000 7f5cb1e00640 1 -- 172.21.15.86:0/890164230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 msgr2=0x55f5bb813ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.109+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.109+0000 7f5cb1e00640 1 -- 172.21.15.86:0/890164230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5bb725920 con 0x55f5bb802990 2024-04-17T07:31:01.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.110+0000 7f5c93e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:01.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.110+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb802990 0x55f5bb80ca80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f5ca009bb50 tx=0x7f5ca009b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a671b9e04f18b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.111+0000 7f5c93e00640 1 -- 172.21.15.86:0/890164230 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0020070 con 0x55f5bb802990 2024-04-17T07:31:01.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.111+0000 7f5c93e00640 1 -- 172.21.15.86:0/890164230 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5ca0098230 con 0x55f5bb802990 2024-04-17T07:31:01.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.111+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/890164230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ca4015070 con 0x55f5bb802990 2024-04-17T07:31:01.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.112+0000 7f5c93e00640 1 -- 172.21.15.86:0/890164230 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5ca009ff70 con 0x55f5bb802990 2024-04-17T07:31:01.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.112+0000 7f5cbc1b8740 10 client.19269 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:01.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.112+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/890164230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5bb6f37f0 con 0x55f5bb802990 2024-04-17T07:31:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.113+0000 7f5c93e00640 1 -- 172.21.15.86:0/890164230 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ca4015070 con 0x55f5bb802990 2024-04-17T07:31:01.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.114+0000 7f5c93e00640 1 -- 172.21.15.86:0/890164230 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5ca0010460 con 0x55f5bb802990 2024-04-17T07:31:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.115+0000 7f5cbc1b8740 10 client.19269 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.115+0000 7f5cbc1b8740 10 client.19269 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:31:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.115+0000 7f5cbc1b8740 20 client.19269 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.115+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f5bb817ed0 0x55f5bb81a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.115+0000 7f5cbc1b8740 4 client.19269 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:01.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.115+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/890164230 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5bb775a40 con 0x55f5bb817ed0 2024-04-17T07:31:01.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.116+0000 7f5cb2800640 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f5bb817ed0 0x55f5bb81a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.118+0000 7f5cb2800640 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f5bb817ed0 0x55f5bb81a390 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50f474947e530a10 server_cookie=f51f589e656c86e3 in_seq=0 out_seq=0 2024-04-17T07:31:01.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.118+0000 7f5c93e00640 10 client.19269 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:31:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.119+0000 7f5c93e00640 1 -- 172.21.15.86:0/890164230 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5bb775a40 con 0x55f5bb817ed0 2024-04-17T07:31:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.119+0000 7f5c93e00640 10 client.19269 handle_command_reply: tid=0 2024-04-17T07:31:01.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.119+0000 7f5cbc1b8740 1 client.19269 shutdown 2024-04-17T07:31:01.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.120+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/890164230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb802990 msgr2=0x55f5bb80ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.120+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb802990 0x55f5bb80ca80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f5ca009bb50 tx=0x7f5ca009b5e0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.120+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/890164230 shutdown_connections 2024-04-17T07:31:01.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.120+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f5bb817ed0 0x55f5bb81a390 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.120+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.120+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb802990 0x55f5bb80ca80 secure :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f5ca009bb50 tx=0x7f5ca009b5e0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.120+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/890164230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb801fa0 0x55f5bb80c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.120+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/890164230 >> 172.21.15.86:0/890164230 conn(0x55f5bb7fd510 msgr2=0x55f5bb7fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.120+0000 7f5c93e00640 0 client.19269 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:31:01.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.120+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/890164230 shutdown_connections 2024-04-17T07:31:01.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.121+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/890164230 wait complete. 2024-04-17T07:31:01.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.121+0000 7f5cbc1b8740 20 client.19269 trim_cache size 0 max 16384 2024-04-17T07:31:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.169+0000 7f5cbc1b8740 1 Processor -- start 2024-04-17T07:31:01.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.170+0000 7f5cbc1b8740 1 -- start start 2024-04-17T07:31:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.170+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb7fbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.170+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.171+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb8045a0 0x55f5bb7fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.171+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb41b9950 con 0x55f5bb801fa0 2024-04-17T07:31:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.171+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb41bb490 con 0x55f5bb807b00 2024-04-17T07:31:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.171+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb41ba020 con 0x55f5bb8045a0 2024-04-17T07:31:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.171+0000 7f5cb1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.171+0000 7f5cb1e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58276/0 (socket says 172.21.15.86:58276) 2024-04-17T07:31:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.171+0000 7f5cb1e00640 1 -- 172.21.15.86:0/557466231 learned_addr learned my addr 172.21.15.86:0/557466231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.171+0000 7f5cb1e00640 1 -- 172.21.15.86:0/557466231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb8045a0 msgr2=0x55f5bb7fc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.171+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/557466231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb8045a0 0x55f5bb7fc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.171+0000 7f5cb1e00640 1 -- 172.21.15.86:0/557466231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 msgr2=0x55f5bb7fbe80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:31:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.172+0000 7f5cb2800640 1 --2- 172.21.15.86:0/557466231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb7fbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.172+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/557466231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb7fbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.172+0000 7f5cb1e00640 1 -- 172.21.15.86:0/557466231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5bb47a3f0 con 0x55f5bb807b00 2024-04-17T07:31:01.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.172+0000 7f5cb2800640 1 --2- 172.21.15.86:0/557466231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb7fbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:01.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.173+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/557466231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813a80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0010850 tx=0x7f5ca009a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c143914770e5be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.173+0000 7f5c9ea00640 1 -- 172.21.15.86:0/557466231 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0020070 con 0x55f5bb807b00 2024-04-17T07:31:01.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.174+0000 7f5c9ea00640 1 -- 172.21.15.86:0/557466231 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ca00a4830 con 0x55f5bb807b00 2024-04-17T07:31:01.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.174+0000 7f5c9ea00640 1 -- 172.21.15.86:0/557466231 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0097910 con 0x55f5bb807b00 2024-04-17T07:31:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.175+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/557466231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 msgr2=0x55f5bb813a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.175+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/557466231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813a80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0010850 tx=0x7f5ca009a0d0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.175+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/557466231 shutdown_connections 2024-04-17T07:31:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.175+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/557466231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb8045a0 0x55f5bb7fc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.175+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/557466231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb807b00 0x55f5bb813a80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.175+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/557466231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb7fbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.175+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/557466231 >> 172.21.15.86:0/557466231 conn(0x55f5bb7fd510 msgr2=0x55f5bb803e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.175+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/557466231 shutdown_connections 2024-04-17T07:31:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.175+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/557466231 wait complete. 2024-04-17T07:31:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.176+0000 7f5cbc1b8740 1 -- start start 2024-04-17T07:31:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.177+0000 7f5cbc1b8740 1 Processor -- start 2024-04-17T07:31:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.178+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb8115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.178+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb8045a0 0x55f5bb811b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.178+0000 7f5cbc1b8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb807b00 0x55f5bb812b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.178+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb41badc0 con 0x55f5bb801fa0 2024-04-17T07:31:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.178+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb41ba6f0 con 0x55f5bb8045a0 2024-04-17T07:31:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.178+0000 7f5cbc1b8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb41b7920 con 0x55f5bb807b00 2024-04-17T07:31:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cbc1b8740 10 client.0 _ll_register_callbacks cb 0x7f5ca009b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cb2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb8115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cb2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb8115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47118/0 (socket says 172.21.15.86:47118) 2024-04-17T07:31:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cb2800640 1 -- 172.21.15.86:0/379262759 learned_addr learned my addr 172.21.15.86:0/379262759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb8045a0 0x55f5bb811b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cb3200640 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb807b00 0x55f5bb812b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cb2800640 1 -- 172.21.15.86:0/379262759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb807b00 msgr2=0x55f5bb812b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cb2800640 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb807b00 0x55f5bb812b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cb2800640 1 -- 172.21.15.86:0/379262759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb8045a0 msgr2=0x55f5bb811b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cb2800640 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb8045a0 0x55f5bb811b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.179+0000 7f5cb2800640 1 -- 172.21.15.86:0/379262759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5bb5447d0 con 0x55f5bb801fa0 2024-04-17T07:31:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.180+0000 7f5cb2800640 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb8115c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b740 tx=0x7f5ca4005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d840de63b4a6f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.180+0000 7f5c93e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.181+0000 7f5c93e00640 1 -- 172.21.15.86:0/379262759 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca402e060 con 0x55f5bb801fa0 2024-04-17T07:31:01.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.182+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/379262759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ca00a4830 con 0x55f5bb801fa0 2024-04-17T07:31:01.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.182+0000 7f5c93e00640 1 -- 172.21.15.86:0/379262759 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5ca4005040 con 0x55f5bb801fa0 2024-04-17T07:31:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.184+0000 7f5c93e00640 1 -- 172.21.15.86:0/379262759 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5ca4035050 con 0x55f5bb801fa0 2024-04-17T07:31:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.184+0000 7f5cbc1b8740 10 client.19272 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.184+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/379262759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5bb87e670 con 0x55f5bb801fa0 2024-04-17T07:31:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.187+0000 7f5c93e00640 1 -- 172.21.15.86:0/379262759 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ca00a4830 con 0x55f5bb801fa0 2024-04-17T07:31:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.187+0000 7f5c93e00640 1 -- 172.21.15.86:0/379262759 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5ca4004600 con 0x55f5bb801fa0 2024-04-17T07:31:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.188+0000 7f5cbc1b8740 10 client.19272 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.188+0000 7f5cbc1b8740 10 client.19272 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:31:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.188+0000 7f5cbc1b8740 20 client.19272 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.188+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f5bb87ed40 0x55f5bb881130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.188+0000 7f5cbc1b8740 4 client.19272 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:31:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.188+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/379262759 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5bb76f8b0 con 0x55f5bb87ed40 2024-04-17T07:31:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.189+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f5bb87ed40 0x55f5bb881130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.194+0000 7f5cb1e00640 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f5bb87ed40 0x55f5bb881130 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=110c67d4c6b855b6 server_cookie=6d93934fb10bfc1f in_seq=0 out_seq=0 2024-04-17T07:31:01.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.194+0000 7f5c93e00640 10 client.19272 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:31:01.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.194+0000 7f5c93e00640 1 -- 172.21.15.86:0/379262759 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5bb76f8b0 con 0x55f5bb87ed40 2024-04-17T07:31:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.194+0000 7f5c93e00640 10 client.19272 handle_command_reply: tid=0 2024-04-17T07:31:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.195+0000 7f5cbc1b8740 1 client.19272 shutdown 2024-04-17T07:31:01.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.196+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/379262759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 msgr2=0x55f5bb8115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.196+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb8115c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b740 tx=0x7f5ca4005d10 comp rx=0 tx=0).stop 2024-04-17T07:31:01.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.196+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/379262759 shutdown_connections 2024-04-17T07:31:01.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.196+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55f5bb87ed40 0x55f5bb881130 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.196+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f5bb807b00 0x55f5bb812b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.196+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5bb8045a0 0x55f5bb811b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.196+0000 7f5cbc1b8740 1 --2- 172.21.15.86:0/379262759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5bb801fa0 0x55f5bb8115c0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.196+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/379262759 >> 172.21.15.86:0/379262759 conn(0x55f5bb7fd510 msgr2=0x55f5bb800770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.196+0000 7f5c93e00640 0 client.19272 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:31:01.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.197+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/379262759 shutdown_connections 2024-04-17T07:31:01.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.197+0000 7f5cbc1b8740 1 -- 172.21.15.86:0/379262759 wait complete. 2024-04-17T07:31:01.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.197+0000 7f5cbc1b8740 20 client.19272 trim_cache size 0 max 16384 2024-04-17T07:31:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:31:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.199+0000 7f5cb9400640 1 -- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5c980a3cb0 msgr2=0x7f5c980a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.199+0000 7f5cb9400640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5c980a3cb0 0x7f5c980a6170 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0099a0 tx=0x7f5cac005fd0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.199+0000 7f5cb9400640 1 -- 172.21.15.86:0/4072185570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb410b8d0 msgr2=0x7f5cb41a74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.199+0000 7f5cb9400640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb410b8d0 0x7f5cb41a74e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000d950 tx=0x7f5ca000de10 comp rx=0 tx=0).stop 2024-04-17T07:31:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.200+0000 7f5cb9400640 1 -- 172.21.15.86:0/4072185570 shutdown_connections 2024-04-17T07:31:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.200+0000 7f5cb9400640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5c980a3cb0 0x7f5c980a6170 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.200+0000 7f5cb9400640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cb410e280 0x7f5cb41ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.200+0000 7f5cb9400640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cb410b8d0 0x7f5cb41a74e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.200+0000 7f5cb9400640 1 --2- 172.21.15.86:0/4072185570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5cb4108f60 0x7f5cb41a6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.200+0000 7f5cb9400640 1 -- 172.21.15.86:0/4072185570 >> 172.21.15.86:0/4072185570 conn(0x7f5cb406f690 msgr2=0x7f5cb410cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.201+0000 7f5cb9400640 1 -- 172.21.15.86:0/4072185570 shutdown_connections 2024-04-17T07:31:01.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.201+0000 7f5cb9400640 1 -- 172.21.15.86:0/4072185570 wait complete. 2024-04-17T07:31:01.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:31:01.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:01 smithi086 ceph-mon[31733]: pgmap v939: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 658 op/s 2024-04-17T07:31:01.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:01 smithi098 ceph-mon[35418]: pgmap v939: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 658 op/s 2024-04-17T07:31:01.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:01 smithi103 ceph-mon[35495]: pgmap v939: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 658 op/s 2024-04-17T07:31:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.594+0000 7f0690c00640 1 -- 172.21.15.86:0/2562064609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f068c105c50 msgr2=0x7f068c10c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.594+0000 7f0690c00640 1 --2- 172.21.15.86:0/2562064609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f068c105c50 0x7f068c10c4e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0684009ca0 tx=0x7f068402c100 comp rx=0 tx=0).stop 2024-04-17T07:31:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.595+0000 7f0690c00640 1 -- 172.21.15.86:0/2562064609 shutdown_connections 2024-04-17T07:31:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.595+0000 7f0690c00640 1 --2- 172.21.15.86:0/2562064609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f068c105c50 0x7f068c10c4e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.595+0000 7f0690c00640 1 --2- 172.21.15.86:0/2562064609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f068c105290 0x7f068c105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.595+0000 7f0690c00640 1 --2- 172.21.15.86:0/2562064609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f068c1040a0 0x7f068c1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.595+0000 7f0690c00640 1 -- 172.21.15.86:0/2562064609 >> 172.21.15.86:0/2562064609 conn(0x7f068c0ff610 msgr2=0x7f068c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.595+0000 7f0690c00640 1 -- 172.21.15.86:0/2562064609 shutdown_connections 2024-04-17T07:31:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.595+0000 7f0690c00640 1 -- 172.21.15.86:0/2562064609 wait complete. 2024-04-17T07:31:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.597+0000 7f0690c00640 1 Processor -- start 2024-04-17T07:31:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.598+0000 7f0690c00640 1 -- start start 2024-04-17T07:31:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.598+0000 7f0690c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f068c1040a0 0x7f068c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.598+0000 7f0690c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f068c105290 0x7f068c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.598+0000 7f0690c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f068c105c50 0x7f068c1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.598+0000 7f0690c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10fbe0 con 0x7f068c1040a0 2024-04-17T07:31:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.598+0000 7f0690c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10fa60 con 0x7f068c105c50 2024-04-17T07:31:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.598+0000 7f0690c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f068c10fd60 con 0x7f068c105290 2024-04-17T07:31:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.599+0000 7f068be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f068c105c50 0x7f068c1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.599+0000 7f068be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f068c105c50 0x7f068c1a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58288/0 (socket says 172.21.15.86:58288) 2024-04-17T07:31:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.599+0000 7f068be00640 1 -- 172.21.15.86:0/72707351 learned_addr learned my addr 172.21.15.86:0/72707351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.599+0000 7f068b400640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f068c1040a0 0x7f068c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.599+0000 7f068aa00640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f068c105290 0x7f068c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.599+0000 7f068b400640 1 -- 172.21.15.86:0/72707351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f068c105290 msgr2=0x7f068c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.599+0000 7f068b400640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f068c105290 0x7f068c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.599+0000 7f068b400640 1 -- 172.21.15.86:0/72707351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f068c105c50 msgr2=0x7f068c1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.599+0000 7f068b400640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f068c105c50 0x7f068c1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.599+0000 7f068b400640 1 -- 172.21.15.86:0/72707351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c1a1e90 con 0x7f068c1040a0 2024-04-17T07:31:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.600+0000 7f068b400640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f068c1040a0 0x7f068c19a270 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f067c00d6a0 tx=0x7f067c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.601+0000 7f0683e00640 1 -- 172.21.15.86:0/72707351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c017070 con 0x7f068c1040a0 2024-04-17T07:31:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.601+0000 7f0683e00640 1 -- 172.21.15.86:0/72707351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f067c009cd0 con 0x7f068c1040a0 2024-04-17T07:31:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.601+0000 7f0683e00640 1 -- 172.21.15.86:0/72707351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c003b80 con 0x7f068c1040a0 2024-04-17T07:31:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.601+0000 7f0690c00640 1 -- 172.21.15.86:0/72707351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068c1a2180 con 0x7f068c1040a0 2024-04-17T07:31:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.601+0000 7f0690c00640 1 -- 172.21.15.86:0/72707351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068c1a2680 con 0x7f068c1040a0 2024-04-17T07:31:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.606+0000 7f0694fbb740 1 Processor -- start 2024-04-17T07:31:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.606+0000 7f0694fbb740 1 -- start start 2024-04-17T07:31:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.606+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.606+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8b60 0x5557997bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.606+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.606+0000 7f0694fbb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55579970d020 con 0x5557997b8170 2024-04-17T07:31:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.606+0000 7f0694fbb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5557996b3460 con 0x5557997b8b60 2024-04-17T07:31:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.606+0000 7f0694fbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555799708fe0 con 0x5557997bdcd0 2024-04-17T07:31:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.607+0000 7f068aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.607+0000 7f068aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54650/0 (socket says 172.21.15.86:54650) 2024-04-17T07:31:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.607+0000 7f068aa00640 1 -- 172.21.15.86:0/2968792428 learned_addr learned my addr 172.21.15.86:0/2968792428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.608+0000 7f068aa00640 1 -- 172.21.15.86:0/2968792428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8b60 msgr2=0x5557997bd660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:31:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.608+0000 7f068aa00640 1 --2- 172.21.15.86:0/2968792428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8b60 0x5557997bd660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.608+0000 7f068aa00640 1 -- 172.21.15.86:0/2968792428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 msgr2=0x5557997b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.608+0000 7f068aa00640 1 --2- 172.21.15.86:0/2968792428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.608+0000 7f068aa00640 1 -- 172.21.15.86:0/2968792428 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557996bb040 con 0x5557997bdcd0 2024-04-17T07:31:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.608+0000 7f0683e00640 1 -- 172.21.15.86:0/72707351 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f067c003d20 con 0x7f068c1040a0 2024-04-17T07:31:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.608+0000 7f068aa00640 1 --2- 172.21.15.86:0/2968792428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c0160 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0678009c90 tx=0x7f0678011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a554cda62fc21ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.609+0000 7f0677e00640 1 -- 172.21.15.86:0/2968792428 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067801e070 con 0x5557997bdcd0 2024-04-17T07:31:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.609+0000 7f0677e00640 1 -- 172.21.15.86:0/2968792428 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0678019ae0 con 0x5557997bdcd0 2024-04-17T07:31:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.609+0000 7f0677e00640 1 -- 172.21.15.86:0/2968792428 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067800e920 con 0x5557997bdcd0 2024-04-17T07:31:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.610+0000 7f0683e00640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06700a3d00 0x7f06700a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.610+0000 7f0694fbb740 1 -- 172.21.15.86:0/2968792428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 msgr2=0x5557997c0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.610+0000 7f0694fbb740 1 --2- 172.21.15.86:0/2968792428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c0160 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0678009c90 tx=0x7f0678011810 comp rx=0 tx=0).stop 2024-04-17T07:31:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.611+0000 7f0683e00640 1 -- 172.21.15.86:0/72707351 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f067c0cffe0 con 0x7f068c1040a0 2024-04-17T07:31:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.611+0000 7f068be00640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06700a3d00 0x7f06700a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.611+0000 7f0694fbb740 1 -- 172.21.15.86:0/2968792428 shutdown_connections 2024-04-17T07:31:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.611+0000 7f0694fbb740 1 --2- 172.21.15.86:0/2968792428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c0160 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.611+0000 7f0694fbb740 1 --2- 172.21.15.86:0/2968792428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8b60 0x5557997bd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.611+0000 7f0694fbb740 1 --2- 172.21.15.86:0/2968792428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.611+0000 7f0694fbb740 1 -- 172.21.15.86:0/2968792428 >> 172.21.15.86:0/2968792428 conn(0x5557997a26d0 msgr2=0x5557997a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.613+0000 7f0694fbb740 1 -- 172.21.15.86:0/2968792428 shutdown_connections 2024-04-17T07:31:01.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.616+0000 7f068be00640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06700a3d00 0x7f06700a61c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x5557997b93c0 tx=0x7f0684004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.616+0000 7f0694fbb740 1 -- 172.21.15.86:0/2968792428 wait complete. 2024-04-17T07:31:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.617+0000 7f0694fbb740 1 -- start start 2024-04-17T07:31:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.617+0000 7f0694fbb740 1 Processor -- start 2024-04-17T07:31:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.619+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8170 0x5557997c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.619+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8b60 0x5557997c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.619+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.619+0000 7f0694fbb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557997c0c80 con 0x5557997b8b60 2024-04-17T07:31:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.619+0000 7f0694fbb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5557996c6830 con 0x5557997b8170 2024-04-17T07:31:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.619+0000 7f0694fbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5557997c0e00 con 0x5557997bdcd0 2024-04-17T07:31:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.619+0000 7f0694fbb740 10 client.0 _ll_register_callbacks cb 0x555799694fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.619+0000 7f068b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8170 0x5557997c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.619+0000 7f068b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8170 0x5557997c2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58290/0 (socket says 172.21.15.86:58290) 2024-04-17T07:31:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.619+0000 7f068b400640 1 -- 172.21.15.86:0/3768909291 learned_addr learned my addr 172.21.15.86:0/3768909291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.620+0000 7f068aa00640 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8b60 0x5557997c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.620+0000 7f068b400640 1 -- 172.21.15.86:0/3768909291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 msgr2=0x5557997c9cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:31:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.620+0000 7f068b400640 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c9cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.620+0000 7f068b400640 1 -- 172.21.15.86:0/3768909291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8b60 msgr2=0x5557997c2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.621+0000 7f068b400640 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8b60 0x5557997c2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.621+0000 7f068b400640 1 -- 172.21.15.86:0/3768909291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557996ca920 con 0x5557997b8170 2024-04-17T07:31:01.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.621+0000 7f068aa00640 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8b60 0x5557997c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:01.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.622+0000 7f068b400640 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8170 0x5557997c2730 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f067c0a4620 tx=0x7f067c09c5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ddd09ad3ddb0220 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.622+0000 7f0674c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:31:01.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.622+0000 7f0674c00640 1 -- 172.21.15.86:0/3768909291 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0b5060 con 0x5557997b8170 2024-04-17T07:31:01.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.622+0000 7f0674c00640 1 -- 172.21.15.86:0/3768909291 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f067c098550 con 0x5557997b8170 2024-04-17T07:31:01.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.622+0000 7f0694fbb740 1 -- 172.21.15.86:0/3768909291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0678019ae0 con 0x5557997b8170 2024-04-17T07:31:01.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.623+0000 7f0674c00640 1 -- 172.21.15.86:0/3768909291 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f067c0a0290 con 0x5557997b8170 2024-04-17T07:31:01.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.623+0000 7f0694fbb740 10 client.38261 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:01.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.623+0000 7f0694fbb740 1 -- 172.21.15.86:0/3768909291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557996987f0 con 0x5557997b8170 2024-04-17T07:31:01.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.623+0000 7f0674c00640 1 -- 172.21.15.86:0/3768909291 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0678019ae0 con 0x5557997b8170 2024-04-17T07:31:01.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.624+0000 7f0674c00640 1 -- 172.21.15.86:0/3768909291 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f067c0097c0 con 0x5557997b8170 2024-04-17T07:31:01.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.624+0000 7f0694fbb740 10 client.38261 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:01.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.624+0000 7f0694fbb740 10 client.38261 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:31:01.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.624+0000 7f0694fbb740 20 client.38261 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:01.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.624+0000 7f0694fbb740 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5557997d6e40 0x5557997d92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.624+0000 7f0694fbb740 4 client.38261 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:01.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.624+0000 7f0694fbb740 1 -- 172.21.15.86:0/3768909291 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55579971aa40 con 0x5557997d6e40 2024-04-17T07:31:01.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.625+0000 7f068aa00640 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5557997d6e40 0x5557997d92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.626+0000 7f068aa00640 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5557997d6e40 0x5557997d92c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71b43b119854c46f server_cookie=991c7ab0d50672c3 in_seq=0 out_seq=0 2024-04-17T07:31:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.626+0000 7f0674c00640 10 client.38261 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:31:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.628+0000 7f0674c00640 1 -- 172.21.15.86:0/3768909291 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55579971aa40 con 0x5557997d6e40 2024-04-17T07:31:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.628+0000 7f0674c00640 10 client.38261 handle_command_reply: tid=0 2024-04-17T07:31:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.628+0000 7f0694fbb740 1 client.38261 shutdown 2024-04-17T07:31:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.629+0000 7f0694fbb740 1 -- 172.21.15.86:0/3768909291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8170 msgr2=0x5557997c2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.630+0000 7f0694fbb740 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8170 0x5557997c2730 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f067c0a4620 tx=0x7f067c09c5e0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.630+0000 7f0694fbb740 1 -- 172.21.15.86:0/3768909291 shutdown_connections 2024-04-17T07:31:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.630+0000 7f0694fbb740 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.630+0000 7f0694fbb740 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8b60 0x5557997c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.630+0000 7f0694fbb740 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5557997d6e40 0x5557997d92c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.630+0000 7f0694fbb740 1 --2- 172.21.15.86:0/3768909291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8170 0x5557997c2730 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.630+0000 7f0694fbb740 1 -- 172.21.15.86:0/3768909291 >> 172.21.15.86:0/3768909291 conn(0x5557997a26d0 msgr2=0x5557997a1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.631+0000 7f0674c00640 0 client.38261 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:31:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.631+0000 7f0694fbb740 1 -- 172.21.15.86:0/3768909291 shutdown_connections 2024-04-17T07:31:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.632+0000 7f0694fbb740 1 -- 172.21.15.86:0/3768909291 wait complete. 2024-04-17T07:31:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.632+0000 7f0694fbb740 20 client.38261 trim_cache size 0 max 16384 2024-04-17T07:31:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.667+0000 7f0694fbb740 1 Processor -- start 2024-04-17T07:31:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.667+0000 7f0694fbb740 1 -- start start 2024-04-17T07:31:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.667+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.667+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997b8b60 0x5557997c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.667+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997bdcd0 0x5557997a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.667+0000 7f0694fbb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f068c0716f0 con 0x5557997b8170 2024-04-17T07:31:01.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.667+0000 7f0694fbb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f068c073230 con 0x5557997bdcd0 2024-04-17T07:31:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.667+0000 7f0694fbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f068c071dc0 con 0x5557997b8b60 2024-04-17T07:31:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.668+0000 7f068be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997bdcd0 0x5557997a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.668+0000 7f068be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997bdcd0 0x5557997a6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58300/0 (socket says 172.21.15.86:58300) 2024-04-17T07:31:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.668+0000 7f068be00640 1 -- 172.21.15.86:0/273778308 learned_addr learned my addr 172.21.15.86:0/273778308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.668+0000 7f068aa00640 1 --2- 172.21.15.86:0/273778308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997b8b60 0x5557997c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.669+0000 7f068b400640 1 --2- 172.21.15.86:0/273778308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.670+0000 7f068be00640 1 -- 172.21.15.86:0/273778308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997b8b60 msgr2=0x5557997c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.670+0000 7f068be00640 1 --2- 172.21.15.86:0/273778308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997b8b60 0x5557997c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.670+0000 7f068be00640 1 -- 172.21.15.86:0/273778308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 msgr2=0x5557997c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.670+0000 7f068be00640 1 --2- 172.21.15.86:0/273778308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.670+0000 7f068be00640 1 -- 172.21.15.86:0/273778308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55579941f3f0 con 0x5557997bdcd0 2024-04-17T07:31:01.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.671+0000 7f068be00640 1 --2- 172.21.15.86:0/273778308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997bdcd0 0x5557997a6040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0684032c70 tx=0x7f068402e050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5870d5534dfb4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.673+0000 7f0677400640 1 -- 172.21.15.86:0/273778308 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684044070 con 0x5557997bdcd0 2024-04-17T07:31:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.674+0000 7f0677400640 1 -- 172.21.15.86:0/273778308 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0684050070 con 0x5557997bdcd0 2024-04-17T07:31:01.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.674+0000 7f0677400640 1 -- 172.21.15.86:0/273778308 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684039040 con 0x5557997bdcd0 2024-04-17T07:31:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.675+0000 7f0694fbb740 1 -- 172.21.15.86:0/273778308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997bdcd0 msgr2=0x5557997a6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.675+0000 7f0694fbb740 1 --2- 172.21.15.86:0/273778308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997bdcd0 0x5557997a6040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0684032c70 tx=0x7f068402e050 comp rx=0 tx=0).stop 2024-04-17T07:31:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.677+0000 7f0694fbb740 1 -- 172.21.15.86:0/273778308 shutdown_connections 2024-04-17T07:31:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.677+0000 7f0694fbb740 1 --2- 172.21.15.86:0/273778308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997bdcd0 0x5557997a6040 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.677+0000 7f0694fbb740 1 --2- 172.21.15.86:0/273778308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997b8b60 0x5557997c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.677+0000 7f0694fbb740 1 --2- 172.21.15.86:0/273778308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.677+0000 7f0694fbb740 1 -- 172.21.15.86:0/273778308 >> 172.21.15.86:0/273778308 conn(0x5557997a26d0 msgr2=0x5557997a5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.678+0000 7f0694fbb740 1 -- 172.21.15.86:0/273778308 shutdown_connections 2024-04-17T07:31:01.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.678+0000 7f0694fbb740 1 -- 172.21.15.86:0/273778308 wait complete. 2024-04-17T07:31:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.679+0000 7f0694fbb740 1 -- start start 2024-04-17T07:31:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.679+0000 7f0694fbb740 1 Processor -- start 2024-04-17T07:31:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.680+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.680+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8b60 0x5557997c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.680+0000 7f0694fbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.681+0000 7f068aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8b60 0x5557997c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.681+0000 7f068b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.681+0000 7f068b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997c70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47164/0 (socket says 172.21.15.86:47164) 2024-04-17T07:31:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.681+0000 7f068b400640 1 -- 172.21.15.86:0/1773613505 learned_addr learned my addr 172.21.15.86:0/1773613505 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.681+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f068c072b60 con 0x5557997b8170 2024-04-17T07:31:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.681+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f068c072490 con 0x5557997b8b60 2024-04-17T07:31:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.681+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f068c06f6c0 con 0x5557997bdcd0 2024-04-17T07:31:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.681+0000 7f0694fbb740 10 client.0 _ll_register_callbacks cb 0x7f067c09ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.682+0000 7f068be00640 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.682+0000 7f068b400640 1 -- 172.21.15.86:0/1773613505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 msgr2=0x5557997c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.682+0000 7f068b400640 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.682+0000 7f068b400640 1 -- 172.21.15.86:0/1773613505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8b60 msgr2=0x5557997c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.683+0000 7f068b400640 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8b60 0x5557997c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.683+0000 7f068b400640 1 -- 172.21.15.86:0/1773613505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557994e97d0 con 0x5557997b8170 2024-04-17T07:31:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.683+0000 7f068be00640 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.683+0000 7f068aa00640 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8b60 0x5557997c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.684+0000 7f068b400640 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997c70b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f067c0a0450 tx=0x7f067c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a71a9238e8bae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.684+0000 7f0674c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.685+0000 7f0674c00640 1 -- 172.21.15.86:0/1773613505 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0b5040 con 0x5557997b8170 2024-04-17T07:31:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.685+0000 7f0674c00640 1 -- 172.21.15.86:0/1773613505 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f067c0b51e0 con 0x5557997b8170 2024-04-17T07:31:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.685+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0684050070 con 0x5557997b8170 2024-04-17T07:31:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.687+0000 7f0674c00640 1 -- 172.21.15.86:0/1773613505 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f067c0a4b50 con 0x5557997b8170 2024-04-17T07:31:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.688+0000 7f0694fbb740 10 client.19284 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.688+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557997c9d20 con 0x5557997b8170 2024-04-17T07:31:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.688+0000 7f0674c00640 1 -- 172.21.15.86:0/1773613505 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0684050070 con 0x5557997b8170 2024-04-17T07:31:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.688+0000 7f0674c00640 1 -- 172.21.15.86:0/1773613505 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f067c0cdcd0 con 0x5557997b8170 2024-04-17T07:31:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.689+0000 7f0694fbb740 10 client.19284 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.689+0000 7f0694fbb740 10 client.19284 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:31:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.689+0000 7f0694fbb740 20 client.19284 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.689+0000 7f0694fbb740 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5557997d0840 0x5557997d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.689+0000 7f0694fbb740 4 client.19284 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:31:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.689+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557997148b0 con 0x5557997d0840 2024-04-17T07:31:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.689+0000 7f068aa00640 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5557997d0840 0x5557997d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.693+0000 7f068aa00640 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5557997d0840 0x5557997d0c20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4aadd9788c8ad988 server_cookie=64dda81a951b60d in_seq=0 out_seq=0 2024-04-17T07:31:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.693+0000 7f0674c00640 10 client.19284 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:31:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.693+0000 7f0674c00640 1 -- 172.21.15.86:0/1773613505 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557997148b0 con 0x5557997d0840 2024-04-17T07:31:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.693+0000 7f0674c00640 10 client.19284 handle_command_reply: tid=0 2024-04-17T07:31:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.693+0000 7f0694fbb740 1 client.19284 shutdown 2024-04-17T07:31:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.695+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 msgr2=0x5557997c70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.695+0000 7f0694fbb740 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997c70b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f067c0a0450 tx=0x7f067c0044c0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.696+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 shutdown_connections 2024-04-17T07:31:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.696+0000 7f0694fbb740 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557997bdcd0 0x5557997c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.696+0000 7f0694fbb740 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5557997d0840 0x5557997d0c20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.696+0000 7f0694fbb740 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557997b8b60 0x5557997c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.696+0000 7f0694fbb740 1 --2- 172.21.15.86:0/1773613505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557997b8170 0x5557997c70b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.696+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 >> 172.21.15.86:0/1773613505 conn(0x5557997a26d0 msgr2=0x5557997c2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.696+0000 7f0674c00640 0 client.19284 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:31:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.696+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 shutdown_connections 2024-04-17T07:31:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.697+0000 7f0694fbb740 1 -- 172.21.15.86:0/1773613505 wait complete. 2024-04-17T07:31:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.697+0000 7f0694fbb740 20 client.19284 trim_cache size 0 max 16384 2024-04-17T07:31:01.700 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:31:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.700+0000 7f0690c00640 1 -- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06700a3d00 msgr2=0x7f06700a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.700+0000 7f0690c00640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06700a3d00 0x7f06700a61c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x5557997b93c0 tx=0x7f0684004ab0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.700+0000 7f0690c00640 1 -- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f068c1040a0 msgr2=0x7f068c19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.700+0000 7f0690c00640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f068c1040a0 0x7f068c19a270 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f067c00d6a0 tx=0x7f067c00db60 comp rx=0 tx=0).stop 2024-04-17T07:31:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.705+0000 7f0690c00640 1 -- 172.21.15.86:0/72707351 shutdown_connections 2024-04-17T07:31:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.705+0000 7f0690c00640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f06700a3d00 0x7f06700a61c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.705+0000 7f0690c00640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f068c105c50 0x7f068c1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.705+0000 7f0690c00640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f068c105290 0x7f068c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.705+0000 7f0690c00640 1 --2- 172.21.15.86:0/72707351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f068c1040a0 0x7f068c19a270 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.705+0000 7f0690c00640 1 -- 172.21.15.86:0/72707351 >> 172.21.15.86:0/72707351 conn(0x7f068c0ff610 msgr2=0x7f068c108610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.705+0000 7f0690c00640 1 -- 172.21.15.86:0/72707351 shutdown_connections 2024-04-17T07:31:01.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:01.705+0000 7f0690c00640 1 -- 172.21.15.86:0/72707351 wait complete. 2024-04-17T07:31:01.747 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:31:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.124+0000 7f7fbbe00640 1 -- 172.21.15.86:0/477209302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb40a6b90 msgr2=0x7f7fb40ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.124+0000 7f7fbbe00640 1 --2- 172.21.15.86:0/477209302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb40a6b90 0x7f7fb40ad420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0074e0 tx=0x7f7fac02c8b0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.124+0000 7f7fbbe00640 1 -- 172.21.15.86:0/477209302 shutdown_connections 2024-04-17T07:31:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.124+0000 7f7fbbe00640 1 --2- 172.21.15.86:0/477209302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb40a6b90 0x7f7fb40ad420 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.124+0000 7f7fbbe00640 1 --2- 172.21.15.86:0/477209302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb40a61d0 0x7f7fb40a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.124+0000 7f7fbbe00640 1 --2- 172.21.15.86:0/477209302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fb40a4fe0 0x7f7fb40a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.125+0000 7f7fbbe00640 1 -- 172.21.15.86:0/477209302 >> 172.21.15.86:0/477209302 conn(0x7f7fb40a0530 msgr2=0x7f7fb40a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.125+0000 7f7fbbe00640 1 -- 172.21.15.86:0/477209302 shutdown_connections 2024-04-17T07:31:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.128+0000 7f7fbbe00640 1 -- 172.21.15.86:0/477209302 wait complete. 2024-04-17T07:31:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.129+0000 7f7fbbe00640 1 Processor -- start 2024-04-17T07:31:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.130+0000 7f7fbbe00640 1 -- start start 2024-04-17T07:31:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.132+0000 7f7fbbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fb40a4fe0 0x7f7fb413ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.132+0000 7f7fbbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb40a61d0 0x7f7fb413b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.132+0000 7f7fbbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb40a6b90 0x7f7fb4153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.132+0000 7f7fbbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb40b2830 con 0x7f7fb40a6b90 2024-04-17T07:31:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.133+0000 7f7fbaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fb40a4fe0 0x7f7fb413ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.133+0000 7f7fbaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fb40a4fe0 0x7f7fb413ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54690/0 (socket says 172.21.15.86:54690) 2024-04-17T07:31:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.133+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3370062684 learned_addr learned my addr 172.21.15.86:0/3370062684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.133+0000 7f7fbbe00640 1 -- 172.21.15.86:0/3370062684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb40b26b0 con 0x7f7fb40a61d0 2024-04-17T07:31:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.133+0000 7f7fbbe00640 1 -- 172.21.15.86:0/3370062684 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb40b29b0 con 0x7f7fb40a4fe0 2024-04-17T07:31:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.133+0000 7f7fba000640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb40a61d0 0x7f7fb413b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.134+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3370062684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb40a61d0 msgr2=0x7f7fb413b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.134+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb40a61d0 0x7f7fb413b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.134+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3370062684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb40a6b90 msgr2=0x7f7fb4153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.134+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb40a6b90 0x7f7fb4153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.134+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3370062684 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb41403d0 con 0x7f7fb40a4fe0 2024-04-17T07:31:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.135+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fb40a4fe0 0x7f7fb413ad00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc067750 tx=0x7f7fbc072a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.135+0000 7f7fb3400640 1 -- 172.21.15.86:0/3370062684 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc079070 con 0x7f7fb40a4fe0 2024-04-17T07:31:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.135+0000 7f7fbbe00640 1 -- 172.21.15.86:0/3370062684 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb4153c60 con 0x7f7fb40a4fe0 2024-04-17T07:31:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.136+0000 7f7fbbe00640 1 -- 172.21.15.86:0/3370062684 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb4154130 con 0x7f7fb40a4fe0 2024-04-17T07:31:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.136+0000 7f7fb3400640 1 -- 172.21.15.86:0/3370062684 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fbc074b30 con 0x7f7fb40a4fe0 2024-04-17T07:31:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.136+0000 7f7fb3400640 1 -- 172.21.15.86:0/3370062684 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc075a00 con 0x7f7fb40a4fe0 2024-04-17T07:31:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.143+0000 7f7fb3400640 1 -- 172.21.15.86:0/3370062684 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f7fbc075ba0 con 0x7f7fb40a4fe0 2024-04-17T07:31:02.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.145+0000 7f7fc4428740 1 Processor -- start 2024-04-17T07:31:02.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.146+0000 7f7fb3400640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7fa00a3d00 0x7f7fa00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.146+0000 7f7fb3400640 1 -- 172.21.15.86:0/3370062684 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7fbc1341e0 con 0x7f7fb40a4fe0 2024-04-17T07:31:02.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.147+0000 7f7fc4428740 1 -- start start 2024-04-17T07:31:02.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.147+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd658580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.147+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd658b50 0x556fbd65d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.147+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd65dcc0 0x556fbd660150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.147+0000 7f7fc4428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556fbd5be020 con 0x556fbd658160 2024-04-17T07:31:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.147+0000 7f7fc4428740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556fbd564460 con 0x556fbd658b50 2024-04-17T07:31:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.147+0000 7f7fc4428740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556fbd5b9fe0 con 0x556fbd65dcc0 2024-04-17T07:31:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.148+0000 7f7fba000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd658b50 0x556fbd65d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.149+0000 7f7fbaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd658580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.149+0000 7f7fbaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd658580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47194/0 (socket says 172.21.15.86:47194) 2024-04-17T07:31:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.149+0000 7f7fbaa00640 1 -- 172.21.15.86:0/1673576764 learned_addr learned my addr 172.21.15.86:0/1673576764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.149+0000 7f7fbb400640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7fa00a3d00 0x7f7fa00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.150+0000 7f7fbaa00640 1 -- 172.21.15.86:0/1673576764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd65dcc0 msgr2=0x556fbd660150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:31:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.150+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/1673576764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd65dcc0 0x556fbd660150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.150+0000 7f7fbaa00640 1 -- 172.21.15.86:0/1673576764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd658b50 msgr2=0x556fbd65d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.150+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/1673576764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd658b50 0x556fbd65d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.150+0000 7f7fbaa00640 1 -- 172.21.15.86:0/1673576764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fbd56c040 con 0x556fbd658160 2024-04-17T07:31:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.151+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/1673576764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd658580 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc065480 tx=0x7f7fbc067d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cb7c89978dab76 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:02.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.152+0000 7f7fab400640 1 -- 172.21.15.86:0/1673576764 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc1007e0 con 0x556fbd658160 2024-04-17T07:31:02.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.152+0000 7f7fab400640 1 -- 172.21.15.86:0/1673576764 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fbc1000e0 con 0x556fbd658160 2024-04-17T07:31:02.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.152+0000 7f7fab400640 1 -- 172.21.15.86:0/1673576764 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0fc750 con 0x556fbd658160 2024-04-17T07:31:02.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.154+0000 7f7fc4428740 1 -- 172.21.15.86:0/1673576764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 msgr2=0x556fbd658580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:02.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.154+0000 7f7fc4428740 1 --2- 172.21.15.86:0/1673576764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd658580 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc065480 tx=0x7f7fbc067d30 comp rx=0 tx=0).stop 2024-04-17T07:31:02.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.154+0000 7f7fbb400640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7fa00a3d00 0x7f7fa00a61c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7fac007320 tx=0x7f7fac005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:02.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.154+0000 7f7fc4428740 1 -- 172.21.15.86:0/1673576764 shutdown_connections 2024-04-17T07:31:02.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.154+0000 7f7fc4428740 1 --2- 172.21.15.86:0/1673576764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd65dcc0 0x556fbd660150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.154+0000 7f7fc4428740 1 --2- 172.21.15.86:0/1673576764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd658b50 0x556fbd65d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.154+0000 7f7fc4428740 1 --2- 172.21.15.86:0/1673576764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd658580 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.154+0000 7f7fc4428740 1 -- 172.21.15.86:0/1673576764 >> 172.21.15.86:0/1673576764 conn(0x556fbd6536d0 msgr2=0x556fbd653ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:02.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.155+0000 7f7fc4428740 1 -- 172.21.15.86:0/1673576764 shutdown_connections 2024-04-17T07:31:02.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.155+0000 7f7fc4428740 1 -- 172.21.15.86:0/1673576764 wait complete. 2024-04-17T07:31:02.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.156+0000 7f7fc4428740 1 -- start start 2024-04-17T07:31:02.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.156+0000 7f7fc4428740 1 Processor -- start 2024-04-17T07:31:02.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.157+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd662630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.157+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd662b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.157+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd669bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.157+0000 7f7fc4428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556fbd660c10 con 0x556fbd658160 2024-04-17T07:31:02.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.157+0000 7f7fc4428740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556fbd577830 con 0x556fbd65dcc0 2024-04-17T07:31:02.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.157+0000 7f7fc4428740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556fbd660d90 con 0x556fbd658b50 2024-04-17T07:31:02.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.157+0000 7f7fc4428740 10 client.0 _ll_register_callbacks cb 0x556fbd545fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:02.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.158+0000 7f7fbb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd669bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.159+0000 7f7fba000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd662b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.159+0000 7f7fba000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd662b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54718/0 (socket says 172.21.15.86:54718) 2024-04-17T07:31:02.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.159+0000 7f7fba000640 1 -- 172.21.15.86:0/1435984829 learned_addr learned my addr 172.21.15.86:0/1435984829 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:02.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.159+0000 7f7fbaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd662630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.160+0000 7f7fbaa00640 1 -- 172.21.15.86:0/1435984829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 msgr2=0x556fbd662b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:02.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.160+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd662b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.160+0000 7f7fbaa00640 1 -- 172.21.15.86:0/1435984829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 msgr2=0x556fbd669bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.159+0000 7f7fbb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd669bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58334/0 (socket says 172.21.15.86:58334) 2024-04-17T07:31:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.160+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd669bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.160+0000 7f7fbaa00640 1 -- 172.21.15.86:0/1435984829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fbd57b920 con 0x556fbd658160 2024-04-17T07:31:02.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.161+0000 7f7fbb400640 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd669bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:02.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.161+0000 7f7fba000640 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd662b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:02.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.161+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd662630 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0ffbb0 tx=0x7f7fbc104580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1f5f27e4c45da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:02.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.161+0000 7f7f9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:02.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.162+0000 7f7f9fe00640 1 -- 172.21.15.86:0/1435984829 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0fc750 con 0x556fbd658160 2024-04-17T07:31:02.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.162+0000 7f7f9fe00640 1 -- 172.21.15.86:0/1435984829 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7fbc114b80 con 0x556fbd658160 2024-04-17T07:31:02.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.162+0000 7f7fc4428740 1 -- 172.21.15.86:0/1435984829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fbc1000e0 con 0x556fbd658160 2024-04-17T07:31:02.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.162+0000 7f7f9fe00640 1 -- 172.21.15.86:0/1435984829 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fbc0fc8f0 con 0x556fbd658160 2024-04-17T07:31:02.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.162+0000 7f7fc4428740 10 client.19293 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:02.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.162+0000 7f7fc4428740 1 -- 172.21.15.86:0/1435984829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fbd5497f0 con 0x556fbd658160 2024-04-17T07:31:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.164+0000 7f7f9fe00640 1 -- 172.21.15.86:0/1435984829 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fbc1000e0 con 0x556fbd658160 2024-04-17T07:31:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.164+0000 7f7f9fe00640 1 -- 172.21.15.86:0/1435984829 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7fbc0fbe30 con 0x556fbd658160 2024-04-17T07:31:02.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.164+0000 7f7fc4428740 10 client.19293 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:02.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.164+0000 7f7fc4428740 10 client.19293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:02.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.165+0000 7f7fc4428740 20 client.19293 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:02.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.165+0000 7f7fc4428740 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556fbd66dd10 0x556fbd670100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.165+0000 7f7fc4428740 4 client.19293 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:02.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.165+0000 7f7fc4428740 1 -- 172.21.15.86:0/1435984829 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fbd5cba40 con 0x556fbd66dd10 2024-04-17T07:31:02.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.166+0000 7f7fba000640 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556fbd66dd10 0x556fbd670100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.167+0000 7f7fba000640 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556fbd66dd10 0x556fbd670100 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1de175bf029462c server_cookie=80ba0d186a7d6085 in_seq=0 out_seq=0 2024-04-17T07:31:02.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.168+0000 7f7f9fe00640 10 client.19293 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.168+0000 7f7f9fe00640 1 -- 172.21.15.86:0/1435984829 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556fbd5cba40 con 0x556fbd66dd10 2024-04-17T07:31:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.168+0000 7f7f9fe00640 10 client.19293 handle_command_reply: tid=0 2024-04-17T07:31:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.168+0000 7f7fc4428740 1 client.19293 shutdown 2024-04-17T07:31:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.170+0000 7f7fc4428740 1 -- 172.21.15.86:0/1435984829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 msgr2=0x556fbd662630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.170+0000 7f7fc4428740 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd662630 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0ffbb0 tx=0x7f7fbc104580 comp rx=0 tx=0).stop 2024-04-17T07:31:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.173+0000 7f7fc4428740 1 -- 172.21.15.86:0/1435984829 shutdown_connections 2024-04-17T07:31:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.173+0000 7f7fc4428740 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556fbd66dd10 0x556fbd670100 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.173+0000 7f7fc4428740 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd669bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.173+0000 7f7fc4428740 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd662b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.173+0000 7f7fc4428740 1 --2- 172.21.15.86:0/1435984829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd662630 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.173+0000 7f7fc4428740 1 -- 172.21.15.86:0/1435984829 >> 172.21.15.86:0/1435984829 conn(0x556fbd6536d0 msgr2=0x556fbd652930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:02.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.173+0000 7f7f9fe00640 0 client.19293 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:02.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.173+0000 7f7fc4428740 1 -- 172.21.15.86:0/1435984829 shutdown_connections 2024-04-17T07:31:02.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.173+0000 7f7fc4428740 1 -- 172.21.15.86:0/1435984829 wait complete. 2024-04-17T07:31:02.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.173+0000 7f7fc4428740 20 client.19293 trim_cache size 0 max 16384 2024-04-17T07:31:02.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.213+0000 7f7fc4428740 1 Processor -- start 2024-04-17T07:31:02.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.213+0000 7f7fc4428740 1 -- start start 2024-04-17T07:31:02.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.217+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd661c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.217+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd669b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.217+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd662190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.217+0000 7f7fc4428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9800fba0 con 0x556fbd658160 2024-04-17T07:31:02.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.217+0000 7f7fc4428740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7f980116e0 con 0x556fbd65dcc0 2024-04-17T07:31:02.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.217+0000 7f7fc4428740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98010270 con 0x556fbd658b50 2024-04-17T07:31:02.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.218+0000 7f7fbb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd662190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.218+0000 7f7fbb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd662190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58340/0 (socket says 172.21.15.86:58340) 2024-04-17T07:31:02.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.218+0000 7f7fbb400640 1 -- 172.21.15.86:0/3953124785 learned_addr learned my addr 172.21.15.86:0/3953124785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:02.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.218+0000 7f7fba000640 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd669b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.218+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd661c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.220+0000 7f7fbb400640 1 -- 172.21.15.86:0/3953124785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 msgr2=0x556fbd669b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:02.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.220+0000 7f7fbb400640 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd669b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.220+0000 7f7fbb400640 1 -- 172.21.15.86:0/3953124785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 msgr2=0x556fbd661c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:02.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.220+0000 7f7fbb400640 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd661c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.220+0000 7f7fbb400640 1 -- 172.21.15.86:0/3953124785 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fbd2d03f0 con 0x556fbd65dcc0 2024-04-17T07:31:02.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.220+0000 7f7fba000640 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd669b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:02.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.220+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd661c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:02.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.221+0000 7f7fbb400640 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd662190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0074e0 tx=0x7f7fac03c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f89f25c34e57c62c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:02.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.223+0000 7f7faaa00640 1 -- 172.21.15.86:0/3953124785 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac04c030 con 0x556fbd65dcc0 2024-04-17T07:31:02.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.223+0000 7f7faaa00640 1 -- 172.21.15.86:0/3953124785 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fac042040 con 0x556fbd65dcc0 2024-04-17T07:31:02.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.223+0000 7f7faaa00640 1 -- 172.21.15.86:0/3953124785 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac00a710 con 0x556fbd65dcc0 2024-04-17T07:31:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.224+0000 7f7fc4428740 1 -- 172.21.15.86:0/3953124785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 msgr2=0x556fbd662190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.224+0000 7f7fc4428740 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd662190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0074e0 tx=0x7f7fac03c000 comp rx=0 tx=0).stop 2024-04-17T07:31:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.224+0000 7f7fc4428740 1 -- 172.21.15.86:0/3953124785 shutdown_connections 2024-04-17T07:31:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.224+0000 7f7fc4428740 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd662190 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.224+0000 7f7fc4428740 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd669b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.224+0000 7f7fc4428740 1 --2- 172.21.15.86:0/3953124785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd661c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.224+0000 7f7fc4428740 1 -- 172.21.15.86:0/3953124785 >> 172.21.15.86:0/3953124785 conn(0x556fbd6536d0 msgr2=0x556fbd6571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:02.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.224+0000 7f7fc4428740 1 -- 172.21.15.86:0/3953124785 shutdown_connections 2024-04-17T07:31:02.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.225+0000 7f7fc4428740 1 -- 172.21.15.86:0/3953124785 wait complete. 2024-04-17T07:31:02.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.225+0000 7f7fc4428740 1 -- start start 2024-04-17T07:31:02.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.225+0000 7f7fc4428740 1 Processor -- start 2024-04-17T07:31:02.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.226+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd667490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.226+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd6679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.226+0000 7f7fc4428740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd668970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.227+0000 7f7fc4428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98011010 con 0x556fbd658160 2024-04-17T07:31:02.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.227+0000 7f7fc4428740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98010940 con 0x556fbd65dcc0 2024-04-17T07:31:02.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.227+0000 7f7fc4428740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9800db30 con 0x556fbd658b50 2024-04-17T07:31:02.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.227+0000 7f7fc4428740 10 client.0 _ll_register_callbacks cb 0x7f7fbc075460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:02.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.228+0000 7f7fba000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd6679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.228+0000 7f7fbb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd668970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.228+0000 7f7fbaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd667490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.228+0000 7f7fbb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd668970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58352/0 (socket says 172.21.15.86:58352) 2024-04-17T07:31:02.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.228+0000 7f7fbb400640 1 -- 172.21.15.86:0/3870310732 learned_addr learned my addr 172.21.15.86:0/3870310732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:02.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.229+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3870310732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 msgr2=0x556fbd6679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:02.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.229+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd6679d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.230+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3870310732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 msgr2=0x556fbd668970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:02.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.230+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd668970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.230+0000 7f7fbaa00640 1 -- 172.21.15.86:0/3870310732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fbd39a7d0 con 0x556fbd658160 2024-04-17T07:31:02.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.230+0000 7f7fbaa00640 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd667490 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc07e8c0 tx=0x7f7fbc067d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee370ee508a185f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:02.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.230+0000 7f7f9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:02.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.231+0000 7f7f9fe00640 1 -- 172.21.15.86:0/3870310732 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0e5070 con 0x556fbd658160 2024-04-17T07:31:02.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.231+0000 7f7f9fe00640 1 -- 172.21.15.86:0/3870310732 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f7fbc0fefa0 con 0x556fbd658160 2024-04-17T07:31:02.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.231+0000 7f7fc4428740 1 -- 172.21.15.86:0/3870310732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fac042040 con 0x556fbd658160 2024-04-17T07:31:02.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.232+0000 7f7f9fe00640 1 -- 172.21.15.86:0/3870310732 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fbc0fc8f0 con 0x556fbd658160 2024-04-17T07:31:02.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.232+0000 7f7fc4428740 10 client.19299 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:02.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.232+0000 7f7fc4428740 1 -- 172.21.15.86:0/3870310732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556fbd6d40b0 con 0x556fbd658160 2024-04-17T07:31:02.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.233+0000 7f7f9fe00640 1 -- 172.21.15.86:0/3870310732 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fac042040 con 0x556fbd658160 2024-04-17T07:31:02.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.233+0000 7f7f9fe00640 1 -- 172.21.15.86:0/3870310732 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f7fbc0fbe30 con 0x556fbd658160 2024-04-17T07:31:02.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.233+0000 7f7fc4428740 10 client.19299 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:02.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.233+0000 7f7fc4428740 10 client.19299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:02.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.233+0000 7f7fc4428740 20 client.19299 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:02.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.233+0000 7f7fc4428740 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556fbd6d4800 0x556fbd6d6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:02.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.233+0000 7f7fc4428740 4 client.19299 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:31:02.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.233+0000 7f7fc4428740 1 -- 172.21.15.86:0/3870310732 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fbd5c58b0 con 0x556fbd6d4800 2024-04-17T07:31:02.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.235+0000 7f7fba000640 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556fbd6d4800 0x556fbd6d6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.236+0000 7f7fba000640 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556fbd6d4800 0x556fbd6d6bf0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5605e59562313c8d server_cookie=6e7057f4697d43b4 in_seq=0 out_seq=0 2024-04-17T07:31:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.236+0000 7f7f9fe00640 10 client.19299 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7f9fe00640 1 -- 172.21.15.86:0/3870310732 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fbd5c58b0 con 0x556fbd6d4800 2024-04-17T07:31:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7f9fe00640 10 client.19299 handle_command_reply: tid=0 2024-04-17T07:31:02.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7fc4428740 1 client.19299 shutdown 2024-04-17T07:31:02.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7fc4428740 1 -- 172.21.15.86:0/3870310732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 msgr2=0x556fbd667490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7fc4428740 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd667490 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc07e8c0 tx=0x7f7fbc067d30 comp rx=0 tx=0).stop 2024-04-17T07:31:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7fc4428740 1 -- 172.21.15.86:0/3870310732 shutdown_connections 2024-04-17T07:31:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7fc4428740 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556fbd6d4800 0x556fbd6d6bf0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7fc4428740 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556fbd65dcc0 0x556fbd668970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7fc4428740 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fbd658b50 0x556fbd6679d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7fc4428740 1 --2- 172.21.15.86:0/3870310732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556fbd658160 0x556fbd667490 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7fc4428740 1 -- 172.21.15.86:0/3870310732 >> 172.21.15.86:0/3870310732 conn(0x556fbd6536d0 msgr2=0x556fbd6520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:02.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.237+0000 7f7f9fe00640 0 client.19299 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:02.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.238+0000 7f7fc4428740 1 -- 172.21.15.86:0/3870310732 shutdown_connections 2024-04-17T07:31:02.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.238+0000 7f7fc4428740 1 -- 172.21.15.86:0/3870310732 wait complete. 2024-04-17T07:31:02.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.238+0000 7f7fc4428740 20 client.19299 trim_cache size 0 max 16384 2024-04-17T07:31:02.242 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:31:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.240+0000 7f7fbbe00640 1 -- 172.21.15.86:0/3370062684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7fa00a3d00 msgr2=0x7f7fa00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:02.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.240+0000 7f7fbbe00640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7fa00a3d00 0x7f7fa00a61c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7fac007320 tx=0x7f7fac005a20 comp rx=0 tx=0).stop 2024-04-17T07:31:02.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.240+0000 7f7fbbe00640 1 -- 172.21.15.86:0/3370062684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fb40a4fe0 msgr2=0x7f7fb413ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:02.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.240+0000 7f7fbbe00640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fb40a4fe0 0x7f7fb413ad00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc067750 tx=0x7f7fbc072a20 comp rx=0 tx=0).stop 2024-04-17T07:31:02.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.240+0000 7f7fbbe00640 1 -- 172.21.15.86:0/3370062684 shutdown_connections 2024-04-17T07:31:02.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.240+0000 7f7fbbe00640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f7fa00a3d00 0x7f7fa00a61c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.240+0000 7f7fbbe00640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb40a6b90 0x7f7fb4153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.240+0000 7f7fbbe00640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb40a61d0 0x7f7fb413b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.240+0000 7f7fbbe00640 1 --2- 172.21.15.86:0/3370062684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fb40a4fe0 0x7f7fb413ad00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:02.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.240+0000 7f7fbbe00640 1 -- 172.21.15.86:0/3370062684 >> 172.21.15.86:0/3370062684 conn(0x7f7fb40a0530 msgr2=0x7f7fb40aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:02.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.241+0000 7f7fbbe00640 1 -- 172.21.15.86:0/3370062684 shutdown_connections 2024-04-17T07:31:02.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:02.241+0000 7f7fbbe00640 1 -- 172.21.15.86:0/3370062684 wait complete. 2024-04-17T07:31:03.282 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:31:03.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:31:03.282 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:31:03.408 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:03 smithi086 ceph-mon[31733]: pgmap v940: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 643 op/s 2024-04-17T07:31:03.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:03 smithi098 ceph-mon[35418]: pgmap v940: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 643 op/s 2024-04-17T07:31:03.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:03 smithi103 ceph-mon[35495]: pgmap v940: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 643 op/s 2024-04-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.664+0000 7f5b8bc00640 1 -- 172.21.15.86:0/542801768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b84105d70 msgr2=0x7f5b8410c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.664+0000 7f5b8bc00640 1 --2- 172.21.15.86:0/542801768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b84105d70 0x7f5b8410c600 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009be0 tx=0x7f5b7c02bef0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.665+0000 7f5b8bc00640 1 -- 172.21.15.86:0/542801768 shutdown_connections 2024-04-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.665+0000 7f5b8bc00640 1 --2- 172.21.15.86:0/542801768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b84105d70 0x7f5b8410c600 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.665+0000 7f5b8bc00640 1 --2- 172.21.15.86:0/542801768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b841053b0 0x7f5b84105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.665+0000 7f5b8bc00640 1 --2- 172.21.15.86:0/542801768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b841041c0 0x7f5b841045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.665+0000 7f5b8bc00640 1 -- 172.21.15.86:0/542801768 >> 172.21.15.86:0/542801768 conn(0x7f5b840ff710 msgr2=0x7f5b84101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.666+0000 7f5b8bc00640 1 -- 172.21.15.86:0/542801768 shutdown_connections 2024-04-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.667+0000 7f5b8bc00640 1 -- 172.21.15.86:0/542801768 wait complete. 2024-04-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.669+0000 7f5b8bc00640 1 Processor -- start 2024-04-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.670+0000 7f5b8bc00640 1 -- start start 2024-04-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.670+0000 7f5b8bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b841041c0 0x7f5b84070990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.670+0000 7f5b8bc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b841053b0 0x7f5b84070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.670+0000 7f5b8bc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b84105d70 0x7f5b841aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.670+0000 7f5b8bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8410fd00 con 0x7f5b841041c0 2024-04-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.670+0000 7f5b8bc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8410fb80 con 0x7f5b84105d70 2024-04-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.670+0000 7f5b8bc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8410fe80 con 0x7f5b841053b0 2024-04-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.671+0000 7f5b89200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b841041c0 0x7f5b84070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.671+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b841053b0 0x7f5b84070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.671+0000 7f5b89200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b841041c0 0x7f5b84070990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47242/0 (socket says 172.21.15.86:47242) 2024-04-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.671+0000 7f5b89200640 1 -- 172.21.15.86:0/470726637 learned_addr learned my addr 172.21.15.86:0/470726637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.671+0000 7f5b89c00640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b84105d70 0x7f5b841aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.672+0000 7f5b89200640 1 -- 172.21.15.86:0/470726637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b841053b0 msgr2=0x7f5b84070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.672+0000 7f5b89200640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b841053b0 0x7f5b84070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.672+0000 7f5b89200640 1 -- 172.21.15.86:0/470726637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b84105d70 msgr2=0x7f5b841aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.672+0000 7f5b89200640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b84105d70 0x7f5b841aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.672+0000 7f5b89200640 1 -- 172.21.15.86:0/470726637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8407a3c0 con 0x7f5b841041c0 2024-04-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.672+0000 7f5b83e00640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b841053b0 0x7f5b84070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.673+0000 7f5b89200640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b841041c0 0x7f5b84070990 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800e9f0 tx=0x7f5b7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.674+0000 7f5b81600640 1 -- 172.21.15.86:0/470726637 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800ccc0 con 0x7f5b841041c0 2024-04-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.674+0000 7f5b81600640 1 -- 172.21.15.86:0/470726637 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b780035f0 con 0x7f5b841041c0 2024-04-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.674+0000 7f5b81600640 1 -- 172.21.15.86:0/470726637 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800ce60 con 0x7f5b841041c0 2024-04-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.675+0000 7f5b8bc00640 1 -- 172.21.15.86:0/470726637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b841af100 con 0x7f5b841041c0 2024-04-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.675+0000 7f5b8bc00640 1 -- 172.21.15.86:0/470726637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b841af4b0 con 0x7f5b841041c0 2024-04-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.684+0000 7f5b8e865740 1 Processor -- start 2024-04-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.685+0000 7f5b8e865740 1 -- start start 2024-04-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.685+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.685+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.685+0000 7f5b81600640 1 -- 172.21.15.86:0/470726637 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5b780037a0 con 0x7f5b841041c0 2024-04-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.685+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.685+0000 7f5b8e865740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557ae731cb0 con 0x5557ae7d1a20 2024-04-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.685+0000 7f5b8e865740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5557ae6d8370 con 0x5557ae7cc8b0 2024-04-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.685+0000 7f5b8e865740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5557ae72dc70 con 0x5557ae7cbec0 2024-04-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.686+0000 7f5b89200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.687+0000 7f5b89200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58388/0 (socket says 172.21.15.86:58388) 2024-04-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.687+0000 7f5b89200640 1 -- 172.21.15.86:0/3115780046 learned_addr learned my addr 172.21.15.86:0/3115780046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.687+0000 7f5b83e00640 1 --2- 172.21.15.86:0/3115780046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.687+0000 7f5b89200640 1 -- 172.21.15.86:0/3115780046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 msgr2=0x5557ae7cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.687+0000 7f5b89200640 1 --2- 172.21.15.86:0/3115780046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.687+0000 7f5b89200640 1 -- 172.21.15.86:0/3115780046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 msgr2=0x5557ae7d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.689+0000 7f5b89200640 1 --2- 172.21.15.86:0/3115780046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.689+0000 7f5b89200640 1 -- 172.21.15.86:0/3115780046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557ae6f7910 con 0x5557ae7cc8b0 2024-04-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.689+0000 7f5b83e00640 1 --2- 172.21.15.86:0/3115780046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.690+0000 7f5b89200640 1 --2- 172.21.15.86:0/3115780046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d13b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5b780d6a90 tx=0x7f5b780d6f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e912ea6c0d70c86e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.690+0000 7f5b81600640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b6c0a3d80 0x7f5b6c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.690+0000 7f5b76000640 1 -- 172.21.15.86:0/3115780046 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780df070 con 0x5557ae7cc8b0 2024-04-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.690+0000 7f5b76000640 1 -- 172.21.15.86:0/3115780046 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b780a6e20 con 0x5557ae7cc8b0 2024-04-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.690+0000 7f5b76000640 1 -- 172.21.15.86:0/3115780046 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780af570 con 0x5557ae7cc8b0 2024-04-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.691+0000 7f5b81600640 1 -- 172.21.15.86:0/470726637 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5b780d0a10 con 0x7f5b841041c0 2024-04-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.691+0000 7f5b89c00640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b6c0a3d80 0x7f5b6c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.692+0000 7f5b8e865740 1 -- 172.21.15.86:0/3115780046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 msgr2=0x5557ae7d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.692+0000 7f5b8e865740 1 --2- 172.21.15.86:0/3115780046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d13b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5b780d6a90 tx=0x7f5b780d6f50 comp rx=0 tx=0).stop 2024-04-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.693+0000 7f5b89c00640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b6c0a3d80 0x7f5b6c0a6240 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c004af0 tx=0x7f5b7c005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.693+0000 7f5b8e865740 1 -- 172.21.15.86:0/3115780046 shutdown_connections 2024-04-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.693+0000 7f5b8e865740 1 --2- 172.21.15.86:0/3115780046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.693+0000 7f5b8e865740 1 --2- 172.21.15.86:0/3115780046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d13b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.693+0000 7f5b8e865740 1 --2- 172.21.15.86:0/3115780046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.693+0000 7f5b8e865740 1 -- 172.21.15.86:0/3115780046 >> 172.21.15.86:0/3115780046 conn(0x5557ae7c7430 msgr2=0x5557ae7c7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.694+0000 7f5b8e865740 1 -- 172.21.15.86:0/3115780046 shutdown_connections 2024-04-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.694+0000 7f5b8e865740 1 -- 172.21.15.86:0/3115780046 wait complete. 2024-04-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.695+0000 7f5b8e865740 1 -- start start 2024-04-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.695+0000 7f5b8e865740 1 Processor -- start 2024-04-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.696+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cbec0 0x5557ae7d64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.696+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.696+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7dda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.696+0000 7f5b8e865740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557ae7d49d0 con 0x5557ae7d1a20 2024-04-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.696+0000 7f5b8e865740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5557ae6eb730 con 0x5557ae7cbec0 2024-04-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.696+0000 7f5b8e865740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5557ae7d4b50 con 0x5557ae7cc8b0 2024-04-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.697+0000 7f5b8e865740 10 client.0 _ll_register_callbacks cb 0x5557ae6edb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:03.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.697+0000 7f5b89c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7dda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.697+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.697+0000 7f5b89200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cbec0 0x5557ae7d64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.697+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54762/0 (socket says 172.21.15.86:54762) 2024-04-17T07:31:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.697+0000 7f5b83e00640 1 -- 172.21.15.86:0/4226700820 learned_addr learned my addr 172.21.15.86:0/4226700820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.698+0000 7f5b83e00640 1 -- 172.21.15.86:0/4226700820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cbec0 msgr2=0x5557ae7d64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.698+0000 7f5b83e00640 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cbec0 0x5557ae7d64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.698+0000 7f5b83e00640 1 -- 172.21.15.86:0/4226700820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 msgr2=0x5557ae7dda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.698+0000 7f5b83e00640 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7dda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.698+0000 7f5b83e00640 1 -- 172.21.15.86:0/4226700820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557ae50c7e0 con 0x5557ae7cc8b0 2024-04-17T07:31:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.700+0000 7f5b83e00640 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d6a10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5b70012770 tx=0x7f5b70012c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f32d664a86920db server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.700+0000 7f5b6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:31:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.701+0000 7f5b6aa00640 1 -- 172.21.15.86:0/4226700820 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70011b50 con 0x5557ae7cc8b0 2024-04-17T07:31:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.701+0000 7f5b6aa00640 1 -- 172.21.15.86:0/4226700820 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5b70011cf0 con 0x5557ae7cc8b0 2024-04-17T07:31:03.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.701+0000 7f5b8e865740 1 -- 172.21.15.86:0/4226700820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b780a6e20 con 0x5557ae7cc8b0 2024-04-17T07:31:03.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.702+0000 7f5b6aa00640 1 -- 172.21.15.86:0/4226700820 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b70004030 con 0x5557ae7cc8b0 2024-04-17T07:31:03.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.702+0000 7f5b8e865740 10 client.28330 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:03.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.702+0000 7f5b8e865740 1 -- 172.21.15.86:0/4226700820 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557ae56cf70 con 0x5557ae7cc8b0 2024-04-17T07:31:03.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.703+0000 7f5b6aa00640 1 -- 172.21.15.86:0/4226700820 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b780a6e20 con 0x5557ae7cc8b0 2024-04-17T07:31:03.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.703+0000 7f5b6aa00640 1 -- 172.21.15.86:0/4226700820 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5b7001c460 con 0x5557ae7cc8b0 2024-04-17T07:31:03.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.703+0000 7f5b8e865740 10 client.28330 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:03.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.703+0000 7f5b8e865740 10 client.28330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:03.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.703+0000 7f5b8e865740 20 client.28330 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:03.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.703+0000 7f5b8e865740 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557ae7e2290 0x5557ae7e4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.703+0000 7f5b8e865740 4 client.28330 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:03.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.703+0000 7f5b8e865740 1 -- 172.21.15.86:0/4226700820 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557ae73f6d0 con 0x5557ae7e2290 2024-04-17T07:31:03.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.706+0000 7f5b89200640 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557ae7e2290 0x5557ae7e4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.707+0000 7f5b89200640 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557ae7e2290 0x5557ae7e4710 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dcc1fce914a8c3e server_cookie=ed584011c44f3f26 in_seq=0 out_seq=0 2024-04-17T07:31:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.707+0000 7f5b6aa00640 10 client.28330 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.708+0000 7f5b6aa00640 1 -- 172.21.15.86:0/4226700820 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557ae73f6d0 con 0x5557ae7e2290 2024-04-17T07:31:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.708+0000 7f5b6aa00640 10 client.28330 handle_command_reply: tid=0 2024-04-17T07:31:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.708+0000 7f5b8e865740 1 client.28330 shutdown 2024-04-17T07:31:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b8e865740 1 -- 172.21.15.86:0/4226700820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cc8b0 msgr2=0x5557ae7d6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b8e865740 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d6a10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5b70012770 tx=0x7f5b70012c30 comp rx=0 tx=0).stop 2024-04-17T07:31:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b8e865740 1 -- 172.21.15.86:0/4226700820 shutdown_connections 2024-04-17T07:31:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b8e865740 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557ae7e2290 0x5557ae7e4710 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b8e865740 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7dda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b8e865740 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cc8b0 0x5557ae7d6a10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b8e865740 1 --2- 172.21.15.86:0/4226700820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cbec0 0x5557ae7d64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b8e865740 1 -- 172.21.15.86:0/4226700820 >> 172.21.15.86:0/4226700820 conn(0x5557ae7c7430 msgr2=0x5557ae7c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b6aa00640 0 client.28330 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b8e865740 1 -- 172.21.15.86:0/4226700820 shutdown_connections 2024-04-17T07:31:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.709+0000 7f5b8e865740 1 -- 172.21.15.86:0/4226700820 wait complete. 2024-04-17T07:31:03.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.710+0000 7f5b8e865740 20 client.28330 trim_cache size 0 max 16384 2024-04-17T07:31:03.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.748+0000 7f5b8e865740 1 Processor -- start 2024-04-17T07:31:03.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.748+0000 7f5b8e865740 1 -- start start 2024-04-17T07:31:03.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.748+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.748+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.748+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.749+0000 7f5b8e865740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b9a80 con 0x5557ae7d1a20 2024-04-17T07:31:03.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.749+0000 7f5b8e865740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841bb5c0 con 0x5557ae7cc8b0 2024-04-17T07:31:03.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.749+0000 7f5b8e865740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841ba150 con 0x5557ae7cbec0 2024-04-17T07:31:03.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.749+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.749+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7dda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58416/0 (socket says 172.21.15.86:58416) 2024-04-17T07:31:03.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.749+0000 7f5b83e00640 1 -- 172.21.15.86:0/3401681333 learned_addr learned my addr 172.21.15.86:0/3401681333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:03.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.749+0000 7f5b89c00640 1 --2- 172.21.15.86:0/3401681333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.749+0000 7f5b89200640 1 --2- 172.21.15.86:0/3401681333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.752+0000 7f5b83e00640 1 -- 172.21.15.86:0/3401681333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 msgr2=0x5557ae7e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:03.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.752+0000 7f5b83e00640 1 --2- 172.21.15.86:0/3401681333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.752+0000 7f5b83e00640 1 -- 172.21.15.86:0/3401681333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 msgr2=0x5557ae7e4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:03.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.752+0000 7f5b83e00640 1 --2- 172.21.15.86:0/3401681333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7e4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.753+0000 7f5b83e00640 1 -- 172.21.15.86:0/3401681333 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557ae404170 con 0x5557ae7cc8b0 2024-04-17T07:31:03.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.753+0000 7f5b89c00640 1 --2- 172.21.15.86:0/3401681333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7e4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.754+0000 7f5b83e00640 1 --2- 172.21.15.86:0/3401681333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7dda10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009e30 tx=0x7f5b700118c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd0e49ca95e756dc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.754+0000 7f5b75600640 1 -- 172.21.15.86:0/3401681333 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7001c520 con 0x5557ae7cc8b0 2024-04-17T07:31:03.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.755+0000 7f5b75600640 1 -- 172.21.15.86:0/3401681333 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b7001caa0 con 0x5557ae7cc8b0 2024-04-17T07:31:03.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.755+0000 7f5b75600640 1 -- 172.21.15.86:0/3401681333 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7001c520 con 0x5557ae7cc8b0 2024-04-17T07:31:03.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.758+0000 7f5b8e865740 1 -- 172.21.15.86:0/3401681333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 msgr2=0x5557ae7dda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.758+0000 7f5b8e865740 1 --2- 172.21.15.86:0/3401681333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7dda10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009e30 tx=0x7f5b700118c0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.758+0000 7f5b8e865740 1 -- 172.21.15.86:0/3401681333 shutdown_connections 2024-04-17T07:31:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.758+0000 7f5b8e865740 1 --2- 172.21.15.86:0/3401681333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7d1a20 0x5557ae7e4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.758+0000 7f5b8e865740 1 --2- 172.21.15.86:0/3401681333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7cc8b0 0x5557ae7dda10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.758+0000 7f5b8e865740 1 --2- 172.21.15.86:0/3401681333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.758+0000 7f5b8e865740 1 -- 172.21.15.86:0/3401681333 >> 172.21.15.86:0/3401681333 conn(0x5557ae7c7430 msgr2=0x5557ae7cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.759+0000 7f5b8e865740 1 -- 172.21.15.86:0/3401681333 shutdown_connections 2024-04-17T07:31:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.759+0000 7f5b8e865740 1 -- 172.21.15.86:0/3401681333 wait complete. 2024-04-17T07:31:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.760+0000 7f5b8e865740 1 -- start start 2024-04-17T07:31:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.760+0000 7f5b8e865740 1 Processor -- start 2024-04-17T07:31:03.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.761+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7dadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.761+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7cc8b0 0x5557ae7db330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.761+0000 7f5b8e865740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7d1a20 0x5557ae7dc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.761+0000 7f5b89200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7dadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.761+0000 7f5b89200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7dadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54784/0 (socket says 172.21.15.86:54784) 2024-04-17T07:31:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.762+0000 7f5b8e865740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841baef0 con 0x5557ae7cc8b0 2024-04-17T07:31:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.762+0000 7f5b8e865740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841ba820 con 0x5557ae7d1a20 2024-04-17T07:31:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.762+0000 7f5b8e865740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b7a50 con 0x5557ae7cbec0 2024-04-17T07:31:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.762+0000 7f5b8e865740 10 client.0 _ll_register_callbacks cb 0x7f5b70009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.762+0000 7f5b89c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7d1a20 0x5557ae7dc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.762+0000 7f5b89200640 1 -- 172.21.15.86:0/2394625674 learned_addr learned my addr 172.21.15.86:0/2394625674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.762+0000 7f5b83e00640 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7cc8b0 0x5557ae7db330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.763+0000 7f5b89200640 1 -- 172.21.15.86:0/2394625674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7d1a20 msgr2=0x5557ae7dc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.763+0000 7f5b89200640 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7d1a20 0x5557ae7dc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.763+0000 7f5b89200640 1 -- 172.21.15.86:0/2394625674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7cc8b0 msgr2=0x5557ae7db330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.763+0000 7f5b89200640 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7cc8b0 0x5557ae7db330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.763+0000 7f5b89200640 1 -- 172.21.15.86:0/2394625674 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557ae7dc930 con 0x5557ae7cbec0 2024-04-17T07:31:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.764+0000 7f5b83e00640 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7cc8b0 0x5557ae7db330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.764+0000 7f5b89200640 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7dadf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5b7809c610 tx=0x7f5b7809c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d446abef867fe7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.764+0000 7f5b6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:31:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.764+0000 7f5b6aa00640 1 -- 172.21.15.86:0/2394625674 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780df040 con 0x5557ae7cbec0 2024-04-17T07:31:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.764+0000 7f5b6aa00640 1 -- 172.21.15.86:0/2394625674 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5b780ce240 con 0x5557ae7cbec0 2024-04-17T07:31:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.765+0000 7f5b8e865740 1 -- 172.21.15.86:0/2394625674 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b7001caa0 con 0x5557ae7cbec0 2024-04-17T07:31:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.766+0000 7f5b6aa00640 1 -- 172.21.15.86:0/2394625674 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b78098bb0 con 0x5557ae7cbec0 2024-04-17T07:31:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.766+0000 7f5b8e865740 10 client.28336 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.766+0000 7f5b8e865740 1 -- 172.21.15.86:0/2394625674 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557ae7ddc70 con 0x5557ae7cbec0 2024-04-17T07:31:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.767+0000 7f5b6aa00640 1 -- 172.21.15.86:0/2394625674 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b7001caa0 con 0x5557ae7cbec0 2024-04-17T07:31:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.767+0000 7f5b6aa00640 1 -- 172.21.15.86:0/2394625674 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5b78098f80 con 0x5557ae7cbec0 2024-04-17T07:31:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.768+0000 7f5b8e865740 10 client.28336 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.768+0000 7f5b8e865740 10 client.28336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.768+0000 7f5b8e865740 20 client.28336 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.768+0000 7f5b8e865740 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557ae848b20 0x5557ae84af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.768+0000 7f5b8e865740 4 client.28336 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:31:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.768+0000 7f5b8e865740 1 -- 172.21.15.86:0/2394625674 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5557ae739540 con 0x5557ae848b20 2024-04-17T07:31:03.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.769+0000 7f5b83e00640 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557ae848b20 0x5557ae84af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:03.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.771+0000 7f5b83e00640 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557ae848b20 0x5557ae84af10 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c7786832470439 server_cookie=5ea58c3b129e6362 in_seq=0 out_seq=0 2024-04-17T07:31:03.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.773+0000 7f5b6aa00640 10 client.28336 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:03.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.775+0000 7f5b6aa00640 1 -- 172.21.15.86:0/2394625674 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5557ae739540 con 0x5557ae848b20 2024-04-17T07:31:03.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.775+0000 7f5b6aa00640 10 client.28336 handle_command_reply: tid=0 2024-04-17T07:31:03.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.775+0000 7f5b8e865740 1 client.28336 shutdown 2024-04-17T07:31:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.775+0000 7f5b8e865740 1 -- 172.21.15.86:0/2394625674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 msgr2=0x5557ae7dadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.775+0000 7f5b8e865740 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7dadf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5b7809c610 tx=0x7f5b7809c450 comp rx=0 tx=0).stop 2024-04-17T07:31:03.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.776+0000 7f5b8e865740 1 -- 172.21.15.86:0/2394625674 shutdown_connections 2024-04-17T07:31:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.776+0000 7f5b8e865740 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5557ae848b20 0x5557ae84af10 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.776+0000 7f5b8e865740 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5557ae7d1a20 0x5557ae7dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.776+0000 7f5b8e865740 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557ae7cc8b0 0x5557ae7db330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.776+0000 7f5b8e865740 1 --2- 172.21.15.86:0/2394625674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5557ae7cbec0 0x5557ae7dadf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.776+0000 7f5b8e865740 1 -- 172.21.15.86:0/2394625674 >> 172.21.15.86:0/2394625674 conn(0x5557ae7c7430 msgr2=0x5557ae7d6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.776+0000 7f5b6aa00640 0 client.28336 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.776+0000 7f5b8e865740 1 -- 172.21.15.86:0/2394625674 shutdown_connections 2024-04-17T07:31:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.776+0000 7f5b8e865740 1 -- 172.21.15.86:0/2394625674 wait complete. 2024-04-17T07:31:03.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.777+0000 7f5b8e865740 20 client.28336 trim_cache size 0 max 16384 2024-04-17T07:31:03.777 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:31:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:31:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "39a63fd3-c052-461b-9259-ffd877e03773", 2024-04-17T07:31:03.778 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:31:03.778 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:31:03.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.778+0000 7f5b8bc00640 1 -- 172.21.15.86:0/470726637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b6c0a3d80 msgr2=0x7f5b6c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:03.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.778+0000 7f5b8bc00640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b6c0a3d80 0x7f5b6c0a6240 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c004af0 tx=0x7f5b7c005c10 comp rx=0 tx=0).stop 2024-04-17T07:31:03.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.778+0000 7f5b8bc00640 1 -- 172.21.15.86:0/470726637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b841041c0 msgr2=0x7f5b84070990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:03.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.779+0000 7f5b8bc00640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b841041c0 0x7f5b84070990 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800e9f0 tx=0x7f5b7800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.779+0000 7f5b8bc00640 1 -- 172.21.15.86:0/470726637 shutdown_connections 2024-04-17T07:31:03.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.779+0000 7f5b8bc00640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5b6c0a3d80 0x7f5b6c0a6240 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.779+0000 7f5b8bc00640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b84105d70 0x7f5b841aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.779+0000 7f5b8bc00640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b841053b0 0x7f5b84070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.779+0000 7f5b8bc00640 1 --2- 172.21.15.86:0/470726637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b841041c0 0x7f5b84070990 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.779+0000 7f5b8bc00640 1 -- 172.21.15.86:0/470726637 >> 172.21.15.86:0/470726637 conn(0x7f5b840ff710 msgr2=0x7f5b841086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.780+0000 7f5b8bc00640 1 -- 172.21.15.86:0/470726637 shutdown_connections 2024-04-17T07:31:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:03.780+0000 7f5b8bc00640 1 -- 172.21.15.86:0/470726637 wait complete. 2024-04-17T07:31:03.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:31:03.826 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:31:04.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.246+0000 7f8f0cc00640 1 -- 172.21.15.86:0/2862231743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f08105c50 msgr2=0x7f8f0810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:04.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.246+0000 7f8f0cc00640 1 --2- 172.21.15.86:0/2862231743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f08105c50 0x7f8f0810c4e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009ca0 tx=0x7f8f0002c100 comp rx=0 tx=0).stop 2024-04-17T07:31:04.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.247+0000 7f8f0cc00640 1 -- 172.21.15.86:0/2862231743 shutdown_connections 2024-04-17T07:31:04.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.247+0000 7f8f0cc00640 1 --2- 172.21.15.86:0/2862231743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f08105c50 0x7f8f0810c4e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.247+0000 7f8f0cc00640 1 --2- 172.21.15.86:0/2862231743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f08105290 0x7f8f08105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.247+0000 7f8f0cc00640 1 --2- 172.21.15.86:0/2862231743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f081040a0 0x7f8f081044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.247+0000 7f8f0cc00640 1 -- 172.21.15.86:0/2862231743 >> 172.21.15.86:0/2862231743 conn(0x7f8f080ff610 msgr2=0x7f8f08101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:04.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.248+0000 7f8f0cc00640 1 -- 172.21.15.86:0/2862231743 shutdown_connections 2024-04-17T07:31:04.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.248+0000 7f8f0cc00640 1 -- 172.21.15.86:0/2862231743 wait complete. 2024-04-17T07:31:04.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.251+0000 7f8f0cc00640 1 Processor -- start 2024-04-17T07:31:04.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.251+0000 7f8f0cc00640 1 -- start start 2024-04-17T07:31:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.252+0000 7f8f0cc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f081040a0 0x7f8f0819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.252+0000 7f8f0cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f08105290 0x7f8f0819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.252+0000 7f8f0cc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f08105c50 0x7f8f081a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.253+0000 7f8f0cc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810fbe0 con 0x7f8f08105290 2024-04-17T07:31:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.253+0000 7f8f0cc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810fa60 con 0x7f8f081040a0 2024-04-17T07:31:04.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.253+0000 7f8f0cc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810fd60 con 0x7f8f08105c50 2024-04-17T07:31:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.253+0000 7f8f07e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f08105c50 0x7f8f081a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.253+0000 7f8f07400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f081040a0 0x7f8f0819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.253+0000 7f8f07e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f08105c50 0x7f8f081a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54806/0 (socket says 172.21.15.86:54806) 2024-04-17T07:31:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.253+0000 7f8f07e00640 1 -- 172.21.15.86:0/3839468907 learned_addr learned my addr 172.21.15.86:0/3839468907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.254+0000 7f8f06a00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f08105290 0x7f8f0819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.253+0000 7f8f07400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f081040a0 0x7f8f0819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58434/0 (socket says 172.21.15.86:58434) 2024-04-17T07:31:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.254+0000 7f8f07e00640 1 -- 172.21.15.86:0/3839468907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f081040a0 msgr2=0x7f8f0819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.254+0000 7f8f07e00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f081040a0 0x7f8f0819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.254+0000 7f8f07e00640 1 -- 172.21.15.86:0/3839468907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f08105290 msgr2=0x7f8f0819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.254+0000 7f8f07e00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f08105290 0x7f8f0819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.254+0000 7f8f07e00640 1 -- 172.21.15.86:0/3839468907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f081a1e40 con 0x7f8f08105c50 2024-04-17T07:31:04.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.255+0000 7f8f07e00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f08105c50 0x7f8f081a17d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8f00037040 tx=0x7f8f00003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.256+0000 7f8effe00640 1 -- 172.21.15.86:0/3839468907 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00039040 con 0x7f8f08105c50 2024-04-17T07:31:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.256+0000 7f8f0cc00640 1 -- 172.21.15.86:0/3839468907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f081a20d0 con 0x7f8f08105c50 2024-04-17T07:31:04.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.256+0000 7f8f0cc00640 1 -- 172.21.15.86:0/3839468907 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f081a2610 con 0x7f8f08105c50 2024-04-17T07:31:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.257+0000 7f8effe00640 1 -- 172.21.15.86:0/3839468907 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f00044070 con 0x7f8f08105c50 2024-04-17T07:31:04.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.257+0000 7f8effe00640 1 -- 172.21.15.86:0/3839468907 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f000043b0 con 0x7f8f08105c50 2024-04-17T07:31:04.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.264+0000 7f8f10fe4740 1 Processor -- start 2024-04-17T07:31:04.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.265+0000 7f8effe00640 1 -- 172.21.15.86:0/3839468907 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8f00004550 con 0x7f8f08105c50 2024-04-17T07:31:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.266+0000 7f8f10fe4740 1 -- start start 2024-04-17T07:31:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.266+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe32f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.267+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32fd60 0x5620fe334860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.267+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe334ed0 0x5620fe337360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.267+0000 7f8f10fe4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5620fe296120 con 0x5620fe32f370 2024-04-17T07:31:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.267+0000 7f8f10fe4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5620fe23c340 con 0x5620fe32fd60 2024-04-17T07:31:04.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.267+0000 7f8f10fe4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5620fe2920e0 con 0x5620fe334ed0 2024-04-17T07:31:04.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.267+0000 7f8f07400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32fd60 0x5620fe334860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.267+0000 7f8f07400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32fd60 0x5620fe334860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58448/0 (socket says 172.21.15.86:58448) 2024-04-17T07:31:04.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.267+0000 7f8f07400640 1 -- 172.21.15.86:0/2628008747 learned_addr learned my addr 172.21.15.86:0/2628008747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.268+0000 7f8f07400640 1 -- 172.21.15.86:0/2628008747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe334ed0 msgr2=0x5620fe337360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.268+0000 7f8f07400640 1 --2- 172.21.15.86:0/2628008747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe334ed0 0x5620fe337360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.268+0000 7f8f07400640 1 -- 172.21.15.86:0/2628008747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 msgr2=0x5620fe32f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.268+0000 7f8f07400640 1 --2- 172.21.15.86:0/2628008747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe32f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.268+0000 7f8f07400640 1 -- 172.21.15.86:0/2628008747 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620fe0727d0 con 0x5620fe32fd60 2024-04-17T07:31:04.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.269+0000 7f8f07400640 1 --2- 172.21.15.86:0/2628008747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32fd60 0x5620fe334860 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000bcb0 tx=0x7f8ef00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=896a192083893d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.270+0000 7f8ef7e00640 1 -- 172.21.15.86:0/2628008747 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef001c480 con 0x5620fe32fd60 2024-04-17T07:31:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.270+0000 7f8ef7e00640 1 -- 172.21.15.86:0/2628008747 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ef00118f0 con 0x5620fe32fd60 2024-04-17T07:31:04.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.270+0000 7f8ef7e00640 1 -- 172.21.15.86:0/2628008747 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef001a900 con 0x5620fe32fd60 2024-04-17T07:31:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.270+0000 7f8effe00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8eec0a3c30 0x7f8eec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.271+0000 7f8effe00640 1 -- 172.21.15.86:0/3839468907 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8f000f2700 con 0x7f8f08105c50 2024-04-17T07:31:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.271+0000 7f8f06a00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8eec0a3c30 0x7f8eec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.271+0000 7f8f10fe4740 1 -- 172.21.15.86:0/2628008747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32fd60 msgr2=0x5620fe334860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:04.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.271+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/2628008747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32fd60 0x5620fe334860 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000bcb0 tx=0x7f8ef00107a0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.272+0000 7f8f10fe4740 1 -- 172.21.15.86:0/2628008747 shutdown_connections 2024-04-17T07:31:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.272+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/2628008747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe334ed0 0x5620fe337360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.272+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/2628008747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32fd60 0x5620fe334860 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.272+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/2628008747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe32f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.272+0000 7f8f10fe4740 1 -- 172.21.15.86:0/2628008747 >> 172.21.15.86:0/2628008747 conn(0x5620fe32a8e0 msgr2=0x5620fe32ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.272+0000 7f8f06a00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8eec0a3c30 0x7f8eec0a60f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8005d90 tx=0x7f8ef8005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.273+0000 7f8f10fe4740 1 -- 172.21.15.86:0/2628008747 shutdown_connections 2024-04-17T07:31:04.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.273+0000 7f8f10fe4740 1 -- 172.21.15.86:0/2628008747 wait complete. 2024-04-17T07:31:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.273+0000 7f8f10fe4740 1 -- start start 2024-04-17T07:31:04.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.274+0000 7f8f10fe4740 1 Processor -- start 2024-04-17T07:31:04.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.275+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32f370 0x5620fe3420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.275+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32fd60 0x5620fe342600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.276+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe334ed0 0x5620fe349640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.276+0000 7f8f07400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32f370 0x5620fe3420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.276+0000 7f8f07400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32f370 0x5620fe3420c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58454/0 (socket says 172.21.15.86:58454) 2024-04-17T07:31:04.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.276+0000 7f8f07400640 1 -- 172.21.15.86:0/3691660341 learned_addr learned my addr 172.21.15.86:0/3691660341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.276+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5620fe340690 con 0x5620fe32fd60 2024-04-17T07:31:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.276+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5620fe24f710 con 0x5620fe32f370 2024-04-17T07:31:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.276+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5620fe340810 con 0x5620fe334ed0 2024-04-17T07:31:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.276+0000 7f8f07e00640 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe334ed0 0x5620fe349640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.277+0000 7f8f10fe4740 10 client.0 _ll_register_callbacks cb 0x5620fe24aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:04.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.277+0000 7f8f06a00640 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32fd60 0x5620fe342600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.278+0000 7f8f07400640 1 -- 172.21.15.86:0/3691660341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe334ed0 msgr2=0x5620fe349640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.278+0000 7f8f07400640 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe334ed0 0x5620fe349640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.278+0000 7f8f07400640 1 -- 172.21.15.86:0/3691660341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32fd60 msgr2=0x5620fe342600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.278+0000 7f8f07400640 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32fd60 0x5620fe342600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.278+0000 7f8f07400640 1 -- 172.21.15.86:0/3691660341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620fdfa83f0 con 0x5620fe32f370 2024-04-17T07:31:04.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.278+0000 7f8f07e00640 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe334ed0 0x5620fe349640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.278+0000 7f8f07400640 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32f370 0x5620fe3420c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0005890 tx=0x7f8ef0011670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4a46121753b9261 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.279+0000 7f8ef4c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:31:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.279+0000 7f8ef4c00640 1 -- 172.21.15.86:0/3691660341 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0005040 con 0x5620fe32f370 2024-04-17T07:31:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.279+0000 7f8ef4c00640 1 -- 172.21.15.86:0/3691660341 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8ef00051e0 con 0x5620fe32f370 2024-04-17T07:31:04.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.279+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef00118f0 con 0x5620fe32f370 2024-04-17T07:31:04.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.280+0000 7f8ef4c00640 1 -- 172.21.15.86:0/3691660341 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ef0032050 con 0x5620fe32f370 2024-04-17T07:31:04.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.280+0000 7f8f10fe4740 10 client.38303 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:04.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.280+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620fdf680f0 con 0x5620fe32f370 2024-04-17T07:31:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.282+0000 7f8ef4c00640 1 -- 172.21.15.86:0/3691660341 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ef00118f0 con 0x5620fe32f370 2024-04-17T07:31:04.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.283+0000 7f8ef4c00640 1 -- 172.21.15.86:0/3691660341 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8ef002d030 con 0x5620fe32f370 2024-04-17T07:31:04.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.284+0000 7f8f10fe4740 10 client.38303 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:04.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.284+0000 7f8f10fe4740 10 client.38303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:04.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.284+0000 7f8f10fe4740 20 client.38303 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:04.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.284+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5620fe34db10 0x5620fe34ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.284+0000 7f8f10fe4740 4 client.38303 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.284+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620fe2a3b40 con 0x5620fe34db10 2024-04-17T07:31:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.285+0000 7f8f06a00640 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5620fe34db10 0x5620fe34ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.286+0000 7f8f06a00640 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5620fe34db10 0x5620fe34ff20 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524bd1461602ab3f server_cookie=783634be7cae154 in_seq=0 out_seq=0 2024-04-17T07:31:04.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.286+0000 7f8ef4c00640 10 client.38303 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:04.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.290+0000 7f8ef4c00640 1 -- 172.21.15.86:0/3691660341 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620fe2a3b40 con 0x5620fe34db10 2024-04-17T07:31:04.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.290+0000 7f8ef4c00640 10 client.38303 handle_command_reply: tid=0 2024-04-17T07:31:04.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.290+0000 7f8f10fe4740 1 client.38303 shutdown 2024-04-17T07:31:04.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.292+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32f370 msgr2=0x5620fe3420c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:04.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.292+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32f370 0x5620fe3420c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0005890 tx=0x7f8ef0011670 comp rx=0 tx=0).stop 2024-04-17T07:31:04.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.294+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 shutdown_connections 2024-04-17T07:31:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.294+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5620fe34db10 0x5620fe34ff20 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.294+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe334ed0 0x5620fe349640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.294+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32fd60 0x5620fe342600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.294+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/3691660341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe32f370 0x5620fe3420c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.294+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 >> 172.21.15.86:0/3691660341 conn(0x5620fe32a8e0 msgr2=0x5620fe329ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.294+0000 7f8ef4c00640 0 client.38303 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.295+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 shutdown_connections 2024-04-17T07:31:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.295+0000 7f8f10fe4740 1 -- 172.21.15.86:0/3691660341 wait complete. 2024-04-17T07:31:04.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.295+0000 7f8f10fe4740 20 client.38303 trim_cache size 0 max 16384 2024-04-17T07:31:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.346+0000 7f8f10fe4740 1 Processor -- start 2024-04-17T07:31:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.347+0000 7f8f10fe4740 1 -- start start 2024-04-17T07:31:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.347+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe354820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.347+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe32fd60 0x5620fe349560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.347+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe354d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.347+0000 7f8f10fe4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee400fc20 con 0x5620fe32f370 2024-04-17T07:31:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.347+0000 7f8f10fe4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4011760 con 0x5620fe334ed0 2024-04-17T07:31:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.347+0000 7f8f10fe4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee40102f0 con 0x5620fe32fd60 2024-04-17T07:31:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.348+0000 7f8f07400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe354820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.348+0000 7f8f07e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe354d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.348+0000 7f8f07400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe354820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47338/0 (socket says 172.21.15.86:47338) 2024-04-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.348+0000 7f8f07e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe354d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:58462/0 (socket says 172.21.15.86:58462) 2024-04-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.348+0000 7f8f07e00640 1 -- 172.21.15.86:0/2949590150 learned_addr learned my addr 172.21.15.86:0/2949590150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.348+0000 7f8f06a00640 1 --2- 172.21.15.86:0/2949590150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe32fd60 0x5620fe349560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.349+0000 7f8f07e00640 1 -- 172.21.15.86:0/2949590150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe32fd60 msgr2=0x5620fe349560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.349+0000 7f8f07e00640 1 --2- 172.21.15.86:0/2949590150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe32fd60 0x5620fe349560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.349+0000 7f8f07e00640 1 -- 172.21.15.86:0/2949590150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 msgr2=0x5620fe354820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.349+0000 7f8f07e00640 1 --2- 172.21.15.86:0/2949590150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe354820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.349+0000 7f8f07e00640 1 -- 172.21.15.86:0/2949590150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620fe32b650 con 0x5620fe334ed0 2024-04-17T07:31:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.349+0000 7f8f07400640 1 --2- 172.21.15.86:0/2949590150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe354820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:04.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.350+0000 7f8f07e00640 1 --2- 172.21.15.86:0/2949590150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe354d60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009430 tx=0x7f8f00032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7da16375e68deb38 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.350+0000 7f8ef7400640 1 -- 172.21.15.86:0/2949590150 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00035e20 con 0x5620fe334ed0 2024-04-17T07:31:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.350+0000 7f8ef7400640 1 -- 172.21.15.86:0/2949590150 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f000bed60 con 0x5620fe334ed0 2024-04-17T07:31:04.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.350+0000 7f8ef7400640 1 -- 172.21.15.86:0/2949590150 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f000f2520 con 0x5620fe334ed0 2024-04-17T07:31:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.352+0000 7f8f10fe4740 1 -- 172.21.15.86:0/2949590150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 msgr2=0x5620fe354d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:04.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.352+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/2949590150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe354d60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009430 tx=0x7f8f00032dc0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.353+0000 7f8f10fe4740 1 -- 172.21.15.86:0/2949590150 shutdown_connections 2024-04-17T07:31:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.353+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/2949590150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe354d60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.353+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/2949590150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe32fd60 0x5620fe349560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.353+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/2949590150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe354820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.353+0000 7f8f10fe4740 1 -- 172.21.15.86:0/2949590150 >> 172.21.15.86:0/2949590150 conn(0x5620fe32a8e0 msgr2=0x5620fe32e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.353+0000 7f8f10fe4740 1 -- 172.21.15.86:0/2949590150 shutdown_connections 2024-04-17T07:31:04.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.353+0000 7f8f10fe4740 1 -- 172.21.15.86:0/2949590150 wait complete. 2024-04-17T07:31:04.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.354+0000 7f8f10fe4740 1 -- start start 2024-04-17T07:31:04.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.354+0000 7f8f10fe4740 1 Processor -- start 2024-04-17T07:31:04.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.355+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe3466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.356+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe32fd60 0x5620fe346c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.356+0000 7f8f10fe4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe347bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.356+0000 7f8f10fe4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4011090 con 0x5620fe32f370 2024-04-17T07:31:04.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.356+0000 7f8f10fe4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee40109c0 con 0x5620fe334ed0 2024-04-17T07:31:04.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.356+0000 7f8f10fe4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee400dbf0 con 0x5620fe32fd60 2024-04-17T07:31:04.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.356+0000 7f8f10fe4740 10 client.0 _ll_register_callbacks cb 0x7f8ef000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:04.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.356+0000 7f8f07400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe3466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.357+0000 7f8f07400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe3466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47348/0 (socket says 172.21.15.86:47348) 2024-04-17T07:31:04.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.357+0000 7f8f07400640 1 -- 172.21.15.86:0/473351863 learned_addr learned my addr 172.21.15.86:0/473351863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:04.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.357+0000 7f8f07e00640 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe347bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.357+0000 7f8f07400640 1 -- 172.21.15.86:0/473351863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe32fd60 msgr2=0x5620fe346c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:04.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.358+0000 7f8f07400640 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe32fd60 0x5620fe346c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.358+0000 7f8f07400640 1 -- 172.21.15.86:0/473351863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 msgr2=0x5620fe347bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:04.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.358+0000 7f8f07400640 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe347bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.358+0000 7f8f07400640 1 -- 172.21.15.86:0/473351863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620fe3481e0 con 0x5620fe32f370 2024-04-17T07:31:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.358+0000 7f8f07e00640 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe347bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.358+0000 7f8f07400640 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe3466f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000b4e0 tx=0x7f8ef0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e812eb7ec4110b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.358+0000 7f8ef4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.359+0000 7f8ef4c00640 1 -- 172.21.15.86:0/473351863 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef001a440 con 0x5620fe32f370 2024-04-17T07:31:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.359+0000 7f8ef4c00640 1 -- 172.21.15.86:0/473351863 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8ef00116e0 con 0x5620fe32f370 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.359+0000 7f8f10fe4740 1 -- 172.21.15.86:0/473351863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f000bed60 con 0x5620fe32f370 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.359+0000 7f8ef4c00640 1 -- 172.21.15.86:0/473351863 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ef00118e0 con 0x5620fe32f370 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.359+0000 7f8f10fe4740 10 client.19329 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.359+0000 7f8f10fe4740 1 -- 172.21.15.86:0/473351863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620fe3494f0 con 0x5620fe32f370 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.359+0000 7f8ef4c00640 1 -- 172.21.15.86:0/473351863 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f000bed60 con 0x5620fe32f370 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.360+0000 7f8ef4c00640 1 -- 172.21.15.86:0/473351863 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8ef002d040 con 0x5620fe32f370 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.360+0000 7f8f10fe4740 10 client.19329 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.360+0000 7f8f10fe4740 10 client.19329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.360+0000 7f8f10fe4740 20 client.19329 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.360+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5620fe3b44e0 0x5620fe3b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.360+0000 7f8f10fe4740 4 client.19329 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:31:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.360+0000 7f8f10fe4740 1 -- 172.21.15.86:0/473351863 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620fe29d9b0 con 0x5620fe3b44e0 2024-04-17T07:31:04.362 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 700.0 s, 493.2 tps, lat 202.376 ms stddev 56.643 2024-04-17T07:31:04.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.360+0000 7f8f06a00640 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5620fe3b44e0 0x5620fe3b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:04.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.361+0000 7f8ef4c00640 10 client.19329 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.363+0000 7f8f06a00640 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5620fe3b44e0 0x5620fe3b6900 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7527ce3bbc0c11c3 server_cookie=9673644a89af82e7 in_seq=0 out_seq=0 2024-04-17T07:31:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8ef4c00640 1 -- 172.21.15.86:0/473351863 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5620fe29d9b0 con 0x5620fe3b44e0 2024-04-17T07:31:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8ef4c00640 10 client.19329 handle_command_reply: tid=0 2024-04-17T07:31:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8f10fe4740 1 client.19329 shutdown 2024-04-17T07:31:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8f10fe4740 1 -- 172.21.15.86:0/473351863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 msgr2=0x5620fe3466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:04.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe3466f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000b4e0 tx=0x7f8ef0005c50 comp rx=0 tx=0).stop 2024-04-17T07:31:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8f10fe4740 1 -- 172.21.15.86:0/473351863 shutdown_connections 2024-04-17T07:31:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5620fe3b44e0 0x5620fe3b6900 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5620fe334ed0 0x5620fe347bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5620fe32fd60 0x5620fe346c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8f10fe4740 1 --2- 172.21.15.86:0/473351863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620fe32f370 0x5620fe3466f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8f10fe4740 1 -- 172.21.15.86:0/473351863 >> 172.21.15.86:0/473351863 conn(0x5620fe32a8e0 msgr2=0x5620fe329250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:04.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.364+0000 7f8ef4c00640 0 client.19329 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:04.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.365+0000 7f8f10fe4740 1 -- 172.21.15.86:0/473351863 shutdown_connections 2024-04-17T07:31:04.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.365+0000 7f8f10fe4740 1 -- 172.21.15.86:0/473351863 wait complete. 2024-04-17T07:31:04.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.365+0000 7f8f10fe4740 20 client.19329 trim_cache size 0 max 16384 2024-04-17T07:31:04.368 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:31:04.368 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:31:04.368 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:31:04.368 INFO:teuthology.orchestra.run.smithi086.stdout: "39a63fd3-c052-461b-9259-ffd877e03773": { 2024-04-17T07:31:04.368 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:31:04.368 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "39a63fd3-c052-461b-9259-ffd877e03773", 2024-04-17T07:31:04.368 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:31:04.368 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:31:04.368 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:31:04.369 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:31:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.368+0000 7f8f0cc00640 1 -- 172.21.15.86:0/3839468907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8eec0a3c30 msgr2=0x7f8eec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.369+0000 7f8f0cc00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8eec0a3c30 0x7f8eec0a60f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8005d90 tx=0x7f8ef8005d20 comp rx=0 tx=0).stop 2024-04-17T07:31:04.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.369+0000 7f8f0cc00640 1 -- 172.21.15.86:0/3839468907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f08105c50 msgr2=0x7f8f081a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.369+0000 7f8f0cc00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f08105c50 0x7f8f081a17d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8f00037040 tx=0x7f8f00003ba0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.370+0000 7f8f0cc00640 1 -- 172.21.15.86:0/3839468907 shutdown_connections 2024-04-17T07:31:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.370+0000 7f8f0cc00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8eec0a3c30 0x7f8eec0a60f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.370+0000 7f8f0cc00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f08105c50 0x7f8f081a17d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.370+0000 7f8f0cc00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f08105290 0x7f8f0819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.370+0000 7f8f0cc00640 1 --2- 172.21.15.86:0/3839468907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f081040a0 0x7f8f0819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:04.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.370+0000 7f8f0cc00640 1 -- 172.21.15.86:0/3839468907 >> 172.21.15.86:0/3839468907 conn(0x7f8f080ff610 msgr2=0x7f8f08108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:04.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.371+0000 7f8f0cc00640 1 -- 172.21.15.86:0/3839468907 shutdown_connections 2024-04-17T07:31:04.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:04.372+0000 7f8f0cc00640 1 -- 172.21.15.86:0/3839468907 wait complete. 2024-04-17T07:31:04.411 INFO:tasks.cephfs.filesystem:scrub status for tag:39a63fd3-c052-461b-9259-ffd877e03773 - {'path': '/', 'tag': '39a63fd3-c052-461b-9259-ffd877e03773', 'options': 'recursive,force'} 2024-04-17T07:31:05.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:05 smithi098 ceph-mon[35418]: pgmap v941: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 633 op/s 2024-04-17T07:31:05.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:05 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:31:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:05 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:05.505 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:05 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:31:05.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:05 smithi103 ceph-mon[35495]: pgmap v941: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 633 op/s 2024-04-17T07:31:05.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:05 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:31:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:05 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:05.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:05 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:31:05.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:05 smithi086 ceph-mon[31733]: pgmap v941: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 633 op/s 2024-04-17T07:31:05.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:05 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:31:05.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:05 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:05.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:05 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:31:06.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:06 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:06.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:06 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:06.586 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:06 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:07.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:07 smithi098 ceph-mon[35418]: pgmap v942: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-04-17T07:31:07.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:07 smithi103 ceph-mon[35495]: pgmap v942: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-04-17T07:31:07.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:07 smithi086 ceph-mon[31733]: pgmap v942: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-04-17T07:31:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 705.0 s, 487.0 tps, lat 204.561 ms stddev 104.252 2024-04-17T07:31:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:09 smithi098 ceph-mon[35418]: pgmap v943: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 605 op/s 2024-04-17T07:31:09.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:09 smithi103 ceph-mon[35495]: pgmap v943: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 605 op/s 2024-04-17T07:31:09.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:09 smithi086 ceph-mon[31733]: pgmap v943: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 605 op/s 2024-04-17T07:31:10.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:10 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:31:10.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:10 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:31:10.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:10 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:31:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:11 smithi098 ceph-mon[35418]: pgmap v944: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 600 op/s 2024-04-17T07:31:11.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:11 smithi103 ceph-mon[35495]: pgmap v944: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 600 op/s 2024-04-17T07:31:11.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:11 smithi086 ceph-mon[31733]: pgmap v944: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 MiB/s wr, 600 op/s 2024-04-17T07:31:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:13 smithi098 ceph-mon[35418]: pgmap v945: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 657 op/s 2024-04-17T07:31:13.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:13 smithi103 ceph-mon[35495]: pgmap v945: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 657 op/s 2024-04-17T07:31:13.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:13 smithi086 ceph-mon[31733]: pgmap v945: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.7 MiB/s wr, 657 op/s 2024-04-17T07:31:14.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 710.0 s, 502.4 tps, lat 200.721 ms stddev 109.085 2024-04-17T07:31:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:15 smithi098 ceph-mon[35418]: pgmap v946: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 663 op/s 2024-04-17T07:31:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:15 smithi103 ceph-mon[35495]: pgmap v946: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 663 op/s 2024-04-17T07:31:15.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:15 smithi086 ceph-mon[31733]: pgmap v946: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.4 MiB/s wr, 663 op/s 2024-04-17T07:31:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:17 smithi098 ceph-mon[35418]: pgmap v947: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.2 MiB/s wr, 624 op/s 2024-04-17T07:31:17.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:17 smithi103 ceph-mon[35495]: pgmap v947: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.2 MiB/s wr, 624 op/s 2024-04-17T07:31:17.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:17 smithi086 ceph-mon[31733]: pgmap v947: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.2 MiB/s wr, 624 op/s 2024-04-17T07:31:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 715.0 s, 431.8 tps, lat 230.914 ms stddev 320.296 2024-04-17T07:31:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:19 smithi098 ceph-mon[35418]: pgmap v948: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.5 MiB/s wr, 710 op/s 2024-04-17T07:31:19.529 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:19 smithi103 ceph-mon[35495]: pgmap v948: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.5 MiB/s wr, 710 op/s 2024-04-17T07:31:19.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:19 smithi086 ceph-mon[31733]: pgmap v948: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.5 MiB/s wr, 710 op/s 2024-04-17T07:31:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:21 smithi098 ceph-mon[35418]: pgmap v949: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 758 op/s 2024-04-17T07:31:21.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:21 smithi103 ceph-mon[35495]: pgmap v949: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 758 op/s 2024-04-17T07:31:21.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:21 smithi086 ceph-mon[31733]: pgmap v949: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 11 MiB/s wr, 758 op/s 2024-04-17T07:31:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:23 smithi098 ceph-mon[35418]: pgmap v950: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 12 MiB/s wr, 797 op/s 2024-04-17T07:31:23.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:23 smithi103 ceph-mon[35495]: pgmap v950: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 12 MiB/s wr, 797 op/s 2024-04-17T07:31:23.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:23 smithi086 ceph-mon[31733]: pgmap v950: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 12 MiB/s wr, 797 op/s 2024-04-17T07:31:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 720.0 s, 683.6 tps, lat 147.271 ms stddev 72.762 2024-04-17T07:31:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:25 smithi098 ceph-mon[35418]: pgmap v951: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 12 MiB/s wr, 767 op/s 2024-04-17T07:31:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:25 smithi103 ceph-mon[35495]: pgmap v951: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 12 MiB/s wr, 767 op/s 2024-04-17T07:31:25.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:25 smithi086 ceph-mon[31733]: pgmap v951: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 12 MiB/s wr, 767 op/s 2024-04-17T07:31:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:27 smithi098 ceph-mon[35418]: pgmap v952: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 745 op/s 2024-04-17T07:31:27.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:27 smithi103 ceph-mon[35495]: pgmap v952: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 745 op/s 2024-04-17T07:31:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:27 smithi086 ceph-mon[31733]: pgmap v952: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 745 op/s 2024-04-17T07:31:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 725.0 s, 686.4 tps, lat 144.740 ms stddev 53.946 2024-04-17T07:31:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:29 smithi098 ceph-mon[35418]: pgmap v953: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 15 MiB/s wr, 806 op/s 2024-04-17T07:31:29.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:29 smithi103 ceph-mon[35495]: pgmap v953: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 15 MiB/s wr, 806 op/s 2024-04-17T07:31:29.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:29 smithi086 ceph-mon[31733]: pgmap v953: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 15 MiB/s wr, 806 op/s 2024-04-17T07:31:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:31 smithi098 ceph-mon[35418]: pgmap v954: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 14 MiB/s wr, 762 op/s 2024-04-17T07:31:31.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:31 smithi103 ceph-mon[35495]: pgmap v954: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 14 MiB/s wr, 762 op/s 2024-04-17T07:31:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:31 smithi086 ceph-mon[31733]: pgmap v954: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 14 MiB/s wr, 762 op/s 2024-04-17T07:31:33.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:33 smithi098 ceph-mon[35418]: pgmap v955: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 751 op/s 2024-04-17T07:31:33.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:33 smithi103 ceph-mon[35495]: pgmap v955: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 751 op/s 2024-04-17T07:31:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:33 smithi086 ceph-mon[31733]: pgmap v955: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 751 op/s 2024-04-17T07:31:34.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 730.0 s, 503.4 tps, lat 199.193 ms stddev 148.700 2024-04-17T07:31:34.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:31:34.412 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:31:34.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.587+0000 7f03d6000640 1 -- 172.21.15.86:0/2765728676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d0101fb0 msgr2=0x7f03d01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:34.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.587+0000 7f03d6000640 1 --2- 172.21.15.86:0/2765728676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d0101fb0 0x7f03d01023d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009ca0 tx=0x7f03c802c0f0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.588+0000 7f03d6000640 1 -- 172.21.15.86:0/2765728676 shutdown_connections 2024-04-17T07:31:34.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.588+0000 7f03d6000640 1 --2- 172.21.15.86:0/2765728676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d0103b60 0x7f03d010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.588+0000 7f03d6000640 1 --2- 172.21.15.86:0/2765728676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03d01031a0 0x7f03d0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.588+0000 7f03d6000640 1 --2- 172.21.15.86:0/2765728676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d0101fb0 0x7f03d01023d0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.588+0000 7f03d6000640 1 -- 172.21.15.86:0/2765728676 >> 172.21.15.86:0/2765728676 conn(0x7f03d00fd540 msgr2=0x7f03d00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.588+0000 7f03d6000640 1 -- 172.21.15.86:0/2765728676 shutdown_connections 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.588+0000 7f03d6000640 1 -- 172.21.15.86:0/2765728676 wait complete. 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.589+0000 7f03d6000640 1 Processor -- start 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.589+0000 7f03d6000640 1 -- start start 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03d6000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d0101fb0 0x7f03d01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03d6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d01031a0 0x7f03d01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03d6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03d0103b60 0x7f03d019f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03d6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03d010dbb0 con 0x7f03d01031a0 2024-04-17T07:31:34.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03d6000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03d010da30 con 0x7f03d0101fb0 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03d6000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03d010dd30 con 0x7f03d0103b60 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d01031a0 0x7f03d01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d01031a0 0x7f03d01988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56726/0 (socket says 172.21.15.86:56726) 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03cea00640 1 -- 172.21.15.86:0/1716678525 learned_addr learned my addr 172.21.15.86:0/1716678525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03cf400640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d0101fb0 0x7f03d01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03cea00640 1 -- 172.21.15.86:0/1716678525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03d0103b60 msgr2=0x7f03d019f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03cea00640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03d0103b60 0x7f03d019f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03cea00640 1 -- 172.21.15.86:0/1716678525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d0101fb0 msgr2=0x7f03d01983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03cea00640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d0101fb0 0x7f03d01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.590+0000 7f03cea00640 1 -- 172.21.15.86:0/1716678525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d019ffc0 con 0x7f03d01031a0 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.591+0000 7f03cea00640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d01031a0 0x7f03d01988e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f03b800b4a0 tx=0x7f03b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.591+0000 7f03c7e00640 1 -- 172.21.15.86:0/1716678525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8014070 con 0x7f03d01031a0 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.591+0000 7f03c7e00640 1 -- 172.21.15.86:0/1716678525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b8010ac0 con 0x7f03d01031a0 2024-04-17T07:31:34.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.591+0000 7f03d6000640 1 -- 172.21.15.86:0/1716678525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d01a02b0 con 0x7f03d01031a0 2024-04-17T07:31:34.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.591+0000 7f03c7e00640 1 -- 172.21.15.86:0/1716678525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8017c30 con 0x7f03d01031a0 2024-04-17T07:31:34.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.591+0000 7f03d6000640 1 -- 172.21.15.86:0/1716678525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d01a0880 con 0x7f03d01031a0 2024-04-17T07:31:34.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.594+0000 7f03d8dea740 1 Processor -- start 2024-04-17T07:31:34.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.594+0000 7f03c7e00640 1 -- 172.21.15.86:0/1716678525 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f03b8017070 con 0x7f03d01031a0 2024-04-17T07:31:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.596+0000 7f03d8dea740 1 -- start start 2024-04-17T07:31:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.596+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 0x55e8f94ea3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.596+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94ea980 0x55e8f94eef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.596+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f94f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.596+0000 7f03d8dea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8f9448120 con 0x55e8f94e9f90 2024-04-17T07:31:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.596+0000 7f03d8dea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e8f93ee340 con 0x55e8f94ea980 2024-04-17T07:31:34.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.596+0000 7f03d8dea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e8f94440e0 con 0x55e8f94ef580 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.597+0000 7f03cf400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94ea980 0x55e8f94eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.597+0000 7f03cf400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94ea980 0x55e8f94eef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:59028/0 (socket says 172.21.15.86:59028) 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.597+0000 7f03cf400640 1 -- 172.21.15.86:0/97346486 learned_addr learned my addr 172.21.15.86:0/97346486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.597+0000 7f03c7e00640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03b40a3cb0 0x7f03b40a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.597+0000 7f03cfe00640 1 --2- 172.21.15.86:0/97346486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 0x55e8f94ea3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.598+0000 7f03cf400640 1 -- 172.21.15.86:0/97346486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 msgr2=0x55e8f94f19f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.598+0000 7f03cf400640 1 --2- 172.21.15.86:0/97346486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f94f19f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.598+0000 7f03cf400640 1 -- 172.21.15.86:0/97346486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 msgr2=0x55e8f94ea3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.598+0000 7f03cf400640 1 --2- 172.21.15.86:0/97346486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 0x55e8f94ea3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.598+0000 7f03cf400640 1 -- 172.21.15.86:0/97346486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f92247d0 con 0x55e8f94ea980 2024-04-17T07:31:34.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.598+0000 7f03cfe00640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03b40a3cb0 0x7f03b40a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.598+0000 7f03c7e00640 1 -- 172.21.15.86:0/1716678525 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f03b80cfbd0 con 0x7f03d01031a0 2024-04-17T07:31:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.599+0000 7f03cf400640 1 --2- 172.21.15.86:0/97346486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94ea980 0x55e8f94eef10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009bf0 tx=0x7f03c8009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=612e45e545260f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:34.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.599+0000 7f03bfe00640 1 -- 172.21.15.86:0/97346486 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8039050 con 0x55e8f94ea980 2024-04-17T07:31:34.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.599+0000 7f03cfe00640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03b40a3cb0 0x7f03b40a6170 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55e8f94e0340 tx=0x7f03c00047d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:34.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.599+0000 7f03bfe00640 1 -- 172.21.15.86:0/97346486 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03c803a040 con 0x55e8f94ea980 2024-04-17T07:31:34.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.600+0000 7f03bfe00640 1 -- 172.21.15.86:0/97346486 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8004040 con 0x55e8f94ea980 2024-04-17T07:31:34.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.602+0000 7f03d8dea740 1 -- 172.21.15.86:0/97346486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94ea980 msgr2=0x55e8f94eef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:34.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.602+0000 7f03d8dea740 1 --2- 172.21.15.86:0/97346486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94ea980 0x55e8f94eef10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009bf0 tx=0x7f03c8009870 comp rx=0 tx=0).stop 2024-04-17T07:31:34.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.603+0000 7f03d8dea740 1 -- 172.21.15.86:0/97346486 shutdown_connections 2024-04-17T07:31:34.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.603+0000 7f03d8dea740 1 --2- 172.21.15.86:0/97346486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f94f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.603+0000 7f03d8dea740 1 --2- 172.21.15.86:0/97346486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94ea980 0x55e8f94eef10 secure :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009bf0 tx=0x7f03c8009870 comp rx=0 tx=0).stop 2024-04-17T07:31:34.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.603+0000 7f03d8dea740 1 --2- 172.21.15.86:0/97346486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 0x55e8f94ea3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.603+0000 7f03d8dea740 1 -- 172.21.15.86:0/97346486 >> 172.21.15.86:0/97346486 conn(0x55e8f94dc680 msgr2=0x55e8f94dca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.606+0000 7f03d8dea740 1 -- 172.21.15.86:0/97346486 shutdown_connections 2024-04-17T07:31:34.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.607+0000 7f03d8dea740 1 -- 172.21.15.86:0/97346486 wait complete. 2024-04-17T07:31:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.607+0000 7f03d8dea740 1 -- start start 2024-04-17T07:31:34.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.608+0000 7f03d8dea740 1 Processor -- start 2024-04-17T07:31:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.608+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 0x55e8f94e3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.608+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94ef580 0x55e8f94e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.608+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94e8a70 0x55e8f94e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.608+0000 7f03d8dea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8f94e0970 con 0x55e8f94ef580 2024-04-17T07:31:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.608+0000 7f03d8dea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e8f9401710 con 0x55e8f94e9f90 2024-04-17T07:31:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.608+0000 7f03d8dea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e8f94e0af0 con 0x55e8f94e8a70 2024-04-17T07:31:34.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.608+0000 7f03d8dea740 10 client.0 _ll_register_callbacks cb 0x55e8f93fcaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:34.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.609+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94e8a70 0x55e8f94e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.610+0000 7f03cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94e8a70 0x55e8f94e8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47156/0 (socket says 172.21.15.86:47156) 2024-04-17T07:31:34.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.610+0000 7f03cfe00640 1 -- 172.21.15.86:0/874828259 learned_addr learned my addr 172.21.15.86:0/874828259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:34.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.610+0000 7f03cf400640 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 0x55e8f94e3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.610+0000 7f03cea00640 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94ef580 0x55e8f94e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.611+0000 7f03cfe00640 1 -- 172.21.15.86:0/874828259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 msgr2=0x55e8f94e3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:34.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.611+0000 7f03cfe00640 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 0x55e8f94e3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.611+0000 7f03cfe00640 1 -- 172.21.15.86:0/874828259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94ef580 msgr2=0x55e8f94e3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:34.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.611+0000 7f03cfe00640 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94ef580 0x55e8f94e3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.611+0000 7f03cfe00640 1 -- 172.21.15.86:0/874828259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f915a3f0 con 0x55e8f94e8a70 2024-04-17T07:31:34.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.611+0000 7f03cf400640 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 0x55e8f94e3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:34.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.611+0000 7f03cea00640 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94ef580 0x55e8f94e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:31:34.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.612+0000 7f03bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:31:34.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.612+0000 7f03cfe00640 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94e8a70 0x55e8f94e8ef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f03c000cf70 tx=0x7f03c0004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=662bad47e24d5e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:34.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.613+0000 7f03bcc00640 1 -- 172.21.15.86:0/874828259 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0022060 con 0x55e8f94e8a70 2024-04-17T07:31:34.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.613+0000 7f03bcc00640 1 -- 172.21.15.86:0/874828259 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f03c0010760 con 0x55e8f94e8a70 2024-04-17T07:31:34.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.616+0000 7f03d8dea740 1 -- 172.21.15.86:0/874828259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c803a040 con 0x55e8f94e8a70 2024-04-17T07:31:34.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.616+0000 7f03bcc00640 1 -- 172.21.15.86:0/874828259 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03c0010960 con 0x55e8f94e8a70 2024-04-17T07:31:34.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.617+0000 7f03d8dea740 10 client.28351 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:34.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.617+0000 7f03d8dea740 1 -- 172.21.15.86:0/874828259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e8f911a0f0 con 0x55e8f94e8a70 2024-04-17T07:31:34.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.618+0000 7f03bcc00640 1 -- 172.21.15.86:0/874828259 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03c803a040 con 0x55e8f94e8a70 2024-04-17T07:31:34.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.618+0000 7f03bcc00640 1 -- 172.21.15.86:0/874828259 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f03c001db60 con 0x55e8f94e8a70 2024-04-17T07:31:34.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.618+0000 7f03d8dea740 10 client.28351 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:34.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.618+0000 7f03d8dea740 10 client.28351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.618+0000 7f03d8dea740 20 client.28351 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.618+0000 7f03d8dea740 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e8f9508aa0 0x55e8f950af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.618+0000 7f03d8dea740 4 client.28351 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.618+0000 7f03d8dea740 1 -- 172.21.15.86:0/874828259 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8f9455b40 con 0x55e8f9508aa0 2024-04-17T07:31:34.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.620+0000 7f03cf400640 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e8f9508aa0 0x55e8f950af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.622+0000 7f03cf400640 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e8f9508aa0 0x55e8f950af20 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e62136bf84a4209 server_cookie=74a4dd7994fef183 in_seq=0 out_seq=0 2024-04-17T07:31:34.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.622+0000 7f03bcc00640 10 client.28351 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:34.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03bcc00640 1 -- 172.21.15.86:0/874828259 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8f9455b40 con 0x55e8f9508aa0 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03bcc00640 10 client.28351 handle_command_reply: tid=0 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03d8dea740 1 client.28351 shutdown 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03d8dea740 1 -- 172.21.15.86:0/874828259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94e8a70 msgr2=0x55e8f94e8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03d8dea740 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94e8a70 0x55e8f94e8ef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f03c000cf70 tx=0x7f03c0004840 comp rx=0 tx=0).stop 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03d8dea740 1 -- 172.21.15.86:0/874828259 shutdown_connections 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03d8dea740 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e8f9508aa0 0x55e8f950af20 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03d8dea740 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94e8a70 0x55e8f94e8ef0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03d8dea740 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94ef580 0x55e8f94e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03d8dea740 1 --2- 172.21.15.86:0/874828259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 0x55e8f94e3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03d8dea740 1 -- 172.21.15.86:0/874828259 >> 172.21.15.86:0/874828259 conn(0x55e8f94dc680 msgr2=0x55e8f94db890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.624+0000 7f03bcc00640 0 client.28351 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:34.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.625+0000 7f03d8dea740 1 -- 172.21.15.86:0/874828259 shutdown_connections 2024-04-17T07:31:34.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.625+0000 7f03d8dea740 1 -- 172.21.15.86:0/874828259 wait complete. 2024-04-17T07:31:34.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.625+0000 7f03d8dea740 20 client.28351 trim_cache size 0 max 16384 2024-04-17T07:31:34.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.660+0000 7f03d8dea740 1 Processor -- start 2024-04-17T07:31:34.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.661+0000 7f03d8dea740 1 -- start start 2024-04-17T07:31:34.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.661+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e8a70 0x55e8f94ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.661+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 0x55e8f94ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.661+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f94e1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.661+0000 7f03d8dea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01a8b80 con 0x55e8f94e9f90 2024-04-17T07:31:34.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.661+0000 7f03d8dea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01aa6c0 con 0x55e8f94e8a70 2024-04-17T07:31:34.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.661+0000 7f03d8dea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01a9250 con 0x55e8f94ef580 2024-04-17T07:31:34.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.662+0000 7f03cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 0x55e8f94ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.662+0000 7f03cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 0x55e8f94ed030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56754/0 (socket says 172.21.15.86:56754) 2024-04-17T07:31:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.662+0000 7f03cea00640 1 -- 172.21.15.86:0/641638787 learned_addr learned my addr 172.21.15.86:0/641638787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.663+0000 7f03cf400640 1 --2- 172.21.15.86:0/641638787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e8a70 0x55e8f94ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.663+0000 7f03cfe00640 1 --2- 172.21.15.86:0/641638787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f94e1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.663+0000 7f03cea00640 1 -- 172.21.15.86:0/641638787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 msgr2=0x55e8f94e1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.663+0000 7f03cea00640 1 --2- 172.21.15.86:0/641638787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f94e1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.663+0000 7f03cea00640 1 -- 172.21.15.86:0/641638787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e8a70 msgr2=0x55e8f94ecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.663+0000 7f03cea00640 1 --2- 172.21.15.86:0/641638787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e8a70 0x55e8f94ecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.663+0000 7f03cea00640 1 -- 172.21.15.86:0/641638787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f950b5a0 con 0x55e8f94e9f90 2024-04-17T07:31:34.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.664+0000 7f03cea00640 1 --2- 172.21.15.86:0/641638787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 0x55e8f94ed030 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f03b8097f30 tx=0x7f03b800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9592d5255f85af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.664+0000 7f03bf400640 1 -- 172.21.15.86:0/641638787 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b801e070 con 0x55e8f94e9f90 2024-04-17T07:31:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.665+0000 7f03bf400640 1 -- 172.21.15.86:0/641638787 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b80a4740 con 0x55e8f94e9f90 2024-04-17T07:31:34.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.665+0000 7f03bf400640 1 -- 172.21.15.86:0/641638787 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b801e070 con 0x55e8f94e9f90 2024-04-17T07:31:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.666+0000 7f03d8dea740 1 -- 172.21.15.86:0/641638787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 msgr2=0x55e8f94ed030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.666+0000 7f03d8dea740 1 --2- 172.21.15.86:0/641638787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 0x55e8f94ed030 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f03b8097f30 tx=0x7f03b800bf90 comp rx=0 tx=0).stop 2024-04-17T07:31:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.667+0000 7f03d8dea740 1 -- 172.21.15.86:0/641638787 shutdown_connections 2024-04-17T07:31:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.667+0000 7f03d8dea740 1 --2- 172.21.15.86:0/641638787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f94e1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.667+0000 7f03d8dea740 1 --2- 172.21.15.86:0/641638787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e9f90 0x55e8f94ed030 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.667+0000 7f03d8dea740 1 --2- 172.21.15.86:0/641638787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e8a70 0x55e8f94ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.667+0000 7f03d8dea740 1 -- 172.21.15.86:0/641638787 >> 172.21.15.86:0/641638787 conn(0x55e8f94dc680 msgr2=0x55e8f94e3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.668+0000 7f03d8dea740 1 -- 172.21.15.86:0/641638787 shutdown_connections 2024-04-17T07:31:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.668+0000 7f03d8dea740 1 -- 172.21.15.86:0/641638787 wait complete. 2024-04-17T07:31:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.668+0000 7f03d8dea740 1 -- start start 2024-04-17T07:31:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.668+0000 7f03d8dea740 1 Processor -- start 2024-04-17T07:31:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.669+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e8a70 0x55e8f9519a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.669+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 0x55e8f9519f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.669+0000 7f03d8dea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f951af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.669+0000 7f03d8dea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01a9ff0 con 0x55e8f94e8a70 2024-04-17T07:31:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.669+0000 7f03d8dea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01a9920 con 0x55e8f94e9f90 2024-04-17T07:31:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.669+0000 7f03d8dea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03d01a6b50 con 0x55e8f94ef580 2024-04-17T07:31:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.669+0000 7f03d8dea740 10 client.0 _ll_register_callbacks cb 0x7f03c0014040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.669+0000 7f03cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e8a70 0x55e8f9519a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.670+0000 7f03cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e8a70 0x55e8f9519a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56760/0 (socket says 172.21.15.86:56760) 2024-04-17T07:31:34.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.670+0000 7f03cf400640 1 -- 172.21.15.86:0/3800792343 learned_addr learned my addr 172.21.15.86:0/3800792343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.670+0000 7f03cea00640 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 0x55e8f9519f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.670+0000 7f03cf400640 1 -- 172.21.15.86:0/3800792343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 msgr2=0x55e8f951af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.670+0000 7f03cfe00640 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f951af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.670+0000 7f03cf400640 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f951af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.670+0000 7f03cf400640 1 -- 172.21.15.86:0/3800792343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 msgr2=0x55e8f9519f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.670+0000 7f03cf400640 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 0x55e8f9519f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.670+0000 7f03cf400640 1 -- 172.21.15.86:0/3800792343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f951b540 con 0x55e8f94e8a70 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.671+0000 7f03cfe00640 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f951af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.671+0000 7f03cf400640 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e8a70 0x55e8f9519a00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f03c802c630 tx=0x7f03c802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13ddce903f06d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.671+0000 7f03bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.671+0000 7f03bcc00640 1 -- 172.21.15.86:0/3800792343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8055030 con 0x55e8f94e8a70 2024-04-17T07:31:34.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.671+0000 7f03d8dea740 1 -- 172.21.15.86:0/3800792343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b80a4740 con 0x55e8f94e8a70 2024-04-17T07:31:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.671+0000 7f03bcc00640 1 -- 172.21.15.86:0/3800792343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f03c8002030 con 0x55e8f94e8a70 2024-04-17T07:31:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.672+0000 7f03bcc00640 1 -- 172.21.15.86:0/3800792343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03c8039020 con 0x55e8f94e8a70 2024-04-17T07:31:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.672+0000 7f03d8dea740 10 client.19344 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.672+0000 7f03d8dea740 1 -- 172.21.15.86:0/3800792343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e8f956eb40 con 0x55e8f94e8a70 2024-04-17T07:31:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.672+0000 7f03bcc00640 1 -- 172.21.15.86:0/3800792343 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b80a4740 con 0x55e8f94e8a70 2024-04-17T07:31:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.672+0000 7f03bcc00640 1 -- 172.21.15.86:0/3800792343 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f03c8040450 con 0x55e8f94e8a70 2024-04-17T07:31:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.673+0000 7f03d8dea740 10 client.19344 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.673+0000 7f03d8dea740 10 client.19344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.673+0000 7f03d8dea740 20 client.19344 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.673+0000 7f03d8dea740 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e8f956f110 0x55e8f9571500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.673+0000 7f03d8dea740 4 client.19344 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:31:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.673+0000 7f03d8dea740 1 -- 172.21.15.86:0/3800792343 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8f944f9b0 con 0x55e8f956f110 2024-04-17T07:31:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.673+0000 7f03cea00640 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e8f956f110 0x55e8f9571500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.674+0000 7f03bcc00640 10 client.19344 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:34.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.676+0000 7f03cea00640 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e8f956f110 0x55e8f9571500 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963eab81ffe37f3c server_cookie=9056156cc5db581b in_seq=0 out_seq=0 2024-04-17T07:31:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.677+0000 7f03bcc00640 1 -- 172.21.15.86:0/3800792343 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e8f944f9b0 con 0x55e8f956f110 2024-04-17T07:31:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.677+0000 7f03bcc00640 10 client.19344 handle_command_reply: tid=0 2024-04-17T07:31:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.677+0000 7f03d8dea740 1 client.19344 shutdown 2024-04-17T07:31:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.678+0000 7f03d8dea740 1 -- 172.21.15.86:0/3800792343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e8a70 msgr2=0x55e8f9519a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:34.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.678+0000 7f03d8dea740 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e8a70 0x55e8f9519a00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f03c802c630 tx=0x7f03c802c0f0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.678+0000 7f03d8dea740 1 -- 172.21.15.86:0/3800792343 shutdown_connections 2024-04-17T07:31:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.678+0000 7f03d8dea740 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e8f956f110 0x55e8f9571500 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.678+0000 7f03d8dea740 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8f94ef580 0x55e8f951af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.678+0000 7f03d8dea740 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e8f94e9f90 0x55e8f9519f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.678+0000 7f03d8dea740 1 --2- 172.21.15.86:0/3800792343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8f94e8a70 0x55e8f9519a00 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.678+0000 7f03d8dea740 1 -- 172.21.15.86:0/3800792343 >> 172.21.15.86:0/3800792343 conn(0x55e8f94dc680 msgr2=0x55e8f94e3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:34.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.678+0000 7f03bcc00640 0 client.19344 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.679+0000 7f03d8dea740 1 -- 172.21.15.86:0/3800792343 shutdown_connections 2024-04-17T07:31:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.679+0000 7f03d8dea740 1 -- 172.21.15.86:0/3800792343 wait complete. 2024-04-17T07:31:34.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.679+0000 7f03d8dea740 20 client.19344 trim_cache size 0 max 16384 2024-04-17T07:31:34.680 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:31:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:31:34.680 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:31:34.680 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:31:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.680+0000 7f03d6000640 1 -- 172.21.15.86:0/1716678525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03b40a3cb0 msgr2=0x7f03b40a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.680+0000 7f03d6000640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03b40a3cb0 0x7f03b40a6170 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55e8f94e0340 tx=0x7f03c00047d0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.680+0000 7f03d6000640 1 -- 172.21.15.86:0/1716678525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d01031a0 msgr2=0x7f03d01988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.680+0000 7f03d6000640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d01031a0 0x7f03d01988e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f03b800b4a0 tx=0x7f03b800b960 comp rx=0 tx=0).stop 2024-04-17T07:31:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.680+0000 7f03d6000640 1 -- 172.21.15.86:0/1716678525 shutdown_connections 2024-04-17T07:31:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.680+0000 7f03d6000640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03b40a3cb0 0x7f03b40a6170 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.680+0000 7f03d6000640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03d0103b60 0x7f03d019f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.680+0000 7f03d6000640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03d01031a0 0x7f03d01988e0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.680+0000 7f03d6000640 1 --2- 172.21.15.86:0/1716678525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03d0101fb0 0x7f03d01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.680+0000 7f03d6000640 1 -- 172.21.15.86:0/1716678525 >> 172.21.15.86:0/1716678525 conn(0x7f03d00fd540 msgr2=0x7f03d0106660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.681+0000 7f03d6000640 1 -- 172.21.15.86:0/1716678525 shutdown_connections 2024-04-17T07:31:34.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:34.681+0000 7f03d6000640 1 -- 172.21.15.86:0/1716678525 wait complete. 2024-04-17T07:31:34.708 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:31:34.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:31:35.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.014+0000 7f5455400640 1 -- 172.21.15.86:0/3269852532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54480945e0 msgr2=0x7f5448096a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:35.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.014+0000 7f5455400640 1 --2- 172.21.15.86:0/3269852532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54480945e0 0x7f5448096a10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5444009900 tx=0x7f544402bc60 comp rx=0 tx=0).stop 2024-04-17T07:31:35.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.015+0000 7f5455400640 1 -- 172.21.15.86:0/3269852532 shutdown_connections 2024-04-17T07:31:35.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.015+0000 7f5455400640 1 --2- 172.21.15.86:0/3269852532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5448099900 0x7f5448099d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:35.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.015+0000 7f5455400640 1 --2- 172.21.15.86:0/3269852532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5448096f50 0x7f54480993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:35.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.015+0000 7f5455400640 1 --2- 172.21.15.86:0/3269852532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54480945e0 0x7f5448096a10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:35.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.015+0000 7f5455400640 1 -- 172.21.15.86:0/3269852532 >> 172.21.15.86:0/3269852532 conn(0x7f54480901c0 msgr2=0x7f5448092620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:35.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.016+0000 7f5455400640 1 -- 172.21.15.86:0/3269852532 shutdown_connections 2024-04-17T07:31:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.017+0000 7f5455400640 1 -- 172.21.15.86:0/3269852532 wait complete. 2024-04-17T07:31:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.018+0000 7f5455400640 1 Processor -- start 2024-04-17T07:31:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.019+0000 7f5455400640 1 -- start start 2024-04-17T07:31:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.019+0000 7f5455400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54480945e0 0x7f544809fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.019+0000 7f5455400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5448096f50 0x7f54480a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.020+0000 7f5455400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5448099900 0x7f5448142620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.020+0000 7f5455400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54480a7f10 con 0x7f54480945e0 2024-04-17T07:31:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.020+0000 7f5455400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f54480a7d90 con 0x7f5448099900 2024-04-17T07:31:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.020+0000 7f5455400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54480a8090 con 0x7f5448096f50 2024-04-17T07:31:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.020+0000 7f544f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5448096f50 0x7f54480a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.020+0000 7f544f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5448096f50 0x7f54480a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47182/0 (socket says 172.21.15.86:47182) 2024-04-17T07:31:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.020+0000 7f544f400640 1 -- 172.21.15.86:0/2917113974 learned_addr learned my addr 172.21.15.86:0/2917113974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:35.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.020+0000 7f5454a00640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5448099900 0x7f5448142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.020+0000 7f544f400640 1 -- 172.21.15.86:0/2917113974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5448099900 msgr2=0x7f5448142620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.020+0000 7f544f400640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5448099900 0x7f5448142620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.021+0000 7f544f400640 1 -- 172.21.15.86:0/2917113974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54480945e0 msgr2=0x7f544809fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.021+0000 7f544f400640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54480945e0 0x7f544809fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:35.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.021+0000 7f544f400640 1 -- 172.21.15.86:0/2917113974 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5448142b60 con 0x7f5448096f50 2024-04-17T07:31:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.021+0000 7f544f400640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5448096f50 0x7f54480a0090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f543c00d950 tx=0x7f543c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:35.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.024+0000 7f544cc00640 1 -- 172.21.15.86:0/2917113974 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c017070 con 0x7f5448096f50 2024-04-17T07:31:35.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.024+0000 7f544cc00640 1 -- 172.21.15.86:0/2917113974 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f543c009c30 con 0x7f5448096f50 2024-04-17T07:31:35.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.024+0000 7f544cc00640 1 -- 172.21.15.86:0/2917113974 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c00f620 con 0x7f5448096f50 2024-04-17T07:31:35.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.026+0000 7f5455400640 1 -- 172.21.15.86:0/2917113974 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5448142d30 con 0x7f5448096f50 2024-04-17T07:31:35.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.026+0000 7f5455400640 1 -- 172.21.15.86:0/2917113974 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5448143140 con 0x7f5448096f50 2024-04-17T07:31:35.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.027+0000 7f5455400640 1 -- 172.21.15.86:0/2917113974 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5448091cb0 con 0x7f5448096f50 2024-04-17T07:31:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.033+0000 7f544cc00640 1 -- 172.21.15.86:0/2917113974 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f543c0109a0 con 0x7f5448096f50 2024-04-17T07:31:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.035+0000 7f544cc00640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54380a3c80 0x7f54380a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.037+0000 7f544cc00640 1 -- 172.21.15.86:0/2917113974 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f543c0d08a0 con 0x7f5448096f50 2024-04-17T07:31:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.037+0000 7f544cc00640 1 -- 172.21.15.86:0/2917113974 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f543c020070 con 0x7f5448096f50 2024-04-17T07:31:35.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.037+0000 7f544fe00640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54380a3c80 0x7f54380a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:35.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.046+0000 7f544fe00640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54380a3c80 0x7f54380a6140 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f54440096c0 tx=0x7f5444000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:35.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:35 smithi103 ceph-mon[35495]: pgmap v956: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 741 op/s 2024-04-17T07:31:35.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:35 smithi086 ceph-mon[31733]: pgmap v956: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 741 op/s 2024-04-17T07:31:35.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.619+0000 7f5455400640 1 -- 172.21.15.86:0/2917113974 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54480022f0 con 0x7f5448096f50 2024-04-17T07:31:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.621+0000 7f544cc00640 1 -- 172.21.15.86:0/2917113974 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f543c009de0 con 0x7f5448096f50 2024-04-17T07:31:35.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:31:35.623 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:31:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:31:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.625+0000 7f5455400640 1 -- 172.21.15.86:0/2917113974 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54380a3c80 msgr2=0x7f54380a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.626+0000 7f5455400640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54380a3c80 0x7f54380a6140 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f54440096c0 tx=0x7f5444000bc0 comp rx=0 tx=0).stop 2024-04-17T07:31:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.626+0000 7f5455400640 1 -- 172.21.15.86:0/2917113974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5448096f50 msgr2=0x7f54480a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.626+0000 7f5455400640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5448096f50 0x7f54480a0090 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f543c00d950 tx=0x7f543c00de10 comp rx=0 tx=0).stop 2024-04-17T07:31:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.626+0000 7f5455400640 1 -- 172.21.15.86:0/2917113974 shutdown_connections 2024-04-17T07:31:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.627+0000 7f5455400640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f54380a3c80 0x7f54380a6140 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.627+0000 7f5455400640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5448099900 0x7f5448142620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.627+0000 7f5455400640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5448096f50 0x7f54480a0090 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.627+0000 7f5455400640 1 --2- 172.21.15.86:0/2917113974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54480945e0 0x7f544809fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.627+0000 7f5455400640 1 -- 172.21.15.86:0/2917113974 >> 172.21.15.86:0/2917113974 conn(0x7f54480901c0 msgr2=0x7f54480978f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.627+0000 7f5455400640 1 -- 172.21.15.86:0/2917113974 shutdown_connections 2024-04-17T07:31:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:35.627+0000 7f5455400640 1 -- 172.21.15.86:0/2917113974 wait complete. 2024-04-17T07:31:35.666 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:31:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:35 smithi098 ceph-mon[35418]: pgmap v956: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 741 op/s 2024-04-17T07:31:36.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.029+0000 7fdaf8600640 1 -- 172.21.15.86:0/199635008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdaf010bd50 msgr2=0x7fdaf0114bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.029+0000 7fdaf8600640 1 --2- 172.21.15.86:0/199635008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdaf010bd50 0x7fdaf0114bf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdae80134c0 tx=0x7fdae8034320 comp rx=0 tx=0).stop 2024-04-17T07:31:36.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.029+0000 7fdaf8600640 1 -- 172.21.15.86:0/199635008 shutdown_connections 2024-04-17T07:31:36.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.029+0000 7fdaf8600640 1 --2- 172.21.15.86:0/199635008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdaf010bd50 0x7fdaf0114bf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.029+0000 7fdaf8600640 1 --2- 172.21.15.86:0/199635008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaf010b8d0 0x7fdaf01126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.029+0000 7fdaf8600640 1 --2- 172.21.15.86:0/199635008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0108f60 0x7fdaf010b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.029+0000 7fdaf8600640 1 -- 172.21.15.86:0/199635008 >> 172.21.15.86:0/199635008 conn(0x7fdaf006edf0 msgr2=0x7fdaf0071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.029+0000 7fdaf8600640 1 -- 172.21.15.86:0/199635008 shutdown_connections 2024-04-17T07:31:36.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.030+0000 7fdaf8600640 1 -- 172.21.15.86:0/199635008 wait complete. 2024-04-17T07:31:36.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.031+0000 7fdaf8600640 1 Processor -- start 2024-04-17T07:31:36.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.032+0000 7fdaf8600640 1 -- start start 2024-04-17T07:31:36.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.033+0000 7fdaf8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0108f60 0x7fdaf019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.033+0000 7fdaf8600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaf010b8d0 0x7fdaf019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.033+0000 7fdaf8600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdaf010bd50 0x7fdaf01a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.033+0000 7fdaf8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf010edb0 con 0x7fdaf0108f60 2024-04-17T07:31:36.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.033+0000 7fdaf8600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf010ec30 con 0x7fdaf010bd50 2024-04-17T07:31:36.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.033+0000 7fdaf8600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf010ef30 con 0x7fdaf010b8d0 2024-04-17T07:31:36.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.033+0000 7fdaf6600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdaf010bd50 0x7fdaf01a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.033+0000 7fdaf6600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdaf010bd50 0x7fdaf01a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:59070/0 (socket says 172.21.15.86:59070) 2024-04-17T07:31:36.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.033+0000 7fdaf6600640 1 -- 172.21.15.86:0/3815954353 learned_addr learned my addr 172.21.15.86:0/3815954353 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.034+0000 7fdaf6600640 1 -- 172.21.15.86:0/3815954353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaf010b8d0 msgr2=0x7fdaf019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.034+0000 7fdaf5200640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaf010b8d0 0x7fdaf019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.034+0000 7fdaf6600640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaf010b8d0 0x7fdaf019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.034+0000 7fdaf6600640 1 -- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0108f60 msgr2=0x7fdaf019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.034+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0108f60 0x7fdaf019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.035+0000 7fdaf6600640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0108f60 0x7fdaf019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.035+0000 7fdaf6600640 1 -- 172.21.15.86:0/3815954353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf01a6430 con 0x7fdaf010bd50 2024-04-17T07:31:36.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.035+0000 7fdaf5200640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaf010b8d0 0x7fdaf019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:36.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.035+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0108f60 0x7fdaf019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:36.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.035+0000 7fdaf6600640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdaf010bd50 0x7fdaf01a5dc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdae8034860 tx=0x7fdae803cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.036+0000 7fdaee000640 1 -- 172.21.15.86:0/3815954353 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8049020 con 0x7fdaf010bd50 2024-04-17T07:31:36.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.036+0000 7fdaee000640 1 -- 172.21.15.86:0/3815954353 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdae8003d80 con 0x7fdaf010bd50 2024-04-17T07:31:36.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.036+0000 7fdaee000640 1 -- 172.21.15.86:0/3815954353 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80045b0 con 0x7fdaf010bd50 2024-04-17T07:31:36.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.036+0000 7fdaf8600640 1 -- 172.21.15.86:0/3815954353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf01a66c0 con 0x7fdaf010bd50 2024-04-17T07:31:36.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.036+0000 7fdaf8600640 1 -- 172.21.15.86:0/3815954353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf01a6bc0 con 0x7fdaf010bd50 2024-04-17T07:31:36.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.041+0000 7fdafb3fb740 1 Processor -- start 2024-04-17T07:31:36.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.041+0000 7fdaee000640 1 -- 172.21.15.86:0/3815954353 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fdae804c020 con 0x7fdaf010bd50 2024-04-17T07:31:36.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.042+0000 7fdafb3fb740 1 -- start start 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.042+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x5620355243c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.042+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035524990 0x562035529490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.042+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 0x56203552bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.042+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56203548a020 con 0x562035524990 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.042+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562035430460 con 0x562035529b00 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.042+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562035485fe0 con 0x562035523fa0 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.043+0000 7fdaf5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x5620355243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.043+0000 7fdaf5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x5620355243c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47208/0 (socket says 172.21.15.86:47208) 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.043+0000 7fdaf5200640 1 -- 172.21.15.86:0/428684437 learned_addr learned my addr 172.21.15.86:0/428684437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.043+0000 7fdaf5200640 1 --2- 172.21.15.86:0/428684437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 0x56203552bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.043+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/428684437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035524990 0x562035529490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.043+0000 7fdaf5200640 1 -- 172.21.15.86:0/428684437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 msgr2=0x56203552bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.043+0000 7fdaf5200640 1 --2- 172.21.15.86:0/428684437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 0x56203552bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.043+0000 7fdaf5200640 1 -- 172.21.15.86:0/428684437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035524990 msgr2=0x562035529490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.043+0000 7fdaf5200640 1 --2- 172.21.15.86:0/428684437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035524990 0x562035529490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.043+0000 7fdaf5200640 1 -- 172.21.15.86:0/428684437 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562035438040 con 0x562035523fa0 2024-04-17T07:31:36.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.044+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/428684437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035524990 0x562035529490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:36.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.044+0000 7fdaf5200640 1 --2- 172.21.15.86:0/428684437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x5620355243c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdadc014900 tx=0x7fdadc009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d65fedab11f4bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.044+0000 7fdae2a00640 1 -- 172.21.15.86:0/428684437 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc005020 con 0x562035523fa0 2024-04-17T07:31:36.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.044+0000 7fdae2a00640 1 -- 172.21.15.86:0/428684437 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdadc0047a0 con 0x562035523fa0 2024-04-17T07:31:36.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.045+0000 7fdae2a00640 1 -- 172.21.15.86:0/428684437 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc00f9c0 con 0x562035523fa0 2024-04-17T07:31:36.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.046+0000 7fdaee000640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdad80a3c30 0x7fdad80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.046+0000 7fdaee000640 1 -- 172.21.15.86:0/3815954353 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdae80fae00 con 0x7fdaf010bd50 2024-04-17T07:31:36.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.047+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdad80a3c30 0x7fdad80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.048+0000 7fdafb3fb740 1 -- 172.21.15.86:0/428684437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 msgr2=0x5620355243c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.048+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/428684437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x5620355243c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdadc014900 tx=0x7fdadc009650 comp rx=0 tx=0).stop 2024-04-17T07:31:36.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.048+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdad80a3c30 0x7fdad80a60f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdae4005fd0 tx=0x7fdae4005e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.048+0000 7fdafb3fb740 1 -- 172.21.15.86:0/428684437 shutdown_connections 2024-04-17T07:31:36.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.048+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/428684437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 0x56203552bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.048+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/428684437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035524990 0x562035529490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.048+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/428684437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x5620355243c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.048+0000 7fdafb3fb740 1 -- 172.21.15.86:0/428684437 >> 172.21.15.86:0/428684437 conn(0x56203551f510 msgr2=0x56203551f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.049+0000 7fdafb3fb740 1 -- 172.21.15.86:0/428684437 shutdown_connections 2024-04-17T07:31:36.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.050+0000 7fdafb3fb740 1 -- 172.21.15.86:0/428684437 wait complete. 2024-04-17T07:31:36.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.052+0000 7fdafb3fb740 1 -- start start 2024-04-17T07:31:36.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.053+0000 7fdafb3fb740 1 Processor -- start 2024-04-17T07:31:36.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 0x56203552eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035529b00 0x562035535b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56203552cab0 con 0x562035529b00 2024-04-17T07:31:36.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562035443830 con 0x562035524990 2024-04-17T07:31:36.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56203552cc30 con 0x562035523fa0 2024-04-17T07:31:36.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdafb3fb740 10 client.0 _ll_register_callbacks cb 0x562035411fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:36.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdaf5c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdaf5c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47210/0 (socket says 172.21.15.86:47210) 2024-04-17T07:31:36.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdaf5c00640 1 -- 172.21.15.86:0/3448320775 learned_addr learned my addr 172.21.15.86:0/3448320775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.054+0000 7fdaf5200640 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 0x56203552eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.055+0000 7fdaf5c00640 1 -- 172.21.15.86:0/3448320775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 msgr2=0x56203552eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.055+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 0x56203552eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.055+0000 7fdaf5c00640 1 -- 172.21.15.86:0/3448320775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035529b00 msgr2=0x562035535b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.055+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035529b00 0x562035535b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.055+0000 7fdaf5c00640 1 -- 172.21.15.86:0/3448320775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562035447920 con 0x562035523fa0 2024-04-17T07:31:36.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.057+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e5a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009fd0 tx=0x7fdae40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=377d437fa3c0bf38 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.057+0000 7fdad7400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:31:36.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.057+0000 7fdad7400640 1 -- 172.21.15.86:0/3448320775 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae400ccf0 con 0x562035523fa0 2024-04-17T07:31:36.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.057+0000 7fdad7400640 1 -- 172.21.15.86:0/3448320775 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdae401c420 con 0x562035523fa0 2024-04-17T07:31:36.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.058+0000 7fdafb3fb740 1 -- 172.21.15.86:0/3448320775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdadc0047a0 con 0x562035523fa0 2024-04-17T07:31:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.058+0000 7fdad7400640 1 -- 172.21.15.86:0/3448320775 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdae402d050 con 0x562035523fa0 2024-04-17T07:31:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.058+0000 7fdafb3fb740 10 client.28369 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.058+0000 7fdafb3fb740 1 -- 172.21.15.86:0/3448320775 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620354157f0 con 0x562035523fa0 2024-04-17T07:31:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.059+0000 7fdad7400640 1 -- 172.21.15.86:0/3448320775 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdadc0047a0 con 0x562035523fa0 2024-04-17T07:31:36.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.060+0000 7fdad7400640 1 -- 172.21.15.86:0/3448320775 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fdae4024660 con 0x562035523fa0 2024-04-17T07:31:36.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.060+0000 7fdafb3fb740 10 client.28369 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:36.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.060+0000 7fdafb3fb740 10 client.28369 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:31:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.060+0000 7fdafb3fb740 20 client.28369 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.060+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56203554b0f0 0x56203554d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.060+0000 7fdafb3fb740 4 client.28369 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.060+0000 7fdafb3fb740 1 -- 172.21.15.86:0/3448320775 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562035497a40 con 0x56203554b0f0 2024-04-17T07:31:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.061+0000 7fdaf5200640 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56203554b0f0 0x56203554d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.063+0000 7fdaf5200640 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56203554b0f0 0x56203554d590 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76539117f768fa8e server_cookie=c774e3caf85e16c3 in_seq=0 out_seq=0 2024-04-17T07:31:36.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.063+0000 7fdad7400640 10 client.28369 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:31:36.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.064+0000 7fdad7400640 1 -- 172.21.15.86:0/3448320775 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562035497a40 con 0x56203554b0f0 2024-04-17T07:31:36.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.064+0000 7fdad7400640 10 client.28369 handle_command_reply: tid=0 2024-04-17T07:31:36.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.064+0000 7fdafb3fb740 1 client.28369 shutdown 2024-04-17T07:31:36.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.065+0000 7fdafb3fb740 1 -- 172.21.15.86:0/3448320775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 msgr2=0x56203552e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.065+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e5a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009fd0 tx=0x7fdae40046c0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.066+0000 7fdafb3fb740 1 -- 172.21.15.86:0/3448320775 shutdown_connections 2024-04-17T07:31:36.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.066+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x56203554b0f0 0x56203554d590 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.066+0000 7fdad7400640 0 client.28369 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:31:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.066+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035529b00 0x562035535b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.066+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 0x56203552eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.066+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/3448320775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e5a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.066+0000 7fdafb3fb740 1 -- 172.21.15.86:0/3448320775 >> 172.21.15.86:0/3448320775 conn(0x56203551f510 msgr2=0x56203551e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.066+0000 7fdafb3fb740 1 -- 172.21.15.86:0/3448320775 shutdown_connections 2024-04-17T07:31:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.067+0000 7fdafb3fb740 1 -- 172.21.15.86:0/3448320775 wait complete. 2024-04-17T07:31:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.067+0000 7fdafb3fb740 20 client.28369 trim_cache size 0 max 16384 2024-04-17T07:31:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.099+0000 7fdafb3fb740 1 Processor -- start 2024-04-17T07:31:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.099+0000 7fdafb3fb740 1 -- start start 2024-04-17T07:31:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.100+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.100+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 0x562035535ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.100+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035529b00 0x56203552e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.100+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b10a0 con 0x562035529b00 2024-04-17T07:31:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.100+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b2be0 con 0x562035524990 2024-04-17T07:31:36.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.100+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b1770 con 0x562035523fa0 2024-04-17T07:31:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.100+0000 7fdaf5c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.101+0000 7fdaf5c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47212/0 (socket says 172.21.15.86:47212) 2024-04-17T07:31:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.101+0000 7fdaf5c00640 1 -- 172.21.15.86:0/436541323 learned_addr learned my addr 172.21.15.86:0/436541323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.101+0000 7fdaf5200640 1 --2- 172.21.15.86:0/436541323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 0x562035535ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.101+0000 7fdaf6600640 1 --2- 172.21.15.86:0/436541323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035529b00 0x56203552e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.101+0000 7fdaf5200640 1 -- 172.21.15.86:0/436541323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 msgr2=0x56203552e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.101+0000 7fdaf5200640 1 --2- 172.21.15.86:0/436541323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.101+0000 7fdaf5200640 1 -- 172.21.15.86:0/436541323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035529b00 msgr2=0x56203552e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.103+0000 7fdaf5200640 1 --2- 172.21.15.86:0/436541323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035529b00 0x56203552e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.103+0000 7fdaf5200640 1 -- 172.21.15.86:0/436541323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56203519c3f0 con 0x562035524990 2024-04-17T07:31:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.103+0000 7fdaf6600640 1 --2- 172.21.15.86:0/436541323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035529b00 0x56203552e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:31:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.103+0000 7fdaf5200640 1 --2- 172.21.15.86:0/436541323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 0x562035535ae0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdadc004fd0 tx=0x7fdadc009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81618b97d0a3308c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.104+0000 7fdae2000640 1 -- 172.21.15.86:0/436541323 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc005000 con 0x562035524990 2024-04-17T07:31:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.105+0000 7fdae2000640 1 -- 172.21.15.86:0/436541323 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdadc004760 con 0x562035524990 2024-04-17T07:31:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.105+0000 7fdae2000640 1 -- 172.21.15.86:0/436541323 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc005000 con 0x562035524990 2024-04-17T07:31:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.107+0000 7fdafb3fb740 1 -- 172.21.15.86:0/436541323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 msgr2=0x562035535ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.107+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/436541323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 0x562035535ae0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdadc004fd0 tx=0x7fdadc009f90 comp rx=0 tx=0).stop 2024-04-17T07:31:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.107+0000 7fdafb3fb740 1 -- 172.21.15.86:0/436541323 shutdown_connections 2024-04-17T07:31:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.107+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/436541323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035529b00 0x56203552e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.107+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/436541323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035524990 0x562035535ae0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.107+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/436541323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035523fa0 0x56203552e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.107+0000 7fdafb3fb740 1 -- 172.21.15.86:0/436541323 >> 172.21.15.86:0/436541323 conn(0x56203551f510 msgr2=0x562035525f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.107+0000 7fdafb3fb740 1 -- 172.21.15.86:0/436541323 shutdown_connections 2024-04-17T07:31:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.107+0000 7fdafb3fb740 1 -- 172.21.15.86:0/436541323 wait complete. 2024-04-17T07:31:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.107+0000 7fdafb3fb740 1 -- start start 2024-04-17T07:31:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.108+0000 7fdafb3fb740 1 Processor -- start 2024-04-17T07:31:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.109+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035523fa0 0x562035532bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.109+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035524990 0x5620355330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.110+0000 7fdafb3fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 0x562035534040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.110+0000 7fdaf5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035524990 0x5620355330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.110+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b2510 con 0x562035523fa0 2024-04-17T07:31:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.110+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01b1e40 con 0x562035529b00 2024-04-17T07:31:36.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.110+0000 7fdafb3fb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf01af070 con 0x562035524990 2024-04-17T07:31:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.110+0000 7fdafb3fb740 10 client.0 _ll_register_callbacks cb 0x7fdae400f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.110+0000 7fdaf5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035524990 0x5620355330f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47218/0 (socket says 172.21.15.86:47218) 2024-04-17T07:31:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.110+0000 7fdaf5200640 1 -- 172.21.15.86:0/464743753 learned_addr learned my addr 172.21.15.86:0/464743753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.110+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035523fa0 0x562035532bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.110+0000 7fdaf6600640 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 0x562035534040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.111+0000 7fdaf6600640 1 -- 172.21.15.86:0/464743753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035524990 msgr2=0x5620355330f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.111+0000 7fdaf6600640 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035524990 0x5620355330f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.111+0000 7fdaf6600640 1 -- 172.21.15.86:0/464743753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035523fa0 msgr2=0x562035532bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.111+0000 7fdaf6600640 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035523fa0 0x562035532bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.111+0000 7fdaf6600640 1 -- 172.21.15.86:0/464743753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620352667d0 con 0x562035529b00 2024-04-17T07:31:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.112+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035523fa0 0x562035532bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.112+0000 7fdaf6600640 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 0x562035534040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdae80f8580 tx=0x7fdae80c68a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=632b880d7135a8b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.112+0000 7fdad7400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:31:36.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.113+0000 7fdad7400640 1 -- 172.21.15.86:0/464743753 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80c6d60 con 0x562035529b00 2024-04-17T07:31:36.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.113+0000 7fdad7400640 1 -- 172.21.15.86:0/464743753 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdae8004750 con 0x562035529b00 2024-04-17T07:31:36.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.113+0000 7fdafb3fb740 1 -- 172.21.15.86:0/464743753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdadc004760 con 0x562035529b00 2024-04-17T07:31:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.114+0000 7fdad7400640 1 -- 172.21.15.86:0/464743753 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdae80cb110 con 0x562035529b00 2024-04-17T07:31:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.114+0000 7fdafb3fb740 10 client.38345 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:36.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.114+0000 7fdafb3fb740 1 -- 172.21.15.86:0/464743753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620355357d0 con 0x562035529b00 2024-04-17T07:31:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.115+0000 7fdad7400640 1 -- 172.21.15.86:0/464743753 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdadc004760 con 0x562035529b00 2024-04-17T07:31:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.115+0000 7fdad7400640 1 -- 172.21.15.86:0/464743753 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fdae80b1040 con 0x562035529b00 2024-04-17T07:31:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.115+0000 7fdafb3fb740 10 client.38345 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.115+0000 7fdafb3fb740 10 client.38345 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:31:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.115+0000 7fdafb3fb740 20 client.38345 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.115+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5620355b1820 0x5620355b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.115+0000 7fdafb3fb740 4 client.38345 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:31:36.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.115+0000 7fdafb3fb740 1 -- 172.21.15.86:0/464743753 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620354918b0 con 0x5620355b1820 2024-04-17T07:31:36.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.116+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5620355b1820 0x5620355b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.117+0000 7fdaf5c00640 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5620355b1820 0x5620355b3c40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5451ef664d320438 server_cookie=b8abca154a178ad in_seq=0 out_seq=0 2024-04-17T07:31:36.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.117+0000 7fdad7400640 10 client.38345 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:31:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdad7400640 1 -- 172.21.15.86:0/464743753 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620354918b0 con 0x5620355b1820 2024-04-17T07:31:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdad7400640 10 client.38345 handle_command_reply: tid=0 2024-04-17T07:31:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdafb3fb740 1 client.38345 shutdown 2024-04-17T07:31:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdafb3fb740 1 -- 172.21.15.86:0/464743753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 msgr2=0x562035534040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 0x562035534040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdae80f8580 tx=0x7fdae80c68a0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdafb3fb740 1 -- 172.21.15.86:0/464743753 shutdown_connections 2024-04-17T07:31:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x5620355b1820 0x5620355b3c40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562035529b00 0x562035534040 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562035524990 0x5620355330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdad7400640 0 client.38345 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:31:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdafb3fb740 1 --2- 172.21.15.86:0/464743753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562035523fa0 0x562035532bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.118+0000 7fdafb3fb740 1 -- 172.21.15.86:0/464743753 >> 172.21.15.86:0/464743753 conn(0x56203551f510 msgr2=0x56203552e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.119+0000 7fdafb3fb740 1 -- 172.21.15.86:0/464743753 shutdown_connections 2024-04-17T07:31:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.119+0000 7fdafb3fb740 1 -- 172.21.15.86:0/464743753 wait complete. 2024-04-17T07:31:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.119+0000 7fdafb3fb740 20 client.38345 trim_cache size 0 max 16384 2024-04-17T07:31:36.120 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:31:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.120+0000 7fdaf8600640 1 -- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdad80a3c30 msgr2=0x7fdad80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.120+0000 7fdaf8600640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdad80a3c30 0x7fdad80a60f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdae4005fd0 tx=0x7fdae4005e60 comp rx=0 tx=0).stop 2024-04-17T07:31:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.120+0000 7fdaf8600640 1 -- 172.21.15.86:0/3815954353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdaf010bd50 msgr2=0x7fdaf01a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.120+0000 7fdaf8600640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdaf010bd50 0x7fdaf01a5dc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdae8034860 tx=0x7fdae803cf90 comp rx=0 tx=0).stop 2024-04-17T07:31:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.121+0000 7fdaf8600640 1 -- 172.21.15.86:0/3815954353 shutdown_connections 2024-04-17T07:31:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.121+0000 7fdaf8600640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fdad80a3c30 0x7fdad80a60f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.121+0000 7fdaf8600640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdaf010bd50 0x7fdaf01a5dc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.121+0000 7fdaf8600640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdaf010b8d0 0x7fdaf019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.121+0000 7fdaf8600640 1 --2- 172.21.15.86:0/3815954353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf0108f60 0x7fdaf019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.121+0000 7fdaf8600640 1 -- 172.21.15.86:0/3815954353 >> 172.21.15.86:0/3815954353 conn(0x7fdaf006edf0 msgr2=0x7fdaf0071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.121+0000 7fdaf8600640 1 -- 172.21.15.86:0/3815954353 shutdown_connections 2024-04-17T07:31:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.121+0000 7fdaf8600640 1 -- 172.21.15.86:0/3815954353 wait complete. 2024-04-17T07:31:36.148 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:31:36.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.446+0000 7f911b200640 1 -- 172.21.15.86:0/2401430121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9114100740 msgr2=0x7f9114100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.446+0000 7f911b200640 1 --2- 172.21.15.86:0/2401430121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9114100740 0x7f9114100b60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9108009900 tx=0x7f910802bc30 comp rx=0 tx=0).stop 2024-04-17T07:31:36.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.446+0000 7f911b200640 1 -- 172.21.15.86:0/2401430121 shutdown_connections 2024-04-17T07:31:36.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.446+0000 7f911b200640 1 --2- 172.21.15.86:0/2401430121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9114101a40 0x7f9114101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.446+0000 7f911b200640 1 --2- 172.21.15.86:0/2401430121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f91141010a0 0x7f9114101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.446+0000 7f911b200640 1 --2- 172.21.15.86:0/2401430121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9114100740 0x7f9114100b60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.446+0000 7f911b200640 1 -- 172.21.15.86:0/2401430121 >> 172.21.15.86:0/2401430121 conn(0x7f91140fc300 msgr2=0x7f91140fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.447+0000 7f911b200640 1 -- 172.21.15.86:0/2401430121 shutdown_connections 2024-04-17T07:31:36.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.447+0000 7f911b200640 1 -- 172.21.15.86:0/2401430121 wait complete. 2024-04-17T07:31:36.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.448+0000 7f911b200640 1 Processor -- start 2024-04-17T07:31:36.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.449+0000 7f911b200640 1 -- start start 2024-04-17T07:31:36.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.450+0000 7f911b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9114100740 0x7f91141061d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.450+0000 7f911b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141010a0 0x7f9114106710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.450+0000 7f911b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9114101a40 0x7f911410b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.450+0000 7f911b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9114103050 con 0x7f9114101a40 2024-04-17T07:31:36.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.450+0000 7f9119e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9114100740 0x7f91141061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.451+0000 7f9119e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9114100740 0x7f91141061d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47244/0 (socket says 172.21.15.86:47244) 2024-04-17T07:31:36.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.451+0000 7f9119e00640 1 -- 172.21.15.86:0/2185763347 learned_addr learned my addr 172.21.15.86:0/2185763347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.451+0000 7f911b200640 1 -- 172.21.15.86:0/2185763347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9114102ed0 con 0x7f91141010a0 2024-04-17T07:31:36.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.451+0000 7f911b200640 1 -- 172.21.15.86:0/2185763347 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91141031d0 con 0x7f9114100740 2024-04-17T07:31:36.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.451+0000 7f9119400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141010a0 0x7f9114106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.451+0000 7f911a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9114101a40 0x7f911410b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.452+0000 7f9119400640 1 -- 172.21.15.86:0/2185763347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9114100740 msgr2=0x7f91141061d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.452+0000 7f9119400640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9114100740 0x7f91141061d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.452+0000 7f9119400640 1 -- 172.21.15.86:0/2185763347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9114101a40 msgr2=0x7f911410b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.452+0000 7f9119400640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9114101a40 0x7f911410b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.452+0000 7f9119400640 1 -- 172.21.15.86:0/2185763347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911410bd90 con 0x7f91141010a0 2024-04-17T07:31:36.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.452+0000 7f9119e00640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9114100740 0x7f91141061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.453+0000 7f9119400640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141010a0 0x7f9114106710 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f910400d6d0 tx=0x7f910400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.453+0000 7f910ea00640 1 -- 172.21.15.86:0/2185763347 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104017070 con 0x7f91141010a0 2024-04-17T07:31:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.453+0000 7f911b200640 1 -- 172.21.15.86:0/2185763347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911410c080 con 0x7f91141010a0 2024-04-17T07:31:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.453+0000 7f910ea00640 1 -- 172.21.15.86:0/2185763347 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9104009820 con 0x7f91141010a0 2024-04-17T07:31:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.453+0000 7f910ea00640 1 -- 172.21.15.86:0/2185763347 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104020060 con 0x7f91141010a0 2024-04-17T07:31:36.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.454+0000 7f911b200640 1 -- 172.21.15.86:0/2185763347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911410c560 con 0x7f91141010a0 2024-04-17T07:31:36.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.458+0000 7f910ea00640 1 -- 172.21.15.86:0/2185763347 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f91040099d0 con 0x7f91141010a0 2024-04-17T07:31:36.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.459+0000 7f911f59d740 1 Processor -- start 2024-04-17T07:31:36.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.459+0000 7f911f59d740 1 -- start start 2024-04-17T07:31:36.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.459+0000 7f911f59d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 0x5632f6000580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.459+0000 7f911f59d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f6005650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.459+0000 7f911f59d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6005cc0 0x5632f6008150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.459+0000 7f911f59d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5632f5f66020 con 0x5632f6000b50 2024-04-17T07:31:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.459+0000 7f911f59d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5632f5f0c460 con 0x5632f6000160 2024-04-17T07:31:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.459+0000 7f911f59d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5632f5f61fe0 con 0x5632f6005cc0 2024-04-17T07:31:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.460+0000 7f911a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 0x5632f6000580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.460+0000 7f911a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 0x5632f6000580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:59142/0 (socket says 172.21.15.86:59142) 2024-04-17T07:31:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.460+0000 7f911a800640 1 -- 172.21.15.86:0/110907970 learned_addr learned my addr 172.21.15.86:0/110907970 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.461+0000 7f910ea00640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90fc0a3c30 0x7f90fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.461+0000 7f910ea00640 1 -- 172.21.15.86:0/2185763347 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f91040d0150 con 0x7f91141010a0 2024-04-17T07:31:36.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.461+0000 7f911a800640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90fc0a3c30 0x7f90fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.461+0000 7f911a800640 1 -- 172.21.15.86:0/110907970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6005cc0 msgr2=0x5632f6008150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:31:36.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.461+0000 7f911a800640 1 --2- 172.21.15.86:0/110907970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6005cc0 0x5632f6008150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.461+0000 7f911a800640 1 -- 172.21.15.86:0/110907970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 msgr2=0x5632f6005650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:31:36.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.461+0000 7f911a800640 1 --2- 172.21.15.86:0/110907970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f6005650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.461+0000 7f911a800640 1 -- 172.21.15.86:0/110907970 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632f5f14040 con 0x5632f6000160 2024-04-17T07:31:36.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.462+0000 7f911a800640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90fc0a3c30 0x7f90fc0a60f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9110014fd0 tx=0x7f911001b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.462+0000 7f911a800640 1 --2- 172.21.15.86:0/110907970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 0x5632f6000580 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9110014700 tx=0x7f9110014bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f799b8bbbbfb6a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.463+0000 7f9103000640 1 -- 172.21.15.86:0/110907970 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911001b7f0 con 0x5632f6000160 2024-04-17T07:31:36.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.464+0000 7f9103000640 1 -- 172.21.15.86:0/110907970 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f911001be40 con 0x5632f6000160 2024-04-17T07:31:36.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.464+0000 7f9103000640 1 -- 172.21.15.86:0/110907970 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110012c00 con 0x5632f6000160 2024-04-17T07:31:36.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.464+0000 7f911f59d740 1 -- 172.21.15.86:0/110907970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 msgr2=0x5632f6000580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.464+0000 7f911f59d740 1 --2- 172.21.15.86:0/110907970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 0x5632f6000580 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9110014700 tx=0x7f9110014bc0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.465+0000 7f911f59d740 1 -- 172.21.15.86:0/110907970 shutdown_connections 2024-04-17T07:31:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.465+0000 7f911f59d740 1 --2- 172.21.15.86:0/110907970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6005cc0 0x5632f6008150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.465+0000 7f911f59d740 1 --2- 172.21.15.86:0/110907970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f6005650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.465+0000 7f911f59d740 1 --2- 172.21.15.86:0/110907970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 0x5632f6000580 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.465+0000 7f911f59d740 1 -- 172.21.15.86:0/110907970 >> 172.21.15.86:0/110907970 conn(0x5632f5ffb6d0 msgr2=0x5632f5ffbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.465+0000 7f911f59d740 1 -- 172.21.15.86:0/110907970 shutdown_connections 2024-04-17T07:31:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.465+0000 7f911f59d740 1 -- 172.21.15.86:0/110907970 wait complete. 2024-04-17T07:31:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.465+0000 7f911f59d740 1 -- start start 2024-04-17T07:31:36.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.466+0000 7f911f59d740 1 Processor -- start 2024-04-17T07:31:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.466+0000 7f911f59d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 0x5632f600a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.466+0000 7f911f59d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f600ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.466+0000 7f911f59d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6005cc0 0x5632f6011c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.466+0000 7f911f59d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5632f6008cc0 con 0x5632f6000b50 2024-04-17T07:31:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.466+0000 7f911f59d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5632f5f1f830 con 0x5632f6005cc0 2024-04-17T07:31:36.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.466+0000 7f911f59d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5632f6008e40 con 0x5632f6000160 2024-04-17T07:31:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f911f59d740 10 client.0 _ll_register_callbacks cb 0x5632f5eedfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f9119e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 0x5632f600a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f9119400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f600ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f9119e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 0x5632f600a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47260/0 (socket says 172.21.15.86:47260) 2024-04-17T07:31:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f9119e00640 1 -- 172.21.15.86:0/3447568492 learned_addr learned my addr 172.21.15.86:0/3447568492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f9119e00640 1 -- 172.21.15.86:0/3447568492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6005cc0 msgr2=0x5632f6011c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f9119e00640 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6005cc0 0x5632f6011c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f9119e00640 1 -- 172.21.15.86:0/3447568492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 msgr2=0x5632f600ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f9119e00640 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f600ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f9119e00640 1 -- 172.21.15.86:0/3447568492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632f5f23920 con 0x5632f6000160 2024-04-17T07:31:36.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.467+0000 7f9119400640 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f600ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:36.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.468+0000 7f9119e00640 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 0x5632f600a710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f910802c240 tx=0x7f9108005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d0c76dea670940a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.468+0000 7f90f7400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:31:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.468+0000 7f90f7400640 1 -- 172.21.15.86:0/3447568492 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108045020 con 0x5632f6000160 2024-04-17T07:31:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.468+0000 7f90f7400640 1 -- 172.21.15.86:0/3447568492 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f91080055d0 con 0x5632f6000160 2024-04-17T07:31:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.468+0000 7f911f59d740 1 -- 172.21.15.86:0/3447568492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f911001be40 con 0x5632f6000160 2024-04-17T07:31:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.469+0000 7f90f7400640 1 -- 172.21.15.86:0/3447568492 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9108050050 con 0x5632f6000160 2024-04-17T07:31:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.469+0000 7f911f59d740 10 client.28384 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.469+0000 7f911f59d740 1 -- 172.21.15.86:0/3447568492 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632f5ef17f0 con 0x5632f6000160 2024-04-17T07:31:36.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.470+0000 7f90f7400640 1 -- 172.21.15.86:0/3447568492 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f911001be40 con 0x5632f6000160 2024-04-17T07:31:36.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.471+0000 7f90f7400640 1 -- 172.21.15.86:0/3447568492 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f91080036f0 con 0x5632f6000160 2024-04-17T07:31:36.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.471+0000 7f911f59d740 10 client.28384 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.471+0000 7f911f59d740 10 client.28384 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:31:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.471+0000 7f911f59d740 20 client.28384 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.471+0000 7f911f59d740 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5632f60162f0 0x5632f6018770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.472+0000 7f911f59d740 4 client.28384 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.472+0000 7f911f59d740 1 -- 172.21.15.86:0/3447568492 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632f5f73a40 con 0x5632f60162f0 2024-04-17T07:31:36.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.472+0000 7f9119400640 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5632f60162f0 0x5632f6018770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.473+0000 7f9119400640 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5632f60162f0 0x5632f6018770 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d14cf614e249f3b server_cookie=d05bb63ca8c70933 in_seq=0 out_seq=0 2024-04-17T07:31:36.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.474+0000 7f90f7400640 10 client.28384 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:31:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.475+0000 7f90f7400640 1 -- 172.21.15.86:0/3447568492 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632f5f73a40 con 0x5632f60162f0 2024-04-17T07:31:36.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.475+0000 7f90f7400640 10 client.28384 handle_command_reply: tid=0 2024-04-17T07:31:36.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.475+0000 7f911f59d740 1 client.28384 shutdown 2024-04-17T07:31:36.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f911f59d740 1 -- 172.21.15.86:0/3447568492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 msgr2=0x5632f600a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f911f59d740 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 0x5632f600a710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f910802c240 tx=0x7f9108005250 comp rx=0 tx=0).stop 2024-04-17T07:31:36.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f911f59d740 1 -- 172.21.15.86:0/3447568492 shutdown_connections 2024-04-17T07:31:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f911f59d740 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5632f60162f0 0x5632f6018770 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f911f59d740 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6005cc0 0x5632f6011c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f911f59d740 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f600ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f911f59d740 1 --2- 172.21.15.86:0/3447568492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 0x5632f600a710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f90f7400640 0 client.28384 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:31:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f911f59d740 1 -- 172.21.15.86:0/3447568492 >> 172.21.15.86:0/3447568492 conn(0x5632f5ffb6d0 msgr2=0x5632f5ffa930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f911f59d740 1 -- 172.21.15.86:0/3447568492 shutdown_connections 2024-04-17T07:31:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.476+0000 7f911f59d740 1 -- 172.21.15.86:0/3447568492 wait complete. 2024-04-17T07:31:36.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.477+0000 7f911f59d740 20 client.28384 trim_cache size 0 max 16384 2024-04-17T07:31:36.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.500+0000 7f911f59d740 1 Processor -- start 2024-04-17T07:31:36.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.501+0000 7f911f59d740 1 -- start start 2024-04-17T07:31:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.501+0000 7f911f59d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 0x5632f6009d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.501+0000 7f911f59d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000b50 0x5632f6011c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.501+0000 7f911f59d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6005cc0 0x5632f600a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.501+0000 7f911f59d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b95d0 con 0x5632f6005cc0 2024-04-17T07:31:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.501+0000 7f911f59d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91141bb110 con 0x5632f6000160 2024-04-17T07:31:36.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.501+0000 7f911f59d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b9ca0 con 0x5632f6000b50 2024-04-17T07:31:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.502+0000 7f9119400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000b50 0x5632f6011c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.502+0000 7f9119400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000b50 0x5632f6011c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:47264/0 (socket says 172.21.15.86:47264) 2024-04-17T07:31:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.502+0000 7f911a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6005cc0 0x5632f600a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.502+0000 7f9119400640 1 -- 172.21.15.86:0/1928155946 learned_addr learned my addr 172.21.15.86:0/1928155946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.502+0000 7f9119e00640 1 --2- 172.21.15.86:0/1928155946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 0x5632f6009d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.503+0000 7f911a800640 1 -- 172.21.15.86:0/1928155946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000b50 msgr2=0x5632f6011c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.503+0000 7f911a800640 1 --2- 172.21.15.86:0/1928155946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000b50 0x5632f6011c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.503+0000 7f911a800640 1 -- 172.21.15.86:0/1928155946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 msgr2=0x5632f6009d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.503+0000 7f911a800640 1 --2- 172.21.15.86:0/1928155946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 0x5632f6009d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.503+0000 7f911a800640 1 -- 172.21.15.86:0/1928155946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632f60097d0 con 0x5632f6005cc0 2024-04-17T07:31:36.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.503+0000 7f9119400640 1 --2- 172.21.15.86:0/1928155946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000b50 0x5632f6011c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:31:36.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.503+0000 7f911a800640 1 --2- 172.21.15.86:0/1928155946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6005cc0 0x5632f600a270 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f9110017040 tx=0x7f9110009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2d00fb8c855687 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.504+0000 7f9102600640 1 -- 172.21.15.86:0/1928155946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110014110 con 0x5632f6005cc0 2024-04-17T07:31:36.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.504+0000 7f9102600640 1 -- 172.21.15.86:0/1928155946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f911002b020 con 0x5632f6005cc0 2024-04-17T07:31:36.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.504+0000 7f9102600640 1 -- 172.21.15.86:0/1928155946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91100050c0 con 0x5632f6005cc0 2024-04-17T07:31:36.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.505+0000 7f911f59d740 1 -- 172.21.15.86:0/1928155946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6005cc0 msgr2=0x5632f600a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.505+0000 7f911f59d740 1 --2- 172.21.15.86:0/1928155946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6005cc0 0x5632f600a270 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f9110017040 tx=0x7f9110009820 comp rx=0 tx=0).stop 2024-04-17T07:31:36.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.505+0000 7f911f59d740 1 -- 172.21.15.86:0/1928155946 shutdown_connections 2024-04-17T07:31:36.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.505+0000 7f911f59d740 1 --2- 172.21.15.86:0/1928155946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6005cc0 0x5632f600a270 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.505+0000 7f911f59d740 1 --2- 172.21.15.86:0/1928155946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000b50 0x5632f6011c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.505+0000 7f911f59d740 1 --2- 172.21.15.86:0/1928155946 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6000160 0x5632f6009d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.505+0000 7f911f59d740 1 -- 172.21.15.86:0/1928155946 >> 172.21.15.86:0/1928155946 conn(0x5632f5ffb6d0 msgr2=0x5632f5fff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.505+0000 7f911f59d740 1 -- 172.21.15.86:0/1928155946 shutdown_connections 2024-04-17T07:31:36.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.505+0000 7f911f59d740 1 -- 172.21.15.86:0/1928155946 wait complete. 2024-04-17T07:31:36.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.506+0000 7f911f59d740 1 -- start start 2024-04-17T07:31:36.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.506+0000 7f911f59d740 1 Processor -- start 2024-04-17T07:31:36.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.507+0000 7f911f59d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 0x5632f600ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f911f59d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f600f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f911f59d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6005cc0 0x5632f60103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f911f59d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91141baa40 con 0x5632f6000b50 2024-04-17T07:31:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f911f59d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91141ba370 con 0x5632f6005cc0 2024-04-17T07:31:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f911f59d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91141b75a0 con 0x5632f6000160 2024-04-17T07:31:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f911f59d740 10 client.0 _ll_register_callbacks cb 0x7f91080043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f9119e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 0x5632f600ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f911a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6005cc0 0x5632f60103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f9119400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f600f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f911a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6005cc0 0x5632f60103b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:59178/0 (socket says 172.21.15.86:59178) 2024-04-17T07:31:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.508+0000 7f911a800640 1 -- 172.21.15.86:0/3121720344 learned_addr learned my addr 172.21.15.86:0/3121720344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.509+0000 7f9119400640 1 -- 172.21.15.86:0/3121720344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 msgr2=0x5632f600ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.509+0000 7f9119400640 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 0x5632f600ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.509+0000 7f9119400640 1 -- 172.21.15.86:0/3121720344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6005cc0 msgr2=0x5632f60103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.509+0000 7f9119400640 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6005cc0 0x5632f60103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.509+0000 7f9119400640 1 -- 172.21.15.86:0/3121720344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632f5c783f0 con 0x5632f6000b50 2024-04-17T07:31:36.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.510+0000 7f9119400640 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f600f3a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f91040cd8d0 tx=0x7f9104005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8986e3c1c7c0001 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.510+0000 7f90f7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.510+0000 7f90f7400640 1 -- 172.21.15.86:0/3121720344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910409a460 con 0x5632f6000b50 2024-04-17T07:31:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.510+0000 7f911f59d740 1 -- 172.21.15.86:0/3121720344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f911002b020 con 0x5632f6000b50 2024-04-17T07:31:36.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.510+0000 7f90f7400640 1 -- 172.21.15.86:0/3121720344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9104009d40 con 0x5632f6000b50 2024-04-17T07:31:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.511+0000 7f90f7400640 1 -- 172.21.15.86:0/3121720344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91040987d0 con 0x5632f6000b50 2024-04-17T07:31:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.511+0000 7f911f59d740 10 client.19365 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.511+0000 7f911f59d740 1 -- 172.21.15.86:0/3121720344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632f5d427d0 con 0x5632f6000b50 2024-04-17T07:31:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.511+0000 7f90f7400640 1 -- 172.21.15.86:0/3121720344 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f911002b020 con 0x5632f6000b50 2024-04-17T07:31:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.511+0000 7f90f7400640 1 -- 172.21.15.86:0/3121720344 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f91040852b0 con 0x5632f6000b50 2024-04-17T07:31:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.512+0000 7f911f59d740 10 client.19365 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.512+0000 7f911f59d740 10 client.19365 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:31:36.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.512+0000 7f911f59d740 20 client.19365 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.512+0000 7f911f59d740 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5632f607ca00 0x5632f607ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.512+0000 7f911f59d740 4 client.19365 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:31:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.512+0000 7f911f59d740 1 -- 172.21.15.86:0/3121720344 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632f5f6d8b0 con 0x5632f607ca00 2024-04-17T07:31:36.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.513+0000 7f9119e00640 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5632f607ca00 0x5632f607ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.514+0000 7f9119e00640 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5632f607ca00 0x5632f607ee20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af279c3766664f66 server_cookie=55668666a66aab53 in_seq=0 out_seq=0 2024-04-17T07:31:36.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.514+0000 7f90f7400640 10 client.19365 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:31:36.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f90f7400640 1 -- 172.21.15.86:0/3121720344 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632f5f6d8b0 con 0x5632f607ca00 2024-04-17T07:31:36.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f90f7400640 10 client.19365 handle_command_reply: tid=0 2024-04-17T07:31:36.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f911f59d740 1 client.19365 shutdown 2024-04-17T07:31:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f911f59d740 1 -- 172.21.15.86:0/3121720344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 msgr2=0x5632f600f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f911f59d740 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f600f3a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f91040cd8d0 tx=0x7f9104005850 comp rx=0 tx=0).stop 2024-04-17T07:31:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f911f59d740 1 -- 172.21.15.86:0/3121720344 shutdown_connections 2024-04-17T07:31:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f911f59d740 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5632f607ca00 0x5632f607ee20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f911f59d740 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5632f6005cc0 0x5632f60103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f911f59d740 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5632f6000b50 0x5632f600f3a0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f911f59d740 1 --2- 172.21.15.86:0/3121720344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5632f6000160 0x5632f600ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.515+0000 7f911f59d740 1 -- 172.21.15.86:0/3121720344 >> 172.21.15.86:0/3121720344 conn(0x5632f5ffb6d0 msgr2=0x5632f5ffa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.516+0000 7f90f7400640 0 client.19365 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:31:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.516+0000 7f911f59d740 1 -- 172.21.15.86:0/3121720344 shutdown_connections 2024-04-17T07:31:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.516+0000 7f911f59d740 1 -- 172.21.15.86:0/3121720344 wait complete. 2024-04-17T07:31:36.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.516+0000 7f911f59d740 20 client.19365 trim_cache size 0 max 16384 2024-04-17T07:31:36.517 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:31:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.517+0000 7f911b200640 1 -- 172.21.15.86:0/2185763347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90fc0a3c30 msgr2=0x7f90fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.517+0000 7f911b200640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90fc0a3c30 0x7f90fc0a60f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9110014fd0 tx=0x7f911001b000 comp rx=0 tx=0).stop 2024-04-17T07:31:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.517+0000 7f911b200640 1 -- 172.21.15.86:0/2185763347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141010a0 msgr2=0x7f9114106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.517+0000 7f911b200640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141010a0 0x7f9114106710 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f910400d6d0 tx=0x7f910400db90 comp rx=0 tx=0).stop 2024-04-17T07:31:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.518+0000 7f911b200640 1 -- 172.21.15.86:0/2185763347 shutdown_connections 2024-04-17T07:31:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.518+0000 7f911b200640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90fc0a3c30 0x7f90fc0a60f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.518+0000 7f911b200640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9114101a40 0x7f911410b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.518+0000 7f911b200640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91141010a0 0x7f9114106710 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.518+0000 7f911b200640 1 --2- 172.21.15.86:0/2185763347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9114100740 0x7f91141061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.518+0000 7f911b200640 1 -- 172.21.15.86:0/2185763347 >> 172.21.15.86:0/2185763347 conn(0x7f91140fc300 msgr2=0x7f91140fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.518+0000 7f911b200640 1 -- 172.21.15.86:0/2185763347 shutdown_connections 2024-04-17T07:31:36.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.518+0000 7f911b200640 1 -- 172.21.15.86:0/2185763347 wait complete. 2024-04-17T07:31:36.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:36 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2917113974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:31:36.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:36 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2917113974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:31:36.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:31:36.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:36 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2917113974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:31:36.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.843+0000 7f90f5000640 1 -- 172.21.15.86:0/971563468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0078500 msgr2=0x7f90f0078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.843+0000 7f90f5000640 1 --2- 172.21.15.86:0/971563468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0078500 0x7f90f0078da0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009900 tx=0x7f90e802bc40 comp rx=0 tx=0).stop 2024-04-17T07:31:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.844+0000 7f90f5000640 1 -- 172.21.15.86:0/971563468 shutdown_connections 2024-04-17T07:31:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.844+0000 7f90f5000640 1 --2- 172.21.15.86:0/971563468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0078500 0x7f90f0078da0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.844+0000 7f90f5000640 1 --2- 172.21.15.86:0/971563468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f0077b60 0x7f90f0077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.844+0000 7f90f5000640 1 --2- 172.21.15.86:0/971563468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90f0076920 0x7f90f0076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.844+0000 7f90f5000640 1 -- 172.21.15.86:0/971563468 >> 172.21.15.86:0/971563468 conn(0x7f90f00ff710 msgr2=0x7f90f0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.844+0000 7f90f5000640 1 -- 172.21.15.86:0/971563468 shutdown_connections 2024-04-17T07:31:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.844+0000 7f90f5000640 1 -- 172.21.15.86:0/971563468 wait complete. 2024-04-17T07:31:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.845+0000 7f90f5000640 1 Processor -- start 2024-04-17T07:31:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.845+0000 7f90f5000640 1 -- start start 2024-04-17T07:31:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90f5000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f0076920 0x7f90f019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90f5000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90f0077b60 0x7f90f019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90f5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0078500 0x7f90f01a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90f5000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0114140 con 0x7f90f0078500 2024-04-17T07:31:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90f5000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0113fc0 con 0x7f90f0077b60 2024-04-17T07:31:36.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90f5000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01142c0 con 0x7f90f0076920 2024-04-17T07:31:36.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90ede00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f0076920 0x7f90f019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90ede00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f0076920 0x7f90f019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42106/0 (socket says 172.21.15.86:42106) 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90ede00640 1 -- 172.21.15.86:0/47333412 learned_addr learned my addr 172.21.15.86:0/47333412 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90ee800640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0078500 0x7f90f01a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.846+0000 7f90e5400640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90f0077b60 0x7f90f019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.847+0000 7f90ede00640 1 -- 172.21.15.86:0/47333412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90f0077b60 msgr2=0x7f90f019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.847+0000 7f90ede00640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90f0077b60 0x7f90f019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.847+0000 7f90ede00640 1 -- 172.21.15.86:0/47333412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0078500 msgr2=0x7f90f01a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.847+0000 7f90ede00640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0078500 0x7f90f01a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.847+0000 7f90ede00640 1 -- 172.21.15.86:0/47333412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f01a63f0 con 0x7f90f0076920 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.847+0000 7f90ede00640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f0076920 0x7f90f019e7d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f90e000db80 tx=0x7f90e000b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.847+0000 7f90e7400640 1 -- 172.21.15.86:0/47333412 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0017070 con 0x7f90f0076920 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.847+0000 7f90e7400640 1 -- 172.21.15.86:0/47333412 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90e0003b70 con 0x7f90f0076920 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.847+0000 7f90e7400640 1 -- 172.21.15.86:0/47333412 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e00104c0 con 0x7f90f0076920 2024-04-17T07:31:36.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.847+0000 7f90f5000640 1 -- 172.21.15.86:0/47333412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f01a66e0 con 0x7f90f0076920 2024-04-17T07:31:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.848+0000 7f90f5000640 1 -- 172.21.15.86:0/47333412 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f01a6be0 con 0x7f90f0076920 2024-04-17T07:31:36.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.851+0000 7f90f7d64740 1 Processor -- start 2024-04-17T07:31:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.852+0000 7f90f7d64740 1 -- start start 2024-04-17T07:31:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.853+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221811fa0 0x55b2218123c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.853+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221812990 0x55b221817490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.853+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221817b00 0x55b221819f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.853+0000 7f90f7d64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b221778020 con 0x55b221812990 2024-04-17T07:31:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.853+0000 7f90f7d64740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b22171e460 con 0x55b221811fa0 2024-04-17T07:31:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.853+0000 7f90f7d64740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b221773fe0 con 0x55b221817b00 2024-04-17T07:31:36.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.853+0000 7f90e5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221817b00 0x55b221819f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.853+0000 7f90e5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221817b00 0x55b221819f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42118/0 (socket says 172.21.15.86:42118) 2024-04-17T07:31:36.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.853+0000 7f90e5400640 1 -- 172.21.15.86:0/2541038653 learned_addr learned my addr 172.21.15.86:0/2541038653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.854+0000 7f90ee800640 1 --2- 172.21.15.86:0/2541038653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221811fa0 0x55b2218123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.854+0000 7f90e7400640 1 -- 172.21.15.86:0/47333412 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f90e0004020 con 0x7f90f0076920 2024-04-17T07:31:36.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.854+0000 7f90e5400640 1 -- 172.21.15.86:0/2541038653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221811fa0 msgr2=0x55b2218123c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.854+0000 7f90e5400640 1 --2- 172.21.15.86:0/2541038653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221811fa0 0x55b2218123c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.854+0000 7f90e5400640 1 -- 172.21.15.86:0/2541038653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221812990 msgr2=0x55b221817490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.854+0000 7f90e5400640 1 --2- 172.21.15.86:0/2541038653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221812990 0x55b221817490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.854+0000 7f90e5400640 1 -- 172.21.15.86:0/2541038653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b221726040 con 0x55b221817b00 2024-04-17T07:31:36.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.856+0000 7f90e5400640 1 --2- 172.21.15.86:0/2541038653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221817b00 0x55b221819f90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f90d800b690 tx=0x7f90d800bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56f045989512875b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.856+0000 7f90e7400640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90d40a3bb0 0x7f90d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.857+0000 7f90df400640 1 -- 172.21.15.86:0/2541038653 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8003bd0 con 0x55b221817b00 2024-04-17T07:31:36.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.857+0000 7f90df400640 1 -- 172.21.15.86:0/2541038653 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90d8015070 con 0x55b221817b00 2024-04-17T07:31:36.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.857+0000 7f90e7400640 1 -- 172.21.15.86:0/47333412 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f90e00d0390 con 0x7f90f0076920 2024-04-17T07:31:36.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.857+0000 7f90df400640 1 -- 172.21.15.86:0/2541038653 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8004e00 con 0x55b221817b00 2024-04-17T07:31:36.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.858+0000 7f90f7d64740 1 -- 172.21.15.86:0/2541038653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221817b00 msgr2=0x55b221819f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.858+0000 7f90f7d64740 1 --2- 172.21.15.86:0/2541038653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221817b00 0x55b221819f90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f90d800b690 tx=0x7f90d800bb50 comp rx=0 tx=0).stop 2024-04-17T07:31:36.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.858+0000 7f90f7d64740 1 -- 172.21.15.86:0/2541038653 shutdown_connections 2024-04-17T07:31:36.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.858+0000 7f90f7d64740 1 --2- 172.21.15.86:0/2541038653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221817b00 0x55b221819f90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.858+0000 7f90f7d64740 1 --2- 172.21.15.86:0/2541038653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221812990 0x55b221817490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.858+0000 7f90f7d64740 1 --2- 172.21.15.86:0/2541038653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221811fa0 0x55b2218123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.858+0000 7f90f7d64740 1 -- 172.21.15.86:0/2541038653 >> 172.21.15.86:0/2541038653 conn(0x55b22180d510 msgr2=0x55b22180d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.858+0000 7f90ee800640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90d40a3bb0 0x7f90d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.858+0000 7f90f7d64740 1 -- 172.21.15.86:0/2541038653 shutdown_connections 2024-04-17T07:31:36.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.859+0000 7f90f7d64740 1 -- 172.21.15.86:0/2541038653 wait complete. 2024-04-17T07:31:36.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.860+0000 7f90f7d64740 1 -- start start 2024-04-17T07:31:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.861+0000 7f90f7d64740 1 Processor -- start 2024-04-17T07:31:36.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.861+0000 7f90ee800640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90d40a3bb0 0x7f90d40a6070 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f90e8006010 tx=0x7f90e802c810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.863+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.863+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 0x55b22181caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.863+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221817b00 0x55b221823ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.863+0000 7f90f7d64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b22181ab50 con 0x55b221811fa0 2024-04-17T07:31:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.863+0000 7f90f7d64740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b221731830 con 0x55b221817b00 2024-04-17T07:31:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.863+0000 7f90f7d64740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b22181acd0 con 0x55b221812990 2024-04-17T07:31:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.863+0000 7f90f7d64740 10 client.0 _ll_register_callbacks cb 0x55b2216fffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.864+0000 7f90ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.864+0000 7f90ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56574/0 (socket says 172.21.15.86:56574) 2024-04-17T07:31:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.864+0000 7f90ede00640 1 -- 172.21.15.86:0/1141653268 learned_addr learned my addr 172.21.15.86:0/1141653268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.864+0000 7f90e5400640 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 0x55b22181caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.864+0000 7f90e5400640 1 -- 172.21.15.86:0/1141653268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221817b00 msgr2=0x55b221823ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.864+0000 7f90e5400640 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221817b00 0x55b221823ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.864+0000 7f90e5400640 1 -- 172.21.15.86:0/1141653268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 msgr2=0x55b22181c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.864+0000 7f90e5400640 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.864+0000 7f90e5400640 1 -- 172.21.15.86:0/1141653268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b221735920 con 0x55b221812990 2024-04-17T07:31:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.865+0000 7f90e5400640 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 0x55b22181caa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55b22180e480 tx=0x7f90d800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1a428c33a534af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.865+0000 7f90d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:31:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.865+0000 7f90d3e00640 1 -- 172.21.15.86:0/1141653268 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8028060 con 0x55b221812990 2024-04-17T07:31:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.865+0000 7f90d3e00640 1 -- 172.21.15.86:0/1141653268 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f90d8011440 con 0x55b221812990 2024-04-17T07:31:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.866+0000 7f90f7d64740 1 -- 172.21.15.86:0/1141653268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d8015070 con 0x55b221812990 2024-04-17T07:31:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.866+0000 7f90d3e00640 1 -- 172.21.15.86:0/1141653268 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90d8011640 con 0x55b221812990 2024-04-17T07:31:36.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.866+0000 7f90f7d64740 10 client.28402 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.866+0000 7f90f7d64740 1 -- 172.21.15.86:0/1141653268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2217037f0 con 0x55b221812990 2024-04-17T07:31:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.866+0000 7f90d3e00640 1 -- 172.21.15.86:0/1141653268 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90d8015070 con 0x55b221812990 2024-04-17T07:31:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.867+0000 7f90d3e00640 1 -- 172.21.15.86:0/1141653268 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f90d8022070 con 0x55b221812990 2024-04-17T07:31:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.867+0000 7f90f7d64740 10 client.28402 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.867+0000 7f90f7d64740 10 client.28402 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:31:36.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.867+0000 7f90f7d64740 20 client.28402 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.867+0000 7f90f7d64740 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b221827c40 0x55b22182a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.867+0000 7f90f7d64740 4 client.28402 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.867+0000 7f90f7d64740 1 -- 172.21.15.86:0/1141653268 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b221785a40 con 0x55b221827c40 2024-04-17T07:31:36.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.868+0000 7f90ede00640 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b221827c40 0x55b22182a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.870+0000 7f90ede00640 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b221827c40 0x55b22182a030 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b218ba8e22c341c server_cookie=eb7ce3ed7ac02ed7 in_seq=0 out_seq=0 2024-04-17T07:31:36.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.870+0000 7f90d3e00640 10 client.28402 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:31:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.872+0000 7f90d3e00640 1 -- 172.21.15.86:0/1141653268 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b221785a40 con 0x55b221827c40 2024-04-17T07:31:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.873+0000 7f90d3e00640 10 client.28402 handle_command_reply: tid=0 2024-04-17T07:31:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.873+0000 7f90f7d64740 1 client.28402 shutdown 2024-04-17T07:31:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.873+0000 7f90f7d64740 1 -- 172.21.15.86:0/1141653268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 msgr2=0x55b22181caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.873+0000 7f90f7d64740 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 0x55b22181caa0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55b22180e480 tx=0x7f90d800bfd0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.874+0000 7f90f7d64740 1 -- 172.21.15.86:0/1141653268 shutdown_connections 2024-04-17T07:31:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.874+0000 7f90f7d64740 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b221827c40 0x55b22182a030 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.874+0000 7f90f7d64740 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221817b00 0x55b221823ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.874+0000 7f90f7d64740 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 0x55b22181caa0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.874+0000 7f90f7d64740 1 --2- 172.21.15.86:0/1141653268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.874+0000 7f90f7d64740 1 -- 172.21.15.86:0/1141653268 >> 172.21.15.86:0/1141653268 conn(0x55b22180d510 msgr2=0x55b22180c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.874+0000 7f90d3e00640 0 client.28402 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:31:36.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.876+0000 7f90f7d64740 1 -- 172.21.15.86:0/1141653268 shutdown_connections 2024-04-17T07:31:36.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.877+0000 7f90f7d64740 1 -- 172.21.15.86:0/1141653268 wait complete. 2024-04-17T07:31:36.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.878+0000 7f90f7d64740 20 client.28402 trim_cache size 0 max 16384 2024-04-17T07:31:36.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.910+0000 7f90f7d64740 1 Processor -- start 2024-04-17T07:31:36.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.911+0000 7f90f7d64740 1 -- start start 2024-04-17T07:31:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.911+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221811fa0 0x55b2218089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.911+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221812990 0x55b221823aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.911+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221817b00 0x55b221810e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.911+0000 7f90f7d64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01b1140 con 0x55b221817b00 2024-04-17T07:31:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.911+0000 7f90f7d64740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01b2c80 con 0x55b221812990 2024-04-17T07:31:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.911+0000 7f90f7d64740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01b1810 con 0x55b221811fa0 2024-04-17T07:31:36.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.912+0000 7f90e5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221812990 0x55b221823aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.912+0000 7f90e5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221812990 0x55b221823aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36442/0 (socket says 172.21.15.86:36442) 2024-04-17T07:31:36.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.912+0000 7f90e5400640 1 -- 172.21.15.86:0/3916318923 learned_addr learned my addr 172.21.15.86:0/3916318923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.912+0000 7f90ede00640 1 --2- 172.21.15.86:0/3916318923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221811fa0 0x55b2218089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.912+0000 7f90ee800640 1 --2- 172.21.15.86:0/3916318923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221817b00 0x55b221810e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.913+0000 7f90e5400640 1 -- 172.21.15.86:0/3916318923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221811fa0 msgr2=0x55b2218089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.913+0000 7f90e5400640 1 --2- 172.21.15.86:0/3916318923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221811fa0 0x55b2218089d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.913+0000 7f90e5400640 1 -- 172.21.15.86:0/3916318923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221817b00 msgr2=0x55b221810e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.913+0000 7f90e5400640 1 --2- 172.21.15.86:0/3916318923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221817b00 0x55b221810e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.913+0000 7f90e5400640 1 -- 172.21.15.86:0/3916318923 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22148a3f0 con 0x55b221812990 2024-04-17T07:31:36.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.913+0000 7f90e5400640 1 --2- 172.21.15.86:0/3916318923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221812990 0x55b221823aa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f90d800b520 tx=0x7f90d8005a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95dd261efc71473a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.914+0000 7f90dea00640 1 -- 172.21.15.86:0/3916318923 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8028070 con 0x55b221812990 2024-04-17T07:31:36.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.914+0000 7f90dea00640 1 -- 172.21.15.86:0/3916318923 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90d8015070 con 0x55b221812990 2024-04-17T07:31:36.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.914+0000 7f90dea00640 1 -- 172.21.15.86:0/3916318923 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8032080 con 0x55b221812990 2024-04-17T07:31:36.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.916+0000 7f90f7d64740 1 -- 172.21.15.86:0/3916318923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221812990 msgr2=0x55b221823aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.916+0000 7f90f7d64740 1 --2- 172.21.15.86:0/3916318923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221812990 0x55b221823aa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f90d800b520 tx=0x7f90d8005a70 comp rx=0 tx=0).stop 2024-04-17T07:31:36.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.916+0000 7f90f7d64740 1 -- 172.21.15.86:0/3916318923 shutdown_connections 2024-04-17T07:31:36.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.916+0000 7f90f7d64740 1 --2- 172.21.15.86:0/3916318923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221817b00 0x55b221810e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.916+0000 7f90f7d64740 1 --2- 172.21.15.86:0/3916318923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221812990 0x55b221823aa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.916+0000 7f90f7d64740 1 --2- 172.21.15.86:0/3916318923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221811fa0 0x55b2218089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.916+0000 7f90f7d64740 1 -- 172.21.15.86:0/3916318923 >> 172.21.15.86:0/3916318923 conn(0x55b22180d510 msgr2=0x55b221810770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.917+0000 7f90f7d64740 1 -- 172.21.15.86:0/3916318923 shutdown_connections 2024-04-17T07:31:36.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.917+0000 7f90f7d64740 1 -- 172.21.15.86:0/3916318923 wait complete. 2024-04-17T07:31:36.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.918+0000 7f90f7d64740 1 -- start start 2024-04-17T07:31:36.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.918+0000 7f90f7d64740 1 Processor -- start 2024-04-17T07:31:36.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.920+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.920+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 0x55b22181f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.920+0000 7f90f7d64740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221817b00 0x55b2218223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.921+0000 7f90e5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 0x55b22181f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.921+0000 7f90e5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 0x55b22181f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42160/0 (socket says 172.21.15.86:42160) 2024-04-17T07:31:36.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.921+0000 7f90ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.921+0000 7f90ede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56600/0 (socket says 172.21.15.86:56600) 2024-04-17T07:31:36.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.921+0000 7f90e5400640 1 -- 172.21.15.86:0/82688915 learned_addr learned my addr 172.21.15.86:0/82688915 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:36.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.921+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01b25b0 con 0x55b221811fa0 2024-04-17T07:31:36.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.922+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01b1ee0 con 0x55b221817b00 2024-04-17T07:31:36.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.922+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01af110 con 0x55b221812990 2024-04-17T07:31:36.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.922+0000 7f90ee800640 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221817b00 0x55b2218223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.922+0000 7f90f7d64740 10 client.0 _ll_register_callbacks cb 0x7f90d80044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:36.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.924+0000 7f90ee800640 1 -- 172.21.15.86:0/82688915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 msgr2=0x55b22181f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.924+0000 7f90ee800640 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 0x55b22181f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.924+0000 7f90ee800640 1 -- 172.21.15.86:0/82688915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 msgr2=0x55b22181ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:36.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.924+0000 7f90ee800640 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.924+0000 7f90ee800640 1 -- 172.21.15.86:0/82688915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2215547d0 con 0x55b221817b00 2024-04-17T07:31:36.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.924+0000 7f90ede00640 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:36.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.925+0000 7f90ee800640 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221817b00 0x55b2218223b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f90e8036040 tx=0x7f90e80052c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f30f73172dbf44 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:36.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.925+0000 7f90d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:31:36.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.925+0000 7f90d3e00640 1 -- 172.21.15.86:0/82688915 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8051070 con 0x55b221817b00 2024-04-17T07:31:36.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.925+0000 7f90d3e00640 1 -- 172.21.15.86:0/82688915 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f90e80048d0 con 0x55b221817b00 2024-04-17T07:31:36.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.926+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d8015070 con 0x55b221817b00 2024-04-17T07:31:36.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.927+0000 7f90d3e00640 1 -- 172.21.15.86:0/82688915 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90e8004ad0 con 0x55b221817b00 2024-04-17T07:31:36.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.927+0000 7f90f7d64740 10 client.38375 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:36.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.927+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b221823b40 con 0x55b221817b00 2024-04-17T07:31:36.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.929+0000 7f90d3e00640 1 -- 172.21.15.86:0/82688915 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90d8015070 con 0x55b221817b00 2024-04-17T07:31:36.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.929+0000 7f90d3e00640 1 -- 172.21.15.86:0/82688915 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f90e8056460 con 0x55b221817b00 2024-04-17T07:31:36.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.929+0000 7f90f7d64740 10 client.38375 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:36.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.929+0000 7f90f7d64740 10 client.38375 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:31:36.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.929+0000 7f90f7d64740 20 client.38375 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:36.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.929+0000 7f90f7d64740 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b22189f4d0 0x55b2218a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:36.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.929+0000 7f90f7d64740 4 client.38375 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:31:36.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.929+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b22177f8b0 con 0x55b22189f4d0 2024-04-17T07:31:36.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.930+0000 7f90ede00640 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b22189f4d0 0x55b2218a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:36.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.933+0000 7f90ede00640 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b22189f4d0 0x55b2218a18f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=187c185694542aa server_cookie=4aa9127d1956a250 in_seq=0 out_seq=0 2024-04-17T07:31:36.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.933+0000 7f90d3e00640 10 client.38375 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:31:36.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.933+0000 7f90d3e00640 1 -- 172.21.15.86:0/82688915 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b22177f8b0 con 0x55b22189f4d0 2024-04-17T07:31:36.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.933+0000 7f90d3e00640 10 client.38375 handle_command_reply: tid=0 2024-04-17T07:31:36.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.934+0000 7f90f7d64740 1 client.38375 shutdown 2024-04-17T07:31:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221817b00 msgr2=0x55b2218223b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90f7d64740 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221817b00 0x55b2218223b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f90e8036040 tx=0x7f90e80052c0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 shutdown_connections 2024-04-17T07:31:36.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90f7d64740 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55b22189f4d0 0x55b2218a18f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90f7d64740 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b221817b00 0x55b2218223b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90f7d64740 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b221812990 0x55b22181f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90f7d64740 1 --2- 172.21.15.86:0/82688915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b221811fa0 0x55b22181ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90d3e00640 0 client.38375 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:31:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 >> 172.21.15.86:0/82688915 conn(0x55b22180d510 msgr2=0x55b221811000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 shutdown_connections 2024-04-17T07:31:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.935+0000 7f90f7d64740 1 -- 172.21.15.86:0/82688915 wait complete. 2024-04-17T07:31:36.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.936+0000 7f90f7d64740 20 client.38375 trim_cache size 0 max 16384 2024-04-17T07:31:36.937 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:31:36.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.937+0000 7f90f5000640 1 -- 172.21.15.86:0/47333412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90d40a3bb0 msgr2=0x7f90d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.937+0000 7f90f5000640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90d40a3bb0 0x7f90d40a6070 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f90e8006010 tx=0x7f90e802c810 comp rx=0 tx=0).stop 2024-04-17T07:31:36.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.937+0000 7f90f5000640 1 -- 172.21.15.86:0/47333412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f0076920 msgr2=0x7f90f019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:36.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.937+0000 7f90f5000640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f0076920 0x7f90f019e7d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f90e000db80 tx=0x7f90e000b630 comp rx=0 tx=0).stop 2024-04-17T07:31:36.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.937+0000 7f90f5000640 1 -- 172.21.15.86:0/47333412 shutdown_connections 2024-04-17T07:31:36.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.937+0000 7f90f5000640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f90d40a3bb0 0x7f90d40a6070 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.937+0000 7f90f5000640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90f0078500 0x7f90f01a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.937+0000 7f90f5000640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90f0077b60 0x7f90f019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.937+0000 7f90f5000640 1 --2- 172.21.15.86:0/47333412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90f0076920 0x7f90f019e7d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:36.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.937+0000 7f90f5000640 1 -- 172.21.15.86:0/47333412 >> 172.21.15.86:0/47333412 conn(0x7f90f00ff710 msgr2=0x7f90f0100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:36.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.938+0000 7f90f5000640 1 -- 172.21.15.86:0/47333412 shutdown_connections 2024-04-17T07:31:36.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:36.938+0000 7f90f5000640 1 -- 172.21.15.86:0/47333412 wait complete. 2024-04-17T07:31:36.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:31:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.307+0000 7f88b7000640 1 -- 172.21.15.86:0/3000392797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88b0101fb0 msgr2=0x7f88b01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.307+0000 7f88b7000640 1 --2- 172.21.15.86:0/3000392797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88b0101fb0 0x7f88b01023d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009900 tx=0x7f88a002bc40 comp rx=0 tx=0).stop 2024-04-17T07:31:37.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.310+0000 7f88b7000640 1 -- 172.21.15.86:0/3000392797 shutdown_connections 2024-04-17T07:31:37.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.310+0000 7f88b7000640 1 --2- 172.21.15.86:0/3000392797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b0103b60 0x7f88b010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.310+0000 7f88b7000640 1 --2- 172.21.15.86:0/3000392797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b01031a0 0x7f88b0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.310+0000 7f88b7000640 1 --2- 172.21.15.86:0/3000392797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88b0101fb0 0x7f88b01023d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.310+0000 7f88b7000640 1 -- 172.21.15.86:0/3000392797 >> 172.21.15.86:0/3000392797 conn(0x7f88b00fd540 msgr2=0x7f88b00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.311+0000 7f88b7000640 1 -- 172.21.15.86:0/3000392797 shutdown_connections 2024-04-17T07:31:37.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.313+0000 7f88b7000640 1 -- 172.21.15.86:0/3000392797 wait complete. 2024-04-17T07:31:37.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.314+0000 7f88b7000640 1 Processor -- start 2024-04-17T07:31:37.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.315+0000 7f88b7000640 1 -- start start 2024-04-17T07:31:37.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.316+0000 7f88b7000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b0101fb0 0x7f88b0070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.316+0000 7f88b7000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b01031a0 0x7f88b0070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.316+0000 7f88b7000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88b0103b60 0x7f88b0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.317+0000 7f88b7000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88b010db00 con 0x7f88b01031a0 2024-04-17T07:31:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.317+0000 7f88b7000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f88b010d980 con 0x7f88b0101fb0 2024-04-17T07:31:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.317+0000 7f88b7000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88b010dc80 con 0x7f88b0103b60 2024-04-17T07:31:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.317+0000 7f88af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b01031a0 0x7f88b0070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.318+0000 7f88afe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b0101fb0 0x7f88b0070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.318+0000 7f88afe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b0101fb0 0x7f88b0070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36474/0 (socket says 172.21.15.86:36474) 2024-04-17T07:31:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.318+0000 7f88afe00640 1 -- 172.21.15.86:0/1192304378 learned_addr learned my addr 172.21.15.86:0/1192304378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.318+0000 7f88b5000640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88b0103b60 0x7f88b0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.319+0000 7f88afe00640 1 -- 172.21.15.86:0/1192304378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88b0103b60 msgr2=0x7f88b0197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.320+0000 7f88afe00640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88b0103b60 0x7f88b0197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.320+0000 7f88afe00640 1 -- 172.21.15.86:0/1192304378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b01031a0 msgr2=0x7f88b0070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.320+0000 7f88afe00640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b01031a0 0x7f88b0070ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.320+0000 7f88afe00640 1 -- 172.21.15.86:0/1192304378 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b0197740 con 0x7f88b0101fb0 2024-04-17T07:31:37.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.321+0000 7f88b5000640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88b0103b60 0x7f88b0197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:37.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.321+0000 7f88af400640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b01031a0 0x7f88b0070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:31:37.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.321+0000 7f88afe00640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b0101fb0 0x7f88b0070980 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f88a002c620 tx=0x7f88a00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.322+0000 7f88acc00640 1 -- 172.21.15.86:0/1192304378 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0043020 con 0x7f88b0101fb0 2024-04-17T07:31:37.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.322+0000 7f88b7000640 1 -- 172.21.15.86:0/1192304378 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b0074190 con 0x7f88b0101fb0 2024-04-17T07:31:37.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.322+0000 7f88b7000640 1 -- 172.21.15.86:0/1192304378 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b0074730 con 0x7f88b0101fb0 2024-04-17T07:31:37.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.323+0000 7f88acc00640 1 -- 172.21.15.86:0/1192304378 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88a00044c0 con 0x7f88b0101fb0 2024-04-17T07:31:37.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.323+0000 7f88acc00640 1 -- 172.21.15.86:0/1192304378 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0003c60 con 0x7f88b0101fb0 2024-04-17T07:31:37.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.328+0000 7f88acc00640 1 -- 172.21.15.86:0/1192304378 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f88a0046020 con 0x7f88b0101fb0 2024-04-17T07:31:37.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.332+0000 7f88acc00640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88940a3cb0 0x7f88940a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.333+0000 7f88acc00640 1 -- 172.21.15.86:0/1192304378 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f88a00f2a10 con 0x7f88b0101fb0 2024-04-17T07:31:37.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.335+0000 7f88af400640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88940a3cb0 0x7f88940a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.337+0000 7f88b9da9740 1 Processor -- start 2024-04-17T07:31:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.340+0000 7f88b9da9740 1 -- start start 2024-04-17T07:31:37.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.340+0000 7f88af400640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88940a3cb0 0x7f88940a6170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f889c005fd0 tx=0x7f889c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f37b0 0x5594e76f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e76f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f9310 0x5594e76fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88b9da9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594e7651020 con 0x5594e76f9310 2024-04-17T07:31:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88b9da9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5594e75f7460 con 0x5594e76f41a0 2024-04-17T07:31:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88b9da9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5594e764cfe0 con 0x5594e76f37b0 2024-04-17T07:31:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88af400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e76f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88b5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f9310 0x5594e76fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88afe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f37b0 0x5594e76f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88afe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f37b0 0x5594e76f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42184/0 (socket says 172.21.15.86:42184) 2024-04-17T07:31:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.342+0000 7f88afe00640 1 -- 172.21.15.86:0/535389702 learned_addr learned my addr 172.21.15.86:0/535389702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.343+0000 7f88b5000640 1 -- 172.21.15.86:0/535389702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f37b0 msgr2=0x5594e76f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.343+0000 7f88b5000640 1 --2- 172.21.15.86:0/535389702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f37b0 0x5594e76f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.343+0000 7f88b5000640 1 -- 172.21.15.86:0/535389702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 msgr2=0x5594e76f8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.343+0000 7f88b5000640 1 --2- 172.21.15.86:0/535389702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e76f8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.343+0000 7f88b5000640 1 -- 172.21.15.86:0/535389702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594e75ff040 con 0x5594e76f9310 2024-04-17T07:31:37.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.344+0000 7f88b5000640 1 --2- 172.21.15.86:0/535389702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f9310 0x5594e76fb7a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f88a8005fd0 tx=0x7f88a8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee2c2b2373176af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.345+0000 7f88a5600640 1 -- 172.21.15.86:0/535389702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a800f940 con 0x5594e76f9310 2024-04-17T07:31:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.345+0000 7f88a5600640 1 -- 172.21.15.86:0/535389702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88a8012450 con 0x5594e76f9310 2024-04-17T07:31:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.345+0000 7f88a5600640 1 -- 172.21.15.86:0/535389702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a801a630 con 0x5594e76f9310 2024-04-17T07:31:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.346+0000 7f88b9da9740 1 -- 172.21.15.86:0/535389702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f9310 msgr2=0x5594e76fb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.346+0000 7f88b9da9740 1 --2- 172.21.15.86:0/535389702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f9310 0x5594e76fb7a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f88a8005fd0 tx=0x7f88a8011c10 comp rx=0 tx=0).stop 2024-04-17T07:31:37.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.347+0000 7f88b9da9740 1 -- 172.21.15.86:0/535389702 shutdown_connections 2024-04-17T07:31:37.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.347+0000 7f88b9da9740 1 --2- 172.21.15.86:0/535389702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f9310 0x5594e76fb7a0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.347+0000 7f88b9da9740 1 --2- 172.21.15.86:0/535389702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e76f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.347+0000 7f88b9da9740 1 --2- 172.21.15.86:0/535389702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f37b0 0x5594e76f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.347+0000 7f88b9da9740 1 -- 172.21.15.86:0/535389702 >> 172.21.15.86:0/535389702 conn(0x5594e76e6510 msgr2=0x5594e76e6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.347+0000 7f88b9da9740 1 -- 172.21.15.86:0/535389702 shutdown_connections 2024-04-17T07:31:37.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.348+0000 7f88b9da9740 1 -- 172.21.15.86:0/535389702 wait complete. 2024-04-17T07:31:37.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.349+0000 7f88b9da9740 1 -- start start 2024-04-17T07:31:37.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.350+0000 7f88b9da9740 1 Processor -- start 2024-04-17T07:31:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.351+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f37b0 0x5594e76fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.351+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f41a0 0x5594e76fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.351+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e77052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.352+0000 7f88b9da9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5594e76fc2c0 con 0x5594e76f41a0 2024-04-17T07:31:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.352+0000 7f88b9da9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5594e760a830 con 0x5594e76f37b0 2024-04-17T07:31:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.352+0000 7f88b9da9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5594e76fc440 con 0x5594e76f9310 2024-04-17T07:31:37.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.352+0000 7f88b9da9740 10 client.0 _ll_register_callbacks cb 0x5594e75d8fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:37.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.352+0000 7f88afe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f37b0 0x5594e76fdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.353+0000 7f88afe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f37b0 0x5594e76fdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36484/0 (socket says 172.21.15.86:36484) 2024-04-17T07:31:37.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.353+0000 7f88afe00640 1 -- 172.21.15.86:0/1028998843 learned_addr learned my addr 172.21.15.86:0/1028998843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.353+0000 7f88b5000640 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e77052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.353+0000 7f88afe00640 1 -- 172.21.15.86:0/1028998843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 msgr2=0x5594e77052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.353+0000 7f88afe00640 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e77052b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.354+0000 7f88afe00640 1 -- 172.21.15.86:0/1028998843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f41a0 msgr2=0x5594e76fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.354+0000 7f88afe00640 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f41a0 0x5594e76fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.354+0000 7f88afe00640 1 -- 172.21.15.86:0/1028998843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594e760e920 con 0x5594e76f37b0 2024-04-17T07:31:37.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.356+0000 7f88afe00640 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f37b0 0x5594e76fdd30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f88a00be610 tx=0x7f88a00035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b300683ff379a74d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.356+0000 7f889a000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:31:37.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.357+0000 7f889a000640 1 -- 172.21.15.86:0/1028998843 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a00c2cc0 con 0x5594e76f37b0 2024-04-17T07:31:37.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.357+0000 7f889a000640 1 -- 172.21.15.86:0/1028998843 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f88a00c70f0 con 0x5594e76f37b0 2024-04-17T07:31:37.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.357+0000 7f88b9da9740 1 -- 172.21.15.86:0/1028998843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a8012450 con 0x5594e76f37b0 2024-04-17T07:31:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.358+0000 7f889a000640 1 -- 172.21.15.86:0/1028998843 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88a00c7580 con 0x5594e76f37b0 2024-04-17T07:31:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.358+0000 7f88b9da9740 10 client.38387 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.358+0000 7f88b9da9740 1 -- 172.21.15.86:0/1028998843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594e75dc7f0 con 0x5594e76f37b0 2024-04-17T07:31:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.358+0000 7f889a000640 1 -- 172.21.15.86:0/1028998843 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88a8012450 con 0x5594e76f37b0 2024-04-17T07:31:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.359+0000 7f889a000640 1 -- 172.21.15.86:0/1028998843 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f88a00ba660 con 0x5594e76f37b0 2024-04-17T07:31:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.359+0000 7f88b9da9740 10 client.38387 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.359+0000 7f88b9da9740 10 client.38387 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:31:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.359+0000 7f88b9da9740 20 client.38387 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.359+0000 7f88b9da9740 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594e77097e0 0x5594e770bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.359+0000 7f88b9da9740 4 client.38387 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:37.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.359+0000 7f88b9da9740 1 -- 172.21.15.86:0/1028998843 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594e765ea40 con 0x5594e77097e0 2024-04-17T07:31:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.360+0000 7f88af400640 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594e77097e0 0x5594e770bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.361+0000 7f88af400640 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594e77097e0 0x5594e770bca0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6157e89a9f71f841 server_cookie=9e7effe04da84684 in_seq=0 out_seq=0 2024-04-17T07:31:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.361+0000 7f889a000640 10 client.38387 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:31:37.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.363+0000 7f889a000640 1 -- 172.21.15.86:0/1028998843 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594e765ea40 con 0x5594e77097e0 2024-04-17T07:31:37.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.363+0000 7f889a000640 10 client.38387 handle_command_reply: tid=0 2024-04-17T07:31:37.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.363+0000 7f88b9da9740 1 client.38387 shutdown 2024-04-17T07:31:37.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.363+0000 7f88b9da9740 1 -- 172.21.15.86:0/1028998843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f37b0 msgr2=0x5594e76fdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.363+0000 7f88b9da9740 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f37b0 0x5594e76fdd30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f88a00be610 tx=0x7f88a00035f0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.364+0000 7f88b9da9740 1 -- 172.21.15.86:0/1028998843 shutdown_connections 2024-04-17T07:31:37.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.364+0000 7f88b9da9740 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e77052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.364+0000 7f88b9da9740 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f41a0 0x5594e76fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.364+0000 7f88b9da9740 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594e77097e0 0x5594e770bca0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.364+0000 7f88b9da9740 1 --2- 172.21.15.86:0/1028998843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f37b0 0x5594e76fdd30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.364+0000 7f88b9da9740 1 -- 172.21.15.86:0/1028998843 >> 172.21.15.86:0/1028998843 conn(0x5594e76e6510 msgr2=0x5594e76e5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.364+0000 7f889a000640 0 client.38387 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:31:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.365+0000 7f88b9da9740 1 -- 172.21.15.86:0/1028998843 shutdown_connections 2024-04-17T07:31:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.365+0000 7f88b9da9740 1 -- 172.21.15.86:0/1028998843 wait complete. 2024-04-17T07:31:37.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.366+0000 7f88b9da9740 20 client.38387 trim_cache size 0 max 16384 2024-04-17T07:31:37.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.409+0000 7f88b9da9740 1 Processor -- start 2024-04-17T07:31:37.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.409+0000 7f88b9da9740 1 -- start start 2024-04-17T07:31:37.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.409+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f37b0 0x5594e76fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.409+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e7705240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.409+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e76fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.409+0000 7f88b9da9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f889000fc30 con 0x5594e76f37b0 2024-04-17T07:31:37.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.409+0000 7f88b9da9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8890011770 con 0x5594e76f41a0 2024-04-17T07:31:37.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.409+0000 7f88b9da9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8890010300 con 0x5594e76f9310 2024-04-17T07:31:37.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.411+0000 7f88af400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e7705240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.411+0000 7f88af400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e7705240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36486/0 (socket says 172.21.15.86:36486) 2024-04-17T07:31:37.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.412+0000 7f88af400640 1 -- 172.21.15.86:0/3451093517 learned_addr learned my addr 172.21.15.86:0/3451093517 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:37.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.412+0000 7f88b5000640 1 --2- 172.21.15.86:0/3451093517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e76fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.412+0000 7f88af400640 1 -- 172.21.15.86:0/3451093517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 msgr2=0x5594e76fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.412+0000 7f88af400640 1 --2- 172.21.15.86:0/3451093517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e76fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.412+0000 7f88af400640 1 -- 172.21.15.86:0/3451093517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f37b0 msgr2=0x5594e76fd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.412+0000 7f88af400640 1 --2- 172.21.15.86:0/3451093517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f37b0 0x5594e76fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.412+0000 7f88af400640 1 -- 172.21.15.86:0/3451093517 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594e73633f0 con 0x5594e76f41a0 2024-04-17T07:31:37.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.413+0000 7f88b5000640 1 --2- 172.21.15.86:0/3451093517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e76fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:37.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.413+0000 7f88af400640 1 --2- 172.21.15.86:0/3451093517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e7705240 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f88b0071aa0 tx=0x7f889c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ec8feb52092ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.414+0000 7f88a4c00640 1 -- 172.21.15.86:0/3451093517 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c0265b0 con 0x5594e76f41a0 2024-04-17T07:31:37.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.414+0000 7f88a4c00640 1 -- 172.21.15.86:0/3451093517 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f889c026750 con 0x5594e76f41a0 2024-04-17T07:31:37.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.414+0000 7f88a4c00640 1 -- 172.21.15.86:0/3451093517 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c0259f0 con 0x5594e76f41a0 2024-04-17T07:31:37.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.418+0000 7f88b9da9740 1 -- 172.21.15.86:0/3451093517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 msgr2=0x5594e7705240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.418+0000 7f88b9da9740 1 --2- 172.21.15.86:0/3451093517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e7705240 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f88b0071aa0 tx=0x7f889c016000 comp rx=0 tx=0).stop 2024-04-17T07:31:37.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.418+0000 7f88b9da9740 1 -- 172.21.15.86:0/3451093517 shutdown_connections 2024-04-17T07:31:37.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.418+0000 7f88b9da9740 1 --2- 172.21.15.86:0/3451093517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e76fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.418+0000 7f88b9da9740 1 --2- 172.21.15.86:0/3451093517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e7705240 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.418+0000 7f88b9da9740 1 --2- 172.21.15.86:0/3451093517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f37b0 0x5594e76fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.418+0000 7f88b9da9740 1 -- 172.21.15.86:0/3451093517 >> 172.21.15.86:0/3451093517 conn(0x5594e76e6510 msgr2=0x5594e76f5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.418+0000 7f88b9da9740 1 -- 172.21.15.86:0/3451093517 shutdown_connections 2024-04-17T07:31:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.419+0000 7f88b9da9740 1 -- 172.21.15.86:0/3451093517 wait complete. 2024-04-17T07:31:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.419+0000 7f88b9da9740 1 -- start start 2024-04-17T07:31:37.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.419+0000 7f88b9da9740 1 Processor -- start 2024-04-17T07:31:37.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.421+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f37b0 0x5594e7702100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.421+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e7702640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.422+0000 7f88b9da9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e776ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.422+0000 7f88afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f37b0 0x5594e7702100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.422+0000 7f88afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f37b0 0x5594e7702100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56648/0 (socket says 172.21.15.86:56648) 2024-04-17T07:31:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.422+0000 7f88b9da9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88900110a0 con 0x5594e76f37b0 2024-04-17T07:31:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.422+0000 7f88b9da9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f88900109d0 con 0x5594e76f41a0 2024-04-17T07:31:37.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.422+0000 7f88b9da9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f889000dc00 con 0x5594e76f9310 2024-04-17T07:31:37.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.422+0000 7f88afe00640 1 -- 172.21.15.86:0/3325889803 learned_addr learned my addr 172.21.15.86:0/3325889803 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:37.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.422+0000 7f88b9da9740 10 client.0 _ll_register_callbacks cb 0x7f88a00be500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:37.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.422+0000 7f88af400640 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e7702640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.423+0000 7f88b5000640 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e776ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.423+0000 7f88b5000640 1 -- 172.21.15.86:0/3325889803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 msgr2=0x5594e7702640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.423+0000 7f88b5000640 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e7702640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.423+0000 7f88b5000640 1 -- 172.21.15.86:0/3325889803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f37b0 msgr2=0x5594e7702100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.423+0000 7f88b5000640 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f37b0 0x5594e7702100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.423+0000 7f88b5000640 1 -- 172.21.15.86:0/3325889803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594e742d7d0 con 0x5594e76f9310 2024-04-17T07:31:37.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.425+0000 7f88b5000640 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e776ddd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5594e7705870 tx=0x7f88a8011f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9761afaabf24744 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.425+0000 7f889a000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:31:37.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.425+0000 7f889a000640 1 -- 172.21.15.86:0/3325889803 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8029020 con 0x5594e76f9310 2024-04-17T07:31:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.426+0000 7f889a000640 1 -- 172.21.15.86:0/3325889803 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f88a8020040 con 0x5594e76f9310 2024-04-17T07:31:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.426+0000 7f88b9da9740 1 -- 172.21.15.86:0/3325889803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f889c026750 con 0x5594e76f9310 2024-04-17T07:31:37.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.426+0000 7f889a000640 1 -- 172.21.15.86:0/3325889803 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88a8034050 con 0x5594e76f9310 2024-04-17T07:31:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.427+0000 7f88b9da9740 10 client.28420 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.427+0000 7f88b9da9740 1 -- 172.21.15.86:0/3325889803 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594e776f320 con 0x5594e76f9310 2024-04-17T07:31:37.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.427+0000 7f889a000640 1 -- 172.21.15.86:0/3325889803 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f889c026750 con 0x5594e76f9310 2024-04-17T07:31:37.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.428+0000 7f889a000640 1 -- 172.21.15.86:0/3325889803 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f88a800f040 con 0x5594e76f9310 2024-04-17T07:31:37.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.428+0000 7f88b9da9740 10 client.28420 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:37.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.428+0000 7f88b9da9740 10 client.28420 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:31:37.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.428+0000 7f88b9da9740 20 client.28420 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.428+0000 7f88b9da9740 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594e776fa20 0x5594e7771e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.428+0000 7f88b9da9740 4 client.28420 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:31:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.428+0000 7f88b9da9740 1 -- 172.21.15.86:0/3325889803 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594e76588b0 con 0x5594e776fa20 2024-04-17T07:31:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.429+0000 7f88afe00640 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594e776fa20 0x5594e7771e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.429+0000 7f889a000640 10 client.28420 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:31:37.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.430+0000 7f88afe00640 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594e776fa20 0x5594e7771e40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79e509e4f046d079 server_cookie=647ef58f045cd7aa in_seq=0 out_seq=0 2024-04-17T07:31:37.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.430+0000 7f889a000640 1 -- 172.21.15.86:0/3325889803 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594e76588b0 con 0x5594e776fa20 2024-04-17T07:31:37.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.430+0000 7f889a000640 10 client.28420 handle_command_reply: tid=0 2024-04-17T07:31:37.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.430+0000 7f88b9da9740 1 client.28420 shutdown 2024-04-17T07:31:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.430+0000 7f88b9da9740 1 -- 172.21.15.86:0/3325889803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 msgr2=0x5594e776ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.430+0000 7f88b9da9740 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e776ddd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x5594e7705870 tx=0x7f88a8011f70 comp rx=0 tx=0).stop 2024-04-17T07:31:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.431+0000 7f88b9da9740 1 -- 172.21.15.86:0/3325889803 shutdown_connections 2024-04-17T07:31:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.431+0000 7f88b9da9740 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5594e76f9310 0x5594e776ddd0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.431+0000 7f88b9da9740 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5594e776fa20 0x5594e7771e40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.431+0000 7f88b9da9740 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5594e76f41a0 0x5594e7702640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.431+0000 7f88b9da9740 1 --2- 172.21.15.86:0/3325889803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5594e76f37b0 0x5594e7702100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.431+0000 7f88b9da9740 1 -- 172.21.15.86:0/3325889803 >> 172.21.15.86:0/3325889803 conn(0x5594e76e6510 msgr2=0x5594e76fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.431+0000 7f889a000640 0 client.28420 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:31:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.431+0000 7f88b9da9740 1 -- 172.21.15.86:0/3325889803 shutdown_connections 2024-04-17T07:31:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.431+0000 7f88b9da9740 1 -- 172.21.15.86:0/3325889803 wait complete. 2024-04-17T07:31:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.431+0000 7f88b9da9740 20 client.28420 trim_cache size 0 max 16384 2024-04-17T07:31:37.432 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:31:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.433+0000 7f88b7000640 1 -- 172.21.15.86:0/1192304378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88940a3cb0 msgr2=0x7f88940a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.433+0000 7f88b7000640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88940a3cb0 0x7f88940a6170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f889c005fd0 tx=0x7f889c005850 comp rx=0 tx=0).stop 2024-04-17T07:31:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.433+0000 7f88b7000640 1 -- 172.21.15.86:0/1192304378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b0101fb0 msgr2=0x7f88b0070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.433+0000 7f88b7000640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b0101fb0 0x7f88b0070980 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f88a002c620 tx=0x7f88a00040c0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.435+0000 7f88b7000640 1 -- 172.21.15.86:0/1192304378 shutdown_connections 2024-04-17T07:31:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.435+0000 7f88b7000640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88940a3cb0 0x7f88940a6170 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.435+0000 7f88b7000640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88b0103b60 0x7f88b0197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.435+0000 7f88b7000640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88b01031a0 0x7f88b0070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.435+0000 7f88b7000640 1 --2- 172.21.15.86:0/1192304378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b0101fb0 0x7f88b0070980 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.435+0000 7f88b7000640 1 -- 172.21.15.86:0/1192304378 >> 172.21.15.86:0/1192304378 conn(0x7f88b00fd540 msgr2=0x7f88b0106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.435+0000 7f88b7000640 1 -- 172.21.15.86:0/1192304378 shutdown_connections 2024-04-17T07:31:37.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.436+0000 7f88b7000640 1 -- 172.21.15.86:0/1192304378 wait complete. 2024-04-17T07:31:37.472 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:31:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:37 smithi086 ceph-mon[31733]: pgmap v957: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 12 MiB/s wr, 689 op/s 2024-04-17T07:31:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:37 smithi098 ceph-mon[35418]: pgmap v957: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 12 MiB/s wr, 689 op/s 2024-04-17T07:31:37.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:37 smithi103 ceph-mon[35495]: pgmap v957: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 12 MiB/s wr, 689 op/s 2024-04-17T07:31:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.809+0000 7ff1eba00640 1 -- 172.21.15.86:0/3659951694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4105d70 msgr2=0x7ff1e410c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.809+0000 7ff1eba00640 1 --2- 172.21.15.86:0/3659951694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4105d70 0x7ff1e410c600 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009be0 tx=0x7ff1dc02bef0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.809+0000 7ff1eba00640 1 -- 172.21.15.86:0/3659951694 shutdown_connections 2024-04-17T07:31:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.809+0000 7ff1eba00640 1 --2- 172.21.15.86:0/3659951694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4105d70 0x7ff1e410c600 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.809+0000 7ff1eba00640 1 --2- 172.21.15.86:0/3659951694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41053b0 0x7ff1e4105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.809+0000 7ff1eba00640 1 --2- 172.21.15.86:0/3659951694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e41041c0 0x7ff1e41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.809+0000 7ff1eba00640 1 -- 172.21.15.86:0/3659951694 >> 172.21.15.86:0/3659951694 conn(0x7ff1e40ff750 msgr2=0x7ff1e4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.809+0000 7ff1eba00640 1 -- 172.21.15.86:0/3659951694 shutdown_connections 2024-04-17T07:31:37.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.810+0000 7ff1eba00640 1 -- 172.21.15.86:0/3659951694 wait complete. 2024-04-17T07:31:37.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.812+0000 7ff1eba00640 1 Processor -- start 2024-04-17T07:31:37.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.812+0000 7ff1eba00640 1 -- start start 2024-04-17T07:31:37.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.812+0000 7ff1eba00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41041c0 0x7ff1e4070940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.812+0000 7ff1eba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e41053b0 0x7ff1e4070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.813+0000 7ff1eba00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4105d70 0x7ff1e41aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.813+0000 7ff1eba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410fdb0 con 0x7ff1e41053b0 2024-04-17T07:31:37.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.813+0000 7ff1eba00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410fc30 con 0x7ff1e41041c0 2024-04-17T07:31:37.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.813+0000 7ff1eba00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410ff30 con 0x7ff1e4105d70 2024-04-17T07:31:37.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.813+0000 7ff1e9000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41041c0 0x7ff1e4070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.813+0000 7ff1e9000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41041c0 0x7ff1e4070940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36520/0 (socket says 172.21.15.86:36520) 2024-04-17T07:31:37.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.813+0000 7ff1e9000640 1 -- 172.21.15.86:0/3546502931 learned_addr learned my addr 172.21.15.86:0/3546502931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:37.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.813+0000 7ff1e9a00640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4105d70 0x7ff1e41aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.814+0000 7ff1e9000640 1 -- 172.21.15.86:0/3546502931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4105d70 msgr2=0x7ff1e41aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.814+0000 7ff1e9000640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4105d70 0x7ff1e41aa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.814+0000 7ff1e9000640 1 -- 172.21.15.86:0/3546502931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e41053b0 msgr2=0x7ff1e4070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.814+0000 7ff1e9000640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e41053b0 0x7ff1e4070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.814+0000 7ff1e9000640 1 -- 172.21.15.86:0/3546502931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e41aaa00 con 0x7ff1e41041c0 2024-04-17T07:31:37.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.815+0000 7ff1e9000640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41041c0 0x7ff1e4070940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800b450 tx=0x7ff1d800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.816+0000 7ff1e1600640 1 -- 172.21.15.86:0/3546502931 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8015020 con 0x7ff1e41041c0 2024-04-17T07:31:37.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.816+0000 7ff1e1600640 1 -- 172.21.15.86:0/3546502931 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d8011d40 con 0x7ff1e41041c0 2024-04-17T07:31:37.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.816+0000 7ff1e1600640 1 -- 172.21.15.86:0/3546502931 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d800f340 con 0x7ff1e41041c0 2024-04-17T07:31:37.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.816+0000 7ff1eba00640 1 -- 172.21.15.86:0/3546502931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e41aabd0 con 0x7ff1e41041c0 2024-04-17T07:31:37.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.816+0000 7ff1eba00640 1 -- 172.21.15.86:0/3546502931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e41ab070 con 0x7ff1e41041c0 2024-04-17T07:31:37.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.822+0000 7ff1ee797740 1 Processor -- start 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.822+0000 7ff1ee797740 1 -- start start 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.822+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2d7b0 0x55de17d2dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.822+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2e1a0 0x55de17d32ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.822+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d33310 0x55de17d357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.822+0000 7ff1ee797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de17c8b020 con 0x55de17d33310 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.822+0000 7ff1ee797740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55de17c31460 con 0x55de17d2d7b0 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.822+0000 7ff1ee797740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55de17c86fe0 con 0x55de17d2e1a0 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.823+0000 7ff1e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d33310 0x55de17d357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.823+0000 7ff1e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d33310 0x55de17d357a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56676/0 (socket says 172.21.15.86:56676) 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.823+0000 7ff1e3e00640 1 -- 172.21.15.86:0/1536345868 learned_addr learned my addr 172.21.15.86:0/1536345868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.824+0000 7ff1e3e00640 1 -- 172.21.15.86:0/1536345868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2e1a0 msgr2=0x55de17d32ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.824+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/1536345868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2e1a0 0x55de17d32ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.824+0000 7ff1e3e00640 1 -- 172.21.15.86:0/1536345868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2d7b0 msgr2=0x55de17d2dbd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:31:37.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.824+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/1536345868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2d7b0 0x55de17d2dbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.824+0000 7ff1e3e00640 1 -- 172.21.15.86:0/1536345868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de17c39040 con 0x55de17d33310 2024-04-17T07:31:37.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.825+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/1536345868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d33310 0x55de17d357a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0004650 tx=0x7ff1d000e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6c9616f432ed78 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.827+0000 7ff1d6000640 1 -- 172.21.15.86:0/1536345868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000ed80 con 0x55de17d33310 2024-04-17T07:31:37.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.827+0000 7ff1e1600640 1 -- 172.21.15.86:0/3546502931 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff1d8018020 con 0x7ff1e41041c0 2024-04-17T07:31:37.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.828+0000 7ff1d6000640 1 -- 172.21.15.86:0/1536345868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d00054d0 con 0x55de17d33310 2024-04-17T07:31:37.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.829+0000 7ff1ee797740 1 -- 172.21.15.86:0/1536345868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d33310 msgr2=0x55de17d357a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.829+0000 7ff1ee797740 1 --2- 172.21.15.86:0/1536345868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d33310 0x55de17d357a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0004650 tx=0x7ff1d000e350 comp rx=0 tx=0).stop 2024-04-17T07:31:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.830+0000 7ff1ee797740 1 -- 172.21.15.86:0/1536345868 shutdown_connections 2024-04-17T07:31:37.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.830+0000 7ff1ee797740 1 --2- 172.21.15.86:0/1536345868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d33310 0x55de17d357a0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.830+0000 7ff1ee797740 1 --2- 172.21.15.86:0/1536345868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2e1a0 0x55de17d32ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.830+0000 7ff1ee797740 1 --2- 172.21.15.86:0/1536345868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2d7b0 0x55de17d2dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.830+0000 7ff1ee797740 1 -- 172.21.15.86:0/1536345868 >> 172.21.15.86:0/1536345868 conn(0x55de17d20510 msgr2=0x55de17d20910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.830+0000 7ff1e1600640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3d80 0x7ff1cc0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.831+0000 7ff1e1600640 1 -- 172.21.15.86:0/3546502931 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff1d80d01e0 con 0x7ff1e41041c0 2024-04-17T07:31:37.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.831+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3d80 0x7ff1cc0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.831+0000 7ff1ee797740 1 -- 172.21.15.86:0/1536345868 shutdown_connections 2024-04-17T07:31:37.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.832+0000 7ff1ee797740 1 -- 172.21.15.86:0/1536345868 wait complete. 2024-04-17T07:31:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.832+0000 7ff1ee797740 1 -- start start 2024-04-17T07:31:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.833+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3d80 0x7ff1cc0a6240 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0004700 tx=0x7ff1d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.833+0000 7ff1ee797740 1 Processor -- start 2024-04-17T07:31:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.834+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.834+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2e1a0 0x55de17d26f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.834+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d33310 0x55de17d2bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.834+0000 7ff1ee797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de17d250f0 con 0x55de17d2d7b0 2024-04-17T07:31:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.834+0000 7ff1ee797740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55de17c44830 con 0x55de17d33310 2024-04-17T07:31:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.834+0000 7ff1ee797740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55de17d25270 con 0x55de17d2e1a0 2024-04-17T07:31:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.835+0000 7ff1e9a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d33310 0x55de17d2bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.834+0000 7ff1ee797740 10 client.0 _ll_register_callbacks cb 0x55de17c12fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:37.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.835+0000 7ff1e9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.835+0000 7ff1e9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56684/0 (socket says 172.21.15.86:56684) 2024-04-17T07:31:37.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.835+0000 7ff1e9000640 1 -- 172.21.15.86:0/2323442768 learned_addr learned my addr 172.21.15.86:0/2323442768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:37.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.836+0000 7ff1e9000640 1 -- 172.21.15.86:0/2323442768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2e1a0 msgr2=0x55de17d26f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.836+0000 7ff1e9000640 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2e1a0 0x55de17d26f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.836+0000 7ff1e9000640 1 -- 172.21.15.86:0/2323442768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d33310 msgr2=0x55de17d2bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.836+0000 7ff1e9000640 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d33310 0x55de17d2bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.836+0000 7ff1e9000640 1 -- 172.21.15.86:0/2323442768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de17c48920 con 0x55de17d2d7b0 2024-04-17T07:31:37.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.837+0000 7ff1e9000640 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26a10 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800be10 tx=0x7ff1d809c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b88342e6813a14e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.837+0000 7ff1caa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:37.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.838+0000 7ff1caa00640 1 -- 172.21.15.86:0/2323442768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80a4d50 con 0x55de17d2d7b0 2024-04-17T07:31:37.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.838+0000 7ff1caa00640 1 -- 172.21.15.86:0/2323442768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff1d80a0170 con 0x55de17d2d7b0 2024-04-17T07:31:37.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.838+0000 7ff1ee797740 1 -- 172.21.15.86:0/2323442768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1d00054d0 con 0x55de17d2d7b0 2024-04-17T07:31:37.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.839+0000 7ff1caa00640 1 -- 172.21.15.86:0/2323442768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff1d8010e80 con 0x55de17d2d7b0 2024-04-17T07:31:37.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.839+0000 7ff1ee797740 10 client.19398 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:37.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.839+0000 7ff1ee797740 1 -- 172.21.15.86:0/2323442768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de17c167f0 con 0x55de17d2d7b0 2024-04-17T07:31:37.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.840+0000 7ff1caa00640 1 -- 172.21.15.86:0/2323442768 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d00054d0 con 0x55de17d2d7b0 2024-04-17T07:31:37.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.840+0000 7ff1caa00640 1 -- 172.21.15.86:0/2323442768 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff1d809b530 con 0x55de17d2d7b0 2024-04-17T07:31:37.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.841+0000 7ff1ee797740 10 client.19398 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:37.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.841+0000 7ff1ee797740 10 client.19398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:37.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.841+0000 7ff1ee797740 20 client.19398 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:37.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.841+0000 7ff1ee797740 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de17d4bf00 0x55de17d4e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.841+0000 7ff1ee797740 4 client.19398 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:37.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.841+0000 7ff1ee797740 1 -- 172.21.15.86:0/2323442768 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de17c98a40 con 0x55de17d4bf00 2024-04-17T07:31:37.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.842+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de17d4bf00 0x55de17d4e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.844+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de17d4bf00 0x55de17d4e390 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b304559efd31c server_cookie=dfdaf065be512598 in_seq=0 out_seq=0 2024-04-17T07:31:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.844+0000 7ff1caa00640 10 client.19398 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.845+0000 7ff1caa00640 1 -- 172.21.15.86:0/2323442768 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de17c98a40 con 0x55de17d4bf00 2024-04-17T07:31:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.845+0000 7ff1caa00640 10 client.19398 handle_command_reply: tid=0 2024-04-17T07:31:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.845+0000 7ff1ee797740 1 client.19398 shutdown 2024-04-17T07:31:37.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.847+0000 7ff1ee797740 1 -- 172.21.15.86:0/2323442768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 msgr2=0x55de17d26a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.847+0000 7ff1ee797740 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26a10 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800be10 tx=0x7ff1d809c7e0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.848+0000 7ff1ee797740 1 -- 172.21.15.86:0/2323442768 shutdown_connections 2024-04-17T07:31:37.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.848+0000 7ff1ee797740 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de17d4bf00 0x55de17d4e390 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.848+0000 7ff1ee797740 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d33310 0x55de17d2bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.848+0000 7ff1ee797740 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2e1a0 0x55de17d26f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.848+0000 7ff1ee797740 1 --2- 172.21.15.86:0/2323442768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26a10 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.848+0000 7ff1ee797740 1 -- 172.21.15.86:0/2323442768 >> 172.21.15.86:0/2323442768 conn(0x55de17d20510 msgr2=0x55de17d1f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.848+0000 7ff1caa00640 0 client.19398 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:37.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.848+0000 7ff1ee797740 1 -- 172.21.15.86:0/2323442768 shutdown_connections 2024-04-17T07:31:37.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.849+0000 7ff1ee797740 1 -- 172.21.15.86:0/2323442768 wait complete. 2024-04-17T07:31:37.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.849+0000 7ff1ee797740 20 client.19398 trim_cache size 0 max 16384 2024-04-17T07:31:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.899+0000 7ff1ee797740 1 Processor -- start 2024-04-17T07:31:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.899+0000 7ff1ee797740 1 -- start start 2024-04-17T07:31:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.899+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.899+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2e1a0 0x55de17d1ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.899+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d33310 0x55de17d1f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1ee797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41b11c0 con 0x55de17d2d7b0 2024-04-17T07:31:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1ee797740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41b2d00 con 0x55de17d2e1a0 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1ee797740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41b1890 con 0x55de17d33310 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2e1a0 0x55de17d1ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2e1a0 0x55de17d1ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36552/0 (socket says 172.21.15.86:36552) 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e3e00640 1 -- 172.21.15.86:0/3732080330 learned_addr learned my addr 172.21.15.86:0/3732080330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e9000640 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e9a00640 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d33310 0x55de17d1f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e3e00640 1 -- 172.21.15.86:0/3732080330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d33310 msgr2=0x55de17d1f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d33310 0x55de17d1f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e3e00640 1 -- 172.21.15.86:0/3732080330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 msgr2=0x55de17d26c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e3e00640 1 -- 172.21.15.86:0/3732080330 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de1799d3f0 con 0x55de17d2e1a0 2024-04-17T07:31:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.900+0000 7ff1e9000640 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:31:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.901+0000 7ff1e9a00640 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d33310 0x55de17d1f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.901+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2e1a0 0x55de17d1ee80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0004620 tx=0x7ff1d0005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ee51dda2ca63c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.902+0000 7ff1d5600640 1 -- 172.21.15.86:0/3732080330 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d002a070 con 0x55de17d2e1a0 2024-04-17T07:31:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.902+0000 7ff1d5600640 1 -- 172.21.15.86:0/3732080330 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d0038020 con 0x55de17d2e1a0 2024-04-17T07:31:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.902+0000 7ff1d5600640 1 -- 172.21.15.86:0/3732080330 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d001d040 con 0x55de17d2e1a0 2024-04-17T07:31:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.903+0000 7ff1ee797740 1 -- 172.21.15.86:0/3732080330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2e1a0 msgr2=0x55de17d1ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.903+0000 7ff1ee797740 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2e1a0 0x55de17d1ee80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0004620 tx=0x7ff1d0005a40 comp rx=0 tx=0).stop 2024-04-17T07:31:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.903+0000 7ff1ee797740 1 -- 172.21.15.86:0/3732080330 shutdown_connections 2024-04-17T07:31:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.903+0000 7ff1ee797740 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d33310 0x55de17d1f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.903+0000 7ff1ee797740 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d2e1a0 0x55de17d1ee80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.903+0000 7ff1ee797740 1 --2- 172.21.15.86:0/3732080330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2d7b0 0x55de17d26c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.903+0000 7ff1ee797740 1 -- 172.21.15.86:0/3732080330 >> 172.21.15.86:0/3732080330 conn(0x55de17d20510 msgr2=0x55de17d25ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.903+0000 7ff1ee797740 1 -- 172.21.15.86:0/3732080330 shutdown_connections 2024-04-17T07:31:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.903+0000 7ff1ee797740 1 -- 172.21.15.86:0/3732080330 wait complete. 2024-04-17T07:31:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.904+0000 7ff1ee797740 1 -- start start 2024-04-17T07:31:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.904+0000 7ff1ee797740 1 Processor -- start 2024-04-17T07:31:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.907+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2d7b0 0x55de17d5f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.907+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2e1a0 0x55de17d5f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.907+0000 7ff1ee797740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d33310 0x55de17d5d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.907+0000 7ff1ee797740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41b2630 con 0x55de17d2e1a0 2024-04-17T07:31:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.907+0000 7ff1e9000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2d7b0 0x55de17d5f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.908+0000 7ff1e9000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2d7b0 0x55de17d5f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42252/0 (socket says 172.21.15.86:42252) 2024-04-17T07:31:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.908+0000 7ff1e9000640 1 -- 172.21.15.86:0/4071065053 learned_addr learned my addr 172.21.15.86:0/4071065053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.907+0000 7ff1e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2e1a0 0x55de17d5f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.910+0000 7ff1ee797740 1 -- 172.21.15.86:0/4071065053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41b1f60 con 0x55de17d33310 2024-04-17T07:31:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.910+0000 7ff1ee797740 1 -- 172.21.15.86:0/4071065053 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e41af190 con 0x55de17d2d7b0 2024-04-17T07:31:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.908+0000 7ff1e9a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d33310 0x55de17d5d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.910+0000 7ff1ee797740 10 client.0 _ll_register_callbacks cb 0x7ff1d809bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.910+0000 7ff1e3e00640 1 -- 172.21.15.86:0/4071065053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2d7b0 msgr2=0x55de17d5f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.910+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2d7b0 0x55de17d5f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.910+0000 7ff1e3e00640 1 -- 172.21.15.86:0/4071065053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d33310 msgr2=0x55de17d5d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.910+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d33310 0x55de17d5d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.910+0000 7ff1e3e00640 1 -- 172.21.15.86:0/4071065053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de17a677d0 con 0x55de17d2e1a0 2024-04-17T07:31:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.911+0000 7ff1e3e00640 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2e1a0 0x55de17d5f6e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x55de17d4e8a0 tx=0x7ff1d0005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b180b3082ad39b93 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:37.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.911+0000 7ff1caa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.912+0000 7ff1caa00640 1 -- 172.21.15.86:0/4071065053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0041020 con 0x55de17d2e1a0 2024-04-17T07:31:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.912+0000 7ff1caa00640 1 -- 172.21.15.86:0/4071065053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff1d0004a70 con 0x55de17d2e1a0 2024-04-17T07:31:37.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.912+0000 7ff1ee797740 1 -- 172.21.15.86:0/4071065053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1d0038020 con 0x55de17d2e1a0 2024-04-17T07:31:37.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.912+0000 7ff1caa00640 1 -- 172.21.15.86:0/4071065053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff1d002a040 con 0x55de17d2e1a0 2024-04-17T07:31:37.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.912+0000 7ff1ee797740 10 client.19404 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:37.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.912+0000 7ff1ee797740 1 -- 172.21.15.86:0/4071065053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de17db2860 con 0x55de17d2e1a0 2024-04-17T07:31:37.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.913+0000 7ff1caa00640 1 -- 172.21.15.86:0/4071065053 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d0038020 con 0x55de17d2e1a0 2024-04-17T07:31:37.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.913+0000 7ff1caa00640 1 -- 172.21.15.86:0/4071065053 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff1d00346f0 con 0x55de17d2e1a0 2024-04-17T07:31:37.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.913+0000 7ff1ee797740 10 client.19404 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:37.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.913+0000 7ff1ee797740 10 client.19404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:37.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.913+0000 7ff1ee797740 20 client.19404 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.913+0000 7ff1ee797740 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de17db3180 0x55de17db55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.913+0000 7ff1ee797740 4 client.19404 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:31:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.913+0000 7ff1ee797740 1 -- 172.21.15.86:0/4071065053 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de17c928b0 con 0x55de17db3180 2024-04-17T07:31:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.914+0000 7ff1e9000640 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de17db3180 0x55de17db55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.916+0000 7ff1caa00640 10 client.19404 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:37.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.918+0000 7ff1e9000640 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de17db3180 0x55de17db55a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4743a85c38b814f server_cookie=bfc01c4c3c06257e in_seq=0 out_seq=0 2024-04-17T07:31:37.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.919+0000 7ff1caa00640 1 -- 172.21.15.86:0/4071065053 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de17c928b0 con 0x55de17db3180 2024-04-17T07:31:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.919+0000 7ff1caa00640 10 client.19404 handle_command_reply: tid=0 2024-04-17T07:31:37.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.920+0000 7ff1ee797740 1 client.19404 shutdown 2024-04-17T07:31:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.920+0000 7ff1ee797740 1 -- 172.21.15.86:0/4071065053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2e1a0 msgr2=0x55de17d5f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.920+0000 7ff1ee797740 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2e1a0 0x55de17d5f6e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x55de17d4e8a0 tx=0x7ff1d0005550 comp rx=0 tx=0).stop 2024-04-17T07:31:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.921+0000 7ff1ee797740 1 -- 172.21.15.86:0/4071065053 shutdown_connections 2024-04-17T07:31:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.921+0000 7ff1ee797740 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55de17db3180 0x55de17db55a0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.921+0000 7ff1ee797740 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55de17d33310 0x55de17d5d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.921+0000 7ff1ee797740 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de17d2e1a0 0x55de17d5f6e0 secure :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x55de17d4e8a0 tx=0x7ff1d0005550 comp rx=0 tx=0).stop 2024-04-17T07:31:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.921+0000 7ff1ee797740 1 --2- 172.21.15.86:0/4071065053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55de17d2d7b0 0x55de17d5f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.921+0000 7ff1ee797740 1 -- 172.21.15.86:0/4071065053 >> 172.21.15.86:0/4071065053 conn(0x55de17d20510 msgr2=0x55de17d25ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.921+0000 7ff1caa00640 0 client.19404 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.921+0000 7ff1ee797740 1 -- 172.21.15.86:0/4071065053 shutdown_connections 2024-04-17T07:31:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.922+0000 7ff1ee797740 1 -- 172.21.15.86:0/4071065053 wait complete. 2024-04-17T07:31:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.923+0000 7ff1ee797740 20 client.19404 trim_cache size 0 max 16384 2024-04-17T07:31:37.924 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:31:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.925+0000 7ff1eba00640 1 -- 172.21.15.86:0/3546502931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3d80 msgr2=0x7ff1cc0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.925+0000 7ff1eba00640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3d80 0x7ff1cc0a6240 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0004700 tx=0x7ff1d0009260 comp rx=0 tx=0).stop 2024-04-17T07:31:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.925+0000 7ff1eba00640 1 -- 172.21.15.86:0/3546502931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41041c0 msgr2=0x7ff1e4070940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.925+0000 7ff1eba00640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41041c0 0x7ff1e4070940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800b450 tx=0x7ff1d800b910 comp rx=0 tx=0).stop 2024-04-17T07:31:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.926+0000 7ff1eba00640 1 -- 172.21.15.86:0/3546502931 shutdown_connections 2024-04-17T07:31:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.926+0000 7ff1eba00640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3d80 0x7ff1cc0a6240 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.926+0000 7ff1eba00640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4105d70 0x7ff1e41aa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.926+0000 7ff1eba00640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e41053b0 0x7ff1e4070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.926+0000 7ff1eba00640 1 --2- 172.21.15.86:0/3546502931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41041c0 0x7ff1e4070940 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:37.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.926+0000 7ff1eba00640 1 -- 172.21.15.86:0/3546502931 >> 172.21.15.86:0/3546502931 conn(0x7ff1e40ff750 msgr2=0x7ff1e4108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.926+0000 7ff1eba00640 1 -- 172.21.15.86:0/3546502931 shutdown_connections 2024-04-17T07:31:37.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:37.927+0000 7ff1eba00640 1 -- 172.21.15.86:0/3546502931 wait complete. 2024-04-17T07:31:38.971 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:31:38.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:31:38.971 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:31:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.328+0000 7f8821600640 1 -- 172.21.15.86:0/99395805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f881c1041c0 msgr2=0x7f881c1045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.328+0000 7f8821600640 1 --2- 172.21.15.86:0/99395805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f881c1041c0 0x7f881c1045e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f880c009900 tx=0x7f880c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:31:39.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.333+0000 7f8821600640 1 -- 172.21.15.86:0/99395805 shutdown_connections 2024-04-17T07:31:39.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.333+0000 7f8821600640 1 --2- 172.21.15.86:0/99395805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c105d70 0x7f881c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.333+0000 7f8821600640 1 --2- 172.21.15.86:0/99395805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881c1053b0 0x7f881c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.333+0000 7f8821600640 1 --2- 172.21.15.86:0/99395805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f881c1041c0 0x7f881c1045e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.333+0000 7f8821600640 1 -- 172.21.15.86:0/99395805 >> 172.21.15.86:0/99395805 conn(0x7f881c0ff750 msgr2=0x7f881c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.334+0000 7f8821600640 1 -- 172.21.15.86:0/99395805 shutdown_connections 2024-04-17T07:31:39.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.335+0000 7f8821600640 1 -- 172.21.15.86:0/99395805 wait complete. 2024-04-17T07:31:39.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.337+0000 7f8821600640 1 Processor -- start 2024-04-17T07:31:39.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.337+0000 7f8821600640 1 -- start start 2024-04-17T07:31:39.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.337+0000 7f8821600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881c1041c0 0x7f881c0709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.337+0000 7f8821600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f881c1053b0 0x7f881c070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.337+0000 7f8821600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c105d70 0x7f881c1aa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.337+0000 7f8821600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f881c10fd50 con 0x7f881c105d70 2024-04-17T07:31:39.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.337+0000 7f8821600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f881c10fbd0 con 0x7f881c1041c0 2024-04-17T07:31:39.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.337+0000 7f8821600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f881c10fed0 con 0x7f881c1053b0 2024-04-17T07:31:39.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.338+0000 7f881a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f881c1053b0 0x7f881c070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.338+0000 7f881a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f881c1053b0 0x7f881c070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42268/0 (socket says 172.21.15.86:42268) 2024-04-17T07:31:39.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.338+0000 7f881a000640 1 -- 172.21.15.86:0/1515707867 learned_addr learned my addr 172.21.15.86:0/1515707867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:39.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.338+0000 7f881aa00640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881c1041c0 0x7f881c0709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.339+0000 7f881b400640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c105d70 0x7f881c1aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.339+0000 7f881a000640 1 -- 172.21.15.86:0/1515707867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881c1041c0 msgr2=0x7f881c0709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.339+0000 7f881a000640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881c1041c0 0x7f881c0709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.339+0000 7f881a000640 1 -- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c105d70 msgr2=0x7f881c1aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.339+0000 7f881a000640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c105d70 0x7f881c1aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.339+0000 7f881a000640 1 -- 172.21.15.86:0/1515707867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c1aa960 con 0x7f881c1053b0 2024-04-17T07:31:39.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.339+0000 7f881b400640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c105d70 0x7f881c1aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:31:39.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.340+0000 7f881a000640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f881c1053b0 0x7f881c070ee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f880800e9f0 tx=0x7f880800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.341+0000 7f8813400640 1 -- 172.21.15.86:0/1515707867 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880800cd80 con 0x7f881c1053b0 2024-04-17T07:31:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.342+0000 7f8813400640 1 -- 172.21.15.86:0/1515707867 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88080108e0 con 0x7f881c1053b0 2024-04-17T07:31:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.342+0000 7f8821600640 1 -- 172.21.15.86:0/1515707867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c0763a0 con 0x7f881c1053b0 2024-04-17T07:31:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.342+0000 7f8813400640 1 -- 172.21.15.86:0/1515707867 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808018920 con 0x7f881c1053b0 2024-04-17T07:31:39.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.342+0000 7f8821600640 1 -- 172.21.15.86:0/1515707867 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c076940 con 0x7f881c1053b0 2024-04-17T07:31:39.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.346+0000 7f88243dc740 1 Processor -- start 2024-04-17T07:31:39.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.348+0000 7f88243dc740 1 -- start start 2024-04-17T07:31:39.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.348+0000 7f88243dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f5cec0 0x561be8f5d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.351+0000 7f88243dc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5d8b0 0x561be8f623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.351+0000 7f88243dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f62a20 0x561be8f64eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.351+0000 7f88243dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561be8ec2cb0 con 0x561be8f5cec0 2024-04-17T07:31:39.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.351+0000 7f88243dc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561be8e69370 con 0x561be8f5d8b0 2024-04-17T07:31:39.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.351+0000 7f88243dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561be8ebec70 con 0x561be8f62a20 2024-04-17T07:31:39.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.352+0000 7f881aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5d8b0 0x561be8f623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.352+0000 7f881aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5d8b0 0x561be8f623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36582/0 (socket says 172.21.15.86:36582) 2024-04-17T07:31:39.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.352+0000 7f881aa00640 1 -- 172.21.15.86:0/2861429263 learned_addr learned my addr 172.21.15.86:0/2861429263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:39.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.352+0000 7f8813400640 1 -- 172.21.15.86:0/1515707867 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8808010a90 con 0x7f881c1053b0 2024-04-17T07:31:39.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.352+0000 7f881b400640 1 --2- 172.21.15.86:0/2861429263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f5cec0 0x561be8f5d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.353+0000 7f881b400640 1 -- 172.21.15.86:0/2861429263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f62a20 msgr2=0x561be8f64eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:31:39.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.353+0000 7f881b400640 1 --2- 172.21.15.86:0/2861429263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f62a20 0x561be8f64eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.353+0000 7f881b400640 1 -- 172.21.15.86:0/2861429263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5d8b0 msgr2=0x561be8f623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.353+0000 7f881b400640 1 --2- 172.21.15.86:0/2861429263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5d8b0 0x561be8f623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.353+0000 7f881b400640 1 -- 172.21.15.86:0/2861429263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be8e88910 con 0x561be8f5cec0 2024-04-17T07:31:39.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.354+0000 7f8813400640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88000a3c30 0x7f88000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.355+0000 7f881aa00640 1 --2- 172.21.15.86:0/2861429263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5d8b0 0x561be8f623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:39.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.355+0000 7f8813400640 1 -- 172.21.15.86:0/1515707867 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f880809aa90 con 0x7f881c1053b0 2024-04-17T07:31:39.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.356+0000 7f881aa00640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88000a3c30 0x7f88000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.357+0000 7f881b400640 1 --2- 172.21.15.86:0/2861429263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f5cec0 0x561be8f5d2e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f8814009ee0 tx=0x7f8814010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1668eac42ab4d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.358+0000 7f8807400640 1 -- 172.21.15.86:0/2861429263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814016070 con 0x561be8f5cec0 2024-04-17T07:31:39.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.358+0000 7f8807400640 1 -- 172.21.15.86:0/2861429263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f881401bb10 con 0x561be8f5cec0 2024-04-17T07:31:39.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.358+0000 7f8807400640 1 -- 172.21.15.86:0/2861429263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814012c30 con 0x561be8f5cec0 2024-04-17T07:31:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.360+0000 7f88243dc740 1 -- 172.21.15.86:0/2861429263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f5cec0 msgr2=0x561be8f5d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.360+0000 7f88243dc740 1 --2- 172.21.15.86:0/2861429263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f5cec0 0x561be8f5d2e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f8814009ee0 tx=0x7f8814010740 comp rx=0 tx=0).stop 2024-04-17T07:31:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.360+0000 7f881aa00640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88000a3c30 0x7f88000a60f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x561be8f5e320 tx=0x7f880c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 735.0 s, 464.4 tps, lat 214.545 ms stddev 195.730 2024-04-17T07:31:39.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.360+0000 7f88243dc740 1 -- 172.21.15.86:0/2861429263 shutdown_connections 2024-04-17T07:31:39.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.360+0000 7f88243dc740 1 --2- 172.21.15.86:0/2861429263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f62a20 0x561be8f64eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.360+0000 7f88243dc740 1 --2- 172.21.15.86:0/2861429263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5d8b0 0x561be8f623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.360+0000 7f88243dc740 1 --2- 172.21.15.86:0/2861429263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f5cec0 0x561be8f5d2e0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.360+0000 7f88243dc740 1 -- 172.21.15.86:0/2861429263 >> 172.21.15.86:0/2861429263 conn(0x561be8f58430 msgr2=0x561be8f58830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.360+0000 7f88243dc740 1 -- 172.21.15.86:0/2861429263 shutdown_connections 2024-04-17T07:31:39.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.360+0000 7f88243dc740 1 -- 172.21.15.86:0/2861429263 wait complete. 2024-04-17T07:31:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.361+0000 7f88243dc740 1 -- start start 2024-04-17T07:31:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.361+0000 7f88243dc740 1 Processor -- start 2024-04-17T07:31:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.362+0000 7f88243dc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5cec0 0x561be8f67480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.362+0000 7f88243dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5d8b0 0x561be8f679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.362+0000 7f88243dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 0x561be8f6ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.362+0000 7f88243dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561be8f659d0 con 0x561be8f62a20 2024-04-17T07:31:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.362+0000 7f88243dc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561be8e7c730 con 0x561be8f5cec0 2024-04-17T07:31:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.363+0000 7f88243dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561be8f65b50 con 0x561be8f5d8b0 2024-04-17T07:31:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.363+0000 7f881b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 0x561be8f6ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.363+0000 7f88243dc740 10 client.0 _ll_register_callbacks cb 0x561be8e7eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.363+0000 7f881aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5cec0 0x561be8f67480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.363+0000 7f881aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5cec0 0x561be8f67480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36592/0 (socket says 172.21.15.86:36592) 2024-04-17T07:31:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.363+0000 7f881aa00640 1 -- 172.21.15.86:0/1971764117 learned_addr learned my addr 172.21.15.86:0/1971764117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.363+0000 7f881a000640 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5d8b0 0x561be8f679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.364+0000 7f881a000640 1 -- 172.21.15.86:0/1971764117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5cec0 msgr2=0x561be8f67480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.364+0000 7f881a000640 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5cec0 0x561be8f67480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.364+0000 7f881a000640 1 -- 172.21.15.86:0/1971764117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 msgr2=0x561be8f6ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.364+0000 7f881a000640 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 0x561be8f6ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.364+0000 7f881a000640 1 -- 172.21.15.86:0/1971764117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be8c9d7e0 con 0x561be8f5d8b0 2024-04-17T07:31:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.364+0000 7f881a000640 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5d8b0 0x561be8f679c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f88080974c0 tx=0x7f88080974f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7826db061d512539 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.364+0000 7f87fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:31:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.365+0000 7f87fbe00640 1 -- 172.21.15.86:0/1971764117 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808099420 con 0x561be8f5d8b0 2024-04-17T07:31:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.365+0000 7f88243dc740 1 -- 172.21.15.86:0/1971764117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f881401bb10 con 0x561be8f5d8b0 2024-04-17T07:31:39.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.365+0000 7f87fbe00640 1 -- 172.21.15.86:0/1971764117 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f88080a55a0 con 0x561be8f5d8b0 2024-04-17T07:31:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.366+0000 7f87fbe00640 1 -- 172.21.15.86:0/1971764117 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88080995c0 con 0x561be8f5d8b0 2024-04-17T07:31:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.366+0000 7f88243dc740 10 client.28438 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:39.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.366+0000 7f88243dc740 1 -- 172.21.15.86:0/1971764117 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561be8cfdf70 con 0x561be8f5d8b0 2024-04-17T07:31:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.368+0000 7f87fbe00640 1 -- 172.21.15.86:0/1971764117 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f881401bb10 con 0x561be8f5d8b0 2024-04-17T07:31:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.368+0000 7f87fbe00640 1 -- 172.21.15.86:0/1971764117 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f880809be40 con 0x561be8f5d8b0 2024-04-17T07:31:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.369+0000 7f88243dc740 10 client.28438 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.369+0000 7f88243dc740 10 client.28438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.369+0000 7f88243dc740 20 client.28438 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:39.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.369+0000 7f88243dc740 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561be8f73240 0x561be8f75630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.369+0000 7f88243dc740 4 client.28438 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:39.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.369+0000 7f88243dc740 1 -- 172.21.15.86:0/1971764117 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561be8ed06d0 con 0x561be8f73240 2024-04-17T07:31:39.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.370+0000 7f881aa00640 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561be8f73240 0x561be8f75630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.373+0000 7f87fbe00640 10 client.28438 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:39.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.373+0000 7f881aa00640 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561be8f73240 0x561be8f75630 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d07f8db78a7a7f9 server_cookie=5b4cd30eb4371396 in_seq=0 out_seq=0 2024-04-17T07:31:39.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.375+0000 7f87fbe00640 1 -- 172.21.15.86:0/1971764117 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561be8ed06d0 con 0x561be8f73240 2024-04-17T07:31:39.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.375+0000 7f87fbe00640 10 client.28438 handle_command_reply: tid=0 2024-04-17T07:31:39.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.376+0000 7f88243dc740 1 client.28438 shutdown 2024-04-17T07:31:39.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.377+0000 7f88243dc740 1 -- 172.21.15.86:0/1971764117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5d8b0 msgr2=0x561be8f679c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.377+0000 7f88243dc740 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5d8b0 0x561be8f679c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f88080974c0 tx=0x7f88080974f0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.377+0000 7f88243dc740 1 -- 172.21.15.86:0/1971764117 shutdown_connections 2024-04-17T07:31:39.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.377+0000 7f88243dc740 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561be8f73240 0x561be8f75630 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.377+0000 7f88243dc740 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 0x561be8f6ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.377+0000 7f88243dc740 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5d8b0 0x561be8f679c0 secure :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f88080974c0 tx=0x7f88080974f0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.377+0000 7f88243dc740 1 --2- 172.21.15.86:0/1971764117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f5cec0 0x561be8f67480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.377+0000 7f88243dc740 1 -- 172.21.15.86:0/1971764117 >> 172.21.15.86:0/1971764117 conn(0x561be8f58430 msgr2=0x561be8f575e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.377+0000 7f87fbe00640 0 client.28438 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:39.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.378+0000 7f88243dc740 1 -- 172.21.15.86:0/1971764117 shutdown_connections 2024-04-17T07:31:39.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.378+0000 7f88243dc740 1 -- 172.21.15.86:0/1971764117 wait complete. 2024-04-17T07:31:39.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.379+0000 7f88243dc740 20 client.28438 trim_cache size 0 max 16384 2024-04-17T07:31:39.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.422+0000 7f88243dc740 1 Processor -- start 2024-04-17T07:31:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.425+0000 7f88243dc740 1 -- start start 2024-04-17T07:31:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.425+0000 7f88243dc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f58430 0x561be8f56d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.425+0000 7f88243dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5cec0 0x561be8f6e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.425+0000 7f88243dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 0x561be8f57290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.426+0000 7f881a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5cec0 0x561be8f6e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.426+0000 7f881b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 0x561be8f57290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.426+0000 7f881b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 0x561be8f57290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56734/0 (socket says 172.21.15.86:56734) 2024-04-17T07:31:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.426+0000 7f881a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5cec0 0x561be8f6e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42284/0 (socket says 172.21.15.86:42284) 2024-04-17T07:31:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.426+0000 7f881aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f58430 0x561be8f56d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.426+0000 7f88243dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1b99f0 con 0x561be8f62a20 2024-04-17T07:31:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.426+0000 7f88243dc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1bb530 con 0x561be8f58430 2024-04-17T07:31:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.426+0000 7f88243dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1ba0c0 con 0x561be8f5cec0 2024-04-17T07:31:39.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.426+0000 7f881b400640 1 -- 172.21.15.86:0/1351095648 learned_addr learned my addr 172.21.15.86:0/1351095648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:39.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.427+0000 7f881b400640 1 -- 172.21.15.86:0/1351095648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5cec0 msgr2=0x561be8f6e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.427+0000 7f881b400640 1 --2- 172.21.15.86:0/1351095648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5cec0 0x561be8f6e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.427+0000 7f881b400640 1 -- 172.21.15.86:0/1351095648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f58430 msgr2=0x561be8f56d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.427+0000 7f881b400640 1 --2- 172.21.15.86:0/1351095648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f58430 0x561be8f56d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.427+0000 7f881b400640 1 -- 172.21.15.86:0/1351095648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be8b95170 con 0x561be8f62a20 2024-04-17T07:31:39.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.428+0000 7f881b400640 1 --2- 172.21.15.86:0/1351095648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 0x561be8f57290 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x561be8f5e110 tx=0x7f8814009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6102fc829575a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.428+0000 7f8806a00640 1 -- 172.21.15.86:0/1351095648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814016040 con 0x561be8f62a20 2024-04-17T07:31:39.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.429+0000 7f8806a00640 1 -- 172.21.15.86:0/1351095648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8814004c00 con 0x561be8f62a20 2024-04-17T07:31:39.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.429+0000 7f8806a00640 1 -- 172.21.15.86:0/1351095648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88140052a0 con 0x561be8f62a20 2024-04-17T07:31:39.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.430+0000 7f88243dc740 1 -- 172.21.15.86:0/1351095648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 msgr2=0x561be8f57290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.430+0000 7f88243dc740 1 --2- 172.21.15.86:0/1351095648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 0x561be8f57290 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x561be8f5e110 tx=0x7f8814009920 comp rx=0 tx=0).stop 2024-04-17T07:31:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.430+0000 7f88243dc740 1 -- 172.21.15.86:0/1351095648 shutdown_connections 2024-04-17T07:31:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.430+0000 7f88243dc740 1 --2- 172.21.15.86:0/1351095648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f62a20 0x561be8f57290 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.430+0000 7f88243dc740 1 --2- 172.21.15.86:0/1351095648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5cec0 0x561be8f6e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.430+0000 7f88243dc740 1 --2- 172.21.15.86:0/1351095648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f58430 0x561be8f56d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.430+0000 7f88243dc740 1 -- 172.21.15.86:0/1351095648 >> 172.21.15.86:0/1351095648 conn(0x561be8f67040 msgr2=0x561be8f65ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.431+0000 7f88243dc740 1 -- 172.21.15.86:0/1351095648 shutdown_connections 2024-04-17T07:31:39.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.432+0000 7f88243dc740 1 -- 172.21.15.86:0/1351095648 wait complete. 2024-04-17T07:31:39.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.433+0000 7f88243dc740 1 -- start start 2024-04-17T07:31:39.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.433+0000 7f88243dc740 1 Processor -- start 2024-04-17T07:31:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.434+0000 7f88243dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f58430 0x561be8f6e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.434+0000 7f88243dc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5cec0 0x561be8f6e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.434+0000 7f88243dc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f62a20 0x561be8f6c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.434+0000 7f88243dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1bae60 con 0x561be8f58430 2024-04-17T07:31:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.434+0000 7f88243dc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1ba790 con 0x561be8f62a20 2024-04-17T07:31:39.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.434+0000 7f88243dc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1b7960 con 0x561be8f5cec0 2024-04-17T07:31:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.434+0000 7f88243dc740 10 client.0 _ll_register_callbacks cb 0x7f8808024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.434+0000 7f881b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f62a20 0x561be8f6c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.435+0000 7f881b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f62a20 0x561be8f6c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36610/0 (socket says 172.21.15.86:36610) 2024-04-17T07:31:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.435+0000 7f881b400640 1 -- 172.21.15.86:0/2483631928 learned_addr learned my addr 172.21.15.86:0/2483631928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.435+0000 7f881b400640 1 -- 172.21.15.86:0/2483631928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5cec0 msgr2=0x561be8f6e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.435+0000 7f881b400640 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5cec0 0x561be8f6e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.435+0000 7f881b400640 1 -- 172.21.15.86:0/2483631928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f58430 msgr2=0x561be8f6e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.435+0000 7f881aa00640 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f58430 0x561be8f6e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.436+0000 7f881b400640 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f58430 0x561be8f6e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.436+0000 7f881b400640 1 -- 172.21.15.86:0/2483631928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be8f6ce40 con 0x561be8f62a20 2024-04-17T07:31:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.436+0000 7f881aa00640 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f58430 0x561be8f6e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.436+0000 7f881b400640 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f62a20 0x561be8f6c7d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8814006010 tx=0x7f8814012890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395ad86991014920 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.436+0000 7f87fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:31:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.438+0000 7f87fbe00640 1 -- 172.21.15.86:0/2483631928 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814034060 con 0x561be8f62a20 2024-04-17T07:31:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.438+0000 7f87fbe00640 1 -- 172.21.15.86:0/2483631928 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f881401b460 con 0x561be8f62a20 2024-04-17T07:31:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.438+0000 7f88243dc740 1 -- 172.21.15.86:0/2483631928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8814004c00 con 0x561be8f62a20 2024-04-17T07:31:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.439+0000 7f87fbe00640 1 -- 172.21.15.86:0/2483631928 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f881403b050 con 0x561be8f62a20 2024-04-17T07:31:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.439+0000 7f88243dc740 10 client.38423 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.439+0000 7f88243dc740 1 -- 172.21.15.86:0/2483631928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561be8fda0f0 con 0x561be8f62a20 2024-04-17T07:31:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.441+0000 7f87fbe00640 1 -- 172.21.15.86:0/2483631928 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8814004c00 con 0x561be8f62a20 2024-04-17T07:31:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.441+0000 7f87fbe00640 1 -- 172.21.15.86:0/2483631928 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f881400f040 con 0x561be8f62a20 2024-04-17T07:31:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.442+0000 7f88243dc740 10 client.38423 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.442+0000 7f88243dc740 10 client.38423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.442+0000 7f88243dc740 20 client.38423 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.442+0000 7f88243dc740 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561be8fda790 0x561be8fdcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.442+0000 7f88243dc740 4 client.38423 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:31:39.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.442+0000 7f88243dc740 1 -- 172.21.15.86:0/2483631928 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561be8eca540 con 0x561be8fda790 2024-04-17T07:31:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.444+0000 7f881aa00640 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561be8fda790 0x561be8fdcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.446+0000 7f881aa00640 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561be8fda790 0x561be8fdcbb0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb6c5da8a166bed server_cookie=c811f6cf8680e431 in_seq=0 out_seq=0 2024-04-17T07:31:39.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.446+0000 7f87fbe00640 10 client.38423 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:39.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.449+0000 7f87fbe00640 1 -- 172.21.15.86:0/2483631928 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561be8eca540 con 0x561be8fda790 2024-04-17T07:31:39.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.449+0000 7f87fbe00640 10 client.38423 handle_command_reply: tid=0 2024-04-17T07:31:39.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.449+0000 7f88243dc740 1 client.38423 shutdown 2024-04-17T07:31:39.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.449+0000 7f88243dc740 1 -- 172.21.15.86:0/2483631928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f62a20 msgr2=0x561be8f6c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.449+0000 7f88243dc740 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f62a20 0x561be8f6c7d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8814006010 tx=0x7f8814012890 comp rx=0 tx=0).stop 2024-04-17T07:31:39.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.450+0000 7f88243dc740 1 -- 172.21.15.86:0/2483631928 shutdown_connections 2024-04-17T07:31:39.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.450+0000 7f88243dc740 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x561be8fda790 0x561be8fdcbb0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.450+0000 7f88243dc740 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561be8f62a20 0x561be8f6c7d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.450+0000 7f88243dc740 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561be8f5cec0 0x561be8f6e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.450+0000 7f88243dc740 1 --2- 172.21.15.86:0/2483631928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561be8f58430 0x561be8f6e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.450+0000 7f88243dc740 1 -- 172.21.15.86:0/2483631928 >> 172.21.15.86:0/2483631928 conn(0x561be8f67040 msgr2=0x561be8f56d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.450+0000 7f87fbe00640 0 client.38423 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:39.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.451+0000 7f88243dc740 1 -- 172.21.15.86:0/2483631928 shutdown_connections 2024-04-17T07:31:39.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.452+0000 7f88243dc740 1 -- 172.21.15.86:0/2483631928 wait complete. 2024-04-17T07:31:39.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.453+0000 7f88243dc740 20 client.38423 trim_cache size 0 max 16384 2024-04-17T07:31:39.453 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:31:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:31:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "3c8c35e0-b183-4598-9f2b-7952d71d0e65", 2024-04-17T07:31:39.454 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:31:39.454 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:31:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.454+0000 7f8821600640 1 -- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88000a3c30 msgr2=0x7f88000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.454+0000 7f8821600640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88000a3c30 0x7f88000a60f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x561be8f5e320 tx=0x7f880c02c7e0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.454+0000 7f8821600640 1 -- 172.21.15.86:0/1515707867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f881c1053b0 msgr2=0x7f881c070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.454+0000 7f8821600640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f881c1053b0 0x7f881c070ee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f880800e9f0 tx=0x7f880800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.455+0000 7f8821600640 1 -- 172.21.15.86:0/1515707867 shutdown_connections 2024-04-17T07:31:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.455+0000 7f8821600640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f88000a3c30 0x7f88000a60f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.455+0000 7f8821600640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f881c105d70 0x7f881c1aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.455+0000 7f8821600640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f881c1053b0 0x7f881c070ee0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.455+0000 7f8821600640 1 --2- 172.21.15.86:0/1515707867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881c1041c0 0x7f881c0709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.455+0000 7f8821600640 1 -- 172.21.15.86:0/1515707867 >> 172.21.15.86:0/1515707867 conn(0x7f881c0ff750 msgr2=0x7f881c108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.455+0000 7f8821600640 1 -- 172.21.15.86:0/1515707867 shutdown_connections 2024-04-17T07:31:39.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.455+0000 7f8821600640 1 -- 172.21.15.86:0/1515707867 wait complete. 2024-04-17T07:31:39.482 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:39 smithi086 ceph-mon[31733]: pgmap v958: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 730 op/s 2024-04-17T07:31:39.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:31:39.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:31:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:39 smithi098 ceph-mon[35418]: pgmap v958: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 730 op/s 2024-04-17T07:31:39.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:39 smithi103 ceph-mon[35495]: pgmap v958: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 730 op/s 2024-04-17T07:31:39.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.854+0000 7fb8fa400640 1 -- 172.21.15.86:0/3719144525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f4105950 msgr2=0x7fb8f4105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.854+0000 7fb8fa400640 1 --2- 172.21.15.86:0/3719144525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f4105950 0x7fb8f4105dd0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00ac70 tx=0x7fb8ec02c0e0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.855+0000 7fb8fa400640 1 -- 172.21.15.86:0/3719144525 shutdown_connections 2024-04-17T07:31:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.855+0000 7fb8fa400640 1 --2- 172.21.15.86:0/3719144525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f4105950 0x7fb8f4105dd0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.855+0000 7fb8fa400640 1 --2- 172.21.15.86:0/3719144525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8f4102fa0 0x7fb8f4105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.855+0000 7fb8fa400640 1 --2- 172.21.15.86:0/3719144525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f4100630 0x7fb8f4102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.855+0000 7fb8fa400640 1 -- 172.21.15.86:0/3719144525 >> 172.21.15.86:0/3719144525 conn(0x7fb8f40fc270 msgr2=0x7fb8f40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.856+0000 7fb8fa400640 1 -- 172.21.15.86:0/3719144525 shutdown_connections 2024-04-17T07:31:39.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.857+0000 7fb8fa400640 1 -- 172.21.15.86:0/3719144525 wait complete. 2024-04-17T07:31:39.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.859+0000 7fb8fa400640 1 Processor -- start 2024-04-17T07:31:39.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.859+0000 7fb8fa400640 1 -- start start 2024-04-17T07:31:39.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.860+0000 7fb8fa400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8f4100630 0x7fb8f4070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.860+0000 7fb8fa400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f4102fa0 0x7fb8f4070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.860+0000 7fb8fa400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f4105950 0x7fb8f41ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.860+0000 7fb8fa400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f4113f10 con 0x7fb8f4102fa0 2024-04-17T07:31:39.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.860+0000 7fb8fa400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f4113d90 con 0x7fb8f4100630 2024-04-17T07:31:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.860+0000 7fb8fa400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f4114090 con 0x7fb8f4105950 2024-04-17T07:31:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.860+0000 7fb8f3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8f4100630 0x7fb8f4070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.860+0000 7fb8f3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8f4100630 0x7fb8f4070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36628/0 (socket says 172.21.15.86:36628) 2024-04-17T07:31:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.860+0000 7fb8f3400640 1 -- 172.21.15.86:0/2319491772 learned_addr learned my addr 172.21.15.86:0/2319491772 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.860+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f4105950 0x7fb8f41ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.861+0000 7fb8f2a00640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f4102fa0 0x7fb8f4070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.861+0000 7fb8f3400640 1 -- 172.21.15.86:0/2319491772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f4105950 msgr2=0x7fb8f41ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.861+0000 7fb8f3400640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f4105950 0x7fb8f41ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.861+0000 7fb8f3400640 1 -- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f4102fa0 msgr2=0x7fb8f4070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.861+0000 7fb8f3400640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f4102fa0 0x7fb8f4070ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.861+0000 7fb8f3400640 1 -- 172.21.15.86:0/2319491772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f41aeb00 con 0x7fb8f4100630 2024-04-17T07:31:39.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.861+0000 7fb8f2a00640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f4102fa0 0x7fb8f4070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:39.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.862+0000 7fb8f3400640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8f4100630 0x7fb8f4070960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800e9f0 tx=0x7fb8e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.862+0000 7fb8e7e00640 1 -- 172.21.15.86:0/2319491772 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e800cd80 con 0x7fb8f4100630 2024-04-17T07:31:39.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.862+0000 7fb8e7e00640 1 -- 172.21.15.86:0/2319491772 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8e80108e0 con 0x7fb8f4100630 2024-04-17T07:31:39.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.862+0000 7fb8e7e00640 1 -- 172.21.15.86:0/2319491772 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8018850 con 0x7fb8f4100630 2024-04-17T07:31:39.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.862+0000 7fb8fa400640 1 -- 172.21.15.86:0/2319491772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f41aecd0 con 0x7fb8f4100630 2024-04-17T07:31:39.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.863+0000 7fb8fa400640 1 -- 172.21.15.86:0/2319491772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f41af1a0 con 0x7fb8f4100630 2024-04-17T07:31:39.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.871+0000 7fb8fd037740 1 Processor -- start 2024-04-17T07:31:39.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.871+0000 7fb8e7e00640 1 -- 172.21.15.86:0/2319491772 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb8e8010a90 con 0x7fb8f4100630 2024-04-17T07:31:39.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.871+0000 7fb8fd037740 1 -- start start 2024-04-17T07:31:39.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.873+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef57778370 0x55ef57778790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.873+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef5777d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.873+0000 7fb8f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef57778370 0x55ef57778790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.874+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef5777ded0 0x55ef57780360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.874+0000 7fb8fd037740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef576df120 con 0x55ef57778370 2024-04-17T07:31:39.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.874+0000 7fb8f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef57778370 0x55ef57778790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56772/0 (socket says 172.21.15.86:56772) 2024-04-17T07:31:39.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.874+0000 7fb8f3e00640 1 -- 172.21.15.86:0/3089468545 learned_addr learned my addr 172.21.15.86:0/3089468545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:39.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.874+0000 7fb8fd037740 1 -- 172.21.15.86:0/3089468545 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ef57685340 con 0x55ef5777ded0 2024-04-17T07:31:39.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.874+0000 7fb8fd037740 1 -- 172.21.15.86:0/3089468545 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef576db0e0 con 0x55ef57778d60 2024-04-17T07:31:39.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.875+0000 7fb8f3400640 1 --2- 172.21.15.86:0/3089468545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef5777d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.875+0000 7fb8f2a00640 1 --2- 172.21.15.86:0/3089468545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef5777ded0 0x55ef57780360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.876+0000 7fb8e7e00640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8dc0a3cb0 0x7fb8dc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.876+0000 7fb8f3400640 1 -- 172.21.15.86:0/3089468545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef5777ded0 msgr2=0x55ef57780360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.876+0000 7fb8f3400640 1 --2- 172.21.15.86:0/3089468545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef5777ded0 0x55ef57780360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.876+0000 7fb8f3400640 1 -- 172.21.15.86:0/3089468545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef57778370 msgr2=0x55ef57778790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.876+0000 7fb8f3400640 1 --2- 172.21.15.86:0/3089468545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef57778370 0x55ef57778790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.876+0000 7fb8f3400640 1 -- 172.21.15.86:0/3089468545 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef574bb7d0 con 0x55ef57778d60 2024-04-17T07:31:39.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.876+0000 7fb8e7e00640 1 -- 172.21.15.86:0/2319491772 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb8e80d0b00 con 0x7fb8f4100630 2024-04-17T07:31:39.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.878+0000 7fb8f3400640 1 --2- 172.21.15.86:0/3089468545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef5777d860 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80ce1c0 tx=0x7fb8e80b10e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e985cda278eb5a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.878+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8dc0a3cb0 0x7fb8dc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.878+0000 7fb8e4c00640 1 -- 172.21.15.86:0/3089468545 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e809c7d0 con 0x55ef57778d60 2024-04-17T07:31:39.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.878+0000 7fb8e4c00640 1 -- 172.21.15.86:0/3089468545 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8e8010de0 con 0x55ef57778d60 2024-04-17T07:31:39.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.878+0000 7fb8e4c00640 1 -- 172.21.15.86:0/3089468545 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e809c360 con 0x55ef57778d60 2024-04-17T07:31:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.880+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8dc0a3cb0 0x7fb8dc0a6170 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec031300 tx=0x7fb8ec00a580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.880+0000 7fb8fd037740 1 -- 172.21.15.86:0/3089468545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 msgr2=0x55ef5777d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.880+0000 7fb8fd037740 1 --2- 172.21.15.86:0/3089468545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef5777d860 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80ce1c0 tx=0x7fb8e80b10e0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.881+0000 7fb8fd037740 1 -- 172.21.15.86:0/3089468545 shutdown_connections 2024-04-17T07:31:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.881+0000 7fb8fd037740 1 --2- 172.21.15.86:0/3089468545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef5777ded0 0x55ef57780360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.881+0000 7fb8fd037740 1 --2- 172.21.15.86:0/3089468545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef5777d860 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.881+0000 7fb8fd037740 1 --2- 172.21.15.86:0/3089468545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef57778370 0x55ef57778790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.881+0000 7fb8fd037740 1 -- 172.21.15.86:0/3089468545 >> 172.21.15.86:0/3089468545 conn(0x55ef577738e0 msgr2=0x55ef57773ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.881+0000 7fb8fd037740 1 -- 172.21.15.86:0/3089468545 shutdown_connections 2024-04-17T07:31:39.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.881+0000 7fb8fd037740 1 -- 172.21.15.86:0/3089468545 wait complete. 2024-04-17T07:31:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.883+0000 7fb8fd037740 1 -- start start 2024-04-17T07:31:39.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.883+0000 7fb8fd037740 1 Processor -- start 2024-04-17T07:31:39.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.884+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 0x55ef577827c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.884+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 0x55ef57782d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.884+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57789d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.884+0000 7fb8f3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 0x55ef577827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.885+0000 7fb8f3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 0x55ef577827c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42336/0 (socket says 172.21.15.86:42336) 2024-04-17T07:31:39.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.885+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 0x55ef57782d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.885+0000 7fb8f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57789d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.885+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef57780dc0 con 0x55ef5777ded0 2024-04-17T07:31:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.885+0000 7fb8f3400640 1 -- 172.21.15.86:0/191833740 learned_addr learned my addr 172.21.15.86:0/191833740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.885+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ef57698710 con 0x55ef57778d60 2024-04-17T07:31:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.885+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef57780f40 con 0x55ef57778370 2024-04-17T07:31:39.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.885+0000 7fb8fd037740 10 client.0 _ll_register_callbacks cb 0x55ef57693af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.886+0000 7fb8f3e00640 1 -- 172.21.15.86:0/191833740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 msgr2=0x55ef577827c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.886+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 0x55ef577827c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.886+0000 7fb8f3e00640 1 -- 172.21.15.86:0/191833740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 msgr2=0x55ef57782d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.886+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 0x55ef57782d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.887+0000 7fb8f3e00640 1 -- 172.21.15.86:0/191833740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef573f13f0 con 0x55ef5777ded0 2024-04-17T07:31:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.887+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57789d40 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00a3a0 tx=0x7fb8ec02c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5fa83c3ac004fc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.887+0000 7fb8d9600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:31:39.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.888+0000 7fb8d9600640 1 -- 172.21.15.86:0/191833740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec04e070 con 0x55ef5777ded0 2024-04-17T07:31:39.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.888+0000 7fb8d9600640 1 -- 172.21.15.86:0/191833740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb8ec034610 con 0x55ef5777ded0 2024-04-17T07:31:39.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.888+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef576ecb40 con 0x55ef5777ded0 2024-04-17T07:31:39.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.888+0000 7fb8d9600640 1 -- 172.21.15.86:0/191833740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8ec031710 con 0x55ef5777ded0 2024-04-17T07:31:39.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.888+0000 7fb8fd037740 10 client.19428 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:39.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.888+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef573b10f0 con 0x55ef5777ded0 2024-04-17T07:31:39.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.889+0000 7fb8d9600640 1 -- 172.21.15.86:0/191833740 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ef576ecb40 con 0x55ef5777ded0 2024-04-17T07:31:39.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.889+0000 7fb8d9600640 1 -- 172.21.15.86:0/191833740 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb8ec04a860 con 0x55ef5777ded0 2024-04-17T07:31:39.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.889+0000 7fb8fd037740 10 client.19428 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:39.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.889+0000 7fb8fd037740 10 client.19428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:39.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.890+0000 7fb8fd037740 20 client.19428 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:39.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.890+0000 7fb8fd037740 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ef5778e5c0 0x55ef57790a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.890+0000 7fb8fd037740 4 client.19428 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:31:39.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.890+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef576e69b0 con 0x55ef5778e5c0 2024-04-17T07:31:39.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.891+0000 7fb8f3400640 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ef5778e5c0 0x55ef57790a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.892+0000 7fb8f3400640 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ef5778e5c0 0x55ef57790a60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cccb679b48d9cdf server_cookie=a22511165677e5de in_seq=0 out_seq=0 2024-04-17T07:31:39.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.892+0000 7fb8d9600640 10 client.19428 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8d9600640 1 -- 172.21.15.86:0/191833740 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef576e69b0 con 0x55ef5778e5c0 2024-04-17T07:31:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8d9600640 10 client.19428 handle_command_reply: tid=0 2024-04-17T07:31:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8fd037740 1 client.19428 shutdown 2024-04-17T07:31:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 msgr2=0x55ef57789d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8fd037740 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57789d40 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00a3a0 tx=0x7fb8ec02c0e0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 shutdown_connections 2024-04-17T07:31:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8fd037740 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ef5778e5c0 0x55ef57790a60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8fd037740 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57789d40 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8fd037740 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 0x55ef57782d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8fd037740 1 --2- 172.21.15.86:0/191833740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 0x55ef577827c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 >> 172.21.15.86:0/191833740 conn(0x55ef577738e0 msgr2=0x55ef57772ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.896+0000 7fb8d9600640 0 client.19428 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.897+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 shutdown_connections 2024-04-17T07:31:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.897+0000 7fb8fd037740 1 -- 172.21.15.86:0/191833740 wait complete. 2024-04-17T07:31:39.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.898+0000 7fb8fd037740 20 client.19428 trim_cache size 0 max 16384 2024-04-17T07:31:39.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.933+0000 7fb8fd037740 1 Processor -- start 2024-04-17T07:31:39.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.933+0000 7fb8fd037740 1 -- start start 2024-04-17T07:31:39.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.933+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 0x55ef57772250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.933+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 0x55ef57789d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.933+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57772790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.933+0000 7fb8fd037740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41b97a0 con 0x55ef5777ded0 2024-04-17T07:31:39.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.933+0000 7fb8fd037740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41bb2e0 con 0x55ef57778d60 2024-04-17T07:31:39.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.933+0000 7fb8fd037740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41b9e70 con 0x55ef57778370 2024-04-17T07:31:39.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.934+0000 7fb8f3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 0x55ef57772250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.934+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 0x55ef57789d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.934+0000 7fb8f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57772790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.934+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 0x55ef57789d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36658/0 (socket says 172.21.15.86:36658) 2024-04-17T07:31:39.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.934+0000 7fb8f2a00640 1 -- 172.21.15.86:0/2229964536 learned_addr learned my addr 172.21.15.86:0/2229964536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:39.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.936+0000 7fb8f3e00640 1 -- 172.21.15.86:0/2229964536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 msgr2=0x55ef57772250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.936+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/2229964536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 0x55ef57772250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.936+0000 7fb8f3e00640 1 -- 172.21.15.86:0/2229964536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 msgr2=0x55ef57789d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.936+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/2229964536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 0x55ef57789d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.936+0000 7fb8f3e00640 1 -- 172.21.15.86:0/2229964536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef57779140 con 0x55ef5777ded0 2024-04-17T07:31:39.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.937+0000 7fb8f2a00640 1 --2- 172.21.15.86:0/2229964536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 0x55ef57789d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:31:39.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.937+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/2229964536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57772790 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00a370 tx=0x7fb8ec001440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6e798894230d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.938+0000 7fb8dbe00640 1 -- 172.21.15.86:0/2229964536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec061070 con 0x55ef5777ded0 2024-04-17T07:31:39.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.938+0000 7fb8dbe00640 1 -- 172.21.15.86:0/2229964536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8ec043040 con 0x55ef5777ded0 2024-04-17T07:31:39.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.938+0000 7fb8dbe00640 1 -- 172.21.15.86:0/2229964536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0528f0 con 0x55ef5777ded0 2024-04-17T07:31:39.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.939+0000 7fb8fd037740 1 -- 172.21.15.86:0/2229964536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 msgr2=0x55ef57772790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.939+0000 7fb8fd037740 1 --2- 172.21.15.86:0/2229964536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57772790 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00a370 tx=0x7fb8ec001440 comp rx=0 tx=0).stop 2024-04-17T07:31:39.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.940+0000 7fb8fd037740 1 -- 172.21.15.86:0/2229964536 shutdown_connections 2024-04-17T07:31:39.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.940+0000 7fb8fd037740 1 --2- 172.21.15.86:0/2229964536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57772790 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.940+0000 7fb8fd037740 1 --2- 172.21.15.86:0/2229964536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778d60 0x55ef57789d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.940+0000 7fb8fd037740 1 --2- 172.21.15.86:0/2229964536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778370 0x55ef57772250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.940+0000 7fb8fd037740 1 -- 172.21.15.86:0/2229964536 >> 172.21.15.86:0/2229964536 conn(0x55ef577738e0 msgr2=0x55ef5777a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.940+0000 7fb8fd037740 1 -- 172.21.15.86:0/2229964536 shutdown_connections 2024-04-17T07:31:39.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.940+0000 7fb8fd037740 1 -- 172.21.15.86:0/2229964536 wait complete. 2024-04-17T07:31:39.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.941+0000 7fb8fd037740 1 -- start start 2024-04-17T07:31:39.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.941+0000 7fb8fd037740 1 Processor -- start 2024-04-17T07:31:39.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.946+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778370 0x55ef57787560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.946+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef57787aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.946+0000 7fb8fd037740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57788ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.947+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef57787aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.947+0000 7fb8f2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef57787aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42346/0 (socket says 172.21.15.86:42346) 2024-04-17T07:31:39.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.948+0000 7fb8fd037740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41bac10 con 0x55ef5777ded0 2024-04-17T07:31:39.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.948+0000 7fb8fd037740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41ba540 con 0x55ef57778370 2024-04-17T07:31:39.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.948+0000 7fb8fd037740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41b7770 con 0x55ef57778d60 2024-04-17T07:31:39.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.948+0000 7fb8fd037740 10 client.0 _ll_register_callbacks cb 0x7fb8ec00aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:31:39.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.948+0000 7fb8f3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778370 0x55ef57787560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.948+0000 7fb8f2a00640 1 -- 172.21.15.86:0/422124697 learned_addr learned my addr 172.21.15.86:0/422124697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:31:39.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.949+0000 7fb8f2a00640 1 -- 172.21.15.86:0/422124697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778370 msgr2=0x55ef57787560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.949+0000 7fb8f2a00640 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778370 0x55ef57787560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.949+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57788ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.949+0000 7fb8f2a00640 1 -- 172.21.15.86:0/422124697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 msgr2=0x55ef57788ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:31:39.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.950+0000 7fb8f2a00640 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57788ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.950+0000 7fb8f2a00640 1 -- 172.21.15.86:0/422124697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef577890a0 con 0x55ef57778d60 2024-04-17T07:31:39.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.950+0000 7fb8f3400640 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778370 0x55ef57787560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:39.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.950+0000 7fb8f3e00640 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57788ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:31:39.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.951+0000 7fb8f2a00640 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef57787aa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0012980 tx=0x7fb8e0012d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2aa3c64c2258cf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:31:39.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.952+0000 7fb8d9600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:31:39.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.952+0000 7fb8d9600640 1 -- 172.21.15.86:0/422124697 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0022060 con 0x55ef57778d60 2024-04-17T07:31:39.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.953+0000 7fb8d9600640 1 -- 172.21.15.86:0/422124697 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb8e0011c70 con 0x55ef57778d60 2024-04-17T07:31:39.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.953+0000 7fb8fd037740 1 -- 172.21.15.86:0/422124697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8ec043040 con 0x55ef57778d60 2024-04-17T07:31:39.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.954+0000 7fb8d9600640 1 -- 172.21.15.86:0/422124697 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8e0029050 con 0x55ef57778d60 2024-04-17T07:31:39.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.955+0000 7fb8fd037740 10 client.28459 fetch_fsmap learned FSMap version 28 2024-04-17T07:31:39.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.955+0000 7fb8fd037740 1 -- 172.21.15.86:0/422124697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef577f4dc0 con 0x55ef57778d60 2024-04-17T07:31:39.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.958+0000 7fb8d9600640 1 -- 172.21.15.86:0/422124697 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8ec043040 con 0x55ef57778d60 2024-04-17T07:31:39.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.958+0000 7fb8d9600640 1 -- 172.21.15.86:0/422124697 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fb8e001c070 con 0x55ef57778d60 2024-04-17T07:31:39.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.961+0000 7fb8fd037740 10 client.28459 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:31:39.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.961+0000 7fb8fd037740 10 client.28459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:31:39.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.961+0000 7fb8fd037740 20 client.28459 populate_metadata read hostname 'smithi086' 2024-04-17T07:31:39.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.961+0000 7fb8fd037740 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ef577f5770 0x55ef577f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:31:39.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.961+0000 7fb8fd037740 4 client.28459 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:31:39.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.961+0000 7fb8fd037740 1 -- 172.21.15.86:0/422124697 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef5776f140 con 0x55ef577f5770 2024-04-17T07:31:39.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.961+0000 7fb8f3400640 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ef577f5770 0x55ef577f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:31:39.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.965+0000 7fb8f3400640 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ef577f5770 0x55ef577f7b90 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a010ec56edfaeb1 server_cookie=d036fd10536946a in_seq=0 out_seq=0 2024-04-17T07:31:39.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.965+0000 7fb8d9600640 10 client.28459 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:39.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.965+0000 7fb8d9600640 1 -- 172.21.15.86:0/422124697 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ef5776f140 con 0x55ef577f5770 2024-04-17T07:31:39.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.965+0000 7fb8d9600640 10 client.28459 handle_command_reply: tid=0 2024-04-17T07:31:39.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.966+0000 7fb8fd037740 1 client.28459 shutdown 2024-04-17T07:31:39.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.966+0000 7fb8fd037740 1 -- 172.21.15.86:0/422124697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 msgr2=0x55ef57787aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.966+0000 7fb8fd037740 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef57787aa0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0012980 tx=0x7fb8e0012d80 comp rx=0 tx=0).stop 2024-04-17T07:31:39.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.966+0000 7fb8fd037740 1 -- 172.21.15.86:0/422124697 shutdown_connections 2024-04-17T07:31:39.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.966+0000 7fb8fd037740 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ef577f5770 0x55ef577f7b90 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.966+0000 7fb8fd037740 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef5777ded0 0x55ef57788ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.966+0000 7fb8d9600640 0 client.28459 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:31:39.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.966+0000 7fb8fd037740 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef57778d60 0x55ef57787aa0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.966+0000 7fb8fd037740 1 --2- 172.21.15.86:0/422124697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ef57778370 0x55ef57787560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.966+0000 7fb8fd037740 1 -- 172.21.15.86:0/422124697 >> 172.21.15.86:0/422124697 conn(0x55ef577738e0 msgr2=0x55ef57776b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.967+0000 7fb8fd037740 1 -- 172.21.15.86:0/422124697 shutdown_connections 2024-04-17T07:31:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.967+0000 7fb8fd037740 1 -- 172.21.15.86:0/422124697 wait complete. 2024-04-17T07:31:39.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.967+0000 7fb8fd037740 20 client.28459 trim_cache size 0 max 16384 2024-04-17T07:31:39.969 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:31:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:31:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:31:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "3c8c35e0-b183-4598-9f2b-7952d71d0e65": { 2024-04-17T07:31:39.969 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:31:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "3c8c35e0-b183-4598-9f2b-7952d71d0e65", 2024-04-17T07:31:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:31:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:31:39.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:31:39.970 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.970+0000 7fb8fa400640 1 -- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8dc0a3cb0 msgr2=0x7fb8dc0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.970+0000 7fb8fa400640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8dc0a3cb0 0x7fb8dc0a6170 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec031300 tx=0x7fb8ec00a580 comp rx=0 tx=0).stop 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.970+0000 7fb8fa400640 1 -- 172.21.15.86:0/2319491772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8f4100630 msgr2=0x7fb8f4070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.970+0000 7fb8fa400640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8f4100630 0x7fb8f4070960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800e9f0 tx=0x7fb8e800eeb0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.970+0000 7fb8fa400640 1 -- 172.21.15.86:0/2319491772 shutdown_connections 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.970+0000 7fb8fa400640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb8dc0a3cb0 0x7fb8dc0a6170 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.970+0000 7fb8fa400640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8f4105950 0x7fb8f41ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.970+0000 7fb8fa400640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8f4102fa0 0x7fb8f4070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.970+0000 7fb8fa400640 1 --2- 172.21.15.86:0/2319491772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8f4100630 0x7fb8f4070960 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.970+0000 7fb8fa400640 1 -- 172.21.15.86:0/2319491772 >> 172.21.15.86:0/2319491772 conn(0x7fb8f40fc270 msgr2=0x7fb8f41012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:31:39.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.971+0000 7fb8fa400640 1 -- 172.21.15.86:0/2319491772 shutdown_connections 2024-04-17T07:31:39.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:31:39.971+0000 7fb8fa400640 1 -- 172.21.15.86:0/2319491772 wait complete. 2024-04-17T07:31:40.017 INFO:tasks.cephfs.filesystem:scrub status for tag:3c8c35e0-b183-4598-9f2b-7952d71d0e65 - {'path': '/', 'tag': '3c8c35e0-b183-4598-9f2b-7952d71d0e65', 'options': 'recursive,force'} 2024-04-17T07:31:40.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:40 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:31:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:40 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:40 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:31:40.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:40 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:40 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:31:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:40 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:40 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:31:40.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:40 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:40.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:40 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:31:40.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:40 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:40.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:40 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:31:40.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:40 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:31:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:41 smithi086 ceph-mon[31733]: pgmap v959: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 700 op/s 2024-04-17T07:31:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:41 smithi098 ceph-mon[35418]: pgmap v959: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 700 op/s 2024-04-17T07:31:41.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:41 smithi103 ceph-mon[35495]: pgmap v959: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 700 op/s 2024-04-17T07:31:43.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:43 smithi086 ceph-mon[31733]: pgmap v960: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 11 MiB/s wr, 715 op/s 2024-04-17T07:31:43.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:43 smithi098 ceph-mon[35418]: pgmap v960: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 11 MiB/s wr, 715 op/s 2024-04-17T07:31:43.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:43 smithi103 ceph-mon[35495]: pgmap v960: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 11 MiB/s wr, 715 op/s 2024-04-17T07:31:44.362 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 740.0 s, 473.4 tps, lat 211.072 ms stddev 140.203 2024-04-17T07:31:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:44 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:31:44.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:44 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:31:44.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:44 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:31:44.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:44 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:31:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:44 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:31:44.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:44 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:31:45.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:45 smithi103 ceph-mon[35495]: pgmap v961: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 718 op/s 2024-04-17T07:31:45.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:45.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:45.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:45.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:45 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:45.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:45 smithi086 ceph-mon[31733]: pgmap v961: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 718 op/s 2024-04-17T07:31:45.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:45.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:45.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:45.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:45 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.086 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:45 smithi098 ceph-mon[35418]: pgmap v961: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 718 op/s 2024-04-17T07:31:46.086 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.086 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.086 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.086 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:45 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:46 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi103 to 176.4M 2024-04-17T07:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:46 smithi098 ceph-mon[35418]: Unable to set osd_memory_target on smithi103 to 185066291: error parsing value: Value '185066291' is below minimum 939524096 2024-04-17T07:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:31:46.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:31:46.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:46 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi103 to 176.4M 2024-04-17T07:31:46.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:46 smithi103 ceph-mon[35495]: Unable to set osd_memory_target on smithi103 to 185066291: error parsing value: Value '185066291' is below minimum 939524096 2024-04-17T07:31:46.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.778 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:31:46.779 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:31:46.779 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:31:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:46 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi103 to 176.4M 2024-04-17T07:31:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:46 smithi086 ceph-mon[31733]: Unable to set osd_memory_target on smithi103 to 185066291: error parsing value: Value '185066291' is below minimum 939524096 2024-04-17T07:31:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-17T07:31:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:31:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:31:46.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:31:47.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:47 smithi098 ceph-mon[35418]: Adjusting osd_memory_target on smithi098 to 176.4M 2024-04-17T07:31:47.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:47 smithi098 ceph-mon[35418]: pgmap v962: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.8 MiB/s wr, 651 op/s 2024-04-17T07:31:47.769 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:47 smithi098 ceph-mon[35418]: Unable to set osd_memory_target on smithi098 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:31:47.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:47 smithi103 ceph-mon[35495]: Adjusting osd_memory_target on smithi098 to 176.4M 2024-04-17T07:31:47.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:47 smithi103 ceph-mon[35495]: pgmap v962: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.8 MiB/s wr, 651 op/s 2024-04-17T07:31:47.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:47 smithi103 ceph-mon[35495]: Unable to set osd_memory_target on smithi098 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:31:47.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:47 smithi086 ceph-mon[31733]: Adjusting osd_memory_target on smithi098 to 176.4M 2024-04-17T07:31:47.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:47 smithi086 ceph-mon[31733]: pgmap v962: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.8 MiB/s wr, 651 op/s 2024-04-17T07:31:47.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:47 smithi086 ceph-mon[31733]: Unable to set osd_memory_target on smithi098 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-17T07:31:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 745.0 s, 552.7 tps, lat 181.027 ms stddev 103.182 2024-04-17T07:31:49.762 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:49 smithi098 ceph-mon[35418]: pgmap v963: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 12 MiB/s wr, 718 op/s 2024-04-17T07:31:49.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:49 smithi103 ceph-mon[35495]: pgmap v963: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 12 MiB/s wr, 718 op/s 2024-04-17T07:31:49.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:49 smithi086 ceph-mon[31733]: pgmap v963: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 12 MiB/s wr, 718 op/s 2024-04-17T07:31:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:51 smithi098 ceph-mon[35418]: pgmap v964: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 10 MiB/s wr, 682 op/s 2024-04-17T07:31:51.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:51 smithi103 ceph-mon[35495]: pgmap v964: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 10 MiB/s wr, 682 op/s 2024-04-17T07:31:51.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:51 smithi086 ceph-mon[31733]: pgmap v964: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 10 MiB/s wr, 682 op/s 2024-04-17T07:31:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:53 smithi098 ceph-mon[35418]: pgmap v965: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 717 op/s 2024-04-17T07:31:53.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:53 smithi103 ceph-mon[35495]: pgmap v965: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 717 op/s 2024-04-17T07:31:53.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:53 smithi086 ceph-mon[31733]: pgmap v965: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 717 op/s 2024-04-17T07:31:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 750.0 s, 476.5 tps, lat 196.267 ms stddev 117.270 2024-04-17T07:31:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:55 smithi098 ceph-mon[35418]: pgmap v966: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 740 op/s 2024-04-17T07:31:55.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:55 smithi103 ceph-mon[35495]: pgmap v966: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 740 op/s 2024-04-17T07:31:55.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:55 smithi086 ceph-mon[31733]: pgmap v966: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 11 MiB/s wr, 740 op/s 2024-04-17T07:31:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:57 smithi098 ceph-mon[35418]: pgmap v967: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.6 MiB/s wr, 672 op/s 2024-04-17T07:31:57.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:57 smithi103 ceph-mon[35495]: pgmap v967: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.6 MiB/s wr, 672 op/s 2024-04-17T07:31:57.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:57 smithi086 ceph-mon[31733]: pgmap v967: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.6 MiB/s wr, 672 op/s 2024-04-17T07:31:59.362 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 755.0 s, 474.2 tps, lat 102.796 ms stddev 47.073 2024-04-17T07:31:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:31:59 smithi098 ceph-mon[35418]: pgmap v968: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 9.6 MiB/s wr, 696 op/s 2024-04-17T07:31:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:31:59 smithi103 ceph-mon[35495]: pgmap v968: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 9.6 MiB/s wr, 696 op/s 2024-04-17T07:31:59.832 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:31:59 smithi086 ceph-mon[31733]: pgmap v968: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 9.6 MiB/s wr, 696 op/s 2024-04-17T07:32:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:01 smithi098 ceph-mon[35418]: pgmap v969: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.0 MiB/s wr, 688 op/s 2024-04-17T07:32:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:01 smithi103 ceph-mon[35495]: pgmap v969: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.0 MiB/s wr, 688 op/s 2024-04-17T07:32:01.829 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:01 smithi086 ceph-mon[31733]: pgmap v969: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.0 MiB/s wr, 688 op/s 2024-04-17T07:32:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:03 smithi098 ceph-mon[35418]: pgmap v970: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 10 MiB/s wr, 762 op/s 2024-04-17T07:32:03.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:03 smithi103 ceph-mon[35495]: pgmap v970: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 10 MiB/s wr, 762 op/s 2024-04-17T07:32:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:03 smithi086 ceph-mon[31733]: pgmap v970: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 10 MiB/s wr, 762 op/s 2024-04-17T07:32:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 760.0 s, 470.6 tps, lat 335.542 ms stddev 1252.089 2024-04-17T07:32:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:05 smithi098 ceph-mon[35418]: pgmap v971: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.6 MiB/s wr, 732 op/s 2024-04-17T07:32:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:05 smithi103 ceph-mon[35495]: pgmap v971: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.6 MiB/s wr, 732 op/s 2024-04-17T07:32:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:05 smithi086 ceph-mon[31733]: pgmap v971: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.6 MiB/s wr, 732 op/s 2024-04-17T07:32:07.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:07 smithi098 ceph-mon[35418]: pgmap v972: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.5 MiB/s wr, 669 op/s 2024-04-17T07:32:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:07 smithi103 ceph-mon[35495]: pgmap v972: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.5 MiB/s wr, 669 op/s 2024-04-17T07:32:07.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:07 smithi086 ceph-mon[31733]: pgmap v972: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.5 MiB/s wr, 669 op/s 2024-04-17T07:32:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 765.0 s, 506.0 tps, lat 196.726 ms stddev 137.368 2024-04-17T07:32:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:09 smithi098 ceph-mon[35418]: pgmap v973: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.5 MiB/s wr, 725 op/s 2024-04-17T07:32:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:09 smithi103 ceph-mon[35495]: pgmap v973: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.5 MiB/s wr, 725 op/s 2024-04-17T07:32:09.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:09 smithi086 ceph-mon[31733]: pgmap v973: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.5 MiB/s wr, 725 op/s 2024-04-17T07:32:10.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:32:10.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:32:10.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.366+0000 7f6de7200640 1 -- 172.21.15.86:0/3489529453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6de81019e0 msgr2=0x7f6de8101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.366+0000 7f6de7200640 1 --2- 172.21.15.86:0/3489529453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6de81019e0 0x7f6de8101e60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009900 tx=0x7f6de002bc40 comp rx=0 tx=0).stop 2024-04-17T07:32:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.366+0000 7f6de7200640 1 -- 172.21.15.86:0/3489529453 shutdown_connections 2024-04-17T07:32:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.366+0000 7f6de7200640 1 --2- 172.21.15.86:0/3489529453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6de81019e0 0x7f6de8101e60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.366+0000 7f6de7200640 1 --2- 172.21.15.86:0/3489529453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de8101040 0x7f6de81014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.366+0000 7f6de7200640 1 --2- 172.21.15.86:0/3489529453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de81006e0 0x7f6de8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.366+0000 7f6de7200640 1 -- 172.21.15.86:0/3489529453 >> 172.21.15.86:0/3489529453 conn(0x7f6de80fc300 msgr2=0x7f6de80fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.366+0000 7f6de7200640 1 -- 172.21.15.86:0/3489529453 shutdown_connections 2024-04-17T07:32:10.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.366+0000 7f6de7200640 1 -- 172.21.15.86:0/3489529453 wait complete. 2024-04-17T07:32:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.368+0000 7f6de7200640 1 Processor -- start 2024-04-17T07:32:10.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.368+0000 7f6de7200640 1 -- start start 2024-04-17T07:32:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.369+0000 7f6de7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de81006e0 0x7f6de819a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.369+0000 7f6de7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de8101040 0x7f6de819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.369+0000 7f6de7200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6de81019e0 0x7f6de81a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.369+0000 7f6de7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6de810fc10 con 0x7f6de8101040 2024-04-17T07:32:10.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.369+0000 7f6de7200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6de810fa90 con 0x7f6de81019e0 2024-04-17T07:32:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.369+0000 7f6de7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6de810fd90 con 0x7f6de81006e0 2024-04-17T07:32:10.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.370+0000 7f6de6800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6de81019e0 0x7f6de81a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.370+0000 7f6de6800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6de81019e0 0x7f6de81a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38340/0 (socket says 172.21.15.86:38340) 2024-04-17T07:32:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.370+0000 7f6de6800640 1 -- 172.21.15.86:0/3335982497 learned_addr learned my addr 172.21.15.86:0/3335982497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.370+0000 7f6de5e00640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de81006e0 0x7f6de819a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.370+0000 7f6de5400640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de8101040 0x7f6de819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.371+0000 7f6de6800640 1 -- 172.21.15.86:0/3335982497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de81006e0 msgr2=0x7f6de819a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.371+0000 7f6de6800640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de81006e0 0x7f6de819a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.371+0000 7f6de6800640 1 -- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de8101040 msgr2=0x7f6de819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.371+0000 7f6de6800640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de8101040 0x7f6de819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.371+0000 7f6de6800640 1 -- 172.21.15.86:0/3335982497 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de81a1df0 con 0x7f6de81019e0 2024-04-17T07:32:10.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.371+0000 7f6de5400640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de8101040 0x7f6de819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:32:10.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.372+0000 7f6de6800640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6de81019e0 0x7f6de81a1780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6de0036040 tx=0x7f6de0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:10.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.372+0000 7f6ddea00640 1 -- 172.21.15.86:0/3335982497 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0043020 con 0x7f6de81019e0 2024-04-17T07:32:10.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.372+0000 7f6de7200640 1 -- 172.21.15.86:0/3335982497 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de81a2080 con 0x7f6de81019e0 2024-04-17T07:32:10.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.372+0000 7f6ddea00640 1 -- 172.21.15.86:0/3335982497 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6de0004490 con 0x7f6de81019e0 2024-04-17T07:32:10.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.372+0000 7f6ddea00640 1 -- 172.21.15.86:0/3335982497 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0003d10 con 0x7f6de81019e0 2024-04-17T07:32:10.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.373+0000 7f6de7200640 1 -- 172.21.15.86:0/3335982497 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de81a2620 con 0x7f6de81019e0 2024-04-17T07:32:10.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.383+0000 7f6ddea00640 1 -- 172.21.15.86:0/3335982497 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f6de0046020 con 0x7f6de81019e0 2024-04-17T07:32:10.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.384+0000 7f6def798740 1 Processor -- start 2024-04-17T07:32:10.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.386+0000 7f6def798740 1 -- start start 2024-04-17T07:32:10.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.386+0000 7f6def798740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21bcd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.387+0000 7f6def798740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 0x563c21bd2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.387+0000 7f6def798740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bd2ed0 0x563c21bd5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.387+0000 7f6def798740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563c21b34120 con 0x563c21bd2ed0 2024-04-17T07:32:10.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.387+0000 7f6def798740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563c21ada340 con 0x563c21bcdd60 2024-04-17T07:32:10.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.387+0000 7f6def798740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563c21b300e0 con 0x563c21bcd370 2024-04-17T07:32:10.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.388+0000 7f6ddea00640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6dcc0a3c30 0x7f6dcc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.388+0000 7f6de5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21bcd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.388+0000 7f6de5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21bcd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41802/0 (socket says 172.21.15.86:41802) 2024-04-17T07:32:10.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.388+0000 7f6de5400640 1 -- 172.21.15.86:0/556858025 learned_addr learned my addr 172.21.15.86:0/556858025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:10.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.389+0000 7f6de5400640 1 --2- 172.21.15.86:0/556858025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bd2ed0 0x563c21bd5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.389+0000 7f6de5e00640 1 --2- 172.21.15.86:0/556858025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 0x563c21bd2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.390+0000 7f6de5400640 1 -- 172.21.15.86:0/556858025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 msgr2=0x563c21bd2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:10.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.390+0000 7f6de5400640 1 --2- 172.21.15.86:0/556858025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 0x563c21bd2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.390+0000 7f6de5400640 1 -- 172.21.15.86:0/556858025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bd2ed0 msgr2=0x563c21bd5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:10.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.390+0000 7f6de5400640 1 --2- 172.21.15.86:0/556858025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bd2ed0 0x563c21bd5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.390+0000 7f6de5400640 1 -- 172.21.15.86:0/556858025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c219107d0 con 0x563c21bcd370 2024-04-17T07:32:10.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.390+0000 7f6de5e00640 1 --2- 172.21.15.86:0/556858025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 0x563c21bd2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:10.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.390+0000 7f6ddea00640 1 -- 172.21.15.86:0/3335982497 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6de00f2bb0 con 0x7f6de81019e0 2024-04-17T07:32:10.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.391+0000 7f6de5400640 1 --2- 172.21.15.86:0/556858025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21bcd790 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4013550 tx=0x7f6dd400b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df23f2d1e5a91c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:10.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.391+0000 7f6de6800640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6dcc0a3c30 0x7f6dcc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.391+0000 7f6dd3000640 1 -- 172.21.15.86:0/556858025 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4019030 con 0x563c21bcd370 2024-04-17T07:32:10.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.392+0000 7f6de6800640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6dcc0a3c30 0x7f6dcc0a60f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6de0005ec0 tx=0x7f6de00df040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:10.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.393+0000 7f6dd3000640 1 -- 172.21.15.86:0/556858025 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dd4005020 con 0x563c21bcd370 2024-04-17T07:32:10.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.393+0000 7f6dd3000640 1 -- 172.21.15.86:0/556858025 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4004b20 con 0x563c21bcd370 2024-04-17T07:32:10.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.394+0000 7f6def798740 1 -- 172.21.15.86:0/556858025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 msgr2=0x563c21bcd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.394+0000 7f6def798740 1 --2- 172.21.15.86:0/556858025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21bcd790 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4013550 tx=0x7f6dd400b660 comp rx=0 tx=0).stop 2024-04-17T07:32:10.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.394+0000 7f6def798740 1 -- 172.21.15.86:0/556858025 shutdown_connections 2024-04-17T07:32:10.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.394+0000 7f6def798740 1 --2- 172.21.15.86:0/556858025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bd2ed0 0x563c21bd5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.394+0000 7f6def798740 1 --2- 172.21.15.86:0/556858025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 0x563c21bd2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.395+0000 7f6def798740 1 --2- 172.21.15.86:0/556858025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21bcd790 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.395+0000 7f6def798740 1 -- 172.21.15.86:0/556858025 >> 172.21.15.86:0/556858025 conn(0x563c21bc88e0 msgr2=0x563c21bc8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:10.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.396+0000 7f6def798740 1 -- 172.21.15.86:0/556858025 shutdown_connections 2024-04-17T07:32:10.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.396+0000 7f6def798740 1 -- 172.21.15.86:0/556858025 wait complete. 2024-04-17T07:32:10.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.398+0000 7f6def798740 1 -- start start 2024-04-17T07:32:10.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.399+0000 7f6def798740 1 Processor -- start 2024-04-17T07:32:10.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.400+0000 7f6def798740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21be00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.400+0000 7f6def798740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcdd60 0x563c21be0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.400+0000 7f6def798740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bd2ed0 0x563c21be7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.400+0000 7f6def798740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563c21bd5e80 con 0x563c21bcdd60 2024-04-17T07:32:10.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.400+0000 7f6def798740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563c21aed710 con 0x563c21bd2ed0 2024-04-17T07:32:10.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.400+0000 7f6def798740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563c21bd6000 con 0x563c21bcd370 2024-04-17T07:32:10.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.401+0000 7f6def798740 10 client.0 _ll_register_callbacks cb 0x563c21ae8af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:10.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.401+0000 7f6de5400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bd2ed0 0x563c21be7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.401+0000 7f6de5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21be00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.401+0000 7f6de5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21be00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41810/0 (socket says 172.21.15.86:41810) 2024-04-17T07:32:10.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.401+0000 7f6de5400640 1 -- 172.21.15.86:0/3953435033 learned_addr learned my addr 172.21.15.86:0/3953435033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:10.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.402+0000 7f6de5e00640 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcdd60 0x563c21be0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.403+0000 7f6de5e00640 1 -- 172.21.15.86:0/3953435033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 msgr2=0x563c21be00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:10.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.403+0000 7f6de5e00640 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21be00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.403+0000 7f6de5e00640 1 -- 172.21.15.86:0/3953435033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bd2ed0 msgr2=0x563c21be7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:10.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.403+0000 7f6de5e00640 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bd2ed0 0x563c21be7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.403+0000 7f6de5e00640 1 -- 172.21.15.86:0/3953435033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c218463f0 con 0x563c21bcdd60 2024-04-17T07:32:10.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.405+0000 7f6de5e00640 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcdd60 0x563c21be0630 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x563c21bce160 tx=0x7f6dd8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1390a19daa82a5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:10.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.405+0000 7f6dc7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.406+0000 7f6dc7400640 1 -- 172.21.15.86:0/3953435033 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8004d20 con 0x563c21bcdd60 2024-04-17T07:32:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.406+0000 7f6dc7400640 1 -- 172.21.15.86:0/3953435033 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6dd800f730 con 0x563c21bcdd60 2024-04-17T07:32:10.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.406+0000 7f6def798740 1 -- 172.21.15.86:0/3953435033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd4005020 con 0x563c21bcdd60 2024-04-17T07:32:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.406+0000 7f6dc7400640 1 -- 172.21.15.86:0/3953435033 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6dd800f930 con 0x563c21bcdd60 2024-04-17T07:32:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.407+0000 7f6def798740 10 client.19443 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:10.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.407+0000 7f6def798740 1 -- 172.21.15.86:0/3953435033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c218060f0 con 0x563c21bcdd60 2024-04-17T07:32:10.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.407+0000 7f6dc7400640 1 -- 172.21.15.86:0/3953435033 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dd4005020 con 0x563c21bcdd60 2024-04-17T07:32:10.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.407+0000 7f6dc7400640 1 -- 172.21.15.86:0/3953435033 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6dd800f040 con 0x563c21bcdd60 2024-04-17T07:32:10.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.408+0000 7f6def798740 10 client.19443 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:10.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.408+0000 7f6def798740 10 client.19443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:10.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.408+0000 7f6def798740 20 client.19443 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:10.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.408+0000 7f6def798740 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563c21bec080 0x563c21bee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.408+0000 7f6def798740 4 client.19443 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:10.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.408+0000 7f6def798740 1 -- 172.21.15.86:0/3953435033 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c21b41b40 con 0x563c21bec080 2024-04-17T07:32:10.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.409+0000 7f6de5400640 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563c21bec080 0x563c21bee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.409+0000 7f6de5400640 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563c21bec080 0x563c21bee4a0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3094fecec97649f8 server_cookie=9fa24aea31421693 in_seq=0 out_seq=0 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.410+0000 7f6dc7400640 10 client.19443 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.411+0000 7f6dc7400640 1 -- 172.21.15.86:0/3953435033 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c21b41b40 con 0x563c21bec080 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.411+0000 7f6dc7400640 10 client.19443 handle_command_reply: tid=0 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.411+0000 7f6def798740 1 client.19443 shutdown 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.412+0000 7f6def798740 1 -- 172.21.15.86:0/3953435033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcdd60 msgr2=0x563c21be0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.412+0000 7f6def798740 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcdd60 0x563c21be0630 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x563c21bce160 tx=0x7f6dd8011af0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.412+0000 7f6def798740 1 -- 172.21.15.86:0/3953435033 shutdown_connections 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.412+0000 7f6def798740 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563c21bec080 0x563c21bee4a0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.412+0000 7f6def798740 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bd2ed0 0x563c21be7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.412+0000 7f6def798740 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcdd60 0x563c21be0630 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.412+0000 7f6def798740 1 --2- 172.21.15.86:0/3953435033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcd370 0x563c21be00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.412+0000 7f6dc7400640 0 client.19443 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:10.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.412+0000 7f6def798740 1 -- 172.21.15.86:0/3953435033 >> 172.21.15.86:0/3953435033 conn(0x563c21bc88e0 msgr2=0x563c21bc7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:10.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.413+0000 7f6def798740 1 -- 172.21.15.86:0/3953435033 shutdown_connections 2024-04-17T07:32:10.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.414+0000 7f6def798740 1 -- 172.21.15.86:0/3953435033 wait complete. 2024-04-17T07:32:10.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.414+0000 7f6def798740 20 client.19443 trim_cache size 0 max 16384 2024-04-17T07:32:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.469+0000 7f6def798740 1 Processor -- start 2024-04-17T07:32:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.470+0000 7f6def798740 1 -- start start 2024-04-17T07:32:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.470+0000 7f6def798740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bc88e0 0x563c21be00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.470+0000 7f6def798740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcd370 0x563c21be7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.470+0000 7f6def798740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcdd60 0x563c21bcc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.470+0000 7f6def798740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81a85f0 con 0x563c21bc88e0 2024-04-17T07:32:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.470+0000 7f6def798740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81aa130 con 0x563c21bcd370 2024-04-17T07:32:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.470+0000 7f6def798740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81a8cc0 con 0x563c21bcdd60 2024-04-17T07:32:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.471+0000 7f6de5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcd370 0x563c21be7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.471+0000 7f6de5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcd370 0x563c21be7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38378/0 (socket says 172.21.15.86:38378) 2024-04-17T07:32:10.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.471+0000 7f6de5e00640 1 -- 172.21.15.86:0/4004360216 learned_addr learned my addr 172.21.15.86:0/4004360216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.471+0000 7f6de5e00640 1 -- 172.21.15.86:0/4004360216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcdd60 msgr2=0x563c21bcc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.471+0000 7f6de5e00640 1 --2- 172.21.15.86:0/4004360216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcdd60 0x563c21bcc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.471+0000 7f6de5e00640 1 -- 172.21.15.86:0/4004360216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bc88e0 msgr2=0x563c21be00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.472+0000 7f6de5e00640 1 --2- 172.21.15.86:0/4004360216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bc88e0 0x563c21be00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.472+0000 7f6de5e00640 1 -- 172.21.15.86:0/4004360216 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c21bcf940 con 0x563c21bcd370 2024-04-17T07:32:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.472+0000 7f6de5e00640 1 --2- 172.21.15.86:0/4004360216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcd370 0x563c21be7630 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8002230 tx=0x7f6dd8005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b7f028ef148b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.473+0000 7f6dd2600640 1 -- 172.21.15.86:0/4004360216 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801a520 con 0x563c21bcd370 2024-04-17T07:32:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.473+0000 7f6dd2600640 1 -- 172.21.15.86:0/4004360216 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dd801a6c0 con 0x563c21bcd370 2024-04-17T07:32:10.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.473+0000 7f6dd2600640 1 -- 172.21.15.86:0/4004360216 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd801ab50 con 0x563c21bcd370 2024-04-17T07:32:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.474+0000 7f6def798740 1 -- 172.21.15.86:0/4004360216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcd370 msgr2=0x563c21be7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.474+0000 7f6def798740 1 --2- 172.21.15.86:0/4004360216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcd370 0x563c21be7630 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8002230 tx=0x7f6dd8005d10 comp rx=0 tx=0).stop 2024-04-17T07:32:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.474+0000 7f6def798740 1 -- 172.21.15.86:0/4004360216 shutdown_connections 2024-04-17T07:32:10.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.474+0000 7f6def798740 1 --2- 172.21.15.86:0/4004360216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bcdd60 0x563c21bcc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.474+0000 7f6def798740 1 --2- 172.21.15.86:0/4004360216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcd370 0x563c21be7630 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.474+0000 7f6def798740 1 --2- 172.21.15.86:0/4004360216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bc88e0 0x563c21be00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.474+0000 7f6def798740 1 -- 172.21.15.86:0/4004360216 >> 172.21.15.86:0/4004360216 conn(0x563c21bceb40 msgr2=0x563c21bcbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:10.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.474+0000 7f6def798740 1 -- 172.21.15.86:0/4004360216 shutdown_connections 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.474+0000 7f6def798740 1 -- 172.21.15.86:0/4004360216 wait complete. 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.475+0000 7f6def798740 1 -- start start 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.475+0000 7f6def798740 1 Processor -- start 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.476+0000 7f6def798740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bc88e0 0x563c21be53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.476+0000 7f6def798740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcd370 0x563c21be5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.476+0000 7f6def798740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 0x563c21be6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.476+0000 7f6def798740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81a9a60 con 0x563c21bcd370 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.476+0000 7f6def798740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81a9390 con 0x563c21bcdd60 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.476+0000 7f6def798740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81a65c0 con 0x563c21bc88e0 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.476+0000 7f6def798740 10 client.0 _ll_register_callbacks cb 0x7f6dd8009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.476+0000 7f6de5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bc88e0 0x563c21be53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.476+0000 7f6de5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcd370 0x563c21be5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.476+0000 7f6de5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bc88e0 0x563c21be53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41842/0 (socket says 172.21.15.86:41842) 2024-04-17T07:32:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.477+0000 7f6de5400640 1 -- 172.21.15.86:0/2471346240 learned_addr learned my addr 172.21.15.86:0/2471346240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.477+0000 7f6de5400640 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 0x563c21be6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.477+0000 7f6de5e00640 1 -- 172.21.15.86:0/2471346240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bc88e0 msgr2=0x563c21be53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.477+0000 7f6de5e00640 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bc88e0 0x563c21be53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.477+0000 7f6de5e00640 1 -- 172.21.15.86:0/2471346240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 msgr2=0x563c21be6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.477+0000 7f6de5e00640 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 0x563c21be6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.477+0000 7f6de5e00640 1 -- 172.21.15.86:0/2471346240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c21bee810 con 0x563c21bcd370 2024-04-17T07:32:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.478+0000 7f6de5e00640 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcd370 0x563c21be5900 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x563c21bd2dc0 tx=0x7f6dd8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f398421953d9f3e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.478+0000 7f6dc7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.478+0000 7f6dc7400640 1 -- 172.21.15.86:0/2471346240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8037070 con 0x563c21bcd370 2024-04-17T07:32:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.478+0000 7f6def798740 1 -- 172.21.15.86:0/2471346240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd801a6c0 con 0x563c21bcd370 2024-04-17T07:32:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.478+0000 7f6dc7400640 1 -- 172.21.15.86:0/2471346240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6dd801ac80 con 0x563c21bcd370 2024-04-17T07:32:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.478+0000 7f6dc7400640 1 -- 172.21.15.86:0/2471346240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6dd8042050 con 0x563c21bcd370 2024-04-17T07:32:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.479+0000 7f6def798740 10 client.19446 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.479+0000 7f6def798740 1 -- 172.21.15.86:0/2471346240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c21c4c810 con 0x563c21bcd370 2024-04-17T07:32:10.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.479+0000 7f6dc7400640 1 -- 172.21.15.86:0/2471346240 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dd801a6c0 con 0x563c21bcd370 2024-04-17T07:32:10.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.479+0000 7f6dc7400640 1 -- 172.21.15.86:0/2471346240 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6dd8005040 con 0x563c21bcd370 2024-04-17T07:32:10.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.479+0000 7f6def798740 10 client.19446 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:10.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.479+0000 7f6def798740 10 client.19446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:10.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.479+0000 7f6def798740 20 client.19446 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:10.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.479+0000 7f6def798740 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563c21bd2ed0 0x563c21c4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.479+0000 7f6def798740 4 client.19446 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:32:10.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.479+0000 7f6def798740 1 -- 172.21.15.86:0/2471346240 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c21b3b9b0 con 0x563c21bd2ed0 2024-04-17T07:32:10.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.480+0000 7f6de5400640 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563c21bd2ed0 0x563c21c4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.481+0000 7f6dc7400640 10 client.19446 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:10.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.481+0000 7f6de5400640 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563c21bd2ed0 0x563c21c4f090 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e5340a53e2c19f server_cookie=a3aa35846898b939 in_seq=0 out_seq=0 2024-04-17T07:32:10.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.482+0000 7f6dc7400640 1 -- 172.21.15.86:0/2471346240 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563c21b3b9b0 con 0x563c21bd2ed0 2024-04-17T07:32:10.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.482+0000 7f6dc7400640 10 client.19446 handle_command_reply: tid=0 2024-04-17T07:32:10.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.482+0000 7f6def798740 1 client.19446 shutdown 2024-04-17T07:32:10.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.482+0000 7f6def798740 1 -- 172.21.15.86:0/2471346240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcd370 msgr2=0x563c21be5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.482+0000 7f6def798740 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcd370 0x563c21be5900 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x563c21bd2dc0 tx=0x7f6dd8005bd0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.483+0000 7f6def798740 1 -- 172.21.15.86:0/2471346240 shutdown_connections 2024-04-17T07:32:10.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.483+0000 7f6def798740 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x563c21bd2ed0 0x563c21c4f090 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.483+0000 7f6def798740 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563c21bcdd60 0x563c21be6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.483+0000 7f6def798740 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563c21bcd370 0x563c21be5900 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.483+0000 7f6def798740 1 --2- 172.21.15.86:0/2471346240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c21bc88e0 0x563c21be53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.483+0000 7f6def798740 1 -- 172.21.15.86:0/2471346240 >> 172.21.15.86:0/2471346240 conn(0x563c21bceb40 msgr2=0x563c21bcc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:10.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.483+0000 7f6dc7400640 0 client.19446 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:10.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.484+0000 7f6def798740 1 -- 172.21.15.86:0/2471346240 shutdown_connections 2024-04-17T07:32:10.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.484+0000 7f6def798740 1 -- 172.21.15.86:0/2471346240 wait complete. 2024-04-17T07:32:10.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.484+0000 7f6def798740 20 client.19446 trim_cache size 0 max 16384 2024-04-17T07:32:10.484 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:32:10.484 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:32:10.484 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:32:10.484 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:32:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.485+0000 7f6de7200640 1 -- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6dcc0a3c30 msgr2=0x7f6dcc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.485+0000 7f6de7200640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6dcc0a3c30 0x7f6dcc0a60f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6de0005ec0 tx=0x7f6de00df040 comp rx=0 tx=0).stop 2024-04-17T07:32:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.485+0000 7f6de7200640 1 -- 172.21.15.86:0/3335982497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6de81019e0 msgr2=0x7f6de81a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.485+0000 7f6de7200640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6de81019e0 0x7f6de81a1780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6de0036040 tx=0x7f6de0004270 comp rx=0 tx=0).stop 2024-04-17T07:32:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.485+0000 7f6de7200640 1 -- 172.21.15.86:0/3335982497 shutdown_connections 2024-04-17T07:32:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.485+0000 7f6de7200640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6dcc0a3c30 0x7f6dcc0a60f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.485+0000 7f6de7200640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6de81019e0 0x7f6de81a1780 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.485+0000 7f6de7200640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de8101040 0x7f6de819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.485+0000 7f6de7200640 1 --2- 172.21.15.86:0/3335982497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6de81006e0 0x7f6de819a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.485+0000 7f6de7200640 1 -- 172.21.15.86:0/3335982497 >> 172.21.15.86:0/3335982497 conn(0x7f6de80fc300 msgr2=0x7f6de80fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.486+0000 7f6de7200640 1 -- 172.21.15.86:0/3335982497 shutdown_connections 2024-04-17T07:32:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.486+0000 7f6de7200640 1 -- 172.21.15.86:0/3335982497 wait complete. 2024-04-17T07:32:10.520 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:32:10.520 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:32:10.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.870+0000 7efcefe00640 1 -- 172.21.15.86:0/542046927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce8101810 msgr2=0x7efce8101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.870+0000 7efcefe00640 1 --2- 172.21.15.86:0/542046927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce8101810 0x7efce8101c90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efcd400d910 tx=0x7efcd402dc00 comp rx=0 tx=0).stop 2024-04-17T07:32:10.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.870+0000 7efcefe00640 1 -- 172.21.15.86:0/542046927 shutdown_connections 2024-04-17T07:32:10.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.870+0000 7efcefe00640 1 --2- 172.21.15.86:0/542046927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce81021d0 0x7efce8068860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.870+0000 7efcefe00640 1 --2- 172.21.15.86:0/542046927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce8101810 0x7efce8101c90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.870+0000 7efcefe00640 1 --2- 172.21.15.86:0/542046927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efce8100620 0x7efce8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.870+0000 7efcefe00640 1 -- 172.21.15.86:0/542046927 >> 172.21.15.86:0/542046927 conn(0x7efce80fbb90 msgr2=0x7efce80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:10.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.871+0000 7efcefe00640 1 -- 172.21.15.86:0/542046927 shutdown_connections 2024-04-17T07:32:10.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.871+0000 7efcefe00640 1 -- 172.21.15.86:0/542046927 wait complete. 2024-04-17T07:32:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.873+0000 7efcefe00640 1 Processor -- start 2024-04-17T07:32:10.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.873+0000 7efcefe00640 1 -- start start 2024-04-17T07:32:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.874+0000 7efcefe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce8100620 0x7efce819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.874+0000 7efcefe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efce8101810 0x7efce819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.874+0000 7efcefe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce81021d0 0x7efce81a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.874+0000 7efcefe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efce810b900 con 0x7efce81021d0 2024-04-17T07:32:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.874+0000 7efcefe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efce810b780 con 0x7efce8101810 2024-04-17T07:32:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.874+0000 7efcefe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efce810ba80 con 0x7efce8100620 2024-04-17T07:32:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.875+0000 7efced400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce8100620 0x7efce819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.875+0000 7efcede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce81021d0 0x7efce81a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.875+0000 7efced400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce8100620 0x7efce819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41862/0 (socket says 172.21.15.86:41862) 2024-04-17T07:32:10.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.875+0000 7efced400640 1 -- 172.21.15.86:0/3026775149 learned_addr learned my addr 172.21.15.86:0/3026775149 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.875+0000 7efceca00640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efce8101810 0x7efce819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.876+0000 7efced400640 1 -- 172.21.15.86:0/3026775149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efce8101810 msgr2=0x7efce819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.876+0000 7efced400640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efce8101810 0x7efce819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.876+0000 7efced400640 1 -- 172.21.15.86:0/3026775149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce81021d0 msgr2=0x7efce81a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:10.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.876+0000 7efced400640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce81021d0 0x7efce81a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:10.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.876+0000 7efced400640 1 -- 172.21.15.86:0/3026775149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce81a1fa0 con 0x7efce8100620 2024-04-17T07:32:10.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.876+0000 7efced400640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce8100620 0x7efce819a380 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00e9c0 tx=0x7efcdc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:10.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.879+0000 7efce6000640 1 -- 172.21.15.86:0/3026775149 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00cd80 con 0x7efce8100620 2024-04-17T07:32:10.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.879+0000 7efce6000640 1 -- 172.21.15.86:0/3026775149 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcdc0035f0 con 0x7efce8100620 2024-04-17T07:32:10.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.879+0000 7efce6000640 1 -- 172.21.15.86:0/3026775149 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc010690 con 0x7efce8100620 2024-04-17T07:32:10.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.879+0000 7efcefe00640 1 -- 172.21.15.86:0/3026775149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce81a2290 con 0x7efce8100620 2024-04-17T07:32:10.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.880+0000 7efcefe00640 1 -- 172.21.15.86:0/3026775149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce81a2700 con 0x7efce8100620 2024-04-17T07:32:10.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.883+0000 7efcefe00640 1 -- 172.21.15.86:0/3026775149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce8105860 con 0x7efce8100620 2024-04-17T07:32:10.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.887+0000 7efce6000640 1 -- 172.21.15.86:0/3026775149 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7efcdc0037a0 con 0x7efce8100620 2024-04-17T07:32:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.889+0000 7efce6000640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efcd00a3d00 0x7efcd00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.889+0000 7efce6000640 1 -- 172.21.15.86:0/3026775149 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7efcdc0d0a00 con 0x7efce8100620 2024-04-17T07:32:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.890+0000 7efceca00640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efcd00a3d00 0x7efcd00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.891+0000 7efceca00640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efcd00a3d00 0x7efcd00a61c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efcd4004c80 tx=0x7efcd402e540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:10.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:10.906+0000 7efce6000640 1 -- 172.21.15.86:0/3026775149 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efcdc09d060 con 0x7efce8100620 2024-04-17T07:32:11.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.520+0000 7efcefe00640 1 -- 172.21.15.86:0/3026775149 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efce802d5d0 con 0x7efce8100620 2024-04-17T07:32:11.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.525+0000 7efce6000640 1 -- 172.21.15.86:0/3026775149 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7efcdc09c960 con 0x7efce8100620 2024-04-17T07:32:11.524 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:32:11.524 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:32:11.524 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:32:11.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.529+0000 7efcefe00640 1 -- 172.21.15.86:0/3026775149 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efcd00a3d00 msgr2=0x7efcd00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:11.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.529+0000 7efcefe00640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efcd00a3d00 0x7efcd00a61c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efcd4004c80 tx=0x7efcd402e540 comp rx=0 tx=0).stop 2024-04-17T07:32:11.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.529+0000 7efcefe00640 1 -- 172.21.15.86:0/3026775149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce8100620 msgr2=0x7efce819a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:11.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.529+0000 7efcefe00640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce8100620 0x7efce819a380 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00e9c0 tx=0x7efcdc00ee80 comp rx=0 tx=0).stop 2024-04-17T07:32:11.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.529+0000 7efcefe00640 1 -- 172.21.15.86:0/3026775149 shutdown_connections 2024-04-17T07:32:11.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.529+0000 7efcefe00640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7efcd00a3d00 0x7efcd00a61c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.529+0000 7efcefe00640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efce81021d0 0x7efce81a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.529+0000 7efcefe00640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efce8101810 0x7efce819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.529+0000 7efcefe00640 1 --2- 172.21.15.86:0/3026775149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efce8100620 0x7efce819a380 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.529+0000 7efcefe00640 1 -- 172.21.15.86:0/3026775149 >> 172.21.15.86:0/3026775149 conn(0x7efce80fbb90 msgr2=0x7efce8102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:11.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.532+0000 7efcefe00640 1 -- 172.21.15.86:0/3026775149 shutdown_connections 2024-04-17T07:32:11.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.532+0000 7efcefe00640 1 -- 172.21.15.86:0/3026775149 wait complete. 2024-04-17T07:32:11.566 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:32:11.735 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:11 smithi086 ceph-mon[31733]: pgmap v974: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 731 op/s 2024-04-17T07:32:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:11 smithi098 ceph-mon[35418]: pgmap v974: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 731 op/s 2024-04-17T07:32:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:11 smithi103 ceph-mon[35495]: pgmap v974: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 10 MiB/s wr, 731 op/s 2024-04-17T07:32:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.957+0000 7f8258c00640 1 -- 172.21.15.86:0/3439656390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8254100760 msgr2=0x7f8254100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.957+0000 7f8258c00640 1 --2- 172.21.15.86:0/3439656390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8254100760 0x7f8254100b80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8244009900 tx=0x7f824402bd10 comp rx=0 tx=0).stop 2024-04-17T07:32:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.957+0000 7f8258c00640 1 -- 172.21.15.86:0/3439656390 shutdown_connections 2024-04-17T07:32:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.957+0000 7f8258c00640 1 --2- 172.21.15.86:0/3439656390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8254101a60 0x7f8254101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.957+0000 7f8258c00640 1 --2- 172.21.15.86:0/3439656390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82541010c0 0x7f8254101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.957+0000 7f8258c00640 1 --2- 172.21.15.86:0/3439656390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8254100760 0x7f8254100b80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.957+0000 7f8258c00640 1 -- 172.21.15.86:0/3439656390 >> 172.21.15.86:0/3439656390 conn(0x7f82540fc300 msgr2=0x7f82540fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:11.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.960+0000 7f8258c00640 1 -- 172.21.15.86:0/3439656390 shutdown_connections 2024-04-17T07:32:11.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.960+0000 7f8258c00640 1 -- 172.21.15.86:0/3439656390 wait complete. 2024-04-17T07:32:11.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.962+0000 7f8258c00640 1 Processor -- start 2024-04-17T07:32:11.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.962+0000 7f8258c00640 1 -- start start 2024-04-17T07:32:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.962+0000 7f8258c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8254100760 0x7f82541051e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.963+0000 7f8258c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82541010c0 0x7f8254105720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.963+0000 7f8258c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8254101a60 0x7f825410c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.963+0000 7f8258c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f825410fd00 con 0x7f8254101a60 2024-04-17T07:32:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.963+0000 7f8258c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f825410fb80 con 0x7f82541010c0 2024-04-17T07:32:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.963+0000 7f8258c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f825410fe80 con 0x7f8254100760 2024-04-17T07:32:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.963+0000 7f8252a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82541010c0 0x7f8254105720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.963+0000 7f8252a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82541010c0 0x7f8254105720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38412/0 (socket says 172.21.15.86:38412) 2024-04-17T07:32:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.963+0000 7f8252a00640 1 -- 172.21.15.86:0/3830538698 learned_addr learned my addr 172.21.15.86:0/3830538698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.964+0000 7f8253400640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8254100760 0x7f82541051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.964+0000 7f8252a00640 1 -- 172.21.15.86:0/3830538698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8254100760 msgr2=0x7f82541051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.964+0000 7f8252a00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8254100760 0x7f82541051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.964+0000 7f8252a00640 1 -- 172.21.15.86:0/3830538698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8254101a60 msgr2=0x7f825410c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:11.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.964+0000 7f8252a00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8254101a60 0x7f825410c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.964+0000 7f8252a00640 1 -- 172.21.15.86:0/3830538698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82541aec80 con 0x7f82541010c0 2024-04-17T07:32:11.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.964+0000 7f8253400640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8254100760 0x7f82541051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:32:11.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.964+0000 7f8252a00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82541010c0 0x7f8254105720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f824000b450 tx=0x7f824000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:11.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.965+0000 7f824be00640 1 -- 172.21.15.86:0/3830538698 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240015020 con 0x7f82541010c0 2024-04-17T07:32:11.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.965+0000 7f824be00640 1 -- 172.21.15.86:0/3830538698 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8240011d40 con 0x7f82541010c0 2024-04-17T07:32:11.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.965+0000 7f824be00640 1 -- 172.21.15.86:0/3830538698 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824000f340 con 0x7f82541010c0 2024-04-17T07:32:11.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.965+0000 7f8258c00640 1 -- 172.21.15.86:0/3830538698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82541aef70 con 0x7f82541010c0 2024-04-17T07:32:11.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.965+0000 7f8258c00640 1 -- 172.21.15.86:0/3830538698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82541af400 con 0x7f82541010c0 2024-04-17T07:32:11.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.971+0000 7f825cf81740 1 Processor -- start 2024-04-17T07:32:11.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.972+0000 7f825cf81740 1 -- start start 2024-04-17T07:32:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.972+0000 7f825cf81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27160 0x55d026e27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.972+0000 7f825cf81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e2c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.972+0000 7f825cf81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e2ccc0 0x55d026e2f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.972+0000 7f825cf81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d026d8d020 con 0x55d026e27b50 2024-04-17T07:32:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.972+0000 7f825cf81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d026d33460 con 0x55d026e2ccc0 2024-04-17T07:32:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.972+0000 7f825cf81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d026d88fe0 con 0x55d026e27160 2024-04-17T07:32:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.973+0000 7f8253400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e2c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.973+0000 7f8253400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e2c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36210/0 (socket says 172.21.15.86:36210) 2024-04-17T07:32:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.973+0000 7f8253400640 1 -- 172.21.15.86:0/2750138816 learned_addr learned my addr 172.21.15.86:0/2750138816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.973+0000 7f8253e00640 1 --2- 172.21.15.86:0/2750138816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27160 0x55d026e27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.974+0000 7f8253400640 1 -- 172.21.15.86:0/2750138816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27160 msgr2=0x55d026e27580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.974+0000 7f8253400640 1 --2- 172.21.15.86:0/2750138816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27160 0x55d026e27580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.974+0000 7f8253400640 1 -- 172.21.15.86:0/2750138816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e2ccc0 msgr2=0x55d026e2f150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:32:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.974+0000 7f8253400640 1 --2- 172.21.15.86:0/2750138816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e2ccc0 0x55d026e2f150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.974+0000 7f8253400640 1 -- 172.21.15.86:0/2750138816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d026d3b040 con 0x55d026e27b50 2024-04-17T07:32:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.975+0000 7f8253400640 1 --2- 172.21.15.86:0/2750138816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e2c650 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f8244000c00 tx=0x7f824402d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a0e26a166108c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.978+0000 7f823fe00640 1 -- 172.21.15.86:0/2750138816 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244042020 con 0x55d026e27b50 2024-04-17T07:32:11.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.978+0000 7f823fe00640 1 -- 172.21.15.86:0/2750138816 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f824402dbe0 con 0x55d026e27b50 2024-04-17T07:32:11.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.978+0000 7f823fe00640 1 -- 172.21.15.86:0/2750138816 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82440046c0 con 0x55d026e27b50 2024-04-17T07:32:11.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.980+0000 7f825cf81740 1 -- 172.21.15.86:0/2750138816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 msgr2=0x55d026e2c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:11.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.980+0000 7f825cf81740 1 --2- 172.21.15.86:0/2750138816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e2c650 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f8244000c00 tx=0x7f824402d9f0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.983+0000 7f825cf81740 1 -- 172.21.15.86:0/2750138816 shutdown_connections 2024-04-17T07:32:11.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.983+0000 7f825cf81740 1 --2- 172.21.15.86:0/2750138816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e2ccc0 0x55d026e2f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.983+0000 7f825cf81740 1 --2- 172.21.15.86:0/2750138816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e2c650 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.983+0000 7f825cf81740 1 --2- 172.21.15.86:0/2750138816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27160 0x55d026e27580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.983+0000 7f825cf81740 1 -- 172.21.15.86:0/2750138816 >> 172.21.15.86:0/2750138816 conn(0x55d026e226d0 msgr2=0x55d026e22ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:11.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.983+0000 7f825cf81740 1 -- 172.21.15.86:0/2750138816 shutdown_connections 2024-04-17T07:32:11.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.985+0000 7f824be00640 1 -- 172.21.15.86:0/3830538698 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8240018020 con 0x7f82541010c0 2024-04-17T07:32:11.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.985+0000 7f825cf81740 1 -- 172.21.15.86:0/2750138816 wait complete. 2024-04-17T07:32:11.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.986+0000 7f825cf81740 1 -- start start 2024-04-17T07:32:11.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.986+0000 7f825cf81740 1 Processor -- start 2024-04-17T07:32:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.987+0000 7f825cf81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27160 0x55d026e31690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.987+0000 7f825cf81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27b50 0x55d026e31bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.987+0000 7f825cf81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e2ccc0 0x55d026e38c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.987+0000 7f825cf81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d026e2fc10 con 0x55d026e27160 2024-04-17T07:32:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.987+0000 7f825cf81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d026d46830 con 0x55d026e2ccc0 2024-04-17T07:32:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.987+0000 7f825cf81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d026e2fd90 con 0x55d026e27b50 2024-04-17T07:32:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.988+0000 7f825cf81740 10 client.0 _ll_register_callbacks cb 0x55d026d14fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.988+0000 7f8253400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27b50 0x55d026e31bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.988+0000 7f8253400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27b50 0x55d026e31bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41888/0 (socket says 172.21.15.86:41888) 2024-04-17T07:32:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.988+0000 7f8253400640 1 -- 172.21.15.86:0/3459843705 learned_addr learned my addr 172.21.15.86:0/3459843705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.988+0000 7f8252a00640 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e2ccc0 0x55d026e38c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:11.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.988+0000 7f8253e00640 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27160 0x55d026e31690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.989+0000 7f824be00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82380a3c10 0x7f82380a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.989+0000 7f8253400640 1 -- 172.21.15.86:0/3459843705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e2ccc0 msgr2=0x55d026e38c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.989+0000 7f8253400640 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e2ccc0 0x55d026e38c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.989+0000 7f8253400640 1 -- 172.21.15.86:0/3459843705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27160 msgr2=0x55d026e31690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.989+0000 7f8253400640 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27160 0x55d026e31690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.989+0000 7f8253400640 1 -- 172.21.15.86:0/3459843705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d026d4a920 con 0x55d026e27b50 2024-04-17T07:32:11.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.989+0000 7f824be00640 1 -- 172.21.15.86:0/3830538698 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f82400d0320 con 0x7f82541010c0 2024-04-17T07:32:11.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.990+0000 7f8252a00640 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e2ccc0 0x55d026e38c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:11.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.990+0000 7f8253400640 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27b50 0x55d026e31bd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f824402c780 tx=0x7f82440042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f01ec4b727951c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:11.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.990+0000 7f823cc00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:32:11.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.991+0000 7f823cc00640 1 -- 172.21.15.86:0/3459843705 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824404b070 con 0x55d026e27b50 2024-04-17T07:32:11.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.991+0000 7f823cc00640 1 -- 172.21.15.86:0/3459843705 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8244003a00 con 0x55d026e27b50 2024-04-17T07:32:11.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.993+0000 7f825cf81740 1 -- 172.21.15.86:0/3459843705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f824402dbe0 con 0x55d026e27b50 2024-04-17T07:32:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.994+0000 7f823cc00640 1 -- 172.21.15.86:0/3459843705 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8244055050 con 0x55d026e27b50 2024-04-17T07:32:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.995+0000 7f825cf81740 10 client.28483 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.995+0000 7f825cf81740 1 -- 172.21.15.86:0/3459843705 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d026d187f0 con 0x55d026e27b50 2024-04-17T07:32:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.995+0000 7f8253e00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82380a3c10 0x7f82380a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.995+0000 7f823cc00640 1 -- 172.21.15.86:0/3459843705 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f824402dbe0 con 0x55d026e27b50 2024-04-17T07:32:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.996+0000 7f823cc00640 1 -- 172.21.15.86:0/3459843705 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f8244047ba0 con 0x55d026e27b50 2024-04-17T07:32:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.996+0000 7f825cf81740 10 client.28483 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.996+0000 7f825cf81740 10 client.28483 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:32:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.997+0000 7f825cf81740 20 client.28483 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.997+0000 7f825cf81740 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55d026e3cd70 0x55d026e3f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.997+0000 7f825cf81740 4 client.28483 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.997+0000 7f825cf81740 1 -- 172.21.15.86:0/3459843705 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d026d9aa40 con 0x55d026e3cd70 2024-04-17T07:32:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.998+0000 7f8253e00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82380a3c10 0x7f82380a60d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x55d026e283b0 tx=0x7f824c004b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:11.999+0000 7f8252a00640 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55d026e3cd70 0x55d026e3f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.000+0000 7f8252a00640 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55d026e3cd70 0x55d026e3f160 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ffe93195cdcbefff server_cookie=9fdd5ab67b6a0bc7 in_seq=0 out_seq=0 2024-04-17T07:32:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.000+0000 7f823cc00640 10 client.28483 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:32:12.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.001+0000 7f823cc00640 1 -- 172.21.15.86:0/3459843705 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d026d9aa40 con 0x55d026e3cd70 2024-04-17T07:32:12.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.001+0000 7f823cc00640 10 client.28483 handle_command_reply: tid=0 2024-04-17T07:32:12.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.001+0000 7f825cf81740 1 client.28483 shutdown 2024-04-17T07:32:12.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.002+0000 7f825cf81740 1 -- 172.21.15.86:0/3459843705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27b50 msgr2=0x55d026e31bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.003+0000 7f825cf81740 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27b50 0x55d026e31bd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f824402c780 tx=0x7f82440042e0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.003+0000 7f825cf81740 1 -- 172.21.15.86:0/3459843705 shutdown_connections 2024-04-17T07:32:12.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.004+0000 7f825cf81740 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55d026e3cd70 0x55d026e3f160 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.004+0000 7f825cf81740 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e2ccc0 0x55d026e38c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.004+0000 7f825cf81740 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e27b50 0x55d026e31bd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.004+0000 7f825cf81740 1 --2- 172.21.15.86:0/3459843705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27160 0x55d026e31690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.004+0000 7f825cf81740 1 -- 172.21.15.86:0/3459843705 >> 172.21.15.86:0/3459843705 conn(0x55d026e226d0 msgr2=0x55d026e21930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.004+0000 7f823cc00640 0 client.28483 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:32:12.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.005+0000 7f825cf81740 1 -- 172.21.15.86:0/3459843705 shutdown_connections 2024-04-17T07:32:12.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.005+0000 7f825cf81740 1 -- 172.21.15.86:0/3459843705 wait complete. 2024-04-17T07:32:12.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.006+0000 7f825cf81740 20 client.28483 trim_cache size 0 max 16384 2024-04-17T07:32:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.047+0000 7f825cf81740 1 Processor -- start 2024-04-17T07:32:12.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.048+0000 7f825cf81740 1 -- start start 2024-04-17T07:32:12.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.049+0000 7f825cf81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 0x55d026e31650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.049+0000 7f825cf81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e31b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.049+0000 7f825cf81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e38bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.050+0000 7f825cf81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82541b9880 con 0x55d026e27b50 2024-04-17T07:32:12.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.050+0000 7f825cf81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82541bb3c0 con 0x55d026e27160 2024-04-17T07:32:12.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.050+0000 7f825cf81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82541b9f50 con 0x55d026e2ccc0 2024-04-17T07:32:12.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.051+0000 7f8253400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 0x55d026e31650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.051+0000 7f8253e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e38bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.052+0000 7f8253400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 0x55d026e31650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38430/0 (socket says 172.21.15.86:38430) 2024-04-17T07:32:12.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.052+0000 7f8253400640 1 -- 172.21.15.86:0/2420005050 learned_addr learned my addr 172.21.15.86:0/2420005050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:12.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.052+0000 7f8252a00640 1 --2- 172.21.15.86:0/2420005050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e31b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.052+0000 7f8253e00640 1 -- 172.21.15.86:0/2420005050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 msgr2=0x55d026e31650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.052+0000 7f8253e00640 1 --2- 172.21.15.86:0/2420005050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 0x55d026e31650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.052+0000 7f8253e00640 1 -- 172.21.15.86:0/2420005050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 msgr2=0x55d026e31b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.052+0000 7f8253e00640 1 --2- 172.21.15.86:0/2420005050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e31b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.052+0000 7f8253e00640 1 -- 172.21.15.86:0/2420005050 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d026a9f3f0 con 0x55d026e2ccc0 2024-04-17T07:32:12.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.054+0000 7f8253e00640 1 --2- 172.21.15.86:0/2420005050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e38bd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f824c00bcb0 tx=0x7f824c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d1e086f690a30a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.054+0000 7f823f400640 1 -- 172.21.15.86:0/2420005050 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c01c050 con 0x55d026e2ccc0 2024-04-17T07:32:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.054+0000 7f823f400640 1 -- 172.21.15.86:0/2420005050 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f824c026070 con 0x55d026e2ccc0 2024-04-17T07:32:12.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.054+0000 7f823f400640 1 -- 172.21.15.86:0/2420005050 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c021c20 con 0x55d026e2ccc0 2024-04-17T07:32:12.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.058+0000 7f825cf81740 1 -- 172.21.15.86:0/2420005050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 msgr2=0x55d026e38bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.058+0000 7f825cf81740 1 --2- 172.21.15.86:0/2420005050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e38bd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f824c00bcb0 tx=0x7f824c014000 comp rx=0 tx=0).stop 2024-04-17T07:32:12.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.059+0000 7f825cf81740 1 -- 172.21.15.86:0/2420005050 shutdown_connections 2024-04-17T07:32:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.059+0000 7f825cf81740 1 --2- 172.21.15.86:0/2420005050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e38bd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.059+0000 7f825cf81740 1 --2- 172.21.15.86:0/2420005050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e31b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.059+0000 7f825cf81740 1 --2- 172.21.15.86:0/2420005050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 0x55d026e31650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.059+0000 7f825cf81740 1 -- 172.21.15.86:0/2420005050 >> 172.21.15.86:0/2420005050 conn(0x55d026e226d0 msgr2=0x55d026e290e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.059+0000 7f825cf81740 1 -- 172.21.15.86:0/2420005050 shutdown_connections 2024-04-17T07:32:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.060+0000 7f825cf81740 1 -- 172.21.15.86:0/2420005050 wait complete. 2024-04-17T07:32:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.060+0000 7f825cf81740 1 -- start start 2024-04-17T07:32:12.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.061+0000 7f825cf81740 1 Processor -- start 2024-04-17T07:32:12.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.062+0000 7f825cf81740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 0x55d026e35d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.062+0000 7f825cf81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.062+0000 7f825cf81740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.062+0000 7f825cf81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82541bacf0 con 0x55d026e27b50 2024-04-17T07:32:12.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.062+0000 7f825cf81740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82541ba620 con 0x55d026e27160 2024-04-17T07:32:12.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.062+0000 7f825cf81740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82541b7850 con 0x55d026e2ccc0 2024-04-17T07:32:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.062+0000 7f825cf81740 10 client.0 _ll_register_callbacks cb 0x7f824402ded0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.063+0000 7f8253400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 0x55d026e35d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.063+0000 7f8253400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 0x55d026e35d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38444/0 (socket says 172.21.15.86:38444) 2024-04-17T07:32:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.063+0000 7f8253e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.063+0000 7f8253e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41894/0 (socket says 172.21.15.86:41894) 2024-04-17T07:32:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.063+0000 7f8253e00640 1 -- 172.21.15.86:0/396807016 learned_addr learned my addr 172.21.15.86:0/396807016 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.064+0000 7f8253e00640 1 -- 172.21.15.86:0/396807016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 msgr2=0x55d026e35d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.064+0000 7f8253e00640 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 0x55d026e35d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.064+0000 7f8253e00640 1 -- 172.21.15.86:0/396807016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 msgr2=0x55d026e362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.064+0000 7f8253e00640 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.064+0000 7f8253e00640 1 -- 172.21.15.86:0/396807016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d026b697d0 con 0x55d026e2ccc0 2024-04-17T07:32:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.065+0000 7f8253e00640 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e372e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f824c016860 tx=0x7f824c016890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97fb00b35fec75d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.065+0000 7f823cc00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:32:12.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.067+0000 7f823cc00640 1 -- 172.21.15.86:0/396807016 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c009530 con 0x55d026e2ccc0 2024-04-17T07:32:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.067+0000 7f823cc00640 1 -- 172.21.15.86:0/396807016 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f824c02a3f0 con 0x55d026e2ccc0 2024-04-17T07:32:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.068+0000 7f825cf81740 1 -- 172.21.15.86:0/396807016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f824c026070 con 0x55d026e2ccc0 2024-04-17T07:32:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.068+0000 7f823cc00640 1 -- 172.21.15.86:0/396807016 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f824c01c020 con 0x55d026e2ccc0 2024-04-17T07:32:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.068+0000 7f825cf81740 10 client.28489 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:12.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.068+0000 7f825cf81740 1 -- 172.21.15.86:0/396807016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d026e38a00 con 0x55d026e2ccc0 2024-04-17T07:32:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.069+0000 7f823cc00640 1 -- 172.21.15.86:0/396807016 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f824c026070 con 0x55d026e2ccc0 2024-04-17T07:32:12.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.069+0000 7f823cc00640 1 -- 172.21.15.86:0/396807016 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f824c03d020 con 0x55d026e2ccc0 2024-04-17T07:32:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.069+0000 7f825cf81740 10 client.28489 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.069+0000 7f825cf81740 10 client.28489 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:32:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.069+0000 7f825cf81740 20 client.28489 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.069+0000 7f825cf81740 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55d026ea3460 0x55d026ea5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.069+0000 7f825cf81740 4 client.28489 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:32:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.069+0000 7f825cf81740 1 -- 172.21.15.86:0/396807016 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d026d948b0 con 0x55d026ea3460 2024-04-17T07:32:12.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.070+0000 7f8253400640 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55d026ea3460 0x55d026ea5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.072+0000 7f8253400640 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55d026ea3460 0x55d026ea5880 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6116ffbdd585911c server_cookie=cf584a081d6ae37c in_seq=0 out_seq=0 2024-04-17T07:32:12.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.072+0000 7f823cc00640 10 client.28489 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:32:12.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.073+0000 7f823cc00640 1 -- 172.21.15.86:0/396807016 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d026d948b0 con 0x55d026ea3460 2024-04-17T07:32:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.073+0000 7f823cc00640 10 client.28489 handle_command_reply: tid=0 2024-04-17T07:32:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.074+0000 7f825cf81740 1 client.28489 shutdown 2024-04-17T07:32:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.074+0000 7f825cf81740 1 -- 172.21.15.86:0/396807016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 msgr2=0x55d026e372e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.074+0000 7f825cf81740 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e372e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f824c016860 tx=0x7f824c016890 comp rx=0 tx=0).stop 2024-04-17T07:32:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.075+0000 7f825cf81740 1 -- 172.21.15.86:0/396807016 shutdown_connections 2024-04-17T07:32:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.075+0000 7f825cf81740 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55d026ea3460 0x55d026ea5880 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.075+0000 7f825cf81740 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d026e2ccc0 0x55d026e372e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.075+0000 7f825cf81740 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d026e27b50 0x55d026e362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.075+0000 7f825cf81740 1 --2- 172.21.15.86:0/396807016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d026e27160 0x55d026e35d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.075+0000 7f825cf81740 1 -- 172.21.15.86:0/396807016 >> 172.21.15.86:0/396807016 conn(0x55d026e226d0 msgr2=0x55d026e31580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.075+0000 7f823cc00640 0 client.28489 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:32:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.075+0000 7f825cf81740 1 -- 172.21.15.86:0/396807016 shutdown_connections 2024-04-17T07:32:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.076+0000 7f825cf81740 1 -- 172.21.15.86:0/396807016 wait complete. 2024-04-17T07:32:12.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.076+0000 7f825cf81740 20 client.28489 trim_cache size 0 max 16384 2024-04-17T07:32:12.076 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:32:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.078+0000 7f8258c00640 1 -- 172.21.15.86:0/3830538698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82380a3c10 msgr2=0x7f82380a60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.078+0000 7f8258c00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82380a3c10 0x7f82380a60d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x55d026e283b0 tx=0x7f824c004b60 comp rx=0 tx=0).stop 2024-04-17T07:32:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.078+0000 7f8258c00640 1 -- 172.21.15.86:0/3830538698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82541010c0 msgr2=0x7f8254105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.078+0000 7f8258c00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82541010c0 0x7f8254105720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f824000b450 tx=0x7f824000b910 comp rx=0 tx=0).stop 2024-04-17T07:32:12.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.079+0000 7f8258c00640 1 -- 172.21.15.86:0/3830538698 shutdown_connections 2024-04-17T07:32:12.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.079+0000 7f8258c00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f82380a3c10 0x7f82380a60d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.079+0000 7f8258c00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8254101a60 0x7f825410c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.079+0000 7f8258c00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82541010c0 0x7f8254105720 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.079+0000 7f8258c00640 1 --2- 172.21.15.86:0/3830538698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8254100760 0x7f82541051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.079+0000 7f8258c00640 1 -- 172.21.15.86:0/3830538698 >> 172.21.15.86:0/3830538698 conn(0x7f82540fc300 msgr2=0x7f82540ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.081+0000 7f8258c00640 1 -- 172.21.15.86:0/3830538698 shutdown_connections 2024-04-17T07:32:12.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.081+0000 7f8258c00640 1 -- 172.21.15.86:0/3830538698 wait complete. 2024-04-17T07:32:12.104 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:32:12.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.452+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3960361616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28103190 msgr2=0x7f3c28103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.452+0000 7f3c2ce00640 1 --2- 172.21.15.86:0/3960361616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28103190 0x7f3c28103610 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009900 tx=0x7f3c1402bc30 comp rx=0 tx=0).stop 2024-04-17T07:32:12.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.452+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3960361616 shutdown_connections 2024-04-17T07:32:12.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.452+0000 7f3c2ce00640 1 --2- 172.21.15.86:0/3960361616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c28103b50 0x7f3c2810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.452+0000 7f3c2ce00640 1 --2- 172.21.15.86:0/3960361616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28103190 0x7f3c28103610 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.452+0000 7f3c2ce00640 1 --2- 172.21.15.86:0/3960361616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c28101fa0 0x7f3c281023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.452+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3960361616 >> 172.21.15.86:0/3960361616 conn(0x7f3c280fd510 msgr2=0x7f3c280ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.453+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3960361616 shutdown_connections 2024-04-17T07:32:12.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.453+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3960361616 wait complete. 2024-04-17T07:32:12.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.455+0000 7f3c2ce00640 1 Processor -- start 2024-04-17T07:32:12.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.455+0000 7f3c2ce00640 1 -- start start 2024-04-17T07:32:12.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.456+0000 7f3c2ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c28101fa0 0x7f3c28070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.456+0000 7f3c2ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28103190 0x7f3c28070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.456+0000 7f3c2ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c28103b50 0x7f3c281a8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.456+0000 7f3c2ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2810db70 con 0x7f3c28101fa0 2024-04-17T07:32:12.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.456+0000 7f3c2ce00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2810d9f0 con 0x7f3c28103b50 2024-04-17T07:32:12.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.456+0000 7f3c2ce00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2810dcf0 con 0x7f3c28103190 2024-04-17T07:32:12.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.456+0000 7f3c26800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c28103b50 0x7f3c281a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.456+0000 7f3c26800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c28103b50 0x7f3c281a8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38462/0 (socket says 172.21.15.86:38462) 2024-04-17T07:32:12.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.456+0000 7f3c26800640 1 -- 172.21.15.86:0/3435343544 learned_addr learned my addr 172.21.15.86:0/3435343544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:12.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.457+0000 7f3c25e00640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c28101fa0 0x7f3c28070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.457+0000 7f3c25400640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28103190 0x7f3c28070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.458+0000 7f3c25400640 1 -- 172.21.15.86:0/3435343544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c28103b50 msgr2=0x7f3c281a8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.458+0000 7f3c25400640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c28103b50 0x7f3c281a8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.458+0000 7f3c25400640 1 -- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c28101fa0 msgr2=0x7f3c28070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.458+0000 7f3c25400640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c28101fa0 0x7f3c28070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.458+0000 7f3c25400640 1 -- 172.21.15.86:0/3435343544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c281a87c0 con 0x7f3c28103190 2024-04-17T07:32:12.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.458+0000 7f3c26800640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c28103b50 0x7f3c281a8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:12.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.458+0000 7f3c25e00640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c28101fa0 0x7f3c28070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:12.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.458+0000 7f3c25400640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28103190 0x7f3c28070ec0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3c1402c170 tx=0x7f3c140040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.459+0000 7f3c1ea00640 1 -- 172.21.15.86:0/3435343544 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14043020 con 0x7f3c28103190 2024-04-17T07:32:12.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.459+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3435343544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c281a8990 con 0x7f3c28103190 2024-04-17T07:32:12.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.459+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3435343544 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c281a8f20 con 0x7f3c28103190 2024-04-17T07:32:12.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.459+0000 7f3c1ea00640 1 -- 172.21.15.86:0/3435343544 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c140044c0 con 0x7f3c28103190 2024-04-17T07:32:12.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.460+0000 7f3c1ea00640 1 -- 172.21.15.86:0/3435343544 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14003ba0 con 0x7f3c28103190 2024-04-17T07:32:12.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.465+0000 7f3c2fbac740 1 Processor -- start 2024-04-17T07:32:12.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.467+0000 7f3c1ea00640 1 -- 172.21.15.86:0/3435343544 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3c14046020 con 0x7f3c28103190 2024-04-17T07:32:12.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.469+0000 7f3c2fbac740 1 -- start start 2024-04-17T07:32:12.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.470+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.471+0000 7f3c1ea00640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3c0c0a3c30 0x7f3c0c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.471+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 0x5560d9d20400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.471+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d20a70 0x5560d9d2d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.471+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560d9c83020 con 0x5560d9d1d910 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.471+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5560d9c29460 con 0x5560d9d20a70 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.471+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560d9c7efe0 con 0x5560d9d1cf20 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.471+0000 7f3c1ea00640 1 -- 172.21.15.86:0/3435343544 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3c140f27f0 con 0x7f3c28103190 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.472+0000 7f3c26800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.472+0000 7f3c26800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41912/0 (socket says 172.21.15.86:41912) 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.472+0000 7f3c26800640 1 -- 172.21.15.86:0/3174449268 learned_addr learned my addr 172.21.15.86:0/3174449268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:12.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.473+0000 7f3c26800640 1 --2- 172.21.15.86:0/3174449268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d20a70 0x5560d9d2d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.473+0000 7f3c25e00640 1 --2- 172.21.15.86:0/3174449268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 0x5560d9d20400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.474+0000 7f3c25400640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3c0c0a3c30 0x7f3c0c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.476+0000 7f3c26800640 1 -- 172.21.15.86:0/3174449268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 msgr2=0x5560d9d1d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.476+0000 7f3c26800640 1 --2- 172.21.15.86:0/3174449268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.476+0000 7f3c26800640 1 -- 172.21.15.86:0/3174449268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 msgr2=0x5560d9d20400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.476+0000 7f3c26800640 1 --2- 172.21.15.86:0/3174449268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 0x5560d9d20400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.476+0000 7f3c26800640 1 -- 172.21.15.86:0/3174449268 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d9c31040 con 0x5560d9d20a70 2024-04-17T07:32:12.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.477+0000 7f3c26800640 1 --2- 172.21.15.86:0/3174449268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d20a70 0x5560d9d2d8c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x5560d9d1e170 tx=0x7f3c20009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae35d7cde91c36e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.481+0000 7f3c13400640 1 -- 172.21.15.86:0/3174449268 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2001f070 con 0x5560d9d20a70 2024-04-17T07:32:12.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.481+0000 7f3c13400640 1 -- 172.21.15.86:0/3174449268 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c20008c50 con 0x5560d9d20a70 2024-04-17T07:32:12.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.481+0000 7f3c13400640 1 -- 172.21.15.86:0/3174449268 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c200185c0 con 0x5560d9d20a70 2024-04-17T07:32:12.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.482+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3174449268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d20a70 msgr2=0x5560d9d2d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.482+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/3174449268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d20a70 0x5560d9d2d8c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x5560d9d1e170 tx=0x7f3c20009710 comp rx=0 tx=0).stop 2024-04-17T07:32:12.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.482+0000 7f3c25400640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3c0c0a3c30 0x7f3c0c0a60f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3c14006010 tx=0x7f3c1404afb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.487+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3174449268 shutdown_connections 2024-04-17T07:32:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.487+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/3174449268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d20a70 0x5560d9d2d8c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.487+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/3174449268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 0x5560d9d20400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.487+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/3174449268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.487+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3174449268 >> 172.21.15.86:0/3174449268 conn(0x5560d9d18510 msgr2=0x5560d9d18910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.488+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3174449268 shutdown_connections 2024-04-17T07:32:12.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.488+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3174449268 wait complete. 2024-04-17T07:32:12.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.489+0000 7f3c2fbac740 1 -- start start 2024-04-17T07:32:12.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.490+0000 7f3c2fbac740 1 Processor -- start 2024-04-17T07:32:12.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.491+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d1cf20 0x5560d9d2fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.491+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 0x5560d9d302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.491+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d20a70 0x5560d9d37330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.491+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560d9d2e2c0 con 0x5560d9d1d910 2024-04-17T07:32:12.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.491+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5560d9c3c830 con 0x5560d9d1cf20 2024-04-17T07:32:12.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.491+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560d9d2e440 con 0x5560d9d20a70 2024-04-17T07:32:12.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.492+0000 7f3c2fbac740 10 client.0 _ll_register_callbacks cb 0x5560d9c0afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:12.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.494+0000 7f3c25e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 0x5560d9d302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.494+0000 7f3c25e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 0x5560d9d302f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36254/0 (socket says 172.21.15.86:36254) 2024-04-17T07:32:12.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.494+0000 7f3c25e00640 1 -- 172.21.15.86:0/3017849005 learned_addr learned my addr 172.21.15.86:0/3017849005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:12.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.494+0000 7f3c25e00640 1 -- 172.21.15.86:0/3017849005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d20a70 msgr2=0x5560d9d37330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:32:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.494+0000 7f3c25e00640 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d20a70 0x5560d9d37330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.494+0000 7f3c25e00640 1 -- 172.21.15.86:0/3017849005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d1cf20 msgr2=0x5560d9d2fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.494+0000 7f3c25e00640 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d1cf20 0x5560d9d2fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.494+0000 7f3c25e00640 1 -- 172.21.15.86:0/3017849005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d9c40920 con 0x5560d9d1d910 2024-04-17T07:32:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.495+0000 7f3c25e00640 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 0x5560d9d302f0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3c18012d00 tx=0x7f3c1800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03f079f732238b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.495+0000 7f3c07e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:12.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.499+0000 7f3c07e00640 1 -- 172.21.15.86:0/3017849005 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c180120d0 con 0x5560d9d1d910 2024-04-17T07:32:12.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.499+0000 7f3c07e00640 1 -- 172.21.15.86:0/3017849005 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3c180047a0 con 0x5560d9d1d910 2024-04-17T07:32:12.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.499+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3017849005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c20008c50 con 0x5560d9d1d910 2024-04-17T07:32:12.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.500+0000 7f3c07e00640 1 -- 172.21.15.86:0/3017849005 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c18012270 con 0x5560d9d1d910 2024-04-17T07:32:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.501+0000 7f3c2fbac740 10 client.19467 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.501+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3017849005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560d9c0e7f0 con 0x5560d9d1d910 2024-04-17T07:32:12.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.502+0000 7f3c07e00640 1 -- 172.21.15.86:0/3017849005 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c20008c50 con 0x5560d9d1d910 2024-04-17T07:32:12.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.502+0000 7f3c07e00640 1 -- 172.21.15.86:0/3017849005 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3c180105b0 con 0x5560d9d1d910 2024-04-17T07:32:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.503+0000 7f3c2fbac740 10 client.19467 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:12.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.506+0000 7f3c2fbac740 10 client.19467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:32:12.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.506+0000 7f3c2fbac740 20 client.19467 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:12.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.506+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560d9d3b8a0 0x5560d9d3dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.506+0000 7f3c2fbac740 4 client.19467 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:12.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.506+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3017849005 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560d9c90a40 con 0x5560d9d3b8a0 2024-04-17T07:32:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.507+0000 7f3c26800640 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560d9d3b8a0 0x5560d9d3dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.509+0000 7f3c26800640 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560d9d3b8a0 0x5560d9d3dd40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b08b482fce363028 server_cookie=4b711547ac951126 in_seq=0 out_seq=0 2024-04-17T07:32:12.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.509+0000 7f3c07e00640 10 client.19467 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:32:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.510+0000 7f3c07e00640 1 -- 172.21.15.86:0/3017849005 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560d9c90a40 con 0x5560d9d3b8a0 2024-04-17T07:32:12.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.510+0000 7f3c07e00640 10 client.19467 handle_command_reply: tid=0 2024-04-17T07:32:12.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.511+0000 7f3c2fbac740 1 client.19467 shutdown 2024-04-17T07:32:12.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.512+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3017849005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 msgr2=0x5560d9d302f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.512+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 0x5560d9d302f0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3c18012d00 tx=0x7f3c1800f790 comp rx=0 tx=0).stop 2024-04-17T07:32:12.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.512+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3017849005 shutdown_connections 2024-04-17T07:32:12.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.512+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560d9d3b8a0 0x5560d9d3dd40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.512+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d20a70 0x5560d9d37330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.512+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d1d910 0x5560d9d302f0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.512+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/3017849005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d1cf20 0x5560d9d2fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.512+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3017849005 >> 172.21.15.86:0/3017849005 conn(0x5560d9d18510 msgr2=0x5560d9d17710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.512+0000 7f3c07e00640 0 client.19467 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:32:12.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.512+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3017849005 shutdown_connections 2024-04-17T07:32:12.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.513+0000 7f3c2fbac740 1 -- 172.21.15.86:0/3017849005 wait complete. 2024-04-17T07:32:12.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.513+0000 7f3c2fbac740 20 client.19467 trim_cache size 0 max 16384 2024-04-17T07:32:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.540+0000 7f3c2fbac740 1 Processor -- start 2024-04-17T07:32:12.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.541+0000 7f3c2fbac740 1 -- start start 2024-04-17T07:32:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.541+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.541+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d20a70 0x5560d9d372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.541+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d18d50 0x5560d9d42600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.541+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281b1190 con 0x5560d9d20a70 2024-04-17T07:32:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.541+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281b2cd0 con 0x5560d9d18d50 2024-04-17T07:32:12.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.541+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281b1860 con 0x5560d9d1cf20 2024-04-17T07:32:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.542+0000 7f3c26800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.542+0000 7f3c26800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41936/0 (socket says 172.21.15.86:41936) 2024-04-17T07:32:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.542+0000 7f3c25e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d20a70 0x5560d9d372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.543+0000 7f3c26800640 1 -- 172.21.15.86:0/960627205 learned_addr learned my addr 172.21.15.86:0/960627205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:12.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.543+0000 7f3c26800640 1 --2- 172.21.15.86:0/960627205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d18d50 0x5560d9d42600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.543+0000 7f3c26800640 1 -- 172.21.15.86:0/960627205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d18d50 msgr2=0x5560d9d42600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.543+0000 7f3c26800640 1 --2- 172.21.15.86:0/960627205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d18d50 0x5560d9d42600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.543+0000 7f3c26800640 1 -- 172.21.15.86:0/960627205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d20a70 msgr2=0x5560d9d372f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.543+0000 7f3c26800640 1 --2- 172.21.15.86:0/960627205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d20a70 0x5560d9d372f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.543+0000 7f3c26800640 1 -- 172.21.15.86:0/960627205 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560d99953f0 con 0x5560d9d1cf20 2024-04-17T07:32:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.545+0000 7f3c26800640 1 --2- 172.21.15.86:0/960627205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1be80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3c20008f60 tx=0x7f3c20007040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71be3ae83923bda0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.546+0000 7f3c12a00640 1 -- 172.21.15.86:0/960627205 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2001f040 con 0x5560d9d1cf20 2024-04-17T07:32:12.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.546+0000 7f3c12a00640 1 -- 172.21.15.86:0/960627205 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c20008c50 con 0x5560d9d1cf20 2024-04-17T07:32:12.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.547+0000 7f3c12a00640 1 -- 172.21.15.86:0/960627205 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2001ac30 con 0x5560d9d1cf20 2024-04-17T07:32:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.548+0000 7f3c2fbac740 1 -- 172.21.15.86:0/960627205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 msgr2=0x5560d9d1be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.548+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/960627205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1be80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3c20008f60 tx=0x7f3c20007040 comp rx=0 tx=0).stop 2024-04-17T07:32:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.549+0000 7f3c2fbac740 1 -- 172.21.15.86:0/960627205 shutdown_connections 2024-04-17T07:32:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.549+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/960627205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d18d50 0x5560d9d42600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.549+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/960627205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d20a70 0x5560d9d372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.549+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/960627205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d1cf20 0x5560d9d1be80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.549+0000 7f3c2fbac740 1 -- 172.21.15.86:0/960627205 >> 172.21.15.86:0/960627205 conn(0x5560d9d18510 msgr2=0x5560d9d1b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.551+0000 7f3c2fbac740 1 -- 172.21.15.86:0/960627205 shutdown_connections 2024-04-17T07:32:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.551+0000 7f3c2fbac740 1 -- 172.21.15.86:0/960627205 wait complete. 2024-04-17T07:32:12.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.552+0000 7f3c2fbac740 1 -- start start 2024-04-17T07:32:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.552+0000 7f3c2fbac740 1 Processor -- start 2024-04-17T07:32:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.553+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d18d50 0x5560d9d32df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.553+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d1cf20 0x5560d9d33330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.553+0000 7f3c2fbac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d20a70 0x5560d9d36350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.553+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281b2600 con 0x5560d9d18d50 2024-04-17T07:32:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.553+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281b1f30 con 0x5560d9d1cf20 2024-04-17T07:32:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.553+0000 7f3c2fbac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281af160 con 0x5560d9d20a70 2024-04-17T07:32:12.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.553+0000 7f3c2fbac740 10 client.0 _ll_register_callbacks cb 0x7f3c18012910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.554+0000 7f3c26800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d18d50 0x5560d9d32df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.554+0000 7f3c26800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d18d50 0x5560d9d32df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36268/0 (socket says 172.21.15.86:36268) 2024-04-17T07:32:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.554+0000 7f3c26800640 1 -- 172.21.15.86:0/1853603044 learned_addr learned my addr 172.21.15.86:0/1853603044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.554+0000 7f3c26800640 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d20a70 0x5560d9d36350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.554+0000 7f3c25e00640 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d1cf20 0x5560d9d33330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.555+0000 7f3c26800640 1 -- 172.21.15.86:0/1853603044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d20a70 msgr2=0x5560d9d36350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.555+0000 7f3c26800640 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d20a70 0x5560d9d36350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.555+0000 7f3c26800640 1 -- 172.21.15.86:0/1853603044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d1cf20 msgr2=0x5560d9d33330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.555+0000 7f3c26800640 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d1cf20 0x5560d9d33330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.555+0000 7f3c26800640 1 -- 172.21.15.86:0/1853603044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560d9a5f7d0 con 0x5560d9d18d50 2024-04-17T07:32:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.558+0000 7f3c26800640 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d18d50 0x5560d9d32df0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c010 tx=0x7f3c20004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c933d9c9b1726f9e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.558+0000 7f3c07e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.561+0000 7f3c07e00640 1 -- 172.21.15.86:0/1853603044 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2001f050 con 0x5560d9d18d50 2024-04-17T07:32:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.561+0000 7f3c07e00640 1 -- 172.21.15.86:0/1853603044 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3c2001a580 con 0x5560d9d18d50 2024-04-17T07:32:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.561+0000 7f3c2fbac740 1 -- 172.21.15.86:0/1853603044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c20008c50 con 0x5560d9d18d50 2024-04-17T07:32:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.562+0000 7f3c07e00640 1 -- 172.21.15.86:0/1853603044 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c2001a9b0 con 0x5560d9d18d50 2024-04-17T07:32:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.562+0000 7f3c2fbac740 10 client.19473 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:12.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.562+0000 7f3c2fbac740 1 -- 172.21.15.86:0/1853603044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560d9d25070 con 0x5560d9d18d50 2024-04-17T07:32:12.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.562+0000 7f3c07e00640 1 -- 172.21.15.86:0/1853603044 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c20008c50 con 0x5560d9d18d50 2024-04-17T07:32:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.563+0000 7f3c07e00640 1 -- 172.21.15.86:0/1853603044 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3c20018040 con 0x5560d9d18d50 2024-04-17T07:32:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.563+0000 7f3c2fbac740 10 client.19473 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.563+0000 7f3c2fbac740 10 client.19473 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:32:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.563+0000 7f3c2fbac740 20 client.19473 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.563+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560d9d257f0 0x5560d9d27c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.563+0000 7f3c2fbac740 4 client.19473 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:32:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.563+0000 7f3c2fbac740 1 -- 172.21.15.86:0/1853603044 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560d9c8a8b0 con 0x5560d9d257f0 2024-04-17T07:32:12.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.563+0000 7f3c25e00640 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560d9d257f0 0x5560d9d27c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.564+0000 7f3c25e00640 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560d9d257f0 0x5560d9d27c10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b733fe10100c69bc server_cookie=6db3fecdd16cca52 in_seq=0 out_seq=0 2024-04-17T07:32:12.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.565+0000 7f3c07e00640 10 client.19473 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:32:12.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.565+0000 7f3c07e00640 1 -- 172.21.15.86:0/1853603044 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560d9c8a8b0 con 0x5560d9d257f0 2024-04-17T07:32:12.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.565+0000 7f3c07e00640 10 client.19473 handle_command_reply: tid=0 2024-04-17T07:32:12.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.565+0000 7f3c2fbac740 1 client.19473 shutdown 2024-04-17T07:32:12.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.567+0000 7f3c2fbac740 1 -- 172.21.15.86:0/1853603044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d18d50 msgr2=0x5560d9d32df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.567+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d18d50 0x5560d9d32df0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c010 tx=0x7f3c20004b50 comp rx=0 tx=0).stop 2024-04-17T07:32:12.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.568+0000 7f3c2fbac740 1 -- 172.21.15.86:0/1853603044 shutdown_connections 2024-04-17T07:32:12.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.568+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560d9d257f0 0x5560d9d27c10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.568+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5560d9d20a70 0x5560d9d36350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.568+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5560d9d1cf20 0x5560d9d33330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.568+0000 7f3c2fbac740 1 --2- 172.21.15.86:0/1853603044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5560d9d18d50 0x5560d9d32df0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.568+0000 7f3c07e00640 0 client.19473 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:32:12.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.568+0000 7f3c2fbac740 1 -- 172.21.15.86:0/1853603044 >> 172.21.15.86:0/1853603044 conn(0x5560d9d18510 msgr2=0x5560d9d1c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.568+0000 7f3c2fbac740 1 -- 172.21.15.86:0/1853603044 shutdown_connections 2024-04-17T07:32:12.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.568+0000 7f3c2fbac740 1 -- 172.21.15.86:0/1853603044 wait complete. 2024-04-17T07:32:12.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.569+0000 7f3c2fbac740 20 client.19473 trim_cache size 0 max 16384 2024-04-17T07:32:12.569 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:32:12.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.570+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3c0c0a3c30 msgr2=0x7f3c0c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.570+0000 7f3c2ce00640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3c0c0a3c30 0x7f3c0c0a60f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3c14006010 tx=0x7f3c1404afb0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.570+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3435343544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28103190 msgr2=0x7f3c28070ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.570+0000 7f3c2ce00640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28103190 0x7f3c28070ec0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3c1402c170 tx=0x7f3c140040c0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.571+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3435343544 shutdown_connections 2024-04-17T07:32:12.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.571+0000 7f3c2ce00640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3c0c0a3c30 0x7f3c0c0a60f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.571+0000 7f3c2ce00640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c28103b50 0x7f3c281a8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.571+0000 7f3c2ce00640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c28103190 0x7f3c28070ec0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.571+0000 7f3c2ce00640 1 --2- 172.21.15.86:0/3435343544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c28101fa0 0x7f3c28070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.571+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3435343544 >> 172.21.15.86:0/3435343544 conn(0x7f3c280fd510 msgr2=0x7f3c28106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.571+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3435343544 shutdown_connections 2024-04-17T07:32:12.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.571+0000 7f3c2ce00640 1 -- 172.21.15.86:0/3435343544 wait complete. 2024-04-17T07:32:12.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:32:12.725 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3026775149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:32:12.752 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3026775149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:32:12.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3026775149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:32:12.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.934+0000 7fc6d2e00640 1 -- 172.21.15.86:0/3675542298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc073240 msgr2=0x7fc6cc073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.934+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/3675542298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc073240 0x7fc6cc073660 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009900 tx=0x7fc6c002bc40 comp rx=0 tx=0).stop 2024-04-17T07:32:12.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.936+0000 7fc6d2e00640 1 -- 172.21.15.86:0/3675542298 shutdown_connections 2024-04-17T07:32:12.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.936+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/3675542298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6cc109f00 0x7fc6cc10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.936+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/3675542298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc109540 0x7fc6cc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.936+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/3675542298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc073240 0x7fc6cc073660 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.936+0000 7fc6d2e00640 1 -- 172.21.15.86:0/3675542298 >> 172.21.15.86:0/3675542298 conn(0x7fc6cc06edb0 msgr2=0x7fc6cc0711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.936+0000 7fc6d2e00640 1 -- 172.21.15.86:0/3675542298 shutdown_connections 2024-04-17T07:32:12.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.936+0000 7fc6d2e00640 1 -- 172.21.15.86:0/3675542298 wait complete. 2024-04-17T07:32:12.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.937+0000 7fc6d2e00640 1 Processor -- start 2024-04-17T07:32:12.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.937+0000 7fc6d2e00640 1 -- start start 2024-04-17T07:32:12.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.938+0000 7fc6d2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc073240 0x7fc6cc1a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.938+0000 7fc6d1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc073240 0x7fc6cc1a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.938+0000 7fc6d1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc073240 0x7fc6cc1a6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36294/0 (socket says 172.21.15.86:36294) 2024-04-17T07:32:12.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.939+0000 7fc6d2e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc109540 0x7fc6cc1a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.939+0000 7fc6d1a00640 1 -- 172.21.15.86:0/594681732 learned_addr learned my addr 172.21.15.86:0/594681732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:12.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.939+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6cc109f00 0x7fc6cc1ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.939+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc10ddb0 con 0x7fc6cc073240 2024-04-17T07:32:12.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.939+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc10dc30 con 0x7fc6cc109f00 2024-04-17T07:32:12.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.939+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc10df30 con 0x7fc6cc109540 2024-04-17T07:32:12.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.939+0000 7fc6d1000640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc109540 0x7fc6cc1a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.940+0000 7fc6d2400640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6cc109f00 0x7fc6cc1ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.941+0000 7fc6d1a00640 1 -- 172.21.15.86:0/594681732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc109540 msgr2=0x7fc6cc1a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.941+0000 7fc6d1a00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc109540 0x7fc6cc1a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.941+0000 7fc6d1a00640 1 -- 172.21.15.86:0/594681732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6cc109f00 msgr2=0x7fc6cc1ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.942+0000 7fc6d1a00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6cc109f00 0x7fc6cc1ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.942+0000 7fc6d1a00640 1 -- 172.21.15.86:0/594681732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6cc1ae7c0 con 0x7fc6cc073240 2024-04-17T07:32:12.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.944+0000 7fc6d1a00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc073240 0x7fc6cc1a6ba0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009850 tx=0x7fc6c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.945+0000 7fc6ca000640 1 -- 172.21.15.86:0/594681732 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0043020 con 0x7fc6cc073240 2024-04-17T07:32:12.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.945+0000 7fc6ca000640 1 -- 172.21.15.86:0/594681732 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6c00054c0 con 0x7fc6cc073240 2024-04-17T07:32:12.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.945+0000 7fc6ca000640 1 -- 172.21.15.86:0/594681732 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0003e20 con 0x7fc6cc073240 2024-04-17T07:32:12.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.945+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6cc1aea50 con 0x7fc6cc073240 2024-04-17T07:32:12.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.946+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6cc1aef50 con 0x7fc6cc073240 2024-04-17T07:32:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.953+0000 7fc6d7169740 1 Processor -- start 2024-04-17T07:32:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.953+0000 7fc6d7169740 1 -- start start 2024-04-17T07:32:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.954+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.954+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e0c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.954+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 0x556057e0f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.954+0000 7fc6d7169740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556057d6d020 con 0x556057e071a0 2024-04-17T07:32:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.954+0000 7fc6d7169740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556057d13460 con 0x556057e0cd00 2024-04-17T07:32:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.954+0000 7fc6d7169740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556057d68fe0 con 0x556057e07b90 2024-04-17T07:32:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.955+0000 7fc6d2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.955+0000 7fc6d2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e075c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36310/0 (socket says 172.21.15.86:36310) 2024-04-17T07:32:12.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.955+0000 7fc6d2400640 1 -- 172.21.15.86:0/1227867612 learned_addr learned my addr 172.21.15.86:0/1227867612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:12.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.956+0000 7fc6ca000640 1 -- 172.21.15.86:0/594681732 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc6c0046020 con 0x7fc6cc073240 2024-04-17T07:32:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.957+0000 7fc6d2400640 1 -- 172.21.15.86:0/1227867612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 msgr2=0x556057e0c690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:32:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.957+0000 7fc6d2400640 1 --2- 172.21.15.86:0/1227867612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e0c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.957+0000 7fc6d2400640 1 -- 172.21.15.86:0/1227867612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 msgr2=0x556057e0f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.957+0000 7fc6d1000640 1 --2- 172.21.15.86:0/1227867612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 0x556057e0f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.957+0000 7fc6d2400640 1 --2- 172.21.15.86:0/1227867612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 0x556057e0f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.957+0000 7fc6d2400640 1 -- 172.21.15.86:0/1227867612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556057d1b040 con 0x556057e071a0 2024-04-17T07:32:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.958+0000 7fc6d1000640 1 --2- 172.21.15.86:0/1227867612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 0x556057e0f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:32:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.958+0000 7fc6d2400640 1 --2- 172.21.15.86:0/1227867612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e075c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009fd0 tx=0x7fc6c4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edec6674beda7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.960+0000 7fc6ca000640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc6b40a3d80 0x7fc6b40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.960+0000 7fc6ca000640 1 -- 172.21.15.86:0/594681732 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc6c00f2ca0 con 0x7fc6cc073240 2024-04-17T07:32:12.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.961+0000 7fc6d1a00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc6b40a3d80 0x7fc6b40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.962+0000 7fc6bea00640 1 -- 172.21.15.86:0/1227867612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4011700 con 0x556057e071a0 2024-04-17T07:32:12.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.962+0000 7fc6bea00640 1 -- 172.21.15.86:0/1227867612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6c4011d50 con 0x556057e071a0 2024-04-17T07:32:12.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.962+0000 7fc6bea00640 1 -- 172.21.15.86:0/1227867612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c401a380 con 0x556057e071a0 2024-04-17T07:32:12.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.963+0000 7fc6d7169740 1 -- 172.21.15.86:0/1227867612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 msgr2=0x556057e075c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.963+0000 7fc6d7169740 1 --2- 172.21.15.86:0/1227867612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e075c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009fd0 tx=0x7fc6c4010860 comp rx=0 tx=0).stop 2024-04-17T07:32:12.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.965+0000 7fc6d1a00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc6b40a3d80 0x7fc6b40a6240 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc6c002c220 tx=0x7fc6c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.965+0000 7fc6d7169740 1 -- 172.21.15.86:0/1227867612 shutdown_connections 2024-04-17T07:32:12.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.965+0000 7fc6d7169740 1 --2- 172.21.15.86:0/1227867612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 0x556057e0f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.965+0000 7fc6d7169740 1 --2- 172.21.15.86:0/1227867612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e0c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.965+0000 7fc6d7169740 1 --2- 172.21.15.86:0/1227867612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e075c0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.965+0000 7fc6d7169740 1 -- 172.21.15.86:0/1227867612 >> 172.21.15.86:0/1227867612 conn(0x556057e02710 msgr2=0x556057e02b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.969+0000 7fc6d7169740 1 -- 172.21.15.86:0/1227867612 shutdown_connections 2024-04-17T07:32:12.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.969+0000 7fc6d7169740 1 -- 172.21.15.86:0/1227867612 wait complete. 2024-04-17T07:32:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.969+0000 7fc6d7169740 1 -- start start 2024-04-17T07:32:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.970+0000 7fc6d7169740 1 Processor -- start 2024-04-17T07:32:12.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.971+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e22790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.971+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.971+0000 7fc6d2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e22790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.971+0000 7fc6d2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e22790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36314/0 (socket says 172.21.15.86:36314) 2024-04-17T07:32:12.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.971+0000 7fc6d2400640 1 -- 172.21.15.86:0/2815974019 learned_addr learned my addr 172.21.15.86:0/2815974019 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:12.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.972+0000 7fc6d7169740 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 0x556057e29d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.972+0000 7fc6d1000640 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.972+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556057e20cc0 con 0x556057e071a0 2024-04-17T07:32:12.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.972+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556057d26830 con 0x556057e0cd00 2024-04-17T07:32:12.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.972+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556057e20e40 con 0x556057e07b90 2024-04-17T07:32:12.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.972+0000 7fc6d7169740 10 client.0 _ll_register_callbacks cb 0x556057cf4fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:12.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.973+0000 7fc6d2400640 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 0x556057e29d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.973+0000 7fc6d1000640 1 -- 172.21.15.86:0/2815974019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 msgr2=0x556057e29d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.973+0000 7fc6d1000640 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 0x556057e29d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.973+0000 7fc6d1000640 1 -- 172.21.15.86:0/2815974019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 msgr2=0x556057e22790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:12.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.973+0000 7fc6d1000640 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e22790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.973+0000 7fc6d1000640 1 -- 172.21.15.86:0/2815974019 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556057d2a920 con 0x556057e07b90 2024-04-17T07:32:12.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.973+0000 7fc6d2400640 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 0x556057e29d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:32:12.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.973+0000 7fc6d1000640 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e22cd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc1a80b0 tx=0x7fc6b8011b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=104043579509901d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:12.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.973+0000 7fc6b3400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:32:12.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.973+0000 7fc6b3400640 1 -- 172.21.15.86:0/2815974019 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b801f060 con 0x556057e07b90 2024-04-17T07:32:12.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.974+0000 7fc6b3400640 1 -- 172.21.15.86:0/2815974019 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc6b800fcc0 con 0x556057e07b90 2024-04-17T07:32:12.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.974+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6c4011d50 con 0x556057e07b90 2024-04-17T07:32:12.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.975+0000 7fc6b3400640 1 -- 172.21.15.86:0/2815974019 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6b8011e70 con 0x556057e07b90 2024-04-17T07:32:12.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.976+0000 7fc6d7169740 10 client.28513 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:12.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.976+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556057cf87f0 con 0x556057e07b90 2024-04-17T07:32:12.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.976+0000 7fc6b3400640 1 -- 172.21.15.86:0/2815974019 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6c4011d50 con 0x556057e07b90 2024-04-17T07:32:12.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.977+0000 7fc6b3400640 1 -- 172.21.15.86:0/2815974019 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc6b801a460 con 0x556057e07b90 2024-04-17T07:32:12.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.977+0000 7fc6d7169740 10 client.28513 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:12.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.977+0000 7fc6d7169740 10 client.28513 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:32:12.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.977+0000 7fc6d7169740 20 client.28513 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:12.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.977+0000 7fc6d7169740 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556057e2e180 0x556057e305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:12.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.977+0000 7fc6d7169740 4 client.28513 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:12.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.977+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556057d7aa40 con 0x556057e2e180 2024-04-17T07:32:12.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.978+0000 7fc6d2400640 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556057e2e180 0x556057e305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:12.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.981+0000 7fc6d2400640 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556057e2e180 0x556057e305e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23e98045de234cb9 server_cookie=32f0f20f85849e88 in_seq=0 out_seq=0 2024-04-17T07:32:12.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.981+0000 7fc6b3400640 10 client.28513 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:32:12.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.983+0000 7fc6b3400640 1 -- 172.21.15.86:0/2815974019 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556057d7aa40 con 0x556057e2e180 2024-04-17T07:32:12.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.983+0000 7fc6b3400640 10 client.28513 handle_command_reply: tid=0 2024-04-17T07:32:12.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.983+0000 7fc6d7169740 1 client.28513 shutdown 2024-04-17T07:32:12.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.983+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 msgr2=0x556057e22cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:12.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.983+0000 7fc6d7169740 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e22cd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc1a80b0 tx=0x7fc6b8011b60 comp rx=0 tx=0).stop 2024-04-17T07:32:12.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.984+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 shutdown_connections 2024-04-17T07:32:12.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.984+0000 7fc6d7169740 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556057e2e180 0x556057e305e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.984+0000 7fc6d7169740 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e0cd00 0x556057e29d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.984+0000 7fc6d7169740 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e22cd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.984+0000 7fc6d7169740 1 --2- 172.21.15.86:0/2815974019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e22790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:12.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.984+0000 7fc6b3400640 0 client.28513 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:32:12.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.985+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 >> 172.21.15.86:0/2815974019 conn(0x556057e02710 msgr2=0x556057e01930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:12.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.985+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 shutdown_connections 2024-04-17T07:32:12.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.985+0000 7fc6d7169740 1 -- 172.21.15.86:0/2815974019 wait complete. 2024-04-17T07:32:12.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:12.986+0000 7fc6d7169740 20 client.28513 trim_cache size 0 max 16384 2024-04-17T07:32:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.029+0000 7fc6d7169740 1 Processor -- start 2024-04-17T07:32:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.030+0000 7fc6d7169740 1 -- start start 2024-04-17T07:32:13.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.031+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e22790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.031+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e07b90 0x556057e22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.031+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e0cd00 0x556057e29cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.031+0000 7fc6d7169740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1b0c70 con 0x556057e071a0 2024-04-17T07:32:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.031+0000 7fc6d7169740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1b27b0 con 0x556057e07b90 2024-04-17T07:32:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.031+0000 7fc6d7169740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1b1340 con 0x556057e0cd00 2024-04-17T07:32:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.032+0000 7fc6d1000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e07b90 0x556057e22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.032+0000 7fc6d2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e22790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.032+0000 7fc6d2400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e0cd00 0x556057e29cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.032+0000 7fc6d1000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e07b90 0x556057e22cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38536/0 (socket says 172.21.15.86:38536) 2024-04-17T07:32:13.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.032+0000 7fc6d1000640 1 -- 172.21.15.86:0/3404315186 learned_addr learned my addr 172.21.15.86:0/3404315186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.033+0000 7fc6d1000640 1 -- 172.21.15.86:0/3404315186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e0cd00 msgr2=0x556057e29cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.034+0000 7fc6d1000640 1 --2- 172.21.15.86:0/3404315186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e0cd00 0x556057e29cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.034+0000 7fc6d1000640 1 -- 172.21.15.86:0/3404315186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 msgr2=0x556057e22790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.034+0000 7fc6d1000640 1 --2- 172.21.15.86:0/3404315186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e22790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.034+0000 7fc6d1000640 1 -- 172.21.15.86:0/3404315186 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556057a7f3f0 con 0x556057e07b90 2024-04-17T07:32:13.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.034+0000 7fc6d2400640 1 --2- 172.21.15.86:0/3404315186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e0cd00 0x556057e29cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:32:13.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.035+0000 7fc6d1000640 1 --2- 172.21.15.86:0/3404315186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e07b90 0x556057e22cd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8000c00 tx=0x7fc6b800f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ff232f26c169d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.038+0000 7fc6be000640 1 -- 172.21.15.86:0/3404315186 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8032070 con 0x556057e07b90 2024-04-17T07:32:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.039+0000 7fc6be000640 1 -- 172.21.15.86:0/3404315186 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b801a640 con 0x556057e07b90 2024-04-17T07:32:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.039+0000 7fc6be000640 1 -- 172.21.15.86:0/3404315186 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8029080 con 0x556057e07b90 2024-04-17T07:32:13.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.042+0000 7fc6d7169740 1 -- 172.21.15.86:0/3404315186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e07b90 msgr2=0x556057e22cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.042+0000 7fc6d7169740 1 --2- 172.21.15.86:0/3404315186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e07b90 0x556057e22cd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8000c00 tx=0x7fc6b800f7b0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.042+0000 7fc6d7169740 1 -- 172.21.15.86:0/3404315186 shutdown_connections 2024-04-17T07:32:13.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.043+0000 7fc6d7169740 1 --2- 172.21.15.86:0/3404315186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e0cd00 0x556057e29cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.044+0000 7fc6d7169740 1 --2- 172.21.15.86:0/3404315186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e07b90 0x556057e22cd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.044+0000 7fc6d7169740 1 --2- 172.21.15.86:0/3404315186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e071a0 0x556057e22790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.044+0000 7fc6d7169740 1 -- 172.21.15.86:0/3404315186 >> 172.21.15.86:0/3404315186 conn(0x556057e02710 msgr2=0x556057e211e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.044+0000 7fc6d7169740 1 -- 172.21.15.86:0/3404315186 shutdown_connections 2024-04-17T07:32:13.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.045+0000 7fc6d7169740 1 -- 172.21.15.86:0/3404315186 wait complete. 2024-04-17T07:32:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.046+0000 7fc6d7169740 1 -- start start 2024-04-17T07:32:13.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.046+0000 7fc6d7169740 1 Processor -- start 2024-04-17T07:32:13.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.051+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e071a0 0x556057e27040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.051+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.052+0000 7fc6d7169740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e0cd00 0x556057e28590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.052+0000 7fc6d7169740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1b20e0 con 0x556057e0cd00 2024-04-17T07:32:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.052+0000 7fc6d7169740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1b1a10 con 0x556057e071a0 2024-04-17T07:32:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.052+0000 7fc6d7169740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc11ea60 con 0x556057e07b90 2024-04-17T07:32:13.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.053+0000 7fc6d7169740 10 client.0 _ll_register_callbacks cb 0x7fc6b800af00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.053+0000 7fc6d1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.053+0000 7fc6d1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e27580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41978/0 (socket says 172.21.15.86:41978) 2024-04-17T07:32:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.053+0000 7fc6d1000640 1 -- 172.21.15.86:0/1367720830 learned_addr learned my addr 172.21.15.86:0/1367720830 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.054+0000 7fc6d2400640 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e071a0 0x556057e27040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.054+0000 7fc6d1000640 1 -- 172.21.15.86:0/1367720830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e071a0 msgr2=0x556057e27040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.054+0000 7fc6d1000640 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e071a0 0x556057e27040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.054+0000 7fc6d1000640 1 -- 172.21.15.86:0/1367720830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e0cd00 msgr2=0x556057e28590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.054+0000 7fc6d1000640 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e0cd00 0x556057e28590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.054+0000 7fc6d1000640 1 -- 172.21.15.86:0/1367720830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556057b497d0 con 0x556057e07b90 2024-04-17T07:32:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.055+0000 7fc6d1000640 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e27580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc6b801aa70 tx=0x7fc6b800fd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701dd79c2df6e257 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.055+0000 7fc6b3400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:32:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.056+0000 7fc6b3400640 1 -- 172.21.15.86:0/1367720830 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8032070 con 0x556057e07b90 2024-04-17T07:32:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.056+0000 7fc6b3400640 1 -- 172.21.15.86:0/1367720830 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc6b8005490 con 0x556057e07b90 2024-04-17T07:32:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.056+0000 7fc6d7169740 1 -- 172.21.15.86:0/1367720830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6b801a640 con 0x556057e07b90 2024-04-17T07:32:13.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.057+0000 7fc6b3400640 1 -- 172.21.15.86:0/1367720830 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6b803e050 con 0x556057e07b90 2024-04-17T07:32:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.057+0000 7fc6d7169740 10 client.28516 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.057+0000 7fc6d7169740 1 -- 172.21.15.86:0/1367720830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556057e29c20 con 0x556057e07b90 2024-04-17T07:32:13.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.058+0000 7fc6b3400640 1 -- 172.21.15.86:0/1367720830 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6b801a640 con 0x556057e07b90 2024-04-17T07:32:13.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.059+0000 7fc6b3400640 1 -- 172.21.15.86:0/1367720830 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc6b80384c0 con 0x556057e07b90 2024-04-17T07:32:13.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.059+0000 7fc6d7169740 10 client.28516 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:13.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.059+0000 7fc6d7169740 10 client.28516 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:32:13.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.059+0000 7fc6d7169740 20 client.28516 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:13.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.059+0000 7fc6d7169740 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556057e2e180 0x556057e96a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.059+0000 7fc6d7169740 4 client.28516 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:32:13.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.059+0000 7fc6d7169740 1 -- 172.21.15.86:0/1367720830 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556057d748b0 con 0x556057e2e180 2024-04-17T07:32:13.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.061+0000 7fc6d2400640 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556057e2e180 0x556057e96a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.062+0000 7fc6d2400640 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556057e2e180 0x556057e96a20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90a54246e0b1dd70 server_cookie=adb101b9c90f2b79 in_seq=0 out_seq=0 2024-04-17T07:32:13.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.062+0000 7fc6b3400640 10 client.28516 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:32:13.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.062+0000 7fc6b3400640 1 -- 172.21.15.86:0/1367720830 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556057d748b0 con 0x556057e2e180 2024-04-17T07:32:13.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.062+0000 7fc6b3400640 10 client.28516 handle_command_reply: tid=0 2024-04-17T07:32:13.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.062+0000 7fc6d7169740 1 client.28516 shutdown 2024-04-17T07:32:13.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.063+0000 7fc6d7169740 1 -- 172.21.15.86:0/1367720830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 msgr2=0x556057e27580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.063+0000 7fc6d7169740 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e27580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc6b801aa70 tx=0x7fc6b800fd70 comp rx=0 tx=0).stop 2024-04-17T07:32:13.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.064+0000 7fc6d7169740 1 -- 172.21.15.86:0/1367720830 shutdown_connections 2024-04-17T07:32:13.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.064+0000 7fc6d7169740 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556057e2e180 0x556057e96a20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.064+0000 7fc6b3400640 0 client.28516 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:32:13.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.064+0000 7fc6d7169740 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556057e0cd00 0x556057e28590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.064+0000 7fc6d7169740 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556057e07b90 0x556057e27580 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.064+0000 7fc6d7169740 1 --2- 172.21.15.86:0/1367720830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556057e071a0 0x556057e27040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.064+0000 7fc6d7169740 1 -- 172.21.15.86:0/1367720830 >> 172.21.15.86:0/1367720830 conn(0x556057e02710 msgr2=0x556057e22770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.064+0000 7fc6d7169740 1 -- 172.21.15.86:0/1367720830 shutdown_connections 2024-04-17T07:32:13.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.064+0000 7fc6d7169740 1 -- 172.21.15.86:0/1367720830 wait complete. 2024-04-17T07:32:13.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.065+0000 7fc6d7169740 20 client.28516 trim_cache size 0 max 16384 2024-04-17T07:32:13.065 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:32:13.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.067+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc6b40a3d80 msgr2=0x7fc6b40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.067+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc6b40a3d80 0x7fc6b40a6240 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc6c002c220 tx=0x7fc6c0005fd0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.067+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc073240 msgr2=0x7fc6cc1a6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.067+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc073240 0x7fc6cc1a6ba0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009850 tx=0x7fc6c00042f0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.067+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 shutdown_connections 2024-04-17T07:32:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.067+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc6b40a3d80 0x7fc6b40a6240 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.067+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6cc109f00 0x7fc6cc1ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.067+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc109540 0x7fc6cc1a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.067+0000 7fc6d2e00640 1 --2- 172.21.15.86:0/594681732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6cc073240 0x7fc6cc1a6ba0 secure :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009850 tx=0x7fc6c00042f0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.067+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 >> 172.21.15.86:0/594681732 conn(0x7fc6cc06edb0 msgr2=0x7fc6cc071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.068+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 shutdown_connections 2024-04-17T07:32:13.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.068+0000 7fc6d2e00640 1 -- 172.21.15.86:0/594681732 wait complete. 2024-04-17T07:32:13.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:32:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.398+0000 7f4e37e00640 1 -- 172.21.15.86:0/1003893096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38101e90 msgr2=0x7f4e381022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.398+0000 7f4e37e00640 1 --2- 172.21.15.86:0/1003893096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38101e90 0x7f4e381022b0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009900 tx=0x7f4e2802bc30 comp rx=0 tx=0).stop 2024-04-17T07:32:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.398+0000 7f4e37e00640 1 -- 172.21.15.86:0/1003893096 shutdown_connections 2024-04-17T07:32:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.398+0000 7f4e37e00640 1 --2- 172.21.15.86:0/1003893096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e381039a0 0x7f4e3810a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.398+0000 7f4e37e00640 1 --2- 172.21.15.86:0/1003893096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e38102fe0 0x7f4e38103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.398+0000 7f4e37e00640 1 --2- 172.21.15.86:0/1003893096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38101e90 0x7f4e381022b0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.398+0000 7f4e37e00640 1 -- 172.21.15.86:0/1003893096 >> 172.21.15.86:0/1003893096 conn(0x7f4e380fd400 msgr2=0x7f4e380ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.398+0000 7f4e37e00640 1 -- 172.21.15.86:0/1003893096 shutdown_connections 2024-04-17T07:32:13.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.398+0000 7f4e37e00640 1 -- 172.21.15.86:0/1003893096 wait complete. 2024-04-17T07:32:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.400+0000 7f4e37e00640 1 Processor -- start 2024-04-17T07:32:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.400+0000 7f4e37e00640 1 -- start start 2024-04-17T07:32:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.400+0000 7f4e37e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e38101e90 0x7f4e381a6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.400+0000 7f4e37e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38102fe0 0x7f4e381a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.400+0000 7f4e37e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e381039a0 0x7f4e381ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.400+0000 7f4e37e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3806c280 con 0x7f4e38102fe0 2024-04-17T07:32:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.400+0000 7f4e37e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3806c100 con 0x7f4e381039a0 2024-04-17T07:32:13.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.400+0000 7f4e37e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3806c400 con 0x7f4e38101e90 2024-04-17T07:32:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.401+0000 7f4e2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38102fe0 0x7f4e381a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.401+0000 7f4e2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38102fe0 0x7f4e381a73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36350/0 (socket says 172.21.15.86:36350) 2024-04-17T07:32:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.401+0000 7f4e2fe00640 1 -- 172.21.15.86:0/1591798161 learned_addr learned my addr 172.21.15.86:0/1591798161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.401+0000 7f4e37400640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e381039a0 0x7f4e381ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.401+0000 7f4e36a00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e38101e90 0x7f4e381a6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.402+0000 7f4e2fe00640 1 -- 172.21.15.86:0/1591798161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e38101e90 msgr2=0x7f4e381a6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.402+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e38101e90 0x7f4e381a6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.402+0000 7f4e2fe00640 1 -- 172.21.15.86:0/1591798161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e381039a0 msgr2=0x7f4e381ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.402+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e381039a0 0x7f4e381ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.402+0000 7f4e2fe00640 1 -- 172.21.15.86:0/1591798161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e381aead0 con 0x7f4e38102fe0 2024-04-17T07:32:13.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.403+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38102fe0 0x7f4e381a73f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000c970 tx=0x7f4e2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.404+0000 7f4e2f400640 1 -- 172.21.15.86:0/1591798161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20007c10 con 0x7f4e38102fe0 2024-04-17T07:32:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.404+0000 7f4e37e00640 1 -- 172.21.15.86:0/1591798161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e381aedc0 con 0x7f4e38102fe0 2024-04-17T07:32:13.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.404+0000 7f4e2f400640 1 -- 172.21.15.86:0/1591798161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e20007db0 con 0x7f4e38102fe0 2024-04-17T07:32:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.404+0000 7f4e2f400640 1 -- 172.21.15.86:0/1591798161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000f7d0 con 0x7f4e38102fe0 2024-04-17T07:32:13.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.404+0000 7f4e37e00640 1 -- 172.21.15.86:0/1591798161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e381af310 con 0x7f4e38102fe0 2024-04-17T07:32:13.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.409+0000 7f4e407c4740 1 Processor -- start 2024-04-17T07:32:13.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.409+0000 7f4e407c4740 1 -- start start 2024-04-17T07:32:13.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.410+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248f160 0x56484248f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.410+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 0x564842494650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.410+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564842494cc0 0x564842497150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.410+0000 7f4e407c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648423f5020 con 0x56484248f160 2024-04-17T07:32:13.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.410+0000 7f4e407c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56484239b460 con 0x56484248fb50 2024-04-17T07:32:13.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.410+0000 7f4e407c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5648423f0fe0 con 0x564842494cc0 2024-04-17T07:32:13.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.410+0000 7f4e2f400640 1 -- 172.21.15.86:0/1591798161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4e2000fa70 con 0x7f4e38102fe0 2024-04-17T07:32:13.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.411+0000 7f4e37400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248f160 0x56484248f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.411+0000 7f4e37400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248f160 0x56484248f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36356/0 (socket says 172.21.15.86:36356) 2024-04-17T07:32:13.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.411+0000 7f4e37400640 1 -- 172.21.15.86:0/1347055756 learned_addr learned my addr 172.21.15.86:0/1347055756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.411+0000 7f4e36a00640 1 --2- 172.21.15.86:0/1347055756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 0x564842494650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.411+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/1347055756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564842494cc0 0x564842497150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.412+0000 7f4e37400640 1 -- 172.21.15.86:0/1347055756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564842494cc0 msgr2=0x564842497150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.412+0000 7f4e37400640 1 --2- 172.21.15.86:0/1347055756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564842494cc0 0x564842497150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.412+0000 7f4e37400640 1 -- 172.21.15.86:0/1347055756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 msgr2=0x564842494650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.412+0000 7f4e37400640 1 --2- 172.21.15.86:0/1347055756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 0x564842494650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.412+0000 7f4e37400640 1 -- 172.21.15.86:0/1347055756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648423a3040 con 0x56484248f160 2024-04-17T07:32:13.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.412+0000 7f4e37400640 1 --2- 172.21.15.86:0/1347055756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248f160 0x56484248f580 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000a430 tx=0x7f4e30009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64757e2929c0d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.412+0000 7f4e2f400640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e1c0a3c00 0x7f4e1c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.413+0000 7f4e27e00640 1 -- 172.21.15.86:0/1347055756 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30013790 con 0x56484248f160 2024-04-17T07:32:13.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.413+0000 7f4e2f400640 1 -- 172.21.15.86:0/1591798161 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4e200d0a20 con 0x7f4e38102fe0 2024-04-17T07:32:13.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.413+0000 7f4e27e00640 1 -- 172.21.15.86:0/1347055756 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e30019070 con 0x56484248f160 2024-04-17T07:32:13.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.413+0000 7f4e27e00640 1 -- 172.21.15.86:0/1347055756 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30013790 con 0x56484248f160 2024-04-17T07:32:13.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.414+0000 7f4e36a00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e1c0a3c00 0x7f4e1c0a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.414+0000 7f4e407c4740 1 -- 172.21.15.86:0/1347055756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248f160 msgr2=0x56484248f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.414+0000 7f4e407c4740 1 --2- 172.21.15.86:0/1347055756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248f160 0x56484248f580 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000a430 tx=0x7f4e30009650 comp rx=0 tx=0).stop 2024-04-17T07:32:13.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.414+0000 7f4e407c4740 1 -- 172.21.15.86:0/1347055756 shutdown_connections 2024-04-17T07:32:13.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.414+0000 7f4e407c4740 1 --2- 172.21.15.86:0/1347055756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564842494cc0 0x564842497150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.414+0000 7f4e407c4740 1 --2- 172.21.15.86:0/1347055756 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 0x564842494650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.414+0000 7f4e407c4740 1 --2- 172.21.15.86:0/1347055756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248f160 0x56484248f580 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.414+0000 7f4e407c4740 1 -- 172.21.15.86:0/1347055756 >> 172.21.15.86:0/1347055756 conn(0x56484248a6d0 msgr2=0x56484248aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.415+0000 7f4e36a00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e1c0a3c00 0x7f4e1c0a60c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4e28006010 tx=0x7f4e28000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.415+0000 7f4e407c4740 1 -- 172.21.15.86:0/1347055756 shutdown_connections 2024-04-17T07:32:13.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.415+0000 7f4e407c4740 1 -- 172.21.15.86:0/1347055756 wait complete. 2024-04-17T07:32:13.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.416+0000 7f4e407c4740 1 -- start start 2024-04-17T07:32:13.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.416+0000 7f4e407c4740 1 Processor -- start 2024-04-17T07:32:13.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.417+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248f160 0x564842499640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.417+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 0x564842499b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.417+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 0x5648424a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.417+0000 7f4e407c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564842497cc0 con 0x564842494cc0 2024-04-17T07:32:13.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.417+0000 7f4e407c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5648423ae830 con 0x56484248fb50 2024-04-17T07:32:13.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.417+0000 7f4e407c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564842497e40 con 0x56484248f160 2024-04-17T07:32:13.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.417+0000 7f4e407c4740 10 client.0 _ll_register_callbacks cb 0x56484237cfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:13.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.418+0000 7f4e37400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 0x5648424a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.418+0000 7f4e2fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 0x564842499b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.418+0000 7f4e2fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 0x564842499b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38596/0 (socket says 172.21.15.86:38596) 2024-04-17T07:32:13.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.418+0000 7f4e2fe00640 1 -- 172.21.15.86:0/2706340707 learned_addr learned my addr 172.21.15.86:0/2706340707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.418+0000 7f4e36a00640 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248f160 0x564842499640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.420+0000 7f4e36a00640 1 -- 172.21.15.86:0/2706340707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 msgr2=0x564842499b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.420+0000 7f4e36a00640 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 0x564842499b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.420+0000 7f4e36a00640 1 -- 172.21.15.86:0/2706340707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 msgr2=0x5648424a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.420+0000 7f4e36a00640 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 0x5648424a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.420+0000 7f4e36a00640 1 -- 172.21.15.86:0/2706340707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648423b2920 con 0x56484248f160 2024-04-17T07:32:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.420+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 0x564842499b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:13.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.420+0000 7f4e37400640 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 0x5648424a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.420+0000 7f4e36a00640 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248f160 0x564842499640 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009850 tx=0x7f4e2802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93330d5eac9b9b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.420+0000 7f4e24c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:32:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.421+0000 7f4e24c00640 1 -- 172.21.15.86:0/2706340707 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2804f030 con 0x56484248f160 2024-04-17T07:32:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.421+0000 7f4e24c00640 1 -- 172.21.15.86:0/2706340707 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4e2803a040 con 0x56484248f160 2024-04-17T07:32:13.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.421+0000 7f4e407c4740 1 -- 172.21.15.86:0/2706340707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e30019070 con 0x56484248f160 2024-04-17T07:32:13.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.422+0000 7f4e24c00640 1 -- 172.21.15.86:0/2706340707 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e2805c050 con 0x56484248f160 2024-04-17T07:32:13.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.422+0000 7f4e407c4740 10 client.28525 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:13.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.422+0000 7f4e407c4740 1 -- 172.21.15.86:0/2706340707 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5648423807f0 con 0x56484248f160 2024-04-17T07:32:13.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.423+0000 7f4e24c00640 1 -- 172.21.15.86:0/2706340707 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e30019070 con 0x56484248f160 2024-04-17T07:32:13.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.423+0000 7f4e24c00640 1 -- 172.21.15.86:0/2706340707 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4e280035f0 con 0x56484248f160 2024-04-17T07:32:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.424+0000 7f4e407c4740 10 client.28525 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:13.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.424+0000 7f4e407c4740 10 client.28525 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:32:13.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.424+0000 7f4e407c4740 20 client.28525 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:13.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.424+0000 7f4e407c4740 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5648424a5110 0x5648424a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.424+0000 7f4e407c4740 4 client.28525 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:13.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.424+0000 7f4e407c4740 1 -- 172.21.15.86:0/2706340707 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564842402a40 con 0x5648424a5110 2024-04-17T07:32:13.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.427+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5648424a5110 0x5648424a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.432+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5648424a5110 0x5648424a7560 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=728eb373231575fe server_cookie=238dd7ba5c653188 in_seq=0 out_seq=0 2024-04-17T07:32:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.432+0000 7f4e24c00640 10 client.28525 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:32:13.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.435+0000 7f4e24c00640 1 -- 172.21.15.86:0/2706340707 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564842402a40 con 0x5648424a5110 2024-04-17T07:32:13.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.435+0000 7f4e24c00640 10 client.28525 handle_command_reply: tid=0 2024-04-17T07:32:13.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.436+0000 7f4e407c4740 1 client.28525 shutdown 2024-04-17T07:32:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.437+0000 7f4e407c4740 1 -- 172.21.15.86:0/2706340707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248f160 msgr2=0x564842499640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.437+0000 7f4e407c4740 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248f160 0x564842499640 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009850 tx=0x7f4e2802bc30 comp rx=0 tx=0).stop 2024-04-17T07:32:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.437+0000 7f4e407c4740 1 -- 172.21.15.86:0/2706340707 shutdown_connections 2024-04-17T07:32:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.437+0000 7f4e407c4740 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 0x5648424a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.437+0000 7f4e407c4740 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5648424a5110 0x5648424a7560 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.437+0000 7f4e407c4740 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248fb50 0x564842499b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.437+0000 7f4e407c4740 1 --2- 172.21.15.86:0/2706340707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248f160 0x564842499640 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.437+0000 7f4e407c4740 1 -- 172.21.15.86:0/2706340707 >> 172.21.15.86:0/2706340707 conn(0x56484248a6d0 msgr2=0x564842489930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.437+0000 7f4e24c00640 0 client.28525 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:32:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.437+0000 7f4e407c4740 1 -- 172.21.15.86:0/2706340707 shutdown_connections 2024-04-17T07:32:13.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.438+0000 7f4e407c4740 1 -- 172.21.15.86:0/2706340707 wait complete. 2024-04-17T07:32:13.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.438+0000 7f4e407c4740 20 client.28525 trim_cache size 0 max 16384 2024-04-17T07:32:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.473+0000 7f4e407c4740 1 Processor -- start 2024-04-17T07:32:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.473+0000 7f4e407c4740 1 -- start start 2024-04-17T07:32:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.474+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 0x5648424857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.474+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248fb50 0x5648424a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.474+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564842494cc0 0x564842499960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.474+0000 7f4e407c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e381b9820 con 0x56484248fb50 2024-04-17T07:32:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.474+0000 7f4e407c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4e381bb360 con 0x56484248f160 2024-04-17T07:32:13.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.474+0000 7f4e407c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e381b9ef0 con 0x564842494cc0 2024-04-17T07:32:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.475+0000 7f4e36a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 0x5648424857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.475+0000 7f4e36a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 0x5648424857c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38602/0 (socket says 172.21.15.86:38602) 2024-04-17T07:32:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.475+0000 7f4e36a00640 1 -- 172.21.15.86:0/1321675361 learned_addr learned my addr 172.21.15.86:0/1321675361 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.475+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/1321675361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248fb50 0x5648424a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.475+0000 7f4e37400640 1 --2- 172.21.15.86:0/1321675361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564842494cc0 0x564842499960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.475+0000 7f4e36a00640 1 -- 172.21.15.86:0/1321675361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564842494cc0 msgr2=0x564842499960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.475+0000 7f4e36a00640 1 --2- 172.21.15.86:0/1321675361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564842494cc0 0x564842499960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.475+0000 7f4e36a00640 1 -- 172.21.15.86:0/1321675361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248fb50 msgr2=0x5648424a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.475+0000 7f4e36a00640 1 --2- 172.21.15.86:0/1321675361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248fb50 0x5648424a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.475+0000 7f4e36a00640 1 -- 172.21.15.86:0/1321675361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648421073f0 con 0x56484248f160 2024-04-17T07:32:13.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.476+0000 7f4e36a00640 1 --2- 172.21.15.86:0/1321675361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 0x5648424857c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4e280096c0 tx=0x7f4e2802c1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1cef5762d710946 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.476+0000 7f4e27400640 1 -- 172.21.15.86:0/1321675361 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28059070 con 0x56484248f160 2024-04-17T07:32:13.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.476+0000 7f4e27400640 1 -- 172.21.15.86:0/1321675361 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e28005640 con 0x56484248f160 2024-04-17T07:32:13.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.476+0000 7f4e27400640 1 -- 172.21.15.86:0/1321675361 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2804f070 con 0x56484248f160 2024-04-17T07:32:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.477+0000 7f4e407c4740 1 -- 172.21.15.86:0/1321675361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 msgr2=0x5648424857c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.477+0000 7f4e407c4740 1 --2- 172.21.15.86:0/1321675361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 0x5648424857c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4e280096c0 tx=0x7f4e2802c1b0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.478+0000 7f4e407c4740 1 -- 172.21.15.86:0/1321675361 shutdown_connections 2024-04-17T07:32:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.478+0000 7f4e407c4740 1 --2- 172.21.15.86:0/1321675361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564842494cc0 0x564842499960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.478+0000 7f4e407c4740 1 --2- 172.21.15.86:0/1321675361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56484248fb50 0x5648424a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.478+0000 7f4e407c4740 1 --2- 172.21.15.86:0/1321675361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 0x5648424857c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.478+0000 7f4e407c4740 1 -- 172.21.15.86:0/1321675361 >> 172.21.15.86:0/1321675361 conn(0x56484248a6d0 msgr2=0x564842499250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.478+0000 7f4e407c4740 1 -- 172.21.15.86:0/1321675361 shutdown_connections 2024-04-17T07:32:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.478+0000 7f4e407c4740 1 -- 172.21.15.86:0/1321675361 wait complete. 2024-04-17T07:32:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.478+0000 7f4e407c4740 1 -- start start 2024-04-17T07:32:13.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.479+0000 7f4e407c4740 1 Processor -- start 2024-04-17T07:32:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.479+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 0x56484249e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.479+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248fb50 0x56484249e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.479+0000 7f4e407c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 0x56484249f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.479+0000 7f4e407c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e381bac90 con 0x564842494cc0 2024-04-17T07:32:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.479+0000 7f4e407c4740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4e381ba5c0 con 0x56484248f160 2024-04-17T07:32:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.479+0000 7f4e407c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e381b77f0 con 0x56484248fb50 2024-04-17T07:32:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.479+0000 7f4e407c4740 10 client.0 _ll_register_callbacks cb 0x7f4e28004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:13.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.480+0000 7f4e2fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248fb50 0x56484249e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.480+0000 7f4e37400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 0x56484249f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.480+0000 7f4e2fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248fb50 0x56484249e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42030/0 (socket says 172.21.15.86:42030) 2024-04-17T07:32:13.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.480+0000 7f4e2fe00640 1 -- 172.21.15.86:0/3166316917 learned_addr learned my addr 172.21.15.86:0/3166316917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.481+0000 7f4e36a00640 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 0x56484249e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.481+0000 7f4e2fe00640 1 -- 172.21.15.86:0/3166316917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 msgr2=0x56484249e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.481+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 0x56484249e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.481+0000 7f4e2fe00640 1 -- 172.21.15.86:0/3166316917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 msgr2=0x56484249f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.481+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 0x56484249f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.481+0000 7f4e2fe00640 1 -- 172.21.15.86:0/3166316917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648421d17d0 con 0x56484248fb50 2024-04-17T07:32:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.482+0000 7f4e2fe00640 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248fb50 0x56484249e7f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4e200ce510 tx=0x7f4e200a3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4d02b890a771b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.482+0000 7f4e24c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:32:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.483+0000 7f4e24c00640 1 -- 172.21.15.86:0/3166316917 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2009c550 con 0x56484248fb50 2024-04-17T07:32:13.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.483+0000 7f4e24c00640 1 -- 172.21.15.86:0/3166316917 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4e2000fd60 con 0x56484248fb50 2024-04-17T07:32:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.483+0000 7f4e407c4740 1 -- 172.21.15.86:0/3166316917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e28005640 con 0x56484248fb50 2024-04-17T07:32:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.484+0000 7f4e24c00640 1 -- 172.21.15.86:0/3166316917 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e2009c370 con 0x56484248fb50 2024-04-17T07:32:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.484+0000 7f4e407c4740 10 client.28528 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:13.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.484+0000 7f4e407c4740 1 -- 172.21.15.86:0/3166316917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56484250b670 con 0x56484248fb50 2024-04-17T07:32:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.485+0000 7f4e24c00640 1 -- 172.21.15.86:0/3166316917 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e28005640 con 0x56484248fb50 2024-04-17T07:32:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.485+0000 7f4e24c00640 1 -- 172.21.15.86:0/3166316917 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4e2009cc70 con 0x56484248fb50 2024-04-17T07:32:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.485+0000 7f4e407c4740 10 client.28528 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:13.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.485+0000 7f4e407c4740 10 client.28528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:32:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.485+0000 7f4e407c4740 20 client.28528 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.485+0000 7f4e407c4740 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56484250be10 0x56484250e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.485+0000 7f4e407c4740 4 client.28528 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:32:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.485+0000 7f4e407c4740 1 -- 172.21.15.86:0/3166316917 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648423fc8b0 con 0x56484250be10 2024-04-17T07:32:13.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.486+0000 7f4e36a00640 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56484250be10 0x56484250e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.489+0000 7f4e36a00640 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56484250be10 0x56484250e230 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9d574e4d28772f2 server_cookie=e1b7e815daefd061 in_seq=0 out_seq=0 2024-04-17T07:32:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.489+0000 7f4e24c00640 10 client.28528 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:32:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.490+0000 7f4e24c00640 1 -- 172.21.15.86:0/3166316917 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648423fc8b0 con 0x56484250be10 2024-04-17T07:32:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.490+0000 7f4e24c00640 10 client.28528 handle_command_reply: tid=0 2024-04-17T07:32:13.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.490+0000 7f4e407c4740 1 client.28528 shutdown 2024-04-17T07:32:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.490+0000 7f4e407c4740 1 -- 172.21.15.86:0/3166316917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248fb50 msgr2=0x56484249e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.490+0000 7f4e407c4740 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248fb50 0x56484249e7f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4e200ce510 tx=0x7f4e200a3040 comp rx=0 tx=0).stop 2024-04-17T07:32:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.490+0000 7f4e407c4740 1 -- 172.21.15.86:0/3166316917 shutdown_connections 2024-04-17T07:32:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.490+0000 7f4e407c4740 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564842494cc0 0x56484249f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.491+0000 7f4e407c4740 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56484248fb50 0x56484249e7f0 secure :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4e200ce510 tx=0x7f4e200a3040 comp rx=0 tx=0).stop 2024-04-17T07:32:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.491+0000 7f4e407c4740 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x56484250be10 0x56484250e230 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.491+0000 7f4e407c4740 1 --2- 172.21.15.86:0/3166316917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56484248f160 0x56484249e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.491+0000 7f4e407c4740 1 -- 172.21.15.86:0/3166316917 >> 172.21.15.86:0/3166316917 conn(0x56484248a6d0 msgr2=0x56484248d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.491+0000 7f4e24c00640 0 client.28528 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:32:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.491+0000 7f4e407c4740 1 -- 172.21.15.86:0/3166316917 shutdown_connections 2024-04-17T07:32:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.491+0000 7f4e407c4740 1 -- 172.21.15.86:0/3166316917 wait complete. 2024-04-17T07:32:13.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.492+0000 7f4e407c4740 20 client.28528 trim_cache size 0 max 16384 2024-04-17T07:32:13.492 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:32:13.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 -- 172.21.15.86:0/1591798161 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e1c0a3c00 msgr2=0x7f4e1c0a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e1c0a3c00 0x7f4e1c0a60c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4e28006010 tx=0x7f4e28000bc0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 -- 172.21.15.86:0/1591798161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38102fe0 msgr2=0x7f4e381a73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38102fe0 0x7f4e381a73f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000c970 tx=0x7f4e2000ce30 comp rx=0 tx=0).stop 2024-04-17T07:32:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 -- 172.21.15.86:0/1591798161 shutdown_connections 2024-04-17T07:32:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4e1c0a3c00 0x7f4e1c0a60c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e381039a0 0x7f4e381ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38102fe0 0x7f4e381a73f0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 --2- 172.21.15.86:0/1591798161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e38101e90 0x7f4e381a6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 -- 172.21.15.86:0/1591798161 >> 172.21.15.86:0/1591798161 conn(0x7f4e380fd400 msgr2=0x7f4e38075460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 -- 172.21.15.86:0/1591798161 shutdown_connections 2024-04-17T07:32:13.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.493+0000 7f4e37e00640 1 -- 172.21.15.86:0/1591798161 wait complete. 2024-04-17T07:32:13.519 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:32:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.728+0000 7f98dae00640 1 -- 172.21.15.86:0/3953790494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d406d4e0 msgr2=0x7f98d406b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.728+0000 7f98dae00640 1 --2- 172.21.15.86:0/3953790494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d406d4e0 0x7f98d406b930 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009930 tx=0x7f98c402bc30 comp rx=0 tx=0).stop 2024-04-17T07:32:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.729+0000 7f98dae00640 1 -- 172.21.15.86:0/3953790494 shutdown_connections 2024-04-17T07:32:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.729+0000 7f98dae00640 1 --2- 172.21.15.86:0/3953790494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d4116d70 0x7f98d4119160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.729+0000 7f98dae00640 1 --2- 172.21.15.86:0/3953790494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d406bf00 0x7f98d406c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.729+0000 7f98dae00640 1 --2- 172.21.15.86:0/3953790494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d406d4e0 0x7f98d406b930 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.729+0000 7f98dae00640 1 -- 172.21.15.86:0/3953790494 >> 172.21.15.86:0/3953790494 conn(0x7f98d4070f50 msgr2=0x7f98d4073370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.729+0000 7f98dae00640 1 -- 172.21.15.86:0/3953790494 shutdown_connections 2024-04-17T07:32:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.729+0000 7f98dae00640 1 -- 172.21.15.86:0/3953790494 wait complete. 2024-04-17T07:32:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.730+0000 7f98dae00640 1 Processor -- start 2024-04-17T07:32:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.731+0000 7f98dae00640 1 -- start start 2024-04-17T07:32:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.731+0000 7f98dae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d406bf00 0x7f98d4113560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.731+0000 7f98dae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d406d4e0 0x7f98d4113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.732+0000 7f98dae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d4116d70 0x7f98d41b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.732+0000 7f98dae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98d411c910 con 0x7f98d406bf00 2024-04-17T07:32:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.732+0000 7f98dae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98d411c790 con 0x7f98d4116d70 2024-04-17T07:32:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.732+0000 7f98dae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98d411ca90 con 0x7f98d406d4e0 2024-04-17T07:32:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.732+0000 7f98d8e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d4116d70 0x7f98d41b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.732+0000 7f98d8e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d4116d70 0x7f98d41b7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38630/0 (socket says 172.21.15.86:38630) 2024-04-17T07:32:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.732+0000 7f98d8e00640 1 -- 172.21.15.86:0/3814896814 learned_addr learned my addr 172.21.15.86:0/3814896814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.732+0000 7f98d3400640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d406d4e0 0x7f98d4113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.732+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d406bf00 0x7f98d4113560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.733+0000 7f98d8e00640 1 -- 172.21.15.86:0/3814896814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d406d4e0 msgr2=0x7f98d4113aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.733+0000 7f98d8e00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d406d4e0 0x7f98d4113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.733+0000 7f98d8e00640 1 -- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d406bf00 msgr2=0x7f98d4113560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.733+0000 7f98d8e00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d406bf00 0x7f98d4113560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.733+0000 7f98d8e00640 1 -- 172.21.15.86:0/3814896814 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d41b7560 con 0x7f98d4116d70 2024-04-17T07:32:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.734+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d406bf00 0x7f98d4113560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.734+0000 7f98d8e00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d4116d70 0x7f98d41b7020 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00cce0 tx=0x7f98cc00a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.735+0000 7f98d0c00640 1 -- 172.21.15.86:0/3814896814 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00f450 con 0x7f98d4116d70 2024-04-17T07:32:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.735+0000 7f98d0c00640 1 -- 172.21.15.86:0/3814896814 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98cc00faa0 con 0x7f98d4116d70 2024-04-17T07:32:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.735+0000 7f98d0c00640 1 -- 172.21.15.86:0/3814896814 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc018be0 con 0x7f98d4116d70 2024-04-17T07:32:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.735+0000 7f98dae00640 1 -- 172.21.15.86:0/3814896814 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d41b7730 con 0x7f98d4116d70 2024-04-17T07:32:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.735+0000 7f98dae00640 1 -- 172.21.15.86:0/3814896814 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d41b7c30 con 0x7f98d4116d70 2024-04-17T07:32:13.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.741+0000 7f98ddb7a740 1 Processor -- start 2024-04-17T07:32:13.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.743+0000 7f98d0c00640 1 -- 172.21.15.86:0/3814896814 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f98cc00fc50 con 0x7f98d4116d70 2024-04-17T07:32:13.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.743+0000 7f98ddb7a740 1 -- start start 2024-04-17T07:32:13.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.743+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d7fa0 0x5615123d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.743+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d8990 0x5615123dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.743+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123ddb00 0x5615123dff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.743+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56151233e020 con 0x5615123d8990 2024-04-17T07:32:13.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.743+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5615122e4460 con 0x5615123ddb00 2024-04-17T07:32:13.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.743+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561512339fe0 con 0x5615123d7fa0 2024-04-17T07:32:13.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.744+0000 7f98d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d8990 0x5615123dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.744+0000 7f98d3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d8990 0x5615123dd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36404/0 (socket says 172.21.15.86:36404) 2024-04-17T07:32:13.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.744+0000 7f98d3e00640 1 -- 172.21.15.86:0/3215012909 learned_addr learned my addr 172.21.15.86:0/3215012909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.744+0000 7f98d3400640 1 --2- 172.21.15.86:0/3215012909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d7fa0 0x5615123d83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.745+0000 7f98d3e00640 1 -- 172.21.15.86:0/3215012909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d7fa0 msgr2=0x5615123d83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.745+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3215012909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d7fa0 0x5615123d83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.745+0000 7f98d0c00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b80a3d30 0x7f98b80a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.745+0000 7f98d3e00640 1 -- 172.21.15.86:0/3215012909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123ddb00 msgr2=0x5615123dff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.745+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3215012909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123ddb00 0x5615123dff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.745+0000 7f98d3400640 1 --2- 172.21.15.86:0/3215012909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d7fa0 0x5615123d83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:32:13.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.745+0000 7f98d0c00640 1 -- 172.21.15.86:0/3814896814 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f98cc0d0b10 con 0x7f98d4116d70 2024-04-17T07:32:13.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.745+0000 7f98d3e00640 1 -- 172.21.15.86:0/3215012909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615122ec040 con 0x5615123d8990 2024-04-17T07:32:13.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.746+0000 7f98d8e00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b80a3d30 0x7f98b80a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.746+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3215012909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d8990 0x5615123dd490 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f98c4000c00 tx=0x7f98c402cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b989a3e810abe491 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.747+0000 7f98d8e00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b80a3d30 0x7f98b80a61f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0184b0 tx=0x7f98cc018400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.747+0000 7f98c9600640 1 -- 172.21.15.86:0/3215012909 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4045020 con 0x5615123d8990 2024-04-17T07:32:13.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.747+0000 7f98c9600640 1 -- 172.21.15.86:0/3215012909 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98c4005040 con 0x5615123d8990 2024-04-17T07:32:13.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.747+0000 7f98c9600640 1 -- 172.21.15.86:0/3215012909 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4004ae0 con 0x5615123d8990 2024-04-17T07:32:13.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.748+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3215012909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d8990 msgr2=0x5615123dd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.748+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/3215012909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d8990 0x5615123dd490 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f98c4000c00 tx=0x7f98c402cdb0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.748+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3215012909 shutdown_connections 2024-04-17T07:32:13.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.748+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/3215012909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123ddb00 0x5615123dff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.748+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/3215012909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d8990 0x5615123dd490 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.748+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/3215012909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d7fa0 0x5615123d83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.748+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3215012909 >> 172.21.15.86:0/3215012909 conn(0x5615123d3510 msgr2=0x5615123d3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.749+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3215012909 shutdown_connections 2024-04-17T07:32:13.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.749+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3215012909 wait complete. 2024-04-17T07:32:13.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.749+0000 7f98ddb7a740 1 -- start start 2024-04-17T07:32:13.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.750+0000 7f98ddb7a740 1 Processor -- start 2024-04-17T07:32:13.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.750+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123d7fa0 0x5615123e24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.750+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d8990 0x5615123e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.750+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123ddb00 0x5615123e9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.750+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5615123e0a50 con 0x5615123ddb00 2024-04-17T07:32:13.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.750+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5615122f7830 con 0x5615123d7fa0 2024-04-17T07:32:13.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.750+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615123e0bd0 con 0x5615123d8990 2024-04-17T07:32:13.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.750+0000 7f98ddb7a740 10 client.0 _ll_register_callbacks cb 0x5615122c5fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:13.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.751+0000 7f98d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123ddb00 0x5615123e9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.751+0000 7f98d3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d8990 0x5615123e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.751+0000 7f98d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123ddb00 0x5615123e9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36420/0 (socket says 172.21.15.86:36420) 2024-04-17T07:32:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.751+0000 7f98d3400640 1 -- 172.21.15.86:0/3306466907 learned_addr learned my addr 172.21.15.86:0/3306466907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.751+0000 7f98d3400640 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123d7fa0 0x5615123e24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.751+0000 7f98d3400640 1 -- 172.21.15.86:0/3306466907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d8990 msgr2=0x5615123e29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.751+0000 7f98d3400640 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d8990 0x5615123e29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.751+0000 7f98d3400640 1 -- 172.21.15.86:0/3306466907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123d7fa0 msgr2=0x5615123e24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.751+0000 7f98d3400640 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123d7fa0 0x5615123e24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.751+0000 7f98d3400640 1 -- 172.21.15.86:0/3306466907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615122fb920 con 0x5615123ddb00 2024-04-17T07:32:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.752+0000 7f98d3400640 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123ddb00 0x5615123e9a30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f98c0012e90 tx=0x7f98c0009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ce36226b52110c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.752+0000 7f98be000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.752+0000 7f98be000640 1 -- 172.21.15.86:0/3306466907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0010e70 con 0x5615123ddb00 2024-04-17T07:32:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.753+0000 7f98be000640 1 -- 172.21.15.86:0/3306466907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f98c0004030 con 0x5615123ddb00 2024-04-17T07:32:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.753+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3306466907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c4005040 con 0x5615123ddb00 2024-04-17T07:32:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.753+0000 7f98be000640 1 -- 172.21.15.86:0/3306466907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98c0027050 con 0x5615123ddb00 2024-04-17T07:32:13.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.753+0000 7f98ddb7a740 10 client.19512 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:13.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.753+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3306466907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615122c97f0 con 0x5615123ddb00 2024-04-17T07:32:13.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:13 smithi098 ceph-mon[35418]: pgmap v975: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.4 MiB/s wr, 707 op/s 2024-04-17T07:32:13.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.754+0000 7f98be000640 1 -- 172.21.15.86:0/3306466907 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98c4005040 con 0x5615123ddb00 2024-04-17T07:32:13.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.754+0000 7f98be000640 1 -- 172.21.15.86:0/3306466907 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f98c00047c0 con 0x5615123ddb00 2024-04-17T07:32:13.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.754+0000 7f98ddb7a740 10 client.19512 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:13.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.754+0000 7f98ddb7a740 10 client.19512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:13.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.754+0000 7f98ddb7a740 20 client.19512 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:13.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.754+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615123edeb0 0x5615123f0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.754+0000 7f98ddb7a740 4 client.19512 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:13.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.754+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3306466907 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56151234ba40 con 0x5615123edeb0 2024-04-17T07:32:13.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.755+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615123edeb0 0x5615123f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.756+0000 7f98d3e00640 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615123edeb0 0x5615123f0350 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f18219bb62da12d server_cookie=3eed654b2b75ae2c in_seq=0 out_seq=0 2024-04-17T07:32:13.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.756+0000 7f98be000640 10 client.19512 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:13.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.758+0000 7f98be000640 1 -- 172.21.15.86:0/3306466907 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56151234ba40 con 0x5615123edeb0 2024-04-17T07:32:13.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.758+0000 7f98be000640 10 client.19512 handle_command_reply: tid=0 2024-04-17T07:32:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.758+0000 7f98ddb7a740 1 client.19512 shutdown 2024-04-17T07:32:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.759+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3306466907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123ddb00 msgr2=0x5615123e9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.759+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123ddb00 0x5615123e9a30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f98c0012e90 tx=0x7f98c0009730 comp rx=0 tx=0).stop 2024-04-17T07:32:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.759+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3306466907 shutdown_connections 2024-04-17T07:32:13.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.759+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615123edeb0 0x5615123f0350 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.759+0000 7f98be000640 0 client.19512 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.759+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123ddb00 0x5615123e9a30 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.759+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d8990 0x5615123e29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.759+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/3306466907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123d7fa0 0x5615123e24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.759+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3306466907 >> 172.21.15.86:0/3306466907 conn(0x5615123d3510 msgr2=0x5615123d2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.760+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3306466907 shutdown_connections 2024-04-17T07:32:13.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.760+0000 7f98ddb7a740 1 -- 172.21.15.86:0/3306466907 wait complete. 2024-04-17T07:32:13.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.760+0000 7f98ddb7a740 20 client.19512 trim_cache size 0 max 16384 2024-04-17T07:32:13.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:13 smithi103 ceph-mon[35495]: pgmap v975: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.4 MiB/s wr, 707 op/s 2024-04-17T07:32:13.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.786+0000 7f98ddb7a740 1 Processor -- start 2024-04-17T07:32:13.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.786+0000 7f98ddb7a740 1 -- start start 2024-04-17T07:32:13.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.787+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123d9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.787+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d8990 0x5615123e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.787+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123ddb00 0x5615123da430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.787+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41c2030 con 0x5615123d7fa0 2024-04-17T07:32:13.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.787+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41c3b70 con 0x5615123ddb00 2024-04-17T07:32:13.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.787+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41c2700 con 0x5615123d8990 2024-04-17T07:32:13.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.787+0000 7f98d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123d9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.787+0000 7f98d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123d9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36430/0 (socket says 172.21.15.86:36430) 2024-04-17T07:32:13.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.787+0000 7f98d3400640 1 -- 172.21.15.86:0/2699108062 learned_addr learned my addr 172.21.15.86:0/2699108062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.788+0000 7f98d3e00640 1 --2- 172.21.15.86:0/2699108062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d8990 0x5615123e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.788+0000 7f98d3400640 1 --2- 172.21.15.86:0/2699108062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123ddb00 0x5615123da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.788+0000 7f98d3400640 1 -- 172.21.15.86:0/2699108062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d8990 msgr2=0x5615123e99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.788+0000 7f98d3400640 1 --2- 172.21.15.86:0/2699108062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d8990 0x5615123e99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.788+0000 7f98d3400640 1 -- 172.21.15.86:0/2699108062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123ddb00 msgr2=0x5615123da430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.788+0000 7f98d3400640 1 --2- 172.21.15.86:0/2699108062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123ddb00 0x5615123da430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.788+0000 7f98d3400640 1 -- 172.21.15.86:0/2699108062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615120503f0 con 0x5615123d7fa0 2024-04-17T07:32:13.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.789+0000 7f98d3400640 1 --2- 172.21.15.86:0/2699108062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123d9ef0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f98c0004b30 tx=0x7f98c0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50cb8793816fb0f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.789+0000 7f98c8c00640 1 -- 172.21.15.86:0/2699108062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0030070 con 0x5615123d7fa0 2024-04-17T07:32:13.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.790+0000 7f98c8c00640 1 -- 172.21.15.86:0/2699108062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98c0009530 con 0x5615123d7fa0 2024-04-17T07:32:13.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.790+0000 7f98c8c00640 1 -- 172.21.15.86:0/2699108062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0005100 con 0x5615123d7fa0 2024-04-17T07:32:13.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.791+0000 7f98ddb7a740 1 -- 172.21.15.86:0/2699108062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 msgr2=0x5615123d9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.791+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/2699108062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123d9ef0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f98c0004b30 tx=0x7f98c0005fd0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.791+0000 7f98ddb7a740 1 -- 172.21.15.86:0/2699108062 shutdown_connections 2024-04-17T07:32:13.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.791+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/2699108062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123ddb00 0x5615123da430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.792+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/2699108062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123d8990 0x5615123e99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.792+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/2699108062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123d9ef0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.792+0000 7f98ddb7a740 1 -- 172.21.15.86:0/2699108062 >> 172.21.15.86:0/2699108062 conn(0x5615123d3510 msgr2=0x5615123d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.792+0000 7f98ddb7a740 1 -- 172.21.15.86:0/2699108062 shutdown_connections 2024-04-17T07:32:13.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.792+0000 7f98ddb7a740 1 -- 172.21.15.86:0/2699108062 wait complete. 2024-04-17T07:32:13.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.793+0000 7f98ddb7a740 1 -- start start 2024-04-17T07:32:13.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.793+0000 7f98ddb7a740 1 Processor -- start 2024-04-17T07:32:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.796+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123e6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.796+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123d8990 0x5615123e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.796+0000 7f98ddb7a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123ddb00 0x5615123e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.796+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41c34a0 con 0x5615123d7fa0 2024-04-17T07:32:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.796+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41c2dd0 con 0x5615123d8990 2024-04-17T07:32:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.796+0000 7f98ddb7a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98d41c0000 con 0x5615123ddb00 2024-04-17T07:32:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.796+0000 7f98ddb7a740 10 client.0 _ll_register_callbacks cb 0x7f98c0012aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.796+0000 7f98d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123e6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.796+0000 7f98d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123e6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36446/0 (socket says 172.21.15.86:36446) 2024-04-17T07:32:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.796+0000 7f98d3400640 1 -- 172.21.15.86:0/1013720815 learned_addr learned my addr 172.21.15.86:0/1013720815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.797+0000 7f98d3e00640 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123d8990 0x5615123e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.797+0000 7f98d3400640 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123ddb00 0x5615123e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.797+0000 7f98d3400640 1 -- 172.21.15.86:0/1013720815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123ddb00 msgr2=0x5615123e84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.797+0000 7f98d3400640 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123ddb00 0x5615123e84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.797+0000 7f98d3400640 1 -- 172.21.15.86:0/1013720815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123d8990 msgr2=0x5615123e74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.797+0000 7f98d3400640 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123d8990 0x5615123e74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.797+0000 7f98d3400640 1 -- 172.21.15.86:0/1013720815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151211a7d0 con 0x5615123d7fa0 2024-04-17T07:32:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.798+0000 7f98d3400640 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123e6f60 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f98c0012e90 tx=0x7f98c0012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2229e84751fa1fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.798+0000 7f98be000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.798+0000 7f98be000640 1 -- 172.21.15.86:0/1013720815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0003aa0 con 0x5615123d7fa0 2024-04-17T07:32:13.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.798+0000 7f98ddb7a740 1 -- 172.21.15.86:0/1013720815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c0009530 con 0x5615123d7fa0 2024-04-17T07:32:13.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.799+0000 7f98be000640 1 -- 172.21.15.86:0/1013720815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f98c0003c40 con 0x5615123d7fa0 2024-04-17T07:32:13.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.799+0000 7f98be000640 1 -- 172.21.15.86:0/1013720815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98c0042050 con 0x5615123d7fa0 2024-04-17T07:32:13.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.799+0000 7f98ddb7a740 10 client.19518 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:13.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.800+0000 7f98ddb7a740 1 -- 172.21.15.86:0/1013720815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615123e9bd0 con 0x5615123d7fa0 2024-04-17T07:32:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.800+0000 7f98be000640 1 -- 172.21.15.86:0/1013720815 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98c0009530 con 0x5615123d7fa0 2024-04-17T07:32:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.800+0000 7f98be000640 1 -- 172.21.15.86:0/1013720815 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f98c00040b0 con 0x5615123d7fa0 2024-04-17T07:32:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.801+0000 7f98ddb7a740 10 client.19518 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.801+0000 7f98ddb7a740 10 client.19518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.801+0000 7f98ddb7a740 20 client.19518 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.801+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615123edeb0 0x5615124569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:13.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.801+0000 7f98ddb7a740 4 client.19518 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:32:13.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.801+0000 7f98ddb7a740 1 -- 172.21.15.86:0/1013720815 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615123458b0 con 0x5615123edeb0 2024-04-17T07:32:13.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.802+0000 7f98d3e00640 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615123edeb0 0x5615124569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:13.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.803+0000 7f98d3e00640 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615123edeb0 0x5615124569d0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7ce5b5c5a7fe7d server_cookie=20861c3187321f78 in_seq=0 out_seq=0 2024-04-17T07:32:13.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.803+0000 7f98be000640 10 client.19518 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:13.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.804+0000 7f98be000640 1 -- 172.21.15.86:0/1013720815 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615123458b0 con 0x5615123edeb0 2024-04-17T07:32:13.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.804+0000 7f98be000640 10 client.19518 handle_command_reply: tid=0 2024-04-17T07:32:13.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.804+0000 7f98ddb7a740 1 client.19518 shutdown 2024-04-17T07:32:13.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.804+0000 7f98ddb7a740 1 -- 172.21.15.86:0/1013720815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 msgr2=0x5615123e6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.805+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123e6f60 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f98c0012e90 tx=0x7f98c0012b50 comp rx=0 tx=0).stop 2024-04-17T07:32:13.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.805+0000 7f98ddb7a740 1 -- 172.21.15.86:0/1013720815 shutdown_connections 2024-04-17T07:32:13.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.805+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5615123edeb0 0x5615124569d0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.805+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615123ddb00 0x5615123e84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.805+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615123d8990 0x5615123e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.805+0000 7f98ddb7a740 1 --2- 172.21.15.86:0/1013720815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5615123d7fa0 0x5615123e6f60 secure :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f98c0012e90 tx=0x7f98c0012b50 comp rx=0 tx=0).stop 2024-04-17T07:32:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.805+0000 7f98ddb7a740 1 -- 172.21.15.86:0/1013720815 >> 172.21.15.86:0/1013720815 conn(0x5615123d3510 msgr2=0x5615123e2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.805+0000 7f98be000640 0 client.19518 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.806+0000 7f98ddb7a740 1 -- 172.21.15.86:0/1013720815 shutdown_connections 2024-04-17T07:32:13.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.806+0000 7f98ddb7a740 1 -- 172.21.15.86:0/1013720815 wait complete. 2024-04-17T07:32:13.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.806+0000 7f98ddb7a740 20 client.19518 trim_cache size 0 max 16384 2024-04-17T07:32:13.806 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:32:13.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.809+0000 7f98dae00640 1 -- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b80a3d30 msgr2=0x7f98b80a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.809+0000 7f98dae00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b80a3d30 0x7f98b80a61f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0184b0 tx=0x7f98cc018400 comp rx=0 tx=0).stop 2024-04-17T07:32:13.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.809+0000 7f98dae00640 1 -- 172.21.15.86:0/3814896814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d4116d70 msgr2=0x7f98d41b7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:13.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.809+0000 7f98dae00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d4116d70 0x7f98d41b7020 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00cce0 tx=0x7f98cc00a620 comp rx=0 tx=0).stop 2024-04-17T07:32:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.809+0000 7f98dae00640 1 -- 172.21.15.86:0/3814896814 shutdown_connections 2024-04-17T07:32:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.809+0000 7f98dae00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98b80a3d30 0x7f98b80a61f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.809+0000 7f98dae00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98d4116d70 0x7f98d41b7020 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.809+0000 7f98dae00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98d406d4e0 0x7f98d4113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.809+0000 7f98dae00640 1 --2- 172.21.15.86:0/3814896814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d406bf00 0x7f98d4113560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.809+0000 7f98dae00640 1 -- 172.21.15.86:0/3814896814 >> 172.21.15.86:0/3814896814 conn(0x7f98d4070f50 msgr2=0x7f98d410a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:13.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.810+0000 7f98dae00640 1 -- 172.21.15.86:0/3814896814 shutdown_connections 2024-04-17T07:32:13.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:13.810+0000 7f98dae00640 1 -- 172.21.15.86:0/3814896814 wait complete. 2024-04-17T07:32:13.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:13 smithi086 ceph-mon[31733]: pgmap v975: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.4 MiB/s wr, 707 op/s 2024-04-17T07:32:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 770.0 s, 435.4 tps, lat 230.220 ms stddev 204.944 2024-04-17T07:32:14.839 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:32:14.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:32:14.839 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:32:15.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.201+0000 7fa0b6800640 1 -- 172.21.15.86:0/3822887996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0b0108f60 msgr2=0x7fa0b010b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.201+0000 7fa0b6800640 1 --2- 172.21.15.86:0/3822887996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0b0108f60 0x7fa0b010b390 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009900 tx=0x7fa0a002bc30 comp rx=0 tx=0).stop 2024-04-17T07:32:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.204+0000 7fa0b6800640 1 -- 172.21.15.86:0/3822887996 shutdown_connections 2024-04-17T07:32:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.204+0000 7fa0b6800640 1 --2- 172.21.15.86:0/3822887996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b01126a0 0x7fa0b0114b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.204+0000 7fa0b6800640 1 --2- 172.21.15.86:0/3822887996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b010b8d0 0x7fa0b010bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.204+0000 7fa0b6800640 1 --2- 172.21.15.86:0/3822887996 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0b0108f60 0x7fa0b010b390 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.204+0000 7fa0b6800640 1 -- 172.21.15.86:0/3822887996 >> 172.21.15.86:0/3822887996 conn(0x7fa0b006f190 msgr2=0x7fa0b00715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.204+0000 7fa0b6800640 1 -- 172.21.15.86:0/3822887996 shutdown_connections 2024-04-17T07:32:15.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.205+0000 7fa0b6800640 1 -- 172.21.15.86:0/3822887996 wait complete. 2024-04-17T07:32:15.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.207+0000 7fa0b6800640 1 Processor -- start 2024-04-17T07:32:15.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.207+0000 7fa0b6800640 1 -- start start 2024-04-17T07:32:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.207+0000 7fa0b6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0b0108f60 0x7fa0b010fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.207+0000 7fa0b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b010b8d0 0x7fa0b0110130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.207+0000 7fa0b6800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b01126a0 0x7fa0b010c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.207+0000 7fa0b6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01184e0 con 0x7fa0b010b8d0 2024-04-17T07:32:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.207+0000 7fa0b6800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0118360 con 0x7fa0b01126a0 2024-04-17T07:32:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.207+0000 7fa0b6800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0118660 con 0x7fa0b0108f60 2024-04-17T07:32:15.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.208+0000 7fa0af400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0b0108f60 0x7fa0b010fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.208+0000 7fa0af400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0b0108f60 0x7fa0b010fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42098/0 (socket says 172.21.15.86:42098) 2024-04-17T07:32:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.208+0000 7fa0af400640 1 -- 172.21.15.86:0/3530178877 learned_addr learned my addr 172.21.15.86:0/3530178877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.208+0000 7fa0afe00640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b01126a0 0x7fa0b010c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.208+0000 7fa0aea00640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b010b8d0 0x7fa0b0110130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.209+0000 7fa0af400640 1 -- 172.21.15.86:0/3530178877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b01126a0 msgr2=0x7fa0b010c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.209+0000 7fa0af400640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b01126a0 0x7fa0b010c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.209+0000 7fa0af400640 1 -- 172.21.15.86:0/3530178877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b010b8d0 msgr2=0x7fa0b0110130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.209+0000 7fa0af400640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b010b8d0 0x7fa0b0110130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.209+0000 7fa0af400640 1 -- 172.21.15.86:0/3530178877 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b010d020 con 0x7fa0b0108f60 2024-04-17T07:32:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.209+0000 7fa0af400640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0b0108f60 0x7fa0b010fbf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0000c00 tx=0x7fa0a00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.210+0000 7fa0a7e00640 1 -- 172.21.15.86:0/3530178877 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0043020 con 0x7fa0b0108f60 2024-04-17T07:32:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.210+0000 7fa0a7e00640 1 -- 172.21.15.86:0/3530178877 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0a0004aa0 con 0x7fa0b0108f60 2024-04-17T07:32:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.210+0000 7fa0a7e00640 1 -- 172.21.15.86:0/3530178877 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0003d00 con 0x7fa0b0108f60 2024-04-17T07:32:15.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.210+0000 7fa0b6800640 1 -- 172.21.15.86:0/3530178877 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b01a2810 con 0x7fa0b0108f60 2024-04-17T07:32:15.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.211+0000 7fa0b6800640 1 -- 172.21.15.86:0/3530178877 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b01a2d10 con 0x7fa0b0108f60 2024-04-17T07:32:15.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.219+0000 7fa0a7e00640 1 -- 172.21.15.86:0/3530178877 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa0a0046020 con 0x7fa0b0108f60 2024-04-17T07:32:15.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.221+0000 7fa0b9409740 1 Processor -- start 2024-04-17T07:32:15.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.222+0000 7fa0b9409740 1 -- start start 2024-04-17T07:32:15.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.222+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 0x5616b54832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.222+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b54883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.222+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 0x5616b548aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.222+0000 7fa0b9409740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616b53e8cb0 con 0x5616b54838b0 2024-04-17T07:32:15.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.222+0000 7fa0b9409740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5616b538f370 con 0x5616b5488a20 2024-04-17T07:32:15.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.222+0000 7fa0b9409740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5616b53e4c70 con 0x5616b5482ec0 2024-04-17T07:32:15.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.223+0000 7fa0aea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 0x5616b548aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.223+0000 7fa0aea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 0x5616b548aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38690/0 (socket says 172.21.15.86:38690) 2024-04-17T07:32:15.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.223+0000 7fa0aea00640 1 -- 172.21.15.86:0/3038383678 learned_addr learned my addr 172.21.15.86:0/3038383678 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:15.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.223+0000 7fa0af400640 1 --2- 172.21.15.86:0/3038383678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b54883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.224+0000 7fa0aea00640 1 -- 172.21.15.86:0/3038383678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 msgr2=0x5616b54832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.224+0000 7fa0afe00640 1 --2- 172.21.15.86:0/3038383678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 0x5616b54832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.224+0000 7fa0aea00640 1 --2- 172.21.15.86:0/3038383678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 0x5616b54832e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.224+0000 7fa0aea00640 1 -- 172.21.15.86:0/3038383678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 msgr2=0x5616b54883b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.224+0000 7fa0aea00640 1 --2- 172.21.15.86:0/3038383678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b54883b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.224+0000 7fa0aea00640 1 -- 172.21.15.86:0/3038383678 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b53ae910 con 0x5616b5488a20 2024-04-17T07:32:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.224+0000 7fa0af400640 1 --2- 172.21.15.86:0/3038383678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b54883b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:32:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.224+0000 7fa0a7e00640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0940a3c30 0x7fa0940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.225+0000 7fa0aea00640 1 --2- 172.21.15.86:0/3038383678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 0x5616b548aeb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00bd10 tx=0x7fa09c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c43838c17dbe54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.225+0000 7fa0a7e00640 1 -- 172.21.15.86:0/3530178877 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa0a00f2aa0 con 0x7fa0b0108f60 2024-04-17T07:32:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.225+0000 7fa0afe00640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0940a3c30 0x7fa0940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.225+0000 7fa0a4c00640 1 -- 172.21.15.86:0/3038383678 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c005220 con 0x5616b5488a20 2024-04-17T07:32:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.225+0000 7fa0a4c00640 1 -- 172.21.15.86:0/3038383678 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa09c004c50 con 0x5616b5488a20 2024-04-17T07:32:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.225+0000 7fa0a4c00640 1 -- 172.21.15.86:0/3038383678 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c005220 con 0x5616b5488a20 2024-04-17T07:32:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.226+0000 7fa0b9409740 1 -- 172.21.15.86:0/3038383678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 msgr2=0x5616b548aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.226+0000 7fa0b9409740 1 --2- 172.21.15.86:0/3038383678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 0x5616b548aeb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00bd10 tx=0x7fa09c0107a0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.226+0000 7fa0afe00640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0940a3c30 0x7fa0940a60f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x5616b5484110 tx=0x7fa0a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.227+0000 7fa0b9409740 1 -- 172.21.15.86:0/3038383678 shutdown_connections 2024-04-17T07:32:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.227+0000 7fa0b9409740 1 --2- 172.21.15.86:0/3038383678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 0x5616b548aeb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.227+0000 7fa0b9409740 1 --2- 172.21.15.86:0/3038383678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b54883b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.227+0000 7fa0b9409740 1 --2- 172.21.15.86:0/3038383678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 0x5616b54832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.227+0000 7fa0b9409740 1 -- 172.21.15.86:0/3038383678 >> 172.21.15.86:0/3038383678 conn(0x5616b547e430 msgr2=0x5616b547e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.227+0000 7fa0b9409740 1 -- 172.21.15.86:0/3038383678 shutdown_connections 2024-04-17T07:32:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.228+0000 7fa0b9409740 1 -- 172.21.15.86:0/3038383678 wait complete. 2024-04-17T07:32:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.228+0000 7fa0b9409740 1 -- start start 2024-04-17T07:32:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.228+0000 7fa0b9409740 1 Processor -- start 2024-04-17T07:32:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.229+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5482ec0 0x5616b548d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.229+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b54838b0 0x5616b548d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.231+0000 7fa0af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5482ec0 0x5616b548d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.231+0000 7fa0aea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b54838b0 0x5616b548d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.231+0000 7fa0aea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b54838b0 0x5616b548d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38702/0 (socket says 172.21.15.86:38702) 2024-04-17T07:32:15.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.231+0000 7fa0aea00640 1 -- 172.21.15.86:0/1359873107 learned_addr learned my addr 172.21.15.86:0/1359873107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.231+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5488a20 0x5616b54947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.231+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5616b548b9d0 con 0x5616b5482ec0 2024-04-17T07:32:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.232+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5616b53a2730 con 0x5616b54838b0 2024-04-17T07:32:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.232+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5616b548bb50 con 0x5616b5488a20 2024-04-17T07:32:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.232+0000 7fa0afe00640 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5488a20 0x5616b54947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.232+0000 7fa0b9409740 10 client.0 _ll_register_callbacks cb 0x5616b53a4b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.233+0000 7fa0aea00640 1 -- 172.21.15.86:0/1359873107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5488a20 msgr2=0x5616b54947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.233+0000 7fa0aea00640 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5488a20 0x5616b54947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.233+0000 7fa0aea00640 1 -- 172.21.15.86:0/1359873107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5482ec0 msgr2=0x5616b548d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.233+0000 7fa0aea00640 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5482ec0 0x5616b548d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.233+0000 7fa0aea00640 1 -- 172.21.15.86:0/1359873107 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b51c37e0 con 0x5616b54838b0 2024-04-17T07:32:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.234+0000 7fa0aea00640 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b54838b0 0x5616b548d760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa09c005d00 tx=0x7fa09c0048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9031b60f1e8aacc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.234+0000 7fa099600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.235+0000 7fa099600640 1 -- 172.21.15.86:0/1359873107 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c004220 con 0x5616b54838b0 2024-04-17T07:32:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.235+0000 7fa099600640 1 -- 172.21.15.86:0/1359873107 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa09c0043c0 con 0x5616b54838b0 2024-04-17T07:32:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.235+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa09c004c50 con 0x5616b54838b0 2024-04-17T07:32:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.235+0000 7fa099600640 1 -- 172.21.15.86:0/1359873107 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa09c030050 con 0x5616b54838b0 2024-04-17T07:32:15.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.236+0000 7fa0b9409740 10 client.38540 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:15.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.236+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616b5223f70 con 0x5616b54838b0 2024-04-17T07:32:15.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.236+0000 7fa099600640 1 -- 172.21.15.86:0/1359873107 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa09c004c50 con 0x5616b54838b0 2024-04-17T07:32:15.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.237+0000 7fa099600640 1 -- 172.21.15.86:0/1359873107 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa09c0035f0 con 0x5616b54838b0 2024-04-17T07:32:15.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.237+0000 7fa0b9409740 10 client.38540 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:15.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.237+0000 7fa0b9409740 10 client.38540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:15.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.237+0000 7fa0b9409740 20 client.38540 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:15.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.237+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5616b5498c90 0x5616b549b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.237+0000 7fa0b9409740 4 client.38540 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:15.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.237+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b53f66d0 con 0x5616b5498c90 2024-04-17T07:32:15.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.237+0000 7fa0af400640 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5616b5498c90 0x5616b549b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.238+0000 7fa099600640 10 client.38540 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:15.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.238+0000 7fa0af400640 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5616b5498c90 0x5616b549b080 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9623da443f108ada server_cookie=4340cc15192b08a7 in_seq=0 out_seq=0 2024-04-17T07:32:15.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.239+0000 7fa099600640 1 -- 172.21.15.86:0/1359873107 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616b53f66d0 con 0x5616b5498c90 2024-04-17T07:32:15.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.240+0000 7fa099600640 10 client.38540 handle_command_reply: tid=0 2024-04-17T07:32:15.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.240+0000 7fa0b9409740 1 client.38540 shutdown 2024-04-17T07:32:15.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.241+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b54838b0 msgr2=0x5616b548d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.241+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b54838b0 0x5616b548d760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa09c005d00 tx=0x7fa09c0048f0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.242+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 shutdown_connections 2024-04-17T07:32:15.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.242+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5616b5498c90 0x5616b549b080 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.242+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5488a20 0x5616b54947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.242+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b54838b0 0x5616b548d760 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.242+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1359873107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b5482ec0 0x5616b548d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.242+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 >> 172.21.15.86:0/1359873107 conn(0x5616b547e430 msgr2=0x5616b547d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.242+0000 7fa099600640 0 client.38540 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:15.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.242+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 shutdown_connections 2024-04-17T07:32:15.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.243+0000 7fa0b9409740 1 -- 172.21.15.86:0/1359873107 wait complete. 2024-04-17T07:32:15.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.243+0000 7fa0b9409740 20 client.38540 trim_cache size 0 max 16384 2024-04-17T07:32:15.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.291+0000 7fa0b9409740 1 Processor -- start 2024-04-17T07:32:15.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.292+0000 7fa0b9409740 1 -- start start 2024-04-17T07:32:15.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.293+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 0x5616b5481da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.293+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b5494760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.293+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 0x5616b547cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.293+0000 7fa0b9409740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01c2120 con 0x5616b54838b0 2024-04-17T07:32:15.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.293+0000 7fa0b9409740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01c3c60 con 0x5616b5488a20 2024-04-17T07:32:15.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.293+0000 7fa0b9409740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01c27f0 con 0x5616b5482ec0 2024-04-17T07:32:15.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.293+0000 7fa0af400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 0x5616b5481da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.294+0000 7fa0af400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 0x5616b5481da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:42132/0 (socket says 172.21.15.86:42132) 2024-04-17T07:32:15.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.294+0000 7fa0af400640 1 -- 172.21.15.86:0/3442507940 learned_addr learned my addr 172.21.15.86:0/3442507940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:15.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.294+0000 7fa0afe00640 1 --2- 172.21.15.86:0/3442507940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 0x5616b547cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.294+0000 7fa0aea00640 1 --2- 172.21.15.86:0/3442507940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b5494760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.294+0000 7fa0af400640 1 -- 172.21.15.86:0/3442507940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 msgr2=0x5616b547cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.294+0000 7fa0af400640 1 --2- 172.21.15.86:0/3442507940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 0x5616b547cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.294+0000 7fa0af400640 1 -- 172.21.15.86:0/3442507940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 msgr2=0x5616b5494760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.294+0000 7fa0af400640 1 --2- 172.21.15.86:0/3442507940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b5494760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.294+0000 7fa0af400640 1 -- 172.21.15.86:0/3442507940 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b54852f0 con 0x5616b5482ec0 2024-04-17T07:32:15.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.295+0000 7fa0af400640 1 --2- 172.21.15.86:0/3442507940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 0x5616b5481da0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00bb370 tx=0x7fa0a00f4040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7273ba4a4dc9b6ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.298+0000 7fa09be00640 1 -- 172.21.15.86:0/3442507940 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00044e0 con 0x5616b5482ec0 2024-04-17T07:32:15.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.298+0000 7fa09be00640 1 -- 172.21.15.86:0/3442507940 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0a00f2e50 con 0x5616b5482ec0 2024-04-17T07:32:15.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.298+0000 7fa09be00640 1 -- 172.21.15.86:0/3442507940 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00c2d50 con 0x5616b5482ec0 2024-04-17T07:32:15.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.300+0000 7fa0b9409740 1 -- 172.21.15.86:0/3442507940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 msgr2=0x5616b5481da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.300+0000 7fa0b9409740 1 --2- 172.21.15.86:0/3442507940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 0x5616b5481da0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00bb370 tx=0x7fa0a00f4040 comp rx=0 tx=0).stop 2024-04-17T07:32:15.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.300+0000 7fa0b9409740 1 -- 172.21.15.86:0/3442507940 shutdown_connections 2024-04-17T07:32:15.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.300+0000 7fa0b9409740 1 --2- 172.21.15.86:0/3442507940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5488a20 0x5616b547cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.300+0000 7fa0b9409740 1 --2- 172.21.15.86:0/3442507940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b5494760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.300+0000 7fa0b9409740 1 --2- 172.21.15.86:0/3442507940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5482ec0 0x5616b5481da0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.300+0000 7fa0b9409740 1 -- 172.21.15.86:0/3442507940 >> 172.21.15.86:0/3442507940 conn(0x5616b547e430 msgr2=0x5616b5481690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.300+0000 7fa0b9409740 1 -- 172.21.15.86:0/3442507940 shutdown_connections 2024-04-17T07:32:15.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.300+0000 7fa0b9409740 1 -- 172.21.15.86:0/3442507940 wait complete. 2024-04-17T07:32:15.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.301+0000 7fa0b9409740 1 -- start start 2024-04-17T07:32:15.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.301+0000 7fa0b9409740 1 Processor -- start 2024-04-17T07:32:15.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.301+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5482ec0 0x5616b5492560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.302+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b5492aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.302+0000 7fa0b9409740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5488a20 0x5616b5493a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.302+0000 7fa0b9409740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01c3590 con 0x5616b54838b0 2024-04-17T07:32:15.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.302+0000 7fa0b9409740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01c2ec0 con 0x5616b5482ec0 2024-04-17T07:32:15.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.302+0000 7fa0b9409740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01c0110 con 0x5616b5488a20 2024-04-17T07:32:15.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.302+0000 7fa0b9409740 10 client.0 _ll_register_callbacks cb 0x7fa09c00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:15.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.302+0000 7fa0af400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5482ec0 0x5616b5492560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.302+0000 7fa0af400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5482ec0 0x5616b5492560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38728/0 (socket says 172.21.15.86:38728) 2024-04-17T07:32:15.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.302+0000 7fa0af400640 1 -- 172.21.15.86:0/1779362616 learned_addr learned my addr 172.21.15.86:0/1779362616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:15.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.302+0000 7fa0afe00640 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5488a20 0x5616b5493a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.303+0000 7fa0af400640 1 -- 172.21.15.86:0/1779362616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5488a20 msgr2=0x5616b5493a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.303+0000 7fa0af400640 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5488a20 0x5616b5493a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.303+0000 7fa0aea00640 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b5492aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.303+0000 7fa0af400640 1 -- 172.21.15.86:0/1779362616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 msgr2=0x5616b5492aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.303+0000 7fa0af400640 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b5492aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.303+0000 7fa0af400640 1 -- 172.21.15.86:0/1779362616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b50bb170 con 0x5616b5482ec0 2024-04-17T07:32:15.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.304+0000 7fa0af400640 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5482ec0 0x5616b5492560 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa0a002c6a0 tx=0x7fa0a0005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9ec8fef0c0c581c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.304+0000 7fa099600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:15.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.304+0000 7fa099600640 1 -- 172.21.15.86:0/1779362616 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00056a0 con 0x5616b5482ec0 2024-04-17T07:32:15.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.304+0000 7fa0b9409740 1 -- 172.21.15.86:0/1779362616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a00f2e50 con 0x5616b5482ec0 2024-04-17T07:32:15.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.305+0000 7fa099600640 1 -- 172.21.15.86:0/1779362616 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa0a00bca40 con 0x5616b5482ec0 2024-04-17T07:32:15.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.305+0000 7fa099600640 1 -- 172.21.15.86:0/1779362616 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa0a00da050 con 0x5616b5482ec0 2024-04-17T07:32:15.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.305+0000 7fa0b9409740 10 client.38546 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:15.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.305+0000 7fa0b9409740 1 -- 172.21.15.86:0/1779362616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616b54ff650 con 0x5616b5482ec0 2024-04-17T07:32:15.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.306+0000 7fa099600640 1 -- 172.21.15.86:0/1779362616 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0a00f2e50 con 0x5616b5482ec0 2024-04-17T07:32:15.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.306+0000 7fa099600640 1 -- 172.21.15.86:0/1779362616 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa0a00c7210 con 0x5616b5482ec0 2024-04-17T07:32:15.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.306+0000 7fa0b9409740 10 client.38546 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:15.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.306+0000 7fa0b9409740 10 client.38546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:15.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.306+0000 7fa0b9409740 20 client.38546 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:15.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.306+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5616b54ffec0 0x5616b55022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.306+0000 7fa0b9409740 4 client.38546 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:32:15.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.306+0000 7fa0b9409740 1 -- 172.21.15.86:0/1779362616 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5616b53f0540 con 0x5616b54ffec0 2024-04-17T07:32:15.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.307+0000 7fa0aea00640 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5616b54ffec0 0x5616b55022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.308+0000 7fa099600640 10 client.38546 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:15.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.308+0000 7fa0aea00640 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5616b54ffec0 0x5616b55022e0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a575ffc3399706 server_cookie=424e8e035d61e07a in_seq=0 out_seq=0 2024-04-17T07:32:15.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.311+0000 7fa099600640 1 -- 172.21.15.86:0/1779362616 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5616b53f0540 con 0x5616b54ffec0 2024-04-17T07:32:15.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.311+0000 7fa099600640 10 client.38546 handle_command_reply: tid=0 2024-04-17T07:32:15.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.311+0000 7fa0b9409740 1 client.38546 shutdown 2024-04-17T07:32:15.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.312+0000 7fa0b9409740 1 -- 172.21.15.86:0/1779362616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5482ec0 msgr2=0x5616b5492560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.312+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5482ec0 0x5616b5492560 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa0a002c6a0 tx=0x7fa0a0005b50 comp rx=0 tx=0).stop 2024-04-17T07:32:15.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.313+0000 7fa0b9409740 1 -- 172.21.15.86:0/1779362616 shutdown_connections 2024-04-17T07:32:15.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.313+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5616b54ffec0 0x5616b55022e0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.313+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5616b5488a20 0x5616b5493a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.313+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5616b54838b0 0x5616b5492aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.313+0000 7fa0b9409740 1 --2- 172.21.15.86:0/1779362616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5616b5482ec0 0x5616b5492560 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.313+0000 7fa0b9409740 1 -- 172.21.15.86:0/1779362616 >> 172.21.15.86:0/1779362616 conn(0x5616b547e430 msgr2=0x5616b547d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.313+0000 7fa099600640 0 client.38546 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:15.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.313+0000 7fa0b9409740 1 -- 172.21.15.86:0/1779362616 shutdown_connections 2024-04-17T07:32:15.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.314+0000 7fa0b9409740 1 -- 172.21.15.86:0/1779362616 wait complete. 2024-04-17T07:32:15.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.314+0000 7fa0b9409740 20 client.38546 trim_cache size 0 max 16384 2024-04-17T07:32:15.314 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:32:15.314 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:32:15.314 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "c19b841b-5096-4d67-8c1e-845a83aaf819", 2024-04-17T07:32:15.315 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:32:15.315 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:32:15.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.316+0000 7fa0b6800640 1 -- 172.21.15.86:0/3530178877 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0940a3c30 msgr2=0x7fa0940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.316+0000 7fa0b6800640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0940a3c30 0x7fa0940a60f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x5616b5484110 tx=0x7fa0a8009500 comp rx=0 tx=0).stop 2024-04-17T07:32:15.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.316+0000 7fa0b6800640 1 -- 172.21.15.86:0/3530178877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0b0108f60 msgr2=0x7fa0b010fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.316+0000 7fa0b6800640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0b0108f60 0x7fa0b010fbf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0000c00 tx=0x7fa0a00042f0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.316+0000 7fa0b6800640 1 -- 172.21.15.86:0/3530178877 shutdown_connections 2024-04-17T07:32:15.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.316+0000 7fa0b6800640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0940a3c30 0x7fa0940a60f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.316+0000 7fa0b6800640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b01126a0 0x7fa0b010c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.316+0000 7fa0b6800640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0b010b8d0 0x7fa0b0110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.316+0000 7fa0b6800640 1 --2- 172.21.15.86:0/3530178877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0b0108f60 0x7fa0b010fbf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.316+0000 7fa0b6800640 1 -- 172.21.15.86:0/3530178877 >> 172.21.15.86:0/3530178877 conn(0x7fa0b006f190 msgr2=0x7fa0b01099e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.317+0000 7fa0b6800640 1 -- 172.21.15.86:0/3530178877 shutdown_connections 2024-04-17T07:32:15.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.317+0000 7fa0b6800640 1 -- 172.21.15.86:0/3530178877 wait complete. 2024-04-17T07:32:15.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:32:15.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:32:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.737+0000 7f98a3800640 1 -- 172.21.15.86:0/2882881972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f989c1041a0 msgr2=0x7f989c1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.737+0000 7f98a3800640 1 --2- 172.21.15.86:0/2882881972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f989c1041a0 0x7f989c1045c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9890009860 tx=0x7f989002bb70 comp rx=0 tx=0).stop 2024-04-17T07:32:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.737+0000 7f98a3800640 1 -- 172.21.15.86:0/2882881972 shutdown_connections 2024-04-17T07:32:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.737+0000 7f98a3800640 1 --2- 172.21.15.86:0/2882881972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f989c105d50 0x7f989c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.737+0000 7f98a3800640 1 --2- 172.21.15.86:0/2882881972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f989c105390 0x7f989c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.737+0000 7f98a3800640 1 --2- 172.21.15.86:0/2882881972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f989c1041a0 0x7f989c1045c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.737+0000 7f98a3800640 1 -- 172.21.15.86:0/2882881972 >> 172.21.15.86:0/2882881972 conn(0x7f989c0ff710 msgr2=0x7f989c101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.737+0000 7f98a3800640 1 -- 172.21.15.86:0/2882881972 shutdown_connections 2024-04-17T07:32:15.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.738+0000 7f98a3800640 1 -- 172.21.15.86:0/2882881972 wait complete. 2024-04-17T07:32:15.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.739+0000 7f98a3800640 1 Processor -- start 2024-04-17T07:32:15.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.739+0000 7f98a3800640 1 -- start start 2024-04-17T07:32:15.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.739+0000 7f98a3800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f989c1041a0 0x7f989c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.739+0000 7f98a3800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f989c105390 0x7f989c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.740+0000 7f98a3800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f989c105d50 0x7f989c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.740+0000 7f98a3800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f989c10fce0 con 0x7f989c105390 2024-04-17T07:32:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.740+0000 7f98a3800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f989c10fb60 con 0x7f989c1041a0 2024-04-17T07:32:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.740+0000 7f98a3800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f989c10fe60 con 0x7f989c105d50 2024-04-17T07:32:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.740+0000 7f98a0e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f989c1041a0 0x7f989c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.740+0000 7f989be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f989c105390 0x7f989c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.740+0000 7f98a0e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f989c1041a0 0x7f989c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38746/0 (socket says 172.21.15.86:38746) 2024-04-17T07:32:15.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.740+0000 7f98a0e00640 1 -- 172.21.15.86:0/1767952613 learned_addr learned my addr 172.21.15.86:0/1767952613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.740+0000 7f98a0e00640 1 -- 172.21.15.86:0/1767952613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f989c105d50 msgr2=0x7f989c1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.741+0000 7f98a0e00640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f989c105d50 0x7f989c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.741+0000 7f98a0e00640 1 -- 172.21.15.86:0/1767952613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f989c105390 msgr2=0x7f989c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.741+0000 7f98a0e00640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f989c105390 0x7f989c19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.741+0000 7f98a0e00640 1 -- 172.21.15.86:0/1767952613 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989c1a2040 con 0x7f989c1041a0 2024-04-17T07:32:15.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.741+0000 7f98a0e00640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f989c1041a0 0x7f989c19a420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f98900057c0 tx=0x7f98900042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.742+0000 7f9899600640 1 -- 172.21.15.86:0/1767952613 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890041030 con 0x7f989c1041a0 2024-04-17T07:32:15.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.742+0000 7f9899600640 1 -- 172.21.15.86:0/1767952613 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f989002c9c0 con 0x7f989c1041a0 2024-04-17T07:32:15.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.742+0000 7f9899600640 1 -- 172.21.15.86:0/1767952613 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890039040 con 0x7f989c1041a0 2024-04-17T07:32:15.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.742+0000 7f98a3800640 1 -- 172.21.15.86:0/1767952613 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989c1a22d0 con 0x7f989c1041a0 2024-04-17T07:32:15.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.742+0000 7f98a3800640 1 -- 172.21.15.86:0/1767952613 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f989c1a2770 con 0x7f989c1041a0 2024-04-17T07:32:15.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.747+0000 7f98a65f2740 1 Processor -- start 2024-04-17T07:32:15.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.747+0000 7f98a65f2740 1 -- start start 2024-04-17T07:32:15.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.748+0000 7f9899600640 1 -- 172.21.15.86:0/1767952613 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9890044070 con 0x7f989c1041a0 2024-04-17T07:32:15.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.749+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435887f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.749+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x5643588849c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.749+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 0x56435889d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.749+0000 7f98a65f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643587eb120 con 0x56435887fec0 2024-04-17T07:32:15.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.749+0000 7f98a65f2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564358791340 con 0x56435887f4d0 2024-04-17T07:32:15.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.749+0000 7f98a65f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5643587e70e0 con 0x564358885030 2024-04-17T07:32:15.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.749+0000 7f98a1800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435887f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.749+0000 7f98a1800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435887f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38748/0 (socket says 172.21.15.86:38748) 2024-04-17T07:32:15.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.749+0000 7f98a1800640 1 -- 172.21.15.86:0/3104197793 learned_addr learned my addr 172.21.15.86:0/3104197793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:15.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.750+0000 7f989be00640 1 --2- 172.21.15.86:0/3104197793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 0x56435889d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.750+0000 7f98a1800640 1 -- 172.21.15.86:0/3104197793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 msgr2=0x56435889d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.750+0000 7f98a1800640 1 --2- 172.21.15.86:0/3104197793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 0x56435889d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.750+0000 7f98a1800640 1 -- 172.21.15.86:0/3104197793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 msgr2=0x5643588849c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.750+0000 7f98a1800640 1 --2- 172.21.15.86:0/3104197793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x5643588849c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.750+0000 7f98a1800640 1 -- 172.21.15.86:0/3104197793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643585c77d0 con 0x56435887f4d0 2024-04-17T07:32:15.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.750+0000 7f989be00640 1 --2- 172.21.15.86:0/3104197793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 0x56435889d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:32:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.751+0000 7f98a1800640 1 --2- 172.21.15.86:0/3104197793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435887f8f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f989400e470 tx=0x7f989400e870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771f6583800764f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.751+0000 7f988e000640 1 -- 172.21.15.86:0/3104197793 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894004d20 con 0x56435887f4d0 2024-04-17T07:32:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.751+0000 7f988e000640 1 -- 172.21.15.86:0/3104197793 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9894003e20 con 0x56435887f4d0 2024-04-17T07:32:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.751+0000 7f9899600640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98840a3d80 0x7f98840a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.751+0000 7f988e000640 1 -- 172.21.15.86:0/3104197793 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989400c8a0 con 0x56435887f4d0 2024-04-17T07:32:15.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.752+0000 7f9899600640 1 -- 172.21.15.86:0/1767952613 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f98900f1cb0 con 0x7f989c1041a0 2024-04-17T07:32:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.752+0000 7f98a0e00640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98840a3d80 0x7f98840a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.752+0000 7f98a65f2740 1 -- 172.21.15.86:0/3104197793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 msgr2=0x56435887f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.752+0000 7f98a65f2740 1 --2- 172.21.15.86:0/3104197793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435887f8f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f989400e470 tx=0x7f989400e870 comp rx=0 tx=0).stop 2024-04-17T07:32:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.752+0000 7f98a65f2740 1 -- 172.21.15.86:0/3104197793 shutdown_connections 2024-04-17T07:32:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.752+0000 7f98a65f2740 1 --2- 172.21.15.86:0/3104197793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 0x56435889d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.753+0000 7f98a65f2740 1 --2- 172.21.15.86:0/3104197793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x5643588849c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.753+0000 7f98a65f2740 1 --2- 172.21.15.86:0/3104197793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435887f8f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.753+0000 7f98a65f2740 1 -- 172.21.15.86:0/3104197793 >> 172.21.15.86:0/3104197793 conn(0x564358887f10 msgr2=0x564358888310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.753+0000 7f98a65f2740 1 -- 172.21.15.86:0/3104197793 shutdown_connections 2024-04-17T07:32:15.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.753+0000 7f98a65f2740 1 -- 172.21.15.86:0/3104197793 wait complete. 2024-04-17T07:32:15.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.753+0000 7f98a0e00640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98840a3d80 0x7f98840a6240 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f98900bd8b0 tx=0x7f9890046f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.753+0000 7f98a65f2740 1 -- start start 2024-04-17T07:32:15.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.754+0000 7f98a65f2740 1 Processor -- start 2024-04-17T07:32:15.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.754+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435889f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.755+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x56435889fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.755+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 0x5643588a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.755+0000 7f98a65f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56435889dcb0 con 0x56435887fec0 2024-04-17T07:32:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.755+0000 7f98a65f2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5643587a4710 con 0x56435887f4d0 2024-04-17T07:32:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.755+0000 7f98a65f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56435889de30 con 0x564358885030 2024-04-17T07:32:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.755+0000 7f98a65f2740 10 client.0 _ll_register_callbacks cb 0x56435879faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.755+0000 7f98a1800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435889f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.755+0000 7f98a1800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 0x5643588a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.755+0000 7f98a1800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435889f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38762/0 (socket says 172.21.15.86:38762) 2024-04-17T07:32:15.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.755+0000 7f98a1800640 1 -- 172.21.15.86:0/3252266869 learned_addr learned my addr 172.21.15.86:0/3252266869 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:15.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.756+0000 7f989be00640 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x56435889fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.756+0000 7f989be00640 1 -- 172.21.15.86:0/3252266869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 msgr2=0x5643588a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.756+0000 7f989be00640 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 0x5643588a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.757+0000 7f989be00640 1 -- 172.21.15.86:0/3252266869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 msgr2=0x56435889f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.757+0000 7f989be00640 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435889f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.757+0000 7f989be00640 1 -- 172.21.15.86:0/3252266869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643584fd3f0 con 0x56435887fec0 2024-04-17T07:32:15.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.757+0000 7f98a1800640 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435889f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:15.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.757+0000 7f989be00640 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x56435889fce0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f989c19b930 tx=0x7f9888013b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9299bb760b5ff407 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.757+0000 7f9882a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:15.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.758+0000 7f9882a00640 1 -- 172.21.15.86:0/3252266869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888004460 con 0x56435887fec0 2024-04-17T07:32:15.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.758+0000 7f9882a00640 1 -- 172.21.15.86:0/3252266869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9888011960 con 0x56435887fec0 2024-04-17T07:32:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.758+0000 7f98a65f2740 1 -- 172.21.15.86:0/3252266869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9894003e20 con 0x56435887fec0 2024-04-17T07:32:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.759+0000 7f9882a00640 1 -- 172.21.15.86:0/3252266869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9888011b60 con 0x56435887fec0 2024-04-17T07:32:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.759+0000 7f98a65f2740 10 client.19539 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.759+0000 7f98a65f2740 1 -- 172.21.15.86:0/3252266869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643584bd0f0 con 0x56435887fec0 2024-04-17T07:32:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.760+0000 7f9882a00640 1 -- 172.21.15.86:0/3252266869 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9894003e20 con 0x56435887fec0 2024-04-17T07:32:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.760+0000 7f9882a00640 1 -- 172.21.15.86:0/3252266869 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f988801a070 con 0x56435887fec0 2024-04-17T07:32:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.760+0000 7f98a65f2740 10 client.19539 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.760+0000 7f98a65f2740 10 client.19539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.760+0000 7f98a65f2740 20 client.19539 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.760+0000 7f98a65f2740 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5643588ab520 0x5643588ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.760+0000 7f98a65f2740 4 client.19539 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.760+0000 7f98a65f2740 1 -- 172.21.15.86:0/3252266869 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643587f8b40 con 0x5643588ab520 2024-04-17T07:32:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.761+0000 7f98a1800640 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5643588ab520 0x5643588ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.762+0000 7f98a1800640 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5643588ab520 0x5643588ad9a0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c1ee3d28f84497f server_cookie=13aece99a4409ab6 in_seq=0 out_seq=0 2024-04-17T07:32:15.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.762+0000 7f9882a00640 10 client.19539 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:15.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.764+0000 7f9882a00640 1 -- 172.21.15.86:0/3252266869 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643587f8b40 con 0x5643588ab520 2024-04-17T07:32:15.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.764+0000 7f9882a00640 10 client.19539 handle_command_reply: tid=0 2024-04-17T07:32:15.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.764+0000 7f98a65f2740 1 client.19539 shutdown 2024-04-17T07:32:15.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.765+0000 7f98a65f2740 1 -- 172.21.15.86:0/3252266869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 msgr2=0x56435889fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.765+0000 7f98a65f2740 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x56435889fce0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f989c19b930 tx=0x7f9888013b70 comp rx=0 tx=0).stop 2024-04-17T07:32:15.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.766+0000 7f98a65f2740 1 -- 172.21.15.86:0/3252266869 shutdown_connections 2024-04-17T07:32:15.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.766+0000 7f98a65f2740 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5643588ab520 0x5643588ad9a0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.766+0000 7f98a65f2740 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564358885030 0x5643588a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.766+0000 7f98a65f2740 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x56435889fce0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.766+0000 7f98a65f2740 1 --2- 172.21.15.86:0/3252266869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887f4d0 0x56435889f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.766+0000 7f98a65f2740 1 -- 172.21.15.86:0/3252266869 >> 172.21.15.86:0/3252266869 conn(0x564358887f10 msgr2=0x56435887e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.767+0000 7f9882a00640 0 client.19539 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.767+0000 7f98a65f2740 1 -- 172.21.15.86:0/3252266869 shutdown_connections 2024-04-17T07:32:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.768+0000 7f98a65f2740 1 -- 172.21.15.86:0/3252266869 wait complete. 2024-04-17T07:32:15.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.768+0000 7f98a65f2740 20 client.19539 trim_cache size 0 max 16384 2024-04-17T07:32:15.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:15 smithi103 ceph-mon[35495]: pgmap v976: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.9 MiB/s wr, 641 op/s 2024-04-17T07:32:15.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.819+0000 7f98a65f2740 1 Processor -- start 2024-04-17T07:32:15.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.820+0000 7f98a65f2740 1 -- start start 2024-04-17T07:32:15.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.820+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 0x5643588882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.820+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887fec0 0x5643588a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.820+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564358885030 0x56435889e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.820+0000 7f98a65f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1a8a00 con 0x564358885030 2024-04-17T07:32:15.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.820+0000 7f98a65f2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1aa540 con 0x56435887fec0 2024-04-17T07:32:15.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.820+0000 7f98a65f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1a90d0 con 0x56435887f4d0 2024-04-17T07:32:15.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.821+0000 7f989be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887fec0 0x5643588a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.821+0000 7f989be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887fec0 0x5643588a6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38764/0 (socket says 172.21.15.86:38764) 2024-04-17T07:32:15.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.821+0000 7f989be00640 1 -- 172.21.15.86:0/285071364 learned_addr learned my addr 172.21.15.86:0/285071364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:15.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.822+0000 7f98a1800640 1 --2- 172.21.15.86:0/285071364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 0x5643588882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.822+0000 7f989be00640 1 -- 172.21.15.86:0/285071364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 msgr2=0x5643588882f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.822+0000 7f989be00640 1 --2- 172.21.15.86:0/285071364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 0x5643588882f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.822+0000 7f989be00640 1 -- 172.21.15.86:0/285071364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564358885030 msgr2=0x56435889e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.822+0000 7f989be00640 1 --2- 172.21.15.86:0/285071364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564358885030 0x56435889e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.822+0000 7f989be00640 1 -- 172.21.15.86:0/285071364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564358879ce0 con 0x56435887fec0 2024-04-17T07:32:15.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.822+0000 7f98a1800640 1 --2- 172.21.15.86:0/285071364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 0x5643588882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:32:15.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.823+0000 7f989be00640 1 --2- 172.21.15.86:0/285071364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887fec0 0x5643588a6ce0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9888001eb0 tx=0x7f9888003670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=454647defd1cc84 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.824+0000 7f988d600640 1 -- 172.21.15.86:0/285071364 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988802a070 con 0x56435887fec0 2024-04-17T07:32:15.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.824+0000 7f988d600640 1 -- 172.21.15.86:0/285071364 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9888011d00 con 0x56435887fec0 2024-04-17T07:32:15.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.824+0000 7f988d600640 1 -- 172.21.15.86:0/285071364 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888003b00 con 0x56435887fec0 2024-04-17T07:32:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.826+0000 7f98a65f2740 1 -- 172.21.15.86:0/285071364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887fec0 msgr2=0x5643588a6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.826+0000 7f98a65f2740 1 --2- 172.21.15.86:0/285071364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887fec0 0x5643588a6ce0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9888001eb0 tx=0x7f9888003670 comp rx=0 tx=0).stop 2024-04-17T07:32:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.827+0000 7f98a65f2740 1 -- 172.21.15.86:0/285071364 shutdown_connections 2024-04-17T07:32:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.827+0000 7f98a65f2740 1 --2- 172.21.15.86:0/285071364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564358885030 0x56435889e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.827+0000 7f98a65f2740 1 --2- 172.21.15.86:0/285071364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56435887fec0 0x5643588a6ce0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.827+0000 7f98a65f2740 1 --2- 172.21.15.86:0/285071364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 0x5643588882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.827+0000 7f98a65f2740 1 -- 172.21.15.86:0/285071364 >> 172.21.15.86:0/285071364 conn(0x564358887f10 msgr2=0x564358881450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.828+0000 7f98a65f2740 1 -- 172.21.15.86:0/285071364 shutdown_connections 2024-04-17T07:32:15.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.828+0000 7f98a65f2740 1 -- 172.21.15.86:0/285071364 wait complete. 2024-04-17T07:32:15.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.829+0000 7f98a65f2740 1 -- start start 2024-04-17T07:32:15.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.829+0000 7f98a65f2740 1 Processor -- start 2024-04-17T07:32:15.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.831+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 0x5643588a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.831+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x5643588a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.831+0000 7f98a65f2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564358885030 0x5643588a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.831+0000 7f98a65f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1a9e70 con 0x56435887fec0 2024-04-17T07:32:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.831+0000 7f98a65f2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1a97a0 con 0x564358885030 2024-04-17T07:32:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.831+0000 7f98a65f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1a69d0 con 0x56435887f4d0 2024-04-17T07:32:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.831+0000 7f98a65f2740 10 client.0 _ll_register_callbacks cb 0x7f9888009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:15.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.832+0000 7f989be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x5643588a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.832+0000 7f989be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x5643588a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36594/0 (socket says 172.21.15.86:36594) 2024-04-17T07:32:15.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.832+0000 7f989be00640 1 -- 172.21.15.86:0/44943419 learned_addr learned my addr 172.21.15.86:0/44943419 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:15.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.832+0000 7f98a1800640 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 0x5643588a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.833 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:15 smithi086 ceph-mon[31733]: pgmap v976: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.9 MiB/s wr, 641 op/s 2024-04-17T07:32:15.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.832+0000 7f98a1800640 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564358885030 0x5643588a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.834+0000 7f989be00640 1 -- 172.21.15.86:0/44943419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 msgr2=0x5643588a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.834+0000 7f989be00640 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 0x5643588a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.834+0000 7f989be00640 1 -- 172.21.15.86:0/44943419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564358885030 msgr2=0x5643588a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:15.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.834+0000 7f989be00640 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564358885030 0x5643588a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.834+0000 7f989be00640 1 -- 172.21.15.86:0/44943419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643588a59c0 con 0x56435887fec0 2024-04-17T07:32:15.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.835+0000 7f98a1800640 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 0x5643588a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:15.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.837+0000 7f989be00640 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x5643588a43c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9888003ab0 tx=0x7f9888003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16bed25838f8085 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:15.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.837+0000 7f9882a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:15.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.839+0000 7f9882a00640 1 -- 172.21.15.86:0/44943419 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988802a040 con 0x56435887fec0 2024-04-17T07:32:15.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.840+0000 7f9882a00640 1 -- 172.21.15.86:0/44943419 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9888004460 con 0x56435887fec0 2024-04-17T07:32:15.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.843+0000 7f98a65f2740 1 -- 172.21.15.86:0/44943419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9888011d00 con 0x56435887fec0 2024-04-17T07:32:15.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.845+0000 7f9882a00640 1 -- 172.21.15.86:0/44943419 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9888003e90 con 0x56435887fec0 2024-04-17T07:32:15.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.845+0000 7f98a65f2740 10 client.19542 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:15.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.845+0000 7f98a65f2740 1 -- 172.21.15.86:0/44943419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643588a6c70 con 0x56435887fec0 2024-04-17T07:32:15.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.846+0000 7f9882a00640 1 -- 172.21.15.86:0/44943419 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9888011d00 con 0x56435887fec0 2024-04-17T07:32:15.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.847+0000 7f9882a00640 1 -- 172.21.15.86:0/44943419 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f988801a070 con 0x56435887fec0 2024-04-17T07:32:15.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.847+0000 7f98a65f2740 10 client.19542 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:15.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.847+0000 7f98a65f2740 10 client.19542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:15.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.847+0000 7f98a65f2740 20 client.19542 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:15.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.847+0000 7f98a65f2740 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564358911ea0 0x5643589142c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:15.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.847+0000 7f98a65f2740 4 client.19542 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:32:15.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.847+0000 7f98a65f2740 1 -- 172.21.15.86:0/44943419 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643587f29b0 con 0x564358911ea0 2024-04-17T07:32:15.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.849+0000 7f98a1800640 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564358911ea0 0x5643589142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.853+0000 7f9882a00640 10 client.19542 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:15.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.853+0000 7f98a1800640 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564358911ea0 0x5643589142c0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128ae762708e9ea5 server_cookie=f26b83119729a25b in_seq=0 out_seq=0 2024-04-17T07:32:15.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.854+0000 7f9882a00640 1 -- 172.21.15.86:0/44943419 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5643587f29b0 con 0x564358911ea0 2024-04-17T07:32:15.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.854+0000 7f9882a00640 10 client.19542 handle_command_reply: tid=0 2024-04-17T07:32:15.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.854+0000 7f98a65f2740 1 client.19542 shutdown 2024-04-17T07:32:15.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.854+0000 7f98a65f2740 1 -- 172.21.15.86:0/44943419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 msgr2=0x5643588a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.854+0000 7f98a65f2740 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x5643588a43c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9888003ab0 tx=0x7f9888003ae0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.855+0000 7f98a65f2740 1 -- 172.21.15.86:0/44943419 shutdown_connections 2024-04-17T07:32:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.855+0000 7f98a65f2740 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564358911ea0 0x5643589142c0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.855+0000 7f98a65f2740 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564358885030 0x5643588a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.855+0000 7f98a65f2740 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56435887fec0 0x5643588a43c0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.855+0000 7f98a65f2740 1 --2- 172.21.15.86:0/44943419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56435887f4d0 0x5643588a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.855+0000 7f98a65f2740 1 -- 172.21.15.86:0/44943419 >> 172.21.15.86:0/44943419 conn(0x564358887f10 msgr2=0x56435889e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.855+0000 7f9882a00640 0 client.19542 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.855+0000 7f98a65f2740 1 -- 172.21.15.86:0/44943419 shutdown_connections 2024-04-17T07:32:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.855+0000 7f98a65f2740 1 -- 172.21.15.86:0/44943419 wait complete. 2024-04-17T07:32:15.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.856+0000 7f98a65f2740 20 client.19542 trim_cache size 0 max 16384 2024-04-17T07:32:15.856 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:32:15.856 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:32:15.856 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:32:15.856 INFO:teuthology.orchestra.run.smithi086.stdout: "c19b841b-5096-4d67-8c1e-845a83aaf819": { 2024-04-17T07:32:15.856 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:32:15.856 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c19b841b-5096-4d67-8c1e-845a83aaf819", 2024-04-17T07:32:15.856 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:32:15.857 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:32:15.857 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:32:15.857 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:32:15.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.857+0000 7f98a3800640 1 -- 172.21.15.86:0/1767952613 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98840a3d80 msgr2=0x7f98840a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.857+0000 7f98a3800640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98840a3d80 0x7f98840a6240 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f98900bd8b0 tx=0x7f9890046f90 comp rx=0 tx=0).stop 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.857+0000 7f98a3800640 1 -- 172.21.15.86:0/1767952613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f989c1041a0 msgr2=0x7f989c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.857+0000 7f98a3800640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f989c1041a0 0x7f989c19a420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f98900057c0 tx=0x7f98900042f0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.857+0000 7f98a3800640 1 -- 172.21.15.86:0/1767952613 shutdown_connections 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.857+0000 7f98a3800640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f98840a3d80 0x7f98840a6240 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.857+0000 7f98a3800640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f989c105d50 0x7f989c1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.857+0000 7f98a3800640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f989c105390 0x7f989c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.857+0000 7f98a3800640 1 --2- 172.21.15.86:0/1767952613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f989c1041a0 0x7f989c19a420 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.857+0000 7f98a3800640 1 -- 172.21.15.86:0/1767952613 >> 172.21.15.86:0/1767952613 conn(0x7f989c0ff710 msgr2=0x7f989c108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.858+0000 7f98a3800640 1 -- 172.21.15.86:0/1767952613 shutdown_connections 2024-04-17T07:32:15.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:15.858+0000 7f98a3800640 1 -- 172.21.15.86:0/1767952613 wait complete. 2024-04-17T07:32:15.893 INFO:tasks.cephfs.filesystem:scrub status for tag:c19b841b-5096-4d67-8c1e-845a83aaf819 - {'path': '/', 'tag': 'c19b841b-5096-4d67-8c1e-845a83aaf819', 'options': 'recursive,force'} 2024-04-17T07:32:16.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:15 smithi098 ceph-mon[35418]: pgmap v976: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.9 MiB/s wr, 641 op/s 2024-04-17T07:32:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:16 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:32:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:16 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:16 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:32:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:16 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:16.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:16 smithi103 ceph-mon[35495]: pgmap v977: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.3 MiB/s wr, 620 op/s 2024-04-17T07:32:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:16 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:32:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:16 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:16 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:32:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:16 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:16.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:16 smithi086 ceph-mon[31733]: pgmap v977: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.3 MiB/s wr, 620 op/s 2024-04-17T07:32:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:16 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:32:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:16 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:16 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:32:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:16 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:17.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:16 smithi098 ceph-mon[35418]: pgmap v977: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.3 MiB/s wr, 620 op/s 2024-04-17T07:32:19.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:18 smithi103 ceph-mon[35495]: pgmap v978: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.4 MiB/s wr, 643 op/s 2024-04-17T07:32:19.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:18 smithi086 ceph-mon[31733]: pgmap v978: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.4 MiB/s wr, 643 op/s 2024-04-17T07:32:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:18 smithi098 ceph-mon[35418]: pgmap v978: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.4 MiB/s wr, 643 op/s 2024-04-17T07:32:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 775.0 s, 590.2 tps, lat 168.224 ms stddev 87.786 2024-04-17T07:32:20.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:19 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:32:20.082 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:19 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:32:20.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:19 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:32:21.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:20 smithi103 ceph-mon[35495]: pgmap v979: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.6 MiB/s wr, 588 op/s 2024-04-17T07:32:21.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:20 smithi086 ceph-mon[31733]: pgmap v979: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.6 MiB/s wr, 588 op/s 2024-04-17T07:32:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:20 smithi098 ceph-mon[35418]: pgmap v979: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.6 MiB/s wr, 588 op/s 2024-04-17T07:32:23.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:22 smithi086 ceph-mon[31733]: pgmap v980: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.9 MiB/s wr, 615 op/s 2024-04-17T07:32:23.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:22 smithi098 ceph-mon[35418]: pgmap v980: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.9 MiB/s wr, 615 op/s 2024-04-17T07:32:23.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:22 smithi103 ceph-mon[35495]: pgmap v980: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.9 MiB/s wr, 615 op/s 2024-04-17T07:32:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 780.0 s, 609.2 tps, lat 165.863 ms stddev 78.444 2024-04-17T07:32:25.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:24 smithi086 ceph-mon[31733]: pgmap v981: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 8.0 MiB/s wr, 626 op/s 2024-04-17T07:32:25.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:24 smithi098 ceph-mon[35418]: pgmap v981: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 8.0 MiB/s wr, 626 op/s 2024-04-17T07:32:25.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:24 smithi103 ceph-mon[35495]: pgmap v981: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 8.0 MiB/s wr, 626 op/s 2024-04-17T07:32:27.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:26 smithi086 ceph-mon[31733]: pgmap v982: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.9 MiB/s wr, 624 op/s 2024-04-17T07:32:27.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:26 smithi098 ceph-mon[35418]: pgmap v982: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.9 MiB/s wr, 624 op/s 2024-04-17T07:32:27.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:26 smithi103 ceph-mon[35495]: pgmap v982: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.9 MiB/s wr, 624 op/s 2024-04-17T07:32:29.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:28 smithi086 ceph-mon[31733]: pgmap v983: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.8 MiB/s wr, 651 op/s 2024-04-17T07:32:29.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:28 smithi098 ceph-mon[35418]: pgmap v983: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.8 MiB/s wr, 651 op/s 2024-04-17T07:32:29.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:28 smithi103 ceph-mon[35495]: pgmap v983: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 7.8 MiB/s wr, 651 op/s 2024-04-17T07:32:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 785.0 s, 619.8 tps, lat 161.449 ms stddev 69.614 2024-04-17T07:32:31.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:30 smithi086 ceph-mon[31733]: pgmap v984: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.0 MiB/s wr, 649 op/s 2024-04-17T07:32:31.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:30 smithi098 ceph-mon[35418]: pgmap v984: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.0 MiB/s wr, 649 op/s 2024-04-17T07:32:31.278 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:30 smithi103 ceph-mon[35495]: pgmap v984: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.0 MiB/s wr, 649 op/s 2024-04-17T07:32:33.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:32 smithi098 ceph-mon[35418]: pgmap v985: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.3 MiB/s wr, 696 op/s 2024-04-17T07:32:33.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:32 smithi103 ceph-mon[35495]: pgmap v985: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.3 MiB/s wr, 696 op/s 2024-04-17T07:32:33.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:32 smithi086 ceph-mon[31733]: pgmap v985: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.3 MiB/s wr, 696 op/s 2024-04-17T07:32:34.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 790.0 s, 551.6 tps, lat 180.553 ms stddev 99.545 2024-04-17T07:32:35.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:34 smithi098 ceph-mon[35418]: pgmap v986: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.1 MiB/s wr, 675 op/s 2024-04-17T07:32:35.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:34 smithi103 ceph-mon[35495]: pgmap v986: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.1 MiB/s wr, 675 op/s 2024-04-17T07:32:35.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:34 smithi086 ceph-mon[31733]: pgmap v986: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.1 MiB/s wr, 675 op/s 2024-04-17T07:32:37.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:36 smithi098 ceph-mon[35418]: pgmap v987: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.6 MiB/s wr, 654 op/s 2024-04-17T07:32:37.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:36 smithi103 ceph-mon[35495]: pgmap v987: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.6 MiB/s wr, 654 op/s 2024-04-17T07:32:37.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:36 smithi086 ceph-mon[31733]: pgmap v987: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.6 MiB/s wr, 654 op/s 2024-04-17T07:32:39.132 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:38 smithi086 ceph-mon[31733]: pgmap v988: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.1 MiB/s wr, 674 op/s 2024-04-17T07:32:39.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:38 smithi098 ceph-mon[35418]: pgmap v988: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.1 MiB/s wr, 674 op/s 2024-04-17T07:32:39.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:38 smithi103 ceph-mon[35495]: pgmap v988: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.1 MiB/s wr, 674 op/s 2024-04-17T07:32:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 795.0 s, 570.0 tps, lat 175.031 ms stddev 120.561 2024-04-17T07:32:41.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:40 smithi098 ceph-mon[35418]: pgmap v989: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 643 op/s 2024-04-17T07:32:41.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:40 smithi103 ceph-mon[35495]: pgmap v989: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 643 op/s 2024-04-17T07:32:41.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:40 smithi086 ceph-mon[31733]: pgmap v989: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 643 op/s 2024-04-17T07:32:43.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:42 smithi098 ceph-mon[35418]: pgmap v990: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.4 MiB/s wr, 668 op/s 2024-04-17T07:32:43.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:42 smithi103 ceph-mon[35495]: pgmap v990: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.4 MiB/s wr, 668 op/s 2024-04-17T07:32:43.332 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:42 smithi086 ceph-mon[31733]: pgmap v990: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.4 MiB/s wr, 668 op/s 2024-04-17T07:32:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 800.0 s, 587.6 tps, lat 171.487 ms stddev 225.833 2024-04-17T07:32:45.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:44 smithi098 ceph-mon[35418]: pgmap v991: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.6 MiB/s wr, 626 op/s 2024-04-17T07:32:45.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:44 smithi103 ceph-mon[35495]: pgmap v991: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.6 MiB/s wr, 626 op/s 2024-04-17T07:32:45.335 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:44 smithi086 ceph-mon[31733]: pgmap v991: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.6 MiB/s wr, 626 op/s 2024-04-17T07:32:45.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:32:45.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:32:46.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.262+0000 7f9725a00640 1 -- 172.21.15.86:0/1772814897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9720100740 msgr2=0x7f9720100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.262+0000 7f9725a00640 1 --2- 172.21.15.86:0/1772814897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9720100740 0x7f9720100b60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9718009860 tx=0x7f971802bb70 comp rx=0 tx=0).stop 2024-04-17T07:32:46.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.262+0000 7f9725a00640 1 -- 172.21.15.86:0/1772814897 shutdown_connections 2024-04-17T07:32:46.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.262+0000 7f9725a00640 1 --2- 172.21.15.86:0/1772814897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9720101a40 0x7f9720101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.262+0000 7f9725a00640 1 --2- 172.21.15.86:0/1772814897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97201010a0 0x7f9720101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.262+0000 7f9725a00640 1 --2- 172.21.15.86:0/1772814897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9720100740 0x7f9720100b60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.262+0000 7f9725a00640 1 -- 172.21.15.86:0/1772814897 >> 172.21.15.86:0/1772814897 conn(0x7f97200fc300 msgr2=0x7f97200fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:46.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.263+0000 7f9725a00640 1 -- 172.21.15.86:0/1772814897 shutdown_connections 2024-04-17T07:32:46.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.263+0000 7f9725a00640 1 -- 172.21.15.86:0/1772814897 wait complete. 2024-04-17T07:32:46.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.265+0000 7f9725a00640 1 Processor -- start 2024-04-17T07:32:46.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.265+0000 7f9725a00640 1 -- start start 2024-04-17T07:32:46.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.266+0000 7f9725a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9720100740 0x7f9720105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.266+0000 7f9725a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97201010a0 0x7f9720105580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.266+0000 7f9725a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9720101a40 0x7f972010c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.266+0000 7f9725a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f972010fbc0 con 0x7f97201010a0 2024-04-17T07:32:46.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.266+0000 7f9725a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f972010fa40 con 0x7f9720101a40 2024-04-17T07:32:46.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.266+0000 7f9725a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f972010fd40 con 0x7f9720100740 2024-04-17T07:32:46.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.266+0000 7f971fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9720100740 0x7f9720105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.266+0000 7f971fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9720100740 0x7f9720105040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:60396/0 (socket says 172.21.15.86:60396) 2024-04-17T07:32:46.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.266+0000 7f971fe00640 1 -- 172.21.15.86:0/323416182 learned_addr learned my addr 172.21.15.86:0/323416182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:46.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.267+0000 7f9725000640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9720101a40 0x7f972010c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.267+0000 7f971fe00640 1 -- 172.21.15.86:0/323416182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9720101a40 msgr2=0x7f972010c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.267+0000 7f971fe00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9720101a40 0x7f972010c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.267+0000 7f971fe00640 1 -- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97201010a0 msgr2=0x7f9720105580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.267+0000 7f971f400640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97201010a0 0x7f9720105580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.267+0000 7f971fe00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97201010a0 0x7f9720105580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.267+0000 7f971fe00640 1 -- 172.21.15.86:0/323416182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972006b980 con 0x7f9720100740 2024-04-17T07:32:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.267+0000 7f971f400640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97201010a0 0x7f9720105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:32:46.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.268+0000 7f971fe00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9720100740 0x7f9720105040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f971802c180 tx=0x7f9718004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:46.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.268+0000 7f971cc00640 1 -- 172.21.15.86:0/323416182 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718039040 con 0x7f9720100740 2024-04-17T07:32:46.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.269+0000 7f971cc00640 1 -- 172.21.15.86:0/323416182 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9718042070 con 0x7f9720100740 2024-04-17T07:32:46.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.269+0000 7f971cc00640 1 -- 172.21.15.86:0/323416182 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718004b60 con 0x7f9720100740 2024-04-17T07:32:46.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.269+0000 7f9725a00640 1 -- 172.21.15.86:0/323416182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972006bc10 con 0x7f9720100740 2024-04-17T07:32:46.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.269+0000 7f9725a00640 1 -- 172.21.15.86:0/323416182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972006c0b0 con 0x7f9720100740 2024-04-17T07:32:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.277+0000 7f9729d9c740 1 Processor -- start 2024-04-17T07:32:46.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.277+0000 7f9729d9c740 1 -- start start 2024-04-17T07:32:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.278+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868131790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.278+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131d60 0x55d868136860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.278+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868139360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.278+0000 7f9729d9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d868098120 con 0x55d868131d60 2024-04-17T07:32:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.278+0000 7f9729d9c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d86803e340 con 0x55d868131370 2024-04-17T07:32:46.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.278+0000 7f9729d9c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d8680940e0 con 0x55d868136ed0 2024-04-17T07:32:46.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.279+0000 7f9725000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.279+0000 7f9725000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868131790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52228/0 (socket says 172.21.15.86:52228) 2024-04-17T07:32:46.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.279+0000 7f9725000640 1 -- 172.21.15.86:0/3864107002 learned_addr learned my addr 172.21.15.86:0/3864107002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:46.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.279+0000 7f971cc00640 1 -- 172.21.15.86:0/323416182 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9718004d00 con 0x7f9720100740 2024-04-17T07:32:46.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.279+0000 7f971f400640 1 --2- 172.21.15.86:0/3864107002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868139360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.280+0000 7f9725000640 1 -- 172.21.15.86:0/3864107002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 msgr2=0x55d868139360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:46.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.280+0000 7f9725000640 1 --2- 172.21.15.86:0/3864107002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868139360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.280+0000 7f9725000640 1 -- 172.21.15.86:0/3864107002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131d60 msgr2=0x55d868136860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:46.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.280+0000 7f9725000640 1 --2- 172.21.15.86:0/3864107002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131d60 0x55d868136860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.280+0000 7f9725000640 1 -- 172.21.15.86:0/3864107002 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d867e747d0 con 0x55d868131370 2024-04-17T07:32:46.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.280+0000 7f971f400640 1 --2- 172.21.15.86:0/3864107002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868139360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:32:46.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.281+0000 7f9725000640 1 --2- 172.21.15.86:0/3864107002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868131790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9710009ee0 tx=0x7f9710010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f9f71b8bb84296 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.281+0000 7f971cc00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97040a3c30 0x7f97040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.282+0000 7f971cc00640 1 -- 172.21.15.86:0/323416182 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f97180f2220 con 0x7f9720100740 2024-04-17T07:32:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.282+0000 7f9715600640 1 -- 172.21.15.86:0/3864107002 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710016070 con 0x55d868131370 2024-04-17T07:32:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.282+0000 7f9715600640 1 -- 172.21.15.86:0/3864107002 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9710004d20 con 0x55d868131370 2024-04-17T07:32:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.282+0000 7f9715600640 1 -- 172.21.15.86:0/3864107002 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710016070 con 0x55d868131370 2024-04-17T07:32:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.283+0000 7f971fe00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97040a3c30 0x7f97040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.285+0000 7f9729d9c740 1 -- 172.21.15.86:0/3864107002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 msgr2=0x55d868131790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.285+0000 7f9729d9c740 1 --2- 172.21.15.86:0/3864107002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868131790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9710009ee0 tx=0x7f9710010740 comp rx=0 tx=0).stop 2024-04-17T07:32:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.285+0000 7f971fe00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97040a3c30 0x7f97040a60f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f97180be110 tx=0x7f971802bb70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.285+0000 7f9729d9c740 1 -- 172.21.15.86:0/3864107002 shutdown_connections 2024-04-17T07:32:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.285+0000 7f9729d9c740 1 --2- 172.21.15.86:0/3864107002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868139360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.285+0000 7f9729d9c740 1 --2- 172.21.15.86:0/3864107002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131d60 0x55d868136860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.285+0000 7f9729d9c740 1 --2- 172.21.15.86:0/3864107002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868131790 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.285+0000 7f9729d9c740 1 -- 172.21.15.86:0/3864107002 >> 172.21.15.86:0/3864107002 conn(0x55d86812c8e0 msgr2=0x55d86812cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.286+0000 7f9729d9c740 1 -- 172.21.15.86:0/3864107002 shutdown_connections 2024-04-17T07:32:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.286+0000 7f9729d9c740 1 -- 172.21.15.86:0/3864107002 wait complete. 2024-04-17T07:32:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.287+0000 7f9729d9c740 1 -- start start 2024-04-17T07:32:46.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.287+0000 7f9729d9c740 1 Processor -- start 2024-04-17T07:32:46.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.288+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.288+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d86813be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.288+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868142e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.288+0000 7f9729d9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d868139e80 con 0x55d868131370 2024-04-17T07:32:46.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.288+0000 7f9729d9c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d868051710 con 0x55d868131d60 2024-04-17T07:32:46.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.288+0000 7f9729d9c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d86813a000 con 0x55d868136ed0 2024-04-17T07:32:46.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.288+0000 7f9729d9c740 10 client.0 _ll_register_callbacks cb 0x55d86804caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:46.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.289+0000 7f9725000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.289+0000 7f971f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d86813be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.289+0000 7f971f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d86813be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52244/0 (socket says 172.21.15.86:52244) 2024-04-17T07:32:46.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.289+0000 7f9725000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59894/0 (socket says 172.21.15.86:59894) 2024-04-17T07:32:46.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.289+0000 7f971f400640 1 -- 172.21.15.86:0/375874146 learned_addr learned my addr 172.21.15.86:0/375874146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:46.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.290+0000 7f9725000640 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.290+0000 7f971f400640 1 -- 172.21.15.86:0/375874146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 msgr2=0x55d868142e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.290+0000 7f971f400640 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868142e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.291+0000 7f971f400640 1 -- 172.21.15.86:0/375874146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 msgr2=0x55d86813b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.291+0000 7f971f400640 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.291+0000 7f971f400640 1 -- 172.21.15.86:0/375874146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d867daa3f0 con 0x55d868131d60 2024-04-17T07:32:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.291+0000 7f971f400640 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d86813be30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9708010650 tx=0x7f9708010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d1bbc001b3b0658 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:46.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.291+0000 7f970e000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.294+0000 7f970e000640 1 -- 172.21.15.86:0/375874146 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708017070 con 0x55d868131d60 2024-04-17T07:32:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.294+0000 7f970e000640 1 -- 172.21.15.86:0/375874146 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9708004220 con 0x55d868131d60 2024-04-17T07:32:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.294+0000 7f9729d9c740 1 -- 172.21.15.86:0/375874146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9710004d20 con 0x55d868131d60 2024-04-17T07:32:46.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.294+0000 7f970e000640 1 -- 172.21.15.86:0/375874146 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9708010e50 con 0x55d868131d60 2024-04-17T07:32:46.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.295+0000 7f9729d9c740 10 client.38576 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:46.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.295+0000 7f9729d9c740 1 -- 172.21.15.86:0/375874146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d867d6a0f0 con 0x55d868131d60 2024-04-17T07:32:46.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.295+0000 7f970e000640 1 -- 172.21.15.86:0/375874146 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9710004d20 con 0x55d868131d60 2024-04-17T07:32:46.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.296+0000 7f970e000640 1 -- 172.21.15.86:0/375874146 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9708003a70 con 0x55d868131d60 2024-04-17T07:32:46.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.296+0000 7f9729d9c740 10 client.38576 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:46.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.296+0000 7f9729d9c740 10 client.38576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:46.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.296+0000 7f9729d9c740 20 client.38576 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:46.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.296+0000 7f9729d9c740 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d868147710 0x55d868149b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.296+0000 7f9729d9c740 4 client.38576 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:46.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.296+0000 7f9729d9c740 1 -- 172.21.15.86:0/375874146 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8680a5b40 con 0x55d868147710 2024-04-17T07:32:46.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.300+0000 7f9725000640 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d868147710 0x55d868149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.301+0000 7f970e000640 10 client.38576 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:46.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.301+0000 7f9725000640 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d868147710 0x55d868149b90 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41bc61786655deb6 server_cookie=a9dc4902530bdc00 in_seq=0 out_seq=0 2024-04-17T07:32:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.302+0000 7f970e000640 1 -- 172.21.15.86:0/375874146 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8680a5b40 con 0x55d868147710 2024-04-17T07:32:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.302+0000 7f970e000640 10 client.38576 handle_command_reply: tid=0 2024-04-17T07:32:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.303+0000 7f9729d9c740 1 client.38576 shutdown 2024-04-17T07:32:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.303+0000 7f9729d9c740 1 -- 172.21.15.86:0/375874146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 msgr2=0x55d86813be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.303+0000 7f9729d9c740 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d86813be30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9708010650 tx=0x7f9708010b10 comp rx=0 tx=0).stop 2024-04-17T07:32:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.304+0000 7f9729d9c740 1 -- 172.21.15.86:0/375874146 shutdown_connections 2024-04-17T07:32:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.304+0000 7f9729d9c740 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d868147710 0x55d868149b90 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.304+0000 7f9729d9c740 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868142e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.304+0000 7f9729d9c740 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d86813be30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.304+0000 7f9729d9c740 1 --2- 172.21.15.86:0/375874146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.304+0000 7f9729d9c740 1 -- 172.21.15.86:0/375874146 >> 172.21.15.86:0/375874146 conn(0x55d86812c8e0 msgr2=0x55d86812bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.304+0000 7f970e000640 0 client.38576 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.305+0000 7f9729d9c740 1 -- 172.21.15.86:0/375874146 shutdown_connections 2024-04-17T07:32:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.305+0000 7f9729d9c740 1 -- 172.21.15.86:0/375874146 wait complete. 2024-04-17T07:32:46.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.305+0000 7f9729d9c740 20 client.38576 trim_cache size 0 max 16384 2024-04-17T07:32:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.360+0000 7f9729d9c740 1 Processor -- start 2024-04-17T07:32:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.361+0000 7f9729d9c740 1 -- start start 2024-04-17T07:32:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.361+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.361+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d868142e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.361+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d86812b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.361+0000 7f9729d9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97201c1c60 con 0x55d868131370 2024-04-17T07:32:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.361+0000 7f9729d9c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f97201c37a0 con 0x55d868131d60 2024-04-17T07:32:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.361+0000 7f9729d9c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97201c2330 con 0x55d868136ed0 2024-04-17T07:32:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.361+0000 7f9725000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.362+0000 7f9725000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59906/0 (socket says 172.21.15.86:59906) 2024-04-17T07:32:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.362+0000 7f9725000640 1 -- 172.21.15.86:0/1521051168 learned_addr learned my addr 172.21.15.86:0/1521051168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.362+0000 7f971f400640 1 --2- 172.21.15.86:0/1521051168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d868142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.362+0000 7f9725000640 1 --2- 172.21.15.86:0/1521051168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d86812b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.362+0000 7f971f400640 1 -- 172.21.15.86:0/1521051168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 msgr2=0x55d86812b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.362+0000 7f971f400640 1 --2- 172.21.15.86:0/1521051168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d86812b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.362+0000 7f971f400640 1 -- 172.21.15.86:0/1521051168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 msgr2=0x55d86813a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.362+0000 7f971f400640 1 --2- 172.21.15.86:0/1521051168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.362+0000 7f971f400640 1 -- 172.21.15.86:0/1521051168 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8681325f0 con 0x55d868131d60 2024-04-17T07:32:46.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.362+0000 7f9725000640 1 --2- 172.21.15.86:0/1521051168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.363+0000 7f971f400640 1 --2- 172.21.15.86:0/1521051168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d868142e30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9708011580 tx=0x7f9708009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b0b6ca8aa1dd9d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.363+0000 7f9714c00640 1 -- 172.21.15.86:0/1521051168 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970802d020 con 0x55d868131d60 2024-04-17T07:32:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.364+0000 7f9714c00640 1 -- 172.21.15.86:0/1521051168 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f970800e930 con 0x55d868131d60 2024-04-17T07:32:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.364+0000 7f9714c00640 1 -- 172.21.15.86:0/1521051168 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708017040 con 0x55d868131d60 2024-04-17T07:32:46.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.365+0000 7f9729d9c740 1 -- 172.21.15.86:0/1521051168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 msgr2=0x55d868142e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.365+0000 7f9729d9c740 1 --2- 172.21.15.86:0/1521051168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d868142e30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9708011580 tx=0x7f9708009460 comp rx=0 tx=0).stop 2024-04-17T07:32:46.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.365+0000 7f9729d9c740 1 -- 172.21.15.86:0/1521051168 shutdown_connections 2024-04-17T07:32:46.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.365+0000 7f9729d9c740 1 --2- 172.21.15.86:0/1521051168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d86812b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.365+0000 7f9729d9c740 1 --2- 172.21.15.86:0/1521051168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131d60 0x55d868142e30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.365+0000 7f9729d9c740 1 --2- 172.21.15.86:0/1521051168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131370 0x55d86813a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.365+0000 7f9729d9c740 1 -- 172.21.15.86:0/1521051168 >> 172.21.15.86:0/1521051168 conn(0x55d86812c8e0 msgr2=0x55d8681332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.367+0000 7f9729d9c740 1 -- 172.21.15.86:0/1521051168 shutdown_connections 2024-04-17T07:32:46.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.367+0000 7f9729d9c740 1 -- 172.21.15.86:0/1521051168 wait complete. 2024-04-17T07:32:46.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.368+0000 7f9729d9c740 1 -- start start 2024-04-17T07:32:46.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.368+0000 7f9729d9c740 1 Processor -- start 2024-04-17T07:32:46.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.369+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868140980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.369+0000 7f9729d9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131d60 0x55d868140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.370+0000 7f9725000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.370+0000 7f9725000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868140980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52272/0 (socket says 172.21.15.86:52272) 2024-04-17T07:32:46.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.370+0000 7f9725000640 1 -- 172.21.15.86:0/774298799 learned_addr learned my addr 172.21.15.86:0/774298799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:46.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.370+0000 7f9729d9c740 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.370+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97201c30d0 con 0x55d868131d60 2024-04-17T07:32:46.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.370+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f97201c2a00 con 0x55d868131370 2024-04-17T07:32:46.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.370+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97201bfc30 con 0x55d868136ed0 2024-04-17T07:32:46.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.370+0000 7f9729d9c740 10 client.0 _ll_register_callbacks cb 0x7f9708009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:46.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.371+0000 7f971f400640 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131d60 0x55d868140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.371+0000 7f9725000640 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.374+0000 7f9725000640 1 -- 172.21.15.86:0/774298799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 msgr2=0x55d868141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:46.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.374+0000 7f9725000640 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.374+0000 7f9725000640 1 -- 172.21.15.86:0/774298799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131d60 msgr2=0x55d868140ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:46.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.374+0000 7f9725000640 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131d60 0x55d868140ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.374+0000 7f9725000640 1 -- 172.21.15.86:0/774298799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8681424c0 con 0x55d868131370 2024-04-17T07:32:46.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.375+0000 7f9725000640 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868140980 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f97100099f0 tx=0x7f9710005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4f4770d47ae100 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:46.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.375+0000 7f970e000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:46.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.375+0000 7f970e000640 1 -- 172.21.15.86:0/774298799 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710022040 con 0x55d868131370 2024-04-17T07:32:46.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.375+0000 7f970e000640 1 -- 172.21.15.86:0/774298799 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f971002b330 con 0x55d868131370 2024-04-17T07:32:46.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.375+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970800e930 con 0x55d868131370 2024-04-17T07:32:46.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.379+0000 7f970e000640 1 -- 172.21.15.86:0/774298799 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9710016040 con 0x55d868131370 2024-04-17T07:32:46.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.379+0000 7f9729d9c740 10 client.38582 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:46.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.379+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8681ae100 con 0x55d868131370 2024-04-17T07:32:46.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.380+0000 7f970e000640 1 -- 172.21.15.86:0/774298799 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f970800e930 con 0x55d868131370 2024-04-17T07:32:46.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.380+0000 7f970e000640 1 -- 172.21.15.86:0/774298799 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9710037070 con 0x55d868131370 2024-04-17T07:32:46.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.380+0000 7f9729d9c740 10 client.38582 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:46.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.380+0000 7f9729d9c740 10 client.38582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:46.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.380+0000 7f9729d9c740 20 client.38582 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:46.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.380+0000 7f9729d9c740 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d8681ae740 0x55d8681b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.380+0000 7f9729d9c740 4 client.38582 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:32:46.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.380+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d86809f9b0 con 0x55d8681ae740 2024-04-17T07:32:46.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.382+0000 7f971f400640 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d8681ae740 0x55d8681b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.384+0000 7f970e000640 10 client.38582 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:46.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.384+0000 7f971f400640 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d8681ae740 0x55d8681b0b60 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68267c57cced7fd server_cookie=706ce328b7b91ec5 in_seq=0 out_seq=0 2024-04-17T07:32:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.385+0000 7f970e000640 1 -- 172.21.15.86:0/774298799 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d86809f9b0 con 0x55d8681ae740 2024-04-17T07:32:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.385+0000 7f970e000640 10 client.38582 handle_command_reply: tid=0 2024-04-17T07:32:46.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.385+0000 7f9729d9c740 1 client.38582 shutdown 2024-04-17T07:32:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.386+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 msgr2=0x55d868140980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.386+0000 7f9729d9c740 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868140980 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f97100099f0 tx=0x7f9710005840 comp rx=0 tx=0).stop 2024-04-17T07:32:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.386+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 shutdown_connections 2024-04-17T07:32:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.386+0000 7f9729d9c740 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55d8681ae740 0x55d8681b0b60 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.386+0000 7f9729d9c740 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d868136ed0 0x55d868141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.386+0000 7f9729d9c740 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d868131d60 0x55d868140ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.386+0000 7f9729d9c740 1 --2- 172.21.15.86:0/774298799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d868131370 0x55d868140980 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.386+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 >> 172.21.15.86:0/774298799 conn(0x55d86812c8e0 msgr2=0x55d86812fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.386+0000 7f970e000640 0 client.38582 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.386+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 shutdown_connections 2024-04-17T07:32:46.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.387+0000 7f9729d9c740 1 -- 172.21.15.86:0/774298799 wait complete. 2024-04-17T07:32:46.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.387+0000 7f9729d9c740 20 client.38582 trim_cache size 0 max 16384 2024-04-17T07:32:46.388 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:32:46.388 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:32:46.388 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:32:46.388 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:32:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.389+0000 7f9725a00640 1 -- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97040a3c30 msgr2=0x7f97040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.389+0000 7f9725a00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97040a3c30 0x7f97040a60f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f97180be110 tx=0x7f971802bb70 comp rx=0 tx=0).stop 2024-04-17T07:32:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.389+0000 7f9725a00640 1 -- 172.21.15.86:0/323416182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9720100740 msgr2=0x7f9720105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.389+0000 7f9725a00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9720100740 0x7f9720105040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f971802c180 tx=0x7f9718004430 comp rx=0 tx=0).stop 2024-04-17T07:32:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.389+0000 7f9725a00640 1 -- 172.21.15.86:0/323416182 shutdown_connections 2024-04-17T07:32:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.389+0000 7f9725a00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f97040a3c30 0x7f97040a60f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.389+0000 7f9725a00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9720101a40 0x7f972010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.389+0000 7f9725a00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97201010a0 0x7f9720105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.389+0000 7f9725a00640 1 --2- 172.21.15.86:0/323416182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9720100740 0x7f9720105040 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.389+0000 7f9725a00640 1 -- 172.21.15.86:0/323416182 >> 172.21.15.86:0/323416182 conn(0x7f97200fc300 msgr2=0x7f97200fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.390+0000 7f9725a00640 1 -- 172.21.15.86:0/323416182 shutdown_connections 2024-04-17T07:32:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.390+0000 7f9725a00640 1 -- 172.21.15.86:0/323416182 wait complete. 2024-04-17T07:32:46.427 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:32:46.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:32:46.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.735+0000 7f2456a00640 1 -- 172.21.15.86:0/440189625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24501085c0 msgr2=0x7f24501089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.735+0000 7f2456a00640 1 --2- 172.21.15.86:0/440189625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24501085c0 0x7f24501089e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2440009900 tx=0x7f244002bc60 comp rx=0 tx=0).stop 2024-04-17T07:32:46.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.735+0000 7f2456a00640 1 -- 172.21.15.86:0/440189625 shutdown_connections 2024-04-17T07:32:46.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.735+0000 7f2456a00640 1 --2- 172.21.15.86:0/440189625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245010a170 0x7f2450110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.735+0000 7f2456a00640 1 --2- 172.21.15.86:0/440189625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24501097b0 0x7f2450109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.735+0000 7f2456a00640 1 --2- 172.21.15.86:0/440189625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24501085c0 0x7f24501089e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.735+0000 7f2456a00640 1 -- 172.21.15.86:0/440189625 >> 172.21.15.86:0/440189625 conn(0x7f2450077580 msgr2=0x7f24500799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.735+0000 7f2456a00640 1 -- 172.21.15.86:0/440189625 shutdown_connections 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.735+0000 7f2456a00640 1 -- 172.21.15.86:0/440189625 wait complete. 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f2456a00640 1 Processor -- start 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f2456a00640 1 -- start start 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f2456a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24501085c0 0x7f245007f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f2456a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24501097b0 0x7f245007fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f2456a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245010a170 0x7f24501ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f2456a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2450114130 con 0x7f24501097b0 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f2456a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2450113fb0 con 0x7f245010a170 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f2456a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24501142b0 con 0x7f24501085c0 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f244f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24501097b0 0x7f245007fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f244f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24501097b0 0x7f245007fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59930/0 (socket says 172.21.15.86:59930) 2024-04-17T07:32:46.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f244f400640 1 -- 172.21.15.86:0/1866125993 learned_addr learned my addr 172.21.15.86:0/1866125993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:46.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.737+0000 7f2454a00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245010a170 0x7f24501ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.738+0000 7f244fe00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24501085c0 0x7f245007f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.738+0000 7f2454a00640 1 -- 172.21.15.86:0/1866125993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24501085c0 msgr2=0x7f245007f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.738+0000 7f2454a00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24501085c0 0x7f245007f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.738+0000 7f2454a00640 1 -- 172.21.15.86:0/1866125993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24501097b0 msgr2=0x7f245007fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:46.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.738+0000 7f2454a00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24501097b0 0x7f245007fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:46.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.738+0000 7f2454a00640 1 -- 172.21.15.86:0/1866125993 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24501aed80 con 0x7f245010a170 2024-04-17T07:32:46.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.738+0000 7f2454a00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245010a170 0x7f24501ae840 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2448009fd0 tx=0x7f244800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:46.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.739+0000 7f244cc00640 1 -- 172.21.15.86:0/1866125993 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244800ce20 con 0x7f245010a170 2024-04-17T07:32:46.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.739+0000 7f244cc00640 1 -- 172.21.15.86:0/1866125993 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2448005020 con 0x7f245010a170 2024-04-17T07:32:46.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.739+0000 7f244cc00640 1 -- 172.21.15.86:0/1866125993 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448010690 con 0x7f245010a170 2024-04-17T07:32:46.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.739+0000 7f2456a00640 1 -- 172.21.15.86:0/1866125993 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24501aef50 con 0x7f245010a170 2024-04-17T07:32:46.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.740+0000 7f2456a00640 1 -- 172.21.15.86:0/1866125993 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24501af3c0 con 0x7f245010a170 2024-04-17T07:32:46.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.741+0000 7f2456a00640 1 -- 172.21.15.86:0/1866125993 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245010f950 con 0x7f245010a170 2024-04-17T07:32:46.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.744+0000 7f244cc00640 1 -- 172.21.15.86:0/1866125993 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f24480051d0 con 0x7f245010a170 2024-04-17T07:32:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.746+0000 7f244cc00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24340a3cb0 0x7f24340a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.746+0000 7f244cc00640 1 -- 172.21.15.86:0/1866125993 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f24480d0e20 con 0x7f245010a170 2024-04-17T07:32:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.748+0000 7f244fe00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24340a3cb0 0x7f24340a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:46.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.752+0000 7f244fe00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24340a3cb0 0x7f24340a6170 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2440000c00 tx=0x7f2440004bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:46.761+0000 7f244cc00640 1 -- 172.21.15.86:0/1866125993 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f244809d420 con 0x7f245010a170 2024-04-17T07:32:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:46 smithi098 ceph-mon[35418]: pgmap v992: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.6 MiB/s wr, 605 op/s 2024-04-17T07:32:47.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:46 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:32:47.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:46 smithi103 ceph-mon[35495]: pgmap v992: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.6 MiB/s wr, 605 op/s 2024-04-17T07:32:47.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:46 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:32:47.334 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:46 smithi086 ceph-mon[31733]: pgmap v992: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.6 MiB/s wr, 605 op/s 2024-04-17T07:32:47.334 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:46 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:32:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.345+0000 7f2456a00640 1 -- 172.21.15.86:0/1866125993 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2450062b60 con 0x7f245010a170 2024-04-17T07:32:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.346+0000 7f244cc00640 1 -- 172.21.15.86:0/1866125993 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f244809cd20 con 0x7f245010a170 2024-04-17T07:32:47.347 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:32:47.347 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:32:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:32:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.352+0000 7f2456a00640 1 -- 172.21.15.86:0/1866125993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24340a3cb0 msgr2=0x7f24340a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.352+0000 7f2456a00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24340a3cb0 0x7f24340a6170 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2440000c00 tx=0x7f2440004bb0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.352+0000 7f2456a00640 1 -- 172.21.15.86:0/1866125993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245010a170 msgr2=0x7f24501ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.352+0000 7f2456a00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245010a170 0x7f24501ae840 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2448009fd0 tx=0x7f244800ef10 comp rx=0 tx=0).stop 2024-04-17T07:32:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.352+0000 7f2456a00640 1 -- 172.21.15.86:0/1866125993 shutdown_connections 2024-04-17T07:32:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.352+0000 7f2456a00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f24340a3cb0 0x7f24340a6170 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.352+0000 7f2456a00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f245010a170 0x7f24501ae840 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.352+0000 7f2456a00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24501097b0 0x7f245007fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.352+0000 7f2456a00640 1 --2- 172.21.15.86:0/1866125993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24501085c0 0x7f245007f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.352+0000 7f2456a00640 1 -- 172.21.15.86:0/1866125993 >> 172.21.15.86:0/1866125993 conn(0x7f2450077580 msgr2=0x7f245010cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:47.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.355+0000 7f2456a00640 1 -- 172.21.15.86:0/1866125993 shutdown_connections 2024-04-17T07:32:47.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.357+0000 7f2456a00640 1 -- 172.21.15.86:0/1866125993 wait complete. 2024-04-17T07:32:47.387 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:32:47.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.778+0000 7f21cc000640 1 -- 172.21.15.86:0/1865594348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21c4100820 msgr2=0x7f21c4102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.778+0000 7f21cc000640 1 --2- 172.21.15.86:0/1865594348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21c4100820 0x7f21c4102c50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009ca0 tx=0x7f21bc02c140 comp rx=0 tx=0).stop 2024-04-17T07:32:47.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.778+0000 7f21cc000640 1 -- 172.21.15.86:0/1865594348 shutdown_connections 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.778+0000 7f21cc000640 1 --2- 172.21.15.86:0/1865594348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c4105b40 0x7f21c4105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.778+0000 7f21cc000640 1 --2- 172.21.15.86:0/1865594348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c4103190 0x7f21c4105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.778+0000 7f21cc000640 1 --2- 172.21.15.86:0/1865594348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21c4100820 0x7f21c4102c50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.778+0000 7f21cc000640 1 -- 172.21.15.86:0/1865594348 >> 172.21.15.86:0/1865594348 conn(0x7f21c40fc420 msgr2=0x7f21c40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.778+0000 7f21cc000640 1 -- 172.21.15.86:0/1865594348 shutdown_connections 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.778+0000 7f21cc000640 1 -- 172.21.15.86:0/1865594348 wait complete. 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.779+0000 7f21cc000640 1 Processor -- start 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.780+0000 7f21cc000640 1 -- start start 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.780+0000 7f21cc000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c4100820 0x7f21c419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.780+0000 7f21cc000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21c4103190 0x7f21c419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.780+0000 7f21cc000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c4105b40 0x7f21c41a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.780+0000 7f21c9600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c4100820 0x7f21c419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.780+0000 7f21c9600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c4100820 0x7f21c419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40668/0 (socket says 172.21.15.86:40668) 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.780+0000 7f21c8c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21c4103190 0x7f21c419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.781+0000 7f21c8c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21c4103190 0x7f21c419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36666/0 (socket says 172.21.15.86:36666) 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.781+0000 7f21ca000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c4105b40 0x7f21c41a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.781+0000 7f21c9600640 1 -- 172.21.15.86:0/4068541565 learned_addr learned my addr 172.21.15.86:0/4068541565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.782+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21c4114130 con 0x7f21c4105b40 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.782+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f21c4113fb0 con 0x7f21c4103190 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.782+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21c41142b0 con 0x7f21c4100820 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.782+0000 7f21c8c00640 1 -- 172.21.15.86:0/4068541565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c4100820 msgr2=0x7f21c419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.782+0000 7f21c8c00640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c4100820 0x7f21c419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.783+0000 7f21c8c00640 1 -- 172.21.15.86:0/4068541565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c4105b40 msgr2=0x7f21c41a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.783+0000 7f21c8c00640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c4105b40 0x7f21c41a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.783+0000 7f21c8c00640 1 -- 172.21.15.86:0/4068541565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c41a64d0 con 0x7f21c4103190 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.783+0000 7f21c8c00640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21c4103190 0x7f21c419edf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f21b800b3f0 tx=0x7f21b800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.784+0000 7f21c2000640 1 -- 172.21.15.86:0/4068541565 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8015020 con 0x7f21c4103190 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.784+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c41a67c0 con 0x7f21c4103190 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.785+0000 7f21c2000640 1 -- 172.21.15.86:0/4068541565 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21b8011ce0 con 0x7f21c4103190 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.785+0000 7f21c2000640 1 -- 172.21.15.86:0/4068541565 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8010d80 con 0x7f21c4103190 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.785+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c41a6de0 con 0x7f21c4103190 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.790+0000 7f21cedab740 1 Processor -- start 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.790+0000 7f21cedab740 1 -- start start 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.791+0000 7f21cedab740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 0x55ae5436f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.791+0000 7f21cedab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436f990 0x55ae54374490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.792+0000 7f21ca000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 0x55ae5436f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.792+0000 7f21ca000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 0x55ae5436f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36668/0 (socket says 172.21.15.86:36668) 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.792+0000 7f21cedab740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae54374b00 0x55ae54376f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.792+0000 7f21ca000640 1 -- 172.21.15.86:0/1916998010 learned_addr learned my addr 172.21.15.86:0/1916998010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.792+0000 7f21cedab740 1 -- 172.21.15.86:0/1916998010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae542d5020 con 0x55ae5436f990 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.792+0000 7f21cedab740 1 -- 172.21.15.86:0/1916998010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ae5427b460 con 0x55ae5436efa0 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.792+0000 7f21cedab740 1 -- 172.21.15.86:0/1916998010 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae542d0fe0 con 0x55ae54374b00 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.793+0000 7f21c9600640 1 --2- 172.21.15.86:0/1916998010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436f990 0x55ae54374490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.793+0000 7f21c2000640 1 -- 172.21.15.86:0/4068541565 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f21b8018020 con 0x7f21c4103190 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.793+0000 7f21ca000640 1 -- 172.21.15.86:0/1916998010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae54374b00 msgr2=0x55ae54376f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.793+0000 7f21ca000640 1 --2- 172.21.15.86:0/1916998010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae54374b00 0x55ae54376f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.793+0000 7f21ca000640 1 -- 172.21.15.86:0/1916998010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436f990 msgr2=0x55ae54374490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.793+0000 7f21ca000640 1 --2- 172.21.15.86:0/1916998010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436f990 0x55ae54374490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.793+0000 7f21ca000640 1 -- 172.21.15.86:0/1916998010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae54283040 con 0x55ae5436efa0 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.795+0000 7f21ca000640 1 --2- 172.21.15.86:0/1916998010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 0x55ae5436f3c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009ca0 tx=0x7f21b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f738d0c88946c32e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.795+0000 7f21b6600640 1 -- 172.21.15.86:0/1916998010 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b001e070 con 0x55ae5436efa0 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.795+0000 7f21b6600640 1 -- 172.21.15.86:0/1916998010 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21b0009e50 con 0x55ae5436efa0 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.795+0000 7f21b6600640 1 -- 172.21.15.86:0/1916998010 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b000ea10 con 0x55ae5436efa0 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.796+0000 7f21c2000640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f21ac0a3c30 0x7f21ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.796+0000 7f21c2000640 1 -- 172.21.15.86:0/4068541565 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f21b80cfe60 con 0x7f21c4103190 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.798+0000 7f21cedab740 1 -- 172.21.15.86:0/1916998010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 msgr2=0x55ae5436f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.798+0000 7f21cedab740 1 --2- 172.21.15.86:0/1916998010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 0x55ae5436f3c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009ca0 tx=0x7f21b0011810 comp rx=0 tx=0).stop 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.798+0000 7f21c9600640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f21ac0a3c30 0x7f21ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.798+0000 7f21cedab740 1 -- 172.21.15.86:0/1916998010 shutdown_connections 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.798+0000 7f21cedab740 1 --2- 172.21.15.86:0/1916998010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae54374b00 0x55ae54376f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.798+0000 7f21cedab740 1 --2- 172.21.15.86:0/1916998010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436f990 0x55ae54374490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.798+0000 7f21cedab740 1 --2- 172.21.15.86:0/1916998010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 0x55ae5436f3c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.798+0000 7f21cedab740 1 -- 172.21.15.86:0/1916998010 >> 172.21.15.86:0/1916998010 conn(0x55ae5436a510 msgr2=0x55ae5436a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.802+0000 7f21c9600640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f21ac0a3c30 0x7f21ac0a60f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55ae54370400 tx=0x7f21bc02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.802+0000 7f21cedab740 1 -- 172.21.15.86:0/1916998010 shutdown_connections 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.803+0000 7f21cedab740 1 -- 172.21.15.86:0/1916998010 wait complete. 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.805+0000 7f21cedab740 1 -- start start 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.807+0000 7f21cedab740 1 Processor -- start 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.809+0000 7f21cedab740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 0x55ae54379550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.809+0000 7f21cedab740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436f990 0x55ae54379a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.809+0000 7f21cedab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae54374b00 0x55ae54380ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.809+0000 7f21cedab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae54377ab0 con 0x55ae54374b00 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.809+0000 7f21cedab740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ae5428e830 con 0x55ae5436efa0 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.809+0000 7f21cedab740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae54377c30 con 0x55ae5436f990 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.809+0000 7f21cedab740 10 client.0 _ll_register_callbacks cb 0x55ae5425cfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.810+0000 7f21c8c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436f990 0x55ae54379a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.810+0000 7f21c8c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436f990 0x55ae54379a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40672/0 (socket says 172.21.15.86:40672) 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.810+0000 7f21c8c00640 1 -- 172.21.15.86:0/1035252228 learned_addr learned my addr 172.21.15.86:0/1035252228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.810+0000 7f21c8c00640 1 -- 172.21.15.86:0/1035252228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 msgr2=0x55ae54379550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.810+0000 7f21c8c00640 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 0x55ae54379550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.810+0000 7f21c8c00640 1 -- 172.21.15.86:0/1035252228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae54374b00 msgr2=0x55ae54380ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.810+0000 7f21c8c00640 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae54374b00 0x55ae54380ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.810+0000 7f21c8c00640 1 -- 172.21.15.86:0/1035252228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae54292920 con 0x55ae5436f990 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.811+0000 7f21c8c00640 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436f990 0x55ae54379a90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f21b800b3c0 tx=0x7f21b809bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74f0996d6c112259 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.811+0000 7f21aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.812+0000 7f21aaa00640 1 -- 172.21.15.86:0/1035252228 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b809c4c0 con 0x55ae5436f990 2024-04-17T07:32:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.812+0000 7f21aaa00640 1 -- 172.21.15.86:0/1035252228 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f21b8011830 con 0x55ae5436f990 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.812+0000 7f21cedab740 1 -- 172.21.15.86:0/1035252228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b0009e50 con 0x55ae5436f990 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.812+0000 7f21aaa00640 1 -- 172.21.15.86:0/1035252228 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21b809bdc0 con 0x55ae5436f990 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.812+0000 7f21cedab740 10 client.28591 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.812+0000 7f21cedab740 1 -- 172.21.15.86:0/1035252228 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae542607f0 con 0x55ae5436f990 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.813+0000 7f21aaa00640 1 -- 172.21.15.86:0/1035252228 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21b0009e50 con 0x55ae5436f990 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.813+0000 7f21aaa00640 1 -- 172.21.15.86:0/1035252228 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f21b8097b10 con 0x55ae5436f990 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.813+0000 7f21cedab740 10 client.28591 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.813+0000 7f21cedab740 10 client.28591 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.813+0000 7f21cedab740 20 client.28591 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.813+0000 7f21cedab740 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ae54385070 0x55ae543874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.813+0000 7f21cedab740 4 client.28591 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.813+0000 7f21cedab740 1 -- 172.21.15.86:0/1035252228 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae542e2a40 con 0x55ae54385070 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.814+0000 7f21c9600640 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ae54385070 0x55ae543874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.815+0000 7f21c9600640 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ae54385070 0x55ae543874c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cff1cd64d80373fc server_cookie=3f8e40c62a45115f in_seq=0 out_seq=0 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.815+0000 7f21aaa00640 10 client.28591 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.816+0000 7f21aaa00640 1 -- 172.21.15.86:0/1035252228 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae542e2a40 con 0x55ae54385070 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.816+0000 7f21aaa00640 10 client.28591 handle_command_reply: tid=0 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.817+0000 7f21cedab740 1 client.28591 shutdown 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.817+0000 7f21cedab740 1 -- 172.21.15.86:0/1035252228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436f990 msgr2=0x55ae54379a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.817+0000 7f21cedab740 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436f990 0x55ae54379a90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f21b800b3c0 tx=0x7f21b809bc50 comp rx=0 tx=0).stop 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.818+0000 7f21cedab740 1 -- 172.21.15.86:0/1035252228 shutdown_connections 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.818+0000 7f21cedab740 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ae54385070 0x55ae543874c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.818+0000 7f21cedab740 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae54374b00 0x55ae54380ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.818+0000 7f21cedab740 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436f990 0x55ae54379a90 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f21b800b3c0 tx=0x7f21b809bc50 comp rx=0 tx=0).stop 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.818+0000 7f21cedab740 1 --2- 172.21.15.86:0/1035252228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae5436efa0 0x55ae54379550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.818+0000 7f21cedab740 1 -- 172.21.15.86:0/1035252228 >> 172.21.15.86:0/1035252228 conn(0x55ae5436a510 msgr2=0x55ae54369710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.818+0000 7f21aaa00640 0 client.28591 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:32:47.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.818+0000 7f21cedab740 1 -- 172.21.15.86:0/1035252228 shutdown_connections 2024-04-17T07:32:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.818+0000 7f21cedab740 1 -- 172.21.15.86:0/1035252228 wait complete. 2024-04-17T07:32:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.818+0000 7f21cedab740 20 client.28591 trim_cache size 0 max 16384 2024-04-17T07:32:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.853+0000 7f21cedab740 1 Processor -- start 2024-04-17T07:32:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.853+0000 7f21cedab740 1 -- start start 2024-04-17T07:32:47.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.853+0000 7f21cedab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436efa0 0x55ae543659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.853+0000 7f21cedab740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54374b00 0x55ae54380a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.853+0000 7f21cedab740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae54379bc0 0x55ae5436de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.853+0000 7f21cedab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21c41b1370 con 0x55ae5436efa0 2024-04-17T07:32:47.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.853+0000 7f21cedab740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f21c41b2eb0 con 0x55ae54374b00 2024-04-17T07:32:47.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.853+0000 7f21cedab740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21c41b1a40 con 0x55ae54379bc0 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.854+0000 7f21c8c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54374b00 0x55ae54380a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.854+0000 7f21c8c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54374b00 0x55ae54380a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36690/0 (socket says 172.21.15.86:36690) 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.854+0000 7f21c8c00640 1 -- 172.21.15.86:0/2014450107 learned_addr learned my addr 172.21.15.86:0/2014450107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.854+0000 7f21c9600640 1 --2- 172.21.15.86:0/2014450107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436efa0 0x55ae543659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.854+0000 7f21ca000640 1 --2- 172.21.15.86:0/2014450107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae54379bc0 0x55ae5436de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.855+0000 7f21c9600640 1 -- 172.21.15.86:0/2014450107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae54379bc0 msgr2=0x55ae5436de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.855+0000 7f21c9600640 1 --2- 172.21.15.86:0/2014450107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae54379bc0 0x55ae5436de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.855+0000 7f21c9600640 1 -- 172.21.15.86:0/2014450107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54374b00 msgr2=0x55ae54380a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.855+0000 7f21c9600640 1 --2- 172.21.15.86:0/2014450107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54374b00 0x55ae54380a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.855+0000 7f21c9600640 1 -- 172.21.15.86:0/2014450107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae53fe73f0 con 0x55ae5436efa0 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.855+0000 7f21c9600640 1 --2- 172.21.15.86:0/2014450107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436efa0 0x55ae543659d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f21bc005e30 tx=0x7f21bc03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1f15ffff412921 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.856+0000 7f21b5c00640 1 -- 172.21.15.86:0/2014450107 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc039040 con 0x55ae5436efa0 2024-04-17T07:32:47.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.856+0000 7f21b5c00640 1 -- 172.21.15.86:0/2014450107 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21bc055030 con 0x55ae5436efa0 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.856+0000 7f21b5c00640 1 -- 172.21.15.86:0/2014450107 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc039040 con 0x55ae5436efa0 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.857+0000 7f21cedab740 1 -- 172.21.15.86:0/2014450107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436efa0 msgr2=0x55ae543659d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.857+0000 7f21cedab740 1 --2- 172.21.15.86:0/2014450107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436efa0 0x55ae543659d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f21bc005e30 tx=0x7f21bc03f000 comp rx=0 tx=0).stop 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.857+0000 7f21cedab740 1 -- 172.21.15.86:0/2014450107 shutdown_connections 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.857+0000 7f21cedab740 1 --2- 172.21.15.86:0/2014450107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae54379bc0 0x55ae5436de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.857+0000 7f21cedab740 1 --2- 172.21.15.86:0/2014450107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54374b00 0x55ae54380a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.857+0000 7f21cedab740 1 --2- 172.21.15.86:0/2014450107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae5436efa0 0x55ae543659d0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.857+0000 7f21cedab740 1 -- 172.21.15.86:0/2014450107 >> 172.21.15.86:0/2014450107 conn(0x55ae5436a510 msgr2=0x55ae5436d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.857+0000 7f21cedab740 1 -- 172.21.15.86:0/2014450107 shutdown_connections 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.857+0000 7f21cedab740 1 -- 172.21.15.86:0/2014450107 wait complete. 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.857+0000 7f21cedab740 1 -- start start 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.858+0000 7f21cedab740 1 Processor -- start 2024-04-17T07:32:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.859+0000 7f21cedab740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436efa0 0x55ae5437e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.859+0000 7f21cedab740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae54374b00 0x55ae5437e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.859+0000 7f21cedab740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54379bc0 0x55ae5437f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21cedab740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21c41b27e0 con 0x55ae54374b00 2024-04-17T07:32:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21cedab740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f21c41b2110 con 0x55ae54379bc0 2024-04-17T07:32:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21cedab740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21c41af340 con 0x55ae5436efa0 2024-04-17T07:32:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21cedab740 10 client.0 _ll_register_callbacks cb 0x7f21b8098150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21ca000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54379bc0 0x55ae5437f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21ca000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54379bc0 0x55ae5437f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36696/0 (socket says 172.21.15.86:36696) 2024-04-17T07:32:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21ca000640 1 -- 172.21.15.86:0/2607979887 learned_addr learned my addr 172.21.15.86:0/2607979887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21c8c00640 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae54374b00 0x55ae5437e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21c9600640 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436efa0 0x55ae5437e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21ca000640 1 -- 172.21.15.86:0/2607979887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436efa0 msgr2=0x55ae5437e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21ca000640 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436efa0 0x55ae5437e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21ca000640 1 -- 172.21.15.86:0/2607979887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae54374b00 msgr2=0x55ae5437e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21ca000640 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae54374b00 0x55ae5437e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.860+0000 7f21ca000640 1 -- 172.21.15.86:0/2607979887 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae540b17d0 con 0x55ae54379bc0 2024-04-17T07:32:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.861+0000 7f21c9600640 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436efa0 0x55ae5437e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:32:47.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.861+0000 7f21c8c00640 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae54374b00 0x55ae5437e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:32:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.861+0000 7f21aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.861+0000 7f21ca000640 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54379bc0 0x55ae5437f700 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009ca0 tx=0x7f21b0005660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ae55531211ed08 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.862+0000 7f21aaa00640 1 -- 172.21.15.86:0/2607979887 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b001e040 con 0x55ae54379bc0 2024-04-17T07:32:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.862+0000 7f21aaa00640 1 -- 172.21.15.86:0/2607979887 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f21b00037f0 con 0x55ae54379bc0 2024-04-17T07:32:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.862+0000 7f21cedab740 1 -- 172.21.15.86:0/2607979887 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21bc055030 con 0x55ae54379bc0 2024-04-17T07:32:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.863+0000 7f21aaa00640 1 -- 172.21.15.86:0/2607979887 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21b0009e50 con 0x55ae54379bc0 2024-04-17T07:32:47.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.863+0000 7f21cedab740 10 client.38603 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:47.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.863+0000 7f21cedab740 1 -- 172.21.15.86:0/2607979887 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae543eb600 con 0x55ae54379bc0 2024-04-17T07:32:47.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.863+0000 7f21aaa00640 1 -- 172.21.15.86:0/2607979887 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21bc055030 con 0x55ae54379bc0 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.864+0000 7f21aaa00640 1 -- 172.21.15.86:0/2607979887 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f21b0024990 con 0x55ae54379bc0 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.864+0000 7f21cedab740 10 client.38603 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.864+0000 7f21cedab740 10 client.38603 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.864+0000 7f21cedab740 20 client.38603 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.864+0000 7f21cedab740 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ae543ebd00 0x55ae543ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.864+0000 7f21cedab740 4 client.38603 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.864+0000 7f21cedab740 1 -- 172.21.15.86:0/2607979887 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae542dc8b0 con 0x55ae543ebd00 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.864+0000 7f21c9600640 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ae543ebd00 0x55ae543ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.865+0000 7f21c9600640 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ae543ebd00 0x55ae543ee120 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea285bc3d6a35355 server_cookie=fbd83ed174ab3186 in_seq=0 out_seq=0 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.865+0000 7f21aaa00640 10 client.38603 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.866+0000 7f21aaa00640 1 -- 172.21.15.86:0/2607979887 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae542dc8b0 con 0x55ae543ebd00 2024-04-17T07:32:47.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.866+0000 7f21aaa00640 10 client.38603 handle_command_reply: tid=0 2024-04-17T07:32:47.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.866+0000 7f21cedab740 1 client.38603 shutdown 2024-04-17T07:32:47.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.866+0000 7f21cedab740 1 -- 172.21.15.86:0/2607979887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54379bc0 msgr2=0x55ae5437f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.866+0000 7f21cedab740 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54379bc0 0x55ae5437f700 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009ca0 tx=0x7f21b0005660 comp rx=0 tx=0).stop 2024-04-17T07:32:47.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.867+0000 7f21cedab740 1 -- 172.21.15.86:0/2607979887 shutdown_connections 2024-04-17T07:32:47.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.867+0000 7f21cedab740 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55ae543ebd00 0x55ae543ee120 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.867+0000 7f21cedab740 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae54379bc0 0x55ae5437f700 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.867+0000 7f21cedab740 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae54374b00 0x55ae5437e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.867+0000 7f21cedab740 1 --2- 172.21.15.86:0/2607979887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae5436efa0 0x55ae5437e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.867+0000 7f21aaa00640 0 client.38603 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:32:47.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.867+0000 7f21cedab740 1 -- 172.21.15.86:0/2607979887 >> 172.21.15.86:0/2607979887 conn(0x55ae5436a510 msgr2=0x55ae5436e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:47.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.867+0000 7f21cedab740 1 -- 172.21.15.86:0/2607979887 shutdown_connections 2024-04-17T07:32:47.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.868+0000 7f21cedab740 1 -- 172.21.15.86:0/2607979887 wait complete. 2024-04-17T07:32:47.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.869+0000 7f21cedab740 20 client.38603 trim_cache size 0 max 16384 2024-04-17T07:32:47.870 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:32:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.872+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f21ac0a3c30 msgr2=0x7f21ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.872+0000 7f21cc000640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f21ac0a3c30 0x7f21ac0a60f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55ae54370400 tx=0x7f21bc02e040 comp rx=0 tx=0).stop 2024-04-17T07:32:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.872+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21c4103190 msgr2=0x7f21c419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.872+0000 7f21cc000640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21c4103190 0x7f21c419edf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f21b800b3f0 tx=0x7f21b800b8b0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.872+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 shutdown_connections 2024-04-17T07:32:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.872+0000 7f21cc000640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f21ac0a3c30 0x7f21ac0a60f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.872+0000 7f21cc000640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c4105b40 0x7f21c41a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.872+0000 7f21cc000640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21c4103190 0x7f21c419edf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.872+0000 7f21cc000640 1 --2- 172.21.15.86:0/4068541565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21c4100820 0x7f21c419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:47.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.872+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 >> 172.21.15.86:0/4068541565 conn(0x7f21c40fc420 msgr2=0x7f21c4103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:47.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.874+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 shutdown_connections 2024-04-17T07:32:47.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:47.875+0000 7f21cc000640 1 -- 172.21.15.86:0/4068541565 wait complete. 2024-04-17T07:32:47.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:32:48.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:47 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1866125993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:32:48.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:47 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1866125993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:32:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.290+0000 7f3cc5e00640 1 -- 172.21.15.86:0/349179135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc0074960 msgr2=0x7f3cc0074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.290+0000 7f3cc5e00640 1 --2- 172.21.15.86:0/349179135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc0074960 0x7f3cc0074dc0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009900 tx=0x7f3cac02bcc0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.291+0000 7f3cc5e00640 1 -- 172.21.15.86:0/349179135 shutdown_connections 2024-04-17T07:32:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.291+0000 7f3cc5e00640 1 --2- 172.21.15.86:0/349179135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0109e30 0x7f3cc010c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.291+0000 7f3cc5e00640 1 --2- 172.21.15.86:0/349179135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc0074960 0x7f3cc0074dc0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.291+0000 7f3cc5e00640 1 --2- 172.21.15.86:0/349179135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0075f40 0x7f3cc0074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.291+0000 7f3cc5e00640 1 -- 172.21.15.86:0/349179135 >> 172.21.15.86:0/349179135 conn(0x7f3cc00fd510 msgr2=0x7f3cc00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.291+0000 7f3cc5e00640 1 -- 172.21.15.86:0/349179135 shutdown_connections 2024-04-17T07:32:48.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.291+0000 7f3cc5e00640 1 -- 172.21.15.86:0/349179135 wait complete. 2024-04-17T07:32:48.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.293+0000 7f3cc5e00640 1 Processor -- start 2024-04-17T07:32:48.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.293+0000 7f3cc5e00640 1 -- start start 2024-04-17T07:32:48.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.293+0000 7f3cc5e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0074960 0x7f3cc019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.293+0000 7f3cc5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc0075f40 0x7f3cc019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.293+0000 7f3cc5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0109e30 0x7f3cc01a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.293+0000 7f3cc5e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc010fce0 con 0x7f3cc0075f40 2024-04-17T07:32:48.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.293+0000 7f3cc5e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc010fb60 con 0x7f3cc0109e30 2024-04-17T07:32:48.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.293+0000 7f3cc5e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc010fe60 con 0x7f3cc0074960 2024-04-17T07:32:48.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.294+0000 7f3cbf400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0109e30 0x7f3cc01a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.294+0000 7f3cbea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0074960 0x7f3cc019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.294+0000 7f3cbf400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0109e30 0x7f3cc01a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36726/0 (socket says 172.21.15.86:36726) 2024-04-17T07:32:48.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.294+0000 7f3cbf400640 1 -- 172.21.15.86:0/2254541472 learned_addr learned my addr 172.21.15.86:0/2254541472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:48.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.294+0000 7f3cbea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0074960 0x7f3cc019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40728/0 (socket says 172.21.15.86:40728) 2024-04-17T07:32:48.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.294+0000 7f3cbe000640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc0075f40 0x7f3cc019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.295+0000 7f3cbf400640 1 -- 172.21.15.86:0/2254541472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0074960 msgr2=0x7f3cc019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.295+0000 7f3cbf400640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0074960 0x7f3cc019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.295+0000 7f3cbf400640 1 -- 172.21.15.86:0/2254541472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc0075f40 msgr2=0x7f3cc019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.295+0000 7f3cbf400640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc0075f40 0x7f3cc019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.295+0000 7f3cbf400640 1 -- 172.21.15.86:0/2254541472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc01a6450 con 0x7f3cc0109e30 2024-04-17T07:32:48.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.295+0000 7f3cbf400640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0109e30 0x7f3cc01a5de0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800e970 tx=0x7f3cb800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.297+0000 7f3cc4a00640 1 -- 172.21.15.86:0/2254541472 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800ccc0 con 0x7f3cc0109e30 2024-04-17T07:32:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.297+0000 7f3cc4a00640 1 -- 172.21.15.86:0/2254541472 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cb800ce60 con 0x7f3cc0109e30 2024-04-17T07:32:48.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.297+0000 7f3cc4a00640 1 -- 172.21.15.86:0/2254541472 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8010430 con 0x7f3cc0109e30 2024-04-17T07:32:48.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.298+0000 7f3cc5e00640 1 -- 172.21.15.86:0/2254541472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc01a6740 con 0x7f3cc0109e30 2024-04-17T07:32:48.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.298+0000 7f3cc5e00640 1 -- 172.21.15.86:0/2254541472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc01a6c10 con 0x7f3cc0109e30 2024-04-17T07:32:48.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.304+0000 7f3cc8a70740 1 Processor -- start 2024-04-17T07:32:48.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.306+0000 7f3cc4a00640 1 -- 172.21.15.86:0/2254541472 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f3cb80046c0 con 0x7f3cc0109e30 2024-04-17T07:32:48.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.306+0000 7f3cc8a70740 1 -- start start 2024-04-17T07:32:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.307+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae1fa0 0x55ab34ae23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.307+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae2990 0x55ab34ae7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.307+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae7b00 0x55ab34ae9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.307+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab34a48020 con 0x55ab34ae7b00 2024-04-17T07:32:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.307+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ab349ee460 con 0x55ab34ae1fa0 2024-04-17T07:32:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.307+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ab34a43fe0 con 0x55ab34ae2990 2024-04-17T07:32:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.308+0000 7f3cbea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae2990 0x55ab34ae7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.308+0000 7f3cbea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae2990 0x55ab34ae7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40738/0 (socket says 172.21.15.86:40738) 2024-04-17T07:32:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.308+0000 7f3cbea00640 1 -- 172.21.15.86:0/1863187601 learned_addr learned my addr 172.21.15.86:0/1863187601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.308+0000 7f3cc4a00640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ca40a3c30 0x7f3ca40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.308+0000 7f3cbe000640 1 --2- 172.21.15.86:0/1863187601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae7b00 0x55ab34ae9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.309+0000 7f3cc4a00640 1 -- 172.21.15.86:0/2254541472 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3cb80d0bb0 con 0x7f3cc0109e30 2024-04-17T07:32:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.309+0000 7f3cbe000640 1 --2- 172.21.15.86:0/1863187601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae1fa0 0x55ab34ae23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.309+0000 7f3cbf400640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ca40a3c30 0x7f3ca40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.310+0000 7f3cbea00640 1 -- 172.21.15.86:0/1863187601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae1fa0 msgr2=0x55ab34ae23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.310+0000 7f3cbea00640 1 --2- 172.21.15.86:0/1863187601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae1fa0 0x55ab34ae23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.310+0000 7f3cbea00640 1 -- 172.21.15.86:0/1863187601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae7b00 msgr2=0x55ab34ae9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.310+0000 7f3cbea00640 1 --2- 172.21.15.86:0/1863187601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae7b00 0x55ab34ae9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.310+0000 7f3cbea00640 1 -- 172.21.15.86:0/1863187601 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab349f6040 con 0x55ab34ae2990 2024-04-17T07:32:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.310+0000 7f3cbe000640 1 --2- 172.21.15.86:0/1863187601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae7b00 0x55ab34ae9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:32:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.311+0000 7f3cbea00640 1 --2- 172.21.15.86:0/1863187601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae2990 0x55ab34ae7490 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00114f0 tx=0x7f3cb00119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ec6f2d1b3c50e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.311+0000 7f3cbf400640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ca40a3c30 0x7f3ca40a60f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80994e0 tx=0x7f3cb80095d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.311+0000 7f3cabe00640 1 -- 172.21.15.86:0/1863187601 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0004d20 con 0x55ab34ae2990 2024-04-17T07:32:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.311+0000 7f3cabe00640 1 -- 172.21.15.86:0/1863187601 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cb0004050 con 0x55ab34ae2990 2024-04-17T07:32:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.311+0000 7f3cabe00640 1 -- 172.21.15.86:0/1863187601 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000f7c0 con 0x55ab34ae2990 2024-04-17T07:32:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.313+0000 7f3cc8a70740 1 -- 172.21.15.86:0/1863187601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae2990 msgr2=0x55ab34ae7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.313+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/1863187601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae2990 0x55ab34ae7490 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00114f0 tx=0x7f3cb00119b0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.313+0000 7f3cc8a70740 1 -- 172.21.15.86:0/1863187601 shutdown_connections 2024-04-17T07:32:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.313+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/1863187601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae7b00 0x55ab34ae9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.313+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/1863187601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae2990 0x55ab34ae7490 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.313+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/1863187601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae1fa0 0x55ab34ae23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.313+0000 7f3cc8a70740 1 -- 172.21.15.86:0/1863187601 >> 172.21.15.86:0/1863187601 conn(0x55ab34add510 msgr2=0x55ab34add910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.314+0000 7f3cc8a70740 1 -- 172.21.15.86:0/1863187601 shutdown_connections 2024-04-17T07:32:48.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.314+0000 7f3cc8a70740 1 -- 172.21.15.86:0/1863187601 wait complete. 2024-04-17T07:32:48.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.315+0000 7f3cc8a70740 1 -- start start 2024-04-17T07:32:48.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.315+0000 7f3cc8a70740 1 Processor -- start 2024-04-17T07:32:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.316+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae1fa0 0x55ab34aec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.316+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae2990 0x55ab34aeca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.316+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae7b00 0x55ab34af3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.316+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ab34aeaab0 con 0x55ab34ae2990 2024-04-17T07:32:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.316+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ab34a01830 con 0x55ab34ae1fa0 2024-04-17T07:32:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.316+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ab34aeac30 con 0x55ab34ae7b00 2024-04-17T07:32:48.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.316+0000 7f3cc8a70740 10 client.0 _ll_register_callbacks cb 0x55ab349cffb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:48.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.317+0000 7f3cbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae2990 0x55ab34aeca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.317+0000 7f3cbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae2990 0x55ab34aeca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53766/0 (socket says 172.21.15.86:53766) 2024-04-17T07:32:48.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.317+0000 7f3cbea00640 1 -- 172.21.15.86:0/112067227 learned_addr learned my addr 172.21.15.86:0/112067227 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:48.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.317+0000 7f3cbe000640 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae1fa0 0x55ab34aec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.318+0000 7f3cbe000640 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae7b00 0x55ab34af3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.318+0000 7f3cbea00640 1 -- 172.21.15.86:0/112067227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae7b00 msgr2=0x55ab34af3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.318+0000 7f3cbea00640 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae7b00 0x55ab34af3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.318+0000 7f3cbea00640 1 -- 172.21.15.86:0/112067227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae1fa0 msgr2=0x55ab34aec4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.318+0000 7f3cbea00640 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae1fa0 0x55ab34aec4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.318+0000 7f3cbea00640 1 -- 172.21.15.86:0/112067227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab34a05920 con 0x55ab34ae2990 2024-04-17T07:32:48.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.318+0000 7f3cbe000640 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae7b00 0x55ab34af3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:32:48.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.319+0000 7f3cbea00640 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae2990 0x55ab34aeca20 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f3cb001b040 tx=0x7f3cb0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6beed60d6a947f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.319+0000 7f3ca8c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:48.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.319+0000 7f3ca8c00640 1 -- 172.21.15.86:0/112067227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0028020 con 0x55ab34ae2990 2024-04-17T07:32:48.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.319+0000 7f3ca8c00640 1 -- 172.21.15.86:0/112067227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3cb0004c50 con 0x55ab34ae2990 2024-04-17T07:32:48.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.321+0000 7f3cc8a70740 1 -- 172.21.15.86:0/112067227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cb0004050 con 0x55ab34ae2990 2024-04-17T07:32:48.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.324+0000 7f3ca8c00640 1 -- 172.21.15.86:0/112067227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3cb0004e50 con 0x55ab34ae2990 2024-04-17T07:32:48.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.324+0000 7f3cc8a70740 10 client.19581 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:48.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.324+0000 7f3cc8a70740 1 -- 172.21.15.86:0/112067227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab349d37f0 con 0x55ab34ae2990 2024-04-17T07:32:48.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.324+0000 7f3ca8c00640 1 -- 172.21.15.86:0/112067227 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cb0004050 con 0x55ab34ae2990 2024-04-17T07:32:48.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.325+0000 7f3ca8c00640 1 -- 172.21.15.86:0/112067227 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3cb00050d0 con 0x55ab34ae2990 2024-04-17T07:32:48.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.325+0000 7f3cc8a70740 10 client.19581 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:48.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.325+0000 7f3cc8a70740 10 client.19581 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:32:48.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.325+0000 7f3cc8a70740 20 client.19581 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:48.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.326+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ab34af7d10 0x55ab34afa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.326+0000 7f3cc8a70740 4 client.19581 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:48.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.326+0000 7f3cc8a70740 1 -- 172.21.15.86:0/112067227 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab34a55a40 con 0x55ab34af7d10 2024-04-17T07:32:48.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.326+0000 7f3cbe000640 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ab34af7d10 0x55ab34afa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.328+0000 7f3cbe000640 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ab34af7d10 0x55ab34afa160 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8784afd112551bf7 server_cookie=d6b6ff4cb58d9ffe in_seq=0 out_seq=0 2024-04-17T07:32:48.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.328+0000 7f3ca8c00640 10 client.19581 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:32:48.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:47 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1866125993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:32:48.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.330+0000 7f3ca8c00640 1 -- 172.21.15.86:0/112067227 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab34a55a40 con 0x55ab34af7d10 2024-04-17T07:32:48.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.330+0000 7f3ca8c00640 10 client.19581 handle_command_reply: tid=0 2024-04-17T07:32:48.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.330+0000 7f3cc8a70740 1 client.19581 shutdown 2024-04-17T07:32:48.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.332+0000 7f3cc8a70740 1 -- 172.21.15.86:0/112067227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae2990 msgr2=0x55ab34aeca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.332+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae2990 0x55ab34aeca20 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f3cb001b040 tx=0x7f3cb0004430 comp rx=0 tx=0).stop 2024-04-17T07:32:48.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.333+0000 7f3cc8a70740 1 -- 172.21.15.86:0/112067227 shutdown_connections 2024-04-17T07:32:48.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.333+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ab34af7d10 0x55ab34afa160 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.333+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34ae7b00 0x55ab34af3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.333+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae2990 0x55ab34aeca20 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.333+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/112067227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae1fa0 0x55ab34aec4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.333+0000 7f3cc8a70740 1 -- 172.21.15.86:0/112067227 >> 172.21.15.86:0/112067227 conn(0x55ab34add510 msgr2=0x55ab34adc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.333+0000 7f3ca8c00640 0 client.19581 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:32:48.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.334+0000 7f3cc8a70740 1 -- 172.21.15.86:0/112067227 shutdown_connections 2024-04-17T07:32:48.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.334+0000 7f3cc8a70740 1 -- 172.21.15.86:0/112067227 wait complete. 2024-04-17T07:32:48.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.334+0000 7f3cc8a70740 20 client.19581 trim_cache size 0 max 16384 2024-04-17T07:32:48.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.372+0000 7f3cc8a70740 1 Processor -- start 2024-04-17T07:32:48.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.372+0000 7f3cc8a70740 1 -- start start 2024-04-17T07:32:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.373+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 0x55ab34ae3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.373+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 0x55ab34ae43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.373+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae2990 0x55ab34af3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.373+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01b1140 con 0x55ab34ae1fa0 2024-04-17T07:32:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.373+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01b2c80 con 0x55ab34ae2990 2024-04-17T07:32:48.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.373+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01b1810 con 0x55ab34add510 2024-04-17T07:32:48.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.374+0000 7f3cbe000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 0x55ab34ae3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.374+0000 7f3cbe000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 0x55ab34ae3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40752/0 (socket says 172.21.15.86:40752) 2024-04-17T07:32:48.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.374+0000 7f3cbe000640 1 -- 172.21.15.86:0/1267761542 learned_addr learned my addr 172.21.15.86:0/1267761542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:48.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.374+0000 7f3cbe000640 1 --2- 172.21.15.86:0/1267761542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae2990 0x55ab34af3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.374+0000 7f3cbea00640 1 --2- 172.21.15.86:0/1267761542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 0x55ab34ae43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.374+0000 7f3cbe000640 1 -- 172.21.15.86:0/1267761542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae2990 msgr2=0x55ab34af3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.374+0000 7f3cbe000640 1 --2- 172.21.15.86:0/1267761542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae2990 0x55ab34af3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.374+0000 7f3cbe000640 1 -- 172.21.15.86:0/1267761542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 msgr2=0x55ab34ae43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.374+0000 7f3cbe000640 1 --2- 172.21.15.86:0/1267761542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 0x55ab34ae43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.376+0000 7f3cbe000640 1 -- 172.21.15.86:0/1267761542 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3475a3f0 con 0x55ab34add510 2024-04-17T07:32:48.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.376+0000 7f3cbe000640 1 --2- 172.21.15.86:0/1267761542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 0x55ab34ae3e90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0096c0 tx=0x7f3cac02bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ebc04d9d8d6bf42 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.377+0000 7f3cab400640 1 -- 172.21.15.86:0/1267761542 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac02de60 con 0x55ab34add510 2024-04-17T07:32:48.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.377+0000 7f3cab400640 1 -- 172.21.15.86:0/1267761542 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cac0055d0 con 0x55ab34add510 2024-04-17T07:32:48.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.377+0000 7f3cab400640 1 -- 172.21.15.86:0/1267761542 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac048820 con 0x55ab34add510 2024-04-17T07:32:48.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.379+0000 7f3cc8a70740 1 -- 172.21.15.86:0/1267761542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 msgr2=0x55ab34ae3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.379+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/1267761542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 0x55ab34ae3e90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0096c0 tx=0x7f3cac02bcc0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.380+0000 7f3cc8a70740 1 -- 172.21.15.86:0/1267761542 shutdown_connections 2024-04-17T07:32:48.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.380+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/1267761542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae2990 0x55ab34af3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.380+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/1267761542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 0x55ab34ae43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.380+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/1267761542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 0x55ab34ae3e90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.380+0000 7f3cc8a70740 1 -- 172.21.15.86:0/1267761542 >> 172.21.15.86:0/1267761542 conn(0x55ab34ae3770 msgr2=0x55ab34aeafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.381+0000 7f3cc8a70740 1 -- 172.21.15.86:0/1267761542 shutdown_connections 2024-04-17T07:32:48.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.381+0000 7f3cc8a70740 1 -- 172.21.15.86:0/1267761542 wait complete. 2024-04-17T07:32:48.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.382+0000 7f3cc8a70740 1 -- start start 2024-04-17T07:32:48.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.383+0000 7f3cc8a70740 1 Processor -- start 2024-04-17T07:32:48.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.385+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 0x55ab34af0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.385+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 0x55ab34af1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.385+0000 7f3cc8a70740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae2990 0x55ab34af2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.385+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01b25b0 con 0x55ab34ae1fa0 2024-04-17T07:32:48.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.385+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01b1ee0 con 0x55ab34ae2990 2024-04-17T07:32:48.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.385+0000 7f3cc8a70740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01af110 con 0x55ab34add510 2024-04-17T07:32:48.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.385+0000 7f3cc8a70740 10 client.0 _ll_register_callbacks cb 0x7f3cb0009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:48.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.386+0000 7f3cbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 0x55ab34af1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.386+0000 7f3cbea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 0x55ab34af1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53782/0 (socket says 172.21.15.86:53782) 2024-04-17T07:32:48.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.386+0000 7f3cbea00640 1 -- 172.21.15.86:0/2309194141 learned_addr learned my addr 172.21.15.86:0/2309194141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.387+0000 7f3cbe000640 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 0x55ab34af0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.387+0000 7f3cbe000640 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae2990 0x55ab34af2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.387+0000 7f3cbea00640 1 -- 172.21.15.86:0/2309194141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 msgr2=0x55ab34af0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.387+0000 7f3cbea00640 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 0x55ab34af0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.387+0000 7f3cbea00640 1 -- 172.21.15.86:0/2309194141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae2990 msgr2=0x55ab34af2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.387+0000 7f3cbea00640 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae2990 0x55ab34af2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.387+0000 7f3cbea00640 1 -- 172.21.15.86:0/2309194141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab348247d0 con 0x55ab34ae1fa0 2024-04-17T07:32:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.388+0000 7f3ca8c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.388+0000 7f3cbea00640 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 0x55ab34af1330 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x55ab34ade2f0 tx=0x7f3cb00114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2148d665e4ec4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.389+0000 7f3ca8c00640 1 -- 172.21.15.86:0/2309194141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb003b030 con 0x55ab34ae1fa0 2024-04-17T07:32:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.389+0000 7f3ca8c00640 1 -- 172.21.15.86:0/2309194141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f3cb0005040 con 0x55ab34ae1fa0 2024-04-17T07:32:48.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.389+0000 7f3cc8a70740 1 -- 172.21.15.86:0/2309194141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cac0055d0 con 0x55ab34ae1fa0 2024-04-17T07:32:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.390+0000 7f3ca8c00640 1 -- 172.21.15.86:0/2309194141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3cb0015030 con 0x55ab34ae1fa0 2024-04-17T07:32:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.390+0000 7f3cc8a70740 10 client.19587 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.390+0000 7f3cc8a70740 1 -- 172.21.15.86:0/2309194141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab34af3a60 con 0x55ab34ae1fa0 2024-04-17T07:32:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.390+0000 7f3ca8c00640 1 -- 172.21.15.86:0/2309194141 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cac0055d0 con 0x55ab34ae1fa0 2024-04-17T07:32:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.390+0000 7f3ca8c00640 1 -- 172.21.15.86:0/2309194141 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3cb000f7c0 con 0x55ab34ae1fa0 2024-04-17T07:32:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.391+0000 7f3cc8a70740 10 client.19587 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.391+0000 7f3cc8a70740 10 client.19587 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:32:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.391+0000 7f3cc8a70740 20 client.19587 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.391+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ab34ae7b00 0x55ab34b71730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.391+0000 7f3cc8a70740 4 client.19587 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:32:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.391+0000 7f3cc8a70740 1 -- 172.21.15.86:0/2309194141 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab34a4f8b0 con 0x55ab34ae7b00 2024-04-17T07:32:48.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.391+0000 7f3cbe000640 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ab34ae7b00 0x55ab34b71730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.392+0000 7f3ca8c00640 10 client.19587 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:32:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.392+0000 7f3cbe000640 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ab34ae7b00 0x55ab34b71730 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a06a47ecc199a98 server_cookie=1a84df92a8d13659 in_seq=0 out_seq=0 2024-04-17T07:32:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.392+0000 7f3ca8c00640 1 -- 172.21.15.86:0/2309194141 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab34a4f8b0 con 0x55ab34ae7b00 2024-04-17T07:32:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.392+0000 7f3ca8c00640 10 client.19587 handle_command_reply: tid=0 2024-04-17T07:32:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.392+0000 7f3cc8a70740 1 client.19587 shutdown 2024-04-17T07:32:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3cc8a70740 1 -- 172.21.15.86:0/2309194141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 msgr2=0x55ab34af1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 0x55ab34af1330 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x55ab34ade2f0 tx=0x7f3cb00114b0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3cc8a70740 1 -- 172.21.15.86:0/2309194141 shutdown_connections 2024-04-17T07:32:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x55ab34ae7b00 0x55ab34b71730 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3ca8c00640 0 client.19587 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:32:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ab34ae2990 0x55ab34af2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ab34ae1fa0 0x55ab34af1330 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3cc8a70740 1 --2- 172.21.15.86:0/2309194141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ab34add510 0x55ab34af0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3cc8a70740 1 -- 172.21.15.86:0/2309194141 >> 172.21.15.86:0/2309194141 conn(0x55ab34ae3770 msgr2=0x55ab34aec4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3cc8a70740 1 -- 172.21.15.86:0/2309194141 shutdown_connections 2024-04-17T07:32:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.393+0000 7f3cc8a70740 1 -- 172.21.15.86:0/2309194141 wait complete. 2024-04-17T07:32:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.394+0000 7f3cc8a70740 20 client.19587 trim_cache size 0 max 16384 2024-04-17T07:32:48.395 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.395+0000 7f3cc5e00640 1 -- 172.21.15.86:0/2254541472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ca40a3c30 msgr2=0x7f3ca40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.395+0000 7f3cc5e00640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ca40a3c30 0x7f3ca40a60f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80994e0 tx=0x7f3cb80095d0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.395+0000 7f3cc5e00640 1 -- 172.21.15.86:0/2254541472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0109e30 msgr2=0x7f3cc01a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.395+0000 7f3cc5e00640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0109e30 0x7f3cc01a5de0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800e970 tx=0x7f3cb800ed70 comp rx=0 tx=0).stop 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.395+0000 7f3cc5e00640 1 -- 172.21.15.86:0/2254541472 shutdown_connections 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.396+0000 7f3cc5e00640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f3ca40a3c30 0x7f3ca40a60f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.396+0000 7f3cc5e00640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cc0109e30 0x7f3cc01a5de0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.396+0000 7f3cc5e00640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc0075f40 0x7f3cc019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.396+0000 7f3cc5e00640 1 --2- 172.21.15.86:0/2254541472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0074960 0x7f3cc019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.396+0000 7f3cc5e00640 1 -- 172.21.15.86:0/2254541472 >> 172.21.15.86:0/2254541472 conn(0x7f3cc00fd510 msgr2=0x7f3cc01019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.396+0000 7f3cc5e00640 1 -- 172.21.15.86:0/2254541472 shutdown_connections 2024-04-17T07:32:48.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.397+0000 7f3cc5e00640 1 -- 172.21.15.86:0/2254541472 wait complete. 2024-04-17T07:32:48.433 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:32:48.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.804+0000 7f9f75c00640 1 -- 172.21.15.86:0/323302722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f7010e2a0 msgr2=0x7f9f7010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.804+0000 7f9f75c00640 1 --2- 172.21.15.86:0/323302722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f7010e2a0 0x7f9f7010e750 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009be0 tx=0x7f9f6802bef0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.805+0000 7f9f75c00640 1 -- 172.21.15.86:0/323302722 shutdown_connections 2024-04-17T07:32:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.805+0000 7f9f75c00640 1 --2- 172.21.15.86:0/323302722 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f7010e2a0 0x7f9f7010e750 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.805+0000 7f9f75c00640 1 --2- 172.21.15.86:0/323302722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7010b8d0 0x7f9f7010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.805+0000 7f9f75c00640 1 --2- 172.21.15.86:0/323302722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f70108f60 0x7f9f7010b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.805+0000 7f9f75c00640 1 -- 172.21.15.86:0/323302722 >> 172.21.15.86:0/323302722 conn(0x7f9f7006edf0 msgr2=0x7f9f70071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.808+0000 7f9f75c00640 1 -- 172.21.15.86:0/323302722 shutdown_connections 2024-04-17T07:32:48.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.808+0000 7f9f75c00640 1 -- 172.21.15.86:0/323302722 wait complete. 2024-04-17T07:32:48.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.810+0000 7f9f75c00640 1 Processor -- start 2024-04-17T07:32:48.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.811+0000 7f9f75c00640 1 -- start start 2024-04-17T07:32:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.811+0000 7f9f75c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f70108f60 0x7f9f7019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.811+0000 7f9f75c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7010b8d0 0x7f9f7019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.812+0000 7f9f75c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f7010e2a0 0x7f9f701a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.812+0000 7f9f6e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7010b8d0 0x7f9f7019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.812+0000 7f9f75c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f70112130 con 0x7f9f70108f60 2024-04-17T07:32:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.812+0000 7f9f75c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f70111fb0 con 0x7f9f7010b8d0 2024-04-17T07:32:48.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.812+0000 7f9f6e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7010b8d0 0x7f9f7019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36784/0 (socket says 172.21.15.86:36784) 2024-04-17T07:32:48.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.812+0000 7f9f6f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f7010e2a0 0x7f9f701a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.812+0000 7f9f6f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f7010e2a0 0x7f9f701a5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40776/0 (socket says 172.21.15.86:40776) 2024-04-17T07:32:48.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.812+0000 7f9f6f400640 1 -- 172.21.15.86:0/1433952007 learned_addr learned my addr 172.21.15.86:0/1433952007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:48.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.812+0000 7f9f75c00640 1 -- 172.21.15.86:0/1433952007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f701122b0 con 0x7f9f7010e2a0 2024-04-17T07:32:48.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.813+0000 7f9f6f400640 1 -- 172.21.15.86:0/1433952007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7010b8d0 msgr2=0x7f9f7019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.813+0000 7f9f6f400640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7010b8d0 0x7f9f7019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.813+0000 7f9f6f400640 1 -- 172.21.15.86:0/1433952007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f70108f60 msgr2=0x7f9f7019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.813+0000 7f9f6f400640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f70108f60 0x7f9f7019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.813+0000 7f9f6f400640 1 -- 172.21.15.86:0/1433952007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f701a6460 con 0x7f9f7010e2a0 2024-04-17T07:32:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.814+0000 7f9f6f400640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f7010e2a0 0x7f9f701a5df0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9f68034fd0 tx=0x7f9f68034700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.814+0000 7f9f67400640 1 -- 172.21.15.86:0/1433952007 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68043020 con 0x7f9f7010e2a0 2024-04-17T07:32:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.815+0000 7f9f75c00640 1 -- 172.21.15.86:0/1433952007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f701a66f0 con 0x7f9f7010e2a0 2024-04-17T07:32:48.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.815+0000 7f9f75c00640 1 -- 172.21.15.86:0/1433952007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f701a6d10 con 0x7f9f7010e2a0 2024-04-17T07:32:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.816+0000 7f9f67400640 1 -- 172.21.15.86:0/1433952007 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f6802ca80 con 0x7f9f7010e2a0 2024-04-17T07:32:48.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.816+0000 7f9f67400640 1 -- 172.21.15.86:0/1433952007 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68043020 con 0x7f9f7010e2a0 2024-04-17T07:32:48.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.824+0000 7f9f67400640 1 -- 172.21.15.86:0/1433952007 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9f68046020 con 0x7f9f7010e2a0 2024-04-17T07:32:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.830+0000 7f9f67400640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9f540a3ef0 0x7f9f540a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.831+0000 7f9f67400640 1 -- 172.21.15.86:0/1433952007 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9f680f2bb0 con 0x7f9f7010e2a0 2024-04-17T07:32:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.832+0000 7f9f78a2a740 1 Processor -- start 2024-04-17T07:32:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.832+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9f540a3ef0 0x7f9f540a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.833+0000 7f9f78a2a740 1 -- start start 2024-04-17T07:32:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.834+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 0x5636a86db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.834+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86db990 0x5636a86e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.834+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.834+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5636a8641020 con 0x5636a86dafa0 2024-04-17T07:32:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.834+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5636a85e7460 con 0x5636a86e0b00 2024-04-17T07:32:48.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.834+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5636a863cfe0 con 0x5636a86db990 2024-04-17T07:32:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.834+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9f540a3ef0 0x7f9f540a63b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005ab0 tx=0x7f9f60005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.835+0000 7f9f6ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 0x5636a86db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.835+0000 7f9f6e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86db990 0x5636a86e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.835+0000 7f9f6f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.835+0000 7f9f6e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86db990 0x5636a86e0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40790/0 (socket says 172.21.15.86:40790) 2024-04-17T07:32:48.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.835+0000 7f9f6e000640 1 -- 172.21.15.86:0/3000148636 learned_addr learned my addr 172.21.15.86:0/3000148636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:48.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.836+0000 7f9f6f400640 1 -- 172.21.15.86:0/3000148636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86db990 msgr2=0x5636a86e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.836+0000 7f9f6f400640 1 --2- 172.21.15.86:0/3000148636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86db990 0x5636a86e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.836+0000 7f9f6f400640 1 -- 172.21.15.86:0/3000148636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 msgr2=0x5636a86db3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.837+0000 7f9f6f400640 1 --2- 172.21.15.86:0/3000148636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 0x5636a86db3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.837+0000 7f9f6f400640 1 -- 172.21.15.86:0/3000148636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a85ef040 con 0x5636a86e0b00 2024-04-17T07:32:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.837+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/3000148636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 0x5636a86db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.837+0000 7f9f6f400640 1 --2- 172.21.15.86:0/3000148636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e2f90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9f6802c4d0 tx=0x7f9f680be410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=580942adafcdb133 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.837+0000 7f9f5be00640 1 -- 172.21.15.86:0/3000148636 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f680c2e60 con 0x5636a86e0b00 2024-04-17T07:32:48.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.838+0000 7f9f5be00640 1 -- 172.21.15.86:0/3000148636 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f680c7720 con 0x5636a86e0b00 2024-04-17T07:32:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.838+0000 7f9f5be00640 1 -- 172.21.15.86:0/3000148636 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f680c2e60 con 0x5636a86e0b00 2024-04-17T07:32:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.839+0000 7f9f78a2a740 1 -- 172.21.15.86:0/3000148636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 msgr2=0x5636a86e2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.839+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/3000148636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e2f90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9f6802c4d0 tx=0x7f9f680be410 comp rx=0 tx=0).stop 2024-04-17T07:32:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.840+0000 7f9f78a2a740 1 -- 172.21.15.86:0/3000148636 shutdown_connections 2024-04-17T07:32:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.840+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/3000148636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e2f90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.840+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/3000148636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86db990 0x5636a86e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.840+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/3000148636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 0x5636a86db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.840+0000 7f9f78a2a740 1 -- 172.21.15.86:0/3000148636 >> 172.21.15.86:0/3000148636 conn(0x5636a86d6510 msgr2=0x5636a86d6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.841+0000 7f9f78a2a740 1 -- 172.21.15.86:0/3000148636 shutdown_connections 2024-04-17T07:32:48.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.841+0000 7f9f78a2a740 1 -- 172.21.15.86:0/3000148636 wait complete. 2024-04-17T07:32:48.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.842+0000 7f9f78a2a740 1 -- start start 2024-04-17T07:32:48.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.842+0000 7f9f78a2a740 1 Processor -- start 2024-04-17T07:32:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.844+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 0x5636a86e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.844+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86db990 0x5636a86e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.844+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86e0b00 0x5636a86ec820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.844+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5636a86e3870 con 0x5636a86dafa0 2024-04-17T07:32:48.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.844+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5636a85fa830 con 0x5636a86db990 2024-04-17T07:32:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.844+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5636a86e39f0 con 0x5636a86e0b00 2024-04-17T07:32:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.844+0000 7f9f78a2a740 10 client.0 _ll_register_callbacks cb 0x5636a85c8fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.845+0000 7f9f6e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86db990 0x5636a86e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.845+0000 7f9f6e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86db990 0x5636a86e57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36802/0 (socket says 172.21.15.86:36802) 2024-04-17T07:32:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.845+0000 7f9f6e000640 1 -- 172.21.15.86:0/2489566953 learned_addr learned my addr 172.21.15.86:0/2489566953 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.845+0000 7f9f6f400640 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86e0b00 0x5636a86ec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.846+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 0x5636a86e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.846+0000 7f9f6ea00640 1 -- 172.21.15.86:0/2489566953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86e0b00 msgr2=0x5636a86ec820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.846+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86e0b00 0x5636a86ec820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.846+0000 7f9f6ea00640 1 -- 172.21.15.86:0/2489566953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86db990 msgr2=0x5636a86e57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.846+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86db990 0x5636a86e57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.846+0000 7f9f6ea00640 1 -- 172.21.15.86:0/2489566953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a85fe920 con 0x5636a86dafa0 2024-04-17T07:32:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.848+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 0x5636a86e52a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005fd0 tx=0x7f9f600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecb3fc048624d29 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.848+0000 7f9f58c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.849+0000 7f9f58c00640 1 -- 172.21.15.86:0/2489566953 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60003d70 con 0x5636a86dafa0 2024-04-17T07:32:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.849+0000 7f9f58c00640 1 -- 172.21.15.86:0/2489566953 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9f60038470 con 0x5636a86dafa0 2024-04-17T07:32:48.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.850+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2489566953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f680c7720 con 0x5636a86dafa0 2024-04-17T07:32:48.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.850+0000 7f9f58c00640 1 -- 172.21.15.86:0/2489566953 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9f60038670 con 0x5636a86dafa0 2024-04-17T07:32:48.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.850+0000 7f9f78a2a740 10 client.19596 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:48.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.850+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2489566953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636a85cc7f0 con 0x5636a86dafa0 2024-04-17T07:32:48.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.854+0000 7f9f58c00640 1 -- 172.21.15.86:0/2489566953 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f680c7720 con 0x5636a86dafa0 2024-04-17T07:32:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.854+0000 7f9f58c00640 1 -- 172.21.15.86:0/2489566953 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9f600218f0 con 0x5636a86dafa0 2024-04-17T07:32:48.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.854+0000 7f9f78a2a740 10 client.19596 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:48.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.854+0000 7f9f78a2a740 10 client.19596 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:32:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.855+0000 7f9f78a2a740 20 client.19596 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.855+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5636a86f0e00 0x5636a86f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.855+0000 7f9f78a2a740 4 client.19596 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.855+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2489566953 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636a864ea40 con 0x5636a86f0e00 2024-04-17T07:32:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.855+0000 7f9f6e000640 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5636a86f0e00 0x5636a86f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.860+0000 7f9f6e000640 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5636a86f0e00 0x5636a86f32a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14b22655251e32c server_cookie=39ea4a8ba1dfef2d in_seq=0 out_seq=0 2024-04-17T07:32:48.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.860+0000 7f9f58c00640 10 client.19596 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:32:48.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.862+0000 7f9f58c00640 1 -- 172.21.15.86:0/2489566953 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636a864ea40 con 0x5636a86f0e00 2024-04-17T07:32:48.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.862+0000 7f9f58c00640 10 client.19596 handle_command_reply: tid=0 2024-04-17T07:32:48.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.862+0000 7f9f78a2a740 1 client.19596 shutdown 2024-04-17T07:32:48.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.863+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2489566953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 msgr2=0x5636a86e52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.863+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 0x5636a86e52a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005fd0 tx=0x7f9f600046c0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.864+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2489566953 shutdown_connections 2024-04-17T07:32:48.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.864+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5636a86f0e00 0x5636a86f32a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.864+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86e0b00 0x5636a86ec820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.864+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86db990 0x5636a86e57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.864+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/2489566953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86dafa0 0x5636a86e52a0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.864+0000 7f9f58c00640 0 client.19596 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:32:48.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.864+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2489566953 >> 172.21.15.86:0/2489566953 conn(0x5636a86d6510 msgr2=0x5636a86d5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.864+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2489566953 shutdown_connections 2024-04-17T07:32:48.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.864+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2489566953 wait complete. 2024-04-17T07:32:48.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.865+0000 7f9f78a2a740 20 client.19596 trim_cache size 0 max 16384 2024-04-17T07:32:48.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.897+0000 7f9f78a2a740 1 Processor -- start 2024-04-17T07:32:48.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.897+0000 7f9f78a2a740 1 -- start start 2024-04-17T07:32:48.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.897+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86dafa0 0x5636a86d9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.897+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 0x5636a86ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.897+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.897+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f701b1170 con 0x5636a86db990 2024-04-17T07:32:48.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.897+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f701b2cb0 con 0x5636a86e0b00 2024-04-17T07:32:48.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.897+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f701b1840 con 0x5636a86dafa0 2024-04-17T07:32:48.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.898+0000 7f9f6f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.898+0000 7f9f6e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 0x5636a86ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.898+0000 7f9f6f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36818/0 (socket says 172.21.15.86:36818) 2024-04-17T07:32:48.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.898+0000 7f9f6f400640 1 -- 172.21.15.86:0/2440532487 learned_addr learned my addr 172.21.15.86:0/2440532487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:48.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.899+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/2440532487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86dafa0 0x5636a86d9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.899+0000 7f9f6e000640 1 -- 172.21.15.86:0/2440532487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86dafa0 msgr2=0x5636a86d9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.899+0000 7f9f6e000640 1 --2- 172.21.15.86:0/2440532487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86dafa0 0x5636a86d9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.899+0000 7f9f6e000640 1 -- 172.21.15.86:0/2440532487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 msgr2=0x5636a86e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.899+0000 7f9f6e000640 1 --2- 172.21.15.86:0/2440532487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.900+0000 7f9f6e000640 1 -- 172.21.15.86:0/2440532487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a83533f0 con 0x5636a86db990 2024-04-17T07:32:48.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.900+0000 7f9f6f400640 1 --2- 172.21.15.86:0/2440532487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:32:48.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.901+0000 7f9f6e000640 1 --2- 172.21.15.86:0/2440532487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 0x5636a86ec7e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x5636a86e6780 tx=0x7f9f5c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a6cf86886d8f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.902+0000 7f9f5b400640 1 -- 172.21.15.86:0/2440532487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c017a10 con 0x5636a86db990 2024-04-17T07:32:48.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.902+0000 7f9f5b400640 1 -- 172.21.15.86:0/2440532487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f5c01e780 con 0x5636a86db990 2024-04-17T07:32:48.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.902+0000 7f9f5b400640 1 -- 172.21.15.86:0/2440532487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c017a10 con 0x5636a86db990 2024-04-17T07:32:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.904+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2440532487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 msgr2=0x5636a86ec7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.904+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/2440532487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 0x5636a86ec7e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x5636a86e6780 tx=0x7f9f5c008040 comp rx=0 tx=0).stop 2024-04-17T07:32:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.905+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2440532487 shutdown_connections 2024-04-17T07:32:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.905+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/2440532487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86e0b00 0x5636a86e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.905+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/2440532487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 0x5636a86ec7e0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.905+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/2440532487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86dafa0 0x5636a86d9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.905+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2440532487 >> 172.21.15.86:0/2440532487 conn(0x5636a86d6510 msgr2=0x5636a86d9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.905+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2440532487 shutdown_connections 2024-04-17T07:32:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.905+0000 7f9f78a2a740 1 -- 172.21.15.86:0/2440532487 wait complete. 2024-04-17T07:32:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.906+0000 7f9f78a2a740 1 -- start start 2024-04-17T07:32:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.910+0000 7f9f78a2a740 1 Processor -- start 2024-04-17T07:32:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.914+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86dafa0 0x5636a86e9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.914+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 0x5636a86e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.916+0000 7f9f78a2a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86e0b00 0x5636a8755380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.916+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f701b25e0 con 0x5636a86db990 2024-04-17T07:32:48.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.916+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f701b1f10 con 0x5636a86dafa0 2024-04-17T07:32:48.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.916+0000 7f9f78a2a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f701af100 con 0x5636a86e0b00 2024-04-17T07:32:48.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.916+0000 7f9f78a2a740 10 client.0 _ll_register_callbacks cb 0x7f9f60004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:48.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.919+0000 7f9f6ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86dafa0 0x5636a86e9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.920+0000 7f9f6e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 0x5636a86e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.920+0000 7f9f6ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86dafa0 0x5636a86e9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36820/0 (socket says 172.21.15.86:36820) 2024-04-17T07:32:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.920+0000 7f9f6ea00640 1 -- 172.21.15.86:0/4080787538 learned_addr learned my addr 172.21.15.86:0/4080787538 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.920+0000 7f9f6f400640 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86e0b00 0x5636a8755380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.921+0000 7f9f6ea00640 1 -- 172.21.15.86:0/4080787538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86e0b00 msgr2=0x5636a8755380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.921+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86e0b00 0x5636a8755380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.921+0000 7f9f6ea00640 1 -- 172.21.15.86:0/4080787538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 msgr2=0x5636a86e9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.922+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 0x5636a86e9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.922+0000 7f9f6ea00640 1 -- 172.21.15.86:0/4080787538 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a841d7d0 con 0x5636a86dafa0 2024-04-17T07:32:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.922+0000 7f9f6e000640 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 0x5636a86e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.923+0000 7f9f6ea00640 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86dafa0 0x5636a86e9780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f60037000 tx=0x7f9f6001a900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ff285b57688420a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.923+0000 7f9f58c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.924+0000 7f9f58c00640 1 -- 172.21.15.86:0/4080787538 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6004f020 con 0x5636a86dafa0 2024-04-17T07:32:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.924+0000 7f9f58c00640 1 -- 172.21.15.86:0/4080787538 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9f600413d0 con 0x5636a86dafa0 2024-04-17T07:32:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.924+0000 7f9f78a2a740 1 -- 172.21.15.86:0/4080787538 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f5c01e780 con 0x5636a86dafa0 2024-04-17T07:32:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.925+0000 7f9f58c00640 1 -- 172.21.15.86:0/4080787538 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9f60047040 con 0x5636a86dafa0 2024-04-17T07:32:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.925+0000 7f9f78a2a740 10 client.38633 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.925+0000 7f9f78a2a740 1 -- 172.21.15.86:0/4080787538 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636a8756a30 con 0x5636a86dafa0 2024-04-17T07:32:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.926+0000 7f9f58c00640 1 -- 172.21.15.86:0/4080787538 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f5c01e780 con 0x5636a86dafa0 2024-04-17T07:32:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.926+0000 7f9f58c00640 1 -- 172.21.15.86:0/4080787538 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9f6004a070 con 0x5636a86dafa0 2024-04-17T07:32:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.927+0000 7f9f78a2a740 10 client.38633 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.927+0000 7f9f78a2a740 10 client.38633 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:32:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.927+0000 7f9f78a2a740 20 client.38633 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.927+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5636a8757370 0x5636a8759790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.928+0000 7f9f78a2a740 4 client.38633 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:32:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.928+0000 7f9f78a2a740 1 -- 172.21.15.86:0/4080787538 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636a86488b0 con 0x5636a8757370 2024-04-17T07:32:48.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.928+0000 7f9f6e000640 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5636a8757370 0x5636a8759790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.929+0000 7f9f6e000640 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5636a8757370 0x5636a8759790 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ed834d40d886438 server_cookie=183dc09a1b68d508 in_seq=0 out_seq=0 2024-04-17T07:32:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.929+0000 7f9f58c00640 10 client.38633 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:32:48.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.930+0000 7f9f58c00640 1 -- 172.21.15.86:0/4080787538 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636a86488b0 con 0x5636a8757370 2024-04-17T07:32:48.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.930+0000 7f9f58c00640 10 client.38633 handle_command_reply: tid=0 2024-04-17T07:32:48.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.930+0000 7f9f78a2a740 1 client.38633 shutdown 2024-04-17T07:32:48.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.931+0000 7f9f78a2a740 1 -- 172.21.15.86:0/4080787538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86dafa0 msgr2=0x5636a86e9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.931+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86dafa0 0x5636a86e9780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f60037000 tx=0x7f9f6001a900 comp rx=0 tx=0).stop 2024-04-17T07:32:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.932+0000 7f9f78a2a740 1 -- 172.21.15.86:0/4080787538 shutdown_connections 2024-04-17T07:32:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.932+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5636a8757370 0x5636a8759790 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.932+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5636a86e0b00 0x5636a8755380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.932+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636a86db990 0x5636a86e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.932+0000 7f9f78a2a740 1 --2- 172.21.15.86:0/4080787538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5636a86dafa0 0x5636a86e9780 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.932+0000 7f9f78a2a740 1 -- 172.21.15.86:0/4080787538 >> 172.21.15.86:0/4080787538 conn(0x5636a86d6510 msgr2=0x5636a86da000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.932+0000 7f9f58c00640 0 client.38633 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:32:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.933+0000 7f9f78a2a740 1 -- 172.21.15.86:0/4080787538 shutdown_connections 2024-04-17T07:32:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.933+0000 7f9f78a2a740 1 -- 172.21.15.86:0/4080787538 wait complete. 2024-04-17T07:32:48.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.933+0000 7f9f78a2a740 20 client.38633 trim_cache size 0 max 16384 2024-04-17T07:32:48.936 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:32:48.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.936+0000 7f9f75c00640 1 -- 172.21.15.86:0/1433952007 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9f540a3ef0 msgr2=0x7f9f540a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.936+0000 7f9f75c00640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9f540a3ef0 0x7f9f540a63b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005ab0 tx=0x7f9f60005a20 comp rx=0 tx=0).stop 2024-04-17T07:32:48.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.936+0000 7f9f75c00640 1 -- 172.21.15.86:0/1433952007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f7010e2a0 msgr2=0x7f9f701a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:48.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.936+0000 7f9f75c00640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f7010e2a0 0x7f9f701a5df0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9f68034fd0 tx=0x7f9f68034700 comp rx=0 tx=0).stop 2024-04-17T07:32:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.937+0000 7f9f75c00640 1 -- 172.21.15.86:0/1433952007 shutdown_connections 2024-04-17T07:32:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.938+0000 7f9f75c00640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9f540a3ef0 0x7f9f540a63b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.938+0000 7f9f75c00640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f7010e2a0 0x7f9f701a5df0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.939+0000 7f9f75c00640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7010b8d0 0x7f9f7019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.939+0000 7f9f75c00640 1 --2- 172.21.15.86:0/1433952007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f70108f60 0x7f9f7019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:48.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.939+0000 7f9f75c00640 1 -- 172.21.15.86:0/1433952007 >> 172.21.15.86:0/1433952007 conn(0x7f9f7006edf0 msgr2=0x7f9f701097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:48.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.939+0000 7f9f75c00640 1 -- 172.21.15.86:0/1433952007 shutdown_connections 2024-04-17T07:32:48.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:48.939+0000 7f9f75c00640 1 -- 172.21.15.86:0/1433952007 wait complete. 2024-04-17T07:32:48.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:32:49.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:48 smithi098 ceph-mon[35418]: pgmap v993: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 8.9 MiB/s wr, 605 op/s 2024-04-17T07:32:49.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:48 smithi103 ceph-mon[35495]: pgmap v993: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 8.9 MiB/s wr, 605 op/s 2024-04-17T07:32:49.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:48 smithi086 ceph-mon[31733]: pgmap v993: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 8.9 MiB/s wr, 605 op/s 2024-04-17T07:32:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 805.0 s, 562.4 tps, lat 176.598 ms stddev 67.389 2024-04-17T07:32:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.417+0000 7f1964400640 1 -- 172.21.15.86:0/234838169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f195c1053d0 msgr2=0x7f195c105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.417+0000 7f1964400640 1 --2- 172.21.15.86:0/234838169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f195c1053d0 0x7f195c105850 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1954009ca0 tx=0x7f195402c100 comp rx=0 tx=0).stop 2024-04-17T07:32:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.418+0000 7f1964400640 1 -- 172.21.15.86:0/234838169 shutdown_connections 2024-04-17T07:32:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.418+0000 7f1964400640 1 --2- 172.21.15.86:0/234838169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f195c105d90 0x7f195c10c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.418+0000 7f1964400640 1 --2- 172.21.15.86:0/234838169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f195c1053d0 0x7f195c105850 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.418+0000 7f1964400640 1 --2- 172.21.15.86:0/234838169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f195c1041e0 0x7f195c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.418+0000 7f1964400640 1 -- 172.21.15.86:0/234838169 >> 172.21.15.86:0/234838169 conn(0x7f195c0ff750 msgr2=0x7f195c101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.419+0000 7f1964400640 1 -- 172.21.15.86:0/234838169 shutdown_connections 2024-04-17T07:32:49.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.419+0000 7f1964400640 1 -- 172.21.15.86:0/234838169 wait complete. 2024-04-17T07:32:49.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.423+0000 7f1964400640 1 Processor -- start 2024-04-17T07:32:49.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.423+0000 7f1964400640 1 -- start start 2024-04-17T07:32:49.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.424+0000 7f1964400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f195c1041e0 0x7f195c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.424+0000 7f1964400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f195c1053d0 0x7f195c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.424+0000 7f1964400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f195c105d90 0x7f195c1a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.424+0000 7f1964400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f195c10fd70 con 0x7f195c105d90 2024-04-17T07:32:49.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.425+0000 7f1964400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f195c10fbf0 con 0x7f195c1041e0 2024-04-17T07:32:49.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.425+0000 7f1961000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f195c1053d0 0x7f195c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.425+0000 7f1964400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f195c10fef0 con 0x7f195c1053d0 2024-04-17T07:32:49.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.425+0000 7f1961a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f195c1041e0 0x7f195c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.425+0000 7f1961a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f195c1041e0 0x7f195c19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36832/0 (socket says 172.21.15.86:36832) 2024-04-17T07:32:49.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.425+0000 7f1961000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f195c1053d0 0x7f195c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40836/0 (socket says 172.21.15.86:40836) 2024-04-17T07:32:49.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.425+0000 7f1961a00640 1 -- 172.21.15.86:0/3806333493 learned_addr learned my addr 172.21.15.86:0/3806333493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:49.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.425+0000 7f1962400640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f195c105d90 0x7f195c1a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.426+0000 7f1961000640 1 -- 172.21.15.86:0/3806333493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f195c1041e0 msgr2=0x7f195c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.427+0000 7f1961000640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f195c1041e0 0x7f195c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.427+0000 7f1961000640 1 -- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f195c105d90 msgr2=0x7f195c1a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.427+0000 7f1961000640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f195c105d90 0x7f195c1a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.427+0000 7f1961000640 1 -- 172.21.15.86:0/3806333493 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c1a6460 con 0x7f195c1053d0 2024-04-17T07:32:49.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.428+0000 7f1962400640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f195c105d90 0x7f195c1a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:32:49.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.428+0000 7f1961000640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f195c1053d0 0x7f195c19ed80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f195402c710 tx=0x7f1954035ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:49.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.428+0000 7f1961a00640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f195c1041e0 0x7f195c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:32:49.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.430+0000 7f195a000640 1 -- 172.21.15.86:0/3806333493 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954039040 con 0x7f195c1053d0 2024-04-17T07:32:49.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.430+0000 7f195a000640 1 -- 172.21.15.86:0/3806333493 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1954003970 con 0x7f195c1053d0 2024-04-17T07:32:49.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.431+0000 7f195a000640 1 -- 172.21.15.86:0/3806333493 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540044a0 con 0x7f195c1053d0 2024-04-17T07:32:49.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.431+0000 7f1964400640 1 -- 172.21.15.86:0/3806333493 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195c1a66f0 con 0x7f195c1053d0 2024-04-17T07:32:49.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.431+0000 7f1964400640 1 -- 172.21.15.86:0/3806333493 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195c1a6b60 con 0x7f195c1053d0 2024-04-17T07:32:49.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.436+0000 7f1967233740 1 Processor -- start 2024-04-17T07:32:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.437+0000 7f1967233740 1 -- start start 2024-04-17T07:32:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.437+0000 7f1967233740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.438+0000 7f1967233740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec0990 0x5572b4ec5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.439+0000 7f1967233740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec5b00 0x5572b4ec7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.439+0000 7f1967233740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5572b4e26020 con 0x5572b4ec5b00 2024-04-17T07:32:49.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.439+0000 7f1967233740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5572b4dcc460 con 0x5572b4ebffa0 2024-04-17T07:32:49.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.439+0000 7f1967233740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5572b4e21fe0 con 0x5572b4ec0990 2024-04-17T07:32:49.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.440+0000 7f195a000640 1 -- 172.21.15.86:0/3806333493 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1954046030 con 0x7f195c1053d0 2024-04-17T07:32:49.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.440+0000 7f1962400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.440+0000 7f1962400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36834/0 (socket says 172.21.15.86:36834) 2024-04-17T07:32:49.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.440+0000 7f1962400640 1 -- 172.21.15.86:0/580246827 learned_addr learned my addr 172.21.15.86:0/580246827 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:49.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.440+0000 7f1961a00640 1 --2- 172.21.15.86:0/580246827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec0990 0x5572b4ec5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.441+0000 7f1961000640 1 --2- 172.21.15.86:0/580246827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec5b00 0x5572b4ec7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.441+0000 7f1962400640 1 -- 172.21.15.86:0/580246827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec0990 msgr2=0x5572b4ec5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.441+0000 7f1962400640 1 --2- 172.21.15.86:0/580246827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec0990 0x5572b4ec5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.441+0000 7f1962400640 1 -- 172.21.15.86:0/580246827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec5b00 msgr2=0x5572b4ec7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.441+0000 7f1962400640 1 --2- 172.21.15.86:0/580246827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec5b00 0x5572b4ec7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.441+0000 7f1962400640 1 -- 172.21.15.86:0/580246827 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b4dd4040 con 0x5572b4ebffa0 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.442+0000 7f1962400640 1 --2- 172.21.15.86:0/580246827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec03c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f194800bca0 tx=0x7f194800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6d05fb47937d063 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.442+0000 7f195a000640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19440a3d30 0x7f19440a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.442+0000 7f194ea00640 1 -- 172.21.15.86:0/580246827 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948015070 con 0x5572b4ebffa0 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.442+0000 7f195a000640 1 -- 172.21.15.86:0/3806333493 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f19540f21d0 con 0x7f195c1053d0 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.442+0000 7f194ea00640 1 -- 172.21.15.86:0/580246827 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19480054d0 con 0x5572b4ebffa0 2024-04-17T07:32:49.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.442+0000 7f194ea00640 1 -- 172.21.15.86:0/580246827 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19480116a0 con 0x5572b4ebffa0 2024-04-17T07:32:49.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.444+0000 7f1967233740 1 -- 172.21.15.86:0/580246827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 msgr2=0x5572b4ec03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.444+0000 7f1967233740 1 --2- 172.21.15.86:0/580246827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec03c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f194800bca0 tx=0x7f194800f730 comp rx=0 tx=0).stop 2024-04-17T07:32:49.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.444+0000 7f1961a00640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19440a3d30 0x7f19440a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.445+0000 7f1967233740 1 -- 172.21.15.86:0/580246827 shutdown_connections 2024-04-17T07:32:49.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.445+0000 7f1967233740 1 --2- 172.21.15.86:0/580246827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec5b00 0x5572b4ec7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.445+0000 7f1967233740 1 --2- 172.21.15.86:0/580246827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec0990 0x5572b4ec5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.445+0000 7f1967233740 1 --2- 172.21.15.86:0/580246827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec03c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.445+0000 7f1967233740 1 -- 172.21.15.86:0/580246827 >> 172.21.15.86:0/580246827 conn(0x5572b4ebb510 msgr2=0x5572b4ebb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:49.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.446+0000 7f1961a00640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19440a3d30 0x7f19440a61f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x5572b4ec0d90 tx=0x7f195000b340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:49.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.446+0000 7f1967233740 1 -- 172.21.15.86:0/580246827 shutdown_connections 2024-04-17T07:32:49.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.446+0000 7f1967233740 1 -- 172.21.15.86:0/580246827 wait complete. 2024-04-17T07:32:49.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.447+0000 7f1967233740 1 -- start start 2024-04-17T07:32:49.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.447+0000 7f1967233740 1 Processor -- start 2024-04-17T07:32:49.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.447+0000 7f1967233740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ebffa0 0x5572b4eca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.447+0000 7f1967233740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec0990 0x5572b4ecaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.448+0000 7f1967233740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ec5b00 0x5572b4ed1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.448+0000 7f1967233740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5572b4ec8ab0 con 0x5572b4ec0990 2024-04-17T07:32:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.448+0000 7f1967233740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5572b4ddf830 con 0x5572b4ec5b00 2024-04-17T07:32:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.448+0000 7f1967233740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5572b4ec8c30 con 0x5572b4ebffa0 2024-04-17T07:32:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.448+0000 7f1967233740 10 client.0 _ll_register_callbacks cb 0x5572b4dadfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.449+0000 7f1961a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ebffa0 0x5572b4eca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.449+0000 7f1961a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ebffa0 0x5572b4eca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40850/0 (socket says 172.21.15.86:40850) 2024-04-17T07:32:49.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.449+0000 7f1961a00640 1 -- 172.21.15.86:0/3962346813 learned_addr learned my addr 172.21.15.86:0/3962346813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:49.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.449+0000 7f1962400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ec5b00 0x5572b4ed1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.450+0000 7f1961a00640 1 -- 172.21.15.86:0/3962346813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ec5b00 msgr2=0x5572b4ed1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.450+0000 7f1961a00640 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ec5b00 0x5572b4ed1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.450+0000 7f1961a00640 1 -- 172.21.15.86:0/3962346813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec0990 msgr2=0x5572b4ecaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.450+0000 7f1961a00640 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec0990 0x5572b4ecaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.450+0000 7f1961a00640 1 -- 172.21.15.86:0/3962346813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b4de3920 con 0x5572b4ebffa0 2024-04-17T07:32:49.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.451+0000 7f1961a00640 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ebffa0 0x5572b4eca540 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f19500126d0 tx=0x7f1950004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61150208d01e9bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:49.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.452+0000 7f1943400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:32:49.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.452+0000 7f1943400640 1 -- 172.21.15.86:0/3962346813 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950023070 con 0x5572b4ebffa0 2024-04-17T07:32:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.452+0000 7f1943400640 1 -- 172.21.15.86:0/3962346813 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1950011930 con 0x5572b4ebffa0 2024-04-17T07:32:49.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.453+0000 7f1967233740 1 -- 172.21.15.86:0/3962346813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19480054d0 con 0x5572b4ebffa0 2024-04-17T07:32:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.454+0000 7f1943400640 1 -- 172.21.15.86:0/3962346813 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f195002d050 con 0x5572b4ebffa0 2024-04-17T07:32:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.455+0000 7f1967233740 10 client.28639 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.455+0000 7f1967233740 1 -- 172.21.15.86:0/3962346813 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5572b4db17f0 con 0x5572b4ebffa0 2024-04-17T07:32:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.456+0000 7f1943400640 1 -- 172.21.15.86:0/3962346813 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19480054d0 con 0x5572b4ebffa0 2024-04-17T07:32:49.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.456+0000 7f1943400640 1 -- 172.21.15.86:0/3962346813 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f195000f040 con 0x5572b4ebffa0 2024-04-17T07:32:49.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.456+0000 7f1967233740 10 client.28639 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:49.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.456+0000 7f1967233740 10 client.28639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:32:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.456+0000 7f1967233740 20 client.28639 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.456+0000 7f1967233740 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5572b4ed6410 0x5572b4ed8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.456+0000 7f1967233740 4 client.28639 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.456+0000 7f1967233740 1 -- 172.21.15.86:0/3962346813 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b4e33a40 con 0x5572b4ed6410 2024-04-17T07:32:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.457+0000 7f1961000640 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5572b4ed6410 0x5572b4ed8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.459+0000 7f1943400640 10 client.28639 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:32:49.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.460+0000 7f1961000640 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5572b4ed6410 0x5572b4ed8890 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3e69684bfb0fb4b server_cookie=27bb8a04f3d72080 in_seq=0 out_seq=0 2024-04-17T07:32:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.462+0000 7f1943400640 1 -- 172.21.15.86:0/3962346813 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572b4e33a40 con 0x5572b4ed6410 2024-04-17T07:32:49.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.462+0000 7f1943400640 10 client.28639 handle_command_reply: tid=0 2024-04-17T07:32:49.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.463+0000 7f1967233740 1 client.28639 shutdown 2024-04-17T07:32:49.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.463+0000 7f1967233740 1 -- 172.21.15.86:0/3962346813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ebffa0 msgr2=0x5572b4eca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.463+0000 7f1967233740 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ebffa0 0x5572b4eca540 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f19500126d0 tx=0x7f1950004730 comp rx=0 tx=0).stop 2024-04-17T07:32:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.463+0000 7f1967233740 1 -- 172.21.15.86:0/3962346813 shutdown_connections 2024-04-17T07:32:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.463+0000 7f1967233740 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5572b4ed6410 0x5572b4ed8890 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.463+0000 7f1943400640 0 client.28639 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:32:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.463+0000 7f1967233740 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ec5b00 0x5572b4ed1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.463+0000 7f1967233740 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec0990 0x5572b4ecaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.463+0000 7f1967233740 1 --2- 172.21.15.86:0/3962346813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ebffa0 0x5572b4eca540 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.463+0000 7f1967233740 1 -- 172.21.15.86:0/3962346813 >> 172.21.15.86:0/3962346813 conn(0x5572b4ebb510 msgr2=0x5572b4eba710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.464+0000 7f1967233740 1 -- 172.21.15.86:0/3962346813 shutdown_connections 2024-04-17T07:32:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.464+0000 7f1967233740 1 -- 172.21.15.86:0/3962346813 wait complete. 2024-04-17T07:32:49.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.464+0000 7f1967233740 20 client.28639 trim_cache size 0 max 16384 2024-04-17T07:32:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.502+0000 7f1967233740 1 Processor -- start 2024-04-17T07:32:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.503+0000 7f1967233740 1 -- start start 2024-04-17T07:32:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.503+0000 7f1967233740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.503+0000 7f1967233740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec0990 0x5572b4ed1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.503+0000 7f1967233740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec5b00 0x5572b4eca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.503+0000 7f1967233740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1b9920 con 0x5572b4ec0990 2024-04-17T07:32:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.503+0000 7f1967233740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1bb460 con 0x5572b4ebffa0 2024-04-17T07:32:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.503+0000 7f1967233740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1b9ff0 con 0x5572b4ec5b00 2024-04-17T07:32:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.504+0000 7f1961a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.504+0000 7f1961000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec0990 0x5572b4ed1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.504+0000 7f1962400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec5b00 0x5572b4eca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.504+0000 7f1962400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec5b00 0x5572b4eca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40864/0 (socket says 172.21.15.86:40864) 2024-04-17T07:32:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.504+0000 7f1961a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36860/0 (socket says 172.21.15.86:36860) 2024-04-17T07:32:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.504+0000 7f1962400640 1 -- 172.21.15.86:0/2712071667 learned_addr learned my addr 172.21.15.86:0/2712071667 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.506+0000 7f1961a00640 1 -- 172.21.15.86:0/2712071667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec5b00 msgr2=0x5572b4eca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.506+0000 7f1961a00640 1 --2- 172.21.15.86:0/2712071667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec5b00 0x5572b4eca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.506+0000 7f1961a00640 1 -- 172.21.15.86:0/2712071667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec0990 msgr2=0x5572b4ed1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.506+0000 7f1961a00640 1 --2- 172.21.15.86:0/2712071667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec0990 0x5572b4ed1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.506+0000 7f1961a00640 1 -- 172.21.15.86:0/2712071667 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b4b383f0 con 0x5572b4ebffa0 2024-04-17T07:32:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.508+0000 7f1961a00640 1 --2- 172.21.15.86:0/2712071667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec1390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1950011010 tx=0x7f19500286f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51aac1f37c9d0541 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.511+0000 7f194e000640 1 -- 172.21.15.86:0/2712071667 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950023070 con 0x5572b4ebffa0 2024-04-17T07:32:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.511+0000 7f194e000640 1 -- 172.21.15.86:0/2712071667 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1950011730 con 0x5572b4ebffa0 2024-04-17T07:32:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.511+0000 7f194e000640 1 -- 172.21.15.86:0/2712071667 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195001f6b0 con 0x5572b4ebffa0 2024-04-17T07:32:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.512+0000 7f1967233740 1 -- 172.21.15.86:0/2712071667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 msgr2=0x5572b4ec1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.512+0000 7f1967233740 1 --2- 172.21.15.86:0/2712071667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec1390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1950011010 tx=0x7f19500286f0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.512+0000 7f1967233740 1 -- 172.21.15.86:0/2712071667 shutdown_connections 2024-04-17T07:32:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.513+0000 7f1967233740 1 --2- 172.21.15.86:0/2712071667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec5b00 0x5572b4eca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.513+0000 7f1967233740 1 --2- 172.21.15.86:0/2712071667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ec0990 0x5572b4ed1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.513+0000 7f1967233740 1 --2- 172.21.15.86:0/2712071667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ebffa0 0x5572b4ec1390 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.513+0000 7f1967233740 1 -- 172.21.15.86:0/2712071667 >> 172.21.15.86:0/2712071667 conn(0x5572b4ebb510 msgr2=0x5572b4ec9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.513+0000 7f1967233740 1 -- 172.21.15.86:0/2712071667 shutdown_connections 2024-04-17T07:32:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.513+0000 7f1967233740 1 -- 172.21.15.86:0/2712071667 wait complete. 2024-04-17T07:32:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.514+0000 7f1967233740 1 -- start start 2024-04-17T07:32:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.514+0000 7f1967233740 1 Processor -- start 2024-04-17T07:32:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.514+0000 7f1967233740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ebffa0 0x5572b4ece9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.514+0000 7f1967233740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec0990 0x5572b4ecef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.515+0000 7f1967233740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ec5b00 0x5572b4f3a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.515+0000 7f1967233740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1bad90 con 0x5572b4ebffa0 2024-04-17T07:32:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.515+0000 7f1967233740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1ba6c0 con 0x5572b4ec5b00 2024-04-17T07:32:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.515+0000 7f1967233740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f195c19d750 con 0x5572b4ec0990 2024-04-17T07:32:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.515+0000 7f1967233740 10 client.0 _ll_register_callbacks cb 0x7f195000be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.515+0000 7f1961a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ebffa0 0x5572b4ece9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.515+0000 7f1961a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ebffa0 0x5572b4ece9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53886/0 (socket says 172.21.15.86:53886) 2024-04-17T07:32:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.515+0000 7f1961a00640 1 -- 172.21.15.86:0/2680800951 learned_addr learned my addr 172.21.15.86:0/2680800951 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.515+0000 7f1961000640 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec0990 0x5572b4ecef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.515+0000 7f1962400640 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ec5b00 0x5572b4f3a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.516+0000 7f1961a00640 1 -- 172.21.15.86:0/2680800951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec0990 msgr2=0x5572b4ecef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:49.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.516+0000 7f1961a00640 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec0990 0x5572b4ecef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.516+0000 7f1961a00640 1 -- 172.21.15.86:0/2680800951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ec5b00 msgr2=0x5572b4f3a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:49.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.516+0000 7f1961a00640 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ec5b00 0x5572b4f3a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.516+0000 7f1961a00640 1 -- 172.21.15.86:0/2680800951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b4c027d0 con 0x5572b4ebffa0 2024-04-17T07:32:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.516+0000 7f1961a00640 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ebffa0 0x5572b4ece9c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f1950006010 tx=0x7f1950005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3550d023c47ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.517+0000 7f1943400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:32:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.517+0000 7f1943400640 1 -- 172.21.15.86:0/2680800951 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950023070 con 0x5572b4ebffa0 2024-04-17T07:32:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.517+0000 7f1943400640 1 -- 172.21.15.86:0/2680800951 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1950028a60 con 0x5572b4ebffa0 2024-04-17T07:32:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.518+0000 7f1967233740 1 -- 172.21.15.86:0/2680800951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1950011730 con 0x5572b4ebffa0 2024-04-17T07:32:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.518+0000 7f1943400640 1 -- 172.21.15.86:0/2680800951 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19500055f0 con 0x5572b4ebffa0 2024-04-17T07:32:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.518+0000 7f1967233740 10 client.19617 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.518+0000 7f1967233740 1 -- 172.21.15.86:0/2680800951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5572b4f3bfd0 con 0x5572b4ebffa0 2024-04-17T07:32:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.519+0000 7f1943400640 1 -- 172.21.15.86:0/2680800951 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1950011730 con 0x5572b4ebffa0 2024-04-17T07:32:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.519+0000 7f1943400640 1 -- 172.21.15.86:0/2680800951 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1950039020 con 0x5572b4ebffa0 2024-04-17T07:32:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.519+0000 7f1967233740 10 client.19617 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.519+0000 7f1967233740 10 client.19617 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:32:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.519+0000 7f1967233740 20 client.19617 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.519+0000 7f1967233740 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5572b4f3c880 0x5572b4f3eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.519+0000 7f1967233740 4 client.19617 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:32:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.519+0000 7f1967233740 1 -- 172.21.15.86:0/2680800951 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572b4e2d8b0 con 0x5572b4f3c880 2024-04-17T07:32:49.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.522+0000 7f1961000640 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5572b4f3c880 0x5572b4f3eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.523+0000 7f1961000640 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5572b4f3c880 0x5572b4f3eca0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5f8e9b7f98f7f09 server_cookie=f703b259cb596b87 in_seq=0 out_seq=0 2024-04-17T07:32:49.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.523+0000 7f1943400640 10 client.19617 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:32:49.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.524+0000 7f1943400640 1 -- 172.21.15.86:0/2680800951 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572b4e2d8b0 con 0x5572b4f3c880 2024-04-17T07:32:49.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.524+0000 7f1943400640 10 client.19617 handle_command_reply: tid=0 2024-04-17T07:32:49.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.525+0000 7f1967233740 1 client.19617 shutdown 2024-04-17T07:32:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.526+0000 7f1967233740 1 -- 172.21.15.86:0/2680800951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ebffa0 msgr2=0x5572b4ece9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.526+0000 7f1967233740 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ebffa0 0x5572b4ece9c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f1950006010 tx=0x7f1950005270 comp rx=0 tx=0).stop 2024-04-17T07:32:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.527+0000 7f1967233740 1 -- 172.21.15.86:0/2680800951 shutdown_connections 2024-04-17T07:32:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.527+0000 7f1967233740 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5572b4f3c880 0x5572b4f3eca0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.527+0000 7f1967233740 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572b4ec5b00 0x5572b4f3a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.527+0000 7f1967233740 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5572b4ec0990 0x5572b4ecef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.527+0000 7f1967233740 1 --2- 172.21.15.86:0/2680800951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572b4ebffa0 0x5572b4ece9c0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.527+0000 7f1967233740 1 -- 172.21.15.86:0/2680800951 >> 172.21.15.86:0/2680800951 conn(0x5572b4ebb510 msgr2=0x5572b4ec1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.527+0000 7f1943400640 0 client.19617 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:32:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.527+0000 7f1967233740 1 -- 172.21.15.86:0/2680800951 shutdown_connections 2024-04-17T07:32:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.527+0000 7f1967233740 1 -- 172.21.15.86:0/2680800951 wait complete. 2024-04-17T07:32:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.528+0000 7f1967233740 20 client.19617 trim_cache size 0 max 16384 2024-04-17T07:32:49.529 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:32:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.529+0000 7f1964400640 1 -- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19440a3d30 msgr2=0x7f19440a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.529+0000 7f1964400640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19440a3d30 0x7f19440a61f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x5572b4ec0d90 tx=0x7f195000b340 comp rx=0 tx=0).stop 2024-04-17T07:32:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.529+0000 7f1964400640 1 -- 172.21.15.86:0/3806333493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f195c1053d0 msgr2=0x7f195c19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.529+0000 7f1964400640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f195c1053d0 0x7f195c19ed80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f195402c710 tx=0x7f1954035ce0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.532+0000 7f1964400640 1 -- 172.21.15.86:0/3806333493 shutdown_connections 2024-04-17T07:32:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.532+0000 7f1964400640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19440a3d30 0x7f19440a61f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.532+0000 7f1964400640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f195c105d90 0x7f195c1a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.532+0000 7f1964400640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f195c1053d0 0x7f195c19ed80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.532+0000 7f1964400640 1 --2- 172.21.15.86:0/3806333493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f195c1041e0 0x7f195c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.532+0000 7f1964400640 1 -- 172.21.15.86:0/3806333493 >> 172.21.15.86:0/3806333493 conn(0x7f195c0ff750 msgr2=0x7f195c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.532+0000 7f1964400640 1 -- 172.21.15.86:0/3806333493 shutdown_connections 2024-04-17T07:32:49.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.532+0000 7f1964400640 1 -- 172.21.15.86:0/3806333493 wait complete. 2024-04-17T07:32:49.574 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:32:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.985+0000 7f423e000640 1 -- 172.21.15.86:0/4147415561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4238101910 msgr2=0x7f42381061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.985+0000 7f423e000640 1 --2- 172.21.15.86:0/4147415561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4238101910 0x7f42381061d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f4230009be0 tx=0x7f423002bef0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.985+0000 7f423e000640 1 -- 172.21.15.86:0/4147415561 shutdown_connections 2024-04-17T07:32:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.985+0000 7f423e000640 1 --2- 172.21.15.86:0/4147415561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4238101910 0x7f42381061d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.985+0000 7f423e000640 1 --2- 172.21.15.86:0/4147415561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4238100f70 0x7f42381013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.985+0000 7f423e000640 1 --2- 172.21.15.86:0/4147415561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42380ffe40 0x7f4238100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.985+0000 7f423e000640 1 -- 172.21.15.86:0/4147415561 >> 172.21.15.86:0/4147415561 conn(0x7f42380fb760 msgr2=0x7f42380fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.986+0000 7f423e000640 1 -- 172.21.15.86:0/4147415561 shutdown_connections 2024-04-17T07:32:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.987+0000 7f423e000640 1 -- 172.21.15.86:0/4147415561 wait complete. 2024-04-17T07:32:49.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.988+0000 7f423e000640 1 Processor -- start 2024-04-17T07:32:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.989+0000 7f423e000640 1 -- start start 2024-04-17T07:32:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.989+0000 7f423e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42380ffe40 0x7f423810fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.989+0000 7f423e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4238100f70 0x7f4238110200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.989+0000 7f423e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4238101910 0x7f4238111240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:49.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.990+0000 7f4237400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42380ffe40 0x7f423810fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.990+0000 7f4237400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42380ffe40 0x7f423810fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40888/0 (socket says 172.21.15.86:40888) 2024-04-17T07:32:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.990+0000 7f4237400640 1 -- 172.21.15.86:0/335897221 learned_addr learned my addr 172.21.15.86:0/335897221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.990+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4238108320 con 0x7f4238100f70 2024-04-17T07:32:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.990+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42381081a0 con 0x7f4238101910 2024-04-17T07:32:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.990+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42381084a0 con 0x7f42380ffe40 2024-04-17T07:32:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.990+0000 7f4237e00640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4238101910 0x7f4238111240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.990+0000 7f4236a00640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4238100f70 0x7f4238110200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.991+0000 7f4237e00640 1 -- 172.21.15.86:0/335897221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42380ffe40 msgr2=0x7f423810fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.991+0000 7f4237e00640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42380ffe40 0x7f423810fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.991+0000 7f4237e00640 1 -- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4238100f70 msgr2=0x7f4238110200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.991+0000 7f4237e00640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4238100f70 0x7f4238110200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.991+0000 7f4237e00640 1 -- 172.21.15.86:0/335897221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42381118b0 con 0x7f4238101910 2024-04-17T07:32:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.991+0000 7f4236a00640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4238100f70 0x7f4238110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:32:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.992+0000 7f4237e00640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4238101910 0x7f4238111240 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4230036040 tx=0x7f4230033c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.992+0000 7f422fe00640 1 -- 172.21.15.86:0/335897221 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230043020 con 0x7f4238101910 2024-04-17T07:32:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.992+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4238111b40 con 0x7f4238101910 2024-04-17T07:32:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.993+0000 7f422fe00640 1 -- 172.21.15.86:0/335897221 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4230003e20 con 0x7f4238101910 2024-04-17T07:32:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.993+0000 7f422fe00640 1 -- 172.21.15.86:0/335897221 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230034430 con 0x7f4238101910 2024-04-17T07:32:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:49.995+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4238112060 con 0x7f4238101910 2024-04-17T07:32:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.000+0000 7f422fe00640 1 -- 172.21.15.86:0/335897221 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4230046020 con 0x7f4238101910 2024-04-17T07:32:50.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.001+0000 7f4240c0b740 1 Processor -- start 2024-04-17T07:32:50.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.001+0000 7f4240c0b740 1 -- start start 2024-04-17T07:32:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.002+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0993a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.002+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d09e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.002+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.002+0000 7f4240c0b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c50cfff020 con 0x55c50d099970 2024-04-17T07:32:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.002+0000 7f4240c0b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c50cfa5460 con 0x55c50d098f80 2024-04-17T07:32:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.002+0000 7f4240c0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c50cffafe0 con 0x55c50d09eae0 2024-04-17T07:32:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.002+0000 7f422fe00640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f421c0a3db0 0x7f421c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.002+0000 7f4236a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.002+0000 7f4236a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.003+0000 7f4236a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36908/0 (socket says 172.21.15.86:36908) 2024-04-17T07:32:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.003+0000 7f4236a00640 1 -- 172.21.15.86:0/1740556199 learned_addr learned my addr 172.21.15.86:0/1740556199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.003+0000 7f4237400640 1 --2- 172.21.15.86:0/1740556199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d09e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.003+0000 7f422fe00640 1 -- 172.21.15.86:0/335897221 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f42300f2460 con 0x7f4238101910 2024-04-17T07:32:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.003+0000 7f4237e00640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f421c0a3db0 0x7f421c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.003+0000 7f4236a00640 1 -- 172.21.15.86:0/1740556199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 msgr2=0x55c50d0a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.003+0000 7f4236a00640 1 --2- 172.21.15.86:0/1740556199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.003+0000 7f4236a00640 1 -- 172.21.15.86:0/1740556199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 msgr2=0x55c50d09e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.003+0000 7f4236a00640 1 --2- 172.21.15.86:0/1740556199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d09e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.003+0000 7f4236a00640 1 -- 172.21.15.86:0/1740556199 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50cfad040 con 0x55c50d098f80 2024-04-17T07:32:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.004+0000 7f4236a00640 1 --2- 172.21.15.86:0/1740556199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0993a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4224012820 tx=0x7f4224005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=574904fb18a941af server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.004+0000 7f4237e00640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f421c0a3db0 0x7f421c0a6270 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4230009b30 tx=0x7f4230009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.005+0000 7f4223e00640 1 -- 172.21.15.86:0/1740556199 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224012e60 con 0x55c50d098f80 2024-04-17T07:32:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.005+0000 7f4223e00640 1 -- 172.21.15.86:0/1740556199 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4224012070 con 0x55c50d098f80 2024-04-17T07:32:50.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.005+0000 7f4223e00640 1 -- 172.21.15.86:0/1740556199 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422400fd80 con 0x55c50d098f80 2024-04-17T07:32:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.006+0000 7f4240c0b740 1 -- 172.21.15.86:0/1740556199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 msgr2=0x55c50d0993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.006+0000 7f4240c0b740 1 --2- 172.21.15.86:0/1740556199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0993a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4224012820 tx=0x7f4224005f90 comp rx=0 tx=0).stop 2024-04-17T07:32:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.006+0000 7f4240c0b740 1 -- 172.21.15.86:0/1740556199 shutdown_connections 2024-04-17T07:32:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.006+0000 7f4240c0b740 1 --2- 172.21.15.86:0/1740556199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.006+0000 7f4240c0b740 1 --2- 172.21.15.86:0/1740556199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d09e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.006+0000 7f4240c0b740 1 --2- 172.21.15.86:0/1740556199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0993a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.006+0000 7f4240c0b740 1 -- 172.21.15.86:0/1740556199 >> 172.21.15.86:0/1740556199 conn(0x55c50d0944f0 msgr2=0x55c50d0948f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.006+0000 7f4240c0b740 1 -- 172.21.15.86:0/1740556199 shutdown_connections 2024-04-17T07:32:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.006+0000 7f4240c0b740 1 -- 172.21.15.86:0/1740556199 wait complete. 2024-04-17T07:32:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.007+0000 7f4240c0b740 1 -- start start 2024-04-17T07:32:50.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.007+0000 7f4240c0b740 1 Processor -- start 2024-04-17T07:32:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.007+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0a3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d099970 0x55c50d0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d09eae0 0x55c50d0aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4240c0b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c50d0a1a30 con 0x55c50d09eae0 2024-04-17T07:32:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4240c0b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c50cfb8830 con 0x55c50d098f80 2024-04-17T07:32:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4240c0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c50d0a1bb0 con 0x55c50d099970 2024-04-17T07:32:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4240c0b740 10 client.0 _ll_register_callbacks cb 0x55c50cf86fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4236a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4236a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36922/0 (socket says 172.21.15.86:36922) 2024-04-17T07:32:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4236a00640 1 -- 172.21.15.86:0/2169657366 learned_addr learned my addr 172.21.15.86:0/2169657366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4236a00640 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d09eae0 0x55c50d0aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.008+0000 7f4237400640 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d099970 0x55c50d0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.009+0000 7f4236a00640 1 -- 172.21.15.86:0/2169657366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d099970 msgr2=0x55c50d0a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.009+0000 7f4236a00640 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d099970 0x55c50d0a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.009+0000 7f4236a00640 1 -- 172.21.15.86:0/2169657366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d09eae0 msgr2=0x55c50d0aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:50.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.009+0000 7f4236a00640 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d09eae0 0x55c50d0aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.009+0000 7f4236a00640 1 -- 172.21.15.86:0/2169657366 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50cfbc920 con 0x55c50d098f80 2024-04-17T07:32:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.010+0000 7f4236a00640 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0a3520 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f422400f890 tx=0x7f4224017040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63559ad104f9618d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.010+0000 7f4220c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.010+0000 7f4220c00640 1 -- 172.21.15.86:0/2169657366 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224012e60 con 0x55c50d098f80 2024-04-17T07:32:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.010+0000 7f4240c0b740 1 -- 172.21.15.86:0/2169657366 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4224012070 con 0x55c50d098f80 2024-04-17T07:32:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.010+0000 7f4220c00640 1 -- 172.21.15.86:0/2169657366 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4224005030 con 0x55c50d098f80 2024-04-17T07:32:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.011+0000 7f4220c00640 1 -- 172.21.15.86:0/2169657366 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4224032050 con 0x55c50d098f80 2024-04-17T07:32:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.011+0000 7f4240c0b740 10 client.38663 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:50.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.011+0000 7f4240c0b740 1 -- 172.21.15.86:0/2169657366 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c50cf8a7f0 con 0x55c50d098f80 2024-04-17T07:32:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.011+0000 7f4220c00640 1 -- 172.21.15.86:0/2169657366 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4224012070 con 0x55c50d098f80 2024-04-17T07:32:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.012+0000 7f4220c00640 1 -- 172.21.15.86:0/2169657366 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f4224005230 con 0x55c50d098f80 2024-04-17T07:32:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.012+0000 7f4240c0b740 10 client.38663 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.012+0000 7f4240c0b740 10 client.38663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.012+0000 7f4240c0b740 20 client.38663 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.012+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c50d0aed20 0x55c50d0b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.012+0000 7f4240c0b740 4 client.38663 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:50.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.012+0000 7f4240c0b740 1 -- 172.21.15.86:0/2169657366 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c50d00ca40 con 0x55c50d0aed20 2024-04-17T07:32:50.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.013+0000 7f4237400640 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c50d0aed20 0x55c50d0b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.014+0000 7f4220c00640 10 client.38663 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:50.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.014+0000 7f4237400640 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c50d0aed20 0x55c50d0b1130 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b954ef0e88e450fc server_cookie=cd9d714b007f8414 in_seq=0 out_seq=0 2024-04-17T07:32:50.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.014+0000 7f4220c00640 1 -- 172.21.15.86:0/2169657366 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c50d00ca40 con 0x55c50d0aed20 2024-04-17T07:32:50.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.015+0000 7f4220c00640 10 client.38663 handle_command_reply: tid=0 2024-04-17T07:32:50.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.015+0000 7f4240c0b740 1 client.38663 shutdown 2024-04-17T07:32:50.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.016+0000 7f4240c0b740 1 -- 172.21.15.86:0/2169657366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 msgr2=0x55c50d0a3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:50.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.016+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0a3520 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f422400f890 tx=0x7f4224017040 comp rx=0 tx=0).stop 2024-04-17T07:32:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.016+0000 7f4240c0b740 1 -- 172.21.15.86:0/2169657366 shutdown_connections 2024-04-17T07:32:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.016+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c50d0aed20 0x55c50d0b1130 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.016+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d09eae0 0x55c50d0aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.016+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d099970 0x55c50d0a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.016+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2169657366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d098f80 0x55c50d0a3520 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.016+0000 7f4220c00640 0 client.38663 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.016+0000 7f4240c0b740 1 -- 172.21.15.86:0/2169657366 >> 172.21.15.86:0/2169657366 conn(0x55c50d0944f0 msgr2=0x55c50d093710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.016+0000 7f4240c0b740 1 -- 172.21.15.86:0/2169657366 shutdown_connections 2024-04-17T07:32:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.018+0000 7f4240c0b740 1 -- 172.21.15.86:0/2169657366 wait complete. 2024-04-17T07:32:50.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.018+0000 7f4240c0b740 20 client.38663 trim_cache size 0 max 16384 2024-04-17T07:32:50.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.063+0000 7f4240c0b740 1 Processor -- start 2024-04-17T07:32:50.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.063+0000 7f4240c0b740 1 -- start start 2024-04-17T07:32:50.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.063+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d092e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.064+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.064+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d093900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.064+0000 7f4240c0b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4238123a70 con 0x55c50d099970 2024-04-17T07:32:50.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.064+0000 7f4240c0b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42381cc2e0 con 0x55c50d0a3b80 2024-04-17T07:32:50.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.064+0000 7f4240c0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4238124140 con 0x55c50d09eae0 2024-04-17T07:32:50.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.065+0000 7f4237400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.065+0000 7f4236a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d093900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.065+0000 7f4236a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d093900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36924/0 (socket says 172.21.15.86:36924) 2024-04-17T07:32:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.065+0000 7f4236a00640 1 -- 172.21.15.86:0/3817363898 learned_addr learned my addr 172.21.15.86:0/3817363898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.065+0000 7f4236a00640 1 --2- 172.21.15.86:0/3817363898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d092e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.065+0000 7f4236a00640 1 -- 172.21.15.86:0/3817363898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 msgr2=0x55c50d0933c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:50.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.065+0000 7f4236a00640 1 --2- 172.21.15.86:0/3817363898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0933c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.065+0000 7f4236a00640 1 -- 172.21.15.86:0/3817363898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 msgr2=0x55c50d092e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:50.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.065+0000 7f4236a00640 1 --2- 172.21.15.86:0/3817363898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d092e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.065+0000 7f4236a00640 1 -- 172.21.15.86:0/3817363898 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50cd113f0 con 0x55c50d0a3b80 2024-04-17T07:32:50.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.068+0000 7f4236a00640 1 --2- 172.21.15.86:0/3817363898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d093900 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4224014040 tx=0x7f4224001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be11170cafbf20e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:50.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.068+0000 7f4223400640 1 -- 172.21.15.86:0/3817363898 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422403b070 con 0x55c50d0a3b80 2024-04-17T07:32:50.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.068+0000 7f4223400640 1 -- 172.21.15.86:0/3817363898 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4224022d50 con 0x55c50d0a3b80 2024-04-17T07:32:50.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.068+0000 7f4223400640 1 -- 172.21.15.86:0/3817363898 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422403b070 con 0x55c50d0a3b80 2024-04-17T07:32:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.070+0000 7f4240c0b740 1 -- 172.21.15.86:0/3817363898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 msgr2=0x55c50d093900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.070+0000 7f4240c0b740 1 --2- 172.21.15.86:0/3817363898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d093900 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4224014040 tx=0x7f4224001e50 comp rx=0 tx=0).stop 2024-04-17T07:32:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.070+0000 7f4240c0b740 1 -- 172.21.15.86:0/3817363898 shutdown_connections 2024-04-17T07:32:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.070+0000 7f4240c0b740 1 --2- 172.21.15.86:0/3817363898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d093900 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.070+0000 7f4240c0b740 1 --2- 172.21.15.86:0/3817363898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0933c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.070+0000 7f4240c0b740 1 --2- 172.21.15.86:0/3817363898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d092e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.070+0000 7f4240c0b740 1 -- 172.21.15.86:0/3817363898 >> 172.21.15.86:0/3817363898 conn(0x55c50d0944f0 msgr2=0x55c50d097fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:50.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.070+0000 7f4240c0b740 1 -- 172.21.15.86:0/3817363898 shutdown_connections 2024-04-17T07:32:50.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.071+0000 7f4240c0b740 1 -- 172.21.15.86:0/3817363898 wait complete. 2024-04-17T07:32:50.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.071+0000 7f4240c0b740 1 -- start start 2024-04-17T07:32:50.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.071+0000 7f4240c0b740 1 Processor -- start 2024-04-17T07:32:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.073+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d0a7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.073+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.073+0000 7f4240c0b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d0a8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.073+0000 7f4240c0b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42381cbc10 con 0x55c50d099970 2024-04-17T07:32:50.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.073+0000 7f4240c0b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42381cb540 con 0x55c50d0a3b80 2024-04-17T07:32:50.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.073+0000 7f4240c0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4238121a60 con 0x55c50d09eae0 2024-04-17T07:32:50.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.073+0000 7f4240c0b740 10 client.0 _ll_register_callbacks cb 0x7f422400ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.074+0000 7f4237400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.074+0000 7f4236a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d0a8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.074+0000 7f4237400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0a7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40922/0 (socket says 172.21.15.86:40922) 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.074+0000 7f4237400640 1 -- 172.21.15.86:0/2279675591 learned_addr learned my addr 172.21.15.86:0/2279675591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.074+0000 7f4236a00640 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d0a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.075+0000 7f4236a00640 1 -- 172.21.15.86:0/2279675591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 msgr2=0x55c50d0a7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.075+0000 7f4236a00640 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0a7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.075+0000 7f4236a00640 1 -- 172.21.15.86:0/2279675591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 msgr2=0x55c50d0a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.075+0000 7f4236a00640 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d0a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.075+0000 7f4236a00640 1 -- 172.21.15.86:0/2279675591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50cddb7d0 con 0x55c50d0a3b80 2024-04-17T07:32:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.075+0000 7f4236a00640 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d0a8bb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4224039040 tx=0x7f4224012f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e09eeaea40299fe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.075+0000 7f4220c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.075+0000 7f4220c00640 1 -- 172.21.15.86:0/2279675591 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422403b040 con 0x55c50d0a3b80 2024-04-17T07:32:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.076+0000 7f4220c00640 1 -- 172.21.15.86:0/2279675591 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4224022110 con 0x55c50d0a3b80 2024-04-17T07:32:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.076+0000 7f4240c0b740 1 -- 172.21.15.86:0/2279675591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4224022d50 con 0x55c50d0a3b80 2024-04-17T07:32:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.076+0000 7f4220c00640 1 -- 172.21.15.86:0/2279675591 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4224009850 con 0x55c50d0a3b80 2024-04-17T07:32:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.076+0000 7f4240c0b740 10 client.38669 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.076+0000 7f4240c0b740 1 -- 172.21.15.86:0/2279675591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c50d0aa2d0 con 0x55c50d0a3b80 2024-04-17T07:32:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.077+0000 7f4220c00640 1 -- 172.21.15.86:0/2279675591 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4224022d50 con 0x55c50d0a3b80 2024-04-17T07:32:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.077+0000 7f4220c00640 1 -- 172.21.15.86:0/2279675591 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f42240035f0 con 0x55c50d0a3b80 2024-04-17T07:32:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.077+0000 7f4240c0b740 10 client.38669 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.077+0000 7f4240c0b740 10 client.38669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:50.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.077+0000 7f4240c0b740 20 client.38669 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:50.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.077+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c50d1158f0 0x55c50d117d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:50.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.077+0000 7f4240c0b740 4 client.38669 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:32:50.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.077+0000 7f4240c0b740 1 -- 172.21.15.86:0/2279675591 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c50d0068b0 con 0x55c50d1158f0 2024-04-17T07:32:50.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.078+0000 7f4237400640 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c50d1158f0 0x55c50d117d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:50.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.079+0000 7f4237400640 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c50d1158f0 0x55c50d117d10 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be239ad32e98771 server_cookie=6e23ee8a39754b1c in_seq=0 out_seq=0 2024-04-17T07:32:50.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.080+0000 7f4220c00640 10 client.38669 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:50.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.080+0000 7f4220c00640 1 -- 172.21.15.86:0/2279675591 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c50d0068b0 con 0x55c50d1158f0 2024-04-17T07:32:50.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.080+0000 7f4220c00640 10 client.38669 handle_command_reply: tid=0 2024-04-17T07:32:50.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.081+0000 7f4240c0b740 1 client.38669 shutdown 2024-04-17T07:32:50.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.081+0000 7f4240c0b740 1 -- 172.21.15.86:0/2279675591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 msgr2=0x55c50d0a8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:50.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.081+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d0a8bb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4224039040 tx=0x7f4224012f60 comp rx=0 tx=0).stop 2024-04-17T07:32:50.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.081+0000 7f4240c0b740 1 -- 172.21.15.86:0/2279675591 shutdown_connections 2024-04-17T07:32:50.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.081+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c50d1158f0 0x55c50d117d10 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.081+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c50d0a3b80 0x55c50d0a8bb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.081+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c50d09eae0 0x55c50d0a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.081+0000 7f4240c0b740 1 --2- 172.21.15.86:0/2279675591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c50d099970 0x55c50d0a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.081+0000 7f4240c0b740 1 -- 172.21.15.86:0/2279675591 >> 172.21.15.86:0/2279675591 conn(0x55c50d0944f0 msgr2=0x55c50d0a25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:50.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.082+0000 7f4220c00640 0 client.38669 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:50.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.082+0000 7f4240c0b740 1 -- 172.21.15.86:0/2279675591 shutdown_connections 2024-04-17T07:32:50.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.086+0000 7f4240c0b740 1 -- 172.21.15.86:0/2279675591 wait complete. 2024-04-17T07:32:50.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.086+0000 7f4240c0b740 20 client.38669 trim_cache size 0 max 16384 2024-04-17T07:32:50.086 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:32:50.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.090+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f421c0a3db0 msgr2=0x7f421c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.090+0000 7f423e000640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f421c0a3db0 0x7f421c0a6270 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4230009b30 tx=0x7f4230009500 comp rx=0 tx=0).stop 2024-04-17T07:32:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.090+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4238101910 msgr2=0x7f4238111240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.090+0000 7f423e000640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4238101910 0x7f4238111240 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4230036040 tx=0x7f4230033c40 comp rx=0 tx=0).stop 2024-04-17T07:32:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.090+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 shutdown_connections 2024-04-17T07:32:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.090+0000 7f423e000640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f421c0a3db0 0x7f421c0a6270 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.090+0000 7f423e000640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4238101910 0x7f4238111240 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.090+0000 7f423e000640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4238100f70 0x7f4238110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.090+0000 7f423e000640 1 --2- 172.21.15.86:0/335897221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42380ffe40 0x7f423810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:50.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.090+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 >> 172.21.15.86:0/335897221 conn(0x7f42380fb760 msgr2=0x7f4238104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:50.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.091+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 shutdown_connections 2024-04-17T07:32:50.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:50.091+0000 7f423e000640 1 -- 172.21.15.86:0/335897221 wait complete. 2024-04-17T07:32:51.147 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:32:51.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:32:51.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:32:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:50 smithi098 ceph-mon[35418]: pgmap v994: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.9 MiB/s wr, 558 op/s 2024-04-17T07:32:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:32:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:32:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:32:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:32:51.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:50 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:32:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:50 smithi103 ceph-mon[35495]: pgmap v994: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.9 MiB/s wr, 558 op/s 2024-04-17T07:32:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:32:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:32:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:32:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:32:51.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:50 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:32:51.277 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:50 smithi086 ceph-mon[31733]: pgmap v994: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.9 MiB/s wr, 558 op/s 2024-04-17T07:32:51.277 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:32:51.277 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:32:51.277 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:32:51.277 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:32:51.277 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:50 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:32:51.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.484+0000 7f38a4e00640 1 -- 172.21.15.86:0/834306509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a01028f0 msgr2=0x7f38a0104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:51.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.484+0000 7f38a4e00640 1 --2- 172.21.15.86:0/834306509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a01028f0 0x7f38a0104ce0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3890009900 tx=0x7f389002bc60 comp rx=0 tx=0).stop 2024-04-17T07:32:51.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.485+0000 7f38a4e00640 1 -- 172.21.15.86:0/834306509 shutdown_connections 2024-04-17T07:32:51.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.485+0000 7f38a4e00640 1 --2- 172.21.15.86:0/834306509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a0107b50 0x7f38a0109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.485+0000 7f38a4e00640 1 --2- 172.21.15.86:0/834306509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a0105220 0x7f38a0107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.485+0000 7f38a4e00640 1 --2- 172.21.15.86:0/834306509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a01028f0 0x7f38a0104ce0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.485+0000 7f38a4e00640 1 -- 172.21.15.86:0/834306509 >> 172.21.15.86:0/834306509 conn(0x7f38a00fc300 msgr2=0x7f38a00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:51.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.486+0000 7f38a4e00640 1 -- 172.21.15.86:0/834306509 shutdown_connections 2024-04-17T07:32:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.487+0000 7f38a4e00640 1 -- 172.21.15.86:0/834306509 wait complete. 2024-04-17T07:32:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.488+0000 7f38a4e00640 1 Processor -- start 2024-04-17T07:32:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.488+0000 7f38a4e00640 1 -- start start 2024-04-17T07:32:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.489+0000 7f38a4e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a01028f0 0x7f38a0195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.489+0000 7f38a4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a0105220 0x7f38a0196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.489+0000 7f38a4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a0107b50 0x7f38a019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.489+0000 7f38a4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38a010d9f0 con 0x7f38a0105220 2024-04-17T07:32:51.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.489+0000 7f38a4e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f38a010d870 con 0x7f38a01028f0 2024-04-17T07:32:51.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.489+0000 7f38a4e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38a010db70 con 0x7f38a0107b50 2024-04-17T07:32:51.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.490+0000 7f389f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a01028f0 0x7f38a0195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.490+0000 7f389f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a01028f0 0x7f38a0195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36936/0 (socket says 172.21.15.86:36936) 2024-04-17T07:32:51.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.490+0000 7f389f400640 1 -- 172.21.15.86:0/1909943229 learned_addr learned my addr 172.21.15.86:0/1909943229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:51.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.490+0000 7f389fe00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a0107b50 0x7f38a019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.492+0000 7f389fe00640 1 -- 172.21.15.86:0/1909943229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a01028f0 msgr2=0x7f38a0195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.492+0000 7f389fe00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a01028f0 0x7f38a0195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.492+0000 7f389fe00640 1 -- 172.21.15.86:0/1909943229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a0105220 msgr2=0x7f38a0196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.492+0000 7f389fe00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a0105220 0x7f38a0196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.492+0000 7f389fe00640 1 -- 172.21.15.86:0/1909943229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a019daf0 con 0x7f38a0107b50 2024-04-17T07:32:51.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.493+0000 7f389fe00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a0107b50 0x7f38a019d480 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f389800e940 tx=0x7f389800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:51.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.493+0000 7f3897e00640 1 -- 172.21.15.86:0/1909943229 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898004d20 con 0x7f38a0107b50 2024-04-17T07:32:51.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.493+0000 7f38a4e00640 1 -- 172.21.15.86:0/1909943229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a019dde0 con 0x7f38a0107b50 2024-04-17T07:32:51.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.493+0000 7f3897e00640 1 -- 172.21.15.86:0/1909943229 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3898005020 con 0x7f38a0107b50 2024-04-17T07:32:51.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.493+0000 7f3897e00640 1 -- 172.21.15.86:0/1909943229 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38980116d0 con 0x7f38a0107b50 2024-04-17T07:32:51.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.493+0000 7f38a4e00640 1 -- 172.21.15.86:0/1909943229 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a019e330 con 0x7f38a0107b50 2024-04-17T07:32:51.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.498+0000 7f38a9060740 1 Processor -- start 2024-04-17T07:32:51.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.503+0000 7f3897e00640 1 -- 172.21.15.86:0/1909943229 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f389800cb20 con 0x7f38a0107b50 2024-04-17T07:32:51.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.503+0000 7f38a9060740 1 -- start start 2024-04-17T07:32:51.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.504+0000 7f38a9060740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 0x56544421f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.504+0000 7f38a9060740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421fa90 0x565444224590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.504+0000 7f38a9060740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 0x565444227090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.504+0000 7f38a9060740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565444184cb0 con 0x56544421f0a0 2024-04-17T07:32:51.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.504+0000 7f38a9060740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56544412b370 con 0x565444224c00 2024-04-17T07:32:51.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.504+0000 7f38a9060740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565444180c70 con 0x56544421fa90 2024-04-17T07:32:51.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.504+0000 7f389f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421fa90 0x565444224590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.504+0000 7f389ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 0x56544421f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.504+0000 7f389f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421fa90 0x565444224590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40964/0 (socket says 172.21.15.86:40964) 2024-04-17T07:32:51.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.504+0000 7f389f400640 1 -- 172.21.15.86:0/2282176440 learned_addr learned my addr 172.21.15.86:0/2282176440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.506+0000 7f389f400640 1 -- 172.21.15.86:0/2282176440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 msgr2=0x565444227090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T07:32:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.506+0000 7f389f400640 1 --2- 172.21.15.86:0/2282176440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 0x565444227090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.506+0000 7f389f400640 1 -- 172.21.15.86:0/2282176440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 msgr2=0x56544421f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.506+0000 7f389f400640 1 --2- 172.21.15.86:0/2282176440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 0x56544421f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.506+0000 7f389f400640 1 -- 172.21.15.86:0/2282176440 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544414a910 con 0x56544421fa90 2024-04-17T07:32:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.506+0000 7f3897e00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38840a3c30 0x7f38840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.506+0000 7f389ea00640 1 --2- 172.21.15.86:0/2282176440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 0x56544421f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:32:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.508+0000 7f389ea00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38840a3c30 0x7f38840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.508+0000 7f389f400640 1 --2- 172.21.15.86:0/2282176440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421fa90 0x565444224590 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3890004740 tx=0x7f3890004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad10ef05b14123d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.508+0000 7f3897e00640 1 -- 172.21.15.86:0/1909943229 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f38980d0880 con 0x7f38a0107b50 2024-04-17T07:32:51.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.509+0000 7f3894c00640 1 -- 172.21.15.86:0/2282176440 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890042020 con 0x56544421fa90 2024-04-17T07:32:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.509+0000 7f3894c00640 1 -- 172.21.15.86:0/2282176440 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f389002c600 con 0x56544421fa90 2024-04-17T07:32:51.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.509+0000 7f3894c00640 1 -- 172.21.15.86:0/2282176440 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890003850 con 0x56544421fa90 2024-04-17T07:32:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.510+0000 7f38a9060740 1 -- 172.21.15.86:0/2282176440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421fa90 msgr2=0x565444224590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.510+0000 7f389ea00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38840a3c30 0x7f38840a60f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f388c005d00 tx=0x7f388c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.510+0000 7f38a9060740 1 --2- 172.21.15.86:0/2282176440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421fa90 0x565444224590 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3890004740 tx=0x7f3890004770 comp rx=0 tx=0).stop 2024-04-17T07:32:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.511+0000 7f38a9060740 1 -- 172.21.15.86:0/2282176440 shutdown_connections 2024-04-17T07:32:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.511+0000 7f38a9060740 1 --2- 172.21.15.86:0/2282176440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 0x565444227090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.511+0000 7f38a9060740 1 --2- 172.21.15.86:0/2282176440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421fa90 0x565444224590 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.511+0000 7f38a9060740 1 --2- 172.21.15.86:0/2282176440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 0x56544421f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.511+0000 7f38a9060740 1 -- 172.21.15.86:0/2282176440 >> 172.21.15.86:0/2282176440 conn(0x56544421a610 msgr2=0x56544421aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.511+0000 7f38a9060740 1 -- 172.21.15.86:0/2282176440 shutdown_connections 2024-04-17T07:32:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.512+0000 7f38a9060740 1 -- 172.21.15.86:0/2282176440 wait complete. 2024-04-17T07:32:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.513+0000 7f38a9060740 1 -- start start 2024-04-17T07:32:51.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.513+0000 7f38a9060740 1 Processor -- start 2024-04-17T07:32:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.514+0000 7f38a9060740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421f0a0 0x565444229580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.514+0000 7f38a9060740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 0x565444229ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.514+0000 7f38a9060740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 0x565444230b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.514+0000 7f38a9060740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565444227b50 con 0x56544421fa90 2024-04-17T07:32:51.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.514+0000 7f38a9060740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56544413e730 con 0x565444224c00 2024-04-17T07:32:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.514+0000 7f38a9060740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565444227cd0 con 0x56544421f0a0 2024-04-17T07:32:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.515+0000 7f38a9060740 10 client.0 _ll_register_callbacks cb 0x565444140b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.515+0000 7f389fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 0x565444230b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.515+0000 7f389ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 0x565444229ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.515+0000 7f389fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 0x565444230b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36948/0 (socket says 172.21.15.86:36948) 2024-04-17T07:32:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.515+0000 7f389fe00640 1 -- 172.21.15.86:0/2019612547 learned_addr learned my addr 172.21.15.86:0/2019612547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.515+0000 7f389f400640 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421f0a0 0x565444229580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.515+0000 7f389fe00640 1 -- 172.21.15.86:0/2019612547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421f0a0 msgr2=0x565444229580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.515+0000 7f389fe00640 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421f0a0 0x565444229580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.515+0000 7f389fe00640 1 -- 172.21.15.86:0/2019612547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 msgr2=0x565444229ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.516+0000 7f389fe00640 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 0x565444229ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.516+0000 7f389fe00640 1 -- 172.21.15.86:0/2019612547 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565443f5f7e0 con 0x565444224c00 2024-04-17T07:32:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.516+0000 7f389ea00640 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 0x565444229ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.517+0000 7f389fe00640 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 0x565444230b00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3898098be0 tx=0x7f389809cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88476e5b72211048 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:51.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.517+0000 7f3889600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.518+0000 7f3889600640 1 -- 172.21.15.86:0/2019612547 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389801f040 con 0x565444224c00 2024-04-17T07:32:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.519+0000 7f3889600640 1 -- 172.21.15.86:0/2019612547 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f389809bbd0 con 0x565444224c00 2024-04-17T07:32:51.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.519+0000 7f38a9060740 1 -- 172.21.15.86:0/2019612547 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389002c600 con 0x565444224c00 2024-04-17T07:32:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.520+0000 7f3889600640 1 -- 172.21.15.86:0/2019612547 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f38980a53f0 con 0x565444224c00 2024-04-17T07:32:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.520+0000 7f38a9060740 10 client.38678 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.520+0000 7f38a9060740 1 -- 172.21.15.86:0/2019612547 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565443fbff70 con 0x565444224c00 2024-04-17T07:32:51.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.521+0000 7f3889600640 1 -- 172.21.15.86:0/2019612547 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f389002c600 con 0x565444224c00 2024-04-17T07:32:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.521+0000 7f3889600640 1 -- 172.21.15.86:0/2019612547 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f3898098040 con 0x565444224c00 2024-04-17T07:32:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.521+0000 7f38a9060740 10 client.38678 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.521+0000 7f38a9060740 10 client.38678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.521+0000 7f38a9060740 20 client.38678 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.521+0000 7f38a9060740 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x565444235390 0x5654442377d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.521+0000 7f38a9060740 4 client.38678 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:51.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.521+0000 7f38a9060740 1 -- 172.21.15.86:0/2019612547 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654441926d0 con 0x565444235390 2024-04-17T07:32:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.524+0000 7f389f400640 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x565444235390 0x5654442377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.525+0000 7f3889600640 10 client.38678 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:51.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.525+0000 7f389f400640 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x565444235390 0x5654442377d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203ca89edbd27dae server_cookie=4a641a1a3a4c2c6b in_seq=0 out_seq=0 2024-04-17T07:32:51.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.526+0000 7f3889600640 1 -- 172.21.15.86:0/2019612547 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654441926d0 con 0x565444235390 2024-04-17T07:32:51.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.526+0000 7f3889600640 10 client.38678 handle_command_reply: tid=0 2024-04-17T07:32:51.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.526+0000 7f38a9060740 1 client.38678 shutdown 2024-04-17T07:32:51.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.528+0000 7f38a9060740 1 -- 172.21.15.86:0/2019612547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 msgr2=0x565444230b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.528+0000 7f38a9060740 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 0x565444230b00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3898098be0 tx=0x7f389809cf40 comp rx=0 tx=0).stop 2024-04-17T07:32:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.528+0000 7f38a9060740 1 -- 172.21.15.86:0/2019612547 shutdown_connections 2024-04-17T07:32:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.528+0000 7f38a9060740 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x565444235390 0x5654442377d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.528+0000 7f38a9060740 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x565444224c00 0x565444230b00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.528+0000 7f38a9060740 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 0x565444229ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.528+0000 7f38a9060740 1 --2- 172.21.15.86:0/2019612547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56544421f0a0 0x565444229580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.528+0000 7f38a9060740 1 -- 172.21.15.86:0/2019612547 >> 172.21.15.86:0/2019612547 conn(0x56544421a610 msgr2=0x565444219990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.529+0000 7f3889600640 0 client.38678 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.529+0000 7f38a9060740 1 -- 172.21.15.86:0/2019612547 shutdown_connections 2024-04-17T07:32:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.530+0000 7f38a9060740 1 -- 172.21.15.86:0/2019612547 wait complete. 2024-04-17T07:32:51.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.530+0000 7f38a9060740 20 client.38678 trim_cache size 0 max 16384 2024-04-17T07:32:51.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.588+0000 7f38a9060740 1 Processor -- start 2024-04-17T07:32:51.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.588+0000 7f38a9060740 1 -- start start 2024-04-17T07:32:51.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.588+0000 7f38a9060740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421f0a0 0x565444228fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.588+0000 7f38a9060740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 0x565444230ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.588+0000 7f38a9060740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 0x565444219100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.588+0000 7f38a9060740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f388000fc00 con 0x56544421fa90 2024-04-17T07:32:51.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.588+0000 7f38a9060740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3880011740 con 0x56544421f0a0 2024-04-17T07:32:51.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.588+0000 7f38a9060740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38800102d0 con 0x565444224c00 2024-04-17T07:32:51.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.589+0000 7f389fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 0x565444219100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.589+0000 7f389fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 0x565444219100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:40978/0 (socket says 172.21.15.86:40978) 2024-04-17T07:32:51.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.589+0000 7f389fe00640 1 -- 172.21.15.86:0/2042228720 learned_addr learned my addr 172.21.15.86:0/2042228720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:51.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.589+0000 7f389ea00640 1 --2- 172.21.15.86:0/2042228720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 0x565444230ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.589+0000 7f389f400640 1 --2- 172.21.15.86:0/2042228720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421f0a0 0x565444228fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.589+0000 7f389fe00640 1 -- 172.21.15.86:0/2042228720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421f0a0 msgr2=0x565444228fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:51.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.590+0000 7f389fe00640 1 --2- 172.21.15.86:0/2042228720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421f0a0 0x565444228fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.590+0000 7f389fe00640 1 -- 172.21.15.86:0/2042228720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 msgr2=0x565444230ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:51.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.590+0000 7f389fe00640 1 --2- 172.21.15.86:0/2042228720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 0x565444230ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.590+0000 7f389fe00640 1 -- 172.21.15.86:0/2042228720 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565443e57170 con 0x565444224c00 2024-04-17T07:32:51.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.590+0000 7f389ea00640 1 --2- 172.21.15.86:0/2042228720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 0x565444230ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:51.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.591+0000 7f389fe00640 1 --2- 172.21.15.86:0/2042228720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 0x565444219100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f389809aa30 tx=0x7f38980a0cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b3fb1d7b86bb083 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:51.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.591+0000 7f388be00640 1 -- 172.21.15.86:0/2042228720 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389801f070 con 0x565444224c00 2024-04-17T07:32:51.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.591+0000 7f388be00640 1 -- 172.21.15.86:0/2042228720 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f389809b730 con 0x565444224c00 2024-04-17T07:32:51.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.591+0000 7f388be00640 1 -- 172.21.15.86:0/2042228720 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389801f070 con 0x565444224c00 2024-04-17T07:32:51.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.593+0000 7f38a9060740 1 -- 172.21.15.86:0/2042228720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 msgr2=0x565444219100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:51.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.593+0000 7f38a9060740 1 --2- 172.21.15.86:0/2042228720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 0x565444219100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f389809aa30 tx=0x7f38980a0cb0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.593+0000 7f38a9060740 1 -- 172.21.15.86:0/2042228720 shutdown_connections 2024-04-17T07:32:51.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.593+0000 7f38a9060740 1 --2- 172.21.15.86:0/2042228720 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 0x565444219100 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.593+0000 7f38a9060740 1 --2- 172.21.15.86:0/2042228720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421fa90 0x565444230ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.593+0000 7f38a9060740 1 --2- 172.21.15.86:0/2042228720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421f0a0 0x565444228fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.593+0000 7f38a9060740 1 -- 172.21.15.86:0/2042228720 >> 172.21.15.86:0/2042228720 conn(0x56544421a610 msgr2=0x5654442288b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:51.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.593+0000 7f38a9060740 1 -- 172.21.15.86:0/2042228720 shutdown_connections 2024-04-17T07:32:51.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.593+0000 7f38a9060740 1 -- 172.21.15.86:0/2042228720 wait complete. 2024-04-17T07:32:51.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.593+0000 7f38a9060740 1 -- start start 2024-04-17T07:32:51.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.594+0000 7f38a9060740 1 Processor -- start 2024-04-17T07:32:51.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.594+0000 7f38a9060740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 0x56544422ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.594+0000 7f38a9060740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421fa90 0x56544422e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.594+0000 7f38a9060740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 0x56544422f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.594+0000 7f38a9060740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3880011070 con 0x56544421f0a0 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.594+0000 7f38a9060740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f38800109a0 con 0x56544421fa90 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.594+0000 7f38a9060740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f388000dbd0 con 0x565444224c00 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.594+0000 7f38a9060740 10 client.0 _ll_register_callbacks cb 0x7f3898004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.595+0000 7f389ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421fa90 0x56544422e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.595+0000 7f389ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421fa90 0x56544422e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36964/0 (socket says 172.21.15.86:36964) 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.595+0000 7f389ea00640 1 -- 172.21.15.86:0/1513405462 learned_addr learned my addr 172.21.15.86:0/1513405462 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.595+0000 7f389fe00640 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 0x56544422f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.595+0000 7f389f400640 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 0x56544422ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.595+0000 7f389ea00640 1 -- 172.21.15.86:0/1513405462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 msgr2=0x56544422f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.595+0000 7f389ea00640 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 0x56544422f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.595+0000 7f389ea00640 1 -- 172.21.15.86:0/1513405462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 msgr2=0x56544422ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:51.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.595+0000 7f389ea00640 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 0x56544422ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.595+0000 7f389ea00640 1 -- 172.21.15.86:0/1513405462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544422f930 con 0x56544421fa90 2024-04-17T07:32:51.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.596+0000 7f389ea00640 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421fa90 0x56544422e330 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f388c0116c0 tx=0x7f388c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6a09f3b7bec4238 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:51.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.596+0000 7f3889600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:51.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.596+0000 7f3889600640 1 -- 172.21.15.86:0/1513405462 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c01f840 con 0x56544421fa90 2024-04-17T07:32:51.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.596+0000 7f38a9060740 1 -- 172.21.15.86:0/1513405462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389809b730 con 0x56544421fa90 2024-04-17T07:32:51.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.596+0000 7f3889600640 1 -- 172.21.15.86:0/1513405462 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f388c01f9e0 con 0x56544421fa90 2024-04-17T07:32:51.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.597+0000 7f3889600640 1 -- 172.21.15.86:0/1513405462 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f388c01fbe0 con 0x56544421fa90 2024-04-17T07:32:51.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.597+0000 7f38a9060740 10 client.38684 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:51.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.597+0000 7f38a9060740 1 -- 172.21.15.86:0/1513405462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565444230d60 con 0x56544421fa90 2024-04-17T07:32:51.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.598+0000 7f3889600640 1 -- 172.21.15.86:0/1513405462 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f389809b730 con 0x56544421fa90 2024-04-17T07:32:51.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.598+0000 7f3889600640 1 -- 172.21.15.86:0/1513405462 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f388c01e420 con 0x56544421fa90 2024-04-17T07:32:51.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.598+0000 7f38a9060740 10 client.38684 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:51.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.598+0000 7f38a9060740 10 client.38684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:51.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.598+0000 7f38a9060740 20 client.38684 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.598+0000 7f38a9060740 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56544429c060 0x56544429e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.599+0000 7f38a9060740 4 client.38684 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:32:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.599+0000 7f38a9060740 1 -- 172.21.15.86:0/1513405462 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56544418c540 con 0x56544429c060 2024-04-17T07:32:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.599+0000 7f389f400640 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56544429c060 0x56544429e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.600+0000 7f3889600640 10 client.38684 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:51.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.600+0000 7f389f400640 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56544429c060 0x56544429e480 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3c78e6a3592c1f3 server_cookie=729ec4dfe0a54691 in_seq=0 out_seq=0 2024-04-17T07:32:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.601+0000 7f3889600640 1 -- 172.21.15.86:0/1513405462 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56544418c540 con 0x56544429c060 2024-04-17T07:32:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.601+0000 7f3889600640 10 client.38684 handle_command_reply: tid=0 2024-04-17T07:32:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 client.38684 shutdown 2024-04-17T07:32:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 -- 172.21.15.86:0/1513405462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421fa90 msgr2=0x56544422e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421fa90 0x56544422e330 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f388c0116c0 tx=0x7f388c005c90 comp rx=0 tx=0).stop 2024-04-17T07:32:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 -- 172.21.15.86:0/1513405462 shutdown_connections 2024-04-17T07:32:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56544429c060 0x56544429e480 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565444224c00 0x56544422f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56544421fa90 0x56544422e330 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 --2- 172.21.15.86:0/1513405462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544421f0a0 0x56544422ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 -- 172.21.15.86:0/1513405462 >> 172.21.15.86:0/1513405462 conn(0x56544421a610 msgr2=0x56544421d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f3889600640 0 client.38684 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 -- 172.21.15.86:0/1513405462 shutdown_connections 2024-04-17T07:32:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.602+0000 7f38a9060740 1 -- 172.21.15.86:0/1513405462 wait complete. 2024-04-17T07:32:51.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.603+0000 7f38a9060740 20 client.38684 trim_cache size 0 max 16384 2024-04-17T07:32:51.605 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:32:51.605 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:32:51.605 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "2ff1db74-fd7f-4159-9f7e-714a1a4ab239", 2024-04-17T07:32:51.605 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:32:51.605 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:32:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 -- 172.21.15.86:0/1909943229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38840a3c30 msgr2=0x7f38840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38840a3c30 0x7f38840a60f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f388c005d00 tx=0x7f388c005f50 comp rx=0 tx=0).stop 2024-04-17T07:32:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 -- 172.21.15.86:0/1909943229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a0107b50 msgr2=0x7f38a019d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a0107b50 0x7f38a019d480 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f389800e940 tx=0x7f389800ee00 comp rx=0 tx=0).stop 2024-04-17T07:32:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 -- 172.21.15.86:0/1909943229 shutdown_connections 2024-04-17T07:32:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f38840a3c30 0x7f38840a60f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38a0107b50 0x7f38a019d480 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38a0105220 0x7f38a0196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 --2- 172.21.15.86:0/1909943229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a01028f0 0x7f38a0195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 -- 172.21.15.86:0/1909943229 >> 172.21.15.86:0/1909943229 conn(0x7f38a00fc300 msgr2=0x7f38a0102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 -- 172.21.15.86:0/1909943229 shutdown_connections 2024-04-17T07:32:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:51.604+0000 7f38a4e00640 1 -- 172.21.15.86:0/1909943229 wait complete. 2024-04-17T07:32:51.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:32:51.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:32:52.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.001+0000 7f288be00640 1 -- 172.21.15.86:0/4118539023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28840a61d0 msgr2=0x7f28840a6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:52.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.001+0000 7f288be00640 1 --2- 172.21.15.86:0/4118539023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28840a61d0 0x7f28840a6650 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f287c009900 tx=0x7f287c02bc40 comp rx=0 tx=0).stop 2024-04-17T07:32:52.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.001+0000 7f288be00640 1 -- 172.21.15.86:0/4118539023 shutdown_connections 2024-04-17T07:32:52.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.001+0000 7f288be00640 1 --2- 172.21.15.86:0/4118539023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28840a6b90 0x7f28840ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.001+0000 7f288be00640 1 --2- 172.21.15.86:0/4118539023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28840a61d0 0x7f28840a6650 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.001+0000 7f288be00640 1 --2- 172.21.15.86:0/4118539023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28840a4fe0 0x7f28840a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.001+0000 7f288be00640 1 -- 172.21.15.86:0/4118539023 >> 172.21.15.86:0/4118539023 conn(0x7f28840a0530 msgr2=0x7f28840a2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:52.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.002+0000 7f288be00640 1 -- 172.21.15.86:0/4118539023 shutdown_connections 2024-04-17T07:32:52.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.003+0000 7f288be00640 1 -- 172.21.15.86:0/4118539023 wait complete. 2024-04-17T07:32:52.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.005+0000 7f288be00640 1 Processor -- start 2024-04-17T07:32:52.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.006+0000 7f288be00640 1 -- start start 2024-04-17T07:32:52.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.006+0000 7f288be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28840a4fe0 0x7f288413ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.006+0000 7f288be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28840a61d0 0x7f288413b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.006+0000 7f288be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28840a6b90 0x7f2884142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.006+0000 7f288be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28840b0730 con 0x7f28840a61d0 2024-04-17T07:32:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.006+0000 7f288be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f28840b05b0 con 0x7f28840a6b90 2024-04-17T07:32:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.006+0000 7f288be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28840b08b0 con 0x7f28840a4fe0 2024-04-17T07:32:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.006+0000 7f288b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28840a6b90 0x7f2884142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.006+0000 7f288b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28840a6b90 0x7f2884142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:36988/0 (socket says 172.21.15.86:36988) 2024-04-17T07:32:52.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.006+0000 7f288b400640 1 -- 172.21.15.86:0/2914236698 learned_addr learned my addr 172.21.15.86:0/2914236698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.007+0000 7f288aa00640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28840a4fe0 0x7f288413ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.007+0000 7f288b400640 1 -- 172.21.15.86:0/2914236698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28840a4fe0 msgr2=0x7f288413ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.007+0000 7f288b400640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28840a4fe0 0x7f288413ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.007+0000 7f288b400640 1 -- 172.21.15.86:0/2914236698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28840a61d0 msgr2=0x7f288413b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.007+0000 7f288b400640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28840a61d0 0x7f288413b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.007+0000 7f288b400640 1 -- 172.21.15.86:0/2914236698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2884153ab0 con 0x7f28840a6b90 2024-04-17T07:32:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.007+0000 7f288b400640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28840a6b90 0x7f2884142430 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f288c050aa0 tx=0x7f288c073bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:52.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.008+0000 7f2883400640 1 -- 172.21.15.86:0/2914236698 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c071880 con 0x7f28840a6b90 2024-04-17T07:32:52.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.008+0000 7f288be00640 1 -- 172.21.15.86:0/2914236698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2884153da0 con 0x7f28840a6b90 2024-04-17T07:32:52.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.008+0000 7f2883400640 1 -- 172.21.15.86:0/2914236698 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f288c07c440 con 0x7f28840a6b90 2024-04-17T07:32:52.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.008+0000 7f2883400640 1 -- 172.21.15.86:0/2914236698 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c068670 con 0x7f28840a6b90 2024-04-17T07:32:52.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.008+0000 7f288be00640 1 -- 172.21.15.86:0/2914236698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2884154270 con 0x7f28840a6b90 2024-04-17T07:32:52.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.015+0000 7f2894996740 1 Processor -- start 2024-04-17T07:32:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.016+0000 7f2883400640 1 -- 172.21.15.86:0/2914236698 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f288c071a20 con 0x7f28840a6b90 2024-04-17T07:32:52.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.017+0000 7f2894996740 1 -- start start 2024-04-17T07:32:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.018+0000 7f2894996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 0x55c8969b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.018+0000 7f2894996740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969b9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.018+0000 7f2894996740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.018+0000 7f2894996740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c89691b120 con 0x55c8969b9ed0 2024-04-17T07:32:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.018+0000 7f2894996740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c8968c1340 con 0x55c8969b4d60 2024-04-17T07:32:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.018+0000 7f2894996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c8969170e0 con 0x55c8969b4370 2024-04-17T07:32:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.019+0000 7f288aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969b9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.019+0000 7f288aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969b9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37000/0 (socket says 172.21.15.86:37000) 2024-04-17T07:32:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.019+0000 7f288aa00640 1 -- 172.21.15.86:0/3593339330 learned_addr learned my addr 172.21.15.86:0/3593339330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.019+0000 7f2883400640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28700a3c30 0x7f28700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.019+0000 7f2883400640 1 -- 172.21.15.86:0/2914236698 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f288c133fe0 con 0x7f28840a6b90 2024-04-17T07:32:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.020+0000 7f288a000640 1 --2- 172.21.15.86:0/3593339330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 0x55c8969b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.020+0000 7f288a000640 1 --2- 172.21.15.86:0/3593339330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.021+0000 7f288aa00640 1 -- 172.21.15.86:0/3593339330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 msgr2=0x55c8969b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.021+0000 7f288b400640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28700a3c30 0x7f28700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.021+0000 7f288aa00640 1 --2- 172.21.15.86:0/3593339330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 0x55c8969b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.021+0000 7f288aa00640 1 -- 172.21.15.86:0/3593339330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 msgr2=0x55c8969bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:52.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.021+0000 7f288aa00640 1 --2- 172.21.15.86:0/3593339330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969bc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.021+0000 7f288aa00640 1 -- 172.21.15.86:0/3593339330 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8966f77d0 con 0x55c8969b4d60 2024-04-17T07:32:52.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.022+0000 7f288b400640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28700a3c30 0x7f28700a60f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f288c065480 tx=0x7f288c067e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:52.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.022+0000 7f288aa00640 1 --2- 172.21.15.86:0/3593339330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969b9860 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f287800cbb0 tx=0x7f287800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11a026f28c39ad54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.023+0000 7f2877e00640 1 -- 172.21.15.86:0/3593339330 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878005220 con 0x55c8969b4d60 2024-04-17T07:32:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.023+0000 7f2877e00640 1 -- 172.21.15.86:0/3593339330 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2878004c50 con 0x55c8969b4d60 2024-04-17T07:32:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.023+0000 7f2877e00640 1 -- 172.21.15.86:0/3593339330 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878010760 con 0x55c8969b4d60 2024-04-17T07:32:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.024+0000 7f2894996740 1 -- 172.21.15.86:0/3593339330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 msgr2=0x55c8969b9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.024+0000 7f2894996740 1 --2- 172.21.15.86:0/3593339330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969b9860 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f287800cbb0 tx=0x7f287800e720 comp rx=0 tx=0).stop 2024-04-17T07:32:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.025+0000 7f2894996740 1 -- 172.21.15.86:0/3593339330 shutdown_connections 2024-04-17T07:32:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.025+0000 7f2894996740 1 --2- 172.21.15.86:0/3593339330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969bc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.025+0000 7f2894996740 1 --2- 172.21.15.86:0/3593339330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969b9860 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.025+0000 7f2894996740 1 --2- 172.21.15.86:0/3593339330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 0x55c8969b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.025+0000 7f2894996740 1 -- 172.21.15.86:0/3593339330 >> 172.21.15.86:0/3593339330 conn(0x55c8969af8e0 msgr2=0x55c8969afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.025+0000 7f2894996740 1 -- 172.21.15.86:0/3593339330 shutdown_connections 2024-04-17T07:32:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.025+0000 7f2894996740 1 -- 172.21.15.86:0/3593339330 wait complete. 2024-04-17T07:32:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.025+0000 7f2894996740 1 -- start start 2024-04-17T07:32:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.026+0000 7f2894996740 1 Processor -- start 2024-04-17T07:32:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.027+0000 7f2894996740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4370 0x55c8969be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.027+0000 7f2894996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4d60 0x55c8969bee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.027+0000 7f2894996740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.027+0000 7f288a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4370 0x55c8969be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.027+0000 7f288a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4370 0x55c8969be950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:37016/0 (socket says 172.21.15.86:37016) 2024-04-17T07:32:52.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.027+0000 7f288a000640 1 -- 172.21.15.86:0/4008611076 learned_addr learned my addr 172.21.15.86:0/4008611076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:52.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.027+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c8969bce80 con 0x55c8969b9ed0 2024-04-17T07:32:52.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.027+0000 7f288aa00640 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4d60 0x55c8969bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.027+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c8968d4710 con 0x55c8969b4370 2024-04-17T07:32:52.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.027+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c8969bd000 con 0x55c8969b4d60 2024-04-17T07:32:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.028+0000 7f2894996740 10 client.0 _ll_register_callbacks cb 0x55c8968cfaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.028+0000 7f288a000640 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.028+0000 7f288a000640 1 -- 172.21.15.86:0/4008611076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4d60 msgr2=0x55c8969bee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.028+0000 7f288a000640 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4d60 0x55c8969bee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.028+0000 7f288a000640 1 -- 172.21.15.86:0/4008611076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 msgr2=0x55c8969c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.028+0000 7f288a000640 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.028+0000 7f288a000640 1 -- 172.21.15.86:0/4008611076 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89662d3f0 con 0x55c8969b4370 2024-04-17T07:32:52.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.028+0000 7f288aa00640 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4d60 0x55c8969bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:52.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.029+0000 7f288a000640 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4370 0x55c8969be950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f287c009230 tx=0x7f287c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99fa1f7f8070536 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:52.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.029+0000 7f2874c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:32:52.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.029+0000 7f2874c00640 1 -- 172.21.15.86:0/4008611076 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c005250 con 0x55c8969b4370 2024-04-17T07:32:52.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.029+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2878004c50 con 0x55c8969b4370 2024-04-17T07:32:52.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.029+0000 7f2874c00640 1 -- 172.21.15.86:0/4008611076 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f287c0053f0 con 0x55c8969b4370 2024-04-17T07:32:52.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.030+0000 7f2874c00640 1 -- 172.21.15.86:0/4008611076 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f287c051050 con 0x55c8969b4370 2024-04-17T07:32:52.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.030+0000 7f2894996740 10 client.38696 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:52.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.030+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c8965ed0f0 con 0x55c8969b4370 2024-04-17T07:32:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.030+0000 7f2874c00640 1 -- 172.21.15.86:0/4008611076 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2878004c50 con 0x55c8969b4370 2024-04-17T07:32:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.030+0000 7f2874c00640 1 -- 172.21.15.86:0/4008611076 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f287c0035f0 con 0x55c8969b4370 2024-04-17T07:32:52.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.031+0000 7f2894996740 10 client.38696 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:52.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.031+0000 7f2894996740 10 client.38696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:52.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.031+0000 7f2894996740 20 client.38696 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:52.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.031+0000 7f2894996740 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c8969ca6a0 0x55c8969ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.031+0000 7f2894996740 4 client.38696 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:32:52.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.031+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c896928b40 con 0x55c8969ca6a0 2024-04-17T07:32:52.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.031+0000 7f288aa00640 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c8969ca6a0 0x55c8969ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.032+0000 7f288aa00640 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c8969ca6a0 0x55c8969ccb10 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39b542446f87a02 server_cookie=d9621f678c7279e3 in_seq=0 out_seq=0 2024-04-17T07:32:52.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.032+0000 7f2874c00640 10 client.38696 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:52.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.033+0000 7f2874c00640 1 -- 172.21.15.86:0/4008611076 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c896928b40 con 0x55c8969ca6a0 2024-04-17T07:32:52.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.033+0000 7f2874c00640 10 client.38696 handle_command_reply: tid=0 2024-04-17T07:32:52.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.034+0000 7f2894996740 1 client.38696 shutdown 2024-04-17T07:32:52.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.034+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4370 msgr2=0x55c8969be950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:52.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.034+0000 7f2894996740 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4370 0x55c8969be950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f287c009230 tx=0x7f287c005fd0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.034+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 shutdown_connections 2024-04-17T07:32:52.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.034+0000 7f2894996740 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c8969ca6a0 0x55c8969ccb10 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.034+0000 7f2894996740 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.034+0000 7f2894996740 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4d60 0x55c8969bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.034+0000 7f2894996740 1 --2- 172.21.15.86:0/4008611076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4370 0x55c8969be950 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.034+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 >> 172.21.15.86:0/4008611076 conn(0x55c8969af8e0 msgr2=0x55c8969aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:52.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.034+0000 7f2874c00640 0 client.38696 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:52.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.035+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 shutdown_connections 2024-04-17T07:32:52.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.035+0000 7f2894996740 1 -- 172.21.15.86:0/4008611076 wait complete. 2024-04-17T07:32:52.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.036+0000 7f2894996740 20 client.38696 trim_cache size 0 max 16384 2024-04-17T07:32:52.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.075+0000 7f2894996740 1 Processor -- start 2024-04-17T07:32:52.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.075+0000 7f2894996740 1 -- start start 2024-04-17T07:32:52.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.076+0000 7f2894996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 0x55c8969b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.076+0000 7f2894996740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.077+0000 7f2894996740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969bdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.077+0000 7f288a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 0x55c8969b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.077+0000 7f288a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 0x55c8969b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41012/0 (socket says 172.21.15.86:41012) 2024-04-17T07:32:52.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.077+0000 7f2894996740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f288415e7f0 con 0x55c8969b9ed0 2024-04-17T07:32:52.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.077+0000 7f2894996740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2884160330 con 0x55c8969b4d60 2024-04-17T07:32:52.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.077+0000 7f2894996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f288415eec0 con 0x55c8969b4370 2024-04-17T07:32:52.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.077+0000 7f288a000640 1 -- 172.21.15.86:0/3464186485 learned_addr learned my addr 172.21.15.86:0/3464186485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:52.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.077+0000 7f288aa00640 1 --2- 172.21.15.86:0/3464186485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.078+0000 7f288a000640 1 --2- 172.21.15.86:0/3464186485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969bdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.079+0000 7f288aa00640 1 -- 172.21.15.86:0/3464186485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 msgr2=0x55c8969b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.079+0000 7f288aa00640 1 --2- 172.21.15.86:0/3464186485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 0x55c8969b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.079+0000 7f288aa00640 1 -- 172.21.15.86:0/3464186485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 msgr2=0x55c8969bdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.079+0000 7f288aa00640 1 --2- 172.21.15.86:0/3464186485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969bdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.079+0000 7f288aa00640 1 -- 172.21.15.86:0/3464186485 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8969b0330 con 0x55c8969b4d60 2024-04-17T07:32:52.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.080+0000 7f288aa00640 1 --2- 172.21.15.86:0/3464186485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969c5e90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55c8969b0850 tx=0x7f2878005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eb618de0f5f7a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:52.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.081+0000 7f2877400640 1 -- 172.21.15.86:0/3464186485 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878004240 con 0x55c8969b4d60 2024-04-17T07:32:52.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.081+0000 7f2877400640 1 -- 172.21.15.86:0/3464186485 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28780043e0 con 0x55c8969b4d60 2024-04-17T07:32:52.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.081+0000 7f2877400640 1 -- 172.21.15.86:0/3464186485 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28780035f0 con 0x55c8969b4d60 2024-04-17T07:32:52.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.083+0000 7f2894996740 1 -- 172.21.15.86:0/3464186485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 msgr2=0x55c8969c5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:52.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.083+0000 7f2894996740 1 --2- 172.21.15.86:0/3464186485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969c5e90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55c8969b0850 tx=0x7f2878005f60 comp rx=0 tx=0).stop 2024-04-17T07:32:52.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.083+0000 7f2894996740 1 -- 172.21.15.86:0/3464186485 shutdown_connections 2024-04-17T07:32:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.083+0000 7f2894996740 1 --2- 172.21.15.86:0/3464186485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b9ed0 0x55c8969bdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.083+0000 7f2894996740 1 --2- 172.21.15.86:0/3464186485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969b4d60 0x55c8969c5e90 secure :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55c8969b0850 tx=0x7f2878005f60 comp rx=0 tx=0).stop 2024-04-17T07:32:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.083+0000 7f2894996740 1 --2- 172.21.15.86:0/3464186485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b4370 0x55c8969b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.083+0000 7f2894996740 1 -- 172.21.15.86:0/3464186485 >> 172.21.15.86:0/3464186485 conn(0x55c8969af8e0 msgr2=0x55c8969b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.083+0000 7f2894996740 1 -- 172.21.15.86:0/3464186485 shutdown_connections 2024-04-17T07:32:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.083+0000 7f2894996740 1 -- 172.21.15.86:0/3464186485 wait complete. 2024-04-17T07:32:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.084+0000 7f2894996740 1 -- start start 2024-04-17T07:32:52.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.084+0000 7f2894996740 1 Processor -- start 2024-04-17T07:32:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.085+0000 7f2894996740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b4370 0x55c8969c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.085+0000 7f2894996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b9ed0 0x55c8969c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.085+0000 7f2894996740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969c43d0 0x55c8969c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.085+0000 7f2894996740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f288415fc60 con 0x55c8969b4370 2024-04-17T07:32:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.085+0000 7f2894996740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f288415f590 con 0x55c8969c43d0 2024-04-17T07:32:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.085+0000 7f2894996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f288415c7c0 con 0x55c8969b9ed0 2024-04-17T07:32:52.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.085+0000 7f2894996740 10 client.0 _ll_register_callbacks cb 0x7f287c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:32:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.086+0000 7f288a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b4370 0x55c8969c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.086+0000 7f288aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b9ed0 0x55c8969c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.086+0000 7f288aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b9ed0 0x55c8969c33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:41028/0 (socket says 172.21.15.86:41028) 2024-04-17T07:32:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.086+0000 7f288a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b4370 0x55c8969c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54064/0 (socket says 172.21.15.86:54064) 2024-04-17T07:32:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.086+0000 7f288aa00640 1 -- 172.21.15.86:0/3502318199 learned_addr learned my addr 172.21.15.86:0/3502318199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:32:52.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.087+0000 7f288aa00640 1 -- 172.21.15.86:0/3502318199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969c43d0 msgr2=0x55c8969c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.087+0000 7f288a000640 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969c43d0 0x55c8969c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.087+0000 7f288aa00640 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969c43d0 0x55c8969c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.087+0000 7f288aa00640 1 -- 172.21.15.86:0/3502318199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b4370 msgr2=0x55c8969c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:32:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.087+0000 7f288aa00640 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b4370 0x55c8969c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.087+0000 7f288aa00640 1 -- 172.21.15.86:0/3502318199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8969c4d90 con 0x55c8969b9ed0 2024-04-17T07:32:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.088+0000 7f288aa00640 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b9ed0 0x55c8969c33c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f287800e760 tx=0x7f287800ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf8039cb78a6a8f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:32:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.088+0000 7f288a000640 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b4370 0x55c8969c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:32:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.088+0000 7f2874c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:32:52.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.088+0000 7f2874c00640 1 -- 172.21.15.86:0/3502318199 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878036060 con 0x55c8969b9ed0 2024-04-17T07:32:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.089+0000 7f2874c00640 1 -- 172.21.15.86:0/3502318199 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2878029120 con 0x55c8969b9ed0 2024-04-17T07:32:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.089+0000 7f2894996740 1 -- 172.21.15.86:0/3502318199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28780043e0 con 0x55c8969b9ed0 2024-04-17T07:32:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.089+0000 7f2874c00640 1 -- 172.21.15.86:0/3502318199 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2878029320 con 0x55c8969b9ed0 2024-04-17T07:32:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.090+0000 7f2894996740 10 client.28696 fetch_fsmap learned FSMap version 28 2024-04-17T07:32:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.090+0000 7f2894996740 1 -- 172.21.15.86:0/3502318199 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c8969c60a0 con 0x55c8969b9ed0 2024-04-17T07:32:52.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.090+0000 7f2874c00640 1 -- 172.21.15.86:0/3502318199 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28780043e0 con 0x55c8969b9ed0 2024-04-17T07:32:52.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.091+0000 7f2874c00640 1 -- 172.21.15.86:0/3502318199 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f28780104c0 con 0x55c8969b9ed0 2024-04-17T07:32:52.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.091+0000 7f2894996740 10 client.28696 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:32:52.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.091+0000 7f2894996740 10 client.28696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:32:52.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.091+0000 7f2894996740 20 client.28696 populate_metadata read hostname 'smithi086' 2024-04-17T07:32:52.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.091+0000 7f2894996740 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c896a30fe0 0x55c896a33400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:32:52.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.091+0000 7f2894996740 4 client.28696 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:32:52.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.091+0000 7f2894996740 1 -- 172.21.15.86:0/3502318199 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8969229b0 con 0x55c896a30fe0 2024-04-17T07:32:52.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.091+0000 7f288a000640 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c896a30fe0 0x55c896a33400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:32:52.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.093+0000 7f2874c00640 10 client.28696 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:52.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.093+0000 7f288a000640 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c896a30fe0 0x55c896a33400 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7bf28f94f844e server_cookie=3c1a905ef78f1969 in_seq=0 out_seq=0 2024-04-17T07:32:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.094+0000 7f2874c00640 1 -- 172.21.15.86:0/3502318199 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c8969229b0 con 0x55c896a30fe0 2024-04-17T07:32:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.094+0000 7f2874c00640 10 client.28696 handle_command_reply: tid=0 2024-04-17T07:32:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.094+0000 7f2894996740 1 client.28696 shutdown 2024-04-17T07:32:52.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.095+0000 7f2894996740 1 -- 172.21.15.86:0/3502318199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b9ed0 msgr2=0x55c8969c33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.095+0000 7f2894996740 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b9ed0 0x55c8969c33c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f287800e760 tx=0x7f287800ceb0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.095+0000 7f2894996740 1 -- 172.21.15.86:0/3502318199 shutdown_connections 2024-04-17T07:32:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.095+0000 7f2894996740 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c896a30fe0 0x55c896a33400 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.095+0000 7f2894996740 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c8969c43d0 0x55c8969c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.095+0000 7f2894996740 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c8969b9ed0 0x55c8969c33c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.095+0000 7f2894996740 1 --2- 172.21.15.86:0/3502318199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8969b4370 0x55c8969c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.095+0000 7f2894996740 1 -- 172.21.15.86:0/3502318199 >> 172.21.15.86:0/3502318199 conn(0x55c8969af8e0 msgr2=0x55c8969b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.095+0000 7f2874c00640 0 client.28696 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:32:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.095+0000 7f2894996740 1 -- 172.21.15.86:0/3502318199 shutdown_connections 2024-04-17T07:32:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.096+0000 7f2894996740 1 -- 172.21.15.86:0/3502318199 wait complete. 2024-04-17T07:32:52.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.096+0000 7f2894996740 20 client.28696 trim_cache size 0 max 16384 2024-04-17T07:32:52.097 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:32:52.097 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:32:52.097 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:32:52.097 INFO:teuthology.orchestra.run.smithi086.stdout: "2ff1db74-fd7f-4159-9f7e-714a1a4ab239": { 2024-04-17T07:32:52.097 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:32:52.097 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "2ff1db74-fd7f-4159-9f7e-714a1a4ab239", 2024-04-17T07:32:52.097 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:32:52.097 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:32:52.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:32:52.098 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:32:52.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.098+0000 7f288be00640 1 -- 172.21.15.86:0/2914236698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28700a3c30 msgr2=0x7f28700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:52.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.099+0000 7f288be00640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28700a3c30 0x7f28700a60f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f288c065480 tx=0x7f288c067e30 comp rx=0 tx=0).stop 2024-04-17T07:32:52.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.099+0000 7f288be00640 1 -- 172.21.15.86:0/2914236698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28840a6b90 msgr2=0x7f2884142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:32:52.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.099+0000 7f288be00640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28840a6b90 0x7f2884142430 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f288c050aa0 tx=0x7f288c073bf0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.099+0000 7f288be00640 1 -- 172.21.15.86:0/2914236698 shutdown_connections 2024-04-17T07:32:52.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.099+0000 7f288be00640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f28700a3c30 0x7f28700a60f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.099+0000 7f288be00640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28840a6b90 0x7f2884142430 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.099+0000 7f288be00640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28840a61d0 0x7f288413b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.099+0000 7f288be00640 1 --2- 172.21.15.86:0/2914236698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28840a4fe0 0x7f288413ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:32:52.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.099+0000 7f288be00640 1 -- 172.21.15.86:0/2914236698 >> 172.21.15.86:0/2914236698 conn(0x7f28840a0530 msgr2=0x7f28840a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:32:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.101+0000 7f288be00640 1 -- 172.21.15.86:0/2914236698 shutdown_connections 2024-04-17T07:32:52.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:32:52.101+0000 7f288be00640 1 -- 172.21.15.86:0/2914236698 wait complete. 2024-04-17T07:32:52.134 INFO:tasks.cephfs.filesystem:scrub status for tag:2ff1db74-fd7f-4159-9f7e-714a1a4ab239 - {'path': '/', 'tag': '2ff1db74-fd7f-4159-9f7e-714a1a4ab239', 'options': 'recursive,force'} 2024-04-17T07:32:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:52 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:32:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:52 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:52 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:32:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:52 smithi098 ceph-mon[35418]: pgmap v995: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-04-17T07:32:53.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:52 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:52 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:32:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:52 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:52 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:32:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:52 smithi103 ceph-mon[35495]: pgmap v995: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-04-17T07:32:53.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:52 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:53.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:52 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:32:53.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:52 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:53.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:52 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:32:53.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:52 smithi086 ceph-mon[31733]: pgmap v995: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.5 MiB/s wr, 585 op/s 2024-04-17T07:32:53.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:52 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:32:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 810.0 s, 534.8 tps, lat 185.823 ms stddev 79.988 2024-04-17T07:32:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:54 smithi098 ceph-mon[35418]: pgmap v996: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-04-17T07:32:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:54 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:32:55.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:54 smithi103 ceph-mon[35495]: pgmap v996: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-04-17T07:32:55.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:54 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:32:55.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:54 smithi086 ceph-mon[31733]: pgmap v996: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-04-17T07:32:55.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:54 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:32:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:56 smithi098 ceph-mon[35418]: pgmap v997: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.7 MiB/s wr, 556 op/s 2024-04-17T07:32:57.279 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:56 smithi103 ceph-mon[35495]: pgmap v997: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.7 MiB/s wr, 556 op/s 2024-04-17T07:32:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:56 smithi086 ceph-mon[31733]: pgmap v997: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.7 MiB/s wr, 556 op/s 2024-04-17T07:32:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:32:58 smithi098 ceph-mon[35418]: pgmap v998: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.7 MiB/s wr, 596 op/s 2024-04-17T07:32:59.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:32:58 smithi103 ceph-mon[35495]: pgmap v998: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.7 MiB/s wr, 596 op/s 2024-04-17T07:32:59.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:32:58 smithi086 ceph-mon[31733]: pgmap v998: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.7 MiB/s wr, 596 op/s 2024-04-17T07:32:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 815.0 s, 551.8 tps, lat 178.516 ms stddev 135.137 2024-04-17T07:33:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:00 smithi098 ceph-mon[35418]: pgmap v999: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-04-17T07:33:01.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:00 smithi103 ceph-mon[35495]: pgmap v999: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-04-17T07:33:01.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:00 smithi086 ceph-mon[31733]: pgmap v999: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2024-04-17T07:33:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:02 smithi098 ceph-mon[35418]: pgmap v1000: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.9 MiB/s wr, 624 op/s 2024-04-17T07:33:03.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:02 smithi103 ceph-mon[35495]: pgmap v1000: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.9 MiB/s wr, 624 op/s 2024-04-17T07:33:03.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:02 smithi086 ceph-mon[31733]: pgmap v1000: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.9 MiB/s wr, 624 op/s 2024-04-17T07:33:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 820.0 s, 585.0 tps, lat 173.331 ms stddev 153.652 2024-04-17T07:33:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:04 smithi098 ceph-mon[35418]: pgmap v1001: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.4 MiB/s wr, 599 op/s 2024-04-17T07:33:05.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:04 smithi103 ceph-mon[35495]: pgmap v1001: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.4 MiB/s wr, 599 op/s 2024-04-17T07:33:05.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:04 smithi086 ceph-mon[31733]: pgmap v1001: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.4 MiB/s wr, 599 op/s 2024-04-17T07:33:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:06 smithi098 ceph-mon[35418]: pgmap v1002: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 606 op/s 2024-04-17T07:33:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:06 smithi103 ceph-mon[35495]: pgmap v1002: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 606 op/s 2024-04-17T07:33:07.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:06 smithi086 ceph-mon[31733]: pgmap v1002: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.7 MiB/s wr, 606 op/s 2024-04-17T07:33:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:09 smithi103 ceph-mon[35495]: pgmap v1003: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.8 MiB/s wr, 619 op/s 2024-04-17T07:33:09.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:09 smithi086 ceph-mon[31733]: pgmap v1003: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.8 MiB/s wr, 619 op/s 2024-04-17T07:33:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 825.0 s, 552.8 tps, lat 183.163 ms stddev 124.248 2024-04-17T07:33:09.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:09 smithi098 ceph-mon[35418]: pgmap v1003: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.8 MiB/s wr, 619 op/s 2024-04-17T07:33:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:11 smithi086 ceph-mon[31733]: pgmap v1004: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.9 MiB/s wr, 561 op/s 2024-04-17T07:33:11.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:11 smithi098 ceph-mon[35418]: pgmap v1004: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.9 MiB/s wr, 561 op/s 2024-04-17T07:33:11.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:11 smithi103 ceph-mon[35495]: pgmap v1004: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.9 MiB/s wr, 561 op/s 2024-04-17T07:33:13.332 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:13 smithi086 ceph-mon[31733]: pgmap v1005: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.3 MiB/s wr, 585 op/s 2024-04-17T07:33:13.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:13 smithi098 ceph-mon[35418]: pgmap v1005: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.3 MiB/s wr, 585 op/s 2024-04-17T07:33:13.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:13 smithi103 ceph-mon[35495]: pgmap v1005: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.3 MiB/s wr, 585 op/s 2024-04-17T07:33:14.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 830.0 s, 565.4 tps, lat 176.105 ms stddev 109.132 2024-04-17T07:33:15.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:15 smithi086 ceph-mon[31733]: pgmap v1006: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.3 MiB/s wr, 565 op/s 2024-04-17T07:33:15.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:15 smithi098 ceph-mon[35418]: pgmap v1006: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.3 MiB/s wr, 565 op/s 2024-04-17T07:33:15.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:15 smithi103 ceph-mon[35495]: pgmap v1006: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.3 MiB/s wr, 565 op/s 2024-04-17T07:33:17.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:17 smithi086 ceph-mon[31733]: pgmap v1007: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.4 MiB/s wr, 576 op/s 2024-04-17T07:33:17.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:17 smithi098 ceph-mon[35418]: pgmap v1007: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.4 MiB/s wr, 576 op/s 2024-04-17T07:33:17.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:17 smithi103 ceph-mon[35495]: pgmap v1007: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.4 MiB/s wr, 576 op/s 2024-04-17T07:33:19.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:19 smithi086 ceph-mon[31733]: pgmap v1008: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-04-17T07:33:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 835.0 s, 569.2 tps, lat 173.130 ms stddev 98.996 2024-04-17T07:33:19.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:19 smithi098 ceph-mon[35418]: pgmap v1008: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-04-17T07:33:19.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:19 smithi103 ceph-mon[35495]: pgmap v1008: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.2 MiB/s wr, 592 op/s 2024-04-17T07:33:21.333 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:21 smithi086 ceph-mon[31733]: pgmap v1009: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.9 MiB/s wr, 565 op/s 2024-04-17T07:33:21.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:21 smithi098 ceph-mon[35418]: pgmap v1009: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.9 MiB/s wr, 565 op/s 2024-04-17T07:33:21.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:21 smithi103 ceph-mon[35495]: pgmap v1009: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.9 MiB/s wr, 565 op/s 2024-04-17T07:33:22.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:33:22.135 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:33:22.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.470+0000 7f0dd3200640 1 -- 172.21.15.86:0/3869123954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dcc0749c0 msgr2=0x7f0dcc074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.470+0000 7f0dd3200640 1 --2- 172.21.15.86:0/3869123954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dcc0749c0 0x7f0dcc074e40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009930 tx=0x7f0db802bc20 comp rx=0 tx=0).stop 2024-04-17T07:33:22.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.471+0000 7f0dd3200640 1 -- 172.21.15.86:0/3869123954 shutdown_connections 2024-04-17T07:33:22.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.471+0000 7f0dd3200640 1 --2- 172.21.15.86:0/3869123954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dcc10e580 0x7f0dcc110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.471+0000 7f0dd3200640 1 --2- 172.21.15.86:0/3869123954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dcc0749c0 0x7f0dcc074e40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.471+0000 7f0dd3200640 1 --2- 172.21.15.86:0/3869123954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc075fa0 0x7f0dcc0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.471+0000 7f0dd3200640 1 -- 172.21.15.86:0/3869123954 >> 172.21.15.86:0/3869123954 conn(0x7f0dcc0fd540 msgr2=0x7f0dcc0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.471+0000 7f0dd3200640 1 -- 172.21.15.86:0/3869123954 shutdown_connections 2024-04-17T07:33:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.471+0000 7f0dd3200640 1 -- 172.21.15.86:0/3869123954 wait complete. 2024-04-17T07:33:22.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.472+0000 7f0dd3200640 1 Processor -- start 2024-04-17T07:33:22.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.472+0000 7f0dd3200640 1 -- start start 2024-04-17T07:33:22.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.473+0000 7f0dd3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dcc0749c0 0x7f0dcc19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.473+0000 7f0dd3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc075fa0 0x7f0dcc19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.473+0000 7f0dd3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dcc10e580 0x7f0dcc1a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.473+0000 7f0dd3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc114100 con 0x7f0dcc075fa0 2024-04-17T07:33:22.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.473+0000 7f0dd3200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc113f80 con 0x7f0dcc0749c0 2024-04-17T07:33:22.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.473+0000 7f0dd3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc114280 con 0x7f0dcc10e580 2024-04-17T07:33:22.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.473+0000 7f0dcb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc075fa0 0x7f0dcc19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.474+0000 7f0dcb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc075fa0 0x7f0dcc19eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43266/0 (socket says 172.21.15.86:43266) 2024-04-17T07:33:22.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.474+0000 7f0dcb400640 1 -- 172.21.15.86:0/3929569398 learned_addr learned my addr 172.21.15.86:0/3929569398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:22.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.474+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dcc0749c0 0x7f0dcc19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.474+0000 7f0dd1200640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dcc10e580 0x7f0dcc1a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.474+0000 7f0dcb400640 1 -- 172.21.15.86:0/3929569398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dcc10e580 msgr2=0x7f0dcc1a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.474+0000 7f0dcb400640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dcc10e580 0x7f0dcc1a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.474+0000 7f0dcb400640 1 -- 172.21.15.86:0/3929569398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dcc0749c0 msgr2=0x7f0dcc19e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.474+0000 7f0dcb400640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dcc0749c0 0x7f0dcc19e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.474+0000 7f0dcb400640 1 -- 172.21.15.86:0/3929569398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc1a6210 con 0x7f0dcc075fa0 2024-04-17T07:33:22.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.475+0000 7f0dcb400640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc075fa0 0x7f0dcc19eb30 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009850 tx=0x7f0db80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:22.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.475+0000 7f0dc8c00640 1 -- 172.21.15.86:0/3929569398 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8043020 con 0x7f0dcc075fa0 2024-04-17T07:33:22.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.475+0000 7f0dc8c00640 1 -- 172.21.15.86:0/3929569398 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db80054c0 con 0x7f0dcc075fa0 2024-04-17T07:33:22.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.475+0000 7f0dc8c00640 1 -- 172.21.15.86:0/3929569398 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db803e450 con 0x7f0dcc075fa0 2024-04-17T07:33:22.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.477+0000 7f0dd3200640 1 -- 172.21.15.86:0/3929569398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc1a64a0 con 0x7f0dcc075fa0 2024-04-17T07:33:22.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.477+0000 7f0dd3200640 1 -- 172.21.15.86:0/3929569398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc1a6970 con 0x7f0dcc075fa0 2024-04-17T07:33:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.482+0000 7f0dd5e43740 1 Processor -- start 2024-04-17T07:33:22.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.482+0000 7f0dd5e43740 1 -- start start 2024-04-17T07:33:22.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.483+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15e8190 0x564dd15e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.483+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15ed680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.483+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15edcf0 0x564dd15f0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.483+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564dd154f120 con 0x564dd15e8190 2024-04-17T07:33:22.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.483+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564dd14f5340 con 0x564dd15edcf0 2024-04-17T07:33:22.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.483+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564dd154b0e0 con 0x564dd15e8b80 2024-04-17T07:33:22.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.484+0000 7f0dcbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.484+0000 7f0dcbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15ed680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45040/0 (socket says 172.21.15.86:45040) 2024-04-17T07:33:22.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.484+0000 7f0dcbe00640 1 -- 172.21.15.86:0/4171416591 learned_addr learned my addr 172.21.15.86:0/4171416591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:22.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.485+0000 7f0dcbe00640 1 -- 172.21.15.86:0/4171416591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15edcf0 msgr2=0x564dd15f0180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:33:22.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.485+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/4171416591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15edcf0 0x564dd15f0180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.485+0000 7f0dcbe00640 1 -- 172.21.15.86:0/4171416591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15e8190 msgr2=0x564dd15e85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:22.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.485+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/4171416591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15e8190 0x564dd15e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.485+0000 7f0dcbe00640 1 -- 172.21.15.86:0/4171416591 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd132b7d0 con 0x564dd15e8b80 2024-04-17T07:33:22.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.485+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/4171416591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15ed680 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00cc60 tx=0x7f0dbc00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ddfe783b633e5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.486+0000 7f0dc1600640 1 -- 172.21.15.86:0/4171416591 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc014070 con 0x564dd15e8b80 2024-04-17T07:33:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.486+0000 7f0dc1600640 1 -- 172.21.15.86:0/4171416591 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dbc0047a0 con 0x564dd15e8b80 2024-04-17T07:33:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.486+0000 7f0dc1600640 1 -- 172.21.15.86:0/4171416591 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc010690 con 0x564dd15e8b80 2024-04-17T07:33:22.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.486+0000 7f0dc8c00640 1 -- 172.21.15.86:0/3929569398 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0db8046020 con 0x7f0dcc075fa0 2024-04-17T07:33:22.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.487+0000 7f0dd5e43740 1 -- 172.21.15.86:0/4171416591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 msgr2=0x564dd15ed680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.487+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/4171416591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15ed680 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00cc60 tx=0x7f0dbc00e720 comp rx=0 tx=0).stop 2024-04-17T07:33:22.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.487+0000 7f0dd5e43740 1 -- 172.21.15.86:0/4171416591 shutdown_connections 2024-04-17T07:33:22.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.487+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/4171416591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15edcf0 0x564dd15f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.487+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/4171416591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15ed680 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.487+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/4171416591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15e8190 0x564dd15e85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.487+0000 7f0dd5e43740 1 -- 172.21.15.86:0/4171416591 >> 172.21.15.86:0/4171416591 conn(0x564dd15e3700 msgr2=0x564dd15e3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:22.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.488+0000 7f0dc8c00640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0db00a3d00 0x7f0db00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.488+0000 7f0dd5e43740 1 -- 172.21.15.86:0/4171416591 shutdown_connections 2024-04-17T07:33:22.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.488+0000 7f0dc8c00640 1 -- 172.21.15.86:0/3929569398 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0db80f2570 con 0x7f0dcc075fa0 2024-04-17T07:33:22.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.489+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0db00a3d00 0x7f0db00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.489+0000 7f0dd5e43740 1 -- 172.21.15.86:0/4171416591 wait complete. 2024-04-17T07:33:22.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.490+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0db00a3d00 0x7f0db00a61c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00c9f0 tx=0x7f0dbc01a670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:22.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.490+0000 7f0dd5e43740 1 -- start start 2024-04-17T07:33:22.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.491+0000 7f0dd5e43740 1 Processor -- start 2024-04-17T07:33:22.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.492+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8190 0x564dd15f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.492+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.492+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15edcf0 0x564dd15f9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.492+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564dd15f0ca0 con 0x564dd15edcf0 2024-04-17T07:33:22.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.492+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564dd1508710 con 0x564dd15e8190 2024-04-17T07:33:22.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.492+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564dd15f0e20 con 0x564dd15e8b80 2024-04-17T07:33:22.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.492+0000 7f0dd5e43740 10 client.0 _ll_register_callbacks cb 0x564dd1503af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:22.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.493+0000 7f0dcb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.493+0000 7f0dcbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8190 0x564dd15f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.493+0000 7f0dcb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15f2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45042/0 (socket says 172.21.15.86:45042) 2024-04-17T07:33:22.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.493+0000 7f0dcb400640 1 -- 172.21.15.86:0/2496615074 learned_addr learned my addr 172.21.15.86:0/2496615074 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:22.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.494+0000 7f0dcb400640 1 -- 172.21.15.86:0/2496615074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8190 msgr2=0x564dd15f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:22.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.494+0000 7f0dcb400640 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8190 0x564dd15f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.494+0000 7f0dcb400640 1 -- 172.21.15.86:0/2496615074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15edcf0 msgr2=0x564dd15f9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:22.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.494+0000 7f0dcb400640 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15edcf0 0x564dd15f9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.494+0000 7f0dcb400640 1 -- 172.21.15.86:0/2496615074 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd12613f0 con 0x564dd15e8b80 2024-04-17T07:33:22.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.495+0000 7f0dcb400640 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15f2ca0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0db80ba930 tx=0x7f0db80ba830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc01e81772547f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:22.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.495+0000 7f0db6000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:33:22.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.495+0000 7f0db6000640 1 -- 172.21.15.86:0/2496615074 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db80d6020 con 0x564dd15e8b80 2024-04-17T07:33:22.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.495+0000 7f0db6000640 1 -- 172.21.15.86:0/2496615074 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0db8003680 con 0x564dd15e8b80 2024-04-17T07:33:22.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.495+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2496615074 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dbc0047a0 con 0x564dd15e8b80 2024-04-17T07:33:22.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.496+0000 7f0db6000640 1 -- 172.21.15.86:0/2496615074 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0db80044e0 con 0x564dd15e8b80 2024-04-17T07:33:22.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.496+0000 7f0dd5e43740 10 client.28708 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:22.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.496+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2496615074 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564dd12210f0 con 0x564dd15e8b80 2024-04-17T07:33:22.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.497+0000 7f0db6000640 1 -- 172.21.15.86:0/2496615074 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dbc0047a0 con 0x564dd15e8b80 2024-04-17T07:33:22.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.497+0000 7f0db6000640 1 -- 172.21.15.86:0/2496615074 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0db80dd2a0 con 0x564dd15e8b80 2024-04-17T07:33:22.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.498+0000 7f0dd5e43740 10 client.28708 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:22.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.498+0000 7f0dd5e43740 10 client.28708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:33:22.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.498+0000 7f0dd5e43740 20 client.28708 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:22.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.498+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564dd15fe4a0 0x564dd1600890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.498+0000 7f0dd5e43740 4 client.28708 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:22.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.498+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2496615074 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dd155cb40 con 0x564dd15fe4a0 2024-04-17T07:33:22.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.500+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564dd15fe4a0 0x564dd1600890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.504+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564dd15fe4a0 0x564dd1600890 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d4b3d3b383eae0 server_cookie=6d8f0cc8dee645ba in_seq=0 out_seq=0 2024-04-17T07:33:22.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.504+0000 7f0db6000640 10 client.28708 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:22.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.506+0000 7f0db6000640 1 -- 172.21.15.86:0/2496615074 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564dd155cb40 con 0x564dd15fe4a0 2024-04-17T07:33:22.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.506+0000 7f0db6000640 10 client.28708 handle_command_reply: tid=0 2024-04-17T07:33:22.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.506+0000 7f0dd5e43740 1 client.28708 shutdown 2024-04-17T07:33:22.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2496615074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 msgr2=0x564dd15f2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15f2ca0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0db80ba930 tx=0x7f0db80ba830 comp rx=0 tx=0).stop 2024-04-17T07:33:22.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2496615074 shutdown_connections 2024-04-17T07:33:22.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564dd15fe4a0 0x564dd1600890 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15edcf0 0x564dd15f9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8b80 0x564dd15f2ca0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0db6000640 0 client.28708 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/2496615074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8190 0x564dd15f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2496615074 >> 172.21.15.86:0/2496615074 conn(0x564dd15e3700 msgr2=0x564dd15e2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2496615074 shutdown_connections 2024-04-17T07:33:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.507+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2496615074 wait complete. 2024-04-17T07:33:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.508+0000 7f0dd5e43740 20 client.28708 trim_cache size 0 max 16384 2024-04-17T07:33:22.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.555+0000 7f0dd5e43740 1 Processor -- start 2024-04-17T07:33:22.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.556+0000 7f0dd5e43740 1 -- start start 2024-04-17T07:33:22.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.557+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 0x564dd15e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.557+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15e8b80 0x564dd15f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.557+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15edcf0 0x564dd15e2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.557+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1b0f30 con 0x564dd15e8b80 2024-04-17T07:33:22.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.557+0000 7f0dcbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 0x564dd15e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.558+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1b2a70 con 0x564dd15edcf0 2024-04-17T07:33:22.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.558+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1b1600 con 0x564dd15e8190 2024-04-17T07:33:22.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.558+0000 7f0dd1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15edcf0 0x564dd15e2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.558+0000 7f0dcbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 0x564dd15e3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45048/0 (socket says 172.21.15.86:45048) 2024-04-17T07:33:22.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.558+0000 7f0dcbe00640 1 -- 172.21.15.86:0/2356883648 learned_addr learned my addr 172.21.15.86:0/2356883648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:22.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.558+0000 7f0dcb400640 1 --2- 172.21.15.86:0/2356883648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15e8b80 0x564dd15f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.559+0000 7f0dcb400640 1 -- 172.21.15.86:0/2356883648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 msgr2=0x564dd15e3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.559+0000 7f0dcb400640 1 --2- 172.21.15.86:0/2356883648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 0x564dd15e3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.559+0000 7f0dcb400640 1 -- 172.21.15.86:0/2356883648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15edcf0 msgr2=0x564dd15e2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.559+0000 7f0dcb400640 1 --2- 172.21.15.86:0/2356883648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15edcf0 0x564dd15e2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.559+0000 7f0dcb400640 1 -- 172.21.15.86:0/2356883648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd15ddce0 con 0x564dd15e8b80 2024-04-17T07:33:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.559+0000 7f0dd1200640 1 --2- 172.21.15.86:0/2356883648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15edcf0 0x564dd15e2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:33:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.560+0000 7f0dcb400640 1 --2- 172.21.15.86:0/2356883648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15e8b80 0x564dd15f9c00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0db80bbec0 tx=0x7f0db80beaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7bf2cde9166813 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:22.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.560+0000 7f0dc0c00640 1 -- 172.21.15.86:0/2356883648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db80d6070 con 0x564dd15e8b80 2024-04-17T07:33:22.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.561+0000 7f0dc0c00640 1 -- 172.21.15.86:0/2356883648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db80bebd0 con 0x564dd15e8b80 2024-04-17T07:33:22.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.561+0000 7f0dc0c00640 1 -- 172.21.15.86:0/2356883648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db80c7140 con 0x564dd15e8b80 2024-04-17T07:33:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.562+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2356883648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15e8b80 msgr2=0x564dd15f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.562+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/2356883648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15e8b80 0x564dd15f9c00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0db80bbec0 tx=0x7f0db80beaf0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.563+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2356883648 shutdown_connections 2024-04-17T07:33:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.563+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/2356883648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15edcf0 0x564dd15e2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.563+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/2356883648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15e8b80 0x564dd15f9c00 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.563+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/2356883648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 0x564dd15e3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.563+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2356883648 >> 172.21.15.86:0/2356883648 conn(0x564dd15e3700 msgr2=0x564dd15ea110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.563+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2356883648 shutdown_connections 2024-04-17T07:33:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.563+0000 7f0dd5e43740 1 -- 172.21.15.86:0/2356883648 wait complete. 2024-04-17T07:33:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.564+0000 7f0dd5e43740 1 -- start start 2024-04-17T07:33:22.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.565+0000 7f0dd5e43740 1 Processor -- start 2024-04-17T07:33:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.566+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 0x564dd15f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.566+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8b80 0x564dd15f7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.566+0000 7f0dd5e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15edcf0 0x564dd15f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.566+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1b23a0 con 0x564dd15edcf0 2024-04-17T07:33:22.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.566+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1b1cd0 con 0x564dd15e8b80 2024-04-17T07:33:22.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.566+0000 7f0dd5e43740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc1aeea0 con 0x564dd15e8190 2024-04-17T07:33:22.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.566+0000 7f0dd5e43740 10 client.0 _ll_register_callbacks cb 0x7f0db8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:22.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.568+0000 7f0dd1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15edcf0 0x564dd15f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.568+0000 7f0dcb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8b80 0x564dd15f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.568+0000 7f0dcb400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8b80 0x564dd15f7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54642/0 (socket says 172.21.15.86:54642) 2024-04-17T07:33:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.568+0000 7f0dcb400640 1 -- 172.21.15.86:0/1384634689 learned_addr learned my addr 172.21.15.86:0/1384634689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.571+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 0x564dd15f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.572+0000 7f0dcb400640 1 -- 172.21.15.86:0/1384634689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 msgr2=0x564dd15f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.572+0000 7f0dcb400640 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 0x564dd15f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.572+0000 7f0dcb400640 1 -- 172.21.15.86:0/1384634689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15edcf0 msgr2=0x564dd15f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.572+0000 7f0dcb400640 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15edcf0 0x564dd15f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.572+0000 7f0dcb400640 1 -- 172.21.15.86:0/1384634689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd15f8dd0 con 0x564dd15e8b80 2024-04-17T07:33:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.572+0000 7f0dcb400640 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8b80 0x564dd15f7820 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0db80bed80 tx=0x7f0db80bc9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc812093d12e31b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.572+0000 7f0db6000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:33:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.573+0000 7f0db6000640 1 -- 172.21.15.86:0/1384634689 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db80cc070 con 0x564dd15e8b80 2024-04-17T07:33:22.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.573+0000 7f0db6000640 1 -- 172.21.15.86:0/1384634689 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0db8003680 con 0x564dd15e8b80 2024-04-17T07:33:22.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.575+0000 7f0dd5e43740 1 -- 172.21.15.86:0/1384634689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db80bebd0 con 0x564dd15e8b80 2024-04-17T07:33:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.576+0000 7f0db6000640 1 -- 172.21.15.86:0/1384634689 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0db80d6050 con 0x564dd15e8b80 2024-04-17T07:33:22.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.576+0000 7f0dd5e43740 10 client.38714 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:22.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.577+0000 7f0dd5e43740 1 -- 172.21.15.86:0/1384634689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564dd1675a20 con 0x564dd15e8b80 2024-04-17T07:33:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.578+0000 7f0db6000640 1 -- 172.21.15.86:0/1384634689 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db80bebd0 con 0x564dd15e8b80 2024-04-17T07:33:22.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.578+0000 7f0db6000640 1 -- 172.21.15.86:0/1384634689 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0db80cf020 con 0x564dd15e8b80 2024-04-17T07:33:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.578+0000 7f0dd5e43740 10 client.38714 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.579+0000 7f0dd5e43740 10 client.38714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:33:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.579+0000 7f0dd5e43740 20 client.38714 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.579+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564dd1676040 0x564dd1678460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.579+0000 7f0dd5e43740 4 client.38714 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:33:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.579+0000 7f0dd5e43740 1 -- 172.21.15.86:0/1384634689 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564dd15569b0 con 0x564dd1676040 2024-04-17T07:33:22.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.580+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564dd1676040 0x564dd1678460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.583+0000 7f0dcbe00640 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564dd1676040 0x564dd1678460 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3e50b16d4e7605 server_cookie=31b928e81f78d88 in_seq=0 out_seq=0 2024-04-17T07:33:22.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.583+0000 7f0db6000640 10 client.38714 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:22.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.583+0000 7f0db6000640 1 -- 172.21.15.86:0/1384634689 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564dd15569b0 con 0x564dd1676040 2024-04-17T07:33:22.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.583+0000 7f0db6000640 10 client.38714 handle_command_reply: tid=0 2024-04-17T07:33:22.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.584+0000 7f0dd5e43740 1 client.38714 shutdown 2024-04-17T07:33:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0dd5e43740 1 -- 172.21.15.86:0/1384634689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8b80 msgr2=0x564dd15f7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8b80 0x564dd15f7820 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0db80bed80 tx=0x7f0db80bc9e0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0dd5e43740 1 -- 172.21.15.86:0/1384634689 shutdown_connections 2024-04-17T07:33:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x564dd1676040 0x564dd1678460 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0db6000640 0 client.38714 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564dd15edcf0 0x564dd15f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564dd15e8b80 0x564dd15f7820 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0dd5e43740 1 --2- 172.21.15.86:0/1384634689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564dd15e8190 0x564dd15f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0dd5e43740 1 -- 172.21.15.86:0/1384634689 >> 172.21.15.86:0/1384634689 conn(0x564dd15e3700 msgr2=0x564dd15e2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:22.589 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:33:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:33:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:33:22.590 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:33:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0dd5e43740 1 -- 172.21.15.86:0/1384634689 shutdown_connections 2024-04-17T07:33:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.588+0000 7f0dd5e43740 1 -- 172.21.15.86:0/1384634689 wait complete. 2024-04-17T07:33:22.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.589+0000 7f0dd5e43740 20 client.38714 trim_cache size 0 max 16384 2024-04-17T07:33:22.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.590+0000 7f0dd3200640 1 -- 172.21.15.86:0/3929569398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0db00a3d00 msgr2=0x7f0db00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.590+0000 7f0dd3200640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0db00a3d00 0x7f0db00a61c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00c9f0 tx=0x7f0dbc01a670 comp rx=0 tx=0).stop 2024-04-17T07:33:22.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.590+0000 7f0dd3200640 1 -- 172.21.15.86:0/3929569398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc075fa0 msgr2=0x7f0dcc19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.590+0000 7f0dd3200640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc075fa0 0x7f0dcc19eb30 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009850 tx=0x7f0db80042f0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.591+0000 7f0dd3200640 1 -- 172.21.15.86:0/3929569398 shutdown_connections 2024-04-17T07:33:22.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.591+0000 7f0dd3200640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0db00a3d00 0x7f0db00a61c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.591+0000 7f0dd3200640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0dcc10e580 0x7f0dcc1a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.591+0000 7f0dd3200640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dcc075fa0 0x7f0dcc19eb30 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.591+0000 7f0dd3200640 1 --2- 172.21.15.86:0/3929569398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0dcc0749c0 0x7f0dcc19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.591+0000 7f0dd3200640 1 -- 172.21.15.86:0/3929569398 >> 172.21.15.86:0/3929569398 conn(0x7f0dcc0fd540 msgr2=0x7f0dcc101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:22.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.591+0000 7f0dd3200640 1 -- 172.21.15.86:0/3929569398 shutdown_connections 2024-04-17T07:33:22.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.591+0000 7f0dd3200640 1 -- 172.21.15.86:0/3929569398 wait complete. 2024-04-17T07:33:22.633 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:33:22.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:33:22.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.944+0000 7ffbdbe00640 1 -- 172.21.15.86:0/9371451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc1019b0 msgr2=0x7ffbdc101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.944+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/9371451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc1019b0 0x7ffbdc101e30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009900 tx=0x7ffbd402bc40 comp rx=0 tx=0).stop 2024-04-17T07:33:22.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.944+0000 7ffbdbe00640 1 -- 172.21.15.86:0/9371451 shutdown_connections 2024-04-17T07:33:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.944+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/9371451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc1019b0 0x7ffbdc101e30 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.944+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/9371451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbdc101010 0x7ffbdc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.944+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/9371451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc1006b0 0x7ffbdc100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.944+0000 7ffbdbe00640 1 -- 172.21.15.86:0/9371451 >> 172.21.15.86:0/9371451 conn(0x7ffbdc0fc230 msgr2=0x7ffbdc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.944+0000 7ffbdbe00640 1 -- 172.21.15.86:0/9371451 shutdown_connections 2024-04-17T07:33:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.944+0000 7ffbdbe00640 1 -- 172.21.15.86:0/9371451 wait complete. 2024-04-17T07:33:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.946+0000 7ffbdbe00640 1 Processor -- start 2024-04-17T07:33:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.946+0000 7ffbdbe00640 1 -- start start 2024-04-17T07:33:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.946+0000 7ffbdbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbdc1006b0 0x7ffbdc106230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.947+0000 7ffbdbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc101010 0x7ffbdc106770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.947+0000 7ffbdaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbdc1006b0 0x7ffbdc106230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.947+0000 7ffbdaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbdc1006b0 0x7ffbdc106230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45058/0 (socket says 172.21.15.86:45058) 2024-04-17T07:33:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.947+0000 7ffbdbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc1019b0 0x7ffbdc102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.947+0000 7ffbdbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc10fb30 con 0x7ffbdc101010 2024-04-17T07:33:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.947+0000 7ffbdbe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc10f9b0 con 0x7ffbdc1019b0 2024-04-17T07:33:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.947+0000 7ffbdbe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc10fcb0 con 0x7ffbdc1006b0 2024-04-17T07:33:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.947+0000 7ffbdaa00640 1 -- 172.21.15.86:0/1447007683 learned_addr learned my addr 172.21.15.86:0/1447007683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.948+0000 7ffbd2000640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc101010 0x7ffbdc106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.948+0000 7ffbdb400640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc1019b0 0x7ffbdc102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.948+0000 7ffbdaa00640 1 -- 172.21.15.86:0/1447007683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc1019b0 msgr2=0x7ffbdc102da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.948+0000 7ffbdaa00640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc1019b0 0x7ffbdc102da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.948+0000 7ffbdaa00640 1 -- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc101010 msgr2=0x7ffbdc106770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.948+0000 7ffbdaa00640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc101010 0x7ffbdc106770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.948+0000 7ffbdaa00640 1 -- 172.21.15.86:0/1447007683 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc103660 con 0x7ffbdc1006b0 2024-04-17T07:33:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.948+0000 7ffbd2000640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc101010 0x7ffbdc106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:33:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.949+0000 7ffbdaa00640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbdc1006b0 0x7ffbdc106230 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00eab0 tx=0x7ffbcc00ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.950+0000 7ffbd3e00640 1 -- 172.21.15.86:0/1447007683 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00ce40 con 0x7ffbdc1006b0 2024-04-17T07:33:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.950+0000 7ffbd3e00640 1 -- 172.21.15.86:0/1447007683 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbcc011920 con 0x7ffbdc1006b0 2024-04-17T07:33:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.950+0000 7ffbd3e00640 1 -- 172.21.15.86:0/1447007683 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc010a00 con 0x7ffbdc1006b0 2024-04-17T07:33:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.950+0000 7ffbdbe00640 1 -- 172.21.15.86:0/1447007683 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbdc1a65f0 con 0x7ffbdc1006b0 2024-04-17T07:33:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.950+0000 7ffbdbe00640 1 -- 172.21.15.86:0/1447007683 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbdc1a6a00 con 0x7ffbdc1006b0 2024-04-17T07:33:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.953+0000 7ffbdbe00640 1 -- 172.21.15.86:0/1447007683 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb80052c0 con 0x7ffbdc1006b0 2024-04-17T07:33:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.958+0000 7ffbd3e00640 1 -- 172.21.15.86:0/1447007683 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ffbcc011470 con 0x7ffbdc1006b0 2024-04-17T07:33:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.963+0000 7ffbd3e00640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffbc00a3d30 0x7ffbc00a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:22.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.964+0000 7ffbd3e00640 1 -- 172.21.15.86:0/1447007683 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ffbcc0d0bc0 con 0x7ffbdc1006b0 2024-04-17T07:33:22.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.965+0000 7ffbd2000640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffbc00a3d30 0x7ffbc00a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:22.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.966+0000 7ffbd2000640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffbc00a3d30 0x7ffbc00a61f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc1041b0 tx=0x7ffbc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:22.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:22.967+0000 7ffbd3e00640 1 -- 172.21.15.86:0/1447007683 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffbcc016030 con 0x7ffbdc1006b0 2024-04-17T07:33:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:23 smithi098 ceph-mon[35418]: pgmap v1010: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-04-17T07:33:23.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.508+0000 7ffbdbe00640 1 -- 172.21.15.86:0/1447007683 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbb80055b0 con 0x7ffbdc1006b0 2024-04-17T07:33:23.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.512+0000 7ffbd3e00640 1 -- 172.21.15.86:0/1447007683 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7ffbcc09d220 con 0x7ffbdc1006b0 2024-04-17T07:33:23.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:33:23.513 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:33:23.514 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:33:23.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.517+0000 7ffbdbe00640 1 -- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffbc00a3d30 msgr2=0x7ffbc00a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.517+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffbc00a3d30 0x7ffbc00a61f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc1041b0 tx=0x7ffbc4005f90 comp rx=0 tx=0).stop 2024-04-17T07:33:23.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.517+0000 7ffbdbe00640 1 -- 172.21.15.86:0/1447007683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbdc1006b0 msgr2=0x7ffbdc106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.517+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbdc1006b0 0x7ffbdc106230 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00eab0 tx=0x7ffbcc00ef70 comp rx=0 tx=0).stop 2024-04-17T07:33:23.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.517+0000 7ffbdbe00640 1 -- 172.21.15.86:0/1447007683 shutdown_connections 2024-04-17T07:33:23.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.517+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ffbc00a3d30 0x7ffbc00a61f0 secure :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc1041b0 tx=0x7ffbc4005f90 comp rx=0 tx=0).stop 2024-04-17T07:33:23.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.517+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbdc1019b0 0x7ffbdc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.517+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbdc101010 0x7ffbdc106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.517+0000 7ffbdbe00640 1 --2- 172.21.15.86:0/1447007683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbdc1006b0 0x7ffbdc106230 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.517+0000 7ffbdbe00640 1 -- 172.21.15.86:0/1447007683 >> 172.21.15.86:0/1447007683 conn(0x7ffbdc0fc230 msgr2=0x7ffbdc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:23.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.518+0000 7ffbdbe00640 1 -- 172.21.15.86:0/1447007683 shutdown_connections 2024-04-17T07:33:23.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.518+0000 7ffbdbe00640 1 -- 172.21.15.86:0/1447007683 wait complete. 2024-04-17T07:33:23.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:23 smithi103 ceph-mon[35495]: pgmap v1010: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-04-17T07:33:23.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:23 smithi086 ceph-mon[31733]: pgmap v1010: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-04-17T07:33:23.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:33:23.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.890+0000 7fc9afe00640 1 -- 172.21.15.86:0/554298192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9a8094fb0 msgr2=0x7fc9a8095410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.890+0000 7fc9afe00640 1 --2- 172.21.15.86:0/554298192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9a8094fb0 0x7fc9a8095410 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009900 tx=0x7fc99c02bc50 comp rx=0 tx=0).stop 2024-04-17T07:33:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.890+0000 7fc9afe00640 1 -- 172.21.15.86:0/554298192 shutdown_connections 2024-04-17T07:33:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.890+0000 7fc9afe00640 1 --2- 172.21.15.86:0/554298192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9a8095950 0x7fc9a8095dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.890+0000 7fc9afe00640 1 --2- 172.21.15.86:0/554298192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9a8094fb0 0x7fc9a8095410 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.890+0000 7fc9afe00640 1 --2- 172.21.15.86:0/554298192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9a8094650 0x7fc9a8094a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.890+0000 7fc9afe00640 1 -- 172.21.15.86:0/554298192 >> 172.21.15.86:0/554298192 conn(0x7fc9a80901d0 msgr2=0x7fc9a8092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.890+0000 7fc9afe00640 1 -- 172.21.15.86:0/554298192 shutdown_connections 2024-04-17T07:33:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.890+0000 7fc9afe00640 1 -- 172.21.15.86:0/554298192 wait complete. 2024-04-17T07:33:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.891+0000 7fc9afe00640 1 Processor -- start 2024-04-17T07:33:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.892+0000 7fc9afe00640 1 -- start start 2024-04-17T07:33:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.892+0000 7fc9afe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9a8094650 0x7fc9a813f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.892+0000 7fc9afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9a8094fb0 0x7fc9a813f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.892+0000 7fc9afe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9a8095950 0x7fc9a81467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.892+0000 7fc9afe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a80a3b80 con 0x7fc9a8094fb0 2024-04-17T07:33:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.892+0000 7fc9afe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a80a3a00 con 0x7fc9a8095950 2024-04-17T07:33:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.893+0000 7fc9afe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a80a3d00 con 0x7fc9a8094650 2024-04-17T07:33:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.893+0000 7fc9aea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9a8094650 0x7fc9a813f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.893+0000 7fc9aea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9a8094650 0x7fc9a813f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45068/0 (socket says 172.21.15.86:45068) 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.893+0000 7fc9aea00640 1 -- 172.21.15.86:0/2486434942 learned_addr learned my addr 172.21.15.86:0/2486434942 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.893+0000 7fc9af400640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9a8095950 0x7fc9a81467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.894+0000 7fc9ae000640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9a8094fb0 0x7fc9a813f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.894+0000 7fc9aea00640 1 -- 172.21.15.86:0/2486434942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9a8095950 msgr2=0x7fc9a81467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.894+0000 7fc9aea00640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9a8095950 0x7fc9a81467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.894+0000 7fc9aea00640 1 -- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9a8094fb0 msgr2=0x7fc9a813f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.894+0000 7fc9aea00640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9a8094fb0 0x7fc9a813f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.894+0000 7fc9aea00640 1 -- 172.21.15.86:0/2486434942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a80968e0 con 0x7fc9a8094650 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.894+0000 7fc9ae000640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9a8094fb0 0x7fc9a813f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:33:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.894+0000 7fc9af400640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9a8095950 0x7fc9a81467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:33:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.897+0000 7fc9aea00640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9a8094650 0x7fc9a813f280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000e9f0 tx=0x7fc9a000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:23.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.898+0000 7fc9a7400640 1 -- 172.21.15.86:0/2486434942 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a000cd80 con 0x7fc9a8094650 2024-04-17T07:33:23.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.898+0000 7fc9afe00640 1 -- 172.21.15.86:0/2486434942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a8096bd0 con 0x7fc9a8094650 2024-04-17T07:33:23.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.898+0000 7fc9a7400640 1 -- 172.21.15.86:0/2486434942 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a00054d0 con 0x7fc9a8094650 2024-04-17T07:33:23.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.898+0000 7fc9afe00640 1 -- 172.21.15.86:0/2486434942 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a80971a0 con 0x7fc9a8094650 2024-04-17T07:33:23.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.898+0000 7fc9a7400640 1 -- 172.21.15.86:0/2486434942 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0010490 con 0x7fc9a8094650 2024-04-17T07:33:23.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.902+0000 7fc9b8842740 1 Processor -- start 2024-04-17T07:33:23.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.903+0000 7fc9b8842740 1 -- start start 2024-04-17T07:33:23.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.903+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 0x557ae121a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.903+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121ab50 0x557ae121f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.903+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 0x557ae1222150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.903+0000 7fc9b8842740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ae1180020 con 0x557ae121ab50 2024-04-17T07:33:23.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.903+0000 7fc9b8842740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557ae1126460 con 0x557ae121fcc0 2024-04-17T07:33:23.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.903+0000 7fc9b8842740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ae117bfe0 con 0x557ae121a160 2024-04-17T07:33:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.904+0000 7fc9ae000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 0x557ae1222150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.904+0000 7fc9ae000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 0x557ae1222150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54676/0 (socket says 172.21.15.86:54676) 2024-04-17T07:33:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.904+0000 7fc9ae000640 1 -- 172.21.15.86:0/28484135 learned_addr learned my addr 172.21.15.86:0/28484135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.904+0000 7fc9af400640 1 --2- 172.21.15.86:0/28484135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 0x557ae121a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.904+0000 7fc9ae000640 1 -- 172.21.15.86:0/28484135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 msgr2=0x557ae121a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.904+0000 7fc9ae000640 1 --2- 172.21.15.86:0/28484135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 0x557ae121a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.904+0000 7fc9ae000640 1 -- 172.21.15.86:0/28484135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121ab50 msgr2=0x557ae121f650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:33:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.904+0000 7fc9ae000640 1 --2- 172.21.15.86:0/28484135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121ab50 0x557ae121f650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.904+0000 7fc9ae000640 1 -- 172.21.15.86:0/28484135 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae112e040 con 0x557ae121fcc0 2024-04-17T07:33:23.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.904+0000 7fc9a7400640 1 -- 172.21.15.86:0/2486434942 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc9a0005020 con 0x7fc9a8094650 2024-04-17T07:33:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.905+0000 7fc9ae000640 1 --2- 172.21.15.86:0/28484135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 0x557ae1222150 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006010 tx=0x7fc99c005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eba12c55964a550c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:23.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.905+0000 7fc99be00640 1 -- 172.21.15.86:0/28484135 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c043020 con 0x557ae121fcc0 2024-04-17T07:33:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.906+0000 7fc99be00640 1 -- 172.21.15.86:0/28484135 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc99c004040 con 0x557ae121fcc0 2024-04-17T07:33:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.906+0000 7fc99be00640 1 -- 172.21.15.86:0/28484135 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0046c0 con 0x557ae121fcc0 2024-04-17T07:33:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.907+0000 7fc9b8842740 1 -- 172.21.15.86:0/28484135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 msgr2=0x557ae1222150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.907+0000 7fc9b8842740 1 --2- 172.21.15.86:0/28484135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 0x557ae1222150 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc99c006010 tx=0x7fc99c005d10 comp rx=0 tx=0).stop 2024-04-17T07:33:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.907+0000 7fc9b8842740 1 -- 172.21.15.86:0/28484135 shutdown_connections 2024-04-17T07:33:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.907+0000 7fc9b8842740 1 --2- 172.21.15.86:0/28484135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 0x557ae1222150 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.907+0000 7fc9b8842740 1 --2- 172.21.15.86:0/28484135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121ab50 0x557ae121f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.907+0000 7fc9b8842740 1 --2- 172.21.15.86:0/28484135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 0x557ae121a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.907+0000 7fc9a7400640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9940a3bb0 0x7fc9940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.907+0000 7fc9b8842740 1 -- 172.21.15.86:0/28484135 >> 172.21.15.86:0/28484135 conn(0x557ae12156d0 msgr2=0x557ae1215ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.907+0000 7fc9a7400640 1 -- 172.21.15.86:0/2486434942 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc9a00d0cc0 con 0x7fc9a8094650 2024-04-17T07:33:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.908+0000 7fc9b8842740 1 -- 172.21.15.86:0/28484135 shutdown_connections 2024-04-17T07:33:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.908+0000 7fc9b8842740 1 -- 172.21.15.86:0/28484135 wait complete. 2024-04-17T07:33:23.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.908+0000 7fc9af400640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9940a3bb0 0x7fc9940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.908+0000 7fc9b8842740 1 -- start start 2024-04-17T07:33:23.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.909+0000 7fc9b8842740 1 Processor -- start 2024-04-17T07:33:23.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.910+0000 7fc9af400640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9940a3bb0 0x7fc9940a6070 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0065480 tx=0x7fc9b006d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:23.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.911+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121a160 0x557ae1224670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.911+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 0x557ae1224bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.912+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 0x557ae122bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.912+0000 7fc9aea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121a160 0x557ae1224670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.912+0000 7fc9b8842740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ae1222c10 con 0x557ae121fcc0 2024-04-17T07:33:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.912+0000 7fc9b8842740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557ae1139830 con 0x557ae121a160 2024-04-17T07:33:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.912+0000 7fc9b8842740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ae1222d90 con 0x557ae121ab50 2024-04-17T07:33:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.912+0000 7fc9b8842740 10 client.0 _ll_register_callbacks cb 0x557ae1107fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.912+0000 7fc9af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 0x557ae122bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.912+0000 7fc9af400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 0x557ae122bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43358/0 (socket says 172.21.15.86:43358) 2024-04-17T07:33:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.912+0000 7fc9af400640 1 -- 172.21.15.86:0/537276993 learned_addr learned my addr 172.21.15.86:0/537276993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:23.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.913+0000 7fc9ae000640 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 0x557ae1224bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.913+0000 7fc9af400640 1 -- 172.21.15.86:0/537276993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 msgr2=0x557ae1224bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.913+0000 7fc9af400640 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 0x557ae1224bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.913+0000 7fc9af400640 1 -- 172.21.15.86:0/537276993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121a160 msgr2=0x557ae1224670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.913+0000 7fc9af400640 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121a160 0x557ae1224670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.914+0000 7fc9af400640 1 -- 172.21.15.86:0/537276993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae113d920 con 0x557ae121fcc0 2024-04-17T07:33:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.914+0000 7fc9af400640 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 0x557ae122bbf0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0067ad0 tx=0x7fc9b0067e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfe0f31cd3ac0b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:23.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.915+0000 7fc998c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:33:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.919+0000 7fc998c00640 1 -- 172.21.15.86:0/537276993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b006acf0 con 0x557ae121fcc0 2024-04-17T07:33:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.919+0000 7fc998c00640 1 -- 172.21.15.86:0/537276993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc9b0079460 con 0x557ae121fcc0 2024-04-17T07:33:23.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.920+0000 7fc9b8842740 1 -- 172.21.15.86:0/537276993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc99c004040 con 0x557ae121fcc0 2024-04-17T07:33:23.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.921+0000 7fc998c00640 1 -- 172.21.15.86:0/537276993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc9b0092050 con 0x557ae121fcc0 2024-04-17T07:33:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.922+0000 7fc9b8842740 10 client.19686 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.922+0000 7fc9b8842740 1 -- 172.21.15.86:0/537276993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ae110b7f0 con 0x557ae121fcc0 2024-04-17T07:33:23.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.922+0000 7fc998c00640 1 -- 172.21.15.86:0/537276993 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc99c004040 con 0x557ae121fcc0 2024-04-17T07:33:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.923+0000 7fc998c00640 1 -- 172.21.15.86:0/537276993 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc9b008d030 con 0x557ae121fcc0 2024-04-17T07:33:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.924+0000 7fc9b8842740 10 client.19686 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.924+0000 7fc9b8842740 10 client.19686 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:33:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.924+0000 7fc9b8842740 20 client.19686 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.924+0000 7fc9b8842740 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557ae1230240 0x557ae1232660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.924+0000 7fc9b8842740 4 client.19686 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:23.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.924+0000 7fc9b8842740 1 -- 172.21.15.86:0/537276993 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ae118da40 con 0x557ae1230240 2024-04-17T07:33:23.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.925+0000 7fc9aea00640 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557ae1230240 0x557ae1232660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.926+0000 7fc9aea00640 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557ae1230240 0x557ae1232660 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89425da6a31d9df7 server_cookie=ce824a9a5dda047a in_seq=0 out_seq=0 2024-04-17T07:33:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.926+0000 7fc998c00640 10 client.19686 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:33:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.927+0000 7fc998c00640 1 -- 172.21.15.86:0/537276993 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ae118da40 con 0x557ae1230240 2024-04-17T07:33:23.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.927+0000 7fc998c00640 10 client.19686 handle_command_reply: tid=0 2024-04-17T07:33:23.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.930+0000 7fc9b8842740 1 client.19686 shutdown 2024-04-17T07:33:23.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.930+0000 7fc9b8842740 1 -- 172.21.15.86:0/537276993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 msgr2=0x557ae122bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.930+0000 7fc9b8842740 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 0x557ae122bbf0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0067ad0 tx=0x7fc9b0067e30 comp rx=0 tx=0).stop 2024-04-17T07:33:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.931+0000 7fc9b8842740 1 -- 172.21.15.86:0/537276993 shutdown_connections 2024-04-17T07:33:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.931+0000 7fc9b8842740 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557ae1230240 0x557ae1232660 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.931+0000 7fc9b8842740 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 0x557ae122bbf0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.931+0000 7fc9b8842740 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 0x557ae1224bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.931+0000 7fc9b8842740 1 --2- 172.21.15.86:0/537276993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121a160 0x557ae1224670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.931+0000 7fc9b8842740 1 -- 172.21.15.86:0/537276993 >> 172.21.15.86:0/537276993 conn(0x557ae12156d0 msgr2=0x557ae1214930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.931+0000 7fc998c00640 0 client.19686 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:33:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.931+0000 7fc9b8842740 1 -- 172.21.15.86:0/537276993 shutdown_connections 2024-04-17T07:33:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.931+0000 7fc9b8842740 1 -- 172.21.15.86:0/537276993 wait complete. 2024-04-17T07:33:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.932+0000 7fc9b8842740 20 client.19686 trim_cache size 0 max 16384 2024-04-17T07:33:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.967+0000 7fc9b8842740 1 Processor -- start 2024-04-17T07:33:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.968+0000 7fc9b8842740 1 -- start start 2024-04-17T07:33:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.968+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121a160 0x557ae12140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.968+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 0x557ae122bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.968+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 0x557ae12145e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.968+0000 7fc9b8842740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8155f20 con 0x557ae121a160 2024-04-17T07:33:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.968+0000 7fc9b8842740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8157a60 con 0x557ae121fcc0 2024-04-17T07:33:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.968+0000 7fc9b8842740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81565f0 con 0x557ae121ab50 2024-04-17T07:33:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.968+0000 7fc9ae000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 0x557ae122bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.968+0000 7fc9ae000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 0x557ae122bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45100/0 (socket says 172.21.15.86:45100) 2024-04-17T07:33:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.968+0000 7fc9ae000640 1 -- 172.21.15.86:0/4204891109 learned_addr learned my addr 172.21.15.86:0/4204891109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.969+0000 7fc9af400640 1 --2- 172.21.15.86:0/4204891109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 0x557ae12145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.969+0000 7fc9aea00640 1 --2- 172.21.15.86:0/4204891109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121a160 0x557ae12140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.969+0000 7fc9ae000640 1 -- 172.21.15.86:0/4204891109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 msgr2=0x557ae12145e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.969+0000 7fc9ae000640 1 --2- 172.21.15.86:0/4204891109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 0x557ae12145e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.970+0000 7fc9ae000640 1 -- 172.21.15.86:0/4204891109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121a160 msgr2=0x557ae12140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.970+0000 7fc9ae000640 1 --2- 172.21.15.86:0/4204891109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121a160 0x557ae12140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.970+0000 7fc9ae000640 1 -- 172.21.15.86:0/4204891109 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae0e923f0 con 0x557ae121ab50 2024-04-17T07:33:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.970+0000 7fc9aea00640 1 --2- 172.21.15.86:0/4204891109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121a160 0x557ae12140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:33:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.970+0000 7fc9ae000640 1 --2- 172.21.15.86:0/4204891109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 0x557ae122bbb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x557ae1216640 tx=0x7fc99c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c01e75b91f0f0808 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.971+0000 7fc99b400640 1 -- 172.21.15.86:0/4204891109 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c043040 con 0x557ae121ab50 2024-04-17T07:33:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.971+0000 7fc99b400640 1 -- 172.21.15.86:0/4204891109 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc99c02cc00 con 0x557ae121ab50 2024-04-17T07:33:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.971+0000 7fc99b400640 1 -- 172.21.15.86:0/4204891109 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c004920 con 0x557ae121ab50 2024-04-17T07:33:23.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.973+0000 7fc9b8842740 1 -- 172.21.15.86:0/4204891109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 msgr2=0x557ae122bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.973+0000 7fc9b8842740 1 --2- 172.21.15.86:0/4204891109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 0x557ae122bbb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x557ae1216640 tx=0x7fc99c005e80 comp rx=0 tx=0).stop 2024-04-17T07:33:23.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.976+0000 7fc9b8842740 1 -- 172.21.15.86:0/4204891109 shutdown_connections 2024-04-17T07:33:23.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.976+0000 7fc9b8842740 1 --2- 172.21.15.86:0/4204891109 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121fcc0 0x557ae12145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.976+0000 7fc9b8842740 1 --2- 172.21.15.86:0/4204891109 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121ab50 0x557ae122bbb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.976+0000 7fc9b8842740 1 --2- 172.21.15.86:0/4204891109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121a160 0x557ae12140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.976+0000 7fc9b8842740 1 -- 172.21.15.86:0/4204891109 >> 172.21.15.86:0/4204891109 conn(0x557ae12156d0 msgr2=0x557ae1224670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.976+0000 7fc9b8842740 1 -- 172.21.15.86:0/4204891109 shutdown_connections 2024-04-17T07:33:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.976+0000 7fc9b8842740 1 -- 172.21.15.86:0/4204891109 wait complete. 2024-04-17T07:33:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.977+0000 7fc9b8842740 1 -- start start 2024-04-17T07:33:23.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.977+0000 7fc9b8842740 1 Processor -- start 2024-04-17T07:33:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.977+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 0x557ae1228d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.977+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121ab50 0x557ae1229250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.977+0000 7fc9b8842740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 0x557ae122a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.978+0000 7fc9aea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 0x557ae1228d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.978+0000 7fc9aea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 0x557ae1228d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45108/0 (socket says 172.21.15.86:45108) 2024-04-17T07:33:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.978+0000 7fc9aea00640 1 -- 172.21.15.86:0/2027567342 learned_addr learned my addr 172.21.15.86:0/2027567342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.978+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8157390 con 0x557ae121fcc0 2024-04-17T07:33:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.978+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8156cc0 con 0x557ae121ab50 2024-04-17T07:33:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.978+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8153ef0 con 0x557ae121a160 2024-04-17T07:33:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.978+0000 7fc9b8842740 10 client.0 _ll_register_callbacks cb 0x7fc9b006b4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.978+0000 7fc9af400640 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 0x557ae122a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.979+0000 7fc9ae000640 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121ab50 0x557ae1229250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.979+0000 7fc9aea00640 1 -- 172.21.15.86:0/2027567342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121ab50 msgr2=0x557ae1229250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:23.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.979+0000 7fc9aea00640 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121ab50 0x557ae1229250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.979+0000 7fc9aea00640 1 -- 172.21.15.86:0/2027567342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 msgr2=0x557ae122a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:23.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.979+0000 7fc9aea00640 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 0x557ae122a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.979+0000 7fc9aea00640 1 -- 172.21.15.86:0/2027567342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae0f5c7d0 con 0x557ae121a160 2024-04-17T07:33:23.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.979+0000 7fc9aea00640 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 0x557ae1228d10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc9a009cb50 tx=0x7fc9a0004aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c14d3cc46694a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:23.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.979+0000 7fc998c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:33:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.981+0000 7fc998c00640 1 -- 172.21.15.86:0/2027567342 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00b5070 con 0x557ae121a160 2024-04-17T07:33:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.981+0000 7fc998c00640 1 -- 172.21.15.86:0/2027567342 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc9a0016dd0 con 0x557ae121a160 2024-04-17T07:33:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.981+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc99c02cc00 con 0x557ae121a160 2024-04-17T07:33:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.981+0000 7fc998c00640 1 -- 172.21.15.86:0/2027567342 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc9a00a0f70 con 0x557ae121a160 2024-04-17T07:33:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.982+0000 7fc9b8842740 10 client.28738 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:23.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.982+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ae122ba50 con 0x557ae121a160 2024-04-17T07:33:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.982+0000 7fc998c00640 1 -- 172.21.15.86:0/2027567342 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc99c02cc00 con 0x557ae121a160 2024-04-17T07:33:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.983+0000 7fc998c00640 1 -- 172.21.15.86:0/2027567342 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc9a0099230 con 0x557ae121a160 2024-04-17T07:33:23.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.983+0000 7fc9b8842740 10 client.28738 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.983+0000 7fc9b8842740 10 client.28738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:33:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.983+0000 7fc9b8842740 20 client.28738 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.983+0000 7fc9b8842740 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557ae1296940 0x557ae1298d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.983+0000 7fc9b8842740 4 client.28738 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:33:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.983+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ae1224460 con 0x557ae1296940 2024-04-17T07:33:23.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.984+0000 7fc9ae000640 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557ae1296940 0x557ae1298d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:23.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.986+0000 7fc998c00640 10 client.28738 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:33:23.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.986+0000 7fc9ae000640 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557ae1296940 0x557ae1298d60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cce9fd51f8479262 server_cookie=9aa2f84b182cd27d in_seq=0 out_seq=0 2024-04-17T07:33:23.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.987+0000 7fc998c00640 1 -- 172.21.15.86:0/2027567342 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ae1224460 con 0x557ae1296940 2024-04-17T07:33:23.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.987+0000 7fc998c00640 10 client.28738 handle_command_reply: tid=0 2024-04-17T07:33:23.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.987+0000 7fc9b8842740 1 client.28738 shutdown 2024-04-17T07:33:23.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.989+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 msgr2=0x557ae1228d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.989+0000 7fc9b8842740 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 0x557ae1228d10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc9a009cb50 tx=0x7fc9a0004aa0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.989+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 shutdown_connections 2024-04-17T07:33:23.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.989+0000 7fc9b8842740 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x557ae1296940 0x557ae1298d60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.989+0000 7fc9b8842740 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557ae121fcc0 0x557ae122a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.989+0000 7fc9b8842740 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557ae121ab50 0x557ae1229250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.989+0000 7fc9b8842740 1 --2- 172.21.15.86:0/2027567342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ae121a160 0x557ae1228d10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.989+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 >> 172.21.15.86:0/2027567342 conn(0x557ae12156d0 msgr2=0x557ae1224670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:23.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.989+0000 7fc998c00640 0 client.28738 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:33:23.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.990+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 shutdown_connections 2024-04-17T07:33:23.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.990+0000 7fc9b8842740 1 -- 172.21.15.86:0/2027567342 wait complete. 2024-04-17T07:33:23.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.990+0000 7fc9b8842740 20 client.28738 trim_cache size 0 max 16384 2024-04-17T07:33:23.991 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:33:23.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.991+0000 7fc9afe00640 1 -- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9940a3bb0 msgr2=0x7fc9940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.991+0000 7fc9afe00640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9940a3bb0 0x7fc9940a6070 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0065480 tx=0x7fc9b006d040 comp rx=0 tx=0).stop 2024-04-17T07:33:23.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.991+0000 7fc9afe00640 1 -- 172.21.15.86:0/2486434942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9a8094650 msgr2=0x7fc9a813f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:23.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.991+0000 7fc9afe00640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9a8094650 0x7fc9a813f280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000e9f0 tx=0x7fc9a000edf0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.992+0000 7fc9afe00640 1 -- 172.21.15.86:0/2486434942 shutdown_connections 2024-04-17T07:33:23.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.992+0000 7fc9afe00640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc9940a3bb0 0x7fc9940a6070 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.992+0000 7fc9afe00640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9a8095950 0x7fc9a81467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.992+0000 7fc9afe00640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9a8094fb0 0x7fc9a813f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.992+0000 7fc9afe00640 1 --2- 172.21.15.86:0/2486434942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9a8094650 0x7fc9a813f280 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:23.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.992+0000 7fc9afe00640 1 -- 172.21.15.86:0/2486434942 >> 172.21.15.86:0/2486434942 conn(0x7fc9a80901d0 msgr2=0x7fc9a8091cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:23.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.992+0000 7fc9afe00640 1 -- 172.21.15.86:0/2486434942 shutdown_connections 2024-04-17T07:33:23.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:23.993+0000 7fc9afe00640 1 -- 172.21.15.86:0/2486434942 wait complete. 2024-04-17T07:33:24.025 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:33:24.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:24 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1447007683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:33:24.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.334+0000 7fce08c00640 1 -- 172.21.15.86:0/2128882784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04077b60 msgr2=0x7fce04077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.334+0000 7fce08c00640 1 --2- 172.21.15.86:0/2128882784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04077b60 0x7fce04077fc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009900 tx=0x7fcdf002bc30 comp rx=0 tx=0).stop 2024-04-17T07:33:24.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.334+0000 7fce08c00640 1 -- 172.21.15.86:0/2128882784 shutdown_connections 2024-04-17T07:33:24.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.334+0000 7fce08c00640 1 --2- 172.21.15.86:0/2128882784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04078500 0x7fce04078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.334+0000 7fce08c00640 1 --2- 172.21.15.86:0/2128882784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04077b60 0x7fce04077fc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.334+0000 7fce08c00640 1 --2- 172.21.15.86:0/2128882784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce04076920 0x7fce04076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.334+0000 7fce08c00640 1 -- 172.21.15.86:0/2128882784 >> 172.21.15.86:0/2128882784 conn(0x7fce040ff750 msgr2=0x7fce04101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.334+0000 7fce08c00640 1 -- 172.21.15.86:0/2128882784 shutdown_connections 2024-04-17T07:33:24.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.334+0000 7fce08c00640 1 -- 172.21.15.86:0/2128882784 wait complete. 2024-04-17T07:33:24.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.335+0000 7fce08c00640 1 Processor -- start 2024-04-17T07:33:24.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.336+0000 7fce08c00640 1 -- start start 2024-04-17T07:33:24.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.336+0000 7fce08c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04076920 0x7fce0419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.336+0000 7fce08c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04077b60 0x7fce0419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.336+0000 7fce08c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce04078500 0x7fce041a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.336+0000 7fce08c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce04114160 con 0x7fce04077b60 2024-04-17T07:33:24.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.336+0000 7fce08c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce04113fe0 con 0x7fce04078500 2024-04-17T07:33:24.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.336+0000 7fce08c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce041142e0 con 0x7fce04076920 2024-04-17T07:33:24.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.337+0000 7fce01e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04076920 0x7fce0419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.337+0000 7fce01e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04076920 0x7fce0419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45126/0 (socket says 172.21.15.86:45126) 2024-04-17T07:33:24.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.337+0000 7fce01e00640 1 -- 172.21.15.86:0/2698611855 learned_addr learned my addr 172.21.15.86:0/2698611855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:24.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.337+0000 7fce01400640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04077b60 0x7fce0419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.337+0000 7fce01e00640 1 -- 172.21.15.86:0/2698611855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce04078500 msgr2=0x7fce041a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.337+0000 7fce01e00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce04078500 0x7fce041a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.337+0000 7fce01e00640 1 -- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04077b60 msgr2=0x7fce0419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.338+0000 7fce01e00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04077b60 0x7fce0419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.338+0000 7fce01e00640 1 -- 172.21.15.86:0/2698611855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0406bab0 con 0x7fce04076920 2024-04-17T07:33:24.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.338+0000 7fce01400640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04077b60 0x7fce0419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:33:24.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.338+0000 7fce01e00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04076920 0x7fce0419e790 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400d770 tx=0x7fcdf400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.339+0000 7fcdfaa00640 1 -- 172.21.15.86:0/2698611855 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4017070 con 0x7fce04076920 2024-04-17T07:33:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.339+0000 7fcdfaa00640 1 -- 172.21.15.86:0/2698611855 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdf40054d0 con 0x7fce04076920 2024-04-17T07:33:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.339+0000 7fcdfaa00640 1 -- 172.21.15.86:0/2698611855 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4004c80 con 0x7fce04076920 2024-04-17T07:33:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.339+0000 7fce08c00640 1 -- 172.21.15.86:0/2698611855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0406bda0 con 0x7fce04076920 2024-04-17T07:33:24.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.340+0000 7fce08c00640 1 -- 172.21.15.86:0/2698611855 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0406c270 con 0x7fce04076920 2024-04-17T07:33:24.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.345+0000 7fce0b818740 1 Processor -- start 2024-04-17T07:33:24.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.346+0000 7fce0b818740 1 -- start start 2024-04-17T07:33:24.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.346+0000 7fcdfaa00640 1 -- 172.21.15.86:0/2698611855 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fcdf4004e20 con 0x7fce04076920 2024-04-17T07:33:24.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.346+0000 7fce0b818740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x5604905263c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.346+0000 7fce0b818740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490526990 0x56049052b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.346+0000 7fce0b818740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x56049052df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.346+0000 7fce0b818740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56049048c020 con 0x560490525fa0 2024-04-17T07:33:24.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.346+0000 7fce0b818740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560490432460 con 0x56049052bb00 2024-04-17T07:33:24.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.346+0000 7fce0b818740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560490487fe0 con 0x560490526990 2024-04-17T07:33:24.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.347+0000 7fce02800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x5604905263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.347+0000 7fce02800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x5604905263c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43410/0 (socket says 172.21.15.86:43410) 2024-04-17T07:33:24.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.348+0000 7fce02800640 1 -- 172.21.15.86:0/397709626 learned_addr learned my addr 172.21.15.86:0/397709626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:24.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.348+0000 7fce02800640 1 -- 172.21.15.86:0/397709626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490526990 msgr2=0x56049052b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.348+0000 7fce02800640 1 --2- 172.21.15.86:0/397709626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490526990 0x56049052b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.348+0000 7fce02800640 1 -- 172.21.15.86:0/397709626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 msgr2=0x56049052df90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:33:24.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.348+0000 7fce02800640 1 --2- 172.21.15.86:0/397709626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x56049052df90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.348+0000 7fce02800640 1 -- 172.21.15.86:0/397709626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56049043a040 con 0x560490525fa0 2024-04-17T07:33:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.349+0000 7fcdfaa00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcde80a3c30 0x7fcde80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.349+0000 7fcdfaa00640 1 -- 172.21.15.86:0/2698611855 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcdf40d01d0 con 0x7fce04076920 2024-04-17T07:33:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.349+0000 7fce02800640 1 --2- 172.21.15.86:0/397709626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x5604905263c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc007f50 tx=0x7fcdfc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7198f39a1b12fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.350+0000 7fce01e00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcde80a3c30 0x7fcde80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.350+0000 7fcdef000640 1 -- 172.21.15.86:0/397709626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc014070 con 0x560490525fa0 2024-04-17T07:33:24.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.350+0000 7fcdef000640 1 -- 172.21.15.86:0/397709626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdfc00fb00 con 0x560490525fa0 2024-04-17T07:33:24.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.350+0000 7fcdef000640 1 -- 172.21.15.86:0/397709626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc014070 con 0x560490525fa0 2024-04-17T07:33:24.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.351+0000 7fce0b818740 1 -- 172.21.15.86:0/397709626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 msgr2=0x5604905263c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.351+0000 7fce0b818740 1 --2- 172.21.15.86:0/397709626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x5604905263c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc007f50 tx=0x7fcdfc00c760 comp rx=0 tx=0).stop 2024-04-17T07:33:24.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.351+0000 7fce01e00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcde80a3c30 0x7fcde80a60f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4098aa0 tx=0x7fcdf4004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.352+0000 7fce0b818740 1 -- 172.21.15.86:0/397709626 shutdown_connections 2024-04-17T07:33:24.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.352+0000 7fce0b818740 1 --2- 172.21.15.86:0/397709626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x56049052df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.352+0000 7fce0b818740 1 --2- 172.21.15.86:0/397709626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490526990 0x56049052b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.352+0000 7fce0b818740 1 --2- 172.21.15.86:0/397709626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x5604905263c0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.352+0000 7fce0b818740 1 -- 172.21.15.86:0/397709626 >> 172.21.15.86:0/397709626 conn(0x560490521510 msgr2=0x560490521910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.353+0000 7fce0b818740 1 -- 172.21.15.86:0/397709626 shutdown_connections 2024-04-17T07:33:24.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.353+0000 7fce0b818740 1 -- 172.21.15.86:0/397709626 wait complete. 2024-04-17T07:33:24.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.353+0000 7fce0b818740 1 -- start start 2024-04-17T07:33:24.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.354+0000 7fce0b818740 1 Processor -- start 2024-04-17T07:33:24.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.354+0000 7fce0b818740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x560490530520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.354+0000 7fce0b818740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490526990 0x560490530a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.354+0000 7fce0b818740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x560490537aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.354+0000 7fce0b818740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56049052eab0 con 0x560490525fa0 2024-04-17T07:33:24.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.354+0000 7fce0b818740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560490445830 con 0x56049052bb00 2024-04-17T07:33:24.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.354+0000 7fce0b818740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56049052ec30 con 0x560490526990 2024-04-17T07:33:24.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.354+0000 7fce0b818740 10 client.0 _ll_register_callbacks cb 0x560490413fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:24.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.355+0000 7fce01400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490526990 0x560490530a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.355+0000 7fce01400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490526990 0x560490530a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45144/0 (socket says 172.21.15.86:45144) 2024-04-17T07:33:24.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.355+0000 7fce01400640 1 -- 172.21.15.86:0/2825593609 learned_addr learned my addr 172.21.15.86:0/2825593609 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:24.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.355+0000 7fce02800640 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x560490530520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.355+0000 7fce02800640 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x560490537aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.355+0000 7fce02800640 1 -- 172.21.15.86:0/2825593609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490526990 msgr2=0x560490530a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.355+0000 7fce02800640 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490526990 0x560490530a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.355+0000 7fce02800640 1 -- 172.21.15.86:0/2825593609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 msgr2=0x560490537aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.355+0000 7fce02800640 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x560490537aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.355+0000 7fce02800640 1 -- 172.21.15.86:0/2825593609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560490449920 con 0x560490525fa0 2024-04-17T07:33:24.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.356+0000 7fce02800640 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x560490530520 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000f80 tx=0x7fcdfc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148ff2699aa865bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.356+0000 7fcde3400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:33:24.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.356+0000 7fcde3400640 1 -- 172.21.15.86:0/2825593609 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc014040 con 0x560490525fa0 2024-04-17T07:33:24.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.356+0000 7fce0b818740 1 -- 172.21.15.86:0/2825593609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdfc00fb00 con 0x560490525fa0 2024-04-17T07:33:24.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.357+0000 7fcde3400640 1 -- 172.21.15.86:0/2825593609 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcdfc003970 con 0x560490525fa0 2024-04-17T07:33:24.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.357+0000 7fcde3400640 1 -- 172.21.15.86:0/2825593609 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcdfc003bb0 con 0x560490525fa0 2024-04-17T07:33:24.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.357+0000 7fce0b818740 10 client.19698 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:24.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.357+0000 7fce0b818740 1 -- 172.21.15.86:0/2825593609 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604904177f0 con 0x560490525fa0 2024-04-17T07:33:24.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.357+0000 7fcde3400640 1 -- 172.21.15.86:0/2825593609 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdfc00fb00 con 0x560490525fa0 2024-04-17T07:33:24.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.358+0000 7fcde3400640 1 -- 172.21.15.86:0/2825593609 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcdfc025070 con 0x560490525fa0 2024-04-17T07:33:24.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.358+0000 7fce0b818740 10 client.19698 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:24.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.358+0000 7fce0b818740 10 client.19698 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:33:24.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.358+0000 7fce0b818740 20 client.19698 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:24.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.358+0000 7fce0b818740 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56049053be00 0x56049053e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.358+0000 7fce0b818740 4 client.19698 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:24.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.358+0000 7fce0b818740 1 -- 172.21.15.86:0/2825593609 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560490499a40 con 0x56049053be00 2024-04-17T07:33:24.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.358+0000 7fce01400640 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56049053be00 0x56049053e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.359+0000 7fcde3400640 10 client.19698 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:33:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.359+0000 7fce01400640 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56049053be00 0x56049053e1f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=823334c856d1eae8 server_cookie=f752d375e02a51f0 in_seq=0 out_seq=0 2024-04-17T07:33:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 840.0 s, 522.6 tps, lat 194.213 ms stddev 173.385 2024-04-17T07:33:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.360+0000 7fcde3400640 1 -- 172.21.15.86:0/2825593609 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560490499a40 con 0x56049053be00 2024-04-17T07:33:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fcde3400640 10 client.19698 handle_command_reply: tid=0 2024-04-17T07:33:24.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fce0b818740 1 client.19698 shutdown 2024-04-17T07:33:24.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fce0b818740 1 -- 172.21.15.86:0/2825593609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 msgr2=0x560490530520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fce0b818740 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x560490530520 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000f80 tx=0x7fcdfc005f90 comp rx=0 tx=0).stop 2024-04-17T07:33:24.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fce0b818740 1 -- 172.21.15.86:0/2825593609 shutdown_connections 2024-04-17T07:33:24.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fce0b818740 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x56049053be00 0x56049053e1f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fce0b818740 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x560490537aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fcde3400640 0 client.19698 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:33:24.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fce0b818740 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490526990 0x560490530a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fce0b818740 1 --2- 172.21.15.86:0/2825593609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490525fa0 0x560490530520 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.361+0000 7fce0b818740 1 -- 172.21.15.86:0/2825593609 >> 172.21.15.86:0/2825593609 conn(0x560490521510 msgr2=0x560490520710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.362+0000 7fce0b818740 1 -- 172.21.15.86:0/2825593609 shutdown_connections 2024-04-17T07:33:24.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.362+0000 7fce0b818740 1 -- 172.21.15.86:0/2825593609 wait complete. 2024-04-17T07:33:24.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.362+0000 7fce0b818740 20 client.19698 trim_cache size 0 max 16384 2024-04-17T07:33:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.393+0000 7fce0b818740 1 Processor -- start 2024-04-17T07:33:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.393+0000 7fce0b818740 1 -- start start 2024-04-17T07:33:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.394+0000 7fce0b818740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 0x5604905309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.394+0000 7fce0b818740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490526990 0x560490537a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.394+0000 7fce0b818740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x560490524e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.394+0000 7fce0b818740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce041b9900 con 0x560490526990 2024-04-17T07:33:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.394+0000 7fce0b818740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce041bb440 con 0x56049052bb00 2024-04-17T07:33:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.394+0000 7fce0b818740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce041b9fd0 con 0x560490525fa0 2024-04-17T07:33:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.394+0000 7fce02800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 0x5604905309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.395+0000 7fce01400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490526990 0x560490537a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.395+0000 7fce01400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490526990 0x560490537a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43430/0 (socket says 172.21.15.86:43430) 2024-04-17T07:33:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.395+0000 7fce01400640 1 -- 172.21.15.86:0/2166839349 learned_addr learned my addr 172.21.15.86:0/2166839349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.395+0000 7fce02800640 1 --2- 172.21.15.86:0/2166839349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x560490524e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.395+0000 7fce01400640 1 -- 172.21.15.86:0/2166839349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 msgr2=0x5604905309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.395+0000 7fce01400640 1 --2- 172.21.15.86:0/2166839349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 0x5604905309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.395+0000 7fce01400640 1 -- 172.21.15.86:0/2166839349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 msgr2=0x560490524e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.395+0000 7fce01400640 1 --2- 172.21.15.86:0/2166839349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x560490524e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.395+0000 7fce01400640 1 -- 172.21.15.86:0/2166839349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56049019e3f0 con 0x560490526990 2024-04-17T07:33:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.396+0000 7fce01400640 1 --2- 172.21.15.86:0/2166839349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490526990 0x560490537a30 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0036040 tx=0x7fcdf0005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3db67272371b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.396+0000 7fcdee600640 1 -- 172.21.15.86:0/2166839349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf004e070 con 0x560490526990 2024-04-17T07:33:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.397+0000 7fcdee600640 1 -- 172.21.15.86:0/2166839349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdf00054f0 con 0x560490526990 2024-04-17T07:33:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.397+0000 7fcdee600640 1 -- 172.21.15.86:0/2166839349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0004a20 con 0x560490526990 2024-04-17T07:33:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.398+0000 7fce0b818740 1 -- 172.21.15.86:0/2166839349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490526990 msgr2=0x560490537a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.398+0000 7fce0b818740 1 --2- 172.21.15.86:0/2166839349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490526990 0x560490537a30 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0036040 tx=0x7fcdf0005900 comp rx=0 tx=0).stop 2024-04-17T07:33:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.399+0000 7fce0b818740 1 -- 172.21.15.86:0/2166839349 shutdown_connections 2024-04-17T07:33:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.399+0000 7fce0b818740 1 --2- 172.21.15.86:0/2166839349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56049052bb00 0x560490524e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.399+0000 7fce0b818740 1 --2- 172.21.15.86:0/2166839349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560490526990 0x560490537a30 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.399+0000 7fce0b818740 1 --2- 172.21.15.86:0/2166839349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 0x5604905309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.399+0000 7fce0b818740 1 -- 172.21.15.86:0/2166839349 >> 172.21.15.86:0/2166839349 conn(0x560490521510 msgr2=0x560490524770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.399+0000 7fce0b818740 1 -- 172.21.15.86:0/2166839349 shutdown_connections 2024-04-17T07:33:24.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.399+0000 7fce0b818740 1 -- 172.21.15.86:0/2166839349 wait complete. 2024-04-17T07:33:24.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.399+0000 7fce0b818740 1 -- start start 2024-04-17T07:33:24.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.400+0000 7fce0b818740 1 Processor -- start 2024-04-17T07:33:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.400+0000 7fce0b818740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 0x560490534e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.401+0000 7fce0b818740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560490526990 0x5604905353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.401+0000 7fce02800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 0x560490534e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.401+0000 7fce02800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 0x560490534e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45156/0 (socket says 172.21.15.86:45156) 2024-04-17T07:33:24.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.401+0000 7fce02800640 1 -- 172.21.15.86:0/4008241121 learned_addr learned my addr 172.21.15.86:0/4008241121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:24.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.401+0000 7fce0b818740 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049052bb00 0x560490536380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.401+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce041bad70 con 0x56049052bb00 2024-04-17T07:33:24.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.401+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce041ba6a0 con 0x560490526990 2024-04-17T07:33:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.401+0000 7fce01400640 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560490526990 0x5604905353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.402+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce04073ec0 con 0x560490525fa0 2024-04-17T07:33:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.402+0000 7fce0b818740 10 client.0 _ll_register_callbacks cb 0x7fcdfc00f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.402+0000 7fce02800640 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049052bb00 0x560490536380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.402+0000 7fce01400640 1 -- 172.21.15.86:0/4008241121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 msgr2=0x560490534e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.402+0000 7fce01400640 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 0x560490534e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.402+0000 7fce01400640 1 -- 172.21.15.86:0/4008241121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049052bb00 msgr2=0x560490536380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.402+0000 7fce01400640 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049052bb00 0x560490536380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.402+0000 7fce01400640 1 -- 172.21.15.86:0/4008241121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604902687d0 con 0x560490526990 2024-04-17T07:33:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.402+0000 7fce02800640 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049052bb00 0x560490536380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:33:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.403+0000 7fce01400640 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560490526990 0x5604905353c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0004100 tx=0x7fcdf0009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8892ffc0335f8522 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.403+0000 7fcde3400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:33:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.403+0000 7fcde3400640 1 -- 172.21.15.86:0/4008241121 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf004e070 con 0x560490526990 2024-04-17T07:33:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.403+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf00054f0 con 0x560490526990 2024-04-17T07:33:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.403+0000 7fcde3400640 1 -- 172.21.15.86:0/4008241121 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcdf00035f0 con 0x560490526990 2024-04-17T07:33:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.403+0000 7fcde3400640 1 -- 172.21.15.86:0/4008241121 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcdf00037f0 con 0x560490526990 2024-04-17T07:33:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.404+0000 7fce0b818740 10 client.38747 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.404+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560490537a80 con 0x560490526990 2024-04-17T07:33:24.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.404+0000 7fcde3400640 1 -- 172.21.15.86:0/4008241121 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdf00054f0 con 0x560490526990 2024-04-17T07:33:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.404+0000 7fcde3400640 1 -- 172.21.15.86:0/4008241121 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fcdf0004e30 con 0x560490526990 2024-04-17T07:33:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.404+0000 7fce0b818740 10 client.38747 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.404+0000 7fce0b818740 10 client.38747 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:33:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.404+0000 7fce0b818740 20 client.38747 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.404+0000 7fce0b818740 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5604905a2770 0x5604905a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.404+0000 7fce0b818740 4 client.38747 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:33:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.404+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604904938b0 con 0x5604905a2770 2024-04-17T07:33:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.405+0000 7fce02800640 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5604905a2770 0x5604905a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.405+0000 7fce02800640 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5604905a2770 0x5604905a4b90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f91fc12643085a4e server_cookie=d9a43dd37520895f in_seq=0 out_seq=0 2024-04-17T07:33:24.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.405+0000 7fcde3400640 10 client.38747 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:33:24.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.406+0000 7fcde3400640 1 -- 172.21.15.86:0/4008241121 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604904938b0 con 0x5604905a2770 2024-04-17T07:33:24.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.406+0000 7fcde3400640 10 client.38747 handle_command_reply: tid=0 2024-04-17T07:33:24.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.406+0000 7fce0b818740 1 client.38747 shutdown 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.406+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560490526990 msgr2=0x5604905353c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.406+0000 7fce0b818740 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560490526990 0x5604905353c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0004100 tx=0x7fcdf0009610 comp rx=0 tx=0).stop 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.407+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 shutdown_connections 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.407+0000 7fce0b818740 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5604905a2770 0x5604905a4b90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.407+0000 7fce0b818740 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56049052bb00 0x560490536380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.407+0000 7fce0b818740 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560490526990 0x5604905353c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.407+0000 7fce0b818740 1 --2- 172.21.15.86:0/4008241121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560490525fa0 0x560490534e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.407+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 >> 172.21.15.86:0/4008241121 conn(0x560490521510 msgr2=0x560490525000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.407+0000 7fcde3400640 0 client.38747 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.407+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 shutdown_connections 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.407+0000 7fce0b818740 1 -- 172.21.15.86:0/4008241121 wait complete. 2024-04-17T07:33:24.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.407+0000 7fce0b818740 20 client.38747 trim_cache size 0 max 16384 2024-04-17T07:33:24.409 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:33:24.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.408+0000 7fce08c00640 1 -- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcde80a3c30 msgr2=0x7fcde80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.408+0000 7fce08c00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcde80a3c30 0x7fcde80a60f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4098aa0 tx=0x7fcdf4004730 comp rx=0 tx=0).stop 2024-04-17T07:33:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.408+0000 7fce08c00640 1 -- 172.21.15.86:0/2698611855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04076920 msgr2=0x7fce0419e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.408+0000 7fce08c00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04076920 0x7fce0419e790 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400d770 tx=0x7fcdf400dc30 comp rx=0 tx=0).stop 2024-04-17T07:33:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.409+0000 7fce08c00640 1 -- 172.21.15.86:0/2698611855 shutdown_connections 2024-04-17T07:33:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.409+0000 7fce08c00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcde80a3c30 0x7fcde80a60f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.409+0000 7fce08c00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce04078500 0x7fce041a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.409+0000 7fce08c00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce04077b60 0x7fce0419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.409+0000 7fce08c00640 1 --2- 172.21.15.86:0/2698611855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce04076920 0x7fce0419e790 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.409+0000 7fce08c00640 1 -- 172.21.15.86:0/2698611855 >> 172.21.15.86:0/2698611855 conn(0x7fce040ff750 msgr2=0x7fce04102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.409+0000 7fce08c00640 1 -- 172.21.15.86:0/2698611855 shutdown_connections 2024-04-17T07:33:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.409+0000 7fce08c00640 1 -- 172.21.15.86:0/2698611855 wait complete. 2024-04-17T07:33:24.439 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:33:24.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:24 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1447007683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:33:24.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:24 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1447007683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:33:24.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.751+0000 7f2c11800640 1 -- 172.21.15.86:0/4285063773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0c101fb0 msgr2=0x7f2c0c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.751+0000 7f2c11800640 1 --2- 172.21.15.86:0/4285063773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0c101fb0 0x7f2c0c1023d0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009900 tx=0x7f2bf802bc30 comp rx=0 tx=0).stop 2024-04-17T07:33:24.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.751+0000 7f2c11800640 1 -- 172.21.15.86:0/4285063773 shutdown_connections 2024-04-17T07:33:24.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.751+0000 7f2c11800640 1 --2- 172.21.15.86:0/4285063773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0c103b60 0x7f2c0c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.751+0000 7f2c11800640 1 --2- 172.21.15.86:0/4285063773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c0c1031a0 0x7f2c0c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.751+0000 7f2c11800640 1 --2- 172.21.15.86:0/4285063773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0c101fb0 0x7f2c0c1023d0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.751+0000 7f2c11800640 1 -- 172.21.15.86:0/4285063773 >> 172.21.15.86:0/4285063773 conn(0x7f2c0c0fd540 msgr2=0x7f2c0c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.751+0000 7f2c11800640 1 -- 172.21.15.86:0/4285063773 shutdown_connections 2024-04-17T07:33:24.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.752+0000 7f2c11800640 1 -- 172.21.15.86:0/4285063773 wait complete. 2024-04-17T07:33:24.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.753+0000 7f2c11800640 1 Processor -- start 2024-04-17T07:33:24.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.753+0000 7f2c11800640 1 -- start start 2024-04-17T07:33:24.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.754+0000 7f2c11800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0c101fb0 0x7f2c0c196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.754+0000 7f2c11800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0c1031a0 0x7f2c0c196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.754+0000 7f2c11800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c0c103b60 0x7f2c0c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.754+0000 7f2c11800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10b950 con 0x7f2c0c101fb0 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.754+0000 7f2c11800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10b7d0 con 0x7f2c0c103b60 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.754+0000 7f2c11800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10bad0 con 0x7f2c0c1031a0 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.754+0000 7f2c0a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0c1031a0 0x7f2c0c196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.754+0000 7f2c0a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0c1031a0 0x7f2c0c196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45172/0 (socket says 172.21.15.86:45172) 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.754+0000 7f2c0a000640 1 -- 172.21.15.86:0/1322285116 learned_addr learned my addr 172.21.15.86:0/1322285116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.754+0000 7f2c0b400640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c0c103b60 0x7f2c0c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.755+0000 7f2c0a000640 1 -- 172.21.15.86:0/1322285116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c0c103b60 msgr2=0x7f2c0c19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.755+0000 7f2c0a000640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c0c103b60 0x7f2c0c19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.755+0000 7f2c0a000640 1 -- 172.21.15.86:0/1322285116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0c101fb0 msgr2=0x7f2c0c196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.755+0000 7f2c0a000640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0c101fb0 0x7f2c0c196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.755+0000 7f2c0a000640 1 -- 172.21.15.86:0/1322285116 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c19dc30 con 0x7f2c0c1031a0 2024-04-17T07:33:24.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.755+0000 7f2c0a000640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0c1031a0 0x7f2c0c196550 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0058c0 tx=0x7f2bfc007590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.756+0000 7f2c03400640 1 -- 172.21.15.86:0/1322285116 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc013020 con 0x7f2c0c1031a0 2024-04-17T07:33:24.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.756+0000 7f2c03400640 1 -- 172.21.15.86:0/1322285116 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bfc013670 con 0x7f2c0c1031a0 2024-04-17T07:33:24.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.756+0000 7f2c03400640 1 -- 172.21.15.86:0/1322285116 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc00ee10 con 0x7f2c0c1031a0 2024-04-17T07:33:24.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.757+0000 7f2c11800640 1 -- 172.21.15.86:0/1322285116 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c19df20 con 0x7f2c0c1031a0 2024-04-17T07:33:24.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.757+0000 7f2c11800640 1 -- 172.21.15.86:0/1322285116 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c19e390 con 0x7f2c0c1031a0 2024-04-17T07:33:24.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.763+0000 7f2c03400640 1 -- 172.21.15.86:0/1322285116 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2bfc02c070 con 0x7f2c0c1031a0 2024-04-17T07:33:24.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.763+0000 7f2c145b9740 1 Processor -- start 2024-04-17T07:33:24.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.764+0000 7f2c145b9740 1 -- start start 2024-04-17T07:33:24.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.764+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4efa0 0x55bb23a4f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.764+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4f990 0x55bb23a54490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.764+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 0x55bb23a56f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.764+0000 7f2c145b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb239b5020 con 0x55bb23a4efa0 2024-04-17T07:33:24.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.764+0000 7f2c145b9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bb2395b460 con 0x55bb23a4f990 2024-04-17T07:33:24.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.764+0000 7f2c145b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb239b0fe0 con 0x55bb23a54b00 2024-04-17T07:33:24.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.765+0000 7f2c0aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4f990 0x55bb23a54490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.765+0000 7f2c0a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 0x55bb23a56f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.765+0000 7f2c0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4efa0 0x55bb23a4f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.765+0000 7f2c0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4efa0 0x55bb23a4f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43476/0 (socket says 172.21.15.86:43476) 2024-04-17T07:33:24.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.765+0000 7f2c0b400640 1 -- 172.21.15.86:0/3583058352 learned_addr learned my addr 172.21.15.86:0/3583058352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:24.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.766+0000 7f2c0b400640 1 -- 172.21.15.86:0/3583058352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 msgr2=0x55bb23a56f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.766+0000 7f2c0b400640 1 --2- 172.21.15.86:0/3583058352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 0x55bb23a56f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.766+0000 7f2c0b400640 1 -- 172.21.15.86:0/3583058352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4f990 msgr2=0x55bb23a54490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.766+0000 7f2c0b400640 1 --2- 172.21.15.86:0/3583058352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4f990 0x55bb23a54490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.766+0000 7f2c0b400640 1 -- 172.21.15.86:0/3583058352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb23963040 con 0x55bb23a4efa0 2024-04-17T07:33:24.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.766+0000 7f2c03400640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2bf00a3c70 0x7f2bf00a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.767+0000 7f2c03400640 1 -- 172.21.15.86:0/1322285116 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2bfc0e09d0 con 0x7f2c0c1031a0 2024-04-17T07:33:24.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.767+0000 7f2c0b400640 1 --2- 172.21.15.86:0/3583058352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4efa0 0x55bb23a4f3c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f2c04010370 tx=0x7f2c04010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e57d9a4f9c51bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.768+0000 7f2c0aa00640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2bf00a3c70 0x7f2bf00a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.768+0000 7f2bf7e00640 1 -- 172.21.15.86:0/3583058352 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04016070 con 0x55bb23a4efa0 2024-04-17T07:33:24.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.768+0000 7f2bf7e00640 1 -- 172.21.15.86:0/3583058352 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c04004d20 con 0x55bb23a4efa0 2024-04-17T07:33:24.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.768+0000 7f2bf7e00640 1 -- 172.21.15.86:0/3583058352 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040053c0 con 0x55bb23a4efa0 2024-04-17T07:33:24.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.769+0000 7f2c0aa00640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2bf00a3c70 0x7f2bf00a6130 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009230 tx=0x7f2bf8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.769+0000 7f2c145b9740 1 -- 172.21.15.86:0/3583058352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4efa0 msgr2=0x55bb23a4f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.769+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3583058352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4efa0 0x55bb23a4f3c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f2c04010370 tx=0x7f2c04010830 comp rx=0 tx=0).stop 2024-04-17T07:33:24.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.770+0000 7f2c145b9740 1 -- 172.21.15.86:0/3583058352 shutdown_connections 2024-04-17T07:33:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.770+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3583058352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 0x55bb23a56f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.770+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3583058352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4f990 0x55bb23a54490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.770+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3583058352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4efa0 0x55bb23a4f3c0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.770+0000 7f2c145b9740 1 -- 172.21.15.86:0/3583058352 >> 172.21.15.86:0/3583058352 conn(0x55bb23a4a510 msgr2=0x55bb23a4a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.770+0000 7f2c145b9740 1 -- 172.21.15.86:0/3583058352 shutdown_connections 2024-04-17T07:33:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.771+0000 7f2c145b9740 1 -- 172.21.15.86:0/3583058352 wait complete. 2024-04-17T07:33:24.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.771+0000 7f2c145b9740 1 -- start start 2024-04-17T07:33:24.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.772+0000 7f2c145b9740 1 Processor -- start 2024-04-17T07:33:24.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.772+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 0x55bb23a59570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.773+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 0x55bb23a59ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.773+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 0x55bb23a60af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.773+0000 7f2c145b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bb23a57ab0 con 0x55bb23a4f990 2024-04-17T07:33:24.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.773+0000 7f2c0aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 0x55bb23a59570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.773+0000 7f2c0aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 0x55bb23a59570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54790/0 (socket says 172.21.15.86:54790) 2024-04-17T07:33:24.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.773+0000 7f2c0aa00640 1 -- 172.21.15.86:0/3187173874 learned_addr learned my addr 172.21.15.86:0/3187173874 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:24.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.773+0000 7f2c145b9740 1 -- 172.21.15.86:0/3187173874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bb2396e830 con 0x55bb23a4efa0 2024-04-17T07:33:24.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.773+0000 7f2c145b9740 1 -- 172.21.15.86:0/3187173874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb23a57c30 con 0x55bb23a54b00 2024-04-17T07:33:24.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.773+0000 7f2c145b9740 10 client.0 _ll_register_callbacks cb 0x55bb2393cfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:24.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.773+0000 7f2c0b400640 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 0x55bb23a60af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.774+0000 7f2c0b400640 1 -- 172.21.15.86:0/3187173874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 msgr2=0x55bb23a59570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.774+0000 7f2c0b400640 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 0x55bb23a59570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.774+0000 7f2c0b400640 1 -- 172.21.15.86:0/3187173874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 msgr2=0x55bb23a59ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.774+0000 7f2c0b400640 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 0x55bb23a59ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.774+0000 7f2c0b400640 1 -- 172.21.15.86:0/3187173874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb23972920 con 0x55bb23a54b00 2024-04-17T07:33:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.775+0000 7f2c0b400640 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 0x55bb23a60af0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2c04006010 tx=0x7f2c04003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5383f3f729fd46a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.775+0000 7f2bf4c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:33:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.775+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3187173874 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04029060 con 0x55bb23a54b00 2024-04-17T07:33:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.775+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3187173874 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2c04004040 con 0x55bb23a54b00 2024-04-17T07:33:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.775+0000 7f2c145b9740 1 -- 172.21.15.86:0/3187173874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c04004d20 con 0x55bb23a54b00 2024-04-17T07:33:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.776+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3187173874 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2c04030050 con 0x55bb23a54b00 2024-04-17T07:33:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.776+0000 7f2c145b9740 10 client.28765 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.776+0000 7f2c145b9740 1 -- 172.21.15.86:0/3187173874 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb239407f0 con 0x55bb23a54b00 2024-04-17T07:33:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.777+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3187173874 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c04004d20 con 0x55bb23a54b00 2024-04-17T07:33:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.777+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3187173874 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2c0400f040 con 0x55bb23a54b00 2024-04-17T07:33:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.777+0000 7f2c145b9740 10 client.28765 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.777+0000 7f2c145b9740 10 client.28765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:33:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.777+0000 7f2c145b9740 20 client.28765 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.777+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55bb23a64ce0 0x55bb23a670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.777+0000 7f2c145b9740 4 client.28765 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:24.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.777+0000 7f2c145b9740 1 -- 172.21.15.86:0/3187173874 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb239c2a40 con 0x55bb23a64ce0 2024-04-17T07:33:24.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.778+0000 7f2c0aa00640 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55bb23a64ce0 0x55bb23a670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.779+0000 7f2c0aa00640 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55bb23a64ce0 0x55bb23a670d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73420e971e913b1e server_cookie=6886f8a8c843cdab in_seq=0 out_seq=0 2024-04-17T07:33:24.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.779+0000 7f2bf4c00640 10 client.28765 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:33:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.780+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3187173874 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb239c2a40 con 0x55bb23a64ce0 2024-04-17T07:33:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.780+0000 7f2bf4c00640 10 client.28765 handle_command_reply: tid=0 2024-04-17T07:33:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.780+0000 7f2c145b9740 1 client.28765 shutdown 2024-04-17T07:33:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.781+0000 7f2c145b9740 1 -- 172.21.15.86:0/3187173874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 msgr2=0x55bb23a60af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.781+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 0x55bb23a60af0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2c04006010 tx=0x7f2c04003970 comp rx=0 tx=0).stop 2024-04-17T07:33:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.782+0000 7f2c145b9740 1 -- 172.21.15.86:0/3187173874 shutdown_connections 2024-04-17T07:33:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.782+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55bb23a64ce0 0x55bb23a670d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.782+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a54b00 0x55bb23a60af0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.782+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 0x55bb23a59ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.782+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3187173874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 0x55bb23a59570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.782+0000 7f2c145b9740 1 -- 172.21.15.86:0/3187173874 >> 172.21.15.86:0/3187173874 conn(0x55bb23a4a510 msgr2=0x55bb23a49710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.782+0000 7f2bf4c00640 0 client.28765 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:33:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.783+0000 7f2c145b9740 1 -- 172.21.15.86:0/3187173874 shutdown_connections 2024-04-17T07:33:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.783+0000 7f2c145b9740 1 -- 172.21.15.86:0/3187173874 wait complete. 2024-04-17T07:33:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.783+0000 7f2c145b9740 20 client.28765 trim_cache size 0 max 16384 2024-04-17T07:33:24.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.809+0000 7f2c145b9740 1 Processor -- start 2024-04-17T07:33:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.810+0000 7f2c145b9740 1 -- start start 2024-04-17T07:33:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.810+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4efa0 0x55bb23a58b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.810+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 0x55bb23a60ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.810+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a54b00 0x55bb23a48e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.810+0000 7f2c145b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c1a8a40 con 0x55bb23a4f990 2024-04-17T07:33:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.810+0000 7f2c145b9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c1aa580 con 0x55bb23a54b00 2024-04-17T07:33:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.810+0000 7f2c145b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c1a9110 con 0x55bb23a4efa0 2024-04-17T07:33:24.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.811+0000 7f2c0aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4efa0 0x55bb23a58b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.811+0000 7f2c0aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4efa0 0x55bb23a58b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45180/0 (socket says 172.21.15.86:45180) 2024-04-17T07:33:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.811+0000 7f2c0a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 0x55bb23a60ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.811+0000 7f2c0a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 0x55bb23a60ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43496/0 (socket says 172.21.15.86:43496) 2024-04-17T07:33:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.811+0000 7f2c0a000640 1 -- 172.21.15.86:0/953808286 learned_addr learned my addr 172.21.15.86:0/953808286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.811+0000 7f2c0b400640 1 --2- 172.21.15.86:0/953808286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a54b00 0x55bb23a48e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.812+0000 7f2c0a000640 1 -- 172.21.15.86:0/953808286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4efa0 msgr2=0x55bb23a58b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.812+0000 7f2c0a000640 1 --2- 172.21.15.86:0/953808286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4efa0 0x55bb23a58b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.812+0000 7f2c0a000640 1 -- 172.21.15.86:0/953808286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a54b00 msgr2=0x55bb23a48e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.812+0000 7f2c0a000640 1 --2- 172.21.15.86:0/953808286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a54b00 0x55bb23a48e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.812+0000 7f2c0a000640 1 -- 172.21.15.86:0/953808286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb23a59360 con 0x55bb23a4f990 2024-04-17T07:33:24.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.812+0000 7f2c0a000640 1 --2- 172.21.15.86:0/953808286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 0x55bb23a60ab0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0ac630 tx=0x7f2bfc0e4060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2596f3e51c71590d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.813+0000 7f2bf7400640 1 -- 172.21.15.86:0/953808286 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0ac290 con 0x55bb23a4f990 2024-04-17T07:33:24.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.813+0000 7f2bf7400640 1 -- 172.21.15.86:0/953808286 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bfc0ad030 con 0x55bb23a4f990 2024-04-17T07:33:24.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.813+0000 7f2bf7400640 1 -- 172.21.15.86:0/953808286 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0aacc0 con 0x55bb23a4f990 2024-04-17T07:33:24.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.814+0000 7f2c145b9740 1 -- 172.21.15.86:0/953808286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 msgr2=0x55bb23a60ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.814+0000 7f2c145b9740 1 --2- 172.21.15.86:0/953808286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 0x55bb23a60ab0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0ac630 tx=0x7f2bfc0e4060 comp rx=0 tx=0).stop 2024-04-17T07:33:24.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.814+0000 7f2c145b9740 1 -- 172.21.15.86:0/953808286 shutdown_connections 2024-04-17T07:33:24.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.814+0000 7f2c145b9740 1 --2- 172.21.15.86:0/953808286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a54b00 0x55bb23a48e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.814+0000 7f2c145b9740 1 --2- 172.21.15.86:0/953808286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a4f990 0x55bb23a60ab0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.814+0000 7f2c145b9740 1 --2- 172.21.15.86:0/953808286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4efa0 0x55bb23a58b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.814+0000 7f2c145b9740 1 -- 172.21.15.86:0/953808286 >> 172.21.15.86:0/953808286 conn(0x55bb23a4a510 msgr2=0x55bb23a59570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.815+0000 7f2c145b9740 1 -- 172.21.15.86:0/953808286 shutdown_connections 2024-04-17T07:33:24.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.815+0000 7f2c145b9740 1 -- 172.21.15.86:0/953808286 wait complete. 2024-04-17T07:33:24.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.815+0000 7f2c145b9740 1 -- start start 2024-04-17T07:33:24.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.815+0000 7f2c145b9740 1 Processor -- start 2024-04-17T07:33:24.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.816+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 0x55bb23a5e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.816+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4f990 0x55bb23a5e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.816+0000 7f2c145b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a54b00 0x55bb23a5f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.816+0000 7f2c145b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c1a9eb0 con 0x55bb23a54b00 2024-04-17T07:33:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.816+0000 7f2c145b9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c1a97e0 con 0x55bb23a4efa0 2024-04-17T07:33:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.816+0000 7f2c145b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c1a6a10 con 0x55bb23a4f990 2024-04-17T07:33:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.816+0000 7f2c145b9740 10 client.0 _ll_register_callbacks cb 0x7f2c04009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.817+0000 7f2c0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a54b00 0x55bb23a5f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.817+0000 7f2c0b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a54b00 0x55bb23a5f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43508/0 (socket says 172.21.15.86:43508) 2024-04-17T07:33:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.817+0000 7f2c0b400640 1 -- 172.21.15.86:0/3649152775 learned_addr learned my addr 172.21.15.86:0/3649152775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.817+0000 7f2c0a000640 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4f990 0x55bb23a5e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.817+0000 7f2c0aa00640 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 0x55bb23a5e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.818+0000 7f2c0b400640 1 -- 172.21.15.86:0/3649152775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4f990 msgr2=0x55bb23a5e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.818+0000 7f2c0b400640 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4f990 0x55bb23a5e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.818+0000 7f2c0b400640 1 -- 172.21.15.86:0/3649152775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 msgr2=0x55bb23a5e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:24.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.818+0000 7f2c0b400640 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 0x55bb23a5e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.818+0000 7f2c0b400640 1 -- 172.21.15.86:0/3649152775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb236c73f0 con 0x55bb23a54b00 2024-04-17T07:33:24.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.818+0000 7f2c0b400640 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a54b00 0x55bb23a5f4e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2c04010870 tx=0x7f2c04003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ceeea901f181e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:24.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.818+0000 7f2bf4c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:33:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.819+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3649152775 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04029070 con 0x55bb23a54b00 2024-04-17T07:33:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.819+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3649152775 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2c04005150 con 0x55bb23a54b00 2024-04-17T07:33:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.819+0000 7f2c145b9740 1 -- 172.21.15.86:0/3649152775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bfc0ad030 con 0x55bb23a54b00 2024-04-17T07:33:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3649152775 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2c04040050 con 0x55bb23a54b00 2024-04-17T07:33:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2c145b9740 10 client.19713 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:24.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2c145b9740 1 -- 172.21.15.86:0/3649152775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb237917d0 con 0x55bb23a54b00 2024-04-17T07:33:24.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3649152775 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bfc0ad030 con 0x55bb23a54b00 2024-04-17T07:33:24.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3649152775 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2c04024460 con 0x55bb23a54b00 2024-04-17T07:33:24.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2c145b9740 10 client.19713 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:24.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2c145b9740 10 client.19713 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:33:24.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2c145b9740 20 client.19713 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:24.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55bb23acb530 0x55bb23acd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:24.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2c145b9740 4 client.19713 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:33:24.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.820+0000 7f2c145b9740 1 -- 172.21.15.86:0/3649152775 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb239bc8b0 con 0x55bb23acb530 2024-04-17T07:33:24.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.821+0000 7f2c0aa00640 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55bb23acb530 0x55bb23acd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:24.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.823+0000 7f2bf4c00640 10 client.19713 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:33:24.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.823+0000 7f2c0aa00640 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55bb23acb530 0x55bb23acd950 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85f7dc6e0f7456c2 server_cookie=7ab3e58e867a0ccc in_seq=0 out_seq=0 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.823+0000 7f2bf4c00640 1 -- 172.21.15.86:0/3649152775 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb239bc8b0 con 0x55bb23acb530 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.823+0000 7f2bf4c00640 10 client.19713 handle_command_reply: tid=0 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.823+0000 7f2c145b9740 1 client.19713 shutdown 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.823+0000 7f2c145b9740 1 -- 172.21.15.86:0/3649152775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a54b00 msgr2=0x55bb23a5f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.823+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a54b00 0x55bb23a5f4e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2c04010870 tx=0x7f2c04003af0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.824+0000 7f2c145b9740 1 -- 172.21.15.86:0/3649152775 shutdown_connections 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.824+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x55bb23acb530 0x55bb23acd950 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.824+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bb23a54b00 0x55bb23a5f4e0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.824+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb23a4f990 0x55bb23a5e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.824+0000 7f2c145b9740 1 --2- 172.21.15.86:0/3649152775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bb23a4efa0 0x55bb23a5e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.824+0000 7f2c145b9740 1 -- 172.21.15.86:0/3649152775 >> 172.21.15.86:0/3649152775 conn(0x55bb23a4a510 msgr2=0x55bb23a59570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.824+0000 7f2bf4c00640 0 client.19713 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:33:24.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.824+0000 7f2c145b9740 1 -- 172.21.15.86:0/3649152775 shutdown_connections 2024-04-17T07:33:24.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.824+0000 7f2c145b9740 1 -- 172.21.15.86:0/3649152775 wait complete. 2024-04-17T07:33:24.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.824+0000 7f2c145b9740 20 client.19713 trim_cache size 0 max 16384 2024-04-17T07:33:24.826 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:33:24.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.825+0000 7f2c11800640 1 -- 172.21.15.86:0/1322285116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2bf00a3c70 msgr2=0x7f2bf00a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.825+0000 7f2c11800640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2bf00a3c70 0x7f2bf00a6130 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009230 tx=0x7f2bf8000bc0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.825+0000 7f2c11800640 1 -- 172.21.15.86:0/1322285116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0c1031a0 msgr2=0x7f2c0c196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.825+0000 7f2c11800640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0c1031a0 0x7f2c0c196550 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0058c0 tx=0x7f2bfc007590 comp rx=0 tx=0).stop 2024-04-17T07:33:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.825+0000 7f2c11800640 1 -- 172.21.15.86:0/1322285116 shutdown_connections 2024-04-17T07:33:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.825+0000 7f2c11800640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2bf00a3c70 0x7f2bf00a6130 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.825+0000 7f2c11800640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c0c103b60 0x7f2c0c19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.825+0000 7f2c11800640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c0c1031a0 0x7f2c0c196550 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.825+0000 7f2c11800640 1 --2- 172.21.15.86:0/1322285116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0c101fb0 0x7f2c0c196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.825+0000 7f2c11800640 1 -- 172.21.15.86:0/1322285116 >> 172.21.15.86:0/1322285116 conn(0x7f2c0c0fd540 msgr2=0x7f2c0c106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.826+0000 7f2c11800640 1 -- 172.21.15.86:0/1322285116 shutdown_connections 2024-04-17T07:33:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:24.826+0000 7f2c11800640 1 -- 172.21.15.86:0/1322285116 wait complete. 2024-04-17T07:33:24.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:33:25.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.177+0000 7f9018e00640 1 -- 172.21.15.86:0/2923512801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90140749c0 msgr2=0x7f9014074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.177+0000 7f9018e00640 1 --2- 172.21.15.86:0/2923512801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90140749c0 0x7f9014074e40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9004009930 tx=0x7f900402bc40 comp rx=0 tx=0).stop 2024-04-17T07:33:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.178+0000 7f9018e00640 1 -- 172.21.15.86:0/2923512801 shutdown_connections 2024-04-17T07:33:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.178+0000 7f9018e00640 1 --2- 172.21.15.86:0/2923512801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f901410e460 0x7f9014110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.178+0000 7f9018e00640 1 --2- 172.21.15.86:0/2923512801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90140749c0 0x7f9014074e40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.178+0000 7f9018e00640 1 --2- 172.21.15.86:0/2923512801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9014075fa0 0x7f90140743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.178+0000 7f9018e00640 1 -- 172.21.15.86:0/2923512801 >> 172.21.15.86:0/2923512801 conn(0x7f90140fd400 msgr2=0x7f90140ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.180+0000 7f9018e00640 1 -- 172.21.15.86:0/2923512801 shutdown_connections 2024-04-17T07:33:25.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.180+0000 7f9018e00640 1 -- 172.21.15.86:0/2923512801 wait complete. 2024-04-17T07:33:25.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.182+0000 7f9018e00640 1 Processor -- start 2024-04-17T07:33:25.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.182+0000 7f9018e00640 1 -- start start 2024-04-17T07:33:25.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.182+0000 7f9018e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90140749c0 0x7f9014070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.182+0000 7f9018e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9014075fa0 0x7f9014070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.182+0000 7f9018e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f901410e460 0x7f90141ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.182+0000 7f9018e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9014114020 con 0x7f9014075fa0 2024-04-17T07:33:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.182+0000 7f9018e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9014113ea0 con 0x7f901410e460 2024-04-17T07:33:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.182+0000 7f9018e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90141141a0 con 0x7f90140749c0 2024-04-17T07:33:25.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.183+0000 7f9012a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9014075fa0 0x7f9014070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.183+0000 7f9013e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f901410e460 0x7f90141ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.183+0000 7f9013400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90140749c0 0x7f9014070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.183+0000 7f9013e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f901410e460 0x7f90141ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54830/0 (socket says 172.21.15.86:54830) 2024-04-17T07:33:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.183+0000 7f9013e00640 1 -- 172.21.15.86:0/3595087541 learned_addr learned my addr 172.21.15.86:0/3595087541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.184+0000 7f9013e00640 1 -- 172.21.15.86:0/3595087541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90140749c0 msgr2=0x7f9014070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.184+0000 7f9013e00640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90140749c0 0x7f9014070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.184+0000 7f9013e00640 1 -- 172.21.15.86:0/3595087541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9014075fa0 msgr2=0x7f9014070e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.184+0000 7f9013e00640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9014075fa0 0x7f9014070e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.184+0000 7f9013e00640 1 -- 172.21.15.86:0/3595087541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90141aec30 con 0x7f901410e460 2024-04-17T07:33:25.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.184+0000 7f9013e00640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f901410e460 0x7f90141ae6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f900c00eaa0 tx=0x7f900c00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.185+0000 7f900be00640 1 -- 172.21.15.86:0/3595087541 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c00cdb0 con 0x7f901410e460 2024-04-17T07:33:25.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.185+0000 7f900be00640 1 -- 172.21.15.86:0/3595087541 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f900c005020 con 0x7f901410e460 2024-04-17T07:33:25.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.185+0000 7f900be00640 1 -- 172.21.15.86:0/3595087541 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c0116d0 con 0x7f901410e460 2024-04-17T07:33:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.185+0000 7f9018e00640 1 -- 172.21.15.86:0/3595087541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90141aee00 con 0x7f901410e460 2024-04-17T07:33:25.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.187+0000 7f9018e00640 1 -- 172.21.15.86:0/3595087541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90141af210 con 0x7f901410e460 2024-04-17T07:33:25.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.192+0000 7f901cff5740 1 Processor -- start 2024-04-17T07:33:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.193+0000 7f900be00640 1 -- 172.21.15.86:0/3595087541 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f900c004b20 con 0x7f901410e460 2024-04-17T07:33:25.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.193+0000 7f901cff5740 1 -- start start 2024-04-17T07:33:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.194+0000 7f901cff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eea93580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.194+0000 7f901cff5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea93b50 0x5588eea98650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.194+0000 7f901cff5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea98cc0 0x5588eea9b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.194+0000 7f901cff5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5588ee9f9020 con 0x5588eea93b50 2024-04-17T07:33:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.194+0000 7f901cff5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5588ee99f460 con 0x5588eea98cc0 2024-04-17T07:33:25.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.194+0000 7f901cff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5588ee9f4fe0 con 0x5588eea93160 2024-04-17T07:33:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.194+0000 7f9012a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eea93580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.194+0000 7f9012a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eea93580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45230/0 (socket says 172.21.15.86:45230) 2024-04-17T07:33:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.194+0000 7f9012a00640 1 -- 172.21.15.86:0/281485348 learned_addr learned my addr 172.21.15.86:0/281485348 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.194+0000 7f9012a00640 1 --2- 172.21.15.86:0/281485348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea98cc0 0x5588eea9b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.195+0000 7f9012a00640 1 -- 172.21.15.86:0/281485348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea98cc0 msgr2=0x5588eea9b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.195+0000 7f9012a00640 1 --2- 172.21.15.86:0/281485348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea98cc0 0x5588eea9b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.195+0000 7f9012a00640 1 -- 172.21.15.86:0/281485348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea93b50 msgr2=0x5588eea98650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.195+0000 7f9012a00640 1 --2- 172.21.15.86:0/281485348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea93b50 0x5588eea98650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.195+0000 7f9012a00640 1 -- 172.21.15.86:0/281485348 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ee9a7040 con 0x5588eea93160 2024-04-17T07:33:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.195+0000 7f9012a00640 1 --2- 172.21.15.86:0/281485348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eea93580 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f900402c250 tx=0x7f9004000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac6759d238d92abc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.196+0000 7f9008c00640 1 -- 172.21.15.86:0/281485348 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900404a070 con 0x5588eea93160 2024-04-17T07:33:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.196+0000 7f900be00640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ff80a3c80 0x7f8ff80a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.196+0000 7f9008c00640 1 -- 172.21.15.86:0/281485348 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9004005500 con 0x5588eea93160 2024-04-17T07:33:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.196+0000 7f9008c00640 1 -- 172.21.15.86:0/281485348 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9004004ae0 con 0x5588eea93160 2024-04-17T07:33:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.196+0000 7f900be00640 1 -- 172.21.15.86:0/3595087541 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f900c0d3170 con 0x7f901410e460 2024-04-17T07:33:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.197+0000 7f901cff5740 1 -- 172.21.15.86:0/281485348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 msgr2=0x5588eea93580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.197+0000 7f901cff5740 1 --2- 172.21.15.86:0/281485348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eea93580 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f900402c250 tx=0x7f9004000bc0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.197+0000 7f9013400640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ff80a3c80 0x7f8ff80a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.199+0000 7f901cff5740 1 -- 172.21.15.86:0/281485348 shutdown_connections 2024-04-17T07:33:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.199+0000 7f901cff5740 1 --2- 172.21.15.86:0/281485348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea98cc0 0x5588eea9b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.199+0000 7f901cff5740 1 --2- 172.21.15.86:0/281485348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea93b50 0x5588eea98650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.199+0000 7f901cff5740 1 --2- 172.21.15.86:0/281485348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eea93580 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.199+0000 7f901cff5740 1 -- 172.21.15.86:0/281485348 >> 172.21.15.86:0/281485348 conn(0x5588eea8e6d0 msgr2=0x5588eea8ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.200+0000 7f9013400640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ff80a3c80 0x7f8ff80a6140 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9000005d90 tx=0x7f9000005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.200+0000 7f901cff5740 1 -- 172.21.15.86:0/281485348 shutdown_connections 2024-04-17T07:33:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.200+0000 7f901cff5740 1 -- 172.21.15.86:0/281485348 wait complete. 2024-04-17T07:33:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.201+0000 7f901cff5740 1 -- start start 2024-04-17T07:33:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.202+0000 7f901cff5740 1 Processor -- start 2024-04-17T07:33:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.202+0000 7f901cff5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 0x5588eea9d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.202+0000 7f901cff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 0x5588eea9dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.202+0000 7f901cff5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eeaa4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f901cff5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5588eea9bc10 con 0x5588eea98cc0 2024-04-17T07:33:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f901cff5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5588ee9b2830 con 0x5588eea93160 2024-04-17T07:33:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f901cff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5588eea9bd90 con 0x5588eea93b50 2024-04-17T07:33:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f9013400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 0x5588eea9d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f9013400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 0x5588eea9d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54842/0 (socket says 172.21.15.86:54842) 2024-04-17T07:33:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f9013400640 1 -- 172.21.15.86:0/1100945792 learned_addr learned my addr 172.21.15.86:0/1100945792 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f901cff5740 10 client.0 _ll_register_callbacks cb 0x5588ee980fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:25.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f9012a00640 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 0x5588eea9dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f9013400640 1 -- 172.21.15.86:0/1100945792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 msgr2=0x5588eea9dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f9013400640 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 0x5588eea9dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f9013400640 1 -- 172.21.15.86:0/1100945792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 msgr2=0x5588eeaa4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f9013400640 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eeaa4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.203+0000 7f9013400640 1 -- 172.21.15.86:0/1100945792 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ee9b6920 con 0x5588eea93160 2024-04-17T07:33:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.204+0000 7f9013400640 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 0x5588eea9d6d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9000004700 tx=0x7f9000004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517d49a7aae95b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.204+0000 7f8ffd600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:33:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.205+0000 7f8ffd600640 1 -- 172.21.15.86:0/1100945792 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000021070 con 0x5588eea93160 2024-04-17T07:33:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.205+0000 7f8ffd600640 1 -- 172.21.15.86:0/1100945792 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f900001d870 con 0x5588eea93160 2024-04-17T07:33:25.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.205+0000 7f901cff5740 1 -- 172.21.15.86:0/1100945792 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9004005500 con 0x5588eea93160 2024-04-17T07:33:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.207+0000 7f8ffd600640 1 -- 172.21.15.86:0/1100945792 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f900002d050 con 0x5588eea93160 2024-04-17T07:33:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.207+0000 7f901cff5740 10 client.38774 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.207+0000 7f901cff5740 1 -- 172.21.15.86:0/1100945792 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588ee9847f0 con 0x5588eea93160 2024-04-17T07:33:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.208+0000 7f8ffd600640 1 -- 172.21.15.86:0/1100945792 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9004005500 con 0x5588eea93160 2024-04-17T07:33:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.208+0000 7f8ffd600640 1 -- 172.21.15.86:0/1100945792 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f900001cad0 con 0x5588eea93160 2024-04-17T07:33:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.208+0000 7f901cff5740 10 client.38774 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.208+0000 7f901cff5740 10 client.38774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:33:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.208+0000 7f901cff5740 20 client.38774 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.208+0000 7f901cff5740 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5588eeaa91d0 0x5588eeaab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.208+0000 7f901cff5740 4 client.38774 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.208+0000 7f901cff5740 1 -- 172.21.15.86:0/1100945792 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588eea06a40 con 0x5588eeaa91d0 2024-04-17T07:33:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.209+0000 7f9012a00640 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5588eeaa91d0 0x5588eeaab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.209+0000 7f9012a00640 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5588eeaa91d0 0x5588eeaab690 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e35d658a83f8f4f server_cookie=6ae417921e003c17 in_seq=0 out_seq=0 2024-04-17T07:33:25.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.209+0000 7f8ffd600640 10 client.38774 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:33:25.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.210+0000 7f8ffd600640 1 -- 172.21.15.86:0/1100945792 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588eea06a40 con 0x5588eeaa91d0 2024-04-17T07:33:25.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.210+0000 7f8ffd600640 10 client.38774 handle_command_reply: tid=0 2024-04-17T07:33:25.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.210+0000 7f901cff5740 1 client.38774 shutdown 2024-04-17T07:33:25.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f901cff5740 1 -- 172.21.15.86:0/1100945792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 msgr2=0x5588eea9d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f901cff5740 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 0x5588eea9d6d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9000004700 tx=0x7f9000004730 comp rx=0 tx=0).stop 2024-04-17T07:33:25.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f901cff5740 1 -- 172.21.15.86:0/1100945792 shutdown_connections 2024-04-17T07:33:25.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f901cff5740 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eeaa4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f901cff5740 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 0x5588eea9dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f901cff5740 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5588eeaa91d0 0x5588eeaab690 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f901cff5740 1 --2- 172.21.15.86:0/1100945792 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 0x5588eea9d6d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f901cff5740 1 -- 172.21.15.86:0/1100945792 >> 172.21.15.86:0/1100945792 conn(0x5588eea8e6d0 msgr2=0x5588eea8d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f8ffd600640 0 client.38774 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:33:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f901cff5740 1 -- 172.21.15.86:0/1100945792 shutdown_connections 2024-04-17T07:33:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.211+0000 7f901cff5740 1 -- 172.21.15.86:0/1100945792 wait complete. 2024-04-17T07:33:25.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.212+0000 7f901cff5740 20 client.38774 trim_cache size 0 max 16384 2024-04-17T07:33:25.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.238+0000 7f901cff5740 1 Processor -- start 2024-04-17T07:33:25.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.238+0000 7f901cff5740 1 -- start start 2024-04-17T07:33:25.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.239+0000 7f901cff5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 0x5588eea9d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.239+0000 7f901cff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 0x5588eeaa4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.239+0000 7f901cff5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eea9d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.239+0000 7f901cff5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90141b9860 con 0x5588eea98cc0 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.239+0000 7f901cff5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90141bb3a0 con 0x5588eea93160 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.239+0000 7f901cff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90141b9f30 con 0x5588eea93b50 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.239+0000 7f9012a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 0x5588eeaa4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.239+0000 7f9012a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 0x5588eeaa4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45238/0 (socket says 172.21.15.86:45238) 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.239+0000 7f9012a00640 1 -- 172.21.15.86:0/1227861345 learned_addr learned my addr 172.21.15.86:0/1227861345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.239+0000 7f9013e00640 1 --2- 172.21.15.86:0/1227861345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eea9d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.240+0000 7f9013400640 1 --2- 172.21.15.86:0/1227861345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 0x5588eea9d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.240+0000 7f9012a00640 1 -- 172.21.15.86:0/1227861345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 msgr2=0x5588eea9d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.240+0000 7f9012a00640 1 --2- 172.21.15.86:0/1227861345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 0x5588eea9d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.240+0000 7f9012a00640 1 -- 172.21.15.86:0/1227861345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 msgr2=0x5588eea9d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.240+0000 7f9012a00640 1 --2- 172.21.15.86:0/1227861345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eea9d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.240+0000 7f9012a00640 1 -- 172.21.15.86:0/1227861345 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ee70b3f0 con 0x5588eea93b50 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.240+0000 7f9013e00640 1 --2- 172.21.15.86:0/1227861345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eea9d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.240+0000 7f9012a00640 1 --2- 172.21.15.86:0/1227861345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 0x5588eeaa4c10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9004048040 tx=0x7f9004005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7c9c935e24d9552 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.241+0000 7f8fffe00640 1 -- 172.21.15.86:0/1227861345 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900407d070 con 0x5588eea93b50 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.241+0000 7f8fffe00640 1 -- 172.21.15.86:0/1227861345 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9004004470 con 0x5588eea93b50 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.242+0000 7f8fffe00640 1 -- 172.21.15.86:0/1227861345 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900404a040 con 0x5588eea93b50 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.242+0000 7f901cff5740 1 -- 172.21.15.86:0/1227861345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 msgr2=0x5588eeaa4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.242+0000 7f901cff5740 1 --2- 172.21.15.86:0/1227861345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 0x5588eeaa4c10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9004048040 tx=0x7f9004005fd0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.242+0000 7f901cff5740 1 -- 172.21.15.86:0/1227861345 shutdown_connections 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.242+0000 7f901cff5740 1 --2- 172.21.15.86:0/1227861345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eea9d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.242+0000 7f901cff5740 1 --2- 172.21.15.86:0/1227861345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93b50 0x5588eeaa4c10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.242+0000 7f901cff5740 1 --2- 172.21.15.86:0/1227861345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93160 0x5588eea9d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.242+0000 7f901cff5740 1 -- 172.21.15.86:0/1227861345 >> 172.21.15.86:0/1227861345 conn(0x5588eea8e6d0 msgr2=0x5588eea950b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.242+0000 7f901cff5740 1 -- 172.21.15.86:0/1227861345 shutdown_connections 2024-04-17T07:33:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.242+0000 7f901cff5740 1 -- 172.21.15.86:0/1227861345 wait complete. 2024-04-17T07:33:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.243+0000 7f901cff5740 1 -- start start 2024-04-17T07:33:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.243+0000 7f901cff5740 1 Processor -- start 2024-04-17T07:33:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.243+0000 7f901cff5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eeaa1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.243+0000 7f901cff5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93b50 0x5588eeaa2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.243+0000 7f901cff5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eeaa3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.244+0000 7f901cff5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90141bacd0 con 0x5588eea98cc0 2024-04-17T07:33:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.244+0000 7f901cff5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90141ba600 con 0x5588eea93b50 2024-04-17T07:33:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.244+0000 7f901cff5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90141b7830 con 0x5588eea93160 2024-04-17T07:33:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.244+0000 7f901cff5740 10 client.0 _ll_register_callbacks cb 0x7f900000ed90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.244+0000 7f9013e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eeaa3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.244+0000 7f9013400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eeaa1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.244+0000 7f9013400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eeaa1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:45246/0 (socket says 172.21.15.86:45246) 2024-04-17T07:33:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.244+0000 7f9013400640 1 -- 172.21.15.86:0/4092964865 learned_addr learned my addr 172.21.15.86:0/4092964865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.245+0000 7f9013400640 1 -- 172.21.15.86:0/4092964865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93b50 msgr2=0x5588eeaa2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.245+0000 7f9012a00640 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93b50 0x5588eeaa2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.245+0000 7f9013400640 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93b50 0x5588eeaa2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.245+0000 7f9013400640 1 -- 172.21.15.86:0/4092964865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 msgr2=0x5588eeaa3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.245+0000 7f9013400640 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eeaa3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.245+0000 7f9013400640 1 -- 172.21.15.86:0/4092964865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ee7d57d0 con 0x5588eea93160 2024-04-17T07:33:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.245+0000 7f9013e00640 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eeaa3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:33:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.245+0000 7f9013400640 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eeaa1dc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9000000c00 tx=0x7f900000f980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ced789a715c1a3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.245+0000 7f8ffd600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:33:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.247+0000 7f8ffd600640 1 -- 172.21.15.86:0/4092964865 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000035070 con 0x5588eea93160 2024-04-17T07:33:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.247+0000 7f8ffd600640 1 -- 172.21.15.86:0/4092964865 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f900001d870 con 0x5588eea93160 2024-04-17T07:33:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.247+0000 7f901cff5740 1 -- 172.21.15.86:0/4092964865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9004004470 con 0x5588eea93160 2024-04-17T07:33:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.247+0000 7f8ffd600640 1 -- 172.21.15.86:0/4092964865 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9000043050 con 0x5588eea93160 2024-04-17T07:33:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.247+0000 7f901cff5740 10 client.28783 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.247+0000 7f901cff5740 1 -- 172.21.15.86:0/4092964865 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588eeaa4a60 con 0x5588eea93160 2024-04-17T07:33:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.248+0000 7f8ffd600640 1 -- 172.21.15.86:0/4092964865 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f900000fe40 con 0x5588eea93160 2024-04-17T07:33:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.248+0000 7f8ffd600640 1 -- 172.21.15.86:0/4092964865 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f900001ca40 con 0x5588eea93160 2024-04-17T07:33:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.248+0000 7f901cff5740 10 client.28783 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.248+0000 7f901cff5740 10 client.28783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:33:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.248+0000 7f901cff5740 20 client.28783 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.248+0000 7f901cff5740 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5588eeb0fac0 0x5588eeb11ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.248+0000 7f901cff5740 4 client.28783 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:33:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.248+0000 7f901cff5740 1 -- 172.21.15.86:0/4092964865 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9004004470 con 0x5588eeb0fac0 2024-04-17T07:33:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.249+0000 7f9012a00640 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5588eeb0fac0 0x5588eeb11ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.251+0000 7f9012a00640 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5588eeb0fac0 0x5588eeb11ee0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0939bbcb35fd003 server_cookie=b71bb6fc9ca8e380 in_seq=0 out_seq=0 2024-04-17T07:33:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.251+0000 7f8ffd600640 10 client.28783 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:33:25.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.251+0000 7f8ffd600640 1 -- 172.21.15.86:0/4092964865 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9004004470 con 0x5588eeb0fac0 2024-04-17T07:33:25.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.251+0000 7f8ffd600640 10 client.28783 handle_command_reply: tid=0 2024-04-17T07:33:25.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.251+0000 7f901cff5740 1 client.28783 shutdown 2024-04-17T07:33:25.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f901cff5740 1 -- 172.21.15.86:0/4092964865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 msgr2=0x5588eeaa1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f901cff5740 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eeaa1dc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9000000c00 tx=0x7f900000f980 comp rx=0 tx=0).stop 2024-04-17T07:33:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f901cff5740 1 -- 172.21.15.86:0/4092964865 shutdown_connections 2024-04-17T07:33:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f901cff5740 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588eea98cc0 0x5588eeaa3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f901cff5740 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x5588eeb0fac0 0x5588eeb11ee0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f901cff5740 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5588eea93b50 0x5588eeaa2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f901cff5740 1 --2- 172.21.15.86:0/4092964865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588eea93160 0x5588eeaa1dc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f901cff5740 1 -- 172.21.15.86:0/4092964865 >> 172.21.15.86:0/4092964865 conn(0x5588eea8e6d0 msgr2=0x5588eea9d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f8ffd600640 0 client.28783 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:33:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f901cff5740 1 -- 172.21.15.86:0/4092964865 shutdown_connections 2024-04-17T07:33:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.252+0000 7f901cff5740 1 -- 172.21.15.86:0/4092964865 wait complete. 2024-04-17T07:33:25.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.253+0000 7f901cff5740 20 client.28783 trim_cache size 0 max 16384 2024-04-17T07:33:25.254 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:33:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 -- 172.21.15.86:0/3595087541 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ff80a3c80 msgr2=0x7f8ff80a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ff80a3c80 0x7f8ff80a6140 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9000005d90 tx=0x7f9000005e40 comp rx=0 tx=0).stop 2024-04-17T07:33:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 -- 172.21.15.86:0/3595087541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f901410e460 msgr2=0x7f90141ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f901410e460 0x7f90141ae6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f900c00eaa0 tx=0x7f900c00ef60 comp rx=0 tx=0).stop 2024-04-17T07:33:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 -- 172.21.15.86:0/3595087541 shutdown_connections 2024-04-17T07:33:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8ff80a3c80 0x7f8ff80a6140 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f901410e460 0x7f90141ae6f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9014075fa0 0x7f9014070e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 --2- 172.21.15.86:0/3595087541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90140749c0 0x7f9014070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 -- 172.21.15.86:0/3595087541 >> 172.21.15.86:0/3595087541 conn(0x7f90140fd400 msgr2=0x7f90141016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.254+0000 7f9018e00640 1 -- 172.21.15.86:0/3595087541 shutdown_connections 2024-04-17T07:33:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.255+0000 7f9018e00640 1 -- 172.21.15.86:0/3595087541 wait complete. 2024-04-17T07:33:25.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:33:25.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:25 smithi098 ceph-mon[35418]: pgmap v1011: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 549 op/s 2024-04-17T07:33:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:25 smithi103 ceph-mon[35495]: pgmap v1011: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 549 op/s 2024-04-17T07:33:25.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:25 smithi086 ceph-mon[31733]: pgmap v1011: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 549 op/s 2024-04-17T07:33:25.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.597+0000 7fa8d7a00640 1 -- 172.21.15.86:0/782773963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d010b9b0 msgr2=0x7fa8d010be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.597+0000 7fa8d7a00640 1 --2- 172.21.15.86:0/782773963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d010b9b0 0x7fa8d010be30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009860 tx=0x7fa8bc02bb90 comp rx=0 tx=0).stop 2024-04-17T07:33:25.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.597+0000 7fa8d7a00640 1 -- 172.21.15.86:0/782773963 shutdown_connections 2024-04-17T07:33:25.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.597+0000 7fa8d7a00640 1 --2- 172.21.15.86:0/782773963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d010c370 0x7fa8d0112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.597+0000 7fa8d7a00640 1 --2- 172.21.15.86:0/782773963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d010b9b0 0x7fa8d010be30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.597+0000 7fa8d7a00640 1 --2- 172.21.15.86:0/782773963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8d010a7c0 0x7fa8d010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.597+0000 7fa8d7a00640 1 -- 172.21.15.86:0/782773963 >> 172.21.15.86:0/782773963 conn(0x7fa8d0070f70 msgr2=0x7fa8d0073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.597+0000 7fa8d7a00640 1 -- 172.21.15.86:0/782773963 shutdown_connections 2024-04-17T07:33:25.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.598+0000 7fa8d7a00640 1 -- 172.21.15.86:0/782773963 wait complete. 2024-04-17T07:33:25.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.599+0000 7fa8d7a00640 1 Processor -- start 2024-04-17T07:33:25.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.599+0000 7fa8d7a00640 1 -- start start 2024-04-17T07:33:25.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.600+0000 7fa8d7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8d010a7c0 0x7fa8d01a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.600+0000 7fa8d7a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d010b9b0 0x7fa8d01a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.600+0000 7fa8d7a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d010c370 0x7fa8d01aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.600+0000 7fa8d7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01162d0 con 0x7fa8d010a7c0 2024-04-17T07:33:25.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.600+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d010b9b0 0x7fa8d01a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.600+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d010b9b0 0x7fa8d01a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54876/0 (socket says 172.21.15.86:54876) 2024-04-17T07:33:25.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.600+0000 7fa8cfe00640 1 -- 172.21.15.86:0/3641335198 learned_addr learned my addr 172.21.15.86:0/3641335198 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.600+0000 7fa8d7a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0116150 con 0x7fa8d010b9b0 2024-04-17T07:33:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.601+0000 7fa8d7a00640 1 -- 172.21.15.86:0/3641335198 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0116450 con 0x7fa8d010c370 2024-04-17T07:33:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.601+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d010c370 0x7fa8d01aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.601+0000 7fa8d5000640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8d010a7c0 0x7fa8d01a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.601+0000 7fa8cfe00640 1 -- 172.21.15.86:0/3641335198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d010c370 msgr2=0x7fa8d01aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.601+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d010c370 0x7fa8d01aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.601+0000 7fa8cfe00640 1 -- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8d010a7c0 msgr2=0x7fa8d01a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.603+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8d010a7c0 0x7fa8d01a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.603+0000 7fa8cfe00640 1 -- 172.21.15.86:0/3641335198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d01aa8f0 con 0x7fa8d010b9b0 2024-04-17T07:33:25.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.603+0000 7fa8d5000640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8d010a7c0 0x7fa8d01a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:33:25.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.603+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d010b9b0 0x7fa8d01a0f40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0057c0 tx=0x7fa8bc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.604+0000 7fa8cd600640 1 -- 172.21.15.86:0/3641335198 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc041030 con 0x7fa8d010b9b0 2024-04-17T07:33:25.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.604+0000 7fa8cd600640 1 -- 172.21.15.86:0/3641335198 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8bc02c9e0 con 0x7fa8d010b9b0 2024-04-17T07:33:25.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.604+0000 7fa8cd600640 1 -- 172.21.15.86:0/3641335198 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc039040 con 0x7fa8d010b9b0 2024-04-17T07:33:25.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.604+0000 7fa8d7a00640 1 -- 172.21.15.86:0/3641335198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d01aaac0 con 0x7fa8d010b9b0 2024-04-17T07:33:25.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.604+0000 7fa8d7a00640 1 -- 172.21.15.86:0/3641335198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d01aafc0 con 0x7fa8d010b9b0 2024-04-17T07:33:25.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.614+0000 7fa8da67b740 1 Processor -- start 2024-04-17T07:33:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.614+0000 7fa8da67b740 1 -- start start 2024-04-17T07:33:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.615+0000 7fa8cd600640 1 -- 172.21.15.86:0/3641335198 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa8bc044070 con 0x7fa8d010b9b0 2024-04-17T07:33:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.615+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e786bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.615+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7871a0 0x56467e78bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.615+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e78e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.615+0000 7fa8da67b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56467e6e4020 con 0x56467e7871a0 2024-04-17T07:33:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.615+0000 7fa8da67b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56467e68a460 con 0x56467e7867b0 2024-04-17T07:33:25.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.615+0000 7fa8da67b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56467e6dffe0 con 0x56467e78c310 2024-04-17T07:33:25.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.616+0000 7fa8d5a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e786bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.616+0000 7fa8d5a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e786bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54882/0 (socket says 172.21.15.86:54882) 2024-04-17T07:33:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.616+0000 7fa8d5a00640 1 -- 172.21.15.86:0/2787769221 learned_addr learned my addr 172.21.15.86:0/2787769221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.617+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/2787769221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e78e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.617+0000 7fa8cd600640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa8b80a3cb0 0x7fa8b80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.618+0000 7fa8d5a00640 1 -- 172.21.15.86:0/2787769221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 msgr2=0x56467e78e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.618+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/2787769221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e78e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.618+0000 7fa8d5a00640 1 -- 172.21.15.86:0/2787769221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7871a0 msgr2=0x56467e78bca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:33:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.618+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/2787769221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7871a0 0x56467e78bca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.618+0000 7fa8d5a00640 1 -- 172.21.15.86:0/2787769221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56467e692040 con 0x56467e7867b0 2024-04-17T07:33:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.618+0000 7fa8cd600640 1 -- 172.21.15.86:0/3641335198 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa8bc0f1de0 con 0x7fa8d010b9b0 2024-04-17T07:33:25.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.619+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa8b80a3cb0 0x7fa8b80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.621+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa8b80a3cb0 0x7fa8b80a6170 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa8c801a000 tx=0x7fa8c801c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.621+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/2787769221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e786bd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8010370 tx=0x7fa8c8010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9bb5561ccdb4428 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.622+0000 7fa8c2000640 1 -- 172.21.15.86:0/2787769221 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8023070 con 0x56467e7867b0 2024-04-17T07:33:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.622+0000 7fa8c2000640 1 -- 172.21.15.86:0/2787769221 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c8011bd0 con 0x56467e7867b0 2024-04-17T07:33:25.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.622+0000 7fa8c2000640 1 -- 172.21.15.86:0/2787769221 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8018040 con 0x56467e7867b0 2024-04-17T07:33:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.623+0000 7fa8da67b740 1 -- 172.21.15.86:0/2787769221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 msgr2=0x56467e786bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.624+0000 7fa8da67b740 1 --2- 172.21.15.86:0/2787769221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e786bd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8010370 tx=0x7fa8c8010830 comp rx=0 tx=0).stop 2024-04-17T07:33:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.624+0000 7fa8da67b740 1 -- 172.21.15.86:0/2787769221 shutdown_connections 2024-04-17T07:33:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.624+0000 7fa8da67b740 1 --2- 172.21.15.86:0/2787769221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e78e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.624+0000 7fa8da67b740 1 --2- 172.21.15.86:0/2787769221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7871a0 0x56467e78bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.624+0000 7fa8da67b740 1 --2- 172.21.15.86:0/2787769221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e786bd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.624+0000 7fa8da67b740 1 -- 172.21.15.86:0/2787769221 >> 172.21.15.86:0/2787769221 conn(0x56467e779510 msgr2=0x56467e779910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.624+0000 7fa8da67b740 1 -- 172.21.15.86:0/2787769221 shutdown_connections 2024-04-17T07:33:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.624+0000 7fa8da67b740 1 -- 172.21.15.86:0/2787769221 wait complete. 2024-04-17T07:33:25.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.625+0000 7fa8da67b740 1 -- start start 2024-04-17T07:33:25.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.625+0000 7fa8da67b740 1 Processor -- start 2024-04-17T07:33:25.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.626+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7867b0 0x56467e77fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.626+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.626+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e785110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.626+0000 7fa8da67b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56467e77e040 con 0x56467e7867b0 2024-04-17T07:33:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.626+0000 7fa8da67b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56467e69d830 con 0x56467e7871a0 2024-04-17T07:33:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.626+0000 7fa8da67b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56467e77e1c0 con 0x56467e78c310 2024-04-17T07:33:25.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.627+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.627+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7800e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54894/0 (socket says 172.21.15.86:54894) 2024-04-17T07:33:25.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.627+0000 7fa8cfe00640 1 -- 172.21.15.86:0/909108180 learned_addr learned my addr 172.21.15.86:0/909108180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:25.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.627+0000 7fa8da67b740 10 client.0 _ll_register_callbacks cb 0x56467e66bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:25.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.627+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e785110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.628+0000 7fa8cfe00640 1 -- 172.21.15.86:0/909108180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 msgr2=0x56467e785110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.628+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e785110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.628+0000 7fa8cfe00640 1 -- 172.21.15.86:0/909108180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7867b0 msgr2=0x56467e77fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.628+0000 7fa8d5000640 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7867b0 0x56467e77fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.628+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7867b0 0x56467e77fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.629+0000 7fa8cfe00640 1 -- 172.21.15.86:0/909108180 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56467e6a1920 con 0x56467e7871a0 2024-04-17T07:33:25.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.629+0000 7fa8d5000640 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7867b0 0x56467e77fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:33:25.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.629+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7800e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc02c1a0 tx=0x7fa8bc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfc45aea094b6b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.629+0000 7fa8b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:33:25.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.630+0000 7fa8b6a00640 1 -- 172.21.15.86:0/909108180 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0c2090 con 0x56467e7871a0 2024-04-17T07:33:25.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.630+0000 7fa8da67b740 1 -- 172.21.15.86:0/909108180 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c8011bd0 con 0x56467e7871a0 2024-04-17T07:33:25.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.630+0000 7fa8b6a00640 1 -- 172.21.15.86:0/909108180 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa8bc005120 con 0x56467e7871a0 2024-04-17T07:33:25.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.631+0000 7fa8b6a00640 1 -- 172.21.15.86:0/909108180 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa8bc0c6950 con 0x56467e7871a0 2024-04-17T07:33:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.634+0000 7fa8da67b740 10 client.38789 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:25.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.634+0000 7fa8da67b740 1 -- 172.21.15.86:0/909108180 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56467e66f7f0 con 0x56467e7871a0 2024-04-17T07:33:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.635+0000 7fa8b6a00640 1 -- 172.21.15.86:0/909108180 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c8011bd0 con 0x56467e7871a0 2024-04-17T07:33:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.635+0000 7fa8b6a00640 1 -- 172.21.15.86:0/909108180 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa8bc04e060 con 0x56467e7871a0 2024-04-17T07:33:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.635+0000 7fa8da67b740 10 client.38789 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.635+0000 7fa8da67b740 10 client.38789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:33:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.635+0000 7fa8da67b740 20 client.38789 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:25.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.635+0000 7fa8da67b740 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56467e7a4f90 0x56467e7a7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.635+0000 7fa8da67b740 4 client.38789 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.635+0000 7fa8da67b740 1 -- 172.21.15.86:0/909108180 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56467e6f1a40 con 0x56467e7a4f90 2024-04-17T07:33:25.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.636+0000 7fa8d5000640 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56467e7a4f90 0x56467e7a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.636+0000 7fa8d5000640 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56467e7a4f90 0x56467e7a7450 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248d48f5dd2d6041 server_cookie=8f6972bb6a337b59 in_seq=0 out_seq=0 2024-04-17T07:33:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.636+0000 7fa8b6a00640 10 client.38789 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.637+0000 7fa8b6a00640 1 -- 172.21.15.86:0/909108180 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56467e6f1a40 con 0x56467e7a4f90 2024-04-17T07:33:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.637+0000 7fa8b6a00640 10 client.38789 handle_command_reply: tid=0 2024-04-17T07:33:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.638+0000 7fa8da67b740 1 client.38789 shutdown 2024-04-17T07:33:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.638+0000 7fa8da67b740 1 -- 172.21.15.86:0/909108180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 msgr2=0x56467e7800e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.638+0000 7fa8da67b740 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7800e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc02c1a0 tx=0x7fa8bc005f90 comp rx=0 tx=0).stop 2024-04-17T07:33:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.639+0000 7fa8da67b740 1 -- 172.21.15.86:0/909108180 shutdown_connections 2024-04-17T07:33:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.639+0000 7fa8da67b740 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56467e7a4f90 0x56467e7a7450 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.639+0000 7fa8da67b740 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e785110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.639+0000 7fa8da67b740 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7800e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.639+0000 7fa8da67b740 1 --2- 172.21.15.86:0/909108180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7867b0 0x56467e77fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.639+0000 7fa8b6a00640 0 client.38789 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.639+0000 7fa8da67b740 1 -- 172.21.15.86:0/909108180 >> 172.21.15.86:0/909108180 conn(0x56467e779510 msgr2=0x56467e778710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.641+0000 7fa8da67b740 1 -- 172.21.15.86:0/909108180 shutdown_connections 2024-04-17T07:33:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.641+0000 7fa8da67b740 1 -- 172.21.15.86:0/909108180 wait complete. 2024-04-17T07:33:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.641+0000 7fa8da67b740 20 client.38789 trim_cache size 0 max 16384 2024-04-17T07:33:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.672+0000 7fa8da67b740 1 Processor -- start 2024-04-17T07:33:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.672+0000 7fa8da67b740 1 -- start start 2024-04-17T07:33:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.673+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e777e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.673+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7871a0 0x56467e7783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.673+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e778900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.673+0000 7fa8da67b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b1190 con 0x56467e7871a0 2024-04-17T07:33:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.673+0000 7fa8da67b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b2cd0 con 0x56467e7867b0 2024-04-17T07:33:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.673+0000 7fa8da67b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b1860 con 0x56467e78c310 2024-04-17T07:33:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.673+0000 7fa8d5000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e777e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.674+0000 7fa8d5000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e777e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54900/0 (socket says 172.21.15.86:54900) 2024-04-17T07:33:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.674+0000 7fa8d5000640 1 -- 172.21.15.86:0/3705286182 learned_addr learned my addr 172.21.15.86:0/3705286182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.674+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/3705286182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e778900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.674+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/3705286182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7871a0 0x56467e7783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.675+0000 7fa8d5a00640 1 -- 172.21.15.86:0/3705286182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 msgr2=0x56467e777e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.675+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/3705286182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e777e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.675+0000 7fa8d5a00640 1 -- 172.21.15.86:0/3705286182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7871a0 msgr2=0x56467e7783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.675+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/3705286182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7871a0 0x56467e7783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.675+0000 7fa8d5a00640 1 -- 172.21.15.86:0/3705286182 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56467e3f63f0 con 0x56467e78c310 2024-04-17T07:33:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.676+0000 7fa8d5a00640 1 --2- 172.21.15.86:0/3705286182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e778900 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa8c801f040 tx=0x7fa8c80049a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=952b1211a8b4921c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.676+0000 7fa8c1600640 1 -- 172.21.15.86:0/3705286182 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8023040 con 0x56467e78c310 2024-04-17T07:33:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.677+0000 7fa8c1600640 1 -- 172.21.15.86:0/3705286182 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c8028b00 con 0x56467e78c310 2024-04-17T07:33:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.677+0000 7fa8c1600640 1 -- 172.21.15.86:0/3705286182 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8018040 con 0x56467e78c310 2024-04-17T07:33:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.678+0000 7fa8da67b740 1 -- 172.21.15.86:0/3705286182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 msgr2=0x56467e778900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.678+0000 7fa8da67b740 1 --2- 172.21.15.86:0/3705286182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e778900 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa8c801f040 tx=0x7fa8c80049a0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.678+0000 7fa8da67b740 1 -- 172.21.15.86:0/3705286182 shutdown_connections 2024-04-17T07:33:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.678+0000 7fa8da67b740 1 --2- 172.21.15.86:0/3705286182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e78c310 0x56467e778900 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.678+0000 7fa8da67b740 1 --2- 172.21.15.86:0/3705286182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e7871a0 0x56467e7783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.678+0000 7fa8da67b740 1 --2- 172.21.15.86:0/3705286182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7867b0 0x56467e777e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.678+0000 7fa8da67b740 1 -- 172.21.15.86:0/3705286182 >> 172.21.15.86:0/3705286182 conn(0x56467e779510 msgr2=0x56467e788700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.678+0000 7fa8da67b740 1 -- 172.21.15.86:0/3705286182 shutdown_connections 2024-04-17T07:33:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.678+0000 7fa8da67b740 1 -- 172.21.15.86:0/3705286182 wait complete. 2024-04-17T07:33:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.679+0000 7fa8da67b740 1 -- start start 2024-04-17T07:33:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.679+0000 7fa8da67b740 1 Processor -- start 2024-04-17T07:33:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.680+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e7867b0 0x56467e7b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.680+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.680+0000 7fa8da67b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e78c310 0x56467e7b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.680+0000 7fa8da67b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b2600 con 0x56467e78c310 2024-04-17T07:33:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.680+0000 7fa8da67b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b1f30 con 0x56467e7871a0 2024-04-17T07:33:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.680+0000 7fa8da67b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01af120 con 0x56467e7867b0 2024-04-17T07:33:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.680+0000 7fa8da67b740 10 client.0 _ll_register_callbacks cb 0x7fa8bc0044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.681+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.681+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:54904/0 (socket says 172.21.15.86:54904) 2024-04-17T07:33:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.681+0000 7fa8cfe00640 1 -- 172.21.15.86:0/135841192 learned_addr learned my addr 172.21.15.86:0/135841192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.681+0000 7fa8cfe00640 1 -- 172.21.15.86:0/135841192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e7867b0 msgr2=0x56467e7b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.681+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e7867b0 0x56467e7b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.681+0000 7fa8cfe00640 1 -- 172.21.15.86:0/135841192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e78c310 msgr2=0x56467e7b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.681+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e78c310 0x56467e7b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.681+0000 7fa8cfe00640 1 -- 172.21.15.86:0/135841192 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56467e4c07d0 con 0x56467e7871a0 2024-04-17T07:33:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.682+0000 7fa8cfe00640 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7b6dd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0bd9e0 tx=0x7fa8bc0ba080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1cbbe20e733403a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.682+0000 7fa8b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:33:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.682+0000 7fa8b6a00640 1 -- 172.21.15.86:0/135841192 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0bdce0 con 0x56467e7871a0 2024-04-17T07:33:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.682+0000 7fa8b6a00640 1 -- 172.21.15.86:0/135841192 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa8bc02cb90 con 0x56467e7871a0 2024-04-17T07:33:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.683+0000 7fa8da67b740 1 -- 172.21.15.86:0/135841192 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c8028b00 con 0x56467e7871a0 2024-04-17T07:33:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.683+0000 7fa8b6a00640 1 -- 172.21.15.86:0/135841192 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa8bc0c64c0 con 0x56467e7871a0 2024-04-17T07:33:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.683+0000 7fa8da67b740 10 client.38795 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.683+0000 7fa8da67b740 1 -- 172.21.15.86:0/135841192 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56467e80b750 con 0x56467e7871a0 2024-04-17T07:33:25.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.684+0000 7fa8b6a00640 1 -- 172.21.15.86:0/135841192 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c8028b00 con 0x56467e7871a0 2024-04-17T07:33:25.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.685+0000 7fa8b6a00640 1 -- 172.21.15.86:0/135841192 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa8bc03c030 con 0x56467e7871a0 2024-04-17T07:33:25.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.685+0000 7fa8da67b740 10 client.38795 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:25.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.685+0000 7fa8da67b740 10 client.38795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:33:25.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.685+0000 7fa8da67b740 20 client.38795 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:25.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.685+0000 7fa8da67b740 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56467e80bdc0 0x56467e80e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:25.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.685+0000 7fa8da67b740 4 client.38795 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:33:25.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.685+0000 7fa8da67b740 1 -- 172.21.15.86:0/135841192 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56467e6eb8b0 con 0x56467e80bdc0 2024-04-17T07:33:25.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.687+0000 7fa8d5000640 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56467e80bdc0 0x56467e80e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.688+0000 7fa8d5000640 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56467e80bdc0 0x56467e80e1e0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524ac6a08d42e6e7 server_cookie=7c60785e517ceea4 in_seq=0 out_seq=0 2024-04-17T07:33:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.688+0000 7fa8b6a00640 10 client.38795 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.688+0000 7fa8b6a00640 1 -- 172.21.15.86:0/135841192 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56467e6eb8b0 con 0x56467e80bdc0 2024-04-17T07:33:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.688+0000 7fa8b6a00640 10 client.38795 handle_command_reply: tid=0 2024-04-17T07:33:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.688+0000 7fa8da67b740 1 client.38795 shutdown 2024-04-17T07:33:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.688+0000 7fa8da67b740 1 -- 172.21.15.86:0/135841192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 msgr2=0x56467e7b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.688+0000 7fa8da67b740 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7b6dd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0bd9e0 tx=0x7fa8bc0ba080 comp rx=0 tx=0).stop 2024-04-17T07:33:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.689+0000 7fa8da67b740 1 -- 172.21.15.86:0/135841192 shutdown_connections 2024-04-17T07:33:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.689+0000 7fa8da67b740 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56467e80bdc0 0x56467e80e1e0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.689+0000 7fa8da67b740 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56467e78c310 0x56467e7b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.689+0000 7fa8da67b740 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56467e7871a0 0x56467e7b6dd0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.689+0000 7fa8da67b740 1 --2- 172.21.15.86:0/135841192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56467e7867b0 0x56467e7b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.689+0000 7fa8da67b740 1 -- 172.21.15.86:0/135841192 >> 172.21.15.86:0/135841192 conn(0x56467e779510 msgr2=0x56467e77c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.689+0000 7fa8b6a00640 0 client.38795 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:25.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.689+0000 7fa8da67b740 1 -- 172.21.15.86:0/135841192 shutdown_connections 2024-04-17T07:33:25.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.689+0000 7fa8da67b740 1 -- 172.21.15.86:0/135841192 wait complete. 2024-04-17T07:33:25.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.689+0000 7fa8da67b740 20 client.38795 trim_cache size 0 max 16384 2024-04-17T07:33:25.691 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:33:25.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.690+0000 7fa8d7a00640 1 -- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa8b80a3cb0 msgr2=0x7fa8b80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.690+0000 7fa8d7a00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa8b80a3cb0 0x7fa8b80a6170 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa8c801a000 tx=0x7fa8c801c000 comp rx=0 tx=0).stop 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.691+0000 7fa8d7a00640 1 -- 172.21.15.86:0/3641335198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d010b9b0 msgr2=0x7fa8d01a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.691+0000 7fa8d7a00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d010b9b0 0x7fa8d01a0f40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0057c0 tx=0x7fa8bc0042f0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.691+0000 7fa8d7a00640 1 -- 172.21.15.86:0/3641335198 shutdown_connections 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.691+0000 7fa8d7a00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa8b80a3cb0 0x7fa8b80a6170 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.691+0000 7fa8d7a00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8d010c370 0x7fa8d01aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.691+0000 7fa8d7a00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d010b9b0 0x7fa8d01a0f40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.691+0000 7fa8d7a00640 1 --2- 172.21.15.86:0/3641335198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8d010a7c0 0x7fa8d01a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.691+0000 7fa8d7a00640 1 -- 172.21.15.86:0/3641335198 >> 172.21.15.86:0/3641335198 conn(0x7fa8d0070f70 msgr2=0x7fa8d010ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.691+0000 7fa8d7a00640 1 -- 172.21.15.86:0/3641335198 shutdown_connections 2024-04-17T07:33:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:25.691+0000 7fa8d7a00640 1 -- 172.21.15.86:0/3641335198 wait complete. 2024-04-17T07:33:26.721 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:33:26.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:33:26.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:33:27.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.063+0000 7fa0e1c00640 1 -- 172.21.15.86:0/1226799263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0dc103b60 msgr2=0x7fa0dc10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.063+0000 7fa0e1c00640 1 --2- 172.21.15.86:0/1226799263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0dc103b60 0x7fa0dc10a3f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009ca0 tx=0x7fa0d402c100 comp rx=0 tx=0).stop 2024-04-17T07:33:27.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.063+0000 7fa0e1c00640 1 -- 172.21.15.86:0/1226799263 shutdown_connections 2024-04-17T07:33:27.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.063+0000 7fa0e1c00640 1 --2- 172.21.15.86:0/1226799263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0dc103b60 0x7fa0dc10a3f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.063+0000 7fa0e1c00640 1 --2- 172.21.15.86:0/1226799263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0dc1031a0 0x7fa0dc103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.063+0000 7fa0e1c00640 1 --2- 172.21.15.86:0/1226799263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc101fb0 0x7fa0dc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.063+0000 7fa0e1c00640 1 -- 172.21.15.86:0/1226799263 >> 172.21.15.86:0/1226799263 conn(0x7fa0dc0fd540 msgr2=0x7fa0dc0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.064+0000 7fa0e1c00640 1 -- 172.21.15.86:0/1226799263 shutdown_connections 2024-04-17T07:33:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.064+0000 7fa0e1c00640 1 -- 172.21.15.86:0/1226799263 wait complete. 2024-04-17T07:33:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.066+0000 7fa0e1c00640 1 Processor -- start 2024-04-17T07:33:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.066+0000 7fa0e1c00640 1 -- start start 2024-04-17T07:33:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.067+0000 7fa0e1c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0dc101fb0 0x7fa0dc0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.067+0000 7fa0e1c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0dc1031a0 0x7fa0dc070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.068+0000 7fa0e1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc103b60 0x7fa0dc1a81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.068+0000 7fa0e1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc10dab0 con 0x7fa0dc103b60 2024-04-17T07:33:27.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.068+0000 7fa0e1c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc10d930 con 0x7fa0dc101fb0 2024-04-17T07:33:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.068+0000 7fa0e1c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc10dc30 con 0x7fa0dc1031a0 2024-04-17T07:33:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.068+0000 7fa0da000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0dc1031a0 0x7fa0dc070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.068+0000 7fa0da000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0dc1031a0 0x7fa0dc070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:57562/0 (socket says 172.21.15.86:57562) 2024-04-17T07:33:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.068+0000 7fa0da000640 1 -- 172.21.15.86:0/3885377304 learned_addr learned my addr 172.21.15.86:0/3885377304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.069+0000 7fa0db400640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc103b60 0x7fa0dc1a81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.069+0000 7fa0db400640 1 -- 172.21.15.86:0/3885377304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0dc1031a0 msgr2=0x7fa0dc070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.069+0000 7fa0db400640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0dc1031a0 0x7fa0dc070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.069+0000 7fa0db400640 1 -- 172.21.15.86:0/3885377304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0dc101fb0 msgr2=0x7fa0dc0709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.069+0000 7fa0db400640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0dc101fb0 0x7fa0dc0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.069+0000 7fa0db400640 1 -- 172.21.15.86:0/3885377304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc1a8700 con 0x7fa0dc103b60 2024-04-17T07:33:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.070+0000 7fa0db400640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc103b60 0x7fa0dc1a81c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4037040 tx=0x7fa0d4009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.070+0000 7fa0d3400640 1 -- 172.21.15.86:0/3885377304 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4039040 con 0x7fa0dc103b60 2024-04-17T07:33:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.070+0000 7fa0d3400640 1 -- 172.21.15.86:0/3885377304 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0d4003e20 con 0x7fa0dc103b60 2024-04-17T07:33:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.070+0000 7fa0d3400640 1 -- 172.21.15.86:0/3885377304 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40046b0 con 0x7fa0dc103b60 2024-04-17T07:33:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.071+0000 7fa0e1c00640 1 -- 172.21.15.86:0/3885377304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0dc074170 con 0x7fa0dc103b60 2024-04-17T07:33:27.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.071+0000 7fa0e1c00640 1 -- 172.21.15.86:0/3885377304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0dc074610 con 0x7fa0dc103b60 2024-04-17T07:33:27.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.076+0000 7fa0d3400640 1 -- 172.21.15.86:0/3885377304 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fa0d4003970 con 0x7fa0dc103b60 2024-04-17T07:33:27.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.078+0000 7fa0d3400640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0c00a3db0 0x7fa0c00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.078+0000 7fa0d3400640 1 -- 172.21.15.86:0/3885377304 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa0d40f7070 con 0x7fa0dc103b60 2024-04-17T07:33:27.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.080+0000 7fa0daa00640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0c00a3db0 0x7fa0c00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.086+0000 7fa0e4a0f740 1 Processor -- start 2024-04-17T07:33:27.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.086+0000 7fa0e4a0f740 1 -- start start 2024-04-17T07:33:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.087+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68741ec0 0x560d687422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.087+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d687473b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.087+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d68749eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.088+0000 7fa0e4a0f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d686a7cb0 con 0x560d687428b0 2024-04-17T07:33:27.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.088+0000 7fa0e4a0f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560d6864e370 con 0x560d68747a20 2024-04-17T07:33:27.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.088+0000 7fa0e4a0f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560d686a3c70 con 0x560d68741ec0 2024-04-17T07:33:27.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.088+0000 7fa0da000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d687473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.088+0000 7fa0db400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d68749eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.088+0000 7fa0da000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d687473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47820/0 (socket says 172.21.15.86:47820) 2024-04-17T07:33:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.088+0000 7fa0da000640 1 -- 172.21.15.86:0/1447436826 learned_addr learned my addr 172.21.15.86:0/1447436826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.089+0000 7fa0da000640 1 -- 172.21.15.86:0/1447436826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68741ec0 msgr2=0x560d687422e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:33:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.089+0000 7fa0da000640 1 --2- 172.21.15.86:0/1447436826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68741ec0 0x560d687422e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.089+0000 7fa0da000640 1 -- 172.21.15.86:0/1447436826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 msgr2=0x560d68749eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.089+0000 7fa0da000640 1 --2- 172.21.15.86:0/1447436826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d68749eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.089+0000 7fa0da000640 1 -- 172.21.15.86:0/1447436826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6866d910 con 0x560d687428b0 2024-04-17T07:33:27.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.089+0000 7fa0db400640 1 --2- 172.21.15.86:0/1447436826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d68749eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:33:27.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.090+0000 7fa0da000640 1 --2- 172.21.15.86:0/1447436826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d687473b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009c50 tx=0x7fa0c80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bad025608371a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.090+0000 7fa0c7e00640 1 -- 172.21.15.86:0/1447436826 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8004020 con 0x560d687428b0 2024-04-17T07:33:27.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.090+0000 7fa0c7e00640 1 -- 172.21.15.86:0/1447436826 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c80041c0 con 0x560d687428b0 2024-04-17T07:33:27.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.091+0000 7fa0c7e00640 1 -- 172.21.15.86:0/1447436826 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8012690 con 0x560d687428b0 2024-04-17T07:33:27.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.091+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/1447436826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 msgr2=0x560d687473b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.091+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/1447436826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d687473b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009c50 tx=0x7fa0c80117b0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.092+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/1447436826 shutdown_connections 2024-04-17T07:33:27.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.092+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/1447436826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d68749eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.092+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/1447436826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d687473b0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.092+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/1447436826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68741ec0 0x560d687422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.092+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/1447436826 >> 172.21.15.86:0/1447436826 conn(0x560d6873d430 msgr2=0x560d6873d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.094+0000 7fa0daa00640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0c00a3db0 0x7fa0c00a6270 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc103000 tx=0x7fa0cc005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.096+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/1447436826 shutdown_connections 2024-04-17T07:33:27.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.096+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/1447436826 wait complete. 2024-04-17T07:33:27.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.097+0000 7fa0e4a0f740 1 -- start start 2024-04-17T07:33:27.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.097+0000 7fa0e4a0f740 1 Processor -- start 2024-04-17T07:33:27.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.098+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d6874c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.098+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d6874cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.099+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 0x560d68753f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.099+0000 7fa0da000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d6874cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.099+0000 7fa0da000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d6874cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38966/0 (socket says 172.21.15.86:38966) 2024-04-17T07:33:27.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.099+0000 7fa0daa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d6874c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.099+0000 7fa0daa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d6874c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47836/0 (socket says 172.21.15.86:47836) 2024-04-17T07:33:27.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.099+0000 7fa0db400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 0x560d68753f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.099+0000 7fa0db400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 0x560d68753f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:57578/0 (socket says 172.21.15.86:57578) 2024-04-17T07:33:27.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.100+0000 7fa0da000640 1 -- 172.21.15.86:0/4123287253 learned_addr learned my addr 172.21.15.86:0/4123287253 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:27.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.100+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560d6874a9d0 con 0x560d687428b0 2024-04-17T07:33:27.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.100+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560d68661730 con 0x560d68747a20 2024-04-17T07:33:27.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.100+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560d6874ab50 con 0x560d68751b60 2024-04-17T07:33:27.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.100+0000 7fa0e4a0f740 10 client.0 _ll_register_callbacks cb 0x560d68663b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:27.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.101+0000 7fa0db400640 1 -- 172.21.15.86:0/4123287253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 msgr2=0x560d6874cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.101+0000 7fa0db400640 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d6874cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.101+0000 7fa0db400640 1 -- 172.21.15.86:0/4123287253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 msgr2=0x560d6874c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.102+0000 7fa0da000640 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d6874cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:33:27.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.102+0000 7fa0db400640 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d6874c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.102+0000 7fa0db400640 1 -- 172.21.15.86:0/4123287253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d684827e0 con 0x560d68751b60 2024-04-17T07:33:27.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.102+0000 7fa0daa00640 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d6874c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:33:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.104+0000 7fa0db400640 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 0x560d68753f50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40c2c70 tx=0x7fa0d40c28d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e83fc897c620f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.105+0000 7fa0c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:33:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.105+0000 7fa0c4c00640 1 -- 172.21.15.86:0/4123287253 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40db070 con 0x560d68751b60 2024-04-17T07:33:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.105+0000 7fa0c4c00640 1 -- 172.21.15.86:0/4123287253 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa0d40dc270 con 0x560d68751b60 2024-04-17T07:33:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.105+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c80041c0 con 0x560d68751b60 2024-04-17T07:33:27.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.105+0000 7fa0c4c00640 1 -- 172.21.15.86:0/4123287253 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa0d40475e0 con 0x560d68751b60 2024-04-17T07:33:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.105+0000 7fa0e4a0f740 10 client.28804 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.105+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d684e2f70 con 0x560d68751b60 2024-04-17T07:33:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.106+0000 7fa0c4c00640 1 -- 172.21.15.86:0/4123287253 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c80041c0 con 0x560d68751b60 2024-04-17T07:33:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.106+0000 7fa0c4c00640 1 -- 172.21.15.86:0/4123287253 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa0d4050070 con 0x560d68751b60 2024-04-17T07:33:27.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.106+0000 7fa0e4a0f740 10 client.28804 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.106+0000 7fa0e4a0f740 10 client.28804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:33:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.106+0000 7fa0e4a0f740 20 client.28804 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.106+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560d68758490 0x560d6875a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.106+0000 7fa0e4a0f740 4 client.28804 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.106+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d686b56d0 con 0x560d68758490 2024-04-17T07:33:27.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.107+0000 7fa0daa00640 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560d68758490 0x560d6875a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.108+0000 7fa0daa00640 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560d68758490 0x560d6875a8e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d1e05b9c7dc0fc server_cookie=99ea4c0962a540ab in_seq=0 out_seq=0 2024-04-17T07:33:27.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.108+0000 7fa0c4c00640 10 client.28804 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:27.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.109+0000 7fa0c4c00640 1 -- 172.21.15.86:0/4123287253 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d686b56d0 con 0x560d68758490 2024-04-17T07:33:27.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.109+0000 7fa0c4c00640 10 client.28804 handle_command_reply: tid=0 2024-04-17T07:33:27.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.110+0000 7fa0e4a0f740 1 client.28804 shutdown 2024-04-17T07:33:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.113+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 msgr2=0x560d68753f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.113+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 0x560d68753f50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40c2c70 tx=0x7fa0d40c28d0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.114+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 shutdown_connections 2024-04-17T07:33:27.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.114+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560d68758490 0x560d6875a8e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.114+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 0x560d68753f50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.114+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d68747a20 0x560d6874cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.114+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/4123287253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d687428b0 0x560d6874c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.114+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 >> 172.21.15.86:0/4123287253 conn(0x560d6873d430 msgr2=0x560d6873c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.114+0000 7fa0c4c00640 0 client.28804 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:27.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.114+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 shutdown_connections 2024-04-17T07:33:27.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.114+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/4123287253 wait complete. 2024-04-17T07:33:27.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.115+0000 7fa0e4a0f740 20 client.28804 trim_cache size 0 max 16384 2024-04-17T07:33:27.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.151+0000 7fa0e4a0f740 1 Processor -- start 2024-04-17T07:33:27.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.151+0000 7fa0e4a0f740 1 -- start start 2024-04-17T07:33:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.151+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 0x560d6873c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.151+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68747a20 0x560d6874aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.151+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68751b60 0x560d6874b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.151+0000 7fa0e4a0f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b1080 con 0x560d68751b60 2024-04-17T07:33:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.151+0000 7fa0e4a0f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b2bc0 con 0x560d687428b0 2024-04-17T07:33:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.151+0000 7fa0e4a0f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b1750 con 0x560d68747a20 2024-04-17T07:33:27.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.152+0000 7fa0db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68751b60 0x560d6874b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.152+0000 7fa0db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68751b60 0x560d6874b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47844/0 (socket says 172.21.15.86:47844) 2024-04-17T07:33:27.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.152+0000 7fa0db400640 1 -- 172.21.15.86:0/2134770424 learned_addr learned my addr 172.21.15.86:0/2134770424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:27.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.152+0000 7fa0da000640 1 --2- 172.21.15.86:0/2134770424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68747a20 0x560d6874aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.152+0000 7fa0daa00640 1 --2- 172.21.15.86:0/2134770424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 0x560d6873c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.153+0000 7fa0db400640 1 -- 172.21.15.86:0/2134770424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68747a20 msgr2=0x560d6874aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.153+0000 7fa0db400640 1 --2- 172.21.15.86:0/2134770424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68747a20 0x560d6874aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.153+0000 7fa0db400640 1 -- 172.21.15.86:0/2134770424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 msgr2=0x560d6873c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.153+0000 7fa0db400640 1 --2- 172.21.15.86:0/2134770424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 0x560d6873c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.153+0000 7fa0db400640 1 -- 172.21.15.86:0/2134770424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6837a170 con 0x560d68751b60 2024-04-17T07:33:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.153+0000 7fa0db400640 1 --2- 172.21.15.86:0/2134770424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68751b60 0x560d6874b430 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40cb6b0 tx=0x7fa0d4032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994dd956a6bac0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.154+0000 7fa0c7400640 1 -- 172.21.15.86:0/2134770424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4004040 con 0x560d68751b60 2024-04-17T07:33:27.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.154+0000 7fa0c7400640 1 -- 172.21.15.86:0/2134770424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0d40dc4c0 con 0x560d68751b60 2024-04-17T07:33:27.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.154+0000 7fa0c7400640 1 -- 172.21.15.86:0/2134770424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4004040 con 0x560d68751b60 2024-04-17T07:33:27.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.155+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/2134770424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68751b60 msgr2=0x560d6874b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.156+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/2134770424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68751b60 0x560d6874b430 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40cb6b0 tx=0x7fa0d4032340 comp rx=0 tx=0).stop 2024-04-17T07:33:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.157+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/2134770424 shutdown_connections 2024-04-17T07:33:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.157+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/2134770424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68751b60 0x560d6874b430 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.157+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/2134770424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68747a20 0x560d6874aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.157+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/2134770424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 0x560d6873c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.157+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/2134770424 >> 172.21.15.86:0/2134770424 conn(0x560d6873d430 msgr2=0x560d6873bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.157+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/2134770424 shutdown_connections 2024-04-17T07:33:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.157+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/2134770424 wait complete. 2024-04-17T07:33:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.158+0000 7fa0e4a0f740 1 -- start start 2024-04-17T07:33:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.158+0000 7fa0e4a0f740 1 Processor -- start 2024-04-17T07:33:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.159+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 0x560d687bb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.159+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68747a20 0x560d687bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.159+0000 7fa0e4a0f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 0x560d6876ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.159+0000 7fa0e4a0f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b24f0 con 0x560d68747a20 2024-04-17T07:33:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.159+0000 7fa0e4a0f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b1e20 con 0x560d687428b0 2024-04-17T07:33:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.159+0000 7fa0e4a0f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1af050 con 0x560d68751b60 2024-04-17T07:33:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.159+0000 7fa0e4a0f740 10 client.0 _ll_register_callbacks cb 0x7fa0d4005f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:27.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.160+0000 7fa0da000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68747a20 0x560d687bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.160+0000 7fa0daa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 0x560d687bb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.160+0000 7fa0daa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 0x560d687bb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:38978/0 (socket says 172.21.15.86:38978) 2024-04-17T07:33:27.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.160+0000 7fa0daa00640 1 -- 172.21.15.86:0/3232640985 learned_addr learned my addr 172.21.15.86:0/3232640985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.161+0000 7fa0db400640 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 0x560d6876ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.161+0000 7fa0daa00640 1 -- 172.21.15.86:0/3232640985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 msgr2=0x560d6876ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.161+0000 7fa0daa00640 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 0x560d6876ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.161+0000 7fa0daa00640 1 -- 172.21.15.86:0/3232640985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68747a20 msgr2=0x560d687bbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.161+0000 7fa0daa00640 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68747a20 0x560d687bbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.161+0000 7fa0daa00640 1 -- 172.21.15.86:0/3232640985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6876b370 con 0x560d687428b0 2024-04-17T07:33:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.162+0000 7fa0daa00640 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 0x560d687bb5d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc012040 tx=0x7fa0cc010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915fc37086b370ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.162+0000 7fa0c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:33:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.162+0000 7fa0c4c00640 1 -- 172.21.15.86:0/3232640985 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc010cb0 con 0x560d687428b0 2024-04-17T07:33:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.162+0000 7fa0c4c00640 1 -- 172.21.15.86:0/3232640985 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fa0cc010ab0 con 0x560d687428b0 2024-04-17T07:33:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.163+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/3232640985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d40dc4c0 con 0x560d687428b0 2024-04-17T07:33:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.163+0000 7fa0c4c00640 1 -- 172.21.15.86:0/3232640985 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa0cc032050 con 0x560d687428b0 2024-04-17T07:33:27.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.163+0000 7fa0e4a0f740 10 client.38807 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.163+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/3232640985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d687c7210 con 0x560d687428b0 2024-04-17T07:33:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.164+0000 7fa0c4c00640 1 -- 172.21.15.86:0/3232640985 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0cc016040 con 0x560d687428b0 2024-04-17T07:33:27.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.165+0000 7fa0c4c00640 1 -- 172.21.15.86:0/3232640985 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fa0cc004aa0 con 0x560d687428b0 2024-04-17T07:33:27.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.165+0000 7fa0e4a0f740 10 client.38807 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:27.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.165+0000 7fa0e4a0f740 10 client.38807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:33:27.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.165+0000 7fa0e4a0f740 20 client.38807 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:27.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.165+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560d687c7ba0 0x560d687c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.165+0000 7fa0e4a0f740 4 client.38807 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:33:27.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.165+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/3232640985 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fa0d40dc4c0 con 0x560d687c7ba0 2024-04-17T07:33:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.165+0000 7fa0da000640 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560d687c7ba0 0x560d687c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.166+0000 7fa0da000640 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560d687c7ba0 0x560d687c9fc0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d593ba0c54f0fb5 server_cookie=d44ac8d9a268c59e in_seq=0 out_seq=0 2024-04-17T07:33:27.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.166+0000 7fa0c4c00640 10 client.38807 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.168+0000 7fa0c4c00640 1 -- 172.21.15.86:0/3232640985 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fa0d40dc4c0 con 0x560d687c7ba0 2024-04-17T07:33:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.168+0000 7fa0c4c00640 10 client.38807 handle_command_reply: tid=0 2024-04-17T07:33:27.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.168+0000 7fa0e4a0f740 1 client.38807 shutdown 2024-04-17T07:33:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.169+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/3232640985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 msgr2=0x560d687bb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.169+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 0x560d687bb5d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc012040 tx=0x7fa0cc010000 comp rx=0 tx=0).stop 2024-04-17T07:33:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.170+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/3232640985 shutdown_connections 2024-04-17T07:33:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.170+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x560d687c7ba0 0x560d687c9fc0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.170+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560d68751b60 0x560d6876ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.170+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d68747a20 0x560d687bbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.170+0000 7fa0e4a0f740 1 --2- 172.21.15.86:0/3232640985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d687428b0 0x560d687bb5d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.170+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/3232640985 >> 172.21.15.86:0/3232640985 conn(0x560d6873d430 msgr2=0x560d6873c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.170+0000 7fa0c4c00640 0 client.38807 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:27.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.170+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/3232640985 shutdown_connections 2024-04-17T07:33:27.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.170+0000 7fa0e4a0f740 1 -- 172.21.15.86:0/3232640985 wait complete. 2024-04-17T07:33:27.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.170+0000 7fa0e4a0f740 20 client.38807 trim_cache size 0 max 16384 2024-04-17T07:33:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:33:27.172 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:33:27.172 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "c089350c-6728-4db0-9d53-dd5090e15bbe", 2024-04-17T07:33:27.172 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:33:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:33:27.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.171+0000 7fa0e1c00640 1 -- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0c00a3db0 msgr2=0x7fa0c00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.171+0000 7fa0e1c00640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0c00a3db0 0x7fa0c00a6270 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc103000 tx=0x7fa0cc005ed0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.171+0000 7fa0e1c00640 1 -- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc103b60 msgr2=0x7fa0dc1a81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.171+0000 7fa0e1c00640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc103b60 0x7fa0dc1a81c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4037040 tx=0x7fa0d4009870 comp rx=0 tx=0).stop 2024-04-17T07:33:27.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.172+0000 7fa0e1c00640 1 -- 172.21.15.86:0/3885377304 shutdown_connections 2024-04-17T07:33:27.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.172+0000 7fa0e1c00640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fa0c00a3db0 0x7fa0c00a6270 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.172+0000 7fa0e1c00640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0dc103b60 0x7fa0dc1a81c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.172+0000 7fa0e1c00640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0dc1031a0 0x7fa0dc070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.172+0000 7fa0e1c00640 1 --2- 172.21.15.86:0/3885377304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0dc101fb0 0x7fa0dc0709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.172+0000 7fa0e1c00640 1 -- 172.21.15.86:0/3885377304 >> 172.21.15.86:0/3885377304 conn(0x7fa0dc0fd540 msgr2=0x7fa0dc1065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.172+0000 7fa0e1c00640 1 -- 172.21.15.86:0/3885377304 shutdown_connections 2024-04-17T07:33:27.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.172+0000 7fa0e1c00640 1 -- 172.21.15.86:0/3885377304 wait complete. 2024-04-17T07:33:27.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:33:27.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:33:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:27 smithi098 ceph-mon[35418]: pgmap v1012: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-04-17T07:33:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:27 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:33:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:27 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:33:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:27 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:33:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:27 smithi103 ceph-mon[35495]: pgmap v1012: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-04-17T07:33:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:27 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:33:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:27 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:33:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:27 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:33:27.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.554+0000 7feb92800640 1 -- 172.21.15.86:0/4159732533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb8c1097b0 msgr2=0x7feb8c109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.554+0000 7feb92800640 1 --2- 172.21.15.86:0/4159732533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb8c1097b0 0x7feb8c109c30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feb78009900 tx=0x7feb7802bc40 comp rx=0 tx=0).stop 2024-04-17T07:33:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.555+0000 7feb92800640 1 -- 172.21.15.86:0/4159732533 shutdown_connections 2024-04-17T07:33:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.555+0000 7feb92800640 1 --2- 172.21.15.86:0/4159732533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb8c10a170 0x7feb8c110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.555+0000 7feb92800640 1 --2- 172.21.15.86:0/4159732533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb8c1097b0 0x7feb8c109c30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.555+0000 7feb92800640 1 --2- 172.21.15.86:0/4159732533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb8c1085c0 0x7feb8c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.555+0000 7feb92800640 1 -- 172.21.15.86:0/4159732533 >> 172.21.15.86:0/4159732533 conn(0x7feb8c077560 msgr2=0x7feb8c0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.556+0000 7feb92800640 1 -- 172.21.15.86:0/4159732533 shutdown_connections 2024-04-17T07:33:27.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.556+0000 7feb92800640 1 -- 172.21.15.86:0/4159732533 wait complete. 2024-04-17T07:33:27.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.557+0000 7feb92800640 1 Processor -- start 2024-04-17T07:33:27.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.559+0000 7feb92800640 1 -- start start 2024-04-17T07:33:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.559+0000 7feb92800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb8c1085c0 0x7feb8c0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.559+0000 7feb92800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb8c1097b0 0x7feb8c070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.559+0000 7feb92800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb8c10a170 0x7feb8c1ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.559+0000 7feb92800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1141a0 con 0x7feb8c1085c0 2024-04-17T07:33:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.560+0000 7feb92800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c114020 con 0x7feb8c1097b0 2024-04-17T07:33:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.560+0000 7feb92800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c114320 con 0x7feb8c10a170 2024-04-17T07:33:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.560+0000 7feb8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb8c1085c0 0x7feb8c0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.560+0000 7feb8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb8c1085c0 0x7feb8c0709d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47884/0 (socket says 172.21.15.86:47884) 2024-04-17T07:33:27.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.560+0000 7feb8b400640 1 -- 172.21.15.86:0/1140245412 learned_addr learned my addr 172.21.15.86:0/1140245412 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.560+0000 7feb8aa00640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb8c1097b0 0x7feb8c070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.561+0000 7feb8b400640 1 -- 172.21.15.86:0/1140245412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb8c10a170 msgr2=0x7feb8c1ae8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T07:33:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.561+0000 7feb8b400640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb8c10a170 0x7feb8c1ae8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.561+0000 7feb8b400640 1 -- 172.21.15.86:0/1140245412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb8c1097b0 msgr2=0x7feb8c070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.561+0000 7feb8b400640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb8c1097b0 0x7feb8c070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.561+0000 7feb8b400640 1 -- 172.21.15.86:0/1140245412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8c1aedf0 con 0x7feb8c1085c0 2024-04-17T07:33:27.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.562+0000 7feb8b400640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb8c1085c0 0x7feb8c0709d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7feb8400ebc0 tx=0x7feb8400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.562+0000 7feb83e00640 1 -- 172.21.15.86:0/1140245412 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84003b70 con 0x7feb8c1085c0 2024-04-17T07:33:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.562+0000 7feb83e00640 1 -- 172.21.15.86:0/1140245412 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb84003d10 con 0x7feb8c1085c0 2024-04-17T07:33:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.563+0000 7feb83e00640 1 -- 172.21.15.86:0/1140245412 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84019840 con 0x7feb8c1085c0 2024-04-17T07:33:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.564+0000 7feb92800640 1 -- 172.21.15.86:0/1140245412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8c1aefc0 con 0x7feb8c1085c0 2024-04-17T07:33:27.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.564+0000 7feb92800640 1 -- 172.21.15.86:0/1140245412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb8c1af3f0 con 0x7feb8c1085c0 2024-04-17T07:33:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.570+0000 7feb955ec740 1 Processor -- start 2024-04-17T07:33:27.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.570+0000 7feb955ec740 1 -- start start 2024-04-17T07:33:27.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.570+0000 7feb83e00640 1 -- 172.21.15.86:0/1140245412 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7feb840199e0 con 0x7feb8c1085c0 2024-04-17T07:33:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.572+0000 7feb955ec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba49f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.572+0000 7feb955ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49fb80 0x55e5ba4a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.572+0000 7feb955ec740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.572+0000 7feb955ec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5ba406120 con 0x55e5ba49f190 2024-04-17T07:33:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.572+0000 7feb955ec740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e5ba3ac340 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.572+0000 7feb955ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e5ba4020e0 con 0x55e5ba49fb80 2024-04-17T07:33:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.573+0000 7feb83e00640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb700a3db0 0x7feb700a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.573+0000 7feb83e00640 1 -- 172.21.15.86:0/1140245412 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feb840d10a0 con 0x7feb8c1085c0 2024-04-17T07:33:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.573+0000 7feb8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba49f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.573+0000 7feb8aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.573+0000 7feb8aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39010/0 (socket says 172.21.15.86:39010) 2024-04-17T07:33:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.573+0000 7feb8aa00640 1 -- 172.21.15.86:0/751272201 learned_addr learned my addr 172.21.15.86:0/751272201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.573+0000 7feb8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba49f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47896/0 (socket says 172.21.15.86:47896) 2024-04-17T07:33:27.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.574+0000 7feb8aa00640 1 -- 172.21.15.86:0/751272201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49fb80 msgr2=0x55e5ba4a4680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:33:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.574+0000 7feb8aa00640 1 --2- 172.21.15.86:0/751272201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49fb80 0x55e5ba4a4680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.574+0000 7feb8aa00640 1 -- 172.21.15.86:0/751272201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 msgr2=0x55e5ba49f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.574+0000 7feb8aa00640 1 --2- 172.21.15.86:0/751272201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba49f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.574+0000 7feb8aa00640 1 -- 172.21.15.86:0/751272201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ba1e27d0 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.574+0000 7feb8be00640 1 --2- 172.21.15.86:0/751272201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba49f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:33:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.574+0000 7feb8be00640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb700a3db0 0x7feb700a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.574+0000 7feb8aa00640 1 --2- 172.21.15.86:0/751272201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a7180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feb78009230 tx=0x7feb7802c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e4d61c4fb66c4bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.575+0000 7feb80c00640 1 -- 172.21.15.86:0/751272201 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78045020 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.575+0000 7feb80c00640 1 -- 172.21.15.86:0/751272201 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb7802ca60 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.575+0000 7feb80c00640 1 -- 172.21.15.86:0/751272201 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780046c0 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.576+0000 7feb955ec740 1 -- 172.21.15.86:0/751272201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 msgr2=0x55e5ba4a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.576+0000 7feb8be00640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb700a3db0 0x7feb700a6270 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7feb7c008fd0 tx=0x7feb7c00a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.576+0000 7feb955ec740 1 --2- 172.21.15.86:0/751272201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a7180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feb78009230 tx=0x7feb7802c8f0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.577+0000 7feb955ec740 1 -- 172.21.15.86:0/751272201 shutdown_connections 2024-04-17T07:33:27.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.577+0000 7feb955ec740 1 --2- 172.21.15.86:0/751272201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a7180 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.577+0000 7feb955ec740 1 --2- 172.21.15.86:0/751272201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49fb80 0x55e5ba4a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.577+0000 7feb955ec740 1 --2- 172.21.15.86:0/751272201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba49f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.577+0000 7feb955ec740 1 -- 172.21.15.86:0/751272201 >> 172.21.15.86:0/751272201 conn(0x55e5ba49a700 msgr2=0x55e5ba49ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.577+0000 7feb955ec740 1 -- 172.21.15.86:0/751272201 shutdown_connections 2024-04-17T07:33:27.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.577+0000 7feb955ec740 1 -- 172.21.15.86:0/751272201 wait complete. 2024-04-17T07:33:27.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.578+0000 7feb955ec740 1 -- start start 2024-04-17T07:33:27.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.578+0000 7feb955ec740 1 Processor -- start 2024-04-17T07:33:27.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.579+0000 7feb955ec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.579+0000 7feb955ec740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba49fb80 0x55e5ba4a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.579+0000 7feb955ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.579+0000 7feb955ec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5ba4a7c40 con 0x55e5ba49f190 2024-04-17T07:33:27.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:27 smithi086 ceph-mon[31733]: pgmap v1012: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-04-17T07:33:27.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:27 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:33:27.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:27 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:33:27.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:27 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:33:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.579+0000 7feb955ec740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e5ba3bf710 con 0x55e5ba49fb80 2024-04-17T07:33:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.579+0000 7feb955ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e5ba4a7dc0 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.579+0000 7feb8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.579+0000 7feb8be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.580+0000 7feb8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4a96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47912/0 (socket says 172.21.15.86:47912) 2024-04-17T07:33:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.580+0000 7feb8b400640 1 -- 172.21.15.86:0/1534914276 learned_addr learned my addr 172.21.15.86:0/1534914276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.580+0000 7feb955ec740 10 client.0 _ll_register_callbacks cb 0x55e5ba3baaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.579+0000 7feb8aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba49fb80 0x55e5ba4a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.580+0000 7feb8b400640 1 -- 172.21.15.86:0/1534914276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba4a4cf0 msgr2=0x55e5ba4b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.580+0000 7feb8b400640 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.580+0000 7feb8b400640 1 -- 172.21.15.86:0/1534914276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba49fb80 msgr2=0x55e5ba4a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.580+0000 7feb8b400640 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba49fb80 0x55e5ba4a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.580+0000 7feb8b400640 1 -- 172.21.15.86:0/1534914276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ba1183f0 con 0x55e5ba49f190 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.581+0000 7feb8b400640 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4a96e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7feb84009270 tx=0x7feb8409cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99990cb437c072c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.581+0000 7feb75600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.581+0000 7feb75600640 1 -- 172.21.15.86:0/1534914276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb840b5070 con 0x55e5ba49f190 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.581+0000 7feb75600640 1 -- 172.21.15.86:0/1534914276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feb840b6270 con 0x55e5ba49f190 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.581+0000 7feb955ec740 1 -- 172.21.15.86:0/1534914276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb7802ca60 con 0x55e5ba49f190 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.582+0000 7feb75600640 1 -- 172.21.15.86:0/1534914276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb840a5e60 con 0x55e5ba49f190 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.582+0000 7feb955ec740 10 client.19758 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.582+0000 7feb955ec740 1 -- 172.21.15.86:0/1534914276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5ba0d80f0 con 0x55e5ba49f190 2024-04-17T07:33:27.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.583+0000 7feb75600640 1 -- 172.21.15.86:0/1534914276 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb7802ca60 con 0x55e5ba49f190 2024-04-17T07:33:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.583+0000 7feb75600640 1 -- 172.21.15.86:0/1534914276 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7feb84004c70 con 0x55e5ba49f190 2024-04-17T07:33:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.583+0000 7feb955ec740 10 client.19758 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.583+0000 7feb955ec740 10 client.19758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:33:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.583+0000 7feb955ec740 20 client.19758 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.583+0000 7feb955ec740 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e5ba4b59d0 0x55e5ba4b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.583+0000 7feb955ec740 4 client.19758 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:27.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.583+0000 7feb955ec740 1 -- 172.21.15.86:0/1534914276 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5ba413b40 con 0x55e5ba4b59d0 2024-04-17T07:33:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.585+0000 7feb8aa00640 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e5ba4b59d0 0x55e5ba4b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.586+0000 7feb8aa00640 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e5ba4b59d0 0x55e5ba4b7e40 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cf635283c633ef server_cookie=8141be2b0153498f in_seq=0 out_seq=0 2024-04-17T07:33:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.586+0000 7feb75600640 10 client.19758 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.587+0000 7feb75600640 1 -- 172.21.15.86:0/1534914276 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5ba413b40 con 0x55e5ba4b59d0 2024-04-17T07:33:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.587+0000 7feb75600640 10 client.19758 handle_command_reply: tid=0 2024-04-17T07:33:27.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.587+0000 7feb955ec740 1 client.19758 shutdown 2024-04-17T07:33:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.588+0000 7feb955ec740 1 -- 172.21.15.86:0/1534914276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 msgr2=0x55e5ba4a96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.588+0000 7feb955ec740 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4a96e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7feb84009270 tx=0x7feb8409cba0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.589+0000 7feb955ec740 1 -- 172.21.15.86:0/1534914276 shutdown_connections 2024-04-17T07:33:27.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.589+0000 7feb955ec740 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e5ba4b59d0 0x55e5ba4b7e40 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.589+0000 7feb955ec740 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.589+0000 7feb955ec740 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba49fb80 0x55e5ba4a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.589+0000 7feb955ec740 1 --2- 172.21.15.86:0/1534914276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4a96e0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.589+0000 7feb75600640 0 client.19758 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:27.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.589+0000 7feb955ec740 1 -- 172.21.15.86:0/1534914276 >> 172.21.15.86:0/1534914276 conn(0x55e5ba49a700 msgr2=0x55e5ba499890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.590+0000 7feb955ec740 1 -- 172.21.15.86:0/1534914276 shutdown_connections 2024-04-17T07:33:27.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.590+0000 7feb955ec740 1 -- 172.21.15.86:0/1534914276 wait complete. 2024-04-17T07:33:27.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.590+0000 7feb955ec740 20 client.19758 trim_cache size 0 max 16384 2024-04-17T07:33:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.644+0000 7feb955ec740 1 Processor -- start 2024-04-17T07:33:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.645+0000 7feb955ec740 1 -- start start 2024-04-17T07:33:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.645+0000 7feb955ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49f190 0x55e5ba4a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.645+0000 7feb955ec740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba49fb80 0x55e5ba4b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.645+0000 7feb955ec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.645+0000 7feb955ec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1b1210 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.645+0000 7feb955ec740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1b2d50 con 0x55e5ba49fb80 2024-04-17T07:33:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.645+0000 7feb955ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1b18e0 con 0x55e5ba49f190 2024-04-17T07:33:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.646+0000 7feb8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.646+0000 7feb8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47922/0 (socket says 172.21.15.86:47922) 2024-04-17T07:33:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.646+0000 7feb8be00640 1 -- 172.21.15.86:0/3497349401 learned_addr learned my addr 172.21.15.86:0/3497349401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.646+0000 7feb8aa00640 1 --2- 172.21.15.86:0/3497349401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba49fb80 0x55e5ba4b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.646+0000 7feb8be00640 1 -- 172.21.15.86:0/3497349401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49f190 msgr2=0x55e5ba4a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.646+0000 7feb8be00640 1 --2- 172.21.15.86:0/3497349401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49f190 0x55e5ba4a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.646+0000 7feb8be00640 1 -- 172.21.15.86:0/3497349401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba49fb80 msgr2=0x55e5ba4b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.646+0000 7feb8be00640 1 --2- 172.21.15.86:0/3497349401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba49fb80 0x55e5ba4b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.646+0000 7feb8be00640 1 -- 172.21.15.86:0/3497349401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5ba494ce0 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.647+0000 7feb8be00640 1 --2- 172.21.15.86:0/3497349401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a9be0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7feb7c006380 tx=0x7feb7c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3bba8c97bb7c0b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.647+0000 7feb77e00640 1 -- 172.21.15.86:0/3497349401 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c004e60 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.648+0000 7feb77e00640 1 -- 172.21.15.86:0/3497349401 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb7c025070 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.648+0000 7feb77e00640 1 -- 172.21.15.86:0/3497349401 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c013e00 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.649+0000 7feb955ec740 1 -- 172.21.15.86:0/3497349401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba4a4cf0 msgr2=0x55e5ba4a9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.649+0000 7feb955ec740 1 --2- 172.21.15.86:0/3497349401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a9be0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7feb7c006380 tx=0x7feb7c010000 comp rx=0 tx=0).stop 2024-04-17T07:33:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.649+0000 7feb955ec740 1 -- 172.21.15.86:0/3497349401 shutdown_connections 2024-04-17T07:33:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.649+0000 7feb955ec740 1 --2- 172.21.15.86:0/3497349401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4a9be0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.649+0000 7feb955ec740 1 --2- 172.21.15.86:0/3497349401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba49fb80 0x55e5ba4b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.649+0000 7feb955ec740 1 --2- 172.21.15.86:0/3497349401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49f190 0x55e5ba4a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.649+0000 7feb955ec740 1 -- 172.21.15.86:0/3497349401 >> 172.21.15.86:0/3497349401 conn(0x55e5ba49a700 msgr2=0x55e5ba4a1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.649+0000 7feb955ec740 1 -- 172.21.15.86:0/3497349401 shutdown_connections 2024-04-17T07:33:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.650+0000 7feb955ec740 1 -- 172.21.15.86:0/3497349401 wait complete. 2024-04-17T07:33:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.650+0000 7feb955ec740 1 -- start start 2024-04-17T07:33:27.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.651+0000 7feb955ec740 1 Processor -- start 2024-04-17T07:33:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.652+0000 7feb955ec740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4ade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.652+0000 7feb955ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49fb80 0x55e5ba4ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.652+0000 7feb955ec740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.652+0000 7feb955ec740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1b2680 con 0x55e5ba49f190 2024-04-17T07:33:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.652+0000 7feb955ec740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1b1fb0 con 0x55e5ba4a4cf0 2024-04-17T07:33:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.652+0000 7feb955ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c082ab0 con 0x55e5ba49fb80 2024-04-17T07:33:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.652+0000 7feb955ec740 10 client.0 _ll_register_callbacks cb 0x7feb840046d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:27.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.653+0000 7feb8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.653+0000 7feb8be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.653+0000 7feb8be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:39046/0 (socket says 172.21.15.86:39046) 2024-04-17T07:33:27.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.653+0000 7feb8be00640 1 -- 172.21.15.86:0/1989362046 learned_addr learned my addr 172.21.15.86:0/1989362046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.653+0000 7feb8aa00640 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49fb80 0x55e5ba4ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.653+0000 7feb8b400640 1 -- 172.21.15.86:0/1989362046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49fb80 msgr2=0x55e5ba4ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.653+0000 7feb8b400640 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49fb80 0x55e5ba4ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.653+0000 7feb8b400640 1 -- 172.21.15.86:0/1989362046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 msgr2=0x55e5ba4af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.653+0000 7feb8b400640 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.653+0000 7feb8b400640 1 -- 172.21.15.86:0/1989362046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ba4af950 con 0x55e5ba49f190 2024-04-17T07:33:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.654+0000 7feb75600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:33:27.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.654+0000 7feb8b400640 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4ade10 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7feb8409cca0 tx=0x7feb840b64a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602ac2e7ae1a784e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.655+0000 7feb75600640 1 -- 172.21.15.86:0/1989362046 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb840ad030 con 0x55e5ba49f190 2024-04-17T07:33:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.655+0000 7feb75600640 1 -- 172.21.15.86:0/1989362046 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7feb84004d70 con 0x55e5ba49f190 2024-04-17T07:33:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.655+0000 7feb955ec740 1 -- 172.21.15.86:0/1989362046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb7c025070 con 0x55e5ba49f190 2024-04-17T07:33:27.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.656+0000 7feb75600640 1 -- 172.21.15.86:0/1989362046 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb840b5070 con 0x55e5ba49f190 2024-04-17T07:33:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.660+0000 7feb955ec740 10 client.19764 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:27.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.660+0000 7feb955ec740 1 -- 172.21.15.86:0/1989362046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5ba4b0cd0 con 0x55e5ba49f190 2024-04-17T07:33:27.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.661+0000 7feb75600640 1 -- 172.21.15.86:0/1989362046 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb7c025070 con 0x55e5ba49f190 2024-04-17T07:33:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.661+0000 7feb75600640 1 -- 172.21.15.86:0/1989362046 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7feb84098dc0 con 0x55e5ba49f190 2024-04-17T07:33:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.663+0000 7feb955ec740 10 client.19764 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.663+0000 7feb955ec740 10 client.19764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:33:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.663+0000 7feb955ec740 20 client.19764 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.663+0000 7feb955ec740 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e5ba51c1d0 0x55e5ba51e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.663+0000 7feb955ec740 4 client.19764 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:33:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.663+0000 7feb955ec740 1 -- 172.21.15.86:0/1989362046 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5ba40d9b0 con 0x55e5ba51c1d0 2024-04-17T07:33:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.665+0000 7feb8aa00640 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e5ba51c1d0 0x55e5ba51e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.666+0000 7feb8aa00640 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e5ba51c1d0 0x55e5ba51e5c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a99242dadc86ca5 server_cookie=169bc9704120bc0 in_seq=0 out_seq=0 2024-04-17T07:33:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.666+0000 7feb75600640 10 client.19764 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.667+0000 7feb75600640 1 -- 172.21.15.86:0/1989362046 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e5ba40d9b0 con 0x55e5ba51c1d0 2024-04-17T07:33:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.668+0000 7feb75600640 10 client.19764 handle_command_reply: tid=0 2024-04-17T07:33:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.668+0000 7feb955ec740 1 client.19764 shutdown 2024-04-17T07:33:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.668+0000 7feb955ec740 1 -- 172.21.15.86:0/1989362046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 msgr2=0x55e5ba4ade10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.669+0000 7feb955ec740 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4ade10 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7feb8409cca0 tx=0x7feb840b64a0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.669+0000 7feb955ec740 1 -- 172.21.15.86:0/1989362046 shutdown_connections 2024-04-17T07:33:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.669+0000 7feb955ec740 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55e5ba51c1d0 0x55e5ba51e5c0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.669+0000 7feb955ec740 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5ba4a4cf0 0x55e5ba4af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.669+0000 7feb955ec740 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5ba49fb80 0x55e5ba4ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.669+0000 7feb955ec740 1 --2- 172.21.15.86:0/1989362046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5ba49f190 0x55e5ba4ade10 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.669+0000 7feb955ec740 1 -- 172.21.15.86:0/1989362046 >> 172.21.15.86:0/1989362046 conn(0x55e5ba49a700 msgr2=0x55e5ba4a96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.670+0000 7feb75600640 0 client.19764 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:27.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.670+0000 7feb955ec740 1 -- 172.21.15.86:0/1989362046 shutdown_connections 2024-04-17T07:33:27.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.670+0000 7feb955ec740 1 -- 172.21.15.86:0/1989362046 wait complete. 2024-04-17T07:33:27.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.671+0000 7feb955ec740 20 client.19764 trim_cache size 0 max 16384 2024-04-17T07:33:27.671 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:33:27.671 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:33:27.672 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:33:27.672 INFO:teuthology.orchestra.run.smithi086.stdout: "c089350c-6728-4db0-9d53-dd5090e15bbe": { 2024-04-17T07:33:27.672 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:33:27.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c089350c-6728-4db0-9d53-dd5090e15bbe", 2024-04-17T07:33:27.672 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:33:27.672 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:33:27.672 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:33:27.672 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:33:27.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.672+0000 7feb92800640 1 -- 172.21.15.86:0/1140245412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb700a3db0 msgr2=0x7feb700a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.672+0000 7feb92800640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb700a3db0 0x7feb700a6270 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7feb7c008fd0 tx=0x7feb7c00a040 comp rx=0 tx=0).stop 2024-04-17T07:33:27.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.672+0000 7feb92800640 1 -- 172.21.15.86:0/1140245412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb8c1085c0 msgr2=0x7feb8c0709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:27.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.672+0000 7feb92800640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb8c1085c0 0x7feb8c0709d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7feb8400ebc0 tx=0x7feb8400c6a0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.673+0000 7feb92800640 1 -- 172.21.15.86:0/1140245412 shutdown_connections 2024-04-17T07:33:27.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.673+0000 7feb92800640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7feb700a3db0 0x7feb700a6270 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.673+0000 7feb92800640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb8c10a170 0x7feb8c1ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.673+0000 7feb92800640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb8c1097b0 0x7feb8c070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.673+0000 7feb92800640 1 --2- 172.21.15.86:0/1140245412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb8c1085c0 0x7feb8c0709d0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:27.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.673+0000 7feb92800640 1 -- 172.21.15.86:0/1140245412 >> 172.21.15.86:0/1140245412 conn(0x7feb8c077560 msgr2=0x7feb8c10cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:27.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.676+0000 7feb92800640 1 -- 172.21.15.86:0/1140245412 shutdown_connections 2024-04-17T07:33:27.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:27.676+0000 7feb92800640 1 -- 172.21.15.86:0/1140245412 wait complete. 2024-04-17T07:33:27.716 INFO:tasks.cephfs.filesystem:scrub status for tag:c089350c-6728-4db0-9d53-dd5090e15bbe - {'path': '/', 'tag': 'c089350c-6728-4db0-9d53-dd5090e15bbe', 'options': 'recursive,force'} 2024-04-17T07:33:28.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:28 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:33:28.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:28 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:33:28.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:28 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:33:29.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 845.0 s, 581.6 tps, lat 169.699 ms stddev 97.186 2024-04-17T07:33:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:29 smithi098 ceph-mon[35418]: pgmap v1013: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 591 op/s 2024-04-17T07:33:29.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:29 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:33:29.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:29 smithi103 ceph-mon[35495]: pgmap v1013: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 591 op/s 2024-04-17T07:33:29.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:29 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:33:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:29 smithi086 ceph-mon[31733]: pgmap v1013: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 591 op/s 2024-04-17T07:33:29.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:29 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:33:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:31 smithi098 ceph-mon[35418]: pgmap v1014: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.8 MiB/s wr, 562 op/s 2024-04-17T07:33:31.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:31 smithi103 ceph-mon[35495]: pgmap v1014: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.8 MiB/s wr, 562 op/s 2024-04-17T07:33:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:31 smithi086 ceph-mon[31733]: pgmap v1014: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.8 MiB/s wr, 562 op/s 2024-04-17T07:33:33.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:33 smithi098 ceph-mon[35418]: pgmap v1015: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.2 MiB/s wr, 587 op/s 2024-04-17T07:33:33.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:33 smithi103 ceph-mon[35495]: pgmap v1015: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.2 MiB/s wr, 587 op/s 2024-04-17T07:33:33.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:33 smithi086 ceph-mon[31733]: pgmap v1015: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.2 MiB/s wr, 587 op/s 2024-04-17T07:33:34.361 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 850.0 s, 572.0 tps, lat 177.903 ms stddev 119.566 2024-04-17T07:33:35.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:35 smithi103 ceph-mon[35495]: pgmap v1016: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 538 op/s 2024-04-17T07:33:35.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:35 smithi086 ceph-mon[31733]: pgmap v1016: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 538 op/s 2024-04-17T07:33:35.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:35 smithi098 ceph-mon[35418]: pgmap v1016: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.6 MiB/s wr, 538 op/s 2024-04-17T07:33:37.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:37 smithi103 ceph-mon[35495]: pgmap v1017: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 600 op/s 2024-04-17T07:33:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:37 smithi086 ceph-mon[31733]: pgmap v1017: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 600 op/s 2024-04-17T07:33:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:37 smithi098 ceph-mon[35418]: pgmap v1017: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 600 op/s 2024-04-17T07:33:39.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 855.0 s, 618.8 tps, lat 160.075 ms stddev 63.457 2024-04-17T07:33:39.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:39 smithi103 ceph-mon[35495]: pgmap v1018: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.3 MiB/s wr, 602 op/s 2024-04-17T07:33:39.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:39 smithi086 ceph-mon[31733]: pgmap v1018: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.3 MiB/s wr, 602 op/s 2024-04-17T07:33:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:39 smithi098 ceph-mon[35418]: pgmap v1018: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.3 MiB/s wr, 602 op/s 2024-04-17T07:33:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:41 smithi086 ceph-mon[31733]: pgmap v1019: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-04-17T07:33:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:41 smithi098 ceph-mon[35418]: pgmap v1019: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-04-17T07:33:41.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:41 smithi103 ceph-mon[35495]: pgmap v1019: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 558 op/s 2024-04-17T07:33:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:43 smithi086 ceph-mon[31733]: pgmap v1020: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.3 MiB/s wr, 592 op/s 2024-04-17T07:33:43.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:43 smithi098 ceph-mon[35418]: pgmap v1020: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.3 MiB/s wr, 592 op/s 2024-04-17T07:33:43.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:43 smithi103 ceph-mon[35495]: pgmap v1020: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.3 MiB/s wr, 592 op/s 2024-04-17T07:33:44.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 860.0 s, 571.4 tps, lat 173.260 ms stddev 95.875 2024-04-17T07:33:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:45 smithi086 ceph-mon[31733]: pgmap v1021: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.6 MiB/s wr, 553 op/s 2024-04-17T07:33:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:45 smithi098 ceph-mon[35418]: pgmap v1021: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.6 MiB/s wr, 553 op/s 2024-04-17T07:33:45.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:45 smithi103 ceph-mon[35495]: pgmap v1021: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.6 MiB/s wr, 553 op/s 2024-04-17T07:33:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:47 smithi086 ceph-mon[31733]: pgmap v1022: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 600 op/s 2024-04-17T07:33:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:47 smithi098 ceph-mon[35418]: pgmap v1022: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 600 op/s 2024-04-17T07:33:47.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:47 smithi103 ceph-mon[35495]: pgmap v1022: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.9 MiB/s wr, 600 op/s 2024-04-17T07:33:49.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 865.0 s, 571.4 tps, lat 178.129 ms stddev 143.223 2024-04-17T07:33:49.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:49 smithi086 ceph-mon[31733]: pgmap v1023: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 607 op/s 2024-04-17T07:33:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:49 smithi098 ceph-mon[35418]: pgmap v1023: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 607 op/s 2024-04-17T07:33:49.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:49 smithi103 ceph-mon[35495]: pgmap v1023: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.1 MiB/s wr, 607 op/s 2024-04-17T07:33:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:51 smithi086 ceph-mon[31733]: pgmap v1024: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 592 op/s 2024-04-17T07:33:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:51 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:33:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:51 smithi098 ceph-mon[35418]: pgmap v1024: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 592 op/s 2024-04-17T07:33:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:51 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:33:51.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:51 smithi103 ceph-mon[35495]: pgmap v1024: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 592 op/s 2024-04-17T07:33:51.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:51 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:33:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:53 smithi098 ceph-mon[35418]: pgmap v1025: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 618 op/s 2024-04-17T07:33:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:53 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:53 smithi103 ceph-mon[35495]: pgmap v1025: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 618 op/s 2024-04-17T07:33:53.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.528 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:53 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:53 smithi086 ceph-mon[31733]: pgmap v1025: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 618 op/s 2024-04-17T07:33:53.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:53.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:53 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 870.0 s, 568.4 tps, lat 173.599 ms stddev 73.672 2024-04-17T07:33:54.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:54 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:54.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:54 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:54 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:55.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:54 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:55.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:54 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:55.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:54 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:55.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:55 smithi086 ceph-mon[31733]: pgmap v1026: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 568 op/s 2024-04-17T07:33:55.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:55.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:55.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:33:55.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:33:55.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:55 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:56.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:55 smithi098 ceph-mon[35418]: pgmap v1026: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 568 op/s 2024-04-17T07:33:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:33:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:33:56.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:55 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:56.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:55 smithi103 ceph-mon[35495]: pgmap v1026: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 568 op/s 2024-04-17T07:33:56.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:56.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:56.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:33:56.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:33:56.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:55 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:33:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:56 smithi086 ceph-mon[31733]: pgmap v1027: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 607 op/s 2024-04-17T07:33:57.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:56 smithi098 ceph-mon[35418]: pgmap v1027: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 607 op/s 2024-04-17T07:33:57.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:56 smithi103 ceph-mon[35495]: pgmap v1027: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 607 op/s 2024-04-17T07:33:57.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:33:57.717 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:33:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.037+0000 7f6136000640 1 -- 172.21.15.86:0/1635007629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6130105d90 msgr2=0x7f613010c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.037+0000 7f6136000640 1 --2- 172.21.15.86:0/1635007629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6130105d90 0x7f613010c620 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6128009ca0 tx=0x7f612802c100 comp rx=0 tx=0).stop 2024-04-17T07:33:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.037+0000 7f6136000640 1 -- 172.21.15.86:0/1635007629 shutdown_connections 2024-04-17T07:33:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.037+0000 7f6136000640 1 --2- 172.21.15.86:0/1635007629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6130105d90 0x7f613010c620 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.037+0000 7f6136000640 1 --2- 172.21.15.86:0/1635007629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61301053d0 0x7f6130105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.037+0000 7f6136000640 1 --2- 172.21.15.86:0/1635007629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61301041e0 0x7f6130104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.037+0000 7f6136000640 1 -- 172.21.15.86:0/1635007629 >> 172.21.15.86:0/1635007629 conn(0x7f61300ff750 msgr2=0x7f6130101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.038+0000 7f6136000640 1 -- 172.21.15.86:0/1635007629 shutdown_connections 2024-04-17T07:33:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.038+0000 7f6136000640 1 -- 172.21.15.86:0/1635007629 wait complete. 2024-04-17T07:33:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.039+0000 7f6136000640 1 Processor -- start 2024-04-17T07:33:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.040+0000 7f6136000640 1 -- start start 2024-04-17T07:33:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.040+0000 7f6136000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61301041e0 0x7f6130079d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.040+0000 7f6136000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61301053d0 0x7f613007a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.040+0000 7f6136000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6130105d90 0x7f6130076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.040+0000 7f6136000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f613010fd20 con 0x7f6130105d90 2024-04-17T07:33:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.040+0000 7f6136000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f613010fba0 con 0x7f61301041e0 2024-04-17T07:33:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.040+0000 7f6136000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f613010fea0 con 0x7f61301053d0 2024-04-17T07:33:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.041+0000 7f612fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6130105d90 0x7f6130076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.041+0000 7f612fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6130105d90 0x7f6130076880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38692/0 (socket says 172.21.15.86:38692) 2024-04-17T07:33:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.041+0000 7f612fe00640 1 -- 172.21.15.86:0/2222788148 learned_addr learned my addr 172.21.15.86:0/2222788148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.041+0000 7f612f400640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61301041e0 0x7f6130079d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.042+0000 7f612fe00640 1 -- 172.21.15.86:0/2222788148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61301053d0 msgr2=0x7f613007a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.042+0000 7f612fe00640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61301053d0 0x7f613007a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.042+0000 7f612fe00640 1 -- 172.21.15.86:0/2222788148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61301041e0 msgr2=0x7f6130079d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.042+0000 7f612fe00640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61301041e0 0x7f6130079d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.042+0000 7f612fe00640 1 -- 172.21.15.86:0/2222788148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6130077140 con 0x7f6130105d90 2024-04-17T07:33:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.043+0000 7f612fe00640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6130105d90 0x7f6130076880 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f6128004af0 tx=0x7f6128003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.043+0000 7f6127e00640 1 -- 172.21.15.86:0/2222788148 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128039040 con 0x7f6130105d90 2024-04-17T07:33:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.043+0000 7f6127e00640 1 -- 172.21.15.86:0/2222788148 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6128044070 con 0x7f6130105d90 2024-04-17T07:33:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.043+0000 7f6127e00640 1 -- 172.21.15.86:0/2222788148 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280329e0 con 0x7f6130105d90 2024-04-17T07:33:58.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.045+0000 7f6136000640 1 -- 172.21.15.86:0/2222788148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61301a6830 con 0x7f6130105d90 2024-04-17T07:33:58.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.046+0000 7f6136000640 1 -- 172.21.15.86:0/2222788148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61301a6be0 con 0x7f6130105d90 2024-04-17T07:33:58.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.051+0000 7f6127e00640 1 -- 172.21.15.86:0/2222788148 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f6128004580 con 0x7f6130105d90 2024-04-17T07:33:58.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.052+0000 7f6138c44740 1 Processor -- start 2024-04-17T07:33:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.053+0000 7f6138c44740 1 -- start start 2024-04-17T07:33:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.055+0000 7f6138c44740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cf190 0x5651041cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.055+0000 7f6138c44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cfb80 0x5651041d4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.055+0000 7f6138c44740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 0x5651041d7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.055+0000 7f6138c44740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565104136120 con 0x5651041cf190 2024-04-17T07:33:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.055+0000 7f6138c44740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5651040dc340 con 0x5651041d4cf0 2024-04-17T07:33:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.055+0000 7f6138c44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651041320e0 con 0x5651041cfb80 2024-04-17T07:33:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.056+0000 7f6127e00640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61140a3df0 0x7f61140a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.057+0000 7f612f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cfb80 0x5651041d4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.057+0000 7f612f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cfb80 0x5651041d4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:53866/0 (socket says 172.21.15.86:53866) 2024-04-17T07:33:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.057+0000 7f612f400640 1 -- 172.21.15.86:0/1161755901 learned_addr learned my addr 172.21.15.86:0/1161755901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.057+0000 7f612ea00640 1 --2- 172.21.15.86:0/1161755901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cf190 0x5651041cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.057+0000 7f612f400640 1 -- 172.21.15.86:0/1161755901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 msgr2=0x5651041d7180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:33:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.057+0000 7f612f400640 1 --2- 172.21.15.86:0/1161755901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 0x5651041d7180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.057+0000 7f612f400640 1 -- 172.21.15.86:0/1161755901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cf190 msgr2=0x5651041cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.057+0000 7f612f400640 1 --2- 172.21.15.86:0/1161755901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cf190 0x5651041cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.057+0000 7f612f400640 1 -- 172.21.15.86:0/1161755901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565103f127d0 con 0x5651041cfb80 2024-04-17T07:33:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.058+0000 7f6127e00640 1 -- 172.21.15.86:0/2222788148 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6128032780 con 0x7f6130105d90 2024-04-17T07:33:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.058+0000 7f612f400640 1 --2- 172.21.15.86:0/1161755901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cfb80 0x5651041d4680 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f612000bc90 tx=0x7f612000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c5514403a2114c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.058+0000 7f611be00640 1 -- 172.21.15.86:0/1161755901 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120015070 con 0x5651041cfb80 2024-04-17T07:33:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.058+0000 7f611be00640 1 -- 172.21.15.86:0/1161755901 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61200035f0 con 0x5651041cfb80 2024-04-17T07:33:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.058+0000 7f611be00640 1 -- 172.21.15.86:0/1161755901 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61200116a0 con 0x5651041cfb80 2024-04-17T07:33:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.058+0000 7f612ea00640 1 --2- 172.21.15.86:0/1161755901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cf190 0x5651041cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:33:58.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.059+0000 7f612fe00640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61140a3df0 0x7f61140a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.060+0000 7f6138c44740 1 -- 172.21.15.86:0/1161755901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cfb80 msgr2=0x5651041d4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.060+0000 7f6138c44740 1 --2- 172.21.15.86:0/1161755901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cfb80 0x5651041d4680 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f612000bc90 tx=0x7f612000f730 comp rx=0 tx=0).stop 2024-04-17T07:33:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.061+0000 7f612fe00640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61140a3df0 0x7f61140a62b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6128037040 tx=0x7f61280095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.061+0000 7f6138c44740 1 -- 172.21.15.86:0/1161755901 shutdown_connections 2024-04-17T07:33:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.061+0000 7f6138c44740 1 --2- 172.21.15.86:0/1161755901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 0x5651041d7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.061+0000 7f6138c44740 1 --2- 172.21.15.86:0/1161755901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cfb80 0x5651041d4680 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.061+0000 7f6138c44740 1 --2- 172.21.15.86:0/1161755901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cf190 0x5651041cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.061+0000 7f6138c44740 1 -- 172.21.15.86:0/1161755901 >> 172.21.15.86:0/1161755901 conn(0x5651041ca700 msgr2=0x5651041cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.062+0000 7f6138c44740 1 -- 172.21.15.86:0/1161755901 shutdown_connections 2024-04-17T07:33:58.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.062+0000 7f6138c44740 1 -- 172.21.15.86:0/1161755901 wait complete. 2024-04-17T07:33:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.062+0000 7f6138c44740 1 -- start start 2024-04-17T07:33:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.063+0000 7f6138c44740 1 Processor -- start 2024-04-17T07:33:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.063+0000 7f6138c44740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041cf190 0x5651041ea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.063+0000 7f6138c44740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cfb80 0x5651041eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.063+0000 7f6138c44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041d4cf0 0x5651041f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.063+0000 7f6138c44740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651041d7ca0 con 0x5651041cfb80 2024-04-17T07:33:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.063+0000 7f6138c44740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5651040ef710 con 0x5651041cf190 2024-04-17T07:33:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.063+0000 7f6138c44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651041d7e20 con 0x5651041d4cf0 2024-04-17T07:33:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.064+0000 7f6138c44740 10 client.0 _ll_register_callbacks cb 0x5651040eaaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:58.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.064+0000 7f612ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041d4cf0 0x5651041f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.065+0000 7f612ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041d4cf0 0x5651041f1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:53870/0 (socket says 172.21.15.86:53870) 2024-04-17T07:33:58.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.065+0000 7f612ea00640 1 -- 172.21.15.86:0/759037106 learned_addr learned my addr 172.21.15.86:0/759037106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:58.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.066+0000 7f612f400640 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cfb80 0x5651041eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.066+0000 7f612ea00640 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041cf190 0x5651041ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.068+0000 7f612f400640 1 -- 172.21.15.86:0/759037106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041d4cf0 msgr2=0x5651041f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.068+0000 7f612f400640 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041d4cf0 0x5651041f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.068+0000 7f612f400640 1 -- 172.21.15.86:0/759037106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041cf190 msgr2=0x5651041ea740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.068+0000 7f612f400640 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041cf190 0x5651041ea740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.068+0000 7f612f400640 1 -- 172.21.15.86:0/759037106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565103e483f0 con 0x5651041cfb80 2024-04-17T07:33:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.069+0000 7f612f400640 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cfb80 0x5651041eac80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x5651041cb670 tx=0x7f61200039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dbb5f05da07a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:58.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.070+0000 7f6118c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:33:58.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.071+0000 7f6118c00640 1 -- 172.21.15.86:0/759037106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120015030 con 0x5651041cfb80 2024-04-17T07:33:58.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.071+0000 7f6118c00640 1 -- 172.21.15.86:0/759037106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6120004000 con 0x5651041cfb80 2024-04-17T07:33:58.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.071+0000 7f6138c44740 1 -- 172.21.15.86:0/759037106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61200035f0 con 0x5651041cfb80 2024-04-17T07:33:58.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.072+0000 7f6118c00640 1 -- 172.21.15.86:0/759037106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6120004200 con 0x5651041cfb80 2024-04-17T07:33:58.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.072+0000 7f6138c44740 10 client.19776 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:58.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.072+0000 7f6138c44740 1 -- 172.21.15.86:0/759037106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565103e080f0 con 0x5651041cfb80 2024-04-17T07:33:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.074+0000 7f6118c00640 1 -- 172.21.15.86:0/759037106 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61200035f0 con 0x5651041cfb80 2024-04-17T07:33:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.074+0000 7f6118c00640 1 -- 172.21.15.86:0/759037106 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6120004960 con 0x5651041cfb80 2024-04-17T07:33:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.074+0000 7f6138c44740 10 client.19776 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:58.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.074+0000 7f6138c44740 10 client.19776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:33:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.074+0000 7f6138c44740 20 client.19776 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.074+0000 7f6138c44740 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5651041f62b0 0x5651041f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.074+0000 7f6138c44740 4 client.19776 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.074+0000 7f6138c44740 1 -- 172.21.15.86:0/759037106 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565104143b40 con 0x5651041f62b0 2024-04-17T07:33:58.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.075+0000 7f612ea00640 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5651041f62b0 0x5651041f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.076+0000 7f612ea00640 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5651041f62b0 0x5651041f86e0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b93ae83ab10e34 server_cookie=deb47aea3b14fc37 in_seq=0 out_seq=0 2024-04-17T07:33:58.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.077+0000 7f6118c00640 10 client.19776 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.078+0000 7f6118c00640 1 -- 172.21.15.86:0/759037106 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565104143b40 con 0x5651041f62b0 2024-04-17T07:33:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.078+0000 7f6118c00640 10 client.19776 handle_command_reply: tid=0 2024-04-17T07:33:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.079+0000 7f6138c44740 1 client.19776 shutdown 2024-04-17T07:33:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.079+0000 7f6138c44740 1 -- 172.21.15.86:0/759037106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cfb80 msgr2=0x5651041eac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.079+0000 7f6138c44740 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cfb80 0x5651041eac80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x5651041cb670 tx=0x7f61200039d0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.079+0000 7f6138c44740 1 -- 172.21.15.86:0/759037106 shutdown_connections 2024-04-17T07:33:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.079+0000 7f6138c44740 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x5651041f62b0 0x5651041f86e0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.079+0000 7f6138c44740 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041d4cf0 0x5651041f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.079+0000 7f6138c44740 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041cfb80 0x5651041eac80 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.079+0000 7f6138c44740 1 --2- 172.21.15.86:0/759037106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041cf190 0x5651041ea740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.079+0000 7f6118c00640 0 client.19776 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.079+0000 7f6138c44740 1 -- 172.21.15.86:0/759037106 >> 172.21.15.86:0/759037106 conn(0x5651041ca700 msgr2=0x5651041c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.080+0000 7f6138c44740 1 -- 172.21.15.86:0/759037106 shutdown_connections 2024-04-17T07:33:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.082+0000 7f6138c44740 1 -- 172.21.15.86:0/759037106 wait complete. 2024-04-17T07:33:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.082+0000 7f6138c44740 20 client.19776 trim_cache size 0 max 16384 2024-04-17T07:33:58.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.118+0000 7f6138c44740 1 Processor -- start 2024-04-17T07:33:58.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.118+0000 7f6138c44740 1 -- start start 2024-04-17T07:33:58.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.119+0000 7f6138c44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cf190 0x5651041d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.119+0000 7f6138c44740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d4cf0 0x5651041f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.119+0000 7f6138c44740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d1820 0x5651041c9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.119+0000 7f6138c44740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61301a8960 con 0x5651041d4cf0 2024-04-17T07:33:58.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.119+0000 7f6138c44740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f61301aa4a0 con 0x5651041d1820 2024-04-17T07:33:58.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.119+0000 7f6138c44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61301a9030 con 0x5651041cf190 2024-04-17T07:33:58.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.119+0000 7f612ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cf190 0x5651041d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.119+0000 7f612ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cf190 0x5651041d8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:53886/0 (socket says 172.21.15.86:53886) 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.119+0000 7f612ea00640 1 -- 172.21.15.86:0/3180557226 learned_addr learned my addr 172.21.15.86:0/3180557226 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.119+0000 7f612f400640 1 --2- 172.21.15.86:0/3180557226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d4cf0 0x5651041f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.120+0000 7f612ea00640 1 --2- 172.21.15.86:0/3180557226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d1820 0x5651041c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.120+0000 7f612ea00640 1 -- 172.21.15.86:0/3180557226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d1820 msgr2=0x5651041c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.120+0000 7f612ea00640 1 --2- 172.21.15.86:0/3180557226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d1820 0x5651041c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.120+0000 7f612ea00640 1 -- 172.21.15.86:0/3180557226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d4cf0 msgr2=0x5651041f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.120+0000 7f612ea00640 1 --2- 172.21.15.86:0/3180557226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d4cf0 0x5651041f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.120+0000 7f612ea00640 1 -- 172.21.15.86:0/3180557226 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651041cb470 con 0x5651041cf190 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.120+0000 7f612f400640 1 --2- 172.21.15.86:0/3180557226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d4cf0 0x5651041f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:33:58.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.121+0000 7f612ea00640 1 --2- 172.21.15.86:0/3180557226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cf190 0x5651041d8220 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x5651041f2280 tx=0x7f611c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee314f9c9601df33 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:58.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.121+0000 7f611b400640 1 -- 172.21.15.86:0/3180557226 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c01b040 con 0x5651041cf190 2024-04-17T07:33:58.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.121+0000 7f611b400640 1 -- 172.21.15.86:0/3180557226 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f611c005020 con 0x5651041cf190 2024-04-17T07:33:58.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.121+0000 7f611b400640 1 -- 172.21.15.86:0/3180557226 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c004a00 con 0x5651041cf190 2024-04-17T07:33:58.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.122+0000 7f6138c44740 1 -- 172.21.15.86:0/3180557226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cf190 msgr2=0x5651041d8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.122+0000 7f6138c44740 1 --2- 172.21.15.86:0/3180557226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cf190 0x5651041d8220 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x5651041f2280 tx=0x7f611c0046c0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.122+0000 7f6138c44740 1 -- 172.21.15.86:0/3180557226 shutdown_connections 2024-04-17T07:33:58.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.122+0000 7f6138c44740 1 --2- 172.21.15.86:0/3180557226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d1820 0x5651041c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.122+0000 7f6138c44740 1 --2- 172.21.15.86:0/3180557226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d4cf0 0x5651041f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.122+0000 7f6138c44740 1 --2- 172.21.15.86:0/3180557226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041cf190 0x5651041d8220 secure :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0x5651041f2280 tx=0x7f611c0046c0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.122+0000 7f6138c44740 1 -- 172.21.15.86:0/3180557226 >> 172.21.15.86:0/3180557226 conn(0x5651041ca700 msgr2=0x5651041d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.123+0000 7f6138c44740 1 -- 172.21.15.86:0/3180557226 shutdown_connections 2024-04-17T07:33:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.123+0000 7f6138c44740 1 -- 172.21.15.86:0/3180557226 wait complete. 2024-04-17T07:33:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.123+0000 7f6138c44740 1 -- start start 2024-04-17T07:33:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.123+0000 7f6138c44740 1 Processor -- start 2024-04-17T07:33:58.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.124+0000 7f6138c44740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d1820 0x5651041ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.124+0000 7f6138c44740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 0x5651041efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.124+0000 7f6138c44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041f0d30 0x5651041f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.124+0000 7f6138c44740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61301a9dd0 con 0x5651041d1820 2024-04-17T07:33:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.124+0000 7f6138c44740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f61301a9700 con 0x5651041d4cf0 2024-04-17T07:33:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.124+0000 7f6138c44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6130073d10 con 0x5651041f0d30 2024-04-17T07:33:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.124+0000 7f6138c44740 10 client.0 _ll_register_callbacks cb 0x7f612000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.125+0000 7f612f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 0x5651041efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.125+0000 7f612ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041f0d30 0x5651041f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.125+0000 7f612f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 0x5651041efd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:51986/0 (socket says 172.21.15.86:51986) 2024-04-17T07:33:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.125+0000 7f612ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041f0d30 0x5651041f11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:53890/0 (socket says 172.21.15.86:53890) 2024-04-17T07:33:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.125+0000 7f612f400640 1 -- 172.21.15.86:0/2003594955 learned_addr learned my addr 172.21.15.86:0/2003594955 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.125+0000 7f612ea00640 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d1820 0x5651041ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.126+0000 7f612ea00640 1 -- 172.21.15.86:0/2003594955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041f0d30 msgr2=0x5651041f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.126+0000 7f612ea00640 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041f0d30 0x5651041f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.126+0000 7f612ea00640 1 -- 172.21.15.86:0/2003594955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 msgr2=0x5651041efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.126+0000 7f612ea00640 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 0x5651041efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.126+0000 7f612ea00640 1 -- 172.21.15.86:0/2003594955 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651041f16f0 con 0x5651041d1820 2024-04-17T07:33:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.126+0000 7f612f400640 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 0x5651041efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:33:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.126+0000 7f612ea00640 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d1820 0x5651041ef7e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x5651041d8d40 tx=0x7f611c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c36c89f30820863 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.127+0000 7f6118c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:33:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.127+0000 7f6118c00640 1 -- 172.21.15.86:0/2003594955 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c005250 con 0x5651041d1820 2024-04-17T07:33:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.127+0000 7f6118c00640 1 -- 172.21.15.86:0/2003594955 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f611c0053f0 con 0x5651041d1820 2024-04-17T07:33:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.128+0000 7f6138c44740 1 -- 172.21.15.86:0/2003594955 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611c005020 con 0x5651041d1820 2024-04-17T07:33:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.129+0000 7f6118c00640 1 -- 172.21.15.86:0/2003594955 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f611c0047a0 con 0x5651041d1820 2024-04-17T07:33:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.129+0000 7f6138c44740 10 client.19782 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.129+0000 7f6138c44740 1 -- 172.21.15.86:0/2003594955 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56510425d010 con 0x5651041d1820 2024-04-17T07:33:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.130+0000 7f6118c00640 1 -- 172.21.15.86:0/2003594955 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f611c005020 con 0x5651041d1820 2024-04-17T07:33:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.130+0000 7f6118c00640 1 -- 172.21.15.86:0/2003594955 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f611c011690 con 0x5651041d1820 2024-04-17T07:33:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.130+0000 7f6138c44740 10 client.19782 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.130+0000 7f6138c44740 10 client.19782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:33:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.130+0000 7f6138c44740 20 client.19782 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.130+0000 7f6138c44740 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56510425d6e0 0x56510425fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.130+0000 7f6138c44740 4 client.19782 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:33:58.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.130+0000 7f6138c44740 1 -- 172.21.15.86:0/2003594955 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56510413d9b0 con 0x56510425d6e0 2024-04-17T07:33:58.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.133+0000 7f612f400640 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56510425d6e0 0x56510425fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.135+0000 7f612f400640 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56510425d6e0 0x56510425fb00 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f12242423bf3cb4 server_cookie=a3108df71b7122da in_seq=0 out_seq=0 2024-04-17T07:33:58.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.135+0000 7f6118c00640 10 client.19782 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.136+0000 7f6118c00640 1 -- 172.21.15.86:0/2003594955 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56510413d9b0 con 0x56510425d6e0 2024-04-17T07:33:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.136+0000 7f6118c00640 10 client.19782 handle_command_reply: tid=0 2024-04-17T07:33:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.136+0000 7f6138c44740 1 client.19782 shutdown 2024-04-17T07:33:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.137+0000 7f6138c44740 1 -- 172.21.15.86:0/2003594955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d1820 msgr2=0x5651041ef7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.137+0000 7f6138c44740 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d1820 0x5651041ef7e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x5651041d8d40 tx=0x7f611c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.137+0000 7f6138c44740 1 -- 172.21.15.86:0/2003594955 shutdown_connections 2024-04-17T07:33:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.137+0000 7f6138c44740 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56510425d6e0 0x56510425fb00 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.137+0000 7f6138c44740 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651041f0d30 0x5651041f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.137+0000 7f6138c44740 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5651041d4cf0 0x5651041efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.137+0000 7f6138c44740 1 --2- 172.21.15.86:0/2003594955 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651041d1820 0x5651041ef7e0 secure :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x5651041d8d40 tx=0x7f611c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.137+0000 7f6138c44740 1 -- 172.21.15.86:0/2003594955 >> 172.21.15.86:0/2003594955 conn(0x5651041ca700 msgr2=0x5651041c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.137+0000 7f6118c00640 0 client.19782 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:33:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.138+0000 7f6138c44740 1 -- 172.21.15.86:0/2003594955 shutdown_connections 2024-04-17T07:33:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.138+0000 7f6138c44740 1 -- 172.21.15.86:0/2003594955 wait complete. 2024-04-17T07:33:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.138+0000 7f6138c44740 20 client.19782 trim_cache size 0 max 16384 2024-04-17T07:33:58.139 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:33:58.140 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:33:58.140 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:33:58.140 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:33:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.140+0000 7f6136000640 1 -- 172.21.15.86:0/2222788148 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61140a3df0 msgr2=0x7f61140a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.140+0000 7f6136000640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61140a3df0 0x7f61140a62b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6128037040 tx=0x7f61280095c0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.140+0000 7f6136000640 1 -- 172.21.15.86:0/2222788148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6130105d90 msgr2=0x7f6130076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.140+0000 7f6136000640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6130105d90 0x7f6130076880 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f6128004af0 tx=0x7f6128003ba0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.141+0000 7f6136000640 1 -- 172.21.15.86:0/2222788148 shutdown_connections 2024-04-17T07:33:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.141+0000 7f6136000640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f61140a3df0 0x7f61140a62b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.141+0000 7f6136000640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6130105d90 0x7f6130076880 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.141+0000 7f6136000640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61301053d0 0x7f613007a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.141+0000 7f6136000640 1 --2- 172.21.15.86:0/2222788148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61301041e0 0x7f6130079d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.141+0000 7f6136000640 1 -- 172.21.15.86:0/2222788148 >> 172.21.15.86:0/2222788148 conn(0x7f61300ff750 msgr2=0x7f61301087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:58.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.141+0000 7f6136000640 1 -- 172.21.15.86:0/2222788148 shutdown_connections 2024-04-17T07:33:58.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.141+0000 7f6136000640 1 -- 172.21.15.86:0/2222788148 wait complete. 2024-04-17T07:33:58.174 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:33:58.174 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:33:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.506+0000 7f6114a00640 1 -- 172.21.15.86:0/3691086951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f611010a170 msgr2=0x7f6110110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.506+0000 7f6114a00640 1 --2- 172.21.15.86:0/3691086951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f611010a170 0x7f6110110a00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6108009ca0 tx=0x7f610802c0f0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.506+0000 7f6114a00640 1 -- 172.21.15.86:0/3691086951 shutdown_connections 2024-04-17T07:33:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.506+0000 7f6114a00640 1 --2- 172.21.15.86:0/3691086951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f611010a170 0x7f6110110a00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.506+0000 7f6114a00640 1 --2- 172.21.15.86:0/3691086951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61101097b0 0x7f6110109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.506+0000 7f6114a00640 1 --2- 172.21.15.86:0/3691086951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61101085c0 0x7f61101089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.506+0000 7f6114a00640 1 -- 172.21.15.86:0/3691086951 >> 172.21.15.86:0/3691086951 conn(0x7f6110077580 msgr2=0x7f61100799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.506+0000 7f6114a00640 1 -- 172.21.15.86:0/3691086951 shutdown_connections 2024-04-17T07:33:58.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.507+0000 7f6114a00640 1 -- 172.21.15.86:0/3691086951 wait complete. 2024-04-17T07:33:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.509+0000 7f6114a00640 1 Processor -- start 2024-04-17T07:33:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.509+0000 7f6114a00640 1 -- start start 2024-04-17T07:33:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.510+0000 7f6114a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61101085c0 0x7f611019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.510+0000 7f6114a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61101097b0 0x7f611019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.511+0000 7f6114a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f611010a170 0x7f61101a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.511+0000 7f6114a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61101140d0 con 0x7f61101085c0 2024-04-17T07:33:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.511+0000 7f6114a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6110113f50 con 0x7f61101097b0 2024-04-17T07:33:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.511+0000 7f6114a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6110114250 con 0x7f611010a170 2024-04-17T07:33:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.511+0000 7f610e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f611010a170 0x7f61101a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.511+0000 7f610e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f611010a170 0x7f61101a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:53910/0 (socket says 172.21.15.86:53910) 2024-04-17T07:33:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.511+0000 7f610e800640 1 -- 172.21.15.86:0/1129915687 learned_addr learned my addr 172.21.15.86:0/1129915687 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.511+0000 7f610d400640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61101097b0 0x7f611019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.511+0000 7f610de00640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61101085c0 0x7f611019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.512+0000 7f610e800640 1 -- 172.21.15.86:0/1129915687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61101097b0 msgr2=0x7f611019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.512+0000 7f610e800640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61101097b0 0x7f611019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.512+0000 7f610e800640 1 -- 172.21.15.86:0/1129915687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61101085c0 msgr2=0x7f611019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.512+0000 7f610e800640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61101085c0 0x7f611019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.512+0000 7f610e800640 1 -- 172.21.15.86:0/1129915687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61101a63f0 con 0x7f611010a170 2024-04-17T07:33:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.512+0000 7f610e800640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f611010a170 0x7f61101a5d80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6108032380 tx=0x7f6108032820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.513+0000 7f6106a00640 1 -- 172.21.15.86:0/1129915687 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108041020 con 0x7f611010a170 2024-04-17T07:33:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.513+0000 7f6114a00640 1 -- 172.21.15.86:0/1129915687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61101a6680 con 0x7f611010a170 2024-04-17T07:33:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.513+0000 7f6114a00640 1 -- 172.21.15.86:0/1129915687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61101a6bc0 con 0x7f611010a170 2024-04-17T07:33:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.513+0000 7f6106a00640 1 -- 172.21.15.86:0/1129915687 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6108003e20 con 0x7f611010a170 2024-04-17T07:33:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.513+0000 7f6106a00640 1 -- 172.21.15.86:0/1129915687 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108039040 con 0x7f611010a170 2024-04-17T07:33:58.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.515+0000 7f6114a00640 1 -- 172.21.15.86:0/1129915687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ec0052c0 con 0x7f611010a170 2024-04-17T07:33:58.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.518+0000 7f6106a00640 1 -- 172.21.15.86:0/1129915687 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f6108044020 con 0x7f611010a170 2024-04-17T07:33:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.521+0000 7f6106a00640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60f40a3c30 0x7f60f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:58.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.521+0000 7f6106a00640 1 -- 172.21.15.86:0/1129915687 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f61080f27a0 con 0x7f611010a170 2024-04-17T07:33:58.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.523+0000 7f610de00640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60f40a3c30 0x7f60f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:58.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.524+0000 7f610de00640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60f40a3c30 0x7f60f40a60f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f61000095f0 tx=0x7f6100005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:58.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:58.534+0000 7f6106a00640 1 -- 172.21.15.86:0/1129915687 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f610804b020 con 0x7f611010a170 2024-04-17T07:33:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.068+0000 7f6114a00640 1 -- 172.21.15.86:0/1129915687 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60ec0055b0 con 0x7f611010a170 2024-04-17T07:33:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.071+0000 7f6106a00640 1 -- 172.21.15.86:0/1129915687 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f61080beda0 con 0x7f611010a170 2024-04-17T07:33:59.072 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:33:59.072 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:33:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.075+0000 7f6114a00640 1 -- 172.21.15.86:0/1129915687 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60f40a3c30 msgr2=0x7f60f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.075+0000 7f6114a00640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60f40a3c30 0x7f60f40a60f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f61000095f0 tx=0x7f6100005f90 comp rx=0 tx=0).stop 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.075+0000 7f6114a00640 1 -- 172.21.15.86:0/1129915687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f611010a170 msgr2=0x7f61101a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.075+0000 7f6114a00640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f611010a170 0x7f61101a5d80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6108032380 tx=0x7f6108032820 comp rx=0 tx=0).stop 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.075+0000 7f6114a00640 1 -- 172.21.15.86:0/1129915687 shutdown_connections 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.075+0000 7f6114a00640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f60f40a3c30 0x7f60f40a60f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.075+0000 7f6114a00640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f611010a170 0x7f61101a5d80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.075+0000 7f6114a00640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61101097b0 0x7f611019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.075+0000 7f6114a00640 1 --2- 172.21.15.86:0/1129915687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61101085c0 0x7f611019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.075+0000 7f6114a00640 1 -- 172.21.15.86:0/1129915687 >> 172.21.15.86:0/1129915687 conn(0x7f6110077580 msgr2=0x7f611010cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.076+0000 7f6114a00640 1 -- 172.21.15.86:0/1129915687 shutdown_connections 2024-04-17T07:33:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.076+0000 7f6114a00640 1 -- 172.21.15.86:0/1129915687 wait complete. 2024-04-17T07:33:59.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:58 smithi086 ceph-mon[31733]: pgmap v1028: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-04-17T07:33:59.108 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:33:59.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:58 smithi098 ceph-mon[35418]: pgmap v1028: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-04-17T07:33:59.278 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:58 smithi103 ceph-mon[35495]: pgmap v1028: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-04-17T07:33:59.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 875.0 s, 531.6 tps, lat 189.267 ms stddev 130.157 2024-04-17T07:33:59.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.433+0000 7f0401800640 1 -- 172.21.15.86:0/511136580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc105220 msgr2=0x7f03fc107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.433+0000 7f0401800640 1 --2- 172.21.15.86:0/511136580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc105220 0x7f03fc107610 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009ca0 tx=0x7f03ec02c120 comp rx=0 tx=0).stop 2024-04-17T07:33:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.434+0000 7f0401800640 1 -- 172.21.15.86:0/511136580 shutdown_connections 2024-04-17T07:33:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.434+0000 7f0401800640 1 --2- 172.21.15.86:0/511136580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03fc107b50 0x7f03fc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.434+0000 7f0401800640 1 --2- 172.21.15.86:0/511136580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc105220 0x7f03fc107610 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.434+0000 7f0401800640 1 --2- 172.21.15.86:0/511136580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03fc1028f0 0x7f03fc104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.434+0000 7f0401800640 1 -- 172.21.15.86:0/511136580 >> 172.21.15.86:0/511136580 conn(0x7f03fc0fc300 msgr2=0x7f03fc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.436+0000 7f0401800640 1 -- 172.21.15.86:0/511136580 shutdown_connections 2024-04-17T07:33:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.437+0000 7f0401800640 1 -- 172.21.15.86:0/511136580 wait complete. 2024-04-17T07:33:59.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.439+0000 7f0401800640 1 Processor -- start 2024-04-17T07:33:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.439+0000 7f0401800640 1 -- start start 2024-04-17T07:33:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.440+0000 7f0401800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03fc1028f0 0x7f03fc196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.440+0000 7f0401800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc105220 0x7f03fc196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.440+0000 7f0401800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03fc107b50 0x7f03fc1935b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.440+0000 7f0401800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc10d9f0 con 0x7f03fc1028f0 2024-04-17T07:33:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.440+0000 7f0401800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc10d870 con 0x7f03fc107b50 2024-04-17T07:33:59.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.440+0000 7f0401800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc10db70 con 0x7f03fc105220 2024-04-17T07:33:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.440+0000 7f0400e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03fc107b50 0x7f03fc1935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.440+0000 7f0400e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03fc107b50 0x7f03fc1935b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52012/0 (socket says 172.21.15.86:52012) 2024-04-17T07:33:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.440+0000 7f0400e00640 1 -- 172.21.15.86:0/1466592712 learned_addr learned my addr 172.21.15.86:0/1466592712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.441+0000 7f03f3400640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc105220 0x7f03fc196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.441+0000 7f03fbe00640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03fc1028f0 0x7f03fc196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.441+0000 7f0400e00640 1 -- 172.21.15.86:0/1466592712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc105220 msgr2=0x7f03fc196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.441+0000 7f0400e00640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc105220 0x7f03fc196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.441+0000 7f0400e00640 1 -- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03fc1028f0 msgr2=0x7f03fc196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.441+0000 7f0400e00640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03fc1028f0 0x7f03fc196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.441+0000 7f0400e00640 1 -- 172.21.15.86:0/1466592712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc193e70 con 0x7f03fc107b50 2024-04-17T07:33:59.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.441+0000 7f03fbe00640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03fc1028f0 0x7f03fc196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:33:59.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.441+0000 7f0400e00640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03fc107b50 0x7f03fc1935b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009fd0 tx=0x7f03f400b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.442+0000 7f03f9600640 1 -- 172.21.15.86:0/1466592712 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4011830 con 0x7f03fc107b50 2024-04-17T07:33:59.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.442+0000 7f03f9600640 1 -- 172.21.15.86:0/1466592712 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03f4016070 con 0x7f03fc107b50 2024-04-17T07:33:59.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.442+0000 7f03f9600640 1 -- 172.21.15.86:0/1466592712 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4010df0 con 0x7f03fc107b50 2024-04-17T07:33:59.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.443+0000 7f0401800640 1 -- 172.21.15.86:0/1466592712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc199f10 con 0x7f03fc107b50 2024-04-17T07:33:59.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.443+0000 7f0401800640 1 -- 172.21.15.86:0/1466592712 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc19a3e0 con 0x7f03fc107b50 2024-04-17T07:33:59.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.451+0000 7f0405b69740 1 Processor -- start 2024-04-17T07:33:59.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.452+0000 7f03f9600640 1 -- 172.21.15.86:0/1466592712 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f03f4010510 con 0x7f03fc107b50 2024-04-17T07:33:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.453+0000 7f0405b69740 1 -- start start 2024-04-17T07:33:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.453+0000 7f0405b69740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6160 0x55f9a6db6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.453+0000 7f0405b69740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dbb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.453+0000 7f0405b69740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dbe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.453+0000 7f0405b69740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9a6d1c020 con 0x55f9a6dbbcc0 2024-04-17T07:33:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.453+0000 7f0405b69740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f9a6cc2460 con 0x55f9a6db6160 2024-04-17T07:33:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.453+0000 7f0405b69740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f9a6d17fe0 con 0x55f9a6db6b50 2024-04-17T07:33:59.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.454+0000 7f03f3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6160 0x55f9a6db6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.454+0000 7f03f3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6160 0x55f9a6db6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52016/0 (socket says 172.21.15.86:52016) 2024-04-17T07:33:59.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.454+0000 7f03f3400640 1 -- 172.21.15.86:0/786933659 learned_addr learned my addr 172.21.15.86:0/786933659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:59.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.454+0000 7f03f3400640 1 --2- 172.21.15.86:0/786933659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dbe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.454+0000 7f03fbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dbb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.455+0000 7f03fbe00640 1 -- 172.21.15.86:0/786933659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6160 msgr2=0x55f9a6db6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.455+0000 7f03fbe00640 1 --2- 172.21.15.86:0/786933659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6160 0x55f9a6db6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.455+0000 7f03fbe00640 1 -- 172.21.15.86:0/786933659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 msgr2=0x55f9a6dbe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.455+0000 7f03fbe00640 1 --2- 172.21.15.86:0/786933659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dbe150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.455+0000 7f03fbe00640 1 -- 172.21.15.86:0/786933659 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a6cca040 con 0x55f9a6db6b50 2024-04-17T07:33:59.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.455+0000 7f03f3400640 1 --2- 172.21.15.86:0/786933659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dbe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:33:59.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.455+0000 7f03f9600640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03e00a3d00 0x7f03e00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.455+0000 7f03f9600640 1 -- 172.21.15.86:0/1466592712 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f03f40cfd80 con 0x7f03fc107b50 2024-04-17T07:33:59.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.456+0000 7f03f3400640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03e00a3d00 0x7f03e00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.456+0000 7f03fbe00640 1 --2- 172.21.15.86:0/786933659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dbb650 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009b90 tx=0x7f03e400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e2cb354d8b118a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.456+0000 7f03f1600640 1 -- 172.21.15.86:0/786933659 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4016070 con 0x55f9a6db6b50 2024-04-17T07:33:59.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.457+0000 7f03f1600640 1 -- 172.21.15.86:0/786933659 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03e40054d0 con 0x55f9a6db6b50 2024-04-17T07:33:59.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.457+0000 7f03f1600640 1 -- 172.21.15.86:0/786933659 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4004e10 con 0x55f9a6db6b50 2024-04-17T07:33:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.458+0000 7f0405b69740 1 -- 172.21.15.86:0/786933659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 msgr2=0x55f9a6dbb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.458+0000 7f0405b69740 1 --2- 172.21.15.86:0/786933659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dbb650 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009b90 tx=0x7f03e400e6c0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.458+0000 7f03f3400640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03e00a3d00 0x7f03e00a61c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x55f9a6db73b0 tx=0x7f03ec032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.460+0000 7f0405b69740 1 -- 172.21.15.86:0/786933659 shutdown_connections 2024-04-17T07:33:59.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.460+0000 7f0405b69740 1 --2- 172.21.15.86:0/786933659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dbe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.460+0000 7f0405b69740 1 --2- 172.21.15.86:0/786933659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dbb650 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.460+0000 7f0405b69740 1 --2- 172.21.15.86:0/786933659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6160 0x55f9a6db6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.460+0000 7f0405b69740 1 -- 172.21.15.86:0/786933659 >> 172.21.15.86:0/786933659 conn(0x55f9a6db16d0 msgr2=0x55f9a6db1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.461+0000 7f0405b69740 1 -- 172.21.15.86:0/786933659 shutdown_connections 2024-04-17T07:33:59.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.461+0000 7f0405b69740 1 -- 172.21.15.86:0/786933659 wait complete. 2024-04-17T07:33:59.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.462+0000 7f0405b69740 1 -- start start 2024-04-17T07:33:59.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.462+0000 7f0405b69740 1 Processor -- start 2024-04-17T07:33:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.463+0000 7f0405b69740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 0x55f9a6dc0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.463+0000 7f0405b69740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.463+0000 7f0405b69740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dc7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.463+0000 7f0405b69740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9a6dbec70 con 0x55f9a6db6160 2024-04-17T07:33:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.463+0000 7f0405b69740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f9a6cd5830 con 0x55f9a6db6b50 2024-04-17T07:33:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.463+0000 7f0405b69740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f9a6dbedf0 con 0x55f9a6dbbcc0 2024-04-17T07:33:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.463+0000 7f0405b69740 10 client.0 _ll_register_callbacks cb 0x55f9a6ca3fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.464+0000 7f0400e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dc7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.464+0000 7f0400e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dc7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:53950/0 (socket says 172.21.15.86:53950) 2024-04-17T07:33:59.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.464+0000 7f0400e00640 1 -- 172.21.15.86:0/904476051 learned_addr learned my addr 172.21.15.86:0/904476051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:59.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.464+0000 7f03f3400640 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.464+0000 7f03fbe00640 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 0x55f9a6dc0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.465+0000 7f03f3400640 1 -- 172.21.15.86:0/904476051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6dbbcc0 msgr2=0x55f9a6dc7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.465+0000 7f03f3400640 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dc7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.465+0000 7f03f3400640 1 -- 172.21.15.86:0/904476051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 msgr2=0x55f9a6dc0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.465+0000 7f03f3400640 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 0x55f9a6dc0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.465+0000 7f03f3400640 1 -- 172.21.15.86:0/904476051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a6cd9920 con 0x55f9a6db6b50 2024-04-17T07:33:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.466+0000 7f03f3400640 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc0bd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009430 tx=0x7f03ec02c120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b3dcd595ff37c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.466+0000 7f03ea000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:33:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.466+0000 7f03ea000640 1 -- 172.21.15.86:0/904476051 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec039030 con 0x55f9a6db6b50 2024-04-17T07:33:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.466+0000 7f0405b69740 1 -- 172.21.15.86:0/904476051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e40054d0 con 0x55f9a6db6b50 2024-04-17T07:33:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.466+0000 7f03ea000640 1 -- 172.21.15.86:0/904476051 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f03ec004250 con 0x55f9a6db6b50 2024-04-17T07:33:59.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.467+0000 7f03ea000640 1 -- 172.21.15.86:0/904476051 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03ec004b40 con 0x55f9a6db6b50 2024-04-17T07:33:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.467+0000 7f0405b69740 10 client.38858 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.467+0000 7f0405b69740 1 -- 172.21.15.86:0/904476051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9a6ca77f0 con 0x55f9a6db6b50 2024-04-17T07:33:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.468+0000 7f03ea000640 1 -- 172.21.15.86:0/904476051 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03e40054d0 con 0x55f9a6db6b50 2024-04-17T07:33:59.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.468+0000 7f03ea000640 1 -- 172.21.15.86:0/904476051 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f03ec03f070 con 0x55f9a6db6b50 2024-04-17T07:33:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.468+0000 7f0405b69740 10 client.38858 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.468+0000 7f0405b69740 10 client.38858 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:33:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.468+0000 7f0405b69740 20 client.38858 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.468+0000 7f0405b69740 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f9a6dcc1a0 0x55f9a6dce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.468+0000 7f0405b69740 4 client.38858 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.468+0000 7f0405b69740 1 -- 172.21.15.86:0/904476051 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9a6d29a40 con 0x55f9a6dcc1a0 2024-04-17T07:33:59.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.469+0000 7f03fbe00640 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f9a6dcc1a0 0x55f9a6dce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.470+0000 7f03fbe00640 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f9a6dcc1a0 0x55f9a6dce610 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d37b712acc9b20d9 server_cookie=684aaf005d254938 in_seq=0 out_seq=0 2024-04-17T07:33:59.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.470+0000 7f03ea000640 10 client.38858 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:33:59.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f03ea000640 1 -- 172.21.15.86:0/904476051 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9a6d29a40 con 0x55f9a6dcc1a0 2024-04-17T07:33:59.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f03ea000640 10 client.38858 handle_command_reply: tid=0 2024-04-17T07:33:59.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f0405b69740 1 client.38858 shutdown 2024-04-17T07:33:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f0405b69740 1 -- 172.21.15.86:0/904476051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 msgr2=0x55f9a6dc0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f0405b69740 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc0bd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009430 tx=0x7f03ec02c120 comp rx=0 tx=0).stop 2024-04-17T07:33:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f0405b69740 1 -- 172.21.15.86:0/904476051 shutdown_connections 2024-04-17T07:33:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f0405b69740 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f9a6dcc1a0 0x55f9a6dce610 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f0405b69740 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dc7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f0405b69740 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc0bd0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f0405b69740 1 --2- 172.21.15.86:0/904476051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 0x55f9a6dc0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f0405b69740 1 -- 172.21.15.86:0/904476051 >> 172.21.15.86:0/904476051 conn(0x55f9a6db16d0 msgr2=0x55f9a6db0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.471+0000 7f03ea000640 0 client.38858 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:33:59.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.472+0000 7f0405b69740 1 -- 172.21.15.86:0/904476051 shutdown_connections 2024-04-17T07:33:59.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.472+0000 7f0405b69740 1 -- 172.21.15.86:0/904476051 wait complete. 2024-04-17T07:33:59.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.472+0000 7f0405b69740 20 client.38858 trim_cache size 0 max 16384 2024-04-17T07:33:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.498+0000 7f0405b69740 1 Processor -- start 2024-04-17T07:33:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.498+0000 7f0405b69740 1 -- start start 2024-04-17T07:33:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.498+0000 7f0405b69740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6160 0x55f9a6dac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.499+0000 7f0405b69740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.499+0000 7f0405b69740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6db00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.499+0000 7f0405b69740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1b9710 con 0x55f9a6dbbcc0 2024-04-17T07:33:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.499+0000 7f0405b69740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1bb250 con 0x55f9a6db6b50 2024-04-17T07:33:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.499+0000 7f0405b69740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1b9de0 con 0x55f9a6db6160 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.499+0000 7f0400e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6db00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.499+0000 7f0400e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6db00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38776/0 (socket says 172.21.15.86:38776) 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.499+0000 7f0400e00640 1 -- 172.21.15.86:0/762510887 learned_addr learned my addr 172.21.15.86:0/762510887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.499+0000 7f03fbe00640 1 --2- 172.21.15.86:0/762510887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6160 0x55f9a6dac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.500+0000 7f03f3400640 1 --2- 172.21.15.86:0/762510887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.500+0000 7f0400e00640 1 -- 172.21.15.86:0/762510887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6160 msgr2=0x55f9a6dac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.500+0000 7f0400e00640 1 --2- 172.21.15.86:0/762510887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6160 0x55f9a6dac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.500+0000 7f0400e00640 1 -- 172.21.15.86:0/762510887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 msgr2=0x55f9a6dc7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.500+0000 7f0400e00640 1 --2- 172.21.15.86:0/762510887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.500+0000 7f0400e00640 1 -- 172.21.15.86:0/762510887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a6a2e3f0 con 0x55f9a6dbbcc0 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.500+0000 7f0400e00640 1 --2- 172.21.15.86:0/762510887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6db00a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f03f40cd870 tx=0x7f03f400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edfc66def3c144e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.500+0000 7f03f0c00640 1 -- 172.21.15.86:0/762510887 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f40b5070 con 0x55f9a6dbbcc0 2024-04-17T07:33:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.500+0000 7f03f0c00640 1 -- 172.21.15.86:0/762510887 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03f40a0090 con 0x55f9a6dbbcc0 2024-04-17T07:33:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.501+0000 7f03f0c00640 1 -- 172.21.15.86:0/762510887 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f40bc0b0 con 0x55f9a6dbbcc0 2024-04-17T07:33:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.501+0000 7f0405b69740 1 -- 172.21.15.86:0/762510887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 msgr2=0x55f9a6db00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.501+0000 7f0405b69740 1 --2- 172.21.15.86:0/762510887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6db00a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f03f40cd870 tx=0x7f03f400bf90 comp rx=0 tx=0).stop 2024-04-17T07:33:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.502+0000 7f0405b69740 1 -- 172.21.15.86:0/762510887 shutdown_connections 2024-04-17T07:33:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.502+0000 7f0405b69740 1 --2- 172.21.15.86:0/762510887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6db00a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.502+0000 7f0405b69740 1 --2- 172.21.15.86:0/762510887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.502+0000 7f0405b69740 1 --2- 172.21.15.86:0/762510887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6160 0x55f9a6dac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.502+0000 7f0405b69740 1 -- 172.21.15.86:0/762510887 >> 172.21.15.86:0/762510887 conn(0x55f9a6db16d0 msgr2=0x55f9a6dc0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.502+0000 7f0405b69740 1 -- 172.21.15.86:0/762510887 shutdown_connections 2024-04-17T07:33:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.502+0000 7f0405b69740 1 -- 172.21.15.86:0/762510887 wait complete. 2024-04-17T07:33:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.502+0000 7f0405b69740 1 -- start start 2024-04-17T07:33:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.503+0000 7f0405b69740 1 Processor -- start 2024-04-17T07:33:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.503+0000 7f0405b69740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 0x55f9a6dc4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.503+0000 7f0405b69740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.503+0000 7f0405b69740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dc6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.504+0000 7f03f3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.504+0000 7f03f3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:53972/0 (socket says 172.21.15.86:53972) 2024-04-17T07:33:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.505+0000 7f03fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 0x55f9a6dc4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.505+0000 7f03f3400640 1 -- 172.21.15.86:0/1658252892 learned_addr learned my addr 172.21.15.86:0/1658252892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.505+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1bab80 con 0x55f9a6db6160 2024-04-17T07:33:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ba4b0 con 0x55f9a6dbbcc0 2024-04-17T07:33:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1b7750 con 0x55f9a6db6b50 2024-04-17T07:33:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f0405b69740 10 client.0 _ll_register_callbacks cb 0x7f03ec032960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f0400e00640 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dc6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f03fbe00640 1 -- 172.21.15.86:0/1658252892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 msgr2=0x55f9a6dc5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f03fbe00640 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f03fbe00640 1 -- 172.21.15.86:0/1658252892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6dbbcc0 msgr2=0x55f9a6dc6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f03fbe00640 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dc6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f03fbe00640 1 -- 172.21.15.86:0/1658252892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a6af87d0 con 0x55f9a6db6160 2024-04-17T07:33:59.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f40a0090 con 0x55f9a6db6160 2024-04-17T07:33:59.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.506+0000 7f03f3400640 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:33:59.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.507+0000 7f03fbe00640 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 0x55f9a6dc4ef0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x55f9a6db2640 tx=0x7f03e4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ba4b2530505085 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.508+0000 7f03ea000640 10 client.19800 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:33:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.508+0000 7f03ea000640 1 -- 172.21.15.86:0/1658252892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4016040 con 0x55f9a6db6160 2024-04-17T07:33:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.508+0000 7f03ea000640 1 -- 172.21.15.86:0/1658252892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f03e4026540 con 0x55f9a6db6160 2024-04-17T07:33:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.508+0000 7f03ea000640 1 -- 172.21.15.86:0/1658252892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f03e4026780 con 0x55f9a6db6160 2024-04-17T07:33:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.508+0000 7f0405b69740 10 client.19800 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.509+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55f9a6dc7b10 con 0x55f9a6db6160 2024-04-17T07:33:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.509+0000 7f03ea000640 1 -- 172.21.15.86:0/1658252892 <== mon.0 v2:172.21.15.86:3300/0 4 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f03e400f040 con 0x55f9a6db6160 2024-04-17T07:33:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.509+0000 7f0405b69740 10 client.19800 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.509+0000 7f0405b69740 10 client.19800 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:33:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.509+0000 7f0405b69740 20 client.19800 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.510+0000 7f0405b69740 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f9a6e329f0 0x55f9a6e34e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.510+0000 7f0405b69740 4 client.19800 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:33:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.510+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9a6d238b0 con 0x55f9a6e329f0 2024-04-17T07:33:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.510+0000 7f03f3400640 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f9a6e329f0 0x55f9a6e34e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.514+0000 7f03f3400640 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f9a6e329f0 0x55f9a6e34e10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cbc6e6e22fa7c4d7 server_cookie=ae7b31f6fa309690 in_seq=0 out_seq=0 2024-04-17T07:33:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.515+0000 7f03ea000640 10 client.19800 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:33:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.515+0000 7f03ea000640 1 -- 172.21.15.86:0/1658252892 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9a6d238b0 con 0x55f9a6e329f0 2024-04-17T07:33:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.515+0000 7f03ea000640 10 client.19800 handle_command_reply: tid=0 2024-04-17T07:33:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.515+0000 7f0405b69740 1 client.19800 shutdown 2024-04-17T07:33:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 msgr2=0x55f9a6dc4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f0405b69740 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 0x55f9a6dc4ef0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x55f9a6db2640 tx=0x7f03e4009310 comp rx=0 tx=0).stop 2024-04-17T07:33:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 shutdown_connections 2024-04-17T07:33:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f0405b69740 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x55f9a6e329f0 0x55f9a6e34e10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f0405b69740 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f9a6dbbcc0 0x55f9a6dc6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f0405b69740 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9a6db6b50 0x55f9a6dc5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f0405b69740 1 --2- 172.21.15.86:0/1658252892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9a6db6160 0x55f9a6dc4ef0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 >> 172.21.15.86:0/1658252892 conn(0x55f9a6db16d0 msgr2=0x55f9a6db4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f03ea000640 0 client.19800 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:33:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 shutdown_connections 2024-04-17T07:33:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.516+0000 7f0405b69740 1 -- 172.21.15.86:0/1658252892 wait complete. 2024-04-17T07:33:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.517+0000 7f0405b69740 20 client.19800 trim_cache size 0 max 16384 2024-04-17T07:33:59.519 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:33:59.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.518+0000 7f0401800640 1 -- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03e00a3d00 msgr2=0x7f03e00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.518+0000 7f0401800640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03e00a3d00 0x7f03e00a61c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x55f9a6db73b0 tx=0x7f03ec032340 comp rx=0 tx=0).stop 2024-04-17T07:33:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.518+0000 7f0401800640 1 -- 172.21.15.86:0/1466592712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03fc107b50 msgr2=0x7f03fc1935b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.518+0000 7f0401800640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03fc107b50 0x7f03fc1935b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009fd0 tx=0x7f03f400b8e0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.518+0000 7f0401800640 1 -- 172.21.15.86:0/1466592712 shutdown_connections 2024-04-17T07:33:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.518+0000 7f0401800640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f03e00a3d00 0x7f03e00a61c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.518+0000 7f0401800640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03fc107b50 0x7f03fc1935b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.518+0000 7f0401800640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc105220 0x7f03fc196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.518+0000 7f0401800640 1 --2- 172.21.15.86:0/1466592712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03fc1028f0 0x7f03fc196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.518+0000 7f0401800640 1 -- 172.21.15.86:0/1466592712 >> 172.21.15.86:0/1466592712 conn(0x7f03fc0fc300 msgr2=0x7f03fc102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.519+0000 7f0401800640 1 -- 172.21.15.86:0/1466592712 shutdown_connections 2024-04-17T07:33:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.519+0000 7f0401800640 1 -- 172.21.15.86:0/1466592712 wait complete. 2024-04-17T07:33:59.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:33:59.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.916+0000 7f42e7800640 1 -- 172.21.15.86:0/1147671483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e010a170 msgr2=0x7f42e0110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.916+0000 7f42e7800640 1 --2- 172.21.15.86:0/1147671483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e010a170 0x7f42e0110a00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009900 tx=0x7f42cc02bc60 comp rx=0 tx=0).stop 2024-04-17T07:33:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.916+0000 7f42e7800640 1 -- 172.21.15.86:0/1147671483 shutdown_connections 2024-04-17T07:33:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.917+0000 7f42e7800640 1 --2- 172.21.15.86:0/1147671483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e010a170 0x7f42e0110a00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.917+0000 7f42e7800640 1 --2- 172.21.15.86:0/1147671483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e01097b0 0x7f42e0109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.917+0000 7f42e7800640 1 --2- 172.21.15.86:0/1147671483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e01085c0 0x7f42e01089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.917+0000 7f42e7800640 1 -- 172.21.15.86:0/1147671483 >> 172.21.15.86:0/1147671483 conn(0x7f42e0077560 msgr2=0x7f42e00799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.917+0000 7f42e7800640 1 -- 172.21.15.86:0/1147671483 shutdown_connections 2024-04-17T07:33:59.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.917+0000 7f42e7800640 1 -- 172.21.15.86:0/1147671483 wait complete. 2024-04-17T07:33:59.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.919+0000 7f42e7800640 1 Processor -- start 2024-04-17T07:33:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.919+0000 7f42e7800640 1 -- start start 2024-04-17T07:33:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.920+0000 7f42e7800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e01085c0 0x7f42e019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.920+0000 7f42e7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e01097b0 0x7f42e019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.920+0000 7f42e7800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e010a170 0x7f42e01a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.920+0000 7f42e7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42e0114120 con 0x7f42e01097b0 2024-04-17T07:33:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.920+0000 7f42e7800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42e0113fa0 con 0x7f42e010a170 2024-04-17T07:33:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.920+0000 7f42e7800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01142a0 con 0x7f42e01085c0 2024-04-17T07:33:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.920+0000 7f42e4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e01085c0 0x7f42e019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.921+0000 7f42e4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e01085c0 0x7f42e019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:53982/0 (socket says 172.21.15.86:53982) 2024-04-17T07:33:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.921+0000 7f42e4e00640 1 -- 172.21.15.86:0/231481757 learned_addr learned my addr 172.21.15.86:0/231481757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.921+0000 7f42dfe00640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e01097b0 0x7f42e019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.921+0000 7f42e5800640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e010a170 0x7f42e01a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.921+0000 7f42e4e00640 1 -- 172.21.15.86:0/231481757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e010a170 msgr2=0x7f42e01a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.921+0000 7f42e4e00640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e010a170 0x7f42e01a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.921+0000 7f42e4e00640 1 -- 172.21.15.86:0/231481757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e01097b0 msgr2=0x7f42e019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.921+0000 7f42e4e00640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e01097b0 0x7f42e019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.921+0000 7f42e4e00640 1 -- 172.21.15.86:0/231481757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e006bab0 con 0x7f42e01085c0 2024-04-17T07:33:59.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.922+0000 7f42e4e00640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e01085c0 0x7f42e019e790 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f42d000e9c0 tx=0x7f42d000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.923+0000 7f42dd600640 1 -- 172.21.15.86:0/231481757 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d000cd80 con 0x7f42e01085c0 2024-04-17T07:33:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.923+0000 7f42dd600640 1 -- 172.21.15.86:0/231481757 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d00035f0 con 0x7f42e01085c0 2024-04-17T07:33:59.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.923+0000 7f42dd600640 1 -- 172.21.15.86:0/231481757 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0010690 con 0x7f42e01085c0 2024-04-17T07:33:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.925+0000 7f42e7800640 1 -- 172.21.15.86:0/231481757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e006bda0 con 0x7f42e01085c0 2024-04-17T07:33:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.926+0000 7f42e7800640 1 -- 172.21.15.86:0/231481757 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e006c270 con 0x7f42e01085c0 2024-04-17T07:33:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.932+0000 7f42ea5e7740 1 Processor -- start 2024-04-17T07:33:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.933+0000 7f42ea5e7740 1 -- start start 2024-04-17T07:33:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.934+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436c973c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.934+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 0x563436c9c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.934+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 0x563436c9ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.934+0000 7f42dd600640 1 -- 172.21.15.86:0/231481757 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f42d00037a0 con 0x7f42e01085c0 2024-04-17T07:33:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.934+0000 7f42ea5e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563436bfd020 con 0x563436c9cb00 2024-04-17T07:33:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.934+0000 7f42ea5e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563436ba3460 con 0x563436c97990 2024-04-17T07:33:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.934+0000 7f42ea5e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563436bf8fe0 con 0x563436c96fa0 2024-04-17T07:33:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.934+0000 7f42dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 0x563436c9ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.934+0000 7f42dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 0x563436c9ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38822/0 (socket says 172.21.15.86:38822) 2024-04-17T07:33:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.934+0000 7f42dfe00640 1 -- 172.21.15.86:0/4049013719 learned_addr learned my addr 172.21.15.86:0/4049013719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.935+0000 7f42e5800640 1 --2- 172.21.15.86:0/4049013719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436c973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.935+0000 7f42e4e00640 1 --2- 172.21.15.86:0/4049013719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 0x563436c9c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.937+0000 7f42e5800640 1 -- 172.21.15.86:0/4049013719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 msgr2=0x563436c9c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.937+0000 7f42e5800640 1 --2- 172.21.15.86:0/4049013719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 0x563436c9c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.937+0000 7f42e5800640 1 -- 172.21.15.86:0/4049013719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 msgr2=0x563436c9ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.937+0000 7f42e5800640 1 --2- 172.21.15.86:0/4049013719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 0x563436c9ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.937+0000 7f42e5800640 1 -- 172.21.15.86:0/4049013719 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563436bab040 con 0x563436c96fa0 2024-04-17T07:33:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.937+0000 7f42dfe00640 1 --2- 172.21.15.86:0/4049013719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 0x563436c9ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:33:59.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.938+0000 7f42e5800640 1 --2- 172.21.15.86:0/4049013719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436c973c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f42cc036040 tx=0x7f42cc009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57ff8c84b4c80e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.938+0000 7f42d6000640 1 -- 172.21.15.86:0/4049013719 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc044020 con 0x563436c96fa0 2024-04-17T07:33:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.939+0000 7f42d6000640 1 -- 172.21.15.86:0/4049013719 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42cc02cb60 con 0x563436c96fa0 2024-04-17T07:33:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.939+0000 7f42d6000640 1 -- 172.21.15.86:0/4049013719 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc004920 con 0x563436c96fa0 2024-04-17T07:33:59.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.939+0000 7f42dd600640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42c40a3c30 0x7f42c40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.940+0000 7f42e4e00640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42c40a3c30 0x7f42c40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.940+0000 7f42dd600640 1 -- 172.21.15.86:0/231481757 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f42d00d0c80 con 0x7f42e01085c0 2024-04-17T07:33:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.942+0000 7f42ea5e7740 1 -- 172.21.15.86:0/4049013719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 msgr2=0x563436c973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.942+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/4049013719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436c973c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f42cc036040 tx=0x7f42cc009630 comp rx=0 tx=0).stop 2024-04-17T07:33:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.942+0000 7f42ea5e7740 1 -- 172.21.15.86:0/4049013719 shutdown_connections 2024-04-17T07:33:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.943+0000 7f42e4e00640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42c40a3c30 0x7f42c40a60f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f42d00ce770 tx=0x7f42d00d4760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.943+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/4049013719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 0x563436c9ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.943+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/4049013719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 0x563436c9c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.943+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/4049013719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436c973c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.943+0000 7f42ea5e7740 1 -- 172.21.15.86:0/4049013719 >> 172.21.15.86:0/4049013719 conn(0x563436c92510 msgr2=0x563436c92910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.943+0000 7f42ea5e7740 1 -- 172.21.15.86:0/4049013719 shutdown_connections 2024-04-17T07:33:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.944+0000 7f42ea5e7740 1 -- 172.21.15.86:0/4049013719 wait complete. 2024-04-17T07:33:59.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.945+0000 7f42ea5e7740 1 -- start start 2024-04-17T07:33:59.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.945+0000 7f42ea5e7740 1 Processor -- start 2024-04-17T07:33:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.947+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.947+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 0x563436ca1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.947+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 0x563436ca8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.948+0000 7f42ea5e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563436c9fa50 con 0x563436c9cb00 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.948+0000 7f42ea5e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563436bb6830 con 0x563436c97990 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.948+0000 7f42ea5e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563436c9fbd0 con 0x563436c96fa0 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.948+0000 7f42ea5e7740 10 client.0 _ll_register_callbacks cb 0x563436b84fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.948+0000 7f42e5800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.948+0000 7f42e5800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54000/0 (socket says 172.21.15.86:54000) 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.948+0000 7f42e5800640 1 -- 172.21.15.86:0/3591155631 learned_addr learned my addr 172.21.15.86:0/3591155631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.948+0000 7f42dfe00640 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 0x563436ca1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.948+0000 7f42e5800640 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 0x563436ca8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.949+0000 7f42e5800640 1 -- 172.21.15.86:0/3591155631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 msgr2=0x563436ca1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.949+0000 7f42e5800640 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 0x563436ca1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.949+0000 7f42e5800640 1 -- 172.21.15.86:0/3591155631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 msgr2=0x563436ca8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.949+0000 7f42e5800640 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 0x563436ca8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.949+0000 7f42e5800640 1 -- 172.21.15.86:0/3591155631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563436bba920 con 0x563436c96fa0 2024-04-17T07:33:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.949+0000 7f42dfe00640 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 0x563436ca1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:33:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.950+0000 7f42e5800640 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca14c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005c40 tx=0x7f42cc02cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fbf5c9559b30eff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.950+0000 7f42caa00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-17T07:33:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.951+0000 7f42caa00640 1 -- 172.21.15.86:0/3591155631 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc044030 con 0x563436c96fa0 2024-04-17T07:33:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.951+0000 7f42caa00640 1 -- 172.21.15.86:0/3591155631 <== mon.1 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f42cc003ab0 con 0x563436c96fa0 2024-04-17T07:33:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.951+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3591155631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42cc02cb60 con 0x563436c96fa0 2024-04-17T07:33:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.952+0000 7f42caa00640 1 -- 172.21.15.86:0/3591155631 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42cc05a050 con 0x563436c96fa0 2024-04-17T07:33:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.952+0000 7f42ea5e7740 10 client.28867 fetch_fsmap learned FSMap version 28 2024-04-17T07:33:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.952+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3591155631 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563436b887f0 con 0x563436c96fa0 2024-04-17T07:33:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.952+0000 7f42caa00640 1 -- 172.21.15.86:0/3591155631 <== mon.1 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42cc02cb60 con 0x563436c96fa0 2024-04-17T07:33:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.953+0000 7f42caa00640 1 -- 172.21.15.86:0/3591155631 <== mon.1 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f42cc034440 con 0x563436c96fa0 2024-04-17T07:33:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.953+0000 7f42ea5e7740 10 client.28867 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:33:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.953+0000 7f42ea5e7740 10 client.28867 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:33:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.953+0000 7f42ea5e7740 20 client.28867 populate_metadata read hostname 'smithi086' 2024-04-17T07:33:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.953+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563436cacba0 0x563436caef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.953+0000 7f42ea5e7740 4 client.28867 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:33:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.953+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3591155631 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563436c0aa40 con 0x563436cacba0 2024-04-17T07:33:59.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.954+0000 7f42dfe00640 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563436cacba0 0x563436caef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.955+0000 7f42dfe00640 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563436cacba0 0x563436caef90 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=608268a1a13e8e6c server_cookie=dffcdf9725869e8a in_seq=0 out_seq=0 2024-04-17T07:33:59.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.955+0000 7f42caa00640 10 client.28867 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:33:59.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.957+0000 7f42caa00640 1 -- 172.21.15.86:0/3591155631 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563436c0aa40 con 0x563436cacba0 2024-04-17T07:33:59.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.957+0000 7f42caa00640 10 client.28867 handle_command_reply: tid=0 2024-04-17T07:33:59.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.957+0000 7f42ea5e7740 1 client.28867 shutdown 2024-04-17T07:33:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.957+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3591155631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 msgr2=0x563436ca14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.957+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca14c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005c40 tx=0x7f42cc02cd10 comp rx=0 tx=0).stop 2024-04-17T07:33:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.958+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3591155631 shutdown_connections 2024-04-17T07:33:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.958+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563436cacba0 0x563436caef90 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.958+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c9cb00 0x563436ca8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.958+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c97990 0x563436ca1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.958+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/3591155631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca14c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.958+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3591155631 >> 172.21.15.86:0/3591155631 conn(0x563436c92510 msgr2=0x563436c91710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.958+0000 7f42caa00640 0 client.28867 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:33:59.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.958+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3591155631 shutdown_connections 2024-04-17T07:33:59.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.959+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3591155631 wait complete. 2024-04-17T07:33:59.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.959+0000 7f42ea5e7740 20 client.28867 trim_cache size 0 max 16384 2024-04-17T07:33:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.989+0000 7f42ea5e7740 1 Processor -- start 2024-04-17T07:33:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.989+0000 7f42ea5e7740 1 -- start start 2024-04-17T07:33:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.989+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.989+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 0x563436ca89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.989+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436cb36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.989+0000 7f42ea5e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01b99b0 con 0x563436c97990 2024-04-17T07:33:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.989+0000 7f42ea5e7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01bb4f0 con 0x563436c9cb00 2024-04-17T07:33:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.989+0000 7f42ea5e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01ba080 con 0x563436c96fa0 2024-04-17T07:33:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.990+0000 7f42e5800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436cb36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.990+0000 7f42e5800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436cb36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52096/0 (socket says 172.21.15.86:52096) 2024-04-17T07:33:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.990+0000 7f42e5800640 1 -- 172.21.15.86:0/3418453740 learned_addr learned my addr 172.21.15.86:0/3418453740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:33:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.990+0000 7f42e5800640 1 --2- 172.21.15.86:0/3418453740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.990+0000 7f42dfe00640 1 --2- 172.21.15.86:0/3418453740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 0x563436ca89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:33:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.991+0000 7f42dfe00640 1 -- 172.21.15.86:0/3418453740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 msgr2=0x563436ca1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.991+0000 7f42dfe00640 1 --2- 172.21.15.86:0/3418453740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.991+0000 7f42dfe00640 1 -- 172.21.15.86:0/3418453740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 msgr2=0x563436cb36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:33:59.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.991+0000 7f42dfe00640 1 --2- 172.21.15.86:0/3418453740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436cb36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.992+0000 7f42dfe00640 1 -- 172.21.15.86:0/3418453740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343690f3f0 con 0x563436c97990 2024-04-17T07:33:59.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.992+0000 7f42dfe00640 1 --2- 172.21.15.86:0/3418453740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 0x563436ca89d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x563436caf5f0 tx=0x7f42d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf71a14a7c6b7d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:33:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.993+0000 7f42d5600640 1 -- 172.21.15.86:0/3418453740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d80142d0 con 0x563436c97990 2024-04-17T07:33:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.993+0000 7f42d5600640 1 -- 172.21.15.86:0/3418453740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d801e500 con 0x563436c97990 2024-04-17T07:33:59.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.993+0000 7f42d5600640 1 -- 172.21.15.86:0/3418453740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d80142d0 con 0x563436c97990 2024-04-17T07:33:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.994+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3418453740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 msgr2=0x563436ca89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:33:59.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.994+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/3418453740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 0x563436ca89d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x563436caf5f0 tx=0x7f42d8004a40 comp rx=0 tx=0).stop 2024-04-17T07:33:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.995+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3418453740 shutdown_connections 2024-04-17T07:33:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.995+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/3418453740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436cb36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.995+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/3418453740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 0x563436ca89d0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.995+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/3418453740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:33:59.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.995+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3418453740 >> 172.21.15.86:0/3418453740 conn(0x563436c92510 msgr2=0x563436c96000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:33:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.996+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3418453740 shutdown_connections 2024-04-17T07:33:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.996+0000 7f42ea5e7740 1 -- 172.21.15.86:0/3418453740 wait complete. 2024-04-17T07:33:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.996+0000 7f42ea5e7740 1 -- start start 2024-04-17T07:33:59.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.997+0000 7f42ea5e7740 1 Processor -- start 2024-04-17T07:33:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.998+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.998+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 0x563436ca6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:33:59.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:33:59.998+0000 7f42ea5e7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436ca7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.000+0000 7f42e5800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436ca7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.000+0000 7f42e5800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436ca7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52108/0 (socket says 172.21.15.86:52108) 2024-04-17T07:34:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.000+0000 7f42e5800640 1 -- 172.21.15.86:0/2288064647 learned_addr learned my addr 172.21.15.86:0/2288064647 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.000+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01bae20 con 0x563436c97990 2024-04-17T07:34:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.000+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01ba750 con 0x563436c9cb00 2024-04-17T07:34:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.000+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f42e0073f60 con 0x563436c96fa0 2024-04-17T07:34:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.001+0000 7f42ea5e7740 10 client.0 _ll_register_callbacks cb 0x7f42cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.001+0000 7f42e5800640 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.001+0000 7f42dfe00640 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 0x563436ca6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.002+0000 7f42e5800640 1 -- 172.21.15.86:0/2288064647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 msgr2=0x563436ca62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.002+0000 7f42e5800640 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.002+0000 7f42e5800640 1 -- 172.21.15.86:0/2288064647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 msgr2=0x563436ca6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.002+0000 7f42e5800640 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 0x563436ca6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.003+0000 7f42e5800640 1 -- 172.21.15.86:0/2288064647 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634369d97d0 con 0x563436c9cb00 2024-04-17T07:34:00.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.003+0000 7f42dfe00640 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 0x563436ca6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:34:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.004+0000 7f42e5800640 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436ca7820 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0349b0 tx=0x7f42cc02cd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=869a81c37912e2a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.004+0000 7f42caa00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:34:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.005+0000 7f42caa00640 1 -- 172.21.15.86:0/2288064647 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc044040 con 0x563436c9cb00 2024-04-17T07:34:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.005+0000 7f42caa00640 1 -- 172.21.15.86:0/2288064647 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f42cc003ab0 con 0x563436c9cb00 2024-04-17T07:34:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.005+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d801e500 con 0x563436c9cb00 2024-04-17T07:34:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.006+0000 7f42caa00640 1 -- 172.21.15.86:0/2288064647 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42cc02cb60 con 0x563436c9cb00 2024-04-17T07:34:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.006+0000 7f42ea5e7740 10 client.38882 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.006+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563436d12f20 con 0x563436c9cb00 2024-04-17T07:34:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.008+0000 7f42caa00640 1 -- 172.21.15.86:0/2288064647 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d801e500 con 0x563436c9cb00 2024-04-17T07:34:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.008+0000 7f42caa00640 1 -- 172.21.15.86:0/2288064647 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f42cc05f440 con 0x563436c9cb00 2024-04-17T07:34:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.008+0000 7f42ea5e7740 10 client.38882 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.008+0000 7f42ea5e7740 10 client.38882 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:34:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.008+0000 7f42ea5e7740 20 client.38882 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.008+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563436d13810 0x563436d15c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.008+0000 7f42ea5e7740 4 client.38882 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:34:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.008+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563436c048b0 con 0x563436d13810 2024-04-17T07:34:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.009+0000 7f42dfe00640 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563436d13810 0x563436d15c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.011+0000 7f42dfe00640 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563436d13810 0x563436d15c30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=564183b4755793c server_cookie=588c43aedf2f7f03 in_seq=0 out_seq=0 2024-04-17T07:34:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.012+0000 7f42caa00640 10 client.38882 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:34:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.012+0000 7f42caa00640 1 -- 172.21.15.86:0/2288064647 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563436c048b0 con 0x563436d13810 2024-04-17T07:34:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.012+0000 7f42caa00640 10 client.38882 handle_command_reply: tid=0 2024-04-17T07:34:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.012+0000 7f42ea5e7740 1 client.38882 shutdown 2024-04-17T07:34:00.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.013+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 msgr2=0x563436ca7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.013+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436ca7820 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0349b0 tx=0x7f42cc02cd10 comp rx=0 tx=0).stop 2024-04-17T07:34:00.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.013+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 shutdown_connections 2024-04-17T07:34:00.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.013+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x563436d13810 0x563436d15c30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.013+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563436c9cb00 0x563436ca7820 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.013+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563436c97990 0x563436ca6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.013+0000 7f42caa00640 0 client.38882 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:34:00.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.013+0000 7f42ea5e7740 1 --2- 172.21.15.86:0/2288064647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563436c96fa0 0x563436ca62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.013+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 >> 172.21.15.86:0/2288064647 conn(0x563436c92510 msgr2=0x563436c96000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.015+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 shutdown_connections 2024-04-17T07:34:00.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.015+0000 7f42ea5e7740 1 -- 172.21.15.86:0/2288064647 wait complete. 2024-04-17T07:34:00.017 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:34:00.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.015+0000 7f42ea5e7740 20 client.38882 trim_cache size 0 max 16384 2024-04-17T07:34:00.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.017+0000 7f42e7800640 1 -- 172.21.15.86:0/231481757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42c40a3c30 msgr2=0x7f42c40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.017+0000 7f42e7800640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42c40a3c30 0x7f42c40a60f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f42d00ce770 tx=0x7f42d00d4760 comp rx=0 tx=0).stop 2024-04-17T07:34:00.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.017+0000 7f42e7800640 1 -- 172.21.15.86:0/231481757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e01085c0 msgr2=0x7f42e019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.017+0000 7f42e7800640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e01085c0 0x7f42e019e790 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f42d000e9c0 tx=0x7f42d000ee80 comp rx=0 tx=0).stop 2024-04-17T07:34:00.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.017+0000 7f42e7800640 1 -- 172.21.15.86:0/231481757 shutdown_connections 2024-04-17T07:34:00.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.017+0000 7f42e7800640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f42c40a3c30 0x7f42c40a60f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.017+0000 7f42e7800640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42e010a170 0x7f42e01a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.017+0000 7f42e7800640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42e01097b0 0x7f42e019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.017+0000 7f42e7800640 1 --2- 172.21.15.86:0/231481757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f42e01085c0 0x7f42e019e790 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.017+0000 7f42e7800640 1 -- 172.21.15.86:0/231481757 >> 172.21.15.86:0/231481757 conn(0x7f42e0077560 msgr2=0x7f42e010cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.018+0000 7f42e7800640 1 -- 172.21.15.86:0/231481757 shutdown_connections 2024-04-17T07:34:00.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.018+0000 7f42e7800640 1 -- 172.21.15.86:0/231481757 wait complete. 2024-04-17T07:34:00.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:34:00.159 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:33:59 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1129915687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:34:00.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:33:59 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1129915687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:34:00.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:33:59 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1129915687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:34:00.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.409+0000 7f5e04c00640 1 -- 172.21.15.86:0/997322084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e00103980 msgr2=0x7f5e0010a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.409+0000 7f5e04c00640 1 --2- 172.21.15.86:0/997322084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e00103980 0x7f5e0010a210 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009be0 tx=0x7f5df802bef0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.412+0000 7f5e04c00640 1 -- 172.21.15.86:0/997322084 shutdown_connections 2024-04-17T07:34:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.412+0000 7f5e04c00640 1 --2- 172.21.15.86:0/997322084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e00103980 0x7f5e0010a210 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.412+0000 7f5e04c00640 1 --2- 172.21.15.86:0/997322084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e00102fc0 0x7f5e00103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.412+0000 7f5e04c00640 1 --2- 172.21.15.86:0/997322084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00101dd0 0x7f5e001021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.412+0000 7f5e04c00640 1 -- 172.21.15.86:0/997322084 >> 172.21.15.86:0/997322084 conn(0x7f5e000fd340 msgr2=0x7f5e000ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.414+0000 7f5e04c00640 1 -- 172.21.15.86:0/997322084 shutdown_connections 2024-04-17T07:34:00.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.414+0000 7f5e04c00640 1 -- 172.21.15.86:0/997322084 wait complete. 2024-04-17T07:34:00.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.415+0000 7f5e04c00640 1 Processor -- start 2024-04-17T07:34:00.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.415+0000 7f5e04c00640 1 -- start start 2024-04-17T07:34:00.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.417+0000 7f5e04c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e00101dd0 0x7f5e00070940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.417+0000 7f5e04c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e00102fc0 0x7f5e00070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.417+0000 7f5e04c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00103980 0x7f5e001a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.417+0000 7f5dff400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e00101dd0 0x7f5e00070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.417+0000 7f5dff400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e00101dd0 0x7f5e00070940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52126/0 (socket says 172.21.15.86:52126) 2024-04-17T07:34:00.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.418+0000 7f5dff400640 1 -- 172.21.15.86:0/2269866106 learned_addr learned my addr 172.21.15.86:0/2269866106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:00.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.418+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0010d8e0 con 0x7f5e00103980 2024-04-17T07:34:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.418+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0010d760 con 0x7f5e00101dd0 2024-04-17T07:34:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.418+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0010da60 con 0x7f5e00102fc0 2024-04-17T07:34:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.418+0000 7f5dfea00640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e00102fc0 0x7f5e00070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.419+0000 7f5dffe00640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00103980 0x7f5e001a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.419+0000 7f5dff400640 1 -- 172.21.15.86:0/2269866106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e00102fc0 msgr2=0x7f5e00070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.419+0000 7f5dff400640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e00102fc0 0x7f5e00070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.419+0000 7f5dff400640 1 -- 172.21.15.86:0/2269866106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00103980 msgr2=0x7f5e001a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.419+0000 7f5dff400640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00103980 0x7f5e001a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.419+0000 7f5dff400640 1 -- 172.21.15.86:0/2269866106 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e001a8530 con 0x7f5e00101dd0 2024-04-17T07:34:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.420+0000 7f5dff400640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e00101dd0 0x7f5e00070940 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5df000e9c0 tx=0x7f5df000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.421+0000 7f5df7e00640 1 -- 172.21.15.86:0/2269866106 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df000cd80 con 0x7f5e00101dd0 2024-04-17T07:34:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.422+0000 7f5df7e00640 1 -- 172.21.15.86:0/2269866106 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5df00108e0 con 0x7f5e00101dd0 2024-04-17T07:34:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.422+0000 7f5df7e00640 1 -- 172.21.15.86:0/2269866106 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df00188e0 con 0x7f5e00101dd0 2024-04-17T07:34:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.422+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e00074190 con 0x7f5e00101dd0 2024-04-17T07:34:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.422+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e00074630 con 0x7f5e00101dd0 2024-04-17T07:34:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.429+0000 7f5e08fca740 1 Processor -- start 2024-04-17T07:34:00.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.430+0000 7f5e08fca740 1 -- start start 2024-04-17T07:34:00.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.431+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.431+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3ec3a0 0x556e1f3f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.431+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.431+0000 7f5e08fca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e1f349020 con 0x556e1f3f1510 2024-04-17T07:34:00.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.431+0000 7f5e08fca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556e1f2ef460 con 0x556e1f3eb9b0 2024-04-17T07:34:00.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.431+0000 7f5e08fca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556e1f344fe0 con 0x556e1f3ec3a0 2024-04-17T07:34:00.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.432+0000 7f5dffe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.432+0000 7f5dffe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ebdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52140/0 (socket says 172.21.15.86:52140) 2024-04-17T07:34:00.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.432+0000 7f5dffe00640 1 -- 172.21.15.86:0/4079757357 learned_addr learned my addr 172.21.15.86:0/4079757357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:00.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.433+0000 7f5dfea00640 1 --2- 172.21.15.86:0/4079757357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.435+0000 7f5dffe00640 1 -- 172.21.15.86:0/4079757357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3ec3a0 msgr2=0x556e1f3f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.434+0000 7f5df7e00640 1 -- 172.21.15.86:0/2269866106 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5df001f020 con 0x7f5e00101dd0 2024-04-17T07:34:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.435+0000 7f5dff400640 1 --2- 172.21.15.86:0/4079757357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3ec3a0 0x556e1f3f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.435+0000 7f5dffe00640 1 --2- 172.21.15.86:0/4079757357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3ec3a0 0x556e1f3f0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.435+0000 7f5dffe00640 1 -- 172.21.15.86:0/4079757357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 msgr2=0x556e1f3f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.437+0000 7f5df7e00640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5de40a3c80 0x7f5de40a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.437+0000 7f5dffe00640 1 --2- 172.21.15.86:0/4079757357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.437+0000 7f5dffe00640 1 -- 172.21.15.86:0/4079757357 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1f2f7040 con 0x556e1f3eb9b0 2024-04-17T07:34:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.437+0000 7f5dfea00640 1 --2- 172.21.15.86:0/4079757357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.438+0000 7f5dffe00640 1 --2- 172.21.15.86:0/4079757357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ebdd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009be0 tx=0x7f5df8009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af59ad5593a19c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.438+0000 7f5dff400640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5de40a3c80 0x7f5de40a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.439+0000 7f5debe00640 1 -- 172.21.15.86:0/4079757357 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8043070 con 0x556e1f3eb9b0 2024-04-17T07:34:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.439+0000 7f5debe00640 1 -- 172.21.15.86:0/4079757357 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5df80334d0 con 0x556e1f3eb9b0 2024-04-17T07:34:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.439+0000 7f5debe00640 1 -- 172.21.15.86:0/4079757357 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8043070 con 0x556e1f3eb9b0 2024-04-17T07:34:00.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.439+0000 7f5df7e00640 1 -- 172.21.15.86:0/2269866106 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5df00d0860 con 0x7f5e00101dd0 2024-04-17T07:34:00.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.440+0000 7f5e08fca740 1 -- 172.21.15.86:0/4079757357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 msgr2=0x556e1f3ebdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.440+0000 7f5dff400640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5de40a3c80 0x7f5de40a6140 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5df00ce350 tx=0x7f5df000cf20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.440+0000 7f5e08fca740 1 --2- 172.21.15.86:0/4079757357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ebdd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009be0 tx=0x7f5df8009940 comp rx=0 tx=0).stop 2024-04-17T07:34:00.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.441+0000 7f5e08fca740 1 -- 172.21.15.86:0/4079757357 shutdown_connections 2024-04-17T07:34:00.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.441+0000 7f5e08fca740 1 --2- 172.21.15.86:0/4079757357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.441+0000 7f5e08fca740 1 --2- 172.21.15.86:0/4079757357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3ec3a0 0x556e1f3f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.441+0000 7f5e08fca740 1 --2- 172.21.15.86:0/4079757357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ebdd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.441+0000 7f5e08fca740 1 -- 172.21.15.86:0/4079757357 >> 172.21.15.86:0/4079757357 conn(0x556e1f3de710 msgr2=0x556e1f3deb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.442+0000 7f5e08fca740 1 -- 172.21.15.86:0/4079757357 shutdown_connections 2024-04-17T07:34:00.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.442+0000 7f5e08fca740 1 -- 172.21.15.86:0/4079757357 wait complete. 2024-04-17T07:34:00.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.443+0000 7f5e08fca740 1 -- start start 2024-04-17T07:34:00.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.444+0000 7f5e08fca740 1 Processor -- start 2024-04-17T07:34:00.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.446+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3eb9b0 0x556e1f3f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.446+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.446+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3fd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.446+0000 7f5e08fca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e1f3f4460 con 0x556e1f3f1510 2024-04-17T07:34:00.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.446+0000 7f5e08fca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556e1f302830 con 0x556e1f3ec3a0 2024-04-17T07:34:00.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.446+0000 7f5e08fca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556e1f3f45e0 con 0x556e1f3eb9b0 2024-04-17T07:34:00.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.447+0000 7f5dfea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.448+0000 7f5e08fca740 10 client.0 _ll_register_callbacks cb 0x556e1f2d0fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:00.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.447+0000 7f5dfea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3f61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52154/0 (socket says 172.21.15.86:52154) 2024-04-17T07:34:00.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.448+0000 7f5dfea00640 1 -- 172.21.15.86:0/4180562753 learned_addr learned my addr 172.21.15.86:0/4180562753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:00.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.448+0000 7f5dffe00640 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3eb9b0 0x556e1f3f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.449+0000 7f5dffe00640 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.449+0000 7f5dfea00640 1 -- 172.21.15.86:0/4180562753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3eb9b0 msgr2=0x556e1f3f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.449+0000 7f5dfea00640 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3eb9b0 0x556e1f3f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.449+0000 7f5dfea00640 1 -- 172.21.15.86:0/4180562753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 msgr2=0x556e1f3fd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.449+0000 7f5dfea00640 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3fd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.449+0000 7f5dfea00640 1 -- 172.21.15.86:0/4180562753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1f306920 con 0x556e1f3ec3a0 2024-04-17T07:34:00.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.449+0000 7f5dffe00640 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3eb9b0 0x556e1f3f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:34:00.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.450+0000 7f5de8c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:34:00.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.450+0000 7f5dfea00640 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3f61f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0136c0 tx=0x7f5dec013b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d499d4bc2ed8ba server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.451+0000 7f5de8c00640 1 -- 172.21.15.86:0/4180562753 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec020060 con 0x556e1f3ec3a0 2024-04-17T07:34:00.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.451+0000 7f5de8c00640 1 -- 172.21.15.86:0/4180562753 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5dec011dd0 con 0x556e1f3ec3a0 2024-04-17T07:34:00.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.451+0000 7f5e08fca740 1 -- 172.21.15.86:0/4180562753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df80334d0 con 0x556e1f3ec3a0 2024-04-17T07:34:00.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.452+0000 7f5de8c00640 1 -- 172.21.15.86:0/4180562753 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5dec01b430 con 0x556e1f3ec3a0 2024-04-17T07:34:00.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.452+0000 7f5e08fca740 10 client.38894 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:00.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.452+0000 7f5e08fca740 1 -- 172.21.15.86:0/4180562753 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556e1f2d47f0 con 0x556e1f3ec3a0 2024-04-17T07:34:00.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.453+0000 7f5de8c00640 1 -- 172.21.15.86:0/4180562753 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5df80334d0 con 0x556e1f3ec3a0 2024-04-17T07:34:00.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.453+0000 7f5de8c00640 1 -- 172.21.15.86:0/4180562753 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5dec00e540 con 0x556e1f3ec3a0 2024-04-17T07:34:00.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.454+0000 7f5e08fca740 10 client.38894 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:00.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.454+0000 7f5e08fca740 10 client.38894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:34:00.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.454+0000 7f5e08fca740 20 client.38894 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:00.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.454+0000 7f5e08fca740 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556e1f401700 0x556e1f403b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.454+0000 7f5e08fca740 4 client.38894 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:00.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.454+0000 7f5e08fca740 1 -- 172.21.15.86:0/4180562753 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e1f356a40 con 0x556e1f401700 2024-04-17T07:34:00.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.454+0000 7f5dffe00640 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556e1f401700 0x556e1f403b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.455+0000 7f5dffe00640 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556e1f401700 0x556e1f403b20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f06a99d90b8e4c78 server_cookie=381902f903c103d1 in_seq=0 out_seq=0 2024-04-17T07:34:00.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.455+0000 7f5de8c00640 10 client.38894 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:34:00.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.457+0000 7f5de8c00640 1 -- 172.21.15.86:0/4180562753 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e1f356a40 con 0x556e1f401700 2024-04-17T07:34:00.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.457+0000 7f5de8c00640 10 client.38894 handle_command_reply: tid=0 2024-04-17T07:34:00.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.457+0000 7f5e08fca740 1 client.38894 shutdown 2024-04-17T07:34:00.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.457+0000 7f5e08fca740 1 -- 172.21.15.86:0/4180562753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 msgr2=0x556e1f3f61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.458+0000 7f5e08fca740 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3f61f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0136c0 tx=0x7f5dec013b80 comp rx=0 tx=0).stop 2024-04-17T07:34:00.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.458+0000 7f5e08fca740 1 -- 172.21.15.86:0/4180562753 shutdown_connections 2024-04-17T07:34:00.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.458+0000 7f5e08fca740 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556e1f401700 0x556e1f403b20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.458+0000 7f5e08fca740 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.458+0000 7f5e08fca740 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3f61f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.458+0000 7f5de8c00640 0 client.38894 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:34:00.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.458+0000 7f5e08fca740 1 --2- 172.21.15.86:0/4180562753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3eb9b0 0x556e1f3f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.458+0000 7f5e08fca740 1 -- 172.21.15.86:0/4180562753 >> 172.21.15.86:0/4180562753 conn(0x556e1f3de710 msgr2=0x556e1f3dd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.460+0000 7f5e08fca740 1 -- 172.21.15.86:0/4180562753 shutdown_connections 2024-04-17T07:34:00.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.460+0000 7f5e08fca740 1 -- 172.21.15.86:0/4180562753 wait complete. 2024-04-17T07:34:00.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.461+0000 7f5e08fca740 20 client.38894 trim_cache size 0 max 16384 2024-04-17T07:34:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.499+0000 7f5e08fca740 1 Processor -- start 2024-04-17T07:34:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.499+0000 7f5e08fca740 1 -- start start 2024-04-17T07:34:00.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.500+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ed700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.500+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3fd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.500+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3f1510 0x556e1f3edc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.500+0000 7f5e08fca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b0ff0 con 0x556e1f3eb9b0 2024-04-17T07:34:00.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.500+0000 7f5e08fca740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b2b30 con 0x556e1f3ec3a0 2024-04-17T07:34:00.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.500+0000 7f5e08fca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b16c0 con 0x556e1f3f1510 2024-04-17T07:34:00.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.501+0000 7f5dffe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3f1510 0x556e1f3edc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.502+0000 7f5dffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.502+0000 7f5dfea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3fd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.502+0000 7f5dfea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3fd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52156/0 (socket says 172.21.15.86:52156) 2024-04-17T07:34:00.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.502+0000 7f5dfea00640 1 -- 172.21.15.86:0/3538801499 learned_addr learned my addr 172.21.15.86:0/3538801499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.504+0000 7f5dfea00640 1 -- 172.21.15.86:0/3538801499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3f1510 msgr2=0x556e1f3edc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.504+0000 7f5dfea00640 1 --2- 172.21.15.86:0/3538801499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3f1510 0x556e1f3edc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.504+0000 7f5dfea00640 1 -- 172.21.15.86:0/3538801499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3eb9b0 msgr2=0x556e1f3ed700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.504+0000 7f5dfea00640 1 --2- 172.21.15.86:0/3538801499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ed700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.504+0000 7f5dfea00640 1 -- 172.21.15.86:0/3538801499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1f05b3f0 con 0x556e1f3ec3a0 2024-04-17T07:34:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.504+0000 7f5dffe00640 1 --2- 172.21.15.86:0/3538801499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ed700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.507+0000 7f5dfea00640 1 --2- 172.21.15.86:0/3538801499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3fd1f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5dec013690 tx=0x7f5dec005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a53ecb92e65c731 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.508+0000 7f5deb400640 1 -- 172.21.15.86:0/3538801499 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec02a080 con 0x556e1f3ec3a0 2024-04-17T07:34:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.508+0000 7f5deb400640 1 -- 172.21.15.86:0/3538801499 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dec011bd0 con 0x556e1f3ec3a0 2024-04-17T07:34:00.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.509+0000 7f5deb400640 1 -- 172.21.15.86:0/3538801499 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec01bd20 con 0x556e1f3ec3a0 2024-04-17T07:34:00.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.511+0000 7f5e08fca740 1 -- 172.21.15.86:0/3538801499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 msgr2=0x556e1f3fd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.511+0000 7f5e08fca740 1 --2- 172.21.15.86:0/3538801499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3fd1f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5dec013690 tx=0x7f5dec005730 comp rx=0 tx=0).stop 2024-04-17T07:34:00.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.511+0000 7f5e08fca740 1 -- 172.21.15.86:0/3538801499 shutdown_connections 2024-04-17T07:34:00.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.511+0000 7f5e08fca740 1 --2- 172.21.15.86:0/3538801499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3f1510 0x556e1f3edc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.511+0000 7f5e08fca740 1 --2- 172.21.15.86:0/3538801499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3ec3a0 0x556e1f3fd1f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.511+0000 7f5e08fca740 1 --2- 172.21.15.86:0/3538801499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3eb9b0 0x556e1f3ed700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.511+0000 7f5e08fca740 1 -- 172.21.15.86:0/3538801499 >> 172.21.15.86:0/3538801499 conn(0x556e1f3de710 msgr2=0x556e1f3f5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.512+0000 7f5e08fca740 1 -- 172.21.15.86:0/3538801499 shutdown_connections 2024-04-17T07:34:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.512+0000 7f5e08fca740 1 -- 172.21.15.86:0/3538801499 wait complete. 2024-04-17T07:34:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.513+0000 7f5e08fca740 1 -- start start 2024-04-17T07:34:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.513+0000 7f5e08fca740 1 Processor -- start 2024-04-17T07:34:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.515+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3fa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.515+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3ec3a0 0x556e1f3fa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.515+0000 7f5e08fca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3fb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.516+0000 7f5dffe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3fa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.516+0000 7f5dffe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3fa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52160/0 (socket says 172.21.15.86:52160) 2024-04-17T07:34:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.516+0000 7f5dffe00640 1 -- 172.21.15.86:0/2700862367 learned_addr learned my addr 172.21.15.86:0/2700862367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.516+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b2460 con 0x556e1f3f1510 2024-04-17T07:34:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.516+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b1d90 con 0x556e1f3eb9b0 2024-04-17T07:34:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.516+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001aefc0 con 0x556e1f3ec3a0 2024-04-17T07:34:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.516+0000 7f5e08fca740 10 client.0 _ll_register_callbacks cb 0x7f5dec009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.516+0000 7f5dffe00640 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3fb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.516+0000 7f5dfea00640 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3ec3a0 0x556e1f3fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.517+0000 7f5dffe00640 1 -- 172.21.15.86:0/2700862367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3ec3a0 msgr2=0x556e1f3fa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.517+0000 7f5dffe00640 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3ec3a0 0x556e1f3fa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.518+0000 7f5dffe00640 1 -- 172.21.15.86:0/2700862367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 msgr2=0x556e1f3fb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.518+0000 7f5dffe00640 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3fb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.518+0000 7f5dffe00640 1 -- 172.21.15.86:0/2700862367 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1f1257d0 con 0x556e1f3eb9b0 2024-04-17T07:34:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.521+0000 7f5dffe00640 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3fa330 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5df8000f80 tx=0x7f5df8009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=402ae0677d3adb30 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.521+0000 7f5de8c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:34:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.521+0000 7f5de8c00640 1 -- 172.21.15.86:0/2700862367 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df805b060 con 0x556e1f3eb9b0 2024-04-17T07:34:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.521+0000 7f5de8c00640 1 -- 172.21.15.86:0/2700862367 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5df8051040 con 0x556e1f3eb9b0 2024-04-17T07:34:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.521+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dec011bd0 con 0x556e1f3eb9b0 2024-04-17T07:34:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.522+0000 7f5de8c00640 1 -- 172.21.15.86:0/2700862367 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5df8062050 con 0x556e1f3eb9b0 2024-04-17T07:34:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.522+0000 7f5e08fca740 10 client.38900 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.522+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556e1f3fcfa0 con 0x556e1f3eb9b0 2024-04-17T07:34:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.523+0000 7f5de8c00640 1 -- 172.21.15.86:0/2700862367 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dec011bd0 con 0x556e1f3eb9b0 2024-04-17T07:34:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.523+0000 7f5de8c00640 1 -- 172.21.15.86:0/2700862367 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f5df8056ce0 con 0x556e1f3eb9b0 2024-04-17T07:34:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.523+0000 7f5e08fca740 10 client.38900 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.523+0000 7f5e08fca740 10 client.38900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:34:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.523+0000 7f5e08fca740 20 client.38900 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.523+0000 7f5e08fca740 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556e1f401700 0x556e1f469e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.523+0000 7f5e08fca740 4 client.38900 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:34:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.523+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e1f3508b0 con 0x556e1f401700 2024-04-17T07:34:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.524+0000 7f5dfea00640 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556e1f401700 0x556e1f469e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.526+0000 7f5dfea00640 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556e1f401700 0x556e1f469e60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36c7f55f285d8799 server_cookie=a64cbfa3d842c788 in_seq=0 out_seq=0 2024-04-17T07:34:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.526+0000 7f5de8c00640 10 client.38900 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:34:00.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.528+0000 7f5de8c00640 1 -- 172.21.15.86:0/2700862367 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e1f3508b0 con 0x556e1f401700 2024-04-17T07:34:00.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.528+0000 7f5de8c00640 10 client.38900 handle_command_reply: tid=0 2024-04-17T07:34:00.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.528+0000 7f5e08fca740 1 client.38900 shutdown 2024-04-17T07:34:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.530+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 msgr2=0x556e1f3fa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.530+0000 7f5e08fca740 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3fa330 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5df8000f80 tx=0x7f5df8009940 comp rx=0 tx=0).stop 2024-04-17T07:34:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.531+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 shutdown_connections 2024-04-17T07:34:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.531+0000 7f5e08fca740 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x556e1f401700 0x556e1f469e60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.531+0000 7f5e08fca740 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1f3f1510 0x556e1f3fb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.531+0000 7f5e08fca740 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556e1f3ec3a0 0x556e1f3fa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.531+0000 7f5e08fca740 1 --2- 172.21.15.86:0/2700862367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e1f3eb9b0 0x556e1f3fa330 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.531+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 >> 172.21.15.86:0/2700862367 conn(0x556e1f3de710 msgr2=0x556e1f3f5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.531+0000 7f5de8c00640 0 client.38900 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:34:00.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.533+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 shutdown_connections 2024-04-17T07:34:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.533+0000 7f5e08fca740 1 -- 172.21.15.86:0/2700862367 wait complete. 2024-04-17T07:34:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.534+0000 7f5e08fca740 20 client.38900 trim_cache size 0 max 16384 2024-04-17T07:34:00.536 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:34:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.536+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5de40a3c80 msgr2=0x7f5de40a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.536+0000 7f5e04c00640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5de40a3c80 0x7f5de40a6140 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5df00ce350 tx=0x7f5df000cf20 comp rx=0 tx=0).stop 2024-04-17T07:34:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.537+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e00101dd0 msgr2=0x7f5e00070940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.537+0000 7f5e04c00640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e00101dd0 0x7f5e00070940 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5df000e9c0 tx=0x7f5df000edc0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.537+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 shutdown_connections 2024-04-17T07:34:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.537+0000 7f5e04c00640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5de40a3c80 0x7f5de40a6140 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.537+0000 7f5e04c00640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e00103980 0x7f5e001a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.537+0000 7f5e04c00640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e00102fc0 0x7f5e00070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.537+0000 7f5e04c00640 1 --2- 172.21.15.86:0/2269866106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e00101dd0 0x7f5e00070940 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.537+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 >> 172.21.15.86:0/2269866106 conn(0x7f5e000fd340 msgr2=0x7f5e00106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.538+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 shutdown_connections 2024-04-17T07:34:00.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.538+0000 7f5e04c00640 1 -- 172.21.15.86:0/2269866106 wait complete. 2024-04-17T07:34:00.580 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:34:00.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.924+0000 7f2f6be00640 1 -- 172.21.15.86:0/2244196071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f64097930 msgr2=0x7f2f6409e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.924+0000 7f2f6be00640 1 --2- 172.21.15.86:0/2244196071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f64097930 0x7f2f6409e1c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009860 tx=0x7f2f5802bb70 comp rx=0 tx=0).stop 2024-04-17T07:34:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.925+0000 7f2f6be00640 1 -- 172.21.15.86:0/2244196071 shutdown_connections 2024-04-17T07:34:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.925+0000 7f2f6be00640 1 --2- 172.21.15.86:0/2244196071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f64097930 0x7f2f6409e1c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.925+0000 7f2f6be00640 1 --2- 172.21.15.86:0/2244196071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f64096f70 0x7f2f640973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.925+0000 7f2f6be00640 1 --2- 172.21.15.86:0/2244196071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f64095d80 0x7f2f640961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.925+0000 7f2f6be00640 1 -- 172.21.15.86:0/2244196071 >> 172.21.15.86:0/2244196071 conn(0x7f2f640912d0 msgr2=0x7f2f64093730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.925+0000 7f2f6be00640 1 -- 172.21.15.86:0/2244196071 shutdown_connections 2024-04-17T07:34:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.925+0000 7f2f6be00640 1 -- 172.21.15.86:0/2244196071 wait complete. 2024-04-17T07:34:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.927+0000 7f2f6be00640 1 Processor -- start 2024-04-17T07:34:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.927+0000 7f2f6be00640 1 -- start start 2024-04-17T07:34:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.928+0000 7f2f6be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f64095d80 0x7f2f64129d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.928+0000 7f2f6be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f64096f70 0x7f2f6412a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.928+0000 7f2f6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f64097930 0x7f2f64142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.928+0000 7f2f6be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6409f6d0 con 0x7f2f64097930 2024-04-17T07:34:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.928+0000 7f2f6be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6409f550 con 0x7f2f64095d80 2024-04-17T07:34:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.928+0000 7f2f6be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6409f850 con 0x7f2f64096f70 2024-04-17T07:34:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.928+0000 7f2f6a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f64096f70 0x7f2f6412a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.928+0000 7f2f6a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f64096f70 0x7f2f6412a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54100/0 (socket says 172.21.15.86:54100) 2024-04-17T07:34:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.928+0000 7f2f6a000640 1 -- 172.21.15.86:0/4214872250 learned_addr learned my addr 172.21.15.86:0/4214872250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.928+0000 7f2f6b400640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f64097930 0x7f2f64142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.929+0000 7f2f6a000640 1 -- 172.21.15.86:0/4214872250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f64095d80 msgr2=0x7f2f64129d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T07:34:00.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.929+0000 7f2f6a000640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f64095d80 0x7f2f64129d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.929+0000 7f2f6a000640 1 -- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f64097930 msgr2=0x7f2f64142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.929+0000 7f2f6a000640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f64097930 0x7f2f64142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.929+0000 7f2f6a000640 1 -- 172.21.15.86:0/4214872250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f64142ad0 con 0x7f2f64096f70 2024-04-17T07:34:00.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.929+0000 7f2f6b400640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f64097930 0x7f2f64142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:34:00.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.929+0000 7f2f6a000640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f64096f70 0x7f2f6412a260 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c920 tx=0x7f2f5c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.930+0000 7f2f63400640 1 -- 172.21.15.86:0/4214872250 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c007c70 con 0x7f2f64096f70 2024-04-17T07:34:00.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.930+0000 7f2f6be00640 1 -- 172.21.15.86:0/4214872250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f64142ca0 con 0x7f2f64096f70 2024-04-17T07:34:00.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.930+0000 7f2f6be00640 1 -- 172.21.15.86:0/4214872250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f64143190 con 0x7f2f64096f70 2024-04-17T07:34:00.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.930+0000 7f2f63400640 1 -- 172.21.15.86:0/4214872250 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f5c007e10 con 0x7f2f64096f70 2024-04-17T07:34:00.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.931+0000 7f2f63400640 1 -- 172.21.15.86:0/4214872250 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00e7a0 con 0x7f2f64096f70 2024-04-17T07:34:00.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.936+0000 7f2f63400640 1 -- 172.21.15.86:0/4214872250 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2f5c00ea80 con 0x7f2f64096f70 2024-04-17T07:34:00.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.939+0000 7f2f63400640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f500a3d00 0x7f2f500a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.939+0000 7f2f63400640 1 -- 172.21.15.86:0/4214872250 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2f5c0d1780 con 0x7f2f64096f70 2024-04-17T07:34:00.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.940+0000 7f2f6aa00640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f500a3d00 0x7f2f500a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.941+0000 7f2f6aa00640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f500a3d00 0x7f2f500a61c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c067ad0 tx=0x7f2f6c050d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.942+0000 7f2f74447740 1 Processor -- start 2024-04-17T07:34:00.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.943+0000 7f2f74447740 1 -- start start 2024-04-17T07:34:00.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.943+0000 7f2f74447740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 0x564f49476580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.943+0000 7f2f74447740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476b50 0x564f4947b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.943+0000 7f2f74447740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f4947bcc0 0x564f4947e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.943+0000 7f2f74447740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f493dc020 con 0x564f49476b50 2024-04-17T07:34:00.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.943+0000 7f2f74447740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564f49382460 con 0x564f49476160 2024-04-17T07:34:00.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.943+0000 7f2f74447740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f493d7fe0 con 0x564f4947bcc0 2024-04-17T07:34:00.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.944+0000 7f2f6aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 0x564f49476580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.944+0000 7f2f6aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 0x564f49476580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52180/0 (socket says 172.21.15.86:52180) 2024-04-17T07:34:00.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.944+0000 7f2f6aa00640 1 -- 172.21.15.86:0/2334595518 learned_addr learned my addr 172.21.15.86:0/2334595518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:00.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.944+0000 7f2f6a000640 1 --2- 172.21.15.86:0/2334595518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476b50 0x564f4947b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.944+0000 7f2f6b400640 1 --2- 172.21.15.86:0/2334595518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f4947bcc0 0x564f4947e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.945+0000 7f2f6a000640 1 -- 172.21.15.86:0/2334595518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f4947bcc0 msgr2=0x564f4947e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.945+0000 7f2f6a000640 1 --2- 172.21.15.86:0/2334595518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f4947bcc0 0x564f4947e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.945+0000 7f2f6a000640 1 -- 172.21.15.86:0/2334595518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 msgr2=0x564f49476580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.945+0000 7f2f6a000640 1 --2- 172.21.15.86:0/2334595518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 0x564f49476580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.945+0000 7f2f6a000640 1 -- 172.21.15.86:0/2334595518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4938a040 con 0x564f49476b50 2024-04-17T07:34:00.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.945+0000 7f2f6a000640 1 --2- 172.21.15.86:0/2334595518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476b50 0x564f4947b650 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0cef00 tx=0x7f2f5c09d100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ed5e1fcbca0e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.945+0000 7f2f57e00640 1 -- 172.21.15.86:0/2334595518 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0b6e60 con 0x564f49476b50 2024-04-17T07:34:00.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.946+0000 7f2f57e00640 1 -- 172.21.15.86:0/2334595518 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f5c0b3070 con 0x564f49476b50 2024-04-17T07:34:00.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.946+0000 7f2f57e00640 1 -- 172.21.15.86:0/2334595518 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c099d50 con 0x564f49476b50 2024-04-17T07:34:00.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.946+0000 7f2f74447740 1 -- 172.21.15.86:0/2334595518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476b50 msgr2=0x564f4947b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.946+0000 7f2f74447740 1 --2- 172.21.15.86:0/2334595518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476b50 0x564f4947b650 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0cef00 tx=0x7f2f5c09d100 comp rx=0 tx=0).stop 2024-04-17T07:34:00.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.947+0000 7f2f74447740 1 -- 172.21.15.86:0/2334595518 shutdown_connections 2024-04-17T07:34:00.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.947+0000 7f2f74447740 1 --2- 172.21.15.86:0/2334595518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f4947bcc0 0x564f4947e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.947+0000 7f2f74447740 1 --2- 172.21.15.86:0/2334595518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476b50 0x564f4947b650 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.947+0000 7f2f74447740 1 --2- 172.21.15.86:0/2334595518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 0x564f49476580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.947+0000 7f2f74447740 1 -- 172.21.15.86:0/2334595518 >> 172.21.15.86:0/2334595518 conn(0x564f494716d0 msgr2=0x564f49471ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.947+0000 7f2f74447740 1 -- 172.21.15.86:0/2334595518 shutdown_connections 2024-04-17T07:34:00.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.948+0000 7f2f74447740 1 -- 172.21.15.86:0/2334595518 wait complete. 2024-04-17T07:34:00.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.948+0000 7f2f74447740 1 -- start start 2024-04-17T07:34:00.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.949+0000 7f2f74447740 1 Processor -- start 2024-04-17T07:34:00.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.950+0000 7f2f74447740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476160 0x564f494806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.950+0000 7f2f74447740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476b50 0x564f49480be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.950+0000 7f2f74447740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 0x564f49487c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.950+0000 7f2f74447740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f4947ec70 con 0x564f4947bcc0 2024-04-17T07:34:00.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.950+0000 7f2f74447740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564f49395830 con 0x564f49476b50 2024-04-17T07:34:00.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.950+0000 7f2f74447740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f4947edf0 con 0x564f49476160 2024-04-17T07:34:00.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.950+0000 7f2f74447740 10 client.0 _ll_register_callbacks cb 0x564f49363fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:00.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.951+0000 7f2f6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 0x564f49487c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.951+0000 7f2f6aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476160 0x564f494806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.951+0000 7f2f6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 0x564f49487c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38950/0 (socket says 172.21.15.86:38950) 2024-04-17T07:34:00.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.951+0000 7f2f6b400640 1 -- 172.21.15.86:0/2281185671 learned_addr learned my addr 172.21.15.86:0/2281185671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:00.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.952+0000 7f2f6b400640 1 -- 172.21.15.86:0/2281185671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476160 msgr2=0x564f494806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:00.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.952+0000 7f2f6b400640 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476160 0x564f494806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.952+0000 7f2f6b400640 1 -- 172.21.15.86:0/2281185671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476b50 msgr2=0x564f49480be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T07:34:00.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.952+0000 7f2f6b400640 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476b50 0x564f49480be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.952+0000 7f2f6b400640 1 -- 172.21.15.86:0/2281185671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f49399920 con 0x564f4947bcc0 2024-04-17T07:34:00.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.953+0000 7f2f6b400640 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 0x564f49487c20 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2f58036000 tx=0x7f2f5802cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fce97ee698eaec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:00.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.953+0000 7f2f54c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:00.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.955+0000 7f2f54c00640 1 -- 172.21.15.86:0/2281185671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58045020 con 0x564f4947bcc0 2024-04-17T07:34:00.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.955+0000 7f2f54c00640 1 -- 172.21.15.86:0/2281185671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2f58004050 con 0x564f4947bcc0 2024-04-17T07:34:00.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.956+0000 7f2f74447740 1 -- 172.21.15.86:0/2281185671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f5c0b3070 con 0x564f4947bcc0 2024-04-17T07:34:00.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.956+0000 7f2f54c00640 1 -- 172.21.15.86:0/2281185671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f58039040 con 0x564f4947bcc0 2024-04-17T07:34:00.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.956+0000 7f2f74447740 10 client.19842 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:00.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.956+0000 7f2f74447740 1 -- 172.21.15.86:0/2281185671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f493677f0 con 0x564f4947bcc0 2024-04-17T07:34:00.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.957+0000 7f2f54c00640 1 -- 172.21.15.86:0/2281185671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f5c0b3070 con 0x564f4947bcc0 2024-04-17T07:34:00.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.957+0000 7f2f54c00640 1 -- 172.21.15.86:0/2281185671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2f5803f040 con 0x564f4947bcc0 2024-04-17T07:34:00.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.958+0000 7f2f74447740 10 client.19842 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:00.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.958+0000 7f2f74447740 10 client.19842 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:34:00.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.958+0000 7f2f74447740 20 client.19842 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:00.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.958+0000 7f2f74447740 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f4948c260 0x564f4948e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:00.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.958+0000 7f2f74447740 4 client.19842 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:00.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.958+0000 7f2f74447740 1 -- 172.21.15.86:0/2281185671 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f493e9a40 con 0x564f4948c260 2024-04-17T07:34:00.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.959+0000 7f2f6aa00640 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f4948c260 0x564f4948e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:00.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.960+0000 7f2f6aa00640 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f4948c260 0x564f4948e6a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b01311cca7a18c69 server_cookie=21297f90524634f0 in_seq=0 out_seq=0 2024-04-17T07:34:00.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.960+0000 7f2f54c00640 10 client.19842 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:34:00.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.962+0000 7f2f54c00640 1 -- 172.21.15.86:0/2281185671 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f493e9a40 con 0x564f4948c260 2024-04-17T07:34:00.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.962+0000 7f2f54c00640 10 client.19842 handle_command_reply: tid=0 2024-04-17T07:34:00.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.962+0000 7f2f74447740 1 client.19842 shutdown 2024-04-17T07:34:00.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.963+0000 7f2f74447740 1 -- 172.21.15.86:0/2281185671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 msgr2=0x564f49487c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:00.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.963+0000 7f2f74447740 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 0x564f49487c20 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2f58036000 tx=0x7f2f5802cd90 comp rx=0 tx=0).stop 2024-04-17T07:34:00.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.963+0000 7f2f74447740 1 -- 172.21.15.86:0/2281185671 shutdown_connections 2024-04-17T07:34:00.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.963+0000 7f2f74447740 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 0x564f49487c20 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.963+0000 7f2f74447740 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f4948c260 0x564f4948e6a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.963+0000 7f2f74447740 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476b50 0x564f49480be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.963+0000 7f2f74447740 1 --2- 172.21.15.86:0/2281185671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476160 0x564f494806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:00.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.963+0000 7f2f74447740 1 -- 172.21.15.86:0/2281185671 >> 172.21.15.86:0/2281185671 conn(0x564f494716d0 msgr2=0x564f49470930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:00.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.963+0000 7f2f54c00640 0 client.19842 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:34:00.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.964+0000 7f2f74447740 1 -- 172.21.15.86:0/2281185671 shutdown_connections 2024-04-17T07:34:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.964+0000 7f2f74447740 1 -- 172.21.15.86:0/2281185671 wait complete. 2024-04-17T07:34:00.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:00.965+0000 7f2f74447740 20 client.19842 trim_cache size 0 max 16384 2024-04-17T07:34:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.002+0000 7f2f74447740 1 Processor -- start 2024-04-17T07:34:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.003+0000 7f2f74447740 1 -- start start 2024-04-17T07:34:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.003+0000 7f2f74447740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476160 0x564f4947f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.003+0000 7f2f74447740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 0x564f49487b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.003+0000 7f2f74447740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f4947bcc0 0x564f49471ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.003+0000 7f2f74447740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c00fc60 con 0x564f49476160 2024-04-17T07:34:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.003+0000 7f2f74447740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c0117a0 con 0x564f4947bcc0 2024-04-17T07:34:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.003+0000 7f2f74447740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c010330 con 0x564f49476b50 2024-04-17T07:34:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.004+0000 7f2f6b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f4947bcc0 0x564f49471ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.004+0000 7f2f6b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f4947bcc0 0x564f49471ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52192/0 (socket says 172.21.15.86:52192) 2024-04-17T07:34:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.004+0000 7f2f6b400640 1 -- 172.21.15.86:0/4233521329 learned_addr learned my addr 172.21.15.86:0/4233521329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.005+0000 7f2f6aa00640 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476160 0x564f4947f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.005+0000 7f2f6a000640 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 0x564f49487b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.005+0000 7f2f6b400640 1 -- 172.21.15.86:0/4233521329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 msgr2=0x564f49487b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.005+0000 7f2f6b400640 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 0x564f49487b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.005+0000 7f2f6b400640 1 -- 172.21.15.86:0/4233521329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476160 msgr2=0x564f4947f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.005+0000 7f2f6b400640 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476160 0x564f4947f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.005+0000 7f2f6b400640 1 -- 172.21.15.86:0/4233521329 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f490ee3f0 con 0x564f4947bcc0 2024-04-17T07:34:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.005+0000 7f2f6a000640 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 0x564f49487b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.005+0000 7f2f6aa00640 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476160 0x564f4947f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:01.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.006+0000 7f2f6b400640 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f4947bcc0 0x564f49471ab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2f5802c0b0 tx=0x7f2f58005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c91008035376d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:01.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.007+0000 7f2f57400640 1 -- 172.21.15.86:0/4233521329 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58057070 con 0x564f4947bcc0 2024-04-17T07:34:01.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.007+0000 7f2f57400640 1 -- 172.21.15.86:0/4233521329 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f5803a040 con 0x564f4947bcc0 2024-04-17T07:34:01.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.007+0000 7f2f57400640 1 -- 172.21.15.86:0/4233521329 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58057070 con 0x564f4947bcc0 2024-04-17T07:34:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.009+0000 7f2f74447740 1 -- 172.21.15.86:0/4233521329 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f4947bcc0 msgr2=0x564f49471ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.009+0000 7f2f74447740 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f4947bcc0 0x564f49471ab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2f5802c0b0 tx=0x7f2f58005c90 comp rx=0 tx=0).stop 2024-04-17T07:34:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.010+0000 7f2f74447740 1 -- 172.21.15.86:0/4233521329 shutdown_connections 2024-04-17T07:34:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.010+0000 7f2f74447740 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f4947bcc0 0x564f49471ab0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.010+0000 7f2f74447740 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 0x564f49487b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.010+0000 7f2f74447740 1 --2- 172.21.15.86:0/4233521329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f49476160 0x564f4947f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.010+0000 7f2f74447740 1 -- 172.21.15.86:0/4233521329 >> 172.21.15.86:0/4233521329 conn(0x564f494716d0 msgr2=0x564f494775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.010+0000 7f2f74447740 1 -- 172.21.15.86:0/4233521329 shutdown_connections 2024-04-17T07:34:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.010+0000 7f2f74447740 1 -- 172.21.15.86:0/4233521329 wait complete. 2024-04-17T07:34:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.011+0000 7f2f74447740 1 -- start start 2024-04-17T07:34:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.011+0000 7f2f74447740 1 Processor -- start 2024-04-17T07:34:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.013+0000 7f2f74447740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 0x564f49484aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.013+0000 7f2f74447740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 0x564f49484fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.013+0000 7f2f74447740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 0x564f494f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.013+0000 7f2f74447740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c0110d0 con 0x564f4947bcc0 2024-04-17T07:34:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.013+0000 7f2f74447740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c010a00 con 0x564f49476160 2024-04-17T07:34:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.013+0000 7f2f74447740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c00dbf0 con 0x564f49476b50 2024-04-17T07:34:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.013+0000 7f2f74447740 10 client.0 _ll_register_callbacks cb 0x7f2f5802c510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.013+0000 7f2f6a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 0x564f49484fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.013+0000 7f2f6a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 0x564f49484fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54132/0 (socket says 172.21.15.86:54132) 2024-04-17T07:34:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.013+0000 7f2f6a000640 1 -- 172.21.15.86:0/1040478240 learned_addr learned my addr 172.21.15.86:0/1040478240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.014+0000 7f2f6b400640 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 0x564f494f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.014+0000 7f2f6aa00640 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 0x564f49484aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.014+0000 7f2f6aa00640 1 -- 172.21.15.86:0/1040478240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 msgr2=0x564f49484fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.014+0000 7f2f6aa00640 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 0x564f49484fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.014+0000 7f2f6aa00640 1 -- 172.21.15.86:0/1040478240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 msgr2=0x564f494f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.014+0000 7f2f6aa00640 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 0x564f494f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.015+0000 7f2f6aa00640 1 -- 172.21.15.86:0/1040478240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f491b87d0 con 0x564f49476160 2024-04-17T07:34:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.015+0000 7f2f6aa00640 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 0x564f49484aa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c06b950 tx=0x7f2f6c067e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d096612dbf3607bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.015+0000 7f2f54c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:34:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.016+0000 7f2f54c00640 1 -- 172.21.15.86:0/1040478240 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c06bb80 con 0x564f49476160 2024-04-17T07:34:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.016+0000 7f2f54c00640 1 -- 172.21.15.86:0/1040478240 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2f6c094250 con 0x564f49476160 2024-04-17T07:34:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.016+0000 7f2f74447740 1 -- 172.21.15.86:0/1040478240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f5803a040 con 0x564f49476160 2024-04-17T07:34:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.017+0000 7f2f54c00640 1 -- 172.21.15.86:0/1040478240 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f6c094450 con 0x564f49476160 2024-04-17T07:34:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.017+0000 7f2f74447740 10 client.38909 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.017+0000 7f2f74447740 1 -- 172.21.15.86:0/1040478240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f49502ed0 con 0x564f49476160 2024-04-17T07:34:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.018+0000 7f2f54c00640 1 -- 172.21.15.86:0/1040478240 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f5803a040 con 0x564f49476160 2024-04-17T07:34:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.018+0000 7f2f54c00640 1 -- 172.21.15.86:0/1040478240 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2f6c068a70 con 0x564f49476160 2024-04-17T07:34:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.018+0000 7f2f74447740 10 client.38909 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.018+0000 7f2f74447740 10 client.38909 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:34:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.018+0000 7f2f74447740 20 client.38909 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.018+0000 7f2f74447740 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f495038d0 0x564f49505cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.018+0000 7f2f74447740 4 client.38909 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:34:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.018+0000 7f2f74447740 1 -- 172.21.15.86:0/1040478240 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f493e38b0 con 0x564f495038d0 2024-04-17T07:34:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.019+0000 7f2f6a000640 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f495038d0 0x564f49505cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.020+0000 7f2f6a000640 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f495038d0 0x564f49505cf0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=176dbf2fee8f2d8f server_cookie=679843af5ca7d017 in_seq=0 out_seq=0 2024-04-17T07:34:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.020+0000 7f2f54c00640 10 client.38909 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:34:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.021+0000 7f2f54c00640 1 -- 172.21.15.86:0/1040478240 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f493e38b0 con 0x564f495038d0 2024-04-17T07:34:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.021+0000 7f2f54c00640 10 client.38909 handle_command_reply: tid=0 2024-04-17T07:34:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.021+0000 7f2f74447740 1 client.38909 shutdown 2024-04-17T07:34:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.024+0000 7f2f74447740 1 -- 172.21.15.86:0/1040478240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 msgr2=0x564f49484aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.024+0000 7f2f74447740 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 0x564f49484aa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c06b950 tx=0x7f2f6c067e30 comp rx=0 tx=0).stop 2024-04-17T07:34:01.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.025+0000 7f2f74447740 1 -- 172.21.15.86:0/1040478240 shutdown_connections 2024-04-17T07:34:01.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.025+0000 7f2f74447740 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f4947bcc0 0x564f494f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.025+0000 7f2f74447740 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f49476b50 0x564f49484fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.025+0000 7f2f74447740 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f495038d0 0x564f49505cf0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.025+0000 7f2f74447740 1 --2- 172.21.15.86:0/1040478240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f49476160 0x564f49484aa0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.025+0000 7f2f74447740 1 -- 172.21.15.86:0/1040478240 >> 172.21.15.86:0/1040478240 conn(0x564f494716d0 msgr2=0x564f4947f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:01.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.025+0000 7f2f54c00640 0 client.38909 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:34:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.025+0000 7f2f74447740 1 -- 172.21.15.86:0/1040478240 shutdown_connections 2024-04-17T07:34:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.025+0000 7f2f74447740 1 -- 172.21.15.86:0/1040478240 wait complete. 2024-04-17T07:34:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.026+0000 7f2f74447740 20 client.38909 trim_cache size 0 max 16384 2024-04-17T07:34:01.027 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:34:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.027+0000 7f2f6be00640 1 -- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f500a3d00 msgr2=0x7f2f500a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.027+0000 7f2f6be00640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f500a3d00 0x7f2f500a61c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c067ad0 tx=0x7f2f6c050d10 comp rx=0 tx=0).stop 2024-04-17T07:34:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.027+0000 7f2f6be00640 1 -- 172.21.15.86:0/4214872250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f64096f70 msgr2=0x7f2f6412a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.027+0000 7f2f6be00640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f64096f70 0x7f2f6412a260 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c920 tx=0x7f2f5c00cde0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.027+0000 7f2f6be00640 1 -- 172.21.15.86:0/4214872250 shutdown_connections 2024-04-17T07:34:01.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.027+0000 7f2f6be00640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2f500a3d00 0x7f2f500a61c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.027+0000 7f2f6be00640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f64097930 0x7f2f64142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.027+0000 7f2f6be00640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f64096f70 0x7f2f6412a260 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.027+0000 7f2f6be00640 1 --2- 172.21.15.86:0/4214872250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f64095d80 0x7f2f64129d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.027+0000 7f2f6be00640 1 -- 172.21.15.86:0/4214872250 >> 172.21.15.86:0/4214872250 conn(0x7f2f640912d0 msgr2=0x7f2f6409a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:01.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.028+0000 7f2f6be00640 1 -- 172.21.15.86:0/4214872250 shutdown_connections 2024-04-17T07:34:01.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.028+0000 7f2f6be00640 1 -- 172.21.15.86:0/4214872250 wait complete. 2024-04-17T07:34:01.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:34:01.194 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:00 smithi086 ceph-mon[31733]: pgmap v1029: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.6 MiB/s wr, 552 op/s 2024-04-17T07:34:01.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:00 smithi098 ceph-mon[35418]: pgmap v1029: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.6 MiB/s wr, 552 op/s 2024-04-17T07:34:01.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:00 smithi103 ceph-mon[35495]: pgmap v1029: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.6 MiB/s wr, 552 op/s 2024-04-17T07:34:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.431+0000 7f1786400640 1 -- 172.21.15.86:0/525321316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f178010a170 msgr2=0x7f1780110a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.431+0000 7f1786400640 1 --2- 172.21.15.86:0/525321316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f178010a170 0x7f1780110a00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1778009be0 tx=0x7f177802bef0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.431+0000 7f1786400640 1 -- 172.21.15.86:0/525321316 shutdown_connections 2024-04-17T07:34:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.431+0000 7f1786400640 1 --2- 172.21.15.86:0/525321316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f178010a170 0x7f1780110a00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.431+0000 7f1786400640 1 --2- 172.21.15.86:0/525321316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17801097b0 0x7f1780109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.431+0000 7f1786400640 1 --2- 172.21.15.86:0/525321316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17801085c0 0x7f17801089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.431+0000 7f1786400640 1 -- 172.21.15.86:0/525321316 >> 172.21.15.86:0/525321316 conn(0x7f1780077560 msgr2=0x7f17800799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.432+0000 7f1786400640 1 -- 172.21.15.86:0/525321316 shutdown_connections 2024-04-17T07:34:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.432+0000 7f1786400640 1 -- 172.21.15.86:0/525321316 wait complete. 2024-04-17T07:34:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.433+0000 7f1786400640 1 Processor -- start 2024-04-17T07:34:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.433+0000 7f1786400640 1 -- start start 2024-04-17T07:34:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.434+0000 7f1786400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17801085c0 0x7f178019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.434+0000 7f1786400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17801097b0 0x7f178019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.434+0000 7f1786400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178010a170 0x7f17801a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.434+0000 7f1786400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17801140d0 con 0x7f178010a170 2024-04-17T07:34:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.434+0000 7f1786400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1780113f50 con 0x7f17801097b0 2024-04-17T07:34:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.434+0000 7f1786400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1780114250 con 0x7f17801085c0 2024-04-17T07:34:01.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.435+0000 7f177ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17801097b0 0x7f178019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.435+0000 7f177ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17801097b0 0x7f178019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52222/0 (socket says 172.21.15.86:52222) 2024-04-17T07:34:01.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.435+0000 7f177ea00640 1 -- 172.21.15.86:0/1976630544 learned_addr learned my addr 172.21.15.86:0/1976630544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:01.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.435+0000 7f177f400640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17801085c0 0x7f178019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.435+0000 7f177fe00640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178010a170 0x7f17801a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.435+0000 7f177fe00640 1 -- 172.21.15.86:0/1976630544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17801085c0 msgr2=0x7f178019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.435+0000 7f177fe00640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17801085c0 0x7f178019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.435+0000 7f177fe00640 1 -- 172.21.15.86:0/1976630544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17801097b0 msgr2=0x7f178019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.435+0000 7f177fe00640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17801097b0 0x7f178019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.435+0000 7f177fe00640 1 -- 172.21.15.86:0/1976630544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17801a63e0 con 0x7f178010a170 2024-04-17T07:34:01.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.436+0000 7f177fe00640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178010a170 0x7f17801a5d70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f1778036040 tx=0x7f1778003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.438+0000 7f1777e00640 1 -- 172.21.15.86:0/1976630544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778043020 con 0x7f178010a170 2024-04-17T07:34:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.438+0000 7f1777e00640 1 -- 172.21.15.86:0/1976630544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1778034c10 con 0x7f178010a170 2024-04-17T07:34:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.438+0000 7f1777e00640 1 -- 172.21.15.86:0/1976630544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778004860 con 0x7f178010a170 2024-04-17T07:34:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.438+0000 7f1786400640 1 -- 172.21.15.86:0/1976630544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17801a6670 con 0x7f178010a170 2024-04-17T07:34:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.438+0000 7f1786400640 1 -- 172.21.15.86:0/1976630544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17801a6be0 con 0x7f178010a170 2024-04-17T07:34:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.444+0000 7f178922d740 1 Processor -- start 2024-04-17T07:34:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.445+0000 7f1777e00640 1 -- 172.21.15.86:0/1976630544 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1778046020 con 0x7f178010a170 2024-04-17T07:34:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.446+0000 7f178922d740 1 -- start start 2024-04-17T07:34:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.447+0000 7f178922d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 0x556a4f6263c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.447+0000 7f178922d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f626990 0x556a4f62b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.447+0000 7f178922d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f62df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.447+0000 7f178922d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a4f58c020 con 0x556a4f62bb00 2024-04-17T07:34:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.447+0000 7f178922d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556a4f532460 con 0x556a4f625fa0 2024-04-17T07:34:01.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.447+0000 7f178922d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a4f587fe0 con 0x556a4f626990 2024-04-17T07:34:01.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.447+0000 7f1777e00640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17640a3c80 0x7f17640a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.447+0000 7f177ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f62df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f62df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38978/0 (socket says 172.21.15.86:38978) 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177ea00640 1 -- 172.21.15.86:0/2749854862 learned_addr learned my addr 172.21.15.86:0/2749854862 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177ea00640 1 --2- 172.21.15.86:0/2749854862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 0x556a4f6263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177fe00640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17640a3c80 0x7f17640a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f1777e00640 1 -- 172.21.15.86:0/1976630544 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f17780f2e00 con 0x7f178010a170 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177f400640 1 --2- 172.21.15.86:0/2749854862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f626990 0x556a4f62b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177ea00640 1 -- 172.21.15.86:0/2749854862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f626990 msgr2=0x556a4f62b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177ea00640 1 --2- 172.21.15.86:0/2749854862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f626990 0x556a4f62b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177ea00640 1 -- 172.21.15.86:0/2749854862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 msgr2=0x556a4f6263c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177ea00640 1 --2- 172.21.15.86:0/2749854862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 0x556a4f6263c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177ea00640 1 -- 172.21.15.86:0/2749854862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a4f53a040 con 0x556a4f62bb00 2024-04-17T07:34:01.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.448+0000 7f177f400640 1 --2- 172.21.15.86:0/2749854862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f626990 0x556a4f62b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.449+0000 7f177ea00640 1 --2- 172.21.15.86:0/2749854862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f62df90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f176c004650 tx=0x7f176c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ee7a926ba9d2f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.449+0000 7f177fe00640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17640a3c80 0x7f17640a6140 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f177804e000 tx=0x7f177802bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.449+0000 7f1774c00640 1 -- 172.21.15.86:0/2749854862 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c00f7c0 con 0x556a4f62bb00 2024-04-17T07:34:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.450+0000 7f1774c00640 1 -- 172.21.15.86:0/2749854862 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f176c00fe10 con 0x556a4f62bb00 2024-04-17T07:34:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.451+0000 7f1774c00640 1 -- 172.21.15.86:0/2749854862 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c01fbc0 con 0x556a4f62bb00 2024-04-17T07:34:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.451+0000 7f178922d740 1 -- 172.21.15.86:0/2749854862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 msgr2=0x556a4f62df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.451+0000 7f178922d740 1 --2- 172.21.15.86:0/2749854862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f62df90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f176c004650 tx=0x7f176c004730 comp rx=0 tx=0).stop 2024-04-17T07:34:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.451+0000 7f178922d740 1 -- 172.21.15.86:0/2749854862 shutdown_connections 2024-04-17T07:34:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.451+0000 7f178922d740 1 --2- 172.21.15.86:0/2749854862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f62df90 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.451+0000 7f178922d740 1 --2- 172.21.15.86:0/2749854862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f626990 0x556a4f62b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.451+0000 7f178922d740 1 --2- 172.21.15.86:0/2749854862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 0x556a4f6263c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.451+0000 7f178922d740 1 -- 172.21.15.86:0/2749854862 >> 172.21.15.86:0/2749854862 conn(0x556a4f621510 msgr2=0x556a4f621910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.451+0000 7f178922d740 1 -- 172.21.15.86:0/2749854862 shutdown_connections 2024-04-17T07:34:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.451+0000 7f178922d740 1 -- 172.21.15.86:0/2749854862 wait complete. 2024-04-17T07:34:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.452+0000 7f178922d740 1 -- start start 2024-04-17T07:34:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.452+0000 7f178922d740 1 Processor -- start 2024-04-17T07:34:01.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f178922d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f625fa0 0x556a4f630540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f178922d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 0x556a4f630a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f178922d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f62bb00 0x556a4f637ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f178922d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a4f62eb50 con 0x556a4f625fa0 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f178922d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556a4f545830 con 0x556a4f626990 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f178922d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a4f62ecd0 con 0x556a4f62bb00 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f178922d740 10 client.0 _ll_register_callbacks cb 0x556a4f513fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f177ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f625fa0 0x556a4f630540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f177f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 0x556a4f630a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f177ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f625fa0 0x556a4f630540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38980/0 (socket says 172.21.15.86:38980) 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f177ea00640 1 -- 172.21.15.86:0/3702847359 learned_addr learned my addr 172.21.15.86:0/3702847359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:01.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.453+0000 7f177ea00640 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f62bb00 0x556a4f637ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.454+0000 7f177ea00640 1 -- 172.21.15.86:0/3702847359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f62bb00 msgr2=0x556a4f637ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.454+0000 7f177ea00640 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f62bb00 0x556a4f637ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.454+0000 7f177ea00640 1 -- 172.21.15.86:0/3702847359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 msgr2=0x556a4f630a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.454+0000 7f177ea00640 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 0x556a4f630a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.454+0000 7f177ea00640 1 -- 172.21.15.86:0/3702847359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a4f549920 con 0x556a4f625fa0 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.454+0000 7f177ea00640 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f625fa0 0x556a4f630540 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f176c004620 tx=0x7f176c009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0ebdce1364aa85 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.454+0000 7f1769600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.455+0000 7f1769600640 1 -- 172.21.15.86:0/3702847359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c01fbd0 con 0x556a4f625fa0 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.455+0000 7f1769600640 1 -- 172.21.15.86:0/3702847359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f176c027040 con 0x556a4f625fa0 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.455+0000 7f178922d740 1 -- 172.21.15.86:0/3702847359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176c00fe10 con 0x556a4f625fa0 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.455+0000 7f1769600640 1 -- 172.21.15.86:0/3702847359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f176c01fd70 con 0x556a4f625fa0 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.455+0000 7f178922d740 10 client.19860 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.455+0000 7f178922d740 1 -- 172.21.15.86:0/3702847359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a4f5177f0 con 0x556a4f625fa0 2024-04-17T07:34:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.456+0000 7f1769600640 1 -- 172.21.15.86:0/3702847359 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f176c00fe10 con 0x556a4f625fa0 2024-04-17T07:34:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.456+0000 7f1769600640 1 -- 172.21.15.86:0/3702847359 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f176c01d020 con 0x556a4f625fa0 2024-04-17T07:34:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.456+0000 7f178922d740 10 client.19860 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.456+0000 7f178922d740 10 client.19860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.456+0000 7f178922d740 20 client.19860 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.456+0000 7f178922d740 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a4f63bc50 0x556a4f63e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.456+0000 7f178922d740 4 client.19860 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.456+0000 7f178922d740 1 -- 172.21.15.86:0/3702847359 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a4f599a40 con 0x556a4f63bc50 2024-04-17T07:34:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.457+0000 7f177f400640 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a4f63bc50 0x556a4f63e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.459+0000 7f177f400640 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a4f63bc50 0x556a4f63e090 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2bf18e274fce41b server_cookie=53d1a7ff03936949 in_seq=0 out_seq=0 2024-04-17T07:34:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.459+0000 7f1769600640 10 client.19860 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.460+0000 7f1769600640 1 -- 172.21.15.86:0/3702847359 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a4f599a40 con 0x556a4f63bc50 2024-04-17T07:34:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.460+0000 7f1769600640 10 client.19860 handle_command_reply: tid=0 2024-04-17T07:34:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.461+0000 7f178922d740 1 client.19860 shutdown 2024-04-17T07:34:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.461+0000 7f178922d740 1 -- 172.21.15.86:0/3702847359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f625fa0 msgr2=0x556a4f630540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.461+0000 7f178922d740 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f625fa0 0x556a4f630540 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f176c004620 tx=0x7f176c009530 comp rx=0 tx=0).stop 2024-04-17T07:34:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.461+0000 7f178922d740 1 -- 172.21.15.86:0/3702847359 shutdown_connections 2024-04-17T07:34:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.462+0000 7f178922d740 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a4f63bc50 0x556a4f63e090 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.462+0000 7f178922d740 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f62bb00 0x556a4f637ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.462+0000 7f178922d740 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 0x556a4f630a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.462+0000 7f178922d740 1 --2- 172.21.15.86:0/3702847359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f625fa0 0x556a4f630540 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.462+0000 7f178922d740 1 -- 172.21.15.86:0/3702847359 >> 172.21.15.86:0/3702847359 conn(0x556a4f621510 msgr2=0x556a4f620710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.462+0000 7f1769600640 0 client.19860 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.462+0000 7f178922d740 1 -- 172.21.15.86:0/3702847359 shutdown_connections 2024-04-17T07:34:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.462+0000 7f178922d740 1 -- 172.21.15.86:0/3702847359 wait complete. 2024-04-17T07:34:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.462+0000 7f178922d740 20 client.19860 trim_cache size 0 max 16384 2024-04-17T07:34:01.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.499+0000 7f178922d740 1 Processor -- start 2024-04-17T07:34:01.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.499+0000 7f178922d740 1 -- start start 2024-04-17T07:34:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.500+0000 7f178922d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 0x556a4f630150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.500+0000 7f178922d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f626990 0x556a4f637a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.500+0000 7f178922d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f62bb00 0x556a4f630690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.500+0000 7f178922d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17801b10c0 con 0x556a4f626990 2024-04-17T07:34:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.500+0000 7f178922d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17801b2c00 con 0x556a4f625fa0 2024-04-17T07:34:01.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.500+0000 7f178922d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17801b1790 con 0x556a4f62bb00 2024-04-17T07:34:01.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.501+0000 7f177ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 0x556a4f630150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.501+0000 7f177f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f626990 0x556a4f637a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.501+0000 7f177ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 0x556a4f630150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52246/0 (socket says 172.21.15.86:52246) 2024-04-17T07:34:01.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.501+0000 7f177ea00640 1 -- 172.21.15.86:0/3107377952 learned_addr learned my addr 172.21.15.86:0/3107377952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:01.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.501+0000 7f177ea00640 1 --2- 172.21.15.86:0/3107377952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f62bb00 0x556a4f630690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.502+0000 7f177ea00640 1 -- 172.21.15.86:0/3107377952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f62bb00 msgr2=0x556a4f630690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.502+0000 7f177ea00640 1 --2- 172.21.15.86:0/3107377952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f62bb00 0x556a4f630690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.502+0000 7f177ea00640 1 -- 172.21.15.86:0/3107377952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f626990 msgr2=0x556a4f637a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.502+0000 7f177ea00640 1 --2- 172.21.15.86:0/3107377952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f626990 0x556a4f637a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.502+0000 7f177ea00640 1 -- 172.21.15.86:0/3107377952 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a4f29e3f0 con 0x556a4f625fa0 2024-04-17T07:34:01.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.503+0000 7f177ea00640 1 --2- 172.21.15.86:0/3107377952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 0x556a4f630150 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f176c011630 tx=0x7f176c01f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1051de349c5f6b11 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:01.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.503+0000 7f176be00640 1 -- 172.21.15.86:0/3107377952 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c009860 con 0x556a4f625fa0 2024-04-17T07:34:01.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.504+0000 7f176be00640 1 -- 172.21.15.86:0/3107377952 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f176c01f550 con 0x556a4f625fa0 2024-04-17T07:34:01.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.504+0000 7f176be00640 1 -- 172.21.15.86:0/3107377952 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c02aaa0 con 0x556a4f625fa0 2024-04-17T07:34:01.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.505+0000 7f178922d740 1 -- 172.21.15.86:0/3107377952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 msgr2=0x556a4f630150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.505+0000 7f178922d740 1 --2- 172.21.15.86:0/3107377952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 0x556a4f630150 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f176c011630 tx=0x7f176c01f440 comp rx=0 tx=0).stop 2024-04-17T07:34:01.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.505+0000 7f178922d740 1 -- 172.21.15.86:0/3107377952 shutdown_connections 2024-04-17T07:34:01.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.505+0000 7f178922d740 1 --2- 172.21.15.86:0/3107377952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f62bb00 0x556a4f630690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.505+0000 7f178922d740 1 --2- 172.21.15.86:0/3107377952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f626990 0x556a4f637a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.505+0000 7f178922d740 1 --2- 172.21.15.86:0/3107377952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f625fa0 0x556a4f630150 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.505+0000 7f178922d740 1 -- 172.21.15.86:0/3107377952 >> 172.21.15.86:0/3107377952 conn(0x556a4f621510 msgr2=0x556a4f627db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:01.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.505+0000 7f178922d740 1 -- 172.21.15.86:0/3107377952 shutdown_connections 2024-04-17T07:34:01.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.505+0000 7f178922d740 1 -- 172.21.15.86:0/3107377952 wait complete. 2024-04-17T07:34:01.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.506+0000 7f178922d740 1 -- start start 2024-04-17T07:34:01.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.506+0000 7f178922d740 1 Processor -- start 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.508+0000 7f178922d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f625fa0 0x556a4f634930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.508+0000 7f178922d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 0x556a4f634e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.508+0000 7f178922d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f6a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.508+0000 7f178922d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17801b2530 con 0x556a4f62bb00 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.508+0000 7f178922d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17801b1e60 con 0x556a4f626990 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.508+0000 7f178922d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17801af090 con 0x556a4f625fa0 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.508+0000 7f178922d740 10 client.0 _ll_register_callbacks cb 0x7f176c00fba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.509+0000 7f177ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f625fa0 0x556a4f634930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.509+0000 7f177ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f625fa0 0x556a4f634930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54190/0 (socket says 172.21.15.86:54190) 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.509+0000 7f177ea00640 1 -- 172.21.15.86:0/2772495785 learned_addr learned my addr 172.21.15.86:0/2772495785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.509+0000 7f177ea00640 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f6a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.509+0000 7f177f400640 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 0x556a4f634e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.509+0000 7f177ea00640 1 -- 172.21.15.86:0/2772495785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f625fa0 msgr2=0x556a4f634930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.510+0000 7f177ea00640 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f625fa0 0x556a4f634930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.510+0000 7f177ea00640 1 -- 172.21.15.86:0/2772495785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 msgr2=0x556a4f634e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.510+0000 7f177ea00640 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 0x556a4f634e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.510+0000 7f177ea00640 1 -- 172.21.15.86:0/2772495785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a4f3687d0 con 0x556a4f62bb00 2024-04-17T07:34:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.510+0000 7f177f400640 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 0x556a4f634e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:01.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.510+0000 7f177ea00640 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f6a01c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f176c00f790 tx=0x7f176c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e735e8d16bbc405c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:01.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.510+0000 7f1769600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:01.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.511+0000 7f1769600640 1 -- 172.21.15.86:0/2772495785 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c02a5f0 con 0x556a4f62bb00 2024-04-17T07:34:01.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.511+0000 7f1769600640 1 -- 172.21.15.86:0/2772495785 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f176c02a790 con 0x556a4f62bb00 2024-04-17T07:34:01.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.511+0000 7f178922d740 1 -- 172.21.15.86:0/2772495785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176c01f550 con 0x556a4f62bb00 2024-04-17T07:34:01.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.511+0000 7f1769600640 1 -- 172.21.15.86:0/2772495785 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f176c02a990 con 0x556a4f62bb00 2024-04-17T07:34:01.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.512+0000 7f178922d740 10 client.19866 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:01.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.512+0000 7f178922d740 1 -- 172.21.15.86:0/2772495785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a4f6a1750 con 0x556a4f62bb00 2024-04-17T07:34:01.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.512+0000 7f1769600640 1 -- 172.21.15.86:0/2772495785 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f176c01f550 con 0x556a4f62bb00 2024-04-17T07:34:01.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.513+0000 7f1769600640 1 -- 172.21.15.86:0/2772495785 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f176c01d070 con 0x556a4f62bb00 2024-04-17T07:34:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.513+0000 7f178922d740 10 client.19866 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.513+0000 7f178922d740 10 client.19866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.513+0000 7f178922d740 20 client.19866 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.513+0000 7f178922d740 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a4f6a1f70 0x556a4f6a4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.513+0000 7f178922d740 4 client.19866 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:34:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.513+0000 7f178922d740 1 -- 172.21.15.86:0/2772495785 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a4f5938b0 con 0x556a4f6a1f70 2024-04-17T07:34:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.514+0000 7f177f400640 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a4f6a1f70 0x556a4f6a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.516+0000 7f177f400640 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a4f6a1f70 0x556a4f6a4390 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dfba4580fd73be7 server_cookie=960cc66de93e0b41 in_seq=0 out_seq=0 2024-04-17T07:34:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.516+0000 7f1769600640 10 client.19866 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.516+0000 7f1769600640 1 -- 172.21.15.86:0/2772495785 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a4f5938b0 con 0x556a4f6a1f70 2024-04-17T07:34:01.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.517+0000 7f1769600640 10 client.19866 handle_command_reply: tid=0 2024-04-17T07:34:01.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.517+0000 7f178922d740 1 client.19866 shutdown 2024-04-17T07:34:01.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.517+0000 7f178922d740 1 -- 172.21.15.86:0/2772495785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 msgr2=0x556a4f6a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.517+0000 7f178922d740 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f6a01c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f176c00f790 tx=0x7f176c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.518+0000 7f178922d740 1 -- 172.21.15.86:0/2772495785 shutdown_connections 2024-04-17T07:34:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.518+0000 7f178922d740 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x556a4f6a1f70 0x556a4f6a4390 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.518+0000 7f178922d740 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a4f62bb00 0x556a4f6a01c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.518+0000 7f178922d740 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556a4f626990 0x556a4f634e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.518+0000 7f1769600640 0 client.19866 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.518+0000 7f178922d740 1 --2- 172.21.15.86:0/2772495785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a4f625fa0 0x556a4f634930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.518+0000 7f178922d740 1 -- 172.21.15.86:0/2772495785 >> 172.21.15.86:0/2772495785 conn(0x556a4f621510 msgr2=0x556a4f630130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.519+0000 7f178922d740 1 -- 172.21.15.86:0/2772495785 shutdown_connections 2024-04-17T07:34:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.519+0000 7f178922d740 1 -- 172.21.15.86:0/2772495785 wait complete. 2024-04-17T07:34:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.519+0000 7f178922d740 20 client.19866 trim_cache size 0 max 16384 2024-04-17T07:34:01.520 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:34:01.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.520+0000 7f1786400640 1 -- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17640a3c80 msgr2=0x7f17640a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.520+0000 7f1786400640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17640a3c80 0x7f17640a6140 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f177804e000 tx=0x7f177802bef0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.520+0000 7f1786400640 1 -- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178010a170 msgr2=0x7f17801a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:01.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.520+0000 7f1786400640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178010a170 0x7f17801a5d70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f1778036040 tx=0x7f1778003f80 comp rx=0 tx=0).stop 2024-04-17T07:34:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.521+0000 7f1786400640 1 -- 172.21.15.86:0/1976630544 shutdown_connections 2024-04-17T07:34:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.521+0000 7f1786400640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f17640a3c80 0x7f17640a6140 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.521+0000 7f1786400640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178010a170 0x7f17801a5d70 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.521+0000 7f1786400640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17801097b0 0x7f178019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.521+0000 7f1786400640 1 --2- 172.21.15.86:0/1976630544 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17801085c0 0x7f178019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.521+0000 7f1786400640 1 -- 172.21.15.86:0/1976630544 >> 172.21.15.86:0/1976630544 conn(0x7f1780077560 msgr2=0x7f178010cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.521+0000 7f1786400640 1 -- 172.21.15.86:0/1976630544 shutdown_connections 2024-04-17T07:34:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:01.521+0000 7f1786400640 1 -- 172.21.15.86:0/1976630544 wait complete. 2024-04-17T07:34:02.559 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:34:02.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:34:02.559 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:34:02.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.968+0000 7fc4e4000640 1 -- 172.21.15.86:0/3244764259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4dc1085c0 msgr2=0x7fc4dc1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:02.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.968+0000 7fc4e4000640 1 --2- 172.21.15.86:0/3244764259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4dc1085c0 0x7fc4dc1089e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009900 tx=0x7fc4d002bc40 comp rx=0 tx=0).stop 2024-04-17T07:34:02.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.969+0000 7fc4e4000640 1 -- 172.21.15.86:0/3244764259 shutdown_connections 2024-04-17T07:34:02.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.969+0000 7fc4e4000640 1 --2- 172.21.15.86:0/3244764259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4dc10a170 0x7fc4dc110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.969+0000 7fc4e4000640 1 --2- 172.21.15.86:0/3244764259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4dc1097b0 0x7fc4dc109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.969+0000 7fc4e4000640 1 --2- 172.21.15.86:0/3244764259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4dc1085c0 0x7fc4dc1089e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.969+0000 7fc4e4000640 1 -- 172.21.15.86:0/3244764259 >> 172.21.15.86:0/3244764259 conn(0x7fc4dc077560 msgr2=0x7fc4dc0799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.969+0000 7fc4e4000640 1 -- 172.21.15.86:0/3244764259 shutdown_connections 2024-04-17T07:34:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.969+0000 7fc4e4000640 1 -- 172.21.15.86:0/3244764259 wait complete. 2024-04-17T07:34:02.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.970+0000 7fc4e4000640 1 Processor -- start 2024-04-17T07:34:02.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.971+0000 7fc4e4000640 1 -- start start 2024-04-17T07:34:02.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.971+0000 7fc4e4000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4dc1085c0 0x7fc4dc0709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.971+0000 7fc4e4000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4dc1097b0 0x7fc4dc070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.971+0000 7fc4e4000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4dc10a170 0x7fc4dc1ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.971+0000 7fc4e4000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1141a0 con 0x7fc4dc1097b0 2024-04-17T07:34:02.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.971+0000 7fc4e4000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc114020 con 0x7fc4dc10a170 2024-04-17T07:34:02.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.971+0000 7fc4e4000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc114320 con 0x7fc4dc1085c0 2024-04-17T07:34:02.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.971+0000 7fc4e2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4dc10a170 0x7fc4dc1ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.972+0000 7fc4e2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4dc10a170 0x7fc4dc1ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52266/0 (socket says 172.21.15.86:52266) 2024-04-17T07:34:02.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.972+0000 7fc4e2000640 1 -- 172.21.15.86:0/3844315132 learned_addr learned my addr 172.21.15.86:0/3844315132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:02.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.972+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4dc1097b0 0x7fc4dc070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.972+0000 7fc4e1600640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4dc1085c0 0x7fc4dc0709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.972+0000 7fc4e2000640 1 -- 172.21.15.86:0/3844315132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4dc1085c0 msgr2=0x7fc4dc0709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:02.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.972+0000 7fc4e2000640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4dc1085c0 0x7fc4dc0709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.972+0000 7fc4e2000640 1 -- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4dc1097b0 msgr2=0x7fc4dc070f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:02.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.972+0000 7fc4e2000640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4dc1097b0 0x7fc4dc070f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.972+0000 7fc4e2000640 1 -- 172.21.15.86:0/3844315132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4dc1aedf0 con 0x7fc4dc10a170 2024-04-17T07:34:02.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.972+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4dc1097b0 0x7fc4dc070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:02.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.973+0000 7fc4e2000640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4dc10a170 0x7fc4dc1ae8b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400eb10 tx=0x7fc4d400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.974+0000 7fc4da000640 1 -- 172.21.15.86:0/3844315132 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4004b20 con 0x7fc4dc10a170 2024-04-17T07:34:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.974+0000 7fc4da000640 1 -- 172.21.15.86:0/3844315132 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4d4004cc0 con 0x7fc4dc10a170 2024-04-17T07:34:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.974+0000 7fc4da000640 1 -- 172.21.15.86:0/3844315132 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4010690 con 0x7fc4dc10a170 2024-04-17T07:34:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.974+0000 7fc4e4000640 1 -- 172.21.15.86:0/3844315132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4dc1aefc0 con 0x7fc4dc10a170 2024-04-17T07:34:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.974+0000 7fc4e4000640 1 -- 172.21.15.86:0/3844315132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4dc1af510 con 0x7fc4dc10a170 2024-04-17T07:34:02.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.978+0000 7fc4e6e10740 1 Processor -- start 2024-04-17T07:34:02.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.978+0000 7fc4da000640 1 -- 172.21.15.86:0/3844315132 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fc4d4005020 con 0x7fc4dc10a170 2024-04-17T07:34:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.980+0000 7fc4da000640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc4c40a3d80 0x7fc4c40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.980+0000 7fc4da000640 1 -- 172.21.15.86:0/3844315132 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc4d40d0db0 con 0x7fc4dc10a170 2024-04-17T07:34:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e6e10740 1 -- start start 2024-04-17T07:34:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc4c40a3d80 0x7fc4c40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 0x559328f0b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0b8b0 0x559328f103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f10a20 0x559328f12eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559328e70cb0 con 0x559328f0b8b0 2024-04-17T07:34:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559328e17370 con 0x559328f0aec0 2024-04-17T07:34:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559328e6cc70 con 0x559328f10a20 2024-04-17T07:34:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0b8b0 0x559328f103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0b8b0 0x559328f103b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39020/0 (socket says 172.21.15.86:39020) 2024-04-17T07:34:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.981+0000 7fc4e0c00640 1 -- 172.21.15.86:0/980257128 learned_addr learned my addr 172.21.15.86:0/980257128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:02.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.982+0000 7fc4e2000640 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f10a20 0x559328f12eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.982+0000 7fc4e1600640 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 0x559328f0b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.982+0000 7fc4e2000640 1 -- 172.21.15.86:0/980257128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 msgr2=0x559328f0b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:02.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.982+0000 7fc4e2000640 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 0x559328f0b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.982+0000 7fc4e2000640 1 -- 172.21.15.86:0/980257128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0b8b0 msgr2=0x559328f103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.982+0000 7fc4e2000640 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0b8b0 0x559328f103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.982+0000 7fc4e2000640 1 -- 172.21.15.86:0/980257128 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559328e36910 con 0x559328f10a20 2024-04-17T07:34:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.983+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0b8b0 0x559328f103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:34:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.983+0000 7fc4e1600640 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 0x559328f0b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.983+0000 7fc4e2000640 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f10a20 0x559328f12eb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ce4d0 tx=0x7fc4d409c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5705d057a14c5fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.983+0000 7fc4ce600640 1 -- 172.21.15.86:0/980257128 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40a1060 con 0x559328f10a20 2024-04-17T07:34:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.983+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc4c40a3d80 0x7fc4c40a6240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8014610 tx=0x7fc4c8009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:02.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.984+0000 7fc4ce600640 1 -- 172.21.15.86:0/980257128 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4d40a5920 con 0x559328f10a20 2024-04-17T07:34:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.984+0000 7fc4ce600640 1 -- 172.21.15.86:0/980257128 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40a56f0 con 0x559328f10a20 2024-04-17T07:34:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.985+0000 7fc4e6e10740 1 -- 172.21.15.86:0/980257128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f10a20 msgr2=0x559328f12eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.985+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f10a20 0x559328f12eb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ce4d0 tx=0x7fc4d409c8b0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.985+0000 7fc4e6e10740 1 -- 172.21.15.86:0/980257128 shutdown_connections 2024-04-17T07:34:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.985+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f10a20 0x559328f12eb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.985+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0b8b0 0x559328f103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.985+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/980257128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 0x559328f0b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.985+0000 7fc4e6e10740 1 -- 172.21.15.86:0/980257128 >> 172.21.15.86:0/980257128 conn(0x559328f06430 msgr2=0x559328f06830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.986+0000 7fc4e6e10740 1 -- 172.21.15.86:0/980257128 shutdown_connections 2024-04-17T07:34:02.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.986+0000 7fc4e6e10740 1 -- 172.21.15.86:0/980257128 wait complete. 2024-04-17T07:34:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.987+0000 7fc4e6e10740 1 -- start start 2024-04-17T07:34:02.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.987+0000 7fc4e6e10740 1 Processor -- start 2024-04-17T07:34:02.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.988+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 0x559328f263e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.988+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 0x559328f26920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.988+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 0x559328f2d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.988+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559328f24980 con 0x559328f10a20 2024-04-17T07:34:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.988+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559328e2a730 con 0x559328f0aec0 2024-04-17T07:34:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.988+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559328f24b00 con 0x559328f0b8b0 2024-04-17T07:34:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.988+0000 7fc4e6e10740 10 client.0 _ll_register_callbacks cb 0x559328e2cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.989+0000 7fc4e1600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 0x559328f263e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.989+0000 7fc4e2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 0x559328f2d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.989+0000 7fc4e0c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 0x559328f26920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.989+0000 7fc4e1600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 0x559328f263e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52290/0 (socket says 172.21.15.86:52290) 2024-04-17T07:34:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.989+0000 7fc4e1600640 1 -- 172.21.15.86:0/1244264547 learned_addr learned my addr 172.21.15.86:0/1244264547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.989+0000 7fc4e2000640 1 -- 172.21.15.86:0/1244264547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 msgr2=0x559328f26920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.989+0000 7fc4e2000640 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 0x559328f26920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.989+0000 7fc4e2000640 1 -- 172.21.15.86:0/1244264547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 msgr2=0x559328f263e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.989+0000 7fc4e2000640 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 0x559328f263e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.989+0000 7fc4e2000640 1 -- 172.21.15.86:0/1244264547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559328c4b7e0 con 0x559328f10a20 2024-04-17T07:34:02.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.990+0000 7fc4e2000640 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 0x559328f2d960 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ce4d0 tx=0x7fc4d409c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1deb7bee2a4d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:02.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.990+0000 7fc4c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:02.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.990+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1244264547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40a1060 con 0x559328f10a20 2024-04-17T07:34:02.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.990+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1244264547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc4d409afb0 con 0x559328f10a20 2024-04-17T07:34:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.991+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1244264547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d40a5920 con 0x559328f10a20 2024-04-17T07:34:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.991+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1244264547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4d40ab050 con 0x559328f10a20 2024-04-17T07:34:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.991+0000 7fc4e6e10740 10 client.19875 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.991+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1244264547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559328cabf70 con 0x559328f10a20 2024-04-17T07:34:02.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.992+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1244264547 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4d40a5920 con 0x559328f10a20 2024-04-17T07:34:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.992+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1244264547 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc4d4098a00 con 0x559328f10a20 2024-04-17T07:34:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.993+0000 7fc4e6e10740 10 client.19875 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.993+0000 7fc4e6e10740 10 client.19875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.993+0000 7fc4e6e10740 20 client.19875 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.994+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559328f31e50 0x559328f34240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.994+0000 7fc4e6e10740 4 client.19875 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:02.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.994+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1244264547 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559328e7e6d0 con 0x559328f31e50 2024-04-17T07:34:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.995+0000 7fc4e1600640 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559328f31e50 0x559328f34240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:02.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.996+0000 7fc4c2a00640 10 client.19875 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:02.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.996+0000 7fc4e1600640 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559328f31e50 0x559328f34240 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2124a42157943e server_cookie=602799200ed2205 in_seq=0 out_seq=0 2024-04-17T07:34:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.997+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1244264547 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559328e7e6d0 con 0x559328f31e50 2024-04-17T07:34:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.997+0000 7fc4c2a00640 10 client.19875 handle_command_reply: tid=0 2024-04-17T07:34:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.997+0000 7fc4e6e10740 1 client.19875 shutdown 2024-04-17T07:34:02.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.997+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1244264547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 msgr2=0x559328f2d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:02.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.997+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 0x559328f2d960 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ce4d0 tx=0x7fc4d409c920 comp rx=0 tx=0).stop 2024-04-17T07:34:02.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.998+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1244264547 shutdown_connections 2024-04-17T07:34:02.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.998+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559328f31e50 0x559328f34240 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.998+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 0x559328f2d960 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.998+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 0x559328f26920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.998+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1244264547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0aec0 0x559328f263e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:02.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.998+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1244264547 >> 172.21.15.86:0/1244264547 conn(0x559328f06430 msgr2=0x559328f055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:02.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.998+0000 7fc4c2a00640 0 client.19875 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:03.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.999+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1244264547 shutdown_connections 2024-04-17T07:34:03.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.999+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1244264547 wait complete. 2024-04-17T07:34:03.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:02.999+0000 7fc4e6e10740 20 client.19875 trim_cache size 0 max 16384 2024-04-17T07:34:03.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.048+0000 7fc4e6e10740 1 Processor -- start 2024-04-17T07:34:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.048+0000 7fc4e6e10740 1 -- start start 2024-04-17T07:34:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.048+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0aec0 0x559328f0ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.048+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 0x559328f0d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.048+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f10a20 0x559328f2d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.048+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1b1140 con 0x559328f0aec0 2024-04-17T07:34:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.048+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1b2c80 con 0x559328f10a20 2024-04-17T07:34:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.048+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1b1810 con 0x559328f0b8b0 2024-04-17T07:34:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.049+0000 7fc4e0c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 0x559328f0d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.049+0000 7fc4e0c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 0x559328f0d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54228/0 (socket says 172.21.15.86:54228) 2024-04-17T07:34:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.049+0000 7fc4e0c00640 1 -- 172.21.15.86:0/330208806 learned_addr learned my addr 172.21.15.86:0/330208806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.050+0000 7fc4e1600640 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0aec0 0x559328f0ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.050+0000 7fc4e0c00640 1 -- 172.21.15.86:0/330208806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f10a20 msgr2=0x559328f2d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.050+0000 7fc4e2000640 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f10a20 0x559328f2d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.050+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f10a20 0x559328f2d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.050+0000 7fc4e0c00640 1 -- 172.21.15.86:0/330208806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0aec0 msgr2=0x559328f0ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.050+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0aec0 0x559328f0ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.050+0000 7fc4e0c00640 1 -- 172.21.15.86:0/330208806 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559328b43170 con 0x559328f0b8b0 2024-04-17T07:34:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.050+0000 7fc4e1600640 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0aec0 0x559328f0ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:34:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.051+0000 7fc4e2000640 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f10a20 0x559328f2d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:03.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.051+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 0x559328f0d390 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009d70 tx=0x7fc4c801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f25ecf739393dca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:03.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.052+0000 7fc4cdc00640 1 -- 172.21.15.86:0/330208806 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c80059c0 con 0x559328f0b8b0 2024-04-17T07:34:03.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.052+0000 7fc4cdc00640 1 -- 172.21.15.86:0/330208806 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4c8010040 con 0x559328f0b8b0 2024-04-17T07:34:03.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.052+0000 7fc4cdc00640 1 -- 172.21.15.86:0/330208806 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8023da0 con 0x559328f0b8b0 2024-04-17T07:34:03.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.053+0000 7fc4e6e10740 1 -- 172.21.15.86:0/330208806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 msgr2=0x559328f0d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.053+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 0x559328f0d390 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009d70 tx=0x7fc4c801c000 comp rx=0 tx=0).stop 2024-04-17T07:34:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.054+0000 7fc4e6e10740 1 -- 172.21.15.86:0/330208806 shutdown_connections 2024-04-17T07:34:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.054+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f10a20 0x559328f2d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.054+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0b8b0 0x559328f0d390 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.054+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/330208806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f0aec0 0x559328f0ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.054+0000 7fc4e6e10740 1 -- 172.21.15.86:0/330208806 >> 172.21.15.86:0/330208806 conn(0x559328f06430 msgr2=0x559328f263e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.054+0000 7fc4e6e10740 1 -- 172.21.15.86:0/330208806 shutdown_connections 2024-04-17T07:34:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.054+0000 7fc4e6e10740 1 -- 172.21.15.86:0/330208806 wait complete. 2024-04-17T07:34:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.055+0000 7fc4e6e10740 1 -- start start 2024-04-17T07:34:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.055+0000 7fc4e6e10740 1 Processor -- start 2024-04-17T07:34:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.056+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0aec0 0x559328f2aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.057+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0b8b0 0x559328f2b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.057+0000 7fc4e6e10740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 0x559328f2c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.057+0000 7fc4e1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0aec0 0x559328f2aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.057+0000 7fc4e1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0aec0 0x559328f2aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54240/0 (socket says 172.21.15.86:54240) 2024-04-17T07:34:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.057+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1b25b0 con 0x559328f10a20 2024-04-17T07:34:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.057+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1b1ee0 con 0x559328f0b8b0 2024-04-17T07:34:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.057+0000 7fc4e6e10740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc082900 con 0x559328f0aec0 2024-04-17T07:34:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.057+0000 7fc4e6e10740 10 client.0 _ll_register_callbacks cb 0x7fc4c8009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.058+0000 7fc4e0c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0b8b0 0x559328f2b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.058+0000 7fc4e1600640 1 -- 172.21.15.86:0/1488665732 learned_addr learned my addr 172.21.15.86:0/1488665732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.058+0000 7fc4e2000640 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 0x559328f2c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.058+0000 7fc4e0c00640 1 -- 172.21.15.86:0/1488665732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0aec0 msgr2=0x559328f2aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.058+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0aec0 0x559328f2aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.058+0000 7fc4e0c00640 1 -- 172.21.15.86:0/1488665732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 msgr2=0x559328f2c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.058+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 0x559328f2c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.058+0000 7fc4e0c00640 1 -- 172.21.15.86:0/1488665732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559328f2cb30 con 0x559328f0b8b0 2024-04-17T07:34:03.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.058+0000 7fc4e2000640 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 0x559328f2c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.060+0000 7fc4e0c00640 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0b8b0 0x559328f2b530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8016d70 tx=0x7fc4c80096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31237efdde3f6520 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.060+0000 7fc4c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:34:03.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.060+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1488665732 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8033070 con 0x559328f0b8b0 2024-04-17T07:34:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.060+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1488665732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c8010040 con 0x559328f0b8b0 2024-04-17T07:34:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.061+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1488665732 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fc4c8023630 con 0x559328f0b8b0 2024-04-17T07:34:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.061+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1488665732 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4c803f050 con 0x559328f0b8b0 2024-04-17T07:34:03.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.061+0000 7fc4e6e10740 10 client.38942 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.061+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1488665732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559328f98270 con 0x559328f0b8b0 2024-04-17T07:34:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.062+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1488665732 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4c8010040 con 0x559328f0b8b0 2024-04-17T07:34:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.062+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1488665732 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fc4c8025470 con 0x559328f0b8b0 2024-04-17T07:34:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.062+0000 7fc4e6e10740 10 client.38942 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.062+0000 7fc4e6e10740 10 client.38942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.062+0000 7fc4e6e10740 20 client.38942 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.062+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559328f98ae0 0x559328f9af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.062+0000 7fc4e6e10740 4 client.38942 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:34:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.062+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1488665732 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559328e78540 con 0x559328f98ae0 2024-04-17T07:34:03.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.063+0000 7fc4e1600640 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559328f98ae0 0x559328f9af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.064+0000 7fc4e1600640 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559328f98ae0 0x559328f9af00 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f4e33827131747 server_cookie=1e0b76028cb40657 in_seq=0 out_seq=0 2024-04-17T07:34:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.064+0000 7fc4c2a00640 10 client.38942 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.067+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1488665732 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559328e78540 con 0x559328f98ae0 2024-04-17T07:34:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.067+0000 7fc4c2a00640 10 client.38942 handle_command_reply: tid=0 2024-04-17T07:34:03.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.067+0000 7fc4e6e10740 1 client.38942 shutdown 2024-04-17T07:34:03.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.068+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1488665732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0b8b0 msgr2=0x559328f2b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.068+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0b8b0 0x559328f2b530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8016d70 tx=0x7fc4c80096b0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.069+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1488665732 shutdown_connections 2024-04-17T07:34:03.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.069+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559328f98ae0 0x559328f9af00 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.069+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559328f10a20 0x559328f2c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.069+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559328f0b8b0 0x559328f2b530 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.069+0000 7fc4e6e10740 1 --2- 172.21.15.86:0/1488665732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559328f0aec0 0x559328f2aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.069+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1488665732 >> 172.21.15.86:0/1488665732 conn(0x559328f06430 msgr2=0x559328f26380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:03.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.069+0000 7fc4c2a00640 0 client.38942 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.069+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1488665732 shutdown_connections 2024-04-17T07:34:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.069+0000 7fc4e6e10740 1 -- 172.21.15.86:0/1488665732 wait complete. 2024-04-17T07:34:03.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.069+0000 7fc4e6e10740 20 client.38942 trim_cache size 0 max 16384 2024-04-17T07:34:03.071 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:34:03.071 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:34:03.071 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "3580aef3-5fdf-4fcc-bab8-b4456de3cb77", 2024-04-17T07:34:03.071 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:34:03.071 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:34:03.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.070+0000 7fc4e4000640 1 -- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc4c40a3d80 msgr2=0x7fc4c40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.070+0000 7fc4e4000640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc4c40a3d80 0x7fc4c40a6240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8014610 tx=0x7fc4c8009940 comp rx=0 tx=0).stop 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.070+0000 7fc4e4000640 1 -- 172.21.15.86:0/3844315132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4dc10a170 msgr2=0x7fc4dc1ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.070+0000 7fc4e4000640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4dc10a170 0x7fc4dc1ae8b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400eb10 tx=0x7fc4d400c2d0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.070+0000 7fc4e4000640 1 -- 172.21.15.86:0/3844315132 shutdown_connections 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.070+0000 7fc4e4000640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fc4c40a3d80 0x7fc4c40a6240 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.070+0000 7fc4e4000640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4dc10a170 0x7fc4dc1ae8b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.070+0000 7fc4e4000640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4dc1097b0 0x7fc4dc070f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.070+0000 7fc4e4000640 1 --2- 172.21.15.86:0/3844315132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4dc1085c0 0x7fc4dc0709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.070+0000 7fc4e4000640 1 -- 172.21.15.86:0/3844315132 >> 172.21.15.86:0/3844315132 conn(0x7fc4dc077560 msgr2=0x7fc4dc10cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.071+0000 7fc4e4000640 1 -- 172.21.15.86:0/3844315132 shutdown_connections 2024-04-17T07:34:03.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.071+0000 7fc4e4000640 1 -- 172.21.15.86:0/3844315132 wait complete. 2024-04-17T07:34:03.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:02 smithi086 ceph-mon[31733]: pgmap v1030: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-04-17T07:34:03.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:34:03.101 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:34:03.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:02 smithi098 ceph-mon[35418]: pgmap v1030: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-04-17T07:34:03.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:02 smithi103 ceph-mon[35495]: pgmap v1030: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-04-17T07:34:03.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.481+0000 7ff5cfe00640 1 -- 172.21.15.86:0/2029811742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d0102fe0 msgr2=0x7ff5d0103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.481+0000 7ff5cfe00640 1 --2- 172.21.15.86:0/2029811742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d0102fe0 0x7ff5d0103460 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009900 tx=0x7ff5b802bc40 comp rx=0 tx=0).stop 2024-04-17T07:34:03.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.483+0000 7ff5cfe00640 1 -- 172.21.15.86:0/2029811742 shutdown_connections 2024-04-17T07:34:03.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.483+0000 7ff5cfe00640 1 --2- 172.21.15.86:0/2029811742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d01039a0 0x7ff5d010a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.483+0000 7ff5cfe00640 1 --2- 172.21.15.86:0/2029811742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d0102fe0 0x7ff5d0103460 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.483+0000 7ff5cfe00640 1 --2- 172.21.15.86:0/2029811742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0101e90 0x7ff5d01022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.483+0000 7ff5cfe00640 1 -- 172.21.15.86:0/2029811742 >> 172.21.15.86:0/2029811742 conn(0x7ff5d00fd400 msgr2=0x7ff5d00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:03.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.483+0000 7ff5cfe00640 1 -- 172.21.15.86:0/2029811742 shutdown_connections 2024-04-17T07:34:03.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.484+0000 7ff5cfe00640 1 -- 172.21.15.86:0/2029811742 wait complete. 2024-04-17T07:34:03.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.486+0000 7ff5cfe00640 1 Processor -- start 2024-04-17T07:34:03.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.486+0000 7ff5cfe00640 1 -- start start 2024-04-17T07:34:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.488+0000 7ff5cfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d0101e90 0x7ff5d019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.488+0000 7ff5cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0102fe0 0x7ff5d019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.488+0000 7ff5cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d01039a0 0x7ff5d01a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.488+0000 7ff5cfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0113fe0 con 0x7ff5d0102fe0 2024-04-17T07:34:03.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.488+0000 7ff5cea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d0101e90 0x7ff5d019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.488+0000 7ff5cfe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0113e60 con 0x7ff5d0101e90 2024-04-17T07:34:03.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.488+0000 7ff5cfe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0114160 con 0x7ff5d01039a0 2024-04-17T07:34:03.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.488+0000 7ff5cea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d0101e90 0x7ff5d019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52330/0 (socket says 172.21.15.86:52330) 2024-04-17T07:34:03.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.489+0000 7ff5cea00640 1 -- 172.21.15.86:0/231455439 learned_addr learned my addr 172.21.15.86:0/231455439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:03.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.489+0000 7ff5cf400640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d01039a0 0x7ff5d01a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.489+0000 7ff5cea00640 1 -- 172.21.15.86:0/231455439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d01039a0 msgr2=0x7ff5d01a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.489+0000 7ff5cea00640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d01039a0 0x7ff5d01a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.489+0000 7ff5cea00640 1 -- 172.21.15.86:0/231455439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0102fe0 msgr2=0x7ff5d019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.492+0000 7ff5ce000640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0102fe0 0x7ff5d019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.492+0000 7ff5cea00640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0102fe0 0x7ff5d019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.492+0000 7ff5cea00640 1 -- 172.21.15.86:0/231455439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d01a6260 con 0x7ff5d0101e90 2024-04-17T07:34:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.492+0000 7ff5cea00640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d0101e90 0x7ff5d019e640 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000c920 tx=0x7ff5c000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.493+0000 7ff5c7400640 1 -- 172.21.15.86:0/231455439 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0007c70 con 0x7ff5d0101e90 2024-04-17T07:34:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.493+0000 7ff5cfe00640 1 -- 172.21.15.86:0/231455439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d01a6550 con 0x7ff5d0101e90 2024-04-17T07:34:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.493+0000 7ff5cfe00640 1 -- 172.21.15.86:0/231455439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d01a6af0 con 0x7ff5d0101e90 2024-04-17T07:34:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.493+0000 7ff5c7400640 1 -- 172.21.15.86:0/231455439 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5c0007e10 con 0x7ff5d0101e90 2024-04-17T07:34:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.493+0000 7ff5c7400640 1 -- 172.21.15.86:0/231455439 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0007c70 con 0x7ff5d0101e90 2024-04-17T07:34:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.498+0000 7ff5c7400640 1 -- 172.21.15.86:0/231455439 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff5c0005080 con 0x7ff5d0101e90 2024-04-17T07:34:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.499+0000 7ff5d8382740 1 Processor -- start 2024-04-17T07:34:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.500+0000 7ff5d8382740 1 -- start start 2024-04-17T07:34:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.500+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe06790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.500+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe0b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.500+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe0e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.500+0000 7ff5d8382740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8cbd6d120 con 0x55a8cbe06370 2024-04-17T07:34:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.500+0000 7ff5d8382740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a8cbd13340 con 0x55a8cbe06d60 2024-04-17T07:34:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.500+0000 7ff5d8382740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a8cbd690e0 con 0x55a8cbe0bed0 2024-04-17T07:34:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.501+0000 7ff5ce000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe0e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.501+0000 7ff5ce000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe0e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54268/0 (socket says 172.21.15.86:54268) 2024-04-17T07:34:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.501+0000 7ff5ce000640 1 -- 172.21.15.86:0/2787456309 learned_addr learned my addr 172.21.15.86:0/2787456309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.501+0000 7ff5cea00640 1 --2- 172.21.15.86:0/2787456309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe0b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.501+0000 7ff5cf400640 1 --2- 172.21.15.86:0/2787456309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe06790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.502+0000 7ff5c7400640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff5b40a3e10 0x7ff5b40a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.503+0000 7ff5c7400640 1 -- 172.21.15.86:0/231455439 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff5c00d0790 con 0x7ff5d0101e90 2024-04-17T07:34:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.503+0000 7ff5ce000640 1 -- 172.21.15.86:0/2787456309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 msgr2=0x55a8cbe0b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.503+0000 7ff5ce000640 1 --2- 172.21.15.86:0/2787456309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe0b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.503+0000 7ff5ce000640 1 -- 172.21.15.86:0/2787456309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 msgr2=0x55a8cbe06790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.503+0000 7ff5ce000640 1 --2- 172.21.15.86:0/2787456309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe06790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.503+0000 7ff5ce000640 1 -- 172.21.15.86:0/2787456309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8cbb497d0 con 0x55a8cbe0bed0 2024-04-17T07:34:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.503+0000 7ff5cea00640 1 --2- 172.21.15.86:0/2787456309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe0b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T07:34:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.504+0000 7ff5ce000640 1 --2- 172.21.15.86:0/2787456309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe0e360 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8006010 tx=0x7ff5b8005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ae22d82e2c5643 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.504+0000 7ff5cf400640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff5b40a3e10 0x7ff5b40a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.504+0000 7ff5bf400640 1 -- 172.21.15.86:0/2787456309 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8043020 con 0x55a8cbe0bed0 2024-04-17T07:34:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.504+0000 7ff5bf400640 1 -- 172.21.15.86:0/2787456309 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5b8004040 con 0x55a8cbe0bed0 2024-04-17T07:34:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.505+0000 7ff5bf400640 1 -- 172.21.15.86:0/2787456309 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8005540 con 0x55a8cbe0bed0 2024-04-17T07:34:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.506+0000 7ff5d8382740 1 -- 172.21.15.86:0/2787456309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 msgr2=0x55a8cbe0e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.506+0000 7ff5d8382740 1 --2- 172.21.15.86:0/2787456309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe0e360 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8006010 tx=0x7ff5b8005be0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.506+0000 7ff5d8382740 1 -- 172.21.15.86:0/2787456309 shutdown_connections 2024-04-17T07:34:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.506+0000 7ff5d8382740 1 --2- 172.21.15.86:0/2787456309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe0e360 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.506+0000 7ff5d8382740 1 --2- 172.21.15.86:0/2787456309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe0b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.506+0000 7ff5d8382740 1 --2- 172.21.15.86:0/2787456309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe06790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.506+0000 7ff5d8382740 1 -- 172.21.15.86:0/2787456309 >> 172.21.15.86:0/2787456309 conn(0x55a8cbe018e0 msgr2=0x55a8cbe01ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.507+0000 7ff5d8382740 1 -- 172.21.15.86:0/2787456309 shutdown_connections 2024-04-17T07:34:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.507+0000 7ff5cf400640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff5b40a3e10 0x7ff5b40a62d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55a8cbe075c0 tx=0x7ff5c8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.507+0000 7ff5d8382740 1 -- 172.21.15.86:0/2787456309 wait complete. 2024-04-17T07:34:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.508+0000 7ff5d8382740 1 -- start start 2024-04-17T07:34:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.508+0000 7ff5d8382740 1 Processor -- start 2024-04-17T07:34:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.509+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe21800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.509+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe21d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.509+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe28d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.510+0000 7ff5cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe21800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.510+0000 7ff5ce000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe21d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.510+0000 7ff5cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe21800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39068/0 (socket says 172.21.15.86:39068) 2024-04-17T07:34:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.510+0000 7ff5ce000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe21d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52350/0 (socket says 172.21.15.86:52350) 2024-04-17T07:34:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.510+0000 7ff5cf400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe28d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.510+0000 7ff5cf400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe28d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54278/0 (socket says 172.21.15.86:54278) 2024-04-17T07:34:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.510+0000 7ff5cea00640 1 -- 172.21.15.86:0/3057332240 learned_addr learned my addr 172.21.15.86:0/3057332240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.511+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8cbe1fe30 con 0x55a8cbe06370 2024-04-17T07:34:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.511+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a8cbd26710 con 0x55a8cbe06d60 2024-04-17T07:34:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.511+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a8cbe1ffb0 con 0x55a8cbe0bed0 2024-04-17T07:34:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.511+0000 7ff5d8382740 10 client.0 _ll_register_callbacks cb 0x55a8cbd21af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.511+0000 7ff5ce000640 1 -- 172.21.15.86:0/3057332240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 msgr2=0x55a8cbe28d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.511+0000 7ff5ce000640 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe28d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.511+0000 7ff5ce000640 1 -- 172.21.15.86:0/3057332240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 msgr2=0x55a8cbe21800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.511+0000 7ff5ce000640 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe21800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.511+0000 7ff5ce000640 1 -- 172.21.15.86:0/3057332240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8cba7f3f0 con 0x55a8cbe06d60 2024-04-17T07:34:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.512+0000 7ff5ce000640 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe21d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8000c00 tx=0x7ff5b80054f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1846c117396a2c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.512+0000 7ff5b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:34:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.512+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3057332240 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b804c070 con 0x55a8cbe06d60 2024-04-17T07:34:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.513+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3057332240 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff5b80046c0 con 0x55a8cbe06d60 2024-04-17T07:34:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.513+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b8004040 con 0x55a8cbe06d60 2024-04-17T07:34:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.513+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3057332240 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5b8043040 con 0x55a8cbe06d60 2024-04-17T07:34:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.514+0000 7ff5d8382740 10 client.38954 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.514+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8cba3f0f0 con 0x55a8cbe06d60 2024-04-17T07:34:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.514+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3057332240 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5b8004040 con 0x55a8cbe06d60 2024-04-17T07:34:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.514+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3057332240 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff5b8003a40 con 0x55a8cbe06d60 2024-04-17T07:34:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.515+0000 7ff5d8382740 10 client.38954 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.515+0000 7ff5d8382740 10 client.38954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.515+0000 7ff5d8382740 20 client.38954 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.515+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a8cbe2d250 0x55a8cbe2f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.515+0000 7ff5d8382740 4 client.38954 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.515+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8cbd7ab40 con 0x55a8cbe2d250 2024-04-17T07:34:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.515+0000 7ff5cea00640 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a8cbe2d250 0x55a8cbe2f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.517+0000 7ff5cea00640 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a8cbe2d250 0x55a8cbe2f640 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab5b3c39386616e server_cookie=2e35986dc87f0c7f in_seq=0 out_seq=0 2024-04-17T07:34:03.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.517+0000 7ff5b3e00640 10 client.38954 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.519+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3057332240 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8cbd7ab40 con 0x55a8cbe2d250 2024-04-17T07:34:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.519+0000 7ff5b3e00640 10 client.38954 handle_command_reply: tid=0 2024-04-17T07:34:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.519+0000 7ff5d8382740 1 client.38954 shutdown 2024-04-17T07:34:03.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.519+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 msgr2=0x55a8cbe21d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.519+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe21d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8000c00 tx=0x7ff5b80054f0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.520+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 shutdown_connections 2024-04-17T07:34:03.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.520+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a8cbe2d250 0x55a8cbe2f640 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.520+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe28d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.520+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe06d60 0x55a8cbe21d40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.520+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3057332240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe21800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.520+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 >> 172.21.15.86:0/3057332240 conn(0x55a8cbe018e0 msgr2=0x55a8cbe00ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:03.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.520+0000 7ff5b3e00640 0 client.38954 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.521+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 shutdown_connections 2024-04-17T07:34:03.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.521+0000 7ff5d8382740 1 -- 172.21.15.86:0/3057332240 wait complete. 2024-04-17T07:34:03.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.522+0000 7ff5d8382740 20 client.38954 trim_cache size 0 max 16384 2024-04-17T07:34:03.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.584+0000 7ff5d8382740 1 Processor -- start 2024-04-17T07:34:03.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.588+0000 7ff5d8382740 1 -- start start 2024-04-17T07:34:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.589+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe05250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.589+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe06d60 0x55a8cbe28d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.589+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe33ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.589+0000 7ff5d8382740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac00fb60 con 0x55a8cbe06370 2024-04-17T07:34:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.589+0000 7ff5d8382740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac0116a0 con 0x55a8cbe0bed0 2024-04-17T07:34:03.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.589+0000 7ff5d8382740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac010230 con 0x55a8cbe06d60 2024-04-17T07:34:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.590+0000 7ff5ce000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe06d60 0x55a8cbe28d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.590+0000 7ff5ce000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe06d60 0x55a8cbe28d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54286/0 (socket says 172.21.15.86:54286) 2024-04-17T07:34:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.590+0000 7ff5ce000640 1 -- 172.21.15.86:0/482947987 learned_addr learned my addr 172.21.15.86:0/482947987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.590+0000 7ff5cf400640 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe33ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.590+0000 7ff5cea00640 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe05250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.591+0000 7ff5ce000640 1 -- 172.21.15.86:0/482947987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 msgr2=0x55a8cbe33ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.591+0000 7ff5ce000640 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe33ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.591+0000 7ff5ce000640 1 -- 172.21.15.86:0/482947987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 msgr2=0x55a8cbe05250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.591+0000 7ff5ce000640 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe05250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.591+0000 7ff5ce000640 1 -- 172.21.15.86:0/482947987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8cbe02650 con 0x55a8cbe06d60 2024-04-17T07:34:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.591+0000 7ff5cf400640 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe33ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:03.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.591+0000 7ff5cea00640 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe05250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.592+0000 7ff5ce000640 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe06d60 0x55a8cbe28d40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009900 tx=0x7ff5b8005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b59f0528dfa5f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.593+0000 7ff5bea00640 1 -- 172.21.15.86:0/482947987 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b804c040 con 0x55a8cbe06d60 2024-04-17T07:34:03.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.593+0000 7ff5bea00640 1 -- 172.21.15.86:0/482947987 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5b802c4c0 con 0x55a8cbe06d60 2024-04-17T07:34:03.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.594+0000 7ff5d8382740 1 -- 172.21.15.86:0/482947987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe06d60 msgr2=0x55a8cbe28d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.594+0000 7ff5d8382740 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe06d60 0x55a8cbe28d40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009900 tx=0x7ff5b8005250 comp rx=0 tx=0).stop 2024-04-17T07:34:03.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.594+0000 7ff5d8382740 1 -- 172.21.15.86:0/482947987 shutdown_connections 2024-04-17T07:34:03.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.594+0000 7ff5d8382740 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe33ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.594+0000 7ff5d8382740 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe06d60 0x55a8cbe28d40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.594+0000 7ff5d8382740 1 --2- 172.21.15.86:0/482947987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe05250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.594+0000 7ff5d8382740 1 -- 172.21.15.86:0/482947987 >> 172.21.15.86:0/482947987 conn(0x55a8cbe018e0 msgr2=0x55a8cbe04b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:03.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.595+0000 7ff5d8382740 1 -- 172.21.15.86:0/482947987 shutdown_connections 2024-04-17T07:34:03.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.595+0000 7ff5d8382740 1 -- 172.21.15.86:0/482947987 wait complete. 2024-04-17T07:34:03.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.596+0000 7ff5d8382740 1 -- start start 2024-04-17T07:34:03.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.596+0000 7ff5d8382740 1 Processor -- start 2024-04-17T07:34:03.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.598+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe26560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.598+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe26aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.598+0000 7ff5cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe26560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.599+0000 7ff5d8382740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe27a10 0x55a8cbe27e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.599+0000 7ff5cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe26560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39084/0 (socket says 172.21.15.86:39084) 2024-04-17T07:34:03.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.599+0000 7ff5cf400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe27a10 0x55a8cbe27e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.599+0000 7ff5cf400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe27a10 0x55a8cbe27e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:54292/0 (socket says 172.21.15.86:54292) 2024-04-17T07:34:03.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.599+0000 7ff5ce000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe26aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.600+0000 7ff5ce000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe26aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.86:52368/0 (socket says 172.21.15.86:52368) 2024-04-17T07:34:03.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.600+0000 7ff5d8382740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac010fd0 con 0x55a8cbe06370 2024-04-17T07:34:03.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.600+0000 7ff5d8382740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac010900 con 0x55a8cbe0bed0 2024-04-17T07:34:03.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.600+0000 7ff5d8382740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac00db30 con 0x55a8cbe27a10 2024-04-17T07:34:03.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.600+0000 7ff5d8382740 10 client.0 _ll_register_callbacks cb 0x7ff5c8009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:03.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.601+0000 7ff5cea00640 1 -- 172.21.15.86:0/3789287516 learned_addr learned my addr 172.21.15.86:0/3789287516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.602+0000 7ff5ce000640 1 -- 172.21.15.86:0/3789287516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe27a10 msgr2=0x55a8cbe27e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.602+0000 7ff5ce000640 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe27a10 0x55a8cbe27e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.602+0000 7ff5ce000640 1 -- 172.21.15.86:0/3789287516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 msgr2=0x55a8cbe26560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:03.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.602+0000 7ff5ce000640 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe26560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.602+0000 7ff5ce000640 1 -- 172.21.15.86:0/3789287516 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8cbe283d0 con 0x55a8cbe0bed0 2024-04-17T07:34:03.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.603+0000 7ff5ce000640 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe26aa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55a8cbdfd2c0 tx=0x7ff5b80058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=425ddb1c3641f4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:03.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.603+0000 7ff5b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-17T07:34:03.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.603+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3789287516 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b804c040 con 0x55a8cbe0bed0 2024-04-17T07:34:03.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.603+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3789287516 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff5b80049e0 con 0x55a8cbe0bed0 2024-04-17T07:34:03.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.603+0000 7ff5d8382740 1 -- 172.21.15.86:0/3789287516 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8cbd749b0 con 0x55a8cbe0bed0 2024-04-17T07:34:03.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.604+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3789287516 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5b8004c00 con 0x55a8cbe0bed0 2024-04-17T07:34:03.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.604+0000 7ff5d8382740 10 client.38960 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:03.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.604+0000 7ff5d8382740 1 -- 172.21.15.86:0/3789287516 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8cbe93c50 con 0x55a8cbe0bed0 2024-04-17T07:34:03.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.605+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3789287516 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a8cbd749b0 con 0x55a8cbe0bed0 2024-04-17T07:34:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.605+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3789287516 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7ff5b8048990 con 0x55a8cbe0bed0 2024-04-17T07:34:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.605+0000 7ff5d8382740 10 client.38960 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.605+0000 7ff5d8382740 10 client.38960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.605+0000 7ff5d8382740 20 client.38960 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.605+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a8cbe94410 0x55a8cbe96830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.605+0000 7ff5d8382740 4 client.38960 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:34:03.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.605+0000 7ff5d8382740 1 -- 172.21.15.86:0/3789287516 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a8cbcf0420 con 0x55a8cbe94410 2024-04-17T07:34:03.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.607+0000 7ff5cea00640 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a8cbe94410 0x55a8cbe96830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:03.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.608+0000 7ff5cea00640 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a8cbe94410 0x55a8cbe96830 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f64ec34780b5d7f4 server_cookie=281c4b6f92c8892c in_seq=0 out_seq=0 2024-04-17T07:34:03.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.608+0000 7ff5b3e00640 10 client.38960 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.608+0000 7ff5b3e00640 1 -- 172.21.15.86:0/3789287516 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a8cbcf0420 con 0x55a8cbe94410 2024-04-17T07:34:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.608+0000 7ff5b3e00640 10 client.38960 handle_command_reply: tid=0 2024-04-17T07:34:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.609+0000 7ff5d8382740 1 client.38960 shutdown 2024-04-17T07:34:03.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.611+0000 7ff5d8382740 1 -- 172.21.15.86:0/3789287516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 msgr2=0x55a8cbe26aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.611+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe26aa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55a8cbdfd2c0 tx=0x7ff5b80058b0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.612+0000 7ff5d8382740 1 -- 172.21.15.86:0/3789287516 shutdown_connections 2024-04-17T07:34:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.612+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55a8cbe94410 0x55a8cbe96830 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.612+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a8cbe27a10 0x55a8cbe27e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.612+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8cbe0bed0 0x55a8cbe26aa0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.612+0000 7ff5b3e00640 0 client.38960 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.612+0000 7ff5d8382740 1 --2- 172.21.15.86:0/3789287516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8cbe06370 0x55a8cbe26560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.612+0000 7ff5d8382740 1 -- 172.21.15.86:0/3789287516 >> 172.21.15.86:0/3789287516 conn(0x55a8cbe018e0 msgr2=0x55a8cbe053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.615+0000 7ff5d8382740 1 -- 172.21.15.86:0/3789287516 shutdown_connections 2024-04-17T07:34:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.615+0000 7ff5d8382740 1 -- 172.21.15.86:0/3789287516 wait complete. 2024-04-17T07:34:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.615+0000 7ff5d8382740 20 client.38960 trim_cache size 0 max 16384 2024-04-17T07:34:03.616 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:34:03.617 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-17T07:34:03.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:34:03.617 INFO:teuthology.orchestra.run.smithi086.stdout: "3580aef3-5fdf-4fcc-bab8-b4456de3cb77": { 2024-04-17T07:34:03.617 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:34:03.617 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "3580aef3-5fdf-4fcc-bab8-b4456de3cb77", 2024-04-17T07:34:03.617 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:34:03.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:34:03.617 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:34:03.617 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:34:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.616+0000 7ff5cfe00640 1 -- 172.21.15.86:0/231455439 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff5b40a3e10 msgr2=0x7ff5b40a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.616+0000 7ff5cfe00640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff5b40a3e10 0x7ff5b40a62d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55a8cbe075c0 tx=0x7ff5c8004ab0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.617+0000 7ff5cfe00640 1 -- 172.21.15.86:0/231455439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d0101e90 msgr2=0x7ff5d019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.617+0000 7ff5cfe00640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d0101e90 0x7ff5d019e640 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000c920 tx=0x7ff5c000cde0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.617+0000 7ff5cfe00640 1 -- 172.21.15.86:0/231455439 shutdown_connections 2024-04-17T07:34:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.617+0000 7ff5cfe00640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff5b40a3e10 0x7ff5b40a62d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.617+0000 7ff5cfe00640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d01039a0 0x7ff5d01a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.617+0000 7ff5cfe00640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5d0102fe0 0x7ff5d019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.617+0000 7ff5cfe00640 1 --2- 172.21.15.86:0/231455439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d0101e90 0x7ff5d019e640 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.617+0000 7ff5cfe00640 1 -- 172.21.15.86:0/231455439 >> 172.21.15.86:0/231455439 conn(0x7ff5d00fd400 msgr2=0x7ff5d0106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.617+0000 7ff5cfe00640 1 -- 172.21.15.86:0/231455439 shutdown_connections 2024-04-17T07:34:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:03.618+0000 7ff5cfe00640 1 -- 172.21.15.86:0/231455439 wait complete. 2024-04-17T07:34:03.653 INFO:tasks.cephfs.filesystem:scrub status for tag:3580aef3-5fdf-4fcc-bab8-b4456de3cb77 - {'path': '/', 'tag': '3580aef3-5fdf-4fcc-bab8-b4456de3cb77', 'options': 'recursive,force'} 2024-04-17T07:34:04.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:03 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:34:04.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:03 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:04.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:03 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:34:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:03 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:34:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:03 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:04.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:03 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:34:04.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:03 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:34:04.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:03 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:04.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:03 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:34:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 880.0 s, 584.8 tps, lat 170.588 ms stddev 64.677 2024-04-17T07:34:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:04 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:04 smithi098 ceph-mon[35418]: pgmap v1031: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.1 MiB/s wr, 502 op/s 2024-04-17T07:34:05.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:04 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:34:05.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:04 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:05.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:04 smithi103 ceph-mon[35495]: pgmap v1031: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.1 MiB/s wr, 502 op/s 2024-04-17T07:34:05.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:04 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:34:05.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:04 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:05.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:04 smithi086 ceph-mon[31733]: pgmap v1031: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.1 MiB/s wr, 502 op/s 2024-04-17T07:34:05.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:04 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:34:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:06 smithi098 ceph-mon[35418]: pgmap v1032: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 557 op/s 2024-04-17T07:34:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:06 smithi103 ceph-mon[35495]: pgmap v1032: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 557 op/s 2024-04-17T07:34:07.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:06 smithi086 ceph-mon[31733]: pgmap v1032: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 557 op/s 2024-04-17T07:34:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:08 smithi098 ceph-mon[35418]: pgmap v1033: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.2 MiB/s wr, 548 op/s 2024-04-17T07:34:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:08 smithi103 ceph-mon[35495]: pgmap v1033: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.2 MiB/s wr, 548 op/s 2024-04-17T07:34:09.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:08 smithi086 ceph-mon[31733]: pgmap v1033: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.2 MiB/s wr, 548 op/s 2024-04-17T07:34:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 885.0 s, 590.0 tps, lat 168.886 ms stddev 97.014 2024-04-17T07:34:11.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:10 smithi098 ceph-mon[35418]: pgmap v1034: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.6 MiB/s wr, 544 op/s 2024-04-17T07:34:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:10 smithi103 ceph-mon[35495]: pgmap v1034: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.6 MiB/s wr, 544 op/s 2024-04-17T07:34:11.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:10 smithi086 ceph-mon[31733]: pgmap v1034: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.6 MiB/s wr, 544 op/s 2024-04-17T07:34:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:12 smithi098 ceph-mon[35418]: pgmap v1035: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 568 op/s 2024-04-17T07:34:13.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:12 smithi103 ceph-mon[35495]: pgmap v1035: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 568 op/s 2024-04-17T07:34:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:12 smithi086 ceph-mon[31733]: pgmap v1035: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 568 op/s 2024-04-17T07:34:14.362 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 890.0 s, 557.3 tps, lat 179.308 ms stddev 122.345 2024-04-17T07:34:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:14 smithi098 ceph-mon[35418]: pgmap v1036: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.9 MiB/s wr, 531 op/s 2024-04-17T07:34:15.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:14 smithi103 ceph-mon[35495]: pgmap v1036: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.9 MiB/s wr, 531 op/s 2024-04-17T07:34:15.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:14 smithi086 ceph-mon[31733]: pgmap v1036: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.9 MiB/s wr, 531 op/s 2024-04-17T07:34:17.252 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:16 smithi098 ceph-mon[35418]: pgmap v1037: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 595 op/s 2024-04-17T07:34:17.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:16 smithi103 ceph-mon[35495]: pgmap v1037: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 595 op/s 2024-04-17T07:34:17.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:16 smithi086 ceph-mon[31733]: pgmap v1037: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 595 op/s 2024-04-17T07:34:19.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:18 smithi098 ceph-mon[35418]: pgmap v1038: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-04-17T07:34:19.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:18 smithi103 ceph-mon[35495]: pgmap v1038: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-04-17T07:34:19.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:18 smithi086 ceph-mon[31733]: pgmap v1038: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 612 op/s 2024-04-17T07:34:19.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 895.0 s, 570.5 tps, lat 175.390 ms stddev 104.169 2024-04-17T07:34:21.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:20 smithi098 ceph-mon[35418]: pgmap v1039: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 593 op/s 2024-04-17T07:34:21.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:20 smithi103 ceph-mon[35495]: pgmap v1039: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 593 op/s 2024-04-17T07:34:21.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:20 smithi086 ceph-mon[31733]: pgmap v1039: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 593 op/s 2024-04-17T07:34:23.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:23 smithi098 ceph-mon[35418]: pgmap v1040: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.8 MiB/s wr, 614 op/s 2024-04-17T07:34:23.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:23 smithi103 ceph-mon[35495]: pgmap v1040: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.8 MiB/s wr, 614 op/s 2024-04-17T07:34:23.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:23 smithi086 ceph-mon[31733]: pgmap v1040: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.8 MiB/s wr, 614 op/s 2024-04-17T07:34:24.360 INFO:teuthology.orchestra.run.smithi086.stderr:progress: 900.0 s, 627.2 tps, lat 160.990 ms stddev 161.282 2024-04-17T07:34:24.689 INFO:teuthology.orchestra.run.smithi086.stdout:transaction type: 2024-04-17T07:34:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:scaling factor: 32 2024-04-17T07:34:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:query mode: simple 2024-04-17T07:34:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:number of clients: 100 2024-04-17T07:34:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:number of threads: 4 2024-04-17T07:34:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:duration: 900 s 2024-04-17T07:34:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:number of transactions actually processed: 552413 2024-04-17T07:34:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:latency average = 162.803 ms 2024-04-17T07:34:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:latency stddev = 184.670 ms 2024-04-17T07:34:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:tps = 613.569200 (including connections establishing) 2024-04-17T07:34:24.690 INFO:teuthology.orchestra.run.smithi086.stdout:tps = 613.583124 (excluding connections establishing) 2024-04-17T07:34:24.698 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-04-17T07:34:25.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:25 smithi098 ceph-mon[35418]: pgmap v1041: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-04-17T07:34:25.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:25 smithi103 ceph-mon[35495]: pgmap v1041: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-04-17T07:34:25.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:25 smithi086 ceph-mon[31733]: pgmap v1041: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-04-17T07:34:27.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:27 smithi098 ceph-mon[35418]: pgmap v1042: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 618 op/s 2024-04-17T07:34:27.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:27 smithi103 ceph-mon[35495]: pgmap v1042: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 618 op/s 2024-04-17T07:34:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:27 smithi086 ceph-mon[31733]: pgmap v1042: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 618 op/s 2024-04-17T07:34:29.504 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:29 smithi098 ceph-mon[35418]: pgmap v1043: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 653 op/s 2024-04-17T07:34:29.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:29 smithi103 ceph-mon[35495]: pgmap v1043: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 653 op/s 2024-04-17T07:34:29.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:29 smithi086 ceph-mon[31733]: pgmap v1043: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 653 op/s 2024-04-17T07:34:31.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:31 smithi098 ceph-mon[35418]: pgmap v1044: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 707 op/s 2024-04-17T07:34:31.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:31 smithi103 ceph-mon[35495]: pgmap v1044: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 707 op/s 2024-04-17T07:34:31.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:31 smithi086 ceph-mon[31733]: pgmap v1044: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.5 MiB/s wr, 707 op/s 2024-04-17T07:34:33.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:33 smithi103 ceph-mon[35495]: pgmap v1045: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.3 MiB/s wr, 879 op/s 2024-04-17T07:34:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:33 smithi086 ceph-mon[31733]: pgmap v1045: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.3 MiB/s wr, 879 op/s 2024-04-17T07:34:33.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:34:33.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:34:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:33 smithi098 ceph-mon[35418]: pgmap v1045: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.3 MiB/s wr, 879 op/s 2024-04-17T07:34:33.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.886+0000 7f0572c00640 1 -- 172.21.15.86:0/2807694420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c1006e0 msgr2=0x7f056c100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:33.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.886+0000 7f0572c00640 1 --2- 172.21.15.86:0/2807694420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c1006e0 0x7f056c100b00 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f0560009900 tx=0x7f056002bc30 comp rx=0 tx=0).stop 2024-04-17T07:34:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.886+0000 7f0572c00640 1 -- 172.21.15.86:0/2807694420 shutdown_connections 2024-04-17T07:34:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.886+0000 7f0572c00640 1 --2- 172.21.15.86:0/2807694420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f056c1019e0 0x7f056c101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.886+0000 7f0572c00640 1 --2- 172.21.15.86:0/2807694420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f056c101040 0x7f056c1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.886+0000 7f0572c00640 1 --2- 172.21.15.86:0/2807694420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c1006e0 0x7f056c100b00 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.886+0000 7f0572c00640 1 -- 172.21.15.86:0/2807694420 >> 172.21.15.86:0/2807694420 conn(0x7f056c0fc300 msgr2=0x7f056c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.886+0000 7f0572c00640 1 -- 172.21.15.86:0/2807694420 shutdown_connections 2024-04-17T07:34:33.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.886+0000 7f0572c00640 1 -- 172.21.15.86:0/2807694420 wait complete. 2024-04-17T07:34:33.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.887+0000 7f0572c00640 1 Processor -- start 2024-04-17T07:34:33.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.887+0000 7f0572c00640 1 -- start start 2024-04-17T07:34:33.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0572c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c1006e0 0x7f056c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0572c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f056c101040 0x7f056c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0572c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f056c1019e0 0x7f056c1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0572c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f056c10fbc0 con 0x7f056c1006e0 2024-04-17T07:34:33.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0572c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f056c10fa40 con 0x7f056c1019e0 2024-04-17T07:34:33.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0572c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f056c10fd40 con 0x7f056c101040 2024-04-17T07:34:33.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0571800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c1006e0 0x7f056c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0571800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c1006e0 0x7f056c19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51294/0 (socket says 172.21.15.86:51294) 2024-04-17T07:34:33.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0571800640 1 -- 172.21.15.86:0/1250868375 learned_addr learned my addr 172.21.15.86:0/1250868375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:33.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0570e00640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f056c101040 0x7f056c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0571800640 1 -- 172.21.15.86:0/1250868375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f056c101040 msgr2=0x7f056c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:33.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0571800640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f056c101040 0x7f056c19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0571800640 1 -- 172.21.15.86:0/1250868375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f056c1019e0 msgr2=0x7f056c1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:33.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0571800640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f056c1019e0 0x7f056c1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.888+0000 7f0571800640 1 -- 172.21.15.86:0/1250868375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c1a1ea0 con 0x7f056c1006e0 2024-04-17T07:34:33.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.889+0000 7f0571800640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c1006e0 0x7f056c19a280 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0560006010 tx=0x7f0560004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:33.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.889+0000 7f056a000640 1 -- 172.21.15.86:0/1250868375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560043020 con 0x7f056c1006e0 2024-04-17T07:34:33.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.889+0000 7f056a000640 1 -- 172.21.15.86:0/1250868375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0560004440 con 0x7f056c1006e0 2024-04-17T07:34:33.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.889+0000 7f0572c00640 1 -- 172.21.15.86:0/1250868375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056c1a2130 con 0x7f056c1006e0 2024-04-17T07:34:33.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.889+0000 7f056a000640 1 -- 172.21.15.86:0/1250868375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560003bf0 con 0x7f056c1006e0 2024-04-17T07:34:33.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.889+0000 7f0572c00640 1 -- 172.21.15.86:0/1250868375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056c1a2680 con 0x7f056c1006e0 2024-04-17T07:34:33.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.893+0000 7f0576fd2740 1 Processor -- start 2024-04-17T07:34:33.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.893+0000 7f056a000640 1 -- 172.21.15.86:0/1250868375 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0560046020 con 0x7f056c1006e0 2024-04-17T07:34:33.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.893+0000 7f0576fd2740 1 -- start start 2024-04-17T07:34:33.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.893+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d7480e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.893+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d74813860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.893+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74816360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.893+0000 7f0576fd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d74775120 con 0x559d74813ed0 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.893+0000 7f0576fd2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559d7471b340 con 0x559d7480e370 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.893+0000 7f0576fd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559d747710e0 con 0x559d7480ed60 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0570e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74816360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0570e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74816360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51298/0 (socket says 172.21.15.86:51298) 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0570e00640 1 -- 172.21.15.86:0/211930360 learned_addr learned my addr 172.21.15.86:0/211930360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0572200640 1 --2- 172.21.15.86:0/211930360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d7480e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0571800640 1 --2- 172.21.15.86:0/211930360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d74813860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0570e00640 1 -- 172.21.15.86:0/211930360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 msgr2=0x559d74813860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0570e00640 1 --2- 172.21.15.86:0/211930360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d74813860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0570e00640 1 -- 172.21.15.86:0/211930360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 msgr2=0x559d7480e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0570e00640 1 --2- 172.21.15.86:0/211930360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d7480e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0570e00640 1 -- 172.21.15.86:0/211930360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d745517d0 con 0x559d74813ed0 2024-04-17T07:34:33.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f0572200640 1 --2- 172.21.15.86:0/211930360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d7480e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:33.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.894+0000 7f056a000640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f05540a3bb0 0x7f05540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.895+0000 7f056a000640 1 -- 172.21.15.86:0/1250868375 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f05600f2c10 con 0x7f056c1006e0 2024-04-17T07:34:33.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.895+0000 7f0572200640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f05540a3bb0 0x7f05540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.895+0000 7f0570e00640 1 --2- 172.21.15.86:0/211930360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74816360 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0558009b60 tx=0x7f055800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99aa08de6db2d4fb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:33.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.895+0000 7f055ea00640 1 -- 172.21.15.86:0/211930360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558016070 con 0x559d74813ed0 2024-04-17T07:34:33.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.895+0000 7f055ea00640 1 -- 172.21.15.86:0/211930360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0558009e50 con 0x559d74813ed0 2024-04-17T07:34:33.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.895+0000 7f055ea00640 1 -- 172.21.15.86:0/211930360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558011470 con 0x559d74813ed0 2024-04-17T07:34:33.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.896+0000 7f0576fd2740 1 -- 172.21.15.86:0/211930360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 msgr2=0x559d74816360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:33.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.896+0000 7f0572200640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f05540a3bb0 0x7f05540a6070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x559d7480f5c0 tx=0x7f0564009ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:33.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.896+0000 7f0576fd2740 1 --2- 172.21.15.86:0/211930360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74816360 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f0558009b60 tx=0x7f055800e6c0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.896+0000 7f0576fd2740 1 -- 172.21.15.86:0/211930360 shutdown_connections 2024-04-17T07:34:33.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.896+0000 7f0576fd2740 1 --2- 172.21.15.86:0/211930360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74816360 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.896+0000 7f0576fd2740 1 --2- 172.21.15.86:0/211930360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d74813860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.896+0000 7f0576fd2740 1 --2- 172.21.15.86:0/211930360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d7480e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.896+0000 7f0576fd2740 1 -- 172.21.15.86:0/211930360 >> 172.21.15.86:0/211930360 conn(0x559d748098e0 msgr2=0x559d74809ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.896+0000 7f0576fd2740 1 -- 172.21.15.86:0/211930360 shutdown_connections 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.896+0000 7f0576fd2740 1 -- 172.21.15.86:0/211930360 wait complete. 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.897+0000 7f0576fd2740 1 -- start start 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.897+0000 7f0576fd2740 1 Processor -- start 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.897+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d748188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.897+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d74818e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.897+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d7481fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.897+0000 7f0576fd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559d74816e80 con 0x559d74813ed0 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.897+0000 7f0576fd2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559d7472e710 con 0x559d7480e370 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.897+0000 7f0576fd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559d74817000 con 0x559d7480ed60 2024-04-17T07:34:33.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.897+0000 7f0576fd2740 10 client.0 _ll_register_callbacks cb 0x559d74729af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0572200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d7481fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0572200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d7481fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51302/0 (socket says 172.21.15.86:51302) 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0572200640 1 -- 172.21.15.86:0/2518317084 learned_addr learned my addr 172.21.15.86:0/2518317084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0571800640 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d748188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0570e00640 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d74818e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0572200640 1 -- 172.21.15.86:0/2518317084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 msgr2=0x559d74818e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0572200640 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d74818e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0572200640 1 -- 172.21.15.86:0/2518317084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 msgr2=0x559d748188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0572200640 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d748188e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0572200640 1 -- 172.21.15.86:0/2518317084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d744873f0 con 0x559d74813ed0 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.898+0000 7f0572200640 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d7481fe60 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0564011c30 tx=0x7f0564004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa06036f5982df54 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:33.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.899+0000 7f0553400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.899+0000 7f0553400640 1 -- 172.21.15.86:0/2518317084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056401e620 con 0x559d74813ed0 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.899+0000 7f0553400640 1 -- 172.21.15.86:0/2518317084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f056401e7c0 con 0x559d74813ed0 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.899+0000 7f0576fd2740 1 -- 172.21.15.86:0/2518317084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0558009e50 con 0x559d74813ed0 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.899+0000 7f0553400640 1 -- 172.21.15.86:0/2518317084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f056400ee90 con 0x559d74813ed0 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.899+0000 7f0576fd2740 10 client.19902 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.899+0000 7f0576fd2740 1 -- 172.21.15.86:0/2518317084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d744470f0 con 0x559d74813ed0 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.899+0000 7f0553400640 1 -- 172.21.15.86:0/2518317084 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0558009e50 con 0x559d74813ed0 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.900+0000 7f0553400640 1 -- 172.21.15.86:0/2518317084 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0564026930 con 0x559d74813ed0 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.900+0000 7f0576fd2740 10 client.19902 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.900+0000 7f0576fd2740 10 client.19902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.900+0000 7f0576fd2740 20 client.19902 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:33.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.900+0000 7f0576fd2740 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559d748247d0 0x559d74826c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.900+0000 7f0576fd2740 4 client.19902 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:33.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.900+0000 7f0576fd2740 1 -- 172.21.15.86:0/2518317084 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d74782b40 con 0x559d748247d0 2024-04-17T07:34:33.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.900+0000 7f0571800640 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559d748247d0 0x559d74826c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.901+0000 7f0571800640 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559d748247d0 0x559d74826c90 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85450d2adbb4e207 server_cookie=3e65cf8d22a42ab5 in_seq=0 out_seq=0 2024-04-17T07:34:33.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.901+0000 7f0553400640 10 client.19902 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:33.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0553400640 1 -- 172.21.15.86:0/2518317084 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d74782b40 con 0x559d748247d0 2024-04-17T07:34:33.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0553400640 10 client.19902 handle_command_reply: tid=0 2024-04-17T07:34:33.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 client.19902 shutdown 2024-04-17T07:34:33.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 -- 172.21.15.86:0/2518317084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 msgr2=0x559d7481fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d7481fe60 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0564011c30 tx=0x7f0564004a40 comp rx=0 tx=0).stop 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 -- 172.21.15.86:0/2518317084 shutdown_connections 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559d748247d0 0x559d74826c90 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d7481fe60 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d74818e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0553400640 0 client.19902 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 --2- 172.21.15.86:0/2518317084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d748188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 -- 172.21.15.86:0/2518317084 >> 172.21.15.86:0/2518317084 conn(0x559d748098e0 msgr2=0x559d74808ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 -- 172.21.15.86:0/2518317084 shutdown_connections 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 1 -- 172.21.15.86:0/2518317084 wait complete. 2024-04-17T07:34:33.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.905+0000 7f0576fd2740 20 client.19902 trim_cache size 0 max 16384 2024-04-17T07:34:33.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.927+0000 7f0576fd2740 1 Processor -- start 2024-04-17T07:34:33.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.927+0000 7f0576fd2740 1 -- start start 2024-04-17T07:34:33.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.927+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d74808250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.927+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d7481fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.927+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74808790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.927+0000 7f0576fd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f056c1aca70 con 0x559d74813ed0 2024-04-17T07:34:33.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.927+0000 7f0576fd2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f056c1ae5b0 con 0x559d7480e370 2024-04-17T07:34:33.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.927+0000 7f0576fd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f056c1ad140 con 0x559d7480ed60 2024-04-17T07:34:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.927+0000 7f0572200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74808790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.928+0000 7f0572200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74808790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51304/0 (socket says 172.21.15.86:51304) 2024-04-17T07:34:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.928+0000 7f0572200640 1 -- 172.21.15.86:0/2338659804 learned_addr learned my addr 172.21.15.86:0/2338659804 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.928+0000 7f0571800640 1 --2- 172.21.15.86:0/2338659804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d74808250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.928+0000 7f0570e00640 1 --2- 172.21.15.86:0/2338659804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d7481fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.928+0000 7f0572200640 1 -- 172.21.15.86:0/2338659804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 msgr2=0x559d7481fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.928+0000 7f0572200640 1 --2- 172.21.15.86:0/2338659804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d7481fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.928+0000 7f0572200640 1 -- 172.21.15.86:0/2338659804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 msgr2=0x559d74808250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.928+0000 7f0572200640 1 --2- 172.21.15.86:0/2338659804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d74808250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.928+0000 7f0572200640 1 -- 172.21.15.86:0/2338659804 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7480a650 con 0x559d74813ed0 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.928+0000 7f0571800640 1 --2- 172.21.15.86:0/2338659804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d74808250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.929+0000 7f0572200640 1 --2- 172.21.15.86:0/2338659804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74808790 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f056400eec0 tx=0x7f056400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2109ca9f8fc11a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.929+0000 7f055e000640 1 -- 172.21.15.86:0/2338659804 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564009df0 con 0x559d74813ed0 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.929+0000 7f055e000640 1 -- 172.21.15.86:0/2338659804 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0564003e20 con 0x559d74813ed0 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.929+0000 7f055e000640 1 -- 172.21.15.86:0/2338659804 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564026660 con 0x559d74813ed0 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 -- 172.21.15.86:0/2338659804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 msgr2=0x559d74808790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 --2- 172.21.15.86:0/2338659804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74808790 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f056400eec0 tx=0x7f056400eef0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 -- 172.21.15.86:0/2338659804 shutdown_connections 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 --2- 172.21.15.86:0/2338659804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d74813ed0 0x559d74808790 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 --2- 172.21.15.86:0/2338659804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d7481fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 --2- 172.21.15.86:0/2338659804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d7480e370 0x559d74808250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 -- 172.21.15.86:0/2338659804 >> 172.21.15.86:0/2338659804 conn(0x559d748098e0 msgr2=0x559d74810260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:33.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 -- 172.21.15.86:0/2338659804 shutdown_connections 2024-04-17T07:34:33.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 -- 172.21.15.86:0/2338659804 wait complete. 2024-04-17T07:34:33.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 -- start start 2024-04-17T07:34:33.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.930+0000 7f0576fd2740 1 Processor -- start 2024-04-17T07:34:33.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d7480e370 0x559d7481dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d7481e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0576fd2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d74813ed0 0x559d7481f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0576fd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f056c1adee0 con 0x559d7480e370 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0576fd2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f056c1ad810 con 0x559d74813ed0 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0576fd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f056c1aaa40 con 0x559d7480ed60 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0576fd2740 10 client.0 _ll_register_callbacks cb 0x7f0564010f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0571800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d7480e370 0x559d7481dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0571800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d7480e370 0x559d7481dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51310/0 (socket says 172.21.15.86:51310) 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0571800640 1 -- 172.21.15.86:0/943126064 learned_addr learned my addr 172.21.15.86:0/943126064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0571800640 1 -- 172.21.15.86:0/943126064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 msgr2=0x559d7481e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0570e00640 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d7481e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0572200640 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d74813ed0 0x559d7481f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.931+0000 7f0571800640 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d7481e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0571800640 1 -- 172.21.15.86:0/943126064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d74813ed0 msgr2=0x559d7481f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0571800640 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d74813ed0 0x559d7481f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0571800640 1 -- 172.21.15.86:0/943126064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7481f810 con 0x559d7480e370 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0572200640 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d74813ed0 0x559d7481f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0571800640 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d7480e370 0x559d7481dcd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0560005ab0 tx=0x7f05600f0200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6db5d5b9c47a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0553400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0553400640 1 -- 172.21.15.86:0/943126064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05600beb10 con 0x559d7480e370 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0553400640 1 -- 172.21.15.86:0/943126064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f05600df0b0 con 0x559d7480e370 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0576fd2740 1 -- 172.21.15.86:0/943126064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0564003e20 con 0x559d7480e370 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0553400640 1 -- 172.21.15.86:0/943126064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05600c2ec0 con 0x559d7480e370 2024-04-17T07:34:33.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0576fd2740 10 client.19908 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:33.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.932+0000 7f0576fd2740 1 -- 172.21.15.86:0/943126064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d74893e20 con 0x559d7480e370 2024-04-17T07:34:33.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.933+0000 7f0553400640 1 -- 172.21.15.86:0/943126064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0564003e20 con 0x559d7480e370 2024-04-17T07:34:33.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.933+0000 7f0553400640 1 -- 172.21.15.86:0/943126064 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f05600bed70 con 0x559d7480e370 2024-04-17T07:34:33.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.933+0000 7f0576fd2740 10 client.19908 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:33.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.933+0000 7f0576fd2740 10 client.19908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:33.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.933+0000 7f0576fd2740 20 client.19908 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:33.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.933+0000 7f0576fd2740 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559d748247d0 0x559d74896830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:33.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.933+0000 7f0576fd2740 4 client.19908 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:34:33.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.933+0000 7f0576fd2740 1 -- 172.21.15.86:0/943126064 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d7477c9b0 con 0x559d748247d0 2024-04-17T07:34:33.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.933+0000 7f0570e00640 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559d748247d0 0x559d74896830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:33.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.935+0000 7f0570e00640 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559d748247d0 0x559d74896830 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc366d3ae1b48f9 server_cookie=9c41346a88af18c in_seq=0 out_seq=0 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.935+0000 7f0553400640 10 client.19908 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0553400640 1 -- 172.21.15.86:0/943126064 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d7477c9b0 con 0x559d748247d0 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0553400640 10 client.19908 handle_command_reply: tid=0 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 client.19908 shutdown 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 -- 172.21.15.86:0/943126064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d7480e370 msgr2=0x559d7481dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d7480e370 0x559d7481dcd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0560005ab0 tx=0x7f05600f0200 comp rx=0 tx=0).stop 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 -- 172.21.15.86:0/943126064 shutdown_connections 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x559d748247d0 0x559d74896830 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559d74813ed0 0x559d7481f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7480ed60 0x559d7481e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 --2- 172.21.15.86:0/943126064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559d7480e370 0x559d7481dcd0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0553400640 0 client.19908 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:33.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 -- 172.21.15.86:0/943126064 >> 172.21.15.86:0/943126064 conn(0x559d748098e0 msgr2=0x559d7480cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:33.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 -- 172.21.15.86:0/943126064 shutdown_connections 2024-04-17T07:34:33.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 1 -- 172.21.15.86:0/943126064 wait complete. 2024-04-17T07:34:33.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.937+0000 7f0576fd2740 20 client.19908 trim_cache size 0 max 16384 2024-04-17T07:34:33.939 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:34:33.939 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:34:33.939 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:34:33.939 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:34:33.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 -- 172.21.15.86:0/1250868375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f05540a3bb0 msgr2=0x7f05540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:33.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f05540a3bb0 0x7f05540a6070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x559d7480f5c0 tx=0x7f0564009ba0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 -- 172.21.15.86:0/1250868375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c1006e0 msgr2=0x7f056c19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:33.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c1006e0 0x7f056c19a280 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f0560006010 tx=0x7f0560004040 comp rx=0 tx=0).stop 2024-04-17T07:34:33.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 -- 172.21.15.86:0/1250868375 shutdown_connections 2024-04-17T07:34:33.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f05540a3bb0 0x7f05540a6070 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f056c1019e0 0x7f056c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f056c101040 0x7f056c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 --2- 172.21.15.86:0/1250868375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f056c1006e0 0x7f056c19a280 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:33.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 -- 172.21.15.86:0/1250868375 >> 172.21.15.86:0/1250868375 conn(0x7f056c0fc300 msgr2=0x7f056c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:33.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 -- 172.21.15.86:0/1250868375 shutdown_connections 2024-04-17T07:34:33.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:33.938+0000 7f0572c00640 1 -- 172.21.15.86:0/1250868375 wait complete. 2024-04-17T07:34:33.955 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:34:33.956 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:34:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.187+0000 7fcaac800640 1 -- 172.21.15.86:0/2648046489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa4101b80 msgr2=0x7fcaa4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.187+0000 7fcaac800640 1 --2- 172.21.15.86:0/2648046489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa4101b80 0x7fcaa4102000 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009ca0 tx=0x7fcaa002c100 comp rx=0 tx=0).stop 2024-04-17T07:34:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.187+0000 7fcaac800640 1 -- 172.21.15.86:0/2648046489 shutdown_connections 2024-04-17T07:34:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.187+0000 7fcaac800640 1 --2- 172.21.15.86:0/2648046489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa4101b80 0x7fcaa4102000 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.187+0000 7fcaac800640 1 --2- 172.21.15.86:0/2648046489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa41011e0 0x7fcaa4101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.187+0000 7fcaac800640 1 --2- 172.21.15.86:0/2648046489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa4100880 0x7fcaa4100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.187+0000 7fcaac800640 1 -- 172.21.15.86:0/2648046489 >> 172.21.15.86:0/2648046489 conn(0x7fcaa40fc440 msgr2=0x7fcaa40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:34.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.187+0000 7fcaac800640 1 -- 172.21.15.86:0/2648046489 shutdown_connections 2024-04-17T07:34:34.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.187+0000 7fcaac800640 1 -- 172.21.15.86:0/2648046489 wait complete. 2024-04-17T07:34:34.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.188+0000 7fcaac800640 1 Processor -- start 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.188+0000 7fcaac800640 1 -- start start 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.188+0000 7fcaac800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa4100880 0x7fcaa419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.188+0000 7fcaac800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa41011e0 0x7fcaa419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.188+0000 7fcaac800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa4101b80 0x7fcaa41a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaac800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa410fd00 con 0x7fcaa4100880 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaac800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa410fb80 con 0x7fcaa41011e0 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaac800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa410fe80 con 0x7fcaa4101b80 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaa9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa4100880 0x7fcaa419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaa9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa4100880 0x7fcaa419a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51328/0 (socket says 172.21.15.86:51328) 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaa9e00640 1 -- 172.21.15.86:0/4240506216 learned_addr learned my addr 172.21.15.86:0/4240506216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaa9e00640 1 -- 172.21.15.86:0/4240506216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa4101b80 msgr2=0x7fcaa41a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaa9e00640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa4101b80 0x7fcaa41a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaa9e00640 1 -- 172.21.15.86:0/4240506216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa41011e0 msgr2=0x7fcaa419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaa9e00640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa41011e0 0x7fcaa419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.189+0000 7fcaa9e00640 1 -- 172.21.15.86:0/4240506216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa41a1f30 con 0x7fcaa4100880 2024-04-17T07:34:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.190+0000 7fcaa9e00640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa4100880 0x7fcaa419a310 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fca9800c920 tx=0x7fca9800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.190+0000 7fca9ea00640 1 -- 172.21.15.86:0/4240506216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98007c70 con 0x7fcaa4100880 2024-04-17T07:34:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.190+0000 7fca9ea00640 1 -- 172.21.15.86:0/4240506216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca98007e10 con 0x7fcaa4100880 2024-04-17T07:34:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.190+0000 7fcaac800640 1 -- 172.21.15.86:0/4240506216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa41a2220 con 0x7fcaa4100880 2024-04-17T07:34:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.190+0000 7fca9ea00640 1 -- 172.21.15.86:0/4240506216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98018820 con 0x7fcaa4100880 2024-04-17T07:34:34.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.190+0000 7fcaac800640 1 -- 172.21.15.86:0/4240506216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa41a2770 con 0x7fcaa4100880 2024-04-17T07:34:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.191+0000 7fcaac800640 1 -- 172.21.15.86:0/4240506216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa410b760 con 0x7fcaa4100880 2024-04-17T07:34:34.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.193+0000 7fca9ea00640 1 -- 172.21.15.86:0/4240506216 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fca980189c0 con 0x7fcaa4100880 2024-04-17T07:34:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.194+0000 7fca9ea00640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fca8c0a3bb0 0x7fca8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.195+0000 7fca9ea00640 1 -- 172.21.15.86:0/4240506216 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fca980d09c0 con 0x7fcaa4100880 2024-04-17T07:34:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.195+0000 7fcaa9400640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fca8c0a3bb0 0x7fca8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.195+0000 7fcaa9400640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fca8c0a3bb0 0x7fca8c0a6070 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fca94005fd0 tx=0x7fca94005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:34.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.200+0000 7fca9ea00640 1 -- 172.21.15.86:0/4240506216 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fca98014030 con 0x7fcaa4100880 2024-04-17T07:34:34.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.467+0000 7fcaac800640 1 -- 172.21.15.86:0/4240506216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaa4062c60 con 0x7fcaa4100880 2024-04-17T07:34:34.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.469+0000 7fca9ea00640 1 -- 172.21.15.86:0/4240506216 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fca9809cfc0 con 0x7fcaa4100880 2024-04-17T07:34:34.468 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:34.469 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:34:34.469 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:34:34.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.472+0000 7fcaac800640 1 -- 172.21.15.86:0/4240506216 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fca8c0a3bb0 msgr2=0x7fca8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.472+0000 7fcaac800640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fca8c0a3bb0 0x7fca8c0a6070 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fca94005fd0 tx=0x7fca94005850 comp rx=0 tx=0).stop 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.472+0000 7fcaac800640 1 -- 172.21.15.86:0/4240506216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa4100880 msgr2=0x7fcaa419a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.472+0000 7fcaac800640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa4100880 0x7fcaa419a310 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fca9800c920 tx=0x7fca9800cde0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.472+0000 7fcaac800640 1 -- 172.21.15.86:0/4240506216 shutdown_connections 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.472+0000 7fcaac800640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fca8c0a3bb0 0x7fca8c0a6070 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.472+0000 7fcaac800640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa4101b80 0x7fcaa41a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.472+0000 7fcaac800640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcaa41011e0 0x7fcaa419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.472+0000 7fcaac800640 1 --2- 172.21.15.86:0/4240506216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa4100880 0x7fcaa419a310 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.472+0000 7fcaac800640 1 -- 172.21.15.86:0/4240506216 >> 172.21.15.86:0/4240506216 conn(0x7fcaa40fc440 msgr2=0x7fcaa40fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.473+0000 7fcaac800640 1 -- 172.21.15.86:0/4240506216 shutdown_connections 2024-04-17T07:34:34.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.473+0000 7fcaac800640 1 -- 172.21.15.86:0/4240506216 wait complete. 2024-04-17T07:34:34.494 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:34:34.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.725+0000 7f9d71200640 1 -- 172.21.15.86:0/1105293776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d6c102fe0 msgr2=0x7f9d6c103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.725+0000 7f9d71200640 1 --2- 172.21.15.86:0/1105293776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d6c102fe0 0x7f9d6c103460 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009900 tx=0x7f9d5802bc30 comp rx=0 tx=0).stop 2024-04-17T07:34:34.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.725+0000 7f9d71200640 1 -- 172.21.15.86:0/1105293776 shutdown_connections 2024-04-17T07:34:34.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.725+0000 7f9d71200640 1 --2- 172.21.15.86:0/1105293776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d6c1039a0 0x7f9d6c10a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.725+0000 7f9d71200640 1 --2- 172.21.15.86:0/1105293776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d6c102fe0 0x7f9d6c103460 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.725+0000 7f9d71200640 1 --2- 172.21.15.86:0/1105293776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c101e90 0x7f9d6c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.725+0000 7f9d71200640 1 -- 172.21.15.86:0/1105293776 >> 172.21.15.86:0/1105293776 conn(0x7f9d6c0fd400 msgr2=0x7f9d6c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:34.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.725+0000 7f9d71200640 1 -- 172.21.15.86:0/1105293776 shutdown_connections 2024-04-17T07:34:34.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.725+0000 7f9d71200640 1 -- 172.21.15.86:0/1105293776 wait complete. 2024-04-17T07:34:34.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.726+0000 7f9d71200640 1 Processor -- start 2024-04-17T07:34:34.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.726+0000 7f9d71200640 1 -- start start 2024-04-17T07:34:34.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.726+0000 7f9d71200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d6c101e90 0x7f9d6c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.726+0000 7f9d71200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c102fe0 0x7f9d6c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.726+0000 7f9d71200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d6c1039a0 0x7f9d6c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d71200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c10d9f0 con 0x7f9d6c101e90 2024-04-17T07:34:34.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d71200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c10d870 con 0x7f9d6c1039a0 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d71200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c10db70 con 0x7f9d6c102fe0 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d6c101e90 0x7f9d6c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d6c101e90 0x7f9d6c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51348/0 (socket says 172.21.15.86:51348) 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6b400640 1 -- 172.21.15.86:0/1958023399 learned_addr learned my addr 172.21.15.86:0/1958023399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6be00640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d6c1039a0 0x7f9d6c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6aa00640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c102fe0 0x7f9d6c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6b400640 1 -- 172.21.15.86:0/1958023399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c102fe0 msgr2=0x7f9d6c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6b400640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c102fe0 0x7f9d6c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6b400640 1 -- 172.21.15.86:0/1958023399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d6c1039a0 msgr2=0x7f9d6c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6b400640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d6c1039a0 0x7f9d6c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6b400640 1 -- 172.21.15.86:0/1958023399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6c19fcc0 con 0x7f9d6c101e90 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.727+0000 7f9d6be00640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d6c1039a0 0x7f9d6c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:34.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.728+0000 7f9d6b400640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d6c101e90 0x7f9d6c1980a0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c970 tx=0x7f9d5c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.728+0000 7f9d63e00640 1 -- 172.21.15.86:0/1958023399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c007bb0 con 0x7f9d6c101e90 2024-04-17T07:34:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.728+0000 7f9d63e00640 1 -- 172.21.15.86:0/1958023399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d5c007d50 con 0x7f9d6c101e90 2024-04-17T07:34:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.728+0000 7f9d71200640 1 -- 172.21.15.86:0/1958023399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6c19ffb0 con 0x7f9d6c101e90 2024-04-17T07:34:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.728+0000 7f9d63e00640 1 -- 172.21.15.86:0/1958023399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c00f450 con 0x7f9d6c101e90 2024-04-17T07:34:34.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.728+0000 7f9d71200640 1 -- 172.21.15.86:0/1958023399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d6c1a03c0 con 0x7f9d6c101e90 2024-04-17T07:34:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d755fe740 1 Processor -- start 2024-04-17T07:34:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d755fe740 1 -- start start 2024-04-17T07:34:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d63e00640 1 -- 172.21.15.86:0/1958023399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f9d5c004f20 con 0x7f9d6c101e90 2024-04-17T07:34:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5e160 0x561daef5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5eb50 0x561daef63650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 0x561daef66150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d755fe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561daeec4020 con 0x561daef5eb50 2024-04-17T07:34:34.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d755fe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561daee6a460 con 0x561daef5e160 2024-04-17T07:34:34.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d755fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561daeebffe0 con 0x561daef63cc0 2024-04-17T07:34:34.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5eb50 0x561daef63650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.732+0000 7f9d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5eb50 0x561daef63650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51364/0 (socket says 172.21.15.86:51364) 2024-04-17T07:34:34.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.733+0000 7f9d6b400640 1 -- 172.21.15.86:0/2988644660 learned_addr learned my addr 172.21.15.86:0/2988644660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:34.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.733+0000 7f9d6aa00640 1 --2- 172.21.15.86:0/2988644660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 0x561daef66150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.733+0000 7f9d6be00640 1 --2- 172.21.15.86:0/2988644660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5e160 0x561daef5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.733+0000 7f9d6b400640 1 -- 172.21.15.86:0/2988644660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 msgr2=0x561daef66150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.733+0000 7f9d6b400640 1 --2- 172.21.15.86:0/2988644660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 0x561daef66150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.733+0000 7f9d6b400640 1 -- 172.21.15.86:0/2988644660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5e160 msgr2=0x561daef5e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.733+0000 7f9d6b400640 1 --2- 172.21.15.86:0/2988644660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5e160 0x561daef5e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.733+0000 7f9d6b400640 1 -- 172.21.15.86:0/2988644660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561daee72040 con 0x561daef5eb50 2024-04-17T07:34:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.733+0000 7f9d6aa00640 1 --2- 172.21.15.86:0/2988644660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 0x561daef66150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.733+0000 7f9d6b400640 1 --2- 172.21.15.86:0/2988644660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5eb50 0x561daef63650 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0d67f0 tx=0x7f9d5c0d6cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed523c742eb89b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.734+0000 7f9d63e00640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9d500a3bb0 0x7f9d500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.734+0000 7f9d60c00640 1 -- 172.21.15.86:0/2988644660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0e3070 con 0x561daef5eb50 2024-04-17T07:34:34.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.734+0000 7f9d60c00640 1 -- 172.21.15.86:0/2988644660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d5c09cc10 con 0x561daef5eb50 2024-04-17T07:34:34.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.734+0000 7f9d60c00640 1 -- 172.21.15.86:0/2988644660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0bce50 con 0x561daef5eb50 2024-04-17T07:34:34.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.734+0000 7f9d6be00640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9d500a3bb0 0x7f9d500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.734+0000 7f9d63e00640 1 -- 172.21.15.86:0/1958023399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9d5c0d0610 con 0x7f9d6c101e90 2024-04-17T07:34:34.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d755fe740 1 -- 172.21.15.86:0/2988644660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5eb50 msgr2=0x561daef63650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d755fe740 1 --2- 172.21.15.86:0/2988644660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5eb50 0x561daef63650 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0d67f0 tx=0x7f9d5c0d6cb0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d6be00640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9d500a3bb0 0x7f9d500a6070 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x561daef5f3b0 tx=0x7f9d64008470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:34.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d755fe740 1 -- 172.21.15.86:0/2988644660 shutdown_connections 2024-04-17T07:34:34.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d755fe740 1 --2- 172.21.15.86:0/2988644660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 0x561daef66150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d755fe740 1 --2- 172.21.15.86:0/2988644660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5eb50 0x561daef63650 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d755fe740 1 --2- 172.21.15.86:0/2988644660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5e160 0x561daef5e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d755fe740 1 -- 172.21.15.86:0/2988644660 >> 172.21.15.86:0/2988644660 conn(0x561daef596d0 msgr2=0x561daef59ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:34.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d755fe740 1 -- 172.21.15.86:0/2988644660 shutdown_connections 2024-04-17T07:34:34.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d755fe740 1 -- 172.21.15.86:0/2988644660 wait complete. 2024-04-17T07:34:34.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.735+0000 7f9d755fe740 1 -- start start 2024-04-17T07:34:34.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.736+0000 7f9d755fe740 1 Processor -- start 2024-04-17T07:34:34.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.736+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5e160 0x561daef6cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.736+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef5eb50 0x561daef6d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.736+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef74010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.736+0000 7f9d755fe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561daef66c70 con 0x561daef63cc0 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.736+0000 7f9d755fe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561daee7d830 con 0x561daef5e160 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.736+0000 7f9d755fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561daef66df0 con 0x561daef5eb50 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.736+0000 7f9d755fe740 10 client.0 _ll_register_callbacks cb 0x561daee4bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.736+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef74010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef74010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51378/0 (socket says 172.21.15.86:51378) 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d6be00640 1 -- 172.21.15.86:0/1529295270 learned_addr learned my addr 172.21.15.86:0/1529295270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d6b400640 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5e160 0x561daef6cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d6aa00640 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef5eb50 0x561daef6d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d6be00640 1 -- 172.21.15.86:0/1529295270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef5eb50 msgr2=0x561daef6d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:34.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d6be00640 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef5eb50 0x561daef6d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d6be00640 1 -- 172.21.15.86:0/1529295270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5e160 msgr2=0x561daef6cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d6be00640 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5e160 0x561daef6cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d6be00640 1 -- 172.21.15.86:0/1529295270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561daee81920 con 0x561daef63cc0 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d6be00640 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef74010 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f9d64010e30 tx=0x7f9d64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fa725664ea1548 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.737+0000 7f9d55600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.738+0000 7f9d55600640 1 -- 172.21.15.86:0/1529295270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6401d620 con 0x561daef63cc0 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.738+0000 7f9d55600640 1 -- 172.21.15.86:0/1529295270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9d6401d7c0 con 0x561daef63cc0 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.738+0000 7f9d755fe740 1 -- 172.21.15.86:0/1529295270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5c09cc10 con 0x561daef63cc0 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.738+0000 7f9d55600640 1 -- 172.21.15.86:0/1529295270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d6401d9c0 con 0x561daef63cc0 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.738+0000 7f9d755fe740 10 client.19926 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.738+0000 7f9d755fe740 1 -- 172.21.15.86:0/1529295270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561daee4f7f0 con 0x561daef63cc0 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.738+0000 7f9d55600640 1 -- 172.21.15.86:0/1529295270 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d5c09cc10 con 0x561daef63cc0 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.738+0000 7f9d55600640 1 -- 172.21.15.86:0/1529295270 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9d64025ae0 con 0x561daef63cc0 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.738+0000 7f9d755fe740 10 client.19926 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:34.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.739+0000 7f9d755fe740 10 client.19926 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:34:34.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.739+0000 7f9d755fe740 20 client.19926 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:34.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.739+0000 7f9d755fe740 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x561daef785d0 0x561daef7aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.739+0000 7f9d755fe740 4 client.19926 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:34.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.739+0000 7f9d755fe740 1 -- 172.21.15.86:0/1529295270 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561daeed1a40 con 0x561daef785d0 2024-04-17T07:34:34.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.739+0000 7f9d6b400640 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x561daef785d0 0x561daef7aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.740+0000 7f9d6b400640 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x561daef785d0 0x561daef7aa70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=988e689ff8b68647 server_cookie=c54a064c68a210d3 in_seq=0 out_seq=0 2024-04-17T07:34:34.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.740+0000 7f9d55600640 10 client.19926 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:34:34.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d55600640 1 -- 172.21.15.86:0/1529295270 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561daeed1a40 con 0x561daef785d0 2024-04-17T07:34:34.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d55600640 10 client.19926 handle_command_reply: tid=0 2024-04-17T07:34:34.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d755fe740 1 client.19926 shutdown 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d755fe740 1 -- 172.21.15.86:0/1529295270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 msgr2=0x561daef74010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d755fe740 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef74010 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f9d64010e30 tx=0x7f9d64004a40 comp rx=0 tx=0).stop 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d755fe740 1 -- 172.21.15.86:0/1529295270 shutdown_connections 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d755fe740 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x561daef785d0 0x561daef7aa70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d755fe740 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef74010 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d755fe740 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef5eb50 0x561daef6d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d55600640 0 client.19926 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d755fe740 1 --2- 172.21.15.86:0/1529295270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5e160 0x561daef6cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.742+0000 7f9d755fe740 1 -- 172.21.15.86:0/1529295270 >> 172.21.15.86:0/1529295270 conn(0x561daef596d0 msgr2=0x561daef58930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.743+0000 7f9d755fe740 1 -- 172.21.15.86:0/1529295270 shutdown_connections 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.743+0000 7f9d755fe740 1 -- 172.21.15.86:0/1529295270 wait complete. 2024-04-17T07:34:34.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.743+0000 7f9d755fe740 20 client.19926 trim_cache size 0 max 16384 2024-04-17T07:34:34.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.759+0000 7f9d755fe740 1 Processor -- start 2024-04-17T07:34:34.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.759+0000 7f9d755fe740 1 -- start start 2024-04-17T07:34:34.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.759+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5e160 0x561daef580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.759+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 0x561daef73fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.759+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 0x561daef585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.759+0000 7f9d755fe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1a8730 con 0x561daef5e160 2024-04-17T07:34:34.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.759+0000 7f9d755fe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1aa270 con 0x561daef5eb50 2024-04-17T07:34:34.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.759+0000 7f9d755fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1a8e00 con 0x561daef63cc0 2024-04-17T07:34:34.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5e160 0x561daef580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5e160 0x561daef580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51382/0 (socket says 172.21.15.86:51382) 2024-04-17T07:34:34.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6b400640 1 -- 172.21.15.86:0/3989434181 learned_addr learned my addr 172.21.15.86:0/3989434181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:34.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6be00640 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 0x561daef585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6aa00640 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 0x561daef73fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6b400640 1 -- 172.21.15.86:0/3989434181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 msgr2=0x561daef585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6b400640 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 0x561daef585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6b400640 1 -- 172.21.15.86:0/3989434181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 msgr2=0x561daef73fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6b400640 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 0x561daef73fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6b400640 1 -- 172.21.15.86:0/3989434181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561daebd63f0 con 0x561daef5e160 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6be00640 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 0x561daef585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6aa00640 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 0x561daef73fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.760+0000 7f9d6b400640 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5e160 0x561daef580a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c098970 tx=0x7f9d5c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b994d0c8cbe5b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.761+0000 7f9d57e00640 1 -- 172.21.15.86:0/3989434181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0c6b40 con 0x561daef5e160 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.761+0000 7f9d57e00640 1 -- 172.21.15.86:0/3989434181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d5c0bce50 con 0x561daef5e160 2024-04-17T07:34:34.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.761+0000 7f9d57e00640 1 -- 172.21.15.86:0/3989434181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0c4650 con 0x561daef5e160 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 -- 172.21.15.86:0/3989434181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5e160 msgr2=0x561daef580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5e160 0x561daef580a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c098970 tx=0x7f9d5c087040 comp rx=0 tx=0).stop 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 -- 172.21.15.86:0/3989434181 shutdown_connections 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef63cc0 0x561daef585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 0x561daef73fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 --2- 172.21.15.86:0/3989434181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef5e160 0x561daef580a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 -- 172.21.15.86:0/3989434181 >> 172.21.15.86:0/3989434181 conn(0x561daef596d0 msgr2=0x561daef67af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 -- 172.21.15.86:0/3989434181 shutdown_connections 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 -- 172.21.15.86:0/3989434181 wait complete. 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 -- start start 2024-04-17T07:34:34.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.762+0000 7f9d755fe740 1 Processor -- start 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef5e160 0x561daef71860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 0x561daef71da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d755fe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef72d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d755fe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1a9ba0 con 0x561daef63cc0 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d755fe740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1a94d0 con 0x561daef5eb50 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d755fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c1a6700 con 0x561daef5e160 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d755fe740 10 client.0 _ll_register_callbacks cb 0x7f9d64010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef72d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef72d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51384/0 (socket says 172.21.15.86:51384) 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d6be00640 1 -- 172.21.15.86:0/3624871260 learned_addr learned my addr 172.21.15.86:0/3624871260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d6aa00640 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 0x561daef71da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d6b400640 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef5e160 0x561daef71860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.763+0000 7f9d6be00640 1 -- 172.21.15.86:0/3624871260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef5e160 msgr2=0x561daef71860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d6be00640 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef5e160 0x561daef71860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d6be00640 1 -- 172.21.15.86:0/3624871260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 msgr2=0x561daef71da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d6be00640 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 0x561daef71da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d6be00640 1 -- 172.21.15.86:0/3624871260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561daeca07d0 con 0x561daef63cc0 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d6aa00640 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 0x561daef71da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d6be00640 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef72d40 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400e010 tx=0x7f9d6401dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df7ab27c62e067d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d55600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d55600640 1 -- 172.21.15.86:0/3624871260 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400dc10 con 0x561daef63cc0 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d55600640 1 -- 172.21.15.86:0/3624871260 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f9d6401b8f0 con 0x561daef63cc0 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d755fe740 1 -- 172.21.15.86:0/3624871260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5c0bce50 con 0x561daef63cc0 2024-04-17T07:34:34.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d55600640 1 -- 172.21.15.86:0/3624871260 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d64046050 con 0x561daef63cc0 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d755fe740 10 client.19932 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.764+0000 7f9d755fe740 1 -- 172.21.15.86:0/3624871260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561daefdebb0 con 0x561daef63cc0 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.765+0000 7f9d55600640 1 -- 172.21.15.86:0/3624871260 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d5c0bce50 con 0x561daef63cc0 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.765+0000 7f9d55600640 1 -- 172.21.15.86:0/3624871260 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f9d64025850 con 0x561daef63cc0 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.765+0000 7f9d755fe740 10 client.19932 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.765+0000 7f9d755fe740 10 client.19932 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.765+0000 7f9d755fe740 20 client.19932 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.765+0000 7f9d755fe740 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x561daefdf4b0 0x561daefe18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.765+0000 7f9d755fe740 4 client.19932 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.765+0000 7f9d755fe740 1 -- 172.21.15.86:0/3624871260 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561daeecb8b0 con 0x561daefdf4b0 2024-04-17T07:34:34.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.766+0000 7f9d6b400640 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x561daefdf4b0 0x561daefe18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.767+0000 7f9d6b400640 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x561daefdf4b0 0x561daefe18d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a4f6c447f5e1f01 server_cookie=f0f7659e408744a0 in_seq=0 out_seq=0 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.767+0000 7f9d55600640 10 client.19932 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.767+0000 7f9d55600640 1 -- 172.21.15.86:0/3624871260 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561daeecb8b0 con 0x561daefdf4b0 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.767+0000 7f9d55600640 10 client.19932 handle_command_reply: tid=0 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.767+0000 7f9d755fe740 1 client.19932 shutdown 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 1 -- 172.21.15.86:0/3624871260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 msgr2=0x561daef72d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef72d40 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400e010 tx=0x7f9d6401dd90 comp rx=0 tx=0).stop 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 1 -- 172.21.15.86:0/3624871260 shutdown_connections 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x561daefdf4b0 0x561daefe18d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561daef63cc0 0x561daef72d40 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561daef5eb50 0x561daef71da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d55600640 0 client.19932 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:34:34.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 1 --2- 172.21.15.86:0/3624871260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561daef5e160 0x561daef71860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 1 -- 172.21.15.86:0/3624871260 >> 172.21.15.86:0/3624871260 conn(0x561daef596d0 msgr2=0x561daef5c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:34.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 1 -- 172.21.15.86:0/3624871260 shutdown_connections 2024-04-17T07:34:34.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 1 -- 172.21.15.86:0/3624871260 wait complete. 2024-04-17T07:34:34.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.768+0000 7f9d755fe740 20 client.19932 trim_cache size 0 max 16384 2024-04-17T07:34:34.771 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:34:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 -- 172.21.15.86:0/1958023399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9d500a3bb0 msgr2=0x7f9d500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9d500a3bb0 0x7f9d500a6070 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x561daef5f3b0 tx=0x7f9d64008470 comp rx=0 tx=0).stop 2024-04-17T07:34:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 -- 172.21.15.86:0/1958023399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d6c101e90 msgr2=0x7f9d6c1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d6c101e90 0x7f9d6c1980a0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c970 tx=0x7f9d5c00ce30 comp rx=0 tx=0).stop 2024-04-17T07:34:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 -- 172.21.15.86:0/1958023399 shutdown_connections 2024-04-17T07:34:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f9d500a3bb0 0x7f9d500a6070 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d6c1039a0 0x7f9d6c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d6c102fe0 0x7f9d6c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 --2- 172.21.15.86:0/1958023399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d6c101e90 0x7f9d6c1980a0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:34.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 -- 172.21.15.86:0/1958023399 >> 172.21.15.86:0/1958023399 conn(0x7f9d6c0fd400 msgr2=0x7f9d6c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 -- 172.21.15.86:0/1958023399 shutdown_connections 2024-04-17T07:34:34.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:34.769+0000 7f9d71200640 1 -- 172.21.15.86:0/1958023399 wait complete. 2024-04-17T07:34:34.787 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:34:35.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.015+0000 7f0ebfe00640 1 -- 172.21.15.86:0/4246613348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec01039a0 msgr2=0x7f0ec010a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.015+0000 7f0ebfe00640 1 --2- 172.21.15.86:0/4246613348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec01039a0 0x7f0ec010a2c0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009ca0 tx=0x7f0eb802c100 comp rx=0 tx=0).stop 2024-04-17T07:34:35.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.015+0000 7f0ebfe00640 1 -- 172.21.15.86:0/4246613348 shutdown_connections 2024-04-17T07:34:35.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.015+0000 7f0ebfe00640 1 --2- 172.21.15.86:0/4246613348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec01039a0 0x7f0ec010a2c0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.015+0000 7f0ebfe00640 1 --2- 172.21.15.86:0/4246613348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ec0102fe0 0x7f0ec0103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.015+0000 7f0ebfe00640 1 --2- 172.21.15.86:0/4246613348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec0101e90 0x7f0ec01022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.015+0000 7f0ebfe00640 1 -- 172.21.15.86:0/4246613348 >> 172.21.15.86:0/4246613348 conn(0x7f0ec00fd400 msgr2=0x7f0ec00ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.015+0000 7f0ebfe00640 1 -- 172.21.15.86:0/4246613348 shutdown_connections 2024-04-17T07:34:35.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.015+0000 7f0ebfe00640 1 -- 172.21.15.86:0/4246613348 wait complete. 2024-04-17T07:34:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.016+0000 7f0ebfe00640 1 Processor -- start 2024-04-17T07:34:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.016+0000 7f0ebfe00640 1 -- start start 2024-04-17T07:34:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebfe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec0101e90 0x7f0ec0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ec0102fe0 0x7f0ec0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec01039a0 0x7f0ec019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec010d9d0 con 0x7f0ec01039a0 2024-04-17T07:34:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebfe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec010d850 con 0x7f0ec0101e90 2024-04-17T07:34:35.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebfe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec010db50 con 0x7f0ec0102fe0 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec01039a0 0x7f0ec019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec01039a0 0x7f0ec019f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51404/0 (socket says 172.21.15.86:51404) 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebf400640 1 -- 172.21.15.86:0/1688894520 learned_addr learned my addr 172.21.15.86:0/1688894520 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ec0102fe0 0x7f0ec0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebea00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec0101e90 0x7f0ec0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebf400640 1 -- 172.21.15.86:0/1688894520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ec0102fe0 msgr2=0x7f0ec0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebf400640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ec0102fe0 0x7f0ec0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebf400640 1 -- 172.21.15.86:0/1688894520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec0101e90 msgr2=0x7f0ec0197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebf400640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec0101e90 0x7f0ec0197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebf400640 1 -- 172.21.15.86:0/1688894520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec019fb20 con 0x7f0ec01039a0 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ec0102fe0 0x7f0ec0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.017+0000 7f0ebea00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec0101e90 0x7f0ec0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:35.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.018+0000 7f0ebf400640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec01039a0 0x7f0ec019f4b0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8032380 tx=0x7f0eb8035e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.018+0000 7f0eb7400640 1 -- 172.21.15.86:0/1688894520 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8039040 con 0x7f0ec01039a0 2024-04-17T07:34:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.019+0000 7f0eb7400640 1 -- 172.21.15.86:0/1688894520 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eb8044070 con 0x7f0ec01039a0 2024-04-17T07:34:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.019+0000 7f0eb7400640 1 -- 172.21.15.86:0/1688894520 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8004670 con 0x7f0ec01039a0 2024-04-17T07:34:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.019+0000 7f0ebfe00640 1 -- 172.21.15.86:0/1688894520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec019fdb0 con 0x7f0ec01039a0 2024-04-17T07:34:35.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.019+0000 7f0ebfe00640 1 -- 172.21.15.86:0/1688894520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01a01f0 con 0x7f0ec01039a0 2024-04-17T07:34:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.023+0000 7f0eb7400640 1 -- 172.21.15.86:0/1688894520 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0eb8003970 con 0x7f0ec01039a0 2024-04-17T07:34:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.023+0000 7f0ec87aa740 1 Processor -- start 2024-04-17T07:34:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.023+0000 7f0ec87aa740 1 -- start start 2024-04-17T07:34:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.023+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e8677570 0x5586e8677990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.023+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e867ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e867d0d0 0x5586e867f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586e85d9020 con 0x5586e8677f60 2024-04-17T07:34:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5586e857f460 con 0x5586e8677570 2024-04-17T07:34:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5586e85d4fe0 con 0x5586e867d0d0 2024-04-17T07:34:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e867ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e867ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51410/0 (socket says 172.21.15.86:51410) 2024-04-17T07:34:35.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ebea00640 1 -- 172.21.15.86:0/3205877620 learned_addr learned my addr 172.21.15.86:0/3205877620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/3205877620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e8677570 0x5586e8677990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ebea00640 1 -- 172.21.15.86:0/3205877620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e867d0d0 msgr2=0x5586e867f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ebea00640 1 --2- 172.21.15.86:0/3205877620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e867d0d0 0x5586e867f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ebea00640 1 -- 172.21.15.86:0/3205877620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e8677570 msgr2=0x5586e8677990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ebea00640 1 --2- 172.21.15.86:0/3205877620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e8677570 0x5586e8677990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.024+0000 7f0ebea00640 1 -- 172.21.15.86:0/3205877620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586e8587040 con 0x5586e8677f60 2024-04-17T07:34:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.025+0000 7f0eb7400640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea40a3d00 0x7f0ea40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.025+0000 7f0ebea00640 1 --2- 172.21.15.86:0/3205877620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e867ca60 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000bc20 tx=0x7f0eb000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fdd6794e23927c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.025+0000 7f0eb7400640 1 -- 172.21.15.86:0/1688894520 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0eb80f30d0 con 0x7f0ec01039a0 2024-04-17T07:34:35.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.025+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea40a3d00 0x7f0ea40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.025+0000 7f0eafe00640 1 -- 172.21.15.86:0/3205877620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0017620 con 0x5586e8677f60 2024-04-17T07:34:35.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.025+0000 7f0eafe00640 1 -- 172.21.15.86:0/3205877620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eb0017c70 con 0x5586e8677f60 2024-04-17T07:34:35.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.025+0000 7f0eafe00640 1 -- 172.21.15.86:0/3205877620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000fc20 con 0x5586e8677f60 2024-04-17T07:34:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.026+0000 7f0ec87aa740 1 -- 172.21.15.86:0/3205877620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 msgr2=0x5586e867ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.026+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea40a3d00 0x7f0ea40a61c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x5586e86787c0 tx=0x7f0ea8005e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.026+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/3205877620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e867ca60 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000bc20 tx=0x7f0eb000d710 comp rx=0 tx=0).stop 2024-04-17T07:34:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.027+0000 7f0ec87aa740 1 -- 172.21.15.86:0/3205877620 shutdown_connections 2024-04-17T07:34:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.027+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/3205877620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e867d0d0 0x5586e867f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.027+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/3205877620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e867ca60 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.027+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/3205877620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e8677570 0x5586e8677990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.027+0000 7f0ec87aa740 1 -- 172.21.15.86:0/3205877620 >> 172.21.15.86:0/3205877620 conn(0x5586e866e6d0 msgr2=0x5586e866ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.027+0000 7f0ec87aa740 1 -- 172.21.15.86:0/3205877620 shutdown_connections 2024-04-17T07:34:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.027+0000 7f0ec87aa740 1 -- 172.21.15.86:0/3205877620 wait complete. 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.027+0000 7f0ec87aa740 1 -- start start 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.027+0000 7f0ec87aa740 1 Processor -- start 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8681ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e8677f60 0x5586e8681ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e867d0d0 0x5586e8689030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586e8680080 con 0x5586e8677570 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5586e8592830 con 0x5586e867d0d0 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5586e8680200 con 0x5586e8677f60 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0ec87aa740 10 client.0 _ll_register_callbacks cb 0x5586e8560fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8681ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8681ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51418/0 (socket says 172.21.15.86:51418) 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0ebea00640 1 -- 172.21.15.86:0/1478991414 learned_addr learned my addr 172.21.15.86:0/1478991414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.028+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e8677f60 0x5586e8681ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0ebf400640 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e867d0d0 0x5586e8689030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0ebea00640 1 -- 172.21.15.86:0/1478991414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e8677f60 msgr2=0x5586e8681ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0ebea00640 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e8677f60 0x5586e8681ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0ebea00640 1 -- 172.21.15.86:0/1478991414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e867d0d0 msgr2=0x5586e8689030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0ebea00640 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e867d0d0 0x5586e8689030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0ebea00640 1 -- 172.21.15.86:0/1478991414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586e8596920 con 0x5586e8677570 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0ebea00640 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8681ab0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000b390 tx=0x7f0eb000f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b103d542252e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0eacc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0eacc00640 1 -- 172.21.15.86:0/1478991414 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0026020 con 0x5586e8677570 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0eacc00640 1 -- 172.21.15.86:0/1478991414 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0eb0017630 con 0x5586e8677570 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.029+0000 7f0ec87aa740 1 -- 172.21.15.86:0/1478991414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb0017c70 con 0x5586e8677570 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.030+0000 7f0eacc00640 1 -- 172.21.15.86:0/1478991414 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0eb0031050 con 0x5586e8677570 2024-04-17T07:34:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.030+0000 7f0ec87aa740 10 client.19944 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:35.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.030+0000 7f0ec87aa740 1 -- 172.21.15.86:0/1478991414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586e85647f0 con 0x5586e8677570 2024-04-17T07:34:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.030+0000 7f0eacc00640 1 -- 172.21.15.86:0/1478991414 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eb0017c70 con 0x5586e8677570 2024-04-17T07:34:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.030+0000 7f0eacc00640 1 -- 172.21.15.86:0/1478991414 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0eb0021a20 con 0x5586e8677570 2024-04-17T07:34:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.030+0000 7f0ec87aa740 10 client.19944 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.030+0000 7f0ec87aa740 10 client.19944 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:34:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.030+0000 7f0ec87aa740 20 client.19944 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.031+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5586e868d2a0 0x5586e868f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.031+0000 7f0ec87aa740 4 client.19944 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.031+0000 7f0ec87aa740 1 -- 172.21.15.86:0/1478991414 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586e85e6a40 con 0x5586e868d2a0 2024-04-17T07:34:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.031+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5586e868d2a0 0x5586e868f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.032+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5586e868d2a0 0x5586e868f690 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f0f518da12948650 server_cookie=52cf8fe726a42950 in_seq=0 out_seq=0 2024-04-17T07:34:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.032+0000 7f0eacc00640 10 client.19944 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:34:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0eacc00640 1 -- 172.21.15.86:0/1478991414 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586e85e6a40 con 0x5586e868d2a0 2024-04-17T07:34:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0eacc00640 10 client.19944 handle_command_reply: tid=0 2024-04-17T07:34:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0ec87aa740 1 client.19944 shutdown 2024-04-17T07:34:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0ec87aa740 1 -- 172.21.15.86:0/1478991414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 msgr2=0x5586e8681ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8681ab0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000b390 tx=0x7f0eb000f650 comp rx=0 tx=0).stop 2024-04-17T07:34:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0ec87aa740 1 -- 172.21.15.86:0/1478991414 shutdown_connections 2024-04-17T07:34:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5586e868d2a0 0x5586e868f690 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e867d0d0 0x5586e8689030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e8677f60 0x5586e8681ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/1478991414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8681ab0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0ec87aa740 1 -- 172.21.15.86:0/1478991414 >> 172.21.15.86:0/1478991414 conn(0x5586e866e6d0 msgr2=0x5586e866d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.034+0000 7f0eacc00640 0 client.19944 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:34:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.035+0000 7f0ec87aa740 1 -- 172.21.15.86:0/1478991414 shutdown_connections 2024-04-17T07:34:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.035+0000 7f0ec87aa740 1 -- 172.21.15.86:0/1478991414 wait complete. 2024-04-17T07:34:35.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.035+0000 7f0ec87aa740 20 client.19944 trim_cache size 0 max 16384 2024-04-17T07:34:35.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.062+0000 7f0ec87aa740 1 Processor -- start 2024-04-17T07:34:35.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.062+0000 7f0ec87aa740 1 -- start start 2024-04-17T07:34:35.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.062+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e8677570 0x5586e866d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.062+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e8688ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.062+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e867d0d0 0x5586e866d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.062+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01a85c0 con 0x5586e8677f60 2024-04-17T07:34:35.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.062+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01aa100 con 0x5586e8677570 2024-04-17T07:34:35.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.062+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01a8c90 con 0x5586e867d0d0 2024-04-17T07:34:35.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.063+0000 7f0eb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e8688ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.063+0000 7f0eb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e8688ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51422/0 (socket says 172.21.15.86:51422) 2024-04-17T07:34:35.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.063+0000 7f0eb7e00640 1 -- 172.21.15.86:0/3457277995 learned_addr learned my addr 172.21.15.86:0/3457277995 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.063+0000 7f0ebf400640 1 --2- 172.21.15.86:0/3457277995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e867d0d0 0x5586e866d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.063+0000 7f0ebea00640 1 --2- 172.21.15.86:0/3457277995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e8677570 0x5586e866d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.063+0000 7f0eb7e00640 1 -- 172.21.15.86:0/3457277995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e867d0d0 msgr2=0x5586e866d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.063+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/3457277995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e867d0d0 0x5586e866d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.063+0000 7f0eb7e00640 1 -- 172.21.15.86:0/3457277995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e8677570 msgr2=0x5586e866d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.063+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/3457277995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e8677570 0x5586e866d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.063+0000 7f0eb7e00640 1 -- 172.21.15.86:0/3457277995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586e82eb3f0 con 0x5586e8677f60 2024-04-17T07:34:35.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.064+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/3457277995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e8688ff0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0199410 tx=0x7f0ea8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a9465cc3f3b8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.064+0000 7f0eaf400640 1 -- 172.21.15.86:0/3457277995 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea802b070 con 0x5586e8677f60 2024-04-17T07:34:35.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.064+0000 7f0eaf400640 1 -- 172.21.15.86:0/3457277995 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ea801f910 con 0x5586e8677f60 2024-04-17T07:34:35.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.064+0000 7f0eaf400640 1 -- 172.21.15.86:0/3457277995 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80255d0 con 0x5586e8677f60 2024-04-17T07:34:35.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.065+0000 7f0ec87aa740 1 -- 172.21.15.86:0/3457277995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 msgr2=0x5586e8688ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.065+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/3457277995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e8688ff0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0199410 tx=0x7f0ea8014000 comp rx=0 tx=0).stop 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.065+0000 7f0ec87aa740 1 -- 172.21.15.86:0/3457277995 shutdown_connections 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.065+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/3457277995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e867d0d0 0x5586e866d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.065+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/3457277995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677f60 0x5586e8688ff0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.065+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/3457277995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e8677570 0x5586e866d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.065+0000 7f0ec87aa740 1 -- 172.21.15.86:0/3457277995 >> 172.21.15.86:0/3457277995 conn(0x5586e866e6d0 msgr2=0x5586e86794a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.065+0000 7f0ec87aa740 1 -- 172.21.15.86:0/3457277995 shutdown_connections 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.065+0000 7f0ec87aa740 1 -- 172.21.15.86:0/3457277995 wait complete. 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.066+0000 7f0ec87aa740 1 -- start start 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.066+0000 7f0ec87aa740 1 Processor -- start 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.066+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8685bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.066+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e8677f60 0x5586e86860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.066+0000 7f0ec87aa740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e867d0d0 0x5586e86f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.066+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01a9a30 con 0x5586e8677570 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.066+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01a9360 con 0x5586e867d0d0 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.066+0000 7f0ec87aa740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01a6590 con 0x5586e8677f60 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.066+0000 7f0ec87aa740 10 client.0 _ll_register_callbacks cb 0x7f0eb000fde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8685bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8685bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51426/0 (socket says 172.21.15.86:51426) 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0ebea00640 1 -- 172.21.15.86:0/2282683881 learned_addr learned my addr 172.21.15.86:0/2282683881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e8677f60 0x5586e86860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0ebf400640 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e867d0d0 0x5586e86f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0ebea00640 1 -- 172.21.15.86:0/2282683881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e8677f60 msgr2=0x5586e86860f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0ebea00640 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e8677f60 0x5586e86860f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0ebea00640 1 -- 172.21.15.86:0/2282683881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e867d0d0 msgr2=0x5586e86f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0ebea00640 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e867d0d0 0x5586e86f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0ebea00640 1 -- 172.21.15.86:0/2282683881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586e83b57d0 con 0x5586e8677570 2024-04-17T07:34:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.067+0000 7f0ebea00640 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8685bb0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0022040 tx=0x7f0eb000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52d3f0c665a1416 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.068+0000 7f0eacc00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.068+0000 7f0eacc00640 1 -- 172.21.15.86:0/2282683881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0026070 con 0x5586e8677570 2024-04-17T07:34:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.068+0000 7f0eacc00640 1 -- 172.21.15.86:0/2282683881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0eb0022840 con 0x5586e8677570 2024-04-17T07:34:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.068+0000 7f0ec87aa740 1 -- 172.21.15.86:0/2282683881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea801f910 con 0x5586e8677570 2024-04-17T07:34:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.068+0000 7f0eacc00640 1 -- 172.21.15.86:0/2282683881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0eb0022600 con 0x5586e8677570 2024-04-17T07:34:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.068+0000 7f0ec87aa740 10 client.19950 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:35.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.068+0000 7f0ec87aa740 1 -- 172.21.15.86:0/2282683881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586e86f2a10 con 0x5586e8677570 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.068+0000 7f0eacc00640 1 -- 172.21.15.86:0/2282683881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ea801f910 con 0x5586e8677570 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.069+0000 7f0eacc00640 1 -- 172.21.15.86:0/2282683881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f0eb0021450 con 0x5586e8677570 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.069+0000 7f0ec87aa740 10 client.19950 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.069+0000 7f0ec87aa740 10 client.19950 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.069+0000 7f0ec87aa740 20 client.19950 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.069+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5586e86f3350 0x5586e86f5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.069+0000 7f0ec87aa740 4 client.19950 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.069+0000 7f0ec87aa740 1 -- 172.21.15.86:0/2282683881 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586e85e08b0 con 0x5586e86f3350 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.069+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5586e86f3350 0x5586e86f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.070+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5586e86f3350 0x5586e86f5770 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e48b81f3f3d72919 server_cookie=a4c308fdedfd499e in_seq=0 out_seq=0 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.070+0000 7f0eacc00640 10 client.19950 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.071+0000 7f0eacc00640 1 -- 172.21.15.86:0/2282683881 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586e85e08b0 con 0x5586e86f3350 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.071+0000 7f0eacc00640 10 client.19950 handle_command_reply: tid=0 2024-04-17T07:34:35.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.071+0000 7f0ec87aa740 1 client.19950 shutdown 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.071+0000 7f0ec87aa740 1 -- 172.21.15.86:0/2282683881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 msgr2=0x5586e8685bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.071+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8685bb0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0022040 tx=0x7f0eb000dc10 comp rx=0 tx=0).stop 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.071+0000 7f0ec87aa740 1 -- 172.21.15.86:0/2282683881 shutdown_connections 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.072+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5586e86f3350 0x5586e86f5770 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.072+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5586e867d0d0 0x5586e86f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.072+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5586e8677f60 0x5586e86860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.072+0000 7f0eacc00640 0 client.19950 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.072+0000 7f0ec87aa740 1 --2- 172.21.15.86:0/2282683881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586e8677570 0x5586e8685bb0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.072+0000 7f0ec87aa740 1 -- 172.21.15.86:0/2282683881 >> 172.21.15.86:0/2282683881 conn(0x5586e866e6d0 msgr2=0x5586e866d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.072+0000 7f0ec87aa740 1 -- 172.21.15.86:0/2282683881 shutdown_connections 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.072+0000 7f0ec87aa740 1 -- 172.21.15.86:0/2282683881 wait complete. 2024-04-17T07:34:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.072+0000 7f0ec87aa740 20 client.19950 trim_cache size 0 max 16384 2024-04-17T07:34:35.074 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:34:35.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 -- 172.21.15.86:0/1688894520 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea40a3d00 msgr2=0x7f0ea40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea40a3d00 0x7f0ea40a61c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x5586e86787c0 tx=0x7f0ea8005e10 comp rx=0 tx=0).stop 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 -- 172.21.15.86:0/1688894520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec01039a0 msgr2=0x7f0ec019f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec01039a0 0x7f0ec019f4b0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8032380 tx=0x7f0eb8035e20 comp rx=0 tx=0).stop 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 -- 172.21.15.86:0/1688894520 shutdown_connections 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f0ea40a3d00 0x7f0ea40a61c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ec01039a0 0x7f0ec019f4b0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ec0102fe0 0x7f0ec0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 --2- 172.21.15.86:0/1688894520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec0101e90 0x7f0ec0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 -- 172.21.15.86:0/1688894520 >> 172.21.15.86:0/1688894520 conn(0x7f0ec00fd400 msgr2=0x7f0ec0106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 -- 172.21.15.86:0/1688894520 shutdown_connections 2024-04-17T07:34:35.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.073+0000 7f0ebfe00640 1 -- 172.21.15.86:0/1688894520 wait complete. 2024-04-17T07:34:35.090 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:34:35.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.285+0000 7f0837e00640 1 -- 172.21.15.86:0/1609530789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0838102400 msgr2=0x7f0838102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.285+0000 7f0837e00640 1 --2- 172.21.15.86:0/1609530789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0838102400 0x7f0838102820 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f0828009900 tx=0x7f082802bc80 comp rx=0 tx=0).stop 2024-04-17T07:34:35.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.286+0000 7f0837e00640 1 -- 172.21.15.86:0/1609530789 shutdown_connections 2024-04-17T07:34:35.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.286+0000 7f0837e00640 1 --2- 172.21.15.86:0/1609530789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0838101980 0x7f0838109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.286+0000 7f0837e00640 1 --2- 172.21.15.86:0/1609530789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0838100fc0 0x7f0838101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.286+0000 7f0837e00640 1 --2- 172.21.15.86:0/1609530789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0838102400 0x7f0838102820 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.286+0000 7f0837e00640 1 -- 172.21.15.86:0/1609530789 >> 172.21.15.86:0/1609530789 conn(0x7f08380fba90 msgr2=0x7f08380fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.286+0000 7f0837e00640 1 -- 172.21.15.86:0/1609530789 shutdown_connections 2024-04-17T07:34:35.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.286+0000 7f0837e00640 1 -- 172.21.15.86:0/1609530789 wait complete. 2024-04-17T07:34:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0837e00640 1 Processor -- start 2024-04-17T07:34:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0837e00640 1 -- start start 2024-04-17T07:34:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0837e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0838100fc0 0x7f083819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0837e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0838101980 0x7f083819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0837e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0838102400 0x7f08381a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0837e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0838100ac0 con 0x7f0838100fc0 2024-04-17T07:34:35.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0837e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0838100940 con 0x7f0838101980 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0837e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0838100c40 con 0x7f0838102400 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0836a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0838100fc0 0x7f083819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0836a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0838100fc0 0x7f083819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51432/0 (socket says 172.21.15.86:51432) 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.287+0000 7f0836a00640 1 -- 172.21.15.86:0/2697214514 learned_addr learned my addr 172.21.15.86:0/2697214514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f0837400640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0838102400 0x7f08381a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f0836a00640 1 -- 172.21.15.86:0/2697214514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0838102400 msgr2=0x7f08381a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f0836a00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0838102400 0x7f08381a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f0836000640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0838101980 0x7f083819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f0836a00640 1 -- 172.21.15.86:0/2697214514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0838101980 msgr2=0x7f083819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f0836a00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0838101980 0x7f083819a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f0836a00640 1 -- 172.21.15.86:0/2697214514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08381a1eb0 con 0x7f0838100fc0 2024-04-17T07:34:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f0837400640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0838102400 0x7f08381a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f0836a00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0838100fc0 0x7f083819a290 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c00 tx=0x7f08280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f082f400640 1 -- 172.21.15.86:0/2697214514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828043020 con 0x7f0838100fc0 2024-04-17T07:34:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.288+0000 7f082f400640 1 -- 172.21.15.86:0/2697214514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08280054c0 con 0x7f0838100fc0 2024-04-17T07:34:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.289+0000 7f0837e00640 1 -- 172.21.15.86:0/2697214514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08381a2140 con 0x7f0838100fc0 2024-04-17T07:34:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.289+0000 7f082f400640 1 -- 172.21.15.86:0/2697214514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828003d70 con 0x7f0838100fc0 2024-04-17T07:34:35.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.289+0000 7f0837e00640 1 -- 172.21.15.86:0/2697214514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08381a2690 con 0x7f0838100fc0 2024-04-17T07:34:35.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.292+0000 7f08403f0740 1 Processor -- start 2024-04-17T07:34:35.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.292+0000 7f08403f0740 1 -- start start 2024-04-17T07:34:35.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.292+0000 7f082f400640 1 -- 172.21.15.86:0/2697214514 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0828046020 con 0x7f0838100fc0 2024-04-17T07:34:35.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f08403f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146160 0x563009146580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f08403f0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 0x56300914b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f08403f0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56300914bcc0 0x56300914e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f08403f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630090ac020 con 0x563009146160 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f08403f0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563009052460 con 0x563009146b50 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f08403f0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5630090a7fe0 con 0x56300914bcc0 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0837400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146160 0x563009146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0837400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146160 0x563009146580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51448/0 (socket says 172.21.15.86:51448) 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0837400640 1 -- 172.21.15.86:0/1947070676 learned_addr learned my addr 172.21.15.86:0/1947070676 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0836000640 1 --2- 172.21.15.86:0/1947070676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56300914bcc0 0x56300914e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0836a00640 1 --2- 172.21.15.86:0/1947070676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 0x56300914b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0837400640 1 -- 172.21.15.86:0/1947070676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56300914bcc0 msgr2=0x56300914e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0837400640 1 --2- 172.21.15.86:0/1947070676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56300914bcc0 0x56300914e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0837400640 1 -- 172.21.15.86:0/1947070676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 msgr2=0x56300914b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0837400640 1 --2- 172.21.15.86:0/1947070676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 0x56300914b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0837400640 1 -- 172.21.15.86:0/1947070676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56300905a040 con 0x563009146160 2024-04-17T07:34:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.293+0000 7f0836a00640 1 --2- 172.21.15.86:0/1947070676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 0x56300914b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.294+0000 7f0837400640 1 --2- 172.21.15.86:0/1947070676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146160 0x563009146580 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f0830008e60 tx=0x7f083000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce4573a92a57be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.294+0000 7f0823400640 1 -- 172.21.15.86:0/1947070676 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083000f830 con 0x563009146160 2024-04-17T07:34:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.294+0000 7f0823400640 1 -- 172.21.15.86:0/1947070676 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0830018480 con 0x563009146160 2024-04-17T07:34:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.294+0000 7f0823400640 1 -- 172.21.15.86:0/1947070676 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083000e5d0 con 0x563009146160 2024-04-17T07:34:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.294+0000 7f082f400640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f081c0a3bb0 0x7f081c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f082f400640 1 -- 172.21.15.86:0/2697214514 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f08280f2be0 con 0x7f0838100fc0 2024-04-17T07:34:35.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f0836a00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f081c0a3bb0 0x7f081c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f08403f0740 1 -- 172.21.15.86:0/1947070676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146160 msgr2=0x563009146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f08403f0740 1 --2- 172.21.15.86:0/1947070676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146160 0x563009146580 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f0830008e60 tx=0x7f083000d710 comp rx=0 tx=0).stop 2024-04-17T07:34:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f08403f0740 1 -- 172.21.15.86:0/1947070676 shutdown_connections 2024-04-17T07:34:35.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f08403f0740 1 --2- 172.21.15.86:0/1947070676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56300914bcc0 0x56300914e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f08403f0740 1 --2- 172.21.15.86:0/1947070676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 0x56300914b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f08403f0740 1 --2- 172.21.15.86:0/1947070676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146160 0x563009146580 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f08403f0740 1 -- 172.21.15.86:0/1947070676 >> 172.21.15.86:0/1947070676 conn(0x5630091416d0 msgr2=0x563009141ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f0836a00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f081c0a3bb0 0x7f081c0a6070 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f08280bea70 tx=0x7f08280df040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f08403f0740 1 -- 172.21.15.86:0/1947070676 shutdown_connections 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.295+0000 7f08403f0740 1 -- 172.21.15.86:0/1947070676 wait complete. 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.296+0000 7f08403f0740 1 -- start start 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.296+0000 7f08403f0740 1 Processor -- start 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.296+0000 7f08403f0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146160 0x563009150680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.296+0000 7f08403f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.296+0000 7f08403f0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56300914bcc0 0x563009157c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.296+0000 7f08403f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56300914ec70 con 0x563009146b50 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.296+0000 7f08403f0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563009065830 con 0x563009146160 2024-04-17T07:34:35.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.296+0000 7f08403f0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56300914edf0 con 0x56300914bcc0 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.296+0000 7f08403f0740 10 client.0 _ll_register_callbacks cb 0x563009033fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0836000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0836000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51460/0 (socket says 172.21.15.86:51460) 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0836000640 1 -- 172.21.15.86:0/303657016 learned_addr learned my addr 172.21.15.86:0/303657016 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0837400640 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146160 0x563009150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0837400640 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56300914bcc0 0x563009157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0836000640 1 -- 172.21.15.86:0/303657016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56300914bcc0 msgr2=0x563009157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0836000640 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56300914bcc0 0x563009157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0836000640 1 -- 172.21.15.86:0/303657016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146160 msgr2=0x563009150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0836000640 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146160 0x563009150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0836000640 1 -- 172.21.15.86:0/303657016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563009069920 con 0x563009146b50 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.297+0000 7f0836000640 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009150bc0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f0824010420 tx=0x7f08240108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3550d6e5475648d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.298+0000 7f0817e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:35.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.298+0000 7f0817e00640 1 -- 172.21.15.86:0/303657016 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824016070 con 0x563009146b50 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.298+0000 7f0817e00640 1 -- 172.21.15.86:0/303657016 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0824012920 con 0x563009146b50 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.298+0000 7f08403f0740 1 -- 172.21.15.86:0/303657016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0830018480 con 0x563009146b50 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.298+0000 7f0817e00640 1 -- 172.21.15.86:0/303657016 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0824024050 con 0x563009146b50 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.298+0000 7f08403f0740 10 client.19962 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.298+0000 7f08403f0740 1 -- 172.21.15.86:0/303657016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630090377f0 con 0x563009146b50 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.298+0000 7f0817e00640 1 -- 172.21.15.86:0/303657016 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0830018480 con 0x563009146b50 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.298+0000 7f0817e00640 1 -- 172.21.15.86:0/303657016 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f082400f040 con 0x563009146b50 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.299+0000 7f08403f0740 10 client.19962 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.299+0000 7f08403f0740 10 client.19962 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.299+0000 7f08403f0740 20 client.19962 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.299+0000 7f08403f0740 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56300915c120 0x56300915e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.299+0000 7f08403f0740 4 client.19962 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.299+0000 7f08403f0740 1 -- 172.21.15.86:0/303657016 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630090b9a40 con 0x56300915c120 2024-04-17T07:34:35.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.299+0000 7f0837400640 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56300915c120 0x56300915e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.301+0000 7f0837400640 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56300915c120 0x56300915e5a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e83affb3520ff57 server_cookie=95a2ed2820766a98 in_seq=0 out_seq=0 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.301+0000 7f0817e00640 10 client.19962 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f0817e00640 1 -- 172.21.15.86:0/303657016 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630090b9a40 con 0x56300915c120 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f0817e00640 10 client.19962 handle_command_reply: tid=0 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 client.19962 shutdown 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 -- 172.21.15.86:0/303657016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 msgr2=0x563009150bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009150bc0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f0824010420 tx=0x7f08240108e0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 -- 172.21.15.86:0/303657016 shutdown_connections 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x56300915c120 0x56300915e5a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56300914bcc0 0x563009157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009150bc0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 --2- 172.21.15.86:0/303657016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146160 0x563009150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f0817e00640 0 client.19962 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:34:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 -- 172.21.15.86:0/303657016 >> 172.21.15.86:0/303657016 conn(0x5630091416d0 msgr2=0x563009140930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 -- 172.21.15.86:0/303657016 shutdown_connections 2024-04-17T07:34:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.303+0000 7f08403f0740 1 -- 172.21.15.86:0/303657016 wait complete. 2024-04-17T07:34:35.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.304+0000 7f08403f0740 20 client.19962 trim_cache size 0 max 16384 2024-04-17T07:34:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.318+0000 7f08403f0740 1 Processor -- start 2024-04-17T07:34:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.319+0000 7f08403f0740 1 -- start start 2024-04-17T07:34:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.319+0000 7f08403f0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563009146160 0x5630091400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.319+0000 7f08403f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.319+0000 7f08403f0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56300914bcc0 0x5630091405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.319+0000 7f08403f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08381aca50 con 0x563009146b50 2024-04-17T07:34:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.319+0000 7f08403f0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f08381ae590 con 0x56300914bcc0 2024-04-17T07:34:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.319+0000 7f08403f0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08381ad120 con 0x563009146160 2024-04-17T07:34:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.319+0000 7f0836000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.319+0000 7f0836000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51472/0 (socket says 172.21.15.86:51472) 2024-04-17T07:34:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.319+0000 7f0836000640 1 -- 172.21.15.86:0/1408442274 learned_addr learned my addr 172.21.15.86:0/1408442274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0837400640 1 --2- 172.21.15.86:0/1408442274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563009146160 0x5630091400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0836000640 1 -- 172.21.15.86:0/1408442274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563009146160 msgr2=0x5630091400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0836000640 1 --2- 172.21.15.86:0/1408442274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563009146160 0x5630091400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0836000640 1 -- 172.21.15.86:0/1408442274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56300914bcc0 msgr2=0x5630091405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0836000640 1 --2- 172.21.15.86:0/1408442274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56300914bcc0 0x5630091405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0836000640 1 -- 172.21.15.86:0/1408442274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563008dbe3f0 con 0x563009146b50 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0837400640 1 --2- 172.21.15.86:0/1408442274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563009146160 0x5630091400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0836000640 1 --2- 172.21.15.86:0/1408442274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009157bc0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f082401be90 tx=0x7f0824009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493bb6ff39e8b46d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0822a00640 1 -- 172.21.15.86:0/1408442274 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082402c020 con 0x563009146b50 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0822a00640 1 -- 172.21.15.86:0/1408442274 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08240054e0 con 0x563009146b50 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.320+0000 7f0822a00640 1 -- 172.21.15.86:0/1408442274 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824016070 con 0x563009146b50 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.321+0000 7f08403f0740 1 -- 172.21.15.86:0/1408442274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 msgr2=0x563009157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.321+0000 7f08403f0740 1 --2- 172.21.15.86:0/1408442274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009157bc0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f082401be90 tx=0x7f0824009580 comp rx=0 tx=0).stop 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.321+0000 7f08403f0740 1 -- 172.21.15.86:0/1408442274 shutdown_connections 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.321+0000 7f08403f0740 1 --2- 172.21.15.86:0/1408442274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56300914bcc0 0x5630091405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.321+0000 7f08403f0740 1 --2- 172.21.15.86:0/1408442274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563009146b50 0x563009157bc0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.321+0000 7f08403f0740 1 --2- 172.21.15.86:0/1408442274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563009146160 0x5630091400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.321+0000 7f08403f0740 1 -- 172.21.15.86:0/1408442274 >> 172.21.15.86:0/1408442274 conn(0x5630091416d0 msgr2=0x563009148010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.321+0000 7f08403f0740 1 -- 172.21.15.86:0/1408442274 shutdown_connections 2024-04-17T07:34:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.321+0000 7f08403f0740 1 -- 172.21.15.86:0/1408442274 wait complete. 2024-04-17T07:34:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.321+0000 7f08403f0740 1 -- start start 2024-04-17T07:34:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f08403f0740 1 Processor -- start 2024-04-17T07:34:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f08403f0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563009146160 0x563009155020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f08403f0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 0x563009155560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f08403f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56300914bcc0 0x563009156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f08403f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08381adec0 con 0x56300914bcc0 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f08403f0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f08381ad7f0 con 0x563009146b50 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f08403f0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08381aaa20 con 0x563009146160 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f08403f0740 10 client.0 _ll_register_callbacks cb 0x7f0824009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f0837400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56300914bcc0 0x563009156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f0837400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56300914bcc0 0x563009156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51482/0 (socket says 172.21.15.86:51482) 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.322+0000 7f0837400640 1 -- 172.21.15.86:0/4138666406 learned_addr learned my addr 172.21.15.86:0/4138666406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0836000640 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 0x563009155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0837400640 1 -- 172.21.15.86:0/4138666406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563009146160 msgr2=0x563009155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0837400640 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563009146160 0x563009155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0837400640 1 -- 172.21.15.86:0/4138666406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 msgr2=0x563009155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0837400640 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 0x563009155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0837400640 1 -- 172.21.15.86:0/4138666406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563008e887d0 con 0x56300914bcc0 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0836000640 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 0x563009155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0837400640 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56300914bcc0 0x563009156570 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0830008e60 tx=0x7f083000dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e1775d9142bb4b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0817e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0817e00640 1 -- 172.21.15.86:0/4138666406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830004c00 con 0x56300914bcc0 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f0817e00640 1 -- 172.21.15.86:0/4138666406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f0830004da0 con 0x56300914bcc0 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.323+0000 7f08403f0740 1 -- 172.21.15.86:0/4138666406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08240054e0 con 0x56300914bcc0 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f0817e00640 1 -- 172.21.15.86:0/4138666406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0830034050 con 0x56300914bcc0 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f08403f0740 10 client.19968 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f08403f0740 1 -- 172.21.15.86:0/4138666406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563009157c30 con 0x56300914bcc0 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f0817e00640 1 -- 172.21.15.86:0/4138666406 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08240054e0 con 0x56300914bcc0 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f0817e00640 1 -- 172.21.15.86:0/4138666406 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f083000f630 con 0x56300914bcc0 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f08403f0740 10 client.19968 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f08403f0740 10 client.19968 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f08403f0740 20 client.19968 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f08403f0740 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630091c2b20 0x5630091c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f08403f0740 4 client.19968 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:34:35.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.324+0000 7f08403f0740 1 -- 172.21.15.86:0/4138666406 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630090b38b0 con 0x5630091c2b20 2024-04-17T07:34:35.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.325+0000 7f0836000640 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630091c2b20 0x5630091c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.326+0000 7f0836000640 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630091c2b20 0x5630091c4f40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b9fab5b30d8bf88 server_cookie=599daebe18011f71 in_seq=0 out_seq=0 2024-04-17T07:34:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.326+0000 7f0817e00640 10 client.19968 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:34:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f0817e00640 1 -- 172.21.15.86:0/4138666406 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630090b38b0 con 0x5630091c2b20 2024-04-17T07:34:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f0817e00640 10 client.19968 handle_command_reply: tid=0 2024-04-17T07:34:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 client.19968 shutdown 2024-04-17T07:34:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 -- 172.21.15.86:0/4138666406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56300914bcc0 msgr2=0x563009156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56300914bcc0 0x563009156570 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0830008e60 tx=0x7f083000dfc0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 -- 172.21.15.86:0/4138666406 shutdown_connections 2024-04-17T07:34:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5630091c2b20 0x5630091c4f40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56300914bcc0 0x563009156570 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563009146b50 0x563009155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f0817e00640 0 client.19968 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:34:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 --2- 172.21.15.86:0/4138666406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563009146160 0x563009155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 -- 172.21.15.86:0/4138666406 >> 172.21.15.86:0/4138666406 conn(0x5630091416d0 msgr2=0x563009144930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 -- 172.21.15.86:0/4138666406 shutdown_connections 2024-04-17T07:34:35.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 1 -- 172.21.15.86:0/4138666406 wait complete. 2024-04-17T07:34:35.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.327+0000 7f08403f0740 20 client.19968 trim_cache size 0 max 16384 2024-04-17T07:34:35.330 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 -- 172.21.15.86:0/2697214514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f081c0a3bb0 msgr2=0x7f081c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f081c0a3bb0 0x7f081c0a6070 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f08280bea70 tx=0x7f08280df040 comp rx=0 tx=0).stop 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 -- 172.21.15.86:0/2697214514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0838100fc0 msgr2=0x7f083819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0838100fc0 0x7f083819a290 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f0828000c00 tx=0x7f08280042f0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 -- 172.21.15.86:0/2697214514 shutdown_connections 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f081c0a3bb0 0x7f081c0a6070 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0838102400 0x7f08381a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0838101980 0x7f083819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 --2- 172.21.15.86:0/2697214514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0838100fc0 0x7f083819a290 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 -- 172.21.15.86:0/2697214514 >> 172.21.15.86:0/2697214514 conn(0x7f08380fba90 msgr2=0x7f083810bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 -- 172.21.15.86:0/2697214514 shutdown_connections 2024-04-17T07:34:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.328+0000 7f0837e00640 1 -- 172.21.15.86:0/2697214514 wait complete. 2024-04-17T07:34:35.342 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:35 smithi086 ceph-mon[31733]: pgmap v1046: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 899 op/s 2024-04-17T07:34:35.343 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:35 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4240506216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:34:35.346 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:34:35.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:35 smithi103 ceph-mon[35495]: pgmap v1046: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 899 op/s 2024-04-17T07:34:35.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:35 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4240506216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:34:35.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.563+0000 7f6af6e00640 1 -- 172.21.15.86:0/499613135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af0109690 msgr2=0x7f6af0109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.563+0000 7f6af6e00640 1 --2- 172.21.15.86:0/499613135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af0109690 0x7f6af0109b10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009900 tx=0x7f6adc02bc40 comp rx=0 tx=0).stop 2024-04-17T07:34:35.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.563+0000 7f6af6e00640 1 -- 172.21.15.86:0/499613135 shutdown_connections 2024-04-17T07:34:35.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.563+0000 7f6af6e00640 1 --2- 172.21.15.86:0/499613135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af010a050 0x7f6af01108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.563+0000 7f6af6e00640 1 --2- 172.21.15.86:0/499613135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af0109690 0x7f6af0109b10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.563+0000 7f6af6e00640 1 --2- 172.21.15.86:0/499613135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af01084a0 0x7f6af01088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.563+0000 7f6af6e00640 1 -- 172.21.15.86:0/499613135 >> 172.21.15.86:0/499613135 conn(0x7f6af0077430 msgr2=0x7f6af0079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.563+0000 7f6af6e00640 1 -- 172.21.15.86:0/499613135 shutdown_connections 2024-04-17T07:34:35.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.563+0000 7f6af6e00640 1 -- 172.21.15.86:0/499613135 wait complete. 2024-04-17T07:34:35.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.564+0000 7f6af6e00640 1 Processor -- start 2024-04-17T07:34:35.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.565+0000 7f6af6e00640 1 -- start start 2024-04-17T07:34:35.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.565+0000 7f6af6e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af01084a0 0x7f6af019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.565+0000 7f6af6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0109690 0x7f6af019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.565+0000 7f6af6e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af010a050 0x7f6af01a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.565+0000 7f6af6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0113fe0 con 0x7f6af0109690 2024-04-17T07:34:35.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.565+0000 7f6af6e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0113e60 con 0x7f6af010a050 2024-04-17T07:34:35.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.565+0000 7f6af6e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0114160 con 0x7f6af01084a0 2024-04-17T07:34:35.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.565+0000 7f6af5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0109690 0x7f6af019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.565+0000 7f6af5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0109690 0x7f6af019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51500/0 (socket says 172.21.15.86:51500) 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.565+0000 7f6af5000640 1 -- 172.21.15.86:0/94699173 learned_addr learned my addr 172.21.15.86:0/94699173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.566+0000 7f6af5a00640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af01084a0 0x7f6af019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.566+0000 7f6af6400640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af010a050 0x7f6af01a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.566+0000 7f6af5000640 1 -- 172.21.15.86:0/94699173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af01084a0 msgr2=0x7f6af019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.566+0000 7f6af5000640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af01084a0 0x7f6af019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.566+0000 7f6af5000640 1 -- 172.21.15.86:0/94699173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af010a050 msgr2=0x7f6af01a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.566+0000 7f6af5000640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af010a050 0x7f6af01a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.566+0000 7f6af5000640 1 -- 172.21.15.86:0/94699173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af01a62b0 con 0x7f6af0109690 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.566+0000 7f6af5000640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0109690 0x7f6af019ebd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f6adc006010 tx=0x7f6adc005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.567+0000 7f6aee000640 1 -- 172.21.15.86:0/94699173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc043020 con 0x7f6af0109690 2024-04-17T07:34:35.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.567+0000 7f6aee000640 1 -- 172.21.15.86:0/94699173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6adc0043d0 con 0x7f6af0109690 2024-04-17T07:34:35.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.567+0000 7f6af6e00640 1 -- 172.21.15.86:0/94699173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af01a6540 con 0x7f6af0109690 2024-04-17T07:34:35.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.567+0000 7f6aee000640 1 -- 172.21.15.86:0/94699173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc003b80 con 0x7f6af0109690 2024-04-17T07:34:35.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.567+0000 7f6af6e00640 1 -- 172.21.15.86:0/94699173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af01a6a90 con 0x7f6af0109690 2024-04-17T07:34:35.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.570+0000 7f6afb1d3740 1 Processor -- start 2024-04-17T07:34:35.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.570+0000 7f6afb1d3740 1 -- start start 2024-04-17T07:34:35.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.570+0000 7f6aee000640 1 -- 172.21.15.86:0/94699173 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f6adc046020 con 0x7f6af0109690 2024-04-17T07:34:35.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.570+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f3d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3db50 0x564f45f42650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f42cc0 0x564f45f45150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f45ea3020 con 0x564f45f3d160 2024-04-17T07:34:35.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564f45e49460 con 0x564f45f3db50 2024-04-17T07:34:35.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f45e9efe0 con 0x564f45f42cc0 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6af6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f3d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6af6400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f3d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51508/0 (socket says 172.21.15.86:51508) 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6af6400640 1 -- 172.21.15.86:0/1565981367 learned_addr learned my addr 172.21.15.86:0/1565981367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6af5a00640 1 --2- 172.21.15.86:0/1565981367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3db50 0x564f45f42650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6af5000640 1 --2- 172.21.15.86:0/1565981367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f42cc0 0x564f45f45150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6af6400640 1 -- 172.21.15.86:0/1565981367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f42cc0 msgr2=0x564f45f45150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6af6400640 1 --2- 172.21.15.86:0/1565981367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f42cc0 0x564f45f45150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6af6400640 1 -- 172.21.15.86:0/1565981367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3db50 msgr2=0x564f45f42650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.571+0000 7f6af6400640 1 --2- 172.21.15.86:0/1565981367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3db50 0x564f45f42650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.572+0000 7f6af6400640 1 -- 172.21.15.86:0/1565981367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f45e51040 con 0x564f45f3d160 2024-04-17T07:34:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.572+0000 7f6af6400640 1 --2- 172.21.15.86:0/1565981367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f3d580 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009fc0 tx=0x7f6ae8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a402a14cc14f397 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.572+0000 7f6ae2a00640 1 -- 172.21.15.86:0/1565981367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8003b70 con 0x564f45f3d160 2024-04-17T07:34:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.572+0000 7f6ae2a00640 1 -- 172.21.15.86:0/1565981367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ae8003d10 con 0x564f45f3d160 2024-04-17T07:34:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.572+0000 7f6ae2a00640 1 -- 172.21.15.86:0/1565981367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8011a00 con 0x564f45f3d160 2024-04-17T07:34:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.572+0000 7f6aee000640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6ad80a3bb0 0x7f6ad80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6aee000640 1 -- 172.21.15.86:0/94699173 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6adc0f2ac0 con 0x7f6af0109690 2024-04-17T07:34:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6af5a00640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6ad80a3bb0 0x7f6ad80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1565981367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 msgr2=0x564f45f3d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/1565981367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f3d580 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009fc0 tx=0x7f6ae8010860 comp rx=0 tx=0).stop 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1565981367 shutdown_connections 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/1565981367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f42cc0 0x564f45f45150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/1565981367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3db50 0x564f45f42650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/1565981367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f3d580 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1565981367 >> 172.21.15.86:0/1565981367 conn(0x564f45f386d0 msgr2=0x564f45f38ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1565981367 shutdown_connections 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.573+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1565981367 wait complete. 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.574+0000 7f6af5a00640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6ad80a3bb0 0x7f6ad80a6070 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x564f45f39640 tx=0x7f6ae4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.574+0000 7f6afb1d3740 1 -- start start 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.574+0000 7f6afb1d3740 1 Processor -- start 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.574+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f47690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f3db50 0x564f45f47bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f42cc0 0x564f45f4ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f45f45c70 con 0x564f45f3d160 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564f45e5c830 con 0x564f45f42cc0 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564f45f45df0 con 0x564f45f3db50 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6afb1d3740 10 client.0 _ll_register_callbacks cb 0x564f45e2afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6af5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f47690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6af5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f47690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51520/0 (socket says 172.21.15.86:51520) 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6af5a00640 1 -- 172.21.15.86:0/3977661416 learned_addr learned my addr 172.21.15.86:0/3977661416 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6af5a00640 1 -- 172.21.15.86:0/3977661416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f3db50 msgr2=0x564f45f47bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6af5a00640 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f3db50 0x564f45f47bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6af5a00640 1 -- 172.21.15.86:0/3977661416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f42cc0 msgr2=0x564f45f4ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6af5a00640 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f42cc0 0x564f45f4ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6af5a00640 1 -- 172.21.15.86:0/3977661416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f45e60920 con 0x564f45f3d160 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.575+0000 7f6af5a00640 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f47690 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005f60 tx=0x7f6ae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c50b7d93d7dafe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.576+0000 7f6ad7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.576+0000 7f6ad7400640 1 -- 172.21.15.86:0/3977661416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4005220 con 0x564f45f3d160 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.576+0000 7f6ad7400640 1 -- 172.21.15.86:0/3977661416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6ae40053c0 con 0x564f45f3d160 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.576+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3977661416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae8003d10 con 0x564f45f3d160 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.576+0000 7f6ad7400640 1 -- 172.21.15.86:0/3977661416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ae4030050 con 0x564f45f3d160 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.576+0000 7f6afb1d3740 10 client.19980 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.576+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3977661416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f45e2e7f0 con 0x564f45f3d160 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.576+0000 7f6ad7400640 1 -- 172.21.15.86:0/3977661416 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ae8003d10 con 0x564f45f3d160 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.577+0000 7f6ad7400640 1 -- 172.21.15.86:0/3977661416 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6ae40118e0 con 0x564f45f3d160 2024-04-17T07:34:35.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.577+0000 7f6afb1d3740 10 client.19980 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:35.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.577+0000 7f6afb1d3740 10 client.19980 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:34:35.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.577+0000 7f6afb1d3740 20 client.19980 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:35.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.577+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f45f532d0 0x564f45f55790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.577+0000 7f6afb1d3740 4 client.19980 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:35.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.577+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3977661416 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f45eb0a40 con 0x564f45f532d0 2024-04-17T07:34:35.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.578+0000 7f6af5000640 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f45f532d0 0x564f45f55790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.579+0000 7f6af5000640 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f45f532d0 0x564f45f55790 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=147502a44a2d47b0 server_cookie=1fb5fdcf4b7ceea9 in_seq=0 out_seq=0 2024-04-17T07:34:35.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.579+0000 7f6ad7400640 10 client.19980 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:34:35.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.580+0000 7f6ad7400640 1 -- 172.21.15.86:0/3977661416 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f45eb0a40 con 0x564f45f532d0 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.580+0000 7f6ad7400640 10 client.19980 handle_command_reply: tid=0 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 client.19980 shutdown 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3977661416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 msgr2=0x564f45f47690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f47690 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005f60 tx=0x7f6ae40046c0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3977661416 shutdown_connections 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f45f532d0 0x564f45f55790 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f42cc0 0x564f45f4ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f3db50 0x564f45f47bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/3977661416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f47690 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6ad7400640 0 client.19980 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3977661416 >> 172.21.15.86:0/3977661416 conn(0x564f45f386d0 msgr2=0x564f45f37930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3977661416 shutdown_connections 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3977661416 wait complete. 2024-04-17T07:34:35.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.581+0000 7f6afb1d3740 20 client.19980 trim_cache size 0 max 16384 2024-04-17T07:34:35.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.596+0000 7f6afb1d3740 1 Processor -- start 2024-04-17T07:34:35.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.596+0000 7f6afb1d3740 1 -- start start 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.596+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3d160 0x564f45f472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.596+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3db50 0x564f45f4ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.596+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f42cc0 0x564f45f477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.596+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b0e90 con 0x564f45f3db50 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.596+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b29d0 con 0x564f45f3d160 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.596+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b1560 con 0x564f45f42cc0 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.596+0000 7f6af5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3db50 0x564f45f4ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3db50 0x564f45f4ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51522/0 (socket says 172.21.15.86:51522) 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af5000640 1 -- 172.21.15.86:0/3189769695 learned_addr learned my addr 172.21.15.86:0/3189769695 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af5a00640 1 --2- 172.21.15.86:0/3189769695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3d160 0x564f45f472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af6400640 1 --2- 172.21.15.86:0/3189769695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f42cc0 0x564f45f477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af5000640 1 -- 172.21.15.86:0/3189769695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f42cc0 msgr2=0x564f45f477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af5000640 1 --2- 172.21.15.86:0/3189769695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f42cc0 0x564f45f477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af5000640 1 -- 172.21.15.86:0/3189769695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3d160 msgr2=0x564f45f472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af5000640 1 --2- 172.21.15.86:0/3189769695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3d160 0x564f45f472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af5000640 1 -- 172.21.15.86:0/3189769695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f45bb53f0 con 0x564f45f3db50 2024-04-17T07:34:35.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af5a00640 1 --2- 172.21.15.86:0/3189769695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3d160 0x564f45f472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:35.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.597+0000 7f6af5000640 1 --2- 172.21.15.86:0/3189769695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3db50 0x564f45f4ebd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0be9b0 tx=0x7f6adc0f0110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00343e92cb531cc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6ae2000640 1 -- 172.21.15.86:0/3189769695 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0f9070 con 0x564f45f3db50 2024-04-17T07:34:35.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6ae2000640 1 -- 172.21.15.86:0/3189769695 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6adc03c350 con 0x564f45f3db50 2024-04-17T07:34:35.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6ae2000640 1 -- 172.21.15.86:0/3189769695 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0c7690 con 0x564f45f3db50 2024-04-17T07:34:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3189769695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3db50 msgr2=0x564f45f4ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/3189769695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3db50 0x564f45f4ebd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0be9b0 tx=0x7f6adc0f0110 comp rx=0 tx=0).stop 2024-04-17T07:34:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3189769695 shutdown_connections 2024-04-17T07:34:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/3189769695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f42cc0 0x564f45f477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/3189769695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3db50 0x564f45f4ebd0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/3189769695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f3d160 0x564f45f472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3189769695 >> 172.21.15.86:0/3189769695 conn(0x564f45f386d0 msgr2=0x564f45f3f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3189769695 shutdown_connections 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.598+0000 7f6afb1d3740 1 -- 172.21.15.86:0/3189769695 wait complete. 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.599+0000 7f6afb1d3740 1 -- start start 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.599+0000 7f6afb1d3740 1 Processor -- start 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.599+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f4b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.599+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f3db50 0x564f45f4bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.599+0000 7f6afb1d3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f42cc0 0x564f45fb78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.599+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b2300 con 0x564f45f3d160 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.599+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01b1c30 con 0x564f45f42cc0 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.599+0000 7f6afb1d3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6af01aee60 con 0x564f45f3db50 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6afb1d3740 10 client.0 _ll_register_callbacks cb 0x7f6ae40041b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:35.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6af5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f4b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6af5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f4b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51534/0 (socket says 172.21.15.86:51534) 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6af5a00640 1 -- 172.21.15.86:0/1828822633 learned_addr learned my addr 172.21.15.86:0/1828822633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6af6400640 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f42cc0 0x564f45fb78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6af5000640 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f3db50 0x564f45f4bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6af5a00640 1 -- 172.21.15.86:0/1828822633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f3db50 msgr2=0x564f45f4bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6af5a00640 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f3db50 0x564f45f4bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6af5a00640 1 -- 172.21.15.86:0/1828822633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f42cc0 msgr2=0x564f45fb78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6af5a00640 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f42cc0 0x564f45fb78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.600+0000 7f6af5a00640 1 -- 172.21.15.86:0/1828822633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f45c7f7d0 con 0x564f45f3d160 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.601+0000 7f6af5a00640 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f4b860 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009ba0 tx=0x7f6ae40286b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246f8107174289b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.601+0000 7f6ad7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.601+0000 7f6ad7400640 1 -- 172.21.15.86:0/1828822633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4005040 con 0x564f45f3d160 2024-04-17T07:34:35.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.601+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1828822633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6adc03c350 con 0x564f45f3d160 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.601+0000 7f6ad7400640 1 -- 172.21.15.86:0/1828822633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6ae40051e0 con 0x564f45f3d160 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.601+0000 7f6ad7400640 1 -- 172.21.15.86:0/1828822633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6ae40053e0 con 0x564f45f3d160 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.601+0000 7f6afb1d3740 10 client.19986 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.601+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1828822633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f45fb8bc0 con 0x564f45f3d160 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.601+0000 7f6ad7400640 1 -- 172.21.15.86:0/1828822633 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6adc03c350 con 0x564f45f3d160 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.602+0000 7f6ad7400640 1 -- 172.21.15.86:0/1828822633 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f6ae4004850 con 0x564f45f3d160 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.602+0000 7f6afb1d3740 10 client.19986 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.602+0000 7f6afb1d3740 10 client.19986 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.602+0000 7f6afb1d3740 20 client.19986 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.602+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f45fb95f0 0x564f45fbba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.602+0000 7f6afb1d3740 4 client.19986 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.602+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1828822633 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f45eaa8b0 con 0x564f45fb95f0 2024-04-17T07:34:35.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.602+0000 7f6af5000640 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f45fb95f0 0x564f45fbba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.603+0000 7f6ad7400640 10 client.19986 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.603+0000 7f6af5000640 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f45fb95f0 0x564f45fbba10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14a077a6139cdecb server_cookie=2f71e4adfef1d843 in_seq=0 out_seq=0 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6ad7400640 1 -- 172.21.15.86:0/1828822633 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f45eaa8b0 con 0x564f45fb95f0 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6ad7400640 10 client.19986 handle_command_reply: tid=0 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6afb1d3740 1 client.19986 shutdown 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1828822633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 msgr2=0x564f45f4b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f4b860 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009ba0 tx=0x7f6ae40286b0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1828822633 shutdown_connections 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x564f45fb95f0 0x564f45fbba10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564f45f42cc0 0x564f45fb78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564f45f3db50 0x564f45f4bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6afb1d3740 1 --2- 172.21.15.86:0/1828822633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f45f3d160 0x564f45f4b860 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6ad7400640 0 client.19986 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:34:35.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1828822633 >> 172.21.15.86:0/1828822633 conn(0x564f45f386d0 msgr2=0x564f45f472a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.604+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1828822633 shutdown_connections 2024-04-17T07:34:35.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6afb1d3740 1 -- 172.21.15.86:0/1828822633 wait complete. 2024-04-17T07:34:35.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6afb1d3740 20 client.19986 trim_cache size 0 max 16384 2024-04-17T07:34:35.609 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:34:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6af6e00640 1 -- 172.21.15.86:0/94699173 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6ad80a3bb0 msgr2=0x7f6ad80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6af6e00640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6ad80a3bb0 0x7f6ad80a6070 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x564f45f39640 tx=0x7f6ae4005f90 comp rx=0 tx=0).stop 2024-04-17T07:34:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6af6e00640 1 -- 172.21.15.86:0/94699173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0109690 msgr2=0x7f6af019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6af6e00640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0109690 0x7f6af019ebd0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f6adc006010 tx=0x7f6adc005e40 comp rx=0 tx=0).stop 2024-04-17T07:34:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6af6e00640 1 -- 172.21.15.86:0/94699173 shutdown_connections 2024-04-17T07:34:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6af6e00640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6ad80a3bb0 0x7f6ad80a6070 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6af6e00640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af010a050 0x7f6af01a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6af6e00640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0109690 0x7f6af019ebd0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.605+0000 7f6af6e00640 1 --2- 172.21.15.86:0/94699173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af01084a0 0x7f6af019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.606+0000 7f6af6e00640 1 -- 172.21.15.86:0/94699173 >> 172.21.15.86:0/94699173 conn(0x7f6af0077430 msgr2=0x7f6af010ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.606+0000 7f6af6e00640 1 -- 172.21.15.86:0/94699173 shutdown_connections 2024-04-17T07:34:35.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.606+0000 7f6af6e00640 1 -- 172.21.15.86:0/94699173 wait complete. 2024-04-17T07:34:35.626 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:34:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:35 smithi098 ceph-mon[35418]: pgmap v1046: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 899 op/s 2024-04-17T07:34:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:35 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4240506216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:34:35.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.849+0000 7f2b71800640 1 -- 172.21.15.86:0/1789904770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b6c105d70 msgr2=0x7f2b6c10c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.849+0000 7f2b71800640 1 --2- 172.21.15.86:0/1789904770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b6c105d70 0x7f2b6c10c600 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009ca0 tx=0x7f2b6402c100 comp rx=0 tx=0).stop 2024-04-17T07:34:35.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.850+0000 7f2b71800640 1 -- 172.21.15.86:0/1789904770 shutdown_connections 2024-04-17T07:34:35.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.850+0000 7f2b71800640 1 --2- 172.21.15.86:0/1789904770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b6c105d70 0x7f2b6c10c600 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.850+0000 7f2b71800640 1 --2- 172.21.15.86:0/1789904770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b6c1053b0 0x7f2b6c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.850+0000 7f2b71800640 1 --2- 172.21.15.86:0/1789904770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6c1041c0 0x7f2b6c1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.850+0000 7f2b71800640 1 -- 172.21.15.86:0/1789904770 >> 172.21.15.86:0/1789904770 conn(0x7f2b6c0ff710 msgr2=0x7f2b6c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.850+0000 7f2b71800640 1 -- 172.21.15.86:0/1789904770 shutdown_connections 2024-04-17T07:34:35.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.850+0000 7f2b71800640 1 -- 172.21.15.86:0/1789904770 wait complete. 2024-04-17T07:34:35.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.851+0000 7f2b71800640 1 Processor -- start 2024-04-17T07:34:35.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.851+0000 7f2b71800640 1 -- start start 2024-04-17T07:34:35.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.851+0000 7f2b71800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b6c1041c0 0x7f2b6c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.851+0000 7f2b71800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b6c1053b0 0x7f2b6c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.851+0000 7f2b71800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6c105d70 0x7f2b6c1a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.851+0000 7f2b71800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c10fde0 con 0x7f2b6c1041c0 2024-04-17T07:34:35.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.851+0000 7f2b71800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c10fc60 con 0x7f2b6c105d70 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.851+0000 7f2b71800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c10ff60 con 0x7f2b6c1053b0 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.851+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b6c1041c0 0x7f2b6c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.852+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b6c1041c0 0x7f2b6c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51562/0 (socket says 172.21.15.86:51562) 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.852+0000 7f2b6aa00640 1 -- 172.21.15.86:0/359619506 learned_addr learned my addr 172.21.15.86:0/359619506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.852+0000 7f2b6b400640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6c105d70 0x7f2b6c1a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.852+0000 7f2b6a000640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b6c1053b0 0x7f2b6c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.852+0000 7f2b6aa00640 1 -- 172.21.15.86:0/359619506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b6c1053b0 msgr2=0x7f2b6c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.852+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b6c1053b0 0x7f2b6c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.852+0000 7f2b6aa00640 1 -- 172.21.15.86:0/359619506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6c105d70 msgr2=0x7f2b6c1a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.852+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6c105d70 0x7f2b6c1a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.852+0000 7f2b6aa00640 1 -- 172.21.15.86:0/359619506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c1a2160 con 0x7f2b6c1041c0 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.852+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b6c1041c0 0x7f2b6c19a540 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00b450 tx=0x7f2b5c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.853+0000 7f2b63400640 1 -- 172.21.15.86:0/359619506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c015020 con 0x7f2b6c1041c0 2024-04-17T07:34:35.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.853+0000 7f2b63400640 1 -- 172.21.15.86:0/359619506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b5c011d40 con 0x7f2b6c1041c0 2024-04-17T07:34:35.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.853+0000 7f2b63400640 1 -- 172.21.15.86:0/359619506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00f340 con 0x7f2b6c1041c0 2024-04-17T07:34:35.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.853+0000 7f2b71800640 1 -- 172.21.15.86:0/359619506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c1a2450 con 0x7f2b6c1041c0 2024-04-17T07:34:35.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.853+0000 7f2b71800640 1 -- 172.21.15.86:0/359619506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c1a29d0 con 0x7f2b6c1041c0 2024-04-17T07:34:35.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.856+0000 7f2b74454740 1 Processor -- start 2024-04-17T07:34:35.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.856+0000 7f2b74454740 1 -- start start 2024-04-17T07:34:35.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b74454740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fcfa0 0x55ae983fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b74454740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fd990 0x55ae98402490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b74454740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae98402b00 0x55ae98404f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b74454740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae98363020 con 0x55ae983fd990 2024-04-17T07:34:35.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b74454740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ae98309460 con 0x55ae983fcfa0 2024-04-17T07:34:35.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b74454740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae9835efe0 con 0x55ae98402b00 2024-04-17T07:34:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b63400640 1 -- 172.21.15.86:0/359619506 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2b5c018020 con 0x7f2b6c1041c0 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fd990 0x55ae98402490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b6a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae98402b00 0x55ae98404f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b6a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae98402b00 0x55ae98404f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.86:49736/0 (socket says 172.21.15.86:49736) 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b6b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fcfa0 0x55ae983fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.857+0000 7f2b6a000640 1 -- 172.21.15.86:0/3471353501 learned_addr learned my addr 172.21.15.86:0/3471353501 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.858+0000 7f2b6aa00640 1 -- 172.21.15.86:0/3471353501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae98402b00 msgr2=0x55ae98404f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.858+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/3471353501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae98402b00 0x55ae98404f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.858+0000 7f2b6aa00640 1 -- 172.21.15.86:0/3471353501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fcfa0 msgr2=0x55ae983fd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.858+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/3471353501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fcfa0 0x55ae983fd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.858+0000 7f2b6aa00640 1 -- 172.21.15.86:0/3471353501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae98311040 con 0x55ae983fd990 2024-04-17T07:34:35.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.858+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/3471353501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fd990 0x55ae98402490 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0cd950 tx=0x7f2b5c0d8880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0390063bbb05cec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.858+0000 7f2b63400640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2b500a3c30 0x7f2b500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.859+0000 7f2b57e00640 1 -- 172.21.15.86:0/3471353501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c0e3070 con 0x55ae983fd990 2024-04-17T07:34:35.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.859+0000 7f2b57e00640 1 -- 172.21.15.86:0/3471353501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b5c09c860 con 0x55ae983fd990 2024-04-17T07:34:35.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.859+0000 7f2b57e00640 1 -- 172.21.15.86:0/3471353501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c086b20 con 0x55ae983fd990 2024-04-17T07:34:35.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.859+0000 7f2b63400640 1 -- 172.21.15.86:0/359619506 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2b5c0d0200 con 0x7f2b6c1041c0 2024-04-17T07:34:35.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.859+0000 7f2b6b400640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2b500a3c30 0x7f2b500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 -- 172.21.15.86:0/3471353501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fd990 msgr2=0x55ae98402490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b6b400640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2b500a3c30 0x7f2b500a60f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2b64006010 tx=0x7f2b64009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 --2- 172.21.15.86:0/3471353501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fd990 0x55ae98402490 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0cd950 tx=0x7f2b5c0d8880 comp rx=0 tx=0).stop 2024-04-17T07:34:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 -- 172.21.15.86:0/3471353501 shutdown_connections 2024-04-17T07:34:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 --2- 172.21.15.86:0/3471353501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae98402b00 0x55ae98404f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 --2- 172.21.15.86:0/3471353501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fd990 0x55ae98402490 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 --2- 172.21.15.86:0/3471353501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fcfa0 0x55ae983fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 -- 172.21.15.86:0/3471353501 >> 172.21.15.86:0/3471353501 conn(0x55ae983f8510 msgr2=0x55ae983f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 -- 172.21.15.86:0/3471353501 shutdown_connections 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 -- 172.21.15.86:0/3471353501 wait complete. 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 -- start start 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.860+0000 7f2b74454740 1 Processor -- start 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b74454740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fcfa0 0x55ae98407450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b74454740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fd990 0x55ae98407990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b74454740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae9840e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b74454740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae98405ab0 con 0x55ae98402b00 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b74454740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ae9831c830 con 0x55ae983fd990 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b74454740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ae98405c30 con 0x55ae983fcfa0 2024-04-17T07:34:35.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b74454740 10 client.0 _ll_register_callbacks cb 0x55ae982eafb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae9840e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae9840e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51570/0 (socket says 172.21.15.86:51570) 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b6b400640 1 -- 172.21.15.86:0/2811368846 learned_addr learned my addr 172.21.15.86:0/2811368846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b6a000640 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fd990 0x55ae98407990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.861+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fcfa0 0x55ae98407450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.862+0000 7f2b6b400640 1 -- 172.21.15.86:0/2811368846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fcfa0 msgr2=0x55ae98407450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.862+0000 7f2b6b400640 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fcfa0 0x55ae98407450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.862+0000 7f2b6b400640 1 -- 172.21.15.86:0/2811368846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fd990 msgr2=0x55ae98407990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.862+0000 7f2b6b400640 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fd990 0x55ae98407990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.862+0000 7f2b6b400640 1 -- 172.21.15.86:0/2811368846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae98320920 con 0x55ae98402b00 2024-04-17T07:34:35.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.862+0000 7f2b6b400640 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae9840e9d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2b64036000 tx=0x7f2b640095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0dc69f569f20412 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.862+0000 7f2b54c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.862+0000 7f2b54c00640 1 -- 172.21.15.86:0/2811368846 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64039040 con 0x55ae98402b00 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.862+0000 7f2b54c00640 1 -- 172.21.15.86:0/2811368846 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2b64004450 con 0x55ae98402b00 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.862+0000 7f2b74454740 1 -- 172.21.15.86:0/2811368846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5c09c860 con 0x55ae98402b00 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b54c00640 1 -- 172.21.15.86:0/2811368846 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b6405a050 con 0x55ae98402b00 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b74454740 10 client.19998 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b74454740 1 -- 172.21.15.86:0/2811368846 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae982ee7f0 con 0x55ae98402b00 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b54c00640 1 -- 172.21.15.86:0/2811368846 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b5c09c860 con 0x55ae98402b00 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b54c00640 1 -- 172.21.15.86:0/2811368846 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2b6404cc10 con 0x55ae98402b00 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b74454740 10 client.19998 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b74454740 10 client.19998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b74454740 20 client.19998 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b74454740 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ae98412fe0 0x55ae98415460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b74454740 4 client.19998 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.863+0000 7f2b74454740 1 -- 172.21.15.86:0/2811368846 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae98370a40 con 0x55ae98412fe0 2024-04-17T07:34:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.864+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ae98412fe0 0x55ae98415460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.865+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ae98412fe0 0x55ae98415460 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4889a3f266afcc9a server_cookie=f34a1d204057d896 in_seq=0 out_seq=0 2024-04-17T07:34:35.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.865+0000 7f2b54c00640 10 client.19998 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b54c00640 1 -- 172.21.15.86:0/2811368846 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae98370a40 con 0x55ae98412fe0 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b54c00640 10 client.19998 handle_command_reply: tid=0 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b74454740 1 client.19998 shutdown 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b74454740 1 -- 172.21.15.86:0/2811368846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 msgr2=0x55ae9840e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b74454740 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae9840e9d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2b64036000 tx=0x7f2b640095c0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b74454740 1 -- 172.21.15.86:0/2811368846 shutdown_connections 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b74454740 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ae98412fe0 0x55ae98415460 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b74454740 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae9840e9d0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b74454740 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fd990 0x55ae98407990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b74454740 1 --2- 172.21.15.86:0/2811368846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fcfa0 0x55ae98407450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b54c00640 0 client.19998 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.869+0000 7f2b74454740 1 -- 172.21.15.86:0/2811368846 >> 172.21.15.86:0/2811368846 conn(0x55ae983f8510 msgr2=0x55ae983f7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.870+0000 7f2b74454740 1 -- 172.21.15.86:0/2811368846 shutdown_connections 2024-04-17T07:34:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.870+0000 7f2b74454740 1 -- 172.21.15.86:0/2811368846 wait complete. 2024-04-17T07:34:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.870+0000 7f2b74454740 20 client.19998 trim_cache size 0 max 16384 2024-04-17T07:34:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.886+0000 7f2b74454740 1 Processor -- start 2024-04-17T07:34:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.886+0000 7f2b74454740 1 -- start start 2024-04-17T07:34:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b74454740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fcfa0 0x55ae983f3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b74454740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fd990 0x55ae9840e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b74454740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae983f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b74454740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1acee0 con 0x55ae98402b00 2024-04-17T07:34:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b74454740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1aea20 con 0x55ae983fcfa0 2024-04-17T07:34:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b74454740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1ad5b0 con 0x55ae983fd990 2024-04-17T07:34:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae983f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae983f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51574/0 (socket says 172.21.15.86:51574) 2024-04-17T07:34:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b6b400640 1 -- 172.21.15.86:0/814433631 learned_addr learned my addr 172.21.15.86:0/814433631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b6b400640 1 -- 172.21.15.86:0/814433631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fd990 msgr2=0x55ae9840e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b6b400640 1 --2- 172.21.15.86:0/814433631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fd990 0x55ae9840e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b6b400640 1 -- 172.21.15.86:0/814433631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fcfa0 msgr2=0x55ae983f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b6b400640 1 --2- 172.21.15.86:0/814433631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fcfa0 0x55ae983f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.887+0000 7f2b6b400640 1 -- 172.21.15.86:0/814433631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae980753f0 con 0x55ae98402b00 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.888+0000 7f2b6b400640 1 --2- 172.21.15.86:0/814433631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae983f6e80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2b64037040 tx=0x7f2b64032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaafa5f058e61f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.888+0000 7f2b57400640 1 -- 172.21.15.86:0/814433631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64062070 con 0x55ae98402b00 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.888+0000 7f2b57400640 1 -- 172.21.15.86:0/814433631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b64051070 con 0x55ae98402b00 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.888+0000 7f2b57400640 1 -- 172.21.15.86:0/814433631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64040b30 con 0x55ae98402b00 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.889+0000 7f2b74454740 1 -- 172.21.15.86:0/814433631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 msgr2=0x55ae983f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.889+0000 7f2b74454740 1 --2- 172.21.15.86:0/814433631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae983f6e80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2b64037040 tx=0x7f2b64032c70 comp rx=0 tx=0).stop 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.889+0000 7f2b74454740 1 -- 172.21.15.86:0/814433631 shutdown_connections 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.889+0000 7f2b74454740 1 --2- 172.21.15.86:0/814433631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae98402b00 0x55ae983f6e80 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.889+0000 7f2b74454740 1 --2- 172.21.15.86:0/814433631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fd990 0x55ae9840e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.889+0000 7f2b74454740 1 --2- 172.21.15.86:0/814433631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae983fcfa0 0x55ae983f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.889+0000 7f2b74454740 1 -- 172.21.15.86:0/814433631 >> 172.21.15.86:0/814433631 conn(0x55ae983f8510 msgr2=0x55ae983fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.889+0000 7f2b74454740 1 -- 172.21.15.86:0/814433631 shutdown_connections 2024-04-17T07:34:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.889+0000 7f2b74454740 1 -- 172.21.15.86:0/814433631 wait complete. 2024-04-17T07:34:35.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.889+0000 7f2b74454740 1 -- start start 2024-04-17T07:34:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b74454740 1 Processor -- start 2024-04-17T07:34:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b74454740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fcfa0 0x55ae9840be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b74454740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fd990 0x55ae9840c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b74454740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae98402b00 0x55ae9840d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b74454740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1ae350 con 0x55ae983fcfa0 2024-04-17T07:34:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b74454740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1adc80 con 0x55ae98402b00 2024-04-17T07:34:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b74454740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c1aaeb0 con 0x55ae983fd990 2024-04-17T07:34:35.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b74454740 10 client.0 _ll_register_callbacks cb 0x7f2b640325c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fcfa0 0x55ae9840be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fcfa0 0x55ae9840be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51584/0 (socket says 172.21.15.86:51584) 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.890+0000 7f2b6aa00640 1 -- 172.21.15.86:0/3392490066 learned_addr learned my addr 172.21.15.86:0/3392490066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b6a000640 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fd990 0x55ae9840c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b6b400640 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae98402b00 0x55ae9840d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b6aa00640 1 -- 172.21.15.86:0/3392490066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fd990 msgr2=0x55ae9840c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fd990 0x55ae9840c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b6aa00640 1 -- 172.21.15.86:0/3392490066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae98402b00 msgr2=0x55ae9840d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae98402b00 0x55ae9840d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b6aa00640 1 -- 172.21.15.86:0/3392490066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae9813f7d0 con 0x55ae983fcfa0 2024-04-17T07:34:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b6aa00640 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fcfa0 0x55ae9840be70 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009fd0 tx=0x7f2b5c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313c1e3efedbb103 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b54c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b54c00640 1 -- 172.21.15.86:0/3392490066 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c089520 con 0x55ae983fcfa0 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b54c00640 1 -- 172.21.15.86:0/3392490066 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2b5c09a460 con 0x55ae983fcfa0 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.891+0000 7f2b74454740 1 -- 172.21.15.86:0/3392490066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b64051070 con 0x55ae983fcfa0 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b54c00640 1 -- 172.21.15.86:0/3392490066 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c089720 con 0x55ae983fcfa0 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b74454740 10 client.20004 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b74454740 1 -- 172.21.15.86:0/3392490066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae9840eb50 con 0x55ae983fcfa0 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b54c00640 1 -- 172.21.15.86:0/3392490066 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b64051070 con 0x55ae983fcfa0 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b54c00640 1 -- 172.21.15.86:0/3392490066 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f2b5c021300 con 0x55ae983fcfa0 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b74454740 10 client.20004 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b74454740 10 client.20004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b74454740 20 client.20004 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b74454740 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ae984797b0 0x55ae9847bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b74454740 4 client.20004 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:34:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.892+0000 7f2b74454740 1 -- 172.21.15.86:0/3392490066 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae9836a8b0 con 0x55ae984797b0 2024-04-17T07:34:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.893+0000 7f2b6a000640 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ae984797b0 0x55ae9847bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.894+0000 7f2b6a000640 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ae984797b0 0x55ae9847bbd0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8969372d0fa85280 server_cookie=d366ef8ac858c884 in_seq=0 out_seq=0 2024-04-17T07:34:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.894+0000 7f2b54c00640 10 client.20004 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:35.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.897+0000 7f2b54c00640 1 -- 172.21.15.86:0/3392490066 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae9836a8b0 con 0x55ae984797b0 2024-04-17T07:34:35.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.897+0000 7f2b54c00640 10 client.20004 handle_command_reply: tid=0 2024-04-17T07:34:35.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.897+0000 7f2b74454740 1 client.20004 shutdown 2024-04-17T07:34:35.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 1 -- 172.21.15.86:0/3392490066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fcfa0 msgr2=0x55ae9840be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fcfa0 0x55ae9840be70 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009fd0 tx=0x7f2b5c011f10 comp rx=0 tx=0).stop 2024-04-17T07:34:35.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 1 -- 172.21.15.86:0/3392490066 shutdown_connections 2024-04-17T07:34:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55ae984797b0 0x55ae9847bbd0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ae98402b00 0x55ae9840d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ae983fd990 0x55ae9840c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 1 --2- 172.21.15.86:0/3392490066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae983fcfa0 0x55ae9840be70 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 1 -- 172.21.15.86:0/3392490066 >> 172.21.15.86:0/3392490066 conn(0x55ae983f8510 msgr2=0x55ae983fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b54c00640 0 client.20004 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 1 -- 172.21.15.86:0/3392490066 shutdown_connections 2024-04-17T07:34:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 1 -- 172.21.15.86:0/3392490066 wait complete. 2024-04-17T07:34:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.898+0000 7f2b74454740 20 client.20004 trim_cache size 0 max 16384 2024-04-17T07:34:35.899 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:34:35.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 -- 172.21.15.86:0/359619506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2b500a3c30 msgr2=0x7f2b500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2b500a3c30 0x7f2b500a60f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2b64006010 tx=0x7f2b64009360 comp rx=0 tx=0).stop 2024-04-17T07:34:35.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 -- 172.21.15.86:0/359619506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b6c1041c0 msgr2=0x7f2b6c19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:35.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b6c1041c0 0x7f2b6c19a540 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00b450 tx=0x7f2b5c00b910 comp rx=0 tx=0).stop 2024-04-17T07:34:35.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 -- 172.21.15.86:0/359619506 shutdown_connections 2024-04-17T07:34:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2b500a3c30 0x7f2b500a60f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6c105d70 0x7f2b6c1a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b6c1053b0 0x7f2b6c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 --2- 172.21.15.86:0/359619506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b6c1041c0 0x7f2b6c19a540 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 -- 172.21.15.86:0/359619506 >> 172.21.15.86:0/359619506 conn(0x7f2b6c0ff710 msgr2=0x7f2b6c1087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 -- 172.21.15.86:0/359619506 shutdown_connections 2024-04-17T07:34:35.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:35.899+0000 7f2b71800640 1 -- 172.21.15.86:0/359619506 wait complete. 2024-04-17T07:34:36.917 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-17T07:34:36.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:34:36.918 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-17T07:34:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.152+0000 7f10bf200640 1 -- 172.21.15.86:0/3643221450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0101e70 msgr2=0x7f10c0102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.152+0000 7f10bf200640 1 --2- 172.21.15.86:0/3643221450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0101e70 0x7f10c0102290 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009900 tx=0x7f10b002bc30 comp rx=0 tx=0).stop 2024-04-17T07:34:37.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.152+0000 7f10bf200640 1 -- 172.21.15.86:0/3643221450 shutdown_connections 2024-04-17T07:34:37.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.152+0000 7f10bf200640 1 --2- 172.21.15.86:0/3643221450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c0103980 0x7f10c010a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.152+0000 7f10bf200640 1 --2- 172.21.15.86:0/3643221450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10c0102fc0 0x7f10c0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.152+0000 7f10bf200640 1 --2- 172.21.15.86:0/3643221450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0101e70 0x7f10c0102290 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.152+0000 7f10bf200640 1 -- 172.21.15.86:0/3643221450 >> 172.21.15.86:0/3643221450 conn(0x7f10c00fd400 msgr2=0x7f10c00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.152+0000 7f10bf200640 1 -- 172.21.15.86:0/3643221450 shutdown_connections 2024-04-17T07:34:37.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.152+0000 7f10bf200640 1 -- 172.21.15.86:0/3643221450 wait complete. 2024-04-17T07:34:37.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.153+0000 7f10bf200640 1 Processor -- start 2024-04-17T07:34:37.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.153+0000 7f10bf200640 1 -- start start 2024-04-17T07:34:37.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.153+0000 7f10bf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0101e70 0x7f10c0198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.153+0000 7f10bf200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c0102fc0 0x7f10c01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.153+0000 7f10bf200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10c0103980 0x7f10c019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.153+0000 7f10bf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010da40 con 0x7f10c0101e70 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.153+0000 7f10bf200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010d8c0 con 0x7f10c0102fc0 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.153+0000 7f10bf200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010dbc0 con 0x7f10c0103980 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0101e70 0x7f10c0198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0101e70 0x7f10c0198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43384/0 (socket says 172.21.15.86:43384) 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bde00640 1 -- 172.21.15.86:0/2718432732 learned_addr learned my addr 172.21.15.86:0/2718432732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10be800640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10c0103980 0x7f10c019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bd400640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c0102fc0 0x7f10c01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bde00640 1 -- 172.21.15.86:0/2718432732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10c0103980 msgr2=0x7f10c019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bde00640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10c0103980 0x7f10c019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bde00640 1 -- 172.21.15.86:0/2718432732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c0102fc0 msgr2=0x7f10c01986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bde00640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c0102fc0 0x7f10c01986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bde00640 1 -- 172.21.15.86:0/2718432732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c019fd90 con 0x7f10c0101e70 2024-04-17T07:34:37.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bd400640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c0102fc0 0x7f10c01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:37.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.154+0000 7f10bde00640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0101e70 0x7f10c0198170 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f10b0036040 tx=0x7f10b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.155+0000 7f10b6a00640 1 -- 172.21.15.86:0/2718432732 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0043020 con 0x7f10c0101e70 2024-04-17T07:34:37.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.155+0000 7f10b6a00640 1 -- 172.21.15.86:0/2718432732 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10b00054c0 con 0x7f10c0101e70 2024-04-17T07:34:37.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.155+0000 7f10bf200640 1 -- 172.21.15.86:0/2718432732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c01a0020 con 0x7f10c0101e70 2024-04-17T07:34:37.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.155+0000 7f10b6a00640 1 -- 172.21.15.86:0/2718432732 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0003d70 con 0x7f10c0101e70 2024-04-17T07:34:37.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.155+0000 7f10bf200640 1 -- 172.21.15.86:0/2718432732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c01a05f0 con 0x7f10c0101e70 2024-04-17T07:34:37.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.158+0000 7f10b6a00640 1 -- 172.21.15.86:0/2718432732 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f10b0046020 con 0x7f10c0101e70 2024-04-17T07:34:37.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.158+0000 7f10c75fc740 1 Processor -- start 2024-04-17T07:34:37.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.159+0000 7f10c75fc740 1 -- start start 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.159+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd74b0 0x555bcabd78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.159+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd7ea0 0x555bcabdc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.159+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabdf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.159+0000 7f10c75fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555bcab38cb0 con 0x555bcabd74b0 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.159+0000 7f10c75fc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555bcaadf370 con 0x555bcabdd010 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.159+0000 7f10c75fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555bcab34c70 con 0x555bcabd7ea0 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.159+0000 7f10be800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd74b0 0x555bcabd78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.159+0000 7f10be800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd74b0 0x555bcabd78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43398/0 (socket says 172.21.15.86:43398) 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.159+0000 7f10be800640 1 -- 172.21.15.86:0/2516846480 learned_addr learned my addr 172.21.15.86:0/2516846480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10bde00640 1 --2- 172.21.15.86:0/2516846480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd7ea0 0x555bcabdc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10bd400640 1 --2- 172.21.15.86:0/2516846480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabdf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10b6a00640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f10a40a3bb0 0x7f10a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10be800640 1 -- 172.21.15.86:0/2516846480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd7ea0 msgr2=0x555bcabdc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10be800640 1 --2- 172.21.15.86:0/2516846480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd7ea0 0x555bcabdc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10be800640 1 -- 172.21.15.86:0/2516846480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 msgr2=0x555bcabdf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10be800640 1 --2- 172.21.15.86:0/2516846480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabdf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10be800640 1 -- 172.21.15.86:0/2516846480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bcaafe910 con 0x555bcabd74b0 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10bde00640 1 --2- 172.21.15.86:0/2516846480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd7ea0 0x555bcabdc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10b6a00640 1 -- 172.21.15.86:0/2718432732 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f10b00f2d70 con 0x7f10c0101e70 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.160+0000 7f10be800640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f10a40a3bb0 0x7f10a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.161+0000 7f10be800640 1 --2- 172.21.15.86:0/2516846480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd74b0 0x555bcabd78d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f10b8008d00 tx=0x7f10b800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73a4d458c6853d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.161+0000 7f10ab000640 1 -- 172.21.15.86:0/2516846480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b801f070 con 0x555bcabd74b0 2024-04-17T07:34:37.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.161+0000 7f10ab000640 1 -- 172.21.15.86:0/2516846480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10b8019db0 con 0x555bcabd74b0 2024-04-17T07:34:37.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.161+0000 7f10ab000640 1 -- 172.21.15.86:0/2516846480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b800ec70 con 0x555bcabd74b0 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.162+0000 7f10c75fc740 1 -- 172.21.15.86:0/2516846480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd74b0 msgr2=0x555bcabd78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.162+0000 7f10be800640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f10a40a3bb0 0x7f10a40a6070 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f10b801c000 tx=0x7f10b801f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.162+0000 7f10c75fc740 1 --2- 172.21.15.86:0/2516846480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd74b0 0x555bcabd78d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f10b8008d00 tx=0x7f10b800d710 comp rx=0 tx=0).stop 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.162+0000 7f10c75fc740 1 -- 172.21.15.86:0/2516846480 shutdown_connections 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.162+0000 7f10c75fc740 1 --2- 172.21.15.86:0/2516846480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabdf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.162+0000 7f10c75fc740 1 --2- 172.21.15.86:0/2516846480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd7ea0 0x555bcabdc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.162+0000 7f10c75fc740 1 --2- 172.21.15.86:0/2516846480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd74b0 0x555bcabd78d0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.162+0000 7f10c75fc740 1 -- 172.21.15.86:0/2516846480 >> 172.21.15.86:0/2516846480 conn(0x555bcabce610 msgr2=0x555bcabcea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.162+0000 7f10c75fc740 1 -- 172.21.15.86:0/2516846480 shutdown_connections 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.162+0000 7f10c75fc740 1 -- 172.21.15.86:0/2516846480 wait complete. 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.163+0000 7f10c75fc740 1 -- start start 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.163+0000 7f10c75fc740 1 Processor -- start 2024-04-17T07:34:37.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.163+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabd74b0 0x555bcabe1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.163+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.163+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabdd010 0x555bcabe8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.163+0000 7f10c75fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555bcabdffc0 con 0x555bcabd7ea0 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10c75fc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555bcaaf2730 con 0x555bcabd74b0 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10c75fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555bcabe0140 con 0x555bcabdd010 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10c75fc740 10 client.0 _ll_register_callbacks cb 0x555bcaaf4b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43412/0 (socket says 172.21.15.86:43412) 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10bd400640 1 -- 172.21.15.86:0/1744766123 learned_addr learned my addr 172.21.15.86:0/1744766123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10be800640 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabdd010 0x555bcabe8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10bde00640 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabd74b0 0x555bcabe1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10bd400640 1 -- 172.21.15.86:0/1744766123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabdd010 msgr2=0x555bcabe8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10bd400640 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabdd010 0x555bcabe8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10bd400640 1 -- 172.21.15.86:0/1744766123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabd74b0 msgr2=0x555bcabe1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10bd400640 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabd74b0 0x555bcabe1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.164+0000 7f10bd400640 1 -- 172.21.15.86:0/1744766123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bca9137e0 con 0x555bcabd7ea0 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.165+0000 7f10bd400640 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe1f80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00e2d0 tx=0x7f10ac00e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b195756dd04dff25 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.165+0000 7f109f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.165+0000 7f109f400640 1 -- 172.21.15.86:0/1744766123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac018030 con 0x555bcabd7ea0 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.165+0000 7f109f400640 1 -- 172.21.15.86:0/1744766123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f10ac00f040 con 0x555bcabd7ea0 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.165+0000 7f10c75fc740 1 -- 172.21.15.86:0/1744766123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b8019db0 con 0x555bcabd7ea0 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.165+0000 7f109f400640 1 -- 172.21.15.86:0/1744766123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10ac022050 con 0x555bcabd7ea0 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.165+0000 7f10c75fc740 10 client.20016 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:37.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.165+0000 7f10c75fc740 1 -- 172.21.15.86:0/1744766123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555bca973f70 con 0x555bcabd7ea0 2024-04-17T07:34:37.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.166+0000 7f109f400640 1 -- 172.21.15.86:0/1744766123 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10b8019db0 con 0x555bcabd7ea0 2024-04-17T07:34:37.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.166+0000 7f109f400640 1 -- 172.21.15.86:0/1744766123 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f10ac01d780 con 0x555bcabd7ea0 2024-04-17T07:34:37.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.166+0000 7f10c75fc740 10 client.20016 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:37.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.166+0000 7f10c75fc740 10 client.20016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:37.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.166+0000 7f10c75fc740 20 client.20016 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:37.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.166+0000 7f10c75fc740 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555bcabed9a0 0x555bcabefe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.166+0000 7f10c75fc740 4 client.20016 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:37.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.166+0000 7f10c75fc740 1 -- 172.21.15.86:0/1744766123 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bcab466d0 con 0x555bcabed9a0 2024-04-17T07:34:37.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.167+0000 7f10bde00640 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555bcabed9a0 0x555bcabefe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.168+0000 7f10bde00640 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555bcabed9a0 0x555bcabefe60 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582a79793871aaa8 server_cookie=bab662db4d51b2a9 in_seq=0 out_seq=0 2024-04-17T07:34:37.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.168+0000 7f109f400640 10 client.20016 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f109f400640 1 -- 172.21.15.86:0/1744766123 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bcab466d0 con 0x555bcabed9a0 2024-04-17T07:34:37.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f109f400640 10 client.20016 handle_command_reply: tid=0 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 client.20016 shutdown 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 -- 172.21.15.86:0/1744766123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 msgr2=0x555bcabe1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe1f80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00e2d0 tx=0x7f10ac00e790 comp rx=0 tx=0).stop 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 -- 172.21.15.86:0/1744766123 shutdown_connections 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555bcabed9a0 0x555bcabefe60 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabdd010 0x555bcabe8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe1f80 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 --2- 172.21.15.86:0/1744766123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabd74b0 0x555bcabe1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f109f400640 0 client.20016 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 -- 172.21.15.86:0/1744766123 >> 172.21.15.86:0/1744766123 conn(0x555bcabce610 msgr2=0x555bcabcd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 -- 172.21.15.86:0/1744766123 shutdown_connections 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.173+0000 7f10c75fc740 1 -- 172.21.15.86:0/1744766123 wait complete. 2024-04-17T07:34:37.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.174+0000 7f10c75fc740 20 client.20016 trim_cache size 0 max 16384 2024-04-17T07:34:37.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.194+0000 7f10c75fc740 1 Processor -- start 2024-04-17T07:34:37.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.194+0000 7f10c75fc740 1 -- start start 2024-04-17T07:34:37.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd74b0 0x555bcabcd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabcd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10c75fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10c01a8760 con 0x555bcabd7ea0 2024-04-17T07:34:37.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10c75fc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10c01aa2a0 con 0x555bcabdd010 2024-04-17T07:34:37.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10c75fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10c01a8e30 con 0x555bcabd74b0 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43416/0 (socket says 172.21.15.86:43416) 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10bd400640 1 -- 172.21.15.86:0/553144567 learned_addr learned my addr 172.21.15.86:0/553144567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10bde00640 1 --2- 172.21.15.86:0/553144567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd74b0 0x555bcabcd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.195+0000 7f10be800640 1 --2- 172.21.15.86:0/553144567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabcd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.196+0000 7f10bd400640 1 -- 172.21.15.86:0/553144567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd74b0 msgr2=0x555bcabcd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.196+0000 7f10bd400640 1 --2- 172.21.15.86:0/553144567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd74b0 0x555bcabcd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.196+0000 7f10bd400640 1 -- 172.21.15.86:0/553144567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 msgr2=0x555bcabcd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.196+0000 7f10bd400640 1 --2- 172.21.15.86:0/553144567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabcd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.196+0000 7f10bd400640 1 -- 172.21.15.86:0/553144567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bca80b170 con 0x555bcabd7ea0 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.196+0000 7f10bd400640 1 --2- 172.21.15.86:0/553144567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe8f80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f10ac001eb0 tx=0x7f10ac00e110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2550945f055f24a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.196+0000 7f10aa600640 1 -- 172.21.15.86:0/553144567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac02a070 con 0x555bcabd7ea0 2024-04-17T07:34:37.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.196+0000 7f10aa600640 1 -- 172.21.15.86:0/553144567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10ac00c370 con 0x555bcabd7ea0 2024-04-17T07:34:37.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.196+0000 7f10aa600640 1 -- 172.21.15.86:0/553144567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac018030 con 0x555bcabd7ea0 2024-04-17T07:34:37.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.197+0000 7f10c75fc740 1 -- 172.21.15.86:0/553144567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 msgr2=0x555bcabe8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.197+0000 7f10c75fc740 1 --2- 172.21.15.86:0/553144567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe8f80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f10ac001eb0 tx=0x7f10ac00e110 comp rx=0 tx=0).stop 2024-04-17T07:34:37.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.197+0000 7f10c75fc740 1 -- 172.21.15.86:0/553144567 shutdown_connections 2024-04-17T07:34:37.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.197+0000 7f10c75fc740 1 --2- 172.21.15.86:0/553144567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabcd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.197+0000 7f10c75fc740 1 --2- 172.21.15.86:0/553144567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe8f80 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.197+0000 7f10c75fc740 1 --2- 172.21.15.86:0/553144567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd74b0 0x555bcabcd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.197+0000 7f10c75fc740 1 -- 172.21.15.86:0/553144567 >> 172.21.15.86:0/553144567 conn(0x555bcabce610 msgr2=0x555bcabce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.197+0000 7f10c75fc740 1 -- 172.21.15.86:0/553144567 shutdown_connections 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.197+0000 7f10c75fc740 1 -- 172.21.15.86:0/553144567 wait complete. 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.198+0000 7f10c75fc740 1 -- start start 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.198+0000 7f10c75fc740 1 Processor -- start 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.198+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd74b0 0x555bcabe6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10c75fc740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabe8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10c75fc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10c01a9bd0 con 0x555bcabd7ea0 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10c75fc740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10c01a9500 con 0x555bcabdd010 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10c75fc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10c01a6730 con 0x555bcabd74b0 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10c75fc740 10 client.0 _ll_register_callbacks cb 0x7f10ac00c190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:37.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43424/0 (socket says 172.21.15.86:43424) 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10bd400640 1 -- 172.21.15.86:0/2078805503 learned_addr learned my addr 172.21.15.86:0/2078805503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10bde00640 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd74b0 0x555bcabe6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10bd400640 1 -- 172.21.15.86:0/2078805503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd74b0 msgr2=0x555bcabe6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10bd400640 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd74b0 0x555bcabe6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10be800640 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabe8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10bd400640 1 -- 172.21.15.86:0/2078805503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 msgr2=0x555bcabe8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10bd400640 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabe8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.199+0000 7f10bd400640 1 -- 172.21.15.86:0/2078805503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bcabe8a00 con 0x555bcabd7ea0 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.200+0000 7f10bd400640 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe7450 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f10ac000c00 tx=0x7f10ac013d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19c23e21d81b84a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.200+0000 7f109f400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.200+0000 7f109f400640 1 -- 172.21.15.86:0/2078805503 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac018050 con 0x555bcabd7ea0 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.200+0000 7f109f400640 1 -- 172.21.15.86:0/2078805503 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f10ac01d600 con 0x555bcabd7ea0 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.200+0000 7f10c75fc740 1 -- 172.21.15.86:0/2078805503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ac00c370 con 0x555bcabd7ea0 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.200+0000 7f109f400640 1 -- 172.21.15.86:0/2078805503 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f10ac03c050 con 0x555bcabd7ea0 2024-04-17T07:34:37.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.200+0000 7f10c75fc740 10 client.20022 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:37.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.200+0000 7f10c75fc740 1 -- 172.21.15.86:0/2078805503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555bcac547e0 con 0x555bcabd7ea0 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.201+0000 7f109f400640 1 -- 172.21.15.86:0/2078805503 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10ac00c370 con 0x555bcabd7ea0 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.201+0000 7f109f400640 1 -- 172.21.15.86:0/2078805503 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f10ac026410 con 0x555bcabd7ea0 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.201+0000 7f10c75fc740 10 client.20022 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.201+0000 7f10c75fc740 10 client.20022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.201+0000 7f10c75fc740 20 client.20022 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.201+0000 7f10c75fc740 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555bcac551a0 0x555bcac575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.201+0000 7f10c75fc740 4 client.20022 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.201+0000 7f10c75fc740 1 -- 172.21.15.86:0/2078805503 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555bcab40540 con 0x555bcac551a0 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.201+0000 7f10bde00640 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555bcac551a0 0x555bcac575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.202+0000 7f10bde00640 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555bcac551a0 0x555bcac575c0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef86d28dbabb0d3b server_cookie=59e1307c36e91c48 in_seq=0 out_seq=0 2024-04-17T07:34:37.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.202+0000 7f109f400640 10 client.20022 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f109f400640 1 -- 172.21.15.86:0/2078805503 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555bcab40540 con 0x555bcac551a0 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f109f400640 10 client.20022 handle_command_reply: tid=0 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f10c75fc740 1 client.20022 shutdown 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f10c75fc740 1 -- 172.21.15.86:0/2078805503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 msgr2=0x555bcabe7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f10c75fc740 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe7450 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f10ac000c00 tx=0x7f10ac013d50 comp rx=0 tx=0).stop 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f10c75fc740 1 -- 172.21.15.86:0/2078805503 shutdown_connections 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f10c75fc740 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x555bcac551a0 0x555bcac575c0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f10c75fc740 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bcabdd010 0x555bcabe8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f10c75fc740 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555bcabd7ea0 0x555bcabe7450 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f10c75fc740 1 --2- 172.21.15.86:0/2078805503 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555bcabd74b0 0x555bcabe6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f109f400640 0 client.20022 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:37.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.206+0000 7f10c75fc740 1 -- 172.21.15.86:0/2078805503 >> 172.21.15.86:0/2078805503 conn(0x555bcabce610 msgr2=0x555bcabd1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.207+0000 7f10c75fc740 1 -- 172.21.15.86:0/2078805503 shutdown_connections 2024-04-17T07:34:37.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.207+0000 7f10c75fc740 1 -- 172.21.15.86:0/2078805503 wait complete. 2024-04-17T07:34:37.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.207+0000 7f10c75fc740 20 client.20022 trim_cache size 0 max 16384 2024-04-17T07:34:37.207 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:34:37.207 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-17T07:34:37.207 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "aa7d64a1-d693-436c-9418-15aab3f6a09d", 2024-04-17T07:34:37.207 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-17T07:34:37.207 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:34:37.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.207+0000 7f10bf200640 1 -- 172.21.15.86:0/2718432732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f10a40a3bb0 msgr2=0x7f10a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f10a40a3bb0 0x7f10a40a6070 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f10b801c000 tx=0x7f10b801f000 comp rx=0 tx=0).stop 2024-04-17T07:34:37.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 -- 172.21.15.86:0/2718432732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0101e70 msgr2=0x7f10c0198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0101e70 0x7f10c0198170 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f10b0036040 tx=0x7f10b00042f0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 -- 172.21.15.86:0/2718432732 shutdown_connections 2024-04-17T07:34:37.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f10a40a3bb0 0x7f10a40a6070 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10c0103980 0x7f10c019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c0102fc0 0x7f10c01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 --2- 172.21.15.86:0/2718432732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10c0101e70 0x7f10c0198170 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 -- 172.21.15.86:0/2718432732 >> 172.21.15.86:0/2718432732 conn(0x7f10c00fd400 msgr2=0x7f10c01063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 -- 172.21.15.86:0/2718432732 shutdown_connections 2024-04-17T07:34:37.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.208+0000 7f10bf200640 1 -- 172.21.15.86:0/2718432732 wait complete. 2024-04-17T07:34:37.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:34:37.227 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:34:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.447+0000 7f1b39e00640 1 -- 172.21.15.86:0/840530101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b34103a40 msgr2=0x7f1b3410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.447+0000 7f1b39e00640 1 --2- 172.21.15.86:0/840530101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b34103a40 0x7f1b3410a360 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009ca0 tx=0x7f1b2c02c0f0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.447+0000 7f1b39e00640 1 -- 172.21.15.86:0/840530101 shutdown_connections 2024-04-17T07:34:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.447+0000 7f1b39e00640 1 --2- 172.21.15.86:0/840530101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b34103a40 0x7f1b3410a360 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.447+0000 7f1b39e00640 1 --2- 172.21.15.86:0/840530101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b34103080 0x7f1b34103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.447+0000 7f1b39e00640 1 --2- 172.21.15.86:0/840530101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b34101ea0 0x7f1b341022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.447+0000 7f1b39e00640 1 -- 172.21.15.86:0/840530101 >> 172.21.15.86:0/840530101 conn(0x7f1b340fd460 msgr2=0x7f1b340ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.448+0000 7f1b39e00640 1 -- 172.21.15.86:0/840530101 shutdown_connections 2024-04-17T07:34:37.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.448+0000 7f1b39e00640 1 -- 172.21.15.86:0/840530101 wait complete. 2024-04-17T07:34:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.449+0000 7f1b39e00640 1 Processor -- start 2024-04-17T07:34:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.449+0000 7f1b39e00640 1 -- start start 2024-04-17T07:34:37.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.449+0000 7f1b39e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b34101ea0 0x7f1b34075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.449+0000 7f1b39e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b34103080 0x7f1b34074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.449+0000 7f1b39e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b34103a40 0x7f1b34074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.449+0000 7f1b39e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3410d9e0 con 0x7f1b34101ea0 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.449+0000 7f1b39e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3410d860 con 0x7f1b34103080 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.449+0000 7f1b39e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3410db60 con 0x7f1b34103a40 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.449+0000 7f1b38a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b34101ea0 0x7f1b34075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.449+0000 7f1b38a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b34101ea0 0x7f1b34075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43444/0 (socket says 172.21.15.86:43444) 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.450+0000 7f1b38a00640 1 -- 172.21.15.86:0/3033965436 learned_addr learned my addr 172.21.15.86:0/3033965436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.450+0000 7f1b33e00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b34103080 0x7f1b34074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.450+0000 7f1b39400640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b34103a40 0x7f1b34074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.450+0000 7f1b38a00640 1 -- 172.21.15.86:0/3033965436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b34103a40 msgr2=0x7f1b34074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.450+0000 7f1b38a00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b34103a40 0x7f1b34074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.450+0000 7f1b38a00640 1 -- 172.21.15.86:0/3033965436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b34103080 msgr2=0x7f1b34074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.450+0000 7f1b38a00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b34103080 0x7f1b34074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.450+0000 7f1b38a00640 1 -- 172.21.15.86:0/3033965436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b34075120 con 0x7f1b34101ea0 2024-04-17T07:34:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.450+0000 7f1b38a00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b34101ea0 0x7f1b34075d10 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400e9c0 tx=0x7f1b2400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.451+0000 7f1b31600640 1 -- 172.21.15.86:0/3033965436 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400cd80 con 0x7f1b34101ea0 2024-04-17T07:34:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.451+0000 7f1b31600640 1 -- 172.21.15.86:0/3033965436 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b240035f0 con 0x7f1b34101ea0 2024-04-17T07:34:37.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.451+0000 7f1b31600640 1 -- 172.21.15.86:0/3033965436 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24010690 con 0x7f1b34101ea0 2024-04-17T07:34:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.451+0000 7f1b39e00640 1 -- 172.21.15.86:0/3033965436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b341a2170 con 0x7f1b34101ea0 2024-04-17T07:34:37.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.451+0000 7f1b39e00640 1 -- 172.21.15.86:0/3033965436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b341a2670 con 0x7f1b34101ea0 2024-04-17T07:34:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.454+0000 7f1b31600640 1 -- 172.21.15.86:0/3033965436 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1b240037a0 con 0x7f1b34101ea0 2024-04-17T07:34:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.454+0000 7f1b3e1e5740 1 Processor -- start 2024-04-17T07:34:37.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.455+0000 7f1b3e1e5740 1 -- start start 2024-04-17T07:34:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.455+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.455+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 0x55eb1cacc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.455+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cacced0 0x55eb1cacf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.455+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ca2e120 con 0x55eb1cac7370 2024-04-17T07:34:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.455+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eb1c9d4340 con 0x55eb1cac7d60 2024-04-17T07:34:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.455+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ca2a0e0 con 0x55eb1cacced0 2024-04-17T07:34:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.455+0000 7f1b39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.455+0000 7f1b39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43458/0 (socket says 172.21.15.86:43458) 2024-04-17T07:34:37.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.455+0000 7f1b39400640 1 -- 172.21.15.86:0/221011596 learned_addr learned my addr 172.21.15.86:0/221011596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b33e00640 1 --2- 172.21.15.86:0/221011596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cacced0 0x55eb1cacf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b38a00640 1 --2- 172.21.15.86:0/221011596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 0x55eb1cacc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b39400640 1 -- 172.21.15.86:0/221011596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cacced0 msgr2=0x55eb1cacf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b39400640 1 --2- 172.21.15.86:0/221011596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cacced0 0x55eb1cacf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b39400640 1 -- 172.21.15.86:0/221011596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 msgr2=0x55eb1cacc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b39400640 1 --2- 172.21.15.86:0/221011596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 0x55eb1cacc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b39400640 1 -- 172.21.15.86:0/221011596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1c80a7d0 con 0x55eb1cac7370 2024-04-17T07:34:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b38a00640 1 --2- 172.21.15.86:0/221011596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 0x55eb1cacc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:34:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b31600640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1b180a3bb0 0x7f1b180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b39400640 1 --2- 172.21.15.86:0/221011596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac7790 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c032550 tx=0x7f1b2c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7ff9bf43cf0605 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.456+0000 7f1b31600640 1 -- 172.21.15.86:0/3033965436 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1b240d0e60 con 0x7f1b34101ea0 2024-04-17T07:34:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.457+0000 7f1b38a00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1b180a3bb0 0x7f1b180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.457+0000 7f1b29c00640 1 -- 172.21.15.86:0/221011596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c043020 con 0x55eb1cac7370 2024-04-17T07:34:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.457+0000 7f1b29c00640 1 -- 172.21.15.86:0/221011596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b2c032db0 con 0x55eb1cac7370 2024-04-17T07:34:37.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.457+0000 7f1b29c00640 1 -- 172.21.15.86:0/221011596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c039040 con 0x55eb1cac7370 2024-04-17T07:34:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.458+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/221011596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 msgr2=0x55eb1cac7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.458+0000 7f1b38a00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1b180a3bb0 0x7f1b180a6070 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1b240ce950 tx=0x7f1b24003d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.458+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/221011596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac7790 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c032550 tx=0x7f1b2c0093e0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.458+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/221011596 shutdown_connections 2024-04-17T07:34:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.458+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/221011596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cacced0 0x55eb1cacf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.458+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/221011596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 0x55eb1cacc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.458+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/221011596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac7790 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.458+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/221011596 >> 172.21.15.86:0/221011596 conn(0x55eb1cac28e0 msgr2=0x55eb1cac2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.458+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/221011596 shutdown_connections 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.459+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/221011596 wait complete. 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.459+0000 7f1b3e1e5740 1 -- start start 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.459+0000 7f1b3e1e5740 1 Processor -- start 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7370 0x55eb1cad18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7d60 0x55eb1cad1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55eb1cacfe20 con 0x55eb1cacced0 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eb1c9e7710 con 0x55eb1cac7370 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eb1cacffa0 con 0x55eb1cac7d60 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b3e1e5740 10 client.0 _ll_register_callbacks cb 0x55eb1c9e2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43470/0 (socket says 172.21.15.86:43470) 2024-04-17T07:34:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b39400640 1 -- 172.21.15.86:0/1344204522 learned_addr learned my addr 172.21.15.86:0/1344204522 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b39400640 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7370 0x55eb1cad18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.460+0000 7f1b33e00640 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7d60 0x55eb1cad1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.461+0000 7f1b39400640 1 -- 172.21.15.86:0/1344204522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7d60 msgr2=0x55eb1cad1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.461+0000 7f1b39400640 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7d60 0x55eb1cad1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.461+0000 7f1b39400640 1 -- 172.21.15.86:0/1344204522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7370 msgr2=0x55eb1cad18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.461+0000 7f1b39400640 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7370 0x55eb1cad18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.461+0000 7f1b39400640 1 -- 172.21.15.86:0/1344204522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1c7403f0 con 0x55eb1cacced0 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.461+0000 7f1b39400640 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad8e30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c033000 tx=0x7f1b2c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c23ede4e57631 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.461+0000 7f1b22000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.461+0000 7f1b22000640 1 -- 172.21.15.86:0/1344204522 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c04d020 con 0x55eb1cacced0 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.461+0000 7f1b22000640 1 -- 172.21.15.86:0/1344204522 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1b2c0043b0 con 0x55eb1cacced0 2024-04-17T07:34:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.461+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/1344204522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2c032db0 con 0x55eb1cacced0 2024-04-17T07:34:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b22000640 1 -- 172.21.15.86:0/1344204522 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1b2c043030 con 0x55eb1cacced0 2024-04-17T07:34:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b3e1e5740 10 client.20034 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/1344204522 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb1c7000f0 con 0x55eb1cacced0 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b22000640 1 -- 172.21.15.86:0/1344204522 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b2c032db0 con 0x55eb1cacced0 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b22000640 1 -- 172.21.15.86:0/1344204522 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1b2c03fa40 con 0x55eb1cacced0 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b3e1e5740 10 client.20034 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b3e1e5740 10 client.20034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b3e1e5740 20 client.20034 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55eb1cadd290 0x55eb1cadf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b3e1e5740 4 client.20034 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.462+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/1344204522 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb1ca3bb40 con 0x55eb1cadd290 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.463+0000 7f1b33e00640 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55eb1cadd290 0x55eb1cadf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.464+0000 7f1b33e00640 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55eb1cadd290 0x55eb1cadf680 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b67f3c9ddf13bd server_cookie=9ace76e766893fc1 in_seq=0 out_seq=0 2024-04-17T07:34:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.464+0000 7f1b22000640 10 client.20034 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b22000640 1 -- 172.21.15.86:0/1344204522 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb1ca3bb40 con 0x55eb1cadd290 2024-04-17T07:34:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b22000640 10 client.20034 handle_command_reply: tid=0 2024-04-17T07:34:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 client.20034 shutdown 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/1344204522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 msgr2=0x55eb1cad8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad8e30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c033000 tx=0x7f1b2c02e040 comp rx=0 tx=0).stop 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/1344204522 shutdown_connections 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55eb1cadd290 0x55eb1cadf680 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad8e30 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7d60 0x55eb1cad1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/1344204522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7370 0x55eb1cad18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/1344204522 >> 172.21.15.86:0/1344204522 conn(0x55eb1cac28e0 msgr2=0x55eb1cac1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b22000640 0 client.20034 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/1344204522 shutdown_connections 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.468+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/1344204522 wait complete. 2024-04-17T07:34:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.469+0000 7f1b3e1e5740 20 client.20034 trim_cache size 0 max 16384 2024-04-17T07:34:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.490+0000 7f1b3e1e5740 1 Processor -- start 2024-04-17T07:34:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.490+0000 7f1b3e1e5740 1 -- start start 2024-04-17T07:34:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.490+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.490+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7d60 0x55eb1cad8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.490+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cacced0 0x55eb1cac1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.490+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341acad0 con 0x55eb1cac7370 2024-04-17T07:34:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.490+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341ae610 con 0x55eb1cacced0 2024-04-17T07:34:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.490+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341ad1a0 con 0x55eb1cac7d60 2024-04-17T07:34:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.491+0000 7f1b39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.491+0000 7f1b39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43486/0 (socket says 172.21.15.86:43486) 2024-04-17T07:34:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.491+0000 7f1b39400640 1 -- 172.21.15.86:0/2499250114 learned_addr learned my addr 172.21.15.86:0/2499250114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.491+0000 7f1b33e00640 1 --2- 172.21.15.86:0/2499250114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7d60 0x55eb1cad8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.491+0000 7f1b39400640 1 --2- 172.21.15.86:0/2499250114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cacced0 0x55eb1cac1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.491+0000 7f1b39400640 1 -- 172.21.15.86:0/2499250114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7d60 msgr2=0x55eb1cad8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.491+0000 7f1b39400640 1 --2- 172.21.15.86:0/2499250114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7d60 0x55eb1cad8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.491+0000 7f1b39400640 1 -- 172.21.15.86:0/2499250114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cacced0 msgr2=0x55eb1cac1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.491+0000 7f1b39400640 1 --2- 172.21.15.86:0/2499250114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cacced0 0x55eb1cac1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.491+0000 7f1b39400640 1 -- 172.21.15.86:0/2499250114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1cacc5c0 con 0x55eb1cac7370 2024-04-17T07:34:37.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.492+0000 7f1b39400640 1 --2- 172.21.15.86:0/2499250114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac1250 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c03fde0 tx=0x7f1b2c032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b533aa71d131903 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.492+0000 7f1b29200640 1 -- 172.21.15.86:0/2499250114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c04d040 con 0x55eb1cac7370 2024-04-17T07:34:37.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.492+0000 7f1b29200640 1 -- 172.21.15.86:0/2499250114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b2c032e30 con 0x55eb1cac7370 2024-04-17T07:34:37.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.492+0000 7f1b29200640 1 -- 172.21.15.86:0/2499250114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c039070 con 0x55eb1cac7370 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.493+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2499250114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 msgr2=0x55eb1cac1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.493+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/2499250114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac1250 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c03fde0 tx=0x7f1b2c032340 comp rx=0 tx=0).stop 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.493+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2499250114 shutdown_connections 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.493+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/2499250114 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cacced0 0x55eb1cac1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.493+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/2499250114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7d60 0x55eb1cad8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.493+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/2499250114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cac7370 0x55eb1cac1250 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.493+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2499250114 >> 172.21.15.86:0/2499250114 conn(0x55eb1cac28e0 msgr2=0x55eb1cad18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.493+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2499250114 shutdown_connections 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.493+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2499250114 wait complete. 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.494+0000 7f1b3e1e5740 1 -- start start 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.494+0000 7f1b3e1e5740 1 Processor -- start 2024-04-17T07:34:37.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.494+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7370 0x55eb1cad6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.494+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 0x55eb1cad6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.494+0000 7f1b3e1e5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.494+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341adf40 con 0x55eb1cacced0 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.494+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341ad870 con 0x55eb1cac7d60 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.494+0000 7f1b3e1e5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341aaaa0 con 0x55eb1cac7370 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.494+0000 7f1b3e1e5740 10 client.0 _ll_register_callbacks cb 0x7f1b24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.495+0000 7f1b39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.495+0000 7f1b39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43496/0 (socket says 172.21.15.86:43496) 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.495+0000 7f1b39400640 1 -- 172.21.15.86:0/2793583516 learned_addr learned my addr 172.21.15.86:0/2793583516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.495+0000 7f1b39400640 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7370 0x55eb1cad6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.495+0000 7f1b33e00640 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 0x55eb1cad6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.495+0000 7f1b39400640 1 -- 172.21.15.86:0/2793583516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7370 msgr2=0x55eb1cad6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.495+0000 7f1b39400640 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7370 0x55eb1cad6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.495+0000 7f1b39400640 1 -- 172.21.15.86:0/2793583516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 msgr2=0x55eb1cad6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.495+0000 7f1b39400640 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 0x55eb1cad6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.495+0000 7f1b39400640 1 -- 172.21.15.86:0/2793583516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1cad84d0 con 0x55eb1cacced0 2024-04-17T07:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.496+0000 7f1b39400640 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad7ee0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c003d00 tx=0x7f1b2c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9736d5aec569cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.496+0000 7f1b22000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.496+0000 7f1b22000640 1 -- 172.21.15.86:0/2793583516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c039040 con 0x55eb1cacced0 2024-04-17T07:34:37.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.496+0000 7f1b22000640 1 -- 172.21.15.86:0/2793583516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1b2c0041b0 con 0x55eb1cacced0 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.496+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2793583516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2c032e30 con 0x55eb1cacced0 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.496+0000 7f1b22000640 1 -- 172.21.15.86:0/2793583516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1b2c034890 con 0x55eb1cacced0 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.496+0000 7f1b3e1e5740 10 client.20040 fetch_fsmap learned FSMap version 28 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.496+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2793583516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb1cad84d0 con 0x55eb1cacced0 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.497+0000 7f1b22000640 1 -- 172.21.15.86:0/2793583516 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b2c032e30 con 0x55eb1cacced0 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.497+0000 7f1b22000640 1 -- 172.21.15.86:0/2793583516 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1b2c03f920 con 0x55eb1cacced0 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.497+0000 7f1b3e1e5740 10 client.20040 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.497+0000 7f1b3e1e5740 10 client.20040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.497+0000 7f1b3e1e5740 20 client.20040 populate_metadata read hostname 'smithi086' 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.497+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55eb1cb440b0 0x55eb1cb464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.497+0000 7f1b3e1e5740 4 client.20040 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.497+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2793583516 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb1cabe140 con 0x55eb1cb440b0 2024-04-17T07:34:37.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.498+0000 7f1b33e00640 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55eb1cb440b0 0x55eb1cb464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:34:37.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.499+0000 7f1b33e00640 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55eb1cb440b0 0x55eb1cb464d0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8271a5addca37ac server_cookie=c7deae97b9b95bcc in_seq=0 out_seq=0 2024-04-17T07:34:37.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.499+0000 7f1b22000640 10 client.20040 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:37.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b22000640 1 -- 172.21.15.86:0/2793583516 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55eb1cabe140 con 0x55eb1cb440b0 2024-04-17T07:34:37.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b22000640 10 client.20040 handle_command_reply: tid=0 2024-04-17T07:34:37.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b3e1e5740 1 client.20040 shutdown 2024-04-17T07:34:37.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2793583516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 msgr2=0x55eb1cad7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad7ee0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c003d00 tx=0x7f1b2c005cf0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2793583516 shutdown_connections 2024-04-17T07:34:37.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55eb1cb440b0 0x55eb1cb464d0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55eb1cacced0 0x55eb1cad7ee0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb1cac7d60 0x55eb1cad6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b3e1e5740 1 --2- 172.21.15.86:0/2793583516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eb1cac7370 0x55eb1cad6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b22000640 0 client.20040 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:34:37.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.502+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2793583516 >> 172.21.15.86:0/2793583516 conn(0x55eb1cac28e0 msgr2=0x55eb1cac5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.503+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2793583516 shutdown_connections 2024-04-17T07:34:37.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.503+0000 7f1b3e1e5740 1 -- 172.21.15.86:0/2793583516 wait complete. 2024-04-17T07:34:37.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.503+0000 7f1b3e1e5740 20 client.20040 trim_cache size 0 max 16384 2024-04-17T07:34:37.503 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:34:37.503 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "scrub active (676 inodes in the stack)", 2024-04-17T07:34:37.503 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-17T07:34:37.503 INFO:teuthology.orchestra.run.smithi086.stdout: "aa7d64a1-d693-436c-9418-15aab3f6a09d": { 2024-04-17T07:34:37.503 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-17T07:34:37.504 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "aa7d64a1-d693-436c-9418-15aab3f6a09d", 2024-04-17T07:34:37.504 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-17T07:34:37.504 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:34:37.504 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:34:37.504 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:34:37.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.503+0000 7f1b39e00640 1 -- 172.21.15.86:0/3033965436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1b180a3bb0 msgr2=0x7f1b180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.503+0000 7f1b39e00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1b180a3bb0 0x7f1b180a6070 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1b240ce950 tx=0x7f1b24003d00 comp rx=0 tx=0).stop 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.503+0000 7f1b39e00640 1 -- 172.21.15.86:0/3033965436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b34101ea0 msgr2=0x7f1b34075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.503+0000 7f1b39e00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b34101ea0 0x7f1b34075d10 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400e9c0 tx=0x7f1b2400ee80 comp rx=0 tx=0).stop 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.504+0000 7f1b39e00640 1 -- 172.21.15.86:0/3033965436 shutdown_connections 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.504+0000 7f1b39e00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1b180a3bb0 0x7f1b180a6070 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.504+0000 7f1b39e00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b34103a40 0x7f1b34074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.504+0000 7f1b39e00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b34103080 0x7f1b34074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.504+0000 7f1b39e00640 1 --2- 172.21.15.86:0/3033965436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b34101ea0 0x7f1b34075d10 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.504+0000 7f1b39e00640 1 -- 172.21.15.86:0/3033965436 >> 172.21.15.86:0/3033965436 conn(0x7f1b340fd460 msgr2=0x7f1b34106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.504+0000 7f1b39e00640 1 -- 172.21.15.86:0/3033965436 shutdown_connections 2024-04-17T07:34:37.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:34:37.504+0000 7f1b39e00640 1 -- 172.21.15.86:0/3033965436 wait complete. 2024-04-17T07:34:37.521 INFO:tasks.cephfs.filesystem:scrub status for tag:aa7d64a1-d693-436c-9418-15aab3f6a09d - {'path': '/', 'tag': 'aa7d64a1-d693-436c-9418-15aab3f6a09d', 'options': 'recursive,force'} 2024-04-17T07:34:37.526 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:37 smithi103 ceph-mon[35495]: pgmap v1047: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 957 op/s 2024-04-17T07:34:37.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:37 smithi086 ceph-mon[31733]: pgmap v1047: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 957 op/s 2024-04-17T07:34:37.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:37 smithi098 ceph-mon[35418]: pgmap v1047: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 957 op/s 2024-04-17T07:34:38.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-04-17T07:34:38.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:38 smithi103 ceph-mon[35495]: scrub queued for path: / 2024-04-17T07:34:38.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:38 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:38.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:38 smithi103 ceph-mon[35495]: scrub summary: active paths [/] 2024-04-17T07:34:38.527 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:38 smithi103 ceph-mon[35495]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:38.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:38 smithi086 ceph-mon[31733]: scrub queued for path: / 2024-04-17T07:34:38.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:38 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:38.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:38 smithi086 ceph-mon[31733]: scrub summary: active paths [/] 2024-04-17T07:34:38.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:38 smithi086 ceph-mon[31733]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:38 smithi098 ceph-mon[35418]: scrub queued for path: / 2024-04-17T07:34:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:38 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:38 smithi098 ceph-mon[35418]: scrub summary: active paths [/] 2024-04-17T07:34:38.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:38 smithi098 ceph-mon[35418]: scrub summary: idle+waiting paths [/] 2024-04-17T07:34:39.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:39 smithi086 ceph-mon[31733]: pgmap v1048: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.1 MiB/s wr, 870 op/s 2024-04-17T07:34:39.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:39 smithi086 ceph-mon[31733]: scrub summary: idle 2024-04-17T07:34:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:39 smithi098 ceph-mon[35418]: pgmap v1048: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.1 MiB/s wr, 870 op/s 2024-04-17T07:34:39.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:39 smithi098 ceph-mon[35418]: scrub summary: idle 2024-04-17T07:34:39.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:39 smithi103 ceph-mon[35495]: pgmap v1048: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.1 MiB/s wr, 870 op/s 2024-04-17T07:34:39.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:39 smithi103 ceph-mon[35495]: scrub summary: idle 2024-04-17T07:34:40.553 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/: 2024-04-17T07:34:40.553 INFO:teuthology.orchestra.run.smithi086.stdout:total 2 2024-04-17T07:34:40.553 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1515689103 Apr 17 07:15 . 2024-04-17T07:34:40.553 INFO:teuthology.orchestra.run.smithi086.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1515689103 Apr 17 07:15 .. 2024-04-17T07:34:40.553 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 data 2024-04-17T07:34:40.553 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:40.553 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data: 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:total 46 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 . 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1515689103 Apr 17 07:15 .. 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 591787573 Apr 17 07:19 base 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Apr 17 07:16 current_logfiles 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 Apr 17 07:16 global 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1047 Apr 17 07:16 log 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_commit_ts 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_dynshmem 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Apr 17 07:15 pg_hba.conf 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Apr 17 07:15 pg_ident.conf 2024-04-17T07:34:40.554 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 17 07:34 pg_logical 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 pg_multixact 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_notify 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_replslot 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_serial 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_snapshots 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 Apr 17 07:34 pg_stat 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:34 pg_stat_tmp 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 17 07:34 pg_subtrans 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_tblspc 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 pg_twophase 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 17 07:15 PG_VERSION 2024-04-17T07:34:40.555 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 939524096 Apr 17 07:34 pg_wal 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 139264 Apr 17 07:15 pg_xact 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Apr 17 07:15 postgresql.auto.conf 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Apr 17 07:15 postgresql.conf 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Apr 17 07:16 postmaster.opts 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/base: 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:total 3 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 591787573 Apr 17 07:19 . 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 17 07:15 1 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 17 07:15 13493 2024-04-17T07:34:40.556 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 575861295 Apr 17 07:22 13494 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:19 pgsql_tmp 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/base/1: 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:total 7778 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 17 07:15 . 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 591787573 Apr 17 07:19 .. 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 112 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 113 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:15 1247 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1247_fsm 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1247_vm 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 17 07:15 1249 2024-04-17T07:34:40.557 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1249_fsm 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1249_vm 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 17 07:15 1255 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1255_fsm 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1255_vm 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 17 07:15 1259 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1259_fsm 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1259_vm 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 17 07:15 13344 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13344_fsm 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13344_vm 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13346 2024-04-17T07:34:40.558 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13348 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13349_fsm 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349_vm 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13351 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13353 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13354_fsm 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354_vm 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13356 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13358 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359 2024-04-17T07:34:40.559 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13359_fsm 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359_vm 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13361 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13363 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1417 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1418 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 174 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 175 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2187 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2224 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2328 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2336 2024-04-17T07:34:40.560 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2337 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2579 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2600 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2600_fsm 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2600_vm 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2601_fsm 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601_vm 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2602 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2602_fsm 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2602_vm 2024-04-17T07:34:40.561 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2603 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2603_fsm 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2603_vm 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2604 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2605 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2605_fsm 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2605_vm 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2606 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2606_fsm 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2606_vm 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2607 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2607_fsm 2024-04-17T07:34:40.562 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2607_vm 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 17 07:15 2608 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2608_fsm 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2608_vm 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 17 07:15 2609 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2609_fsm 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2609_vm 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2610 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2610_fsm 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2610_vm 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2611 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612 2024-04-17T07:34:40.563 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2612_fsm 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612_vm 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2613 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2615_fsm 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615_vm 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2616 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2616_fsm 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2616_vm 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 17 07:15 2617 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2617_fsm 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2617_vm 2024-04-17T07:34:40.564 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 17 07:15 2618 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2618_fsm 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2618_vm 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Apr 17 07:15 2619 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2619_fsm 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2619_vm 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2620 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2650 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2651 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2652 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2653 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2654 2024-04-17T07:34:40.565 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2655 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2656 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2657 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 17 07:15 2658 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:15 2659 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2660 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2661 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2662 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2663 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2664 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2665 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2666 2024-04-17T07:34:40.566 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2667 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2668 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2669 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2670 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 17 07:15 2673 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 17 07:15 2674 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 17 07:15 2675 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2678 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2679 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2680 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2681 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2682 2024-04-17T07:34:40.567 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2683 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2684 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2685 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2686 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2687 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2688 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2689 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 17 07:15 2690 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 17 07:15 2691 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2692 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2693 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2696 2024-04-17T07:34:40.568 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2699 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2701 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2702 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2703 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2704 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2753 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2753_fsm 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2753_vm 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2754 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2755 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2756 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2757 2024-04-17T07:34:40.569 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2830 2024-04-17T07:34:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2831 2024-04-17T07:34:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2832 2024-04-17T07:34:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2833 2024-04-17T07:34:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2834 2024-04-17T07:34:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2835 2024-04-17T07:34:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2836 2024-04-17T07:34:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2837 2024-04-17T07:34:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 17 07:15 2838 2024-04-17T07:34:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2838_fsm 2024-04-17T07:34:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2838_vm 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2839 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840_fsm 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2840_vm 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2841 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2995 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2996 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3079_fsm 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079_vm 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3080 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3081 2024-04-17T07:34:40.571 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3085 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3118 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3119 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 17 07:15 3164 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3256 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3257 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3258 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3350 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3351 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3379 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3380 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3381 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3394 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3394_fsm 2024-04-17T07:34:40.572 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3394_vm 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3395 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3429 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3430 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3431 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3433 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3439 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3440 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3455 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 17 07:15 3456 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3456_fsm 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3456_vm 2024-04-17T07:34:40.573 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3466 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3467 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3468 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3501 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3502 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3503 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3534 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3541_fsm 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541_vm 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3542 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3574 2024-04-17T07:34:40.574 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3575 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3576 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3596 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3597 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3598 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3599 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3600_fsm 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600_vm 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3601_fsm 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601_vm 2024-04-17T07:34:40.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3602_fsm 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602_vm 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603_fsm 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3603_vm 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3604 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3605 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3606 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3607 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3608 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 3609 2024-04-17T07:34:40.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3712 2024-04-17T07:34:40.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764 2024-04-17T07:34:40.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3764_fsm 2024-04-17T07:34:40.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764_vm 2024-04-17T07:34:40.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3766 2024-04-17T07:34:40.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3767 2024-04-17T07:34:40.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3997 2024-04-17T07:34:40.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4143 2024-04-17T07:34:40.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4144 2024-04-17T07:34:40.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4145 2024-04-17T07:34:40.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4146 2024-04-17T07:34:40.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4147 2024-04-17T07:34:40.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4148 2024-04-17T07:34:40.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4149 2024-04-17T07:34:40.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4150 2024-04-17T07:34:40.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4151 2024-04-17T07:34:41.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:41 smithi086 ceph-mon[31733]: pgmap v1049: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.7 MiB/s wr, 721 op/s 2024-04-17T07:34:41.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:41 smithi098 ceph-mon[35418]: pgmap v1049: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.7 MiB/s wr, 721 op/s 2024-04-17T07:34:41.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:41 smithi103 ceph-mon[35495]: pgmap v1049: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.7 MiB/s wr, 721 op/s 2024-04-17T07:34:42.224 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4152 2024-04-17T07:34:42.224 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4153 2024-04-17T07:34:42.224 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4154 2024-04-17T07:34:42.224 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4155 2024-04-17T07:34:42.224 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4156 2024-04-17T07:34:42.224 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4157 2024-04-17T07:34:42.224 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4158 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4159 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4160 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4161 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4162 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4163 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4164 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4165 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4166 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4167 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4168 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4169 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4170 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4171 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4172 2024-04-17T07:34:42.225 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4173 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4174 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 5002 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 548 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 549 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6102 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6104 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6106 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6110 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6111 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6112 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6113 2024-04-17T07:34:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6117 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 826 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 827 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 828 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 17 07:15 pg_filenode.map 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 17 07:15 PG_VERSION 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:total 7778 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Apr 17 07:15 . 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 591787573 Apr 17 07:19 .. 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 112 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 113 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:15 1247 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1247_fsm 2024-04-17T07:34:42.227 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1247_vm 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 17 07:15 1249 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1249_fsm 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1249_vm 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 17 07:15 1255 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1255_fsm 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1255_vm 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 17 07:15 1259 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1259_fsm 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1259_vm 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 17 07:15 13344 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13344_fsm 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13344_vm 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13346 2024-04-17T07:34:42.228 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13348 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13349_fsm 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349_vm 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13351 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13353 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13354_fsm 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354_vm 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13356 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13358 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13359_fsm 2024-04-17T07:34:42.229 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359_vm 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13361 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13363 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1417 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1418 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 174 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 175 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2187 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2224 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2328 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2336 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2337 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2579 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2600 2024-04-17T07:34:42.230 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2600_fsm 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2600_vm 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2601_fsm 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601_vm 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2602 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2602_fsm 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2602_vm 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2603 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2603_fsm 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2603_vm 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2604 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2605 2024-04-17T07:34:42.231 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2605_fsm 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2605_vm 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2606 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2606_fsm 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2606_vm 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2607 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2607_fsm 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2607_vm 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 17 07:15 2608 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2608_fsm 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2608_vm 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 17 07:15 2609 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2609_fsm 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2609_vm 2024-04-17T07:34:42.232 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2610 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2610_fsm 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2610_vm 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2611 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2612_fsm 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612_vm 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2613 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2615_fsm 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615_vm 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2616 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2616_fsm 2024-04-17T07:34:42.233 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2616_vm 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 17 07:15 2617 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2617_fsm 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2617_vm 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 17 07:15 2618 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2618_fsm 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2618_vm 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Apr 17 07:15 2619 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2619_fsm 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2619_vm 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2620 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2650 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2651 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2652 2024-04-17T07:34:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2653 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2654 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2655 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2656 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2657 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 17 07:15 2658 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:15 2659 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2660 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2661 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2662 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2663 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2664 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2665 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2666 2024-04-17T07:34:42.235 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2667 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2668 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2669 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2670 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 17 07:15 2673 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 17 07:15 2674 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 17 07:15 2675 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2678 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2679 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2680 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2681 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2682 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2683 2024-04-17T07:34:42.236 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2684 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2685 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2686 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2687 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2688 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2689 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 17 07:15 2690 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 17 07:15 2691 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2692 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2693 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2696 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2699 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2701 2024-04-17T07:34:42.237 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2702 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2703 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2704 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2753 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2753_fsm 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2753_vm 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2754 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2755 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2756 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2757 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2830 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2831 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2832 2024-04-17T07:34:42.238 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2833 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2834 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2835 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2836 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2837 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 17 07:15 2838 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2838_fsm 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2838_vm 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2839 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840_fsm 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2840_vm 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2841 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2995 2024-04-17T07:34:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2996 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3079_fsm 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079_vm 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3080 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3081 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3085 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3118 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3119 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 17 07:15 3164 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3256 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3257 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3258 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3350 2024-04-17T07:34:42.240 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3351 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3379 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3380 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3381 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3394 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3394_fsm 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3394_vm 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3395 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3429 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3430 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3431 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3433 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3439 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3440 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3455 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 17 07:15 3456 2024-04-17T07:34:42.241 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3456_fsm 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3456_vm 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3466 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3467 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3468 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3501 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3502 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3503 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3534 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3541_fsm 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541_vm 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3542 2024-04-17T07:34:42.242 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3574 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3575 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3576 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3596 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3597 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3598 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3599 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3600_fsm 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600_vm 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3601_fsm 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601_vm 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3602_fsm 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602_vm 2024-04-17T07:34:42.243 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603_fsm 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3603_vm 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3604 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3605 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3606 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3607 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3608 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 3609 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3712 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3764_fsm 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764_vm 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3766 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3767 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3997 2024-04-17T07:34:42.244 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4143 2024-04-17T07:34:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4144 2024-04-17T07:34:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4145 2024-04-17T07:34:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4146 2024-04-17T07:34:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4147 2024-04-17T07:34:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4148 2024-04-17T07:34:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4149 2024-04-17T07:34:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4150 2024-04-17T07:34:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4151 2024-04-17T07:34:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4152 2024-04-17T07:34:42.245 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4153 2024-04-17T07:34:43.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:43 smithi086 ceph-mon[31733]: pgmap v1050: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.1 MiB/s wr, 540 op/s 2024-04-17T07:34:43.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:43 smithi098 ceph-mon[35418]: pgmap v1050: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.1 MiB/s wr, 540 op/s 2024-04-17T07:34:43.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:43 smithi103 ceph-mon[35495]: pgmap v1050: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.1 MiB/s wr, 540 op/s 2024-04-17T07:34:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4154 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4155 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4156 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4157 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4158 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4159 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4160 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4161 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4162 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4163 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4164 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4165 2024-04-17T07:34:44.152 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4166 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4167 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4168 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4169 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4170 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4171 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4172 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4173 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4174 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 5002 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 548 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 549 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6102 2024-04-17T07:34:44.153 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6104 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6106 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6110 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6111 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6112 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6113 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6117 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 826 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 827 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 828 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 17 07:15 pg_filenode.map 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 17 07:15 PG_VERSION 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.154 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:total 562367 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 575861295 Apr 17 07:22 . 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 591787573 Apr 17 07:19 .. 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 112 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 113 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:19 1247 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1247_fsm 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:19 1247_vm 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Apr 17 07:19 1249 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:19 1249_fsm 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:19 1249_vm 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Apr 17 07:15 1255 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1255_fsm 2024-04-17T07:34:44.155 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1255_vm 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Apr 17 07:34 1259 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1259_fsm 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:19 1259_vm 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Apr 17 07:15 13344 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13344_fsm 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13344_vm 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13346 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13348 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13349_fsm 2024-04-17T07:34:44.156 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13349_vm 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13351 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13353 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13354_fsm 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13354_vm 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13356 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13358 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 13359_fsm 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13359_vm 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 13361 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 13363 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1417 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 1418 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 451076096 Apr 17 07:34 16396 2024-04-17T07:34:44.157 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Apr 17 07:33 16396_fsm 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:34 16396_vm 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 7880704 Apr 17 07:34 16397 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:34 16397_fsm 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:34 16397_vm 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 6455296 Apr 17 07:34 16399 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:34 16399_fsm 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:34 16399_vm 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:34 16400 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 17 07:34 16402 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Apr 17 07:34 16404 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30040064 Apr 17 07:34 16406 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:34 16406_fsm 2024-04-17T07:34:44.158 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:32 16406_vm 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 174 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 175 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2187 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2224 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2328 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2336 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2337 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:19 2579 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2600 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2600_fsm 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2600_vm 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2601_fsm 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2601_vm 2024-04-17T07:34:44.159 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2602 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2602_fsm 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2602_vm 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2603 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2603_fsm 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2603_vm 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2604 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2605 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2605_fsm 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2605_vm 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:19 2606 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2606_fsm 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:19 2606_vm 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2607 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2607_fsm 2024-04-17T07:34:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2607_vm 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Apr 17 07:19 2608 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2608_fsm 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:19 2608_vm 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Apr 17 07:15 2609 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2609_fsm 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2609_vm 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:19 2610 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2610_fsm 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:19 2610_vm 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2611 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2612_fsm 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2612_vm 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2613 2024-04-17T07:34:44.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2615_fsm 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2615_vm 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2616 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2616_fsm 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2616_vm 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 17 07:15 2617 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2617_fsm 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2617_vm 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Apr 17 07:15 2618 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2618_fsm 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2618_vm 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Apr 17 07:34 2619 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:32 2619_fsm 2024-04-17T07:34:44.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:19 2619_vm 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2620 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2650 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2651 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2652 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2653 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2654 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2655 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2656 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2657 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Apr 17 07:19 2658 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Apr 17 07:19 2659 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2660 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2661 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:19 2662 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:19 2663 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:19 2664 2024-04-17T07:34:44.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:19 2665 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:19 2666 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:19 2667 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2668 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2669 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2670 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Apr 17 07:19 2673 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Apr 17 07:19 2674 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Apr 17 07:15 2675 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:19 2678 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:19 2679 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2680 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2681 2024-04-17T07:34:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2682 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2683 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2684 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2685 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2686 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2687 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 2688 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Apr 17 07:15 2689 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Apr 17 07:15 2690 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Apr 17 07:15 2691 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2692 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2693 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:33 2696 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2699 2024-04-17T07:34:44.165 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2701 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2702 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:19 2703 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:19 2704 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2753 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2753_fsm 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2753_vm 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2754 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2755 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2756 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 2757 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2830 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2831 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2832 2024-04-17T07:34:44.166 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2833 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2834 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2835 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2836 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2837 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Apr 17 07:15 2838 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2838_fsm 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2838_vm 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2839 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2840_fsm 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2840_vm 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2841 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2995 2024-04-17T07:34:44.167 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2996 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3079_fsm 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3079_vm 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3080 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3081 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:15 3085 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3118 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3119 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Apr 17 07:15 3164 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3256 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3257 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3258 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3350 2024-04-17T07:34:44.168 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3351 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3379 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3380 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3381 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3394 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3394_fsm 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3394_vm 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3395 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3429 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3430 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3431 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3433 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3439 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3440 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Apr 17 07:19 3455 2024-04-17T07:34:44.169 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Apr 17 07:15 3456 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3456_fsm 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3456_vm 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3466 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3467 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3468 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3501 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3502 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3503 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3534 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3541_fsm 2024-04-17T07:34:44.170 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3541_vm 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3542 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3574 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3575 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3576 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3596 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3597 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3598 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3599 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3600_fsm 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3600_vm 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3601_fsm 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3601_vm 2024-04-17T07:34:44.171 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3602_fsm 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3602_vm 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3603_fsm 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3603_vm 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3604 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3605 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3606 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3607 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3608 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Apr 17 07:15 3609 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3712 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 3764_fsm 2024-04-17T07:34:44.172 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3764_vm 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3766 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 3767 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3997 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4143 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4144 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4145 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4146 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4147 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4148 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4149 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4150 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4151 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4152 2024-04-17T07:34:44.173 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4153 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4154 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4155 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4156 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4157 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4158 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4159 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4160 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4161 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4162 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4163 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4164 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4165 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4166 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4167 2024-04-17T07:34:44.174 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4168 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4169 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4170 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4171 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4172 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4173 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4174 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 5002 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 548 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 549 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6102 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6104 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6106 2024-04-17T07:34:44.175 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6110 2024-04-17T07:34:44.487 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6111 2024-04-17T07:34:44.487 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6112 2024-04-17T07:34:44.487 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6113 2024-04-17T07:34:44.487 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6117 2024-04-17T07:34:44.487 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 826 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 827 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 828 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 17 07:15 pg_filenode.map 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Apr 17 07:16 pg_internal.init 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Apr 17 07:15 PG_VERSION 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:19 . 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 591787573 Apr 17 07:19 .. 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/global: 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:total 561 2024-04-17T07:34:44.488 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 Apr 17 07:16 . 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1213 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1213_fsm 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1213_vm 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1214 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1214_fsm 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1214_vm 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 1232 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 1233 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1260 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1260_fsm 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1260_vm 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1261 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1261_fsm 2024-04-17T07:34:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1261_vm 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1262 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 1262_fsm 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 1262_vm 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2396 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Apr 17 07:15 2396_fsm 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2396_vm 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2397 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2671 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2672 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2676 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2677 2024-04-17T07:34:44.490 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2694 2024-04-17T07:34:44.575 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2695 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2697 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 2698 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2846 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2847 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2964 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2965 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 2966 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 2967 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 3592 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 3593 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4060 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4061 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4175 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4176 2024-04-17T07:34:44.576 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4177 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4178 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4181 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4182 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4183 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4184 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 4185 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 4186 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6000 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6001 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6002 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 6100 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6114 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 6115 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:34 pg_control 2024-04-17T07:34:44.577 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Apr 17 07:15 pg_filenode.map 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Apr 17 07:16 pg_internal.init 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/log: 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:total 3 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 Apr 17 07:16 . 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 Apr 17 07:34 postgresql-Wed.log 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.578 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-04-17T07:34:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.579 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-04-17T07:34:44.579 INFO:teuthology.orchestra.run.smithi086.stdout:total 3 2024-04-17T07:34:44.816 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 17 07:34 . 2024-04-17T07:34:44.816 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.816 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 mappings 2024-04-17T07:34:44.816 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 17 07:34 replorigin_checkpoint 2024-04-17T07:34:44.816 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 snapshots 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 17 07:34 .. 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Apr 17 07:34 .. 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:total 2 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 . 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.817 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 members 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 offsets 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:total 9 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 . 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 .. 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 0000 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:total 9 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:15 . 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Apr 17 07:15 .. 2024-04-17T07:34:44.818 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Apr 17 07:21 0000 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:44.819 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:total 16 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 Apr 17 07:34 . 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Apr 17 07:34 db_0.stat 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Apr 17 07:34 db_13494.stat 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Apr 17 07:34 global.stat 2024-04-17T07:34:44.820 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.821 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-04-17T07:34:44.821 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:44.821 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:34 . 2024-04-17T07:34:44.821 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:44.821 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:44.821 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-04-17T07:34:44.821 INFO:teuthology.orchestra.run.smithi086.stdout:total 113 2024-04-17T07:34:44.821 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 17 07:34 . 2024-04-17T07:34:44.821 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:45.161 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Apr 17 07:34 0008 2024-04-17T07:34:45.161 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:45.161 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-04-17T07:34:45.161 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:45.161 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:45.161 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:45.161 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:45.161 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-04-17T07:34:45.161 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:45.161 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:total 917506 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 939524096 Apr 17 07:34 . 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:34 00000001000000000000007C 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 00000001000000000000007D 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:28 00000001000000000000007E 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 00000001000000000000007F 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 000000010000000000000080 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:30 000000010000000000000081 2024-04-17T07:34:45.162 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:28 000000010000000000000082 2024-04-17T07:34:45.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 000000010000000000000083 2024-04-17T07:34:45.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:27 000000010000000000000084 2024-04-17T07:34:45.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:29 000000010000000000000085 2024-04-17T07:34:45.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 000000010000000000000086 2024-04-17T07:34:45.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 000000010000000000000087 2024-04-17T07:34:45.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:28 000000010000000000000088 2024-04-17T07:34:45.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:27 000000010000000000000089 2024-04-17T07:34:45.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:29 00000001000000000000008A 2024-04-17T07:34:45.163 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:30 00000001000000000000008B 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 00000001000000000000008C 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:27 00000001000000000000008D 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:27 00000001000000000000008E 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:27 00000001000000000000008F 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:27 000000010000000000000090 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:28 000000010000000000000091 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 000000010000000000000092 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 000000010000000000000093 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 000000010000000000000094 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:26 000000010000000000000095 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 000000010000000000000096 2024-04-17T07:34:45.164 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 000000010000000000000097 2024-04-17T07:34:45.199 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 000000010000000000000098 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:32 000000010000000000000099 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:33 00000001000000000000009A 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:33 00000001000000000000009B 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:32 00000001000000000000009C 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 00000001000000000000009D 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:33 00000001000000000000009E 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 00000001000000000000009F 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 0000000100000000000000A0 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 0000000100000000000000A1 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:32 0000000100000000000000A2 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 0000000100000000000000A3 2024-04-17T07:34:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 0000000100000000000000A4 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 0000000100000000000000A5 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:32 0000000100000000000000A6 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:32 0000000100000000000000A7 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 0000000100000000000000A8 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 0000000100000000000000A9 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 0000000100000000000000AA 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:32 0000000100000000000000AB 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 0000000100000000000000AC 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:32 0000000100000000000000AD 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:31 0000000100000000000000AE 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:33 0000000100000000000000AF 2024-04-17T07:34:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:32 0000000100000000000000B0 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:34 0000000100000000000000B1 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:33 0000000100000000000000B2 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Apr 17 07:32 0000000100000000000000B3 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 archive_status 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:total 1 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Apr 17 07:15 . 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 939524096 Apr 17 07:34 .. 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:total 137 2024-04-17T07:34:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 139264 Apr 17 07:15 . 2024-04-17T07:34:45.203 INFO:teuthology.orchestra.run.smithi086.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1532204068 Apr 17 07:34 .. 2024-04-17T07:34:45.203 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 139264 Apr 17 07:34 0000 2024-04-17T07:34:45.209 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-17T07:34:45.220 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-17T07:34:45.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:45 smithi086 ceph-mon[31733]: pgmap v1051: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.7 MiB/s wr, 293 op/s 2024-04-17T07:34:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:45 smithi098 ceph-mon[35418]: pgmap v1051: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.7 MiB/s wr, 293 op/s 2024-04-17T07:34:45.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:45 smithi103 ceph-mon[35495]: pgmap v1051: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.7 MiB/s wr, 293 op/s 2024-04-17T07:34:47.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:47 smithi086 ceph-mon[31733]: pgmap v1052: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-04-17T07:34:47.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:47 smithi098 ceph-mon[35418]: pgmap v1052: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-04-17T07:34:47.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:47 smithi103 ceph-mon[35495]: pgmap v1052: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-04-17T07:34:49.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:49 smithi086 ceph-mon[31733]: pgmap v1053: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 336 KiB/s wr, 37 op/s 2024-04-17T07:34:49.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:49 smithi098 ceph-mon[35418]: pgmap v1053: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 336 KiB/s wr, 37 op/s 2024-04-17T07:34:49.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:49 smithi103 ceph-mon[35495]: pgmap v1053: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 336 KiB/s wr, 37 op/s 2024-04-17T07:34:51.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:51 smithi086 ceph-mon[31733]: pgmap v1054: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-17T07:34:51.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:51 smithi098 ceph-mon[35418]: pgmap v1054: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-17T07:34:51.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:51 smithi103 ceph-mon[35495]: pgmap v1054: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-04-17T07:34:53.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:53 smithi098 ceph-mon[35418]: pgmap v1055: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T07:34:53.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:53 smithi103 ceph-mon[35495]: pgmap v1055: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T07:34:53.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:53 smithi086 ceph-mon[31733]: pgmap v1055: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-17T07:34:55.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:55 smithi098 ceph-mon[35418]: pgmap v1056: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-17T07:34:55.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:55 smithi103 ceph-mon[35495]: pgmap v1056: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-17T07:34:55.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:55 smithi086 ceph-mon[31733]: pgmap v1056: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-17T07:34:56.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:56 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:34:56.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:56 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:34:56.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:56 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:34:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:57 smithi098 ceph-mon[35418]: pgmap v1057: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-04-17T07:34:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:34:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:34:57.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:34:57.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:57 smithi103 ceph-mon[35495]: pgmap v1057: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-04-17T07:34:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:34:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:34:57.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:34:57.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:57 smithi086 ceph-mon[31733]: pgmap v1057: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-04-17T07:34:57.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:34:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:34:57.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:34:59.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:34:59 smithi098 ceph-mon[35418]: pgmap v1058: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-17T07:34:59.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:34:59 smithi103 ceph-mon[35495]: pgmap v1058: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-17T07:34:59.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:34:59 smithi086 ceph-mon[31733]: pgmap v1058: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-17T07:35:01.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:01 smithi098 ceph-mon[35418]: pgmap v1059: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-17T07:35:01.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:01 smithi103 ceph-mon[35495]: pgmap v1059: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-17T07:35:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:01 smithi086 ceph-mon[31733]: pgmap v1059: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-17T07:35:03.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:03 smithi098 ceph-mon[35418]: pgmap v1060: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-17T07:35:03.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:03 smithi103 ceph-mon[35495]: pgmap v1060: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-17T07:35:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:03 smithi086 ceph-mon[31733]: pgmap v1060: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-17T07:35:05.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:05 smithi098 ceph-mon[35418]: pgmap v1061: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-17T07:35:05.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:05 smithi103 ceph-mon[35495]: pgmap v1061: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-17T07:35:05.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:05 smithi086 ceph-mon[31733]: pgmap v1061: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-17T07:35:07.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-17T07:35:07.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-17T07:35:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.748+0000 7f1f16600640 1 -- 172.21.15.86:0/236563106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1010a050 msgr2=0x7f1f101108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.748+0000 7f1f16600640 1 --2- 172.21.15.86:0/236563106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1010a050 0x7f1f101108e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009ca0 tx=0x7f1f0802c100 comp rx=0 tx=0).stop 2024-04-17T07:35:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.748+0000 7f1f16600640 1 -- 172.21.15.86:0/236563106 shutdown_connections 2024-04-17T07:35:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.748+0000 7f1f16600640 1 --2- 172.21.15.86:0/236563106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1010a050 0x7f1f101108e0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.748+0000 7f1f16600640 1 --2- 172.21.15.86:0/236563106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f10109690 0x7f1f10109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.748+0000 7f1f16600640 1 --2- 172.21.15.86:0/236563106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f101084a0 0x7f1f101088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.748+0000 7f1f16600640 1 -- 172.21.15.86:0/236563106 >> 172.21.15.86:0/236563106 conn(0x7f1f10077430 msgr2=0x7f1f10079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.748+0000 7f1f16600640 1 -- 172.21.15.86:0/236563106 shutdown_connections 2024-04-17T07:35:07.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.748+0000 7f1f16600640 1 -- 172.21.15.86:0/236563106 wait complete. 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.749+0000 7f1f16600640 1 Processor -- start 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.749+0000 7f1f16600640 1 -- start start 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f16600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f101084a0 0x7f1f1019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f16600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f10109690 0x7f1f1019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f16600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1010a050 0x7f1f101a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f16600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f10113fe0 con 0x7f1f1010a050 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f16600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f10113e60 con 0x7f1f101084a0 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f16600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f10114160 con 0x7f1f10109690 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f15c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1010a050 0x7f1f101a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f15c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1010a050 0x7f1f101a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51084/0 (socket says 172.21.15.86:51084) 2024-04-17T07:35:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f15c00640 1 -- 172.21.15.86:0/3520478494 learned_addr learned my addr 172.21.15.86:0/3520478494 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f15200640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f101084a0 0x7f1f1019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f07e00640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f10109690 0x7f1f1019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f15c00640 1 -- 172.21.15.86:0/3520478494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f10109690 msgr2=0x7f1f1019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f15c00640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f10109690 0x7f1f1019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f15c00640 1 -- 172.21.15.86:0/3520478494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f101084a0 msgr2=0x7f1f1019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f15c00640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f101084a0 0x7f1f1019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f15c00640 1 -- 172.21.15.86:0/3520478494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f101a6220 con 0x7f1f1010a050 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.750+0000 7f1f07e00640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f10109690 0x7f1f1019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.751+0000 7f1f15c00640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1010a050 0x7f1f101a5bb0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1f0802c710 tx=0x7f1f08035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.751+0000 7f1f0e000640 1 -- 172.21.15.86:0/3520478494 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08039040 con 0x7f1f1010a050 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.751+0000 7f1f16600640 1 -- 172.21.15.86:0/3520478494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f101a64b0 con 0x7f1f1010a050 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.751+0000 7f1f0e000640 1 -- 172.21.15.86:0/3520478494 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f08044070 con 0x7f1f1010a050 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.751+0000 7f1f0e000640 1 -- 172.21.15.86:0/3520478494 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08004530 con 0x7f1f1010a050 2024-04-17T07:35:07.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.751+0000 7f1f16600640 1 -- 172.21.15.86:0/3520478494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f101a69f0 con 0x7f1f1010a050 2024-04-17T07:35:07.754 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:07 smithi098 ceph-mon[35418]: pgmap v1062: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-17T07:35:07.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f0e000640 1 -- 172.21.15.86:0/3520478494 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1f080327c0 con 0x7f1f1010a050 2024-04-17T07:35:07.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f1aa2d740 1 Processor -- start 2024-04-17T07:35:07.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f1aa2d740 1 -- start start 2024-04-17T07:35:07.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d419790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d41e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d41eed0 0x56061d421360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56061d380120 con 0x56061d419d60 2024-04-17T07:35:07.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56061d326340 con 0x56061d41eed0 2024-04-17T07:35:07.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56061d37c0e0 con 0x56061d419370 2024-04-17T07:35:07.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d41e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d41e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51094/0 (socket says 172.21.15.86:51094) 2024-04-17T07:35:07.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.755+0000 7f1f15200640 1 -- 172.21.15.86:0/1718137220 learned_addr learned my addr 172.21.15.86:0/1718137220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.756+0000 7f1f07e00640 1 --2- 172.21.15.86:0/1718137220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d419790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.756+0000 7f1f15200640 1 -- 172.21.15.86:0/1718137220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 msgr2=0x56061d419790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.756+0000 7f1f15200640 1 --2- 172.21.15.86:0/1718137220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d419790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.756+0000 7f1f15200640 1 -- 172.21.15.86:0/1718137220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d41eed0 msgr2=0x56061d421360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.756+0000 7f1f15200640 1 --2- 172.21.15.86:0/1718137220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d41eed0 0x56061d421360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.756+0000 7f1f15200640 1 -- 172.21.15.86:0/1718137220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56061d15c7d0 con 0x56061d419d60 2024-04-17T07:35:07.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.756+0000 7f1f07e00640 1 --2- 172.21.15.86:0/1718137220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d419790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.756+0000 7f1f15200640 1 --2- 172.21.15.86:0/1718137220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d41e860 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000bc90 tx=0x7f1f0000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c676176dcc232a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.756+0000 7f1f0e000640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ef40a3c30 0x7f1ef40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.756+0000 7f1f06000640 1 -- 172.21.15.86:0/1718137220 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00015070 con 0x56061d419d60 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.757+0000 7f1f0e000640 1 -- 172.21.15.86:0/3520478494 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1f080f2f40 con 0x7f1f1010a050 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.757+0000 7f1f07e00640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ef40a3c30 0x7f1ef40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.757+0000 7f1f06000640 1 -- 172.21.15.86:0/1718137220 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f000035f0 con 0x56061d419d60 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.757+0000 7f1f06000640 1 -- 172.21.15.86:0/1718137220 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f000116a0 con 0x56061d419d60 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.757+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1718137220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 msgr2=0x56061d41e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.757+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/1718137220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d41e860 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000bc90 tx=0x7f1f0000f730 comp rx=0 tx=0).stop 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.757+0000 7f1f07e00640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ef40a3c30 0x7f1ef40a60f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8005fd0 tx=0x7f1ef800b360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.758+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1718137220 shutdown_connections 2024-04-17T07:35:07.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.758+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/1718137220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d41eed0 0x56061d421360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.758+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/1718137220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d41e860 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.758+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/1718137220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d419790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.758+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1718137220 >> 172.21.15.86:0/1718137220 conn(0x56061d4148e0 msgr2=0x56061d414ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.758+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1718137220 shutdown_connections 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.758+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1718137220 wait complete. 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.758+0000 7f1f1aa2d740 1 -- start start 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.758+0000 7f1f1aa2d740 1 Processor -- start 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d419370 0x56061d4238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d423e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d41eed0 0x56061d42ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56061d421e80 con 0x56061d419d60 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56061d339710 con 0x56061d419370 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56061d422000 con 0x56061d41eed0 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f1aa2d740 10 client.0 _ll_register_callbacks cb 0x56061d334af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:07.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d423e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d423e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51106/0 (socket says 172.21.15.86:51106) 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f07e00640 1 -- 172.21.15.86:0/391202906 learned_addr learned my addr 172.21.15.86:0/391202906 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f15200640 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d419370 0x56061d4238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f15c00640 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d41eed0 0x56061d42ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f07e00640 1 -- 172.21.15.86:0/391202906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d41eed0 msgr2=0x56061d42ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f07e00640 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d41eed0 0x56061d42ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f07e00640 1 -- 172.21.15.86:0/391202906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d419370 msgr2=0x56061d4238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.759+0000 7f1f07e00640 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d419370 0x56061d4238e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.760+0000 7f1f07e00640 1 -- 172.21.15.86:0/391202906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56061d0923f0 con 0x56061d419d60 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.760+0000 7f1f07e00640 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d423e20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800bfc0 tx=0x7f1ef8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff2ac25aa410da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.760+0000 7f1efea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.760+0000 7f1efea00640 1 -- 172.21.15.86:0/391202906 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8022070 con 0x56061d419d60 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.760+0000 7f1efea00640 1 -- 172.21.15.86:0/391202906 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1ef801e860 con 0x56061d419d60 2024-04-17T07:35:07.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.760+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/391202906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f000035f0 con 0x56061d419d60 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.760+0000 7f1efea00640 1 -- 172.21.15.86:0/391202906 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ef802e050 con 0x56061d419d60 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.760+0000 7f1f1aa2d740 10 client.20052 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.760+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/391202906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56061d0520f0 con 0x56061d419d60 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.761+0000 7f1efea00640 1 -- 172.21.15.86:0/391202906 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f000035f0 con 0x56061d419d60 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.761+0000 7f1efea00640 1 -- 172.21.15.86:0/391202906 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1ef8010be0 con 0x56061d419d60 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.761+0000 7f1f1aa2d740 10 client.20052 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.761+0000 7f1f1aa2d740 10 client.20052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.761+0000 7f1f1aa2d740 20 client.20052 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.761+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56061d42f700 0x56061d431b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.761+0000 7f1f1aa2d740 4 client.20052 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:35:07.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.761+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/391202906 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56061d38db40 con 0x56061d42f700 2024-04-17T07:35:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.762+0000 7f1f15200640 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56061d42f700 0x56061d431b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.763+0000 7f1f15200640 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56061d42f700 0x56061d431b90 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa9a87386413cab server_cookie=1656f98676d44cf2 in_seq=0 out_seq=0 2024-04-17T07:35:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.763+0000 7f1efea00640 10 client.20052 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:35:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1efea00640 1 -- 172.21.15.86:0/391202906 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56061d38db40 con 0x56061d42f700 2024-04-17T07:35:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1efea00640 10 client.20052 handle_command_reply: tid=0 2024-04-17T07:35:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 client.20052 shutdown 2024-04-17T07:35:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/391202906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 msgr2=0x56061d423e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d423e20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800bfc0 tx=0x7f1ef8005c90 comp rx=0 tx=0).stop 2024-04-17T07:35:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/391202906 shutdown_connections 2024-04-17T07:35:07.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56061d42f700 0x56061d431b90 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d41eed0 0x56061d42ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d423e20 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1efea00640 0 client.20052 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:35:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/391202906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d419370 0x56061d4238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/391202906 >> 172.21.15.86:0/391202906 conn(0x56061d4148e0 msgr2=0x56061d413ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/391202906 shutdown_connections 2024-04-17T07:35:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/391202906 wait complete. 2024-04-17T07:35:07.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.765+0000 7f1f1aa2d740 20 client.20052 trim_cache size 0 max 16384 2024-04-17T07:35:07.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:07 smithi103 ceph-mon[35495]: pgmap v1062: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-17T07:35:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.784+0000 7f1f1aa2d740 1 Processor -- start 2024-04-17T07:35:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.784+0000 7f1f1aa2d740 1 -- start start 2024-04-17T07:35:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d413250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d42ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d41eed0 0x56061d413790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101b0dd0 con 0x56061d419d60 2024-04-17T07:35:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101b2910 con 0x56061d41eed0 2024-04-17T07:35:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101b14a0 con 0x56061d419370 2024-04-17T07:35:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d42ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d42ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51112/0 (socket says 172.21.15.86:51112) 2024-04-17T07:35:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f07e00640 1 -- 172.21.15.86:0/543933541 learned_addr learned my addr 172.21.15.86:0/543933541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f07e00640 1 -- 172.21.15.86:0/543933541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 msgr2=0x56061d413250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f07e00640 1 --2- 172.21.15.86:0/543933541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d413250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f07e00640 1 -- 172.21.15.86:0/543933541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d41eed0 msgr2=0x56061d413790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f07e00640 1 --2- 172.21.15.86:0/543933541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d41eed0 0x56061d413790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.785+0000 7f1f07e00640 1 -- 172.21.15.86:0/543933541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56061d415650 con 0x56061d419d60 2024-04-17T07:35:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.786+0000 7f1f07e00640 1 --2- 172.21.15.86:0/543933541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d42ae20 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800b570 tx=0x7f1ef8005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a480aac364a1a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.786+0000 7f1f05600640 1 -- 172.21.15.86:0/543933541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8036030 con 0x56061d419d60 2024-04-17T07:35:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.786+0000 7f1f05600640 1 -- 172.21.15.86:0/543933541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef80054d0 con 0x56061d419d60 2024-04-17T07:35:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.786+0000 7f1f05600640 1 -- 172.21.15.86:0/543933541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8022050 con 0x56061d419d60 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.786+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/543933541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 msgr2=0x56061d42ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.786+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/543933541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d42ae20 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800b570 tx=0x7f1ef8005ee0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.787+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/543933541 shutdown_connections 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.787+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/543933541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d41eed0 0x56061d413790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.787+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/543933541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d419d60 0x56061d42ae20 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.787+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/543933541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d413250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.787+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/543933541 >> 172.21.15.86:0/543933541 conn(0x56061d4148e0 msgr2=0x56061d41b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.787+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/543933541 shutdown_connections 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.787+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/543933541 wait complete. 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.787+0000 7f1f1aa2d740 1 -- start start 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.787+0000 7f1f1aa2d740 1 Processor -- start 2024-04-17T07:35:07.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d428390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d419d60 0x56061d4288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f1aa2d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d41eed0 0x56061d4298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101b2240 con 0x56061d41eed0 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101b1b70 con 0x56061d419d60 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f1aa2d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f101aeda0 con 0x56061d419370 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f1aa2d740 10 client.0 _ll_register_callbacks cb 0x7f1ef8015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f15c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d41eed0 0x56061d4298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f15c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d41eed0 0x56061d4298e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51120/0 (socket says 172.21.15.86:51120) 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f15c00640 1 -- 172.21.15.86:0/1978940124 learned_addr learned my addr 172.21.15.86:0/1978940124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f07e00640 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d419d60 0x56061d4288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f15200640 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d428390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f15c00640 1 -- 172.21.15.86:0/1978940124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 msgr2=0x56061d428390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f15c00640 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d428390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f15c00640 1 -- 172.21.15.86:0/1978940124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d419d60 msgr2=0x56061d4288d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f15c00640 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d419d60 0x56061d4288d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.788+0000 7f1f15c00640 1 -- 172.21.15.86:0/1978940124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56061d429ed0 con 0x56061d41eed0 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.789+0000 7f1f15c00640 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d41eed0 0x56061d4298e0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1f080bb2a0 tx=0x7f1f080be860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236f6c12c2835c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.789+0000 7f1efea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.789+0000 7f1efea00640 1 -- 172.21.15.86:0/1978940124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f080c1070 con 0x56061d41eed0 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.789+0000 7f1efea00640 1 -- 172.21.15.86:0/1978940124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1f080bf0a0 con 0x56061d41eed0 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.789+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1978940124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef80054d0 con 0x56061d41eed0 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.789+0000 7f1efea00640 1 -- 172.21.15.86:0/1978940124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f080bee40 con 0x56061d41eed0 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.789+0000 7f1f1aa2d740 10 client.20058 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.789+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1978940124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56061d495cd0 con 0x56061d41eed0 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.789+0000 7f1efea00640 1 -- 172.21.15.86:0/1978940124 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef80054d0 con 0x56061d41eed0 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.789+0000 7f1efea00640 1 -- 172.21.15.86:0/1978940124 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f1f0803c030 con 0x56061d41eed0 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.790+0000 7f1f1aa2d740 10 client.20058 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:07.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.790+0000 7f1f1aa2d740 10 client.20058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:35:07.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.790+0000 7f1f1aa2d740 20 client.20058 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:07.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.790+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56061d496610 0x56061d498a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:07.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.790+0000 7f1f1aa2d740 4 client.20058 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-17T07:35:07.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.790+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1978940124 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56061d3879b0 con 0x56061d496610 2024-04-17T07:35:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.790+0000 7f1f15200640 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56061d496610 0x56061d498a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.791+0000 7f1f15200640 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56061d496610 0x56061d498a30 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5d4854dafac444 server_cookie=2fc281165648884b in_seq=0 out_seq=0 2024-04-17T07:35:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.791+0000 7f1efea00640 10 client.20058 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:35:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1efea00640 1 -- 172.21.15.86:0/1978940124 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56061d3879b0 con 0x56061d496610 2024-04-17T07:35:07.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1efea00640 10 client.20058 handle_command_reply: tid=0 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 client.20058 shutdown 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1978940124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d41eed0 msgr2=0x56061d4298e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d41eed0 0x56061d4298e0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f1f080bb2a0 tx=0x7f1f080be860 comp rx=0 tx=0).stop 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1978940124 shutdown_connections 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x56061d496610 0x56061d498a30 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56061d41eed0 0x56061d4298e0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56061d419d60 0x56061d4288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1efea00640 0 client.20058 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 --2- 172.21.15.86:0/1978940124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56061d419370 0x56061d428390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1978940124 >> 172.21.15.86:0/1978940124 conn(0x56061d4148e0 msgr2=0x56061d417b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1978940124 shutdown_connections 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.792+0000 7f1f1aa2d740 1 -- 172.21.15.86:0/1978940124 wait complete. 2024-04-17T07:35:07.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f1aa2d740 20 client.20058 trim_cache size 0 max 16384 2024-04-17T07:35:07.796 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:35:07.796 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-17T07:35:07.796 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-17T07:35:07.796 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:35:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 -- 172.21.15.86:0/3520478494 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ef40a3c30 msgr2=0x7f1ef40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ef40a3c30 0x7f1ef40a60f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8005fd0 tx=0x7f1ef800b360 comp rx=0 tx=0).stop 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 -- 172.21.15.86:0/3520478494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1010a050 msgr2=0x7f1f101a5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1010a050 0x7f1f101a5bb0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1f0802c710 tx=0x7f1f08035d60 comp rx=0 tx=0).stop 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 -- 172.21.15.86:0/3520478494 shutdown_connections 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1ef40a3c30 0x7f1ef40a60f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f1010a050 0x7f1f101a5bb0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f10109690 0x7f1f1019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 --2- 172.21.15.86:0/3520478494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f101084a0 0x7f1f1019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 -- 172.21.15.86:0/3520478494 >> 172.21.15.86:0/3520478494 conn(0x7f1f10077430 msgr2=0x7f1f1010c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 -- 172.21.15.86:0/3520478494 shutdown_connections 2024-04-17T07:35:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:07.793+0000 7f1f16600640 1 -- 172.21.15.86:0/3520478494 wait complete. 2024-04-17T07:35:07.808 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:07 smithi086 ceph-mon[31733]: pgmap v1062: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-17T07:35:07.811 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-17T07:35:07.811 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T07:35:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.022+0000 7f40a9800640 1 -- 172.21.15.86:0/761725035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a4109c80 msgr2=0x7f40a410c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.022+0000 7f40a9800640 1 --2- 172.21.15.86:0/761725035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a4109c80 0x7f40a410c110 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f409c009ca0 tx=0x7f409c02c180 comp rx=0 tx=0).stop 2024-04-17T07:35:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.022+0000 7f40a9800640 1 -- 172.21.15.86:0/761725035 shutdown_connections 2024-04-17T07:35:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.022+0000 7f40a9800640 1 --2- 172.21.15.86:0/761725035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a4109c80 0x7f40a410c110 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.022+0000 7f40a9800640 1 --2- 172.21.15.86:0/761725035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40a4074940 0x7f40a4074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.022+0000 7f40a9800640 1 --2- 172.21.15.86:0/761725035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a4075f20 0x7f40a4074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.022+0000 7f40a9800640 1 -- 172.21.15.86:0/761725035 >> 172.21.15.86:0/761725035 conn(0x7f40a40fd340 msgr2=0x7f40a40ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.023+0000 7f40a9800640 1 -- 172.21.15.86:0/761725035 shutdown_connections 2024-04-17T07:35:08.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.023+0000 7f40a9800640 1 -- 172.21.15.86:0/761725035 wait complete. 2024-04-17T07:35:08.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.023+0000 7f40a9800640 1 Processor -- start 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.024+0000 7f40a9800640 1 -- start start 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.024+0000 7f40a9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a4074940 0x7f40a419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.024+0000 7f40a9800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a4075f20 0x7f40a419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.024+0000 7f40a9800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40a4109c80 0x7f40a41a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.024+0000 7f40a9800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40a410fb20 con 0x7f40a4074940 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.024+0000 7f40a9800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40a410f9a0 con 0x7f40a4109c80 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.024+0000 7f40a9800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f40a410fca0 con 0x7f40a4075f20 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.024+0000 7f40a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a4074940 0x7f40a419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.024+0000 7f40a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a4074940 0x7f40a419a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51132/0 (socket says 172.21.15.86:51132) 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.024+0000 7f40a3e00640 1 -- 172.21.15.86:0/3735746757 learned_addr learned my addr 172.21.15.86:0/3735746757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f40a8e00640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40a4109c80 0x7f40a41a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f409be00640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a4075f20 0x7f40a419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f40a3e00640 1 -- 172.21.15.86:0/3735746757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a4075f20 msgr2=0x7f40a419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f40a3e00640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a4075f20 0x7f40a419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f40a3e00640 1 -- 172.21.15.86:0/3735746757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40a4109c80 msgr2=0x7f40a41a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f40a3e00640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40a4109c80 0x7f40a41a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f40a3e00640 1 -- 172.21.15.86:0/3735746757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a41a1de0 con 0x7f40a4074940 2024-04-17T07:35:08.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f40a3e00640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a4074940 0x7f40a419a1c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f409400d770 tx=0x7f409400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f40a1600640 1 -- 172.21.15.86:0/3735746757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094017070 con 0x7f40a4074940 2024-04-17T07:35:08.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f40a1600640 1 -- 172.21.15.86:0/3735746757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40940054d0 con 0x7f40a4074940 2024-04-17T07:35:08.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.025+0000 7f40a1600640 1 -- 172.21.15.86:0/3735746757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094004cf0 con 0x7f40a4074940 2024-04-17T07:35:08.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.026+0000 7f40a9800640 1 -- 172.21.15.86:0/3735746757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a41a20d0 con 0x7f40a4074940 2024-04-17T07:35:08.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.026+0000 7f40a9800640 1 -- 172.21.15.86:0/3735746757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a41a25d0 con 0x7f40a4074940 2024-04-17T07:35:08.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.027+0000 7f40a9800640 1 -- 172.21.15.86:0/3735746757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a4108a40 con 0x7f40a4074940 2024-04-17T07:35:08.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.029+0000 7f40a1600640 1 -- 172.21.15.86:0/3735746757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4094004e90 con 0x7f40a4074940 2024-04-17T07:35:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.031+0000 7f40a1600640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40880a3bb0 0x7f40880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.031+0000 7f40a1600640 1 -- 172.21.15.86:0/3735746757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f40940d0300 con 0x7f40a4074940 2024-04-17T07:35:08.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.031+0000 7f409be00640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40880a3bb0 0x7f40880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.032+0000 7f409be00640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40880a3bb0 0x7f40880a6070 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f408c005fd0 tx=0x7f408c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.036+0000 7f40a1600640 1 -- 172.21.15.86:0/3735746757 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f409409c900 con 0x7f40a4074940 2024-04-17T07:35:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.284+0000 7f40a9800640 1 -- 172.21.15.86:0/3735746757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40a402d5d0 con 0x7f40a4074940 2024-04-17T07:35:08.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.286+0000 7f40a1600640 1 -- 172.21.15.86:0/3735746757 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f409409c200 con 0x7f40a4074940 2024-04-17T07:35:08.286 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:35:08.286 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14583,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/22224874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":22224874},{"type":"v1","addr":"172.21.15.86:6835","nonce":22224874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/267440661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":267440661},{"type":"v1","addr":"172.21.15.86:6837","nonce":267440661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-17T07:11:54.934275+0000","modified":"2024-04-17T07:15:00.638260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34490,"mds_1":34460,"mds_2":34430,"mds_3":34400,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/2923053510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":2923053510},{"type":"v1","addr":"172.21.15.86:6839","nonce":2923053510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14700":{"gid":14700,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/4256331668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":4256331668},{"type":"v1","addr":"172.21.15.86:6841","nonce":4256331668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/1707978686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1707978686},{"type":"v1","addr":"172.21.15.103:6833","nonce":1707978686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/423518277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":423518277},{"type":"v1","addr":"172.21.15.103:6835","nonce":423518277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/299191474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":299191474},{"type":"v1","addr":"172.21.15.103:6837","nonce":299191474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.103:6839/2592108290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2592108290},{"type":"v1","addr":"172.21.15.103:6839","nonce":2592108290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34400":{"gid":34400,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6833/2318991416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2318991416},{"type":"v1","addr":"172.21.15.98:6833","nonce":2318991416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34430":{"gid":34430,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":19,"addr":"172.21.15.98:6835/133134430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":133134430},{"type":"v1","addr":"172.21.15.98:6835","nonce":133134430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34460":{"gid":34460,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.98:6837/2794896256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2794896256},{"type":"v1","addr":"172.21.15.98:6837","nonce":2794896256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34490":{"gid":34490,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6839/809350102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":809350102},{"type":"v1","addr":"172.21.15.98:6839","nonce":809350102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34490,"qdb_cluster":[24505,34400,34430,34460,34490]},"id":1}]} 2024-04-17T07:35:08.287 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.288+0000 7f40a9800640 1 -- 172.21.15.86:0/3735746757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40880a3bb0 msgr2=0x7f40880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.288+0000 7f40a9800640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40880a3bb0 0x7f40880a6070 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f408c005fd0 tx=0x7f408c005850 comp rx=0 tx=0).stop 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.288+0000 7f40a9800640 1 -- 172.21.15.86:0/3735746757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a4074940 msgr2=0x7f40a419a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.288+0000 7f40a9800640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a4074940 0x7f40a419a1c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f409400d770 tx=0x7f409400dc30 comp rx=0 tx=0).stop 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.288+0000 7f40a9800640 1 -- 172.21.15.86:0/3735746757 shutdown_connections 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.288+0000 7f40a9800640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40880a3bb0 0x7f40880a6070 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.288+0000 7f40a9800640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40a4109c80 0x7f40a41a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.288+0000 7f40a9800640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40a4075f20 0x7f40a419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.289+0000 7f40a9800640 1 --2- 172.21.15.86:0/3735746757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a4074940 0x7f40a419a1c0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.289+0000 7f40a9800640 1 -- 172.21.15.86:0/3735746757 >> 172.21.15.86:0/3735746757 conn(0x7f40a40fd340 msgr2=0x7f40a4105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.289+0000 7f40a9800640 1 -- 172.21.15.86:0/3735746757 shutdown_connections 2024-04-17T07:35:08.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.289+0000 7f40a9800640 1 -- 172.21.15.86:0/3735746757 wait complete. 2024-04-17T07:35:08.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-17T07:35:08.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.525+0000 7f19bf200640 1 -- 172.21.15.86:0/1472149306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0101e70 msgr2=0x7f19c0102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.525+0000 7f19bf200640 1 --2- 172.21.15.86:0/1472149306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0101e70 0x7f19c0102290 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009900 tx=0x7f19ac02bc30 comp rx=0 tx=0).stop 2024-04-17T07:35:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.525+0000 7f19bf200640 1 -- 172.21.15.86:0/1472149306 shutdown_connections 2024-04-17T07:35:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.525+0000 7f19bf200640 1 --2- 172.21.15.86:0/1472149306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c0103980 0x7f19c010a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.525+0000 7f19bf200640 1 --2- 172.21.15.86:0/1472149306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c0102fc0 0x7f19c0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.525+0000 7f19bf200640 1 --2- 172.21.15.86:0/1472149306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0101e70 0x7f19c0102290 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.525+0000 7f19bf200640 1 -- 172.21.15.86:0/1472149306 >> 172.21.15.86:0/1472149306 conn(0x7f19c00fd400 msgr2=0x7f19c00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.525+0000 7f19bf200640 1 -- 172.21.15.86:0/1472149306 shutdown_connections 2024-04-17T07:35:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.525+0000 7f19bf200640 1 -- 172.21.15.86:0/1472149306 wait complete. 2024-04-17T07:35:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.526+0000 7f19bf200640 1 Processor -- start 2024-04-17T07:35:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19bf200640 1 -- start start 2024-04-17T07:35:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19bf200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c0101e70 0x7f19c01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19bf200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c0102fc0 0x7f19c01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19bf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0103980 0x7f19c019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19bf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19c010da10 con 0x7f19c0103980 2024-04-17T07:35:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19bf200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19c010d890 con 0x7f19c0102fc0 2024-04-17T07:35:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19bf200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19c010db90 con 0x7f19c0101e70 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19be800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0103980 0x7f19c019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19be800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0103980 0x7f19c019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51148/0 (socket says 172.21.15.86:51148) 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19be800640 1 -- 172.21.15.86:0/361021903 learned_addr learned my addr 172.21.15.86:0/361021903 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19bd400640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c0102fc0 0x7f19c01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.527+0000 7f19bde00640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c0101e70 0x7f19c01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19be800640 1 -- 172.21.15.86:0/361021903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c0101e70 msgr2=0x7f19c01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19be800640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c0101e70 0x7f19c01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19be800640 1 -- 172.21.15.86:0/361021903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c0102fc0 msgr2=0x7f19c01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19be800640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c0102fc0 0x7f19c01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19be800640 1 -- 172.21.15.86:0/361021903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c019fcd0 con 0x7f19c0103980 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19bde00640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c0101e70 0x7f19c01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:35:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19bd400640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c0102fc0 0x7f19c01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:35:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19be800640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0103980 0x7f19c019f660 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f19b800eaa0 tx=0x7f19b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19b6a00640 1 -- 172.21.15.86:0/361021903 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b800cdb0 con 0x7f19c0103980 2024-04-17T07:35:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19b6a00640 1 -- 172.21.15.86:0/361021903 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19b8005020 con 0x7f19c0103980 2024-04-17T07:35:08.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19b6a00640 1 -- 172.21.15.86:0/361021903 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80116d0 con 0x7f19c0103980 2024-04-17T07:35:08.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.528+0000 7f19bf200640 1 -- 172.21.15.86:0/361021903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c019ffc0 con 0x7f19c0103980 2024-04-17T07:35:08.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.529+0000 7f19bf200640 1 -- 172.21.15.86:0/361021903 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c01a0480 con 0x7f19c0103980 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.533+0000 7f19c7fb1740 1 Processor -- start 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.533+0000 7f19b6a00640 1 -- 172.21.15.86:0/361021903 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f19b80051d0 con 0x7f19c0103980 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.533+0000 7f19c7fb1740 1 -- start start 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8e8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559a7c842020 con 0x559a7c8e60d0 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559a7c7e8460 con 0x559a7c8e0570 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559a7c83dfe0 con 0x559a7c8e0f60 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8e8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8e8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51154/0 (socket says 172.21.15.86:51154) 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19bd400640 1 -- 172.21.15.86:0/3777909460 learned_addr learned my addr 172.21.15.86:0/3777909460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19bde00640 1 --2- 172.21.15.86:0/3777909460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.534+0000 7f19bd400640 1 --2- 172.21.15.86:0/3777909460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.535+0000 7f19b6a00640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19a40a3bb0 0x7f19a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.535+0000 7f19bd400640 1 -- 172.21.15.86:0/3777909460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 msgr2=0x559a7c8e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.535+0000 7f19bd400640 1 --2- 172.21.15.86:0/3777909460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.535+0000 7f19bd400640 1 -- 172.21.15.86:0/3777909460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 msgr2=0x559a7c8e0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.535+0000 7f19bd400640 1 --2- 172.21.15.86:0/3777909460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8e0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.535+0000 7f19bd400640 1 -- 172.21.15.86:0/3777909460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7c7f0040 con 0x559a7c8e60d0 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.535+0000 7f19b6a00640 1 -- 172.21.15.86:0/361021903 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f19b80d09a0 con 0x7f19c0103980 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.535+0000 7f19bd400640 1 --2- 172.21.15.86:0/3777909460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8e8560 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f19b000cfa0 tx=0x7f19b000c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aec4a343246adf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.535+0000 7f19be800640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19a40a3bb0 0x7f19a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.535+0000 7f19ab400640 1 -- 172.21.15.86:0/3777909460 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0019030 con 0x559a7c8e60d0 2024-04-17T07:35:08.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.536+0000 7f19ab400640 1 -- 172.21.15.86:0/3777909460 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19b00054d0 con 0x559a7c8e60d0 2024-04-17T07:35:08.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.536+0000 7f19ab400640 1 -- 172.21.15.86:0/3777909460 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0011800 con 0x559a7c8e60d0 2024-04-17T07:35:08.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.536+0000 7f19c7fb1740 1 -- 172.21.15.86:0/3777909460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 msgr2=0x559a7c8e8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.536+0000 7f19be800640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19a40a3bb0 0x7f19a40a6070 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f19b80a5040 tx=0x7f19b800cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.536+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/3777909460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8e8560 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f19b000cfa0 tx=0x7f19b000c720 comp rx=0 tx=0).stop 2024-04-17T07:35:08.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.536+0000 7f19c7fb1740 1 -- 172.21.15.86:0/3777909460 shutdown_connections 2024-04-17T07:35:08.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.536+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/3777909460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8e8560 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.536+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/3777909460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.536+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/3777909460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.536+0000 7f19c7fb1740 1 -- 172.21.15.86:0/3777909460 >> 172.21.15.86:0/3777909460 conn(0x559a7c8d76d0 msgr2=0x559a7c8d7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.537+0000 7f19c7fb1740 1 -- 172.21.15.86:0/3777909460 shutdown_connections 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.537+0000 7f19c7fb1740 1 -- 172.21.15.86:0/3777909460 wait complete. 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.537+0000 7f19c7fb1740 1 -- start start 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.537+0000 7f19c7fb1740 1 Processor -- start 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0570 0x559a7c8eaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0f60 0x559a7c8eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559a7c8e9080 con 0x559a7c8e60d0 2024-04-17T07:35:08.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559a7c7fb830 con 0x559a7c8e0f60 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559a7c8e9200 con 0x559a7c8e0570 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19c7fb1740 10 client.0 _ll_register_callbacks cb 0x559a7c7c9fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51168/0 (socket says 172.21.15.86:51168) 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19bd400640 1 -- 172.21.15.86:0/1411495564 learned_addr learned my addr 172.21.15.86:0/1411495564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19bd400640 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0570 0x559a7c8eaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19bde00640 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0f60 0x559a7c8eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19bd400640 1 -- 172.21.15.86:0/1411495564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0570 msgr2=0x559a7c8eaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19bd400640 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0570 0x559a7c8eaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19bd400640 1 -- 172.21.15.86:0/1411495564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0f60 msgr2=0x559a7c8eafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.538+0000 7f19bd400640 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0f60 0x559a7c8eafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.539+0000 7f19bd400640 1 -- 172.21.15.86:0/1411495564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7c7ff920 con 0x559a7c8e60d0 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.539+0000 7f19bd400640 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8f1fe0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f19b0004c30 tx=0x7f19b0004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8107713a1d023d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.539+0000 7f199fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.539+0000 7f199fe00640 1 -- 172.21.15.86:0/1411495564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0019040 con 0x559a7c8e60d0 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.539+0000 7f199fe00640 1 -- 172.21.15.86:0/1411495564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f19b0004f00 con 0x559a7c8e60d0 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.539+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1411495564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b00054d0 con 0x559a7c8e60d0 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.539+0000 7f199fe00640 1 -- 172.21.15.86:0/1411495564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19b0032050 con 0x559a7c8e60d0 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.539+0000 7f19c7fb1740 10 client.20076 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.539+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1411495564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a7c7cd7f0 con 0x559a7c8e60d0 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.540+0000 7f199fe00640 1 -- 172.21.15.86:0/1411495564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19b00054d0 con 0x559a7c8e60d0 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.540+0000 7f199fe00640 1 -- 172.21.15.86:0/1411495564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f19b001e580 con 0x559a7c8e60d0 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.540+0000 7f19c7fb1740 10 client.20076 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.540+0000 7f19c7fb1740 10 client.20076 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:35:08.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.540+0000 7f19c7fb1740 20 client.20076 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:08.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.540+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559a7c8f6140 0x559a7c8f8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.540+0000 7f19c7fb1740 4 client.20076 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:35:08.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.540+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1411495564 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a7c84fa40 con 0x559a7c8f6140 2024-04-17T07:35:08.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.541+0000 7f19bde00640 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559a7c8f6140 0x559a7c8f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.542+0000 7f19bde00640 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559a7c8f6140 0x559a7c8f8530 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5364858d0b671de5 server_cookie=7c0edf6ff8a4addf in_seq=0 out_seq=0 2024-04-17T07:35:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.542+0000 7f199fe00640 10 client.20076 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:35:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f199fe00640 1 -- 172.21.15.86:0/1411495564 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a7c84fa40 con 0x559a7c8f6140 2024-04-17T07:35:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f199fe00640 10 client.20076 handle_command_reply: tid=0 2024-04-17T07:35:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f19c7fb1740 1 client.20076 shutdown 2024-04-17T07:35:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1411495564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 msgr2=0x559a7c8f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8f1fe0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f19b0004c30 tx=0x7f19b0004b80 comp rx=0 tx=0).stop 2024-04-17T07:35:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1411495564 shutdown_connections 2024-04-17T07:35:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559a7c8f6140 0x559a7c8f8530 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8f1fe0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0f60 0x559a7c8eafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1411495564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0570 0x559a7c8eaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1411495564 >> 172.21.15.86:0/1411495564 conn(0x559a7c8d76d0 msgr2=0x559a7c8d6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f199fe00640 0 client.20076 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:35:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.544+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1411495564 shutdown_connections 2024-04-17T07:35:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.545+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1411495564 wait complete. 2024-04-17T07:35:08.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.545+0000 7f19c7fb1740 20 client.20076 trim_cache size 0 max 16384 2024-04-17T07:35:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.559+0000 7f19c7fb1740 1 Processor -- start 2024-04-17T07:35:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.559+0000 7f19c7fb1740 1 -- start start 2024-04-17T07:35:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01a8780 con 0x559a7c8e60d0 2024-04-17T07:35:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01aa2c0 con 0x559a7c8e0570 2024-04-17T07:35:08.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01a8e50 con 0x559a7c8e0f60 2024-04-17T07:35:08.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8d60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51180/0 (socket says 172.21.15.86:51180) 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19bd400640 1 -- 172.21.15.86:0/1888600655 learned_addr learned my addr 172.21.15.86:0/1888600655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19bd400640 1 --2- 172.21.15.86:0/1888600655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.560+0000 7f19bde00640 1 --2- 172.21.15.86:0/1888600655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.561+0000 7f19bd400640 1 -- 172.21.15.86:0/1888600655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 msgr2=0x559a7c8f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.561+0000 7f19bd400640 1 --2- 172.21.15.86:0/1888600655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.561+0000 7f19bd400640 1 -- 172.21.15.86:0/1888600655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 msgr2=0x559a7c8e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.561+0000 7f19bd400640 1 --2- 172.21.15.86:0/1888600655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.561+0000 7f19bd400640 1 -- 172.21.15.86:0/1888600655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7c5543f0 con 0x559a7c8e60d0 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.561+0000 7f19bd400640 1 --2- 172.21.15.86:0/1888600655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8d60a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f19b000c2f0 tx=0x7f19b001ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e97160008250a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.561+0000 7f19aaa00640 1 -- 172.21.15.86:0/1888600655 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b003a020 con 0x559a7c8e60d0 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.561+0000 7f19aaa00640 1 -- 172.21.15.86:0/1888600655 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19b000cbf0 con 0x559a7c8e60d0 2024-04-17T07:35:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.561+0000 7f19aaa00640 1 -- 172.21.15.86:0/1888600655 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0019040 con 0x559a7c8e60d0 2024-04-17T07:35:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.562+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1888600655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 msgr2=0x559a7c8d60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.562+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1888600655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8d60a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f19b000c2f0 tx=0x7f19b001ffc0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.562+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1888600655 shutdown_connections 2024-04-17T07:35:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.562+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1888600655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c8d60a0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.562+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1888600655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.562+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1888600655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.562+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1888600655 >> 172.21.15.86:0/1888600655 conn(0x559a7c8d76d0 msgr2=0x559a7c8e2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.562+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1888600655 shutdown_connections 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.562+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1888600655 wait complete. 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.562+0000 7f19c7fb1740 1 -- start start 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19c7fb1740 1 Processor -- start 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8ef020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19c7fb1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c95a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01a9bf0 con 0x559a7c8e60d0 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01a9520 con 0x559a7c8e0570 2024-04-17T07:35:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19c7fb1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19c01a6750 con 0x559a7c8e0f60 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19c7fb1740 10 client.0 _ll_register_callbacks cb 0x7f19b000c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c95a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c95a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51192/0 (socket says 172.21.15.86:51192) 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.563+0000 7f19bd400640 1 -- 172.21.15.86:0/1901368840 learned_addr learned my addr 172.21.15.86:0/1901368840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.564+0000 7f19bd400640 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.564+0000 7f19bde00640 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.564+0000 7f19bd400640 1 -- 172.21.15.86:0/1901368840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 msgr2=0x559a7c8ef560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.564+0000 7f19bd400640 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8ef560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.564+0000 7f19bd400640 1 -- 172.21.15.86:0/1901368840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 msgr2=0x559a7c8ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.564+0000 7f19bd400640 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8ef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.564+0000 7f19bd400640 1 -- 172.21.15.86:0/1901368840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7c61e7d0 con 0x559a7c8e60d0 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.564+0000 7f19bde00640 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.564+0000 7f19bd400640 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c95a610 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f19b00134c0 tx=0x7f19b0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227de1a9e2dd9e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.564+0000 7f199fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.565+0000 7f199fe00640 1 -- 172.21.15.86:0/1901368840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b003a030 con 0x559a7c8e60d0 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.565+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1901368840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b000cbf0 con 0x559a7c8e60d0 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.565+0000 7f199fe00640 1 -- 172.21.15.86:0/1901368840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f19b001ed30 con 0x559a7c8e60d0 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.565+0000 7f199fe00640 1 -- 172.21.15.86:0/1901368840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19b004b050 con 0x559a7c8e60d0 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.565+0000 7f19c7fb1740 10 client.20082 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.565+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1901368840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a7c95bd00 con 0x559a7c8e60d0 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.565+0000 7f199fe00640 1 -- 172.21.15.86:0/1901368840 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19b000cbf0 con 0x559a7c8e60d0 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.565+0000 7f199fe00640 1 -- 172.21.15.86:0/1901368840 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f19b003f020 con 0x559a7c8e60d0 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.566+0000 7f19c7fb1740 10 client.20082 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.566+0000 7f19c7fb1740 10 client.20082 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.566+0000 7f19c7fb1740 20 client.20082 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.566+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559a7c95c5a0 0x559a7c95e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.566+0000 7f19c7fb1740 4 client.20082 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:35:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.566+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1901368840 --> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a7c8498b0 con 0x559a7c95c5a0 2024-04-17T07:35:08.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.566+0000 7f19bde00640 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559a7c95c5a0 0x559a7c95e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.567+0000 7f199fe00640 10 client.20082 ms_handle_connect on v2:172.21.15.103:6838/2592108290 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.567+0000 7f19bde00640 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559a7c95c5a0 0x559a7c95e990 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d22d58af7580d94f server_cookie=ef530267a83d700a in_seq=0 out_seq=0 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f199fe00640 1 -- 172.21.15.86:0/1901368840 <== mds.4 v2:172.21.15.103:6838/2592108290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a7c8498b0 con 0x559a7c95c5a0 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f199fe00640 10 client.20082 handle_command_reply: tid=0 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 client.20082 shutdown 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1901368840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 msgr2=0x559a7c95a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c95a610 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f19b00134c0 tx=0x7f19b0015040 comp rx=0 tx=0).stop 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1901368840 shutdown_connections 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.103:6838/2592108290,v1:172.21.15.103:6839/2592108290] conn(0x559a7c95c5a0 0x559a7c95e990 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a7c8e60d0 0x559a7c95a610 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559a7c8e0f60 0x559a7c8ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 --2- 172.21.15.86:0/1901368840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559a7c8e0570 0x559a7c8ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f199fe00640 0 client.20082 ms_handle_reset on v2:172.21.15.103:6838/2592108290 2024-04-17T07:35:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1901368840 >> 172.21.15.86:0/1901368840 conn(0x559a7c8d76d0 msgr2=0x559a7c8e2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1901368840 shutdown_connections 2024-04-17T07:35:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.568+0000 7f19c7fb1740 1 -- 172.21.15.86:0/1901368840 wait complete. 2024-04-17T07:35:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19c7fb1740 20 client.20082 trim_cache size 0 max 16384 2024-04-17T07:35:08.572 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 -- 172.21.15.86:0/361021903 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19a40a3bb0 msgr2=0x7f19a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19a40a3bb0 0x7f19a40a6070 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f19b80a5040 tx=0x7f19b800cf50 comp rx=0 tx=0).stop 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 -- 172.21.15.86:0/361021903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0103980 msgr2=0x7f19c019f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0103980 0x7f19c019f660 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f19b800eaa0 tx=0x7f19b800ef60 comp rx=0 tx=0).stop 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 -- 172.21.15.86:0/361021903 shutdown_connections 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f19a40a3bb0 0x7f19a40a6070 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19c0103980 0x7f19c019f660 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c0102fc0 0x7f19c01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 --2- 172.21.15.86:0/361021903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19c0101e70 0x7f19c01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 -- 172.21.15.86:0/361021903 >> 172.21.15.86:0/361021903 conn(0x7f19c00fd400 msgr2=0x7f19c0106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 -- 172.21.15.86:0/361021903 shutdown_connections 2024-04-17T07:35:08.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.569+0000 7f19bf200640 1 -- 172.21.15.86:0/361021903 wait complete. 2024-04-17T07:35:08.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:08 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3735746757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:35:08.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-17T07:35:08.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:08 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3735746757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:35:08.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:08 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3735746757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-17T07:35:08.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.783+0000 7fddf3200640 1 -- 172.21.15.86:0/3871563661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf4109690 msgr2=0x7fddf4109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.783+0000 7fddf3200640 1 --2- 172.21.15.86:0/3871563661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf4109690 0x7fddf4109b10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009900 tx=0x7fdddc02bc50 comp rx=0 tx=0).stop 2024-04-17T07:35:08.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.784+0000 7fddf3200640 1 -- 172.21.15.86:0/3871563661 shutdown_connections 2024-04-17T07:35:08.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.784+0000 7fddf3200640 1 --2- 172.21.15.86:0/3871563661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddf410a050 0x7fddf41108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.784+0000 7fddf3200640 1 --2- 172.21.15.86:0/3871563661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf4109690 0x7fddf4109b10 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.784+0000 7fddf3200640 1 --2- 172.21.15.86:0/3871563661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf41084a0 0x7fddf41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.784+0000 7fddf3200640 1 -- 172.21.15.86:0/3871563661 >> 172.21.15.86:0/3871563661 conn(0x7fddf4077430 msgr2=0x7fddf4079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.784+0000 7fddf3200640 1 -- 172.21.15.86:0/3871563661 shutdown_connections 2024-04-17T07:35:08.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.784+0000 7fddf3200640 1 -- 172.21.15.86:0/3871563661 wait complete. 2024-04-17T07:35:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.785+0000 7fddf3200640 1 Processor -- start 2024-04-17T07:35:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.785+0000 7fddf3200640 1 -- start start 2024-04-17T07:35:08.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.785+0000 7fddf3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf41084a0 0x7fddf419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.785+0000 7fddf3200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddf4109690 0x7fddf419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.785+0000 7fddf3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf410a050 0x7fddf41a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.785+0000 7fddf3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fddf4113fe0 con 0x7fddf410a050 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.785+0000 7fddf3200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fddf4113e60 con 0x7fddf4109690 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.785+0000 7fddf3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fddf4114160 con 0x7fddf41084a0 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fddf2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf410a050 0x7fddf41a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fddf2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf410a050 0x7fddf41a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51216/0 (socket says 172.21.15.86:51216) 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fddf2800640 1 -- 172.21.15.86:0/1155372664 learned_addr learned my addr 172.21.15.86:0/1155372664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fdde9400640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddf4109690 0x7fddf419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fddf1e00640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf41084a0 0x7fddf419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fddf2800640 1 -- 172.21.15.86:0/1155372664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf41084a0 msgr2=0x7fddf419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fddf2800640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf41084a0 0x7fddf419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fddf2800640 1 -- 172.21.15.86:0/1155372664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddf4109690 msgr2=0x7fddf419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fddf2800640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddf4109690 0x7fddf419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fddf2800640 1 -- 172.21.15.86:0/1155372664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf41a62e0 con 0x7fddf410a050 2024-04-17T07:35:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.786+0000 7fddf1e00640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf41084a0 0x7fddf419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.787+0000 7fddf2800640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf410a050 0x7fddf41a5c70 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fddec00eaa0 tx=0x7fddec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.787+0000 7fddeb400640 1 -- 172.21.15.86:0/1155372664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec00cdb0 con 0x7fddf410a050 2024-04-17T07:35:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.787+0000 7fddeb400640 1 -- 172.21.15.86:0/1155372664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddec005020 con 0x7fddf410a050 2024-04-17T07:35:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.787+0000 7fddeb400640 1 -- 172.21.15.86:0/1155372664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0116d0 con 0x7fddf410a050 2024-04-17T07:35:08.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.787+0000 7fddf3200640 1 -- 172.21.15.86:0/1155372664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf41a65d0 con 0x7fddf410a050 2024-04-17T07:35:08.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.787+0000 7fddf3200640 1 -- 172.21.15.86:0/1155372664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf41a6ad0 con 0x7fddf410a050 2024-04-17T07:35:08.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.790+0000 7fddfba7a740 1 Processor -- start 2024-04-17T07:35:08.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.791+0000 7fddeb400640 1 -- 172.21.15.86:0/1155372664 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fddec0051d0 con 0x7fddf410a050 2024-04-17T07:35:08.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.791+0000 7fddfba7a740 1 -- start start 2024-04-17T07:35:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.791+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528c160 0x55607528c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.791+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x556075291650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.791+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 0x556075294150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.791+0000 7fddfba7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5560751f2020 con 0x55607528cb50 2024-04-17T07:35:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.791+0000 7fddfba7a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556075198460 con 0x556075291cc0 2024-04-17T07:35:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.791+0000 7fddfba7a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5560751edfe0 con 0x55607528c160 2024-04-17T07:35:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fddf1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x556075291650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fddf1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x556075291650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51222/0 (socket says 172.21.15.86:51222) 2024-04-17T07:35:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fddf1e00640 1 -- 172.21.15.86:0/1118009558 learned_addr learned my addr 172.21.15.86:0/1118009558 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fdde9400640 1 --2- 172.21.15.86:0/1118009558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528c160 0x55607528c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fddf1e00640 1 -- 172.21.15.86:0/1118009558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528c160 msgr2=0x55607528c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fdde9400640 1 --2- 172.21.15.86:0/1118009558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 0x556075294150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fddf1e00640 1 --2- 172.21.15.86:0/1118009558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528c160 0x55607528c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fddf1e00640 1 -- 172.21.15.86:0/1118009558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 msgr2=0x556075294150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fddf1e00640 1 --2- 172.21.15.86:0/1118009558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 0x556075294150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fddf1e00640 1 -- 172.21.15.86:0/1118009558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560751a0040 con 0x55607528cb50 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.792+0000 7fdde9400640 1 --2- 172.21.15.86:0/1118009558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 0x556075294150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.793+0000 7fddeb400640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddd40a3c30 0x7fddd40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.793+0000 7fddeb400640 1 -- 172.21.15.86:0/1155372664 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fddec0d09b0 con 0x7fddf410a050 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.793+0000 7fddf1e00640 1 --2- 172.21.15.86:0/1118009558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x556075291650 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009c30 tx=0x7fdde400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46e66ab2676b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.793+0000 7fdde9400640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddd40a3c30 0x7fddd40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.793+0000 7fdde3400640 1 -- 172.21.15.86:0/1118009558 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4016070 con 0x55607528cb50 2024-04-17T07:35:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.793+0000 7fdde3400640 1 -- 172.21.15.86:0/1118009558 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdde40054d0 con 0x55607528cb50 2024-04-17T07:35:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.793+0000 7fdde3400640 1 -- 172.21.15.86:0/1118009558 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4011470 con 0x55607528cb50 2024-04-17T07:35:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.794+0000 7fddfba7a740 1 -- 172.21.15.86:0/1118009558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 msgr2=0x556075291650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.794+0000 7fdde9400640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddd40a3c30 0x7fddd40a60f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fddf419fbd0 tx=0x7fdddc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.794+0000 7fddfba7a740 1 --2- 172.21.15.86:0/1118009558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x556075291650 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009c30 tx=0x7fdde400e6c0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.794+0000 7fddfba7a740 1 -- 172.21.15.86:0/1118009558 shutdown_connections 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.794+0000 7fddfba7a740 1 --2- 172.21.15.86:0/1118009558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 0x556075294150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.794+0000 7fddfba7a740 1 --2- 172.21.15.86:0/1118009558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x556075291650 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.794+0000 7fddfba7a740 1 --2- 172.21.15.86:0/1118009558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528c160 0x55607528c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.794+0000 7fddfba7a740 1 -- 172.21.15.86:0/1118009558 >> 172.21.15.86:0/1118009558 conn(0x5560752876d0 msgr2=0x556075287ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.794+0000 7fddfba7a740 1 -- 172.21.15.86:0/1118009558 shutdown_connections 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.794+0000 7fddfba7a740 1 -- 172.21.15.86:0/1118009558 wait complete. 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.795+0000 7fddfba7a740 1 -- start start 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.795+0000 7fddfba7a740 1 Processor -- start 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.795+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528c160 0x556075296640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.795+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528cb50 0x556075296b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.795+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 0x55607529dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.795+0000 7fddfba7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556075294c70 con 0x55607528c160 2024-04-17T07:35:08.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.795+0000 7fddfba7a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5560751ab830 con 0x556075291cc0 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.795+0000 7fddfba7a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556075294df0 con 0x55607528cb50 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddfba7a740 10 client.0 _ll_register_callbacks cb 0x556075179fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddf1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528c160 0x556075296640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddf1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528c160 0x556075296640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51238/0 (socket says 172.21.15.86:51238) 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddf1e00640 1 -- 172.21.15.86:0/3150031024 learned_addr learned my addr 172.21.15.86:0/3150031024 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fdde9400640 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528cb50 0x556075296b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddf2800640 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 0x55607529dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddf1e00640 1 -- 172.21.15.86:0/3150031024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528cb50 msgr2=0x556075296b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddf1e00640 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528cb50 0x556075296b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddf1e00640 1 -- 172.21.15.86:0/3150031024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 msgr2=0x55607529dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddf1e00640 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 0x55607529dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddf1e00640 1 -- 172.21.15.86:0/3150031024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560751af920 con 0x55607528c160 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fddf1e00640 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528c160 0x556075296640 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fdde4005e70 tx=0x7fdde4009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56028b3c2a2f955 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.796+0000 7fdddbe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:08.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.797+0000 7fdddbe00640 1 -- 172.21.15.86:0/3150031024 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4028060 con 0x55607528c160 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.797+0000 7fdddbe00640 1 -- 172.21.15.86:0/3150031024 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdde4004ee0 con 0x55607528c160 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.797+0000 7fddfba7a740 1 -- 172.21.15.86:0/3150031024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde40054d0 con 0x55607528c160 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.797+0000 7fdddbe00640 1 -- 172.21.15.86:0/3150031024 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdde402f050 con 0x55607528c160 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.797+0000 7fddfba7a740 10 client.20094 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.797+0000 7fddfba7a740 1 -- 172.21.15.86:0/3150031024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55607517d7f0 con 0x55607528c160 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.797+0000 7fdddbe00640 1 -- 172.21.15.86:0/3150031024 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdde40054d0 con 0x55607528c160 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.798+0000 7fdddbe00640 1 -- 172.21.15.86:0/3150031024 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fdde400f040 con 0x55607528c160 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.798+0000 7fddfba7a740 10 client.20094 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.798+0000 7fddfba7a740 10 client.20094 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.798+0000 7fddfba7a740 20 client.20094 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.798+0000 7fddfba7a740 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560752a1db0 0x5560752a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.798+0000 7fddfba7a740 4 client.20094 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:35:08.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.798+0000 7fddfba7a740 1 -- 172.21.15.86:0/3150031024 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560751ffa40 con 0x5560752a1db0 2024-04-17T07:35:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.798+0000 7fdde9400640 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560752a1db0 0x5560752a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.800+0000 7fdde9400640 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560752a1db0 0x5560752a41a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bce80bb9d91660ba server_cookie=204caec7939b4099 in_seq=0 out_seq=0 2024-04-17T07:35:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.800+0000 7fdddbe00640 10 client.20094 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:35:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.801+0000 7fdddbe00640 1 -- 172.21.15.86:0/3150031024 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560751ffa40 con 0x5560752a1db0 2024-04-17T07:35:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.801+0000 7fdddbe00640 10 client.20094 handle_command_reply: tid=0 2024-04-17T07:35:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.801+0000 7fddfba7a740 1 client.20094 shutdown 2024-04-17T07:35:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 1 -- 172.21.15.86:0/3150031024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528c160 msgr2=0x556075296640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528c160 0x556075296640 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fdde4005e70 tx=0x7fdde4009970 comp rx=0 tx=0).stop 2024-04-17T07:35:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 1 -- 172.21.15.86:0/3150031024 shutdown_connections 2024-04-17T07:35:08.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x5560752a1db0 0x5560752a41a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556075291cc0 0x55607529dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528cb50 0x556075296b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fdddbe00640 0 client.20094 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:35:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 1 --2- 172.21.15.86:0/3150031024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528c160 0x556075296640 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 1 -- 172.21.15.86:0/3150031024 >> 172.21.15.86:0/3150031024 conn(0x5560752876d0 msgr2=0x556075286930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 1 -- 172.21.15.86:0/3150031024 shutdown_connections 2024-04-17T07:35:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 1 -- 172.21.15.86:0/3150031024 wait complete. 2024-04-17T07:35:08.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.802+0000 7fddfba7a740 20 client.20094 trim_cache size 0 max 16384 2024-04-17T07:35:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.817+0000 7fddfba7a740 1 Processor -- start 2024-04-17T07:35:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.817+0000 7fddfba7a740 1 -- start start 2024-04-17T07:35:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.817+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 0x556075295c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.817+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528cb50 0x55607529dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.817+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556075291cc0 0x5560752961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.817+0000 7fddfba7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41b0ea0 con 0x556075291cc0 2024-04-17T07:35:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.817+0000 7fddfba7a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41b29e0 con 0x55607528c160 2024-04-17T07:35:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.817+0000 7fddfba7a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41b1570 con 0x55607528cb50 2024-04-17T07:35:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fddf2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556075291cc0 0x5560752961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fddf2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556075291cc0 0x5560752961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51246/0 (socket says 172.21.15.86:51246) 2024-04-17T07:35:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fddf2800640 1 -- 172.21.15.86:0/601807621 learned_addr learned my addr 172.21.15.86:0/601807621 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fddf1e00640 1 --2- 172.21.15.86:0/601807621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 0x556075295c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fdde9400640 1 --2- 172.21.15.86:0/601807621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528cb50 0x55607529dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fddf2800640 1 -- 172.21.15.86:0/601807621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528cb50 msgr2=0x55607529dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fddf2800640 1 --2- 172.21.15.86:0/601807621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528cb50 0x55607529dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fddf2800640 1 -- 172.21.15.86:0/601807621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 msgr2=0x556075295c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fddf2800640 1 --2- 172.21.15.86:0/601807621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 0x556075295c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fddf2800640 1 -- 172.21.15.86:0/601807621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556074f043f0 con 0x556075291cc0 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.818+0000 7fddf1e00640 1 --2- 172.21.15.86:0/601807621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 0x556075295c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.819+0000 7fddf2800640 1 --2- 172.21.15.86:0/601807621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556075291cc0 0x5560752961a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fddec09c8a0 tx=0x7fddec09d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cfa1c143ce3137 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.819+0000 7fdde2a00640 1 -- 172.21.15.86:0/601807621 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec01f040 con 0x556075291cc0 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.819+0000 7fdde2a00640 1 -- 172.21.15.86:0/601807621 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddec09ac10 con 0x556075291cc0 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.819+0000 7fdde2a00640 1 -- 172.21.15.86:0/601807621 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec087340 con 0x556075291cc0 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.820+0000 7fddfba7a740 1 -- 172.21.15.86:0/601807621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556075291cc0 msgr2=0x5560752961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.820+0000 7fddfba7a740 1 --2- 172.21.15.86:0/601807621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556075291cc0 0x5560752961a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fddec09c8a0 tx=0x7fddec09d150 comp rx=0 tx=0).stop 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.820+0000 7fddfba7a740 1 -- 172.21.15.86:0/601807621 shutdown_connections 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.820+0000 7fddfba7a740 1 --2- 172.21.15.86:0/601807621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556075291cc0 0x5560752961a0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.820+0000 7fddfba7a740 1 --2- 172.21.15.86:0/601807621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55607528cb50 0x55607529dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.820+0000 7fddfba7a740 1 --2- 172.21.15.86:0/601807621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 0x556075295c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.820+0000 7fddfba7a740 1 -- 172.21.15.86:0/601807621 >> 172.21.15.86:0/601807621 conn(0x5560752876d0 msgr2=0x55607528b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.820+0000 7fddfba7a740 1 -- 172.21.15.86:0/601807621 shutdown_connections 2024-04-17T07:35:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.820+0000 7fddfba7a740 1 -- 172.21.15.86:0/601807621 wait complete. 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.820+0000 7fddfba7a740 1 -- start start 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fddfba7a740 1 Processor -- start 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 0x55607529ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x55607529b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fddfba7a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556075291cc0 0x55607529c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fddfba7a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41b2310 con 0x55607528cb50 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fddfba7a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41b1c40 con 0x55607528c160 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fddfba7a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fddf41aee70 con 0x556075291cc0 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fddfba7a740 10 client.0 _ll_register_callbacks cb 0x7fdde40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fdde9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x55607529b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fdde9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x55607529b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51254/0 (socket says 172.21.15.86:51254) 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.821+0000 7fdde9400640 1 -- 172.21.15.86:0/1308947394 learned_addr learned my addr 172.21.15.86:0/1308947394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.822+0000 7fddf1e00640 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 0x55607529ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.822+0000 7fddf2800640 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556075291cc0 0x55607529c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.822+0000 7fdde9400640 1 -- 172.21.15.86:0/1308947394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556075291cc0 msgr2=0x55607529c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.822+0000 7fdde9400640 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556075291cc0 0x55607529c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.822+0000 7fdde9400640 1 -- 172.21.15.86:0/1308947394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 msgr2=0x55607529ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.822+0000 7fdde9400640 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 0x55607529ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.822+0000 7fdde9400640 1 -- 172.21.15.86:0/1308947394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556074fce7d0 con 0x55607528cb50 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.822+0000 7fddf2800640 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556075291cc0 0x55607529c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.822+0000 7fdde9400640 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x55607529b340 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fdddc02c190 tx=0x7fdddc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e719d8bd87655e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.822+0000 7fdddbe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.823+0000 7fdddbe00640 1 -- 172.21.15.86:0/1308947394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc056070 con 0x55607528cb50 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.823+0000 7fdddbe00640 1 -- 172.21.15.86:0/1308947394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fdddc005490 con 0x55607528cb50 2024-04-17T07:35:08.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.823+0000 7fddfba7a740 1 -- 172.21.15.86:0/1308947394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddec09ac10 con 0x55607528cb50 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.823+0000 7fdddbe00640 1 -- 172.21.15.86:0/1308947394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdddc0046c0 con 0x55607528cb50 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.823+0000 7fddfba7a740 10 client.20100 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.823+0000 7fddfba7a740 1 -- 172.21.15.86:0/1308947394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55607529da80 con 0x55607528cb50 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.824+0000 7fdddbe00640 1 -- 172.21.15.86:0/1308947394 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddec09ac10 con 0x55607528cb50 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.824+0000 7fdddbe00640 1 -- 172.21.15.86:0/1308947394 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7fdde4009d20 con 0x55607528cb50 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.824+0000 7fddfba7a740 10 client.20100 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.824+0000 7fddfba7a740 10 client.20100 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.824+0000 7fddfba7a740 20 client.20100 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.824+0000 7fddfba7a740 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x556075308770 0x55607530ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.824+0000 7fddfba7a740 4 client.20100 mds_command: new command op to 34400 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.824+0000 7fddfba7a740 1 -- 172.21.15.86:0/1308947394 --> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560751f98b0 con 0x556075308770 2024-04-17T07:35:08.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.825+0000 7fddf1e00640 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x556075308770 0x55607530ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.826+0000 7fddf1e00640 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x556075308770 0x55607530ab90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ae552f17fbbdb23 server_cookie=e02a3a43e8091a15 in_seq=0 out_seq=0 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.826+0000 7fdddbe00640 10 client.20100 ms_handle_connect on v2:172.21.15.98:6832/2318991416 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.826+0000 7fdddbe00640 1 -- 172.21.15.86:0/1308947394 <== mds.3 v2:172.21.15.98:6832/2318991416 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560751f98b0 con 0x556075308770 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.826+0000 7fdddbe00640 10 client.20100 handle_command_reply: tid=0 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.826+0000 7fddfba7a740 1 client.20100 shutdown 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.826+0000 7fddfba7a740 1 -- 172.21.15.86:0/1308947394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 msgr2=0x55607529b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.826+0000 7fddfba7a740 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x55607529b340 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fdddc02c190 tx=0x7fdddc009260 comp rx=0 tx=0).stop 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddfba7a740 1 -- 172.21.15.86:0/1308947394 shutdown_connections 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddfba7a740 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.98:6832/2318991416,v1:172.21.15.98:6833/2318991416] conn(0x556075308770 0x55607530ab90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddfba7a740 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556075291cc0 0x55607529c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddfba7a740 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55607528cb50 0x55607529b340 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fdddbe00640 0 client.20100 ms_handle_reset on v2:172.21.15.98:6832/2318991416 2024-04-17T07:35:08.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddfba7a740 1 --2- 172.21.15.86:0/1308947394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55607528c160 0x55607529ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddfba7a740 1 -- 172.21.15.86:0/1308947394 >> 172.21.15.86:0/1308947394 conn(0x5560752876d0 msgr2=0x5560752860a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddfba7a740 1 -- 172.21.15.86:0/1308947394 shutdown_connections 2024-04-17T07:35:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddfba7a740 1 -- 172.21.15.86:0/1308947394 wait complete. 2024-04-17T07:35:08.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddfba7a740 20 client.20100 trim_cache size 0 max 16384 2024-04-17T07:35:08.830 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddf3200640 1 -- 172.21.15.86:0/1155372664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddd40a3c30 msgr2=0x7fddd40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.827+0000 7fddf3200640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddd40a3c30 0x7fddd40a60f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fddf419fbd0 tx=0x7fdddc000bc0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.828+0000 7fddf3200640 1 -- 172.21.15.86:0/1155372664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf410a050 msgr2=0x7fddf41a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.828+0000 7fddf3200640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf410a050 0x7fddf41a5c70 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fddec00eaa0 tx=0x7fddec00ef60 comp rx=0 tx=0).stop 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.828+0000 7fddf3200640 1 -- 172.21.15.86:0/1155372664 shutdown_connections 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.828+0000 7fddf3200640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fddd40a3c30 0x7fddd40a60f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.828+0000 7fddf3200640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddf410a050 0x7fddf41a5c70 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.828+0000 7fddf3200640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddf4109690 0x7fddf419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.828+0000 7fddf3200640 1 --2- 172.21.15.86:0/1155372664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fddf41084a0 0x7fddf419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.828+0000 7fddf3200640 1 -- 172.21.15.86:0/1155372664 >> 172.21.15.86:0/1155372664 conn(0x7fddf4077430 msgr2=0x7fddf410cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:08.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.828+0000 7fddf3200640 1 -- 172.21.15.86:0/1155372664 shutdown_connections 2024-04-17T07:35:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:08.828+0000 7fddf3200640 1 -- 172.21.15.86:0/1155372664 wait complete. 2024-04-17T07:35:08.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-17T07:35:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.051+0000 7f8806600640 1 -- 172.21.15.86:0/3597686953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88001084a0 msgr2=0x7f88001088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.051+0000 7f8806600640 1 --2- 172.21.15.86:0/3597686953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88001084a0 0x7f88001088c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009900 tx=0x7f87f002bc30 comp rx=0 tx=0).stop 2024-04-17T07:35:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.051+0000 7f8806600640 1 -- 172.21.15.86:0/3597686953 shutdown_connections 2024-04-17T07:35:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.051+0000 7f8806600640 1 --2- 172.21.15.86:0/3597686953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880010a050 0x7f88001108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.051+0000 7f8806600640 1 --2- 172.21.15.86:0/3597686953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8800109690 0x7f8800109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.051+0000 7f8806600640 1 --2- 172.21.15.86:0/3597686953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88001084a0 0x7f88001088c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.051+0000 7f8806600640 1 -- 172.21.15.86:0/3597686953 >> 172.21.15.86:0/3597686953 conn(0x7f8800077430 msgr2=0x7f8800079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.051+0000 7f8806600640 1 -- 172.21.15.86:0/3597686953 shutdown_connections 2024-04-17T07:35:09.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.051+0000 7f8806600640 1 -- 172.21.15.86:0/3597686953 wait complete. 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.052+0000 7f8806600640 1 Processor -- start 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8806600640 1 -- start start 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8806600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88001084a0 0x7f880019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8806600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8800109690 0x7f880019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8806600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880010a050 0x7f88001a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8806600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8800114000 con 0x7f88001084a0 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8806600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8800113e80 con 0x7f880010a050 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8806600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8800114180 con 0x7f8800109690 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8805200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88001084a0 0x7f880019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8805200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88001084a0 0x7f880019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51274/0 (socket says 172.21.15.86:51274) 2024-04-17T07:35:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8805200640 1 -- 172.21.15.86:0/1551297398 learned_addr learned my addr 172.21.15.86:0/1551297398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8805200640 1 -- 172.21.15.86:0/1551297398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8800109690 msgr2=0x7f880019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8805200640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8800109690 0x7f880019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8805200640 1 -- 172.21.15.86:0/1551297398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880010a050 msgr2=0x7f88001a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8805200640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880010a050 0x7f88001a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.053+0000 7f8805200640 1 -- 172.21.15.86:0/1551297398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88001a6320 con 0x7f88001084a0 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.054+0000 7f8805200640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88001084a0 0x7f880019e700 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f87f0005ab0 tx=0x7f87f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.054+0000 7f87fd600640 1 -- 172.21.15.86:0/1551297398 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0043020 con 0x7f88001084a0 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.054+0000 7f87fd600640 1 -- 172.21.15.86:0/1551297398 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87f00054c0 con 0x7f88001084a0 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.054+0000 7f8806600640 1 -- 172.21.15.86:0/1551297398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88001a65b0 con 0x7f88001084a0 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.054+0000 7f87fd600640 1 -- 172.21.15.86:0/1551297398 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0003d10 con 0x7f88001084a0 2024-04-17T07:35:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.054+0000 7f8806600640 1 -- 172.21.15.86:0/1551297398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88001a6b00 con 0x7f88001084a0 2024-04-17T07:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.058+0000 7f880a9a0740 1 Processor -- start 2024-04-17T07:35:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.058+0000 7f880a9a0740 1 -- start start 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8caa160 0x5640a8caa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8caf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8cafcc0 0x5640a8cb2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f880a9a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640a8c10020 con 0x5640a8cafcc0 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f880a9a0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5640a8bb6460 con 0x5640a8caab50 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f880a9a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5640a8c0bfe0 con 0x5640a8caa160 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f87fd600640 1 -- 172.21.15.86:0/1551297398 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f87f0046020 con 0x7f88001084a0 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f87ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8cafcc0 0x5640a8cb2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f87ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8cafcc0 0x5640a8cb2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51288/0 (socket says 172.21.15.86:51288) 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f87ffe00640 1 -- 172.21.15.86:0/412181257 learned_addr learned my addr 172.21.15.86:0/412181257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f8805c00640 1 --2- 172.21.15.86:0/412181257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8caa160 0x5640a8caa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.059+0000 7f8805200640 1 --2- 172.21.15.86:0/412181257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8caf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87ffe00640 1 -- 172.21.15.86:0/412181257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8caa160 msgr2=0x5640a8caa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87ffe00640 1 --2- 172.21.15.86:0/412181257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8caa160 0x5640a8caa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87ffe00640 1 -- 172.21.15.86:0/412181257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 msgr2=0x5640a8caf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87ffe00640 1 --2- 172.21.15.86:0/412181257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8caf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87ffe00640 1 -- 172.21.15.86:0/412181257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a8bbe040 con 0x5640a8cafcc0 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87ffe00640 1 --2- 172.21.15.86:0/412181257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8cafcc0 0x5640a8cb2150 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00bae0 tx=0x7f87ec00d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47355fd9e57b92a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87fd600640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87e40a3bb0 0x7f87e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87fd600640 1 -- 172.21.15.86:0/1551297398 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f87f00f2b80 con 0x7f88001084a0 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87f6000640 1 -- 172.21.15.86:0/412181257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec016070 con 0x5640a8cafcc0 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87f6000640 1 -- 172.21.15.86:0/412181257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87ec0054d0 con 0x5640a8cafcc0 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.060+0000 7f87f6000640 1 -- 172.21.15.86:0/412181257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec010460 con 0x5640a8cafcc0 2024-04-17T07:35:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.061+0000 7f8805c00640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87e40a3bb0 0x7f87e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.061+0000 7f880a9a0740 1 -- 172.21.15.86:0/412181257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8cafcc0 msgr2=0x5640a8cb2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.061+0000 7f8805c00640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87e40a3bb0 0x7f87e40a6070 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009270 tx=0x7f87f80095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.061+0000 7f880a9a0740 1 --2- 172.21.15.86:0/412181257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8cafcc0 0x5640a8cb2150 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00bae0 tx=0x7f87ec00d2e0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.061+0000 7f880a9a0740 1 -- 172.21.15.86:0/412181257 shutdown_connections 2024-04-17T07:35:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.062+0000 7f880a9a0740 1 --2- 172.21.15.86:0/412181257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8cafcc0 0x5640a8cb2150 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.062+0000 7f880a9a0740 1 --2- 172.21.15.86:0/412181257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8caf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.062+0000 7f880a9a0740 1 --2- 172.21.15.86:0/412181257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8caa160 0x5640a8caa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.062+0000 7f880a9a0740 1 -- 172.21.15.86:0/412181257 >> 172.21.15.86:0/412181257 conn(0x5640a8ca56d0 msgr2=0x5640a8ca5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.062+0000 7f880a9a0740 1 -- 172.21.15.86:0/412181257 shutdown_connections 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.062+0000 7f880a9a0740 1 -- 172.21.15.86:0/412181257 wait complete. 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.062+0000 7f880a9a0740 1 -- start start 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.062+0000 7f880a9a0740 1 Processor -- start 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8cb4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8cafcc0 0x5640a8cbbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f880a9a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640a8cb2c70 con 0x5640a8caa160 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f880a9a0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5640a8bc9830 con 0x5640a8caab50 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f880a9a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5640a8cb2df0 con 0x5640a8cafcc0 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f880a9a0740 10 client.0 _ll_register_callbacks cb 0x5640a8b97fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f8805200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f8805200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51290/0 (socket says 172.21.15.86:51290) 2024-04-17T07:35:09.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f8805200640 1 -- 172.21.15.86:0/3694127549 learned_addr learned my addr 172.21.15.86:0/3694127549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f87ffe00640 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8cb4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f8805c00640 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8cafcc0 0x5640a8cbbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f8805200640 1 -- 172.21.15.86:0/3694127549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8cafcc0 msgr2=0x5640a8cbbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f8805200640 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8cafcc0 0x5640a8cbbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f8805200640 1 -- 172.21.15.86:0/3694127549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 msgr2=0x5640a8cb4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f8805200640 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8cb4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f8805200640 1 -- 172.21.15.86:0/3694127549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a8bcd920 con 0x5640a8caa160 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.063+0000 7f87ffe00640 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8cb4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.064+0000 7f8805200640 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4690 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f87f00f0300 tx=0x7f87f0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd98c3a89a4b2b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.064+0000 7f87eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.064+0000 7f87eaa00640 1 -- 172.21.15.86:0/3694127549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00be620 con 0x5640a8caa160 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.064+0000 7f87eaa00640 1 -- 172.21.15.86:0/3694127549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f87f00bcde0 con 0x5640a8caa160 2024-04-17T07:35:09.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.064+0000 7f880a9a0740 1 -- 172.21.15.86:0/3694127549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ec0054d0 con 0x5640a8caa160 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.064+0000 7f87eaa00640 1 -- 172.21.15.86:0/3694127549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87f00044e0 con 0x5640a8caa160 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.064+0000 7f880a9a0740 10 client.20112 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.064+0000 7f880a9a0740 1 -- 172.21.15.86:0/3694127549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640a8b9b7f0 con 0x5640a8caa160 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.065+0000 7f87eaa00640 1 -- 172.21.15.86:0/3694127549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87ec0054d0 con 0x5640a8caa160 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.065+0000 7f87eaa00640 1 -- 172.21.15.86:0/3694127549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f87f0083bd0 con 0x5640a8caa160 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.065+0000 7f880a9a0740 10 client.20112 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.065+0000 7f880a9a0740 10 client.20112 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.065+0000 7f880a9a0740 20 client.20112 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.065+0000 7f880a9a0740 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5640a8cc0100 0x5640a8cc25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.065+0000 7f880a9a0740 4 client.20112 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.065+0000 7f880a9a0740 1 -- 172.21.15.86:0/3694127549 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640a8c1da40 con 0x5640a8cc0100 2024-04-17T07:35:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.066+0000 7f87ffe00640 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5640a8cc0100 0x5640a8cc25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.067+0000 7f87ffe00640 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5640a8cc0100 0x5640a8cc25a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34b847d78a08c42b server_cookie=5c7c912afa4c5ce5 in_seq=0 out_seq=0 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.067+0000 7f87eaa00640 10 client.20112 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.068+0000 7f87eaa00640 1 -- 172.21.15.86:0/3694127549 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640a8c1da40 con 0x5640a8cc0100 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.068+0000 7f87eaa00640 10 client.20112 handle_command_reply: tid=0 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 client.20112 shutdown 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 -- 172.21.15.86:0/3694127549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 msgr2=0x5640a8cb4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4690 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f87f00f0300 tx=0x7f87f0009560 comp rx=0 tx=0).stop 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 -- 172.21.15.86:0/3694127549 shutdown_connections 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5640a8cc0100 0x5640a8cc25a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8cafcc0 0x5640a8cbbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8cb4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 --2- 172.21.15.86:0/3694127549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4690 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f87eaa00640 0 client.20112 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:35:09.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 -- 172.21.15.86:0/3694127549 >> 172.21.15.86:0/3694127549 conn(0x5640a8ca56d0 msgr2=0x5640a8ca4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 -- 172.21.15.86:0/3694127549 shutdown_connections 2024-04-17T07:35:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 1 -- 172.21.15.86:0/3694127549 wait complete. 2024-04-17T07:35:09.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.069+0000 7f880a9a0740 20 client.20112 trim_cache size 0 max 16384 2024-04-17T07:35:09.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.084+0000 7f880a9a0740 1 Processor -- start 2024-04-17T07:35:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.084+0000 7f880a9a0740 1 -- start start 2024-04-17T07:35:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.084+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.084+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8caab50 0x5640a8cbbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.084+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8cafcc0 0x5640a8ca9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.084+0000 7f880a9a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88001b0e70 con 0x5640a8caa160 2024-04-17T07:35:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.084+0000 7f880a9a0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f88001b29b0 con 0x5640a8cafcc0 2024-04-17T07:35:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.084+0000 7f880a9a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88001b1540 con 0x5640a8caab50 2024-04-17T07:35:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.084+0000 7f8805200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f8805200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51306/0 (socket says 172.21.15.86:51306) 2024-04-17T07:35:09.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f8805200640 1 -- 172.21.15.86:0/1197466014 learned_addr learned my addr 172.21.15.86:0/1197466014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f8805c00640 1 --2- 172.21.15.86:0/1197466014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8cafcc0 0x5640a8ca9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f8805200640 1 -- 172.21.15.86:0/1197466014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8caab50 msgr2=0x5640a8cbbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f8805200640 1 --2- 172.21.15.86:0/1197466014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8caab50 0x5640a8cbbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f8805200640 1 -- 172.21.15.86:0/1197466014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8cafcc0 msgr2=0x5640a8ca9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f8805200640 1 --2- 172.21.15.86:0/1197466014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8cafcc0 0x5640a8ca9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f8805200640 1 -- 172.21.15.86:0/1197466014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a89223f0 con 0x5640a8caa160 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f8805c00640 1 --2- 172.21.15.86:0/1197466014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8cafcc0 0x5640a8ca9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f8805200640 1 --2- 172.21.15.86:0/1197466014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4b30 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f87f00baf40 tx=0x7f87f00f04e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73100be978bdac44 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.085+0000 7f87f5600640 1 -- 172.21.15.86:0/1197466014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00be440 con 0x5640a8caa160 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f87f5600640 1 -- 172.21.15.86:0/1197466014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87f0088070 con 0x5640a8caa160 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f87f5600640 1 -- 172.21.15.86:0/1197466014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00beae0 con 0x5640a8caa160 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f880a9a0740 1 -- 172.21.15.86:0/1197466014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 msgr2=0x5640a8cb4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f880a9a0740 1 --2- 172.21.15.86:0/1197466014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4b30 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f87f00baf40 tx=0x7f87f00f04e0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f880a9a0740 1 -- 172.21.15.86:0/1197466014 shutdown_connections 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f880a9a0740 1 --2- 172.21.15.86:0/1197466014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8cafcc0 0x5640a8ca9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f880a9a0740 1 --2- 172.21.15.86:0/1197466014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8caab50 0x5640a8cbbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f880a9a0740 1 --2- 172.21.15.86:0/1197466014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb4b30 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f880a9a0740 1 -- 172.21.15.86:0/1197466014 >> 172.21.15.86:0/1197466014 conn(0x5640a8ca56d0 msgr2=0x5640a8ca8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f880a9a0740 1 -- 172.21.15.86:0/1197466014 shutdown_connections 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.086+0000 7f880a9a0740 1 -- 172.21.15.86:0/1197466014 wait complete. 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.087+0000 7f880a9a0740 1 -- start start 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.087+0000 7f880a9a0740 1 Processor -- start 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.087+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.087+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8cb9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.087+0000 7f880a9a0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8cafcc0 0x5640a8cbaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.087+0000 7f880a9a0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88001b22e0 con 0x5640a8caa160 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.087+0000 7f880a9a0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f88001b1c10 con 0x5640a8caab50 2024-04-17T07:35:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.087+0000 7f880a9a0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88001aee40 con 0x5640a8cafcc0 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.087+0000 7f880a9a0740 10 client.0 _ll_register_callbacks cb 0x7f87f0005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f8805200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f8805200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51314/0 (socket says 172.21.15.86:51314) 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f8805200640 1 -- 172.21.15.86:0/954280084 learned_addr learned my addr 172.21.15.86:0/954280084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f8805200640 1 -- 172.21.15.86:0/954280084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8cafcc0 msgr2=0x5640a8cbaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f8805200640 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8cafcc0 0x5640a8cbaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f8805200640 1 -- 172.21.15.86:0/954280084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 msgr2=0x5640a8cb9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f87ffe00640 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8cb9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f8805200640 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8cb9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f8805200640 1 -- 172.21.15.86:0/954280084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a89ec7d0 con 0x5640a8caa160 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f8805200640 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb9540 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f87f004e000 tx=0x7f87f0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f1e2129c9c84bd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f87eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.088+0000 7f87eaa00640 1 -- 172.21.15.86:0/954280084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f00bad80 con 0x5640a8caa160 2024-04-17T07:35:09.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f87eaa00640 1 -- 172.21.15.86:0/954280084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f87f00c74d0 con 0x5640a8caa160 2024-04-17T07:35:09.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f880a9a0740 1 -- 172.21.15.86:0/954280084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f0088070 con 0x5640a8caa160 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f87eaa00640 1 -- 172.21.15.86:0/954280084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87f0004c50 con 0x5640a8caa160 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f880a9a0740 10 client.20118 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f880a9a0740 1 -- 172.21.15.86:0/954280084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640a89ec7d0 con 0x5640a8caa160 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f87eaa00640 1 -- 172.21.15.86:0/954280084 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87f0088070 con 0x5640a8caa160 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f87eaa00640 1 -- 172.21.15.86:0/954280084 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f87f00ba830 con 0x5640a8caa160 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f880a9a0740 10 client.20118 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f880a9a0740 10 client.20118 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f880a9a0740 20 client.20118 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f880a9a0740 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5640a8d26cb0 0x5640a8d290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f880a9a0740 4 client.20118 mds_command: new command op to 34430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:35:09.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.089+0000 7f880a9a0740 1 -- 172.21.15.86:0/954280084 --> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640a8c178b0 con 0x5640a8d26cb0 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.090+0000 7f87ffe00640 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5640a8d26cb0 0x5640a8d290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.091+0000 7f87ffe00640 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5640a8d26cb0 0x5640a8d290d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=300c231119748a7f server_cookie=42e6f02783aa42a in_seq=0 out_seq=0 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.091+0000 7f87eaa00640 10 client.20118 ms_handle_connect on v2:172.21.15.98:6834/133134430 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.091+0000 7f87eaa00640 1 -- 172.21.15.86:0/954280084 <== mds.2 v2:172.21.15.98:6834/133134430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640a8c178b0 con 0x5640a8d26cb0 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.091+0000 7f87eaa00640 10 client.20118 handle_command_reply: tid=0 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.091+0000 7f880a9a0740 1 client.20118 shutdown 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 1 -- 172.21.15.86:0/954280084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 msgr2=0x5640a8cb9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb9540 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f87f004e000 tx=0x7f87f0009560 comp rx=0 tx=0).stop 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 1 -- 172.21.15.86:0/954280084 shutdown_connections 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.98:6834/133134430,v1:172.21.15.98:6835/133134430] conn(0x5640a8d26cb0 0x5640a8d290d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640a8cafcc0 0x5640a8cbaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640a8caab50 0x5640a8cb9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 1 --2- 172.21.15.86:0/954280084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a8caa160 0x5640a8cb9540 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f87eaa00640 0 client.20118 ms_handle_reset on v2:172.21.15.98:6834/133134430 2024-04-17T07:35:09.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 1 -- 172.21.15.86:0/954280084 >> 172.21.15.86:0/954280084 conn(0x5640a8ca56d0 msgr2=0x5640a8cb49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 1 -- 172.21.15.86:0/954280084 shutdown_connections 2024-04-17T07:35:09.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 1 -- 172.21.15.86:0/954280084 wait complete. 2024-04-17T07:35:09.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.092+0000 7f880a9a0740 20 client.20118 trim_cache size 0 max 16384 2024-04-17T07:35:09.096 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:35:09.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 -- 172.21.15.86:0/1551297398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87e40a3bb0 msgr2=0x7f87e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87e40a3bb0 0x7f87e40a6070 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009270 tx=0x7f87f80095e0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 -- 172.21.15.86:0/1551297398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88001084a0 msgr2=0x7f880019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88001084a0 0x7f880019e700 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f87f0005ab0 tx=0x7f87f00042f0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 -- 172.21.15.86:0/1551297398 shutdown_connections 2024-04-17T07:35:09.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f87e40a3bb0 0x7f87e40a6070 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880010a050 0x7f88001a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8800109690 0x7f880019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 --2- 172.21.15.86:0/1551297398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88001084a0 0x7f880019e700 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 -- 172.21.15.86:0/1551297398 >> 172.21.15.86:0/1551297398 conn(0x7f8800077430 msgr2=0x7f880010ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 -- 172.21.15.86:0/1551297398 shutdown_connections 2024-04-17T07:35:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.093+0000 7f8806600640 1 -- 172.21.15.86:0/1551297398 wait complete. 2024-04-17T07:35:09.111 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-17T07:35:09.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.323+0000 7f8356800640 1 -- 172.21.15.86:0/4123112403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8350105c50 msgr2=0x7f835010c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.323+0000 7f8356800640 1 --2- 172.21.15.86:0/4123112403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8350105c50 0x7f835010c4e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f8348009ca0 tx=0x7f834802c100 comp rx=0 tx=0).stop 2024-04-17T07:35:09.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.323+0000 7f8356800640 1 -- 172.21.15.86:0/4123112403 shutdown_connections 2024-04-17T07:35:09.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.323+0000 7f8356800640 1 --2- 172.21.15.86:0/4123112403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8350105c50 0x7f835010c4e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.323+0000 7f8356800640 1 --2- 172.21.15.86:0/4123112403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8350105290 0x7f8350105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.323+0000 7f8356800640 1 --2- 172.21.15.86:0/4123112403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83501040a0 0x7f83501044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.323+0000 7f8356800640 1 -- 172.21.15.86:0/4123112403 >> 172.21.15.86:0/4123112403 conn(0x7f83500ff610 msgr2=0x7f8350101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.323+0000 7f8356800640 1 -- 172.21.15.86:0/4123112403 shutdown_connections 2024-04-17T07:35:09.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.323+0000 7f8356800640 1 -- 172.21.15.86:0/4123112403 wait complete. 2024-04-17T07:35:09.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.324+0000 7f8356800640 1 Processor -- start 2024-04-17T07:35:09.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.324+0000 7f8356800640 1 -- start start 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.324+0000 7f8356800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83501040a0 0x7f835019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8356800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8350105290 0x7f835019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8356800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8350105c50 0x7f83501a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8356800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f835010fbe0 con 0x7f83501040a0 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8356800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f835010fa60 con 0x7f8350105290 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8356800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f835010fd60 con 0x7f8350105c50 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8355400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83501040a0 0x7f835019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8355400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83501040a0 0x7f835019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51336/0 (socket says 172.21.15.86:51336) 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8355400640 1 -- 172.21.15.86:0/3062130079 learned_addr learned my addr 172.21.15.86:0/3062130079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8355400640 1 -- 172.21.15.86:0/3062130079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8350105c50 msgr2=0x7f83501a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8355400640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8350105c50 0x7f83501a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8355400640 1 -- 172.21.15.86:0/3062130079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8350105290 msgr2=0x7f835019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8355400640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8350105290 0x7f835019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.325+0000 7f8355400640 1 -- 172.21.15.86:0/3062130079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83501a1dc0 con 0x7f83501040a0 2024-04-17T07:35:09.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.326+0000 7f8355400640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83501040a0 0x7f835019a1a0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f834400d950 tx=0x7f834400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.326+0000 7f834e000640 1 -- 172.21.15.86:0/3062130079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344017070 con 0x7f83501040a0 2024-04-17T07:35:09.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.326+0000 7f834e000640 1 -- 172.21.15.86:0/3062130079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8344009c30 con 0x7f83501040a0 2024-04-17T07:35:09.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.326+0000 7f8356800640 1 -- 172.21.15.86:0/3062130079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83501a20b0 con 0x7f83501040a0 2024-04-17T07:35:09.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.326+0000 7f834e000640 1 -- 172.21.15.86:0/3062130079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834400f620 con 0x7f83501040a0 2024-04-17T07:35:09.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.326+0000 7f8356800640 1 -- 172.21.15.86:0/3062130079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83501a25b0 con 0x7f83501040a0 2024-04-17T07:35:09.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.330+0000 7f835aa24740 1 Processor -- start 2024-04-17T07:35:09.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.330+0000 7f835aa24740 1 -- start start 2024-04-17T07:35:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.331+0000 7f835aa24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1160 0x55e2f0eb1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.331+0000 7f835aa24740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0eb6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.331+0000 7f835aa24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0eb9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.331+0000 7f835aa24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2f0e17020 con 0x55e2f0eb1160 2024-04-17T07:35:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.331+0000 7f835aa24740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e2f0dbd460 con 0x55e2f0eb1b50 2024-04-17T07:35:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.331+0000 7f835aa24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e2f0e12fe0 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.331+0000 7f834e000640 1 -- 172.21.15.86:0/3062130079 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f834401a070 con 0x7f83501040a0 2024-04-17T07:35:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.331+0000 7f8355e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1160 0x55e2f0eb1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.331+0000 7f8355e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1160 0x55e2f0eb1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51338/0 (socket says 172.21.15.86:51338) 2024-04-17T07:35:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.331+0000 7f8355e00640 1 -- 172.21.15.86:0/3630838192 learned_addr learned my addr 172.21.15.86:0/3630838192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f8355400640 1 --2- 172.21.15.86:0/3630838192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0eb6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f8355e00640 1 -- 172.21.15.86:0/3630838192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb6cc0 msgr2=0x55e2f0eb9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f8355e00640 1 --2- 172.21.15.86:0/3630838192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0eb9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f8355e00640 1 -- 172.21.15.86:0/3630838192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 msgr2=0x55e2f0eb6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f8355e00640 1 --2- 172.21.15.86:0/3630838192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0eb6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f8355e00640 1 -- 172.21.15.86:0/3630838192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f0dc5040 con 0x55e2f0eb1160 2024-04-17T07:35:09.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f8355e00640 1 --2- 172.21.15.86:0/3630838192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1160 0x55e2f0eb1580 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8348005e30 tx=0x7f8348005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45db32e4dc68f1cd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f8342600640 1 -- 172.21.15.86:0/3630838192 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348041020 con 0x55e2f0eb1160 2024-04-17T07:35:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f8342600640 1 -- 172.21.15.86:0/3630838192 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8348032dc0 con 0x55e2f0eb1160 2024-04-17T07:35:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f8342600640 1 -- 172.21.15.86:0/3630838192 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348039040 con 0x55e2f0eb1160 2024-04-17T07:35:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.332+0000 7f834e000640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83380a3bb0 0x7f83380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.333+0000 7f834e000640 1 -- 172.21.15.86:0/3062130079 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f83440d0300 con 0x7f83501040a0 2024-04-17T07:35:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.333+0000 7f8355400640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83380a3bb0 0x7f83380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.333+0000 7f835aa24740 1 -- 172.21.15.86:0/3630838192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1160 msgr2=0x55e2f0eb1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.333+0000 7f835aa24740 1 --2- 172.21.15.86:0/3630838192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1160 0x55e2f0eb1580 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8348005e30 tx=0x7f8348005f80 comp rx=0 tx=0).stop 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.333+0000 7f835aa24740 1 -- 172.21.15.86:0/3630838192 shutdown_connections 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.333+0000 7f835aa24740 1 --2- 172.21.15.86:0/3630838192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0eb9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.333+0000 7f835aa24740 1 --2- 172.21.15.86:0/3630838192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0eb6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.333+0000 7f835aa24740 1 --2- 172.21.15.86:0/3630838192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1160 0x55e2f0eb1580 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.333+0000 7f8355400640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83380a3bb0 0x7f83380a6070 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f834409c190 tx=0x7f8344005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.333+0000 7f835aa24740 1 -- 172.21.15.86:0/3630838192 >> 172.21.15.86:0/3630838192 conn(0x55e2f0eac6d0 msgr2=0x55e2f0eacad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.334+0000 7f835aa24740 1 -- 172.21.15.86:0/3630838192 shutdown_connections 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.334+0000 7f835aa24740 1 -- 172.21.15.86:0/3630838192 wait complete. 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.334+0000 7f835aa24740 1 -- start start 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.334+0000 7f835aa24740 1 Processor -- start 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f835aa24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ebb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f835aa24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ebbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f835aa24740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f835aa24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2f0eb9c10 con 0x55e2f0eb1b50 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f835aa24740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e2f0dd0830 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f835aa24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e2f0eb9d90 con 0x55e2f0eb1160 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f835aa24740 10 client.0 _ll_register_callbacks cb 0x55e2f0d9efb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f8354a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ebbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f8354a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ebbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51350/0 (socket says 172.21.15.86:51350) 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f8354a00640 1 -- 172.21.15.86:0/1368754437 learned_addr learned my addr 172.21.15.86:0/1368754437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f8355e00640 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ebb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f8355e00640 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f8354a00640 1 -- 172.21.15.86:0/1368754437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 msgr2=0x55e2f0ebb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f8354a00640 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ebb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f8354a00640 1 -- 172.21.15.86:0/1368754437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb6cc0 msgr2=0x55e2f0ec2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f8354a00640 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.335+0000 7f8354a00640 1 -- 172.21.15.86:0/1368754437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f0dd4920 con 0x55e2f0eb1b50 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.336+0000 7f8354a00640 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ebbb60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f833c00afd0 tx=0x7f833c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef5c395b620acc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.336+0000 7f8336a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.336+0000 7f8336a00640 1 -- 172.21.15.86:0/1368754437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c004220 con 0x55e2f0eb1b50 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.336+0000 7f8336a00640 1 -- 172.21.15.86:0/1368754437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f833c0043c0 con 0x55e2f0eb1b50 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.336+0000 7f835aa24740 1 -- 172.21.15.86:0/1368754437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8348032dc0 con 0x55e2f0eb1b50 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.336+0000 7f8336a00640 1 -- 172.21.15.86:0/1368754437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f833c01f050 con 0x55e2f0eb1b50 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.336+0000 7f835aa24740 10 client.20130 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.336+0000 7f835aa24740 1 -- 172.21.15.86:0/1368754437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2f0da27f0 con 0x55e2f0eb1b50 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.337+0000 7f8336a00640 1 -- 172.21.15.86:0/1368754437 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8348032dc0 con 0x55e2f0eb1b50 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.337+0000 7f8336a00640 1 -- 172.21.15.86:0/1368754437 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f833c00f660 con 0x55e2f0eb1b50 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.337+0000 7f835aa24740 10 client.20130 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.337+0000 7f835aa24740 10 client.20130 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:35:09.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.337+0000 7f835aa24740 20 client.20130 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:09.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.337+0000 7f835aa24740 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2f0ec71b0 0x55e2f0ec95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.337+0000 7f835aa24740 4 client.20130 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:35:09.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.337+0000 7f835aa24740 1 -- 172.21.15.86:0/1368754437 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2f0e24a40 con 0x55e2f0ec71b0 2024-04-17T07:35:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.338+0000 7f8355e00640 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2f0ec71b0 0x55e2f0ec95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.339+0000 7f8355e00640 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2f0ec71b0 0x55e2f0ec95f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c48f2741127af5e6 server_cookie=7a22ecbd2d3f2cce in_seq=0 out_seq=0 2024-04-17T07:35:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.339+0000 7f8336a00640 10 client.20130 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:35:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.340+0000 7f8336a00640 1 -- 172.21.15.86:0/1368754437 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2f0e24a40 con 0x55e2f0ec71b0 2024-04-17T07:35:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.340+0000 7f8336a00640 10 client.20130 handle_command_reply: tid=0 2024-04-17T07:35:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 client.20130 shutdown 2024-04-17T07:35:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 -- 172.21.15.86:0/1368754437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1b50 msgr2=0x55e2f0ebbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ebbb60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f833c00afd0 tx=0x7f833c00c830 comp rx=0 tx=0).stop 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 -- 172.21.15.86:0/1368754437 shutdown_connections 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2f0ec71b0 0x55e2f0ec95f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ebbb60 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 --2- 172.21.15.86:0/1368754437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ebb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f8336a00640 0 client.20130 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 -- 172.21.15.86:0/1368754437 >> 172.21.15.86:0/1368754437 conn(0x55e2f0eac6d0 msgr2=0x55e2f0eab930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 -- 172.21.15.86:0/1368754437 shutdown_connections 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 1 -- 172.21.15.86:0/1368754437 wait complete. 2024-04-17T07:35:09.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.341+0000 7f835aa24740 20 client.20130 trim_cache size 0 max 16384 2024-04-17T07:35:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f835aa24740 1 Processor -- start 2024-04-17T07:35:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f835aa24740 1 -- start start 2024-04-17T07:35:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f835aa24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ebb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f835aa24740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ebbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f835aa24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f835aa24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83501ac910 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f835aa24740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f83501ae450 con 0x55e2f0eb1b50 2024-04-17T07:35:09.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f835aa24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83501acfe0 con 0x55e2f0eb1160 2024-04-17T07:35:09.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f8355e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f8355e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51358/0 (socket says 172.21.15.86:51358) 2024-04-17T07:35:09.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.356+0000 7f8355e00640 1 -- 172.21.15.86:0/2512910063 learned_addr learned my addr 172.21.15.86:0/2512910063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8354a00640 1 --2- 172.21.15.86:0/2512910063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ebbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8355e00640 1 --2- 172.21.15.86:0/2512910063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ebb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8355e00640 1 -- 172.21.15.86:0/2512910063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 msgr2=0x55e2f0ebb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8355e00640 1 --2- 172.21.15.86:0/2512910063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ebb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8355e00640 1 -- 172.21.15.86:0/2512910063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 msgr2=0x55e2f0ebbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8355e00640 1 --2- 172.21.15.86:0/2512910063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ebbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8355e00640 1 -- 172.21.15.86:0/2512910063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f0b293f0 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8355e00640 1 --2- 172.21.15.86:0/2512910063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec2b60 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8348000f80 tx=0x7f8348032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f621ed0c85e3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8341c00640 1 -- 172.21.15.86:0/2512910063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348054070 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8341c00640 1 -- 172.21.15.86:0/2512910063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f834804ebc0 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.357+0000 7f8341c00640 1 -- 172.21.15.86:0/2512910063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348039050 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.358+0000 7f835aa24740 1 -- 172.21.15.86:0/2512910063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 msgr2=0x55e2f0ec2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.358+0000 7f835aa24740 1 --2- 172.21.15.86:0/2512910063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec2b60 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8348000f80 tx=0x7f8348032340 comp rx=0 tx=0).stop 2024-04-17T07:35:09.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.358+0000 7f835aa24740 1 -- 172.21.15.86:0/2512910063 shutdown_connections 2024-04-17T07:35:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.358+0000 7f835aa24740 1 --2- 172.21.15.86:0/2512910063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec2b60 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.358+0000 7f835aa24740 1 --2- 172.21.15.86:0/2512910063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ebbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.358+0000 7f835aa24740 1 --2- 172.21.15.86:0/2512910063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ebb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.358+0000 7f835aa24740 1 -- 172.21.15.86:0/2512910063 >> 172.21.15.86:0/2512910063 conn(0x55e2f0eac6d0 msgr2=0x55e2f0eacab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.358+0000 7f835aa24740 1 -- 172.21.15.86:0/2512910063 shutdown_connections 2024-04-17T07:35:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.358+0000 7f835aa24740 1 -- 172.21.15.86:0/2512910063 wait complete. 2024-04-17T07:35:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.359+0000 7f835aa24740 1 -- start start 2024-04-17T07:35:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.359+0000 7f835aa24740 1 Processor -- start 2024-04-17T07:35:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.359+0000 7f835aa24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ec0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.359+0000 7f835aa24740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ec0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.359+0000 7f835aa24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.359+0000 7f835aa24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83501add80 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.359+0000 7f835aa24740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f83501ad6b0 con 0x55e2f0eb1b50 2024-04-17T07:35:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.359+0000 7f835aa24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83501aa8e0 con 0x55e2f0eb1160 2024-04-17T07:35:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.359+0000 7f835aa24740 10 client.0 _ll_register_callbacks cb 0x7f833c00ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:09.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.359+0000 7f8355e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8355e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51366/0 (socket says 172.21.15.86:51366) 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8355e00640 1 -- 172.21.15.86:0/3007291982 learned_addr learned my addr 172.21.15.86:0/3007291982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8355e00640 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ec0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8354a00640 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ec0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8355e00640 1 -- 172.21.15.86:0/3007291982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 msgr2=0x55e2f0ec0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8355e00640 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ec0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8355e00640 1 -- 172.21.15.86:0/3007291982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 msgr2=0x55e2f0ec0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8355e00640 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ec0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8355e00640 1 -- 172.21.15.86:0/3007291982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f0bf37d0 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8355e00640 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec1c90 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8348004af0 tx=0x7f83480039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4d317018a05fec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8336a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8336a00640 1 -- 172.21.15.86:0/3007291982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348039070 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f8336a00640 1 -- 172.21.15.86:0/3007291982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8348032d20 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.360+0000 7f835aa24740 1 -- 172.21.15.86:0/3007291982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f834804ebc0 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f8336a00640 1 -- 172.21.15.86:0/3007291982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8348054040 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f835aa24740 10 client.20136 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f835aa24740 1 -- 172.21.15.86:0/3007291982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2f0f2da20 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f8336a00640 1 -- 172.21.15.86:0/3007291982 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f834804ebc0 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f8336a00640 1 -- 172.21.15.86:0/3007291982 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f834805a070 con 0x55e2f0eb6cc0 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f835aa24740 10 client.20136 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f835aa24740 10 client.20136 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f835aa24740 20 client.20136 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f835aa24740 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2f0f2de10 0x55e2f0f30230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f835aa24740 4 client.20136 mds_command: new command op to 34460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.361+0000 7f835aa24740 1 -- 172.21.15.86:0/3007291982 --> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2f0e1e8b0 con 0x55e2f0f2de10 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.362+0000 7f8354a00640 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2f0f2de10 0x55e2f0f30230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.363+0000 7f8354a00640 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2f0f2de10 0x55e2f0f30230 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21eafe4b8e0eca6a server_cookie=98352328305c4ae1 in_seq=0 out_seq=0 2024-04-17T07:35:09.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.363+0000 7f8336a00640 10 client.20136 ms_handle_connect on v2:172.21.15.98:6836/2794896256 2024-04-17T07:35:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.363+0000 7f8336a00640 1 -- 172.21.15.86:0/3007291982 <== mds.1 v2:172.21.15.98:6836/2794896256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2f0e1e8b0 con 0x55e2f0f2de10 2024-04-17T07:35:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.363+0000 7f8336a00640 10 client.20136 handle_command_reply: tid=0 2024-04-17T07:35:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.363+0000 7f835aa24740 1 client.20136 shutdown 2024-04-17T07:35:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 1 -- 172.21.15.86:0/3007291982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 msgr2=0x55e2f0ec1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec1c90 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8348004af0 tx=0x7f83480039f0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 1 -- 172.21.15.86:0/3007291982 shutdown_connections 2024-04-17T07:35:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2f0eb6cc0 0x55e2f0ec1c90 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.98:6836/2794896256,v1:172.21.15.98:6837/2794896256] conn(0x55e2f0f2de10 0x55e2f0f30230 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e2f0eb1b50 0x55e2f0ec0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 1 --2- 172.21.15.86:0/3007291982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e2f0eb1160 0x55e2f0ec0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f8336a00640 0 client.20136 ms_handle_reset on v2:172.21.15.98:6836/2794896256 2024-04-17T07:35:09.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 1 -- 172.21.15.86:0/3007291982 >> 172.21.15.86:0/3007291982 conn(0x55e2f0eac6d0 msgr2=0x55e2f0eaf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 1 -- 172.21.15.86:0/3007291982 shutdown_connections 2024-04-17T07:35:09.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 1 -- 172.21.15.86:0/3007291982 wait complete. 2024-04-17T07:35:09.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.364+0000 7f835aa24740 20 client.20136 trim_cache size 0 max 16384 2024-04-17T07:35:09.368 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:35:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 -- 172.21.15.86:0/3062130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83380a3bb0 msgr2=0x7f83380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83380a3bb0 0x7f83380a6070 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f834409c190 tx=0x7f8344005f30 comp rx=0 tx=0).stop 2024-04-17T07:35:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 -- 172.21.15.86:0/3062130079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83501040a0 msgr2=0x7f835019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83501040a0 0x7f835019a1a0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f834400d950 tx=0x7f834400de10 comp rx=0 tx=0).stop 2024-04-17T07:35:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 -- 172.21.15.86:0/3062130079 shutdown_connections 2024-04-17T07:35:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f83380a3bb0 0x7f83380a6070 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8350105c50 0x7f83501a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8350105290 0x7f835019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 --2- 172.21.15.86:0/3062130079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83501040a0 0x7f835019a1a0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 -- 172.21.15.86:0/3062130079 >> 172.21.15.86:0/3062130079 conn(0x7f83500ff610 msgr2=0x7f8350108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 -- 172.21.15.86:0/3062130079 shutdown_connections 2024-04-17T07:35:09.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.365+0000 7f8356800640 1 -- 172.21.15.86:0/3062130079 wait complete. 2024-04-17T07:35:09.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-17T07:35:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.599+0000 7f40d4a00640 1 -- 172.21.15.86:0/3035861123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40d0101e70 msgr2=0x7f40d0102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.599+0000 7f40d4a00640 1 --2- 172.21.15.86:0/3035861123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40d0101e70 0x7f40d0102290 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009900 tx=0x7f40c002bc30 comp rx=0 tx=0).stop 2024-04-17T07:35:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.599+0000 7f40d4a00640 1 -- 172.21.15.86:0/3035861123 shutdown_connections 2024-04-17T07:35:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.599+0000 7f40d4a00640 1 --2- 172.21.15.86:0/3035861123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40d0103980 0x7f40d010a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.599+0000 7f40d4a00640 1 --2- 172.21.15.86:0/3035861123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40d0102fc0 0x7f40d0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.599+0000 7f40d4a00640 1 --2- 172.21.15.86:0/3035861123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40d0101e70 0x7f40d0102290 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.599+0000 7f40d4a00640 1 -- 172.21.15.86:0/3035861123 >> 172.21.15.86:0/3035861123 conn(0x7f40d00fd400 msgr2=0x7f40d00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.600+0000 7f40d4a00640 1 -- 172.21.15.86:0/3035861123 shutdown_connections 2024-04-17T07:35:09.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.600+0000 7f40d4a00640 1 -- 172.21.15.86:0/3035861123 wait complete. 2024-04-17T07:35:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.601+0000 7f40d4a00640 1 Processor -- start 2024-04-17T07:35:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.601+0000 7f40d4a00640 1 -- start start 2024-04-17T07:35:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.601+0000 7f40d4a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40d0101e70 0x7f40d019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.601+0000 7f40d4a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40d0102fc0 0x7f40d019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.601+0000 7f40d4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40d0103980 0x7f40d01a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.601+0000 7f40d4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40d010d9a0 con 0x7f40d0103980 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.601+0000 7f40d4a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40d010d820 con 0x7f40d0101e70 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.601+0000 7f40d4a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f40d010db20 con 0x7f40d0102fc0 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.602+0000 7f40cf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40d0103980 0x7f40d01a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.602+0000 7f40cf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40d0103980 0x7f40d01a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51386/0 (socket says 172.21.15.86:51386) 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.602+0000 7f40cf200640 1 -- 172.21.15.86:0/3114191605 learned_addr learned my addr 172.21.15.86:0/3114191605 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.602+0000 7f40ce800640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40d0101e70 0x7f40d019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.602+0000 7f40cf200640 1 -- 172.21.15.86:0/3114191605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40d0102fc0 msgr2=0x7f40d019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.602+0000 7f40cf200640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40d0102fc0 0x7f40d019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.602+0000 7f40cf200640 1 -- 172.21.15.86:0/3114191605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40d0101e70 msgr2=0x7f40d019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.602+0000 7f40cf200640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40d0101e70 0x7f40d019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.602+0000 7f40cf200640 1 -- 172.21.15.86:0/3114191605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d01a1e50 con 0x7f40d0103980 2024-04-17T07:35:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.603+0000 7f40cf200640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40d0103980 0x7f40d01a17e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f40c800cce0 tx=0x7f40c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.603+0000 7f40c7e00640 1 -- 172.21.15.86:0/3114191605 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c800f450 con 0x7f40d0103980 2024-04-17T07:35:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.603+0000 7f40c7e00640 1 -- 172.21.15.86:0/3114191605 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40c800faa0 con 0x7f40d0103980 2024-04-17T07:35:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.603+0000 7f40d4a00640 1 -- 172.21.15.86:0/3114191605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d01a2140 con 0x7f40d0103980 2024-04-17T07:35:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.603+0000 7f40c7e00640 1 -- 172.21.15.86:0/3114191605 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8018b70 con 0x7f40d0103980 2024-04-17T07:35:09.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.603+0000 7f40d4a00640 1 -- 172.21.15.86:0/3114191605 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d01a2640 con 0x7f40d0103980 2024-04-17T07:35:09.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.606+0000 7f40d81ce740 1 Processor -- start 2024-04-17T07:35:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.606+0000 7f40c7e00640 1 -- 172.21.15.86:0/3114191605 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f40c8018d10 con 0x7f40d0103980 2024-04-17T07:35:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.607+0000 7f40d81ce740 1 -- start start 2024-04-17T07:35:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.607+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135160 0x55c6d4135580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.607+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135b50 0x55c6d413a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.607+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d413d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.607+0000 7f40d81ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6d409b020 con 0x55c6d4135b50 2024-04-17T07:35:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.607+0000 7f40d81ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c6d4041460 con 0x55c6d413acc0 2024-04-17T07:35:09.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.607+0000 7f40d81ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c6d4096fe0 con 0x55c6d4135160 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.607+0000 7f40ce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135b50 0x55c6d413a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.607+0000 7f40ce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135b50 0x55c6d413a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51394/0 (socket says 172.21.15.86:51394) 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.607+0000 7f40ce800640 1 -- 172.21.15.86:0/2694425080 learned_addr learned my addr 172.21.15.86:0/2694425080 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40c5e00640 1 --2- 172.21.15.86:0/2694425080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135160 0x55c6d4135580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40ce800640 1 -- 172.21.15.86:0/2694425080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135160 msgr2=0x55c6d4135580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40ce800640 1 --2- 172.21.15.86:0/2694425080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135160 0x55c6d4135580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40ce800640 1 -- 172.21.15.86:0/2694425080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 msgr2=0x55c6d413d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40ce800640 1 --2- 172.21.15.86:0/2694425080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d413d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40ce800640 1 -- 172.21.15.86:0/2694425080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d4049040 con 0x55c6d4135b50 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40c5e00640 1 --2- 172.21.15.86:0/2694425080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135160 0x55c6d4135580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:09.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40c7e00640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40b40a3bb0 0x7f40b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40ce800640 1 --2- 172.21.15.86:0/2694425080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135b50 0x55c6d413a650 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009900 tx=0x7f40c0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e9a90269082bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40c7e00640 1 -- 172.21.15.86:0/3114191605 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f40c80d0c70 con 0x7f40d0103980 2024-04-17T07:35:09.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.608+0000 7f40c5e00640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40b40a3bb0 0x7f40b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.609+0000 7f40bf400640 1 -- 172.21.15.86:0/2694425080 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0044020 con 0x55c6d4135b50 2024-04-17T07:35:09.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.609+0000 7f40bf400640 1 -- 172.21.15.86:0/2694425080 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40c0005040 con 0x55c6d4135b50 2024-04-17T07:35:09.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.609+0000 7f40bf400640 1 -- 172.21.15.86:0/2694425080 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0004900 con 0x55c6d4135b50 2024-04-17T07:35:09.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 -- 172.21.15.86:0/2694425080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135b50 msgr2=0x55c6d413a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2694425080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135b50 0x55c6d413a650 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009900 tx=0x7f40c0005e80 comp rx=0 tx=0).stop 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40c5e00640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40b40a3bb0 0x7f40b40a6070 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55c6d41363b0 tx=0x7f40b800a270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 -- 172.21.15.86:0/2694425080 shutdown_connections 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2694425080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d413d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2694425080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135b50 0x55c6d413a650 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2694425080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135160 0x55c6d4135580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 -- 172.21.15.86:0/2694425080 >> 172.21.15.86:0/2694425080 conn(0x55c6d41306d0 msgr2=0x55c6d4130ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 -- 172.21.15.86:0/2694425080 shutdown_connections 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 -- 172.21.15.86:0/2694425080 wait complete. 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 -- start start 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.610+0000 7f40d81ce740 1 Processor -- start 2024-04-17T07:35:09.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.611+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.611+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 0x55c6d413fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.611+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d4146b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.611+0000 7f40d81ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6d413dc10 con 0x55c6d4135160 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.611+0000 7f40d81ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c6d4054830 con 0x55c6d413acc0 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.611+0000 7f40d81ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c6d413dd90 con 0x55c6d4135b50 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.611+0000 7f40d81ce740 10 client.0 _ll_register_callbacks cb 0x55c6d4022fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.611+0000 7f40ce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.611+0000 7f40ce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51406/0 (socket says 172.21.15.86:51406) 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.611+0000 7f40ce800640 1 -- 172.21.15.86:0/1194628810 learned_addr learned my addr 172.21.15.86:0/1194628810 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40cf200640 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d4146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40ce800640 1 -- 172.21.15.86:0/1194628810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 msgr2=0x55c6d413fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40c5e00640 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 0x55c6d413fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40ce800640 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 0x55c6d413fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40ce800640 1 -- 172.21.15.86:0/1194628810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 msgr2=0x55c6d4146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40ce800640 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d4146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40ce800640 1 -- 172.21.15.86:0/1194628810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d4058920 con 0x55c6d4135160 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40cf200640 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d4146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40c5e00640 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 0x55c6d413fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40ce800640 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413f600 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f40c0006010 tx=0x7f40c0004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753825681d60c88b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40b3e00640 1 -- 172.21.15.86:0/1194628810 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0044030 con 0x55c6d4135160 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40b3e00640 1 -- 172.21.15.86:0/1194628810 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f40c0040470 con 0x55c6d4135160 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.612+0000 7f40d81ce740 1 -- 172.21.15.86:0/1194628810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40c0005040 con 0x55c6d4135160 2024-04-17T07:35:09.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40b3e00640 1 -- 172.21.15.86:0/1194628810 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f40c0059050 con 0x55c6d4135160 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40d81ce740 10 client.20148 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40d81ce740 1 -- 172.21.15.86:0/1194628810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6d40267f0 con 0x55c6d4135160 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40b3e00640 1 -- 172.21.15.86:0/1194628810 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40c0005040 con 0x55c6d4135160 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40b3e00640 1 -- 172.21.15.86:0/1194628810 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f40c00346a0 con 0x55c6d4135160 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40d81ce740 10 client.20148 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40d81ce740 10 client.20148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40d81ce740 20 client.20148 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40d81ce740 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6d414ac50 0x55c6d414d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40d81ce740 4 client.20148 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.613+0000 7f40d81ce740 1 -- 172.21.15.86:0/1194628810 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6d40a8a40 con 0x55c6d414ac50 2024-04-17T07:35:09.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.614+0000 7f40c5e00640 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6d414ac50 0x55c6d414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.615+0000 7f40c5e00640 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6d414ac50 0x55c6d414d040 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e9ea6eea985c64 server_cookie=e67e20492afe6eaa in_seq=0 out_seq=0 2024-04-17T07:35:09.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.615+0000 7f40b3e00640 10 client.20148 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:35:09.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40b3e00640 1 -- 172.21.15.86:0/1194628810 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6d40a8a40 con 0x55c6d414ac50 2024-04-17T07:35:09.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40b3e00640 10 client.20148 handle_command_reply: tid=0 2024-04-17T07:35:09.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 client.20148 shutdown 2024-04-17T07:35:09.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 -- 172.21.15.86:0/1194628810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 msgr2=0x55c6d413f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413f600 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f40c0006010 tx=0x7f40c0004970 comp rx=0 tx=0).stop 2024-04-17T07:35:09.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 -- 172.21.15.86:0/1194628810 shutdown_connections 2024-04-17T07:35:09.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6d414ac50 0x55c6d414d040 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d4146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 0x55c6d413fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 --2- 172.21.15.86:0/1194628810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413f600 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40b3e00640 0 client.20148 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:35:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 -- 172.21.15.86:0/1194628810 >> 172.21.15.86:0/1194628810 conn(0x55c6d41306d0 msgr2=0x55c6d412f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 -- 172.21.15.86:0/1194628810 shutdown_connections 2024-04-17T07:35:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 1 -- 172.21.15.86:0/1194628810 wait complete. 2024-04-17T07:35:09.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.617+0000 7f40d81ce740 20 client.20148 trim_cache size 0 max 16384 2024-04-17T07:35:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.632+0000 7f40d81ce740 1 Processor -- start 2024-04-17T07:35:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.632+0000 7f40d81ce740 1 -- start start 2024-04-17T07:35:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.632+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.632+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 0x55c6d4146b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.632+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d413f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.632+0000 7f40d81ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01ac9b0 con 0x55c6d4135160 2024-04-17T07:35:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.632+0000 7f40d81ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01ae4f0 con 0x55c6d413acc0 2024-04-17T07:35:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.632+0000 7f40d81ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01ad080 con 0x55c6d4135b50 2024-04-17T07:35:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40ce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40ce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51420/0 (socket says 172.21.15.86:51420) 2024-04-17T07:35:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40ce800640 1 -- 172.21.15.86:0/2802561999 learned_addr learned my addr 172.21.15.86:0/2802561999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40c5e00640 1 --2- 172.21.15.86:0/2802561999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 0x55c6d4146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40cf200640 1 --2- 172.21.15.86:0/2802561999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d413f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40ce800640 1 -- 172.21.15.86:0/2802561999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 msgr2=0x55c6d4146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40ce800640 1 --2- 172.21.15.86:0/2802561999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 0x55c6d4146b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40ce800640 1 -- 172.21.15.86:0/2802561999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 msgr2=0x55c6d413f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40ce800640 1 --2- 172.21.15.86:0/2802561999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d413f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40ce800640 1 -- 172.21.15.86:0/2802561999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d3dad3f0 con 0x55c6d4135160 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40ce800640 1 --2- 172.21.15.86:0/2802561999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413ec20 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f40c0034ac0 tx=0x7f40c002c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e37b5b87e3cdc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.633+0000 7f40bea00640 1 -- 172.21.15.86:0/2802561999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0056020 con 0x55c6d4135160 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40bea00640 1 -- 172.21.15.86:0/2802561999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40c0004d70 con 0x55c6d4135160 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40bea00640 1 -- 172.21.15.86:0/2802561999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0044040 con 0x55c6d4135160 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40d81ce740 1 -- 172.21.15.86:0/2802561999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 msgr2=0x55c6d413ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2802561999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413ec20 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f40c0034ac0 tx=0x7f40c002c360 comp rx=0 tx=0).stop 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40d81ce740 1 -- 172.21.15.86:0/2802561999 shutdown_connections 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2802561999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d413acc0 0x55c6d413f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2802561999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135b50 0x55c6d4146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2802561999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d4135160 0x55c6d413ec20 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40d81ce740 1 -- 172.21.15.86:0/2802561999 >> 172.21.15.86:0/2802561999 conn(0x55c6d41306d0 msgr2=0x55c6d41341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40d81ce740 1 -- 172.21.15.86:0/2802561999 shutdown_connections 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.634+0000 7f40d81ce740 1 -- 172.21.15.86:0/2802561999 wait complete. 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.635+0000 7f40d81ce740 1 -- start start 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.635+0000 7f40d81ce740 1 Processor -- start 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.635+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135160 0x55c6d4143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.635+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d4135b50 0x55c6d4144110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.635+0000 7f40d81ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d413acc0 0x55c6d4145120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.635+0000 7f40d81ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01ade20 con 0x55c6d413acc0 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.635+0000 7f40d81ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01ad750 con 0x55c6d4135b50 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.635+0000 7f40d81ce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01aa980 con 0x55c6d4135160 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.635+0000 7f40d81ce740 10 client.0 _ll_register_callbacks cb 0x7f40c0005520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40cf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d413acc0 0x55c6d4145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40cf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d413acc0 0x55c6d4145120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51436/0 (socket says 172.21.15.86:51436) 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40cf200640 1 -- 172.21.15.86:0/2440113159 learned_addr learned my addr 172.21.15.86:0/2440113159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40cf200640 1 -- 172.21.15.86:0/2440113159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135160 msgr2=0x55c6d4143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40cf200640 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135160 0x55c6d4143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40cf200640 1 -- 172.21.15.86:0/2440113159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d4135b50 msgr2=0x55c6d4144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40cf200640 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d4135b50 0x55c6d4144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40cf200640 1 -- 172.21.15.86:0/2440113159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d3e777d0 con 0x55c6d413acc0 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40cf200640 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d413acc0 0x55c6d4145120 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f40c800ab20 tx=0x7f40c800fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb19e892778392ab server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40b3e00640 1 -- 172.21.15.86:0/2440113159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c80a0f80 con 0x55c6d413acc0 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40b3e00640 1 -- 172.21.15.86:0/2440113159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f40c80bd0c0 con 0x55c6d413acc0 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.636+0000 7f40d81ce740 1 -- 172.21.15.86:0/2440113159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40c0004d70 con 0x55c6d413acc0 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40b3e00640 1 -- 172.21.15.86:0/2440113159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f40c80a5840 con 0x55c6d413acc0 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40d81ce740 10 client.20154 fetch_fsmap learned FSMap version 28 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40d81ce740 1 -- 172.21.15.86:0/2440113159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6d4146910 con 0x55c6d413acc0 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40b3e00640 1 -- 172.21.15.86:0/2440113159 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40c0004d70 con 0x55c6d413acc0 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40b3e00640 1 -- 172.21.15.86:0/2440113159 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8363+0+0 (secure 0 0 0) 0x7f40c8061ba0 con 0x55c6d413acc0 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40d81ce740 10 client.20154 fetch_fsmap finished waiting for FSMap version 28 2024-04-17T07:35:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40d81ce740 10 client.20154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-17T07:35:09.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40d81ce740 20 client.20154 populate_metadata read hostname 'smithi086' 2024-04-17T07:35:09.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6d41b9ae0 0x55c6d41bbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:09.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40d81ce740 4 client.20154 mds_command: new command op to 34490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-17T07:35:09.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.637+0000 7f40d81ce740 1 -- 172.21.15.86:0/2440113159 --> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6d40a28b0 con 0x55c6d41b9ae0 2024-04-17T07:35:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.638+0000 7f40ce800640 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6d41b9ae0 0x55c6d41bbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40ce800640 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6d41b9ae0 0x55c6d41bbef0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5748455fb6658de1 server_cookie=54e4f85635456a2f in_seq=0 out_seq=0 2024-04-17T07:35:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40b3e00640 10 client.20154 ms_handle_connect on v2:172.21.15.98:6838/809350102 2024-04-17T07:35:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40b3e00640 1 -- 172.21.15.86:0/2440113159 <== mds.0 v2:172.21.15.98:6838/809350102 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6d40a28b0 con 0x55c6d41b9ae0 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40b3e00640 10 client.20154 handle_command_reply: tid=0 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40d81ce740 1 client.20154 shutdown 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40d81ce740 1 -- 172.21.15.86:0/2440113159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d413acc0 msgr2=0x55c6d4145120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d413acc0 0x55c6d4145120 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f40c800ab20 tx=0x7f40c800fd10 comp rx=0 tx=0).stop 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40d81ce740 1 -- 172.21.15.86:0/2440113159 shutdown_connections 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.98:6838/809350102,v1:172.21.15.98:6839/809350102] conn(0x55c6d41b9ae0 0x55c6d41bbef0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6d413acc0 0x55c6d4145120 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6d4135b50 0x55c6d4144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40d81ce740 1 --2- 172.21.15.86:0/2440113159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c6d4135160 0x55c6d4143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40b3e00640 0 client.20154 ms_handle_reset on v2:172.21.15.98:6838/809350102 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40d81ce740 1 -- 172.21.15.86:0/2440113159 >> 172.21.15.86:0/2440113159 conn(0x55c6d41306d0 msgr2=0x55c6d412f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.639+0000 7f40d81ce740 1 -- 172.21.15.86:0/2440113159 shutdown_connections 2024-04-17T07:35:09.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d81ce740 1 -- 172.21.15.86:0/2440113159 wait complete. 2024-04-17T07:35:09.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d81ce740 20 client.20154 trim_cache size 0 max 16384 2024-04-17T07:35:09.645 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-17T07:35:09.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d4a00640 1 -- 172.21.15.86:0/3114191605 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40b40a3bb0 msgr2=0x7f40b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d4a00640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40b40a3bb0 0x7f40b40a6070 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55c6d41363b0 tx=0x7f40b800a270 comp rx=0 tx=0).stop 2024-04-17T07:35:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d4a00640 1 -- 172.21.15.86:0/3114191605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40d0103980 msgr2=0x7f40d01a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d4a00640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40d0103980 0x7f40d01a17e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f40c800cce0 tx=0x7f40c800a620 comp rx=0 tx=0).stop 2024-04-17T07:35:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d4a00640 1 -- 172.21.15.86:0/3114191605 shutdown_connections 2024-04-17T07:35:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d4a00640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f40b40a3bb0 0x7f40b40a6070 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d4a00640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40d0103980 0x7f40d01a17e0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d4a00640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f40d0102fc0 0x7f40d019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d4a00640 1 --2- 172.21.15.86:0/3114191605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40d0101e70 0x7f40d019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.640+0000 7f40d4a00640 1 -- 172.21.15.86:0/3114191605 >> 172.21.15.86:0/3114191605 conn(0x7f40d00fd400 msgr2=0x7f40d0074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.641+0000 7f40d4a00640 1 -- 172.21.15.86:0/3114191605 shutdown_connections 2024-04-17T07:35:09.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:09.641+0000 7f40d4a00640 1 -- 172.21.15.86:0/3114191605 wait complete. 2024-04-17T07:35:09.655 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:09 smithi086 ceph-mon[31733]: pgmap v1063: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-17T07:35:09.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:09 smithi098 ceph-mon[35418]: pgmap v1063: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-17T07:35:09.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:09 smithi103 ceph-mon[35495]: pgmap v1063: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-17T07:35:10.659 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-17T07:35:10.659 INFO:tasks.fwd_scrub:done joining 2024-04-17T07:35:10.659 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-17T07:35:10.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:35:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.892+0000 7fcc4b200640 1 -- 172.21.15.86:0/2786367101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc4c107ac0 msgr2=0x7fcc4c109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.892+0000 7fcc4b200640 1 --2- 172.21.15.86:0/2786367101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc4c107ac0 0x7fcc4c109f10 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009ca0 tx=0x7fcc4402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:35:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.892+0000 7fcc4b200640 1 -- 172.21.15.86:0/2786367101 shutdown_connections 2024-04-17T07:35:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.892+0000 7fcc4b200640 1 --2- 172.21.15.86:0/2786367101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc4c107ac0 0x7fcc4c109f10 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.892+0000 7fcc4b200640 1 --2- 172.21.15.86:0/2786367101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc4c105190 0x7fcc4c107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.892+0000 7fcc4b200640 1 --2- 172.21.15.86:0/2786367101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc4c102860 0x7fcc4c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.892+0000 7fcc4b200640 1 -- 172.21.15.86:0/2786367101 >> 172.21.15.86:0/2786367101 conn(0x7fcc4c0fc230 msgr2=0x7fcc4c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.892+0000 7fcc4b200640 1 -- 172.21.15.86:0/2786367101 shutdown_connections 2024-04-17T07:35:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.892+0000 7fcc4b200640 1 -- 172.21.15.86:0/2786367101 wait complete. 2024-04-17T07:35:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.893+0000 7fcc4b200640 1 Processor -- start 2024-04-17T07:35:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.893+0000 7fcc4b200640 1 -- start start 2024-04-17T07:35:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc4c102860 0x7fcc4c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc4c105190 0x7fcc4c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:10.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc4c107ac0 0x7fcc4c1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c10d930 con 0x7fcc4c107ac0 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c10d7b0 con 0x7fcc4c105190 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c10dab0 con 0x7fcc4c102860 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc4c107ac0 0x7fcc4c1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc4c107ac0 0x7fcc4c1a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51446/0 (socket says 172.21.15.86:51446) 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4a800640 1 -- 172.21.15.86:0/2324273958 learned_addr learned my addr 172.21.15.86:0/2324273958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4a800640 1 -- 172.21.15.86:0/2324273958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc4c102860 msgr2=0x7fcc4c19a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4a800640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc4c102860 0x7fcc4c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4a800640 1 -- 172.21.15.86:0/2324273958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc4c105190 msgr2=0x7fcc4c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4a800640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc4c105190 0x7fcc4c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.894+0000 7fcc4a800640 1 -- 172.21.15.86:0/2324273958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c1a1e40 con 0x7fcc4c107ac0 2024-04-17T07:35:10.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.895+0000 7fcc4a800640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc4c107ac0 0x7fcc4c1a17d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fcc4402c700 tx=0x7fcc440328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.895+0000 7fcc43400640 1 -- 172.21.15.86:0/2324273958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44041020 con 0x7fcc4c107ac0 2024-04-17T07:35:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.895+0000 7fcc43400640 1 -- 172.21.15.86:0/2324273958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc44003e20 con 0x7fcc4c107ac0 2024-04-17T07:35:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.895+0000 7fcc43400640 1 -- 172.21.15.86:0/2324273958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44039040 con 0x7fcc4c107ac0 2024-04-17T07:35:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.895+0000 7fcc4b200640 1 -- 172.21.15.86:0/2324273958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4c1a20d0 con 0x7fcc4c107ac0 2024-04-17T07:35:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.895+0000 7fcc4b200640 1 -- 172.21.15.86:0/2324273958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4c1a2570 con 0x7fcc4c107ac0 2024-04-17T07:35:10.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.896+0000 7fcc4b200640 1 -- 172.21.15.86:0/2324273958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4c1095b0 con 0x7fcc4c107ac0 2024-04-17T07:35:10.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.898+0000 7fcc43400640 1 -- 172.21.15.86:0/2324273958 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fcc44044020 con 0x7fcc4c107ac0 2024-04-17T07:35:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.899+0000 7fcc43400640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc300a3c30 0x7fcc300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.900+0000 7fcc43400640 1 -- 172.21.15.86:0/2324273958 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fcc440f2fe0 con 0x7fcc4c107ac0 2024-04-17T07:35:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.900+0000 7fcc49e00640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc300a3c30 0x7fcc300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:10.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.900+0000 7fcc49e00640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc300a3c30 0x7fcc300a60f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c005fd0 tx=0x7fcc3c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:10.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:10.903+0000 7fcc43400640 1 -- 172.21.15.86:0/2324273958 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcc4404b020 con 0x7fcc4c107ac0 2024-04-17T07:35:11.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.107+0000 7fcc4b200640 1 -- 172.21.15.86:0/2324273958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc4c02d880 con 0x7fcc4c107ac0 2024-04-17T07:35:11.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.109+0000 7fcc43400640 1 -- 172.21.15.86:0/2324273958 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7fcc440333a0 con 0x7fcc4c107ac0 2024-04-17T07:35:11.110 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:35:11.111 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:35:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.112+0000 7fcc4b200640 1 -- 172.21.15.86:0/2324273958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc300a3c30 msgr2=0x7fcc300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.112+0000 7fcc4b200640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc300a3c30 0x7fcc300a60f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c005fd0 tx=0x7fcc3c005850 comp rx=0 tx=0).stop 2024-04-17T07:35:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.112+0000 7fcc4b200640 1 -- 172.21.15.86:0/2324273958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc4c107ac0 msgr2=0x7fcc4c1a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.112+0000 7fcc4b200640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc4c107ac0 0x7fcc4c1a17d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fcc4402c700 tx=0x7fcc440328a0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.113+0000 7fcc4b200640 1 -- 172.21.15.86:0/2324273958 shutdown_connections 2024-04-17T07:35:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.113+0000 7fcc4b200640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fcc300a3c30 0x7fcc300a60f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.113+0000 7fcc4b200640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc4c107ac0 0x7fcc4c1a17d0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.113+0000 7fcc4b200640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc4c105190 0x7fcc4c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.113+0000 7fcc4b200640 1 --2- 172.21.15.86:0/2324273958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc4c102860 0x7fcc4c19a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.113+0000 7fcc4b200640 1 -- 172.21.15.86:0/2324273958 >> 172.21.15.86:0/2324273958 conn(0x7fcc4c0fc230 msgr2=0x7fcc4c102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:11.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.113+0000 7fcc4b200640 1 -- 172.21.15.86:0/2324273958 shutdown_connections 2024-04-17T07:35:11.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.113+0000 7fcc4b200640 1 -- 172.21.15.86:0/2324273958 wait complete. 2024-04-17T07:35:11.132 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:35:11.133 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:35:11.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.353+0000 7f767ca00640 1 -- 172.21.15.86:0/3337095506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76781041c0 msgr2=0x7f76781045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.353+0000 7f767ca00640 1 --2- 172.21.15.86:0/3337095506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76781041c0 0x7f76781045e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7668009900 tx=0x7f766802bc40 comp rx=0 tx=0).stop 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.353+0000 7f767ca00640 1 -- 172.21.15.86:0/3337095506 shutdown_connections 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.353+0000 7f767ca00640 1 --2- 172.21.15.86:0/3337095506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7678105d70 0x7f767810c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.353+0000 7f767ca00640 1 --2- 172.21.15.86:0/3337095506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76781053b0 0x7f7678105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.353+0000 7f767ca00640 1 --2- 172.21.15.86:0/3337095506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76781041c0 0x7f76781045e0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.353+0000 7f767ca00640 1 -- 172.21.15.86:0/3337095506 >> 172.21.15.86:0/3337095506 conn(0x7f76780ff750 msgr2=0x7f7678101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.353+0000 7f767ca00640 1 -- 172.21.15.86:0/3337095506 shutdown_connections 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.353+0000 7f767ca00640 1 -- 172.21.15.86:0/3337095506 wait complete. 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.354+0000 7f767ca00640 1 Processor -- start 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.354+0000 7f767ca00640 1 -- start start 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.354+0000 7f767ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76781041c0 0x7f767819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:11.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.354+0000 7f767ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76781053b0 0x7f767819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:11.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.354+0000 7f767ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678105d70 0x7f76781a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:11.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.354+0000 7f767ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f767810fcd0 con 0x7f76781053b0 2024-04-17T07:35:11.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.354+0000 7f767ca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f767810fb50 con 0x7f76781041c0 2024-04-17T07:35:11.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.354+0000 7f767ca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f767810fe50 con 0x7f7678105d70 2024-04-17T07:35:11.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.354+0000 7f7675400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76781053b0 0x7f767819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:11.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f7675400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76781053b0 0x7f767819a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51474/0 (socket says 172.21.15.86:51474) 2024-04-17T07:35:11.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f7675400640 1 -- 172.21.15.86:0/1107202960 learned_addr learned my addr 172.21.15.86:0/1107202960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:11.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f7675e00640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76781041c0 0x7f767819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:11.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f7676800640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678105d70 0x7f76781a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:11.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f7675400640 1 -- 172.21.15.86:0/1107202960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678105d70 msgr2=0x7f76781a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:11.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f7675400640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678105d70 0x7f76781a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f7675400640 1 -- 172.21.15.86:0/1107202960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76781041c0 msgr2=0x7f767819a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:11.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f7675400640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76781041c0 0x7f767819a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f7675400640 1 -- 172.21.15.86:0/1107202960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76781a1fd0 con 0x7f76781053b0 2024-04-17T07:35:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f7675400640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76781053b0 0x7f767819a8f0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f766000c920 tx=0x7f766000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.355+0000 7f766ea00640 1 -- 172.21.15.86:0/1107202960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660007c70 con 0x7f76781053b0 2024-04-17T07:35:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.356+0000 7f767ca00640 1 -- 172.21.15.86:0/1107202960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76781a22c0 con 0x7f76781053b0 2024-04-17T07:35:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.356+0000 7f766ea00640 1 -- 172.21.15.86:0/1107202960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7660007e10 con 0x7f76781053b0 2024-04-17T07:35:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.356+0000 7f766ea00640 1 -- 172.21.15.86:0/1107202960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660018820 con 0x7f76781053b0 2024-04-17T07:35:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.356+0000 7f767ca00640 1 -- 172.21.15.86:0/1107202960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781a2810 con 0x7f76781053b0 2024-04-17T07:35:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.357+0000 7f767ca00640 1 -- 172.21.15.86:0/1107202960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f767810b530 con 0x7f76781053b0 2024-04-17T07:35:11.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.359+0000 7f766ea00640 1 -- 172.21.15.86:0/1107202960 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f76600189c0 con 0x7f76781053b0 2024-04-17T07:35:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.360+0000 7f766ea00640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f765c0a3bb0 0x7f765c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.360+0000 7f766ea00640 1 -- 172.21.15.86:0/1107202960 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f76600d09c0 con 0x7f76781053b0 2024-04-17T07:35:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.360+0000 7f7675e00640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f765c0a3bb0 0x7f765c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.361+0000 7f7675e00640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f765c0a3bb0 0x7f765c0a6070 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7668006010 tx=0x7f766802c810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.365+0000 7f766ea00640 1 -- 172.21.15.86:0/1107202960 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7660014030 con 0x7f76781053b0 2024-04-17T07:35:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.568+0000 7f767ca00640 1 -- 172.21.15.86:0/1107202960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7678062b60 con 0x7f76781053b0 2024-04-17T07:35:11.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.569+0000 7f766ea00640 1 -- 172.21.15.86:0/1107202960 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f766009cfc0 con 0x7f76781053b0 2024-04-17T07:35:11.570 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:35:11.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 -- 172.21.15.86:0/1107202960 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f765c0a3bb0 msgr2=0x7f765c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f765c0a3bb0 0x7f765c0a6070 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7668006010 tx=0x7f766802c810 comp rx=0 tx=0).stop 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 -- 172.21.15.86:0/1107202960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76781053b0 msgr2=0x7f767819a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76781053b0 0x7f767819a8f0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f766000c920 tx=0x7f766000cde0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 -- 172.21.15.86:0/1107202960 shutdown_connections 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f765c0a3bb0 0x7f765c0a6070 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7678105d70 0x7f76781a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76781053b0 0x7f767819a8f0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 --2- 172.21.15.86:0/1107202960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f76781041c0 0x7f767819a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 -- 172.21.15.86:0/1107202960 >> 172.21.15.86:0/1107202960 conn(0x7f76780ff750 msgr2=0x7f7678108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 -- 172.21.15.86:0/1107202960 shutdown_connections 2024-04-17T07:35:11.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.572+0000 7f767ca00640 1 -- 172.21.15.86:0/1107202960 wait complete. 2024-04-17T07:35:11.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:35:11.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:11 smithi086 ceph-mon[31733]: pgmap v1064: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-17T07:35:11.674 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:11 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2324273958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:35:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:11 smithi098 ceph-mon[35418]: pgmap v1064: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-17T07:35:11.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:11 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2324273958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:35:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:11 smithi103 ceph-mon[35495]: pgmap v1064: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-17T07:35:11.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:11 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2324273958' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:35:11.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.813+0000 7f2fdbe00640 1 -- 172.21.15.86:0/3675787939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fdc078500 msgr2=0x7f2fdc078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.813+0000 7f2fdbe00640 1 --2- 172.21.15.86:0/3675787939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fdc078500 0x7f2fdc078da0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009ca0 tx=0x7f2fd402c0f0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.813+0000 7f2fdbe00640 1 -- 172.21.15.86:0/3675787939 shutdown_connections 2024-04-17T07:35:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.813+0000 7f2fdbe00640 1 --2- 172.21.15.86:0/3675787939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fdc078500 0x7f2fdc078da0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.813+0000 7f2fdbe00640 1 --2- 172.21.15.86:0/3675787939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc077b60 0x7f2fdc077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.813+0000 7f2fdbe00640 1 --2- 172.21.15.86:0/3675787939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fdc076920 0x7f2fdc076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.813+0000 7f2fdbe00640 1 -- 172.21.15.86:0/3675787939 >> 172.21.15.86:0/3675787939 conn(0x7f2fdc0ff610 msgr2=0x7f2fdc101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.813+0000 7f2fdbe00640 1 -- 172.21.15.86:0/3675787939 shutdown_connections 2024-04-17T07:35:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.813+0000 7f2fdbe00640 1 -- 172.21.15.86:0/3675787939 wait complete. 2024-04-17T07:35:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.814+0000 7f2fdbe00640 1 Processor -- start 2024-04-17T07:35:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.814+0000 7f2fdbe00640 1 -- start start 2024-04-17T07:35:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.815+0000 7f2fdbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fdc076920 0x7f2fdc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.815+0000 7f2fdbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc077b60 0x7f2fdc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.815+0000 7f2fdbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fdc078500 0x7f2fdc1a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.815+0000 7f2fdbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc113fe0 con 0x7f2fdc078500 2024-04-17T07:35:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.815+0000 7f2fdbe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc113e60 con 0x7f2fdc077b60 2024-04-17T07:35:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.815+0000 7f2fdbe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc114160 con 0x7f2fdc076920 2024-04-17T07:35:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.815+0000 7f2fdb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fdc078500 0x7f2fdc1a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.815+0000 7f2fdb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fdc078500 0x7f2fdc1a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51502/0 (socket says 172.21.15.86:51502) 2024-04-17T07:35:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.815+0000 7f2fdb400640 1 -- 172.21.15.86:0/2032144146 learned_addr learned my addr 172.21.15.86:0/2032144146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.815+0000 7f2fdaa00640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fdc076920 0x7f2fdc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.816+0000 7f2fda000640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc077b60 0x7f2fdc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.816+0000 7f2fdb400640 1 -- 172.21.15.86:0/2032144146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fdc076920 msgr2=0x7f2fdc19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.816+0000 7f2fdb400640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fdc076920 0x7f2fdc19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.816+0000 7f2fdb400640 1 -- 172.21.15.86:0/2032144146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc077b60 msgr2=0x7f2fdc19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.816+0000 7f2fdb400640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc077b60 0x7f2fdc19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.816+0000 7f2fdb400640 1 -- 172.21.15.86:0/2032144146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc1a6250 con 0x7f2fdc078500 2024-04-17T07:35:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.816+0000 7f2fdb400640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fdc078500 0x7f2fdc1a5be0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4032380 tx=0x7f2fd4032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.816+0000 7f2fe0a00640 1 -- 172.21.15.86:0/2032144146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4041020 con 0x7f2fdc078500 2024-04-17T07:35:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.816+0000 7f2fe0a00640 1 -- 172.21.15.86:0/2032144146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fd4003e20 con 0x7f2fdc078500 2024-04-17T07:35:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.816+0000 7f2fdbe00640 1 -- 172.21.15.86:0/2032144146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc1a64e0 con 0x7f2fdc078500 2024-04-17T07:35:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.817+0000 7f2fe0a00640 1 -- 172.21.15.86:0/2032144146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4039040 con 0x7f2fdc078500 2024-04-17T07:35:11.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.817+0000 7f2fdbe00640 1 -- 172.21.15.86:0/2032144146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc1a69d0 con 0x7f2fdc078500 2024-04-17T07:35:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.818+0000 7f2fdbe00640 1 -- 172.21.15.86:0/2032144146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fdc110a10 con 0x7f2fdc078500 2024-04-17T07:35:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.820+0000 7f2fe0a00640 1 -- 172.21.15.86:0/2032144146 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f2fd4044020 con 0x7f2fdc078500 2024-04-17T07:35:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.822+0000 7f2fe0a00640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2fc00a3bb0 0x7f2fc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.822+0000 7f2fe0a00640 1 -- 172.21.15.86:0/2032144146 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f2fd40f2910 con 0x7f2fdc078500 2024-04-17T07:35:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.822+0000 7f2fdaa00640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2fc00a3bb0 0x7f2fc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.823+0000 7f2fdaa00640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2fc00a3bb0 0x7f2fc00a6070 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009bd0 tx=0x7f2fc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:11.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:11.827+0000 7f2fe0a00640 1 -- 172.21.15.86:0/2032144146 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2fd404b020 con 0x7f2fdc078500 2024-04-17T07:35:12.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.031+0000 7f2fdbe00640 1 -- 172.21.15.86:0/2032144146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2fdc062b60 con 0x7f2fdc078500 2024-04-17T07:35:12.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.031+0000 7f2fe0a00640 1 -- 172.21.15.86:0/2032144146 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f2fd40bef10 con 0x7f2fdc078500 2024-04-17T07:35:12.032 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:35:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 -- 172.21.15.86:0/2032144146 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2fc00a3bb0 msgr2=0x7f2fc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2fc00a3bb0 0x7f2fc00a6070 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009bd0 tx=0x7f2fc4005f90 comp rx=0 tx=0).stop 2024-04-17T07:35:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 -- 172.21.15.86:0/2032144146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fdc078500 msgr2=0x7f2fdc1a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fdc078500 0x7f2fdc1a5be0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4032380 tx=0x7f2fd4032820 comp rx=0 tx=0).stop 2024-04-17T07:35:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 -- 172.21.15.86:0/2032144146 shutdown_connections 2024-04-17T07:35:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f2fc00a3bb0 0x7f2fc00a6070 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fdc078500 0x7f2fdc1a5be0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fdc077b60 0x7f2fdc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 --2- 172.21.15.86:0/2032144146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fdc076920 0x7f2fdc19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 -- 172.21.15.86:0/2032144146 >> 172.21.15.86:0/2032144146 conn(0x7f2fdc0ff610 msgr2=0x7f2fdc101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 -- 172.21.15.86:0/2032144146 shutdown_connections 2024-04-17T07:35:12.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.034+0000 7f2fdbe00640 1 -- 172.21.15.86:0/2032144146 wait complete. 2024-04-17T07:35:12.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:35:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.273+0000 7f1b12000640 1 -- 172.21.15.86:0/1718247381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0c105d70 msgr2=0x7f1b0c10c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.273+0000 7f1b12000640 1 --2- 172.21.15.86:0/1718247381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0c105d70 0x7f1b0c10c600 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009ca0 tx=0x7f1b0402c100 comp rx=0 tx=0).stop 2024-04-17T07:35:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.273+0000 7f1b12000640 1 -- 172.21.15.86:0/1718247381 shutdown_connections 2024-04-17T07:35:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.273+0000 7f1b12000640 1 --2- 172.21.15.86:0/1718247381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0c105d70 0x7f1b0c10c600 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.273+0000 7f1b12000640 1 --2- 172.21.15.86:0/1718247381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b0c1053b0 0x7f1b0c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.273+0000 7f1b12000640 1 --2- 172.21.15.86:0/1718247381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c1041c0 0x7f1b0c1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.273+0000 7f1b12000640 1 -- 172.21.15.86:0/1718247381 >> 172.21.15.86:0/1718247381 conn(0x7f1b0c0ff750 msgr2=0x7f1b0c101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:12.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.273+0000 7f1b12000640 1 -- 172.21.15.86:0/1718247381 shutdown_connections 2024-04-17T07:35:12.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.273+0000 7f1b12000640 1 -- 172.21.15.86:0/1718247381 wait complete. 2024-04-17T07:35:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.274+0000 7f1b12000640 1 Processor -- start 2024-04-17T07:35:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.274+0000 7f1b12000640 1 -- start start 2024-04-17T07:35:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b12000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c1041c0 0x7f1b0c19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b12000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b0c1053b0 0x7f1b0c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b12000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0c105d70 0x7f1b0c1a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b12000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c10fd00 con 0x7f1b0c105d70 2024-04-17T07:35:12.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b12000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c10fb80 con 0x7f1b0c1053b0 2024-04-17T07:35:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b12000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c10fe80 con 0x7f1b0c1041c0 2024-04-17T07:35:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b0be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0c105d70 0x7f1b0c1a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b0be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0c105d70 0x7f1b0c1a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51512/0 (socket says 172.21.15.86:51512) 2024-04-17T07:35:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b0be00640 1 -- 172.21.15.86:0/2511399578 learned_addr learned my addr 172.21.15.86:0/2511399578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b0b400640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c1041c0 0x7f1b0c19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.275+0000 7f1b02a00640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b0c1053b0 0x7f1b0c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b0be00640 1 -- 172.21.15.86:0/2511399578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c1041c0 msgr2=0x7f1b0c19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b0be00640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c1041c0 0x7f1b0c19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b0be00640 1 -- 172.21.15.86:0/2511399578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b0c1053b0 msgr2=0x7f1b0c19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b0be00640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b0c1053b0 0x7f1b0c19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b0be00640 1 -- 172.21.15.86:0/2511399578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c1a1ee0 con 0x7f1b0c105d70 2024-04-17T07:35:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b02a00640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b0c1053b0 0x7f1b0c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b0be00640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0c105d70 0x7f1b0c1a1870 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1b04032520 tx=0x7f1b04034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b08c00640 1 -- 172.21.15.86:0/2511399578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04039040 con 0x7f1b0c105d70 2024-04-17T07:35:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b08c00640 1 -- 172.21.15.86:0/2511399578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b04044070 con 0x7f1b0c105d70 2024-04-17T07:35:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b12000640 1 -- 172.21.15.86:0/2511399578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0c1a2170 con 0x7f1b0c105d70 2024-04-17T07:35:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b08c00640 1 -- 172.21.15.86:0/2511399578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b040043c0 con 0x7f1b0c105d70 2024-04-17T07:35:12.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.276+0000 7f1b12000640 1 -- 172.21.15.86:0/2511399578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0c1a26b0 con 0x7f1b0c105d70 2024-04-17T07:35:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.278+0000 7f1b12000640 1 -- 172.21.15.86:0/2511399578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0c10b480 con 0x7f1b0c105d70 2024-04-17T07:35:12.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.280+0000 7f1b08c00640 1 -- 172.21.15.86:0/2511399578 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f1b04004560 con 0x7f1b0c105d70 2024-04-17T07:35:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.281+0000 7f1b08c00640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1af00a3c30 0x7f1af00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.281+0000 7f1b08c00640 1 -- 172.21.15.86:0/2511399578 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f1b040f2c30 con 0x7f1b0c105d70 2024-04-17T07:35:12.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.281+0000 7f1b0b400640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1af00a3c30 0x7f1af00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:12.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.282+0000 7f1b0b400640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1af00a3c30 0x7f1af00a60f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1afc005fd0 tx=0x7f1afc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:12.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.286+0000 7f1b08c00640 1 -- 172.21.15.86:0/2511399578 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1b040f9080 con 0x7f1b0c105d70 2024-04-17T07:35:12.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.496+0000 7f1b12000640 1 -- 172.21.15.86:0/2511399578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1b0c062b60 con 0x7f1b0c105d70 2024-04-17T07:35:12.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.496+0000 7f1b08c00640 1 -- 172.21.15.86:0/2511399578 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f1b0404e020 con 0x7f1b0c105d70 2024-04-17T07:35:12.497 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:35:12.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 -- 172.21.15.86:0/2511399578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1af00a3c30 msgr2=0x7f1af00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1af00a3c30 0x7f1af00a60f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1afc005fd0 tx=0x7f1afc005850 comp rx=0 tx=0).stop 2024-04-17T07:35:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 -- 172.21.15.86:0/2511399578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0c105d70 msgr2=0x7f1b0c1a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0c105d70 0x7f1b0c1a1870 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f1b04032520 tx=0x7f1b04034880 comp rx=0 tx=0).stop 2024-04-17T07:35:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 -- 172.21.15.86:0/2511399578 shutdown_connections 2024-04-17T07:35:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f1af00a3c30 0x7f1af00a60f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b0c105d70 0x7f1b0c1a1870 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b0c1053b0 0x7f1b0c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 --2- 172.21.15.86:0/2511399578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b0c1041c0 0x7f1b0c19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 -- 172.21.15.86:0/2511399578 >> 172.21.15.86:0/2511399578 conn(0x7f1b0c0ff750 msgr2=0x7f1b0c108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 -- 172.21.15.86:0/2511399578 shutdown_connections 2024-04-17T07:35:12.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.499+0000 7f1b12000640 1 -- 172.21.15.86:0/2511399578 wait complete. 2024-04-17T07:35:12.520 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:35:12.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.739+0000 7fbce4000640 1 -- 172.21.15.86:0/1881114652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcdc1053b0 msgr2=0x7fbcdc1077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.739+0000 7fbce4000640 1 --2- 172.21.15.86:0/1881114652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcdc1053b0 0x7fbcdc1077a0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009900 tx=0x7fbcc802bc50 comp rx=0 tx=0).stop 2024-04-17T07:35:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.739+0000 7fbce4000640 1 -- 172.21.15.86:0/1881114652 shutdown_connections 2024-04-17T07:35:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.739+0000 7fbce4000640 1 --2- 172.21.15.86:0/1881114652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcdc107ce0 0x7fbcdc10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.739+0000 7fbce4000640 1 --2- 172.21.15.86:0/1881114652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcdc1053b0 0x7fbcdc1077a0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.739+0000 7fbce4000640 1 --2- 172.21.15.86:0/1881114652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc102a80 0x7fbcdc104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.739+0000 7fbce4000640 1 -- 172.21.15.86:0/1881114652 >> 172.21.15.86:0/1881114652 conn(0x7fbcdc0fc410 msgr2=0x7fbcdc0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.739+0000 7fbce4000640 1 -- 172.21.15.86:0/1881114652 shutdown_connections 2024-04-17T07:35:12.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.739+0000 7fbce4000640 1 -- 172.21.15.86:0/1881114652 wait complete. 2024-04-17T07:35:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.740+0000 7fbce4000640 1 Processor -- start 2024-04-17T07:35:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.740+0000 7fbce4000640 1 -- start start 2024-04-17T07:35:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.740+0000 7fbce4000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcdc102a80 0x7fbcdc198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.740+0000 7fbce4000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc1053b0 0x7fbcdc1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:12.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.740+0000 7fbce4000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcdc107ce0 0x7fbcdc19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.740+0000 7fbce4000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc10dae0 con 0x7fbcdc107ce0 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.740+0000 7fbce4000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc10d960 con 0x7fbcdc1053b0 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.740+0000 7fbce4000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc10dc60 con 0x7fbcdc102a80 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcdc107ce0 0x7fbcdc19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcdc107ce0 0x7fbcdc19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51536/0 (socket says 172.21.15.86:51536) 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce2000640 1 -- 172.21.15.86:0/427450525 learned_addr learned my addr 172.21.15.86:0/427450525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce0c00640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc1053b0 0x7fbcdc1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce1600640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcdc102a80 0x7fbcdc198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce2000640 1 -- 172.21.15.86:0/427450525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcdc102a80 msgr2=0x7fbcdc198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce2000640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcdc102a80 0x7fbcdc198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce2000640 1 -- 172.21.15.86:0/427450525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc1053b0 msgr2=0x7fbcdc1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce2000640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc1053b0 0x7fbcdc1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce2000640 1 -- 172.21.15.86:0/427450525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc19fe80 con 0x7fbcdc107ce0 2024-04-17T07:35:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.741+0000 7fbce1600640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcdc102a80 0x7fbcdc198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.742+0000 7fbce2000640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcdc107ce0 0x7fbcdc19f810 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400cad0 tx=0x7fbcd400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.742+0000 7fbcda000640 1 -- 172.21.15.86:0/427450525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400ae20 con 0x7fbcdc107ce0 2024-04-17T07:35:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.742+0000 7fbcda000640 1 -- 172.21.15.86:0/427450525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcd4005020 con 0x7fbcdc107ce0 2024-04-17T07:35:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.742+0000 7fbcda000640 1 -- 172.21.15.86:0/427450525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400f6b0 con 0x7fbcdc107ce0 2024-04-17T07:35:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.742+0000 7fbce4000640 1 -- 172.21.15.86:0/427450525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcdc1a0170 con 0x7fbcdc107ce0 2024-04-17T07:35:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.742+0000 7fbce4000640 1 -- 172.21.15.86:0/427450525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcdc1a0640 con 0x7fbcdc107ce0 2024-04-17T07:35:12.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.743+0000 7fbce4000640 1 -- 172.21.15.86:0/427450525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcdc0fdf00 con 0x7fbcdc107ce0 2024-04-17T07:35:12.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.745+0000 7fbcda000640 1 -- 172.21.15.86:0/427450525 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fbcd40051d0 con 0x7fbcdc107ce0 2024-04-17T07:35:12.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.747+0000 7fbcda000640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbcc40a3c30 0x7fbcc40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.747+0000 7fbcda000640 1 -- 172.21.15.86:0/427450525 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fbcd40d0850 con 0x7fbcdc107ce0 2024-04-17T07:35:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.748+0000 7fbce1600640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbcc40a3c30 0x7fbcc40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:12.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.748+0000 7fbce1600640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbcc40a3c30 0x7fbcc40a60f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00058c0 tx=0x7fbcd0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:12.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.752+0000 7fbcda000640 1 -- 172.21.15.86:0/427450525 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbcd4014030 con 0x7fbcdc107ce0 2024-04-17T07:35:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1107202960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:35:12.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:12 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2032144146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:35:12.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1107202960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:35:12.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:12 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2032144146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:35:12.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1107202960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:35:12.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:12 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2032144146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:35:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.953+0000 7fbce4000640 1 -- 172.21.15.86:0/427450525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbcdc062c00 con 0x7fbcdc107ce0 2024-04-17T07:35:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.954+0000 7fbcda000640 1 -- 172.21.15.86:0/427450525 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fbcd409ce50 con 0x7fbcdc107ce0 2024-04-17T07:35:12.954 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:35:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 -- 172.21.15.86:0/427450525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbcc40a3c30 msgr2=0x7fbcc40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbcc40a3c30 0x7fbcc40a60f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00058c0 tx=0x7fbcd0005d30 comp rx=0 tx=0).stop 2024-04-17T07:35:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 -- 172.21.15.86:0/427450525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcdc107ce0 msgr2=0x7fbcdc19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:12.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcdc107ce0 0x7fbcdc19f810 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400cad0 tx=0x7fbcd400cf90 comp rx=0 tx=0).stop 2024-04-17T07:35:12.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 -- 172.21.15.86:0/427450525 shutdown_connections 2024-04-17T07:35:12.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fbcc40a3c30 0x7fbcc40a60f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcdc107ce0 0x7fbcdc19f810 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbcdc1053b0 0x7fbcdc1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 --2- 172.21.15.86:0/427450525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbcdc102a80 0x7fbcdc198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:12.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 -- 172.21.15.86:0/427450525 >> 172.21.15.86:0/427450525 conn(0x7fbcdc0fc410 msgr2=0x7fbcdc1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:12.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 -- 172.21.15.86:0/427450525 shutdown_connections 2024-04-17T07:35:12.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:12.957+0000 7fbce4000640 1 -- 172.21.15.86:0/427450525 wait complete. 2024-04-17T07:35:12.978 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-17T07:35:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.182+0000 7fb729c00640 1 -- 172.21.15.86:0/874479948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7241097b0 msgr2=0x7fb724109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.182+0000 7fb729c00640 1 --2- 172.21.15.86:0/874479948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7241097b0 0x7fb724109c30 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009900 tx=0x7fb70c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:35:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.182+0000 7fb729c00640 1 -- 172.21.15.86:0/874479948 shutdown_connections 2024-04-17T07:35:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.182+0000 7fb729c00640 1 --2- 172.21.15.86:0/874479948 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72410a170 0x7fb724110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.182+0000 7fb729c00640 1 --2- 172.21.15.86:0/874479948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7241097b0 0x7fb724109c30 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.182+0000 7fb729c00640 1 --2- 172.21.15.86:0/874479948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7241085c0 0x7fb7241089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.182+0000 7fb729c00640 1 -- 172.21.15.86:0/874479948 >> 172.21.15.86:0/874479948 conn(0x7fb724077570 msgr2=0x7fb724079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:13.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.182+0000 7fb729c00640 1 -- 172.21.15.86:0/874479948 shutdown_connections 2024-04-17T07:35:13.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.182+0000 7fb729c00640 1 -- 172.21.15.86:0/874479948 wait complete. 2024-04-17T07:35:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.183+0000 7fb729c00640 1 Processor -- start 2024-04-17T07:35:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.183+0000 7fb729c00640 1 -- start start 2024-04-17T07:35:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb729c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7241085c0 0x7fb72419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb729c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7241097b0 0x7fb72419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb729c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72410a170 0x7fb7241a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb729c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7241140f0 con 0x7fb7241085c0 2024-04-17T07:35:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb729c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb724113f70 con 0x7fb7241097b0 2024-04-17T07:35:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb729c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb724114270 con 0x7fb72410a170 2024-04-17T07:35:13.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb722a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7241085c0 0x7fb72419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb722a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7241085c0 0x7fb72419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51546/0 (socket says 172.21.15.86:51546) 2024-04-17T07:35:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb722a00640 1 -- 172.21.15.86:0/83250898 learned_addr learned my addr 172.21.15.86:0/83250898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:35:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb71a000640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7241097b0 0x7fb72419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.184+0000 7fb723400640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72410a170 0x7fb7241a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:13.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb722a00640 1 -- 172.21.15.86:0/83250898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72410a170 msgr2=0x7fb7241a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb722a00640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72410a170 0x7fb7241a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:13.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb722a00640 1 -- 172.21.15.86:0/83250898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7241097b0 msgr2=0x7fb72419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb722a00640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7241097b0 0x7fb72419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb722a00640 1 -- 172.21.15.86:0/83250898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7241a6380 con 0x7fb7241085c0 2024-04-17T07:35:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb723400640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72410a170 0x7fb7241a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:35:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb722a00640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7241085c0 0x7fb72419e760 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb71400d770 tx=0x7fb71400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb71be00640 1 -- 172.21.15.86:0/83250898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714017070 con 0x7fb7241085c0 2024-04-17T07:35:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb71be00640 1 -- 172.21.15.86:0/83250898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb714009c30 con 0x7fb7241085c0 2024-04-17T07:35:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb729c00640 1 -- 172.21.15.86:0/83250898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7241a6670 con 0x7fb7241085c0 2024-04-17T07:35:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb71be00640 1 -- 172.21.15.86:0/83250898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714003d00 con 0x7fb7241085c0 2024-04-17T07:35:13.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.185+0000 7fb729c00640 1 -- 172.21.15.86:0/83250898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7241a6bc0 con 0x7fb7241085c0 2024-04-17T07:35:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.187+0000 7fb729c00640 1 -- 172.21.15.86:0/83250898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72410f8e0 con 0x7fb7241085c0 2024-04-17T07:35:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.189+0000 7fb71be00640 1 -- 172.21.15.86:0/83250898 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7fb714003ea0 con 0x7fb7241085c0 2024-04-17T07:35:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.190+0000 7fb71be00640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7080a3bb0 0x7fb7080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:35:13.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.190+0000 7fb71be00640 1 -- 172.21.15.86:0/83250898 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7fb7140d05a0 con 0x7fb7241085c0 2024-04-17T07:35:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.190+0000 7fb71a000640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7080a3bb0 0x7fb7080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:35:13.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.191+0000 7fb71a000640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7080a3bb0 0x7fb7080a6070 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb72419fc70 tx=0x7fb70c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:35:13.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.194+0000 7fb71be00640 1 -- 172.21.15.86:0/83250898 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb71409cba0 con 0x7fb7241085c0 2024-04-17T07:35:13.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.452+0000 7fb729c00640 1 -- 172.21.15.86:0/83250898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fb724062b60 con 0x7fb7241085c0 2024-04-17T07:35:13.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.471+0000 7fb71be00640 1 -- 172.21.15.86:0/83250898 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+745528 (secure 0 0 0) 0x7fb714009de0 con 0x7fb7241085c0 2024-04-17T07:35:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:35:13.485 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:35:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 20, 2024-04-17T07:35:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "active_gid": 14152, 2024-04-17T07:35:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "active_name": "x", 2024-04-17T07:35:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addrs": { 2024-04-17T07:35:13.485 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6800", 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 561620030 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6801", 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 561620030 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-17T07:35:13.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addr": "172.21.15.86:6801/561620030", 2024-04-17T07:35:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "active_change": "2024-04-17T07:02:26.882232+0000", 2024-04-17T07:35:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "active_mgr_features": 4540701547738038271, 2024-04-17T07:35:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "available": true, 2024-04-17T07:35:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "standbys": [ 2024-04-17T07:35:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 24109, 2024-04-17T07:35:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "z", 2024-04-17T07:35:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540701547738038271, 2024-04-17T07:35:13.487 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.488 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.489 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.490 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.491 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.492 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.493 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-17T07:35:13.494 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.495 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.496 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-17T07:35:13.497 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-17T07:35:13.498 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-17T07:35:13.499 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-17T07:35:13.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.502 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.503 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-17T07:35:13.504 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-17T07:35:13.505 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.506 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-17T07:35:13.507 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.508 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-17T07:35:13.509 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.510 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.513 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.514 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:13.515 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.516 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-17T07:35:13.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:13.518 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.519 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.520 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-17T07:35:13.521 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.522 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-17T07:35:13.524 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-17T07:35:13.525 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-17T07:35:13.526 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-17T07:35:13.528 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.529 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.530 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-17T07:35:13.532 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.533 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-17T07:35:13.534 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.536 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.538 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-17T07:35:13.539 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-17T07:35:13.540 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-17T07:35:13.541 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.542 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.543 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.546 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-17T07:35:13.547 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-17T07:35:13.548 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-17T07:35:13.549 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.550 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.551 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.552 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.554 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:13.555 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.556 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.558 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-17T07:35:13.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.561 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.562 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.563 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.564 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.565 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.566 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-17T07:35:13.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-17T07:35:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.568 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.569 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-17T07:35:13.570 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.571 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.572 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.573 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.574 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.575 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.576 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.579 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-17T07:35:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.583 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.584 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-17T07:35:13.585 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.586 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.587 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.588 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.589 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.590 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.591 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-17T07:35:13.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.593 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-17T07:35:13.594 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.595 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.596 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.600 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.601 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.602 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.604 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.606 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.607 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.608 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-17T07:35:13.609 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.611 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.612 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.613 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-17T07:35:13.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.618 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.619 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.620 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.623 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.624 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.625 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.626 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.627 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.628 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.629 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.630 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.631 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-17T07:35:13.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.633 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-17T07:35:13.634 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.636 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.637 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.638 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.640 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.642 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.643 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.644 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.648 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.651 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.654 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.655 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.658 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.659 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.660 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.661 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-17T07:35:13.662 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.664 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.665 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.666 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.667 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.669 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-17T07:35:13.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.673 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.674 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.675 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.678 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.680 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.681 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-17T07:35:13.682 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-17T07:35:13.683 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.685 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-17T07:35:13.686 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-17T07:35:13.687 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.688 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.689 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.690 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.691 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.693 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.694 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.695 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.696 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.698 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.699 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.700 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-17T07:35:13.701 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.702 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.703 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.704 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.705 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.706 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.707 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.709 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.710 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.711 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-17T07:35:13.712 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-17T07:35:13.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.714 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.715 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.716 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.717 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.719 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.720 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.721 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.722 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.723 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.724 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.725 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.726 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-17T07:35:13.727 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.729 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.730 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.731 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.732 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.733 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.734 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.735 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.737 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.738 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.741 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.742 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.743 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.744 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.746 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.747 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.749 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.750 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.752 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.753 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-17T07:35:13.754 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.755 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.756 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.757 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.758 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.759 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.760 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.761 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.762 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.763 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.764 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.765 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.767 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.768 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.769 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.770 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.771 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:13.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-17T07:35:13.773 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.775 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.776 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.777 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.778 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.779 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.780 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.781 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-17T07:35:13.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.783 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.785 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.786 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.787 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.788 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.789 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.790 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.792 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.793 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.794 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-17T07:35:13.795 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.796 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.797 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.800 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.803 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.804 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.805 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-17T07:35:13.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-17T07:35:13.807 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.808 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-17T07:35:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-17T07:35:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.809 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-17T07:35:13.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-17T07:35:13.811 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.813 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.816 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-17T07:35:13.817 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-17T07:35:13.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-17T07:35:13.819 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.820 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.821 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.822 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.823 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.824 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.825 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.826 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.828 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.829 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.830 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-17T07:35:13.831 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-17T07:35:13.832 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.840 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-17T07:35:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.844 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.845 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.846 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.847 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.848 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.849 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-17T07:35:13.850 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-17T07:35:13.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.853 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.854 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.857 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.858 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.859 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.860 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.861 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.862 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.863 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.864 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.865 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.866 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.867 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.869 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-17T07:35:13.870 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:13.871 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.874 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.878 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.879 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-17T07:35:13.880 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-17T07:35:13.881 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-17T07:35:13.882 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.885 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.886 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.887 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.888 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.889 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.890 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.891 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.892 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-17T07:35:13.893 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.894 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.895 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.896 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.897 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-17T07:35:13.899 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-17T07:35:13.900 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.902 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.903 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.905 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.906 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.907 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.908 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.909 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.910 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.911 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.912 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.913 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-17T07:35:13.914 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-17T07:35:13.915 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.916 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.917 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.918 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.919 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.920 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.922 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.924 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.925 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-17T07:35:13.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-17T07:35:13.928 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.929 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 34109, 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "y", 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540701547738038271, 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:13.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.931 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.932 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.933 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.934 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.935 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-17T07:35:13.936 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-17T07:35:13.937 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.938 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-17T07:35:13.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.940 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.942 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.944 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.945 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.946 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:13.947 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-17T07:35:13.948 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-17T07:35:13.949 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.950 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-17T07:35:13.951 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-17T07:35:13.952 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.953 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.954 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.955 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.956 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:13.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.958 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-17T07:35:13.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.960 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-17T07:35:13.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:13.964 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.965 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.967 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.968 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.969 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.970 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-17T07:35:13.971 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-17T07:35:13.972 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-17T07:35:13.973 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.974 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:13.975 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-17T07:35:13.976 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.977 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.978 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.979 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.980 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.981 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.982 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-17T07:35:13.983 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-17T07:35:13.984 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-17T07:35:13.985 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-17T07:35:13.986 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.988 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.989 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.990 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.991 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.992 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-17T07:35:13.993 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.994 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-17T07:35:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-17T07:35:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-17T07:35:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-17T07:35:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.997 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-17T07:35:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-17T07:35:13.998 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-17T07:35:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:13.999 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-17T07:35:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-17T07:35:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-17T07:35:14.000 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.003 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.004 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.005 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.006 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.010 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.012 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-17T07:35:14.013 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-17T07:35:14.014 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-17T07:35:14.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.019 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.020 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.022 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.024 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.025 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.026 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-17T07:35:14.027 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-17T07:35:14.028 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.029 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.030 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.031 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.032 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.033 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.034 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.035 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.038 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.039 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.040 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.041 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.042 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.043 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.045 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.046 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.047 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.048 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.049 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.050 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.051 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-17T07:35:14.053 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.054 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.055 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.056 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.057 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.058 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.059 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.060 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.061 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-17T07:35:14.062 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.063 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.064 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.065 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.066 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.067 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.068 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.069 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.071 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.072 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.073 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.075 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.077 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.078 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-17T07:35:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.079 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.082 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.083 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.085 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.086 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-17T07:35:14.089 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.090 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-17T07:35:14.091 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.092 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.093 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.094 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.095 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.096 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.097 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.099 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.100 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.101 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.102 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.103 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.104 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-17T07:35:14.105 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.106 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-17T07:35:14.107 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.108 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.110 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.114 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-17T07:35:14.115 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-17T07:35:14.116 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.117 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.118 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.119 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.120 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.121 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.122 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.123 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.124 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.125 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-17T07:35:14.126 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.127 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:14.128 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.129 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-17T07:35:14.130 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.131 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.132 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-17T07:35:14.133 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.134 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.135 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.137 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.138 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.139 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.140 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.141 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-17T07:35:14.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.143 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.144 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.145 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.146 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.147 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.148 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.149 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.150 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.151 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.152 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.154 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.157 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.158 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.159 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.160 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.162 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.163 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-17T07:35:14.164 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.165 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.166 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.167 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.169 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.170 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.171 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.172 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.173 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.174 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-17T07:35:14.175 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.176 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.177 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.178 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.179 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.182 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.183 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.184 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-17T07:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.189 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.190 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.191 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.193 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.195 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.196 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.198 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.199 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.200 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.201 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.202 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.203 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.204 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.205 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.207 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.210 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.211 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.212 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.213 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.214 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.215 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.216 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.217 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:14.218 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-17T07:35:14.219 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.220 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.221 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.222 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.223 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.224 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.225 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.226 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.228 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.229 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.230 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.231 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-17T07:35:14.232 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.233 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.234 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.235 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.236 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.237 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.238 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.239 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.240 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.241 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-17T07:35:14.242 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-17T07:35:14.243 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-17T07:35:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.245 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.246 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.247 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.248 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.250 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.251 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.252 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.254 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-17T07:35:14.255 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-17T07:35:14.256 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.258 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.259 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.260 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.265 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.266 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.267 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.268 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.269 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.271 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.272 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.273 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.274 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.275 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.276 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.277 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.278 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.279 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.280 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.281 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.282 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.283 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.284 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.285 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.286 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.287 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.288 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.289 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.290 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.291 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.292 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.294 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-17T07:35:14.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-17T07:35:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-17T07:35:14.296 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.297 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.298 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.299 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.301 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.302 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.303 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.304 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.305 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.306 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.307 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.308 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.309 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.311 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.312 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.313 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.315 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.316 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.317 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.318 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.320 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.321 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.322 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.323 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.325 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.327 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.328 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.330 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.331 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.332 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.333 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.334 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-17T07:35:14.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.337 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.339 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.340 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.341 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.342 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.343 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.345 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.346 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.347 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.349 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.350 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.352 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.353 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.354 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.355 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.358 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.359 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-17T07:35:14.360 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.361 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.362 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.363 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.364 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.365 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.366 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.367 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.368 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.369 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.370 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.371 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "modules": [ 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard", 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "iostat", 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "nfs", 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "restful" 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.372 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.374 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.376 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.379 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-17T07:35:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-17T07:35:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-17T07:35:14.380 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.381 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.383 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.384 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.385 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.386 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-17T07:35:14.387 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.388 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.389 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T07:35:14.390 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.391 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.392 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.393 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.394 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.395 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.396 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.397 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.398 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.399 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.400 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.401 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.402 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-17T07:35:14.403 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:14.404 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-17T07:35:14.405 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.406 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.407 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.408 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.409 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.411 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.412 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:14.413 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-17T07:35:14.414 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.415 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.416 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.417 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-17T07:35:14.418 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:14.419 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-17T07:35:14.420 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-17T07:35:14.421 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.422 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.423 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.424 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.425 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.426 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.427 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.428 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-17T07:35:14.429 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-17T07:35:14.430 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.431 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.432 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.434 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.435 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.436 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.437 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.438 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.439 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.440 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.441 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.442 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-17T07:35:14.444 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-17T07:35:14.445 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.446 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.447 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.448 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.449 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.450 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.451 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-17T07:35:14.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.453 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.454 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.455 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.456 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.457 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-17T07:35:14.458 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-17T07:35:14.459 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.460 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.462 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.463 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.464 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.465 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.466 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.467 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.468 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.470 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.471 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.472 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.473 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.474 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.476 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.477 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.478 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.479 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-17T07:35:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-17T07:35:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.480 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.481 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.482 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.484 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.485 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.486 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.487 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.488 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.489 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.490 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.495 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.496 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.497 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.498 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.500 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.501 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-17T07:35:14.502 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.503 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.504 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.505 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.506 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.507 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.509 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.510 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-17T07:35:14.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-17T07:35:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-17T07:35:14.515 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-17T07:35:14.516 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-17T07:35:14.517 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-17T07:35:14.518 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-17T07:35:14.519 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.520 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.522 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.525 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-17T07:35:14.526 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-17T07:35:14.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-17T07:35:14.528 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.529 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.532 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.533 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-17T07:35:14.534 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.535 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.536 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.537 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.539 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.540 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.542 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.543 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.544 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.545 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.546 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.547 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.548 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.549 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.550 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.552 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.553 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.555 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.556 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-17T07:35:14.557 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-17T07:35:14.558 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-17T07:35:14.559 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-17T07:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-17T07:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-17T07:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-17T07:35:14.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.562 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.564 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.565 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.566 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.567 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.568 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.571 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-17T07:35:14.573 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.574 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.577 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.578 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.581 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.582 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.584 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.585 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:14.586 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.587 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.588 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.591 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.595 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.598 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.599 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.600 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.601 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.603 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.605 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.606 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.609 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.610 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.612 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-17T07:35:14.614 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-17T07:35:14.615 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.616 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.619 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.620 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.621 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.622 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.623 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.625 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.626 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.627 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.629 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.631 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.632 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.633 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.634 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.635 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.637 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.638 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.639 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.641 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.642 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.643 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.644 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.645 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.646 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.649 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.650 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-17T07:35:14.651 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.652 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.653 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.654 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.655 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.657 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.658 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.659 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.660 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.661 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.662 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.663 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.664 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.668 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.669 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.670 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.671 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-17T07:35:14.672 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-17T07:35:14.673 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-17T07:35:14.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-17T07:35:14.675 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.676 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.677 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.680 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.681 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.683 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.684 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.685 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.686 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.687 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.688 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-17T07:35:14.689 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.690 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.691 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.692 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.693 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.694 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.695 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.697 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.699 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.700 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-17T07:35:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.702 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.703 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.704 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.705 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.706 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.707 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-17T07:35:14.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-17T07:35:14.709 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.710 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.711 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.712 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.715 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.716 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.717 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.720 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.721 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-17T07:35:14.722 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.723 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.725 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.726 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.727 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.728 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.729 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.730 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.731 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.732 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.733 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.734 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.735 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.736 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.737 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.738 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.739 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-17T07:35:14.740 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-17T07:35:14.741 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.742 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.743 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.744 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.746 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.748 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.749 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.750 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.751 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.752 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-17T07:35:14.753 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.754 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.755 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.757 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.758 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.759 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-17T07:35:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.763 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.764 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.765 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-17T07:35:14.766 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.767 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.768 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.769 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.770 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.771 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.772 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.773 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.774 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.775 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.777 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.779 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.780 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.781 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.782 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.783 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.784 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.786 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.787 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.789 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.791 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.792 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.793 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.794 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.797 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.799 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.800 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-17T07:35:14.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.802 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.803 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.804 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.806 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.808 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-17T07:35:14.809 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-17T07:35:14.810 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-17T07:35:14.811 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-17T07:35:14.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "services": { 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard": "https://172.21.15.86:8443/" 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "always_on_modules": { 2024-04-17T07:35:14.814 INFO:teuthology.orchestra.run.smithi086.stdout: "octopus": [ 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "pacific": [ 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-17T07:35:14.815 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "quincy": [ 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-17T07:35:14.816 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "reef": [ 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-17T07:35:14.817 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "squid": [ 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-17T07:35:14.818 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "last_failure_osd_epoch": 3, 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "active_clients": [ 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3234336563 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.819 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "libcephsqlite", 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3393751077 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.820 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1168582328 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2573645088 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.821 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-17T07:35:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:14.822 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-17T07:35:14.822 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:35:14.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:13 smithi103 ceph-mon[35495]: pgmap v1065: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-17T07:35:14.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:13 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2511399578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:35:14.823 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:13 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/427450525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:35:14.824 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:14 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/83250898' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-17T07:35:14.824 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:13 smithi098 ceph-mon[35418]: pgmap v1065: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-17T07:35:14.824 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:13 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2511399578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:35:14.824 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:13 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/427450525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:35:14.824 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:14 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/83250898' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-17T07:35:14.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 -- 172.21.15.86:0/83250898 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7080a3bb0 msgr2=0x7fb7080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:14.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7080a3bb0 0x7fb7080a6070 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb72419fc70 tx=0x7fb70c005be0 comp rx=0 tx=0).stop 2024-04-17T07:35:14.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 -- 172.21.15.86:0/83250898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7241085c0 msgr2=0x7fb72419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:35:14.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7241085c0 0x7fb72419e760 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb71400d770 tx=0x7fb71400dc30 comp rx=0 tx=0).stop 2024-04-17T07:35:14.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 -- 172.21.15.86:0/83250898 shutdown_connections 2024-04-17T07:35:14.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7fb7080a3bb0 0x7fb7080a6070 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:14.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb72410a170 0x7fb7241a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:14.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb7241097b0 0x7fb72419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:14.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 --2- 172.21.15.86:0/83250898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7241085c0 0x7fb72419e760 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:35:14.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 -- 172.21.15.86:0/83250898 >> 172.21.15.86:0/83250898 conn(0x7fb724077570 msgr2=0x7fb72410cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:35:14.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 -- 172.21.15.86:0/83250898 shutdown_connections 2024-04-17T07:35:14.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:35:13.486+0000 7fb729c00640 1 -- 172.21.15.86:0/83250898 wait complete. 2024-04-17T07:35:14.878 DEBUG:tasks.check_counter:Getting stats from a 2024-04-17T07:35:14.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-17T07:35:14.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:13 smithi086 ceph-mon[31733]: pgmap v1065: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-17T07:35:14.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:13 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2511399578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:35:14.879 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:13 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/427450525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:35:14.880 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:14 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/83250898' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-17T07:35:15.365 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:35:15.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:15 smithi098 ceph-mon[35418]: pgmap v1066: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-17T07:35:15.759 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:15 smithi086 ceph-mon[31733]: pgmap v1066: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-17T07:35:15.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:15 smithi103 ceph-mon[35495]: pgmap v1066: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-17T07:35:17.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:17 smithi086 ceph-mon[31733]: pgmap v1067: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-17T07:35:17.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:17 smithi098 ceph-mon[35418]: pgmap v1067: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-17T07:35:17.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:17 smithi103 ceph-mon[35495]: pgmap v1067: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-17T07:35:18.491 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:35:18.491 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:18.491 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2727, 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2729, 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 816740, 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 867187, 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.510019538, 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.159872641, 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.121520663, 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.167114986, 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:18.492 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2727, 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.390232439, 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000143099 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1218, 2024-04-17T07:35:18.493 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1503, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 300194, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 965842, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.274648748, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.103987533, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.047807380, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.093550200, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1501, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.215767802, 2024-04-17T07:35:18.494 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000143749 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 578768 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 393, 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 385, 2024-04-17T07:35:18.495 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 830972, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 234046, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.135302168, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.062919558, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.058386106, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.017705347, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 385, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.058874492, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000152920 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.496 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 830704, 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 233728 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 6353, 2024-04-17T07:35:18.497 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.347412125, 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000054684 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-04-17T07:35:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-04-17T07:35:18.499 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-04-17T07:35:18.500 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-04-17T07:35:18.501 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-04-17T07:35:18.502 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-04-17T07:35:18.503 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-04-17T07:35:18.504 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.505 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-04-17T07:35:18.506 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-04-17T07:35:18.506 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-04-17T07:35:18.506 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-04-17T07:35:18.506 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-04-17T07:35:18.506 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 5, 2024-04-17T07:35:18.506 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-04-17T07:35:18.506 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-04-17T07:35:18.506 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-04-17T07:35:18.506 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-04-17T07:35:18.506 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4195901, 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 4195760, 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 5 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 3, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 0, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 1, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 0, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 0, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 0, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 0, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-04-17T07:35:18.508 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 48896, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215416 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.509 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.510 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-04-17T07:35:18.511 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.512 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-04-17T07:35:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.514 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.514 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.514 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-04-17T07:35:18.514 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.514 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.514 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.514 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.515 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-04-17T07:35:18.516 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.517 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.517 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.517 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-04-17T07:35:18.517 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.517 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.517 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.517 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-04-17T07:35:18.517 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.518 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 2, 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 1, 2024-04-17T07:35:18.521 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 2, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 4069, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:18.522 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:18.523 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:18.524 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 190207, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 53, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8792, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 168, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:18.525 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 8512, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 13, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-04-17T07:35:18.526 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 3945, 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 3945, 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3945, 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 8.034829148, 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002036712 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3945, 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3946 2024-04-17T07:35:18.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 3945, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 3945, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 2575, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 1368, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-04-17T07:35:18.528 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-04-17T07:35:18.529 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-04-17T07:35:18.530 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 93, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 2, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 5, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 7, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 2, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-04-17T07:35:18.531 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T07:35:18.532 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 4334, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1534874, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 4334, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-17T07:35:18.533 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 4334, 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1534874, 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-17T07:35:18.534 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3945, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1602, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3945, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1602, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-04-17T07:35:18.535 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3945, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 3945, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3945, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 3945, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-04-17T07:35:18.536 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.537 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55c1312d2780": { 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-17T07:35:18.538 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-17T07:35:18.539 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-17T07:35:18.539 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-17T07:35:18.539 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:18.539 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:18.539 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:18.539 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:18.539 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:18.539 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:18.539 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:35:19.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:19 smithi086 ceph-mon[31733]: pgmap v1068: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-17T07:35:19.741 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-17T07:35:19.742 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-17T07:35:19.742 DEBUG:tasks.check_counter:Getting stats from d 2024-04-17T07:35:19.742 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-17T07:35:19.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:19 smithi098 ceph-mon[35418]: pgmap v1068: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-17T07:35:19.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:19 smithi103 ceph-mon[35495]: pgmap v1068: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-17T07:35:20.215 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:35:21.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:21 smithi098 ceph-mon[35418]: pgmap v1069: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:21.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:21 smithi103 ceph-mon[35495]: pgmap v1069: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:21.800 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:21 smithi086 ceph-mon[31733]: pgmap v1069: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:22.864 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:35:22.864 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:22.864 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 393, 2024-04-17T07:35:22.864 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 386, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 809712, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 238000, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.142153458, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.061160256, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.070499339, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.016501173, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 386, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.058977982, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000152792 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.865 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 809712, 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 238000 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2, 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 294, 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 320, 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 208064, 2024-04-17T07:35:22.866 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.051781583, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.043310181, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.003666521, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 292, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.067763884, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000232068 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:22.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 208064 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.001006105, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-04-17T07:35:22.868 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000817761, 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T07:35:22.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:22.870 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:22.872 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 148704, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 29, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:22.873 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-17T07:35:22.874 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 391, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 751727, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 391, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 391, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 751727, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:22.875 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:22.876 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:35:23.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:23 smithi098 ceph-mon[35418]: pgmap v1070: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:23.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:23 smithi103 ceph-mon[35495]: pgmap v1070: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:23.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:23 smithi086 ceph-mon[31733]: pgmap v1070: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:23.905 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-17T07:35:23.905 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-17T07:35:23.905 DEBUG:tasks.check_counter:Getting stats from g 2024-04-17T07:35:23.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-17T07:35:24.361 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:35:25.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:25 smithi098 ceph-mon[35418]: pgmap v1071: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:25.776 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:25 smithi086 ceph-mon[31733]: pgmap v1071: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:25.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:25 smithi103 ceph-mon[35495]: pgmap v1071: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:26.886 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1368, 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1370, 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 459593, 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 435025, 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.334598545, 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.143062791, 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.067161601, 2024-04-17T07:35:26.887 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.096199664, 2024-04-17T07:35:26.888 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:26.888 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1368, 2024-04-17T07:35:26.888 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.210251991, 2024-04-17T07:35:26.888 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000153692 2024-04-17T07:35:26.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.888 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:26.888 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.888 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.888 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2981, 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2975, 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1488669, 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 1058877, 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.519713682, 2024-04-17T07:35:26.889 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.174998008, 2024-04-17T07:35:26.890 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.144757871, 2024-04-17T07:35:26.890 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.179026805, 2024-04-17T07:35:26.890 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:26.890 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2974, 2024-04-17T07:35:26.890 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.369075485, 2024-04-17T07:35:26.890 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000124100 2024-04-17T07:35:26.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.890 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:26.890 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 828528, 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 235200 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 9, 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 295, 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 4660, 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 671733, 2024-04-17T07:35:26.891 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-04-17T07:35:26.892 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-04-17T07:35:26.892 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.053610445, 2024-04-17T07:35:26.892 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.043840263, 2024-04-17T07:35:26.892 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.004692086, 2024-04-17T07:35:26.892 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000066566, 2024-04-17T07:35:26.892 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:26.892 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 294, 2024-04-17T07:35:26.892 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.062721402, 2024-04-17T07:35:26.892 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000213338 2024-04-17T07:35:26.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 670768 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-17T07:35:26.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 6385, 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.309331798, 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000048446 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-17T07:35:26.894 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.895 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.896 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.896 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-04-17T07:35:26.897 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-04-17T07:35:26.898 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-04-17T07:35:26.899 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T07:35:26.900 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-04-17T07:35:26.901 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-04-17T07:35:26.902 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-04-17T07:35:26.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 3, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-04-17T07:35:26.904 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4195369, 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 4194892, 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.905 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 3 2024-04-17T07:35:26.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.906 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-04-17T07:35:26.906 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3, 2024-04-17T07:35:26.906 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 3, 2024-04-17T07:35:26.906 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 0, 2024-04-17T07:35:26.906 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1, 2024-04-17T07:35:26.906 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 1, 2024-04-17T07:35:26.906 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 0, 2024-04-17T07:35:26.906 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 0, 2024-04-17T07:35:26.906 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 0, 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 0, 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 49024, 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215416 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-04-17T07:35:26.907 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.908 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.910 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.911 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.912 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.913 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.913 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-04-17T07:35:26.913 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.913 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.914 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.915 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-04-17T07:35:26.916 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.918 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-04-17T07:35:26.918 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.918 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.918 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.918 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-04-17T07:35:26.918 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.918 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.918 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.919 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-04-17T07:35:26.920 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.921 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.922 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.922 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:26.922 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.922 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-04-17T07:35:26.922 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-04-17T07:35:26.922 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 1, 2024-04-17T07:35:26.922 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 2651, 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:26.923 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:26.924 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 198287, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 57, 2024-04-17T07:35:26.926 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 8792, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 168, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:26.927 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 8512, 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 13, 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-04-17T07:35:26.928 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 3961, 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 3961, 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3961, 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 9.049807959, 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.002284728 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3961, 2024-04-17T07:35:26.929 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 3962 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 3961, 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 3961, 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 2587, 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 1372, 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-04-17T07:35:26.930 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-04-17T07:35:26.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-04-17T07:35:26.932 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 93, 2024-04-17T07:35:26.933 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 2, 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 6, 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 7, 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 1, 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-04-17T07:35:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-04-17T07:35:26.935 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 4354, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1537891, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 4354, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 4354, 2024-04-17T07:35:26.936 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1537891, 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3961, 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1070, 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3961, 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1070, 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.938 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 3961, 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 3961, 2024-04-17T07:35:26.939 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 3961, 2024-04-17T07:35:26.940 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 3961, 2024-04-17T07:35:26.940 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:26.940 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.940 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.940 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.940 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:26.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.940 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-04-17T07:35:26.940 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:26.940 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.941 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.942 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.942 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:26.942 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:26.942 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55dc56496780": { 2024-04-17T07:35:26.942 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:26.942 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-17T07:35:26.942 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:26.942 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-17T07:35:26.942 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-17T07:35:26.942 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:26.943 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:26.944 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:35:27.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:27 smithi086 ceph-mon[31733]: pgmap v1072: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:27.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:27 smithi098 ceph-mon[35418]: pgmap v1072: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:27.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:27 smithi103 ceph-mon[35495]: pgmap v1072: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:27.818 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-17T07:35:27.818 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-17T07:35:27.818 DEBUG:tasks.check_counter:Getting stats from j 2024-04-17T07:35:27.819 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-17T07:35:28.265 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.a/config 2024-04-17T07:35:29.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:29 smithi098 ceph-mon[35418]: pgmap v1073: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:29.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:29 smithi103 ceph-mon[35495]: pgmap v1073: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:29.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:29 smithi086 ceph-mon[31733]: pgmap v1073: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:30.417 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-17T07:35:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 4, 2024-04-17T07:35:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 4, 2024-04-17T07:35:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 2992, 2024-04-17T07:35:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 448, 2024-04-17T07:35:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-17T07:35:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-04-17T07:35:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.004232039, 2024-04-17T07:35:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000080182, 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.001392300, 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3, 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.001794001, 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000598000 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-17T07:35:30.419 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 448 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 2, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 296, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 320, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 208416, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.047805190, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.040270455, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.002371196, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-17T07:35:30.420 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 294, 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.038779000, 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000131901 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 208416 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 392, 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 387, 2024-04-17T07:35:30.421 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 807536, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 239472, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.131108618, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.063208530, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.050566784, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.017403372, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 386, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.066939758, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000173419 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:30.422 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 807536, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 239472 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:30.423 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:30.424 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:30.425 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 149752, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 31, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:30.426 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 394, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 752117, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 394, 2024-04-17T07:35:30.427 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-17T07:35:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-17T07:35:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 394, 2024-04-17T07:35:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 752117, 2024-04-17T07:35:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-17T07:35:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-17T07:35:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-17T07:35:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-17T07:35:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:30.428 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-17T07:35:30.428 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-17T07:35:31.297 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-17T07:35:31.297 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-17T07:35:31.297 DEBUG:tasks.check_counter:Getting stats from b 2024-04-17T07:35:31.297 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-17T07:35:31.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:31 smithi098 ceph-mon[35418]: pgmap v1074: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:31.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:31 smithi103 ceph-mon[35495]: pgmap v1074: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:31.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:31 smithi086 ceph-mon[31733]: pgmap v1074: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:35:33.580 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:33 smithi086 ceph-mon[31733]: pgmap v1075: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:33.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:33 smithi098 ceph-mon[35418]: pgmap v1075: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:33.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:33 smithi103 ceph-mon[35495]: pgmap v1075: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:34.379 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 14456, 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 14386, 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 4538623, 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 21145509, 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 30, 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 2.153499327, 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.719479442, 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.520256136, 2024-04-17T07:35:34.380 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.593262741, 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 14385, 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.369743539, 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000095220 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1882, 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001150284, 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000611 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 880592, 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 282448 2024-04-17T07:35:34.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 12728, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 12544, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 3479717, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 12897922, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 10, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 1.844679158, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.669240156, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.444724075, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.603031896, 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:34.382 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 12549, 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.160620229, 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000092487 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3016, 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001063778, 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000352 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 627072 2024-04-17T07:35:34.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 47442, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 46117, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 14527313, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 46583821, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 272, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 10, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 8.106660741, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 3.346846421, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 4.498938731, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.863693452, 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:34.384 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 46109, 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 4.726822272, 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000102514 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 56835, 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.025334994, 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000445 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 97472, 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 31616 2024-04-17T07:35:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.386 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-17T07:35:34.386 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-17T07:35:34.386 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-17T07:35:34.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.386 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 35174, 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 20.464945282, 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000581820 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1114, 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.123402937, 2024-04-17T07:35:34.387 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000110774 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 9119, 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 9119, 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 9119, 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 29.258733233, 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003208546 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2024-04-17T07:35:34.388 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 39, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 3, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 1426, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 1426, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 1426, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 1044, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 1044, 2024-04-17T07:35:34.389 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 4, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 14506, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 13271, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 22, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-04-17T07:35:34.390 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 34248, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 1017, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 1532204373, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 34248, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 1190, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 34, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 1156, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 33058, 2024-04-17T07:35:34.391 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 1311, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 3132, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 18837, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 12601, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 81, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 2853 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-17T07:35:34.392 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 8, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 446, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 267, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 179, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-17T07:35:34.393 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 34, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 3, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 20945, 2024-04-17T07:35:34.394 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 20945, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 26, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 26, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 3, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 52753257, 2024-04-17T07:35:34.395 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3817, 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 34.375803255, 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.009005974 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 1065, 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 1366, 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 301, 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 68, 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 73, 2024-04-17T07:35:34.396 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 5, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 1426, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 1697, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 271, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 1044, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 1324, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 280, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 68564, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 231800 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 10023, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 1, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 9119, 2024-04-17T07:35:34.397 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 335, 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 2, 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1282, 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 6.573518083, 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005127549 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 471, 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.349979815, 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000743056 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.398 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 179, 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.754672343, 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.004216046 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3975, 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 11.822572785, 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002974232 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.399 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.401 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-17T07:35:34.401 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.401 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.401 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.401 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-17T07:35:34.401 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.401 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.401 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.402 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-17T07:35:34.402 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.402 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.402 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.402 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-17T07:35:34.402 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 37, 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.124979713, 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003377830 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2446, 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 4.424885297, 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001809029 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 79, 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.327446138, 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.004144887 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 428, 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 4.146057918, 2024-04-17T07:35:34.404 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.009687051 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.015490075, 2024-04-17T07:35:34.405 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.007745037 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.003822305, 2024-04-17T07:35:34.406 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001911152 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.407 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 9, 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.052177360, 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005797484 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 209, 2024-04-17T07:35:34.408 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.663131401, 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003172877 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-04-17T07:35:34.409 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 4, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 3, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 1735, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 1735, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 1235, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:34.410 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:34.411 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:34.412 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 293194, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 119, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:34.413 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8792, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 168, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 3848072, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 36655, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-17T07:35:34.414 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 40291, 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 49355040, 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 37, 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 40291, 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 40291, 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 104.051666047, 2024-04-17T07:35:34.415 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002582503 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 40291, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 40969 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 40291, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 34462, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 5829, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 11, 2024-04-17T07:35:34.416 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 180, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 3963, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 185, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 3, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 1094, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 34282, 2024-04-17T07:35:34.417 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 22, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-17T07:35:34.418 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-17T07:35:34.419 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 93, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 9, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 12, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 12, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 967, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 262 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-04-17T07:35:34.420 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 23, 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 5, 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 571, 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 267, 2024-04-17T07:35:34.421 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 74622, 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 15790396, 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 74622, 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:34.422 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 74622, 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 15790396, 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:34.423 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 40291, 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 49384127, 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 40291, 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 49384127, 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:34.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 40291, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 40291, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 40291, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 40291, 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:34.425 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 267, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 29087, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 267, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:34.426 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 146, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 29087, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55cb07fc6500": { 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 20945, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 48558953, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:34.427 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 20945, 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3817, 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 48558953, 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:34.428 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-17T07:35:35.454 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-17T07:35:35.455 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-17T07:35:35.455 DEBUG:tasks.check_counter:Getting stats from e 2024-04-17T07:35:35.455 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-17T07:35:35.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:35 smithi098 ceph-mon[35418]: pgmap v1076: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:35.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:35 smithi103 ceph-mon[35495]: pgmap v1076: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:35.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:35 smithi086 ceph-mon[31733]: pgmap v1076: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:35.935 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:35:37.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:37 smithi103 ceph-mon[35495]: pgmap v1077: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:37.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:37 smithi086 ceph-mon[31733]: pgmap v1077: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:37.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:37 smithi098 ceph-mon[35418]: pgmap v1077: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:38.515 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-17T07:35:38.515 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:38.515 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 24, 2024-04-17T07:35:38.515 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 24, 2024-04-17T07:35:38.515 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 8428, 2024-04-17T07:35:38.515 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 10595, 2024-04-17T07:35:38.515 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 9, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.028312904, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000745660, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.008026083, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000633109, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 23, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.134860255, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005863489 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.516 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 448 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 36, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 329, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 7245, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 673500, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 13, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.047704890, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.027430348, 2024-04-17T07:35:38.517 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.013501554, 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000493859, 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 327, 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.110432797, 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000337714 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 36, 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000014517, 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000403 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 611872 2024-04-17T07:35:38.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1239, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1411, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1022684, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 1393946, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 74, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.463181117, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.205178924, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.231232268, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.029835985, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1410, 2024-04-17T07:35:38.519 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.176442886, 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000125136 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1702, 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000700229, 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000411 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 855520, 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 245312 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-17T07:35:38.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 50, 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.002061487, 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000041229 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000010225, 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000005112 2024-04-17T07:35:38.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 1, 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 1, 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000762494, 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000762494 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-17T07:35:38.522 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 11, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 11, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 11, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 11, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 1, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 1, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 2, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 2, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 1, 2024-04-17T07:35:38.523 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 1, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 1019, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 1532466553, 2024-04-17T07:35:38.524 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T07:35:38.525 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 2, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-17T07:35:38.526 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-04-17T07:35:38.527 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 3, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 3, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 2, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-04-17T07:35:38.528 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4195369, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.004921745, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002460872 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.529 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 14, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 14, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 0, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 12, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 13, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 1, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 11, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 11, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 0, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 1, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 1, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 51200, 2024-04-17T07:35:38.530 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 223608 2024-04-17T07:35:38.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.531 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-17T07:35:38.531 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 2, 2024-04-17T07:35:38.531 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-04-17T07:35:38.531 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 1, 2024-04-17T07:35:38.531 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 15, 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000762494, 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000762494 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.532 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.533 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-17T07:35:38.534 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.535 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.535 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.535 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-17T07:35:38.535 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.535 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.535 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.535 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-17T07:35:38.536 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.536 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.536 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.536 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-17T07:35:38.536 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.537 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.537 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.537 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.539 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.541 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.542 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-17T07:35:38.543 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-04-17T07:35:38.544 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 1, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 0, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 1, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 1, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 1237, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:38.545 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:38.546 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:38.547 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 247030, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 81, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8792, 2024-04-17T07:35:38.548 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 168, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 55448, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 215, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-17T07:35:38.549 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 38, 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 11371, 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 27, 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 38, 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 38, 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 7.799899033, 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.205260500 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 38, 2024-04-17T07:35:38.550 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 61 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 38, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 38, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 2, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 6, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-17T07:35:38.551 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 6, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-17T07:35:38.552 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-17T07:35:38.553 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 93, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 3, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 9, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 11, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 2, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 16, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-04-17T07:35:38.554 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 1 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-17T07:35:38.555 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 1295, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 896711, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 1295, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:38.556 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 1295, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 896711, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:38.557 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 38, 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 11371, 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 38, 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 11371, 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-17T07:35:38.558 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 38, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 38, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 38, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 38, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:38.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.560 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55db58654500": { 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 3, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1065, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 3, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:38.561 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 2, 2024-04-17T07:35:38.562 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1065, 2024-04-17T07:35:38.562 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:38.562 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:38.562 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:38.562 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:38.562 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:38.562 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:38.562 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-17T07:35:39.679 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-17T07:35:39.679 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-17T07:35:39.679 DEBUG:tasks.check_counter:Getting stats from h 2024-04-17T07:35:39.679 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-17T07:35:39.774 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:39 smithi098 ceph-mon[35418]: pgmap v1078: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:39.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:39 smithi103 ceph-mon[35495]: pgmap v1078: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:39.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:39 smithi086 ceph-mon[31733]: pgmap v1078: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:40.146 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:35:41.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:41 smithi086 ceph-mon[31733]: pgmap v1079: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:41.863 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:41 smithi098 ceph-mon[35418]: pgmap v1079: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:42.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:41 smithi103 ceph-mon[35495]: pgmap v1079: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1396, 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1436, 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1070083, 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 635095, 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 8, 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.470842661, 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.236492667, 2024-04-17T07:35:42.421 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.200044763, 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.030243090, 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1435, 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.240485208, 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000167585 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1900, 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000793786, 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000417 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 860080, 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 244528 2024-04-17T07:35:42.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 44, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 334, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 11282, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 702847, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 14, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.058099186, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.029235329, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.021716132, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000424787, 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:42.423 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 333, 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.112930052, 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000339129 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 51, 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000016205, 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000317 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 5248, 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 647360 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 84, 2024-04-17T07:35:42.424 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 86, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 14235, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 810108, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 75, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.077976539, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.006979545, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.064310013, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000849059, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 84, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.031617719, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000376401 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.425 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 68, 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000013571, 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000199 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-17T07:35:42.426 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 54, 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.002698287, 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000049968 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000012513, 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000006256 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:42.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 5, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 3, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.004484468, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001494822 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 2, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 11, 2024-04-17T07:35:42.428 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 12, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 12, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 12, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 2, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 2, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 2, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 9, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 5, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-04-17T07:35:42.429 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 2, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-17T07:35:42.430 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 1019, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 1532466553, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T07:35:42.431 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 2 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 3, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-04-17T07:35:42.432 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T07:35:42.433 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 5, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 5, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-04-17T07:35:42.434 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 2, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4195901, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.016511922, 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.004127980 2024-04-17T07:35:42.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 15, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 15, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 0, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 13, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 14, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 1, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 12, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 12, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 0, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 2, 2024-04-17T07:35:42.436 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 3, 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 1, 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 51564, 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 223608 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 10, 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 5, 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 19, 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 1, 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.437 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3, 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.004484468, 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001494822 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.438 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-17T07:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.442 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.442 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.442 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-17T07:35:42.442 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.444 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-17T07:35:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2024-04-17T07:35:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 1, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 2, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 2, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 1239, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-17T07:35:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 248801, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 88, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8792, 2024-04-17T07:35:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 168, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 60048, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 235, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-17T07:35:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 41, 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 12054, 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 33, 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 41, 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 41, 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.570992617, 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.013926649 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 41, 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 64 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 41, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 41, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 4, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 7, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 6, 2024-04-17T07:35:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-04-17T07:35:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-17T07:35:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 93, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 4, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 9, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 11, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 2, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 16, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 1 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-17T07:35:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-04-17T07:35:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 1520, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 933138, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 1520, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 1520, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 933138, 2024-04-17T07:35:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 41, 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 12054, 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 41, 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 12054, 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 41, 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 41, 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 41, 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 41, 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x5624310ba500": { 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 5, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1597, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 5, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 4, 2024-04-17T07:35:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1597, 2024-04-17T07:35:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:42.467 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-17T07:35:43.355 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-17T07:35:43.355 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-17T07:35:43.355 DEBUG:tasks.check_counter:Getting stats from k 2024-04-17T07:35:43.356 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-17T07:35:43.811 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.b/config 2024-04-17T07:35:43.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:43 smithi086 ceph-mon[31733]: pgmap v1080: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:43.955 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:43 smithi098 ceph-mon[35418]: pgmap v1080: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:44.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:43 smithi103 ceph-mon[35495]: pgmap v1080: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:45.753 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:45 smithi098 ceph-mon[35418]: pgmap v1081: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:45.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:45 smithi086 ceph-mon[31733]: pgmap v1081: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:45.997 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-17T07:35:45.997 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1475, 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1179, 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 337631, 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 483637, 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 14, 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.377222766, 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.199532160, 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.176987456, 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.012968004, 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:45.998 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1177, 2024-04-17T07:35:45.999 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.197850427, 2024-04-17T07:35:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000168097 2024-04-17T07:35:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2500, 2024-04-17T07:35:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001190933, 2024-04-17T07:35:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000476 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 46, 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 342, 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 11859, 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 674619, 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 15, 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.048073025, 2024-04-17T07:35:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.027560708, 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.013451342, 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000641555, 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 341, 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.052651463, 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000154403 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 40, 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000014393, 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000359 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 614640 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 520, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 510, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 865781, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 1006223, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 72, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 6, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.203632051, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.062131493, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.131115957, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.017519972, 2024-04-17T07:35:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 509, 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.085171085, 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000167330 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 81, 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000041966, 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000518 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 854640, 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 252832 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 56, 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.002221708, 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000039673 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-17T07:35:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000005262, 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000002631 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 7, 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 4, 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-17T07:35:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.004737805, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001184451 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 3, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 11, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 14, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-04-17T07:35:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 14, 2024-04-17T07:35:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-17T07:35:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 14, 2024-04-17T07:35:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2024-04-17T07:35:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 1, 2024-04-17T07:35:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 1, 2024-04-17T07:35:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 2, 2024-04-17T07:35:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 9, 2024-04-17T07:35:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 7, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 1, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-17T07:35:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 1019, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 1532466553, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-04-17T07:35:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 1, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 1, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 1 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 2, 2024-04-17T07:35:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-04-17T07:35:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 5, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-04-17T07:35:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 5, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 2, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4195901, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.013564472, 2024-04-17T07:35:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003391118 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 17, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 17, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 0, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 15, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 16, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 1, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 14, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 14, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 0, 2024-04-17T07:35:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 1, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 4, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 3, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 51072, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 223608 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 8, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 7, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 19, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.003178246, 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001589123 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-17T07:35:46.018 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001559559, 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000779779 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.019 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-17T07:35:46.020 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.020 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.020 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.020 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-17T07:35:46.020 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.020 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.020 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.021 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-17T07:35:46.022 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.022 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.022 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.022 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-17T07:35:46.023 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.023 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.023 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.023 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-17T07:35:46.023 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.023 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.023 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.023 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.026 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-17T07:35:46.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 1, 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 3, 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 1, 2024-04-17T07:35:46.030 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 1244, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:46.031 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:46.032 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 257375, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 96, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 176, 2024-04-17T07:35:46.033 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 2, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8792, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 168, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:46.034 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 68101, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 296, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 43, 2024-04-17T07:35:46.035 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 12478, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 33, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 43, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 43, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.519993270, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.012092866 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 43, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 68 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 43, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-04-17T07:35:46.036 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 43, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 4, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 7, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 6, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-17T07:35:46.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-17T07:35:46.038 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-17T07:35:46.039 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 93, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 2, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 9, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 15, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 6, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 20, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 1 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-17T07:35:46.040 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-17T07:35:46.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 2037, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1006036, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 2037, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 2037, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1006036, 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:46.042 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 43, 2024-04-17T07:35:46.043 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 12478, 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 43, 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 12478, 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:46.044 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 43, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 43, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 43, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 43, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-17T07:35:46.045 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.046 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55eb985da500": { 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 5, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1597, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 5, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 4, 2024-04-17T07:35:46.047 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1597, 2024-04-17T07:35:46.048 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-17T07:35:46.048 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-17T07:35:46.048 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-17T07:35:46.048 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-17T07:35:46.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:46.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-17T07:35:46.048 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-17T07:35:46.048 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:45 smithi103 ceph-mon[35495]: pgmap v1081: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:46.865 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-17T07:35:46.865 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-17T07:35:46.865 DEBUG:tasks.check_counter:Getting stats from c 2024-04-17T07:35:46.866 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-17T07:35:47.391 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:35:47.805 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:47 smithi103 ceph-mon[35495]: pgmap v1082: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:47.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:47 smithi086 ceph-mon[31733]: pgmap v1082: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:48.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:47 smithi098 ceph-mon[35418]: pgmap v1082: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:49.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:49 smithi103 ceph-mon[35495]: pgmap v1083: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:49.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:49 smithi086 ceph-mon[31733]: pgmap v1083: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:49.870 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-17T07:35:49.870 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:49.870 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2905, 2024-04-17T07:35:49.870 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2907, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 21827840, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 923800, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 7, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.561517098, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.178819673, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.180746146, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.150776722, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2905, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.256581622, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000088324 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.871 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1414, 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1709, 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 19560748, 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 1064362, 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 7, 2024-04-17T07:35:49.872 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.316603377, 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.103990329, 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.118481789, 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.067223280, 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1707, 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.174966946, 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000102499 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.873 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 615280 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 845, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 837, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 9334678, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 381507, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.248133317, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.091656108, 2024-04-17T07:35:49.874 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.111706849, 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.042512874, 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 837, 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.085659245, 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000102340 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 835024, 2024-04-17T07:35:49.875 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 241904 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 7958, 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.367878116, 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000046227 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.876 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 1692, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 1692, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 1692, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-04-17T07:35:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 2, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-04-17T07:35:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 1017, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 1532204373, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T07:35:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-04-17T07:35:49.881 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-17T07:35:49.882 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-04-17T07:35:49.883 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 20945, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 26, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 3, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 52753257, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 52748620, 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-17T07:35:49.884 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 20945 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 1064, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 1331, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 267, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 67, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 72, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 5, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 1692, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 1696, 2024-04-17T07:35:49.885 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 4, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 56080, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215416 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-04-17T07:35:49.886 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.890 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-17T07:35:49.890 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.890 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.890 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.890 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-17T07:35:49.890 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.890 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.890 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.891 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-17T07:35:49.891 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.891 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.891 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.891 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-17T07:35:49.891 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.891 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.891 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.892 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.893 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.894 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-17T07:35:49.897 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 4, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 3, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 2, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 4095, 2024-04-17T07:35:49.898 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:49.899 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:49.900 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 215999, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 68, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:49.901 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8792, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 168, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 3543472, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 22915, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-17T07:35:49.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 4756, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 26, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 4, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 4756, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4756, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 8.443840054, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001775407 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-17T07:35:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4756, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 4757 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 4756, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 4756, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 2778, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1975, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-17T07:35:49.904 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 2, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-17T07:35:49.905 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-17T07:35:49.906 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 93, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 2, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 8, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 8, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-17T07:35:49.907 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T07:35:49.908 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 5160, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 50235142, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 5160, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:35:49.909 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 5160, 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 50235142, 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:49.910 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 4756, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 48568222, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4756, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 48568222, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-17T07:35:49.911 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 4756, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 4756, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4756, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 4756, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-17T07:35:49.912 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.913 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x556319dd6780": { 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T07:35:49.914 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:49.915 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:49.915 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:49.915 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:49.915 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:49.915 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:49.915 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-17T07:35:50.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:49 smithi098 ceph-mon[35418]: pgmap v1083: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:50.887 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-17T07:35:50.888 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-17T07:35:50.888 DEBUG:tasks.check_counter:Getting stats from f 2024-04-17T07:35:50.888 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-17T07:35:51.353 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:35:51.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:51 smithi103 ceph-mon[35495]: pgmap v1084: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:51.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:51 smithi086 ceph-mon[31733]: pgmap v1084: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:52.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:51 smithi098 ceph-mon[35418]: pgmap v1084: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:35:53.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:52 smithi098 ceph-mon[35418]: pgmap v1085: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:53.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:52 smithi103 ceph-mon[35495]: pgmap v1085: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:53.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:52 smithi086 ceph-mon[31733]: pgmap v1085: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:54.019 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-17T07:35:54.019 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:54.019 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1799, 2024-04-17T07:35:54.019 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1793, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1301472, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 686504, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.390703857, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.139685598, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.134621563, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.093843038, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1792, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.179111311, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000099950 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 832416, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 242576 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 8, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 302, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3868, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 616602, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-04-17T07:35:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.035655018, 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.024560872, 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.005461179, 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000065052, 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 301, 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.037748655, 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000125410 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 615904 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2712, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2714, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 691730, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 862423, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.412056390, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.145956531, 2024-04-17T07:35:54.023 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.103723243, 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.135368742, 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2712, 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.201283427, 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000074219 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T07:35:54.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6833, 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.303655505, 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000044439 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.025 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-17T07:35:54.026 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T07:35:54.026 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T07:35:54.026 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.026 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.026 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.026 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:54.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-04-17T07:35:54.027 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-04-17T07:35:54.028 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-04-17T07:35:54.029 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-17T07:35:54.030 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-04-17T07:35:54.031 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-17T07:35:54.032 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.033 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 5, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-17T07:35:54.034 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4195901, 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 4195424, 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 5 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 3, 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 3, 2024-04-17T07:35:54.035 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 1, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 1, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 0, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 0, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 49252, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215416 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-17T07:35:54.036 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.037 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.038 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.039 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.040 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-17T07:35:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.041 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.042 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.044 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-17T07:35:54.045 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.047 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.048 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 2, 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 1, 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 2671, 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-17T07:35:54.049 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:54.050 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:54.051 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 182087, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 48, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-17T07:35:54.052 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8792, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 168, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 8512, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 13, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-17T07:35:54.053 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 4110, 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 4110, 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4110, 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 8.984829928, 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002186090 2024-04-17T07:35:54.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4110, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 4111 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 4110, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 4110, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 2706, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1402, 2024-04-17T07:35:54.055 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-17T07:35:54.056 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-17T07:35:54.057 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 93, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-17T07:35:54.058 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 2, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 4, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 5, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 1, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-17T07:35:54.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.060 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 4515, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1566938, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 4515, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4515, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1566938, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.061 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 4110, 2024-04-17T07:35:54.062 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 1602, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4110, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1602, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:54.063 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 4110, 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 4110, 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4110, 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 4110, 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:54.064 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.065 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x5630c125c780": { 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T07:35:54.066 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:54.067 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:54.067 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-17T07:35:54.915 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-17T07:35:54.915 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-17T07:35:54.915 DEBUG:tasks.check_counter:Getting stats from i 2024-04-17T07:35:54.915 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-17T07:35:55.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:54 smithi098 ceph-mon[35418]: pgmap v1086: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:55.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:54 smithi103 ceph-mon[35495]: pgmap v1086: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:55.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:54 smithi086 ceph-mon[31733]: pgmap v1086: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:35:55.378 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:35:57.162 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:56 smithi103 ceph-mon[35495]: pgmap v1087: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:57.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:56 smithi098 ceph-mon[35418]: pgmap v1087: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:57.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:56 smithi086 ceph-mon[31733]: pgmap v1087: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:35:57.537 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 92, 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 92, 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 18123, 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 807100, 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 77, 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.088405216, 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.005982169, 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.059693988, 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000768932, 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:57.538 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 91, 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.052005793, 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000571492 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 68, 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000014328, 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000210 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1012, 2024-04-17T07:35:57.539 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1292, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 959849, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 587879, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 8, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.303665877, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.127761045, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.150116274, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.028628444, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1291, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.157056823, 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000121655 2024-04-17T07:35:57.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1424, 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000713565, 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000501 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 844336, 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 246096 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 30, 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 325, 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 5505, 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 672852, 2024-04-17T07:35:57.541 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 15, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.048247135, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.025967128, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.014231968, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000381020, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 323, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.046562452, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000144156 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 34, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000012070, 2024-04-17T07:35:57.542 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000355 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 618512 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 50, 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001770348, 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000035406 2024-04-17T07:35:57.543 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000011228, 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000005614 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-17T07:35:57.544 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 11, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 10, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 10, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-04-17T07:35:57.545 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 10, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-17T07:35:57.546 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 1019, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 1532466553, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-17T07:35:57.547 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.548 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 1, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-17T07:35:57.549 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-17T07:35:57.550 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 3, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 3, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 2, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-04-17T07:35:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4195369, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 0, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.005982633, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002991316 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 0 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 13, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 13, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 11, 2024-04-17T07:35:57.552 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 12, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 1, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 10, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 10, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 50560, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 223608 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-04-17T07:35:57.553 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 15, 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.554 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-17T07:35:57.555 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-17T07:35:57.556 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-17T07:35:57.557 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.558 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.558 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.560 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-17T07:35:57.561 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.562 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.563 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 1, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 0, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-17T07:35:57.565 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1258, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:35:57.566 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:35:57.567 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:35:57.568 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 246919, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 82, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8792, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 168, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:35:57.569 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 50872, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 198, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 37, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 11277, 2024-04-17T07:35:57.570 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 33, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 37, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 37, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.489990698, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.013242991 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 37, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 59 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 37, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 0, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 37, 2024-04-17T07:35:57.571 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 0, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 10, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 0, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 2, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 6, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 6, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-17T07:35:57.572 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 20, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-17T07:35:57.573 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-17T07:35:57.574 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 93, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 2, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 10, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 15, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 5, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 14, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 0, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 1 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 1, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-17T07:35:57.575 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-17T07:35:57.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 1130, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 856629, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 1130, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1130, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 856629, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.577 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 37, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 11277, 2024-04-17T07:35:57.578 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 37, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 11277, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:57.579 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 37, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 37, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 37, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 37, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:57.580 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x56293a30c500": { 2024-04-17T07:35:57.581 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 3, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1065, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 3, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1065, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:35:57.582 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:35:57.583 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:57.583 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:35:57.583 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-17T07:35:58.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:57 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:35:58.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:57 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:35:58.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:57 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-17T07:35:58.646 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-17T07:35:58.646 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-17T07:35:58.647 DEBUG:tasks.check_counter:Getting stats from l 2024-04-17T07:35:58.647 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-17T07:35:59.080 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/mon.c/config 2024-04-17T07:35:59.161 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:58 smithi098 ceph-mon[35418]: pgmap v1088: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:59.162 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:58 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:35:59.162 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:35:58 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:35:59.227 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:58 smithi103 ceph-mon[35495]: pgmap v1088: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:59.227 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:58 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:35:59.227 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:35:58 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:35:59.326 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:58 smithi086 ceph-mon[31733]: pgmap v1088: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:35:59.327 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:58 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:35:59.327 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:35:58 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:01.444 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-17T07:36:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 412, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 406, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 835522, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 247684, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.138135761, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.055461800, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.068223187, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.018051358, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 405, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.055330743, 2024-04-17T07:36:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000136619 2024-04-17T07:36:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:36:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 834688, 2024-04-17T07:36:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 247360 2024-04-17T07:36:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-17T07:36:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1261, 2024-04-17T07:36:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1563, 2024-04-17T07:36:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 310142, 2024-04-17T07:36:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 964813, 2024-04-17T07:36:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-04-17T07:36:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-04-17T07:36:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.249070192, 2024-04-17T07:36:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.091095718, 2024-04-17T07:36:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.053130772, 2024-04-17T07:36:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.077027003, 2024-04-17T07:36:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:36:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1561, 2024-04-17T07:36:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.149567159, 2024-04-17T07:36:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000095814 2024-04-17T07:36:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:36:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-17T07:36:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 564128 2024-04-17T07:36:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-17T07:36:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2818, 2024-04-17T07:36:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2818, 2024-04-17T07:36:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 845330, 2024-04-17T07:36:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 895358, 2024-04-17T07:36:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-04-17T07:36:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-04-17T07:36:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.495333570, 2024-04-17T07:36:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.165784152, 2024-04-17T07:36:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.131781991, 2024-04-17T07:36:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.146503327, 2024-04-17T07:36:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T07:36:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2817, 2024-04-17T07:36:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.246808106, 2024-04-17T07:36:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000087613 2024-04-17T07:36:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-17T07:36:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-17T07:36:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-04-17T07:36:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-17T07:36:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-17T07:36:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-17T07:36:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-17T07:36:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T07:36:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T07:36:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6707, 2024-04-17T07:36:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.326180892, 2024-04-17T07:36:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000048632 2024-04-17T07:36:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:36:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-17T07:36:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-17T07:36:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-17T07:36:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:36:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-17T07:36:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-04-17T07:36:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-04-17T07:36:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-17T07:36:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-17T07:36:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-04-17T07:36:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-17T07:36:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-17T07:36:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-04-17T07:36:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-17T07:36:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-17T07:36:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 0, 2024-04-17T07:36:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-04-17T07:36:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 0, 2024-04-17T07:36:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-04-17T07:36:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 0, 2024-04-17T07:36:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-04-17T07:36:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-04-17T07:36:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-04-17T07:36:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-04-17T07:36:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-04-17T07:36:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-04-17T07:36:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-04-17T07:36:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-04-17T07:36:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-17T07:36:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-17T07:36:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-17T07:36:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-17T07:36:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-17T07:36:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-04-17T07:36:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-04-17T07:36:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-04-17T07:36:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-04-17T07:36:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-17T07:36:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-17T07:36:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-04-17T07:36:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-04-17T07:36:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-04-17T07:36:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-04-17T07:36:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T07:36:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-17T07:36:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T07:36:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-17T07:36:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-17T07:36:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T07:36:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T07:36:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-17T07:36:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-17T07:36:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T07:36:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-04-17T07:36:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T07:36:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T07:36:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T07:36:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-04-17T07:36:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-04-17T07:36:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-04-17T07:36:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-04-17T07:36:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-17T07:36:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-04-17T07:36:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-04-17T07:36:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-04-17T07:36:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-04-17T07:36:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-04-17T07:36:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-04-17T07:36:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-17T07:36:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-17T07:36:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-04-17T07:36:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-04-17T07:36:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-17T07:36:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-17T07:36:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-17T07:36:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-17T07:36:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-17T07:36:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-17T07:36:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-17T07:36:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-04-17T07:36:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-04-17T07:36:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T07:36:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-04-17T07:36:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-17T07:36:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-17T07:36:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-17T07:36:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-17T07:36:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-17T07:36:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-17T07:36:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-04-17T07:36:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-04-17T07:36:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-04-17T07:36:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 3, 2024-04-17T07:36:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-17T07:36:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-04-17T07:36:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-04-17T07:36:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-04-17T07:36:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-04-17T07:36:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-04-17T07:36:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-04-17T07:36:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-17T07:36:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-04-17T07:36:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-04-17T07:36:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4195369, 2024-04-17T07:36:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 4194892, 2024-04-17T07:36:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-17T07:36:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 3 2024-04-17T07:36:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-17T07:36:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 3, 2024-04-17T07:36:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 3, 2024-04-17T07:36:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-04-17T07:36:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 1, 2024-04-17T07:36:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 1, 2024-04-17T07:36:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-04-17T07:36:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 0, 2024-04-17T07:36:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 0, 2024-04-17T07:36:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-04-17T07:36:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-04-17T07:36:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-04-17T07:36:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-04-17T07:36:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 49280, 2024-04-17T07:36:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215416 2024-04-17T07:36:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-17T07:36:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-04-17T07:36:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-04-17T07:36:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-04-17T07:36:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-04-17T07:36:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-04-17T07:36:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-17T07:36:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-17T07:36:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-17T07:36:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-17T07:36:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-17T07:36:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-17T07:36:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-17T07:36:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-17T07:36:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-17T07:36:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-17T07:36:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-17T07:36:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-17T07:36:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-17T07:36:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-17T07:36:01.494 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-17T07:36:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-04-17T07:36:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 1, 2024-04-17T07:36:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 0, 2024-04-17T07:36:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2024-04-17T07:36:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-17T07:36:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-04-17T07:36:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-04-17T07:36:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 2681, 2024-04-17T07:36:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-17T07:36:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-17T07:36:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-17T07:36:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-17T07:36:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T07:36:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-17T07:36:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T07:36:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-17T07:36:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T07:36:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T07:36:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T07:36:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T07:36:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T07:36:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-17T07:36:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T07:36:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T07:36:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-17T07:36:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-17T07:36:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-17T07:36:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-17T07:36:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T07:36:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T07:36:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T07:36:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T07:36:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T07:36:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-17T07:36:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-17T07:36:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-17T07:36:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T07:36:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T07:36:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-17T07:36:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-17T07:36:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-17T07:36:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-17T07:36:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T07:36:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-17T07:36:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T07:36:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-17T07:36:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 191008, 2024-04-17T07:36:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 56, 2024-04-17T07:36:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-17T07:36:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-17T07:36:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-17T07:36:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-17T07:36:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-17T07:36:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-17T07:36:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-17T07:36:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-17T07:36:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8792, 2024-04-17T07:36:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 168, 2024-04-17T07:36:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T07:36:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-17T07:36:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-17T07:36:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-17T07:36:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 8512, 2024-04-17T07:36:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 13, 2024-04-17T07:36:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-17T07:36:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-17T07:36:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-17T07:36:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-17T07:36:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-17T07:36:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-17T07:36:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-17T07:36:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 4075, 2024-04-17T07:36:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-04-17T07:36:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-04-17T07:36:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 4075, 2024-04-17T07:36:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-17T07:36:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4075, 2024-04-17T07:36:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 6.654873706, 2024-04-17T07:36:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001633097 2024-04-17T07:36:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-17T07:36:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-17T07:36:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4075, 2024-04-17T07:36:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 4076 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 4075, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 4075, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 2663, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1410, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-04-17T07:36:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-17T07:36:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-17T07:36:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 93, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 2, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 5, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 6, 2024-04-17T07:36:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 1, 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-17T07:36:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T07:36:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 4487, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1563016, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 4487, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4487, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1563016, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:36:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 4075, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 1070, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4075, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1070, 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:36:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:36:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 4075, 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 4075, 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4075, 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 4075, 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:36:01.519 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-17T07:36:01.520 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x56371ed78780": { 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-17T07:36:01.521 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-17T07:36:01.522 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:36:01.522 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-17T07:36:01.522 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-17T07:36:01.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:01 smithi103 ceph-mon[35495]: pgmap v1089: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:36:01.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:01.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:01.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:01.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:01 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:01 smithi086 ceph-mon[31733]: pgmap v1089: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:36:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:01.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:01 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:01 smithi098 ceph-mon[35418]: pgmap v1089: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-17T07:36:02.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:02.004 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:01 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:02.869 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-17T07:36:02.869 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-17T07:36:02.870 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} 2024-04-17T07:36:02.871 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-17T07:36:02.884 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-17T07:36:02.884 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:36:02.884 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T07:36:02.955 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:36:02.955 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T07:36:03.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:36:03.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.236+0000 7ff1eac00640 1 -- 172.21.15.86:0/3212303548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4102ff0 msgr2=0x7ff1e4103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:03.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.236+0000 7ff1eac00640 1 --2- 172.21.15.86:0/3212303548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4102ff0 0x7ff1e4103470 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009900 tx=0x7ff1d002bc30 comp rx=0 tx=0).stop 2024-04-17T07:36:03.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.236+0000 7ff1eac00640 1 -- 172.21.15.86:0/3212303548 shutdown_connections 2024-04-17T07:36:03.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.236+0000 7ff1eac00640 1 --2- 172.21.15.86:0/3212303548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e41039b0 0x7ff1e410a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.236+0000 7ff1eac00640 1 --2- 172.21.15.86:0/3212303548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4102ff0 0x7ff1e4103470 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.236+0000 7ff1eac00640 1 --2- 172.21.15.86:0/3212303548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e4101e00 0x7ff1e4102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.236+0000 7ff1eac00640 1 -- 172.21.15.86:0/3212303548 >> 172.21.15.86:0/3212303548 conn(0x7ff1e40fd370 msgr2=0x7ff1e40ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:03.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.236+0000 7ff1eac00640 1 -- 172.21.15.86:0/3212303548 shutdown_connections 2024-04-17T07:36:03.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.236+0000 7ff1eac00640 1 -- 172.21.15.86:0/3212303548 wait complete. 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.237+0000 7ff1eac00640 1 Processor -- start 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1eac00640 1 -- start start 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1eac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4101e00 0x7ff1e4075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1eac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4102ff0 0x7ff1e4074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1eac00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41039b0 0x7ff1e4074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1eac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410d960 con 0x7ff1e4101e00 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1eac00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410d7e0 con 0x7ff1e41039b0 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1eac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e410dae0 con 0x7ff1e4102ff0 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1e9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4101e00 0x7ff1e4075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1e9800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4101e00 0x7ff1e4075d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51650/0 (socket says 172.21.15.86:51650) 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1e9800640 1 -- 172.21.15.86:0/2807887531 learned_addr learned my addr 172.21.15.86:0/2807887531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1e8e00640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4102ff0 0x7ff1e4074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:03.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.238+0000 7ff1ea200640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41039b0 0x7ff1e4074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.239+0000 7ff1e9800640 1 -- 172.21.15.86:0/2807887531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4102ff0 msgr2=0x7ff1e4074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.239+0000 7ff1e9800640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4102ff0 0x7ff1e4074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.239+0000 7ff1e9800640 1 -- 172.21.15.86:0/2807887531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41039b0 msgr2=0x7ff1e4074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.239+0000 7ff1e9800640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41039b0 0x7ff1e4074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.239+0000 7ff1e9800640 1 -- 172.21.15.86:0/2807887531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e4075120 con 0x7ff1e4101e00 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.239+0000 7ff1e9800640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4101e00 0x7ff1e4075d10 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800d770 tx=0x7ff1d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.239+0000 7ff1e2000640 1 -- 172.21.15.86:0/2807887531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8017070 con 0x7ff1e4101e00 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.239+0000 7ff1e2000640 1 -- 172.21.15.86:0/2807887531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d8009c30 con 0x7ff1e4101e00 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.239+0000 7ff1e2000640 1 -- 172.21.15.86:0/2807887531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8003d00 con 0x7ff1e4101e00 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.239+0000 7ff1eac00640 1 -- 172.21.15.86:0/2807887531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e41a2180 con 0x7ff1e4101e00 2024-04-17T07:36:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.240+0000 7ff1eac00640 1 -- 172.21.15.86:0/2807887531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e41a26a0 con 0x7ff1e4101e00 2024-04-17T07:36:03.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.241+0000 7ff1eac00640 1 -- 172.21.15.86:0/2807887531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e4109190 con 0x7ff1e4101e00 2024-04-17T07:36:03.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.243+0000 7ff1e2000640 1 -- 172.21.15.86:0/2807887531 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff1d8003ea0 con 0x7ff1e4101e00 2024-04-17T07:36:03.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.245+0000 7ff1e2000640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3bb0 0x7ff1cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:03.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.245+0000 7ff1e2000640 1 -- 172.21.15.86:0/2807887531 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff1d80d0490 con 0x7ff1e4101e00 2024-04-17T07:36:03.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.245+0000 7ff1e8e00640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3bb0 0x7ff1cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:03.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.246+0000 7ff1e8e00640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3bb0 0x7ff1cc0a6070 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0000c00 tx=0x7ff1d002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:03.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.250+0000 7ff1e2000640 1 -- 172.21.15.86:0/2807887531 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff1d809caf0 con 0x7ff1e4101e00 2024-04-17T07:36:03.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.452+0000 7ff1eac00640 1 -- 172.21.15.86:0/2807887531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1e4062b60 con 0x7ff1e4101e00 2024-04-17T07:36:03.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.456+0000 7ff1e2000640 1 -- 172.21.15.86:0/2807887531 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7ff1d8009de0 con 0x7ff1e4101e00 2024-04-17T07:36:03.457 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:36:03.457 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:36:03.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 -- 172.21.15.86:0/2807887531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3bb0 msgr2=0x7ff1cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3bb0 0x7ff1cc0a6070 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0000c00 tx=0x7ff1d002c7e0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 -- 172.21.15.86:0/2807887531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4101e00 msgr2=0x7ff1e4075d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4101e00 0x7ff1e4075d10 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800d770 tx=0x7ff1d800dc30 comp rx=0 tx=0).stop 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 -- 172.21.15.86:0/2807887531 shutdown_connections 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1cc0a3bb0 0x7ff1cc0a6070 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e41039b0 0x7ff1e4074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1e4102ff0 0x7ff1e4074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 --2- 172.21.15.86:0/2807887531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1e4101e00 0x7ff1e4075d10 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 -- 172.21.15.86:0/2807887531 >> 172.21.15.86:0/2807887531 conn(0x7ff1e40fd370 msgr2=0x7ff1e4106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 -- 172.21.15.86:0/2807887531 shutdown_connections 2024-04-17T07:36:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.459+0000 7ff1eac00640 1 -- 172.21.15.86:0/2807887531 wait complete. 2024-04-17T07:36:03.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:36:03.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:36:03.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.703+0000 7f435ae00640 1 -- 172.21.15.86:0/1808147909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43540749c0 msgr2=0x7f4354074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:03.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.703+0000 7f435ae00640 1 --2- 172.21.15.86:0/1808147909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43540749c0 0x7f4354074e40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4340009900 tx=0x7f434002bc30 comp rx=0 tx=0).stop 2024-04-17T07:36:03.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.703+0000 7f435ae00640 1 -- 172.21.15.86:0/1808147909 shutdown_connections 2024-04-17T07:36:03.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.703+0000 7f435ae00640 1 --2- 172.21.15.86:0/1808147909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4354109c70 0x7f435410c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.703+0000 7f435ae00640 1 --2- 172.21.15.86:0/1808147909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43540749c0 0x7f4354074e40 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.703+0000 7f435ae00640 1 --2- 172.21.15.86:0/1808147909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4354075fa0 0x7f43540743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.703+0000 7f435ae00640 1 -- 172.21.15.86:0/1808147909 >> 172.21.15.86:0/1808147909 conn(0x7f43540fd340 msgr2=0x7f43540ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:03.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.704+0000 7f435ae00640 1 -- 172.21.15.86:0/1808147909 shutdown_connections 2024-04-17T07:36:03.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.704+0000 7f435ae00640 1 -- 172.21.15.86:0/1808147909 wait complete. 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.704+0000 7f435ae00640 1 Processor -- start 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f435ae00640 1 -- start start 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f435ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43540749c0 0x7f435419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f435ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4354075fa0 0x7f435419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f435ae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4354109c70 0x7f43541a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f435ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f435410fb90 con 0x7f43540749c0 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f435ae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f435410fa10 con 0x7f4354109c70 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f435ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f435410fd10 con 0x7f4354075fa0 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f4359a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43540749c0 0x7f435419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f4359a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43540749c0 0x7f435419a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51672/0 (socket says 172.21.15.86:51672) 2024-04-17T07:36:03.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f4359a00640 1 -- 172.21.15.86:0/2339478680 learned_addr learned my addr 172.21.15.86:0/2339478680 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.705+0000 7f4359000640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4354075fa0 0x7f435419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.706+0000 7f435a400640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4354109c70 0x7f43541a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.706+0000 7f4359a00640 1 -- 172.21.15.86:0/2339478680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4354075fa0 msgr2=0x7f435419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.706+0000 7f4359a00640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4354075fa0 0x7f435419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.706+0000 7f4359a00640 1 -- 172.21.15.86:0/2339478680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4354109c70 msgr2=0x7f43541a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.706+0000 7f4359a00640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4354109c70 0x7f43541a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.706+0000 7f4359a00640 1 -- 172.21.15.86:0/2339478680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43541a1e80 con 0x7f43540749c0 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.706+0000 7f435a400640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4354109c70 0x7f43541a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.706+0000 7f4359a00640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43540749c0 0x7f435419a260 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f434800b4a0 tx=0x7f434800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.706+0000 7f4352000640 1 -- 172.21.15.86:0/2339478680 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348014070 con 0x7f43540749c0 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.707+0000 7f4352000640 1 -- 172.21.15.86:0/2339478680 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4348010ac0 con 0x7f43540749c0 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.707+0000 7f4352000640 1 -- 172.21.15.86:0/2339478680 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348017c30 con 0x7f43540749c0 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.707+0000 7f435ae00640 1 -- 172.21.15.86:0/2339478680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43541a2170 con 0x7f43540749c0 2024-04-17T07:36:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.707+0000 7f435ae00640 1 -- 172.21.15.86:0/2339478680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43541a2710 con 0x7f43540749c0 2024-04-17T07:36:03.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.708+0000 7f435ae00640 1 -- 172.21.15.86:0/2339478680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43541089a0 con 0x7f43540749c0 2024-04-17T07:36:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.711+0000 7f4352000640 1 -- 172.21.15.86:0/2339478680 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4348017070 con 0x7f43540749c0 2024-04-17T07:36:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.714+0000 7f4352000640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f433c0a3bb0 0x7f433c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.714+0000 7f4352000640 1 -- 172.21.15.86:0/2339478680 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f43480cfeb0 con 0x7f43540749c0 2024-04-17T07:36:03.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.714+0000 7f4359000640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f433c0a3bb0 0x7f433c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.715+0000 7f4359000640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f433c0a3bb0 0x7f433c0a6070 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4340000c00 tx=0x7f434002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:03.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.719+0000 7f4352000640 1 -- 172.21.15.86:0/2339478680 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f434809c4b0 con 0x7f43540749c0 2024-04-17T07:36:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:03 smithi086 ceph-mon[31733]: pgmap v1090: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:36:03.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:03 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2807887531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:36:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.931+0000 7f435ae00640 1 -- 172.21.15.86:0/2339478680 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4354062b60 con 0x7f43540749c0 2024-04-17T07:36:03.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.932+0000 7f4352000640 1 -- 172.21.15.86:0/2339478680 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f434809bdb0 con 0x7f43540749c0 2024-04-17T07:36:03.932 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:36:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 -- 172.21.15.86:0/2339478680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f433c0a3bb0 msgr2=0x7f433c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f433c0a3bb0 0x7f433c0a6070 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4340000c00 tx=0x7f434002c7e0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 -- 172.21.15.86:0/2339478680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43540749c0 msgr2=0x7f435419a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43540749c0 0x7f435419a260 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f434800b4a0 tx=0x7f434800b960 comp rx=0 tx=0).stop 2024-04-17T07:36:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 -- 172.21.15.86:0/2339478680 shutdown_connections 2024-04-17T07:36:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f433c0a3bb0 0x7f433c0a6070 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4354109c70 0x7f43541a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4354075fa0 0x7f435419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 --2- 172.21.15.86:0/2339478680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43540749c0 0x7f435419a260 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 -- 172.21.15.86:0/2339478680 >> 172.21.15.86:0/2339478680 conn(0x7f43540fd340 msgr2=0x7f4354105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 -- 172.21.15.86:0/2339478680 shutdown_connections 2024-04-17T07:36:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:03.935+0000 7f435ae00640 1 -- 172.21.15.86:0/2339478680 wait complete. 2024-04-17T07:36:03.954 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:36:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:03 smithi098 ceph-mon[35418]: pgmap v1090: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:36:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:03 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2807887531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:36:04.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:03 smithi103 ceph-mon[35495]: pgmap v1090: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:36:04.026 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:03 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2807887531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:36:04.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.172+0000 7f02b5c00640 1 -- 172.21.15.86:0/1269880062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0109d20 msgr2=0x7f02b010c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:04.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.172+0000 7f02b5c00640 1 --2- 172.21.15.86:0/1269880062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0109d20 0x7f02b010c1b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009ca0 tx=0x7f02a802c180 comp rx=0 tx=0).stop 2024-04-17T07:36:04.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.172+0000 7f02b5c00640 1 -- 172.21.15.86:0/1269880062 shutdown_connections 2024-04-17T07:36:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.172+0000 7f02b5c00640 1 --2- 172.21.15.86:0/1269880062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0109d20 0x7f02b010c1b0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.172+0000 7f02b5c00640 1 --2- 172.21.15.86:0/1269880062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b0074940 0x7f02b0074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.172+0000 7f02b5c00640 1 --2- 172.21.15.86:0/1269880062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0075f20 0x7f02b0074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.172+0000 7f02b5c00640 1 -- 172.21.15.86:0/1269880062 >> 172.21.15.86:0/1269880062 conn(0x7f02b00fd400 msgr2=0x7f02b00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.172+0000 7f02b5c00640 1 -- 172.21.15.86:0/1269880062 shutdown_connections 2024-04-17T07:36:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.172+0000 7f02b5c00640 1 -- 172.21.15.86:0/1269880062 wait complete. 2024-04-17T07:36:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.173+0000 7f02b5c00640 1 Processor -- start 2024-04-17T07:36:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.173+0000 7f02b5c00640 1 -- start start 2024-04-17T07:36:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.173+0000 7f02b5c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b0074940 0x7f02b019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.173+0000 7f02b5c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0075f20 0x7f02b019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.173+0000 7f02b5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0109d20 0x7f02b01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.173+0000 7f02b5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02b010fbc0 con 0x7f02b0109d20 2024-04-17T07:36:04.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.173+0000 7f02b5c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f02b010fa40 con 0x7f02b0074940 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.173+0000 7f02b5c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f02b010fd40 con 0x7f02b0075f20 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02b5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0109d20 0x7f02b01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02b5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0109d20 0x7f02b01a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51688/0 (socket says 172.21.15.86:51688) 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02b5200640 1 -- 172.21.15.86:0/1853121285 learned_addr learned my addr 172.21.15.86:0/1853121285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02af400640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0075f20 0x7f02b019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02afe00640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b0074940 0x7f02b019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02b5200640 1 -- 172.21.15.86:0/1853121285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0075f20 msgr2=0x7f02b019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02b5200640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0075f20 0x7f02b019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02b5200640 1 -- 172.21.15.86:0/1853121285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b0074940 msgr2=0x7f02b019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02b5200640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b0074940 0x7f02b019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02b5200640 1 -- 172.21.15.86:0/1853121285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b01a1e10 con 0x7f02b0109d20 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.174+0000 7f02b5200640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0109d20 0x7f02b01a17a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f02a802c6c0 tx=0x7f02a8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.175+0000 7f02acc00640 1 -- 172.21.15.86:0/1853121285 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8039040 con 0x7f02b0109d20 2024-04-17T07:36:04.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.175+0000 7f02b5c00640 1 -- 172.21.15.86:0/1853121285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b01a20a0 con 0x7f02b0109d20 2024-04-17T07:36:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.175+0000 7f02acc00640 1 -- 172.21.15.86:0/1853121285 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02a8044070 con 0x7f02b0109d20 2024-04-17T07:36:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.175+0000 7f02acc00640 1 -- 172.21.15.86:0/1853121285 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a80042a0 con 0x7f02b0109d20 2024-04-17T07:36:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.175+0000 7f02b5c00640 1 -- 172.21.15.86:0/1853121285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b01a2590 con 0x7f02b0109d20 2024-04-17T07:36:04.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.176+0000 7f02b5c00640 1 -- 172.21.15.86:0/1853121285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b0108ae0 con 0x7f02b0109d20 2024-04-17T07:36:04.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.178+0000 7f02acc00640 1 -- 172.21.15.86:0/1853121285 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f02a8004440 con 0x7f02b0109d20 2024-04-17T07:36:04.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.179+0000 7f02acc00640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02940a3bb0 0x7f02940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.179+0000 7f02acc00640 1 -- 172.21.15.86:0/1853121285 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f02a80f2e80 con 0x7f02b0109d20 2024-04-17T07:36:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.180+0000 7f02afe00640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02940a3bb0 0x7f02940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.180+0000 7f02afe00640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02940a3bb0 0x7f02940a6070 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009b60 tx=0x7f02a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.184+0000 7f02acc00640 1 -- 172.21.15.86:0/1853121285 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f02a80bf480 con 0x7f02b0109d20 2024-04-17T07:36:04.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.388+0000 7f02b5c00640 1 -- 172.21.15.86:0/1853121285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f02b0062b60 con 0x7f02b0109d20 2024-04-17T07:36:04.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.389+0000 7f02acc00640 1 -- 172.21.15.86:0/1853121285 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f02a80bed80 con 0x7f02b0109d20 2024-04-17T07:36:04.390 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:36:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.391+0000 7f02b5c00640 1 -- 172.21.15.86:0/1853121285 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02940a3bb0 msgr2=0x7f02940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.391+0000 7f02b5c00640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02940a3bb0 0x7f02940a6070 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009b60 tx=0x7f02a0005f90 comp rx=0 tx=0).stop 2024-04-17T07:36:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.391+0000 7f02b5c00640 1 -- 172.21.15.86:0/1853121285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0109d20 msgr2=0x7f02b01a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.391+0000 7f02b5c00640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0109d20 0x7f02b01a17a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f02a802c6c0 tx=0x7f02a8003ba0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.392+0000 7f02b5c00640 1 -- 172.21.15.86:0/1853121285 shutdown_connections 2024-04-17T07:36:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.392+0000 7f02b5c00640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f02940a3bb0 0x7f02940a6070 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.392+0000 7f02b5c00640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b0109d20 0x7f02b01a17a0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.392+0000 7f02b5c00640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02b0075f20 0x7f02b019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.392+0000 7f02b5c00640 1 --2- 172.21.15.86:0/1853121285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b0074940 0x7f02b019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.392+0000 7f02b5c00640 1 -- 172.21.15.86:0/1853121285 >> 172.21.15.86:0/1853121285 conn(0x7f02b00fd400 msgr2=0x7f02b0105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:04.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.392+0000 7f02b5c00640 1 -- 172.21.15.86:0/1853121285 shutdown_connections 2024-04-17T07:36:04.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.392+0000 7f02b5c00640 1 -- 172.21.15.86:0/1853121285 wait complete. 2024-04-17T07:36:04.412 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:36:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.629+0000 7ff133200640 1 -- 172.21.15.86:0/1498463073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1341083c0 msgr2=0x7ff1341087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.629+0000 7ff133200640 1 --2- 172.21.15.86:0/1498463073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1341083c0 0x7ff1341087e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7ff124009900 tx=0x7ff12402bc30 comp rx=0 tx=0).stop 2024-04-17T07:36:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.629+0000 7ff133200640 1 -- 172.21.15.86:0/1498463073 shutdown_connections 2024-04-17T07:36:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.629+0000 7ff133200640 1 --2- 172.21.15.86:0/1498463073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff134109f70 0x7ff134110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.629+0000 7ff133200640 1 --2- 172.21.15.86:0/1498463073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1341095b0 0x7ff134109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.629+0000 7ff133200640 1 --2- 172.21.15.86:0/1498463073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1341083c0 0x7ff1341087e0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.629+0000 7ff133200640 1 -- 172.21.15.86:0/1498463073 >> 172.21.15.86:0/1498463073 conn(0x7ff134077360 msgr2=0x7ff1340797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.629+0000 7ff133200640 1 -- 172.21.15.86:0/1498463073 shutdown_connections 2024-04-17T07:36:04.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.629+0000 7ff133200640 1 -- 172.21.15.86:0/1498463073 wait complete. 2024-04-17T07:36:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.630+0000 7ff133200640 1 Processor -- start 2024-04-17T07:36:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.630+0000 7ff133200640 1 -- start start 2024-04-17T07:36:04.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff133200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1341083c0 0x7ff13419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff133200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1341095b0 0x7ff13419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff133200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff134109f70 0x7ff1341a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff133200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff134113f00 con 0x7ff1341095b0 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff133200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff134113d80 con 0x7ff134109f70 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff133200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff134114080 con 0x7ff1341083c0 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff129400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1341095b0 0x7ff13419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff129400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1341095b0 0x7ff13419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51706/0 (socket says 172.21.15.86:51706) 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff129400640 1 -- 172.21.15.86:0/1203265057 learned_addr learned my addr 172.21.15.86:0/1203265057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff131e00640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1341083c0 0x7ff13419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff132800640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff134109f70 0x7ff1341a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff129400640 1 -- 172.21.15.86:0/1203265057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1341083c0 msgr2=0x7ff13419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff129400640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1341083c0 0x7ff13419e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff129400640 1 -- 172.21.15.86:0/1203265057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff134109f70 msgr2=0x7ff1341a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:04.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff129400640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff134109f70 0x7ff1341a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.631+0000 7ff129400640 1 -- 172.21.15.86:0/1203265057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1341a6210 con 0x7ff1341095b0 2024-04-17T07:36:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.632+0000 7ff129400640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1341095b0 0x7ff13419eb80 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00eab0 tx=0x7ff11c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.632+0000 7ff12b400640 1 -- 172.21.15.86:0/1203265057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00ce40 con 0x7ff1341095b0 2024-04-17T07:36:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.632+0000 7ff12b400640 1 -- 172.21.15.86:0/1203265057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff11c011920 con 0x7ff1341095b0 2024-04-17T07:36:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.632+0000 7ff133200640 1 -- 172.21.15.86:0/1203265057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1341a6500 con 0x7ff1341095b0 2024-04-17T07:36:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.632+0000 7ff12b400640 1 -- 172.21.15.86:0/1203265057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c010a00 con 0x7ff1341095b0 2024-04-17T07:36:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.632+0000 7ff133200640 1 -- 172.21.15.86:0/1203265057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1341a6a50 con 0x7ff1341095b0 2024-04-17T07:36:04.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.633+0000 7ff133200640 1 -- 172.21.15.86:0/1203265057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13410f700 con 0x7ff1341095b0 2024-04-17T07:36:04.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.636+0000 7ff12b400640 1 -- 172.21.15.86:0/1203265057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7ff11c011ad0 con 0x7ff1341095b0 2024-04-17T07:36:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.637+0000 7ff12b400640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1180a3bb0 0x7ff1180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.637+0000 7ff12b400640 1 -- 172.21.15.86:0/1203265057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7ff11c0d0e20 con 0x7ff1341095b0 2024-04-17T07:36:04.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.637+0000 7ff131e00640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1180a3bb0 0x7ff1180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:04.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.638+0000 7ff131e00640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1180a3bb0 0x7ff1180a6070 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff124005ab0 tx=0x7ff124005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:04.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.641+0000 7ff12b400640 1 -- 172.21.15.86:0/1203265057 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff11c016030 con 0x7ff1341095b0 2024-04-17T07:36:04.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.845+0000 7ff133200640 1 -- 172.21.15.86:0/1203265057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff134062b60 con 0x7ff1341095b0 2024-04-17T07:36:04.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.846+0000 7ff12b400640 1 -- 172.21.15.86:0/1203265057 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7ff11c09d420 con 0x7ff1341095b0 2024-04-17T07:36:04.847 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:36:04.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 -- 172.21.15.86:0/1203265057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1180a3bb0 msgr2=0x7ff1180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:04.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1180a3bb0 0x7ff1180a6070 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff124005ab0 tx=0x7ff124005fd0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 -- 172.21.15.86:0/1203265057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1341095b0 msgr2=0x7ff13419eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:04.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1341095b0 0x7ff13419eb80 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00eab0 tx=0x7ff11c00ef70 comp rx=0 tx=0).stop 2024-04-17T07:36:04.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 -- 172.21.15.86:0/1203265057 shutdown_connections 2024-04-17T07:36:04.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7ff1180a3bb0 0x7ff1180a6070 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff134109f70 0x7ff1341a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1341095b0 0x7ff13419eb80 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 --2- 172.21.15.86:0/1203265057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1341083c0 0x7ff13419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:04.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 -- 172.21.15.86:0/1203265057 >> 172.21.15.86:0/1203265057 conn(0x7ff134077360 msgr2=0x7ff13410c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:04.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 -- 172.21.15.86:0/1203265057 shutdown_connections 2024-04-17T07:36:04.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:04.850+0000 7ff133200640 1 -- 172.21.15.86:0/1203265057 wait complete. 2024-04-17T07:36:04.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:36:05.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:04 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:05.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:04 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:05.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:04 smithi103 ceph-mon[35495]: pgmap v1091: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:36:05.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:04 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2339478680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:36:05.027 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:04 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1853121285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:36:05.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.078+0000 7f4d65200640 1 -- 172.21.15.86:0/1805860226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d60102a20 msgr2=0x7f4d60104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:05.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.078+0000 7f4d65200640 1 --2- 172.21.15.86:0/1805860226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d60102a20 0x7f4d60104e10 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009900 tx=0x7f4d5002bc30 comp rx=0 tx=0).stop 2024-04-17T07:36:05.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.078+0000 7f4d65200640 1 -- 172.21.15.86:0/1805860226 shutdown_connections 2024-04-17T07:36:05.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.078+0000 7f4d65200640 1 --2- 172.21.15.86:0/1805860226 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d60107c80 0x7f4d6010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.078+0000 7f4d65200640 1 --2- 172.21.15.86:0/1805860226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d60105350 0x7f4d60107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.078+0000 7f4d65200640 1 --2- 172.21.15.86:0/1805860226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d60102a20 0x7f4d60104e10 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.078+0000 7f4d65200640 1 -- 172.21.15.86:0/1805860226 >> 172.21.15.86:0/1805860226 conn(0x7f4d600fc450 msgr2=0x7f4d600fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:05.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.078+0000 7f4d65200640 1 -- 172.21.15.86:0/1805860226 shutdown_connections 2024-04-17T07:36:05.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.078+0000 7f4d65200640 1 -- 172.21.15.86:0/1805860226 wait complete. 2024-04-17T07:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.079+0000 7f4d65200640 1 Processor -- start 2024-04-17T07:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.079+0000 7f4d65200640 1 -- start start 2024-04-17T07:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.079+0000 7f4d65200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d60102a20 0x7f4d60102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.079+0000 7f4d65200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d60105350 0x7f4d601009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:05.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:04 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:05.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:04 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:05.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:04 smithi086 ceph-mon[31733]: pgmap v1091: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:36:05.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:04 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2339478680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:36:05.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:04 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1853121285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d65200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d60107c80 0x7f4d60100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d65200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6010dae0 con 0x7f4d60102a20 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d65200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6010d960 con 0x7f4d60105350 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d65200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6010dc60 con 0x7f4d60107c80 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d5de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d60102a20 0x7f4d60102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d5de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d60102a20 0x7f4d60102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51730/0 (socket says 172.21.15.86:51730) 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d5de00640 1 -- 172.21.15.86:0/2946978043 learned_addr learned my addr 172.21.15.86:0/2946978043 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d5d400640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d60105350 0x7f4d601009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d5e800640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d60107c80 0x7f4d60100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d5de00640 1 -- 172.21.15.86:0/2946978043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d60107c80 msgr2=0x7f4d60100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d5de00640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d60107c80 0x7f4d60100f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d5de00640 1 -- 172.21.15.86:0/2946978043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d60105350 msgr2=0x7f4d601009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d5de00640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d60105350 0x7f4d601009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.080+0000 7f4d5de00640 1 -- 172.21.15.86:0/2946978043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d601017a0 con 0x7f4d60102a20 2024-04-17T07:36:05.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.081+0000 7f4d5de00640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d60102a20 0x7f4d60102390 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f4d50006010 tx=0x7f4d500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:05.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.081+0000 7f4d56a00640 1 -- 172.21.15.86:0/2946978043 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50043020 con 0x7f4d60102a20 2024-04-17T07:36:05.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.081+0000 7f4d56a00640 1 -- 172.21.15.86:0/2946978043 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d50004aa0 con 0x7f4d60102a20 2024-04-17T07:36:05.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.081+0000 7f4d56a00640 1 -- 172.21.15.86:0/2946978043 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50003c60 con 0x7f4d60102a20 2024-04-17T07:36:05.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.081+0000 7f4d65200640 1 -- 172.21.15.86:0/2946978043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d601a2340 con 0x7f4d60102a20 2024-04-17T07:36:05.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.081+0000 7f4d65200640 1 -- 172.21.15.86:0/2946978043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d601a2840 con 0x7f4d60102a20 2024-04-17T07:36:05.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.083+0000 7f4d65200640 1 -- 172.21.15.86:0/2946978043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d600fdf40 con 0x7f4d60102a20 2024-04-17T07:36:05.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.084+0000 7f4d56a00640 1 -- 172.21.15.86:0/2946978043 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f4d50046020 con 0x7f4d60102a20 2024-04-17T07:36:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.086+0000 7f4d56a00640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d440a3c30 0x7f4d440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.086+0000 7f4d56a00640 1 -- 172.21.15.86:0/2946978043 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f4d500f2bd0 con 0x7f4d60102a20 2024-04-17T07:36:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.086+0000 7f4d5d400640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d440a3c30 0x7f4d440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:05.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.087+0000 7f4d5d400640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d440a3c30 0x7f4d440a60f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c005fd0 tx=0x7f4d4c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:05.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.090+0000 7f4d56a00640 1 -- 172.21.15.86:0/2946978043 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4d5004d020 con 0x7f4d60102a20 2024-04-17T07:36:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:04 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:04 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:04 smithi098 ceph-mon[35418]: pgmap v1091: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-17T07:36:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:04 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2339478680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:36:05.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:04 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1853121285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:36:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.301+0000 7f4d65200640 1 -- 172.21.15.86:0/2946978043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4d60062b60 con 0x7f4d60102a20 2024-04-17T07:36:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.302+0000 7f4d56a00640 1 -- 172.21.15.86:0/2946978043 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f4d500bf1d0 con 0x7f4d60102a20 2024-04-17T07:36:05.302 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.305+0000 7f4d65200640 1 -- 172.21.15.86:0/2946978043 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d440a3c30 msgr2=0x7f4d440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.305+0000 7f4d65200640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d440a3c30 0x7f4d440a60f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c005fd0 tx=0x7f4d4c005850 comp rx=0 tx=0).stop 2024-04-17T07:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.306+0000 7f4d65200640 1 -- 172.21.15.86:0/2946978043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d60102a20 msgr2=0x7f4d60102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.306+0000 7f4d65200640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d60102a20 0x7f4d60102390 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f4d50006010 tx=0x7f4d500042f0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.306+0000 7f4d65200640 1 -- 172.21.15.86:0/2946978043 shutdown_connections 2024-04-17T07:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.306+0000 7f4d65200640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f4d440a3c30 0x7f4d440a60f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.306+0000 7f4d65200640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4d60107c80 0x7f4d60100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.306+0000 7f4d65200640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d60105350 0x7f4d601009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.306+0000 7f4d65200640 1 --2- 172.21.15.86:0/2946978043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d60102a20 0x7f4d60102390 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.306+0000 7f4d65200640 1 -- 172.21.15.86:0/2946978043 >> 172.21.15.86:0/2946978043 conn(0x7f4d600fc450 msgr2=0x7f4d60103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.306+0000 7f4d65200640 1 -- 172.21.15.86:0/2946978043 shutdown_connections 2024-04-17T07:36:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.306+0000 7f4d65200640 1 -- 172.21.15.86:0/2946978043 wait complete. 2024-04-17T07:36:05.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T07:36:05.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.533+0000 7f5667200640 1 -- 172.21.15.86:0/1831603860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5660101f60 msgr2=0x7f56601023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:05.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.533+0000 7f5667200640 1 --2- 172.21.15.86:0/1831603860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5660101f60 0x7f56601023e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f5650009900 tx=0x7f565002bc30 comp rx=0 tx=0).stop 2024-04-17T07:36:05.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.533+0000 7f5667200640 1 -- 172.21.15.86:0/1831603860 shutdown_connections 2024-04-17T07:36:05.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.533+0000 7f5667200640 1 --2- 172.21.15.86:0/1831603860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5660102920 0x7f56601091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.534+0000 7f5667200640 1 --2- 172.21.15.86:0/1831603860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5660101f60 0x7f56601023e0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.534+0000 7f5667200640 1 --2- 172.21.15.86:0/1831603860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5660100d70 0x7f5660101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.534+0000 7f5667200640 1 -- 172.21.15.86:0/1831603860 >> 172.21.15.86:0/1831603860 conn(0x7f56600fc300 msgr2=0x7f56600fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:05.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.534+0000 7f5667200640 1 -- 172.21.15.86:0/1831603860 shutdown_connections 2024-04-17T07:36:05.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.534+0000 7f5667200640 1 -- 172.21.15.86:0/1831603860 wait complete. 2024-04-17T07:36:05.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.535+0000 7f5667200640 1 Processor -- start 2024-04-17T07:36:05.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.535+0000 7f5667200640 1 -- start start 2024-04-17T07:36:05.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.535+0000 7f5667200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5660100d70 0x7f566019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:05.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.535+0000 7f5667200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5660101f60 0x7f566019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:05.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.535+0000 7f5667200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5660102920 0x7f56601a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:05.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.535+0000 7f5667200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f566010c8a0 con 0x7f5660101f60 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.535+0000 7f5667200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f566010c720 con 0x7f5660100d70 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.535+0000 7f5667200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f566010ca20 con 0x7f5660102920 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.536+0000 7f5665400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5660101f60 0x7f566019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.536+0000 7f5665400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5660101f60 0x7f566019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51746/0 (socket says 172.21.15.86:51746) 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.536+0000 7f5665400640 1 -- 172.21.15.86:0/1710030273 learned_addr learned my addr 172.21.15.86:0/1710030273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.536+0000 7f5665e00640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5660100d70 0x7f566019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.536+0000 7f5666800640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5660102920 0x7f56601a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.536+0000 7f5665400640 1 -- 172.21.15.86:0/1710030273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5660102920 msgr2=0x7f56601a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.536+0000 7f5665400640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5660102920 0x7f56601a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.536+0000 7f5665400640 1 -- 172.21.15.86:0/1710030273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5660100d70 msgr2=0x7f566019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.536+0000 7f5665400640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5660100d70 0x7f566019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.536+0000 7f5665400640 1 -- 172.21.15.86:0/1710030273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56601a1ef0 con 0x7f5660101f60 2024-04-17T07:36:05.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.537+0000 7f5665400640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5660101f60 0x7f566019a810 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f5650006010 tx=0x7f56500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:05.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.537+0000 7f5656a00640 1 -- 172.21.15.86:0/1710030273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650043020 con 0x7f5660101f60 2024-04-17T07:36:05.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.537+0000 7f5656a00640 1 -- 172.21.15.86:0/1710030273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5650004aa0 con 0x7f5660101f60 2024-04-17T07:36:05.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.537+0000 7f5667200640 1 -- 172.21.15.86:0/1710030273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56601a2180 con 0x7f5660101f60 2024-04-17T07:36:05.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.537+0000 7f5656a00640 1 -- 172.21.15.86:0/1710030273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650003c60 con 0x7f5660101f60 2024-04-17T07:36:05.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.537+0000 7f5667200640 1 -- 172.21.15.86:0/1710030273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56601a2720 con 0x7f5660101f60 2024-04-17T07:36:05.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.538+0000 7f5667200640 1 -- 172.21.15.86:0/1710030273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56601a2920 con 0x7f5660101f60 2024-04-17T07:36:05.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.540+0000 7f5656a00640 1 -- 172.21.15.86:0/1710030273 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5650046020 con 0x7f5660101f60 2024-04-17T07:36:05.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.541+0000 7f5656a00640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f564c0a3c30 0x7f564c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:05.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.542+0000 7f5656a00640 1 -- 172.21.15.86:0/1710030273 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f56500f2bd0 con 0x7f5660101f60 2024-04-17T07:36:05.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.542+0000 7f5665e00640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f564c0a3c30 0x7f564c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:05.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.543+0000 7f5665e00640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f564c0a3c30 0x7f564c0a60f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5660101dc0 tx=0x7f5658009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:05.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.546+0000 7f5656a00640 1 -- 172.21.15.86:0/1710030273 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f565004d020 con 0x7f5660101f60 2024-04-17T07:36:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.761+0000 7f5667200640 1 -- 172.21.15.86:0/1710030273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f56601a2af0 con 0x7f5660101f60 2024-04-17T07:36:05.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.762+0000 7f5656a00640 1 -- 172.21.15.86:0/1710030273 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+591 (secure 0 0 0) 0x7f56500bf1d0 con 0x7f5660101f60 2024-04-17T07:36:05.762 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/174556107 2024-04-18T07:02:26.881886+0000 2024-04-17T07:36:05.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2497304884 2024-04-18T07:02:26.881886+0000 2024-04-17T07:36:05.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2497304884 2024-04-18T07:02:26.881886+0000 2024-04-17T07:36:05.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2537127051 2024-04-18T07:01:52.772350+0000 2024-04-17T07:36:05.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2158768684 2024-04-18T07:02:26.881886+0000 2024-04-17T07:36:05.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3172534015 2024-04-18T07:02:26.881886+0000 2024-04-17T07:36:05.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/4176177513 2024-04-18T07:01:52.772350+0000 2024-04-17T07:36:05.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3239168822 2024-04-18T07:01:52.772350+0000 2024-04-17T07:36:05.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/4176177513 2024-04-18T07:01:52.772350+0000 2024-04-17T07:36:05.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3864604926 2024-04-18T07:01:52.772350+0000 2024-04-17T07:36:05.763 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-04-17T07:36:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.764+0000 7f5667200640 1 -- 172.21.15.86:0/1710030273 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f564c0a3c30 msgr2=0x7f564c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.764+0000 7f5667200640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f564c0a3c30 0x7f564c0a60f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5660101dc0 tx=0x7f5658009870 comp rx=0 tx=0).stop 2024-04-17T07:36:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.764+0000 7f5667200640 1 -- 172.21.15.86:0/1710030273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5660101f60 msgr2=0x7f566019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.764+0000 7f5667200640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5660101f60 0x7f566019a810 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f5650006010 tx=0x7f56500042f0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.765+0000 7f5667200640 1 -- 172.21.15.86:0/1710030273 shutdown_connections 2024-04-17T07:36:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.765+0000 7f5667200640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f564c0a3c30 0x7f564c0a60f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.765+0000 7f5667200640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5660102920 0x7f56601a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.765+0000 7f5667200640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5660101f60 0x7f566019a810 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.765+0000 7f5667200640 1 --2- 172.21.15.86:0/1710030273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5660100d70 0x7f566019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.765+0000 7f5667200640 1 -- 172.21.15.86:0/1710030273 >> 172.21.15.86:0/1710030273 conn(0x7f56600fc300 msgr2=0x7f56600fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.765+0000 7f5667200640 1 -- 172.21.15.86:0/1710030273 shutdown_connections 2024-04-17T07:36:05.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:05.765+0000 7f5667200640 1 -- 172.21.15.86:0/1710030273 wait complete. 2024-04-17T07:36:05.782 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-17T07:36:05.782 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T07:36:05.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T07:36:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:05 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1203265057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:36:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:36:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:36:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:05 smithi086 ceph-mon[31733]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:05 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2946978043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:36:06.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:05 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/1710030273' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:36:06.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.085+0000 7f6e67200640 1 -- 172.21.15.86:0/1774340735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e68101dd0 msgr2=0x7f6e681021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.085+0000 7f6e67200640 1 --2- 172.21.15.86:0/1774340735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e68101dd0 0x7f6e681021f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009900 tx=0x7f6e5802bc30 comp rx=0 tx=0).stop 2024-04-17T07:36:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.085+0000 7f6e67200640 1 -- 172.21.15.86:0/1774340735 shutdown_connections 2024-04-17T07:36:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.085+0000 7f6e67200640 1 --2- 172.21.15.86:0/1774340735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e68103980 0x7f6e6810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.085+0000 7f6e67200640 1 --2- 172.21.15.86:0/1774340735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68102fc0 0x7f6e68103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.085+0000 7f6e67200640 1 --2- 172.21.15.86:0/1774340735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e68101dd0 0x7f6e681021f0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.085+0000 7f6e67200640 1 -- 172.21.15.86:0/1774340735 >> 172.21.15.86:0/1774340735 conn(0x7f6e680fd340 msgr2=0x7f6e680ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.085+0000 7f6e67200640 1 -- 172.21.15.86:0/1774340735 shutdown_connections 2024-04-17T07:36:06.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.085+0000 7f6e67200640 1 -- 172.21.15.86:0/1774340735 wait complete. 2024-04-17T07:36:06.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.086+0000 7f6e67200640 1 Processor -- start 2024-04-17T07:36:06.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.086+0000 7f6e67200640 1 -- start start 2024-04-17T07:36:06.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e67200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e68101dd0 0x7f6e6819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e67200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e68102fc0 0x7f6e6819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e67200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68103980 0x7f6e681a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e67200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6810d920 con 0x7f6e68102fc0 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e67200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6810d7a0 con 0x7f6e68103980 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e67200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6810daa0 con 0x7f6e68101dd0 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e5d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e68102fc0 0x7f6e6819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e5d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e68102fc0 0x7f6e6819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51750/0 (socket says 172.21.15.86:51750) 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e5d400640 1 -- 172.21.15.86:0/423900074 learned_addr learned my addr 172.21.15.86:0/423900074 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e65e00640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e68101dd0 0x7f6e6819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e66800640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68103980 0x7f6e681a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e5d400640 1 -- 172.21.15.86:0/423900074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e68101dd0 msgr2=0x7f6e6819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e5d400640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e68101dd0 0x7f6e6819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e5d400640 1 -- 172.21.15.86:0/423900074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68103980 msgr2=0x7f6e681a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e5d400640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68103980 0x7f6e681a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e5d400640 1 -- 172.21.15.86:0/423900074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e681a1ec0 con 0x7f6e68102fc0 2024-04-17T07:36:06.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.087+0000 7f6e65e00640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e68101dd0 0x7f6e6819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:36:06.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.088+0000 7f6e5d400640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e68102fc0 0x7f6e6819a7e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000d950 tx=0x7f6e5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:06.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.088+0000 7f6e5f400640 1 -- 172.21.15.86:0/423900074 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50017070 con 0x7f6e68102fc0 2024-04-17T07:36:06.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.088+0000 7f6e5f400640 1 -- 172.21.15.86:0/423900074 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e50009c30 con 0x7f6e68102fc0 2024-04-17T07:36:06.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.088+0000 7f6e5f400640 1 -- 172.21.15.86:0/423900074 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000f620 con 0x7f6e68102fc0 2024-04-17T07:36:06.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.088+0000 7f6e67200640 1 -- 172.21.15.86:0/423900074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e681a21b0 con 0x7f6e68102fc0 2024-04-17T07:36:06.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.088+0000 7f6e67200640 1 -- 172.21.15.86:0/423900074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e681a26b0 con 0x7f6e68102fc0 2024-04-17T07:36:06.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.089+0000 7f6e67200640 1 -- 172.21.15.86:0/423900074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e681a2910 con 0x7f6e68102fc0 2024-04-17T07:36:06.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.092+0000 7f6e5f400640 1 -- 172.21.15.86:0/423900074 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f6e500109a0 con 0x7f6e68102fc0 2024-04-17T07:36:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.093+0000 7f6e5f400640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e4c0a3c30 0x7f6e4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.093+0000 7f6e5f400640 1 -- 172.21.15.86:0/423900074 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f6e500d03f0 con 0x7f6e68102fc0 2024-04-17T07:36:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.093+0000 7f6e65e00640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e4c0a3c30 0x7f6e4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.094+0000 7f6e65e00640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e4c0a3c30 0x7f6e4c0a60f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6e58036040 tx=0x7f6e58005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.095+0000 7f6e5f400640 1 -- 172.21.15.86:0/423900074 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6e50009de0 con 0x7f6e68102fc0 2024-04-17T07:36:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:05 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1203265057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:36:06.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:36:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:36:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:05 smithi098 ceph-mon[35418]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:05 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2946978043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:36:06.254 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:05 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/1710030273' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:36:06.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:05 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1203265057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:36:06.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-17T07:36:06.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-17T07:36:06.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:05 smithi103 ceph-mon[35495]: from='mgr.14152 172.21.15.86:0/524712874' entity='mgr.x' 2024-04-17T07:36:06.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:05 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2946978043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:36:06.278 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:05 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/1710030273' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:36:06.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.303+0000 7f6e67200640 1 -- 172.21.15.86:0/423900074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e681a2ca0 con 0x7f6e68102fc0 2024-04-17T07:36:06.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.306+0000 7f6e5f400640 1 -- 172.21.15.86:0/423900074 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24632 (secure 0 0 0) 0x7f6e5009ca70 con 0x7f6e68102fc0 2024-04-17T07:36:06.307 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-17T07:36:06.307 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"1f7ed99a-fc88-11ee-bc91-c7b262605968","created":"2024-04-17T07:01:19.941306+0000","modified":"2024-04-17T07:13:29.152264+0000","last_up_change":"2024-04-17T07:10:47.722805+0000","last_in_change":"2024-04-17T07:10:21.903254+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T07:05:37.060296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T07:11:52.505855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T07:11:53.895972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T07:11:57.496923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"770a17ec-a9cd-4339-b3d5-1a841ac60e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6803","nonce":871033288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6805","nonce":871033288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6809","nonce":871033288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":871033288},{"type":"v1","addr":"172.21.15.86:6807","nonce":871033288}]},"public_addr":"172.21.15.86:6803/871033288","cluster_addr":"172.21.15.86:6805/871033288","heartbeat_back_addr":"172.21.15.86:6809/871033288","heartbeat_front_addr":"172.21.15.86:6807/871033288","state":["exists","up"]},{"osd":1,"uuid":"ded7c4aa-ef6b-4065-9f7f-b26aca9c1299","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6801","nonce":140628921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6803","nonce":140628921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6807","nonce":140628921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":140628921},{"type":"v1","addr":"172.21.15.98:6805","nonce":140628921}]},"public_addr":"172.21.15.98:6801/140628921","cluster_addr":"172.21.15.98:6803/140628921","heartbeat_back_addr":"172.21.15.98:6807/140628921","heartbeat_front_addr":"172.21.15.98:6805/140628921","state":["exists","up"]},{"osd":2,"uuid":"29d6fda2-deeb-4a3f-aa06-16bec72d0cc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6801","nonce":3728485789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6803","nonce":3728485789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6807","nonce":3728485789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":3728485789},{"type":"v1","addr":"172.21.15.103:6805","nonce":3728485789}]},"public_addr":"172.21.15.103:6801/3728485789","cluster_addr":"172.21.15.103:6803/3728485789","heartbeat_back_addr":"172.21.15.103:6807/3728485789","heartbeat_front_addr":"172.21.15.103:6805/3728485789","state":["exists","up"]},{"osd":3,"uuid":"2f6b9975-ef1c-4fd4-bb10-a3c92aa7dd9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6811","nonce":3089932177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6813","nonce":3089932177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6817","nonce":3089932177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":3089932177},{"type":"v1","addr":"172.21.15.86:6815","nonce":3089932177}]},"public_addr":"172.21.15.86:6811/3089932177","cluster_addr":"172.21.15.86:6813/3089932177","heartbeat_back_addr":"172.21.15.86:6817/3089932177","heartbeat_front_addr":"172.21.15.86:6815/3089932177","state":["exists","up"]},{"osd":4,"uuid":"0591e547-4420-43e7-9454-663746016d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6809","nonce":2532868809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6811","nonce":2532868809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6815","nonce":2532868809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2532868809},{"type":"v1","addr":"172.21.15.98:6813","nonce":2532868809}]},"public_addr":"172.21.15.98:6809/2532868809","cluster_addr":"172.21.15.98:6811/2532868809","heartbeat_back_addr":"172.21.15.98:6815/2532868809","heartbeat_front_addr":"172.21.15.98:6813/2532868809","state":["exists","up"]},{"osd":5,"uuid":"3965d5b0-7cd1-40fe-9b78-287e5a584a5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6809","nonce":3545437589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6811","nonce":3545437589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6815","nonce":3545437589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":3545437589},{"type":"v1","addr":"172.21.15.103:6813","nonce":3545437589}]},"public_addr":"172.21.15.103:6809/3545437589","cluster_addr":"172.21.15.103:6811/3545437589","heartbeat_back_addr":"172.21.15.103:6815/3545437589","heartbeat_front_addr":"172.21.15.103:6813/3545437589","state":["exists","up"]},{"osd":6,"uuid":"c695ed77-6e45-4218-88cd-40094d317983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6819","nonce":994878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6821","nonce":994878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6825","nonce":994878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":994878330},{"type":"v1","addr":"172.21.15.86:6823","nonce":994878330}]},"public_addr":"172.21.15.86:6819/994878330","cluster_addr":"172.21.15.86:6821/994878330","heartbeat_back_addr":"172.21.15.86:6825/994878330","heartbeat_front_addr":"172.21.15.86:6823/994878330","state":["exists","up"]},{"osd":7,"uuid":"eacf793e-e041-433e-ba5b-374fb4f1c6d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6817","nonce":3839591799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6819","nonce":3839591799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6823","nonce":3839591799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3839591799},{"type":"v1","addr":"172.21.15.98:6821","nonce":3839591799}]},"public_addr":"172.21.15.98:6817/3839591799","cluster_addr":"172.21.15.98:6819/3839591799","heartbeat_back_addr":"172.21.15.98:6823/3839591799","heartbeat_front_addr":"172.21.15.98:6821/3839591799","state":["exists","up"]},{"osd":8,"uuid":"6c7c6330-3ffb-45e1-b9b5-77db0f63a735","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6817","nonce":55757000}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6819","nonce":55757000}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6823","nonce":55757000}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":55757000},{"type":"v1","addr":"172.21.15.103:6821","nonce":55757000}]},"public_addr":"172.21.15.103:6817/55757000","cluster_addr":"172.21.15.103:6819/55757000","heartbeat_back_addr":"172.21.15.103:6823/55757000","heartbeat_front_addr":"172.21.15.103:6821/55757000","state":["exists","up"]},{"osd":9,"uuid":"c4f5184c-d482-4569-9b76-78377f60226d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6827","nonce":3368302207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6829","nonce":3368302207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6833","nonce":3368302207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3368302207},{"type":"v1","addr":"172.21.15.86:6831","nonce":3368302207}]},"public_addr":"172.21.15.86:6827/3368302207","cluster_addr":"172.21.15.86:6829/3368302207","heartbeat_back_addr":"172.21.15.86:6833/3368302207","heartbeat_front_addr":"172.21.15.86:6831/3368302207","state":["exists","up"]},{"osd":10,"uuid":"f42db156-1323-4e33-9d20-d3401ef400fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6825","nonce":2308079272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6827","nonce":2308079272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6831","nonce":2308079272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":2308079272},{"type":"v1","addr":"172.21.15.98:6829","nonce":2308079272}]},"public_addr":"172.21.15.98:6825/2308079272","cluster_addr":"172.21.15.98:6827/2308079272","heartbeat_back_addr":"172.21.15.98:6831/2308079272","heartbeat_front_addr":"172.21.15.98:6829/2308079272","state":["exists","up"]},{"osd":11,"uuid":"86c2d8c2-4547-437f-a8f4-d12f4a8047b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6825","nonce":1742700108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6827","nonce":1742700108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6831","nonce":1742700108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":1742700108},{"type":"v1","addr":"172.21.15.103:6829","nonce":1742700108}]},"public_addr":"172.21.15.103:6825/1742700108","cluster_addr":"172.21.15.103:6827/1742700108","heartbeat_back_addr":"172.21.15.103:6831/1742700108","heartbeat_front_addr":"172.21.15.103:6829/1742700108","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:04:29.711665+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:02.450586+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:05:34.142982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:07.869437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:06:41.880470+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:15.105005+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:07:49.961468+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:24.980473+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:08:59.282417+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:09:34.352417+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:09.456505+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T07:10:45.235292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.f","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/174556107":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6801/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/2497304884":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/2537127051":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/2158768684":"2024-04-18T07:02:26.881886+0000","172.21.15.86:0/3172534015":"2024-04-18T07:02:26.881886+0000","172.21.15.86:6800/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3239168822":"2024-04-18T07:01:52.772350+0000","172.21.15.86:6801/4176177513":"2024-04-18T07:01:52.772350+0000","172.21.15.86:0/3864604926":"2024-04-18T07:01:52.772350+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T07:36:06.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.309+0000 7f6e67200640 1 -- 172.21.15.86:0/423900074 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e4c0a3c30 msgr2=0x7f6e4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.309+0000 7f6e67200640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e4c0a3c30 0x7f6e4c0a60f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6e58036040 tx=0x7f6e58005fd0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.309+0000 7f6e67200640 1 -- 172.21.15.86:0/423900074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e68102fc0 msgr2=0x7f6e6819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.309+0000 7f6e67200640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e68102fc0 0x7f6e6819a7e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000d950 tx=0x7f6e5000de10 comp rx=0 tx=0).stop 2024-04-17T07:36:06.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.309+0000 7f6e67200640 1 -- 172.21.15.86:0/423900074 shutdown_connections 2024-04-17T07:36:06.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.309+0000 7f6e67200640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f6e4c0a3c30 0x7f6e4c0a60f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.309+0000 7f6e67200640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68103980 0x7f6e681a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.309+0000 7f6e67200640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e68102fc0 0x7f6e6819a7e0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.309+0000 7f6e67200640 1 --2- 172.21.15.86:0/423900074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e68101dd0 0x7f6e6819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.309+0000 7f6e67200640 1 -- 172.21.15.86:0/423900074 >> 172.21.15.86:0/423900074 conn(0x7f6e680fd340 msgr2=0x7f6e68074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:06.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.310+0000 7f6e67200640 1 -- 172.21.15.86:0/423900074 shutdown_connections 2024-04-17T07:36:06.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.310+0000 7f6e67200640 1 -- 172.21.15.86:0/423900074 wait complete. 2024-04-17T07:36:06.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T07:05:37.060296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T07:11:52.505855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T07:11:53.895972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T07:11:57.496923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T07:36:06.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T07:36:06.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.547+0000 7f14fbe00640 1 -- 172.21.15.86:0/2708042143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc103fc0 msgr2=0x7f14fc1043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.547+0000 7f14fbe00640 1 --2- 172.21.15.86:0/2708042143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc103fc0 0x7f14fc1043e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009900 tx=0x7f14ec02bc30 comp rx=0 tx=0).stop 2024-04-17T07:36:06.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.548+0000 7f14fbe00640 1 -- 172.21.15.86:0/2708042143 shutdown_connections 2024-04-17T07:36:06.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.548+0000 7f14fbe00640 1 --2- 172.21.15.86:0/2708042143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc105b70 0x7f14fc10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.548+0000 7f14fbe00640 1 --2- 172.21.15.86:0/2708042143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc1051b0 0x7f14fc105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.548+0000 7f14fbe00640 1 --2- 172.21.15.86:0/2708042143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc103fc0 0x7f14fc1043e0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.548+0000 7f14fbe00640 1 -- 172.21.15.86:0/2708042143 >> 172.21.15.86:0/2708042143 conn(0x7f14fc0ff530 msgr2=0x7f14fc101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:06.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.548+0000 7f14fbe00640 1 -- 172.21.15.86:0/2708042143 shutdown_connections 2024-04-17T07:36:06.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.548+0000 7f14fbe00640 1 -- 172.21.15.86:0/2708042143 wait complete. 2024-04-17T07:36:06.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.549+0000 7f14fbe00640 1 Processor -- start 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.549+0000 7f14fbe00640 1 -- start start 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.549+0000 7f14fbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc103fc0 0x7f14fc19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.549+0000 7f14fbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc1051b0 0x7f14fc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.549+0000 7f14fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc105b70 0x7f14fc1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.549+0000 7f14fbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc10fb50 con 0x7f14fc105b70 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.549+0000 7f14fbe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc10f9d0 con 0x7f14fc1051b0 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.549+0000 7f14fbe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc10fcd0 con 0x7f14fc103fc0 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.549+0000 7f14fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc105b70 0x7f14fc1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc105b70 0x7f14fc1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51780/0 (socket says 172.21.15.86:51780) 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14fb400640 1 -- 172.21.15.86:0/4154093815 learned_addr learned my addr 172.21.15.86:0/4154093815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14faa00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc103fc0 0x7f14fc19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14f3e00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc1051b0 0x7f14fc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14fb400640 1 -- 172.21.15.86:0/4154093815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc103fc0 msgr2=0x7f14fc19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14fb400640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc103fc0 0x7f14fc19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14fb400640 1 -- 172.21.15.86:0/4154093815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc1051b0 msgr2=0x7f14fc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14fb400640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc1051b0 0x7f14fc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14fb400640 1 -- 172.21.15.86:0/4154093815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc1a1df0 con 0x7f14fc105b70 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14f3e00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc1051b0 0x7f14fc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.550+0000 7f14fb400640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc105b70 0x7f14fc1a1780 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f14f400eaa0 tx=0x7f14f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.551+0000 7f14f3400640 1 -- 172.21.15.86:0/4154093815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f400cdb0 con 0x7f14fc105b70 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.551+0000 7f14f3400640 1 -- 172.21.15.86:0/4154093815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14f4005020 con 0x7f14fc105b70 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.551+0000 7f14fbe00640 1 -- 172.21.15.86:0/4154093815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc1a20e0 con 0x7f14fc105b70 2024-04-17T07:36:06.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.551+0000 7f14f3400640 1 -- 172.21.15.86:0/4154093815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40116d0 con 0x7f14fc105b70 2024-04-17T07:36:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.551+0000 7f14fbe00640 1 -- 172.21.15.86:0/4154093815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1a2630 con 0x7f14fc105b70 2024-04-17T07:36:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.552+0000 7f14fbe00640 1 -- 172.21.15.86:0/4154093815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14fc10b360 con 0x7f14fc105b70 2024-04-17T07:36:06.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.554+0000 7f14f3400640 1 -- 172.21.15.86:0/4154093815 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f14f40051d0 con 0x7f14fc105b70 2024-04-17T07:36:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.556+0000 7f14f3400640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14e00a3bb0 0x7f14e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.556+0000 7f14f3400640 1 -- 172.21.15.86:0/4154093815 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f14f40d0dd0 con 0x7f14fc105b70 2024-04-17T07:36:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.556+0000 7f14faa00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14e00a3bb0 0x7f14e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.557+0000 7f14faa00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14e00a3bb0 0x7f14e00a6070 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f14ec02c170 tx=0x7f14ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:06.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.559+0000 7f14f3400640 1 -- 172.21.15.86:0/4154093815 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f14f4016030 con 0x7f14fc105b70 2024-04-17T07:36:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.758+0000 7f14fbe00640 1 -- 172.21.15.86:0/4154093815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f14fc02d5d0 con 0x7f14fc105b70 2024-04-17T07:36:06.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.759+0000 7f14f3400640 1 -- 172.21.15.86:0/4154093815 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f14f409d430 con 0x7f14fc105b70 2024-04-17T07:36:06.760 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-17T07:36:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 -- 172.21.15.86:0/4154093815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14e00a3bb0 msgr2=0x7f14e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14e00a3bb0 0x7f14e00a6070 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f14ec02c170 tx=0x7f14ec000bc0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 -- 172.21.15.86:0/4154093815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc105b70 msgr2=0x7f14fc1a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc105b70 0x7f14fc1a1780 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f14f400eaa0 tx=0x7f14f400ef60 comp rx=0 tx=0).stop 2024-04-17T07:36:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 -- 172.21.15.86:0/4154093815 shutdown_connections 2024-04-17T07:36:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f14e00a3bb0 0x7f14e00a6070 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14fc105b70 0x7f14fc1a1780 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14fc1051b0 0x7f14fc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 --2- 172.21.15.86:0/4154093815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14fc103fc0 0x7f14fc19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 -- 172.21.15.86:0/4154093815 >> 172.21.15.86:0/4154093815 conn(0x7f14fc0ff530 msgr2=0x7f14fc108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 -- 172.21.15.86:0/4154093815 shutdown_connections 2024-04-17T07:36:06.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.762+0000 7f14fbe00640 1 -- 172.21.15.86:0/4154093815 wait complete. 2024-04-17T07:36:06.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T07:36:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.989+0000 7f5a9b200640 1 -- 172.21.15.86:0/2147980028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9c105230 msgr2=0x7f5a9c107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.989+0000 7f5a9b200640 1 --2- 172.21.15.86:0/2147980028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9c105230 0x7f5a9c107620 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009ca0 tx=0x7f5a8c02c100 comp rx=0 tx=0).stop 2024-04-17T07:36:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.990+0000 7f5a9b200640 1 -- 172.21.15.86:0/2147980028 shutdown_connections 2024-04-17T07:36:06.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.990+0000 7f5a9b200640 1 --2- 172.21.15.86:0/2147980028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c107b60 0x7f5a9c109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.990+0000 7f5a9b200640 1 --2- 172.21.15.86:0/2147980028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9c105230 0x7f5a9c107620 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.990+0000 7f5a9b200640 1 --2- 172.21.15.86:0/2147980028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a9c102900 0x7f5a9c104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.990+0000 7f5a9b200640 1 -- 172.21.15.86:0/2147980028 >> 172.21.15.86:0/2147980028 conn(0x7f5a9c0fc310 msgr2=0x7f5a9c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.990+0000 7f5a9b200640 1 -- 172.21.15.86:0/2147980028 shutdown_connections 2024-04-17T07:36:06.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.990+0000 7f5a9b200640 1 -- 172.21.15.86:0/2147980028 wait complete. 2024-04-17T07:36:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9b200640 1 Processor -- start 2024-04-17T07:36:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9b200640 1 -- start start 2024-04-17T07:36:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a9c102900 0x7f5a9c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105230 0x7f5a9c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9c107b60 0x7f5a9c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c10d9b0 con 0x7f5a9c107b60 2024-04-17T07:36:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c10d830 con 0x7f5a9c102900 2024-04-17T07:36:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c10db30 con 0x7f5a9c105230 2024-04-17T07:36:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9c107b60 0x7f5a9c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9c107b60 0x7f5a9c19f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42946/0 (socket says 172.21.15.86:42946) 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a9a800640 1 -- 172.21.15.86:0/3943207850 learned_addr learned my addr 172.21.15.86:0/3943207850 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.991+0000 7f5a99e00640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a9c102900 0x7f5a9c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a99400640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105230 0x7f5a9c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a9a800640 1 -- 172.21.15.86:0/3943207850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105230 msgr2=0x7f5a9c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a9a800640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105230 0x7f5a9c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a9a800640 1 -- 172.21.15.86:0/3943207850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a9c102900 msgr2=0x7f5a9c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a9a800640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a9c102900 0x7f5a9c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a9a800640 1 -- 172.21.15.86:0/3943207850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c19fc80 con 0x7f5a9c107b60 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a99400640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105230 0x7f5a9c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a9a800640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9c107b60 0x7f5a9c19f610 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c970 tx=0x7f5a9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a92a00640 1 -- 172.21.15.86:0/3943207850 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94007bb0 con 0x7f5a9c107b60 2024-04-17T07:36:06.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a92a00640 1 -- 172.21.15.86:0/3943207850 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a94007d50 con 0x7f5a9c107b60 2024-04-17T07:36:06.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a92a00640 1 -- 172.21.15.86:0/3943207850 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94004e10 con 0x7f5a9c107b60 2024-04-17T07:36:06.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a9b200640 1 -- 172.21.15.86:0/3943207850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9c19ff70 con 0x7f5a9c107b60 2024-04-17T07:36:06.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.992+0000 7f5a9b200640 1 -- 172.21.15.86:0/3943207850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9c1a0480 con 0x7f5a9c107b60 2024-04-17T07:36:06.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.993+0000 7f5a9b200640 1 -- 172.21.15.86:0/3943207850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a9c0fde00 con 0x7f5a9c107b60 2024-04-17T07:36:06.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.995+0000 7f5a92a00640 1 -- 172.21.15.86:0/3943207850 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f5a94005020 con 0x7f5a9c107b60 2024-04-17T07:36:06.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.997+0000 7f5a92a00640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5a800a3c30 0x7f5a800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:06.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.998+0000 7f5a92a00640 1 -- 172.21.15.86:0/3943207850 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f5a940d07a0 con 0x7f5a9c107b60 2024-04-17T07:36:06.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.998+0000 7f5a99e00640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5a800a3c30 0x7f5a800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:06.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:06.998+0000 7f5a99e00640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5a800a3c30 0x7f5a800a60f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5a84005e10 tx=0x7f5a84005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:07.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.002+0000 7f5a92a00640 1 -- 172.21.15.86:0/3943207850 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5a94014030 con 0x7f5a9c107b60 2024-04-17T07:36:07.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:06 smithi086 ceph-mon[31733]: pgmap v1092: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:36:07.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:06 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/423900074' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:36:07.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:06 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/4154093815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:36:07.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.203+0000 7f5a9b200640 1 -- 172.21.15.86:0/3943207850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5a9c062b60 con 0x7f5a9c107b60 2024-04-17T07:36:07.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.204+0000 7f5a92a00640 1 -- 172.21.15.86:0/3943207850 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f5a9409cda0 con 0x7f5a9c107b60 2024-04-17T07:36:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:36:07.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.207+0000 7f5a9b200640 1 -- 172.21.15.86:0/3943207850 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5a800a3c30 msgr2=0x7f5a800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:07.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.207+0000 7f5a9b200640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5a800a3c30 0x7f5a800a60f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5a84005e10 tx=0x7f5a84005da0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.207+0000 7f5a9b200640 1 -- 172.21.15.86:0/3943207850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9c107b60 msgr2=0x7f5a9c19f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:07.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.207+0000 7f5a9b200640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9c107b60 0x7f5a9c19f610 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c970 tx=0x7f5a9400ce30 comp rx=0 tx=0).stop 2024-04-17T07:36:07.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.208+0000 7f5a9b200640 1 -- 172.21.15.86:0/3943207850 shutdown_connections 2024-04-17T07:36:07.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.208+0000 7f5a9b200640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f5a800a3c30 0x7f5a800a60f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.208+0000 7f5a9b200640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a9c107b60 0x7f5a9c19f610 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.208+0000 7f5a9b200640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c105230 0x7f5a9c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.208+0000 7f5a9b200640 1 --2- 172.21.15.86:0/3943207850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a9c102900 0x7f5a9c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.208+0000 7f5a9b200640 1 -- 172.21.15.86:0/3943207850 >> 172.21.15.86:0/3943207850 conn(0x7f5a9c0fc310 msgr2=0x7f5a9c1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:07.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.208+0000 7f5a9b200640 1 -- 172.21.15.86:0/3943207850 shutdown_connections 2024-04-17T07:36:07.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.208+0000 7f5a9b200640 1 -- 172.21.15.86:0/3943207850 wait complete. 2024-04-17T07:36:07.227 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T07:36:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:06 smithi098 ceph-mon[35418]: pgmap v1092: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:36:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:06 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/423900074' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:36:07.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:06 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/4154093815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:36:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:06 smithi103 ceph-mon[35495]: pgmap v1092: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-17T07:36:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:06 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/423900074' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-17T07:36:07.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:06 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/4154093815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-17T07:36:07.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.443+0000 7f8565200640 1 -- 172.21.15.86:0/2688994956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560105190 msgr2=0x7f8560105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:07.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.443+0000 7f8565200640 1 --2- 172.21.15.86:0/2688994956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560105190 0x7f8560105610 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f854c009900 tx=0x7f854c02bc30 comp rx=0 tx=0).stop 2024-04-17T07:36:07.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.443+0000 7f8565200640 1 -- 172.21.15.86:0/2688994956 shutdown_connections 2024-04-17T07:36:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.443+0000 7f8565200640 1 --2- 172.21.15.86:0/2688994956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8560105b50 0x7f856010c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.443+0000 7f8565200640 1 --2- 172.21.15.86:0/2688994956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560105190 0x7f8560105610 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.443+0000 7f8565200640 1 --2- 172.21.15.86:0/2688994956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8560103fa0 0x7f85601043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.443+0000 7f8565200640 1 -- 172.21.15.86:0/2688994956 >> 172.21.15.86:0/2688994956 conn(0x7f85600ff530 msgr2=0x7f8560101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.443+0000 7f8565200640 1 -- 172.21.15.86:0/2688994956 shutdown_connections 2024-04-17T07:36:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.443+0000 7f8565200640 1 -- 172.21.15.86:0/2688994956 wait complete. 2024-04-17T07:36:07.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.444+0000 7f8565200640 1 Processor -- start 2024-04-17T07:36:07.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.444+0000 7f8565200640 1 -- start start 2024-04-17T07:36:07.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.444+0000 7f8565200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560103fa0 0x7f856019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:07.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.444+0000 7f8565200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8560105190 0x7f856019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:07.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.444+0000 7f8565200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8560105b50 0x7f85601a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:07.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.444+0000 7f8565200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f856010fb90 con 0x7f8560103fa0 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.444+0000 7f8565200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f856010fa10 con 0x7f8560105190 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.444+0000 7f8565200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f856010fd10 con 0x7f8560105b50 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.444+0000 7f855f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560103fa0 0x7f856019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560103fa0 0x7f856019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42964/0 (socket says 172.21.15.86:42964) 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855f400640 1 -- 172.21.15.86:0/3956487860 learned_addr learned my addr 172.21.15.86:0/3956487860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855ea00640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8560105190 0x7f856019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855fe00640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8560105b50 0x7f85601a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855f400640 1 -- 172.21.15.86:0/3956487860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8560105b50 msgr2=0x7f85601a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855f400640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8560105b50 0x7f85601a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855f400640 1 -- 172.21.15.86:0/3956487860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8560105190 msgr2=0x7f856019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855f400640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8560105190 0x7f856019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855f400640 1 -- 172.21.15.86:0/3956487860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85601a1e20 con 0x7f8560103fa0 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855fe00640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8560105b50 0x7f85601a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:36:07.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f855f400640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560103fa0 0x7f856019a200 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f855000c970 tx=0x7f855000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.445+0000 7f8557e00640 1 -- 172.21.15.86:0/3956487860 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550004220 con 0x7f8560103fa0 2024-04-17T07:36:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.446+0000 7f8557e00640 1 -- 172.21.15.86:0/3956487860 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85500043c0 con 0x7f8560103fa0 2024-04-17T07:36:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.446+0000 7f8565200640 1 -- 172.21.15.86:0/3956487860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85601a2110 con 0x7f8560103fa0 2024-04-17T07:36:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.446+0000 7f8557e00640 1 -- 172.21.15.86:0/3956487860 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550004a00 con 0x7f8560103fa0 2024-04-17T07:36:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.446+0000 7f8565200640 1 -- 172.21.15.86:0/3956487860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85601a2690 con 0x7f8560103fa0 2024-04-17T07:36:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.447+0000 7f8565200640 1 -- 172.21.15.86:0/3956487860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856010b3b0 con 0x7f8560103fa0 2024-04-17T07:36:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.449+0000 7f8557e00640 1 -- 172.21.15.86:0/3956487860 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f85500079b0 con 0x7f8560103fa0 2024-04-17T07:36:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.451+0000 7f8557e00640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85440a3bb0 0x7f85440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.451+0000 7f8557e00640 1 -- 172.21.15.86:0/3956487860 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f85500d05f0 con 0x7f8560103fa0 2024-04-17T07:36:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.452+0000 7f855ea00640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85440a3bb0 0x7f85440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.452+0000 7f855ea00640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85440a3bb0 0x7f85440a6070 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f856019b710 tx=0x7f854c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:07.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.456+0000 7f8557e00640 1 -- 172.21.15.86:0/3956487860 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8550014030 con 0x7f8560103fa0 2024-04-17T07:36:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.662+0000 7f8565200640 1 -- 172.21.15.86:0/3956487860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8560062b60 con 0x7f8560103fa0 2024-04-17T07:36:07.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.663+0000 7f8557e00640 1 -- 172.21.15.86:0/3956487860 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f855009cbf0 con 0x7f8560103fa0 2024-04-17T07:36:07.663 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-17T07:36:07.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 -- 172.21.15.86:0/3956487860 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85440a3bb0 msgr2=0x7f85440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:07.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85440a3bb0 0x7f85440a6070 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f856019b710 tx=0x7f854c000bc0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 -- 172.21.15.86:0/3956487860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560103fa0 msgr2=0x7f856019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:07.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560103fa0 0x7f856019a200 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f855000c970 tx=0x7f855000ce30 comp rx=0 tx=0).stop 2024-04-17T07:36:07.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 -- 172.21.15.86:0/3956487860 shutdown_connections 2024-04-17T07:36:07.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f85440a3bb0 0x7f85440a6070 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8560105b50 0x7f85601a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8560105190 0x7f856019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 --2- 172.21.15.86:0/3956487860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8560103fa0 0x7f856019a200 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 -- 172.21.15.86:0/3956487860 >> 172.21.15.86:0/3956487860 conn(0x7f85600ff530 msgr2=0x7f8560108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:07.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 -- 172.21.15.86:0/3956487860 shutdown_connections 2024-04-17T07:36:07.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.667+0000 7f8565200640 1 -- 172.21.15.86:0/3956487860 wait complete. 2024-04-17T07:36:07.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T07:36:07.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.907+0000 7f047ee00640 1 -- 172.21.15.86:0/3227355716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478078500 msgr2=0x7f0478078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:07.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.908+0000 7f047ee00640 1 --2- 172.21.15.86:0/3227355716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478078500 0x7f0478078da0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f0470009ca0 tx=0x7f047002c100 comp rx=0 tx=0).stop 2024-04-17T07:36:07.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.908+0000 7f047ee00640 1 -- 172.21.15.86:0/3227355716 shutdown_connections 2024-04-17T07:36:07.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.908+0000 7f047ee00640 1 --2- 172.21.15.86:0/3227355716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478078500 0x7f0478078da0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.908+0000 7f047ee00640 1 --2- 172.21.15.86:0/3227355716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0478077b60 0x7f0478077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.908+0000 7f047ee00640 1 --2- 172.21.15.86:0/3227355716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0478076920 0x7f0478076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.908+0000 7f047ee00640 1 -- 172.21.15.86:0/3227355716 >> 172.21.15.86:0/3227355716 conn(0x7f04780ff610 msgr2=0x7f0478101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:07.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.908+0000 7f047ee00640 1 -- 172.21.15.86:0/3227355716 shutdown_connections 2024-04-17T07:36:07.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.908+0000 7f047ee00640 1 -- 172.21.15.86:0/3227355716 wait complete. 2024-04-17T07:36:07.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047ee00640 1 Processor -- start 2024-04-17T07:36:07.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047ee00640 1 -- start start 2024-04-17T07:36:07.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047ee00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0478076920 0x7f047819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:07.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047ee00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0478077b60 0x7f047819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:07.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478078500 0x7f04781a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:07.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0478113fe0 con 0x7f0478078500 2024-04-17T07:36:07.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047ee00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0478113e60 con 0x7f0478077b60 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047ee00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0478114160 con 0x7f0478076920 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478078500 0x7f04781a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478078500 0x7f04781a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42974/0 (socket says 172.21.15.86:42974) 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.909+0000 7f047e400640 1 -- 172.21.15.86:0/2245408869 learned_addr learned my addr 172.21.15.86:0/2245408869 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f047d000640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0478077b60 0x7f047819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f047da00640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0478076920 0x7f047819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f047e400640 1 -- 172.21.15.86:0/2245408869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0478076920 msgr2=0x7f047819e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f047e400640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0478076920 0x7f047819e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f047e400640 1 -- 172.21.15.86:0/2245408869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0478077b60 msgr2=0x7f047819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f047e400640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0478077b60 0x7f047819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f047e400640 1 -- 172.21.15.86:0/2245408869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04781a6220 con 0x7f0478078500 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f047da00640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0478076920 0x7f047819e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f047e400640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478078500 0x7f04781a5bb0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0470032520 tx=0x7f0470034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:07.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f0476000640 1 -- 172.21.15.86:0/2245408869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470039040 con 0x7f0478078500 2024-04-17T07:36:07.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f0476000640 1 -- 172.21.15.86:0/2245408869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0470044070 con 0x7f0478078500 2024-04-17T07:36:07.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.910+0000 7f0476000640 1 -- 172.21.15.86:0/2245408869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470004430 con 0x7f0478078500 2024-04-17T07:36:07.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.911+0000 7f047ee00640 1 -- 172.21.15.86:0/2245408869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04781a64b0 con 0x7f0478078500 2024-04-17T07:36:07.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.911+0000 7f047ee00640 1 -- 172.21.15.86:0/2245408869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04781a6950 con 0x7f0478078500 2024-04-17T07:36:07.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.912+0000 7f047ee00640 1 -- 172.21.15.86:0/2245408869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0478110a10 con 0x7f0478078500 2024-04-17T07:36:07.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.914+0000 7f0476000640 1 -- 172.21.15.86:0/2245408869 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f0470003bc0 con 0x7f0478078500 2024-04-17T07:36:07.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.915+0000 7f0476000640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04600a3bb0 0x7f04600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:07.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.916+0000 7f0476000640 1 -- 172.21.15.86:0/2245408869 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f04700f35a0 con 0x7f0478078500 2024-04-17T07:36:07.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.916+0000 7f047da00640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04600a3bb0 0x7f04600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:07.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.917+0000 7f047da00640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04600a3bb0 0x7f04600a6070 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f046c009b60 tx=0x7f046c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:07.918+0000 7f0476000640 1 -- 172.21.15.86:0/2245408869 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f04700bfba0 con 0x7f0478078500 2024-04-17T07:36:08.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:07 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3943207850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:36:08.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:07 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/3956487860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:36:08.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.122+0000 7f047ee00640 1 -- 172.21.15.86:0/2245408869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0478062b60 con 0x7f0478078500 2024-04-17T07:36:08.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.123+0000 7f0476000640 1 -- 172.21.15.86:0/2245408869 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f04700bf4a0 con 0x7f0478078500 2024-04-17T07:36:08.127 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-17T07:36:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.125+0000 7f047ee00640 1 -- 172.21.15.86:0/2245408869 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04600a3bb0 msgr2=0x7f04600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.125+0000 7f047ee00640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04600a3bb0 0x7f04600a6070 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f046c009b60 tx=0x7f046c005e70 comp rx=0 tx=0).stop 2024-04-17T07:36:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.126+0000 7f047ee00640 1 -- 172.21.15.86:0/2245408869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478078500 msgr2=0x7f04781a5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.126+0000 7f047ee00640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478078500 0x7f04781a5bb0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0470032520 tx=0x7f0470034880 comp rx=0 tx=0).stop 2024-04-17T07:36:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.126+0000 7f047ee00640 1 -- 172.21.15.86:0/2245408869 shutdown_connections 2024-04-17T07:36:08.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.126+0000 7f047ee00640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f04600a3bb0 0x7f04600a6070 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.126+0000 7f047ee00640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0478078500 0x7f04781a5bb0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.126+0000 7f047ee00640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0478077b60 0x7f047819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.126+0000 7f047ee00640 1 --2- 172.21.15.86:0/2245408869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0478076920 0x7f047819e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.126+0000 7f047ee00640 1 -- 172.21.15.86:0/2245408869 >> 172.21.15.86:0/2245408869 conn(0x7f04780ff610 msgr2=0x7f0478101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.126+0000 7f047ee00640 1 -- 172.21.15.86:0/2245408869 shutdown_connections 2024-04-17T07:36:08.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.126+0000 7f047ee00640 1 -- 172.21.15.86:0/2245408869 wait complete. 2024-04-17T07:36:08.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T07:36:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:07 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3943207850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:36:08.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:07 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/3956487860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:36:08.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:07 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3943207850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-17T07:36:08.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:07 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/3956487860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-17T07:36:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.352+0000 7f8f9a600640 1 -- 172.21.15.86:0/3117949025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94109690 msgr2=0x7f8f94109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:08.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.352+0000 7f8f9a600640 1 --2- 172.21.15.86:0/3117949025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94109690 0x7f8f94109b10 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009900 tx=0x7f8f8002bc40 comp rx=0 tx=0).stop 2024-04-17T07:36:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.352+0000 7f8f9a600640 1 -- 172.21.15.86:0/3117949025 shutdown_connections 2024-04-17T07:36:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.352+0000 7f8f9a600640 1 --2- 172.21.15.86:0/3117949025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f9410a050 0x7f8f941108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.352+0000 7f8f9a600640 1 --2- 172.21.15.86:0/3117949025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94109690 0x7f8f94109b10 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.352+0000 7f8f9a600640 1 --2- 172.21.15.86:0/3117949025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f941084a0 0x7f8f941088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.352+0000 7f8f9a600640 1 -- 172.21.15.86:0/3117949025 >> 172.21.15.86:0/3117949025 conn(0x7f8f94077430 msgr2=0x7f8f94079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.352+0000 7f8f9a600640 1 -- 172.21.15.86:0/3117949025 shutdown_connections 2024-04-17T07:36:08.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.352+0000 7f8f9a600640 1 -- 172.21.15.86:0/3117949025 wait complete. 2024-04-17T07:36:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.353+0000 7f8f9a600640 1 Processor -- start 2024-04-17T07:36:08.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.353+0000 7f8f9a600640 1 -- start start 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.353+0000 7f8f9a600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f941084a0 0x7f8f9419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.353+0000 7f8f9a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94109690 0x7f8f9419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.353+0000 7f8f9a600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f9410a050 0x7f8f941a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f9a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94114090 con 0x7f8f94109690 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f9a600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94113f10 con 0x7f8f9410a050 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f9a600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94114210 con 0x7f8f941084a0 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94109690 0x7f8f9419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94109690 0x7f8f9419eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42994/0 (socket says 172.21.15.86:42994) 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f93e00640 1 -- 172.21.15.86:0/2387151561 learned_addr learned my addr 172.21.15.86:0/2387151561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f99200640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f941084a0 0x7f8f9419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f99c00640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f9410a050 0x7f8f941a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f93e00640 1 -- 172.21.15.86:0/2387151561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f941084a0 msgr2=0x7f8f9419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f93e00640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f941084a0 0x7f8f9419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f93e00640 1 -- 172.21.15.86:0/2387151561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f9410a050 msgr2=0x7f8f941a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f93e00640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f9410a050 0x7f8f941a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f93e00640 1 -- 172.21.15.86:0/2387151561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f941a6380 con 0x7f8f94109690 2024-04-17T07:36:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f99c00640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f9410a050 0x7f8f941a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:36:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.354+0000 7f8f99200640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f941084a0 0x7f8f9419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T07:36:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.355+0000 7f8f93e00640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94109690 0x7f8f9419eca0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000c00 tx=0x7f8f800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.355+0000 7f8f91600640 1 -- 172.21.15.86:0/2387151561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80043020 con 0x7f8f94109690 2024-04-17T07:36:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.355+0000 7f8f91600640 1 -- 172.21.15.86:0/2387151561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f800054c0 con 0x7f8f94109690 2024-04-17T07:36:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.355+0000 7f8f9a600640 1 -- 172.21.15.86:0/2387151561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f941a6610 con 0x7f8f94109690 2024-04-17T07:36:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.355+0000 7f8f91600640 1 -- 172.21.15.86:0/2387151561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80003d80 con 0x7f8f94109690 2024-04-17T07:36:08.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.355+0000 7f8f9a600640 1 -- 172.21.15.86:0/2387151561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f941a6c30 con 0x7f8f94109690 2024-04-17T07:36:08.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.356+0000 7f8f9a600640 1 -- 172.21.15.86:0/2387151561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9410f860 con 0x7f8f94109690 2024-04-17T07:36:08.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.358+0000 7f8f91600640 1 -- 172.21.15.86:0/2387151561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138646+0+0 (secure 0 0 0) 0x7f8f80046020 con 0x7f8f94109690 2024-04-17T07:36:08.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.360+0000 7f8f91600640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f7c0a3c30 0x7f8f7c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T07:36:08.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.361+0000 7f8f91600640 1 -- 172.21.15.86:0/2387151561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9121+0+0 (secure 0 0 0) 0x7f8f800f2bf0 con 0x7f8f94109690 2024-04-17T07:36:08.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.361+0000 7f8f99200640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f7c0a3c30 0x7f8f7c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T07:36:08.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.361+0000 7f8f99200640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f7c0a3c30 0x7f8f7c0a60f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8f88006fd0 tx=0x7f8f88008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T07:36:08.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.365+0000 7f8f91600640 1 -- 172.21.15.86:0/2387151561 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8f8004d020 con 0x7f8f94109690 2024-04-17T07:36:08.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.584+0000 7f8f9a600640 1 -- 172.21.15.86:0/2387151561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8f9402d5d0 con 0x7f8f94109690 2024-04-17T07:36:08.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.586+0000 7f8f91600640 1 -- 172.21.15.86:0/2387151561 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+591 (secure 0 0 0) 0x7f8f800bf1f0 con 0x7f8f94109690 2024-04-17T07:36:08.586 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/174556107 2024-04-18T07:02:26.881886+0000 2024-04-17T07:36:08.586 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2497304884 2024-04-18T07:02:26.881886+0000 2024-04-17T07:36:08.586 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2497304884 2024-04-18T07:02:26.881886+0000 2024-04-17T07:36:08.586 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2537127051 2024-04-18T07:01:52.772350+0000 2024-04-17T07:36:08.586 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2158768684 2024-04-18T07:02:26.881886+0000 2024-04-17T07:36:08.586 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3172534015 2024-04-18T07:02:26.881886+0000 2024-04-17T07:36:08.586 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/4176177513 2024-04-18T07:01:52.772350+0000 2024-04-17T07:36:08.586 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3239168822 2024-04-18T07:01:52.772350+0000 2024-04-17T07:36:08.586 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/4176177513 2024-04-18T07:01:52.772350+0000 2024-04-17T07:36:08.586 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3864604926 2024-04-18T07:01:52.772350+0000 2024-04-17T07:36:08.587 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 -- 172.21.15.86:0/2387151561 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f7c0a3c30 msgr2=0x7f8f7c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f7c0a3c30 0x7f8f7c0a60f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8f88006fd0 tx=0x7f8f88008040 comp rx=0 tx=0).stop 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 -- 172.21.15.86:0/2387151561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94109690 msgr2=0x7f8f9419eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94109690 0x7f8f9419eca0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000c00 tx=0x7f8f800042f0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 -- 172.21.15.86:0/2387151561 shutdown_connections 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.86:6800/561620030,v1:172.21.15.86:6801/561620030] conn(0x7f8f7c0a3c30 0x7f8f7c0a60f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f9410a050 0x7f8f941a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f94109690 0x7f8f9419eca0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 --2- 172.21.15.86:0/2387151561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f941084a0 0x7f8f9419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 -- 172.21.15.86:0/2387151561 >> 172.21.15.86:0/2387151561 conn(0x7f8f94077430 msgr2=0x7f8f9410ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.588+0000 7f8f9a600640 1 -- 172.21.15.86:0/2387151561 shutdown_connections 2024-04-17T07:36:08.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-17T07:36:08.589+0000 7f8f9a600640 1 -- 172.21.15.86:0/2387151561 wait complete. 2024-04-17T07:36:08.609 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-17T07:36:08.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:36:08.609 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /tmp/cephfs 2024-04-17T07:36:08.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[83951]: fuse finished with error 0 and tester_r 0 2024-04-17T07:36:08.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:08 smithi086 ceph-mon[31733]: pgmap v1093: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:36:08.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:08 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2245408869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:36:08.950 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:08 smithi086 ceph-mon[31733]: from='client.? 172.21.15.86:0/2387151561' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:36:08.953 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T07:36:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:08 smithi098 ceph-mon[35418]: pgmap v1093: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:36:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:08 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2245408869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:36:09.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:08 smithi098 ceph-mon[35418]: from='client.? 172.21.15.86:0/2387151561' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:36:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:08 smithi103 ceph-mon[35495]: pgmap v1093: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-17T07:36:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:08 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2245408869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-17T07:36:09.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:08 smithi103 ceph-mon[35495]: from='client.? 172.21.15.86:0/2387151561' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-17T07:36:11.080 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:10 smithi086 ceph-mon[31733]: pgmap v1094: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-17T07:36:11.216 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:10 smithi098 ceph-mon[35418]: pgmap v1094: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-17T07:36:11.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:10 smithi103 ceph-mon[35495]: pgmap v1094: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-17T07:36:13.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:12 smithi098 ceph-mon[35418]: pgmap v1095: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-17T07:36:13.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:12 smithi103 ceph-mon[35495]: pgmap v1095: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-17T07:36:13.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:12 smithi086 ceph-mon[31733]: pgmap v1095: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-17T07:36:15.056 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-17T07:36:15.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:36:15.057 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-04-17T07:36:15.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:36:15.137 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-17T07:36:15.189 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-04-17T07:36:15.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:36:15.191 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-04-17T07:36:15.253 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:14 smithi098 ceph-mon[35418]: pgmap v1096: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-17T07:36:15.272 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:14 smithi086 ceph-mon[31733]: pgmap v1096: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-17T07:36:15.276 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:14 smithi103 ceph-mon[35495]: pgmap v1096: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-17T07:36:15.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T07:36:15.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-17T07:36:15.407 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-17T07:36:15.419 INFO:tasks.cephadm:Teardown begin 2024-04-17T07:36:15.419 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/cephadm.py", line 2246, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} 2024-04-17T07:36:15.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:36:15.519 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:36:15.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:36:15.723 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-17T07:36:15.723 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-17T07:36:15.780 DEBUG:teuthology.orchestra.run.smithi098:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-17T07:36:15.843 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-17T07:36:15.915 INFO:tasks.cephadm:Stopping all daemons... 2024-04-17T07:36:15.916 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-17T07:36:15.916 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.a 2024-04-17T07:36:16.331 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:16 smithi086 systemd[1]: Stopping Ceph mon.a for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:36:16.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:16 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a[31729]: 2024-04-17T07:36:16.549+0000 7f8797600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:36:16.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:16 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a[31729]: 2024-04-17T07:36:16.549+0000 7f8797600700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-17T07:36:16.831 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:16 smithi086 podman[114455]: 2024-04-17 07:36:16.62090844 +0000 UTC m=+0.223966221 container died c352ceee4038beb26a994da8880f0ab0d213c1f1e1dac4497541958f6e1fb122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:36:17.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:17 smithi086 podman[114455]: 2024-04-17 07:36:17.125523612 +0000 UTC m=+0.728581397 container cleanup c352ceee4038beb26a994da8880f0ab0d213c1f1e1dac4497541958f6e1fb122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:36:17.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:17 smithi086 bash[114455]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a 2024-04-17T07:36:17.830 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 17 07:36:17 smithi086 podman[114469]: 2024-04-17 07:36:17.36444085 +0000 UTC m=+0.731674407 container remove c352ceee4038beb26a994da8880f0ab0d213c1f1e1dac4497541958f6e1fb122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-17T07:36:17.907 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.a.service' 2024-04-17T07:36:18.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:36:18.049 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-17T07:36:18.049 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-17T07:36:18.049 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.b 2024-04-17T07:36:18.503 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:18 smithi098 systemd[1]: Stopping Ceph mon.b for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:36:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:18 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-b[35414]: 2024-04-17T07:36:18.743+0000 7fb830800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:36:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:18 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-b[35414]: 2024-04-17T07:36:18.743+0000 7fb830800700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-17T07:36:19.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:18 smithi098 podman[72324]: 2024-04-17 07:36:18.98004119 +0000 UTC m=+0.391128098 container died 35a692606e822c829b6d2c25081ab972f8c15656794bc9cd38a42d34a238d37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-17T07:36:19.742 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:19 smithi098 podman[72324]: 2024-04-17 07:36:19.439469284 +0000 UTC m=+0.850556186 container cleanup 35a692606e822c829b6d2c25081ab972f8c15656794bc9cd38a42d34a238d37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-17T07:36:19.743 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:19 smithi098 bash[72324]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-b 2024-04-17T07:36:20.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 17 07:36:19 smithi098 podman[72338]: 2024-04-17 07:36:19.741138652 +0000 UTC m=+0.743650100 container remove 35a692606e822c829b6d2c25081ab972f8c15656794bc9cd38a42d34a238d37a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-17T07:36:20.223 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.b.service' 2024-04-17T07:36:20.371 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:36:20.371 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-17T07:36:20.371 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-17T07:36:20.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.c 2024-04-17T07:36:20.776 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:20 smithi103 systemd[1]: Stopping Ceph mon.c for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:36:21.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-c[35491]: 2024-04-17T07:36:21.013+0000 7f1a45400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:36:21.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-c[35491]: 2024-04-17T07:36:21.013+0000 7f1a45400700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-17T07:36:21.277 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:21 smithi103 podman[73813]: 2024-04-17 07:36:21.087140484 +0000 UTC m=+0.239646282 container died 819a7f1e426ca449ea42647d3e7dbaad8bc4a0ebe724a949ceb31f6398b88bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-c, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-17T07:36:21.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:21 smithi103 podman[73813]: 2024-04-17 07:36:21.528620038 +0000 UTC m=+0.681125843 container cleanup 819a7f1e426ca449ea42647d3e7dbaad8bc4a0ebe724a949ceb31f6398b88bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-c, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-17T07:36:21.777 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:21 smithi103 bash[73813]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-c 2024-04-17T07:36:22.267 INFO:journalctl@ceph.mon.c.smithi103.stdout:Apr 17 07:36:21 smithi103 podman[73827]: 2024-04-17 07:36:21.808052126 +0000 UTC m=+0.704994157 container remove 819a7f1e426ca449ea42647d3e7dbaad8bc4a0ebe724a949ceb31f6398b88bb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mon-c, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-17T07:36:22.306 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mon.c.service' 2024-04-17T07:36:22.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:36:22.487 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-17T07:36:22.487 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-17T07:36:22.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.x 2024-04-17T07:36:22.831 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:36:22 smithi086 systemd[1]: Stopping Ceph mgr.x for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:36:23.331 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:36:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:36:23.062+0000 7ffaecc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:36:23.331 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:36:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x[31970]: 2024-04-17T07:36:23.062+0000 7ffaecc00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-17T07:36:23.331 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:36:23 smithi086 podman[114576]: 2024-04-17 07:36:23.156645634 +0000 UTC m=+0.237107170 container died e4f189e0cff0f8b221bf89b8875d6315a7d6c33a207f0363e747a41f80f1034c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:36:23.830 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:36:23 smithi086 podman[114576]: 2024-04-17 07:36:23.623689783 +0000 UTC m=+0.704151312 container cleanup e4f189e0cff0f8b221bf89b8875d6315a7d6c33a207f0363e747a41f80f1034c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-17T07:36:23.830 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:36:23 smithi086 bash[114576]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x 2024-04-17T07:36:24.330 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 17 07:36:23 smithi086 podman[114590]: 2024-04-17 07:36:23.875570303 +0000 UTC m=+0.697693928 container remove e4f189e0cff0f8b221bf89b8875d6315a7d6c33a207f0363e747a41f80f1034c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-17T07:36:24.452 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.x.service' 2024-04-17T07:36:24.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:36:24.562 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-17T07:36:24.562 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-17T07:36:24.562 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.y 2024-04-17T07:36:25.003 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:36:24 smithi098 systemd[1]: Stopping Ceph mgr.y for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:36:25.503 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:36:25 smithi098 podman[72457]: 2024-04-17 07:36:25.241406854 +0000 UTC m=+0.220518143 container died 7f5ef95ce723a59537bf9beaf7b81e4e744596ab08a7b4ab3db9bbbec0d30c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-17T07:36:25.983 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:36:25 smithi098 podman[72457]: 2024-04-17 07:36:25.779343741 +0000 UTC m=+0.758455004 container cleanup 7f5ef95ce723a59537bf9beaf7b81e4e744596ab08a7b4ab3db9bbbec0d30c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-17T07:36:25.983 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:36:25 smithi098 bash[72457]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y 2024-04-17T07:36:26.253 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:36:25 smithi098 podman[72471]: 2024-04-17 07:36:25.983171424 +0000 UTC m=+0.727793369 container remove 7f5ef95ce723a59537bf9beaf7b81e4e744596ab08a7b4ab3db9bbbec0d30c16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:36:26.253 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 17 07:36:25 smithi098 systemd[1]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-17T07:36:26.451 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.y.service' 2024-04-17T07:36:26.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:36:26.573 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-17T07:36:26.573 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-17T07:36:26.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.z 2024-04-17T07:36:27.026 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:36:26 smithi103 systemd[1]: Stopping Ceph mgr.z for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:36:27.526 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:36:27 smithi103 podman[73947]: 2024-04-17 07:36:27.275929712 +0000 UTC m=+0.278719505 container died afa69e5a544e93d35f242876214a13db4306cc95f994a247af597ef92e2ec8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-17T07:36:27.887 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:36:27 smithi103 podman[73947]: 2024-04-17 07:36:27.659994024 +0000 UTC m=+0.662783827 container cleanup afa69e5a544e93d35f242876214a13db4306cc95f994a247af597ef92e2ec8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-17T07:36:27.887 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:36:27 smithi103 bash[73947]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z 2024-04-17T07:36:28.277 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:36:27 smithi103 podman[73961]: 2024-04-17 07:36:27.886662212 +0000 UTC m=+0.594236843 container remove afa69e5a544e93d35f242876214a13db4306cc95f994a247af597ef92e2ec8fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:36:28.277 INFO:journalctl@ceph.mgr.z.smithi103.stdout:Apr 17 07:36:27 smithi103 systemd[1]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-17T07:36:28.415 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mgr.z.service' 2024-04-17T07:36:28.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:36:28.543 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-17T07:36:28.543 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-17T07:36:28.543 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.0 2024-04-17T07:36:29.080 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:28 smithi086 systemd[1]: Stopping Ceph osd.0 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:36:29.581 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0[44267]: 2024-04-17T07:36:29.109+0000 7f71d5600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:36:29.582 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0[44267]: 2024-04-17T07:36:29.109+0000 7f71d5600700 -1 osd.0 93 *** Got signal Terminated *** 2024-04-17T07:36:29.582 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0[44267]: 2024-04-17T07:36:29.109+0000 7f71d5600700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:36:35.259 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:34 smithi086 podman[114698]: 2024-04-17 07:36:34.920048662 +0000 UTC m=+5.949252827 container died d035413793074cbd136bb4b2f36a77e0ed9b4ca93c88f78f941628be1b8b7a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:36:35.580 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:35 smithi086 podman[114698]: 2024-04-17 07:36:35.387492411 +0000 UTC m=+6.416696569 container cleanup d035413793074cbd136bb4b2f36a77e0ed9b4ca93c88f78f941628be1b8b7a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:36:35.580 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:35 smithi086 bash[114698]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0 2024-04-17T07:36:36.080 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:35 smithi086 podman[114720]: 2024-04-17 07:36:35.613645378 +0000 UTC m=+0.676438966 container remove d035413793074cbd136bb4b2f36a77e0ed9b4ca93c88f78f941628be1b8b7a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-17T07:36:36.777 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:36 smithi086 podman[114804]: 2024-04-17 07:36:36.333265502 +0000 UTC m=+0.036560349 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:36:36.777 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:36 smithi086 podman[114804]: 2024-04-17 07:36:36.456444837 +0000 UTC m=+0.159739713 container create b46cb1543f945f1bab62ce88e22a71171e45e5d024c8a1876a73f6d1f41deac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:36:36.778 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:36 smithi086 podman[114804]: 2024-04-17 07:36:36.777087727 +0000 UTC m=+0.480382610 container init b46cb1543f945f1bab62ce88e22a71171e45e5d024c8a1876a73f6d1f41deac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:36:37.080 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:36 smithi086 podman[114804]: 2024-04-17 07:36:36.788645242 +0000 UTC m=+0.491940124 container start b46cb1543f945f1bab62ce88e22a71171e45e5d024c8a1876a73f6d1f41deac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:36:37.081 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:36 smithi086 podman[114804]: 2024-04-17 07:36:36.942006345 +0000 UTC m=+0.645301196 container attach b46cb1543f945f1bab62ce88e22a71171e45e5d024c8a1876a73f6d1f41deac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-17T07:36:37.950 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 17 07:36:37 smithi086 podman[114804]: 2024-04-17 07:36:37.598139332 +0000 UTC m=+1.301434195 container died b46cb1543f945f1bab62ce88e22a71171e45e5d024c8a1876a73f6d1f41deac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, ceph=True) 2024-04-17T07:36:38.373 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.0.service' 2024-04-17T07:36:38.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:36:38.495 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-17T07:36:38.496 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-17T07:36:38.496 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.1 2024-04-17T07:36:39.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:38 smithi098 systemd[1]: Stopping Ceph osd.1 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:36:39.503 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1[40012]: 2024-04-17T07:36:39.109+0000 7f89bc000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:36:39.503 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1[40012]: 2024-04-17T07:36:39.109+0000 7f89bc000700 -1 osd.1 93 *** Got signal Terminated *** 2024-04-17T07:36:39.504 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1[40012]: 2024-04-17T07:36:39.109+0000 7f89bc000700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:36:45.253 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:44 smithi098 podman[72580]: 2024-04-17 07:36:44.964659247 +0000 UTC m=+6.025024211 container died 7397415dd010349ddb599fe58f43b9ba2387f77f51b13e1a5033f194b7d17507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:36:45.664 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:45 smithi098 podman[72580]: 2024-04-17 07:36:45.420363273 +0000 UTC m=+6.480728274 container cleanup 7397415dd010349ddb599fe58f43b9ba2387f77f51b13e1a5033f194b7d17507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:36:45.664 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:45 smithi098 bash[72580]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1 2024-04-17T07:36:46.003 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:45 smithi098 podman[72597]: 2024-04-17 07:36:45.663893377 +0000 UTC m=+0.684356707 container remove 7397415dd010349ddb599fe58f43b9ba2387f77f51b13e1a5033f194b7d17507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:36:46.754 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:46 smithi098 podman[72688]: 2024-04-17 07:36:46.347784198 +0000 UTC m=+0.033878372 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:36:46.754 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:46 smithi098 podman[72688]: 2024-04-17 07:36:46.465071381 +0000 UTC m=+0.151165592 container create 9423187b03aa2b2aaa3757072f90c01c100e598bbdd63131b47d9b330e99cfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:36:47.254 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:46 smithi098 podman[72688]: 2024-04-17 07:36:46.777415339 +0000 UTC m=+0.463509560 container init 9423187b03aa2b2aaa3757072f90c01c100e598bbdd63131b47d9b330e99cfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-17T07:36:47.254 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:46 smithi098 podman[72688]: 2024-04-17 07:36:46.788342777 +0000 UTC m=+0.474436991 container start 9423187b03aa2b2aaa3757072f90c01c100e598bbdd63131b47d9b330e99cfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-17T07:36:47.254 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:46 smithi098 podman[72688]: 2024-04-17 07:36:46.884658058 +0000 UTC m=+0.570752306 container attach 9423187b03aa2b2aaa3757072f90c01c100e598bbdd63131b47d9b330e99cfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:36:47.753 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 17 07:36:47 smithi098 podman[72688]: 2024-04-17 07:36:47.417374339 +0000 UTC m=+1.103468544 container died 9423187b03aa2b2aaa3757072f90c01c100e598bbdd63131b47d9b330e99cfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-17T07:36:48.180 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.1.service' 2024-04-17T07:36:48.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:36:48.307 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-17T07:36:48.307 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-17T07:36:48.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.2 2024-04-17T07:36:48.777 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:48 smithi103 systemd[1]: Stopping Ceph osd.2 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:36:49.277 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2[40737]: 2024-04-17T07:36:48.897+0000 7f4771e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:36:49.277 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2[40737]: 2024-04-17T07:36:48.897+0000 7f4771e00700 -1 osd.2 93 *** Got signal Terminated *** 2024-04-17T07:36:49.277 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2[40737]: 2024-04-17T07:36:48.897+0000 7f4771e00700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:36:55.152 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:54 smithi103 podman[74069]: 2024-04-17 07:36:54.820078987 +0000 UTC m=+6.059586290 container died b000872e2c248de6900db56227f49751046fba9698d2c658d2651ae180ee8b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-17T07:36:55.460 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:55 smithi103 podman[74069]: 2024-04-17 07:36:55.283227047 +0000 UTC m=+6.522734347 container cleanup b000872e2c248de6900db56227f49751046fba9698d2c658d2651ae180ee8b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-17T07:36:55.460 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:55 smithi103 bash[74069]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2 2024-04-17T07:36:55.776 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:55 smithi103 podman[74087]: 2024-04-17 07:36:55.459785068 +0000 UTC m=+0.629257801 container remove b000872e2c248de6900db56227f49751046fba9698d2c658d2651ae180ee8b09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-17T07:36:56.527 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:56 smithi103 podman[74172]: 2024-04-17 07:36:56.134746465 +0000 UTC m=+0.036195482 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:36:56.527 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:56 smithi103 podman[74172]: 2024-04-17 07:36:56.25261786 +0000 UTC m=+0.154066890 container create f2dd4ad7840475db324854036c5af3cf58fcdcfd0adda95c6a04baeebb023024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS) 2024-04-17T07:36:57.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:36:56 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:36:56.765+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:36:57.028 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:36:56 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:36:56.722+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:36:57.028 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:56 smithi103 podman[74172]: 2024-04-17 07:36:56.566477348 +0000 UTC m=+0.467926442 container init f2dd4ad7840475db324854036c5af3cf58fcdcfd0adda95c6a04baeebb023024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-04-17T07:36:57.028 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:56 smithi103 podman[74172]: 2024-04-17 07:36:56.579669851 +0000 UTC m=+0.481118899 container start f2dd4ad7840475db324854036c5af3cf58fcdcfd0adda95c6a04baeebb023024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-17T07:36:57.028 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:56 smithi103 podman[74172]: 2024-04-17 07:36:56.655673277 +0000 UTC m=+0.557122387 container attach f2dd4ad7840475db324854036c5af3cf58fcdcfd0adda95c6a04baeebb023024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-17T07:36:57.082 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:36:56 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:36:56.613+0000 7f3c96800700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.427115+0000 front 2024-04-17T07:36:29.427025+0000 (oldest deadline 2024-04-17T07:36:55.728856+0000) 2024-04-17T07:36:57.527 INFO:journalctl@ceph.osd.2.smithi103.stdout:Apr 17 07:36:57 smithi103 podman[74172]: 2024-04-17 07:36:57.181438539 +0000 UTC m=+1.082887606 container died f2dd4ad7840475db324854036c5af3cf58fcdcfd0adda95c6a04baeebb023024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:36:57.815 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:36:57 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:36:57.740+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:36:57.950 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.2.service' 2024-04-17T07:36:58.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:36:58.059 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-17T07:36:58.059 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-17T07:36:58.060 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.3 2024-04-17T07:36:58.080 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:36:57 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:36:57.658+0000 7f3c96800700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.427115+0000 front 2024-04-17T07:36:29.427025+0000 (oldest deadline 2024-04-17T07:36:55.728856+0000) 2024-04-17T07:36:58.276 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:36:57 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:36:57.815+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:36:58.459 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:36:58 smithi086 systemd[1]: Stopping Ceph osd.3 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:36:58.459 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:36:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:36:58.238+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:36:58.773 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:36:58 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:36:58.282+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:36:58.775 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:36:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:36:58.624+0000 7f3c96800700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.427115+0000 front 2024-04-17T07:36:29.427025+0000 (oldest deadline 2024-04-17T07:36:55.728856+0000) 2024-04-17T07:36:58.776 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:36:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:36:58.690+0000 7f3ca0000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:36:58.776 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:36:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:36:58.690+0000 7f3ca0000700 -1 osd.3 93 *** Got signal Terminated *** 2024-04-17T07:36:58.776 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:36:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:36:58.690+0000 7f3ca0000700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:36:59.026 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:36:58 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:36:58.782+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:36:59.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:36:58 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:36:58.772+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:36:59.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:36:59 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:36:59.247+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:36:59.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:36:59 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:36:59.200+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:36:59.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:36:59 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:36:59.359+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:00.080 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:36:59 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:36:59.580+0000 7f3c96800700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.427115+0000 front 2024-04-17T07:36:29.427025+0000 (oldest deadline 2024-04-17T07:36:55.728856+0000) 2024-04-17T07:37:00.232 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:36:59 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:36:59.798+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:00.232 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:36:59 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:36:59.798+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:00.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:00.232+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:00.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:00 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:00.312+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:00.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:00 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:00.152+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:00.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:00 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:00.407+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:01.080 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:00 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:37:00.581+0000 7f3c96800700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.427115+0000 front 2024-04-17T07:36:29.427025+0000 (oldest deadline 2024-04-17T07:36:55.728856+0000) 2024-04-17T07:37:01.245 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:00.799+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:01.245 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:00.783+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:01.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:01.245+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:01.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:01.310+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:01.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:01.161+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:01.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:01 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:01.374+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:02.080 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:37:01.629+0000 7f3c96800700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.427115+0000 front 2024-04-17T07:36:29.427025+0000 (oldest deadline 2024-04-17T07:36:55.728856+0000) 2024-04-17T07:37:02.276 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:01.789+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:02.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:01.823+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:02.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:02 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:02.313+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:02.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:02 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:02.196+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:02.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:02 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:02.337+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:02.754 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:02.294+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:03.026 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:02.754+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:03.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:02.840+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:03.080 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:02 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3[50816]: 2024-04-17T07:37:02.679+0000 7f3c96800700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.427115+0000 front 2024-04-17T07:36:29.427025+0000 (oldest deadline 2024-04-17T07:36:55.728856+0000) 2024-04-17T07:37:03.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:03 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:03.324+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:03.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:03 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:03.220+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:03.655 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:03.359+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:03.776 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:03.293+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:04.276 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:03.799+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:04.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:03.817+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:04.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:04 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:04.335+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:04.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:04 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:04.186+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:04.686 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:04 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:04.330+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:04.776 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:04.284+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:04.929 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:04 smithi086 podman[114909]: 2024-04-17 07:37:04.618604147 +0000 UTC m=+6.099646423 container died 58df7478181b7338660f668c83acde5c6940ccf6b4b121039307dd06c3771c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:37:05.230 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:05 smithi086 podman[114909]: 2024-04-17 07:37:05.100306599 +0000 UTC m=+6.581348910 container cleanup 58df7478181b7338660f668c83acde5c6940ccf6b4b121039307dd06c3771c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-17T07:37:05.230 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:05 smithi086 bash[114909]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3 2024-04-17T07:37:05.256 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:04.840+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:05.256 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:04.781+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:05.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:05.256+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:05.581 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:05 smithi086 podman[114937]: 2024-04-17 07:37:05.404802509 +0000 UTC m=+0.773801907 container remove 58df7478181b7338660f668c83acde5c6940ccf6b4b121039307dd06c3771c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-17T07:37:05.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:05 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:05.322+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:05.582 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:05 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:05.229+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:05.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:05.300+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:05.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:05.300+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:05.754 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:05.455+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:06.246 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:05.806+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:06.247 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:05.825+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:06.248 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:05 smithi086 podman[115029]: 2024-04-17 07:37:05.898625081 +0000 UTC m=+0.033524642 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:37:06.249 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:06 smithi086 podman[115029]: 2024-04-17 07:37:06.022544398 +0000 UTC m=+0.157443991 container create 87b722b8576ccb777232c12d78ea2f934994a0b92d13ebbb1a4e4761f3d9b235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:37:06.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:06.492+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:06.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:06.262+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:06.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:06.262+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:06.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:06.245+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:06.575 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:06 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:06.352+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:06.575 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:06 smithi086 podman[115029]: 2024-04-17 07:37:06.484915971 +0000 UTC m=+0.619815571 container init 87b722b8576ccb777232c12d78ea2f934994a0b92d13ebbb1a4e4761f3d9b235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:37:06.576 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:06 smithi086 podman[115029]: 2024-04-17 07:37:06.498122442 +0000 UTC m=+0.633022042 container start 87b722b8576ccb777232c12d78ea2f934994a0b92d13ebbb1a4e4761f3d9b235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:37:06.576 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:06 smithi086 podman[115029]: 2024-04-17 07:37:06.575388096 +0000 UTC m=+0.710287733 container attach 87b722b8576ccb777232c12d78ea2f934994a0b92d13ebbb1a4e4761f3d9b235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-17T07:37:06.576 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:06 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:06.254+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:06.576 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:06 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:06.254+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:07.005 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4[46846]: 2024-04-17T07:37:06.646+0000 7f7866e00700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.372981+0000 front 2024-04-17T07:36:42.373098+0000 (oldest deadline 2024-04-17T07:37:05.873300+0000) 2024-04-17T07:37:07.257 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 17 07:37:07 smithi086 podman[115055]: 2024-04-17 07:37:07.011762111 +0000 UTC m=+0.040417403 container died 87b722b8576ccb777232c12d78ea2f934994a0b92d13ebbb1a4e4761f3d9b235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-17T07:37:07.265 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:06.823+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:07.266 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:06.823+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:07.266 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:06.832+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:07.266 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:06.833+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:07.519 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:07.252+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:07.519 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:07.252+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:07.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:07.264+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:07.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:07 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:07.326+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:07.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:07 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:07.326+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:07.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:07 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:07.256+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:07.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:07 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:07.256+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:07.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.3.service' 2024-04-17T07:37:07.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:37:07.955 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-17T07:37:07.955 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-17T07:37:07.955 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.4 2024-04-17T07:37:08.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:07.519+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:08.004 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4[46846]: 2024-04-17T07:37:07.600+0000 7f7866e00700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.372981+0000 front 2024-04-17T07:36:42.373098+0000 (oldest deadline 2024-04-17T07:37:05.873300+0000) 2024-04-17T07:37:08.254 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:07.810+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:08.254 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:07.810+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:08.254 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:07.870+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:08.254 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:07.870+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:08.497 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:08 smithi098 systemd[1]: Stopping Ceph osd.4 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:37:08.497 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:08.278+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:08.497 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:08.278+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:08.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:08.292+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:08.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:08.293+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:08.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:08 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:08.326+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:08.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:08 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:08.326+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:08.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:08 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:08.275+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:08.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:08 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:08.275+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:08.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:08.495+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:08.753 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4[46846]: 2024-04-17T07:37:08.582+0000 7f786b600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:37:08.753 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4[46846]: 2024-04-17T07:37:08.582+0000 7f786b600700 -1 osd.4 93 *** Got signal Terminated *** 2024-04-17T07:37:08.754 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4[46846]: 2024-04-17T07:37:08.582+0000 7f786b600700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:37:08.754 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4[46846]: 2024-04-17T07:37:08.648+0000 7f7866e00700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.372981+0000 front 2024-04-17T07:36:42.373098+0000 (oldest deadline 2024-04-17T07:37:05.873300+0000) 2024-04-17T07:37:09.273 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:08.827+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:09.273 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:08.827+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:09.273 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:08.835+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:09.273 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:08.835+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:09.526 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:09.272+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:09.526 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:09.272+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:09.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:09.272+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:09.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:09.272+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:09.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:09 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:09.273+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:09.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:09 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:09.273+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:09.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:09 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:09.370+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:09.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:09 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:09.370+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:10.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:09.524+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:10.003 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4[46846]: 2024-04-17T07:37:09.645+0000 7f7866e00700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.372981+0000 front 2024-04-17T07:36:42.373098+0000 (oldest deadline 2024-04-17T07:37:05.873300+0000) 2024-04-17T07:37:10.263 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:09.853+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:10.263 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:09.853+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:10.264 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:09.841+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:10.264 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:09.841+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:10.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:10.262+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:10.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:10.262+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:10.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:10 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:10.398+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:10.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:10 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:10.398+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:10.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:10 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:10.312+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:10.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:10 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:10.312+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:10.664 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:10.511+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:10.665 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:10.285+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:10.665 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:10.285+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:11.003 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4[46846]: 2024-04-17T07:37:10.663+0000 7f7866e00700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.372981+0000 front 2024-04-17T07:36:42.373098+0000 (oldest deadline 2024-04-17T07:37:05.873300+0000) 2024-04-17T07:37:11.224 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:10.852+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:11.224 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:10.852+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:11.224 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:10.883+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:11.224 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:10.883+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:11.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:11.223+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:11.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:11.223+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:11.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:11 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:11.387+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:11.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:11 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:11.387+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:11.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:11 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:11.302+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:11.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:11 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:11.302+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:11.712 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:11.483+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:11.712 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:11.314+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:11.712 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:11.314+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:12.003 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4[46846]: 2024-04-17T07:37:11.710+0000 7f7866e00700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.372981+0000 front 2024-04-17T07:36:42.373098+0000 (oldest deadline 2024-04-17T07:37:05.873300+0000) 2024-04-17T07:37:12.196 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:11.845+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:12.196 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:11.845+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:12.196 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:11.910+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:12.196 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:11.910+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:12.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:12.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:12.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:12.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:12.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:12 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:12.362+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:12.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:12 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:12.362+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:12.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:12 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:12.322+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:12.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:12 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:12.322+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:12.717 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:12 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:12.532+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:12.718 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:12 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:12.310+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:12.718 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:12 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:12.310+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:13.003 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:12 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4[46846]: 2024-04-17T07:37:12.716+0000 7f7866e00700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.372981+0000 front 2024-04-17T07:36:42.373098+0000 (oldest deadline 2024-04-17T07:37:05.873300+0000) 2024-04-17T07:37:13.173 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:12.819+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:13.173 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:12.819+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:13.174 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:12.871+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:13.174 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:12.871+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:13.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:13.172+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:13.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:13.172+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:13.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:13 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:13.489+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:13.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:13 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:13.277+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:13.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:13 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:13.277+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:13.830 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:13 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:13.402+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:13.830 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:13 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:13.402+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:13.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:13 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:13.348+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:13.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:13 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:13.348+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:14.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:13.774+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:14.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:13.774+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:14.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:13.920+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:14.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:13.920+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:14.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:14.175+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:14.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:14.175+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:14.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:14 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:14.419+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:14.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:14 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:14.419+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:14.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:14 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:14.315+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:14.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:14 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:14.315+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:14.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:14 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:14.462+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:14.754 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:14 smithi098 podman[72805]: 2024-04-17 07:37:14.44080473 +0000 UTC m=+6.021314022 container died 8c1ea0f9bfca6673e6a32e78dca95391e5b9df420c02b4009e4e4014d8771f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-17T07:37:14.754 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:14 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:14.279+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:14.754 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:14 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:14.279+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:15.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:14.738+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:15.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:14.738+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:15.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:14.966+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:15.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:14.966+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:15.162 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:14 smithi098 podman[72805]: 2024-04-17 07:37:14.94362756 +0000 UTC m=+6.524136845 container cleanup 8c1ea0f9bfca6673e6a32e78dca95391e5b9df420c02b4009e4e4014d8771f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-17T07:37:15.162 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:14 smithi098 bash[72805]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4 2024-04-17T07:37:15.501 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:15 smithi098 podman[72824]: 2024-04-17 07:37:15.16193573 +0000 UTC m=+0.709734959 container remove 8c1ea0f9bfca6673e6a32e78dca95391e5b9df420c02b4009e4e4014d8771f84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-17T07:37:15.501 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:15 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:15.237+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:15.501 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:15 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:15.237+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:15.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:15.161+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:15.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:15.161+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:15.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:15 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:15.413+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:15.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:15 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:15.413+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:15.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:15 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:15.325+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:15.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:15 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:15.325+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:15.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:15 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:15.499+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:16.026 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:15.699+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:16.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:15.699+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:16.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:15.916+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:16.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:15.916+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:16.027 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:15.916+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:16.253 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:15 smithi098 podman[72910]: 2024-04-17 07:37:15.871405645 +0000 UTC m=+0.032416792 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:37:16.254 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:15 smithi098 podman[72910]: 2024-04-17 07:37:15.987901362 +0000 UTC m=+0.148912516 container create 0ac0397c4c2c856637b865b36c9856af5781c114c8acc8883c17cef0409db235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-17T07:37:16.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:16.144+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:16.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:16.144+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:16.534 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:16 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:16.280+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:16.534 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:16 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:16.280+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:16.534 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:16 smithi098 podman[72910]: 2024-04-17 07:37:16.326661716 +0000 UTC m=+0.487672947 container init 0ac0397c4c2c856637b865b36c9856af5781c114c8acc8883c17cef0409db235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-17T07:37:16.534 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:16 smithi098 podman[72910]: 2024-04-17 07:37:16.338226001 +0000 UTC m=+0.499237160 container start 0ac0397c4c2c856637b865b36c9856af5781c114c8acc8883c17cef0409db235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-17T07:37:16.535 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:16 smithi098 podman[72910]: 2024-04-17 07:37:16.457878592 +0000 UTC m=+0.618889789 container attach 0ac0397c4c2c856637b865b36c9856af5781c114c8acc8883c17cef0409db235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-17T07:37:16.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:16 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:16.285+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:16.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:16 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:16.285+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:16.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:16 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:16.381+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:16.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:16 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:16.381+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:16.954 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:16.690+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:16.955 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:16.690+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:16.955 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:16.690+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.957261+0000 front 2024-04-17T07:36:51.957341+0000 (oldest deadline 2024-04-17T07:37:16.657494+0000) 2024-04-17T07:37:17.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:16 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:16.532+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:17.003 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 17 07:37:16 smithi098 podman[72910]: 2024-04-17 07:37:16.770291927 +0000 UTC m=+0.931303104 container died 0ac0397c4c2c856637b865b36c9856af5781c114c8acc8883c17cef0409db235 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-17T07:37:17.277 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:17.142+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:17.277 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:17.142+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:17.277 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:17.142+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:17.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:16.953+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:17.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:16.953+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:17.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:16.953+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:17.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:17 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:17.320+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:17.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:17 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:17.320+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:17.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:17 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:17.320+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:17.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:17 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:17.408+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:17.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:17 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:17.408+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:17.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:17 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:17.264+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:17.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:17 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:17.264+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:17.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:17 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:17.264+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:17.594 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.4.service' 2024-04-17T07:37:17.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:37:17.761 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-17T07:37:17.761 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-17T07:37:17.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.5 2024-04-17T07:37:17.999 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:17.711+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:17.999 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:17.711+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:17.999 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:17.711+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.957261+0000 front 2024-04-17T07:36:51.957341+0000 (oldest deadline 2024-04-17T07:37:16.657494+0000) 2024-04-17T07:37:17.999 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:17 smithi103 systemd[1]: Stopping Ceph osd.5 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:37:18.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:17 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:17.539+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:18.277 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:18.161+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:18.277 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:18.161+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:18.277 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:18.161+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:18.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:17.997+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:18.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:17.997+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:18.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:17.997+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:18.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:18 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:18.361+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:18.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:18 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:18.361+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:18.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:18 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:18.285+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:18.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:18 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:18.285+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:18.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:18 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:18.285+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:18.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:18 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:18.535+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:18.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:18 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:18.318+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:18.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:18 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:18.318+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:18.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:18 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:18.318+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:18.757 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:18.372+0000 7f5b3ec00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:37:18.757 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:18.372+0000 7f5b3ec00700 -1 osd.5 93 *** Got signal Terminated *** 2024-04-17T07:37:18.757 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:18.372+0000 7f5b3ec00700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:37:19.026 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:18.755+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:19.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:18.755+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:19.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:18.755+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.957261+0000 front 2024-04-17T07:36:51.957341+0000 (oldest deadline 2024-04-17T07:37:16.657494+0000) 2024-04-17T07:37:19.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:19.037+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:19.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:19.037+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:19.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:19.037+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:19.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:19.200+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:19.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:19.200+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:19.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:19.200+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:19.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:19 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:19.386+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:19.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:19 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:19.386+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:19.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:19 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:19.386+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:19.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:19 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:19.241+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:19.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:19 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:19.241+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:19.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:19 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:19.241+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:19.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:19 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:19.559+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:19.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:19 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:19.559+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:19.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:19 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:19.341+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:19.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:19 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:19.341+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:19.754 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:19 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:19.341+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:20.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:19.741+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:20.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:19.741+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:20.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:19.741+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.957261+0000 front 2024-04-17T07:36:51.957341+0000 (oldest deadline 2024-04-17T07:37:16.657494+0000) 2024-04-17T07:37:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:20.079+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:20.079+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:20.079+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:20.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:20.244+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:20.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:20.244+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:20.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:20.244+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:20.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:20.389+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:20.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:20.389+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:20.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:20.389+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:20.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:20.200+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:20.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:20.200+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:20.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:20 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:20.200+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:20.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:20 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:20.547+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:20.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:20 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:20.547+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:20.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:20 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:20.313+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:20.754 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:20 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:20.313+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:20.754 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:20 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:20.313+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:21.113 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:20.777+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:21.113 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:20.777+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:21.113 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:20.777+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.957261+0000 front 2024-04-17T07:36:51.957341+0000 (oldest deadline 2024-04-17T07:37:16.657494+0000) 2024-04-17T07:37:21.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:21.111+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:21.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:21.111+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:21.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:21.111+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:21.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:21.262+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:21.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:21.262+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:21.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:21.262+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:21.566 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:21 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:21.277+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:21.567 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:21 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:21.277+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:21.567 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:21 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:21.277+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:21.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:21 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:21.346+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:21.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:21 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:21.346+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:21.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:21 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:21.346+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:21.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:21 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:21.245+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:21.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:21 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:21.245+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:21.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:21 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:21.245+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:22.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:21 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:21.565+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:22.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:21 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:21.565+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:22.077 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:21.813+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:22.077 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:21.813+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:22.077 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:21.813+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.957261+0000 front 2024-04-17T07:36:51.957341+0000 (oldest deadline 2024-04-17T07:37:16.657494+0000) 2024-04-17T07:37:22.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:22 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:22.309+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:22.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:22 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:22.309+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:22.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:22 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:22.309+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:22.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:22 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:22.076+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:22.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:22 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:22.076+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:22.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:22 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:22.076+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:22.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:22 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:22.337+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:22.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:22 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:22.337+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:22.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:22 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:22.337+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:22.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:22 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:22.237+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:22.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:22 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:22.237+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:22.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:22 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:22.237+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:22.590 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:22 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:22.260+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:22.590 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:22 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:22.260+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:22.590 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:22 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:22.260+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:23.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:22 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:22.589+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:23.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:22 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:22.589+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:23.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:22 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:22.770+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:32.146482+0000 front 2024-04-17T07:36:32.146146+0000 (oldest deadline 2024-04-17T07:36:56.246376+0000) 2024-04-17T07:37:23.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:22 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:22.770+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.252074+0000 front 2024-04-17T07:36:43.252176+0000 (oldest deadline 2024-04-17T07:37:06.752298+0000) 2024-04-17T07:37:23.027 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:22 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5[47340]: 2024-04-17T07:37:22.770+0000 7f5b35400700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.957261+0000 front 2024-04-17T07:36:51.957341+0000 (oldest deadline 2024-04-17T07:37:16.657494+0000) 2024-04-17T07:37:23.340 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:23 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:23.086+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:23.340 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:23 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:23.086+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:23.340 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:23 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:23.086+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:23.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:23 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:23.271+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:23.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:23 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:23.271+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:23.549 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:23 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:23.271+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:23.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:23.197+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:23.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:23.197+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:23.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:23.197+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:23.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:23.343+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:23.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:23.343+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:23.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:23 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:23.343+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:23.776 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:23 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:23.339+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:23.777 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:23 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:23.339+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:23.777 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:23 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:23.339+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:24.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:23 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:23.547+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:24.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:23 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:23.548+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:24.502 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:24 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:24.243+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:24.502 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:24 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:24.243+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:24.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:24 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:24.243+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:24.523 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:24 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:24.293+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:24.523 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:24 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:24.293+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:24.523 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:24 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:24.293+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:24.524 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:24 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:24.106+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:24.524 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:24 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:24.106+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:24.524 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:24 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:24.106+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:24.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:24.304+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:24.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:24.304+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:24.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:24.304+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:24.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:24.164+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:24.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:24.164+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:24.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:24 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:24.164+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:24.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:24 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:24.502+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:24.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:24 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:24.502+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:24.776 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:24 smithi103 podman[74288]: 2024-04-17 07:37:24.523294065 +0000 UTC m=+6.307240921 container died 5d7ac68c443099726a96181ca843e23da347ed6dcc18617be559d1d2e5509dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:37:25.487 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:25 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:25.233+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:25.487 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:25 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:25.233+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:25.487 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:25 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:25.233+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:25.487 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:25 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:25.233+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:25.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:25 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:25.132+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:25.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:25 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:25.132+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:25.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:25 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:25.132+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:25.528 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:25 smithi103 podman[74288]: 2024-04-17 07:37:25.208179213 +0000 UTC m=+6.992126117 container cleanup 5d7ac68c443099726a96181ca843e23da347ed6dcc18617be559d1d2e5509dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-17T07:37:25.528 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:25 smithi103 bash[74288]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5 2024-04-17T07:37:25.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:25 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:25.265+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:25.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:25 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:25.265+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:25.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:25 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:25.265+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:25.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:25 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:25.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:25.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:25 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:25.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:25.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:25 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:25.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:25.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:25 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:25.340+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:25.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:25 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:25.340+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:25.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:25 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:25.340+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:25.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:25 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:25.486+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:25.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:25 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:25.486+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:25.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:25 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:25.486+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:26.026 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:25 smithi103 podman[74309]: 2024-04-17 07:37:25.594978031 +0000 UTC m=+1.053659660 container remove 5d7ac68c443099726a96181ca843e23da347ed6dcc18617be559d1d2e5509dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-17T07:37:26.369 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:26 smithi103 podman[74396]: 2024-04-17 07:37:26.055394272 +0000 UTC m=+0.033022100 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:37:26.370 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:26 smithi103 podman[74396]: 2024-04-17 07:37:26.186005778 +0000 UTC m=+0.163633640 container create ec02d380b70ac36547a15efe0bddf667ae009c1e640b46a7374f940d935b6dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-17T07:37:26.370 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:26 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:26.308+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:26.370 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:26 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:26.308+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:26.370 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:26 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:26.308+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:26.370 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:26 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:26.308+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:26.370 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:26 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:26.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:26.370 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:26 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:26.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:26.371 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:26 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:26.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:26.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:26 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:26.213+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:26.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:26 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:26.213+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:26.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:26 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:26.213+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:26.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:26 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:26.213+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:26.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:26 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:26.189+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:26.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:26 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:26.189+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:26.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:26 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:26.189+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:26.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:26 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:26.312+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:26.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:26 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:26.312+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:26.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:26 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:26.312+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:26.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:26 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:26.312+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:01.973087+0000 front 2024-04-17T07:37:01.973046+0000 (oldest deadline 2024-04-17T07:37:25.374291+0000) 2024-04-17T07:37:26.622 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:26 smithi103 podman[74396]: 2024-04-17 07:37:26.473442257 +0000 UTC m=+0.451070129 container init ec02d380b70ac36547a15efe0bddf667ae009c1e640b46a7374f940d935b6dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2) 2024-04-17T07:37:26.622 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:26 smithi103 podman[74396]: 2024-04-17 07:37:26.484769635 +0000 UTC m=+0.462397474 container start ec02d380b70ac36547a15efe0bddf667ae009c1e640b46a7374f940d935b6dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-17T07:37:26.954 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:26 smithi103 podman[74396]: 2024-04-17 07:37:26.622634138 +0000 UTC m=+0.600261969 container attach ec02d380b70ac36547a15efe0bddf667ae009c1e640b46a7374f940d935b6dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-17T07:37:27.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:26 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:26.516+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:27.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:26 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:26.516+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:27.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:26 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:26.516+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:27.274 INFO:journalctl@ceph.osd.5.smithi103.stdout:Apr 17 07:37:27 smithi103 podman[74427]: 2024-04-17 07:37:27.004197211 +0000 UTC m=+0.030679008 container died ec02d380b70ac36547a15efe0bddf667ae009c1e640b46a7374f940d935b6dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=) 2024-04-17T07:37:27.274 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:27 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:27.165+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:27.274 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:27 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:27.165+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:27.275 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:27 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:27.165+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:27.482 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:27 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:27.321+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:27.483 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:27 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:27.321+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:27.483 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:27 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:27.321+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:27.483 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:27 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:27.321+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:01.973087+0000 front 2024-04-17T07:37:01.973046+0000 (oldest deadline 2024-04-17T07:37:25.374291+0000) 2024-04-17T07:37:27.483 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:27 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:27.210+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:27.483 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:27 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:27.210+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:27.483 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:27 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:27.210+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:27.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:27 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:27.469+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:27.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:27 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:27.469+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:27.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:27 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:27.469+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:27.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:27 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:27.222+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:27.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:27 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:27.222+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:27.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:27 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:27.222+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:27.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:27 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:27.222+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:27.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:27 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:27.273+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:27.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:27 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:27.273+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:27.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:27 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:27.273+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:27.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:27 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:27.273+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:27.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.5.service' 2024-04-17T07:37:27.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:37:27.866 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-17T07:37:27.866 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-17T07:37:27.866 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.6 2024-04-17T07:37:28.176 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:27 smithi086 systemd[1]: Stopping Ceph osd.6 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:37:28.463 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:28 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:28.318+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:28.463 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:28 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:28.318+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:28.463 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:28 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:28.318+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:28.463 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:28 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:28.318+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:01.973087+0000 front 2024-04-17T07:37:01.973046+0000 (oldest deadline 2024-04-17T07:37:25.374291+0000) 2024-04-17T07:37:28.463 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:28 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:28.175+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:28.463 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:28 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:28.175+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:28.463 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:28 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:28.175+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:28.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:28 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:28.184+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:28.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:28 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:28.184+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:28.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:28 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:28.184+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:28.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:28 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:28.184+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:28.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:28 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:28.144+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:28.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:28 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:28.144+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:28.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:28 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:28.144+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:28.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:28 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:28.252+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:28.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:28 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:28.252+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:28.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:28 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:28.252+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:28.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:28 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:28.252+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:28.830 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:28 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:28.462+0000 7f590a800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:37:28.831 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:28 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:28.462+0000 7f590a800700 -1 osd.6 93 *** Got signal Terminated *** 2024-04-17T07:37:28.831 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:28 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:28.462+0000 7f590a800700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:37:29.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:28 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:28.506+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:29.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:28 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:28.506+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:29.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:28 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:28.506+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:29.500 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:29 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:29.165+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:29.501 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:29 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:29.165+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:29.501 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:29 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:29.165+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:29.501 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:29 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:29.165+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:29.504 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:29.129+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:29.504 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:29.129+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:29.505 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:29.129+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:29.505 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:29.129+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:29.505 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:29.356+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:29.505 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:29.356+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:29.505 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:29.356+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:29.505 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:29 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:29.356+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:01.973087+0000 front 2024-04-17T07:37:01.973046+0000 (oldest deadline 2024-04-17T07:37:25.374291+0000) 2024-04-17T07:37:29.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:29 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:29.274+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:29.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:29 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:29.274+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:29.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:29 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:29.274+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:29.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:29 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:29.274+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:29.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:29 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:29.097+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:29.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:29 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:29.097+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:29.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:29 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:29.097+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:29.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:29 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:29.499+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:29.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:29 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:29.499+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:29.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:29 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:29.499+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:30.479 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:30 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:30.168+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:30.479 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:30 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:30.168+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:30.479 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:30 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:30.168+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:30.479 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:30 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:30.168+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:30.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:30 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:30.077+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:30.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:30 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:30.077+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:30.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:30 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:30.077+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:30.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:30 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:30.261+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:30.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:30 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:30.261+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:30.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:30 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:30.261+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:30.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:30 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:30.261+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:30.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:30.140+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:30.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:30.140+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:30.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:30.140+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:30.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:30.140+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:30.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:30.379+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:30.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:30.379+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:30.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:30.379+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:30.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:30.379+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:01.973087+0000 front 2024-04-17T07:37:01.973046+0000 (oldest deadline 2024-04-17T07:37:25.374291+0000) 2024-04-17T07:37:30.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:30 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:30.478+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:30.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:30 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:30.478+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:30.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:30 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:30.478+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:31.402 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:31.143+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:31.402 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:31.143+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:31.402 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:31.143+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:31.402 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:31.143+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:31.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:31 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:31.171+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:31.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:31 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:31.171+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:31.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:31 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:31.171+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:31.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:31 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:31.171+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:31.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:31 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:31.278+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:31.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:31 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:31.278+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:31.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:31 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:31.278+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:31.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:31 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:31.278+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:31.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:31 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:31.119+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:31.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:31 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:31.119+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:31.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:31 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:31.119+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:31.830 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:31.401+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:31.831 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:31.401+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:31.831 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:31.401+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:31.831 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:31 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:31.401+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:01.973087+0000 front 2024-04-17T07:37:01.973046+0000 (oldest deadline 2024-04-17T07:37:25.374291+0000) 2024-04-17T07:37:32.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:31 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:31.505+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:32.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:31 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:31.505+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:32.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:31 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:31.505+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:32.477 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:32 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:32.130+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:32.477 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:32 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:32.131+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:32.477 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:32 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:32.131+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:32.477 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:32 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:32.131+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:32.526 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:32 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:32.118+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:32.526 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:32 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:32.118+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:32.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:32 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:32.118+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:32.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:32 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:32.248+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:32.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:32 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:32.248+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:32.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:32 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:32.248+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:32.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:32 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:32.248+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:32.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:32.415+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:32.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:32.415+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:32.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:32.415+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:32.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:32.415+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:01.973087+0000 front 2024-04-17T07:37:01.973046+0000 (oldest deadline 2024-04-17T07:37:25.374291+0000) 2024-04-17T07:37:32.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:32.179+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:32.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:32.179+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:32.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:32.179+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:32.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:32 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:32.179+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:32.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:32 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:32.476+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:32.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:32 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:32.476+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:32.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:32 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:32.476+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:33.488 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:33 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:33.163+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:33.488 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:33 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:33.163+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:33.488 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:33 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:33.163+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:33.488 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:33 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:33.163+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:33.526 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:33 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:33.072+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:33.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:33 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:33.072+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:33.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:33 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:33.072+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:33.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:33 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:33.240+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:33.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:33 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:33.240+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:33.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:33 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:33.240+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:33.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:33 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:33.240+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:33.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:33 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:33.380+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.461095+0000 front 2024-04-17T07:36:29.461249+0000 (oldest deadline 2024-04-17T07:36:58.862705+0000) 2024-04-17T07:37:33.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:33 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:33.380+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:42.866331+0000 front 2024-04-17T07:36:42.865691+0000 (oldest deadline 2024-04-17T07:37:06.365989+0000) 2024-04-17T07:37:33.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:33 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:33.380+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.870379+0000 front 2024-04-17T07:36:53.870630+0000 (oldest deadline 2024-04-17T07:37:18.570163+0000) 2024-04-17T07:37:33.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:33 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6[57763]: 2024-04-17T07:37:33.380+0000 7f5901000700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:01.973087+0000 front 2024-04-17T07:37:01.973046+0000 (oldest deadline 2024-04-17T07:37:25.374291+0000) 2024-04-17T07:37:33.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:33 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:33.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:33.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:33 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:33.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:33.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:33 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:33.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:33.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:33 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:33.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:33.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:33 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:33.488+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:33.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:33 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:33.488+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:33.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:33 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:33.488+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:34.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:34 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:34.201+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:34.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:34 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:34.201+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:34.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:34 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:34.201+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:34.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:34 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:34.202+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:34.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:34 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:34.438+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:34.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:34 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:34.438+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:34.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:34 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:34.438+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:34.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:34 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:34.063+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:34.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:34 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:34.063+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:34.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:34 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:34.063+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:34.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:34 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:34.200+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:34.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:34 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:34.200+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:34.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:34 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:34.200+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:34.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:34 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:34.200+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:34.580 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:34 smithi086 podman[115137]: 2024-04-17 07:37:34.365072982 +0000 UTC m=+6.046443863 container died 1c06539ce62f98c4c618d59f4863a7a75bbc0d9b7d6ba031415dcdc64470d257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-17T07:37:34.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:34 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:34.202+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:34.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:34 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:34.202+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:34.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:34 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:34.202+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:34.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:34 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:34.202+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:35.041 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:34 smithi086 podman[115137]: 2024-04-17 07:37:34.832100298 +0000 UTC m=+6.513471169 container cleanup 1c06539ce62f98c4c618d59f4863a7a75bbc0d9b7d6ba031415dcdc64470d257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-17T07:37:35.042 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:34 smithi086 bash[115137]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6 2024-04-17T07:37:35.330 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:35 smithi086 podman[115167]: 2024-04-17 07:37:35.041489266 +0000 UTC m=+0.663621836 container remove 1c06539ce62f98c4c618d59f4863a7a75bbc0d9b7d6ba031415dcdc64470d257 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-17T07:37:35.330 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:35.206+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:35.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:35.206+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:35.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:35.206+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:35.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:35.206+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:35.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:35 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:35.206+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:35.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:35 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:35.391+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:35.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:35 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:35.391+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:35.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:35 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:35.391+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:35.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:35 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:35.194+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:35.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:35 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:35.194+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:35.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:35 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:35.194+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:35.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:35 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:35.194+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:35.526 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:35 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:35.063+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:35.526 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:35 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:35.063+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:35.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:35 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:35.063+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:35.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:35 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:35.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:35.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:35 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:35.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:35.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:35 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:35.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:35.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:35 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:35.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:35.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:35 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:35.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:36.076 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:35 smithi086 podman[115256]: 2024-04-17 07:37:35.72371978 +0000 UTC m=+0.038724087 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:37:36.076 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:35 smithi086 podman[115256]: 2024-04-17 07:37:35.892496641 +0000 UTC m=+0.207500936 container create 86921084a7af8d4996838bb4a7910a0102f76bd5d6d67f00f14991571c15222c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:37:36.356 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:36 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:36.221+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:36.356 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:36 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:36.221+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:36.356 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:36 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:36.221+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:36.356 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:36 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:36.221+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:36.356 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:36 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:36.221+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:36.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:36 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:36.075+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:36.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:36 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:36.075+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:36.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:36 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:36.075+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:36.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:36 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:36.075+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:36.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:36 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:36.183+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:36.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:36 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:36.183+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:36.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:36 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:36.183+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:36.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:36 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:36.183+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:36.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:36 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:36.183+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:36.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:36 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:36.177+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:36.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:36 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:36.177+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:36.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:36 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:36.177+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:36.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:36 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:36.177+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:36.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:36 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:36.177+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:36.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:36 smithi086 podman[115256]: 2024-04-17 07:37:36.188399359 +0000 UTC m=+0.503403694 container init 86921084a7af8d4996838bb4a7910a0102f76bd5d6d67f00f14991571c15222c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-17T07:37:36.581 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:36 smithi086 podman[115256]: 2024-04-17 07:37:36.200045615 +0000 UTC m=+0.515049946 container start 86921084a7af8d4996838bb4a7910a0102f76bd5d6d67f00f14991571c15222c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:37:36.582 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:36 smithi086 podman[115256]: 2024-04-17 07:37:36.303361914 +0000 UTC m=+0.618366249 container attach 86921084a7af8d4996838bb4a7910a0102f76bd5d6d67f00f14991571c15222c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-17T07:37:36.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:36 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:36.354+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:36.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:36 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:36.354+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:36.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:36 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:36.354+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:37.001 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 17 07:37:36 smithi086 podman[115256]: 2024-04-17 07:37:36.663804857 +0000 UTC m=+0.978809182 container died 86921084a7af8d4996838bb4a7910a0102f76bd5d6d67f00f14991571c15222c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-17T07:37:37.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:37 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:37.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:37.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:37 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:37.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:37.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:37 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:37.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:37.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:37 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:37.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:37.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:37 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:37.165+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:37.375 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:37 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:37.049+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:37.375 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:37 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:37.049+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:37.375 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:37 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:37.049+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:37.375 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:37 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:37.049+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:37.376 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:37 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:37.142+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:37.376 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:37 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:37.142+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:37.376 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:37 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:37.142+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:37.376 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:37 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:37.142+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:37.376 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:37 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:37.142+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:37.414 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.6.service' 2024-04-17T07:37:37.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:37 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:37.336+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:37.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:37 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:37.336+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:37.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:37 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:37.336+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:37.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:37 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:37.246+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:37.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:37 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:37.246+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:37.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:37 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:37.246+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:37.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:37 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:37.246+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:37.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:37 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:37.246+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:37.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:37:37.541 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-17T07:37:37.541 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-17T07:37:37.541 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.7 2024-04-17T07:37:38.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:37 smithi098 systemd[1]: Stopping Ceph osd.7 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:37:38.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:38.221+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:38.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:38.221+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:38.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:38.221+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:38.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:38.221+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:38.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:38.221+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:38.505 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:38.097+0000 7f266a000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:37:38.505 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:38.097+0000 7f266a000700 -1 osd.7 93 *** Got signal Terminated *** 2024-04-17T07:37:38.505 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:38.097+0000 7f266a000700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:37:38.505 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:38.293+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:38.505 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:38.293+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:38.505 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:38 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:38.293+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:38.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:38 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:38.063+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:38.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:38 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:38.063+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:38.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:38 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:38.063+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:38.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:38 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:38.063+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:38.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:38 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:38.135+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:38.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:38 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:38.135+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:38.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:38 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:38.135+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:38.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:38 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:38.135+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:38.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:38 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:38.135+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:38.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:38 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:38.181+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:38.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:38 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:38.181+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:38.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:38 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:38.181+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:38.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:38 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:38.181+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:38.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:38 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:38.181+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:39.330 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:39 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:39.175+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:39.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:39 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:39.175+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:39.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:39 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:39.175+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:39.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:39 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:39.175+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:39.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:39 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:39.175+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:39.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:39.302+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:39.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:39.302+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:39.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:39.302+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:39.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:39.302+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:13.137388+0000 front 2024-04-17T07:37:13.137455+0000 (oldest deadline 2024-04-17T07:37:39.037613+0000) 2024-04-17T07:37:39.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:39.178+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:39.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:39.178+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:39.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:39.178+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:39.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:39.178+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:39.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:39 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:39.178+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:39.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:39 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:39.077+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:39.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:39 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:39.077+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:39.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:39 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:39.077+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:39.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:39 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:39.077+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:39.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:39 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:39.126+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:39.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:39 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:39.126+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:39.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:39 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:39.126+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:39.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:39 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:39.126+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:39.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:39 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:39.126+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:40.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:40 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:40.297+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:40.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:40 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:40.297+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:40.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:40 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:40.297+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:40.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:40 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:40.297+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:13.137388+0000 front 2024-04-17T07:37:13.137455+0000 (oldest deadline 2024-04-17T07:37:39.037613+0000) 2024-04-17T07:37:40.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:40 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:40.187+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:40.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:40 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:40.187+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:40.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:40 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:40.187+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:40.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:40 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:40.187+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:40.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:40 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:40.187+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:40.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:40 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:40.060+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:40.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:40 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:40.060+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:40.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:40 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:40.060+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:40.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:40 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:40.060+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:40.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:40 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:40.169+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:40.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:40 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:40.169+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:40.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:40 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:40.169+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:40.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:40 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:40.169+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:40.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:40 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:40.169+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:40.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:40 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:40.166+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:40.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:40 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:40.166+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:40.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:40 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:40.166+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:40.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:40 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:40.166+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:40.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:40 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:40.166+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:41.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:41 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:41.276+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:41.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:41 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:41.276+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:41.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:41 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:41.276+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:41.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:41 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:41.276+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:13.137388+0000 front 2024-04-17T07:37:13.137455+0000 (oldest deadline 2024-04-17T07:37:39.037613+0000) 2024-04-17T07:37:41.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:41 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:41.232+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:41.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:41 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:41.232+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:41.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:41 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:41.232+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:41.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:41 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:41.232+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:41.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:41 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:41.232+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:41.526 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:41 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:41.068+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:41.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:41 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:41.068+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:41.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:41 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:41.068+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:41.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:41 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:41.068+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:41.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:41 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:41.144+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:41.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:41 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:41.144+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:41.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:41 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:41.144+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:41.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:41 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:41.144+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:41.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:41 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:41.144+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:41.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:41 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:41.191+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:41.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:41 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:41.191+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:41.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:41 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:41.191+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:41.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:41 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:41.191+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:41.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:41 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:41.191+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:42.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:42.311+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.723940+0000 front 2024-04-17T07:36:43.724314+0000 (oldest deadline 2024-04-17T07:37:05.324954+0000) 2024-04-17T07:37:42.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:42.311+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.428675+0000 front 2024-04-17T07:36:53.428491+0000 (oldest deadline 2024-04-17T07:37:18.728412+0000) 2024-04-17T07:37:42.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:42.311+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.830391+0000 front 2024-04-17T07:36:59.830278+0000 (oldest deadline 2024-04-17T07:37:24.932847+0000) 2024-04-17T07:37:42.504 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7[53950]: 2024-04-17T07:37:42.311+0000 7f2665800700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:13.137388+0000 front 2024-04-17T07:37:13.137455+0000 (oldest deadline 2024-04-17T07:37:39.037613+0000) 2024-04-17T07:37:42.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:42.186+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:42.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:42.186+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:42.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:42.186+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:42.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:42.186+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:42.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:42.186+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:42.526 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:42 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:42.108+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:42.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:42 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:42.108+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:42.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:42 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:42.108+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:42.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:42 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:42.108+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:42.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:42 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:42.110+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:42.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:42 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:42.110+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:42.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:42 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:42.110+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:42.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:42 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:42.110+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:42.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:42 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:42.110+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:42.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:42 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:42.196+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:42.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:42 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:42.196+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:42.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:42 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:42.196+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:42.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:42 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:42.196+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:42.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:42 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:42.196+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:43.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:43 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:43.152+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:43.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:43 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:43.152+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:43.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:43 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:43.152+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:43.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:43 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:43.152+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:43.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:43 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:43.152+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:43.526 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:43 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:43.116+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:43.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:43 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:43.116+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:43.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:43 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:43.116+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:43.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:43 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:43.116+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:43.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:43 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:43.102+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:43.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:43 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:43.102+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:43.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:43 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:43.102+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:43.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:43 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:43.102+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:43.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:43 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:43.102+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:43.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:43 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:43.241+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:43.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:43 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:43.241+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:43.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:43 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:43.241+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:43.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:43 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:43.241+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:43.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:43 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:43.241+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:44.253 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:43 smithi098 podman[73027]: 2024-04-17 07:37:43.963354117 +0000 UTC m=+6.018429347 container died f213f7017453e85612b8a27b8931413843c58ee9cb7cb7b1231689cdfa0b0732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-17T07:37:44.253 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:44 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:44.163+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:44.253 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:44 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:44.163+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:44.253 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:44 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:44.163+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:44.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:44 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:44.163+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:44.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:44 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:44.163+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:44.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:44 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:44.120+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:44.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:44 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:44.120+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:44.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:44 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:44.120+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:44.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:44 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:44.120+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:44.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:44 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:44.120+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:44.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:44 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:44.120+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:44.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:44 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:44.120+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:44.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:44 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:44.120+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:44.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:44 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:44.120+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:44.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:44 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:44.263+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:44.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:44 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:44.263+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:44.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:44 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:44.263+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:44.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:44 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:44.264+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:44.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:44 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:44.264+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:44.643 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:44 smithi098 podman[73027]: 2024-04-17 07:37:44.433351613 +0000 UTC m=+6.488426866 container cleanup f213f7017453e85612b8a27b8931413843c58ee9cb7cb7b1231689cdfa0b0732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:37:44.643 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:44 smithi098 bash[73027]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7 2024-04-17T07:37:45.003 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:44 smithi098 podman[73043]: 2024-04-17 07:37:44.642958795 +0000 UTC m=+0.665325018 container remove f213f7017453e85612b8a27b8931413843c58ee9cb7cb7b1231689cdfa0b0732 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-17T07:37:45.486 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:45 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:45.200+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:45.487 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:45 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:45.200+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:45.487 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:45 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:45.200+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:45.487 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:45 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:45.200+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:45.487 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:45 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:45.200+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:45.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:45 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:45.087+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:45.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:45 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:45.087+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:45.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:45 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:45.087+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:45.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:45 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:45.087+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:45.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:45 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:45.087+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:45.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:45 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:45.136+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:45.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:45 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:45.136+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:45.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:45 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:45.136+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:45.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:45 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:45.136+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:45.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:45 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:45.136+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:45.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:45 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:45.245+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:45.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:45 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:45.245+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:45.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:45 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:45.245+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:45.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:45 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:45.245+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:45.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:45 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:45.245+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:45.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:45 smithi098 podman[73135]: 2024-04-17 07:37:45.396529546 +0000 UTC m=+0.041250486 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:37:45.753 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:45 smithi098 podman[73135]: 2024-04-17 07:37:45.535988948 +0000 UTC m=+0.180709907 container create add54fd7f57e33b79bd8dc08289d1da745707e44b9bb2385d5629ddcb29de4fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-17T07:37:46.146 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:45 smithi098 podman[73135]: 2024-04-17 07:37:45.839760753 +0000 UTC m=+0.484481715 container init add54fd7f57e33b79bd8dc08289d1da745707e44b9bb2385d5629ddcb29de4fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-17T07:37:46.146 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:45 smithi098 podman[73135]: 2024-04-17 07:37:45.85119379 +0000 UTC m=+0.495914752 container start add54fd7f57e33b79bd8dc08289d1da745707e44b9bb2385d5629ddcb29de4fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-17T07:37:46.146 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:45 smithi098 podman[73135]: 2024-04-17 07:37:45.930348128 +0000 UTC m=+0.575069154 container attach add54fd7f57e33b79bd8dc08289d1da745707e44b9bb2385d5629ddcb29de4fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:37:46.503 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 17 07:37:46 smithi098 podman[73135]: 2024-04-17 07:37:46.315160604 +0000 UTC m=+0.959881531 container died add54fd7f57e33b79bd8dc08289d1da745707e44b9bb2385d5629ddcb29de4fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-17T07:37:46.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:46 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:46.178+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:46.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:46 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:46.178+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:46.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:46 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:46.178+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:46.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:46 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:46.178+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:46.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:46 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:46.178+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:46.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:46.119+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:46.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:46.119+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:46.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:46.119+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:46.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:46.119+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:46.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:46.119+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:46.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:46.093+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:46.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:46.093+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:46.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:46.093+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:46.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:46.093+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:46.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:46 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:46.093+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:46.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:46.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:46.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:46.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:46.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:46.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:46.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:46.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:46.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:46 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:46.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:47.011 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.7.service' 2024-04-17T07:37:47.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:37:47.125 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-17T07:37:47.125 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-17T07:37:47.125 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.8 2024-04-17T07:37:47.253 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:47 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:47.144+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:47.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:47 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:47.144+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:47.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:47 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:47.144+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:47.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:47 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:47.144+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:47.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:47 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:47.144+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:47.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:47.099+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:47.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:47.099+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:47.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:47.099+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:47.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:47.099+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:47.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:47.099+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:47.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:47.125+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:47.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:47.125+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:47.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:47.125+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:47.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:47.125+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:47.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:47.125+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:47.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:47.125+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.130595+0000 front 2024-04-17T07:37:22.130658+0000 (oldest deadline 2024-04-17T07:37:46.830756+0000) 2024-04-17T07:37:47.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:47 smithi103 systemd[1]: Stopping Ceph osd.8 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:37:47.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:47 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:47.304+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:47.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:47 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:47.304+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:47.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:47 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:47.304+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:47.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:47 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:47.304+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:47.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:47 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:47.304+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:47.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:47 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:47.304+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:48.027 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:47.713+0000 7f4968600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:37:48.027 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:47.713+0000 7f4968600700 -1 osd.8 93 *** Got signal Terminated *** 2024-04-17T07:37:48.027 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:47 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:47.713+0000 7f4968600700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:37:48.501 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:48 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:48.146+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:48.501 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:48 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:48.146+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:48.501 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:48 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:48.146+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:48.501 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:48 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:48.146+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:48.501 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:48 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:48.146+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:48.526 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:48.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:48.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:48.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:48.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:48.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:48.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:48.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:48.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:48.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:48.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:48.150+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:48.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:48.150+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:48.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:48.150+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:48.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:48.150+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:48.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:48.150+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:48.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:48 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:48.150+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.130595+0000 front 2024-04-17T07:37:22.130658+0000 (oldest deadline 2024-04-17T07:37:46.830756+0000) 2024-04-17T07:37:48.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:48 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:48.314+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:48.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:48 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:48.314+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:48.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:48 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:48.314+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:48.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:48 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:48.314+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:48.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:48 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:48.315+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:48.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:48 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:48.315+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:49.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:49 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:49.171+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:49.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:49 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:49.171+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:49.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:49 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:49.171+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:49.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:49 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:49.171+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:49.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:49 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:49.171+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:49.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:49 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:49.171+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:49.133+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:49.133+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:49.133+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:49.133+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:49.133+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:49.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:49.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:49.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:49.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:49.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:49.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:49 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:49.195+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.130595+0000 front 2024-04-17T07:37:22.130658+0000 (oldest deadline 2024-04-17T07:37:46.830756+0000) 2024-04-17T07:37:49.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:49 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:49.309+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:49.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:49 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:49.309+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:49.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:49 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:49.309+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:49.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:49 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:49.309+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:49.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:49 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:49.309+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:49.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:49 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:49.309+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:50.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:50 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:50.193+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:50.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:50 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:50.193+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:50.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:50 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:50.193+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:50.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:50 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:50.193+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:50.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:50 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:50.193+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:50.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:50 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:50.193+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:50.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:50.133+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:50.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:50.133+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:50.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:50.134+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:50.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:50.134+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:50.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:50.134+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:50.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:50.154+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:50.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:50.154+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:50.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:50.154+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:50.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:50.154+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:50.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:50.154+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:50.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:50 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:50.154+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.130595+0000 front 2024-04-17T07:37:22.130658+0000 (oldest deadline 2024-04-17T07:37:46.830756+0000) 2024-04-17T07:37:50.830 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:50 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:50.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:50.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:50 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:50.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:50.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:50 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:50.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:50.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:50 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:50.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:50.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:50 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:50.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:50.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:50 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:50.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:51.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:51 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:51.151+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:51.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:51 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:51.151+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:51.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:51 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:51.151+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:51.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:51 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:51.151+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:51.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:51 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:51.151+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:51.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:51 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:51.151+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:51.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:51.131+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:51.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:51.131+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:51.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:51.131+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:51.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:51.131+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:51.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:51.131+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:51.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:51.170+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:51.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:51.170+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:51.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:51.170+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:51.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:51.170+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:51.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:51.170+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:51.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:51 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:51.170+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.130595+0000 front 2024-04-17T07:37:22.130658+0000 (oldest deadline 2024-04-17T07:37:46.830756+0000) 2024-04-17T07:37:51.830 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:51 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:51.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:51.830 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:51 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:51.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:51.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:51 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:51.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:51.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:51 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:51.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:51.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:51 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:51.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:51.831 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:51 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:51.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:52.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:52 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:52.112+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:52.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:52 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:52.112+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:52.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:52 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:52.112+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:52.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:52 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:52.112+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:52.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:52 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:52.112+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:52.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:52 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:52.112+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:52.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:52.159+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:52.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:52.159+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:52.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:52.159+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:52.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:52.159+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:52.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:52.159+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:52.527 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:52.196+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:29.305145+0000 front 2024-04-17T07:36:29.304548+0000 (oldest deadline 2024-04-17T07:36:57.505837+0000) 2024-04-17T07:37:52.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:52.196+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.910302+0000 front 2024-04-17T07:36:43.909918+0000 (oldest deadline 2024-04-17T07:37:08.010165+0000) 2024-04-17T07:37:52.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:52.196+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:53.716281+0000 front 2024-04-17T07:36:53.715920+0000 (oldest deadline 2024-04-17T07:37:16.616079+0000) 2024-04-17T07:37:52.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:52.196+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:00.118320+0000 front 2024-04-17T07:37:00.118372+0000 (oldest deadline 2024-04-17T07:37:26.018632+0000) 2024-04-17T07:37:52.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:52.196+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:12.324002+0000 front 2024-04-17T07:37:12.323891+0000 (oldest deadline 2024-04-17T07:37:34.624152+0000) 2024-04-17T07:37:52.528 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:52 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8[54324]: 2024-04-17T07:37:52.196+0000 7f495ee00700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.130595+0000 front 2024-04-17T07:37:22.130658+0000 (oldest deadline 2024-04-17T07:37:46.830756+0000) 2024-04-17T07:37:52.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:52 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:52.310+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:52.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:52 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:52.310+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:52.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:52 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:52.310+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:52.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:52 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:52.310+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:52.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:52 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:52.310+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:52.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:52 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:52.310+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:53.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:53 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:53.124+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:53.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:53 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:53.124+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:53.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:53 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:53.124+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:53.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:53 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:53.124+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:53.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:53 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:53.124+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:53.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:53 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:53.124+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:53.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:53 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:53.158+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:53.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:53 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:53.158+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:53.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:53 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:53.158+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:53.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:53 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:53.158+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:53.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:53 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:53.158+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:53.834 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:53 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:53.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:53.834 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:53 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:53.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:53.835 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:53 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:53.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:53.835 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:53 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:53.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:53.835 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:53 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:53.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:53.835 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:53 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:53.336+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:53.955 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:53 smithi103 podman[74515]: 2024-04-17 07:37:53.686319735 +0000 UTC m=+6.117973874 container died 5a4acff0b847e79be828c8cc8bacda29fa9449d22ce0a32ff7472229a9cb2870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-17T07:37:54.274 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:54 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:54.137+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:54.274 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:54 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:54.137+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:54.274 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:54 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:54.137+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:54.274 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:54 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:54.137+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:54.275 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:54 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:54.137+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:54.275 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:54 smithi103 podman[74515]: 2024-04-17 07:37:54.047480268 +0000 UTC m=+6.479134400 container cleanup 5a4acff0b847e79be828c8cc8bacda29fa9449d22ce0a32ff7472229a9cb2870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-17T07:37:54.275 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:54 smithi103 bash[74515]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8 2024-04-17T07:37:54.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:54 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:54.101+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:54.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:54 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:54.101+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:54.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:54 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:54.101+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:54.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:54 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:54.101+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:54.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:54 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:54.101+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:54.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:54 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:54.101+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:54.526 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:54 smithi103 podman[74533]: 2024-04-17 07:37:54.273784094 +0000 UTC m=+0.575225684 container remove 5a4acff0b847e79be828c8cc8bacda29fa9449d22ce0a32ff7472229a9cb2870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-17T07:37:54.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:54 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:54.326+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:54.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:54 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:54.326+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:54.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:54 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:54.326+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:54.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:54 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:54.326+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:54.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:54 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:54.326+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:54.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:54 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:54.326+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:55.429 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:55 smithi103 podman[74616]: 2024-04-17 07:37:54.999073313 +0000 UTC m=+0.035848369 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:37:55.429 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:55 smithi103 podman[74616]: 2024-04-17 07:37:55.124975849 +0000 UTC m=+0.161750906 container create a89030234a054fa9776522663cd5c9ba9f74f219a420d7ebc1e59ed6c7893e55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-17T07:37:55.429 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:55 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:55.129+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:55.429 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:55 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:55.129+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:55.429 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:55 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:55.129+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:55.429 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:55 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:55.129+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:55.429 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:55 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:55.129+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:55.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:55 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:55.102+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:55.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:55 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:55.102+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:55.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:55 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:55.102+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:55.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:55 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:55.102+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:55.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:55 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:55.102+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:55.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:55 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:55.102+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:55.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:55 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:55.294+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:55.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:55 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:55.294+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:55.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:55 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:55.294+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:55.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:55 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:55.294+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:55.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:55 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:55.294+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:55.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:55 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:55.294+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:55.777 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:55 smithi103 podman[74616]: 2024-04-17 07:37:55.465931637 +0000 UTC m=+0.502706730 container init a89030234a054fa9776522663cd5c9ba9f74f219a420d7ebc1e59ed6c7893e55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:37:55.777 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:55 smithi103 podman[74616]: 2024-04-17 07:37:55.478806408 +0000 UTC m=+0.515581502 container start a89030234a054fa9776522663cd5c9ba9f74f219a420d7ebc1e59ed6c7893e55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-17T07:37:55.777 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:55 smithi103 podman[74616]: 2024-04-17 07:37:55.578155136 +0000 UTC m=+0.614930283 container attach a89030234a054fa9776522663cd5c9ba9f74f219a420d7ebc1e59ed6c7893e55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:37:56.276 INFO:journalctl@ceph.osd.8.smithi103.stdout:Apr 17 07:37:55 smithi103 podman[74616]: 2024-04-17 07:37:55.941251378 +0000 UTC m=+0.978026471 container died a89030234a054fa9776522663cd5c9ba9f74f219a420d7ebc1e59ed6c7893e55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:37:56.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:56 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:56.172+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:56.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:56 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:56.172+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:56.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:56 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:56.172+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:56.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:56 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:56.172+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:56.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:56 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:56.172+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:56.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:56 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:56.172+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:37:56.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:56 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:56.077+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:56.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:56 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:56.077+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:56.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:56 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:56.077+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:56.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:56 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:56.077+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:56.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:56 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:56.077+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:56.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:56 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:56.077+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:56.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:56 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:56.254+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:56.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:56 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:56.254+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:56.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:56 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:56.254+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:56.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:56 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:56.254+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:56.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:56 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:56.254+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:56.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:56 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:56.254+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:56.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.8.service' 2024-04-17T07:37:56.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:37:56.791 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-17T07:37:56.791 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-17T07:37:56.791 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.9 2024-04-17T07:37:57.265 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:56 smithi086 systemd[1]: Stopping Ceph osd.9 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:37:57.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:57 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:57.033+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:57.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:57 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:57.033+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:57.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:57 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:57.033+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:57.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:57 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:57.033+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:57.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:57 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:57.033+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:57.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:57 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:57.033+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:57.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:57 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:57.205+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:57.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:57 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:57.205+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:57.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:57 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:57.205+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:57.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:57 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:57.205+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:57.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:57 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:57.205+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:57.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:57 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:57.205+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:37:57.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:57 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:57.263+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:57.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:57 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:57.263+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:57.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:57 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:57.263+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:57.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:57 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:57.263+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:57.582 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:57 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:57.263+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:57.582 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:57 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:57.263+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:57.582 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:57 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:57.264+0000 7fbb8e200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:37:57.582 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:57 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:57.264+0000 7fbb8e200700 -1 osd.9 93 *** Got signal Terminated *** 2024-04-17T07:37:57.582 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:57 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:57.264+0000 7fbb8e200700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:37:58.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:58 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:58.080+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:58.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:58 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:58.080+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:58.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:58 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:58.080+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:58.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:58 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:58.080+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:58.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:58 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:58.080+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:58.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:58 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:58.080+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:58.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:58 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:58.080+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:37:58.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:58 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:58.238+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:58.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:58 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:58.238+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:58.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:58 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:58.238+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:58.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:58 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:58.238+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:58.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:58 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:58.238+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:58.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:58 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:58.238+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:37:58.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:58.300+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:58.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:58.300+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:58.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:58.300+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:58.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:58.300+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:58.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:58.300+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:58.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:58 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:58.300+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:37:59.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:59 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:59.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:37:59.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:59 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:59.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:37:59.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:59 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:59.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:37:59.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:59 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:59.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:37:59.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:59 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:59.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:37:59.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:59 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:59.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:37:59.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:37:59 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:37:59.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:37:59.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:59 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:59.190+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:37:59.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:59 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:59.190+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:37:59.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:59 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:59.190+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:37:59.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:59 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:59.190+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:37:59.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:59 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:59.190+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:37:59.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:37:59 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:37:59.190+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:37:59.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:59 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:59.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:37:59.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:59 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:59.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:37:59.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:59 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:59.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:37:59.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:59 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:59.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:37:59.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:59 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:59.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:37:59.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:37:59 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:37:59.276+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:38:00.253 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:00 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:00.099+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:00.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:00 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:00.099+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:00.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:00 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:00.099+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:00.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:00 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:00.099+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:00.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:00 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:00.099+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:00.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:00 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:00.099+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:00.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:00 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:00.099+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:00.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:00.191+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:00.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:00.191+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:00.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:00.191+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:00.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:00.191+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:00.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:00.191+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:00.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:00 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:00.192+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:00.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:00 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:00.289+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:38:00.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:00 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:00.289+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:38:00.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:00 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:00.289+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:38:00.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:00 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:00.289+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:38:00.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:00 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:00.289+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:38:00.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:00 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:00.289+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:38:01.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:01 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:01.059+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:01.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:01 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:01.059+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:01.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:01 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:01.059+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:01.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:01 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:01.059+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:01.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:01 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:01.059+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:01.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:01 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:01.059+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:01.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:01 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:01.059+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:01.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:01.167+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:01.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:01.167+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:01.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:01.167+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:01.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:01.167+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:01.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:01.167+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:01.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:01 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:01.167+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:01.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:01.251+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:38:01.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:01.251+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:38:01.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:01.251+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:38:01.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:01.251+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:38:01.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:01.251+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:38:01.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:01 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:01.251+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:38:02.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:02 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:02.054+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:02.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:02 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:02.054+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:02.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:02 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:02.054+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:02.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:02 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:02.054+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:02.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:02 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:02.054+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:02.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:02 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:02.054+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:02.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:02 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:02.054+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:02.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:02.156+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:02.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:02.156+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:02.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:02.156+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:02.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:02.156+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:02.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:02.156+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:02.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:02 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:02.156+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:02.580 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:02 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:02.258+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:33.244960+0000 front 2024-04-17T07:36:33.245036+0000 (oldest deadline 2024-04-17T07:36:57.345116+0000) 2024-04-17T07:38:02.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:02 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:02.258+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:43.248020+0000 front 2024-04-17T07:36:43.247675+0000 (oldest deadline 2024-04-17T07:37:05.547924+0000) 2024-04-17T07:38:02.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:02 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:02.258+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.752894+0000 front 2024-04-17T07:36:51.753102+0000 (oldest deadline 2024-04-17T07:37:16.353917+0000) 2024-04-17T07:38:02.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:02 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:02.258+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:36:59.255585+0000 front 2024-04-17T07:36:59.255690+0000 (oldest deadline 2024-04-17T07:37:28.656789+0000) 2024-04-17T07:38:02.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:02 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:02.258+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:08.657858+0000 front 2024-04-17T07:37:08.658154+0000 (oldest deadline 2024-04-17T07:37:34.557550+0000) 2024-04-17T07:38:02.581 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:02 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9[64392]: 2024-04-17T07:38:02.258+0000 7fbb84a00700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.962134+0000 front 2024-04-17T07:37:21.961943+0000 (oldest deadline 2024-04-17T07:37:47.261421+0000) 2024-04-17T07:38:03.405 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:03.096+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:03.405 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:03.096+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:03.405 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:03.096+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:03.406 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:03.096+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:03.406 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:03.096+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:03.406 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:03.096+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:03.406 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:03 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:03.096+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:03.472 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:03 smithi086 podman[115360]: 2024-04-17 07:38:03.153591024 +0000 UTC m=+5.991489219 container died 0471db46e5d9e7c8e0efc5ebd5e9705a9ea9bb40ccb01b32308db837698aa980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:38:03.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:03.165+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:03.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:03.165+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:03.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:03.165+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:03.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:03.165+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:03.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:03.165+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:03.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:03 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:03.165+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:03.830 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:03 smithi086 podman[115360]: 2024-04-17 07:38:03.583948707 +0000 UTC m=+6.421846900 container cleanup 0471db46e5d9e7c8e0efc5ebd5e9705a9ea9bb40ccb01b32308db837698aa980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-17T07:38:03.830 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:03 smithi086 bash[115360]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9 2024-04-17T07:38:04.330 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:03 smithi086 podman[115390]: 2024-04-17 07:38:03.860046509 +0000 UTC m=+0.687991149 container remove 0471db46e5d9e7c8e0efc5ebd5e9705a9ea9bb40ccb01b32308db837698aa980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-17T07:38:04.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:04 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:04.050+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:04.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:04 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:04.050+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:04.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:04 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:04.050+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:04.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:04 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:04.050+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:04.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:04 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:04.050+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:04.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:04 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:04.050+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:04.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:04 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:04.050+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:04.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:04.150+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:04.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:04.150+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:04.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:04.150+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:04.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:04.150+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:04.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:04.150+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:04.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:04 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:04.150+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:04.934 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:04 smithi086 podman[115478]: 2024-04-17 07:38:04.562316354 +0000 UTC m=+0.038906697 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:38:04.934 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:04 smithi086 podman[115478]: 2024-04-17 07:38:04.686035054 +0000 UTC m=+0.162625386 container create 0a0d206c952bbdcea0f2253ada65a8afdd2b0101eeffdebc85eb6a68b99c4597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:38:04.934 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:04 smithi086 podman[115478]: 2024-04-17 07:38:04.933397207 +0000 UTC m=+0.409987563 container init 0a0d206c952bbdcea0f2253ada65a8afdd2b0101eeffdebc85eb6a68b99c4597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=) 2024-04-17T07:38:05.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:04 smithi086 podman[115478]: 2024-04-17 07:38:04.946306557 +0000 UTC m=+0.422896896 container start 0a0d206c952bbdcea0f2253ada65a8afdd2b0101eeffdebc85eb6a68b99c4597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:38:05.331 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:05 smithi086 podman[115478]: 2024-04-17 07:38:05.029936215 +0000 UTC m=+0.506526561 container attach 0a0d206c952bbdcea0f2253ada65a8afdd2b0101eeffdebc85eb6a68b99c4597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:38:05.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:05.021+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:05.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:05.021+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:05.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:05.021+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:05.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:05.021+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:05.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:05.021+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:05.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:05.021+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:05.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:05 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:05.021+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:05.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:05.177+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:05.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:05.177+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:05.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:05.177+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:05.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:05.177+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:05.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:05.177+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:05.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:05.177+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:05.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:05 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:05.177+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:05.695 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 17 07:38:05 smithi086 podman[115478]: 2024-04-17 07:38:05.39278438 +0000 UTC m=+0.869374736 container died 0a0d206c952bbdcea0f2253ada65a8afdd2b0101eeffdebc85eb6a68b99c4597 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-17T07:38:06.092 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.9.service' 2024-04-17T07:38:06.210 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:38:06.211 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-17T07:38:06.211 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-17T07:38:06.211 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.10 2024-04-17T07:38:06.354 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:06.066+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:06.354 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:06.066+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:06.354 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:06.066+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:06.354 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:06.066+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:06.354 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:06.066+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:06.354 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:06.066+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:06.355 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:06.066+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:06.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:06.139+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:06.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:06.139+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:06.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:06.139+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:06.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:06.139+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:06.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:06.139+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:06.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:06.139+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:06.530 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:06 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:06.139+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:06.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 systemd[1]: Stopping Ceph osd.10 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:38:07.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:06.861+0000 7fab80e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:38:07.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:06.861+0000 7fab80e00700 -1 osd.10 93 *** Got signal Terminated *** 2024-04-17T07:38:07.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:06 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:06.861+0000 7fab80e00700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:38:07.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:07.078+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:07.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:07.078+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:07.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:07.078+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:07.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:07.078+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:07.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:07.078+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:07.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:07.078+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:07.254 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:07 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:07.078+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:07.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:07.111+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:07.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:07.111+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:07.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:07.111+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:07.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:07.111+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:07.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:07.111+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:07.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:07.111+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:07.277 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:07 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:07.111+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:08.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:08.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:08.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:08.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:08.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:08.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:08.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:08.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:08.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:08.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:08.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:08.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:08.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:08 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:08.086+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:08.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:08.084+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:08.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:08.084+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:08.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:08.084+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:08.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:08.084+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:08.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:08.084+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:08.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:08.084+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:08.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:08.084+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:09.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:09.115+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:09.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:09.115+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:09.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:09.115+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:09.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:09.115+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:09.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:09.115+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:09.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:09.115+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:09.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:09 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:09.115+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:09.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:09.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:09.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:09.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:09.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:09.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:09.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:09.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:09.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:09.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:09.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:09.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:09.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:09 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:09.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:10.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:10.073+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:10.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:10.073+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:10.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:10.073+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:10.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:10.073+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:10.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:10.073+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:10.505 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:10.073+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:10.505 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:10 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:10.073+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:10.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:10.087+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:10.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:10.087+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:10.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:10.087+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:10.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:10.087+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:10.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:10.087+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:10.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:10.087+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:10.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:10 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:10.087+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:11.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:11.117+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:34.054352+0000 front 2024-04-17T07:36:34.054407+0000 (oldest deadline 2024-04-17T07:36:59.954603+0000) 2024-04-17T07:38:11.503 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:11.117+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:39.955478+0000 front 2024-04-17T07:36:39.955650+0000 (oldest deadline 2024-04-17T07:37:05.157416+0000) 2024-04-17T07:38:11.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:11.117+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:51.560551+0000 front 2024-04-17T07:36:51.560405+0000 (oldest deadline 2024-04-17T07:37:16.860480+0000) 2024-04-17T07:38:11.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:11.117+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-17T07:37:02.564675+0000 front 2024-04-17T07:37:02.565051+0000 (oldest deadline 2024-04-17T07:37:24.864613+0000) 2024-04-17T07:38:11.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:11.117+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:11.768107+0000 front 2024-04-17T07:37:11.768190+0000 (oldest deadline 2024-04-17T07:37:35.267964+0000) 2024-04-17T07:38:11.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:11.117+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:22.672287+0000 front 2024-04-17T07:37:22.672197+0000 (oldest deadline 2024-04-17T07:37:48.572013+0000) 2024-04-17T07:38:11.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:11 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10[60961]: 2024-04-17T07:38:11.117+0000 7fab77600700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.474319+0000 front 2024-04-17T07:37:31.474370+0000 (oldest deadline 2024-04-17T07:37:57.874682+0000) 2024-04-17T07:38:11.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:11.096+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:11.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:11.096+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:11.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:11.096+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:11.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:11.096+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:11.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:11.096+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:11.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:11.096+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:11.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:11 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:11.096+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:12.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:12.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:12.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:12.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:12.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:12.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:12.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:12.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:12.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:12.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:12.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:12.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:12.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:12 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:12.051+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:12.892 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:12 smithi098 podman[73255]: 2024-04-17 07:38:12.630792388 +0000 UTC m=+5.921123323 container died 6ce9930a838b46846182082696af669b307e2e8e43863ac3d9589e623783a968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-17T07:38:13.253 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:13 smithi098 podman[73255]: 2024-04-17 07:38:13.03764735 +0000 UTC m=+6.327978338 container cleanup 6ce9930a838b46846182082696af669b307e2e8e43863ac3d9589e623783a968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:38:13.253 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:13 smithi098 bash[73255]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10 2024-04-17T07:38:13.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:13.040+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:13.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:13.040+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:13.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:13.040+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:13.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:13.040+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:13.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:13.040+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:13.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:13.040+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:13.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:13 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:13.040+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:13.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:13 smithi098 podman[73274]: 2024-04-17 07:38:13.280404083 +0000 UTC m=+0.633065567 container remove 6ce9930a838b46846182082696af669b307e2e8e43863ac3d9589e623783a968 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-17T07:38:14.319 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:14 smithi098 podman[73358]: 2024-04-17 07:38:13.94599286 +0000 UTC m=+0.032492320 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:38:14.319 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:14 smithi098 podman[73358]: 2024-04-17 07:38:14.064859724 +0000 UTC m=+0.151359209 container create 2cadba347d05df60f9909b8ff1b122f334b5e5ed1865a502f78694acb13d06a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-17T07:38:14.319 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:14 smithi098 podman[73358]: 2024-04-17 07:38:14.318807443 +0000 UTC m=+0.405306935 container init 2cadba347d05df60f9909b8ff1b122f334b5e5ed1865a502f78694acb13d06a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-17T07:38:14.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:14.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:14.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:14.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:14.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:14.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:14.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:14.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:14.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:14.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:14.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:14.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:14.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:14.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:14.753 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:14 smithi098 podman[73358]: 2024-04-17 07:38:14.330221886 +0000 UTC m=+0.416721378 container start 2cadba347d05df60f9909b8ff1b122f334b5e5ed1865a502f78694acb13d06a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-04-17T07:38:14.754 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:14 smithi098 podman[73358]: 2024-04-17 07:38:14.417223116 +0000 UTC m=+0.503722621 container attach 2cadba347d05df60f9909b8ff1b122f334b5e5ed1865a502f78694acb13d06a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-17T07:38:15.131 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 17 07:38:14 smithi098 podman[73358]: 2024-04-17 07:38:14.78586803 +0000 UTC m=+0.872367525 container died 2cadba347d05df60f9909b8ff1b122f334b5e5ed1865a502f78694acb13d06a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:38:15.526 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.10.service' 2024-04-17T07:38:15.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:15.059+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:15.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:15.059+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:15.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:15.059+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:15.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:15.059+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:15.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:15.059+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:15.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:15.059+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:15.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:15 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:15.059+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:15.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:38:15.656 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-17T07:38:15.656 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-17T07:38:15.657 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.11 2024-04-17T07:38:16.026 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:15 smithi103 systemd[1]: Stopping Ceph osd.11 for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:38:16.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:16.039+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:16.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:16.039+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:16.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:16.039+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:16.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:16.039+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:16.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:16.039+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:16.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:16.039+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:16.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:16.039+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:16.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:16.224+0000 7f1626800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:38:16.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:16.224+0000 7f1626800700 -1 osd.11 93 *** Got signal Terminated *** 2024-04-17T07:38:16.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:16 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:16.224+0000 7f1626800700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-17T07:38:17.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:17.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:17.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:17.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:17.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:17.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:17.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:17.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:17.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:17.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:17.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:17.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:17.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:17 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:17.085+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:18.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:18.124+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:18.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:18.124+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:18.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:18.124+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:18.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:18.124+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:18.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:18.124+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:18.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:18.124+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:18.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:18.124+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:18.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:18.124+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6820 osd.8 since back 2024-04-17T07:37:52.671854+0000 front 2024-04-17T07:37:52.672268+0000 (oldest deadline 2024-04-17T07:38:17.971759+0000) 2024-04-17T07:38:19.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:19.152+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:19.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:19.152+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:19.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:19.152+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:19.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:19.152+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:19.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:19.152+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:19.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:19.152+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:19.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:19.152+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:19.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:19 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:19.152+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6820 osd.8 since back 2024-04-17T07:37:52.671854+0000 front 2024-04-17T07:37:52.672268+0000 (oldest deadline 2024-04-17T07:38:17.971759+0000) 2024-04-17T07:38:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:20.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:20.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:20.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:20.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:20.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:20.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:20.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:20.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:20 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:20.117+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6820 osd.8 since back 2024-04-17T07:37:52.671854+0000 front 2024-04-17T07:37:52.672268+0000 (oldest deadline 2024-04-17T07:38:17.971759+0000) 2024-04-17T07:38:21.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:21.123+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-17T07:36:31.733357+0000 front 2024-04-17T07:36:31.733429+0000 (oldest deadline 2024-04-17T07:36:55.833550+0000) 2024-04-17T07:38:21.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:21.123+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-17T07:36:41.540399+0000 front 2024-04-17T07:36:41.540032+0000 (oldest deadline 2024-04-17T07:37:06.239810+0000) 2024-04-17T07:38:21.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:21.123+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6804 osd.2 since back 2024-04-17T07:36:49.642636+0000 front 2024-04-17T07:36:49.642745+0000 (oldest deadline 2024-04-17T07:37:15.542671+0000) 2024-04-17T07:38:21.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:21.123+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-17T07:37:09.754041+0000 front 2024-04-17T07:37:09.753922+0000 (oldest deadline 2024-04-17T07:37:35.654113+0000) 2024-04-17T07:38:21.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:21.123+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6812 osd.5 since back 2024-04-17T07:37:21.360011+0000 front 2024-04-17T07:37:21.360153+0000 (oldest deadline 2024-04-17T07:37:44.860184+0000) 2024-04-17T07:38:21.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:21.123+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-17T07:37:31.263143+0000 front 2024-04-17T07:37:31.262902+0000 (oldest deadline 2024-04-17T07:37:55.962960+0000) 2024-04-17T07:38:21.527 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:21.123+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-17T07:37:41.865244+0000 front 2024-04-17T07:37:41.865364+0000 (oldest deadline 2024-04-17T07:38:04.765346+0000) 2024-04-17T07:38:21.528 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:21 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11[61446]: 2024-04-17T07:38:21.123+0000 7f161d000700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.103:6820 osd.8 since back 2024-04-17T07:37:52.671854+0000 front 2024-04-17T07:37:52.672268+0000 (oldest deadline 2024-04-17T07:38:17.971759+0000) 2024-04-17T07:38:22.367 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:22 smithi103 podman[74733]: 2024-04-17 07:38:22.073996306 +0000 UTC m=+5.991568602 container died d3816451d8f4e8b254215ce5369569795e1a6c157ba7891bbd56a85ddff2fbf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-17T07:38:22.678 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:22 smithi103 podman[74733]: 2024-04-17 07:38:22.494136942 +0000 UTC m=+6.411709231 container cleanup d3816451d8f4e8b254215ce5369569795e1a6c157ba7891bbd56a85ddff2fbf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-17T07:38:22.678 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:22 smithi103 bash[74733]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11 2024-04-17T07:38:23.026 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:22 smithi103 podman[74752]: 2024-04-17 07:38:22.678410066 +0000 UTC m=+0.594498726 container remove d3816451d8f4e8b254215ce5369569795e1a6c157ba7891bbd56a85ddff2fbf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-17T07:38:23.714 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:23 smithi103 podman[74840]: 2024-04-17 07:38:23.36489488 +0000 UTC m=+0.035426979 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-17T07:38:23.714 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:23 smithi103 podman[74840]: 2024-04-17 07:38:23.471282247 +0000 UTC m=+0.141814374 container create e9032b9018b96a803f6c7a337794f2dcfa52ae5521d0d822d816b551be6c0fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-17T07:38:23.714 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:23 smithi103 podman[74840]: 2024-04-17 07:38:23.702536919 +0000 UTC m=+0.373069053 container init e9032b9018b96a803f6c7a337794f2dcfa52ae5521d0d822d816b551be6c0fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:38:23.714 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:23 smithi103 podman[74840]: 2024-04-17 07:38:23.714257944 +0000 UTC m=+0.384790107 container start e9032b9018b96a803f6c7a337794f2dcfa52ae5521d0d822d816b551be6c0fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-17T07:38:24.026 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:23 smithi103 podman[74840]: 2024-04-17 07:38:23.8072064 +0000 UTC m=+0.477738530 container attach e9032b9018b96a803f6c7a337794f2dcfa52ae5521d0d822d816b551be6c0fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-17T07:38:24.526 INFO:journalctl@ceph.osd.11.smithi103.stdout:Apr 17 07:38:24 smithi103 podman[74840]: 2024-04-17 07:38:24.174053172 +0000 UTC m=+0.844585306 container died e9032b9018b96a803f6c7a337794f2dcfa52ae5521d0d822d816b551be6c0fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:38:24.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@osd.11.service' 2024-04-17T07:38:25.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:38:25.043 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-17T07:38:25.043 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-17T07:38:25.043 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.a 2024-04-17T07:38:25.580 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:38:25 smithi086 systemd[1]: Stopping Ceph mds.a for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:38:26.080 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:38:25 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-a[71154]: 2024-04-17T07:38:25.590+0000 7f07b6a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:38:26.081 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:38:25 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-a[71154]: 2024-04-17T07:38:25.590+0000 7f07b6a00700 -1 mds.a *** got signal Terminated *** 2024-04-17T07:38:29.374 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:38:29 smithi086 podman[115581]: 2024-04-17 07:38:29.12032598 +0000 UTC m=+3.644841240 container died f0157b6dc715473c6b19f5b21465f444e49c573c6e7d7634bd0d4c3e67fb4c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:38:29.693 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:38:29 smithi086 podman[115581]: 2024-04-17 07:38:29.467822537 +0000 UTC m=+3.992337787 container cleanup f0157b6dc715473c6b19f5b21465f444e49c573c6e7d7634bd0d4c3e67fb4c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-17T07:38:29.694 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:38:29 smithi086 bash[115581]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-a 2024-04-17T07:38:30.080 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 17 07:38:29 smithi086 podman[115595]: 2024-04-17 07:38:29.693764124 +0000 UTC m=+0.561600397 container remove f0157b6dc715473c6b19f5b21465f444e49c573c6e7d7634bd0d4c3e67fb4c66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-17T07:38:30.205 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.a.service' 2024-04-17T07:38:30.324 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:38:30.324 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-17T07:38:30.324 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-17T07:38:30.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.d 2024-04-17T07:38:30.830 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 17 07:38:30 smithi086 systemd[1]: Stopping Ceph mds.d for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:38:31.331 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 17 07:38:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-d[70092]: 2024-04-17T07:38:30.969+0000 7f1c67000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:38:31.331 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 17 07:38:30 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-d[70092]: 2024-04-17T07:38:30.969+0000 7f1c67000700 -1 mds.d *** got signal Terminated *** 2024-04-17T07:38:32.831 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 17 07:38:32 smithi086 podman[115700]: 2024-04-17 07:38:32.542012391 +0000 UTC m=+1.710902263 container stop 67a4d7830c90073b3b6f65c5a2a3a921c3cd9ec1b49a1f4cf8371468adba5453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:38:32.831 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 17 07:38:32 smithi086 podman[115700]: 2024-04-17 07:38:32.569579013 +0000 UTC m=+1.738468848 container died 67a4d7830c90073b3b6f65c5a2a3a921c3cd9ec1b49a1f4cf8371468adba5453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-17T07:38:33.172 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 17 07:38:32 smithi086 podman[115700]: 2024-04-17 07:38:32.988131237 +0000 UTC m=+2.157021096 container cleanup 67a4d7830c90073b3b6f65c5a2a3a921c3cd9ec1b49a1f4cf8371468adba5453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-17T07:38:33.172 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 17 07:38:32 smithi086 bash[115700]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-d 2024-04-17T07:38:33.580 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 17 07:38:33 smithi086 podman[115725]: 2024-04-17 07:38:33.172568788 +0000 UTC m=+0.613588916 container remove 67a4d7830c90073b3b6f65c5a2a3a921c3cd9ec1b49a1f4cf8371468adba5453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-17T07:38:33.680 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.d.service' 2024-04-17T07:38:33.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:38:33.792 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-17T07:38:33.792 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-17T07:38:33.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.g 2024-04-17T07:38:34.330 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:38:33 smithi086 systemd[1]: Stopping Ceph mds.g for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:38:34.830 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:38:34 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-g[70811]: 2024-04-17T07:38:34.423+0000 7fe978600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:38:34.830 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:38:34 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-g[70811]: 2024-04-17T07:38:34.423+0000 7fe978600700 -1 mds.g *** got signal Terminated *** 2024-04-17T07:38:37.330 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:38:37 smithi086 podman[115832]: 2024-04-17 07:38:37.045476422 +0000 UTC m=+2.750645869 container died ed9543eec6ff30c144e770a438537b88689dbb73ad9b4c0daf7f5f9be8f1869b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-17T07:38:37.580 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:38:37 smithi086 podman[115832]: 2024-04-17 07:38:37.45008937 +0000 UTC m=+3.155258858 container cleanup ed9543eec6ff30c144e770a438537b88689dbb73ad9b4c0daf7f5f9be8f1869b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-17T07:38:37.580 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:38:37 smithi086 bash[115832]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-g 2024-04-17T07:38:38.000 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 17 07:38:37 smithi086 podman[115846]: 2024-04-17 07:38:37.601569878 +0000 UTC m=+0.541581750 container remove ed9543eec6ff30c144e770a438537b88689dbb73ad9b4c0daf7f5f9be8f1869b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-17T07:38:38.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.g.service' 2024-04-17T07:38:38.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:38:38.148 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-17T07:38:38.149 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-17T07:38:38.149 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.j 2024-04-17T07:38:38.580 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:38:38 smithi086 systemd[1]: Stopping Ceph mds.j for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:38:39.080 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:38:38 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-j[69710]: 2024-04-17T07:38:38.752+0000 7f3af8800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:38:39.081 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:38:38 smithi086 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-j[69710]: 2024-04-17T07:38:38.752+0000 7f3af8800700 -1 mds.j *** got signal Terminated *** 2024-04-17T07:38:40.699 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:38:40 smithi086 podman[115952]: 2024-04-17 07:38:40.401071078 +0000 UTC m=+1.784247153 container died 97cd7ae16a23e8cebac3a9faa576bff966348322c40061fbc4c5c9e8aa6db61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-j, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-17T07:38:41.080 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:38:40 smithi086 podman[115952]: 2024-04-17 07:38:40.795810315 +0000 UTC m=+2.178986413 container cleanup 97cd7ae16a23e8cebac3a9faa576bff966348322c40061fbc4c5c9e8aa6db61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-17T07:38:41.080 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:38:40 smithi086 bash[115952]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-j 2024-04-17T07:38:41.386 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 17 07:38:41 smithi086 podman[115966]: 2024-04-17 07:38:41.121473908 +0000 UTC m=+0.707958557 container remove 97cd7ae16a23e8cebac3a9faa576bff966348322c40061fbc4c5c9e8aa6db61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-17T07:38:41.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.j.service' 2024-04-17T07:38:41.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:38:41.519 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-17T07:38:41.519 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-17T07:38:41.519 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.b 2024-04-17T07:38:41.884 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:38:41 smithi098 systemd[1]: Stopping Ceph mds.b for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:38:42.251 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:38:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b[66166]: 2024-04-17T07:38:42.069+0000 7f64d5a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:38:42.251 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:38:42 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b[66166]: 2024-04-17T07:38:42.069+0000 7f64d5a00700 -1 mds.b *** got signal Terminated *** 2024-04-17T07:38:47.378 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:38:47 smithi098 podman[73462]: 2024-04-17 07:38:47.097575345 +0000 UTC m=+5.151238529 container died 28e0e021f6c9f36ec9fc8e2a5b2f9da8016b2fe61e34b064eaeb8d3862f225b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-17T07:38:47.691 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:38:47 smithi098 podman[73462]: 2024-04-17 07:38:47.506382782 +0000 UTC m=+5.560045988 container cleanup 28e0e021f6c9f36ec9fc8e2a5b2f9da8016b2fe61e34b064eaeb8d3862f225b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:38:47.691 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:38:47 smithi098 bash[73462]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b 2024-04-17T07:38:48.002 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 17 07:38:47 smithi098 podman[73487]: 2024-04-17 07:38:47.690945187 +0000 UTC m=+0.574130646 container remove 28e0e021f6c9f36ec9fc8e2a5b2f9da8016b2fe61e34b064eaeb8d3862f225b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-17T07:38:48.193 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.b.service' 2024-04-17T07:38:48.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:38:48.313 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-17T07:38:48.313 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-17T07:38:48.313 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.e 2024-04-17T07:38:48.754 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:38:48 smithi098 systemd[1]: Stopping Ceph mds.e for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:38:49.253 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:38:48 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-e[65550]: 2024-04-17T07:38:48.926+0000 7f1b95c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:38:49.253 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:38:48 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-e[65550]: 2024-04-17T07:38:48.926+0000 7f1b95c00700 -1 mds.e *** got signal Terminated *** 2024-04-17T07:38:53.753 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:38:53 smithi098 podman[73594]: 2024-04-17 07:38:53.493512799 +0000 UTC m=+4.744135165 container died 19cc2df441fd76c09b0450f17eac474a11fc52d8f1a87fd5a4c3452dd6ce529d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-e, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-17T07:38:54.169 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:38:53 smithi098 podman[73594]: 2024-04-17 07:38:53.972321946 +0000 UTC m=+5.222944340 container cleanup 19cc2df441fd76c09b0450f17eac474a11fc52d8f1a87fd5a4c3452dd6ce529d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:38:54.170 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:38:53 smithi098 bash[73594]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-e 2024-04-17T07:38:54.503 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 17 07:38:54 smithi098 podman[73608]: 2024-04-17 07:38:54.169735321 +0000 UTC m=+0.661475056 container remove 19cc2df441fd76c09b0450f17eac474a11fc52d8f1a87fd5a4c3452dd6ce529d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:38:54.665 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.e.service' 2024-04-17T07:38:54.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:38:54.782 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-17T07:38:54.782 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-17T07:38:54.782 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.h 2024-04-17T07:38:55.253 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:38:54 smithi098 systemd[1]: Stopping Ceph mds.h for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:38:55.753 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:38:55 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-h[65851]: 2024-04-17T07:38:55.454+0000 7f1f69400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:38:55.753 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:38:55 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-h[65851]: 2024-04-17T07:38:55.454+0000 7f1f69400700 -1 mds.h *** got signal Terminated *** 2024-04-17T07:39:00.311 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:39:00 smithi098 podman[73716]: 2024-04-17 07:39:00.012241973 +0000 UTC m=+4.703547218 container died 2590f5f40a5fc92449cb813d6746588c87b0940bc1a50e5237cd467844365266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:39:00.673 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:39:00 smithi098 podman[73716]: 2024-04-17 07:39:00.447388473 +0000 UTC m=+5.138693728 container cleanup 2590f5f40a5fc92449cb813d6746588c87b0940bc1a50e5237cd467844365266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-h, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-17T07:39:00.674 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:39:00 smithi098 bash[73716]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-h 2024-04-17T07:39:01.003 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 17 07:39:00 smithi098 podman[73730]: 2024-04-17 07:39:00.673651444 +0000 UTC m=+0.645854900 container remove 2590f5f40a5fc92449cb813d6746588c87b0940bc1a50e5237cd467844365266 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-17T07:39:01.146 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.h.service' 2024-04-17T07:39:01.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:39:01.253 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-17T07:39:01.254 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-17T07:39:01.254 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.k 2024-04-17T07:39:01.503 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:39:01 smithi098 systemd[1]: Stopping Ceph mds.k for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:39:02.253 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:39:01 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-k[65234]: 2024-04-17T07:39:01.808+0000 7f47eb000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:39:02.253 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:39:01 smithi098 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-k[65234]: 2024-04-17T07:39:01.808+0000 7f47eb000700 -1 mds.k *** got signal Terminated *** 2024-04-17T07:39:06.753 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:39:06 smithi098 podman[73837]: 2024-04-17 07:39:06.414271509 +0000 UTC m=+4.735756557 container died 6b3e665830c96c209a39bb971f362fda288a0c4f127f9265335097409985749a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-17T07:39:07.143 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:39:06 smithi098 podman[73837]: 2024-04-17 07:39:06.901002919 +0000 UTC m=+5.222487964 container cleanup 6b3e665830c96c209a39bb971f362fda288a0c4f127f9265335097409985749a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-17T07:39:07.143 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:39:06 smithi098 bash[73837]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-k 2024-04-17T07:39:07.503 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 17 07:39:07 smithi098 podman[73851]: 2024-04-17 07:39:07.143364798 +0000 UTC m=+0.667908461 container remove 6b3e665830c96c209a39bb971f362fda288a0c4f127f9265335097409985749a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-k, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-17T07:39:07.645 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.k.service' 2024-04-17T07:39:07.760 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:39:07.761 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-17T07:39:07.761 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-17T07:39:07.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.c 2024-04-17T07:39:08.277 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:39:07 smithi103 systemd[1]: Stopping Ceph mds.c for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:39:08.776 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:39:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-c[66786]: 2024-04-17T07:39:08.326+0000 7f6fd0600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:39:08.777 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:39:08 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-c[66786]: 2024-04-17T07:39:08.326+0000 7f6fd0600700 -1 mds.c *** got signal Terminated *** 2024-04-17T07:39:12.820 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:39:12 smithi103 podman[74957]: 2024-04-17 07:39:12.552988151 +0000 UTC m=+4.371775097 container died e7940f396d362baa7ad05506634effde6f59ffcbad0db602ab0b2ff7ab7fe12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-17T07:39:13.156 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:39:12 smithi103 podman[74957]: 2024-04-17 07:39:12.912389846 +0000 UTC m=+4.731176854 container cleanup e7940f396d362baa7ad05506634effde6f59ffcbad0db602ab0b2ff7ab7fe12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:39:13.156 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:39:12 smithi103 bash[74957]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-c 2024-04-17T07:39:13.526 INFO:journalctl@ceph.mds.c.smithi103.stdout:Apr 17 07:39:13 smithi103 podman[74983]: 2024-04-17 07:39:13.156276838 +0000 UTC m=+0.587153446 container remove e7940f396d362baa7ad05506634effde6f59ffcbad0db602ab0b2ff7ab7fe12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-17T07:39:13.638 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.c.service' 2024-04-17T07:39:13.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:39:13.759 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-17T07:39:13.759 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-17T07:39:13.759 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.f 2024-04-17T07:39:14.277 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:39:13 smithi103 systemd[1]: Stopping Ceph mds.f for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:39:14.776 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:39:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-f[66472]: 2024-04-17T07:39:14.288+0000 7ff950c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:39:14.777 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:39:14 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-f[66472]: 2024-04-17T07:39:14.288+0000 7ff950c00700 -1 mds.f *** got signal Terminated *** 2024-04-17T07:39:16.926 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:39:16 smithi103 podman[75089]: 2024-04-17 07:39:16.635900561 +0000 UTC m=+2.466484568 container died 7b5269347c63d0b52bc31fdb37d33f096b594db53eb6ec4ddb64b0388b84a4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-17T07:39:17.276 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:39:17 smithi103 podman[75089]: 2024-04-17 07:39:17.041650905 +0000 UTC m=+2.872234909 container cleanup 7b5269347c63d0b52bc31fdb37d33f096b594db53eb6ec4ddb64b0388b84a4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:39:17.276 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:39:17 smithi103 bash[75089]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-f 2024-04-17T07:39:17.626 INFO:journalctl@ceph.mds.f.smithi103.stdout:Apr 17 07:39:17 smithi103 podman[75103]: 2024-04-17 07:39:17.359776965 +0000 UTC m=+0.703862357 container remove 7b5269347c63d0b52bc31fdb37d33f096b594db53eb6ec4ddb64b0388b84a4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:39:17.643 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.f.service' 2024-04-17T07:39:17.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:39:17.756 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-17T07:39:17.756 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-17T07:39:17.756 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.i 2024-04-17T07:39:18.276 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:39:17 smithi103 systemd[1]: Stopping Ceph mds.i for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:39:18.776 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:39:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-i[67092]: 2024-04-17T07:39:18.387+0000 7f8a49600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:39:18.776 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:39:18 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-i[67092]: 2024-04-17T07:39:18.387+0000 7f8a49600700 -1 mds.i *** got signal Terminated *** 2024-04-17T07:39:23.918 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:39:23 smithi103 podman[75209]: 2024-04-17 07:39:23.578443478 +0000 UTC m=+5.327508953 container died 8e3e7493cc70d8a7c9e861ea01966912ee235ae7cd6bc1d032c07e9980f22294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-17T07:39:24.238 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:39:24 smithi103 podman[75209]: 2024-04-17 07:39:24.029222988 +0000 UTC m=+5.778288507 container cleanup 8e3e7493cc70d8a7c9e861ea01966912ee235ae7cd6bc1d032c07e9980f22294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:39:24.239 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:39:24 smithi103 bash[75209]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-i 2024-04-17T07:39:24.526 INFO:journalctl@ceph.mds.i.smithi103.stdout:Apr 17 07:39:24 smithi103 podman[75223]: 2024-04-17 07:39:24.238608012 +0000 UTC m=+0.647912212 container remove 8e3e7493cc70d8a7c9e861ea01966912ee235ae7cd6bc1d032c07e9980f22294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-17T07:39:24.783 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.i.service' 2024-04-17T07:39:24.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:39:24.898 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-17T07:39:24.898 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-17T07:39:24.898 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.l 2024-04-17T07:39:25.276 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:39:25 smithi103 systemd[1]: Stopping Ceph mds.l for 1f7ed99a-fc88-11ee-bc91-c7b262605968... 2024-04-17T07:39:25.776 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:39:25 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-l[66159]: 2024-04-17T07:39:25.522+0000 7fe4fee00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-17T07:39:25.777 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:39:25 smithi103 ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-l[66159]: 2024-04-17T07:39:25.522+0000 7fe4fee00700 -1 mds.l *** got signal Terminated *** 2024-04-17T07:39:28.661 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:39:28 smithi103 podman[75330]: 2024-04-17 07:39:28.402243401 +0000 UTC m=+3.022911149 container died 8cccce9551256533fc1da323353b8dc824e98278d843ada695a3b4f5ce3ada4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-17T07:39:28.942 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:39:28 smithi103 podman[75330]: 2024-04-17 07:39:28.749677686 +0000 UTC m=+3.370345428 container cleanup 8cccce9551256533fc1da323353b8dc824e98278d843ada695a3b4f5ce3ada4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-17T07:39:28.942 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:39:28 smithi103 bash[75330]: ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-l 2024-04-17T07:39:29.276 INFO:journalctl@ceph.mds.l.smithi103.stdout:Apr 17 07:39:28 smithi103 podman[75343]: 2024-04-17 07:39:28.942164411 +0000 UTC m=+0.511880346 container remove 8cccce9551256533fc1da323353b8dc824e98278d843ada695a3b4f5ce3ada4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968-mds-l, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-17T07:39:29.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1f7ed99a-fc88-11ee-bc91-c7b262605968@mds.l.service' 2024-04-17T07:39:29.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-17T07:39:29.581 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-17T07:39:29.581 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 --force --keep-logs 2024-04-17T07:39:29.907 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:39:34.914 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 --force --keep-logs 2024-04-17T07:39:35.245 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:39:40.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 --force --keep-logs 2024-04-17T07:39:40.649 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:39:45.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:39:45.506 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:39:45.611 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-17T07:39:45.711 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-17T07:39:45.713 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891/remote/smithi086/crash 2024-04-17T07:39:45.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/crash -- . 2024-04-17T07:39:45.839 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/crash: Cannot open: No such file or directory 2024-04-17T07:39:45.840 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-04-17T07:39:45.844 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891/remote/smithi098/crash 2024-04-17T07:39:45.845 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/crash -- . 2024-04-17T07:39:45.948 INFO:teuthology.orchestra.run.smithi098.stderr:tar: /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/crash: Cannot open: No such file or directory 2024-04-17T07:39:45.949 INFO:teuthology.orchestra.run.smithi098.stderr:tar: Error is not recoverable: exiting now 2024-04-17T07:39:45.953 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891/remote/smithi103/crash 2024-04-17T07:39:45.954 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/crash -- . 2024-04-17T07:39:46.074 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/crash: Cannot open: No such file or directory 2024-04-17T07:39:46.074 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-04-17T07:39:46.078 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-17T07:39:46.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-17T07:39:46.240 INFO:tasks.cephadm:Compressing logs... 2024-04-17T07:39:46.240 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T07:39:46.245 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T07:39:46.251 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-17T07:39:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-17T07:39:46.337 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-17T07:39:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-17T07:39:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-17T07:39:46.340 INFO:teuthology.orchestra.run.smithi098.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-17T07:39:46.340 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-17T07:39:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.6.log 2024-04-17T07:39:46.341 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.10.log 2024-04-17T07:39:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mon.a.log 2024-04-17T07:39:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.audit.log 2024-04-17T07:39:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.g.log 2024-04-17T07:39:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.j.log 2024-04-17T07:39:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.3.log 2024-04-17T07:39:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mgr.x.log 2024-04-17T07:39:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.0.log 2024-04-17T07:39:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.cephadm.log 2024-04-17T07:39:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.log 2024-04-17T07:39:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-volume.log 2024-04-17T07:39:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.9.log 2024-04-17T07:39:46.360 INFO:teuthology.orchestra.run.smithi086.stderr: 94.4% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.j.log.gz 2024-04-17T07:39:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.d.log 2024-04-17T07:39:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.a.log 2024-04-17T07:39:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.d.log: 87.5% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.log.gz 2024-04-17T07:39:46.361 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.audit.log 2024-04-17T07:39:46.361 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.b.log 2024-04-17T07:39:46.361 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.4.log 2024-04-17T07:39:46.362 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.audit.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-17T07:39:46.362 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.1.log 2024-04-17T07:39:46.362 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.k.log 2024-04-17T07:39:46.362 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.h.log 2024-04-17T07:39:46.362 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.7.log 2024-04-17T07:39:46.363 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.e.log 2024-04-17T07:39:46.363 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mgr.y.log 2024-04-17T07:39:46.363 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.cephadm.log 2024-04-17T07:39:46.363 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.log 2024-04-17T07:39:46.363 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-volume.log 2024-04-17T07:39:46.363 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mon.b.log 2024-04-17T07:39:46.364 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-volume.log: /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mon.b.log: 93.2% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mgr.y.log.gz 2024-04-17T07:39:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.a.log: 94.4% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.d.log.gz 2024-04-17T07:39:46.368 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-17T07:39:46.369 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-17T07:39:46.370 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.l.log 2024-04-17T07:39:46.372 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.audit.log 2024-04-17T07:39:46.373 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.i.log 2024-04-17T07:39:46.373 INFO:teuthology.orchestra.run.smithi103.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-17T07:39:46.375 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.11.log 2024-04-17T07:39:46.376 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mgr.z.log 2024-04-17T07:39:46.377 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.8.log 2024-04-17T07:39:46.378 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.5.log 2024-04-17T07:39:46.379 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.f.log 2024-04-17T07:39:46.380 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.cephadm.log 2024-04-17T07:39:46.381 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.log 2024-04-17T07:39:46.382 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.2.log 2024-04-17T07:39:46.383 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.c.log 2024-04-17T07:39:46.384 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-volume.log 2024-04-17T07:39:46.385 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mon.c.log 2024-04-17T07:39:46.385 INFO:teuthology.orchestra.run.smithi103.stderr: 93.3% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mgr.z.log.gz 2024-04-17T07:39:46.407 INFO:teuthology.orchestra.run.smithi098.stderr: 83.3% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.cephadm.log.gz 2024-04-17T07:39:46.419 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-volume.log: /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mon.c.log: 83.3% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.cephadm.log.gz 2024-04-17T07:39:46.444 INFO:teuthology.orchestra.run.smithi098.stderr: 90.3% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.audit.log.gz 2024-04-17T07:39:46.486 INFO:teuthology.orchestra.run.smithi086.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-17T07:39:46.489 INFO:teuthology.orchestra.run.smithi103.stderr: 90.4% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.audit.log.gz 2024-04-17T07:39:46.498 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.log.gz 2024-04-17T07:39:46.505 INFO:teuthology.orchestra.run.smithi103.stderr: 87.5% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.log.gz 2024-04-17T07:39:46.538 INFO:teuthology.orchestra.run.smithi086.stderr: 90.2% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.audit.log.gz 2024-04-17T07:39:46.577 INFO:teuthology.orchestra.run.smithi086.stderr: 90.8% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-volume.log.gz 2024-04-17T07:39:46.597 INFO:teuthology.orchestra.run.smithi098.stderr: 93.6% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.h.log.gz 2024-04-17T07:39:46.598 INFO:teuthology.orchestra.run.smithi098.stderr: 93.5% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.k.log.gz 2024-04-17T07:39:46.611 INFO:teuthology.orchestra.run.smithi098.stderr: 93.8% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.e.log.gz 2024-04-17T07:39:46.625 INFO:teuthology.orchestra.run.smithi098.stderr: 90.8% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-volume.log.gz 2024-04-17T07:39:46.629 INFO:teuthology.orchestra.run.smithi103.stderr: 93.9% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.i.log.gz 2024-04-17T07:39:46.642 INFO:teuthology.orchestra.run.smithi086.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-17T07:39:46.664 INFO:teuthology.orchestra.run.smithi103.stderr: 90.8% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-volume.log.gz 2024-04-17T07:39:46.698 INFO:teuthology.orchestra.run.smithi103.stderr: 95.0% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.l.log.gz 2024-04-17T07:39:46.747 INFO:teuthology.orchestra.run.smithi086.stderr: 95.0% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.g.log.gz 2024-04-17T07:39:46.832 INFO:teuthology.orchestra.run.smithi086.stderr: 95.0% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.a.log.gz 2024-04-17T07:39:46.905 INFO:teuthology.orchestra.run.smithi086.stderr: 83.3% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph.cephadm.log.gz 2024-04-17T07:39:46.916 INFO:teuthology.orchestra.run.smithi103.stderr: 95.2% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.f.log.gz 2024-04-17T07:39:47.686 INFO:teuthology.orchestra.run.smithi086.stderr: 90.5% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mgr.x.log.gz 2024-04-17T07:39:47.778 INFO:teuthology.orchestra.run.smithi098.stderr: 92.0% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mon.b.log.gz 2024-04-17T07:39:48.849 INFO:teuthology.orchestra.run.smithi103.stderr: 91.8% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mon.c.log.gz 2024-04-17T07:39:49.959 INFO:teuthology.orchestra.run.smithi103.stderr: 95.5% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.c.log.gz 2024-04-17T07:39:56.284 INFO:teuthology.orchestra.run.smithi086.stderr: 90.2% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mon.a.log.gz 2024-04-17T07:40:37.159 INFO:teuthology.orchestra.run.smithi098.stderr: 94.3% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-mds.b.log.gz 2024-04-17T07:41:52.756 INFO:teuthology.orchestra.run.smithi098.stderr: 90.6% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.1.log.gz 2024-04-17T07:41:54.325 INFO:teuthology.orchestra.run.smithi098.stderr: 90.6% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.10.log.gz 2024-04-17T07:42:23.054 INFO:teuthology.orchestra.run.smithi098.stderr: 90.5% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.7.log.gz 2024-04-17T07:42:43.200 INFO:teuthology.orchestra.run.smithi098.stderr: 90.7% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.4.log.gz 2024-04-17T07:42:43.225 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-04-17T07:42:43.225 INFO:teuthology.orchestra.run.smithi098.stderr:real 2m56.912s 2024-04-17T07:42:43.226 INFO:teuthology.orchestra.run.smithi098.stderr:user 6m0.250s 2024-04-17T07:42:43.226 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m34.507s 2024-04-17T07:42:43.541 INFO:teuthology.orchestra.run.smithi103.stderr: 90.6% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.8.log.gz 2024-04-17T07:42:52.967 INFO:teuthology.orchestra.run.smithi103.stderr: 90.7% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.11.log.gz 2024-04-17T07:43:12.962 INFO:teuthology.orchestra.run.smithi103.stderr: 90.7% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.2.log.gz 2024-04-17T07:43:40.615 INFO:teuthology.orchestra.run.smithi103.stderr: 90.5% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.5.log.gz 2024-04-17T07:43:40.629 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-04-17T07:43:40.629 INFO:teuthology.orchestra.run.smithi103.stderr:real 3m54.310s 2024-04-17T07:43:40.629 INFO:teuthology.orchestra.run.smithi103.stderr:user 7m10.643s 2024-04-17T07:43:40.629 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m42.903s 2024-04-17T07:49:36.971 INFO:teuthology.orchestra.run.smithi086.stderr: 90.6% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.0.log.gz 2024-04-17T07:50:44.947 INFO:teuthology.orchestra.run.smithi086.stderr: 90.6% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.6.log.gz 2024-04-17T07:52:22.457 INFO:teuthology.orchestra.run.smithi086.stderr: 90.5% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.3.log.gz 2024-04-17T07:52:56.724 INFO:teuthology.orchestra.run.smithi086.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-17T07:53:12.258 INFO:teuthology.orchestra.run.smithi086.stderr: 90.5% -- replaced with /var/log/ceph/1f7ed99a-fc88-11ee-bc91-c7b262605968/ceph-osd.9.log.gz 2024-04-17T07:53:12.263 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-17T07:53:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:real 13m25.979s 2024-04-17T07:53:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:user 15m33.458s 2024-04-17T07:53:12.263 INFO:teuthology.orchestra.run.smithi086.stderr:sys 1m56.651s 2024-04-17T07:53:12.264 INFO:tasks.cephadm:Archiving logs... 2024-04-17T07:53:12.266 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891/remote/smithi086/log 2024-04-17T07:53:12.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T07:54:20.578 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891/remote/smithi098/log 2024-04-17T07:54:20.580 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T07:55:09.763 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891/remote/smithi103/log 2024-04-17T07:55:09.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-17T07:56:02.570 INFO:tasks.cephadm:Removing cluster... 2024-04-17T07:56:02.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 --force 2024-04-17T07:56:03.123 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:56:04.945 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 --force 2024-04-17T07:56:05.392 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:56:07.042 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1f7ed99a-fc88-11ee-bc91-c7b262605968 --force 2024-04-17T07:56:07.460 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 1f7ed99a-fc88-11ee-bc91-c7b262605968 2024-04-17T07:56:09.177 INFO:tasks.cephadm:Removing cephadm ... 2024-04-17T07:56:09.178 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-17T07:56:09.246 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-17T07:56:09.305 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-17T07:56:09.364 INFO:tasks.cephadm:Teardown complete 2024-04-17T07:56:09.364 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-17T07:56:09.380 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} 2024-04-17T07:56:09.380 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-17T07:56:09.381 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T07:56:09.388 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T07:56:09.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-17T07:56:09.562 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-17T07:56:09.573 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-17T07:56:09.573 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T07:56:09.581 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T07:56:09.589 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T07:56:09.629 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-17T07:56:09.637 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-04-17T07:56:09.645 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-04-17T07:56:09.730 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T07:56:09.730 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-17T07:56:09.730 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv01.front.sepia.ceph.com 3 6 377 58 +740us[ +742us] +/- 44ms 2024-04-17T07:56:09.730 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 -897us[ -897us] +/- 72ms 2024-04-17T07:56:09.730 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -464us[ -464us] +/- 82ms 2024-04-17T07:56:09.730 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:56:09.760 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T07:56:09.760 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-04-17T07:56:09.760 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +748us[ +740us] +/- 44ms 2024-04-17T07:56:09.760 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -875us[ -883us] +/- 71ms 2024-04-17T07:56:09.760 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 -454us[ -462us] +/- 82ms 2024-04-17T07:56:09.761 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:56:09.761 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T07:56:09.761 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-04-17T07:56:09.761 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 58 +759us[ +771us] +/- 44ms 2024-04-17T07:56:09.762 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -892us[ -879us] +/- 71ms 2024-04-17T07:56:09.762 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -447us[ -447us] +/- 82ms 2024-04-17T07:56:09.762 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T07:56:09.765 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-17T07:56:09.775 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-17T07:56:09.776 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-17T07:56:09.788 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T07:56:10.273 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T07:56:10.499 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-17T07:56:10.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T07:56:10.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:56:10.862 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T07:56:10.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:56:10.964 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T07:56:11.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:56:11.070 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-17T07:56:11.081 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-17T07:56:11.104 INFO:teuthology.task.internal:Duration was 3860.388934 seconds 2024-04-17T07:56:11.105 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-17T07:56:11.121 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-17T07:56:11.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T07:56:11.129 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T07:56:11.137 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-17T07:56:11.300 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T07:56:11.333 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T07:56:11.345 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T07:56:11.704 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-17T07:56:11.704 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-04-17T07:56:11.704 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T07:56:11.777 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2024-04-17T07:56:11.777 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T07:56:11.851 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-04-17T07:56:11.851 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-17T07:56:11.939 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-17T07:56:11.939 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T07:56:11.947 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T07:56:11.955 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-17T07:56:12.057 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-17T07:56:12.057 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T07:56:12.063 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T07:56:12.068 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-17T07:56:12.610 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-17T07:56:12.622 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-17T07:56:12.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T07:56:12.714 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T07:56:12.814 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-17T07:56:12.938 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-17T07:56:12.950 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T07:56:12.958 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T07:56:12.966 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-17T07:56:13.084 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-04-17T07:56:13.094 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2024-04-17T07:56:13.097 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-04-17T07:56:13.161 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T07:56:13.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:56:13.211 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T07:56:13.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:56:13.265 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-17T07:56:13.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T07:56:13.322 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-17T07:56:13.336 INFO:teuthology.task.internal:Transferring archived files... 2024-04-17T07:56:13.337 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891/remote/smithi086 2024-04-17T07:56:13.337 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T07:56:13.459 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891/remote/smithi098 2024-04-17T07:56:13.459 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T07:56:13.568 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891/remote/smithi103 2024-04-17T07:56:13.568 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-17T07:56:13.679 INFO:teuthology.task.internal:Removing archive directory... 2024-04-17T07:56:13.680 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T07:56:13.687 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T07:56:13.695 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-17T07:56:13.753 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-17T07:56:13.768 INFO:teuthology.task.internal:Not uploading archives. 2024-04-17T07:56:13.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-17T07:56:13.781 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-17T07:56:13.782 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T07:56:13.789 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T07:56:13.797 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-17T07:56:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: 393829 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 07:56 /home/ubuntu/cephtest 2024-04-17T07:56:13.847 INFO:teuthology.orchestra.run.smithi098.stdout: 395890 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 07:56 /home/ubuntu/cephtest 2024-04-17T07:56:13.853 INFO:teuthology.orchestra.run.smithi103.stdout: 395495 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 17 07:56 /home/ubuntu/cephtest 2024-04-17T07:56:13.857 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-17T07:56:13.868 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-17T07:56:13.882 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} 2024-04-17T07:56:13.882 INFO:teuthology.nuke:Checking targets against current locks 2024-04-17T07:56:13.909 DEBUG:teuthology.nuke:shortname: smithi086 2024-04-17T07:56:13.910 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T07:56:13.934 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 06:39:46.425817', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDWvvi3c9NgpR+VwJWnnDD7K28JTAVPxd+qkTRepn6rnRXmH1+Z9mTT5reI8v39KHZWRjvbw0HYADM8XiGNoVbc='} 2024-04-17T07:56:13.939 DEBUG:teuthology.nuke:shortname: smithi098 2024-04-17T07:56:13.939 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T07:56:13.964 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 06:39:46.423580', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNliSfdHzPHo430Yz8TMQz9ReCoSEUJUFaOJjMSzkqlCFzWqtiPzzbfPADZ31P6iRJDHwH7Wh83CBKVAgbkkicA='} 2024-04-17T07:56:13.965 DEBUG:teuthology.nuke:shortname: smithi103 2024-04-17T07:56:13.966 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T07:56:13.989 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 06:39:46.424858', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrtn37WGU5iIjblYaX6uLqlB+qcsPvB/jHlbyyMWDiroyLnkQBHtHGJUNEc0B4qUjQTEQuEtwvv0EhoEzBeprI='} 2024-04-17T07:56:13.999 INFO:teuthology.orchestra.console.smithi086:Power off 2024-04-17T07:56:13.999 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T07:56:14.023 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2024-04-17T07:56:14.024 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T07:56:14.041 INFO:teuthology.orchestra.console.smithi098:Power off 2024-04-17T07:56:14.041 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T07:56:14.058 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-17T07:56:14.065 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off 2024-04-17T07:56:14.065 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T07:56:14.088 INFO:teuthology.orchestra.console.smithi103:Power off 2024-04-17T07:56:14.089 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T07:56:14.097 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2024-04-17T07:56:14.110 DEBUG:teuthology.orchestra.console.smithi103:power off output: Chassis Power Control: Down/Off 2024-04-17T07:56:14.110 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T07:56:14.133 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on 2024-04-17T07:56:18.059 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T07:56:18.098 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T07:56:18.134 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T07:56:18.170 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-17T07:56:18.210 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2024-04-17T07:56:18.247 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on 2024-04-17T07:56:22.174 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T07:56:22.211 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T07:56:22.248 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T07:56:22.286 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-04-17T07:56:22.322 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2024-04-17T07:56:22.358 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off 2024-04-17T07:56:22.388 INFO:teuthology.orchestra.console.smithi086:Power off completed 2024-04-17T07:56:22.460 INFO:teuthology.orchestra.console.smithi103:Power off completed 2024-04-17T07:56:26.326 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T07:56:26.436 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2024-04-17T07:56:26.537 INFO:teuthology.orchestra.console.smithi098:Power off completed 2024-04-17T07:56:26.643 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} duration: 3860.388934135437 failure_reason: 'The following counters failed to be set on mds daemons: {''mds.exported'', ''mds.imported''}' flavor: default mon.a-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 mon.b-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 mon.c-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 owner: scheduled_rishabh@teuthology status: fail success: false 2024-04-17T07:56:26.643 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T07:56:26.737 INFO:teuthology.run:FAIL